2024-03-30T00:24:01.637 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-30T00:24:01.638 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} --name rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi --verbose --archive /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246 --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246/orig.config.yaml 2024-03-30T00:24:01.682 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-30T00:24:01.781 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246 branch: wip-rishabh-testing-20240327.051042-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: null first_in_suite: false job_id: '7629246' kernel: flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rishabh-testing-20240327.051042-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240327.051042-reef sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8052 sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f sleep_before_teardown: 0 subset: 497/512 suite: fs suite_branch: wip-rishabh-testing-20240327.051042-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHquc1fXzOkR+Oc1lu0Vh5hvEbwh7hdzwryuzCoCkRz2kM9DTitUMYF4yCUJnGjeYGWwCNtdmZwlb/AZBCdkHzs= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF3D/q5JOaLuZ73PSCANMnVcs6eIMU7C3S/m8GabWZjBMBkYYlRzUfN9LKxOP3IvKTZGl+HXiPoGXl3fFjGlbzA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-29_18:05:24 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-30T00:24:01.782 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa; will attempt to use it 2024-03-30T00:24:01.782 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks 2024-03-30T00:24:01.782 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-30T00:24:01.784 INFO:teuthology.task.internal:Checking packages... 2024-03-30T00:24:01.810 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '470100a4ccbcf7769052bf09a5413881ccf1b97f' 2024-03-30T00:24:01.810 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-30T00:24:01.810 INFO:teuthology.packaging:ref: None 2024-03-30T00:24:01.810 INFO:teuthology.packaging:tag: None 2024-03-30T00:24:01.810 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240327.051042-reef 2024-03-30T00:24:01.810 INFO:teuthology.packaging:sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:24:01.811 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rishabh-testing-20240327.051042-reef 2024-03-30T00:24:02.034 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1018-g470100a4-1jammy 2024-03-30T00:24:02.036 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-30T00:24:02.042 INFO:teuthology.task.internal:no buildpackages task found 2024-03-30T00:24:02.042 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-30T00:24:02.088 INFO:teuthology.task.internal:Saving configuration 2024-03-30T00:24:02.105 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-30T00:24:02.111 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T00:24:02.139 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 00:16:51.577550', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILdKTluXz/lzhkQaAdgYw2IDT3iYP0EnpnsMb4ry7Hxv'} 2024-03-30T00:24:02.159 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 00:16:51.579006', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOCJRrOtb1tF4AKyeoOqyB1zYGldYczIAbLbzFkybMeo'} 2024-03-30T00:24:02.159 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-30T00:24:02.165 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-30T00:24:02.165 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-30T00:24:02.166 INFO:teuthology.run_tasks:Running task console_log... 2024-03-30T00:24:02.231 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8c2a82a4c0>, signals=[15]) 2024-03-30T00:24:02.232 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-30T00:24:02.238 INFO:teuthology.task.internal:Opening connections... 2024-03-30T00:24:02.238 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:24:02.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:24:02.324 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:24:02.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:24:02.404 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-30T00:24:02.411 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-03-30T00:24:02.416 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-03-30T00:24:02.416 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-03-30T00:24:02.467 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-30T00:24:02.467 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="Ubuntu" 2024-03-30T00:24:02.467 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="22.04" 2024-03-30T00:24:02.467 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-30T00:24:02.467 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_CODENAME=jammy 2024-03-30T00:24:02.467 INFO:teuthology.orchestra.run.smithi066.stdout:ID=ubuntu 2024-03-30T00:24:02.467 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE=debian 2024-03-30T00:24:02.467 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-30T00:24:02.467 INFO:teuthology.orchestra.run.smithi066.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-30T00:24:02.467 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-30T00:24:02.467 INFO:teuthology.orchestra.run.smithi066.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-30T00:24:02.468 INFO:teuthology.orchestra.run.smithi066.stdout:UBUNTU_CODENAME=jammy 2024-03-30T00:24:02.468 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-03-30T00:24:02.493 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2024-03-30T00:24:02.498 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2024-03-30T00:24:02.498 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2024-03-30T00:24:02.547 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-30T00:24:02.547 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Ubuntu" 2024-03-30T00:24:02.547 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="22.04" 2024-03-30T00:24:02.547 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-30T00:24:02.547 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_CODENAME=jammy 2024-03-30T00:24:02.547 INFO:teuthology.orchestra.run.smithi156.stdout:ID=ubuntu 2024-03-30T00:24:02.547 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE=debian 2024-03-30T00:24:02.547 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-30T00:24:02.547 INFO:teuthology.orchestra.run.smithi156.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-30T00:24:02.547 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-30T00:24:02.547 INFO:teuthology.orchestra.run.smithi156.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-30T00:24:02.548 INFO:teuthology.orchestra.run.smithi156.stdout:UBUNTU_CODENAME=jammy 2024-03-30T00:24:02.548 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2024-03-30T00:24:02.567 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-30T00:24:02.579 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-30T00:24:02.600 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-30T00:24:02.600 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-03-30T00:24:02.603 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2024-03-30T00:24:02.607 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-30T00:24:02.613 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-30T00:24:02.613 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-03-30T00:24:02.652 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2024-03-30T00:24:02.669 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-30T00:24:02.719 INFO:teuthology.run_tasks:Running task kernel... 2024-03-30T00:24:02.733 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-30T00:24:02.733 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, timeout 300 2024-03-30T00:24:02.733 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T00:24:02.734 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T00:24:02.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:24:02.742 DEBUG:teuthology.task.kernel:sha1 for mon.a is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:24:02.742 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-30T00:24:02.742 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-03-30T00:24:02.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:24:02.743 DEBUG:teuthology.task.kernel:sha1 for mon.b is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:24:02.743 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-30T00:24:02.743 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2024-03-30T00:24:02.787 INFO:teuthology.orchestra.run.smithi066.stdout:5.15.0-56-generic 2024-03-30T00:24:02.787 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:24:02.787 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-30T00:24:02.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:24:02.795 INFO:teuthology.orchestra.run.smithi156.stdout:5.15.0-56-generic 2024-03-30T00:24:02.796 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:24:02.796 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-30T00:24:02.796 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:24:02.931 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2024-03-30T00:24:02.931 DEBUG:teuthology.orchestra.run.smithi066:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-03-30T00:24:02.934 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2024-03-30T00:24:02.935 DEBUG:teuthology.orchestra.run.smithi156:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-03-30T00:24:02.946 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-03-30T00:24:02.958 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-03-30T00:24:03.096 INFO:teuthology.orchestra.run.smithi156.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-03-30T00:24:03.097 INFO:teuthology.orchestra.run.smithi156.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-03-30T00:24:03.097 INFO:teuthology.orchestra.run.smithi156.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-03-30T00:24:03.097 INFO:teuthology.orchestra.run.smithi156.stderr:hint: 2024-03-30T00:24:03.097 INFO:teuthology.orchestra.run.smithi156.stderr:hint: git config --global init.defaultBranch 2024-03-30T00:24:03.097 INFO:teuthology.orchestra.run.smithi156.stderr:hint: 2024-03-30T00:24:03.097 INFO:teuthology.orchestra.run.smithi156.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-03-30T00:24:03.097 INFO:teuthology.orchestra.run.smithi156.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-03-30T00:24:03.097 INFO:teuthology.orchestra.run.smithi156.stderr:hint: 2024-03-30T00:24:03.098 INFO:teuthology.orchestra.run.smithi156.stderr:hint: git branch -m 2024-03-30T00:24:03.098 INFO:teuthology.orchestra.run.smithi156.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-03-30T00:24:03.100 DEBUG:teuthology.orchestra.run.smithi156:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-03-30T00:24:03.103 INFO:teuthology.orchestra.run.smithi066.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-03-30T00:24:03.103 INFO:teuthology.orchestra.run.smithi066.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-03-30T00:24:03.103 INFO:teuthology.orchestra.run.smithi066.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-03-30T00:24:03.103 INFO:teuthology.orchestra.run.smithi066.stderr:hint: 2024-03-30T00:24:03.103 INFO:teuthology.orchestra.run.smithi066.stderr:hint: git config --global init.defaultBranch 2024-03-30T00:24:03.103 INFO:teuthology.orchestra.run.smithi066.stderr:hint: 2024-03-30T00:24:03.103 INFO:teuthology.orchestra.run.smithi066.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-03-30T00:24:03.103 INFO:teuthology.orchestra.run.smithi066.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-03-30T00:24:03.103 INFO:teuthology.orchestra.run.smithi066.stderr:hint: 2024-03-30T00:24:03.104 INFO:teuthology.orchestra.run.smithi066.stderr:hint: git branch -m 2024-03-30T00:24:03.104 INFO:teuthology.orchestra.run.smithi066.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-03-30T00:24:03.106 DEBUG:teuthology.orchestra.run.smithi066:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-03-30T00:24:03.173 DEBUG:teuthology.orchestra.run.smithi156:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-03-30T00:24:03.175 DEBUG:teuthology.orchestra.run.smithi066:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-03-30T00:24:03.229 DEBUG:teuthology.orchestra.run.smithi156:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-03-30T00:24:03.233 DEBUG:teuthology.orchestra.run.smithi066:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-03-30T00:24:45.841 INFO:teuthology.orchestra.run.smithi156.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-03-30T00:24:45.842 INFO:teuthology.orchestra.run.smithi156.stderr: * [new branch] main -> origin/main 2024-03-30T00:24:45.874 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20190312 -> 20190312 2024-03-30T00:24:45.875 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20190416 -> 20190416 2024-03-30T00:24:45.875 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20190514 -> 20190514 2024-03-30T00:24:45.875 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20190618 -> 20190618 2024-03-30T00:24:45.875 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20190717 -> 20190717 2024-03-30T00:24:45.876 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20190815 -> 20190815 2024-03-30T00:24:45.876 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20190923 -> 20190923 2024-03-30T00:24:45.876 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20191022 -> 20191022 2024-03-30T00:24:45.876 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20191215 -> 20191215 2024-03-30T00:24:45.876 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20200122 -> 20200122 2024-03-30T00:24:45.876 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20200316 -> 20200316 2024-03-30T00:24:45.876 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20200421 -> 20200421 2024-03-30T00:24:45.876 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20200519 -> 20200519 2024-03-30T00:24:45.877 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20200619 -> 20200619 2024-03-30T00:24:45.878 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20200721 -> 20200721 2024-03-30T00:24:45.878 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20200817 -> 20200817 2024-03-30T00:24:45.878 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20200918 -> 20200918 2024-03-30T00:24:45.878 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20201022 -> 20201022 2024-03-30T00:24:45.878 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20201118 -> 20201118 2024-03-30T00:24:45.878 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20201218 -> 20201218 2024-03-30T00:24:45.878 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20210208 -> 20210208 2024-03-30T00:24:45.878 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20210315 -> 20210315 2024-03-30T00:24:45.879 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20210511 -> 20210511 2024-03-30T00:24:45.879 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20210716 -> 20210716 2024-03-30T00:24:45.879 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20210818 -> 20210818 2024-03-30T00:24:45.879 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20210919 -> 20210919 2024-03-30T00:24:45.879 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20211027 -> 20211027 2024-03-30T00:24:45.879 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20211216 -> 20211216 2024-03-30T00:24:45.879 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20220209 -> 20220209 2024-03-30T00:24:45.880 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20220310 -> 20220310 2024-03-30T00:24:45.880 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20220411 -> 20220411 2024-03-30T00:24:45.881 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20220509 -> 20220509 2024-03-30T00:24:45.881 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20220610 -> 20220610 2024-03-30T00:24:45.881 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20220708 -> 20220708 2024-03-30T00:24:45.881 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20220815 -> 20220815 2024-03-30T00:24:45.881 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20220913 -> 20220913 2024-03-30T00:24:45.882 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20221012 -> 20221012 2024-03-30T00:24:45.882 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20221109 -> 20221109 2024-03-30T00:24:45.882 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20221214 -> 20221214 2024-03-30T00:24:45.882 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20230117 -> 20230117 2024-03-30T00:24:45.882 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20230210 -> 20230210 2024-03-30T00:24:45.882 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20230310 -> 20230310 2024-03-30T00:24:45.882 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20230404 -> 20230404 2024-03-30T00:24:45.883 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20230515 -> 20230515 2024-03-30T00:24:45.883 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20230625 -> 20230625 2024-03-30T00:24:45.883 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20230804 -> 20230804 2024-03-30T00:24:45.883 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20230919 -> 20230919 2024-03-30T00:24:45.883 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20231030 -> 20231030 2024-03-30T00:24:45.884 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20231111 -> 20231111 2024-03-30T00:24:45.884 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20231211 -> 20231211 2024-03-30T00:24:45.884 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20240115 -> 20240115 2024-03-30T00:24:45.884 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20240220 -> 20240220 2024-03-30T00:24:45.884 INFO:teuthology.orchestra.run.smithi156.stderr: * [new tag] 20240312 -> 20240312 2024-03-30T00:24:49.569 INFO:teuthology.orchestra.run.smithi066.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-03-30T00:24:49.570 INFO:teuthology.orchestra.run.smithi066.stderr: * [new branch] main -> origin/main 2024-03-30T00:24:49.604 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190312 -> 20190312 2024-03-30T00:24:49.604 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190416 -> 20190416 2024-03-30T00:24:49.604 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190514 -> 20190514 2024-03-30T00:24:49.604 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190618 -> 20190618 2024-03-30T00:24:49.604 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190717 -> 20190717 2024-03-30T00:24:49.604 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190815 -> 20190815 2024-03-30T00:24:49.604 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20190923 -> 20190923 2024-03-30T00:24:49.606 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20191022 -> 20191022 2024-03-30T00:24:49.606 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20191215 -> 20191215 2024-03-30T00:24:49.606 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200122 -> 20200122 2024-03-30T00:24:49.606 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200316 -> 20200316 2024-03-30T00:24:49.606 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200421 -> 20200421 2024-03-30T00:24:49.606 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200519 -> 20200519 2024-03-30T00:24:49.606 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200619 -> 20200619 2024-03-30T00:24:49.606 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200721 -> 20200721 2024-03-30T00:24:49.606 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200817 -> 20200817 2024-03-30T00:24:49.606 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20200918 -> 20200918 2024-03-30T00:24:49.608 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20201022 -> 20201022 2024-03-30T00:24:49.608 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20201118 -> 20201118 2024-03-30T00:24:49.608 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20201218 -> 20201218 2024-03-30T00:24:49.608 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210208 -> 20210208 2024-03-30T00:24:49.608 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210315 -> 20210315 2024-03-30T00:24:49.608 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210511 -> 20210511 2024-03-30T00:24:49.608 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210716 -> 20210716 2024-03-30T00:24:49.608 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210818 -> 20210818 2024-03-30T00:24:49.608 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20210919 -> 20210919 2024-03-30T00:24:49.608 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20211027 -> 20211027 2024-03-30T00:24:49.608 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20211216 -> 20211216 2024-03-30T00:24:49.609 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220209 -> 20220209 2024-03-30T00:24:49.609 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220310 -> 20220310 2024-03-30T00:24:49.610 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220411 -> 20220411 2024-03-30T00:24:49.610 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220509 -> 20220509 2024-03-30T00:24:49.610 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220610 -> 20220610 2024-03-30T00:24:49.610 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220708 -> 20220708 2024-03-30T00:24:49.610 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220815 -> 20220815 2024-03-30T00:24:49.611 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20220913 -> 20220913 2024-03-30T00:24:49.611 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20221012 -> 20221012 2024-03-30T00:24:49.611 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20221109 -> 20221109 2024-03-30T00:24:49.611 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20221214 -> 20221214 2024-03-30T00:24:49.611 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230117 -> 20230117 2024-03-30T00:24:49.611 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230210 -> 20230210 2024-03-30T00:24:49.611 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230310 -> 20230310 2024-03-30T00:24:49.611 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230404 -> 20230404 2024-03-30T00:24:49.611 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230515 -> 20230515 2024-03-30T00:24:49.612 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230625 -> 20230625 2024-03-30T00:24:49.612 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230804 -> 20230804 2024-03-30T00:24:49.612 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20230919 -> 20230919 2024-03-30T00:24:49.612 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20231030 -> 20231030 2024-03-30T00:24:49.612 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20231111 -> 20231111 2024-03-30T00:24:49.613 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20231211 -> 20231211 2024-03-30T00:24:49.613 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240115 -> 20240115 2024-03-30T00:24:49.613 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240220 -> 20240220 2024-03-30T00:24:49.613 INFO:teuthology.orchestra.run.smithi066.stderr: * [new tag] 20240312 -> 20240312 2024-03-30T00:24:50.083 INFO:teuthology.orchestra.run.smithi156.stderr:Updating files: 51% (1850/3598) Updating files: 52% (1871/3598) Updating files: 53% (1907/3598) Updating files: 54% (1943/3598) Updating files: 55% (1979/3598) Updating files: 56% (2015/3598) Updating files: 57% (2051/3598) Updating files: 58% (2087/3598) Updating files: 59% (2123/3598) Updating files: 60% (2159/3598) Updating files: 61% (2195/3598) Updating files: 62% (2231/3598) Updating files: 63% (2267/3598) Updating files: 64% (2303/3598) Updating files: 64% (2326/3598) Updating files: 65% (2339/3598) Updating files: 66% (2375/3598) Updating files: 67% (2411/3598) Updating files: 68% (2447/3598) Updating files: 69% (2483/3598) Updating files: 70% (2519/3598) Updating files: 71% (2555/3598) Updating files: 72% (2591/3598) Updating files: 73% (2627/3598) Updating files: 74% (2663/3598) Updating files: 75% (2699/3598) Updating files: 76% (2735/3598) Updating files: 77% (2771/3598) Updating files: 77% (2785/3598) Updating files: 78% (2807/3598) Updating files: 79% (2843/3598) Updating files: 80% (2879/3598) Updating files: 81% (2915/3598) Updating files: 82% (2951/3598) Updating files: 83% (2987/3598) Updating files: 84% (3023/3598) Updating files: 85% (3059/3598) Updating files: 86% (3095/3598) Updating files: 87% (3131/3598) Updating files: 88% (3167/3598) Updating files: 88% (3175/3598) Updating files: 89% (3203/3598) Updating files: 90% (3239/3598) Updating files: 91% (3275/3598) Updating files: 92% (3311/3598) Updating files: 93% (3347/3598) Updating files: 94% (3383/3598) Updating files: 95% (3419/3598) Updating files: 96% (3455/3598) Updating files: 97% (3491/3598) Updating files: 98% (3527/3598) Updating files: 99% (3563/3598) Updating files: 100% (3598/3598) Updating files: 100% (3598/3598), done. 2024-03-30T00:24:50.088 INFO:teuthology.orchestra.run.smithi156.stdout:HEAD is now at e068ccf0 Merge branch 'robot/pr-0-1711100280' into 'main' 2024-03-30T00:24:50.110 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-30T00:24:50.110 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:24:50.339 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/ 2024-03-30T00:24:50.339 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /tmp/linux-image.deb && echo linux-image-6.8.0-g1df0c7e8a235_6.8.0-g1df0c7e8a235-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/ --input-file=- 2024-03-30T00:24:50.342 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi156.front.sepia.ceph.com to complete... 2024-03-30T00:24:52.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30 00:24:52 URL:https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/linux-image-6.8.0-g1df0c7e8a235_6.8.0-g1df0c7e8a235-1_amd64.deb [39278344/39278344] -> "/tmp/linux-image.deb" [1] 2024-03-30T00:24:52.153 INFO:teuthology.orchestra.run.smithi156.stderr:FINISHED --2024-03-30 00:24:52-- 2024-03-30T00:24:52.153 INFO:teuthology.orchestra.run.smithi156.stderr:Total wall clock time: 1.8s 2024-03-30T00:24:52.154 INFO:teuthology.orchestra.run.smithi156.stderr:Downloaded: 1 files, 37M in 1.6s (24.0 MB/s) 2024-03-30T00:24:52.158 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-30T00:24:52.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo dpkg -i /tmp/linux-image.deb 2024-03-30T00:24:52.414 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package linux-image-6.8.0-g1df0c7e8a235. 2024-03-30T00:24:54.060 INFO:teuthology.orchestra.run.smithi066.stderr:Updating files: 48% (1746/3598) Updating files: 49% (1764/3598) Updating files: 50% (1799/3598) Updating files: 51% (1835/3598) Updating files: 52% (1871/3598) Updating files: 53% (1907/3598) Updating files: 54% (1943/3598) Updating files: 55% (1979/3598) Updating files: 56% (2015/3598) Updating files: 57% (2051/3598) Updating files: 58% (2087/3598) Updating files: 59% (2123/3598) Updating files: 60% (2159/3598) Updating files: 61% (2195/3598) Updating files: 61% (2220/3598) Updating files: 62% (2231/3598) Updating files: 63% (2267/3598) Updating files: 64% (2303/3598) Updating files: 65% (2339/3598) Updating files: 66% (2375/3598) Updating files: 67% (2411/3598) Updating files: 68% (2447/3598) Updating files: 69% (2483/3598) Updating files: 70% (2519/3598) Updating files: 71% (2555/3598) Updating files: 71% (2580/3598) Updating files: 72% (2591/3598) Updating files: 73% (2627/3598) Updating files: 74% (2663/3598) Updating files: 75% (2699/3598) Updating files: 76% (2735/3598) Updating files: 77% (2771/3598) Updating files: 78% (2807/3598) Updating files: 79% (2843/3598) Updating files: 80% (2879/3598) Updating files: 81% (2915/3598) Updating files: 82% (2951/3598) Updating files: 83% (2987/3598) Updating files: 83% (3008/3598) Updating files: 84% (3023/3598) Updating files: 85% (3059/3598) Updating files: 86% (3095/3598) Updating files: 87% (3131/3598) Updating files: 88% (3167/3598) Updating files: 89% (3203/3598) Updating files: 90% (3239/3598) Updating files: 91% (3275/3598) Updating files: 92% (3311/3598) Updating files: 93% (3347/3598) Updating files: 94% (3383/3598) Updating files: 95% (3419/3598) Updating files: 96% (3455/3598) Updating files: 97% (3491/3598) Updating files: 98% (3527/3598) Updating files: 99% (3563/3598) Updating files: 100% (3598/3598) Updating files: 100% (3598/3598), done. 2024-03-30T00:24:54.065 INFO:teuthology.orchestra.run.smithi066.stdout:HEAD is now at e068ccf0 Merge branch 'robot/pr-0-1711100280' into 'main' 2024-03-30T00:24:54.087 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-30T00:24:54.088 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:24:54.308 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/ 2024-03-30T00:24:54.308 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /tmp/linux-image.deb && echo linux-image-6.8.0-g1df0c7e8a235_6.8.0-g1df0c7e8a235-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/ --input-file=- 2024-03-30T00:24:54.311 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi066.front.sepia.ceph.com to complete... 2024-03-30T00:24:54.420 INFO:teuthology.orchestra.run.smithi156.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-03-30T00:24:54.425 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-03-30T00:24:54.552 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking linux-image-6.8.0-g1df0c7e8a235 (6.8.0-g1df0c7e8a235-1) ... 2024-03-30T00:24:56.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30 00:24:55 URL:https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/linux-image-6.8.0-g1df0c7e8a235_6.8.0-g1df0c7e8a235-1_amd64.deb [39278344/39278344] -> "/tmp/linux-image.deb" [1] 2024-03-30T00:24:56.024 INFO:teuthology.orchestra.run.smithi066.stderr:FINISHED --2024-03-30 00:24:55-- 2024-03-30T00:24:56.024 INFO:teuthology.orchestra.run.smithi066.stderr:Total wall clock time: 1.7s 2024-03-30T00:24:56.024 INFO:teuthology.orchestra.run.smithi066.stderr:Downloaded: 1 files, 37M in 1.5s (25.5 MB/s) 2024-03-30T00:24:56.028 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-30T00:24:56.029 DEBUG:teuthology.orchestra.run.smithi066:> sudo dpkg -i /tmp/linux-image.deb 2024-03-30T00:24:56.242 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package linux-image-6.8.0-g1df0c7e8a235. 2024-03-30T00:24:58.150 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up linux-image-6.8.0-g1df0c7e8a235 (6.8.0-g1df0c7e8a235-1) ... 2024-03-30T00:24:58.185 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-03-30T00:24:58.189 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-03-30T00:24:58.261 INFO:teuthology.orchestra.run.smithi156.stderr:update-initramfs: Generating /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-30T00:24:58.301 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking linux-image-6.8.0-g1df0c7e8a235 (6.8.0-g1df0c7e8a235-1) ... 2024-03-30T00:25:01.815 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up linux-image-6.8.0-g1df0c7e8a235 (6.8.0-g1df0c7e8a235-1) ... 2024-03-30T00:25:01.926 INFO:teuthology.orchestra.run.smithi066.stderr:update-initramfs: Generating /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-30T00:25:20.033 INFO:teuthology.orchestra.run.smithi156.stderr:Sourcing file `/etc/default/grub' 2024-03-30T00:25:20.041 INFO:teuthology.orchestra.run.smithi156.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-30T00:25:20.064 INFO:teuthology.orchestra.run.smithi156.stderr:Generating grub configuration file ... 2024-03-30T00:25:20.407 INFO:teuthology.orchestra.run.smithi156.stderr:Found linux image: /boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-30T00:25:20.422 INFO:teuthology.orchestra.run.smithi156.stderr:Found initrd image: /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-30T00:25:20.735 INFO:teuthology.orchestra.run.smithi156.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-30T00:25:20.743 INFO:teuthology.orchestra.run.smithi156.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-30T00:25:20.809 INFO:teuthology.orchestra.run.smithi156.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-30T00:25:20.815 INFO:teuthology.orchestra.run.smithi156.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-30T00:25:20.964 INFO:teuthology.orchestra.run.smithi156.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-30T00:25:20.964 INFO:teuthology.orchestra.run.smithi156.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-30T00:25:20.964 INFO:teuthology.orchestra.run.smithi156.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-30T00:25:21.006 INFO:teuthology.orchestra.run.smithi156.stderr:done 2024-03-30T00:25:21.199 DEBUG:teuthology.orchestra.run.smithi156:> dpkg-deb -c /tmp/linux-image.deb 2024-03-30T00:25:21.206 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./ 2024-03-30T00:25:21.207 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./boot/ 2024-03-30T00:25:21.207 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 4474334 2024-03-25 03:29 ./boot/System.map-6.8.0-g1df0c7e8a235 2024-03-30T00:25:21.216 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 170899 2024-03-25 03:29 ./boot/config-6.8.0-g1df0c7e8a235 2024-03-30T00:25:21.217 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11551744 2024-03-25 03:29 ./boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-30T00:25:21.236 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/ 2024-03-30T00:25:21.236 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/ 2024-03-30T00:25:21.236 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/postinst.d/ 2024-03-30T00:25:21.236 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/postrm.d/ 2024-03-30T00:25:21.236 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/preinst.d/ 2024-03-30T00:25:21.236 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/prerm.d/ 2024-03-30T00:25:21.236 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/ 2024-03-30T00:25:21.236 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/ 2024-03-30T00:25:21.236 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/ 2024-03-30T00:25:21.236 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/ 2024-03-30T00:25:21.237 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/ 2024-03-30T00:25:21.237 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/ 2024-03-30T00:25:21.237 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ 2024-03-30T00:25:21.237 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 490441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/aesni-intel.ko 2024-03-30T00:25:21.239 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 93073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 106689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 152137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 83849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-30T00:25:21.240 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 58481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 50385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/ 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/ 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/ 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpuid.ko 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/msr.ko 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/ 2024-03-30T00:25:21.241 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 354225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-30T00:25:21.242 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 886129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-30T00:25:21.242 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 2294793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-30T00:25:21.309 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 53833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/af_alg.ko 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/algif_hash.ko 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/algif_skcipher.ko 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/asymmetric_keys/ 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/ 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-30T00:25:21.310 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-30T00:25:21.311 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cryptd.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_simd.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lrw.ko 2024-03-30T00:25:21.312 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 143025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-30T00:25:21.313 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xts.ko 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_extlog.ko 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/apei/ 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/apei/einj.ko 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-30T00:25:21.314 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 104417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 89929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 88305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-30T00:25:21.315 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 52169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_legacy.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-30T00:25:21.316 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_radisys.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rz1000.ko 2024-03-30T00:25:21.317 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 83249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 70297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-30T00:25:21.318 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 44697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/ 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 124385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/ 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/ 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 113801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/ 2024-03-30T00:25:21.319 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 964745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 169697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/ 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 94209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 116329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/ 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 110321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 98185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 263633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/ 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 98105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/ 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 58761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ 2024-03-30T00:25:21.320 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/applicom.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/ 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 148513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 139305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 70865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-30T00:25:21.321 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 53409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/ 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 84001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/nvram.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/ 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-30T00:25:21.322 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/ 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_class.ko 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 50841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_core.ko 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/ 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-palmas.ko 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-si5351.ko 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-twl6040.ko 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-wm831x.ko 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/ 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-30T00:25:21.323 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 70425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 246777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/ 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/ 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/ 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 570217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/ 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 70409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-30T00:25:21.324 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/ 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/ 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 122793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 80297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-30T00:25:21.325 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 56729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/ 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-gpio.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max14577.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max77693.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max8997.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-palmas.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-rt8973a.ko 2024-03-30T00:25:21.326 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-sm5502.ko 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/ 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/dmi-sysfs.ko 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/efi/ 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/efi/efi-pstore.ko 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/ 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-adp5520.ko 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-amd8111.ko 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-arizona.ko 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-da9052.ko 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-da9055.ko 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-dln2.ko 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 77745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-f7188x.ko 2024-03-30T00:25:21.327 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-generic.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-kempld.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-lp3943.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max7300.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max730x.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max732x.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-pca953x.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-pcf857x.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-rdc321x.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-sch.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-sch311x.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-tps65912.ko 2024-03-30T00:25:21.328 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-twl4030.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-twl6040.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-vx855.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm831x.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm8350.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm8994.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/ 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-a4tech.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-apple.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-belkin.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cherry.ko 2024-03-30T00:25:21.329 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-chicony.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 60657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cp2112.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cypress.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ezkey.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-generic.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-30T00:25:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kensington.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lg-g15.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 57969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 101537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 80897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech.ko 2024-03-30T00:25:21.331 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-magicmouse.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-microsoft.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-monterey.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 50273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ntrig.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 72841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-30T00:25:21.332 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-30T00:25:21.333 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 70977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 143377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-30T00:25:21.334 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 269209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid.ko 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/ 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 125625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbhid.ko 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbkbd.ko 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbmouse.ko 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 218865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/ 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 64297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 90089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 304913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ 2024-03-30T00:25:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 66793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 61809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 97881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 45457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 74313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-30T00:25:21.336 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 69305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 55105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 58521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 40721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/da9052-hwmon.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/da9055-hwmon.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 99505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-30T00:25:21.337 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc2103.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 66889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 94593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 55353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 46705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-30T00:25:21.338 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 45433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hih6130.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 140129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/jc42.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 48505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-30T00:25:21.339 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 72737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 46057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 108745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-30T00:25:21.340 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-30T00:25:21.341 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mc13783-adc.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 119841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 61433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 59641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 82713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 54337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-30T00:25:21.342 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 82473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-30T00:25:21.343 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 40481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 40353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 43441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 52361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 53201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-30T00:25:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 58257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 79833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 71249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 63129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 71225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 101377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 119409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/ 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/ 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-30T00:25:21.345 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/ 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 50289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-30T00:25:21.346 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 67681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 50969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-30T00:25:21.347 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-30T00:25:21.348 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/ 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 273513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 883777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 69177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 324233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 97825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 282329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 66049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ 2024-03-30T00:25:21.349 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/ 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 493129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/ 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 420529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/ 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 741609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 296785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ocrdma/ 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 153705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/usnic/ 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 169089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ 2024-03-30T00:25:21.350 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 266529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 100257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 111329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 128657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 129793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/ 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/evbug.ko 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/ff-memless.ko 2024-03-30T00:25:21.351 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/ 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/ 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lkkbd.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lm8323.ko 2024-03-30T00:25:21.352 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lm8333.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/newtonkbd.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/qt1070.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/qt2160.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/stowaway.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/sunkbd.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-03-30T00:25:21.353 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/xtkbd.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/matrix-keymap.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/ 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 123177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 89377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/gpio_mouse.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 335857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/psmouse.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-30T00:25:21.354 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/ 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 109553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ct82c710.ko 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/parkbd.ko 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/pcips2.ko 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ps2mult.ko 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serport.ko 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/ 2024-03-30T00:25:21.355 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-88pm860x.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-adp5520.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 59649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-bd2802.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-da903x.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-da9052.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-gpio.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3533.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm355x.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3642.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp8788.ko 2024-03-30T00:25:21.356 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-max8997.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-mc13783.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-menf21bmc.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca9532.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca955x.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca963x.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-tca6507.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-wm831x-status.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-wm8350.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-30T00:25:21.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/ 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/ 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 647033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 40473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 93313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bufio.ko 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 155801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-30T00:25:21.358 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 107929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 53761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-era.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-mirror.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 77865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 70977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-region-hash.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 109433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-snapshot.ko 2024-03-30T00:25:21.359 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 181761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 53729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-zero.ko 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/ 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 199825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 100833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 126361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 368481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/ 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/ 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 178041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 63233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-30T00:25:21.360 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 46241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 123225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 80913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 52857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/ 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm800.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm805.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm80x.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/arizona-i2c.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 123625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/arizona.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/bcm590xx.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/cros_ec_dev.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/dln2.ko 2024-03-30T00:25:21.361 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/janz-cmodio.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 53481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/kempld-core.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lm3533-core.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lp3943.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 56961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/max8907.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mc13xxx-core.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/menf21bmc.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633-adc.ko 2024-03-30T00:25:21.362 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633-gpio.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/retu-mfd.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 80329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/si476x-core.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 48361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/tps65010.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/tps6507x.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 44249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wm8994.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-03-30T00:25:21.363 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 48385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ad525x_dpot.ko 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/ 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 46489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 45489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/ 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/core.ko 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/ 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ds1682.ko 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/dummy-irq.ko 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/echo/ 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/echo/echo.ko 2024-03-30T00:25:21.364 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/ 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/genwqe/ 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 152041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hmc6352.ko 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ibmasm/ 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 72505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ics932s401.ko 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/ 2024-03-30T00:25:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/ 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 126137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 56361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 313545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/phantom.ko 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ti-st/ 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 74289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ti-st/st_drv.ko 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 48337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-30T00:25:21.366 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/ 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 183513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/ 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 491561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/ 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/ 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 105929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 40209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/ 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 458905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ 2024-03-30T00:25:21.367 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/ 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 45953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 130657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 68777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 64609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 101609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/ 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 54617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/ 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 69161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-30T00:25:21.368 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/ 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 59985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/ 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/ 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 54609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 87449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/ 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/ 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 97185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 102481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-30T00:25:21.369 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 90185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/ 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 81313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 69537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/ 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 78377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 176689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1434449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 128361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/ 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 131265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 314321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-30T00:25:21.370 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/ 2024-03-30T00:25:21.371 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/ 2024-03-30T00:25:21.371 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 323681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-30T00:25:21.371 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/ 2024-03-30T00:25:21.371 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-03-30T00:25:21.371 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 193537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-30T00:25:21.446 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-03-30T00:25:21.446 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 347161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-30T00:25:21.446 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-03-30T00:25:21.446 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1045321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 166129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/ 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/ 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 204185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/ 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/ 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 69425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 61417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 141153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-30T00:25:21.447 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 62825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/ 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 60665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 62825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 45481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ec_bhf.ko 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/ 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/ 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 281193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fujitsu/ 2024-03-30T00:25:21.448 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 45545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 90297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/ 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 295673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/ 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 584297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/ 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 245977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/ 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 944449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/ 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 288481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/ 2024-03-30T00:25:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 514337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/ 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 93465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 735129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 151649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 86985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/ 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 102433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 122601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/ 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-03-30T00:25:21.450 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 727377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 278825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1122065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 52977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 40873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 95569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/ 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 108401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ 2024-03-30T00:25:21.451 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 73609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/ 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 151193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/ 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 136793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/ 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 50425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/ 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 227841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 87009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-03-30T00:25:21.452 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 671985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/ 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/ 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 67273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 83801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 194057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/ 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 104001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/ 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1066817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/ 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/ 2024-03-30T00:25:21.453 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 54721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 63193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/ 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 61417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 67289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 73249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 68041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/ 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 495641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/ 2024-03-30T00:25:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 103177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 166041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 66265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 78713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/ 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 50009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/ 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 92337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/ 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 79929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 83713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/ 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 46305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-30T00:25:21.455 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/ 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 59977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/ 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/defxx.ko 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/skfp/ 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 217025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/skfp/skfp.ko 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 74985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/ 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 237241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/ 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 85841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 62889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-30T00:25:21.456 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/ 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-gpio.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 45049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/ 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-30T00:25:21.457 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 44625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 44817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 81953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 76561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 99001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-30T00:25:21.458 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/plip/ 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 43761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/plip/plip.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-30T00:25:21.459 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sb1000.ko 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/ 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 64553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/ 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 107225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 86817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/ 2024-03-30T00:25:21.460 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 144345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/ 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 307169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/ 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 143993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/ 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/ 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 267721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 103353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/ 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 131297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-30T00:25:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 85209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/ 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/ 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pci-stub.ko 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 46361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/ 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 146129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia.ko 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 62065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia_core.ko 2024-03-30T00:25:21.462 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 112785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/ 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/broadcom/ 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/samsung/ 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/ 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/ 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 69025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/ 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/ 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-03-30T00:25:21.463 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/ 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 73593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 59561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 131937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-30T00:25:21.464 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 40161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/ibm_rtl.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/ 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 44529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-30T00:25:21.465 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 48473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/ 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/88pm860x_battery.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/88pm860x_charger.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 54953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq2415x_charger.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 59497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq24190_charger.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq24735-charger.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-03-30T00:25:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/da9030_battery.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/da9052-battery.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2760_battery.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2780_battery.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2781_battery.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2782_battery.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/gpio-charger.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/isp1704_charger.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/lp8727_charger.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max14577_charger.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max17042_battery.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max8903_charger.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max8925_power.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/pcf50633-charger.ko 2024-03-30T00:25:21.467 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/sbs-battery.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/test_power.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/tps65090-charger.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm831x_backup.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm831x_power.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm8350_power.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/ 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 58097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/ 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/ 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-30T00:25:21.468 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 70849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/ 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-88pm80x.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-88pm860x.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9052.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9055.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9063.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 68473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-30T00:25:21.469 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-lp8788.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t86.ko 2024-03-30T00:25:21.470 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8907.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8925.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8997.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8998.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-palmas.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 46529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf50633.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rc5t583.ko 2024-03-30T00:25:21.471 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-s35390a.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-tps6586x.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-tps65910.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-wm831x.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-wm8350.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 77521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-30T00:25:21.472 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 57713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 55009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 135249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/ 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 271385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 100529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/ 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 356833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 295889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic94xx/ 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 189089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/ 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 108193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-30T00:25:21.473 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/ 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 246913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/ 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1171857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/ 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 238369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/ 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 115305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 40777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/ 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 308905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/ 2024-03-30T00:25:21.474 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 76033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 101417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 146745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 75969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/ 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 212329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 60497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/ 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 78217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 157713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fdomain.ko 2024-03-30T00:25:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/ 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 294881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 201721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 48937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/imm.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 43969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 265937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 101873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/ 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 317129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 52857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/ 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 315273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-30T00:25:21.476 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 145489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 53801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/ 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 216345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/ 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 2193953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/ 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 73465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 310665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 85513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/ 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 659585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/ 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 119465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-30T00:25:21.477 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 78033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/ 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 81545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/ 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 366305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 124593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ppa.ko 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 55329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/ 2024-03-30T00:25:21.478 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1876657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 85185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/ 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 558897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qlogicfas408.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 261073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 173921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 280105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 97617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 73857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 44881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 115641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-30T00:25:21.479 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 46729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/ 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 164865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 44849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 50225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/xen-scsifront.ko 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 129345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/ 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/ 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 736665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/ 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-30T00:25:21.480 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 931433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 44145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 122481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/ 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 75497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/ 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/ 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/ 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-30T00:25:21.481 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ 2024-03-30T00:25:21.482 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 78329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 57769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/moxa.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 50857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/mxser.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 108481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 67833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/ 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/ 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 59393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/altera_uart.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 72545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/ 2024-03-30T00:25:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 101969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/rp2.ko 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 53169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/sc16is7xx.ko 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/ 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 545377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/ 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 44465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_dmem_genirq.ko 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_mf624.ko 2024-03-30T00:25:21.484 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_netx.ko 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/ 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/c67x00/ 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 73201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/c67x00/c67x00.ko 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 145753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/ 2024-03-30T00:25:21.485 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 87001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 89761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/common/ 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/common/ulpi.ko 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc2/ 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/ 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 452785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3.ko 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/ 2024-03-30T00:25:21.486 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/ 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 60321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/u_ether.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/u_serial.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 92401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 50937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 122241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 59857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-03-30T00:25:21.487 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 57625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 85505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 58009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 67345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/ 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-03-30T00:25:21.488 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 62361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 161193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/libcomposite.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/ 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/bdc/ 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 157593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 52921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 69609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 55945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-03-30T00:25:21.489 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 75961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/net2272.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 74977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/net2280.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 73009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 65873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 65993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 81401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 156321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/ 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/bcma-hcd.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 80009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/isp116x-hcd.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 87249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 78769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811_cs.ko 2024-03-30T00:25:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/ssb-hcd.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/ 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/cytherm.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ehset.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-30T00:25:21.491 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/lvstest.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/ 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 64737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 70713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbtest.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/mon/ 2024-03-30T00:25:21.492 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 72801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/mon/usbmon.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/musb/ 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 301593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/musb/musb_hdrc.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/ 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-isp1301.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-tahvo.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 70273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-30T00:25:21.493 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 58113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 44177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/f81232.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 175097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 53009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 136977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 124737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 67673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 58129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 107425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-30T00:25:21.494 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/metro-usb.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 66353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 78121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 56241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mxuport.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 191817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 54257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-30T00:25:21.495 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 61385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 125513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usbserial.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/wishbone-serial.ko 2024-03-30T00:25:21.496 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 52753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-30T00:25:21.497 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 180281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/ 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 67385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 71001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 103977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/ 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/ 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 162841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-30T00:25:21.498 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 103649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 76433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/ 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 113657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 57993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 89049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/ 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/ 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/aat2870_bl.ko 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp5520_bl.ko 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp8860_bl.ko 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp8870_bl.ko 2024-03-30T00:25:21.499 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/as3711_bl.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/bd6107.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/da903x_bl.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/da9052_bl.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/gpio_backlight.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/kb3886_bl.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lm3533_bl.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lm3639_bl.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lv5207lp.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/max8925_bl.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/pandora_bl.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-03-30T00:25:21.500 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/wm831x_bl.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/ 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/matrox_w1.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/w1-gpio.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/ 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-30T00:25:21.501 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 56329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 103025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/acquirewdt.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/advantechwdt.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/cpu5wdt.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9052_wdt.ko 2024-03-30T00:25:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9055_wdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9063_wdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/dw_wdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/eurotechwdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/kempld_wdt.ko 2024-03-30T00:25:21.503 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-30T00:25:21.504 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/mena21_wdt.ko 2024-03-30T00:25:21.504 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-03-30T00:25:21.504 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-30T00:25:21.504 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-03-30T00:25:21.504 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pc87413_wdt.ko 2024-03-30T00:25:21.504 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-30T00:25:21.504 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-30T00:25:21.504 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/retu_wdt.ko 2024-03-30T00:25:21.504 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-03-30T00:25:21.504 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-03-30T00:25:21.504 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-30T00:25:21.505 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sc1200wdt.ko 2024-03-30T00:25:21.505 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-30T00:25:21.505 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-03-30T00:25:21.505 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-30T00:25:21.505 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-30T00:25:21.505 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/twl4030_wdt.ko 2024-03-30T00:25:21.505 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wafer5823wdt.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wm831x_wdt.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wm8350_wdt.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/ 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/ 2024-03-30T00:25:21.506 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 176337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 40081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 71145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-scsiback.ko 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/ 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/ 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 158217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/adfs/ 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 61681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/adfs/adfs.ko 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/ 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 163369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/afs/ 2024-03-30T00:25:21.507 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1019049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/afs/kafs.ko 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/autofs/ 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 103185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/autofs/autofs4.ko 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/ 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 64585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/bfs/ 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 41001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/bfs/bfs.ko 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/ 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 3487249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/ 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 305841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1335577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/ 2024-03-30T00:25:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 97713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/configfs/ 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 108545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/configfs/configfs.ko 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/ 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/ 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 644161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/efs/ 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/efs/efs.ko 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/ 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1527561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/ 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/freevxfs/ 2024-03-30T00:25:21.509 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/freevxfs/freevxfs.ko 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/ 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/ 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1231233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/ 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 141625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/ 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 223177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hpfs/ 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 193481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hpfs/hpfs.ko 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/ 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 74689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-30T00:25:21.510 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/ 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 420329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/ 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 281617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/ 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 88897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/ 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 490409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/ 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/ 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 81433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/ 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 59337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/ 2024-03-30T00:25:21.511 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 117361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1058273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 66345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv2.ko 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 120169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 2041705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/ 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/ 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1511641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/ 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 582409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/ 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-30T00:25:21.512 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ascii.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-30T00:25:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-30T00:25:21.514 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 94033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 142569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 165641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 118017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-30T00:25:21.515 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ntfs/ 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 220457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ntfs/ntfs.ko 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 244633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 440633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-30T00:25:21.516 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 54161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 2590681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/omfs/ 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 53681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/omfs/omfs.ko 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/ 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 360897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 56929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx4/ 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx4/qnx4.ko 2024-03-30T00:25:21.517 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx6/ 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 46601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx6/qnx6.ko 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/ 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_tree.ko 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_v1.ko 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_v2.ko 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/ 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 532161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/ 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/ 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/ 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 2712385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-30T00:25:21.518 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/ 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/ 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 134705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/ 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 91281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/ 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 248329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 162737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/ 2024-03-30T00:25:21.519 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 4035417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-30T00:25:21.553 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/ 2024-03-30T00:25:21.553 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/locking/ 2024-03-30T00:25:21.553 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 80481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/locking/locktorture.ko 2024-03-30T00:25:21.553 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/time/ 2024-03-30T00:25:21.553 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/time/test_udelay.ko 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 60257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc7.ko 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/ 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/interval_tree_test.ko 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/ 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-30T00:25:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/ 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/memory-notifier-error-inject.ko 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/notifier-error-inject.ko 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/percpu_test.ko 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/pm-notifier-error-inject.ko 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/ 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 143257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/rbtree_test.ko 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/ 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test-kstrtox.ko 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test-string_helpers.ko 2024-03-30T00:25:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 4863217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_bpf.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 48985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_firmware.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_module.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_user_copy.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/xz/ 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/xz/xz_dec_test.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/ 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/ 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 55753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-30T00:25:21.556 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 6897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/ 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/ 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 82865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/ 2024-03-30T00:25:21.557 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 198809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/ 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 667353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/ 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 67753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 790545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-30T00:25:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-30T00:25:21.559 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 55937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/ 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1046689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/ 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 125697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/ 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 194441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 58737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 61393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-30T00:25:21.560 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/ 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 390033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/hsr/ 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 102033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/hsr/hsr.ko 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 45481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 284665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 47873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ 2024-03-30T00:25:21.561 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 48233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 51209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 61129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 61505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-30T00:25:21.562 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ip_tables.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-30T00:25:21.563 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-30T00:25:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 45041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-30T00:25:21.565 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 79921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_gre.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 94161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 54265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/mip6.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-30T00:25:21.566 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-30T00:25:21.567 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 74873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/ 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 84393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/ 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 125201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 25809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-30T00:25:21.568 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 53225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/lapb/ 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 48097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/lapb/lapb.ko 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/ 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 268449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc2.ko 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/ 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 309361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/ 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 119689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-30T00:25:21.569 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 99985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 100273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 102545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 102337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 111665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 59185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 108337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 112137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 112401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 110329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-30T00:25:21.570 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 113281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 412433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-30T00:25:21.571 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 411665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 118145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 108305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 66529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-30T00:25:21.572 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 92217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 96489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 44129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 623337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 42505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-30T00:25:21.573 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 22929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 40801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-30T00:25:21.574 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_osf.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_synproxy.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tproxy.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 89225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/x_tables.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-30T00:25:21.575 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 18073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 38769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-30T00:25:21.576 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-30T00:25:21.577 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 43193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-30T00:25:21.578 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-30T00:25:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 29993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 16809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 8449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpudp.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/ 2024-03-30T00:25:21.580 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 14433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/ 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 12329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/ 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 384137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/ 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/ 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 112521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/phonet.ko 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 55041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/pn_pep.ko 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/ 2024-03-30T00:25:21.581 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 253881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 165049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 68065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rxrpc/ 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 828145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rxrpc/rxrpc.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/ 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 37081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 43345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-30T00:25:21.582 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 19177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_cgroup.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 39513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 62649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 13313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 33417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-30T00:25:21.583 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 34585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq_codel.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 35449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 49841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 21969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 66441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 30217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 36145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 27401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-30T00:25:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 24769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 48529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 32809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 26785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 31313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 28265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/ 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 932785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/ 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/ 2024-03-30T00:25:21.585 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 336969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 55009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 1513769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/ 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 704697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/ 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 10393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 867121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/ 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/ 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 87737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 67937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 98089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-30T00:25:21.586 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 11073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 17809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/x25/ 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 131209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/x25/x25.ko 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/ 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_algo.ko 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 20545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 105209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_user.ko 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/ 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/ 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 15433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 642383 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 625850 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 7677 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 23498 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-30T00:25:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 9916 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 70732 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 147214 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 233213 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 210 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 72614 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 879 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 257955 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 312782 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/ 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/ 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/doc/ 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/ 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 169 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/changelog.Debian.gz 2024-03-30T00:25:21.588 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r-- root/root 691 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/copyright 2024-03-30T00:25:21.593 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-30T00:25:21.593 INFO:teuthology.task.kernel:searching for kernel 6.8.0-g1df0c7e8a235 2024-03-30T00:25:21.593 DEBUG:teuthology.orchestra.run.smithi156:> egrep '(submenu|menuentry.*6.8.0-g1df0c7e8a235).*{' /boot/grub/grub.cfg 2024-03-30T00:25:21.600 INFO:teuthology.orchestra.run.smithi156.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-30T00:25:21.600 INFO:teuthology.orchestra.run.smithi156.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g1df0c7e8a235' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g1df0c7e8a235-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-30T00:25:21.600 INFO:teuthology.orchestra.run.smithi156.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g1df0c7e8a235 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g1df0c7e8a235-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-30T00:25:21.601 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-03-30T00:25:21.601 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.8.0-g1df0c7e8a235 2024-03-30T00:25:21.601 DEBUG:teuthology.orchestra.run.smithi156:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-03-30T00:25:21.644 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi156.front.sepia.ceph.com to complete... 2024-03-30T00:25:22.068 INFO:teuthology.orchestra.run.smithi156.stderr:Sourcing file `/etc/default/grub' 2024-03-30T00:25:22.068 INFO:teuthology.orchestra.run.smithi156.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-30T00:25:22.090 INFO:teuthology.orchestra.run.smithi156.stderr:Generating grub configuration file ... 2024-03-30T00:25:22.268 INFO:teuthology.orchestra.run.smithi156.stderr:Found linux image: /boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-30T00:25:22.274 INFO:teuthology.orchestra.run.smithi156.stderr:Found initrd image: /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-30T00:25:22.655 INFO:teuthology.orchestra.run.smithi156.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-30T00:25:22.660 INFO:teuthology.orchestra.run.smithi156.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-30T00:25:22.697 INFO:teuthology.orchestra.run.smithi156.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-30T00:25:22.702 INFO:teuthology.orchestra.run.smithi156.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-30T00:25:22.788 INFO:teuthology.orchestra.run.smithi066.stderr:Sourcing file `/etc/default/grub' 2024-03-30T00:25:22.800 INFO:teuthology.orchestra.run.smithi066.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-30T00:25:22.816 INFO:teuthology.orchestra.run.smithi156.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-30T00:25:22.816 INFO:teuthology.orchestra.run.smithi156.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-30T00:25:22.816 INFO:teuthology.orchestra.run.smithi156.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-30T00:25:22.823 INFO:teuthology.orchestra.run.smithi066.stderr:Generating grub configuration file ... 2024-03-30T00:25:22.831 INFO:teuthology.orchestra.run.smithi156.stderr:done 2024-03-30T00:25:23.118 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-30T00:25:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-30T00:25:23.430 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-30T00:25:23.437 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-30T00:25:23.506 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-30T00:25:23.651 INFO:teuthology.orchestra.run.smithi066.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-30T00:25:23.651 INFO:teuthology.orchestra.run.smithi066.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-30T00:25:23.651 INFO:teuthology.orchestra.run.smithi066.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-30T00:25:23.694 INFO:teuthology.orchestra.run.smithi066.stderr:done 2024-03-30T00:25:23.879 DEBUG:teuthology.orchestra.run.smithi066:> dpkg-deb -c /tmp/linux-image.deb 2024-03-30T00:25:23.887 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./ 2024-03-30T00:25:23.888 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./boot/ 2024-03-30T00:25:23.888 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 4474334 2024-03-25 03:29 ./boot/System.map-6.8.0-g1df0c7e8a235 2024-03-30T00:25:23.900 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 170899 2024-03-25 03:29 ./boot/config-6.8.0-g1df0c7e8a235 2024-03-30T00:25:23.900 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11551744 2024-03-25 03:29 ./boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-30T00:25:23.922 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/ 2024-03-30T00:25:23.922 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/ 2024-03-30T00:25:23.922 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/postinst.d/ 2024-03-30T00:25:23.922 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/postrm.d/ 2024-03-30T00:25:23.922 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/preinst.d/ 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/prerm.d/ 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/ 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/ 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/ 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/ 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/ 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/ 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 490441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/aesni-intel.ko 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-30T00:25:23.923 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 93073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 106689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 152137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 83849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-30T00:25:23.924 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-30T00:25:23.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-30T00:25:23.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-30T00:25:23.925 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/ 2024-03-30T00:25:23.925 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/ 2024-03-30T00:25:23.925 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/ 2024-03-30T00:25:23.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-30T00:25:23.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpuid.ko 2024-03-30T00:25:23.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/msr.ko 2024-03-30T00:25:23.925 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/ 2024-03-30T00:25:23.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 354225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-30T00:25:23.925 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 886129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-30T00:25:23.926 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 2294793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-30T00:25:24.000 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ 2024-03-30T00:25:24.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/af_alg.ko 2024-03-30T00:25:24.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/algif_hash.ko 2024-03-30T00:25:24.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/algif_skcipher.ko 2024-03-30T00:25:24.000 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-30T00:25:24.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-30T00:25:24.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-30T00:25:24.001 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/asymmetric_keys/ 2024-03-30T00:25:24.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-03-30T00:25:24.001 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/ 2024-03-30T00:25:24.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-30T00:25:24.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-30T00:25:24.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-30T00:25:24.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-30T00:25:24.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-30T00:25:24.001 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-30T00:25:24.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-30T00:25:24.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-30T00:25:24.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-30T00:25:24.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-30T00:25:24.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-30T00:25:24.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-30T00:25:24.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-30T00:25:24.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-30T00:25:24.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-30T00:25:24.002 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cryptd.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_simd.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-30T00:25:24.003 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lrw.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 143025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-30T00:25:24.004 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xts.ko 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_extlog.ko 2024-03-30T00:25:24.005 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-30T00:25:24.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-30T00:25:24.006 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/apei/ 2024-03-30T00:25:24.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/apei/einj.ko 2024-03-30T00:25:24.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-30T00:25:24.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-30T00:25:24.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-30T00:25:24.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-30T00:25:24.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 104417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-30T00:25:24.006 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ 2024-03-30T00:25:24.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-30T00:25:24.006 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 89929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 88305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-30T00:25:24.007 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_legacy.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-30T00:25:24.008 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_radisys.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rz1000.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-30T00:25:24.009 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 83249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-30T00:25:24.010 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/ 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 124385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/ 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/ 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 113801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/ 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 964745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 169697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/ 2024-03-30T00:25:24.011 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 94209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 116329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/ 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 110321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 98185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 263633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/ 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 98105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/ 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/applicom.ko 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/ 2024-03-30T00:25:24.012 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 148513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 139305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/ 2024-03-30T00:25:24.013 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 84001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/nvram.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/ 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/ 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_class.ko 2024-03-30T00:25:24.014 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_core.ko 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/ 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-palmas.ko 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-si5351.ko 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-twl6040.ko 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-wm831x.ko 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/ 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ 2024-03-30T00:25:24.015 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-30T00:25:24.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 246777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-30T00:25:24.016 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/ 2024-03-30T00:25:24.016 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/ 2024-03-30T00:25:24.016 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/ 2024-03-30T00:25:24.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 570217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-30T00:25:24.016 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-03-30T00:25:24.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-30T00:25:24.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-30T00:25:24.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-30T00:25:24.016 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/ 2024-03-30T00:25:24.016 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-30T00:25:24.017 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/ 2024-03-30T00:25:24.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-30T00:25:24.017 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ 2024-03-30T00:25:24.017 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/ 2024-03-30T00:25:24.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-30T00:25:24.017 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ 2024-03-30T00:25:24.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 122793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-30T00:25:24.017 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ 2024-03-30T00:25:24.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 80297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-30T00:25:24.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-30T00:25:24.017 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-30T00:25:24.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-30T00:25:24.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-30T00:25:24.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-30T00:25:24.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-30T00:25:24.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-30T00:25:24.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-30T00:25:24.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-30T00:25:24.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-30T00:25:24.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-30T00:25:24.018 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/ 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-gpio.ko 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max14577.ko 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max77693.ko 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max8997.ko 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-palmas.ko 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-rt8973a.ko 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-sm5502.ko 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/ 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/dmi-sysfs.ko 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/efi/ 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/efi/efi-pstore.ko 2024-03-30T00:25:24.019 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/ 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-adp5520.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-amd8111.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-arizona.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-da9052.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-da9055.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-dln2.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 77745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-f7188x.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-generic.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-kempld.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-lp3943.ko 2024-03-30T00:25:24.020 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max7300.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max730x.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max732x.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-pca953x.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-pcf857x.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-rdc321x.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-sch.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-sch311x.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-tps65912.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-twl4030.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-twl6040.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-vx855.ko 2024-03-30T00:25:24.021 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm831x.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm8350.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm8994.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/ 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-a4tech.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-apple.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-belkin.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cherry.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-chicony.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cp2112.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cypress.ko 2024-03-30T00:25:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ezkey.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-generic.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kensington.ko 2024-03-30T00:25:24.023 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lg-g15.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 101537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 80897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-magicmouse.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-microsoft.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-monterey.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ntrig.ko 2024-03-30T00:25:24.024 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 72841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-30T00:25:24.025 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-30T00:25:24.026 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 143377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 269209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/ 2024-03-30T00:25:24.027 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 125625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbhid.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbkbd.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbmouse.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 218865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/ 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 64297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 304913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-30T00:25:24.028 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 97881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-30T00:25:24.029 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-30T00:25:24.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-30T00:25:24.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-30T00:25:24.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-30T00:25:24.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-30T00:25:24.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-30T00:25:24.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/da9052-hwmon.ko 2024-03-30T00:25:24.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/da9055-hwmon.ko 2024-03-30T00:25:24.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-30T00:25:24.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 99505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-30T00:25:24.030 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc2103.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 94593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hih6130.ko 2024-03-30T00:25:24.031 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 140129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/jc42.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-30T00:25:24.032 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 72737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 108745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-30T00:25:24.033 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mc13783-adc.ko 2024-03-30T00:25:24.034 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-30T00:25:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-03-30T00:25:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-30T00:25:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 119841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-30T00:25:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-30T00:25:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-30T00:25:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 82713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-30T00:25:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-30T00:25:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-30T00:25:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ 2024-03-30T00:25:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-30T00:25:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-30T00:25:24.036 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-30T00:25:24.036 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-30T00:25:24.036 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-30T00:25:24.036 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-30T00:25:24.036 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-30T00:25:24.036 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 82473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-30T00:25:24.036 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-30T00:25:24.036 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-30T00:25:24.036 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-30T00:25:24.036 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-30T00:25:24.036 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-30T00:25:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 79833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 71249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 63129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-30T00:25:24.038 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 71225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 101377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 119409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/ 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/ 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/ 2024-03-30T00:25:24.039 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-03-30T00:25:24.040 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-03-30T00:25:24.041 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-03-30T00:25:24.042 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-30T00:25:24.042 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-03-30T00:25:24.042 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-30T00:25:24.042 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-30T00:25:24.042 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-30T00:25:24.042 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-30T00:25:24.042 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-03-30T00:25:24.042 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-30T00:25:24.042 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-30T00:25:24.042 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-30T00:25:24.043 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/ 2024-03-30T00:25:24.043 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-03-30T00:25:24.043 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-03-30T00:25:24.043 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-03-30T00:25:24.043 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ 2024-03-30T00:25:24.043 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ 2024-03-30T00:25:24.043 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 273513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-30T00:25:24.043 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 883777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-30T00:25:24.043 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-30T00:25:24.043 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 324233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-30T00:25:24.043 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 97825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-30T00:25:24.044 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 282329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-30T00:25:24.044 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-30T00:25:24.044 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ 2024-03-30T00:25:24.044 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/ 2024-03-30T00:25:24.044 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 493129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-30T00:25:24.044 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/ 2024-03-30T00:25:24.044 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 420529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-30T00:25:24.044 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/ 2024-03-30T00:25:24.044 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 741609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-30T00:25:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ 2024-03-30T00:25:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 296785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-30T00:25:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ocrdma/ 2024-03-30T00:25:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 153705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-03-30T00:25:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/usnic/ 2024-03-30T00:25:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 169089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-03-30T00:25:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ 2024-03-30T00:25:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ 2024-03-30T00:25:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 266529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-30T00:25:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ 2024-03-30T00:25:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 100257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-30T00:25:24.046 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ 2024-03-30T00:25:24.046 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 111329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-30T00:25:24.046 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ 2024-03-30T00:25:24.046 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 128657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-30T00:25:24.046 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ 2024-03-30T00:25:24.046 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 129793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-30T00:25:24.046 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/ 2024-03-30T00:25:24.046 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/evbug.ko 2024-03-30T00:25:24.046 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/ff-memless.ko 2024-03-30T00:25:24.046 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/ 2024-03-30T00:25:24.047 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-30T00:25:24.047 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-30T00:25:24.047 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-30T00:25:24.047 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-30T00:25:24.047 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/ 2024-03-30T00:25:24.047 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-03-30T00:25:24.047 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-03-30T00:25:24.047 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-03-30T00:25:24.047 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-03-30T00:25:24.047 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-30T00:25:24.047 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-03-30T00:25:24.048 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lkkbd.ko 2024-03-30T00:25:24.048 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lm8323.ko 2024-03-30T00:25:24.048 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lm8333.ko 2024-03-30T00:25:24.048 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-03-30T00:25:24.048 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-03-30T00:25:24.048 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-03-30T00:25:24.048 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-03-30T00:25:24.048 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/newtonkbd.ko 2024-03-30T00:25:24.048 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-03-30T00:25:24.048 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/qt1070.ko 2024-03-30T00:25:24.048 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/qt2160.ko 2024-03-30T00:25:24.049 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-03-30T00:25:24.049 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/stowaway.ko 2024-03-30T00:25:24.049 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/sunkbd.ko 2024-03-30T00:25:24.049 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-03-30T00:25:24.049 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-03-30T00:25:24.049 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-03-30T00:25:24.049 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/xtkbd.ko 2024-03-30T00:25:24.049 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/matrix-keymap.ko 2024-03-30T00:25:24.049 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/ 2024-03-30T00:25:24.049 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-30T00:25:24.050 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-30T00:25:24.050 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 123177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-30T00:25:24.050 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 89377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-30T00:25:24.050 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/gpio_mouse.ko 2024-03-30T00:25:24.050 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 335857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/psmouse.ko 2024-03-30T00:25:24.050 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-30T00:25:24.050 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-30T00:25:24.050 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-30T00:25:24.050 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-30T00:25:24.051 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/ 2024-03-30T00:25:24.051 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 109553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-30T00:25:24.051 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ 2024-03-30T00:25:24.051 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-30T00:25:24.051 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-30T00:25:24.051 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ct82c710.ko 2024-03-30T00:25:24.051 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-30T00:25:24.051 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/parkbd.ko 2024-03-30T00:25:24.051 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/pcips2.ko 2024-03-30T00:25:24.051 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ps2mult.ko 2024-03-30T00:25:24.051 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serport.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/ 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-88pm860x.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-adp5520.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-bd2802.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-da903x.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-da9052.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-gpio.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3533.ko 2024-03-30T00:25:24.052 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm355x.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3642.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp8788.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-max8997.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-mc13783.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-menf21bmc.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca9532.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca955x.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca963x.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-tca6507.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-wm831x-status.ko 2024-03-30T00:25:24.053 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-wm8350.ko 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/ 2024-03-30T00:25:24.054 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/ 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 647033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 93313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bufio.ko 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 155801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 107929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-era.ko 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log.ko 2024-03-30T00:25:24.055 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-mirror.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 77865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-region-hash.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 109433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-snapshot.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 181761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-zero.ko 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/ 2024-03-30T00:25:24.056 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 199825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 100833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 126361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 368481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/ 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/ 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 178041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 63233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 123225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 80913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-30T00:25:24.057 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/ 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm800.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm805.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm80x.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/arizona-i2c.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 123625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/arizona.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/bcm590xx.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/cros_ec_dev.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/dln2.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/janz-cmodio.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/kempld-core.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lm3533-core.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lp3943.ko 2024-03-30T00:25:24.058 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/max8907.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mc13xxx-core.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/menf21bmc.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633-adc.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633-gpio.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/retu-mfd.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 80329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/si476x-core.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/tps65010.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/tps6507x.ko 2024-03-30T00:25:24.059 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wm8994.ko 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ad525x_dpot.ko 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/ 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/ 2024-03-30T00:25:24.060 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/core.ko 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/ 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ds1682.ko 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/dummy-irq.ko 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/echo/ 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/echo/echo.ko 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/ 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-30T00:25:24.061 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/genwqe/ 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 152041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hmc6352.ko 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ibmasm/ 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 72505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ics932s401.ko 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/ 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-30T00:25:24.062 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/ 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 126137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 313545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/phantom.ko 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ti-st/ 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ti-st/st_drv.ko 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/ 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 183513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-30T00:25:24.063 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/ 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 491561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/ 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/ 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 105929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/ 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 458905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-30T00:25:24.064 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/ 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 130657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 68777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 64609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 101609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/ 2024-03-30T00:25:24.065 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/ 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/ 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/ 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/ 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 87449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/ 2024-03-30T00:25:24.066 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/ 2024-03-30T00:25:24.067 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 97185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-30T00:25:24.067 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-03-30T00:25:24.067 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 102481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-30T00:25:24.067 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-03-30T00:25:24.067 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-30T00:25:24.067 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/ 2024-03-30T00:25:24.067 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 81313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-30T00:25:24.067 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-30T00:25:24.067 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/ 2024-03-30T00:25:24.067 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-30T00:25:24.068 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 176689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-30T00:25:24.068 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-03-30T00:25:24.068 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1434449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-30T00:25:24.159 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 128361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-30T00:25:24.160 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/ 2024-03-30T00:25:24.160 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 131265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-30T00:25:24.160 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 314321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-30T00:25:24.160 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/ 2024-03-30T00:25:24.160 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/ 2024-03-30T00:25:24.160 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 323681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-30T00:25:24.160 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/ 2024-03-30T00:25:24.160 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-03-30T00:25:24.160 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 193537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-30T00:25:24.160 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 347161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1045321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 166129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/ 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/ 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 204185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/ 2024-03-30T00:25:24.161 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/ 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 141153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/ 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ec_bhf.ko 2024-03-30T00:25:24.162 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/ 2024-03-30T00:25:24.163 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/ 2024-03-30T00:25:24.163 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 281193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-30T00:25:24.163 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-30T00:25:24.163 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-30T00:25:24.163 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fujitsu/ 2024-03-30T00:25:24.163 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-03-30T00:25:24.163 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ 2024-03-30T00:25:24.163 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 90297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-30T00:25:24.163 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/ 2024-03-30T00:25:24.163 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 295673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-30T00:25:24.163 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/ 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 584297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/ 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 245977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/ 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 944449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/ 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 288481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/ 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 514337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/ 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 93465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 735129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-30T00:25:24.164 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 151649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 86985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/ 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 102433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 122601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/ 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 727377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 278825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-03-30T00:25:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1122065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 95569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/ 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 108401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 73609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/ 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 151193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-30T00:25:24.166 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/ 2024-03-30T00:25:24.167 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 136793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-30T00:25:24.167 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/ 2024-03-30T00:25:24.167 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-30T00:25:24.167 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-30T00:25:24.167 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/ 2024-03-30T00:25:24.167 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-03-30T00:25:24.167 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 227841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-30T00:25:24.167 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 87009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-30T00:25:24.167 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-03-30T00:25:24.167 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 671985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-30T00:25:24.167 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/ 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/ 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 83801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 194057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/ 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 104001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/ 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1066817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-30T00:25:24.168 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/ 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/ 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 63193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/ 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 73249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 68041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/ 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-30T00:25:24.169 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 495641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/ 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 103177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 166041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/ 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/ 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 92337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/ 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 79929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-30T00:25:24.170 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 83713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/ 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/ 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/ 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/defxx.ko 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/skfp/ 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 217025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/skfp/skfp.ko 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/ 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 237241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-30T00:25:24.171 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/ 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/ 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-gpio.ko 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-30T00:25:24.172 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/ 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-30T00:25:24.173 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 81953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 76561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 99001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/plip/ 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/plip/plip.ko 2024-03-30T00:25:24.174 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ 2024-03-30T00:25:24.175 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-30T00:25:24.175 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-30T00:25:24.175 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-30T00:25:24.175 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-30T00:25:24.175 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-30T00:25:24.175 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-30T00:25:24.175 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-30T00:25:24.175 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-30T00:25:24.175 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sb1000.ko 2024-03-30T00:25:24.175 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/ 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 64553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/ 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 107225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 86817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/ 2024-03-30T00:25:24.176 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 144345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/ 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 307169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/ 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 143993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/ 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/ 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 267721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 103353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/ 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 131297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-30T00:25:24.177 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/ 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/ 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pci-stub.ko 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/ 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 146129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia.ko 2024-03-30T00:25:24.178 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia_core.ko 2024-03-30T00:25:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-03-30T00:25:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-30T00:25:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 112785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-30T00:25:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/ 2024-03-30T00:25:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/broadcom/ 2024-03-30T00:25:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-03-30T00:25:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/samsung/ 2024-03-30T00:25:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-03-30T00:25:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/ 2024-03-30T00:25:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/ 2024-03-30T00:25:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/ 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/ 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec.ko 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/ 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 73593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-30T00:25:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 131937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/ibm_rtl.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/ 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-30T00:25:24.181 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/ 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/88pm860x_battery.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/88pm860x_charger.ko 2024-03-30T00:25:24.182 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq2415x_charger.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq24190_charger.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq24735-charger.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/da9030_battery.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/da9052-battery.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2760_battery.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2780_battery.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2781_battery.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2782_battery.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/gpio-charger.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/isp1704_charger.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/lp8727_charger.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max14577_charger.ko 2024-03-30T00:25:24.183 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max17042_battery.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max8903_charger.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max8925_power.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/pcf50633-charger.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/sbs-battery.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/test_power.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/tps65090-charger.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm831x_backup.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm831x_power.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm8350_power.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/ 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/ 2024-03-30T00:25:24.184 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/ 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/ 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-88pm80x.ko 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-88pm860x.ko 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9052.ko 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9055.ko 2024-03-30T00:25:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9063.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 68473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-30T00:25:24.186 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-lp8788.ko 2024-03-30T00:25:24.187 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-30T00:25:24.187 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-30T00:25:24.187 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-30T00:25:24.187 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t86.ko 2024-03-30T00:25:24.187 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-30T00:25:24.187 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8907.ko 2024-03-30T00:25:24.187 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8925.ko 2024-03-30T00:25:24.187 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8997.ko 2024-03-30T00:25:24.187 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8998.ko 2024-03-30T00:25:24.187 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-03-30T00:25:24.187 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-palmas.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf50633.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rc5t583.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-s35390a.ko 2024-03-30T00:25:24.188 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-tps6586x.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-tps65910.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-wm831x.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-wm8350.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 77521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 135249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/ 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 271385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-30T00:25:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 100529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/ 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 356833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 295889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic94xx/ 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 189089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/ 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 108193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/ 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 246913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/ 2024-03-30T00:25:24.190 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1171857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/ 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 238369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/ 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 115305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/ 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 308905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/ 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 76033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 101417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 146745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-30T00:25:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 75969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/ 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 212329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/ 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 157713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fdomain.ko 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/ 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 294881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 201721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-30T00:25:24.192 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/imm.ko 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 265937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 101873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/ 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 317129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/ 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 315273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 145489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-30T00:25:24.193 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/ 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 216345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/ 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 2193953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/ 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 73465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 310665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/ 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 659585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/ 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 119465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-30T00:25:24.194 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/ 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 81545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/ 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 366305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 124593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ppa.ko 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/ 2024-03-30T00:25:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1876657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/ 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 558897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qlogicfas408.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 261073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 173921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 280105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 97617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 73857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-30T00:25:24.196 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 115641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/ 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 164865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/xen-scsifront.ko 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 129345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/ 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/ 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 736665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/ 2024-03-30T00:25:24.197 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-30T00:25:24.198 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-30T00:25:24.198 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-30T00:25:24.198 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 931433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-30T00:25:24.198 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-30T00:25:24.198 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 122481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-30T00:25:24.198 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/ 2024-03-30T00:25:24.198 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 75497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-30T00:25:24.198 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/ 2024-03-30T00:25:24.199 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/ 2024-03-30T00:25:24.199 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/ 2024-03-30T00:25:24.199 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-30T00:25:24.199 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-30T00:25:24.199 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-30T00:25:24.199 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-30T00:25:24.199 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-30T00:25:24.199 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-30T00:25:24.199 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-30T00:25:24.199 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-30T00:25:24.200 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-30T00:25:24.200 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-30T00:25:24.200 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-30T00:25:24.200 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-30T00:25:24.200 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-30T00:25:24.200 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-30T00:25:24.200 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-30T00:25:24.200 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-03-30T00:25:24.200 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-30T00:25:24.201 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-30T00:25:24.201 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-30T00:25:24.201 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ 2024-03-30T00:25:24.201 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ 2024-03-30T00:25:24.201 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-30T00:25:24.201 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/moxa.ko 2024-03-30T00:25:24.201 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/mxser.ko 2024-03-30T00:25:24.201 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 108481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-30T00:25:24.201 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-30T00:25:24.201 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-30T00:25:24.201 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/ 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/ 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/altera_uart.ko 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 72545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/ 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 101969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/rp2.ko 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/sc16is7xx.ko 2024-03-30T00:25:24.202 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/ 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 545377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/ 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_dmem_genirq.ko 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_mf624.ko 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_netx.ko 2024-03-30T00:25:24.203 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/ 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/c67x00/ 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 73201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/c67x00/c67x00.ko 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 145753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/ 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 87001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-30T00:25:24.204 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 89761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/common/ 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/common/ulpi.ko 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc2/ 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/ 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 452785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3.ko 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/ 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/ 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/u_ether.ko 2024-03-30T00:25:24.205 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/u_serial.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 92401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 50937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 122241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 85505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-03-30T00:25:24.206 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/ 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-03-30T00:25:24.207 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 161193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/libcomposite.ko 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/ 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/bdc/ 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 157593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 69609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-03-30T00:25:24.208 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 75961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/net2272.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/net2280.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 73009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 65873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 65993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 81401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 156321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/ 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/bcma-hcd.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 80009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/isp116x-hcd.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 87249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-03-30T00:25:24.209 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811_cs.ko 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/ssb-hcd.ko 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/ 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/cytherm.ko 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ehset.ko 2024-03-30T00:25:24.210 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/lvstest.ko 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/ 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 64737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-30T00:25:24.211 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbtest.ko 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/mon/ 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 72801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/mon/usbmon.ko 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/musb/ 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 301593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/musb/musb_hdrc.ko 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/ 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-isp1301.ko 2024-03-30T00:25:24.212 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-tahvo.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/f81232.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 175097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-30T00:25:24.213 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 136977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 124737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 107425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/metro-usb.ko 2024-03-30T00:25:24.214 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 78121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mxuport.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 191817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-30T00:25:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 125513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usbserial.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/wishbone-serial.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ 2024-03-30T00:25:24.216 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 52753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-30T00:25:24.217 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 180281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/ 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 71001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 103977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/ 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/ 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 162841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 103649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 76433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-30T00:25:24.218 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/ 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 113657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 57993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 89049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/ 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/ 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/aat2870_bl.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp5520_bl.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp8860_bl.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp8870_bl.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/as3711_bl.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/bd6107.ko 2024-03-30T00:25:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/da903x_bl.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/da9052_bl.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/gpio_backlight.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/kb3886_bl.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lm3533_bl.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lm3639_bl.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lv5207lp.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/max8925_bl.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/pandora_bl.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/wm831x_bl.ko 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/ 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ 2024-03-30T00:25:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/matrox_w1.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/w1-gpio.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/ 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-30T00:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 103025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-30T00:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ 2024-03-30T00:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/acquirewdt.ko 2024-03-30T00:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/advantechwdt.ko 2024-03-30T00:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-30T00:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-30T00:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/cpu5wdt.ko 2024-03-30T00:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9052_wdt.ko 2024-03-30T00:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9055_wdt.ko 2024-03-30T00:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9063_wdt.ko 2024-03-30T00:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/dw_wdt.ko 2024-03-30T00:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/eurotechwdt.ko 2024-03-30T00:25:24.223 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-30T00:25:24.223 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-30T00:25:24.223 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-30T00:25:24.223 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-30T00:25:24.223 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-30T00:25:24.223 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-30T00:25:24.223 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-30T00:25:24.224 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-30T00:25:24.224 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-30T00:25:24.224 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-30T00:25:24.224 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/kempld_wdt.ko 2024-03-30T00:25:24.224 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-30T00:25:24.224 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/mena21_wdt.ko 2024-03-30T00:25:24.224 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-03-30T00:25:24.224 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-30T00:25:24.224 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-03-30T00:25:24.224 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pc87413_wdt.ko 2024-03-30T00:25:24.224 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/retu_wdt.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sc1200wdt.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/twl4030_wdt.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-30T00:25:24.225 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wafer5823wdt.ko 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wm831x_wdt.ko 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wm8350_wdt.ko 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/ 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/ 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 176337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-30T00:25:24.226 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 71145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-scsiback.ko 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/ 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/ 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 158217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/adfs/ 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/adfs/adfs.ko 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/ 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 163369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/afs/ 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1019049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/afs/kafs.ko 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/autofs/ 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 103185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/autofs/autofs4.ko 2024-03-30T00:25:24.227 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/ 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 64585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/bfs/ 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 41001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/bfs/bfs.ko 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/ 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 3487249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/ 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 305841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1335577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/ 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 97713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/configfs/ 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 108545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/configfs/configfs.ko 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/ 2024-03-30T00:25:24.228 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/ 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 644161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/efs/ 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/efs/efs.ko 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/ 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1527561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/ 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/freevxfs/ 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/freevxfs/freevxfs.ko 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/ 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/ 2024-03-30T00:25:24.229 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1231233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/ 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 141625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/ 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 223177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hpfs/ 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 193481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hpfs/hpfs.ko 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/ 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/ 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 420329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/ 2024-03-30T00:25:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 281617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/ 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 88897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/ 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 490409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/ 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/ 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 81433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/ 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/ 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 117361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1058273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv2.ko 2024-03-30T00:25:24.231 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 120169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 2041705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/ 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/ 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1511641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/ 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 582409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/ 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-30T00:25:24.232 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ascii.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-30T00:25:24.233 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-30T00:25:24.234 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 94033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 142569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 165641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 118017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-30T00:25:24.235 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ntfs/ 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 220457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ntfs/ntfs.ko 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ 2024-03-30T00:25:24.236 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 244633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 440633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 2590681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/omfs/ 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/omfs/omfs.ko 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/ 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 360897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ 2024-03-30T00:25:24.237 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 56929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx4/ 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx4/qnx4.ko 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx6/ 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 46601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx6/qnx6.ko 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/ 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_tree.ko 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_v1.ko 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_v2.ko 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/ 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 532161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/ 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-30T00:25:24.238 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/ 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/ 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 2712385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/ 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/ 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 134705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/ 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 91281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/ 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 248329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ 2024-03-30T00:25:24.239 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 162737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-30T00:25:24.240 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/ 2024-03-30T00:25:24.240 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 4035417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-30T00:25:24.280 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/ 2024-03-30T00:25:24.280 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/locking/ 2024-03-30T00:25:24.280 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 80481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/locking/locktorture.ko 2024-03-30T00:25:24.280 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/time/ 2024-03-30T00:25:24.280 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/time/test_udelay.ko 2024-03-30T00:25:24.280 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 60257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-30T00:25:24.280 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ 2024-03-30T00:25:24.280 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-30T00:25:24.280 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc7.ko 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/ 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/interval_tree_test.ko 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/ 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/ 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/memory-notifier-error-inject.ko 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/notifier-error-inject.ko 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/percpu_test.ko 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/pm-notifier-error-inject.ko 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/ 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 143257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/rbtree_test.ko 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/ 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test-kstrtox.ko 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test-string_helpers.ko 2024-03-30T00:25:24.282 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 4863217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_bpf.ko 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_firmware.ko 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_module.ko 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_user_copy.ko 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/xz/ 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/xz/xz_dec_test.ko 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/ 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/ 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-30T00:25:24.283 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-30T00:25:24.284 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-30T00:25:24.285 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-30T00:25:24.285 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-30T00:25:24.285 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-30T00:25:24.285 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 6897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-30T00:25:24.285 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-30T00:25:24.286 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/ 2024-03-30T00:25:24.286 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-30T00:25:24.286 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-30T00:25:24.286 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-30T00:25:24.286 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-30T00:25:24.286 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-30T00:25:24.286 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/ 2024-03-30T00:25:24.286 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 82865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-30T00:25:24.286 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/ 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 198809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/ 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 667353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/ 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 790545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-30T00:25:24.287 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-30T00:25:24.288 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/ 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1046689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/ 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 125697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/ 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 194441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-30T00:25:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 58737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/ 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 390033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/hsr/ 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 102033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/hsr/hsr.ko 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-30T00:25:24.290 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 284665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 47873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 51209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 61505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ 2024-03-30T00:25:24.291 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ip_tables.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-30T00:25:24.292 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-30T00:25:24.293 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-30T00:25:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 45041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 79921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_gre.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 94161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 54265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/mip6.ko 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ 2024-03-30T00:25:24.295 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-30T00:25:24.296 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 74873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-30T00:25:24.297 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/ 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 84393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/ 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 125201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 25809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 53225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/lapb/ 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/lapb/lapb.ko 2024-03-30T00:25:24.298 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/ 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 268449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc2.ko 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/ 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 309361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/ 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 119689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-30T00:25:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 99985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 100273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 102545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 102337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 111665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 59185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 108337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 112137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 112401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 110329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 113281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 412433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-30T00:25:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-30T00:25:24.301 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-30T00:25:24.302 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-30T00:25:24.302 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 411665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-30T00:25:24.302 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-30T00:25:24.302 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-30T00:25:24.302 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-30T00:25:24.302 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 118145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-30T00:25:24.302 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-30T00:25:24.302 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-30T00:25:24.302 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 108305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-30T00:25:24.302 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-30T00:25:24.302 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 92217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 96489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-30T00:25:24.303 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 44129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 623337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 42505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 22929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-30T00:25:24.304 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 40801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_osf.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-30T00:25:24.305 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_synproxy.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tproxy.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 89225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/x_tables.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 18073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-30T00:25:24.306 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 38769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-30T00:25:24.307 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-30T00:25:24.308 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-30T00:25:24.309 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-30T00:25:24.310 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 29993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 16809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 8449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-30T00:25:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpudp.ko 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/ 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 14433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/ 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 12329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/ 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 384137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/ 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-30T00:25:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/ 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 112521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/phonet.ko 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/pn_pep.ko 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/ 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 253881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 165049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 68065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rxrpc/ 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 828145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rxrpc/rxrpc.ko 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/ 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 37081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-30T00:25:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 43345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 19177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_cgroup.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 39513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 62649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 13313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-30T00:25:24.314 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 33417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 34585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq_codel.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 35449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 49841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 21969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 66441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-30T00:25:24.315 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 30217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 36145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 27401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 24769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 48529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 32809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 26785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 31313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 28265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/ 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 932785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-30T00:25:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-30T00:25:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/ 2024-03-30T00:25:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/ 2024-03-30T00:25:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 336969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-30T00:25:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 55009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-30T00:25:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 1513769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-30T00:25:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/ 2024-03-30T00:25:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 704697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-30T00:25:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/ 2024-03-30T00:25:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 10393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-30T00:25:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 867121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-30T00:25:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/ 2024-03-30T00:25:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-30T00:25:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/ 2024-03-30T00:25:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 87737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-30T00:25:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 67937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-30T00:25:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 98089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-30T00:25:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 11073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-30T00:25:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 17809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-30T00:25:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/x25/ 2024-03-30T00:25:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 131209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/x25/x25.ko 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/ 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_algo.ko 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 20545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 105209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_user.ko 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/ 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/ 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 15433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 642383 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 625850 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 7677 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 23498 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 9916 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-30T00:25:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 70732 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 147214 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 233213 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 210 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 72614 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 879 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 257955 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 312782 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/ 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/ 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/doc/ 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/ 2024-03-30T00:25:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 169 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/changelog.Debian.gz 2024-03-30T00:25:24.321 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r-- root/root 691 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/copyright 2024-03-30T00:25:24.325 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-30T00:25:24.325 INFO:teuthology.task.kernel:searching for kernel 6.8.0-g1df0c7e8a235 2024-03-30T00:25:24.325 DEBUG:teuthology.orchestra.run.smithi066:> egrep '(submenu|menuentry.*6.8.0-g1df0c7e8a235).*{' /boot/grub/grub.cfg 2024-03-30T00:25:24.333 INFO:teuthology.orchestra.run.smithi066.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-30T00:25:24.333 INFO:teuthology.orchestra.run.smithi066.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g1df0c7e8a235' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g1df0c7e8a235-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-30T00:25:24.333 INFO:teuthology.orchestra.run.smithi066.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g1df0c7e8a235 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g1df0c7e8a235-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-30T00:25:24.333 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-03-30T00:25:24.333 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.8.0-g1df0c7e8a235 2024-03-30T00:25:24.334 DEBUG:teuthology.orchestra.run.smithi066:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-03-30T00:25:24.376 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi066.front.sepia.ceph.com to complete... 2024-03-30T00:25:24.723 INFO:teuthology.orchestra.run.smithi066.stderr:Sourcing file `/etc/default/grub' 2024-03-30T00:25:24.724 INFO:teuthology.orchestra.run.smithi066.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-30T00:25:24.755 INFO:teuthology.orchestra.run.smithi066.stderr:Generating grub configuration file ... 2024-03-30T00:25:24.957 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-30T00:25:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-30T00:25:25.242 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-30T00:25:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-30T00:25:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-30T00:25:25.449 INFO:teuthology.orchestra.run.smithi066.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-30T00:25:25.449 INFO:teuthology.orchestra.run.smithi066.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-30T00:25:25.449 INFO:teuthology.orchestra.run.smithi066.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-30T00:25:25.461 INFO:teuthology.orchestra.run.smithi066.stderr:done 2024-03-30T00:25:53.974 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-30T00:25:53.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-03-30T00:25:53.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:25:56.589 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-30T00:25:56.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T00:25:56.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:26:28.511 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2024-03-30T00:26:31.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-03-30T00:26:37.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-03-30T00:26:37.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:26:40.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T00:26:40.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:26:43.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-03-30T00:26:55.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T00:26:55.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:26:58.207 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-03-30T00:27:13.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T00:27:13.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:27:16.290 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-03-30T00:27:34.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T00:27:34.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:27:37.521 DEBUG:teuthology.orchestra.remote:timed out 2024-03-30T00:27:49.530 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-03-30T00:27:49.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:27:50.071 DEBUG:teuthology.orchestra.run.smithi156:> true 2024-03-30T00:27:50.756 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-03-30T00:27:50.756 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g1df0c7e8a235"... 2024-03-30T00:27:50.756 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2024-03-30T00:27:50.807 INFO:teuthology.orchestra.run.smithi156.stdout:6.8.0-g1df0c7e8a235 2024-03-30T00:27:50.807 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-30T00:27:50.807 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-30T00:27:50.808 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-30T00:27:51.808 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-30T00:27:51.808 DEBUG:teuthology.orchestra.run.smithi156:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-30T00:27:51.881 INFO:teuthology.orchestra.run.smithi156.stdout:ttyS1 2024-03-30T00:27:51.931 DEBUG:teuthology.parallel:result is None 2024-03-30T00:27:52.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-03-30T00:28:13.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T00:28:13.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:28:14.031 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-03-30T00:28:14.685 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T00:28:14.686 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g1df0c7e8a235"... 2024-03-30T00:28:14.686 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-03-30T00:28:14.734 INFO:teuthology.orchestra.run.smithi066.stdout:6.8.0-g1df0c7e8a235 2024-03-30T00:28:14.734 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-30T00:28:14.735 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-30T00:28:14.735 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-30T00:28:15.736 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-30T00:28:15.736 DEBUG:teuthology.orchestra.run.smithi066:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-30T00:28:15.828 INFO:teuthology.orchestra.run.smithi066.stdout:ttyS1 2024-03-30T00:28:15.874 DEBUG:teuthology.parallel:result is None 2024-03-30T00:28:15.874 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-30T00:28:15.888 INFO:teuthology.task.internal:Creating test directory... 2024-03-30T00:28:15.888 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T00:28:15.892 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T00:28:15.901 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-30T00:28:15.908 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-30T00:28:15.929 INFO:teuthology.task.internal:Creating archive directory... 2024-03-30T00:28:15.929 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T00:28:15.941 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T00:28:15.963 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-30T00:28:15.969 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-30T00:28:15.969 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T00:28:15.997 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T00:28:16.022 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T00:28:16.027 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T00:28:16.035 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T00:28:16.041 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T00:28:16.044 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-30T00:28:16.051 INFO:teuthology.task.internal:Configuring sudo... 2024-03-30T00:28:16.051 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T00:28:16.090 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T00:28:16.110 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-30T00:28:16.120 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-30T00:28:16.120 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T00:28:16.157 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T00:28:16.166 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T00:28:16.216 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T00:28:16.267 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:28:16.267 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T00:28:16.343 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T00:28:16.352 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T00:28:16.404 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:28:16.404 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T00:28:16.479 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-03-30T00:28:16.482 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2024-03-30T00:28:16.595 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-30T00:28:16.604 INFO:teuthology.task.internal:Starting timer... 2024-03-30T00:28:16.604 INFO:teuthology.run_tasks:Running task pcp... 2024-03-30T00:28:16.615 INFO:teuthology.run_tasks:Running task selinux... 2024-03-30T00:28:16.624 DEBUG:teuthology.task.selinux:Excluding smithi066: OS 'ubuntu' does not support SELinux 2024-03-30T00:28:16.625 DEBUG:teuthology.task.selinux:Excluding smithi156: OS 'ubuntu' does not support SELinux 2024-03-30T00:28:16.625 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-30T00:28:16.625 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-30T00:28:16.625 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-30T00:28:16.625 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-30T00:28:16.634 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-30T00:28:16.640 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-30T00:28:16.764 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-30T00:28:16.780 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-30T00:28:16.782 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi066.front.sepia.ceph.com,smithi156.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-30T00:35:58.636 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com')] 2024-03-30T00:35:58.638 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T00:35:58.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:35:58.720 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-03-30T00:35:58.865 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T00:35:58.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-03-30T00:35:58.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:35:58.954 DEBUG:teuthology.orchestra.run.smithi156:> true 2024-03-30T00:35:59.065 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-03-30T00:35:59.065 INFO:teuthology.run_tasks:Running task clock... 2024-03-30T00:35:59.076 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-30T00:35:59.076 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T00:35:59.077 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T00:35:59.080 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T00:35:59.080 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T00:35:59.117 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-30T00:35:59.117 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: Command line: ntpd -gq 2024-03-30T00:35:59.117 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: ---------------------------------------------------- 2024-03-30T00:35:59.117 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: ntp-4 is maintained by Network Time Foundation, 2024-03-30T00:35:59.117 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-30T00:35:59.117 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: corporation. Support and training for ntp-4 are 2024-03-30T00:35:59.117 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: available at https://www.nwtime.org/support 2024-03-30T00:35:59.117 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: ---------------------------------------------------- 2024-03-30T00:35:59.118 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: proto: precision = 0.045 usec (-24) 2024-03-30T00:35:59.118 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: basedate set to 2022-02-04 2024-03-30T00:35:59.118 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: gps base set to 2022-02-06 (week 2196) 2024-03-30T00:35:59.119 INFO:teuthology.orchestra.run.smithi066.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-30T00:35:59.119 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-30T00:35:59.119 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: restrict ::: KOD does nothing without LIMITED. 2024-03-30T00:35:59.119 INFO:teuthology.orchestra.run.smithi066.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-30T00:35:59.120 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: Listen and drop on 0 v6wildcard [::]:123 2024-03-30T00:35:59.120 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-30T00:35:59.121 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: Listen normally on 2 lo 127.0.0.1:123 2024-03-30T00:35:59.121 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: Listen normally on 3 enp3s0f1 172.21.15.66:123 2024-03-30T00:35:59.121 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: Listen normally on 4 lo [::1]:123 2024-03-30T00:35:59.121 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8bd%5]:123 2024-03-30T00:35:59.121 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:35:59 ntpd[17202]: Listening on routing socket on fd #22 for interface updates 2024-03-30T00:35:59.151 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-30T00:35:59.151 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: Command line: ntpd -gq 2024-03-30T00:35:59.151 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: ---------------------------------------------------- 2024-03-30T00:35:59.151 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: ntp-4 is maintained by Network Time Foundation, 2024-03-30T00:35:59.151 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-30T00:35:59.151 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: corporation. Support and training for ntp-4 are 2024-03-30T00:35:59.151 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: available at https://www.nwtime.org/support 2024-03-30T00:35:59.151 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: ---------------------------------------------------- 2024-03-30T00:35:59.152 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: proto: precision = 0.041 usec (-24) 2024-03-30T00:35:59.152 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: basedate set to 2022-02-04 2024-03-30T00:35:59.152 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: gps base set to 2022-02-06 (week 2196) 2024-03-30T00:35:59.152 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-30T00:35:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-30T00:35:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-30T00:35:59.153 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: restrict ::: KOD does nothing without LIMITED. 2024-03-30T00:35:59.154 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: Listen and drop on 0 v6wildcard [::]:123 2024-03-30T00:35:59.155 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-30T00:35:59.155 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: Listen normally on 2 lo 127.0.0.1:123 2024-03-30T00:35:59.155 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: Listen normally on 3 enp3s0f1 172.21.15.156:123 2024-03-30T00:35:59.155 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: Listen normally on 4 lo [::1]:123 2024-03-30T00:35:59.155 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6dfd%5]:123 2024-03-30T00:35:59.155 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:35:59 ntpd[17183]: Listening on routing socket on fd #22 for interface updates 2024-03-30T00:36:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:00 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:00 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:00 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:00 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:00 ntpd[17202]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:00 ntpd[17202]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:00.152 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:00.152 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:00.152 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:00.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:00.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:00.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:00.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:00.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:00.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:00.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:00.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:00.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:00.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:00.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:00 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:02.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:02 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:02.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:02 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:02.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:02 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:02.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:02 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:02.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:02 ntpd[17202]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:02.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:02 ntpd[17202]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:02 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:04.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:04 ntpd[17202]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:04.152 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:04.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:04.154 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:04 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:06.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:06.118 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:06.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:06.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:06.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:06.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:06.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:06.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:06.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:06.119 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:06.119 INFO:teuthology.orchestra.run.smithi066.stdout:30 Mar 00:36:06 ntpd[17202]: ntpd: time slew +0.000315 s 2024-03-30T00:36:06.119 INFO:teuthology.orchestra.run.smithi066.stdout:ntpd: time slew +0.000315s 2024-03-30T00:36:06.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-30T00:36:06.120 INFO:teuthology.orchestra.run.smithi066.stderr:30 Mar 00:36:06 ntpd[17202]: can't open /var/log/ntpstats/loopstats.20240330: Permission denied 2024-03-30T00:36:06.152 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:06.152 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:06.152 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:06.152 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:06.152 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:06.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:06.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:06.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:06.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:06.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:06.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T00:36:06.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T00:36:06.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T00:36:06.153 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T00:36:06.153 INFO:teuthology.orchestra.run.smithi156.stdout:30 Mar 00:36:06 ntpd[17183]: ntpd: time slew -0.001783 s 2024-03-30T00:36:06.153 INFO:teuthology.orchestra.run.smithi156.stdout:ntpd: time slew -0.001783s 2024-03-30T00:36:06.154 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-30T00:36:06.154 INFO:teuthology.orchestra.run.smithi156.stderr:30 Mar 00:36:06 ntpd[17183]: can't open /var/log/ntpstats/loopstats.20240330: Permission denied 2024-03-30T00:36:06.232 INFO:teuthology.orchestra.run.smithi066.stdout: remote refid st t when poll reach delay offset jitter 2024-03-30T00:36:06.232 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================== 2024-03-30T00:36:06.233 INFO:teuthology.orchestra.run.smithi066.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T00:36:06.233 INFO:teuthology.orchestra.run.smithi066.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T00:36:06.233 INFO:teuthology.orchestra.run.smithi066.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T00:36:06.233 INFO:teuthology.orchestra.run.smithi066.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T00:36:06.266 INFO:teuthology.orchestra.run.smithi156.stdout: remote refid st t when poll reach delay offset jitter 2024-03-30T00:36:06.266 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================== 2024-03-30T00:36:06.266 INFO:teuthology.orchestra.run.smithi156.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T00:36:06.266 INFO:teuthology.orchestra.run.smithi156.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T00:36:06.266 INFO:teuthology.orchestra.run.smithi156.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T00:36:06.267 INFO:teuthology.orchestra.run.smithi156.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T00:36:06.268 INFO:teuthology.run_tasks:Running task install... 2024-03-30T00:36:06.278 DEBUG:teuthology.task.install:project ceph 2024-03-30T00:36:06.278 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}} 2024-03-30T00:36:06.278 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'} 2024-03-30T00:36:06.278 INFO:teuthology.task.install:Using flavor: default 2024-03-30T00:36:06.284 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-30T00:36:06.284 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-30T00:36:06.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo apt-key list | grep Ceph 2024-03-30T00:36:06.285 DEBUG:teuthology.orchestra.run.smithi156:> sudo apt-key list | grep Ceph 2024-03-30T00:36:06.388 INFO:teuthology.orchestra.run.smithi066.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-30T00:36:06.409 INFO:teuthology.orchestra.run.smithi156.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-30T00:36:06.440 INFO:teuthology.orchestra.run.smithi066.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-30T00:36:06.440 INFO:teuthology.orchestra.run.smithi066.stdout:uid [ unknown] Ceph.com (release key) 2024-03-30T00:36:06.441 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-30T00:36:06.441 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-30T00:36:06.441 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:36:06.459 INFO:teuthology.orchestra.run.smithi156.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-30T00:36:06.459 INFO:teuthology.orchestra.run.smithi156.stdout:uid [ unknown] Ceph.com (release key) 2024-03-30T00:36:06.460 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-30T00:36:06.460 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-30T00:36:06.460 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:36:06.660 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default/ 2024-03-30T00:36:06.661 INFO:teuthology.task.install.deb:Package version is 18.2.2-1018-g470100a4-1jammy 2024-03-30T00:36:06.673 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default/ 2024-03-30T00:36:06.674 INFO:teuthology.task.install.deb:Package version is 18.2.2-1018-g470100a4-1jammy 2024-03-30T00:36:06.970 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:36:06.970 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-30T00:36:06.990 DEBUG:teuthology.orchestra.run.smithi156:> sudo apt-get update 2024-03-30T00:36:07.014 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:36:07.014 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-30T00:36:07.035 DEBUG:teuthology.orchestra.run.smithi066:> sudo apt-get update 2024-03-30T00:36:07.182 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-30T00:36:07.193 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-30T00:36:07.215 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-30T00:36:07.246 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-30T00:36:07.250 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-30T00:36:07.252 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-30T00:36:07.288 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-30T00:36:07.322 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-30T00:36:07.464 INFO:teuthology.orchestra.run.smithi156.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy InRelease 2024-03-30T00:36:07.514 INFO:teuthology.orchestra.run.smithi066.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy InRelease 2024-03-30T00:36:07.546 INFO:teuthology.orchestra.run.smithi156.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-30T00:36:07.596 INFO:teuthology.orchestra.run.smithi066.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-30T00:36:07.628 INFO:teuthology.orchestra.run.smithi156.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-30T00:36:07.682 INFO:teuthology.orchestra.run.smithi066.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-30T00:36:07.710 INFO:teuthology.orchestra.run.smithi156.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2024-03-30T00:36:07.762 INFO:teuthology.orchestra.run.smithi066.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2024-03-30T00:36:07.803 INFO:teuthology.orchestra.run.smithi156.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2024-03-30T00:36:07.856 INFO:teuthology.orchestra.run.smithi066.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2024-03-30T00:36:07.995 INFO:teuthology.orchestra.run.smithi156.stdout:Fetched 28.5 kB in 1s (36.6 kB/s) 2024-03-30T00:36:08.054 INFO:teuthology.orchestra.run.smithi066.stdout:Fetched 28.5 kB in 1s (36.3 kB/s) 2024-03-30T00:36:08.958 INFO:teuthology.orchestra.run.smithi156.stdout:Reading package lists... 2024-03-30T00:36:08.973 DEBUG:teuthology.orchestra.run.smithi156:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1018-g470100a4-1jammy cephadm=18.2.2-1018-g470100a4-1jammy ceph-mds=18.2.2-1018-g470100a4-1jammy ceph-mgr=18.2.2-1018-g470100a4-1jammy ceph-common=18.2.2-1018-g470100a4-1jammy ceph-fuse=18.2.2-1018-g470100a4-1jammy ceph-test=18.2.2-1018-g470100a4-1jammy ceph-volume=18.2.2-1018-g470100a4-1jammy radosgw=18.2.2-1018-g470100a4-1jammy python3-rados=18.2.2-1018-g470100a4-1jammy python3-rgw=18.2.2-1018-g470100a4-1jammy python3-cephfs=18.2.2-1018-g470100a4-1jammy python3-rbd=18.2.2-1018-g470100a4-1jammy libcephfs2=18.2.2-1018-g470100a4-1jammy libcephfs-dev=18.2.2-1018-g470100a4-1jammy librados2=18.2.2-1018-g470100a4-1jammy librbd1=18.2.2-1018-g470100a4-1jammy rbd-fuse=18.2.2-1018-g470100a4-1jammy python3-cephfs=18.2.2-1018-g470100a4-1jammy cephfs-shell=18.2.2-1018-g470100a4-1jammy cephfs-top=18.2.2-1018-g470100a4-1jammy cephfs-mirror=18.2.2-1018-g470100a4-1jammy 2024-03-30T00:36:09.048 INFO:teuthology.orchestra.run.smithi156.stdout:Reading package lists... 2024-03-30T00:36:09.051 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-03-30T00:36:09.065 DEBUG:teuthology.orchestra.run.smithi066:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1018-g470100a4-1jammy cephadm=18.2.2-1018-g470100a4-1jammy ceph-mds=18.2.2-1018-g470100a4-1jammy ceph-mgr=18.2.2-1018-g470100a4-1jammy ceph-common=18.2.2-1018-g470100a4-1jammy ceph-fuse=18.2.2-1018-g470100a4-1jammy ceph-test=18.2.2-1018-g470100a4-1jammy ceph-volume=18.2.2-1018-g470100a4-1jammy radosgw=18.2.2-1018-g470100a4-1jammy python3-rados=18.2.2-1018-g470100a4-1jammy python3-rgw=18.2.2-1018-g470100a4-1jammy python3-cephfs=18.2.2-1018-g470100a4-1jammy python3-rbd=18.2.2-1018-g470100a4-1jammy libcephfs2=18.2.2-1018-g470100a4-1jammy libcephfs-dev=18.2.2-1018-g470100a4-1jammy librados2=18.2.2-1018-g470100a4-1jammy librbd1=18.2.2-1018-g470100a4-1jammy rbd-fuse=18.2.2-1018-g470100a4-1jammy python3-cephfs=18.2.2-1018-g470100a4-1jammy cephfs-shell=18.2.2-1018-g470100a4-1jammy cephfs-top=18.2.2-1018-g470100a4-1jammy cephfs-mirror=18.2.2-1018-g470100a4-1jammy 2024-03-30T00:36:09.141 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-03-30T00:36:09.228 INFO:teuthology.orchestra.run.smithi156.stdout:Building dependency tree... 2024-03-30T00:36:09.228 INFO:teuthology.orchestra.run.smithi156.stdout:Reading state information... 2024-03-30T00:36:09.317 INFO:teuthology.orchestra.run.smithi066.stdout:Building dependency tree... 2024-03-30T00:36:09.317 INFO:teuthology.orchestra.run.smithi066.stdout:Reading state information... 2024-03-30T00:36:09.408 INFO:teuthology.orchestra.run.smithi156.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T00:36:09.408 INFO:teuthology.orchestra.run.smithi156.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T00:36:09.408 INFO:teuthology.orchestra.run.smithi156.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T00:36:09.408 INFO:teuthology.orchestra.run.smithi156.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T00:36:09.409 INFO:teuthology.orchestra.run.smithi156.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-30T00:36:09.409 INFO:teuthology.orchestra.run.smithi156.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-30T00:36:09.409 INFO:teuthology.orchestra.run.smithi156.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T00:36:09.410 INFO:teuthology.orchestra.run.smithi156.stdout:The following additional packages will be installed: 2024-03-30T00:36:09.410 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-30T00:36:09.410 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-30T00:36:09.411 INFO:teuthology.orchestra.run.smithi156.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-03-30T00:36:09.411 INFO:teuthology.orchestra.run.smithi156.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-30T00:36:09.411 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-30T00:36:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-30T00:36:09.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-30T00:36:09.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-30T00:36:09.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-30T00:36:09.413 INFO:teuthology.orchestra.run.smithi156.stdout: qttranslations5-l10n socat xmlstarlet 2024-03-30T00:36:09.413 INFO:teuthology.orchestra.run.smithi156.stdout:Suggested packages: 2024-03-30T00:36:09.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-03-30T00:36:09.413 INFO:teuthology.orchestra.run.smithi156.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-03-30T00:36:09.413 INFO:teuthology.orchestra.run.smithi156.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-03-30T00:36:09.413 INFO:teuthology.orchestra.run.smithi156.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-03-30T00:36:09.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-03-30T00:36:09.414 INFO:teuthology.orchestra.run.smithi156.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-03-30T00:36:09.414 INFO:teuthology.orchestra.run.smithi156.stdout: python3-watchdog 2024-03-30T00:36:09.414 INFO:teuthology.orchestra.run.smithi156.stdout:Recommended packages: 2024-03-30T00:36:09.414 INFO:teuthology.orchestra.run.smithi156.stdout: btrfs-tools 2024-03-30T00:36:09.496 INFO:teuthology.orchestra.run.smithi066.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T00:36:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T00:36:09.496 INFO:teuthology.orchestra.run.smithi066.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T00:36:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T00:36:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-30T00:36:09.497 INFO:teuthology.orchestra.run.smithi066.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-30T00:36:09.497 INFO:teuthology.orchestra.run.smithi066.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T00:36:09.498 INFO:teuthology.orchestra.run.smithi066.stdout:The following additional packages will be installed: 2024-03-30T00:36:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-30T00:36:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-30T00:36:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-03-30T00:36:09.498 INFO:teuthology.orchestra.run.smithi066.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-30T00:36:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-03-30T00:36:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-30T00:36:09.499 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-30T00:36:09.500 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-30T00:36:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-30T00:36:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: qttranslations5-l10n socat xmlstarlet 2024-03-30T00:36:09.501 INFO:teuthology.orchestra.run.smithi066.stdout:Suggested packages: 2024-03-30T00:36:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-03-30T00:36:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-03-30T00:36:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-03-30T00:36:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-03-30T00:36:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-03-30T00:36:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-03-30T00:36:09.501 INFO:teuthology.orchestra.run.smithi066.stdout: python3-watchdog 2024-03-30T00:36:09.502 INFO:teuthology.orchestra.run.smithi066.stdout:Recommended packages: 2024-03-30T00:36:09.502 INFO:teuthology.orchestra.run.smithi066.stdout: btrfs-tools 2024-03-30T00:36:09.560 INFO:teuthology.orchestra.run.smithi156.stdout:The following NEW packages will be installed: 2024-03-30T00:36:09.560 INFO:teuthology.orchestra.run.smithi156.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-30T00:36:09.560 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-30T00:36:09.560 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-30T00:36:09.560 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-30T00:36:09.560 INFO:teuthology.orchestra.run.smithi156.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-03-30T00:36:09.560 INFO:teuthology.orchestra.run.smithi156.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-30T00:36:09.560 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-03-30T00:36:09.561 INFO:teuthology.orchestra.run.smithi156.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-30T00:36:09.562 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-30T00:36:09.563 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-30T00:36:09.563 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-30T00:36:09.563 INFO:teuthology.orchestra.run.smithi156.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-30T00:36:09.563 INFO:teuthology.orchestra.run.smithi156.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-30T00:36:09.563 INFO:teuthology.orchestra.run.smithi156.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-30T00:36:09.563 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-30T00:36:09.563 INFO:teuthology.orchestra.run.smithi156.stdout: socat xmlstarlet 2024-03-30T00:36:09.653 INFO:teuthology.orchestra.run.smithi066.stdout:The following NEW packages will be installed: 2024-03-30T00:36:09.653 INFO:teuthology.orchestra.run.smithi066.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-30T00:36:09.653 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-30T00:36:09.653 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-30T00:36:09.653 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-30T00:36:09.653 INFO:teuthology.orchestra.run.smithi066.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-03-30T00:36:09.654 INFO:teuthology.orchestra.run.smithi066.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-30T00:36:09.654 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-03-30T00:36:09.655 INFO:teuthology.orchestra.run.smithi066.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-03-30T00:36:09.655 INFO:teuthology.orchestra.run.smithi066.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-30T00:36:09.655 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-30T00:36:09.655 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-30T00:36:09.655 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-30T00:36:09.655 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-30T00:36:09.655 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-30T00:36:09.655 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-30T00:36:09.655 INFO:teuthology.orchestra.run.smithi066.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-30T00:36:09.655 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-30T00:36:09.655 INFO:teuthology.orchestra.run.smithi066.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-30T00:36:09.656 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-30T00:36:09.656 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-30T00:36:09.656 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-30T00:36:09.656 INFO:teuthology.orchestra.run.smithi066.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-30T00:36:09.656 INFO:teuthology.orchestra.run.smithi066.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-30T00:36:09.656 INFO:teuthology.orchestra.run.smithi066.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-30T00:36:09.656 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-30T00:36:09.656 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-30T00:36:09.656 INFO:teuthology.orchestra.run.smithi066.stdout: socat xmlstarlet 2024-03-30T00:36:09.668 INFO:teuthology.orchestra.run.smithi156.stdout:0 upgraded, 115 newly installed, 0 to remove and 301 not upgraded. 2024-03-30T00:36:09.668 INFO:teuthology.orchestra.run.smithi156.stdout:Need to get 162 MB of archives. 2024-03-30T00:36:09.668 INFO:teuthology.orchestra.run.smithi156.stdout:After this operation, 664 MB of additional disk space will be used. 2024-03-30T00:36:09.668 INFO:teuthology.orchestra.run.smithi156.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-30T00:36:09.705 INFO:teuthology.orchestra.run.smithi156.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-30T00:36:09.726 INFO:teuthology.orchestra.run.smithi156.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-30T00:36:09.805 INFO:teuthology.orchestra.run.smithi156.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-03-30T00:36:09.821 INFO:teuthology.orchestra.run.smithi156.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-30T00:36:09.850 INFO:teuthology.orchestra.run.smithi156.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-30T00:36:09.854 INFO:teuthology.orchestra.run.smithi156.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-30T00:36:09.872 INFO:teuthology.orchestra.run.smithi156.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-30T00:36:09.882 INFO:teuthology.orchestra.run.smithi066.stdout:0 upgraded, 115 newly installed, 0 to remove and 301 not upgraded. 2024-03-30T00:36:09.882 INFO:teuthology.orchestra.run.smithi066.stdout:Need to get 162 MB of archives. 2024-03-30T00:36:09.883 INFO:teuthology.orchestra.run.smithi066.stdout:After this operation, 664 MB of additional disk space will be used. 2024-03-30T00:36:09.883 INFO:teuthology.orchestra.run.smithi066.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-30T00:36:09.936 INFO:teuthology.orchestra.run.smithi156.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-30T00:36:09.939 INFO:teuthology.orchestra.run.smithi156.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-30T00:36:09.943 INFO:teuthology.orchestra.run.smithi156.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.2-1018-g470100a4-1jammy [3,405 kB] 2024-03-30T00:36:09.949 INFO:teuthology.orchestra.run.smithi156.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-30T00:36:09.952 INFO:teuthology.orchestra.run.smithi156.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-30T00:36:09.953 INFO:teuthology.orchestra.run.smithi156.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-30T00:36:09.960 INFO:teuthology.orchestra.run.smithi156.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-30T00:36:09.960 INFO:teuthology.orchestra.run.smithi156.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-30T00:36:09.961 INFO:teuthology.orchestra.run.smithi156.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-30T00:36:09.962 INFO:teuthology.orchestra.run.smithi156.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-30T00:36:09.964 INFO:teuthology.orchestra.run.smithi156.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-30T00:36:09.981 INFO:teuthology.orchestra.run.smithi066.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-30T00:36:09.991 INFO:teuthology.orchestra.run.smithi156.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-30T00:36:09.991 INFO:teuthology.orchestra.run.smithi156.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-30T00:36:09.992 INFO:teuthology.orchestra.run.smithi156.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-30T00:36:09.992 INFO:teuthology.orchestra.run.smithi156.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-30T00:36:09.993 INFO:teuthology.orchestra.run.smithi156.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-30T00:36:09.995 INFO:teuthology.orchestra.run.smithi156.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-30T00:36:09.996 INFO:teuthology.orchestra.run.smithi156.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-30T00:36:09.996 INFO:teuthology.orchestra.run.smithi156.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-30T00:36:09.996 INFO:teuthology.orchestra.run.smithi156.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-30T00:36:10.022 INFO:teuthology.orchestra.run.smithi156.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-30T00:36:10.022 INFO:teuthology.orchestra.run.smithi156.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-30T00:36:10.022 INFO:teuthology.orchestra.run.smithi156.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-30T00:36:10.036 INFO:teuthology.orchestra.run.smithi066.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-30T00:36:10.039 INFO:teuthology.orchestra.run.smithi066.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.2-1018-g470100a4-1jammy [3,405 kB] 2024-03-30T00:36:10.050 INFO:teuthology.orchestra.run.smithi156.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-30T00:36:10.051 INFO:teuthology.orchestra.run.smithi156.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-30T00:36:10.051 INFO:teuthology.orchestra.run.smithi156.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-30T00:36:10.056 INFO:teuthology.orchestra.run.smithi156.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-30T00:36:10.057 INFO:teuthology.orchestra.run.smithi156.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-30T00:36:10.057 INFO:teuthology.orchestra.run.smithi156.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-30T00:36:10.058 INFO:teuthology.orchestra.run.smithi156.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-30T00:36:10.079 INFO:teuthology.orchestra.run.smithi156.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-30T00:36:10.081 INFO:teuthology.orchestra.run.smithi156.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-30T00:36:10.082 INFO:teuthology.orchestra.run.smithi156.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-30T00:36:10.110 INFO:teuthology.orchestra.run.smithi156.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-30T00:36:10.111 INFO:teuthology.orchestra.run.smithi156.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-30T00:36:10.113 INFO:teuthology.orchestra.run.smithi156.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-30T00:36:10.114 INFO:teuthology.orchestra.run.smithi156.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-30T00:36:10.117 INFO:teuthology.orchestra.run.smithi156.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-30T00:36:10.172 INFO:teuthology.orchestra.run.smithi156.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-30T00:36:10.174 INFO:teuthology.orchestra.run.smithi156.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-30T00:36:10.176 INFO:teuthology.orchestra.run.smithi156.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-30T00:36:10.176 INFO:teuthology.orchestra.run.smithi156.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-30T00:36:10.177 INFO:teuthology.orchestra.run.smithi156.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-30T00:36:10.185 INFO:teuthology.orchestra.run.smithi156.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-30T00:36:10.186 INFO:teuthology.orchestra.run.smithi156.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-30T00:36:10.186 INFO:teuthology.orchestra.run.smithi156.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-30T00:36:10.189 INFO:teuthology.orchestra.run.smithi156.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-30T00:36:10.220 INFO:teuthology.orchestra.run.smithi156.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-30T00:36:10.222 INFO:teuthology.orchestra.run.smithi156.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-30T00:36:10.243 INFO:teuthology.orchestra.run.smithi156.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-30T00:36:10.246 INFO:teuthology.orchestra.run.smithi156.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-30T00:36:10.246 INFO:teuthology.orchestra.run.smithi156.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-30T00:36:10.263 INFO:teuthology.orchestra.run.smithi066.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-03-30T00:36:10.269 INFO:teuthology.orchestra.run.smithi156.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-30T00:36:10.269 INFO:teuthology.orchestra.run.smithi156.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-30T00:36:10.270 INFO:teuthology.orchestra.run.smithi156.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-30T00:36:10.271 INFO:teuthology.orchestra.run.smithi156.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-30T00:36:10.271 INFO:teuthology.orchestra.run.smithi156.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-30T00:36:10.296 INFO:teuthology.orchestra.run.smithi156.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-30T00:36:10.297 INFO:teuthology.orchestra.run.smithi156.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-30T00:36:10.297 INFO:teuthology.orchestra.run.smithi156.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-30T00:36:10.301 INFO:teuthology.orchestra.run.smithi156.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-30T00:36:10.303 INFO:teuthology.orchestra.run.smithi066.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-30T00:36:10.311 INFO:teuthology.orchestra.run.smithi156.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.2-1018-g470100a4-1jammy [3,105 kB] 2024-03-30T00:36:10.333 INFO:teuthology.orchestra.run.smithi156.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-30T00:36:10.335 INFO:teuthology.orchestra.run.smithi156.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-30T00:36:10.336 INFO:teuthology.orchestra.run.smithi156.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-30T00:36:10.340 INFO:teuthology.orchestra.run.smithi156.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-30T00:36:10.343 INFO:teuthology.orchestra.run.smithi156.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-30T00:36:10.343 INFO:teuthology.orchestra.run.smithi156.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-30T00:36:10.370 INFO:teuthology.orchestra.run.smithi156.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-30T00:36:10.374 INFO:teuthology.orchestra.run.smithi156.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-30T00:36:10.375 INFO:teuthology.orchestra.run.smithi156.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-30T00:36:10.376 INFO:teuthology.orchestra.run.smithi156.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-30T00:36:10.387 INFO:teuthology.orchestra.run.smithi066.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-30T00:36:10.400 INFO:teuthology.orchestra.run.smithi066.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-30T00:36:10.408 INFO:teuthology.orchestra.run.smithi156.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-30T00:36:10.408 INFO:teuthology.orchestra.run.smithi156.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-30T00:36:10.409 INFO:teuthology.orchestra.run.smithi156.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-30T00:36:10.434 INFO:teuthology.orchestra.run.smithi156.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.2-1018-g470100a4-1jammy [674 kB] 2024-03-30T00:36:10.449 INFO:teuthology.orchestra.run.smithi066.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-30T00:36:10.539 INFO:teuthology.orchestra.run.smithi156.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.2-1018-g470100a4-1jammy [356 kB] 2024-03-30T00:36:10.542 INFO:teuthology.orchestra.run.smithi156.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.2-1018-g470100a4-1jammy [32.8 kB] 2024-03-30T00:36:10.543 INFO:teuthology.orchestra.run.smithi156.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.2-1018-g470100a4-1jammy [180 kB] 2024-03-30T00:36:10.545 INFO:teuthology.orchestra.run.smithi156.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.2-1018-g470100a4-1jammy [63.7 kB] 2024-03-30T00:36:10.554 INFO:teuthology.orchestra.run.smithi156.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.2-1018-g470100a4-1jammy [330 kB] 2024-03-30T00:36:10.557 INFO:teuthology.orchestra.run.smithi156.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.2-1018-g470100a4-1jammy [5,892 kB] 2024-03-30T00:36:10.612 INFO:teuthology.orchestra.run.smithi066.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.2-1018-g470100a4-1jammy [3,105 kB] 2024-03-30T00:36:10.647 INFO:teuthology.orchestra.run.smithi066.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-30T00:36:10.657 INFO:teuthology.orchestra.run.smithi066.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-30T00:36:10.689 INFO:teuthology.orchestra.run.smithi066.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-30T00:36:10.749 INFO:teuthology.orchestra.run.smithi156.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.2-1018-g470100a4-1jammy [112 kB] 2024-03-30T00:36:10.990 INFO:teuthology.orchestra.run.smithi066.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-30T00:36:10.991 INFO:teuthology.orchestra.run.smithi156.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.2-1018-g470100a4-1jammy [441 kB] 2024-03-30T00:36:10.991 INFO:teuthology.orchestra.run.smithi156.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.2-1018-g470100a4-1jammy [22.0 MB] 2024-03-30T00:36:11.034 INFO:teuthology.orchestra.run.smithi066.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.2-1018-g470100a4-1jammy [674 kB] 2024-03-30T00:36:11.100 INFO:teuthology.orchestra.run.smithi066.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.2-1018-g470100a4-1jammy [356 kB] 2024-03-30T00:36:11.140 INFO:teuthology.orchestra.run.smithi066.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-30T00:36:11.160 INFO:teuthology.orchestra.run.smithi066.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.2-1018-g470100a4-1jammy [32.8 kB] 2024-03-30T00:36:11.160 INFO:teuthology.orchestra.run.smithi066.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.2-1018-g470100a4-1jammy [180 kB] 2024-03-30T00:36:11.166 INFO:teuthology.orchestra.run.smithi066.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.2-1018-g470100a4-1jammy [63.7 kB] 2024-03-30T00:36:11.167 INFO:teuthology.orchestra.run.smithi066.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.2-1018-g470100a4-1jammy [330 kB] 2024-03-30T00:36:11.225 INFO:teuthology.orchestra.run.smithi066.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.2-1018-g470100a4-1jammy [5,892 kB] 2024-03-30T00:36:11.592 INFO:teuthology.orchestra.run.smithi066.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-30T00:36:11.601 INFO:teuthology.orchestra.run.smithi066.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-30T00:36:11.602 INFO:teuthology.orchestra.run.smithi066.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-30T00:36:11.616 INFO:teuthology.orchestra.run.smithi066.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-30T00:36:11.617 INFO:teuthology.orchestra.run.smithi066.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-30T00:36:11.619 INFO:teuthology.orchestra.run.smithi066.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-30T00:36:11.621 INFO:teuthology.orchestra.run.smithi066.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-30T00:36:11.623 INFO:teuthology.orchestra.run.smithi066.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-30T00:36:11.624 INFO:teuthology.orchestra.run.smithi066.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-30T00:36:11.723 INFO:teuthology.orchestra.run.smithi156.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.2-1018-g470100a4-1jammy [5,001 kB] 2024-03-30T00:36:11.817 INFO:teuthology.orchestra.run.smithi066.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-30T00:36:11.932 INFO:teuthology.orchestra.run.smithi156.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.2-1018-g470100a4-1jammy [241 kB] 2024-03-30T00:36:11.955 INFO:teuthology.orchestra.run.smithi156.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1018-g470100a4-1jammy [130 kB] 2024-03-30T00:36:11.958 INFO:teuthology.orchestra.run.smithi156.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.2-1018-g470100a4-1jammy [1,486 kB] 2024-03-30T00:36:12.010 INFO:teuthology.orchestra.run.smithi066.stdout:Get:33 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.2-1018-g470100a4-1jammy [112 kB] 2024-03-30T00:36:12.012 INFO:teuthology.orchestra.run.smithi066.stdout:Get:34 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.2-1018-g470100a4-1jammy [441 kB] 2024-03-30T00:36:12.025 INFO:teuthology.orchestra.run.smithi156.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.2-1018-g470100a4-1jammy [6,079 kB] 2024-03-30T00:36:12.077 INFO:teuthology.orchestra.run.smithi066.stdout:Get:35 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.2-1018-g470100a4-1jammy [22.0 MB] 2024-03-30T00:36:12.294 INFO:teuthology.orchestra.run.smithi156.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.2-1018-g470100a4-1jammy [22.3 MB] 2024-03-30T00:36:12.329 INFO:teuthology.orchestra.run.smithi066.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-30T00:36:12.341 INFO:teuthology.orchestra.run.smithi066.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-30T00:36:12.347 INFO:teuthology.orchestra.run.smithi066.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-30T00:36:12.352 INFO:teuthology.orchestra.run.smithi066.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-30T00:36:12.373 INFO:teuthology.orchestra.run.smithi066.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-30T00:36:12.377 INFO:teuthology.orchestra.run.smithi066.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-30T00:36:12.380 INFO:teuthology.orchestra.run.smithi066.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-30T00:36:12.426 INFO:teuthology.orchestra.run.smithi066.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-30T00:36:12.443 INFO:teuthology.orchestra.run.smithi066.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-30T00:36:12.448 INFO:teuthology.orchestra.run.smithi066.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-30T00:36:12.730 INFO:teuthology.orchestra.run.smithi066.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-30T00:36:12.823 INFO:teuthology.orchestra.run.smithi066.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-30T00:36:12.937 INFO:teuthology.orchestra.run.smithi066.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-30T00:36:13.015 INFO:teuthology.orchestra.run.smithi066.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-30T00:36:13.135 INFO:teuthology.orchestra.run.smithi066.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-30T00:36:13.231 INFO:teuthology.orchestra.run.smithi066.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-30T00:36:13.253 INFO:teuthology.orchestra.run.smithi066.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-30T00:36:13.265 INFO:teuthology.orchestra.run.smithi156.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.2-1018-g470100a4-1jammy [14.1 kB] 2024-03-30T00:36:13.266 INFO:teuthology.orchestra.run.smithi156.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.2-1018-g470100a4-1jammy [826 kB] 2024-03-30T00:36:13.288 INFO:teuthology.orchestra.run.smithi066.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-30T00:36:13.302 INFO:teuthology.orchestra.run.smithi066.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-30T00:36:13.304 INFO:teuthology.orchestra.run.smithi156.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.2-1018-g470100a4-1jammy [2,176 kB] 2024-03-30T00:36:13.345 INFO:teuthology.orchestra.run.smithi066.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-30T00:36:13.354 INFO:teuthology.orchestra.run.smithi066.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-30T00:36:13.395 INFO:teuthology.orchestra.run.smithi156.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.2-1018-g470100a4-1jammy [238 kB] 2024-03-30T00:36:13.402 INFO:teuthology.orchestra.run.smithi156.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.2-1018-g470100a4-1jammy [144 kB] 2024-03-30T00:36:13.407 INFO:teuthology.orchestra.run.smithi156.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.2-1018-g470100a4-1jammy [1,931 kB] 2024-03-30T00:36:13.490 INFO:teuthology.orchestra.run.smithi156.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1018-g470100a4-1jammy [8,625 kB] 2024-03-30T00:36:13.567 INFO:teuthology.orchestra.run.smithi066.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-30T00:36:13.869 INFO:teuthology.orchestra.run.smithi156.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.2-1018-g470100a4-1jammy [14.3 kB] 2024-03-30T00:36:13.870 INFO:teuthology.orchestra.run.smithi156.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.2-1018-g470100a4-1jammy [42.7 MB] 2024-03-30T00:36:14.535 INFO:teuthology.orchestra.run.smithi066.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-30T00:36:14.536 INFO:teuthology.orchestra.run.smithi066.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-30T00:36:14.537 INFO:teuthology.orchestra.run.smithi066.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-30T00:36:14.538 INFO:teuthology.orchestra.run.smithi066.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-30T00:36:14.538 INFO:teuthology.orchestra.run.smithi066.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-30T00:36:14.540 INFO:teuthology.orchestra.run.smithi066.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-30T00:36:14.540 INFO:teuthology.orchestra.run.smithi066.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-30T00:36:14.540 INFO:teuthology.orchestra.run.smithi066.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-30T00:36:14.543 INFO:teuthology.orchestra.run.smithi066.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-30T00:36:14.629 INFO:teuthology.orchestra.run.smithi066.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-30T00:36:14.815 INFO:teuthology.orchestra.run.smithi066.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-30T00:36:14.997 INFO:teuthology.orchestra.run.smithi066.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.2-1018-g470100a4-1jammy [5,001 kB] 2024-03-30T00:36:15.660 INFO:teuthology.orchestra.run.smithi066.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-30T00:36:15.666 INFO:teuthology.orchestra.run.smithi066.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-30T00:36:15.666 INFO:teuthology.orchestra.run.smithi066.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-30T00:36:15.701 INFO:teuthology.orchestra.run.smithi156.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.2-1018-g470100a4-1jammy [121 kB] 2024-03-30T00:36:15.702 INFO:teuthology.orchestra.run.smithi156.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.2-1018-g470100a4-1jammy [230 kB] 2024-03-30T00:36:15.704 INFO:teuthology.orchestra.run.smithi156.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.2-1018-g470100a4-1jammy [34.0 kB] 2024-03-30T00:36:15.704 INFO:teuthology.orchestra.run.smithi156.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.2-1018-g470100a4-1jammy [23.5 kB] 2024-03-30T00:36:15.705 INFO:teuthology.orchestra.run.smithi156.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.2-1018-g470100a4-1jammy [38.3 kB] 2024-03-30T00:36:15.706 INFO:teuthology.orchestra.run.smithi156.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.2-1018-g470100a4-1jammy [11.6 MB] 2024-03-30T00:36:15.727 INFO:teuthology.orchestra.run.smithi066.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-30T00:36:15.728 INFO:teuthology.orchestra.run.smithi066.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-30T00:36:15.755 INFO:teuthology.orchestra.run.smithi066.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-30T00:36:15.756 INFO:teuthology.orchestra.run.smithi066.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-30T00:36:15.757 INFO:teuthology.orchestra.run.smithi066.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-30T00:36:15.759 INFO:teuthology.orchestra.run.smithi066.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-30T00:36:15.759 INFO:teuthology.orchestra.run.smithi066.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-30T00:36:15.806 INFO:teuthology.orchestra.run.smithi066.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.2-1018-g470100a4-1jammy [241 kB] 2024-03-30T00:36:15.873 INFO:teuthology.orchestra.run.smithi066.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1018-g470100a4-1jammy [130 kB] 2024-03-30T00:36:15.877 INFO:teuthology.orchestra.run.smithi066.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.2-1018-g470100a4-1jammy [1,486 kB] 2024-03-30T00:36:15.930 INFO:teuthology.orchestra.run.smithi066.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-30T00:36:16.110 INFO:teuthology.orchestra.run.smithi066.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.2-1018-g470100a4-1jammy [6,079 kB] 2024-03-30T00:36:16.177 INFO:teuthology.orchestra.run.smithi156.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.2-1018-g470100a4-1jammy [93.1 kB] 2024-03-30T00:36:16.433 INFO:teuthology.orchestra.run.smithi066.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.2-1018-g470100a4-1jammy [22.3 MB] 2024-03-30T00:36:16.488 INFO:teuthology.orchestra.run.smithi066.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-30T00:36:16.520 INFO:teuthology.orchestra.run.smithi066.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-30T00:36:16.545 INFO:teuthology.orchestra.run.smithi066.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-30T00:36:16.555 INFO:teuthology.orchestra.run.smithi066.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-30T00:36:16.649 INFO:teuthology.orchestra.run.smithi066.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-30T00:36:16.688 INFO:teuthology.orchestra.run.smithi066.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-30T00:36:16.691 INFO:teuthology.orchestra.run.smithi066.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-30T00:36:16.734 INFO:teuthology.orchestra.run.smithi066.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-30T00:36:16.738 INFO:teuthology.orchestra.run.smithi156.stdout:Fetched 162 MB in 7s (24.6 MB/s) 2024-03-30T00:36:16.763 INFO:teuthology.orchestra.run.smithi066.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-30T00:36:16.767 INFO:teuthology.orchestra.run.smithi066.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-30T00:36:16.844 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-30T00:36:16.909 INFO:teuthology.orchestra.run.smithi156.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132923 files and directories currently installed.) 2024-03-30T00:36:16.912 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-30T00:36:16.951 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-30T00:36:16.951 INFO:teuthology.orchestra.run.smithi066.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-30T00:36:17.195 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-30T00:36:17.209 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-30T00:36:17.245 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-30T00:36:17.400 INFO:teuthology.orchestra.run.smithi066.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.2-1018-g470100a4-1jammy [14.1 kB] 2024-03-30T00:36:17.401 INFO:teuthology.orchestra.run.smithi066.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.2-1018-g470100a4-1jammy [826 kB] 2024-03-30T00:36:17.429 INFO:teuthology.orchestra.run.smithi066.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.2-1018-g470100a4-1jammy [2,176 kB] 2024-03-30T00:36:17.526 INFO:teuthology.orchestra.run.smithi066.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.2-1018-g470100a4-1jammy [238 kB] 2024-03-30T00:36:17.532 INFO:teuthology.orchestra.run.smithi066.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.2-1018-g470100a4-1jammy [144 kB] 2024-03-30T00:36:17.538 INFO:teuthology.orchestra.run.smithi066.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.2-1018-g470100a4-1jammy [1,931 kB] 2024-03-30T00:36:17.581 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-cmd2. 2024-03-30T00:36:17.595 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-30T00:36:17.621 INFO:teuthology.orchestra.run.smithi066.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1018-g470100a4-1jammy [8,625 kB] 2024-03-30T00:36:17.639 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-30T00:36:17.643 INFO:teuthology.orchestra.run.smithi066.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-30T00:36:17.645 INFO:teuthology.orchestra.run.smithi066.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-30T00:36:17.650 INFO:teuthology.orchestra.run.smithi066.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-30T00:36:17.951 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libfmt8:amd64. 2024-03-30T00:36:17.965 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-03-30T00:36:17.981 INFO:teuthology.orchestra.run.smithi066.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.2-1018-g470100a4-1jammy [14.3 kB] 2024-03-30T00:36:17.982 INFO:teuthology.orchestra.run.smithi066.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.2-1018-g470100a4-1jammy [42.7 MB] 2024-03-30T00:36:18.051 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-03-30T00:36:18.353 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-30T00:36:18.368 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-30T00:36:18.412 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-30T00:36:18.782 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-30T00:36:18.797 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-30T00:36:18.840 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-30T00:36:19.177 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-30T00:36:19.192 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-30T00:36:19.235 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-30T00:36:19.605 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-30T00:36:19.620 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T00:36:19.662 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:36:19.701 INFO:teuthology.orchestra.run.smithi066.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.2-1018-g470100a4-1jammy [121 kB] 2024-03-30T00:36:19.702 INFO:teuthology.orchestra.run.smithi066.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.2-1018-g470100a4-1jammy [230 kB] 2024-03-30T00:36:19.705 INFO:teuthology.orchestra.run.smithi066.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.2-1018-g470100a4-1jammy [34.0 kB] 2024-03-30T00:36:19.705 INFO:teuthology.orchestra.run.smithi066.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.2-1018-g470100a4-1jammy [23.5 kB] 2024-03-30T00:36:19.705 INFO:teuthology.orchestra.run.smithi066.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.2-1018-g470100a4-1jammy [38.3 kB] 2024-03-30T00:36:19.706 INFO:teuthology.orchestra.run.smithi066.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.2-1018-g470100a4-1jammy [11.6 MB] 2024-03-30T00:36:20.083 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-30T00:36:20.097 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T00:36:20.141 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:36:20.146 INFO:teuthology.orchestra.run.smithi066.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.2-1018-g470100a4-1jammy [93.1 kB] 2024-03-30T00:36:20.461 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-30T00:36:20.476 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T00:36:20.519 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:36:20.735 INFO:teuthology.orchestra.run.smithi066.stdout:Fetched 162 MB in 10s (15.5 MB/s) 2024-03-30T00:36:20.814 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-30T00:36:20.829 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-30T00:36:20.864 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-30T00:36:20.866 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-30T00:36:20.935 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132923 files and directories currently installed.) 2024-03-30T00:36:20.939 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-30T00:36:20.981 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-30T00:36:21.284 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-30T00:36:21.298 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-30T00:36:21.342 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-30T00:36:21.475 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package librados2. 2024-03-30T00:36:21.490 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../011-librados2_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:21.533 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking librados2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:21.670 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cmd2. 2024-03-30T00:36:21.684 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-30T00:36:21.729 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-30T00:36:22.044 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package librbd1. 2024-03-30T00:36:22.060 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../012-librbd1_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:22.103 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking librbd1 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:22.323 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libfmt8:amd64. 2024-03-30T00:36:22.338 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-03-30T00:36:22.423 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-03-30T00:36:22.489 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libcephfs2. 2024-03-30T00:36:22.503 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../013-libcephfs2_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:22.538 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libcephfs2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:22.701 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-30T00:36:22.715 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-30T00:36:22.751 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-30T00:36:22.791 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-rados. 2024-03-30T00:36:22.806 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../014-python3-rados_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:22.849 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-rados (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:23.105 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-30T00:36:23.119 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-30T00:36:23.163 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-30T00:36:23.185 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-30T00:36:23.200 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:23.244 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-ceph-argparse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:23.496 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-cephfs. 2024-03-30T00:36:23.499 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-30T00:36:23.511 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../016-python3-cephfs_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:23.514 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-30T00:36:23.554 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-cephfs (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:23.557 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-30T00:36:23.840 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-30T00:36:23.855 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:23.898 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-ceph-common (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:23.977 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-30T00:36:23.993 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T00:36:24.045 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:36:24.270 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-prettytable. 2024-03-30T00:36:24.285 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-03-30T00:36:24.328 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-30T00:36:24.464 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-30T00:36:24.479 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T00:36:24.522 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:36:24.581 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-rbd. 2024-03-30T00:36:24.596 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../019-python3-rbd_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:24.639 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-rbd (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:24.867 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-30T00:36:24.882 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T00:36:24.925 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:36:24.992 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-30T00:36:25.007 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-30T00:36:25.050 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-30T00:36:25.287 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-30T00:36:25.302 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-30T00:36:25.337 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package librgw2. 2024-03-30T00:36:25.402 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../021-librgw2_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:25.437 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking librgw2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:25.589 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librados2. 2024-03-30T00:36:25.604 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../011-librados2_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:25.647 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librados2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:25.889 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-rgw. 2024-03-30T00:36:25.904 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../022-python3-rgw_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:25.948 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-rgw (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:26.083 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librbd1. 2024-03-30T00:36:26.098 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../012-librbd1_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:26.141 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librbd1 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:26.301 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-30T00:36:26.316 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-30T00:36:26.359 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-30T00:36:26.586 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libcephfs2. 2024-03-30T00:36:26.600 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../013-libcephfs2_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:26.645 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libcephfs2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:26.703 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libradosstriper1. 2024-03-30T00:36:26.718 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../024-libradosstriper1_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:26.762 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libradosstriper1 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:26.897 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rados. 2024-03-30T00:36:26.911 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../014-python3-rados_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:26.947 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rados (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:27.165 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-common. 2024-03-30T00:36:27.180 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../025-ceph-common_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:27.223 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-common (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:27.375 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-30T00:36:27.390 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:27.433 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-ceph-argparse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:27.710 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cephfs. 2024-03-30T00:36:27.725 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../016-python3-cephfs_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:27.768 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cephfs (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:28.154 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-ceph-common (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:28.254 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-base. 2024-03-30T00:36:28.269 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../026-ceph-base_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:28.321 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-base (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:28.482 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-prettytable. 2024-03-30T00:36:28.498 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-03-30T00:36:28.532 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-30T00:36:28.768 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rbd. 2024-03-30T00:36:28.784 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../019-python3-rbd_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:28.818 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rbd (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:28.891 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-30T00:36:28.905 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-30T00:36:28.950 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-30T00:36:29.171 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-30T00:36:29.186 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-30T00:36:29.229 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-30T00:36:29.244 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-cheroot. 2024-03-30T00:36:29.258 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-30T00:36:29.328 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-30T00:36:29.565 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librgw2. 2024-03-30T00:36:29.580 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../021-librgw2_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:29.622 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-30T00:36:29.623 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librgw2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:29.637 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-30T00:36:29.672 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-30T00:36:30.000 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-30T00:36:30.014 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-30T00:36:30.058 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-30T00:36:30.093 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rgw. 2024-03-30T00:36:30.108 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../022-python3-rgw_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:30.142 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rgw (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:30.361 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-30T00:36:30.376 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-30T00:36:30.412 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-30T00:36:30.419 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-30T00:36:30.427 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-30T00:36:30.470 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-30T00:36:30.739 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-tempora. 2024-03-30T00:36:30.754 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2024-03-30T00:36:30.789 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-30T00:36:30.790 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libradosstriper1. 2024-03-30T00:36:30.805 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../024-libradosstriper1_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:30.848 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libradosstriper1 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:31.078 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-portend. 2024-03-30T00:36:31.091 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2024-03-30T00:36:31.133 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-30T00:36:31.293 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-common. 2024-03-30T00:36:31.308 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../025-ceph-common_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:31.351 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-common (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:31.420 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-30T00:36:31.435 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-30T00:36:31.478 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-30T00:36:31.764 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-30T00:36:31.779 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-30T00:36:31.814 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-30T00:36:32.176 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-natsort. 2024-03-30T00:36:32.191 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2024-03-30T00:36:32.234 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-30T00:36:32.399 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-base. 2024-03-30T00:36:32.414 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../026-ceph-base_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:32.457 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-base (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:32.554 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-logutils. 2024-03-30T00:36:32.570 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2024-03-30T00:36:32.613 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-30T00:36:32.915 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-30T00:36:32.930 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-30T00:36:32.973 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-30T00:36:33.019 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-30T00:36:33.033 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-30T00:36:33.077 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-30T00:36:33.235 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-mako. 2024-03-30T00:36:33.250 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-30T00:36:33.284 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-30T00:36:33.339 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cheroot. 2024-03-30T00:36:33.353 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-30T00:36:33.388 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-30T00:36:33.596 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-30T00:36:33.612 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-30T00:36:33.646 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-30T00:36:33.683 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-30T00:36:33.698 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-30T00:36:33.733 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-30T00:36:33.932 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-30T00:36:33.947 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-30T00:36:33.990 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-30T00:36:34.027 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-30T00:36:34.042 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-30T00:36:34.086 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-30T00:36:34.293 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-webob. 2024-03-30T00:36:34.308 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-30T00:36:34.360 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-30T00:36:34.388 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-30T00:36:34.404 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-30T00:36:34.447 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-30T00:36:34.730 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-waitress. 2024-03-30T00:36:34.741 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-tempora. 2024-03-30T00:36:34.745 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-30T00:36:34.757 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2024-03-30T00:36:34.791 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-30T00:36:34.791 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-30T00:36:35.027 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-portend. 2024-03-30T00:36:35.042 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2024-03-30T00:36:35.085 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-30T00:36:35.108 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-tempita. 2024-03-30T00:36:35.123 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-30T00:36:35.157 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-30T00:36:35.405 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-30T00:36:35.420 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-30T00:36:35.435 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-paste. 2024-03-30T00:36:35.450 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-30T00:36:35.463 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-30T00:36:35.494 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-30T00:36:35.774 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-30T00:36:35.790 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-30T00:36:35.814 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-30T00:36:35.828 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-30T00:36:35.841 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-30T00:36:35.863 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-30T00:36:36.175 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-30T00:36:36.190 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-30T00:36:36.233 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-30T00:36:36.244 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-natsort. 2024-03-30T00:36:36.260 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2024-03-30T00:36:36.310 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-30T00:36:36.706 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-logutils. 2024-03-30T00:36:36.719 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-webtest. 2024-03-30T00:36:36.721 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2024-03-30T00:36:36.734 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2024-03-30T00:36:36.772 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-30T00:36:36.786 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-30T00:36:37.081 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pecan. 2024-03-30T00:36:37.096 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-30T00:36:37.134 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-30T00:36:37.139 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-30T00:36:37.149 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-30T00:36:37.191 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-30T00:36:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-certifi. 2024-03-30T00:36:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2024-03-30T00:36:37.544 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-mako. 2024-03-30T00:36:37.560 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-30T00:36:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-30T00:36:37.811 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-30T00:36:37.897 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-idna. 2024-03-30T00:36:37.912 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2024-03-30T00:36:37.955 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-30T00:36:38.206 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-30T00:36:38.221 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-30T00:36:38.264 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-30T00:36:38.274 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-urllib3. 2024-03-30T00:36:38.289 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-30T00:36:38.332 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-30T00:36:38.567 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-30T00:36:38.582 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-30T00:36:38.625 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-30T00:36:38.702 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-requests. 2024-03-30T00:36:38.718 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-30T00:36:38.761 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-30T00:36:38.878 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-webob. 2024-03-30T00:36:38.893 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-30T00:36:38.927 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-30T00:36:39.072 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-30T00:36:39.087 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-30T00:36:39.130 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-30T00:36:39.230 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-waitress. 2024-03-30T00:36:39.246 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-30T00:36:39.291 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-30T00:36:39.417 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-30T00:36:39.432 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:39.475 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:39.608 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-tempita. 2024-03-30T00:36:39.624 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-30T00:36:39.666 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-30T00:36:39.847 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-30T00:36:39.861 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:39.930 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:39.969 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-paste. 2024-03-30T00:36:39.985 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-30T00:36:40.027 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-30T00:36:40.216 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-openssl. 2024-03-30T00:36:40.231 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2024-03-30T00:36:40.274 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-30T00:36:40.355 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-30T00:36:40.371 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-30T00:36:40.413 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-30T00:36:40.619 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr. 2024-03-30T00:36:40.634 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../058-ceph-mgr_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:40.677 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:40.750 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-30T00:36:40.765 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-30T00:36:40.808 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-30T00:36:41.022 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mon. 2024-03-30T00:36:41.037 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../059-ceph-mon_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:41.052 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-webtest. 2024-03-30T00:36:41.067 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2024-03-30T00:36:41.071 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mon (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:41.102 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-30T00:36:41.414 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pecan. 2024-03-30T00:36:41.429 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-30T00:36:41.472 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-30T00:36:41.617 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-osd. 2024-03-30T00:36:41.632 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../060-ceph-osd_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:41.675 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-osd (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:41.835 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-certifi. 2024-03-30T00:36:41.850 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2024-03-30T00:36:41.893 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-30T00:36:42.922 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-idna. 2024-03-30T00:36:42.937 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2024-03-30T00:36:42.979 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-30T00:36:43.071 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph. 2024-03-30T00:36:43.086 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../061-ceph_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:43.120 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:43.275 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-urllib3. 2024-03-30T00:36:43.289 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-30T00:36:43.323 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-30T00:36:43.423 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-fuse. 2024-03-30T00:36:43.438 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../062-ceph-fuse_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:43.481 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-fuse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:43.601 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-requests. 2024-03-30T00:36:43.617 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-30T00:36:43.651 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-30T00:36:43.835 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mds. 2024-03-30T00:36:43.850 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../063-ceph-mds_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:43.893 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mds (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:43.913 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-30T00:36:43.928 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-30T00:36:43.988 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-30T00:36:44.241 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-30T00:36:44.254 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package cephadm. 2024-03-30T00:36:44.255 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:44.269 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../064-cephadm_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:44.290 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:44.304 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking cephadm (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:44.582 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-30T00:36:44.597 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-30T00:36:44.640 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-30T00:36:44.687 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-30T00:36:44.701 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:44.753 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:44.943 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-30T00:36:44.958 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-30T00:36:45.002 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-30T00:36:45.081 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-openssl. 2024-03-30T00:36:45.098 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2024-03-30T00:36:45.147 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-30T00:36:45.434 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr. 2024-03-30T00:36:45.449 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../058-ceph-mgr_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:45.483 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:45.878 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mon. 2024-03-30T00:36:45.894 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../059-ceph-mon_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:45.936 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mon (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:45.973 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-babel. 2024-03-30T00:36:45.988 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-30T00:36:46.032 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-30T00:36:46.360 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-jinja2. 2024-03-30T00:36:46.374 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-30T00:36:46.418 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-30T00:36:46.549 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-osd. 2024-03-30T00:36:46.564 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../060-ceph-osd_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:46.606 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-osd (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:46.980 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-30T00:36:46.995 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:47.038 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:47.344 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph. 2024-03-30T00:36:47.358 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../061-ceph_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:47.366 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-30T00:36:47.384 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-30T00:36:47.393 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:47.424 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-30T00:36:47.704 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-fuse. 2024-03-30T00:36:47.719 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../062-ceph-fuse_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:47.736 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pluggy. 2024-03-30T00:36:47.751 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-30T00:36:47.762 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-fuse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:47.794 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-30T00:36:48.097 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-py. 2024-03-30T00:36:48.112 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2024-03-30T00:36:48.156 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-30T00:36:48.307 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mds. 2024-03-30T00:36:48.323 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../063-ceph-mds_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:48.365 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mds (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:48.467 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-30T00:36:48.482 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-30T00:36:48.526 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-30T00:36:48.768 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package cephadm. 2024-03-30T00:36:48.783 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../064-cephadm_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:48.812 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-toml. 2024-03-30T00:36:48.826 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking cephadm (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:48.827 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2024-03-30T00:36:48.870 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-30T00:36:49.138 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-30T00:36:49.153 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-30T00:36:49.196 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-30T00:36:49.198 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pytest. 2024-03-30T00:36:49.213 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-30T00:36:49.256 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-30T00:36:49.549 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-30T00:36:49.564 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-30T00:36:49.577 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-30T00:36:49.594 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-03-30T00:36:49.607 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-30T00:36:49.626 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-30T00:36:49.896 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-routes. 2024-03-30T00:36:49.911 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-30T00:36:49.954 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-30T00:36:50.266 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-30T00:36:50.282 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:50.324 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:50.578 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-babel. 2024-03-30T00:36:50.592 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-30T00:36:50.637 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-30T00:36:50.753 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-30T00:36:50.767 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-30T00:36:50.811 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-30T00:36:50.973 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jinja2. 2024-03-30T00:36:50.987 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-30T00:36:51.023 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-30T00:36:51.281 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-joblib. 2024-03-30T00:36:51.293 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-30T00:36:51.331 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-30T00:36:51.351 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-30T00:36:51.366 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:51.401 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:51.676 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-30T00:36:51.692 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-30T00:36:51.713 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-30T00:36:51.727 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-30T00:36:51.734 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-30T00:36:51.762 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-30T00:36:52.032 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pluggy. 2024-03-30T00:36:52.047 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-30T00:36:52.071 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-sklearn. 2024-03-30T00:36:52.087 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-30T00:36:52.123 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-30T00:36:52.129 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-30T00:36:52.435 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-py. 2024-03-30T00:36:52.454 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2024-03-30T00:36:52.494 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-30T00:36:52.822 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-30T00:36:52.837 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-30T00:36:52.848 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-30T00:36:52.862 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:52.888 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-30T00:36:52.906 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:53.199 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-toml. 2024-03-30T00:36:53.215 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2024-03-30T00:36:53.250 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-30T00:36:53.519 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pytest. 2024-03-30T00:36:53.534 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-30T00:36:53.577 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-30T00:36:53.842 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-cachetools. 2024-03-30T00:36:53.857 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-03-30T00:36:53.892 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-30T00:36:53.939 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-30T00:36:53.954 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-03-30T00:36:53.988 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-30T00:36:54.153 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-30T00:36:54.168 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-30T00:36:54.203 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-30T00:36:54.241 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-routes. 2024-03-30T00:36:54.257 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-30T00:36:54.291 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-30T00:36:54.549 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-30T00:36:54.563 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-30T00:36:54.607 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-30T00:36:54.620 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-30T00:36:54.635 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:54.669 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:54.910 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-rsa. 2024-03-30T00:36:54.924 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-03-30T00:36:54.968 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-30T00:36:55.075 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-30T00:36:55.090 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-30T00:36:55.125 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-30T00:36:55.314 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-google-auth. 2024-03-30T00:36:55.331 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-03-30T00:36:55.372 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-30T00:36:55.636 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-joblib. 2024-03-30T00:36:55.652 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-30T00:36:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-30T00:36:55.692 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-30T00:36:55.707 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-30T00:36:55.750 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-30T00:36:56.039 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-30T00:36:56.044 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-websocket. 2024-03-30T00:36:56.049 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-30T00:36:56.060 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-03-30T00:36:56.089 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-30T00:36:56.094 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-30T00:36:56.392 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-sklearn. 2024-03-30T00:36:56.402 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-30T00:36:56.406 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-30T00:36:56.421 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-30T00:36:56.442 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-30T00:36:56.464 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-30T00:36:57.202 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-30T00:36:57.212 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:57.252 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:57.370 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-30T00:36:57.384 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:36:57.428 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:57.723 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-30T00:36:57.737 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-30T00:36:57.773 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-30T00:36:58.092 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-30T00:36:58.107 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-30T00:36:58.150 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-30T00:36:58.388 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cachetools. 2024-03-30T00:36:58.402 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-03-30T00:36:58.411 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package jq. 2024-03-30T00:36:58.427 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-30T00:36:58.438 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-30T00:36:58.495 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-30T00:36:58.699 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-30T00:36:58.711 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-30T00:36:58.748 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package socat. 2024-03-30T00:36:58.749 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-30T00:36:58.763 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-30T00:36:58.806 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-30T00:36:59.084 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package xmlstarlet. 2024-03-30T00:36:59.086 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-30T00:36:59.098 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-30T00:36:59.099 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-30T00:36:59.134 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-30T00:36:59.161 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-30T00:36:59.440 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-test. 2024-03-30T00:36:59.454 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../098-ceph-test_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:36:59.472 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rsa. 2024-03-30T00:36:59.483 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-03-30T00:36:59.490 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-test (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:36:59.522 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-30T00:36:59.859 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-google-auth. 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-03-30T00:36:59.909 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-30T00:37:00.212 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-30T00:37:00.223 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-30T00:37:00.283 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-30T00:37:00.556 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-websocket. 2024-03-30T00:37:00.568 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-03-30T00:37:00.597 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-30T00:37:00.909 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-30T00:37:00.924 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-30T00:37:00.967 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-30T00:37:01.095 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-volume. 2024-03-30T00:37:01.110 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../099-ceph-volume_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:37:01.152 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-volume (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:01.507 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-30T00:37:01.521 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:37:01.565 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking cephfs-mirror (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:01.681 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-30T00:37:01.693 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:37:01.731 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:01.868 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-colorama. 2024-03-30T00:37:01.882 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2024-03-30T00:37:01.926 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-30T00:37:02.042 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-30T00:37:02.054 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-30T00:37:02.092 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-30T00:37:02.188 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package cephfs-shell. 2024-03-30T00:37:02.201 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../102-cephfs-shell_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:37:02.246 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking cephfs-shell (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:02.378 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-30T00:37:02.389 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-30T00:37:02.419 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-30T00:37:02.493 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package cephfs-top. 2024-03-30T00:37:02.506 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../103-cephfs-top_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:37:02.549 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking cephfs-top (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:02.714 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package jq. 2024-03-30T00:37:02.730 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-30T00:37:02.772 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-30T00:37:02.803 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-30T00:37:02.815 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:37:02.859 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libcephfs-dev (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:03.000 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package socat. 2024-03-30T00:37:03.013 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-30T00:37:03.042 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-30T00:37:03.328 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package xmlstarlet. 2024-03-30T00:37:03.341 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-30T00:37:03.377 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-30T00:37:03.438 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package nvme-cli. 2024-03-30T00:37:03.451 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-30T00:37:03.496 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-30T00:37:03.823 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-test. 2024-03-30T00:37:03.836 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../098-ceph-test_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:37:03.843 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-30T00:37:03.855 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-30T00:37:03.873 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-test (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:03.884 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-30T00:37:04.246 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pastescript. 2024-03-30T00:37:04.258 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-03-30T00:37:04.287 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-30T00:37:04.549 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-psutil. 2024-03-30T00:37:04.564 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-30T00:37:04.599 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-30T00:37:04.910 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pygments. 2024-03-30T00:37:04.923 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-30T00:37:04.968 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-30T00:37:05.447 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-30T00:37:05.460 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-30T00:37:05.505 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-30T00:37:05.594 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-volume. 2024-03-30T00:37:05.609 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../099-ceph-volume_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:37:05.661 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-volume (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:05.808 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-simplejson. 2024-03-30T00:37:05.821 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-30T00:37:05.867 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-30T00:37:06.090 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-30T00:37:06.103 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:37:06.119 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-30T00:37:06.132 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-30T00:37:06.159 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking cephfs-mirror (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:06.177 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-30T00:37:06.517 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-colorama. 2024-03-30T00:37:06.528 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2024-03-30T00:37:06.571 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-30T00:37:06.828 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package cephfs-shell. 2024-03-30T00:37:06.838 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../102-cephfs-shell_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:37:06.878 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking cephfs-shell (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:06.898 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package radosgw. 2024-03-30T00:37:06.911 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../113-radosgw_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:37:06.956 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking radosgw (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:07.147 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package cephfs-top. 2024-03-30T00:37:07.158 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../103-cephfs-top_18.2.2-1018-g470100a4-1jammy_all.deb ... 2024-03-30T00:37:07.197 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking cephfs-top (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:07.434 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-30T00:37:07.444 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:37:07.483 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libcephfs-dev (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:07.484 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package rbd-fuse. 2024-03-30T00:37:07.498 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../114-rbd-fuse_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:37:07.542 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking rbd-fuse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:07.828 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package nvme-cli. 2024-03-30T00:37:07.833 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-30T00:37:07.838 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-30T00:37:07.878 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-30T00:37:08.106 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-30T00:37:08.199 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-30T00:37:08.209 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-30T00:37:08.240 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-30T00:37:08.240 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-30T00:37:08.501 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-30T00:37:08.610 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pastescript. 2024-03-30T00:37:08.620 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-03-30T00:37:08.652 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-30T00:37:08.915 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-30T00:37:08.930 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-psutil. 2024-03-30T00:37:08.940 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-30T00:37:08.972 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-30T00:37:09.299 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pygments. 2024-03-30T00:37:09.310 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-30T00:37:09.341 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-30T00:37:09.493 INFO:teuthology.orchestra.run.smithi156.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-30T00:37:09.535 INFO:teuthology.orchestra.run.smithi156.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-03-30T00:37:09.575 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up cephadm (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:09.770 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-30T00:37:09.779 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-30T00:37:09.820 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-30T00:37:10.027 INFO:teuthology.orchestra.run.smithi156.stdout:Adding system user cephadm....done 2024-03-30T00:37:10.164 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-simplejson. 2024-03-30T00:37:10.173 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-30T00:37:10.176 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-30T00:37:10.214 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-30T00:37:10.483 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-30T00:37:10.483 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-30T00:37:10.494 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-30T00:37:10.541 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-30T00:37:10.725 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-30T00:37:10.851 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-30T00:37:11.111 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-30T00:37:11.295 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package radosgw. 2024-03-30T00:37:11.306 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../113-radosgw_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:37:11.345 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking radosgw (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:11.370 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-30T00:37:11.472 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-30T00:37:11.798 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-30T00:37:11.906 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package rbd-fuse. 2024-03-30T00:37:11.917 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../114-rbd-fuse_18.2.2-1018-g470100a4-1jammy_amd64.deb ... 2024-03-30T00:37:11.947 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking rbd-fuse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:12.159 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-30T00:37:12.230 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-30T00:37:12.410 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-30T00:37:12.553 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-30T00:37:12.678 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-30T00:37:12.711 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-30T00:37:12.946 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-30T00:37:12.987 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-30T00:37:13.247 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-30T00:37:13.371 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-30T00:37:13.565 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-30T00:37:13.908 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-ceph-argparse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:13.997 INFO:teuthology.orchestra.run.smithi066.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-30T00:37:14.034 INFO:teuthology.orchestra.run.smithi066.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-03-30T00:37:14.080 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up cephadm (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:14.175 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-30T00:37:14.301 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-30T00:37:14.577 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-30T00:37:14.590 INFO:teuthology.orchestra.run.smithi066.stdout:Adding system user cephadm....done 2024-03-30T00:37:14.777 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-30T00:37:14.837 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-30T00:37:14.962 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-30T00:37:15.087 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-30T00:37:15.322 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-30T00:37:15.371 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-30T00:37:15.457 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-30T00:37:15.497 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-30T00:37:15.748 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-30T00:37:15.758 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-30T00:37:16.025 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-30T00:37:16.041 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-30T00:37:16.193 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-30T00:37:16.335 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-30T00:37:16.527 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-30T00:37:16.913 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-30T00:37:17.029 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-30T00:37:17.230 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-30T00:37:17.305 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-30T00:37:17.431 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-30T00:37:17.481 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-30T00:37:17.565 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-30T00:37:17.766 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-30T00:37:18.158 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-30T00:37:18.167 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-30T00:37:18.452 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-30T00:37:18.543 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-30T00:37:18.745 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-ceph-argparse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:18.802 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-30T00:37:19.037 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-30T00:37:19.163 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-30T00:37:19.178 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-30T00:37:19.439 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-30T00:37:19.463 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-30T00:37:19.707 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-30T00:37:19.781 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-30T00:37:19.807 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-30T00:37:20.141 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-30T00:37:20.192 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-30T00:37:20.343 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-30T00:37:20.392 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-30T00:37:20.518 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-ceph-common (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:20.669 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-30T00:37:20.819 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-idna (3.3-1) ... 2024-03-30T00:37:20.945 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-30T00:37:21.145 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-30T00:37:21.288 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-30T00:37:21.999 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-30T00:37:22.283 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-30T00:37:22.376 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:37:22.442 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-30T00:37:22.510 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-30T00:37:22.593 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-30T00:37:22.769 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-30T00:37:22.895 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-30T00:37:22.936 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-30T00:37:23.021 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-30T00:37:23.323 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-30T00:37:23.379 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-30T00:37:23.630 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-30T00:37:23.655 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-30T00:37:23.875 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:37:24.002 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-03-30T00:37:24.049 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-30T00:37:24.127 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-30T00:37:24.366 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-30T00:37:24.420 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-30T00:37:24.671 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-30T00:37:24.701 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-30T00:37:24.965 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-30T00:37:25.110 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-30T00:37:25.232 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-30T00:37:25.395 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-30T00:37:25.475 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-30T00:37:25.521 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-ceph-common (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:25.734 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-30T00:37:25.830 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-idna (3.3-1) ... 2024-03-30T00:37:25.860 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-30T00:37:26.136 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-30T00:37:26.173 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-30T00:37:26.287 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-30T00:37:26.580 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-30T00:37:26.898 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-30T00:37:27.149 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-30T00:37:27.462 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:37:27.534 INFO:teuthology.orchestra.run.smithi156.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-30T00:37:27.621 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-30T00:37:27.802 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-30T00:37:27.897 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-30T00:37:27.928 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-30T00:37:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-30T00:37:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-30T00:37:28.321 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-30T00:37:28.451 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-30T00:37:28.597 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-30T00:37:28.727 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-30T00:37:28.905 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:37:29.011 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:37:29.033 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-30T00:37:29.154 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-03-30T00:37:29.279 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-30T00:37:29.367 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-30T00:37:29.598 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-30T00:37:29.677 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-30T00:37:29.857 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-30T00:37:29.936 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-30T00:37:30.037 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-30T00:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-30T00:37:30.305 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-30T00:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-30T00:37:30.590 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-30T00:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-30T00:37:30.866 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-30T00:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-30T00:37:31.129 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-30T00:37:31.234 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-30T00:37:31.438 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-30T00:37:31.535 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-30T00:37:31.547 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-30T00:37:31.832 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-30T00:37:31.887 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-30T00:37:32.013 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up librados2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:32.133 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-30T00:37:32.139 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up librgw2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:32.334 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:32.409 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-30T00:37:32.460 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libcephfs2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:32.586 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libradosstriper1 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:32.712 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-30T00:37:32.842 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-30T00:37:33.039 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up librbd1 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:33.104 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-30T00:37:33.165 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr-modules-core (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:33.263 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-30T00:37:33.291 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-fuse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:33.501 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-30T00:37:33.501 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-30T00:37:33.665 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-30T00:37:33.916 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-30T00:37:34.176 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libcephfs-dev (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:34.183 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T00:37:34.302 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-rados (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:34.309 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-30T00:37:34.428 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up cephfs-top (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:34.554 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-30T00:37:34.861 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-30T00:37:35.195 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-30T00:37:35.479 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-30T00:37:35.630 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-30T00:37:35.826 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-rbd (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:35.915 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-30T00:37:35.951 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up rbd-fuse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:36.061 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-rgw (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:36.162 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-cephfs (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up cephfs-shell (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-common (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi156.stdout:Adding group ceph....done 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi156.stdout:Adding system user ceph....done 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:Setting system user ceph properties....done 2024-03-30T00:37:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-30T00:37:37.682 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-30T00:37:37.731 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librados2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:37.832 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librgw2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:38.060 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:38.078 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-30T00:37:38.186 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libcephfs2 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:38.313 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libradosstriper1 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:38.438 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-30T00:37:38.732 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librbd1 (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:38.757 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up cephfs-mirror (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:38.882 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-modules-core (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:38.972 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-30T00:37:38.972 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-30T00:37:39.059 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-fuse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:39.296 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-30T00:37:39.296 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-30T00:37:39.571 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-test (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:39.696 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up radosgw (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:39.968 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libcephfs-dev (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:40.094 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rados (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:40.211 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-30T00:37:40.211 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-30T00:37:40.245 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up cephfs-top (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:40.371 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-30T00:37:40.807 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-base (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:41.153 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-30T00:37:41.711 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rbd (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:41.784 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mds (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:41.862 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up rbd-fuse (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:42.004 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rgw (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:42.007 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-30T00:37:42.007 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-30T00:37:42.131 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cephfs (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:42.265 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up cephfs-shell (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:42.416 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-common (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:42.635 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:42.830 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-30T00:37:42.831 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-30T00:37:43.119 INFO:teuthology.orchestra.run.smithi066.stdout:Adding group ceph....done 2024-03-30T00:37:43.445 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-osd (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:43.511 INFO:teuthology.orchestra.run.smithi066.stdout:Adding system user ceph....done 2024-03-30T00:37:43.705 INFO:teuthology.orchestra.run.smithi066.stdout:Setting system user ceph properties....done 2024-03-30T00:37:43.728 INFO:teuthology.orchestra.run.smithi066.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-30T00:37:43.770 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-30T00:37:43.770 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-30T00:37:43.877 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-30T00:37:44.278 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-30T00:37:44.355 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:44.490 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:44.632 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mon (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:44.858 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-30T00:37:44.858 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-30T00:37:45.126 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up cephfs-mirror (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:45.378 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-30T00:37:45.379 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-30T00:37:45.492 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr-cephadm (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:45.618 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:45.727 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr-dashboard (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:45.870 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-volume (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:46.036 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-test (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:46.162 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up radosgw (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:46.231 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-30T00:37:46.753 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-30T00:37:46.753 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-30T00:37:47.348 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-30T00:37:47.422 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-base (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:47.754 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-30T00:37:47.820 INFO:teuthology.orchestra.run.smithi156.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-30T00:37:47.825 DEBUG:teuthology.orchestra.run.smithi156:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-30T00:37:47.898 INFO:teuthology.orchestra.run.smithi156.stdout:Reading package lists... 2024-03-30T00:37:48.071 INFO:teuthology.orchestra.run.smithi156.stdout:Building dependency tree... 2024-03-30T00:37:48.072 INFO:teuthology.orchestra.run.smithi156.stdout:Reading state information... 2024-03-30T00:37:48.238 INFO:teuthology.orchestra.run.smithi156.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-30T00:37:48.238 INFO:teuthology.orchestra.run.smithi156.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-30T00:37:48.238 INFO:teuthology.orchestra.run.smithi156.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-30T00:37:48.238 INFO:teuthology.orchestra.run.smithi156.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T00:37:48.238 INFO:teuthology.orchestra.run.smithi156.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T00:37:48.238 INFO:teuthology.orchestra.run.smithi156.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T00:37:48.238 INFO:teuthology.orchestra.run.smithi156.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T00:37:48.238 INFO:teuthology.orchestra.run.smithi156.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-30T00:37:48.240 INFO:teuthology.orchestra.run.smithi156.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-30T00:37:48.240 INFO:teuthology.orchestra.run.smithi156.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T00:37:48.240 INFO:teuthology.orchestra.run.smithi156.stdout:The following additional packages will be installed: 2024-03-30T00:37:48.240 INFO:teuthology.orchestra.run.smithi156.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-30T00:37:48.240 INFO:teuthology.orchestra.run.smithi156.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-30T00:37:48.240 INFO:teuthology.orchestra.run.smithi156.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-03-30T00:37:48.240 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-30T00:37:48.240 INFO:teuthology.orchestra.run.smithi156.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-30T00:37:48.241 INFO:teuthology.orchestra.run.smithi156.stdout:Suggested packages: 2024-03-30T00:37:48.241 INFO:teuthology.orchestra.run.smithi156.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-30T00:37:48.241 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-03-30T00:37:48.241 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-03-30T00:37:48.294 INFO:teuthology.orchestra.run.smithi156.stdout:The following NEW packages will be installed: 2024-03-30T00:37:48.295 INFO:teuthology.orchestra.run.smithi156.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-30T00:37:48.295 INFO:teuthology.orchestra.run.smithi156.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-30T00:37:48.296 INFO:teuthology.orchestra.run.smithi156.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-30T00:37:48.296 INFO:teuthology.orchestra.run.smithi156.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-30T00:37:48.296 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-30T00:37:48.296 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-30T00:37:48.296 INFO:teuthology.orchestra.run.smithi156.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-30T00:37:48.297 INFO:teuthology.orchestra.run.smithi156.stdout:The following packages will be upgraded: 2024-03-30T00:37:48.298 INFO:teuthology.orchestra.run.smithi156.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-03-30T00:37:48.390 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mds (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:48.549 INFO:teuthology.orchestra.run.smithi156.stdout:4 upgraded, 39 newly installed, 0 to remove and 297 not upgraded. 2024-03-30T00:37:48.549 INFO:teuthology.orchestra.run.smithi156.stdout:Need to get 55.7 MB of archives. 2024-03-30T00:37:48.549 INFO:teuthology.orchestra.run.smithi156.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-30T00:37:48.549 INFO:teuthology.orchestra.run.smithi156.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.3 [1,063 kB] 2024-03-30T00:37:48.636 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-30T00:37:48.636 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-30T00:37:49.297 INFO:teuthology.orchestra.run.smithi156.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-30T00:37:49.300 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:49.317 INFO:teuthology.orchestra.run.smithi156.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-30T00:37:49.457 INFO:teuthology.orchestra.run.smithi156.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.3 [33.1 kB] 2024-03-30T00:37:49.458 INFO:teuthology.orchestra.run.smithi156.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.3 [24.4 kB] 2024-03-30T00:37:49.460 INFO:teuthology.orchestra.run.smithi156.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-30T00:37:49.490 INFO:teuthology.orchestra.run.smithi156.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-30T00:37:49.492 INFO:teuthology.orchestra.run.smithi156.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-30T00:37:49.495 INFO:teuthology.orchestra.run.smithi156.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-30T00:37:49.498 INFO:teuthology.orchestra.run.smithi156.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-30T00:37:49.498 INFO:teuthology.orchestra.run.smithi156.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-30T00:37:49.563 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-30T00:37:49.563 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-30T00:37:49.686 INFO:teuthology.orchestra.run.smithi156.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-30T00:37:50.026 INFO:teuthology.orchestra.run.smithi156.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-30T00:37:50.038 INFO:teuthology.orchestra.run.smithi156.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-30T00:37:50.045 INFO:teuthology.orchestra.run.smithi156.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-30T00:37:50.139 INFO:teuthology.orchestra.run.smithi156.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-30T00:37:50.289 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-osd (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:50.636 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-30T00:37:50.636 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-30T00:37:51.215 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:51.267 INFO:teuthology.orchestra.run.smithi156.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-30T00:37:51.269 INFO:teuthology.orchestra.run.smithi156.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-30T00:37:51.269 INFO:teuthology.orchestra.run.smithi156.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-30T00:37:51.304 INFO:teuthology.orchestra.run.smithi156.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-30T00:37:51.305 INFO:teuthology.orchestra.run.smithi156.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-30T00:37:51.309 INFO:teuthology.orchestra.run.smithi156.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-30T00:37:51.341 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:51.484 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mon (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:51.499 INFO:teuthology.orchestra.run.smithi156.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-30T00:37:51.707 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-30T00:37:51.707 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-30T00:37:52.202 INFO:teuthology.orchestra.run.smithi156.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-30T00:37:52.202 INFO:teuthology.orchestra.run.smithi156.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-30T00:37:52.352 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-cephadm (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:52.411 INFO:teuthology.orchestra.run.smithi156.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-30T00:37:52.411 INFO:teuthology.orchestra.run.smithi156.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-30T00:37:52.456 INFO:teuthology.orchestra.run.smithi156.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-30T00:37:52.486 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:52.520 INFO:teuthology.orchestra.run.smithi156.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-30T00:37:52.526 INFO:teuthology.orchestra.run.smithi156.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-30T00:37:52.527 INFO:teuthology.orchestra.run.smithi156.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-30T00:37:52.528 INFO:teuthology.orchestra.run.smithi156.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-30T00:37:52.528 INFO:teuthology.orchestra.run.smithi156.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-30T00:37:52.612 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-dashboard (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:52.755 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-volume (18.2.2-1018-g470100a4-1jammy) ... 2024-03-30T00:37:52.772 INFO:teuthology.orchestra.run.smithi156.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-30T00:37:53.155 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-30T00:37:53.270 INFO:teuthology.orchestra.run.smithi156.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-30T00:37:54.128 INFO:teuthology.orchestra.run.smithi156.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-30T00:37:54.128 INFO:teuthology.orchestra.run.smithi156.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-30T00:37:54.128 INFO:teuthology.orchestra.run.smithi156.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-30T00:37:54.129 INFO:teuthology.orchestra.run.smithi156.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-30T00:37:54.129 INFO:teuthology.orchestra.run.smithi156.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-30T00:37:54.129 INFO:teuthology.orchestra.run.smithi156.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-30T00:37:54.168 INFO:teuthology.orchestra.run.smithi156.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-30T00:37:54.169 INFO:teuthology.orchestra.run.smithi156.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-30T00:37:54.545 INFO:teuthology.orchestra.run.smithi156.stdout:Preconfiguring packages ... 2024-03-30T00:37:54.550 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-30T00:37:54.688 INFO:teuthology.orchestra.run.smithi156.stdout:Fetched 55.7 MB in 6s (9,505 kB/s) 2024-03-30T00:37:54.977 INFO:teuthology.orchestra.run.smithi156.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139840 files and directories currently installed.) 2024-03-30T00:37:54.981 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T00:37:55.058 INFO:teuthology.orchestra.run.smithi066.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-30T00:37:55.063 DEBUG:teuthology.orchestra.run.smithi066:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-30T00:37:55.130 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking util-linux (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T00:37:55.141 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-03-30T00:37:55.317 INFO:teuthology.orchestra.run.smithi066.stdout:Building dependency tree... 2024-03-30T00:37:55.318 INFO:teuthology.orchestra.run.smithi066.stdout:Reading state information... 2024-03-30T00:37:55.493 INFO:teuthology.orchestra.run.smithi066.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-30T00:37:55.494 INFO:teuthology.orchestra.run.smithi066.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-30T00:37:55.494 INFO:teuthology.orchestra.run.smithi066.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-30T00:37:55.494 INFO:teuthology.orchestra.run.smithi066.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T00:37:55.494 INFO:teuthology.orchestra.run.smithi066.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T00:37:55.494 INFO:teuthology.orchestra.run.smithi066.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T00:37:55.494 INFO:teuthology.orchestra.run.smithi066.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T00:37:55.495 INFO:teuthology.orchestra.run.smithi066.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-30T00:37:55.495 INFO:teuthology.orchestra.run.smithi066.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-30T00:37:55.495 INFO:teuthology.orchestra.run.smithi066.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T00:37:55.495 INFO:teuthology.orchestra.run.smithi066.stdout:The following additional packages will be installed: 2024-03-30T00:37:55.495 INFO:teuthology.orchestra.run.smithi066.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-30T00:37:55.495 INFO:teuthology.orchestra.run.smithi066.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-30T00:37:55.496 INFO:teuthology.orchestra.run.smithi066.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-03-30T00:37:55.496 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-30T00:37:55.496 INFO:teuthology.orchestra.run.smithi066.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-30T00:37:55.497 INFO:teuthology.orchestra.run.smithi066.stdout:Suggested packages: 2024-03-30T00:37:55.497 INFO:teuthology.orchestra.run.smithi066.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-30T00:37:55.497 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-03-30T00:37:55.497 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-03-30T00:37:55.552 INFO:teuthology.orchestra.run.smithi066.stdout:The following NEW packages will be installed: 2024-03-30T00:37:55.552 INFO:teuthology.orchestra.run.smithi066.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-30T00:37:55.552 INFO:teuthology.orchestra.run.smithi066.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-30T00:37:55.553 INFO:teuthology.orchestra.run.smithi066.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-30T00:37:55.553 INFO:teuthology.orchestra.run.smithi066.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-30T00:37:55.553 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-30T00:37:55.553 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-30T00:37:55.553 INFO:teuthology.orchestra.run.smithi066.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-30T00:37:55.554 INFO:teuthology.orchestra.run.smithi066.stdout:The following packages will be upgraded: 2024-03-30T00:37:55.555 INFO:teuthology.orchestra.run.smithi066.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-03-30T00:37:55.644 INFO:teuthology.orchestra.run.smithi066.stdout:4 upgraded, 39 newly installed, 0 to remove and 297 not upgraded. 2024-03-30T00:37:55.644 INFO:teuthology.orchestra.run.smithi066.stdout:Need to get 55.7 MB of archives. 2024-03-30T00:37:55.644 INFO:teuthology.orchestra.run.smithi066.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-30T00:37:55.644 INFO:teuthology.orchestra.run.smithi066.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.3 [1,063 kB] 2024-03-30T00:37:55.770 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up util-linux (2.37.2-4ubuntu3.3) ... 2024-03-30T00:37:55.870 INFO:teuthology.orchestra.run.smithi066.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-30T00:37:55.877 INFO:teuthology.orchestra.run.smithi066.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-30T00:37:55.928 INFO:teuthology.orchestra.run.smithi066.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.3 [33.1 kB] 2024-03-30T00:37:55.928 INFO:teuthology.orchestra.run.smithi066.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.3 [24.4 kB] 2024-03-30T00:37:55.929 INFO:teuthology.orchestra.run.smithi066.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-30T00:37:55.941 INFO:teuthology.orchestra.run.smithi066.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-30T00:37:55.942 INFO:teuthology.orchestra.run.smithi066.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-30T00:37:55.943 INFO:teuthology.orchestra.run.smithi066.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-30T00:37:55.944 INFO:teuthology.orchestra.run.smithi066.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-30T00:37:55.945 INFO:teuthology.orchestra.run.smithi066.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-30T00:37:55.946 INFO:teuthology.orchestra.run.smithi066.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-30T00:37:55.972 INFO:teuthology.orchestra.run.smithi066.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-30T00:37:55.972 INFO:teuthology.orchestra.run.smithi066.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-30T00:37:55.972 INFO:teuthology.orchestra.run.smithi066.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-30T00:37:56.008 INFO:teuthology.orchestra.run.smithi066.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-30T00:37:56.309 INFO:teuthology.orchestra.run.smithi066.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-30T00:37:56.311 INFO:teuthology.orchestra.run.smithi066.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-30T00:37:56.311 INFO:teuthology.orchestra.run.smithi066.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-30T00:37:56.314 INFO:teuthology.orchestra.run.smithi066.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-30T00:37:56.381 INFO:teuthology.orchestra.run.smithi066.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-30T00:37:56.550 INFO:teuthology.orchestra.run.smithi066.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-30T00:37:56.562 INFO:teuthology.orchestra.run.smithi066.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi066.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi066.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-30T00:37:56.640 INFO:teuthology.orchestra.run.smithi066.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-30T00:37:56.641 INFO:teuthology.orchestra.run.smithi066.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-30T00:37:56.652 INFO:teuthology.orchestra.run.smithi066.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-30T00:37:56.673 INFO:teuthology.orchestra.run.smithi066.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-30T00:37:56.677 INFO:teuthology.orchestra.run.smithi066.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-30T00:37:56.677 INFO:teuthology.orchestra.run.smithi066.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-30T00:37:56.678 INFO:teuthology.orchestra.run.smithi066.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-30T00:37:56.678 INFO:teuthology.orchestra.run.smithi066.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-30T00:37:56.690 INFO:teuthology.orchestra.run.smithi066.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-30T00:37:56.692 INFO:teuthology.orchestra.run.smithi066.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-30T00:37:56.827 INFO:teuthology.orchestra.run.smithi156.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-03-30T00:37:56.874 INFO:teuthology.orchestra.run.smithi066.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-30T00:37:56.875 INFO:teuthology.orchestra.run.smithi066.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-30T00:37:56.875 INFO:teuthology.orchestra.run.smithi066.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-30T00:37:56.875 INFO:teuthology.orchestra.run.smithi066.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-30T00:37:56.875 INFO:teuthology.orchestra.run.smithi066.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-30T00:37:56.875 INFO:teuthology.orchestra.run.smithi066.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-30T00:37:56.889 INFO:teuthology.orchestra.run.smithi066.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-30T00:37:56.889 INFO:teuthology.orchestra.run.smithi066.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-30T00:37:57.085 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package flex. 2024-03-30T00:37:57.157 INFO:teuthology.orchestra.run.smithi156.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139840 files and directories currently installed.) 2024-03-30T00:37:57.161 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-30T00:37:57.208 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-30T00:37:57.281 INFO:teuthology.orchestra.run.smithi066.stdout:Preconfiguring packages ... 2024-03-30T00:37:57.449 INFO:teuthology.orchestra.run.smithi066.stdout:Fetched 55.7 MB in 1s (42.0 MB/s) 2024-03-30T00:37:57.705 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-30T00:37:57.738 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139840 files and directories currently installed.) 2024-03-30T00:37:57.741 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T00:37:57.883 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-30T00:37:57.898 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking util-linux (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T00:37:58.289 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T00:37:58.550 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up util-linux (2.37.2-4ubuntu3.3) ... 2024-03-30T00:37:58.829 INFO:teuthology.orchestra.run.smithi156.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139913 files and directories currently installed.) 2024-03-30T00:37:58.832 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T00:37:58.974 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T00:37:59.370 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T00:37:59.488 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T00:37:59.750 INFO:teuthology.orchestra.run.smithi066.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-03-30T00:37:59.794 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-30T00:38:00.035 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package flex. 2024-03-30T00:38:00.108 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139840 files and directories currently installed.) 2024-03-30T00:38:00.112 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-30T00:38:00.150 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-30T00:38:00.164 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package bison. 2024-03-30T00:38:00.236 INFO:teuthology.orchestra.run.smithi156.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139913 files and directories currently installed.) 2024-03-30T00:38:00.240 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-30T00:38:00.282 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-30T00:38:00.603 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-30T00:38:00.717 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-30T00:38:00.731 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-30T00:38:00.775 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-30T00:38:00.781 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-30T00:38:01.103 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package dump. 2024-03-30T00:38:01.118 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-30T00:38:01.161 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking dump (0.4b47-3) ... 2024-03-30T00:38:01.187 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T00:38:01.507 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package indent. 2024-03-30T00:38:01.522 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:01.565 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-30T00:38:01.796 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139913 files and directories currently installed.) 2024-03-30T00:38:01.799 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T00:38:01.818 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-30T00:38:01.833 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-30T00:38:01.868 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-30T00:38:01.905 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T00:38:02.230 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-30T00:38:02.246 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-30T00:38:02.288 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-30T00:38:02.393 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T00:38:02.558 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-30T00:38:02.570 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T00:38:02.573 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-30T00:38:02.616 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-30T00:38:02.928 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-30T00:38:02.943 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-30T00:38:02.944 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-30T00:38:02.978 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-30T00:38:03.214 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-30T00:38:03.229 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-30T00:38:03.297 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-30T00:38:03.337 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package bison. 2024-03-30T00:38:03.416 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139913 files and directories currently installed.) 2024-03-30T00:38:03.420 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-30T00:38:03.464 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-30T00:38:03.558 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libjson-perl. 2024-03-30T00:38:03.574 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-30T00:38:03.616 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-30T00:38:03.886 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-30T00:38:03.902 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-30T00:38:03.923 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-30T00:38:03.936 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-30T00:38:03.938 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-30T00:38:03.981 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-30T00:38:04.392 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package dump. 2024-03-30T00:38:04.409 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-30T00:38:04.459 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking dump (0.4b47-3) ... 2024-03-30T00:38:04.805 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package indent. 2024-03-30T00:38:04.821 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:04.871 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-30T00:38:05.207 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-30T00:38:05.224 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-30T00:38:05.265 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-30T00:38:05.314 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-30T00:38:05.330 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-30T00:38:05.372 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-30T00:38:05.649 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-30T00:38:05.661 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-30T00:38:05.677 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-30T00:38:05.691 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T00:38:05.728 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-30T00:38:05.977 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-30T00:38:05.993 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-30T00:38:06.027 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T00:38:06.063 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-30T00:38:06.080 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-30T00:38:06.130 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-30T00:38:06.364 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-30T00:38:06.380 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-30T00:38:06.422 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-30T00:38:06.500 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-30T00:38:06.516 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-30T00:38:06.558 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-30T00:38:06.750 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-30T00:38:06.766 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-30T00:38:06.808 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-30T00:38:06.836 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-30T00:38:06.852 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-30T00:38:06.903 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-30T00:38:07.144 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-30T00:38:07.160 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:07.202 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:07.247 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libjson-perl. 2024-03-30T00:38:07.263 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-30T00:38:07.314 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-30T00:38:07.530 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-30T00:38:07.545 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:07.589 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T00:38:07.600 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-30T00:38:07.616 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-30T00:38:07.650 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-30T00:38:07.916 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-30T00:38:07.932 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:07.975 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T00:38:08.185 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-30T00:38:08.200 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-30T00:38:08.235 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T00:38:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-30T00:38:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-30T00:38:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-30T00:38:09.011 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-30T00:38:09.028 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-30T00:38:09.070 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T00:38:09.167 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package modemmanager. 2024-03-30T00:38:09.182 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-30T00:38:09.225 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T00:38:09.339 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-30T00:38:09.355 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-30T00:38:09.397 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T00:38:09.733 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-30T00:38:09.749 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-30T00:38:09.791 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T00:38:09.857 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package network-manager. 2024-03-30T00:38:09.873 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-30T00:38:09.923 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-30T00:38:10.127 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-30T00:38:10.143 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-30T00:38:10.194 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-30T00:38:10.530 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-30T00:38:10.538 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ppp. 2024-03-30T00:38:10.547 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-30T00:38:10.554 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-30T00:38:10.588 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-30T00:38:10.883 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-30T00:38:10.899 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:10.909 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-30T00:38:10.933 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:11.227 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package pptp-linux. 2024-03-30T00:38:11.241 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-30T00:38:11.277 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-30T00:38:11.285 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-30T00:38:11.294 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:11.335 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T00:38:11.546 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-30T00:38:11.561 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-30T00:38:11.630 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-30T00:38:11.638 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-30T00:38:11.647 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:11.688 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T00:38:11.924 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-30T00:38:11.940 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-30T00:38:11.941 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-30T00:38:11.955 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-30T00:38:11.982 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T00:38:11.999 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking postgresql-client-common (238) ... 2024-03-30T00:38:12.303 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-30T00:38:12.318 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:12.361 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:12.444 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-30T00:38:12.460 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-30T00:38:12.494 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-30T00:38:12.841 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package postgresql-common. 2024-03-30T00:38:12.847 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package modemmanager. 2024-03-30T00:38:12.855 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-30T00:38:12.863 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-30T00:38:12.905 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T00:38:12.907 INFO:teuthology.orchestra.run.smithi156.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-30T00:38:12.964 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking postgresql-common (238) ... 2024-03-30T00:38:13.398 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package postgresql-14. 2024-03-30T00:38:13.412 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:13.458 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:13.512 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package network-manager. 2024-03-30T00:38:13.528 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-30T00:38:13.570 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-30T00:38:14.194 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ppp. 2024-03-30T00:38:14.210 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-30T00:38:14.535 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-30T00:38:14.719 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package postgresql. 2024-03-30T00:38:14.733 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-30T00:38:14.776 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking postgresql (14+238) ... 2024-03-30T00:38:14.848 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package pptp-linux. 2024-03-30T00:38:14.863 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-30T00:38:14.898 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-30T00:38:15.020 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package postgresql-client. 2024-03-30T00:38:15.034 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-30T00:38:15.078 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking postgresql-client (14+238) ... 2024-03-30T00:38:15.130 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-30T00:38:15.219 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-30T00:38:15.219 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-30T00:38:15.322 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-30T00:38:15.336 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-30T00:38:15.380 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-30T00:38:15.496 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-30T00:38:15.511 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-30T00:38:15.554 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-client-common (238) ... 2024-03-30T00:38:15.666 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package tcl8.6. 2024-03-30T00:38:15.681 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-30T00:38:15.716 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-30T00:38:15.866 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-30T00:38:15.881 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:15.916 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:16.002 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package tcl. 2024-03-30T00:38:16.018 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-30T00:38:16.065 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-30T00:38:16.413 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package wpasupplicant. 2024-03-30T00:38:16.429 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-30T00:38:16.471 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-30T00:38:16.487 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-common. 2024-03-30T00:38:16.503 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-30T00:38:16.564 INFO:teuthology.orchestra.run.smithi066.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-30T00:38:16.602 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-common (238) ... 2024-03-30T00:38:16.817 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-30T00:38:16.834 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-30T00:38:16.892 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-30T00:38:17.102 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-14. 2024-03-30T00:38:17.118 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T00:38:17.164 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:17.254 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-30T00:38:17.270 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-30T00:38:17.320 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-30T00:38:17.604 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up postgresql-client-common (238) ... 2024-03-30T00:38:17.809 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-30T00:38:17.911 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-30T00:38:18.011 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:18.112 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-30T00:38:18.255 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-30T00:38:18.313 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql. 2024-03-30T00:38:18.328 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-30T00:38:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql (14+238) ... 2024-03-30T00:38:18.381 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-client. 2024-03-30T00:38:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-30T00:38:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-client (14+238) ... 2024-03-30T00:38:18.909 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-30T00:38:18.924 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-30T00:38:18.959 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-30T00:38:19.236 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package tcl8.6. 2024-03-30T00:38:19.252 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-30T00:38:19.286 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-30T00:38:19.336 INFO:teuthology.orchestra.run.smithi156.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-30T00:38:19.564 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package tcl. 2024-03-30T00:38:19.580 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-30T00:38:19.627 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-30T00:38:19.795 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-30T00:38:19.975 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package wpasupplicant. 2024-03-30T00:38:19.992 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-30T00:38:20.033 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-30T00:38:20.158 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-30T00:38:20.158 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-30T00:38:20.856 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-30T00:38:20.885 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-30T00:38:20.901 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-30T00:38:20.970 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-30T00:38:20.990 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-30T00:38:21.091 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T00:38:21.209 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-30T00:38:21.282 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-30T00:38:21.298 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-30T00:38:21.335 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-30T00:38:21.340 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-30T00:38:21.461 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-30T00:38:21.587 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-30T00:38:21.625 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-client-common (238) ... 2024-03-30T00:38:21.797 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-30T00:38:21.879 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-30T00:38:22.005 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-30T00:38:22.131 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:22.274 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-30T00:38:22.425 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-30T00:38:22.452 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-30T00:38:22.576 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:22.578 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up dump (0.4b47-3) ... 2024-03-30T00:38:22.673 INFO:teuthology.orchestra.run.smithi156.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-30T00:38:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T00:38:22.864 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-30T00:38:22.990 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-30T00:38:23.116 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-30T00:38:23.234 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-30T00:38:23.315 INFO:teuthology.orchestra.run.smithi156.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-30T00:38:23.377 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-30T00:38:23.503 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-30T00:38:23.584 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-30T00:38:23.683 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-30T00:38:23.704 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-30T00:38:23.830 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T00:38:23.956 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-30T00:38:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-30T00:38:24.092 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-30T00:38:24.116 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up postgresql-client (14+238) ... 2024-03-30T00:38:24.234 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-30T00:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-30T00:38:24.962 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-30T00:38:25.130 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T00:38:25.149 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:25.149 INFO:teuthology.orchestra.run.smithi156.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-30T00:38:25.149 INFO:teuthology.orchestra.run.smithi156.stdout:which means they are currently configured by ifupdown: 2024-03-30T00:38:25.150 INFO:teuthology.orchestra.run.smithi156.stdout:- enp3s0f1 2024-03-30T00:38:25.150 INFO:teuthology.orchestra.run.smithi156.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-30T00:38:25.150 INFO:teuthology.orchestra.run.smithi156.stdout:remove their configuration from /etc/network/interfaces. 2024-03-30T00:38:25.150 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:25.256 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-30T00:38:25.282 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-30T00:38:25.391 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-30T00:38:25.508 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-30T00:38:25.678 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-30T00:38:26.034 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-30T00:38:26.067 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-30T00:38:26.160 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-30T00:38:26.745 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T00:38:26.840 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-30T00:38:26.888 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-30T00:38:26.949 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up dump (0.4b47-3) ... 2024-03-30T00:38:27.014 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-30T00:38:27.044 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-30T00:38:27.109 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T00:38:27.140 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up postgresql-common (238) ... 2024-03-30T00:38:27.235 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-30T00:38:27.361 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-30T00:38:27.512 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-30T00:38:27.638 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-30T00:38:27.728 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-30T00:38:27.790 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-30T00:38:27.891 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-30T00:38:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-30T00:38:28.066 INFO:teuthology.orchestra.run.smithi156.stdout:Adding user postgres to group ssl-cert 2024-03-30T00:38:28.176 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T00:38:28.302 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-30T00:38:28.326 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:28.326 INFO:teuthology.orchestra.run.smithi156.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-30T00:38:28.470 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-client (14+238) ... 2024-03-30T00:38:28.582 INFO:teuthology.orchestra.run.smithi156.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-30T00:38:28.582 INFO:teuthology.orchestra.run.smithi156.stdout:Removing obsolete dictionary files: 2024-03-30T00:38:28.621 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-30T00:38:29.215 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:29.215 INFO:teuthology.orchestra.run.smithi066.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-30T00:38:29.215 INFO:teuthology.orchestra.run.smithi066.stdout:which means they are currently configured by ifupdown: 2024-03-30T00:38:29.215 INFO:teuthology.orchestra.run.smithi066.stdout:- enp3s0f1 2024-03-30T00:38:29.215 INFO:teuthology.orchestra.run.smithi066.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-30T00:38:29.215 INFO:teuthology.orchestra.run.smithi066.stdout:remove their configuration from /etc/network/interfaces. 2024-03-30T00:38:29.215 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:29.291 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-30T00:38:29.355 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-30T00:38:29.758 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:29.790 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-30T00:38:30.083 INFO:teuthology.orchestra.run.smithi156.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-30T00:38:30.150 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-30T00:38:30.214 INFO:teuthology.orchestra.run.smithi156.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-30T00:38:30.214 INFO:teuthology.orchestra.run.smithi156.stdout:This user must also own the server process. 2024-03-30T00:38:30.214 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:30.215 INFO:teuthology.orchestra.run.smithi156.stdout:The database cluster will be initialized with locale "en_US". 2024-03-30T00:38:30.215 INFO:teuthology.orchestra.run.smithi156.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-30T00:38:30.215 INFO:teuthology.orchestra.run.smithi156.stdout:The default text search configuration will be set to "english". 2024-03-30T00:38:30.215 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:30.215 INFO:teuthology.orchestra.run.smithi156.stdout:Data page checksums are disabled. 2024-03-30T00:38:30.215 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:30.215 INFO:teuthology.orchestra.run.smithi156.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-30T00:38:30.216 INFO:teuthology.orchestra.run.smithi156.stdout:creating subdirectories ... ok 2024-03-30T00:38:30.216 INFO:teuthology.orchestra.run.smithi156.stdout:selecting dynamic shared memory implementation ... posix 2024-03-30T00:38:30.227 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-30T00:38:30.239 INFO:teuthology.orchestra.run.smithi156.stdout:selecting default max_connections ... 100 2024-03-30T00:38:30.273 INFO:teuthology.orchestra.run.smithi156.stdout:selecting default shared_buffers ... 128MB 2024-03-30T00:38:30.287 INFO:teuthology.orchestra.run.smithi156.stdout:selecting default time zone ... Etc/UTC 2024-03-30T00:38:30.287 INFO:teuthology.orchestra.run.smithi156.stdout:creating configuration files ... ok 2024-03-30T00:38:30.450 INFO:teuthology.orchestra.run.smithi156.stdout:running bootstrap script ... ok 2024-03-30T00:38:30.940 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T00:38:31.058 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-30T00:38:31.093 INFO:teuthology.orchestra.run.smithi156.stdout:performing post-bootstrap initialization ... ok 2024-03-30T00:38:31.193 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-30T00:38:31.318 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-common (238) ... 2024-03-30T00:38:31.631 INFO:teuthology.orchestra.run.smithi156.stdout:syncing data to disk ... ok 2024-03-30T00:38:32.128 INFO:teuthology.orchestra.run.smithi156.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-30T00:38:32.415 INFO:teuthology.orchestra.run.smithi066.stdout:Adding user postgres to group ssl-cert 2024-03-30T00:38:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:32.662 INFO:teuthology.orchestra.run.smithi066.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-30T00:38:32.936 INFO:teuthology.orchestra.run.smithi066.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-30T00:38:32.936 INFO:teuthology.orchestra.run.smithi066.stdout:Removing obsolete dictionary files: 2024-03-30T00:38:33.676 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-30T00:38:34.178 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T00:38:34.542 INFO:teuthology.orchestra.run.smithi066.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-30T00:38:34.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-30T00:38:34.617 INFO:teuthology.orchestra.run.smithi066.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-30T00:38:34.617 INFO:teuthology.orchestra.run.smithi066.stdout:This user must also own the server process. 2024-03-30T00:38:34.617 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:34.617 INFO:teuthology.orchestra.run.smithi066.stdout:The database cluster will be initialized with locale "en_US". 2024-03-30T00:38:34.617 INFO:teuthology.orchestra.run.smithi066.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-30T00:38:34.617 INFO:teuthology.orchestra.run.smithi066.stdout:The default text search configuration will be set to "english". 2024-03-30T00:38:34.617 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:34.617 INFO:teuthology.orchestra.run.smithi066.stdout:Data page checksums are disabled. 2024-03-30T00:38:34.617 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:34.617 INFO:teuthology.orchestra.run.smithi066.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-30T00:38:34.618 INFO:teuthology.orchestra.run.smithi066.stdout:creating subdirectories ... ok 2024-03-30T00:38:34.618 INFO:teuthology.orchestra.run.smithi066.stdout:selecting dynamic shared memory implementation ... posix 2024-03-30T00:38:34.642 INFO:teuthology.orchestra.run.smithi066.stdout:selecting default max_connections ... 100 2024-03-30T00:38:34.678 INFO:teuthology.orchestra.run.smithi066.stdout:selecting default shared_buffers ... 128MB 2024-03-30T00:38:34.692 INFO:teuthology.orchestra.run.smithi066.stdout:selecting default time zone ... Etc/UTC 2024-03-30T00:38:34.692 INFO:teuthology.orchestra.run.smithi066.stdout:creating configuration files ... ok 2024-03-30T00:38:34.866 INFO:teuthology.orchestra.run.smithi066.stdout:running bootstrap script ... ok 2024-03-30T00:38:35.223 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T00:38:35.350 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-30T00:38:35.484 INFO:teuthology.orchestra.run.smithi066.stdout:performing post-bootstrap initialization ... ok 2024-03-30T00:38:35.509 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up postgresql (14+238) ... 2024-03-30T00:38:35.618 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T00:38:35.727 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T00:38:35.989 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-30T00:38:35.989 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-30T00:38:36.181 INFO:teuthology.orchestra.run.smithi066.stdout:syncing data to disk ... ok 2024-03-30T00:38:36.693 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-30T00:38:36.713 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-30T00:38:36.923 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-30T00:38:37.258 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-30T00:38:37.410 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-30T00:38:38.787 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-30T00:38:38.870 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-30T00:38:39.542 INFO:teuthology.orchestra.run.smithi156.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-30T00:38:39.549 DEBUG:teuthology.parallel:result is None 2024-03-30T00:38:39.652 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T00:38:39.778 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-30T00:38:39.904 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up postgresql (14+238) ... 2024-03-30T00:38:40.030 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T00:38:40.165 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T00:38:40.407 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-30T00:38:40.407 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-30T00:38:41.200 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-30T00:38:41.443 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-30T00:38:41.745 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-30T00:38:41.897 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-30T00:38:43.366 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-30T00:38:43.466 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-30T00:38:44.209 INFO:teuthology.orchestra.run.smithi066.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-30T00:38:44.215 DEBUG:teuthology.parallel:result is None 2024-03-30T00:38:44.216 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:38:44.429 DEBUG:teuthology.orchestra.run.smithi066:> dpkg-query -W -f '${Version}' ceph 2024-03-30T00:38:44.452 INFO:teuthology.orchestra.run.smithi066.stdout:18.2.2-1018-g470100a4-1jammy 2024-03-30T00:38:44.452 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1018-g470100a4-1jammy 2024-03-30T00:38:44.452 INFO:teuthology.task.install:The correct ceph version 18.2.2-1018-g470100a4-1jammy is installed. 2024-03-30T00:38:44.454 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:38:44.670 DEBUG:teuthology.orchestra.run.smithi156:> dpkg-query -W -f '${Version}' ceph 2024-03-30T00:38:44.695 INFO:teuthology.orchestra.run.smithi156.stdout:18.2.2-1018-g470100a4-1jammy 2024-03-30T00:38:44.695 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1018-g470100a4-1jammy 2024-03-30T00:38:44.695 INFO:teuthology.task.install:The correct ceph version 18.2.2-1018-g470100a4-1jammy is installed. 2024-03-30T00:38:44.697 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-30T00:38:44.697 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:38:44.697 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T00:38:44.718 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:38:44.718 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T00:38:44.758 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-30T00:38:44.758 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:38:44.758 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T00:38:44.780 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T00:38:44.844 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:38:44.844 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T00:38:44.865 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T00:38:44.932 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-30T00:38:44.933 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:38:44.933 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T00:38:44.952 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T00:38:45.016 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:38:45.016 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T00:38:45.037 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T00:38:45.104 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-30T00:38:45.105 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:38:45.105 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T00:38:45.125 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T00:38:45.192 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:38:45.192 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T00:38:45.213 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T00:38:45.277 INFO:teuthology.run_tasks:Running task ceph... 2024-03-30T00:38:45.369 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-30T00:38:45.369 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /var/log/ceph 2024-03-30T00:38:45.372 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /var/log/ceph 2024-03-30T00:38:45.393 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-30T00:38:45.393 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-30T00:38:45.439 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-30T00:38:45.460 INFO:tasks.ceph:Creating extra log directories... 2024-03-30T00:38:45.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-30T00:38:45.503 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-30T00:38:45.531 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-30T00:38:45.531 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-30T00:38:45.531 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-30T00:38:45.531 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:38:45.531 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-30T00:38:45.573 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-30T00:38:45.666 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:38:45.667 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-30T00:38:45.676 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-30T00:38:45.771 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-30T00:38:45.771 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-30T00:38:45.772 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246', 'branch': 'wip-rishabh-testing-20240327.051042-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': None, 'first_in_suite': False, 'job_id': '7629246', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240327.051042-reef'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240327.051042-reef', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8052, 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'sleep_before_teardown': 0, 'subset': '497/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240327.051042-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi066.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHquc1fXzOkR+Oc1lu0Vh5hvEbwh7hdzwryuzCoCkRz2kM9DTitUMYF4yCUJnGjeYGWwCNtdmZwlb/AZBCdkHzs=', 'smithi156.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF3D/q5JOaLuZ73PSCANMnVcs6eIMU7C3S/m8GabWZjBMBkYYlRzUfN9LKxOP3IvKTZGl+HXiPoGXl3fFjGlbzA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-29_18:05:24', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-30T00:38:45.772 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-30T00:38:45.776 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-30T00:38:45.823 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/run/ceph 2024-03-30T00:38:45.827 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/run/ceph 2024-03-30T00:38:45.884 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:38:45.885 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T00:38:45.897 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T00:38:45.897 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-03-30T00:38:45.949 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T00:38:45.949 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:38:45.949 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 884 Links: 1 2024-03-30T00:38:45.950 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:38:45.950 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-30 00:32:29.172482994 +0000 2024-03-30T00:38:45.950 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-30 00:32:28.908485352 +0000 2024-03-30T00:38:45.950 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-30 00:32:28.908485352 +0000 2024-03-30T00:38:45.950 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-30 00:32:28.908485352 +0000 2024-03-30T00:38:45.950 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T00:38:46.011 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-30T00:38:46.011 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-30T00:38:46.012 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000404698 s, 1.3 MB/s 2024-03-30T00:38:46.014 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T00:38:46.071 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-03-30T00:38:46.121 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T00:38:46.121 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:38:46.121 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-03-30T00:38:46.121 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:38:46.122 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-30 00:32:29.708478207 +0000 2024-03-30T00:38:46.122 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-30 00:32:29.424480744 +0000 2024-03-30T00:38:46.122 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-30 00:32:29.424480744 +0000 2024-03-30T00:38:46.122 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-30 00:32:29.424480744 +0000 2024-03-30T00:38:46.122 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T00:38:46.184 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-30T00:38:46.184 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-30T00:38:46.184 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000639962 s, 800 kB/s 2024-03-30T00:38:46.186 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T00:38:46.239 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-03-30T00:38:46.289 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T00:38:46.289 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:38:46.289 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 916 Links: 1 2024-03-30T00:38:46.289 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:38:46.290 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-30 00:32:30.228473563 +0000 2024-03-30T00:38:46.290 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-30 00:32:29.976475813 +0000 2024-03-30T00:38:46.290 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-30 00:32:29.976475813 +0000 2024-03-30T00:38:46.290 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-30 00:32:29.976475813 +0000 2024-03-30T00:38:46.290 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T00:38:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-30T00:38:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-30T00:38:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000373762 s, 1.4 MB/s 2024-03-30T00:38:46.354 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T00:38:46.407 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-03-30T00:38:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T00:38:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:38:46.457 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 935 Links: 1 2024-03-30T00:38:46.457 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:38:46.458 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-30 00:32:30.740468989 +0000 2024-03-30T00:38:46.458 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-30 00:32:30.484471276 +0000 2024-03-30T00:38:46.458 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-30 00:32:30.484471276 +0000 2024-03-30T00:38:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-30 00:32:30.484471276 +0000 2024-03-30T00:38:46.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T00:38:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-30T00:38:46.520 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-30T00:38:46.520 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.00038847 s, 1.3 MB/s 2024-03-30T00:38:46.522 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T00:38:46.575 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T00:38:46.575 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:38:46.575 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T00:38:46.585 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T00:38:46.585 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2024-03-30T00:38:46.637 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T00:38:46.637 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:38:46.637 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-03-30T00:38:46.637 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:38:46.637 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-30 00:32:26.428072152 +0000 2024-03-30T00:38:46.637 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-30 00:32:26.156074581 +0000 2024-03-30T00:38:46.637 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-30 00:32:26.156074581 +0000 2024-03-30T00:38:46.637 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-30 00:32:26.156074581 +0000 2024-03-30T00:38:46.638 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T00:38:46.698 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-30T00:38:46.698 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-30T00:38:46.698 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000289877 s, 1.8 MB/s 2024-03-30T00:38:46.701 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T00:38:46.755 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2024-03-30T00:38:46.805 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T00:38:46.805 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:38:46.805 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-03-30T00:38:46.805 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:38:46.805 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-30 00:32:26.948067505 +0000 2024-03-30T00:38:46.805 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-30 00:32:26.680069900 +0000 2024-03-30T00:38:46.806 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-30 00:32:26.680069900 +0000 2024-03-30T00:38:46.806 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-30 00:32:26.680069900 +0000 2024-03-30T00:38:46.806 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T00:38:46.871 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-30T00:38:46.871 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-30T00:38:46.871 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000275278 s, 1.9 MB/s 2024-03-30T00:38:46.873 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T00:38:46.931 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2024-03-30T00:38:46.986 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T00:38:46.986 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:38:46.986 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-03-30T00:38:46.986 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:38:46.986 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-30 00:32:27.440063112 +0000 2024-03-30T00:38:46.986 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-30 00:32:27.180065435 +0000 2024-03-30T00:38:46.986 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-30 00:32:27.180065435 +0000 2024-03-30T00:38:46.986 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-30 00:32:27.180065435 +0000 2024-03-30T00:38:46.987 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T00:38:47.050 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-30T00:38:47.050 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-30T00:38:47.050 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000275876 s, 1.9 MB/s 2024-03-30T00:38:47.051 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T00:38:47.103 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2024-03-30T00:38:47.154 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T00:38:47.154 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:38:47.154 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 940 Links: 1 2024-03-30T00:38:47.154 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:38:47.154 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-30 00:32:27.924058790 +0000 2024-03-30T00:38:47.154 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-30 00:32:27.672061041 +0000 2024-03-30T00:38:47.154 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-30 00:32:27.672061041 +0000 2024-03-30T00:38:47.154 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-30 00:32:27.672061041 +0000 2024-03-30T00:38:47.155 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T00:38:47.216 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-30T00:38:47.216 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-30T00:38:47.216 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000272782 s, 1.9 MB/s 2024-03-30T00:38:47.218 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T00:38:47.271 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T00:38:47.271 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-30T00:38:47.271 INFO:tasks.ceph:Generating config... 2024-03-30T00:38:47.272 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-30T00:38:47.273 INFO:tasks.ceph:[client] debug client = 20 2024-03-30T00:38:47.273 INFO:tasks.ceph:[client] debug ms = 1 2024-03-30T00:38:47.273 INFO:tasks.ceph:[client] pid file = 2024-03-30T00:38:47.273 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-30T00:38:47.273 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-30T00:38:47.273 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-30T00:38:47.273 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-30T00:38:47.273 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-30T00:38:47.273 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-30T00:38:47.273 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-30T00:38:47.273 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-30T00:38:47.273 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-30T00:38:47.273 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-30T00:38:47.273 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-30T00:38:47.274 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-30T00:38:47.274 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-30T00:38:47.274 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-30T00:38:47.274 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-30T00:38:47.274 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-30T00:38:47.274 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-30T00:38:47.274 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-30T00:38:47.274 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-30T00:38:47.274 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-30T00:38:47.274 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-30T00:38:47.274 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-30T00:38:47.274 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-30T00:38:47.274 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-30T00:38:47.274 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-30T00:38:47.275 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-30T00:38:47.275 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-30T00:38:47.275 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-30T00:38:47.275 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-30T00:38:47.275 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-30T00:38:47.275 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-30T00:38:47.275 INFO:tasks.ceph:Setting up mon.a... 2024-03-30T00:38:47.275 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-30T00:38:47.323 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.keyring 2024-03-30T00:38:47.332 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-30T00:38:47.382 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T00:38:47.403 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.66'), ('mon.b', '172.21.15.156'), ('mon.c', '[v2:172.21.15.156:3301,v1:172.21.15.156:6790]')] 2024-03-30T00:38:47.404 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.66,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-30T00:38:47.405 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:38:47.405 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-30T00:38:47.452 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.66 --add b 172.21.15.156 --addv c '[v2:172.21.15.156:3301,v1:172.21.15.156:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:38:47.544 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: generated fsid 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:setting min_mon_release = pacific 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:epoch 0 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:fsid 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:last_changed 2024-03-30T00:38:47.540258+0000 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:created 2024-03-30T00:38:47.540258+0000 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:min_mon_release 16 (pacific) 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:election_strategy: 1 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:1: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.b 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:2: [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] mon.c 2024-03-30T00:38:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-30T00:38:47.548 DEBUG:teuthology.orchestra.run.smithi066:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-30T00:38:47.555 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0aec3283-d125-4c04-9eb7-d2c2f49c35a9... 2024-03-30T00:38:47.556 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T00:38:47.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T00:38:47.640 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-03-30T00:38:47.640 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-03-30T00:38:47.640 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T00:38:47.640 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-03-30T00:38:47.640 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.640 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-03-30T00:38:47.640 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.640 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-03-30T00:38:47.640 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.640 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-03-30T00:38:47.641 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-03-30T00:38:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-03-30T00:38:47.643 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-03-30T00:38:47.644 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.644 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-03-30T00:38:47.644 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-03-30T00:38:47.644 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.644 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-03-30T00:38:47.644 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-03-30T00:38:47.644 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-03-30T00:38:47.644 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.644 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-03-30T00:38:47.645 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-03-30T00:38:47.645 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.645 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-03-30T00:38:47.645 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.645 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-03-30T00:38:47.645 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.645 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-03-30T00:38:47.645 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.646 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-03-30T00:38:47.646 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-03-30T00:38:47.646 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.646 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-03-30T00:38:47.646 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-03-30T00:38:47.646 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-03-30T00:38:47.646 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-03-30T00:38:47.646 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 20 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 20 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 10 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-03-30T00:38:47.647 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T00:38:47.648 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T00:38:47.648 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:38:47.648 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:38:47.648 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-03-30T00:38:47.648 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.648 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-03-30T00:38:47.648 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:38:47.648 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T00:38:47.648 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-03-30T00:38:47.648 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T00:38:47.648 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T00:38:47.649 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.650 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-03-30T00:38:47.650 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-03-30T00:38:47.650 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.650 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-03-30T00:38:47.650 INFO:teuthology.orchestra.run.smithi066.stdout: mon op complaint time = 120 2024-03-30T00:38:47.650 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:38:47.650 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-03-30T00:38:47.650 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-03-30T00:38:47.650 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-03-30T00:38:47.650 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-03-30T00:38:47.650 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T00:38:47.651 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T00:38:47.651 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 600 2024-03-30T00:38:47.651 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-03-30T00:38:47.651 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:38:47.651 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-03-30T00:38:47.651 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T00:38:47.651 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T00:38:47.651 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-03-30T00:38:47.651 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-03-30T00:38:47.651 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-03-30T00:38:47.651 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds = 20 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds balancer = 20 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug frag = True 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug scatterstat = True 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout: mds op complaint time = 180 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout: mds verify scatter = True 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T00:38:47.652 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T00:38:47.653 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T00:38:47.653 INFO:teuthology.orchestra.run.smithi156.stdout:[global] 2024-03-30T00:38:47.653 INFO:teuthology.orchestra.run.smithi156.stdout: chdir = "" 2024-03-30T00:38:47.653 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T00:38:47.653 INFO:teuthology.orchestra.run.smithi156.stdout: auth supported = cephx 2024-03-30T00:38:47.653 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: filestore xattr use omap = true 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: mon clock drift allowed = 1.000 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: osd crush chooseleaf type = 0 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: auth debug = true 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on old message = true 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on bug = true 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: mon pg warn max object skew = 0 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.654 INFO:teuthology.orchestra.run.smithi156.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default size = 2 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow primary affinity = true 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow pg remap = true 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on legacy crush tunables = false 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on crush straw calc version zero = false 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on no sortbitwise = false 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on osd down out interval zero = false 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on too few osds = false 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T00:38:47.655 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: mon_allow_pool_size_one = true 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: osd default data pool replay window = 5 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: mon allow pool delete = true 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: mon cluster log file level = debug 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: debug asserts on shutdown = true 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: mon health detail to clog = false 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: mon host = "172.21.15.66,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: fsid = 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.656 INFO:teuthology.orchestra.run.smithi156.stdout:[osd] 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: osd journal size = 100 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub load threshold = 5.0 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub max interval = 600 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: osd mclock profile = high_recovery_ops 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: osd recover clone overlap = true 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: osd recovery max chunk = 1048576 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug shutdown = true 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug op order = true 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug verify stray on activate = true 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.657 INFO:teuthology.orchestra.run.smithi156.stdout: osd open classes on start = true 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug pg log writeout = true 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: osd deep scrub update digest min age = 30 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: osd map max advance = 10 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: journal zero on create = true 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: filestore ondisk finisher threads = 3 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: filestore apply finisher threads = 3 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: bdev debug aio = true 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug misdirected ops = true 2024-03-30T00:38:47.658 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore block size = 96636764160 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore fsck on mount = True 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluefs = 20 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluestore = 20 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: debug osd = 20 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: debug rocksdb = 10 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd full ratio = 0.9 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd nearfull ratio = 0.8 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: osd failsafe full ratio = 0.95 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: osd objectstore = bluestore 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.659 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr] 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgr = 20 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: mgr/telemetry/nag = false 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout:[mon] 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: debug paxos = 20 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2024-03-30T00:38:47.660 INFO:teuthology.orchestra.run.smithi156.stdout: mon data avail warn = 5 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: mon mgr mkfs grace = 240 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd reporter subtree level = osd 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd prime pg temp = true 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.661 INFO:teuthology.orchestra.run.smithi156.stdout: # 1m isn't quite enough 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: mon_down_mkfs_grace = 2m 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_filestore_osds = false 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: mon op complaint time = 120 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout:[client] 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: rgw cache enabled = true 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable ops log = true 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable usage log = true 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: client mount timeout = 600 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = "" 2024-03-30T00:38:47.662 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.a] 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.b] 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.c] 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout:[mds] 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds = 20 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds balancer = 20 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug frag = True 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug scatterstat = True 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout: mds op complaint time = 180 2024-03-30T00:38:47.663 INFO:teuthology.orchestra.run.smithi156.stdout: mds verify scatter = True 2024-03-30T00:38:47.664 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2024-03-30T00:38:47.664 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2024-03-30T00:38:47.664 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2024-03-30T00:38:47.665 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-30T00:38:47.665 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-30T00:38:47.769 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-30T00:38:47.769 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:38:47.769 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-30T00:38:47.778 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:38:47.778 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-30T00:38:47.828 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:38:47.828 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:38:47.829 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-30T00:38:47.829 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T00:38:47.904 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:38:47.904 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:38:47.956 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:38:47.957 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:38:47.957 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-30T00:38:47.957 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T00:38:47.993 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:38:47.993 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:38:48.049 INFO:tasks.ceph:Setting up mon nodes... 2024-03-30T00:38:48.049 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-30T00:38:48.049 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-30T00:38:48.104 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-30T00:38:48.109 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-30T00:38:48.168 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-30T00:38:48.179 INFO:tasks.ceph:Setting up mds nodes... 2024-03-30T00:38:48.179 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-30T00:38:48.236 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-30T00:38:48.241 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-30T00:38:48.261 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-30T00:38:48.363 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-30T00:38:48.368 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-30T00:38:48.388 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-30T00:38:48.446 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-30T00:38:48.451 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-30T00:38:48.471 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-30T00:38:48.567 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-30T00:38:48.571 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-30T00:38:48.590 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-30T00:38:48.590 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-30T00:38:48.633 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-30T00:38:48.650 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-30T00:38:48.734 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-30T00:38:48.751 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-30T00:38:48.793 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-30T00:38:48.810 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-30T00:38:48.893 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-30T00:38:48.910 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-30T00:38:48.910 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-30T00:38:48.910 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-30T00:38:48.930 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T00:38:48.930 INFO:tasks.ceph:role: osd.0 2024-03-30T00:38:48.931 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:38:48.931 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-30T00:38:49.003 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T00:38:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T00:38:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T00:38:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T00:38:49.004 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T00:38:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-03-30T00:38:49.004 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T00:38:49.004 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T00:38:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T00:38:49.004 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T00:38:49.461 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-03-30T00:38:49.464 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-03-30T00:38:49.464 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-30T00:38:49.569 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-30T00:38:49.585 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T00:38:49.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:38:49.586 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-30T00:38:49.651 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T00:38:49.651 INFO:tasks.ceph:role: osd.1 2024-03-30T00:38:49.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:38:49.652 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-30T00:38:49.719 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T00:38:49.719 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T00:38:49.719 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T00:38:49.719 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T00:38:49.719 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T00:38:49.719 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-03-30T00:38:49.720 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T00:38:49.720 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T00:38:49.720 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T00:38:49.720 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T00:38:50.203 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-03-30T00:38:50.206 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-03-30T00:38:50.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-30T00:38:50.359 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-30T00:38:50.376 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T00:38:50.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:38:50.377 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-30T00:38:50.440 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T00:38:50.440 INFO:tasks.ceph:role: osd.2 2024-03-30T00:38:50.440 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:38:50.440 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-30T00:38:50.511 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T00:38:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T00:38:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T00:38:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T00:38:50.512 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T00:38:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-03-30T00:38:50.512 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T00:38:50.512 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T00:38:50.512 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T00:38:50.512 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T00:38:50.993 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-03-30T00:38:50.996 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-03-30T00:38:50.996 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-30T00:38:51.133 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-30T00:38:51.151 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T00:38:51.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:38:51.152 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-30T00:38:51.213 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T00:38:51.213 INFO:tasks.ceph:role: osd.3 2024-03-30T00:38:51.213 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:38:51.213 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-30T00:38:51.288 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T00:38:51.289 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T00:38:51.289 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T00:38:51.289 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T00:38:51.289 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T00:38:51.289 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-03-30T00:38:51.289 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T00:38:51.289 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T00:38:51.289 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T00:38:51.289 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T00:38:51.732 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-03-30T00:38:51.735 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-03-30T00:38:51.735 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-30T00:38:51.831 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-30T00:38:51.848 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T00:38:51.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:38:51.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:38:51.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:51.986+0000 7ff63dfc6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-30T00:38:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:51.986+0000 7ff63dfc6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-30T00:38:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:51.986+0000 7ff63dfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T00:38:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:51.986+0000 7ff63dfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T00:38:51.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:51.986+0000 7ff63dfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T00:38:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:51.998+0000 7ff63dfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-30T00:38:54.972 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-30T00:38:54.993 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:38:55.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:55.082+0000 7fc958c2d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-30T00:38:55.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:55.082+0000 7fc958c2d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-30T00:38:55.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:55.082+0000 7fc958c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T00:38:55.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:55.082+0000 7fc958c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T00:38:55.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:55.082+0000 7fc958c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T00:38:55.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:55.082+0000 7fc958c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-30T00:38:58.059 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-30T00:38:58.081 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:38:58.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:58.174+0000 7efdf9a56640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-30T00:38:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:58.174+0000 7efdf9a56640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-30T00:38:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:58.174+0000 7efdf9a56640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T00:38:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:58.174+0000 7efdf9a56640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T00:38:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:58.174+0000 7efdf9a56640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T00:38:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:38:58.174+0000 7efdf9a56640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-30T00:39:01.169 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-30T00:39:01.189 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:39:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:01.274+0000 7f6b65319640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-30T00:39:01.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:01.274+0000 7f6b65319640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-30T00:39:01.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:01.274+0000 7f6b65319640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T00:39:01.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:01.274+0000 7f6b65319640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T00:39:01.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:01.274+0000 7f6b65319640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T00:39:01.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:01.278+0000 7f6b65319640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-30T00:39:04.278 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-30T00:39:04.299 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-30T00:39:04.321 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T00:39:04.321 INFO:tasks.ceph:role: osd.4 2024-03-30T00:39:04.321 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:39:04.322 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-30T00:39:04.385 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T00:39:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T00:39:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T00:39:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T00:39:04.385 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T00:39:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2024-03-30T00:39:04.385 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T00:39:04.385 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T00:39:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T00:39:04.386 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T00:39:04.409 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2024-03-30T00:39:04.412 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2024-03-30T00:39:04.412 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-30T00:39:04.481 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-30T00:39:04.540 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T00:39:04.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:39:04.541 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-30T00:39:04.603 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T00:39:04.603 INFO:tasks.ceph:role: osd.5 2024-03-30T00:39:04.603 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:39:04.603 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-30T00:39:04.663 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T00:39:04.663 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T00:39:04.663 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T00:39:04.663 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T00:39:04.663 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T00:39:04.663 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2024-03-30T00:39:04.664 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T00:39:04.664 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T00:39:04.664 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T00:39:04.664 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T00:39:04.687 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2024-03-30T00:39:04.689 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2024-03-30T00:39:04.689 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-30T00:39:04.760 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-30T00:39:04.815 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T00:39:04.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:39:04.816 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-30T00:39:04.879 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T00:39:04.879 INFO:tasks.ceph:role: osd.6 2024-03-30T00:39:04.879 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:39:04.879 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-30T00:39:04.938 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T00:39:04.938 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T00:39:04.939 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T00:39:04.939 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T00:39:04.939 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T00:39:04.939 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2024-03-30T00:39:04.939 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T00:39:04.939 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T00:39:04.939 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T00:39:04.939 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T00:39:04.966 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2024-03-30T00:39:04.969 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2024-03-30T00:39:04.969 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-30T00:39:05.042 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-30T00:39:05.100 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T00:39:05.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:39:05.101 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-30T00:39:05.167 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T00:39:05.167 INFO:tasks.ceph:role: osd.7 2024-03-30T00:39:05.167 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:39:05.167 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-30T00:39:05.226 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T00:39:05.226 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T00:39:05.226 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T00:39:05.226 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T00:39:05.226 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T00:39:05.226 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2024-03-30T00:39:05.227 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T00:39:05.227 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T00:39:05.227 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T00:39:05.227 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T00:39:05.251 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2024-03-30T00:39:05.254 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2024-03-30T00:39:05.254 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-30T00:39:05.319 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-30T00:39:05.375 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T00:39:05.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:39:05.376 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:39:05.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:05.482+0000 7f95aca95640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-30T00:39:05.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:05.486+0000 7f95aca95640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-30T00:39:05.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:05.486+0000 7f95aca95640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T00:39:05.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:05.486+0000 7f95aca95640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T00:39:05.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:05.486+0000 7f95aca95640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T00:39:05.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:05.486+0000 7f95aca95640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-30T00:39:08.475 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-30T00:39:08.495 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:39:08.592 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:08.582+0000 7f31726ac640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-30T00:39:08.592 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:08.582+0000 7f31726ac640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-30T00:39:08.592 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:08.582+0000 7f31726ac640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T00:39:08.592 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:08.582+0000 7f31726ac640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T00:39:08.592 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:08.582+0000 7f31726ac640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T00:39:08.593 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:08.586+0000 7f31726ac640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-30T00:39:11.559 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-30T00:39:11.586 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:39:11.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:11.674+0000 7fd28b82b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-30T00:39:11.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:11.674+0000 7fd28b82b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-30T00:39:11.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:11.674+0000 7fd28b82b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T00:39:11.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:11.674+0000 7fd28b82b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T00:39:11.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:11.674+0000 7fd28b82b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T00:39:11.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:11.678+0000 7fd28b82b640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-30T00:39:14.659 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-30T00:39:14.680 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:39:14.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:14.770+0000 7f8ade931640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-30T00:39:14.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:14.774+0000 7f8ade931640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-30T00:39:14.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:14.774+0000 7f8ade931640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T00:39:14.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:14.774+0000 7f8ade931640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T00:39:14.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:14.774+0000 7f8ade931640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T00:39:14.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:14.774+0000 7f8ade931640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-30T00:39:15.774 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:39:15.778 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:39:17.749 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-30T00:39:17.770 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-30T00:39:17.770 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:17.770 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-30T00:39:17.792 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:17.792 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-30T00:39:17.857 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:17.858 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-30T00:39:17.921 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:17.922 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-30T00:39:17.985 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:17.985 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-30T00:39:18.049 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:18.049 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-30T00:39:18.113 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:18.113 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-30T00:39:18.177 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:18.177 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-30T00:39:18.198 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:18.198 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-30T00:39:18.259 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:18.259 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-30T00:39:18.319 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:18.319 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-30T00:39:18.379 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:18.379 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-30T00:39:18.439 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:18.439 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-30T00:39:18.499 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:18.499 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-30T00:39:18.559 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:18.559 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-30T00:39:18.568 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:18.569 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-30T00:39:18.620 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:18.620 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-30T00:39:18.628 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:18.629 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-30T00:39:18.680 INFO:tasks.ceph:Adding keys to all mons... 2024-03-30T00:39:18.680 DEBUG:teuthology.orchestra.run.smithi066:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-30T00:39:18.683 DEBUG:teuthology.orchestra.run.smithi156:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-30T00:39:18.724 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.y] 2024-03-30T00:39:18.724 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAYXwdmMg32BRAAUWBF+QbwRGsopcAoBay8wA== 2024-03-30T00:39:18.724 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-03-30T00:39:18.724 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAYXwdm857MDRAAu2pldF20deZfoSDDXuTF/A== 2024-03-30T00:39:18.724 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-03-30T00:39:18.724 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAYXwdmuiBsFRAA2kez6ylAylKTWsQ1S2cahA== 2024-03-30T00:39:18.724 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.0] 2024-03-30T00:39:18.724 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAbXwdmf3HtOhAAyRcyb0YUJcJca5fjmZ3Frw== 2024-03-30T00:39:18.724 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.1] 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAfXwdml7cdBRAAs639bBHTPyfhfVZMPRMfYA== 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.2] 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAiXwdmeS6KChAAZjLAbKSL7Vswi3qQ4304bA== 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.3] 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAlXwdmeuigEBAAiPIPheowKBnn3UZWwpdr+Q== 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.x] 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAYXwdmEF+xCRAAqRVFguki881+4azP8lsf0g== 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAYXwdmMmNFGhAAlHS6ZpIoJc7tlYazCISiLw== 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAYXwdm/oV3IRAAWn/zG1cLK+Raa2pYPzHd7Q== 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.4] 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQApXwdm+bQNHRAAAb/lufdb7Fe+EFevd23gtA== 2024-03-30T00:39:18.725 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.5] 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAsXwdmReL1IhAAwCs1cS+P6J8VCOYdrnrPQg== 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.6] 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAvXwdmA15vKBAAmpnKrDM/Y0xuBGxdyjrkWA== 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.7] 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAyXwdmrl02LhAA6JfEnugoz6l4nsXl6/vq0Q== 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAYXwdmZjh7JRAA3EFOImM03iOljcwmA7/xWw== 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout:[client.2] 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAYXwdmLG1/KxAAmNIZ9ZXdqPRQjCXvz7wC9A== 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout:[client.1] 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAYXwdmkZv0LhAAgwCqW25+J1VYO31mWwDtLQ== 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout:[client.3] 2024-03-30T00:39:18.726 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQAYXwdmyJ/nNBAAIgiUoUCbTeLEjL1iaPFw1A== 2024-03-30T00:39:18.737 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.y] 2024-03-30T00:39:18.737 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAYXwdmMg32BRAAUWBF+QbwRGsopcAoBay8wA== 2024-03-30T00:39:18.737 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2024-03-30T00:39:18.737 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAYXwdm857MDRAAu2pldF20deZfoSDDXuTF/A== 2024-03-30T00:39:18.737 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2024-03-30T00:39:18.737 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAYXwdmuiBsFRAA2kez6ylAylKTWsQ1S2cahA== 2024-03-30T00:39:18.737 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.0] 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAbXwdmf3HtOhAAyRcyb0YUJcJca5fjmZ3Frw== 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.1] 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAfXwdml7cdBRAAs639bBHTPyfhfVZMPRMfYA== 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.2] 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAiXwdmeS6KChAAZjLAbKSL7Vswi3qQ4304bA== 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.3] 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAlXwdmeuigEBAAiPIPheowKBnn3UZWwpdr+Q== 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.x] 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAYXwdmEF+xCRAAqRVFguki881+4azP8lsf0g== 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAYXwdmMmNFGhAAlHS6ZpIoJc7tlYazCISiLw== 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2024-03-30T00:39:18.738 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAYXwdm/oV3IRAAWn/zG1cLK+Raa2pYPzHd7Q== 2024-03-30T00:39:18.739 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.4] 2024-03-30T00:39:18.739 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQApXwdm+bQNHRAAAb/lufdb7Fe+EFevd23gtA== 2024-03-30T00:39:18.739 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.5] 2024-03-30T00:39:18.739 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAsXwdmReL1IhAAwCs1cS+P6J8VCOYdrnrPQg== 2024-03-30T00:39:18.739 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.6] 2024-03-30T00:39:18.739 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAvXwdmA15vKBAAmpnKrDM/Y0xuBGxdyjrkWA== 2024-03-30T00:39:18.740 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.7] 2024-03-30T00:39:18.740 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAyXwdmrl02LhAA6JfEnugoz6l4nsXl6/vq0Q== 2024-03-30T00:39:18.740 INFO:teuthology.orchestra.run.smithi156.stdout:[client.0] 2024-03-30T00:39:18.740 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAYXwdmZjh7JRAA3EFOImM03iOljcwmA7/xWw== 2024-03-30T00:39:18.740 INFO:teuthology.orchestra.run.smithi156.stdout:[client.2] 2024-03-30T00:39:18.740 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAYXwdmLG1/KxAAmNIZ9ZXdqPRQjCXvz7wC9A== 2024-03-30T00:39:18.740 INFO:teuthology.orchestra.run.smithi156.stdout:[client.1] 2024-03-30T00:39:18.740 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAYXwdmkZv0LhAAgwCqW25+J1VYO31mWwDtLQ== 2024-03-30T00:39:18.740 INFO:teuthology.orchestra.run.smithi156.stdout:[client.3] 2024-03-30T00:39:18.740 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAYXwdmyJ/nNBAAIgiUoUCbTeLEjL1iaPFw1A== 2024-03-30T00:39:18.741 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T00:39:18.776 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T00:39:18.838 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T00:39:18.841 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T00:39:18.898 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T00:39:18.901 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T00:39:18.957 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:18.960 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.012 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.015 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.067 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.070 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.123 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.126 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.178 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T00:39:19.181 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T00:39:19.234 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T00:39:19.237 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T00:39:19.292 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T00:39:19.295 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T00:39:19.351 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.354 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.407 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.409 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.462 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.465 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.520 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T00:39:19.573 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T00:39:19.577 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T00:39:19.632 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T00:39:19.634 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T00:39:19.690 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T00:39:19.693 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T00:39:19.748 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T00:39:19.751 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T00:39:19.808 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-30T00:39:19.808 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-30T00:39:19.828 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T00:39:19.935 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-30T00:39:19.955 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-30T00:39:19.974 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T00:39:20.082 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-30T00:39:20.102 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-30T00:39:20.162 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T00:39:20.270 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-30T00:39:20.290 DEBUG:teuthology.orchestra.run.smithi066:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:39:20.293 DEBUG:teuthology.orchestra.run.smithi156:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-30T00:39:20.339 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-30T00:39:20.339 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-30T00:39:20.340 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-30T00:39:20.342 INFO:tasks.ceph.mon.a:Started 2024-03-30T00:39:20.342 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-30T00:39:20.343 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-30T00:39:20.387 INFO:tasks.ceph.mon.b:Started 2024-03-30T00:39:20.387 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-30T00:39:20.387 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-30T00:39:20.390 INFO:tasks.ceph.mon.c:Started 2024-03-30T00:39:20.390 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-30T00:39:20.390 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-30T00:39:20.391 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-30T00:39:20.394 INFO:tasks.ceph.mgr.y:Started 2024-03-30T00:39:20.394 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-30T00:39:20.394 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-30T00:39:20.397 INFO:tasks.ceph.mgr.x:Started 2024-03-30T00:39:20.397 INFO:tasks.ceph:Setting crush tunables to default 2024-03-30T00:39:20.397 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd crush tunables default 2024-03-30T00:39:20.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f7626800640 1 Processor -- start 2024-03-30T00:39:20.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f7626800640 1 -- start start 2024-03-30T00:39:20.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f7626800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:20.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f7626800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76200cbb90 con 0x7f762015de30 2024-03-30T00:39:20.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f7626800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76200ce1b0 con 0x7f76200cee50 2024-03-30T00:39:20.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f7626800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76200afa10 con 0x7f76200af420 2024-03-30T00:39:20.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f761f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f762015de30 0x7f76200ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57976/0 (socket says 172.21.15.66:57976) 2024-03-30T00:39:20.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 learned_addr learned my addr 172.21.15.66:0/1515954418 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:39:20.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:20.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f761fe00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 msgr2=0x7f76200cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-30T00:39:20.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f761fe00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 msgr2=0x7f76200cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:20.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:20.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.530+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T00:39:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.734+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.734+0000 7f761fe00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 msgr2=0x7f76200cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-30T00:39:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.734+0000 7f761fe00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 msgr2=0x7f76200cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.734+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:20.734+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T00:39:21.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:21.138+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:21.138+0000 7f761fe00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 msgr2=0x7f76200cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-30T00:39:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:21.138+0000 7f761fe00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 msgr2=0x7f76200cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:21.138+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:21.138+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-30T00:39:21.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:21.942+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:21.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:21.942+0000 7f761fe00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 msgr2=0x7f76200cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-30T00:39:21.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:21.942+0000 7f761fe00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 msgr2=0x7f76200cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:21.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:21.942+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:21.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:21.942+0000 7f761fe00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-30T00:39:23.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761d600640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 msgr2=0x7f76200cc040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:39:23.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761d600640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f76200cc040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:23.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761d600640 1 -- 172.21.15.66:0/1515954418 >> v1:172.21.15.156:6789/0 conn(0x7f76200cee50 legacy=0x7f762006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:23.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761d600640 1 -- 172.21.15.66:0/1515954418 >> v1:172.21.15.66:6789/0 conn(0x7f762015de30 legacy=0x7f76200ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:23.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761d600640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:23.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761d600640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:23.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761d600640 1 -- 172.21.15.66:0/1515954418 --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76200afa10 con 0x7f75fc006200 2024-03-30T00:39:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761d600640 1 -- 172.21.15.66:0/1515954418 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc008d70 con 0x7f75fc003740 2024-03-30T00:39:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:23.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f7616a00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 msgr2=0x7f75fc0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-30T00:39:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f7616a00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 msgr2=0x7f75fc0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T00:39:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 msgr2=0x7f75fc005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-30T00:39:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 msgr2=0x7f75fc005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:23.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.534+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T00:39:23.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.738+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.738+0000 7f7616a00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 msgr2=0x7f75fc0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-30T00:39:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.738+0000 7f7616a00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 msgr2=0x7f75fc0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.738+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.738+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T00:39:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.738+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.738+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 msgr2=0x7f75fc005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-30T00:39:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.738+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 msgr2=0x7f75fc005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.738+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:23.738+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T00:39:24.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.142+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:24.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.142+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:24.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.142+0000 7f7616a00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 msgr2=0x7f75fc0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-30T00:39:24.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.142+0000 7f7616a00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 msgr2=0x7f75fc0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:24.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.142+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:24.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.142+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-30T00:39:24.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.142+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 msgr2=0x7f75fc005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-30T00:39:24.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.142+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 msgr2=0x7f75fc005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:24.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.142+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:24.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.142+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-30T00:39:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.946+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.946+0000 7f7616a00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 msgr2=0x7f75fc0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-30T00:39:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.946+0000 7f7616a00640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 msgr2=0x7f75fc0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.946+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.946+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.946+0000 7f7616a00640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-30T00:39:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.946+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 msgr2=0x7f75fc005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-30T00:39:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.946+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 msgr2=0x7f75fc005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T00:39:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.946+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:39:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:24.946+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-30T00:39:25.914 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:25.906+0000 7f0a61649180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T00:39:26.423 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:26.414+0000 7f0a61649180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T00:39:26.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761d600640 1 -- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 msgr2=0x7f75fc005b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:39:26.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761d600640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc003740 0x7f75fc005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:26.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761d600640 1 -- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 msgr2=0x7f75fc0086c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:39:26.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761d600640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.66:3300/0 conn(0x7f75fc006200 0x7f75fc0086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:26.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761d600640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f75fc00b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:26.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761fe00640 1 -- 172.21.15.66:0/1515954418 reap_dead start 2024-03-30T00:39:26.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761d600640 1 -- 172.21.15.66:0/1515954418 --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76200cf430 con 0x7f75fc0066b0 2024-03-30T00:39:26.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761d600640 1 -- 172.21.15.66:0/1515954418 --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75fc005f10 con 0x7f75fc005240 2024-03-30T00:39:26.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761d600640 1 -- 172.21.15.66:0/1515954418 --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76200afa10 con 0x7f76200af420 2024-03-30T00:39:26.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> v2:172.21.15.156:3300/0 conn(0x7f76200af420 0x7f75fc00b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:26.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761ea00640 1 -- 172.21.15.66:0/1515954418 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702396796 0 0) 0x7f76200cf430 con 0x7f75fc0066b0 2024-03-30T00:39:26.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761ea00640 1 -- 172.21.15.66:0/1515954418 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7604003530 con 0x7f75fc0066b0 2024-03-30T00:39:26.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761ea00640 1 -- 172.21.15.66:0/1515954418 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288076688 0 0) 0x7f7618009640 con 0x7f75fc005240 2024-03-30T00:39:26.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761ea00640 1 -- 172.21.15.66:0/1515954418 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76200cf430 con 0x7f75fc005240 2024-03-30T00:39:26.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 >> v1:172.21.15.156:6789/0 conn(0x7f75fc005240 legacy=0x7f75fc0056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:26.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 >> v1:172.21.15.66:6789/0 conn(0x7f75fc0066b0 legacy=0x7f75fc003b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:26.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.530+0000 7f761f400640 1 -- 172.21.15.66:0/1515954418 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc010760 con 0x7f76200af420 2024-03-30T00:39:26.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.534+0000 7f761f400640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76200af420 0x7f75fc00b180 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7610007d30 tx=0x7f761001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9ffd2822caf73cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:26.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.534+0000 7f761ea00640 1 -- 172.21.15.66:0/1515954418 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76100a5dd0 con 0x7f76200af420 2024-03-30T00:39:26.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.534+0000 7f761ea00640 1 -- 172.21.15.66:0/1515954418 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76100a9070 con 0x7f76200af420 2024-03-30T00:39:26.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.534+0000 7f761ea00640 1 -- 172.21.15.66:0/1515954418 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76100af640 con 0x7f76200af420 2024-03-30T00:39:26.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.534+0000 7f7626800640 1 -- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76200af420 msgr2=0x7f75fc00b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:26.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.534+0000 7f7626800640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76200af420 0x7f75fc00b180 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7610007d30 tx=0x7f761001d000 comp rx=0 tx=0).stop 2024-03-30T00:39:26.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.534+0000 7f7626800640 1 -- 172.21.15.66:0/1515954418 shutdown_connections 2024-03-30T00:39:26.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.534+0000 7f7626800640 1 --2- 172.21.15.66:0/1515954418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76200af420 0x7f75fc00b180 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:26.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.534+0000 7f7626800640 1 -- 172.21.15.66:0/1515954418 >> 172.21.15.66:0/1515954418 conn(0x7f76200d0540 msgr2=0x7f76200d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:26.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.534+0000 7f7626800640 1 -- 172.21.15.66:0/1515954418 shutdown_connections 2024-03-30T00:39:26.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.534+0000 7f7626800640 1 -- 172.21.15.66:0/1515954418 wait complete. 2024-03-30T00:39:26.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7626800640 1 Processor -- start 2024-03-30T00:39:26.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7626800640 1 -- start start 2024-03-30T00:39:26.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7626800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc005240 0x7f76200b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:26.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7626800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75fc0066b0 0x7f76200c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:26.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7626800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76200af420 0x7f76200c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:26.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7626800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76200c6f40 con 0x7f76200af420 2024-03-30T00:39:26.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7626800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76200b42a0 con 0x7f75fc0066b0 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7626800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76200b2990 con 0x7f75fc005240 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f761fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76200af420 0x7f76200c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f761fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76200af420 0x7f76200c45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36336/0 (socket says 172.21.15.66:36336) 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f761fe00640 1 -- 172.21.15.66:0/240076323 learned_addr learned my addr 172.21.15.66:0/240076323 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7616a00640 1 --2- 172.21.15.66:0/240076323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75fc0066b0 0x7f76200c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f761f400640 1 --2- 172.21.15.66:0/240076323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc005240 0x7f76200b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f761fe00640 1 -- 172.21.15.66:0/240076323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc005240 msgr2=0x7f76200b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f761fe00640 1 --2- 172.21.15.66:0/240076323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc005240 0x7f76200b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f761fe00640 1 -- 172.21.15.66:0/240076323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75fc0066b0 msgr2=0x7f76200c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f761fe00640 1 --2- 172.21.15.66:0/240076323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75fc0066b0 0x7f76200c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f761fe00640 1 -- 172.21.15.66:0/240076323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761000f040 con 0x7f76200af420 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f761f400640 1 --2- 172.21.15.66:0/240076323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc005240 0x7f76200b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:26.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f761fe00640 1 --2- 172.21.15.66:0/240076323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76200af420 0x7f76200c45b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f75fc008c80 tx=0x7f761800ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:26.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7617e00640 1 -- 172.21.15.66:0/240076323 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618007cd0 con 0x7f76200af420 2024-03-30T00:39:26.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7617e00640 1 -- 172.21.15.66:0/240076323 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761800f040 con 0x7f76200af420 2024-03-30T00:39:26.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7626800640 1 -- 172.21.15.66:0/240076323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76200b39b0 con 0x7f76200af420 2024-03-30T00:39:26.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7617e00640 1 -- 172.21.15.66:0/240076323 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618015650 con 0x7f76200af420 2024-03-30T00:39:26.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.538+0000 7f7626800640 1 -- 172.21.15.66:0/240076323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76200b3f30 con 0x7f76200af420 2024-03-30T00:39:26.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.542+0000 7f7617e00640 1 -- 172.21.15.66:0/240076323 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7618015ab0 con 0x7f76200af420 2024-03-30T00:39:26.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.542+0000 7f7617e00640 1 -- 172.21.15.66:0/240076323 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7618021990 con 0x7f76200af420 2024-03-30T00:39:26.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.542+0000 7f7626800640 1 -- 172.21.15.66:0/240076323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76200cdf60 con 0x7f76200af420 2024-03-30T00:39:26.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.550+0000 7f7617e00640 1 -- 172.21.15.66:0/240076323 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f75fc005f10 con 0x7f76200af420 2024-03-30T00:39:26.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.610+0000 7f7626800640 1 -- 172.21.15.66:0/240076323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7620066c30 con 0x7f76200af420 2024-03-30T00:39:26.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.614+0000 7f7617e00640 1 -- 172.21.15.66:0/240076323 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7618015e50 con 0x7f76200af420 2024-03-30T00:39:26.622 INFO:teuthology.orchestra.run.smithi066.stderr:adjusted tunables profile to default 2024-03-30T00:39:26.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.614+0000 7f7626800640 1 -- 172.21.15.66:0/240076323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76200af420 msgr2=0x7f76200c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:26.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.614+0000 7f7626800640 1 --2- 172.21.15.66:0/240076323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76200af420 0x7f76200c45b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f75fc008c80 tx=0x7f761800ae00 comp rx=0 tx=0).stop 2024-03-30T00:39:26.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.614+0000 7f7626800640 1 -- 172.21.15.66:0/240076323 shutdown_connections 2024-03-30T00:39:26.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.614+0000 7f7626800640 1 --2- 172.21.15.66:0/240076323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76200af420 0x7f76200c45b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:26.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.614+0000 7f7626800640 1 --2- 172.21.15.66:0/240076323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75fc0066b0 0x7f76200c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:26.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.614+0000 7f7626800640 1 --2- 172.21.15.66:0/240076323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75fc005240 0x7f76200b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:26.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.614+0000 7f7626800640 1 -- 172.21.15.66:0/240076323 >> 172.21.15.66:0/240076323 conn(0x7f76200d0540 msgr2=0x7f76200ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:26.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.614+0000 7f7626800640 1 -- 172.21.15.66:0/240076323 shutdown_connections 2024-03-30T00:39:26.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:26.614+0000 7f7626800640 1 -- 172.21.15.66:0/240076323 wait complete. 2024-03-30T00:39:26.641 INFO:tasks.ceph:check_enable_crimson: False 2024-03-30T00:39:26.641 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-30T00:39:26.641 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:26.641 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-30T00:39:26.650 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:26.642+0000 7f0a61649180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T00:39:26.661 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:26.661 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-30T00:39:26.709 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:26.702+0000 7f19d2a38180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T00:39:26.724 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:26.724 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-30T00:39:26.783 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:39:26.783 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-30T00:39:26.843 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:26.844 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-30T00:39:26.864 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:26.864 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-30T00:39:26.924 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:26.924 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-30T00:39:26.991 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:39:26.991 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-30T00:39:27.053 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 7c5db6f7-404b-4949-93f0-9378144404fb 0 2024-03-30T00:39:27.208 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:27.194+0000 7f19d2a38180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T00:39:27.235 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.226+0000 7f517d800640 1 Processor -- start 2024-03-30T00:39:27.236 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.226+0000 7f517d800640 1 -- start start 2024-03-30T00:39:27.237 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f517d800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f517815a8e0 0x7f51780713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.237 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f517d800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 0x7f517815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.237 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f517d800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517806e490 con 0x7f51780ce600 2024-03-30T00:39:27.237 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f517d800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517806e1b0 con 0x7f517815a8e0 2024-03-30T00:39:27.237 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f517d800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51780cdb20 con 0x7f517806b4b0 2024-03-30T00:39:27.237 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f516fe00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f51780ce600 0x7f517806ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51456/0 (socket says 172.21.15.156:51456) 2024-03-30T00:39:27.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f5177400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 0x7f517815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f5176a00640 1 --2- 172.21.15.156:0/2303898336 >> v2:172.21.15.66:3300/0 conn(0x7f517815a8e0 0x7f51780713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f516fe00640 1 -- 172.21.15.156:0/2303898336 learned_addr learned my addr 172.21.15.156:0/2303898336 (peer_addr_for_me v1:172.21.15.156:0/0) 2024-03-30T00:39:27.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f5176a00640 1 -- 172.21.15.156:0/2303898336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 msgr2=0x7f517815acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:27.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f5176a00640 1 --2- 172.21.15.156:0/2303898336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 0x7f517815acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f5176a00640 1 -- 172.21.15.156:0/2303898336 >> v1:172.21.15.156:6789/0 conn(0x7f51780ce600 legacy=0x7f517806ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f5176000640 1 -- 172.21.15.156:0/2303898336 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863390721 0 0) 0x7f517806e490 con 0x7f51780ce600 2024-03-30T00:39:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f5176a00640 1 -- 172.21.15.156:0/2303898336 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51780cde00 con 0x7f517815a8e0 2024-03-30T00:39:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.230+0000 7f5177400640 1 --2- 172.21.15.156:0/2303898336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 0x7f517815acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f5176a00640 1 --2- 172.21.15.156:0/2303898336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517815a8e0 0x7f51780713c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f51680097c0 tx=0x7f516801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37320d95916c29a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:27.243 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f5176000640 1 -- 172.21.15.156:0/2303898336 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51680ab650 con 0x7f517815a8e0 2024-03-30T00:39:27.243 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f5176000640 1 -- 172.21.15.156:0/2303898336 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51680ab7b0 con 0x7f517815a8e0 2024-03-30T00:39:27.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f5176000640 1 -- 172.21.15.156:0/2303898336 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51680abb40 con 0x7f517815a8e0 2024-03-30T00:39:27.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f517d800640 1 -- 172.21.15.156:0/2303898336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517815a8e0 msgr2=0x7f51780713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f517d800640 1 --2- 172.21.15.156:0/2303898336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517815a8e0 0x7f51780713c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f51680097c0 tx=0x7f516801c470 comp rx=0 tx=0).stop 2024-03-30T00:39:27.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f517d800640 1 -- 172.21.15.156:0/2303898336 shutdown_connections 2024-03-30T00:39:27.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f517d800640 1 --2- 172.21.15.156:0/2303898336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 0x7f517815acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f517d800640 1 --2- 172.21.15.156:0/2303898336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517815a8e0 0x7f51780713c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f517d800640 1 -- 172.21.15.156:0/2303898336 >> 172.21.15.156:0/2303898336 conn(0x7f51780708a0 msgr2=0x7f5178070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:27.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f517d800640 1 -- 172.21.15.156:0/2303898336 shutdown_connections 2024-03-30T00:39:27.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.234+0000 7f517d800640 1 -- 172.21.15.156:0/2303898336 wait complete. 2024-03-30T00:39:27.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f517d800640 1 Processor -- start 2024-03-30T00:39:27.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f517d800640 1 -- start start 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f517d800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 0x7f517814dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f517d800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51780ce600 0x7f517806c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f517d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517815a8e0 0x7f517806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f517d800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5178147b20 con 0x7f517815a8e0 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f517d800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5178146ea0 con 0x7f51780ce600 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f517d800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5178147180 con 0x7f517806b4b0 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f5176a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 0x7f517814dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f5176a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 0x7f517814dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:60888/0 (socket says 172.21.15.156:60888) 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f516fe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51780ce600 0x7f517806c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f5176a00640 1 -- 172.21.15.156:0/2600742188 learned_addr learned my addr 172.21.15.156:0/2600742188 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f5177400640 1 --2- 172.21.15.156:0/2600742188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517815a8e0 0x7f517806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f5176a00640 1 -- 172.21.15.156:0/2600742188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51780ce600 msgr2=0x7f517806c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f5176a00640 1 --2- 172.21.15.156:0/2600742188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51780ce600 0x7f517806c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f5176a00640 1 -- 172.21.15.156:0/2600742188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517815a8e0 msgr2=0x7f517806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f5176a00640 1 --2- 172.21.15.156:0/2600742188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517815a8e0 0x7f517806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f5176a00640 1 -- 172.21.15.156:0/2600742188 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5170009a70 con 0x7f517806b4b0 2024-03-30T00:39:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f5176a00640 1 --2- 172.21.15.156:0/2600742188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 0x7f517814dd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f51680ad040 tx=0x7f516801acb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f516ea00640 1 -- 172.21.15.156:0/2600742188 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51680bf610 con 0x7f517806b4b0 2024-03-30T00:39:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f516ea00640 1 -- 172.21.15.156:0/2600742188 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51680bf790 con 0x7f517806b4b0 2024-03-30T00:39:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f517d800640 1 -- 172.21.15.156:0/2600742188 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5168009420 con 0x7f517806b4b0 2024-03-30T00:39:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f516ea00640 1 -- 172.21.15.156:0/2600742188 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51680bfb40 con 0x7f517806b4b0 2024-03-30T00:39:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f517d800640 1 -- 172.21.15.156:0/2600742188 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5178147740 con 0x7f517806b4b0 2024-03-30T00:39:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f516ea00640 1 -- 172.21.15.156:0/2600742188 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f51680b5400 con 0x7f517806b4b0 2024-03-30T00:39:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.238+0000 7f516ea00640 1 -- 172.21.15.156:0/2600742188 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f51680c83f0 con 0x7f517806b4b0 2024-03-30T00:39:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.242+0000 7f517d800640 1 -- 172.21.15.156:0/2600742188 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5178067340 con 0x7f517806b4b0 2024-03-30T00:39:27.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.242+0000 7f516ea00640 1 -- 172.21.15.156:0/2600742188 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f51680b5700 con 0x7f517806b4b0 2024-03-30T00:39:27.292 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:27.282+0000 7f0a61649180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T00:39:27.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.310+0000 7f517d800640 1 -- 172.21.15.156:0/2600742188 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7c5db6f7-404b-4949-93f0-9378144404fb", "id": 0} v 0) v1 -- 0x7f517815dfb0 con 0x7f517806b4b0 2024-03-30T00:39:27.323 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.314+0000 7f516ea00640 1 -- 172.21.15.156:0/2600742188 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7c5db6f7-404b-4949-93f0-9378144404fb", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f51680b58a0 con 0x7f517806b4b0 2024-03-30T00:39:27.323 INFO:teuthology.orchestra.run.smithi156.stdout:0 2024-03-30T00:39:27.324 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.314+0000 7f517d800640 1 -- 172.21.15.156:0/2600742188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 msgr2=0x7f517814dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.324 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.314+0000 7f517d800640 1 --2- 172.21.15.156:0/2600742188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 0x7f517814dd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f51680ad040 tx=0x7f516801acb0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.324 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.314+0000 7f517d800640 1 -- 172.21.15.156:0/2600742188 shutdown_connections 2024-03-30T00:39:27.324 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.314+0000 7f517d800640 1 --2- 172.21.15.156:0/2600742188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517815a8e0 0x7f517806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.314+0000 7f517d800640 1 --2- 172.21.15.156:0/2600742188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51780ce600 0x7f517806c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.314+0000 7f517d800640 1 --2- 172.21.15.156:0/2600742188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f517806b4b0 0x7f517814dd70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.314+0000 7f517d800640 1 -- 172.21.15.156:0/2600742188 >> 172.21.15.156:0/2600742188 conn(0x7f51780708a0 msgr2=0x7f517806ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:27.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.318+0000 7f517d800640 1 -- 172.21.15.156:0/2600742188 shutdown_connections 2024-03-30T00:39:27.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.318+0000 7f517d800640 1 -- 172.21.15.156:0/2600742188 wait complete. 2024-03-30T00:39:27.342 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new f6b429b0-c16f-44fb-b35d-ee6774a74577 1 2024-03-30T00:39:27.360 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:27.350+0000 7f0a61649180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T00:39:27.424 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:27.414+0000 7f0a61649180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T00:39:27.435 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:27.426+0000 7f19d2a38180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T00:39:27.477 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcbb0c00640 1 Processor -- start 2024-03-30T00:39:27.477 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcbb0c00640 1 -- start start 2024-03-30T00:39:27.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcbb0c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac15de30 0x7fcbac0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcbb0c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcbac0cee50 0x7fcbac0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcbb0c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac0cbb90 con 0x7fcbac0af420 2024-03-30T00:39:27.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcbb0c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac0ce1b0 con 0x7fcbac0cee50 2024-03-30T00:39:27.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcbb0c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0afa10 con 0x7fcbac15de30 2024-03-30T00:39:27.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcba9e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac15de30 0x7fcbac0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcba9e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac15de30 0x7fcbac0cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:60890/0 (socket says 172.21.15.156:60890) 2024-03-30T00:39:27.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcbaa800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcbac0cee50 0x7fcbac0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcba9e00640 1 -- 172.21.15.156:0/38216867 learned_addr learned my addr 172.21.15.156:0/38216867 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-30T00:39:27.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcba9e00640 1 -- 172.21.15.156:0/38216867 >> v2:172.21.15.66:3300/0 conn(0x7fcbac0cee50 msgr2=0x7fcbac0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:27.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcba9e00640 1 --2- 172.21.15.156:0/38216867 >> v2:172.21.15.66:3300/0 conn(0x7fcbac0cee50 0x7fcbac0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcba9e00640 1 -- 172.21.15.156:0/38216867 >> v1:172.21.15.66:6789/0 conn(0x7fcbac0af420 legacy=0x7fcbac0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.470+0000 7fcba9e00640 1 -- 172.21.15.156:0/38216867 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0cf430 con 0x7fcbac15de30 2024-03-30T00:39:27.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcba9e00640 1 --2- 172.21.15.156:0/38216867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac15de30 0x7fcbac0cc040 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009870 tx=0x7fcb9c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b47527df9d47ab66 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:27.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcba9400640 1 -- 172.21.15.156:0/38216867 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0a7070 con 0x7fcbac15de30 2024-03-30T00:39:27.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcba9400640 1 -- 172.21.15.156:0/38216867 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c0ab430 con 0x7fcbac15de30 2024-03-30T00:39:27.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcba9400640 1 -- 172.21.15.156:0/38216867 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0ab7c0 con 0x7fcbac15de30 2024-03-30T00:39:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcbb0c00640 1 -- 172.21.15.156:0/38216867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac15de30 msgr2=0x7fcbac0cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcbb0c00640 1 --2- 172.21.15.156:0/38216867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac15de30 0x7fcbac0cc040 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009870 tx=0x7fcb9c01c4d0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcbb0c00640 1 -- 172.21.15.156:0/38216867 shutdown_connections 2024-03-30T00:39:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcbb0c00640 1 --2- 172.21.15.156:0/38216867 >> v2:172.21.15.66:3300/0 conn(0x7fcbac0cee50 0x7fcbac0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcbb0c00640 1 --2- 172.21.15.156:0/38216867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac15de30 0x7fcbac0cc040 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcbb0c00640 1 -- 172.21.15.156:0/38216867 >> 172.21.15.156:0/38216867 conn(0x7fcbac0d0540 msgr2=0x7fcbac0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcbb0c00640 1 -- 172.21.15.156:0/38216867 shutdown_connections 2024-03-30T00:39:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcbb0c00640 1 -- 172.21.15.156:0/38216867 wait complete. 2024-03-30T00:39:27.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcbb0c00640 1 Processor -- start 2024-03-30T00:39:27.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.474+0000 7fcbb0c00640 1 -- start start 2024-03-30T00:39:27.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcbb0c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbac0af420 0x7fcbac0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcbb0c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac0cee50 0x7fcbac0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcbb0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbac15de30 0x7fcbac1f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcbb0c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0ae650 con 0x7fcbac15de30 2024-03-30T00:39:27.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcbb0c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac1f8580 con 0x7fcbac0af420 2024-03-30T00:39:27.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcbb0c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0b0e80 con 0x7fcbac0cee50 2024-03-30T00:39:27.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba9e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbac0af420 0x7fcbac0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba9e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbac0af420 0x7fcbac0b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54282/0 (socket says 172.21.15.156:54282) 2024-03-30T00:39:27.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba9e00640 1 -- 172.21.15.156:0/2848563885 learned_addr learned my addr 172.21.15.156:0/2848563885 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-30T00:39:27.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba3e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac0cee50 0x7fcbac0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcbaa800640 1 --2- 172.21.15.156:0/2848563885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbac15de30 0x7fcbac1f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba9e00640 1 -- 172.21.15.156:0/2848563885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac0cee50 msgr2=0x7fcbac0b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba9e00640 1 --2- 172.21.15.156:0/2848563885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac0cee50 0x7fcbac0b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba9e00640 1 -- 172.21.15.156:0/2848563885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbac15de30 msgr2=0x7fcbac1f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba9e00640 1 --2- 172.21.15.156:0/2848563885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbac15de30 0x7fcbac1f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba9e00640 1 -- 172.21.15.156:0/2848563885 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c0094d0 con 0x7fcbac0af420 2024-03-30T00:39:27.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba9e00640 1 --2- 172.21.15.156:0/2848563885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbac0af420 0x7fcbac0b0940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0ad040 tx=0x7fcb9c019270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:27.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba2000640 1 -- 172.21.15.156:0/2848563885 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0b4800 con 0x7fcbac0af420 2024-03-30T00:39:27.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba2000640 1 -- 172.21.15.156:0/2848563885 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c0b4980 con 0x7fcbac0af420 2024-03-30T00:39:27.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcbb0c00640 1 -- 172.21.15.156:0/2848563885 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac1f54b0 con 0x7fcbac0af420 2024-03-30T00:39:27.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba2000640 1 -- 172.21.15.156:0/2848563885 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0b4d30 con 0x7fcbac0af420 2024-03-30T00:39:27.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcbb0c00640 1 -- 172.21.15.156:0/2848563885 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac1f5a70 con 0x7fcbac0af420 2024-03-30T00:39:27.489 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba2000640 1 -- 172.21.15.156:0/2848563885 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcb9c0be710 con 0x7fcbac0af420 2024-03-30T00:39:27.489 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcba2000640 1 -- 172.21.15.156:0/2848563885 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fcb9c0b3380 con 0x7fcbac0af420 2024-03-30T00:39:27.489 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.478+0000 7fcbb0c00640 1 -- 172.21.15.156:0/2848563885 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac0cdf60 con 0x7fcbac0af420 2024-03-30T00:39:27.490 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.482+0000 7fcba2000640 1 -- 172.21.15.156:0/2848563885 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fcb9c0b3540 con 0x7fcbac0af420 2024-03-30T00:39:27.517 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:27.510+0000 7f0a61649180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T00:39:27.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.546+0000 7fcbb0c00640 1 -- 172.21.15.156:0/2848563885 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f6b429b0-c16f-44fb-b35d-ee6774a74577", "id": 1} v 0) v1 -- 0x7fcbac066c30 con 0x7fcbac0af420 2024-03-30T00:39:27.563 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.554+0000 7fcba2000640 1 -- 172.21.15.156:0/2848563885 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f6b429b0-c16f-44fb-b35d-ee6774a74577", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcb9c0ce020 con 0x7fcbac0af420 2024-03-30T00:39:27.564 INFO:teuthology.orchestra.run.smithi156.stdout:1 2024-03-30T00:39:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.558+0000 7fcbb0c00640 1 -- 172.21.15.156:0/2848563885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbac0af420 msgr2=0x7fcbac0b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.558+0000 7fcbb0c00640 1 --2- 172.21.15.156:0/2848563885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbac0af420 0x7fcbac0b0940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0ad040 tx=0x7fcb9c019270 comp rx=0 tx=0).stop 2024-03-30T00:39:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.558+0000 7fcbb0c00640 1 -- 172.21.15.156:0/2848563885 shutdown_connections 2024-03-30T00:39:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.558+0000 7fcbb0c00640 1 --2- 172.21.15.156:0/2848563885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbac15de30 0x7fcbac1f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.558+0000 7fcbb0c00640 1 --2- 172.21.15.156:0/2848563885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac0cee50 0x7fcbac0b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.558+0000 7fcbb0c00640 1 --2- 172.21.15.156:0/2848563885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbac0af420 0x7fcbac0b0940 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.558+0000 7fcbb0c00640 1 -- 172.21.15.156:0/2848563885 >> 172.21.15.156:0/2848563885 conn(0x7fcbac0d0540 msgr2=0x7fcbac06b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.558+0000 7fcbb0c00640 1 -- 172.21.15.156:0/2848563885 shutdown_connections 2024-03-30T00:39:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.558+0000 7fcbb0c00640 1 -- 172.21.15.156:0/2848563885 wait complete. 2024-03-30T00:39:27.581 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:27.574+0000 7f0a61649180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T00:39:27.583 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 17dced93-3dd9-42c1-9cc8-2ce56c7845fb 2 2024-03-30T00:39:27.675 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:27.666+0000 7f0a61649180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T00:39:27.718 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b7200640 1 Processor -- start 2024-03-30T00:39:27.718 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b7200640 1 -- start start 2024-03-30T00:39:27.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f21b00cdf50 0x7f21b00cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b7200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f21b00cc420 0x7f21b00c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b7200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b015ada0 con 0x7f21b0159990 2024-03-30T00:39:27.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b00cd450 con 0x7f21b00cdf50 2024-03-30T00:39:27.730 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b7200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b015a910 con 0x7f21b00cc420 2024-03-30T00:39:27.730 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21afe00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f21b0159990 0x7f21b00cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51466/0 (socket says 172.21.15.156:51466) 2024-03-30T00:39:27.730 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21afe00640 1 -- 172.21.15.156:0/906140571 learned_addr learned my addr 172.21.15.156:0/906140571 (peer_addr_for_me v1:172.21.15.156:0/0) 2024-03-30T00:39:27.730 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b5200640 1 --2- 172.21.15.156:0/906140571 >> v2:172.21.15.156:3300/0 conn(0x7f21b00cc420 0x7f21b00c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.730 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21a7e00640 1 --2- 172.21.15.156:0/906140571 >> v2:172.21.15.66:3300/0 conn(0x7f21b00cdf50 0x7f21b00cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.730 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21af400640 1 -- 172.21.15.156:0/906140571 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096384281 0 0) 0x7f21b015ada0 con 0x7f21b0159990 2024-03-30T00:39:27.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21af400640 1 -- 172.21.15.156:0/906140571 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2194003530 con 0x7f21b0159990 2024-03-30T00:39:27.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b5200640 1 -- 172.21.15.156:0/906140571 >> v2:172.21.15.66:3300/0 conn(0x7f21b00cdf50 msgr2=0x7f21b00cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:27.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b5200640 1 --2- 172.21.15.156:0/906140571 >> v2:172.21.15.66:3300/0 conn(0x7f21b00cdf50 0x7f21b00cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b5200640 1 -- 172.21.15.156:0/906140571 >> v1:172.21.15.156:6789/0 conn(0x7f21b0159990 legacy=0x7f21b00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21b5200640 1 -- 172.21.15.156:0/906140571 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b0159d70 con 0x7f21b00cc420 2024-03-30T00:39:27.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.710+0000 7f21a7e00640 1 --2- 172.21.15.156:0/906140571 >> v2:172.21.15.66:3300/0 conn(0x7f21b00cdf50 0x7f21b00cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:27.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21b5200640 1 --2- 172.21.15.156:0/906140571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b00cc420 0x7f21b00c9f70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f21b00722f0 tx=0x7f21a801c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aefce0fe7c3e05c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:27.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21af400640 1 -- 172.21.15.156:0/906140571 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a80a5850 con 0x7f21b00cc420 2024-03-30T00:39:27.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21af400640 1 -- 172.21.15.156:0/906140571 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a80a59b0 con 0x7f21b00cc420 2024-03-30T00:39:27.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21af400640 1 -- 172.21.15.156:0/906140571 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a80a5d40 con 0x7f21b00cc420 2024-03-30T00:39:27.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21b7200640 1 -- 172.21.15.156:0/906140571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b00cc420 msgr2=0x7f21b00c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21b7200640 1 --2- 172.21.15.156:0/906140571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b00cc420 0x7f21b00c9f70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f21b00722f0 tx=0x7f21a801c8a0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21b7200640 1 -- 172.21.15.156:0/906140571 shutdown_connections 2024-03-30T00:39:27.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21b7200640 1 --2- 172.21.15.156:0/906140571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b00cc420 0x7f21b00c9f70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21b7200640 1 --2- 172.21.15.156:0/906140571 >> v2:172.21.15.66:3300/0 conn(0x7f21b00cdf50 0x7f21b00cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.733 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21b7200640 1 -- 172.21.15.156:0/906140571 >> 172.21.15.156:0/906140571 conn(0x7f21b0098a20 msgr2=0x7f21b00cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:27.733 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21b7200640 1 -- 172.21.15.156:0/906140571 shutdown_connections 2024-03-30T00:39:27.733 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.714+0000 7f21b7200640 1 -- 172.21.15.156:0/906140571 wait complete. 2024-03-30T00:39:27.733 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21b7200640 1 Processor -- start 2024-03-30T00:39:27.733 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21b7200640 1 -- start start 2024-03-30T00:39:27.733 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21b7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21b00cc420 0x7f21b0157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.733 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21b7200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b00cdf50 0x7f21b01f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.733 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21b7200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21b0159990 0x7f21b01f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21b7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b0155e30 con 0x7f21b00cc420 2024-03-30T00:39:27.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21b7200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b00c9110 con 0x7f21b00cdf50 2024-03-30T00:39:27.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21b7200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b0157fd0 con 0x7f21b0159990 2024-03-30T00:39:27.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21a7e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b00cdf50 0x7f21b01f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21b5200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21b0159990 0x7f21b01f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21a7e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b00cdf50 0x7f21b01f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54294/0 (socket says 172.21.15.156:54294) 2024-03-30T00:39:27.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21a7e00640 1 -- 172.21.15.156:0/839844802 learned_addr learned my addr 172.21.15.156:0/839844802 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-30T00:39:27.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21afe00640 1 --2- 172.21.15.156:0/839844802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21b00cc420 0x7f21b0157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21a7e00640 1 -- 172.21.15.156:0/839844802 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21b0159990 msgr2=0x7f21b01f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21a7e00640 1 --2- 172.21.15.156:0/839844802 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21b0159990 0x7f21b01f4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21a7e00640 1 -- 172.21.15.156:0/839844802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21b00cc420 msgr2=0x7f21b0157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21a7e00640 1 --2- 172.21.15.156:0/839844802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21b00cc420 0x7f21b0157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21a7e00640 1 -- 172.21.15.156:0/839844802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219800ba70 con 0x7f21b00cdf50 2024-03-30T00:39:27.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21a7e00640 1 --2- 172.21.15.156:0/839844802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b00cdf50 0x7f21b01f47f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2198010fc0 tx=0x7f219800e490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:27.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21acc00640 1 -- 172.21.15.156:0/839844802 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198016680 con 0x7f21b00cdf50 2024-03-30T00:39:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21acc00640 1 -- 172.21.15.156:0/839844802 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2198011040 con 0x7f21b00cdf50 2024-03-30T00:39:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21acc00640 1 -- 172.21.15.156:0/839844802 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198016a00 con 0x7f21b00cdf50 2024-03-30T00:39:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21b7200640 1 -- 172.21.15.156:0/839844802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a8009a70 con 0x7f21b00cdf50 2024-03-30T00:39:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21b7200640 1 -- 172.21.15.156:0/839844802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b0158830 con 0x7f21b00cdf50 2024-03-30T00:39:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21acc00640 1 -- 172.21.15.156:0/839844802 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2198016b60 con 0x7f21b00cdf50 2024-03-30T00:39:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.718+0000 7f21acc00640 1 -- 172.21.15.156:0/839844802 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f219801fe10 con 0x7f21b00cdf50 2024-03-30T00:39:27.736 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.722+0000 7f21b7200640 1 -- 172.21.15.156:0/839844802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b0066c30 con 0x7f21b00cdf50 2024-03-30T00:39:27.737 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.722+0000 7f21acc00640 1 -- 172.21.15.156:0/839844802 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f219803db50 con 0x7f21b00cdf50 2024-03-30T00:39:27.739 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:27.730+0000 7f0a61649180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T00:39:27.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.786+0000 7f21b7200640 1 -- 172.21.15.156:0/839844802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "17dced93-3dd9-42c1-9cc8-2ce56c7845fb", "id": 2} v 0) v1 -- 0x7f21b00712f0 con 0x7f21b00cdf50 2024-03-30T00:39:27.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.794+0000 7f21acc00640 1 -- 172.21.15.156:0/839844802 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "17dced93-3dd9-42c1-9cc8-2ce56c7845fb", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f219802c030 con 0x7f21b00cdf50 2024-03-30T00:39:27.804 INFO:teuthology.orchestra.run.smithi156.stdout:2 2024-03-30T00:39:27.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.798+0000 7f21b7200640 1 -- 172.21.15.156:0/839844802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b00cdf50 msgr2=0x7f21b01f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.798+0000 7f21b7200640 1 --2- 172.21.15.156:0/839844802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b00cdf50 0x7f21b01f47f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2198010fc0 tx=0x7f219800e490 comp rx=0 tx=0).stop 2024-03-30T00:39:27.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.798+0000 7f21b7200640 1 -- 172.21.15.156:0/839844802 shutdown_connections 2024-03-30T00:39:27.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.798+0000 7f21b7200640 1 --2- 172.21.15.156:0/839844802 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21b0159990 0x7f21b01f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.798+0000 7f21b7200640 1 --2- 172.21.15.156:0/839844802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21b00cdf50 0x7f21b01f47f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.798+0000 7f21b7200640 1 --2- 172.21.15.156:0/839844802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21b00cc420 0x7f21b0157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.798+0000 7f21b7200640 1 -- 172.21.15.156:0/839844802 >> 172.21.15.156:0/839844802 conn(0x7f21b0098a20 msgr2=0x7f21b00b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:27.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.798+0000 7f21b7200640 1 -- 172.21.15.156:0/839844802 shutdown_connections 2024-03-30T00:39:27.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.798+0000 7f21b7200640 1 -- 172.21.15.156:0/839844802 wait complete. 2024-03-30T00:39:27.824 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 38a5cdb5-b81c-42d3-80d3-239be5d4cccc 3 2024-03-30T00:39:27.916 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:27.906+0000 7f0a61649180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T00:39:27.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.950+0000 7f132aa00640 1 Processor -- start 2024-03-30T00:39:27.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.950+0000 7f132aa00640 1 -- start start 2024-03-30T00:39:27.961 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f132aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f132415a850 0x7f13240cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.961 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f132aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13240aef10 0x7f13240caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.961 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f132aa00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13240cc450 con 0x7f13240b0230 2024-03-30T00:39:27.961 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f132aa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13240cce20 con 0x7f132415a850 2024-03-30T00:39:27.961 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f132aa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132415ac30 con 0x7f13240aef10 2024-03-30T00:39:27.962 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f1323400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13240aef10 0x7f13240caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.962 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f1328a00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f13240b0230 0x7f13240ac310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51476/0 (socket says 172.21.15.156:51476) 2024-03-30T00:39:27.962 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f1328a00640 1 -- 172.21.15.156:0/1430780202 learned_addr learned my addr 172.21.15.156:0/1430780202 (peer_addr_for_me v1:172.21.15.156:0/0) 2024-03-30T00:39:27.962 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f1323e00640 1 --2- 172.21.15.156:0/1430780202 >> v2:172.21.15.66:3300/0 conn(0x7f132415a850 0x7f13240cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.962 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f1322a00640 1 -- 172.21.15.156:0/1430780202 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602519042 0 0) 0x7f13240cc450 con 0x7f13240b0230 2024-03-30T00:39:27.962 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f1323400640 1 -- 172.21.15.156:0/1430780202 >> v2:172.21.15.66:3300/0 conn(0x7f132415a850 msgr2=0x7f13240cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:27.962 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f1323400640 1 --2- 172.21.15.156:0/1430780202 >> v2:172.21.15.66:3300/0 conn(0x7f132415a850 0x7f13240cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.962 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f1323400640 1 -- 172.21.15.156:0/1430780202 >> v1:172.21.15.156:6789/0 conn(0x7f13240b0230 legacy=0x7f13240ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.962 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.954+0000 7f1323400640 1 -- 172.21.15.156:0/1430780202 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13240cb840 con 0x7f13240aef10 2024-03-30T00:39:27.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f1323400640 1 --2- 172.21.15.156:0/1430780202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13240aef10 0x7f13240caff0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f13100097c0 tx=0x7f131001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94d2070efe551b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:27.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f1322a00640 1 -- 172.21.15.156:0/1430780202 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100a6040 con 0x7f13240aef10 2024-03-30T00:39:27.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f1322a00640 1 -- 172.21.15.156:0/1430780202 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13100aa600 con 0x7f13240aef10 2024-03-30T00:39:27.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f1322a00640 1 -- 172.21.15.156:0/1430780202 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100aa990 con 0x7f13240aef10 2024-03-30T00:39:27.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f132aa00640 1 -- 172.21.15.156:0/1430780202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13240aef10 msgr2=0x7f13240caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f132aa00640 1 --2- 172.21.15.156:0/1430780202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13240aef10 0x7f13240caff0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f13100097c0 tx=0x7f131001c4a0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f132aa00640 1 -- 172.21.15.156:0/1430780202 shutdown_connections 2024-03-30T00:39:27.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f132aa00640 1 --2- 172.21.15.156:0/1430780202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13240aef10 0x7f13240caff0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f132aa00640 1 --2- 172.21.15.156:0/1430780202 >> v2:172.21.15.66:3300/0 conn(0x7f132415a850 0x7f13240cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f132aa00640 1 -- 172.21.15.156:0/1430780202 >> 172.21.15.156:0/1430780202 conn(0x7f1324098a20 msgr2=0x7f13240b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:27.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f132aa00640 1 -- 172.21.15.156:0/1430780202 shutdown_connections 2024-03-30T00:39:27.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.958+0000 7f132aa00640 1 -- 172.21.15.156:0/1430780202 wait complete. 2024-03-30T00:39:27.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f132aa00640 1 Processor -- start 2024-03-30T00:39:27.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f132aa00640 1 -- start start 2024-03-30T00:39:27.972 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f132aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13240aef10 0x7f13240ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f132aa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13240b0230 0x7f13240c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f132aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f132415a850 0x7f13240c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f132aa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13240c4e30 con 0x7f13240aef10 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f132aa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13240c82c0 con 0x7f13240b0230 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f132aa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13240c95a0 con 0x7f132415a850 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f1328a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f132415a850 0x7f13240c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f1323400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13240b0230 0x7f13240c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f1323400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13240b0230 0x7f13240c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54306/0 (socket says 172.21.15.156:54306) 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f1323400640 1 -- 172.21.15.156:0/2171694563 learned_addr learned my addr 172.21.15.156:0/2171694563 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.962+0000 7f1323e00640 1 --2- 172.21.15.156:0/2171694563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13240aef10 0x7f13240ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f1328a00640 1 -- 172.21.15.156:0/2171694563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13240b0230 msgr2=0x7f13240c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f1328a00640 1 --2- 172.21.15.156:0/2171694563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13240b0230 0x7f13240c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f1328a00640 1 -- 172.21.15.156:0/2171694563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13240aef10 msgr2=0x7f13240ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:27.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f1328a00640 1 --2- 172.21.15.156:0/2171694563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13240aef10 0x7f13240ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:27.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f1328a00640 1 -- 172.21.15.156:0/2171694563 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1310009420 con 0x7f132415a850 2024-03-30T00:39:27.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f1323400640 1 --2- 172.21.15.156:0/2171694563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13240b0230 0x7f13240c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:27.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f1328a00640 1 --2- 172.21.15.156:0/2171694563 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f132415a850 0x7f13240c8a50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f131c00c780 tx=0x7f131c00cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:27.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f131be00640 1 -- 172.21.15.156:0/2171694563 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00ce30 con 0x7f132415a850 2024-03-30T00:39:27.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f131be00640 1 -- 172.21.15.156:0/2171694563 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131c00a640 con 0x7f132415a850 2024-03-30T00:39:27.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f131be00640 1 -- 172.21.15.156:0/2171694563 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00a9f0 con 0x7f132415a850 2024-03-30T00:39:27.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f132aa00640 1 -- 172.21.15.156:0/2171694563 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13240c47e0 con 0x7f132415a850 2024-03-30T00:39:27.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f132aa00640 1 -- 172.21.15.156:0/2171694563 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13240c4420 con 0x7f132415a850 2024-03-30T00:39:27.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f131be00640 1 -- 172.21.15.156:0/2171694563 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f131c0070d0 con 0x7f132415a850 2024-03-30T00:39:27.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f131be00640 1 -- 172.21.15.156:0/2171694563 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f131c007810 con 0x7f132415a850 2024-03-30T00:39:27.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.966+0000 7f132aa00640 1 -- 172.21.15.156:0/2171694563 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13240c4fa0 con 0x7f132415a850 2024-03-30T00:39:27.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:27.970+0000 7f131be00640 1 -- 172.21.15.156:0/2171694563 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f131c018070 con 0x7f132415a850 2024-03-30T00:39:27.989 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:27.982+0000 7f0a61649180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T00:39:28.043 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.034+0000 7f132aa00640 1 -- 172.21.15.156:0/2171694563 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd new", "uuid": "38a5cdb5-b81c-42d3-80d3-239be5d4cccc", "id": 3} v 0) v1 -- 0x7f1324067340 con 0x7f132415a850 2024-03-30T00:39:28.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.042+0000 7f131be00640 1 -- 172.21.15.156:0/2171694563 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "38a5cdb5-b81c-42d3-80d3-239be5d4cccc", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f131c007380 con 0x7f132415a850 2024-03-30T00:39:28.050 INFO:teuthology.orchestra.run.smithi156.stdout:3 2024-03-30T00:39:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.042+0000 7f132aa00640 1 -- 172.21.15.156:0/2171694563 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f132415a850 msgr2=0x7f13240c8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.042+0000 7f132aa00640 1 --2- 172.21.15.156:0/2171694563 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f132415a850 0x7f13240c8a50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f131c00c780 tx=0x7f131c00cc40 comp rx=0 tx=0).stop 2024-03-30T00:39:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.042+0000 7f132aa00640 1 -- 172.21.15.156:0/2171694563 shutdown_connections 2024-03-30T00:39:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.042+0000 7f132aa00640 1 --2- 172.21.15.156:0/2171694563 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f132415a850 0x7f13240c8a50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.042+0000 7f132aa00640 1 --2- 172.21.15.156:0/2171694563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13240b0230 0x7f13240c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.042+0000 7f132aa00640 1 --2- 172.21.15.156:0/2171694563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13240aef10 0x7f13240ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.042+0000 7f132aa00640 1 -- 172.21.15.156:0/2171694563 >> 172.21.15.156:0/2171694563 conn(0x7f1324098a20 msgr2=0x7f1324159640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.042+0000 7f132aa00640 1 -- 172.21.15.156:0/2171694563 shutdown_connections 2024-03-30T00:39:28.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.042+0000 7f132aa00640 1 -- 172.21.15.156:0/2171694563 wait complete. 2024-03-30T00:39:28.070 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 56195e4d-4f62-45b8-9cca-9ab4eccec014 4 2024-03-30T00:39:28.070 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:28.062+0000 7f19d2a38180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T00:39:28.133 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:28.126+0000 7f19d2a38180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T00:39:28.196 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:28.186+0000 7f19d2a38180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T00:39:28.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.194+0000 7f47d8200640 1 Processor -- start 2024-03-30T00:39:28.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.194+0000 7f47d8200640 1 -- start start 2024-03-30T00:39:28.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d8200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d00ce560 0x7f47d00b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d8200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f47d006b4b0 0x7f47d00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d8200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d015a570 con 0x7f47d01570e0 2024-03-30T00:39:28.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d8200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d00ce9f0 con 0x7f47d006b4b0 2024-03-30T00:39:28.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d8200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d0159c40 con 0x7f47d00ce560 2024-03-30T00:39:28.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d4e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d00ce560 0x7f47d00b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d5800640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f47d01570e0 0x7f47d006ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51492/0 (socket says 172.21.15.156:51492) 2024-03-30T00:39:28.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d5800640 1 -- 172.21.15.156:0/4105328574 learned_addr learned my addr 172.21.15.156:0/4105328574 (peer_addr_for_me v1:172.21.15.156:0/0) 2024-03-30T00:39:28.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d6200640 1 --2- 172.21.15.156:0/4105328574 >> v2:172.21.15.156:3300/0 conn(0x7f47d006b4b0 0x7f47d00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d6200640 1 -- 172.21.15.156:0/4105328574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d00ce560 msgr2=0x7f47d00b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:28.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d6200640 1 --2- 172.21.15.156:0/4105328574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d00ce560 0x7f47d00b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d6200640 1 -- 172.21.15.156:0/4105328574 >> v1:172.21.15.156:6789/0 conn(0x7f47d01570e0 legacy=0x7f47d006ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d6200640 1 -- 172.21.15.156:0/4105328574 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d0159f20 con 0x7f47d006b4b0 2024-03-30T00:39:28.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.198+0000 7f47d4e00640 1 --2- 172.21.15.156:0/4105328574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d00ce560 0x7f47d00b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:28.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47d6200640 1 --2- 172.21.15.156:0/4105328574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d006b4b0 0x7f47d00ccad0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009d90 tx=0x7f47c801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7165112e7f037f80 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:28.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47cfe00640 1 -- 172.21.15.156:0/4105328574 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c80a59a0 con 0x7f47d006b4b0 2024-03-30T00:39:28.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47cfe00640 1 -- 172.21.15.156:0/4105328574 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c80a5b00 con 0x7f47d006b4b0 2024-03-30T00:39:28.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47cfe00640 1 -- 172.21.15.156:0/4105328574 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c80a5e90 con 0x7f47d006b4b0 2024-03-30T00:39:28.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47d8200640 1 -- 172.21.15.156:0/4105328574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d006b4b0 msgr2=0x7f47d00ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47d8200640 1 --2- 172.21.15.156:0/4105328574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d006b4b0 0x7f47d00ccad0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009d90 tx=0x7f47c801c940 comp rx=0 tx=0).stop 2024-03-30T00:39:28.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47d8200640 1 -- 172.21.15.156:0/4105328574 shutdown_connections 2024-03-30T00:39:28.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47d8200640 1 --2- 172.21.15.156:0/4105328574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d006b4b0 0x7f47d00ccad0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47d8200640 1 --2- 172.21.15.156:0/4105328574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d00ce560 0x7f47d00b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47d8200640 1 -- 172.21.15.156:0/4105328574 >> 172.21.15.156:0/4105328574 conn(0x7f47d0070860 msgr2=0x7f47d0070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:28.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47d8200640 1 -- 172.21.15.156:0/4105328574 shutdown_connections 2024-03-30T00:39:28.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47d8200640 1 -- 172.21.15.156:0/4105328574 wait complete. 2024-03-30T00:39:28.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47d8200640 1 Processor -- start 2024-03-30T00:39:28.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.202+0000 7f47d8200640 1 -- start start 2024-03-30T00:39:28.214 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d8200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d006b4b0 0x7f47d014f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d8200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d01570e0 0x7f47d014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d8200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47d014d190 0x7f47d015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d8200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d014f590 con 0x7f47d014d190 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d8200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d014da30 con 0x7f47d01570e0 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d8200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d014d750 con 0x7f47d006b4b0 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d4e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d01570e0 0x7f47d014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d5800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d006b4b0 0x7f47d014f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d5800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d006b4b0 0x7f47d014f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:60946/0 (socket says 172.21.15.156:60946) 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d5800640 1 -- 172.21.15.156:0/3354557931 learned_addr learned my addr 172.21.15.156:0/3354557931 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d4e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d01570e0 0x7f47d014cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54336/0 (socket says 172.21.15.156:54336) 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d6200640 1 --2- 172.21.15.156:0/3354557931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47d014d190 0x7f47d015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d5800640 1 -- 172.21.15.156:0/3354557931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d01570e0 msgr2=0x7f47d014cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d5800640 1 --2- 172.21.15.156:0/3354557931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d01570e0 0x7f47d014cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.215 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d5800640 1 -- 172.21.15.156:0/3354557931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47d014d190 msgr2=0x7f47d015bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d5800640 1 --2- 172.21.15.156:0/3354557931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47d014d190 0x7f47d015bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d5800640 1 -- 172.21.15.156:0/3354557931 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc00b6c0 con 0x7f47d006b4b0 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d6200640 1 --2- 172.21.15.156:0/3354557931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47d014d190 0x7f47d015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d4e00640 1 --2- 172.21.15.156:0/3354557931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d01570e0 0x7f47d014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d5800640 1 --2- 172.21.15.156:0/3354557931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d006b4b0 0x7f47d014f050 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f47c400c400 tx=0x7f47c400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47cd600640 1 -- 172.21.15.156:0/3354557931 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4005cf0 con 0x7f47d006b4b0 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47cd600640 1 -- 172.21.15.156:0/3354557931 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c4005e50 con 0x7f47d006b4b0 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47cd600640 1 -- 172.21.15.156:0/3354557931 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40072d0 con 0x7f47d006b4b0 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d8200640 1 -- 172.21.15.156:0/3354557931 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c8009a70 con 0x7f47d006b4b0 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d8200640 1 -- 172.21.15.156:0/3354557931 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d015b6f0 con 0x7f47d006b4b0 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47cd600640 1 -- 172.21.15.156:0/3354557931 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f47c401c440 con 0x7f47d006b4b0 2024-03-30T00:39:28.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47cd600640 1 -- 172.21.15.156:0/3354557931 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f47c401cc00 con 0x7f47d006b4b0 2024-03-30T00:39:28.217 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.206+0000 7f47d8200640 1 -- 172.21.15.156:0/3354557931 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d015b8e0 con 0x7f47d006b4b0 2024-03-30T00:39:28.217 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.210+0000 7f47cd600640 1 -- 172.21.15.156:0/3354557931 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f47c4021060 con 0x7f47d006b4b0 2024-03-30T00:39:28.283 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.274+0000 7f47d8200640 1 -- 172.21.15.156:0/3354557931 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd new", "uuid": "56195e4d-4f62-45b8-9cca-9ab4eccec014", "id": 4} v 0) v1 -- 0x7f47d0067340 con 0x7f47d006b4b0 2024-03-30T00:39:28.288 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:28.278+0000 7f19d2a38180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T00:39:28.291 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.282+0000 7f47cd600640 1 -- 172.21.15.156:0/3354557931 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "56195e4d-4f62-45b8-9cca-9ab4eccec014", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f47c4017070 con 0x7f47d006b4b0 2024-03-30T00:39:28.291 INFO:teuthology.orchestra.run.smithi156.stdout:4 2024-03-30T00:39:28.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.282+0000 7f47d8200640 1 -- 172.21.15.156:0/3354557931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d006b4b0 msgr2=0x7f47d014f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.282+0000 7f47d8200640 1 --2- 172.21.15.156:0/3354557931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d006b4b0 0x7f47d014f050 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f47c400c400 tx=0x7f47c400c8c0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.282+0000 7f47d8200640 1 -- 172.21.15.156:0/3354557931 shutdown_connections 2024-03-30T00:39:28.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.282+0000 7f47d8200640 1 --2- 172.21.15.156:0/3354557931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47d014d190 0x7f47d015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.282+0000 7f47d8200640 1 --2- 172.21.15.156:0/3354557931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47d01570e0 0x7f47d014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.282+0000 7f47d8200640 1 --2- 172.21.15.156:0/3354557931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47d006b4b0 0x7f47d014f050 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.282+0000 7f47d8200640 1 -- 172.21.15.156:0/3354557931 >> 172.21.15.156:0/3354557931 conn(0x7f47d0070860 msgr2=0x7f47d0157750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:28.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.282+0000 7f47d8200640 1 -- 172.21.15.156:0/3354557931 shutdown_connections 2024-03-30T00:39:28.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.282+0000 7f47d8200640 1 -- 172.21.15.156:0/3354557931 wait complete. 2024-03-30T00:39:28.310 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new e2b93e58-a16b-409a-9980-3197387fe368 5 2024-03-30T00:39:28.353 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:28.346+0000 7f19d2a38180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T00:39:28.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73df600640 1 Processor -- start 2024-03-30T00:39:28.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73df600640 1 -- start start 2024-03-30T00:39:28.447 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f19d2a38180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T00:39:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73df600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80cee50 0x7f73d80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73df600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f73d80af420 0x7f73d80d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73df600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d80cbb90 con 0x7f73d815de30 2024-03-30T00:39:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73df600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d80ce1b0 con 0x7f73d80af420 2024-03-30T00:39:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73df600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80afa10 con 0x7f73d80cee50 2024-03-30T00:39:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73d7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80cee50 0x7f73d80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73d7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80cee50 0x7f73d80cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:60948/0 (socket says 172.21.15.156:60948) 2024-03-30T00:39:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73d7e00640 1 -- 172.21.15.156:0/1245698233 learned_addr learned my addr 172.21.15.156:0/1245698233 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-30T00:39:28.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73dd600640 1 --2- 172.21.15.156:0/1245698233 >> v2:172.21.15.66:3300/0 conn(0x7f73d80af420 0x7f73d80d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73d7e00640 1 -- 172.21.15.156:0/1245698233 >> v2:172.21.15.66:3300/0 conn(0x7f73d80af420 msgr2=0x7f73d80d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:28.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73d7e00640 1 --2- 172.21.15.156:0/1245698233 >> v2:172.21.15.66:3300/0 conn(0x7f73d80af420 0x7f73d80d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73d7e00640 1 -- 172.21.15.156:0/1245698233 >> v1:172.21.15.66:6789/0 conn(0x7f73d815de30 legacy=0x7f73d80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.438+0000 7f73d7e00640 1 -- 172.21.15.156:0/1245698233 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d80cf430 con 0x7f73d80cee50 2024-03-30T00:39:28.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73d7e00640 1 --2- 172.21.15.156:0/1245698233 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80cee50 0x7f73d80cc040 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009870 tx=0x7f73c401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2151e7698b188679 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:28.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73d7400640 1 -- 172.21.15.156:0/1245698233 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c40a7070 con 0x7f73d80cee50 2024-03-30T00:39:28.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73d7400640 1 -- 172.21.15.156:0/1245698233 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c40ab430 con 0x7f73d80cee50 2024-03-30T00:39:28.452 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73d7400640 1 -- 172.21.15.156:0/1245698233 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c40ab7c0 con 0x7f73d80cee50 2024-03-30T00:39:28.452 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73df600640 1 -- 172.21.15.156:0/1245698233 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80cee50 msgr2=0x7f73d80cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.452 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73df600640 1 --2- 172.21.15.156:0/1245698233 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80cee50 0x7f73d80cc040 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009870 tx=0x7f73c401c4d0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.452 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73df600640 1 -- 172.21.15.156:0/1245698233 shutdown_connections 2024-03-30T00:39:28.452 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73df600640 1 --2- 172.21.15.156:0/1245698233 >> v2:172.21.15.66:3300/0 conn(0x7f73d80af420 0x7f73d80d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.452 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73df600640 1 --2- 172.21.15.156:0/1245698233 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80cee50 0x7f73d80cc040 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.453 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73df600640 1 -- 172.21.15.156:0/1245698233 >> 172.21.15.156:0/1245698233 conn(0x7f73d80d0540 msgr2=0x7f73d80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:28.453 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73df600640 1 -- 172.21.15.156:0/1245698233 shutdown_connections 2024-03-30T00:39:28.453 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.442+0000 7f73df600640 1 -- 172.21.15.156:0/1245698233 wait complete. 2024-03-30T00:39:28.454 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73df600640 1 Processor -- start 2024-03-30T00:39:28.454 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73df600640 1 -- start start 2024-03-30T00:39:28.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73df600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80af420 0x7f73d814e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73df600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d80cee50 0x7f73d8162830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73df600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73d815de30 0x7f73d8151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73df600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d806b070 con 0x7f73d80cee50 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73df600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d806b2c0 con 0x7f73d815de30 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73df600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d806b5a0 con 0x7f73d80af420 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73dcc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80af420 0x7f73d814e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73dcc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80af420 0x7f73d814e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:60950/0 (socket says 172.21.15.156:60950) 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73dcc00640 1 -- 172.21.15.156:0/317173356 learned_addr learned my addr 172.21.15.156:0/317173356 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73dd600640 1 --2- 172.21.15.156:0/317173356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73d815de30 0x7f73d8151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73d7e00640 1 --2- 172.21.15.156:0/317173356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d80cee50 0x7f73d8162830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73dcc00640 1 -- 172.21.15.156:0/317173356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73d815de30 msgr2=0x7f73d8151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73dcc00640 1 --2- 172.21.15.156:0/317173356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73d815de30 0x7f73d8151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73dcc00640 1 -- 172.21.15.156:0/317173356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d80cee50 msgr2=0x7f73d8162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73dcc00640 1 --2- 172.21.15.156:0/317173356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d80cee50 0x7f73d8162830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73dcc00640 1 -- 172.21.15.156:0/317173356 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c40094d0 con 0x7f73d80af420 2024-03-30T00:39:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73d7e00640 1 --2- 172.21.15.156:0/317173356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d80cee50 0x7f73d8162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73dcc00640 1 --2- 172.21.15.156:0/317173356 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80af420 0x7f73d814e280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c500 tx=0x7f73cc00c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73d4c00640 1 -- 172.21.15.156:0/317173356 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc005220 con 0x7f73d80af420 2024-03-30T00:39:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73d4c00640 1 -- 172.21.15.156:0/317173356 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc005380 con 0x7f73d80af420 2024-03-30T00:39:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73d4c00640 1 -- 172.21.15.156:0/317173356 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc007270 con 0x7f73d80af420 2024-03-30T00:39:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.446+0000 7f73df600640 1 -- 172.21.15.156:0/317173356 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d806b880 con 0x7f73d80af420 2024-03-30T00:39:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.450+0000 7f73df600640 1 -- 172.21.15.156:0/317173356 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d806be40 con 0x7f73d80af420 2024-03-30T00:39:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.450+0000 7f73d4c00640 1 -- 172.21.15.156:0/317173356 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f73cc0067e0 con 0x7f73d80af420 2024-03-30T00:39:28.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.450+0000 7f73d4c00640 1 -- 172.21.15.156:0/317173356 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f73cc006a90 con 0x7f73d80af420 2024-03-30T00:39:28.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.450+0000 7f73df600640 1 -- 172.21.15.156:0/317173356 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d80cdf60 con 0x7f73d80af420 2024-03-30T00:39:28.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.450+0000 7f73d4c00640 1 -- 172.21.15.156:0/317173356 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f73cc016020 con 0x7f73d80af420 2024-03-30T00:39:28.511 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:28.502+0000 7f19d2a38180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T00:39:28.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.518+0000 7f73df600640 1 -- 172.21.15.156:0/317173356 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e2b93e58-a16b-409a-9980-3197387fe368", "id": 5} v 0) v1 -- 0x7f73d8066c30 con 0x7f73d80af420 2024-03-30T00:39:28.531 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.522+0000 7f73d4c00640 1 -- 172.21.15.156:0/317173356 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e2b93e58-a16b-409a-9980-3197387fe368", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f73cc02a0b0 con 0x7f73d80af420 2024-03-30T00:39:28.531 INFO:teuthology.orchestra.run.smithi156.stdout:5 2024-03-30T00:39:28.532 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.522+0000 7f73df600640 1 -- 172.21.15.156:0/317173356 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80af420 msgr2=0x7f73d814e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.532 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.522+0000 7f73df600640 1 --2- 172.21.15.156:0/317173356 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80af420 0x7f73d814e280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c500 tx=0x7f73cc00c9c0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.532 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.522+0000 7f73df600640 1 -- 172.21.15.156:0/317173356 shutdown_connections 2024-03-30T00:39:28.533 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.522+0000 7f73df600640 1 --2- 172.21.15.156:0/317173356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73d815de30 0x7f73d8151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.533 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.522+0000 7f73df600640 1 --2- 172.21.15.156:0/317173356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d80cee50 0x7f73d8162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.533 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.522+0000 7f73df600640 1 --2- 172.21.15.156:0/317173356 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73d80af420 0x7f73d814e280 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.533 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.522+0000 7f73df600640 1 -- 172.21.15.156:0/317173356 >> 172.21.15.156:0/317173356 conn(0x7f73d80d0540 msgr2=0x7f73d80d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:28.533 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.522+0000 7f73df600640 1 -- 172.21.15.156:0/317173356 shutdown_connections 2024-03-30T00:39:28.533 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.522+0000 7f73df600640 1 -- 172.21.15.156:0/317173356 wait complete. 2024-03-30T00:39:28.550 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new b930ed94-76bd-432e-8316-5114a343f8ce 6 2024-03-30T00:39:28.633 INFO:tasks.ceph.mgr.y.smithi066.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-30T00:39:28.633 INFO:tasks.ceph.mgr.y.smithi066.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-30T00:39:28.633 INFO:tasks.ceph.mgr.y.smithi066.stderr: from numpy import show_config as show_numpy_config 2024-03-30T00:39:28.673 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:28.666+0000 7f0a61649180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T00:39:28.686 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:28.678+0000 7f19d2a38180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T00:39:28.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33cb000640 1 Processor -- start 2024-03-30T00:39:28.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33cb000640 1 -- start start 2024-03-30T00:39:28.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33cb000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c40adb00 0x7f33c40ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33cb000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f33c406a4a0 0x7f33c415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33cb000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c415ada0 con 0x7f33c415e060 2024-03-30T00:39:28.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33cb000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c40cdf80 con 0x7f33c406a4a0 2024-03-30T00:39:28.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33cb000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40adee0 con 0x7f33c40adb00 2024-03-30T00:39:28.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33c9200640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f33c415e060 0x7f33c4069d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51504/0 (socket says 172.21.15.156:51504) 2024-03-30T00:39:28.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33c9200640 1 -- 172.21.15.156:0/1953699004 learned_addr learned my addr 172.21.15.156:0/1953699004 (peer_addr_for_me v1:172.21.15.156:0/0) 2024-03-30T00:39:28.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33c9c00640 1 --2- 172.21.15.156:0/1953699004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c40adb00 0x7f33c40ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33ca600640 1 --2- 172.21.15.156:0/1953699004 >> v2:172.21.15.66:3300/0 conn(0x7f33c406a4a0 0x7f33c415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33c3e00640 1 -- 172.21.15.156:0/1953699004 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266711868 0 0) 0x7f33c415ada0 con 0x7f33c415e060 2024-03-30T00:39:28.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33c3e00640 1 -- 172.21.15.156:0/1953699004 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33ac003530 con 0x7f33c415e060 2024-03-30T00:39:28.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33c9c00640 1 -- 172.21.15.156:0/1953699004 >> v2:172.21.15.66:3300/0 conn(0x7f33c406a4a0 msgr2=0x7f33c415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:28.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33c9c00640 1 --2- 172.21.15.156:0/1953699004 >> v2:172.21.15.66:3300/0 conn(0x7f33c406a4a0 0x7f33c415e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33c9c00640 1 -- 172.21.15.156:0/1953699004 >> v1:172.21.15.156:6789/0 conn(0x7f33c415e060 legacy=0x7f33c4069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.722+0000 7f33c9c00640 1 -- 172.21.15.156:0/1953699004 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c415ebd0 con 0x7f33c40adb00 2024-03-30T00:39:28.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33c9c00640 1 --2- 172.21.15.156:0/1953699004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c40adb00 0x7f33c40ce3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f33b80097c0 tx=0x7f33b801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19620647ff8d0480 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:28.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33c3e00640 1 -- 172.21.15.156:0/1953699004 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b80ad640 con 0x7f33c40adb00 2024-03-30T00:39:28.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33c3e00640 1 -- 172.21.15.156:0/1953699004 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b80ad7a0 con 0x7f33c40adb00 2024-03-30T00:39:28.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33c3e00640 1 -- 172.21.15.156:0/1953699004 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b80adb30 con 0x7f33c40adb00 2024-03-30T00:39:28.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33cb000640 1 -- 172.21.15.156:0/1953699004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c40adb00 msgr2=0x7f33c40ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33cb000640 1 --2- 172.21.15.156:0/1953699004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c40adb00 0x7f33c40ce3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f33b80097c0 tx=0x7f33b801c4a0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33cb000640 1 -- 172.21.15.156:0/1953699004 shutdown_connections 2024-03-30T00:39:28.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33cb000640 1 --2- 172.21.15.156:0/1953699004 >> v2:172.21.15.66:3300/0 conn(0x7f33c406a4a0 0x7f33c415e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33cb000640 1 --2- 172.21.15.156:0/1953699004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c40adb00 0x7f33c40ce3a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33cb000640 1 -- 172.21.15.156:0/1953699004 >> 172.21.15.156:0/1953699004 conn(0x7f33c40b0690 msgr2=0x7f33c40b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:28.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33cb000640 1 -- 172.21.15.156:0/1953699004 shutdown_connections 2024-03-30T00:39:28.735 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33cb000640 1 -- 172.21.15.156:0/1953699004 wait complete. 2024-03-30T00:39:28.736 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.726+0000 7f33cb000640 1 Processor -- start 2024-03-30T00:39:28.736 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33cb000640 1 -- start start 2024-03-30T00:39:28.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33cb000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33c406a4a0 0x7f33c4143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33cb000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c40adb00 0x7f33c414c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33cb000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c415e060 0x7f33c40b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33cb000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4143fd0 con 0x7f33c406a4a0 2024-03-30T00:39:28.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33cb000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40b28b0 con 0x7f33c40adb00 2024-03-30T00:39:28.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33cb000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40b1ce0 con 0x7f33c415e060 2024-03-30T00:39:28.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c9200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c40adb00 0x7f33c414c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.739 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33ca600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c415e060 0x7f33c40b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.739 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c9200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c40adb00 0x7f33c414c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54356/0 (socket says 172.21.15.156:54356) 2024-03-30T00:39:28.739 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c9200640 1 -- 172.21.15.156:0/2314723167 learned_addr learned my addr 172.21.15.156:0/2314723167 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-30T00:39:28.739 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c9c00640 1 --2- 172.21.15.156:0/2314723167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33c406a4a0 0x7f33c4143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.739 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c9200640 1 -- 172.21.15.156:0/2314723167 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c415e060 msgr2=0x7f33c40b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.740 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c9200640 1 --2- 172.21.15.156:0/2314723167 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c415e060 0x7f33c40b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c9200640 1 -- 172.21.15.156:0/2314723167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33c406a4a0 msgr2=0x7f33c4143a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c9200640 1 --2- 172.21.15.156:0/2314723167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33c406a4a0 0x7f33c4143a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c9200640 1 -- 172.21.15.156:0/2314723167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b8009420 con 0x7f33c40adb00 2024-03-30T00:39:28.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33ca600640 1 --2- 172.21.15.156:0/2314723167 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c415e060 0x7f33c40b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:28.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c9200640 1 --2- 172.21.15.156:0/2314723167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c40adb00 0x7f33c414c8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c500 tx=0x7f33b000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:28.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c1600640 1 -- 172.21.15.156:0/2314723167 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0005cf0 con 0x7f33c40adb00 2024-03-30T00:39:28.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c1600640 1 -- 172.21.15.156:0/2314723167 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b0005e50 con 0x7f33c40adb00 2024-03-30T00:39:28.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33cb000640 1 -- 172.21.15.156:0/2314723167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c40b1f80 con 0x7f33c40adb00 2024-03-30T00:39:28.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c1600640 1 -- 172.21.15.156:0/2314723167 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b00072f0 con 0x7f33c40adb00 2024-03-30T00:39:28.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33cb000640 1 -- 172.21.15.156:0/2314723167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c40b2540 con 0x7f33c40adb00 2024-03-30T00:39:28.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c1600640 1 -- 172.21.15.156:0/2314723167 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f33b001c440 con 0x7f33c40adb00 2024-03-30T00:39:28.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33c1600640 1 -- 172.21.15.156:0/2314723167 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f33b001cce0 con 0x7f33c40adb00 2024-03-30T00:39:28.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.730+0000 7f33cb000640 1 -- 172.21.15.156:0/2314723167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c4073e00 con 0x7f33c40adb00 2024-03-30T00:39:28.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.734+0000 7f33c1600640 1 -- 172.21.15.156:0/2314723167 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f33b0021060 con 0x7f33c40adb00 2024-03-30T00:39:28.759 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:28.750+0000 7f19d2a38180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T00:39:28.808 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.798+0000 7f33cb000640 1 -- 172.21.15.156:0/2314723167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b930ed94-76bd-432e-8316-5114a343f8ce", "id": 6} v 0) v1 -- 0x7f33c415daa0 con 0x7f33c40adb00 2024-03-30T00:39:28.812 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.802+0000 7f33c1600640 1 -- 172.21.15.156:0/2314723167 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b930ed94-76bd-432e-8316-5114a343f8ce", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f33b0017070 con 0x7f33c40adb00 2024-03-30T00:39:28.812 INFO:teuthology.orchestra.run.smithi156.stdout:6 2024-03-30T00:39:28.813 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.806+0000 7f33cb000640 1 -- 172.21.15.156:0/2314723167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c40adb00 msgr2=0x7f33c414c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.813 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.806+0000 7f33cb000640 1 --2- 172.21.15.156:0/2314723167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c40adb00 0x7f33c414c8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c500 tx=0x7f33b000c9c0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.806+0000 7f33cb000640 1 -- 172.21.15.156:0/2314723167 shutdown_connections 2024-03-30T00:39:28.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.806+0000 7f33cb000640 1 --2- 172.21.15.156:0/2314723167 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33c415e060 0x7f33c40b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.806+0000 7f33cb000640 1 --2- 172.21.15.156:0/2314723167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c40adb00 0x7f33c414c8f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.806+0000 7f33cb000640 1 --2- 172.21.15.156:0/2314723167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33c406a4a0 0x7f33c4143a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.806+0000 7f33cb000640 1 -- 172.21.15.156:0/2314723167 >> 172.21.15.156:0/2314723167 conn(0x7f33c40b0690 msgr2=0x7f33c41598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:28.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.806+0000 7f33cb000640 1 -- 172.21.15.156:0/2314723167 shutdown_connections 2024-03-30T00:39:28.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.806+0000 7f33cb000640 1 -- 172.21.15.156:0/2314723167 wait complete. 2024-03-30T00:39:28.830 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:28.822+0000 7f0a61649180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:39:28.831 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new ba829def-38d8-4dd7-9d66-91dddaa8395b 7 2024-03-30T00:39:28.964 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.954+0000 7fccbe600640 1 Processor -- start 2024-03-30T00:39:28.964 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.954+0000 7fccbe600640 1 -- start start 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccbe600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fccb80cec30 0x7fccb8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccbe600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fccb80cde70 0x7fccb815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccbe600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb815a6e0 con 0x7fccb80cda90 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccbe600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb80b1b80 con 0x7fccb80cec30 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccbe600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb80cd3e0 con 0x7fccb80cde70 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccb7e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fccb80cde70 0x7fccb815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccb7e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fccb80cde70 0x7fccb815a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54372/0 (socket says 172.21.15.156:54372) 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccb7e00640 1 -- 172.21.15.156:0/2646640920 learned_addr learned my addr 172.21.15.156:0/2646640920 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccb6a00640 1 --2- 172.21.15.156:0/2646640920 >> v2:172.21.15.66:3300/0 conn(0x7fccb80cec30 0x7fccb8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccb7e00640 1 -- 172.21.15.156:0/2646640920 >> v2:172.21.15.66:3300/0 conn(0x7fccb80cec30 msgr2=0x7fccb8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccb7e00640 1 --2- 172.21.15.156:0/2646640920 >> v2:172.21.15.66:3300/0 conn(0x7fccb80cec30 0x7fccb8159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccb7e00640 1 -- 172.21.15.156:0/2646640920 >> v1:172.21.15.66:6789/0 conn(0x7fccb80cda90 legacy=0x7fccb80ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.958+0000 7fccb7e00640 1 -- 172.21.15.156:0/2646640920 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb815a990 con 0x7fccb80cde70 2024-03-30T00:39:28.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccb7e00640 1 --2- 172.21.15.156:0/2646640920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccb80cde70 0x7fccb815a1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fccb80adcf0 tx=0x7fccb001c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7fd5e7fb6d6440 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:28.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccb6000640 1 -- 172.21.15.156:0/2646640920 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb00a5820 con 0x7fccb80cde70 2024-03-30T00:39:28.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccb6000640 1 -- 172.21.15.156:0/2646640920 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb00a5980 con 0x7fccb80cde70 2024-03-30T00:39:28.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccb6000640 1 -- 172.21.15.156:0/2646640920 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb00a5d10 con 0x7fccb80cde70 2024-03-30T00:39:28.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 -- 172.21.15.156:0/2646640920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccb80cde70 msgr2=0x7fccb815a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 --2- 172.21.15.156:0/2646640920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccb80cde70 0x7fccb815a1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fccb80adcf0 tx=0x7fccb001c960 comp rx=0 tx=0).stop 2024-03-30T00:39:28.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 -- 172.21.15.156:0/2646640920 shutdown_connections 2024-03-30T00:39:28.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 --2- 172.21.15.156:0/2646640920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccb80cde70 0x7fccb815a1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 --2- 172.21.15.156:0/2646640920 >> v2:172.21.15.66:3300/0 conn(0x7fccb80cec30 0x7fccb8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 -- 172.21.15.156:0/2646640920 >> 172.21.15.156:0/2646640920 conn(0x7fccb8069f60 msgr2=0x7fccb806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:28.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 -- 172.21.15.156:0/2646640920 shutdown_connections 2024-03-30T00:39:28.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 -- 172.21.15.156:0/2646640920 wait complete. 2024-03-30T00:39:28.971 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 Processor -- start 2024-03-30T00:39:28.971 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 -- start start 2024-03-30T00:39:28.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccb80cda90 0x7fccb81642a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccb80cde70 0x7fccb80add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccb80cec30 0x7fccb8159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:28.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb815a4c0 con 0x7fccb80cda90 2024-03-30T00:39:28.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80ae460 con 0x7fccb80cde70 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccbe600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80ae740 con 0x7fccb80cec30 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccb6a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccb80cde70 0x7fccb80add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.962+0000 7fccb7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccb80cec30 0x7fccb8159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccb6a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccb80cde70 0x7fccb80add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54376/0 (socket says 172.21.15.156:54376) 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccb7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccb80cda90 0x7fccb81642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccb6a00640 1 -- 172.21.15.156:0/1100981905 learned_addr learned my addr 172.21.15.156:0/1100981905 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccb6a00640 1 -- 172.21.15.156:0/1100981905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccb80cec30 msgr2=0x7fccb8159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccb6a00640 1 --2- 172.21.15.156:0/1100981905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccb80cec30 0x7fccb8159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccb6a00640 1 -- 172.21.15.156:0/1100981905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccb80cda90 msgr2=0x7fccb81642a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccb6a00640 1 --2- 172.21.15.156:0/1100981905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccb80cda90 0x7fccb81642a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccb6a00640 1 -- 172.21.15.156:0/1100981905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb0009b30 con 0x7fccb80cde70 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccb7e00640 1 --2- 172.21.15.156:0/1100981905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccb80cec30 0x7fccb8159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:28.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccb6a00640 1 --2- 172.21.15.156:0/1100981905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccb80cde70 0x7fccb80add50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcca400d5c0 tx=0x7fcca400da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:28.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccaf400640 1 -- 172.21.15.156:0/1100981905 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca40125f0 con 0x7fccb80cde70 2024-03-30T00:39:28.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccaf400640 1 -- 172.21.15.156:0/1100981905 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca4012750 con 0x7fccb80cde70 2024-03-30T00:39:28.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccbe600640 1 -- 172.21.15.156:0/1100981905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb80aea20 con 0x7fccb80cde70 2024-03-30T00:39:28.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccaf400640 1 -- 172.21.15.156:0/1100981905 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4012b00 con 0x7fccb80cde70 2024-03-30T00:39:28.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccbe600640 1 -- 172.21.15.156:0/1100981905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb80aefe0 con 0x7fccb80cde70 2024-03-30T00:39:28.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccaf400640 1 -- 172.21.15.156:0/1100981905 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcca4004020 con 0x7fccb80cde70 2024-03-30T00:39:28.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccaf400640 1 -- 172.21.15.156:0/1100981905 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fcca40042d0 con 0x7fccb80cde70 2024-03-30T00:39:28.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.966+0000 7fccbe600640 1 -- 172.21.15.156:0/1100981905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb8066c30 con 0x7fccb80cde70 2024-03-30T00:39:28.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:28.970+0000 7fccaf400640 1 -- 172.21.15.156:0/1100981905 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fcca4003c60 con 0x7fccb80cde70 2024-03-30T00:39:29.012 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:29.002+0000 7f0a61649180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T00:39:29.054 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:29.046+0000 7fccbe600640 1 -- 172.21.15.156:0/1100981905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ba829def-38d8-4dd7-9d66-91dddaa8395b", "id": 7} v 0) v1 -- 0x7fccb80b3170 con 0x7fccb80cde70 2024-03-30T00:39:29.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:29.050+0000 7fccaf400640 1 -- 172.21.15.156:0/1100981905 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ba829def-38d8-4dd7-9d66-91dddaa8395b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcca4003e00 con 0x7fccb80cde70 2024-03-30T00:39:29.058 INFO:teuthology.orchestra.run.smithi156.stdout:7 2024-03-30T00:39:29.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:29.050+0000 7fccbe600640 1 -- 172.21.15.156:0/1100981905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccb80cde70 msgr2=0x7fccb80add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:29.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:29.050+0000 7fccbe600640 1 --2- 172.21.15.156:0/1100981905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccb80cde70 0x7fccb80add50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcca400d5c0 tx=0x7fcca400da80 comp rx=0 tx=0).stop 2024-03-30T00:39:29.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:29.050+0000 7fccbe600640 1 -- 172.21.15.156:0/1100981905 shutdown_connections 2024-03-30T00:39:29.060 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:29.050+0000 7fccbe600640 1 --2- 172.21.15.156:0/1100981905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccb80cec30 0x7fccb8159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:29.060 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:29.050+0000 7fccbe600640 1 --2- 172.21.15.156:0/1100981905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccb80cde70 0x7fccb80add50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:29.060 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:29.050+0000 7fccbe600640 1 --2- 172.21.15.156:0/1100981905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccb80cda90 0x7fccb81642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:29.060 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:29.050+0000 7fccbe600640 1 -- 172.21.15.156:0/1100981905 >> 172.21.15.156:0/1100981905 conn(0x7fccb8069f60 msgr2=0x7fccb8164a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:29.060 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:29.050+0000 7fccbe600640 1 -- 172.21.15.156:0/1100981905 shutdown_connections 2024-03-30T00:39:29.060 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-30T00:39:29.050+0000 7fccbe600640 1 -- 172.21.15.156:0/1100981905 wait complete. 2024-03-30T00:39:29.076 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-30T00:39:29.077 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-30T00:39:29.080 INFO:tasks.ceph.osd.0:Started 2024-03-30T00:39:29.080 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-30T00:39:29.080 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-30T00:39:29.083 INFO:tasks.ceph.osd.1:Started 2024-03-30T00:39:29.083 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-30T00:39:29.083 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-30T00:39:29.086 INFO:tasks.ceph.osd.2:Started 2024-03-30T00:39:29.086 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-30T00:39:29.087 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-30T00:39:29.091 INFO:tasks.ceph.osd.3:Started 2024-03-30T00:39:29.091 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-30T00:39:29.091 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-30T00:39:29.091 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:29.082+0000 7f0a61649180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T00:39:29.095 INFO:tasks.ceph.osd.4:Started 2024-03-30T00:39:29.096 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-30T00:39:29.096 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-30T00:39:29.108 INFO:tasks.ceph.osd.5:Started 2024-03-30T00:39:29.108 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-30T00:39:29.109 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-30T00:39:29.113 INFO:tasks.ceph.osd.6:Started 2024-03-30T00:39:29.113 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-30T00:39:29.113 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-30T00:39:29.128 INFO:tasks.ceph.osd.7:Started 2024-03-30T00:39:29.129 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:39:29.286 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:29.278+0000 7f0a61649180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:39:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.294+0000 7f301a600640 1 Processor -- start 2024-03-30T00:39:29.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.294+0000 7f301a600640 1 -- start start 2024-03-30T00:39:29.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f301a600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f301415de30 0x7f30140cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:29.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f301a600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30140cbb90 con 0x7f30140af420 2024-03-30T00:39:29.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f301a600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30140ce1b0 con 0x7f30140cee50 2024-03-30T00:39:29.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f301a600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30140afa10 con 0x7f301415de30 2024-03-30T00:39:29.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f300aa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f30140af420 0x7f30140ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39762/0 (socket says 172.21.15.66:39762) 2024-03-30T00:39:29.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f300aa00640 1 -- 172.21.15.66:0/2423783794 learned_addr learned my addr 172.21.15.66:0/2423783794 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:39:29.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f3013400640 1 --2- 172.21.15.66:0/2423783794 >> v2:172.21.15.156:3300/0 conn(0x7f301415de30 0x7f30140cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f3012a00640 1 -- 172.21.15.66:0/2423783794 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425769245 0 0) 0x7f30140cbb90 con 0x7f30140af420 2024-03-30T00:39:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f3012a00640 1 -- 172.21.15.66:0/2423783794 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff8003530 con 0x7f30140af420 2024-03-30T00:39:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f3012a00640 1 -- 172.21.15.66:0/2423783794 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156059745 0 0) 0x7f30140ce1b0 con 0x7f30140cee50 2024-03-30T00:39:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f3012a00640 1 -- 172.21.15.66:0/2423783794 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30140cbb90 con 0x7f30140cee50 2024-03-30T00:39:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f3013400640 1 -- 172.21.15.66:0/2423783794 >> v1:172.21.15.156:6789/0 conn(0x7f30140cee50 legacy=0x7f301406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f3013400640 1 -- 172.21.15.66:0/2423783794 >> v1:172.21.15.66:6789/0 conn(0x7f30140af420 legacy=0x7f30140ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.298+0000 7f3013400640 1 -- 172.21.15.66:0/2423783794 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30140cf430 con 0x7f301415de30 2024-03-30T00:39:29.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f3013400640 1 --2- 172.21.15.66:0/2423783794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f301415de30 0x7f30140cc040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f30040097c0 tx=0x7f300401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acd3e4c463536cc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:29.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f3012a00640 1 -- 172.21.15.66:0/2423783794 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30040ab600 con 0x7f301415de30 2024-03-30T00:39:29.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f3012a00640 1 -- 172.21.15.66:0/2423783794 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30040a7050 con 0x7f301415de30 2024-03-30T00:39:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f3012a00640 1 -- 172.21.15.66:0/2423783794 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30040ab980 con 0x7f301415de30 2024-03-30T00:39:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f301a600640 1 -- 172.21.15.66:0/2423783794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f301415de30 msgr2=0x7f30140cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f301a600640 1 --2- 172.21.15.66:0/2423783794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f301415de30 0x7f30140cc040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f30040097c0 tx=0x7f300401c410 comp rx=0 tx=0).stop 2024-03-30T00:39:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f301a600640 1 -- 172.21.15.66:0/2423783794 shutdown_connections 2024-03-30T00:39:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f301a600640 1 --2- 172.21.15.66:0/2423783794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f301415de30 0x7f30140cc040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f301a600640 1 -- 172.21.15.66:0/2423783794 >> 172.21.15.66:0/2423783794 conn(0x7f30140d0540 msgr2=0x7f30140d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f301a600640 1 -- 172.21.15.66:0/2423783794 shutdown_connections 2024-03-30T00:39:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f301a600640 1 -- 172.21.15.66:0/2423783794 wait complete. 2024-03-30T00:39:29.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f301a600640 1 Processor -- start 2024-03-30T00:39:29.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.302+0000 7f301a600640 1 -- start start 2024-03-30T00:39:29.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f301a600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30140af420 0x7f301406b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:29.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f301a600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30140cee50 0x7f3014150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:29.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f301a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f301415de30 0x7f3014150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:29.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f301a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30140c5c20 con 0x7f301415de30 2024-03-30T00:39:29.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f301a600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30140c6ea0 con 0x7f30140af420 2024-03-30T00:39:29.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f301a600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30140c4d00 con 0x7f30140cee50 2024-03-30T00:39:29.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f3013e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f301415de30 0x7f3014150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:29.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f3013e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f301415de30 0x7f3014150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58328/0 (socket says 172.21.15.66:58328) 2024-03-30T00:39:29.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f3013e00640 1 -- 172.21.15.66:0/707926745 learned_addr learned my addr 172.21.15.66:0/707926745 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:29.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f300aa00640 1 --2- 172.21.15.66:0/707926745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30140cee50 0x7f3014150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:29.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f3013400640 1 --2- 172.21.15.66:0/707926745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30140af420 0x7f301406b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:29.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f3013e00640 1 -- 172.21.15.66:0/707926745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30140cee50 msgr2=0x7f3014150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:29.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f3013e00640 1 --2- 172.21.15.66:0/707926745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30140cee50 0x7f3014150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:29.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f3013e00640 1 -- 172.21.15.66:0/707926745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30140af420 msgr2=0x7f301406b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:29.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f3013e00640 1 --2- 172.21.15.66:0/707926745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30140af420 0x7f301406b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:29.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f3013e00640 1 -- 172.21.15.66:0/707926745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3004009420 con 0x7f301415de30 2024-03-30T00:39:29.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f3013e00640 1 --2- 172.21.15.66:0/707926745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f301415de30 0x7f3014150ed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f300c00c790 tx=0x7f300c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f300be00640 1 -- 172.21.15.66:0/707926745 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c00cea0 con 0x7f301415de30 2024-03-30T00:39:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f300be00640 1 -- 172.21.15.66:0/707926745 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c003b80 con 0x7f301415de30 2024-03-30T00:39:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f301a600640 1 -- 172.21.15.66:0/707926745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30140c6040 con 0x7f301415de30 2024-03-30T00:39:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f300be00640 1 -- 172.21.15.66:0/707926745 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c00a850 con 0x7f301415de30 2024-03-30T00:39:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f301a600640 1 -- 172.21.15.66:0/707926745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30140c5280 con 0x7f301415de30 2024-03-30T00:39:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f300be00640 1 -- 172.21.15.66:0/707926745 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f300c00acb0 con 0x7f301415de30 2024-03-30T00:39:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f300be00640 1 -- 172.21.15.66:0/707926745 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f300c012470 con 0x7f301415de30 2024-03-30T00:39:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.306+0000 7f301a600640 1 -- 172.21.15.66:0/707926745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30140c5d60 con 0x7f301415de30 2024-03-30T00:39:29.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.310+0000 7f300be00640 1 -- 172.21.15.66:0/707926745 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f30140c5d60 con 0x7f301415de30 2024-03-30T00:39:29.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.366+0000 7f301a600640 1 -- 172.21.15.66:0/707926745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3014066c30 con 0x7f301415de30 2024-03-30T00:39:29.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.366+0000 7f300be00640 1 -- 172.21.15.66:0/707926745 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f300c0050d0 con 0x7f301415de30 2024-03-30T00:39:29.377 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:39:29.377 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":10,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:29.050525+0000","last_up_change":"0.000000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:39:29.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.370+0000 7f301a600640 1 -- 172.21.15.66:0/707926745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f301415de30 msgr2=0x7f3014150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:29.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.370+0000 7f301a600640 1 --2- 172.21.15.66:0/707926745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f301415de30 0x7f3014150ed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f300c00c790 tx=0x7f300c00cc50 comp rx=0 tx=0).stop 2024-03-30T00:39:29.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.370+0000 7f301a600640 1 -- 172.21.15.66:0/707926745 shutdown_connections 2024-03-30T00:39:29.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.370+0000 7f301a600640 1 --2- 172.21.15.66:0/707926745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f301415de30 0x7f3014150ed0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:29.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.370+0000 7f301a600640 1 --2- 172.21.15.66:0/707926745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30140cee50 0x7f3014150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:29.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.370+0000 7f301a600640 1 --2- 172.21.15.66:0/707926745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30140af420 0x7f301406b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:29.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.370+0000 7f301a600640 1 -- 172.21.15.66:0/707926745 >> 172.21.15.66:0/707926745 conn(0x7f30140d0540 msgr2=0x7f30140ae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:29.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.370+0000 7f301a600640 1 -- 172.21.15.66:0/707926745 shutdown_connections 2024-03-30T00:39:29.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:29.370+0000 7f301a600640 1 -- 172.21.15.66:0/707926745 wait complete. 2024-03-30T00:39:29.397 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-30T00:39:29.397 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-30T00:39:29.464 INFO:tasks.ceph.mgr.x.smithi156.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-30T00:39:29.464 INFO:tasks.ceph.mgr.x.smithi156.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-30T00:39:29.465 INFO:tasks.ceph.mgr.x.smithi156.stderr: from numpy import show_config as show_numpy_config 2024-03-30T00:39:29.507 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:29.498+0000 7f19d2a38180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T00:39:29.653 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:29.646+0000 7f19d2a38180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:39:29.734 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-03-30T00:39:29.726+0000 7f5c5e62e640 -1 Falling back to public interface 2024-03-30T00:39:29.762 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-03-30T00:39:29.754+0000 7fc2dac95640 -1 Falling back to public interface 2024-03-30T00:39:29.774 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-03-30T00:39:29.766+0000 7f6991d2b640 -1 Falling back to public interface 2024-03-30T00:39:29.777 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-03-30T00:39:29.770+0000 7f1f7b384640 -1 Falling back to public interface 2024-03-30T00:39:29.790 INFO:tasks.ceph.osd.4.smithi156.stderr:2024-03-30T00:39:29.782+0000 7f4490d67640 -1 Falling back to public interface 2024-03-30T00:39:29.797 INFO:tasks.ceph.osd.5.smithi156.stderr:2024-03-30T00:39:29.790+0000 7f8eecb07640 -1 Falling back to public interface 2024-03-30T00:39:29.801 INFO:tasks.ceph.osd.6.smithi156.stderr:2024-03-30T00:39:29.794+0000 7f92639bd640 -1 Falling back to public interface 2024-03-30T00:39:29.809 INFO:tasks.ceph.osd.7.smithi156.stderr:2024-03-30T00:39:29.802+0000 7f5d6bdda640 -1 Falling back to public interface 2024-03-30T00:39:29.831 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:29.822+0000 7f19d2a38180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T00:39:29.904 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:29.894+0000 7f19d2a38180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T00:39:30.014 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:30.006+0000 7f0a61649180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T00:39:30.051 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:30.042+0000 7f19d2a38180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:39:30.106 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:30.098+0000 7f0a61649180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T00:39:30.204 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T00:39:30.321 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:30.314+0000 7f0a61649180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T00:39:30.337 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd245e00640 1 Processor -- start 2024-03-30T00:39:30.337 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd245e00640 1 -- start start 2024-03-30T00:39:30.338 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd245e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2400cd500 0x7fd2400ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:30.338 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd245e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2400cb060 con 0x7fd24015cac0 2024-03-30T00:39:30.338 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd245e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2400d0220 con 0x7fd24006c150 2024-03-30T00:39:30.338 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd245e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400cdf30 con 0x7fd2400cd500 2024-03-30T00:39:30.338 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd23ea00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd24015cac0 0x7fd2400ccdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39764/0 (socket says 172.21.15.66:39764) 2024-03-30T00:39:30.338 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd23ea00640 1 -- 172.21.15.66:0/4157945039 learned_addr learned my addr 172.21.15.66:0/4157945039 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:39:30.339 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd23e000640 1 --2- 172.21.15.66:0/4157945039 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2400cd500 0x7fd2400ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:30.339 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd23d600640 1 -- 172.21.15.66:0/4157945039 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505987783 0 0) 0x7fd2400cb060 con 0x7fd24015cac0 2024-03-30T00:39:30.339 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd23d600640 1 -- 172.21.15.66:0/4157945039 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd224003530 con 0x7fd24015cac0 2024-03-30T00:39:30.339 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd23d600640 1 -- 172.21.15.66:0/4157945039 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023699697 0 0) 0x7fd2400d0220 con 0x7fd24006c150 2024-03-30T00:39:30.339 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd23e000640 1 -- 172.21.15.66:0/4157945039 >> v1:172.21.15.156:6789/0 conn(0x7fd24006c150 legacy=0x7fd2400cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:30.339 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd23e000640 1 -- 172.21.15.66:0/4157945039 >> v1:172.21.15.66:6789/0 conn(0x7fd24015cac0 legacy=0x7fd2400ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:30.339 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.330+0000 7fd23e000640 1 -- 172.21.15.66:0/4157945039 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2400cf990 con 0x7fd2400cd500 2024-03-30T00:39:30.342 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd23e000640 1 --2- 172.21.15.66:0/4157945039 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2400cd500 0x7fd2400ce9a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0097f0 tx=0x7fd22c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c763253f5ebb5e38 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:30.342 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd23d600640 1 -- 172.21.15.66:0/4157945039 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c0ab620 con 0x7fd2400cd500 2024-03-30T00:39:30.342 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd23d600640 1 -- 172.21.15.66:0/4157945039 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c0ab780 con 0x7fd2400cd500 2024-03-30T00:39:30.342 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd23d600640 1 -- 172.21.15.66:0/4157945039 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c0abb10 con 0x7fd2400cd500 2024-03-30T00:39:30.343 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd245e00640 1 -- 172.21.15.66:0/4157945039 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2400cd500 msgr2=0x7fd2400ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:30.343 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd245e00640 1 --2- 172.21.15.66:0/4157945039 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2400cd500 0x7fd2400ce9a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0097f0 tx=0x7fd22c01c420 comp rx=0 tx=0).stop 2024-03-30T00:39:30.343 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd245e00640 1 -- 172.21.15.66:0/4157945039 shutdown_connections 2024-03-30T00:39:30.343 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd245e00640 1 --2- 172.21.15.66:0/4157945039 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2400cd500 0x7fd2400ce9a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:30.343 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd245e00640 1 -- 172.21.15.66:0/4157945039 >> 172.21.15.66:0/4157945039 conn(0x7fd2400d0570 msgr2=0x7fd2400d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:30.343 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd245e00640 1 -- 172.21.15.66:0/4157945039 shutdown_connections 2024-03-30T00:39:30.343 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd245e00640 1 -- 172.21.15.66:0/4157945039 wait complete. 2024-03-30T00:39:30.344 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd245e00640 1 Processor -- start 2024-03-30T00:39:30.344 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.334+0000 7fd245e00640 1 -- start start 2024-03-30T00:39:30.347 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd245e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd24006c150 0x7fd240152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:30.347 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd245e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2400cd500 0x7fd2400c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:30.347 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd245e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd24015cac0 0x7fd240151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:30.347 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd245e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24014b000 con 0x7fd2400cd500 2024-03-30T00:39:30.347 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd245e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd240153280 con 0x7fd24015cac0 2024-03-30T00:39:30.347 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd245e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400c3fd0 con 0x7fd24006c150 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2400cd500 0x7fd2400c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2400cd500 0x7fd2400c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58340/0 (socket says 172.21.15.66:58340) 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23e000640 1 -- 172.21.15.66:0/3807810298 learned_addr learned my addr 172.21.15.66:0/3807810298 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23ea00640 1 --2- 172.21.15.66:0/3807810298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd24006c150 0x7fd240152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23f400640 1 --2- 172.21.15.66:0/3807810298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd24015cac0 0x7fd240151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23e000640 1 -- 172.21.15.66:0/3807810298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd24006c150 msgr2=0x7fd240152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23e000640 1 --2- 172.21.15.66:0/3807810298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd24006c150 0x7fd240152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23e000640 1 -- 172.21.15.66:0/3807810298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd24015cac0 msgr2=0x7fd240151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23e000640 1 --2- 172.21.15.66:0/3807810298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd24015cac0 0x7fd240151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23e000640 1 -- 172.21.15.66:0/3807810298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c009420 con 0x7fd2400cd500 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23f400640 1 --2- 172.21.15.66:0/3807810298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd24015cac0 0x7fd240151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd23e000640 1 --2- 172.21.15.66:0/3807810298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2400cd500 0x7fd2400c4d90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0ad040 tx=0x7fd22c01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd237400640 1 -- 172.21.15.66:0/3807810298 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c0abdc0 con 0x7fd2400cd500 2024-03-30T00:39:30.348 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd237400640 1 -- 172.21.15.66:0/3807810298 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c0bf410 con 0x7fd2400cd500 2024-03-30T00:39:30.349 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd237400640 1 -- 172.21.15.66:0/3807810298 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c0bf7c0 con 0x7fd2400cd500 2024-03-30T00:39:30.349 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd245e00640 1 -- 172.21.15.66:0/3807810298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2400c42b0 con 0x7fd2400cd500 2024-03-30T00:39:30.349 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd245e00640 1 -- 172.21.15.66:0/3807810298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2400c4870 con 0x7fd2400cd500 2024-03-30T00:39:30.349 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd237400640 1 -- 172.21.15.66:0/3807810298 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd22c0bfc20 con 0x7fd2400cd500 2024-03-30T00:39:30.349 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd237400640 1 -- 172.21.15.66:0/3807810298 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd22c0b5bd0 con 0x7fd2400cd500 2024-03-30T00:39:30.349 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.338+0000 7fd245e00640 1 -- 172.21.15.66:0/3807810298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd240067150 con 0x7fd2400cd500 2024-03-30T00:39:30.350 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.342+0000 7fd237400640 1 -- 172.21.15.66:0/3807810298 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fd22c0b5db0 con 0x7fd2400cd500 2024-03-30T00:39:30.409 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.398+0000 7fd245e00640 1 -- 172.21.15.66:0/3807810298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2400c4a60 con 0x7fd2400cd500 2024-03-30T00:39:30.409 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.402+0000 7fd237400640 1 -- 172.21.15.66:0/3807810298 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd22c0d0090 con 0x7fd2400cd500 2024-03-30T00:39:30.409 INFO:teuthology.misc.health.smithi066.stdout: 2024-03-30T00:39:30.409 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":10,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:29.050525+0000","last_up_change":"0.000000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:39:30.410 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.402+0000 7fd245e00640 1 -- 172.21.15.66:0/3807810298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2400cd500 msgr2=0x7fd2400c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:30.410 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.402+0000 7fd245e00640 1 --2- 172.21.15.66:0/3807810298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2400cd500 0x7fd2400c4d90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0ad040 tx=0x7fd22c01ac60 comp rx=0 tx=0).stop 2024-03-30T00:39:30.411 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.402+0000 7fd245e00640 1 -- 172.21.15.66:0/3807810298 shutdown_connections 2024-03-30T00:39:30.411 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.402+0000 7fd245e00640 1 --2- 172.21.15.66:0/3807810298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd24015cac0 0x7fd240151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:30.411 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.402+0000 7fd245e00640 1 --2- 172.21.15.66:0/3807810298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2400cd500 0x7fd2400c4d90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:30.411 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.402+0000 7fd245e00640 1 --2- 172.21.15.66:0/3807810298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd24006c150 0x7fd240152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:30.411 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.402+0000 7fd245e00640 1 -- 172.21.15.66:0/3807810298 >> 172.21.15.66:0/3807810298 conn(0x7fd2400d0570 msgr2=0x7fd2400ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:30.411 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.402+0000 7fd245e00640 1 -- 172.21.15.66:0/3807810298 shutdown_connections 2024-03-30T00:39:30.411 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:30.402+0000 7fd245e00640 1 -- 172.21.15.66:0/3807810298 wait complete. 2024-03-30T00:39:30.414 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:30.406+0000 7f0a61649180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T00:39:30.427 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-30T00:39:30.760 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:30.754+0000 7f19d2a38180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T00:39:30.765 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:30.758+0000 7f0a61649180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T00:39:30.840 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:30.830+0000 7f0a61649180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T00:39:30.849 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:30.842+0000 7f19d2a38180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T00:39:31.055 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:31.046+0000 7f19d2a38180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T00:39:31.140 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:31.130+0000 7f19d2a38180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T00:39:31.466 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:31.458+0000 7f19d2a38180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T00:39:31.536 INFO:tasks.ceph.mgr.x.smithi156.stderr:2024-03-30T00:39:31.526+0000 7f19d2a38180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T00:39:31.861 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T00:39:31.854+0000 7f0a06e00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-30T00:39:33.542 INFO:tasks.ceph.osd.4.smithi156.stderr:2024-03-30T00:39:33.534+0000 7f4490d67640 -1 osd.4 0 log_to_monitors true 2024-03-30T00:39:33.553 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-03-30T00:39:33.546+0000 7f5c5e62e640 -1 osd.0 0 log_to_monitors true 2024-03-30T00:39:33.653 INFO:tasks.ceph.osd.5.smithi156.stderr:2024-03-30T00:39:33.646+0000 7f8eecb07640 -1 osd.5 0 log_to_monitors true 2024-03-30T00:39:33.713 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-03-30T00:39:33.706+0000 7fc2dac95640 -1 osd.3 0 log_to_monitors true 2024-03-30T00:39:33.724 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-03-30T00:39:33.714+0000 7f1f7b384640 -1 osd.1 0 log_to_monitors true 2024-03-30T00:39:33.726 INFO:tasks.ceph.osd.7.smithi156.stderr:2024-03-30T00:39:33.718+0000 7f5d6bdda640 -1 osd.7 0 log_to_monitors true 2024-03-30T00:39:33.749 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-03-30T00:39:33.742+0000 7f6991d2b640 -1 osd.2 0 log_to_monitors true 2024-03-30T00:39:33.769 INFO:tasks.ceph.osd.6.smithi156.stderr:2024-03-30T00:39:33.762+0000 7f92639bd640 -1 osd.6 0 log_to_monitors true 2024-03-30T00:39:36.599 INFO:tasks.ceph.osd.4.smithi156.stderr:2024-03-30T00:39:36.590+0000 7f448bc00640 -1 osd.4 0 waiting for initial osdmap 2024-03-30T00:39:36.630 INFO:tasks.ceph.osd.4.smithi156.stderr:2024-03-30T00:39:36.622+0000 7f4481c00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:39:36.636 INFO:tasks.ceph.osd.6.smithi156.stderr:2024-03-30T00:39:36.626+0000 7f925f200640 -1 osd.6 0 waiting for initial osdmap 2024-03-30T00:39:36.643 INFO:tasks.ceph.osd.5.smithi156.stderr:2024-03-30T00:39:36.634+0000 7f8ee7a00640 -1 osd.5 0 waiting for initial osdmap 2024-03-30T00:39:36.658 INFO:tasks.ceph.osd.7.smithi156.stderr:2024-03-30T00:39:36.650+0000 7f5d66c00640 -1 osd.7 0 waiting for initial osdmap 2024-03-30T00:39:36.669 INFO:tasks.ceph.osd.6.smithi156.stderr:2024-03-30T00:39:36.662+0000 7f9254800640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:39:36.674 INFO:tasks.ceph.osd.5.smithi156.stderr:2024-03-30T00:39:36.666+0000 7f8edda00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:39:36.689 INFO:tasks.ceph.osd.7.smithi156.stderr:2024-03-30T00:39:36.682+0000 7f5d5cc00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:39:37.107 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-03-30T00:39:37.098+0000 7f5c58c00640 -1 osd.0 0 waiting for initial osdmap 2024-03-30T00:39:37.146 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-03-30T00:39:37.138+0000 7f5c4ec00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:39:37.172 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-03-30T00:39:37.162+0000 7f1f76200640 -1 osd.1 0 waiting for initial osdmap 2024-03-30T00:39:37.202 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-03-30T00:39:37.194+0000 7f1f6c200640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:39:37.222 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-03-30T00:39:37.214+0000 7f698cc00640 -1 osd.2 0 waiting for initial osdmap 2024-03-30T00:39:37.232 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-03-30T00:39:37.222+0000 7fc2d5c00640 -1 osd.3 0 waiting for initial osdmap 2024-03-30T00:39:37.235 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T00:39:37.258 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-03-30T00:39:37.250+0000 7fc2cbc00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:39:37.258 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-03-30T00:39:37.250+0000 7f6982c00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:39:37.365 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f451c800640 1 Processor -- start 2024-03-30T00:39:37.366 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f451c800640 1 -- start start 2024-03-30T00:39:37.366 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f451c800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f451415df90 0x7f45140d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:37.366 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f451c800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f45140ccf80 0x7f45140cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:37.367 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f451c800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45140ce970 con 0x7f451406eab0 2024-03-30T00:39:37.367 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f451c800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45140cd940 con 0x7f45140ccf80 2024-03-30T00:39:37.367 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f451c800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451415e370 con 0x7f451415df90 2024-03-30T00:39:37.368 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f4519400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f45140ccf80 0x7f45140cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:37.368 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f451a800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f451406eab0 0x7f451406ef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39780/0 (socket says 172.21.15.66:39780) 2024-03-30T00:39:37.368 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f451a800640 1 -- 172.21.15.66:0/4103331298 learned_addr learned my addr 172.21.15.66:0/4103331298 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:39:37.368 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f4519e00640 1 --2- 172.21.15.66:0/4103331298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f451415df90 0x7f45140d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:37.368 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f4519400640 1 -- 172.21.15.66:0/4103331298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f451415df90 msgr2=0x7f45140d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:37.368 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f4519400640 1 --2- 172.21.15.66:0/4103331298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f451415df90 0x7f45140d1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:37.368 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f4519400640 1 -- 172.21.15.66:0/4103331298 >> v1:172.21.15.66:6789/0 conn(0x7f451406eab0 legacy=0x7f451406ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:37.368 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f4519400640 1 -- 172.21.15.66:0/4103331298 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140ce3f0 con 0x7f45140ccf80 2024-03-30T00:39:37.368 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.358+0000 7f4518a00640 1 -- 172.21.15.66:0/4103331298 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337289018 0 0) 0x7f45100014a0 con 0x7f451406eab0 2024-03-30T00:39:37.371 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f4519400640 1 --2- 172.21.15.66:0/4103331298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45140ccf80 0x7f45140cd400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f45040097f0 tx=0x7f450401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e803b75cf7eb5802 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:37.371 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f4518a00640 1 -- 172.21.15.66:0/4103331298 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45040ad680 con 0x7f45140ccf80 2024-03-30T00:39:37.371 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f4518a00640 1 -- 172.21.15.66:0/4103331298 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45040ad7e0 con 0x7f45140ccf80 2024-03-30T00:39:37.371 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f4518a00640 1 -- 172.21.15.66:0/4103331298 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45040adb70 con 0x7f45140ccf80 2024-03-30T00:39:37.371 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f451c800640 1 -- 172.21.15.66:0/4103331298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45140ccf80 msgr2=0x7f45140cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:37.371 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f451c800640 1 --2- 172.21.15.66:0/4103331298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45140ccf80 0x7f45140cd400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f45040097f0 tx=0x7f450401c470 comp rx=0 tx=0).stop 2024-03-30T00:39:37.371 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f451c800640 1 -- 172.21.15.66:0/4103331298 shutdown_connections 2024-03-30T00:39:37.371 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f451c800640 1 --2- 172.21.15.66:0/4103331298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45140ccf80 0x7f45140cd400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:37.371 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f451c800640 1 --2- 172.21.15.66:0/4103331298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f451415df90 0x7f45140d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:37.372 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f451c800640 1 -- 172.21.15.66:0/4103331298 >> 172.21.15.66:0/4103331298 conn(0x7f45140d0570 msgr2=0x7f45140d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:37.372 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f451c800640 1 -- 172.21.15.66:0/4103331298 shutdown_connections 2024-03-30T00:39:37.372 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.362+0000 7f451c800640 1 -- 172.21.15.66:0/4103331298 wait complete. 2024-03-30T00:39:37.373 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451c800640 1 Processor -- start 2024-03-30T00:39:37.373 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451c800640 1 -- start start 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451c800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406eab0 0x7f45140ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451c800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45140ccf80 0x7f4514156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451c800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f451415df90 0x7f4514154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451c800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45140af100 con 0x7f451406eab0 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451c800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4514158880 con 0x7f45140ccf80 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451c800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4514158b20 con 0x7f451415df90 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f4519e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406eab0 0x7f45140ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f4519e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406eab0 0x7f45140ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58452/0 (socket says 172.21.15.66:58452) 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f4519e00640 1 -- 172.21.15.66:0/925544062 learned_addr learned my addr 172.21.15.66:0/925544062 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451a800640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f451415df90 0x7f4514154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f4519400640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45140ccf80 0x7f4514156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f4519e00640 1 -- 172.21.15.66:0/925544062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f451415df90 msgr2=0x7f4514154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:37.376 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f4519e00640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f451415df90 0x7f4514154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:37.377 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f4519e00640 1 -- 172.21.15.66:0/925544062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45140ccf80 msgr2=0x7f4514156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:37.377 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f4519e00640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45140ccf80 0x7f4514156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:37.377 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f4519e00640 1 -- 172.21.15.66:0/925544062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4504009420 con 0x7f451406eab0 2024-03-30T00:39:37.377 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f4519400640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45140ccf80 0x7f4514156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:39:37.377 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451a800640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f451415df90 0x7f4514154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:39:37.377 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f4519e00640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406eab0 0x7f45140ae3e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f450800e930 tx=0x7f450800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:37.377 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f450e000640 1 -- 172.21.15.66:0/925544062 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508009690 con 0x7f451406eab0 2024-03-30T00:39:37.377 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f450e000640 1 -- 172.21.15.66:0/925544062 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450800ca00 con 0x7f451406eab0 2024-03-30T00:39:37.377 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f450e000640 1 -- 172.21.15.66:0/925544062 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450800cd70 con 0x7f451406eab0 2024-03-30T00:39:37.377 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451c800640 1 -- 172.21.15.66:0/925544062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4514158e00 con 0x7f451406eab0 2024-03-30T00:39:37.377 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451c800640 1 -- 172.21.15.66:0/925544062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45141593c0 con 0x7f451406eab0 2024-03-30T00:39:37.378 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.366+0000 7f451c800640 1 -- 172.21.15.66:0/925544062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4514073d20 con 0x7f451406eab0 2024-03-30T00:39:37.378 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.370+0000 7f450e000640 1 -- 172.21.15.66:0/925544062 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) v1 ==== 86028+0+0 (secure 0 0 0) 0x7f4508013470 con 0x7f451406eab0 2024-03-30T00:39:37.378 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.370+0000 7f450e000640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44f8066600 0x7f44f8068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:37.378 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.370+0000 7f450e000640 1 -- 172.21.15.66:0/925544062 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f45080835f0 con 0x7f451406eab0 2024-03-30T00:39:37.378 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.370+0000 7f4519400640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44f8066600 0x7f44f8068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:37.379 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.370+0000 7f4519400640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44f8066600 0x7f44f8068ac0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f45040af000 tx=0x7f4504008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:37.383 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.374+0000 7f450e000640 1 -- 172.21.15.66:0/925544062 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f45080517f0 con 0x7f451406eab0 2024-03-30T00:39:37.549 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 -- 172.21.15.66:0/925544062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4514164f50 con 0x7f451406eab0 2024-03-30T00:39:37.549 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f450e000640 1 -- 172.21.15.66:0/925544062 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4508021090 con 0x7f451406eab0 2024-03-30T00:39:37.550 INFO:teuthology.misc.health.smithi066.stdout: 2024-03-30T00:39:37.550 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":12,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:34.860556+0000","last_up_change":"0.000000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:39:37.552 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 -- 172.21.15.66:0/925544062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44f8066600 msgr2=0x7f44f8068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:37.552 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44f8066600 0x7f44f8068ac0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f45040af000 tx=0x7f4504008040 comp rx=0 tx=0).stop 2024-03-30T00:39:37.552 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 -- 172.21.15.66:0/925544062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406eab0 msgr2=0x7f45140ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:37.552 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406eab0 0x7f45140ae3e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f450800e930 tx=0x7f450800edf0 comp rx=0 tx=0).stop 2024-03-30T00:39:37.552 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 -- 172.21.15.66:0/925544062 shutdown_connections 2024-03-30T00:39:37.552 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f451415df90 0x7f4514154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:37.552 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45140ccf80 0x7f4514156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:37.552 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44f8066600 0x7f44f8068ac0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:37.553 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 --2- 172.21.15.66:0/925544062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451406eab0 0x7f45140ae3e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:37.553 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 -- 172.21.15.66:0/925544062 >> 172.21.15.66:0/925544062 conn(0x7f45140d0570 msgr2=0x7f451406eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:37.553 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 -- 172.21.15.66:0/925544062 shutdown_connections 2024-03-30T00:39:37.553 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:37.542+0000 7f451c800640 1 -- 172.21.15.66:0/925544062 wait complete. 2024-03-30T00:39:37.568 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-30T00:39:44.375 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T00:39:44.517 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.506+0000 7f588b200640 1 Processor -- start 2024-03-30T00:39:44.517 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f588b200640 1 -- start start 2024-03-30T00:39:44.518 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f588b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58840b2870 0x7f588406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.518 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f588b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cd440 0x7f58840cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f588b200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f58840cee70 0x7f58840cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f588b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58840cddd0 con 0x7f58840b2870 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f588b200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58840ce0b0 con 0x7f58840cee70 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f588b200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58840ce390 con 0x7f58840cd440 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f5883e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58840b2870 0x7f588406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f5883e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58840b2870 0x7f588406db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42068/0 (socket says 172.21.15.66:42068) 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f5883e00640 1 -- 172.21.15.66:0/2210737503 learned_addr learned my addr 172.21.15.66:0/2210737503 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f587be00640 1 --2- 172.21.15.66:0/2210737503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cd440 0x7f58840cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f5889200640 1 --2- 172.21.15.66:0/2210737503 >> v2:172.21.15.156:3300/0 conn(0x7f58840cee70 0x7f58840cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f5883e00640 1 -- 172.21.15.66:0/2210737503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cd440 msgr2=0x7f58840cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f5883e00640 1 --2- 172.21.15.66:0/2210737503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cd440 0x7f58840cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f5883e00640 1 -- 172.21.15.66:0/2210737503 >> v2:172.21.15.156:3300/0 conn(0x7f58840cee70 msgr2=0x7f58840cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f5883e00640 1 --2- 172.21.15.66:0/2210737503 >> v2:172.21.15.156:3300/0 conn(0x7f58840cee70 0x7f58840cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.519 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f5883e00640 1 -- 172.21.15.66:0/2210737503 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840ce670 con 0x7f58840b2870 2024-03-30T00:39:44.520 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f587be00640 1 --2- 172.21.15.66:0/2210737503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cd440 0x7f58840cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:44.520 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.510+0000 7f5889200640 1 --2- 172.21.15.66:0/2210737503 >> v2:172.21.15.156:3300/0 conn(0x7f58840cee70 0x7f58840cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:44.521 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f5883e00640 1 --2- 172.21.15.66:0/2210737503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58840b2870 0x7f588406db10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f588406c860 tx=0x7f587401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99f2215d6748c13 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:44.522 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f5883400640 1 -- 172.21.15.66:0/2210737503 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588406e260 con 0x7f58840b2870 2024-03-30T00:39:44.522 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f5883400640 1 -- 172.21.15.66:0/2210737503 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58740a7070 con 0x7f58840b2870 2024-03-30T00:39:44.522 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f5883400640 1 -- 172.21.15.66:0/2210737503 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58740ab860 con 0x7f58840b2870 2024-03-30T00:39:44.522 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f588b200640 1 -- 172.21.15.66:0/2210737503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58840b2870 msgr2=0x7f588406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:44.523 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f588b200640 1 --2- 172.21.15.66:0/2210737503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58840b2870 0x7f588406db10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f588406c860 tx=0x7f587401c460 comp rx=0 tx=0).stop 2024-03-30T00:39:44.523 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f588b200640 1 -- 172.21.15.66:0/2210737503 shutdown_connections 2024-03-30T00:39:44.523 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f588b200640 1 --2- 172.21.15.66:0/2210737503 >> v2:172.21.15.156:3300/0 conn(0x7f58840cee70 0x7f58840cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.523 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f588b200640 1 --2- 172.21.15.66:0/2210737503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cd440 0x7f58840cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.523 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f588b200640 1 --2- 172.21.15.66:0/2210737503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58840b2870 0x7f588406db10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.523 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f588b200640 1 -- 172.21.15.66:0/2210737503 >> 172.21.15.66:0/2210737503 conn(0x7f5884069f60 msgr2=0x7f588406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:44.523 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f588b200640 1 -- 172.21.15.66:0/2210737503 shutdown_connections 2024-03-30T00:39:44.523 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f588b200640 1 -- 172.21.15.66:0/2210737503 wait complete. 2024-03-30T00:39:44.524 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f588b200640 1 Processor -- start 2024-03-30T00:39:44.524 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.514+0000 7f588b200640 1 -- start start 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f588b200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58840b2870 0x7f58840aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f588b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58840cd440 0x7f5884152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f588b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cee70 0x7f5884097540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f588b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588415acf0 con 0x7f58840cd440 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f588b200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5884159090 con 0x7f58840b2870 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f588b200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5884159370 con 0x7f58840cee70 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f587be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58840cd440 0x7f5884152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f587be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58840cd440 0x7f5884152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42080/0 (socket says 172.21.15.66:42080) 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f587be00640 1 -- 172.21.15.66:0/4147434517 learned_addr learned my addr 172.21.15.66:0/4147434517 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f5883e00640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58840b2870 0x7f58840aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f5889200640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cee70 0x7f5884097540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f587be00640 1 -- 172.21.15.66:0/4147434517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cee70 msgr2=0x7f5884097540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f587be00640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cee70 0x7f5884097540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.527 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f587be00640 1 -- 172.21.15.66:0/4147434517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58840b2870 msgr2=0x7f58840aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:44.528 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f587be00640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58840b2870 0x7f58840aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.528 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f587be00640 1 -- 172.21.15.66:0/4147434517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740094d0 con 0x7f58840cd440 2024-03-30T00:39:44.528 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f5889200640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cee70 0x7f5884097540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:44.528 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f587be00640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58840cd440 0x7f5884152db0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f586c00e810 tx=0x7f586c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:44.528 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f5880c00640 1 -- 172.21.15.66:0/4147434517 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c004030 con 0x7f58840cd440 2024-03-30T00:39:44.528 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f5880c00640 1 -- 172.21.15.66:0/4147434517 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c004190 con 0x7f58840cd440 2024-03-30T00:39:44.528 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f5880c00640 1 -- 172.21.15.66:0/4147434517 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c004520 con 0x7f58840cd440 2024-03-30T00:39:44.528 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f588b200640 1 -- 172.21.15.66:0/4147434517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5884159650 con 0x7f58840cd440 2024-03-30T00:39:44.528 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f588b200640 1 -- 172.21.15.66:0/4147434517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5884158c60 con 0x7f58840cd440 2024-03-30T00:39:44.528 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f588b200640 1 -- 172.21.15.66:0/4147434517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5884159b00 con 0x7f58840cd440 2024-03-30T00:39:44.529 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.518+0000 7f5880c00640 1 -- 172.21.15.66:0/4147434517 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f586c00c6d0 con 0x7f58840cd440 2024-03-30T00:39:44.529 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.522+0000 7f5880c00640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f58600666c0 0x7f5860068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.529 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.522+0000 7f5880c00640 1 -- 172.21.15.66:0/4147434517 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f586c0838f0 con 0x7f58840cd440 2024-03-30T00:39:44.529 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.522+0000 7f5883e00640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f58600666c0 0x7f5860068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.530 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.522+0000 7f5883e00640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f58600666c0 0x7f5860068b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f58740af000 tx=0x7f58740b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:44.534 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.526+0000 7f5880c00640 1 -- 172.21.15.66:0/4147434517 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f586c051050 con 0x7f58840cd440 2024-03-30T00:39:44.696 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.686+0000 7f588b200640 1 -- 172.21.15.66:0/4147434517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5884073d20 con 0x7f58840cd440 2024-03-30T00:39:44.696 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.686+0000 7f5880c00640 1 -- 172.21.15.66:0/4147434517 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f586c020090 con 0x7f58840cd440 2024-03-30T00:39:44.697 INFO:teuthology.misc.health.smithi066.stdout: 2024-03-30T00:39:44.697 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":17,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:42.625906+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:39:44.699 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 -- 172.21.15.66:0/4147434517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f58600666c0 msgr2=0x7f5860068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:44.699 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f58600666c0 0x7f5860068b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f58740af000 tx=0x7f58740b6040 comp rx=0 tx=0).stop 2024-03-30T00:39:44.699 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 -- 172.21.15.66:0/4147434517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58840cd440 msgr2=0x7f5884152db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:44.700 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58840cd440 0x7f5884152db0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f586c00e810 tx=0x7f586c00ecd0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.700 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 -- 172.21.15.66:0/4147434517 shutdown_connections 2024-03-30T00:39:44.700 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58840cee70 0x7f5884097540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.700 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f58600666c0 0x7f5860068b80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.700 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58840cd440 0x7f5884152db0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.700 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 --2- 172.21.15.66:0/4147434517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58840b2870 0x7f58840aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.700 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 -- 172.21.15.66:0/4147434517 >> 172.21.15.66:0/4147434517 conn(0x7f5884069f60 msgr2=0x7f58840ab0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:44.700 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 -- 172.21.15.66:0/4147434517 shutdown_connections 2024-03-30T00:39:44.700 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T00:39:44.690+0000 7f588b200640 1 -- 172.21.15.66:0/4147434517 wait complete. 2024-03-30T00:39:44.716 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-30T00:39:44.716 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-30T00:39:44.716 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T00:39:44.717 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T00:39:44.719 INFO:tasks.ceph.mds.a:Started 2024-03-30T00:39:44.720 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T00:39:44.720 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T00:39:44.722 INFO:tasks.ceph.mds.c:Started 2024-03-30T00:39:44.723 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T00:39:44.723 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T00:39:44.726 INFO:tasks.ceph.mds.b:Started 2024-03-30T00:39:44.726 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T00:39:44.727 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T00:39:44.729 INFO:tasks.ceph.mds.d:Started 2024-03-30T00:39:44.730 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-30T00:39:44.730 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:39:44.837 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2024-03-30T00:39:44.837 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2024-03-30T00:39:44.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.862+0000 7fe0edc00640 1 Processor -- start 2024-03-30T00:39:44.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.862+0000 7fe0edc00640 1 -- start start 2024-03-30T00:39:44.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0edc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0e80cda90 0x7fe0e80ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0edc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe0e815a370 0x7fe0e80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0edc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e815a990 con 0x7fe0e80cde70 2024-03-30T00:39:44.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0edc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e80b1b80 con 0x7fe0e815a370 2024-03-30T00:39:44.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0edc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80cd3e0 con 0x7fe0e80cda90 2024-03-30T00:39:44.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0e6a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0e80cda90 0x7fe0e80ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0de000640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe0e80cde70 0x7fe0e8159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:58490/0 (socket says 172.21.15.66:58490) 2024-03-30T00:39:44.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0de000640 1 -- 172.21.15.66:0/928921248 learned_addr learned my addr 172.21.15.66:0/928921248 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:39:44.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0e7400640 1 --2- 172.21.15.66:0/928921248 >> v2:172.21.15.156:3300/0 conn(0x7fe0e815a370 0x7fe0e80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0e6a00640 1 -- 172.21.15.66:0/928921248 >> v2:172.21.15.156:3300/0 conn(0x7fe0e815a370 msgr2=0x7fe0e80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0e6a00640 1 --2- 172.21.15.66:0/928921248 >> v2:172.21.15.156:3300/0 conn(0x7fe0e815a370 0x7fe0e80cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0e6a00640 1 -- 172.21.15.66:0/928921248 >> v1:172.21.15.156:6789/0 conn(0x7fe0e80cde70 legacy=0x7fe0e8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:44.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0e6000640 1 -- 172.21.15.66:0/928921248 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601548861 0 0) 0x7fe0e815a990 con 0x7fe0e80cde70 2024-03-30T00:39:44.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.866+0000 7fe0e6a00640 1 -- 172.21.15.66:0/928921248 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e80b11e0 con 0x7fe0e80cda90 2024-03-30T00:39:44.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0e6a00640 1 --2- 172.21.15.66:0/928921248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0e80cda90 0x7fe0e80ce520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800bfa0 tx=0x7fe0d801eae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d03f2a8c050b5691 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:44.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0e6000640 1 -- 172.21.15.66:0/928921248 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d800b2d0 con 0x7fe0e80cda90 2024-03-30T00:39:44.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0e6000640 1 -- 172.21.15.66:0/928921248 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d80a78c0 con 0x7fe0e80cda90 2024-03-30T00:39:44.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0e6000640 1 -- 172.21.15.66:0/928921248 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80a7c50 con 0x7fe0e80cda90 2024-03-30T00:39:44.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0edc00640 1 -- 172.21.15.66:0/928921248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0e80cda90 msgr2=0x7fe0e80ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:44.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0edc00640 1 --2- 172.21.15.66:0/928921248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0e80cda90 0x7fe0e80ce520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800bfa0 tx=0x7fe0d801eae0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0edc00640 1 -- 172.21.15.66:0/928921248 shutdown_connections 2024-03-30T00:39:44.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0edc00640 1 --2- 172.21.15.66:0/928921248 >> v2:172.21.15.156:3300/0 conn(0x7fe0e815a370 0x7fe0e80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0edc00640 1 --2- 172.21.15.66:0/928921248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0e80cda90 0x7fe0e80ce520 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0edc00640 1 -- 172.21.15.66:0/928921248 >> 172.21.15.66:0/928921248 conn(0x7fe0e8069f60 msgr2=0x7fe0e806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:44.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0edc00640 1 -- 172.21.15.66:0/928921248 shutdown_connections 2024-03-30T00:39:44.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0edc00640 1 -- 172.21.15.66:0/928921248 wait complete. 2024-03-30T00:39:44.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0edc00640 1 Processor -- start 2024-03-30T00:39:44.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.870+0000 7fe0edc00640 1 -- start start 2024-03-30T00:39:44.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0edc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0e80cda90 0x7fe0e81535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0edc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0e80cde70 0x7fe0e8155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0edc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0e815a370 0x7fe0e81641b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0edc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e8155fc0 con 0x7fe0e815a370 2024-03-30T00:39:44.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0edc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e8153e60 con 0x7fe0e80cda90 2024-03-30T00:39:44.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0edc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e8154140 con 0x7fe0e80cde70 2024-03-30T00:39:44.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0e7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0e815a370 0x7fe0e81641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0e7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0e815a370 0x7fe0e81641b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42090/0 (socket says 172.21.15.66:42090) 2024-03-30T00:39:44.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0e7400640 1 -- 172.21.15.66:0/2385480145 learned_addr learned my addr 172.21.15.66:0/2385480145 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:44.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0e6a00640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0e80cda90 0x7fe0e81535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0de000640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0e80cde70 0x7fe0e8155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0e7400640 1 -- 172.21.15.66:0/2385480145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0e80cde70 msgr2=0x7fe0e8155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:44.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0e7400640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0e80cde70 0x7fe0e8155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0e7400640 1 -- 172.21.15.66:0/2385480145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0e80cda90 msgr2=0x7fe0e81535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:44.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0e7400640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0e80cda90 0x7fe0e81535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:44.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0e7400640 1 -- 172.21.15.66:0/2385480145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d800bc80 con 0x7fe0e815a370 2024-03-30T00:39:44.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.874+0000 7fe0e7400640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0e815a370 0x7fe0e81641b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000e930 tx=0x7fe0e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:44.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.962+0000 7fe0df400640 1 -- 172.21.15.66:0/2385480145 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e00096b0 con 0x7fe0e815a370 2024-03-30T00:39:44.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.962+0000 7fe0edc00640 1 -- 172.21.15.66:0/2385480145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e8163c90 con 0x7fe0e815a370 2024-03-30T00:39:44.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.962+0000 7fe0edc00640 1 -- 172.21.15.66:0/2385480145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e8154420 con 0x7fe0e815a370 2024-03-30T00:39:44.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.962+0000 7fe0df400640 1 -- 172.21.15.66:0/2385480145 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e00059a0 con 0x7fe0e815a370 2024-03-30T00:39:44.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.962+0000 7fe0df400640 1 -- 172.21.15.66:0/2385480145 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0005d10 con 0x7fe0e815a370 2024-03-30T00:39:44.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.966+0000 7fe0edc00640 1 -- 172.21.15.66:0/2385480145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c0005260 con 0x7fe0e815a370 2024-03-30T00:39:44.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.966+0000 7fe0df400640 1 -- 172.21.15.66:0/2385480145 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe0e000c9a0 con 0x7fe0e815a370 2024-03-30T00:39:44.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.966+0000 7fe0df400640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe0c4066690 0x7fe0c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:44.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.966+0000 7fe0df400640 1 -- 172.21.15.66:0/2385480145 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe0e0083f50 con 0x7fe0e815a370 2024-03-30T00:39:44.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.966+0000 7fe0e6a00640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe0c4066690 0x7fe0c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.966+0000 7fe0e6a00640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe0c4066690 0x7fe0c4068b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80b0000 tx=0x7fe0d80a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:44.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:44.974+0000 7fe0df400640 1 -- 172.21.15.66:0/2385480145 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe0e0051770 con 0x7fe0e815a370 2024-03-30T00:39:45.014 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:39:45.026 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:39:45.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 -- 172.21.15.66:0/2385480145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0c0005510 con 0x7fe0e815a370 2024-03-30T00:39:45.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0df400640 1 -- 172.21.15.66:0/2385480145 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12382 (secure 0 0 0) 0x7fe0e0021090 con 0x7fe0e815a370 2024-03-30T00:39:45.166 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:39:45.167 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:42.625906+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 -- 172.21.15.66:0/2385480145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe0c4066690 msgr2=0x7fe0c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe0c4066690 0x7fe0c4068b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80b0000 tx=0x7fe0d80a9070 comp rx=0 tx=0).stop 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 -- 172.21.15.66:0/2385480145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0e815a370 msgr2=0x7fe0e81641b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0e815a370 0x7fe0e81641b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000e930 tx=0x7fe0e000edf0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 -- 172.21.15.66:0/2385480145 shutdown_connections 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe0c4066690 0x7fe0c4068b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0e815a370 0x7fe0e81641b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0e80cde70 0x7fe0e8155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 --2- 172.21.15.66:0/2385480145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0e80cda90 0x7fe0e81535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 -- 172.21.15.66:0/2385480145 >> 172.21.15.66:0/2385480145 conn(0x7fe0e8069f60 msgr2=0x7fe0e815b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 -- 172.21.15.66:0/2385480145 shutdown_connections 2024-03-30T00:39:45.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.158+0000 7fe0edc00640 1 -- 172.21.15.66:0/2385480145 wait complete. 2024-03-30T00:39:45.188 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:39:45.188 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:39:45.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.322+0000 7f8396200640 1 Processor -- start 2024-03-30T00:39:45.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.322+0000 7f8396200640 1 -- start start 2024-03-30T00:39:45.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8396200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839015c6b0 0x7f83900d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8396200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f839006cb90 0x7f839015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8396200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f83900cdef0 0x7f83900ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8396200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839006b1f0 con 0x7f839006cb90 2024-03-30T00:39:45.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8396200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83900ce880 con 0x7f83900cdef0 2024-03-30T00:39:45.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8396200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839015cfd0 con 0x7f839015c6b0 2024-03-30T00:39:45.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8387e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f839006cb90 0x7f839015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8394e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839015c6b0 0x7f83900d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8387e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f839006cb90 0x7f839015ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42114/0 (socket says 172.21.15.66:42114) 2024-03-30T00:39:45.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8387e00640 1 -- 172.21.15.66:0/3886947256 learned_addr learned my addr 172.21.15.66:0/3886947256 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:45.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8395800640 1 --2- 172.21.15.66:0/3886947256 >> v2:172.21.15.156:3300/0 conn(0x7f83900cdef0 0x7f83900ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8394e00640 1 -- 172.21.15.66:0/3886947256 >> v2:172.21.15.156:3300/0 conn(0x7f83900cdef0 msgr2=0x7f83900ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:45.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8394e00640 1 --2- 172.21.15.66:0/3886947256 >> v2:172.21.15.156:3300/0 conn(0x7f83900cdef0 0x7f83900ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8394e00640 1 -- 172.21.15.66:0/3886947256 >> v2:172.21.15.66:3300/0 conn(0x7f839006cb90 msgr2=0x7f839015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:45.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8394e00640 1 --2- 172.21.15.66:0/3886947256 >> v2:172.21.15.66:3300/0 conn(0x7f839006cb90 0x7f839015ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8394e00640 1 -- 172.21.15.66:0/3886947256 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839015d250 con 0x7f839015c6b0 2024-03-30T00:39:45.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.326+0000 7f8395800640 1 --2- 172.21.15.66:0/3886947256 >> v2:172.21.15.156:3300/0 conn(0x7f83900cdef0 0x7f83900ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8394e00640 1 --2- 172.21.15.66:0/3886947256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839015c6b0 0x7f83900d0c50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f83900cf9a0 tx=0x7f838001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5182e96a0d4a10d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:45.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f838fe00640 1 -- 172.21.15.66:0/3886947256 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83800ab600 con 0x7f839015c6b0 2024-03-30T00:39:45.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f838fe00640 1 -- 172.21.15.66:0/3886947256 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83800a7050 con 0x7f839015c6b0 2024-03-30T00:39:45.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f838fe00640 1 -- 172.21.15.66:0/3886947256 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83800ab980 con 0x7f839015c6b0 2024-03-30T00:39:45.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 -- 172.21.15.66:0/3886947256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839015c6b0 msgr2=0x7f83900d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 --2- 172.21.15.66:0/3886947256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839015c6b0 0x7f83900d0c50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f83900cf9a0 tx=0x7f838001c450 comp rx=0 tx=0).stop 2024-03-30T00:39:45.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 -- 172.21.15.66:0/3886947256 shutdown_connections 2024-03-30T00:39:45.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 --2- 172.21.15.66:0/3886947256 >> v2:172.21.15.156:3300/0 conn(0x7f83900cdef0 0x7f83900ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 --2- 172.21.15.66:0/3886947256 >> v2:172.21.15.66:3300/0 conn(0x7f839006cb90 0x7f839015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 --2- 172.21.15.66:0/3886947256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839015c6b0 0x7f83900d0c50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 -- 172.21.15.66:0/3886947256 >> 172.21.15.66:0/3886947256 conn(0x7f8390069d90 msgr2=0x7f839006a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:45.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 -- 172.21.15.66:0/3886947256 shutdown_connections 2024-03-30T00:39:45.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 -- 172.21.15.66:0/3886947256 wait complete. 2024-03-30T00:39:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 Processor -- start 2024-03-30T00:39:45.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 -- start start 2024-03-30T00:39:45.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839006cb90 0x7f83900b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83900cdef0 0x7f83900b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839015c6b0 0x7f83900b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83900b1b30 con 0x7f839015c6b0 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8390154c00 con 0x7f83900cdef0 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.330+0000 7f8396200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8390154080 con 0x7f839006cb90 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8394e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839006cb90 0x7f83900b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8395800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839015c6b0 0x7f83900b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8395800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839015c6b0 0x7f83900b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42116/0 (socket says 172.21.15.66:42116) 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8395800640 1 -- 172.21.15.66:0/2462102443 learned_addr learned my addr 172.21.15.66:0/2462102443 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8387e00640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83900cdef0 0x7f83900b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8394e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839006cb90 0x7f83900b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:34182/0 (socket says 172.21.15.66:34182) 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8395800640 1 -- 172.21.15.66:0/2462102443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839006cb90 msgr2=0x7f83900b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8395800640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839006cb90 0x7f83900b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8395800640 1 -- 172.21.15.66:0/2462102443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83900cdef0 msgr2=0x7f83900b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8395800640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83900cdef0 0x7f83900b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8395800640 1 -- 172.21.15.66:0/2462102443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8388009a70 con 0x7f839015c6b0 2024-03-30T00:39:45.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8394e00640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839006cb90 0x7f83900b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:45.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8395800640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839015c6b0 0x7f83900b15f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f838800efc0 tx=0x7f8388009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:45.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f838d600640 1 -- 172.21.15.66:0/2462102443 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388009d70 con 0x7f839015c6b0 2024-03-30T00:39:45.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f838d600640 1 -- 172.21.15.66:0/2462102443 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838800f040 con 0x7f839015c6b0 2024-03-30T00:39:45.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f838d600640 1 -- 172.21.15.66:0/2462102443 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83880039a0 con 0x7f839015c6b0 2024-03-30T00:39:45.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8396200640 1 -- 172.21.15.66:0/2462102443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8380009420 con 0x7f839015c6b0 2024-03-30T00:39:45.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8396200640 1 -- 172.21.15.66:0/2462102443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8390154640 con 0x7f839015c6b0 2024-03-30T00:39:45.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f8396200640 1 -- 172.21.15.66:0/2462102443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8390073e00 con 0x7f839015c6b0 2024-03-30T00:39:45.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.334+0000 7f838d600640 1 -- 172.21.15.66:0/2462102443 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8388014480 con 0x7f839015c6b0 2024-03-30T00:39:45.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.338+0000 7f838d600640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f836c0666c0 0x7f836c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.338+0000 7f838d600640 1 -- 172.21.15.66:0/2462102443 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8388084960 con 0x7f839015c6b0 2024-03-30T00:39:45.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.338+0000 7f8394e00640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f836c0666c0 0x7f836c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.338+0000 7f8394e00640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f836c0666c0 0x7f836c068b80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f83800ad000 tx=0x7f8380008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:45.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.342+0000 7f838d600640 1 -- 172.21.15.66:0/2462102443 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f83880520c0 con 0x7f839015c6b0 2024-03-30T00:39:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 -- 172.21.15.66:0/2462102443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83900b62a0 con 0x7f839015c6b0 2024-03-30T00:39:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f838d600640 1 -- 172.21.15.66:0/2462102443 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83880180a0 con 0x7f839015c6b0 2024-03-30T00:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:39:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 -- 172.21.15.66:0/2462102443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f836c0666c0 msgr2=0x7f836c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f836c0666c0 0x7f836c068b80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f83800ad000 tx=0x7f8380008040 comp rx=0 tx=0).stop 2024-03-30T00:39:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 -- 172.21.15.66:0/2462102443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839015c6b0 msgr2=0x7f83900b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839015c6b0 0x7f83900b15f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f838800efc0 tx=0x7f8388009f90 comp rx=0 tx=0).stop 2024-03-30T00:39:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 -- 172.21.15.66:0/2462102443 shutdown_connections 2024-03-30T00:39:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f836c0666c0 0x7f836c068b80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839015c6b0 0x7f83900b15f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83900cdef0 0x7f83900b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 --2- 172.21.15.66:0/2462102443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f839006cb90 0x7f83900b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 -- 172.21.15.66:0/2462102443 >> 172.21.15.66:0/2462102443 conn(0x7f8390069d90 msgr2=0x7f83900b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 -- 172.21.15.66:0/2462102443 shutdown_connections 2024-03-30T00:39:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.506+0000 7f8396200640 1 -- 172.21.15.66:0/2462102443 wait complete. 2024-03-30T00:39:45.534 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:39:45.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.666+0000 7f01df200640 1 Processor -- start 2024-03-30T00:39:45.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.666+0000 7f01df200640 1 -- start start 2024-03-30T00:39:45.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.666+0000 7f01df200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d815a670 0x7f01d80b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.666+0000 7f01df200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f01d80ce710 0x7f01d80ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.666+0000 7f01df200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f01d80cd330 0x7f01d80d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.666+0000 7f01df200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d80cb520 con 0x7f01d80cd330 2024-03-30T00:39:45.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.666+0000 7f01df200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d80cd7d0 con 0x7f01d80ce710 2024-03-30T00:39:45.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.666+0000 7f01df200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80cdab0 con 0x7f01d815a670 2024-03-30T00:39:45.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.666+0000 7f01dd200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f01d80cd330 0x7f01d80d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01dd200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f01d80cd330 0x7f01d80d01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42122/0 (socket says 172.21.15.66:42122) 2024-03-30T00:39:45.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01dd200640 1 -- 172.21.15.66:0/2162770582 learned_addr learned my addr 172.21.15.66:0/2162770582 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:45.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01d7e00640 1 --2- 172.21.15.66:0/2162770582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d815a670 0x7f01d80b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01cf400640 1 --2- 172.21.15.66:0/2162770582 >> v2:172.21.15.156:3300/0 conn(0x7f01d80ce710 0x7f01d80ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01dd200640 1 -- 172.21.15.66:0/2162770582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d815a670 msgr2=0x7f01d80b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:45.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01dd200640 1 --2- 172.21.15.66:0/2162770582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d815a670 0x7f01d80b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01dd200640 1 -- 172.21.15.66:0/2162770582 >> v2:172.21.15.156:3300/0 conn(0x7f01d80ce710 msgr2=0x7f01d80ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:45.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01dd200640 1 --2- 172.21.15.66:0/2162770582 >> v2:172.21.15.156:3300/0 conn(0x7f01d80ce710 0x7f01d80ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01dd200640 1 -- 172.21.15.66:0/2162770582 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80cdd90 con 0x7f01d80cd330 2024-03-30T00:39:45.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01dd200640 1 --2- 172.21.15.66:0/2162770582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d80cd330 0x7f01d80d01d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f01d80ce110 tx=0x7f01d001cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d449cf5d25ee41 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:45.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01d7400640 1 -- 172.21.15.66:0/2162770582 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00aa430 con 0x7f01d80cd330 2024-03-30T00:39:45.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01d7400640 1 -- 172.21.15.66:0/2162770582 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d00aa590 con 0x7f01d80cd330 2024-03-30T00:39:45.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.670+0000 7f01d7400640 1 -- 172.21.15.66:0/2162770582 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00aa920 con 0x7f01d80cd330 2024-03-30T00:39:45.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 -- 172.21.15.66:0/2162770582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d80cd330 msgr2=0x7f01d80d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 --2- 172.21.15.66:0/2162770582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d80cd330 0x7f01d80d01d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f01d80ce110 tx=0x7f01d001cf50 comp rx=0 tx=0).stop 2024-03-30T00:39:45.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 -- 172.21.15.66:0/2162770582 shutdown_connections 2024-03-30T00:39:45.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 --2- 172.21.15.66:0/2162770582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d80cd330 0x7f01d80d01d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 --2- 172.21.15.66:0/2162770582 >> v2:172.21.15.156:3300/0 conn(0x7f01d80ce710 0x7f01d80ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 --2- 172.21.15.66:0/2162770582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d815a670 0x7f01d80b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 -- 172.21.15.66:0/2162770582 >> 172.21.15.66:0/2162770582 conn(0x7f01d8099870 msgr2=0x7f01d8089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:45.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 -- 172.21.15.66:0/2162770582 shutdown_connections 2024-03-30T00:39:45.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 -- 172.21.15.66:0/2162770582 wait complete. 2024-03-30T00:39:45.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 Processor -- start 2024-03-30T00:39:45.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 -- start start 2024-03-30T00:39:45.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d80cd330 0x7f01d81543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d80ce710 0x7f01d806c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d815a670 0x7f01d806c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80c9ac0 con 0x7f01d815a670 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80c9220 con 0x7f01d80ce710 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01df200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80c9500 con 0x7f01d80cd330 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01dd200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d815a670 0x7f01d806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01dd200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d815a670 0x7f01d806c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42138/0 (socket says 172.21.15.66:42138) 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01dd200640 1 -- 172.21.15.66:0/240453107 learned_addr learned my addr 172.21.15.66:0/240453107 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01cf400640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d80ce710 0x7f01d806c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01d7e00640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d80cd330 0x7f01d81543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01dd200640 1 -- 172.21.15.66:0/240453107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d80cd330 msgr2=0x7f01d81543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01dd200640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d80cd330 0x7f01d81543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01dd200640 1 -- 172.21.15.66:0/240453107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d80ce710 msgr2=0x7f01d806c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01dd200640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d80ce710 0x7f01d806c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01dd200640 1 -- 172.21.15.66:0/240453107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d00094d0 con 0x7f01d815a670 2024-03-30T00:39:45.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01d7e00640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d80cd330 0x7f01d81543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:45.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.674+0000 7f01cf400640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d80ce710 0x7f01d806c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:45.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.678+0000 7f01dd200640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d815a670 0x7f01d806c560 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01d001aba0 tx=0x7f01d001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:45.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.678+0000 7f01d4c00640 1 -- 172.21.15.66:0/240453107 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00aa430 con 0x7f01d815a670 2024-03-30T00:39:45.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.678+0000 7f01d4c00640 1 -- 172.21.15.66:0/240453107 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d00b3800 con 0x7f01d815a670 2024-03-30T00:39:45.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.678+0000 7f01d4c00640 1 -- 172.21.15.66:0/240453107 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00b3bb0 con 0x7f01d815a670 2024-03-30T00:39:45.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.678+0000 7f01df200640 1 -- 172.21.15.66:0/240453107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d815abd0 con 0x7f01d815a670 2024-03-30T00:39:45.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.678+0000 7f01df200640 1 -- 172.21.15.66:0/240453107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d80c41b0 con 0x7f01d815a670 2024-03-30T00:39:45.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.678+0000 7f01df200640 1 -- 172.21.15.66:0/240453107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d8066c30 con 0x7f01d815a670 2024-03-30T00:39:45.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.678+0000 7f01d4c00640 1 -- 172.21.15.66:0/240453107 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f01d00b3d10 con 0x7f01d815a670 2024-03-30T00:39:45.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.678+0000 7f01d4c00640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01b4066690 0x7f01b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:45.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.678+0000 7f01d4c00640 1 -- 172.21.15.66:0/240453107 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f01d011e1a0 con 0x7f01d815a670 2024-03-30T00:39:45.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.678+0000 7f01d7e00640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01b4066690 0x7f01b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:45.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.682+0000 7f01d7e00640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01b4066690 0x7f01b4068b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01d80affa0 tx=0x7f01c800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:45.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.682+0000 7f01d4c00640 1 -- 172.21.15.66:0/240453107 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f01d00eb980 con 0x7f01d815a670 2024-03-30T00:39:45.848 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:39:45.852 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:39:45.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.878+0000 7f01df200640 1 -- 172.21.15.66:0/240453107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01d80c4370 con 0x7f01d815a670 2024-03-30T00:39:45.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.878+0000 7f01d4c00640 1 -- 172.21.15.66:0/240453107 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7f01d00eefa0 con 0x7f01d815a670 2024-03-30T00:39:45.888 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:39:45.888 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-30T00:39:45.888 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 2 2024-03-30T00:39:45.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 -- 172.21.15.66:0/240453107 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01b4066690 msgr2=0x7f01b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01b4066690 0x7f01b4068b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01d80affa0 tx=0x7f01c800b040 comp rx=0 tx=0).stop 2024-03-30T00:39:45.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 -- 172.21.15.66:0/240453107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d815a670 msgr2=0x7f01d806c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d815a670 0x7f01d806c560 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01d001aba0 tx=0x7f01d001ad30 comp rx=0 tx=0).stop 2024-03-30T00:39:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 -- 172.21.15.66:0/240453107 shutdown_connections 2024-03-30T00:39:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01b4066690 0x7f01b4068b50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01d815a670 0x7f01d806c560 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d80ce710 0x7f01d806c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 --2- 172.21.15.66:0/240453107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d80cd330 0x7f01d81543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 -- 172.21.15.66:0/240453107 >> 172.21.15.66:0/240453107 conn(0x7f01d8099870 msgr2=0x7f01d80b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 -- 172.21.15.66:0/240453107 shutdown_connections 2024-03-30T00:39:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:45.882+0000 7f01df200640 1 -- 172.21.15.66:0/240453107 wait complete. 2024-03-30T00:39:47.910 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:39:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448c400640 1 Processor -- start 2024-03-30T00:39:48.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448c400640 1 -- start start 2024-03-30T00:39:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448c400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cf2f0 0x7f448406b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448c400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44840cb570 0x7f44840cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448c400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44840cb0e0 con 0x7f448415ca80 2024-03-30T00:39:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448c400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44840cf7f0 con 0x7f44840cb570 2024-03-30T00:39:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448c400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44840cfad0 con 0x7f44840cf2f0 2024-03-30T00:39:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448a400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44840cb570 0x7f44840cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448a400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44840cb570 0x7f44840cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42152/0 (socket says 172.21.15.66:42152) 2024-03-30T00:39:48.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448a400640 1 -- 172.21.15.66:0/2746254125 learned_addr learned my addr 172.21.15.66:0/2746254125 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f4489000640 1 --2- 172.21.15.66:0/2746254125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cf2f0 0x7f448406b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448a400640 1 -- 172.21.15.66:0/2746254125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cf2f0 msgr2=0x7f448406b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448a400640 1 --2- 172.21.15.66:0/2746254125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cf2f0 0x7f448406b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448a400640 1 -- 172.21.15.66:0/2746254125 >> v1:172.21.15.156:6789/0 conn(0x7f448415ca80 legacy=0x7f448406b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.054+0000 7f448a400640 1 -- 172.21.15.66:0/2746254125 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44840cad70 con 0x7f44840cb570 2024-03-30T00:39:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f448a400640 1 --2- 172.21.15.66:0/2746254125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44840cb570 0x7f44840cba20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f447c009e50 tx=0x7f447c01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e12bc1d47e6c78 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f4483e00640 1 -- 172.21.15.66:0/2746254125 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0af400 con 0x7f44840cb570 2024-03-30T00:39:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f4483e00640 1 -- 172.21.15.66:0/2746254125 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c0ab050 con 0x7f44840cb570 2024-03-30T00:39:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f4483e00640 1 -- 172.21.15.66:0/2746254125 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0af780 con 0x7f44840cb570 2024-03-30T00:39:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f448c400640 1 -- 172.21.15.66:0/2746254125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44840cb570 msgr2=0x7f44840cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f448c400640 1 --2- 172.21.15.66:0/2746254125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44840cb570 0x7f44840cba20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f447c009e50 tx=0x7f447c01f210 comp rx=0 tx=0).stop 2024-03-30T00:39:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f448c400640 1 -- 172.21.15.66:0/2746254125 shutdown_connections 2024-03-30T00:39:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f448c400640 1 --2- 172.21.15.66:0/2746254125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44840cb570 0x7f44840cba20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f448c400640 1 --2- 172.21.15.66:0/2746254125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cf2f0 0x7f448406b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f448c400640 1 -- 172.21.15.66:0/2746254125 >> 172.21.15.66:0/2746254125 conn(0x7f44840b2090 msgr2=0x7f44840b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f448c400640 1 -- 172.21.15.66:0/2746254125 shutdown_connections 2024-03-30T00:39:48.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.058+0000 7f448c400640 1 -- 172.21.15.66:0/2746254125 wait complete. 2024-03-30T00:39:48.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448c400640 1 Processor -- start 2024-03-30T00:39:48.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448c400640 1 -- start start 2024-03-30T00:39:48.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448c400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44840cb570 0x7f44840b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448c400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cf2f0 0x7f448415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448c400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448415ca80 0x7f448409bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448c400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448409b730 con 0x7f448415ca80 2024-03-30T00:39:48.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448c400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44840c99d0 con 0x7f44840cb570 2024-03-30T00:39:48.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448c400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44840b1df0 con 0x7f44840cf2f0 2024-03-30T00:39:48.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448415ca80 0x7f448409bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448415ca80 0x7f448409bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42164/0 (socket says 172.21.15.66:42164) 2024-03-30T00:39:48.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448a400640 1 -- 172.21.15.66:0/4043282629 learned_addr learned my addr 172.21.15.66:0/4043282629 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:48.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f4489a00640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44840cb570 0x7f44840b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f4489000640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cf2f0 0x7f448415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448a400640 1 -- 172.21.15.66:0/4043282629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cf2f0 msgr2=0x7f448415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448a400640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cf2f0 0x7f448415adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448a400640 1 -- 172.21.15.66:0/4043282629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44840cb570 msgr2=0x7f44840b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448a400640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44840cb570 0x7f44840b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448a400640 1 -- 172.21.15.66:0/4043282629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c009b30 con 0x7f448415ca80 2024-03-30T00:39:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f4489000640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cf2f0 0x7f448415adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448a400640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448415ca80 0x7f448409bd10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f447c0b2040 tx=0x7f447c01bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f4481600640 1 -- 172.21.15.66:0/4043282629 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0afd30 con 0x7f448415ca80 2024-03-30T00:39:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f4481600640 1 -- 172.21.15.66:0/4043282629 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c009510 con 0x7f448415ca80 2024-03-30T00:39:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f4481600640 1 -- 172.21.15.66:0/4043282629 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0ba660 con 0x7f448415ca80 2024-03-30T00:39:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448c400640 1 -- 172.21.15.66:0/4043282629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44840c9c20 con 0x7f448415ca80 2024-03-30T00:39:48.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448c400640 1 -- 172.21.15.66:0/4043282629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448409b3c0 con 0x7f448415ca80 2024-03-30T00:39:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.062+0000 7f448c400640 1 -- 172.21.15.66:0/4043282629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448409b8e0 con 0x7f448415ca80 2024-03-30T00:39:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.066+0000 7f4481600640 1 -- 172.21.15.66:0/4043282629 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f447c0b6040 con 0x7f448415ca80 2024-03-30T00:39:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.066+0000 7f4481600640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44640666c0 0x7f4464068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.066+0000 7f4481600640 1 -- 172.21.15.66:0/4043282629 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f447c121f60 con 0x7f448415ca80 2024-03-30T00:39:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.066+0000 7f4489a00640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44640666c0 0x7f4464068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.066+0000 7f4489a00640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44640666c0 0x7f4464068b80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f44840b6110 tx=0x7f4478009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:48.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.070+0000 7f4481600640 1 -- 172.21.15.66:0/4043282629 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f448409b8e0 con 0x7f448415ca80 2024-03-30T00:39:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 -- 172.21.15.66:0/4043282629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f448409b8e0 con 0x7f448415ca80 2024-03-30T00:39:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f4481600640 1 -- 172.21.15.66:0/4043282629 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7f448409b8e0 con 0x7f448415ca80 2024-03-30T00:39:48.274 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:39:48.275 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4122431016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":4122431016},{"type":"v1","addr":"172.21.15.156:6835","nonce":4122431016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3882342907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3882342907},{"type":"v1","addr":"172.21.15.66:6835","nonce":3882342907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/703895528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":703895528},{"type":"v1","addr":"172.21.15.66:6837","nonce":703895528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-30T00:39:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 3 2024-03-30T00:39:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 -- 172.21.15.66:0/4043282629 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44640666c0 msgr2=0x7f4464068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44640666c0 0x7f4464068b80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f44840b6110 tx=0x7f4478009040 comp rx=0 tx=0).stop 2024-03-30T00:39:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 -- 172.21.15.66:0/4043282629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448415ca80 msgr2=0x7f448409bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448415ca80 0x7f448409bd10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f447c0b2040 tx=0x7f447c01bfb0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 -- 172.21.15.66:0/4043282629 shutdown_connections 2024-03-30T00:39:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44640666c0 0x7f4464068b80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f448415ca80 0x7f448409bd10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cf2f0 0x7f448415adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 --2- 172.21.15.66:0/4043282629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44840cb570 0x7f44840b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 -- 172.21.15.66:0/4043282629 >> 172.21.15.66:0/4043282629 conn(0x7f44840b2090 msgr2=0x7f448406ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:48.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 -- 172.21.15.66:0/4043282629 shutdown_connections 2024-03-30T00:39:48.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.266+0000 7f448c400640 1 -- 172.21.15.66:0/4043282629 wait complete. 2024-03-30T00:39:48.296 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:39:48.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7aaa00640 1 Processor -- start 2024-03-30T00:39:48.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7aaa00640 1 -- start start 2024-03-30T00:39:48.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7aaa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7a4160490 0x7fb7a4160870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7aaa00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb7a40b0290 0x7fb7a40b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7aaa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7a415c200 0x7fb7a415c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7aaa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a40b15d0 con 0x7fb7a4160490 2024-03-30T00:39:48.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7aaa00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a40afdf0 con 0x7fb7a40b0290 2024-03-30T00:39:48.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7aaa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4160db0 con 0x7fb7a415c200 2024-03-30T00:39:48.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7a3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7a4160490 0x7fb7a4160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7a3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7a4160490 0x7fb7a4160870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42172/0 (socket says 172.21.15.66:42172) 2024-03-30T00:39:48.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7a3e00640 1 -- 172.21.15.66:0/1078919972 learned_addr learned my addr 172.21.15.66:0/1078919972 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:48.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7a8a00640 1 --2- 172.21.15.66:0/1078919972 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7a415c200 0x7fb7a415c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb79b400640 1 --2- 172.21.15.66:0/1078919972 >> v2:172.21.15.156:3300/0 conn(0x7fb7a40b0290 0x7fb7a40b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7a3e00640 1 -- 172.21.15.66:0/1078919972 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7a415c200 msgr2=0x7fb7a415c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:48.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7a3e00640 1 --2- 172.21.15.66:0/1078919972 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7a415c200 0x7fb7a415c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7a3e00640 1 -- 172.21.15.66:0/1078919972 >> v2:172.21.15.156:3300/0 conn(0x7fb7a40b0290 msgr2=0x7fb7a40b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:48.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7a3e00640 1 --2- 172.21.15.66:0/1078919972 >> v2:172.21.15.156:3300/0 conn(0x7fb7a40b0290 0x7fb7a40b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb7a3e00640 1 -- 172.21.15.66:0/1078919972 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a40b1010 con 0x7fb7a4160490 2024-03-30T00:39:48.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.426+0000 7fb79b400640 1 --2- 172.21.15.66:0/1078919972 >> v2:172.21.15.156:3300/0 conn(0x7fb7a40b0290 0x7fb7a40b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:39:48.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7a3e00640 1 --2- 172.21.15.66:0/1078919972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7a4160490 0x7fb7a4160870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7a406d060 tx=0x7fb79401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e15877d16b6b41 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7a3400640 1 -- 172.21.15.66:0/1078919972 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940aa610 con 0x7fb7a4160490 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7a3400640 1 -- 172.21.15.66:0/1078919972 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7940a6040 con 0x7fb7a4160490 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7a3400640 1 -- 172.21.15.66:0/1078919972 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940aa990 con 0x7fb7a4160490 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7aaa00640 1 -- 172.21.15.66:0/1078919972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7a4160490 msgr2=0x7fb7a4160870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7aaa00640 1 --2- 172.21.15.66:0/1078919972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7a4160490 0x7fb7a4160870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7a406d060 tx=0x7fb79401c4f0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7aaa00640 1 -- 172.21.15.66:0/1078919972 shutdown_connections 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7aaa00640 1 --2- 172.21.15.66:0/1078919972 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7a415c200 0x7fb7a415c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7aaa00640 1 --2- 172.21.15.66:0/1078919972 >> v2:172.21.15.156:3300/0 conn(0x7fb7a40b0290 0x7fb7a40b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7aaa00640 1 --2- 172.21.15.66:0/1078919972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7a4160490 0x7fb7a4160870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7aaa00640 1 -- 172.21.15.66:0/1078919972 >> 172.21.15.66:0/1078919972 conn(0x7fb7a4099870 msgr2=0x7fb7a4089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7aaa00640 1 -- 172.21.15.66:0/1078919972 shutdown_connections 2024-03-30T00:39:48.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.430+0000 7fb7aaa00640 1 -- 172.21.15.66:0/1078919972 wait complete. 2024-03-30T00:39:48.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7aaa00640 1 Processor -- start 2024-03-30T00:39:48.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7aaa00640 1 -- start start 2024-03-30T00:39:48.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7aaa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7a40b0290 0x7fb7a4151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7aaa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7a415c200 0x7fb7a4154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7aaa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a4160490 0x7fb7a4156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7aaa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a41558f0 con 0x7fb7a415c200 2024-03-30T00:39:48.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7aaa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4154c70 con 0x7fb7a4160490 2024-03-30T00:39:48.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7aaa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4154f50 con 0x7fb7a40b0290 2024-03-30T00:39:48.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb79b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7a415c200 0x7fb7a4154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb79b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7a415c200 0x7fb7a4154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42176/0 (socket says 172.21.15.66:42176) 2024-03-30T00:39:48.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb79b400640 1 -- 172.21.15.66:0/2765292402 learned_addr learned my addr 172.21.15.66:0/2765292402 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:48.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb79b400640 1 -- 172.21.15.66:0/2765292402 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7a40b0290 msgr2=0x7fb7a4151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb79b400640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7a40b0290 0x7fb7a4151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb79b400640 1 -- 172.21.15.66:0/2765292402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a4160490 msgr2=0x7fb7a4156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb79b400640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a4160490 0x7fb7a4156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb79b400640 1 -- 172.21.15.66:0/2765292402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7940094d0 con 0x7fb7a415c200 2024-03-30T00:39:48.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb79b400640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7a415c200 0x7fb7a4154420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00c7c0 tx=0x7fb78c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:48.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7a0c00640 1 -- 172.21.15.66:0/2765292402 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c007a30 con 0x7fb7a415c200 2024-03-30T00:39:48.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7a0c00640 1 -- 172.21.15.66:0/2765292402 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c007b90 con 0x7fb7a415c200 2024-03-30T00:39:48.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7aaa00640 1 -- 172.21.15.66:0/2765292402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a40b0de0 con 0x7fb7a415c200 2024-03-30T00:39:48.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7a0c00640 1 -- 172.21.15.66:0/2765292402 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c00ce70 con 0x7fb7a415c200 2024-03-30T00:39:48.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.434+0000 7fb7aaa00640 1 -- 172.21.15.66:0/2765292402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a4155510 con 0x7fb7a415c200 2024-03-30T00:39:48.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.438+0000 7fb7aaa00640 1 -- 172.21.15.66:0/2765292402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a4066c30 con 0x7fb7a415c200 2024-03-30T00:39:48.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.438+0000 7fb7a0c00640 1 -- 172.21.15.66:0/2765292402 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb78c0113f0 con 0x7fb7a415c200 2024-03-30T00:39:48.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.438+0000 7fb7a0c00640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7800666c0 0x7fb780068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.438+0000 7fb7a0c00640 1 -- 172.21.15.66:0/2765292402 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb78c083fd0 con 0x7fb7a415c200 2024-03-30T00:39:48.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.438+0000 7fb7a3e00640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7800666c0 0x7fb780068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.438+0000 7fb7a3e00640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7800666c0 0x7fb780068b80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb7940ae000 tx=0x7fb794008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:48.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.442+0000 7fb7a0c00640 1 -- 172.21.15.66:0/2765292402 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb78c051680 con 0x7fb7a415c200 2024-03-30T00:39:48.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.606+0000 7fb7aaa00640 1 -- 172.21.15.66:0/2765292402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7a415bfe0 con 0x7fb7a415c200 2024-03-30T00:39:48.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.606+0000 7fb7a0c00640 1 -- 172.21.15.66:0/2765292402 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12382 (secure 0 0 0) 0x7fb78c01f090 con 0x7fb7a415c200 2024-03-30T00:39:48.616 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:39:48.616 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:42.625906+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:39:48.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 -- 172.21.15.66:0/2765292402 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7800666c0 msgr2=0x7fb780068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7800666c0 0x7fb780068b80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb7940ae000 tx=0x7fb794008040 comp rx=0 tx=0).stop 2024-03-30T00:39:48.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 -- 172.21.15.66:0/2765292402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7a415c200 msgr2=0x7fb7a4154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7a415c200 0x7fb7a4154420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00c7c0 tx=0x7fb78c00cc80 comp rx=0 tx=0).stop 2024-03-30T00:39:48.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 -- 172.21.15.66:0/2765292402 shutdown_connections 2024-03-30T00:39:48.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7a4160490 0x7fb7a4156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7800666c0 0x7fb780068b80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7a415c200 0x7fb7a4154420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 --2- 172.21.15.66:0/2765292402 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7a40b0290 0x7fb7a4151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 -- 172.21.15.66:0/2765292402 >> 172.21.15.66:0/2765292402 conn(0x7fb7a4099870 msgr2=0x7fb7a406e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:48.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 -- 172.21.15.66:0/2765292402 shutdown_connections 2024-03-30T00:39:48.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.610+0000 7fb7aaa00640 1 -- 172.21.15.66:0/2765292402 wait complete. 2024-03-30T00:39:48.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:39:48.638 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:39:48.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.770+0000 7f557f400640 1 Processor -- start 2024-03-30T00:39:48.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.770+0000 7f557f400640 1 -- start start 2024-03-30T00:39:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f557f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578099060 0x7f55780aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f557f400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55780b0470 0x7f55780cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f557f400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5578099760 con 0x7f5578098c80 2024-03-30T00:39:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f557f400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55780cb8f0 con 0x7f55780b0470 2024-03-30T00:39:48.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f557f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55780999b0 con 0x7f5578099060 2024-03-30T00:39:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f5577e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578099060 0x7f55780aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f557d400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55780b0470 0x7f55780cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f5577e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578099060 0x7f55780aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:36974/0 (socket says 172.21.15.66:36974) 2024-03-30T00:39:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f5577e00640 1 -- 172.21.15.66:0/3651100676 learned_addr learned my addr 172.21.15.66:0/3651100676 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f5577e00640 1 -- 172.21.15.66:0/3651100676 >> v2:172.21.15.156:3300/0 conn(0x7f55780b0470 msgr2=0x7f55780cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f5577e00640 1 --2- 172.21.15.66:0/3651100676 >> v2:172.21.15.156:3300/0 conn(0x7f55780b0470 0x7f55780cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f5577e00640 1 -- 172.21.15.66:0/3651100676 >> v1:172.21.15.156:6789/0 conn(0x7f5578098c80 legacy=0x7f55780ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.774+0000 7f5577e00640 1 -- 172.21.15.66:0/3651100676 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5578099c90 con 0x7f5578099060 2024-03-30T00:39:48.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f5577e00640 1 --2- 172.21.15.66:0/3651100676 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578099060 0x7f55780aff30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5564009870 tx=0x7f556401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c3bc09a1cfe957 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:48.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f5577400640 1 -- 172.21.15.66:0/3651100676 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55640ac410 con 0x7f5578099060 2024-03-30T00:39:48.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f5577400640 1 -- 172.21.15.66:0/3651100676 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55640ac570 con 0x7f5578099060 2024-03-30T00:39:48.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f5577400640 1 -- 172.21.15.66:0/3651100676 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55640ac900 con 0x7f5578099060 2024-03-30T00:39:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f557f400640 1 -- 172.21.15.66:0/3651100676 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578099060 msgr2=0x7f55780aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f557f400640 1 --2- 172.21.15.66:0/3651100676 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578099060 0x7f55780aff30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5564009870 tx=0x7f556401c4a0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f557f400640 1 -- 172.21.15.66:0/3651100676 shutdown_connections 2024-03-30T00:39:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f557f400640 1 --2- 172.21.15.66:0/3651100676 >> v2:172.21.15.156:3300/0 conn(0x7f55780b0470 0x7f55780cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f557f400640 1 --2- 172.21.15.66:0/3651100676 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578099060 0x7f55780aff30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f557f400640 1 -- 172.21.15.66:0/3651100676 >> 172.21.15.66:0/3651100676 conn(0x7f55780b20c0 msgr2=0x7f55780b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f557f400640 1 -- 172.21.15.66:0/3651100676 shutdown_connections 2024-03-30T00:39:48.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f557f400640 1 -- 172.21.15.66:0/3651100676 wait complete. 2024-03-30T00:39:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f557f400640 1 Processor -- start 2024-03-30T00:39:48.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.778+0000 7f557f400640 1 -- start start 2024-03-30T00:39:48.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578098c80 0x7f55780c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5578099060 0x7f55780c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55780b0470 0x7f5578157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55780c4b00 con 0x7f5578099060 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557f400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55780c8490 con 0x7f55780b0470 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55780c7ed0 con 0x7f5578098c80 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5577e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5578099060 0x7f55780c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5577e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5578099060 0x7f55780c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51280/0 (socket says 172.21.15.66:51280) 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5577e00640 1 -- 172.21.15.66:0/786508092 learned_addr learned my addr 172.21.15.66:0/786508092 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557d400640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55780b0470 0x7f5578157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557ca00640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578098c80 0x7f55780c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5577e00640 1 -- 172.21.15.66:0/786508092 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578098c80 msgr2=0x7f55780c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5577e00640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578098c80 0x7f55780c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5577e00640 1 -- 172.21.15.66:0/786508092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55780b0470 msgr2=0x7f5578157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5577e00640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55780b0470 0x7f5578157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5577e00640 1 -- 172.21.15.66:0/786508092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55640094d0 con 0x7f5578099060 2024-03-30T00:39:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557d400640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55780b0470 0x7f5578157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5577e00640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5578099060 0x7f55780c76a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f55640af040 tx=0x7f5564019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5574c00640 1 -- 172.21.15.66:0/786508092 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55640bb070 con 0x7f5578099060 2024-03-30T00:39:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5574c00640 1 -- 172.21.15.66:0/786508092 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55640ace70 con 0x7f5578099060 2024-03-30T00:39:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f5574c00640 1 -- 172.21.15.66:0/786508092 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55640ac410 con 0x7f5578099060 2024-03-30T00:39:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557f400640 1 -- 172.21.15.66:0/786508092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55780c81b0 con 0x7f5578099060 2024-03-30T00:39:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557f400640 1 -- 172.21.15.66:0/786508092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55780c7be0 con 0x7f5578099060 2024-03-30T00:39:48.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.782+0000 7f557f400640 1 -- 172.21.15.66:0/786508092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55780c4960 con 0x7f5578099060 2024-03-30T00:39:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.786+0000 7f5574c00640 1 -- 172.21.15.66:0/786508092 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f55640b65f0 con 0x7f5578099060 2024-03-30T00:39:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.786+0000 7f5574c00640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5558066690 0x7f5558068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:48.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.786+0000 7f5574c00640 1 -- 172.21.15.66:0/786508092 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f556411dda0 con 0x7f5578099060 2024-03-30T00:39:48.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.786+0000 7f557ca00640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5558066690 0x7f5558068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:48.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.786+0000 7f557ca00640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5558066690 0x7f5558068b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f556c007fd0 tx=0x7f556c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:48.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.790+0000 7f5574c00640 1 -- 172.21.15.66:0/786508092 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f55780c4960 con 0x7f5578099060 2024-03-30T00:39:48.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 -- 172.21.15.66:0/786508092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55780c4fe0 con 0x7f5578099060 2024-03-30T00:39:48.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f5574c00640 1 -- 172.21.15.66:0/786508092 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55780c4fe0 con 0x7f5578099060 2024-03-30T00:39:48.958 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:39:48.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 -- 172.21.15.66:0/786508092 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5558066690 msgr2=0x7f5558068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5558066690 0x7f5558068b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f556c007fd0 tx=0x7f556c009040 comp rx=0 tx=0).stop 2024-03-30T00:39:48.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 -- 172.21.15.66:0/786508092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5578099060 msgr2=0x7f55780c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:48.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5578099060 0x7f55780c76a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f55640af040 tx=0x7f5564019240 comp rx=0 tx=0).stop 2024-03-30T00:39:48.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 -- 172.21.15.66:0/786508092 shutdown_connections 2024-03-30T00:39:48.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55780b0470 0x7f5578157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5558066690 0x7f5558068b50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5578099060 0x7f55780c76a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 --2- 172.21.15.66:0/786508092 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5578098c80 0x7f55780c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:48.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 -- 172.21.15.66:0/786508092 >> 172.21.15.66:0/786508092 conn(0x7f55780b20c0 msgr2=0x7f55780aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:48.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 -- 172.21.15.66:0/786508092 shutdown_connections 2024-03-30T00:39:48.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:48.950+0000 7f557f400640 1 -- 172.21.15.66:0/786508092 wait complete. 2024-03-30T00:39:48.979 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T00:39:49.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f8400640 1 Processor -- start 2024-03-30T00:39:49.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f8400640 1 -- start start 2024-03-30T00:39:49.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f8400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06f00cda90 0x7f06f00ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f8400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f06f00cde70 0x7f06f00cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f8400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f06f0159c60 0x7f06f015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f8400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f00b1b80 con 0x7f06f00cde70 2024-03-30T00:39:49.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f8400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f015a990 con 0x7f06f0159c60 2024-03-30T00:39:49.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f8400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00b11e0 con 0x7f06f00cda90 2024-03-30T00:39:49.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f5000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f06f00cde70 0x7f06f00cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f5000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f06f00cde70 0x7f06f00cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51296/0 (socket says 172.21.15.66:51296) 2024-03-30T00:39:49.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f5000640 1 -- 172.21.15.66:0/330679609 learned_addr learned my addr 172.21.15.66:0/330679609 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:49.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f6400640 1 --2- 172.21.15.66:0/330679609 >> v2:172.21.15.156:3300/0 conn(0x7f06f0159c60 0x7f06f015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f5a00640 1 --2- 172.21.15.66:0/330679609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06f00cda90 0x7f06f00ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f5000640 1 -- 172.21.15.66:0/330679609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06f00cda90 msgr2=0x7f06f00ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:49.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f5000640 1 --2- 172.21.15.66:0/330679609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06f00cda90 0x7f06f00ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f5000640 1 -- 172.21.15.66:0/330679609 >> v2:172.21.15.156:3300/0 conn(0x7f06f0159c60 msgr2=0x7f06f015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:49.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f5000640 1 --2- 172.21.15.66:0/330679609 >> v2:172.21.15.156:3300/0 conn(0x7f06f0159c60 0x7f06f015a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.110+0000 7f06f5000640 1 -- 172.21.15.66:0/330679609 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f00b14c0 con 0x7f06f00cde70 2024-03-30T00:39:49.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06f5000640 1 --2- 172.21.15.66:0/330679609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f00cde70 0x7f06f00cea60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06f00724d0 tx=0x7f06dc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e4a024c00b47ce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:49.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06efe00640 1 -- 172.21.15.66:0/330679609 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0ab670 con 0x7f06f00cde70 2024-03-30T00:39:49.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06efe00640 1 -- 172.21.15.66:0/330679609 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc0ab7d0 con 0x7f06f00cde70 2024-03-30T00:39:49.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06efe00640 1 -- 172.21.15.66:0/330679609 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0abb60 con 0x7f06f00cde70 2024-03-30T00:39:49.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06f8400640 1 -- 172.21.15.66:0/330679609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f00cde70 msgr2=0x7f06f00cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06f8400640 1 --2- 172.21.15.66:0/330679609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f00cde70 0x7f06f00cea60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06f00724d0 tx=0x7f06dc01c490 comp rx=0 tx=0).stop 2024-03-30T00:39:49.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06f8400640 1 -- 172.21.15.66:0/330679609 shutdown_connections 2024-03-30T00:39:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06f8400640 1 --2- 172.21.15.66:0/330679609 >> v2:172.21.15.156:3300/0 conn(0x7f06f0159c60 0x7f06f015a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06f8400640 1 --2- 172.21.15.66:0/330679609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f00cde70 0x7f06f00cea60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06f8400640 1 --2- 172.21.15.66:0/330679609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06f00cda90 0x7f06f00ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06f8400640 1 -- 172.21.15.66:0/330679609 >> 172.21.15.66:0/330679609 conn(0x7f06f0069f60 msgr2=0x7f06f006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06f8400640 1 -- 172.21.15.66:0/330679609 shutdown_connections 2024-03-30T00:39:49.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.114+0000 7f06f8400640 1 -- 172.21.15.66:0/330679609 wait complete. 2024-03-30T00:39:49.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f8400640 1 Processor -- start 2024-03-30T00:39:49.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f8400640 1 -- start start 2024-03-30T00:39:49.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f8400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06f00cda90 0x7f06f00aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f8400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f00cde70 0x7f06f0071690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f8400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f0159c60 0x7f06f00c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f8400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00ac390 con 0x7f06f00cde70 2024-03-30T00:39:49.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f8400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00ac670 con 0x7f06f0159c60 2024-03-30T00:39:49.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f8400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00ac950 con 0x7f06f00cda90 2024-03-30T00:39:49.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f00cde70 0x7f06f0071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f00cde70 0x7f06f0071690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51310/0 (socket says 172.21.15.66:51310) 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f5000640 1 -- 172.21.15.66:0/1481617293 learned_addr learned my addr 172.21.15.66:0/1481617293 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f5a00640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06f00cda90 0x7f06f00aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f6400640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f0159c60 0x7f06f00c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f5000640 1 -- 172.21.15.66:0/1481617293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06f00cda90 msgr2=0x7f06f00aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f5000640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06f00cda90 0x7f06f00aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f5000640 1 -- 172.21.15.66:0/1481617293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f0159c60 msgr2=0x7f06f00c2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f5000640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f0159c60 0x7f06f00c2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f5000640 1 -- 172.21.15.66:0/1481617293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc0094d0 con 0x7f06f00cde70 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f5a00640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06f00cda90 0x7f06f00aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f5000640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f00cde70 0x7f06f0071690 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06dc01ab40 tx=0x7f06dc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06ed600640 1 -- 172.21.15.66:0/1481617293 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0b4800 con 0x7f06f00cde70 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06ed600640 1 -- 172.21.15.66:0/1481617293 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc0b4980 con 0x7f06f00cde70 2024-03-30T00:39:49.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06ed600640 1 -- 172.21.15.66:0/1481617293 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0b4d30 con 0x7f06f00cde70 2024-03-30T00:39:49.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f8400640 1 -- 172.21.15.66:0/1481617293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f00acc30 con 0x7f06f00cde70 2024-03-30T00:39:49.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.118+0000 7f06f8400640 1 -- 172.21.15.66:0/1481617293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f00abf60 con 0x7f06f00cde70 2024-03-30T00:39:49.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.122+0000 7f06f8400640 1 -- 172.21.15.66:0/1481617293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f00ac150 con 0x7f06f00cde70 2024-03-30T00:39:49.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.122+0000 7f06ed600640 1 -- 172.21.15.66:0/1481617293 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f06dc0b3380 con 0x7f06f00cde70 2024-03-30T00:39:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.122+0000 7f06ed600640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f06d0066690 0x7f06d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.122+0000 7f06ed600640 1 -- 172.21.15.66:0/1481617293 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f06dc11e4e0 con 0x7f06f00cde70 2024-03-30T00:39:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.122+0000 7f06f5a00640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f06d0066690 0x7f06d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.122+0000 7f06f5a00640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f06d0066690 0x7f06d0068b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f06e400ba50 tx=0x7f06e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:49.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.126+0000 7f06ed600640 1 -- 172.21.15.66:0/1481617293 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f06f00ac150 con 0x7f06f00cde70 2024-03-30T00:39:49.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.290+0000 7f06f8400640 1 -- 172.21.15.66:0/1481617293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f06f00ac150 con 0x7f06f00cde70 2024-03-30T00:39:49.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.290+0000 7f06ed600640 1 -- 172.21.15.66:0/1481617293 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f06f00ac150 con 0x7f06f00cde70 2024-03-30T00:39:49.299 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:39:49.300 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:39:49.300 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:39:49.300 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-03-30T00:39:49.300 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-03-30T00:39:49.300 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:39:49.300 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:39:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 -- 172.21.15.66:0/1481617293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f06d0066690 msgr2=0x7f06d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f06d0066690 0x7f06d0068b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f06e400ba50 tx=0x7f06e4008040 comp rx=0 tx=0).stop 2024-03-30T00:39:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 -- 172.21.15.66:0/1481617293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f00cde70 msgr2=0x7f06f0071690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f00cde70 0x7f06f0071690 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06dc01ab40 tx=0x7f06dc01acd0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 -- 172.21.15.66:0/1481617293 shutdown_connections 2024-03-30T00:39:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f0159c60 0x7f06f00c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f06d0066690 0x7f06d0068b50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f00cde70 0x7f06f0071690 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 --2- 172.21.15.66:0/1481617293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06f00cda90 0x7f06f00aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 -- 172.21.15.66:0/1481617293 >> 172.21.15.66:0/1481617293 conn(0x7f06f0069f60 msgr2=0x7f06f006ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 -- 172.21.15.66:0/1481617293 shutdown_connections 2024-03-30T00:39:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.294+0000 7f06f8400640 1 -- 172.21.15.66:0/1481617293 wait complete. 2024-03-30T00:39:49.321 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T00:39:49.322 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T00:39:49.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe718e00640 1 Processor -- start 2024-03-30T00:39:49.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe718e00640 1 -- start start 2024-03-30T00:39:49.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe718e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7140acaf0 0x7fe7140b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe718e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe71415ba00 0x7fe7140ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe718e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7140cbf80 con 0x7fe7140cecf0 2024-03-30T00:39:49.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe718e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7140cf180 con 0x7fe71415ba00 2024-03-30T00:39:49.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe718e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7140cc820 con 0x7fe7140acaf0 2024-03-30T00:39:49.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe709400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe71415ba00 0x7fe7140ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe709400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe71415ba00 0x7fe7140ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51312/0 (socket says 172.21.15.66:51312) 2024-03-30T00:39:49.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe709400640 1 -- 172.21.15.66:0/89706762 learned_addr learned my addr 172.21.15.66:0/89706762 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:49.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe711e00640 1 --2- 172.21.15.66:0/89706762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7140acaf0 0x7fe7140b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe709400640 1 -- 172.21.15.66:0/89706762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7140acaf0 msgr2=0x7fe7140b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:49.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe709400640 1 --2- 172.21.15.66:0/89706762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7140acaf0 0x7fe7140b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe709400640 1 -- 172.21.15.66:0/89706762 >> v1:172.21.15.66:6789/0 conn(0x7fe7140cecf0 legacy=0x7fe7141579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.458+0000 7fe709400640 1 -- 172.21.15.66:0/89706762 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7140b2dd0 con 0x7fe71415ba00 2024-03-30T00:39:49.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe709400640 1 --2- 172.21.15.66:0/89706762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe71415ba00 0x7fe7140ce7b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009870 tx=0x7fe6fc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67141de4971e4387 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:49.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe711400640 1 -- 172.21.15.66:0/89706762 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0ac600 con 0x7fe71415ba00 2024-03-30T00:39:49.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe711400640 1 -- 172.21.15.66:0/89706762 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6fc0a8050 con 0x7fe71415ba00 2024-03-30T00:39:49.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe711400640 1 -- 172.21.15.66:0/89706762 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0ac980 con 0x7fe71415ba00 2024-03-30T00:39:49.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe718e00640 1 -- 172.21.15.66:0/89706762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe71415ba00 msgr2=0x7fe7140ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe718e00640 1 --2- 172.21.15.66:0/89706762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe71415ba00 0x7fe7140ce7b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009870 tx=0x7fe6fc01c4a0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe718e00640 1 -- 172.21.15.66:0/89706762 shutdown_connections 2024-03-30T00:39:49.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe718e00640 1 --2- 172.21.15.66:0/89706762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe71415ba00 0x7fe7140ce7b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe718e00640 1 --2- 172.21.15.66:0/89706762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7140acaf0 0x7fe7140b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe718e00640 1 -- 172.21.15.66:0/89706762 >> 172.21.15.66:0/89706762 conn(0x7fe71406a190 msgr2=0x7fe71406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe718e00640 1 -- 172.21.15.66:0/89706762 shutdown_connections 2024-03-30T00:39:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.462+0000 7fe718e00640 1 -- 172.21.15.66:0/89706762 wait complete. 2024-03-30T00:39:49.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe718e00640 1 Processor -- start 2024-03-30T00:39:49.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe718e00640 1 -- start start 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe718e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7140acaf0 0x7fe714097080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe718e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7140cecf0 0x7fe71406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe718e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe71415ba00 0x7fe714154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe718e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7140975c0 con 0x7fe7140cecf0 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe718e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe714159490 con 0x7fe71415ba00 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe718e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe714159770 con 0x7fe7140acaf0 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe709400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7140cecf0 0x7fe71406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe709400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7140cecf0 0x7fe71406d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51314/0 (socket says 172.21.15.66:51314) 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe709400640 1 -- 172.21.15.66:0/55956034 learned_addr learned my addr 172.21.15.66:0/55956034 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe711e00640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7140acaf0 0x7fe714097080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe709400640 1 -- 172.21.15.66:0/55956034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7140acaf0 msgr2=0x7fe714097080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe709400640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7140acaf0 0x7fe714097080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe709400640 1 -- 172.21.15.66:0/55956034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe71415ba00 msgr2=0x7fe714154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe712800640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe71415ba00 0x7fe714154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe709400640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe71415ba00 0x7fe714154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe709400640 1 -- 172.21.15.66:0/55956034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc0094d0 con 0x7fe7140cecf0 2024-03-30T00:39:49.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe711e00640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7140acaf0 0x7fe714097080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:39:49.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe712800640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe71415ba00 0x7fe714154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:39:49.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe709400640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7140cecf0 0x7fe71406d5b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc01ab50 tx=0x7fe6fc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:49.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe70aa00640 1 -- 172.21.15.66:0/55956034 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0b5460 con 0x7fe7140cecf0 2024-03-30T00:39:49.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe70aa00640 1 -- 172.21.15.66:0/55956034 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6fc0b55c0 con 0x7fe7140cecf0 2024-03-30T00:39:49.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe718e00640 1 -- 172.21.15.66:0/55956034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe714159a50 con 0x7fe7140cecf0 2024-03-30T00:39:49.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe70aa00640 1 -- 172.21.15.66:0/55956034 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0b5950 con 0x7fe7140cecf0 2024-03-30T00:39:49.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe718e00640 1 -- 172.21.15.66:0/55956034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71415a010 con 0x7fe7140cecf0 2024-03-30T00:39:49.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.466+0000 7fe718e00640 1 -- 172.21.15.66:0/55956034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe714066c30 con 0x7fe7140cecf0 2024-03-30T00:39:49.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.470+0000 7fe70aa00640 1 -- 172.21.15.66:0/55956034 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe6fc0b3070 con 0x7fe7140cecf0 2024-03-30T00:39:49.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.470+0000 7fe70aa00640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6f0066690 0x7fe6f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.470+0000 7fe70aa00640 1 -- 172.21.15.66:0/55956034 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe6fc11e180 con 0x7fe7140cecf0 2024-03-30T00:39:49.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.470+0000 7fe711e00640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6f0066690 0x7fe6f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.470+0000 7fe711e00640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6f0066690 0x7fe6f0068b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe704009a40 tx=0x7fe704008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:49.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.474+0000 7fe70aa00640 1 -- 172.21.15.66:0/55956034 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe6fc0eb830 con 0x7fe7140cecf0 2024-03-30T00:39:49.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.634+0000 7fe718e00640 1 -- 172.21.15.66:0/55956034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe7140b2390 con 0x7fe7140cecf0 2024-03-30T00:39:49.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.734+0000 7fe70aa00640 1 -- 172.21.15.66:0/55956034 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe6fc0eee50 con 0x7fe7140cecf0 2024-03-30T00:39:49.744 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-03-30T00:39:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 -- 172.21.15.66:0/55956034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6f0066690 msgr2=0x7fe6f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6f0066690 0x7fe6f0068b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe704009a40 tx=0x7fe704008040 comp rx=0 tx=0).stop 2024-03-30T00:39:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 -- 172.21.15.66:0/55956034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7140cecf0 msgr2=0x7fe71406d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7140cecf0 0x7fe71406d5b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc01ab50 tx=0x7fe6fc019240 comp rx=0 tx=0).stop 2024-03-30T00:39:49.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 -- 172.21.15.66:0/55956034 shutdown_connections 2024-03-30T00:39:49.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe71415ba00 0x7fe714154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6f0066690 0x7fe6f0068b50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7140cecf0 0x7fe71406d5b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 --2- 172.21.15.66:0/55956034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7140acaf0 0x7fe714097080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 -- 172.21.15.66:0/55956034 >> 172.21.15.66:0/55956034 conn(0x7fe71406a190 msgr2=0x7fe7140af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:49.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 -- 172.21.15.66:0/55956034 shutdown_connections 2024-03-30T00:39:49.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.738+0000 7fe718e00640 1 -- 172.21.15.66:0/55956034 wait complete. 2024-03-30T00:39:49.771 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-30T00:39:49.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c7800640 1 Processor -- start 2024-03-30T00:39:49.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c7800640 1 -- start start 2024-03-30T00:39:49.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c7800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c015ca00 0x7fc3c00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c7800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc3c0097640 0x7fc3c00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c7800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c00b16e0 con 0x7fc3c00b11d0 2024-03-30T00:39:49.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c7800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c00b1980 con 0x7fc3c0097640 2024-03-30T00:39:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c7800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c00b1c20 con 0x7fc3c015ca00 2024-03-30T00:39:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3b7e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc3c00b11d0 0x7fc3c00ab550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48066/0 (socket says 172.21.15.66:48066) 2024-03-30T00:39:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3b7e00640 1 -- 172.21.15.66:0/4080203853 learned_addr learned my addr 172.21.15.66:0/4080203853 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:39:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c5800640 1 --2- 172.21.15.66:0/4080203853 >> v2:172.21.15.66:3300/0 conn(0x7fc3c0097640 0x7fc3c00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c4e00640 1 --2- 172.21.15.66:0/4080203853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c015ca00 0x7fc3c00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c5800640 1 -- 172.21.15.66:0/4080203853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c015ca00 msgr2=0x7fc3c00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c5800640 1 --2- 172.21.15.66:0/4080203853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c015ca00 0x7fc3c00ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c5800640 1 -- 172.21.15.66:0/4080203853 >> v1:172.21.15.66:6789/0 conn(0x7fc3c00b11d0 legacy=0x7fc3c00ab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c5800640 1 -- 172.21.15.66:0/4080203853 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c00b1ee0 con 0x7fc3c0097640 2024-03-30T00:39:49.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.910+0000 7fc3c4e00640 1 --2- 172.21.15.66:0/4080203853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c015ca00 0x7fc3c00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:49.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3c5800640 1 --2- 172.21.15.66:0/4080203853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c0097640 0x7fc3c00cecc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009d90 tx=0x7fc3b801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5feb728835ac5df2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:49.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3bfe00640 1 -- 172.21.15.66:0/4080203853 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b80a58b0 con 0x7fc3c0097640 2024-03-30T00:39:49.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3bfe00640 1 -- 172.21.15.66:0/4080203853 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b80a5a10 con 0x7fc3c0097640 2024-03-30T00:39:49.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3bfe00640 1 -- 172.21.15.66:0/4080203853 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b80a5da0 con 0x7fc3c0097640 2024-03-30T00:39:49.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3c7800640 1 -- 172.21.15.66:0/4080203853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c0097640 msgr2=0x7fc3c00cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3c7800640 1 --2- 172.21.15.66:0/4080203853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c0097640 0x7fc3c00cecc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009d90 tx=0x7fc3b801c8e0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3c7800640 1 -- 172.21.15.66:0/4080203853 shutdown_connections 2024-03-30T00:39:49.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3c7800640 1 --2- 172.21.15.66:0/4080203853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c0097640 0x7fc3c00cecc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3c7800640 1 --2- 172.21.15.66:0/4080203853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c015ca00 0x7fc3c00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3c7800640 1 -- 172.21.15.66:0/4080203853 >> 172.21.15.66:0/4080203853 conn(0x7fc3c006a190 msgr2=0x7fc3c006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:49.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3c7800640 1 -- 172.21.15.66:0/4080203853 shutdown_connections 2024-03-30T00:39:49.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.914+0000 7fc3c7800640 1 -- 172.21.15.66:0/4080203853 wait complete. 2024-03-30T00:39:49.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c7800640 1 Processor -- start 2024-03-30T00:39:49.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c7800640 1 -- start start 2024-03-30T00:39:49.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c7800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c0097640 0x7fc3c015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c7800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c00b11d0 0x7fc3c006d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c7800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3c015ca00 0x7fc3c0154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c7800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c01563e0 con 0x7fc3c00b11d0 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c7800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c00c6990 con 0x7fc3c015ca00 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c7800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c015a7a0 con 0x7fc3c0097640 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3b7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c00b11d0 0x7fc3c006d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3b7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c00b11d0 0x7fc3c006d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51322/0 (socket says 172.21.15.66:51322) 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3b7e00640 1 -- 172.21.15.66:0/1933201564 learned_addr learned my addr 172.21.15.66:0/1933201564 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c4e00640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c0097640 0x7fc3c015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c5800640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3c015ca00 0x7fc3c0154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3b7e00640 1 -- 172.21.15.66:0/1933201564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c0097640 msgr2=0x7fc3c015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3b7e00640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c0097640 0x7fc3c015a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3b7e00640 1 -- 172.21.15.66:0/1933201564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3c015ca00 msgr2=0x7fc3c0154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:49.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3b7e00640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3c015ca00 0x7fc3c0154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:49.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3b7e00640 1 -- 172.21.15.66:0/1933201564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b8009a70 con 0x7fc3c00b11d0 2024-03-30T00:39:49.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c5800640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3c015ca00 0x7fc3c0154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:39:49.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3b7e00640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c00b11d0 0x7fc3c006d610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800c500 tx=0x7fc3a800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:49.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3bd600640 1 -- 172.21.15.66:0/1933201564 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8005cf0 con 0x7fc3c00b11d0 2024-03-30T00:39:49.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3bd600640 1 -- 172.21.15.66:0/1933201564 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8005e50 con 0x7fc3c00b11d0 2024-03-30T00:39:49.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3bd600640 1 -- 172.21.15.66:0/1933201564 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80072f0 con 0x7fc3c00b11d0 2024-03-30T00:39:49.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c7800640 1 -- 172.21.15.66:0/1933201564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c015aa80 con 0x7fc3c00b11d0 2024-03-30T00:39:49.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c7800640 1 -- 172.21.15.66:0/1933201564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c015b040 con 0x7fc3c00b11d0 2024-03-30T00:39:49.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.918+0000 7fc3c7800640 1 -- 172.21.15.66:0/1933201564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c00c6b90 con 0x7fc3c00b11d0 2024-03-30T00:39:49.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.922+0000 7fc3bd600640 1 -- 172.21.15.66:0/1933201564 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc3a8007450 con 0x7fc3c00b11d0 2024-03-30T00:39:49.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.922+0000 7fc3bd600640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc39c066690 0x7fc39c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:49.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.922+0000 7fc3bd600640 1 -- 172.21.15.66:0/1933201564 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc3a80836b0 con 0x7fc3c00b11d0 2024-03-30T00:39:49.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.922+0000 7fc3c4e00640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc39c066690 0x7fc39c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:49.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.922+0000 7fc3c4e00640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc39c066690 0x7fc39c068b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000c900 tx=0x7fc3b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:49.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:49.926+0000 7fc3bd600640 1 -- 172.21.15.66:0/1933201564 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc3a8050e20 con 0x7fc3c00b11d0 2024-03-30T00:39:50.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.086+0000 7fc3c7800640 1 -- 172.21.15.66:0/1933201564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fc3c00c6b90 con 0x7fc3c00b11d0 2024-03-30T00:39:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.742+0000 7fc3bd600640 1 -- 172.21.15.66:0/1933201564 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7fc3c00c6b90 con 0x7fc3c00b11d0 2024-03-30T00:39:50.752 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-03-30T00:39:50.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.750+0000 7fc3c7800640 1 -- 172.21.15.66:0/1933201564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc39c066690 msgr2=0x7fc39c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:50.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.750+0000 7fc3c7800640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc39c066690 0x7fc39c068b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000c900 tx=0x7fc3b0008040 comp rx=0 tx=0).stop 2024-03-30T00:39:50.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.750+0000 7fc3c7800640 1 -- 172.21.15.66:0/1933201564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c00b11d0 msgr2=0x7fc3c006d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:50.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.750+0000 7fc3c7800640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c00b11d0 0x7fc3c006d610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800c500 tx=0x7fc3a800c9c0 comp rx=0 tx=0).stop 2024-03-30T00:39:50.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.750+0000 7fc3c7800640 1 -- 172.21.15.66:0/1933201564 shutdown_connections 2024-03-30T00:39:50.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.750+0000 7fc3c7800640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3c015ca00 0x7fc3c0154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:50.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.750+0000 7fc3c7800640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc39c066690 0x7fc39c068b50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:50.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.750+0000 7fc3c7800640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c00b11d0 0x7fc3c006d610 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:50.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.750+0000 7fc3c7800640 1 --2- 172.21.15.66:0/1933201564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c0097640 0x7fc3c015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:50.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.750+0000 7fc3c7800640 1 -- 172.21.15.66:0/1933201564 >> 172.21.15.66:0/1933201564 conn(0x7fc3c006a190 msgr2=0x7fc3c006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:50.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.754+0000 7fc3c7800640 1 -- 172.21.15.66:0/1933201564 shutdown_connections 2024-03-30T00:39:50.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.754+0000 7fc3c7800640 1 -- 172.21.15.66:0/1933201564 wait complete. 2024-03-30T00:39:50.791 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T00:39:50.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8dfec00640 1 Processor -- start 2024-03-30T00:39:50.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8dfec00640 1 -- start start 2024-03-30T00:39:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8dfec00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8df80cabb0 0x7f8df80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8dfec00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df806d070 con 0x7f8df815a640 2024-03-30T00:39:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8dfec00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df80cbba0 con 0x7f8df80ce520 2024-03-30T00:39:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8dfec00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df80cb8c0 con 0x7f8df80cabb0 2024-03-30T00:39:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8df7e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f8df815a640 0x7f8df806dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48068/0 (socket says 172.21.15.66:48068) 2024-03-30T00:39:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8df7e00640 1 -- 172.21.15.66:0/3899347922 learned_addr learned my addr 172.21.15.66:0/3899347922 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:39:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8dfcc00640 1 --2- 172.21.15.66:0/3899347922 >> v2:172.21.15.66:3300/0 conn(0x7f8df80cabb0 0x7f8df80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:50.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8dfcc00640 1 -- 172.21.15.66:0/3899347922 >> v1:172.21.15.156:6789/0 conn(0x7f8df80ce520 legacy=0x7f8df80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:50.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8dfcc00640 1 -- 172.21.15.66:0/3899347922 >> v1:172.21.15.66:6789/0 conn(0x7f8df815a640 legacy=0x7f8df806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:50.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.926+0000 7f8dfcc00640 1 -- 172.21.15.66:0/3899347922 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df806d2a0 con 0x7f8df80cabb0 2024-03-30T00:39:50.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.930+0000 7f8dfcc00640 1 --2- 172.21.15.66:0/3899347922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df80cabb0 0x7f8df80cafe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8df00097c0 tx=0x7f8df001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3a7d3cd500c918 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.930+0000 7f8df7400640 1 -- 172.21.15.66:0/3899347922 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00ab670 con 0x7f8df80cabb0 2024-03-30T00:39:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.930+0000 7f8df7400640 1 -- 172.21.15.66:0/3899347922 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df00ab7d0 con 0x7f8df80cabb0 2024-03-30T00:39:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.930+0000 7f8df7400640 1 -- 172.21.15.66:0/3899347922 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00abb60 con 0x7f8df80cabb0 2024-03-30T00:39:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.930+0000 7f8dfec00640 1 -- 172.21.15.66:0/3899347922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df80cabb0 msgr2=0x7f8df80cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.930+0000 7f8dfec00640 1 --2- 172.21.15.66:0/3899347922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df80cabb0 0x7f8df80cafe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8df00097c0 tx=0x7f8df001c470 comp rx=0 tx=0).stop 2024-03-30T00:39:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.930+0000 7f8dfec00640 1 -- 172.21.15.66:0/3899347922 shutdown_connections 2024-03-30T00:39:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.930+0000 7f8dfec00640 1 --2- 172.21.15.66:0/3899347922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df80cabb0 0x7f8df80cafe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.930+0000 7f8dfec00640 1 -- 172.21.15.66:0/3899347922 >> 172.21.15.66:0/3899347922 conn(0x7f8df8069f60 msgr2=0x7f8df806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.930+0000 7f8dfec00640 1 -- 172.21.15.66:0/3899347922 shutdown_connections 2024-03-30T00:39:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.930+0000 7f8dfec00640 1 -- 172.21.15.66:0/3899347922 wait complete. 2024-03-30T00:39:50.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfec00640 1 Processor -- start 2024-03-30T00:39:50.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfec00640 1 -- start start 2024-03-30T00:39:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfec00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8df80cabb0 0x7f8df80ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfec00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df80ce520 0x7f8df8097810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df815a640 0x7f8df8158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80c53c0 con 0x7f8df815a640 2024-03-30T00:39:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfec00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80c5610 con 0x7f8df80ce520 2024-03-30T00:39:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfec00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80cd8a0 con 0x7f8df80cabb0 2024-03-30T00:39:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfcc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df815a640 0x7f8df8158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfcc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df815a640 0x7f8df8158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51332/0 (socket says 172.21.15.66:51332) 2024-03-30T00:39:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfcc00640 1 -- 172.21.15.66:0/823457008 learned_addr learned my addr 172.21.15.66:0/823457008 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8def400640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df80ce520 0x7f8df8097810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8df7e00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8df80cabb0 0x7f8df80ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfcc00640 1 -- 172.21.15.66:0/823457008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8df80cabb0 msgr2=0x7f8df80ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfcc00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8df80cabb0 0x7f8df80ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfcc00640 1 -- 172.21.15.66:0/823457008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df80ce520 msgr2=0x7f8df8097810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfcc00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df80ce520 0x7f8df8097810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfcc00640 1 -- 172.21.15.66:0/823457008 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df0009420 con 0x7f8df815a640 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8df7e00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8df80cabb0 0x7f8df80ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfcc00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df815a640 0x7f8df8158860 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8df00ad040 tx=0x7f8df001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8df4c00640 1 -- 172.21.15.66:0/823457008 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00b5810 con 0x7f8df815a640 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8df4c00640 1 -- 172.21.15.66:0/823457008 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df00b5970 con 0x7f8df815a640 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfec00640 1 -- 172.21.15.66:0/823457008 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df80cdb80 con 0x7f8df815a640 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8df4c00640 1 -- 172.21.15.66:0/823457008 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00b5d20 con 0x7f8df815a640 2024-03-30T00:39:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.934+0000 7f8dfec00640 1 -- 172.21.15.66:0/823457008 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df80c58f0 con 0x7f8df815a640 2024-03-30T00:39:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.938+0000 7f8dfec00640 1 -- 172.21.15.66:0/823457008 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df80cddc0 con 0x7f8df815a640 2024-03-30T00:39:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.938+0000 7f8df4c00640 1 -- 172.21.15.66:0/823457008 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8df00bf410 con 0x7f8df815a640 2024-03-30T00:39:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.938+0000 7f8df4c00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dd4066690 0x7f8dd4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.938+0000 7f8df4c00640 1 -- 172.21.15.66:0/823457008 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f8df011ed20 con 0x7f8df815a640 2024-03-30T00:39:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.938+0000 7f8df7e00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dd4066690 0x7f8dd4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:50.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.938+0000 7f8df7e00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dd4066690 0x7f8dd4068b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8de8005c10 tx=0x7f8de8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:50.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:50.942+0000 7f8df4c00640 1 -- 172.21.15.66:0/823457008 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8df80cddc0 con 0x7f8df815a640 2024-03-30T00:39:51.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.138+0000 7f8dfec00640 1 -- 172.21.15.66:0/823457008 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8df80cdd70 con 0x7f8df815a640 2024-03-30T00:39:51.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.146+0000 7f8df4c00640 1 -- 172.21.15.66:0/823457008 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8df00ec080 con 0x7f8df815a640 2024-03-30T00:39:51.158 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 2 and data pool 3 2024-03-30T00:39:51.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.154+0000 7f8dfec00640 1 -- 172.21.15.66:0/823457008 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dd4066690 msgr2=0x7f8dd4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:51.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.154+0000 7f8dfec00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dd4066690 0x7f8dd4068b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8de8005c10 tx=0x7f8de8009040 comp rx=0 tx=0).stop 2024-03-30T00:39:51.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.154+0000 7f8dfec00640 1 -- 172.21.15.66:0/823457008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df815a640 msgr2=0x7f8df8158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:51.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.154+0000 7f8dfec00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df815a640 0x7f8df8158860 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8df00ad040 tx=0x7f8df001acb0 comp rx=0 tx=0).stop 2024-03-30T00:39:51.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.158+0000 7f8dfec00640 1 -- 172.21.15.66:0/823457008 shutdown_connections 2024-03-30T00:39:51.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.158+0000 7f8dfec00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dd4066690 0x7f8dd4068b50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:51.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.158+0000 7f8dfec00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df815a640 0x7f8df8158860 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:51.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.158+0000 7f8dfec00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df80ce520 0x7f8df8097810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:51.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.158+0000 7f8dfec00640 1 --2- 172.21.15.66:0/823457008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8df80cabb0 0x7f8df80ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:51.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.158+0000 7f8dfec00640 1 -- 172.21.15.66:0/823457008 >> 172.21.15.66:0/823457008 conn(0x7f8df8069f60 msgr2=0x7f8df80aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:51.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.162+0000 7f8dfec00640 1 -- 172.21.15.66:0/823457008 shutdown_connections 2024-03-30T00:39:51.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.166+0000 7f8dfec00640 1 -- 172.21.15.66:0/823457008 wait complete. 2024-03-30T00:39:51.207 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-30T00:39:51.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-30T00:39:51.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.342+0000 7f3ad9600640 1 Processor -- start 2024-03-30T00:39:51.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.342+0000 7f3ad9600640 1 -- start start 2024-03-30T00:39:51.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad9600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ad4158e80 0x7f3ad40ad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:51.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad9600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3ad40ca940 0x7f3ad40cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:51.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad9600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad406df80 con 0x7f3ad40b24e0 2024-03-30T00:39:51.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad9600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad406c4e0 con 0x7f3ad4158e80 2024-03-30T00:39:51.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad9600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad40cb2d0 con 0x7f3ad40ca940 2024-03-30T00:39:51.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ad4158e80 0x7f3ad40ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:51.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ad4158e80 0x7f3ad40ad590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51334/0 (socket says 172.21.15.66:51334) 2024-03-30T00:39:51.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad2a00640 1 -- 172.21.15.66:0/2853284930 learned_addr learned my addr 172.21.15.66:0/2853284930 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:51.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad2a00640 1 -- 172.21.15.66:0/2853284930 >> v2:172.21.15.156:3300/0 conn(0x7f3ad40ca940 msgr2=0x7f3ad40cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:51.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad2a00640 1 --2- 172.21.15.66:0/2853284930 >> v2:172.21.15.156:3300/0 conn(0x7f3ad40ca940 0x7f3ad40cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:51.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad2a00640 1 -- 172.21.15.66:0/2853284930 >> v1:172.21.15.156:6789/0 conn(0x7f3ad40b24e0 legacy=0x7f3ad406bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:51.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.346+0000 7f3ad2a00640 1 -- 172.21.15.66:0/2853284930 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad40c93d0 con 0x7f3ad4158e80 2024-03-30T00:39:51.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad2a00640 1 --2- 172.21.15.66:0/2853284930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad4158e80 0x7f3ad40ad590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40adad0 tx=0x7f3ac4020ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e662b12d1c77b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:51.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad2000640 1 -- 172.21.15.66:0/2853284930 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40aabe0 con 0x7f3ad4158e80 2024-03-30T00:39:51.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad2000640 1 -- 172.21.15.66:0/2853284930 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac40aad40 con 0x7f3ad4158e80 2024-03-30T00:39:51.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad2000640 1 -- 172.21.15.66:0/2853284930 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40b2660 con 0x7f3ad4158e80 2024-03-30T00:39:51.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 -- 172.21.15.66:0/2853284930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad4158e80 msgr2=0x7f3ad40ad590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:51.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 --2- 172.21.15.66:0/2853284930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad4158e80 0x7f3ad40ad590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40adad0 tx=0x7f3ac4020ee0 comp rx=0 tx=0).stop 2024-03-30T00:39:51.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 -- 172.21.15.66:0/2853284930 shutdown_connections 2024-03-30T00:39:51.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 --2- 172.21.15.66:0/2853284930 >> v2:172.21.15.156:3300/0 conn(0x7f3ad40ca940 0x7f3ad40cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:51.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 --2- 172.21.15.66:0/2853284930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad4158e80 0x7f3ad40ad590 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:51.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 -- 172.21.15.66:0/2853284930 >> 172.21.15.66:0/2853284930 conn(0x7f3ad406a190 msgr2=0x7f3ad406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:51.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 -- 172.21.15.66:0/2853284930 shutdown_connections 2024-03-30T00:39:51.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 -- 172.21.15.66:0/2853284930 wait complete. 2024-03-30T00:39:51.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 Processor -- start 2024-03-30T00:39:51.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 -- start start 2024-03-30T00:39:51.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ad40b24e0 0x7f3ad4155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:51.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad40ca940 0x7f3ad40c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:51.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ad4158e80 0x7f3ad40c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:51.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40c70d0 con 0x7f3ad40ca940 2024-03-30T00:39:51.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad4156200 con 0x7f3ad4158e80 2024-03-30T00:39:51.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.350+0000 7f3ad9600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40c6450 con 0x7f3ad40b24e0 2024-03-30T00:39:51.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad40ca940 0x7f3ad40c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:51.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad40ca940 0x7f3ad40c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51340/0 (socket says 172.21.15.66:51340) 2024-03-30T00:39:51.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acbe00640 1 -- 172.21.15.66:0/1101236049 learned_addr learned my addr 172.21.15.66:0/1101236049 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:51.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3ad3400640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ad4158e80 0x7f3ad40c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:51.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3ad2a00640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ad40b24e0 0x7f3ad4155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:51.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acbe00640 1 -- 172.21.15.66:0/1101236049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ad40b24e0 msgr2=0x7f3ad4155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:51.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acbe00640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ad40b24e0 0x7f3ad4155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:51.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acbe00640 1 -- 172.21.15.66:0/1101236049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ad4158e80 msgr2=0x7f3ad40c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:51.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acbe00640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ad4158e80 0x7f3ad40c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:51.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acbe00640 1 -- 172.21.15.66:0/1101236049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac40094d0 con 0x7f3ad40ca940 2024-03-30T00:39:51.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acbe00640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad40ca940 0x7f3ad40c9860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c490 tx=0x7f3abc001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:51.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acaa00640 1 -- 172.21.15.66:0/1101236049 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc003600 con 0x7f3ad40ca940 2024-03-30T00:39:51.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acaa00640 1 -- 172.21.15.66:0/1101236049 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc003780 con 0x7f3ad40ca940 2024-03-30T00:39:51.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acaa00640 1 -- 172.21.15.66:0/1101236049 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc003b30 con 0x7f3ad40ca940 2024-03-30T00:39:51.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3ad9600640 1 -- 172.21.15.66:0/1101236049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad40c6730 con 0x7f3ad40ca940 2024-03-30T00:39:51.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3ad9600640 1 -- 172.21.15.66:0/1101236049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad40c6cf0 con 0x7f3ad40ca940 2024-03-30T00:39:51.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3ad9600640 1 -- 172.21.15.66:0/1101236049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad40c7210 con 0x7f3ad40ca940 2024-03-30T00:39:51.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acaa00640 1 -- 172.21.15.66:0/1101236049 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3abc003c90 con 0x7f3ad40ca940 2024-03-30T00:39:51.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acaa00640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ab0066690 0x7f3ab0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:51.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.354+0000 7f3acaa00640 1 -- 172.21.15.66:0/1101236049 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3abc0833a0 con 0x7f3ad40ca940 2024-03-30T00:39:51.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.358+0000 7f3ad2a00640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ab0066690 0x7f3ab0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:51.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.358+0000 7f3ad2a00640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ab0066690 0x7f3ab0068b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40b4000 tx=0x7f3ac4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:51.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.362+0000 7f3acaa00640 1 -- 172.21.15.66:0/1101236049 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3ad40c7210 con 0x7f3ad40ca940 2024-03-30T00:39:51.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:51.526+0000 7f3ad9600640 1 -- 172.21.15.66:0/1101236049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f3ad40c7210 con 0x7f3ad40ca940 2024-03-30T00:39:52.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.146+0000 7f3acaa00640 1 -- 172.21.15.66:0/1101236049 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f3ad40c7210 con 0x7f3ad40ca940 2024-03-30T00:39:52.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 -- 172.21.15.66:0/1101236049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ab0066690 msgr2=0x7f3ab0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:52.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ab0066690 0x7f3ab0068b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40b4000 tx=0x7f3ac4008040 comp rx=0 tx=0).stop 2024-03-30T00:39:52.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 -- 172.21.15.66:0/1101236049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad40ca940 msgr2=0x7f3ad40c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:52.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad40ca940 0x7f3ad40c9860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c490 tx=0x7f3abc001e80 comp rx=0 tx=0).stop 2024-03-30T00:39:52.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 -- 172.21.15.66:0/1101236049 shutdown_connections 2024-03-30T00:39:52.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ad4158e80 0x7f3ad40c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:52.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ab0066690 0x7f3ab0068b50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:52.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad40ca940 0x7f3ad40c9860 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:52.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 --2- 172.21.15.66:0/1101236049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ad40b24e0 0x7f3ad4155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:52.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 -- 172.21.15.66:0/1101236049 >> 172.21.15.66:0/1101236049 conn(0x7f3ad406a190 msgr2=0x7f3ad40ab220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:52.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 -- 172.21.15.66:0/1101236049 shutdown_connections 2024-03-30T00:39:52.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.150+0000 7f3ad9600640 1 -- 172.21.15.66:0/1101236049 wait complete. 2024-03-30T00:39:52.180 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T00:39:52.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62c000640 1 Processor -- start 2024-03-30T00:39:52.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62c000640 1 -- start start 2024-03-30T00:39:52.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62c000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe624154de0 0x7fe62415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:52.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62c000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6240b3070 0x7fe6240ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:52.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62c000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6240b1810 con 0x7fe6240b2c90 2024-03-30T00:39:52.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62c000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62415b0f0 con 0x7fe6240b3070 2024-03-30T00:39:52.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62c000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62415b3d0 con 0x7fe624154de0 2024-03-30T00:39:52.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62a000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6240b3070 0x7fe6240ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:52.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62a000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6240b3070 0x7fe6240ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51356/0 (socket says 172.21.15.66:51356) 2024-03-30T00:39:52.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62a000640 1 -- 172.21.15.66:0/556941506 learned_addr learned my addr 172.21.15.66:0/556941506 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:52.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe628c00640 1 --2- 172.21.15.66:0/556941506 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe624154de0 0x7fe62415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:52.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62a000640 1 -- 172.21.15.66:0/556941506 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe624154de0 msgr2=0x7fe62415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:52.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62a000640 1 --2- 172.21.15.66:0/556941506 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe624154de0 0x7fe62415aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:52.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62a000640 1 -- 172.21.15.66:0/556941506 >> v1:172.21.15.156:6789/0 conn(0x7fe6240b2c90 legacy=0x7fe62406bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:52.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe62a000640 1 -- 172.21.15.66:0/556941506 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62415b6b0 con 0x7fe6240b3070 2024-03-30T00:39:52.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.322+0000 7fe628c00640 1 --2- 172.21.15.66:0/556941506 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe624154de0 0x7fe62415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:39:52.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe62a000640 1 --2- 172.21.15.66:0/556941506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6240b3070 0x7fe6240ab440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009e50 tx=0x7fe61c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3430bdf4077e43d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:52.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe623e00640 1 -- 172.21.15.66:0/556941506 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0a57d0 con 0x7fe6240b3070 2024-03-30T00:39:52.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe623e00640 1 -- 172.21.15.66:0/556941506 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c0a5930 con 0x7fe6240b3070 2024-03-30T00:39:52.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe623e00640 1 -- 172.21.15.66:0/556941506 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0a5cc0 con 0x7fe6240b3070 2024-03-30T00:39:52.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe62c000640 1 -- 172.21.15.66:0/556941506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6240b3070 msgr2=0x7fe6240ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:52.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe62c000640 1 --2- 172.21.15.66:0/556941506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6240b3070 0x7fe6240ab440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009e50 tx=0x7fe61c01c9c0 comp rx=0 tx=0).stop 2024-03-30T00:39:52.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe62c000640 1 -- 172.21.15.66:0/556941506 shutdown_connections 2024-03-30T00:39:52.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe62c000640 1 --2- 172.21.15.66:0/556941506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6240b3070 0x7fe6240ab440 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:52.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe62c000640 1 --2- 172.21.15.66:0/556941506 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe624154de0 0x7fe62415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:52.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe62c000640 1 -- 172.21.15.66:0/556941506 >> 172.21.15.66:0/556941506 conn(0x7fe6240aff40 msgr2=0x7fe6240b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:52.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe62c000640 1 -- 172.21.15.66:0/556941506 shutdown_connections 2024-03-30T00:39:52.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.326+0000 7fe62c000640 1 -- 172.21.15.66:0/556941506 wait complete. 2024-03-30T00:39:52.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62c000640 1 Processor -- start 2024-03-30T00:39:52.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62c000640 1 -- start start 2024-03-30T00:39:52.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62c000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6240b2c90 0x7fe6241f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:52.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62c000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6240b3070 0x7fe62415c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:52.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62c000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe624071860 0x7fe624071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:52.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62c000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624151680 con 0x7fe624071860 2024-03-30T00:39:52.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62c000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624151960 con 0x7fe6240b3070 2024-03-30T00:39:52.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62c000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe624151c40 con 0x7fe6240b2c90 2024-03-30T00:39:52.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe624071860 0x7fe624071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:52.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe624071860 0x7fe624071ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51358/0 (socket says 172.21.15.66:51358) 2024-03-30T00:39:52.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62a000640 1 -- 172.21.15.66:0/1087577033 learned_addr learned my addr 172.21.15.66:0/1087577033 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:52.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe629600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6240b2c90 0x7fe6241f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:52.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe628c00640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6240b3070 0x7fe62415c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:52.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62a000640 1 -- 172.21.15.66:0/1087577033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6240b2c90 msgr2=0x7fe6241f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:52.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62a000640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6240b2c90 0x7fe6241f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:52.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62a000640 1 -- 172.21.15.66:0/1087577033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6240b3070 msgr2=0x7fe62415c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:52.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62a000640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6240b3070 0x7fe62415c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:52.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62a000640 1 -- 172.21.15.66:0/1087577033 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c009b30 con 0x7fe624071860 2024-03-30T00:39:52.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62a000640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe624071860 0x7fe624071ce0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0af040 tx=0x7fe61c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:52.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe621600640 1 -- 172.21.15.66:0/1087577033 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0b8670 con 0x7fe624071860 2024-03-30T00:39:52.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe621600640 1 -- 172.21.15.66:0/1087577033 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c0b87f0 con 0x7fe624071860 2024-03-30T00:39:52.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe621600640 1 -- 172.21.15.66:0/1087577033 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0b8ba0 con 0x7fe624071860 2024-03-30T00:39:52.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.330+0000 7fe62c000640 1 -- 172.21.15.66:0/1087577033 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62415bbd0 con 0x7fe624071860 2024-03-30T00:39:52.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.334+0000 7fe62c000640 1 -- 172.21.15.66:0/1087577033 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62415c910 con 0x7fe624071860 2024-03-30T00:39:52.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.334+0000 7fe616a00640 1 -- 172.21.15.66:0/1087577033 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62406d4d0 con 0x7fe624071860 2024-03-30T00:39:52.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.334+0000 7fe621600640 1 -- 172.21.15.66:0/1087577033 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe61c0b8d00 con 0x7fe624071860 2024-03-30T00:39:52.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.334+0000 7fe621600640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6040666c0 0x7fe604068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:52.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.334+0000 7fe621600640 1 -- 172.21.15.66:0/1087577033 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5935+0+0 (secure 0 0 0) 0x7fe61c11f480 con 0x7fe624071860 2024-03-30T00:39:52.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.334+0000 7fe629600640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6040666c0 0x7fe604068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:52.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.334+0000 7fe629600640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6040666c0 0x7fe604068b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6180098e0 tx=0x7fe618009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:52.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.338+0000 7fe621600640 1 -- 172.21.15.66:0/1087577033 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe61c0ec840 con 0x7fe624071860 2024-03-30T00:39:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:52.498+0000 7fe616a00640 1 -- 172.21.15.66:0/1087577033 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe62406d4d0 con 0x7fe624071860 2024-03-30T00:39:54.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.154+0000 7fe621600640 1 -- 172.21.15.66:0/1087577033 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7fe61c0efe60 con 0x7fe624071860 2024-03-30T00:39:54.162 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data_ec' created 2024-03-30T00:39:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.158+0000 7fe616a00640 1 -- 172.21.15.66:0/1087577033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6040666c0 msgr2=0x7fe604068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.158+0000 7fe616a00640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6040666c0 0x7fe604068b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6180098e0 tx=0x7fe618009060 comp rx=0 tx=0).stop 2024-03-30T00:39:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.158+0000 7fe616a00640 1 -- 172.21.15.66:0/1087577033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe624071860 msgr2=0x7fe624071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.158+0000 7fe616a00640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe624071860 0x7fe624071ce0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0af040 tx=0x7fe61c019760 comp rx=0 tx=0).stop 2024-03-30T00:39:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.158+0000 7fe616a00640 1 -- 172.21.15.66:0/1087577033 shutdown_connections 2024-03-30T00:39:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.158+0000 7fe616a00640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6040666c0 0x7fe604068b80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.158+0000 7fe616a00640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe624071860 0x7fe624071ce0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.158+0000 7fe616a00640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6240b3070 0x7fe62415c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.162+0000 7fe616a00640 1 --2- 172.21.15.66:0/1087577033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6240b2c90 0x7fe6241f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.162+0000 7fe616a00640 1 -- 172.21.15.66:0/1087577033 >> 172.21.15.66:0/1087577033 conn(0x7fe6240aff40 msgr2=0x7fe62406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:54.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.162+0000 7fe616a00640 1 -- 172.21.15.66:0/1087577033 shutdown_connections 2024-03-30T00:39:54.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.166+0000 7fe616a00640 1 -- 172.21.15.66:0/1087577033 wait complete. 2024-03-30T00:39:54.200 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-30T00:39:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.338+0000 7f3991400640 1 Processor -- start 2024-03-30T00:39:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.338+0000 7f3991400640 1 -- start start 2024-03-30T00:39:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f3991400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f398c1594f0 0x7f398c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f3991400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f398c0cec60 0x7f398c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f3991400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c0cf0d0 con 0x7f398c0ce7b0 2024-03-30T00:39:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f3991400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c0b1a60 con 0x7f398c1594f0 2024-03-30T00:39:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f3991400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0cda90 con 0x7f398c0cec60 2024-03-30T00:39:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f398a800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f398c1594f0 0x7f398c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f398b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f398c0cec60 0x7f398c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f398a800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f398c1594f0 0x7f398c0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:37080/0 (socket says 172.21.15.66:37080) 2024-03-30T00:39:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f3981e00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f398c0ce7b0 0x7f398c1598d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:51936/0 (socket says 172.21.15.66:51936) 2024-03-30T00:39:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f398a800640 1 -- 172.21.15.66:0/2713742637 learned_addr learned my addr 172.21.15.66:0/2713742637 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f398a800640 1 -- 172.21.15.66:0/2713742637 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f398c0cec60 msgr2=0x7f398c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f398a800640 1 --2- 172.21.15.66:0/2713742637 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f398c0cec60 0x7f398c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f398a800640 1 -- 172.21.15.66:0/2713742637 >> v1:172.21.15.156:6789/0 conn(0x7f398c0ce7b0 legacy=0x7f398c1598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.342+0000 7f398a800640 1 -- 172.21.15.66:0/2713742637 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0b13c0 con 0x7f398c1594f0 2024-03-30T00:39:54.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f398a800640 1 --2- 172.21.15.66:0/2713742637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398c1594f0 0x7f398c0b1e80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f397c009870 tx=0x7f397c01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a788c61beaec5122 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:54.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3989e00640 1 -- 172.21.15.66:0/2713742637 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c0a9640 con 0x7f398c1594f0 2024-03-30T00:39:54.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3989e00640 1 -- 172.21.15.66:0/2713742637 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c0a97a0 con 0x7f398c1594f0 2024-03-30T00:39:54.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3989e00640 1 -- 172.21.15.66:0/2713742637 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c0a9b30 con 0x7f398c1594f0 2024-03-30T00:39:54.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 -- 172.21.15.66:0/2713742637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398c1594f0 msgr2=0x7f398c0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:54.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 --2- 172.21.15.66:0/2713742637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398c1594f0 0x7f398c0b1e80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f397c009870 tx=0x7f397c01c560 comp rx=0 tx=0).stop 2024-03-30T00:39:54.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 -- 172.21.15.66:0/2713742637 shutdown_connections 2024-03-30T00:39:54.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 --2- 172.21.15.66:0/2713742637 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f398c0cec60 0x7f398c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:54.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 --2- 172.21.15.66:0/2713742637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398c1594f0 0x7f398c0b1e80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:54.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 -- 172.21.15.66:0/2713742637 >> 172.21.15.66:0/2713742637 conn(0x7f398c06a190 msgr2=0x7f398c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:54.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 -- 172.21.15.66:0/2713742637 shutdown_connections 2024-03-30T00:39:54.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 -- 172.21.15.66:0/2713742637 wait complete. 2024-03-30T00:39:54.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 Processor -- start 2024-03-30T00:39:54.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 -- start start 2024-03-30T00:39:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f398c0ce7b0 0x7f398c097960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398c0cec60 0x7f398c1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398c1594f0 0x7f398c157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0af870 con 0x7f398c1594f0 2024-03-30T00:39:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c097ea0 con 0x7f398c0cec60 2024-03-30T00:39:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f3991400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0aefd0 con 0x7f398c0ce7b0 2024-03-30T00:39:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.346+0000 7f398b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398c1594f0 0x7f398c157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f398b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398c1594f0 0x7f398c157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51360/0 (socket says 172.21.15.66:51360) 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f398b200640 1 -- 172.21.15.66:0/2782165662 learned_addr learned my addr 172.21.15.66:0/2782165662 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f398a800640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f398c0ce7b0 0x7f398c097960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f3981e00640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398c0cec60 0x7f398c1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f398b200640 1 -- 172.21.15.66:0/2782165662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f398c0ce7b0 msgr2=0x7f398c097960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f398b200640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f398c0ce7b0 0x7f398c097960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f398b200640 1 -- 172.21.15.66:0/2782165662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398c0cec60 msgr2=0x7f398c1577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f398b200640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398c0cec60 0x7f398c1577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f398b200640 1 -- 172.21.15.66:0/2782165662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0094d0 con 0x7f398c1594f0 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f398a800640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f398c0ce7b0 0x7f398c097960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f398b200640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398c1594f0 0x7f398c157ce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f398400efc0 tx=0x7f398400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f3983400640 1 -- 172.21.15.66:0/2782165662 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984014480 con 0x7f398c1594f0 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f3983400640 1 -- 172.21.15.66:0/2782165662 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398400f040 con 0x7f398c1594f0 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f3983400640 1 -- 172.21.15.66:0/2782165662 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984014820 con 0x7f398c1594f0 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f3991400640 1 -- 172.21.15.66:0/2782165662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c0af2b0 con 0x7f398c1594f0 2024-03-30T00:39:54.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f3991400640 1 -- 172.21.15.66:0/2782165662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c0c5a40 con 0x7f398c1594f0 2024-03-30T00:39:54.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f3991400640 1 -- 172.21.15.66:0/2782165662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c066c30 con 0x7f398c1594f0 2024-03-30T00:39:54.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f3983400640 1 -- 172.21.15.66:0/2782165662 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3984014980 con 0x7f398c1594f0 2024-03-30T00:39:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.350+0000 7f3983400640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3968066690 0x7f3968068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.354+0000 7f3983400640 1 -- 172.21.15.66:0/2782165662 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f3984084f70 con 0x7f398c1594f0 2024-03-30T00:39:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.354+0000 7f398a800640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3968066690 0x7f3968068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.354+0000 7f398a800640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3968066690 0x7f3968068b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f397c0ae000 tx=0x7f397c0b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:54.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.358+0000 7f3983400640 1 -- 172.21.15.66:0/2782165662 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3984051fa0 con 0x7f398c1594f0 2024-03-30T00:39:54.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:54.514+0000 7f3991400640 1 -- 172.21.15.66:0/2782165662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f398c06a7c0 con 0x7f398c1594f0 2024-03-30T00:39:55.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.153+0000 7f3983400640 1 -- 172.21.15.66:0/2782165662 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f39840180c0 con 0x7f398c1594f0 2024-03-30T00:39:55.165 INFO:teuthology.orchestra.run.smithi066.stderr:set pool 4 allow_ec_overwrites to true 2024-03-30T00:39:55.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.161+0000 7f3991400640 1 -- 172.21.15.66:0/2782165662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3968066690 msgr2=0x7f3968068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.161+0000 7f3991400640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3968066690 0x7f3968068b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f397c0ae000 tx=0x7f397c0b3040 comp rx=0 tx=0).stop 2024-03-30T00:39:55.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.161+0000 7f3991400640 1 -- 172.21.15.66:0/2782165662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398c1594f0 msgr2=0x7f398c157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.161+0000 7f3991400640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398c1594f0 0x7f398c157ce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f398400efc0 tx=0x7f398400c490 comp rx=0 tx=0).stop 2024-03-30T00:39:55.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.165+0000 7f3991400640 1 -- 172.21.15.66:0/2782165662 shutdown_connections 2024-03-30T00:39:55.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.165+0000 7f3991400640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3968066690 0x7f3968068b50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.165+0000 7f3991400640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398c1594f0 0x7f398c157ce0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.165+0000 7f3991400640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398c0cec60 0x7f398c1577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.165+0000 7f3991400640 1 --2- 172.21.15.66:0/2782165662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f398c0ce7b0 0x7f398c097960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.165+0000 7f3991400640 1 -- 172.21.15.66:0/2782165662 >> 172.21.15.66:0/2782165662 conn(0x7f398c06a190 msgr2=0x7f398c0b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:55.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.165+0000 7f3991400640 1 -- 172.21.15.66:0/2782165662 shutdown_connections 2024-03-30T00:39:55.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.165+0000 7f3991400640 1 -- 172.21.15.66:0/2782165662 wait complete. 2024-03-30T00:39:55.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-30T00:39:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0ff2200640 1 Processor -- start 2024-03-30T00:39:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0ff2200640 1 -- start start 2024-03-30T00:39:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0ff2200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0fec0cda90 0x7f0fec0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0ff2200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0fec15a370 0x7f0fec0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0ff2200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fec15a990 con 0x7f0fec0cde70 2024-03-30T00:39:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0ff2200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fec0b1b80 con 0x7f0fec15a370 2024-03-30T00:39:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0ff2200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fec0cd3e0 con 0x7f0fec0cda90 2024-03-30T00:39:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0fe2a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0fec0cde70 0x7f0fec159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48080/0 (socket says 172.21.15.66:48080) 2024-03-30T00:39:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0fe2a00640 1 -- 172.21.15.66:0/1958521681 learned_addr learned my addr 172.21.15.66:0/1958521681 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:39:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0febe00640 1 --2- 172.21.15.66:0/1958521681 >> v2:172.21.15.66:3300/0 conn(0x7f0fec15a370 0x7f0fec0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0feb400640 1 --2- 172.21.15.66:0/1958521681 >> v2:172.21.15.156:3300/0 conn(0x7f0fec0cda90 0x7f0fec0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0febe00640 1 -- 172.21.15.66:0/1958521681 >> v2:172.21.15.156:3300/0 conn(0x7f0fec0cda90 msgr2=0x7f0fec0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0febe00640 1 --2- 172.21.15.66:0/1958521681 >> v2:172.21.15.156:3300/0 conn(0x7f0fec0cda90 0x7f0fec0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0febe00640 1 -- 172.21.15.66:0/1958521681 >> v1:172.21.15.66:6789/0 conn(0x7f0fec0cde70 legacy=0x7f0fec159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.349+0000 7f0febe00640 1 -- 172.21.15.66:0/1958521681 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec0b11e0 con 0x7f0fec15a370 2024-03-30T00:39:55.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0febe00640 1 --2- 172.21.15.66:0/1958521681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fec15a370 0x7f0fec0cea60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0cf3f0 tx=0x7f0fe401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ec9753bdd421cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:55.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0feaa00640 1 -- 172.21.15.66:0/1958521681 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40a5870 con 0x7f0fec15a370 2024-03-30T00:39:55.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0feaa00640 1 -- 172.21.15.66:0/1958521681 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe40a59d0 con 0x7f0fec15a370 2024-03-30T00:39:55.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0feaa00640 1 -- 172.21.15.66:0/1958521681 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40a5d60 con 0x7f0fec15a370 2024-03-30T00:39:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0ff2200640 1 -- 172.21.15.66:0/1958521681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fec15a370 msgr2=0x7f0fec0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0ff2200640 1 --2- 172.21.15.66:0/1958521681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fec15a370 0x7f0fec0cea60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0cf3f0 tx=0x7f0fe401c8a0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0ff2200640 1 -- 172.21.15.66:0/1958521681 shutdown_connections 2024-03-30T00:39:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0ff2200640 1 --2- 172.21.15.66:0/1958521681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fec15a370 0x7f0fec0cea60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0ff2200640 1 --2- 172.21.15.66:0/1958521681 >> v2:172.21.15.156:3300/0 conn(0x7f0fec0cda90 0x7f0fec0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0ff2200640 1 -- 172.21.15.66:0/1958521681 >> 172.21.15.66:0/1958521681 conn(0x7f0fec069f60 msgr2=0x7f0fec06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0ff2200640 1 -- 172.21.15.66:0/1958521681 shutdown_connections 2024-03-30T00:39:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0ff2200640 1 -- 172.21.15.66:0/1958521681 wait complete. 2024-03-30T00:39:55.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.353+0000 7f0ff2200640 1 Processor -- start 2024-03-30T00:39:55.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0ff2200640 1 -- start start 2024-03-30T00:39:55.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0ff2200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0fec0cda90 0x7f0fec155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0ff2200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec0cde70 0x7f0fec0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0ff2200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fec15a370 0x7f0fec0c2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0ff2200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0abe90 con 0x7f0fec15a370 2024-03-30T00:39:55.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0ff2200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0ac170 con 0x7f0fec0cde70 2024-03-30T00:39:55.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0ff2200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0ac450 con 0x7f0fec0cda90 2024-03-30T00:39:55.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0febe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fec15a370 0x7f0fec0c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0febe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fec15a370 0x7f0fec0c2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51390/0 (socket says 172.21.15.66:51390) 2024-03-30T00:39:55.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0febe00640 1 -- 172.21.15.66:0/2666981130 learned_addr learned my addr 172.21.15.66:0/2666981130 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:55.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0fe2a00640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec0cde70 0x7f0fec0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0feb400640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0fec0cda90 0x7f0fec155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0febe00640 1 -- 172.21.15.66:0/2666981130 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0fec0cda90 msgr2=0x7f0fec155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0febe00640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0fec0cda90 0x7f0fec155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0febe00640 1 -- 172.21.15.66:0/2666981130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec0cde70 msgr2=0x7f0fec0aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0febe00640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec0cde70 0x7f0fec0aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0febe00640 1 -- 172.21.15.66:0/2666981130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe4009a70 con 0x7f0fec15a370 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0fe2a00640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec0cde70 0x7f0fec0aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0febe00640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fec15a370 0x7f0fec0c2200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40194b0 tx=0x7f0fe4019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0fe3e00640 1 -- 172.21.15.66:0/2666981130 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40b9860 con 0x7f0fec15a370 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0fe3e00640 1 -- 172.21.15.66:0/2666981130 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe40b99c0 con 0x7f0fec15a370 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0ff2200640 1 -- 172.21.15.66:0/2666981130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec0ac730 con 0x7f0fec15a370 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0fe3e00640 1 -- 172.21.15.66:0/2666981130 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40b9d70 con 0x7f0fec15a370 2024-03-30T00:39:55.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0ff2200640 1 -- 172.21.15.66:0/2666981130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec1566d0 con 0x7f0fec15a370 2024-03-30T00:39:55.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.357+0000 7f0ff2200640 1 -- 172.21.15.66:0/2666981130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec066c30 con 0x7f0fec15a370 2024-03-30T00:39:55.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.361+0000 7f0fe3e00640 1 -- 172.21.15.66:0/2666981130 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0fe40ad440 con 0x7f0fec15a370 2024-03-30T00:39:55.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.361+0000 7f0fe3e00640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0fc80666e0 0x7f0fc8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.361+0000 7f0fe3e00640 1 -- 172.21.15.66:0/2666981130 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f0fe411f570 con 0x7f0fec15a370 2024-03-30T00:39:55.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.361+0000 7f0feb400640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0fc80666e0 0x7f0fc8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.361+0000 7f0feb400640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0fc80666e0 0x7f0fc8068ba0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009a50 tx=0x7f0fdc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.365+0000 7f0fe3e00640 1 -- 172.21.15.66:0/2666981130 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0fe40ec700 con 0x7f0fec15a370 2024-03-30T00:39:55.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.561+0000 7f0ff2200640 1 -- 172.21.15.66:0/2666981130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f0fec165280 con 0x7f0fec15a370 2024-03-30T00:39:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.565+0000 7f0fe3e00640 1 -- 172.21.15.66:0/2666981130 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f0fe40efd20 con 0x7f0fec15a370 2024-03-30T00:39:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:added data pool 4 to fsmap 2024-03-30T00:39:55.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 -- 172.21.15.66:0/2666981130 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0fc80666e0 msgr2=0x7f0fc8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0fc80666e0 0x7f0fc8068ba0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009a50 tx=0x7f0fdc008040 comp rx=0 tx=0).stop 2024-03-30T00:39:55.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 -- 172.21.15.66:0/2666981130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fec15a370 msgr2=0x7f0fec0c2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fec15a370 0x7f0fec0c2200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40194b0 tx=0x7f0fe4019640 comp rx=0 tx=0).stop 2024-03-30T00:39:55.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 -- 172.21.15.66:0/2666981130 shutdown_connections 2024-03-30T00:39:55.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0fc80666e0 0x7f0fc8068ba0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fec15a370 0x7f0fec0c2200 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec0cde70 0x7f0fec0aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 --2- 172.21.15.66:0/2666981130 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0fec0cda90 0x7f0fec155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 -- 172.21.15.66:0/2666981130 >> 172.21.15.66:0/2666981130 conn(0x7f0fec069f60 msgr2=0x7f0fec15ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:55.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 -- 172.21.15.66:0/2666981130 shutdown_connections 2024-03-30T00:39:55.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.569+0000 7f0ff2200640 1 -- 172.21.15.66:0/2666981130 wait complete. 2024-03-30T00:39:55.597 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:39:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3ac6400640 1 Processor -- start 2024-03-30T00:39:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3ac6400640 1 -- start start 2024-03-30T00:39:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3ac6400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3ac00cad00 0x7f3ac00c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3ac6400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cd2d0 0x7f3ac00cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3ac6400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ac00cc8d0 0x7f3ac006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3ac6400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac00cbec0 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3ac6400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac00c8e80 con 0x7f3ac00cad00 2024-03-30T00:39:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3ac6400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00cb620 con 0x7f3ac00cd2d0 2024-03-30T00:39:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3abfe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ac00cc8d0 0x7f3ac006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3abfe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ac00cc8d0 0x7f3ac006c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51396/0 (socket says 172.21.15.66:51396) 2024-03-30T00:39:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3abfe00640 1 -- 172.21.15.66:0/1565441600 learned_addr learned my addr 172.21.15.66:0/1565441600 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3ab7e00640 1 --2- 172.21.15.66:0/1565441600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cd2d0 0x7f3ac00cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3abf400640 1 --2- 172.21.15.66:0/1565441600 >> v2:172.21.15.156:3300/0 conn(0x7f3ac00cad00 0x7f3ac00c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3abfe00640 1 -- 172.21.15.66:0/1565441600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cd2d0 msgr2=0x7f3ac00cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3abfe00640 1 --2- 172.21.15.66:0/1565441600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cd2d0 0x7f3ac00cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3abfe00640 1 -- 172.21.15.66:0/1565441600 >> v2:172.21.15.156:3300/0 conn(0x7f3ac00cad00 msgr2=0x7f3ac00c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3abfe00640 1 --2- 172.21.15.66:0/1565441600 >> v2:172.21.15.156:3300/0 conn(0x7f3ac00cad00 0x7f3ac00c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.729+0000 7f3abfe00640 1 -- 172.21.15.66:0/1565441600 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac00ca660 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3abfe00640 1 --2- 172.21.15.66:0/1565441600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cc8d0 0x7f3ac006c030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00c1a00 tx=0x7f3ab801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5543fc8871c42 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3abea00640 1 -- 172.21.15.66:0/1565441600 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80a5840 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3abea00640 1 -- 172.21.15.66:0/1565441600 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab80a59a0 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3abea00640 1 -- 172.21.15.66:0/1565441600 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80a5d30 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3ac6400640 1 -- 172.21.15.66:0/1565441600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cc8d0 msgr2=0x7f3ac006c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3ac6400640 1 --2- 172.21.15.66:0/1565441600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cc8d0 0x7f3ac006c030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00c1a00 tx=0x7f3ab801c9c0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3ac6400640 1 -- 172.21.15.66:0/1565441600 shutdown_connections 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3ac6400640 1 --2- 172.21.15.66:0/1565441600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cc8d0 0x7f3ac006c030 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3ac6400640 1 --2- 172.21.15.66:0/1565441600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cd2d0 0x7f3ac00cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3ac6400640 1 --2- 172.21.15.66:0/1565441600 >> v2:172.21.15.156:3300/0 conn(0x7f3ac00cad00 0x7f3ac00c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3ac6400640 1 -- 172.21.15.66:0/1565441600 >> 172.21.15.66:0/1565441600 conn(0x7f3ac0069f60 msgr2=0x7f3ac006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3ac6400640 1 -- 172.21.15.66:0/1565441600 shutdown_connections 2024-03-30T00:39:55.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3ac6400640 1 -- 172.21.15.66:0/1565441600 wait complete. 2024-03-30T00:39:55.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.733+0000 7f3ac6400640 1 Processor -- start 2024-03-30T00:39:55.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ac6400640 1 -- start start 2024-03-30T00:39:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ac6400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cad00 0x7f3ac00acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ac6400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cc8d0 0x7f3ac00c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ac6400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ac00cd2d0 0x7f3ac00c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ac6400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac01669b0 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ac6400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac0165910 con 0x7f3ac00cd2d0 2024-03-30T00:39:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ac6400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac0165bf0 con 0x7f3ac00cad00 2024-03-30T00:39:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cc8d0 0x7f3ac00c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cc8d0 0x7f3ac00c2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51412/0 (socket says 172.21.15.66:51412) 2024-03-30T00:39:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7e00640 1 -- 172.21.15.66:0/2037701692 learned_addr learned my addr 172.21.15.66:0/2037701692 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3abf400640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cad00 0x7f3ac00acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3abfe00640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ac00cd2d0 0x7f3ac00c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7e00640 1 -- 172.21.15.66:0/2037701692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cad00 msgr2=0x7f3ac00acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7e00640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cad00 0x7f3ac00acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7e00640 1 -- 172.21.15.66:0/2037701692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ac00cd2d0 msgr2=0x7f3ac00c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7e00640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ac00cd2d0 0x7f3ac00c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7e00640 1 -- 172.21.15.66:0/2037701692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab8009b30 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7e00640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cc8d0 0x7f3ac00c2cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800e8d0 tx=0x7f3aa800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7400640 1 -- 172.21.15.66:0/2037701692 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800c940 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7400640 1 -- 172.21.15.66:0/2037701692 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa800caa0 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ab7400640 1 -- 172.21.15.66:0/2037701692 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800ce10 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ac6400640 1 -- 172.21.15.66:0/2037701692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac0165ed0 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ac6400640 1 -- 172.21.15.66:0/2037701692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac0166490 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.737+0000 7f3ac6400640 1 -- 172.21.15.66:0/2037701692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac0066c30 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.741+0000 7f3ab7400640 1 -- 172.21.15.66:0/2037701692 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3aa8013470 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.741+0000 7f3ab7400640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 0x7f3a9c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:55.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.741+0000 7f3ab7400640 1 -- 172.21.15.66:0/2037701692 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3aa80846d0 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.741+0000 7f3abf400640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 0x7f3a9c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:55.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.741+0000 7f3abf400640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 0x7f3a9c068b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009a40 tx=0x7f3ab0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:55.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.745+0000 7f3ab7400640 1 -- 172.21.15.66:0/2037701692 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3aa8051760 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.937+0000 7f3ac6400640 1 -- 172.21.15.66:0/2037701692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ac015a420 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.937+0000 7f3ab7400640 1 -- 172.21.15.66:0/2037701692 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f3aa8054d80 con 0x7f3ac00cc8d0 2024-03-30T00:39:55.949 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:39:55.949 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4122431016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":4122431016},{"type":"v1","addr":"172.21.15.156:6835","nonce":4122431016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3882342907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3882342907},{"type":"v1","addr":"172.21.15.66:6835","nonce":3882342907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:39:51.142575+0000","modified":"2024-03-30T00:39:55.567140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/703895528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":703895528},{"type":"v1","addr":"172.21.15.66:6837","nonce":703895528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-30T00:39:55.949 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 7 2024-03-30T00:39:55.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 -- 172.21.15.66:0/2037701692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 msgr2=0x7f3a9c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 0x7f3a9c068b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009a40 tx=0x7f3ab0009190 comp rx=0 tx=0).stop 2024-03-30T00:39:55.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 -- 172.21.15.66:0/2037701692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cc8d0 msgr2=0x7f3ac00c2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:55.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cc8d0 0x7f3ac00c2cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800e8d0 tx=0x7f3aa800ed90 comp rx=0 tx=0).stop 2024-03-30T00:39:55.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 -- 172.21.15.66:0/2037701692 shutdown_connections 2024-03-30T00:39:55.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ac00cd2d0 0x7f3ac00c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 0x7f3a9c068b50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cc8d0 0x7f3ac00c2cc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 --2- 172.21.15.66:0/2037701692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cad00 0x7f3ac00acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:55.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 -- 172.21.15.66:0/2037701692 >> 172.21.15.66:0/2037701692 conn(0x7f3ac0069f60 msgr2=0x7f3ac00af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:55.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 -- 172.21.15.66:0/2037701692 shutdown_connections 2024-03-30T00:39:55.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:55.941+0000 7f3ac6400640 1 -- 172.21.15.66:0/2037701692 wait complete. 2024-03-30T00:39:55.969 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:39:55.982 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-03-30T00:39:55.973+0000 7fbc06000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:39:56.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.097+0000 7fb54e600640 1 Processor -- start 2024-03-30T00:39:56.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.097+0000 7fb54e600640 1 -- start start 2024-03-30T00:39:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb54e600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5480cad00 0x7fb5480c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb54e600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb5480cc8d0 0x7fb54806c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb54e600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5480c8e80 con 0x7fb5480cd2d0 2024-03-30T00:39:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb54e600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480cbec0 con 0x7fb5480cc8d0 2024-03-30T00:39:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb54e600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480ca660 con 0x7fb5480cad00 2024-03-30T00:39:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb53ea00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb5480cd2d0 0x7fb5480cb0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48082/0 (socket says 172.21.15.66:48082) 2024-03-30T00:39:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb53ea00640 1 -- 172.21.15.66:0/1258252258 learned_addr learned my addr 172.21.15.66:0/1258252258 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:39:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb547e00640 1 --2- 172.21.15.66:0/1258252258 >> v2:172.21.15.156:3300/0 conn(0x7fb5480cc8d0 0x7fb54806c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb547400640 1 --2- 172.21.15.66:0/1258252258 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5480cad00 0x7fb5480c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb546a00640 1 -- 172.21.15.66:0/1258252258 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650583089 0 0) 0x7fb5480c8e80 con 0x7fb5480cd2d0 2024-03-30T00:39:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb546a00640 1 -- 172.21.15.66:0/1258252258 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb52c003530 con 0x7fb5480cd2d0 2024-03-30T00:39:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb547e00640 1 -- 172.21.15.66:0/1258252258 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5480cad00 msgr2=0x7fb5480c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb547e00640 1 --2- 172.21.15.66:0/1258252258 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5480cad00 0x7fb5480c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb547e00640 1 -- 172.21.15.66:0/1258252258 >> v1:172.21.15.66:6789/0 conn(0x7fb5480cd2d0 legacy=0x7fb5480cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.101+0000 7fb547e00640 1 -- 172.21.15.66:0/1258252258 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480ca940 con 0x7fb5480cc8d0 2024-03-30T00:39:56.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb547e00640 1 --2- 172.21.15.66:0/1258252258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5480cc8d0 0x7fb54806c030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb540009d90 tx=0x7fb54001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20be23b1c33acab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:56.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb546a00640 1 -- 172.21.15.66:0/1258252258 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5400a5a00 con 0x7fb5480cc8d0 2024-03-30T00:39:56.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb546a00640 1 -- 172.21.15.66:0/1258252258 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5400a5b60 con 0x7fb5480cc8d0 2024-03-30T00:39:56.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb546a00640 1 -- 172.21.15.66:0/1258252258 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5400ad440 con 0x7fb5480cc8d0 2024-03-30T00:39:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb54e600640 1 -- 172.21.15.66:0/1258252258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5480cc8d0 msgr2=0x7fb54806c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb54e600640 1 --2- 172.21.15.66:0/1258252258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5480cc8d0 0x7fb54806c030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb540009d90 tx=0x7fb54001c940 comp rx=0 tx=0).stop 2024-03-30T00:39:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb54e600640 1 -- 172.21.15.66:0/1258252258 shutdown_connections 2024-03-30T00:39:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb54e600640 1 --2- 172.21.15.66:0/1258252258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5480cc8d0 0x7fb54806c030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb54e600640 1 --2- 172.21.15.66:0/1258252258 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5480cad00 0x7fb5480c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb54e600640 1 -- 172.21.15.66:0/1258252258 >> 172.21.15.66:0/1258252258 conn(0x7fb548069f60 msgr2=0x7fb54806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb54e600640 1 -- 172.21.15.66:0/1258252258 shutdown_connections 2024-03-30T00:39:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb54e600640 1 -- 172.21.15.66:0/1258252258 wait complete. 2024-03-30T00:39:56.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb54e600640 1 Processor -- start 2024-03-30T00:39:56.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.105+0000 7fb54e600640 1 -- start start 2024-03-30T00:39:56.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb54e600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5480cad00 0x7fb548070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb54e600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5480cc8d0 0x7fb5481eb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb54e600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5480cd2d0 0x7fb5480c1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb54e600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548159880 con 0x7fb5480cc8d0 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb54e600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548159b60 con 0x7fb5480cd2d0 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb54e600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb548159e40 con 0x7fb5480cad00 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5480cc8d0 0x7fb5481eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5480cc8d0 0x7fb5481eb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51422/0 (socket says 172.21.15.66:51422) 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53ea00640 1 -- 172.21.15.66:0/17760066 learned_addr learned my addr 172.21.15.66:0/17760066 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53ea00640 1 -- 172.21.15.66:0/17760066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5480cad00 msgr2=0x7fb548070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb547400640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5480cad00 0x7fb548070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb547e00640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5480cd2d0 0x7fb5480c1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53ea00640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5480cad00 0x7fb548070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53ea00640 1 -- 172.21.15.66:0/17760066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5480cd2d0 msgr2=0x7fb5480c1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53ea00640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5480cd2d0 0x7fb5480c1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53ea00640 1 -- 172.21.15.66:0/17760066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540009a70 con 0x7fb5480cc8d0 2024-03-30T00:39:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb547400640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5480cad00 0x7fb548070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:39:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53ea00640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5480cc8d0 0x7fb5481eb850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb53000c420 tx=0x7fb53000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53fe00640 1 -- 172.21.15.66:0/17760066 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530005cf0 con 0x7fb5480cc8d0 2024-03-30T00:39:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53fe00640 1 -- 172.21.15.66:0/17760066 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb530005e70 con 0x7fb5480cc8d0 2024-03-30T00:39:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb54e600640 1 -- 172.21.15.66:0/17760066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54815a120 con 0x7fb5480cc8d0 2024-03-30T00:39:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53fe00640 1 -- 172.21.15.66:0/17760066 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300072f0 con 0x7fb5480cc8d0 2024-03-30T00:39:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb54e600640 1 -- 172.21.15.66:0/17760066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb548070fe0 con 0x7fb5480cc8d0 2024-03-30T00:39:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb54e600640 1 -- 172.21.15.66:0/17760066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548066c30 con 0x7fb5480cc8d0 2024-03-30T00:39:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.109+0000 7fb53fe00640 1 -- 172.21.15.66:0/17760066 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb530007490 con 0x7fb5480cc8d0 2024-03-30T00:39:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.113+0000 7fb53fe00640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb524066690 0x7fb524068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.113+0000 7fb53fe00640 1 -- 172.21.15.66:0/17760066 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb530083b90 con 0x7fb5480cc8d0 2024-03-30T00:39:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.113+0000 7fb547400640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb524066690 0x7fb524068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.113+0000 7fb547400640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb524066690 0x7fb524068b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb53800ab40 tx=0x7fb538008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.117+0000 7fb53fe00640 1 -- 172.21.15.66:0/17760066 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb5300546c0 con 0x7fb5480cc8d0 2024-03-30T00:39:56.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.277+0000 7fb54e600640 1 -- 172.21.15.66:0/17760066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb54815a5d0 con 0x7fb5480cc8d0 2024-03-30T00:39:56.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb53fe00640 1 -- 172.21.15.66:0/17760066 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fb530017070 con 0x7fb5480cc8d0 2024-03-30T00:39:56.290 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:39:56.290 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 -- 172.21.15.66:0/17760066 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb524066690 msgr2=0x7fb524068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb524066690 0x7fb524068b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb53800ab40 tx=0x7fb538008040 comp rx=0 tx=0).stop 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 -- 172.21.15.66:0/17760066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5480cc8d0 msgr2=0x7fb5481eb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5480cc8d0 0x7fb5481eb850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb53000c420 tx=0x7fb53000c8e0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 -- 172.21.15.66:0/17760066 shutdown_connections 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5480cd2d0 0x7fb5480c1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb524066690 0x7fb524068b50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5480cc8d0 0x7fb5481eb850 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 --2- 172.21.15.66:0/17760066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5480cad00 0x7fb548070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 -- 172.21.15.66:0/17760066 >> 172.21.15.66:0/17760066 conn(0x7fb548069f60 msgr2=0x7fb54806a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 -- 172.21.15.66:0/17760066 shutdown_connections 2024-03-30T00:39:56.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.281+0000 7fb54e600640 1 -- 172.21.15.66:0/17760066 wait complete. 2024-03-30T00:39:56.311 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:39:56.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.481+0000 7f239ae00640 1 Processor -- start 2024-03-30T00:39:56.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.481+0000 7f239ae00640 1 -- start start 2024-03-30T00:39:56.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.481+0000 7f239ae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23940cec30 0x7f2394159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.481+0000 7f239ae00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23940cde70 0x7f239415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.481+0000 7f239ae00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239415a6e0 con 0x7f23940cda90 2024-03-30T00:39:56.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.481+0000 7f239ae00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23940b1b80 con 0x7f23940cde70 2024-03-30T00:39:56.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.481+0000 7f239ae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23940cd3e0 con 0x7f23940cec30 2024-03-30T00:39:56.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.481+0000 7f2398e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23940cde70 0x7f239415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.481+0000 7f238b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23940cec30 0x7f2394159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.481+0000 7f2398e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23940cde70 0x7f239415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:37154/0 (socket says 172.21.15.66:37154) 2024-03-30T00:39:56.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.481+0000 7f2398e00640 1 -- 172.21.15.66:0/2635872909 learned_addr learned my addr 172.21.15.66:0/2635872909 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:56.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f238b400640 1 -- 172.21.15.66:0/2635872909 >> v2:172.21.15.156:3300/0 conn(0x7f23940cde70 msgr2=0x7f239415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:56.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f238b400640 1 --2- 172.21.15.66:0/2635872909 >> v2:172.21.15.156:3300/0 conn(0x7f23940cde70 0x7f239415a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f238b400640 1 -- 172.21.15.66:0/2635872909 >> v1:172.21.15.156:6789/0 conn(0x7f23940cda90 legacy=0x7f23940ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f238b400640 1 -- 172.21.15.66:0/2635872909 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239415a990 con 0x7f23940cec30 2024-03-30T00:39:56.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f238b400640 1 --2- 172.21.15.66:0/2635872909 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23940cec30 0x7f2394159c60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f237c0097c0 tx=0x7f237c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06f89a7f3b23944 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:56.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f2393400640 1 -- 172.21.15.66:0/2635872909 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0ab610 con 0x7f23940cec30 2024-03-30T00:39:56.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f2393400640 1 -- 172.21.15.66:0/2635872909 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c0ab770 con 0x7f23940cec30 2024-03-30T00:39:56.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f2393400640 1 -- 172.21.15.66:0/2635872909 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0abb00 con 0x7f23940cec30 2024-03-30T00:39:56.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f239ae00640 1 -- 172.21.15.66:0/2635872909 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23940cec30 msgr2=0x7f2394159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f239ae00640 1 --2- 172.21.15.66:0/2635872909 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23940cec30 0x7f2394159c60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f237c0097c0 tx=0x7f237c01c410 comp rx=0 tx=0).stop 2024-03-30T00:39:56.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f239ae00640 1 -- 172.21.15.66:0/2635872909 shutdown_connections 2024-03-30T00:39:56.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f239ae00640 1 --2- 172.21.15.66:0/2635872909 >> v2:172.21.15.156:3300/0 conn(0x7f23940cde70 0x7f239415a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f239ae00640 1 --2- 172.21.15.66:0/2635872909 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23940cec30 0x7f2394159c60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.485+0000 7f239ae00640 1 -- 172.21.15.66:0/2635872909 >> 172.21.15.66:0/2635872909 conn(0x7f2394069f60 msgr2=0x7f239406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:56.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f239ae00640 1 -- 172.21.15.66:0/2635872909 shutdown_connections 2024-03-30T00:39:56.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f239ae00640 1 -- 172.21.15.66:0/2635872909 wait complete. 2024-03-30T00:39:56.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f239ae00640 1 Processor -- start 2024-03-30T00:39:56.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f239ae00640 1 -- start start 2024-03-30T00:39:56.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f239ae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23940cda90 0x7f2394150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f239ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23940cde70 0x7f2394159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f239ae00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23940cec30 0x7f23941588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f239ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2394070960 con 0x7f23940cde70 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f239ae00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2394097740 con 0x7f23940cec30 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f239ae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2394070c00 con 0x7f23940cda90 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f238b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23940cde70 0x7f2394159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f238b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23940cde70 0x7f2394159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51428/0 (socket says 172.21.15.66:51428) 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f238b400640 1 -- 172.21.15.66:0/866055535 learned_addr learned my addr 172.21.15.66:0/866055535 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f2398e00640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23940cec30 0x7f23941588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f2393e00640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23940cda90 0x7f2394150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f238b400640 1 -- 172.21.15.66:0/866055535 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23940cda90 msgr2=0x7f2394150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f238b400640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23940cda90 0x7f2394150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f238b400640 1 -- 172.21.15.66:0/866055535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23940cec30 msgr2=0x7f23941588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f238b400640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23940cec30 0x7f23941588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f238b400640 1 -- 172.21.15.66:0/866055535 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c009420 con 0x7f23940cde70 2024-03-30T00:39:56.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f238b400640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23940cde70 0x7f2394159330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f237c01a040 tx=0x7f237c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:56.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f2390c00640 1 -- 172.21.15.66:0/866055535 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0abdb0 con 0x7f23940cde70 2024-03-30T00:39:56.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f2390c00640 1 -- 172.21.15.66:0/866055535 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c0bf450 con 0x7f23940cde70 2024-03-30T00:39:56.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f2390c00640 1 -- 172.21.15.66:0/866055535 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c0bf7e0 con 0x7f23940cde70 2024-03-30T00:39:56.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.489+0000 7f239ae00640 1 -- 172.21.15.66:0/866055535 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2394150a50 con 0x7f23940cde70 2024-03-30T00:39:56.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.493+0000 7f239ae00640 1 -- 172.21.15.66:0/866055535 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2394151b00 con 0x7f23940cde70 2024-03-30T00:39:56.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.493+0000 7f239ae00640 1 -- 172.21.15.66:0/866055535 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2394150c40 con 0x7f23940cde70 2024-03-30T00:39:56.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.493+0000 7f2390c00640 1 -- 172.21.15.66:0/866055535 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f237c0b43f0 con 0x7f23940cde70 2024-03-30T00:39:56.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.493+0000 7f2390c00640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2370066690 0x7f2370068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.493+0000 7f2390c00640 1 -- 172.21.15.66:0/866055535 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f237c11e780 con 0x7f23940cde70 2024-03-30T00:39:56.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.493+0000 7f2393e00640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2370066690 0x7f2370068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.493+0000 7f2393e00640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2370066690 0x7f2370068b50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2384007fd0 tx=0x7f2384009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:56.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.497+0000 7f2390c00640 1 -- 172.21.15.66:0/866055535 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f237c0eb780 con 0x7f23940cde70 2024-03-30T00:39:56.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.661+0000 7f239ae00640 1 -- 172.21.15.66:0/866055535 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2394066c30 con 0x7f23940cde70 2024-03-30T00:39:56.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.661+0000 7f2390c00640 1 -- 172.21.15.66:0/866055535 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f237c0eeda0 con 0x7f23940cde70 2024-03-30T00:39:56.672 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:39:56.672 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:39:56.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 -- 172.21.15.66:0/866055535 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2370066690 msgr2=0x7f2370068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2370066690 0x7f2370068b50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2384007fd0 tx=0x7f2384009040 comp rx=0 tx=0).stop 2024-03-30T00:39:56.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 -- 172.21.15.66:0/866055535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23940cde70 msgr2=0x7f2394159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23940cde70 0x7f2394159330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f237c01a040 tx=0x7f237c0191b0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 -- 172.21.15.66:0/866055535 shutdown_connections 2024-03-30T00:39:56.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23940cec30 0x7f23941588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2370066690 0x7f2370068b50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23940cde70 0x7f2394159330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 --2- 172.21.15.66:0/866055535 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23940cda90 0x7f2394150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 -- 172.21.15.66:0/866055535 >> 172.21.15.66:0/866055535 conn(0x7f2394069f60 msgr2=0x7f23940afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:56.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 -- 172.21.15.66:0/866055535 shutdown_connections 2024-03-30T00:39:56.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.665+0000 7f239ae00640 1 -- 172.21.15.66:0/866055535 wait complete. 2024-03-30T00:39:56.710 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:39:56.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffaa4a00640 1 Processor -- start 2024-03-30T00:39:56.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffaa4a00640 1 -- start start 2024-03-30T00:39:56.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffaa4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa0159950 0x7ffaa015a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffaa4a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa015adb0 con 0x7ffaa015d390 2024-03-30T00:39:56.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffaa4a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa015d930 con 0x7ffaa006dc40 2024-03-30T00:39:56.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffaa4a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0159d30 con 0x7ffaa0159950 2024-03-30T00:39:56.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffa97e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ffaa015d390 0x7ffaa006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48086/0 (socket says 172.21.15.66:48086) 2024-03-30T00:39:56.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffa97e00640 1 -- 172.21.15.66:0/2351015937 learned_addr learned my addr 172.21.15.66:0/2351015937 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:39:56.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffa9de00640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa0159950 0x7ffaa015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829107350 0 0) 0x7ffaa015adb0 con 0x7ffaa015d390 2024-03-30T00:39:56.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa84003530 con 0x7ffaa015d390 2024-03-30T00:39:56.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 788310855 0 0) 0x7ffa84003530 con 0x7ffaa015d390 2024-03-30T00:39:56.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa0159950 msgr2=0x7ffaa015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:56.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffa9d400640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa0159950 0x7ffaa015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 >> v1:172.21.15.156:6789/0 conn(0x7ffaa006dc40 legacy=0x7ffaa00ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa015dc10 con 0x7ffaa015d390 2024-03-30T00:39:56.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.841+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7ffa88001680 con 0x7ffaa015d390 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 >> v1:172.21.15.66:6789/0 conn(0x7ffaa015d390 legacy=0x7ffaa006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9d400640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa0159950 0x7ffa840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9d400640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 0x7ffa840047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9d400640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa84004d10 0x7ffa84005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0159d30 con 0x7ffa84003f30 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8400ba40 con 0x7ffa84004d10 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8400bc00 con 0x7ffaa0159950 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa97e00640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 0x7ffa840047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9de00640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa0159950 0x7ffa840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9e800640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa84004d10 0x7ffa84005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa97e00640 1 -- 172.21.15.66:0/2351015937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa0159950 msgr2=0x7ffa840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa97e00640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa0159950 0x7ffa840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa97e00640 1 -- 172.21.15.66:0/2351015937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa84004d10 msgr2=0x7ffa84005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:56.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa97e00640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa84004d10 0x7ffa84005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa97e00640 1 -- 172.21.15.66:0/2351015937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa015dc10 con 0x7ffa84003f30 2024-03-30T00:39:56.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa97e00640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 0x7ffa840047d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800c340 tx=0x7ffa8801f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8e292b0c2ba391 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:56.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88005ae0 con 0x7ffa84003f30 2024-03-30T00:39:56.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa880ab070 con 0x7ffa84003f30 2024-03-30T00:39:56.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.845+0000 7ffa9d400640 1 -- 172.21.15.66:0/2351015937 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88007070 con 0x7ffa84003f30 2024-03-30T00:39:56.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2351015937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 msgr2=0x7ffa840047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 --2- 172.21.15.66:0/2351015937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 0x7ffa840047d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800c340 tx=0x7ffa8801f490 comp rx=0 tx=0).stop 2024-03-30T00:39:56.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffa9e800640 1 -- 172.21.15.66:0/2351015937 reap_dead start 2024-03-30T00:39:56.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2351015937 shutdown_connections 2024-03-30T00:39:56.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2351015937 >> 172.21.15.66:0/2351015937 conn(0x7ffaa0098a20 msgr2=0x7ffaa00cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:56.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2351015937 shutdown_connections 2024-03-30T00:39:56.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2351015937 wait complete. 2024-03-30T00:39:56.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 Processor -- start 2024-03-30T00:39:56.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 -- start start 2024-03-30T00:39:56.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 0x7ffaa00b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa00cf5f0 0x7ffaa014fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaa00b0a60 0x7ffaa00b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00b41b0 con 0x7ffa84003f30 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00b1420 con 0x7ffaa00b0a60 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffaa4a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00b4fd0 con 0x7ffaa00cf5f0 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffa9de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 0x7ffaa00b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffa9de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 0x7ffaa00b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51446/0 (socket says 172.21.15.66:51446) 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffa9de00640 1 -- 172.21.15.66:0/2782552497 learned_addr learned my addr 172.21.15.66:0/2782552497 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffa97e00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa00cf5f0 0x7ffaa014fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffa9e800640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaa00b0a60 0x7ffaa00b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffa9de00640 1 -- 172.21.15.66:0/2782552497 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa00cf5f0 msgr2=0x7ffaa014fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffa9de00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa00cf5f0 0x7ffaa014fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffa9de00640 1 -- 172.21.15.66:0/2782552497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaa00b0a60 msgr2=0x7ffaa00b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:56.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffa9de00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaa00b0a60 0x7ffaa00b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:56.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.849+0000 7ffa9de00640 1 -- 172.21.15.66:0/2782552497 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00b43c0 con 0x7ffa84003f30 2024-03-30T00:39:56.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.853+0000 7ffa9de00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 0x7ffaa00b3c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa90011a00 tx=0x7ffa90011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:56.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.853+0000 7ffa96000640 1 -- 172.21.15.66:0/2782552497 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000fb50 con 0x7ffa84003f30 2024-03-30T00:39:56.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.853+0000 7ffa96000640 1 -- 172.21.15.66:0/2782552497 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9000fcb0 con 0x7ffa84003f30 2024-03-30T00:39:56.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.853+0000 7ffa96000640 1 -- 172.21.15.66:0/2782552497 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90014440 con 0x7ffa84003f30 2024-03-30T00:39:56.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.853+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2782552497 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa00b46a0 con 0x7ffa84003f30 2024-03-30T00:39:56.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.853+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2782552497 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa00b4c60 con 0x7ffa84003f30 2024-03-30T00:39:56.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.853+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2782552497 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa0072da0 con 0x7ffa84003f30 2024-03-30T00:39:56.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.853+0000 7ffa96000640 1 -- 172.21.15.66:0/2782552497 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ffa900145a0 con 0x7ffa84003f30 2024-03-30T00:39:56.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.857+0000 7ffa96000640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa7c066690 0x7ffa7c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:56.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.857+0000 7ffa96000640 1 -- 172.21.15.66:0/2782552497 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffa90084e10 con 0x7ffa84003f30 2024-03-30T00:39:56.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.857+0000 7ffa97e00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa7c066690 0x7ffa7c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:56.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.857+0000 7ffa97e00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa7c066690 0x7ffa7c068b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa880af000 tx=0x7ffa880aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:56.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:56.861+0000 7ffa96000640 1 -- 172.21.15.66:0/2782552497 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ffa90088030 con 0x7ffa84003f30 2024-03-30T00:39:57.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.057+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2782552497 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaa00c6f80 con 0x7ffa84003f30 2024-03-30T00:39:57.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.057+0000 7ffa96000640 1 -- 172.21.15.66:0/2782552497 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4730 (secure 0 0 0) 0x7ffa90022020 con 0x7ffa84003f30 2024-03-30T00:39:57.068 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:39:57.068 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4122431016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":4122431016},{"type":"v1","addr":"172.21.15.156:6835","nonce":4122431016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3882342907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3882342907},{"type":"v1","addr":"172.21.15.66:6835","nonce":3882342907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:39:51.142575+0000","modified":"2024-03-30T00:39:55.567140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/703895528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":703895528},{"type":"v1","addr":"172.21.15.66:6837","nonce":703895528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-30T00:39:57.068 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 7 2024-03-30T00:39:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2782552497 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa7c066690 msgr2=0x7ffa7c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa7c066690 0x7ffa7c068b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa880af000 tx=0x7ffa880aa000 comp rx=0 tx=0).stop 2024-03-30T00:39:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2782552497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 msgr2=0x7ffaa00b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 0x7ffaa00b3c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa90011a00 tx=0x7ffa90011ec0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2782552497 shutdown_connections 2024-03-30T00:39:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaa00b0a60 0x7ffaa00b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaa00cf5f0 0x7ffaa014fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa7c066690 0x7ffa7c068b50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 --2- 172.21.15.66:0/2782552497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa84003f30 0x7ffaa00b3c70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2782552497 >> 172.21.15.66:0/2782552497 conn(0x7ffaa0098a20 msgr2=0x7ffaa00ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2782552497 shutdown_connections 2024-03-30T00:39:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.061+0000 7ffaa4a00640 1 -- 172.21.15.66:0/2782552497 wait complete. 2024-03-30T00:39:57.089 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:39:57.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.221+0000 7fb028000640 1 Processor -- start 2024-03-30T00:39:57.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.221+0000 7fb028000640 1 -- start start 2024-03-30T00:39:57.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb028000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb020098c80 0x7fb020099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb028000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb020099ca0 0x7fb0200ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb028000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0200cbd90 con 0x7fb0200cf320 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb028000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0200cc070 con 0x7fb020098c80 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb028000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0200cc350 con 0x7fb020099ca0 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb025600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb020098c80 0x7fb020099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb025600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb020098c80 0x7fb020099760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51454/0 (socket says 172.21.15.66:51454) 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb025600640 1 -- 172.21.15.66:0/1451635406 learned_addr learned my addr 172.21.15.66:0/1451635406 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb026000640 1 --2- 172.21.15.66:0/1451635406 >> v2:172.21.15.156:3300/0 conn(0x7fb020099ca0 0x7fb0200ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb025600640 1 -- 172.21.15.66:0/1451635406 >> v2:172.21.15.156:3300/0 conn(0x7fb020099ca0 msgr2=0x7fb0200ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb025600640 1 --2- 172.21.15.66:0/1451635406 >> v2:172.21.15.156:3300/0 conn(0x7fb020099ca0 0x7fb0200ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb025600640 1 -- 172.21.15.66:0/1451635406 >> v1:172.21.15.156:6789/0 conn(0x7fb0200cf320 legacy=0x7fb0200ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:57.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.225+0000 7fb025600640 1 -- 172.21.15.66:0/1451635406 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0200b3530 con 0x7fb020098c80 2024-03-30T00:39:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb025600640 1 --2- 172.21.15.66:0/1451635406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb020098c80 0x7fb020099760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb0200b3200 tx=0x7fb01401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6782854bb5f1dcbe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:57.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb01fe00640 1 -- 172.21.15.66:0/1451635406 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140a89d0 con 0x7fb020098c80 2024-03-30T00:39:57.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb01fe00640 1 -- 172.21.15.66:0/1451635406 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0140a8b30 con 0x7fb020098c80 2024-03-30T00:39:57.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb01fe00640 1 -- 172.21.15.66:0/1451635406 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140b0440 con 0x7fb020098c80 2024-03-30T00:39:57.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 -- 172.21.15.66:0/1451635406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb020098c80 msgr2=0x7fb020099760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:57.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 --2- 172.21.15.66:0/1451635406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb020098c80 0x7fb020099760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb0200b3200 tx=0x7fb01401ecd0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 -- 172.21.15.66:0/1451635406 shutdown_connections 2024-03-30T00:39:57.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 --2- 172.21.15.66:0/1451635406 >> v2:172.21.15.156:3300/0 conn(0x7fb020099ca0 0x7fb0200ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 --2- 172.21.15.66:0/1451635406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb020098c80 0x7fb020099760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 -- 172.21.15.66:0/1451635406 >> 172.21.15.66:0/1451635406 conn(0x7fb0200b2130 msgr2=0x7fb0200b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:57.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 -- 172.21.15.66:0/1451635406 shutdown_connections 2024-03-30T00:39:57.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 -- 172.21.15.66:0/1451635406 wait complete. 2024-03-30T00:39:57.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 Processor -- start 2024-03-30T00:39:57.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 -- start start 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020098c80 0x7fb020151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb020099ca0 0x7fb0200b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0200cf320 0x7fb0200b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb020152090 con 0x7fb020099ca0 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0200ad360 con 0x7fb020098c80 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb028000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0200ad640 con 0x7fb0200cf320 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.229+0000 7fb024c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb020099ca0 0x7fb0200b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb024c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb020099ca0 0x7fb0200b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51462/0 (socket says 172.21.15.66:51462) 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb024c00640 1 -- 172.21.15.66:0/2437884011 learned_addr learned my addr 172.21.15.66:0/2437884011 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb025600640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020098c80 0x7fb020151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb026000640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0200cf320 0x7fb0200b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb024c00640 1 -- 172.21.15.66:0/2437884011 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0200cf320 msgr2=0x7fb0200b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:57.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb024c00640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0200cf320 0x7fb0200b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb024c00640 1 -- 172.21.15.66:0/2437884011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020098c80 msgr2=0x7fb020151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:57.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb024c00640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020098c80 0x7fb020151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb024c00640 1 -- 172.21.15.66:0/2437884011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0140094d0 con 0x7fb020099ca0 2024-03-30T00:39:57.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb024c00640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb020099ca0 0x7fb0200b0820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00b410 tx=0x7fb00c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:57.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb01d600640 1 -- 172.21.15.66:0/2437884011 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c004020 con 0x7fb020099ca0 2024-03-30T00:39:57.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb01d600640 1 -- 172.21.15.66:0/2437884011 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c004180 con 0x7fb020099ca0 2024-03-30T00:39:57.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb01d600640 1 -- 172.21.15.66:0/2437884011 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c011630 con 0x7fb020099ca0 2024-03-30T00:39:57.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb028000640 1 -- 172.21.15.66:0/2437884011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0200ad920 con 0x7fb020099ca0 2024-03-30T00:39:57.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb028000640 1 -- 172.21.15.66:0/2437884011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0200adee0 con 0x7fb020099ca0 2024-03-30T00:39:57.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb028000640 1 -- 172.21.15.66:0/2437884011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0200677d0 con 0x7fb020099ca0 2024-03-30T00:39:57.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb01d600640 1 -- 172.21.15.66:0/2437884011 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb00c011790 con 0x7fb020099ca0 2024-03-30T00:39:57.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb01d600640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb0000666c0 0x7fb000068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:57.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.233+0000 7fb01d600640 1 -- 172.21.15.66:0/2437884011 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb00c084160 con 0x7fb020099ca0 2024-03-30T00:39:57.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.237+0000 7fb025600640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb0000666c0 0x7fb000068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:57.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.237+0000 7fb025600640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb0000666c0 0x7fb000068b80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb0140b2000 tx=0x7fb014009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:57.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.237+0000 7fb01d600640 1 -- 172.21.15.66:0/2437884011 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb00c0511f0 con 0x7fb020099ca0 2024-03-30T00:39:57.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.401+0000 7fb028000640 1 -- 172.21.15.66:0/2437884011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0200ae100 con 0x7fb020099ca0 2024-03-30T00:39:57.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb01d600640 1 -- 172.21.15.66:0/2437884011 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fb00c054810 con 0x7fb020099ca0 2024-03-30T00:39:57.413 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:39:57.414 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:39:57.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 -- 172.21.15.66:0/2437884011 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb0000666c0 msgr2=0x7fb000068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:57.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb0000666c0 0x7fb000068b80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb0140b2000 tx=0x7fb014009040 comp rx=0 tx=0).stop 2024-03-30T00:39:57.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 -- 172.21.15.66:0/2437884011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb020099ca0 msgr2=0x7fb0200b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb020099ca0 0x7fb0200b0820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00b410 tx=0x7fb00c00b8d0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 -- 172.21.15.66:0/2437884011 shutdown_connections 2024-03-30T00:39:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0200cf320 0x7fb0200b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb0000666c0 0x7fb000068b80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb020099ca0 0x7fb0200b0820 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 --2- 172.21.15.66:0/2437884011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb020098c80 0x7fb020151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 -- 172.21.15.66:0/2437884011 >> 172.21.15.66:0/2437884011 conn(0x7fb0200b2130 msgr2=0x7fb0200b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 -- 172.21.15.66:0/2437884011 shutdown_connections 2024-03-30T00:39:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.405+0000 7fb028000640 1 -- 172.21.15.66:0/2437884011 wait complete. 2024-03-30T00:39:57.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:39:51.142575+0000', 'modified': '2024-03-30T00:39:55.567140+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6837/703895528', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 703895528}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 703895528}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-30T00:39:57.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:39:57.837 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T00:39:57.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee8bc00640 1 Processor -- start 2024-03-30T00:39:57.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee8bc00640 1 -- start start 2024-03-30T00:39:57.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee8bc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee8415de30 0x7fee840cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:57.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee8bc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fee840cee50 0x7fee840d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee8bc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee840cbb90 con 0x7fee840af420 2024-03-30T00:39:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee8bc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee840ce1b0 con 0x7fee840cee50 2024-03-30T00:39:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee8bc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee840afa10 con 0x7fee8415de30 2024-03-30T00:39:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fee840af420 0x7fee840ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48096/0 (socket says 172.21.15.66:48096) 2024-03-30T00:39:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83e00640 1 -- 172.21.15.66:0/2806789586 learned_addr learned my addr 172.21.15.66:0/2806789586 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:39:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee89c00640 1 --2- 172.21.15.66:0/2806789586 >> v2:172.21.15.156:3300/0 conn(0x7fee840cee50 0x7fee840d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee89200640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee8415de30 0x7fee840cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1313073314 0 0) 0x7fee840cbb90 con 0x7fee840af420 2024-03-30T00:39:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee6c003530 con 0x7fee840af420 2024-03-30T00:39:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3826288847 0 0) 0x7fee6c003530 con 0x7fee840af420 2024-03-30T00:39:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee8415de30 msgr2=0x7fee840cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83400640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee8415de30 0x7fee840cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 >> v2:172.21.15.156:3300/0 conn(0x7fee840cee50 msgr2=0x7fee840d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83400640 1 --2- 172.21.15.66:0/2806789586 >> v2:172.21.15.156:3300/0 conn(0x7fee840cee50 0x7fee840d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee840cf430 con 0x7fee840af420 2024-03-30T00:39:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fee700016a0 con 0x7fee840af420 2024-03-30T00:39:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee89200640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee8415de30 0x7fee840cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee89c00640 1 --2- 172.21.15.66:0/2806789586 >> v2:172.21.15.156:3300/0 conn(0x7fee840cee50 0x7fee840d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 >> v1:172.21.15.66:6789/0 conn(0x7fee840af420 legacy=0x7fee840ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.981+0000 7fee83400640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee6c003980 0x7fee6c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83400640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 0x7fee6c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83400640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee8415de30 0x7fee6c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee840ce1b0 con 0x7fee6c0047d0 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee840afa10 con 0x7fee6c003980 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c00b990 con 0x7fee8415de30 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83e00640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 0x7fee6c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee89200640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee6c003980 0x7fee6c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee89c00640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee8415de30 0x7fee6c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83e00640 1 -- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee8415de30 msgr2=0x7fee6c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83e00640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee8415de30 0x7fee6c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83e00640 1 -- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee6c003980 msgr2=0x7fee6c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83e00640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee6c003980 0x7fee6c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83e00640 1 -- 172.21.15.66:0/2806789586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee78009420 con 0x7fee6c0047d0 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83e00640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 0x7fee6c004c50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c210 tx=0x7fee7001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305b4164acd80973 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee700058e0 con 0x7fee6c0047d0 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee70005a40 con 0x7fee6c0047d0 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.985+0000 7fee83400640 1 -- 172.21.15.66:0/2806789586 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70005df0 con 0x7fee6c0047d0 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 -- 172.21.15.66:0/2806789586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 msgr2=0x7fee6c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 --2- 172.21.15.66:0/2806789586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 0x7fee6c004c50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c210 tx=0x7fee7001cc60 comp rx=0 tx=0).stop 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee89c00640 1 -- 172.21.15.66:0/2806789586 reap_dead start 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 -- 172.21.15.66:0/2806789586 shutdown_connections 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 -- 172.21.15.66:0/2806789586 >> 172.21.15.66:0/2806789586 conn(0x7fee840d0540 msgr2=0x7fee840d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 -- 172.21.15.66:0/2806789586 shutdown_connections 2024-03-30T00:39:58.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 -- 172.21.15.66:0/2806789586 wait complete. 2024-03-30T00:39:58.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 Processor -- start 2024-03-30T00:39:58.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 -- start start 2024-03-30T00:39:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 0x7fee840b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee840b5b50 0x7fee8406c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840c6a20 0x7fee84148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee840b5730 con 0x7fee6c0047d0 2024-03-30T00:39:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee840b4ab0 con 0x7fee840c6a20 2024-03-30T00:39:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee8bc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee840b4d90 con 0x7fee840b5b50 2024-03-30T00:39:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee89200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 0x7fee840b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee89200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 0x7fee840b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51472/0 (socket says 172.21.15.66:51472) 2024-03-30T00:39:58.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee89200640 1 -- 172.21.15.66:0/1586899366 learned_addr learned my addr 172.21.15.66:0/1586899366 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:58.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee89c00640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840c6a20 0x7fee84148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee83e00640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee840b5b50 0x7fee8406c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee89200640 1 -- 172.21.15.66:0/1586899366 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee840b5b50 msgr2=0x7fee8406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee89200640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee840b5b50 0x7fee8406c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee89200640 1 -- 172.21.15.66:0/1586899366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840c6a20 msgr2=0x7fee84148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee89200640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840c6a20 0x7fee84148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee89200640 1 -- 172.21.15.66:0/1586899366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee840b5070 con 0x7fee6c0047d0 2024-03-30T00:39:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.989+0000 7fee89200640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 0x7fee840b43b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fee78011ca0 tx=0x7fee7800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.993+0000 7fee80c00640 1 -- 172.21.15.66:0/1586899366 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7800fd20 con 0x7fee6c0047d0 2024-03-30T00:39:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.993+0000 7fee80c00640 1 -- 172.21.15.66:0/1586899366 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7800fe80 con 0x7fee6c0047d0 2024-03-30T00:39:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.993+0000 7fee80c00640 1 -- 172.21.15.66:0/1586899366 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78004240 con 0x7fee6c0047d0 2024-03-30T00:39:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.993+0000 7fee8bc00640 1 -- 172.21.15.66:0/1586899366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee840b5350 con 0x7fee6c0047d0 2024-03-30T00:39:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.993+0000 7fee8bc00640 1 -- 172.21.15.66:0/1586899366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee84149040 con 0x7fee6c0047d0 2024-03-30T00:39:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.993+0000 7fee80c00640 1 -- 172.21.15.66:0/1586899366 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fee780043a0 con 0x7fee6c0047d0 2024-03-30T00:39:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.993+0000 7fee80c00640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee640666c0 0x7fee64068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.993+0000 7fee80c00640 1 -- 172.21.15.66:0/1586899366 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fee78084970 con 0x7fee6c0047d0 2024-03-30T00:39:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.993+0000 7fee83e00640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee640666c0 0x7fee64068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.993+0000 7fee8f0c2000 1 Processor -- start 2024-03-30T00:39:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.993+0000 7fee8f0c2000 1 -- start start 2024-03-30T00:39:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee83e00640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee640666c0 0x7fee64068b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fee700aa000 tx=0x7fee70001aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f78db0 0x55da44f79190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f796d0 0x55da44f2ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 0x55da44f2f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da44df75a0 con 0x55da44f2f4c0 2024-03-30T00:39:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da44d84af0 con 0x55da44f796d0 2024-03-30T00:39:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da44d820c0 con 0x55da44f78db0 2024-03-30T00:39:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee89200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f78db0 0x55da44f79190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee83e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f796d0 0x55da44f2ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee89c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 0x55da44f2f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee89200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f78db0 0x55da44f79190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:37206/0 (socket says 172.21.15.66:37206) 2024-03-30T00:39:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee89200640 1 -- 172.21.15.66:0/2825296199 learned_addr learned my addr 172.21.15.66:0/2825296199 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee89200640 1 -- 172.21.15.66:0/2825296199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f796d0 msgr2=0x55da44f2ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee89200640 1 --2- 172.21.15.66:0/2825296199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f796d0 0x55da44f2ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee89200640 1 -- 172.21.15.66:0/2825296199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 msgr2=0x55da44f2f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee89200640 1 --2- 172.21.15.66:0/2825296199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 0x55da44f2f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee89200640 1 -- 172.21.15.66:0/2825296199 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da44a79890 con 0x55da44f78db0 2024-03-30T00:39:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee89200640 1 --2- 172.21.15.66:0/2825296199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f78db0 0x55da44f79190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee78055450 tx=0x7fee78055290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7347d0b912c750f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee75600640 1 -- 172.21.15.66:0/2825296199 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7806d070 con 0x55da44f78db0 2024-03-30T00:39:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee75600640 1 -- 172.21.15.66:0/2825296199 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee780594d0 con 0x55da44f78db0 2024-03-30T00:39:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee75600640 1 -- 172.21.15.66:0/2825296199 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7805da90 con 0x55da44f78db0 2024-03-30T00:39:58.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2825296199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f78db0 msgr2=0x55da44f79190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/2825296199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f78db0 0x55da44f79190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee78055450 tx=0x7fee78055290 comp rx=0 tx=0).stop 2024-03-30T00:39:58.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2825296199 shutdown_connections 2024-03-30T00:39:58.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/2825296199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 0x55da44f2f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/2825296199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f796d0 0x55da44f2ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/2825296199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f78db0 0x55da44f79190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2825296199 >> 172.21.15.66:0/2825296199 conn(0x55da44f35830 msgr2=0x55da44f2d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:58.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2825296199 shutdown_connections 2024-03-30T00:39:58.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2825296199 wait complete. 2024-03-30T00:39:58.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 -- start start 2024-03-30T00:39:58.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:57.997+0000 7fee8f0c2000 1 Processor -- start 2024-03-30T00:39:58.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f2f4c0 0x55da44f34090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44f3a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f3aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da44d7d9b0 con 0x55da44f796d0 2024-03-30T00:39:58.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da44d1cc00 con 0x55da44f78db0 2024-03-30T00:39:58.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da44d3b070 con 0x55da44f2f4c0 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 10 client.0 _ll_register_callbacks cb 0x55da44f1b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f3aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f3aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51488/0 (socket says 172.21.15.66:51488) 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89c00640 1 -- 172.21.15.66:0/2546366916 learned_addr learned my addr 172.21.15.66:0/2546366916 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89200640 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f2f4c0 0x55da44f34090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee83e00640 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44f3a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89c00640 1 -- 172.21.15.66:0/2546366916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f2f4c0 msgr2=0x55da44f34090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89c00640 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f2f4c0 0x55da44f34090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89c00640 1 -- 172.21.15.66:0/2546366916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 msgr2=0x55da44f3a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89c00640 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44f3a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89c00640 1 -- 172.21.15.66:0/2546366916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee84149040 con 0x55da44f796d0 2024-03-30T00:39:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89c00640 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f3aaf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee840cc6d0 tx=0x7fee7c015760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc1754324eee524 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee6a000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee6a000640 1 -- 172.21.15.66:0/2546366916 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00e040 con 0x55da44f796d0 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee6a000640 1 -- 172.21.15.66:0/2546366916 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fee7c020d90 con 0x55da44f796d0 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2546366916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da44d3a5f0 con 0x55da44f796d0 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee6a000640 1 -- 172.21.15.66:0/2546366916 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee7c0039a0 con 0x55da44f796d0 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 10 client.4422 fetch_fsmap learned FSMap version 7 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2546366916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55da44d42ab0 con 0x55da44f796d0 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee6a000640 1 -- 172.21.15.66:0/2546366916 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da44d3a5f0 con 0x55da44f796d0 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee6a000640 1 -- 172.21.15.66:0/2546366916 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fee7c003b00 con 0x55da44f796d0 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 10 client.4422 fetch_fsmap finished waiting for FSMap version 7 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 10 client.4422 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:39:58.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 20 client.4422 populate_metadata read hostname 'smithi066' 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55da44f7f6b0 0x55da44f7fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 4 client.4422 mds_command: new command op to 4299 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2546366916 --> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da44d7a790 con 0x55da44f7f6b0 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89200640 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55da44f7f6b0 0x55da44f7fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee89200640 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55da44f7f6b0 0x55da44f7fa90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f03c387a426e7d8 server_cookie=9fe486ce290d12bf in_seq=0 out_seq=0 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee6a000640 10 client.4422 ms_handle_connect on v2:172.21.15.66:6836/703895528 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee6a000640 1 -- 172.21.15.66:0/2546366916 <== mds.0 v2:172.21.15.66:6836/703895528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55da44d7a790 con 0x55da44f7f6b0 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee6a000640 10 client.4422 handle_command_reply: tid=0 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.001+0000 7fee8f0c2000 1 client.4422 shutdown 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2546366916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 msgr2=0x55da44f3aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f3aaf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee840cc6d0 tx=0x7fee7c015760 comp rx=0 tx=0).stop 2024-03-30T00:39:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2546366916 shutdown_connections 2024-03-30T00:39:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55da44f7f6b0 0x55da44f7fa90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f3aaf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44f3a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/2546366916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f2f4c0 0x55da44f34090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee6a000640 0 client.4422 ms_handle_reset on v2:172.21.15.66:6836/703895528 2024-03-30T00:39:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2546366916 >> 172.21.15.66:0/2546366916 conn(0x55da44f35830 msgr2=0x55da44f2df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2546366916 shutdown_connections 2024-03-30T00:39:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee8f0c2000 1 -- 172.21.15.66:0/2546366916 wait complete. 2024-03-30T00:39:58.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.005+0000 7fee8f0c2000 20 client.4422 trim_cache size 0 max 16384 2024-03-30T00:39:58.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.017+0000 7fee8f0c2000 1 Processor -- start 2024-03-30T00:39:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.017+0000 7fee8f0c2000 1 -- start start 2024-03-30T00:39:58.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 0x55da44fd2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44fd3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f796d0 0x55da44fd36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da44a3a940 con 0x55da44f2f4c0 2024-03-30T00:39:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da44d7b1c0 con 0x55da44f78db0 2024-03-30T00:39:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da44f32dc0 con 0x55da44f796d0 2024-03-30T00:39:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee89200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 0x55da44fd2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee89200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 0x55da44fd2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51496/0 (socket says 172.21.15.66:51496) 2024-03-30T00:39:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee89200640 1 -- 172.21.15.66:0/3606257620 learned_addr learned my addr 172.21.15.66:0/3606257620 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee89c00640 1 --2- 172.21.15.66:0/3606257620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f796d0 0x55da44fd36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee83e00640 1 --2- 172.21.15.66:0/3606257620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44fd3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee89200640 1 -- 172.21.15.66:0/3606257620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f796d0 msgr2=0x55da44fd36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee89200640 1 --2- 172.21.15.66:0/3606257620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f796d0 0x55da44fd36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee89200640 1 -- 172.21.15.66:0/3606257620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 msgr2=0x55da44fd3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee89200640 1 --2- 172.21.15.66:0/3606257620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44fd3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee89200640 1 -- 172.21.15.66:0/3606257620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da44d28c00 con 0x55da44f2f4c0 2024-03-30T00:39:58.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee89c00640 1 --2- 172.21.15.66:0/3606257620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f796d0 0x55da44fd36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:39:58.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee83e00640 1 --2- 172.21.15.66:0/3606257620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44fd3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:39:58.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee89200640 1 --2- 172.21.15.66:0/3606257620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 0x55da44fd2c30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee78009850 tx=0x7fee780553c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c683553a1696ace1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:58.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee76000640 1 -- 172.21.15.66:0/3606257620 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7806d040 con 0x55da44f2f4c0 2024-03-30T00:39:58.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee76000640 1 -- 172.21.15.66:0/3606257620 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7805da90 con 0x55da44f2f4c0 2024-03-30T00:39:58.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee76000640 1 -- 172.21.15.66:0/3606257620 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78059170 con 0x55da44f2f4c0 2024-03-30T00:39:58.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- 172.21.15.66:0/3606257620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 msgr2=0x55da44fd2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/3606257620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 0x55da44fd2c30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee78009850 tx=0x7fee780553c0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- 172.21.15.66:0/3606257620 shutdown_connections 2024-03-30T00:39:58.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/3606257620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f796d0 0x55da44fd36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/3606257620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44fd3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/3606257620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f2f4c0 0x55da44fd2c30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- 172.21.15.66:0/3606257620 >> 172.21.15.66:0/3606257620 conn(0x55da44f35830 msgr2=0x55da44f77540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:58.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- 172.21.15.66:0/3606257620 shutdown_connections 2024-03-30T00:39:58.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- 172.21.15.66:0/3606257620 wait complete. 2024-03-30T00:39:58.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- start start 2024-03-30T00:39:58.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 Processor -- start 2024-03-30T00:39:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f2f4c0 0x55da44fd71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44fd76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f7d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da44d71530 con 0x55da44f796d0 2024-03-30T00:39:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da44d3f6c0 con 0x55da44f78db0 2024-03-30T00:39:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da44d48820 con 0x55da44f2f4c0 2024-03-30T00:39:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.021+0000 7fee8f0c2000 10 client.0 _ll_register_callbacks cb 0x55da44d42590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:39:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f7d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f2f4c0 0x55da44fd71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f7d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51500/0 (socket says 172.21.15.66:51500) 2024-03-30T00:39:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89c00640 1 -- 172.21.15.66:0/1313353195 learned_addr learned my addr 172.21.15.66:0/1313353195 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89200640 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f2f4c0 0x55da44fd71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:37236/0 (socket says 172.21.15.66:37236) 2024-03-30T00:39:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee83e00640 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44fd76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89c00640 1 -- 172.21.15.66:0/1313353195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f2f4c0 msgr2=0x55da44fd71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89c00640 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f2f4c0 0x55da44fd71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89c00640 1 -- 172.21.15.66:0/1313353195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 msgr2=0x55da44fd76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:39:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89c00640 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44fd76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89c00640 1 -- 172.21.15.66:0/1313353195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da44a79890 con 0x55da44f796d0 2024-03-30T00:39:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89c00640 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f7d2e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0157a0 tx=0x7fee7c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06417bf5518f00b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee6a000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:39:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee6a000640 1 -- 172.21.15.66:0/1313353195 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00e040 con 0x55da44f796d0 2024-03-30T00:39:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee6a000640 1 -- 172.21.15.66:0/1313353195 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fee7c004250 con 0x55da44f796d0 2024-03-30T00:39:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 -- 172.21.15.66:0/1313353195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da44d4d460 con 0x55da44f796d0 2024-03-30T00:39:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee6a000640 1 -- 172.21.15.66:0/1313353195 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee7c0098f0 con 0x55da44f796d0 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 10 client.4428 fetch_fsmap learned FSMap version 7 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 -- 172.21.15.66:0/1313353195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55da44a79cc0 con 0x55da44f796d0 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee6a000640 1 -- 172.21.15.66:0/1313353195 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da44d4d460 con 0x55da44f796d0 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee6a000640 1 -- 172.21.15.66:0/1313353195 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fee7c004410 con 0x55da44f796d0 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 10 client.4428 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 20 client.4428 populate_metadata read hostname 'smithi066' 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55da44fde010 0x55da44fe0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 4 client.4428 mds_command: new command op to 4299 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 -- 172.21.15.66:0/1313353195 --> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55da44d469c0 con 0x55da44fde010 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89200640 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55da44fde010 0x55da44fe0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee89200640 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55da44fde010 0x55da44fe0400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc239b4c1ac6de62 server_cookie=b9ce16f52b550f3f in_seq=0 out_seq=0 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee6a000640 10 client.4428 ms_handle_connect on v2:172.21.15.66:6836/703895528 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee6a000640 1 -- 172.21.15.66:0/1313353195 <== mds.0 v2:172.21.15.66:6836/703895528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55da44d469c0 con 0x55da44fde010 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee6a000640 10 client.4428 handle_command_reply: tid=0 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 client.4428 shutdown 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 -- 172.21.15.66:0/1313353195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 msgr2=0x55da44f7d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f7d2e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0157a0 tx=0x7fee7c004ac0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 -- 172.21.15.66:0/1313353195 shutdown_connections 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55da44fde010 0x55da44fe0400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55da44f796d0 0x55da44f7d2e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da44f78db0 0x55da44fd76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 --2- 172.21.15.66:0/1313353195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55da44f2f4c0 0x55da44fd71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee6a000640 0 client.4428 ms_handle_reset on v2:172.21.15.66:6836/703895528 2024-03-30T00:39:58.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 -- 172.21.15.66:0/1313353195 >> 172.21.15.66:0/1313353195 conn(0x55da44f35830 msgr2=0x55da44f34c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 -- 172.21.15.66:0/1313353195 shutdown_connections 2024-03-30T00:39:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 1 -- 172.21.15.66:0/1313353195 wait complete. 2024-03-30T00:39:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8f0c2000 20 client.4428 trim_cache size 0 max 16384 2024-03-30T00:39:58.043 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:39:58.043 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:39:58.043 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:39:58.043 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4299, 2024-03-30T00:39:58.043 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:39:58.044 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:39:58.044 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:39:58.044 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 6.8852187799999998, 2024-03-30T00:39:58.044 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 7, 2024-03-30T00:39:58.044 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 25, 2024-03-30T00:39:58.044 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 24, 2024-03-30T00:39:58.044 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 13.004521629999999 2024-03-30T00:39:58.044 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:39:58.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 -- 172.21.15.66:0/1586899366 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee640666c0 msgr2=0x7fee64068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee640666c0 0x7fee64068b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fee700aa000 tx=0x7fee70001aa0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 -- 172.21.15.66:0/1586899366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 msgr2=0x7fee840b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 0x7fee840b43b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fee78011ca0 tx=0x7fee7800f610 comp rx=0 tx=0).stop 2024-03-30T00:39:58.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 -- 172.21.15.66:0/1586899366 shutdown_connections 2024-03-30T00:39:58.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840c6a20 0x7fee84148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee840b5b50 0x7fee8406c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee640666c0 0x7fee64068b80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 --2- 172.21.15.66:0/1586899366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee6c0047d0 0x7fee840b43b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 -- 172.21.15.66:0/1586899366 >> 172.21.15.66:0/1586899366 conn(0x7fee840d0540 msgr2=0x7fee8415ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:58.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 -- 172.21.15.66:0/1586899366 shutdown_connections 2024-03-30T00:39:58.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.025+0000 7fee8bc00640 1 -- 172.21.15.66:0/1586899366 wait complete. 2024-03-30T00:39:58.055 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T00:39:58.056 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-30T00:39:58.056 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:39:58.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f7600600640 1 Processor -- start 2024-03-30T00:39:58.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f7600600640 1 -- start start 2024-03-30T00:39:58.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f7600600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f75f806e800 0x7f75f80d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f7600600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f806b0b0 con 0x7f75f806f8e0 2024-03-30T00:39:58.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f7600600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f806b390 con 0x7f75f80cc560 2024-03-30T00:39:58.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f7600600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f8160640 con 0x7f75f806e800 2024-03-30T00:39:58.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f75fd200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f75f806e800 0x7f75f80d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f75fd200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f75f806e800 0x7f75f80d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51506/0 (socket says 172.21.15.66:51506) 2024-03-30T00:39:58.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f75fd200640 1 -- 172.21.15.66:0/2230382660 learned_addr learned my addr 172.21.15.66:0/2230382660 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:58.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f75fd200640 1 -- 172.21.15.66:0/2230382660 >> v1:172.21.15.156:6789/0 conn(0x7f75f80cc560 legacy=0x7f75f806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f75fd200640 1 -- 172.21.15.66:0/2230382660 >> v1:172.21.15.66:6789/0 conn(0x7f75f806f8e0 legacy=0x7f75f80ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.189+0000 7f75fd200640 1 -- 172.21.15.66:0/2230382660 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f806b670 con 0x7f75f806e800 2024-03-30T00:39:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.193+0000 7f75fd200640 1 --2- 172.21.15.66:0/2230382660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f806e800 0x7f75f80d1060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009870 tx=0x7f75e401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537c7b5ad51c958a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.193+0000 7f75f7e00640 1 -- 172.21.15.66:0/2230382660 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e40a9610 con 0x7f75f806e800 2024-03-30T00:39:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.193+0000 7f75f7e00640 1 -- 172.21.15.66:0/2230382660 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e40a9770 con 0x7f75f806e800 2024-03-30T00:39:58.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.193+0000 7f75f7e00640 1 -- 172.21.15.66:0/2230382660 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e40a9b00 con 0x7f75f806e800 2024-03-30T00:39:58.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.193+0000 7f7600600640 1 -- 172.21.15.66:0/2230382660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f806e800 msgr2=0x7f75f80d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.193+0000 7f7600600640 1 --2- 172.21.15.66:0/2230382660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f806e800 0x7f75f80d1060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009870 tx=0x7f75e401c530 comp rx=0 tx=0).stop 2024-03-30T00:39:58.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.193+0000 7f7600600640 1 -- 172.21.15.66:0/2230382660 shutdown_connections 2024-03-30T00:39:58.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.193+0000 7f7600600640 1 --2- 172.21.15.66:0/2230382660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f806e800 0x7f75f80d1060 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.193+0000 7f7600600640 1 -- 172.21.15.66:0/2230382660 >> 172.21.15.66:0/2230382660 conn(0x7f75f80d0540 msgr2=0x7f75f80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:58.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.193+0000 7f7600600640 1 -- 172.21.15.66:0/2230382660 shutdown_connections 2024-03-30T00:39:58.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.193+0000 7f7600600640 1 -- 172.21.15.66:0/2230382660 wait complete. 2024-03-30T00:39:58.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f7600600640 1 Processor -- start 2024-03-30T00:39:58.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f7600600640 1 -- start start 2024-03-30T00:39:58.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f7600600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75f806e800 0x7f75f81519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f7600600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75f806f8e0 0x7f75f80c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f7600600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80cc560 0x7f75f80b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f7600600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f8151f20 con 0x7f75f80cc560 2024-03-30T00:39:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f7600600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80c4730 con 0x7f75f806e800 2024-03-30T00:39:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f7600600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80c9010 con 0x7f75f806f8e0 2024-03-30T00:39:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fe600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80cc560 0x7f75f80b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fe600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80cc560 0x7f75f80b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51512/0 (socket says 172.21.15.66:51512) 2024-03-30T00:39:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fe600640 1 -- 172.21.15.66:0/1919599815 learned_addr learned my addr 172.21.15.66:0/1919599815 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:39:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fd200640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75f806f8e0 0x7f75f80c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fdc00640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75f806e800 0x7f75f81519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fd200640 1 -- 172.21.15.66:0/1919599815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75f806e800 msgr2=0x7f75f81519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fd200640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75f806e800 0x7f75f81519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fd200640 1 -- 172.21.15.66:0/1919599815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80cc560 msgr2=0x7f75f80b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fd200640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80cc560 0x7f75f80b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fd200640 1 -- 172.21.15.66:0/1919599815 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e40094d0 con 0x7f75f806f8e0 2024-03-30T00:39:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fe600640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80cc560 0x7f75f80b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:39:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75fd200640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75f806f8e0 0x7f75f80c9360 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f75e40aa000 tx=0x7f75e401ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75f5600640 1 -- 172.21.15.66:0/1919599815 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e40b16b0 con 0x7f75f806f8e0 2024-03-30T00:39:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75f5600640 1 -- 172.21.15.66:0/1919599815 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e40b1830 con 0x7f75f806f8e0 2024-03-30T00:39:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f75f5600640 1 -- 172.21.15.66:0/1919599815 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e40b1be0 con 0x7f75f806f8e0 2024-03-30T00:39:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f7600600640 1 -- 172.21.15.66:0/1919599815 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f80c8400 con 0x7f75f806f8e0 2024-03-30T00:39:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.197+0000 7f7600600640 1 -- 172.21.15.66:0/1919599815 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f80c89c0 con 0x7f75f806f8e0 2024-03-30T00:39:58.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.201+0000 7f7600600640 1 -- 172.21.15.66:0/1919599815 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f80c48a0 con 0x7f75f806f8e0 2024-03-30T00:39:58.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.201+0000 7f75f5600640 1 -- 172.21.15.66:0/1919599815 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f75e40b1d40 con 0x7f75f806f8e0 2024-03-30T00:39:58.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.201+0000 7f75f5600640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f75d8066690 0x7f75d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:39:58.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.201+0000 7f75f5600640 1 -- 172.21.15.66:0/1919599815 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f75e411e820 con 0x7f75f806f8e0 2024-03-30T00:39:58.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.201+0000 7f75fdc00640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f75d8066690 0x7f75d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:39:58.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.201+0000 7f75fdc00640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f75d8066690 0x7f75d8068b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f75f806c260 tx=0x7f75ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:39:58.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.205+0000 7f75f5600640 1 -- 172.21.15.66:0/1919599815 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f75f80c48a0 con 0x7f75f806f8e0 2024-03-30T00:39:58.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.401+0000 7f7600600640 1 -- 172.21.15.66:0/1919599815 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f75f80c48a0 con 0x7f75f806f8e0 2024-03-30T00:39:58.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.401+0000 7f75f5600640 1 -- 172.21.15.66:0/1919599815 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f75f80c48a0 con 0x7f75f806f8e0 2024-03-30T00:39:58.412 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T00:39:58.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 -- 172.21.15.66:0/1919599815 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f75d8066690 msgr2=0x7f75d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f75d8066690 0x7f75d8068b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f75f806c260 tx=0x7f75ec009040 comp rx=0 tx=0).stop 2024-03-30T00:39:58.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 -- 172.21.15.66:0/1919599815 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75f806f8e0 msgr2=0x7f75f80c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:39:58.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75f806f8e0 0x7f75f80c9360 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f75e40aa000 tx=0x7f75e401ad70 comp rx=0 tx=0).stop 2024-03-30T00:39:58.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 -- 172.21.15.66:0/1919599815 shutdown_connections 2024-03-30T00:39:58.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f75d8066690 0x7f75d8068b50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80cc560 0x7f75f80b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75f806f8e0 0x7f75f80c9360 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 --2- 172.21.15.66:0/1919599815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75f806e800 0x7f75f81519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:39:58.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 -- 172.21.15.66:0/1919599815 >> 172.21.15.66:0/1919599815 conn(0x7f75f80d0540 msgr2=0x7f75f806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:39:58.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 -- 172.21.15.66:0/1919599815 shutdown_connections 2024-03-30T00:39:58.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:39:58.405+0000 7f7600600640 1 -- 172.21.15.66:0/1919599815 wait complete. 2024-03-30T00:39:58.432 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-30T00:39:58.432 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:39:58.433 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:39:58.433 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-30T00:39:58.433 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-30T00:39:58.433 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-30T00:39:58.433 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:39:58.433 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:39:58.433 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-30T00:39:58.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:58.433 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:39:58.442 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:39:58.442 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:39:58.442 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:39:58.442 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:39:58.442 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:39:58.442 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:39:58.442 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:39:58.442 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:39:58.442 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:39:58.442 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42480sec preferred_lft 42480sec 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:39:58.443 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:39:58.444 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T00:39:58.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:58.445 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:39:58.445 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:39:58.445 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2024-03-30T00:39:58.445 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2024-03-30T00:39:58.445 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2024-03-30T00:39:58.445 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T00:39:58.445 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:39:58.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:58.556 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:39:58.574 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:39:58.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:58.576 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:39:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:39:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:39:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:39:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:39:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:39:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:39:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:39:58.677 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:39:58.677 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:39:58.677 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:39:58.678 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:39:58.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:58.678 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:39:58.678 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:39:58.678 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:39:58.678 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:39:58.678 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:39:58.678 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:39:58.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:58.728 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:39:58.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:58.738 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-03-30T00:39:58.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:58.789 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:39:58.789 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:39:58.790 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T00:39:58.790 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-30T00:39:58.790 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:39:58.871 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-30T00:39:58.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:58.872 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:39:58.872 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:39:58.872 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-30T00:39:58.872 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T00:39:58.872 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-30T00:39:58.872 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-30T00:39:58.872 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-30T00:39:58.872 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:39:59.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:59.034 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:39:59.035 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:39:59.035 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:39:59.035 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T00:39:59.035 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:39:59.075 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-30T00:39:59.075 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:39:59.075 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-30T00:39:59.125 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-30T00:39:59.125 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:39:59.125 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-30T00:39:59.176 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:39:59.241 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:39:59.241 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:39:59.302 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:39:59.304 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:39:59.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:59.304 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:39:59.358 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:39:59.358 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-30T00:39:59.407 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:39:59.427 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:39:59.428 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:39:59.486 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:39:59.488 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:39:59.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:39:59.489 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:39:59.506 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:2024-03-30T00:39:59.497+0000 7f757ef64580 -1 init, newargv = 0x564529ef0aa0 newargc=15 2024-03-30T00:39:59.506 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[33740]: starting ceph client 2024-03-30T00:39:59.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[33740]: starting fuse 2024-03-30T00:40:00.616 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:40:00.641 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:40:00.641 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:40:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:40:00.709 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:40:00.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:00.710 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:40:00.761 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:40:00.761 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:40:00.761 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:40:00.761 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:40:00.761 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:40:00.761 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:40:00.761 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:40:00.761 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:00.761 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:40:00.761 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-30T00:40:00.761 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:40:00.762 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:40:00.763 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:40:00.763 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:00.763 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.admin")) 2024-03-30T00:40:00.763 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:40:00.892 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.admin.33740.asok 2024-03-30T00:40:01.076 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.33740.asok 2024-03-30T00:40:01.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:01.076 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.33740.asok status 2024-03-30T00:40:01.257 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:40:01.258 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:40:01.258 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:40:01.258 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:40:01.258 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "admin", 2024-03-30T00:40:01.258 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:40:01.258 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-30T00:40:01.258 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "33740", 2024-03-30T00:40:01.258 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:40:01.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:01.258 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:40:01.258 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4443, 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 4443 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3994719049 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:40:01.259 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:40:01.260 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3994719049 2024-03-30T00:40:01.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:01.260 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.4443 192.168.0.1:0/3994719049", 2024-03-30T00:40:01.260 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/3994719049", 2024-03-30T00:40:01.260 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:40:01.260 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 7, 2024-03-30T00:40:01.260 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 25, 2024-03-30T00:40:01.260 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:40:01.260 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:40:01.260 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:40:01.260 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:40:01.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:01.273 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:40:01.273 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-30T00:40:01.283 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:40:01.283 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-30T00:40:01.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:01.283 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-30T00:40:01.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:01.344 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-30T00:40:01.411 INFO:teuthology.orchestra.run.smithi066.stdout:# file: . 2024-03-30T00:40:01.411 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-30T00:40:01.411 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:01.412 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:40:01.412 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:40:01.464 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:01.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.637+0000 7f6152600640 1 Processor -- start 2024-03-30T00:40:01.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.637+0000 7f6152600640 1 -- start start 2024-03-30T00:40:01.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.641+0000 7f6152600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f614c0cdcc0 0x7f614c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:01.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.641+0000 7f6152600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614c0accb0 con 0x7f614c15dc40 2024-03-30T00:40:01.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.641+0000 7f6152600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614c0acf00 con 0x7f614c15e020 2024-03-30T00:40:01.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.641+0000 7f6152600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614c06f190 con 0x7f614c0cdcc0 2024-03-30T00:40:01.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.641+0000 7f6143e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f614c0cdcc0 0x7f614c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:01.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.641+0000 7f6143e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f614c0cdcc0 0x7f614c06e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52894/0 (socket says 172.21.15.66:52894) 2024-03-30T00:40:01.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.641+0000 7f6143e00640 1 -- 172.21.15.66:0/2526242030 learned_addr learned my addr 172.21.15.66:0/2526242030 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:01.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.641+0000 7f6143e00640 1 -- 172.21.15.66:0/2526242030 >> v1:172.21.15.156:6789/0 conn(0x7f614c15e020 legacy=0x7f614c0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:01.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.641+0000 7f6143e00640 1 -- 172.21.15.66:0/2526242030 >> v1:172.21.15.66:6789/0 conn(0x7f614c15dc40 legacy=0x7f614c0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:01.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.641+0000 7f6143e00640 1 -- 172.21.15.66:0/2526242030 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c0ad1e0 con 0x7f614c0cdcc0 2024-03-30T00:40:01.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.641+0000 7f614aa00640 1 -- 172.21.15.66:0/2526242030 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395223067 0 0) 0x7f614c0accb0 con 0x7f614c15dc40 2024-03-30T00:40:01.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f6143e00640 1 --2- 172.21.15.66:0/2526242030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f614c0cdcc0 0x7f614c06e290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6134009870 tx=0x7f613401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a008b00334971b36 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:01.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f614aa00640 1 -- 172.21.15.66:0/2526242030 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61340a6040 con 0x7f614c0cdcc0 2024-03-30T00:40:01.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f614aa00640 1 -- 172.21.15.66:0/2526242030 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61340aa600 con 0x7f614c0cdcc0 2024-03-30T00:40:01.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f614aa00640 1 -- 172.21.15.66:0/2526242030 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61340aa990 con 0x7f614c0cdcc0 2024-03-30T00:40:01.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f6152600640 1 -- 172.21.15.66:0/2526242030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f614c0cdcc0 msgr2=0x7f614c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:01.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f6152600640 1 --2- 172.21.15.66:0/2526242030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f614c0cdcc0 0x7f614c06e290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6134009870 tx=0x7f613401c4a0 comp rx=0 tx=0).stop 2024-03-30T00:40:01.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f6152600640 1 -- 172.21.15.66:0/2526242030 shutdown_connections 2024-03-30T00:40:01.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f6152600640 1 --2- 172.21.15.66:0/2526242030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f614c0cdcc0 0x7f614c06e290 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f6152600640 1 -- 172.21.15.66:0/2526242030 >> 172.21.15.66:0/2526242030 conn(0x7f614c098a20 msgr2=0x7f614c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f6152600640 1 -- 172.21.15.66:0/2526242030 shutdown_connections 2024-03-30T00:40:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f6152600640 1 -- 172.21.15.66:0/2526242030 wait complete. 2024-03-30T00:40:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f6152600640 1 Processor -- start 2024-03-30T00:40:01.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.645+0000 7f6152600640 1 -- start start 2024-03-30T00:40:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6152600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f614c0cdcc0 0x7f614c0b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6152600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f614c15dc40 0x7f614c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6152600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f614c15e020 0x7f614c153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6152600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0b52f0 con 0x7f614c15dc40 2024-03-30T00:40:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6152600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0b3d10 con 0x7f614c15e020 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6152600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0b3ff0 con 0x7f614c0cdcc0 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f614c15dc40 0x7f614c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f614c15dc40 0x7f614c0b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52910/0 (socket says 172.21.15.66:52910) 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143e00640 1 -- 172.21.15.66:0/1619135737 learned_addr learned my addr 172.21.15.66:0/1619135737 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f614b400640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f614c0cdcc0 0x7f614c0b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f614be00640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f614c15e020 0x7f614c153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143e00640 1 -- 172.21.15.66:0/1619135737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f614c0cdcc0 msgr2=0x7f614c0b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143e00640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f614c0cdcc0 0x7f614c0b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143e00640 1 -- 172.21.15.66:0/1619135737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f614c15e020 msgr2=0x7f614c153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143e00640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f614c15e020 0x7f614c153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143e00640 1 -- 172.21.15.66:0/1619135737 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61340094d0 con 0x7f614c15dc40 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143e00640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f614c15dc40 0x7f614c0b4db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f61340190b0 tx=0x7f613401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143400640 1 -- 172.21.15.66:0/1619135737 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61340b3800 con 0x7f614c15dc40 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143400640 1 -- 172.21.15.66:0/1619135737 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61340b3960 con 0x7f614c15dc40 2024-03-30T00:40:01.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143400640 1 -- 172.21.15.66:0/1619135737 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61340b3cd0 con 0x7f614c15dc40 2024-03-30T00:40:01.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6152600640 1 -- 172.21.15.66:0/1619135737 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614c0b42d0 con 0x7f614c15dc40 2024-03-30T00:40:01.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6152600640 1 -- 172.21.15.66:0/1619135737 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614c0b4890 con 0x7f614c15dc40 2024-03-30T00:40:01.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6152600640 1 -- 172.21.15.66:0/1619135737 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614c0b4ad0 con 0x7f614c15dc40 2024-03-30T00:40:01.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.649+0000 7f6143400640 1 -- 172.21.15.66:0/1619135737 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f61340b2380 con 0x7f614c15dc40 2024-03-30T00:40:01.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.653+0000 7f6143400640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6128066690 0x7f6128068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:01.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.653+0000 7f6143400640 1 -- 172.21.15.66:0/1619135737 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f613411e810 con 0x7f614c15dc40 2024-03-30T00:40:01.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.653+0000 7f614b400640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6128066690 0x7f6128068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:01.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.653+0000 7f614b400640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6128066690 0x7f6128068b50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f613c000f70 tx=0x7f613c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:01.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.657+0000 7f6143400640 1 -- 172.21.15.66:0/1619135737 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f614c0b4ad0 con 0x7f614c15dc40 2024-03-30T00:40:01.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.821+0000 7f6152600640 1 -- 172.21.15.66:0/1619135737 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f614c0b4ad0 con 0x7f614c15dc40 2024-03-30T00:40:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.821+0000 7f6143400640 1 -- 172.21.15.66:0/1619135737 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f614c0b4ad0 con 0x7f614c15dc40 2024-03-30T00:40:01.831 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:01.831 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 -- 172.21.15.66:0/1619135737 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6128066690 msgr2=0x7f6128068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6128066690 0x7f6128068b50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f613c000f70 tx=0x7f613c009040 comp rx=0 tx=0).stop 2024-03-30T00:40:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 -- 172.21.15.66:0/1619135737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f614c15dc40 msgr2=0x7f614c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f614c15dc40 0x7f614c0b4db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f61340190b0 tx=0x7f613401ace0 comp rx=0 tx=0).stop 2024-03-30T00:40:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 -- 172.21.15.66:0/1619135737 shutdown_connections 2024-03-30T00:40:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f614c15e020 0x7f614c153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6128066690 0x7f6128068b50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f614c15dc40 0x7f614c0b4db0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 --2- 172.21.15.66:0/1619135737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f614c0cdcc0 0x7f614c0b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 -- 172.21.15.66:0/1619135737 >> 172.21.15.66:0/1619135737 conn(0x7f614c098a20 msgr2=0x7f614c0b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 -- 172.21.15.66:0/1619135737 shutdown_connections 2024-03-30T00:40:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.825+0000 7f6152600640 1 -- 172.21.15.66:0/1619135737 wait complete. 2024-03-30T00:40:01.852 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:01.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:01.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.985+0000 7fb968800640 1 Processor -- start 2024-03-30T00:40:01.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.985+0000 7fb968800640 1 -- start start 2024-03-30T00:40:01.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.985+0000 7fb968800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb9600cc580 0x7fb960167630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:01.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.985+0000 7fb968800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9600cb0e0 con 0x7fb96015ca80 2024-03-30T00:40:01.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.985+0000 7fb968800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96006bed0 con 0x7fb9600cf2f0 2024-03-30T00:40:01.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.985+0000 7fb968800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9600cf7f0 con 0x7fb9600cc580 2024-03-30T00:40:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.985+0000 7fb966800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb9600cc580 0x7fb960167630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.985+0000 7fb966800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb9600cc580 0x7fb960167630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52920/0 (socket says 172.21.15.66:52920) 2024-03-30T00:40:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.985+0000 7fb966800640 1 -- 172.21.15.66:0/1333571719 learned_addr learned my addr 172.21.15.66:0/1333571719 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.985+0000 7fb965e00640 1 --1- 172.21.15.66:0/1333571719 >> v1:172.21.15.66:6789/0 conn(0x7fb96015ca80 0x7fb96006b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53236/0 (socket says 172.21.15.66:53236) 2024-03-30T00:40:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb966800640 1 -- 172.21.15.66:0/1333571719 >> v1:172.21.15.156:6789/0 conn(0x7fb9600cf2f0 legacy=0x7fb96006b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb966800640 1 -- 172.21.15.66:0/1333571719 >> v1:172.21.15.66:6789/0 conn(0x7fb96015ca80 legacy=0x7fb96006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:01.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb966800640 1 -- 172.21.15.66:0/1333571719 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9600cfad0 con 0x7fb9600cc580 2024-03-30T00:40:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb966800640 1 --2- 172.21.15.66:0/1333571719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9600cc580 0x7fb960167630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009e50 tx=0x7fb95c01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747af3861629b675 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb964a00640 1 -- 172.21.15.66:0/1333571719 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0af400 con 0x7fb9600cc580 2024-03-30T00:40:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb964a00640 1 -- 172.21.15.66:0/1333571719 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c0ab050 con 0x7fb9600cc580 2024-03-30T00:40:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb964a00640 1 -- 172.21.15.66:0/1333571719 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0af780 con 0x7fb9600cc580 2024-03-30T00:40:02.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb968800640 1 -- 172.21.15.66:0/1333571719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9600cc580 msgr2=0x7fb960167630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb968800640 1 --2- 172.21.15.66:0/1333571719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9600cc580 0x7fb960167630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009e50 tx=0x7fb95c01f210 comp rx=0 tx=0).stop 2024-03-30T00:40:02.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb968800640 1 -- 172.21.15.66:0/1333571719 shutdown_connections 2024-03-30T00:40:02.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb968800640 1 --2- 172.21.15.66:0/1333571719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9600cc580 0x7fb960167630 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.989+0000 7fb968800640 1 -- 172.21.15.66:0/1333571719 >> 172.21.15.66:0/1333571719 conn(0x7fb9600b2090 msgr2=0x7fb9600b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:02.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 -- 172.21.15.66:0/1333571719 shutdown_connections 2024-03-30T00:40:02.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 -- 172.21.15.66:0/1333571719 wait complete. 2024-03-30T00:40:02.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 Processor -- start 2024-03-30T00:40:02.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 -- start start 2024-03-30T00:40:02.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb9600cc580 0x7fb9601f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9600cf2f0 0x7fb960151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb96015ca80 0x7fb9600c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600b4f80 con 0x7fb96015ca80 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb960152080 con 0x7fb9600cf2f0 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600b4020 con 0x7fb9600cc580 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb966800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb96015ca80 0x7fb9600c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb966800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb96015ca80 0x7fb9600c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52922/0 (socket says 172.21.15.66:52922) 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb966800640 1 -- 172.21.15.66:0/3922818709 learned_addr learned my addr 172.21.15.66:0/3922818709 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb965e00640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb9600cc580 0x7fb9601f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb965400640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9600cf2f0 0x7fb960151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb966800640 1 -- 172.21.15.66:0/3922818709 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb9600cc580 msgr2=0x7fb9601f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb966800640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb9600cc580 0x7fb9601f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb966800640 1 -- 172.21.15.66:0/3922818709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9600cf2f0 msgr2=0x7fb960151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb966800640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9600cf2f0 0x7fb960151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb966800640 1 -- 172.21.15.66:0/3922818709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c009b30 con 0x7fb96015ca80 2024-03-30T00:40:02.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb965e00640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb9600cc580 0x7fb9601f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:02.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb965400640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9600cf2f0 0x7fb960151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:02.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb966800640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb96015ca80 0x7fb9600c9f90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb95c01be20 tx=0x7fb95c01bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:02.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb95a000640 1 -- 172.21.15.66:0/3922818709 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0afde0 con 0x7fb96015ca80 2024-03-30T00:40:02.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb95a000640 1 -- 172.21.15.66:0/3922818709 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c009510 con 0x7fb96015ca80 2024-03-30T00:40:02.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 -- 172.21.15.66:0/3922818709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9600b4300 con 0x7fb96015ca80 2024-03-30T00:40:02.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb95a000640 1 -- 172.21.15.66:0/3922818709 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0b8640 con 0x7fb96015ca80 2024-03-30T00:40:02.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.993+0000 7fb968800640 1 -- 172.21.15.66:0/3922818709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9600b48c0 con 0x7fb96015ca80 2024-03-30T00:40:02.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.997+0000 7fb968800640 1 -- 172.21.15.66:0/3922818709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9600b50c0 con 0x7fb96015ca80 2024-03-30T00:40:02.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.997+0000 7fb95a000640 1 -- 172.21.15.66:0/3922818709 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb95c0b6040 con 0x7fb96015ca80 2024-03-30T00:40:02.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.997+0000 7fb95a000640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9440666c0 0x7fb944068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.997+0000 7fb95a000640 1 -- 172.21.15.66:0/3922818709 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb95c122430 con 0x7fb96015ca80 2024-03-30T00:40:02.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.997+0000 7fb965e00640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9440666c0 0x7fb944068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:01.997+0000 7fb965e00640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9440666c0 0x7fb944068b80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb954000f70 tx=0x7fb954007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:02.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.001+0000 7fb95a000640 1 -- 172.21.15.66:0/3922818709 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb9600b50c0 con 0x7fb96015ca80 2024-03-30T00:40:02.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.161+0000 7fb968800640 1 -- 172.21.15.66:0/3922818709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9600b50c0 con 0x7fb96015ca80 2024-03-30T00:40:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.161+0000 7fb95a000640 1 -- 172.21.15.66:0/3922818709 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9600b50c0 con 0x7fb96015ca80 2024-03-30T00:40:02.172 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 -- 172.21.15.66:0/3922818709 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9440666c0 msgr2=0x7fb944068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9440666c0 0x7fb944068b80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb954000f70 tx=0x7fb954007d50 comp rx=0 tx=0).stop 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 -- 172.21.15.66:0/3922818709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb96015ca80 msgr2=0x7fb9600c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb96015ca80 0x7fb9600c9f90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb95c01be20 tx=0x7fb95c01bfb0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 -- 172.21.15.66:0/3922818709 shutdown_connections 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9440666c0 0x7fb944068b80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb96015ca80 0x7fb9600c9f90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9600cf2f0 0x7fb960151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 --2- 172.21.15.66:0/3922818709 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb9600cc580 0x7fb9601f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 -- 172.21.15.66:0/3922818709 >> 172.21.15.66:0/3922818709 conn(0x7fb9600b2090 msgr2=0x7fb96015bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 -- 172.21.15.66:0/3922818709 shutdown_connections 2024-03-30T00:40:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.165+0000 7fb968800640 1 -- 172.21.15.66:0/3922818709 wait complete. 2024-03-30T00:40:02.194 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd114c00640 1 Processor -- start 2024-03-30T00:40:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd114c00640 1 -- start start 2024-03-30T00:40:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd114c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1100b19a0 0x7fd1100c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd114c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1100c1800 0x7fd1100cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd114c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1100c97f0 con 0x7fd110160c10 2024-03-30T00:40:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd114c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1100c1c90 con 0x7fd1100c1800 2024-03-30T00:40:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd114c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1100c9a30 con 0x7fd1100b19a0 2024-03-30T00:40:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd10e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1100c1800 0x7fd1100cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd10e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1100c1800 0x7fd1100cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52928/0 (socket says 172.21.15.66:52928) 2024-03-30T00:40:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd10e800640 1 -- 172.21.15.66:0/4205590944 learned_addr learned my addr 172.21.15.66:0/4205590944 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd10d400640 1 --2- 172.21.15.66:0/4205590944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1100b19a0 0x7fd1100c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd10e800640 1 -- 172.21.15.66:0/4205590944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1100b19a0 msgr2=0x7fd1100c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd10e800640 1 --2- 172.21.15.66:0/4205590944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1100b19a0 0x7fd1100c12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd10e800640 1 -- 172.21.15.66:0/4205590944 >> v1:172.21.15.156:6789/0 conn(0x7fd110160c10 legacy=0x7fd1100c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.329+0000 7fd10e800640 1 -- 172.21.15.66:0/4205590944 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1100c9d10 con 0x7fd1100c1800 2024-03-30T00:40:02.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd10e800640 1 --2- 172.21.15.66:0/4205590944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1100c1800 0x7fd1100cc280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd108009e50 tx=0x7fd10801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f032deef76ce523 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:02.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd10ca00640 1 -- 172.21.15.66:0/4205590944 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1080a58d0 con 0x7fd1100c1800 2024-03-30T00:40:02.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd10ca00640 1 -- 172.21.15.66:0/4205590944 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1080a5a30 con 0x7fd1100c1800 2024-03-30T00:40:02.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd10ca00640 1 -- 172.21.15.66:0/4205590944 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1080a5dc0 con 0x7fd1100c1800 2024-03-30T00:40:02.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd114c00640 1 -- 172.21.15.66:0/4205590944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1100c1800 msgr2=0x7fd1100cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd114c00640 1 --2- 172.21.15.66:0/4205590944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1100c1800 0x7fd1100cc280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd108009e50 tx=0x7fd10801ca00 comp rx=0 tx=0).stop 2024-03-30T00:40:02.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd114c00640 1 -- 172.21.15.66:0/4205590944 shutdown_connections 2024-03-30T00:40:02.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd114c00640 1 --2- 172.21.15.66:0/4205590944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1100c1800 0x7fd1100cc280 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd114c00640 1 --2- 172.21.15.66:0/4205590944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1100b19a0 0x7fd1100c12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd114c00640 1 -- 172.21.15.66:0/4205590944 >> 172.21.15.66:0/4205590944 conn(0x7fd110069f60 msgr2=0x7fd11006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:02.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd114c00640 1 -- 172.21.15.66:0/4205590944 shutdown_connections 2024-03-30T00:40:02.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd114c00640 1 -- 172.21.15.66:0/4205590944 wait complete. 2024-03-30T00:40:02.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd114c00640 1 Processor -- start 2024-03-30T00:40:02.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.333+0000 7fd114c00640 1 -- start start 2024-03-30T00:40:02.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd114c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1100b19a0 0x7fd1100979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd114c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1100c1800 0x7fd1100af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd114c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd110160c10 0x7fd11015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd114c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11006dc20 con 0x7fd1100c1800 2024-03-30T00:40:02.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd114c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd110097f10 con 0x7fd110160c10 2024-03-30T00:40:02.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd114c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11006cfa0 con 0x7fd1100b19a0 2024-03-30T00:40:02.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd10d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1100c1800 0x7fd1100af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd10de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1100b19a0 0x7fd1100979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd10d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1100c1800 0x7fd1100af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52936/0 (socket says 172.21.15.66:52936) 2024-03-30T00:40:02.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd10d400640 1 -- 172.21.15.66:0/2613039573 learned_addr learned my addr 172.21.15.66:0/2613039573 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:02.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd10e800640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd110160c10 0x7fd11015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd10d400640 1 -- 172.21.15.66:0/2613039573 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1100b19a0 msgr2=0x7fd1100979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd10d400640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1100b19a0 0x7fd1100979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd10d400640 1 -- 172.21.15.66:0/2613039573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd110160c10 msgr2=0x7fd11015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd10d400640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd110160c10 0x7fd11015f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd10d400640 1 -- 172.21.15.66:0/2613039573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd108009b30 con 0x7fd1100c1800 2024-03-30T00:40:02.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd10d400640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1100c1800 0x7fd1100af5c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00e930 tx=0x7fd0fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd106000640 1 -- 172.21.15.66:0/2613039573 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc009690 con 0x7fd1100c1800 2024-03-30T00:40:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd106000640 1 -- 172.21.15.66:0/2613039573 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0fc00ca00 con 0x7fd1100c1800 2024-03-30T00:40:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd106000640 1 -- 172.21.15.66:0/2613039573 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc00cd70 con 0x7fd1100c1800 2024-03-30T00:40:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd114c00640 1 -- 172.21.15.66:0/2613039573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11006d280 con 0x7fd1100c1800 2024-03-30T00:40:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd114c00640 1 -- 172.21.15.66:0/2613039573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11006d840 con 0x7fd1100c1800 2024-03-30T00:40:02.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd114c00640 1 -- 172.21.15.66:0/2613039573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11006dd60 con 0x7fd1100c1800 2024-03-30T00:40:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.337+0000 7fd106000640 1 -- 172.21.15.66:0/2613039573 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd0fc013470 con 0x7fd1100c1800 2024-03-30T00:40:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.341+0000 7fd106000640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd0f0066690 0x7fd0f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.341+0000 7fd106000640 1 -- 172.21.15.66:0/2613039573 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd0fc084760 con 0x7fd1100c1800 2024-03-30T00:40:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.341+0000 7fd10de00640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd0f0066690 0x7fd0f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.341+0000 7fd10de00640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd0f0066690 0x7fd0f0068b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd100007fd0 tx=0x7fd100009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:02.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.345+0000 7fd106000640 1 -- 172.21.15.66:0/2613039573 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd11006dd60 con 0x7fd1100c1800 2024-03-30T00:40:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 -- 172.21.15.66:0/2613039573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd11006dd60 con 0x7fd1100c1800 2024-03-30T00:40:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd106000640 1 -- 172.21.15.66:0/2613039573 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd11006dd60 con 0x7fd1100c1800 2024-03-30T00:40:02.518 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 -- 172.21.15.66:0/2613039573 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd0f0066690 msgr2=0x7fd0f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd0f0066690 0x7fd0f0068b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd100007fd0 tx=0x7fd100009040 comp rx=0 tx=0).stop 2024-03-30T00:40:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 -- 172.21.15.66:0/2613039573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1100c1800 msgr2=0x7fd1100af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1100c1800 0x7fd1100af5c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00e930 tx=0x7fd0fc00edf0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 -- 172.21.15.66:0/2613039573 shutdown_connections 2024-03-30T00:40:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd110160c10 0x7fd11015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd0f0066690 0x7fd0f0068b50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1100c1800 0x7fd1100af5c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 --2- 172.21.15.66:0/2613039573 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1100b19a0 0x7fd1100979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 -- 172.21.15.66:0/2613039573 >> 172.21.15.66:0/2613039573 conn(0x7fd110069f60 msgr2=0x7fd1100ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 -- 172.21.15.66:0/2613039573 shutdown_connections 2024-03-30T00:40:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.509+0000 7fd114c00640 1 -- 172.21.15.66:0/2613039573 wait complete. 2024-03-30T00:40:02.539 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:02.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae2e800640 1 Processor -- start 2024-03-30T00:40:02.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae2e800640 1 -- start start 2024-03-30T00:40:02.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae2e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae28098c80 0x7fae28099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae2e800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae280cf320 0x7fae280ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae2e800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae280cbd90 con 0x7fae28099ca0 2024-03-30T00:40:02.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae2e800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae280cc070 con 0x7fae28098c80 2024-03-30T00:40:02.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae2e800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae280cc350 con 0x7fae280cf320 2024-03-30T00:40:02.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae27400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae28098c80 0x7fae28099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae27400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae28098c80 0x7fae28099760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52942/0 (socket says 172.21.15.66:52942) 2024-03-30T00:40:02.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae27400640 1 -- 172.21.15.66:0/4074346380 learned_addr learned my addr 172.21.15.66:0/4074346380 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:02.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae1ea00640 1 --2- 172.21.15.66:0/4074346380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae280cf320 0x7fae280ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae27400640 1 -- 172.21.15.66:0/4074346380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae280cf320 msgr2=0x7fae280ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:02.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae27400640 1 --2- 172.21.15.66:0/4074346380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae280cf320 0x7fae280ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae27400640 1 -- 172.21.15.66:0/4074346380 >> v1:172.21.15.156:6789/0 conn(0x7fae28099ca0 legacy=0x7fae280ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.673+0000 7fae27400640 1 -- 172.21.15.66:0/4074346380 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae280b3530 con 0x7fae28098c80 2024-03-30T00:40:02.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.677+0000 7fae27400640 1 --2- 172.21.15.66:0/4074346380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae28098c80 0x7fae28099760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fae18009870 tx=0x7fae1801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74a06b15cd58d02 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:02.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.677+0000 7fae26a00640 1 -- 172.21.15.66:0/4074346380 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180a8a50 con 0x7fae28098c80 2024-03-30T00:40:02.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.677+0000 7fae26a00640 1 -- 172.21.15.66:0/4074346380 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae180a8bb0 con 0x7fae28098c80 2024-03-30T00:40:02.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.677+0000 7fae26a00640 1 -- 172.21.15.66:0/4074346380 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180b0440 con 0x7fae28098c80 2024-03-30T00:40:02.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.677+0000 7fae2e800640 1 -- 172.21.15.66:0/4074346380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae28098c80 msgr2=0x7fae28099760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.677+0000 7fae2e800640 1 --2- 172.21.15.66:0/4074346380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae28098c80 0x7fae28099760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fae18009870 tx=0x7fae1801ed50 comp rx=0 tx=0).stop 2024-03-30T00:40:02.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.681+0000 7fae2e800640 1 -- 172.21.15.66:0/4074346380 shutdown_connections 2024-03-30T00:40:02.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.681+0000 7fae2e800640 1 --2- 172.21.15.66:0/4074346380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae280cf320 0x7fae280ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.681+0000 7fae2e800640 1 --2- 172.21.15.66:0/4074346380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae28098c80 0x7fae28099760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.681+0000 7fae2e800640 1 -- 172.21.15.66:0/4074346380 >> 172.21.15.66:0/4074346380 conn(0x7fae280b2130 msgr2=0x7fae280b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:02.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.681+0000 7fae2e800640 1 -- 172.21.15.66:0/4074346380 shutdown_connections 2024-03-30T00:40:02.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.681+0000 7fae2e800640 1 -- 172.21.15.66:0/4074346380 wait complete. 2024-03-30T00:40:02.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.681+0000 7fae2e800640 1 Processor -- start 2024-03-30T00:40:02.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.681+0000 7fae2e800640 1 -- start start 2024-03-30T00:40:02.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae2e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae28098c80 0x7fae280c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae2e800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae28099ca0 0x7fae280c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae2e800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae280cf320 0x7fae28159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae2e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c6760 con 0x7fae28098c80 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae2e800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c5960 con 0x7fae28099ca0 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae2e800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c9750 con 0x7fae280cf320 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae27400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae28098c80 0x7fae280c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae27400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae28098c80 0x7fae280c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52956/0 (socket says 172.21.15.66:52956) 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae27400640 1 -- 172.21.15.66:0/3469459062 learned_addr learned my addr 172.21.15.66:0/3469459062 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae1ea00640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae28099ca0 0x7fae280c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae27400640 1 -- 172.21.15.66:0/3469459062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae280cf320 msgr2=0x7fae28159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae27e00640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae280cf320 0x7fae28159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae27400640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae280cf320 0x7fae28159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae27400640 1 -- 172.21.15.66:0/3469459062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae28099ca0 msgr2=0x7fae280c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae27400640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae28099ca0 0x7fae280c5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae27400640 1 -- 172.21.15.66:0/3469459062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae180094d0 con 0x7fae28098c80 2024-03-30T00:40:02.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae1ea00640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae28099ca0 0x7fae280c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:02.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae27e00640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae280cf320 0x7fae28159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:02.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae27400640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae28098c80 0x7fae280c6220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae1801d400 tx=0x7fae1801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:02.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae1fe00640 1 -- 172.21.15.66:0/3469459062 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180ba600 con 0x7fae28098c80 2024-03-30T00:40:02.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae1fe00640 1 -- 172.21.15.66:0/3469459062 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae180ba780 con 0x7fae28098c80 2024-03-30T00:40:02.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae1fe00640 1 -- 172.21.15.66:0/3469459062 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180bab30 con 0x7fae28098c80 2024-03-30T00:40:02.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae2e800640 1 -- 172.21.15.66:0/3469459062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae280c8b40 con 0x7fae28098c80 2024-03-30T00:40:02.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae2e800640 1 -- 172.21.15.66:0/3469459062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae280c9100 con 0x7fae28098c80 2024-03-30T00:40:02.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae2e800640 1 -- 172.21.15.66:0/3469459062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae280677d0 con 0x7fae28098c80 2024-03-30T00:40:02.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.685+0000 7fae1fe00640 1 -- 172.21.15.66:0/3469459062 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fae180bac90 con 0x7fae28098c80 2024-03-30T00:40:02.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.689+0000 7fae1fe00640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae04066690 0x7fae04068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:02.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.689+0000 7fae1fe00640 1 -- 172.21.15.66:0/3469459062 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fae18122a00 con 0x7fae28098c80 2024-03-30T00:40:02.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.689+0000 7fae1ea00640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae04066690 0x7fae04068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:02.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.689+0000 7fae1ea00640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae04066690 0x7fae04068b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae280b1740 tx=0x7fae10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:02.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.693+0000 7fae1fe00640 1 -- 172.21.15.66:0/3469459062 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fae180efa80 con 0x7fae28098c80 2024-03-30T00:40:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 -- 172.21.15.66:0/3469459062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae28066c30 con 0x7fae28098c80 2024-03-30T00:40:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae1fe00640 1 -- 172.21.15.66:0/3469459062 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae180b5020 con 0x7fae28098c80 2024-03-30T00:40:02.862 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 -- 172.21.15.66:0/3469459062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae04066690 msgr2=0x7fae04068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae04066690 0x7fae04068b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae280b1740 tx=0x7fae10008040 comp rx=0 tx=0).stop 2024-03-30T00:40:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 -- 172.21.15.66:0/3469459062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae28098c80 msgr2=0x7fae280c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae28098c80 0x7fae280c6220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae1801d400 tx=0x7fae1801baf0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 -- 172.21.15.66:0/3469459062 shutdown_connections 2024-03-30T00:40:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae280cf320 0x7fae28159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae28099ca0 0x7fae280c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae04066690 0x7fae04068b50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 --2- 172.21.15.66:0/3469459062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae28098c80 0x7fae280c6220 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 -- 172.21.15.66:0/3469459062 >> 172.21.15.66:0/3469459062 conn(0x7fae280b2130 msgr2=0x7fae280cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 -- 172.21.15.66:0/3469459062 shutdown_connections 2024-03-30T00:40:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:02.853+0000 7fae2e800640 1 -- 172.21.15.66:0/3469459062 wait complete. 2024-03-30T00:40:02.884 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:03.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.017+0000 7f3667200640 1 Processor -- start 2024-03-30T00:40:03.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.017+0000 7f3667200640 1 -- start start 2024-03-30T00:40:03.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.017+0000 7f3667200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f366815c890 0x7f36680cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.017+0000 7f3667200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f36680d1060 0x7f366806b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.017+0000 7f3667200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36680cf0d0 con 0x7f36680ce190 2024-03-30T00:40:03.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.017+0000 7f3667200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36680cf390 con 0x7f36680d1060 2024-03-30T00:40:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.017+0000 7f3667200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36680cf670 con 0x7f366815c890 2024-03-30T00:40:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.017+0000 7f3666800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f36680d1060 0x7f366806b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.017+0000 7f365d400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f36680ce190 0x7f36680d0950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53244/0 (socket says 172.21.15.66:53244) 2024-03-30T00:40:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.017+0000 7f365d400640 1 -- 172.21.15.66:0/1651593955 learned_addr learned my addr 172.21.15.66:0/1651593955 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3665e00640 1 --2- 172.21.15.66:0/1651593955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f366815c890 0x7f36680cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3666800640 1 -- 172.21.15.66:0/1651593955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f366815c890 msgr2=0x7f36680cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3666800640 1 --2- 172.21.15.66:0/1651593955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f366815c890 0x7f36680cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3666800640 1 -- 172.21.15.66:0/1651593955 >> v1:172.21.15.66:6789/0 conn(0x7f36680ce190 legacy=0x7f36680d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3666800640 1 -- 172.21.15.66:0/1651593955 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36680cf950 con 0x7f36680d1060 2024-03-30T00:40:03.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3665400640 1 -- 172.21.15.66:0/1651593955 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126665836 0 0) 0x7f36680cf0d0 con 0x7f36680ce190 2024-03-30T00:40:03.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3665e00640 1 --2- 172.21.15.66:0/1651593955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f366815c890 0x7f36680cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3666800640 1 --2- 172.21.15.66:0/1651593955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36680d1060 0x7f366806b840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f36600097c0 tx=0x7f366001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa95227d9ffa858 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:03.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3665400640 1 -- 172.21.15.66:0/1651593955 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36600ab600 con 0x7f36680d1060 2024-03-30T00:40:03.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3665400640 1 -- 172.21.15.66:0/1651593955 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36600ab760 con 0x7f36680d1060 2024-03-30T00:40:03.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3665400640 1 -- 172.21.15.66:0/1651593955 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36600abaf0 con 0x7f36680d1060 2024-03-30T00:40:03.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3667200640 1 -- 172.21.15.66:0/1651593955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36680d1060 msgr2=0x7f366806b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.021+0000 7f3667200640 1 --2- 172.21.15.66:0/1651593955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36680d1060 0x7f366806b840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f36600097c0 tx=0x7f366001c470 comp rx=0 tx=0).stop 2024-03-30T00:40:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 -- 172.21.15.66:0/1651593955 shutdown_connections 2024-03-30T00:40:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 --2- 172.21.15.66:0/1651593955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36680d1060 0x7f366806b840 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 --2- 172.21.15.66:0/1651593955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f366815c890 0x7f36680cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 -- 172.21.15.66:0/1651593955 >> 172.21.15.66:0/1651593955 conn(0x7f366809b8d0 msgr2=0x7f3668089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 -- 172.21.15.66:0/1651593955 shutdown_connections 2024-03-30T00:40:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 -- 172.21.15.66:0/1651593955 wait complete. 2024-03-30T00:40:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 Processor -- start 2024-03-30T00:40:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 -- start start 2024-03-30T00:40:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36680ce190 0x7f36680cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36680d1060 0x7f366814f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f366815c890 0x7f366815bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680c7be0 con 0x7f36680ce190 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680c7e30 con 0x7f366815c890 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36680c8110 con 0x7f36680d1060 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3665e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36680ce190 0x7f36680cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3665e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36680ce190 0x7f36680cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52972/0 (socket says 172.21.15.66:52972) 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3665e00640 1 -- 172.21.15.66:0/2654994693 learned_addr learned my addr 172.21.15.66:0/2654994693 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3665e00640 1 -- 172.21.15.66:0/2654994693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36680d1060 msgr2=0x7f366814f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3665e00640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36680d1060 0x7f366814f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3665e00640 1 -- 172.21.15.66:0/2654994693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f366815c890 msgr2=0x7f366815bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3665e00640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f366815c890 0x7f366815bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3665e00640 1 -- 172.21.15.66:0/2654994693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3660009420 con 0x7f36680ce190 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3665e00640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36680ce190 0x7f36680cbe60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f365800c7f0 tx=0x7f365800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:03.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f365ea00640 1 -- 172.21.15.66:0/2654994693 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658007820 con 0x7f36680ce190 2024-03-30T00:40:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f365ea00640 1 -- 172.21.15.66:0/2654994693 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3658007980 con 0x7f36680ce190 2024-03-30T00:40:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f365ea00640 1 -- 172.21.15.66:0/2654994693 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658007d50 con 0x7f36680ce190 2024-03-30T00:40:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 -- 172.21.15.66:0/2654994693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36680c83f0 con 0x7f36680ce190 2024-03-30T00:40:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.025+0000 7f3667200640 1 -- 172.21.15.66:0/2654994693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366814fd00 con 0x7f36680ce190 2024-03-30T00:40:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.029+0000 7f3667200640 1 -- 172.21.15.66:0/2654994693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36680c8910 con 0x7f36680ce190 2024-03-30T00:40:03.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.029+0000 7f365ea00640 1 -- 172.21.15.66:0/2654994693 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f36580113f0 con 0x7f36680ce190 2024-03-30T00:40:03.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.029+0000 7f365ea00640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3644066690 0x7f3644068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.029+0000 7f365ea00640 1 -- 172.21.15.66:0/2654994693 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3658084880 con 0x7f36680ce190 2024-03-30T00:40:03.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.029+0000 7f365d400640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3644066690 0x7f3644068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.029+0000 7f365d400640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3644066690 0x7f3644068b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3650005c10 tx=0x7f36500016a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:03.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.033+0000 7f365ea00640 1 -- 172.21.15.66:0/2654994693 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f36680c8910 con 0x7f36680ce190 2024-03-30T00:40:03.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.197+0000 7f3667200640 1 -- 172.21.15.66:0/2654994693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f36680c8910 con 0x7f36680ce190 2024-03-30T00:40:03.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.197+0000 7f365ea00640 1 -- 172.21.15.66:0/2654994693 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f365801f020 con 0x7f36680ce190 2024-03-30T00:40:03.207 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:03.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 -- 172.21.15.66:0/2654994693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3644066690 msgr2=0x7f3644068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3644066690 0x7f3644068b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3650005c10 tx=0x7f36500016a0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 -- 172.21.15.66:0/2654994693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36680ce190 msgr2=0x7f36680cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36680ce190 0x7f36680cbe60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f365800c7f0 tx=0x7f365800ccb0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 -- 172.21.15.66:0/2654994693 shutdown_connections 2024-03-30T00:40:03.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f366815c890 0x7f366815bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36680d1060 0x7f366814f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3644066690 0x7f3644068b50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 --2- 172.21.15.66:0/2654994693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36680ce190 0x7f36680cbe60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 -- 172.21.15.66:0/2654994693 >> 172.21.15.66:0/2654994693 conn(0x7f366809b8d0 msgr2=0x7f36680ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:03.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 -- 172.21.15.66:0/2654994693 shutdown_connections 2024-03-30T00:40:03.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.201+0000 7f3667200640 1 -- 172.21.15.66:0/2654994693 wait complete. 2024-03-30T00:40:03.229 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:40:03.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3acc00640 1 Processor -- start 2024-03-30T00:40:03.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3acc00640 1 -- start start 2024-03-30T00:40:03.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3acc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3a815de30 0x7ff3a80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3acc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff3a80af420 0x7ff3a80d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3acc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff3a80ce740 0x7ff3a80cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3acc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a80cf110 con 0x7ff3a80af420 2024-03-30T00:40:03.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3acc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a80cbb90 con 0x7ff3a80ce740 2024-03-30T00:40:03.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3acc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80ce1b0 con 0x7ff3a815de30 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff39d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff3a80af420 0x7ff3a80d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff39d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff3a80af420 0x7ff3a80d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52980/0 (socket says 172.21.15.66:52980) 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff39d400640 1 -- 172.21.15.66:0/2014274767 learned_addr learned my addr 172.21.15.66:0/2014274767 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3a5e00640 1 --2- 172.21.15.66:0/2014274767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3a815de30 0x7ff3a80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3a6800640 1 --2- 172.21.15.66:0/2014274767 >> v2:172.21.15.156:3300/0 conn(0x7ff3a80ce740 0x7ff3a80cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff39d400640 1 -- 172.21.15.66:0/2014274767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3a815de30 msgr2=0x7ff3a80cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff39d400640 1 --2- 172.21.15.66:0/2014274767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3a815de30 0x7ff3a80cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff39d400640 1 -- 172.21.15.66:0/2014274767 >> v2:172.21.15.156:3300/0 conn(0x7ff3a80ce740 msgr2=0x7ff3a80cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff39d400640 1 --2- 172.21.15.66:0/2014274767 >> v2:172.21.15.156:3300/0 conn(0x7ff3a80ce740 0x7ff3a80cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff39d400640 1 -- 172.21.15.66:0/2014274767 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80afa10 con 0x7ff3a80af420 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3a5e00640 1 --2- 172.21.15.66:0/2014274767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3a815de30 0x7ff3a80cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:03.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.361+0000 7ff3a6800640 1 --2- 172.21.15.66:0/2014274767 >> v2:172.21.15.156:3300/0 conn(0x7ff3a80ce740 0x7ff3a80cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:03.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff39d400640 1 --2- 172.21.15.66:0/2014274767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a80af420 0x7ff3a80d1060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8070b00 tx=0x7ff39001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf627a2cb72edd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:03.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3a5400640 1 -- 172.21.15.66:0/2014274767 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3900ab610 con 0x7ff3a80af420 2024-03-30T00:40:03.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3a5400640 1 -- 172.21.15.66:0/2014274767 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3900ab770 con 0x7ff3a80af420 2024-03-30T00:40:03.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3a5400640 1 -- 172.21.15.66:0/2014274767 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3900abb00 con 0x7ff3a80af420 2024-03-30T00:40:03.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3acc00640 1 -- 172.21.15.66:0/2014274767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a80af420 msgr2=0x7ff3a80d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3acc00640 1 --2- 172.21.15.66:0/2014274767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a80af420 0x7ff3a80d1060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8070b00 tx=0x7ff39001c430 comp rx=0 tx=0).stop 2024-03-30T00:40:03.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3acc00640 1 -- 172.21.15.66:0/2014274767 shutdown_connections 2024-03-30T00:40:03.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3acc00640 1 --2- 172.21.15.66:0/2014274767 >> v2:172.21.15.156:3300/0 conn(0x7ff3a80ce740 0x7ff3a80cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3acc00640 1 --2- 172.21.15.66:0/2014274767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a80af420 0x7ff3a80d1060 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3acc00640 1 --2- 172.21.15.66:0/2014274767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3a815de30 0x7ff3a80cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3acc00640 1 -- 172.21.15.66:0/2014274767 >> 172.21.15.66:0/2014274767 conn(0x7ff3a80d0540 msgr2=0x7ff3a80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:03.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3acc00640 1 -- 172.21.15.66:0/2014274767 shutdown_connections 2024-03-30T00:40:03.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.365+0000 7ff3acc00640 1 -- 172.21.15.66:0/2014274767 wait complete. 2024-03-30T00:40:03.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3acc00640 1 Processor -- start 2024-03-30T00:40:03.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3acc00640 1 -- start start 2024-03-30T00:40:03.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3acc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a80af420 0x7ff3a80c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3acc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3a80ce740 0x7ff3a80b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3acc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a815de30 0x7ff3a80ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3acc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80b55e0 con 0x7ff3a815de30 2024-03-30T00:40:03.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3acc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80b4960 con 0x7ff3a80af420 2024-03-30T00:40:03.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3acc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80b4c40 con 0x7ff3a80ce740 2024-03-30T00:40:03.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3a6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a815de30 0x7ff3a80ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3a6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a815de30 0x7ff3a80ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52994/0 (socket says 172.21.15.66:52994) 2024-03-30T00:40:03.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3a6800640 1 -- 172.21.15.66:0/4287884775 learned_addr learned my addr 172.21.15.66:0/4287884775 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:03.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3a5e00640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a80af420 0x7ff3a80c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff39d400640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3a80ce740 0x7ff3a80b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3a6800640 1 -- 172.21.15.66:0/4287884775 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3a80ce740 msgr2=0x7ff3a80b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3a6800640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3a80ce740 0x7ff3a80b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3a6800640 1 -- 172.21.15.66:0/4287884775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a80af420 msgr2=0x7ff3a80c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3a6800640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a80af420 0x7ff3a80c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3a6800640 1 -- 172.21.15.66:0/4287884775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3900094d0 con 0x7ff3a815de30 2024-03-30T00:40:03.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3a6800640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a815de30 0x7ff3a80ca240 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000cef0 tx=0x7ff3a000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:03.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff39ea00640 1 -- 172.21.15.66:0/4287884775 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a000d840 con 0x7ff3a815de30 2024-03-30T00:40:03.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff39ea00640 1 -- 172.21.15.66:0/4287884775 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a000d9a0 con 0x7ff3a815de30 2024-03-30T00:40:03.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff39ea00640 1 -- 172.21.15.66:0/4287884775 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a000dd50 con 0x7ff3a815de30 2024-03-30T00:40:03.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3acc00640 1 -- 172.21.15.66:0/4287884775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a80b4f20 con 0x7ff3a815de30 2024-03-30T00:40:03.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.369+0000 7ff3acc00640 1 -- 172.21.15.66:0/4287884775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a80c66d0 con 0x7ff3a815de30 2024-03-30T00:40:03.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.373+0000 7ff3acc00640 1 -- 172.21.15.66:0/4287884775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a80cdf60 con 0x7ff3a815de30 2024-03-30T00:40:03.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.373+0000 7ff39ea00640 1 -- 172.21.15.66:0/4287884775 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff3a001e3b0 con 0x7ff3a815de30 2024-03-30T00:40:03.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.373+0000 7ff39ea00640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff384066690 0x7ff384068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.373+0000 7ff39ea00640 1 -- 172.21.15.66:0/4287884775 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff3a0084d90 con 0x7ff3a815de30 2024-03-30T00:40:03.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.373+0000 7ff3a5e00640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff384066690 0x7ff384068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.373+0000 7ff3a5e00640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff384066690 0x7ff384068b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80b25b0 tx=0x7ff398008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:03.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.377+0000 7ff39ea00640 1 -- 172.21.15.66:0/4287884775 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff3a0051f50 con 0x7ff3a815de30 2024-03-30T00:40:03.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 -- 172.21.15.66:0/4287884775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff3a80b5720 con 0x7ff3a815de30 2024-03-30T00:40:03.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff39ea00640 1 -- 172.21.15.66:0/4287884775 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff3a80b5720 con 0x7ff3a815de30 2024-03-30T00:40:03.562 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:40:03.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 -- 172.21.15.66:0/4287884775 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff384066690 msgr2=0x7ff384068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff384066690 0x7ff384068b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80b25b0 tx=0x7ff398008040 comp rx=0 tx=0).stop 2024-03-30T00:40:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 -- 172.21.15.66:0/4287884775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a815de30 msgr2=0x7ff3a80ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a815de30 0x7ff3a80ca240 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000cef0 tx=0x7ff3a000a710 comp rx=0 tx=0).stop 2024-03-30T00:40:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 -- 172.21.15.66:0/4287884775 shutdown_connections 2024-03-30T00:40:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff384066690 0x7ff384068b50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a815de30 0x7ff3a80ca240 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3a80ce740 0x7ff3a80b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 --2- 172.21.15.66:0/4287884775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a80af420 0x7ff3a80c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 -- 172.21.15.66:0/4287884775 >> 172.21.15.66:0/4287884775 conn(0x7ff3a80d0540 msgr2=0x7ff3a815aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 -- 172.21.15.66:0/4287884775 shutdown_connections 2024-03-30T00:40:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.553+0000 7ff3acc00640 1 -- 172.21.15.66:0/4287884775 wait complete. 2024-03-30T00:40:03.584 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:40:03.584 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:40:03.592 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:03.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.765+0000 7fdd55200640 1 Processor -- start 2024-03-30T00:40:03.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.765+0000 7fdd55200640 1 -- start start 2024-03-30T00:40:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.765+0000 7fdd55200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cda90 0x7fdd500ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.765+0000 7fdd55200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd500cd3e0 con 0x7fdd5015a370 2024-03-30T00:40:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.765+0000 7fdd55200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5015a990 con 0x7fdd500cde70 2024-03-30T00:40:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.765+0000 7fdd55200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500b1b80 con 0x7fdd500cda90 2024-03-30T00:40:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.765+0000 7fdd4e800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdd5015a370 0x7fdd50156910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53256/0 (socket says 172.21.15.66:53256) 2024-03-30T00:40:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.765+0000 7fdd4e800640 1 -- 172.21.15.66:0/2016812387 learned_addr learned my addr 172.21.15.66:0/2016812387 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.765+0000 7fdd4de00640 1 --2- 172.21.15.66:0/2016812387 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cda90 0x7fdd500ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.765+0000 7fdd4d400640 1 -- 172.21.15.66:0/2016812387 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465509383 0 0) 0x7fdd500cd3e0 con 0x7fdd5015a370 2024-03-30T00:40:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.765+0000 7fdd4d400640 1 -- 172.21.15.66:0/2016812387 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd34003530 con 0x7fdd5015a370 2024-03-30T00:40:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.769+0000 7fdd4de00640 1 -- 172.21.15.66:0/2016812387 >> v1:172.21.15.156:6789/0 conn(0x7fdd500cde70 legacy=0x7fdd50159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.769+0000 7fdd4d400640 1 -- 172.21.15.66:0/2016812387 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272194211 0 0) 0x7fdd5015a990 con 0x7fdd500cde70 2024-03-30T00:40:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.769+0000 7fdd4de00640 1 -- 172.21.15.66:0/2016812387 >> v1:172.21.15.66:6789/0 conn(0x7fdd5015a370 legacy=0x7fdd50156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.769+0000 7fdd4de00640 1 -- 172.21.15.66:0/2016812387 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd500b11e0 con 0x7fdd500cda90 2024-03-30T00:40:03.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.769+0000 7fdd4de00640 1 --2- 172.21.15.66:0/2016812387 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cda90 0x7fdd500ce520 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdd400097f0 tx=0x7fdd4001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71fce1d75ae1a2e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:03.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.769+0000 7fdd4d400640 1 -- 172.21.15.66:0/2016812387 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd400ad660 con 0x7fdd500cda90 2024-03-30T00:40:03.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.769+0000 7fdd4d400640 1 -- 172.21.15.66:0/2016812387 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd400ad7c0 con 0x7fdd500cda90 2024-03-30T00:40:03.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.769+0000 7fdd4d400640 1 -- 172.21.15.66:0/2016812387 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd400adb50 con 0x7fdd500cda90 2024-03-30T00:40:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.769+0000 7fdd55200640 1 -- 172.21.15.66:0/2016812387 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cda90 msgr2=0x7fdd500ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.769+0000 7fdd55200640 1 --2- 172.21.15.66:0/2016812387 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cda90 0x7fdd500ce520 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdd400097f0 tx=0x7fdd4001c450 comp rx=0 tx=0).stop 2024-03-30T00:40:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 -- 172.21.15.66:0/2016812387 shutdown_connections 2024-03-30T00:40:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 --2- 172.21.15.66:0/2016812387 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cda90 0x7fdd500ce520 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 -- 172.21.15.66:0/2016812387 >> 172.21.15.66:0/2016812387 conn(0x7fdd50069f60 msgr2=0x7fdd5006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 -- 172.21.15.66:0/2016812387 shutdown_connections 2024-03-30T00:40:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 -- 172.21.15.66:0/2016812387 wait complete. 2024-03-30T00:40:03.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 Processor -- start 2024-03-30T00:40:03.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 -- start start 2024-03-30T00:40:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd500cda90 0x7fdd50164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd500cde70 0x7fdd500acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5015a370 0x7fdd5006fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd50165130 con 0x7fdd500cda90 2024-03-30T00:40:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500ab0b0 con 0x7fdd500cde70 2024-03-30T00:40:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500ab390 con 0x7fdd5015a370 2024-03-30T00:40:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd4de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd500cda90 0x7fdd50164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd4de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd500cda90 0x7fdd50164bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52996/0 (socket says 172.21.15.66:52996) 2024-03-30T00:40:03.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd4de00640 1 -- 172.21.15.66:0/470842401 learned_addr learned my addr 172.21.15.66:0/470842401 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:03.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd4de00640 1 -- 172.21.15.66:0/470842401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5015a370 msgr2=0x7fdd5006fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd4de00640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5015a370 0x7fdd5006fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd4de00640 1 -- 172.21.15.66:0/470842401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd500cde70 msgr2=0x7fdd500acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd4de00640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd500cde70 0x7fdd500acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd4de00640 1 -- 172.21.15.66:0/470842401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd40009420 con 0x7fdd500cda90 2024-03-30T00:40:03.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd4de00640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd500cda90 0x7fdd50164bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdd400b0040 tx=0x7fdd4001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd46a00640 1 -- 172.21.15.66:0/470842401 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40004020 con 0x7fdd500cda90 2024-03-30T00:40:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd46a00640 1 -- 172.21.15.66:0/470842401 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd400041a0 con 0x7fdd500cda90 2024-03-30T00:40:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd46a00640 1 -- 172.21.15.66:0/470842401 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40003620 con 0x7fdd500cda90 2024-03-30T00:40:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.773+0000 7fdd55200640 1 -- 172.21.15.66:0/470842401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd50157840 con 0x7fdd500cda90 2024-03-30T00:40:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.777+0000 7fdd55200640 1 -- 172.21.15.66:0/470842401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd500ad020 con 0x7fdd500cda90 2024-03-30T00:40:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.777+0000 7fdd55200640 1 -- 172.21.15.66:0/470842401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd501576a0 con 0x7fdd500cda90 2024-03-30T00:40:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.777+0000 7fdd46a00640 1 -- 172.21.15.66:0/470842401 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdd40003780 con 0x7fdd500cda90 2024-03-30T00:40:03.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.777+0000 7fdd46a00640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 0x7fdd2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:03.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.777+0000 7fdd46a00640 1 -- 172.21.15.66:0/470842401 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdd4011edb0 con 0x7fdd500cda90 2024-03-30T00:40:03.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.777+0000 7fdd45400640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 0x7fdd2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:03.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.777+0000 7fdd45400640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 0x7fdd2c068b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd38007fd0 tx=0x7fdd38007920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:03.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.781+0000 7fdd46a00640 1 -- 172.21.15.66:0/470842401 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdd501576a0 con 0x7fdd500cda90 2024-03-30T00:40:03.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.941+0000 7fdd55200640 1 -- 172.21.15.66:0/470842401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd501576a0 con 0x7fdd500cda90 2024-03-30T00:40:03.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd46a00640 1 -- 172.21.15.66:0/470842401 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fdd501576a0 con 0x7fdd500cda90 2024-03-30T00:40:03.954 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:03.954 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 -- 172.21.15.66:0/470842401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 msgr2=0x7fdd2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 0x7fdd2c068b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd38007fd0 tx=0x7fdd38007920 comp rx=0 tx=0).stop 2024-03-30T00:40:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 -- 172.21.15.66:0/470842401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd500cda90 msgr2=0x7fdd50164bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd500cda90 0x7fdd50164bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdd400b0040 tx=0x7fdd4001ac90 comp rx=0 tx=0).stop 2024-03-30T00:40:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 -- 172.21.15.66:0/470842401 shutdown_connections 2024-03-30T00:40:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5015a370 0x7fdd5006fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd500cde70 0x7fdd500acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 0x7fdd2c068b50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 --2- 172.21.15.66:0/470842401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd500cda90 0x7fdd50164bf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 -- 172.21.15.66:0/470842401 >> 172.21.15.66:0/470842401 conn(0x7fdd50069f60 msgr2=0x7fdd5006d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 -- 172.21.15.66:0/470842401 shutdown_connections 2024-03-30T00:40:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:03.945+0000 7fdd55200640 1 -- 172.21.15.66:0/470842401 wait complete. 2024-03-30T00:40:03.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:03.976 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:04.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f052dc00640 1 Processor -- start 2024-03-30T00:40:04.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f052dc00640 1 -- start start 2024-03-30T00:40:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f052dc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0528157060 0x7f052806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f052dc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f05281610c0 0x7f05281604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f052dc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f05281609f0 0x7f052815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f052dc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0528157440 con 0x7f05281609f0 2024-03-30T00:40:04.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f052dc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0528160ea0 con 0x7f05281610c0 2024-03-30T00:40:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f052dc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052806dc60 con 0x7f0528157060 2024-03-30T00:40:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f0527400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f05281609f0 0x7f052815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f0527400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f05281609f0 0x7f052815cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52998/0 (socket says 172.21.15.66:52998) 2024-03-30T00:40:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f0527400640 1 -- 172.21.15.66:0/131503784 learned_addr learned my addr 172.21.15.66:0/131503784 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f0526a00640 1 --2- 172.21.15.66:0/131503784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0528157060 0x7f052806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f051fe00640 1 --2- 172.21.15.66:0/131503784 >> v2:172.21.15.156:3300/0 conn(0x7f05281610c0 0x7f05281604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f0527400640 1 -- 172.21.15.66:0/131503784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0528157060 msgr2=0x7f052806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f0527400640 1 --2- 172.21.15.66:0/131503784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0528157060 0x7f052806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f0527400640 1 -- 172.21.15.66:0/131503784 >> v2:172.21.15.156:3300/0 conn(0x7f05281610c0 msgr2=0x7f05281604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f0527400640 1 --2- 172.21.15.66:0/131503784 >> v2:172.21.15.156:3300/0 conn(0x7f05281610c0 0x7f05281604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.109+0000 7f0527400640 1 -- 172.21.15.66:0/131503784 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052806df40 con 0x7f05281609f0 2024-03-30T00:40:04.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f0527400640 1 --2- 172.21.15.66:0/131503784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05281609f0 0x7f052815cec0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0528066920 tx=0x7f052001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cd09835b235f73 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:04.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f0526000640 1 -- 172.21.15.66:0/131503784 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200a5640 con 0x7f05281609f0 2024-03-30T00:40:04.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f0526000640 1 -- 172.21.15.66:0/131503784 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05200a57a0 con 0x7f05281609f0 2024-03-30T00:40:04.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f0526000640 1 -- 172.21.15.66:0/131503784 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200a5b30 con 0x7f05281609f0 2024-03-30T00:40:04.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f052dc00640 1 -- 172.21.15.66:0/131503784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05281609f0 msgr2=0x7f052815cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f052dc00640 1 --2- 172.21.15.66:0/131503784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05281609f0 0x7f052815cec0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0528066920 tx=0x7f052001c8d0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f052dc00640 1 -- 172.21.15.66:0/131503784 shutdown_connections 2024-03-30T00:40:04.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f052dc00640 1 --2- 172.21.15.66:0/131503784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05281609f0 0x7f052815cec0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f052dc00640 1 --2- 172.21.15.66:0/131503784 >> v2:172.21.15.156:3300/0 conn(0x7f05281610c0 0x7f05281604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f052dc00640 1 --2- 172.21.15.66:0/131503784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0528157060 0x7f052806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f052dc00640 1 -- 172.21.15.66:0/131503784 >> 172.21.15.66:0/131503784 conn(0x7f0528071750 msgr2=0x7f0528069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:04.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f052dc00640 1 -- 172.21.15.66:0/131503784 shutdown_connections 2024-03-30T00:40:04.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.113+0000 7f052dc00640 1 -- 172.21.15.66:0/131503784 wait complete. 2024-03-30T00:40:04.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f052dc00640 1 Processor -- start 2024-03-30T00:40:04.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f052dc00640 1 -- start start 2024-03-30T00:40:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f052dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528157060 0x7f05281406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f052dc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05281609f0 0x7f052815b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f052dc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05281610c0 0x7f052815bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f052dc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052815fea0 con 0x7f0528157060 2024-03-30T00:40:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f052dc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052815ee00 con 0x7f05281609f0 2024-03-30T00:40:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f052dc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052815f0e0 con 0x7f05281610c0 2024-03-30T00:40:04.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f0526a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528157060 0x7f05281406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f0526a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528157060 0x7f05281406a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53008/0 (socket says 172.21.15.66:53008) 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f0526a00640 1 -- 172.21.15.66:0/3270902855 learned_addr learned my addr 172.21.15.66:0/3270902855 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f0527400640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05281610c0 0x7f052815bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f051fe00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05281609f0 0x7f052815b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f0526a00640 1 -- 172.21.15.66:0/3270902855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05281610c0 msgr2=0x7f052815bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f0526a00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05281610c0 0x7f052815bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f0526a00640 1 -- 172.21.15.66:0/3270902855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05281609f0 msgr2=0x7f052815b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f0526a00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05281609f0 0x7f052815b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f0526a00640 1 -- 172.21.15.66:0/3270902855 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0520009b30 con 0x7f0528157060 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f0526a00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528157060 0x7f05281406a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f051800d800 tx=0x7f051800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f051ea00640 1 -- 172.21.15.66:0/3270902855 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05180096a0 con 0x7f0528157060 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f051ea00640 1 -- 172.21.15.66:0/3270902855 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0518009800 con 0x7f0528157060 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f051ea00640 1 -- 172.21.15.66:0/3270902855 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518009990 con 0x7f0528157060 2024-03-30T00:40:04.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f052dc00640 1 -- 172.21.15.66:0/3270902855 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052815f3c0 con 0x7f0528157060 2024-03-30T00:40:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.117+0000 7f052dc00640 1 -- 172.21.15.66:0/3270902855 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052815f980 con 0x7f0528157060 2024-03-30T00:40:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.121+0000 7f052dc00640 1 -- 172.21.15.66:0/3270902855 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052815fb40 con 0x7f0528157060 2024-03-30T00:40:04.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.121+0000 7f051ea00640 1 -- 172.21.15.66:0/3270902855 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0518009af0 con 0x7f0528157060 2024-03-30T00:40:04.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.121+0000 7f051ea00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0504066690 0x7f0504068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.121+0000 7f051ea00640 1 -- 172.21.15.66:0/3270902855 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0518084840 con 0x7f0528157060 2024-03-30T00:40:04.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.121+0000 7f051fe00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0504066690 0x7f0504068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.121+0000 7f051fe00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0504066690 0x7f0504068b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f05280acb40 tx=0x7f0510008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:04.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.125+0000 7f051ea00640 1 -- 172.21.15.66:0/3270902855 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f05180518d0 con 0x7f0528157060 2024-03-30T00:40:04.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.285+0000 7f052dc00640 1 -- 172.21.15.66:0/3270902855 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0528066c30 con 0x7f0528157060 2024-03-30T00:40:04.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.285+0000 7f051ea00640 1 -- 172.21.15.66:0/3270902855 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0518054ef0 con 0x7f0528157060 2024-03-30T00:40:04.296 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:04.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 -- 172.21.15.66:0/3270902855 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0504066690 msgr2=0x7f0504068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0504066690 0x7f0504068b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f05280acb40 tx=0x7f0510008040 comp rx=0 tx=0).stop 2024-03-30T00:40:04.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 -- 172.21.15.66:0/3270902855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528157060 msgr2=0x7f05281406a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528157060 0x7f05281406a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f051800d800 tx=0x7f051800dcc0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 -- 172.21.15.66:0/3270902855 shutdown_connections 2024-03-30T00:40:04.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05281610c0 0x7f052815bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05281609f0 0x7f052815b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0504066690 0x7f0504068b50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 --2- 172.21.15.66:0/3270902855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528157060 0x7f05281406a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 -- 172.21.15.66:0/3270902855 >> 172.21.15.66:0/3270902855 conn(0x7f0528071750 msgr2=0x7f05280acdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:04.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 -- 172.21.15.66:0/3270902855 shutdown_connections 2024-03-30T00:40:04.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.289+0000 7f052dc00640 1 -- 172.21.15.66:0/3270902855 wait complete. 2024-03-30T00:40:04.318 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:04.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.449+0000 7f297fe00640 1 Processor -- start 2024-03-30T00:40:04.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.449+0000 7f297fe00640 1 -- start start 2024-03-30T00:40:04.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2978098c80 0x7f2978099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297fe00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f29780cf320 0x7f29780ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297fe00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29780cbd90 con 0x7f2978099ca0 2024-03-30T00:40:04.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297fe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29780cc070 con 0x7f2978098c80 2024-03-30T00:40:04.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297fe00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29780cc350 con 0x7f29780cf320 2024-03-30T00:40:04.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2978098c80 0x7f2978099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2978098c80 0x7f2978099760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53014/0 (socket says 172.21.15.66:53014) 2024-03-30T00:40:04.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297d400640 1 -- 172.21.15.66:0/2129804921 learned_addr learned my addr 172.21.15.66:0/2129804921 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:04.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297ca00640 1 --2- 172.21.15.66:0/2129804921 >> v2:172.21.15.156:3300/0 conn(0x7f29780cf320 0x7f29780ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297d400640 1 -- 172.21.15.66:0/2129804921 >> v2:172.21.15.156:3300/0 conn(0x7f29780cf320 msgr2=0x7f29780ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:04.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297d400640 1 --2- 172.21.15.66:0/2129804921 >> v2:172.21.15.156:3300/0 conn(0x7f29780cf320 0x7f29780ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297d400640 1 -- 172.21.15.66:0/2129804921 >> v1:172.21.15.156:6789/0 conn(0x7f2978099ca0 legacy=0x7f29780ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:04.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297d400640 1 -- 172.21.15.66:0/2129804921 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29780b3530 con 0x7f2978098c80 2024-03-30T00:40:04.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.453+0000 7f297ca00640 1 --2- 172.21.15.66:0/2129804921 >> v2:172.21.15.156:3300/0 conn(0x7f29780cf320 0x7f29780ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:04.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297d400640 1 --2- 172.21.15.66:0/2129804921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978098c80 0x7f2978099760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f297806e280 tx=0x7f296c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d48d55dbc48b11 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:04.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f2977e00640 1 -- 172.21.15.66:0/2129804921 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0a89d0 con 0x7f2978098c80 2024-03-30T00:40:04.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f2977e00640 1 -- 172.21.15.66:0/2129804921 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c0a8b30 con 0x7f2978098c80 2024-03-30T00:40:04.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f2977e00640 1 -- 172.21.15.66:0/2129804921 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0b0440 con 0x7f2978098c80 2024-03-30T00:40:04.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 -- 172.21.15.66:0/2129804921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978098c80 msgr2=0x7f2978099760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 --2- 172.21.15.66:0/2129804921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978098c80 0x7f2978099760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f297806e280 tx=0x7f296c01ecd0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 -- 172.21.15.66:0/2129804921 shutdown_connections 2024-03-30T00:40:04.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 --2- 172.21.15.66:0/2129804921 >> v2:172.21.15.156:3300/0 conn(0x7f29780cf320 0x7f29780ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 --2- 172.21.15.66:0/2129804921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978098c80 0x7f2978099760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 -- 172.21.15.66:0/2129804921 >> 172.21.15.66:0/2129804921 conn(0x7f29780b2130 msgr2=0x7f29780b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:04.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 -- 172.21.15.66:0/2129804921 shutdown_connections 2024-03-30T00:40:04.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 -- 172.21.15.66:0/2129804921 wait complete. 2024-03-30T00:40:04.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 Processor -- start 2024-03-30T00:40:04.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 -- start start 2024-03-30T00:40:04.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978098c80 0x7f297815ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2978099ca0 0x7f297806f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29780cf320 0x7f297806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.457+0000 7f297fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29780c9f40 con 0x7f2978098c80 2024-03-30T00:40:04.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297fe00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297815dff0 con 0x7f2978099ca0 2024-03-30T00:40:04.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297fe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297815e2d0 con 0x7f29780cf320 2024-03-30T00:40:04.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978098c80 0x7f297815ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978098c80 0x7f297815ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53022/0 (socket says 172.21.15.66:53022) 2024-03-30T00:40:04.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297d400640 1 -- 172.21.15.66:0/1981008230 learned_addr learned my addr 172.21.15.66:0/1981008230 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:04.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297de00640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29780cf320 0x7f297806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297ca00640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2978099ca0 0x7f297806f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297d400640 1 -- 172.21.15.66:0/1981008230 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29780cf320 msgr2=0x7f297806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297d400640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29780cf320 0x7f297806f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297d400640 1 -- 172.21.15.66:0/1981008230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2978099ca0 msgr2=0x7f297806f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297d400640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2978099ca0 0x7f297806f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297d400640 1 -- 172.21.15.66:0/1981008230 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0094d0 con 0x7f2978098c80 2024-03-30T00:40:04.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297d400640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978098c80 0x7f297815ccd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b3040 tx=0x7f296c01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:04.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f2975600640 1 -- 172.21.15.66:0/1981008230 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0b0e80 con 0x7f2978098c80 2024-03-30T00:40:04.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f2975600640 1 -- 172.21.15.66:0/1981008230 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c0bb480 con 0x7f2978098c80 2024-03-30T00:40:04.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f2975600640 1 -- 172.21.15.66:0/1981008230 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0bb830 con 0x7f2978098c80 2024-03-30T00:40:04.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297fe00640 1 -- 172.21.15.66:0/1981008230 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29780c9980 con 0x7f2978098c80 2024-03-30T00:40:04.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f297fe00640 1 -- 172.21.15.66:0/1981008230 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29780c4400 con 0x7f2978098c80 2024-03-30T00:40:04.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f296aa00640 1 -- 172.21.15.66:0/1981008230 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29780677d0 con 0x7f2978098c80 2024-03-30T00:40:04.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f2975600640 1 -- 172.21.15.66:0/1981008230 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f296c0bb990 con 0x7f2978098c80 2024-03-30T00:40:04.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.461+0000 7f2975600640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2958066690 0x7f2958068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.465+0000 7f2975600640 1 -- 172.21.15.66:0/1981008230 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f296c122c50 con 0x7f2978098c80 2024-03-30T00:40:04.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.465+0000 7f297ca00640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2958066690 0x7f2958068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.465+0000 7f297ca00640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2958066690 0x7f2958068b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2964006fd0 tx=0x7f2964008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:04.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.465+0000 7f2975600640 1 -- 172.21.15.66:0/1981008230 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f296c0efc30 con 0x7f2978098c80 2024-03-30T00:40:04.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.625+0000 7f296aa00640 1 -- 172.21.15.66:0/1981008230 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29780677d0 con 0x7f2978098c80 2024-03-30T00:40:04.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.625+0000 7f2975600640 1 -- 172.21.15.66:0/1981008230 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f296c0f3250 con 0x7f2978098c80 2024-03-30T00:40:04.636 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:04.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 -- 172.21.15.66:0/1981008230 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2958066690 msgr2=0x7f2958068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2958066690 0x7f2958068b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2964006fd0 tx=0x7f2964008040 comp rx=0 tx=0).stop 2024-03-30T00:40:04.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 -- 172.21.15.66:0/1981008230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978098c80 msgr2=0x7f297815ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978098c80 0x7f297815ccd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b3040 tx=0x7f296c01ba70 comp rx=0 tx=0).stop 2024-03-30T00:40:04.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 -- 172.21.15.66:0/1981008230 shutdown_connections 2024-03-30T00:40:04.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29780cf320 0x7f297806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2978099ca0 0x7f297806f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2958066690 0x7f2958068b50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 --2- 172.21.15.66:0/1981008230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978098c80 0x7f297815ccd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 -- 172.21.15.66:0/1981008230 >> 172.21.15.66:0/1981008230 conn(0x7f29780b2130 msgr2=0x7f297815a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:04.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 -- 172.21.15.66:0/1981008230 shutdown_connections 2024-03-30T00:40:04.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.629+0000 7f296aa00640 1 -- 172.21.15.66:0/1981008230 wait complete. 2024-03-30T00:40:04.657 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:04.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.789+0000 7fb8aa600640 1 Processor -- start 2024-03-30T00:40:04.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.789+0000 7fb8aa600640 1 -- start start 2024-03-30T00:40:04.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.789+0000 7fb8aa600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb8a415a670 0x7fb8a40b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.789+0000 7fb8aa600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40d01d0 0x7fb8a40d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.789+0000 7fb8aa600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a40cb520 con 0x7fb8a40ce710 2024-03-30T00:40:04.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.789+0000 7fb8aa600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a40d0b80 con 0x7fb8a415a670 2024-03-30T00:40:04.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.789+0000 7fb8aa600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a40cd500 con 0x7fb8a40d01d0 2024-03-30T00:40:04.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb8a415a670 0x7fb8a40b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb89be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb8a40ce710 0x7fb8a40ccdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53268/0 (socket says 172.21.15.66:53268) 2024-03-30T00:40:04.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb89be00640 1 -- 172.21.15.66:0/1068044447 learned_addr learned my addr 172.21.15.66:0/1068044447 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:04.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb8a415a670 0x7fb8a40b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53024/0 (socket says 172.21.15.66:53024) 2024-03-30T00:40:04.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a3e00640 1 --2- 172.21.15.66:0/1068044447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40d01d0 0x7fb8a40d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a3400640 1 -- 172.21.15.66:0/1068044447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40d01d0 msgr2=0x7fb8a40d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:04.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a3400640 1 --2- 172.21.15.66:0/1068044447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40d01d0 0x7fb8a40d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a3400640 1 -- 172.21.15.66:0/1068044447 >> v1:172.21.15.66:6789/0 conn(0x7fb8a40ce710 legacy=0x7fb8a40ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a3400640 1 -- 172.21.15.66:0/1068044447 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a40cd7d0 con 0x7fb8a415a670 2024-03-30T00:40:04.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a2a00640 1 -- 172.21.15.66:0/1068044447 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203624999 0 0) 0x7fb8a40cb520 con 0x7fb8a40ce710 2024-03-30T00:40:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a3400640 1 --2- 172.21.15.66:0/1068044447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a415a670 0x7fb8a40b0200 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb8940097c0 tx=0x7fb89401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4b4eec0b299fa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a2a00640 1 -- 172.21.15.66:0/1068044447 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8940ab670 con 0x7fb8a415a670 2024-03-30T00:40:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a2a00640 1 -- 172.21.15.66:0/1068044447 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8940ab7d0 con 0x7fb8a415a670 2024-03-30T00:40:04.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.793+0000 7fb8a2a00640 1 -- 172.21.15.66:0/1068044447 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8940abb60 con 0x7fb8a415a670 2024-03-30T00:40:04.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 -- 172.21.15.66:0/1068044447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a415a670 msgr2=0x7fb8a40b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 --2- 172.21.15.66:0/1068044447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a415a670 0x7fb8a40b0200 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb8940097c0 tx=0x7fb89401c470 comp rx=0 tx=0).stop 2024-03-30T00:40:04.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 -- 172.21.15.66:0/1068044447 shutdown_connections 2024-03-30T00:40:04.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 --2- 172.21.15.66:0/1068044447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40d01d0 0x7fb8a40d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 --2- 172.21.15.66:0/1068044447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a415a670 0x7fb8a40b0200 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 -- 172.21.15.66:0/1068044447 >> 172.21.15.66:0/1068044447 conn(0x7fb8a4099870 msgr2=0x7fb8a4089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:04.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 -- 172.21.15.66:0/1068044447 shutdown_connections 2024-03-30T00:40:04.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 -- 172.21.15.66:0/1068044447 wait complete. 2024-03-30T00:40:04.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 Processor -- start 2024-03-30T00:40:04.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 -- start start 2024-03-30T00:40:04.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40ce710 0x7fb8a41f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8a40d01d0 0x7fb8a41597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a415a670 0x7fb8a4159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a41f4dc0 con 0x7fb8a415a670 2024-03-30T00:40:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a4154ba0 con 0x7fb8a40d01d0 2024-03-30T00:40:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8aa600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a40c40c0 con 0x7fb8a40ce710 2024-03-30T00:40:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8a3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a415a670 0x7fb8a4159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8a3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a415a670 0x7fb8a4159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53040/0 (socket says 172.21.15.66:53040) 2024-03-30T00:40:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8a3e00640 1 -- 172.21.15.66:0/2062484434 learned_addr learned my addr 172.21.15.66:0/2062484434 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8a3400640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40ce710 0x7fb8a41f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb89be00640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8a40d01d0 0x7fb8a41597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8a3e00640 1 -- 172.21.15.66:0/2062484434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40ce710 msgr2=0x7fb8a41f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8a3e00640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40ce710 0x7fb8a41f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8a3e00640 1 -- 172.21.15.66:0/2062484434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8a40d01d0 msgr2=0x7fb8a41597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8a3e00640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8a40d01d0 0x7fb8a41597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb8a3e00640 1 -- 172.21.15.66:0/2062484434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb894009420 con 0x7fb8a415a670 2024-03-30T00:40:04.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.797+0000 7fb89be00640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8a40d01d0 0x7fb8a41597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:04.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.801+0000 7fb8a3e00640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a415a670 0x7fb8a4159cf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00efc0 tx=0x7fb89c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:04.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.801+0000 7fb89b400640 1 -- 172.21.15.66:0/2062484434 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c014480 con 0x7fb8a415a670 2024-03-30T00:40:04.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.801+0000 7fb89b400640 1 -- 172.21.15.66:0/2062484434 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c00f040 con 0x7fb8a415a670 2024-03-30T00:40:04.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.801+0000 7fb8aa600640 1 -- 172.21.15.66:0/2062484434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a415abd0 con 0x7fb8a415a670 2024-03-30T00:40:04.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.801+0000 7fb89b400640 1 -- 172.21.15.66:0/2062484434 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c014820 con 0x7fb8a415a670 2024-03-30T00:40:04.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.801+0000 7fb8aa600640 1 -- 172.21.15.66:0/2062484434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a41545e0 con 0x7fb8a415a670 2024-03-30T00:40:04.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.801+0000 7fb8aa600640 1 -- 172.21.15.66:0/2062484434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a40b0720 con 0x7fb8a415a670 2024-03-30T00:40:04.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.801+0000 7fb89b400640 1 -- 172.21.15.66:0/2062484434 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb89c014980 con 0x7fb8a415a670 2024-03-30T00:40:04.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.801+0000 7fb89b400640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb880066690 0x7fb880068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:04.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.801+0000 7fb89b400640 1 -- 172.21.15.66:0/2062484434 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb89c085300 con 0x7fb8a415a670 2024-03-30T00:40:04.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.801+0000 7fb8a3400640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb880066690 0x7fb880068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:04.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.805+0000 7fb8a3400640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb880066690 0x7fb880068b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8940af000 tx=0x7fb8940a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:04.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.805+0000 7fb89b400640 1 -- 172.21.15.66:0/2062484434 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb8a40b0720 con 0x7fb8a415a670 2024-03-30T00:40:04.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.965+0000 7fb8aa600640 1 -- 172.21.15.66:0/2062484434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8a40b0720 con 0x7fb8a415a670 2024-03-30T00:40:04.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.965+0000 7fb89b400640 1 -- 172.21.15.66:0/2062484434 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8a40b0720 con 0x7fb8a415a670 2024-03-30T00:40:04.976 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:04.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 -- 172.21.15.66:0/2062484434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb880066690 msgr2=0x7fb880068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb880066690 0x7fb880068b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8940af000 tx=0x7fb8940a7000 comp rx=0 tx=0).stop 2024-03-30T00:40:04.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 -- 172.21.15.66:0/2062484434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a415a670 msgr2=0x7fb8a4159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:04.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a415a670 0x7fb8a4159cf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00efc0 tx=0x7fb89c00c490 comp rx=0 tx=0).stop 2024-03-30T00:40:04.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 -- 172.21.15.66:0/2062484434 shutdown_connections 2024-03-30T00:40:04.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb880066690 0x7fb880068b50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a415a670 0x7fb8a4159cf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8a40d01d0 0x7fb8a41597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 --2- 172.21.15.66:0/2062484434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40ce710 0x7fb8a41f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:04.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 -- 172.21.15.66:0/2062484434 >> 172.21.15.66:0/2062484434 conn(0x7fb8a4099870 msgr2=0x7fb8a4164a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:04.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 -- 172.21.15.66:0/2062484434 shutdown_connections 2024-03-30T00:40:04.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:04.969+0000 7fb8aa600640 1 -- 172.21.15.66:0/2062484434 wait complete. 2024-03-30T00:40:04.998 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:05.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51bda00640 1 Processor -- start 2024-03-30T00:40:05.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51bda00640 1 -- start start 2024-03-30T00:40:05.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51bda00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f51b815a640 0x7f51b806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:05.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51bda00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b806d070 con 0x7f51b80cb2c0 2024-03-30T00:40:05.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51bda00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b80cbba0 con 0x7f51b80ce520 2024-03-30T00:40:05.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51bda00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b806e2c0 con 0x7f51b815a640 2024-03-30T00:40:05.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51b7400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f51b80cb2c0 0x7f51b80ab010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53282/0 (socket says 172.21.15.66:53282) 2024-03-30T00:40:05.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51b7400640 1 -- 172.21.15.66:0/3481514193 learned_addr learned my addr 172.21.15.66:0/3481514193 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:05.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51b6a00640 1 --2- 172.21.15.66:0/3481514193 >> v2:172.21.15.156:3300/0 conn(0x7f51b815a640 0x7f51b806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:05.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51b6000640 1 -- 172.21.15.66:0/3481514193 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771383173 0 0) 0x7f51b806d070 con 0x7f51b80cb2c0 2024-03-30T00:40:05.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51b6000640 1 -- 172.21.15.66:0/3481514193 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f519c003530 con 0x7f51b80cb2c0 2024-03-30T00:40:05.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51b6a00640 1 -- 172.21.15.66:0/3481514193 >> v1:172.21.15.156:6789/0 conn(0x7f51b80ce520 legacy=0x7f51b80cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51b6000640 1 -- 172.21.15.66:0/3481514193 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133072641 0 0) 0x7f51b80cbba0 con 0x7f51b80ce520 2024-03-30T00:40:05.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51b6a00640 1 -- 172.21.15.66:0/3481514193 >> v1:172.21.15.66:6789/0 conn(0x7f51b80cb2c0 legacy=0x7f51b80ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.133+0000 7f51b6a00640 1 -- 172.21.15.66:0/3481514193 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b80cb8c0 con 0x7f51b815a640 2024-03-30T00:40:05.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.137+0000 7f51b6a00640 1 --2- 172.21.15.66:0/3481514193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51b815a640 0x7f51b806dd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f51a80097c0 tx=0x7f51a801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52b75ac4bc746764 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:05.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.137+0000 7f51b6000640 1 -- 172.21.15.66:0/3481514193 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a80a8050 con 0x7f51b815a640 2024-03-30T00:40:05.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.137+0000 7f51b6000640 1 -- 172.21.15.66:0/3481514193 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a80ac5f0 con 0x7f51b815a640 2024-03-30T00:40:05.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.137+0000 7f51b6000640 1 -- 172.21.15.66:0/3481514193 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a80ac980 con 0x7f51b815a640 2024-03-30T00:40:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.137+0000 7f51bda00640 1 -- 172.21.15.66:0/3481514193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51b815a640 msgr2=0x7f51b806dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.137+0000 7f51bda00640 1 --2- 172.21.15.66:0/3481514193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51b815a640 0x7f51b806dd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f51a80097c0 tx=0x7f51a801c4a0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.137+0000 7f51bda00640 1 -- 172.21.15.66:0/3481514193 shutdown_connections 2024-03-30T00:40:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.137+0000 7f51bda00640 1 --2- 172.21.15.66:0/3481514193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51b815a640 0x7f51b806dd80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.137+0000 7f51bda00640 1 -- 172.21.15.66:0/3481514193 >> 172.21.15.66:0/3481514193 conn(0x7f51b8069f60 msgr2=0x7f51b806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.137+0000 7f51bda00640 1 -- 172.21.15.66:0/3481514193 shutdown_connections 2024-03-30T00:40:05.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.137+0000 7f51bda00640 1 -- 172.21.15.66:0/3481514193 wait complete. 2024-03-30T00:40:05.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51bda00640 1 Processor -- start 2024-03-30T00:40:05.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51bda00640 1 -- start start 2024-03-30T00:40:05.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51bda00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b80cb2c0 0x7f51b814bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51bda00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51b80ce520 0x7f51b80cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51bda00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51b815a640 0x7f51b8157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51bda00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b814c2c0 con 0x7f51b80cb2c0 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51bda00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b80cd410 con 0x7f51b815a640 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51bda00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b80adaa0 con 0x7f51b80ce520 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51b6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b80cb2c0 0x7f51b814bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51b6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b80cb2c0 0x7f51b814bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53048/0 (socket says 172.21.15.66:53048) 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51b6a00640 1 -- 172.21.15.66:0/990166555 learned_addr learned my addr 172.21.15.66:0/990166555 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51b7400640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51b815a640 0x7f51b8157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51afe00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51b80ce520 0x7f51b80cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51b6a00640 1 -- 172.21.15.66:0/990166555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51b80ce520 msgr2=0x7f51b80cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51b6a00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51b80ce520 0x7f51b80cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51b6a00640 1 -- 172.21.15.66:0/990166555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51b815a640 msgr2=0x7f51b8157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51b6a00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51b815a640 0x7f51b8157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51b6a00640 1 -- 172.21.15.66:0/990166555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a8009420 con 0x7f51b80cb2c0 2024-03-30T00:40:05.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51afe00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51b80ce520 0x7f51b80cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:05.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51b6a00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b80cb2c0 0x7f51b814bd80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f51a80097c0 tx=0x7f51a8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:05.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51aea00640 1 -- 172.21.15.66:0/990166555 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a8006e20 con 0x7f51b80cb2c0 2024-03-30T00:40:05.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51aea00640 1 -- 172.21.15.66:0/990166555 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a8004020 con 0x7f51b80cb2c0 2024-03-30T00:40:05.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51aea00640 1 -- 172.21.15.66:0/990166555 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a8004390 con 0x7f51b80cb2c0 2024-03-30T00:40:05.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51bda00640 1 -- 172.21.15.66:0/990166555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b80cdd50 con 0x7f51b80cb2c0 2024-03-30T00:40:05.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.141+0000 7f51bda00640 1 -- 172.21.15.66:0/990166555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b80cce50 con 0x7f51b80cb2c0 2024-03-30T00:40:05.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.145+0000 7f51bda00640 1 -- 172.21.15.66:0/990166555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b80b1cc0 con 0x7f51b80cb2c0 2024-03-30T00:40:05.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.145+0000 7f51aea00640 1 -- 172.21.15.66:0/990166555 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f51a80b3040 con 0x7f51b80cb2c0 2024-03-30T00:40:05.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.145+0000 7f51aea00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5194066690 0x7f5194068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:05.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.145+0000 7f51aea00640 1 -- 172.21.15.66:0/990166555 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f51a811e3f0 con 0x7f51b80cb2c0 2024-03-30T00:40:05.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.145+0000 7f51afe00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5194066690 0x7f5194068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:05.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.145+0000 7f51afe00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5194066690 0x7f5194068b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51b81f8db0 tx=0x7f51a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:05.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.149+0000 7f51aea00640 1 -- 172.21.15.66:0/990166555 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f51a80eb460 con 0x7f51b80cb2c0 2024-03-30T00:40:05.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.309+0000 7f51bda00640 1 -- 172.21.15.66:0/990166555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f51b8066c30 con 0x7f51b80cb2c0 2024-03-30T00:40:05.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.309+0000 7f51aea00640 1 -- 172.21.15.66:0/990166555 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f51a80044f0 con 0x7f51b80cb2c0 2024-03-30T00:40:05.320 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:05.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 -- 172.21.15.66:0/990166555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5194066690 msgr2=0x7f5194068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5194066690 0x7f5194068b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51b81f8db0 tx=0x7f51a0009040 comp rx=0 tx=0).stop 2024-03-30T00:40:05.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 -- 172.21.15.66:0/990166555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b80cb2c0 msgr2=0x7f51b814bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b80cb2c0 0x7f51b814bd80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f51a80097c0 tx=0x7f51a8019240 comp rx=0 tx=0).stop 2024-03-30T00:40:05.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 -- 172.21.15.66:0/990166555 shutdown_connections 2024-03-30T00:40:05.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51b815a640 0x7f51b8157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51b80ce520 0x7f51b80cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5194066690 0x7f5194068b50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 --2- 172.21.15.66:0/990166555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b80cb2c0 0x7f51b814bd80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 -- 172.21.15.66:0/990166555 >> 172.21.15.66:0/990166555 conn(0x7f51b8069f60 msgr2=0x7f51b80c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:05.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 -- 172.21.15.66:0/990166555 shutdown_connections 2024-03-30T00:40:05.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.313+0000 7f51bda00640 1 -- 172.21.15.66:0/990166555 wait complete. 2024-03-30T00:40:05.340 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:40:05.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a6ea00640 1 Processor -- start 2024-03-30T00:40:05.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a6ea00640 1 -- start start 2024-03-30T00:40:05.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a6ea00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a6815de30 0x7f1a680cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:05.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a6ea00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680af420 0x7f1a680d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:05.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a6ea00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a680cbb90 con 0x7f1a680ce740 2024-03-30T00:40:05.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a6ea00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a680ce1b0 con 0x7f1a6815de30 2024-03-30T00:40:05.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a6ea00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680afa10 con 0x7f1a680af420 2024-03-30T00:40:05.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a6ca00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1a680ce740 0x7f1a6806e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53292/0 (socket says 172.21.15.66:53292) 2024-03-30T00:40:05.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a67e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a6815de30 0x7f1a680cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:05.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a6ca00640 1 -- 172.21.15.66:0/3767879151 learned_addr learned my addr 172.21.15.66:0/3767879151 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:05.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a5f400640 1 --2- 172.21.15.66:0/3767879151 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680af420 0x7f1a680d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:05.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a67400640 1 -- 172.21.15.66:0/3767879151 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232544907 0 0) 0x7f1a680cbb90 con 0x7f1a680ce740 2024-03-30T00:40:05.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a67400640 1 -- 172.21.15.66:0/3767879151 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a4c003530 con 0x7f1a680ce740 2024-03-30T00:40:05.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a5f400640 1 -- 172.21.15.66:0/3767879151 >> v2:172.21.15.156:3300/0 conn(0x7f1a6815de30 msgr2=0x7f1a680cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:05.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a5f400640 1 --2- 172.21.15.66:0/3767879151 >> v2:172.21.15.156:3300/0 conn(0x7f1a6815de30 0x7f1a680cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a5f400640 1 -- 172.21.15.66:0/3767879151 >> v1:172.21.15.66:6789/0 conn(0x7f1a680ce740 legacy=0x7f1a6806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.473+0000 7f1a5f400640 1 -- 172.21.15.66:0/3767879151 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680cf430 con 0x7f1a680af420 2024-03-30T00:40:05.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a5f400640 1 --2- 172.21.15.66:0/3767879151 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680af420 0x7f1a680d1060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a500097c0 tx=0x7f1a5001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2e629de5df86f0c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:05.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a67400640 1 -- 172.21.15.66:0/3767879151 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500ab600 con 0x7f1a680af420 2024-03-30T00:40:05.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a67400640 1 -- 172.21.15.66:0/3767879151 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a500a7050 con 0x7f1a680af420 2024-03-30T00:40:05.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a67400640 1 -- 172.21.15.66:0/3767879151 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500ab980 con 0x7f1a680af420 2024-03-30T00:40:05.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a6ea00640 1 -- 172.21.15.66:0/3767879151 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680af420 msgr2=0x7f1a680d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a6ea00640 1 --2- 172.21.15.66:0/3767879151 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680af420 0x7f1a680d1060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a500097c0 tx=0x7f1a5001c410 comp rx=0 tx=0).stop 2024-03-30T00:40:05.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a6ea00640 1 -- 172.21.15.66:0/3767879151 shutdown_connections 2024-03-30T00:40:05.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a6ea00640 1 --2- 172.21.15.66:0/3767879151 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680af420 0x7f1a680d1060 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a6ea00640 1 --2- 172.21.15.66:0/3767879151 >> v2:172.21.15.156:3300/0 conn(0x7f1a6815de30 0x7f1a680cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a6ea00640 1 -- 172.21.15.66:0/3767879151 >> 172.21.15.66:0/3767879151 conn(0x7f1a680d0540 msgr2=0x7f1a680d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:05.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a6ea00640 1 -- 172.21.15.66:0/3767879151 shutdown_connections 2024-03-30T00:40:05.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.477+0000 7f1a6ea00640 1 -- 172.21.15.66:0/3767879151 wait complete. 2024-03-30T00:40:05.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ea00640 1 Processor -- start 2024-03-30T00:40:05.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ea00640 1 -- start start 2024-03-30T00:40:05.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a680af420 0x7f1a68156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:05.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ea00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680ce740 0x7f1a68153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ea00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6815de30 0x7f1a68150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681581e0 con 0x7f1a680af420 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ea00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681575d0 con 0x7f1a6815de30 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ea00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681578b0 con 0x7f1a680ce740 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a67e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a680af420 0x7f1a68156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a67e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a680af420 0x7f1a68156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53058/0 (socket says 172.21.15.66:53058) 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a67e00640 1 -- 172.21.15.66:0/1752431251 learned_addr learned my addr 172.21.15.66:0/1752431251 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ca00640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6815de30 0x7f1a68150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a5f400640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680ce740 0x7f1a68153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a67e00640 1 -- 172.21.15.66:0/1752431251 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680ce740 msgr2=0x7f1a68153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a67e00640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680ce740 0x7f1a68153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a67e00640 1 -- 172.21.15.66:0/1752431251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6815de30 msgr2=0x7f1a68150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a67e00640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6815de30 0x7f1a68150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a67e00640 1 -- 172.21.15.66:0/1752431251 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a50009420 con 0x7f1a680af420 2024-03-30T00:40:05.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a5f400640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680ce740 0x7f1a68153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:05.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a67e00640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a680af420 0x7f1a68156cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800e870 tx=0x7f1a5800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:05.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a64c00640 1 -- 172.21.15.66:0/1752431251 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58009510 con 0x7f1a680af420 2024-03-30T00:40:05.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a64c00640 1 -- 172.21.15.66:0/1752431251 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5800eec0 con 0x7f1a680af420 2024-03-30T00:40:05.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a64c00640 1 -- 172.21.15.66:0/1752431251 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800cb40 con 0x7f1a680af420 2024-03-30T00:40:05.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ea00640 1 -- 172.21.15.66:0/1752431251 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a68157b90 con 0x7f1a680af420 2024-03-30T00:40:05.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ea00640 1 -- 172.21.15.66:0/1752431251 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a68157210 con 0x7f1a680af420 2024-03-30T00:40:05.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.481+0000 7f1a6ea00640 1 -- 172.21.15.66:0/1752431251 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a680cdf60 con 0x7f1a680af420 2024-03-30T00:40:05.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.485+0000 7f1a64c00640 1 -- 172.21.15.66:0/1752431251 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1a5800cca0 con 0x7f1a680af420 2024-03-30T00:40:05.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.485+0000 7f1a64c00640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a44066690 0x7f1a44068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:05.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.485+0000 7f1a64c00640 1 -- 172.21.15.66:0/1752431251 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1a580844d0 con 0x7f1a680af420 2024-03-30T00:40:05.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.485+0000 7f1a5f400640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a44066690 0x7f1a44068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:05.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.485+0000 7f1a5f400640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a44066690 0x7f1a44068b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1a500ad000 tx=0x7f1a50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:05.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.489+0000 7f1a64c00640 1 -- 172.21.15.66:0/1752431251 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1a58051600 con 0x7f1a680af420 2024-03-30T00:40:05.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.661+0000 7f1a6ea00640 1 -- 172.21.15.66:0/1752431251 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1a68066c30 con 0x7f1a680af420 2024-03-30T00:40:05.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.661+0000 7f1a64c00640 1 -- 172.21.15.66:0/1752431251 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1a58054c20 con 0x7f1a680af420 2024-03-30T00:40:05.672 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:40:05.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 -- 172.21.15.66:0/1752431251 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a44066690 msgr2=0x7f1a44068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a44066690 0x7f1a44068b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1a500ad000 tx=0x7f1a50008040 comp rx=0 tx=0).stop 2024-03-30T00:40:05.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 -- 172.21.15.66:0/1752431251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a680af420 msgr2=0x7f1a68156cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a680af420 0x7f1a68156cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800e870 tx=0x7f1a5800ec70 comp rx=0 tx=0).stop 2024-03-30T00:40:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 -- 172.21.15.66:0/1752431251 shutdown_connections 2024-03-30T00:40:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6815de30 0x7f1a68150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a680ce740 0x7f1a68153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a44066690 0x7f1a44068b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 --2- 172.21.15.66:0/1752431251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a680af420 0x7f1a68156cd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 -- 172.21.15.66:0/1752431251 >> 172.21.15.66:0/1752431251 conn(0x7f1a680d0540 msgr2=0x7f1a680ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 -- 172.21.15.66:0/1752431251 shutdown_connections 2024-03-30T00:40:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:05.665+0000 7f1a6ea00640 1 -- 172.21.15.66:0/1752431251 wait complete. 2024-03-30T00:40:05.694 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:40:05.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:05.694 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-30T00:40:05.720 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[33740]: fuse finished with error 0 and tester_r 0 2024-03-30T00:40:05.723 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:40:11.825 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:40:11.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:11.825 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-30T00:40:11.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:12.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.033+0000 7f76e4a00640 1 Processor -- start 2024-03-30T00:40:12.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.033+0000 7f76e4a00640 1 -- start start 2024-03-30T00:40:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.033+0000 7f76e4a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f76e00cfd80 0x7f76e00cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.033+0000 7f76e4a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e00c5c80 con 0x7f76e0159670 2024-03-30T00:40:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.033+0000 7f76e4a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e0071920 con 0x7f76e00b22a0 2024-03-30T00:40:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.033+0000 7f76e4a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e0159a50 con 0x7f76e00cfd80 2024-03-30T00:40:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.033+0000 7f76de800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f76e0159670 0x7f76e00cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36314/0 (socket says 172.21.15.66:36314) 2024-03-30T00:40:12.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.033+0000 7f76de800640 1 -- 172.21.15.66:0/3381429068 learned_addr learned my addr 172.21.15.66:0/3381429068 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.033+0000 7f76d5e00640 1 --2- 172.21.15.66:0/3381429068 >> v2:172.21.15.156:3300/0 conn(0x7f76e00cfd80 0x7f76e00cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.033+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446382631 0 0) 0x7f76e00c5c80 con 0x7f76e0159670 2024-03-30T00:40:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.033+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c4003530 con 0x7f76e0159670 2024-03-30T00:40:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 827712200 0 0) 0x7f76c4003530 con 0x7f76e0159670 2024-03-30T00:40:12.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 >> v2:172.21.15.156:3300/0 conn(0x7f76e00cfd80 msgr2=0x7f76e00cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 --2- 172.21.15.66:0/3381429068 >> v2:172.21.15.156:3300/0 conn(0x7f76e00cfd80 0x7f76e00cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 >> v1:172.21.15.156:6789/0 conn(0x7f76e00b22a0 legacy=0x7f76e00b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e00c5350 con 0x7f76e0159670 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f76d0001680 con 0x7f76e0159670 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 >> v1:172.21.15.66:6789/0 conn(0x7f76e0159670 legacy=0x7f76e00cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c40039d0 0x7f76c4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c40047d0 0x7f76c4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76c40051e0 0x7f76c400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0159a50 con 0x7f76c40039d0 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c400c130 con 0x7f76c40047d0 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c400c2f0 con 0x7f76c40051e0 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76de800640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c40039d0 0x7f76c4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76d5e00640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c40047d0 0x7f76c4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76df200640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76c40051e0 0x7f76c400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76de800640 1 -- 172.21.15.66:0/3381429068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76c40051e0 msgr2=0x7f76c400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:12.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76de800640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76c40051e0 0x7f76c400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76de800640 1 -- 172.21.15.66:0/3381429068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c40047d0 msgr2=0x7f76c4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:12.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76de800640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c40047d0 0x7f76c4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76de800640 1 -- 172.21.15.66:0/3381429068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c4003530 con 0x7f76c40039d0 2024-03-30T00:40:12.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76d5e00640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c40047d0 0x7f76c4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:12.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76df200640 1 -- 172.21.15.66:0/3381429068 reap_dead start 2024-03-30T00:40:12.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.037+0000 7f76de800640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c40039d0 0x7f76c4003df0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f76d0001ec0 tx=0x7f76d001cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caa2ebedf1ea5dc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0005ae0 con 0x7f76c40039d0 2024-03-30T00:40:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d0005c40 con 0x7f76c40039d0 2024-03-30T00:40:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76dde00640 1 -- 172.21.15.66:0/3381429068 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d00072b0 con 0x7f76c40039d0 2024-03-30T00:40:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 -- 172.21.15.66:0/3381429068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c40039d0 msgr2=0x7f76c4003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c40039d0 0x7f76c4003df0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f76d0001ec0 tx=0x7f76d001cb30 comp rx=0 tx=0).stop 2024-03-30T00:40:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 -- 172.21.15.66:0/3381429068 shutdown_connections 2024-03-30T00:40:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 --2- 172.21.15.66:0/3381429068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c40039d0 0x7f76c4003df0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 -- 172.21.15.66:0/3381429068 >> 172.21.15.66:0/3381429068 conn(0x7f76e006b070 msgr2=0x7f76e006b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 -- 172.21.15.66:0/3381429068 shutdown_connections 2024-03-30T00:40:12.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 -- 172.21.15.66:0/3381429068 wait complete. 2024-03-30T00:40:12.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 Processor -- start 2024-03-30T00:40:12.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 -- start start 2024-03-30T00:40:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c40039d0 0x7f76e014fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76e0150460 0x7f76e00c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76e0153960 0x7f76e0153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0154320 con 0x7f76e0150460 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00c7af0 con 0x7f76c40039d0 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76e4a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e015ac80 con 0x7f76e0153960 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76d5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76e0150460 0x7f76e00c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76d5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76e0150460 0x7f76e00c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56552/0 (socket says 172.21.15.66:56552) 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76d5e00640 1 -- 172.21.15.66:0/2301482043 learned_addr learned my addr 172.21.15.66:0/2301482043 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76de800640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c40039d0 0x7f76e014fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.041+0000 7f76df200640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76e0153960 0x7f76e0153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d5e00640 1 -- 172.21.15.66:0/2301482043 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76e0153960 msgr2=0x7f76e0153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d5e00640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76e0153960 0x7f76e0153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d5e00640 1 -- 172.21.15.66:0/2301482043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c40039d0 msgr2=0x7f76e014fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d5e00640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c40039d0 0x7f76e014fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d5e00640 1 -- 172.21.15.66:0/2301482043 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e00c6270 con 0x7f76e0150460 2024-03-30T00:40:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d5e00640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76e0150460 0x7f76e00c6c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f76c80101f0 tx=0x7f76c80106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d7400640 1 -- 172.21.15.66:0/2301482043 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c800e050 con 0x7f76e0150460 2024-03-30T00:40:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d7400640 1 -- 172.21.15.66:0/2301482043 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c800f040 con 0x7f76e0150460 2024-03-30T00:40:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d7400640 1 -- 172.21.15.66:0/2301482043 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8003bc0 con 0x7f76e0150460 2024-03-30T00:40:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76e4a00640 1 -- 172.21.15.66:0/2301482043 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e015b240 con 0x7f76e0150460 2024-03-30T00:40:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76e4a00640 1 -- 172.21.15.66:0/2301482043 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e00c7810 con 0x7f76e0150460 2024-03-30T00:40:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76e4a00640 1 -- 172.21.15.66:0/2301482043 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e015b3d0 con 0x7f76e0150460 2024-03-30T00:40:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d7400640 1 -- 172.21.15.66:0/2301482043 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f76c8003d20 con 0x7f76e0150460 2024-03-30T00:40:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d7400640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f76bc0666c0 0x7f76bc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76d7400640 1 -- 172.21.15.66:0/2301482043 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f76c80851a0 con 0x7f76e0150460 2024-03-30T00:40:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.045+0000 7f76de800640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f76bc0666c0 0x7f76bc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.049+0000 7f76de800640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f76bc0666c0 0x7f76bc068b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f76d00ab000 tx=0x7f76d00b1040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.049+0000 7f76d7400640 1 -- 172.21.15.66:0/2301482043 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f76c8052230 con 0x7f76e0150460 2024-03-30T00:40:12.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.213+0000 7f76e4a00640 1 -- 172.21.15.66:0/2301482043 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76e0073cf0 con 0x7f76e0150460 2024-03-30T00:40:12.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76d7400640 1 -- 172.21.15.66:0/2301482043 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f76c8055850 con 0x7f76e0150460 2024-03-30T00:40:12.225 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:12.225 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:12.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 -- 172.21.15.66:0/2301482043 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f76bc0666c0 msgr2=0x7f76bc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f76bc0666c0 0x7f76bc068b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f76d00ab000 tx=0x7f76d00b1040 comp rx=0 tx=0).stop 2024-03-30T00:40:12.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 -- 172.21.15.66:0/2301482043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76e0150460 msgr2=0x7f76e00c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76e0150460 0x7f76e00c6c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f76c80101f0 tx=0x7f76c80106b0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 -- 172.21.15.66:0/2301482043 shutdown_connections 2024-03-30T00:40:12.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76e0153960 0x7f76e0153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f76bc0666c0 0x7f76bc068b80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76e0150460 0x7f76e00c6c30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 --2- 172.21.15.66:0/2301482043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c40039d0 0x7f76e014fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 -- 172.21.15.66:0/2301482043 >> 172.21.15.66:0/2301482043 conn(0x7f76e006b070 msgr2=0x7f76e0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:12.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 -- 172.21.15.66:0/2301482043 shutdown_connections 2024-03-30T00:40:12.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.217+0000 7f76e4a00640 1 -- 172.21.15.66:0/2301482043 wait complete. 2024-03-30T00:40:12.247 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:12.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.377+0000 7f003ea00640 1 Processor -- start 2024-03-30T00:40:12.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.377+0000 7f003ea00640 1 -- start start 2024-03-30T00:40:12.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.377+0000 7f003ea00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f00380cad00 0x7f00380c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.377+0000 7f003ea00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00380c8e80 con 0x7f00380cd2d0 2024-03-30T00:40:12.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.377+0000 7f003ea00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00380ca660 con 0x7f00380cc8d0 2024-03-30T00:40:12.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.377+0000 7f003ea00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00380cbec0 con 0x7f00380cad00 2024-03-30T00:40:12.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.377+0000 7f002fe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f00380cd2d0 0x7f00380cb0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36322/0 (socket says 172.21.15.66:36322) 2024-03-30T00:40:12.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.377+0000 7f002fe00640 1 -- 172.21.15.66:0/3286359690 learned_addr learned my addr 172.21.15.66:0/3286359690 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:12.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037e00640 1 --2- 172.21.15.66:0/3286359690 >> v2:172.21.15.156:3300/0 conn(0x7f00380cad00 0x7f00380c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037400640 1 -- 172.21.15.66:0/3286359690 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199749699 0 0) 0x7f00380c8e80 con 0x7f00380cd2d0 2024-03-30T00:40:12.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037400640 1 -- 172.21.15.66:0/3286359690 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f001c003530 con 0x7f00380cd2d0 2024-03-30T00:40:12.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037400640 1 -- 172.21.15.66:0/3286359690 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114845991 0 0) 0x7f00380ca660 con 0x7f00380cc8d0 2024-03-30T00:40:12.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037400640 1 -- 172.21.15.66:0/3286359690 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00380c8e80 con 0x7f00380cc8d0 2024-03-30T00:40:12.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037e00640 1 -- 172.21.15.66:0/3286359690 >> v1:172.21.15.156:6789/0 conn(0x7f00380cc8d0 legacy=0x7f003806c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037e00640 1 -- 172.21.15.66:0/3286359690 >> v1:172.21.15.66:6789/0 conn(0x7f00380cd2d0 legacy=0x7f00380cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037400640 1 -- 172.21.15.66:0/3286359690 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3877390587 0 0) 0x7f001c003530 con 0x7f00380cd2d0 2024-03-30T00:40:12.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037e00640 1 -- 172.21.15.66:0/3286359690 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00380ca940 con 0x7f00380cad00 2024-03-30T00:40:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037e00640 1 --2- 172.21.15.66:0/3286359690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00380cad00 0x7f00380c8940 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f00280097c0 tx=0x7f002801c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45c302ff74e8ea83 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037400640 1 -- 172.21.15.66:0/3286359690 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020001680 con 0x7f00380cad00 2024-03-30T00:40:12.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037400640 1 -- 172.21.15.66:0/3286359690 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00280ac440 con 0x7f00380cad00 2024-03-30T00:40:12.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.381+0000 7f0037400640 1 -- 172.21.15.66:0/3286359690 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00280ac7b0 con 0x7f00380cad00 2024-03-30T00:40:12.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 -- 172.21.15.66:0/3286359690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00380cad00 msgr2=0x7f00380c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 --2- 172.21.15.66:0/3286359690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00380cad00 0x7f00380c8940 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f00280097c0 tx=0x7f002801c390 comp rx=0 tx=0).stop 2024-03-30T00:40:12.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 -- 172.21.15.66:0/3286359690 shutdown_connections 2024-03-30T00:40:12.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 --2- 172.21.15.66:0/3286359690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00380cad00 0x7f00380c8940 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 -- 172.21.15.66:0/3286359690 >> 172.21.15.66:0/3286359690 conn(0x7f0038069f60 msgr2=0x7f003806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:12.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 -- 172.21.15.66:0/3286359690 shutdown_connections 2024-03-30T00:40:12.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 -- 172.21.15.66:0/3286359690 wait complete. 2024-03-30T00:40:12.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 Processor -- start 2024-03-30T00:40:12.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 -- start start 2024-03-30T00:40:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00380cad00 0x7f0038151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00380cc8d0 0x7f00381573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00380cd2d0 0x7f0038157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0038097320 con 0x7f00380cd2d0 2024-03-30T00:40:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00380add00 con 0x7f00380cc8d0 2024-03-30T00:40:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ea00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0038159340 con 0x7f00380cad00 2024-03-30T00:40:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00380cd2d0 0x7f0038157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00380cd2d0 0x7f0038157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56566/0 (socket says 172.21.15.66:56566) 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ca00640 1 -- 172.21.15.66:0/2934272137 learned_addr learned my addr 172.21.15.66:0/2934272137 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f0037e00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00380cad00 0x7f0038151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f002fe00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00380cc8d0 0x7f00381573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ca00640 1 -- 172.21.15.66:0/2934272137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00380cad00 msgr2=0x7f0038151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ca00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00380cad00 0x7f0038151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ca00640 1 -- 172.21.15.66:0/2934272137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00380cc8d0 msgr2=0x7f00381573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ca00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00380cc8d0 0x7f00381573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f003ca00640 1 -- 172.21.15.66:0/2934272137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0028009420 con 0x7f00380cd2d0 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.385+0000 7f002fe00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00380cc8d0 0x7f00381573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.389+0000 7f003ca00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00380cd2d0 0x7f0038157910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f003000c790 tx=0x7f003000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.389+0000 7f0034c00640 1 -- 172.21.15.66:0/2934272137 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003000cea0 con 0x7f00380cd2d0 2024-03-30T00:40:12.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.389+0000 7f0034c00640 1 -- 172.21.15.66:0/2934272137 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0030003b80 con 0x7f00380cd2d0 2024-03-30T00:40:12.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.389+0000 7f0034c00640 1 -- 172.21.15.66:0/2934272137 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003000a850 con 0x7f00380cd2d0 2024-03-30T00:40:12.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.389+0000 7f003ea00640 1 -- 172.21.15.66:0/2934272137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00381597d0 con 0x7f00380cd2d0 2024-03-30T00:40:12.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.389+0000 7f003ea00640 1 -- 172.21.15.66:0/2934272137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003806a610 con 0x7f00380cd2d0 2024-03-30T00:40:12.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.389+0000 7f003ea00640 1 -- 172.21.15.66:0/2934272137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00381594b0 con 0x7f00380cd2d0 2024-03-30T00:40:12.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.389+0000 7f0034c00640 1 -- 172.21.15.66:0/2934272137 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f003000a9b0 con 0x7f00380cd2d0 2024-03-30T00:40:12.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.389+0000 7f0034c00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0014066690 0x7f0014068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.389+0000 7f0034c00640 1 -- 172.21.15.66:0/2934272137 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f00300842c0 con 0x7f00380cd2d0 2024-03-30T00:40:12.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.393+0000 7f0037e00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0014066690 0x7f0014068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.393+0000 7f0037e00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0014066690 0x7f0014068b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f00280ae000 tx=0x7f00280a6070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:12.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.397+0000 7f0034c00640 1 -- 172.21.15.66:0/2934272137 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0030051340 con 0x7f00380cd2d0 2024-03-30T00:40:12.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.557+0000 7f003ea00640 1 -- 172.21.15.66:0/2934272137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0038066c30 con 0x7f00380cd2d0 2024-03-30T00:40:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f0034c00640 1 -- 172.21.15.66:0/2934272137 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f0030054960 con 0x7f00380cd2d0 2024-03-30T00:40:12.570 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 -- 172.21.15.66:0/2934272137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0014066690 msgr2=0x7f0014068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0014066690 0x7f0014068b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f00280ae000 tx=0x7f00280a6070 comp rx=0 tx=0).stop 2024-03-30T00:40:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 -- 172.21.15.66:0/2934272137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00380cd2d0 msgr2=0x7f0038157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00380cd2d0 0x7f0038157910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f003000c790 tx=0x7f003000cc50 comp rx=0 tx=0).stop 2024-03-30T00:40:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 -- 172.21.15.66:0/2934272137 shutdown_connections 2024-03-30T00:40:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0014066690 0x7f0014068b50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00380cd2d0 0x7f0038157910 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00380cc8d0 0x7f00381573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 --2- 172.21.15.66:0/2934272137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00380cad00 0x7f0038151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 -- 172.21.15.66:0/2934272137 >> 172.21.15.66:0/2934272137 conn(0x7f0038069f60 msgr2=0x7f003806cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:12.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 -- 172.21.15.66:0/2934272137 shutdown_connections 2024-03-30T00:40:12.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.561+0000 7f003ea00640 1 -- 172.21.15.66:0/2934272137 wait complete. 2024-03-30T00:40:12.591 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T00:40:12.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.725+0000 7fbc90400640 1 Processor -- start 2024-03-30T00:40:12.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.725+0000 7fbc90400640 1 -- start start 2024-03-30T00:40:12.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.725+0000 7fbc90400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbc8806bb90 0x7fbc88098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.725+0000 7fbc90400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbc8806bff0 0x7fbc88098f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.725+0000 7fbc90400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc880b0940 con 0x7fbc880af790 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.725+0000 7fbc90400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8806b140 con 0x7fbc8806bff0 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.725+0000 7fbc90400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8806b700 con 0x7fbc8806bb90 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.725+0000 7fbc8e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbc8806bff0 0x7fbc88098f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.725+0000 7fbc8e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbc8806bff0 0x7fbc88098f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56580/0 (socket says 172.21.15.66:56580) 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.725+0000 7fbc8e400640 1 -- 172.21.15.66:0/309677474 learned_addr learned my addr 172.21.15.66:0/309677474 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.725+0000 7fbc8d000640 1 --2- 172.21.15.66:0/309677474 >> v2:172.21.15.156:3300/0 conn(0x7fbc8806bb90 0x7fbc88098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc8e400640 1 -- 172.21.15.66:0/309677474 >> v2:172.21.15.156:3300/0 conn(0x7fbc8806bb90 msgr2=0x7fbc88098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc8e400640 1 --2- 172.21.15.66:0/309677474 >> v2:172.21.15.156:3300/0 conn(0x7fbc8806bb90 0x7fbc88098a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc8e400640 1 -- 172.21.15.66:0/309677474 >> v1:172.21.15.156:6789/0 conn(0x7fbc880af790 legacy=0x7fbc88099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc8e400640 1 -- 172.21.15.66:0/309677474 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880b0b90 con 0x7fbc8806bff0 2024-03-30T00:40:12.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc8d000640 1 --2- 172.21.15.66:0/309677474 >> v2:172.21.15.156:3300/0 conn(0x7fbc8806bb90 0x7fbc88098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:12.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc8e400640 1 --2- 172.21.15.66:0/309677474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc8806bff0 0x7fbc88098f60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc880d05a0 tx=0x7fbc8001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ba972c414cea63 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:12.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc87e00640 1 -- 172.21.15.66:0/309677474 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800a5880 con 0x7fbc8806bff0 2024-03-30T00:40:12.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc87e00640 1 -- 172.21.15.66:0/309677474 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc800a59e0 con 0x7fbc8806bff0 2024-03-30T00:40:12.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc87e00640 1 -- 172.21.15.66:0/309677474 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800a5d70 con 0x7fbc8806bff0 2024-03-30T00:40:12.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc90400640 1 -- 172.21.15.66:0/309677474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc8806bff0 msgr2=0x7fbc88098f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc90400640 1 --2- 172.21.15.66:0/309677474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc8806bff0 0x7fbc88098f60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc880d05a0 tx=0x7fbc8001c9c0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc90400640 1 -- 172.21.15.66:0/309677474 shutdown_connections 2024-03-30T00:40:12.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc90400640 1 --2- 172.21.15.66:0/309677474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc8806bff0 0x7fbc88098f60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc90400640 1 --2- 172.21.15.66:0/309677474 >> v2:172.21.15.156:3300/0 conn(0x7fbc8806bb90 0x7fbc88098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc90400640 1 -- 172.21.15.66:0/309677474 >> 172.21.15.66:0/309677474 conn(0x7fbc880b20c0 msgr2=0x7fbc880b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:12.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc90400640 1 -- 172.21.15.66:0/309677474 shutdown_connections 2024-03-30T00:40:12.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.729+0000 7fbc90400640 1 -- 172.21.15.66:0/309677474 wait complete. 2024-03-30T00:40:12.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc90400640 1 Processor -- start 2024-03-30T00:40:12.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc90400640 1 -- start start 2024-03-30T00:40:12.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc90400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc8806bb90 0x7fbc8815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc90400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc8806bff0 0x7fbc88155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc90400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc880af790 0x7fbc880b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc90400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88155980 con 0x7fbc8806bb90 2024-03-30T00:40:12.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc90400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880b3730 con 0x7fbc880af790 2024-03-30T00:40:12.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc90400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880b39d0 con 0x7fbc8806bff0 2024-03-30T00:40:12.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc8806bb90 0x7fbc8815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc8806bb90 0x7fbc8815a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56596/0 (socket says 172.21.15.66:56596) 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8da00640 1 -- 172.21.15.66:0/3737238771 learned_addr learned my addr 172.21.15.66:0/3737238771 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8d000640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc8806bff0 0x7fbc88155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8e400640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc880af790 0x7fbc880b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8da00640 1 -- 172.21.15.66:0/3737238771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc8806bff0 msgr2=0x7fbc88155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8da00640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc8806bff0 0x7fbc88155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8da00640 1 -- 172.21.15.66:0/3737238771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc880af790 msgr2=0x7fbc880b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8da00640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc880af790 0x7fbc880b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8da00640 1 -- 172.21.15.66:0/3737238771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc80009b30 con 0x7fbc8806bb90 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8e400640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc880af790 0x7fbc880b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc8da00640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc8806bb90 0x7fbc8815a960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c3c0 tx=0x7fbc7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc85600640 1 -- 172.21.15.66:0/3737238771 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0052d0 con 0x7fbc8806bb90 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc85600640 1 -- 172.21.15.66:0/3737238771 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c005430 con 0x7fbc8806bb90 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc90400640 1 -- 172.21.15.66:0/3737238771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc880b3cb0 con 0x7fbc8806bb90 2024-03-30T00:40:12.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc85600640 1 -- 172.21.15.66:0/3737238771 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c003840 con 0x7fbc8806bb90 2024-03-30T00:40:12.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.733+0000 7fbc90400640 1 -- 172.21.15.66:0/3737238771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc880b4270 con 0x7fbc8806bb90 2024-03-30T00:40:12.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.737+0000 7fbc90400640 1 -- 172.21.15.66:0/3737238771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc88066c30 con 0x7fbc8806bb90 2024-03-30T00:40:12.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.737+0000 7fbc85600640 1 -- 172.21.15.66:0/3737238771 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbc7c0039a0 con 0x7fbc8806bb90 2024-03-30T00:40:12.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.737+0000 7fbc85600640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbc68066690 0x7fbc68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:12.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.737+0000 7fbc85600640 1 -- 172.21.15.66:0/3737238771 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbc7c0839a0 con 0x7fbc8806bb90 2024-03-30T00:40:12.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.737+0000 7fbc8d000640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbc68066690 0x7fbc68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:12.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.737+0000 7fbc8d000640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbc68066690 0x7fbc68068b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbc8815af90 tx=0x7fbc74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:12.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.741+0000 7fbc85600640 1 -- 172.21.15.66:0/3737238771 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbc7c050a30 con 0x7fbc8806bb90 2024-03-30T00:40:12.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:12.937+0000 7fbc90400640 1 -- 172.21.15.66:0/3737238771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbc880b2d20 con 0x7fbc8806bb90 2024-03-30T00:40:13.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.497+0000 7fbc85600640 1 -- 172.21.15.66:0/3737238771 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbc7c054050 con 0x7fbc8806bb90 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.497+0000 7fbc90400640 1 -- 172.21.15.66:0/3737238771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbc68066690 msgr2=0x7fbc68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.497+0000 7fbc90400640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbc68066690 0x7fbc68068b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbc8815af90 tx=0x7fbc74008040 comp rx=0 tx=0).stop 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.497+0000 7fbc90400640 1 -- 172.21.15.66:0/3737238771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc8806bb90 msgr2=0x7fbc8815a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.497+0000 7fbc90400640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc8806bb90 0x7fbc8815a960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c3c0 tx=0x7fbc7c00c880 comp rx=0 tx=0).stop 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.497+0000 7fbc90400640 1 -- 172.21.15.66:0/3737238771 shutdown_connections 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.497+0000 7fbc90400640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc880af790 0x7fbc880b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.497+0000 7fbc90400640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc8806bff0 0x7fbc88155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.497+0000 7fbc90400640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbc68066690 0x7fbc68068b50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.497+0000 7fbc90400640 1 --2- 172.21.15.66:0/3737238771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc8806bb90 0x7fbc8815a960 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.497+0000 7fbc90400640 1 -- 172.21.15.66:0/3737238771 >> 172.21.15.66:0/3737238771 conn(0x7fbc880b20c0 msgr2=0x7fbc880ad510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.501+0000 7fbc90400640 1 -- 172.21.15.66:0/3737238771 shutdown_connections 2024-03-30T00:40:13.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.501+0000 7fbc90400640 1 -- 172.21.15.66:0/3737238771 wait complete. 2024-03-30T00:40:13.530 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-30T00:40:13.530 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T00:40:13.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f367ba00640 1 Processor -- start 2024-03-30T00:40:13.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f367ba00640 1 -- start start 2024-03-30T00:40:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f367ba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36740af790 0x7f3674099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f367ba00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f367406bb90 0x7f3674098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f367ba00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36740b0940 con 0x7f367406bff0 2024-03-30T00:40:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f367ba00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f367406b140 con 0x7f367406bb90 2024-03-30T00:40:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f367ba00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367406b700 con 0x7f36740af790 2024-03-30T00:40:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f366be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f367406bb90 0x7f3674098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f366be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f367406bb90 0x7f3674098a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56600/0 (socket says 172.21.15.66:56600) 2024-03-30T00:40:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f366be00640 1 -- 172.21.15.66:0/3678543821 learned_addr learned my addr 172.21.15.66:0/3678543821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:13.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f3679000640 1 --2- 172.21.15.66:0/3678543821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36740af790 0x7f3674099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:13.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f366be00640 1 -- 172.21.15.66:0/3678543821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36740af790 msgr2=0x7f3674099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:13.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f366be00640 1 --2- 172.21.15.66:0/3678543821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36740af790 0x7f3674099760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:13.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f366be00640 1 -- 172.21.15.66:0/3678543821 >> v1:172.21.15.156:6789/0 conn(0x7f367406bff0 legacy=0x7f36740ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:13.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f366be00640 1 -- 172.21.15.66:0/3678543821 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36740b0b90 con 0x7f367406bb90 2024-03-30T00:40:13.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.665+0000 7f3679000640 1 --2- 172.21.15.66:0/3678543821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36740af790 0x7f3674099760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:13.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f366be00640 1 --2- 172.21.15.66:0/3678543821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f367406bb90 0x7f3674098a20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f365c009870 tx=0x7f365c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ef59f0a836ba34 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:13.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f3673e00640 1 -- 172.21.15.66:0/3678543821 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c0ac620 con 0x7f367406bb90 2024-03-30T00:40:13.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f3673e00640 1 -- 172.21.15.66:0/3678543821 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c0ac780 con 0x7f367406bb90 2024-03-30T00:40:13.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f3673e00640 1 -- 172.21.15.66:0/3678543821 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c0acb10 con 0x7f367406bb90 2024-03-30T00:40:13.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f367ba00640 1 -- 172.21.15.66:0/3678543821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f367406bb90 msgr2=0x7f3674098a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:13.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f367ba00640 1 --2- 172.21.15.66:0/3678543821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f367406bb90 0x7f3674098a20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f365c009870 tx=0x7f365c01c4f0 comp rx=0 tx=0).stop 2024-03-30T00:40:13.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f367ba00640 1 -- 172.21.15.66:0/3678543821 shutdown_connections 2024-03-30T00:40:13.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f367ba00640 1 --2- 172.21.15.66:0/3678543821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f367406bb90 0x7f3674098a20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:13.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f367ba00640 1 --2- 172.21.15.66:0/3678543821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36740af790 0x7f3674099760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:13.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f367ba00640 1 -- 172.21.15.66:0/3678543821 >> 172.21.15.66:0/3678543821 conn(0x7f36740b20c0 msgr2=0x7f36740b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:13.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f367ba00640 1 -- 172.21.15.66:0/3678543821 shutdown_connections 2024-03-30T00:40:13.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.669+0000 7f367ba00640 1 -- 172.21.15.66:0/3678543821 wait complete. 2024-03-30T00:40:13.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f367ba00640 1 Processor -- start 2024-03-30T00:40:13.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f367ba00640 1 -- start start 2024-03-30T00:40:13.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f367ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f367406bb90 0x7f367409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:13.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f367ba00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f367406bff0 0x7f367409ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:13.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f367ba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36740af790 0x7f36740ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:13.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f367ba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3674157ce0 con 0x7f367406bb90 2024-03-30T00:40:13.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f367ba00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3674143d70 con 0x7f367406bff0 2024-03-30T00:40:13.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f367ba00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3674157e80 con 0x7f36740af790 2024-03-30T00:40:13.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3679000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f367406bb90 0x7f367409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:13.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3679000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f367406bb90 0x7f367409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56604/0 (socket says 172.21.15.66:56604) 2024-03-30T00:40:13.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3679000640 1 -- 172.21.15.66:0/1114974399 learned_addr learned my addr 172.21.15.66:0/1114974399 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:13.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3679a00640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36740af790 0x7f36740ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:13.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f366be00640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f367406bff0 0x7f367409ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:13.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3679000640 1 -- 172.21.15.66:0/1114974399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36740af790 msgr2=0x7f36740ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:13.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3679000640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36740af790 0x7f36740ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:13.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3679000640 1 -- 172.21.15.66:0/1114974399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f367406bff0 msgr2=0x7f367409ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:13.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3679000640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f367406bff0 0x7f367409ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:13.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3679000640 1 -- 172.21.15.66:0/1114974399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0094d0 con 0x7f367406bb90 2024-03-30T00:40:13.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3679000640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f367406bb90 0x7f367409dac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f366400c7f0 tx=0x7f366400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:13.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3671600640 1 -- 172.21.15.66:0/1114974399 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664007820 con 0x7f367406bb90 2024-03-30T00:40:13.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3671600640 1 -- 172.21.15.66:0/1114974399 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3664007980 con 0x7f367406bb90 2024-03-30T00:40:13.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f3671600640 1 -- 172.21.15.66:0/1114974399 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664007d30 con 0x7f367406bb90 2024-03-30T00:40:13.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f367ba00640 1 -- 172.21.15.66:0/1114974399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367409b2a0 con 0x7f367406bb90 2024-03-30T00:40:13.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.673+0000 7f367ba00640 1 -- 172.21.15.66:0/1114974399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3674143fc0 con 0x7f367406bb90 2024-03-30T00:40:13.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.677+0000 7f367ba00640 1 -- 172.21.15.66:0/1114974399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3674066c30 con 0x7f367406bb90 2024-03-30T00:40:13.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.677+0000 7f3671600640 1 -- 172.21.15.66:0/1114974399 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f36640113f0 con 0x7f367406bb90 2024-03-30T00:40:13.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.677+0000 7f3671600640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3650066690 0x7f3650068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:13.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.677+0000 7f3671600640 1 -- 172.21.15.66:0/1114974399 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3664084760 con 0x7f367406bb90 2024-03-30T00:40:13.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.677+0000 7f366be00640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3650066690 0x7f3650068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:13.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.677+0000 7f366be00640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3650066690 0x7f3650068b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f365c0ae000 tx=0x7f365c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:13.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.681+0000 7f3671600640 1 -- 172.21.15.66:0/1114974399 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3664051880 con 0x7f367406bb90 2024-03-30T00:40:13.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:13.873+0000 7f367ba00640 1 -- 172.21.15.66:0/1114974399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3674099a80 con 0x7f367406bb90 2024-03-30T00:40:14.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.497+0000 7f3671600640 1 -- 172.21.15.66:0/1114974399 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3664054ea0 con 0x7f367406bb90 2024-03-30T00:40:14.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 -- 172.21.15.66:0/1114974399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3650066690 msgr2=0x7f3650068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:14.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3650066690 0x7f3650068b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f365c0ae000 tx=0x7f365c0a7000 comp rx=0 tx=0).stop 2024-03-30T00:40:14.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 -- 172.21.15.66:0/1114974399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f367406bb90 msgr2=0x7f367409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:14.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f367406bb90 0x7f367409dac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f366400c7f0 tx=0x7f366400ccb0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 -- 172.21.15.66:0/1114974399 shutdown_connections 2024-03-30T00:40:14.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36740af790 0x7f36740ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f367406bff0 0x7f367409ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3650066690 0x7f3650068b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 --2- 172.21.15.66:0/1114974399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f367406bb90 0x7f367409dac0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 -- 172.21.15.66:0/1114974399 >> 172.21.15.66:0/1114974399 conn(0x7f36740b20c0 msgr2=0x7f36740add60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:14.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 -- 172.21.15.66:0/1114974399 shutdown_connections 2024-03-30T00:40:14.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.501+0000 7f367ba00640 1 -- 172.21.15.66:0/1114974399 wait complete. 2024-03-30T00:40:14.535 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T00:40:14.535 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:40:14.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.669+0000 7fa802200640 1 Processor -- start 2024-03-30T00:40:14.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.669+0000 7fa802200640 1 -- start start 2024-03-30T00:40:14.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.669+0000 7fa802200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa7fc157060 0x7fa7fc06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:14.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.669+0000 7fa802200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7fc06dc60 con 0x7fa7fc160bc0 2024-03-30T00:40:14.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.669+0000 7fa802200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7fc06df40 con 0x7fa7fc1610c0 2024-03-30T00:40:14.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.669+0000 7fa802200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7fc157440 con 0x7fa7fc157060 2024-03-30T00:40:14.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.669+0000 7fa7fb400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa7fc157060 0x7fa7fc06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:14.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.669+0000 7fa7fb400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa7fc157060 0x7fa7fc06d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56620/0 (socket says 172.21.15.66:56620) 2024-03-30T00:40:14.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.669+0000 7fa7fb400640 1 -- 172.21.15.66:0/624069182 learned_addr learned my addr 172.21.15.66:0/624069182 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:14.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa7fb400640 1 -- 172.21.15.66:0/624069182 >> v1:172.21.15.156:6789/0 conn(0x7fa7fc1610c0 legacy=0x7fa7fc1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:14.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa7fb400640 1 -- 172.21.15.66:0/624069182 >> v1:172.21.15.66:6789/0 conn(0x7fa7fc160bc0 legacy=0x7fa7fc0abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:14.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa7fb400640 1 -- 172.21.15.66:0/624069182 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc06e220 con 0x7fa7fc157060 2024-03-30T00:40:14.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa7fb400640 1 --2- 172.21.15.66:0/624069182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc157060 0x7fa7fc06d310 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009870 tx=0x7fa7ec01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7e6c5699fd9c0b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa7faa00640 1 -- 172.21.15.66:0/624069182 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc06da60 con 0x7fa7fc157060 2024-03-30T00:40:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa7faa00640 1 -- 172.21.15.66:0/624069182 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec0ac640 con 0x7fa7fc157060 2024-03-30T00:40:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa7faa00640 1 -- 172.21.15.66:0/624069182 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0ac9d0 con 0x7fa7fc157060 2024-03-30T00:40:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa802200640 1 -- 172.21.15.66:0/624069182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc157060 msgr2=0x7fa7fc06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa802200640 1 --2- 172.21.15.66:0/624069182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc157060 0x7fa7fc06d310 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009870 tx=0x7fa7ec01c400 comp rx=0 tx=0).stop 2024-03-30T00:40:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa802200640 1 -- 172.21.15.66:0/624069182 shutdown_connections 2024-03-30T00:40:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa802200640 1 --2- 172.21.15.66:0/624069182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc157060 0x7fa7fc06d310 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa802200640 1 -- 172.21.15.66:0/624069182 >> 172.21.15.66:0/624069182 conn(0x7fa7fc071750 msgr2=0x7fa7fc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.673+0000 7fa802200640 1 -- 172.21.15.66:0/624069182 shutdown_connections 2024-03-30T00:40:14.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa802200640 1 -- 172.21.15.66:0/624069182 wait complete. 2024-03-30T00:40:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa802200640 1 Processor -- start 2024-03-30T00:40:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa802200640 1 -- start start 2024-03-30T00:40:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa802200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7fc157060 0x7fa7fc15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa802200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7fc160bc0 0x7fa7fc154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa802200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc1610c0 0x7fa7fc151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa802200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc15f2a0 con 0x7fa7fc1610c0 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa802200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc153ea0 con 0x7fa7fc157060 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa802200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc154180 con 0x7fa7fc160bc0 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7fbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc1610c0 0x7fa7fc151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7fbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc1610c0 0x7fa7fc151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56630/0 (socket says 172.21.15.66:56630) 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7fbe00640 1 -- 172.21.15.66:0/3921069331 learned_addr learned my addr 172.21.15.66:0/3921069331 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7f3e00640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7fc160bc0 0x7fa7fc154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7fb400640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7fc157060 0x7fa7fc15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7fbe00640 1 -- 172.21.15.66:0/3921069331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7fc160bc0 msgr2=0x7fa7fc154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7fbe00640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7fc160bc0 0x7fa7fc154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7fbe00640 1 -- 172.21.15.66:0/3921069331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7fc157060 msgr2=0x7fa7fc15fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7fbe00640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7fc157060 0x7fa7fc15fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7fbe00640 1 -- 172.21.15.66:0/3921069331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec0094d0 con 0x7fa7fc1610c0 2024-03-30T00:40:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7fb400640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7fc157060 0x7fa7fc15fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7fbe00640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc1610c0 0x7fa7fc151090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400c700 tx=0x7fa7f400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7f3400640 1 -- 172.21.15.66:0/3921069331 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4003b80 con 0x7fa7fc1610c0 2024-03-30T00:40:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7f3400640 1 -- 172.21.15.66:0/3921069331 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f4003ce0 con 0x7fa7fc1610c0 2024-03-30T00:40:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa802200640 1 -- 172.21.15.66:0/3921069331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7fc154460 con 0x7fa7fc1610c0 2024-03-30T00:40:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa7f3400640 1 -- 172.21.15.66:0/3921069331 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f400aa90 con 0x7fa7fc1610c0 2024-03-30T00:40:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.677+0000 7fa802200640 1 -- 172.21.15.66:0/3921069331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7fc154a20 con 0x7fa7fc1610c0 2024-03-30T00:40:14.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.681+0000 7fa802200640 1 -- 172.21.15.66:0/3921069331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7fc154c10 con 0x7fa7fc1610c0 2024-03-30T00:40:14.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.681+0000 7fa7f3400640 1 -- 172.21.15.66:0/3921069331 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa7f400abf0 con 0x7fa7fc1610c0 2024-03-30T00:40:14.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.681+0000 7fa7f3400640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7d8066690 0x7fa7d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:14.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.681+0000 7fa7f3400640 1 -- 172.21.15.66:0/3921069331 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa7f4084350 con 0x7fa7fc1610c0 2024-03-30T00:40:14.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.681+0000 7fa7fb400640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7d8066690 0x7fa7d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:14.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.681+0000 7fa7fb400640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7d8066690 0x7fa7d8068b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ae000 tx=0x7fa7ec0a6070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:14.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.685+0000 7fa7f3400640 1 -- 172.21.15.66:0/3921069331 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa7f4051330 con 0x7fa7fc1610c0 2024-03-30T00:40:14.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.881+0000 7fa802200640 1 -- 172.21.15.66:0/3921069331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7fc066c30 con 0x7fa7fc1610c0 2024-03-30T00:40:14.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.881+0000 7fa7f3400640 1 -- 172.21.15.66:0/3921069331 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4730 (secure 0 0 0) 0x7fa7f4054950 con 0x7fa7fc1610c0 2024-03-30T00:40:14.892 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:14.892 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4122431016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":4122431016},{"type":"v1","addr":"172.21.15.156:6835","nonce":4122431016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3882342907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3882342907},{"type":"v1","addr":"172.21.15.66:6835","nonce":3882342907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:39:51.142575+0000","modified":"2024-03-30T00:40:14.500142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/703895528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":703895528},{"type":"v1","addr":"172.21.15.66:6837","nonce":703895528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:40:14.893 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2024-03-30T00:40:14.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 -- 172.21.15.66:0/3921069331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7d8066690 msgr2=0x7fa7d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:14.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7d8066690 0x7fa7d8068b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ae000 tx=0x7fa7ec0a6070 comp rx=0 tx=0).stop 2024-03-30T00:40:14.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 -- 172.21.15.66:0/3921069331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc1610c0 msgr2=0x7fa7fc151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:14.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc1610c0 0x7fa7fc151090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400c700 tx=0x7fa7f400cbc0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 -- 172.21.15.66:0/3921069331 shutdown_connections 2024-03-30T00:40:14.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7d8066690 0x7fa7d8068b50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc1610c0 0x7fa7fc151090 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7fc160bc0 0x7fa7fc154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 --2- 172.21.15.66:0/3921069331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7fc157060 0x7fa7fc15fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:14.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 -- 172.21.15.66:0/3921069331 >> 172.21.15.66:0/3921069331 conn(0x7fa7fc071750 msgr2=0x7fa7fc15c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:14.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 -- 172.21.15.66:0/3921069331 shutdown_connections 2024-03-30T00:40:14.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:14.885+0000 7fa802200640 1 -- 172.21.15.66:0/3921069331 wait complete. 2024-03-30T00:40:14.913 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:15.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1249400640 1 Processor -- start 2024-03-30T00:40:15.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1249400640 1 -- start start 2024-03-30T00:40:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1249400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f12440ce880 0x7f12440ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1249400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1244069d90 0x7f124406a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1249400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12440aecf0 con 0x7f12440cb110 2024-03-30T00:40:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1249400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124406a760 con 0x7f1244069d90 2024-03-30T00:40:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1249400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12440aefd0 con 0x7f12440ce880 2024-03-30T00:40:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1243200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1244069d90 0x7f124406a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1243200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1244069d90 0x7f124406a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56632/0 (socket says 172.21.15.66:56632) 2024-03-30T00:40:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1243200640 1 -- 172.21.15.66:0/135877072 learned_addr learned my addr 172.21.15.66:0/135877072 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1241e00640 1 --2- 172.21.15.66:0/135877072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f12440ce880 0x7f12440ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1243200640 1 -- 172.21.15.66:0/135877072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f12440ce880 msgr2=0x7f12440ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1243200640 1 --2- 172.21.15.66:0/135877072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f12440ce880 0x7f12440ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1243200640 1 -- 172.21.15.66:0/135877072 >> v1:172.21.15.156:6789/0 conn(0x7f12440cb110 legacy=0x7f12440afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.045+0000 7f1243200640 1 -- 172.21.15.66:0/135877072 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12440af2b0 con 0x7f1244069d90 2024-03-30T00:40:15.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1243200640 1 --2- 172.21.15.66:0/135877072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1244069d90 0x7f124406a220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f123c009e50 tx=0x7f123c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5862656be658f67a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:15.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1241400640 1 -- 172.21.15.66:0/135877072 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c0a57f0 con 0x7f1244069d90 2024-03-30T00:40:15.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1241400640 1 -- 172.21.15.66:0/135877072 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c0a5950 con 0x7f1244069d90 2024-03-30T00:40:15.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1241400640 1 -- 172.21.15.66:0/135877072 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c0a5ce0 con 0x7f1244069d90 2024-03-30T00:40:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1249400640 1 -- 172.21.15.66:0/135877072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1244069d90 msgr2=0x7f124406a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1249400640 1 --2- 172.21.15.66:0/135877072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1244069d90 0x7f124406a220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f123c009e50 tx=0x7f123c01ca30 comp rx=0 tx=0).stop 2024-03-30T00:40:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1249400640 1 -- 172.21.15.66:0/135877072 shutdown_connections 2024-03-30T00:40:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1249400640 1 --2- 172.21.15.66:0/135877072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1244069d90 0x7f124406a220 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1249400640 1 --2- 172.21.15.66:0/135877072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f12440ce880 0x7f12440ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1249400640 1 -- 172.21.15.66:0/135877072 >> 172.21.15.66:0/135877072 conn(0x7f1244098a20 msgr2=0x7f12440cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1249400640 1 -- 172.21.15.66:0/135877072 shutdown_connections 2024-03-30T00:40:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1249400640 1 -- 172.21.15.66:0/135877072 wait complete. 2024-03-30T00:40:15.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1249400640 1 Processor -- start 2024-03-30T00:40:15.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.049+0000 7f1249400640 1 -- start start 2024-03-30T00:40:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1249400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1244069d90 0x7f1244159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1249400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f12440cb110 0x7f12440cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1249400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12440ce880 0x7f12440b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1249400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12440c5500 con 0x7f12440ce880 2024-03-30T00:40:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1249400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12440c5ca0 con 0x7f1244069d90 2024-03-30T00:40:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1249400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12440c3510 con 0x7f12440cb110 2024-03-30T00:40:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1243200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12440ce880 0x7f12440b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1243200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12440ce880 0x7f12440b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56648/0 (socket says 172.21.15.66:56648) 2024-03-30T00:40:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1243200640 1 -- 172.21.15.66:0/550329385 learned_addr learned my addr 172.21.15.66:0/550329385 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1241e00640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f12440cb110 0x7f12440cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1242800640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1244069d90 0x7f1244159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1243200640 1 -- 172.21.15.66:0/550329385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f12440cb110 msgr2=0x7f12440cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1243200640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f12440cb110 0x7f12440cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1243200640 1 -- 172.21.15.66:0/550329385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1244069d90 msgr2=0x7f1244159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1243200640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1244069d90 0x7f1244159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1243200640 1 -- 172.21.15.66:0/550329385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c009b30 con 0x7f12440ce880 2024-03-30T00:40:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1241e00640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f12440cb110 0x7f12440cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1243200640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12440ce880 0x7f12440b31f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f123c0af040 tx=0x7f123c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f123aa00640 1 -- 172.21.15.66:0/550329385 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c0b8670 con 0x7f12440ce880 2024-03-30T00:40:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f123aa00640 1 -- 172.21.15.66:0/550329385 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c0b87d0 con 0x7f12440ce880 2024-03-30T00:40:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f123aa00640 1 -- 172.21.15.66:0/550329385 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c0b8b80 con 0x7f12440ce880 2024-03-30T00:40:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1249400640 1 -- 172.21.15.66:0/550329385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12440c56e0 con 0x7f12440ce880 2024-03-30T00:40:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1249400640 1 -- 172.21.15.66:0/550329385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12440c31a0 con 0x7f12440ce880 2024-03-30T00:40:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f1249400640 1 -- 172.21.15.66:0/550329385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12440c3000 con 0x7f12440ce880 2024-03-30T00:40:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.053+0000 7f123aa00640 1 -- 172.21.15.66:0/550329385 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f123c0b8ce0 con 0x7f12440ce880 2024-03-30T00:40:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.057+0000 7f123aa00640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1224066690 0x7f1224068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.057+0000 7f123aa00640 1 -- 172.21.15.66:0/550329385 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f123c11fa70 con 0x7f12440ce880 2024-03-30T00:40:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.057+0000 7f1242800640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1224066690 0x7f1224068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.057+0000 7f1242800640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1224066690 0x7f1224068b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f12440ccda0 tx=0x7f1234009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.061+0000 7f123aa00640 1 -- 172.21.15.66:0/550329385 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f12440c3000 con 0x7f12440ce880 2024-03-30T00:40:15.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.225+0000 7f1249400640 1 -- 172.21.15.66:0/550329385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12440c3000 con 0x7f12440ce880 2024-03-30T00:40:15.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.225+0000 7f123aa00640 1 -- 172.21.15.66:0/550329385 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f12440c3000 con 0x7f12440ce880 2024-03-30T00:40:15.236 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:15.236 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 -- 172.21.15.66:0/550329385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1224066690 msgr2=0x7f1224068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1224066690 0x7f1224068b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f12440ccda0 tx=0x7f1234009040 comp rx=0 tx=0).stop 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 -- 172.21.15.66:0/550329385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12440ce880 msgr2=0x7f12440b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12440ce880 0x7f12440b31f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f123c0af040 tx=0x7f123c0197d0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 -- 172.21.15.66:0/550329385 shutdown_connections 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1224066690 0x7f1224068b50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12440ce880 0x7f12440b31f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f12440cb110 0x7f12440cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 --2- 172.21.15.66:0/550329385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1244069d90 0x7f1244159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 -- 172.21.15.66:0/550329385 >> 172.21.15.66:0/550329385 conn(0x7f1244098a20 msgr2=0x7f124415fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 -- 172.21.15.66:0/550329385 shutdown_connections 2024-03-30T00:40:15.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.229+0000 7f1249400640 1 -- 172.21.15.66:0/550329385 wait complete. 2024-03-30T00:40:15.257 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T00:40:15.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:40:15.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565ec00640 1 Processor -- start 2024-03-30T00:40:15.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565ec00640 1 -- start start 2024-03-30T00:40:15.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565ec00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f56580cee50 0x7f56580cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565ec00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f56580af420 0x7f56580d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565ec00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56580cbb90 con 0x7f565815de30 2024-03-30T00:40:15.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565ec00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56580ce1b0 con 0x7f56580af420 2024-03-30T00:40:15.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565ec00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56580afa10 con 0x7f56580cee50 2024-03-30T00:40:15.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565cc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f56580af420 0x7f56580d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565cc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f56580af420 0x7f56580d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56652/0 (socket says 172.21.15.66:56652) 2024-03-30T00:40:15.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565cc00640 1 -- 172.21.15.66:0/3587178424 learned_addr learned my addr 172.21.15.66:0/3587178424 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:15.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f5657400640 1 --2- 172.21.15.66:0/3587178424 >> v2:172.21.15.156:3300/0 conn(0x7f56580cee50 0x7f56580cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565cc00640 1 -- 172.21.15.66:0/3587178424 >> v2:172.21.15.156:3300/0 conn(0x7f56580cee50 msgr2=0x7f56580cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:15.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565cc00640 1 --2- 172.21.15.66:0/3587178424 >> v2:172.21.15.156:3300/0 conn(0x7f56580cee50 0x7f56580cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565cc00640 1 -- 172.21.15.66:0/3587178424 >> v1:172.21.15.156:6789/0 conn(0x7f565815de30 legacy=0x7f56580ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.389+0000 7f565cc00640 1 -- 172.21.15.66:0/3587178424 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56580cf430 con 0x7f56580af420 2024-03-30T00:40:15.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f565cc00640 1 --2- 172.21.15.66:0/3587178424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56580af420 0x7f56580d1060 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5658070ad0 tx=0x7f565001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e4a46e8c3f6ad1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:15.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f5656a00640 1 -- 172.21.15.66:0/3587178424 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56500a5820 con 0x7f56580af420 2024-03-30T00:40:15.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f5656a00640 1 -- 172.21.15.66:0/3587178424 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56500a5980 con 0x7f56580af420 2024-03-30T00:40:15.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f5656a00640 1 -- 172.21.15.66:0/3587178424 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56500a5d10 con 0x7f56580af420 2024-03-30T00:40:15.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f565ec00640 1 -- 172.21.15.66:0/3587178424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56580af420 msgr2=0x7f56580d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f565ec00640 1 --2- 172.21.15.66:0/3587178424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56580af420 0x7f56580d1060 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5658070ad0 tx=0x7f565001c9f0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f565ec00640 1 -- 172.21.15.66:0/3587178424 shutdown_connections 2024-03-30T00:40:15.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f565ec00640 1 --2- 172.21.15.66:0/3587178424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56580af420 0x7f56580d1060 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f565ec00640 1 --2- 172.21.15.66:0/3587178424 >> v2:172.21.15.156:3300/0 conn(0x7f56580cee50 0x7f56580cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f565ec00640 1 -- 172.21.15.66:0/3587178424 >> 172.21.15.66:0/3587178424 conn(0x7f56580d0540 msgr2=0x7f56580d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:15.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f565ec00640 1 -- 172.21.15.66:0/3587178424 shutdown_connections 2024-03-30T00:40:15.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f565ec00640 1 -- 172.21.15.66:0/3587178424 wait complete. 2024-03-30T00:40:15.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f565ec00640 1 Processor -- start 2024-03-30T00:40:15.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.393+0000 7f565ec00640 1 -- start start 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565ec00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56580af420 0x7f5658154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565ec00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56580cee50 0x7f5658162830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f565815de30 0x7f565806aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56580c8170 con 0x7f565815de30 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565ec00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56580c8710 con 0x7f56580cee50 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565ec00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565806be40 con 0x7f56580af420 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f565815de30 0x7f565806aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f565815de30 0x7f565806aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56658/0 (socket says 172.21.15.66:56658) 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565cc00640 1 -- 172.21.15.66:0/707914474 learned_addr learned my addr 172.21.15.66:0/707914474 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f5657e00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56580af420 0x7f5658154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f5657400640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56580cee50 0x7f5658162830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565cc00640 1 -- 172.21.15.66:0/707914474 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56580af420 msgr2=0x7f5658154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565cc00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56580af420 0x7f5658154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565cc00640 1 -- 172.21.15.66:0/707914474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56580cee50 msgr2=0x7f5658162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565cc00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56580cee50 0x7f5658162830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565cc00640 1 -- 172.21.15.66:0/707914474 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5650009b30 con 0x7f565815de30 2024-03-30T00:40:15.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565cc00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f565815de30 0x7f565806aea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5650019600 tx=0x7f5650019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:15.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f564fe00640 1 -- 172.21.15.66:0/707914474 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56500b8670 con 0x7f565815de30 2024-03-30T00:40:15.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f564fe00640 1 -- 172.21.15.66:0/707914474 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56500b87d0 con 0x7f565815de30 2024-03-30T00:40:15.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f564fe00640 1 -- 172.21.15.66:0/707914474 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56500b8b80 con 0x7f565815de30 2024-03-30T00:40:15.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565ec00640 1 -- 172.21.15.66:0/707914474 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565806b880 con 0x7f565815de30 2024-03-30T00:40:15.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565ec00640 1 -- 172.21.15.66:0/707914474 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56580c83a0 con 0x7f565815de30 2024-03-30T00:40:15.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f565ec00640 1 -- 172.21.15.66:0/707914474 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56580cdf60 con 0x7f565815de30 2024-03-30T00:40:15.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.397+0000 7f564fe00640 1 -- 172.21.15.66:0/707914474 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f56500b8ce0 con 0x7f565815de30 2024-03-30T00:40:15.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.401+0000 7f564fe00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56380666e0 0x7f5638068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.401+0000 7f564fe00640 1 -- 172.21.15.66:0/707914474 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f565011fc60 con 0x7f565815de30 2024-03-30T00:40:15.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.401+0000 7f5657e00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56380666e0 0x7f5638068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.401+0000 7f5657e00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56380666e0 0x7f5638068ba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5648000f70 tx=0x7f5648009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:15.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.405+0000 7f564fe00640 1 -- 172.21.15.66:0/707914474 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f56500ece00 con 0x7f565815de30 2024-03-30T00:40:15.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.565+0000 7f565ec00640 1 -- 172.21.15.66:0/707914474 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56580c8880 con 0x7f56380666e0 2024-03-30T00:40:15.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.573+0000 7f564fe00640 1 -- 172.21.15.66:0/707914474 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345302 (secure 0 0 0) 0x7f56580c8880 con 0x7f56380666e0 2024-03-30T00:40:15.589 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:15.591 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:40:15.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.577+0000 7f565ec00640 1 -- 172.21.15.66:0/707914474 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56380666e0 msgr2=0x7f5638068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.577+0000 7f565ec00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56380666e0 0x7f5638068ba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5648000f70 tx=0x7f5648009040 comp rx=0 tx=0).stop 2024-03-30T00:40:15.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.577+0000 7f565ec00640 1 -- 172.21.15.66:0/707914474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f565815de30 msgr2=0x7f565806aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.577+0000 7f565ec00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f565815de30 0x7f565806aea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5650019600 tx=0x7f5650019790 comp rx=0 tx=0).stop 2024-03-30T00:40:15.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.581+0000 7f565ec00640 1 -- 172.21.15.66:0/707914474 shutdown_connections 2024-03-30T00:40:15.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.581+0000 7f565ec00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56380666e0 0x7f5638068ba0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.581+0000 7f565ec00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f565815de30 0x7f565806aea0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.581+0000 7f565ec00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56580cee50 0x7f5658162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.581+0000 7f565ec00640 1 --2- 172.21.15.66:0/707914474 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56580af420 0x7f5658154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.581+0000 7f565ec00640 1 -- 172.21.15.66:0/707914474 >> 172.21.15.66:0/707914474 conn(0x7f56580d0540 msgr2=0x7f56580cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:15.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.581+0000 7f565ec00640 1 -- 172.21.15.66:0/707914474 shutdown_connections 2024-03-30T00:40:15.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.581+0000 7f565ec00640 1 -- 172.21.15.66:0/707914474 wait complete. 2024-03-30T00:40:15.608 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-03-30T00:40:14.855664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235864,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738856,"statfs":{"total":773094113280,"available":772852588544,"internally_reserved":0,"allocated":3383296,"data_stored":2025532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002921"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184484+0000","last_change":"2024-03-30T00:39:50.768525+0000","last_active":"2024-03-30T00:39:55.184484+0000","last_peered":"2024-03-30T00:39:55.184484+0000","last_clean":"2024-03-30T00:39:55.184484+0000","last_became_active":"2024-03-30T00:39:50.768196+0000","last_became_peered":"2024-03-30T00:39:50.768196+0000","last_unstale":"2024-03-30T00:39:55.184484+0000","last_undegraded":"2024-03-30T00:39:55.184484+0000","last_fullsized":"2024-03-30T00:39:55.184484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:26.877416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168811+0000","last_change":"2024-03-30T00:39:51.184250+0000","last_active":"2024-03-30T00:39:55.168811+0000","last_peered":"2024-03-30T00:39:55.168811+0000","last_clean":"2024-03-30T00:39:55.168811+0000","last_became_active":"2024-03-30T00:39:51.184025+0000","last_became_peered":"2024-03-30T00:39:51.184025+0000","last_unstale":"2024-03-30T00:39:55.168811+0000","last_undegraded":"2024-03-30T00:39:55.168811+0000","last_fullsized":"2024-03-30T00:39:55.168811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:49.107527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571479+0000","last_change":"2024-03-30T00:39:55.177962+0000","last_active":"2024-03-30T00:39:55.571479+0000","last_peered":"2024-03-30T00:39:55.571479+0000","last_clean":"2024-03-30T00:39:55.571479+0000","last_became_active":"2024-03-30T00:39:55.177403+0000","last_became_peered":"2024-03-30T00:39:55.177403+0000","last_unstale":"2024-03-30T00:39:55.571479+0000","last_undegraded":"2024-03-30T00:39:55.571479+0000","last_fullsized":"2024-03-30T00:39:55.571479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:44:30.994000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168955+0000","last_change":"2024-03-30T00:39:50.765754+0000","last_active":"2024-03-30T00:39:55.168955+0000","last_peered":"2024-03-30T00:39:55.168955+0000","last_clean":"2024-03-30T00:39:55.168955+0000","last_became_active":"2024-03-30T00:39:50.764611+0000","last_became_peered":"2024-03-30T00:39:50.764611+0000","last_unstale":"2024-03-30T00:39:55.168955+0000","last_undegraded":"2024-03-30T00:39:55.168955+0000","last_fullsized":"2024-03-30T00:39:55.168955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:20.857915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184332+0000","last_change":"2024-03-30T00:39:51.180718+0000","last_active":"2024-03-30T00:39:55.184332+0000","last_peered":"2024-03-30T00:39:55.184332+0000","last_clean":"2024-03-30T00:39:55.184332+0000","last_became_active":"2024-03-30T00:39:51.175872+0000","last_became_peered":"2024-03-30T00:39:51.175872+0000","last_unstale":"2024-03-30T00:39:55.184332+0000","last_undegraded":"2024-03-30T00:39:55.184332+0000","last_fullsized":"2024-03-30T00:39:55.184332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272688+0000","last_change":"2024-03-30T00:39:55.184652+0000","last_active":"2024-03-30T00:39:57.272688+0000","last_peered":"2024-03-30T00:39:57.272688+0000","last_clean":"2024-03-30T00:39:57.272688+0000","last_became_active":"2024-03-30T00:39:55.184290+0000","last_became_peered":"2024-03-30T00:39:55.184290+0000","last_unstale":"2024-03-30T00:39:57.272688+0000","last_undegraded":"2024-03-30T00:39:57.272688+0000","last_fullsized":"2024-03-30T00:39:57.272688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:15:03.826593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163001+0000","last_change":"2024-03-30T00:39:50.771435+0000","last_active":"2024-03-30T00:39:55.163001+0000","last_peered":"2024-03-30T00:39:55.163001+0000","last_clean":"2024-03-30T00:39:55.163001+0000","last_became_active":"2024-03-30T00:39:50.771153+0000","last_became_peered":"2024-03-30T00:39:50.771153+0000","last_unstale":"2024-03-30T00:39:55.163001+0000","last_undegraded":"2024-03-30T00:39:55.163001+0000","last_fullsized":"2024-03-30T00:39:55.163001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:43.959952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170301+0000","last_change":"2024-03-30T00:39:51.183510+0000","last_active":"2024-03-30T00:39:55.170301+0000","last_peered":"2024-03-30T00:39:55.170301+0000","last_clean":"2024-03-30T00:39:55.170301+0000","last_became_active":"2024-03-30T00:39:51.183236+0000","last_became_peered":"2024-03-30T00:39:51.183236+0000","last_unstale":"2024-03-30T00:39:55.170301+0000","last_undegraded":"2024-03-30T00:39:55.170301+0000","last_fullsized":"2024-03-30T00:39:55.170301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273177+0000","last_change":"2024-03-30T00:39:55.179167+0000","last_active":"2024-03-30T00:39:57.273177+0000","last_peered":"2024-03-30T00:39:57.273177+0000","last_clean":"2024-03-30T00:39:57.273177+0000","last_became_active":"2024-03-30T00:39:55.178708+0000","last_became_peered":"2024-03-30T00:39:55.178708+0000","last_unstale":"2024-03-30T00:39:57.273177+0000","last_undegraded":"2024-03-30T00:39:57.273177+0000","last_fullsized":"2024-03-30T00:39:57.273177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161291+0000","last_change":"2024-03-30T00:39:50.773829+0000","last_active":"2024-03-30T00:39:55.161291+0000","last_peered":"2024-03-30T00:39:55.161291+0000","last_clean":"2024-03-30T00:39:55.161291+0000","last_became_active":"2024-03-30T00:39:50.773527+0000","last_became_peered":"2024-03-30T00:39:50.773527+0000","last_unstale":"2024-03-30T00:39:55.161291+0000","last_undegraded":"2024-03-30T00:39:55.161291+0000","last_fullsized":"2024-03-30T00:39:55.161291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.179506+0000","last_change":"2024-03-30T00:39:51.183689+0000","last_active":"2024-03-30T00:39:55.179506+0000","last_peered":"2024-03-30T00:39:55.179506+0000","last_clean":"2024-03-30T00:39:55.179506+0000","last_became_active":"2024-03-30T00:39:51.183421+0000","last_became_peered":"2024-03-30T00:39:51.183421+0000","last_unstale":"2024-03-30T00:39:55.179506+0000","last_undegraded":"2024-03-30T00:39:55.179506+0000","last_fullsized":"2024-03-30T00:39:55.179506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:54.542537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272671+0000","last_change":"2024-03-30T00:39:55.186210+0000","last_active":"2024-03-30T00:39:57.272671+0000","last_peered":"2024-03-30T00:39:57.272671+0000","last_clean":"2024-03-30T00:39:57.272671+0000","last_became_active":"2024-03-30T00:39:55.185849+0000","last_became_peered":"2024-03-30T00:39:55.185849+0000","last_unstale":"2024-03-30T00:39:57.272671+0000","last_undegraded":"2024-03-30T00:39:57.272671+0000","last_fullsized":"2024-03-30T00:39:57.272671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:45:20.370665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171140+0000","last_change":"2024-03-30T00:39:51.172454+0000","last_active":"2024-03-30T00:39:55.171140+0000","last_peered":"2024-03-30T00:39:55.171140+0000","last_clean":"2024-03-30T00:39:55.171140+0000","last_became_active":"2024-03-30T00:39:51.172196+0000","last_became_peered":"2024-03-30T00:39:51.172196+0000","last_unstale":"2024-03-30T00:39:55.171140+0000","last_undegraded":"2024-03-30T00:39:55.171140+0000","last_fullsized":"2024-03-30T00:39:55.171140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:02.372510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161442+0000","last_change":"2024-03-30T00:39:50.764668+0000","last_active":"2024-03-30T00:39:55.161442+0000","last_peered":"2024-03-30T00:39:55.161442+0000","last_clean":"2024-03-30T00:39:55.161442+0000","last_became_active":"2024-03-30T00:39:50.763661+0000","last_became_peered":"2024-03-30T00:39:50.763661+0000","last_unstale":"2024-03-30T00:39:55.161442+0000","last_undegraded":"2024-03-30T00:39:55.161442+0000","last_fullsized":"2024-03-30T00:39:55.161442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273301+0000","last_change":"2024-03-30T00:39:55.190348+0000","last_active":"2024-03-30T00:39:57.273301+0000","last_peered":"2024-03-30T00:39:57.273301+0000","last_clean":"2024-03-30T00:39:57.273301+0000","last_became_active":"2024-03-30T00:39:55.189987+0000","last_became_peered":"2024-03-30T00:39:55.189987+0000","last_unstale":"2024-03-30T00:39:57.273301+0000","last_undegraded":"2024-03-30T00:39:57.273301+0000","last_fullsized":"2024-03-30T00:39:57.273301+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:15:12.303669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169221+0000","last_change":"2024-03-30T00:39:51.168741+0000","last_active":"2024-03-30T00:39:55.169221+0000","last_peered":"2024-03-30T00:39:55.169221+0000","last_clean":"2024-03-30T00:39:55.169221+0000","last_became_active":"2024-03-30T00:39:51.168476+0000","last_became_peered":"2024-03-30T00:39:51.168476+0000","last_unstale":"2024-03-30T00:39:55.169221+0000","last_undegraded":"2024-03-30T00:39:55.169221+0000","last_fullsized":"2024-03-30T00:39:55.169221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:10:20.544219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172219+0000","last_change":"2024-03-30T00:39:50.766156+0000","last_active":"2024-03-30T00:39:55.172219+0000","last_peered":"2024-03-30T00:39:55.172219+0000","last_clean":"2024-03-30T00:39:55.172219+0000","last_became_active":"2024-03-30T00:39:50.765730+0000","last_became_peered":"2024-03-30T00:39:50.765730+0000","last_unstale":"2024-03-30T00:39:55.172219+0000","last_undegraded":"2024-03-30T00:39:55.172219+0000","last_fullsized":"2024-03-30T00:39:55.172219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:27.304749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272886+0000","last_change":"2024-03-30T00:39:55.180712+0000","last_active":"2024-03-30T00:39:57.272886+0000","last_peered":"2024-03-30T00:39:57.272886+0000","last_clean":"2024-03-30T00:39:57.272886+0000","last_became_active":"2024-03-30T00:39:55.179741+0000","last_became_peered":"2024-03-30T00:39:55.179741+0000","last_unstale":"2024-03-30T00:39:57.272886+0000","last_undegraded":"2024-03-30T00:39:57.272886+0000","last_fullsized":"2024-03-30T00:39:57.272886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.166309+0000","last_change":"2024-03-30T00:39:51.182604+0000","last_active":"2024-03-30T00:39:55.166309+0000","last_peered":"2024-03-30T00:39:55.166309+0000","last_clean":"2024-03-30T00:39:55.166309+0000","last_became_active":"2024-03-30T00:39:51.182323+0000","last_became_peered":"2024-03-30T00:39:51.182323+0000","last_unstale":"2024-03-30T00:39:55.166309+0000","last_undegraded":"2024-03-30T00:39:55.166309+0000","last_fullsized":"2024-03-30T00:39:55.166309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172036+0000","last_change":"2024-03-30T00:39:50.758774+0000","last_active":"2024-03-30T00:39:55.172036+0000","last_peered":"2024-03-30T00:39:55.172036+0000","last_clean":"2024-03-30T00:39:55.172036+0000","last_became_active":"2024-03-30T00:39:50.758508+0000","last_became_peered":"2024-03-30T00:39:50.758508+0000","last_unstale":"2024-03-30T00:39:55.172036+0000","last_undegraded":"2024-03-30T00:39:55.172036+0000","last_fullsized":"2024-03-30T00:39:55.172036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.129001+0000","last_change":"2024-03-30T00:39:55.182422+0000","last_active":"2024-03-30T00:39:57.129001+0000","last_peered":"2024-03-30T00:39:57.129001+0000","last_clean":"2024-03-30T00:39:57.129001+0000","last_became_active":"2024-03-30T00:39:55.182026+0000","last_became_peered":"2024-03-30T00:39:55.182026+0000","last_unstale":"2024-03-30T00:39:57.129001+0000","last_undegraded":"2024-03-30T00:39:57.129001+0000","last_fullsized":"2024-03-30T00:39:57.129001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:18:37.750989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160976+0000","last_change":"2024-03-30T00:39:51.161273+0000","last_active":"2024-03-30T00:39:55.160976+0000","last_peered":"2024-03-30T00:39:55.160976+0000","last_clean":"2024-03-30T00:39:55.160976+0000","last_became_active":"2024-03-30T00:39:51.156256+0000","last_became_peered":"2024-03-30T00:39:51.156256+0000","last_unstale":"2024-03-30T00:39:55.160976+0000","last_undegraded":"2024-03-30T00:39:55.160976+0000","last_fullsized":"2024-03-30T00:39:55.160976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:17.102921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159893+0000","last_change":"2024-03-30T00:39:50.768022+0000","last_active":"2024-03-30T00:39:55.159893+0000","last_peered":"2024-03-30T00:39:55.159893+0000","last_clean":"2024-03-30T00:39:55.159893+0000","last_became_active":"2024-03-30T00:39:50.767791+0000","last_became_peered":"2024-03-30T00:39:50.767791+0000","last_unstale":"2024-03-30T00:39:55.159893+0000","last_undegraded":"2024-03-30T00:39:55.159893+0000","last_fullsized":"2024-03-30T00:39:55.159893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:16.100201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273213+0000","last_change":"2024-03-30T00:39:55.176378+0000","last_active":"2024-03-30T00:39:57.273213+0000","last_peered":"2024-03-30T00:39:57.273213+0000","last_clean":"2024-03-30T00:39:57.273213+0000","last_became_active":"2024-03-30T00:39:55.176090+0000","last_became_peered":"2024-03-30T00:39:55.176090+0000","last_unstale":"2024-03-30T00:39:57.273213+0000","last_undegraded":"2024-03-30T00:39:57.273213+0000","last_fullsized":"2024-03-30T00:39:57.273213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184191+0000","last_change":"2024-03-30T00:39:51.181030+0000","last_active":"2024-03-30T00:39:55.184191+0000","last_peered":"2024-03-30T00:39:55.184191+0000","last_clean":"2024-03-30T00:39:55.184191+0000","last_became_active":"2024-03-30T00:39:51.180794+0000","last_became_peered":"2024-03-30T00:39:51.180794+0000","last_unstale":"2024-03-30T00:39:55.184191+0000","last_undegraded":"2024-03-30T00:39:55.184191+0000","last_fullsized":"2024-03-30T00:39:55.184191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:02.372510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172255+0000","last_change":"2024-03-30T00:39:50.771845+0000","last_active":"2024-03-30T00:39:55.172255+0000","last_peered":"2024-03-30T00:39:55.172255+0000","last_clean":"2024-03-30T00:39:55.172255+0000","last_became_active":"2024-03-30T00:39:50.771049+0000","last_became_peered":"2024-03-30T00:39:50.771049+0000","last_unstale":"2024-03-30T00:39:55.172255+0000","last_undegraded":"2024-03-30T00:39:55.172255+0000","last_fullsized":"2024-03-30T00:39:55.172255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:20.857915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272571+0000","last_change":"2024-03-30T00:39:55.174666+0000","last_active":"2024-03-30T00:39:57.272571+0000","last_peered":"2024-03-30T00:39:57.272571+0000","last_clean":"2024-03-30T00:39:57.272571+0000","last_became_active":"2024-03-30T00:39:55.174259+0000","last_became_peered":"2024-03-30T00:39:55.174259+0000","last_unstale":"2024-03-30T00:39:57.272571+0000","last_undegraded":"2024-03-30T00:39:57.272571+0000","last_fullsized":"2024-03-30T00:39:57.272571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165004+0000","last_change":"2024-03-30T00:39:51.178352+0000","last_active":"2024-03-30T00:39:55.165004+0000","last_peered":"2024-03-30T00:39:55.165004+0000","last_clean":"2024-03-30T00:39:55.165004+0000","last_became_active":"2024-03-30T00:39:51.178080+0000","last_became_peered":"2024-03-30T00:39:51.178080+0000","last_unstale":"2024-03-30T00:39:55.165004+0000","last_undegraded":"2024-03-30T00:39:55.165004+0000","last_fullsized":"2024-03-30T00:39:55.165004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172086+0000","last_change":"2024-03-30T00:39:50.768206+0000","last_active":"2024-03-30T00:39:55.172086+0000","last_peered":"2024-03-30T00:39:55.172086+0000","last_clean":"2024-03-30T00:39:55.172086+0000","last_became_active":"2024-03-30T00:39:50.767891+0000","last_became_peered":"2024-03-30T00:39:50.767891+0000","last_unstale":"2024-03-30T00:39:55.172086+0000","last_undegraded":"2024-03-30T00:39:55.172086+0000","last_fullsized":"2024-03-30T00:39:55.172086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:39.163294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571335+0000","last_change":"2024-03-30T00:39:55.192391+0000","last_active":"2024-03-30T00:39:55.571335+0000","last_peered":"2024-03-30T00:39:55.571335+0000","last_clean":"2024-03-30T00:39:55.571335+0000","last_became_active":"2024-03-30T00:39:55.189816+0000","last_became_peered":"2024-03-30T00:39:55.189816+0000","last_unstale":"2024-03-30T00:39:55.571335+0000","last_undegraded":"2024-03-30T00:39:55.571335+0000","last_fullsized":"2024-03-30T00:39:55.571335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:05:10.673253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171550+0000","last_change":"2024-03-30T00:39:51.161793+0000","last_active":"2024-03-30T00:39:55.171550+0000","last_peered":"2024-03-30T00:39:55.171550+0000","last_clean":"2024-03-30T00:39:55.171550+0000","last_became_active":"2024-03-30T00:39:51.161523+0000","last_became_peered":"2024-03-30T00:39:51.161523+0000","last_unstale":"2024-03-30T00:39:55.171550+0000","last_undegraded":"2024-03-30T00:39:55.171550+0000","last_fullsized":"2024-03-30T00:39:55.171550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163175+0000","last_change":"2024-03-30T00:39:50.769035+0000","last_active":"2024-03-30T00:39:55.163175+0000","last_peered":"2024-03-30T00:39:55.163175+0000","last_clean":"2024-03-30T00:39:55.163175+0000","last_became_active":"2024-03-30T00:39:50.768789+0000","last_became_peered":"2024-03-30T00:39:50.768789+0000","last_unstale":"2024-03-30T00:39:55.163175+0000","last_undegraded":"2024-03-30T00:39:55.163175+0000","last_fullsized":"2024-03-30T00:39:55.163175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:39.163294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273625+0000","last_change":"2024-03-30T00:39:55.200272+0000","last_active":"2024-03-30T00:39:57.273625+0000","last_peered":"2024-03-30T00:39:57.273625+0000","last_clean":"2024-03-30T00:39:57.273625+0000","last_became_active":"2024-03-30T00:39:55.199688+0000","last_became_peered":"2024-03-30T00:39:55.199688+0000","last_unstale":"2024-03-30T00:39:57.273625+0000","last_undegraded":"2024-03-30T00:39:57.273625+0000","last_fullsized":"2024-03-30T00:39:57.273625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:30.942593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.174408+0000","last_change":"2024-03-30T00:39:51.174886+0000","last_active":"2024-03-30T00:39:55.174408+0000","last_peered":"2024-03-30T00:39:55.174408+0000","last_clean":"2024-03-30T00:39:55.174408+0000","last_became_active":"2024-03-30T00:39:51.174274+0000","last_became_peered":"2024-03-30T00:39:51.174274+0000","last_unstale":"2024-03-30T00:39:55.174408+0000","last_undegraded":"2024-03-30T00:39:55.174408+0000","last_fullsized":"2024-03-30T00:39:55.174408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:24:05.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171643+0000","last_change":"2024-03-30T00:39:50.769092+0000","last_active":"2024-03-30T00:39:55.171643+0000","last_peered":"2024-03-30T00:39:55.171643+0000","last_clean":"2024-03-30T00:39:55.171643+0000","last_became_active":"2024-03-30T00:39:50.768750+0000","last_became_peered":"2024-03-30T00:39:50.768750+0000","last_unstale":"2024-03-30T00:39:55.171643+0000","last_undegraded":"2024-03-30T00:39:55.171643+0000","last_fullsized":"2024-03-30T00:39:55.171643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128439+0000","last_change":"2024-03-30T00:39:55.187382+0000","last_active":"2024-03-30T00:39:57.128439+0000","last_peered":"2024-03-30T00:39:57.128439+0000","last_clean":"2024-03-30T00:39:57.128439+0000","last_became_active":"2024-03-30T00:39:55.186804+0000","last_became_peered":"2024-03-30T00:39:55.186804+0000","last_unstale":"2024-03-30T00:39:57.128439+0000","last_undegraded":"2024-03-30T00:39:57.128439+0000","last_fullsized":"2024-03-30T00:39:57.128439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161837+0000","last_change":"2024-03-30T00:39:51.172750+0000","last_active":"2024-03-30T00:39:55.161837+0000","last_peered":"2024-03-30T00:39:55.161837+0000","last_clean":"2024-03-30T00:39:55.161837+0000","last_became_active":"2024-03-30T00:39:51.172456+0000","last_became_peered":"2024-03-30T00:39:51.172456+0000","last_unstale":"2024-03-30T00:39:55.161837+0000","last_undegraded":"2024-03-30T00:39:55.161837+0000","last_fullsized":"2024-03-30T00:39:55.161837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:40.166014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168217+0000","last_change":"2024-03-30T00:39:50.775126+0000","last_active":"2024-03-30T00:39:55.168217+0000","last_peered":"2024-03-30T00:39:55.168217+0000","last_clean":"2024-03-30T00:39:55.168217+0000","last_became_active":"2024-03-30T00:39:50.773078+0000","last_became_peered":"2024-03-30T00:39:50.773078+0000","last_unstale":"2024-03-30T00:39:55.168217+0000","last_undegraded":"2024-03-30T00:39:55.168217+0000","last_fullsized":"2024-03-30T00:39:55.168217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128669+0000","last_change":"2024-03-30T00:39:55.182489+0000","last_active":"2024-03-30T00:39:57.128669+0000","last_peered":"2024-03-30T00:39:57.128669+0000","last_clean":"2024-03-30T00:39:57.128669+0000","last_became_active":"2024-03-30T00:39:55.181876+0000","last_became_peered":"2024-03-30T00:39:55.181876+0000","last_unstale":"2024-03-30T00:39:57.128669+0000","last_undegraded":"2024-03-30T00:39:57.128669+0000","last_fullsized":"2024-03-30T00:39:57.128669+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171723+0000","last_change":"2024-03-30T00:39:51.157937+0000","last_active":"2024-03-30T00:39:55.171723+0000","last_peered":"2024-03-30T00:39:55.171723+0000","last_clean":"2024-03-30T00:39:55.171723+0000","last_became_active":"2024-03-30T00:39:51.155936+0000","last_became_peered":"2024-03-30T00:39:51.155936+0000","last_unstale":"2024-03-30T00:39:55.171723+0000","last_undegraded":"2024-03-30T00:39:55.171723+0000","last_fullsized":"2024-03-30T00:39:55.171723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159139+0000","last_change":"2024-03-30T00:39:50.765709+0000","last_active":"2024-03-30T00:39:55.159139+0000","last_peered":"2024-03-30T00:39:55.159139+0000","last_clean":"2024-03-30T00:39:55.159139+0000","last_became_active":"2024-03-30T00:39:50.765450+0000","last_became_peered":"2024-03-30T00:39:50.765450+0000","last_unstale":"2024-03-30T00:39:55.159139+0000","last_undegraded":"2024-03-30T00:39:55.159139+0000","last_fullsized":"2024-03-30T00:39:55.159139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.129028+0000","last_change":"2024-03-30T00:39:55.193764+0000","last_active":"2024-03-30T00:39:57.129028+0000","last_peered":"2024-03-30T00:39:57.129028+0000","last_clean":"2024-03-30T00:39:57.129028+0000","last_became_active":"2024-03-30T00:39:55.188761+0000","last_became_peered":"2024-03-30T00:39:55.188761+0000","last_unstale":"2024-03-30T00:39:57.129028+0000","last_undegraded":"2024-03-30T00:39:57.129028+0000","last_fullsized":"2024-03-30T00:39:57.129028+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.182494+0000","last_change":"2024-03-30T00:39:51.176755+0000","last_active":"2024-03-30T00:39:55.182494+0000","last_peered":"2024-03-30T00:39:55.182494+0000","last_clean":"2024-03-30T00:39:55.182494+0000","last_became_active":"2024-03-30T00:39:51.176507+0000","last_became_peered":"2024-03-30T00:39:51.176507+0000","last_unstale":"2024-03-30T00:39:55.182494+0000","last_undegraded":"2024-03-30T00:39:55.182494+0000","last_fullsized":"2024-03-30T00:39:55.182494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160037+0000","last_change":"2024-03-30T00:39:50.763226+0000","last_active":"2024-03-30T00:39:55.160037+0000","last_peered":"2024-03-30T00:39:55.160037+0000","last_clean":"2024-03-30T00:39:55.160037+0000","last_became_active":"2024-03-30T00:39:50.762972+0000","last_became_peered":"2024-03-30T00:39:50.762972+0000","last_unstale":"2024-03-30T00:39:55.160037+0000","last_undegraded":"2024-03-30T00:39:55.160037+0000","last_fullsized":"2024-03-30T00:39:55.160037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:27.304749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273996+0000","last_change":"2024-03-30T00:39:55.198804+0000","last_active":"2024-03-30T00:39:57.273996+0000","last_peered":"2024-03-30T00:39:57.273996+0000","last_clean":"2024-03-30T00:39:57.273996+0000","last_became_active":"2024-03-30T00:39:55.198382+0000","last_became_peered":"2024-03-30T00:39:55.198382+0000","last_unstale":"2024-03-30T00:39:57.273996+0000","last_undegraded":"2024-03-30T00:39:57.273996+0000","last_fullsized":"2024-03-30T00:39:57.273996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165312+0000","last_change":"2024-03-30T00:39:51.176690+0000","last_active":"2024-03-30T00:39:55.165312+0000","last_peered":"2024-03-30T00:39:55.165312+0000","last_clean":"2024-03-30T00:39:55.165312+0000","last_became_active":"2024-03-30T00:39:51.175614+0000","last_became_peered":"2024-03-30T00:39:51.175614+0000","last_unstale":"2024-03-30T00:39:55.165312+0000","last_undegraded":"2024-03-30T00:39:55.165312+0000","last_fullsized":"2024-03-30T00:39:55.165312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:13.959522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.176056+0000","last_change":"2024-03-30T00:39:50.766293+0000","last_active":"2024-03-30T00:39:55.176056+0000","last_peered":"2024-03-30T00:39:55.176056+0000","last_clean":"2024-03-30T00:39:55.176056+0000","last_became_active":"2024-03-30T00:39:50.765977+0000","last_became_peered":"2024-03-30T00:39:50.765977+0000","last_unstale":"2024-03-30T00:39:55.176056+0000","last_undegraded":"2024-03-30T00:39:55.176056+0000","last_fullsized":"2024-03-30T00:39:55.176056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160226+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:39:55.160226+0000","last_peered":"2024-03-30T00:39:55.160226+0000","last_clean":"2024-03-30T00:39:55.160226+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:39:55.160226+0000","last_undegraded":"2024-03-30T00:39:55.160226+0000","last_fullsized":"2024-03-30T00:39:55.160226+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572655+0000","last_change":"2024-03-30T00:39:55.202116+0000","last_active":"2024-03-30T00:39:55.572655+0000","last_peered":"2024-03-30T00:39:55.572655+0000","last_clean":"2024-03-30T00:39:55.572655+0000","last_became_active":"2024-03-30T00:39:55.201638+0000","last_became_peered":"2024-03-30T00:39:55.201638+0000","last_unstale":"2024-03-30T00:39:55.572655+0000","last_undegraded":"2024-03-30T00:39:55.572655+0000","last_fullsized":"2024-03-30T00:39:55.572655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:46:36.593302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162909+0000","last_change":"2024-03-30T00:39:51.176768+0000","last_active":"2024-03-30T00:39:55.162909+0000","last_peered":"2024-03-30T00:39:55.162909+0000","last_clean":"2024-03-30T00:39:55.162909+0000","last_became_active":"2024-03-30T00:39:51.176215+0000","last_became_peered":"2024-03-30T00:39:51.176215+0000","last_unstale":"2024-03-30T00:39:55.162909+0000","last_undegraded":"2024-03-30T00:39:55.162909+0000","last_fullsized":"2024-03-30T00:39:55.162909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:26.397679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159355+0000","last_change":"2024-03-30T00:39:50.763227+0000","last_active":"2024-03-30T00:39:55.159355+0000","last_peered":"2024-03-30T00:39:55.159355+0000","last_clean":"2024-03-30T00:39:55.159355+0000","last_became_active":"2024-03-30T00:39:50.762934+0000","last_became_peered":"2024-03-30T00:39:50.762934+0000","last_unstale":"2024-03-30T00:39:55.159355+0000","last_undegraded":"2024-03-30T00:39:55.159355+0000","last_fullsized":"2024-03-30T00:39:55.159355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572469+0000","last_change":"2024-03-30T00:39:55.195709+0000","last_active":"2024-03-30T00:39:55.572469+0000","last_peered":"2024-03-30T00:39:55.572469+0000","last_clean":"2024-03-30T00:39:55.572469+0000","last_became_active":"2024-03-30T00:39:55.194810+0000","last_became_peered":"2024-03-30T00:39:55.194810+0000","last_unstale":"2024-03-30T00:39:55.572469+0000","last_undegraded":"2024-03-30T00:39:55.572469+0000","last_fullsized":"2024-03-30T00:39:55.572469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:30.942593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170424+0000","last_change":"2024-03-30T00:39:51.182838+0000","last_active":"2024-03-30T00:39:55.170424+0000","last_peered":"2024-03-30T00:39:55.170424+0000","last_clean":"2024-03-30T00:39:55.170424+0000","last_became_active":"2024-03-30T00:39:51.178980+0000","last_became_peered":"2024-03-30T00:39:51.178980+0000","last_unstale":"2024-03-30T00:39:55.170424+0000","last_undegraded":"2024-03-30T00:39:55.170424+0000","last_fullsized":"2024-03-30T00:39:55.170424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:13.959522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172688+0000","last_change":"2024-03-30T00:39:50.755295+0000","last_active":"2024-03-30T00:39:55.172688+0000","last_peered":"2024-03-30T00:39:55.172688+0000","last_clean":"2024-03-30T00:39:55.172688+0000","last_became_active":"2024-03-30T00:39:50.754957+0000","last_became_peered":"2024-03-30T00:39:50.754957+0000","last_unstale":"2024-03-30T00:39:55.172688+0000","last_undegraded":"2024-03-30T00:39:55.172688+0000","last_fullsized":"2024-03-30T00:39:55.172688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:16.100201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273833+0000","last_change":"2024-03-30T00:39:55.199975+0000","last_active":"2024-03-30T00:39:57.273833+0000","last_peered":"2024-03-30T00:39:57.273833+0000","last_clean":"2024-03-30T00:39:57.273833+0000","last_became_active":"2024-03-30T00:39:55.199191+0000","last_became_peered":"2024-03-30T00:39:55.199191+0000","last_unstale":"2024-03-30T00:39:57.273833+0000","last_undegraded":"2024-03-30T00:39:57.273833+0000","last_fullsized":"2024-03-30T00:39:57.273833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163333+0000","last_change":"2024-03-30T00:39:51.163933+0000","last_active":"2024-03-30T00:39:55.163333+0000","last_peered":"2024-03-30T00:39:55.163333+0000","last_clean":"2024-03-30T00:39:55.163333+0000","last_became_active":"2024-03-30T00:39:51.163663+0000","last_became_peered":"2024-03-30T00:39:51.163663+0000","last_unstale":"2024-03-30T00:39:55.163333+0000","last_undegraded":"2024-03-30T00:39:55.163333+0000","last_fullsized":"2024-03-30T00:39:55.163333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:41.330935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159267+0000","last_change":"2024-03-30T00:39:50.766833+0000","last_active":"2024-03-30T00:39:55.159267+0000","last_peered":"2024-03-30T00:39:55.159267+0000","last_clean":"2024-03-30T00:39:55.159267+0000","last_became_active":"2024-03-30T00:39:50.766534+0000","last_became_peered":"2024-03-30T00:39:50.766534+0000","last_unstale":"2024-03-30T00:39:55.159267+0000","last_undegraded":"2024-03-30T00:39:55.159267+0000","last_fullsized":"2024-03-30T00:39:55.159267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:56.358073+0000","last_change":"2024-03-30T00:39:55.184862+0000","last_active":"2024-03-30T00:39:56.358073+0000","last_peered":"2024-03-30T00:39:56.358073+0000","last_clean":"2024-03-30T00:39:56.358073+0000","last_became_active":"2024-03-30T00:39:55.184374+0000","last_became_peered":"2024-03-30T00:39:55.184374+0000","last_unstale":"2024-03-30T00:39:56.358073+0000","last_undegraded":"2024-03-30T00:39:56.358073+0000","last_fullsized":"2024-03-30T00:39:56.358073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163385+0000","last_change":"2024-03-30T00:39:51.162489+0000","last_active":"2024-03-30T00:39:55.163385+0000","last_peered":"2024-03-30T00:39:55.163385+0000","last_clean":"2024-03-30T00:39:55.163385+0000","last_became_active":"2024-03-30T00:39:51.162205+0000","last_became_peered":"2024-03-30T00:39:51.162205+0000","last_unstale":"2024-03-30T00:39:55.163385+0000","last_undegraded":"2024-03-30T00:39:55.163385+0000","last_fullsized":"2024-03-30T00:39:55.163385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:36.973826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162109+0000","last_change":"2024-03-30T00:39:50.768761+0000","last_active":"2024-03-30T00:39:55.162109+0000","last_peered":"2024-03-30T00:39:55.162109+0000","last_clean":"2024-03-30T00:39:55.162109+0000","last_became_active":"2024-03-30T00:39:50.768372+0000","last_became_peered":"2024-03-30T00:39:50.768372+0000","last_unstale":"2024-03-30T00:39:55.162109+0000","last_undegraded":"2024-03-30T00:39:55.162109+0000","last_fullsized":"2024-03-30T00:39:55.162109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:35:25.209340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272560+0000","last_change":"2024-03-30T00:39:55.177966+0000","last_active":"2024-03-30T00:39:57.272560+0000","last_peered":"2024-03-30T00:39:57.272560+0000","last_clean":"2024-03-30T00:39:57.272560+0000","last_became_active":"2024-03-30T00:39:55.177485+0000","last_became_peered":"2024-03-30T00:39:55.177485+0000","last_unstale":"2024-03-30T00:39:57.272560+0000","last_undegraded":"2024-03-30T00:39:57.272560+0000","last_fullsized":"2024-03-30T00:39:57.272560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161004+0000","last_change":"2024-03-30T00:39:51.157566+0000","last_active":"2024-03-30T00:39:55.161004+0000","last_peered":"2024-03-30T00:39:55.161004+0000","last_clean":"2024-03-30T00:39:55.161004+0000","last_became_active":"2024-03-30T00:39:51.157243+0000","last_became_peered":"2024-03-30T00:39:51.157243+0000","last_unstale":"2024-03-30T00:39:55.161004+0000","last_undegraded":"2024-03-30T00:39:55.161004+0000","last_fullsized":"2024-03-30T00:39:55.161004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:35.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159067+0000","last_change":"2024-03-30T00:39:50.760164+0000","last_active":"2024-03-30T00:39:55.159067+0000","last_peered":"2024-03-30T00:39:55.159067+0000","last_clean":"2024-03-30T00:39:55.159067+0000","last_became_active":"2024-03-30T00:39:50.759887+0000","last_became_peered":"2024-03-30T00:39:50.759887+0000","last_unstale":"2024-03-30T00:39:55.159067+0000","last_undegraded":"2024-03-30T00:39:55.159067+0000","last_fullsized":"2024-03-30T00:39:55.159067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273934+0000","last_change":"2024-03-30T00:39:55.196574+0000","last_active":"2024-03-30T00:39:57.273934+0000","last_peered":"2024-03-30T00:39:57.273934+0000","last_clean":"2024-03-30T00:39:57.273934+0000","last_became_active":"2024-03-30T00:39:55.196182+0000","last_became_peered":"2024-03-30T00:39:55.196182+0000","last_unstale":"2024-03-30T00:39:57.273934+0000","last_undegraded":"2024-03-30T00:39:57.273934+0000","last_fullsized":"2024-03-30T00:39:57.273934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170724+0000","last_change":"2024-03-30T00:39:51.167097+0000","last_active":"2024-03-30T00:39:55.170724+0000","last_peered":"2024-03-30T00:39:55.170724+0000","last_clean":"2024-03-30T00:39:55.170724+0000","last_became_active":"2024-03-30T00:39:51.166629+0000","last_became_peered":"2024-03-30T00:39:51.166629+0000","last_unstale":"2024-03-30T00:39:55.170724+0000","last_undegraded":"2024-03-30T00:39:55.170724+0000","last_fullsized":"2024-03-30T00:39:55.170724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170864+0000","last_change":"2024-03-30T00:39:50.767642+0000","last_active":"2024-03-30T00:39:55.170864+0000","last_peered":"2024-03-30T00:39:55.170864+0000","last_clean":"2024-03-30T00:39:55.170864+0000","last_became_active":"2024-03-30T00:39:50.767166+0000","last_became_peered":"2024-03-30T00:39:50.767166+0000","last_unstale":"2024-03-30T00:39:55.170864+0000","last_undegraded":"2024-03-30T00:39:55.170864+0000","last_fullsized":"2024-03-30T00:39:55.170864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572362+0000","last_change":"2024-03-30T00:39:55.192042+0000","last_active":"2024-03-30T00:39:55.572362+0000","last_peered":"2024-03-30T00:39:55.572362+0000","last_clean":"2024-03-30T00:39:55.572362+0000","last_became_active":"2024-03-30T00:39:55.190080+0000","last_became_peered":"2024-03-30T00:39:55.190080+0000","last_unstale":"2024-03-30T00:39:55.572362+0000","last_undegraded":"2024-03-30T00:39:55.572362+0000","last_fullsized":"2024-03-30T00:39:55.572362+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:25.792128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161353+0000","last_change":"2024-03-30T00:39:51.161405+0000","last_active":"2024-03-30T00:39:55.161353+0000","last_peered":"2024-03-30T00:39:55.161353+0000","last_clean":"2024-03-30T00:39:55.161353+0000","last_became_active":"2024-03-30T00:39:51.161053+0000","last_became_peered":"2024-03-30T00:39:51.161053+0000","last_unstale":"2024-03-30T00:39:55.161353+0000","last_undegraded":"2024-03-30T00:39:55.161353+0000","last_fullsized":"2024-03-30T00:39:55.161353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:49.107527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168130+0000","last_change":"2024-03-30T00:39:50.774925+0000","last_active":"2024-03-30T00:39:55.168130+0000","last_peered":"2024-03-30T00:39:55.168130+0000","last_clean":"2024-03-30T00:39:55.168130+0000","last_became_active":"2024-03-30T00:39:50.773221+0000","last_became_peered":"2024-03-30T00:39:50.773221+0000","last_unstale":"2024-03-30T00:39:55.168130+0000","last_undegraded":"2024-03-30T00:39:55.168130+0000","last_fullsized":"2024-03-30T00:39:55.168130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572142+0000","last_change":"2024-03-30T00:39:55.191576+0000","last_active":"2024-03-30T00:39:55.572142+0000","last_peered":"2024-03-30T00:39:55.572142+0000","last_clean":"2024-03-30T00:39:55.572142+0000","last_became_active":"2024-03-30T00:39:55.191188+0000","last_became_peered":"2024-03-30T00:39:55.191188+0000","last_unstale":"2024-03-30T00:39:55.572142+0000","last_undegraded":"2024-03-30T00:39:55.572142+0000","last_fullsized":"2024-03-30T00:39:55.572142+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:58.020154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184454+0000","last_change":"2024-03-30T00:39:51.169704+0000","last_active":"2024-03-30T00:39:55.184454+0000","last_peered":"2024-03-30T00:39:55.184454+0000","last_clean":"2024-03-30T00:39:55.184454+0000","last_became_active":"2024-03-30T00:39:51.168350+0000","last_became_peered":"2024-03-30T00:39:51.168350+0000","last_unstale":"2024-03-30T00:39:55.184454+0000","last_undegraded":"2024-03-30T00:39:55.184454+0000","last_fullsized":"2024-03-30T00:39:55.184454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183870+0000","last_change":"2024-03-30T00:39:50.760857+0000","last_active":"2024-03-30T00:39:55.183870+0000","last_peered":"2024-03-30T00:39:55.183870+0000","last_clean":"2024-03-30T00:39:55.183870+0000","last_became_active":"2024-03-30T00:39:50.760607+0000","last_became_peered":"2024-03-30T00:39:50.760607+0000","last_unstale":"2024-03-30T00:39:55.183870+0000","last_undegraded":"2024-03-30T00:39:55.183870+0000","last_fullsized":"2024-03-30T00:39:55.183870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:56.358741+0000","last_change":"2024-03-30T00:39:55.188289+0000","last_active":"2024-03-30T00:39:56.358741+0000","last_peered":"2024-03-30T00:39:56.358741+0000","last_clean":"2024-03-30T00:39:56.358741+0000","last_became_active":"2024-03-30T00:39:55.187869+0000","last_became_peered":"2024-03-30T00:39:55.187869+0000","last_unstale":"2024-03-30T00:39:56.358741+0000","last_undegraded":"2024-03-30T00:39:56.358741+0000","last_fullsized":"2024-03-30T00:39:56.358741+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162815+0000","last_change":"2024-03-30T00:39:51.177240+0000","last_active":"2024-03-30T00:39:55.162815+0000","last_peered":"2024-03-30T00:39:55.162815+0000","last_clean":"2024-03-30T00:39:55.162815+0000","last_became_active":"2024-03-30T00:39:51.176388+0000","last_became_peered":"2024-03-30T00:39:51.176388+0000","last_unstale":"2024-03-30T00:39:55.162815+0000","last_undegraded":"2024-03-30T00:39:55.162815+0000","last_fullsized":"2024-03-30T00:39:55.162815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:37:05.851113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:40:01.400156+0000","last_change":"2024-03-30T00:39:50.766631+0000","last_active":"2024-03-30T00:40:01.400156+0000","last_peered":"2024-03-30T00:40:01.400156+0000","last_clean":"2024-03-30T00:40:01.400156+0000","last_became_active":"2024-03-30T00:39:50.766362+0000","last_became_peered":"2024-03-30T00:39:50.766362+0000","last_unstale":"2024-03-30T00:40:01.400156+0000","last_undegraded":"2024-03-30T00:40:01.400156+0000","last_fullsized":"2024-03-30T00:40:01.400156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128582+0000","last_change":"2024-03-30T00:39:55.187191+0000","last_active":"2024-03-30T00:39:57.128582+0000","last_peered":"2024-03-30T00:39:57.128582+0000","last_clean":"2024-03-30T00:39:57.128582+0000","last_became_active":"2024-03-30T00:39:55.186712+0000","last_became_peered":"2024-03-30T00:39:55.186712+0000","last_unstale":"2024-03-30T00:39:57.128582+0000","last_undegraded":"2024-03-30T00:39:57.128582+0000","last_fullsized":"2024-03-30T00:39:57.128582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162239+0000","last_change":"2024-03-30T00:39:51.154183+0000","last_active":"2024-03-30T00:39:55.162239+0000","last_peered":"2024-03-30T00:39:55.162239+0000","last_clean":"2024-03-30T00:39:55.162239+0000","last_became_active":"2024-03-30T00:39:51.153909+0000","last_became_peered":"2024-03-30T00:39:51.153909+0000","last_unstale":"2024-03-30T00:39:55.162239+0000","last_undegraded":"2024-03-30T00:39:55.162239+0000","last_fullsized":"2024-03-30T00:39:55.162239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:51.868242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172499+0000","last_change":"2024-03-30T00:39:50.763665+0000","last_active":"2024-03-30T00:39:55.172499+0000","last_peered":"2024-03-30T00:39:55.172499+0000","last_clean":"2024-03-30T00:39:55.172499+0000","last_became_active":"2024-03-30T00:39:50.763390+0000","last_became_peered":"2024-03-30T00:39:50.763390+0000","last_unstale":"2024-03-30T00:39:55.172499+0000","last_undegraded":"2024-03-30T00:39:55.172499+0000","last_fullsized":"2024-03-30T00:39:55.172499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272944+0000","last_change":"2024-03-30T00:39:55.185767+0000","last_active":"2024-03-30T00:39:57.272944+0000","last_peered":"2024-03-30T00:39:57.272944+0000","last_clean":"2024-03-30T00:39:57.272944+0000","last_became_active":"2024-03-30T00:39:55.185430+0000","last_became_peered":"2024-03-30T00:39:55.185430+0000","last_unstale":"2024-03-30T00:39:57.272944+0000","last_undegraded":"2024-03-30T00:39:57.272944+0000","last_fullsized":"2024-03-30T00:39:57.272944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:59:28.443255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184382+0000","last_change":"2024-03-30T00:39:51.177311+0000","last_active":"2024-03-30T00:39:55.184382+0000","last_peered":"2024-03-30T00:39:55.184382+0000","last_clean":"2024-03-30T00:39:55.184382+0000","last_became_active":"2024-03-30T00:39:51.175904+0000","last_became_peered":"2024-03-30T00:39:51.175904+0000","last_unstale":"2024-03-30T00:39:55.184382+0000","last_undegraded":"2024-03-30T00:39:55.184382+0000","last_fullsized":"2024-03-30T00:39:55.184382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167202+0000","last_change":"2024-03-30T00:39:50.769375+0000","last_active":"2024-03-30T00:39:55.167202+0000","last_peered":"2024-03-30T00:39:55.167202+0000","last_clean":"2024-03-30T00:39:55.167202+0000","last_became_active":"2024-03-30T00:39:50.768671+0000","last_became_peered":"2024-03-30T00:39:50.768671+0000","last_unstale":"2024-03-30T00:39:55.167202+0000","last_undegraded":"2024-03-30T00:39:55.167202+0000","last_fullsized":"2024-03-30T00:39:55.167202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571487+0000","last_change":"2024-03-30T00:39:55.192307+0000","last_active":"2024-03-30T00:39:55.571487+0000","last_peered":"2024-03-30T00:39:55.571487+0000","last_clean":"2024-03-30T00:39:55.571487+0000","last_became_active":"2024-03-30T00:39:55.191803+0000","last_became_peered":"2024-03-30T00:39:55.191803+0000","last_unstale":"2024-03-30T00:39:55.571487+0000","last_undegraded":"2024-03-30T00:39:55.571487+0000","last_fullsized":"2024-03-30T00:39:55.571487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:33:50.238803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167108+0000","last_change":"2024-03-30T00:39:51.168083+0000","last_active":"2024-03-30T00:39:55.167108+0000","last_peered":"2024-03-30T00:39:55.167108+0000","last_clean":"2024-03-30T00:39:55.167108+0000","last_became_active":"2024-03-30T00:39:51.167756+0000","last_became_peered":"2024-03-30T00:39:51.167756+0000","last_unstale":"2024-03-30T00:39:55.167108+0000","last_undegraded":"2024-03-30T00:39:55.167108+0000","last_fullsized":"2024-03-30T00:39:55.167108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172080+0000","last_change":"2024-03-30T00:39:50.775634+0000","last_active":"2024-03-30T00:39:55.172080+0000","last_peered":"2024-03-30T00:39:55.172080+0000","last_clean":"2024-03-30T00:39:55.172080+0000","last_became_active":"2024-03-30T00:39:50.775337+0000","last_became_peered":"2024-03-30T00:39:50.775337+0000","last_unstale":"2024-03-30T00:39:55.172080+0000","last_undegraded":"2024-03-30T00:39:55.172080+0000","last_fullsized":"2024-03-30T00:39:55.172080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571487+0000","last_change":"2024-03-30T00:39:55.175243+0000","last_active":"2024-03-30T00:39:55.571487+0000","last_peered":"2024-03-30T00:39:55.571487+0000","last_clean":"2024-03-30T00:39:55.571487+0000","last_became_active":"2024-03-30T00:39:55.174556+0000","last_became_peered":"2024-03-30T00:39:55.174556+0000","last_unstale":"2024-03-30T00:39:55.571487+0000","last_undegraded":"2024-03-30T00:39:55.571487+0000","last_fullsized":"2024-03-30T00:39:55.571487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.274064+0000","last_change":"2024-03-30T00:39:55.203406+0000","last_active":"2024-03-30T00:39:57.274064+0000","last_peered":"2024-03-30T00:39:57.274064+0000","last_clean":"2024-03-30T00:39:57.274064+0000","last_became_active":"2024-03-30T00:39:55.203027+0000","last_became_peered":"2024-03-30T00:39:55.203027+0000","last_unstale":"2024-03-30T00:39:57.274064+0000","last_undegraded":"2024-03-30T00:39:57.274064+0000","last_fullsized":"2024-03-30T00:39:57.274064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171850+0000","last_change":"2024-03-30T00:39:51.175309+0000","last_active":"2024-03-30T00:39:55.171850+0000","last_peered":"2024-03-30T00:39:55.171850+0000","last_clean":"2024-03-30T00:39:55.171850+0000","last_became_active":"2024-03-30T00:39:51.174946+0000","last_became_peered":"2024-03-30T00:39:51.174946+0000","last_unstale":"2024-03-30T00:39:55.171850+0000","last_undegraded":"2024-03-30T00:39:55.171850+0000","last_fullsized":"2024-03-30T00:39:55.171850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167862+0000","last_change":"2024-03-30T00:39:50.762581+0000","last_active":"2024-03-30T00:39:55.167862+0000","last_peered":"2024-03-30T00:39:55.167862+0000","last_clean":"2024-03-30T00:39:55.167862+0000","last_became_active":"2024-03-30T00:39:50.762206+0000","last_became_peered":"2024-03-30T00:39:50.762206+0000","last_unstale":"2024-03-30T00:39:55.167862+0000","last_undegraded":"2024-03-30T00:39:55.167862+0000","last_fullsized":"2024-03-30T00:39:55.167862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.573135+0000","last_change":"2024-03-30T00:39:55.188289+0000","last_active":"2024-03-30T00:39:55.573135+0000","last_peered":"2024-03-30T00:39:55.573135+0000","last_clean":"2024-03-30T00:39:55.573135+0000","last_became_active":"2024-03-30T00:39:55.187825+0000","last_became_peered":"2024-03-30T00:39:55.187825+0000","last_unstale":"2024-03-30T00:39:55.573135+0000","last_undegraded":"2024-03-30T00:39:55.573135+0000","last_fullsized":"2024-03-30T00:39:55.573135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.166225+0000","last_change":"2024-03-30T00:39:51.172769+0000","last_active":"2024-03-30T00:39:55.166225+0000","last_peered":"2024-03-30T00:39:55.166225+0000","last_clean":"2024-03-30T00:39:55.166225+0000","last_became_active":"2024-03-30T00:39:51.170913+0000","last_became_peered":"2024-03-30T00:39:51.170913+0000","last_unstale":"2024-03-30T00:39:55.166225+0000","last_undegraded":"2024-03-30T00:39:55.166225+0000","last_fullsized":"2024-03-30T00:39:55.166225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165283+0000","last_change":"2024-03-30T00:39:50.771732+0000","last_active":"2024-03-30T00:39:55.165283+0000","last_peered":"2024-03-30T00:39:55.165283+0000","last_clean":"2024-03-30T00:39:55.165283+0000","last_became_active":"2024-03-30T00:39:50.771443+0000","last_became_peered":"2024-03-30T00:39:50.771443+0000","last_unstale":"2024-03-30T00:39:55.165283+0000","last_undegraded":"2024-03-30T00:39:55.165283+0000","last_fullsized":"2024-03-30T00:39:55.165283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171239+0000","last_change":"2024-03-30T00:39:50.768147+0000","last_active":"2024-03-30T00:39:55.171239+0000","last_peered":"2024-03-30T00:39:55.171239+0000","last_clean":"2024-03-30T00:39:55.171239+0000","last_became_active":"2024-03-30T00:39:50.767258+0000","last_became_peered":"2024-03-30T00:39:50.767258+0000","last_unstale":"2024-03-30T00:39:55.171239+0000","last_undegraded":"2024-03-30T00:39:55.171239+0000","last_fullsized":"2024-03-30T00:39:55.171239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183786+0000","last_change":"2024-03-30T00:39:51.173636+0000","last_active":"2024-03-30T00:39:55.183786+0000","last_peered":"2024-03-30T00:39:55.183786+0000","last_clean":"2024-03-30T00:39:55.183786+0000","last_became_active":"2024-03-30T00:39:51.173356+0000","last_became_peered":"2024-03-30T00:39:51.173356+0000","last_unstale":"2024-03-30T00:39:55.183786+0000","last_undegraded":"2024-03-30T00:39:55.183786+0000","last_fullsized":"2024-03-30T00:39:55.183786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:10:20.544219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273686+0000","last_change":"2024-03-30T00:39:55.187062+0000","last_active":"2024-03-30T00:39:57.273686+0000","last_peered":"2024-03-30T00:39:57.273686+0000","last_clean":"2024-03-30T00:39:57.273686+0000","last_became_active":"2024-03-30T00:39:55.184977+0000","last_became_peered":"2024-03-30T00:39:55.184977+0000","last_unstale":"2024-03-30T00:39:57.273686+0000","last_undegraded":"2024-03-30T00:39:57.273686+0000","last_fullsized":"2024-03-30T00:39:57.273686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:23.899767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169318+0000","last_change":"2024-03-30T00:39:50.767360+0000","last_active":"2024-03-30T00:39:55.169318+0000","last_peered":"2024-03-30T00:39:55.169318+0000","last_clean":"2024-03-30T00:39:55.169318+0000","last_became_active":"2024-03-30T00:39:50.764301+0000","last_became_peered":"2024-03-30T00:39:50.764301+0000","last_unstale":"2024-03-30T00:39:55.169318+0000","last_undegraded":"2024-03-30T00:39:55.169318+0000","last_fullsized":"2024-03-30T00:39:55.169318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:26.877416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160985+0000","last_change":"2024-03-30T00:39:51.163323+0000","last_active":"2024-03-30T00:39:55.160985+0000","last_peered":"2024-03-30T00:39:55.160985+0000","last_clean":"2024-03-30T00:39:55.160985+0000","last_became_active":"2024-03-30T00:39:51.163080+0000","last_became_peered":"2024-03-30T00:39:51.163080+0000","last_unstale":"2024-03-30T00:39:55.160985+0000","last_undegraded":"2024-03-30T00:39:55.160985+0000","last_fullsized":"2024-03-30T00:39:55.160985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:28.307469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572341+0000","last_change":"2024-03-30T00:39:55.194121+0000","last_active":"2024-03-30T00:39:55.572341+0000","last_peered":"2024-03-30T00:39:55.572341+0000","last_clean":"2024-03-30T00:39:55.572341+0000","last_became_active":"2024-03-30T00:39:55.190121+0000","last_became_peered":"2024-03-30T00:39:55.190121+0000","last_unstale":"2024-03-30T00:39:55.572341+0000","last_undegraded":"2024-03-30T00:39:55.572341+0000","last_fullsized":"2024-03-30T00:39:55.572341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:10:34.870687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169878+0000","last_change":"2024-03-30T00:39:51.168112+0000","last_active":"2024-03-30T00:39:55.169878+0000","last_peered":"2024-03-30T00:39:55.169878+0000","last_clean":"2024-03-30T00:39:55.169878+0000","last_became_active":"2024-03-30T00:39:51.167770+0000","last_became_peered":"2024-03-30T00:39:51.167770+0000","last_unstale":"2024-03-30T00:39:55.169878+0000","last_undegraded":"2024-03-30T00:39:55.169878+0000","last_fullsized":"2024-03-30T00:39:55.169878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159083+0000","last_change":"2024-03-30T00:39:51.169867+0000","last_active":"2024-03-30T00:39:55.159083+0000","last_peered":"2024-03-30T00:39:55.159083+0000","last_clean":"2024-03-30T00:39:55.159083+0000","last_became_active":"2024-03-30T00:39:51.169515+0000","last_became_peered":"2024-03-30T00:39:51.169515+0000","last_unstale":"2024-03-30T00:39:55.159083+0000","last_undegraded":"2024-03-30T00:39:55.159083+0000","last_fullsized":"2024-03-30T00:39:55.159083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:44.962672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168036+0000","last_change":"2024-03-30T00:39:51.175788+0000","last_active":"2024-03-30T00:39:55.168036+0000","last_peered":"2024-03-30T00:39:55.168036+0000","last_clean":"2024-03-30T00:39:55.168036+0000","last_became_active":"2024-03-30T00:39:51.175288+0000","last_became_peered":"2024-03-30T00:39:51.175288+0000","last_unstale":"2024-03-30T00:39:55.168036+0000","last_undegraded":"2024-03-30T00:39:55.168036+0000","last_fullsized":"2024-03-30T00:39:55.168036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:35.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183710+0000","last_change":"2024-03-30T00:39:51.173618+0000","last_active":"2024-03-30T00:39:55.183710+0000","last_peered":"2024-03-30T00:39:55.183710+0000","last_clean":"2024-03-30T00:39:55.183710+0000","last_became_active":"2024-03-30T00:39:51.173324+0000","last_became_peered":"2024-03-30T00:39:51.173324+0000","last_unstale":"2024-03-30T00:39:55.183710+0000","last_undegraded":"2024-03-30T00:39:55.183710+0000","last_fullsized":"2024-03-30T00:39:55.183710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168094+0000","last_change":"2024-03-30T00:39:51.184101+0000","last_active":"2024-03-30T00:39:55.168094+0000","last_peered":"2024-03-30T00:39:55.168094+0000","last_clean":"2024-03-30T00:39:55.168094+0000","last_became_active":"2024-03-30T00:39:51.183843+0000","last_became_peered":"2024-03-30T00:39:51.183843+0000","last_unstale":"2024-03-30T00:39:55.168094+0000","last_undegraded":"2024-03-30T00:39:55.168094+0000","last_fullsized":"2024-03-30T00:39:55.168094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:51.868242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172390+0000","last_change":"2024-03-30T00:39:51.174001+0000","last_active":"2024-03-30T00:39:55.172390+0000","last_peered":"2024-03-30T00:39:55.172390+0000","last_clean":"2024-03-30T00:39:55.172390+0000","last_became_active":"2024-03-30T00:39:51.173753+0000","last_became_peered":"2024-03-30T00:39:51.173753+0000","last_unstale":"2024-03-30T00:39:55.172390+0000","last_undegraded":"2024-03-30T00:39:55.172390+0000","last_fullsized":"2024-03-30T00:39:55.172390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171728+0000","last_change":"2024-03-30T00:39:51.165845+0000","last_active":"2024-03-30T00:39:55.171728+0000","last_peered":"2024-03-30T00:39:55.171728+0000","last_clean":"2024-03-30T00:39:55.171728+0000","last_became_active":"2024-03-30T00:39:51.164657+0000","last_became_peered":"2024-03-30T00:39:51.164657+0000","last_unstale":"2024-03-30T00:39:55.171728+0000","last_undegraded":"2024-03-30T00:39:55.171728+0000","last_fullsized":"2024-03-30T00:39:55.171728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.164918+0000","last_change":"2024-03-30T00:39:51.185222+0000","last_active":"2024-03-30T00:39:55.164918+0000","last_peered":"2024-03-30T00:39:55.164918+0000","last_clean":"2024-03-30T00:39:55.164918+0000","last_became_active":"2024-03-30T00:39:51.184954+0000","last_became_peered":"2024-03-30T00:39:51.184954+0000","last_unstale":"2024-03-30T00:39:55.164918+0000","last_undegraded":"2024-03-30T00:39:55.164918+0000","last_fullsized":"2024-03-30T00:39:55.164918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167766+0000","last_change":"2024-03-30T00:39:51.174598+0000","last_active":"2024-03-30T00:39:55.167766+0000","last_peered":"2024-03-30T00:39:55.167766+0000","last_clean":"2024-03-30T00:39:55.167766+0000","last_became_active":"2024-03-30T00:39:51.174250+0000","last_became_peered":"2024-03-30T00:39:51.174250+0000","last_unstale":"2024-03-30T00:39:55.167766+0000","last_undegraded":"2024-03-30T00:39:55.167766+0000","last_fullsized":"2024-03-30T00:39:55.167766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165202+0000","last_change":"2024-03-30T00:39:51.184460+0000","last_active":"2024-03-30T00:39:55.165202+0000","last_peered":"2024-03-30T00:39:55.165202+0000","last_clean":"2024-03-30T00:39:55.165202+0000","last_became_active":"2024-03-30T00:39:51.184185+0000","last_became_peered":"2024-03-30T00:39:51.184185+0000","last_unstale":"2024-03-30T00:39:55.165202+0000","last_undegraded":"2024-03-30T00:39:55.165202+0000","last_fullsized":"2024-03-30T00:39:55.165202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167944+0000","last_change":"2024-03-30T00:39:51.168064+0000","last_active":"2024-03-30T00:39:55.167944+0000","last_peered":"2024-03-30T00:39:55.167944+0000","last_clean":"2024-03-30T00:39:55.167944+0000","last_became_active":"2024-03-30T00:39:51.167772+0000","last_became_peered":"2024-03-30T00:39:51.167772+0000","last_unstale":"2024-03-30T00:39:55.167944+0000","last_undegraded":"2024-03-30T00:39:55.167944+0000","last_fullsized":"2024-03-30T00:39:55.167944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:12.512040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183617+0000","last_change":"2024-03-30T00:39:51.183269+0000","last_active":"2024-03-30T00:39:55.183617+0000","last_peered":"2024-03-30T00:39:55.183617+0000","last_clean":"2024-03-30T00:39:55.183617+0000","last_became_active":"2024-03-30T00:39:51.182687+0000","last_became_peered":"2024-03-30T00:39:51.182687+0000","last_unstale":"2024-03-30T00:39:55.183617+0000","last_undegraded":"2024-03-30T00:39:55.183617+0000","last_fullsized":"2024-03-30T00:39:55.183617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171193+0000","last_change":"2024-03-30T00:39:51.171426+0000","last_active":"2024-03-30T00:39:55.171193+0000","last_peered":"2024-03-30T00:39:55.171193+0000","last_clean":"2024-03-30T00:39:55.171193+0000","last_became_active":"2024-03-30T00:39:51.171182+0000","last_became_peered":"2024-03-30T00:39:51.171182+0000","last_unstale":"2024-03-30T00:39:55.171193+0000","last_undegraded":"2024-03-30T00:39:55.171193+0000","last_fullsized":"2024-03-30T00:39:55.171193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:51:41.648914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163087+0000","last_change":"2024-03-30T00:39:51.159899+0000","last_active":"2024-03-30T00:39:55.163087+0000","last_peered":"2024-03-30T00:39:55.163087+0000","last_clean":"2024-03-30T00:39:55.163087+0000","last_became_active":"2024-03-30T00:39:51.159626+0000","last_became_peered":"2024-03-30T00:39:51.159626+0000","last_unstale":"2024-03-30T00:39:55.163087+0000","last_undegraded":"2024-03-30T00:39:55.163087+0000","last_fullsized":"2024-03-30T00:39:55.163087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159252+0000","last_change":"2024-03-30T00:39:51.171375+0000","last_active":"2024-03-30T00:39:55.159252+0000","last_peered":"2024-03-30T00:39:55.159252+0000","last_clean":"2024-03-30T00:39:55.159252+0000","last_became_active":"2024-03-30T00:39:51.170689+0000","last_became_peered":"2024-03-30T00:39:51.170689+0000","last_unstale":"2024-03-30T00:39:55.159252+0000","last_undegraded":"2024-03-30T00:39:55.159252+0000","last_fullsized":"2024-03-30T00:39:55.159252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:27.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167585+0000","last_change":"2024-03-30T00:39:51.164848+0000","last_active":"2024-03-30T00:39:55.167585+0000","last_peered":"2024-03-30T00:39:55.167585+0000","last_clean":"2024-03-30T00:39:55.167585+0000","last_became_active":"2024-03-30T00:39:51.164575+0000","last_became_peered":"2024-03-30T00:39:51.164575+0000","last_unstale":"2024-03-30T00:39:55.167585+0000","last_undegraded":"2024-03-30T00:39:55.167585+0000","last_fullsized":"2024-03-30T00:39:55.167585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167681+0000","last_change":"2024-03-30T00:39:51.173535+0000","last_active":"2024-03-30T00:39:55.167681+0000","last_peered":"2024-03-30T00:39:55.167681+0000","last_clean":"2024-03-30T00:39:55.167681+0000","last_became_active":"2024-03-30T00:39:51.173313+0000","last_became_peered":"2024-03-30T00:39:51.173313+0000","last_unstale":"2024-03-30T00:39:55.167681+0000","last_undegraded":"2024-03-30T00:39:55.167681+0000","last_fullsized":"2024-03-30T00:39:55.167681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171944+0000","last_change":"2024-03-30T00:39:51.171986+0000","last_active":"2024-03-30T00:39:55.171944+0000","last_peered":"2024-03-30T00:39:55.171944+0000","last_clean":"2024-03-30T00:39:55.171944+0000","last_became_active":"2024-03-30T00:39:51.164756+0000","last_became_peered":"2024-03-30T00:39:51.164756+0000","last_unstale":"2024-03-30T00:39:55.171944+0000","last_undegraded":"2024-03-30T00:39:55.171944+0000","last_fullsized":"2024-03-30T00:39:55.171944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184365+0000","last_change":"2024-03-30T00:39:51.181455+0000","last_active":"2024-03-30T00:39:55.184365+0000","last_peered":"2024-03-30T00:39:55.184365+0000","last_clean":"2024-03-30T00:39:55.184365+0000","last_became_active":"2024-03-30T00:39:51.181207+0000","last_became_peered":"2024-03-30T00:39:51.181207+0000","last_unstale":"2024-03-30T00:39:55.184365+0000","last_undegraded":"2024-03-30T00:39:55.184365+0000","last_fullsized":"2024-03-30T00:39:55.184365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:01:48.059238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169138+0000","last_change":"2024-03-30T00:39:51.175074+0000","last_active":"2024-03-30T00:39:55.169138+0000","last_peered":"2024-03-30T00:39:55.169138+0000","last_clean":"2024-03-30T00:39:55.169138+0000","last_became_active":"2024-03-30T00:39:51.174836+0000","last_became_peered":"2024-03-30T00:39:51.174836+0000","last_unstale":"2024-03-30T00:39:55.169138+0000","last_undegraded":"2024-03-30T00:39:55.169138+0000","last_fullsized":"2024-03-30T00:39:55.169138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162196+0000","last_change":"2024-03-30T00:39:51.169964+0000","last_active":"2024-03-30T00:39:55.162196+0000","last_peered":"2024-03-30T00:39:55.162196+0000","last_clean":"2024-03-30T00:39:55.162196+0000","last_became_active":"2024-03-30T00:39:51.169647+0000","last_became_peered":"2024-03-30T00:39:51.169647+0000","last_unstale":"2024-03-30T00:39:55.162196+0000","last_undegraded":"2024-03-30T00:39:55.162196+0000","last_fullsized":"2024-03-30T00:39:55.162196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:44.962672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171503+0000","last_change":"2024-03-30T00:39:51.178908+0000","last_active":"2024-03-30T00:39:55.171503+0000","last_peered":"2024-03-30T00:39:55.171503+0000","last_clean":"2024-03-30T00:39:55.171503+0000","last_became_active":"2024-03-30T00:39:51.178663+0000","last_became_peered":"2024-03-30T00:39:51.178663+0000","last_unstale":"2024-03-30T00:39:55.171503+0000","last_undegraded":"2024-03-30T00:39:55.171503+0000","last_fullsized":"2024-03-30T00:39:55.171503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171643+0000","last_change":"2024-03-30T00:39:51.179913+0000","last_active":"2024-03-30T00:39:55.171643+0000","last_peered":"2024-03-30T00:39:55.171643+0000","last_clean":"2024-03-30T00:39:55.171643+0000","last_became_active":"2024-03-30T00:39:51.179652+0000","last_became_peered":"2024-03-30T00:39:51.179652+0000","last_unstale":"2024-03-30T00:39:55.171643+0000","last_undegraded":"2024-03-30T00:39:55.171643+0000","last_fullsized":"2024-03-30T00:39:55.171643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163237+0000","last_change":"2024-03-30T00:39:51.170941+0000","last_active":"2024-03-30T00:39:55.163237+0000","last_peered":"2024-03-30T00:39:55.163237+0000","last_clean":"2024-03-30T00:39:55.163237+0000","last_became_active":"2024-03-30T00:39:51.166779+0000","last_became_peered":"2024-03-30T00:39:51.166779+0000","last_unstale":"2024-03-30T00:39:55.163237+0000","last_undegraded":"2024-03-30T00:39:55.163237+0000","last_fullsized":"2024-03-30T00:39:55.163237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:12.512040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167960+0000","last_change":"2024-03-30T00:39:51.176410+0000","last_active":"2024-03-30T00:39:55.167960+0000","last_peered":"2024-03-30T00:39:55.167960+0000","last_clean":"2024-03-30T00:39:55.167960+0000","last_became_active":"2024-03-30T00:39:51.175971+0000","last_became_peered":"2024-03-30T00:39:51.175971+0000","last_unstale":"2024-03-30T00:39:55.167960+0000","last_undegraded":"2024-03-30T00:39:55.167960+0000","last_fullsized":"2024-03-30T00:39:55.167960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160661+0000","last_change":"2024-03-30T00:39:51.167648+0000","last_active":"2024-03-30T00:39:55.160661+0000","last_peered":"2024-03-30T00:39:55.160661+0000","last_clean":"2024-03-30T00:39:55.160661+0000","last_became_active":"2024-03-30T00:39:51.167382+0000","last_became_peered":"2024-03-30T00:39:51.167382+0000","last_unstale":"2024-03-30T00:39:55.160661+0000","last_undegraded":"2024-03-30T00:39:55.160661+0000","last_fullsized":"2024-03-30T00:39:55.160661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:39:43.189821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167861+0000","last_change":"2024-03-30T00:39:51.169776+0000","last_active":"2024-03-30T00:39:55.167861+0000","last_peered":"2024-03-30T00:39:55.167861+0000","last_clean":"2024-03-30T00:39:55.167861+0000","last_became_active":"2024-03-30T00:39:51.169462+0000","last_became_peered":"2024-03-30T00:39:51.169462+0000","last_unstale":"2024-03-30T00:39:55.167861+0000","last_undegraded":"2024-03-30T00:39:55.167861+0000","last_fullsized":"2024-03-30T00:39:55.167861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:37:05.851113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161236+0000","last_change":"2024-03-30T00:39:51.163469+0000","last_active":"2024-03-30T00:39:55.161236+0000","last_peered":"2024-03-30T00:39:55.161236+0000","last_clean":"2024-03-30T00:39:55.161236+0000","last_became_active":"2024-03-30T00:39:51.163200+0000","last_became_peered":"2024-03-30T00:39:51.163200+0000","last_unstale":"2024-03-30T00:39:55.161236+0000","last_undegraded":"2024-03-30T00:39:55.161236+0000","last_fullsized":"2024-03-30T00:39:55.161236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:24:05.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170209+0000","last_change":"2024-03-30T00:39:51.183006+0000","last_active":"2024-03-30T00:39:55.170209+0000","last_peered":"2024-03-30T00:39:55.170209+0000","last_clean":"2024-03-30T00:39:55.170209+0000","last_became_active":"2024-03-30T00:39:51.179310+0000","last_became_peered":"2024-03-30T00:39:51.179310+0000","last_unstale":"2024-03-30T00:39:55.170209+0000","last_undegraded":"2024-03-30T00:39:55.170209+0000","last_fullsized":"2024-03-30T00:39:55.170209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:41.330935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183618+0000","last_change":"2024-03-30T00:39:51.180621+0000","last_active":"2024-03-30T00:39:55.183618+0000","last_peered":"2024-03-30T00:39:55.183618+0000","last_clean":"2024-03-30T00:39:55.183618+0000","last_became_active":"2024-03-30T00:39:51.175749+0000","last_became_peered":"2024-03-30T00:39:51.175749+0000","last_unstale":"2024-03-30T00:39:55.183618+0000","last_undegraded":"2024-03-30T00:39:55.183618+0000","last_fullsized":"2024-03-30T00:39:55.183618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:27.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171993+0000","last_change":"2024-03-30T00:39:51.168331+0000","last_active":"2024-03-30T00:39:55.171993+0000","last_peered":"2024-03-30T00:39:55.171993+0000","last_clean":"2024-03-30T00:39:55.171993+0000","last_became_active":"2024-03-30T00:39:51.168052+0000","last_became_peered":"2024-03-30T00:39:51.168052+0000","last_unstale":"2024-03-30T00:39:55.171993+0000","last_undegraded":"2024-03-30T00:39:55.171993+0000","last_fullsized":"2024-03-30T00:39:55.171993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163158+0000","last_change":"2024-03-30T00:39:51.167745+0000","last_active":"2024-03-30T00:39:55.163158+0000","last_peered":"2024-03-30T00:39:55.163158+0000","last_clean":"2024-03-30T00:39:55.163158+0000","last_became_active":"2024-03-30T00:39:51.166397+0000","last_became_peered":"2024-03-30T00:39:51.166397+0000","last_unstale":"2024-03-30T00:39:55.163158+0000","last_undegraded":"2024-03-30T00:39:55.163158+0000","last_fullsized":"2024-03-30T00:39:55.163158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:50:20.941634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:40:15.615 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:40:15.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5139a00640 1 Processor -- start 2024-03-30T00:40:15.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5139a00640 1 -- start start 2024-03-30T00:40:15.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5139a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f51340ade30 0x7f51340c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5139a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5134070cb0 0x7f51340ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5139a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51340cb290 con 0x7f51340ca370 2024-03-30T00:40:15.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5139a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51340ca7d0 con 0x7f51340ade30 2024-03-30T00:40:15.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5139a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51340ca9f0 con 0x7f5134070cb0 2024-03-30T00:40:15.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5133e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f51340ade30 0x7f51340c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5133e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f51340ade30 0x7f51340c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56660/0 (socket says 172.21.15.66:56660) 2024-03-30T00:40:15.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5133e00640 1 -- 172.21.15.66:0/3226701535 learned_addr learned my addr 172.21.15.66:0/3226701535 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:15.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5133e00640 1 -- 172.21.15.66:0/3226701535 >> v2:172.21.15.156:3300/0 conn(0x7f5134070cb0 msgr2=0x7f51340ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:15.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5133e00640 1 --2- 172.21.15.66:0/3226701535 >> v2:172.21.15.156:3300/0 conn(0x7f5134070cb0 0x7f51340ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5133e00640 1 -- 172.21.15.66:0/3226701535 >> v1:172.21.15.66:6789/0 conn(0x7f51340ca370 legacy=0x7f51340705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.781+0000 7f5133e00640 1 -- 172.21.15.66:0/3226701535 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51340cacd0 con 0x7f51340ade30 2024-03-30T00:40:15.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5133e00640 1 --2- 172.21.15.66:0/3226701535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ade30 0x7f51340c9e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f51340cf7c0 tx=0x7f512401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450e52234a2cf48c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:15.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5132a00640 1 -- 172.21.15.66:0/3226701535 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240ac630 con 0x7f51340ade30 2024-03-30T00:40:15.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5132a00640 1 -- 172.21.15.66:0/3226701535 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51240ac790 con 0x7f51340ade30 2024-03-30T00:40:15.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5132a00640 1 -- 172.21.15.66:0/3226701535 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240acb20 con 0x7f51340ade30 2024-03-30T00:40:15.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5139a00640 1 -- 172.21.15.66:0/3226701535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ade30 msgr2=0x7f51340c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5139a00640 1 --2- 172.21.15.66:0/3226701535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ade30 0x7f51340c9e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f51340cf7c0 tx=0x7f512401c3d0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5139a00640 1 -- 172.21.15.66:0/3226701535 shutdown_connections 2024-03-30T00:40:15.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5139a00640 1 --2- 172.21.15.66:0/3226701535 >> v2:172.21.15.156:3300/0 conn(0x7f5134070cb0 0x7f51340ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5139a00640 1 --2- 172.21.15.66:0/3226701535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ade30 0x7f51340c9e30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5139a00640 1 -- 172.21.15.66:0/3226701535 >> 172.21.15.66:0/3226701535 conn(0x7f51340ce9f0 msgr2=0x7f51340cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:15.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5139a00640 1 -- 172.21.15.66:0/3226701535 shutdown_connections 2024-03-30T00:40:15.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5139a00640 1 -- 172.21.15.66:0/3226701535 wait complete. 2024-03-30T00:40:15.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5139a00640 1 Processor -- start 2024-03-30T00:40:15.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.785+0000 7f5139a00640 1 -- start start 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5134070cb0 0x7f51341504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51340ade30 0x7f513415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ca370 0x7f51341f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5134159d80 con 0x7f51340ca370 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513415a300 con 0x7f51340ade30 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51340c85a0 con 0x7f5134070cb0 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ca370 0x7f51341f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ca370 0x7f51341f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56670/0 (socket says 172.21.15.66:56670) 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139000640 1 -- 172.21.15.66:0/3011649908 learned_addr learned my addr 172.21.15.66:0/3011649908 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5133e00640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5134070cb0 0x7f51341504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5133400640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51340ade30 0x7f513415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139000640 1 -- 172.21.15.66:0/3011649908 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5134070cb0 msgr2=0x7f51341504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139000640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5134070cb0 0x7f51341504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139000640 1 -- 172.21.15.66:0/3011649908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51340ade30 msgr2=0x7f513415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139000640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51340ade30 0x7f513415bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139000640 1 -- 172.21.15.66:0/3011649908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240094d0 con 0x7f51340ca370 2024-03-30T00:40:15.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5133e00640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5134070cb0 0x7f51341504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:15.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139000640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ca370 0x7f51341f63b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f512c00b9e0 tx=0x7f512c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f512be00640 1 -- 172.21.15.66:0/3011649908 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c00cb20 con 0x7f51340ca370 2024-03-30T00:40:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f512be00640 1 -- 172.21.15.66:0/3011649908 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c00cca0 con 0x7f51340ca370 2024-03-30T00:40:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139a00640 1 -- 172.21.15.66:0/3011649908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51340c8880 con 0x7f51340ca370 2024-03-30T00:40:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f512be00640 1 -- 172.21.15.66:0/3011649908 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c014680 con 0x7f51340ca370 2024-03-30T00:40:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139a00640 1 -- 172.21.15.66:0/3011649908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513415c230 con 0x7f51340ca370 2024-03-30T00:40:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f5139a00640 1 -- 172.21.15.66:0/3011649908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513406aee0 con 0x7f51340ca370 2024-03-30T00:40:15.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.789+0000 7f512be00640 1 -- 172.21.15.66:0/3011649908 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f512c01c360 con 0x7f51340ca370 2024-03-30T00:40:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.793+0000 7f512be00640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5114066690 0x7f5114068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.793+0000 7f512be00640 1 -- 172.21.15.66:0/3011649908 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f512c0843c0 con 0x7f51340ca370 2024-03-30T00:40:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.793+0000 7f5133e00640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5114066690 0x7f5114068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:15.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.793+0000 7f5133e00640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5114066690 0x7f5114068b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f51240ae000 tx=0x7f51240a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.797+0000 7f512be00640 1 -- 172.21.15.66:0/3011649908 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f513406aee0 con 0x7f51340ca370 2024-03-30T00:40:15.912 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:40:15.916 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:40:15.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.953+0000 7f5139a00640 1 -- 172.21.15.66:0/3011649908 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f513415a160 con 0x7f5114066690 2024-03-30T00:40:15.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.965+0000 7f512be00640 1 -- 172.21.15.66:0/3011649908 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345302 (secure 0 0 0) 0x7f51240bc090 con 0x7f5114066690 2024-03-30T00:40:15.976 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:15.978 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 -- 172.21.15.66:0/3011649908 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5114066690 msgr2=0x7f5114068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5114066690 0x7f5114068b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f51240ae000 tx=0x7f51240a7000 comp rx=0 tx=0).stop 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 -- 172.21.15.66:0/3011649908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ca370 msgr2=0x7f51341f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ca370 0x7f51341f63b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f512c00b9e0 tx=0x7f512c00bea0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 -- 172.21.15.66:0/3011649908 shutdown_connections 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5114066690 0x7f5114068b50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ca370 0x7f51341f63b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51340ade30 0x7f513415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 --2- 172.21.15.66:0/3011649908 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5134070cb0 0x7f51341504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 -- 172.21.15.66:0/3011649908 >> 172.21.15.66:0/3011649908 conn(0x7f51340ce9f0 msgr2=0x7f5134069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 -- 172.21.15.66:0/3011649908 shutdown_connections 2024-03-30T00:40:15.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:15.969+0000 7f5139a00640 1 -- 172.21.15.66:0/3011649908 wait complete. 2024-03-30T00:40:15.997 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-03-30T00:40:14.855664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235864,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738856,"statfs":{"total":773094113280,"available":772852588544,"internally_reserved":0,"allocated":3383296,"data_stored":2025532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002921"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184484+0000","last_change":"2024-03-30T00:39:50.768525+0000","last_active":"2024-03-30T00:39:55.184484+0000","last_peered":"2024-03-30T00:39:55.184484+0000","last_clean":"2024-03-30T00:39:55.184484+0000","last_became_active":"2024-03-30T00:39:50.768196+0000","last_became_peered":"2024-03-30T00:39:50.768196+0000","last_unstale":"2024-03-30T00:39:55.184484+0000","last_undegraded":"2024-03-30T00:39:55.184484+0000","last_fullsized":"2024-03-30T00:39:55.184484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:26.877416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168811+0000","last_change":"2024-03-30T00:39:51.184250+0000","last_active":"2024-03-30T00:39:55.168811+0000","last_peered":"2024-03-30T00:39:55.168811+0000","last_clean":"2024-03-30T00:39:55.168811+0000","last_became_active":"2024-03-30T00:39:51.184025+0000","last_became_peered":"2024-03-30T00:39:51.184025+0000","last_unstale":"2024-03-30T00:39:55.168811+0000","last_undegraded":"2024-03-30T00:39:55.168811+0000","last_fullsized":"2024-03-30T00:39:55.168811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:49.107527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571479+0000","last_change":"2024-03-30T00:39:55.177962+0000","last_active":"2024-03-30T00:39:55.571479+0000","last_peered":"2024-03-30T00:39:55.571479+0000","last_clean":"2024-03-30T00:39:55.571479+0000","last_became_active":"2024-03-30T00:39:55.177403+0000","last_became_peered":"2024-03-30T00:39:55.177403+0000","last_unstale":"2024-03-30T00:39:55.571479+0000","last_undegraded":"2024-03-30T00:39:55.571479+0000","last_fullsized":"2024-03-30T00:39:55.571479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:44:30.994000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168955+0000","last_change":"2024-03-30T00:39:50.765754+0000","last_active":"2024-03-30T00:39:55.168955+0000","last_peered":"2024-03-30T00:39:55.168955+0000","last_clean":"2024-03-30T00:39:55.168955+0000","last_became_active":"2024-03-30T00:39:50.764611+0000","last_became_peered":"2024-03-30T00:39:50.764611+0000","last_unstale":"2024-03-30T00:39:55.168955+0000","last_undegraded":"2024-03-30T00:39:55.168955+0000","last_fullsized":"2024-03-30T00:39:55.168955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:20.857915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184332+0000","last_change":"2024-03-30T00:39:51.180718+0000","last_active":"2024-03-30T00:39:55.184332+0000","last_peered":"2024-03-30T00:39:55.184332+0000","last_clean":"2024-03-30T00:39:55.184332+0000","last_became_active":"2024-03-30T00:39:51.175872+0000","last_became_peered":"2024-03-30T00:39:51.175872+0000","last_unstale":"2024-03-30T00:39:55.184332+0000","last_undegraded":"2024-03-30T00:39:55.184332+0000","last_fullsized":"2024-03-30T00:39:55.184332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272688+0000","last_change":"2024-03-30T00:39:55.184652+0000","last_active":"2024-03-30T00:39:57.272688+0000","last_peered":"2024-03-30T00:39:57.272688+0000","last_clean":"2024-03-30T00:39:57.272688+0000","last_became_active":"2024-03-30T00:39:55.184290+0000","last_became_peered":"2024-03-30T00:39:55.184290+0000","last_unstale":"2024-03-30T00:39:57.272688+0000","last_undegraded":"2024-03-30T00:39:57.272688+0000","last_fullsized":"2024-03-30T00:39:57.272688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:15:03.826593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163001+0000","last_change":"2024-03-30T00:39:50.771435+0000","last_active":"2024-03-30T00:39:55.163001+0000","last_peered":"2024-03-30T00:39:55.163001+0000","last_clean":"2024-03-30T00:39:55.163001+0000","last_became_active":"2024-03-30T00:39:50.771153+0000","last_became_peered":"2024-03-30T00:39:50.771153+0000","last_unstale":"2024-03-30T00:39:55.163001+0000","last_undegraded":"2024-03-30T00:39:55.163001+0000","last_fullsized":"2024-03-30T00:39:55.163001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:43.959952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170301+0000","last_change":"2024-03-30T00:39:51.183510+0000","last_active":"2024-03-30T00:39:55.170301+0000","last_peered":"2024-03-30T00:39:55.170301+0000","last_clean":"2024-03-30T00:39:55.170301+0000","last_became_active":"2024-03-30T00:39:51.183236+0000","last_became_peered":"2024-03-30T00:39:51.183236+0000","last_unstale":"2024-03-30T00:39:55.170301+0000","last_undegraded":"2024-03-30T00:39:55.170301+0000","last_fullsized":"2024-03-30T00:39:55.170301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273177+0000","last_change":"2024-03-30T00:39:55.179167+0000","last_active":"2024-03-30T00:39:57.273177+0000","last_peered":"2024-03-30T00:39:57.273177+0000","last_clean":"2024-03-30T00:39:57.273177+0000","last_became_active":"2024-03-30T00:39:55.178708+0000","last_became_peered":"2024-03-30T00:39:55.178708+0000","last_unstale":"2024-03-30T00:39:57.273177+0000","last_undegraded":"2024-03-30T00:39:57.273177+0000","last_fullsized":"2024-03-30T00:39:57.273177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161291+0000","last_change":"2024-03-30T00:39:50.773829+0000","last_active":"2024-03-30T00:39:55.161291+0000","last_peered":"2024-03-30T00:39:55.161291+0000","last_clean":"2024-03-30T00:39:55.161291+0000","last_became_active":"2024-03-30T00:39:50.773527+0000","last_became_peered":"2024-03-30T00:39:50.773527+0000","last_unstale":"2024-03-30T00:39:55.161291+0000","last_undegraded":"2024-03-30T00:39:55.161291+0000","last_fullsized":"2024-03-30T00:39:55.161291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.179506+0000","last_change":"2024-03-30T00:39:51.183689+0000","last_active":"2024-03-30T00:39:55.179506+0000","last_peered":"2024-03-30T00:39:55.179506+0000","last_clean":"2024-03-30T00:39:55.179506+0000","last_became_active":"2024-03-30T00:39:51.183421+0000","last_became_peered":"2024-03-30T00:39:51.183421+0000","last_unstale":"2024-03-30T00:39:55.179506+0000","last_undegraded":"2024-03-30T00:39:55.179506+0000","last_fullsized":"2024-03-30T00:39:55.179506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:54.542537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272671+0000","last_change":"2024-03-30T00:39:55.186210+0000","last_active":"2024-03-30T00:39:57.272671+0000","last_peered":"2024-03-30T00:39:57.272671+0000","last_clean":"2024-03-30T00:39:57.272671+0000","last_became_active":"2024-03-30T00:39:55.185849+0000","last_became_peered":"2024-03-30T00:39:55.185849+0000","last_unstale":"2024-03-30T00:39:57.272671+0000","last_undegraded":"2024-03-30T00:39:57.272671+0000","last_fullsized":"2024-03-30T00:39:57.272671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:45:20.370665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171140+0000","last_change":"2024-03-30T00:39:51.172454+0000","last_active":"2024-03-30T00:39:55.171140+0000","last_peered":"2024-03-30T00:39:55.171140+0000","last_clean":"2024-03-30T00:39:55.171140+0000","last_became_active":"2024-03-30T00:39:51.172196+0000","last_became_peered":"2024-03-30T00:39:51.172196+0000","last_unstale":"2024-03-30T00:39:55.171140+0000","last_undegraded":"2024-03-30T00:39:55.171140+0000","last_fullsized":"2024-03-30T00:39:55.171140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:02.372510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161442+0000","last_change":"2024-03-30T00:39:50.764668+0000","last_active":"2024-03-30T00:39:55.161442+0000","last_peered":"2024-03-30T00:39:55.161442+0000","last_clean":"2024-03-30T00:39:55.161442+0000","last_became_active":"2024-03-30T00:39:50.763661+0000","last_became_peered":"2024-03-30T00:39:50.763661+0000","last_unstale":"2024-03-30T00:39:55.161442+0000","last_undegraded":"2024-03-30T00:39:55.161442+0000","last_fullsized":"2024-03-30T00:39:55.161442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273301+0000","last_change":"2024-03-30T00:39:55.190348+0000","last_active":"2024-03-30T00:39:57.273301+0000","last_peered":"2024-03-30T00:39:57.273301+0000","last_clean":"2024-03-30T00:39:57.273301+0000","last_became_active":"2024-03-30T00:39:55.189987+0000","last_became_peered":"2024-03-30T00:39:55.189987+0000","last_unstale":"2024-03-30T00:39:57.273301+0000","last_undegraded":"2024-03-30T00:39:57.273301+0000","last_fullsized":"2024-03-30T00:39:57.273301+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:15:12.303669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169221+0000","last_change":"2024-03-30T00:39:51.168741+0000","last_active":"2024-03-30T00:39:55.169221+0000","last_peered":"2024-03-30T00:39:55.169221+0000","last_clean":"2024-03-30T00:39:55.169221+0000","last_became_active":"2024-03-30T00:39:51.168476+0000","last_became_peered":"2024-03-30T00:39:51.168476+0000","last_unstale":"2024-03-30T00:39:55.169221+0000","last_undegraded":"2024-03-30T00:39:55.169221+0000","last_fullsized":"2024-03-30T00:39:55.169221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:10:20.544219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172219+0000","last_change":"2024-03-30T00:39:50.766156+0000","last_active":"2024-03-30T00:39:55.172219+0000","last_peered":"2024-03-30T00:39:55.172219+0000","last_clean":"2024-03-30T00:39:55.172219+0000","last_became_active":"2024-03-30T00:39:50.765730+0000","last_became_peered":"2024-03-30T00:39:50.765730+0000","last_unstale":"2024-03-30T00:39:55.172219+0000","last_undegraded":"2024-03-30T00:39:55.172219+0000","last_fullsized":"2024-03-30T00:39:55.172219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:27.304749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272886+0000","last_change":"2024-03-30T00:39:55.180712+0000","last_active":"2024-03-30T00:39:57.272886+0000","last_peered":"2024-03-30T00:39:57.272886+0000","last_clean":"2024-03-30T00:39:57.272886+0000","last_became_active":"2024-03-30T00:39:55.179741+0000","last_became_peered":"2024-03-30T00:39:55.179741+0000","last_unstale":"2024-03-30T00:39:57.272886+0000","last_undegraded":"2024-03-30T00:39:57.272886+0000","last_fullsized":"2024-03-30T00:39:57.272886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.166309+0000","last_change":"2024-03-30T00:39:51.182604+0000","last_active":"2024-03-30T00:39:55.166309+0000","last_peered":"2024-03-30T00:39:55.166309+0000","last_clean":"2024-03-30T00:39:55.166309+0000","last_became_active":"2024-03-30T00:39:51.182323+0000","last_became_peered":"2024-03-30T00:39:51.182323+0000","last_unstale":"2024-03-30T00:39:55.166309+0000","last_undegraded":"2024-03-30T00:39:55.166309+0000","last_fullsized":"2024-03-30T00:39:55.166309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172036+0000","last_change":"2024-03-30T00:39:50.758774+0000","last_active":"2024-03-30T00:39:55.172036+0000","last_peered":"2024-03-30T00:39:55.172036+0000","last_clean":"2024-03-30T00:39:55.172036+0000","last_became_active":"2024-03-30T00:39:50.758508+0000","last_became_peered":"2024-03-30T00:39:50.758508+0000","last_unstale":"2024-03-30T00:39:55.172036+0000","last_undegraded":"2024-03-30T00:39:55.172036+0000","last_fullsized":"2024-03-30T00:39:55.172036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.129001+0000","last_change":"2024-03-30T00:39:55.182422+0000","last_active":"2024-03-30T00:39:57.129001+0000","last_peered":"2024-03-30T00:39:57.129001+0000","last_clean":"2024-03-30T00:39:57.129001+0000","last_became_active":"2024-03-30T00:39:55.182026+0000","last_became_peered":"2024-03-30T00:39:55.182026+0000","last_unstale":"2024-03-30T00:39:57.129001+0000","last_undegraded":"2024-03-30T00:39:57.129001+0000","last_fullsized":"2024-03-30T00:39:57.129001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:18:37.750989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160976+0000","last_change":"2024-03-30T00:39:51.161273+0000","last_active":"2024-03-30T00:39:55.160976+0000","last_peered":"2024-03-30T00:39:55.160976+0000","last_clean":"2024-03-30T00:39:55.160976+0000","last_became_active":"2024-03-30T00:39:51.156256+0000","last_became_peered":"2024-03-30T00:39:51.156256+0000","last_unstale":"2024-03-30T00:39:55.160976+0000","last_undegraded":"2024-03-30T00:39:55.160976+0000","last_fullsized":"2024-03-30T00:39:55.160976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:17.102921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159893+0000","last_change":"2024-03-30T00:39:50.768022+0000","last_active":"2024-03-30T00:39:55.159893+0000","last_peered":"2024-03-30T00:39:55.159893+0000","last_clean":"2024-03-30T00:39:55.159893+0000","last_became_active":"2024-03-30T00:39:50.767791+0000","last_became_peered":"2024-03-30T00:39:50.767791+0000","last_unstale":"2024-03-30T00:39:55.159893+0000","last_undegraded":"2024-03-30T00:39:55.159893+0000","last_fullsized":"2024-03-30T00:39:55.159893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:16.100201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273213+0000","last_change":"2024-03-30T00:39:55.176378+0000","last_active":"2024-03-30T00:39:57.273213+0000","last_peered":"2024-03-30T00:39:57.273213+0000","last_clean":"2024-03-30T00:39:57.273213+0000","last_became_active":"2024-03-30T00:39:55.176090+0000","last_became_peered":"2024-03-30T00:39:55.176090+0000","last_unstale":"2024-03-30T00:39:57.273213+0000","last_undegraded":"2024-03-30T00:39:57.273213+0000","last_fullsized":"2024-03-30T00:39:57.273213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184191+0000","last_change":"2024-03-30T00:39:51.181030+0000","last_active":"2024-03-30T00:39:55.184191+0000","last_peered":"2024-03-30T00:39:55.184191+0000","last_clean":"2024-03-30T00:39:55.184191+0000","last_became_active":"2024-03-30T00:39:51.180794+0000","last_became_peered":"2024-03-30T00:39:51.180794+0000","last_unstale":"2024-03-30T00:39:55.184191+0000","last_undegraded":"2024-03-30T00:39:55.184191+0000","last_fullsized":"2024-03-30T00:39:55.184191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:02.372510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172255+0000","last_change":"2024-03-30T00:39:50.771845+0000","last_active":"2024-03-30T00:39:55.172255+0000","last_peered":"2024-03-30T00:39:55.172255+0000","last_clean":"2024-03-30T00:39:55.172255+0000","last_became_active":"2024-03-30T00:39:50.771049+0000","last_became_peered":"2024-03-30T00:39:50.771049+0000","last_unstale":"2024-03-30T00:39:55.172255+0000","last_undegraded":"2024-03-30T00:39:55.172255+0000","last_fullsized":"2024-03-30T00:39:55.172255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:20.857915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272571+0000","last_change":"2024-03-30T00:39:55.174666+0000","last_active":"2024-03-30T00:39:57.272571+0000","last_peered":"2024-03-30T00:39:57.272571+0000","last_clean":"2024-03-30T00:39:57.272571+0000","last_became_active":"2024-03-30T00:39:55.174259+0000","last_became_peered":"2024-03-30T00:39:55.174259+0000","last_unstale":"2024-03-30T00:39:57.272571+0000","last_undegraded":"2024-03-30T00:39:57.272571+0000","last_fullsized":"2024-03-30T00:39:57.272571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165004+0000","last_change":"2024-03-30T00:39:51.178352+0000","last_active":"2024-03-30T00:39:55.165004+0000","last_peered":"2024-03-30T00:39:55.165004+0000","last_clean":"2024-03-30T00:39:55.165004+0000","last_became_active":"2024-03-30T00:39:51.178080+0000","last_became_peered":"2024-03-30T00:39:51.178080+0000","last_unstale":"2024-03-30T00:39:55.165004+0000","last_undegraded":"2024-03-30T00:39:55.165004+0000","last_fullsized":"2024-03-30T00:39:55.165004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172086+0000","last_change":"2024-03-30T00:39:50.768206+0000","last_active":"2024-03-30T00:39:55.172086+0000","last_peered":"2024-03-30T00:39:55.172086+0000","last_clean":"2024-03-30T00:39:55.172086+0000","last_became_active":"2024-03-30T00:39:50.767891+0000","last_became_peered":"2024-03-30T00:39:50.767891+0000","last_unstale":"2024-03-30T00:39:55.172086+0000","last_undegraded":"2024-03-30T00:39:55.172086+0000","last_fullsized":"2024-03-30T00:39:55.172086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:39.163294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571335+0000","last_change":"2024-03-30T00:39:55.192391+0000","last_active":"2024-03-30T00:39:55.571335+0000","last_peered":"2024-03-30T00:39:55.571335+0000","last_clean":"2024-03-30T00:39:55.571335+0000","last_became_active":"2024-03-30T00:39:55.189816+0000","last_became_peered":"2024-03-30T00:39:55.189816+0000","last_unstale":"2024-03-30T00:39:55.571335+0000","last_undegraded":"2024-03-30T00:39:55.571335+0000","last_fullsized":"2024-03-30T00:39:55.571335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:05:10.673253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171550+0000","last_change":"2024-03-30T00:39:51.161793+0000","last_active":"2024-03-30T00:39:55.171550+0000","last_peered":"2024-03-30T00:39:55.171550+0000","last_clean":"2024-03-30T00:39:55.171550+0000","last_became_active":"2024-03-30T00:39:51.161523+0000","last_became_peered":"2024-03-30T00:39:51.161523+0000","last_unstale":"2024-03-30T00:39:55.171550+0000","last_undegraded":"2024-03-30T00:39:55.171550+0000","last_fullsized":"2024-03-30T00:39:55.171550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163175+0000","last_change":"2024-03-30T00:39:50.769035+0000","last_active":"2024-03-30T00:39:55.163175+0000","last_peered":"2024-03-30T00:39:55.163175+0000","last_clean":"2024-03-30T00:39:55.163175+0000","last_became_active":"2024-03-30T00:39:50.768789+0000","last_became_peered":"2024-03-30T00:39:50.768789+0000","last_unstale":"2024-03-30T00:39:55.163175+0000","last_undegraded":"2024-03-30T00:39:55.163175+0000","last_fullsized":"2024-03-30T00:39:55.163175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:39.163294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273625+0000","last_change":"2024-03-30T00:39:55.200272+0000","last_active":"2024-03-30T00:39:57.273625+0000","last_peered":"2024-03-30T00:39:57.273625+0000","last_clean":"2024-03-30T00:39:57.273625+0000","last_became_active":"2024-03-30T00:39:55.199688+0000","last_became_peered":"2024-03-30T00:39:55.199688+0000","last_unstale":"2024-03-30T00:39:57.273625+0000","last_undegraded":"2024-03-30T00:39:57.273625+0000","last_fullsized":"2024-03-30T00:39:57.273625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:30.942593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.174408+0000","last_change":"2024-03-30T00:39:51.174886+0000","last_active":"2024-03-30T00:39:55.174408+0000","last_peered":"2024-03-30T00:39:55.174408+0000","last_clean":"2024-03-30T00:39:55.174408+0000","last_became_active":"2024-03-30T00:39:51.174274+0000","last_became_peered":"2024-03-30T00:39:51.174274+0000","last_unstale":"2024-03-30T00:39:55.174408+0000","last_undegraded":"2024-03-30T00:39:55.174408+0000","last_fullsized":"2024-03-30T00:39:55.174408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:24:05.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171643+0000","last_change":"2024-03-30T00:39:50.769092+0000","last_active":"2024-03-30T00:39:55.171643+0000","last_peered":"2024-03-30T00:39:55.171643+0000","last_clean":"2024-03-30T00:39:55.171643+0000","last_became_active":"2024-03-30T00:39:50.768750+0000","last_became_peered":"2024-03-30T00:39:50.768750+0000","last_unstale":"2024-03-30T00:39:55.171643+0000","last_undegraded":"2024-03-30T00:39:55.171643+0000","last_fullsized":"2024-03-30T00:39:55.171643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128439+0000","last_change":"2024-03-30T00:39:55.187382+0000","last_active":"2024-03-30T00:39:57.128439+0000","last_peered":"2024-03-30T00:39:57.128439+0000","last_clean":"2024-03-30T00:39:57.128439+0000","last_became_active":"2024-03-30T00:39:55.186804+0000","last_became_peered":"2024-03-30T00:39:55.186804+0000","last_unstale":"2024-03-30T00:39:57.128439+0000","last_undegraded":"2024-03-30T00:39:57.128439+0000","last_fullsized":"2024-03-30T00:39:57.128439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161837+0000","last_change":"2024-03-30T00:39:51.172750+0000","last_active":"2024-03-30T00:39:55.161837+0000","last_peered":"2024-03-30T00:39:55.161837+0000","last_clean":"2024-03-30T00:39:55.161837+0000","last_became_active":"2024-03-30T00:39:51.172456+0000","last_became_peered":"2024-03-30T00:39:51.172456+0000","last_unstale":"2024-03-30T00:39:55.161837+0000","last_undegraded":"2024-03-30T00:39:55.161837+0000","last_fullsized":"2024-03-30T00:39:55.161837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:40.166014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168217+0000","last_change":"2024-03-30T00:39:50.775126+0000","last_active":"2024-03-30T00:39:55.168217+0000","last_peered":"2024-03-30T00:39:55.168217+0000","last_clean":"2024-03-30T00:39:55.168217+0000","last_became_active":"2024-03-30T00:39:50.773078+0000","last_became_peered":"2024-03-30T00:39:50.773078+0000","last_unstale":"2024-03-30T00:39:55.168217+0000","last_undegraded":"2024-03-30T00:39:55.168217+0000","last_fullsized":"2024-03-30T00:39:55.168217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128669+0000","last_change":"2024-03-30T00:39:55.182489+0000","last_active":"2024-03-30T00:39:57.128669+0000","last_peered":"2024-03-30T00:39:57.128669+0000","last_clean":"2024-03-30T00:39:57.128669+0000","last_became_active":"2024-03-30T00:39:55.181876+0000","last_became_peered":"2024-03-30T00:39:55.181876+0000","last_unstale":"2024-03-30T00:39:57.128669+0000","last_undegraded":"2024-03-30T00:39:57.128669+0000","last_fullsized":"2024-03-30T00:39:57.128669+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171723+0000","last_change":"2024-03-30T00:39:51.157937+0000","last_active":"2024-03-30T00:39:55.171723+0000","last_peered":"2024-03-30T00:39:55.171723+0000","last_clean":"2024-03-30T00:39:55.171723+0000","last_became_active":"2024-03-30T00:39:51.155936+0000","last_became_peered":"2024-03-30T00:39:51.155936+0000","last_unstale":"2024-03-30T00:39:55.171723+0000","last_undegraded":"2024-03-30T00:39:55.171723+0000","last_fullsized":"2024-03-30T00:39:55.171723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159139+0000","last_change":"2024-03-30T00:39:50.765709+0000","last_active":"2024-03-30T00:39:55.159139+0000","last_peered":"2024-03-30T00:39:55.159139+0000","last_clean":"2024-03-30T00:39:55.159139+0000","last_became_active":"2024-03-30T00:39:50.765450+0000","last_became_peered":"2024-03-30T00:39:50.765450+0000","last_unstale":"2024-03-30T00:39:55.159139+0000","last_undegraded":"2024-03-30T00:39:55.159139+0000","last_fullsized":"2024-03-30T00:39:55.159139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.129028+0000","last_change":"2024-03-30T00:39:55.193764+0000","last_active":"2024-03-30T00:39:57.129028+0000","last_peered":"2024-03-30T00:39:57.129028+0000","last_clean":"2024-03-30T00:39:57.129028+0000","last_became_active":"2024-03-30T00:39:55.188761+0000","last_became_peered":"2024-03-30T00:39:55.188761+0000","last_unstale":"2024-03-30T00:39:57.129028+0000","last_undegraded":"2024-03-30T00:39:57.129028+0000","last_fullsized":"2024-03-30T00:39:57.129028+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.182494+0000","last_change":"2024-03-30T00:39:51.176755+0000","last_active":"2024-03-30T00:39:55.182494+0000","last_peered":"2024-03-30T00:39:55.182494+0000","last_clean":"2024-03-30T00:39:55.182494+0000","last_became_active":"2024-03-30T00:39:51.176507+0000","last_became_peered":"2024-03-30T00:39:51.176507+0000","last_unstale":"2024-03-30T00:39:55.182494+0000","last_undegraded":"2024-03-30T00:39:55.182494+0000","last_fullsized":"2024-03-30T00:39:55.182494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160037+0000","last_change":"2024-03-30T00:39:50.763226+0000","last_active":"2024-03-30T00:39:55.160037+0000","last_peered":"2024-03-30T00:39:55.160037+0000","last_clean":"2024-03-30T00:39:55.160037+0000","last_became_active":"2024-03-30T00:39:50.762972+0000","last_became_peered":"2024-03-30T00:39:50.762972+0000","last_unstale":"2024-03-30T00:39:55.160037+0000","last_undegraded":"2024-03-30T00:39:55.160037+0000","last_fullsized":"2024-03-30T00:39:55.160037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:27.304749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273996+0000","last_change":"2024-03-30T00:39:55.198804+0000","last_active":"2024-03-30T00:39:57.273996+0000","last_peered":"2024-03-30T00:39:57.273996+0000","last_clean":"2024-03-30T00:39:57.273996+0000","last_became_active":"2024-03-30T00:39:55.198382+0000","last_became_peered":"2024-03-30T00:39:55.198382+0000","last_unstale":"2024-03-30T00:39:57.273996+0000","last_undegraded":"2024-03-30T00:39:57.273996+0000","last_fullsized":"2024-03-30T00:39:57.273996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165312+0000","last_change":"2024-03-30T00:39:51.176690+0000","last_active":"2024-03-30T00:39:55.165312+0000","last_peered":"2024-03-30T00:39:55.165312+0000","last_clean":"2024-03-30T00:39:55.165312+0000","last_became_active":"2024-03-30T00:39:51.175614+0000","last_became_peered":"2024-03-30T00:39:51.175614+0000","last_unstale":"2024-03-30T00:39:55.165312+0000","last_undegraded":"2024-03-30T00:39:55.165312+0000","last_fullsized":"2024-03-30T00:39:55.165312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:13.959522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.176056+0000","last_change":"2024-03-30T00:39:50.766293+0000","last_active":"2024-03-30T00:39:55.176056+0000","last_peered":"2024-03-30T00:39:55.176056+0000","last_clean":"2024-03-30T00:39:55.176056+0000","last_became_active":"2024-03-30T00:39:50.765977+0000","last_became_peered":"2024-03-30T00:39:50.765977+0000","last_unstale":"2024-03-30T00:39:55.176056+0000","last_undegraded":"2024-03-30T00:39:55.176056+0000","last_fullsized":"2024-03-30T00:39:55.176056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160226+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:39:55.160226+0000","last_peered":"2024-03-30T00:39:55.160226+0000","last_clean":"2024-03-30T00:39:55.160226+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:39:55.160226+0000","last_undegraded":"2024-03-30T00:39:55.160226+0000","last_fullsized":"2024-03-30T00:39:55.160226+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572655+0000","last_change":"2024-03-30T00:39:55.202116+0000","last_active":"2024-03-30T00:39:55.572655+0000","last_peered":"2024-03-30T00:39:55.572655+0000","last_clean":"2024-03-30T00:39:55.572655+0000","last_became_active":"2024-03-30T00:39:55.201638+0000","last_became_peered":"2024-03-30T00:39:55.201638+0000","last_unstale":"2024-03-30T00:39:55.572655+0000","last_undegraded":"2024-03-30T00:39:55.572655+0000","last_fullsized":"2024-03-30T00:39:55.572655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:46:36.593302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162909+0000","last_change":"2024-03-30T00:39:51.176768+0000","last_active":"2024-03-30T00:39:55.162909+0000","last_peered":"2024-03-30T00:39:55.162909+0000","last_clean":"2024-03-30T00:39:55.162909+0000","last_became_active":"2024-03-30T00:39:51.176215+0000","last_became_peered":"2024-03-30T00:39:51.176215+0000","last_unstale":"2024-03-30T00:39:55.162909+0000","last_undegraded":"2024-03-30T00:39:55.162909+0000","last_fullsized":"2024-03-30T00:39:55.162909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:26.397679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159355+0000","last_change":"2024-03-30T00:39:50.763227+0000","last_active":"2024-03-30T00:39:55.159355+0000","last_peered":"2024-03-30T00:39:55.159355+0000","last_clean":"2024-03-30T00:39:55.159355+0000","last_became_active":"2024-03-30T00:39:50.762934+0000","last_became_peered":"2024-03-30T00:39:50.762934+0000","last_unstale":"2024-03-30T00:39:55.159355+0000","last_undegraded":"2024-03-30T00:39:55.159355+0000","last_fullsized":"2024-03-30T00:39:55.159355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572469+0000","last_change":"2024-03-30T00:39:55.195709+0000","last_active":"2024-03-30T00:39:55.572469+0000","last_peered":"2024-03-30T00:39:55.572469+0000","last_clean":"2024-03-30T00:39:55.572469+0000","last_became_active":"2024-03-30T00:39:55.194810+0000","last_became_peered":"2024-03-30T00:39:55.194810+0000","last_unstale":"2024-03-30T00:39:55.572469+0000","last_undegraded":"2024-03-30T00:39:55.572469+0000","last_fullsized":"2024-03-30T00:39:55.572469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:30.942593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170424+0000","last_change":"2024-03-30T00:39:51.182838+0000","last_active":"2024-03-30T00:39:55.170424+0000","last_peered":"2024-03-30T00:39:55.170424+0000","last_clean":"2024-03-30T00:39:55.170424+0000","last_became_active":"2024-03-30T00:39:51.178980+0000","last_became_peered":"2024-03-30T00:39:51.178980+0000","last_unstale":"2024-03-30T00:39:55.170424+0000","last_undegraded":"2024-03-30T00:39:55.170424+0000","last_fullsized":"2024-03-30T00:39:55.170424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:13.959522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172688+0000","last_change":"2024-03-30T00:39:50.755295+0000","last_active":"2024-03-30T00:39:55.172688+0000","last_peered":"2024-03-30T00:39:55.172688+0000","last_clean":"2024-03-30T00:39:55.172688+0000","last_became_active":"2024-03-30T00:39:50.754957+0000","last_became_peered":"2024-03-30T00:39:50.754957+0000","last_unstale":"2024-03-30T00:39:55.172688+0000","last_undegraded":"2024-03-30T00:39:55.172688+0000","last_fullsized":"2024-03-30T00:39:55.172688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:16.100201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273833+0000","last_change":"2024-03-30T00:39:55.199975+0000","last_active":"2024-03-30T00:39:57.273833+0000","last_peered":"2024-03-30T00:39:57.273833+0000","last_clean":"2024-03-30T00:39:57.273833+0000","last_became_active":"2024-03-30T00:39:55.199191+0000","last_became_peered":"2024-03-30T00:39:55.199191+0000","last_unstale":"2024-03-30T00:39:57.273833+0000","last_undegraded":"2024-03-30T00:39:57.273833+0000","last_fullsized":"2024-03-30T00:39:57.273833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163333+0000","last_change":"2024-03-30T00:39:51.163933+0000","last_active":"2024-03-30T00:39:55.163333+0000","last_peered":"2024-03-30T00:39:55.163333+0000","last_clean":"2024-03-30T00:39:55.163333+0000","last_became_active":"2024-03-30T00:39:51.163663+0000","last_became_peered":"2024-03-30T00:39:51.163663+0000","last_unstale":"2024-03-30T00:39:55.163333+0000","last_undegraded":"2024-03-30T00:39:55.163333+0000","last_fullsized":"2024-03-30T00:39:55.163333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:41.330935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159267+0000","last_change":"2024-03-30T00:39:50.766833+0000","last_active":"2024-03-30T00:39:55.159267+0000","last_peered":"2024-03-30T00:39:55.159267+0000","last_clean":"2024-03-30T00:39:55.159267+0000","last_became_active":"2024-03-30T00:39:50.766534+0000","last_became_peered":"2024-03-30T00:39:50.766534+0000","last_unstale":"2024-03-30T00:39:55.159267+0000","last_undegraded":"2024-03-30T00:39:55.159267+0000","last_fullsized":"2024-03-30T00:39:55.159267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:56.358073+0000","last_change":"2024-03-30T00:39:55.184862+0000","last_active":"2024-03-30T00:39:56.358073+0000","last_peered":"2024-03-30T00:39:56.358073+0000","last_clean":"2024-03-30T00:39:56.358073+0000","last_became_active":"2024-03-30T00:39:55.184374+0000","last_became_peered":"2024-03-30T00:39:55.184374+0000","last_unstale":"2024-03-30T00:39:56.358073+0000","last_undegraded":"2024-03-30T00:39:56.358073+0000","last_fullsized":"2024-03-30T00:39:56.358073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163385+0000","last_change":"2024-03-30T00:39:51.162489+0000","last_active":"2024-03-30T00:39:55.163385+0000","last_peered":"2024-03-30T00:39:55.163385+0000","last_clean":"2024-03-30T00:39:55.163385+0000","last_became_active":"2024-03-30T00:39:51.162205+0000","last_became_peered":"2024-03-30T00:39:51.162205+0000","last_unstale":"2024-03-30T00:39:55.163385+0000","last_undegraded":"2024-03-30T00:39:55.163385+0000","last_fullsized":"2024-03-30T00:39:55.163385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:36.973826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162109+0000","last_change":"2024-03-30T00:39:50.768761+0000","last_active":"2024-03-30T00:39:55.162109+0000","last_peered":"2024-03-30T00:39:55.162109+0000","last_clean":"2024-03-30T00:39:55.162109+0000","last_became_active":"2024-03-30T00:39:50.768372+0000","last_became_peered":"2024-03-30T00:39:50.768372+0000","last_unstale":"2024-03-30T00:39:55.162109+0000","last_undegraded":"2024-03-30T00:39:55.162109+0000","last_fullsized":"2024-03-30T00:39:55.162109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:35:25.209340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272560+0000","last_change":"2024-03-30T00:39:55.177966+0000","last_active":"2024-03-30T00:39:57.272560+0000","last_peered":"2024-03-30T00:39:57.272560+0000","last_clean":"2024-03-30T00:39:57.272560+0000","last_became_active":"2024-03-30T00:39:55.177485+0000","last_became_peered":"2024-03-30T00:39:55.177485+0000","last_unstale":"2024-03-30T00:39:57.272560+0000","last_undegraded":"2024-03-30T00:39:57.272560+0000","last_fullsized":"2024-03-30T00:39:57.272560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161004+0000","last_change":"2024-03-30T00:39:51.157566+0000","last_active":"2024-03-30T00:39:55.161004+0000","last_peered":"2024-03-30T00:39:55.161004+0000","last_clean":"2024-03-30T00:39:55.161004+0000","last_became_active":"2024-03-30T00:39:51.157243+0000","last_became_peered":"2024-03-30T00:39:51.157243+0000","last_unstale":"2024-03-30T00:39:55.161004+0000","last_undegraded":"2024-03-30T00:39:55.161004+0000","last_fullsized":"2024-03-30T00:39:55.161004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:35.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159067+0000","last_change":"2024-03-30T00:39:50.760164+0000","last_active":"2024-03-30T00:39:55.159067+0000","last_peered":"2024-03-30T00:39:55.159067+0000","last_clean":"2024-03-30T00:39:55.159067+0000","last_became_active":"2024-03-30T00:39:50.759887+0000","last_became_peered":"2024-03-30T00:39:50.759887+0000","last_unstale":"2024-03-30T00:39:55.159067+0000","last_undegraded":"2024-03-30T00:39:55.159067+0000","last_fullsized":"2024-03-30T00:39:55.159067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273934+0000","last_change":"2024-03-30T00:39:55.196574+0000","last_active":"2024-03-30T00:39:57.273934+0000","last_peered":"2024-03-30T00:39:57.273934+0000","last_clean":"2024-03-30T00:39:57.273934+0000","last_became_active":"2024-03-30T00:39:55.196182+0000","last_became_peered":"2024-03-30T00:39:55.196182+0000","last_unstale":"2024-03-30T00:39:57.273934+0000","last_undegraded":"2024-03-30T00:39:57.273934+0000","last_fullsized":"2024-03-30T00:39:57.273934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170724+0000","last_change":"2024-03-30T00:39:51.167097+0000","last_active":"2024-03-30T00:39:55.170724+0000","last_peered":"2024-03-30T00:39:55.170724+0000","last_clean":"2024-03-30T00:39:55.170724+0000","last_became_active":"2024-03-30T00:39:51.166629+0000","last_became_peered":"2024-03-30T00:39:51.166629+0000","last_unstale":"2024-03-30T00:39:55.170724+0000","last_undegraded":"2024-03-30T00:39:55.170724+0000","last_fullsized":"2024-03-30T00:39:55.170724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170864+0000","last_change":"2024-03-30T00:39:50.767642+0000","last_active":"2024-03-30T00:39:55.170864+0000","last_peered":"2024-03-30T00:39:55.170864+0000","last_clean":"2024-03-30T00:39:55.170864+0000","last_became_active":"2024-03-30T00:39:50.767166+0000","last_became_peered":"2024-03-30T00:39:50.767166+0000","last_unstale":"2024-03-30T00:39:55.170864+0000","last_undegraded":"2024-03-30T00:39:55.170864+0000","last_fullsized":"2024-03-30T00:39:55.170864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572362+0000","last_change":"2024-03-30T00:39:55.192042+0000","last_active":"2024-03-30T00:39:55.572362+0000","last_peered":"2024-03-30T00:39:55.572362+0000","last_clean":"2024-03-30T00:39:55.572362+0000","last_became_active":"2024-03-30T00:39:55.190080+0000","last_became_peered":"2024-03-30T00:39:55.190080+0000","last_unstale":"2024-03-30T00:39:55.572362+0000","last_undegraded":"2024-03-30T00:39:55.572362+0000","last_fullsized":"2024-03-30T00:39:55.572362+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:25.792128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161353+0000","last_change":"2024-03-30T00:39:51.161405+0000","last_active":"2024-03-30T00:39:55.161353+0000","last_peered":"2024-03-30T00:39:55.161353+0000","last_clean":"2024-03-30T00:39:55.161353+0000","last_became_active":"2024-03-30T00:39:51.161053+0000","last_became_peered":"2024-03-30T00:39:51.161053+0000","last_unstale":"2024-03-30T00:39:55.161353+0000","last_undegraded":"2024-03-30T00:39:55.161353+0000","last_fullsized":"2024-03-30T00:39:55.161353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:49.107527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168130+0000","last_change":"2024-03-30T00:39:50.774925+0000","last_active":"2024-03-30T00:39:55.168130+0000","last_peered":"2024-03-30T00:39:55.168130+0000","last_clean":"2024-03-30T00:39:55.168130+0000","last_became_active":"2024-03-30T00:39:50.773221+0000","last_became_peered":"2024-03-30T00:39:50.773221+0000","last_unstale":"2024-03-30T00:39:55.168130+0000","last_undegraded":"2024-03-30T00:39:55.168130+0000","last_fullsized":"2024-03-30T00:39:55.168130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572142+0000","last_change":"2024-03-30T00:39:55.191576+0000","last_active":"2024-03-30T00:39:55.572142+0000","last_peered":"2024-03-30T00:39:55.572142+0000","last_clean":"2024-03-30T00:39:55.572142+0000","last_became_active":"2024-03-30T00:39:55.191188+0000","last_became_peered":"2024-03-30T00:39:55.191188+0000","last_unstale":"2024-03-30T00:39:55.572142+0000","last_undegraded":"2024-03-30T00:39:55.572142+0000","last_fullsized":"2024-03-30T00:39:55.572142+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:58.020154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184454+0000","last_change":"2024-03-30T00:39:51.169704+0000","last_active":"2024-03-30T00:39:55.184454+0000","last_peered":"2024-03-30T00:39:55.184454+0000","last_clean":"2024-03-30T00:39:55.184454+0000","last_became_active":"2024-03-30T00:39:51.168350+0000","last_became_peered":"2024-03-30T00:39:51.168350+0000","last_unstale":"2024-03-30T00:39:55.184454+0000","last_undegraded":"2024-03-30T00:39:55.184454+0000","last_fullsized":"2024-03-30T00:39:55.184454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183870+0000","last_change":"2024-03-30T00:39:50.760857+0000","last_active":"2024-03-30T00:39:55.183870+0000","last_peered":"2024-03-30T00:39:55.183870+0000","last_clean":"2024-03-30T00:39:55.183870+0000","last_became_active":"2024-03-30T00:39:50.760607+0000","last_became_peered":"2024-03-30T00:39:50.760607+0000","last_unstale":"2024-03-30T00:39:55.183870+0000","last_undegraded":"2024-03-30T00:39:55.183870+0000","last_fullsized":"2024-03-30T00:39:55.183870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:56.358741+0000","last_change":"2024-03-30T00:39:55.188289+0000","last_active":"2024-03-30T00:39:56.358741+0000","last_peered":"2024-03-30T00:39:56.358741+0000","last_clean":"2024-03-30T00:39:56.358741+0000","last_became_active":"2024-03-30T00:39:55.187869+0000","last_became_peered":"2024-03-30T00:39:55.187869+0000","last_unstale":"2024-03-30T00:39:56.358741+0000","last_undegraded":"2024-03-30T00:39:56.358741+0000","last_fullsized":"2024-03-30T00:39:56.358741+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162815+0000","last_change":"2024-03-30T00:39:51.177240+0000","last_active":"2024-03-30T00:39:55.162815+0000","last_peered":"2024-03-30T00:39:55.162815+0000","last_clean":"2024-03-30T00:39:55.162815+0000","last_became_active":"2024-03-30T00:39:51.176388+0000","last_became_peered":"2024-03-30T00:39:51.176388+0000","last_unstale":"2024-03-30T00:39:55.162815+0000","last_undegraded":"2024-03-30T00:39:55.162815+0000","last_fullsized":"2024-03-30T00:39:55.162815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:37:05.851113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:40:01.400156+0000","last_change":"2024-03-30T00:39:50.766631+0000","last_active":"2024-03-30T00:40:01.400156+0000","last_peered":"2024-03-30T00:40:01.400156+0000","last_clean":"2024-03-30T00:40:01.400156+0000","last_became_active":"2024-03-30T00:39:50.766362+0000","last_became_peered":"2024-03-30T00:39:50.766362+0000","last_unstale":"2024-03-30T00:40:01.400156+0000","last_undegraded":"2024-03-30T00:40:01.400156+0000","last_fullsized":"2024-03-30T00:40:01.400156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128582+0000","last_change":"2024-03-30T00:39:55.187191+0000","last_active":"2024-03-30T00:39:57.128582+0000","last_peered":"2024-03-30T00:39:57.128582+0000","last_clean":"2024-03-30T00:39:57.128582+0000","last_became_active":"2024-03-30T00:39:55.186712+0000","last_became_peered":"2024-03-30T00:39:55.186712+0000","last_unstale":"2024-03-30T00:39:57.128582+0000","last_undegraded":"2024-03-30T00:39:57.128582+0000","last_fullsized":"2024-03-30T00:39:57.128582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162239+0000","last_change":"2024-03-30T00:39:51.154183+0000","last_active":"2024-03-30T00:39:55.162239+0000","last_peered":"2024-03-30T00:39:55.162239+0000","last_clean":"2024-03-30T00:39:55.162239+0000","last_became_active":"2024-03-30T00:39:51.153909+0000","last_became_peered":"2024-03-30T00:39:51.153909+0000","last_unstale":"2024-03-30T00:39:55.162239+0000","last_undegraded":"2024-03-30T00:39:55.162239+0000","last_fullsized":"2024-03-30T00:39:55.162239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:51.868242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172499+0000","last_change":"2024-03-30T00:39:50.763665+0000","last_active":"2024-03-30T00:39:55.172499+0000","last_peered":"2024-03-30T00:39:55.172499+0000","last_clean":"2024-03-30T00:39:55.172499+0000","last_became_active":"2024-03-30T00:39:50.763390+0000","last_became_peered":"2024-03-30T00:39:50.763390+0000","last_unstale":"2024-03-30T00:39:55.172499+0000","last_undegraded":"2024-03-30T00:39:55.172499+0000","last_fullsized":"2024-03-30T00:39:55.172499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272944+0000","last_change":"2024-03-30T00:39:55.185767+0000","last_active":"2024-03-30T00:39:57.272944+0000","last_peered":"2024-03-30T00:39:57.272944+0000","last_clean":"2024-03-30T00:39:57.272944+0000","last_became_active":"2024-03-30T00:39:55.185430+0000","last_became_peered":"2024-03-30T00:39:55.185430+0000","last_unstale":"2024-03-30T00:39:57.272944+0000","last_undegraded":"2024-03-30T00:39:57.272944+0000","last_fullsized":"2024-03-30T00:39:57.272944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:59:28.443255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184382+0000","last_change":"2024-03-30T00:39:51.177311+0000","last_active":"2024-03-30T00:39:55.184382+0000","last_peered":"2024-03-30T00:39:55.184382+0000","last_clean":"2024-03-30T00:39:55.184382+0000","last_became_active":"2024-03-30T00:39:51.175904+0000","last_became_peered":"2024-03-30T00:39:51.175904+0000","last_unstale":"2024-03-30T00:39:55.184382+0000","last_undegraded":"2024-03-30T00:39:55.184382+0000","last_fullsized":"2024-03-30T00:39:55.184382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167202+0000","last_change":"2024-03-30T00:39:50.769375+0000","last_active":"2024-03-30T00:39:55.167202+0000","last_peered":"2024-03-30T00:39:55.167202+0000","last_clean":"2024-03-30T00:39:55.167202+0000","last_became_active":"2024-03-30T00:39:50.768671+0000","last_became_peered":"2024-03-30T00:39:50.768671+0000","last_unstale":"2024-03-30T00:39:55.167202+0000","last_undegraded":"2024-03-30T00:39:55.167202+0000","last_fullsized":"2024-03-30T00:39:55.167202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571487+0000","last_change":"2024-03-30T00:39:55.192307+0000","last_active":"2024-03-30T00:39:55.571487+0000","last_peered":"2024-03-30T00:39:55.571487+0000","last_clean":"2024-03-30T00:39:55.571487+0000","last_became_active":"2024-03-30T00:39:55.191803+0000","last_became_peered":"2024-03-30T00:39:55.191803+0000","last_unstale":"2024-03-30T00:39:55.571487+0000","last_undegraded":"2024-03-30T00:39:55.571487+0000","last_fullsized":"2024-03-30T00:39:55.571487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:33:50.238803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167108+0000","last_change":"2024-03-30T00:39:51.168083+0000","last_active":"2024-03-30T00:39:55.167108+0000","last_peered":"2024-03-30T00:39:55.167108+0000","last_clean":"2024-03-30T00:39:55.167108+0000","last_became_active":"2024-03-30T00:39:51.167756+0000","last_became_peered":"2024-03-30T00:39:51.167756+0000","last_unstale":"2024-03-30T00:39:55.167108+0000","last_undegraded":"2024-03-30T00:39:55.167108+0000","last_fullsized":"2024-03-30T00:39:55.167108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172080+0000","last_change":"2024-03-30T00:39:50.775634+0000","last_active":"2024-03-30T00:39:55.172080+0000","last_peered":"2024-03-30T00:39:55.172080+0000","last_clean":"2024-03-30T00:39:55.172080+0000","last_became_active":"2024-03-30T00:39:50.775337+0000","last_became_peered":"2024-03-30T00:39:50.775337+0000","last_unstale":"2024-03-30T00:39:55.172080+0000","last_undegraded":"2024-03-30T00:39:55.172080+0000","last_fullsized":"2024-03-30T00:39:55.172080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571487+0000","last_change":"2024-03-30T00:39:55.175243+0000","last_active":"2024-03-30T00:39:55.571487+0000","last_peered":"2024-03-30T00:39:55.571487+0000","last_clean":"2024-03-30T00:39:55.571487+0000","last_became_active":"2024-03-30T00:39:55.174556+0000","last_became_peered":"2024-03-30T00:39:55.174556+0000","last_unstale":"2024-03-30T00:39:55.571487+0000","last_undegraded":"2024-03-30T00:39:55.571487+0000","last_fullsized":"2024-03-30T00:39:55.571487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.274064+0000","last_change":"2024-03-30T00:39:55.203406+0000","last_active":"2024-03-30T00:39:57.274064+0000","last_peered":"2024-03-30T00:39:57.274064+0000","last_clean":"2024-03-30T00:39:57.274064+0000","last_became_active":"2024-03-30T00:39:55.203027+0000","last_became_peered":"2024-03-30T00:39:55.203027+0000","last_unstale":"2024-03-30T00:39:57.274064+0000","last_undegraded":"2024-03-30T00:39:57.274064+0000","last_fullsized":"2024-03-30T00:39:57.274064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171850+0000","last_change":"2024-03-30T00:39:51.175309+0000","last_active":"2024-03-30T00:39:55.171850+0000","last_peered":"2024-03-30T00:39:55.171850+0000","last_clean":"2024-03-30T00:39:55.171850+0000","last_became_active":"2024-03-30T00:39:51.174946+0000","last_became_peered":"2024-03-30T00:39:51.174946+0000","last_unstale":"2024-03-30T00:39:55.171850+0000","last_undegraded":"2024-03-30T00:39:55.171850+0000","last_fullsized":"2024-03-30T00:39:55.171850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167862+0000","last_change":"2024-03-30T00:39:50.762581+0000","last_active":"2024-03-30T00:39:55.167862+0000","last_peered":"2024-03-30T00:39:55.167862+0000","last_clean":"2024-03-30T00:39:55.167862+0000","last_became_active":"2024-03-30T00:39:50.762206+0000","last_became_peered":"2024-03-30T00:39:50.762206+0000","last_unstale":"2024-03-30T00:39:55.167862+0000","last_undegraded":"2024-03-30T00:39:55.167862+0000","last_fullsized":"2024-03-30T00:39:55.167862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.573135+0000","last_change":"2024-03-30T00:39:55.188289+0000","last_active":"2024-03-30T00:39:55.573135+0000","last_peered":"2024-03-30T00:39:55.573135+0000","last_clean":"2024-03-30T00:39:55.573135+0000","last_became_active":"2024-03-30T00:39:55.187825+0000","last_became_peered":"2024-03-30T00:39:55.187825+0000","last_unstale":"2024-03-30T00:39:55.573135+0000","last_undegraded":"2024-03-30T00:39:55.573135+0000","last_fullsized":"2024-03-30T00:39:55.573135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.166225+0000","last_change":"2024-03-30T00:39:51.172769+0000","last_active":"2024-03-30T00:39:55.166225+0000","last_peered":"2024-03-30T00:39:55.166225+0000","last_clean":"2024-03-30T00:39:55.166225+0000","last_became_active":"2024-03-30T00:39:51.170913+0000","last_became_peered":"2024-03-30T00:39:51.170913+0000","last_unstale":"2024-03-30T00:39:55.166225+0000","last_undegraded":"2024-03-30T00:39:55.166225+0000","last_fullsized":"2024-03-30T00:39:55.166225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165283+0000","last_change":"2024-03-30T00:39:50.771732+0000","last_active":"2024-03-30T00:39:55.165283+0000","last_peered":"2024-03-30T00:39:55.165283+0000","last_clean":"2024-03-30T00:39:55.165283+0000","last_became_active":"2024-03-30T00:39:50.771443+0000","last_became_peered":"2024-03-30T00:39:50.771443+0000","last_unstale":"2024-03-30T00:39:55.165283+0000","last_undegraded":"2024-03-30T00:39:55.165283+0000","last_fullsized":"2024-03-30T00:39:55.165283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171239+0000","last_change":"2024-03-30T00:39:50.768147+0000","last_active":"2024-03-30T00:39:55.171239+0000","last_peered":"2024-03-30T00:39:55.171239+0000","last_clean":"2024-03-30T00:39:55.171239+0000","last_became_active":"2024-03-30T00:39:50.767258+0000","last_became_peered":"2024-03-30T00:39:50.767258+0000","last_unstale":"2024-03-30T00:39:55.171239+0000","last_undegraded":"2024-03-30T00:39:55.171239+0000","last_fullsized":"2024-03-30T00:39:55.171239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183786+0000","last_change":"2024-03-30T00:39:51.173636+0000","last_active":"2024-03-30T00:39:55.183786+0000","last_peered":"2024-03-30T00:39:55.183786+0000","last_clean":"2024-03-30T00:39:55.183786+0000","last_became_active":"2024-03-30T00:39:51.173356+0000","last_became_peered":"2024-03-30T00:39:51.173356+0000","last_unstale":"2024-03-30T00:39:55.183786+0000","last_undegraded":"2024-03-30T00:39:55.183786+0000","last_fullsized":"2024-03-30T00:39:55.183786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:10:20.544219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273686+0000","last_change":"2024-03-30T00:39:55.187062+0000","last_active":"2024-03-30T00:39:57.273686+0000","last_peered":"2024-03-30T00:39:57.273686+0000","last_clean":"2024-03-30T00:39:57.273686+0000","last_became_active":"2024-03-30T00:39:55.184977+0000","last_became_peered":"2024-03-30T00:39:55.184977+0000","last_unstale":"2024-03-30T00:39:57.273686+0000","last_undegraded":"2024-03-30T00:39:57.273686+0000","last_fullsized":"2024-03-30T00:39:57.273686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:23.899767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169318+0000","last_change":"2024-03-30T00:39:50.767360+0000","last_active":"2024-03-30T00:39:55.169318+0000","last_peered":"2024-03-30T00:39:55.169318+0000","last_clean":"2024-03-30T00:39:55.169318+0000","last_became_active":"2024-03-30T00:39:50.764301+0000","last_became_peered":"2024-03-30T00:39:50.764301+0000","last_unstale":"2024-03-30T00:39:55.169318+0000","last_undegraded":"2024-03-30T00:39:55.169318+0000","last_fullsized":"2024-03-30T00:39:55.169318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:26.877416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160985+0000","last_change":"2024-03-30T00:39:51.163323+0000","last_active":"2024-03-30T00:39:55.160985+0000","last_peered":"2024-03-30T00:39:55.160985+0000","last_clean":"2024-03-30T00:39:55.160985+0000","last_became_active":"2024-03-30T00:39:51.163080+0000","last_became_peered":"2024-03-30T00:39:51.163080+0000","last_unstale":"2024-03-30T00:39:55.160985+0000","last_undegraded":"2024-03-30T00:39:55.160985+0000","last_fullsized":"2024-03-30T00:39:55.160985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:28.307469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572341+0000","last_change":"2024-03-30T00:39:55.194121+0000","last_active":"2024-03-30T00:39:55.572341+0000","last_peered":"2024-03-30T00:39:55.572341+0000","last_clean":"2024-03-30T00:39:55.572341+0000","last_became_active":"2024-03-30T00:39:55.190121+0000","last_became_peered":"2024-03-30T00:39:55.190121+0000","last_unstale":"2024-03-30T00:39:55.572341+0000","last_undegraded":"2024-03-30T00:39:55.572341+0000","last_fullsized":"2024-03-30T00:39:55.572341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:10:34.870687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169878+0000","last_change":"2024-03-30T00:39:51.168112+0000","last_active":"2024-03-30T00:39:55.169878+0000","last_peered":"2024-03-30T00:39:55.169878+0000","last_clean":"2024-03-30T00:39:55.169878+0000","last_became_active":"2024-03-30T00:39:51.167770+0000","last_became_peered":"2024-03-30T00:39:51.167770+0000","last_unstale":"2024-03-30T00:39:55.169878+0000","last_undegraded":"2024-03-30T00:39:55.169878+0000","last_fullsized":"2024-03-30T00:39:55.169878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159083+0000","last_change":"2024-03-30T00:39:51.169867+0000","last_active":"2024-03-30T00:39:55.159083+0000","last_peered":"2024-03-30T00:39:55.159083+0000","last_clean":"2024-03-30T00:39:55.159083+0000","last_became_active":"2024-03-30T00:39:51.169515+0000","last_became_peered":"2024-03-30T00:39:51.169515+0000","last_unstale":"2024-03-30T00:39:55.159083+0000","last_undegraded":"2024-03-30T00:39:55.159083+0000","last_fullsized":"2024-03-30T00:39:55.159083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:44.962672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168036+0000","last_change":"2024-03-30T00:39:51.175788+0000","last_active":"2024-03-30T00:39:55.168036+0000","last_peered":"2024-03-30T00:39:55.168036+0000","last_clean":"2024-03-30T00:39:55.168036+0000","last_became_active":"2024-03-30T00:39:51.175288+0000","last_became_peered":"2024-03-30T00:39:51.175288+0000","last_unstale":"2024-03-30T00:39:55.168036+0000","last_undegraded":"2024-03-30T00:39:55.168036+0000","last_fullsized":"2024-03-30T00:39:55.168036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:35.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183710+0000","last_change":"2024-03-30T00:39:51.173618+0000","last_active":"2024-03-30T00:39:55.183710+0000","last_peered":"2024-03-30T00:39:55.183710+0000","last_clean":"2024-03-30T00:39:55.183710+0000","last_became_active":"2024-03-30T00:39:51.173324+0000","last_became_peered":"2024-03-30T00:39:51.173324+0000","last_unstale":"2024-03-30T00:39:55.183710+0000","last_undegraded":"2024-03-30T00:39:55.183710+0000","last_fullsized":"2024-03-30T00:39:55.183710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168094+0000","last_change":"2024-03-30T00:39:51.184101+0000","last_active":"2024-03-30T00:39:55.168094+0000","last_peered":"2024-03-30T00:39:55.168094+0000","last_clean":"2024-03-30T00:39:55.168094+0000","last_became_active":"2024-03-30T00:39:51.183843+0000","last_became_peered":"2024-03-30T00:39:51.183843+0000","last_unstale":"2024-03-30T00:39:55.168094+0000","last_undegraded":"2024-03-30T00:39:55.168094+0000","last_fullsized":"2024-03-30T00:39:55.168094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:51.868242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172390+0000","last_change":"2024-03-30T00:39:51.174001+0000","last_active":"2024-03-30T00:39:55.172390+0000","last_peered":"2024-03-30T00:39:55.172390+0000","last_clean":"2024-03-30T00:39:55.172390+0000","last_became_active":"2024-03-30T00:39:51.173753+0000","last_became_peered":"2024-03-30T00:39:51.173753+0000","last_unstale":"2024-03-30T00:39:55.172390+0000","last_undegraded":"2024-03-30T00:39:55.172390+0000","last_fullsized":"2024-03-30T00:39:55.172390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171728+0000","last_change":"2024-03-30T00:39:51.165845+0000","last_active":"2024-03-30T00:39:55.171728+0000","last_peered":"2024-03-30T00:39:55.171728+0000","last_clean":"2024-03-30T00:39:55.171728+0000","last_became_active":"2024-03-30T00:39:51.164657+0000","last_became_peered":"2024-03-30T00:39:51.164657+0000","last_unstale":"2024-03-30T00:39:55.171728+0000","last_undegraded":"2024-03-30T00:39:55.171728+0000","last_fullsized":"2024-03-30T00:39:55.171728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.164918+0000","last_change":"2024-03-30T00:39:51.185222+0000","last_active":"2024-03-30T00:39:55.164918+0000","last_peered":"2024-03-30T00:39:55.164918+0000","last_clean":"2024-03-30T00:39:55.164918+0000","last_became_active":"2024-03-30T00:39:51.184954+0000","last_became_peered":"2024-03-30T00:39:51.184954+0000","last_unstale":"2024-03-30T00:39:55.164918+0000","last_undegraded":"2024-03-30T00:39:55.164918+0000","last_fullsized":"2024-03-30T00:39:55.164918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167766+0000","last_change":"2024-03-30T00:39:51.174598+0000","last_active":"2024-03-30T00:39:55.167766+0000","last_peered":"2024-03-30T00:39:55.167766+0000","last_clean":"2024-03-30T00:39:55.167766+0000","last_became_active":"2024-03-30T00:39:51.174250+0000","last_became_peered":"2024-03-30T00:39:51.174250+0000","last_unstale":"2024-03-30T00:39:55.167766+0000","last_undegraded":"2024-03-30T00:39:55.167766+0000","last_fullsized":"2024-03-30T00:39:55.167766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165202+0000","last_change":"2024-03-30T00:39:51.184460+0000","last_active":"2024-03-30T00:39:55.165202+0000","last_peered":"2024-03-30T00:39:55.165202+0000","last_clean":"2024-03-30T00:39:55.165202+0000","last_became_active":"2024-03-30T00:39:51.184185+0000","last_became_peered":"2024-03-30T00:39:51.184185+0000","last_unstale":"2024-03-30T00:39:55.165202+0000","last_undegraded":"2024-03-30T00:39:55.165202+0000","last_fullsized":"2024-03-30T00:39:55.165202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167944+0000","last_change":"2024-03-30T00:39:51.168064+0000","last_active":"2024-03-30T00:39:55.167944+0000","last_peered":"2024-03-30T00:39:55.167944+0000","last_clean":"2024-03-30T00:39:55.167944+0000","last_became_active":"2024-03-30T00:39:51.167772+0000","last_became_peered":"2024-03-30T00:39:51.167772+0000","last_unstale":"2024-03-30T00:39:55.167944+0000","last_undegraded":"2024-03-30T00:39:55.167944+0000","last_fullsized":"2024-03-30T00:39:55.167944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:12.512040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183617+0000","last_change":"2024-03-30T00:39:51.183269+0000","last_active":"2024-03-30T00:39:55.183617+0000","last_peered":"2024-03-30T00:39:55.183617+0000","last_clean":"2024-03-30T00:39:55.183617+0000","last_became_active":"2024-03-30T00:39:51.182687+0000","last_became_peered":"2024-03-30T00:39:51.182687+0000","last_unstale":"2024-03-30T00:39:55.183617+0000","last_undegraded":"2024-03-30T00:39:55.183617+0000","last_fullsized":"2024-03-30T00:39:55.183617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171193+0000","last_change":"2024-03-30T00:39:51.171426+0000","last_active":"2024-03-30T00:39:55.171193+0000","last_peered":"2024-03-30T00:39:55.171193+0000","last_clean":"2024-03-30T00:39:55.171193+0000","last_became_active":"2024-03-30T00:39:51.171182+0000","last_became_peered":"2024-03-30T00:39:51.171182+0000","last_unstale":"2024-03-30T00:39:55.171193+0000","last_undegraded":"2024-03-30T00:39:55.171193+0000","last_fullsized":"2024-03-30T00:39:55.171193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:51:41.648914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163087+0000","last_change":"2024-03-30T00:39:51.159899+0000","last_active":"2024-03-30T00:39:55.163087+0000","last_peered":"2024-03-30T00:39:55.163087+0000","last_clean":"2024-03-30T00:39:55.163087+0000","last_became_active":"2024-03-30T00:39:51.159626+0000","last_became_peered":"2024-03-30T00:39:51.159626+0000","last_unstale":"2024-03-30T00:39:55.163087+0000","last_undegraded":"2024-03-30T00:39:55.163087+0000","last_fullsized":"2024-03-30T00:39:55.163087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159252+0000","last_change":"2024-03-30T00:39:51.171375+0000","last_active":"2024-03-30T00:39:55.159252+0000","last_peered":"2024-03-30T00:39:55.159252+0000","last_clean":"2024-03-30T00:39:55.159252+0000","last_became_active":"2024-03-30T00:39:51.170689+0000","last_became_peered":"2024-03-30T00:39:51.170689+0000","last_unstale":"2024-03-30T00:39:55.159252+0000","last_undegraded":"2024-03-30T00:39:55.159252+0000","last_fullsized":"2024-03-30T00:39:55.159252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:27.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167585+0000","last_change":"2024-03-30T00:39:51.164848+0000","last_active":"2024-03-30T00:39:55.167585+0000","last_peered":"2024-03-30T00:39:55.167585+0000","last_clean":"2024-03-30T00:39:55.167585+0000","last_became_active":"2024-03-30T00:39:51.164575+0000","last_became_peered":"2024-03-30T00:39:51.164575+0000","last_unstale":"2024-03-30T00:39:55.167585+0000","last_undegraded":"2024-03-30T00:39:55.167585+0000","last_fullsized":"2024-03-30T00:39:55.167585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167681+0000","last_change":"2024-03-30T00:39:51.173535+0000","last_active":"2024-03-30T00:39:55.167681+0000","last_peered":"2024-03-30T00:39:55.167681+0000","last_clean":"2024-03-30T00:39:55.167681+0000","last_became_active":"2024-03-30T00:39:51.173313+0000","last_became_peered":"2024-03-30T00:39:51.173313+0000","last_unstale":"2024-03-30T00:39:55.167681+0000","last_undegraded":"2024-03-30T00:39:55.167681+0000","last_fullsized":"2024-03-30T00:39:55.167681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171944+0000","last_change":"2024-03-30T00:39:51.171986+0000","last_active":"2024-03-30T00:39:55.171944+0000","last_peered":"2024-03-30T00:39:55.171944+0000","last_clean":"2024-03-30T00:39:55.171944+0000","last_became_active":"2024-03-30T00:39:51.164756+0000","last_became_peered":"2024-03-30T00:39:51.164756+0000","last_unstale":"2024-03-30T00:39:55.171944+0000","last_undegraded":"2024-03-30T00:39:55.171944+0000","last_fullsized":"2024-03-30T00:39:55.171944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184365+0000","last_change":"2024-03-30T00:39:51.181455+0000","last_active":"2024-03-30T00:39:55.184365+0000","last_peered":"2024-03-30T00:39:55.184365+0000","last_clean":"2024-03-30T00:39:55.184365+0000","last_became_active":"2024-03-30T00:39:51.181207+0000","last_became_peered":"2024-03-30T00:39:51.181207+0000","last_unstale":"2024-03-30T00:39:55.184365+0000","last_undegraded":"2024-03-30T00:39:55.184365+0000","last_fullsized":"2024-03-30T00:39:55.184365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:01:48.059238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169138+0000","last_change":"2024-03-30T00:39:51.175074+0000","last_active":"2024-03-30T00:39:55.169138+0000","last_peered":"2024-03-30T00:39:55.169138+0000","last_clean":"2024-03-30T00:39:55.169138+0000","last_became_active":"2024-03-30T00:39:51.174836+0000","last_became_peered":"2024-03-30T00:39:51.174836+0000","last_unstale":"2024-03-30T00:39:55.169138+0000","last_undegraded":"2024-03-30T00:39:55.169138+0000","last_fullsized":"2024-03-30T00:39:55.169138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162196+0000","last_change":"2024-03-30T00:39:51.169964+0000","last_active":"2024-03-30T00:39:55.162196+0000","last_peered":"2024-03-30T00:39:55.162196+0000","last_clean":"2024-03-30T00:39:55.162196+0000","last_became_active":"2024-03-30T00:39:51.169647+0000","last_became_peered":"2024-03-30T00:39:51.169647+0000","last_unstale":"2024-03-30T00:39:55.162196+0000","last_undegraded":"2024-03-30T00:39:55.162196+0000","last_fullsized":"2024-03-30T00:39:55.162196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:44.962672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171503+0000","last_change":"2024-03-30T00:39:51.178908+0000","last_active":"2024-03-30T00:39:55.171503+0000","last_peered":"2024-03-30T00:39:55.171503+0000","last_clean":"2024-03-30T00:39:55.171503+0000","last_became_active":"2024-03-30T00:39:51.178663+0000","last_became_peered":"2024-03-30T00:39:51.178663+0000","last_unstale":"2024-03-30T00:39:55.171503+0000","last_undegraded":"2024-03-30T00:39:55.171503+0000","last_fullsized":"2024-03-30T00:39:55.171503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171643+0000","last_change":"2024-03-30T00:39:51.179913+0000","last_active":"2024-03-30T00:39:55.171643+0000","last_peered":"2024-03-30T00:39:55.171643+0000","last_clean":"2024-03-30T00:39:55.171643+0000","last_became_active":"2024-03-30T00:39:51.179652+0000","last_became_peered":"2024-03-30T00:39:51.179652+0000","last_unstale":"2024-03-30T00:39:55.171643+0000","last_undegraded":"2024-03-30T00:39:55.171643+0000","last_fullsized":"2024-03-30T00:39:55.171643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163237+0000","last_change":"2024-03-30T00:39:51.170941+0000","last_active":"2024-03-30T00:39:55.163237+0000","last_peered":"2024-03-30T00:39:55.163237+0000","last_clean":"2024-03-30T00:39:55.163237+0000","last_became_active":"2024-03-30T00:39:51.166779+0000","last_became_peered":"2024-03-30T00:39:51.166779+0000","last_unstale":"2024-03-30T00:39:55.163237+0000","last_undegraded":"2024-03-30T00:39:55.163237+0000","last_fullsized":"2024-03-30T00:39:55.163237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:12.512040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167960+0000","last_change":"2024-03-30T00:39:51.176410+0000","last_active":"2024-03-30T00:39:55.167960+0000","last_peered":"2024-03-30T00:39:55.167960+0000","last_clean":"2024-03-30T00:39:55.167960+0000","last_became_active":"2024-03-30T00:39:51.175971+0000","last_became_peered":"2024-03-30T00:39:51.175971+0000","last_unstale":"2024-03-30T00:39:55.167960+0000","last_undegraded":"2024-03-30T00:39:55.167960+0000","last_fullsized":"2024-03-30T00:39:55.167960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160661+0000","last_change":"2024-03-30T00:39:51.167648+0000","last_active":"2024-03-30T00:39:55.160661+0000","last_peered":"2024-03-30T00:39:55.160661+0000","last_clean":"2024-03-30T00:39:55.160661+0000","last_became_active":"2024-03-30T00:39:51.167382+0000","last_became_peered":"2024-03-30T00:39:51.167382+0000","last_unstale":"2024-03-30T00:39:55.160661+0000","last_undegraded":"2024-03-30T00:39:55.160661+0000","last_fullsized":"2024-03-30T00:39:55.160661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:39:43.189821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167861+0000","last_change":"2024-03-30T00:39:51.169776+0000","last_active":"2024-03-30T00:39:55.167861+0000","last_peered":"2024-03-30T00:39:55.167861+0000","last_clean":"2024-03-30T00:39:55.167861+0000","last_became_active":"2024-03-30T00:39:51.169462+0000","last_became_peered":"2024-03-30T00:39:51.169462+0000","last_unstale":"2024-03-30T00:39:55.167861+0000","last_undegraded":"2024-03-30T00:39:55.167861+0000","last_fullsized":"2024-03-30T00:39:55.167861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:37:05.851113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161236+0000","last_change":"2024-03-30T00:39:51.163469+0000","last_active":"2024-03-30T00:39:55.161236+0000","last_peered":"2024-03-30T00:39:55.161236+0000","last_clean":"2024-03-30T00:39:55.161236+0000","last_became_active":"2024-03-30T00:39:51.163200+0000","last_became_peered":"2024-03-30T00:39:51.163200+0000","last_unstale":"2024-03-30T00:39:55.161236+0000","last_undegraded":"2024-03-30T00:39:55.161236+0000","last_fullsized":"2024-03-30T00:39:55.161236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:24:05.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170209+0000","last_change":"2024-03-30T00:39:51.183006+0000","last_active":"2024-03-30T00:39:55.170209+0000","last_peered":"2024-03-30T00:39:55.170209+0000","last_clean":"2024-03-30T00:39:55.170209+0000","last_became_active":"2024-03-30T00:39:51.179310+0000","last_became_peered":"2024-03-30T00:39:51.179310+0000","last_unstale":"2024-03-30T00:39:55.170209+0000","last_undegraded":"2024-03-30T00:39:55.170209+0000","last_fullsized":"2024-03-30T00:39:55.170209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:41.330935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183618+0000","last_change":"2024-03-30T00:39:51.180621+0000","last_active":"2024-03-30T00:39:55.183618+0000","last_peered":"2024-03-30T00:39:55.183618+0000","last_clean":"2024-03-30T00:39:55.183618+0000","last_became_active":"2024-03-30T00:39:51.175749+0000","last_became_peered":"2024-03-30T00:39:51.175749+0000","last_unstale":"2024-03-30T00:39:55.183618+0000","last_undegraded":"2024-03-30T00:39:55.183618+0000","last_fullsized":"2024-03-30T00:39:55.183618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:27.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171993+0000","last_change":"2024-03-30T00:39:51.168331+0000","last_active":"2024-03-30T00:39:55.171993+0000","last_peered":"2024-03-30T00:39:55.171993+0000","last_clean":"2024-03-30T00:39:55.171993+0000","last_became_active":"2024-03-30T00:39:51.168052+0000","last_became_peered":"2024-03-30T00:39:51.168052+0000","last_unstale":"2024-03-30T00:39:55.171993+0000","last_undegraded":"2024-03-30T00:39:55.171993+0000","last_fullsized":"2024-03-30T00:39:55.171993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163158+0000","last_change":"2024-03-30T00:39:51.167745+0000","last_active":"2024-03-30T00:39:55.163158+0000","last_peered":"2024-03-30T00:39:55.163158+0000","last_clean":"2024-03-30T00:39:55.163158+0000","last_became_active":"2024-03-30T00:39:51.166397+0000","last_became_peered":"2024-03-30T00:39:51.166397+0000","last_unstale":"2024-03-30T00:39:55.163158+0000","last_undegraded":"2024-03-30T00:39:55.163158+0000","last_fullsized":"2024-03-30T00:39:55.163158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:50:20.941634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:40:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T00:40:16.003 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-30T00:40:16.003 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-30T00:40:16.003 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-30T00:40:16.003 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-30T00:40:16.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.169+0000 7fc984000640 1 Processor -- start 2024-03-30T00:40:16.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.169+0000 7fc984000640 1 -- start start 2024-03-30T00:40:16.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc984000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc97c15ca10 0x7fc97c06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc984000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c06e7d0 con 0x7fc97c0ce820 2024-03-30T00:40:16.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc984000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c06f1a0 con 0x7fc97c0ae4e0 2024-03-30T00:40:16.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc984000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0b1f70 con 0x7fc97c15ca10 2024-03-30T00:40:16.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc982000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc97c0ce820 0x7fc97c160510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36348/0 (socket says 172.21.15.66:36348) 2024-03-30T00:40:16.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc982000640 1 -- 172.21.15.66:0/3791745146 learned_addr learned my addr 172.21.15.66:0/3791745146 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:16.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc981600640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc97c15ca10 0x7fc97c06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134492786 0 0) 0x7fc97c06e7d0 con 0x7fc97c0ce820 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc964003530 con 0x7fc97c0ce820 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1509012544 0 0) 0x7fc964003530 con 0x7fc97c0ce820 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc97c15ca10 msgr2=0x7fc97c06ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc97c15ca10 0x7fc97c06ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 >> v1:172.21.15.156:6789/0 conn(0x7fc97c0ae4e0 legacy=0x7fc97c0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c15cdf0 con 0x7fc97c0ce820 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fc974001a00 con 0x7fc97c0ce820 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 >> v1:172.21.15.66:6789/0 conn(0x7fc97c0ce820 legacy=0x7fc97c160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc97c15ca10 0x7fc9640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc964003ef0 0x7fc9640047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc964004d10 0x7fc964005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0b1f70 con 0x7fc964003ef0 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96400b9f0 con 0x7fc964004d10 2024-03-30T00:40:16.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96400bbb0 con 0x7fc97c15ca10 2024-03-30T00:40:16.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc980c00640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc964003ef0 0x7fc9640047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc981600640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc97c15ca10 0x7fc9640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc982000640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc964004d10 0x7fc964005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc980c00640 1 -- 172.21.15.66:0/3791745146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc97c15ca10 msgr2=0x7fc9640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:16.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc980c00640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc97c15ca10 0x7fc9640039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc980c00640 1 -- 172.21.15.66:0/3791745146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc964004d10 msgr2=0x7fc964005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:16.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc980c00640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc964004d10 0x7fc964005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.173+0000 7fc980c00640 1 -- 172.21.15.66:0/3791745146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc964003530 con 0x7fc964003ef0 2024-03-30T00:40:16.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc980c00640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc964003ef0 0x7fc9640047d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc968001860 tx=0x7fc96801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1407fc1fe089b35 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:16.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9680ab600 con 0x7fc964003ef0 2024-03-30T00:40:16.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9680a7050 con 0x7fc964003ef0 2024-03-30T00:40:16.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc97be00640 1 -- 172.21.15.66:0/3791745146 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9680ab9e0 con 0x7fc964003ef0 2024-03-30T00:40:16.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc984000640 1 -- 172.21.15.66:0/3791745146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc964003ef0 msgr2=0x7fc9640047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc984000640 1 --2- 172.21.15.66:0/3791745146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc964003ef0 0x7fc9640047d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc968001860 tx=0x7fc96801c410 comp rx=0 tx=0).stop 2024-03-30T00:40:16.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc982000640 1 -- 172.21.15.66:0/3791745146 reap_dead start 2024-03-30T00:40:16.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc984000640 1 -- 172.21.15.66:0/3791745146 shutdown_connections 2024-03-30T00:40:16.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc984000640 1 -- 172.21.15.66:0/3791745146 >> 172.21.15.66:0/3791745146 conn(0x7fc97c098a20 msgr2=0x7fc97c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:16.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc984000640 1 -- 172.21.15.66:0/3791745146 shutdown_connections 2024-03-30T00:40:16.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc984000640 1 -- 172.21.15.66:0/3791745146 wait complete. 2024-03-30T00:40:16.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc984000640 1 Processor -- start 2024-03-30T00:40:16.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.177+0000 7fc984000640 1 -- start start 2024-03-30T00:40:16.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc984000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc964003ef0 0x7fc97c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc984000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc97c0af400 0x7fc97c0ad180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc984000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc97c1568e0 0x7fc97c156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc984000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0af880 con 0x7fc97c1568e0 2024-03-30T00:40:16.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc984000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c157280 con 0x7fc97c0af400 2024-03-30T00:40:16.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc984000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0b03b0 con 0x7fc964003ef0 2024-03-30T00:40:16.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc982000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc97c1568e0 0x7fc97c156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc982000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc97c1568e0 0x7fc97c156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56678/0 (socket says 172.21.15.66:56678) 2024-03-30T00:40:16.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc982000640 1 -- 172.21.15.66:0/2986512977 learned_addr learned my addr 172.21.15.66:0/2986512977 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:16.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc981600640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc964003ef0 0x7fc97c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc980c00640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc97c0af400 0x7fc97c0ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc982000640 1 -- 172.21.15.66:0/2986512977 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc964003ef0 msgr2=0x7fc97c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc982000640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc964003ef0 0x7fc97c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc982000640 1 -- 172.21.15.66:0/2986512977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc97c0af400 msgr2=0x7fc97c0ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc982000640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc97c0af400 0x7fc97c0ad180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc982000640 1 -- 172.21.15.66:0/2986512977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c0b4680 con 0x7fc97c1568e0 2024-03-30T00:40:16.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc980c00640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc97c0af400 0x7fc97c0ad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:16.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc982000640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc97c1568e0 0x7fc97c156d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0c2f90 tx=0x7fc97400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:16.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc979600640 1 -- 172.21.15.66:0/2986512977 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97400ec40 con 0x7fc97c1568e0 2024-03-30T00:40:16.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc979600640 1 -- 172.21.15.66:0/2986512977 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97400edc0 con 0x7fc97c1568e0 2024-03-30T00:40:16.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc979600640 1 -- 172.21.15.66:0/2986512977 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974014670 con 0x7fc97c1568e0 2024-03-30T00:40:16.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc984000640 1 -- 172.21.15.66:0/2986512977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c0c6d20 con 0x7fc97c1568e0 2024-03-30T00:40:16.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc984000640 1 -- 172.21.15.66:0/2986512977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c0b3de0 con 0x7fc97c1568e0 2024-03-30T00:40:16.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.181+0000 7fc984000640 1 -- 172.21.15.66:0/2986512977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97c066c30 con 0x7fc97c1568e0 2024-03-30T00:40:16.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.185+0000 7fc979600640 1 -- 172.21.15.66:0/2986512977 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc9740147d0 con 0x7fc97c1568e0 2024-03-30T00:40:16.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.185+0000 7fc979600640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc95c0666c0 0x7fc95c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.185+0000 7fc979600640 1 -- 172.21.15.66:0/2986512977 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc974084140 con 0x7fc97c1568e0 2024-03-30T00:40:16.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.185+0000 7fc981600640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc95c0666c0 0x7fc95c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.185+0000 7fc981600640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc95c0666c0 0x7fc95c068b80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc970009be0 tx=0x7fc970009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:16.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.189+0000 7fc979600640 1 -- 172.21.15.66:0/2986512977 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc9740511c0 con 0x7fc97c1568e0 2024-03-30T00:40:16.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.389+0000 7fc984000640 1 -- 172.21.15.66:0/2986512977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc97c0c6eb0 con 0x7fc97c1568e0 2024-03-30T00:40:16.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc979600640 1 -- 172.21.15.66:0/2986512977 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164875 (secure 0 0 0) 0x7fc9740547e0 con 0x7fc97c1568e0 2024-03-30T00:40:16.403 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 -- 172.21.15.66:0/2986512977 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc95c0666c0 msgr2=0x7fc95c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc95c0666c0 0x7fc95c068b80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc970009be0 tx=0x7fc970009190 comp rx=0 tx=0).stop 2024-03-30T00:40:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 -- 172.21.15.66:0/2986512977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc97c1568e0 msgr2=0x7fc97c156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc97c1568e0 0x7fc97c156d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0c2f90 tx=0x7fc97400e530 comp rx=0 tx=0).stop 2024-03-30T00:40:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 -- 172.21.15.66:0/2986512977 shutdown_connections 2024-03-30T00:40:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc95c0666c0 0x7fc95c068b80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc97c1568e0 0x7fc97c156d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc97c0af400 0x7fc97c0ad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 --2- 172.21.15.66:0/2986512977 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc964003ef0 0x7fc97c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 -- 172.21.15.66:0/2986512977 >> 172.21.15.66:0/2986512977 conn(0x7fc97c098a20 msgr2=0x7fc97c15bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 -- 172.21.15.66:0/2986512977 shutdown_connections 2024-03-30T00:40:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.393+0000 7fc984000640 1 -- 172.21.15.66:0/2986512977 wait complete. 2024-03-30T00:40:16.423 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":27382},{"type":"v1","addr":"172.21.15.66:6833","nonce":27382}]},"active_addr":"172.21.15.66:6833/27382","active_change":"2024-03-30T00:39:30.837329+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":2766508840}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":2092393986}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":1817349224}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":2248997976}]}]} 2024-03-30T00:40:16.426 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-30T00:40:16.426 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-30T00:40:16.426 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:16.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.557+0000 7f7a55600640 1 Processor -- start 2024-03-30T00:40:16.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.557+0000 7f7a55600640 1 -- start start 2024-03-30T00:40:16.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.557+0000 7f7a55600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7a5015a670 0x7f7a500b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.557+0000 7f7a55600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a500cb520 con 0x7f7a500d01d0 2024-03-30T00:40:16.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.557+0000 7f7a55600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a500cd500 con 0x7f7a500ce710 2024-03-30T00:40:16.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.557+0000 7f7a55600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a500cd7d0 con 0x7f7a5015a670 2024-03-30T00:40:16.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.557+0000 7f7a4f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7a500d01d0 0x7f7a500d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36356/0 (socket says 172.21.15.66:36356) 2024-03-30T00:40:16.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.557+0000 7f7a4f400640 1 -- 172.21.15.66:0/2868966887 learned_addr learned my addr 172.21.15.66:0/2868966887 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.557+0000 7f7a4ea00640 1 --2- 172.21.15.66:0/2868966887 >> v2:172.21.15.156:3300/0 conn(0x7f7a5015a670 0x7f7a500b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369382463 0 0) 0x7f7a500cb520 con 0x7f7a500d01d0 2024-03-30T00:40:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a34003530 con 0x7f7a500d01d0 2024-03-30T00:40:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4043650564 0 0) 0x7f7a34003530 con 0x7f7a500d01d0 2024-03-30T00:40:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 >> v2:172.21.15.156:3300/0 conn(0x7f7a5015a670 msgr2=0x7f7a500b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 --2- 172.21.15.66:0/2868966887 >> v2:172.21.15.156:3300/0 conn(0x7f7a5015a670 0x7f7a500b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 >> v1:172.21.15.156:6789/0 conn(0x7f7a500ce710 legacy=0x7f7a500ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a500cdab0 con 0x7f7a500d01d0 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f7a48001680 con 0x7f7a500d01d0 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 >> v1:172.21.15.66:6789/0 conn(0x7f7a500d01d0 legacy=0x7f7a500d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 --2- 172.21.15.66:0/2868966887 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a34003980 0x7f7a34003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 --2- 172.21.15.66:0/2868966887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a34004380 0x7f7a34004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 --2- 172.21.15.66:0/2868966887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a34005050 0x7f7a3400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500cd7d0 con 0x7f7a34004380 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3400bfb0 con 0x7f7a34005050 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3400c170 con 0x7f7a34003980 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a46000640 1 --2- 172.21.15.66:0/2868966887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a34004380 0x7f7a34004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4f400640 1 --2- 172.21.15.66:0/2868966887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a34005050 0x7f7a3400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4ea00640 1 --2- 172.21.15.66:0/2868966887 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a34003980 0x7f7a34003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a46000640 1 -- 172.21.15.66:0/2868966887 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a34003980 msgr2=0x7f7a34003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:16.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a46000640 1 --2- 172.21.15.66:0/2868966887 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a34003980 0x7f7a34003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a46000640 1 -- 172.21.15.66:0/2868966887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a34005050 msgr2=0x7f7a3400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:16.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a46000640 1 --2- 172.21.15.66:0/2868966887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a34005050 0x7f7a3400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a46000640 1 -- 172.21.15.66:0/2868966887 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a34003530 con 0x7f7a34004380 2024-03-30T00:40:16.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.561+0000 7f7a4f400640 1 -- 172.21.15.66:0/2868966887 reap_dead start 2024-03-30T00:40:16.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a46000640 1 --2- 172.21.15.66:0/2868966887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a34004380 0x7f7a34004850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a38001910 tx=0x7f7a3801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84d53d7b8454336 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:16.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a380a7070 con 0x7f7a34004380 2024-03-30T00:40:16.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a380ab430 con 0x7f7a34004380 2024-03-30T00:40:16.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a4e000640 1 -- 172.21.15.66:0/2868966887 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a380ab800 con 0x7f7a34004380 2024-03-30T00:40:16.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 -- 172.21.15.66:0/2868966887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a34004380 msgr2=0x7f7a34004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 --2- 172.21.15.66:0/2868966887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a34004380 0x7f7a34004850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a38001910 tx=0x7f7a3801c510 comp rx=0 tx=0).stop 2024-03-30T00:40:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 -- 172.21.15.66:0/2868966887 shutdown_connections 2024-03-30T00:40:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 --2- 172.21.15.66:0/2868966887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a34004380 0x7f7a34004850 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 -- 172.21.15.66:0/2868966887 >> 172.21.15.66:0/2868966887 conn(0x7f7a50099870 msgr2=0x7f7a50089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 -- 172.21.15.66:0/2868966887 shutdown_connections 2024-03-30T00:40:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 -- 172.21.15.66:0/2868966887 wait complete. 2024-03-30T00:40:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 Processor -- start 2024-03-30T00:40:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 -- start start 2024-03-30T00:40:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a34004380 0x7f7a500b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a50151aa0 0x7f7a500c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a500c9400 0x7f7a500ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500c8fe0 con 0x7f7a500c9400 2024-03-30T00:40:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500c86a0 con 0x7f7a50151aa0 2024-03-30T00:40:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a55600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500b4dd0 con 0x7f7a34004380 2024-03-30T00:40:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a4f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a500c9400 0x7f7a500ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a4f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a500c9400 0x7f7a500ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56694/0 (socket says 172.21.15.66:56694) 2024-03-30T00:40:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a4f400640 1 -- 172.21.15.66:0/1089431003 learned_addr learned my addr 172.21.15.66:0/1089431003 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a4ea00640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a34004380 0x7f7a500b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.565+0000 7f7a46000640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a50151aa0 0x7f7a500c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a4f400640 1 -- 172.21.15.66:0/1089431003 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a34004380 msgr2=0x7f7a500b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a4f400640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a34004380 0x7f7a500b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a4f400640 1 -- 172.21.15.66:0/1089431003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a50151aa0 msgr2=0x7f7a500c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a4f400640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a50151aa0 0x7f7a500c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a4f400640 1 -- 172.21.15.66:0/1089431003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5015abd0 con 0x7f7a500c9400 2024-03-30T00:40:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a4f400640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a500c9400 0x7f7a500ae2e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a500cad80 tx=0x7f7a4800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:16.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a47400640 1 -- 172.21.15.66:0/1089431003 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48010510 con 0x7f7a500c9400 2024-03-30T00:40:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a47400640 1 -- 172.21.15.66:0/1089431003 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a48010670 con 0x7f7a500c9400 2024-03-30T00:40:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a47400640 1 -- 172.21.15.66:0/1089431003 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48010a20 con 0x7f7a500c9400 2024-03-30T00:40:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a55600640 1 -- 172.21.15.66:0/1089431003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a500c8920 con 0x7f7a500c9400 2024-03-30T00:40:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a55600640 1 -- 172.21.15.66:0/1089431003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a500c6ea0 con 0x7f7a500c9400 2024-03-30T00:40:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a55600640 1 -- 172.21.15.66:0/1089431003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a500c9120 con 0x7f7a500c9400 2024-03-30T00:40:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a47400640 1 -- 172.21.15.66:0/1089431003 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7a48010b80 con 0x7f7a500c9400 2024-03-30T00:40:16.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a47400640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7a2c066690 0x7f7a2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a47400640 1 -- 172.21.15.66:0/1089431003 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7a48083950 con 0x7f7a500c9400 2024-03-30T00:40:16.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.569+0000 7f7a4ea00640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7a2c066690 0x7f7a2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.573+0000 7f7a4ea00640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7a2c066690 0x7f7a2c068b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a50068fe0 tx=0x7f7a4000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:16.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.573+0000 7f7a47400640 1 -- 172.21.15.66:0/1089431003 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7a500c9120 con 0x7f7a500c9400 2024-03-30T00:40:16.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.737+0000 7f7a55600640 1 -- 172.21.15.66:0/1089431003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a500c9120 con 0x7f7a500c9400 2024-03-30T00:40:16.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.737+0000 7f7a47400640 1 -- 172.21.15.66:0/1089431003 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f7a500c9120 con 0x7f7a500c9400 2024-03-30T00:40:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:16.747 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:16.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 -- 172.21.15.66:0/1089431003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7a2c066690 msgr2=0x7f7a2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7a2c066690 0x7f7a2c068b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a50068fe0 tx=0x7f7a4000b040 comp rx=0 tx=0).stop 2024-03-30T00:40:16.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 -- 172.21.15.66:0/1089431003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a500c9400 msgr2=0x7f7a500ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a500c9400 0x7f7a500ae2e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a500cad80 tx=0x7f7a4800e960 comp rx=0 tx=0).stop 2024-03-30T00:40:16.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 -- 172.21.15.66:0/1089431003 shutdown_connections 2024-03-30T00:40:16.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7a2c066690 0x7f7a2c068b50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a500c9400 0x7f7a500ae2e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a50151aa0 0x7f7a500c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 --2- 172.21.15.66:0/1089431003 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a34004380 0x7f7a500b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 -- 172.21.15.66:0/1089431003 >> 172.21.15.66:0/1089431003 conn(0x7f7a50099870 msgr2=0x7f7a5006e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:16.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 -- 172.21.15.66:0/1089431003 shutdown_connections 2024-03-30T00:40:16.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.741+0000 7f7a55600640 1 -- 172.21.15.66:0/1089431003 wait complete. 2024-03-30T00:40:16.768 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-30T00:40:16.768 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:16.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee2c00640 1 Processor -- start 2024-03-30T00:40:16.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee2c00640 1 -- start start 2024-03-30T00:40:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee2c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5edc0cda90 0x7f5edc0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee2c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc15a370 0x7f5edc0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee2c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5edc15a990 con 0x7f5edc0cde70 2024-03-30T00:40:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee2c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5edc0b1b80 con 0x7f5edc0cda90 2024-03-30T00:40:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee2c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc0cd3e0 con 0x7f5edc15a370 2024-03-30T00:40:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5edb400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5edc0cde70 0x7f5edc159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36358/0 (socket says 172.21.15.66:36358) 2024-03-30T00:40:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5edb400640 1 -- 172.21.15.66:0/1824986071 learned_addr learned my addr 172.21.15.66:0/1824986071 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee0c00640 1 --2- 172.21.15.66:0/1824986071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc15a370 0x7f5edc0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5edbe00640 1 --2- 172.21.15.66:0/1824986071 >> v2:172.21.15.156:3300/0 conn(0x7f5edc0cda90 0x7f5edc0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee0c00640 1 -- 172.21.15.66:0/1824986071 >> v2:172.21.15.156:3300/0 conn(0x7f5edc0cda90 msgr2=0x7f5edc0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:16.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee0c00640 1 --2- 172.21.15.66:0/1824986071 >> v2:172.21.15.156:3300/0 conn(0x7f5edc0cda90 0x7f5edc0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee0c00640 1 -- 172.21.15.66:0/1824986071 >> v1:172.21.15.66:6789/0 conn(0x7f5edc0cde70 legacy=0x7f5edc159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5ee0c00640 1 -- 172.21.15.66:0/1824986071 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc0b11e0 con 0x7f5edc15a370 2024-03-30T00:40:16.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.901+0000 7f5edaa00640 1 -- 172.21.15.66:0/1824986071 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615052015 0 0) 0x7f5edc15a990 con 0x7f5edc0cde70 2024-03-30T00:40:16.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5ee0c00640 1 --2- 172.21.15.66:0/1824986071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc15a370 0x7f5edc0cea60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009d90 tx=0x7f5ed401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10a5d62c2c88bf72 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:16.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5edaa00640 1 -- 172.21.15.66:0/1824986071 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40a58b0 con 0x7f5edc15a370 2024-03-30T00:40:16.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5edaa00640 1 -- 172.21.15.66:0/1824986071 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed40a5a10 con 0x7f5edc15a370 2024-03-30T00:40:16.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5edaa00640 1 -- 172.21.15.66:0/1824986071 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40a5da0 con 0x7f5edc15a370 2024-03-30T00:40:16.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5ee2c00640 1 -- 172.21.15.66:0/1824986071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc15a370 msgr2=0x7f5edc0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5ee2c00640 1 --2- 172.21.15.66:0/1824986071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc15a370 0x7f5edc0cea60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009d90 tx=0x7f5ed401c8e0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5ee2c00640 1 -- 172.21.15.66:0/1824986071 shutdown_connections 2024-03-30T00:40:16.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5ee2c00640 1 --2- 172.21.15.66:0/1824986071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc15a370 0x7f5edc0cea60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5ee2c00640 1 --2- 172.21.15.66:0/1824986071 >> v2:172.21.15.156:3300/0 conn(0x7f5edc0cda90 0x7f5edc0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5ee2c00640 1 -- 172.21.15.66:0/1824986071 >> 172.21.15.66:0/1824986071 conn(0x7f5edc069f60 msgr2=0x7f5edc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:16.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5ee2c00640 1 -- 172.21.15.66:0/1824986071 shutdown_connections 2024-03-30T00:40:16.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.905+0000 7f5ee2c00640 1 -- 172.21.15.66:0/1824986071 wait complete. 2024-03-30T00:40:16.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee2c00640 1 Processor -- start 2024-03-30T00:40:16.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee2c00640 1 -- start start 2024-03-30T00:40:16.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee2c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5edc0cda90 0x7f5edc157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee2c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc0cde70 0x7f5edc157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee2c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5edc15a370 0x7f5edc070a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee2c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc162e80 con 0x7f5edc0cda90 2024-03-30T00:40:16.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee2c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc071810 con 0x7f5edc15a370 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee2c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc1627c0 con 0x7f5edc0cde70 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5edbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5edc0cda90 0x7f5edc157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5edbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5edc0cda90 0x7f5edc157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56696/0 (socket says 172.21.15.66:56696) 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5edbe00640 1 -- 172.21.15.66:0/2221972426 learned_addr learned my addr 172.21.15.66:0/2221972426 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee0c00640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5edc15a370 0x7f5edc070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5edb400640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc0cde70 0x7f5edc157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5edbe00640 1 -- 172.21.15.66:0/2221972426 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc0cde70 msgr2=0x7f5edc157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5edbe00640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc0cde70 0x7f5edc157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5edbe00640 1 -- 172.21.15.66:0/2221972426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5edc15a370 msgr2=0x7f5edc070a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5edbe00640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5edc15a370 0x7f5edc070a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5edbe00640 1 -- 172.21.15.66:0/2221972426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed4009a70 con 0x7f5edc0cda90 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5edb400640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc0cde70 0x7f5edc157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5edbe00640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5edc0cda90 0x7f5edc157680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00e870 tx=0x7f5ecc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ed3e00640 1 -- 172.21.15.66:0/2221972426 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc009530 con 0x7f5edc0cda90 2024-03-30T00:40:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ed3e00640 1 -- 172.21.15.66:0/2221972426 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ecc00c940 con 0x7f5edc0cda90 2024-03-30T00:40:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ed3e00640 1 -- 172.21.15.66:0/2221972426 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc00ccd0 con 0x7f5edc0cda90 2024-03-30T00:40:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee2c00640 1 -- 172.21.15.66:0/2221972426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5edc162aa0 con 0x7f5edc0cda90 2024-03-30T00:40:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee2c00640 1 -- 172.21.15.66:0/2221972426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5edc097b80 con 0x7f5edc0cda90 2024-03-30T00:40:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.909+0000 7f5ee2c00640 1 -- 172.21.15.66:0/2221972426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5edc162620 con 0x7f5edc0cda90 2024-03-30T00:40:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.913+0000 7f5ed3e00640 1 -- 172.21.15.66:0/2221972426 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5ecc013470 con 0x7f5edc0cda90 2024-03-30T00:40:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.913+0000 7f5ed3e00640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ebc066690 0x7f5ebc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.913+0000 7f5ed3e00640 1 -- 172.21.15.66:0/2221972426 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5ecc0845a0 con 0x7f5edc0cda90 2024-03-30T00:40:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.913+0000 7f5edb400640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ebc066690 0x7f5ebc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.913+0000 7f5edb400640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ebc066690 0x7f5ebc068b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5edc071d30 tx=0x7f5ec8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:16.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:16.917+0000 7f5ed3e00640 1 -- 172.21.15.66:0/2221972426 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5edc162620 con 0x7f5edc0cda90 2024-03-30T00:40:17.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.077+0000 7f5ee2c00640 1 -- 172.21.15.66:0/2221972426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5edc162fc0 con 0x7f5edc0cda90 2024-03-30T00:40:17.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.077+0000 7f5ed3e00640 1 -- 172.21.15.66:0/2221972426 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f5edc162fc0 con 0x7f5edc0cda90 2024-03-30T00:40:17.088 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:17.088 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:17.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 -- 172.21.15.66:0/2221972426 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ebc066690 msgr2=0x7f5ebc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ebc066690 0x7f5ebc068b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5edc071d30 tx=0x7f5ec8009040 comp rx=0 tx=0).stop 2024-03-30T00:40:17.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 -- 172.21.15.66:0/2221972426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5edc0cda90 msgr2=0x7f5edc157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5edc0cda90 0x7f5edc157680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00e870 tx=0x7f5ecc00ed30 comp rx=0 tx=0).stop 2024-03-30T00:40:17.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 -- 172.21.15.66:0/2221972426 shutdown_connections 2024-03-30T00:40:17.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5edc15a370 0x7f5edc070a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5edc0cde70 0x7f5edc157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ebc066690 0x7f5ebc068b50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 --2- 172.21.15.66:0/2221972426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5edc0cda90 0x7f5edc157680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 -- 172.21.15.66:0/2221972426 >> 172.21.15.66:0/2221972426 conn(0x7f5edc069f60 msgr2=0x7f5edc164b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 -- 172.21.15.66:0/2221972426 shutdown_connections 2024-03-30T00:40:17.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.081+0000 7f5ee2c00640 1 -- 172.21.15.66:0/2221972426 wait complete. 2024-03-30T00:40:17.110 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-30T00:40:17.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-30T00:40:17.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-30T00:40:17.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-30T00:40:17.112 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-30T00:40:17.112 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-30T00:40:17.112 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-30T00:40:17.113 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-30T00:40:17.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7f7742000640 1 Processor -- start 2024-03-30T00:40:17.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7f7742000640 1 -- start start 2024-03-30T00:40:17.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7f7742000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f773c160b80 0x7f773c071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7f7742000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f773c160f60 0x7f773c071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7f7742000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f773c0cc870 0x7f773c164870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7ff559800640 1 Processor -- start 2024-03-30T00:40:17.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7ff559800640 1 -- start start 2024-03-30T00:40:17.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7f7742000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773c06f670 con 0x7f773c160b80 2024-03-30T00:40:17.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7f7742000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773c06ea60 con 0x7f773c160f60 2024-03-30T00:40:17.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7f7742000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c06ed40 con 0x7f773c0cc870 2024-03-30T00:40:17.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7f7740c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f773c160b80 0x7f773c071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7f7740c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f773c160b80 0x7f773c071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56710/0 (socket says 172.21.15.66:56710) 2024-03-30T00:40:17.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.345+0000 7f7740c00640 1 -- 172.21.15.66:0/3295577295 learned_addr learned my addr 172.21.15.66:0/3295577295 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7f7741600640 1 --2- 172.21.15.66:0/3295577295 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f773c0cc870 0x7f773c164870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7f773be00640 1 --2- 172.21.15.66:0/3295577295 >> v2:172.21.15.156:3300/0 conn(0x7f773c160f60 0x7f773c071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7ff559800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff554160dc0 0x7ff554161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7ff559800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5540b0b90 con 0x7ff5540712c0 2024-03-30T00:40:17.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7ff559800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5540ca760 con 0x7ff5540ca210 2024-03-30T00:40:17.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7ff559800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5541620b0 con 0x7ff554160dc0 2024-03-30T00:40:17.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7f7741600640 1 -- 172.21.15.66:0/3295577295 >> v2:172.21.15.156:3300/0 conn(0x7f773c160f60 msgr2=0x7f773c071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7f7741600640 1 --2- 172.21.15.66:0/3295577295 >> v2:172.21.15.156:3300/0 conn(0x7f773c160f60 0x7f773c071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7f7741600640 1 -- 172.21.15.66:0/3295577295 >> v2:172.21.15.66:3300/0 conn(0x7f773c160b80 msgr2=0x7f773c071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7f773be00640 1 --2- 172.21.15.66:0/3295577295 >> v2:172.21.15.156:3300/0 conn(0x7f773c160f60 0x7f773c071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:17.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7f7741600640 1 --2- 172.21.15.66:0/3295577295 >> v2:172.21.15.66:3300/0 conn(0x7f773c160b80 0x7f773c071840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7f7741600640 1 -- 172.21.15.66:0/3295577295 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c06f020 con 0x7f773c0cc870 2024-03-30T00:40:17.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7f7740c00640 1 --2- 172.21.15.66:0/3295577295 >> v2:172.21.15.66:3300/0 conn(0x7f773c160b80 0x7f773c071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:17.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7ff552a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff5540712c0 0x7ff5540716a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36370/0 (socket says 172.21.15.66:36370) 2024-03-30T00:40:17.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7ff552a00640 1 -- 172.21.15.66:0/2738737314 learned_addr learned my addr 172.21.15.66:0/2738737314 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:17.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7ff552000640 1 --2- 172.21.15.66:0/2738737314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff554160dc0 0x7ff554161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7fc3e7400640 1 Processor -- start 2024-03-30T00:40:17.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7fc3e7400640 1 -- start start 2024-03-30T00:40:17.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7ff552000640 1 -- 172.21.15.66:0/2738737314 >> v1:172.21.15.156:6789/0 conn(0x7ff5540ca210 legacy=0x7ff554163340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:40:17.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7ff552000640 1 -- 172.21.15.66:0/2738737314 >> v1:172.21.15.66:6789/0 conn(0x7ff5540712c0 legacy=0x7ff5540716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7ff552000640 1 -- 172.21.15.66:0/2738737314 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5541622a0 con 0x7ff554160dc0 2024-03-30T00:40:17.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7fc3e7400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3e00d0cc0 0x7fc3e015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7fc3e7400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc3e00d0130 0x7fc3e00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7fc3e7400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e01552f0 con 0x7fc3e00cd8d0 2024-03-30T00:40:17.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7fc3e7400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e00b6390 con 0x7fc3e00d0130 2024-03-30T00:40:17.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.349+0000 7fc3e7400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e006ceb0 con 0x7fc3e00d0cc0 2024-03-30T00:40:17.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3e5400640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc3e00cd8d0 0x7fc3e00cdf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:54962/0 (socket says 172.21.15.66:54962) 2024-03-30T00:40:17.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3e5400640 1 -- 172.21.15.66:0/3645572390 learned_addr learned my addr 172.21.15.66:0/3645572390 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:17.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f7741600640 1 --2- 172.21.15.66:0/3295577295 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f773c0cc870 0x7f773c164870 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f773c06db50 tx=0x7f773401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0af9ea250f4796f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f773b400640 1 -- 172.21.15.66:0/3295577295 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77340a59a0 con 0x7f773c0cc870 2024-03-30T00:40:17.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f773b400640 1 -- 172.21.15.66:0/3295577295 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77340a5b00 con 0x7f773c0cc870 2024-03-30T00:40:17.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f773b400640 1 -- 172.21.15.66:0/3295577295 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77340a5e90 con 0x7f773c0cc870 2024-03-30T00:40:17.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3e4a00640 1 --2- 172.21.15.66:0/3645572390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3e00d0cc0 0x7fc3e015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252636300 0 0) 0x7fc3e01552f0 con 0x7fc3e00cd8d0 2024-03-30T00:40:17.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3dfe00640 1 --2- 172.21.15.66:0/3645572390 >> v2:172.21.15.156:3300/0 conn(0x7fc3e00d0130 0x7fc3e00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3c4003530 con 0x7fc3e00cd8d0 2024-03-30T00:40:17.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f7742000640 1 -- 172.21.15.66:0/3295577295 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f773c0cc870 msgr2=0x7f773c164870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f7742000640 1 --2- 172.21.15.66:0/3295577295 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f773c0cc870 0x7f773c164870 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f773c06db50 tx=0x7f773401c900 comp rx=0 tx=0).stop 2024-03-30T00:40:17.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f7742000640 1 -- 172.21.15.66:0/3295577295 shutdown_connections 2024-03-30T00:40:17.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f7742000640 1 --2- 172.21.15.66:0/3295577295 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f773c0cc870 0x7f773c164870 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f7742000640 1 --2- 172.21.15.66:0/3295577295 >> v2:172.21.15.156:3300/0 conn(0x7f773c160f60 0x7f773c071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f7742000640 1 --2- 172.21.15.66:0/3295577295 >> v2:172.21.15.66:3300/0 conn(0x7f773c160b80 0x7f773c071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7ff552000640 1 --2- 172.21.15.66:0/2738737314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff554160dc0 0x7ff554161b70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff5400097f0 tx=0x7ff54001c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4f7af62ee5c189e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f7742000640 1 -- 172.21.15.66:0/3295577295 >> 172.21.15.66:0/3295577295 conn(0x7f773c06cf90 msgr2=0x7f773c06d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7ff551600640 1 -- 172.21.15.66:0/2738737314 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400ab600 con 0x7ff554160dc0 2024-03-30T00:40:17.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7ff551600640 1 -- 172.21.15.66:0/2738737314 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5400a7050 con 0x7ff554160dc0 2024-03-30T00:40:17.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7ff551600640 1 -- 172.21.15.66:0/2738737314 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400ab980 con 0x7ff554160dc0 2024-03-30T00:40:17.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7f7742000640 1 -- 172.21.15.66:0/3295577295 shutdown_connections 2024-03-30T00:40:17.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7f7742000640 1 -- 172.21.15.66:0/3295577295 wait complete. 2024-03-30T00:40:17.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1053668236 0 0) 0x7fc3c4003530 con 0x7fc3e00cd8d0 2024-03-30T00:40:17.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3e00d0cc0 msgr2=0x7fc3e015ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 --2- 172.21.15.66:0/3645572390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3e00d0cc0 0x7fc3e015ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 >> v2:172.21.15.156:3300/0 conn(0x7fc3e00d0130 msgr2=0x7fc3e00cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 --2- 172.21.15.66:0/3645572390 >> v2:172.21.15.156:3300/0 conn(0x7fc3e00d0130 0x7fc3e00cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e015b610 con 0x7fc3e00cd8d0 2024-03-30T00:40:17.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fc3d8001a00 con 0x7fc3e00cd8d0 2024-03-30T00:40:17.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 >> v1:172.21.15.156:6789/0 conn(0x7fc3e00cd8d0 legacy=0x7fc3e00cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 --2- 172.21.15.66:0/3645572390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3c4003980 0x7fc3c4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 --2- 172.21.15.66:0/3645572390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c40047d0 0x7fc3c4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 --2- 172.21.15.66:0/3645572390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c4005190 0x7fc3c400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e00b6390 con 0x7fc3c40047d0 2024-03-30T00:40:17.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e006ceb0 con 0x7fc3c4003980 2024-03-30T00:40:17.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.353+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c400bf00 con 0x7fc3c4005190 2024-03-30T00:40:17.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7fc3dfe00640 1 --2- 172.21.15.66:0/3645572390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c40047d0 0x7fc3c4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7fc3e5400640 1 --2- 172.21.15.66:0/3645572390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c4005190 0x7fc3c400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7ff559800640 1 -- 172.21.15.66:0/2738737314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff554160dc0 msgr2=0x7ff554161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7ff559800640 1 --2- 172.21.15.66:0/2738737314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff554160dc0 0x7ff554161b70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff5400097f0 tx=0x7ff54001c300 comp rx=0 tx=0).stop 2024-03-30T00:40:17.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7ff559800640 1 -- 172.21.15.66:0/2738737314 shutdown_connections 2024-03-30T00:40:17.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7ff559800640 1 --2- 172.21.15.66:0/2738737314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff554160dc0 0x7ff554161b70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7ff559800640 1 -- 172.21.15.66:0/2738737314 >> 172.21.15.66:0/2738737314 conn(0x7ff554070410 msgr2=0x7ff5540afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7fc3dfe00640 1 -- 172.21.15.66:0/3645572390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c4005190 msgr2=0x7fc3c400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7fc3dfe00640 1 --2- 172.21.15.66:0/3645572390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3c4005190 0x7fc3c400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7fc3dfe00640 1 -- 172.21.15.66:0/3645572390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3c4003980 msgr2=0x7fc3c4003da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:40:17.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7fc3dfe00640 1 --2- 172.21.15.66:0/3645572390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3c4003980 0x7fc3c4003da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7fc3dfe00640 1 -- 172.21.15.66:0/3645572390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e015b610 con 0x7fc3c40047d0 2024-03-30T00:40:17.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7f7742000640 1 Processor -- start 2024-03-30T00:40:17.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7fd89a400640 1 Processor -- start 2024-03-30T00:40:17.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7ff559800640 1 -- 172.21.15.66:0/2738737314 shutdown_connections 2024-03-30T00:40:17.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7f7742000640 1 -- start start 2024-03-30T00:40:17.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7ff559800640 1 -- 172.21.15.66:0/2738737314 wait complete. 2024-03-30T00:40:17.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.357+0000 7fd89a400640 1 -- start start 2024-03-30T00:40:17.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f1709a00640 1 Processor -- start 2024-03-30T00:40:17.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f1709a00640 1 -- start start 2024-03-30T00:40:17.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7742000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f773c0cc870 0x7f773c14e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7742000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f773c160b80 0x7f773c0afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7742000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c160f60 0x7f773c159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7742000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c14f3e0 con 0x7f773c160b80 2024-03-30T00:40:17.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7742000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c14f6c0 con 0x7f773c160f60 2024-03-30T00:40:17.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7742000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c14f9a0 con 0x7f773c0cc870 2024-03-30T00:40:17.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7ff559800640 1 Processor -- start 2024-03-30T00:40:17.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7fd89a400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd894161140 0x7fd8940cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7ff559800640 1 -- start start 2024-03-30T00:40:17.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7741600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c160f60 0x7f773c159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7741600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c160f60 0x7f773c159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:32934/0 (socket says 172.21.15.66:32934) 2024-03-30T00:40:17.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7741600640 1 -- 172.21.15.66:0/1540660424 learned_addr learned my addr 172.21.15.66:0/1540660424 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7fd89a400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8940cb3e0 con 0x7fd8940d1060 2024-03-30T00:40:17.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7fd893400640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd894160d60 0x7fd8940cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:54966/0 (socket says 172.21.15.66:54966) 2024-03-30T00:40:17.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7fd893400640 1 -- 172.21.15.66:0/3252743491 learned_addr learned my addr 172.21.15.66:0/3252743491 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:17.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f1709a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1704165010 0x7f17040b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f1709a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f17040b1a00 0x7f17040cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7fd892a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd894161140 0x7fd8940cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7fd89a400640 1 -- 172.21.15.66:0/3252743491 --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8940cb6c0 con 0x7fd894160d60 2024-03-30T00:40:17.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7fd89a400640 1 -- 172.21.15.66:0/3252743491 --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8940cb9a0 con 0x7fd894161140 2024-03-30T00:40:17.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7740c00640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f773c0cc870 0x7f773c14e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7ff559800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5540712c0 0x7ff554150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7ff559800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5540ca210 0x7ff5540cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7741600640 1 -- 172.21.15.66:0/1540660424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f773c0cc870 msgr2=0x7f773c14e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7741600640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f773c0cc870 0x7f773c14e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7741600640 1 -- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f773c160b80 msgr2=0x7f773c0afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7741600640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f773c160b80 0x7f773c0afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f7741600640 1 -- 172.21.15.66:0/1540660424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7734009a70 con 0x7f773c160f60 2024-03-30T00:40:17.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f1702a00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f1704164c30 0x7f17040cf8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:54978/0 (socket says 172.21.15.66:54978) 2024-03-30T00:40:17.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f1702000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1704165010 0x7f17040b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f1702000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1704165010 0x7f17040b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:32940/0 (socket says 172.21.15.66:32940) 2024-03-30T00:40:17.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7ff559800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff554150a50 0x7ff5540cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7f234f400640 1 Processor -- start 2024-03-30T00:40:17.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7fd892000640 1 -- 172.21.15.66:0/3252743491 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382180718 0 0) 0x7fd8940cb6c0 con 0x7fd894160d60 2024-03-30T00:40:17.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7ff559800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540cc9e0 con 0x7ff5540712c0 2024-03-30T00:40:17.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7fd892000640 1 -- 172.21.15.66:0/3252743491 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd878003530 con 0x7fd894160d60 2024-03-30T00:40:17.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7ff559800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540cccb0 con 0x7ff554150a50 2024-03-30T00:40:17.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7ff559800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540ccf90 con 0x7ff5540ca210 2024-03-30T00:40:17.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7ff552a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5540712c0 0x7ff554150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7ff552a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5540712c0 0x7ff554150510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56734/0 (socket says 172.21.15.66:56734) 2024-03-30T00:40:17.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.361+0000 7ff552a00640 1 -- 172.21.15.66:0/613565977 learned_addr learned my addr 172.21.15.66:0/613565977 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f7741600640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c160f60 0x7f773c159a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f773401afb0 tx=0x7f77340196a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7ff552000640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5540ca210 0x7ff5540cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7ff553400640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff554150a50 0x7ff5540cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7fd892a00640 1 -- 172.21.15.66:0/3252743491 >> v1:172.21.15.156:6789/0 conn(0x7fd894160d60 legacy=0x7fd8940cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f234f400640 1 -- start start 2024-03-30T00:40:17.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7fd892000640 1 -- 172.21.15.66:0/3252743491 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4061101514 0 0) 0x7fd878003530 con 0x7fd894160d60 2024-03-30T00:40:17.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7fd892a00640 1 -- 172.21.15.66:0/3252743491 >> v1:172.21.15.66:6789/0 conn(0x7fd8940d1060 legacy=0x7fd8940c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7fd892a00640 1 -- 172.21.15.66:0/3252743491 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8940cbc80 con 0x7fd894161140 2024-03-30T00:40:17.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f7738c00640 1 -- 172.21.15.66:0/1540660424 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77340adcf0 con 0x7f773c160f60 2024-03-30T00:40:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f7738c00640 1 -- 172.21.15.66:0/1540660424 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77340ade70 con 0x7f773c160f60 2024-03-30T00:40:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f7738c00640 1 -- 172.21.15.66:0/1540660424 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77340b9670 con 0x7f773c160f60 2024-03-30T00:40:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f7742000640 1 -- 172.21.15.66:0/1540660424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c14fc80 con 0x7f773c160f60 2024-03-30T00:40:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f1709a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17040c5ca0 con 0x7f1704164c30 2024-03-30T00:40:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f1709a00640 1 -- 172.21.15.66:0/1796063351 --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17040cffd0 con 0x7f17040b1a00 2024-03-30T00:40:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f1709a00640 1 -- 172.21.15.66:0/1796063351 --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17040d1390 con 0x7f1704165010 2024-03-30T00:40:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f7742000640 1 -- 172.21.15.66:0/1540660424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c14ed30 con 0x7f773c160f60 2024-03-30T00:40:17.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7fc3dfe00640 1 --2- 172.21.15.66:0/3645572390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c40047d0 0x7fc3c4004c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009ab0 tx=0x7fc3cc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c55b67acd297856 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f1702a00640 1 -- 172.21.15.66:0/1796063351 learned_addr learned my addr 172.21.15.66:0/1796063351 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:17.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7ff552000640 1 -- 172.21.15.66:0/613565977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff554150a50 msgr2=0x7ff5540cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f98ce600640 1 Processor -- start 2024-03-30T00:40:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7ff552000640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff554150a50 0x7ff5540cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7ff552000640 1 -- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5540712c0 msgr2=0x7ff554150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7ff552000640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5540712c0 0x7ff554150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7ff552000640 1 -- 172.21.15.66:0/613565977 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff540009420 con 0x7ff5540ca210 2024-03-30T00:40:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f98ce600640 1 -- start start 2024-03-30T00:40:17.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f1703400640 1 --2- 172.21.15.66:0/1796063351 >> v2:172.21.15.66:3300/0 conn(0x7f17040b1a00 0x7f17040cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f234f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f234815c620 0x7f234806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0a5680 con 0x7fc3c40047d0 2024-03-30T00:40:17.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3cc0a57e0 con 0x7fc3c40047d0 2024-03-30T00:40:17.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7fc3df400640 1 -- 172.21.15.66:0/3645572390 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0a5bd0 con 0x7fc3c40047d0 2024-03-30T00:40:17.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f234f400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23480cce00 con 0x7f23480cc7e0 2024-03-30T00:40:17.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f7742000640 1 -- 172.21.15.66:0/1540660424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f773c073e00 con 0x7f773c160f60 2024-03-30T00:40:17.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f98ce600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f98c806cdc0 0x7f98c806d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f98ce600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f98c80cbec0 0x7f98c80cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f98ce600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c8160540 con 0x7f98c80713c0 2024-03-30T00:40:17.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f98ce600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c80cc890 con 0x7f98c80cbec0 2024-03-30T00:40:17.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f98ce600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c80cb8c0 con 0x7f98c806cdc0 2024-03-30T00:40:17.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f234ca00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f23480cee10 0x7f23480cc0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:54984/0 (socket says 172.21.15.66:54984) 2024-03-30T00:40:17.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f234ca00640 1 -- 172.21.15.66:0/1619610748 learned_addr learned my addr 172.21.15.66:0/1619610748 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:17.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f234f400640 1 -- 172.21.15.66:0/1619610748 --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234806c540 con 0x7f23480cee10 2024-03-30T00:40:17.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f234f400640 1 -- 172.21.15.66:0/1619610748 --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23480cda50 con 0x7f234815c620 2024-03-30T00:40:17.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069705357 0 0) 0x7f234806c540 con 0x7f23480cee10 2024-03-30T00:40:17.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f232c005740 con 0x7f23480cee10 2024-03-30T00:40:17.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f98c7400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f98c806cdc0 0x7f98c806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f98c7400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f98c806cdc0 0x7f98c806d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:32964/0 (socket says 172.21.15.66:32964) 2024-03-30T00:40:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f98c7400640 1 -- 172.21.15.66:0/2664987809 learned_addr learned my addr 172.21.15.66:0/2664987809 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3405157264 0 0) 0x7f232c005740 con 0x7f23480cee10 2024-03-30T00:40:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 >> v2:172.21.15.66:3300/0 conn(0x7f234815c620 msgr2=0x7f234806d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 --2- 172.21.15.66:0/1619610748 >> v2:172.21.15.66:3300/0 conn(0x7f234815c620 0x7f234806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 >> v1:172.21.15.66:6789/0 conn(0x7f23480cc7e0 legacy=0x7f234806ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234806c820 con 0x7f23480cee10 2024-03-30T00:40:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f2338001680 con 0x7f23480cee10 2024-03-30T00:40:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 >> v1:172.21.15.156:6789/0 conn(0x7f23480cee10 legacy=0x7f23480cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 --2- 172.21.15.66:0/1619610748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f232c005b90 0x7f232c005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 --2- 172.21.15.66:0/1619610748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c0069e0 0x7f232c006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 --2- 172.21.15.66:0/1619610748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f232c0074c0 0x7f232c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23480cda50 con 0x7f232c0069e0 2024-03-30T00:40:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c00e3c0 con 0x7f232c005b90 2024-03-30T00:40:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232c00e580 con 0x7f232c0074c0 2024-03-30T00:40:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7fc3e7400640 1 -- 172.21.15.66:0/3645572390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c40047d0 msgr2=0x7fc3c4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7fc3e7400640 1 --2- 172.21.15.66:0/3645572390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c40047d0 0x7fc3c4004c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009ab0 tx=0x7fc3cc01ca30 comp rx=0 tx=0).stop 2024-03-30T00:40:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7fc3e5400640 1 -- 172.21.15.66:0/3645572390 reap_dead start 2024-03-30T00:40:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7ff552000640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5540ca210 0x7ff5540cbf60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff5400ab000 tx=0x7ff5400190a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7ff54aa00640 1 -- 172.21.15.66:0/613565977 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400b4600 con 0x7ff5540ca210 2024-03-30T00:40:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7ff54aa00640 1 -- 172.21.15.66:0/613565977 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5400b4760 con 0x7ff5540ca210 2024-03-30T00:40:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7ff54aa00640 1 -- 172.21.15.66:0/613565977 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400b4b10 con 0x7ff5540ca210 2024-03-30T00:40:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7fc3e7400640 1 -- 172.21.15.66:0/3645572390 shutdown_connections 2024-03-30T00:40:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7fc3e7400640 1 -- 172.21.15.66:0/3645572390 >> 172.21.15.66:0/3645572390 conn(0x7fc3e00b1f00 msgr2=0x7fc3e00b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f234ca00640 1 --2- 172.21.15.66:0/1619610748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f232c005b90 0x7f232c005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f5871400640 1 Processor -- start 2024-03-30T00:40:17.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7ff559800640 1 -- 172.21.15.66:0/613565977 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5540cd270 con 0x7ff5540ca210 2024-03-30T00:40:17.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7fd892a00640 1 --2- 172.21.15.66:0/3252743491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd894161140 0x7fd8940cab90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd884009740 tx=0x7fd88401c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dabdd53cdf014b7f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f234d400640 1 --2- 172.21.15.66:0/1619610748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f232c0074c0 0x7f232c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7ff559800640 1 -- 172.21.15.66:0/613565977 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5540cd830 con 0x7ff5540ca210 2024-03-30T00:40:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7fd892000640 1 -- 172.21.15.66:0/3252743491 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c001a00 con 0x7fd894161140 2024-03-30T00:40:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f1702000640 1 -- 172.21.15.66:0/1796063351 >> v2:172.21.15.66:3300/0 conn(0x7f17040b1a00 msgr2=0x7f17040cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f1702000640 1 --2- 172.21.15.66:0/1796063351 >> v2:172.21.15.66:3300/0 conn(0x7f17040b1a00 0x7f17040cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f1702000640 1 -- 172.21.15.66:0/1796063351 >> v1:172.21.15.156:6789/0 conn(0x7f1704164c30 legacy=0x7f17040cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.365+0000 7f1702000640 1 -- 172.21.15.66:0/1796063351 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17040d04a0 con 0x7f1704165010 2024-03-30T00:40:17.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7fd892000640 1 -- 172.21.15.66:0/3252743491 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8840a7050 con 0x7fd894161140 2024-03-30T00:40:17.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7fc3e7400640 1 -- 172.21.15.66:0/3645572390 shutdown_connections 2024-03-30T00:40:17.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7fd892000640 1 -- 172.21.15.66:0/3252743491 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8840ab620 con 0x7fd894161140 2024-03-30T00:40:17.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f98c6a00640 1 --2- 172.21.15.66:0/2664987809 >> v2:172.21.15.66:3300/0 conn(0x7f98c80cbec0 0x7f98c80cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f98c6a00640 1 -- 172.21.15.66:0/2664987809 >> v2:172.21.15.156:3300/0 conn(0x7f98c806cdc0 msgr2=0x7f98c806d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f98c6a00640 1 --2- 172.21.15.66:0/2664987809 >> v2:172.21.15.156:3300/0 conn(0x7f98c806cdc0 0x7f98c806d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f98c6a00640 1 -- 172.21.15.66:0/2664987809 >> v1:172.21.15.66:6789/0 conn(0x7f98c80713c0 legacy=0x7f98c8071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f98c6a00640 1 -- 172.21.15.66:0/2664987809 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c815deb0 con 0x7f98c80cbec0 2024-03-30T00:40:17.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f5871400640 1 -- start start 2024-03-30T00:40:17.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f5871400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f586c0b19a0 0x7f586c0c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f5871400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c0c1800 0x7f586c0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f5871400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586c0c97f0 con 0x7f586c160c10 2024-03-30T00:40:17.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f5871400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586c0c1c90 con 0x7f586c0b19a0 2024-03-30T00:40:17.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f5871400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0c9a30 con 0x7f586c0c1800 2024-03-30T00:40:17.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7fd89a400640 1 -- 172.21.15.66:0/3252743491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd894161140 msgr2=0x7fd8940cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f586a800640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f586c160c10 0x7f586c0c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:55000/0 (socket says 172.21.15.66:55000) 2024-03-30T00:40:17.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7fd89a400640 1 --2- 172.21.15.66:0/3252743491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd894161140 0x7fd8940cab90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd884009740 tx=0x7fd88401c2d0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f586b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c0c1800 0x7f586c0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f586b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c0c1800 0x7f586c0cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:50502/0 (socket says 172.21.15.66:50502) 2024-03-30T00:40:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f586b200640 1 -- 172.21.15.66:0/1930318448 learned_addr learned my addr 172.21.15.66:0/1930318448 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7ff559800640 1 -- 172.21.15.66:0/613565977 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff554162e20 con 0x7ff5540ca210 2024-03-30T00:40:17.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7ff54aa00640 1 -- 172.21.15.66:0/613565977 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff5400b2040 con 0x7ff5540ca210 2024-03-30T00:40:17.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f7738c00640 1 -- 172.21.15.66:0/1540660424 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f77340b97d0 con 0x7f773c160f60 2024-03-30T00:40:17.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f7738c00640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7720066760 0x7f7720068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f7738c00640 1 -- 172.21.15.66:0/1540660424 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f773c06e800 con 0x7f773c160f60 2024-03-30T00:40:17.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f7738c00640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x7f772006fd20 0x7f7720072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f7738c00640 1 -- 172.21.15.66:0/1540660424 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7720072830 con 0x7f772006fd20 2024-03-30T00:40:17.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f586b200640 1 -- 172.21.15.66:0/1930318448 >> v2:172.21.15.66:3300/0 conn(0x7f586c0b19a0 msgr2=0x7f586c0c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7fc3e7400640 1 -- 172.21.15.66:0/3645572390 wait complete. 2024-03-30T00:40:17.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f586b200640 1 --2- 172.21.15.66:0/1930318448 >> v2:172.21.15.66:3300/0 conn(0x7f586c0b19a0 0x7f586c0c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3e7400640 1 Processor -- start 2024-03-30T00:40:17.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f586b200640 1 -- 172.21.15.66:0/1930318448 >> v1:172.21.15.156:6789/0 conn(0x7f586c160c10 legacy=0x7f586c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f7740c00640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7720066760 0x7f7720068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f586b200640 1 -- 172.21.15.66:0/1930318448 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586c0c9d10 con 0x7f586c0c1800 2024-03-30T00:40:17.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fd89a400640 1 -- 172.21.15.66:0/3252743491 shutdown_connections 2024-03-30T00:40:17.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fd89a400640 1 --2- 172.21.15.66:0/3252743491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd894161140 0x7fd8940cab90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fd89a400640 1 -- 172.21.15.66:0/3252743491 >> 172.21.15.66:0/3252743491 conn(0x7fd8940d0540 msgr2=0x7fd8940d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f2347e00640 1 --2- 172.21.15.66:0/1619610748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c0069e0 0x7f232c006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f234ca00640 1 -- 172.21.15.66:0/1619610748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f232c0074c0 msgr2=0x7f232c00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f234ca00640 1 --2- 172.21.15.66:0/1619610748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f232c0074c0 0x7f232c00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f234ca00640 1 -- 172.21.15.66:0/1619610748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c0069e0 msgr2=0x7f232c006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f234ca00640 1 --2- 172.21.15.66:0/1619610748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c0069e0 0x7f232c006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.369+0000 7f234ca00640 1 -- 172.21.15.66:0/1619610748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234806c820 con 0x7f232c005b90 2024-03-30T00:40:17.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f234d400640 1 -- 172.21.15.66:0/1619610748 reap_dead start 2024-03-30T00:40:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f7738c00640 1 -- 172.21.15.66:0/1540660424 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77340ecae0 con 0x7f773c160f60 2024-03-30T00:40:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f1702000640 1 --2- 172.21.15.66:0/1796063351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1704165010 0x7f17040b14c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17040c61a0 tx=0x7f16f401c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b74e6db29ed0d650 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f773be00640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x7f772006fd20 0x7f7720072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3e7400640 1 -- start start 2024-03-30T00:40:17.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7ff54aa00640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff534066760 0x7ff534068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fd89a400640 1 -- 172.21.15.66:0/3252743491 shutdown_connections 2024-03-30T00:40:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fd89a400640 1 -- 172.21.15.66:0/3252743491 wait complete. 2024-03-30T00:40:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f1701600640 1 -- 172.21.15.66:0/1796063351 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f40ab600 con 0x7f1704165010 2024-03-30T00:40:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f1701600640 1 -- 172.21.15.66:0/1796063351 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f40a7050 con 0x7f1704165010 2024-03-30T00:40:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f1701600640 1 -- 172.21.15.66:0/1796063351 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f40ab980 con 0x7f1704165010 2024-03-30T00:40:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7ff54aa00640 1 -- 172.21.15.66:0/613565977 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff54011d9b0 con 0x7ff5540ca210 2024-03-30T00:40:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7ff552a00640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff534066760 0x7ff534068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f234ca00640 1 --2- 172.21.15.66:0/1619610748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f232c005b90 0x7f232c005f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f233800c260 tx=0x7f233801f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cc9f3f22a35fac8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7ff54aa00640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x7ff53406fda0 0x7ff5340721e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f773be00640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x7f772006fd20 0x7f7720072160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3e7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c40047d0 0x7fc3e015d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3e7400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3e00b14e0 0x7fc3e00cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f1709a00640 1 -- 172.21.15.66:0/1796063351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1704165010 msgr2=0x7f17040b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f1709a00640 1 --2- 172.21.15.66:0/1796063351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1704165010 0x7f17040b14c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17040c61a0 tx=0x7f16f401c300 comp rx=0 tx=0).stop 2024-03-30T00:40:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3e7400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3e00cf470 0x7fc3e01495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3e7400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e00cf8f0 con 0x7fc3c40047d0 2024-03-30T00:40:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3e7400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e015dd20 con 0x7fc3e00cf470 2024-03-30T00:40:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3e7400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e00b46b0 con 0x7fc3e00b14e0 2024-03-30T00:40:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7ff54aa00640 1 -- 172.21.15.66:0/613565977 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5340728b0 con 0x7ff53406fda0 2024-03-30T00:40:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338005ae0 con 0x7f232c005b90 2024-03-30T00:40:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23380ab070 con 0x7f232c005b90 2024-03-30T00:40:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7f2347400640 1 -- 172.21.15.66:0/1619610748 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338005e80 con 0x7f232c005b90 2024-03-30T00:40:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7ff54aa00640 1 -- 172.21.15.66:0/613565977 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5401222a0 con 0x7ff5540ca210 2024-03-30T00:40:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3e5400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3e00cf470 0x7fc3e01495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3e5400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3e00cf470 0x7fc3e01495f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:32984/0 (socket says 172.21.15.66:32984) 2024-03-30T00:40:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3e5400640 1 -- 172.21.15.66:0/2666277424 learned_addr learned my addr 172.21.15.66:0/2666277424 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7fc3dfe00640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3e00b14e0 0x7fc3e00cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f1709a00640 1 -- 172.21.15.66:0/1796063351 shutdown_connections 2024-03-30T00:40:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3e4a00640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c40047d0 0x7fc3e015d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f1709a00640 1 --2- 172.21.15.66:0/1796063351 >> v2:172.21.15.66:3300/0 conn(0x7f17040b1a00 0x7f17040cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f1709a00640 1 --2- 172.21.15.66:0/1796063351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1704165010 0x7f17040b14c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.373+0000 7ff553400640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x7ff53406fda0 0x7ff5340721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f98c6a00640 1 --2- 172.21.15.66:0/2664987809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98c80cbec0 0x7f98c80cc350 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f98c8072420 tx=0x7f98b801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e9fea01af3e08 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3e5400640 1 -- 172.21.15.66:0/2666277424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3e00b14e0 msgr2=0x7fc3e00cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3e5400640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3e00b14e0 0x7fc3e00cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3e5400640 1 -- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c40047d0 msgr2=0x7fc3e015d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3e5400640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c40047d0 0x7fc3e015d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3e5400640 1 -- 172.21.15.66:0/2666277424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e0148e60 con 0x7fc3e00cf470 2024-03-30T00:40:17.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3dfe00640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3e00b14e0 0x7fc3e00cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f234f400640 1 -- 172.21.15.66:0/1619610748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f232c005b90 msgr2=0x7f232c005f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f234f400640 1 --2- 172.21.15.66:0/1619610748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f232c005b90 0x7f232c005f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f233800c260 tx=0x7f233801f410 comp rx=0 tx=0).stop 2024-03-30T00:40:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f1709a00640 1 -- 172.21.15.66:0/1796063351 >> 172.21.15.66:0/1796063351 conn(0x7f170409ba30 msgr2=0x7f1704089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f7738c00640 1 -- 172.21.15.66:0/1540660424 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7720072830 con 0x7f772006fd20 2024-03-30T00:40:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f234f400640 1 -- 172.21.15.66:0/1619610748 shutdown_connections 2024-03-30T00:40:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f234f400640 1 --2- 172.21.15.66:0/1619610748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f232c005b90 0x7f232c005f90 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f233800c260 tx=0x7f233801f410 comp rx=0 tx=0).stop 2024-03-30T00:40:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f234f400640 1 -- 172.21.15.66:0/1619610748 >> 172.21.15.66:0/1619610748 conn(0x7f23480ce2f0 msgr2=0x7f23480ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f234f400640 1 -- 172.21.15.66:0/1619610748 shutdown_connections 2024-03-30T00:40:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f234f400640 1 -- 172.21.15.66:0/1619610748 wait complete. 2024-03-30T00:40:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f98c6000640 1 -- 172.21.15.66:0/2664987809 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b80ab680 con 0x7f98c80cbec0 2024-03-30T00:40:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f98c6000640 1 -- 172.21.15.66:0/2664987809 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b80ab7e0 con 0x7f98c80cbec0 2024-03-30T00:40:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f98c6000640 1 -- 172.21.15.66:0/2664987809 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b80abb70 con 0x7f98c80cbec0 2024-03-30T00:40:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f7740c00640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7720066760 0x7f7720068c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f772c009890 tx=0x7f772c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f1709a00640 1 -- 172.21.15.66:0/1796063351 shutdown_connections 2024-03-30T00:40:17.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f1709a00640 1 -- 172.21.15.66:0/1796063351 wait complete. 2024-03-30T00:40:17.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3e5400640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3e00cf470 0x7fc3e01495f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00d0510 tx=0x7fc3d8010f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f586b200640 1 --2- 172.21.15.66:0/1930318448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c0c1800 0x7f586c0cc280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5864009e50 tx=0x7f586401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c536226db56c02b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7ff553400640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x7ff53406fda0 0x7ff5340721e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7ff552a00640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff534066760 0x7ff534068c20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff544000c10 tx=0x7ff544007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f5869400640 1 -- 172.21.15.66:0/1930318448 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58640a58d0 con 0x7f586c0c1800 2024-03-30T00:40:17.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3dcc00640 1 -- 172.21.15.66:0/2666277424 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80072d0 con 0x7fc3e00cf470 2024-03-30T00:40:17.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3dcc00640 1 -- 172.21.15.66:0/2666277424 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d800e7c0 con 0x7fc3e00cf470 2024-03-30T00:40:17.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3dcc00640 1 -- 172.21.15.66:0/2666277424 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d800eb70 con 0x7fc3e00cf470 2024-03-30T00:40:17.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f98ce600640 1 -- 172.21.15.66:0/2664987809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98c80cbec0 msgr2=0x7f98c80cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f98ce600640 1 --2- 172.21.15.66:0/2664987809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98c80cbec0 0x7f98c80cc350 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f98c8072420 tx=0x7f98b801c4c0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f5869400640 1 -- 172.21.15.66:0/1930318448 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58640a5a30 con 0x7f586c0c1800 2024-03-30T00:40:17.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f5869400640 1 -- 172.21.15.66:0/1930318448 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58640a5dc0 con 0x7f586c0c1800 2024-03-30T00:40:17.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3e7400640 1 -- 172.21.15.66:0/2666277424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e00b4900 con 0x7fc3e00cf470 2024-03-30T00:40:17.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fc3e7400640 1 -- 172.21.15.66:0/2666277424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e01490d0 con 0x7fc3e00cf470 2024-03-30T00:40:17.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fd89a400640 1 Processor -- start 2024-03-30T00:40:17.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7fd89a400640 1 -- start start 2024-03-30T00:40:17.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f98ce600640 1 -- 172.21.15.66:0/2664987809 shutdown_connections 2024-03-30T00:40:17.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f98ce600640 1 --2- 172.21.15.66:0/2664987809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98c80cbec0 0x7f98c80cc350 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f98ce600640 1 --2- 172.21.15.66:0/2664987809 >> v2:172.21.15.156:3300/0 conn(0x7f98c806cdc0 0x7f98c806d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f98ce600640 1 -- 172.21.15.66:0/2664987809 >> 172.21.15.66:0/2664987809 conn(0x7f98c80ce2f0 msgr2=0x7f98c80ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7ff54aa00640 1 -- 172.21.15.66:0/613565977 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ff5340728b0 con 0x7ff53406fda0 2024-03-30T00:40:17.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f5871400640 1 -- 172.21.15.66:0/1930318448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c0c1800 msgr2=0x7f586c0cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.377+0000 7f5871400640 1 --2- 172.21.15.66:0/1930318448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c0c1800 0x7f586c0cc280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5864009e50 tx=0x7f586401ca00 comp rx=0 tx=0).stop 2024-03-30T00:40:17.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f5871400640 1 -- 172.21.15.66:0/1930318448 shutdown_connections 2024-03-30T00:40:17.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f5871400640 1 --2- 172.21.15.66:0/1930318448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c0c1800 0x7f586c0cc280 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f5871400640 1 --2- 172.21.15.66:0/1930318448 >> v2:172.21.15.66:3300/0 conn(0x7f586c0b19a0 0x7f586c0c12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f5871400640 1 -- 172.21.15.66:0/1930318448 >> 172.21.15.66:0/1930318448 conn(0x7f586c069f60 msgr2=0x7f586c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fc3e7400640 1 -- 172.21.15.66:0/2666277424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc3e00cce70 con 0x7fc3e00cf470 2024-03-30T00:40:17.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f234f400640 1 Processor -- start 2024-03-30T00:40:17.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f98ce600640 1 -- 172.21.15.66:0/2664987809 shutdown_connections 2024-03-30T00:40:17.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f234f400640 1 -- start start 2024-03-30T00:40:17.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f5871400640 1 -- 172.21.15.66:0/1930318448 shutdown_connections 2024-03-30T00:40:17.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f98ce600640 1 -- 172.21.15.66:0/2664987809 wait complete. 2024-03-30T00:40:17.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fd89a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8940d1060 0x7fd8940c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f1709a00640 1 Processor -- start 2024-03-30T00:40:17.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fc3dcc00640 1 -- 172.21.15.66:0/2666277424 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc3d800ecd0 con 0x7fc3e00cf470 2024-03-30T00:40:17.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fd89a400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd894160d60 0x7fd8941581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f1709a00640 1 -- start start 2024-03-30T00:40:17.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fd893400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8940d1060 0x7fd8940c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fd893400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8940d1060 0x7fd8940c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56790/0 (socket says 172.21.15.66:56790) 2024-03-30T00:40:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fd893400640 1 -- 172.21.15.66:0/1261833090 learned_addr learned my addr 172.21.15.66:0/1261833090 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fd89a400640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd894161140 0x7fd894156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f234f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23480c8bf0 0x7f2348151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f234f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23480c8fd0 0x7f2348096b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940c6e50 con 0x7fd8940d1060 2024-03-30T00:40:17.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd894158720 con 0x7fd894160d60 2024-03-30T00:40:17.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940b5ff0 con 0x7fd894161140 2024-03-30T00:40:17.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f234f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2348097090 0x7f23480ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f234f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23480c9ee0 con 0x7f2348097090 2024-03-30T00:40:17.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f234f400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2348097540 con 0x7f23480c8fd0 2024-03-30T00:40:17.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f234f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23480ca300 con 0x7f23480c8bf0 2024-03-30T00:40:17.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f5871400640 1 -- 172.21.15.66:0/1930318448 wait complete. 2024-03-30T00:40:17.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f5871400640 1 Processor -- start 2024-03-30T00:40:17.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f1709a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17040b1a00 0x7f17040b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f1709a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1704164c30 0x7f17040b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f1709a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1704165010 0x7f17040b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f1709a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17040b6940 con 0x7f1704164c30 2024-03-30T00:40:17.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f1709a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17040b6c20 con 0x7f17040b1a00 2024-03-30T00:40:17.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f1709a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17040b6f00 con 0x7f1704165010 2024-03-30T00:40:17.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fc3dcc00640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc3c0066760 0x7fc3c0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fc3dcc00640 1 -- 172.21.15.66:0/2666277424 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc3d80869e0 con 0x7fc3e00cf470 2024-03-30T00:40:17.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fd892a00640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd894160d60 0x7fd8941581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7fd893e00640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd894161140 0x7fd894156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd893e00640 1 -- 172.21.15.66:0/1261833090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd894160d60 msgr2=0x7fd8941581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd893e00640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd894160d60 0x7fd8941581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd893e00640 1 -- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8940d1060 msgr2=0x7fd8940c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd893e00640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8940d1060 0x7fd8940c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd893e00640 1 -- 172.21.15.66:0/1261833090 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd884009420 con 0x7fd894161140 2024-03-30T00:40:17.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd893e00640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd894161140 0x7fd894156bf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd88000c410 tx=0x7fd88000c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.381+0000 7f234ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23480c8bf0 0x7f2348151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f234ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23480c8bf0 0x7f2348151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:50524/0 (socket says 172.21.15.66:50524) 2024-03-30T00:40:17.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f234ca00640 1 -- 172.21.15.66:0/1158453265 learned_addr learned my addr 172.21.15.66:0/1158453265 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f2347e00640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23480c8fd0 0x7f2348096b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f234ca00640 1 -- 172.21.15.66:0/1158453265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23480c8fd0 msgr2=0x7f2348096b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f234ca00640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23480c8fd0 0x7f2348096b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f234ca00640 1 -- 172.21.15.66:0/1158453265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2348097090 msgr2=0x7f23480ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f234ca00640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2348097090 0x7f23480ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f234ca00640 1 -- 172.21.15.66:0/1158453265 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23481525e0 con 0x7f23480c8bf0 2024-03-30T00:40:17.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f1702a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17040b1a00 0x7f17040b79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f1702a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17040b1a00 0x7f17040b79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:33000/0 (socket says 172.21.15.66:33000) 2024-03-30T00:40:17.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f1702a00640 1 -- 172.21.15.66:0/1425883500 learned_addr learned my addr 172.21.15.66:0/1425883500 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f1703400640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1704165010 0x7f17040b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f1702a00640 1 -- 172.21.15.66:0/1425883500 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1704165010 msgr2=0x7f17040b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f1702a00640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1704165010 0x7f17040b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f1702a00640 1 -- 172.21.15.66:0/1425883500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1704164c30 msgr2=0x7f17040b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f1702a00640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1704164c30 0x7f17040b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f1702a00640 1 -- 172.21.15.66:0/1425883500 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f4009420 con 0x7f17040b1a00 2024-03-30T00:40:17.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98ce600640 1 Processor -- start 2024-03-30T00:40:17.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98ce600640 1 -- start start 2024-03-30T00:40:17.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f5871400640 1 -- start start 2024-03-30T00:40:17.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd88b400640 1 -- 172.21.15.66:0/1261833090 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880005230 con 0x7fd894161140 2024-03-30T00:40:17.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd88b400640 1 -- 172.21.15.66:0/1261833090 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd880005390 con 0x7fd894161140 2024-03-30T00:40:17.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd88b400640 1 -- 172.21.15.66:0/1261833090 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880011890 con 0x7fd894161140 2024-03-30T00:40:17.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f234ca00640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23480c8bf0 0x7f2348151d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f233801fcb0 tx=0x7f232c007a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f1702a00640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17040b1a00 0x7f17040b79e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00caf0 tx=0x7f16fc002150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f2344c00640 1 -- 172.21.15.66:0/1158453265 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338004c10 con 0x7f23480c8bf0 2024-03-30T00:40:17.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8940b5a30 con 0x7fd894161140 2024-03-30T00:40:17.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8940c4530 con 0x7fd894161140 2024-03-30T00:40:17.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f2344c00640 1 -- 172.21.15.66:0/1158453265 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2338004d70 con 0x7f23480c8bf0 2024-03-30T00:40:17.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f2344c00640 1 -- 172.21.15.66:0/1158453265 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23380072b0 con 0x7f23480c8bf0 2024-03-30T00:40:17.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f234f400640 1 -- 172.21.15.66:0/1158453265 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23480c9540 con 0x7f23480c8bf0 2024-03-30T00:40:17.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fc3dcc00640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x7fc3c006fda0 0x7fc3c00721e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fc3e4a00640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc3c0066760 0x7fc3c0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fc3e4a00640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc3c0066760 0x7fc3c0068c20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3e006f000 tx=0x7fc3d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fc3dcc00640 1 -- 172.21.15.66:0/2666277424 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3c00728b0 con 0x7fc3c006fda0 2024-03-30T00:40:17.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fc3dcc00640 1 -- 172.21.15.66:0/2666277424 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3d808b2a0 con 0x7fc3e00cf470 2024-03-30T00:40:17.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f234f400640 1 -- 172.21.15.66:0/1158453265 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23480c9b00 con 0x7f23480c8bf0 2024-03-30T00:40:17.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98ce600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98c806cdc0 0x7f98c8201990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98ce600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98c80713c0 0x7f98c80c1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98ce600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c80cbec0 0x7f98c80c2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98ce600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c80b20e0 con 0x7f98c806cdc0 2024-03-30T00:40:17.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98ce600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c8201ed0 con 0x7f98c80cbec0 2024-03-30T00:40:17.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98ce600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c80c51d0 con 0x7f98c80713c0 2024-03-30T00:40:17.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98c806cdc0 0x7f98c8201990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98c806cdc0 0x7f98c8201990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56812/0 (socket says 172.21.15.66:56812) 2024-03-30T00:40:17.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98c7400640 1 -- 172.21.15.66:0/3717725501 learned_addr learned my addr 172.21.15.66:0/3717725501 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f234f400640 1 -- 172.21.15.66:0/1158453265 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2348159ec0 con 0x7f23480c8bf0 2024-03-30T00:40:17.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f98c7e00640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c80cbec0 0x7f98c80c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f16faa00640 1 -- 172.21.15.66:0/1425883500 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00a7c0 con 0x7f17040b1a00 2024-03-30T00:40:17.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd894066c30 con 0x7fd894161140 2024-03-30T00:40:17.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f16faa00640 1 -- 172.21.15.66:0/1425883500 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc00a920 con 0x7f17040b1a00 2024-03-30T00:40:17.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.385+0000 7f16faa00640 1 -- 172.21.15.66:0/1425883500 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00acf0 con 0x7f17040b1a00 2024-03-30T00:40:17.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98c6a00640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98c80713c0 0x7f98c80c1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98c7400640 1 -- 172.21.15.66:0/3717725501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98c80713c0 msgr2=0x7f98c80c1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98c7400640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98c80713c0 0x7f98c80c1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98c7400640 1 -- 172.21.15.66:0/3717725501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c80cbec0 msgr2=0x7f98c80c2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98c7400640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c80cbec0 0x7f98c80c2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98c7400640 1 -- 172.21.15.66:0/3717725501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b8009420 con 0x7f98c806cdc0 2024-03-30T00:40:17.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98c7400640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98c806cdc0 0x7f98c8201990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f98b400e820 tx=0x7f98b400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f1709a00640 1 -- 172.21.15.66:0/1425883500 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17040b71e0 con 0x7f17040b1a00 2024-03-30T00:40:17.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f1709a00640 1 -- 172.21.15.66:0/1425883500 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17040b64a0 con 0x7f17040b1a00 2024-03-30T00:40:17.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5871400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c0b19a0 0x7f586c071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7fd88b400640 1 -- 172.21.15.66:0/1261833090 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd8800119f0 con 0x7fd894161140 2024-03-30T00:40:17.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7fc3dfe00640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x7fc3c006fda0 0x7fc3c00721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98bf400640 1 -- 172.21.15.66:0/3717725501 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4009580 con 0x7f98c806cdc0 2024-03-30T00:40:17.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98ce600640 1 -- 172.21.15.66:0/3717725501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c80b0090 con 0x7f98c806cdc0 2024-03-30T00:40:17.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5871400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586c0c1800 0x7f586c162d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f586a800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c0b19a0 0x7f586c071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5871400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c160c10 0x7f586c163280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5871400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0c4750 con 0x7f586c0c1800 2024-03-30T00:40:17.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5871400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c071ed0 con 0x7f586c0b19a0 2024-03-30T00:40:17.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f586a800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c0b19a0 0x7f586c071990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:33016/0 (socket says 172.21.15.66:33016) 2024-03-30T00:40:17.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f586a800640 1 -- 172.21.15.66:0/1182875318 learned_addr learned my addr 172.21.15.66:0/1182875318 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0c3880 con 0x7f586c160c10 2024-03-30T00:40:17.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5869e00640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586c0c1800 0x7f586c162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7fd88b400640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd8740666e0 0x7fd874068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f1709a00640 1 -- 172.21.15.66:0/1425883500 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f17040b6d90 con 0x7f17040b1a00 2024-03-30T00:40:17.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98bf400640 1 -- 172.21.15.66:0/3717725501 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b400c940 con 0x7f98c806cdc0 2024-03-30T00:40:17.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98bf400640 1 -- 172.21.15.66:0/3717725501 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4009580 con 0x7f98c806cdc0 2024-03-30T00:40:17.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f98ce600640 1 -- 172.21.15.66:0/3717725501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c80c4c10 con 0x7f98c806cdc0 2024-03-30T00:40:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7fd88b400640 1 -- 172.21.15.66:0/1261833090 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd8800837d0 con 0x7fd894161140 2024-03-30T00:40:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7fc3dfe00640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x7fc3c006fda0 0x7fc3c00721e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7fd88b400640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x7fd87406fd20 0x7fd874072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7fd88b400640 1 -- 172.21.15.66:0/1261833090 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd874072830 con 0x7fd87406fd20 2024-03-30T00:40:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7fd88b400640 1 -- 172.21.15.66:0/1261833090 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8800882c0 con 0x7fd894161140 2024-03-30T00:40:17.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5869e00640 1 -- 172.21.15.66:0/1182875318 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c160c10 msgr2=0x7f586c163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5869e00640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c160c10 0x7f586c163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5869e00640 1 -- 172.21.15.66:0/1182875318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c0b19a0 msgr2=0x7f586c071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5869e00640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c0b19a0 0x7f586c071990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f5869e00640 1 -- 172.21.15.66:0/1182875318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5864009b30 con 0x7f586c0c1800 2024-03-30T00:40:17.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7fd893400640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd8740666e0 0x7fd874068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.389+0000 7f2344c00640 1 -- 172.21.15.66:0/1158453265 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2338007410 con 0x7f23480c8bf0 2024-03-30T00:40:17.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f5869e00640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586c0c1800 0x7f586c162d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f585800b420 tx=0x7f585800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f98ce600640 1 -- 172.21.15.66:0/3717725501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98c806dc80 con 0x7f98c806cdc0 2024-03-30T00:40:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f2344c00640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2328066790 0x7f2328068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f2344c00640 1 -- 172.21.15.66:0/1158453265 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f23380b7070 con 0x7f23480c8bf0 2024-03-30T00:40:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f2344c00640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x7f232806fdd0 0x7f2328072210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f5862a00640 1 -- 172.21.15.66:0/1182875318 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5858011830 con 0x7f586c0c1800 2024-03-30T00:40:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7fd893400640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd8740666e0 0x7fd874068ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0012f0 tx=0x7fd88c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f16faa00640 1 -- 172.21.15.66:0/1425883500 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f16fc0070d0 con 0x7f17040b1a00 2024-03-30T00:40:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f5862a00640 1 -- 172.21.15.66:0/1182875318 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5858011990 con 0x7f586c0c1800 2024-03-30T00:40:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f5862a00640 1 -- 172.21.15.66:0/1182875318 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5858011d00 con 0x7f586c0c1800 2024-03-30T00:40:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c0c3ad0 con 0x7f586c0c1800 2024-03-30T00:40:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586c0c4090 con 0x7f586c0c1800 2024-03-30T00:40:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7fc3dcc00640 1 -- 172.21.15.66:0/2666277424 <== osd.2 v2:172.21.15.66:6816/27464 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc3c00728b0 con 0x7fc3c006fda0 2024-03-30T00:40:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f16faa00640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f16e4066710 0x7f16e4068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f16faa00640 1 -- 172.21.15.66:0/1425883500 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f16fc083fd0 con 0x7f17040b1a00 2024-03-30T00:40:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f16faa00640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x7f16e406fd50 0x7f16e4072190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f16faa00640 1 -- 172.21.15.66:0/1425883500 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16e4072860 con 0x7f16e406fd50 2024-03-30T00:40:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f16faa00640 1 -- 172.21.15.66:0/1425883500 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16fc0866e0 con 0x7f17040b1a00 2024-03-30T00:40:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f1702000640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f16e4066710 0x7f16e4068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5844000fd0 con 0x7f586c0c1800 2024-03-30T00:40:17.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7fd892a00640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x7fd87406fd20 0x7fd874072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f1703400640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x7f16e406fd50 0x7f16e4072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f2344c00640 1 -- 172.21.15.66:0/1158453265 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23280728e0 con 0x7f232806fdd0 2024-03-30T00:40:17.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f2344c00640 1 -- 172.21.15.66:0/1158453265 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23381236c0 con 0x7f23480c8bf0 2024-03-30T00:40:17.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f2347e00640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2328066790 0x7f2328068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.393+0000 7f234d400640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x7f232806fdd0 0x7f2328072210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f1702000640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f16e4066710 0x7f16e4068bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16f40ad000 tx=0x7f16f4004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f2347e00640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2328066790 0x7f2328068c50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f234815bc20 tx=0x7f2334009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f234d400640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x7f232806fdd0 0x7f2328072210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7fd892a00640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x7fd87406fd20 0x7fd874072160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f98bf400640 1 -- 172.21.15.66:0/3717725501 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f98b400cab0 con 0x7f98c806cdc0 2024-03-30T00:40:17.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f2344c00640 1 -- 172.21.15.66:0/1158453265 <== osd.4 v2:172.21.15.156:6800/27641 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f23280728e0 con 0x7f232806fdd0 2024-03-30T00:40:17.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f1703400640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x7f16e406fd50 0x7f16e4072190 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7fd88b400640 1 -- 172.21.15.66:0/1261833090 <== osd.3 v2:172.21.15.66:6808/27471 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fd874072830 con 0x7fd87406fd20 2024-03-30T00:40:17.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f5862a00640 1 -- 172.21.15.66:0/1182875318 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f585801b460 con 0x7f586c0c1800 2024-03-30T00:40:17.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f98bf400640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f98a80666f0 0x7f98a8068bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f98bf400640 1 -- 172.21.15.66:0/3717725501 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f98b4084510 con 0x7f98c806cdc0 2024-03-30T00:40:17.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f98bf400640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x7f98a806fcb0 0x7f98a80720f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f98bf400640 1 -- 172.21.15.66:0/3717725501 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f98a80727c0 con 0x7f98a806fcb0 2024-03-30T00:40:17.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f98bf400640 1 -- 172.21.15.66:0/3717725501 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98b4084880 con 0x7f98c806cdc0 2024-03-30T00:40:17.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f98c6a00640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f98a80666f0 0x7f98a8068bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.397+0000 7f5862a00640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f584c066690 0x7f584c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f98c7e00640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x7f98a806fcb0 0x7f98a80720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f5862a00640 1 -- 172.21.15.66:0/1182875318 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5858084bb0 con 0x7f586c0c1800 2024-03-30T00:40:17.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f98c6a00640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f98a80666f0 0x7f98a8068bb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f98b80af000 tx=0x7f98b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f5862a00640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x7f584c06fcd0 0x7f584c072110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f5862a00640 1 -- 172.21.15.66:0/1182875318 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f584c0727e0 con 0x7f584c06fcd0 2024-03-30T00:40:17.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f5862a00640 1 -- 172.21.15.66:0/1182875318 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5858084f60 con 0x7f586c0c1800 2024-03-30T00:40:17.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f586a800640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f584c066690 0x7f584c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f98c7e00640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x7f98a806fcb0 0x7f98a80720f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f586b200640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x7f584c06fcd0 0x7f584c072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f586a800640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f584c066690 0x7f584c068b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f585c000f70 tx=0x7f585c007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f586b200640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x7f584c06fcd0 0x7f584c072110 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f98bf400640 1 -- 172.21.15.66:0/3717725501 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f98a80727c0 con 0x7f98a806fcb0 2024-03-30T00:40:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.401+0000 7f5862a00640 1 -- 172.21.15.66:0/1182875318 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f584c0727e0 con 0x7f584c06fcd0 2024-03-30T00:40:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.405+0000 7f16faa00640 1 -- 172.21.15.66:0/1425883500 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f16e4072860 con 0x7f16e406fd50 2024-03-30T00:40:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.405+0000 7f7742000640 1 -- 172.21.15.66:0/1540660424 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f773c1501a0 con 0x7f772006fd20 2024-03-30T00:40:17.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.405+0000 7ff559800640 1 -- 172.21.15.66:0/613565977 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff554073e00 con 0x7ff53406fda0 2024-03-30T00:40:17.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff54aa00640 1 -- 172.21.15.66:0/613565977 <== osd.0 v2:172.21.15.66:6800/27456 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff554073e00 con 0x7ff53406fda0 2024-03-30T00:40:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff559800640 1 -- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x7ff53406fda0 msgr2=0x7ff5340721e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff559800640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x7ff53406fda0 0x7ff5340721e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff559800640 1 -- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff534066760 msgr2=0x7ff534068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff559800640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff534066760 0x7ff534068c20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff544000c10 tx=0x7ff544007cf0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff559800640 1 -- 172.21.15.66:0/613565977 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5540ca210 msgr2=0x7ff5540cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff559800640 1 --2- 172.21.15.66:0/613565977 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5540ca210 0x7ff5540cbf60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff5400ab000 tx=0x7ff5400190a0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff553400640 1 -- 172.21.15.66:0/613565977 reap_dead start 2024-03-30T00:40:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff559800640 1 -- 172.21.15.66:0/613565977 shutdown_connections 2024-03-30T00:40:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff559800640 1 -- 172.21.15.66:0/613565977 >> 172.21.15.66:0/613565977 conn(0x7ff554070410 msgr2=0x7ff554154380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7f7738c00640 1 -- 172.21.15.66:0/1540660424 <== osd.1 v2:172.21.15.66:6821/27458 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f773c1501a0 con 0x7f772006fd20 2024-03-30T00:40:17.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff559800640 1 -- 172.21.15.66:0/613565977 shutdown_connections 2024-03-30T00:40:17.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7ff559800640 1 -- 172.21.15.66:0/613565977 wait complete. 2024-03-30T00:40:17.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7f7742000640 1 -- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x7f772006fd20 msgr2=0x7f7720072160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.409+0000 7f7742000640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x7f772006fd20 0x7f7720072160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.413+0000 7f7742000640 1 -- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7720066760 msgr2=0x7f7720068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.413+0000 7f7742000640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7720066760 0x7f7720068c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f772c009890 tx=0x7f772c008040 comp rx=0 tx=0).stop 2024-03-30T00:40:17.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.413+0000 7f7742000640 1 -- 172.21.15.66:0/1540660424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c160f60 msgr2=0x7f773c159a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.413+0000 7f7742000640 1 --2- 172.21.15.66:0/1540660424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c160f60 0x7f773c159a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f773401afb0 tx=0x7f77340196a0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.413+0000 7f7741600640 1 -- 172.21.15.66:0/1540660424 reap_dead start 2024-03-30T00:40:17.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.413+0000 7f7742000640 1 -- 172.21.15.66:0/1540660424 shutdown_connections 2024-03-30T00:40:17.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.413+0000 7f7742000640 1 -- 172.21.15.66:0/1540660424 >> 172.21.15.66:0/1540660424 conn(0x7f773c06cf90 msgr2=0x7f773c162340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.413+0000 7f7742000640 1 -- 172.21.15.66:0/1540660424 shutdown_connections 2024-03-30T00:40:17.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.413+0000 7f7742000640 1 -- 172.21.15.66:0/1540660424 wait complete. 2024-03-30T00:40:17.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.417+0000 7f234f400640 1 -- 172.21.15.66:0/1158453265 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2348159be0 con 0x7f232806fdd0 2024-03-30T00:40:17.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.417+0000 7f2344c00640 1 -- 172.21.15.66:0/1158453265 <== osd.4 v2:172.21.15.156:6800/27641 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2340009b10 con 0x7f232806fdd0 2024-03-30T00:40:17.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.417+0000 7f234f400640 1 -- 172.21.15.66:0/1158453265 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x7f232806fdd0 msgr2=0x7f2328072210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.417+0000 7f234f400640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x7f232806fdd0 0x7f2328072210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.417+0000 7f234f400640 1 -- 172.21.15.66:0/1158453265 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2328066790 msgr2=0x7f2328068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.417+0000 7f234f400640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2328066790 0x7f2328068c50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f234815bc20 tx=0x7f2334009040 comp rx=0 tx=0).stop 2024-03-30T00:40:17.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.417+0000 7f234f400640 1 -- 172.21.15.66:0/1158453265 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23480c8bf0 msgr2=0x7f2348151d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.417+0000 7f234f400640 1 --2- 172.21.15.66:0/1158453265 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23480c8bf0 0x7f2348151d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f233801fcb0 tx=0x7f232c007a10 comp rx=0 tx=0).stop 2024-03-30T00:40:17.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.417+0000 7f234d400640 1 -- 172.21.15.66:0/1158453265 reap_dead start 2024-03-30T00:40:17.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.417+0000 7f234f400640 1 -- 172.21.15.66:0/1158453265 shutdown_connections 2024-03-30T00:40:17.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.417+0000 7f234f400640 1 -- 172.21.15.66:0/1158453265 >> 172.21.15.66:0/1158453265 conn(0x7f23480ce2f0 msgr2=0x7f23480acb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.421+0000 7f234f400640 1 -- 172.21.15.66:0/1158453265 shutdown_connections 2024-03-30T00:40:17.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.421+0000 7f234f400640 1 -- 172.21.15.66:0/1158453265 wait complete. 2024-03-30T00:40:17.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.421+0000 7fc3e7400640 1 -- 172.21.15.66:0/2666277424 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc3e015bbd0 con 0x7fc3c006fda0 2024-03-30T00:40:17.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.425+0000 7fc3dcc00640 1 -- 172.21.15.66:0/2666277424 <== osd.2 v2:172.21.15.66:6816/27464 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc3cc0b7440 con 0x7fc3c006fda0 2024-03-30T00:40:17.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fc3e7400640 1 -- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x7fc3c006fda0 msgr2=0x7fc3c00721e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fc3e7400640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x7fc3c006fda0 0x7fc3c00721e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fc3e7400640 1 -- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc3c0066760 msgr2=0x7fc3c0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fc3e7400640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc3c0066760 0x7fc3c0068c20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3e006f000 tx=0x7fc3d0008040 comp rx=0 tx=0).stop 2024-03-30T00:40:17.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fc3e7400640 1 -- 172.21.15.66:0/2666277424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3e00cf470 msgr2=0x7fc3e01495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fc3e7400640 1 --2- 172.21.15.66:0/2666277424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3e00cf470 0x7fc3e01495f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00d0510 tx=0x7fc3d8010f90 comp rx=0 tx=0).stop 2024-03-30T00:40:17.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fc3e5400640 1 -- 172.21.15.66:0/2666277424 reap_dead start 2024-03-30T00:40:17.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fc3e7400640 1 -- 172.21.15.66:0/2666277424 shutdown_connections 2024-03-30T00:40:17.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fc3e7400640 1 -- 172.21.15.66:0/2666277424 >> 172.21.15.66:0/2666277424 conn(0x7fc3e00b1f00 msgr2=0x7fc3e00b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fc3e7400640 1 -- 172.21.15.66:0/2666277424 shutdown_connections 2024-03-30T00:40:17.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fc3e7400640 1 -- 172.21.15.66:0/2666277424 wait complete. 2024-03-30T00:40:17.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd89406e900 con 0x7fd87406fd20 2024-03-30T00:40:17.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5844002cc0 con 0x7f584c06fcd0 2024-03-30T00:40:17.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.429+0000 7f98ce600640 1 -- 172.21.15.66:0/3717725501 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f98c8161030 con 0x7f98a806fcb0 2024-03-30T00:40:17.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f1709a00640 1 -- 172.21.15.66:0/1425883500 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f17040b1e40 con 0x7f16e406fd50 2024-03-30T00:40:17.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd88b400640 1 -- 172.21.15.66:0/1261833090 <== osd.3 v2:172.21.15.66:6808/27471 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd89406e900 con 0x7fd87406fd20 2024-03-30T00:40:17.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f98bf400640 1 -- 172.21.15.66:0/3717725501 <== osd.6 v2:172.21.15.156:6816/27642 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f98a80727c0 con 0x7f98a806fcb0 2024-03-30T00:40:17.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f5862a00640 1 -- 172.21.15.66:0/1182875318 <== osd.7 v2:172.21.15.156:6824/27643 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5844002cc0 con 0x7f584c06fcd0 2024-03-30T00:40:17.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f16faa00640 1 -- 172.21.15.66:0/1425883500 <== osd.5 v2:172.21.15.156:6808/27644 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f16f00075c0 con 0x7f16e406fd50 2024-03-30T00:40:17.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x7fd87406fd20 msgr2=0x7fd874072160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd89a400640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x7fd87406fd20 0x7fd874072160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x7f584c06fcd0 msgr2=0x7f584c072110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f5871400640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x7f584c06fcd0 0x7f584c072110 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd8740666e0 msgr2=0x7fd874068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd89a400640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd8740666e0 0x7fd874068ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0012f0 tx=0x7fd88c009040 comp rx=0 tx=0).stop 2024-03-30T00:40:17.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd894161140 msgr2=0x7fd894156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd89a400640 1 --2- 172.21.15.66:0/1261833090 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd894161140 0x7fd894156bf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd88000c410 tx=0x7fd88000c8d0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f584c066690 msgr2=0x7f584c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f5871400640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f584c066690 0x7f584c068b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f585c000f70 tx=0x7f585c007ce0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586c0c1800 msgr2=0x7f586c162d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f5871400640 1 --2- 172.21.15.66:0/1182875318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586c0c1800 0x7f586c162d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f585800b420 tx=0x7f585800b8e0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd893e00640 1 -- 172.21.15.66:0/1261833090 reap_dead start 2024-03-30T00:40:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f586b200640 1 -- 172.21.15.66:0/1182875318 reap_dead start 2024-03-30T00:40:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f1709a00640 1 -- 172.21.15.66:0/1425883500 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x7f16e406fd50 msgr2=0x7f16e4072190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f1709a00640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x7f16e406fd50 0x7f16e4072190 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f98bcc00640 1 -- 172.21.15.66:0/3717725501 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x7f98a806fcb0 msgr2=0x7f98a80720f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 shutdown_connections 2024-03-30T00:40:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 >> 172.21.15.66:0/1182875318 conn(0x7f586c069f60 msgr2=0x7f586c070030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f98bcc00640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x7f98a806fcb0 0x7f98a80720f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 shutdown_connections 2024-03-30T00:40:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 >> 172.21.15.66:0/1261833090 conn(0x7fd8940d0540 msgr2=0x7fd894159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 shutdown_connections 2024-03-30T00:40:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f5871400640 1 -- 172.21.15.66:0/1182875318 wait complete. 2024-03-30T00:40:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f1709a00640 1 -- 172.21.15.66:0/1425883500 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f16e4066710 msgr2=0x7f16e4068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f1709a00640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f16e4066710 0x7f16e4068bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16f40ad000 tx=0x7f16f4004740 comp rx=0 tx=0).stop 2024-03-30T00:40:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f1709a00640 1 -- 172.21.15.66:0/1425883500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17040b1a00 msgr2=0x7f17040b79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f1709a00640 1 --2- 172.21.15.66:0/1425883500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17040b1a00 0x7f17040b79e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00caf0 tx=0x7f16fc002150 comp rx=0 tx=0).stop 2024-03-30T00:40:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f98bcc00640 1 -- 172.21.15.66:0/3717725501 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f98a80666f0 msgr2=0x7f98a8068bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f1703400640 1 -- 172.21.15.66:0/1425883500 reap_dead start 2024-03-30T00:40:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f98bcc00640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f98a80666f0 0x7f98a8068bb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f98b80af000 tx=0x7f98b8008040 comp rx=0 tx=0).stop 2024-03-30T00:40:17.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f1709a00640 1 -- 172.21.15.66:0/1425883500 shutdown_connections 2024-03-30T00:40:17.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f1709a00640 1 -- 172.21.15.66:0/1425883500 >> 172.21.15.66:0/1425883500 conn(0x7f170409ba30 msgr2=0x7f170406c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 shutdown_connections 2024-03-30T00:40:17.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7fd89a400640 1 -- 172.21.15.66:0/1261833090 wait complete. 2024-03-30T00:40:17.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f98bcc00640 1 -- 172.21.15.66:0/3717725501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98c806cdc0 msgr2=0x7f98c8201990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f98bcc00640 1 --2- 172.21.15.66:0/3717725501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98c806cdc0 0x7f98c8201990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f98b400e820 tx=0x7f98b400ece0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.433+0000 7f98c7e00640 1 -- 172.21.15.66:0/3717725501 reap_dead start 2024-03-30T00:40:17.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.437+0000 7f1709a00640 1 -- 172.21.15.66:0/1425883500 shutdown_connections 2024-03-30T00:40:17.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.437+0000 7f1709a00640 1 -- 172.21.15.66:0/1425883500 wait complete. 2024-03-30T00:40:17.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.437+0000 7f98bcc00640 1 -- 172.21.15.66:0/3717725501 shutdown_connections 2024-03-30T00:40:17.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.437+0000 7f98bcc00640 1 -- 172.21.15.66:0/3717725501 >> 172.21.15.66:0/3717725501 conn(0x7f98c80ce2f0 msgr2=0x7f98c80ab010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.437+0000 7f98bcc00640 1 -- 172.21.15.66:0/3717725501 shutdown_connections 2024-03-30T00:40:17.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.437+0000 7f98bcc00640 1 -- 172.21.15.66:0/3717725501 wait complete. 2024-03-30T00:40:17.488 INFO:teuthology.orchestra.run.smithi066.stdout:55834574858 2024-03-30T00:40:17.488 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-30T00:40:17.489 INFO:teuthology.orchestra.run.smithi066.stdout:55834574858 2024-03-30T00:40:17.489 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-30T00:40:17.490 INFO:teuthology.orchestra.run.smithi066.stdout:55834574857 2024-03-30T00:40:17.490 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-30T00:40:17.491 INFO:teuthology.orchestra.run.smithi066.stdout:55834574858 2024-03-30T00:40:17.491 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-30T00:40:17.492 INFO:teuthology.orchestra.run.smithi066.stdout:55834574857 2024-03-30T00:40:17.492 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-30T00:40:17.493 INFO:teuthology.orchestra.run.smithi066.stdout:55834574857 2024-03-30T00:40:17.493 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-30T00:40:17.494 INFO:teuthology.orchestra.run.smithi066.stdout:55834574857 2024-03-30T00:40:17.495 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-30T00:40:17.495 INFO:teuthology.orchestra.run.smithi066.stdout:55834574859 2024-03-30T00:40:17.496 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-30T00:40:17.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.761+0000 7f0203c00640 1 Processor -- start 2024-03-30T00:40:17.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.761+0000 7f0203c00640 1 -- start start 2024-03-30T00:40:17.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0203c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01fc0ca620 0x7f01fc097870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0203c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f01fc0719d0 0x7f01fc16be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0203c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01fc16b9a0 con 0x7f01fc16a460 2024-03-30T00:40:17.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0203c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01fc16b070 con 0x7f01fc0719d0 2024-03-30T00:40:17.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0203c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc16b350 con 0x7f01fc0ca620 2024-03-30T00:40:17.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0201c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f01fc0719d0 0x7f01fc16be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0201c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f01fc0719d0 0x7f01fc16be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56820/0 (socket says 172.21.15.66:56820) 2024-03-30T00:40:17.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0201c00640 1 -- 172.21.15.66:0/3328377471 learned_addr learned my addr 172.21.15.66:0/3328377471 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0201200640 1 --2- 172.21.15.66:0/3328377471 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01fc0ca620 0x7f01fc097870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0201c00640 1 -- 172.21.15.66:0/3328377471 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01fc0ca620 msgr2=0x7f01fc097870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0201c00640 1 --2- 172.21.15.66:0/3328377471 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01fc0ca620 0x7f01fc097870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0201c00640 1 -- 172.21.15.66:0/3328377471 >> v1:172.21.15.66:6789/0 conn(0x7f01fc16a460 legacy=0x7f01fc0712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.765+0000 7f0201c00640 1 -- 172.21.15.66:0/3328377471 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc16b630 con 0x7f01fc0719d0 2024-03-30T00:40:17.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.769+0000 7f0201c00640 1 --2- 172.21.15.66:0/3328377471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01fc0719d0 0x7f01fc16be30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009e50 tx=0x7f01f401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f0dceee0b0e598 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.769+0000 7f01fb400640 1 -- 172.21.15.66:0/3328377471 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40af400 con 0x7f01fc0719d0 2024-03-30T00:40:17.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.769+0000 7f01fb400640 1 -- 172.21.15.66:0/3328377471 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f40ab050 con 0x7f01fc0719d0 2024-03-30T00:40:17.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.769+0000 7f01fb400640 1 -- 172.21.15.66:0/3328377471 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40af780 con 0x7f01fc0719d0 2024-03-30T00:40:17.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f41f9000640 1 Processor -- start 2024-03-30T00:40:17.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f0203c00640 1 -- 172.21.15.66:0/3328377471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01fc0719d0 msgr2=0x7f01fc16be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f0203c00640 1 --2- 172.21.15.66:0/3328377471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01fc0719d0 0x7f01fc16be30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009e50 tx=0x7f01f401f210 comp rx=0 tx=0).stop 2024-03-30T00:40:17.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f41f9000640 1 -- start start 2024-03-30T00:40:17.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7fdee3200640 1 Processor -- start 2024-03-30T00:40:17.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7fdee3200640 1 -- start start 2024-03-30T00:40:17.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f0203c00640 1 -- 172.21.15.66:0/3328377471 shutdown_connections 2024-03-30T00:40:17.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f0203c00640 1 --2- 172.21.15.66:0/3328377471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01fc0719d0 0x7f01fc16be30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f0203c00640 1 --2- 172.21.15.66:0/3328377471 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01fc0ca620 0x7f01fc097870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f0203c00640 1 -- 172.21.15.66:0/3328377471 >> 172.21.15.66:0/3328377471 conn(0x7f01fc070410 msgr2=0x7f01fc0afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f41f9000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f41f415c890 0x7f41f40cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f41f9000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41f40ce190 0x7f41f40d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f41f9000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f40cf0d0 con 0x7f41f40d0e90 2024-03-30T00:40:17.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f41f9000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f40d1320 con 0x7f41f415c890 2024-03-30T00:40:17.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f41f9000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40cf390 con 0x7f41f40ce190 2024-03-30T00:40:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7fdee3200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdedc050180 0x7fdedc0df280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7fdee3200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdedc033cd0 0x7fdedc0ea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f0203c00640 1 -- 172.21.15.66:0/3328377471 shutdown_connections 2024-03-30T00:40:17.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7fdee3200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdedc0540f0 con 0x7fdedc00aab0 2024-03-30T00:40:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f41f3400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f41f415c890 0x7f41f40cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7fdee3200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdedc038060 con 0x7fdedc033cd0 2024-03-30T00:40:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7fdee3200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdedc0543d0 con 0x7fdedc050180 2024-03-30T00:40:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7fdee1400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdedc050180 0x7fdedc0df280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7fdee1400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdedc050180 0x7fdedc0df280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:33036/0 (socket says 172.21.15.66:33036) 2024-03-30T00:40:17.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7fdee1400640 1 -- 172.21.15.66:0/4030995419 learned_addr learned my addr 172.21.15.66:0/4030995419 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f41f3e00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f41f40d0e90 0x7f41f40b4020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:55008/0 (socket says 172.21.15.66:55008) 2024-03-30T00:40:17.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f41f3e00640 1 -- 172.21.15.66:0/3850442171 learned_addr learned my addr 172.21.15.66:0/3850442171 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:17.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f41f2a00640 1 --2- 172.21.15.66:0/3850442171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41f40ce190 0x7f41f40d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7fdee2800640 1 --2- 172.21.15.66:0/4030995419 >> v2:172.21.15.66:3300/0 conn(0x7fdedc033cd0 0x7fdedc0ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.773+0000 7f0203c00640 1 -- 172.21.15.66:0/3328377471 wait complete. 2024-03-30T00:40:17.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7f0203c00640 1 Processor -- start 2024-03-30T00:40:17.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7fdee1400640 1 -- 172.21.15.66:0/4030995419 >> v2:172.21.15.66:3300/0 conn(0x7fdedc033cd0 msgr2=0x7fdedc0ea010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7fdee1400640 1 --2- 172.21.15.66:0/4030995419 >> v2:172.21.15.66:3300/0 conn(0x7fdedc033cd0 0x7fdedc0ea010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7fdee1400640 1 -- 172.21.15.66:0/4030995419 >> v1:172.21.15.66:6789/0 conn(0x7fdedc00aab0 legacy=0x7fdedc04fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7fdee1400640 1 -- 172.21.15.66:0/4030995419 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc0546b0 con 0x7fdedc050180 2024-03-30T00:40:17.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7f0203c00640 1 -- start start 2024-03-30T00:40:17.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7f41f2000640 1 -- 172.21.15.66:0/3850442171 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734403108 0 0) 0x7f41f40cf0d0 con 0x7f41f40d0e90 2024-03-30T00:40:17.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7f41f2000640 1 -- 172.21.15.66:0/3850442171 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41d8003530 con 0x7f41f40d0e90 2024-03-30T00:40:17.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7f41f3400640 1 -- 172.21.15.66:0/3850442171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41f40ce190 msgr2=0x7f41f40d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7f41f3400640 1 --2- 172.21.15.66:0/3850442171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41f40ce190 0x7f41f40d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7f41f3400640 1 -- 172.21.15.66:0/3850442171 >> v1:172.21.15.156:6789/0 conn(0x7f41f40d0e90 legacy=0x7f41f40b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7f41f3400640 1 -- 172.21.15.66:0/3850442171 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f40cf670 con 0x7f41f415c890 2024-03-30T00:40:17.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7f1e41200640 1 Processor -- start 2024-03-30T00:40:17.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0203c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01fc0719d0 0x7f01fc1ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0203c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01fc0ca620 0x7f01fc0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0203c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01fc16a460 0x7f01fc09b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.777+0000 7f1e41200640 1 -- start start 2024-03-30T00:40:17.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0201200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01fc0719d0 0x7f01fc1ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f01fbe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01fc0ca620 0x7f01fc0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0201200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01fc0719d0 0x7f01fc1ff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56838/0 (socket says 172.21.15.66:56838) 2024-03-30T00:40:17.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f01fbe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01fc0ca620 0x7f01fc0c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:50568/0 (socket says 172.21.15.66:50568) 2024-03-30T00:40:17.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0203c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0ac1c0 con 0x7f01fc0719d0 2024-03-30T00:40:17.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0203c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc1ffbd0 con 0x7f01fc16a460 2024-03-30T00:40:17.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0203c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0ab640 con 0x7f01fc0ca620 2024-03-30T00:40:17.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0201200640 1 -- 172.21.15.66:0/2779662022 learned_addr learned my addr 172.21.15.66:0/2779662022 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0201c00640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01fc16a460 0x7f01fc09b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0201200640 1 -- 172.21.15.66:0/2779662022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01fc0ca620 msgr2=0x7f01fc0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0201200640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01fc0ca620 0x7f01fc0c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0201200640 1 -- 172.21.15.66:0/2779662022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01fc16a460 msgr2=0x7f01fc09b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0201200640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01fc16a460 0x7f01fc09b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0201200640 1 -- 172.21.15.66:0/2779662022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4009b30 con 0x7f01fc0719d0 2024-03-30T00:40:17.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0201200640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01fc0719d0 0x7f01fc1ff690 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01f000c7f0 tx=0x7f01f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f01f8c00640 1 -- 172.21.15.66:0/2779662022 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0007820 con 0x7f01fc0719d0 2024-03-30T00:40:17.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f01f8c00640 1 -- 172.21.15.66:0/2779662022 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f0007980 con 0x7f01fc0719d0 2024-03-30T00:40:17.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f01f8c00640 1 -- 172.21.15.66:0/2779662022 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0007d30 con 0x7f01fc0719d0 2024-03-30T00:40:17.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0203c00640 1 -- 172.21.15.66:0/2779662022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01fc0ab920 con 0x7f01fc0719d0 2024-03-30T00:40:17.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e39e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1e3c0cf090 0x7f1e3c0cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36414/0 (socket says 172.21.15.66:36414) 2024-03-30T00:40:17.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e39e00640 1 -- 172.21.15.66:0/280537020 learned_addr learned my addr 172.21.15.66:0/280537020 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:17.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e41200640 1 --2- 172.21.15.66:0/280537020 >> v2:172.21.15.66:3300/0 conn(0x7f1e3c15d670 0x7f1e3c15db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e41200640 1 --2- 172.21.15.66:0/280537020 >> v2:172.21.15.156:3300/0 conn(0x7f1e3c0d0270 0x7f1e3c0d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e41200640 1 -- 172.21.15.66:0/280537020 --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3c0ce450 con 0x7f1e3c0cf090 2024-03-30T00:40:17.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e39400640 1 --2- 172.21.15.66:0/280537020 >> v2:172.21.15.66:3300/0 conn(0x7f1e3c15d670 0x7f1e3c15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e41200640 1 -- 172.21.15.66:0/280537020 --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c0d33b0 con 0x7f1e3c15d670 2024-03-30T00:40:17.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e41200640 1 -- 172.21.15.66:0/280537020 --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c0cd840 con 0x7f1e3c0d0270 2024-03-30T00:40:17.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e3a800640 1 --2- 172.21.15.66:0/280537020 >> v2:172.21.15.156:3300/0 conn(0x7f1e3c0d0270 0x7f1e3c0d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7fd1b4800640 1 Processor -- start 2024-03-30T00:40:17.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f41f3400640 1 --2- 172.21.15.66:0/3850442171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f415c890 0x7f41f40cdc50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009d90 tx=0x7f41ec01c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55359f0811e26e50 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7fd1b4800640 1 -- start start 2024-03-30T00:40:17.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e3a800640 1 -- 172.21.15.66:0/280537020 >> v2:172.21.15.66:3300/0 conn(0x7f1e3c15d670 msgr2=0x7f1e3c15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e3a800640 1 --2- 172.21.15.66:0/280537020 >> v2:172.21.15.66:3300/0 conn(0x7f1e3c15d670 0x7f1e3c15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f0203c00640 1 -- 172.21.15.66:0/2779662022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01fc0abee0 con 0x7f01fc0719d0 2024-03-30T00:40:17.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e3a800640 1 -- 172.21.15.66:0/280537020 >> v1:172.21.15.66:6789/0 conn(0x7f1e3c0cf090 legacy=0x7f1e3c0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e38a00640 1 -- 172.21.15.66:0/280537020 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577860257 0 0) 0x7f1e3c0ce450 con 0x7f1e3c0cf090 2024-03-30T00:40:17.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7fdee1400640 1 --2- 172.21.15.66:0/4030995419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdedc050180 0x7fdedc0df280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdedc055a90 tx=0x7fded001c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5917cfff70534057 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e3a800640 1 -- 172.21.15.66:0/280537020 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c0cdb20 con 0x7f1e3c0d0270 2024-03-30T00:40:17.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f1e39400640 1 --2- 172.21.15.66:0/280537020 >> v2:172.21.15.66:3300/0 conn(0x7f1e3c15d670 0x7f1e3c15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:17.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.781+0000 7f3b38400640 1 Processor -- start 2024-03-30T00:40:17.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f3b38400640 1 -- start start 2024-03-30T00:40:17.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f41f2000640 1 -- 172.21.15.66:0/3850442171 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0a5980 con 0x7f41f415c890 2024-03-30T00:40:17.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f0203c00640 1 -- 172.21.15.66:0/2779662022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01fc067150 con 0x7f01fc0719d0 2024-03-30T00:40:17.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f41f2000640 1 -- 172.21.15.66:0/3850442171 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec0a5ae0 con 0x7f41f415c890 2024-03-30T00:40:17.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f41f2000640 1 -- 172.21.15.66:0/3850442171 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0a5e70 con 0x7f41f415c890 2024-03-30T00:40:17.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdb9f200640 1 Processor -- start 2024-03-30T00:40:17.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdee0a00640 1 -- 172.21.15.66:0/4030995419 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00ab600 con 0x7fdedc050180 2024-03-30T00:40:17.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdee0a00640 1 -- 172.21.15.66:0/4030995419 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded00a7050 con 0x7fdedc050180 2024-03-30T00:40:17.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdee0a00640 1 -- 172.21.15.66:0/4030995419 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00ab600 con 0x7fdedc050180 2024-03-30T00:40:17.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f41f9000640 1 -- 172.21.15.66:0/3850442171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f415c890 msgr2=0x7f41f40cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f41f9000640 1 --2- 172.21.15.66:0/3850442171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f415c890 0x7f41f40cdc50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009d90 tx=0x7f41ec01c900 comp rx=0 tx=0).stop 2024-03-30T00:40:17.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b1e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd1ac160e20 0x7fd1ac0af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36426/0 (socket says 172.21.15.66:36426) 2024-03-30T00:40:17.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f41f9000640 1 -- 172.21.15.66:0/3850442171 shutdown_connections 2024-03-30T00:40:17.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f41f9000640 1 --2- 172.21.15.66:0/3850442171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41f40ce190 0x7f41f40d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f41f9000640 1 --2- 172.21.15.66:0/3850442171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f415c890 0x7f41f40cdc50 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009d90 tx=0x7f41ec01c900 comp rx=0 tx=0).stop 2024-03-30T00:40:17.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f41f9000640 1 -- 172.21.15.66:0/3850442171 >> 172.21.15.66:0/3850442171 conn(0x7f41f409b8d0 msgr2=0x7f41f4089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b1e00640 1 -- 172.21.15.66:0/2443142798 learned_addr learned my addr 172.21.15.66:0/2443142798 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:17.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b4800640 1 --2- 172.21.15.66:0/2443142798 >> v2:172.21.15.156:3300/0 conn(0x7fd1ac161200 0x7fd1ac0c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f3b38400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3b30160c10 0x7f3b300c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f41f9000640 1 -- 172.21.15.66:0/3850442171 shutdown_connections 2024-03-30T00:40:17.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f41f9000640 1 -- 172.21.15.66:0/3850442171 wait complete. 2024-03-30T00:40:17.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b4800640 1 --2- 172.21.15.66:0/2443142798 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1ac0c29e0 0x7fd1ac0cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b4800640 1 -- 172.21.15.66:0/2443142798 --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac165450 con 0x7fd1ac160e20 2024-03-30T00:40:17.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b1400640 1 --2- 172.21.15.66:0/2443142798 >> v2:172.21.15.156:3300/0 conn(0x7fd1ac161200 0x7fd1ac0c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b4800640 1 -- 172.21.15.66:0/2443142798 --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac0c2e90 con 0x7fd1ac161200 2024-03-30T00:40:17.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b4800640 1 -- 172.21.15.66:0/2443142798 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac164b40 con 0x7fd1ac0c29e0 2024-03-30T00:40:17.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdb9f200640 1 -- start start 2024-03-30T00:40:17.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdb9f200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 0x7fdb981604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdb9f200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdb980cd260 0x7fdb981609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f3b38400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3b300c19d0 0x7f3b300cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f01f8c00640 1 -- 172.21.15.66:0/2779662022 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f01f00113f0 con 0x7f01fc0719d0 2024-03-30T00:40:17.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b2800640 1 --2- 172.21.15.66:0/2443142798 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1ac0c29e0 0x7fd1ac0cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b0a00640 1 -- 172.21.15.66:0/2443142798 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393795918 0 0) 0x7fd1ac165450 con 0x7fd1ac160e20 2024-03-30T00:40:17.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b0a00640 1 -- 172.21.15.66:0/2443142798 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd194003530 con 0x7fd1ac160e20 2024-03-30T00:40:17.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f2c57a00640 1 Processor -- start 2024-03-30T00:40:17.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b1400640 1 -- 172.21.15.66:0/2443142798 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1ac0c29e0 msgr2=0x7fd1ac0cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b1400640 1 --2- 172.21.15.66:0/2443142798 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1ac0c29e0 0x7fd1ac0cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b1400640 1 -- 172.21.15.66:0/2443142798 >> v1:172.21.15.66:6789/0 conn(0x7fd1ac160e20 legacy=0x7fd1ac0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fd1b1400640 1 -- 172.21.15.66:0/2443142798 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac0b0530 con 0x7fd1ac161200 2024-03-30T00:40:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f3b38400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b300c9a30 con 0x7f3b300b19a0 2024-03-30T00:40:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f3b38400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b300c9d10 con 0x7f3b30160c10 2024-03-30T00:40:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f3b35a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3b30160c10 0x7f3b300c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f3b35a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3b30160c10 0x7f3b300c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56842/0 (socket says 172.21.15.66:56842) 2024-03-30T00:40:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f3b35a00640 1 -- 172.21.15.66:0/2289744484 learned_addr learned my addr 172.21.15.66:0/2289744484 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdb9f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb980b1d00 con 0x7fdb9815c780 2024-03-30T00:40:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdb9f200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb980cd820 con 0x7fdb980cd260 2024-03-30T00:40:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdb9f200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980cdb00 con 0x7fdb9815cb60 2024-03-30T00:40:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdb97400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 0x7fdb981604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdb97400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 0x7fdb981604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:50582/0 (socket says 172.21.15.66:50582) 2024-03-30T00:40:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdb97400640 1 -- 172.21.15.66:0/1000816872 learned_addr learned my addr 172.21.15.66:0/1000816872 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7fdb97e00640 1 --1- 172.21.15.66:0/1000816872 >> v1:172.21.15.66:6789/0 conn(0x7fdb9815c780 0x7fdb980cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36434/0 (socket says 172.21.15.66:36434) 2024-03-30T00:40:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f2c57a00640 1 -- start start 2024-03-30T00:40:17.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdee3200640 1 -- 172.21.15.66:0/4030995419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdedc050180 msgr2=0x7fdedc0df280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdee3200640 1 --2- 172.21.15.66:0/4030995419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdedc050180 0x7fdedc0df280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdedc055a90 tx=0x7fded001c390 comp rx=0 tx=0).stop 2024-03-30T00:40:17.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdee3200640 1 -- 172.21.15.66:0/4030995419 shutdown_connections 2024-03-30T00:40:17.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdee3200640 1 --2- 172.21.15.66:0/4030995419 >> v2:172.21.15.66:3300/0 conn(0x7fdedc033cd0 0x7fdedc0ea010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdee3200640 1 --2- 172.21.15.66:0/4030995419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdedc050180 0x7fdedc0df280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7fdee3200640 1 -- 172.21.15.66:0/4030995419 >> 172.21.15.66:0/4030995419 conn(0x7fdedc009f90 msgr2=0x7fdedc00a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e3a800640 1 --2- 172.21.15.66:0/280537020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0270 0x7f1e3c0d06c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c08a810 tx=0x7f1e3401f110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7b2229341334959 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f01f8c00640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01dc066760 0x7f01dc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f41f9000640 1 Processor -- start 2024-03-30T00:40:17.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e38a00640 1 -- 172.21.15.66:0/280537020 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340af600 con 0x7f1e3c0d0270 2024-03-30T00:40:17.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e38a00640 1 -- 172.21.15.66:0/280537020 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e340ab050 con 0x7f1e3c0d0270 2024-03-30T00:40:17.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e38a00640 1 -- 172.21.15.66:0/280537020 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340af980 con 0x7f1e3c0d0270 2024-03-30T00:40:17.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f41f9000640 1 -- start start 2024-03-30T00:40:17.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7fdb97400640 1 -- 172.21.15.66:0/1000816872 >> v2:172.21.15.66:3300/0 conn(0x7fdb980cd260 msgr2=0x7fdb981609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f01f8c00640 1 -- 172.21.15.66:0/2779662022 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f01f00844b0 con 0x7f01fc0719d0 2024-03-30T00:40:17.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7fdee3200640 1 -- 172.21.15.66:0/4030995419 shutdown_connections 2024-03-30T00:40:17.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7fdee3200640 1 -- 172.21.15.66:0/4030995419 wait complete. 2024-03-30T00:40:17.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7fdb97400640 1 --2- 172.21.15.66:0/1000816872 >> v2:172.21.15.66:3300/0 conn(0x7fdb980cd260 0x7fdb981609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7fdb97400640 1 -- 172.21.15.66:0/1000816872 >> v1:172.21.15.66:6789/0 conn(0x7fdb9815c780 legacy=0x7fdb980cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7fdb97400640 1 -- 172.21.15.66:0/1000816872 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb980cdde0 con 0x7fdb9815cb60 2024-03-30T00:40:17.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e41200640 1 -- 172.21.15.66:0/280537020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0270 msgr2=0x7f1e3c0d06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e41200640 1 --2- 172.21.15.66:0/280537020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0270 0x7f1e3c0d06c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c08a810 tx=0x7f1e3401f110 comp rx=0 tx=0).stop 2024-03-30T00:40:17.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e41200640 1 -- 172.21.15.66:0/280537020 shutdown_connections 2024-03-30T00:40:17.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e41200640 1 --2- 172.21.15.66:0/280537020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0270 0x7f1e3c0d06c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e41200640 1 --2- 172.21.15.66:0/280537020 >> v2:172.21.15.66:3300/0 conn(0x7f1e3c15d670 0x7f1e3c15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e41200640 1 -- 172.21.15.66:0/280537020 >> 172.21.15.66:0/280537020 conn(0x7f1e3c089fb0 msgr2=0x7f1e3c0d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e41200640 1 -- 172.21.15.66:0/280537020 shutdown_connections 2024-03-30T00:40:17.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f1e41200640 1 -- 172.21.15.66:0/280537020 wait complete. 2024-03-30T00:40:17.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f41f9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f40ce190 0x7f41f41575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f41f9000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41f40d0e90 0x7f41f40b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f41f9000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f40b63e0 0x7f41f40b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f2c57a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c500cb5b0 0x7f2c500c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f41f9000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40b5130 con 0x7f41f40ce190 2024-03-30T00:40:17.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f41f9000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40b7f60 con 0x7f41f40b63e0 2024-03-30T00:40:17.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f41f9000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40b5340 con 0x7f41f40d0e90 2024-03-30T00:40:17.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f2c55000640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2c5006df10 0x7f2c500cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:55012/0 (socket says 172.21.15.66:55012) 2024-03-30T00:40:17.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f2c57a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2c500c8e80 0x7f2c500ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f2c55000640 1 -- 172.21.15.66:0/1737145905 learned_addr learned my addr 172.21.15.66:0/1737145905 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:17.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f41f3e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f40b63e0 0x7f41f40b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f41f3e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f40b63e0 0x7f41f40b7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:33078/0 (socket says 172.21.15.66:33078) 2024-03-30T00:40:17.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f41f3e00640 1 -- 172.21.15.66:0/310385177 learned_addr learned my addr 172.21.15.66:0/310385177 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.785+0000 7f3b38400640 1 -- 172.21.15.66:0/2289744484 --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b300c9ff0 con 0x7f3b300c19d0 2024-03-30T00:40:17.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7fdee3200640 1 Processor -- start 2024-03-30T00:40:17.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.789+0000 7f2c57a00640 1 -- 172.21.15.66:0/1737145905 --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5006e2f0 con 0x7f2c5006df10 2024-03-30T00:40:17.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c57a00640 1 -- 172.21.15.66:0/1737145905 --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c500ca180 con 0x7f2c500c8e80 2024-03-30T00:40:17.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c57a00640 1 -- 172.21.15.66:0/1737145905 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500cba30 con 0x7f2c500cb5b0 2024-03-30T00:40:17.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee3200640 1 -- start start 2024-03-30T00:40:17.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f01fbe00640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01dc066760 0x7f01dc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f01fbe00640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01dc066760 0x7f01dc068c20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0ac830 tx=0x7f01e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41f3400640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f40ce190 0x7f41f41575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41f3e00640 1 -- 172.21.15.66:0/310385177 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41f40d0e90 msgr2=0x7f41f40b74e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T00:40:17.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41f3e00640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41f40d0e90 0x7f41f40b74e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41f3e00640 1 -- 172.21.15.66:0/310385177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f40ce190 msgr2=0x7f41f41575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41f3e00640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f40ce190 0x7f41f41575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41f3e00640 1 -- 172.21.15.66:0/310385177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec009a70 con 0x7f41f40b63e0 2024-03-30T00:40:17.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee3200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdedc00aab0 0x7fdedc0d8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee3200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdedc033cd0 0x7fdedc038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdedc050180 0x7fdedc036aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee3200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0d94e0 con 0x7fdedc050180 2024-03-30T00:40:17.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee3200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0e1820 con 0x7fdedc033cd0 2024-03-30T00:40:17.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee3200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0e1b00 con 0x7fdedc00aab0 2024-03-30T00:40:17.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f1e41200640 1 Processor -- start 2024-03-30T00:40:17.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f1e41200640 1 -- start start 2024-03-30T00:40:17.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b1400640 1 --2- 172.21.15.66:0/2443142798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1ac161200 0x7fd1ac0c24a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009d90 tx=0x7fd1a801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ecf9d3fa57fb03 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b0a00640 1 -- 172.21.15.66:0/2443142798 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80a5a00 con 0x7fd1ac161200 2024-03-30T00:40:17.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b0a00640 1 -- 172.21.15.66:0/2443142798 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a80a5b60 con 0x7fd1ac161200 2024-03-30T00:40:17.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b0a00640 1 -- 172.21.15.66:0/2443142798 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80ad440 con 0x7fd1ac161200 2024-03-30T00:40:17.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b4800640 1 -- 172.21.15.66:0/2443142798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1ac161200 msgr2=0x7fd1ac0c24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b4800640 1 --2- 172.21.15.66:0/2443142798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1ac161200 0x7fd1ac0c24a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009d90 tx=0x7fd1a801c940 comp rx=0 tx=0).stop 2024-03-30T00:40:17.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b4800640 1 -- 172.21.15.66:0/2443142798 shutdown_connections 2024-03-30T00:40:17.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b4800640 1 --2- 172.21.15.66:0/2443142798 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1ac0c29e0 0x7fd1ac0cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b4800640 1 --2- 172.21.15.66:0/2443142798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1ac161200 0x7fd1ac0c24a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b4800640 1 -- 172.21.15.66:0/2443142798 >> 172.21.15.66:0/2443142798 conn(0x7fd1ac098a20 msgr2=0x7fd1ac0cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdb97400640 1 --2- 172.21.15.66:0/1000816872 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 0x7fdb981604a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009e50 tx=0x7fdb9001c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22433855cd42afa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420412493 0 0) 0x7f2c5006e2f0 con 0x7f2c5006df10 2024-03-30T00:40:17.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c38003530 con 0x7f2c5006df10 2024-03-30T00:40:17.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4fe00640 1 --2- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c500cb5b0 0x7f2c500c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c55a00640 1 --2- 172.21.15.66:0/1737145905 >> v2:172.21.15.66:3300/0 conn(0x7f2c500c8e80 0x7f2c500ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2915228176 0 0) 0x7f2c38003530 con 0x7f2c5006df10 2024-03-30T00:40:17.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c500cb5b0 msgr2=0x7f2c500c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 --2- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c500cb5b0 0x7f2c500c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 >> v2:172.21.15.66:3300/0 conn(0x7f2c500c8e80 msgr2=0x7f2c500ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 --2- 172.21.15.66:0/1737145905 >> v2:172.21.15.66:3300/0 conn(0x7f2c500c8e80 0x7f2c500ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500c97e0 con 0x7f2c5006df10 2024-03-30T00:40:17.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f2c3c001680 con 0x7f2c5006df10 2024-03-30T00:40:17.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 >> v1:172.21.15.156:6789/0 conn(0x7f2c5006df10 legacy=0x7f2c500cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 --2- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c38003980 0x7f2c38003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 --2- 172.21.15.66:0/1737145905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c380047d0 0x7f2c38004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 --2- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c38005190 0x7f2c3800fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500ca180 con 0x7f2c380047d0 2024-03-30T00:40:17.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500cba30 con 0x7f2c38005190 2024-03-30T00:40:17.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c38010420 con 0x7f2c38003980 2024-03-30T00:40:17.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c55000640 1 --2- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c38003980 0x7f2c38003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f3b36400640 1 --2- 172.21.15.66:0/2289744484 >> v2:172.21.15.156:3300/0 conn(0x7f3b300c19d0 0x7f3b300cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c55a00640 1 --2- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c38005190 0x7f2c3800fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee1e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdedc00aab0 0x7fdedc0d8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee1400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdedc033cd0 0x7fdedc038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee1400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdedc033cd0 0x7fdedc038bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:33100/0 (socket says 172.21.15.66:33100) 2024-03-30T00:40:17.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee1400640 1 -- 172.21.15.66:0/1162432757 learned_addr learned my addr 172.21.15.66:0/1162432757 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c55000640 1 -- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c38005190 msgr2=0x7f2c3800fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c55000640 1 --2- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c38005190 0x7f2c3800fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c55000640 1 -- 172.21.15.66:0/1737145905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c380047d0 msgr2=0x7f2c38004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c55000640 1 --2- 172.21.15.66:0/1737145905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c380047d0 0x7f2c38004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f2c55000640 1 -- 172.21.15.66:0/1737145905 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c38003530 con 0x7f2c38003980 2024-03-30T00:40:17.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41f3e00640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f40b63e0 0x7f41f40b7a20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41e000c390 tx=0x7f41e000c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdb96a00640 1 -- 172.21.15.66:0/1000816872 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb900a5840 con 0x7fdb9815cb60 2024-03-30T00:40:17.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdb96a00640 1 -- 172.21.15.66:0/1000816872 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb900a59a0 con 0x7fdb9815cb60 2024-03-30T00:40:17.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdb96a00640 1 -- 172.21.15.66:0/1000816872 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb900a5d30 con 0x7fdb9815cb60 2024-03-30T00:40:17.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fdee2800640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdedc050180 0x7fdedc036aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b4800640 1 -- 172.21.15.66:0/2443142798 shutdown_connections 2024-03-30T00:40:17.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41eb400640 1 -- 172.21.15.66:0/310385177 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e00074e0 con 0x7f41f40b63e0 2024-03-30T00:40:17.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41f9000640 1 -- 172.21.15.66:0/310385177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f40b5620 con 0x7f41f40b63e0 2024-03-30T00:40:17.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41eb400640 1 -- 172.21.15.66:0/310385177 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e0007640 con 0x7f41f40b63e0 2024-03-30T00:40:17.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41eb400640 1 -- 172.21.15.66:0/310385177 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0004a10 con 0x7f41f40b63e0 2024-03-30T00:40:17.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f41f9000640 1 -- 172.21.15.66:0/310385177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f40b5be0 con 0x7f41f40b63e0 2024-03-30T00:40:17.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7fd1b4800640 1 -- 172.21.15.66:0/2443142798 wait complete. 2024-03-30T00:40:17.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f3b36400640 1 -- 172.21.15.66:0/2289744484 >> v2:172.21.15.66:3300/0 conn(0x7f3b30160c10 msgr2=0x7f3b300c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:17.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f3b36400640 1 --2- 172.21.15.66:0/2289744484 >> v2:172.21.15.66:3300/0 conn(0x7f3b30160c10 0x7f3b300c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdee1e00640 1 -- 172.21.15.66:0/1162432757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdedc033cd0 msgr2=0x7fdedc038bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdb9f200640 1 -- 172.21.15.66:0/1000816872 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 msgr2=0x7fdb981604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdb9f200640 1 --2- 172.21.15.66:0/1000816872 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 0x7fdb981604a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009e50 tx=0x7fdb9001c9c0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdee1e00640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdedc033cd0 0x7fdedc038bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdee1e00640 1 -- 172.21.15.66:0/1162432757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdedc050180 msgr2=0x7fdedc036aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f1e41200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c0cf090 0x7f1e3c0b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f1e41200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e3c0d0270 0x7f1e3c147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f1e41200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c15d670 0x7f1e3c147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f1e41200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0b5cd0 con 0x7f1e3c0cf090 2024-03-30T00:40:17.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f1e41200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0cce10 con 0x7f1e3c15d670 2024-03-30T00:40:17.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.793+0000 7f1e41200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0b6ac0 con 0x7f1e3c0d0270 2024-03-30T00:40:17.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e3a800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c15d670 0x7f1e3c147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdb9f200640 1 -- 172.21.15.66:0/1000816872 shutdown_connections 2024-03-30T00:40:17.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdb9f200640 1 --2- 172.21.15.66:0/1000816872 >> v2:172.21.15.66:3300/0 conn(0x7fdb980cd260 0x7fdb981609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdb9f200640 1 --2- 172.21.15.66:0/1000816872 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 0x7fdb981604a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdb9f200640 1 -- 172.21.15.66:0/1000816872 >> 172.21.15.66:0/1000816872 conn(0x7fdb9809ba30 msgr2=0x7fdb98089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e3a800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c15d670 0x7f1e3c147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:33110/0 (socket says 172.21.15.66:33110) 2024-03-30T00:40:17.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e3a800640 1 -- 172.21.15.66:0/1778708330 learned_addr learned my addr 172.21.15.66:0/1778708330 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e39400640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e3c0d0270 0x7f1e3c147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f3b36400640 1 -- 172.21.15.66:0/2289744484 >> v1:172.21.15.66:6789/0 conn(0x7f3b300b19a0 legacy=0x7f3b300c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdb9f200640 1 -- 172.21.15.66:0/1000816872 shutdown_connections 2024-03-30T00:40:17.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdee1e00640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdedc050180 0x7fdedc036aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f41f9000640 1 -- 172.21.15.66:0/310385177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f4073e00 con 0x7f41f40b63e0 2024-03-30T00:40:17.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f01f8c00640 1 -- 172.21.15.66:0/2779662022 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f01f00515a0 con 0x7f01fc0719d0 2024-03-30T00:40:17.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e3a800640 1 -- 172.21.15.66:0/1778708330 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e3c0d0270 msgr2=0x7f1e3c147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e3a800640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e3c0d0270 0x7f1e3c147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e3a800640 1 -- 172.21.15.66:0/1778708330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c0cf090 msgr2=0x7f1e3c0b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e3a800640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c0cf090 0x7f1e3c0b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e3a800640 1 -- 172.21.15.66:0/1778708330 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c009420 con 0x7f1e3c15d670 2024-03-30T00:40:17.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e39400640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e3c0d0270 0x7f1e3c147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:17.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e3a800640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c15d670 0x7f1e3c147dc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1e3401cd40 tx=0x7f1e3401beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f3b36400640 1 -- 172.21.15.66:0/2289744484 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b300ca2d0 con 0x7f3b300c19d0 2024-03-30T00:40:17.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f3b35a00640 1 --2- 172.21.15.66:0/2289744484 >> v2:172.21.15.66:3300/0 conn(0x7f3b30160c10 0x7f3b300c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:17.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdee1e00640 1 -- 172.21.15.66:0/1162432757 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00094d0 con 0x7fdedc00aab0 2024-03-30T00:40:17.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdee1400640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdedc033cd0 0x7fdedc038bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:17.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdee2800640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdedc050180 0x7fdedc036aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:17.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdb9f200640 1 -- 172.21.15.66:0/1000816872 wait complete. 2024-03-30T00:40:17.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e32000640 1 -- 172.21.15.66:0/1778708330 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340afb70 con 0x7f1e3c15d670 2024-03-30T00:40:17.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f41eb400640 1 -- 172.21.15.66:0/310385177 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f41e0004b70 con 0x7f41f40b63e0 2024-03-30T00:40:17.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e32000640 1 -- 172.21.15.66:0/1778708330 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e340afcd0 con 0x7f1e3c15d670 2024-03-30T00:40:17.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e32000640 1 -- 172.21.15.66:0/1778708330 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340ba640 con 0x7f1e3c15d670 2024-03-30T00:40:17.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fd1b4800640 1 Processor -- start 2024-03-30T00:40:17.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e41200640 1 -- 172.21.15.66:0/1778708330 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e34009a70 con 0x7f1e3c15d670 2024-03-30T00:40:17.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f1e41200640 1 -- 172.21.15.66:0/1778708330 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3c0b6190 con 0x7f1e3c15d670 2024-03-30T00:40:17.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdb9f200640 1 Processor -- start 2024-03-30T00:40:17.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fd1b4800640 1 -- start start 2024-03-30T00:40:17.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7f41eb400640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41d40666e0 0x7f41d4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.797+0000 7fdee1e00640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdedc00aab0 0x7fdedc0d8a00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fded400c3e0 tx=0x7fded400c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb9f200640 1 -- start start 2024-03-30T00:40:17.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7f41eb400640 1 -- 172.21.15.66:0/310385177 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f41e0084170 con 0x7f41f40b63e0 2024-03-30T00:40:17.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7f2c55000640 1 --2- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c38003980 0x7f2c38003da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00c190 tx=0x7f2c3c01cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e46a34115f5b7c58 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdeda000640 1 -- 172.21.15.66:0/1162432757 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded4011790 con 0x7fdedc00aab0 2024-03-30T00:40:17.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7f41f3400640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41d40666e0 0x7f41d4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdee3200640 1 -- 172.21.15.66:0/1162432757 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdedc0e1de0 con 0x7fdedc00aab0 2024-03-30T00:40:17.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdee3200640 1 -- 172.21.15.66:0/1162432757 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdedc0e23a0 con 0x7fdedc00aab0 2024-03-30T00:40:17.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdeda000640 1 -- 172.21.15.66:0/1162432757 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded40118f0 con 0x7fdedc00aab0 2024-03-30T00:40:17.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdeda000640 1 -- 172.21.15.66:0/1162432757 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded4011ce0 con 0x7fdedc00aab0 2024-03-30T00:40:17.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7f1e41200640 1 -- 172.21.15.66:0/1778708330 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3c067340 con 0x7f1e3c15d670 2024-03-30T00:40:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0058e0 con 0x7f2c38003980 2024-03-30T00:40:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c005a60 con 0x7f2c38003980 2024-03-30T00:40:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7f2c4f400640 1 -- 172.21.15.66:0/1737145905 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c005e50 con 0x7f2c38003980 2024-03-30T00:40:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb9f200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb980cd260 0x7fdb9815b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb9f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb9815c780 0x7fdb980b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7f41f3400640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41d40666e0 0x7f41d4068ba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0af000 tx=0x7f41ec0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fd1b4800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1ac0c29e0 0x7fd1ac0c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb9f200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 0x7fdb980b7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb9f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980c9000 con 0x7fdb9815c780 2024-03-30T00:40:17.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb9f200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980b6340 con 0x7fdb980cd260 2024-03-30T00:40:17.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb9f200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980b6620 con 0x7fdb9815cb60 2024-03-30T00:40:17.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fd1b4800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ac160e20 0x7fd1ac0b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7f2c57a00640 1 -- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c38003980 msgr2=0x7f2c38003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb97e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb980cd260 0x7fdb9815b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb97e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb980cd260 0x7fdb9815b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:33112/0 (socket says 172.21.15.66:33112) 2024-03-30T00:40:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb97e00640 1 -- 172.21.15.66:0/960592679 learned_addr learned my addr 172.21.15.66:0/960592679 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb9d200640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 0x7fdb980b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7f2c57a00640 1 --2- 172.21.15.66:0/1737145905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c38003980 0x7f2c38003da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00c190 tx=0x7f2c3c01cc40 comp rx=0 tx=0).stop 2024-03-30T00:40:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fd1b1e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1ac0c29e0 0x7fd1ac0c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fd1b1e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1ac0c29e0 0x7fd1ac0c4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:33122/0 (socket says 172.21.15.66:33122) 2024-03-30T00:40:17.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fd1b1e00640 1 -- 172.21.15.66:0/3259092158 learned_addr learned my addr 172.21.15.66:0/3259092158 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb97e00640 1 -- 172.21.15.66:0/960592679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 msgr2=0x7fdb980b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb97e00640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 0x7fdb980b7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb97e00640 1 -- 172.21.15.66:0/960592679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb9815c780 msgr2=0x7fdb980b7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb97e00640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb9815c780 0x7fdb980b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fd1b4800640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1ac161200 0x7fd1ac0b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb97e00640 1 -- 172.21.15.66:0/960592679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb90009b30 con 0x7fdb980cd260 2024-03-30T00:40:17.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0b2ee0 con 0x7fd1ac160e20 2024-03-30T00:40:17.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7f2c55a00640 1 -- 172.21.15.66:0/1737145905 reap_dead start 2024-03-30T00:40:17.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fd1b1400640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ac160e20 0x7fd1ac0b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdb97e00640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb980cd260 0x7fdb9815b9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c460 tx=0x7fdb8800c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fd1b2800640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1ac161200 0x7fd1ac0b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0c4e90 con 0x7fd1ac0c29e0 2024-03-30T00:40:17.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0b2260 con 0x7fd1ac161200 2024-03-30T00:40:17.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.801+0000 7fdee3200640 1 -- 172.21.15.66:0/1162432757 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdebc005260 con 0x7fdedc00aab0 2024-03-30T00:40:17.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f2c57a00640 1 -- 172.21.15.66:0/1737145905 shutdown_connections 2024-03-30T00:40:17.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f2c57a00640 1 -- 172.21.15.66:0/1737145905 >> 172.21.15.66:0/1737145905 conn(0x7f2c50069f60 msgr2=0x7f2c5006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f2c57a00640 1 -- 172.21.15.66:0/1737145905 shutdown_connections 2024-03-30T00:40:17.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f3b36400640 1 --2- 172.21.15.66:0/2289744484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b300c19d0 0x7f3b300cc280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b300cf340 tx=0x7f3b2801c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51217272113cc4c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f2c57a00640 1 -- 172.21.15.66:0/1737145905 wait complete. 2024-03-30T00:40:17.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fd1b2800640 1 -- 172.21.15.66:0/3259092158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1ac0c29e0 msgr2=0x7fd1ac0c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fd1b2800640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1ac0c29e0 0x7fd1ac0c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fd1b2800640 1 -- 172.21.15.66:0/3259092158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ac160e20 msgr2=0x7fd1ac0b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fd1b2800640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ac160e20 0x7fd1ac0b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fd1b2800640 1 -- 172.21.15.66:0/3259092158 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a8009a70 con 0x7fd1ac161200 2024-03-30T00:40:17.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fd1b1400640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ac160e20 0x7fd1ac0b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:17.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fd1b2800640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1ac161200 0x7fd1ac0b41f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000d530 tx=0x7fd1a000d9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fdeda000640 1 -- 172.21.15.66:0/1162432757 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fded401c460 con 0x7fdedc00aab0 2024-03-30T00:40:17.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fdeda000640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdec40666e0 0x7fdec4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fdeda000640 1 -- 172.21.15.66:0/1162432757 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fded4084200 con 0x7fdedc00aab0 2024-03-30T00:40:17.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fdee1400640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdec40666e0 0x7fdec4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f2c57a00640 1 Processor -- start 2024-03-30T00:40:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f41eb400640 1 -- 172.21.15.66:0/310385177 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f41e00511d0 con 0x7f41f40b63e0 2024-03-30T00:40:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fdb8fe00640 1 -- 172.21.15.66:0/960592679 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88007480 con 0x7fdb980cd260 2024-03-30T00:40:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fdb8fe00640 1 -- 172.21.15.66:0/960592679 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb88007600 con 0x7fdb980cd260 2024-03-30T00:40:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fdb8fe00640 1 -- 172.21.15.66:0/960592679 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88005a60 con 0x7fdb980cd260 2024-03-30T00:40:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fdb9f200640 1 -- 172.21.15.66:0/960592679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb980b6900 con 0x7fdb980cd260 2024-03-30T00:40:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fdb9f200640 1 -- 172.21.15.66:0/960592679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9815cfa0 con 0x7fdb980cd260 2024-03-30T00:40:17.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f3b2fe00640 1 -- 172.21.15.66:0/2289744484 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b300c9830 con 0x7f3b300c19d0 2024-03-30T00:40:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f3b2fe00640 1 -- 172.21.15.66:0/2289744484 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b280a5a10 con 0x7f3b300c19d0 2024-03-30T00:40:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7fdee1400640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdec40666e0 0x7fdec4068ba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fded001c3d0 tx=0x7fded00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f3b2fe00640 1 -- 172.21.15.66:0/2289744484 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b280a5da0 con 0x7f3b300c19d0 2024-03-30T00:40:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f3b38400640 1 -- 172.21.15.66:0/2289744484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b300c19d0 msgr2=0x7f3b300cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f3b38400640 1 --2- 172.21.15.66:0/2289744484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b300c19d0 0x7f3b300cc280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b300cf340 tx=0x7f3b2801c930 comp rx=0 tx=0).stop 2024-03-30T00:40:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.805+0000 7f2c57a00640 1 -- start start 2024-03-30T00:40:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f3b38400640 1 -- 172.21.15.66:0/2289744484 shutdown_connections 2024-03-30T00:40:17.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7fdb9f200640 1 -- 172.21.15.66:0/960592679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb98066c30 con 0x7fdb980cd260 2024-03-30T00:40:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f3b38400640 1 --2- 172.21.15.66:0/2289744484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b300c19d0 0x7f3b300cc280 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f3b38400640 1 --2- 172.21.15.66:0/2289744484 >> v2:172.21.15.66:3300/0 conn(0x7f3b30160c10 0x7f3b300c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f3b38400640 1 -- 172.21.15.66:0/2289744484 >> 172.21.15.66:0/2289744484 conn(0x7f3b30069f60 msgr2=0x7f3b3006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c57a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c38003980 0x7f2c5015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f3b38400640 1 -- 172.21.15.66:0/2289744484 shutdown_connections 2024-03-30T00:40:17.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f3b38400640 1 -- 172.21.15.66:0/2289744484 wait complete. 2024-03-30T00:40:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c57a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c500c7170 0x7f2c500c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c57a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c501e2760 0x7f2c501e2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c57a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c501e3120 con 0x7f2c38003980 2024-03-30T00:40:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c57a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500c6820 con 0x7f2c500c7170 2024-03-30T00:40:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c57a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500c5c10 con 0x7f2c501e2760 2024-03-30T00:40:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c55a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c501e2760 0x7f2c501e2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c55000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c38003980 0x7f2c5015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c55a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c501e2760 0x7f2c501e2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:50664/0 (socket says 172.21.15.66:50664) 2024-03-30T00:40:17.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c55a00640 1 -- 172.21.15.66:0/3728239790 learned_addr learned my addr 172.21.15.66:0/3728239790 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f1e32000640 1 -- 172.21.15.66:0/1778708330 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1e340ba7a0 con 0x7f1e3c15d670 2024-03-30T00:40:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c4fe00640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c500c7170 0x7f2c500c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c55a00640 1 -- 172.21.15.66:0/3728239790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c500c7170 msgr2=0x7f2c500c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c55a00640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c500c7170 0x7f2c500c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c55a00640 1 -- 172.21.15.66:0/3728239790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c38003980 msgr2=0x7f2c5015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c55a00640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c38003980 0x7f2c5015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c55a00640 1 -- 172.21.15.66:0/3728239790 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500c5ef0 con 0x7f2c501e2760 2024-03-30T00:40:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c55000640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c38003980 0x7f2c5015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:17.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f2c55a00640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c501e2760 0x7f2c501e2be0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c5006acc0 tx=0x7f2c48009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f1e32000640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e1c0666e0 0x7f1e1c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f1e32000640 1 -- 172.21.15.66:0/1778708330 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1e34122ff0 con 0x7f1e3c15d670 2024-03-30T00:40:17.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f1e39e00640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e1c0666e0 0x7f1e1c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f1e39e00640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e1c0666e0 0x7f1e1c068ba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1e28007fd0 tx=0x7f1e280078c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.809+0000 7f3b38400640 1 Processor -- start 2024-03-30T00:40:17.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b38400640 1 -- start start 2024-03-30T00:40:17.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b38400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b300b19a0 0x7f3b300c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b38400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b300c19d0 0x7f3b30071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b38400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b30160c10 0x7f3b30071950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b38400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b301639a0 con 0x7f3b300c19d0 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b38400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30163c80 con 0x7f3b300b19a0 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b38400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30163f60 con 0x7f3b30160c10 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b35a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b300b19a0 0x7f3b300c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b35000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b300c19d0 0x7f3b30071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b35a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b300b19a0 0x7f3b300c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:33134/0 (socket says 172.21.15.66:33134) 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b35a00640 1 -- 172.21.15.66:0/2192724435 learned_addr learned my addr 172.21.15.66:0/2192724435 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b36400640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b30160c10 0x7f3b30071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b35a00640 1 -- 172.21.15.66:0/2192724435 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b30160c10 msgr2=0x7f3b30071950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b35a00640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b30160c10 0x7f3b30071950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b35a00640 1 -- 172.21.15.66:0/2192724435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b300c19d0 msgr2=0x7f3b30071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b35a00640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b300c19d0 0x7f3b30071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b35a00640 1 -- 172.21.15.66:0/2192724435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b28009a70 con 0x7f3b300b19a0 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b36400640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b30160c10 0x7f3b30071950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:17.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.813+0000 7f3b35a00640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b300b19a0 0x7f3b300c7b60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00e810 tx=0x7f3b1c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f1e32000640 1 -- 172.21.15.66:0/1778708330 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1e340f3c90 con 0x7f1e3c15d670 2024-03-30T00:40:17.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7fd1a6000640 1 -- 172.21.15.66:0/3259092158 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00125f0 con 0x7fd1ac161200 2024-03-30T00:40:17.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac0b2540 con 0x7fd1ac161200 2024-03-30T00:40:17.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f2c4cc00640 1 -- 172.21.15.66:0/3728239790 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c48007db0 con 0x7f2c501e2760 2024-03-30T00:40:17.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f2c4cc00640 1 -- 172.21.15.66:0/3728239790 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4800f050 con 0x7f2c501e2760 2024-03-30T00:40:17.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f2c4cc00640 1 -- 172.21.15.66:0/3728239790 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c48014650 con 0x7f2c501e2760 2024-03-30T00:40:17.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7fd1a6000640 1 -- 172.21.15.66:0/3259092158 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a0012750 con 0x7fd1ac161200 2024-03-30T00:40:17.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7fd1a6000640 1 -- 172.21.15.66:0/3259092158 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0012b00 con 0x7fd1ac161200 2024-03-30T00:40:17.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac0b2b00 con 0x7fd1ac161200 2024-03-30T00:40:17.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f2c57a00640 1 -- 172.21.15.66:0/3728239790 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c500c61d0 con 0x7f2c501e2760 2024-03-30T00:40:17.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f2c57a00640 1 -- 172.21.15.66:0/3728239790 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c500c5900 con 0x7f2c501e2760 2024-03-30T00:40:17.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f3b2d600640 1 -- 172.21.15.66:0/2192724435 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c004030 con 0x7f3b300b19a0 2024-03-30T00:40:17.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f3b2d600640 1 -- 172.21.15.66:0/2192724435 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c004190 con 0x7f3b300b19a0 2024-03-30T00:40:17.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f3b2d600640 1 -- 172.21.15.66:0/2192724435 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c004520 con 0x7f3b300b19a0 2024-03-30T00:40:17.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7fdeda000640 1 -- 172.21.15.66:0/1162432757 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fded4051260 con 0x7fdedc00aab0 2024-03-30T00:40:17.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f2c57a00640 1 -- 172.21.15.66:0/3728239790 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c50066c30 con 0x7f2c501e2760 2024-03-30T00:40:17.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f3b38400640 1 -- 172.21.15.66:0/2192724435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b300c5540 con 0x7f3b300b19a0 2024-03-30T00:40:17.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7f3b38400640 1 -- 172.21.15.66:0/2192724435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b30163720 con 0x7f3b300b19a0 2024-03-30T00:40:17.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7fdb8fe00640 1 -- 172.21.15.66:0/960592679 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdb8801b3b0 con 0x7fdb980cd260 2024-03-30T00:40:17.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.817+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ac066c30 con 0x7fd1ac161200 2024-03-30T00:40:17.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.821+0000 7f2c4cc00640 1 -- 172.21.15.66:0/3728239790 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2c480147b0 con 0x7f2c501e2760 2024-03-30T00:40:17.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.821+0000 7fdb8fe00640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb78066760 0x7fdb78068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.821+0000 7fdb8fe00640 1 -- 172.21.15.66:0/960592679 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdb88083d40 con 0x7fdb980cd260 2024-03-30T00:40:17.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.821+0000 7fdb97400640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb78066760 0x7fdb78068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.821+0000 7f3b38400640 1 -- 172.21.15.66:0/2192724435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b300c53a0 con 0x7f3b300b19a0 2024-03-30T00:40:17.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.821+0000 7f2c4cc00640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c30066710 0x7f2c30068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.821+0000 7f2c4cc00640 1 -- 172.21.15.66:0/3728239790 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2c48084810 con 0x7f2c501e2760 2024-03-30T00:40:17.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.821+0000 7f2c55000640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c30066710 0x7f2c30068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.821+0000 7fd1a6000640 1 -- 172.21.15.66:0/3259092158 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd1a0012c60 con 0x7fd1ac161200 2024-03-30T00:40:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.821+0000 7fd1a6000640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1900666c0 0x7fd190068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.821+0000 7fd1a6000640 1 -- 172.21.15.66:0/3259092158 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd1a0084740 con 0x7fd1ac161200 2024-03-30T00:40:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.825+0000 7fdb97400640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb78066760 0x7fdb78068c20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb900ae000 tx=0x7fdb90004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.825+0000 7fdb8fe00640 1 -- 172.21.15.66:0/960592679 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdb88050e50 con 0x7fdb980cd260 2024-03-30T00:40:17.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.825+0000 7fd1b1e00640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1900666c0 0x7fd190068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.825+0000 7f3b2d600640 1 -- 172.21.15.66:0/2192724435 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3b1c0096c0 con 0x7f3b300b19a0 2024-03-30T00:40:17.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.825+0000 7fd1b1e00640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1900666c0 0x7fd190068b80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0011f0 tx=0x7fd19c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.825+0000 7f2c55000640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c30066710 0x7f2c30068bd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0aa000 tx=0x7f2c3c001940 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.825+0000 7f3b2d600640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3b10066760 0x7f3b10068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:17.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.825+0000 7f3b2d600640 1 -- 172.21.15.66:0/2192724435 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3b1c0841f0 con 0x7f3b300b19a0 2024-03-30T00:40:17.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.825+0000 7f3b35000640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3b10066760 0x7f3b10068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:17.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.825+0000 7f3b35000640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3b10066760 0x7f3b10068c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b300b06e0 tx=0x7f3b20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:17.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.829+0000 7fd1a6000640 1 -- 172.21.15.66:0/3259092158 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd1a0051860 con 0x7fd1ac161200 2024-03-30T00:40:17.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.833+0000 7f3b2d600640 1 -- 172.21.15.66:0/2192724435 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3b1c051390 con 0x7f3b300b19a0 2024-03-30T00:40:17.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:17.837+0000 7f2c4cc00640 1 -- 172.21.15.66:0/3728239790 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2c48051900 con 0x7f2c501e2760 2024-03-30T00:40:18.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.133+0000 7f41f9000640 1 -- 172.21.15.66:0/310385177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f41f40cd540 con 0x7f41f40b63e0 2024-03-30T00:40:18.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.137+0000 7f41eb400640 1 -- 172.21.15.66:0/310385177 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f41e00547f0 con 0x7f41f40b63e0 2024-03-30T00:40:18.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.137+0000 7f0203c00640 1 -- 172.21.15.66:0/2779662022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f01fc1ffe60 con 0x7f01fc0719d0 2024-03-30T00:40:18.146 INFO:teuthology.orchestra.run.smithi066.stdout:55834574857 2024-03-30T00:40:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.137+0000 7f01f8c00640 1 -- 172.21.15.66:0/2779662022 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01f0054bc0 con 0x7f01fc0719d0 2024-03-30T00:40:18.148 INFO:teuthology.orchestra.run.smithi066.stdout:55834574857 2024-03-30T00:40:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 -- 172.21.15.66:0/310385177 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41d40666e0 msgr2=0x7f41d4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41d40666e0 0x7f41d4068ba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0af000 tx=0x7f41ec0a7070 comp rx=0 tx=0).stop 2024-03-30T00:40:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 -- 172.21.15.66:0/310385177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f40b63e0 msgr2=0x7f41f40b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f40b63e0 0x7f41f40b7a20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41e000c390 tx=0x7f41e000c850 comp rx=0 tx=0).stop 2024-03-30T00:40:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 -- 172.21.15.66:0/310385177 shutdown_connections 2024-03-30T00:40:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41f40b63e0 0x7f41f40b7a20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41f40d0e90 0x7f41f40b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41d40666e0 0x7f41d4068ba0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0af000 tx=0x7f41ec0a7070 comp rx=0 tx=0).stop 2024-03-30T00:40:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 --2- 172.21.15.66:0/310385177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41f40ce190 0x7f41f41575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 -- 172.21.15.66:0/310385177 >> 172.21.15.66:0/310385177 conn(0x7f41f409b8d0 msgr2=0x7f41f40b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:18.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 -- 172.21.15.66:0/310385177 shutdown_connections 2024-03-30T00:40:18.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f41f9000640 1 -- 172.21.15.66:0/310385177 wait complete. 2024-03-30T00:40:18.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f0203c00640 1 -- 172.21.15.66:0/2779662022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01dc066760 msgr2=0x7f01dc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f0203c00640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01dc066760 0x7f01dc068c20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0ac830 tx=0x7f01e8009040 comp rx=0 tx=0).stop 2024-03-30T00:40:18.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f0203c00640 1 -- 172.21.15.66:0/2779662022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01fc0719d0 msgr2=0x7f01fc1ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f0203c00640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01fc0719d0 0x7f01fc1ff690 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f01f000c7f0 tx=0x7f01f000ccb0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f0203c00640 1 -- 172.21.15.66:0/2779662022 shutdown_connections 2024-03-30T00:40:18.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f0203c00640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01fc16a460 0x7f01fc09b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f0203c00640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01fc0ca620 0x7f01fc0c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f0203c00640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01dc066760 0x7f01dc068c20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f0203c00640 1 --2- 172.21.15.66:0/2779662022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01fc0719d0 0x7f01fc1ff690 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.141+0000 7f0203c00640 1 -- 172.21.15.66:0/2779662022 >> 172.21.15.66:0/2779662022 conn(0x7f01fc070410 msgr2=0x7f01fc0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:18.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.145+0000 7f0203c00640 1 -- 172.21.15.66:0/2779662022 shutdown_connections 2024-03-30T00:40:18.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.145+0000 7f0203c00640 1 -- 172.21.15.66:0/2779662022 wait complete. 2024-03-30T00:40:18.174 INFO:teuthology.orchestra.run.smithi066.stdout:55834574856 2024-03-30T00:40:18.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.161+0000 7fdb9f200640 1 -- 172.21.15.66:0/960592679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fdb980b6480 con 0x7fdb980cd260 2024-03-30T00:40:18.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.161+0000 7fdb8fe00640 1 -- 172.21.15.66:0/960592679 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdb980b6480 con 0x7fdb980cd260 2024-03-30T00:40:18.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdee3200640 1 -- 172.21.15.66:0/1162432757 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdebc005510 con 0x7fdedc00aab0 2024-03-30T00:40:18.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdb9f200640 1 -- 172.21.15.66:0/960592679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb78066760 msgr2=0x7fdb78068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdb9f200640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb78066760 0x7fdb78068c20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb900ae000 tx=0x7fdb90004ac0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdb9f200640 1 -- 172.21.15.66:0/960592679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb980cd260 msgr2=0x7fdb9815b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdb9f200640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb980cd260 0x7fdb9815b9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c460 tx=0x7fdb8800c920 comp rx=0 tx=0).stop 2024-03-30T00:40:18.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdeda000640 1 -- 172.21.15.66:0/1162432757 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fded4054880 con 0x7fdedc00aab0 2024-03-30T00:40:18.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdb9f200640 1 -- 172.21.15.66:0/960592679 shutdown_connections 2024-03-30T00:40:18.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdb9f200640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb9815cb60 0x7fdb980b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdb9f200640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb78066760 0x7fdb78068c20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdb9f200640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb9815c780 0x7fdb980b7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdb9f200640 1 --2- 172.21.15.66:0/960592679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb980cd260 0x7fdb9815b9f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.165+0000 7fdb9f200640 1 -- 172.21.15.66:0/960592679 >> 172.21.15.66:0/960592679 conn(0x7fdb9809ba30 msgr2=0x7fdb980b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:18.177 INFO:teuthology.orchestra.run.smithi066.stdout:55834574856 2024-03-30T00:40:18.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.169+0000 7fdb9f200640 1 -- 172.21.15.66:0/960592679 shutdown_connections 2024-03-30T00:40:18.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.169+0000 7fdb9f200640 1 -- 172.21.15.66:0/960592679 wait complete. 2024-03-30T00:40:18.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.169+0000 7fdee3200640 1 -- 172.21.15.66:0/1162432757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdec40666e0 msgr2=0x7fdec4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.169+0000 7fdee3200640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdec40666e0 0x7fdec4068ba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fded001c3d0 tx=0x7fded00a6000 comp rx=0 tx=0).stop 2024-03-30T00:40:18.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.169+0000 7fdee3200640 1 -- 172.21.15.66:0/1162432757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdedc00aab0 msgr2=0x7fdedc0d8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.169+0000 7fdee3200640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdedc00aab0 0x7fdedc0d8a00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fded400c3e0 tx=0x7fded400c8a0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fdee3200640 1 -- 172.21.15.66:0/1162432757 shutdown_connections 2024-03-30T00:40:18.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fdee3200640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdec40666e0 0x7fdec4068ba0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fdee3200640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdedc050180 0x7fdedc036aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fdee3200640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdedc033cd0 0x7fdedc038bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fdee3200640 1 --2- 172.21.15.66:0/1162432757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdedc00aab0 0x7fdedc0d8a00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd1ac164880 con 0x7fd1ac161200 2024-03-30T00:40:18.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fdee3200640 1 -- 172.21.15.66:0/1162432757 >> 172.21.15.66:0/1162432757 conn(0x7fdedc009f90 msgr2=0x7fdedc0347e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:18.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fdee3200640 1 -- 172.21.15.66:0/1162432757 shutdown_connections 2024-03-30T00:40:18.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fdee3200640 1 -- 172.21.15.66:0/1162432757 wait complete. 2024-03-30T00:40:18.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fd1a6000640 1 -- 172.21.15.66:0/3259092158 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd1a0054e80 con 0x7fd1ac161200 2024-03-30T00:40:18.183 INFO:teuthology.orchestra.run.smithi066.stdout:55834574856 2024-03-30T00:40:18.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7f1e41200640 1 -- 172.21.15.66:0/1778708330 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1e3c0b6c00 con 0x7f1e3c15d670 2024-03-30T00:40:18.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7f1e32000640 1 -- 172.21.15.66:0/1778708330 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1e340c8020 con 0x7f1e3c15d670 2024-03-30T00:40:18.184 INFO:teuthology.orchestra.run.smithi066.stdout:55834574857 2024-03-30T00:40:18.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-03-30T00:40:18.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1900666c0 msgr2=0x7fd190068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fd1b4800640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1900666c0 0x7fd190068b80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0011f0 tx=0x7fd19c009040 comp rx=0 tx=0).stop 2024-03-30T00:40:18.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1ac161200 msgr2=0x7fd1ac0b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.173+0000 7fd1b4800640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1ac161200 0x7fd1ac0b41f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000d530 tx=0x7fd1a000d9f0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 shutdown_connections 2024-03-30T00:40:18.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7fd1b4800640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1ac161200 0x7fd1ac0b41f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7fd1b4800640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1900666c0 0x7fd190068b80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7fd1b4800640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ac160e20 0x7fd1ac0b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7fd1b4800640 1 --2- 172.21.15.66:0/3259092158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1ac0c29e0 0x7fd1ac0c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 >> 172.21.15.66:0/3259092158 conn(0x7fd1ac098a20 msgr2=0x7fd1ac160070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:18.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f3b38400640 1 -- 172.21.15.66:0/2192724435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3b300c53a0 con 0x7f3b300b19a0 2024-03-30T00:40:18.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 shutdown_connections 2024-03-30T00:40:18.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7fd1b4800640 1 -- 172.21.15.66:0/3259092158 wait complete. 2024-03-30T00:40:18.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f3b2d600640 1 -- 172.21.15.66:0/2192724435 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b300c53a0 con 0x7f3b300b19a0 2024-03-30T00:40:18.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 -- 172.21.15.66:0/1778708330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e1c0666e0 msgr2=0x7f1e1c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e1c0666e0 0x7f1e1c068ba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1e28007fd0 tx=0x7f1e280078c0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 -- 172.21.15.66:0/1778708330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c15d670 msgr2=0x7f1e3c147dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c15d670 0x7f1e3c147dc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1e3401cd40 tx=0x7f1e3401beb0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.188 INFO:teuthology.orchestra.run.smithi066.stdout:55834574858 2024-03-30T00:40:18.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 -- 172.21.15.66:0/1778708330 shutdown_connections 2024-03-30T00:40:18.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c15d670 0x7f1e3c147dc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e3c0d0270 0x7f1e3c147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e1c0666e0 0x7f1e1c068ba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 --2- 172.21.15.66:0/1778708330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c0cf090 0x7f1e3c0b7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 -- 172.21.15.66:0/1778708330 >> 172.21.15.66:0/1778708330 conn(0x7f1e3c089fb0 msgr2=0x7f1e3c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:18.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 -- 172.21.15.66:0/1778708330 shutdown_connections 2024-03-30T00:40:18.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.177+0000 7f1e41200640 1 -- 172.21.15.66:0/1778708330 wait complete. 2024-03-30T00:40:18.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-03-30T00:40:18.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 -- 172.21.15.66:0/2192724435 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3b10066760 msgr2=0x7f3b10068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3b10066760 0x7f3b10068c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b300b06e0 tx=0x7f3b20009040 comp rx=0 tx=0).stop 2024-03-30T00:40:18.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 -- 172.21.15.66:0/2192724435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b300b19a0 msgr2=0x7f3b300c7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b300b19a0 0x7f3b300c7b60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00e810 tx=0x7f3b1c00ecd0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 -- 172.21.15.66:0/2192724435 shutdown_connections 2024-03-30T00:40:18.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b30160c10 0x7f3b30071950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3b10066760 0x7f3b10068c20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b300c19d0 0x7f3b30071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 --2- 172.21.15.66:0/2192724435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b300b19a0 0x7f3b300c7b60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 -- 172.21.15.66:0/2192724435 >> 172.21.15.66:0/2192724435 conn(0x7f3b30069f60 msgr2=0x7f3b3006dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:18.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 -- 172.21.15.66:0/2192724435 shutdown_connections 2024-03-30T00:40:18.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.181+0000 7f3b38400640 1 -- 172.21.15.66:0/2192724435 wait complete. 2024-03-30T00:40:18.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.185+0000 7f2c57a00640 1 -- 172.21.15.66:0/3728239790 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2c500aff60 con 0x7f2c501e2760 2024-03-30T00:40:18.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.185+0000 7f2c4cc00640 1 -- 172.21.15.66:0/3728239790 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2c48054f20 con 0x7f2c501e2760 2024-03-30T00:40:18.197 INFO:teuthology.orchestra.run.smithi066.stdout:55834574856 2024-03-30T00:40:18.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 -- 172.21.15.66:0/3728239790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c30066710 msgr2=0x7f2c30068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c30066710 0x7f2c30068bd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0aa000 tx=0x7f2c3c001940 comp rx=0 tx=0).stop 2024-03-30T00:40:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 -- 172.21.15.66:0/3728239790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c501e2760 msgr2=0x7f2c501e2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c501e2760 0x7f2c501e2be0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c5006acc0 tx=0x7f2c48009dc0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 -- 172.21.15.66:0/3728239790 shutdown_connections 2024-03-30T00:40:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c501e2760 0x7f2c501e2be0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c500c7170 0x7f2c500c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c30066710 0x7f2c30068bd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 --2- 172.21.15.66:0/3728239790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c38003980 0x7f2c5015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 -- 172.21.15.66:0/3728239790 >> 172.21.15.66:0/3728239790 conn(0x7f2c50069f60 msgr2=0x7f2c5006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 -- 172.21.15.66:0/3728239790 shutdown_connections 2024-03-30T00:40:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:18.189+0000 7f2c57a00640 1 -- 172.21.15.66:0/3728239790 wait complete. 2024-03-30T00:40:18.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2024-03-30T00:40:18.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2024-03-30T00:40:18.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-03-30T00:40:18.229 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2024-03-30T00:40:18.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2024-03-30T00:40:18.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2024-03-30T00:40:19.186 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-30T00:40:19.195 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-30T00:40:19.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-30T00:40:19.216 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-30T00:40:19.227 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-30T00:40:19.229 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-30T00:40:19.233 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-30T00:40:19.234 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-30T00:40:19.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.405+0000 7f817de00640 1 Processor -- start 2024-03-30T00:40:19.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.405+0000 7f817de00640 1 -- start start 2024-03-30T00:40:19.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.405+0000 7f817de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f81780af970 0x7f81780b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.405+0000 7f817de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81780b1f60 0x7f81780adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.405+0000 7f817de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8178071630 con 0x7f81780af590 2024-03-30T00:40:19.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.405+0000 7f817de00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81780722b0 con 0x7f81780af970 2024-03-30T00:40:19.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.405+0000 7f817de00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81780b23d0 con 0x7f81780b1f60 2024-03-30T00:40:19.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.409+0000 7f8176a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f81780af590 0x7f8178159720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:43214/0 (socket says 172.21.15.66:43214) 2024-03-30T00:40:19.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.409+0000 7f8176a00640 1 -- 172.21.15.66:0/1495283308 learned_addr learned my addr 172.21.15.66:0/1495283308 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:19.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.409+0000 7f8176000640 1 --2- 172.21.15.66:0/1495283308 >> v2:172.21.15.66:3300/0 conn(0x7f81780af970 0x7f81780b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.409+0000 7f8176000640 1 -- 172.21.15.66:0/1495283308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81780b1f60 msgr2=0x7f81780adbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:40:19.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.409+0000 7f8176000640 1 --2- 172.21.15.66:0/1495283308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81780b1f60 0x7f81780adbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.409+0000 7f8176000640 1 -- 172.21.15.66:0/1495283308 >> v1:172.21.15.66:6789/0 conn(0x7f81780af590 legacy=0x7f8178159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.409+0000 7f8176000640 1 -- 172.21.15.66:0/1495283308 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8178071910 con 0x7f81780af970 2024-03-30T00:40:19.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.413+0000 7f8176000640 1 --2- 172.21.15.66:0/1495283308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81780af970 0x7f81780b1a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f816800a990 tx=0x7f816801d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76eb249c33b66cf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.413+0000 7f8175600640 1 -- 172.21.15.66:0/1495283308 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81680a68a0 con 0x7f81780af970 2024-03-30T00:40:19.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.413+0000 7f8175600640 1 -- 172.21.15.66:0/1495283308 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81680a6a00 con 0x7f81780af970 2024-03-30T00:40:19.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.413+0000 7f8175600640 1 -- 172.21.15.66:0/1495283308 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81680a6dd0 con 0x7f81780af970 2024-03-30T00:40:19.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.417+0000 7f817de00640 1 -- 172.21.15.66:0/1495283308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81780af970 msgr2=0x7f81780b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.417+0000 7f817de00640 1 --2- 172.21.15.66:0/1495283308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81780af970 0x7f81780b1a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f816800a990 tx=0x7f816801d7f0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.417+0000 7f817de00640 1 -- 172.21.15.66:0/1495283308 shutdown_connections 2024-03-30T00:40:19.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.417+0000 7f817de00640 1 --2- 172.21.15.66:0/1495283308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81780b1f60 0x7f81780adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.417+0000 7f817de00640 1 --2- 172.21.15.66:0/1495283308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81780af970 0x7f81780b1a20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.417+0000 7f817de00640 1 -- 172.21.15.66:0/1495283308 >> 172.21.15.66:0/1495283308 conn(0x7f8178070410 msgr2=0x7f81780afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.417+0000 7f817de00640 1 -- 172.21.15.66:0/1495283308 shutdown_connections 2024-03-30T00:40:19.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.417+0000 7f817de00640 1 -- 172.21.15.66:0/1495283308 wait complete. 2024-03-30T00:40:19.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.417+0000 7f817de00640 1 Processor -- start 2024-03-30T00:40:19.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.417+0000 7f817de00640 1 -- start start 2024-03-30T00:40:19.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f817de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81780af590 0x7f817814a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f817de00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81780b1f60 0x7f817815aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f817de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81780cd340 0x7f8178097860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f817de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8178149e40 con 0x7f81780af590 2024-03-30T00:40:19.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f8176a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81780af590 0x7f817814a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f8176a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81780af590 0x7f817814a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33958/0 (socket says 172.21.15.66:33958) 2024-03-30T00:40:19.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817814a810 con 0x7f81780b1f60 2024-03-30T00:40:19.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817814b040 con 0x7f81780cd340 2024-03-30T00:40:19.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f8176a00640 1 -- 172.21.15.66:0/4157890182 learned_addr learned my addr 172.21.15.66:0/4157890182 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f8177400640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81780cd340 0x7f8178097860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f8176000640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81780b1f60 0x7f817815aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f8177400640 1 -- 172.21.15.66:0/4157890182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81780b1f60 msgr2=0x7f817815aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f8177400640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81780b1f60 0x7f817815aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.421+0000 7f8177400640 1 -- 172.21.15.66:0/4157890182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81780af590 msgr2=0x7f817814a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.425+0000 7f8177400640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81780af590 0x7f817814a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.425+0000 7f8177400640 1 -- 172.21.15.66:0/4157890182 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816800a5c0 con 0x7f81780cd340 2024-03-30T00:40:19.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.425+0000 7f8177400640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81780cd340 0x7f8178097860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f816400b410 tx=0x7f816400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.425+0000 7f816f400640 1 -- 172.21.15.66:0/4157890182 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8164011830 con 0x7f81780cd340 2024-03-30T00:40:19.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.425+0000 7f816f400640 1 -- 172.21.15.66:0/4157890182 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8164011990 con 0x7f81780cd340 2024-03-30T00:40:19.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.425+0000 7f816f400640 1 -- 172.21.15.66:0/4157890182 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8164011d40 con 0x7f81780cd340 2024-03-30T00:40:19.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.425+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817814b320 con 0x7f81780cd340 2024-03-30T00:40:19.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.425+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817815afb0 con 0x7f81780cd340 2024-03-30T00:40:19.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.425+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81780cdfa0 con 0x7f81780cd340 2024-03-30T00:40:19.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.429+0000 7f816f400640 1 -- 172.21.15.66:0/4157890182 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f81640193b0 con 0x7f81780cd340 2024-03-30T00:40:19.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.429+0000 7f816f400640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8158066760 0x7f8158068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.429+0000 7f816f400640 1 -- 172.21.15.66:0/4157890182 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8164084090 con 0x7f81780cd340 2024-03-30T00:40:19.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.433+0000 7f8176a00640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8158066760 0x7f8158068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.433+0000 7f8176a00640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8158066760 0x7f8158068c20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f81700012f0 tx=0x7f817000a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.433+0000 7f816f400640 1 -- 172.21.15.66:0/4157890182 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8164051230 con 0x7f81780cd340 2024-03-30T00:40:19.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.453+0000 7fc6b0e00640 1 Processor -- start 2024-03-30T00:40:19.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.453+0000 7f2443200640 1 Processor -- start 2024-03-30T00:40:19.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.453+0000 7f2443200640 1 -- start start 2024-03-30T00:40:19.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.453+0000 7fc6b0e00640 1 -- start start 2024-03-30T00:40:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2443200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f244415d070 0x7f24440cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2443200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24440aebb0 0x7f24440717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2443200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244406ab80 con 0x7f2444071320 2024-03-30T00:40:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2443200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24440aefb0 con 0x7f244415d070 2024-03-30T00:40:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2443200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2444071cf0 con 0x7f24440aebb0 2024-03-30T00:40:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7fc6b0e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af590 0x7fc6ac159720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7fc6b0e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc6ac0b2130 0x7fc6ac0adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7fc6b0e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ac159c60 con 0x7fc6ac0af970 2024-03-30T00:40:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7fc6b0e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ac0722b0 con 0x7fc6ac0b2130 2024-03-30T00:40:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7fc6b0e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac071630 con 0x7fc6ac0af590 2024-03-30T00:40:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2442800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24440aebb0 0x7f24440717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7fc6a9e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af590 0x7fc6ac159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7fc6a9e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af590 0x7fc6ac159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:60688/0 (socket says 172.21.15.66:60688) 2024-03-30T00:40:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7fc6a9e00640 1 -- 172.21.15.66:0/887022870 learned_addr learned my addr 172.21.15.66:0/887022870 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2442800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24440aebb0 0x7f24440717b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:60684/0 (socket says 172.21.15.66:60684) 2024-03-30T00:40:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2442800640 1 -- 172.21.15.66:0/3252888616 learned_addr learned my addr 172.21.15.66:0/3252888616 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2441e00640 1 --2- 172.21.15.66:0/3252888616 >> v2:172.21.15.156:3300/0 conn(0x7f244415d070 0x7f24440cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2442800640 1 -- 172.21.15.66:0/3252888616 >> v2:172.21.15.156:3300/0 conn(0x7f244415d070 msgr2=0x7f24440cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2442800640 1 --2- 172.21.15.66:0/3252888616 >> v2:172.21.15.156:3300/0 conn(0x7f244415d070 0x7f24440cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2442800640 1 -- 172.21.15.66:0/3252888616 >> v1:172.21.15.66:6789/0 conn(0x7f2444071320 legacy=0x7f24440c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7f2442800640 1 -- 172.21.15.66:0/3252888616 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440af190 con 0x7f24440aebb0 2024-03-30T00:40:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.457+0000 7fc6aa800640 1 --2- 172.21.15.66:0/887022870 >> v2:172.21.15.156:3300/0 conn(0x7fc6ac0b2130 0x7fc6ac0adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.461+0000 7fc6a8a00640 1 -- 172.21.15.66:0/887022870 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304438480 0 0) 0x7fc6ac159c60 con 0x7fc6ac0af970 2024-03-30T00:40:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.461+0000 7fc6a8a00640 1 -- 172.21.15.66:0/887022870 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc690005740 con 0x7fc6ac0af970 2024-03-30T00:40:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.461+0000 7fc6aa800640 1 -- 172.21.15.66:0/887022870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af590 msgr2=0x7fc6ac159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.461+0000 7fc6a8a00640 1 -- 172.21.15.66:0/887022870 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2066331805 0 0) 0x7fc690005740 con 0x7fc6ac0af970 2024-03-30T00:40:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.461+0000 7fc6aa800640 1 --2- 172.21.15.66:0/887022870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af590 0x7fc6ac159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.461+0000 7fc6aa800640 1 -- 172.21.15.66:0/887022870 >> v1:172.21.15.156:6789/0 conn(0x7fc6ac0af970 legacy=0x7fc6ac0b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.461+0000 7fc6aa800640 1 -- 172.21.15.66:0/887022870 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac071910 con 0x7fc6ac0b2130 2024-03-30T00:40:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f2442800640 1 --2- 172.21.15.66:0/3252888616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24440aebb0 0x7f24440717b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2430009870 tx=0x7f243001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81df8bb66f1ed55e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7fc6aa800640 1 --2- 172.21.15.66:0/887022870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6ac0b2130 0x7fc6ac0adbe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc6980097c0 tx=0x7fc69801c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3150da6d2125a064 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f2440a00640 1 -- 172.21.15.66:0/3252888616 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24300ac620 con 0x7f24440aebb0 2024-03-30T00:40:19.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f2440a00640 1 -- 172.21.15.66:0/3252888616 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24300ac780 con 0x7f24440aebb0 2024-03-30T00:40:19.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f73b4c00640 1 Processor -- start 2024-03-30T00:40:19.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7fc6a8a00640 1 -- 172.21.15.66:0/887022870 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c001660 con 0x7fc6ac0b2130 2024-03-30T00:40:19.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7fc6a8a00640 1 -- 172.21.15.66:0/887022870 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6980a7050 con 0x7fc6ac0b2130 2024-03-30T00:40:19.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7fc6a8a00640 1 -- 172.21.15.66:0/887022870 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6980ab620 con 0x7fc6ac0b2130 2024-03-30T00:40:19.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f2440a00640 1 -- 172.21.15.66:0/3252888616 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24300acb10 con 0x7f24440aebb0 2024-03-30T00:40:19.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f2443200640 1 -- 172.21.15.66:0/3252888616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24440aebb0 msgr2=0x7f24440717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f2443200640 1 --2- 172.21.15.66:0/3252888616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24440aebb0 0x7f24440717b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2430009870 tx=0x7f243001c400 comp rx=0 tx=0).stop 2024-03-30T00:40:19.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f73b4c00640 1 -- start start 2024-03-30T00:40:19.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f2443200640 1 -- 172.21.15.66:0/3252888616 shutdown_connections 2024-03-30T00:40:19.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f2443200640 1 --2- 172.21.15.66:0/3252888616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24440aebb0 0x7f24440717b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f2443200640 1 --2- 172.21.15.66:0/3252888616 >> v2:172.21.15.156:3300/0 conn(0x7f244415d070 0x7f24440cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7f2443200640 1 -- 172.21.15.66:0/3252888616 >> 172.21.15.66:0/3252888616 conn(0x7f24440ce1e0 msgr2=0x7f24440ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7fc6b0e00640 1 -- 172.21.15.66:0/887022870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6ac0b2130 msgr2=0x7fc6ac0adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.465+0000 7fc6b0e00640 1 --2- 172.21.15.66:0/887022870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6ac0b2130 0x7fc6ac0adbe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc6980097c0 tx=0x7fc69801c300 comp rx=0 tx=0).stop 2024-03-30T00:40:19.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7fc6b0e00640 1 -- 172.21.15.66:0/887022870 shutdown_connections 2024-03-30T00:40:19.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f2443200640 1 -- 172.21.15.66:0/3252888616 shutdown_connections 2024-03-30T00:40:19.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7fc6b0e00640 1 --2- 172.21.15.66:0/887022870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6ac0b2130 0x7fc6ac0adbe0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7fc6b0e00640 1 --2- 172.21.15.66:0/887022870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af590 0x7fc6ac159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7fc6b0e00640 1 -- 172.21.15.66:0/887022870 >> 172.21.15.66:0/887022870 conn(0x7fc6ac070410 msgr2=0x7fc6ac0afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73b4c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f73b0160ad0 0x7f73b006cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73b4c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73b00cdae0 0x7f73b00b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73b4c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b00b17e0 con 0x7f73b00c11b0 2024-03-30T00:40:19.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73b4c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b006d7b0 con 0x7f73b0160ad0 2024-03-30T00:40:19.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73b4c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00b11d0 con 0x7f73b00cdae0 2024-03-30T00:40:19.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f2443200640 1 -- 172.21.15.66:0/3252888616 wait complete. 2024-03-30T00:40:19.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73af400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f73b0160ad0 0x7f73b006cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73af400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f73b0160ad0 0x7f73b006cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33970/0 (socket says 172.21.15.66:33970) 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73af400640 1 -- 172.21.15.66:0/3975096845 learned_addr learned my addr 172.21.15.66:0/3975096845 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73aea00640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73b00cdae0 0x7f73b00b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609803949 0 0) 0x7f73b00b17e0 con 0x7f73b00c11b0 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7394003530 con 0x7f73b00c11b0 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2372078873 0 0) 0x7f7394003530 con 0x7f73b00c11b0 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73b00cdae0 msgr2=0x7f73b00b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73b00cdae0 0x7f73b00b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 >> v2:172.21.15.66:3300/0 conn(0x7f73b0160ad0 msgr2=0x7f73b006cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 --2- 172.21.15.66:0/3975096845 >> v2:172.21.15.66:3300/0 conn(0x7f73b0160ad0 0x7f73b006cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b006d4d0 con 0x7f73b00c11b0 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f73a80019e0 con 0x7f73b00c11b0 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 >> v1:172.21.15.156:6789/0 conn(0x7f73b00c11b0 legacy=0x7f73b00cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73940039a0 0x7f7394003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73940047d0 0x7f7394004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7394005190 0x7f739400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b006d7b0 con 0x7f73940039a0 2024-03-30T00:40:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00b11d0 con 0x7f73940047d0 2024-03-30T00:40:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739400c040 con 0x7f7394005190 2024-03-30T00:40:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73afe00640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7394005190 0x7f739400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73aea00640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73940047d0 0x7f7394004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7fc6b0e00640 1 -- 172.21.15.66:0/887022870 shutdown_connections 2024-03-30T00:40:19.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7fc6b0e00640 1 -- 172.21.15.66:0/887022870 wait complete. 2024-03-30T00:40:19.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73af400640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73940039a0 0x7f7394003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f2443200640 1 Processor -- start 2024-03-30T00:40:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73afe00640 1 -- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73940047d0 msgr2=0x7f7394004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73afe00640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73940047d0 0x7f7394004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73afe00640 1 -- 172.21.15.66:0/3975096845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73940039a0 msgr2=0x7f7394003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73afe00640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73940039a0 0x7f7394003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.469+0000 7f73afe00640 1 -- 172.21.15.66:0/3975096845 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7394003530 con 0x7f7394005190 2024-03-30T00:40:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2443200640 1 -- start start 2024-03-30T00:40:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f73af400640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73940039a0 0x7f7394003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:19.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2443200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2444071320 0x7f24440c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2443200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24440aebb0 0x7f24441f9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2443200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f244415d070 0x7f24440c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2443200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24441fa290 con 0x7f244415d070 2024-03-30T00:40:19.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2443200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24441fa570 con 0x7f2444071320 2024-03-30T00:40:19.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2443200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24441fa850 con 0x7f24440aebb0 2024-03-30T00:40:19.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2442800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f244415d070 0x7f24440c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2442800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f244415d070 0x7f24440c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33976/0 (socket says 172.21.15.66:33976) 2024-03-30T00:40:19.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2442800640 1 -- 172.21.15.66:0/3056203998 learned_addr learned my addr 172.21.15.66:0/3056203998 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7fc6b0e00640 1 Processor -- start 2024-03-30T00:40:19.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2442800640 1 -- 172.21.15.66:0/3056203998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24440aebb0 msgr2=0x7f24441f9980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T00:40:19.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7fc6b0e00640 1 -- start start 2024-03-30T00:40:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2442800640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24440aebb0 0x7f24441f9980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2442800640 1 -- 172.21.15.66:0/3056203998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2444071320 msgr2=0x7f24440c7bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T00:40:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2442800640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2444071320 0x7f24440c7bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.473+0000 7f2442800640 1 -- 172.21.15.66:0/3056203998 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24300094d0 con 0x7f244415d070 2024-03-30T00:40:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f2442800640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f244415d070 0x7f24440c5cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f243001a030 tx=0x7f24300191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f243a000640 1 -- 172.21.15.66:0/3056203998 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24300ac620 con 0x7f244415d070 2024-03-30T00:40:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f243a000640 1 -- 172.21.15.66:0/3056203998 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24300b6440 con 0x7f244415d070 2024-03-30T00:40:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f2443200640 1 -- 172.21.15.66:0/3056203998 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24441fae30 con 0x7f244415d070 2024-03-30T00:40:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f243a000640 1 -- 172.21.15.66:0/3056203998 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24300b67f0 con 0x7f244415d070 2024-03-30T00:40:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f2443200640 1 -- 172.21.15.66:0/3056203998 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24441f9ec0 con 0x7f244415d070 2024-03-30T00:40:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f73afe00640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7394005190 0x7f739400b990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f73a800c490 tx=0x7f73a801cd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2c9e1e4dd119fc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7fc6b0e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6ac0af590 0x7fc6ac14e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7fc6b0e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af970 0x7fc6ac097a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7fc6b0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ac0b2130 0x7fc6ac150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7fc6b0e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac14b9e0 con 0x7fc6ac0b2130 2024-03-30T00:40:19.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7fc6b0e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac14bcc0 con 0x7fc6ac0af590 2024-03-30T00:40:19.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7fc6b0e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac150110 con 0x7fc6ac0af970 2024-03-30T00:40:19.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f2443200640 1 -- 172.21.15.66:0/3056203998 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241c005260 con 0x7f244415d070 2024-03-30T00:40:19.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a80a5a90 con 0x7f7394005190 2024-03-30T00:40:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7fc6a9e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6ac0af590 0x7fc6ac14e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7fc6a9e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6ac0af590 0x7fc6ac14e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:34658/0 (socket says 172.21.15.66:34658) 2024-03-30T00:40:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7fc6a9e00640 1 -- 172.21.15.66:0/1985691916 learned_addr learned my addr 172.21.15.66:0/1985691916 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a80a5bf0 con 0x7f7394005190 2024-03-30T00:40:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f73ae000640 1 -- 172.21.15.66:0/3975096845 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8003980 con 0x7f7394005190 2024-03-30T00:40:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7fc6a9400640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af970 0x7fc6ac097a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.477+0000 7f9a5f200640 1 Processor -- start 2024-03-30T00:40:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f9a5f200640 1 -- start start 2024-03-30T00:40:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7fc6a9400640 1 -- 172.21.15.66:0/1985691916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6ac0af590 msgr2=0x7fc6ac14e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7fc6a9400640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6ac0af590 0x7fc6ac14e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7fc6a9400640 1 -- 172.21.15.66:0/1985691916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ac0b2130 msgr2=0x7fc6ac150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7fc6aa800640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ac0b2130 0x7fc6ac150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f73b4c00640 1 -- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7394005190 msgr2=0x7f739400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f73b4c00640 1 --2- 172.21.15.66:0/3975096845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7394005190 0x7f739400b990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f73a800c490 tx=0x7f73a801cd90 comp rx=0 tx=0).stop 2024-03-30T00:40:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f73afe00640 1 -- 172.21.15.66:0/3975096845 reap_dead start 2024-03-30T00:40:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7fc6a9400640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ac0b2130 0x7fc6ac150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7fc6a9400640 1 -- 172.21.15.66:0/1985691916 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698009420 con 0x7fc6ac0af970 2024-03-30T00:40:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f73b4c00640 1 -- 172.21.15.66:0/3975096845 shutdown_connections 2024-03-30T00:40:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f73b4c00640 1 -- 172.21.15.66:0/3975096845 >> 172.21.15.66:0/3975096845 conn(0x7f73b006a190 msgr2=0x7f73b006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7fc6aa800640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ac0b2130 0x7fc6ac150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f243a000640 1 -- 172.21.15.66:0/3056203998 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f24300aa070 con 0x7f244415d070 2024-03-30T00:40:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f73b4c00640 1 -- 172.21.15.66:0/3975096845 shutdown_connections 2024-03-30T00:40:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7fc6a9400640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af970 0x7fc6ac097a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c520 tx=0x7fc69c00c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f9a5f200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a600c18c0 0x7f9a600cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f9a5f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a600b17e0 con 0x7f9a6006cef0 2024-03-30T00:40:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f9a5f200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a600ce020 con 0x7f9a60160ad0 2024-03-30T00:40:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f9a5f200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6006d7b0 con 0x7f9a600c18c0 2024-03-30T00:40:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7fc6a2000640 1 -- 172.21.15.66:0/1985691916 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c005480 con 0x7fc6ac0af970 2024-03-30T00:40:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f9a5e800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a600c18c0 0x7f9a600cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f9a5e800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a600c18c0 0x7f9a600cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:60730/0 (socket says 172.21.15.66:60730) 2024-03-30T00:40:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f9a5e800640 1 -- 172.21.15.66:0/1923824524 learned_addr learned my addr 172.21.15.66:0/1923824524 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7fc6b0e00640 1 -- 172.21.15.66:0/1985691916 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac1503f0 con 0x7fc6ac0af970 2024-03-30T00:40:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7fc6b0e00640 1 -- 172.21.15.66:0/1985691916 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac151020 con 0x7fc6ac0af970 2024-03-30T00:40:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f9a5e800640 1 -- 172.21.15.66:0/1923824524 >> v1:172.21.15.156:6789/0 conn(0x7f9a60160ad0 legacy=0x7f9a600b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f9a5e800640 1 -- 172.21.15.66:0/1923824524 >> v1:172.21.15.66:6789/0 conn(0x7f9a6006cef0 legacy=0x7f9a600c11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f73b4c00640 1 -- 172.21.15.66:0/3975096845 wait complete. 2024-03-30T00:40:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73b4c00640 1 Processor -- start 2024-03-30T00:40:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73b4c00640 1 -- start start 2024-03-30T00:40:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f243a000640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2424066690 0x7f2424068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f243a000640 1 -- 172.21.15.66:0/3056203998 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f243011e660 con 0x7f244415d070 2024-03-30T00:40:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f2441e00640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2424066690 0x7f2424068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f2441e00640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2424066690 0x7f2424068b50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2434004660 tx=0x7f2434008180 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.481+0000 7f9a5e800640 1 -- 172.21.15.66:0/1923824524 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a600b2380 con 0x7f9a600c18c0 2024-03-30T00:40:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7fc6a2000640 1 -- 172.21.15.66:0/1985691916 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c003600 con 0x7fc6ac0af970 2024-03-30T00:40:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7fc6a2000640 1 -- 172.21.15.66:0/1985691916 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c003970 con 0x7fc6ac0af970 2024-03-30T00:40:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73b4c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7394005190 0x7f73b0089fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73b4c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73b0097570 0x7f73b00c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73b4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b00c76f0 0x7f73b00c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73b4c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00c80b0 con 0x7f73b00c76f0 2024-03-30T00:40:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73b4c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0164960 con 0x7f7394005190 2024-03-30T00:40:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73b4c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00c2400 con 0x7f73b0097570 2024-03-30T00:40:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7fc6a2000640 1 -- 172.21.15.66:0/1985691916 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc69c003b90 con 0x7fc6ac0af970 2024-03-30T00:40:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73af400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7394005190 0x7f73b0089fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73aea00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73b0097570 0x7f73b00c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73af400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7394005190 0x7f73b0089fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:34668/0 (socket says 172.21.15.66:34668) 2024-03-30T00:40:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.485+0000 7f73af400640 1 -- 172.21.15.66:0/3360750218 learned_addr learned my addr 172.21.15.66:0/3360750218 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73afe00640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b00c76f0 0x7f73b00c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7fc697400640 1 -- 172.21.15.66:0/1985691916 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac1511b0 con 0x7fc6ac0af970 2024-03-30T00:40:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73af400640 1 -- 172.21.15.66:0/3360750218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73b0097570 msgr2=0x7f73b00c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73af400640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73b0097570 0x7f73b00c1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73af400640 1 -- 172.21.15.66:0/3360750218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b00c76f0 msgr2=0x7f73b00c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73af400640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b00c76f0 0x7f73b00c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73af400640 1 -- 172.21.15.66:0/3360750218 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b008a640 con 0x7f7394005190 2024-03-30T00:40:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73afe00640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b00c76f0 0x7f73b00c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f9a5e800640 1 --2- 172.21.15.66:0/1923824524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a600c18c0 0x7f9a600cdae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0098a0 tx=0x7f9a4c01c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83db5d57de2f1496 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7ff8bba00640 1 Processor -- start 2024-03-30T00:40:19.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73af400640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7394005190 0x7f73b0089fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f73a0010890 tx=0x7f73a0010d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7fc6a2000640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc68c066690 0x7fc68c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7ff8bba00640 1 -- start start 2024-03-30T00:40:19.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73a7400640 1 -- 172.21.15.66:0/3360750218 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0004220 con 0x7f7394005190 2024-03-30T00:40:19.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73a7400640 1 -- 172.21.15.66:0/3360750218 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a0004380 con 0x7f7394005190 2024-03-30T00:40:19.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73a7400640 1 -- 172.21.15.66:0/3360750218 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000eba0 con 0x7f7394005190 2024-03-30T00:40:19.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73b4c00640 1 -- 172.21.15.66:0/3360750218 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b008a920 con 0x7f7394005190 2024-03-30T00:40:19.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7fc6a2000640 1 -- 172.21.15.66:0/1985691916 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc69c0839e0 con 0x7fc6ac0af970 2024-03-30T00:40:19.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7fc6a9e00640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc68c066690 0x7fc68c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7f73b4c00640 1 -- 172.21.15.66:0/3360750218 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b01645f0 con 0x7f7394005190 2024-03-30T00:40:19.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7ff8bba00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff8b4160e20 0x7ff8b40af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7ff8bba00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8b4161200 0x7ff8b40aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7ff8bba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8b40c24a0 0x7ff8b40c2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7ff8bba00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b40c2e90 con 0x7ff8b4161200 2024-03-30T00:40:19.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7ff8bba00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b4165450 con 0x7ff8b4160e20 2024-03-30T00:40:19.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.489+0000 7ff8bba00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b4164b40 con 0x7ff8b40c24a0 2024-03-30T00:40:19.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7fc6a9e00640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc68c066690 0x7fc68c068b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac14c0c0 tx=0x7fc6a400c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7f73b4c00640 1 -- 172.21.15.66:0/3360750218 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b0073e00 con 0x7f7394005190 2024-03-30T00:40:19.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7ff8b9a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8b40c24a0 0x7ff8b40c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7ff8b3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8b4161200 0x7ff8b40aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7ff8b3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8b4161200 0x7ff8b40aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33988/0 (socket says 172.21.15.66:33988) 2024-03-30T00:40:19.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7ff8b9a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8b40c24a0 0x7ff8b40c2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:60740/0 (socket says 172.21.15.66:60740) 2024-03-30T00:40:19.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7ff8b9a00640 1 -- 172.21.15.66:0/3715427913 learned_addr learned my addr 172.21.15.66:0/3715427913 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7ff8b9000640 1 --2- 172.21.15.66:0/3715427913 >> v2:172.21.15.156:3300/0 conn(0x7ff8b4160e20 0x7ff8b40af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7ff8b9000640 1 -- 172.21.15.66:0/3715427913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8b40c24a0 msgr2=0x7ff8b40c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7ff8b9000640 1 --2- 172.21.15.66:0/3715427913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8b40c24a0 0x7ff8b40c2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7ff8b9000640 1 -- 172.21.15.66:0/3715427913 >> v2:172.21.15.66:3300/0 conn(0x7ff8b4161200 msgr2=0x7ff8b40aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7ff8b9000640 1 --2- 172.21.15.66:0/3715427913 >> v2:172.21.15.66:3300/0 conn(0x7ff8b4161200 0x7ff8b40aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7ff8b9000640 1 -- 172.21.15.66:0/3715427913 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b40b0530 con 0x7ff8b4160e20 2024-03-30T00:40:19.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7f73a7400640 1 -- 172.21.15.66:0/3360750218 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f73a000ed00 con 0x7f7394005190 2024-03-30T00:40:19.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7f9a5ca00640 1 -- 172.21.15.66:0/1923824524 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0ac410 con 0x7f9a600c18c0 2024-03-30T00:40:19.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7f9a5ca00640 1 -- 172.21.15.66:0/1923824524 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c0ac570 con 0x7f9a600c18c0 2024-03-30T00:40:19.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7f9a5ca00640 1 -- 172.21.15.66:0/1923824524 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0ac900 con 0x7f9a600c18c0 2024-03-30T00:40:19.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7fc6a2000640 1 -- 172.21.15.66:0/1985691916 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc69c050c10 con 0x7fc6ac0af970 2024-03-30T00:40:19.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7f73a7400640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73900666e0 0x7f7390068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7f9a5f200640 1 -- 172.21.15.66:0/1923824524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a600c18c0 msgr2=0x7f9a600cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7f9a5f200640 1 --2- 172.21.15.66:0/1923824524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a600c18c0 0x7f9a600cdae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0098a0 tx=0x7f9a4c01c500 comp rx=0 tx=0).stop 2024-03-30T00:40:19.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.493+0000 7f73a7400640 1 -- 172.21.15.66:0/3360750218 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f73a0084750 con 0x7f7394005190 2024-03-30T00:40:19.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7f73aea00640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73900666e0 0x7f7390068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7f9a5f200640 1 -- 172.21.15.66:0/1923824524 shutdown_connections 2024-03-30T00:40:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7f9a5f200640 1 --2- 172.21.15.66:0/1923824524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a600c18c0 0x7f9a600cdae0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7f9a5f200640 1 -- 172.21.15.66:0/1923824524 >> 172.21.15.66:0/1923824524 conn(0x7f9a6006a190 msgr2=0x7f9a6006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7f9a5f200640 1 -- 172.21.15.66:0/1923824524 shutdown_connections 2024-03-30T00:40:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7f9a5f200640 1 -- 172.21.15.66:0/1923824524 wait complete. 2024-03-30T00:40:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7f243a000640 1 -- 172.21.15.66:0/3056203998 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f24300eb770 con 0x7f244415d070 2024-03-30T00:40:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7ff8b9000640 1 --2- 172.21.15.66:0/3715427913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8b4160e20 0x7ff8b40af9c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40b0f90 tx=0x7ff8ac01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d90e7b31273d5ff2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7f73aea00640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73900666e0 0x7f7390068ba0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f73b00c82b0 tx=0x7f739c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7ff8b3400640 1 -- 172.21.15.66:0/3715427913 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0ab610 con 0x7ff8b4160e20 2024-03-30T00:40:19.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7f73a7400640 1 -- 172.21.15.66:0/3360750218 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f73a0051840 con 0x7f7394005190 2024-03-30T00:40:19.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7ff8b3400640 1 -- 172.21.15.66:0/3715427913 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac0ab770 con 0x7ff8b4160e20 2024-03-30T00:40:19.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.497+0000 7ff8b3400640 1 -- 172.21.15.66:0/3715427913 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0abb00 con 0x7ff8b4160e20 2024-03-30T00:40:19.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7ff8bba00640 1 -- 172.21.15.66:0/3715427913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8b4160e20 msgr2=0x7ff8b40af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7ff8bba00640 1 --2- 172.21.15.66:0/3715427913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8b4160e20 0x7ff8b40af9c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40b0f90 tx=0x7ff8ac01c450 comp rx=0 tx=0).stop 2024-03-30T00:40:19.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7ff8bba00640 1 -- 172.21.15.66:0/3715427913 shutdown_connections 2024-03-30T00:40:19.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7ff8bba00640 1 --2- 172.21.15.66:0/3715427913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8b40c24a0 0x7ff8b40c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7ff8bba00640 1 --2- 172.21.15.66:0/3715427913 >> v2:172.21.15.66:3300/0 conn(0x7ff8b4161200 0x7ff8b40aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7ff8bba00640 1 --2- 172.21.15.66:0/3715427913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8b4160e20 0x7ff8b40af9c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7ff8bba00640 1 -- 172.21.15.66:0/3715427913 >> 172.21.15.66:0/3715427913 conn(0x7ff8b4098a20 msgr2=0x7ff8b40cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7ff8bba00640 1 -- 172.21.15.66:0/3715427913 shutdown_connections 2024-03-30T00:40:19.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7ff8bba00640 1 -- 172.21.15.66:0/3715427913 wait complete. 2024-03-30T00:40:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7f9a5f200640 1 Processor -- start 2024-03-30T00:40:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7f9a5f200640 1 -- start start 2024-03-30T00:40:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5f200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a6006cef0 0x7f9a60155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7ff8bba00640 1 Processor -- start 2024-03-30T00:40:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.501+0000 7ff8bba00640 1 -- start start 2024-03-30T00:40:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a600c18c0 0x7f9a600c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5f200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a60160ad0 0x7f9a601622e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a60155690 con 0x7f9a600c18c0 2024-03-30T00:40:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5f200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a600ca3b0 con 0x7f9a60160ad0 2024-03-30T00:40:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5f200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a600ca690 con 0x7f9a6006cef0 2024-03-30T00:40:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5e800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a60160ad0 0x7f9a601622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5e800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a60160ad0 0x7f9a601622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:34674/0 (socket says 172.21.15.66:34674) 2024-03-30T00:40:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5e800640 1 -- 172.21.15.66:0/4277933231 learned_addr learned my addr 172.21.15.66:0/4277933231 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8bba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8b40c24a0 0x7ff8b4159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8bba00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8b4160e20 0x7ff8b4149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5de00640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a6006cef0 0x7f9a60155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8bba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4161200 0x7ff8b40b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8bba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b4157ad0 con 0x7ff8b4161200 2024-03-30T00:40:19.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8bba00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b414a540 con 0x7ff8b4160e20 2024-03-30T00:40:19.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8bba00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b414a820 con 0x7ff8b40c24a0 2024-03-30T00:40:19.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5d400640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a600c18c0 0x7f9a600c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8b9000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8b40c24a0 0x7ff8b4159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5de00640 1 -- 172.21.15.66:0/4277933231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a60160ad0 msgr2=0x7f9a601622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5de00640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a60160ad0 0x7f9a601622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5de00640 1 -- 172.21.15.66:0/4277933231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a600c18c0 msgr2=0x7f9a600c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8b3e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8b4160e20 0x7ff8b4149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5de00640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a600c18c0 0x7f9a600c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5de00640 1 -- 172.21.15.66:0/4277933231 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c0094d0 con 0x7f9a6006cef0 2024-03-30T00:40:19.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7f9a5d400640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a600c18c0 0x7f9a600c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8b9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4161200 0x7ff8b40b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8b9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4161200 0x7ff8b40b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33996/0 (socket says 172.21.15.66:33996) 2024-03-30T00:40:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8b9a00640 1 -- 172.21.15.66:0/4258731392 learned_addr learned my addr 172.21.15.66:0/4258731392 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8b9a00640 1 -- 172.21.15.66:0/4258731392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8b40c24a0 msgr2=0x7ff8b4159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8b9a00640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8b40c24a0 0x7ff8b4159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8b9a00640 1 -- 172.21.15.66:0/4258731392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8b4160e20 msgr2=0x7ff8b4149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8b9a00640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8b4160e20 0x7ff8b4149cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.505+0000 7ff8b9a00640 1 -- 172.21.15.66:0/4258731392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac009420 con 0x7ff8b4161200 2024-03-30T00:40:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7f9a5de00640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a6006cef0 0x7f9a60155150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000c3e0 tx=0x7f9a5000c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7ff8b9a00640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4161200 0x7ff8b40b2320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400efc0 tx=0x7ff8a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7f9a56000640 1 -- 172.21.15.66:0/4277933231 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50011790 con 0x7f9a6006cef0 2024-03-30T00:40:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7f9a5f200640 1 -- 172.21.15.66:0/4277933231 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a600ca970 con 0x7f9a6006cef0 2024-03-30T00:40:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7f9a56000640 1 -- 172.21.15.66:0/4277933231 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a500118f0 con 0x7f9a6006cef0 2024-03-30T00:40:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7ff8b0c00640 1 -- 172.21.15.66:0/4258731392 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a400cd20 con 0x7ff8b4161200 2024-03-30T00:40:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7ff8bba00640 1 -- 172.21.15.66:0/4258731392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b414ab00 con 0x7ff8b4161200 2024-03-30T00:40:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7ff8bba00640 1 -- 172.21.15.66:0/4258731392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b414a1f0 con 0x7ff8b4161200 2024-03-30T00:40:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7f9a56000640 1 -- 172.21.15.66:0/4277933231 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50011ce0 con 0x7f9a6006cef0 2024-03-30T00:40:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7ff8b0c00640 1 -- 172.21.15.66:0/4258731392 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a400f040 con 0x7ff8b4161200 2024-03-30T00:40:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7ff8b0c00640 1 -- 172.21.15.66:0/4258731392 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4003be0 con 0x7ff8b4161200 2024-03-30T00:40:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7ff8bba00640 1 -- 172.21.15.66:0/4258731392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b4157f60 con 0x7ff8b4161200 2024-03-30T00:40:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.509+0000 7f9a5f200640 1 -- 172.21.15.66:0/4277933231 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a600caf30 con 0x7f9a6006cef0 2024-03-30T00:40:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.513+0000 7f9a56000640 1 -- 172.21.15.66:0/4277933231 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9a5001c460 con 0x7f9a6006cef0 2024-03-30T00:40:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.513+0000 7f9a56000640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a40066690 0x7f9a40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.513+0000 7f9a56000640 1 -- 172.21.15.66:0/4277933231 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9a50011e40 con 0x7f9a6006cef0 2024-03-30T00:40:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.513+0000 7f9a5f200640 1 -- 172.21.15.66:0/4277933231 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a600c5290 con 0x7f9a6006cef0 2024-03-30T00:40:19.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.517+0000 7f9a5d400640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a40066690 0x7f9a40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.517+0000 7f9a5d400640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a40066690 0x7f9a40068b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a60162290 tx=0x7f9a58009550 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.521+0000 7ff8b0c00640 1 -- 172.21.15.66:0/4258731392 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff8a4003d40 con 0x7ff8b4161200 2024-03-30T00:40:19.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.521+0000 7f9a56000640 1 -- 172.21.15.66:0/4277933231 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9a50050a90 con 0x7f9a6006cef0 2024-03-30T00:40:19.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.521+0000 7ff8b0c00640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8900666f0 0x7ff890068bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.521+0000 7ff8b0c00640 1 -- 172.21.15.66:0/4258731392 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff8a4085680 con 0x7ff8b4161200 2024-03-30T00:40:19.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.521+0000 7ff8b9000640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8900666f0 0x7ff890068bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.525+0000 7ff8b0c00640 1 -- 172.21.15.66:0/4258731392 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff8a4052790 con 0x7ff8b4161200 2024-03-30T00:40:19.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.529+0000 7ff8b9000640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8900666f0 0x7ff890068bb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0af000 tx=0x7ff8ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.537+0000 7f0460c00640 1 Processor -- start 2024-03-30T00:40:19.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.537+0000 7f0460c00640 1 -- start start 2024-03-30T00:40:19.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.541+0000 7f7f21800640 1 Processor -- start 2024-03-30T00:40:19.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.541+0000 7f7f21800640 1 -- start start 2024-03-30T00:40:19.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.541+0000 7f0460c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f045404e730 0x7f0454033990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.541+0000 7f0460c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0454050cf0 con 0x7f0454038610 2024-03-30T00:40:19.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.541+0000 7f0460c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0454050fd0 con 0x7f0454039e30 2024-03-30T00:40:19.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.541+0000 7f0460c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04540512b0 con 0x7f045404e730 2024-03-30T00:40:19.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.541+0000 7f045b400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f045404e730 0x7f0454033990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.541+0000 7f045b400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f045404e730 0x7f0454033990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:34690/0 (socket says 172.21.15.66:34690) 2024-03-30T00:40:19.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.541+0000 7f045b400640 1 -- 172.21.15.66:0/970775520 learned_addr learned my addr 172.21.15.66:0/970775520 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f045b400640 1 -- 172.21.15.66:0/970775520 >> v1:172.21.15.156:6789/0 conn(0x7f0454039e30 legacy=0x7f045403a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f7f21800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7f1404fa50 0x7f7f1404fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f7f1b400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7f140df360 0x7f7f14007160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:43262/0 (socket says 172.21.15.66:43262) 2024-03-30T00:40:19.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f7f21800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7f1404f590 0x7f7f14037ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f7f21800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f14005e70 con 0x7f7f140df360 2024-03-30T00:40:19.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f7f21800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f14007870 con 0x7f7f1404fa50 2024-03-30T00:40:19.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f7f21800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f14006cd0 con 0x7f7f1404f590 2024-03-30T00:40:19.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f7f1b400640 1 -- 172.21.15.66:0/3154561011 learned_addr learned my addr 172.21.15.66:0/3154561011 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:19.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f7f20e00640 1 --2- 172.21.15.66:0/3154561011 >> v2:172.21.15.156:3300/0 conn(0x7f7f1404f590 0x7f7f14037ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f7f1be00640 1 --2- 172.21.15.66:0/3154561011 >> v2:172.21.15.66:3300/0 conn(0x7f7f1404fa50 0x7f7f1404fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f045b400640 1 -- 172.21.15.66:0/970775520 >> v1:172.21.15.66:6789/0 conn(0x7f0454038610 legacy=0x7f04540500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.545+0000 7f045b400640 1 -- 172.21.15.66:0/970775520 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045404f5c0 con 0x7f045404e730 2024-03-30T00:40:19.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f7f1aa00640 1 -- 172.21.15.66:0/3154561011 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190205278 0 0) 0x7f7f14005e70 con 0x7f7f140df360 2024-03-30T00:40:19.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f7f1aa00640 1 -- 172.21.15.66:0/3154561011 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f04003430 con 0x7f7f140df360 2024-03-30T00:40:19.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f045b400640 1 --2- 172.21.15.66:0/970775520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f045404e730 0x7f0454033990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f045c02c890 tx=0x7f045c0960d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe2a46ec1a875cb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f045a000640 1 -- 172.21.15.66:0/970775520 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c11ec70 con 0x7f045404e730 2024-03-30T00:40:19.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f045a000640 1 -- 172.21.15.66:0/970775520 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045c11edd0 con 0x7f045404e730 2024-03-30T00:40:19.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f045a000640 1 -- 172.21.15.66:0/970775520 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c126640 con 0x7f045404e730 2024-03-30T00:40:19.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f0460c00640 1 -- 172.21.15.66:0/970775520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f045404e730 msgr2=0x7f0454033990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f0460c00640 1 --2- 172.21.15.66:0/970775520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f045404e730 0x7f0454033990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f045c02c890 tx=0x7f045c0960d0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f0460c00640 1 -- 172.21.15.66:0/970775520 shutdown_connections 2024-03-30T00:40:19.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f0460c00640 1 --2- 172.21.15.66:0/970775520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f045404e730 0x7f0454033990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f0460c00640 1 -- 172.21.15.66:0/970775520 >> 172.21.15.66:0/970775520 conn(0x7f0454009f90 msgr2=0x7f045400a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f7f20e00640 1 -- 172.21.15.66:0/3154561011 >> v2:172.21.15.66:3300/0 conn(0x7f7f1404fa50 msgr2=0x7f7f1404fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:19.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f7f20e00640 1 --2- 172.21.15.66:0/3154561011 >> v2:172.21.15.66:3300/0 conn(0x7f7f1404fa50 0x7f7f1404fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f7f20e00640 1 -- 172.21.15.66:0/3154561011 >> v1:172.21.15.66:6789/0 conn(0x7f7f140df360 legacy=0x7f7f14007160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.549+0000 7f7f20e00640 1 -- 172.21.15.66:0/3154561011 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f14054360 con 0x7f7f1404f590 2024-03-30T00:40:19.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.553+0000 7f0460c00640 1 -- 172.21.15.66:0/970775520 shutdown_connections 2024-03-30T00:40:19.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.553+0000 7f0460c00640 1 -- 172.21.15.66:0/970775520 wait complete. 2024-03-30T00:40:19.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.553+0000 7f7f20e00640 1 --2- 172.21.15.66:0/3154561011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f1404f590 0x7f7f14037ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7f14054790 tx=0x7f7f0801c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a11c1cd9f4e655d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.553+0000 7f7f1aa00640 1 -- 172.21.15.66:0/3154561011 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f080ab680 con 0x7f7f1404f590 2024-03-30T00:40:19.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.553+0000 7f7f1aa00640 1 -- 172.21.15.66:0/3154561011 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f080ab7e0 con 0x7f7f1404f590 2024-03-30T00:40:19.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.553+0000 7f7f1aa00640 1 -- 172.21.15.66:0/3154561011 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f080abb70 con 0x7f7f1404f590 2024-03-30T00:40:19.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.553+0000 7f7f21800640 1 -- 172.21.15.66:0/3154561011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f1404f590 msgr2=0x7f7f14037ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.553+0000 7f7f21800640 1 --2- 172.21.15.66:0/3154561011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f1404f590 0x7f7f14037ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7f14054790 tx=0x7f7f0801c4c0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f7f21800640 1 -- 172.21.15.66:0/3154561011 shutdown_connections 2024-03-30T00:40:19.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f7f21800640 1 --2- 172.21.15.66:0/3154561011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f1404f590 0x7f7f14037ff0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f7f21800640 1 --2- 172.21.15.66:0/3154561011 >> v2:172.21.15.66:3300/0 conn(0x7f7f1404fa50 0x7f7f1404fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f7f21800640 1 -- 172.21.15.66:0/3154561011 >> 172.21.15.66:0/3154561011 conn(0x7f7f14009f90 msgr2=0x7f7f1400a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f7f21800640 1 -- 172.21.15.66:0/3154561011 shutdown_connections 2024-03-30T00:40:19.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f7f21800640 1 -- 172.21.15.66:0/3154561011 wait complete. 2024-03-30T00:40:19.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f0460c00640 1 Processor -- start 2024-03-30T00:40:19.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f0460c00640 1 -- start start 2024-03-30T00:40:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f0460c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0454038610 0x7f04540e17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f0460c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0454039e30 0x7f04540db450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f0460c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0454053740 0x7f04540db990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f0460c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04540cca20 con 0x7f0454039e30 2024-03-30T00:40:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f0460c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04540d3970 con 0x7f0454053740 2024-03-30T00:40:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.557+0000 7f0460c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04540e2030 con 0x7f0454038610 2024-03-30T00:40:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f21800640 1 Processor -- start 2024-03-30T00:40:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f21800640 1 -- start start 2024-03-30T00:40:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0454039e30 0x7f04540db450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0454039e30 0x7f04540db450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34018/0 (socket says 172.21.15.66:34018) 2024-03-30T00:40:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045aa00640 1 -- 172.21.15.66:0/2795669289 learned_addr learned my addr 172.21.15.66:0/2795669289 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045be00640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0454053740 0x7f04540db990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045b400640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0454038610 0x7f04540e17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045be00640 1 -- 172.21.15.66:0/2795669289 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0454038610 msgr2=0x7f04540e17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f21800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f1404f590 0x7f7f140d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f21800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f1404fa50 0x7f7f1404abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045be00640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0454038610 0x7f04540e17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045be00640 1 -- 172.21.15.66:0/2795669289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0454039e30 msgr2=0x7f04540db450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045be00640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0454039e30 0x7f04540db450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045be00640 1 -- 172.21.15.66:0/2795669289 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c04efd0 con 0x7f0454053740 2024-03-30T00:40:19.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f21800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140df360 0x7f7f14049c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045b400640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0454038610 0x7f04540e17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:19.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f21800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1404bb70 con 0x7f7f140df360 2024-03-30T00:40:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f21800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140c9960 con 0x7f7f1404fa50 2024-03-30T00:40:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f21800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140d1560 con 0x7f7f1404f590 2024-03-30T00:40:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f1b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f1404fa50 0x7f7f1404abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f1b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f1404fa50 0x7f7f1404abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:34726/0 (socket says 172.21.15.66:34726) 2024-03-30T00:40:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f1b400640 1 -- 172.21.15.66:0/3413069019 learned_addr learned my addr 172.21.15.66:0/3413069019 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f045be00640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0454053740 0x7f04540db990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f045000c410 tx=0x7f045000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f1be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f1404f590 0x7f7f140d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f1b400640 1 -- 172.21.15.66:0/3413069019 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f1404f590 msgr2=0x7f7f140d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f7f20e00640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140df360 0x7f7f14049c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f044f400640 1 -- 172.21.15.66:0/2795669289 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450005230 con 0x7f0454053740 2024-03-30T00:40:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f044f400640 1 -- 172.21.15.66:0/2795669289 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0450005390 con 0x7f0454053740 2024-03-30T00:40:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f044f400640 1 -- 172.21.15.66:0/2795669289 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450011870 con 0x7f0454053740 2024-03-30T00:40:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.561+0000 7f0460c00640 1 -- 172.21.15.66:0/2795669289 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04540e2310 con 0x7f0454053740 2024-03-30T00:40:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f0460c00640 1 -- 172.21.15.66:0/2795669289 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04540e1d20 con 0x7f0454053740 2024-03-30T00:40:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f7f1b400640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f1404f590 0x7f7f140d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f7f1b400640 1 -- 172.21.15.66:0/3413069019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140df360 msgr2=0x7f7f14049c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f7f1b400640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140df360 0x7f7f14049c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f7f1b400640 1 -- 172.21.15.66:0/3413069019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f08009420 con 0x7f7f1404fa50 2024-03-30T00:40:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f7f20e00640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140df360 0x7f7f14049c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f7f1b400640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f1404fa50 0x7f7f1404abe0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c068ee0 tx=0x7f7f1c0733b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f7f0fe00640 1 -- 172.21.15.66:0/3413069019 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0734e0 con 0x7f7f1404fa50 2024-03-30T00:40:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f7f21800640 1 -- 172.21.15.66:0/3413069019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f140d1840 con 0x7f7f1404fa50 2024-03-30T00:40:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f7f21800640 1 -- 172.21.15.66:0/3413069019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f140d1e00 con 0x7f7f1404fa50 2024-03-30T00:40:19.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f0460c00640 1 -- 172.21.15.66:0/2795669289 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04540d3ae0 con 0x7f0454053740 2024-03-30T00:40:19.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f044f400640 1 -- 172.21.15.66:0/2795669289 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f04500119d0 con 0x7f0454053740 2024-03-30T00:40:19.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.569+0000 7f044f400640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f043c066690 0x7f043c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.565+0000 7f7f0fe00640 1 -- 172.21.15.66:0/3413069019 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c073640 con 0x7f7f1404fa50 2024-03-30T00:40:19.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.569+0000 7f7f0fe00640 1 -- 172.21.15.66:0/3413069019 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0734e0 con 0x7f7f1404fa50 2024-03-30T00:40:19.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.569+0000 7f7f1be00640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f1404f590 0x7f7f140d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:19.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.569+0000 7f045b400640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f043c066690 0x7f043c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.569+0000 7f044f400640 1 -- 172.21.15.66:0/2795669289 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f04500546c0 con 0x7f0454053740 2024-03-30T00:40:19.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.569+0000 7f045b400640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f043c066690 0x7f043c068b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f045400b360 tx=0x7f045c122000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.569+0000 7f7f21800640 1 -- 172.21.15.66:0/3413069019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f14005370 con 0x7f7f1404fa50 2024-03-30T00:40:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.573+0000 7f044f400640 1 -- 172.21.15.66:0/2795669289 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0450051190 con 0x7f0454053740 2024-03-30T00:40:19.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.573+0000 7f7f0fe00640 1 -- 172.21.15.66:0/3413069019 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7f1c0721c0 con 0x7f7f1404fa50 2024-03-30T00:40:19.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.577+0000 7f7f0fe00640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7efc066750 0x7f7efc068c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:19.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.577+0000 7f7f0fe00640 1 -- 172.21.15.66:0/3413069019 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7f1c0fda00 con 0x7f7f1404fa50 2024-03-30T00:40:19.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.577+0000 7f7f1be00640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7efc066750 0x7f7efc068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:19.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.577+0000 7f7f1be00640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7efc066750 0x7f7efc068c10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000afd0 tx=0x7f7f10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:19.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.581+0000 7f7f0fe00640 1 -- 172.21.15.66:0/3413069019 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7f1c0ca9f0 con 0x7f7f1404fa50 2024-03-30T00:40:19.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.773+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f81780af340 con 0x7f81780cd340 2024-03-30T00:40:19.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.773+0000 7f816f400640 1 -- 172.21.15.66:0/4157890182 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8164054850 con 0x7f81780cd340 2024-03-30T00:40:19.785 INFO:teuthology.orchestra.run.smithi066.stdout:55834574859 2024-03-30T00:40:19.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.777+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8158066760 msgr2=0x7f8158068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.777+0000 7f817de00640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8158066760 0x7f8158068c20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f81700012f0 tx=0x7f817000a040 comp rx=0 tx=0).stop 2024-03-30T00:40:19.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.777+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81780cd340 msgr2=0x7f8178097860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.777+0000 7f817de00640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81780cd340 0x7f8178097860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f816400b410 tx=0x7f816400b8d0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.781+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 shutdown_connections 2024-03-30T00:40:19.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.781+0000 7f817de00640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81780cd340 0x7f8178097860 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.781+0000 7f817de00640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81780b1f60 0x7f817815aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.781+0000 7f817de00640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8158066760 0x7f8158068c20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.781+0000 7f817de00640 1 --2- 172.21.15.66:0/4157890182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81780af590 0x7f817814a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.781+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 >> 172.21.15.66:0/4157890182 conn(0x7f8178070410 msgr2=0x7f817806d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.781+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 shutdown_connections 2024-03-30T00:40:19.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.781+0000 7f817de00640 1 -- 172.21.15.66:0/4157890182 wait complete. 2024-03-30T00:40:19.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.1 2024-03-30T00:40:19.821 DEBUG:teuthology.parallel:result is None 2024-03-30T00:40:19.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.825+0000 7f73b4c00640 1 -- 172.21.15.66:0/3360750218 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f73b01643d0 con 0x7f7394005190 2024-03-30T00:40:19.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.825+0000 7f73a7400640 1 -- 172.21.15.66:0/3360750218 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f73a0054e60 con 0x7f7394005190 2024-03-30T00:40:19.836 INFO:teuthology.orchestra.run.smithi066.stdout:55834574859 2024-03-30T00:40:19.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.829+0000 7f73b4c00640 1 -- 172.21.15.66:0/3360750218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73900666e0 msgr2=0x7f7390068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.829+0000 7f73b4c00640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73900666e0 0x7f7390068ba0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f73b00c82b0 tx=0x7f739c008040 comp rx=0 tx=0).stop 2024-03-30T00:40:19.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.829+0000 7f73b4c00640 1 -- 172.21.15.66:0/3360750218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7394005190 msgr2=0x7f73b0089fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.829+0000 7f73b4c00640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7394005190 0x7f73b0089fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f73a0010890 tx=0x7f73a0010d50 comp rx=0 tx=0).stop 2024-03-30T00:40:19.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.829+0000 7f73b4c00640 1 -- 172.21.15.66:0/3360750218 shutdown_connections 2024-03-30T00:40:19.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.829+0000 7f73b4c00640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73900666e0 0x7f7390068ba0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.829+0000 7f73b4c00640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b00c76f0 0x7f73b00c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.829+0000 7f73b4c00640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f73b0097570 0x7f73b00c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.829+0000 7f73b4c00640 1 --2- 172.21.15.66:0/3360750218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7394005190 0x7f73b0089fb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.829+0000 7f73b4c00640 1 -- 172.21.15.66:0/3360750218 >> 172.21.15.66:0/3360750218 conn(0x7f73b006a190 msgr2=0x7f73b0070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.833+0000 7f73b4c00640 1 -- 172.21.15.66:0/3360750218 shutdown_connections 2024-03-30T00:40:19.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.833+0000 7f73b4c00640 1 -- 172.21.15.66:0/3360750218 wait complete. 2024-03-30T00:40:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.837+0000 7fc697400640 1 -- 172.21.15.66:0/1985691916 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc6ac14bf00 con 0x7fc6ac0af970 2024-03-30T00:40:19.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.845+0000 7fc6a2000640 1 -- 172.21.15.66:0/1985691916 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc6ac14bf00 con 0x7fc6ac0af970 2024-03-30T00:40:19.853 INFO:teuthology.orchestra.run.smithi066.stdout:55834574858 2024-03-30T00:40:19.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.845+0000 7fc697400640 1 -- 172.21.15.66:0/1985691916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc68c066690 msgr2=0x7fc68c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.845+0000 7fc697400640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc68c066690 0x7fc68c068b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac14c0c0 tx=0x7fc6a400c040 comp rx=0 tx=0).stop 2024-03-30T00:40:19.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.845+0000 7fc697400640 1 -- 172.21.15.66:0/1985691916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af970 msgr2=0x7fc6ac097a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.845+0000 7fc697400640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af970 0x7fc6ac097a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c520 tx=0x7fc69c00c9e0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.845+0000 7fc697400640 1 -- 172.21.15.66:0/1985691916 shutdown_connections 2024-03-30T00:40:19.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.845+0000 7fc697400640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc68c066690 0x7fc68c068b50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.845+0000 7fc697400640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ac0b2130 0x7fc6ac150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.845+0000 7fc697400640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6ac0af970 0x7fc6ac097a60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.845+0000 7fc697400640 1 --2- 172.21.15.66:0/1985691916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6ac0af590 0x7fc6ac14e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.845+0000 7fc697400640 1 -- 172.21.15.66:0/1985691916 >> 172.21.15.66:0/1985691916 conn(0x7fc6ac070410 msgr2=0x7fc6ac15b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.849+0000 7fc697400640 1 -- 172.21.15.66:0/1985691916 shutdown_connections 2024-03-30T00:40:19.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.849+0000 7fc697400640 1 -- 172.21.15.66:0/1985691916 wait complete. 2024-03-30T00:40:19.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.849+0000 7f2443200640 1 -- 172.21.15.66:0/3056203998 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f241c005510 con 0x7f244415d070 2024-03-30T00:40:19.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.849+0000 7f243a000640 1 -- 172.21.15.66:0/3056203998 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f24300eed90 con 0x7f244415d070 2024-03-30T00:40:19.860 INFO:teuthology.orchestra.run.smithi066.stdout:55834574858 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 -- 172.21.15.66:0/3056203998 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2424066690 msgr2=0x7f2424068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2424066690 0x7f2424068b50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2434004660 tx=0x7f2434008180 comp rx=0 tx=0).stop 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 -- 172.21.15.66:0/3056203998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f244415d070 msgr2=0x7f24440c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f244415d070 0x7f24440c5cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f243001a030 tx=0x7f24300191a0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 -- 172.21.15.66:0/3056203998 shutdown_connections 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2424066690 0x7f2424068b50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f244415d070 0x7f24440c5cc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24440aebb0 0x7f24441f9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 --2- 172.21.15.66:0/3056203998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2444071320 0x7f24440c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 -- 172.21.15.66:0/3056203998 >> 172.21.15.66:0/3056203998 conn(0x7f24440ce1e0 msgr2=0x7f244406d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 -- 172.21.15.66:0/3056203998 shutdown_connections 2024-03-30T00:40:19.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.853+0000 7f2443200640 1 -- 172.21.15.66:0/3056203998 wait complete. 2024-03-30T00:40:19.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.0 2024-03-30T00:40:19.865 DEBUG:teuthology.parallel:result is None 2024-03-30T00:40:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.865+0000 7f9a5f200640 1 -- 172.21.15.66:0/4277933231 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9a60073e00 con 0x7f9a6006cef0 2024-03-30T00:40:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.865+0000 7f9a56000640 1 -- 172.21.15.66:0/4277933231 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9a500540b0 con 0x7f9a6006cef0 2024-03-30T00:40:19.875 INFO:teuthology.orchestra.run.smithi066.stdout:55834574858 2024-03-30T00:40:19.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.865+0000 7ff8bba00640 1 -- 172.21.15.66:0/4258731392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff8b4157f60 con 0x7ff8b4161200 2024-03-30T00:40:19.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.865+0000 7ff8b0c00640 1 -- 172.21.15.66:0/4258731392 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff8b4157f60 con 0x7ff8b4161200 2024-03-30T00:40:19.876 INFO:teuthology.orchestra.run.smithi066.stdout:55834574859 2024-03-30T00:40:19.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 -- 172.21.15.66:0/4277933231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a40066690 msgr2=0x7f9a40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a40066690 0x7f9a40068b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a60162290 tx=0x7f9a58009550 comp rx=0 tx=0).stop 2024-03-30T00:40:19.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 -- 172.21.15.66:0/4277933231 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a6006cef0 msgr2=0x7f9a60155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a6006cef0 0x7f9a60155150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000c3e0 tx=0x7f9a5000c8a0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 -- 172.21.15.66:0/4277933231 shutdown_connections 2024-03-30T00:40:19.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a60160ad0 0x7f9a601622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a40066690 0x7f9a40068b50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a600c18c0 0x7f9a600c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 --2- 172.21.15.66:0/4277933231 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a6006cef0 0x7f9a60155150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 -- 172.21.15.66:0/4277933231 >> 172.21.15.66:0/4277933231 conn(0x7f9a6006a190 msgr2=0x7f9a600ab9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 -- 172.21.15.66:0/4277933231 shutdown_connections 2024-03-30T00:40:19.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7f9a5f200640 1 -- 172.21.15.66:0/4277933231 wait complete. 2024-03-30T00:40:19.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 -- 172.21.15.66:0/4258731392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8900666f0 msgr2=0x7ff890068bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8900666f0 0x7ff890068bb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0af000 tx=0x7ff8ac008040 comp rx=0 tx=0).stop 2024-03-30T00:40:19.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 -- 172.21.15.66:0/4258731392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4161200 msgr2=0x7ff8b40b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4161200 0x7ff8b40b2320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400efc0 tx=0x7ff8a400c490 comp rx=0 tx=0).stop 2024-03-30T00:40:19.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 -- 172.21.15.66:0/4258731392 shutdown_connections 2024-03-30T00:40:19.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8900666f0 0x7ff890068bb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4161200 0x7ff8b40b2320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8b4160e20 0x7ff8b4149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 --2- 172.21.15.66:0/4258731392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8b40c24a0 0x7ff8b4159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 -- 172.21.15.66:0/4258731392 >> 172.21.15.66:0/4258731392 conn(0x7ff8b4098a20 msgr2=0x7ff8b406d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 -- 172.21.15.66:0/4258731392 shutdown_connections 2024-03-30T00:40:19.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.869+0000 7ff8bba00640 1 -- 172.21.15.66:0/4258731392 wait complete. 2024-03-30T00:40:19.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.5 2024-03-30T00:40:19.889 DEBUG:teuthology.parallel:result is None 2024-03-30T00:40:19.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.4 2024-03-30T00:40:19.894 DEBUG:teuthology.parallel:result is None 2024-03-30T00:40:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.893+0000 7f0460c00640 1 -- 172.21.15.66:0/2795669289 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f04540d3ab0 con 0x7f0454053740 2024-03-30T00:40:19.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.893+0000 7f044f400640 1 -- 172.21.15.66:0/2795669289 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0450017070 con 0x7f0454053740 2024-03-30T00:40:19.904 INFO:teuthology.orchestra.run.smithi066.stdout:55834574858 2024-03-30T00:40:19.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 -- 172.21.15.66:0/2795669289 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f043c066690 msgr2=0x7f043c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f043c066690 0x7f043c068b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f045400b360 tx=0x7f045c122000 comp rx=0 tx=0).stop 2024-03-30T00:40:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 -- 172.21.15.66:0/2795669289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0454053740 msgr2=0x7f04540db990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0454053740 0x7f04540db990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f045000c410 tx=0x7f045000c8d0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 -- 172.21.15.66:0/2795669289 shutdown_connections 2024-03-30T00:40:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0454053740 0x7f04540db990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f043c066690 0x7f043c068b50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0454039e30 0x7f04540db450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 --2- 172.21.15.66:0/2795669289 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0454038610 0x7f04540e17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 -- 172.21.15.66:0/2795669289 >> 172.21.15.66:0/2795669289 conn(0x7f0454009f90 msgr2=0x7f045404c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 -- 172.21.15.66:0/2795669289 shutdown_connections 2024-03-30T00:40:19.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.897+0000 7f0460c00640 1 -- 172.21.15.66:0/2795669289 wait complete. 2024-03-30T00:40:19.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.901+0000 7f7f21800640 1 -- 172.21.15.66:0/3413069019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7f1404f160 con 0x7f7f1404fa50 2024-03-30T00:40:19.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.901+0000 7f7f0fe00640 1 -- 172.21.15.66:0/3413069019 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f1c0ca9f0 con 0x7f7f1404fa50 2024-03-30T00:40:19.910 INFO:teuthology.orchestra.run.smithi066.stdout:55834574860 2024-03-30T00:40:19.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.3 2024-03-30T00:40:19.911 DEBUG:teuthology.parallel:result is None 2024-03-30T00:40:19.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.901+0000 7f7f21800640 1 -- 172.21.15.66:0/3413069019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7efc066750 msgr2=0x7f7efc068c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.901+0000 7f7f21800640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7efc066750 0x7f7efc068c10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000afd0 tx=0x7f7f10008040 comp rx=0 tx=0).stop 2024-03-30T00:40:19.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.901+0000 7f7f21800640 1 -- 172.21.15.66:0/3413069019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f1404fa50 msgr2=0x7f7f1404abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:19.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.901+0000 7f7f21800640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f1404fa50 0x7f7f1404abe0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c068ee0 tx=0x7f7f1c0733b0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.901+0000 7f7f21800640 1 -- 172.21.15.66:0/3413069019 shutdown_connections 2024-03-30T00:40:19.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.901+0000 7f7f21800640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7efc066750 0x7f7efc068c10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.901+0000 7f7f21800640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140df360 0x7f7f14049c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.901+0000 7f7f21800640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f1404fa50 0x7f7f1404abe0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.901+0000 7f7f21800640 1 --2- 172.21.15.66:0/3413069019 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f1404f590 0x7f7f140d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:19.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.905+0000 7f7f21800640 1 -- 172.21.15.66:0/3413069019 >> 172.21.15.66:0/3413069019 conn(0x7f7f14009f90 msgr2=0x7f7f140e19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:19.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.905+0000 7f7f21800640 1 -- 172.21.15.66:0/3413069019 shutdown_connections 2024-03-30T00:40:19.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:19.905+0000 7f7f21800640 1 -- 172.21.15.66:0/3413069019 wait complete. 2024-03-30T00:40:19.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.2 2024-03-30T00:40:19.914 DEBUG:teuthology.parallel:result is None 2024-03-30T00:40:19.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.7 2024-03-30T00:40:19.930 DEBUG:teuthology.parallel:result is None 2024-03-30T00:40:19.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.6 2024-03-30T00:40:19.936 DEBUG:teuthology.parallel:result is None 2024-03-30T00:40:19.936 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-30T00:40:19.936 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:40:20.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.105+0000 7f1510c00640 1 Processor -- start 2024-03-30T00:40:20.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.105+0000 7f1510c00640 1 -- start start 2024-03-30T00:40:20.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1510c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f150c0cec30 0x7f150c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1510c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150c0cd3e0 con 0x7f150c0cda90 2024-03-30T00:40:20.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1510c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150c15a990 con 0x7f150c0cde70 2024-03-30T00:40:20.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1510c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c0b1b80 con 0x7f150c0cec30 2024-03-30T00:40:20.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1509e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f150c0cda90 0x7f150c0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:43276/0 (socket says 172.21.15.66:43276) 2024-03-30T00:40:20.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1509e00640 1 -- 172.21.15.66:0/2489537758 learned_addr learned my addr 172.21.15.66:0/2489537758 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:20.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1503e00640 1 --2- 172.21.15.66:0/2489537758 >> v2:172.21.15.156:3300/0 conn(0x7f150c0cec30 0x7f150c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1509400640 1 -- 172.21.15.66:0/2489537758 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568861043 0 0) 0x7f150c0cd3e0 con 0x7f150c0cda90 2024-03-30T00:40:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1509400640 1 -- 172.21.15.66:0/2489537758 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f0005740 con 0x7f150c0cda90 2024-03-30T00:40:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1509400640 1 -- 172.21.15.66:0/2489537758 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2423391259 0 0) 0x7f14f0005740 con 0x7f150c0cda90 2024-03-30T00:40:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1503e00640 1 -- 172.21.15.66:0/2489537758 >> v1:172.21.15.156:6789/0 conn(0x7f150c0cde70 legacy=0x7f150c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1503e00640 1 -- 172.21.15.66:0/2489537758 >> v1:172.21.15.66:6789/0 conn(0x7f150c0cda90 legacy=0x7f150c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.109+0000 7f1503e00640 1 -- 172.21.15.66:0/2489537758 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c0b11e0 con 0x7f150c0cec30 2024-03-30T00:40:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1503e00640 1 --2- 172.21.15.66:0/2489537758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f150c0cec30 0x7f150c159c60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f14f40097c0 tx=0x7f14f401c1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59aa135ff33f2ab server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1509400640 1 -- 172.21.15.66:0/2489537758 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc001680 con 0x7f150c0cec30 2024-03-30T00:40:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1509400640 1 -- 172.21.15.66:0/2489537758 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f40a7050 con 0x7f150c0cec30 2024-03-30T00:40:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1509400640 1 -- 172.21.15.66:0/2489537758 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f40ab600 con 0x7f150c0cec30 2024-03-30T00:40:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1510c00640 1 -- 172.21.15.66:0/2489537758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f150c0cec30 msgr2=0x7f150c159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1510c00640 1 --2- 172.21.15.66:0/2489537758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f150c0cec30 0x7f150c159c60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f14f40097c0 tx=0x7f14f401c1e0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1510c00640 1 -- 172.21.15.66:0/2489537758 shutdown_connections 2024-03-30T00:40:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1510c00640 1 --2- 172.21.15.66:0/2489537758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f150c0cec30 0x7f150c159c60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1510c00640 1 -- 172.21.15.66:0/2489537758 >> 172.21.15.66:0/2489537758 conn(0x7f150c069f60 msgr2=0x7f150c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1510c00640 1 -- 172.21.15.66:0/2489537758 shutdown_connections 2024-03-30T00:40:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1510c00640 1 -- 172.21.15.66:0/2489537758 wait complete. 2024-03-30T00:40:20.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1510c00640 1 Processor -- start 2024-03-30T00:40:20.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.113+0000 7f1510c00640 1 -- start start 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1510c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c0cda90 0x7f150c06fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1510c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f150c0cde70 0x7f150c0ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1510c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f150c0cec30 0x7f150c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1510c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c162720 con 0x7f150c0cda90 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1510c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c0afa50 con 0x7f150c0cde70 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1510c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c158aa0 con 0x7f150c0cec30 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1509e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c0cda90 0x7f150c06fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1509e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c0cda90 0x7f150c06fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34032/0 (socket says 172.21.15.66:34032) 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1509e00640 1 -- 172.21.15.66:0/2564605377 learned_addr learned my addr 172.21.15.66:0/2564605377 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1503e00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f150c0cde70 0x7f150c0ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f150a800640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f150c0cec30 0x7f150c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1509e00640 1 -- 172.21.15.66:0/2564605377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f150c0cec30 msgr2=0x7f150c155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1509e00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f150c0cec30 0x7f150c155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1509e00640 1 -- 172.21.15.66:0/2564605377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f150c0cde70 msgr2=0x7f150c0ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1509e00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f150c0cde70 0x7f150c0ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1509e00640 1 -- 172.21.15.66:0/2564605377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f4009420 con 0x7f150c0cda90 2024-03-30T00:40:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f150a800640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f150c0cec30 0x7f150c155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1509e00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c0cda90 0x7f150c06fed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f14fc007fd0 tx=0x7f14fc00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:20.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1502000640 1 -- 172.21.15.66:0/2564605377 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc0072d0 con 0x7f150c0cda90 2024-03-30T00:40:20.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1502000640 1 -- 172.21.15.66:0/2564605377 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc007450 con 0x7f150c0cda90 2024-03-30T00:40:20.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1510c00640 1 -- 172.21.15.66:0/2564605377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c158d80 con 0x7f150c0cda90 2024-03-30T00:40:20.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1502000640 1 -- 172.21.15.66:0/2564605377 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc0047f0 con 0x7f150c0cda90 2024-03-30T00:40:20.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1510c00640 1 -- 172.21.15.66:0/2564605377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c159340 con 0x7f150c0cda90 2024-03-30T00:40:20.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1510c00640 1 -- 172.21.15.66:0/2564605377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c066c30 con 0x7f150c0cda90 2024-03-30T00:40:20.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.117+0000 7f1502000640 1 -- 172.21.15.66:0/2564605377 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f14fc004950 con 0x7f150c0cda90 2024-03-30T00:40:20.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.121+0000 7f1502000640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f14e8066690 0x7f14e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.121+0000 7f1502000640 1 -- 172.21.15.66:0/2564605377 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f14fc083c30 con 0x7f150c0cda90 2024-03-30T00:40:20.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.121+0000 7f1503e00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f14e8066690 0x7f14e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.121+0000 7f1503e00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f14e8066690 0x7f14e8068b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14f40ad000 tx=0x7f14f40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:20.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.125+0000 7f1502000640 1 -- 172.21.15.66:0/2564605377 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f14fc050b80 con 0x7f150c0cda90 2024-03-30T00:40:20.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.285+0000 7f1510c00640 1 -- 172.21.15.66:0/2564605377 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f150c0b3170 con 0x7f14e8066690 2024-03-30T00:40:20.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.297+0000 7f1502000640 1 -- 172.21.15.66:0/2564605377 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345229 (secure 0 0 0) 0x7f150c0b3170 con 0x7f14e8066690 2024-03-30T00:40:20.308 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:20.310 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 -- 172.21.15.66:0/2564605377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f14e8066690 msgr2=0x7f14e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f14e8066690 0x7f14e8068b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14f40ad000 tx=0x7f14f40a6000 comp rx=0 tx=0).stop 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 -- 172.21.15.66:0/2564605377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c0cda90 msgr2=0x7f150c06fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c0cda90 0x7f150c06fed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f14fc007fd0 tx=0x7f14fc00cbe0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 -- 172.21.15.66:0/2564605377 shutdown_connections 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f150c0cec30 0x7f150c155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f150c0cde70 0x7f150c0ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f14e8066690 0x7f14e8068b50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 --2- 172.21.15.66:0/2564605377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c0cda90 0x7f150c06fed0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 -- 172.21.15.66:0/2564605377 >> 172.21.15.66:0/2564605377 conn(0x7f150c069f60 msgr2=0x7f150c0c4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 -- 172.21.15.66:0/2564605377 shutdown_connections 2024-03-30T00:40:20.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.301+0000 7f1510c00640 1 -- 172.21.15.66:0/2564605377 wait complete. 2024-03-30T00:40:20.329 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-03-30T00:40:18.856621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235864,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738856,"statfs":{"total":773094113280,"available":772852588544,"internally_reserved":0,"allocated":3383296,"data_stored":2025532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002905"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184484+0000","last_change":"2024-03-30T00:39:50.768525+0000","last_active":"2024-03-30T00:39:55.184484+0000","last_peered":"2024-03-30T00:39:55.184484+0000","last_clean":"2024-03-30T00:39:55.184484+0000","last_became_active":"2024-03-30T00:39:50.768196+0000","last_became_peered":"2024-03-30T00:39:50.768196+0000","last_unstale":"2024-03-30T00:39:55.184484+0000","last_undegraded":"2024-03-30T00:39:55.184484+0000","last_fullsized":"2024-03-30T00:39:55.184484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:26.877416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168811+0000","last_change":"2024-03-30T00:39:51.184250+0000","last_active":"2024-03-30T00:39:55.168811+0000","last_peered":"2024-03-30T00:39:55.168811+0000","last_clean":"2024-03-30T00:39:55.168811+0000","last_became_active":"2024-03-30T00:39:51.184025+0000","last_became_peered":"2024-03-30T00:39:51.184025+0000","last_unstale":"2024-03-30T00:39:55.168811+0000","last_undegraded":"2024-03-30T00:39:55.168811+0000","last_fullsized":"2024-03-30T00:39:55.168811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:49.107527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571479+0000","last_change":"2024-03-30T00:39:55.177962+0000","last_active":"2024-03-30T00:39:55.571479+0000","last_peered":"2024-03-30T00:39:55.571479+0000","last_clean":"2024-03-30T00:39:55.571479+0000","last_became_active":"2024-03-30T00:39:55.177403+0000","last_became_peered":"2024-03-30T00:39:55.177403+0000","last_unstale":"2024-03-30T00:39:55.571479+0000","last_undegraded":"2024-03-30T00:39:55.571479+0000","last_fullsized":"2024-03-30T00:39:55.571479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:44:30.994000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168955+0000","last_change":"2024-03-30T00:39:50.765754+0000","last_active":"2024-03-30T00:39:55.168955+0000","last_peered":"2024-03-30T00:39:55.168955+0000","last_clean":"2024-03-30T00:39:55.168955+0000","last_became_active":"2024-03-30T00:39:50.764611+0000","last_became_peered":"2024-03-30T00:39:50.764611+0000","last_unstale":"2024-03-30T00:39:55.168955+0000","last_undegraded":"2024-03-30T00:39:55.168955+0000","last_fullsized":"2024-03-30T00:39:55.168955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:20.857915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184332+0000","last_change":"2024-03-30T00:39:51.180718+0000","last_active":"2024-03-30T00:39:55.184332+0000","last_peered":"2024-03-30T00:39:55.184332+0000","last_clean":"2024-03-30T00:39:55.184332+0000","last_became_active":"2024-03-30T00:39:51.175872+0000","last_became_peered":"2024-03-30T00:39:51.175872+0000","last_unstale":"2024-03-30T00:39:55.184332+0000","last_undegraded":"2024-03-30T00:39:55.184332+0000","last_fullsized":"2024-03-30T00:39:55.184332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272688+0000","last_change":"2024-03-30T00:39:55.184652+0000","last_active":"2024-03-30T00:39:57.272688+0000","last_peered":"2024-03-30T00:39:57.272688+0000","last_clean":"2024-03-30T00:39:57.272688+0000","last_became_active":"2024-03-30T00:39:55.184290+0000","last_became_peered":"2024-03-30T00:39:55.184290+0000","last_unstale":"2024-03-30T00:39:57.272688+0000","last_undegraded":"2024-03-30T00:39:57.272688+0000","last_fullsized":"2024-03-30T00:39:57.272688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:15:03.826593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163001+0000","last_change":"2024-03-30T00:39:50.771435+0000","last_active":"2024-03-30T00:39:55.163001+0000","last_peered":"2024-03-30T00:39:55.163001+0000","last_clean":"2024-03-30T00:39:55.163001+0000","last_became_active":"2024-03-30T00:39:50.771153+0000","last_became_peered":"2024-03-30T00:39:50.771153+0000","last_unstale":"2024-03-30T00:39:55.163001+0000","last_undegraded":"2024-03-30T00:39:55.163001+0000","last_fullsized":"2024-03-30T00:39:55.163001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:43.959952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170301+0000","last_change":"2024-03-30T00:39:51.183510+0000","last_active":"2024-03-30T00:39:55.170301+0000","last_peered":"2024-03-30T00:39:55.170301+0000","last_clean":"2024-03-30T00:39:55.170301+0000","last_became_active":"2024-03-30T00:39:51.183236+0000","last_became_peered":"2024-03-30T00:39:51.183236+0000","last_unstale":"2024-03-30T00:39:55.170301+0000","last_undegraded":"2024-03-30T00:39:55.170301+0000","last_fullsized":"2024-03-30T00:39:55.170301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273177+0000","last_change":"2024-03-30T00:39:55.179167+0000","last_active":"2024-03-30T00:39:57.273177+0000","last_peered":"2024-03-30T00:39:57.273177+0000","last_clean":"2024-03-30T00:39:57.273177+0000","last_became_active":"2024-03-30T00:39:55.178708+0000","last_became_peered":"2024-03-30T00:39:55.178708+0000","last_unstale":"2024-03-30T00:39:57.273177+0000","last_undegraded":"2024-03-30T00:39:57.273177+0000","last_fullsized":"2024-03-30T00:39:57.273177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161291+0000","last_change":"2024-03-30T00:39:50.773829+0000","last_active":"2024-03-30T00:39:55.161291+0000","last_peered":"2024-03-30T00:39:55.161291+0000","last_clean":"2024-03-30T00:39:55.161291+0000","last_became_active":"2024-03-30T00:39:50.773527+0000","last_became_peered":"2024-03-30T00:39:50.773527+0000","last_unstale":"2024-03-30T00:39:55.161291+0000","last_undegraded":"2024-03-30T00:39:55.161291+0000","last_fullsized":"2024-03-30T00:39:55.161291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.179506+0000","last_change":"2024-03-30T00:39:51.183689+0000","last_active":"2024-03-30T00:39:55.179506+0000","last_peered":"2024-03-30T00:39:55.179506+0000","last_clean":"2024-03-30T00:39:55.179506+0000","last_became_active":"2024-03-30T00:39:51.183421+0000","last_became_peered":"2024-03-30T00:39:51.183421+0000","last_unstale":"2024-03-30T00:39:55.179506+0000","last_undegraded":"2024-03-30T00:39:55.179506+0000","last_fullsized":"2024-03-30T00:39:55.179506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:54.542537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272671+0000","last_change":"2024-03-30T00:39:55.186210+0000","last_active":"2024-03-30T00:39:57.272671+0000","last_peered":"2024-03-30T00:39:57.272671+0000","last_clean":"2024-03-30T00:39:57.272671+0000","last_became_active":"2024-03-30T00:39:55.185849+0000","last_became_peered":"2024-03-30T00:39:55.185849+0000","last_unstale":"2024-03-30T00:39:57.272671+0000","last_undegraded":"2024-03-30T00:39:57.272671+0000","last_fullsized":"2024-03-30T00:39:57.272671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:45:20.370665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171140+0000","last_change":"2024-03-30T00:39:51.172454+0000","last_active":"2024-03-30T00:39:55.171140+0000","last_peered":"2024-03-30T00:39:55.171140+0000","last_clean":"2024-03-30T00:39:55.171140+0000","last_became_active":"2024-03-30T00:39:51.172196+0000","last_became_peered":"2024-03-30T00:39:51.172196+0000","last_unstale":"2024-03-30T00:39:55.171140+0000","last_undegraded":"2024-03-30T00:39:55.171140+0000","last_fullsized":"2024-03-30T00:39:55.171140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:02.372510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161442+0000","last_change":"2024-03-30T00:39:50.764668+0000","last_active":"2024-03-30T00:39:55.161442+0000","last_peered":"2024-03-30T00:39:55.161442+0000","last_clean":"2024-03-30T00:39:55.161442+0000","last_became_active":"2024-03-30T00:39:50.763661+0000","last_became_peered":"2024-03-30T00:39:50.763661+0000","last_unstale":"2024-03-30T00:39:55.161442+0000","last_undegraded":"2024-03-30T00:39:55.161442+0000","last_fullsized":"2024-03-30T00:39:55.161442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273301+0000","last_change":"2024-03-30T00:39:55.190348+0000","last_active":"2024-03-30T00:39:57.273301+0000","last_peered":"2024-03-30T00:39:57.273301+0000","last_clean":"2024-03-30T00:39:57.273301+0000","last_became_active":"2024-03-30T00:39:55.189987+0000","last_became_peered":"2024-03-30T00:39:55.189987+0000","last_unstale":"2024-03-30T00:39:57.273301+0000","last_undegraded":"2024-03-30T00:39:57.273301+0000","last_fullsized":"2024-03-30T00:39:57.273301+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:15:12.303669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169221+0000","last_change":"2024-03-30T00:39:51.168741+0000","last_active":"2024-03-30T00:39:55.169221+0000","last_peered":"2024-03-30T00:39:55.169221+0000","last_clean":"2024-03-30T00:39:55.169221+0000","last_became_active":"2024-03-30T00:39:51.168476+0000","last_became_peered":"2024-03-30T00:39:51.168476+0000","last_unstale":"2024-03-30T00:39:55.169221+0000","last_undegraded":"2024-03-30T00:39:55.169221+0000","last_fullsized":"2024-03-30T00:39:55.169221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:10:20.544219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172219+0000","last_change":"2024-03-30T00:39:50.766156+0000","last_active":"2024-03-30T00:39:55.172219+0000","last_peered":"2024-03-30T00:39:55.172219+0000","last_clean":"2024-03-30T00:39:55.172219+0000","last_became_active":"2024-03-30T00:39:50.765730+0000","last_became_peered":"2024-03-30T00:39:50.765730+0000","last_unstale":"2024-03-30T00:39:55.172219+0000","last_undegraded":"2024-03-30T00:39:55.172219+0000","last_fullsized":"2024-03-30T00:39:55.172219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:27.304749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272886+0000","last_change":"2024-03-30T00:39:55.180712+0000","last_active":"2024-03-30T00:39:57.272886+0000","last_peered":"2024-03-30T00:39:57.272886+0000","last_clean":"2024-03-30T00:39:57.272886+0000","last_became_active":"2024-03-30T00:39:55.179741+0000","last_became_peered":"2024-03-30T00:39:55.179741+0000","last_unstale":"2024-03-30T00:39:57.272886+0000","last_undegraded":"2024-03-30T00:39:57.272886+0000","last_fullsized":"2024-03-30T00:39:57.272886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.166309+0000","last_change":"2024-03-30T00:39:51.182604+0000","last_active":"2024-03-30T00:39:55.166309+0000","last_peered":"2024-03-30T00:39:55.166309+0000","last_clean":"2024-03-30T00:39:55.166309+0000","last_became_active":"2024-03-30T00:39:51.182323+0000","last_became_peered":"2024-03-30T00:39:51.182323+0000","last_unstale":"2024-03-30T00:39:55.166309+0000","last_undegraded":"2024-03-30T00:39:55.166309+0000","last_fullsized":"2024-03-30T00:39:55.166309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172036+0000","last_change":"2024-03-30T00:39:50.758774+0000","last_active":"2024-03-30T00:39:55.172036+0000","last_peered":"2024-03-30T00:39:55.172036+0000","last_clean":"2024-03-30T00:39:55.172036+0000","last_became_active":"2024-03-30T00:39:50.758508+0000","last_became_peered":"2024-03-30T00:39:50.758508+0000","last_unstale":"2024-03-30T00:39:55.172036+0000","last_undegraded":"2024-03-30T00:39:55.172036+0000","last_fullsized":"2024-03-30T00:39:55.172036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.129001+0000","last_change":"2024-03-30T00:39:55.182422+0000","last_active":"2024-03-30T00:39:57.129001+0000","last_peered":"2024-03-30T00:39:57.129001+0000","last_clean":"2024-03-30T00:39:57.129001+0000","last_became_active":"2024-03-30T00:39:55.182026+0000","last_became_peered":"2024-03-30T00:39:55.182026+0000","last_unstale":"2024-03-30T00:39:57.129001+0000","last_undegraded":"2024-03-30T00:39:57.129001+0000","last_fullsized":"2024-03-30T00:39:57.129001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:18:37.750989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160976+0000","last_change":"2024-03-30T00:39:51.161273+0000","last_active":"2024-03-30T00:39:55.160976+0000","last_peered":"2024-03-30T00:39:55.160976+0000","last_clean":"2024-03-30T00:39:55.160976+0000","last_became_active":"2024-03-30T00:39:51.156256+0000","last_became_peered":"2024-03-30T00:39:51.156256+0000","last_unstale":"2024-03-30T00:39:55.160976+0000","last_undegraded":"2024-03-30T00:39:55.160976+0000","last_fullsized":"2024-03-30T00:39:55.160976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:17.102921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159893+0000","last_change":"2024-03-30T00:39:50.768022+0000","last_active":"2024-03-30T00:39:55.159893+0000","last_peered":"2024-03-30T00:39:55.159893+0000","last_clean":"2024-03-30T00:39:55.159893+0000","last_became_active":"2024-03-30T00:39:50.767791+0000","last_became_peered":"2024-03-30T00:39:50.767791+0000","last_unstale":"2024-03-30T00:39:55.159893+0000","last_undegraded":"2024-03-30T00:39:55.159893+0000","last_fullsized":"2024-03-30T00:39:55.159893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:16.100201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273213+0000","last_change":"2024-03-30T00:39:55.176378+0000","last_active":"2024-03-30T00:39:57.273213+0000","last_peered":"2024-03-30T00:39:57.273213+0000","last_clean":"2024-03-30T00:39:57.273213+0000","last_became_active":"2024-03-30T00:39:55.176090+0000","last_became_peered":"2024-03-30T00:39:55.176090+0000","last_unstale":"2024-03-30T00:39:57.273213+0000","last_undegraded":"2024-03-30T00:39:57.273213+0000","last_fullsized":"2024-03-30T00:39:57.273213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184191+0000","last_change":"2024-03-30T00:39:51.181030+0000","last_active":"2024-03-30T00:39:55.184191+0000","last_peered":"2024-03-30T00:39:55.184191+0000","last_clean":"2024-03-30T00:39:55.184191+0000","last_became_active":"2024-03-30T00:39:51.180794+0000","last_became_peered":"2024-03-30T00:39:51.180794+0000","last_unstale":"2024-03-30T00:39:55.184191+0000","last_undegraded":"2024-03-30T00:39:55.184191+0000","last_fullsized":"2024-03-30T00:39:55.184191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:02.372510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172255+0000","last_change":"2024-03-30T00:39:50.771845+0000","last_active":"2024-03-30T00:39:55.172255+0000","last_peered":"2024-03-30T00:39:55.172255+0000","last_clean":"2024-03-30T00:39:55.172255+0000","last_became_active":"2024-03-30T00:39:50.771049+0000","last_became_peered":"2024-03-30T00:39:50.771049+0000","last_unstale":"2024-03-30T00:39:55.172255+0000","last_undegraded":"2024-03-30T00:39:55.172255+0000","last_fullsized":"2024-03-30T00:39:55.172255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:20.857915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272571+0000","last_change":"2024-03-30T00:39:55.174666+0000","last_active":"2024-03-30T00:39:57.272571+0000","last_peered":"2024-03-30T00:39:57.272571+0000","last_clean":"2024-03-30T00:39:57.272571+0000","last_became_active":"2024-03-30T00:39:55.174259+0000","last_became_peered":"2024-03-30T00:39:55.174259+0000","last_unstale":"2024-03-30T00:39:57.272571+0000","last_undegraded":"2024-03-30T00:39:57.272571+0000","last_fullsized":"2024-03-30T00:39:57.272571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165004+0000","last_change":"2024-03-30T00:39:51.178352+0000","last_active":"2024-03-30T00:39:55.165004+0000","last_peered":"2024-03-30T00:39:55.165004+0000","last_clean":"2024-03-30T00:39:55.165004+0000","last_became_active":"2024-03-30T00:39:51.178080+0000","last_became_peered":"2024-03-30T00:39:51.178080+0000","last_unstale":"2024-03-30T00:39:55.165004+0000","last_undegraded":"2024-03-30T00:39:55.165004+0000","last_fullsized":"2024-03-30T00:39:55.165004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172086+0000","last_change":"2024-03-30T00:39:50.768206+0000","last_active":"2024-03-30T00:39:55.172086+0000","last_peered":"2024-03-30T00:39:55.172086+0000","last_clean":"2024-03-30T00:39:55.172086+0000","last_became_active":"2024-03-30T00:39:50.767891+0000","last_became_peered":"2024-03-30T00:39:50.767891+0000","last_unstale":"2024-03-30T00:39:55.172086+0000","last_undegraded":"2024-03-30T00:39:55.172086+0000","last_fullsized":"2024-03-30T00:39:55.172086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:39.163294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571335+0000","last_change":"2024-03-30T00:39:55.192391+0000","last_active":"2024-03-30T00:39:55.571335+0000","last_peered":"2024-03-30T00:39:55.571335+0000","last_clean":"2024-03-30T00:39:55.571335+0000","last_became_active":"2024-03-30T00:39:55.189816+0000","last_became_peered":"2024-03-30T00:39:55.189816+0000","last_unstale":"2024-03-30T00:39:55.571335+0000","last_undegraded":"2024-03-30T00:39:55.571335+0000","last_fullsized":"2024-03-30T00:39:55.571335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:05:10.673253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171550+0000","last_change":"2024-03-30T00:39:51.161793+0000","last_active":"2024-03-30T00:39:55.171550+0000","last_peered":"2024-03-30T00:39:55.171550+0000","last_clean":"2024-03-30T00:39:55.171550+0000","last_became_active":"2024-03-30T00:39:51.161523+0000","last_became_peered":"2024-03-30T00:39:51.161523+0000","last_unstale":"2024-03-30T00:39:55.171550+0000","last_undegraded":"2024-03-30T00:39:55.171550+0000","last_fullsized":"2024-03-30T00:39:55.171550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163175+0000","last_change":"2024-03-30T00:39:50.769035+0000","last_active":"2024-03-30T00:39:55.163175+0000","last_peered":"2024-03-30T00:39:55.163175+0000","last_clean":"2024-03-30T00:39:55.163175+0000","last_became_active":"2024-03-30T00:39:50.768789+0000","last_became_peered":"2024-03-30T00:39:50.768789+0000","last_unstale":"2024-03-30T00:39:55.163175+0000","last_undegraded":"2024-03-30T00:39:55.163175+0000","last_fullsized":"2024-03-30T00:39:55.163175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:39.163294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273625+0000","last_change":"2024-03-30T00:39:55.200272+0000","last_active":"2024-03-30T00:39:57.273625+0000","last_peered":"2024-03-30T00:39:57.273625+0000","last_clean":"2024-03-30T00:39:57.273625+0000","last_became_active":"2024-03-30T00:39:55.199688+0000","last_became_peered":"2024-03-30T00:39:55.199688+0000","last_unstale":"2024-03-30T00:39:57.273625+0000","last_undegraded":"2024-03-30T00:39:57.273625+0000","last_fullsized":"2024-03-30T00:39:57.273625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:30.942593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.174408+0000","last_change":"2024-03-30T00:39:51.174886+0000","last_active":"2024-03-30T00:39:55.174408+0000","last_peered":"2024-03-30T00:39:55.174408+0000","last_clean":"2024-03-30T00:39:55.174408+0000","last_became_active":"2024-03-30T00:39:51.174274+0000","last_became_peered":"2024-03-30T00:39:51.174274+0000","last_unstale":"2024-03-30T00:39:55.174408+0000","last_undegraded":"2024-03-30T00:39:55.174408+0000","last_fullsized":"2024-03-30T00:39:55.174408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:24:05.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171643+0000","last_change":"2024-03-30T00:39:50.769092+0000","last_active":"2024-03-30T00:39:55.171643+0000","last_peered":"2024-03-30T00:39:55.171643+0000","last_clean":"2024-03-30T00:39:55.171643+0000","last_became_active":"2024-03-30T00:39:50.768750+0000","last_became_peered":"2024-03-30T00:39:50.768750+0000","last_unstale":"2024-03-30T00:39:55.171643+0000","last_undegraded":"2024-03-30T00:39:55.171643+0000","last_fullsized":"2024-03-30T00:39:55.171643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128439+0000","last_change":"2024-03-30T00:39:55.187382+0000","last_active":"2024-03-30T00:39:57.128439+0000","last_peered":"2024-03-30T00:39:57.128439+0000","last_clean":"2024-03-30T00:39:57.128439+0000","last_became_active":"2024-03-30T00:39:55.186804+0000","last_became_peered":"2024-03-30T00:39:55.186804+0000","last_unstale":"2024-03-30T00:39:57.128439+0000","last_undegraded":"2024-03-30T00:39:57.128439+0000","last_fullsized":"2024-03-30T00:39:57.128439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161837+0000","last_change":"2024-03-30T00:39:51.172750+0000","last_active":"2024-03-30T00:39:55.161837+0000","last_peered":"2024-03-30T00:39:55.161837+0000","last_clean":"2024-03-30T00:39:55.161837+0000","last_became_active":"2024-03-30T00:39:51.172456+0000","last_became_peered":"2024-03-30T00:39:51.172456+0000","last_unstale":"2024-03-30T00:39:55.161837+0000","last_undegraded":"2024-03-30T00:39:55.161837+0000","last_fullsized":"2024-03-30T00:39:55.161837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:40.166014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168217+0000","last_change":"2024-03-30T00:39:50.775126+0000","last_active":"2024-03-30T00:39:55.168217+0000","last_peered":"2024-03-30T00:39:55.168217+0000","last_clean":"2024-03-30T00:39:55.168217+0000","last_became_active":"2024-03-30T00:39:50.773078+0000","last_became_peered":"2024-03-30T00:39:50.773078+0000","last_unstale":"2024-03-30T00:39:55.168217+0000","last_undegraded":"2024-03-30T00:39:55.168217+0000","last_fullsized":"2024-03-30T00:39:55.168217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128669+0000","last_change":"2024-03-30T00:39:55.182489+0000","last_active":"2024-03-30T00:39:57.128669+0000","last_peered":"2024-03-30T00:39:57.128669+0000","last_clean":"2024-03-30T00:39:57.128669+0000","last_became_active":"2024-03-30T00:39:55.181876+0000","last_became_peered":"2024-03-30T00:39:55.181876+0000","last_unstale":"2024-03-30T00:39:57.128669+0000","last_undegraded":"2024-03-30T00:39:57.128669+0000","last_fullsized":"2024-03-30T00:39:57.128669+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171723+0000","last_change":"2024-03-30T00:39:51.157937+0000","last_active":"2024-03-30T00:39:55.171723+0000","last_peered":"2024-03-30T00:39:55.171723+0000","last_clean":"2024-03-30T00:39:55.171723+0000","last_became_active":"2024-03-30T00:39:51.155936+0000","last_became_peered":"2024-03-30T00:39:51.155936+0000","last_unstale":"2024-03-30T00:39:55.171723+0000","last_undegraded":"2024-03-30T00:39:55.171723+0000","last_fullsized":"2024-03-30T00:39:55.171723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159139+0000","last_change":"2024-03-30T00:39:50.765709+0000","last_active":"2024-03-30T00:39:55.159139+0000","last_peered":"2024-03-30T00:39:55.159139+0000","last_clean":"2024-03-30T00:39:55.159139+0000","last_became_active":"2024-03-30T00:39:50.765450+0000","last_became_peered":"2024-03-30T00:39:50.765450+0000","last_unstale":"2024-03-30T00:39:55.159139+0000","last_undegraded":"2024-03-30T00:39:55.159139+0000","last_fullsized":"2024-03-30T00:39:55.159139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.129028+0000","last_change":"2024-03-30T00:39:55.193764+0000","last_active":"2024-03-30T00:39:57.129028+0000","last_peered":"2024-03-30T00:39:57.129028+0000","last_clean":"2024-03-30T00:39:57.129028+0000","last_became_active":"2024-03-30T00:39:55.188761+0000","last_became_peered":"2024-03-30T00:39:55.188761+0000","last_unstale":"2024-03-30T00:39:57.129028+0000","last_undegraded":"2024-03-30T00:39:57.129028+0000","last_fullsized":"2024-03-30T00:39:57.129028+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.182494+0000","last_change":"2024-03-30T00:39:51.176755+0000","last_active":"2024-03-30T00:39:55.182494+0000","last_peered":"2024-03-30T00:39:55.182494+0000","last_clean":"2024-03-30T00:39:55.182494+0000","last_became_active":"2024-03-30T00:39:51.176507+0000","last_became_peered":"2024-03-30T00:39:51.176507+0000","last_unstale":"2024-03-30T00:39:55.182494+0000","last_undegraded":"2024-03-30T00:39:55.182494+0000","last_fullsized":"2024-03-30T00:39:55.182494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160037+0000","last_change":"2024-03-30T00:39:50.763226+0000","last_active":"2024-03-30T00:39:55.160037+0000","last_peered":"2024-03-30T00:39:55.160037+0000","last_clean":"2024-03-30T00:39:55.160037+0000","last_became_active":"2024-03-30T00:39:50.762972+0000","last_became_peered":"2024-03-30T00:39:50.762972+0000","last_unstale":"2024-03-30T00:39:55.160037+0000","last_undegraded":"2024-03-30T00:39:55.160037+0000","last_fullsized":"2024-03-30T00:39:55.160037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:27.304749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273996+0000","last_change":"2024-03-30T00:39:55.198804+0000","last_active":"2024-03-30T00:39:57.273996+0000","last_peered":"2024-03-30T00:39:57.273996+0000","last_clean":"2024-03-30T00:39:57.273996+0000","last_became_active":"2024-03-30T00:39:55.198382+0000","last_became_peered":"2024-03-30T00:39:55.198382+0000","last_unstale":"2024-03-30T00:39:57.273996+0000","last_undegraded":"2024-03-30T00:39:57.273996+0000","last_fullsized":"2024-03-30T00:39:57.273996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165312+0000","last_change":"2024-03-30T00:39:51.176690+0000","last_active":"2024-03-30T00:39:55.165312+0000","last_peered":"2024-03-30T00:39:55.165312+0000","last_clean":"2024-03-30T00:39:55.165312+0000","last_became_active":"2024-03-30T00:39:51.175614+0000","last_became_peered":"2024-03-30T00:39:51.175614+0000","last_unstale":"2024-03-30T00:39:55.165312+0000","last_undegraded":"2024-03-30T00:39:55.165312+0000","last_fullsized":"2024-03-30T00:39:55.165312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:13.959522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.176056+0000","last_change":"2024-03-30T00:39:50.766293+0000","last_active":"2024-03-30T00:39:55.176056+0000","last_peered":"2024-03-30T00:39:55.176056+0000","last_clean":"2024-03-30T00:39:55.176056+0000","last_became_active":"2024-03-30T00:39:50.765977+0000","last_became_peered":"2024-03-30T00:39:50.765977+0000","last_unstale":"2024-03-30T00:39:55.176056+0000","last_undegraded":"2024-03-30T00:39:55.176056+0000","last_fullsized":"2024-03-30T00:39:55.176056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160226+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:39:55.160226+0000","last_peered":"2024-03-30T00:39:55.160226+0000","last_clean":"2024-03-30T00:39:55.160226+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:39:55.160226+0000","last_undegraded":"2024-03-30T00:39:55.160226+0000","last_fullsized":"2024-03-30T00:39:55.160226+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572655+0000","last_change":"2024-03-30T00:39:55.202116+0000","last_active":"2024-03-30T00:39:55.572655+0000","last_peered":"2024-03-30T00:39:55.572655+0000","last_clean":"2024-03-30T00:39:55.572655+0000","last_became_active":"2024-03-30T00:39:55.201638+0000","last_became_peered":"2024-03-30T00:39:55.201638+0000","last_unstale":"2024-03-30T00:39:55.572655+0000","last_undegraded":"2024-03-30T00:39:55.572655+0000","last_fullsized":"2024-03-30T00:39:55.572655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:46:36.593302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162909+0000","last_change":"2024-03-30T00:39:51.176768+0000","last_active":"2024-03-30T00:39:55.162909+0000","last_peered":"2024-03-30T00:39:55.162909+0000","last_clean":"2024-03-30T00:39:55.162909+0000","last_became_active":"2024-03-30T00:39:51.176215+0000","last_became_peered":"2024-03-30T00:39:51.176215+0000","last_unstale":"2024-03-30T00:39:55.162909+0000","last_undegraded":"2024-03-30T00:39:55.162909+0000","last_fullsized":"2024-03-30T00:39:55.162909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:26.397679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159355+0000","last_change":"2024-03-30T00:39:50.763227+0000","last_active":"2024-03-30T00:39:55.159355+0000","last_peered":"2024-03-30T00:39:55.159355+0000","last_clean":"2024-03-30T00:39:55.159355+0000","last_became_active":"2024-03-30T00:39:50.762934+0000","last_became_peered":"2024-03-30T00:39:50.762934+0000","last_unstale":"2024-03-30T00:39:55.159355+0000","last_undegraded":"2024-03-30T00:39:55.159355+0000","last_fullsized":"2024-03-30T00:39:55.159355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572469+0000","last_change":"2024-03-30T00:39:55.195709+0000","last_active":"2024-03-30T00:39:55.572469+0000","last_peered":"2024-03-30T00:39:55.572469+0000","last_clean":"2024-03-30T00:39:55.572469+0000","last_became_active":"2024-03-30T00:39:55.194810+0000","last_became_peered":"2024-03-30T00:39:55.194810+0000","last_unstale":"2024-03-30T00:39:55.572469+0000","last_undegraded":"2024-03-30T00:39:55.572469+0000","last_fullsized":"2024-03-30T00:39:55.572469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:30.942593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170424+0000","last_change":"2024-03-30T00:39:51.182838+0000","last_active":"2024-03-30T00:39:55.170424+0000","last_peered":"2024-03-30T00:39:55.170424+0000","last_clean":"2024-03-30T00:39:55.170424+0000","last_became_active":"2024-03-30T00:39:51.178980+0000","last_became_peered":"2024-03-30T00:39:51.178980+0000","last_unstale":"2024-03-30T00:39:55.170424+0000","last_undegraded":"2024-03-30T00:39:55.170424+0000","last_fullsized":"2024-03-30T00:39:55.170424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:13.959522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172688+0000","last_change":"2024-03-30T00:39:50.755295+0000","last_active":"2024-03-30T00:39:55.172688+0000","last_peered":"2024-03-30T00:39:55.172688+0000","last_clean":"2024-03-30T00:39:55.172688+0000","last_became_active":"2024-03-30T00:39:50.754957+0000","last_became_peered":"2024-03-30T00:39:50.754957+0000","last_unstale":"2024-03-30T00:39:55.172688+0000","last_undegraded":"2024-03-30T00:39:55.172688+0000","last_fullsized":"2024-03-30T00:39:55.172688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:16.100201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273833+0000","last_change":"2024-03-30T00:39:55.199975+0000","last_active":"2024-03-30T00:39:57.273833+0000","last_peered":"2024-03-30T00:39:57.273833+0000","last_clean":"2024-03-30T00:39:57.273833+0000","last_became_active":"2024-03-30T00:39:55.199191+0000","last_became_peered":"2024-03-30T00:39:55.199191+0000","last_unstale":"2024-03-30T00:39:57.273833+0000","last_undegraded":"2024-03-30T00:39:57.273833+0000","last_fullsized":"2024-03-30T00:39:57.273833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163333+0000","last_change":"2024-03-30T00:39:51.163933+0000","last_active":"2024-03-30T00:39:55.163333+0000","last_peered":"2024-03-30T00:39:55.163333+0000","last_clean":"2024-03-30T00:39:55.163333+0000","last_became_active":"2024-03-30T00:39:51.163663+0000","last_became_peered":"2024-03-30T00:39:51.163663+0000","last_unstale":"2024-03-30T00:39:55.163333+0000","last_undegraded":"2024-03-30T00:39:55.163333+0000","last_fullsized":"2024-03-30T00:39:55.163333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:41.330935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159267+0000","last_change":"2024-03-30T00:39:50.766833+0000","last_active":"2024-03-30T00:39:55.159267+0000","last_peered":"2024-03-30T00:39:55.159267+0000","last_clean":"2024-03-30T00:39:55.159267+0000","last_became_active":"2024-03-30T00:39:50.766534+0000","last_became_peered":"2024-03-30T00:39:50.766534+0000","last_unstale":"2024-03-30T00:39:55.159267+0000","last_undegraded":"2024-03-30T00:39:55.159267+0000","last_fullsized":"2024-03-30T00:39:55.159267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:56.358073+0000","last_change":"2024-03-30T00:39:55.184862+0000","last_active":"2024-03-30T00:39:56.358073+0000","last_peered":"2024-03-30T00:39:56.358073+0000","last_clean":"2024-03-30T00:39:56.358073+0000","last_became_active":"2024-03-30T00:39:55.184374+0000","last_became_peered":"2024-03-30T00:39:55.184374+0000","last_unstale":"2024-03-30T00:39:56.358073+0000","last_undegraded":"2024-03-30T00:39:56.358073+0000","last_fullsized":"2024-03-30T00:39:56.358073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163385+0000","last_change":"2024-03-30T00:39:51.162489+0000","last_active":"2024-03-30T00:39:55.163385+0000","last_peered":"2024-03-30T00:39:55.163385+0000","last_clean":"2024-03-30T00:39:55.163385+0000","last_became_active":"2024-03-30T00:39:51.162205+0000","last_became_peered":"2024-03-30T00:39:51.162205+0000","last_unstale":"2024-03-30T00:39:55.163385+0000","last_undegraded":"2024-03-30T00:39:55.163385+0000","last_fullsized":"2024-03-30T00:39:55.163385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:36.973826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162109+0000","last_change":"2024-03-30T00:39:50.768761+0000","last_active":"2024-03-30T00:39:55.162109+0000","last_peered":"2024-03-30T00:39:55.162109+0000","last_clean":"2024-03-30T00:39:55.162109+0000","last_became_active":"2024-03-30T00:39:50.768372+0000","last_became_peered":"2024-03-30T00:39:50.768372+0000","last_unstale":"2024-03-30T00:39:55.162109+0000","last_undegraded":"2024-03-30T00:39:55.162109+0000","last_fullsized":"2024-03-30T00:39:55.162109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:35:25.209340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272560+0000","last_change":"2024-03-30T00:39:55.177966+0000","last_active":"2024-03-30T00:39:57.272560+0000","last_peered":"2024-03-30T00:39:57.272560+0000","last_clean":"2024-03-30T00:39:57.272560+0000","last_became_active":"2024-03-30T00:39:55.177485+0000","last_became_peered":"2024-03-30T00:39:55.177485+0000","last_unstale":"2024-03-30T00:39:57.272560+0000","last_undegraded":"2024-03-30T00:39:57.272560+0000","last_fullsized":"2024-03-30T00:39:57.272560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161004+0000","last_change":"2024-03-30T00:39:51.157566+0000","last_active":"2024-03-30T00:39:55.161004+0000","last_peered":"2024-03-30T00:39:55.161004+0000","last_clean":"2024-03-30T00:39:55.161004+0000","last_became_active":"2024-03-30T00:39:51.157243+0000","last_became_peered":"2024-03-30T00:39:51.157243+0000","last_unstale":"2024-03-30T00:39:55.161004+0000","last_undegraded":"2024-03-30T00:39:55.161004+0000","last_fullsized":"2024-03-30T00:39:55.161004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:35.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159067+0000","last_change":"2024-03-30T00:39:50.760164+0000","last_active":"2024-03-30T00:39:55.159067+0000","last_peered":"2024-03-30T00:39:55.159067+0000","last_clean":"2024-03-30T00:39:55.159067+0000","last_became_active":"2024-03-30T00:39:50.759887+0000","last_became_peered":"2024-03-30T00:39:50.759887+0000","last_unstale":"2024-03-30T00:39:55.159067+0000","last_undegraded":"2024-03-30T00:39:55.159067+0000","last_fullsized":"2024-03-30T00:39:55.159067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273934+0000","last_change":"2024-03-30T00:39:55.196574+0000","last_active":"2024-03-30T00:39:57.273934+0000","last_peered":"2024-03-30T00:39:57.273934+0000","last_clean":"2024-03-30T00:39:57.273934+0000","last_became_active":"2024-03-30T00:39:55.196182+0000","last_became_peered":"2024-03-30T00:39:55.196182+0000","last_unstale":"2024-03-30T00:39:57.273934+0000","last_undegraded":"2024-03-30T00:39:57.273934+0000","last_fullsized":"2024-03-30T00:39:57.273934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170724+0000","last_change":"2024-03-30T00:39:51.167097+0000","last_active":"2024-03-30T00:39:55.170724+0000","last_peered":"2024-03-30T00:39:55.170724+0000","last_clean":"2024-03-30T00:39:55.170724+0000","last_became_active":"2024-03-30T00:39:51.166629+0000","last_became_peered":"2024-03-30T00:39:51.166629+0000","last_unstale":"2024-03-30T00:39:55.170724+0000","last_undegraded":"2024-03-30T00:39:55.170724+0000","last_fullsized":"2024-03-30T00:39:55.170724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170864+0000","last_change":"2024-03-30T00:39:50.767642+0000","last_active":"2024-03-30T00:39:55.170864+0000","last_peered":"2024-03-30T00:39:55.170864+0000","last_clean":"2024-03-30T00:39:55.170864+0000","last_became_active":"2024-03-30T00:39:50.767166+0000","last_became_peered":"2024-03-30T00:39:50.767166+0000","last_unstale":"2024-03-30T00:39:55.170864+0000","last_undegraded":"2024-03-30T00:39:55.170864+0000","last_fullsized":"2024-03-30T00:39:55.170864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572362+0000","last_change":"2024-03-30T00:39:55.192042+0000","last_active":"2024-03-30T00:39:55.572362+0000","last_peered":"2024-03-30T00:39:55.572362+0000","last_clean":"2024-03-30T00:39:55.572362+0000","last_became_active":"2024-03-30T00:39:55.190080+0000","last_became_peered":"2024-03-30T00:39:55.190080+0000","last_unstale":"2024-03-30T00:39:55.572362+0000","last_undegraded":"2024-03-30T00:39:55.572362+0000","last_fullsized":"2024-03-30T00:39:55.572362+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:25.792128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161353+0000","last_change":"2024-03-30T00:39:51.161405+0000","last_active":"2024-03-30T00:39:55.161353+0000","last_peered":"2024-03-30T00:39:55.161353+0000","last_clean":"2024-03-30T00:39:55.161353+0000","last_became_active":"2024-03-30T00:39:51.161053+0000","last_became_peered":"2024-03-30T00:39:51.161053+0000","last_unstale":"2024-03-30T00:39:55.161353+0000","last_undegraded":"2024-03-30T00:39:55.161353+0000","last_fullsized":"2024-03-30T00:39:55.161353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:49.107527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168130+0000","last_change":"2024-03-30T00:39:50.774925+0000","last_active":"2024-03-30T00:39:55.168130+0000","last_peered":"2024-03-30T00:39:55.168130+0000","last_clean":"2024-03-30T00:39:55.168130+0000","last_became_active":"2024-03-30T00:39:50.773221+0000","last_became_peered":"2024-03-30T00:39:50.773221+0000","last_unstale":"2024-03-30T00:39:55.168130+0000","last_undegraded":"2024-03-30T00:39:55.168130+0000","last_fullsized":"2024-03-30T00:39:55.168130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572142+0000","last_change":"2024-03-30T00:39:55.191576+0000","last_active":"2024-03-30T00:39:55.572142+0000","last_peered":"2024-03-30T00:39:55.572142+0000","last_clean":"2024-03-30T00:39:55.572142+0000","last_became_active":"2024-03-30T00:39:55.191188+0000","last_became_peered":"2024-03-30T00:39:55.191188+0000","last_unstale":"2024-03-30T00:39:55.572142+0000","last_undegraded":"2024-03-30T00:39:55.572142+0000","last_fullsized":"2024-03-30T00:39:55.572142+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:58.020154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184454+0000","last_change":"2024-03-30T00:39:51.169704+0000","last_active":"2024-03-30T00:39:55.184454+0000","last_peered":"2024-03-30T00:39:55.184454+0000","last_clean":"2024-03-30T00:39:55.184454+0000","last_became_active":"2024-03-30T00:39:51.168350+0000","last_became_peered":"2024-03-30T00:39:51.168350+0000","last_unstale":"2024-03-30T00:39:55.184454+0000","last_undegraded":"2024-03-30T00:39:55.184454+0000","last_fullsized":"2024-03-30T00:39:55.184454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183870+0000","last_change":"2024-03-30T00:39:50.760857+0000","last_active":"2024-03-30T00:39:55.183870+0000","last_peered":"2024-03-30T00:39:55.183870+0000","last_clean":"2024-03-30T00:39:55.183870+0000","last_became_active":"2024-03-30T00:39:50.760607+0000","last_became_peered":"2024-03-30T00:39:50.760607+0000","last_unstale":"2024-03-30T00:39:55.183870+0000","last_undegraded":"2024-03-30T00:39:55.183870+0000","last_fullsized":"2024-03-30T00:39:55.183870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:56.358741+0000","last_change":"2024-03-30T00:39:55.188289+0000","last_active":"2024-03-30T00:39:56.358741+0000","last_peered":"2024-03-30T00:39:56.358741+0000","last_clean":"2024-03-30T00:39:56.358741+0000","last_became_active":"2024-03-30T00:39:55.187869+0000","last_became_peered":"2024-03-30T00:39:55.187869+0000","last_unstale":"2024-03-30T00:39:56.358741+0000","last_undegraded":"2024-03-30T00:39:56.358741+0000","last_fullsized":"2024-03-30T00:39:56.358741+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162815+0000","last_change":"2024-03-30T00:39:51.177240+0000","last_active":"2024-03-30T00:39:55.162815+0000","last_peered":"2024-03-30T00:39:55.162815+0000","last_clean":"2024-03-30T00:39:55.162815+0000","last_became_active":"2024-03-30T00:39:51.176388+0000","last_became_peered":"2024-03-30T00:39:51.176388+0000","last_unstale":"2024-03-30T00:39:55.162815+0000","last_undegraded":"2024-03-30T00:39:55.162815+0000","last_fullsized":"2024-03-30T00:39:55.162815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:37:05.851113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:40:01.400156+0000","last_change":"2024-03-30T00:39:50.766631+0000","last_active":"2024-03-30T00:40:01.400156+0000","last_peered":"2024-03-30T00:40:01.400156+0000","last_clean":"2024-03-30T00:40:01.400156+0000","last_became_active":"2024-03-30T00:39:50.766362+0000","last_became_peered":"2024-03-30T00:39:50.766362+0000","last_unstale":"2024-03-30T00:40:01.400156+0000","last_undegraded":"2024-03-30T00:40:01.400156+0000","last_fullsized":"2024-03-30T00:40:01.400156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128582+0000","last_change":"2024-03-30T00:39:55.187191+0000","last_active":"2024-03-30T00:39:57.128582+0000","last_peered":"2024-03-30T00:39:57.128582+0000","last_clean":"2024-03-30T00:39:57.128582+0000","last_became_active":"2024-03-30T00:39:55.186712+0000","last_became_peered":"2024-03-30T00:39:55.186712+0000","last_unstale":"2024-03-30T00:39:57.128582+0000","last_undegraded":"2024-03-30T00:39:57.128582+0000","last_fullsized":"2024-03-30T00:39:57.128582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162239+0000","last_change":"2024-03-30T00:39:51.154183+0000","last_active":"2024-03-30T00:39:55.162239+0000","last_peered":"2024-03-30T00:39:55.162239+0000","last_clean":"2024-03-30T00:39:55.162239+0000","last_became_active":"2024-03-30T00:39:51.153909+0000","last_became_peered":"2024-03-30T00:39:51.153909+0000","last_unstale":"2024-03-30T00:39:55.162239+0000","last_undegraded":"2024-03-30T00:39:55.162239+0000","last_fullsized":"2024-03-30T00:39:55.162239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:51.868242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172499+0000","last_change":"2024-03-30T00:39:50.763665+0000","last_active":"2024-03-30T00:39:55.172499+0000","last_peered":"2024-03-30T00:39:55.172499+0000","last_clean":"2024-03-30T00:39:55.172499+0000","last_became_active":"2024-03-30T00:39:50.763390+0000","last_became_peered":"2024-03-30T00:39:50.763390+0000","last_unstale":"2024-03-30T00:39:55.172499+0000","last_undegraded":"2024-03-30T00:39:55.172499+0000","last_fullsized":"2024-03-30T00:39:55.172499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272944+0000","last_change":"2024-03-30T00:39:55.185767+0000","last_active":"2024-03-30T00:39:57.272944+0000","last_peered":"2024-03-30T00:39:57.272944+0000","last_clean":"2024-03-30T00:39:57.272944+0000","last_became_active":"2024-03-30T00:39:55.185430+0000","last_became_peered":"2024-03-30T00:39:55.185430+0000","last_unstale":"2024-03-30T00:39:57.272944+0000","last_undegraded":"2024-03-30T00:39:57.272944+0000","last_fullsized":"2024-03-30T00:39:57.272944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:59:28.443255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184382+0000","last_change":"2024-03-30T00:39:51.177311+0000","last_active":"2024-03-30T00:39:55.184382+0000","last_peered":"2024-03-30T00:39:55.184382+0000","last_clean":"2024-03-30T00:39:55.184382+0000","last_became_active":"2024-03-30T00:39:51.175904+0000","last_became_peered":"2024-03-30T00:39:51.175904+0000","last_unstale":"2024-03-30T00:39:55.184382+0000","last_undegraded":"2024-03-30T00:39:55.184382+0000","last_fullsized":"2024-03-30T00:39:55.184382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167202+0000","last_change":"2024-03-30T00:39:50.769375+0000","last_active":"2024-03-30T00:39:55.167202+0000","last_peered":"2024-03-30T00:39:55.167202+0000","last_clean":"2024-03-30T00:39:55.167202+0000","last_became_active":"2024-03-30T00:39:50.768671+0000","last_became_peered":"2024-03-30T00:39:50.768671+0000","last_unstale":"2024-03-30T00:39:55.167202+0000","last_undegraded":"2024-03-30T00:39:55.167202+0000","last_fullsized":"2024-03-30T00:39:55.167202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571487+0000","last_change":"2024-03-30T00:39:55.192307+0000","last_active":"2024-03-30T00:39:55.571487+0000","last_peered":"2024-03-30T00:39:55.571487+0000","last_clean":"2024-03-30T00:39:55.571487+0000","last_became_active":"2024-03-30T00:39:55.191803+0000","last_became_peered":"2024-03-30T00:39:55.191803+0000","last_unstale":"2024-03-30T00:39:55.571487+0000","last_undegraded":"2024-03-30T00:39:55.571487+0000","last_fullsized":"2024-03-30T00:39:55.571487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:33:50.238803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167108+0000","last_change":"2024-03-30T00:39:51.168083+0000","last_active":"2024-03-30T00:39:55.167108+0000","last_peered":"2024-03-30T00:39:55.167108+0000","last_clean":"2024-03-30T00:39:55.167108+0000","last_became_active":"2024-03-30T00:39:51.167756+0000","last_became_peered":"2024-03-30T00:39:51.167756+0000","last_unstale":"2024-03-30T00:39:55.167108+0000","last_undegraded":"2024-03-30T00:39:55.167108+0000","last_fullsized":"2024-03-30T00:39:55.167108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172080+0000","last_change":"2024-03-30T00:39:50.775634+0000","last_active":"2024-03-30T00:39:55.172080+0000","last_peered":"2024-03-30T00:39:55.172080+0000","last_clean":"2024-03-30T00:39:55.172080+0000","last_became_active":"2024-03-30T00:39:50.775337+0000","last_became_peered":"2024-03-30T00:39:50.775337+0000","last_unstale":"2024-03-30T00:39:55.172080+0000","last_undegraded":"2024-03-30T00:39:55.172080+0000","last_fullsized":"2024-03-30T00:39:55.172080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571487+0000","last_change":"2024-03-30T00:39:55.175243+0000","last_active":"2024-03-30T00:39:55.571487+0000","last_peered":"2024-03-30T00:39:55.571487+0000","last_clean":"2024-03-30T00:39:55.571487+0000","last_became_active":"2024-03-30T00:39:55.174556+0000","last_became_peered":"2024-03-30T00:39:55.174556+0000","last_unstale":"2024-03-30T00:39:55.571487+0000","last_undegraded":"2024-03-30T00:39:55.571487+0000","last_fullsized":"2024-03-30T00:39:55.571487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.274064+0000","last_change":"2024-03-30T00:39:55.203406+0000","last_active":"2024-03-30T00:39:57.274064+0000","last_peered":"2024-03-30T00:39:57.274064+0000","last_clean":"2024-03-30T00:39:57.274064+0000","last_became_active":"2024-03-30T00:39:55.203027+0000","last_became_peered":"2024-03-30T00:39:55.203027+0000","last_unstale":"2024-03-30T00:39:57.274064+0000","last_undegraded":"2024-03-30T00:39:57.274064+0000","last_fullsized":"2024-03-30T00:39:57.274064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171850+0000","last_change":"2024-03-30T00:39:51.175309+0000","last_active":"2024-03-30T00:39:55.171850+0000","last_peered":"2024-03-30T00:39:55.171850+0000","last_clean":"2024-03-30T00:39:55.171850+0000","last_became_active":"2024-03-30T00:39:51.174946+0000","last_became_peered":"2024-03-30T00:39:51.174946+0000","last_unstale":"2024-03-30T00:39:55.171850+0000","last_undegraded":"2024-03-30T00:39:55.171850+0000","last_fullsized":"2024-03-30T00:39:55.171850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167862+0000","last_change":"2024-03-30T00:39:50.762581+0000","last_active":"2024-03-30T00:39:55.167862+0000","last_peered":"2024-03-30T00:39:55.167862+0000","last_clean":"2024-03-30T00:39:55.167862+0000","last_became_active":"2024-03-30T00:39:50.762206+0000","last_became_peered":"2024-03-30T00:39:50.762206+0000","last_unstale":"2024-03-30T00:39:55.167862+0000","last_undegraded":"2024-03-30T00:39:55.167862+0000","last_fullsized":"2024-03-30T00:39:55.167862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.573135+0000","last_change":"2024-03-30T00:39:55.188289+0000","last_active":"2024-03-30T00:39:55.573135+0000","last_peered":"2024-03-30T00:39:55.573135+0000","last_clean":"2024-03-30T00:39:55.573135+0000","last_became_active":"2024-03-30T00:39:55.187825+0000","last_became_peered":"2024-03-30T00:39:55.187825+0000","last_unstale":"2024-03-30T00:39:55.573135+0000","last_undegraded":"2024-03-30T00:39:55.573135+0000","last_fullsized":"2024-03-30T00:39:55.573135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.166225+0000","last_change":"2024-03-30T00:39:51.172769+0000","last_active":"2024-03-30T00:39:55.166225+0000","last_peered":"2024-03-30T00:39:55.166225+0000","last_clean":"2024-03-30T00:39:55.166225+0000","last_became_active":"2024-03-30T00:39:51.170913+0000","last_became_peered":"2024-03-30T00:39:51.170913+0000","last_unstale":"2024-03-30T00:39:55.166225+0000","last_undegraded":"2024-03-30T00:39:55.166225+0000","last_fullsized":"2024-03-30T00:39:55.166225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165283+0000","last_change":"2024-03-30T00:39:50.771732+0000","last_active":"2024-03-30T00:39:55.165283+0000","last_peered":"2024-03-30T00:39:55.165283+0000","last_clean":"2024-03-30T00:39:55.165283+0000","last_became_active":"2024-03-30T00:39:50.771443+0000","last_became_peered":"2024-03-30T00:39:50.771443+0000","last_unstale":"2024-03-30T00:39:55.165283+0000","last_undegraded":"2024-03-30T00:39:55.165283+0000","last_fullsized":"2024-03-30T00:39:55.165283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171239+0000","last_change":"2024-03-30T00:39:50.768147+0000","last_active":"2024-03-30T00:39:55.171239+0000","last_peered":"2024-03-30T00:39:55.171239+0000","last_clean":"2024-03-30T00:39:55.171239+0000","last_became_active":"2024-03-30T00:39:50.767258+0000","last_became_peered":"2024-03-30T00:39:50.767258+0000","last_unstale":"2024-03-30T00:39:55.171239+0000","last_undegraded":"2024-03-30T00:39:55.171239+0000","last_fullsized":"2024-03-30T00:39:55.171239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183786+0000","last_change":"2024-03-30T00:39:51.173636+0000","last_active":"2024-03-30T00:39:55.183786+0000","last_peered":"2024-03-30T00:39:55.183786+0000","last_clean":"2024-03-30T00:39:55.183786+0000","last_became_active":"2024-03-30T00:39:51.173356+0000","last_became_peered":"2024-03-30T00:39:51.173356+0000","last_unstale":"2024-03-30T00:39:55.183786+0000","last_undegraded":"2024-03-30T00:39:55.183786+0000","last_fullsized":"2024-03-30T00:39:55.183786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:10:20.544219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273686+0000","last_change":"2024-03-30T00:39:55.187062+0000","last_active":"2024-03-30T00:39:57.273686+0000","last_peered":"2024-03-30T00:39:57.273686+0000","last_clean":"2024-03-30T00:39:57.273686+0000","last_became_active":"2024-03-30T00:39:55.184977+0000","last_became_peered":"2024-03-30T00:39:55.184977+0000","last_unstale":"2024-03-30T00:39:57.273686+0000","last_undegraded":"2024-03-30T00:39:57.273686+0000","last_fullsized":"2024-03-30T00:39:57.273686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:23.899767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169318+0000","last_change":"2024-03-30T00:39:50.767360+0000","last_active":"2024-03-30T00:39:55.169318+0000","last_peered":"2024-03-30T00:39:55.169318+0000","last_clean":"2024-03-30T00:39:55.169318+0000","last_became_active":"2024-03-30T00:39:50.764301+0000","last_became_peered":"2024-03-30T00:39:50.764301+0000","last_unstale":"2024-03-30T00:39:55.169318+0000","last_undegraded":"2024-03-30T00:39:55.169318+0000","last_fullsized":"2024-03-30T00:39:55.169318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:26.877416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160985+0000","last_change":"2024-03-30T00:39:51.163323+0000","last_active":"2024-03-30T00:39:55.160985+0000","last_peered":"2024-03-30T00:39:55.160985+0000","last_clean":"2024-03-30T00:39:55.160985+0000","last_became_active":"2024-03-30T00:39:51.163080+0000","last_became_peered":"2024-03-30T00:39:51.163080+0000","last_unstale":"2024-03-30T00:39:55.160985+0000","last_undegraded":"2024-03-30T00:39:55.160985+0000","last_fullsized":"2024-03-30T00:39:55.160985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:28.307469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572341+0000","last_change":"2024-03-30T00:39:55.194121+0000","last_active":"2024-03-30T00:39:55.572341+0000","last_peered":"2024-03-30T00:39:55.572341+0000","last_clean":"2024-03-30T00:39:55.572341+0000","last_became_active":"2024-03-30T00:39:55.190121+0000","last_became_peered":"2024-03-30T00:39:55.190121+0000","last_unstale":"2024-03-30T00:39:55.572341+0000","last_undegraded":"2024-03-30T00:39:55.572341+0000","last_fullsized":"2024-03-30T00:39:55.572341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:10:34.870687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169878+0000","last_change":"2024-03-30T00:39:51.168112+0000","last_active":"2024-03-30T00:39:55.169878+0000","last_peered":"2024-03-30T00:39:55.169878+0000","last_clean":"2024-03-30T00:39:55.169878+0000","last_became_active":"2024-03-30T00:39:51.167770+0000","last_became_peered":"2024-03-30T00:39:51.167770+0000","last_unstale":"2024-03-30T00:39:55.169878+0000","last_undegraded":"2024-03-30T00:39:55.169878+0000","last_fullsized":"2024-03-30T00:39:55.169878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159083+0000","last_change":"2024-03-30T00:39:51.169867+0000","last_active":"2024-03-30T00:39:55.159083+0000","last_peered":"2024-03-30T00:39:55.159083+0000","last_clean":"2024-03-30T00:39:55.159083+0000","last_became_active":"2024-03-30T00:39:51.169515+0000","last_became_peered":"2024-03-30T00:39:51.169515+0000","last_unstale":"2024-03-30T00:39:55.159083+0000","last_undegraded":"2024-03-30T00:39:55.159083+0000","last_fullsized":"2024-03-30T00:39:55.159083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:44.962672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168036+0000","last_change":"2024-03-30T00:39:51.175788+0000","last_active":"2024-03-30T00:39:55.168036+0000","last_peered":"2024-03-30T00:39:55.168036+0000","last_clean":"2024-03-30T00:39:55.168036+0000","last_became_active":"2024-03-30T00:39:51.175288+0000","last_became_peered":"2024-03-30T00:39:51.175288+0000","last_unstale":"2024-03-30T00:39:55.168036+0000","last_undegraded":"2024-03-30T00:39:55.168036+0000","last_fullsized":"2024-03-30T00:39:55.168036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:35.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183710+0000","last_change":"2024-03-30T00:39:51.173618+0000","last_active":"2024-03-30T00:39:55.183710+0000","last_peered":"2024-03-30T00:39:55.183710+0000","last_clean":"2024-03-30T00:39:55.183710+0000","last_became_active":"2024-03-30T00:39:51.173324+0000","last_became_peered":"2024-03-30T00:39:51.173324+0000","last_unstale":"2024-03-30T00:39:55.183710+0000","last_undegraded":"2024-03-30T00:39:55.183710+0000","last_fullsized":"2024-03-30T00:39:55.183710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168094+0000","last_change":"2024-03-30T00:39:51.184101+0000","last_active":"2024-03-30T00:39:55.168094+0000","last_peered":"2024-03-30T00:39:55.168094+0000","last_clean":"2024-03-30T00:39:55.168094+0000","last_became_active":"2024-03-30T00:39:51.183843+0000","last_became_peered":"2024-03-30T00:39:51.183843+0000","last_unstale":"2024-03-30T00:39:55.168094+0000","last_undegraded":"2024-03-30T00:39:55.168094+0000","last_fullsized":"2024-03-30T00:39:55.168094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:51.868242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172390+0000","last_change":"2024-03-30T00:39:51.174001+0000","last_active":"2024-03-30T00:39:55.172390+0000","last_peered":"2024-03-30T00:39:55.172390+0000","last_clean":"2024-03-30T00:39:55.172390+0000","last_became_active":"2024-03-30T00:39:51.173753+0000","last_became_peered":"2024-03-30T00:39:51.173753+0000","last_unstale":"2024-03-30T00:39:55.172390+0000","last_undegraded":"2024-03-30T00:39:55.172390+0000","last_fullsized":"2024-03-30T00:39:55.172390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171728+0000","last_change":"2024-03-30T00:39:51.165845+0000","last_active":"2024-03-30T00:39:55.171728+0000","last_peered":"2024-03-30T00:39:55.171728+0000","last_clean":"2024-03-30T00:39:55.171728+0000","last_became_active":"2024-03-30T00:39:51.164657+0000","last_became_peered":"2024-03-30T00:39:51.164657+0000","last_unstale":"2024-03-30T00:39:55.171728+0000","last_undegraded":"2024-03-30T00:39:55.171728+0000","last_fullsized":"2024-03-30T00:39:55.171728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.164918+0000","last_change":"2024-03-30T00:39:51.185222+0000","last_active":"2024-03-30T00:39:55.164918+0000","last_peered":"2024-03-30T00:39:55.164918+0000","last_clean":"2024-03-30T00:39:55.164918+0000","last_became_active":"2024-03-30T00:39:51.184954+0000","last_became_peered":"2024-03-30T00:39:51.184954+0000","last_unstale":"2024-03-30T00:39:55.164918+0000","last_undegraded":"2024-03-30T00:39:55.164918+0000","last_fullsized":"2024-03-30T00:39:55.164918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167766+0000","last_change":"2024-03-30T00:39:51.174598+0000","last_active":"2024-03-30T00:39:55.167766+0000","last_peered":"2024-03-30T00:39:55.167766+0000","last_clean":"2024-03-30T00:39:55.167766+0000","last_became_active":"2024-03-30T00:39:51.174250+0000","last_became_peered":"2024-03-30T00:39:51.174250+0000","last_unstale":"2024-03-30T00:39:55.167766+0000","last_undegraded":"2024-03-30T00:39:55.167766+0000","last_fullsized":"2024-03-30T00:39:55.167766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165202+0000","last_change":"2024-03-30T00:39:51.184460+0000","last_active":"2024-03-30T00:39:55.165202+0000","last_peered":"2024-03-30T00:39:55.165202+0000","last_clean":"2024-03-30T00:39:55.165202+0000","last_became_active":"2024-03-30T00:39:51.184185+0000","last_became_peered":"2024-03-30T00:39:51.184185+0000","last_unstale":"2024-03-30T00:39:55.165202+0000","last_undegraded":"2024-03-30T00:39:55.165202+0000","last_fullsized":"2024-03-30T00:39:55.165202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167944+0000","last_change":"2024-03-30T00:39:51.168064+0000","last_active":"2024-03-30T00:39:55.167944+0000","last_peered":"2024-03-30T00:39:55.167944+0000","last_clean":"2024-03-30T00:39:55.167944+0000","last_became_active":"2024-03-30T00:39:51.167772+0000","last_became_peered":"2024-03-30T00:39:51.167772+0000","last_unstale":"2024-03-30T00:39:55.167944+0000","last_undegraded":"2024-03-30T00:39:55.167944+0000","last_fullsized":"2024-03-30T00:39:55.167944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:12.512040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183617+0000","last_change":"2024-03-30T00:39:51.183269+0000","last_active":"2024-03-30T00:39:55.183617+0000","last_peered":"2024-03-30T00:39:55.183617+0000","last_clean":"2024-03-30T00:39:55.183617+0000","last_became_active":"2024-03-30T00:39:51.182687+0000","last_became_peered":"2024-03-30T00:39:51.182687+0000","last_unstale":"2024-03-30T00:39:55.183617+0000","last_undegraded":"2024-03-30T00:39:55.183617+0000","last_fullsized":"2024-03-30T00:39:55.183617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171193+0000","last_change":"2024-03-30T00:39:51.171426+0000","last_active":"2024-03-30T00:39:55.171193+0000","last_peered":"2024-03-30T00:39:55.171193+0000","last_clean":"2024-03-30T00:39:55.171193+0000","last_became_active":"2024-03-30T00:39:51.171182+0000","last_became_peered":"2024-03-30T00:39:51.171182+0000","last_unstale":"2024-03-30T00:39:55.171193+0000","last_undegraded":"2024-03-30T00:39:55.171193+0000","last_fullsized":"2024-03-30T00:39:55.171193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:51:41.648914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163087+0000","last_change":"2024-03-30T00:39:51.159899+0000","last_active":"2024-03-30T00:39:55.163087+0000","last_peered":"2024-03-30T00:39:55.163087+0000","last_clean":"2024-03-30T00:39:55.163087+0000","last_became_active":"2024-03-30T00:39:51.159626+0000","last_became_peered":"2024-03-30T00:39:51.159626+0000","last_unstale":"2024-03-30T00:39:55.163087+0000","last_undegraded":"2024-03-30T00:39:55.163087+0000","last_fullsized":"2024-03-30T00:39:55.163087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159252+0000","last_change":"2024-03-30T00:39:51.171375+0000","last_active":"2024-03-30T00:39:55.159252+0000","last_peered":"2024-03-30T00:39:55.159252+0000","last_clean":"2024-03-30T00:39:55.159252+0000","last_became_active":"2024-03-30T00:39:51.170689+0000","last_became_peered":"2024-03-30T00:39:51.170689+0000","last_unstale":"2024-03-30T00:39:55.159252+0000","last_undegraded":"2024-03-30T00:39:55.159252+0000","last_fullsized":"2024-03-30T00:39:55.159252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:27.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167585+0000","last_change":"2024-03-30T00:39:51.164848+0000","last_active":"2024-03-30T00:39:55.167585+0000","last_peered":"2024-03-30T00:39:55.167585+0000","last_clean":"2024-03-30T00:39:55.167585+0000","last_became_active":"2024-03-30T00:39:51.164575+0000","last_became_peered":"2024-03-30T00:39:51.164575+0000","last_unstale":"2024-03-30T00:39:55.167585+0000","last_undegraded":"2024-03-30T00:39:55.167585+0000","last_fullsized":"2024-03-30T00:39:55.167585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167681+0000","last_change":"2024-03-30T00:39:51.173535+0000","last_active":"2024-03-30T00:39:55.167681+0000","last_peered":"2024-03-30T00:39:55.167681+0000","last_clean":"2024-03-30T00:39:55.167681+0000","last_became_active":"2024-03-30T00:39:51.173313+0000","last_became_peered":"2024-03-30T00:39:51.173313+0000","last_unstale":"2024-03-30T00:39:55.167681+0000","last_undegraded":"2024-03-30T00:39:55.167681+0000","last_fullsized":"2024-03-30T00:39:55.167681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171944+0000","last_change":"2024-03-30T00:39:51.171986+0000","last_active":"2024-03-30T00:39:55.171944+0000","last_peered":"2024-03-30T00:39:55.171944+0000","last_clean":"2024-03-30T00:39:55.171944+0000","last_became_active":"2024-03-30T00:39:51.164756+0000","last_became_peered":"2024-03-30T00:39:51.164756+0000","last_unstale":"2024-03-30T00:39:55.171944+0000","last_undegraded":"2024-03-30T00:39:55.171944+0000","last_fullsized":"2024-03-30T00:39:55.171944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184365+0000","last_change":"2024-03-30T00:39:51.181455+0000","last_active":"2024-03-30T00:39:55.184365+0000","last_peered":"2024-03-30T00:39:55.184365+0000","last_clean":"2024-03-30T00:39:55.184365+0000","last_became_active":"2024-03-30T00:39:51.181207+0000","last_became_peered":"2024-03-30T00:39:51.181207+0000","last_unstale":"2024-03-30T00:39:55.184365+0000","last_undegraded":"2024-03-30T00:39:55.184365+0000","last_fullsized":"2024-03-30T00:39:55.184365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:01:48.059238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169138+0000","last_change":"2024-03-30T00:39:51.175074+0000","last_active":"2024-03-30T00:39:55.169138+0000","last_peered":"2024-03-30T00:39:55.169138+0000","last_clean":"2024-03-30T00:39:55.169138+0000","last_became_active":"2024-03-30T00:39:51.174836+0000","last_became_peered":"2024-03-30T00:39:51.174836+0000","last_unstale":"2024-03-30T00:39:55.169138+0000","last_undegraded":"2024-03-30T00:39:55.169138+0000","last_fullsized":"2024-03-30T00:39:55.169138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162196+0000","last_change":"2024-03-30T00:39:51.169964+0000","last_active":"2024-03-30T00:39:55.162196+0000","last_peered":"2024-03-30T00:39:55.162196+0000","last_clean":"2024-03-30T00:39:55.162196+0000","last_became_active":"2024-03-30T00:39:51.169647+0000","last_became_peered":"2024-03-30T00:39:51.169647+0000","last_unstale":"2024-03-30T00:39:55.162196+0000","last_undegraded":"2024-03-30T00:39:55.162196+0000","last_fullsized":"2024-03-30T00:39:55.162196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:44.962672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171503+0000","last_change":"2024-03-30T00:39:51.178908+0000","last_active":"2024-03-30T00:39:55.171503+0000","last_peered":"2024-03-30T00:39:55.171503+0000","last_clean":"2024-03-30T00:39:55.171503+0000","last_became_active":"2024-03-30T00:39:51.178663+0000","last_became_peered":"2024-03-30T00:39:51.178663+0000","last_unstale":"2024-03-30T00:39:55.171503+0000","last_undegraded":"2024-03-30T00:39:55.171503+0000","last_fullsized":"2024-03-30T00:39:55.171503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171643+0000","last_change":"2024-03-30T00:39:51.179913+0000","last_active":"2024-03-30T00:39:55.171643+0000","last_peered":"2024-03-30T00:39:55.171643+0000","last_clean":"2024-03-30T00:39:55.171643+0000","last_became_active":"2024-03-30T00:39:51.179652+0000","last_became_peered":"2024-03-30T00:39:51.179652+0000","last_unstale":"2024-03-30T00:39:55.171643+0000","last_undegraded":"2024-03-30T00:39:55.171643+0000","last_fullsized":"2024-03-30T00:39:55.171643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163237+0000","last_change":"2024-03-30T00:39:51.170941+0000","last_active":"2024-03-30T00:39:55.163237+0000","last_peered":"2024-03-30T00:39:55.163237+0000","last_clean":"2024-03-30T00:39:55.163237+0000","last_became_active":"2024-03-30T00:39:51.166779+0000","last_became_peered":"2024-03-30T00:39:51.166779+0000","last_unstale":"2024-03-30T00:39:55.163237+0000","last_undegraded":"2024-03-30T00:39:55.163237+0000","last_fullsized":"2024-03-30T00:39:55.163237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:12.512040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167960+0000","last_change":"2024-03-30T00:39:51.176410+0000","last_active":"2024-03-30T00:39:55.167960+0000","last_peered":"2024-03-30T00:39:55.167960+0000","last_clean":"2024-03-30T00:39:55.167960+0000","last_became_active":"2024-03-30T00:39:51.175971+0000","last_became_peered":"2024-03-30T00:39:51.175971+0000","last_unstale":"2024-03-30T00:39:55.167960+0000","last_undegraded":"2024-03-30T00:39:55.167960+0000","last_fullsized":"2024-03-30T00:39:55.167960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160661+0000","last_change":"2024-03-30T00:39:51.167648+0000","last_active":"2024-03-30T00:39:55.160661+0000","last_peered":"2024-03-30T00:39:55.160661+0000","last_clean":"2024-03-30T00:39:55.160661+0000","last_became_active":"2024-03-30T00:39:51.167382+0000","last_became_peered":"2024-03-30T00:39:51.167382+0000","last_unstale":"2024-03-30T00:39:55.160661+0000","last_undegraded":"2024-03-30T00:39:55.160661+0000","last_fullsized":"2024-03-30T00:39:55.160661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:39:43.189821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167861+0000","last_change":"2024-03-30T00:39:51.169776+0000","last_active":"2024-03-30T00:39:55.167861+0000","last_peered":"2024-03-30T00:39:55.167861+0000","last_clean":"2024-03-30T00:39:55.167861+0000","last_became_active":"2024-03-30T00:39:51.169462+0000","last_became_peered":"2024-03-30T00:39:51.169462+0000","last_unstale":"2024-03-30T00:39:55.167861+0000","last_undegraded":"2024-03-30T00:39:55.167861+0000","last_fullsized":"2024-03-30T00:39:55.167861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:37:05.851113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161236+0000","last_change":"2024-03-30T00:39:51.163469+0000","last_active":"2024-03-30T00:39:55.161236+0000","last_peered":"2024-03-30T00:39:55.161236+0000","last_clean":"2024-03-30T00:39:55.161236+0000","last_became_active":"2024-03-30T00:39:51.163200+0000","last_became_peered":"2024-03-30T00:39:51.163200+0000","last_unstale":"2024-03-30T00:39:55.161236+0000","last_undegraded":"2024-03-30T00:39:55.161236+0000","last_fullsized":"2024-03-30T00:39:55.161236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:24:05.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170209+0000","last_change":"2024-03-30T00:39:51.183006+0000","last_active":"2024-03-30T00:39:55.170209+0000","last_peered":"2024-03-30T00:39:55.170209+0000","last_clean":"2024-03-30T00:39:55.170209+0000","last_became_active":"2024-03-30T00:39:51.179310+0000","last_became_peered":"2024-03-30T00:39:51.179310+0000","last_unstale":"2024-03-30T00:39:55.170209+0000","last_undegraded":"2024-03-30T00:39:55.170209+0000","last_fullsized":"2024-03-30T00:39:55.170209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:41.330935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183618+0000","last_change":"2024-03-30T00:39:51.180621+0000","last_active":"2024-03-30T00:39:55.183618+0000","last_peered":"2024-03-30T00:39:55.183618+0000","last_clean":"2024-03-30T00:39:55.183618+0000","last_became_active":"2024-03-30T00:39:51.175749+0000","last_became_peered":"2024-03-30T00:39:51.175749+0000","last_unstale":"2024-03-30T00:39:55.183618+0000","last_undegraded":"2024-03-30T00:39:55.183618+0000","last_fullsized":"2024-03-30T00:39:55.183618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:27.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171993+0000","last_change":"2024-03-30T00:39:51.168331+0000","last_active":"2024-03-30T00:39:55.171993+0000","last_peered":"2024-03-30T00:39:55.171993+0000","last_clean":"2024-03-30T00:39:55.171993+0000","last_became_active":"2024-03-30T00:39:51.168052+0000","last_became_peered":"2024-03-30T00:39:51.168052+0000","last_unstale":"2024-03-30T00:39:55.171993+0000","last_undegraded":"2024-03-30T00:39:55.171993+0000","last_fullsized":"2024-03-30T00:39:55.171993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163158+0000","last_change":"2024-03-30T00:39:51.167745+0000","last_active":"2024-03-30T00:39:55.163158+0000","last_peered":"2024-03-30T00:39:55.163158+0000","last_clean":"2024-03-30T00:39:55.163158+0000","last_became_active":"2024-03-30T00:39:51.166397+0000","last_became_peered":"2024-03-30T00:39:51.166397+0000","last_unstale":"2024-03-30T00:39:55.163158+0000","last_undegraded":"2024-03-30T00:39:55.163158+0000","last_fullsized":"2024-03-30T00:39:55.163158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:50:20.941634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0490000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3170000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:40:20.335 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:40:20.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.501+0000 7f9987e00640 1 Processor -- start 2024-03-30T00:40:20.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.501+0000 7f9987e00640 1 -- start start 2024-03-30T00:40:20.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9987e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f998015ca80 0x7f998006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9987e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99800cf2f0 0x7f998006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9987e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99800cb0e0 con 0x7f998006bb30 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9987e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99800cf7f0 con 0x7f998015ca80 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9987e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99800cfad0 con 0x7f99800cf2f0 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9985400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f998015ca80 0x7f998006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9985400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f998015ca80 0x7f998006b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34038/0 (socket says 172.21.15.66:34038) 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9985400640 1 -- 172.21.15.66:0/4126815810 learned_addr learned my addr 172.21.15.66:0/4126815810 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9984a00640 1 --2- 172.21.15.66:0/4126815810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99800cf2f0 0x7f998006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9985400640 1 -- 172.21.15.66:0/4126815810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99800cf2f0 msgr2=0x7f998006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9985400640 1 --2- 172.21.15.66:0/4126815810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99800cf2f0 0x7f998006b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9985400640 1 -- 172.21.15.66:0/4126815810 >> v1:172.21.15.156:6789/0 conn(0x7f998006bb30 legacy=0x7f9980098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9985400640 1 -- 172.21.15.66:0/4126815810 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99800cad70 con 0x7f998015ca80 2024-03-30T00:40:20.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.505+0000 7f9985400640 1 --2- 172.21.15.66:0/4126815810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f998015ca80 0x7f998006b0b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9974009870 tx=0x7f997401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc60feb4b6635fce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:20.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f997fe00640 1 -- 172.21.15.66:0/4126815810 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740a8a50 con 0x7f998015ca80 2024-03-30T00:40:20.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f997fe00640 1 -- 172.21.15.66:0/4126815810 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99740a8bb0 con 0x7f998015ca80 2024-03-30T00:40:20.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f997fe00640 1 -- 172.21.15.66:0/4126815810 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740b0440 con 0x7f998015ca80 2024-03-30T00:40:20.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 -- 172.21.15.66:0/4126815810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f998015ca80 msgr2=0x7f998006b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 --2- 172.21.15.66:0/4126815810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f998015ca80 0x7f998006b0b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9974009870 tx=0x7f997401ed50 comp rx=0 tx=0).stop 2024-03-30T00:40:20.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 -- 172.21.15.66:0/4126815810 shutdown_connections 2024-03-30T00:40:20.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 --2- 172.21.15.66:0/4126815810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99800cf2f0 0x7f998006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 --2- 172.21.15.66:0/4126815810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f998015ca80 0x7f998006b0b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 -- 172.21.15.66:0/4126815810 >> 172.21.15.66:0/4126815810 conn(0x7f99800b2090 msgr2=0x7f99800b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:20.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 -- 172.21.15.66:0/4126815810 shutdown_connections 2024-03-30T00:40:20.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 -- 172.21.15.66:0/4126815810 wait complete. 2024-03-30T00:40:20.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 Processor -- start 2024-03-30T00:40:20.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 -- start start 2024-03-30T00:40:20.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f998006bb30 0x7f9980158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99800cf2f0 0x7f99800c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f998015ca80 0x7f99800b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99800c8f70 con 0x7f998015ca80 2024-03-30T00:40:20.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99800c9250 con 0x7f998006bb30 2024-03-30T00:40:20.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9987e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99800c9530 con 0x7f99800cf2f0 2024-03-30T00:40:20.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9985e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f998015ca80 0x7f99800b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9985400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f998006bb30 0x7f9980158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9985e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f998015ca80 0x7f99800b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34054/0 (socket says 172.21.15.66:34054) 2024-03-30T00:40:20.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9985e00640 1 -- 172.21.15.66:0/3671691199 learned_addr learned my addr 172.21.15.66:0/3671691199 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:20.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.509+0000 7f9984a00640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99800cf2f0 0x7f99800c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f9985e00640 1 -- 172.21.15.66:0/3671691199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99800cf2f0 msgr2=0x7f99800c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f9985e00640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99800cf2f0 0x7f99800c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f9985e00640 1 -- 172.21.15.66:0/3671691199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f998006bb30 msgr2=0x7f9980158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f9985e00640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f998006bb30 0x7f9980158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f9985e00640 1 -- 172.21.15.66:0/3671691199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740094d0 con 0x7f998015ca80 2024-03-30T00:40:20.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f9985e00640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f998015ca80 0x7f99800b4360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f997800c770 tx=0x7f997800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:20.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f997d600640 1 -- 172.21.15.66:0/3671691199 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978003b80 con 0x7f998015ca80 2024-03-30T00:40:20.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f997d600640 1 -- 172.21.15.66:0/3671691199 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9978003ce0 con 0x7f998015ca80 2024-03-30T00:40:20.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f997d600640 1 -- 172.21.15.66:0/3671691199 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997800aa90 con 0x7f998015ca80 2024-03-30T00:40:20.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f9987e00640 1 -- 172.21.15.66:0/3671691199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99800c9810 con 0x7f998015ca80 2024-03-30T00:40:20.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f9987e00640 1 -- 172.21.15.66:0/3671691199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99801590f0 con 0x7f998015ca80 2024-03-30T00:40:20.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f9987e00640 1 -- 172.21.15.66:0/3671691199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998006fb20 con 0x7f998015ca80 2024-03-30T00:40:20.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.513+0000 7f997d600640 1 -- 172.21.15.66:0/3671691199 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f997800abf0 con 0x7f998015ca80 2024-03-30T00:40:20.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.517+0000 7f997d600640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9960066690 0x7f9960068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.517+0000 7f997d600640 1 -- 172.21.15.66:0/3671691199 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9978084220 con 0x7f998015ca80 2024-03-30T00:40:20.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.517+0000 7f9985400640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9960066690 0x7f9960068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.517+0000 7f9985400640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9960066690 0x7f9960068b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99740b2000 tx=0x7f9974009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:20.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.521+0000 7f997d600640 1 -- 172.21.15.66:0/3671691199 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f99780512b0 con 0x7f998015ca80 2024-03-30T00:40:20.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.677+0000 7f9987e00640 1 -- 172.21.15.66:0/3671691199 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99800cc580 con 0x7f9960066690 2024-03-30T00:40:20.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.685+0000 7f997d600640 1 -- 172.21.15.66:0/3671691199 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345229 (secure 0 0 0) 0x7f99740b04c0 con 0x7f9960066690 2024-03-30T00:40:20.700 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:20.702 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:40:20.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.689+0000 7f9987e00640 1 -- 172.21.15.66:0/3671691199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9960066690 msgr2=0x7f9960068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.689+0000 7f9987e00640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9960066690 0x7f9960068b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99740b2000 tx=0x7f9974009040 comp rx=0 tx=0).stop 2024-03-30T00:40:20.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.693+0000 7f9987e00640 1 -- 172.21.15.66:0/3671691199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f998015ca80 msgr2=0x7f99800b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.693+0000 7f9987e00640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f998015ca80 0x7f99800b4360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f997800c770 tx=0x7f997800cc30 comp rx=0 tx=0).stop 2024-03-30T00:40:20.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.693+0000 7f9987e00640 1 -- 172.21.15.66:0/3671691199 shutdown_connections 2024-03-30T00:40:20.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.693+0000 7f9987e00640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9960066690 0x7f9960068b50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.693+0000 7f9987e00640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f998015ca80 0x7f99800b4360 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.693+0000 7f9987e00640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99800cf2f0 0x7f99800c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.693+0000 7f9987e00640 1 --2- 172.21.15.66:0/3671691199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f998006bb30 0x7f9980158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.693+0000 7f9987e00640 1 -- 172.21.15.66:0/3671691199 >> 172.21.15.66:0/3671691199 conn(0x7f99800b2090 msgr2=0x7f99800ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:20.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.693+0000 7f9987e00640 1 -- 172.21.15.66:0/3671691199 shutdown_connections 2024-03-30T00:40:20.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.693+0000 7f9987e00640 1 -- 172.21.15.66:0/3671691199 wait complete. 2024-03-30T00:40:20.720 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-03-30T00:40:18.856621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235864,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738856,"statfs":{"total":773094113280,"available":772852588544,"internally_reserved":0,"allocated":3383296,"data_stored":2025532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002905"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184484+0000","last_change":"2024-03-30T00:39:50.768525+0000","last_active":"2024-03-30T00:39:55.184484+0000","last_peered":"2024-03-30T00:39:55.184484+0000","last_clean":"2024-03-30T00:39:55.184484+0000","last_became_active":"2024-03-30T00:39:50.768196+0000","last_became_peered":"2024-03-30T00:39:50.768196+0000","last_unstale":"2024-03-30T00:39:55.184484+0000","last_undegraded":"2024-03-30T00:39:55.184484+0000","last_fullsized":"2024-03-30T00:39:55.184484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:26.877416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168811+0000","last_change":"2024-03-30T00:39:51.184250+0000","last_active":"2024-03-30T00:39:55.168811+0000","last_peered":"2024-03-30T00:39:55.168811+0000","last_clean":"2024-03-30T00:39:55.168811+0000","last_became_active":"2024-03-30T00:39:51.184025+0000","last_became_peered":"2024-03-30T00:39:51.184025+0000","last_unstale":"2024-03-30T00:39:55.168811+0000","last_undegraded":"2024-03-30T00:39:55.168811+0000","last_fullsized":"2024-03-30T00:39:55.168811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:49.107527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571479+0000","last_change":"2024-03-30T00:39:55.177962+0000","last_active":"2024-03-30T00:39:55.571479+0000","last_peered":"2024-03-30T00:39:55.571479+0000","last_clean":"2024-03-30T00:39:55.571479+0000","last_became_active":"2024-03-30T00:39:55.177403+0000","last_became_peered":"2024-03-30T00:39:55.177403+0000","last_unstale":"2024-03-30T00:39:55.571479+0000","last_undegraded":"2024-03-30T00:39:55.571479+0000","last_fullsized":"2024-03-30T00:39:55.571479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:44:30.994000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168955+0000","last_change":"2024-03-30T00:39:50.765754+0000","last_active":"2024-03-30T00:39:55.168955+0000","last_peered":"2024-03-30T00:39:55.168955+0000","last_clean":"2024-03-30T00:39:55.168955+0000","last_became_active":"2024-03-30T00:39:50.764611+0000","last_became_peered":"2024-03-30T00:39:50.764611+0000","last_unstale":"2024-03-30T00:39:55.168955+0000","last_undegraded":"2024-03-30T00:39:55.168955+0000","last_fullsized":"2024-03-30T00:39:55.168955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:20.857915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184332+0000","last_change":"2024-03-30T00:39:51.180718+0000","last_active":"2024-03-30T00:39:55.184332+0000","last_peered":"2024-03-30T00:39:55.184332+0000","last_clean":"2024-03-30T00:39:55.184332+0000","last_became_active":"2024-03-30T00:39:51.175872+0000","last_became_peered":"2024-03-30T00:39:51.175872+0000","last_unstale":"2024-03-30T00:39:55.184332+0000","last_undegraded":"2024-03-30T00:39:55.184332+0000","last_fullsized":"2024-03-30T00:39:55.184332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272688+0000","last_change":"2024-03-30T00:39:55.184652+0000","last_active":"2024-03-30T00:39:57.272688+0000","last_peered":"2024-03-30T00:39:57.272688+0000","last_clean":"2024-03-30T00:39:57.272688+0000","last_became_active":"2024-03-30T00:39:55.184290+0000","last_became_peered":"2024-03-30T00:39:55.184290+0000","last_unstale":"2024-03-30T00:39:57.272688+0000","last_undegraded":"2024-03-30T00:39:57.272688+0000","last_fullsized":"2024-03-30T00:39:57.272688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:15:03.826593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163001+0000","last_change":"2024-03-30T00:39:50.771435+0000","last_active":"2024-03-30T00:39:55.163001+0000","last_peered":"2024-03-30T00:39:55.163001+0000","last_clean":"2024-03-30T00:39:55.163001+0000","last_became_active":"2024-03-30T00:39:50.771153+0000","last_became_peered":"2024-03-30T00:39:50.771153+0000","last_unstale":"2024-03-30T00:39:55.163001+0000","last_undegraded":"2024-03-30T00:39:55.163001+0000","last_fullsized":"2024-03-30T00:39:55.163001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:43.959952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170301+0000","last_change":"2024-03-30T00:39:51.183510+0000","last_active":"2024-03-30T00:39:55.170301+0000","last_peered":"2024-03-30T00:39:55.170301+0000","last_clean":"2024-03-30T00:39:55.170301+0000","last_became_active":"2024-03-30T00:39:51.183236+0000","last_became_peered":"2024-03-30T00:39:51.183236+0000","last_unstale":"2024-03-30T00:39:55.170301+0000","last_undegraded":"2024-03-30T00:39:55.170301+0000","last_fullsized":"2024-03-30T00:39:55.170301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273177+0000","last_change":"2024-03-30T00:39:55.179167+0000","last_active":"2024-03-30T00:39:57.273177+0000","last_peered":"2024-03-30T00:39:57.273177+0000","last_clean":"2024-03-30T00:39:57.273177+0000","last_became_active":"2024-03-30T00:39:55.178708+0000","last_became_peered":"2024-03-30T00:39:55.178708+0000","last_unstale":"2024-03-30T00:39:57.273177+0000","last_undegraded":"2024-03-30T00:39:57.273177+0000","last_fullsized":"2024-03-30T00:39:57.273177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161291+0000","last_change":"2024-03-30T00:39:50.773829+0000","last_active":"2024-03-30T00:39:55.161291+0000","last_peered":"2024-03-30T00:39:55.161291+0000","last_clean":"2024-03-30T00:39:55.161291+0000","last_became_active":"2024-03-30T00:39:50.773527+0000","last_became_peered":"2024-03-30T00:39:50.773527+0000","last_unstale":"2024-03-30T00:39:55.161291+0000","last_undegraded":"2024-03-30T00:39:55.161291+0000","last_fullsized":"2024-03-30T00:39:55.161291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.179506+0000","last_change":"2024-03-30T00:39:51.183689+0000","last_active":"2024-03-30T00:39:55.179506+0000","last_peered":"2024-03-30T00:39:55.179506+0000","last_clean":"2024-03-30T00:39:55.179506+0000","last_became_active":"2024-03-30T00:39:51.183421+0000","last_became_peered":"2024-03-30T00:39:51.183421+0000","last_unstale":"2024-03-30T00:39:55.179506+0000","last_undegraded":"2024-03-30T00:39:55.179506+0000","last_fullsized":"2024-03-30T00:39:55.179506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:54.542537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272671+0000","last_change":"2024-03-30T00:39:55.186210+0000","last_active":"2024-03-30T00:39:57.272671+0000","last_peered":"2024-03-30T00:39:57.272671+0000","last_clean":"2024-03-30T00:39:57.272671+0000","last_became_active":"2024-03-30T00:39:55.185849+0000","last_became_peered":"2024-03-30T00:39:55.185849+0000","last_unstale":"2024-03-30T00:39:57.272671+0000","last_undegraded":"2024-03-30T00:39:57.272671+0000","last_fullsized":"2024-03-30T00:39:57.272671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:45:20.370665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171140+0000","last_change":"2024-03-30T00:39:51.172454+0000","last_active":"2024-03-30T00:39:55.171140+0000","last_peered":"2024-03-30T00:39:55.171140+0000","last_clean":"2024-03-30T00:39:55.171140+0000","last_became_active":"2024-03-30T00:39:51.172196+0000","last_became_peered":"2024-03-30T00:39:51.172196+0000","last_unstale":"2024-03-30T00:39:55.171140+0000","last_undegraded":"2024-03-30T00:39:55.171140+0000","last_fullsized":"2024-03-30T00:39:55.171140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:02.372510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161442+0000","last_change":"2024-03-30T00:39:50.764668+0000","last_active":"2024-03-30T00:39:55.161442+0000","last_peered":"2024-03-30T00:39:55.161442+0000","last_clean":"2024-03-30T00:39:55.161442+0000","last_became_active":"2024-03-30T00:39:50.763661+0000","last_became_peered":"2024-03-30T00:39:50.763661+0000","last_unstale":"2024-03-30T00:39:55.161442+0000","last_undegraded":"2024-03-30T00:39:55.161442+0000","last_fullsized":"2024-03-30T00:39:55.161442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273301+0000","last_change":"2024-03-30T00:39:55.190348+0000","last_active":"2024-03-30T00:39:57.273301+0000","last_peered":"2024-03-30T00:39:57.273301+0000","last_clean":"2024-03-30T00:39:57.273301+0000","last_became_active":"2024-03-30T00:39:55.189987+0000","last_became_peered":"2024-03-30T00:39:55.189987+0000","last_unstale":"2024-03-30T00:39:57.273301+0000","last_undegraded":"2024-03-30T00:39:57.273301+0000","last_fullsized":"2024-03-30T00:39:57.273301+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:15:12.303669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169221+0000","last_change":"2024-03-30T00:39:51.168741+0000","last_active":"2024-03-30T00:39:55.169221+0000","last_peered":"2024-03-30T00:39:55.169221+0000","last_clean":"2024-03-30T00:39:55.169221+0000","last_became_active":"2024-03-30T00:39:51.168476+0000","last_became_peered":"2024-03-30T00:39:51.168476+0000","last_unstale":"2024-03-30T00:39:55.169221+0000","last_undegraded":"2024-03-30T00:39:55.169221+0000","last_fullsized":"2024-03-30T00:39:55.169221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:10:20.544219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172219+0000","last_change":"2024-03-30T00:39:50.766156+0000","last_active":"2024-03-30T00:39:55.172219+0000","last_peered":"2024-03-30T00:39:55.172219+0000","last_clean":"2024-03-30T00:39:55.172219+0000","last_became_active":"2024-03-30T00:39:50.765730+0000","last_became_peered":"2024-03-30T00:39:50.765730+0000","last_unstale":"2024-03-30T00:39:55.172219+0000","last_undegraded":"2024-03-30T00:39:55.172219+0000","last_fullsized":"2024-03-30T00:39:55.172219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:27.304749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272886+0000","last_change":"2024-03-30T00:39:55.180712+0000","last_active":"2024-03-30T00:39:57.272886+0000","last_peered":"2024-03-30T00:39:57.272886+0000","last_clean":"2024-03-30T00:39:57.272886+0000","last_became_active":"2024-03-30T00:39:55.179741+0000","last_became_peered":"2024-03-30T00:39:55.179741+0000","last_unstale":"2024-03-30T00:39:57.272886+0000","last_undegraded":"2024-03-30T00:39:57.272886+0000","last_fullsized":"2024-03-30T00:39:57.272886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.166309+0000","last_change":"2024-03-30T00:39:51.182604+0000","last_active":"2024-03-30T00:39:55.166309+0000","last_peered":"2024-03-30T00:39:55.166309+0000","last_clean":"2024-03-30T00:39:55.166309+0000","last_became_active":"2024-03-30T00:39:51.182323+0000","last_became_peered":"2024-03-30T00:39:51.182323+0000","last_unstale":"2024-03-30T00:39:55.166309+0000","last_undegraded":"2024-03-30T00:39:55.166309+0000","last_fullsized":"2024-03-30T00:39:55.166309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172036+0000","last_change":"2024-03-30T00:39:50.758774+0000","last_active":"2024-03-30T00:39:55.172036+0000","last_peered":"2024-03-30T00:39:55.172036+0000","last_clean":"2024-03-30T00:39:55.172036+0000","last_became_active":"2024-03-30T00:39:50.758508+0000","last_became_peered":"2024-03-30T00:39:50.758508+0000","last_unstale":"2024-03-30T00:39:55.172036+0000","last_undegraded":"2024-03-30T00:39:55.172036+0000","last_fullsized":"2024-03-30T00:39:55.172036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.129001+0000","last_change":"2024-03-30T00:39:55.182422+0000","last_active":"2024-03-30T00:39:57.129001+0000","last_peered":"2024-03-30T00:39:57.129001+0000","last_clean":"2024-03-30T00:39:57.129001+0000","last_became_active":"2024-03-30T00:39:55.182026+0000","last_became_peered":"2024-03-30T00:39:55.182026+0000","last_unstale":"2024-03-30T00:39:57.129001+0000","last_undegraded":"2024-03-30T00:39:57.129001+0000","last_fullsized":"2024-03-30T00:39:57.129001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:18:37.750989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160976+0000","last_change":"2024-03-30T00:39:51.161273+0000","last_active":"2024-03-30T00:39:55.160976+0000","last_peered":"2024-03-30T00:39:55.160976+0000","last_clean":"2024-03-30T00:39:55.160976+0000","last_became_active":"2024-03-30T00:39:51.156256+0000","last_became_peered":"2024-03-30T00:39:51.156256+0000","last_unstale":"2024-03-30T00:39:55.160976+0000","last_undegraded":"2024-03-30T00:39:55.160976+0000","last_fullsized":"2024-03-30T00:39:55.160976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:17.102921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159893+0000","last_change":"2024-03-30T00:39:50.768022+0000","last_active":"2024-03-30T00:39:55.159893+0000","last_peered":"2024-03-30T00:39:55.159893+0000","last_clean":"2024-03-30T00:39:55.159893+0000","last_became_active":"2024-03-30T00:39:50.767791+0000","last_became_peered":"2024-03-30T00:39:50.767791+0000","last_unstale":"2024-03-30T00:39:55.159893+0000","last_undegraded":"2024-03-30T00:39:55.159893+0000","last_fullsized":"2024-03-30T00:39:55.159893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:16.100201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273213+0000","last_change":"2024-03-30T00:39:55.176378+0000","last_active":"2024-03-30T00:39:57.273213+0000","last_peered":"2024-03-30T00:39:57.273213+0000","last_clean":"2024-03-30T00:39:57.273213+0000","last_became_active":"2024-03-30T00:39:55.176090+0000","last_became_peered":"2024-03-30T00:39:55.176090+0000","last_unstale":"2024-03-30T00:39:57.273213+0000","last_undegraded":"2024-03-30T00:39:57.273213+0000","last_fullsized":"2024-03-30T00:39:57.273213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184191+0000","last_change":"2024-03-30T00:39:51.181030+0000","last_active":"2024-03-30T00:39:55.184191+0000","last_peered":"2024-03-30T00:39:55.184191+0000","last_clean":"2024-03-30T00:39:55.184191+0000","last_became_active":"2024-03-30T00:39:51.180794+0000","last_became_peered":"2024-03-30T00:39:51.180794+0000","last_unstale":"2024-03-30T00:39:55.184191+0000","last_undegraded":"2024-03-30T00:39:55.184191+0000","last_fullsized":"2024-03-30T00:39:55.184191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:02.372510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172255+0000","last_change":"2024-03-30T00:39:50.771845+0000","last_active":"2024-03-30T00:39:55.172255+0000","last_peered":"2024-03-30T00:39:55.172255+0000","last_clean":"2024-03-30T00:39:55.172255+0000","last_became_active":"2024-03-30T00:39:50.771049+0000","last_became_peered":"2024-03-30T00:39:50.771049+0000","last_unstale":"2024-03-30T00:39:55.172255+0000","last_undegraded":"2024-03-30T00:39:55.172255+0000","last_fullsized":"2024-03-30T00:39:55.172255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:20.857915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272571+0000","last_change":"2024-03-30T00:39:55.174666+0000","last_active":"2024-03-30T00:39:57.272571+0000","last_peered":"2024-03-30T00:39:57.272571+0000","last_clean":"2024-03-30T00:39:57.272571+0000","last_became_active":"2024-03-30T00:39:55.174259+0000","last_became_peered":"2024-03-30T00:39:55.174259+0000","last_unstale":"2024-03-30T00:39:57.272571+0000","last_undegraded":"2024-03-30T00:39:57.272571+0000","last_fullsized":"2024-03-30T00:39:57.272571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165004+0000","last_change":"2024-03-30T00:39:51.178352+0000","last_active":"2024-03-30T00:39:55.165004+0000","last_peered":"2024-03-30T00:39:55.165004+0000","last_clean":"2024-03-30T00:39:55.165004+0000","last_became_active":"2024-03-30T00:39:51.178080+0000","last_became_peered":"2024-03-30T00:39:51.178080+0000","last_unstale":"2024-03-30T00:39:55.165004+0000","last_undegraded":"2024-03-30T00:39:55.165004+0000","last_fullsized":"2024-03-30T00:39:55.165004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172086+0000","last_change":"2024-03-30T00:39:50.768206+0000","last_active":"2024-03-30T00:39:55.172086+0000","last_peered":"2024-03-30T00:39:55.172086+0000","last_clean":"2024-03-30T00:39:55.172086+0000","last_became_active":"2024-03-30T00:39:50.767891+0000","last_became_peered":"2024-03-30T00:39:50.767891+0000","last_unstale":"2024-03-30T00:39:55.172086+0000","last_undegraded":"2024-03-30T00:39:55.172086+0000","last_fullsized":"2024-03-30T00:39:55.172086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:39.163294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571335+0000","last_change":"2024-03-30T00:39:55.192391+0000","last_active":"2024-03-30T00:39:55.571335+0000","last_peered":"2024-03-30T00:39:55.571335+0000","last_clean":"2024-03-30T00:39:55.571335+0000","last_became_active":"2024-03-30T00:39:55.189816+0000","last_became_peered":"2024-03-30T00:39:55.189816+0000","last_unstale":"2024-03-30T00:39:55.571335+0000","last_undegraded":"2024-03-30T00:39:55.571335+0000","last_fullsized":"2024-03-30T00:39:55.571335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:05:10.673253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171550+0000","last_change":"2024-03-30T00:39:51.161793+0000","last_active":"2024-03-30T00:39:55.171550+0000","last_peered":"2024-03-30T00:39:55.171550+0000","last_clean":"2024-03-30T00:39:55.171550+0000","last_became_active":"2024-03-30T00:39:51.161523+0000","last_became_peered":"2024-03-30T00:39:51.161523+0000","last_unstale":"2024-03-30T00:39:55.171550+0000","last_undegraded":"2024-03-30T00:39:55.171550+0000","last_fullsized":"2024-03-30T00:39:55.171550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163175+0000","last_change":"2024-03-30T00:39:50.769035+0000","last_active":"2024-03-30T00:39:55.163175+0000","last_peered":"2024-03-30T00:39:55.163175+0000","last_clean":"2024-03-30T00:39:55.163175+0000","last_became_active":"2024-03-30T00:39:50.768789+0000","last_became_peered":"2024-03-30T00:39:50.768789+0000","last_unstale":"2024-03-30T00:39:55.163175+0000","last_undegraded":"2024-03-30T00:39:55.163175+0000","last_fullsized":"2024-03-30T00:39:55.163175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:39.163294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273625+0000","last_change":"2024-03-30T00:39:55.200272+0000","last_active":"2024-03-30T00:39:57.273625+0000","last_peered":"2024-03-30T00:39:57.273625+0000","last_clean":"2024-03-30T00:39:57.273625+0000","last_became_active":"2024-03-30T00:39:55.199688+0000","last_became_peered":"2024-03-30T00:39:55.199688+0000","last_unstale":"2024-03-30T00:39:57.273625+0000","last_undegraded":"2024-03-30T00:39:57.273625+0000","last_fullsized":"2024-03-30T00:39:57.273625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:30.942593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.174408+0000","last_change":"2024-03-30T00:39:51.174886+0000","last_active":"2024-03-30T00:39:55.174408+0000","last_peered":"2024-03-30T00:39:55.174408+0000","last_clean":"2024-03-30T00:39:55.174408+0000","last_became_active":"2024-03-30T00:39:51.174274+0000","last_became_peered":"2024-03-30T00:39:51.174274+0000","last_unstale":"2024-03-30T00:39:55.174408+0000","last_undegraded":"2024-03-30T00:39:55.174408+0000","last_fullsized":"2024-03-30T00:39:55.174408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:24:05.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171643+0000","last_change":"2024-03-30T00:39:50.769092+0000","last_active":"2024-03-30T00:39:55.171643+0000","last_peered":"2024-03-30T00:39:55.171643+0000","last_clean":"2024-03-30T00:39:55.171643+0000","last_became_active":"2024-03-30T00:39:50.768750+0000","last_became_peered":"2024-03-30T00:39:50.768750+0000","last_unstale":"2024-03-30T00:39:55.171643+0000","last_undegraded":"2024-03-30T00:39:55.171643+0000","last_fullsized":"2024-03-30T00:39:55.171643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128439+0000","last_change":"2024-03-30T00:39:55.187382+0000","last_active":"2024-03-30T00:39:57.128439+0000","last_peered":"2024-03-30T00:39:57.128439+0000","last_clean":"2024-03-30T00:39:57.128439+0000","last_became_active":"2024-03-30T00:39:55.186804+0000","last_became_peered":"2024-03-30T00:39:55.186804+0000","last_unstale":"2024-03-30T00:39:57.128439+0000","last_undegraded":"2024-03-30T00:39:57.128439+0000","last_fullsized":"2024-03-30T00:39:57.128439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161837+0000","last_change":"2024-03-30T00:39:51.172750+0000","last_active":"2024-03-30T00:39:55.161837+0000","last_peered":"2024-03-30T00:39:55.161837+0000","last_clean":"2024-03-30T00:39:55.161837+0000","last_became_active":"2024-03-30T00:39:51.172456+0000","last_became_peered":"2024-03-30T00:39:51.172456+0000","last_unstale":"2024-03-30T00:39:55.161837+0000","last_undegraded":"2024-03-30T00:39:55.161837+0000","last_fullsized":"2024-03-30T00:39:55.161837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:40.166014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168217+0000","last_change":"2024-03-30T00:39:50.775126+0000","last_active":"2024-03-30T00:39:55.168217+0000","last_peered":"2024-03-30T00:39:55.168217+0000","last_clean":"2024-03-30T00:39:55.168217+0000","last_became_active":"2024-03-30T00:39:50.773078+0000","last_became_peered":"2024-03-30T00:39:50.773078+0000","last_unstale":"2024-03-30T00:39:55.168217+0000","last_undegraded":"2024-03-30T00:39:55.168217+0000","last_fullsized":"2024-03-30T00:39:55.168217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128669+0000","last_change":"2024-03-30T00:39:55.182489+0000","last_active":"2024-03-30T00:39:57.128669+0000","last_peered":"2024-03-30T00:39:57.128669+0000","last_clean":"2024-03-30T00:39:57.128669+0000","last_became_active":"2024-03-30T00:39:55.181876+0000","last_became_peered":"2024-03-30T00:39:55.181876+0000","last_unstale":"2024-03-30T00:39:57.128669+0000","last_undegraded":"2024-03-30T00:39:57.128669+0000","last_fullsized":"2024-03-30T00:39:57.128669+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171723+0000","last_change":"2024-03-30T00:39:51.157937+0000","last_active":"2024-03-30T00:39:55.171723+0000","last_peered":"2024-03-30T00:39:55.171723+0000","last_clean":"2024-03-30T00:39:55.171723+0000","last_became_active":"2024-03-30T00:39:51.155936+0000","last_became_peered":"2024-03-30T00:39:51.155936+0000","last_unstale":"2024-03-30T00:39:55.171723+0000","last_undegraded":"2024-03-30T00:39:55.171723+0000","last_fullsized":"2024-03-30T00:39:55.171723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159139+0000","last_change":"2024-03-30T00:39:50.765709+0000","last_active":"2024-03-30T00:39:55.159139+0000","last_peered":"2024-03-30T00:39:55.159139+0000","last_clean":"2024-03-30T00:39:55.159139+0000","last_became_active":"2024-03-30T00:39:50.765450+0000","last_became_peered":"2024-03-30T00:39:50.765450+0000","last_unstale":"2024-03-30T00:39:55.159139+0000","last_undegraded":"2024-03-30T00:39:55.159139+0000","last_fullsized":"2024-03-30T00:39:55.159139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.129028+0000","last_change":"2024-03-30T00:39:55.193764+0000","last_active":"2024-03-30T00:39:57.129028+0000","last_peered":"2024-03-30T00:39:57.129028+0000","last_clean":"2024-03-30T00:39:57.129028+0000","last_became_active":"2024-03-30T00:39:55.188761+0000","last_became_peered":"2024-03-30T00:39:55.188761+0000","last_unstale":"2024-03-30T00:39:57.129028+0000","last_undegraded":"2024-03-30T00:39:57.129028+0000","last_fullsized":"2024-03-30T00:39:57.129028+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.182494+0000","last_change":"2024-03-30T00:39:51.176755+0000","last_active":"2024-03-30T00:39:55.182494+0000","last_peered":"2024-03-30T00:39:55.182494+0000","last_clean":"2024-03-30T00:39:55.182494+0000","last_became_active":"2024-03-30T00:39:51.176507+0000","last_became_peered":"2024-03-30T00:39:51.176507+0000","last_unstale":"2024-03-30T00:39:55.182494+0000","last_undegraded":"2024-03-30T00:39:55.182494+0000","last_fullsized":"2024-03-30T00:39:55.182494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160037+0000","last_change":"2024-03-30T00:39:50.763226+0000","last_active":"2024-03-30T00:39:55.160037+0000","last_peered":"2024-03-30T00:39:55.160037+0000","last_clean":"2024-03-30T00:39:55.160037+0000","last_became_active":"2024-03-30T00:39:50.762972+0000","last_became_peered":"2024-03-30T00:39:50.762972+0000","last_unstale":"2024-03-30T00:39:55.160037+0000","last_undegraded":"2024-03-30T00:39:55.160037+0000","last_fullsized":"2024-03-30T00:39:55.160037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:27.304749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273996+0000","last_change":"2024-03-30T00:39:55.198804+0000","last_active":"2024-03-30T00:39:57.273996+0000","last_peered":"2024-03-30T00:39:57.273996+0000","last_clean":"2024-03-30T00:39:57.273996+0000","last_became_active":"2024-03-30T00:39:55.198382+0000","last_became_peered":"2024-03-30T00:39:55.198382+0000","last_unstale":"2024-03-30T00:39:57.273996+0000","last_undegraded":"2024-03-30T00:39:57.273996+0000","last_fullsized":"2024-03-30T00:39:57.273996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:35.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165312+0000","last_change":"2024-03-30T00:39:51.176690+0000","last_active":"2024-03-30T00:39:55.165312+0000","last_peered":"2024-03-30T00:39:55.165312+0000","last_clean":"2024-03-30T00:39:55.165312+0000","last_became_active":"2024-03-30T00:39:51.175614+0000","last_became_peered":"2024-03-30T00:39:51.175614+0000","last_unstale":"2024-03-30T00:39:55.165312+0000","last_undegraded":"2024-03-30T00:39:55.165312+0000","last_fullsized":"2024-03-30T00:39:55.165312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:13.959522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.176056+0000","last_change":"2024-03-30T00:39:50.766293+0000","last_active":"2024-03-30T00:39:55.176056+0000","last_peered":"2024-03-30T00:39:55.176056+0000","last_clean":"2024-03-30T00:39:55.176056+0000","last_became_active":"2024-03-30T00:39:50.765977+0000","last_became_peered":"2024-03-30T00:39:50.765977+0000","last_unstale":"2024-03-30T00:39:55.176056+0000","last_undegraded":"2024-03-30T00:39:55.176056+0000","last_fullsized":"2024-03-30T00:39:55.176056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160226+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:39:55.160226+0000","last_peered":"2024-03-30T00:39:55.160226+0000","last_clean":"2024-03-30T00:39:55.160226+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:39:55.160226+0000","last_undegraded":"2024-03-30T00:39:55.160226+0000","last_fullsized":"2024-03-30T00:39:55.160226+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572655+0000","last_change":"2024-03-30T00:39:55.202116+0000","last_active":"2024-03-30T00:39:55.572655+0000","last_peered":"2024-03-30T00:39:55.572655+0000","last_clean":"2024-03-30T00:39:55.572655+0000","last_became_active":"2024-03-30T00:39:55.201638+0000","last_became_peered":"2024-03-30T00:39:55.201638+0000","last_unstale":"2024-03-30T00:39:55.572655+0000","last_undegraded":"2024-03-30T00:39:55.572655+0000","last_fullsized":"2024-03-30T00:39:55.572655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:46:36.593302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162909+0000","last_change":"2024-03-30T00:39:51.176768+0000","last_active":"2024-03-30T00:39:55.162909+0000","last_peered":"2024-03-30T00:39:55.162909+0000","last_clean":"2024-03-30T00:39:55.162909+0000","last_became_active":"2024-03-30T00:39:51.176215+0000","last_became_peered":"2024-03-30T00:39:51.176215+0000","last_unstale":"2024-03-30T00:39:55.162909+0000","last_undegraded":"2024-03-30T00:39:55.162909+0000","last_fullsized":"2024-03-30T00:39:55.162909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:26.397679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159355+0000","last_change":"2024-03-30T00:39:50.763227+0000","last_active":"2024-03-30T00:39:55.159355+0000","last_peered":"2024-03-30T00:39:55.159355+0000","last_clean":"2024-03-30T00:39:55.159355+0000","last_became_active":"2024-03-30T00:39:50.762934+0000","last_became_peered":"2024-03-30T00:39:50.762934+0000","last_unstale":"2024-03-30T00:39:55.159355+0000","last_undegraded":"2024-03-30T00:39:55.159355+0000","last_fullsized":"2024-03-30T00:39:55.159355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572469+0000","last_change":"2024-03-30T00:39:55.195709+0000","last_active":"2024-03-30T00:39:55.572469+0000","last_peered":"2024-03-30T00:39:55.572469+0000","last_clean":"2024-03-30T00:39:55.572469+0000","last_became_active":"2024-03-30T00:39:55.194810+0000","last_became_peered":"2024-03-30T00:39:55.194810+0000","last_unstale":"2024-03-30T00:39:55.572469+0000","last_undegraded":"2024-03-30T00:39:55.572469+0000","last_fullsized":"2024-03-30T00:39:55.572469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:30.942593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170424+0000","last_change":"2024-03-30T00:39:51.182838+0000","last_active":"2024-03-30T00:39:55.170424+0000","last_peered":"2024-03-30T00:39:55.170424+0000","last_clean":"2024-03-30T00:39:55.170424+0000","last_became_active":"2024-03-30T00:39:51.178980+0000","last_became_peered":"2024-03-30T00:39:51.178980+0000","last_unstale":"2024-03-30T00:39:55.170424+0000","last_undegraded":"2024-03-30T00:39:55.170424+0000","last_fullsized":"2024-03-30T00:39:55.170424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:13.959522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172688+0000","last_change":"2024-03-30T00:39:50.755295+0000","last_active":"2024-03-30T00:39:55.172688+0000","last_peered":"2024-03-30T00:39:55.172688+0000","last_clean":"2024-03-30T00:39:55.172688+0000","last_became_active":"2024-03-30T00:39:50.754957+0000","last_became_peered":"2024-03-30T00:39:50.754957+0000","last_unstale":"2024-03-30T00:39:55.172688+0000","last_undegraded":"2024-03-30T00:39:55.172688+0000","last_fullsized":"2024-03-30T00:39:55.172688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:16.100201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273833+0000","last_change":"2024-03-30T00:39:55.199975+0000","last_active":"2024-03-30T00:39:57.273833+0000","last_peered":"2024-03-30T00:39:57.273833+0000","last_clean":"2024-03-30T00:39:57.273833+0000","last_became_active":"2024-03-30T00:39:55.199191+0000","last_became_peered":"2024-03-30T00:39:55.199191+0000","last_unstale":"2024-03-30T00:39:57.273833+0000","last_undegraded":"2024-03-30T00:39:57.273833+0000","last_fullsized":"2024-03-30T00:39:57.273833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163333+0000","last_change":"2024-03-30T00:39:51.163933+0000","last_active":"2024-03-30T00:39:55.163333+0000","last_peered":"2024-03-30T00:39:55.163333+0000","last_clean":"2024-03-30T00:39:55.163333+0000","last_became_active":"2024-03-30T00:39:51.163663+0000","last_became_peered":"2024-03-30T00:39:51.163663+0000","last_unstale":"2024-03-30T00:39:55.163333+0000","last_undegraded":"2024-03-30T00:39:55.163333+0000","last_fullsized":"2024-03-30T00:39:55.163333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:41.330935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159267+0000","last_change":"2024-03-30T00:39:50.766833+0000","last_active":"2024-03-30T00:39:55.159267+0000","last_peered":"2024-03-30T00:39:55.159267+0000","last_clean":"2024-03-30T00:39:55.159267+0000","last_became_active":"2024-03-30T00:39:50.766534+0000","last_became_peered":"2024-03-30T00:39:50.766534+0000","last_unstale":"2024-03-30T00:39:55.159267+0000","last_undegraded":"2024-03-30T00:39:55.159267+0000","last_fullsized":"2024-03-30T00:39:55.159267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:56.358073+0000","last_change":"2024-03-30T00:39:55.184862+0000","last_active":"2024-03-30T00:39:56.358073+0000","last_peered":"2024-03-30T00:39:56.358073+0000","last_clean":"2024-03-30T00:39:56.358073+0000","last_became_active":"2024-03-30T00:39:55.184374+0000","last_became_peered":"2024-03-30T00:39:55.184374+0000","last_unstale":"2024-03-30T00:39:56.358073+0000","last_undegraded":"2024-03-30T00:39:56.358073+0000","last_fullsized":"2024-03-30T00:39:56.358073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163385+0000","last_change":"2024-03-30T00:39:51.162489+0000","last_active":"2024-03-30T00:39:55.163385+0000","last_peered":"2024-03-30T00:39:55.163385+0000","last_clean":"2024-03-30T00:39:55.163385+0000","last_became_active":"2024-03-30T00:39:51.162205+0000","last_became_peered":"2024-03-30T00:39:51.162205+0000","last_unstale":"2024-03-30T00:39:55.163385+0000","last_undegraded":"2024-03-30T00:39:55.163385+0000","last_fullsized":"2024-03-30T00:39:55.163385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:36.973826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162109+0000","last_change":"2024-03-30T00:39:50.768761+0000","last_active":"2024-03-30T00:39:55.162109+0000","last_peered":"2024-03-30T00:39:55.162109+0000","last_clean":"2024-03-30T00:39:55.162109+0000","last_became_active":"2024-03-30T00:39:50.768372+0000","last_became_peered":"2024-03-30T00:39:50.768372+0000","last_unstale":"2024-03-30T00:39:55.162109+0000","last_undegraded":"2024-03-30T00:39:55.162109+0000","last_fullsized":"2024-03-30T00:39:55.162109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:35:25.209340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272560+0000","last_change":"2024-03-30T00:39:55.177966+0000","last_active":"2024-03-30T00:39:57.272560+0000","last_peered":"2024-03-30T00:39:57.272560+0000","last_clean":"2024-03-30T00:39:57.272560+0000","last_became_active":"2024-03-30T00:39:55.177485+0000","last_became_peered":"2024-03-30T00:39:55.177485+0000","last_unstale":"2024-03-30T00:39:57.272560+0000","last_undegraded":"2024-03-30T00:39:57.272560+0000","last_fullsized":"2024-03-30T00:39:57.272560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:13:47.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161004+0000","last_change":"2024-03-30T00:39:51.157566+0000","last_active":"2024-03-30T00:39:55.161004+0000","last_peered":"2024-03-30T00:39:55.161004+0000","last_clean":"2024-03-30T00:39:55.161004+0000","last_became_active":"2024-03-30T00:39:51.157243+0000","last_became_peered":"2024-03-30T00:39:51.157243+0000","last_unstale":"2024-03-30T00:39:55.161004+0000","last_undegraded":"2024-03-30T00:39:55.161004+0000","last_fullsized":"2024-03-30T00:39:55.161004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:35.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159067+0000","last_change":"2024-03-30T00:39:50.760164+0000","last_active":"2024-03-30T00:39:55.159067+0000","last_peered":"2024-03-30T00:39:55.159067+0000","last_clean":"2024-03-30T00:39:55.159067+0000","last_became_active":"2024-03-30T00:39:50.759887+0000","last_became_peered":"2024-03-30T00:39:50.759887+0000","last_unstale":"2024-03-30T00:39:55.159067+0000","last_undegraded":"2024-03-30T00:39:55.159067+0000","last_fullsized":"2024-03-30T00:39:55.159067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273934+0000","last_change":"2024-03-30T00:39:55.196574+0000","last_active":"2024-03-30T00:39:57.273934+0000","last_peered":"2024-03-30T00:39:57.273934+0000","last_clean":"2024-03-30T00:39:57.273934+0000","last_became_active":"2024-03-30T00:39:55.196182+0000","last_became_peered":"2024-03-30T00:39:55.196182+0000","last_unstale":"2024-03-30T00:39:57.273934+0000","last_undegraded":"2024-03-30T00:39:57.273934+0000","last_fullsized":"2024-03-30T00:39:57.273934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170724+0000","last_change":"2024-03-30T00:39:51.167097+0000","last_active":"2024-03-30T00:39:55.170724+0000","last_peered":"2024-03-30T00:39:55.170724+0000","last_clean":"2024-03-30T00:39:55.170724+0000","last_became_active":"2024-03-30T00:39:51.166629+0000","last_became_peered":"2024-03-30T00:39:51.166629+0000","last_unstale":"2024-03-30T00:39:55.170724+0000","last_undegraded":"2024-03-30T00:39:55.170724+0000","last_fullsized":"2024-03-30T00:39:55.170724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170864+0000","last_change":"2024-03-30T00:39:50.767642+0000","last_active":"2024-03-30T00:39:55.170864+0000","last_peered":"2024-03-30T00:39:55.170864+0000","last_clean":"2024-03-30T00:39:55.170864+0000","last_became_active":"2024-03-30T00:39:50.767166+0000","last_became_peered":"2024-03-30T00:39:50.767166+0000","last_unstale":"2024-03-30T00:39:55.170864+0000","last_undegraded":"2024-03-30T00:39:55.170864+0000","last_fullsized":"2024-03-30T00:39:55.170864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:03.620861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572362+0000","last_change":"2024-03-30T00:39:55.192042+0000","last_active":"2024-03-30T00:39:55.572362+0000","last_peered":"2024-03-30T00:39:55.572362+0000","last_clean":"2024-03-30T00:39:55.572362+0000","last_became_active":"2024-03-30T00:39:55.190080+0000","last_became_peered":"2024-03-30T00:39:55.190080+0000","last_unstale":"2024-03-30T00:39:55.572362+0000","last_undegraded":"2024-03-30T00:39:55.572362+0000","last_fullsized":"2024-03-30T00:39:55.572362+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:25.792128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161353+0000","last_change":"2024-03-30T00:39:51.161405+0000","last_active":"2024-03-30T00:39:55.161353+0000","last_peered":"2024-03-30T00:39:55.161353+0000","last_clean":"2024-03-30T00:39:55.161353+0000","last_became_active":"2024-03-30T00:39:51.161053+0000","last_became_peered":"2024-03-30T00:39:51.161053+0000","last_unstale":"2024-03-30T00:39:55.161353+0000","last_undegraded":"2024-03-30T00:39:55.161353+0000","last_fullsized":"2024-03-30T00:39:55.161353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:49.107527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168130+0000","last_change":"2024-03-30T00:39:50.774925+0000","last_active":"2024-03-30T00:39:55.168130+0000","last_peered":"2024-03-30T00:39:55.168130+0000","last_clean":"2024-03-30T00:39:55.168130+0000","last_became_active":"2024-03-30T00:39:50.773221+0000","last_became_peered":"2024-03-30T00:39:50.773221+0000","last_unstale":"2024-03-30T00:39:55.168130+0000","last_undegraded":"2024-03-30T00:39:55.168130+0000","last_fullsized":"2024-03-30T00:39:55.168130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572142+0000","last_change":"2024-03-30T00:39:55.191576+0000","last_active":"2024-03-30T00:39:55.572142+0000","last_peered":"2024-03-30T00:39:55.572142+0000","last_clean":"2024-03-30T00:39:55.572142+0000","last_became_active":"2024-03-30T00:39:55.191188+0000","last_became_peered":"2024-03-30T00:39:55.191188+0000","last_unstale":"2024-03-30T00:39:55.572142+0000","last_undegraded":"2024-03-30T00:39:55.572142+0000","last_fullsized":"2024-03-30T00:39:55.572142+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:58.020154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184454+0000","last_change":"2024-03-30T00:39:51.169704+0000","last_active":"2024-03-30T00:39:55.184454+0000","last_peered":"2024-03-30T00:39:55.184454+0000","last_clean":"2024-03-30T00:39:55.184454+0000","last_became_active":"2024-03-30T00:39:51.168350+0000","last_became_peered":"2024-03-30T00:39:51.168350+0000","last_unstale":"2024-03-30T00:39:55.184454+0000","last_undegraded":"2024-03-30T00:39:55.184454+0000","last_fullsized":"2024-03-30T00:39:55.184454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183870+0000","last_change":"2024-03-30T00:39:50.760857+0000","last_active":"2024-03-30T00:39:55.183870+0000","last_peered":"2024-03-30T00:39:55.183870+0000","last_clean":"2024-03-30T00:39:55.183870+0000","last_became_active":"2024-03-30T00:39:50.760607+0000","last_became_peered":"2024-03-30T00:39:50.760607+0000","last_unstale":"2024-03-30T00:39:55.183870+0000","last_undegraded":"2024-03-30T00:39:55.183870+0000","last_fullsized":"2024-03-30T00:39:55.183870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:56.358741+0000","last_change":"2024-03-30T00:39:55.188289+0000","last_active":"2024-03-30T00:39:56.358741+0000","last_peered":"2024-03-30T00:39:56.358741+0000","last_clean":"2024-03-30T00:39:56.358741+0000","last_became_active":"2024-03-30T00:39:55.187869+0000","last_became_peered":"2024-03-30T00:39:55.187869+0000","last_unstale":"2024-03-30T00:39:56.358741+0000","last_undegraded":"2024-03-30T00:39:56.358741+0000","last_fullsized":"2024-03-30T00:39:56.358741+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162815+0000","last_change":"2024-03-30T00:39:51.177240+0000","last_active":"2024-03-30T00:39:55.162815+0000","last_peered":"2024-03-30T00:39:55.162815+0000","last_clean":"2024-03-30T00:39:55.162815+0000","last_became_active":"2024-03-30T00:39:51.176388+0000","last_became_peered":"2024-03-30T00:39:51.176388+0000","last_unstale":"2024-03-30T00:39:55.162815+0000","last_undegraded":"2024-03-30T00:39:55.162815+0000","last_fullsized":"2024-03-30T00:39:55.162815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:37:05.851113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:40:01.400156+0000","last_change":"2024-03-30T00:39:50.766631+0000","last_active":"2024-03-30T00:40:01.400156+0000","last_peered":"2024-03-30T00:40:01.400156+0000","last_clean":"2024-03-30T00:40:01.400156+0000","last_became_active":"2024-03-30T00:39:50.766362+0000","last_became_peered":"2024-03-30T00:39:50.766362+0000","last_unstale":"2024-03-30T00:40:01.400156+0000","last_undegraded":"2024-03-30T00:40:01.400156+0000","last_fullsized":"2024-03-30T00:40:01.400156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.128582+0000","last_change":"2024-03-30T00:39:55.187191+0000","last_active":"2024-03-30T00:39:57.128582+0000","last_peered":"2024-03-30T00:39:57.128582+0000","last_clean":"2024-03-30T00:39:57.128582+0000","last_became_active":"2024-03-30T00:39:55.186712+0000","last_became_peered":"2024-03-30T00:39:55.186712+0000","last_unstale":"2024-03-30T00:39:57.128582+0000","last_undegraded":"2024-03-30T00:39:57.128582+0000","last_fullsized":"2024-03-30T00:39:57.128582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162239+0000","last_change":"2024-03-30T00:39:51.154183+0000","last_active":"2024-03-30T00:39:55.162239+0000","last_peered":"2024-03-30T00:39:55.162239+0000","last_clean":"2024-03-30T00:39:55.162239+0000","last_became_active":"2024-03-30T00:39:51.153909+0000","last_became_peered":"2024-03-30T00:39:51.153909+0000","last_unstale":"2024-03-30T00:39:55.162239+0000","last_undegraded":"2024-03-30T00:39:55.162239+0000","last_fullsized":"2024-03-30T00:39:55.162239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:51.868242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172499+0000","last_change":"2024-03-30T00:39:50.763665+0000","last_active":"2024-03-30T00:39:55.172499+0000","last_peered":"2024-03-30T00:39:55.172499+0000","last_clean":"2024-03-30T00:39:55.172499+0000","last_became_active":"2024-03-30T00:39:50.763390+0000","last_became_peered":"2024-03-30T00:39:50.763390+0000","last_unstale":"2024-03-30T00:39:55.172499+0000","last_undegraded":"2024-03-30T00:39:55.172499+0000","last_fullsized":"2024-03-30T00:39:55.172499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.272944+0000","last_change":"2024-03-30T00:39:55.185767+0000","last_active":"2024-03-30T00:39:57.272944+0000","last_peered":"2024-03-30T00:39:57.272944+0000","last_clean":"2024-03-30T00:39:57.272944+0000","last_became_active":"2024-03-30T00:39:55.185430+0000","last_became_peered":"2024-03-30T00:39:55.185430+0000","last_unstale":"2024-03-30T00:39:57.272944+0000","last_undegraded":"2024-03-30T00:39:57.272944+0000","last_fullsized":"2024-03-30T00:39:57.272944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:59:28.443255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184382+0000","last_change":"2024-03-30T00:39:51.177311+0000","last_active":"2024-03-30T00:39:55.184382+0000","last_peered":"2024-03-30T00:39:55.184382+0000","last_clean":"2024-03-30T00:39:55.184382+0000","last_became_active":"2024-03-30T00:39:51.175904+0000","last_became_peered":"2024-03-30T00:39:51.175904+0000","last_unstale":"2024-03-30T00:39:55.184382+0000","last_undegraded":"2024-03-30T00:39:55.184382+0000","last_fullsized":"2024-03-30T00:39:55.184382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:45.366551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167202+0000","last_change":"2024-03-30T00:39:50.769375+0000","last_active":"2024-03-30T00:39:55.167202+0000","last_peered":"2024-03-30T00:39:55.167202+0000","last_clean":"2024-03-30T00:39:55.167202+0000","last_became_active":"2024-03-30T00:39:50.768671+0000","last_became_peered":"2024-03-30T00:39:50.768671+0000","last_unstale":"2024-03-30T00:39:55.167202+0000","last_undegraded":"2024-03-30T00:39:55.167202+0000","last_fullsized":"2024-03-30T00:39:55.167202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:45.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571487+0000","last_change":"2024-03-30T00:39:55.192307+0000","last_active":"2024-03-30T00:39:55.571487+0000","last_peered":"2024-03-30T00:39:55.571487+0000","last_clean":"2024-03-30T00:39:55.571487+0000","last_became_active":"2024-03-30T00:39:55.191803+0000","last_became_peered":"2024-03-30T00:39:55.191803+0000","last_unstale":"2024-03-30T00:39:55.571487+0000","last_undegraded":"2024-03-30T00:39:55.571487+0000","last_fullsized":"2024-03-30T00:39:55.571487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:33:50.238803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167108+0000","last_change":"2024-03-30T00:39:51.168083+0000","last_active":"2024-03-30T00:39:55.167108+0000","last_peered":"2024-03-30T00:39:55.167108+0000","last_clean":"2024-03-30T00:39:55.167108+0000","last_became_active":"2024-03-30T00:39:51.167756+0000","last_became_peered":"2024-03-30T00:39:51.167756+0000","last_unstale":"2024-03-30T00:39:55.167108+0000","last_undegraded":"2024-03-30T00:39:55.167108+0000","last_fullsized":"2024-03-30T00:39:55.167108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172080+0000","last_change":"2024-03-30T00:39:50.775634+0000","last_active":"2024-03-30T00:39:55.172080+0000","last_peered":"2024-03-30T00:39:55.172080+0000","last_clean":"2024-03-30T00:39:55.172080+0000","last_became_active":"2024-03-30T00:39:50.775337+0000","last_became_peered":"2024-03-30T00:39:50.775337+0000","last_unstale":"2024-03-30T00:39:55.172080+0000","last_undegraded":"2024-03-30T00:39:55.172080+0000","last_fullsized":"2024-03-30T00:39:55.172080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:50.865522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.571487+0000","last_change":"2024-03-30T00:39:55.175243+0000","last_active":"2024-03-30T00:39:55.571487+0000","last_peered":"2024-03-30T00:39:55.571487+0000","last_clean":"2024-03-30T00:39:55.571487+0000","last_became_active":"2024-03-30T00:39:55.174556+0000","last_became_peered":"2024-03-30T00:39:55.174556+0000","last_unstale":"2024-03-30T00:39:55.571487+0000","last_undegraded":"2024-03-30T00:39:55.571487+0000","last_fullsized":"2024-03-30T00:39:55.571487+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:32:58.696045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.274064+0000","last_change":"2024-03-30T00:39:55.203406+0000","last_active":"2024-03-30T00:39:57.274064+0000","last_peered":"2024-03-30T00:39:57.274064+0000","last_clean":"2024-03-30T00:39:57.274064+0000","last_became_active":"2024-03-30T00:39:55.203027+0000","last_became_peered":"2024-03-30T00:39:55.203027+0000","last_unstale":"2024-03-30T00:39:57.274064+0000","last_undegraded":"2024-03-30T00:39:57.274064+0000","last_fullsized":"2024-03-30T00:39:57.274064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:51:09.198004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171850+0000","last_change":"2024-03-30T00:39:51.175309+0000","last_active":"2024-03-30T00:39:55.171850+0000","last_peered":"2024-03-30T00:39:55.171850+0000","last_clean":"2024-03-30T00:39:55.171850+0000","last_became_active":"2024-03-30T00:39:51.174946+0000","last_became_peered":"2024-03-30T00:39:51.174946+0000","last_unstale":"2024-03-30T00:39:55.171850+0000","last_undegraded":"2024-03-30T00:39:55.171850+0000","last_fullsized":"2024-03-30T00:39:55.171850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167862+0000","last_change":"2024-03-30T00:39:50.762581+0000","last_active":"2024-03-30T00:39:55.167862+0000","last_peered":"2024-03-30T00:39:55.167862+0000","last_clean":"2024-03-30T00:39:55.167862+0000","last_became_active":"2024-03-30T00:39:50.762206+0000","last_became_peered":"2024-03-30T00:39:50.762206+0000","last_unstale":"2024-03-30T00:39:55.167862+0000","last_undegraded":"2024-03-30T00:39:55.167862+0000","last_fullsized":"2024-03-30T00:39:55.167862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.573135+0000","last_change":"2024-03-30T00:39:55.188289+0000","last_active":"2024-03-30T00:39:55.573135+0000","last_peered":"2024-03-30T00:39:55.573135+0000","last_clean":"2024-03-30T00:39:55.573135+0000","last_became_active":"2024-03-30T00:39:55.187825+0000","last_became_peered":"2024-03-30T00:39:55.187825+0000","last_unstale":"2024-03-30T00:39:55.573135+0000","last_undegraded":"2024-03-30T00:39:55.573135+0000","last_fullsized":"2024-03-30T00:39:55.573135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:01.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.166225+0000","last_change":"2024-03-30T00:39:51.172769+0000","last_active":"2024-03-30T00:39:55.166225+0000","last_peered":"2024-03-30T00:39:55.166225+0000","last_clean":"2024-03-30T00:39:55.166225+0000","last_became_active":"2024-03-30T00:39:51.170913+0000","last_became_peered":"2024-03-30T00:39:51.170913+0000","last_unstale":"2024-03-30T00:39:55.166225+0000","last_undegraded":"2024-03-30T00:39:55.166225+0000","last_fullsized":"2024-03-30T00:39:55.166225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165283+0000","last_change":"2024-03-30T00:39:50.771732+0000","last_active":"2024-03-30T00:39:55.165283+0000","last_peered":"2024-03-30T00:39:55.165283+0000","last_clean":"2024-03-30T00:39:55.165283+0000","last_became_active":"2024-03-30T00:39:50.771443+0000","last_became_peered":"2024-03-30T00:39:50.771443+0000","last_unstale":"2024-03-30T00:39:55.165283+0000","last_undegraded":"2024-03-30T00:39:55.165283+0000","last_fullsized":"2024-03-30T00:39:55.165283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:51.594108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171239+0000","last_change":"2024-03-30T00:39:50.768147+0000","last_active":"2024-03-30T00:39:55.171239+0000","last_peered":"2024-03-30T00:39:55.171239+0000","last_clean":"2024-03-30T00:39:55.171239+0000","last_became_active":"2024-03-30T00:39:50.767258+0000","last_became_peered":"2024-03-30T00:39:50.767258+0000","last_unstale":"2024-03-30T00:39:55.171239+0000","last_undegraded":"2024-03-30T00:39:55.171239+0000","last_fullsized":"2024-03-30T00:39:55.171239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:27.487056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183786+0000","last_change":"2024-03-30T00:39:51.173636+0000","last_active":"2024-03-30T00:39:55.183786+0000","last_peered":"2024-03-30T00:39:55.183786+0000","last_clean":"2024-03-30T00:39:55.183786+0000","last_became_active":"2024-03-30T00:39:51.173356+0000","last_became_peered":"2024-03-30T00:39:51.173356+0000","last_unstale":"2024-03-30T00:39:55.183786+0000","last_undegraded":"2024-03-30T00:39:55.183786+0000","last_fullsized":"2024-03-30T00:39:55.183786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:10:20.544219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:57.273686+0000","last_change":"2024-03-30T00:39:55.187062+0000","last_active":"2024-03-30T00:39:57.273686+0000","last_peered":"2024-03-30T00:39:57.273686+0000","last_clean":"2024-03-30T00:39:57.273686+0000","last_became_active":"2024-03-30T00:39:55.184977+0000","last_became_peered":"2024-03-30T00:39:55.184977+0000","last_unstale":"2024-03-30T00:39:57.273686+0000","last_undegraded":"2024-03-30T00:39:57.273686+0000","last_fullsized":"2024-03-30T00:39:57.273686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:23.899767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169318+0000","last_change":"2024-03-30T00:39:50.767360+0000","last_active":"2024-03-30T00:39:55.169318+0000","last_peered":"2024-03-30T00:39:55.169318+0000","last_clean":"2024-03-30T00:39:55.169318+0000","last_became_active":"2024-03-30T00:39:50.764301+0000","last_became_peered":"2024-03-30T00:39:50.764301+0000","last_unstale":"2024-03-30T00:39:55.169318+0000","last_undegraded":"2024-03-30T00:39:55.169318+0000","last_fullsized":"2024-03-30T00:39:55.169318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:49.735223+0000","last_clean_scrub_stamp":"2024-03-30T00:39:49.735223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:26.877416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160985+0000","last_change":"2024-03-30T00:39:51.163323+0000","last_active":"2024-03-30T00:39:55.160985+0000","last_peered":"2024-03-30T00:39:55.160985+0000","last_clean":"2024-03-30T00:39:55.160985+0000","last_became_active":"2024-03-30T00:39:51.163080+0000","last_became_peered":"2024-03-30T00:39:51.163080+0000","last_unstale":"2024-03-30T00:39:55.160985+0000","last_undegraded":"2024-03-30T00:39:55.160985+0000","last_fullsized":"2024-03-30T00:39:55.160985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:28.307469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.572341+0000","last_change":"2024-03-30T00:39:55.194121+0000","last_active":"2024-03-30T00:39:55.572341+0000","last_peered":"2024-03-30T00:39:55.572341+0000","last_clean":"2024-03-30T00:39:55.572341+0000","last_became_active":"2024-03-30T00:39:55.190121+0000","last_became_peered":"2024-03-30T00:39:55.190121+0000","last_unstale":"2024-03-30T00:39:55.572341+0000","last_undegraded":"2024-03-30T00:39:55.572341+0000","last_fullsized":"2024-03-30T00:39:55.572341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:54.152557+0000","last_clean_scrub_stamp":"2024-03-30T00:39:54.152557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:10:34.870687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169878+0000","last_change":"2024-03-30T00:39:51.168112+0000","last_active":"2024-03-30T00:39:55.169878+0000","last_peered":"2024-03-30T00:39:55.169878+0000","last_clean":"2024-03-30T00:39:55.169878+0000","last_became_active":"2024-03-30T00:39:51.167770+0000","last_became_peered":"2024-03-30T00:39:51.167770+0000","last_unstale":"2024-03-30T00:39:55.169878+0000","last_undegraded":"2024-03-30T00:39:55.169878+0000","last_fullsized":"2024-03-30T00:39:55.169878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159083+0000","last_change":"2024-03-30T00:39:51.169867+0000","last_active":"2024-03-30T00:39:55.159083+0000","last_peered":"2024-03-30T00:39:55.159083+0000","last_clean":"2024-03-30T00:39:55.159083+0000","last_became_active":"2024-03-30T00:39:51.169515+0000","last_became_peered":"2024-03-30T00:39:51.169515+0000","last_unstale":"2024-03-30T00:39:55.159083+0000","last_undegraded":"2024-03-30T00:39:55.159083+0000","last_fullsized":"2024-03-30T00:39:55.159083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:44.962672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168036+0000","last_change":"2024-03-30T00:39:51.175788+0000","last_active":"2024-03-30T00:39:55.168036+0000","last_peered":"2024-03-30T00:39:55.168036+0000","last_clean":"2024-03-30T00:39:55.168036+0000","last_became_active":"2024-03-30T00:39:51.175288+0000","last_became_peered":"2024-03-30T00:39:51.175288+0000","last_unstale":"2024-03-30T00:39:55.168036+0000","last_undegraded":"2024-03-30T00:39:55.168036+0000","last_fullsized":"2024-03-30T00:39:55.168036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:16:35.369846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183710+0000","last_change":"2024-03-30T00:39:51.173618+0000","last_active":"2024-03-30T00:39:55.183710+0000","last_peered":"2024-03-30T00:39:55.183710+0000","last_clean":"2024-03-30T00:39:55.183710+0000","last_became_active":"2024-03-30T00:39:51.173324+0000","last_became_peered":"2024-03-30T00:39:51.173324+0000","last_unstale":"2024-03-30T00:39:55.183710+0000","last_undegraded":"2024-03-30T00:39:55.183710+0000","last_fullsized":"2024-03-30T00:39:55.183710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.168094+0000","last_change":"2024-03-30T00:39:51.184101+0000","last_active":"2024-03-30T00:39:55.168094+0000","last_peered":"2024-03-30T00:39:55.168094+0000","last_clean":"2024-03-30T00:39:55.168094+0000","last_became_active":"2024-03-30T00:39:51.183843+0000","last_became_peered":"2024-03-30T00:39:51.183843+0000","last_unstale":"2024-03-30T00:39:55.168094+0000","last_undegraded":"2024-03-30T00:39:55.168094+0000","last_fullsized":"2024-03-30T00:39:55.168094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:16:51.868242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.172390+0000","last_change":"2024-03-30T00:39:51.174001+0000","last_active":"2024-03-30T00:39:55.172390+0000","last_peered":"2024-03-30T00:39:55.172390+0000","last_clean":"2024-03-30T00:39:55.172390+0000","last_became_active":"2024-03-30T00:39:51.173753+0000","last_became_peered":"2024-03-30T00:39:51.173753+0000","last_unstale":"2024-03-30T00:39:55.172390+0000","last_undegraded":"2024-03-30T00:39:55.172390+0000","last_fullsized":"2024-03-30T00:39:55.172390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171728+0000","last_change":"2024-03-30T00:39:51.165845+0000","last_active":"2024-03-30T00:39:55.171728+0000","last_peered":"2024-03-30T00:39:55.171728+0000","last_clean":"2024-03-30T00:39:55.171728+0000","last_became_active":"2024-03-30T00:39:51.164657+0000","last_became_peered":"2024-03-30T00:39:51.164657+0000","last_unstale":"2024-03-30T00:39:55.171728+0000","last_undegraded":"2024-03-30T00:39:55.171728+0000","last_fullsized":"2024-03-30T00:39:55.171728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.164918+0000","last_change":"2024-03-30T00:39:51.185222+0000","last_active":"2024-03-30T00:39:55.164918+0000","last_peered":"2024-03-30T00:39:55.164918+0000","last_clean":"2024-03-30T00:39:55.164918+0000","last_became_active":"2024-03-30T00:39:51.184954+0000","last_became_peered":"2024-03-30T00:39:51.184954+0000","last_unstale":"2024-03-30T00:39:55.164918+0000","last_undegraded":"2024-03-30T00:39:55.164918+0000","last_fullsized":"2024-03-30T00:39:55.164918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167766+0000","last_change":"2024-03-30T00:39:51.174598+0000","last_active":"2024-03-30T00:39:55.167766+0000","last_peered":"2024-03-30T00:39:55.167766+0000","last_clean":"2024-03-30T00:39:55.167766+0000","last_became_active":"2024-03-30T00:39:51.174250+0000","last_became_peered":"2024-03-30T00:39:51.174250+0000","last_unstale":"2024-03-30T00:39:55.167766+0000","last_undegraded":"2024-03-30T00:39:55.167766+0000","last_fullsized":"2024-03-30T00:39:55.167766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.165202+0000","last_change":"2024-03-30T00:39:51.184460+0000","last_active":"2024-03-30T00:39:55.165202+0000","last_peered":"2024-03-30T00:39:55.165202+0000","last_clean":"2024-03-30T00:39:55.165202+0000","last_became_active":"2024-03-30T00:39:51.184185+0000","last_became_peered":"2024-03-30T00:39:51.184185+0000","last_unstale":"2024-03-30T00:39:55.165202+0000","last_undegraded":"2024-03-30T00:39:55.165202+0000","last_fullsized":"2024-03-30T00:39:55.165202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:03:20.829166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167944+0000","last_change":"2024-03-30T00:39:51.168064+0000","last_active":"2024-03-30T00:39:55.167944+0000","last_peered":"2024-03-30T00:39:55.167944+0000","last_clean":"2024-03-30T00:39:55.167944+0000","last_became_active":"2024-03-30T00:39:51.167772+0000","last_became_peered":"2024-03-30T00:39:51.167772+0000","last_unstale":"2024-03-30T00:39:55.167944+0000","last_undegraded":"2024-03-30T00:39:55.167944+0000","last_fullsized":"2024-03-30T00:39:55.167944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:12.512040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183617+0000","last_change":"2024-03-30T00:39:51.183269+0000","last_active":"2024-03-30T00:39:55.183617+0000","last_peered":"2024-03-30T00:39:55.183617+0000","last_clean":"2024-03-30T00:39:55.183617+0000","last_became_active":"2024-03-30T00:39:51.182687+0000","last_became_peered":"2024-03-30T00:39:51.182687+0000","last_unstale":"2024-03-30T00:39:55.183617+0000","last_undegraded":"2024-03-30T00:39:55.183617+0000","last_fullsized":"2024-03-30T00:39:55.183617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171193+0000","last_change":"2024-03-30T00:39:51.171426+0000","last_active":"2024-03-30T00:39:55.171193+0000","last_peered":"2024-03-30T00:39:55.171193+0000","last_clean":"2024-03-30T00:39:55.171193+0000","last_became_active":"2024-03-30T00:39:51.171182+0000","last_became_peered":"2024-03-30T00:39:51.171182+0000","last_unstale":"2024-03-30T00:39:55.171193+0000","last_undegraded":"2024-03-30T00:39:55.171193+0000","last_fullsized":"2024-03-30T00:39:55.171193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:51:41.648914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163087+0000","last_change":"2024-03-30T00:39:51.159899+0000","last_active":"2024-03-30T00:39:55.163087+0000","last_peered":"2024-03-30T00:39:55.163087+0000","last_clean":"2024-03-30T00:39:55.163087+0000","last_became_active":"2024-03-30T00:39:51.159626+0000","last_became_peered":"2024-03-30T00:39:51.159626+0000","last_unstale":"2024-03-30T00:39:55.163087+0000","last_undegraded":"2024-03-30T00:39:55.163087+0000","last_fullsized":"2024-03-30T00:39:55.163087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.159252+0000","last_change":"2024-03-30T00:39:51.171375+0000","last_active":"2024-03-30T00:39:55.159252+0000","last_peered":"2024-03-30T00:39:55.159252+0000","last_clean":"2024-03-30T00:39:55.159252+0000","last_became_active":"2024-03-30T00:39:51.170689+0000","last_became_peered":"2024-03-30T00:39:51.170689+0000","last_unstale":"2024-03-30T00:39:55.159252+0000","last_undegraded":"2024-03-30T00:39:55.159252+0000","last_fullsized":"2024-03-30T00:39:55.159252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:27.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167585+0000","last_change":"2024-03-30T00:39:51.164848+0000","last_active":"2024-03-30T00:39:55.167585+0000","last_peered":"2024-03-30T00:39:55.167585+0000","last_clean":"2024-03-30T00:39:55.167585+0000","last_became_active":"2024-03-30T00:39:51.164575+0000","last_became_peered":"2024-03-30T00:39:51.164575+0000","last_unstale":"2024-03-30T00:39:55.167585+0000","last_undegraded":"2024-03-30T00:39:55.167585+0000","last_fullsized":"2024-03-30T00:39:55.167585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167681+0000","last_change":"2024-03-30T00:39:51.173535+0000","last_active":"2024-03-30T00:39:55.167681+0000","last_peered":"2024-03-30T00:39:55.167681+0000","last_clean":"2024-03-30T00:39:55.167681+0000","last_became_active":"2024-03-30T00:39:51.173313+0000","last_became_peered":"2024-03-30T00:39:51.173313+0000","last_unstale":"2024-03-30T00:39:55.167681+0000","last_undegraded":"2024-03-30T00:39:55.167681+0000","last_fullsized":"2024-03-30T00:39:55.167681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:16.568454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171944+0000","last_change":"2024-03-30T00:39:51.171986+0000","last_active":"2024-03-30T00:39:55.171944+0000","last_peered":"2024-03-30T00:39:55.171944+0000","last_clean":"2024-03-30T00:39:55.171944+0000","last_became_active":"2024-03-30T00:39:51.164756+0000","last_became_peered":"2024-03-30T00:39:51.164756+0000","last_unstale":"2024-03-30T00:39:55.171944+0000","last_undegraded":"2024-03-30T00:39:55.171944+0000","last_fullsized":"2024-03-30T00:39:55.171944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.184365+0000","last_change":"2024-03-30T00:39:51.181455+0000","last_active":"2024-03-30T00:39:55.184365+0000","last_peered":"2024-03-30T00:39:55.184365+0000","last_clean":"2024-03-30T00:39:55.184365+0000","last_became_active":"2024-03-30T00:39:51.181207+0000","last_became_peered":"2024-03-30T00:39:51.181207+0000","last_unstale":"2024-03-30T00:39:55.184365+0000","last_undegraded":"2024-03-30T00:39:55.184365+0000","last_fullsized":"2024-03-30T00:39:55.184365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:01:48.059238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.169138+0000","last_change":"2024-03-30T00:39:51.175074+0000","last_active":"2024-03-30T00:39:55.169138+0000","last_peered":"2024-03-30T00:39:55.169138+0000","last_clean":"2024-03-30T00:39:55.169138+0000","last_became_active":"2024-03-30T00:39:51.174836+0000","last_became_peered":"2024-03-30T00:39:51.174836+0000","last_unstale":"2024-03-30T00:39:55.169138+0000","last_undegraded":"2024-03-30T00:39:55.169138+0000","last_fullsized":"2024-03-30T00:39:55.169138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.162196+0000","last_change":"2024-03-30T00:39:51.169964+0000","last_active":"2024-03-30T00:39:55.162196+0000","last_peered":"2024-03-30T00:39:55.162196+0000","last_clean":"2024-03-30T00:39:55.162196+0000","last_became_active":"2024-03-30T00:39:51.169647+0000","last_became_peered":"2024-03-30T00:39:51.169647+0000","last_unstale":"2024-03-30T00:39:55.162196+0000","last_undegraded":"2024-03-30T00:39:55.162196+0000","last_fullsized":"2024-03-30T00:39:55.162196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:44.962672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171503+0000","last_change":"2024-03-30T00:39:51.178908+0000","last_active":"2024-03-30T00:39:55.171503+0000","last_peered":"2024-03-30T00:39:55.171503+0000","last_clean":"2024-03-30T00:39:55.171503+0000","last_became_active":"2024-03-30T00:39:51.178663+0000","last_became_peered":"2024-03-30T00:39:51.178663+0000","last_unstale":"2024-03-30T00:39:55.171503+0000","last_undegraded":"2024-03-30T00:39:55.171503+0000","last_fullsized":"2024-03-30T00:39:55.171503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:06:20.485152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171643+0000","last_change":"2024-03-30T00:39:51.179913+0000","last_active":"2024-03-30T00:39:55.171643+0000","last_peered":"2024-03-30T00:39:55.171643+0000","last_clean":"2024-03-30T00:39:55.171643+0000","last_became_active":"2024-03-30T00:39:51.179652+0000","last_became_peered":"2024-03-30T00:39:51.179652+0000","last_unstale":"2024-03-30T00:39:55.171643+0000","last_undegraded":"2024-03-30T00:39:55.171643+0000","last_fullsized":"2024-03-30T00:39:55.171643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:39:59.215019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163237+0000","last_change":"2024-03-30T00:39:51.170941+0000","last_active":"2024-03-30T00:39:55.163237+0000","last_peered":"2024-03-30T00:39:55.163237+0000","last_clean":"2024-03-30T00:39:55.163237+0000","last_became_active":"2024-03-30T00:39:51.166779+0000","last_became_peered":"2024-03-30T00:39:51.166779+0000","last_unstale":"2024-03-30T00:39:55.163237+0000","last_undegraded":"2024-03-30T00:39:55.163237+0000","last_fullsized":"2024-03-30T00:39:55.163237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:12.512040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167960+0000","last_change":"2024-03-30T00:39:51.176410+0000","last_active":"2024-03-30T00:39:55.167960+0000","last_peered":"2024-03-30T00:39:55.167960+0000","last_clean":"2024-03-30T00:39:55.167960+0000","last_became_active":"2024-03-30T00:39:51.175971+0000","last_became_peered":"2024-03-30T00:39:51.175971+0000","last_unstale":"2024-03-30T00:39:55.167960+0000","last_undegraded":"2024-03-30T00:39:55.167960+0000","last_fullsized":"2024-03-30T00:39:55.167960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:37.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.160661+0000","last_change":"2024-03-30T00:39:51.167648+0000","last_active":"2024-03-30T00:39:55.160661+0000","last_peered":"2024-03-30T00:39:55.160661+0000","last_clean":"2024-03-30T00:39:55.160661+0000","last_became_active":"2024-03-30T00:39:51.167382+0000","last_became_peered":"2024-03-30T00:39:51.167382+0000","last_unstale":"2024-03-30T00:39:55.160661+0000","last_undegraded":"2024-03-30T00:39:55.160661+0000","last_fullsized":"2024-03-30T00:39:55.160661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:39:43.189821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.167861+0000","last_change":"2024-03-30T00:39:51.169776+0000","last_active":"2024-03-30T00:39:55.167861+0000","last_peered":"2024-03-30T00:39:55.167861+0000","last_clean":"2024-03-30T00:39:55.167861+0000","last_became_active":"2024-03-30T00:39:51.169462+0000","last_became_peered":"2024-03-30T00:39:51.169462+0000","last_unstale":"2024-03-30T00:39:55.167861+0000","last_undegraded":"2024-03-30T00:39:55.167861+0000","last_fullsized":"2024-03-30T00:39:55.167861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:37:05.851113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.161236+0000","last_change":"2024-03-30T00:39:51.163469+0000","last_active":"2024-03-30T00:39:55.161236+0000","last_peered":"2024-03-30T00:39:55.161236+0000","last_clean":"2024-03-30T00:39:55.161236+0000","last_became_active":"2024-03-30T00:39:51.163200+0000","last_became_peered":"2024-03-30T00:39:51.163200+0000","last_unstale":"2024-03-30T00:39:55.161236+0000","last_undegraded":"2024-03-30T00:39:55.161236+0000","last_fullsized":"2024-03-30T00:39:55.161236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:24:05.856817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.170209+0000","last_change":"2024-03-30T00:39:51.183006+0000","last_active":"2024-03-30T00:39:55.170209+0000","last_peered":"2024-03-30T00:39:55.170209+0000","last_clean":"2024-03-30T00:39:55.170209+0000","last_became_active":"2024-03-30T00:39:51.179310+0000","last_became_peered":"2024-03-30T00:39:51.179310+0000","last_unstale":"2024-03-30T00:39:55.170209+0000","last_undegraded":"2024-03-30T00:39:55.170209+0000","last_fullsized":"2024-03-30T00:39:55.170209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:41.330935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.183618+0000","last_change":"2024-03-30T00:39:51.180621+0000","last_active":"2024-03-30T00:39:55.183618+0000","last_peered":"2024-03-30T00:39:55.183618+0000","last_clean":"2024-03-30T00:39:55.183618+0000","last_became_active":"2024-03-30T00:39:51.175749+0000","last_became_peered":"2024-03-30T00:39:51.175749+0000","last_unstale":"2024-03-30T00:39:55.183618+0000","last_undegraded":"2024-03-30T00:39:55.183618+0000","last_fullsized":"2024-03-30T00:39:55.183618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:27.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.171993+0000","last_change":"2024-03-30T00:39:51.168331+0000","last_active":"2024-03-30T00:39:55.171993+0000","last_peered":"2024-03-30T00:39:55.171993+0000","last_clean":"2024-03-30T00:39:55.171993+0000","last_became_active":"2024-03-30T00:39:51.168052+0000","last_became_peered":"2024-03-30T00:39:51.168052+0000","last_unstale":"2024-03-30T00:39:55.171993+0000","last_undegraded":"2024-03-30T00:39:55.171993+0000","last_fullsized":"2024-03-30T00:39:55.171993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:25.407309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T00:39:55.163158+0000","last_change":"2024-03-30T00:39:51.167745+0000","last_active":"2024-03-30T00:39:55.163158+0000","last_peered":"2024-03-30T00:39:55.163158+0000","last_clean":"2024-03-30T00:39:55.163158+0000","last_became_active":"2024-03-30T00:39:51.166397+0000","last_became_peered":"2024-03-30T00:39:51.166397+0000","last_unstale":"2024-03-30T00:39:55.163158+0000","last_undegraded":"2024-03-30T00:39:55.163158+0000","last_fullsized":"2024-03-30T00:39:55.163158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:50.737942+0000","last_clean_scrub_stamp":"2024-03-30T00:39:50.737942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:50:20.941634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0490000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3170000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:40:20.725 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-30T00:40:20.725 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-30T00:40:20.725 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-30T00:40:20.725 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-30T00:40:20.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.889+0000 7efe00a00640 1 Processor -- start 2024-03-30T00:40:20.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.889+0000 7efe00a00640 1 -- start start 2024-03-30T00:40:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efe00a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efdfc0b2b60 0x7efdfc06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efe00a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdfc0ce070 con 0x7efdfc06ba80 2024-03-30T00:40:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efe00a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdfc0ce350 con 0x7efdfc15c9f0 2024-03-30T00:40:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efe00a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc06cf60 con 0x7efdfc0b2b60 2024-03-30T00:40:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efdf2a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efdfc06ba80 0x7efdfc0caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:43286/0 (socket says 172.21.15.66:43286) 2024-03-30T00:40:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efdf2a00640 1 -- 172.21.15.66:0/3792010250 learned_addr learned my addr 172.21.15.66:0/3792010250 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:20.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efdfbe00640 1 --2- 172.21.15.66:0/3792010250 >> v2:172.21.15.156:3300/0 conn(0x7efdfc0b2b60 0x7efdfc06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efdfaa00640 1 -- 172.21.15.66:0/3792010250 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337872959 0 0) 0x7efdfc0ce070 con 0x7efdfc06ba80 2024-03-30T00:40:20.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efdfaa00640 1 -- 172.21.15.66:0/3792010250 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efde0003530 con 0x7efdfc06ba80 2024-03-30T00:40:20.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efdfbe00640 1 -- 172.21.15.66:0/3792010250 >> v1:172.21.15.156:6789/0 conn(0x7efdfc15c9f0 legacy=0x7efdfc06b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efdfbe00640 1 -- 172.21.15.66:0/3792010250 >> v1:172.21.15.66:6789/0 conn(0x7efdfc06ba80 legacy=0x7efdfc0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.893+0000 7efdfbe00640 1 -- 172.21.15.66:0/3792010250 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0ce630 con 0x7efdfc0b2b60 2024-03-30T00:40:20.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efdfbe00640 1 --2- 172.21.15.66:0/3792010250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0b2b60 0x7efdfc06ca20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009d90 tx=0x7efdf401c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c787ec023b2ed11 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:20.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efdfaa00640 1 -- 172.21.15.66:0/3792010250 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf40a5970 con 0x7efdfc0b2b60 2024-03-30T00:40:20.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efdfaa00640 1 -- 172.21.15.66:0/3792010250 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf40a5ad0 con 0x7efdfc0b2b60 2024-03-30T00:40:20.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efdfaa00640 1 -- 172.21.15.66:0/3792010250 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf40a5e60 con 0x7efdfc0b2b60 2024-03-30T00:40:20.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 -- 172.21.15.66:0/3792010250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0b2b60 msgr2=0x7efdfc06ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 --2- 172.21.15.66:0/3792010250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0b2b60 0x7efdfc06ca20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009d90 tx=0x7efdf401c8b0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 -- 172.21.15.66:0/3792010250 shutdown_connections 2024-03-30T00:40:20.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 --2- 172.21.15.66:0/3792010250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0b2b60 0x7efdfc06ca20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 -- 172.21.15.66:0/3792010250 >> 172.21.15.66:0/3792010250 conn(0x7efdfc06e0d0 msgr2=0x7efdfc06e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:20.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 -- 172.21.15.66:0/3792010250 shutdown_connections 2024-03-30T00:40:20.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 -- 172.21.15.66:0/3792010250 wait complete. 2024-03-30T00:40:20.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 Processor -- start 2024-03-30T00:40:20.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 -- start start 2024-03-30T00:40:20.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc06ba80 0x7efdfc0c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0b2b60 0x7efdfc1f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc15c9f0 0x7efdfc0b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0c7ad0 con 0x7efdfc0b2b60 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0b47a0 con 0x7efdfc15c9f0 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efe00a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0b4a80 con 0x7efdfc06ba80 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efdf2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0b2b60 0x7efdfc1f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efdf2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0b2b60 0x7efdfc1f7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34068/0 (socket says 172.21.15.66:34068) 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.897+0000 7efdf2a00640 1 -- 172.21.15.66:0/3593332688 learned_addr learned my addr 172.21.15.66:0/3593332688 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdfb400640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc06ba80 0x7efdfc0c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdfbe00640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc15c9f0 0x7efdfc0b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf2a00640 1 -- 172.21.15.66:0/3593332688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc06ba80 msgr2=0x7efdfc0c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf2a00640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc06ba80 0x7efdfc0c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf2a00640 1 -- 172.21.15.66:0/3593332688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc15c9f0 msgr2=0x7efdfc0b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:20.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf2a00640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc15c9f0 0x7efdfc0b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf2a00640 1 -- 172.21.15.66:0/3593332688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4009a70 con 0x7efdfc0b2b60 2024-03-30T00:40:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf2a00640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0b2b60 0x7efdfc1f7ca0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efde400c4d0 tx=0x7efde400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf3e00640 1 -- 172.21.15.66:0/3593332688 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4005cf0 con 0x7efdfc0b2b60 2024-03-30T00:40:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf3e00640 1 -- 172.21.15.66:0/3593332688 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde4005e50 con 0x7efdfc0b2b60 2024-03-30T00:40:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf3e00640 1 -- 172.21.15.66:0/3593332688 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4007270 con 0x7efdfc0b2b60 2024-03-30T00:40:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efe00a00640 1 -- 172.21.15.66:0/3593332688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc0b4d60 con 0x7efdfc0b2b60 2024-03-30T00:40:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efe00a00640 1 -- 172.21.15.66:0/3593332688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc0b5320 con 0x7efdfc0b2b60 2024-03-30T00:40:20.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efe00a00640 1 -- 172.21.15.66:0/3593332688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc0b5510 con 0x7efdfc0b2b60 2024-03-30T00:40:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf3e00640 1 -- 172.21.15.66:0/3593332688 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efde40073d0 con 0x7efdfc0b2b60 2024-03-30T00:40:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf3e00640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd8066690 0x7efdd8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.901+0000 7efdf3e00640 1 -- 172.21.15.66:0/3593332688 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efde4083a00 con 0x7efdfc0b2b60 2024-03-30T00:40:20.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.905+0000 7efdfb400640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd8066690 0x7efdd8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:20.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.905+0000 7efdfb400640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd8066690 0x7efdd8068b50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdec000f70 tx=0x7efdec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:20.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:20.905+0000 7efdf3e00640 1 -- 172.21.15.66:0/3593332688 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efde4050b20 con 0x7efdfc0b2b60 2024-03-30T00:40:21.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efe00a00640 1 -- 172.21.15.66:0/3593332688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7efdfc073cf0 con 0x7efdfc0b2b60 2024-03-30T00:40:21.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efdf3e00640 1 -- 172.21.15.66:0/3593332688 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7efde4017040 con 0x7efdfc0b2b60 2024-03-30T00:40:21.111 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:21.111 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efe00a00640 1 -- 172.21.15.66:0/3593332688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd8066690 msgr2=0x7efdd8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efe00a00640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd8066690 0x7efdd8068b50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdec000f70 tx=0x7efdec009040 comp rx=0 tx=0).stop 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efe00a00640 1 -- 172.21.15.66:0/3593332688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0b2b60 msgr2=0x7efdfc1f7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efe00a00640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0b2b60 0x7efdfc1f7ca0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efde400c4d0 tx=0x7efde400c990 comp rx=0 tx=0).stop 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efe00a00640 1 -- 172.21.15.66:0/3593332688 shutdown_connections 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efe00a00640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc15c9f0 0x7efdfc0b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efe00a00640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd8066690 0x7efdd8068b50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efe00a00640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0b2b60 0x7efdfc1f7ca0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efe00a00640 1 --2- 172.21.15.66:0/3593332688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc06ba80 0x7efdfc0c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.101+0000 7efe00a00640 1 -- 172.21.15.66:0/3593332688 >> 172.21.15.66:0/3593332688 conn(0x7efdfc06e0d0 msgr2=0x7efdfc15d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.105+0000 7efe00a00640 1 -- 172.21.15.66:0/3593332688 shutdown_connections 2024-03-30T00:40:21.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.105+0000 7efe00a00640 1 -- 172.21.15.66:0/3593332688 wait complete. 2024-03-30T00:40:21.133 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-30T00:40:21.133 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:21.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.261+0000 7f639e200640 1 Processor -- start 2024-03-30T00:40:21.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.261+0000 7f639e200640 1 -- start start 2024-03-30T00:40:21.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f639e200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f639815ca80 0x7f639806b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f639e200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f63980cb570 0x7f63980cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f639e200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63980cb0e0 con 0x7f63980cf2f0 2024-03-30T00:40:21.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f639e200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63980cf7f0 con 0x7f63980cb570 2024-03-30T00:40:21.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f639e200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63980cfad0 con 0x7f639815ca80 2024-03-30T00:40:21.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f6397e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f63980cb570 0x7f63980cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f6397e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f63980cb570 0x7f63980cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34072/0 (socket says 172.21.15.66:34072) 2024-03-30T00:40:21.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f6397e00640 1 -- 172.21.15.66:0/1484546570 learned_addr learned my addr 172.21.15.66:0/1484546570 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:21.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f6397400640 1 --2- 172.21.15.66:0/1484546570 >> v2:172.21.15.156:3300/0 conn(0x7f639815ca80 0x7f639806b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f6397e00640 1 -- 172.21.15.66:0/1484546570 >> v2:172.21.15.156:3300/0 conn(0x7f639815ca80 msgr2=0x7f639806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:21.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f6397e00640 1 --2- 172.21.15.66:0/1484546570 >> v2:172.21.15.156:3300/0 conn(0x7f639815ca80 0x7f639806b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f6397e00640 1 -- 172.21.15.66:0/1484546570 >> v1:172.21.15.66:6789/0 conn(0x7f63980cf2f0 legacy=0x7f639806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.265+0000 7f6397e00640 1 -- 172.21.15.66:0/1484546570 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63980cad70 con 0x7f63980cb570 2024-03-30T00:40:21.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f6397e00640 1 --2- 172.21.15.66:0/1484546570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63980cb570 0x7f63980cba20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6398070ed0 tx=0x7f639001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb31f9cbcf48b83d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:21.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f6396a00640 1 -- 172.21.15.66:0/1484546570 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900a8a10 con 0x7f63980cb570 2024-03-30T00:40:21.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f6396a00640 1 -- 172.21.15.66:0/1484546570 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63900a8b70 con 0x7f63980cb570 2024-03-30T00:40:21.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f6396a00640 1 -- 172.21.15.66:0/1484546570 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900b0440 con 0x7f63980cb570 2024-03-30T00:40:21.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f639e200640 1 -- 172.21.15.66:0/1484546570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63980cb570 msgr2=0x7f63980cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f639e200640 1 --2- 172.21.15.66:0/1484546570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63980cb570 0x7f63980cba20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6398070ed0 tx=0x7f639001ed10 comp rx=0 tx=0).stop 2024-03-30T00:40:21.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f639e200640 1 -- 172.21.15.66:0/1484546570 shutdown_connections 2024-03-30T00:40:21.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f639e200640 1 --2- 172.21.15.66:0/1484546570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63980cb570 0x7f63980cba20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f639e200640 1 --2- 172.21.15.66:0/1484546570 >> v2:172.21.15.156:3300/0 conn(0x7f639815ca80 0x7f639806b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f639e200640 1 -- 172.21.15.66:0/1484546570 >> 172.21.15.66:0/1484546570 conn(0x7f63980b2090 msgr2=0x7f63980b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:21.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f639e200640 1 -- 172.21.15.66:0/1484546570 shutdown_connections 2024-03-30T00:40:21.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f639e200640 1 -- 172.21.15.66:0/1484546570 wait complete. 2024-03-30T00:40:21.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f639e200640 1 Processor -- start 2024-03-30T00:40:21.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f639e200640 1 -- start start 2024-03-30T00:40:21.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.269+0000 7f639e200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980cb570 0x7f639815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f639e200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63980cf2f0 0x7f639809dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f639e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f639815ca80 0x7f639809bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f639e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639815b780 con 0x7f639815ca80 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f639e200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63980b1c70 con 0x7f63980cf2f0 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f639e200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639809b290 con 0x7f63980cb570 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f6397e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f639815ca80 0x7f639809bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f6397e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f639815ca80 0x7f639809bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34084/0 (socket says 172.21.15.66:34084) 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f6397e00640 1 -- 172.21.15.66:0/966088834 learned_addr learned my addr 172.21.15.66:0/966088834 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f6397400640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980cb570 0x7f639815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f638ea00640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63980cf2f0 0x7f639809dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f6397e00640 1 -- 172.21.15.66:0/966088834 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980cb570 msgr2=0x7f639815b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f6397e00640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980cb570 0x7f639815b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f6397e00640 1 -- 172.21.15.66:0/966088834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63980cf2f0 msgr2=0x7f639809dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f6397e00640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63980cf2f0 0x7f639809dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f6397e00640 1 -- 172.21.15.66:0/966088834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63900094d0 con 0x7f639815ca80 2024-03-30T00:40:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f6397e00640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f639815ca80 0x7f639809bba0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63900b3040 tx=0x7f639001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f638fe00640 1 -- 172.21.15.66:0/966088834 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900b0e20 con 0x7f639815ca80 2024-03-30T00:40:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f638fe00640 1 -- 172.21.15.66:0/966088834 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63900bb480 con 0x7f639815ca80 2024-03-30T00:40:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f638fe00640 1 -- 172.21.15.66:0/966088834 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900bb7f0 con 0x7f639815ca80 2024-03-30T00:40:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f639e200640 1 -- 172.21.15.66:0/966088834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639809b570 con 0x7f639815ca80 2024-03-30T00:40:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f639e200640 1 -- 172.21.15.66:0/966088834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63980c7670 con 0x7f639815ca80 2024-03-30T00:40:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f639e200640 1 -- 172.21.15.66:0/966088834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6398066c30 con 0x7f639815ca80 2024-03-30T00:40:21.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.273+0000 7f638fe00640 1 -- 172.21.15.66:0/966088834 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f63900bb950 con 0x7f639815ca80 2024-03-30T00:40:21.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.277+0000 7f638fe00640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6374066690 0x7f6374068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.277+0000 7f638fe00640 1 -- 172.21.15.66:0/966088834 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f63901229b0 con 0x7f639815ca80 2024-03-30T00:40:21.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.277+0000 7f6397400640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6374066690 0x7f6374068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.277+0000 7f6397400640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6374066690 0x7f6374068b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63981f58b0 tx=0x7f6388009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:21.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.281+0000 7f638fe00640 1 -- 172.21.15.66:0/966088834 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f63900efb60 con 0x7f639815ca80 2024-03-30T00:40:21.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.441+0000 7f639e200640 1 -- 172.21.15.66:0/966088834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63980b2bb0 con 0x7f639815ca80 2024-03-30T00:40:21.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.441+0000 7f638fe00640 1 -- 172.21.15.66:0/966088834 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f63900b5090 con 0x7f639815ca80 2024-03-30T00:40:21.452 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:21.452 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 -- 172.21.15.66:0/966088834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6374066690 msgr2=0x7f6374068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6374066690 0x7f6374068b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63981f58b0 tx=0x7f6388009830 comp rx=0 tx=0).stop 2024-03-30T00:40:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 -- 172.21.15.66:0/966088834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f639815ca80 msgr2=0x7f639809bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f639815ca80 0x7f639809bba0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63900b3040 tx=0x7f639001bab0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 -- 172.21.15.66:0/966088834 shutdown_connections 2024-03-30T00:40:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6374066690 0x7f6374068b50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f639815ca80 0x7f639809bba0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63980cf2f0 0x7f639809dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 --2- 172.21.15.66:0/966088834 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980cb570 0x7f639815b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 -- 172.21.15.66:0/966088834 >> 172.21.15.66:0/966088834 conn(0x7f63980b2090 msgr2=0x7f63980ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:21.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 -- 172.21.15.66:0/966088834 shutdown_connections 2024-03-30T00:40:21.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.445+0000 7f639e200640 1 -- 172.21.15.66:0/966088834 wait complete. 2024-03-30T00:40:21.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:21.473 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:21.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f1957200640 1 Processor -- start 2024-03-30T00:40:21.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f1957200640 1 -- start start 2024-03-30T00:40:21.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f1957200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19500af420 0x7f19500cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f1957200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19500cbb90 con 0x7f195015de30 2024-03-30T00:40:21.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f1957200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19500ce1b0 con 0x7f19500cee50 2024-03-30T00:40:21.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f1957200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19500afa10 con 0x7f19500af420 2024-03-30T00:40:21.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194fe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f195015de30 0x7f19500ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:43306/0 (socket says 172.21.15.66:43306) 2024-03-30T00:40:21.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194fe00640 1 -- 172.21.15.66:0/4083896386 learned_addr learned my addr 172.21.15.66:0/4083896386 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f1955200640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19500af420 0x7f19500cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072926449 0 0) 0x7f19500cbb90 con 0x7f195015de30 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1934003530 con 0x7f195015de30 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780112228 0 0) 0x7f19500ce1b0 con 0x7f19500cee50 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19500cbb90 con 0x7f19500cee50 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2342431793 0 0) 0x7f1934003530 con 0x7f195015de30 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19500af420 msgr2=0x7f19500cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19500af420 0x7f19500cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 >> v1:172.21.15.156:6789/0 conn(0x7f19500cee50 legacy=0x7f195006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19500cf430 con 0x7f195015de30 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f1940001a00 con 0x7f195015de30 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 >> v1:172.21.15.66:6789/0 conn(0x7f195015de30 legacy=0x7f19500ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1934003b40 0x7f1934003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19500af420 0x7f19340047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1934004d10 0x7f1934005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19500afa10 con 0x7f1934004d10 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193400bb20 con 0x7f1934003b40 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.609+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193400bcb0 con 0x7f19500af420 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f1955200640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1934004d10 0x7f1934005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f1947400640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19500af420 0x7f19340047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f194fe00640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1934003b40 0x7f1934003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f1955200640 1 -- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19500af420 msgr2=0x7f19340047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f1955200640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19500af420 0x7f19340047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f1955200640 1 -- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1934003b40 msgr2=0x7f1934003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f1955200640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1934003b40 0x7f1934003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f1955200640 1 -- 172.21.15.66:0/4083896386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934003530 con 0x7f1934004d10 2024-03-30T00:40:21.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f194fe00640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1934003b40 0x7f1934003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:21.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f1955200640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1934004d10 0x7f1934005190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1948009b10 tx=0x7f194801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2de21555f482e79 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:21.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19480a5660 con 0x7f1934004d10 2024-03-30T00:40:21.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19480a57c0 con 0x7f1934004d10 2024-03-30T00:40:21.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.613+0000 7f194f400640 1 -- 172.21.15.66:0/4083896386 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19480a5b70 con 0x7f1934004d10 2024-03-30T00:40:21.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 -- 172.21.15.66:0/4083896386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1934004d10 msgr2=0x7f1934005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 --2- 172.21.15.66:0/4083896386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1934004d10 0x7f1934005190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1948009b10 tx=0x7f194801ca70 comp rx=0 tx=0).stop 2024-03-30T00:40:21.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1955200640 1 -- 172.21.15.66:0/4083896386 reap_dead start 2024-03-30T00:40:21.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 -- 172.21.15.66:0/4083896386 shutdown_connections 2024-03-30T00:40:21.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 -- 172.21.15.66:0/4083896386 >> 172.21.15.66:0/4083896386 conn(0x7f19500d0540 msgr2=0x7f19500d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:21.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 -- 172.21.15.66:0/4083896386 shutdown_connections 2024-03-30T00:40:21.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 -- 172.21.15.66:0/4083896386 wait complete. 2024-03-30T00:40:21.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 Processor -- start 2024-03-30T00:40:21.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 -- start start 2024-03-30T00:40:21.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1934004d10 0x7f1950154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950159b10 0x7f19500b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1950158990 0x7f19500b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1950155050 con 0x7f1950159b10 2024-03-30T00:40:21.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19501591c0 con 0x7f1934004d10 2024-03-30T00:40:21.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1957200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19500b1410 con 0x7f1950158990 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1947400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950159b10 0x7f19500b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1947400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950159b10 0x7f19500b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34104/0 (socket says 172.21.15.66:34104) 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1947400640 1 -- 172.21.15.66:0/3194162719 learned_addr learned my addr 172.21.15.66:0/3194162719 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f194fe00640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1934004d10 0x7f1950154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1955200640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1950158990 0x7f19500b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1947400640 1 -- 172.21.15.66:0/3194162719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1950158990 msgr2=0x7f19500b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1947400640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1950158990 0x7f19500b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1947400640 1 -- 172.21.15.66:0/3194162719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1934004d10 msgr2=0x7f1950154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1947400640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1934004d10 0x7f1950154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1947400640 1 -- 172.21.15.66:0/3194162719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19500b1de0 con 0x7f1950159b10 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f194fe00640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1934004d10 0x7f1950154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:21.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.617+0000 7f1947400640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950159b10 0x7f19500b18a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f19380129a0 tx=0x7f1938012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:21.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.621+0000 7f194cc00640 1 -- 172.21.15.66:0/3194162719 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19380070a0 con 0x7f1950159b10 2024-03-30T00:40:21.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.621+0000 7f194cc00640 1 -- 172.21.15.66:0/3194162719 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1938010bd0 con 0x7f1950159b10 2024-03-30T00:40:21.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.621+0000 7f1957200640 1 -- 172.21.15.66:0/3194162719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19500b0ae0 con 0x7f1950159b10 2024-03-30T00:40:21.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.621+0000 7f194cc00640 1 -- 172.21.15.66:0/3194162719 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938016480 con 0x7f1950159b10 2024-03-30T00:40:21.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.621+0000 7f1957200640 1 -- 172.21.15.66:0/3194162719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19500b10a0 con 0x7f1950159b10 2024-03-30T00:40:21.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.621+0000 7f1957200640 1 -- 172.21.15.66:0/3194162719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19500cdf60 con 0x7f1950159b10 2024-03-30T00:40:21.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.621+0000 7f194cc00640 1 -- 172.21.15.66:0/3194162719 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f19380165e0 con 0x7f1950159b10 2024-03-30T00:40:21.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.621+0000 7f194cc00640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f192c0666c0 0x7f192c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.621+0000 7f194cc00640 1 -- 172.21.15.66:0/3194162719 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1938087d10 con 0x7f1950159b10 2024-03-30T00:40:21.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.621+0000 7f194fe00640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f192c0666c0 0x7f192c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.625+0000 7f194fe00640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f192c0666c0 0x7f192c068b80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1940001590 tx=0x7f1940009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:21.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.625+0000 7f194cc00640 1 -- 172.21.15.66:0/3194162719 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f193808d020 con 0x7f1950159b10 2024-03-30T00:40:21.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.785+0000 7f1957200640 1 -- 172.21.15.66:0/3194162719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1950066c30 con 0x7f1950159b10 2024-03-30T00:40:21.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.785+0000 7f194cc00640 1 -- 172.21.15.66:0/3194162719 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1938054cf0 con 0x7f1950159b10 2024-03-30T00:40:21.796 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:21.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 -- 172.21.15.66:0/3194162719 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f192c0666c0 msgr2=0x7f192c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f192c0666c0 0x7f192c068b80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1940001590 tx=0x7f1940009040 comp rx=0 tx=0).stop 2024-03-30T00:40:21.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 -- 172.21.15.66:0/3194162719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950159b10 msgr2=0x7f19500b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950159b10 0x7f19500b18a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f19380129a0 tx=0x7f1938012e60 comp rx=0 tx=0).stop 2024-03-30T00:40:21.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 -- 172.21.15.66:0/3194162719 shutdown_connections 2024-03-30T00:40:21.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1950158990 0x7f19500b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f192c0666c0 0x7f192c068b80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950159b10 0x7f19500b18a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 --2- 172.21.15.66:0/3194162719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1934004d10 0x7f1950154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 -- 172.21.15.66:0/3194162719 >> 172.21.15.66:0/3194162719 conn(0x7f19500d0540 msgr2=0x7f195006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:21.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 -- 172.21.15.66:0/3194162719 shutdown_connections 2024-03-30T00:40:21.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.789+0000 7f1957200640 1 -- 172.21.15.66:0/3194162719 wait complete. 2024-03-30T00:40:21.818 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:21.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f2587000640 1 Processor -- start 2024-03-30T00:40:21.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f2587000640 1 -- start start 2024-03-30T00:40:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f2587000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f25800cad00 0x7f25800c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f2587000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25800cd2d0 0x7f25800cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f2587000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25800c8e80 con 0x7f25800cc8d0 2024-03-30T00:40:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f2587000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25800cbec0 con 0x7f25800cad00 2024-03-30T00:40:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f2587000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25800cb620 con 0x7f25800cd2d0 2024-03-30T00:40:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f257fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f25800cad00 0x7f25800c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f257fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f25800cad00 0x7f25800c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34110/0 (socket says 172.21.15.66:34110) 2024-03-30T00:40:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f257fe00640 1 -- 172.21.15.66:0/3473040743 learned_addr learned my addr 172.21.15.66:0/3473040743 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f2577e00640 1 --2- 172.21.15.66:0/3473040743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25800cd2d0 0x7f25800cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f257fe00640 1 -- 172.21.15.66:0/3473040743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25800cd2d0 msgr2=0x7f25800cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:21.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f257fe00640 1 --2- 172.21.15.66:0/3473040743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25800cd2d0 0x7f25800cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f257fe00640 1 -- 172.21.15.66:0/3473040743 >> v1:172.21.15.66:6789/0 conn(0x7f25800cc8d0 legacy=0x7f258006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f257fe00640 1 -- 172.21.15.66:0/3473040743 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25800ca660 con 0x7f25800cad00 2024-03-30T00:40:21.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.953+0000 7f2577e00640 1 --2- 172.21.15.66:0/3473040743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25800cd2d0 0x7f25800cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:21.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f257fe00640 1 --2- 172.21.15.66:0/3473040743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25800cad00 0x7f25800c8940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2570009870 tx=0x7f257001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2629145f8d4d6c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:21.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f257f400640 1 -- 172.21.15.66:0/3473040743 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25700a95f0 con 0x7f25800cad00 2024-03-30T00:40:21.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f257f400640 1 -- 172.21.15.66:0/3473040743 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25700a9750 con 0x7f25800cad00 2024-03-30T00:40:21.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f257f400640 1 -- 172.21.15.66:0/3473040743 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25700a9ae0 con 0x7f25800cad00 2024-03-30T00:40:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f2587000640 1 -- 172.21.15.66:0/3473040743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25800cad00 msgr2=0x7f25800c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f2587000640 1 --2- 172.21.15.66:0/3473040743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25800cad00 0x7f25800c8940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2570009870 tx=0x7f257001c530 comp rx=0 tx=0).stop 2024-03-30T00:40:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f2587000640 1 -- 172.21.15.66:0/3473040743 shutdown_connections 2024-03-30T00:40:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f2587000640 1 --2- 172.21.15.66:0/3473040743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25800cd2d0 0x7f25800cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f2587000640 1 --2- 172.21.15.66:0/3473040743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25800cad00 0x7f25800c8940 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f2587000640 1 -- 172.21.15.66:0/3473040743 >> 172.21.15.66:0/3473040743 conn(0x7f2580069f60 msgr2=0x7f258006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f2587000640 1 -- 172.21.15.66:0/3473040743 shutdown_connections 2024-03-30T00:40:21.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f2587000640 1 -- 172.21.15.66:0/3473040743 wait complete. 2024-03-30T00:40:21.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f2587000640 1 Processor -- start 2024-03-30T00:40:21.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.957+0000 7f2587000640 1 -- start start 2024-03-30T00:40:21.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2587000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25800cad00 0x7f25801f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2587000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25800cc8d0 0x7f2580070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2587000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25800cd2d0 0x7f2580071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2587000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2580165850 con 0x7f25800cd2d0 2024-03-30T00:40:21.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2587000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25801f5ea0 con 0x7f25800cad00 2024-03-30T00:40:21.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2587000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25800af190 con 0x7f25800cc8d0 2024-03-30T00:40:21.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2585000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25800cd2d0 0x7f2580071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2585000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25800cd2d0 0x7f2580071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34118/0 (socket says 172.21.15.66:34118) 2024-03-30T00:40:21.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2585000640 1 -- 172.21.15.66:0/3411430904 learned_addr learned my addr 172.21.15.66:0/3411430904 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:21.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2585000640 1 -- 172.21.15.66:0/3411430904 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25800cc8d0 msgr2=0x7f2580070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2585000640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25800cc8d0 0x7f2580070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2585000640 1 -- 172.21.15.66:0/3411430904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25800cad00 msgr2=0x7f25801f68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:21.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2585000640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25800cad00 0x7f25801f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:21.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2585000640 1 -- 172.21.15.66:0/3411430904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25700094d0 con 0x7f25800cd2d0 2024-03-30T00:40:21.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2585000640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25800cd2d0 0x7f2580071460 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f257800c7a0 tx=0x7f257800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:21.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f257cc00640 1 -- 172.21.15.66:0/3411430904 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578003b80 con 0x7f25800cd2d0 2024-03-30T00:40:21.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f257cc00640 1 -- 172.21.15.66:0/3411430904 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2578003ce0 con 0x7f25800cd2d0 2024-03-30T00:40:21.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f257cc00640 1 -- 172.21.15.66:0/3411430904 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800aa90 con 0x7f25800cd2d0 2024-03-30T00:40:21.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2587000640 1 -- 172.21.15.66:0/3411430904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25801f6420 con 0x7f25800cd2d0 2024-03-30T00:40:21.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2587000640 1 -- 172.21.15.66:0/3411430904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25801f60b0 con 0x7f25800cd2d0 2024-03-30T00:40:21.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f2587000640 1 -- 172.21.15.66:0/3411430904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2580066c30 con 0x7f25800cd2d0 2024-03-30T00:40:21.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.961+0000 7f257cc00640 1 -- 172.21.15.66:0/3411430904 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f257800abf0 con 0x7f25800cd2d0 2024-03-30T00:40:21.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.965+0000 7f257cc00640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f255c066690 0x7f255c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:21.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.965+0000 7f257cc00640 1 -- 172.21.15.66:0/3411430904 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f25780842f0 con 0x7f25800cd2d0 2024-03-30T00:40:21.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.965+0000 7f257fe00640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f255c066690 0x7f255c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:21.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.965+0000 7f257fe00640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f255c066690 0x7f255c068b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25700ae000 tx=0x7f2570008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:21.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:21.969+0000 7f257cc00640 1 -- 172.21.15.66:0/3411430904 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2578051360 con 0x7f25800cd2d0 2024-03-30T00:40:22.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.129+0000 7f2587000640 1 -- 172.21.15.66:0/3411430904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25801f65b0 con 0x7f25800cd2d0 2024-03-30T00:40:22.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.129+0000 7f257cc00640 1 -- 172.21.15.66:0/3411430904 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2578054980 con 0x7f25800cd2d0 2024-03-30T00:40:22.140 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 -- 172.21.15.66:0/3411430904 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f255c066690 msgr2=0x7f255c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f255c066690 0x7f255c068b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25700ae000 tx=0x7f2570008040 comp rx=0 tx=0).stop 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 -- 172.21.15.66:0/3411430904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25800cd2d0 msgr2=0x7f2580071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25800cd2d0 0x7f2580071460 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f257800c7a0 tx=0x7f257800cc60 comp rx=0 tx=0).stop 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 -- 172.21.15.66:0/3411430904 shutdown_connections 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f255c066690 0x7f255c068b50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25800cd2d0 0x7f2580071460 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25800cc8d0 0x7f2580070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 --2- 172.21.15.66:0/3411430904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25800cad00 0x7f25801f68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 -- 172.21.15.66:0/3411430904 >> 172.21.15.66:0/3411430904 conn(0x7f2580069f60 msgr2=0x7f258006cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 -- 172.21.15.66:0/3411430904 shutdown_connections 2024-03-30T00:40:22.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.133+0000 7f2587000640 1 -- 172.21.15.66:0/3411430904 wait complete. 2024-03-30T00:40:22.161 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:22.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.293+0000 7fc2dcc00640 1 Processor -- start 2024-03-30T00:40:22.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.293+0000 7fc2dcc00640 1 -- start start 2024-03-30T00:40:22.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2dcc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d815a640 0x7fc2d806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2dcc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc2d80ce520 0x7fc2d80ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2dcc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2d80cabb0 0x7fc2d80cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2dcc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d806d070 con 0x7fc2d80cabb0 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2dcc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d80cbba0 con 0x7fc2d80ce520 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2dcc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d806e2c0 con 0x7fc2d815a640 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d6800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2d80cabb0 0x7fc2d80cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d6800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2d80cabb0 0x7fc2d80cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34134/0 (socket says 172.21.15.66:34134) 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d6800640 1 -- 172.21.15.66:0/342578627 learned_addr learned my addr 172.21.15.66:0/342578627 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2cd400640 1 --2- 172.21.15.66:0/342578627 >> v2:172.21.15.156:3300/0 conn(0x7fc2d80ce520 0x7fc2d80ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d5e00640 1 --2- 172.21.15.66:0/342578627 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d815a640 0x7fc2d806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d6800640 1 -- 172.21.15.66:0/342578627 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d815a640 msgr2=0x7fc2d806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d6800640 1 --2- 172.21.15.66:0/342578627 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d815a640 0x7fc2d806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d6800640 1 -- 172.21.15.66:0/342578627 >> v2:172.21.15.156:3300/0 conn(0x7fc2d80ce520 msgr2=0x7fc2d80ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:22.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d6800640 1 --2- 172.21.15.66:0/342578627 >> v2:172.21.15.156:3300/0 conn(0x7fc2d80ce520 0x7fc2d80ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d6800640 1 -- 172.21.15.66:0/342578627 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d80cee60 con 0x7fc2d80cabb0 2024-03-30T00:40:22.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d5e00640 1 --2- 172.21.15.66:0/342578627 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d815a640 0x7fc2d806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:22.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d6800640 1 --2- 172.21.15.66:0/342578627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2d80cabb0 0x7fc2d80cb000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80cbe00 tx=0x7fc2d001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9bd395ae52b15f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:22.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.297+0000 7fc2d5400640 1 -- 172.21.15.66:0/342578627 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00a5840 con 0x7fc2d80cabb0 2024-03-30T00:40:22.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2d5400640 1 -- 172.21.15.66:0/342578627 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d00a59a0 con 0x7fc2d80cabb0 2024-03-30T00:40:22.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2d5400640 1 -- 172.21.15.66:0/342578627 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00a5d30 con 0x7fc2d80cabb0 2024-03-30T00:40:22.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 -- 172.21.15.66:0/342578627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2d80cabb0 msgr2=0x7fc2d80cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 --2- 172.21.15.66:0/342578627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2d80cabb0 0x7fc2d80cb000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80cbe00 tx=0x7fc2d001c9c0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 -- 172.21.15.66:0/342578627 shutdown_connections 2024-03-30T00:40:22.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 --2- 172.21.15.66:0/342578627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2d80cabb0 0x7fc2d80cb000 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 --2- 172.21.15.66:0/342578627 >> v2:172.21.15.156:3300/0 conn(0x7fc2d80ce520 0x7fc2d80ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 --2- 172.21.15.66:0/342578627 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d815a640 0x7fc2d806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 -- 172.21.15.66:0/342578627 >> 172.21.15.66:0/342578627 conn(0x7fc2d8069f60 msgr2=0x7fc2d806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:22.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 -- 172.21.15.66:0/342578627 shutdown_connections 2024-03-30T00:40:22.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 -- 172.21.15.66:0/342578627 wait complete. 2024-03-30T00:40:22.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 Processor -- start 2024-03-30T00:40:22.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 -- start start 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d80cabb0 0x7fc2d80715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d80ce520 0x7fc2d80abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2d815a640 0x7fc2d80ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d8163c10 con 0x7fc2d815a640 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d81630b0 con 0x7fc2d80cabb0 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2dcc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d8163370 con 0x7fc2d80ce520 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2d6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2d815a640 0x7fc2d80ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2d6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2d815a640 0x7fc2d80ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34144/0 (socket says 172.21.15.66:34144) 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2d6800640 1 -- 172.21.15.66:0/591446821 learned_addr learned my addr 172.21.15.66:0/591446821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2cd400640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d80ce520 0x7fc2d80abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.301+0000 7fc2d5e00640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d80cabb0 0x7fc2d80715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2d6800640 1 -- 172.21.15.66:0/591446821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d80ce520 msgr2=0x7fc2d80abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2d6800640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d80ce520 0x7fc2d80abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2d6800640 1 -- 172.21.15.66:0/591446821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d80cabb0 msgr2=0x7fc2d80715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2d6800640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d80cabb0 0x7fc2d80715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2d6800640 1 -- 172.21.15.66:0/591446821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d0009b30 con 0x7fc2d815a640 2024-03-30T00:40:22.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2d6800640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2d815a640 0x7fc2d80ac170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00af040 tx=0x7fc2d0019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:22.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2cea00640 1 -- 172.21.15.66:0/591446821 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00b8670 con 0x7fc2d815a640 2024-03-30T00:40:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2cea00640 1 -- 172.21.15.66:0/591446821 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d00b87d0 con 0x7fc2d815a640 2024-03-30T00:40:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2cea00640 1 -- 172.21.15.66:0/591446821 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00b8b80 con 0x7fc2d815a640 2024-03-30T00:40:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2dcc00640 1 -- 172.21.15.66:0/591446821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d8163650 con 0x7fc2d815a640 2024-03-30T00:40:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2dcc00640 1 -- 172.21.15.66:0/591446821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d80cbfb0 con 0x7fc2d815a640 2024-03-30T00:40:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2dcc00640 1 -- 172.21.15.66:0/591446821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d80cc1a0 con 0x7fc2d815a640 2024-03-30T00:40:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2cea00640 1 -- 172.21.15.66:0/591446821 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc2d00b8ce0 con 0x7fc2d815a640 2024-03-30T00:40:22.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.305+0000 7fc2cea00640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2b4066690 0x7fc2b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.309+0000 7fc2cea00640 1 -- 172.21.15.66:0/591446821 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc2d011f930 con 0x7fc2d815a640 2024-03-30T00:40:22.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.309+0000 7fc2d5e00640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2b4066690 0x7fc2b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.309+0000 7fc2d5e00640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2b4066690 0x7fc2b4068b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80045f0 tx=0x7fc2c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.313+0000 7fc2cea00640 1 -- 172.21.15.66:0/591446821 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc2d00ecc40 con 0x7fc2d815a640 2024-03-30T00:40:22.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 -- 172.21.15.66:0/591446821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2d8066c30 con 0x7fc2d815a640 2024-03-30T00:40:22.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2cea00640 1 -- 172.21.15.66:0/591446821 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc2d00f0260 con 0x7fc2d815a640 2024-03-30T00:40:22.481 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:22.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 -- 172.21.15.66:0/591446821 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2b4066690 msgr2=0x7fc2b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2b4066690 0x7fc2b4068b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80045f0 tx=0x7fc2c8008040 comp rx=0 tx=0).stop 2024-03-30T00:40:22.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 -- 172.21.15.66:0/591446821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2d815a640 msgr2=0x7fc2d80ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2d815a640 0x7fc2d80ac170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00af040 tx=0x7fc2d0019760 comp rx=0 tx=0).stop 2024-03-30T00:40:22.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 -- 172.21.15.66:0/591446821 shutdown_connections 2024-03-30T00:40:22.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2b4066690 0x7fc2b4068b50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2d815a640 0x7fc2d80ac170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d80ce520 0x7fc2d80abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 --2- 172.21.15.66:0/591446821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d80cabb0 0x7fc2d80715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 -- 172.21.15.66:0/591446821 >> 172.21.15.66:0/591446821 conn(0x7fc2d8069f60 msgr2=0x7fc2d80ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:22.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 -- 172.21.15.66:0/591446821 shutdown_connections 2024-03-30T00:40:22.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.473+0000 7fc2dcc00640 1 -- 172.21.15.66:0/591446821 wait complete. 2024-03-30T00:40:22.503 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:22.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.637+0000 7f2c84a00640 1 Processor -- start 2024-03-30T00:40:22.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.637+0000 7f2c84a00640 1 -- start start 2024-03-30T00:40:22.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.637+0000 7f2c84a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2c8015ca80 0x7f2c8006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.637+0000 7f2c84a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c800cb0e0 con 0x7f2c800cf2f0 2024-03-30T00:40:22.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.637+0000 7f2c84a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c800cf7f0 con 0x7f2c800cb570 2024-03-30T00:40:22.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.637+0000 7f2c84a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c800cfad0 con 0x7f2c8015ca80 2024-03-30T00:40:22.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.637+0000 7f2c7de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2c8015ca80 0x7f2c8006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.637+0000 7f2c7de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2c8015ca80 0x7f2c8006b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34156/0 (socket says 172.21.15.66:34156) 2024-03-30T00:40:22.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.637+0000 7f2c7de00640 1 -- 172.21.15.66:0/1486983147 learned_addr learned my addr 172.21.15.66:0/1486983147 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:22.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c7de00640 1 -- 172.21.15.66:0/1486983147 >> v1:172.21.15.156:6789/0 conn(0x7f2c800cb570 legacy=0x7f2c800cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:22.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c7de00640 1 -- 172.21.15.66:0/1486983147 >> v1:172.21.15.66:6789/0 conn(0x7f2c800cf2f0 legacy=0x7f2c8006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c7de00640 1 -- 172.21.15.66:0/1486983147 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800cad70 con 0x7f2c8015ca80 2024-03-30T00:40:22.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c7de00640 1 --2- 172.21.15.66:0/1486983147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c8015ca80 0x7f2c8006b0b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009870 tx=0x7f2c7001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578e3716954dd069 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:22.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c7d400640 1 -- 172.21.15.66:0/1486983147 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c700a8a10 con 0x7f2c8015ca80 2024-03-30T00:40:22.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c7d400640 1 -- 172.21.15.66:0/1486983147 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c700a8b70 con 0x7f2c8015ca80 2024-03-30T00:40:22.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c7d400640 1 -- 172.21.15.66:0/1486983147 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c700b0440 con 0x7f2c8015ca80 2024-03-30T00:40:22.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c84a00640 1 -- 172.21.15.66:0/1486983147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c8015ca80 msgr2=0x7f2c8006b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c84a00640 1 --2- 172.21.15.66:0/1486983147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c8015ca80 0x7f2c8006b0b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009870 tx=0x7f2c7001ed10 comp rx=0 tx=0).stop 2024-03-30T00:40:22.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c84a00640 1 -- 172.21.15.66:0/1486983147 shutdown_connections 2024-03-30T00:40:22.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c84a00640 1 --2- 172.21.15.66:0/1486983147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c8015ca80 0x7f2c8006b0b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c84a00640 1 -- 172.21.15.66:0/1486983147 >> 172.21.15.66:0/1486983147 conn(0x7f2c800b2090 msgr2=0x7f2c800b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:22.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c84a00640 1 -- 172.21.15.66:0/1486983147 shutdown_connections 2024-03-30T00:40:22.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.641+0000 7f2c84a00640 1 -- 172.21.15.66:0/1486983147 wait complete. 2024-03-30T00:40:22.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c84a00640 1 Processor -- start 2024-03-30T00:40:22.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c84a00640 1 -- start start 2024-03-30T00:40:22.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c84a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c800cb570 0x7f2c80157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c84a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c800cf2f0 0x7f2c8009a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c84a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c8015ca80 0x7f2c8009d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c84a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80158290 con 0x7f2c800cf2f0 2024-03-30T00:40:22.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c84a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8006e430 con 0x7f2c8015ca80 2024-03-30T00:40:22.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c84a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8006e710 con 0x7f2c800cb570 2024-03-30T00:40:22.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c75400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c800cf2f0 0x7f2c8009a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c75400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c800cf2f0 0x7f2c8009a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34168/0 (socket says 172.21.15.66:34168) 2024-03-30T00:40:22.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c75400640 1 -- 172.21.15.66:0/3335986202 learned_addr learned my addr 172.21.15.66:0/3335986202 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:22.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c7de00640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c800cb570 0x7f2c80157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c7e800640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c8015ca80 0x7f2c8009d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c75400640 1 -- 172.21.15.66:0/3335986202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c800cb570 msgr2=0x7f2c80157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c75400640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c800cb570 0x7f2c80157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c75400640 1 -- 172.21.15.66:0/3335986202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c8015ca80 msgr2=0x7f2c8009d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c75400640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c8015ca80 0x7f2c8009d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c75400640 1 -- 172.21.15.66:0/3335986202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c700094d0 con 0x7f2c800cf2f0 2024-03-30T00:40:22.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.645+0000 7f2c75400640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c800cf2f0 0x7f2c8009a310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800c500 tx=0x7f2c6800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:22.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.649+0000 7f2c76a00640 1 -- 172.21.15.66:0/3335986202 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68005220 con 0x7f2c800cf2f0 2024-03-30T00:40:22.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.649+0000 7f2c76a00640 1 -- 172.21.15.66:0/3335986202 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c68005380 con 0x7f2c800cf2f0 2024-03-30T00:40:22.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.649+0000 7f2c84a00640 1 -- 172.21.15.66:0/3335986202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8006e9f0 con 0x7f2c800cf2f0 2024-03-30T00:40:22.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.649+0000 7f2c76a00640 1 -- 172.21.15.66:0/3335986202 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68007270 con 0x7f2c800cf2f0 2024-03-30T00:40:22.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.649+0000 7f2c84a00640 1 -- 172.21.15.66:0/3335986202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8006ecd0 con 0x7f2c800cf2f0 2024-03-30T00:40:22.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.649+0000 7f2c84a00640 1 -- 172.21.15.66:0/3335986202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c80066c30 con 0x7f2c800cf2f0 2024-03-30T00:40:22.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.649+0000 7f2c76a00640 1 -- 172.21.15.66:0/3335986202 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2c680073d0 con 0x7f2c800cf2f0 2024-03-30T00:40:22.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.649+0000 7f2c76a00640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c5c066690 0x7f2c5c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.649+0000 7f2c76a00640 1 -- 172.21.15.66:0/3335986202 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2c68083c60 con 0x7f2c800cf2f0 2024-03-30T00:40:22.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.649+0000 7f2c7de00640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c5c066690 0x7f2c5c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.653+0000 7f2c7de00640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c5c066690 0x7f2c5c068b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c700b2000 tx=0x7f2c70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:22.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.653+0000 7f2c76a00640 1 -- 172.21.15.66:0/3335986202 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2c68050da0 con 0x7f2c800cf2f0 2024-03-30T00:40:22.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.817+0000 7f2c84a00640 1 -- 172.21.15.66:0/3335986202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2c800b3b60 con 0x7f2c800cf2f0 2024-03-30T00:40:22.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.817+0000 7f2c76a00640 1 -- 172.21.15.66:0/3335986202 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2c680543c0 con 0x7f2c800cf2f0 2024-03-30T00:40:22.828 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:22.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 -- 172.21.15.66:0/3335986202 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c5c066690 msgr2=0x7f2c5c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c5c066690 0x7f2c5c068b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c700b2000 tx=0x7f2c70008040 comp rx=0 tx=0).stop 2024-03-30T00:40:22.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 -- 172.21.15.66:0/3335986202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c800cf2f0 msgr2=0x7f2c8009a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c800cf2f0 0x7f2c8009a310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800c500 tx=0x7f2c6800c9c0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 -- 172.21.15.66:0/3335986202 shutdown_connections 2024-03-30T00:40:22.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c8015ca80 0x7f2c8009d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c5c066690 0x7f2c5c068b50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c800cf2f0 0x7f2c8009a310 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 --2- 172.21.15.66:0/3335986202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c800cb570 0x7f2c80157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 -- 172.21.15.66:0/3335986202 >> 172.21.15.66:0/3335986202 conn(0x7f2c800b2090 msgr2=0x7f2c800cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:22.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 -- 172.21.15.66:0/3335986202 shutdown_connections 2024-03-30T00:40:22.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.821+0000 7f2c84a00640 1 -- 172.21.15.66:0/3335986202 wait complete. 2024-03-30T00:40:22.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:40:22.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e27e00640 1 Processor -- start 2024-03-30T00:40:22.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e27e00640 1 -- start start 2024-03-30T00:40:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e27e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e200cda90 0x7f2e200ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e27e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2e2015a370 0x7f2e200cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e27e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2015a990 con 0x7f2e200cde70 2024-03-30T00:40:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e27e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e200b1b80 con 0x7f2e200cda90 2024-03-30T00:40:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e27e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e200cd3e0 con 0x7f2e2015a370 2024-03-30T00:40:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e25400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e200cda90 0x7f2e200ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e25400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e200cda90 0x7f2e200ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34180/0 (socket says 172.21.15.66:34180) 2024-03-30T00:40:22.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e25400640 1 -- 172.21.15.66:0/1194394826 learned_addr learned my addr 172.21.15.66:0/1194394826 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:22.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e25e00640 1 --2- 172.21.15.66:0/1194394826 >> v2:172.21.15.156:3300/0 conn(0x7f2e2015a370 0x7f2e200cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:22.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e25400640 1 -- 172.21.15.66:0/1194394826 >> v2:172.21.15.156:3300/0 conn(0x7f2e2015a370 msgr2=0x7f2e200cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:22.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e25400640 1 --2- 172.21.15.66:0/1194394826 >> v2:172.21.15.156:3300/0 conn(0x7f2e2015a370 0x7f2e200cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e25400640 1 -- 172.21.15.66:0/1194394826 >> v1:172.21.15.156:6789/0 conn(0x7f2e200cde70 legacy=0x7f2e20159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:22.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e25400640 1 -- 172.21.15.66:0/1194394826 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e200b11e0 con 0x7f2e200cda90 2024-03-30T00:40:22.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.981+0000 7f2e25e00640 1 --2- 172.21.15.66:0/1194394826 >> v2:172.21.15.156:3300/0 conn(0x7f2e2015a370 0x7f2e200cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:22.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e25400640 1 --2- 172.21.15.66:0/1194394826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e200cda90 0x7f2e200ce520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e200b3140 tx=0x7f2e1401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab57e9df841c04f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:22.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e1fe00640 1 -- 172.21.15.66:0/1194394826 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e140ad660 con 0x7f2e200cda90 2024-03-30T00:40:22.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e1fe00640 1 -- 172.21.15.66:0/1194394826 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e140ad7c0 con 0x7f2e200cda90 2024-03-30T00:40:22.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e1fe00640 1 -- 172.21.15.66:0/1194394826 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e140adb50 con 0x7f2e200cda90 2024-03-30T00:40:22.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e27e00640 1 -- 172.21.15.66:0/1194394826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e200cda90 msgr2=0x7f2e200ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:22.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e27e00640 1 --2- 172.21.15.66:0/1194394826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e200cda90 0x7f2e200ce520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e200b3140 tx=0x7f2e1401c450 comp rx=0 tx=0).stop 2024-03-30T00:40:22.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e27e00640 1 -- 172.21.15.66:0/1194394826 shutdown_connections 2024-03-30T00:40:22.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e27e00640 1 --2- 172.21.15.66:0/1194394826 >> v2:172.21.15.156:3300/0 conn(0x7f2e2015a370 0x7f2e200cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e27e00640 1 --2- 172.21.15.66:0/1194394826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e200cda90 0x7f2e200ce520 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:22.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e27e00640 1 -- 172.21.15.66:0/1194394826 >> 172.21.15.66:0/1194394826 conn(0x7f2e20069f60 msgr2=0x7f2e2006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:22.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e27e00640 1 -- 172.21.15.66:0/1194394826 shutdown_connections 2024-03-30T00:40:22.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.985+0000 7f2e27e00640 1 -- 172.21.15.66:0/1194394826 wait complete. 2024-03-30T00:40:22.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e27e00640 1 Processor -- start 2024-03-30T00:40:22.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e27e00640 1 -- start start 2024-03-30T00:40:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e27e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e200cda90 0x7f2e20155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e27e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e200cde70 0x7f2e20158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e27e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e2015a370 0x7f2e201592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e27e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e20158530 con 0x7f2e200cde70 2024-03-30T00:40:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e27e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e201f8fe0 con 0x7f2e200cda90 2024-03-30T00:40:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e27e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e201f8360 con 0x7f2e2015a370 2024-03-30T00:40:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e24a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e200cde70 0x7f2e20158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e24a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e200cde70 0x7f2e20158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34188/0 (socket says 172.21.15.66:34188) 2024-03-30T00:40:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e24a00640 1 -- 172.21.15.66:0/1753635167 learned_addr learned my addr 172.21.15.66:0/1753635167 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:23.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e24a00640 1 -- 172.21.15.66:0/1753635167 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e2015a370 msgr2=0x7f2e201592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e24a00640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e2015a370 0x7f2e201592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e24a00640 1 -- 172.21.15.66:0/1753635167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e200cda90 msgr2=0x7f2e20155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e24a00640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e200cda90 0x7f2e20155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e24a00640 1 -- 172.21.15.66:0/1753635167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140094d0 con 0x7f2e200cde70 2024-03-30T00:40:23.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e24a00640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e200cde70 0x7f2e20158d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00c440 tx=0x7f2e0c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:23.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e1d600640 1 -- 172.21.15.66:0/1753635167 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c012830 con 0x7f2e200cde70 2024-03-30T00:40:23.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e1d600640 1 -- 172.21.15.66:0/1753635167 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c012990 con 0x7f2e200cde70 2024-03-30T00:40:23.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e1d600640 1 -- 172.21.15.66:0/1753635167 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c012d40 con 0x7f2e200cde70 2024-03-30T00:40:23.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.989+0000 7f2e27e00640 1 -- 172.21.15.66:0/1753635167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e201f8640 con 0x7f2e200cde70 2024-03-30T00:40:23.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.993+0000 7f2e27e00640 1 -- 172.21.15.66:0/1753635167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e201f8c00 con 0x7f2e200cde70 2024-03-30T00:40:23.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.993+0000 7f2e27e00640 1 -- 172.21.15.66:0/1753635167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e20066c30 con 0x7f2e200cde70 2024-03-30T00:40:23.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.993+0000 7f2e1d600640 1 -- 172.21.15.66:0/1753635167 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2e0c01b400 con 0x7f2e200cde70 2024-03-30T00:40:23.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.993+0000 7f2e1d600640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2e000666e0 0x7f2e00068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.993+0000 7f2e1d600640 1 -- 172.21.15.66:0/1753635167 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2e0c084720 con 0x7f2e200cde70 2024-03-30T00:40:23.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.993+0000 7f2e25400640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2e000666e0 0x7f2e00068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.993+0000 7f2e25400640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2e000666e0 0x7f2e00068ba0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2e140af000 tx=0x7f2e14004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:23.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:22.997+0000 7f2e1d600640 1 -- 172.21.15.66:0/1753635167 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2e0c051780 con 0x7f2e200cde70 2024-03-30T00:40:23.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.193+0000 7f2e27e00640 1 -- 172.21.15.66:0/1753635167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e200b3170 con 0x7f2e200cde70 2024-03-30T00:40:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.193+0000 7f2e1d600640 1 -- 172.21.15.66:0/1753635167 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f2e0c054da0 con 0x7f2e200cde70 2024-03-30T00:40:23.204 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:23.204 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4122431016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":4122431016},{"type":"v1","addr":"172.21.15.156:6835","nonce":4122431016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3882342907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3882342907},{"type":"v1","addr":"172.21.15.66:6835","nonce":3882342907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:39:51.142575+0000","modified":"2024-03-30T00:40:14.500142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/703895528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":703895528},{"type":"v1","addr":"172.21.15.66:6837","nonce":703895528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:40:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2024-03-30T00:40:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 -- 172.21.15.66:0/1753635167 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2e000666e0 msgr2=0x7f2e00068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2e000666e0 0x7f2e00068ba0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2e140af000 tx=0x7f2e14004740 comp rx=0 tx=0).stop 2024-03-30T00:40:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 -- 172.21.15.66:0/1753635167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e200cde70 msgr2=0x7f2e20158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e200cde70 0x7f2e20158d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00c440 tx=0x7f2e0c00c900 comp rx=0 tx=0).stop 2024-03-30T00:40:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 -- 172.21.15.66:0/1753635167 shutdown_connections 2024-03-30T00:40:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e2015a370 0x7f2e201592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2e000666e0 0x7f2e00068ba0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e200cde70 0x7f2e20158d80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 --2- 172.21.15.66:0/1753635167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e200cda90 0x7f2e20155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 -- 172.21.15.66:0/1753635167 >> 172.21.15.66:0/1753635167 conn(0x7f2e20069f60 msgr2=0x7f2e2006d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:23.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 -- 172.21.15.66:0/1753635167 shutdown_connections 2024-03-30T00:40:23.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.197+0000 7f2e27e00640 1 -- 172.21.15.66:0/1753635167 wait complete. 2024-03-30T00:40:23.226 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:23.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f554ba00640 1 Processor -- start 2024-03-30T00:40:23.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f554ba00640 1 -- start start 2024-03-30T00:40:23.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f554ba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55440cee50 0x7f55440cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f554ba00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55440af420 0x7f55440d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f554ba00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55440cbb90 con 0x7f554415de30 2024-03-30T00:40:23.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f554ba00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55440ce1b0 con 0x7f55440af420 2024-03-30T00:40:23.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f554ba00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55440afa10 con 0x7f55440cee50 2024-03-30T00:40:23.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f5543e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55440cee50 0x7f55440cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f5549a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55440af420 0x7f55440d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f5549a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55440af420 0x7f55440d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:34874/0 (socket says 172.21.15.66:34874) 2024-03-30T00:40:23.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f5549000640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f554415de30 0x7f55440ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:59186/0 (socket says 172.21.15.66:59186) 2024-03-30T00:40:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f5549a00640 1 -- 172.21.15.66:0/3260204400 learned_addr learned my addr 172.21.15.66:0/3260204400 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f5543e00640 1 -- 172.21.15.66:0/3260204400 >> v2:172.21.15.156:3300/0 conn(0x7f55440af420 msgr2=0x7f55440d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f5543400640 1 -- 172.21.15.66:0/3260204400 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373039530 0 0) 0x7f55440cbb90 con 0x7f554415de30 2024-03-30T00:40:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f5543e00640 1 --2- 172.21.15.66:0/3260204400 >> v2:172.21.15.156:3300/0 conn(0x7f55440af420 0x7f55440d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f5543e00640 1 -- 172.21.15.66:0/3260204400 >> v1:172.21.15.156:6789/0 conn(0x7f554415de30 legacy=0x7f55440ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f5543e00640 1 -- 172.21.15.66:0/3260204400 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55440cf430 con 0x7f55440cee50 2024-03-30T00:40:23.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.357+0000 7f5549a00640 1 --2- 172.21.15.66:0/3260204400 >> v2:172.21.15.156:3300/0 conn(0x7f55440af420 0x7f55440d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:23.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f5543e00640 1 --2- 172.21.15.66:0/3260204400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55440cee50 0x7f55440cc040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f55300097c0 tx=0x7f553001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=487a74b61d202f8d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:23.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f5543400640 1 -- 172.21.15.66:0/3260204400 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55300ab600 con 0x7f55440cee50 2024-03-30T00:40:23.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f5543400640 1 -- 172.21.15.66:0/3260204400 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55300a7050 con 0x7f55440cee50 2024-03-30T00:40:23.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f5543400640 1 -- 172.21.15.66:0/3260204400 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55300ab980 con 0x7f55440cee50 2024-03-30T00:40:23.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f554ba00640 1 -- 172.21.15.66:0/3260204400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55440cee50 msgr2=0x7f55440cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f554ba00640 1 --2- 172.21.15.66:0/3260204400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55440cee50 0x7f55440cc040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f55300097c0 tx=0x7f553001c410 comp rx=0 tx=0).stop 2024-03-30T00:40:23.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f554ba00640 1 -- 172.21.15.66:0/3260204400 shutdown_connections 2024-03-30T00:40:23.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f554ba00640 1 --2- 172.21.15.66:0/3260204400 >> v2:172.21.15.156:3300/0 conn(0x7f55440af420 0x7f55440d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f554ba00640 1 --2- 172.21.15.66:0/3260204400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55440cee50 0x7f55440cc040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f554ba00640 1 -- 172.21.15.66:0/3260204400 >> 172.21.15.66:0/3260204400 conn(0x7f55440d0540 msgr2=0x7f55440d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:23.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f554ba00640 1 -- 172.21.15.66:0/3260204400 shutdown_connections 2024-03-30T00:40:23.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.361+0000 7f554ba00640 1 -- 172.21.15.66:0/3260204400 wait complete. 2024-03-30T00:40:23.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f554ba00640 1 Processor -- start 2024-03-30T00:40:23.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f554ba00640 1 -- start start 2024-03-30T00:40:23.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f554ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440af420 0x7f55441f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f554ba00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55440cee50 0x7f554406aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f554ba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f554415de30 0x7f5544156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f554ba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5544162830 con 0x7f55440af420 2024-03-30T00:40:23.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f554ba00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554406b810 con 0x7f55440cee50 2024-03-30T00:40:23.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f554ba00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554406baf0 con 0x7f554415de30 2024-03-30T00:40:23.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5549000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440af420 0x7f55441f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5549000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440af420 0x7f55441f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34194/0 (socket says 172.21.15.66:34194) 2024-03-30T00:40:23.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5549000640 1 -- 172.21.15.66:0/3376819999 learned_addr learned my addr 172.21.15.66:0/3376819999 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:23.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5543e00640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55440cee50 0x7f554406aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5549a00640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f554415de30 0x7f5544156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5549000640 1 -- 172.21.15.66:0/3376819999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f554415de30 msgr2=0x7f5544156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5549000640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f554415de30 0x7f5544156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5549000640 1 -- 172.21.15.66:0/3376819999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55440cee50 msgr2=0x7f554406aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5549000640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55440cee50 0x7f554406aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5549000640 1 -- 172.21.15.66:0/3376819999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c009b30 con 0x7f55440af420 2024-03-30T00:40:23.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5549000640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440af420 0x7f55441f5990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f553800c480 tx=0x7f553800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:23.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5540c00640 1 -- 172.21.15.66:0/3376819999 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55380074e0 con 0x7f55440af420 2024-03-30T00:40:23.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5540c00640 1 -- 172.21.15.66:0/3376819999 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5538007660 con 0x7f55440af420 2024-03-30T00:40:23.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f554ba00640 1 -- 172.21.15.66:0/3376819999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5530009420 con 0x7f55440af420 2024-03-30T00:40:23.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f5540c00640 1 -- 172.21.15.66:0/3376819999 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538004a30 con 0x7f55440af420 2024-03-30T00:40:23.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.365+0000 7f554ba00640 1 -- 172.21.15.66:0/3376819999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554406c0b0 con 0x7f55440af420 2024-03-30T00:40:23.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.369+0000 7f554ba00640 1 -- 172.21.15.66:0/3376819999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55440cdf60 con 0x7f55440af420 2024-03-30T00:40:23.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.369+0000 7f5540c00640 1 -- 172.21.15.66:0/3376819999 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5538004b90 con 0x7f55440af420 2024-03-30T00:40:23.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.369+0000 7f5540c00640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5524066690 0x7f5524068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.369+0000 7f5540c00640 1 -- 172.21.15.66:0/3376819999 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5538083860 con 0x7f55440af420 2024-03-30T00:40:23.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.369+0000 7f5543e00640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5524066690 0x7f5524068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.369+0000 7f5543e00640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5524066690 0x7f5524068b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f55300ad000 tx=0x7f55300a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:23.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.373+0000 7f5540c00640 1 -- 172.21.15.66:0/3376819999 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5538050a20 con 0x7f55440af420 2024-03-30T00:40:23.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 -- 172.21.15.66:0/3376819999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5544066c30 con 0x7f55440af420 2024-03-30T00:40:23.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f5540c00640 1 -- 172.21.15.66:0/3376819999 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f5538018090 con 0x7f55440af420 2024-03-30T00:40:23.546 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:23.546 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 -- 172.21.15.66:0/3376819999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5524066690 msgr2=0x7f5524068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5524066690 0x7f5524068b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f55300ad000 tx=0x7f55300a6000 comp rx=0 tx=0).stop 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 -- 172.21.15.66:0/3376819999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440af420 msgr2=0x7f55441f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440af420 0x7f55441f5990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f553800c480 tx=0x7f553800c940 comp rx=0 tx=0).stop 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 -- 172.21.15.66:0/3376819999 shutdown_connections 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f554415de30 0x7f5544156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55440cee50 0x7f554406aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5524066690 0x7f5524068b50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 --2- 172.21.15.66:0/3376819999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440af420 0x7f55441f5990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 -- 172.21.15.66:0/3376819999 >> 172.21.15.66:0/3376819999 conn(0x7f55440d0540 msgr2=0x7f55440ad330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.537+0000 7f554ba00640 1 -- 172.21.15.66:0/3376819999 shutdown_connections 2024-03-30T00:40:23.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.541+0000 7f554ba00640 1 -- 172.21.15.66:0/3376819999 wait complete. 2024-03-30T00:40:23.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:23.570 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:23.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.701+0000 7f5e89e00640 1 Processor -- start 2024-03-30T00:40:23.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.701+0000 7f5e89e00640 1 -- start start 2024-03-30T00:40:23.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e89e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cec30 0x7f5e84159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e89e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e840cd3e0 con 0x7f5e840cda90 2024-03-30T00:40:23.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e89e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8415a990 con 0x7f5e840cde70 2024-03-30T00:40:23.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e89e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840b1b80 con 0x7f5e840cec30 2024-03-30T00:40:23.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e82a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5e840cda90 0x7f5e840ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:43326/0 (socket says 172.21.15.66:43326) 2024-03-30T00:40:23.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e82a00640 1 -- 172.21.15.66:0/1439283230 learned_addr learned my addr 172.21.15.66:0/1439283230 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:23.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e7a000640 1 --2- 172.21.15.66:0/1439283230 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cec30 0x7f5e84159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e82000640 1 -- 172.21.15.66:0/1439283230 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445520105 0 0) 0x7f5e840cd3e0 con 0x7f5e840cda90 2024-03-30T00:40:23.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e82000640 1 -- 172.21.15.66:0/1439283230 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e68003530 con 0x7f5e840cda90 2024-03-30T00:40:23.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e82000640 1 -- 172.21.15.66:0/1439283230 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255022125 0 0) 0x7f5e8415a990 con 0x7f5e840cde70 2024-03-30T00:40:23.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e82000640 1 -- 172.21.15.66:0/1439283230 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e840cd3e0 con 0x7f5e840cde70 2024-03-30T00:40:23.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e7a000640 1 -- 172.21.15.66:0/1439283230 >> v1:172.21.15.156:6789/0 conn(0x7f5e840cde70 legacy=0x7f5e8415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e7a000640 1 -- 172.21.15.66:0/1439283230 >> v1:172.21.15.66:6789/0 conn(0x7f5e840cda90 legacy=0x7f5e840ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.705+0000 7f5e7a000640 1 -- 172.21.15.66:0/1439283230 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e840b11e0 con 0x7f5e840cec30 2024-03-30T00:40:23.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e7a000640 1 --2- 172.21.15.66:0/1439283230 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cec30 0x7f5e84159c60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0097f0 tx=0x7f5e6c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fa123c199e30b5d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:23.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e82000640 1 -- 172.21.15.66:0/1439283230 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0ab650 con 0x7f5e840cec30 2024-03-30T00:40:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e82000640 1 -- 172.21.15.66:0/1439283230 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c0ab7b0 con 0x7f5e840cec30 2024-03-30T00:40:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e82000640 1 -- 172.21.15.66:0/1439283230 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0abb40 con 0x7f5e840cec30 2024-03-30T00:40:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e89e00640 1 -- 172.21.15.66:0/1439283230 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cec30 msgr2=0x7f5e84159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e89e00640 1 --2- 172.21.15.66:0/1439283230 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cec30 0x7f5e84159c60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0097f0 tx=0x7f5e6c01c450 comp rx=0 tx=0).stop 2024-03-30T00:40:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e89e00640 1 -- 172.21.15.66:0/1439283230 shutdown_connections 2024-03-30T00:40:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e89e00640 1 --2- 172.21.15.66:0/1439283230 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cec30 0x7f5e84159c60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e89e00640 1 -- 172.21.15.66:0/1439283230 >> 172.21.15.66:0/1439283230 conn(0x7f5e84069f60 msgr2=0x7f5e8406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e89e00640 1 -- 172.21.15.66:0/1439283230 shutdown_connections 2024-03-30T00:40:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e89e00640 1 -- 172.21.15.66:0/1439283230 wait complete. 2024-03-30T00:40:23.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e89e00640 1 Processor -- start 2024-03-30T00:40:23.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.709+0000 7f5e89e00640 1 -- start start 2024-03-30T00:40:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e89e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e840cda90 0x7f5e84159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e89e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cde70 0x7f5e840b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e89e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e840cec30 0x7f5e840c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e89e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e841f4d10 con 0x7f5e840cec30 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e89e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8415a450 con 0x7f5e840cda90 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e89e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e84159940 con 0x7f5e840cde70 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e83400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e840cec30 0x7f5e840c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e83400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e840cec30 0x7f5e840c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34196/0 (socket says 172.21.15.66:34196) 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e83400640 1 -- 172.21.15.66:0/858946826 learned_addr learned my addr 172.21.15.66:0/858946826 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e7a000640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cde70 0x7f5e840b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e82a00640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e840cda90 0x7f5e84159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e83400640 1 -- 172.21.15.66:0/858946826 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cde70 msgr2=0x7f5e840b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e83400640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cde70 0x7f5e840b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e83400640 1 -- 172.21.15.66:0/858946826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e840cda90 msgr2=0x7f5e84159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e83400640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e840cda90 0x7f5e84159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e83400640 1 -- 172.21.15.66:0/858946826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c009420 con 0x7f5e840cec30 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e82a00640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e840cda90 0x7f5e84159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e7a000640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cde70 0x7f5e840b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e83400640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e840cec30 0x7f5e840c7870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c7f0 tx=0x7f5e7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:23.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e7be00640 1 -- 172.21.15.66:0/858946826 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c003b80 con 0x7f5e840cec30 2024-03-30T00:40:23.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e7be00640 1 -- 172.21.15.66:0/858946826 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7c003ce0 con 0x7f5e840cec30 2024-03-30T00:40:23.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e7be00640 1 -- 172.21.15.66:0/858946826 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c00aa50 con 0x7f5e840cec30 2024-03-30T00:40:23.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e89e00640 1 -- 172.21.15.66:0/858946826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e840b0740 con 0x7f5e840cec30 2024-03-30T00:40:23.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e89e00640 1 -- 172.21.15.66:0/858946826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e84159e90 con 0x7f5e840cec30 2024-03-30T00:40:23.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e89e00640 1 -- 172.21.15.66:0/858946826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e84066c30 con 0x7f5e840cec30 2024-03-30T00:40:23.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.713+0000 7f5e7be00640 1 -- 172.21.15.66:0/858946826 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5e7c00abb0 con 0x7f5e840cec30 2024-03-30T00:40:23.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.717+0000 7f5e7be00640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5e60066690 0x7f5e60068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:23.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.717+0000 7f5e7be00640 1 -- 172.21.15.66:0/858946826 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5e7c084390 con 0x7f5e840cec30 2024-03-30T00:40:23.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.717+0000 7f5e82a00640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5e60066690 0x7f5e60068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:23.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.717+0000 7f5e82a00640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5e60066690 0x7f5e60068b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5e740011f0 tx=0x7f5e74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:23.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.721+0000 7f5e7be00640 1 -- 172.21.15.66:0/858946826 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5e7c051410 con 0x7f5e840cec30 2024-03-30T00:40:23.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 -- 172.21.15.66:0/858946826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e840b3170 con 0x7f5e840cec30 2024-03-30T00:40:23.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e7be00640 1 -- 172.21.15.66:0/858946826 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e7c00ae60 con 0x7f5e840cec30 2024-03-30T00:40:23.890 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:23.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 -- 172.21.15.66:0/858946826 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5e60066690 msgr2=0x7f5e60068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5e60066690 0x7f5e60068b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5e740011f0 tx=0x7f5e74009040 comp rx=0 tx=0).stop 2024-03-30T00:40:23.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 -- 172.21.15.66:0/858946826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e840cec30 msgr2=0x7f5e840c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:23.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e840cec30 0x7f5e840c7870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c7f0 tx=0x7f5e7c00ccb0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 -- 172.21.15.66:0/858946826 shutdown_connections 2024-03-30T00:40:23.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5e60066690 0x7f5e60068b50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e840cec30 0x7f5e840c7870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e840cde70 0x7f5e840b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 --2- 172.21.15.66:0/858946826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e840cda90 0x7f5e84159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:23.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 -- 172.21.15.66:0/858946826 >> 172.21.15.66:0/858946826 conn(0x7f5e84069f60 msgr2=0x7f5e8406c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:23.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 -- 172.21.15.66:0/858946826 shutdown_connections 2024-03-30T00:40:23.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:23.881+0000 7f5e89e00640 1 -- 172.21.15.66:0/858946826 wait complete. 2024-03-30T00:40:23.911 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:24.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb809e00640 1 Processor -- start 2024-03-30T00:40:24.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb809e00640 1 -- start start 2024-03-30T00:40:24.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb809e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8040ce780 0x7fb8040cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb809e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb8040ceb60 0x7fb80415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb809e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb80415d360 0x7fb8040cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb809e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8040cbec0 con 0x7fb80415d360 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb809e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8040cce50 con 0x7fb8040ceb60 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb809e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80415d970 con 0x7fb8040ce780 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb803400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb80415d360 0x7fb8040cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb803400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb80415d360 0x7fb8040cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34210/0 (socket says 172.21.15.66:34210) 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb803400640 1 -- 172.21.15.66:0/2091471803 learned_addr learned my addr 172.21.15.66:0/2091471803 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb802a00640 1 --2- 172.21.15.66:0/2091471803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8040ce780 0x7fb8040cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb7fa000640 1 --2- 172.21.15.66:0/2091471803 >> v2:172.21.15.156:3300/0 conn(0x7fb8040ceb60 0x7fb80415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb803400640 1 -- 172.21.15.66:0/2091471803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8040ce780 msgr2=0x7fb8040cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb803400640 1 --2- 172.21.15.66:0/2091471803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8040ce780 0x7fb8040cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb803400640 1 -- 172.21.15.66:0/2091471803 >> v2:172.21.15.156:3300/0 conn(0x7fb8040ceb60 msgr2=0x7fb80415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:24.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb803400640 1 --2- 172.21.15.66:0/2091471803 >> v2:172.21.15.156:3300/0 conn(0x7fb8040ceb60 0x7fb80415ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.045+0000 7fb803400640 1 -- 172.21.15.66:0/2091471803 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80415dc50 con 0x7fb80415d360 2024-03-30T00:40:24.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb803400640 1 --2- 172.21.15.66:0/2091471803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb80415d360 0x7fb8040cd5e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb8040cbe00 tx=0x7fb7fc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3397bd7e8fd53a49 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:24.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb802000640 1 -- 172.21.15.66:0/2091471803 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0ab620 con 0x7fb80415d360 2024-03-30T00:40:24.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb802000640 1 -- 172.21.15.66:0/2091471803 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc0ab780 con 0x7fb80415d360 2024-03-30T00:40:24.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb802000640 1 -- 172.21.15.66:0/2091471803 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0abb10 con 0x7fb80415d360 2024-03-30T00:40:24.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb809e00640 1 -- 172.21.15.66:0/2091471803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb80415d360 msgr2=0x7fb8040cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb809e00640 1 --2- 172.21.15.66:0/2091471803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb80415d360 0x7fb8040cd5e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb8040cbe00 tx=0x7fb7fc01c460 comp rx=0 tx=0).stop 2024-03-30T00:40:24.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb809e00640 1 -- 172.21.15.66:0/2091471803 shutdown_connections 2024-03-30T00:40:24.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb809e00640 1 --2- 172.21.15.66:0/2091471803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb80415d360 0x7fb8040cd5e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb809e00640 1 --2- 172.21.15.66:0/2091471803 >> v2:172.21.15.156:3300/0 conn(0x7fb8040ceb60 0x7fb80415ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb809e00640 1 --2- 172.21.15.66:0/2091471803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8040ce780 0x7fb8040cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb809e00640 1 -- 172.21.15.66:0/2091471803 >> 172.21.15.66:0/2091471803 conn(0x7fb804069f60 msgr2=0x7fb80406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:24.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb809e00640 1 -- 172.21.15.66:0/2091471803 shutdown_connections 2024-03-30T00:40:24.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb809e00640 1 -- 172.21.15.66:0/2091471803 wait complete. 2024-03-30T00:40:24.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb809e00640 1 Processor -- start 2024-03-30T00:40:24.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.049+0000 7fb809e00640 1 -- start start 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb809e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8040ce780 0x7fb8041552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb809e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8040ceb60 0x7fb804099cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb809e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb80415d360 0x7fb80409a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb809e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80409a770 con 0x7fb8040ceb60 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb809e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb804155a90 con 0x7fb8040ce780 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb809e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb804156330 con 0x7fb80415d360 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fa000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8040ceb60 0x7fb804099cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fa000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8040ceb60 0x7fb804099cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34224/0 (socket says 172.21.15.66:34224) 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fa000640 1 -- 172.21.15.66:0/2124221145 learned_addr learned my addr 172.21.15.66:0/2124221145 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb803400640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb80415d360 0x7fb80409a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb802a00640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8040ce780 0x7fb8041552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fa000640 1 -- 172.21.15.66:0/2124221145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb80415d360 msgr2=0x7fb80409a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fa000640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb80415d360 0x7fb80409a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fa000640 1 -- 172.21.15.66:0/2124221145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8040ce780 msgr2=0x7fb8041552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fa000640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8040ce780 0x7fb8041552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fa000640 1 -- 172.21.15.66:0/2124221145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc0094d0 con 0x7fb8040ceb60 2024-03-30T00:40:24.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb802a00640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8040ce780 0x7fb8041552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:24.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fa000640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8040ceb60 0x7fb804099cf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009db0 tx=0x7fb7ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:24.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fbe00640 1 -- 172.21.15.66:0/2124221145 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec014480 con 0x7fb8040ceb60 2024-03-30T00:40:24.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fbe00640 1 -- 172.21.15.66:0/2124221145 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec00f040 con 0x7fb8040ceb60 2024-03-30T00:40:24.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fbe00640 1 -- 172.21.15.66:0/2124221145 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec014800 con 0x7fb8040ceb60 2024-03-30T00:40:24.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb809e00640 1 -- 172.21.15.66:0/2124221145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb804099a10 con 0x7fb8040ceb60 2024-03-30T00:40:24.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb809e00640 1 -- 172.21.15.66:0/2124221145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb804155fc0 con 0x7fb8040ceb60 2024-03-30T00:40:24.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb809e00640 1 -- 172.21.15.66:0/2124221145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8041564c0 con 0x7fb8040ceb60 2024-03-30T00:40:24.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.053+0000 7fb7fbe00640 1 -- 172.21.15.66:0/2124221145 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb7ec014960 con 0x7fb8040ceb60 2024-03-30T00:40:24.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.057+0000 7fb7fbe00640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7e0066690 0x7fb7e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.057+0000 7fb7fbe00640 1 -- 172.21.15.66:0/2124221145 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb7ec084f80 con 0x7fb8040ceb60 2024-03-30T00:40:24.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.057+0000 7fb802a00640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7e0066690 0x7fb7e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.057+0000 7fb802a00640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7e0066690 0x7fb7e0068b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb8040c7740 tx=0x7fb7f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:24.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.061+0000 7fb7fbe00640 1 -- 172.21.15.66:0/2124221145 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb7ec051ff0 con 0x7fb8040ceb60 2024-03-30T00:40:24.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 -- 172.21.15.66:0/2124221145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8040ad160 con 0x7fb8040ceb60 2024-03-30T00:40:24.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb7fbe00640 1 -- 172.21.15.66:0/2124221145 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7ec055610 con 0x7fb8040ceb60 2024-03-30T00:40:24.230 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 -- 172.21.15.66:0/2124221145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7e0066690 msgr2=0x7fb7e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7e0066690 0x7fb7e0068b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb8040c7740 tx=0x7fb7f4008040 comp rx=0 tx=0).stop 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 -- 172.21.15.66:0/2124221145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8040ceb60 msgr2=0x7fb804099cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8040ceb60 0x7fb804099cf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009db0 tx=0x7fb7ec00c490 comp rx=0 tx=0).stop 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 -- 172.21.15.66:0/2124221145 shutdown_connections 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb80415d360 0x7fb80409a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7e0066690 0x7fb7e0068b50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8040ceb60 0x7fb804099cf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 --2- 172.21.15.66:0/2124221145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8040ce780 0x7fb8041552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 -- 172.21.15.66:0/2124221145 >> 172.21.15.66:0/2124221145 conn(0x7fb804069f60 msgr2=0x7fb8040ab2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 -- 172.21.15.66:0/2124221145 shutdown_connections 2024-03-30T00:40:24.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.221+0000 7fb809e00640 1 -- 172.21.15.66:0/2124221145 wait complete. 2024-03-30T00:40:24.251 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:24.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d0dc00640 1 Processor -- start 2024-03-30T00:40:24.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d0dc00640 1 -- start start 2024-03-30T00:40:24.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d0dc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d08099130 0x7f6d080ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d0dc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d080b0940 con 0x7f6d0806bb90 2024-03-30T00:40:24.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d0dc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d0806b140 con 0x7f6d080af790 2024-03-30T00:40:24.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d0dc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0806b700 con 0x7f6d08099130 2024-03-30T00:40:24.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6cffe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6d0806bb90 0x7f6d08098a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:43340/0 (socket says 172.21.15.66:43340) 2024-03-30T00:40:24.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6cffe00640 1 -- 172.21.15.66:0/1299711550 learned_addr learned my addr 172.21.15.66:0/1299711550 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:24.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d07400640 1 --2- 172.21.15.66:0/1299711550 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d08099130 0x7f6d080ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d06000640 1 -- 172.21.15.66:0/1299711550 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503677819 0 0) 0x7f6d080b0940 con 0x7f6d0806bb90 2024-03-30T00:40:24.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d06000640 1 -- 172.21.15.66:0/1299711550 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cec003530 con 0x7f6d0806bb90 2024-03-30T00:40:24.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d06000640 1 -- 172.21.15.66:0/1299711550 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582952690 0 0) 0x7f6d0806b140 con 0x7f6d080af790 2024-03-30T00:40:24.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d06000640 1 -- 172.21.15.66:0/1299711550 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d080b0940 con 0x7f6d080af790 2024-03-30T00:40:24.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d07400640 1 -- 172.21.15.66:0/1299711550 >> v1:172.21.15.156:6789/0 conn(0x7f6d080af790 legacy=0x7f6d08099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d07400640 1 -- 172.21.15.66:0/1299711550 >> v1:172.21.15.66:6789/0 conn(0x7f6d0806bb90 legacy=0x7f6d08098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.385+0000 7f6d07400640 1 -- 172.21.15.66:0/1299711550 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080b0b90 con 0x7f6d08099130 2024-03-30T00:40:24.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d07400640 1 --2- 172.21.15.66:0/1299711550 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d08099130 0x7f6d080ce900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009dc0 tx=0x7f6d0001c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67f634fd0a4a7b04 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:24.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d06000640 1 -- 172.21.15.66:0/1299711550 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d000a5930 con 0x7f6d08099130 2024-03-30T00:40:24.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d06000640 1 -- 172.21.15.66:0/1299711550 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d000a5a90 con 0x7f6d08099130 2024-03-30T00:40:24.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d06000640 1 -- 172.21.15.66:0/1299711550 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d000a5e00 con 0x7f6d08099130 2024-03-30T00:40:24.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d0dc00640 1 -- 172.21.15.66:0/1299711550 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d08099130 msgr2=0x7f6d080ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d0dc00640 1 --2- 172.21.15.66:0/1299711550 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d08099130 0x7f6d080ce900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009dc0 tx=0x7f6d0001c960 comp rx=0 tx=0).stop 2024-03-30T00:40:24.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d0dc00640 1 -- 172.21.15.66:0/1299711550 shutdown_connections 2024-03-30T00:40:24.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d0dc00640 1 --2- 172.21.15.66:0/1299711550 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d08099130 0x7f6d080ce900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d0dc00640 1 -- 172.21.15.66:0/1299711550 >> 172.21.15.66:0/1299711550 conn(0x7f6d080b20c0 msgr2=0x7f6d080b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:24.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d0dc00640 1 -- 172.21.15.66:0/1299711550 shutdown_connections 2024-03-30T00:40:24.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d0dc00640 1 -- 172.21.15.66:0/1299711550 wait complete. 2024-03-30T00:40:24.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.389+0000 7f6d0dc00640 1 Processor -- start 2024-03-30T00:40:24.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d0dc00640 1 -- start start 2024-03-30T00:40:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d0dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d0806bb90 0x7f6d081564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d0dc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d08099130 0x7f6d08150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d0dc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d080af790 0x7f6d080b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d0dc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d08150f80 con 0x7f6d0806bb90 2024-03-30T00:40:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d0dc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080b4cf0 con 0x7f6d08099130 2024-03-30T00:40:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d0dc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d08156af0 con 0x7f6d080af790 2024-03-30T00:40:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d06a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d0806bb90 0x7f6d081564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d06a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d0806bb90 0x7f6d081564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34230/0 (socket says 172.21.15.66:34230) 2024-03-30T00:40:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d06a00640 1 -- 172.21.15.66:0/2500264114 learned_addr learned my addr 172.21.15.66:0/2500264114 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d07400640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d080af790 0x7f6d080b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6cffe00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d08099130 0x7f6d08150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d06a00640 1 -- 172.21.15.66:0/2500264114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d080af790 msgr2=0x7f6d080b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d06a00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d080af790 0x7f6d080b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d06a00640 1 -- 172.21.15.66:0/2500264114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d08099130 msgr2=0x7f6d08150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d06a00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d08099130 0x7f6d08150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d06a00640 1 -- 172.21.15.66:0/2500264114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d00009a70 con 0x7f6d0806bb90 2024-03-30T00:40:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d07400640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d080af790 0x7f6d080b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d06a00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d0806bb90 0x7f6d081564c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c500 tx=0x7f6cf800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6cfea00640 1 -- 172.21.15.66:0/2500264114 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf80052d0 con 0x7f6d0806bb90 2024-03-30T00:40:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6cfea00640 1 -- 172.21.15.66:0/2500264114 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf8005450 con 0x7f6d0806bb90 2024-03-30T00:40:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d0dc00640 1 -- 172.21.15.66:0/2500264114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d08156dd0 con 0x7f6d0806bb90 2024-03-30T00:40:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6cfea00640 1 -- 172.21.15.66:0/2500264114 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8012870 con 0x7f6d0806bb90 2024-03-30T00:40:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d0dc00640 1 -- 172.21.15.66:0/2500264114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d08157390 con 0x7f6d0806bb90 2024-03-30T00:40:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.393+0000 7f6d0dc00640 1 -- 172.21.15.66:0/2500264114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d08066c30 con 0x7f6d0806bb90 2024-03-30T00:40:24.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.397+0000 7f6cfea00640 1 -- 172.21.15.66:0/2500264114 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6cf80129d0 con 0x7f6d0806bb90 2024-03-30T00:40:24.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.397+0000 7f6cfea00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6ce40666c0 0x7f6ce4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.397+0000 7f6cfea00640 1 -- 172.21.15.66:0/2500264114 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6cf8083fa0 con 0x7f6d0806bb90 2024-03-30T00:40:24.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.397+0000 7f6cffe00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6ce40666c0 0x7f6ce4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.397+0000 7f6cffe00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6ce40666c0 0x7f6ce4068b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d080c7050 tx=0x7f6cf0007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:24.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.401+0000 7f6cfea00640 1 -- 172.21.15.66:0/2500264114 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6cf80b5980 con 0x7f6d0806bb90 2024-03-30T00:40:24.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 -- 172.21.15.66:0/2500264114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d080b1380 con 0x7f6d0806bb90 2024-03-30T00:40:24.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6cfea00640 1 -- 172.21.15.66:0/2500264114 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6cf8054ac0 con 0x7f6d0806bb90 2024-03-30T00:40:24.570 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:24.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 -- 172.21.15.66:0/2500264114 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6ce40666c0 msgr2=0x7f6ce4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6ce40666c0 0x7f6ce4068b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d080c7050 tx=0x7f6cf0007d40 comp rx=0 tx=0).stop 2024-03-30T00:40:24.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 -- 172.21.15.66:0/2500264114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d0806bb90 msgr2=0x7f6d081564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d0806bb90 0x7f6d081564c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c500 tx=0x7f6cf800c9c0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 -- 172.21.15.66:0/2500264114 shutdown_connections 2024-03-30T00:40:24.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d080af790 0x7f6d080b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d08099130 0x7f6d08150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6ce40666c0 0x7f6ce4068b80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 --2- 172.21.15.66:0/2500264114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d0806bb90 0x7f6d081564c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 -- 172.21.15.66:0/2500264114 >> 172.21.15.66:0/2500264114 conn(0x7f6d080b20c0 msgr2=0x7f6d0806fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:24.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 -- 172.21.15.66:0/2500264114 shutdown_connections 2024-03-30T00:40:24.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.561+0000 7f6d0dc00640 1 -- 172.21.15.66:0/2500264114 wait complete. 2024-03-30T00:40:24.590 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:24.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc546a00640 1 Processor -- start 2024-03-30T00:40:24.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc546a00640 1 -- start start 2024-03-30T00:40:24.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc546a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc5400cee50 0x7fc5400cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc546a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc5400af420 0x7fc5400d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc546a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5400cbb90 con 0x7fc54015de30 2024-03-30T00:40:24.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc546a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5400ce1b0 con 0x7fc5400cee50 2024-03-30T00:40:24.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc546a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5400afa10 con 0x7fc5400af420 2024-03-30T00:40:24.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc537400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc5400cee50 0x7fc5400cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc537400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc5400cee50 0x7fc5400cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34240/0 (socket says 172.21.15.66:34240) 2024-03-30T00:40:24.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc537400640 1 -- 172.21.15.66:0/323611726 learned_addr learned my addr 172.21.15.66:0/323611726 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:24.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc544a00640 1 --2- 172.21.15.66:0/323611726 >> v2:172.21.15.156:3300/0 conn(0x7fc5400af420 0x7fc5400d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc537400640 1 -- 172.21.15.66:0/323611726 >> v2:172.21.15.156:3300/0 conn(0x7fc5400af420 msgr2=0x7fc5400d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:24.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc537400640 1 --2- 172.21.15.66:0/323611726 >> v2:172.21.15.156:3300/0 conn(0x7fc5400af420 0x7fc5400d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc537400640 1 -- 172.21.15.66:0/323611726 >> v1:172.21.15.156:6789/0 conn(0x7fc54015de30 legacy=0x7fc5400ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc537400640 1 -- 172.21.15.66:0/323611726 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5400cf430 con 0x7fc5400cee50 2024-03-30T00:40:24.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.725+0000 7fc544a00640 1 --2- 172.21.15.66:0/323611726 >> v2:172.21.15.156:3300/0 conn(0x7fc5400af420 0x7fc5400d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:24.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc537400640 1 --2- 172.21.15.66:0/323611726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5400cee50 0x7fc5400cc040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc5400cac20 tx=0x7fc52801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bab9f9eb284c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:24.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc53f400640 1 -- 172.21.15.66:0/323611726 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5280ab680 con 0x7fc5400cee50 2024-03-30T00:40:24.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc53f400640 1 -- 172.21.15.66:0/323611726 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5280ab7e0 con 0x7fc5400cee50 2024-03-30T00:40:24.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc53f400640 1 -- 172.21.15.66:0/323611726 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5280abb70 con 0x7fc5400cee50 2024-03-30T00:40:24.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc546a00640 1 -- 172.21.15.66:0/323611726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5400cee50 msgr2=0x7fc5400cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc546a00640 1 --2- 172.21.15.66:0/323611726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5400cee50 0x7fc5400cc040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc5400cac20 tx=0x7fc52801c520 comp rx=0 tx=0).stop 2024-03-30T00:40:24.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc546a00640 1 -- 172.21.15.66:0/323611726 shutdown_connections 2024-03-30T00:40:24.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc546a00640 1 --2- 172.21.15.66:0/323611726 >> v2:172.21.15.156:3300/0 conn(0x7fc5400af420 0x7fc5400d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc546a00640 1 --2- 172.21.15.66:0/323611726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5400cee50 0x7fc5400cc040 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc546a00640 1 -- 172.21.15.66:0/323611726 >> 172.21.15.66:0/323611726 conn(0x7fc5400d0540 msgr2=0x7fc5400d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:24.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc546a00640 1 -- 172.21.15.66:0/323611726 shutdown_connections 2024-03-30T00:40:24.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.729+0000 7fc546a00640 1 -- 172.21.15.66:0/323611726 wait complete. 2024-03-30T00:40:24.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc546a00640 1 Processor -- start 2024-03-30T00:40:24.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc546a00640 1 -- start start 2024-03-30T00:40:24.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc546a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5400af420 0x7fc54014ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc546a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5400cee50 0x7fc54015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc546a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc54015de30 0x7fc5401518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc546a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54015af10 con 0x7fc5400af420 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc546a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54014f210 con 0x7fc54015de30 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc546a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54015a950 con 0x7fc5400cee50 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5400af420 0x7fc54014ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5400af420 0x7fc54014ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34256/0 (socket says 172.21.15.66:34256) 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53fe00640 1 -- 172.21.15.66:0/2411334511 learned_addr learned my addr 172.21.15.66:0/2411334511 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc537400640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5400cee50 0x7fc54015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc544a00640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc54015de30 0x7fc5401518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53fe00640 1 -- 172.21.15.66:0/2411334511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5400cee50 msgr2=0x7fc54015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53fe00640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5400cee50 0x7fc54015a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53fe00640 1 -- 172.21.15.66:0/2411334511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc54015de30 msgr2=0x7fc5401518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53fe00640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc54015de30 0x7fc5401518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53fe00640 1 -- 172.21.15.66:0/2411334511 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280094d0 con 0x7fc5400af420 2024-03-30T00:40:24.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53fe00640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5400af420 0x7fc54014ecd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c3a0 tx=0x7fc53000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:24.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53cc00640 1 -- 172.21.15.66:0/2411334511 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5300052d0 con 0x7fc5400af420 2024-03-30T00:40:24.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53cc00640 1 -- 172.21.15.66:0/2411334511 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc530005430 con 0x7fc5400af420 2024-03-30T00:40:24.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc53cc00640 1 -- 172.21.15.66:0/2411334511 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530007260 con 0x7fc5400af420 2024-03-30T00:40:24.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc546a00640 1 -- 172.21.15.66:0/2411334511 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54015ac30 con 0x7fc5400af420 2024-03-30T00:40:24.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc546a00640 1 -- 172.21.15.66:0/2411334511 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc540156310 con 0x7fc5400af420 2024-03-30T00:40:24.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.733+0000 7fc546a00640 1 -- 172.21.15.66:0/2411334511 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5400cdf60 con 0x7fc5400af420 2024-03-30T00:40:24.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.737+0000 7fc53cc00640 1 -- 172.21.15.66:0/2411334511 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc5300073c0 con 0x7fc5400af420 2024-03-30T00:40:24.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.737+0000 7fc53cc00640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc51c066690 0x7fc51c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:24.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.737+0000 7fc53cc00640 1 -- 172.21.15.66:0/2411334511 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc5300838e0 con 0x7fc5400af420 2024-03-30T00:40:24.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.737+0000 7fc537400640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc51c066690 0x7fc51c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:24.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.737+0000 7fc537400640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc51c066690 0x7fc51c068b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc5280af000 tx=0x7fc5280a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:24.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.741+0000 7fc53cc00640 1 -- 172.21.15.66:0/2411334511 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc5300508c0 con 0x7fc5400af420 2024-03-30T00:40:24.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.901+0000 7fc546a00640 1 -- 172.21.15.66:0/2411334511 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc540156500 con 0x7fc5400af420 2024-03-30T00:40:24.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.901+0000 7fc53cc00640 1 -- 172.21.15.66:0/2411334511 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc530053ee0 con 0x7fc5400af420 2024-03-30T00:40:24.911 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:24.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 -- 172.21.15.66:0/2411334511 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc51c066690 msgr2=0x7fc51c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc51c066690 0x7fc51c068b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc5280af000 tx=0x7fc5280a7000 comp rx=0 tx=0).stop 2024-03-30T00:40:24.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 -- 172.21.15.66:0/2411334511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5400af420 msgr2=0x7fc54014ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:24.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5400af420 0x7fc54014ecd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c3a0 tx=0x7fc53000c860 comp rx=0 tx=0).stop 2024-03-30T00:40:24.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 -- 172.21.15.66:0/2411334511 shutdown_connections 2024-03-30T00:40:24.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc54015de30 0x7fc5401518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5400cee50 0x7fc54015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc51c066690 0x7fc51c068b50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 --2- 172.21.15.66:0/2411334511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5400af420 0x7fc54014ecd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:24.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 -- 172.21.15.66:0/2411334511 >> 172.21.15.66:0/2411334511 conn(0x7fc5400d0540 msgr2=0x7fc5400cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:24.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 -- 172.21.15.66:0/2411334511 shutdown_connections 2024-03-30T00:40:24.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:24.905+0000 7fc546a00640 1 -- 172.21.15.66:0/2411334511 wait complete. 2024-03-30T00:40:24.933 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:40:25.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd233400640 1 Processor -- start 2024-03-30T00:40:25.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd233400640 1 -- start start 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd233400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c0ce820 0x7fd22c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd233400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c06e7d0 con 0x7fd22c15ca10 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd233400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c15cdf0 con 0x7fd22c0ae4e0 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd233400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0b1f70 con 0x7fd22c0ce820 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd230a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd22c15ca10 0x7fd22c06ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:43350/0 (socket says 172.21.15.66:43350) 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd230a00640 1 -- 172.21.15.66:0/1884336022 learned_addr learned my addr 172.21.15.66:0/1884336022 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd231400640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c0ce820 0x7fd22c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294049173 0 0) 0x7fd22c06e7d0 con 0x7fd22c15ca10 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd210003530 con 0x7fd22c15ca10 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513899010 0 0) 0x7fd22c15cdf0 con 0x7fd22c0ae4e0 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c06e7d0 con 0x7fd22c0ae4e0 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 448976111 0 0) 0x7fd210003530 con 0x7fd22c15ca10 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c0ce820 msgr2=0x7fd22c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:25.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c0ce820 0x7fd22c0cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 >> v1:172.21.15.156:6789/0 conn(0x7fd22c0ae4e0 legacy=0x7fd22c0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c0b16d0 con 0x7fd22c15ca10 2024-03-30T00:40:25.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fd21c001680 con 0x7fd22c15ca10 2024-03-30T00:40:25.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd231400640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c0ce820 0x7fd22c0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:25.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 >> v1:172.21.15.66:6789/0 conn(0x7fd22c15ca10 legacy=0x7fd22c06ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd210003b40 0x7fd210003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2100047d0 0x7fd210004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c0ce820 0x7fd210005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0b1f70 con 0x7fd210003b40 2024-03-30T00:40:25.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21000ba40 con 0x7fd2100047d0 2024-03-30T00:40:25.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21000bc00 con 0x7fd22c0ce820 2024-03-30T00:40:25.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd230a00640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd210003b40 0x7fd210003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd223e00640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2100047d0 0x7fd210004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.065+0000 7fd231400640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c0ce820 0x7fd210005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd230a00640 1 -- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c0ce820 msgr2=0x7fd210005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:25.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd230a00640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c0ce820 0x7fd210005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd230a00640 1 -- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2100047d0 msgr2=0x7fd210004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:25.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd230a00640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2100047d0 0x7fd210004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd230a00640 1 -- 172.21.15.66:0/1884336022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd224009a70 con 0x7fd210003b40 2024-03-30T00:40:25.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd230a00640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd210003b40 0x7fd210003f40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd21c001ec0 tx=0x7fd21c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58320413168dd18 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:25.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c005ae0 con 0x7fd210003b40 2024-03-30T00:40:25.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21c005c40 con 0x7fd210003b40 2024-03-30T00:40:25.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd22be00640 1 -- 172.21.15.66:0/1884336022 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0a57a0 con 0x7fd210003b40 2024-03-30T00:40:25.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd233400640 1 -- 172.21.15.66:0/1884336022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd210003b40 msgr2=0x7fd210003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd233400640 1 --2- 172.21.15.66:0/1884336022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd210003b40 0x7fd210003f40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd21c001ec0 tx=0x7fd21c01cab0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd231400640 1 -- 172.21.15.66:0/1884336022 reap_dead start 2024-03-30T00:40:25.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd233400640 1 -- 172.21.15.66:0/1884336022 shutdown_connections 2024-03-30T00:40:25.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd233400640 1 -- 172.21.15.66:0/1884336022 >> 172.21.15.66:0/1884336022 conn(0x7fd22c098a20 msgr2=0x7fd22c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:25.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd233400640 1 -- 172.21.15.66:0/1884336022 shutdown_connections 2024-03-30T00:40:25.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.069+0000 7fd233400640 1 -- 172.21.15.66:0/1884336022 wait complete. 2024-03-30T00:40:25.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd233400640 1 Processor -- start 2024-03-30T00:40:25.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd233400640 1 -- start start 2024-03-30T00:40:25.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd233400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd210003b40 0x7fd22c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd233400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd22c154a90 0x7fd22c154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd233400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c1553e0 0x7fd22c0cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd233400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0b31e0 con 0x7fd22c154a90 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd233400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c155da0 con 0x7fd210003b40 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd233400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c156080 con 0x7fd22c1553e0 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd223e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd22c154a90 0x7fd22c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd223e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd22c154a90 0x7fd22c154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34268/0 (socket says 172.21.15.66:34268) 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd223e00640 1 -- 172.21.15.66:0/1008252485 learned_addr learned my addr 172.21.15.66:0/1008252485 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd230a00640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd210003b40 0x7fd22c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd231400640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c1553e0 0x7fd22c0cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd223e00640 1 -- 172.21.15.66:0/1008252485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c1553e0 msgr2=0x7fd22c0cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd223e00640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c1553e0 0x7fd22c0cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd223e00640 1 -- 172.21.15.66:0/1008252485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd210003b40 msgr2=0x7fd22c0b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd223e00640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd210003b40 0x7fd22c0b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd223e00640 1 -- 172.21.15.66:0/1008252485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c0b24a0 con 0x7fd22c154a90 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd231400640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c1553e0 0x7fd22c0cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:25.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd223e00640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd22c154a90 0x7fd22c154160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd21400f6f0 tx=0x7fd21400fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:25.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd229600640 1 -- 172.21.15.66:0/1008252485 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214005110 con 0x7fd22c154a90 2024-03-30T00:40:25.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd229600640 1 -- 172.21.15.66:0/1008252485 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd214005290 con 0x7fd22c154a90 2024-03-30T00:40:25.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd233400640 1 -- 172.21.15.66:0/1008252485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22c0b2780 con 0x7fd22c154a90 2024-03-30T00:40:25.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd229600640 1 -- 172.21.15.66:0/1008252485 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2140067e0 con 0x7fd22c154a90 2024-03-30T00:40:25.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.073+0000 7fd233400640 1 -- 172.21.15.66:0/1008252485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c155890 con 0x7fd22c154a90 2024-03-30T00:40:25.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.077+0000 7fd233400640 1 -- 172.21.15.66:0/1008252485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22c155b70 con 0x7fd22c154a90 2024-03-30T00:40:25.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.077+0000 7fd229600640 1 -- 172.21.15.66:0/1008252485 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd214006940 con 0x7fd22c154a90 2024-03-30T00:40:25.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.077+0000 7fd229600640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd208066690 0x7fd208068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.077+0000 7fd229600640 1 -- 172.21.15.66:0/1008252485 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd214083c50 con 0x7fd22c154a90 2024-03-30T00:40:25.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.077+0000 7fd230a00640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd208066690 0x7fd208068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.077+0000 7fd230a00640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd208066690 0x7fd208068b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0ab000 tx=0x7fd21c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:25.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.081+0000 7fd229600640 1 -- 172.21.15.66:0/1008252485 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd214050ce0 con 0x7fd22c154a90 2024-03-30T00:40:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 -- 172.21.15.66:0/1008252485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd22c0b2300 con 0x7fd22c154a90 2024-03-30T00:40:25.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd229600640 1 -- 172.21.15.66:0/1008252485 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4730 (secure 0 0 0) 0x7fd22c0b2300 con 0x7fd22c154a90 2024-03-30T00:40:25.286 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:25.286 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4122431016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":4122431016},{"type":"v1","addr":"172.21.15.156:6835","nonce":4122431016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3882342907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3882342907},{"type":"v1","addr":"172.21.15.66:6835","nonce":3882342907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:39:51.142575+0000","modified":"2024-03-30T00:40:14.500142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/703895528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":703895528},{"type":"v1","addr":"172.21.15.66:6837","nonce":703895528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:40:25.286 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2024-03-30T00:40:25.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 -- 172.21.15.66:0/1008252485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd208066690 msgr2=0x7fd208068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd208066690 0x7fd208068b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0ab000 tx=0x7fd21c0a7070 comp rx=0 tx=0).stop 2024-03-30T00:40:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 -- 172.21.15.66:0/1008252485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd22c154a90 msgr2=0x7fd22c154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd22c154a90 0x7fd22c154160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd21400f6f0 tx=0x7fd21400fbb0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 -- 172.21.15.66:0/1008252485 shutdown_connections 2024-03-30T00:40:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd22c1553e0 0x7fd22c0cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd208066690 0x7fd208068b50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd22c154a90 0x7fd22c154160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 --2- 172.21.15.66:0/1008252485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd210003b40 0x7fd22c0b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 -- 172.21.15.66:0/1008252485 >> 172.21.15.66:0/1008252485 conn(0x7fd22c098a20 msgr2=0x7fd22c06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 -- 172.21.15.66:0/1008252485 shutdown_connections 2024-03-30T00:40:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.277+0000 7fd233400640 1 -- 172.21.15.66:0/1008252485 wait complete. 2024-03-30T00:40:25.307 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:25.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd452000640 1 Processor -- start 2024-03-30T00:40:25.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd452000640 1 -- start start 2024-03-30T00:40:25.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd452000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd44c0cda90 0x7fd44c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd452000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd44c0cde70 0x7fd44c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd452000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c15a990 con 0x7fd44c159c60 2024-03-30T00:40:25.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd452000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44c0b1b80 con 0x7fd44c0cda90 2024-03-30T00:40:25.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd452000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0cd3e0 con 0x7fd44c0cde70 2024-03-30T00:40:25.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd44b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd44c0cda90 0x7fd44c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd44b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd44c0cda90 0x7fd44c0ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34284/0 (socket says 172.21.15.66:34284) 2024-03-30T00:40:25.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd44b400640 1 -- 172.21.15.66:0/2948958966 learned_addr learned my addr 172.21.15.66:0/2948958966 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:25.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd44b400640 1 -- 172.21.15.66:0/2948958966 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd44c0cde70 msgr2=0x7fd44c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:25.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd44b400640 1 --2- 172.21.15.66:0/2948958966 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd44c0cde70 0x7fd44c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd44b400640 1 -- 172.21.15.66:0/2948958966 >> v1:172.21.15.156:6789/0 conn(0x7fd44c159c60 legacy=0x7fd44c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:25.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.437+0000 7fd44b400640 1 -- 172.21.15.66:0/2948958966 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0b11e0 con 0x7fd44c0cda90 2024-03-30T00:40:25.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd44b400640 1 --2- 172.21.15.66:0/2948958966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd44c0cda90 0x7fd44c0ce520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009870 tx=0x7fd43c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e88c2e0af67395 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:25.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd44aa00640 1 -- 172.21.15.66:0/2948958966 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0ad660 con 0x7fd44c0cda90 2024-03-30T00:40:25.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd44aa00640 1 -- 172.21.15.66:0/2948958966 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c0ad7c0 con 0x7fd44c0cda90 2024-03-30T00:40:25.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd44aa00640 1 -- 172.21.15.66:0/2948958966 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0adb50 con 0x7fd44c0cda90 2024-03-30T00:40:25.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd452000640 1 -- 172.21.15.66:0/2948958966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd44c0cda90 msgr2=0x7fd44c0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd452000640 1 --2- 172.21.15.66:0/2948958966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd44c0cda90 0x7fd44c0ce520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009870 tx=0x7fd43c01c490 comp rx=0 tx=0).stop 2024-03-30T00:40:25.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd452000640 1 -- 172.21.15.66:0/2948958966 shutdown_connections 2024-03-30T00:40:25.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd452000640 1 --2- 172.21.15.66:0/2948958966 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd44c0cde70 0x7fd44c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd452000640 1 --2- 172.21.15.66:0/2948958966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd44c0cda90 0x7fd44c0ce520 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd452000640 1 -- 172.21.15.66:0/2948958966 >> 172.21.15.66:0/2948958966 conn(0x7fd44c069f60 msgr2=0x7fd44c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:25.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd452000640 1 -- 172.21.15.66:0/2948958966 shutdown_connections 2024-03-30T00:40:25.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.441+0000 7fd452000640 1 -- 172.21.15.66:0/2948958966 wait complete. 2024-03-30T00:40:25.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd452000640 1 Processor -- start 2024-03-30T00:40:25.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd452000640 1 -- start start 2024-03-30T00:40:25.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd452000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd44c0cda90 0x7fd44c0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd452000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd44c0cde70 0x7fd44c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd452000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd44c159c60 0x7fd44c1f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd452000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c1f4d70 con 0x7fd44c0cda90 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd452000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0acff0 con 0x7fd44c159c60 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd452000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c1f8eb0 con 0x7fd44c0cde70 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd44b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd44c0cda90 0x7fd44c0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd44b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd44c0cda90 0x7fd44c0c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34300/0 (socket says 172.21.15.66:34300) 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd44b400640 1 -- 172.21.15.66:0/189641743 learned_addr learned my addr 172.21.15.66:0/189641743 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd44be00640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd44c159c60 0x7fd44c1f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd442a00640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd44c0cde70 0x7fd44c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd44b400640 1 -- 172.21.15.66:0/189641743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd44c0cde70 msgr2=0x7fd44c0ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd44b400640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd44c0cde70 0x7fd44c0ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd44b400640 1 -- 172.21.15.66:0/189641743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd44c159c60 msgr2=0x7fd44c1f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd44b400640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd44c159c60 0x7fd44c1f83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd44b400640 1 -- 172.21.15.66:0/189641743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c0094d0 con 0x7fd44c0cda90 2024-03-30T00:40:25.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd442a00640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd44c0cde70 0x7fd44c0ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:25.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd44b400640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd44c0cda90 0x7fd44c0c42d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0b0040 tx=0x7fd43c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:25.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd443e00640 1 -- 172.21.15.66:0/189641743 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0b8830 con 0x7fd44c0cda90 2024-03-30T00:40:25.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd443e00640 1 -- 172.21.15.66:0/189641743 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c0b89b0 con 0x7fd44c0cda90 2024-03-30T00:40:25.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd443e00640 1 -- 172.21.15.66:0/189641743 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0b8d60 con 0x7fd44c0cda90 2024-03-30T00:40:25.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd452000640 1 -- 172.21.15.66:0/189641743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c1f9190 con 0x7fd44c0cda90 2024-03-30T00:40:25.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.445+0000 7fd452000640 1 -- 172.21.15.66:0/189641743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c0ac970 con 0x7fd44c0cda90 2024-03-30T00:40:25.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.449+0000 7fd452000640 1 -- 172.21.15.66:0/189641743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c066c30 con 0x7fd44c0cda90 2024-03-30T00:40:25.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.449+0000 7fd443e00640 1 -- 172.21.15.66:0/189641743 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd43c0b4040 con 0x7fd44c0cda90 2024-03-30T00:40:25.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.449+0000 7fd443e00640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd428066690 0x7fd428068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.449+0000 7fd443e00640 1 -- 172.21.15.66:0/189641743 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd43c11ec00 con 0x7fd44c0cda90 2024-03-30T00:40:25.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.449+0000 7fd442a00640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd428066690 0x7fd428068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.449+0000 7fd442a00640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd428066690 0x7fd428068b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd44c1f8350 tx=0x7fd434008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:25.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.453+0000 7fd443e00640 1 -- 172.21.15.66:0/189641743 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd43c125080 con 0x7fd44c0cda90 2024-03-30T00:40:25.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.617+0000 7fd452000640 1 -- 172.21.15.66:0/189641743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd44c0b3170 con 0x7fd44c0cda90 2024-03-30T00:40:25.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.617+0000 7fd443e00640 1 -- 172.21.15.66:0/189641743 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fd43c0ebda0 con 0x7fd44c0cda90 2024-03-30T00:40:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:25.628 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:39:55.566951+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:25.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 -- 172.21.15.66:0/189641743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd428066690 msgr2=0x7fd428068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd428066690 0x7fd428068b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd44c1f8350 tx=0x7fd434008040 comp rx=0 tx=0).stop 2024-03-30T00:40:25.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 -- 172.21.15.66:0/189641743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd44c0cda90 msgr2=0x7fd44c0c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd44c0cda90 0x7fd44c0c42d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0b0040 tx=0x7fd43c01acd0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 -- 172.21.15.66:0/189641743 shutdown_connections 2024-03-30T00:40:25.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd44c159c60 0x7fd44c1f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd44c0cde70 0x7fd44c0ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd428066690 0x7fd428068b50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 --2- 172.21.15.66:0/189641743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd44c0cda90 0x7fd44c0c42d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 -- 172.21.15.66:0/189641743 >> 172.21.15.66:0/189641743 conn(0x7fd44c069f60 msgr2=0x7fd44c15da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:25.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 -- 172.21.15.66:0/189641743 shutdown_connections 2024-03-30T00:40:25.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.621+0000 7fd452000640 1 -- 172.21.15.66:0/189641743 wait complete. 2024-03-30T00:40:25.649 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:40:25.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4554400640 1 Processor -- start 2024-03-30T00:40:25.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4554400640 1 -- start start 2024-03-30T00:40:25.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4554400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f454c0cc560 0x7f454c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4554400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f454c06f8e0 0x7f454c06e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4554400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454c06b0b0 con 0x7f454c06e630 2024-03-30T00:40:25.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4554400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c06eac0 con 0x7f454c0cc560 2024-03-30T00:40:25.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4554400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c160640 con 0x7f454c06f8e0 2024-03-30T00:40:25.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4551a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f454c0cc560 0x7f454c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4551a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f454c0cc560 0x7f454c0d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34308/0 (socket says 172.21.15.66:34308) 2024-03-30T00:40:25.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4551a00640 1 -- 172.21.15.66:0/377360616 learned_addr learned my addr 172.21.15.66:0/377360616 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:25.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4551a00640 1 -- 172.21.15.66:0/377360616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f454c06f8e0 msgr2=0x7f454c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:25.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4551a00640 1 --2- 172.21.15.66:0/377360616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f454c06f8e0 0x7f454c06e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4551a00640 1 -- 172.21.15.66:0/377360616 >> v1:172.21.15.156:6789/0 conn(0x7f454c06e630 legacy=0x7f454c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:25.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.781+0000 7f4551a00640 1 -- 172.21.15.66:0/377360616 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c06b390 con 0x7f454c0cc560 2024-03-30T00:40:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f4551a00640 1 --2- 172.21.15.66:0/377360616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f454c0cc560 0x7f454c0d1060 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4540009870 tx=0x7f454001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2476feb2eebbe172 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f454be00640 1 -- 172.21.15.66:0/377360616 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45400ac630 con 0x7f454c0cc560 2024-03-30T00:40:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f454be00640 1 -- 172.21.15.66:0/377360616 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45400ac790 con 0x7f454c0cc560 2024-03-30T00:40:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f454be00640 1 -- 172.21.15.66:0/377360616 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45400acb20 con 0x7f454c0cc560 2024-03-30T00:40:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f4554400640 1 -- 172.21.15.66:0/377360616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f454c0cc560 msgr2=0x7f454c0d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f4554400640 1 --2- 172.21.15.66:0/377360616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f454c0cc560 0x7f454c0d1060 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4540009870 tx=0x7f454001c4f0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f4554400640 1 -- 172.21.15.66:0/377360616 shutdown_connections 2024-03-30T00:40:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f4554400640 1 --2- 172.21.15.66:0/377360616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f454c06f8e0 0x7f454c06e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f4554400640 1 --2- 172.21.15.66:0/377360616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f454c0cc560 0x7f454c0d1060 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f4554400640 1 -- 172.21.15.66:0/377360616 >> 172.21.15.66:0/377360616 conn(0x7f454c0d0540 msgr2=0x7f454c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f4554400640 1 -- 172.21.15.66:0/377360616 shutdown_connections 2024-03-30T00:40:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f4554400640 1 -- 172.21.15.66:0/377360616 wait complete. 2024-03-30T00:40:25.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.785+0000 7f4554400640 1 Processor -- start 2024-03-30T00:40:25.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4554400640 1 -- start start 2024-03-30T00:40:25.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4554400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f454c06e630 0x7f454c152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4554400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f454c06f8e0 0x7f454c0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4554400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f454c0cc560 0x7f454c0ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4554400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0b4150 con 0x7f454c0cc560 2024-03-30T00:40:25.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4554400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0b3e70 con 0x7f454c06e630 2024-03-30T00:40:25.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4554400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0c4110 con 0x7f454c06f8e0 2024-03-30T00:40:25.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4552400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f454c0cc560 0x7f454c0ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4552400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f454c0cc560 0x7f454c0ae460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34310/0 (socket says 172.21.15.66:34310) 2024-03-30T00:40:25.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4552400640 1 -- 172.21.15.66:0/3646150216 learned_addr learned my addr 172.21.15.66:0/3646150216 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:25.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4551a00640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f454c06e630 0x7f454c152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4551000640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f454c06f8e0 0x7f454c0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4552400640 1 -- 172.21.15.66:0/3646150216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f454c06f8e0 msgr2=0x7f454c0adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4552400640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f454c06f8e0 0x7f454c0adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4552400640 1 -- 172.21.15.66:0/3646150216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f454c06e630 msgr2=0x7f454c152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4552400640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f454c06e630 0x7f454c152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4552400640 1 -- 172.21.15.66:0/3646150216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45400094d0 con 0x7f454c0cc560 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4551a00640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f454c06e630 0x7f454c152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4552400640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f454c0cc560 0x7f454c0ae460 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f454400c780 tx=0x7f454400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4549600640 1 -- 172.21.15.66:0/3646150216 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544003d90 con 0x7f454c0cc560 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4549600640 1 -- 172.21.15.66:0/3646150216 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454400ce90 con 0x7f454c0cc560 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4549600640 1 -- 172.21.15.66:0/3646150216 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440052b0 con 0x7f454c0cc560 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4554400640 1 -- 172.21.15.66:0/3646150216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454c0c43f0 con 0x7f454c0cc560 2024-03-30T00:40:25.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4554400640 1 -- 172.21.15.66:0/3646150216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454c152d90 con 0x7f454c0cc560 2024-03-30T00:40:25.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.789+0000 7f4554400640 1 -- 172.21.15.66:0/3646150216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c0c5d70 con 0x7f454c0cc560 2024-03-30T00:40:25.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.793+0000 7f4549600640 1 -- 172.21.15.66:0/3646150216 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4544005410 con 0x7f454c0cc560 2024-03-30T00:40:25.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.793+0000 7f4549600640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f452c0666e0 0x7f452c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:25.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.793+0000 7f4549600640 1 -- 172.21.15.66:0/3646150216 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4544083d70 con 0x7f454c0cc560 2024-03-30T00:40:25.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.793+0000 7f4551a00640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f452c0666e0 0x7f452c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:25.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.793+0000 7f4551a00640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f452c0666e0 0x7f452c068ba0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f45400ae000 tx=0x7f45400a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:25.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.797+0000 7f4549600640 1 -- 172.21.15.66:0/3646150216 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4544050dd0 con 0x7f454c0cc560 2024-03-30T00:40:25.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.989+0000 7f4554400640 1 -- 172.21.15.66:0/3646150216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f454c066c30 con 0x7f454c0cc560 2024-03-30T00:40:25.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.989+0000 7f4549600640 1 -- 172.21.15.66:0/3646150216 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f45440543f0 con 0x7f454c0cc560 2024-03-30T00:40:25.999 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:26.000 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4122431016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":4122431016},{"type":"v1","addr":"172.21.15.156:6835","nonce":4122431016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3882342907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3882342907},{"type":"v1","addr":"172.21.15.66:6835","nonce":3882342907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:39:51.142575+0000","modified":"2024-03-30T00:40:14.500142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/703895528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":703895528},{"type":"v1","addr":"172.21.15.66:6837","nonce":703895528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:40:26.000 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2024-03-30T00:40:26.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 -- 172.21.15.66:0/3646150216 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f452c0666e0 msgr2=0x7f452c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f452c0666e0 0x7f452c068ba0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f45400ae000 tx=0x7f45400a7000 comp rx=0 tx=0).stop 2024-03-30T00:40:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 -- 172.21.15.66:0/3646150216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f454c0cc560 msgr2=0x7f454c0ae460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f454c0cc560 0x7f454c0ae460 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f454400c780 tx=0x7f454400cc40 comp rx=0 tx=0).stop 2024-03-30T00:40:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 -- 172.21.15.66:0/3646150216 shutdown_connections 2024-03-30T00:40:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f452c0666e0 0x7f452c068ba0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f454c0cc560 0x7f454c0ae460 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f454c06f8e0 0x7f454c0adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 --2- 172.21.15.66:0/3646150216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f454c06e630 0x7f454c152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 -- 172.21.15.66:0/3646150216 >> 172.21.15.66:0/3646150216 conn(0x7f454c0d0540 msgr2=0x7f454c0cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 -- 172.21.15.66:0/3646150216 shutdown_connections 2024-03-30T00:40:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:25.993+0000 7f4554400640 1 -- 172.21.15.66:0/3646150216 wait complete. 2024-03-30T00:40:26.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:39:51.142575+0000', 'modified': '2024-03-30T00:40:14.500142+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6837/703895528', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 703895528}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 703895528}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:40:26.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:40:26.423 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T00:40:26.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.557+0000 7f822e200640 1 Processor -- start 2024-03-30T00:40:26.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.557+0000 7f822e200640 1 -- start start 2024-03-30T00:40:26.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.561+0000 7f822e200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f82280cad00 0x7f82280c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.561+0000 7f822e200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82280c8e80 con 0x7f82280cd2d0 2024-03-30T00:40:26.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.561+0000 7f822e200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82280ca660 con 0x7f82280cc8d0 2024-03-30T00:40:26.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.561+0000 7f822e200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82280cbec0 con 0x7f82280cad00 2024-03-30T00:40:26.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.561+0000 7f8226a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f82280cd2d0 0x7f82280cb0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:43366/0 (socket says 172.21.15.66:43366) 2024-03-30T00:40:26.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.561+0000 7f8226a00640 1 -- 172.21.15.66:0/3008671678 learned_addr learned my addr 172.21.15.66:0/3008671678 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:26.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.561+0000 7f8227400640 1 --2- 172.21.15.66:0/3008671678 >> v2:172.21.15.156:3300/0 conn(0x7f82280cad00 0x7f82280c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.561+0000 7f8227400640 1 -- 172.21.15.66:0/3008671678 >> v1:172.21.15.156:6789/0 conn(0x7f82280cc8d0 legacy=0x7f822806c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.561+0000 7f8227400640 1 -- 172.21.15.66:0/3008671678 >> v1:172.21.15.66:6789/0 conn(0x7f82280cd2d0 legacy=0x7f82280cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.561+0000 7f8227400640 1 -- 172.21.15.66:0/3008671678 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82280ca940 con 0x7f82280cad00 2024-03-30T00:40:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f8227400640 1 --2- 172.21.15.66:0/3008671678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82280cad00 0x7f82280c8940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f82180097c0 tx=0x7f821801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b33161679120e36a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f8226000640 1 -- 172.21.15.66:0/3008671678 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180ab670 con 0x7f82280cad00 2024-03-30T00:40:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f8226000640 1 -- 172.21.15.66:0/3008671678 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82180ab7d0 con 0x7f82280cad00 2024-03-30T00:40:26.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f8226000640 1 -- 172.21.15.66:0/3008671678 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180abb60 con 0x7f82280cad00 2024-03-30T00:40:26.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f822e200640 1 -- 172.21.15.66:0/3008671678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82280cad00 msgr2=0x7f82280c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f822e200640 1 --2- 172.21.15.66:0/3008671678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82280cad00 0x7f82280c8940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f82180097c0 tx=0x7f821801c470 comp rx=0 tx=0).stop 2024-03-30T00:40:26.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f822e200640 1 -- 172.21.15.66:0/3008671678 shutdown_connections 2024-03-30T00:40:26.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f822e200640 1 --2- 172.21.15.66:0/3008671678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82280cad00 0x7f82280c8940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f822e200640 1 -- 172.21.15.66:0/3008671678 >> 172.21.15.66:0/3008671678 conn(0x7f8228069f60 msgr2=0x7f822806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:26.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f822e200640 1 -- 172.21.15.66:0/3008671678 shutdown_connections 2024-03-30T00:40:26.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f822e200640 1 -- 172.21.15.66:0/3008671678 wait complete. 2024-03-30T00:40:26.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f822e200640 1 Processor -- start 2024-03-30T00:40:26.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.565+0000 7f822e200640 1 -- start start 2024-03-30T00:40:26.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f822e200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82280cad00 0x7f822815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f822e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82280cc8d0 0x7f82280ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f822e200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82280cd2d0 0x7f82280acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f822e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8228097860 con 0x7f82280cc8d0 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f822e200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82280c4a70 con 0x7f82280cad00 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f822e200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82280c2010 con 0x7f82280cd2d0 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8226a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82280cc8d0 0x7f82280ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8226a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82280cc8d0 0x7f82280ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34314/0 (socket says 172.21.15.66:34314) 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8226a00640 1 -- 172.21.15.66:0/2840267720 learned_addr learned my addr 172.21.15.66:0/2840267720 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8227400640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82280cad00 0x7f822815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8227e00640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82280cd2d0 0x7f82280acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8226a00640 1 -- 172.21.15.66:0/2840267720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82280cd2d0 msgr2=0x7f82280acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8226a00640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82280cd2d0 0x7f82280acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8226a00640 1 -- 172.21.15.66:0/2840267720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82280cad00 msgr2=0x7f822815bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8226a00640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82280cad00 0x7f822815bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8226a00640 1 -- 172.21.15.66:0/2840267720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8218009420 con 0x7f82280cc8d0 2024-03-30T00:40:26.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8227400640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82280cad00 0x7f822815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:26.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f8226a00640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82280cc8d0 0x7f82280ac600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f821400c450 tx=0x7f821400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:26.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f821f400640 1 -- 172.21.15.66:0/2840267720 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214005cf0 con 0x7f82280cc8d0 2024-03-30T00:40:26.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f821f400640 1 -- 172.21.15.66:0/2840267720 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8214005e50 con 0x7f82280cc8d0 2024-03-30T00:40:26.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f821f400640 1 -- 172.21.15.66:0/2840267720 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214007270 con 0x7f82280cc8d0 2024-03-30T00:40:26.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f822e200640 1 -- 172.21.15.66:0/2840267720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82280c1790 con 0x7f82280cc8d0 2024-03-30T00:40:26.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f822e200640 1 -- 172.21.15.66:0/2840267720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82280c1a50 con 0x7f82280cc8d0 2024-03-30T00:40:26.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.569+0000 7f821f400640 1 -- 172.21.15.66:0/2840267720 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f82140073d0 con 0x7f82280cc8d0 2024-03-30T00:40:26.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f821f400640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8208066690 0x7f8208068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f821f400640 1 -- 172.21.15.66:0/2840267720 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8214083930 con 0x7f82280cc8d0 2024-03-30T00:40:26.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8227400640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8208066690 0x7f8208068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 Processor -- start 2024-03-30T00:40:26.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8227400640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8208066690 0x7f8208068b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f82180af000 tx=0x7f82180a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:26.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 -- start start 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6b2db0 0x55ec9d6b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d668f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6694c0 0x55ec9d6698a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d5315a0 con 0x55ec9d6b36d0 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d4beaf0 con 0x55ec9d6b2db0 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d4bc0c0 con 0x55ec9d6694c0 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8226a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d668f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8226a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d668f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34322/0 (socket says 172.21.15.66:34322) 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8226a00640 1 -- 172.21.15.66:0/1639555176 learned_addr learned my addr 172.21.15.66:0/1639555176 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8226a00640 1 -- 172.21.15.66:0/1639555176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6694c0 msgr2=0x55ec9d6698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8226a00640 1 --2- 172.21.15.66:0/1639555176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6694c0 0x55ec9d6698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8226a00640 1 -- 172.21.15.66:0/1639555176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6b2db0 msgr2=0x55ec9d6b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8226a00640 1 --2- 172.21.15.66:0/1639555176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6b2db0 0x55ec9d6b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8226a00640 1 -- 172.21.15.66:0/1639555176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec9d1b3890 con 0x55ec9d6b36d0 2024-03-30T00:40:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8226a00640 1 --2- 172.21.15.66:0/1639555176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d668f80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8214054380 tx=0x7f8214054800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b52f4751930434 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8213e00640 1 -- 172.21.15.66:0/1639555176 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214007680 con 0x55ec9d6b36d0 2024-03-30T00:40:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8213e00640 1 -- 172.21.15.66:0/1639555176 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8214058120 con 0x55ec9d6b36d0 2024-03-30T00:40:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f8213e00640 1 -- 172.21.15.66:0/1639555176 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214053b70 con 0x55ec9d6b36d0 2024-03-30T00:40:26.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 -- 172.21.15.66:0/1639555176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 msgr2=0x55ec9d668f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 --2- 172.21.15.66:0/1639555176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d668f80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8214054380 tx=0x7f8214054800 comp rx=0 tx=0).stop 2024-03-30T00:40:26.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 -- 172.21.15.66:0/1639555176 shutdown_connections 2024-03-30T00:40:26.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 --2- 172.21.15.66:0/1639555176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6694c0 0x55ec9d6698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 --2- 172.21.15.66:0/1639555176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d668f80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.573+0000 7f823163e000 1 --2- 172.21.15.66:0/1639555176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6b2db0 0x55ec9d6b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 -- 172.21.15.66:0/1639555176 >> 172.21.15.66:0/1639555176 conn(0x55ec9d66f830 msgr2=0x55ec9d667860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:26.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 -- 172.21.15.66:0/1639555176 shutdown_connections 2024-03-30T00:40:26.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 -- 172.21.15.66:0/1639555176 wait complete. 2024-03-30T00:40:26.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 -- start start 2024-03-30T00:40:26.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 Processor -- start 2024-03-30T00:40:26.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6694c0 0x55ec9d6b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d66d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6b36d0 0x55ec9d66d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d4b79b0 con 0x55ec9d6b2db0 2024-03-30T00:40:26.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d456c00 con 0x55ec9d6b36d0 2024-03-30T00:40:26.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d475070 con 0x55ec9d6694c0 2024-03-30T00:40:26.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 10 client.0 _ll_register_callbacks cb 0x55ec9d6550e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:40:26.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8226a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d66d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8227400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6694c0 0x55ec9d6b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8226a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d66d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34332/0 (socket says 172.21.15.66:34332) 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8226a00640 1 -- 172.21.15.66:0/1040203448 learned_addr learned my addr 172.21.15.66:0/1040203448 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8227e00640 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6b36d0 0x55ec9d66d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8226a00640 1 -- 172.21.15.66:0/1040203448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6694c0 msgr2=0x55ec9d6b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8226a00640 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6694c0 0x55ec9d6b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8226a00640 1 -- 172.21.15.66:0/1040203448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6b36d0 msgr2=0x55ec9d66d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8226a00640 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6b36d0 0x55ec9d66d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8226a00640 1 -- 172.21.15.66:0/1040203448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82280c1a50 con 0x55ec9d6b2db0 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8226a00640 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d66d230 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f821400c420 tx=0x7f8214000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee474aa55c06de8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8210c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8210c00640 1 -- 172.21.15.66:0/1040203448 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214053d70 con 0x55ec9d6b2db0 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8210c00640 1 -- 172.21.15.66:0/1040203448 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8214007740 con 0x55ec9d6b2db0 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 -- 172.21.15.66:0/1040203448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec9d4745f0 con 0x55ec9d6b2db0 2024-03-30T00:40:26.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8210c00640 1 -- 172.21.15.66:0/1040203448 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f821405c530 con 0x55ec9d6b2db0 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 10 client.4821 fetch_fsmap learned FSMap version 9 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 -- 172.21.15.66:0/1040203448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55ec9d47cab0 con 0x55ec9d6b2db0 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8210c00640 1 -- 172.21.15.66:0/1040203448 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec9d4745f0 con 0x55ec9d6b2db0 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8210c00640 1 -- 172.21.15.66:0/1040203448 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f821405a3a0 con 0x55ec9d6b2db0 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 10 client.4821 fetch_fsmap finished waiting for FSMap version 9 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 10 client.4821 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 20 client.4821 populate_metadata read hostname 'smithi066' 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55ec9d66e450 0x55ec9d66e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 4 client.4821 mds_command: new command op to 4299 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f823163e000 1 -- 172.21.15.66:0/1040203448 --> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec9d4b4790 con 0x55ec9d66e450 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.577+0000 7f8227400640 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55ec9d66e450 0x55ec9d66e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f8210c00640 10 client.4821 ms_handle_connect on v2:172.21.15.66:6836/703895528 2024-03-30T00:40:26.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f8227400640 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55ec9d66e450 0x55ec9d66e830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4b0cde9f4aa4d2 server_cookie=f59c01ac53f78f6f in_seq=0 out_seq=0 2024-03-30T00:40:26.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f8210c00640 1 -- 172.21.15.66:0/1040203448 <== mds.0 v2:172.21.15.66:6836/703895528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec9d4b4790 con 0x55ec9d66e450 2024-03-30T00:40:26.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f8210c00640 10 client.4821 handle_command_reply: tid=0 2024-03-30T00:40:26.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 1 client.4821 shutdown 2024-03-30T00:40:26.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 1 -- 172.21.15.66:0/1040203448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 msgr2=0x55ec9d66d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d66d230 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f821400c420 tx=0x7f8214000ee0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 1 -- 172.21.15.66:0/1040203448 shutdown_connections 2024-03-30T00:40:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6b36d0 0x55ec9d66d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55ec9d66e450 0x55ec9d66e830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d66d230 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 1 --2- 172.21.15.66:0/1040203448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6694c0 0x55ec9d6b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f8210c00640 0 client.4821 ms_handle_reset on v2:172.21.15.66:6836/703895528 2024-03-30T00:40:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 1 -- 172.21.15.66:0/1040203448 >> 172.21.15.66:0/1040203448 conn(0x55ec9d66f830 msgr2=0x55ec9d667f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 1 -- 172.21.15.66:0/1040203448 shutdown_connections 2024-03-30T00:40:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 1 -- 172.21.15.66:0/1040203448 wait complete. 2024-03-30T00:40:26.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.581+0000 7f823163e000 20 client.4821 trim_cache size 0 max 16384 2024-03-30T00:40:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 Processor -- start 2024-03-30T00:40:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 -- start start 2024-03-30T00:40:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6694c0 0x55ec9d6b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6b2db0 0x55ec9d6b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d6c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d174940 con 0x55ec9d6b36d0 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d6acbd0 con 0x55ec9d6694c0 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d454c10 con 0x55ec9d6b2db0 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8227e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d6c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8227e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d6c32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34344/0 (socket says 172.21.15.66:34344) 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8227e00640 1 -- 172.21.15.66:0/2068485996 learned_addr learned my addr 172.21.15.66:0/2068485996 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8226a00640 1 --2- 172.21.15.66:0/2068485996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6b2db0 0x55ec9d6b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8227400640 1 --2- 172.21.15.66:0/2068485996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6694c0 0x55ec9d6b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8227e00640 1 -- 172.21.15.66:0/2068485996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6b2db0 msgr2=0x55ec9d6b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8227e00640 1 --2- 172.21.15.66:0/2068485996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6b2db0 0x55ec9d6b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8227e00640 1 -- 172.21.15.66:0/2068485996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6694c0 msgr2=0x55ec9d6b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8227e00640 1 --2- 172.21.15.66:0/2068485996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6694c0 0x55ec9d6b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8227e00640 1 -- 172.21.15.66:0/2068485996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec9d462c00 con 0x55ec9d6b36d0 2024-03-30T00:40:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8226a00640 1 --2- 172.21.15.66:0/2068485996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6b2db0 0x55ec9d6b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f8227e00640 1 --2- 172.21.15.66:0/2068485996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d6c32b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8220003f70 tx=0x7f82200054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd01c5472a925c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f821cc00640 1 -- 172.21.15.66:0/2068485996 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220001850 con 0x55ec9d6b36d0 2024-03-30T00:40:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f821cc00640 1 -- 172.21.15.66:0/2068485996 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82200019f0 con 0x55ec9d6b36d0 2024-03-30T00:40:26.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f821cc00640 1 -- 172.21.15.66:0/2068485996 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220001da0 con 0x55ec9d6b36d0 2024-03-30T00:40:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 -- 172.21.15.66:0/2068485996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 msgr2=0x55ec9d6c32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 --2- 172.21.15.66:0/2068485996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d6c32b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8220003f70 tx=0x7f82200054e0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 -- 172.21.15.66:0/2068485996 shutdown_connections 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 --2- 172.21.15.66:0/2068485996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b36d0 0x55ec9d6c32b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 --2- 172.21.15.66:0/2068485996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6b2db0 0x55ec9d6b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 --2- 172.21.15.66:0/2068485996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6694c0 0x55ec9d6b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 -- 172.21.15.66:0/2068485996 >> 172.21.15.66:0/2068485996 conn(0x55ec9d66f830 msgr2=0x55ec9d6b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 -- 172.21.15.66:0/2068485996 shutdown_connections 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.597+0000 7f823163e000 1 -- 172.21.15.66:0/2068485996 wait complete. 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 -- start start 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 Processor -- start 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6694c0 0x55ec9d6c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d6c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6b36d0 0x55ec9d6c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d4bfd80 con 0x55ec9d6b2db0 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d4faac0 con 0x55ec9d6694c0 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec9d470f80 con 0x55ec9d6b36d0 2024-03-30T00:40:26.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 10 client.0 _ll_register_callbacks cb 0x55ec9d47c590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8226a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d6c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8226a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d6c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34348/0 (socket says 172.21.15.66:34348) 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8226a00640 1 -- 172.21.15.66:0/326064614 learned_addr learned my addr 172.21.15.66:0/326064614 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8227400640 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6694c0 0x55ec9d6c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8227e00640 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6b36d0 0x55ec9d6c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8226a00640 1 -- 172.21.15.66:0/326064614 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6b36d0 msgr2=0x55ec9d6c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8226a00640 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6b36d0 0x55ec9d6c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8226a00640 1 -- 172.21.15.66:0/326064614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6694c0 msgr2=0x55ec9d6c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8226a00640 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6694c0 0x55ec9d6c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8226a00640 1 -- 172.21.15.66:0/326064614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822000f040 con 0x55ec9d6b2db0 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8227400640 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6694c0 0x55ec9d6c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8226a00640 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d6c6a10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8214054610 tx=0x7f8214054580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cdd776dac335d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8210c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:40:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8210c00640 1 -- 172.21.15.66:0/326064614 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400e040 con 0x55ec9d6b2db0 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8210c00640 1 -- 172.21.15.66:0/326064614 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8214007740 con 0x55ec9d6b2db0 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 -- 172.21.15.66:0/326064614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec9d487460 con 0x55ec9d6b2db0 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8210c00640 1 -- 172.21.15.66:0/326064614 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82140509c0 con 0x55ec9d6b2db0 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 10 client.4827 fetch_fsmap learned FSMap version 9 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 -- 172.21.15.66:0/326064614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55ec9d1b3cc0 con 0x55ec9d6b2db0 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8210c00640 1 -- 172.21.15.66:0/326064614 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec9d487460 con 0x55ec9d6b2db0 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8210c00640 1 -- 172.21.15.66:0/326064614 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f8214053b70 con 0x55ec9d6b2db0 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 10 client.4827 fetch_fsmap finished waiting for FSMap version 9 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 10 client.4827 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 20 client.4827 populate_metadata read hostname 'smithi066' 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55ec9d6c7490 0x55ec9d6c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 4 client.4827 mds_command: new command op to 4299 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f823163e000 1 -- 172.21.15.66:0/326064614 --> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ec9d4809c0 con 0x55ec9d6c7490 2024-03-30T00:40:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8227400640 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55ec9d6c7490 0x55ec9d6c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:26.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8227400640 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55ec9d6c7490 0x55ec9d6c7870 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6935b004e299665 server_cookie=5c5b611ff51ed895 in_seq=0 out_seq=0 2024-03-30T00:40:26.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.601+0000 7f8210c00640 10 client.4827 ms_handle_connect on v2:172.21.15.66:6836/703895528 2024-03-30T00:40:26.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f8210c00640 1 -- 172.21.15.66:0/326064614 <== mds.0 v2:172.21.15.66:6836/703895528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55ec9d4809c0 con 0x55ec9d6c7490 2024-03-30T00:40:26.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f8210c00640 10 client.4827 handle_command_reply: tid=0 2024-03-30T00:40:26.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 1 client.4827 shutdown 2024-03-30T00:40:26.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 1 -- 172.21.15.66:0/326064614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 msgr2=0x55ec9d6c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d6c6a10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8214054610 tx=0x7f8214054580 comp rx=0 tx=0).stop 2024-03-30T00:40:26.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 1 -- 172.21.15.66:0/326064614 shutdown_connections 2024-03-30T00:40:26.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec9d6b36d0 0x55ec9d6c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.66:6836/703895528,v1:172.21.15.66:6837/703895528] conn(0x55ec9d6c7490 0x55ec9d6c7870 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec9d6b2db0 0x55ec9d6c6a10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 1 --2- 172.21.15.66:0/326064614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec9d6694c0 0x55ec9d6c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f8210c00640 0 client.4827 ms_handle_reset on v2:172.21.15.66:6836/703895528 2024-03-30T00:40:26.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 1 -- 172.21.15.66:0/326064614 >> 172.21.15.66:0/326064614 conn(0x55ec9d66f830 msgr2=0x55ec9d6b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:26.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 1 -- 172.21.15.66:0/326064614 shutdown_connections 2024-03-30T00:40:26.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 1 -- 172.21.15.66:0/326064614 wait complete. 2024-03-30T00:40:26.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f823163e000 20 client.4827 trim_cache size 0 max 16384 2024-03-30T00:40:26.618 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:40:26.618 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:40:26.618 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:40:26.618 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4299, 2024-03-30T00:40:26.618 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:40:26.618 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:40:26.618 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:40:26.619 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 35.462892277000002, 2024-03-30T00:40:26.619 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 9, 2024-03-30T00:40:26.619 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 25, 2024-03-30T00:40:26.619 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 25, 2024-03-30T00:40:26.619 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 41.582235541999999 2024-03-30T00:40:26.619 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:40:26.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 -- 172.21.15.66:0/2840267720 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8208066690 msgr2=0x7f8208068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8208066690 0x7f8208068b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f82180af000 tx=0x7f82180a7000 comp rx=0 tx=0).stop 2024-03-30T00:40:26.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 -- 172.21.15.66:0/2840267720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82280cc8d0 msgr2=0x7f82280ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:26.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82280cc8d0 0x7f82280ac600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f821400c450 tx=0x7f821400c910 comp rx=0 tx=0).stop 2024-03-30T00:40:26.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 -- 172.21.15.66:0/2840267720 shutdown_connections 2024-03-30T00:40:26.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82280cd2d0 0x7f82280acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8208066690 0x7f8208068b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82280cc8d0 0x7f82280ac600 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 --2- 172.21.15.66:0/2840267720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82280cad00 0x7f822815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:26.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 -- 172.21.15.66:0/2840267720 >> 172.21.15.66:0/2840267720 conn(0x7f8228069f60 msgr2=0x7f82280ae140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:26.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 -- 172.21.15.66:0/2840267720 shutdown_connections 2024-03-30T00:40:26.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:26.605+0000 7f822e200640 1 -- 172.21.15.66:0/2840267720 wait complete. 2024-03-30T00:40:26.633 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-30T00:40:26.643 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-30T00:40:26.644 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-30T00:40:26.644 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-30T00:40:26.644 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T00:40:26.644 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:40:26.644 INFO:tasks.ceph_fuse:client.2 config is {} 2024-03-30T00:40:26.644 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T00:40:26.644 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-30T00:40:26.644 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-30T00:40:26.644 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T00:40:26.644 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T00:40:26.645 INFO:tasks.ceph_fuse:client.3 config is {} 2024-03-30T00:40:26.645 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T00:40:26.645 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-30T00:40:26.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:26.645 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2024-03-30T00:40:26.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:26.655 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2024-03-30T00:40:26.719 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2024-03-30T00:40:26.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:40:26.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:26.721 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:40:26.730 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-30T00:40:26.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:26.731 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T00:40:26.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:26.792 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-03-30T00:40:27.011 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-30T00:40:27.011 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:40:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd184800640 1 Processor -- start 2024-03-30T00:40:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd184800640 1 -- start start 2024-03-30T00:40:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd184800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd17c15de30 0x7fd17c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd184800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd17c0af420 0x7fd17c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd184800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd17c0ce740 0x7fd17c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd184800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17c0cf0c0 con 0x7fd17c15de30 2024-03-30T00:40:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd184800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17c0cbb90 con 0x7fd17c0af420 2024-03-30T00:40:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd184800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0ce1b0 con 0x7fd17c0ce740 2024-03-30T00:40:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd181e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd17c15de30 0x7fd17c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd181e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd17c15de30 0x7fd17c0cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34358/0 (socket says 172.21.15.66:34358) 2024-03-30T00:40:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd181e00640 1 -- 172.21.15.66:0/831535459 learned_addr learned my addr 172.21.15.66:0/831535459 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd182800640 1 --2- 172.21.15.66:0/831535459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd17c0ce740 0x7fd17c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd181400640 1 --2- 172.21.15.66:0/831535459 >> v2:172.21.15.156:3300/0 conn(0x7fd17c0af420 0x7fd17c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd181e00640 1 -- 172.21.15.66:0/831535459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd17c0ce740 msgr2=0x7fd17c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd181e00640 1 --2- 172.21.15.66:0/831535459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd17c0ce740 0x7fd17c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd181e00640 1 -- 172.21.15.66:0/831535459 >> v2:172.21.15.156:3300/0 conn(0x7fd17c0af420 msgr2=0x7fd17c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd181e00640 1 --2- 172.21.15.66:0/831535459 >> v2:172.21.15.156:3300/0 conn(0x7fd17c0af420 0x7fd17c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd181e00640 1 -- 172.21.15.66:0/831535459 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0afa10 con 0x7fd17c15de30 2024-03-30T00:40:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.141+0000 7fd181400640 1 --2- 172.21.15.66:0/831535459 >> v2:172.21.15.156:3300/0 conn(0x7fd17c0af420 0x7fd17c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd181e00640 1 --2- 172.21.15.66:0/831535459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd17c15de30 0x7fd17c0cc040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd17c070080 tx=0x7fd17001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f0fca154f3778b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd180a00640 1 -- 172.21.15.66:0/831535459 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1700ab610 con 0x7fd17c15de30 2024-03-30T00:40:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd180a00640 1 -- 172.21.15.66:0/831535459 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1700ab770 con 0x7fd17c15de30 2024-03-30T00:40:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd180a00640 1 -- 172.21.15.66:0/831535459 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1700abb00 con 0x7fd17c15de30 2024-03-30T00:40:27.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd184800640 1 -- 172.21.15.66:0/831535459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd17c15de30 msgr2=0x7fd17c0cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:27.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd184800640 1 --2- 172.21.15.66:0/831535459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd17c15de30 0x7fd17c0cc040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd17c070080 tx=0x7fd17001c430 comp rx=0 tx=0).stop 2024-03-30T00:40:27.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd184800640 1 -- 172.21.15.66:0/831535459 shutdown_connections 2024-03-30T00:40:27.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd184800640 1 --2- 172.21.15.66:0/831535459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd17c0ce740 0x7fd17c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd184800640 1 --2- 172.21.15.66:0/831535459 >> v2:172.21.15.156:3300/0 conn(0x7fd17c0af420 0x7fd17c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd184800640 1 --2- 172.21.15.66:0/831535459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd17c15de30 0x7fd17c0cc040 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd184800640 1 -- 172.21.15.66:0/831535459 >> 172.21.15.66:0/831535459 conn(0x7fd17c0d0540 msgr2=0x7fd17c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd184800640 1 -- 172.21.15.66:0/831535459 shutdown_connections 2024-03-30T00:40:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.145+0000 7fd184800640 1 -- 172.21.15.66:0/831535459 wait complete. 2024-03-30T00:40:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd184800640 1 Processor -- start 2024-03-30T00:40:27.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd184800640 1 -- start start 2024-03-30T00:40:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd184800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd17c0af420 0x7fd17c0b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd184800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c0ce740 0x7fd17c147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd184800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd17c15de30 0x7fd17c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd184800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c156fe0 con 0x7fd17c15de30 2024-03-30T00:40:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd184800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c1572c0 con 0x7fd17c0ce740 2024-03-30T00:40:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd184800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c1575a0 con 0x7fd17c0af420 2024-03-30T00:40:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd182800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd17c15de30 0x7fd17c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd182800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd17c15de30 0x7fd17c0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34370/0 (socket says 172.21.15.66:34370) 2024-03-30T00:40:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd182800640 1 -- 172.21.15.66:0/1843049108 learned_addr learned my addr 172.21.15.66:0/1843049108 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd181400640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c0ce740 0x7fd17c147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd181e00640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd17c0af420 0x7fd17c0b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd182800640 1 -- 172.21.15.66:0/1843049108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd17c0af420 msgr2=0x7fd17c0b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd182800640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd17c0af420 0x7fd17c0b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd182800640 1 -- 172.21.15.66:0/1843049108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c0ce740 msgr2=0x7fd17c147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd182800640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c0ce740 0x7fd17c147710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd182800640 1 -- 172.21.15.66:0/1843049108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700094d0 con 0x7fd17c15de30 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd181400640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c0ce740 0x7fd17c147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd182800640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd17c15de30 0x7fd17c0caf60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd178007e40 tx=0x7fd17800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd176000640 1 -- 172.21.15.66:0/1843049108 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178014620 con 0x7fd17c15de30 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd176000640 1 -- 172.21.15.66:0/1843049108 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17800ee90 con 0x7fd17c15de30 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd176000640 1 -- 172.21.15.66:0/1843049108 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1780149e0 con 0x7fd17c15de30 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd184800640 1 -- 172.21.15.66:0/1843049108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c157880 con 0x7fd17c15de30 2024-03-30T00:40:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.149+0000 7fd184800640 1 -- 172.21.15.66:0/1843049108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c0b2d80 con 0x7fd17c15de30 2024-03-30T00:40:27.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.153+0000 7fd184800640 1 -- 172.21.15.66:0/1843049108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c0cdf60 con 0x7fd17c15de30 2024-03-30T00:40:27.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.153+0000 7fd176000640 1 -- 172.21.15.66:0/1843049108 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd178014b40 con 0x7fd17c15de30 2024-03-30T00:40:27.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.153+0000 7fd176000640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd160066690 0x7fd160068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:27.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.153+0000 7fd176000640 1 -- 172.21.15.66:0/1843049108 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd178084d90 con 0x7fd17c15de30 2024-03-30T00:40:27.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.153+0000 7fd181e00640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd160066690 0x7fd160068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:27.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.153+0000 7fd181e00640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd160066690 0x7fd160068b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd1700af000 tx=0x7fd1700b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:27.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.157+0000 7fd176000640 1 -- 172.21.15.66:0/1843049108 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd178051e10 con 0x7fd17c15de30 2024-03-30T00:40:27.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 -- 172.21.15.66:0/1843049108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd17c066c30 con 0x7fd17c15de30 2024-03-30T00:40:27.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd176000640 1 -- 172.21.15.66:0/1843049108 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd178055430 con 0x7fd17c15de30 2024-03-30T00:40:27.366 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T00:40:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 -- 172.21.15.66:0/1843049108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd160066690 msgr2=0x7fd160068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd160066690 0x7fd160068b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd1700af000 tx=0x7fd1700b6040 comp rx=0 tx=0).stop 2024-03-30T00:40:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 -- 172.21.15.66:0/1843049108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd17c15de30 msgr2=0x7fd17c0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd17c15de30 0x7fd17c0caf60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd178007e40 tx=0x7fd17800e4b0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 -- 172.21.15.66:0/1843049108 shutdown_connections 2024-03-30T00:40:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd160066690 0x7fd160068b50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd17c15de30 0x7fd17c0caf60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c0ce740 0x7fd17c147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 --2- 172.21.15.66:0/1843049108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd17c0af420 0x7fd17c0b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 -- 172.21.15.66:0/1843049108 >> 172.21.15.66:0/1843049108 conn(0x7fd17c0d0540 msgr2=0x7fd17c15ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 -- 172.21.15.66:0/1843049108 shutdown_connections 2024-03-30T00:40:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:27.357+0000 7fd184800640 1 -- 172.21.15.66:0/1843049108 wait complete. 2024-03-30T00:40:27.387 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:40:27.387 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:40:27.387 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:40:27.387 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:40:27.387 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:40:27.387 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:40:27.387 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:40:27.387 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:40:27.388 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:40:27.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:27.388 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:27.397 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42451sec preferred_lft 42451sec 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:40:27.398 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:27.399 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T00:40:27.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:27.399 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:40:27.399 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:40:27.400 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2024-03-30T00:40:27.400 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2024-03-30T00:40:27.400 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2024-03-30T00:40:27.400 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T00:40:27.400 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:40:27.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:27.507 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:40:27.526 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:40:27.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:27.529 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:40:27.584 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:40:27.584 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:40:27.584 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:40:27.584 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:40:27.584 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:40:27.584 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:40:27.584 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:40:27.584 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:27.584 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:27.585 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:27.585 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:40:27.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:27.585 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:40:27.585 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:40:27.585 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:40:27.585 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:40:27.585 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:40:27.585 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:40:27.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:27.673 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:40:27.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:27.683 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-03-30T00:40:27.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:27.737 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:40:27.737 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:40:27.737 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:40:27.737 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-30T00:40:27.737 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:40:27.818 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-30T00:40:27.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:27.818 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:40:27.818 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:40:27.818 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-30T00:40:27.818 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T00:40:27.819 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-30T00:40:27.819 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-30T00:40:27.819 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-30T00:40:27.819 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:40:27.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:27.955 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:40:27.955 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:40:27.955 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:40:27.955 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T00:40:27.956 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:40:27.995 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:40:27.995 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:40:27.996 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:40:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:40:28.045 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:40:28.045 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:40:28.096 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:40:28.156 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:40:28.156 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:40:28.218 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:40:28.220 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:40:28.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:28.220 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:40:28.272 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:40:28.272 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:40:28.319 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:40:28.339 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:40:28.340 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:40:28.403 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:40:28.405 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:40:28.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:28.405 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:40:28.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:40:28.405+0000 7f146d646580 -1 init, newargv = 0x55de6cafbe00 newargc=15 2024-03-30T00:40:28.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[35647]: starting ceph client 2024-03-30T00:40:28.421 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[35647]: starting fuse 2024-03-30T00:40:28.424 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:40:28.424 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:40:28.424 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:40:28.424 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:40:28.424 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:40:28.425 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:40:28.426 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:40:28.426 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:40:28.426 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:40:28.426 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:40:28.426 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:40:28.426 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:28.426 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:40:28.426 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:40:28.552 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.35647.asok 2024-03-30T00:40:28.736 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35647.asok 2024-03-30T00:40:28.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:28.737 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35647.asok status 2024-03-30T00:40:28.917 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:40:28.917 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:40:28.917 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:40:28.917 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:40:28.918 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:40:28.918 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:40:28.918 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:40:28.918 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "35647", 2024-03-30T00:40:28.918 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:40:28.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:28.918 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:40:28.918 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:40:28.918 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4839, 2024-03-30T00:40:28.918 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:40:28.918 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 4839 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3831244225 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:40:28.919 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3831244225 2024-03-30T00:40:28.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:28.920 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.4839 192.168.0.1:0/3831244225", 2024-03-30T00:40:28.920 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/3831244225", 2024-03-30T00:40:28.920 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:40:28.920 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 9, 2024-03-30T00:40:28.920 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 25, 2024-03-30T00:40:28.920 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:40:28.920 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:40:28.920 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:40:28.920 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:40:28.932 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:40:29.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.061+0000 7f781aa00640 1 Processor -- start 2024-03-30T00:40:29.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.061+0000 7f781aa00640 1 -- start start 2024-03-30T00:40:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.061+0000 7f781aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f78140cc8d0 0x7f781406c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.061+0000 7f781aa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78140ca660 con 0x7f78140cd2d0 2024-03-30T00:40:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.061+0000 7f781aa00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78140ca940 con 0x7f78140cad00 2024-03-30T00:40:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.061+0000 7f781aa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78140cbec0 con 0x7f78140cc8d0 2024-03-30T00:40:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f780b400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f78140cd2d0 0x7f78140cb0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46512/0 (socket says 172.21.15.66:46512) 2024-03-30T00:40:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f780b400640 1 -- 172.21.15.66:0/135445597 learned_addr learned my addr 172.21.15.66:0/135445597 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f7818a00640 1 --2- 172.21.15.66:0/135445597 >> v2:172.21.15.66:3300/0 conn(0x7f78140cc8d0 0x7f781406c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f7813400640 1 -- 172.21.15.66:0/135445597 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55447682 0 0) 0x7f78140ca660 con 0x7f78140cd2d0 2024-03-30T00:40:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f7813400640 1 -- 172.21.15.66:0/135445597 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77f8003530 con 0x7f78140cd2d0 2024-03-30T00:40:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f7818a00640 1 -- 172.21.15.66:0/135445597 >> v1:172.21.15.156:6789/0 conn(0x7f78140cad00 legacy=0x7f78140c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f7818a00640 1 -- 172.21.15.66:0/135445597 >> v1:172.21.15.66:6789/0 conn(0x7f78140cd2d0 legacy=0x7f78140cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f7818a00640 1 -- 172.21.15.66:0/135445597 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78140cc0d0 con 0x7f78140cc8d0 2024-03-30T00:40:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f7818a00640 1 --2- 172.21.15.66:0/135445597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78140cc8d0 0x7f781406c030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f780c009d90 tx=0x7f780c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d63a6903d304a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f7813400640 1 -- 172.21.15.66:0/135445597 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c0a5a20 con 0x7f78140cc8d0 2024-03-30T00:40:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f7813400640 1 -- 172.21.15.66:0/135445597 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c0a5b80 con 0x7f78140cc8d0 2024-03-30T00:40:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.065+0000 7f7813400640 1 -- 172.21.15.66:0/135445597 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c0ad440 con 0x7f78140cc8d0 2024-03-30T00:40:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 -- 172.21.15.66:0/135445597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78140cc8d0 msgr2=0x7f781406c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 --2- 172.21.15.66:0/135445597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78140cc8d0 0x7f781406c030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f780c009d90 tx=0x7f780c01c940 comp rx=0 tx=0).stop 2024-03-30T00:40:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 -- 172.21.15.66:0/135445597 shutdown_connections 2024-03-30T00:40:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 --2- 172.21.15.66:0/135445597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78140cc8d0 0x7f781406c030 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 -- 172.21.15.66:0/135445597 >> 172.21.15.66:0/135445597 conn(0x7f7814069f60 msgr2=0x7f781406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 -- 172.21.15.66:0/135445597 shutdown_connections 2024-03-30T00:40:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 -- 172.21.15.66:0/135445597 wait complete. 2024-03-30T00:40:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 Processor -- start 2024-03-30T00:40:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 -- start start 2024-03-30T00:40:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78140cad00 0x7f781415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78140cc8d0 0x7f7814157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78140cd2d0 0x7f7814167c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781415c310 con 0x7f78140cd2d0 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781415b940 con 0x7f78140cc8d0 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f781aa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7814168610 con 0x7f78140cad00 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f780b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78140cc8d0 0x7f7814157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f7813e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78140cad00 0x7f781415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f7813e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78140cad00 0x7f781415bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:53664/0 (socket says 172.21.15.66:53664) 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f780b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78140cc8d0 0x7f7814157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:36528/0 (socket says 172.21.15.66:36528) 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f7813e00640 1 -- 172.21.15.66:0/3113578528 learned_addr learned my addr 172.21.15.66:0/3113578528 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f7818a00640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78140cd2d0 0x7f7814167c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f7813e00640 1 -- 172.21.15.66:0/3113578528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78140cc8d0 msgr2=0x7f7814157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f7813e00640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78140cc8d0 0x7f7814157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f7813e00640 1 -- 172.21.15.66:0/3113578528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78140cd2d0 msgr2=0x7f7814167c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:29.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f7813e00640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78140cd2d0 0x7f7814167c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.069+0000 7f7813e00640 1 -- 172.21.15.66:0/3113578528 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c009a70 con 0x7f78140cad00 2024-03-30T00:40:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.073+0000 7f7813e00640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78140cad00 0x7f781415bdd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f780400c530 tx=0x7f780400c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.073+0000 7f7810c00640 1 -- 172.21.15.66:0/3113578528 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804005220 con 0x7f78140cad00 2024-03-30T00:40:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.073+0000 7f7810c00640 1 -- 172.21.15.66:0/3113578528 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7804005380 con 0x7f78140cad00 2024-03-30T00:40:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.073+0000 7f7810c00640 1 -- 172.21.15.66:0/3113578528 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804007270 con 0x7f78140cad00 2024-03-30T00:40:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.073+0000 7f781aa00640 1 -- 172.21.15.66:0/3113578528 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78141688f0 con 0x7f78140cad00 2024-03-30T00:40:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.073+0000 7f781aa00640 1 -- 172.21.15.66:0/3113578528 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781415b280 con 0x7f78140cad00 2024-03-30T00:40:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.073+0000 7f781aa00640 1 -- 172.21.15.66:0/3113578528 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7814066c30 con 0x7f78140cad00 2024-03-30T00:40:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.073+0000 7f7810c00640 1 -- 172.21.15.66:0/3113578528 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f78040073d0 con 0x7f78140cad00 2024-03-30T00:40:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.073+0000 7f7810c00640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77f0066690 0x7f77f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.073+0000 7f7810c00640 1 -- 172.21.15.66:0/3113578528 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7804083c10 con 0x7f78140cad00 2024-03-30T00:40:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.073+0000 7f780b400640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77f0066690 0x7f77f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.077+0000 7f780b400640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77f0066690 0x7f77f0068b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f78140c88a0 tx=0x7f77fc007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:29.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.077+0000 7f7810c00640 1 -- 172.21.15.66:0/3113578528 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7804050d30 con 0x7f78140cad00 2024-03-30T00:40:29.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.273+0000 7f781aa00640 1 -- 172.21.15.66:0/3113578528 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f781406a340 con 0x7f78140cad00 2024-03-30T00:40:29.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.273+0000 7f7810c00640 1 -- 172.21.15.66:0/3113578528 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7804054350 con 0x7f78140cad00 2024-03-30T00:40:29.284 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T00:40:29.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 -- 172.21.15.66:0/3113578528 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77f0066690 msgr2=0x7f77f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:29.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77f0066690 0x7f77f0068b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f78140c88a0 tx=0x7f77fc007d40 comp rx=0 tx=0).stop 2024-03-30T00:40:29.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 -- 172.21.15.66:0/3113578528 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78140cad00 msgr2=0x7f781415bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:29.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78140cad00 0x7f781415bdd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f780400c530 tx=0x7f780400c9f0 comp rx=0 tx=0).stop 2024-03-30T00:40:29.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 -- 172.21.15.66:0/3113578528 shutdown_connections 2024-03-30T00:40:29.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77f0066690 0x7f77f0068b50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:29.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78140cd2d0 0x7f7814167c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:29.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78140cc8d0 0x7f7814157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:29.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 --2- 172.21.15.66:0/3113578528 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78140cad00 0x7f781415bdd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:29.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 -- 172.21.15.66:0/3113578528 >> 172.21.15.66:0/3113578528 conn(0x7f7814069f60 msgr2=0x7f78140af020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:29.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 -- 172.21.15.66:0/3113578528 shutdown_connections 2024-03-30T00:40:29.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:29.277+0000 7f781aa00640 1 -- 172.21.15.66:0/3113578528 wait complete. 2024-03-30T00:40:29.305 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:40:29.305 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:40:29.305 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:40:29.305 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-30T00:40:29.305 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-30T00:40:29.305 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:40:29.305 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:40:29.305 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:40:29.306 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-30T00:40:29.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:29.306 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:40:29.314 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:29.315 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42449sec preferred_lft 42449sec 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:29.316 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:40:29.317 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.317 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:40:29.317 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.317 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:40:29.317 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:40:29.317 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-03-30T00:40:29.317 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:29.318 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:40:29.378 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:40:29.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:29.381 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:40:29.436 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:40:29.437 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:40:29.437 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:40:29.437 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:40:29.437 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:40:29.437 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:40:29.437 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:40:29.437 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:29.437 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:29.437 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:29.437 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:40:29.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:29.438 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:40:29.438 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:40:29.438 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:40:29.438 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:40:29.438 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:40:29.438 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:40:29.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:29.534 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:40:29.543 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:40:29.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:29.544 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-03-30T00:40:29.592 INFO:teuthology.orchestra.run.smithi066.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-30T00:40:29.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:29.593 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:40:29.593 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:40:29.593 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:40:29.593 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-30T00:40:29.594 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:40:29.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:29.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-30T00:40:29.741 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:40:29.741 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:40:29.742 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:40:29.742 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.742 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:40:29.742 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.742 INFO:teuthology.orchestra.run.smithi066.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:40:29.742 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 56:ed:79:4f:bc:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T00:40:29.742 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T00:40:29.742 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.742 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::54ed:79ff:fe4f:bc9f/64 scope link tentative 2024-03-30T00:40:29.742 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:29.744 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-30T00:40:29.804 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:40:29.804 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:40:29.805 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:40:29.805 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.805 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:40:29.805 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.805 INFO:teuthology.orchestra.run.smithi066.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:40:29.805 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 56:ed:79:4f:bc:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T00:40:29.805 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T00:40:29.805 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.805 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::54ed:79ff:fe4f:bc9f/64 scope link tentative 2024-03-30T00:40:29.805 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:29.807 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-30T00:40:29.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:29.807 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:40:29.807 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:40:29.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-30T00:40:29.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-30T00:40:29.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-30T00:40:29.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-30T00:40:29.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-30T00:40:29.807 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:40:29.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:29.946 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:40:29.946 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:40:29.947 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.1 up 2024-03-30T00:40:29.947 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.1 master ceph-brx 2024-03-30T00:40:29.947 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:40:29.985 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-03-30T00:40:29.985 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:40:29.985 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-30T00:40:30.040 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-30T00:40:30.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:40:30.041 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-30T00:40:30.092 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:40:30.152 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:40:30.152 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:40:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:40:30.220 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:40:30.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:30.221 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:40:30.272 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:40:30.272 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-03-30T00:40:30.272 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-03-30T00:40:30.315 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:40:30.335 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:40:30.335 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:40:30.394 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:40:30.396 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:40:30.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:30.397 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:40:30.410 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi066.stderr:2024-03-30T00:40:30.401+0000 7f341d248580 -1 init, newargv = 0x55950723ff60 newargc=15 2024-03-30T00:40:30.411 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi066.stderr:ceph-fuse[35781]: starting ceph client 2024-03-30T00:40:30.418 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:40:30.419 INFO:teuthology.orchestra.run.smithi066.stdout:44 2024-03-30T00:40:30.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi066.stderr:ceph-fuse[35781]: starting fuse 2024-03-30T00:40:30.419 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-03-30T00:40:30.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:40:30.420 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:30.421 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:40:30.421 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:40:30.421 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:40:30.421 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:40:30.421 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:40:30.421 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:40:30.421 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:40:30.421 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:40:30.421 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:40:30.421 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.2")) 2024-03-30T00:40:30.421 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:40:30.541 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.2.35781.asok 2024-03-30T00:40:30.725 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35781.asok 2024-03-30T00:40:30.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:30.726 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35781.asok status 2024-03-30T00:40:30.900 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:40:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:40:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:40:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:40:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "2", 2024-03-30T00:40:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "35781", 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4854, 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 4854 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.2:0", 2024-03-30T00:40:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3696152160 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.2:0", 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3696152160 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.4854 192.168.0.2:0/3696152160", 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.2:0/3696152160", 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 9, 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 25, 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:40:30.902 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:40:30.915 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:40:31.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc047e00640 1 Processor -- start 2024-03-30T00:40:31.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc047e00640 1 -- start start 2024-03-30T00:40:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc047e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0400cda90 0x7fc0400ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc047e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc0400cde70 0x7fc0400cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc047e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc040159c60 0x7fc04015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc047e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0400b1b80 con 0x7fc040159c60 2024-03-30T00:40:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc047e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04015a990 con 0x7fc0400cde70 2024-03-30T00:40:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc047e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0400b11e0 con 0x7fc0400cda90 2024-03-30T00:40:31.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc045e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc040159c60 0x7fc04015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:31.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc045e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc040159c60 0x7fc04015a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37406/0 (socket says 172.21.15.66:37406) 2024-03-30T00:40:31.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc045e00640 1 -- 172.21.15.66:0/302387923 learned_addr learned my addr 172.21.15.66:0/302387923 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:31.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc044a00640 1 --2- 172.21.15.66:0/302387923 >> v2:172.21.15.156:3300/0 conn(0x7fc0400cde70 0x7fc0400cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:31.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc045400640 1 --2- 172.21.15.66:0/302387923 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0400cda90 0x7fc0400ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:31.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc045e00640 1 -- 172.21.15.66:0/302387923 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0400cda90 msgr2=0x7fc0400ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:31.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc045e00640 1 --2- 172.21.15.66:0/302387923 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0400cda90 0x7fc0400ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:31.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc045e00640 1 -- 172.21.15.66:0/302387923 >> v2:172.21.15.156:3300/0 conn(0x7fc0400cde70 msgr2=0x7fc0400cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:31.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc045e00640 1 --2- 172.21.15.66:0/302387923 >> v2:172.21.15.156:3300/0 conn(0x7fc0400cde70 0x7fc0400cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:31.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.049+0000 7fc045e00640 1 -- 172.21.15.66:0/302387923 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0400b14c0 con 0x7fc040159c60 2024-03-30T00:40:31.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc045e00640 1 --2- 172.21.15.66:0/302387923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc040159c60 0x7fc04015a0b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc0400cf0d0 tx=0x7fc03801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89d946be083ebf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:31.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc03fe00640 1 -- 172.21.15.66:0/302387923 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0380a5800 con 0x7fc040159c60 2024-03-30T00:40:31.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc03fe00640 1 -- 172.21.15.66:0/302387923 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0380a5960 con 0x7fc040159c60 2024-03-30T00:40:31.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc03fe00640 1 -- 172.21.15.66:0/302387923 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0380a5cf0 con 0x7fc040159c60 2024-03-30T00:40:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc047e00640 1 -- 172.21.15.66:0/302387923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc040159c60 msgr2=0x7fc04015a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc047e00640 1 --2- 172.21.15.66:0/302387923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc040159c60 0x7fc04015a0b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc0400cf0d0 tx=0x7fc03801c960 comp rx=0 tx=0).stop 2024-03-30T00:40:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc047e00640 1 -- 172.21.15.66:0/302387923 shutdown_connections 2024-03-30T00:40:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc047e00640 1 --2- 172.21.15.66:0/302387923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc040159c60 0x7fc04015a0b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc047e00640 1 --2- 172.21.15.66:0/302387923 >> v2:172.21.15.156:3300/0 conn(0x7fc0400cde70 0x7fc0400cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc047e00640 1 --2- 172.21.15.66:0/302387923 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0400cda90 0x7fc0400ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc047e00640 1 -- 172.21.15.66:0/302387923 >> 172.21.15.66:0/302387923 conn(0x7fc040069f60 msgr2=0x7fc04006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc047e00640 1 -- 172.21.15.66:0/302387923 shutdown_connections 2024-03-30T00:40:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc047e00640 1 -- 172.21.15.66:0/302387923 wait complete. 2024-03-30T00:40:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc047e00640 1 Processor -- start 2024-03-30T00:40:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.053+0000 7fc047e00640 1 -- start start 2024-03-30T00:40:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc047e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0400cda90 0x7fc040097790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc047e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0400cde70 0x7fc0400c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc047e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc040159c60 0x7fc0400c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc047e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0400af720 con 0x7fc040159c60 2024-03-30T00:40:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc047e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0400abc30 con 0x7fc0400cda90 2024-03-30T00:40:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc047e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04006d0b0 con 0x7fc0400cde70 2024-03-30T00:40:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc045e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc040159c60 0x7fc0400c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:31.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc045e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc040159c60 0x7fc0400c0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37414/0 (socket says 172.21.15.66:37414) 2024-03-30T00:40:31.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc045e00640 1 -- 172.21.15.66:0/399118433 learned_addr learned my addr 172.21.15.66:0/399118433 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:31.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc045400640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0400cda90 0x7fc040097790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:31.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc044a00640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0400cde70 0x7fc0400c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:31.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc045e00640 1 -- 172.21.15.66:0/399118433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0400cde70 msgr2=0x7fc0400c28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:31.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc045e00640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0400cde70 0x7fc0400c28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:31.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc045e00640 1 -- 172.21.15.66:0/399118433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0400cda90 msgr2=0x7fc040097790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:31.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc045e00640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0400cda90 0x7fc040097790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:31.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc045e00640 1 -- 172.21.15.66:0/399118433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc038009b30 con 0x7fc040159c60 2024-03-30T00:40:31.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc045e00640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc040159c60 0x7fc0400c0fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc0380af040 tx=0x7fc038019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc03d600640 1 -- 172.21.15.66:0/399118433 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0380b8640 con 0x7fc040159c60 2024-03-30T00:40:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc03d600640 1 -- 172.21.15.66:0/399118433 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0380b87a0 con 0x7fc040159c60 2024-03-30T00:40:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc03d600640 1 -- 172.21.15.66:0/399118433 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0380b8b50 con 0x7fc040159c60 2024-03-30T00:40:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc047e00640 1 -- 172.21.15.66:0/399118433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0400974b0 con 0x7fc040159c60 2024-03-30T00:40:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc047e00640 1 -- 172.21.15.66:0/399118433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0400c2de0 con 0x7fc040159c60 2024-03-30T00:40:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.057+0000 7fc047e00640 1 -- 172.21.15.66:0/399118433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0400abda0 con 0x7fc040159c60 2024-03-30T00:40:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.061+0000 7fc03d600640 1 -- 172.21.15.66:0/399118433 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc0380b8cb0 con 0x7fc040159c60 2024-03-30T00:40:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.061+0000 7fc03d600640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0200666c0 0x7fc020068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:31.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.061+0000 7fc03d600640 1 -- 172.21.15.66:0/399118433 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc03811f340 con 0x7fc040159c60 2024-03-30T00:40:31.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.061+0000 7fc045400640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0200666c0 0x7fc020068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:31.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.061+0000 7fc045400640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0200666c0 0x7fc020068b80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0340045f0 tx=0x7fc034008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:31.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.065+0000 7fc03d600640 1 -- 172.21.15.66:0/399118433 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc0380ec460 con 0x7fc040159c60 2024-03-30T00:40:31.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 -- 172.21.15.66:0/399118433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc0400aba20 con 0x7fc040159c60 2024-03-30T00:40:31.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc03d600640 1 -- 172.21.15.66:0/399118433 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc0380efa80 con 0x7fc040159c60 2024-03-30T00:40:31.274 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T00:40:31.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 -- 172.21.15.66:0/399118433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0200666c0 msgr2=0x7fc020068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:31.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0200666c0 0x7fc020068b80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0340045f0 tx=0x7fc034008040 comp rx=0 tx=0).stop 2024-03-30T00:40:31.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 -- 172.21.15.66:0/399118433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc040159c60 msgr2=0x7fc0400c0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:31.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc040159c60 0x7fc0400c0fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc0380af040 tx=0x7fc038019700 comp rx=0 tx=0).stop 2024-03-30T00:40:31.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 -- 172.21.15.66:0/399118433 shutdown_connections 2024-03-30T00:40:31.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0200666c0 0x7fc020068b80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:31.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc040159c60 0x7fc0400c0fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:31.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0400cde70 0x7fc0400c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:31.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 --2- 172.21.15.66:0/399118433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0400cda90 0x7fc040097790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:31.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 -- 172.21.15.66:0/399118433 >> 172.21.15.66:0/399118433 conn(0x7fc040069f60 msgr2=0x7fc0400cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:31.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 -- 172.21.15.66:0/399118433 shutdown_connections 2024-03-30T00:40:31.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:31.265+0000 7fc047e00640 1 -- 172.21.15.66:0/399118433 wait complete. 2024-03-30T00:40:31.294 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:40:31.294 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:40:31.294 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2024-03-30T00:40:31.294 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-30T00:40:31.294 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T00:40:31.294 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:40:31.294 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:40:31.295 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:40:31.295 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T00:40:31.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:31.295 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:31.305 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42403sec preferred_lft 42403sec 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::ec4:7aff:fe88:6dfd/64 scope link 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 02:42:aa:4a:b5:00 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:40:31.306 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:31.307 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T00:40:31.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:31.308 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2024-03-30T00:40:31.308 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-03-30T00:40:31.308 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add name ceph-brx type bridge 2024-03-30T00:40:31.308 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr flush dev ceph-brx 2024-03-30T00:40:31.308 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set ceph-brx up 2024-03-30T00:40:31.308 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T00:40:31.308 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-03-30T00:40:31.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:31.421 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:40:31.440 INFO:teuthology.orchestra.run.smithi156.stdout:1 2024-03-30T00:40:31.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:31.442 DEBUG:teuthology.orchestra.run.smithi156:> route 2024-03-30T00:40:31.514 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2024-03-30T00:40:31.514 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:40:31.514 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:40:31.515 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:40:31.515 INFO:teuthology.orchestra.run.smithi156.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:40:31.515 INFO:teuthology.orchestra.run.smithi156.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:40:31.515 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:40:31.515 INFO:teuthology.orchestra.run.smithi156.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:31.515 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:31.515 INFO:teuthology.orchestra.run.smithi156.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:31.515 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:40:31.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:31.516 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2024-03-30T00:40:31.516 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-03-30T00:40:31.516 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:40:31.516 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:40:31.516 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:40:31.516 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-03-30T00:40:31.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:31.612 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2024-03-30T00:40:31.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:31.622 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list-id 2024-03-30T00:40:31.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:31.672 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2024-03-30T00:40:31.672 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-03-30T00:40:31.672 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T00:40:31.672 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-30T00:40:31.672 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-03-30T00:40:31.757 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-30T00:40:31.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:31.758 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2024-03-30T00:40:31.758 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-03-30T00:40:31.758 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-30T00:40:31.758 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T00:40:31.758 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-30T00:40:31.758 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-30T00:40:31.758 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-30T00:40:31.758 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-03-30T00:40:31.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:31.919 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2024-03-30T00:40:31.919 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-03-30T00:40:31.919 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2024-03-30T00:40:31.919 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T00:40:31.919 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-03-30T00:40:31.957 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-30T00:40:31.957 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:40:31.957 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-30T00:40:32.011 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-30T00:40:32.012 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:40:32.012 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-30T00:40:32.058 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2024-03-30T00:40:32.119 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:40:32.119 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:40:32.177 INFO:teuthology.orchestra.run.smithi156.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:40:32.179 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:40:32.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:32.179 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2024-03-30T00:40:32.227 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:40:32.227 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-30T00:40:32.271 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2024-03-30T00:40:32.291 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:40:32.291 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:40:32.350 INFO:teuthology.orchestra.run.smithi156.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:40:32.352 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:40:32.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:32.352 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2024-03-30T00:40:32.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi156.stderr:2024-03-30T00:40:32.353+0000 7fb7c0621580 -1 init, newargv = 0x559b851891d0 newargc=15 2024-03-30T00:40:32.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi156.stderr:ceph-fuse[31957]: starting ceph client 2024-03-30T00:40:32.373 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi156.stderr:ceph-fuse[31957]: starting fuse 2024-03-30T00:40:33.473 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2024-03-30T00:40:33.498 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:40:33.498 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:40:33.559 INFO:teuthology.orchestra.run.smithi156.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:40:33.561 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:40:33.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:33.562 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2024-03-30T00:40:33.612 INFO:teuthology.orchestra.run.smithi156.stdout:43 2024-03-30T00:40:33.613 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:40:33.613 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:40:33.613 DEBUG:teuthology.orchestra.run.smithi156:> import glob 2024-03-30T00:40:33.613 DEBUG:teuthology.orchestra.run.smithi156:> import re 2024-03-30T00:40:33.613 DEBUG:teuthology.orchestra.run.smithi156:> import os 2024-03-30T00:40:33.613 DEBUG:teuthology.orchestra.run.smithi156:> import subprocess 2024-03-30T00:40:33.613 DEBUG:teuthology.orchestra.run.smithi156:> 2024-03-30T00:40:33.613 DEBUG:teuthology.orchestra.run.smithi156:> def _find_admin_socket(client_name): 2024-03-30T00:40:33.613 DEBUG:teuthology.orchestra.run.smithi156:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-30T00:40:33.613 DEBUG:teuthology.orchestra.run.smithi156:> files = glob.glob(asok_path) 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> # Given a non-glob path, it better be there 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> if "*" not in asok_path: 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> assert(len(files) == 1) 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> return files[0] 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> for f in files: 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> contents = proc_f.read() 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> if mountpoint in contents: 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> return f 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> print(_find_admin_socket("client.1")) 2024-03-30T00:40:33.614 DEBUG:teuthology.orchestra.run.smithi156:> ' 2024-03-30T00:40:33.740 INFO:teuthology.orchestra.run.smithi156.stdout:/var/run/ceph/ceph-client.1.31957.asok 2024-03-30T00:40:33.925 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.31957.asok 2024-03-30T00:40:33.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:33.925 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.31957.asok status 2024-03-30T00:40:34.104 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-30T00:40:34.104 INFO:teuthology.orchestra.run.smithi156.stdout: "metadata": { 2024-03-30T00:40:34.104 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:40:34.104 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "entity_id": "1", 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "hostname": "smithi156", 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "pid": "31957", 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "root": "/" 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "dentry_count": 0, 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "dentry_pinned_count": 0, 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 4804, 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "inst": { 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "name": { 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "client", 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "num": 4804 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-30T00:40:34.105 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": { 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "v1", 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: "nonce": 4294604261 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": { 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "v1", 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: "nonce": 4294604261 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: "inst_str": "client.4804 192.168.0.1:0/4294604261", 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: "addr_str": "192.168.0.1:0/4294604261", 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: "inode_count": 1, 2024-03-30T00:40:34.106 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_epoch": 9, 2024-03-30T00:40:34.107 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_epoch": 26, 2024-03-30T00:40:34.107 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:40:34.107 INFO:teuthology.orchestra.run.smithi156.stdout: "blocklisted": false, 2024-03-30T00:40:34.107 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs" 2024-03-30T00:40:34.107 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-30T00:40:34.119 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:40:34.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b93600640 1 Processor -- start 2024-03-30T00:40:34.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b93600640 1 -- start start 2024-03-30T00:40:34.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b93600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b8c0ae4e0 0x7f9b8c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:34.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b93600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8c06e7d0 con 0x7f9b8c15ca10 2024-03-30T00:40:34.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b93600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8c15cdf0 con 0x7f9b8c0ce820 2024-03-30T00:40:34.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b93600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0b1f70 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b90c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9b8c15ca10 0x7f9b8c06ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46516/0 (socket says 172.21.15.66:46516) 2024-03-30T00:40:34.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b90c00640 1 -- 172.21.15.66:0/3371313250 learned_addr learned my addr 172.21.15.66:0/3371313250 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:34.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b8be00640 1 --2- 172.21.15.66:0/3371313250 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b8c0ae4e0 0x7f9b8c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:34.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b8b400640 1 -- 172.21.15.66:0/3371313250 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443666884 0 0) 0x7f9b8c06e7d0 con 0x7f9b8c15ca10 2024-03-30T00:40:34.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b8b400640 1 -- 172.21.15.66:0/3371313250 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b74003530 con 0x7f9b8c15ca10 2024-03-30T00:40:34.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b8b400640 1 -- 172.21.15.66:0/3371313250 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561643535 0 0) 0x7f9b8c15cdf0 con 0x7f9b8c0ce820 2024-03-30T00:40:34.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b8b400640 1 -- 172.21.15.66:0/3371313250 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b8c06e7d0 con 0x7f9b8c0ce820 2024-03-30T00:40:34.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b8be00640 1 -- 172.21.15.66:0/3371313250 >> v1:172.21.15.156:6789/0 conn(0x7f9b8c0ce820 legacy=0x7f9b8c160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:34.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b8b400640 1 -- 172.21.15.66:0/3371313250 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 244510351 0 0) 0x7f9b74003530 con 0x7f9b8c15ca10 2024-03-30T00:40:34.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b8be00640 1 -- 172.21.15.66:0/3371313250 >> v1:172.21.15.66:6789/0 conn(0x7f9b8c15ca10 legacy=0x7f9b8c06ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:34.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.269+0000 7f9b8be00640 1 -- 172.21.15.66:0/3371313250 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c0b16d0 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.273+0000 7f9b8be00640 1 --2- 172.21.15.66:0/3371313250 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b8c0ae4e0 0x7f9b8c0cb620 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b780097f0 tx=0x7f9b7801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f01e7310fdb9aae server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:34.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.273+0000 7f9b8b400640 1 -- 172.21.15.66:0/3371313250 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80001680 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.273+0000 7f9b8b400640 1 -- 172.21.15.66:0/3371313250 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b780a7050 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.273+0000 7f9b8b400640 1 -- 172.21.15.66:0/3371313250 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b780ab620 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.273+0000 7f9b93600640 1 -- 172.21.15.66:0/3371313250 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b8c0ae4e0 msgr2=0x7f9b8c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:34.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.273+0000 7f9b93600640 1 --2- 172.21.15.66:0/3371313250 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b8c0ae4e0 0x7f9b8c0cb620 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b780097f0 tx=0x7f9b7801c400 comp rx=0 tx=0).stop 2024-03-30T00:40:34.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.273+0000 7f9b93600640 1 -- 172.21.15.66:0/3371313250 shutdown_connections 2024-03-30T00:40:34.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.273+0000 7f9b93600640 1 --2- 172.21.15.66:0/3371313250 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b8c0ae4e0 0x7f9b8c0cb620 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:34.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.273+0000 7f9b93600640 1 -- 172.21.15.66:0/3371313250 >> 172.21.15.66:0/3371313250 conn(0x7f9b8c098a20 msgr2=0x7f9b8c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:34.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.273+0000 7f9b93600640 1 -- 172.21.15.66:0/3371313250 shutdown_connections 2024-03-30T00:40:34.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.273+0000 7f9b93600640 1 -- 172.21.15.66:0/3371313250 wait complete. 2024-03-30T00:40:34.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b93600640 1 Processor -- start 2024-03-30T00:40:34.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b93600640 1 -- start start 2024-03-30T00:40:34.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b93600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b8c0ae4e0 0x7f9b8c150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:34.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b93600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b8c0ce820 0x7f9b8c15fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:34.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b93600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c15ca10 0x7f9b8c15d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b93600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0c7dd0 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b93600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c15d550 con 0x7f9b8c15ca10 2024-03-30T00:40:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b93600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0b46a0 con 0x7f9b8c0ce820 2024-03-30T00:40:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b90c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b8c0ae4e0 0x7f9b8c150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b90c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b8c0ae4e0 0x7f9b8c150500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37424/0 (socket says 172.21.15.66:37424) 2024-03-30T00:40:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b90c00640 1 -- 172.21.15.66:0/2475092531 learned_addr learned my addr 172.21.15.66:0/2475092531 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b8be00640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b8c0ce820 0x7f9b8c15fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b91600640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c15ca10 0x7f9b8c15d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b90c00640 1 -- 172.21.15.66:0/2475092531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b8c0ce820 msgr2=0x7f9b8c15fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b90c00640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b8c0ce820 0x7f9b8c15fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b90c00640 1 -- 172.21.15.66:0/2475092531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c15ca10 msgr2=0x7f9b8c15d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b90c00640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c15ca10 0x7f9b8c15d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b90c00640 1 -- 172.21.15.66:0/2475092531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b78009420 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b90c00640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b8c0ae4e0 0x7f9b8c150500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000c780 tx=0x7f9b8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b88c00640 1 -- 172.21.15.66:0/2475092531 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80005cf0 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b88c00640 1 -- 172.21.15.66:0/2475092531 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b80005e50 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b88c00640 1 -- 172.21.15.66:0/2475092531 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b800072b0 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b93600640 1 -- 172.21.15.66:0/2475092531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c0b4980 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b93600640 1 -- 172.21.15.66:0/2475092531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c0c8290 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.277+0000 7f9b93600640 1 -- 172.21.15.66:0/2475092531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c0c84b0 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.281+0000 7f9b88c00640 1 -- 172.21.15.66:0/2475092531 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9b80007410 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.281+0000 7f9b88c00640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b6c066690 0x7f9b6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.281+0000 7f9b88c00640 1 -- 172.21.15.66:0/2475092531 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f9b80084290 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.281+0000 7f9b8be00640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b6c066690 0x7f9b6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.281+0000 7f9b8be00640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b6c066690 0x7f9b6c068b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b780ad000 tx=0x7f9b78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:34.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.285+0000 7f9b88c00640 1 -- 172.21.15.66:0/2475092531 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9b80051210 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 -- 172.21.15.66:0/2475092531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9b8c066c30 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b88c00640 1 -- 172.21.15.66:0/2475092531 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9b80054830 con 0x7f9b8c0ae4e0 2024-03-30T00:40:34.490 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T00:40:34.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 -- 172.21.15.66:0/2475092531 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b6c066690 msgr2=0x7f9b6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:34.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b6c066690 0x7f9b6c068b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b780ad000 tx=0x7f9b78008040 comp rx=0 tx=0).stop 2024-03-30T00:40:34.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 -- 172.21.15.66:0/2475092531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b8c0ae4e0 msgr2=0x7f9b8c150500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:34.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b8c0ae4e0 0x7f9b8c150500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000c780 tx=0x7f9b8000cc40 comp rx=0 tx=0).stop 2024-03-30T00:40:34.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 -- 172.21.15.66:0/2475092531 shutdown_connections 2024-03-30T00:40:34.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c15ca10 0x7f9b8c15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:34.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b8c0ce820 0x7f9b8c15fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:34.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b6c066690 0x7f9b6c068b50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:34.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 --2- 172.21.15.66:0/2475092531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b8c0ae4e0 0x7f9b8c150500 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:34.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 -- 172.21.15.66:0/2475092531 >> 172.21.15.66:0/2475092531 conn(0x7f9b8c098a20 msgr2=0x7f9b8c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:34.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 -- 172.21.15.66:0/2475092531 shutdown_connections 2024-03-30T00:40:34.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:34.481+0000 7f9b93600640 1 -- 172.21.15.66:0/2475092531 wait complete. 2024-03-30T00:40:34.510 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:40:34.510 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:40:34.510 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2024-03-30T00:40:34.511 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-30T00:40:34.511 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-30T00:40:34.511 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:40:34.511 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:40:34.511 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:40:34.511 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-30T00:40:34.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:34.511 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2024-03-30T00:40:34.522 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:40:34.522 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:40:34.522 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:40:34.522 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.522 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2024-03-30T00:40:34.522 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.522 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:34.522 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:34.522 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2024-03-30T00:40:34.522 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42399sec preferred_lft 42399sec 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::ec4:7aff:fe88:6dfd/64 scope link 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 02:42:aa:4a:b5:00 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:40:34.523 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:40:34.524 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:40:34.524 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.524 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:40:34.524 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.524 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:40:34.524 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T00:40:34.524 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:40:34.524 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:34.524 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:40:34.581 INFO:teuthology.orchestra.run.smithi156.stdout:1 2024-03-30T00:40:34.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:34.583 DEBUG:teuthology.orchestra.run.smithi156:> route 2024-03-30T00:40:34.634 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2024-03-30T00:40:34.634 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:40:34.635 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:40:34.635 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:40:34.635 INFO:teuthology.orchestra.run.smithi156.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:40:34.635 INFO:teuthology.orchestra.run.smithi156.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:40:34.635 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:40:34.635 INFO:teuthology.orchestra.run.smithi156.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:34.635 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:34.635 INFO:teuthology.orchestra.run.smithi156.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:40:34.635 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:40:34.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:34.635 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2024-03-30T00:40:34.635 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-03-30T00:40:34.636 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:40:34.636 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:40:34.636 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:40:34.636 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-03-30T00:40:34.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:34.724 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2024-03-30T00:40:34.732 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T00:40:34.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:34.733 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list-id 2024-03-30T00:40:34.784 INFO:teuthology.orchestra.run.smithi156.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-30T00:40:34.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:34.784 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2024-03-30T00:40:34.784 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-03-30T00:40:34.784 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T00:40:34.784 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-30T00:40:34.785 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-03-30T00:40:34.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:34.864 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-30T00:40:34.931 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:40:34.931 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:40:34.931 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:40:34.931 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.931 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2024-03-30T00:40:34.932 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.932 INFO:teuthology.orchestra.run.smithi156.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:40:34.932 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 22:2a:34:ae:26:9d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T00:40:34.932 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T00:40:34.932 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.932 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::202a:34ff:feae:269d/64 scope link 2024-03-30T00:40:34.932 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:34.933 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-30T00:40:34.995 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:40:34.995 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:40:34.995 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:40:34.995 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.995 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2024-03-30T00:40:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.996 INFO:teuthology.orchestra.run.smithi156.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:40:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 22:2a:34:ae:26:9d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T00:40:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T00:40:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::202a:34ff:feae:269d/64 scope link 2024-03-30T00:40:34.996 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:40:34.997 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-30T00:40:34.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:34.998 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2024-03-30T00:40:34.998 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-03-30T00:40:34.998 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-30T00:40:34.998 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-30T00:40:34.998 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-30T00:40:34.998 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-30T00:40:34.998 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-30T00:40:34.998 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-03-30T00:40:35.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:35.132 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2024-03-30T00:40:35.132 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-03-30T00:40:35.133 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.1 up 2024-03-30T00:40:35.133 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set dev brx.1 master ceph-brx 2024-03-30T00:40:35.133 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-03-30T00:40:35.170 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-03-30T00:40:35.171 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:40:35.171 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-30T00:40:35.219 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-30T00:40:35.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:40:35.220 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-30T00:40:35.271 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2024-03-30T00:40:35.331 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:40:35.331 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:40:35.389 INFO:teuthology.orchestra.run.smithi156.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:40:35.391 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:40:35.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:35.391 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2024-03-30T00:40:35.443 INFO:teuthology.orchestra.run.smithi156.stdout:43 2024-03-30T00:40:35.443 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-03-30T00:40:35.443 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-03-30T00:40:35.486 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2024-03-30T00:40:35.507 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:40:35.507 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:40:35.566 INFO:teuthology.orchestra.run.smithi156.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:40:35.568 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:40:35.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:35.569 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2024-03-30T00:40:35.581 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi156.stderr:2024-03-30T00:40:35.573+0000 7fcee584a580 -1 init, newargv = 0x55c2571d2020 newargc=15 2024-03-30T00:40:35.581 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi156.stderr:ceph-fuse[32071]: starting ceph client 2024-03-30T00:40:35.588 INFO:teuthology.orchestra.run.smithi156.stdout:43 2024-03-30T00:40:35.589 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi156.stderr:ceph-fuse[32071]: starting fuse 2024-03-30T00:40:36.690 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2024-03-30T00:40:36.713 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:40:36.713 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:40:36.778 INFO:teuthology.orchestra.run.smithi156.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:40:36.780 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:40:36.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:36.781 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2024-03-30T00:40:36.832 INFO:teuthology.orchestra.run.smithi156.stdout:43 2024-03-30T00:40:36.832 INFO:teuthology.orchestra.run.smithi156.stdout:44 2024-03-30T00:40:36.832 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-03-30T00:40:36.832 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:40:36.832 DEBUG:teuthology.orchestra.run.smithi156:> import glob 2024-03-30T00:40:36.832 DEBUG:teuthology.orchestra.run.smithi156:> import re 2024-03-30T00:40:36.832 DEBUG:teuthology.orchestra.run.smithi156:> import os 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> import subprocess 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> def _find_admin_socket(client_name): 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> files = glob.glob(asok_path) 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> # Given a non-glob path, it better be there 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> if "*" not in asok_path: 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> assert(len(files) == 1) 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> return files[0] 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> for f in files: 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:40:36.833 DEBUG:teuthology.orchestra.run.smithi156:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:40:36.834 DEBUG:teuthology.orchestra.run.smithi156:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:40:36.834 DEBUG:teuthology.orchestra.run.smithi156:> contents = proc_f.read() 2024-03-30T00:40:36.834 DEBUG:teuthology.orchestra.run.smithi156:> if mountpoint in contents: 2024-03-30T00:40:36.834 DEBUG:teuthology.orchestra.run.smithi156:> return f 2024-03-30T00:40:36.834 DEBUG:teuthology.orchestra.run.smithi156:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:40:36.834 DEBUG:teuthology.orchestra.run.smithi156:> 2024-03-30T00:40:36.834 DEBUG:teuthology.orchestra.run.smithi156:> print(_find_admin_socket("client.3")) 2024-03-30T00:40:36.834 DEBUG:teuthology.orchestra.run.smithi156:> ' 2024-03-30T00:40:36.959 INFO:teuthology.orchestra.run.smithi156.stdout:/var/run/ceph/ceph-client.3.32071.asok 2024-03-30T00:40:37.143 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.32071.asok 2024-03-30T00:40:37.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:37.143 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.32071.asok status 2024-03-30T00:40:37.313 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-30T00:40:37.313 INFO:teuthology.orchestra.run.smithi156.stdout: "metadata": { 2024-03-30T00:40:37.313 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:40:37.313 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:40:37.313 INFO:teuthology.orchestra.run.smithi156.stdout: "entity_id": "3", 2024-03-30T00:40:37.313 INFO:teuthology.orchestra.run.smithi156.stdout: "hostname": "smithi156", 2024-03-30T00:40:37.313 INFO:teuthology.orchestra.run.smithi156.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-03-30T00:40:37.313 INFO:teuthology.orchestra.run.smithi156.stdout: "pid": "32071", 2024-03-30T00:40:37.313 INFO:teuthology.orchestra.run.smithi156.stdout: "root": "/" 2024-03-30T00:40:37.313 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: "dentry_count": 0, 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: "dentry_pinned_count": 0, 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 4658, 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: "inst": { 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: "name": { 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "client", 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: "num": 4658 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": { 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "v1", 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": "192.168.0.2:0", 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: "nonce": 3161658429 2024-03-30T00:40:37.314 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": { 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "v1", 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": "192.168.0.2:0", 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "nonce": 3161658429 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "inst_str": "client.4658 192.168.0.2:0/3161658429", 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "addr_str": "192.168.0.2:0/3161658429", 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "inode_count": 1, 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_epoch": 9, 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_epoch": 27, 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "blocklisted": false, 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs" 2024-03-30T00:40:37.315 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-30T00:40:37.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:37.328 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:40:37.328 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:40:37.338 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:40:37.338 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:40:37.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:37.338 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:40:37.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:37.403 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:40:37.403 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.2 2024-03-30T00:40:37.453 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:40:37.454 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-03-30T00:40:37.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:37.454 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-30T00:40:37.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:37.518 DEBUG:teuthology.orchestra.run.smithi156:> stat --file-system '--printf=%T 2024-03-30T00:40:37.519 DEBUG:teuthology.orchestra.run.smithi156:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-30T00:40:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:fuseblk 2024-03-30T00:40:37.527 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-30T00:40:37.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:37.528 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-30T00:40:37.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:37.587 DEBUG:teuthology.orchestra.run.smithi156:> stat --file-system '--printf=%T 2024-03-30T00:40:37.587 DEBUG:teuthology.orchestra.run.smithi156:> ' -- /home/ubuntu/cephtest/mnt.3 2024-03-30T00:40:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:fuseblk 2024-03-30T00:40:37.636 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-03-30T00:40:37.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:37.636 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-30T00:40:37.699 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-30T00:40:37.715 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:37.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3b4a00640 1 Processor -- start 2024-03-30T00:40:37.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3b4a00640 1 -- start start 2024-03-30T00:40:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3b4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3b006e800 0x7fe3b00d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3b4a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b006b0b0 con 0x7fe3b006f8e0 2024-03-30T00:40:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3b4a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b006b390 con 0x7fe3b00cc560 2024-03-30T00:40:37.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3b4a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0160640 con 0x7fe3b006e800 2024-03-30T00:40:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3ae800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe3b006f8e0 0x7fe3b00ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46528/0 (socket says 172.21.15.66:46528) 2024-03-30T00:40:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3ae800640 1 -- 172.21.15.66:0/2943577033 learned_addr learned my addr 172.21.15.66:0/2943577033 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:37.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3a7e00640 1 --2- 172.21.15.66:0/2943577033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3b006e800 0x7fe3b00d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3ad400640 1 -- 172.21.15.66:0/2943577033 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688128059 0 0) 0x7fe3b006b0b0 con 0x7fe3b006f8e0 2024-03-30T00:40:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3ad400640 1 -- 172.21.15.66:0/2943577033 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe394003530 con 0x7fe3b006f8e0 2024-03-30T00:40:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3ad400640 1 -- 172.21.15.66:0/2943577033 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345987857 0 0) 0x7fe3b006b390 con 0x7fe3b00cc560 2024-03-30T00:40:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3ad400640 1 -- 172.21.15.66:0/2943577033 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3b006b0b0 con 0x7fe3b00cc560 2024-03-30T00:40:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3a7e00640 1 -- 172.21.15.66:0/2943577033 >> v1:172.21.15.156:6789/0 conn(0x7fe3b00cc560 legacy=0x7fe3b006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3a7e00640 1 -- 172.21.15.66:0/2943577033 >> v1:172.21.15.66:6789/0 conn(0x7fe3b006f8e0 legacy=0x7fe3b00ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:37.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.857+0000 7fe3a7e00640 1 -- 172.21.15.66:0/2943577033 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b006b670 con 0x7fe3b006e800 2024-03-30T00:40:37.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.861+0000 7fe3a7e00640 1 --2- 172.21.15.66:0/2943577033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3b006e800 0x7fe3b00d1060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe3980097f0 tx=0x7fe39801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c8673a78c0e9104 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:37.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.861+0000 7fe3ad400640 1 -- 172.21.15.66:0/2943577033 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980aa430 con 0x7fe3b006e800 2024-03-30T00:40:37.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.861+0000 7fe3ad400640 1 -- 172.21.15.66:0/2943577033 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3980aa590 con 0x7fe3b006e800 2024-03-30T00:40:37.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.861+0000 7fe3ad400640 1 -- 172.21.15.66:0/2943577033 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980aa920 con 0x7fe3b006e800 2024-03-30T00:40:37.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.861+0000 7fe3b4a00640 1 -- 172.21.15.66:0/2943577033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3b006e800 msgr2=0x7fe3b00d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:37.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.861+0000 7fe3b4a00640 1 --2- 172.21.15.66:0/2943577033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3b006e800 0x7fe3b00d1060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe3980097f0 tx=0x7fe39801c4b0 comp rx=0 tx=0).stop 2024-03-30T00:40:37.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.861+0000 7fe3b4a00640 1 -- 172.21.15.66:0/2943577033 shutdown_connections 2024-03-30T00:40:37.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.861+0000 7fe3b4a00640 1 --2- 172.21.15.66:0/2943577033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3b006e800 0x7fe3b00d1060 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:37.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.861+0000 7fe3b4a00640 1 -- 172.21.15.66:0/2943577033 >> 172.21.15.66:0/2943577033 conn(0x7fe3b00d0540 msgr2=0x7fe3b00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:37.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.861+0000 7fe3b4a00640 1 -- 172.21.15.66:0/2943577033 shutdown_connections 2024-03-30T00:40:37.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.861+0000 7fe3b4a00640 1 -- 172.21.15.66:0/2943577033 wait complete. 2024-03-30T00:40:37.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3b4a00640 1 Processor -- start 2024-03-30T00:40:37.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3b4a00640 1 -- start start 2024-03-30T00:40:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3b4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3b006e800 0x7fe3b01f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3b4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3b006f8e0 0x7fe3b00b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3b4a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3b00cc560 0x7fe3b00cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3b4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00b0600 con 0x7fe3b006e800 2024-03-30T00:40:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3b4a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b01f5e20 con 0x7fe3b00cc560 2024-03-30T00:40:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3b4a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00c85e0 con 0x7fe3b006f8e0 2024-03-30T00:40:37.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3ade00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3b006e800 0x7fe3b01f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3ade00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3b006e800 0x7fe3b01f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37434/0 (socket says 172.21.15.66:37434) 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3ade00640 1 -- 172.21.15.66:0/866388679 learned_addr learned my addr 172.21.15.66:0/866388679 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3a7e00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3b006f8e0 0x7fe3b00b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3ae800640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3b00cc560 0x7fe3b00cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3ade00640 1 -- 172.21.15.66:0/866388679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3b006f8e0 msgr2=0x7fe3b00b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3ade00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3b006f8e0 0x7fe3b00b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3ade00640 1 -- 172.21.15.66:0/866388679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3b00cc560 msgr2=0x7fe3b00cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3ade00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3b00cc560 0x7fe3b00cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3ade00640 1 -- 172.21.15.66:0/866388679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398009420 con 0x7fe3b006e800 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3ae800640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3b00cc560 0x7fe3b00cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3ade00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3b006e800 0x7fe3b01f58e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000c430 tx=0x7fe3a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3a6000640 1 -- 172.21.15.66:0/866388679 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a00052d0 con 0x7fe3b006e800 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3a6000640 1 -- 172.21.15.66:0/866388679 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0005450 con 0x7fe3b006e800 2024-03-30T00:40:37.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3b4a00640 1 -- 172.21.15.66:0/866388679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b00b0850 con 0x7fe3b006e800 2024-03-30T00:40:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3a6000640 1 -- 172.21.15.66:0/866388679 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0003840 con 0x7fe3b006e800 2024-03-30T00:40:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.865+0000 7fe3b4a00640 1 -- 172.21.15.66:0/866388679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b00b0e10 con 0x7fe3b006e800 2024-03-30T00:40:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.869+0000 7fe3b4a00640 1 -- 172.21.15.66:0/866388679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b0066c30 con 0x7fe3b006e800 2024-03-30T00:40:37.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.869+0000 7fe3a6000640 1 -- 172.21.15.66:0/866388679 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe3a00039a0 con 0x7fe3b006e800 2024-03-30T00:40:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.869+0000 7fe3a6000640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe38c0666c0 0x7fe38c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.869+0000 7fe3a6000640 1 -- 172.21.15.66:0/866388679 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fe3a0083bd0 con 0x7fe3b006e800 2024-03-30T00:40:37.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.869+0000 7fe3a7e00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe38c0666c0 0x7fe38c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.869+0000 7fe3a7e00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe38c0666c0 0x7fe38c068b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe3980ae000 tx=0x7fe3980a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:37.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:37.873+0000 7fe3a6000640 1 -- 172.21.15.66:0/866388679 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe3a0050b50 con 0x7fe3b006e800 2024-03-30T00:40:38.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.037+0000 7fe3b4a00640 1 -- 172.21.15.66:0/866388679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3b006aa80 con 0x7fe3b006e800 2024-03-30T00:40:38.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.037+0000 7fe3a6000640 1 -- 172.21.15.66:0/866388679 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7fe3a0017090 con 0x7fe3b006e800 2024-03-30T00:40:38.048 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:38.049 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:38.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 -- 172.21.15.66:0/866388679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe38c0666c0 msgr2=0x7fe38c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe38c0666c0 0x7fe38c068b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe3980ae000 tx=0x7fe3980a7000 comp rx=0 tx=0).stop 2024-03-30T00:40:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 -- 172.21.15.66:0/866388679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3b006e800 msgr2=0x7fe3b01f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3b006e800 0x7fe3b01f58e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000c430 tx=0x7fe3a000c8f0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 -- 172.21.15.66:0/866388679 shutdown_connections 2024-03-30T00:40:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3b00cc560 0x7fe3b00cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3b006f8e0 0x7fe3b00b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe38c0666c0 0x7fe38c068b80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 --2- 172.21.15.66:0/866388679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3b006e800 0x7fe3b01f58e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 -- 172.21.15.66:0/866388679 >> 172.21.15.66:0/866388679 conn(0x7fe3b00d0540 msgr2=0x7fe3b00ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 -- 172.21.15.66:0/866388679 shutdown_connections 2024-03-30T00:40:38.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.041+0000 7fe3b4a00640 1 -- 172.21.15.66:0/866388679 wait complete. 2024-03-30T00:40:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:38.070 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:38.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff981000640 1 Processor -- start 2024-03-30T00:40:38.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff981000640 1 -- start start 2024-03-30T00:40:38.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff981000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff97c0cde70 0x7ff97c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff981000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff97c0cd3e0 con 0x7ff97c0cda90 2024-03-30T00:40:38.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff981000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff97c15a990 con 0x7ff97c0cec30 2024-03-30T00:40:38.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff981000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c0b1b80 con 0x7ff97c0cde70 2024-03-30T00:40:38.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff979e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff97c0cda90 0x7ff97c0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46540/0 (socket says 172.21.15.66:46540) 2024-03-30T00:40:38.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff979e00640 1 -- 172.21.15.66:0/990283842 learned_addr learned my addr 172.21.15.66:0/990283842 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:38.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff97a800640 1 --2- 172.21.15.66:0/990283842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff97c0cde70 0x7ff97c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff979400640 1 -- 172.21.15.66:0/990283842 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164424629 0 0) 0x7ff97c0cd3e0 con 0x7ff97c0cda90 2024-03-30T00:40:38.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff979400640 1 -- 172.21.15.66:0/990283842 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff960005740 con 0x7ff97c0cda90 2024-03-30T00:40:38.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff979400640 1 -- 172.21.15.66:0/990283842 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644310635 0 0) 0x7ff97c15a990 con 0x7ff97c0cec30 2024-03-30T00:40:38.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff979400640 1 -- 172.21.15.66:0/990283842 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff97c0cd3e0 con 0x7ff97c0cec30 2024-03-30T00:40:38.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff97a800640 1 -- 172.21.15.66:0/990283842 >> v1:172.21.15.156:6789/0 conn(0x7ff97c0cec30 legacy=0x7ff97c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff97a800640 1 -- 172.21.15.66:0/990283842 >> v1:172.21.15.66:6789/0 conn(0x7ff97c0cda90 legacy=0x7ff97c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.205+0000 7ff97a800640 1 -- 172.21.15.66:0/990283842 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c0b11e0 con 0x7ff97c0cde70 2024-03-30T00:40:38.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.209+0000 7ff97a800640 1 --2- 172.21.15.66:0/990283842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff97c0cde70 0x7ff97c15a370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff974009dc0 tx=0x7ff97401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0310e8988a1522a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:38.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.209+0000 7ff979400640 1 -- 172.21.15.66:0/990283842 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9740a59c0 con 0x7ff97c0cde70 2024-03-30T00:40:38.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.209+0000 7ff979400640 1 -- 172.21.15.66:0/990283842 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9740a5b20 con 0x7ff97c0cde70 2024-03-30T00:40:38.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.209+0000 7ff979400640 1 -- 172.21.15.66:0/990283842 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9740ad440 con 0x7ff97c0cde70 2024-03-30T00:40:38.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.209+0000 7ff981000640 1 -- 172.21.15.66:0/990283842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff97c0cde70 msgr2=0x7ff97c15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.209+0000 7ff981000640 1 --2- 172.21.15.66:0/990283842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff97c0cde70 0x7ff97c15a370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff974009dc0 tx=0x7ff97401c900 comp rx=0 tx=0).stop 2024-03-30T00:40:38.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.209+0000 7ff981000640 1 -- 172.21.15.66:0/990283842 shutdown_connections 2024-03-30T00:40:38.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.209+0000 7ff981000640 1 --2- 172.21.15.66:0/990283842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff97c0cde70 0x7ff97c15a370 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.209+0000 7ff981000640 1 -- 172.21.15.66:0/990283842 >> 172.21.15.66:0/990283842 conn(0x7ff97c069f60 msgr2=0x7ff97c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:38.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.209+0000 7ff981000640 1 -- 172.21.15.66:0/990283842 shutdown_connections 2024-03-30T00:40:38.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.209+0000 7ff981000640 1 -- 172.21.15.66:0/990283842 wait complete. 2024-03-30T00:40:38.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff981000640 1 Processor -- start 2024-03-30T00:40:38.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff981000640 1 -- start start 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff981000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff97c0cda90 0x7ff97c1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff981000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97c0cde70 0x7ff97c071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff981000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff97c0cec30 0x7ff97c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff981000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c0ab010 con 0x7ff97c0cde70 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff981000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c0af180 con 0x7ff97c0cda90 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff981000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c0af460 con 0x7ff97c0cec30 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff973e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97c0cde70 0x7ff97c071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff973e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97c0cde70 0x7ff97c071ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37442/0 (socket says 172.21.15.66:37442) 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff973e00640 1 -- 172.21.15.66:0/4220478499 learned_addr learned my addr 172.21.15.66:0/4220478499 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff97a800640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff97c0cec30 0x7ff97c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff979e00640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff97c0cda90 0x7ff97c1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff973e00640 1 -- 172.21.15.66:0/4220478499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff97c0cec30 msgr2=0x7ff97c06c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff973e00640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff97c0cec30 0x7ff97c06c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff973e00640 1 -- 172.21.15.66:0/4220478499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff97c0cda90 msgr2=0x7ff97c1560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff973e00640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff97c0cda90 0x7ff97c1560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff973e00640 1 -- 172.21.15.66:0/4220478499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff974009a70 con 0x7ff97c0cde70 2024-03-30T00:40:38.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff979e00640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff97c0cda90 0x7ff97c1560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:38.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff973e00640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97c0cde70 0x7ff97c071ad0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c500 tx=0x7ff96400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:38.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff972000640 1 -- 172.21.15.66:0/4220478499 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9640052d0 con 0x7ff97c0cde70 2024-03-30T00:40:38.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff972000640 1 -- 172.21.15.66:0/4220478499 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff964005430 con 0x7ff97c0cde70 2024-03-30T00:40:38.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff972000640 1 -- 172.21.15.66:0/4220478499 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964012870 con 0x7ff97c0cde70 2024-03-30T00:40:38.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff981000640 1 -- 172.21.15.66:0/4220478499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff97c0af740 con 0x7ff97c0cde70 2024-03-30T00:40:38.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.213+0000 7ff981000640 1 -- 172.21.15.66:0/4220478499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97c1565f0 con 0x7ff97c0cde70 2024-03-30T00:40:38.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.217+0000 7ff981000640 1 -- 172.21.15.66:0/4220478499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff97c0ab180 con 0x7ff97c0cde70 2024-03-30T00:40:38.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.217+0000 7ff972000640 1 -- 172.21.15.66:0/4220478499 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff9640129d0 con 0x7ff97c0cde70 2024-03-30T00:40:38.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.217+0000 7ff972000640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff958066690 0x7ff958068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.217+0000 7ff972000640 1 -- 172.21.15.66:0/4220478499 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff964084280 con 0x7ff97c0cde70 2024-03-30T00:40:38.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.217+0000 7ff979e00640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff958066690 0x7ff958068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.217+0000 7ff979e00640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff958066690 0x7ff958068b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff96c005890 tx=0x7ff96c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:38.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.221+0000 7ff972000640 1 -- 172.21.15.66:0/4220478499 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff964051160 con 0x7ff97c0cde70 2024-03-30T00:40:38.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 -- 172.21.15.66:0/4220478499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff97c066c30 con 0x7ff97c0cde70 2024-03-30T00:40:38.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff972000640 1 -- 172.21.15.66:0/4220478499 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff964054780 con 0x7ff97c0cde70 2024-03-30T00:40:38.390 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 -- 172.21.15.66:0/4220478499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff958066690 msgr2=0x7ff958068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff958066690 0x7ff958068b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff96c005890 tx=0x7ff96c009040 comp rx=0 tx=0).stop 2024-03-30T00:40:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 -- 172.21.15.66:0/4220478499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97c0cde70 msgr2=0x7ff97c071ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97c0cde70 0x7ff97c071ad0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c500 tx=0x7ff96400c9c0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 -- 172.21.15.66:0/4220478499 shutdown_connections 2024-03-30T00:40:38.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff97c0cec30 0x7ff97c06c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff958066690 0x7ff958068b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff97c0cde70 0x7ff97c071ad0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 --2- 172.21.15.66:0/4220478499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff97c0cda90 0x7ff97c1560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 -- 172.21.15.66:0/4220478499 >> 172.21.15.66:0/4220478499 conn(0x7ff97c069f60 msgr2=0x7ff97c0cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:38.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 -- 172.21.15.66:0/4220478499 shutdown_connections 2024-03-30T00:40:38.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.381+0000 7ff981000640 1 -- 172.21.15.66:0/4220478499 wait complete. 2024-03-30T00:40:38.412 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:38.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f9132e00640 1 Processor -- start 2024-03-30T00:40:38.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f9132e00640 1 -- start start 2024-03-30T00:40:38.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f9132e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f912c15ca80 0x7f912c06b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f9132e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f912c0cf2f0 0x7f912c06b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f9132e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912c0cb0e0 con 0x7f912c06bb30 2024-03-30T00:40:38.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f9132e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912c0cf7f0 con 0x7f912c15ca80 2024-03-30T00:40:38.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f9132e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912c0cfad0 con 0x7f912c0cf2f0 2024-03-30T00:40:38.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f912be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f912c15ca80 0x7f912c06b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f912be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f912c15ca80 0x7f912c06b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37456/0 (socket says 172.21.15.66:37456) 2024-03-30T00:40:38.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f912be00640 1 -- 172.21.15.66:0/1075686139 learned_addr learned my addr 172.21.15.66:0/1075686139 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:38.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f9123400640 1 --2- 172.21.15.66:0/1075686139 >> v2:172.21.15.156:3300/0 conn(0x7f912c0cf2f0 0x7f912c06b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f912be00640 1 -- 172.21.15.66:0/1075686139 >> v2:172.21.15.156:3300/0 conn(0x7f912c0cf2f0 msgr2=0x7f912c06b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:38.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f912be00640 1 --2- 172.21.15.66:0/1075686139 >> v2:172.21.15.156:3300/0 conn(0x7f912c0cf2f0 0x7f912c06b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f912be00640 1 -- 172.21.15.66:0/1075686139 >> v1:172.21.15.156:6789/0 conn(0x7f912c06bb30 legacy=0x7f912c098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.545+0000 7f912be00640 1 -- 172.21.15.66:0/1075686139 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c0cad70 con 0x7f912c15ca80 2024-03-30T00:40:38.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f912be00640 1 --2- 172.21.15.66:0/1075686139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c15ca80 0x7f912c06b0b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f912c06ced0 tx=0x7f911c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ed4bc61a60ef99 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:38.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f912b400640 1 -- 172.21.15.66:0/1075686139 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0a8a10 con 0x7f912c15ca80 2024-03-30T00:40:38.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f912b400640 1 -- 172.21.15.66:0/1075686139 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c0a8b70 con 0x7f912c15ca80 2024-03-30T00:40:38.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f912b400640 1 -- 172.21.15.66:0/1075686139 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0b0440 con 0x7f912c15ca80 2024-03-30T00:40:38.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f9132e00640 1 -- 172.21.15.66:0/1075686139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c15ca80 msgr2=0x7f912c06b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f9132e00640 1 --2- 172.21.15.66:0/1075686139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c15ca80 0x7f912c06b0b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f912c06ced0 tx=0x7f911c01ed10 comp rx=0 tx=0).stop 2024-03-30T00:40:38.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f9132e00640 1 -- 172.21.15.66:0/1075686139 shutdown_connections 2024-03-30T00:40:38.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f9132e00640 1 --2- 172.21.15.66:0/1075686139 >> v2:172.21.15.156:3300/0 conn(0x7f912c0cf2f0 0x7f912c06b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f9132e00640 1 --2- 172.21.15.66:0/1075686139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c15ca80 0x7f912c06b0b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f9132e00640 1 -- 172.21.15.66:0/1075686139 >> 172.21.15.66:0/1075686139 conn(0x7f912c0b2090 msgr2=0x7f912c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:38.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f9132e00640 1 -- 172.21.15.66:0/1075686139 shutdown_connections 2024-03-30T00:40:38.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.549+0000 7f9132e00640 1 -- 172.21.15.66:0/1075686139 wait complete. 2024-03-30T00:40:38.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9132e00640 1 Processor -- start 2024-03-30T00:40:38.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9132e00640 1 -- start start 2024-03-30T00:40:38.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9132e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f912c06bb30 0x7f912c0b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9132e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0cf2f0 0x7f912c1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9132e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f912c15ca80 0x7f912c0c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9132e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0c89d0 con 0x7f912c0cf2f0 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9132e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0b4140 con 0x7f912c15ca80 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9132e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0b4420 con 0x7f912c06bb30 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9123400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0cf2f0 0x7f912c1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f912be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f912c06bb30 0x7f912c0b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f912be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f912c06bb30 0x7f912c0b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:53754/0 (socket says 172.21.15.66:53754) 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9123400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0cf2f0 0x7f912c1555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37464/0 (socket says 172.21.15.66:37464) 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f912be00640 1 -- 172.21.15.66:0/2327384963 learned_addr learned my addr 172.21.15.66:0/2327384963 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9130e00640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f912c15ca80 0x7f912c0c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9123400640 1 -- 172.21.15.66:0/2327384963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f912c06bb30 msgr2=0x7f912c0b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9123400640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f912c06bb30 0x7f912c0b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9123400640 1 -- 172.21.15.66:0/2327384963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f912c15ca80 msgr2=0x7f912c0c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9123400640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f912c15ca80 0x7f912c0c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9123400640 1 -- 172.21.15.66:0/2327384963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0094d0 con 0x7f912c0cf2f0 2024-03-30T00:40:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f912be00640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f912c06bb30 0x7f912c0b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9123400640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0cf2f0 0x7f912c1555a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f911400e870 tx=0x7f911400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9128c00640 1 -- 172.21.15.66:0/2327384963 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91140095d0 con 0x7f912c0cf2f0 2024-03-30T00:40:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9128c00640 1 -- 172.21.15.66:0/2327384963 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911400c940 con 0x7f912c0cf2f0 2024-03-30T00:40:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9128c00640 1 -- 172.21.15.66:0/2327384963 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911400ccb0 con 0x7f912c0cf2f0 2024-03-30T00:40:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9132e00640 1 -- 172.21.15.66:0/2327384963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c0b4700 con 0x7f912c0cf2f0 2024-03-30T00:40:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.553+0000 7f9132e00640 1 -- 172.21.15.66:0/2327384963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c155ae0 con 0x7f912c0cf2f0 2024-03-30T00:40:38.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.557+0000 7f9132e00640 1 -- 172.21.15.66:0/2327384963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912c155d60 con 0x7f912c0cf2f0 2024-03-30T00:40:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.557+0000 7f9128c00640 1 -- 172.21.15.66:0/2327384963 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9114013470 con 0x7f912c0cf2f0 2024-03-30T00:40:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.557+0000 7f9128c00640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91080666c0 0x7f9108068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.557+0000 7f9128c00640 1 -- 172.21.15.66:0/2327384963 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f9114084610 con 0x7f912c0cf2f0 2024-03-30T00:40:38.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.557+0000 7f912be00640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91080666c0 0x7f9108068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.557+0000 7f912be00640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91080666c0 0x7f9108068b80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f911c01ed50 tx=0x7f911c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:38.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.561+0000 7f9128c00640 1 -- 172.21.15.66:0/2327384963 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9114051620 con 0x7f912c0cf2f0 2024-03-30T00:40:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.721+0000 7f9132e00640 1 -- 172.21.15.66:0/2327384963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f912c0b4bb0 con 0x7f912c0cf2f0 2024-03-30T00:40:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.721+0000 7f9128c00640 1 -- 172.21.15.66:0/2327384963 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9114054c40 con 0x7f912c0cf2f0 2024-03-30T00:40:38.732 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 -- 172.21.15.66:0/2327384963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91080666c0 msgr2=0x7f9108068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91080666c0 0x7f9108068b80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f911c01ed50 tx=0x7f911c009040 comp rx=0 tx=0).stop 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 -- 172.21.15.66:0/2327384963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0cf2f0 msgr2=0x7f912c1555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0cf2f0 0x7f912c1555a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f911400e870 tx=0x7f911400ed30 comp rx=0 tx=0).stop 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 -- 172.21.15.66:0/2327384963 shutdown_connections 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f912c15ca80 0x7f912c0c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91080666c0 0x7f9108068b80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0cf2f0 0x7f912c1555a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 --2- 172.21.15.66:0/2327384963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f912c06bb30 0x7f912c0b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 -- 172.21.15.66:0/2327384963 >> 172.21.15.66:0/2327384963 conn(0x7f912c0b2090 msgr2=0x7f912c0ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 -- 172.21.15.66:0/2327384963 shutdown_connections 2024-03-30T00:40:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.725+0000 7f9132e00640 1 -- 172.21.15.66:0/2327384963 wait complete. 2024-03-30T00:40:38.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:38.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc580e00640 1 Processor -- start 2024-03-30T00:40:38.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc580e00640 1 -- start start 2024-03-30T00:40:38.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc580e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc57c0cda90 0x7fc57c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc580e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc57c0cde70 0x7fc57c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc580e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc57c159c60 0x7fc57c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc580e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57c0b1b80 con 0x7fc57c0cde70 2024-03-30T00:40:38.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc580e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57c15a990 con 0x7fc57c159c60 2024-03-30T00:40:38.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc580e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0b11e0 con 0x7fc57c0cda90 2024-03-30T00:40:38.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc573e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc57c0cde70 0x7fc57c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc573e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc57c0cde70 0x7fc57c0cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60572/0 (socket says 172.21.15.66:60572) 2024-03-30T00:40:38.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc573e00640 1 -- 172.21.15.66:0/2493308042 learned_addr learned my addr 172.21.15.66:0/2493308042 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:38.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc573e00640 1 -- 172.21.15.66:0/2493308042 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc57c0cda90 msgr2=0x7fc57c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:38.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc579e00640 1 --2- 172.21.15.66:0/2493308042 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc57c0cda90 0x7fc57c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc57a800640 1 --2- 172.21.15.66:0/2493308042 >> v2:172.21.15.156:3300/0 conn(0x7fc57c159c60 0x7fc57c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc573e00640 1 --2- 172.21.15.66:0/2493308042 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc57c0cda90 0x7fc57c0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc573e00640 1 -- 172.21.15.66:0/2493308042 >> v2:172.21.15.156:3300/0 conn(0x7fc57c159c60 msgr2=0x7fc57c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:38.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc573e00640 1 --2- 172.21.15.66:0/2493308042 >> v2:172.21.15.156:3300/0 conn(0x7fc57c159c60 0x7fc57c15a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc573e00640 1 -- 172.21.15.66:0/2493308042 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c0b14c0 con 0x7fc57c0cde70 2024-03-30T00:40:38.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc579e00640 1 --2- 172.21.15.66:0/2493308042 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc57c0cda90 0x7fc57c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:38.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.889+0000 7fc57a800640 1 --2- 172.21.15.66:0/2493308042 >> v2:172.21.15.156:3300/0 conn(0x7fc57c159c60 0x7fc57c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:38.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc573e00640 1 --2- 172.21.15.66:0/2493308042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc57c0cde70 0x7fc57c0cea60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0724d0 tx=0x7fc56401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c563415ed9407bb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:38.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc579400640 1 -- 172.21.15.66:0/2493308042 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5640ab670 con 0x7fc57c0cde70 2024-03-30T00:40:38.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc579400640 1 -- 172.21.15.66:0/2493308042 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5640ab7d0 con 0x7fc57c0cde70 2024-03-30T00:40:38.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc579400640 1 -- 172.21.15.66:0/2493308042 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5640abb60 con 0x7fc57c0cde70 2024-03-30T00:40:38.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc580e00640 1 -- 172.21.15.66:0/2493308042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc57c0cde70 msgr2=0x7fc57c0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc580e00640 1 --2- 172.21.15.66:0/2493308042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc57c0cde70 0x7fc57c0cea60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0724d0 tx=0x7fc56401c490 comp rx=0 tx=0).stop 2024-03-30T00:40:38.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc580e00640 1 -- 172.21.15.66:0/2493308042 shutdown_connections 2024-03-30T00:40:38.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc580e00640 1 --2- 172.21.15.66:0/2493308042 >> v2:172.21.15.156:3300/0 conn(0x7fc57c159c60 0x7fc57c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc580e00640 1 --2- 172.21.15.66:0/2493308042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc57c0cde70 0x7fc57c0cea60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc580e00640 1 --2- 172.21.15.66:0/2493308042 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc57c0cda90 0x7fc57c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc580e00640 1 -- 172.21.15.66:0/2493308042 >> 172.21.15.66:0/2493308042 conn(0x7fc57c069f60 msgr2=0x7fc57c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:38.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc580e00640 1 -- 172.21.15.66:0/2493308042 shutdown_connections 2024-03-30T00:40:38.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc580e00640 1 -- 172.21.15.66:0/2493308042 wait complete. 2024-03-30T00:40:38.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.893+0000 7fc580e00640 1 Processor -- start 2024-03-30T00:40:38.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc580e00640 1 -- start start 2024-03-30T00:40:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc580e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc57c0cda90 0x7fc57c164310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc580e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc57c0cde70 0x7fc57c159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc580e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc57c159c60 0x7fc57c0ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc580e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0c8e30 con 0x7fc57c159c60 2024-03-30T00:40:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc580e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0ad000 con 0x7fc57c0cda90 2024-03-30T00:40:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc580e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c159700 con 0x7fc57c0cde70 2024-03-30T00:40:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc57a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc57c159c60 0x7fc57c0ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc57a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc57c159c60 0x7fc57c0ad490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60584/0 (socket says 172.21.15.66:60584) 2024-03-30T00:40:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc579e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc57c0cda90 0x7fc57c164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc57a800640 1 -- 172.21.15.66:0/2400114018 learned_addr learned my addr 172.21.15.66:0/2400114018 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc573e00640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc57c0cde70 0x7fc57c159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc57a800640 1 -- 172.21.15.66:0/2400114018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc57c0cde70 msgr2=0x7fc57c159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc57a800640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc57c0cde70 0x7fc57c159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc57a800640 1 -- 172.21.15.66:0/2400114018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc57c0cda90 msgr2=0x7fc57c164310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc57a800640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc57c0cda90 0x7fc57c164310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc57a800640 1 -- 172.21.15.66:0/2400114018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5640094d0 con 0x7fc57c159c60 2024-03-30T00:40:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc57a800640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc57c159c60 0x7fc57c0ad490 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc57400cef0 tx=0x7fc57400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc572000640 1 -- 172.21.15.66:0/2400114018 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57400d840 con 0x7fc57c159c60 2024-03-30T00:40:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc572000640 1 -- 172.21.15.66:0/2400114018 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57400d9c0 con 0x7fc57c159c60 2024-03-30T00:40:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc572000640 1 -- 172.21.15.66:0/2400114018 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57400dd70 con 0x7fc57c159c60 2024-03-30T00:40:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc580e00640 1 -- 172.21.15.66:0/2400114018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c0ac9b0 con 0x7fc57c159c60 2024-03-30T00:40:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc580e00640 1 -- 172.21.15.66:0/2400114018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57c0ac660 con 0x7fc57c159c60 2024-03-30T00:40:38.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.897+0000 7fc580e00640 1 -- 172.21.15.66:0/2400114018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57c066c30 con 0x7fc57c159c60 2024-03-30T00:40:38.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.901+0000 7fc572000640 1 -- 172.21.15.66:0/2400114018 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc57401e3b0 con 0x7fc57c159c60 2024-03-30T00:40:38.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.901+0000 7fc572000640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc558066690 0x7fc558068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:38.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.901+0000 7fc572000640 1 -- 172.21.15.66:0/2400114018 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fc574084f70 con 0x7fc57c159c60 2024-03-30T00:40:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.901+0000 7fc579e00640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc558066690 0x7fc558068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:38.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.901+0000 7fc579e00640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc558066690 0x7fc558068b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc57c1f83c0 tx=0x7fc56c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:38.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:38.905+0000 7fc572000640 1 -- 172.21.15.66:0/2400114018 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc574052040 con 0x7fc57c159c60 2024-03-30T00:40:39.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 -- 172.21.15.66:0/2400114018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc57c0b3170 con 0x7fc57c159c60 2024-03-30T00:40:39.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc572000640 1 -- 172.21.15.66:0/2400114018 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc57401a020 con 0x7fc57c159c60 2024-03-30T00:40:39.074 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:39.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 -- 172.21.15.66:0/2400114018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc558066690 msgr2=0x7fc558068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc558066690 0x7fc558068b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc57c1f83c0 tx=0x7fc56c008040 comp rx=0 tx=0).stop 2024-03-30T00:40:39.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 -- 172.21.15.66:0/2400114018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc57c159c60 msgr2=0x7fc57c0ad490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc57c159c60 0x7fc57c0ad490 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc57400cef0 tx=0x7fc57400a710 comp rx=0 tx=0).stop 2024-03-30T00:40:39.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 -- 172.21.15.66:0/2400114018 shutdown_connections 2024-03-30T00:40:39.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc558066690 0x7fc558068b50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc57c159c60 0x7fc57c0ad490 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc57c0cde70 0x7fc57c159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 --2- 172.21.15.66:0/2400114018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc57c0cda90 0x7fc57c164310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 -- 172.21.15.66:0/2400114018 >> 172.21.15.66:0/2400114018 conn(0x7fc57c069f60 msgr2=0x7fc57c0cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:39.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 -- 172.21.15.66:0/2400114018 shutdown_connections 2024-03-30T00:40:39.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.065+0000 7fc580e00640 1 -- 172.21.15.66:0/2400114018 wait complete. 2024-03-30T00:40:39.096 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:39.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85fe00640 1 Processor -- start 2024-03-30T00:40:39.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85fe00640 1 -- start start 2024-03-30T00:40:39.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8580cca40 0x7fe8580cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85fe00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe858099960 0x7fe85806b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85fe00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85806bac0 con 0x7fe85806b140 2024-03-30T00:40:39.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85fe00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe858098ac0 con 0x7fe858099960 2024-03-30T00:40:39.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85fe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580cf3f0 con 0x7fe8580cca40 2024-03-30T00:40:39.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85de00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe858099960 0x7fe85806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85d400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8580cca40 0x7fe8580cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85de00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe858099960 0x7fe85806b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:58568/0 (socket says 172.21.15.66:58568) 2024-03-30T00:40:39.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85de00640 1 -- 172.21.15.66:0/1559630839 learned_addr learned my addr 172.21.15.66:0/1559630839 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:39.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85ca00640 1 --1- 172.21.15.66:0/1559630839 >> v1:172.21.15.156:6789/0 conn(0x7fe85806b140 0x7fe858099250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:50974/0 (socket says 172.21.15.66:50974) 2024-03-30T00:40:39.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85de00640 1 -- 172.21.15.66:0/1559630839 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8580cca40 msgr2=0x7fe8580cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:39.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85de00640 1 --2- 172.21.15.66:0/1559630839 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8580cca40 0x7fe8580cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85de00640 1 -- 172.21.15.66:0/1559630839 >> v1:172.21.15.156:6789/0 conn(0x7fe85806b140 legacy=0x7fe858099250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.233+0000 7fe85de00640 1 -- 172.21.15.66:0/1559630839 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe858099db0 con 0x7fe858099960 2024-03-30T00:40:39.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe85de00640 1 --2- 172.21.15.66:0/1559630839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe858099960 0x7fe85806b580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe850009e50 tx=0x7fe85001c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d110088f23070d45 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:39.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe857e00640 1 -- 172.21.15.66:0/1559630839 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8500a5890 con 0x7fe858099960 2024-03-30T00:40:39.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe857e00640 1 -- 172.21.15.66:0/1559630839 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8500a59f0 con 0x7fe858099960 2024-03-30T00:40:39.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe857e00640 1 -- 172.21.15.66:0/1559630839 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8500a5d80 con 0x7fe858099960 2024-03-30T00:40:39.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe85fe00640 1 -- 172.21.15.66:0/1559630839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe858099960 msgr2=0x7fe85806b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe85fe00640 1 --2- 172.21.15.66:0/1559630839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe858099960 0x7fe85806b580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe850009e50 tx=0x7fe85001c9c0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe85fe00640 1 -- 172.21.15.66:0/1559630839 shutdown_connections 2024-03-30T00:40:39.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe85fe00640 1 --2- 172.21.15.66:0/1559630839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe858099960 0x7fe85806b580 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe85fe00640 1 --2- 172.21.15.66:0/1559630839 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8580cca40 0x7fe8580cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe85fe00640 1 -- 172.21.15.66:0/1559630839 >> 172.21.15.66:0/1559630839 conn(0x7fe8580b2130 msgr2=0x7fe8580b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:39.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe85fe00640 1 -- 172.21.15.66:0/1559630839 shutdown_connections 2024-03-30T00:40:39.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe85fe00640 1 -- 172.21.15.66:0/1559630839 wait complete. 2024-03-30T00:40:39.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe85fe00640 1 Processor -- start 2024-03-30T00:40:39.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.237+0000 7fe85fe00640 1 -- start start 2024-03-30T00:40:39.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe85806b140 0x7fe858153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe858099960 0x7fe8580b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8580cca40 0x7fe8580b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858156b60 con 0x7fe858099960 2024-03-30T00:40:39.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85fe00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85815f9d0 con 0x7fe85806b140 2024-03-30T00:40:39.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85fe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85815fcb0 con 0x7fe8580cca40 2024-03-30T00:40:39.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe858099960 0x7fe8580b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe858099960 0x7fe8580b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60600/0 (socket says 172.21.15.66:60600) 2024-03-30T00:40:39.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85ca00640 1 -- 172.21.15.66:0/1464473327 learned_addr learned my addr 172.21.15.66:0/1464473327 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:39.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85d400640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe85806b140 0x7fe858153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85de00640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8580cca40 0x7fe8580b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85ca00640 1 -- 172.21.15.66:0/1464473327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8580cca40 msgr2=0x7fe8580b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85ca00640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8580cca40 0x7fe8580b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85ca00640 1 -- 172.21.15.66:0/1464473327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe85806b140 msgr2=0x7fe858153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85ca00640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe85806b140 0x7fe858153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85ca00640 1 -- 172.21.15.66:0/1464473327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe850009b30 con 0x7fe858099960 2024-03-30T00:40:39.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85de00640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8580cca40 0x7fe8580b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:39.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85ca00640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe858099960 0x7fe8580b3a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe84400c490 tx=0x7fe84400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:39.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe855600640 1 -- 172.21.15.66:0/1464473327 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844007480 con 0x7fe858099960 2024-03-30T00:40:39.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe855600640 1 -- 172.21.15.66:0/1464473327 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8440075e0 con 0x7fe858099960 2024-03-30T00:40:39.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe855600640 1 -- 172.21.15.66:0/1464473327 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844004a10 con 0x7fe858099960 2024-03-30T00:40:39.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85fe00640 1 -- 172.21.15.66:0/1464473327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85815ff90 con 0x7fe858099960 2024-03-30T00:40:39.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85fe00640 1 -- 172.21.15.66:0/1464473327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8580b3260 con 0x7fe858099960 2024-03-30T00:40:39.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe85fe00640 1 -- 172.21.15.66:0/1464473327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8580677d0 con 0x7fe858099960 2024-03-30T00:40:39.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.241+0000 7fe855600640 1 -- 172.21.15.66:0/1464473327 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe844004b70 con 0x7fe858099960 2024-03-30T00:40:39.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.245+0000 7fe855600640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8380666c0 0x7fe838068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.245+0000 7fe855600640 1 -- 172.21.15.66:0/1464473327 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fe844083670 con 0x7fe858099960 2024-03-30T00:40:39.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.245+0000 7fe85d400640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8380666c0 0x7fe838068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.245+0000 7fe85d400640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8380666c0 0x7fe838068b80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009a50 tx=0x7fe84c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:39.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.249+0000 7fe855600640 1 -- 172.21.15.66:0/1464473327 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe8440505d0 con 0x7fe858099960 2024-03-30T00:40:39.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 -- 172.21.15.66:0/1464473327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe858066c30 con 0x7fe858099960 2024-03-30T00:40:39.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe855600640 1 -- 172.21.15.66:0/1464473327 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe844053bf0 con 0x7fe858099960 2024-03-30T00:40:39.418 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 -- 172.21.15.66:0/1464473327 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8380666c0 msgr2=0x7fe838068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8380666c0 0x7fe838068b80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009a50 tx=0x7fe84c008040 comp rx=0 tx=0).stop 2024-03-30T00:40:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 -- 172.21.15.66:0/1464473327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe858099960 msgr2=0x7fe8580b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe858099960 0x7fe8580b3a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe84400c490 tx=0x7fe84400c950 comp rx=0 tx=0).stop 2024-03-30T00:40:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 -- 172.21.15.66:0/1464473327 shutdown_connections 2024-03-30T00:40:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8580cca40 0x7fe8580b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8380666c0 0x7fe838068b80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe858099960 0x7fe8580b3a60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 --2- 172.21.15.66:0/1464473327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe85806b140 0x7fe858153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 -- 172.21.15.66:0/1464473327 >> 172.21.15.66:0/1464473327 conn(0x7fe8580b2130 msgr2=0x7fe85806fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:39.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 -- 172.21.15.66:0/1464473327 shutdown_connections 2024-03-30T00:40:39.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.409+0000 7fe85fe00640 1 -- 172.21.15.66:0/1464473327 wait complete. 2024-03-30T00:40:39.440 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:39.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.569+0000 7fbbdde00640 1 Processor -- start 2024-03-30T00:40:39.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.569+0000 7fbbdde00640 1 -- start start 2024-03-30T00:40:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.569+0000 7fbbdde00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80cec30 0x7fbbd8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbdde00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbbd80cde70 0x7fbbd815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbdde00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd815a6e0 con 0x7fbbd80cda90 2024-03-30T00:40:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbdde00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd80b1b80 con 0x7fbbd80cde70 2024-03-30T00:40:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbdde00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80cd3e0 con 0x7fbbd80cec30 2024-03-30T00:40:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fbbd80cda90 0x7fbbd80ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42392/0 (socket says 172.21.15.66:42392) 2024-03-30T00:40:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6a00640 1 -- 172.21.15.66:0/3957901277 learned_addr learned my addr 172.21.15.66:0/3957901277 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891157785 0 0) 0x7fbbd815a6e0 con 0x7fbbd80cda90 2024-03-30T00:40:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd7400640 1 --2- 172.21.15.66:0/3957901277 >> v2:172.21.15.156:3300/0 conn(0x7fbbd80cde70 0x7fbbd815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbbc003530 con 0x7fbbd80cda90 2024-03-30T00:40:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbce000640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80cec30 0x7fbbd8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 806765125 0 0) 0x7fbbbc003530 con 0x7fbbd80cda90 2024-03-30T00:40:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80cec30 msgr2=0x7fbbd8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80cec30 0x7fbbd8159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 >> v2:172.21.15.156:3300/0 conn(0x7fbbd80cde70 msgr2=0x7fbbd815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 --2- 172.21.15.66:0/3957901277 >> v2:172.21.15.156:3300/0 conn(0x7fbbd80cde70 0x7fbbd815a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd815a990 con 0x7fbbd80cda90 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fbbc8001680 con 0x7fbbd80cda90 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 >> v1:172.21.15.66:6789/0 conn(0x7fbbd80cda90 legacy=0x7fbbd80ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80cec30 0x7fbbbc003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc003f00 0x7fbbbc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbbc004d10 0x7fbbbc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80b1b80 con 0x7fbbbc004d10 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80cd3e0 con 0x7fbbbc003f00 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc00b9e0 con 0x7fbbd80cec30 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd7400640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbbc004d10 0x7fbbbc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbce000640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc003f00 0x7fbbbc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd6a00640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80cec30 0x7fbbbc003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd7400640 1 -- 172.21.15.66:0/3957901277 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80cec30 msgr2=0x7fbbbc003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:39.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd7400640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80cec30 0x7fbbbc003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd7400640 1 -- 172.21.15.66:0/3957901277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc003f00 msgr2=0x7fbbbc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:39.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd7400640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc003f00 0x7fbbbc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.573+0000 7fbbd7400640 1 -- 172.21.15.66:0/3957901277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc003530 con 0x7fbbbc004d10 2024-03-30T00:40:39.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbd7400640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbbc004d10 0x7fbbbc005190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009bd0 tx=0x7fbbd001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c430ee51c03347d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:39.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00a5cb0 con 0x7fbbbc004d10 2024-03-30T00:40:39.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd000f040 con 0x7fbbbc004d10 2024-03-30T00:40:39.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbd6000640 1 -- 172.21.15.66:0/3957901277 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00ad680 con 0x7fbbbc004d10 2024-03-30T00:40:39.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbdde00640 1 -- 172.21.15.66:0/3957901277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbbc004d10 msgr2=0x7fbbbc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbdde00640 1 --2- 172.21.15.66:0/3957901277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbbc004d10 0x7fbbbc005190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009bd0 tx=0x7fbbd001cca0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbd7400640 1 -- 172.21.15.66:0/3957901277 reap_dead start 2024-03-30T00:40:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbdde00640 1 -- 172.21.15.66:0/3957901277 shutdown_connections 2024-03-30T00:40:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbdde00640 1 -- 172.21.15.66:0/3957901277 >> 172.21.15.66:0/3957901277 conn(0x7fbbd8069f60 msgr2=0x7fbbd806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbdde00640 1 -- 172.21.15.66:0/3957901277 shutdown_connections 2024-03-30T00:40:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbdde00640 1 -- 172.21.15.66:0/3957901277 wait complete. 2024-03-30T00:40:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbdde00640 1 Processor -- start 2024-03-30T00:40:39.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.577+0000 7fbbdde00640 1 -- start start 2024-03-30T00:40:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbdde00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc004d10 0x7fbbd80b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbdde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd80c53c0 0x7fbbd8097940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbdde00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80adac0 0x7fbbd80adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbdde00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80aeb70 con 0x7fbbd80c53c0 2024-03-30T00:40:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbdde00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80c2b10 con 0x7fbbbc004d10 2024-03-30T00:40:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbdde00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd8097e80 con 0x7fbbd80adac0 2024-03-30T00:40:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbce000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd80c53c0 0x7fbbd8097940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbce000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd80c53c0 0x7fbbd8097940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60620/0 (socket says 172.21.15.66:60620) 2024-03-30T00:40:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbce000640 1 -- 172.21.15.66:0/3438759489 learned_addr learned my addr 172.21.15.66:0/3438759489 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbce000640 1 -- 172.21.15.66:0/3438759489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80adac0 msgr2=0x7fbbd80adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbce000640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80adac0 0x7fbbd80adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbce000640 1 -- 172.21.15.66:0/3438759489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc004d10 msgr2=0x7fbbd80b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbce000640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc004d10 0x7fbbd80b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbce000640 1 -- 172.21.15.66:0/3438759489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd80ae890 con 0x7fbbd80c53c0 2024-03-30T00:40:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbce000640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd80c53c0 0x7fbbd8097940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0011960 tx=0x7fbbc0011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbcfe00640 1 -- 172.21.15.66:0/3438759489 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc000fa70 con 0x7fbbd80c53c0 2024-03-30T00:40:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbcfe00640 1 -- 172.21.15.66:0/3438759489 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc000fbd0 con 0x7fbbd80c53c0 2024-03-30T00:40:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbdde00640 1 -- 172.21.15.66:0/3438759489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd80c2d20 con 0x7fbbd80c53c0 2024-03-30T00:40:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbcfe00640 1 -- 172.21.15.66:0/3438759489 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0014440 con 0x7fbbd80c53c0 2024-03-30T00:40:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbdde00640 1 -- 172.21.15.66:0/3438759489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd81522b0 con 0x7fbbd80c53c0 2024-03-30T00:40:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbdde00640 1 -- 172.21.15.66:0/3438759489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd8066c30 con 0x7fbbd80c53c0 2024-03-30T00:40:39.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.581+0000 7fbbcfe00640 1 -- 172.21.15.66:0/3438759489 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbbc00145a0 con 0x7fbbd80c53c0 2024-03-30T00:40:39.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.585+0000 7fbbcfe00640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbbb4066690 0x7fbbb4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.585+0000 7fbbcfe00640 1 -- 172.21.15.66:0/3438759489 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fbbc0084dc0 con 0x7fbbd80c53c0 2024-03-30T00:40:39.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.585+0000 7fbbd6a00640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbbb4066690 0x7fbbb4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.585+0000 7fbbd6a00640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbbb4066690 0x7fbbb4068b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80b1d30 tx=0x7fbbc8001440 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:39.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.589+0000 7fbbcfe00640 1 -- 172.21.15.66:0/3438759489 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbbc0051d30 con 0x7fbbd80c53c0 2024-03-30T00:40:39.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 -- 172.21.15.66:0/3438759489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbd815a3b0 con 0x7fbbd80c53c0 2024-03-30T00:40:39.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbcfe00640 1 -- 172.21.15.66:0/3438759489 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7fbbc0022020 con 0x7fbbd80c53c0 2024-03-30T00:40:39.766 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:39.766 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:39.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 -- 172.21.15.66:0/3438759489 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbbb4066690 msgr2=0x7fbbb4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbbb4066690 0x7fbbb4068b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80b1d30 tx=0x7fbbc8001440 comp rx=0 tx=0).stop 2024-03-30T00:40:39.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 -- 172.21.15.66:0/3438759489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd80c53c0 msgr2=0x7fbbd8097940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd80c53c0 0x7fbbd8097940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0011960 tx=0x7fbbc0011e20 comp rx=0 tx=0).stop 2024-03-30T00:40:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 -- 172.21.15.66:0/3438759489 shutdown_connections 2024-03-30T00:40:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbd80adac0 0x7fbbd80adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbbb4066690 0x7fbbb4068b50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbd80c53c0 0x7fbbd8097940 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 --2- 172.21.15.66:0/3438759489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc004d10 0x7fbbd80b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 -- 172.21.15.66:0/3438759489 >> 172.21.15.66:0/3438759489 conn(0x7fbbd8069f60 msgr2=0x7fbbd80cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 -- 172.21.15.66:0/3438759489 shutdown_connections 2024-03-30T00:40:39.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.757+0000 7fbbdde00640 1 -- 172.21.15.66:0/3438759489 wait complete. 2024-03-30T00:40:39.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:39.788 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:39.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.921+0000 7f09ab400640 1 Processor -- start 2024-03-30T00:40:39.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.921+0000 7f09ab400640 1 -- start start 2024-03-30T00:40:39.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.921+0000 7f09ab400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f09a40cd2d0 0x7f09a40cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.921+0000 7f09ab400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a40cb620 con 0x7f09a40cad00 2024-03-30T00:40:39.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09ab400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a40ca660 con 0x7f09a40cc8d0 2024-03-30T00:40:39.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09ab400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a40cbec0 con 0x7f09a40cd2d0 2024-03-30T00:40:39.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09a3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f09a40cd2d0 0x7f09a40cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09a8a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f09a40cad00 0x7f09a40c8940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42404/0 (socket says 172.21.15.66:42404) 2024-03-30T00:40:39.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09a8a00640 1 -- 172.21.15.66:0/1018677854 learned_addr learned my addr 172.21.15.66:0/1018677854 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09a3e00640 1 -- 172.21.15.66:0/1018677854 >> v1:172.21.15.156:6789/0 conn(0x7f09a40cc8d0 legacy=0x7f09a406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09a3e00640 1 -- 172.21.15.66:0/1018677854 >> v1:172.21.15.66:6789/0 conn(0x7f09a40cad00 legacy=0x7f09a40c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09a3e00640 1 -- 172.21.15.66:0/1018677854 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a40ca940 con 0x7f09a40cd2d0 2024-03-30T00:40:39.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09a3e00640 1 --2- 172.21.15.66:0/1018677854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09a40cd2d0 0x7f09a40cb0e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f09900097c0 tx=0x7f099001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d794a59261b54178 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:39.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09a3400640 1 -- 172.21.15.66:0/1018677854 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09900a7070 con 0x7f09a40cd2d0 2024-03-30T00:40:39.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09a3400640 1 -- 172.21.15.66:0/1018677854 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09900ab430 con 0x7f09a40cd2d0 2024-03-30T00:40:39.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.925+0000 7f09a3400640 1 -- 172.21.15.66:0/1018677854 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09900ab7c0 con 0x7f09a40cd2d0 2024-03-30T00:40:39.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 -- 172.21.15.66:0/1018677854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09a40cd2d0 msgr2=0x7f09a40cb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 --2- 172.21.15.66:0/1018677854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09a40cd2d0 0x7f09a40cb0e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f09900097c0 tx=0x7f099001c430 comp rx=0 tx=0).stop 2024-03-30T00:40:39.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 -- 172.21.15.66:0/1018677854 shutdown_connections 2024-03-30T00:40:39.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 --2- 172.21.15.66:0/1018677854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09a40cd2d0 0x7f09a40cb0e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 -- 172.21.15.66:0/1018677854 >> 172.21.15.66:0/1018677854 conn(0x7f09a4069f60 msgr2=0x7f09a406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:39.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 -- 172.21.15.66:0/1018677854 shutdown_connections 2024-03-30T00:40:39.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 -- 172.21.15.66:0/1018677854 wait complete. 2024-03-30T00:40:39.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 Processor -- start 2024-03-30T00:40:39.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 -- start start 2024-03-30T00:40:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09a40cad00 0x7f09a4154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09a40cc8d0 0x7f09a415a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09a40cd2d0 0x7f09a40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40c3350 con 0x7f09a40cd2d0 2024-03-30T00:40:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4154bc0 con 0x7f09a40cc8d0 2024-03-30T00:40:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09ab400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a415abe0 con 0x7f09a40cad00 2024-03-30T00:40:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09a40cd2d0 0x7f09a40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09a40cd2d0 0x7f09a40c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60638/0 (socket says 172.21.15.66:60638) 2024-03-30T00:40:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a9400640 1 -- 172.21.15.66:0/1352476040 learned_addr learned my addr 172.21.15.66:0/1352476040 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a8a00640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09a40cad00 0x7f09a4154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a3e00640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09a40cc8d0 0x7f09a415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a9400640 1 -- 172.21.15.66:0/1352476040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09a40cad00 msgr2=0x7f09a4154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a9400640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09a40cad00 0x7f09a4154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a9400640 1 -- 172.21.15.66:0/1352476040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09a40cc8d0 msgr2=0x7f09a415a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a9400640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09a40cc8d0 0x7f09a415a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a9400640 1 -- 172.21.15.66:0/1352476040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0990009420 con 0x7f09a40cd2d0 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a3e00640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09a40cc8d0 0x7f09a415a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a8a00640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09a40cad00 0x7f09a4154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.929+0000 7f09a9400640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09a40cd2d0 0x7f09a40c5bd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f099c00c780 tx=0x7f099c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.933+0000 7f09a0c00640 1 -- 172.21.15.66:0/1352476040 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c004b70 con 0x7f09a40cd2d0 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.933+0000 7f09a0c00640 1 -- 172.21.15.66:0/1352476040 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c00ce90 con 0x7f09a40cd2d0 2024-03-30T00:40:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.933+0000 7f09a0c00640 1 -- 172.21.15.66:0/1352476040 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c00a850 con 0x7f09a40cd2d0 2024-03-30T00:40:39.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.933+0000 7f09ab400640 1 -- 172.21.15.66:0/1352476040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a415aec0 con 0x7f09a40cd2d0 2024-03-30T00:40:39.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.933+0000 7f09ab400640 1 -- 172.21.15.66:0/1352476040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a40c2f70 con 0x7f09a40cd2d0 2024-03-30T00:40:39.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.933+0000 7f09ab400640 1 -- 172.21.15.66:0/1352476040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a4072c00 con 0x7f09a40cd2d0 2024-03-30T00:40:39.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.933+0000 7f09a0c00640 1 -- 172.21.15.66:0/1352476040 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f099c00a9b0 con 0x7f09a40cd2d0 2024-03-30T00:40:39.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.933+0000 7f09a0c00640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0984066690 0x7f0984068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:39.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.933+0000 7f09a0c00640 1 -- 172.21.15.66:0/1352476040 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f099c084720 con 0x7f09a40cd2d0 2024-03-30T00:40:39.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.933+0000 7f09a8a00640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0984066690 0x7f0984068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:39.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.937+0000 7f09a8a00640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0984066690 0x7f0984068b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0998005890 tx=0x7f0998009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:39.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:39.937+0000 7f09a0c00640 1 -- 172.21.15.66:0/1352476040 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f099c051720 con 0x7f09a40cd2d0 2024-03-30T00:40:40.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 -- 172.21.15.66:0/1352476040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09a40c3490 con 0x7f09a40cd2d0 2024-03-30T00:40:40.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09a0c00640 1 -- 172.21.15.66:0/1352476040 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09a40c3490 con 0x7f09a40cd2d0 2024-03-30T00:40:40.109 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:40.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 -- 172.21.15.66:0/1352476040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0984066690 msgr2=0x7f0984068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0984066690 0x7f0984068b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0998005890 tx=0x7f0998009040 comp rx=0 tx=0).stop 2024-03-30T00:40:40.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 -- 172.21.15.66:0/1352476040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09a40cd2d0 msgr2=0x7f09a40c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09a40cd2d0 0x7f09a40c5bd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f099c00c780 tx=0x7f099c00cc40 comp rx=0 tx=0).stop 2024-03-30T00:40:40.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 -- 172.21.15.66:0/1352476040 shutdown_connections 2024-03-30T00:40:40.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0984066690 0x7f0984068b50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09a40cd2d0 0x7f09a40c5bd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09a40cc8d0 0x7f09a415a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 --2- 172.21.15.66:0/1352476040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09a40cad00 0x7f09a4154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 -- 172.21.15.66:0/1352476040 >> 172.21.15.66:0/1352476040 conn(0x7f09a4069f60 msgr2=0x7f09a406d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:40.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 -- 172.21.15.66:0/1352476040 shutdown_connections 2024-03-30T00:40:40.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.101+0000 7f09ab400640 1 -- 172.21.15.66:0/1352476040 wait complete. 2024-03-30T00:40:40.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:40.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd28d400640 1 Processor -- start 2024-03-30T00:40:40.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd28d400640 1 -- start start 2024-03-30T00:40:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd28d400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd2880cc560 0x7fd2880d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd28d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd28806e800 0x7fd2880ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd28d400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28806b0b0 con 0x7fd28806f8e0 2024-03-30T00:40:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd28d400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd288160640 con 0x7fd28806e800 2024-03-30T00:40:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd28d400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd28806b390 con 0x7fd2880cc560 2024-03-30T00:40:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd287200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd28806e800 0x7fd2880ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd285e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd28806f8e0 0x7fd28806e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42418/0 (socket says 172.21.15.66:42418) 2024-03-30T00:40:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd287200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd28806e800 0x7fd2880ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60642/0 (socket says 172.21.15.66:60642) 2024-03-30T00:40:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd285e00640 1 -- 172.21.15.66:0/2778527540 learned_addr learned my addr 172.21.15.66:0/2778527540 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd286800640 1 --2- 172.21.15.66:0/2778527540 >> v2:172.21.15.156:3300/0 conn(0x7fd2880cc560 0x7fd2880d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd287200640 1 -- 172.21.15.66:0/2778527540 >> v2:172.21.15.156:3300/0 conn(0x7fd2880cc560 msgr2=0x7fd2880d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:40.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd287200640 1 --2- 172.21.15.66:0/2778527540 >> v2:172.21.15.156:3300/0 conn(0x7fd2880cc560 0x7fd2880d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd287200640 1 -- 172.21.15.66:0/2778527540 >> v1:172.21.15.66:6789/0 conn(0x7fd28806f8e0 legacy=0x7fd28806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd287200640 1 -- 172.21.15.66:0/2778527540 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28806b670 con 0x7fd28806e800 2024-03-30T00:40:40.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.265+0000 7fd285400640 1 -- 172.21.15.66:0/2778527540 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906308608 0 0) 0x7fd28806b0b0 con 0x7fd28806f8e0 2024-03-30T00:40:40.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd287200640 1 --2- 172.21.15.66:0/2778527540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd28806e800 0x7fd2880ce8d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd288072c50 tx=0x7fd28001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5a036415c22bcf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:40.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd285400640 1 -- 172.21.15.66:0/2778527540 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2800a59c0 con 0x7fd28806e800 2024-03-30T00:40:40.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd285400640 1 -- 172.21.15.66:0/2778527540 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2800a5b20 con 0x7fd28806e800 2024-03-30T00:40:40.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd285400640 1 -- 172.21.15.66:0/2778527540 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2800ad440 con 0x7fd28806e800 2024-03-30T00:40:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd28d400640 1 -- 172.21.15.66:0/2778527540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd28806e800 msgr2=0x7fd2880ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd28d400640 1 --2- 172.21.15.66:0/2778527540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd28806e800 0x7fd2880ce8d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd288072c50 tx=0x7fd28001c900 comp rx=0 tx=0).stop 2024-03-30T00:40:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd28d400640 1 -- 172.21.15.66:0/2778527540 shutdown_connections 2024-03-30T00:40:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd28d400640 1 --2- 172.21.15.66:0/2778527540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd28806e800 0x7fd2880ce8d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd28d400640 1 --2- 172.21.15.66:0/2778527540 >> v2:172.21.15.156:3300/0 conn(0x7fd2880cc560 0x7fd2880d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd28d400640 1 -- 172.21.15.66:0/2778527540 >> 172.21.15.66:0/2778527540 conn(0x7fd2880d0540 msgr2=0x7fd2880d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd28d400640 1 -- 172.21.15.66:0/2778527540 shutdown_connections 2024-03-30T00:40:40.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd28d400640 1 -- 172.21.15.66:0/2778527540 wait complete. 2024-03-30T00:40:40.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd28d400640 1 Processor -- start 2024-03-30T00:40:40.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.269+0000 7fd28d400640 1 -- start start 2024-03-30T00:40:40.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd28d400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd28806e800 0x7fd288156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd28d400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd28806f8e0 0x7fd2880c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd28d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2880cc560 0x7fd2880c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd28d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880c9c80 con 0x7fd2880cc560 2024-03-30T00:40:40.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd28d400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880c90d0 con 0x7fd28806e800 2024-03-30T00:40:40.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd28d400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880c8420 con 0x7fd28806f8e0 2024-03-30T00:40:40.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd287200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2880cc560 0x7fd2880c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd287200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2880cc560 0x7fd2880c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60644/0 (socket says 172.21.15.66:60644) 2024-03-30T00:40:40.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd287200640 1 -- 172.21.15.66:0/3817827636 learned_addr learned my addr 172.21.15.66:0/3817827636 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:40.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd285e00640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd28806f8e0 0x7fd2880c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd286800640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd28806e800 0x7fd288156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd287200640 1 -- 172.21.15.66:0/3817827636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd28806f8e0 msgr2=0x7fd2880c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd287200640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd28806f8e0 0x7fd2880c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd287200640 1 -- 172.21.15.66:0/3817827636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd28806e800 msgr2=0x7fd288156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd287200640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd28806e800 0x7fd288156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd287200640 1 -- 172.21.15.66:0/3817827636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd280009a70 con 0x7fd2880cc560 2024-03-30T00:40:40.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd287200640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2880cc560 0x7fd2880c7610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd280019510 tx=0x7fd2800196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:40.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd27ea00640 1 -- 172.21.15.66:0/3817827636 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2800bf070 con 0x7fd2880cc560 2024-03-30T00:40:40.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd27ea00640 1 -- 172.21.15.66:0/3817827636 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280009450 con 0x7fd2880cc560 2024-03-30T00:40:40.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd27ea00640 1 -- 172.21.15.66:0/3817827636 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2800aee00 con 0x7fd2880cc560 2024-03-30T00:40:40.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd28d400640 1 -- 172.21.15.66:0/3817827636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2880c92e0 con 0x7fd2880cc560 2024-03-30T00:40:40.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd28d400640 1 -- 172.21.15.66:0/3817827636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2880c98a0 con 0x7fd2880cc560 2024-03-30T00:40:40.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd28d400640 1 -- 172.21.15.66:0/3817827636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2880c9dc0 con 0x7fd2880cc560 2024-03-30T00:40:40.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.273+0000 7fd27ea00640 1 -- 172.21.15.66:0/3817827636 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd2800b4050 con 0x7fd2880cc560 2024-03-30T00:40:40.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.277+0000 7fd27ea00640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2680666c0 0x7fd268068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.277+0000 7fd27ea00640 1 -- 172.21.15.66:0/3817827636 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fd28011f1d0 con 0x7fd2880cc560 2024-03-30T00:40:40.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.277+0000 7fd286800640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2680666c0 0x7fd268068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.277+0000 7fd286800640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2680666c0 0x7fd268068b80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd278009a50 tx=0x7fd278008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:40.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.281+0000 7fd27ea00640 1 -- 172.21.15.66:0/3817827636 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd2880c9dc0 con 0x7fd2880cc560 2024-03-30T00:40:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 -- 172.21.15.66:0/3817827636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2880c9dc0 con 0x7fd2880cc560 2024-03-30T00:40:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd27ea00640 1 -- 172.21.15.66:0/3817827636 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2880c9dc0 con 0x7fd2880cc560 2024-03-30T00:40:40.449 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:40.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 -- 172.21.15.66:0/3817827636 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2680666c0 msgr2=0x7fd268068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2680666c0 0x7fd268068b80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd278009a50 tx=0x7fd278008040 comp rx=0 tx=0).stop 2024-03-30T00:40:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 -- 172.21.15.66:0/3817827636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2880cc560 msgr2=0x7fd2880c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2880cc560 0x7fd2880c7610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd280019510 tx=0x7fd2800196a0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 -- 172.21.15.66:0/3817827636 shutdown_connections 2024-03-30T00:40:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2680666c0 0x7fd268068b80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2880cc560 0x7fd2880c7610 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd28806f8e0 0x7fd2880c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 --2- 172.21.15.66:0/3817827636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd28806e800 0x7fd288156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 -- 172.21.15.66:0/3817827636 >> 172.21.15.66:0/3817827636 conn(0x7fd2880d0540 msgr2=0x7fd2880ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 -- 172.21.15.66:0/3817827636 shutdown_connections 2024-03-30T00:40:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.441+0000 7fd28d400640 1 -- 172.21.15.66:0/3817827636 wait complete. 2024-03-30T00:40:40.470 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:40.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09a00640 1 Processor -- start 2024-03-30T00:40:40.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09a00640 1 -- start start 2024-03-30T00:40:40.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce0406cb90 0x7fce040d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fce040cdef0 0x7fce040ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce040ce880 con 0x7fce0415c6b0 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0406b1f0 con 0x7fce040cdef0 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0415d250 con 0x7fce0406cb90 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce03e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fce0415c6b0 0x7fce0415ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42432/0 (socket says 172.21.15.66:42432) 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce03e00640 1 -- 172.21.15.66:0/3709165329 learned_addr learned my addr 172.21.15.66:0/3709165329 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09000640 1 --2- 172.21.15.66:0/3709165329 >> v2:172.21.15.66:3300/0 conn(0x7fce040cdef0 0x7fce040ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fcdfbe00640 1 --2- 172.21.15.66:0/3709165329 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce0406cb90 0x7fce040d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce03400640 1 -- 172.21.15.66:0/3709165329 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165189006 0 0) 0x7fcdf4001120 con 0x7fce0415c6b0 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce03400640 1 -- 172.21.15.66:0/3709165329 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde8003530 con 0x7fce0415c6b0 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09000640 1 -- 172.21.15.66:0/3709165329 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce0406cb90 msgr2=0x7fce040d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:40.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09000640 1 --2- 172.21.15.66:0/3709165329 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce0406cb90 0x7fce040d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09000640 1 -- 172.21.15.66:0/3709165329 >> v1:172.21.15.66:6789/0 conn(0x7fce0415c6b0 legacy=0x7fce0415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.605+0000 7fce09000640 1 -- 172.21.15.66:0/3709165329 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0415d530 con 0x7fce040cdef0 2024-03-30T00:40:40.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce09000640 1 --2- 172.21.15.66:0/3709165329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce040cdef0 0x7fce040ce340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009d90 tx=0x7fcdfc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8746a22abf68f616 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:40.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce03400640 1 -- 172.21.15.66:0/3709165329 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0a5910 con 0x7fce040cdef0 2024-03-30T00:40:40.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce03400640 1 -- 172.21.15.66:0/3709165329 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc0a5a70 con 0x7fce040cdef0 2024-03-30T00:40:40.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce03400640 1 -- 172.21.15.66:0/3709165329 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0a5de0 con 0x7fce040cdef0 2024-03-30T00:40:40.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce09a00640 1 -- 172.21.15.66:0/3709165329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce040cdef0 msgr2=0x7fce040ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce09a00640 1 --2- 172.21.15.66:0/3709165329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce040cdef0 0x7fce040ce340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009d90 tx=0x7fcdfc01c940 comp rx=0 tx=0).stop 2024-03-30T00:40:40.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce09a00640 1 -- 172.21.15.66:0/3709165329 shutdown_connections 2024-03-30T00:40:40.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce09a00640 1 --2- 172.21.15.66:0/3709165329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce040cdef0 0x7fce040ce340 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce09a00640 1 --2- 172.21.15.66:0/3709165329 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce0406cb90 0x7fce040d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce09a00640 1 -- 172.21.15.66:0/3709165329 >> 172.21.15.66:0/3709165329 conn(0x7fce04069d90 msgr2=0x7fce0406a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:40.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce09a00640 1 -- 172.21.15.66:0/3709165329 shutdown_connections 2024-03-30T00:40:40.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.609+0000 7fce09a00640 1 -- 172.21.15.66:0/3709165329 wait complete. 2024-03-30T00:40:40.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09a00640 1 Processor -- start 2024-03-30T00:40:40.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09a00640 1 -- start start 2024-03-30T00:40:40.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce0406cb90 0x7fce04152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce040cdef0 0x7fce040b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce0415c6b0 0x7fce0406bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce040b4290 con 0x7fce040cdef0 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce040b35d0 con 0x7fce0406cb90 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce04153030 con 0x7fce0415c6b0 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fcdfbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce040cdef0 0x7fce040b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fcdfbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce040cdef0 0x7fce040b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60666/0 (socket says 172.21.15.66:60666) 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fcdfbe00640 1 -- 172.21.15.66:0/1799651525 learned_addr learned my addr 172.21.15.66:0/1799651525 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09000640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce0415c6b0 0x7fce0406bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce03e00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce0406cb90 0x7fce04152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fcdfbe00640 1 -- 172.21.15.66:0/1799651525 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce0415c6b0 msgr2=0x7fce0406bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fcdfbe00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce0415c6b0 0x7fce0406bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fcdfbe00640 1 -- 172.21.15.66:0/1799651525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce0406cb90 msgr2=0x7fce04152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fcdfbe00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce0406cb90 0x7fce04152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fcdfbe00640 1 -- 172.21.15.66:0/1799651525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc009a70 con 0x7fce040cdef0 2024-03-30T00:40:40.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09000640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce0415c6b0 0x7fce0406bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:40.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fcdfbe00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce040cdef0 0x7fce040b2df0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00e960 tx=0x7fcdec00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:40.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce00c00640 1 -- 172.21.15.66:0/1799651525 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec009690 con 0x7fce040cdef0 2024-03-30T00:40:40.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce00c00640 1 -- 172.21.15.66:0/1799651525 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec00ca00 con 0x7fce040cdef0 2024-03-30T00:40:40.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce00c00640 1 -- 172.21.15.66:0/1799651525 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00cd70 con 0x7fce040cdef0 2024-03-30T00:40:40.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09a00640 1 -- 172.21.15.66:0/1799651525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce040b3330 con 0x7fce040cdef0 2024-03-30T00:40:40.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.613+0000 7fce09a00640 1 -- 172.21.15.66:0/1799651525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce040b3bd0 con 0x7fce040cdef0 2024-03-30T00:40:40.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.617+0000 7fce09a00640 1 -- 172.21.15.66:0/1799651525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce04073e00 con 0x7fce040cdef0 2024-03-30T00:40:40.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.617+0000 7fce00c00640 1 -- 172.21.15.66:0/1799651525 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcdec013470 con 0x7fce040cdef0 2024-03-30T00:40:40.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.617+0000 7fce00c00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcde00666e0 0x7fcde0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.617+0000 7fce00c00640 1 -- 172.21.15.66:0/1799651525 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fcdec084710 con 0x7fce040cdef0 2024-03-30T00:40:40.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.617+0000 7fce03e00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcde00666e0 0x7fcde0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.617+0000 7fce03e00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcde00666e0 0x7fcde0068ba0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4005890 tx=0x7fcdf4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:40.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.625+0000 7fce00c00640 1 -- 172.21.15.66:0/1799651525 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcdec0516f0 con 0x7fce040cdef0 2024-03-30T00:40:40.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.781+0000 7fce09a00640 1 -- 172.21.15.66:0/1799651525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce040b62a0 con 0x7fce040cdef0 2024-03-30T00:40:40.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce00c00640 1 -- 172.21.15.66:0/1799651525 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcdec054d10 con 0x7fce040cdef0 2024-03-30T00:40:40.793 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 -- 172.21.15.66:0/1799651525 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcde00666e0 msgr2=0x7fcde0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcde00666e0 0x7fcde0068ba0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4005890 tx=0x7fcdf4009040 comp rx=0 tx=0).stop 2024-03-30T00:40:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 -- 172.21.15.66:0/1799651525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce040cdef0 msgr2=0x7fce040b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce040cdef0 0x7fce040b2df0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00e960 tx=0x7fcdec00ee20 comp rx=0 tx=0).stop 2024-03-30T00:40:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 -- 172.21.15.66:0/1799651525 shutdown_connections 2024-03-30T00:40:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce0415c6b0 0x7fce0406bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcde00666e0 0x7fcde0068ba0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce040cdef0 0x7fce040b2df0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 --2- 172.21.15.66:0/1799651525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce0406cb90 0x7fce04152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 -- 172.21.15.66:0/1799651525 >> 172.21.15.66:0/1799651525 conn(0x7fce04069d90 msgr2=0x7fce040b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 -- 172.21.15.66:0/1799651525 shutdown_connections 2024-03-30T00:40:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.785+0000 7fce09a00640 1 -- 172.21.15.66:0/1799651525 wait complete. 2024-03-30T00:40:40.815 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:40.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.949+0000 7f4496c00640 1 Processor -- start 2024-03-30T00:40:40.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.949+0000 7f4496c00640 1 -- start start 2024-03-30T00:40:40.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.949+0000 7f4496c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44900cf2f0 0x7f449006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.949+0000 7f4496c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f44900cb570 0x7f44900cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.949+0000 7f4496c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44900cb0e0 con 0x7f449015ca80 2024-03-30T00:40:40.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.949+0000 7f4496c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44900cf7f0 con 0x7f44900cf2f0 2024-03-30T00:40:40.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.949+0000 7f4496c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44900cfad0 con 0x7f44900cb570 2024-03-30T00:40:40.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.949+0000 7f4487400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44900cf2f0 0x7f449006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.949+0000 7f4487400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44900cf2f0 0x7f449006b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60674/0 (socket says 172.21.15.66:60674) 2024-03-30T00:40:40.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.949+0000 7f4487400640 1 -- 172.21.15.66:0/2755594768 learned_addr learned my addr 172.21.15.66:0/2755594768 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:40.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.953+0000 7f4494c00640 1 --2- 172.21.15.66:0/2755594768 >> v2:172.21.15.156:3300/0 conn(0x7f44900cb570 0x7f44900cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.953+0000 7f4487400640 1 -- 172.21.15.66:0/2755594768 >> v2:172.21.15.156:3300/0 conn(0x7f44900cb570 msgr2=0x7f44900cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:40.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.953+0000 7f4487400640 1 --2- 172.21.15.66:0/2755594768 >> v2:172.21.15.156:3300/0 conn(0x7f44900cb570 0x7f44900cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.953+0000 7f4487400640 1 -- 172.21.15.66:0/2755594768 >> v1:172.21.15.156:6789/0 conn(0x7f449015ca80 legacy=0x7f449006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.953+0000 7f4487400640 1 -- 172.21.15.66:0/2755594768 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44900cad70 con 0x7f44900cf2f0 2024-03-30T00:40:40.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.953+0000 7f4487400640 1 --2- 172.21.15.66:0/2755594768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44900cf2f0 0x7f449006b7c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4490071d80 tx=0x7f447801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f5b3427fc33c09 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:40.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.953+0000 7f448f400640 1 -- 172.21.15.66:0/2755594768 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780a8a10 con 0x7f44900cf2f0 2024-03-30T00:40:40.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.953+0000 7f448f400640 1 -- 172.21.15.66:0/2755594768 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44780a8b70 con 0x7f44900cf2f0 2024-03-30T00:40:40.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.953+0000 7f448f400640 1 -- 172.21.15.66:0/2755594768 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780b0440 con 0x7f44900cf2f0 2024-03-30T00:40:40.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.953+0000 7f4496c00640 1 -- 172.21.15.66:0/2755594768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44900cf2f0 msgr2=0x7f449006b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.953+0000 7f4496c00640 1 --2- 172.21.15.66:0/2755594768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44900cf2f0 0x7f449006b7c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4490071d80 tx=0x7f447801ed10 comp rx=0 tx=0).stop 2024-03-30T00:40:40.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 -- 172.21.15.66:0/2755594768 shutdown_connections 2024-03-30T00:40:40.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 --2- 172.21.15.66:0/2755594768 >> v2:172.21.15.156:3300/0 conn(0x7f44900cb570 0x7f44900cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 --2- 172.21.15.66:0/2755594768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44900cf2f0 0x7f449006b7c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 -- 172.21.15.66:0/2755594768 >> 172.21.15.66:0/2755594768 conn(0x7f44900b2090 msgr2=0x7f44900b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:40.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 -- 172.21.15.66:0/2755594768 shutdown_connections 2024-03-30T00:40:40.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 -- 172.21.15.66:0/2755594768 wait complete. 2024-03-30T00:40:40.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 Processor -- start 2024-03-30T00:40:40.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 -- start start 2024-03-30T00:40:40.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44900cb570 0x7f4490153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44900cf2f0 0x7f44900c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449015ca80 0x7f44900c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449015ff80 con 0x7f44900cf2f0 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449015f300 con 0x7f449015ca80 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449015f5e0 con 0x7f44900cb570 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4487400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44900cf2f0 0x7f44900c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4487400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44900cf2f0 0x7f44900c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60680/0 (socket says 172.21.15.66:60680) 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4487400640 1 -- 172.21.15.66:0/4155624225 learned_addr learned my addr 172.21.15.66:0/4155624225 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f448fe00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44900cb570 0x7f4490153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4494c00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449015ca80 0x7f44900c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4487400640 1 -- 172.21.15.66:0/4155624225 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44900cb570 msgr2=0x7f4490153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4487400640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44900cb570 0x7f4490153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4487400640 1 -- 172.21.15.66:0/4155624225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449015ca80 msgr2=0x7f44900c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:40.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4487400640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449015ca80 0x7f44900c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:40.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4487400640 1 -- 172.21.15.66:0/4155624225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780094d0 con 0x7f44900cf2f0 2024-03-30T00:40:40.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4494c00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449015ca80 0x7f44900c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:40.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4487400640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44900cf2f0 0x7f44900c58d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4478000c10 tx=0x7f447801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:40.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f448cc00640 1 -- 172.21.15.66:0/4155624225 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780b1de0 con 0x7f44900cf2f0 2024-03-30T00:40:40.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f448cc00640 1 -- 172.21.15.66:0/4155624225 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44780b9410 con 0x7f44900cf2f0 2024-03-30T00:40:40.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f448cc00640 1 -- 172.21.15.66:0/4155624225 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780b97c0 con 0x7f44900cf2f0 2024-03-30T00:40:40.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.957+0000 7f4496c00640 1 -- 172.21.15.66:0/4155624225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449015f8c0 con 0x7f44900cf2f0 2024-03-30T00:40:40.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.961+0000 7f4496c00640 1 -- 172.21.15.66:0/4155624225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44900b4ae0 con 0x7f44900cf2f0 2024-03-30T00:40:40.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.961+0000 7f4496c00640 1 -- 172.21.15.66:0/4155624225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44901600c0 con 0x7f44900cf2f0 2024-03-30T00:40:40.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.961+0000 7f448cc00640 1 -- 172.21.15.66:0/4155624225 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f44780b7040 con 0x7f44900cf2f0 2024-03-30T00:40:40.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.961+0000 7f448cc00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f446c066690 0x7f446c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:40.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.961+0000 7f448cc00640 1 -- 172.21.15.66:0/4155624225 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f4478122020 con 0x7f44900cf2f0 2024-03-30T00:40:40.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.961+0000 7f448fe00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f446c066690 0x7f446c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:40.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.961+0000 7f448fe00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f446c066690 0x7f446c068b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f44800015b0 tx=0x7f4480009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:40.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:40.965+0000 7f448cc00640 1 -- 172.21.15.66:0/4155624225 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f44901600c0 con 0x7f44900cf2f0 2024-03-30T00:40:41.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.125+0000 7f4496c00640 1 -- 172.21.15.66:0/4155624225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f44901600c0 con 0x7f44900cf2f0 2024-03-30T00:40:41.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.125+0000 7f448cc00640 1 -- 172.21.15.66:0/4155624225 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f44901600c0 con 0x7f44900cf2f0 2024-03-30T00:40:41.135 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:41.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.125+0000 7f4496c00640 1 -- 172.21.15.66:0/4155624225 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f446c066690 msgr2=0x7f446c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.125+0000 7f4496c00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f446c066690 0x7f446c068b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f44800015b0 tx=0x7f4480009040 comp rx=0 tx=0).stop 2024-03-30T00:40:41.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.125+0000 7f4496c00640 1 -- 172.21.15.66:0/4155624225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44900cf2f0 msgr2=0x7f44900c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.125+0000 7f4496c00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44900cf2f0 0x7f44900c58d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4478000c10 tx=0x7f447801bab0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.129+0000 7f4496c00640 1 -- 172.21.15.66:0/4155624225 shutdown_connections 2024-03-30T00:40:41.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.129+0000 7f4496c00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449015ca80 0x7f44900c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.129+0000 7f4496c00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f446c066690 0x7f446c068b50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.129+0000 7f4496c00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44900cf2f0 0x7f44900c58d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.129+0000 7f4496c00640 1 --2- 172.21.15.66:0/4155624225 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44900cb570 0x7f4490153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.129+0000 7f4496c00640 1 -- 172.21.15.66:0/4155624225 >> 172.21.15.66:0/4155624225 conn(0x7f44900b2090 msgr2=0x7f449006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:41.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.129+0000 7f4496c00640 1 -- 172.21.15.66:0/4155624225 shutdown_connections 2024-03-30T00:40:41.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.129+0000 7f4496c00640 1 -- 172.21.15.66:0/4155624225 wait complete. 2024-03-30T00:40:41.156 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:41.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0baa00640 1 Processor -- start 2024-03-30T00:40:41.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0baa00640 1 -- start start 2024-03-30T00:40:41.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0baa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff0b415a640 0x7ff0b406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0baa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0b40ce520 0x7ff0b40ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0baa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b406d070 con 0x7ff0b40cabb0 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0baa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b40cbba0 con 0x7ff0b415a640 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0baa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b406e2c0 con 0x7ff0b40ce520 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0b3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff0b415a640 0x7ff0b406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0b3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff0b415a640 0x7ff0b406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60690/0 (socket says 172.21.15.66:60690) 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0b3e00640 1 -- 172.21.15.66:0/1339198974 learned_addr learned my addr 172.21.15.66:0/1339198974 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0b8a00640 1 --1- 172.21.15.66:0/1339198974 >> v1:172.21.15.66:6789/0 conn(0x7ff0b40cabb0 0x7ff0b40cb000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42434/0 (socket says 172.21.15.66:42434) 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0ab400640 1 --2- 172.21.15.66:0/1339198974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0b40ce520 0x7ff0b40ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0b3e00640 1 -- 172.21.15.66:0/1339198974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0b40ce520 msgr2=0x7ff0b40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0b3e00640 1 --2- 172.21.15.66:0/1339198974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0b40ce520 0x7ff0b40ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0b3e00640 1 -- 172.21.15.66:0/1339198974 >> v1:172.21.15.66:6789/0 conn(0x7ff0b40cabb0 legacy=0x7ff0b40cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0b3e00640 1 -- 172.21.15.66:0/1339198974 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b40cee60 con 0x7ff0b415a640 2024-03-30T00:40:41.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.289+0000 7ff0b3400640 1 -- 172.21.15.66:0/1339198974 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700808940 0 0) 0x7ff0b406d070 con 0x7ff0b40cabb0 2024-03-30T00:40:41.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0b3e00640 1 --2- 172.21.15.66:0/1339198974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b415a640 0x7ff0b406dd80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009870 tx=0x7ff0a401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948e4562bea41b8b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:41.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0b3400640 1 -- 172.21.15.66:0/1339198974 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40ab650 con 0x7ff0b415a640 2024-03-30T00:40:41.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0b3400640 1 -- 172.21.15.66:0/1339198974 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a40ab7b0 con 0x7ff0b415a640 2024-03-30T00:40:41.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0b3400640 1 -- 172.21.15.66:0/1339198974 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40abb40 con 0x7ff0b415a640 2024-03-30T00:40:41.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0baa00640 1 -- 172.21.15.66:0/1339198974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b415a640 msgr2=0x7ff0b406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0baa00640 1 --2- 172.21.15.66:0/1339198974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b415a640 0x7ff0b406dd80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009870 tx=0x7ff0a401c560 comp rx=0 tx=0).stop 2024-03-30T00:40:41.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0baa00640 1 -- 172.21.15.66:0/1339198974 shutdown_connections 2024-03-30T00:40:41.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0baa00640 1 --2- 172.21.15.66:0/1339198974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0b40ce520 0x7ff0b40ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0baa00640 1 --2- 172.21.15.66:0/1339198974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b415a640 0x7ff0b406dd80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0baa00640 1 -- 172.21.15.66:0/1339198974 >> 172.21.15.66:0/1339198974 conn(0x7ff0b4069f60 msgr2=0x7ff0b406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:41.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0baa00640 1 -- 172.21.15.66:0/1339198974 shutdown_connections 2024-03-30T00:40:41.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.293+0000 7ff0baa00640 1 -- 172.21.15.66:0/1339198974 wait complete. 2024-03-30T00:40:41.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0baa00640 1 Processor -- start 2024-03-30T00:40:41.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0baa00640 1 -- start start 2024-03-30T00:40:41.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0baa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b40cabb0 0x7ff0b40cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0baa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b40ce520 0x7ff0b4071800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0baa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0b415a640 0x7ff0b40aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0baa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40c88f0 con 0x7ff0b40cabb0 2024-03-30T00:40:41.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0baa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40cd0d0 con 0x7ff0b40ce520 2024-03-30T00:40:41.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0baa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40cdea0 con 0x7ff0b415a640 2024-03-30T00:40:41.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b40cabb0 0x7ff0b40cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b40cabb0 0x7ff0b40cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60700/0 (socket says 172.21.15.66:60700) 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b3e00640 1 -- 172.21.15.66:0/1500867724 learned_addr learned my addr 172.21.15.66:0/1500867724 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b3e00640 1 -- 172.21.15.66:0/1500867724 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0b415a640 msgr2=0x7ff0b40aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b3e00640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0b415a640 0x7ff0b40aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b3e00640 1 -- 172.21.15.66:0/1500867724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b40ce520 msgr2=0x7ff0b4071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b3e00640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b40ce520 0x7ff0b4071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b3e00640 1 -- 172.21.15.66:0/1500867724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a40094d0 con 0x7ff0b40cabb0 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b3e00640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b40cabb0 0x7ff0b40cd960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009870 tx=0x7ff0a401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b0c00640 1 -- 172.21.15.66:0/1500867724 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40abdf0 con 0x7ff0b40cabb0 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b0c00640 1 -- 172.21.15.66:0/1500867724 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a40b8070 con 0x7ff0b40cabb0 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0b0c00640 1 -- 172.21.15.66:0/1500867724 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4004230 con 0x7ff0b40cabb0 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0baa00640 1 -- 172.21.15.66:0/1500867724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b40c1e00 con 0x7ff0b40cabb0 2024-03-30T00:40:41.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0baa00640 1 -- 172.21.15.66:0/1500867724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b40c8b40 con 0x7ff0b40cabb0 2024-03-30T00:40:41.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.297+0000 7ff0baa00640 1 -- 172.21.15.66:0/1500867724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b4066c30 con 0x7ff0b40cabb0 2024-03-30T00:40:41.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.301+0000 7ff0b0c00640 1 -- 172.21.15.66:0/1500867724 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff0a4004390 con 0x7ff0b40cabb0 2024-03-30T00:40:41.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.301+0000 7ff0b0c00640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0900666c0 0x7ff090068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.301+0000 7ff0b0c00640 1 -- 172.21.15.66:0/1500867724 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff0a411e6d0 con 0x7ff0b40cabb0 2024-03-30T00:40:41.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.301+0000 7ff0ab400640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0900666c0 0x7ff090068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.301+0000 7ff0ab400640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0900666c0 0x7ff090068b80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40720d0 tx=0x7ff09c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:41.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.305+0000 7ff0b0c00640 1 -- 172.21.15.66:0/1500867724 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff0a40eb5a0 con 0x7ff0b40cabb0 2024-03-30T00:40:41.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.469+0000 7ff0baa00640 1 -- 172.21.15.66:0/1500867724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0b406a3e0 con 0x7ff0b40cabb0 2024-03-30T00:40:41.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0b0c00640 1 -- 172.21.15.66:0/1500867724 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7ff0a40eebc0 con 0x7ff0b40cabb0 2024-03-30T00:40:41.481 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:41.481 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:41.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 -- 172.21.15.66:0/1500867724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0900666c0 msgr2=0x7ff090068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0900666c0 0x7ff090068b80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40720d0 tx=0x7ff09c008040 comp rx=0 tx=0).stop 2024-03-30T00:40:41.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 -- 172.21.15.66:0/1500867724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b40cabb0 msgr2=0x7ff0b40cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b40cabb0 0x7ff0b40cd960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009870 tx=0x7ff0a401ada0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 -- 172.21.15.66:0/1500867724 shutdown_connections 2024-03-30T00:40:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0b415a640 0x7ff0b40aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b40ce520 0x7ff0b4071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0900666c0 0x7ff090068b80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 --2- 172.21.15.66:0/1500867724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b40cabb0 0x7ff0b40cd960 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 -- 172.21.15.66:0/1500867724 >> 172.21.15.66:0/1500867724 conn(0x7ff0b4069f60 msgr2=0x7ff0b40b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 -- 172.21.15.66:0/1500867724 shutdown_connections 2024-03-30T00:40:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.473+0000 7ff0baa00640 1 -- 172.21.15.66:0/1500867724 wait complete. 2024-03-30T00:40:41.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:41.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:41.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f161cc00640 1 Processor -- start 2024-03-30T00:40:41.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f161cc00640 1 -- start start 2024-03-30T00:40:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f161cc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161815a810 0x7f16180ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f161cc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f16180cf290 0x7f16180c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f161cc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16180c4600 con 0x7f1618099430 2024-03-30T00:40:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f161cc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16180cc510 con 0x7f16180cf290 2024-03-30T00:40:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f161cc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16180998c0 con 0x7f161815a810 2024-03-30T00:40:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f1616800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f16180cf290 0x7f16180c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f1615e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161815a810 0x7f16180ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f1616800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f16180cf290 0x7f16180c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:58674/0 (socket says 172.21.15.66:58674) 2024-03-30T00:40:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f1616800640 1 -- 172.21.15.66:0/2091310216 learned_addr learned my addr 172.21.15.66:0/2091310216 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:41.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f1616800640 1 -- 172.21.15.66:0/2091310216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161815a810 msgr2=0x7f16180ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:41.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f1616800640 1 --2- 172.21.15.66:0/2091310216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161815a810 0x7f16180ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f1616800640 1 -- 172.21.15.66:0/2091310216 >> v1:172.21.15.156:6789/0 conn(0x7f1618099430 legacy=0x7f16180ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.637+0000 7f1616800640 1 -- 172.21.15.66:0/2091310216 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1618099ba0 con 0x7f16180cf290 2024-03-30T00:40:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f1616800640 1 --2- 172.21.15.66:0/2091310216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16180cf290 0x7f16180c4a90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1610009870 tx=0x7f161001ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed957a712ae0fd80 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f1615400640 1 -- 172.21.15.66:0/2091310216 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16100a8a50 con 0x7f16180cf290 2024-03-30T00:40:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f1615400640 1 -- 172.21.15.66:0/2091310216 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16100a8bb0 con 0x7f16180cf290 2024-03-30T00:40:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f1615400640 1 -- 172.21.15.66:0/2091310216 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16100b0440 con 0x7f16180cf290 2024-03-30T00:40:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f161cc00640 1 -- 172.21.15.66:0/2091310216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16180cf290 msgr2=0x7f16180c4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f161cc00640 1 --2- 172.21.15.66:0/2091310216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16180cf290 0x7f16180c4a90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1610009870 tx=0x7f161001ed50 comp rx=0 tx=0).stop 2024-03-30T00:40:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f161cc00640 1 -- 172.21.15.66:0/2091310216 shutdown_connections 2024-03-30T00:40:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f161cc00640 1 --2- 172.21.15.66:0/2091310216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16180cf290 0x7f16180c4a90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f161cc00640 1 --2- 172.21.15.66:0/2091310216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161815a810 0x7f16180ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f161cc00640 1 -- 172.21.15.66:0/2091310216 >> 172.21.15.66:0/2091310216 conn(0x7f16180b2090 msgr2=0x7f16180b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:41.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f161cc00640 1 -- 172.21.15.66:0/2091310216 shutdown_connections 2024-03-30T00:40:41.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.641+0000 7f161cc00640 1 -- 172.21.15.66:0/2091310216 wait complete. 2024-03-30T00:40:41.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f161cc00640 1 Processor -- start 2024-03-30T00:40:41.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f161cc00640 1 -- start start 2024-03-30T00:40:41.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f161cc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16180cf290 0x7f161815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f161cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161815a810 0x7f16180c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f161cc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16180c7bb0 0x7f16180c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f161cc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16181f3740 con 0x7f161815a810 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f161cc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16181f33e0 con 0x7f16180c7bb0 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f161cc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16180c8130 con 0x7f16180cf290 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161815a810 0x7f16180c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161815a810 0x7f16180c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60704/0 (socket says 172.21.15.66:60704) 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f1615e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16180cf290 0x7f161815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160d400640 1 -- 172.21.15.66:0/611243475 learned_addr learned my addr 172.21.15.66:0/611243475 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f1616800640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16180c7bb0 0x7f16180c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160d400640 1 -- 172.21.15.66:0/611243475 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16180cf290 msgr2=0x7f161815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160d400640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16180cf290 0x7f161815db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160d400640 1 -- 172.21.15.66:0/611243475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16180c7bb0 msgr2=0x7f16180c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160d400640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16180c7bb0 0x7f16180c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160d400640 1 -- 172.21.15.66:0/611243475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16100094d0 con 0x7f161815a810 2024-03-30T00:40:41.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160d400640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161815a810 0x7f16180c7670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f160000c500 tx=0x7f160000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:41.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160ea00640 1 -- 172.21.15.66:0/611243475 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600005220 con 0x7f161815a810 2024-03-30T00:40:41.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160ea00640 1 -- 172.21.15.66:0/611243475 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1600005380 con 0x7f161815a810 2024-03-30T00:40:41.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f160ea00640 1 -- 172.21.15.66:0/611243475 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600007270 con 0x7f161815a810 2024-03-30T00:40:41.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f161cc00640 1 -- 172.21.15.66:0/611243475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16180c8410 con 0x7f161815a810 2024-03-30T00:40:41.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.645+0000 7f161cc00640 1 -- 172.21.15.66:0/611243475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16181f2590 con 0x7f161815a810 2024-03-30T00:40:41.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.649+0000 7f161cc00640 1 -- 172.21.15.66:0/611243475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16181f3880 con 0x7f161815a810 2024-03-30T00:40:41.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.649+0000 7f160ea00640 1 -- 172.21.15.66:0/611243475 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f16000073d0 con 0x7f161815a810 2024-03-30T00:40:41.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.649+0000 7f160ea00640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15f40666c0 0x7f15f4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.649+0000 7f160ea00640 1 -- 172.21.15.66:0/611243475 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f1600083d70 con 0x7f161815a810 2024-03-30T00:40:41.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.649+0000 7f1615e00640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15f40666c0 0x7f15f4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.649+0000 7f1615e00640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15f40666c0 0x7f15f4068b80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16180c60c0 tx=0x7f160800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:41.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.653+0000 7f160ea00640 1 -- 172.21.15.66:0/611243475 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f16181f3880 con 0x7f161815a810 2024-03-30T00:40:41.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.813+0000 7f161cc00640 1 -- 172.21.15.66:0/611243475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16181f3880 con 0x7f161815a810 2024-03-30T00:40:41.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.813+0000 7f160ea00640 1 -- 172.21.15.66:0/611243475 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16181f3880 con 0x7f161815a810 2024-03-30T00:40:41.825 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 -- 172.21.15.66:0/611243475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15f40666c0 msgr2=0x7f15f4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15f40666c0 0x7f15f4068b80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16180c60c0 tx=0x7f160800b040 comp rx=0 tx=0).stop 2024-03-30T00:40:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 -- 172.21.15.66:0/611243475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161815a810 msgr2=0x7f16180c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161815a810 0x7f16180c7670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f160000c500 tx=0x7f160000c9c0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 -- 172.21.15.66:0/611243475 shutdown_connections 2024-03-30T00:40:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16180c7bb0 0x7f16180c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15f40666c0 0x7f15f4068b80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f161815a810 0x7f16180c7670 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 --2- 172.21.15.66:0/611243475 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16180cf290 0x7f161815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 -- 172.21.15.66:0/611243475 >> 172.21.15.66:0/611243475 conn(0x7f16180b2090 msgr2=0x7f16180b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 -- 172.21.15.66:0/611243475 shutdown_connections 2024-03-30T00:40:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.817+0000 7f161cc00640 1 -- 172.21.15.66:0/611243475 wait complete. 2024-03-30T00:40:41.847 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:41.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77fda00640 1 Processor -- start 2024-03-30T00:40:41.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77fda00640 1 -- start start 2024-03-30T00:40:41.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77fda00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f77f80cda90 0x7f77f80ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77fda00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77f80cde70 0x7f77f80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77fda00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f77f8159c60 0x7f77f815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77fda00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f80b1b80 con 0x7f77f8159c60 2024-03-30T00:40:41.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77fda00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f815a990 con 0x7f77f80cda90 2024-03-30T00:40:41.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77fda00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80b11e0 con 0x7f77f80cde70 2024-03-30T00:40:41.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77f7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f77f8159c60 0x7f77f815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77f7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f77f8159c60 0x7f77f815a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60712/0 (socket says 172.21.15.66:60712) 2024-03-30T00:40:41.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77f7400640 1 -- 172.21.15.66:0/1459535623 learned_addr learned my addr 172.21.15.66:0/1459535623 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:41.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77efe00640 1 --2- 172.21.15.66:0/1459535623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77f80cde70 0x7f77f80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77f6a00640 1 --2- 172.21.15.66:0/1459535623 >> v2:172.21.15.156:3300/0 conn(0x7f77f80cda90 0x7f77f80ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:41.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77f7400640 1 -- 172.21.15.66:0/1459535623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77f80cde70 msgr2=0x7f77f80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:41.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77f7400640 1 --2- 172.21.15.66:0/1459535623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77f80cde70 0x7f77f80cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77f7400640 1 -- 172.21.15.66:0/1459535623 >> v2:172.21.15.156:3300/0 conn(0x7f77f80cda90 msgr2=0x7f77f80ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:41.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77f7400640 1 --2- 172.21.15.66:0/1459535623 >> v2:172.21.15.156:3300/0 conn(0x7f77f80cda90 0x7f77f80ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77f7400640 1 -- 172.21.15.66:0/1459535623 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f80b14c0 con 0x7f77f8159c60 2024-03-30T00:40:41.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.981+0000 7f77efe00640 1 --2- 172.21.15.66:0/1459535623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77f80cde70 0x7f77f80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77f7400640 1 --2- 172.21.15.66:0/1459535623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77f8159c60 0x7f77f815a0b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f77f80cf3f0 tx=0x7f77f001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf65948cee6790d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77f6000640 1 -- 172.21.15.66:0/1459535623 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f00a5800 con 0x7f77f8159c60 2024-03-30T00:40:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77f6000640 1 -- 172.21.15.66:0/1459535623 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f00a5960 con 0x7f77f8159c60 2024-03-30T00:40:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77f6000640 1 -- 172.21.15.66:0/1459535623 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f00a5cf0 con 0x7f77f8159c60 2024-03-30T00:40:41.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77fda00640 1 -- 172.21.15.66:0/1459535623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77f8159c60 msgr2=0x7f77f815a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:41.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77fda00640 1 --2- 172.21.15.66:0/1459535623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77f8159c60 0x7f77f815a0b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f77f80cf3f0 tx=0x7f77f001c960 comp rx=0 tx=0).stop 2024-03-30T00:40:41.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77fda00640 1 -- 172.21.15.66:0/1459535623 shutdown_connections 2024-03-30T00:40:41.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77fda00640 1 --2- 172.21.15.66:0/1459535623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77f8159c60 0x7f77f815a0b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77fda00640 1 --2- 172.21.15.66:0/1459535623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77f80cde70 0x7f77f80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77fda00640 1 --2- 172.21.15.66:0/1459535623 >> v2:172.21.15.156:3300/0 conn(0x7f77f80cda90 0x7f77f80ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:41.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77fda00640 1 -- 172.21.15.66:0/1459535623 >> 172.21.15.66:0/1459535623 conn(0x7f77f8069f60 msgr2=0x7f77f806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:41.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77fda00640 1 -- 172.21.15.66:0/1459535623 shutdown_connections 2024-03-30T00:40:41.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.985+0000 7f77fda00640 1 -- 172.21.15.66:0/1459535623 wait complete. 2024-03-30T00:40:41.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77fda00640 1 Processor -- start 2024-03-30T00:40:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77fda00640 1 -- start start 2024-03-30T00:40:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77fda00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77f80cda90 0x7f77f809b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77fda00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77f80cde70 0x7f77f8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77fda00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77f8159c60 0x7f77f80ac670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77fda00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80acbb0 con 0x7f77f8159c60 2024-03-30T00:40:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77fda00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80afb70 con 0x7f77f80cda90 2024-03-30T00:40:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77fda00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80acde0 con 0x7f77f80cde70 2024-03-30T00:40:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77f7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77f8159c60 0x7f77f80ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77f7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77f8159c60 0x7f77f80ac670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60728/0 (socket says 172.21.15.66:60728) 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77f7400640 1 -- 172.21.15.66:0/3017703263 learned_addr learned my addr 172.21.15.66:0/3017703263 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77f6a00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77f80cda90 0x7f77f809b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77efe00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77f80cde70 0x7f77f8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77f7400640 1 -- 172.21.15.66:0/3017703263 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77f80cde70 msgr2=0x7f77f8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77f7400640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77f80cde70 0x7f77f8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77f7400640 1 -- 172.21.15.66:0/3017703263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77f80cda90 msgr2=0x7f77f809b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77f7400640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77f80cda90 0x7f77f809b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77f7400640 1 -- 172.21.15.66:0/3017703263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f0009b30 con 0x7f77f8159c60 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77f6a00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77f80cda90 0x7f77f809b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77f7400640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77f8159c60 0x7f77f80ac670 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f77f00af040 tx=0x7f77f0019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77eea00640 1 -- 172.21.15.66:0/3017703263 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f00b8670 con 0x7f77f8159c60 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77eea00640 1 -- 172.21.15.66:0/3017703263 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f00b87d0 con 0x7f77f8159c60 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77eea00640 1 -- 172.21.15.66:0/3017703263 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f00b8b60 con 0x7f77f8159c60 2024-03-30T00:40:42.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77fda00640 1 -- 172.21.15.66:0/3017703263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f8070050 con 0x7f77f8159c60 2024-03-30T00:40:42.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77fda00640 1 -- 172.21.15.66:0/3017703263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f8070610 con 0x7f77f8159c60 2024-03-30T00:40:42.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.989+0000 7f77fda00640 1 -- 172.21.15.66:0/3017703263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f8066c30 con 0x7f77f8159c60 2024-03-30T00:40:42.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.993+0000 7f77eea00640 1 -- 172.21.15.66:0/3017703263 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f77f00b8cc0 con 0x7f77f8159c60 2024-03-30T00:40:42.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.993+0000 7f77eea00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77d4066690 0x7f77d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.993+0000 7f77eea00640 1 -- 172.21.15.66:0/3017703263 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f77f011faa0 con 0x7f77f8159c60 2024-03-30T00:40:42.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.993+0000 7f77f6a00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77d4066690 0x7f77d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.993+0000 7f77f6a00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77d4066690 0x7f77d4068b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f77e800ba50 tx=0x7f77e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:41.997+0000 7f77eea00640 1 -- 172.21.15.66:0/3017703263 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f77f00ecac0 con 0x7f77f8159c60 2024-03-30T00:40:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.153+0000 7f77fda00640 1 -- 172.21.15.66:0/3017703263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f77f80ad4d0 con 0x7f77f8159c60 2024-03-30T00:40:42.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77eea00640 1 -- 172.21.15.66:0/3017703263 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f77f00f00e0 con 0x7f77f8159c60 2024-03-30T00:40:42.165 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:42.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 -- 172.21.15.66:0/3017703263 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77d4066690 msgr2=0x7f77d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77d4066690 0x7f77d4068b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f77e800ba50 tx=0x7f77e8008040 comp rx=0 tx=0).stop 2024-03-30T00:40:42.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 -- 172.21.15.66:0/3017703263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77f8159c60 msgr2=0x7f77f80ac670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77f8159c60 0x7f77f80ac670 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f77f00af040 tx=0x7f77f0019700 comp rx=0 tx=0).stop 2024-03-30T00:40:42.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 -- 172.21.15.66:0/3017703263 shutdown_connections 2024-03-30T00:40:42.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77d4066690 0x7f77d4068b50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77f8159c60 0x7f77f80ac670 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77f80cde70 0x7f77f8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 --2- 172.21.15.66:0/3017703263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77f80cda90 0x7f77f809b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 -- 172.21.15.66:0/3017703263 >> 172.21.15.66:0/3017703263 conn(0x7f77f8069f60 msgr2=0x7f77f806a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:42.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 -- 172.21.15.66:0/3017703263 shutdown_connections 2024-03-30T00:40:42.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.157+0000 7f77fda00640 1 -- 172.21.15.66:0/3017703263 wait complete. 2024-03-30T00:40:42.186 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a79600640 1 Processor -- start 2024-03-30T00:40:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a79600640 1 -- start start 2024-03-30T00:40:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a79600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740b4a60 0x7f9a7415f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a79600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a74099e70 con 0x7f9a740cf320 2024-03-30T00:40:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a79600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a740cbd90 con 0x7f9a74098c80 2024-03-30T00:40:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a79600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740cc070 con 0x7f9a740b4a60 2024-03-30T00:40:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a6a000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9a740cf320 0x7f9a7415eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42442/0 (socket says 172.21.15.66:42442) 2024-03-30T00:40:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a6a000640 1 -- 172.21.15.66:0/4257082911 learned_addr learned my addr 172.21.15.66:0/4257082911 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a73400640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740b4a60 0x7f9a7415f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614755946 0 0) 0x7f9a74099e70 con 0x7f9a740cf320 2024-03-30T00:40:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a58003530 con 0x7f9a740cf320 2024-03-30T00:40:42.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1131498432 0 0) 0x7f9a58003530 con 0x7f9a740cf320 2024-03-30T00:40:42.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740b4a60 msgr2=0x7f9a7415f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:42.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740b4a60 0x7f9a7415f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 >> v1:172.21.15.156:6789/0 conn(0x7f9a74098c80 legacy=0x7f9a74099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a740cc350 con 0x7f9a740cf320 2024-03-30T00:40:42.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f9a5c0016a0 con 0x7f9a740cf320 2024-03-30T00:40:42.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 >> v1:172.21.15.66:6789/0 conn(0x7f9a740cf320 legacy=0x7f9a7415eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740b4a60 0x7f9a580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a58003f30 0x7f9a580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 0x7f9a58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740cc070 con 0x7f9a58004d10 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5800bb00 con 0x7f9a58003f30 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5800bcc0 con 0x7f9a740b4a60 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a72a00640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740b4a60 0x7f9a580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.321+0000 7f9a73400640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 0x7f9a58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a6a000640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a58003f30 0x7f9a580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a73400640 1 -- 172.21.15.66:0/4257082911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740b4a60 msgr2=0x7f9a580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a73400640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740b4a60 0x7f9a580039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a73400640 1 -- 172.21.15.66:0/4257082911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a58003f30 msgr2=0x7f9a580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a73400640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a58003f30 0x7f9a580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a73400640 1 -- 172.21.15.66:0/4257082911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a58003530 con 0x7f9a58004d10 2024-03-30T00:40:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a73400640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 0x7f9a58005190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009fc0 tx=0x7f9a6c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e737468e9f190fff server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0a5c90 con 0x7f9a58004d10 2024-03-30T00:40:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c00f040 con 0x7f9a58004d10 2024-03-30T00:40:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a72000640 1 -- 172.21.15.66:0/4257082911 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0ad440 con 0x7f9a58004d10 2024-03-30T00:40:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a79600640 1 -- 172.21.15.66:0/4257082911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 msgr2=0x7f9a58005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a79600640 1 --2- 172.21.15.66:0/4257082911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 0x7f9a58005190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009fc0 tx=0x7f9a6c01cc80 comp rx=0 tx=0).stop 2024-03-30T00:40:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a73400640 1 -- 172.21.15.66:0/4257082911 reap_dead start 2024-03-30T00:40:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a79600640 1 -- 172.21.15.66:0/4257082911 shutdown_connections 2024-03-30T00:40:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.325+0000 7f9a79600640 1 -- 172.21.15.66:0/4257082911 >> 172.21.15.66:0/4257082911 conn(0x7f9a740b2130 msgr2=0x7f9a740b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a79600640 1 -- 172.21.15.66:0/4257082911 shutdown_connections 2024-03-30T00:40:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a79600640 1 -- 172.21.15.66:0/4257082911 wait complete. 2024-03-30T00:40:42.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a79600640 1 Processor -- start 2024-03-30T00:40:42.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a79600640 1 -- start start 2024-03-30T00:40:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a79600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 0x7f9a74158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a79600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a741514e0 0x7f9a740c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a79600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740c98e0 0x7f9a740c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a79600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740ca2d0 con 0x7f9a58004d10 2024-03-30T00:40:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a79600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740c59e0 con 0x7f9a741514e0 2024-03-30T00:40:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a79600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740c5cc0 con 0x7f9a740c98e0 2024-03-30T00:40:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a72a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 0x7f9a74158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a72a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 0x7f9a74158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60752/0 (socket says 172.21.15.66:60752) 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a72a00640 1 -- 172.21.15.66:0/3338700722 learned_addr learned my addr 172.21.15.66:0/3338700722 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a6a000640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a741514e0 0x7f9a740c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a73400640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740c98e0 0x7f9a740c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a72a00640 1 -- 172.21.15.66:0/3338700722 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740c98e0 msgr2=0x7f9a740c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a72a00640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740c98e0 0x7f9a740c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a72a00640 1 -- 172.21.15.66:0/3338700722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a741514e0 msgr2=0x7f9a740c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a72a00640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a741514e0 0x7f9a740c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a72a00640 1 -- 172.21.15.66:0/3338700722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a740c49a0 con 0x7f9a58004d10 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a73400640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740c98e0 0x7f9a740c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a72a00640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 0x7f9a74158190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400f610 tx=0x7f9a6400fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a6b400640 1 -- 172.21.15.66:0/3338700722 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a640052e0 con 0x7f9a58004d10 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a6b400640 1 -- 172.21.15.66:0/3338700722 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a64005440 con 0x7f9a58004d10 2024-03-30T00:40:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a6b400640 1 -- 172.21.15.66:0/3338700722 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64003820 con 0x7f9a58004d10 2024-03-30T00:40:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.329+0000 7f9a79600640 1 -- 172.21.15.66:0/3338700722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a740c4c80 con 0x7f9a58004d10 2024-03-30T00:40:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.333+0000 7f9a79600640 1 -- 172.21.15.66:0/3338700722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a740c46b0 con 0x7f9a58004d10 2024-03-30T00:40:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.333+0000 7f9a79600640 1 -- 172.21.15.66:0/3338700722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a740677d0 con 0x7f9a58004d10 2024-03-30T00:40:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.333+0000 7f9a6b400640 1 -- 172.21.15.66:0/3338700722 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9a64003980 con 0x7f9a58004d10 2024-03-30T00:40:42.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.333+0000 7f9a6b400640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a500666c0 0x7f9a50068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.333+0000 7f9a6b400640 1 -- 172.21.15.66:0/3338700722 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f9a64084310 con 0x7f9a58004d10 2024-03-30T00:40:42.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.333+0000 7f9a6a000640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a500666c0 0x7f9a50068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.333+0000 7f9a6a000640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a500666c0 0x7f9a50068b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a740cddc0 tx=0x7f9a5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.337+0000 7f9a6b400640 1 -- 172.21.15.66:0/3338700722 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9a64051310 con 0x7f9a58004d10 2024-03-30T00:40:42.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.497+0000 7f9a79600640 1 -- 172.21.15.66:0/3338700722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a74066c30 con 0x7f9a58004d10 2024-03-30T00:40:42.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.497+0000 7f9a6b400640 1 -- 172.21.15.66:0/3338700722 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a64054930 con 0x7f9a58004d10 2024-03-30T00:40:42.507 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 -- 172.21.15.66:0/3338700722 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a500666c0 msgr2=0x7f9a50068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a500666c0 0x7f9a50068b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a740cddc0 tx=0x7f9a5c009040 comp rx=0 tx=0).stop 2024-03-30T00:40:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 -- 172.21.15.66:0/3338700722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 msgr2=0x7f9a74158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 0x7f9a74158190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400f610 tx=0x7f9a6400fad0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 -- 172.21.15.66:0/3338700722 shutdown_connections 2024-03-30T00:40:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a740c98e0 0x7f9a740c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a741514e0 0x7f9a740c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a500666c0 0x7f9a50068b80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 --2- 172.21.15.66:0/3338700722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a58004d10 0x7f9a74158190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 -- 172.21.15.66:0/3338700722 >> 172.21.15.66:0/3338700722 conn(0x7f9a740b2130 msgr2=0x7f9a7406e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 -- 172.21.15.66:0/3338700722 shutdown_connections 2024-03-30T00:40:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.501+0000 7f9a79600640 1 -- 172.21.15.66:0/3338700722 wait complete. 2024-03-30T00:40:42.528 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:42.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.661+0000 7f65d7e00640 1 Processor -- start 2024-03-30T00:40:42.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.661+0000 7f65d7e00640 1 -- start start 2024-03-30T00:40:42.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d80b1810 0x7f65d80ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d7e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d80acea0 con 0x7f65d80acac0 2024-03-30T00:40:42.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d7e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d815a9d0 con 0x7f65d80ceb20 2024-03-30T00:40:42.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d7e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80cefa0 con 0x7f65d80b1810 2024-03-30T00:40:42.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f65d80acac0 0x7f65d806d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42448/0 (socket says 172.21.15.66:42448) 2024-03-30T00:40:42.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6a00640 1 -- 172.21.15.66:0/3146178998 learned_addr learned my addr 172.21.15.66:0/3146178998 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:42.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65ce000640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d80b1810 0x7f65d80ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568297824 0 0) 0x7f65d80acea0 con 0x7f65d80acac0 2024-03-30T00:40:42.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65bc003530 con 0x7f65d80acac0 2024-03-30T00:40:42.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2909572292 0 0) 0x7f65bc003530 con 0x7f65d80acac0 2024-03-30T00:40:42.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d80b1810 msgr2=0x7f65d80ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d80b1810 0x7f65d80ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 >> v1:172.21.15.156:6789/0 conn(0x7f65d80ceb20 legacy=0x7f65d80ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d80b1dc0 con 0x7f65d80acac0 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f65c8001a00 con 0x7f65d80acac0 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 >> v1:172.21.15.66:6789/0 conn(0x7f65d80acac0 legacy=0x7f65d806d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65bc0039b0 0x7f65bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65bc0047d0 0x7f65bc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65bc005200 0x7f65bc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80cefa0 con 0x7f65bc0039b0 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc00bf80 con 0x7f65bc0047d0 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc00c0f0 con 0x7f65bc005200 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6a00640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65bc0039b0 0x7f65bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65ce000640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65bc0047d0 0x7f65bc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d7400640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65bc005200 0x7f65bc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6a00640 1 -- 172.21.15.66:0/3146178998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65bc005200 msgr2=0x7f65bc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:42.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6a00640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65bc005200 0x7f65bc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6a00640 1 -- 172.21.15.66:0/3146178998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65bc0047d0 msgr2=0x7f65bc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:42.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6a00640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65bc0047d0 0x7f65bc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.665+0000 7f65d6a00640 1 -- 172.21.15.66:0/3146178998 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc003530 con 0x7f65bc0039b0 2024-03-30T00:40:42.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d6a00640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65bc0039b0 0x7f65bc003dd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f65c800a830 tx=0x7f65c801d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb5ef7ada25887c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:42.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80070d0 con 0x7f65bc0039b0 2024-03-30T00:40:42.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80a8070 con 0x7f65bc0039b0 2024-03-30T00:40:42.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d6000640 1 -- 172.21.15.66:0/3146178998 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80074b0 con 0x7f65bc0039b0 2024-03-30T00:40:42.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d7e00640 1 -- 172.21.15.66:0/3146178998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65bc0039b0 msgr2=0x7f65bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d7e00640 1 --2- 172.21.15.66:0/3146178998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65bc0039b0 0x7f65bc003dd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f65c800a830 tx=0x7f65c801d330 comp rx=0 tx=0).stop 2024-03-30T00:40:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d7400640 1 -- 172.21.15.66:0/3146178998 reap_dead start 2024-03-30T00:40:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d7e00640 1 -- 172.21.15.66:0/3146178998 shutdown_connections 2024-03-30T00:40:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d7e00640 1 -- 172.21.15.66:0/3146178998 >> 172.21.15.66:0/3146178998 conn(0x7f65d806a190 msgr2=0x7f65d806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d7e00640 1 -- 172.21.15.66:0/3146178998 shutdown_connections 2024-03-30T00:40:42.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d7e00640 1 -- 172.21.15.66:0/3146178998 wait complete. 2024-03-30T00:40:42.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d7e00640 1 Processor -- start 2024-03-30T00:40:42.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.669+0000 7f65d7e00640 1 -- start start 2024-03-30T00:40:42.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65bc0039b0 0x7f65d8155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65d8157b20 0x7f65d815c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d80c84f0 0x7f65d815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80b17f0 con 0x7f65d80c84f0 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8160d10 con 0x7f65d8157b20 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80aeb10 con 0x7f65bc0039b0 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d80c84f0 0x7f65d815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d80c84f0 0x7f65d815abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60768/0 (socket says 172.21.15.66:60768) 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7400640 1 -- 172.21.15.66:0/2015308441 learned_addr learned my addr 172.21.15.66:0/2015308441 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d6a00640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65bc0039b0 0x7f65d8155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65ce000640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65d8157b20 0x7f65d815c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7400640 1 -- 172.21.15.66:0/2015308441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65bc0039b0 msgr2=0x7f65d8155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7400640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65bc0039b0 0x7f65d8155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7400640 1 -- 172.21.15.66:0/2015308441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65d8157b20 msgr2=0x7f65d815c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7400640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65d8157b20 0x7f65d815c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7400640 1 -- 172.21.15.66:0/2015308441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d8160a30 con 0x7f65d80c84f0 2024-03-30T00:40:42.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d6a00640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65bc0039b0 0x7f65d8155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7400640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d80c84f0 0x7f65d815abe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f65d80b42f0 tx=0x7f65d000e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65cf400640 1 -- 172.21.15.66:0/2015308441 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d00052e0 con 0x7f65d80c84f0 2024-03-30T00:40:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65cf400640 1 -- 172.21.15.66:0/2015308441 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d0005460 con 0x7f65d80c84f0 2024-03-30T00:40:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7e00640 1 -- 172.21.15.66:0/2015308441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d815b8b0 con 0x7f65d80c84f0 2024-03-30T00:40:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65cf400640 1 -- 172.21.15.66:0/2015308441 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0003840 con 0x7f65d80c84f0 2024-03-30T00:40:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7e00640 1 -- 172.21.15.66:0/2015308441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d80b1480 con 0x7f65d80c84f0 2024-03-30T00:40:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65d7e00640 1 -- 172.21.15.66:0/2015308441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d8073e00 con 0x7f65d80c84f0 2024-03-30T00:40:42.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.673+0000 7f65cf400640 1 -- 172.21.15.66:0/2015308441 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f65d00039a0 con 0x7f65d80c84f0 2024-03-30T00:40:42.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.677+0000 7f65cf400640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65b4066710 0x7f65b4068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:42.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.677+0000 7f65cf400640 1 -- 172.21.15.66:0/2015308441 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f65d0083c70 con 0x7f65d80c84f0 2024-03-30T00:40:42.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.677+0000 7f65d6a00640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65b4066710 0x7f65b4068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:42.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.677+0000 7f65d6a00640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65b4066710 0x7f65b4068bd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f65c80ad000 tx=0x7f65c8009050 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:42.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.681+0000 7f65cf400640 1 -- 172.21.15.66:0/2015308441 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f65d0050bc0 con 0x7f65d80c84f0 2024-03-30T00:40:42.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 -- 172.21.15.66:0/2015308441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f65d815bd60 con 0x7f65d80c84f0 2024-03-30T00:40:42.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65cf400640 1 -- 172.21.15.66:0/2015308441 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f65d00190a0 con 0x7f65d80c84f0 2024-03-30T00:40:42.850 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:42.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 -- 172.21.15.66:0/2015308441 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65b4066710 msgr2=0x7f65b4068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65b4066710 0x7f65b4068bd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f65c80ad000 tx=0x7f65c8009050 comp rx=0 tx=0).stop 2024-03-30T00:40:42.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 -- 172.21.15.66:0/2015308441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d80c84f0 msgr2=0x7f65d815abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:42.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d80c84f0 0x7f65d815abe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f65d80b42f0 tx=0x7f65d000e8a0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 -- 172.21.15.66:0/2015308441 shutdown_connections 2024-03-30T00:40:42.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65b4066710 0x7f65b4068bd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d80c84f0 0x7f65d815abe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65d8157b20 0x7f65d815c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 --2- 172.21.15.66:0/2015308441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65bc0039b0 0x7f65d8155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:42.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 -- 172.21.15.66:0/2015308441 >> 172.21.15.66:0/2015308441 conn(0x7f65d806a190 msgr2=0x7f65d80acea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:42.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 -- 172.21.15.66:0/2015308441 shutdown_connections 2024-03-30T00:40:42.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:42.841+0000 7f65d7e00640 1 -- 172.21.15.66:0/2015308441 wait complete. 2024-03-30T00:40:42.872 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:43.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.001+0000 7f7b89800640 1 Processor -- start 2024-03-30T00:40:43.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.001+0000 7f7b89800640 1 -- start start 2024-03-30T00:40:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b89800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b8415a640 0x7f7b8406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b89800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7b840ce520 0x7f7b840ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b89800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b8406d070 con 0x7f7b840cabb0 2024-03-30T00:40:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b89800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b840cbba0 con 0x7f7b840ce520 2024-03-30T00:40:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b89800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8406e2c0 con 0x7f7b8415a640 2024-03-30T00:40:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b82a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b8415a640 0x7f7b8406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b7be00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7b840ce520 0x7f7b840ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b83400640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7b840cabb0 0x7f7b840cb000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:51018/0 (socket says 172.21.15.66:51018) 2024-03-30T00:40:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b83400640 1 -- 172.21.15.66:0/2109108998 learned_addr learned my addr 172.21.15.66:0/2109108998 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b82000640 1 -- 172.21.15.66:0/2109108998 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375914222 0 0) 0x7f7b8406d070 con 0x7f7b840cabb0 2024-03-30T00:40:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b82000640 1 -- 172.21.15.66:0/2109108998 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b68003530 con 0x7f7b840cabb0 2024-03-30T00:40:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b82a00640 1 -- 172.21.15.66:0/2109108998 >> v2:172.21.15.156:3300/0 conn(0x7f7b840ce520 msgr2=0x7f7b840ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b82a00640 1 --2- 172.21.15.66:0/2109108998 >> v2:172.21.15.156:3300/0 conn(0x7f7b840ce520 0x7f7b840ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b82a00640 1 -- 172.21.15.66:0/2109108998 >> v1:172.21.15.156:6789/0 conn(0x7f7b840cabb0 legacy=0x7f7b840cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b82a00640 1 -- 172.21.15.66:0/2109108998 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b840cee60 con 0x7f7b8415a640 2024-03-30T00:40:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.005+0000 7f7b7be00640 1 --2- 172.21.15.66:0/2109108998 >> v2:172.21.15.156:3300/0 conn(0x7f7b840ce520 0x7f7b840ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:43.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b82a00640 1 --2- 172.21.15.66:0/2109108998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b8415a640 0x7f7b8406dd80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7b740097c0 tx=0x7f7b7401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a340c45b553bac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b82000640 1 -- 172.21.15.66:0/2109108998 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740ad640 con 0x7f7b8415a640 2024-03-30T00:40:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b82000640 1 -- 172.21.15.66:0/2109108998 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b740ad7a0 con 0x7f7b8415a640 2024-03-30T00:40:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b82000640 1 -- 172.21.15.66:0/2109108998 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740adb30 con 0x7f7b8415a640 2024-03-30T00:40:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b89800640 1 -- 172.21.15.66:0/2109108998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b8415a640 msgr2=0x7f7b8406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b89800640 1 --2- 172.21.15.66:0/2109108998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b8415a640 0x7f7b8406dd80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7b740097c0 tx=0x7f7b7401c4a0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b89800640 1 -- 172.21.15.66:0/2109108998 shutdown_connections 2024-03-30T00:40:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b89800640 1 --2- 172.21.15.66:0/2109108998 >> v2:172.21.15.156:3300/0 conn(0x7f7b840ce520 0x7f7b840ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b89800640 1 --2- 172.21.15.66:0/2109108998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b8415a640 0x7f7b8406dd80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b89800640 1 -- 172.21.15.66:0/2109108998 >> 172.21.15.66:0/2109108998 conn(0x7f7b84069f60 msgr2=0x7f7b8406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b89800640 1 -- 172.21.15.66:0/2109108998 shutdown_connections 2024-03-30T00:40:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b89800640 1 -- 172.21.15.66:0/2109108998 wait complete. 2024-03-30T00:40:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b89800640 1 Processor -- start 2024-03-30T00:40:43.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.009+0000 7f7b89800640 1 -- start start 2024-03-30T00:40:43.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b89800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b840cabb0 0x7f7b84097af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b89800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b840ce520 0x7f7b841548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b89800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b8415a640 0x7f7b84151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b89800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b84155660 con 0x7f7b8415a640 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b89800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b84155940 con 0x7f7b840ce520 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b89800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b84155c20 con 0x7f7b840cabb0 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b83400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b8415a640 0x7f7b84151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b83400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b8415a640 0x7f7b84151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60778/0 (socket says 172.21.15.66:60778) 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b83400640 1 -- 172.21.15.66:0/429655363 learned_addr learned my addr 172.21.15.66:0/429655363 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b82a00640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b840cabb0 0x7f7b84097af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b7be00640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b840ce520 0x7f7b841548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b83400640 1 -- 172.21.15.66:0/429655363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b840cabb0 msgr2=0x7f7b84097af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b83400640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b840cabb0 0x7f7b84097af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b83400640 1 -- 172.21.15.66:0/429655363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b840ce520 msgr2=0x7f7b841548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b83400640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b840ce520 0x7f7b841548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b83400640 1 -- 172.21.15.66:0/429655363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c009420 con 0x7f7b8415a640 2024-03-30T00:40:43.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b83400640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b8415a640 0x7f7b84151b70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00cb40 tx=0x7f7b7c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:43.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b7aa00640 1 -- 172.21.15.66:0/429655363 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c00aca0 con 0x7f7b8415a640 2024-03-30T00:40:43.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b7aa00640 1 -- 172.21.15.66:0/429655363 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c00ae00 con 0x7f7b8415a640 2024-03-30T00:40:43.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b7aa00640 1 -- 172.21.15.66:0/429655363 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c012620 con 0x7f7b8415a640 2024-03-30T00:40:43.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b89800640 1 -- 172.21.15.66:0/429655363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b74009420 con 0x7f7b8415a640 2024-03-30T00:40:43.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b89800640 1 -- 172.21.15.66:0/429655363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b841561e0 con 0x7f7b8415a640 2024-03-30T00:40:43.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b89800640 1 -- 172.21.15.66:0/429655363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b84072c00 con 0x7f7b8415a640 2024-03-30T00:40:43.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.013+0000 7f7b7aa00640 1 -- 172.21.15.66:0/429655363 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7b7c007070 con 0x7f7b8415a640 2024-03-30T00:40:43.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.017+0000 7f7b7aa00640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b60066690 0x7f7b60068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.017+0000 7f7b7aa00640 1 -- 172.21.15.66:0/429655363 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f7b7c084520 con 0x7f7b8415a640 2024-03-30T00:40:43.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.017+0000 7f7b82a00640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b60066690 0x7f7b60068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.017+0000 7f7b82a00640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b60066690 0x7f7b60068b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b740af000 tx=0x7f7b740a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:43.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.021+0000 7f7b7aa00640 1 -- 172.21.15.66:0/429655363 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7b7c051530 con 0x7f7b8415a640 2024-03-30T00:40:43.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.181+0000 7f7b89800640 1 -- 172.21.15.66:0/429655363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b841563d0 con 0x7f7b8415a640 2024-03-30T00:40:43.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b7aa00640 1 -- 172.21.15.66:0/429655363 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7f7b7c054b50 con 0x7f7b8415a640 2024-03-30T00:40:43.194 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:43.194 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:43.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 -- 172.21.15.66:0/429655363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b60066690 msgr2=0x7f7b60068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b60066690 0x7f7b60068b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b740af000 tx=0x7f7b740a7070 comp rx=0 tx=0).stop 2024-03-30T00:40:43.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 -- 172.21.15.66:0/429655363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b8415a640 msgr2=0x7f7b84151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b8415a640 0x7f7b84151b70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00cb40 tx=0x7f7b7c00a060 comp rx=0 tx=0).stop 2024-03-30T00:40:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 -- 172.21.15.66:0/429655363 shutdown_connections 2024-03-30T00:40:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b60066690 0x7f7b60068b50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b8415a640 0x7f7b84151b70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b840ce520 0x7f7b841548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 --2- 172.21.15.66:0/429655363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b840cabb0 0x7f7b84097af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 -- 172.21.15.66:0/429655363 >> 172.21.15.66:0/429655363 conn(0x7f7b84069f60 msgr2=0x7f7b840ae780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 -- 172.21.15.66:0/429655363 shutdown_connections 2024-03-30T00:40:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.185+0000 7f7b89800640 1 -- 172.21.15.66:0/429655363 wait complete. 2024-03-30T00:40:43.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:43.218 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:43.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc260400640 1 Processor -- start 2024-03-30T00:40:43.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc260400640 1 -- start start 2024-03-30T00:40:43.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc260400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2580cda90 0x7fc2580ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc260400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 0x7fc2580cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc260400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25815a990 con 0x7fc2580cde70 2024-03-30T00:40:43.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc260400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2580b1b80 con 0x7fc2580cda90 2024-03-30T00:40:43.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc260400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580cd3e0 con 0x7fc25815a370 2024-03-30T00:40:43.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc25da00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2580cda90 0x7fc2580ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc25d000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc2580cde70 0x7fc258159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42460/0 (socket says 172.21.15.66:42460) 2024-03-30T00:40:43.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc25d000640 1 -- 172.21.15.66:0/2666843318 learned_addr learned my addr 172.21.15.66:0/2666843318 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:43.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc25da00640 1 -- 172.21.15.66:0/2666843318 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 msgr2=0x7fc2580cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:43.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc25e400640 1 --2- 172.21.15.66:0/2666843318 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 0x7fc2580cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc25da00640 1 --2- 172.21.15.66:0/2666843318 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 0x7fc2580cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc257e00640 1 -- 172.21.15.66:0/2666843318 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436923301 0 0) 0x7fc25815a990 con 0x7fc2580cde70 2024-03-30T00:40:43.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc25da00640 1 -- 172.21.15.66:0/2666843318 >> v1:172.21.15.66:6789/0 conn(0x7fc2580cde70 legacy=0x7fc258159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc25da00640 1 -- 172.21.15.66:0/2666843318 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2580b11e0 con 0x7fc2580cda90 2024-03-30T00:40:43.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.353+0000 7fc25e400640 1 --2- 172.21.15.66:0/2666843318 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 0x7fc2580cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:43.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc25da00640 1 --2- 172.21.15.66:0/2666843318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2580cda90 0x7fc2580ce520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0097c0 tx=0x7fc24c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacf62e5df91b96b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:43.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc257e00640 1 -- 172.21.15.66:0/2666843318 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0ab600 con 0x7fc2580cda90 2024-03-30T00:40:43.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc257e00640 1 -- 172.21.15.66:0/2666843318 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c0a7050 con 0x7fc2580cda90 2024-03-30T00:40:43.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc257e00640 1 -- 172.21.15.66:0/2666843318 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0ab980 con 0x7fc2580cda90 2024-03-30T00:40:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc260400640 1 -- 172.21.15.66:0/2666843318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2580cda90 msgr2=0x7fc2580ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc260400640 1 --2- 172.21.15.66:0/2666843318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2580cda90 0x7fc2580ce520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0097c0 tx=0x7fc24c01c410 comp rx=0 tx=0).stop 2024-03-30T00:40:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc260400640 1 -- 172.21.15.66:0/2666843318 shutdown_connections 2024-03-30T00:40:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc260400640 1 --2- 172.21.15.66:0/2666843318 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 0x7fc2580cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc260400640 1 --2- 172.21.15.66:0/2666843318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2580cda90 0x7fc2580ce520 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc260400640 1 -- 172.21.15.66:0/2666843318 >> 172.21.15.66:0/2666843318 conn(0x7fc258069f60 msgr2=0x7fc25806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc260400640 1 -- 172.21.15.66:0/2666843318 shutdown_connections 2024-03-30T00:40:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc260400640 1 -- 172.21.15.66:0/2666843318 wait complete. 2024-03-30T00:40:43.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.357+0000 7fc260400640 1 Processor -- start 2024-03-30T00:40:43.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc260400640 1 -- start start 2024-03-30T00:40:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc260400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2580cda90 0x7fc2581524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc260400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2580cde70 0x7fc258157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc260400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 0x7fc258164900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc260400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580c18a0 con 0x7fc2580cda90 2024-03-30T00:40:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc260400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25814f790 con 0x7fc2580cde70 2024-03-30T00:40:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc260400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25814fa30 con 0x7fc25815a370 2024-03-30T00:40:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2580cda90 0x7fc2581524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2580cda90 0x7fc2581524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60790/0 (socket says 172.21.15.66:60790) 2024-03-30T00:40:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25da00640 1 -- 172.21.15.66:0/2115289078 learned_addr learned my addr 172.21.15.66:0/2115289078 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25d000640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2580cde70 0x7fc258157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25e400640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 0x7fc258164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25da00640 1 -- 172.21.15.66:0/2115289078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 msgr2=0x7fc258164900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25da00640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 0x7fc258164900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25da00640 1 -- 172.21.15.66:0/2115289078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2580cde70 msgr2=0x7fc258157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25da00640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2580cde70 0x7fc258157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25da00640 1 -- 172.21.15.66:0/2115289078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c009420 con 0x7fc2580cda90 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25e400640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 0x7fc258164900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc25da00640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2580cda90 0x7fc2581524c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc24c01a040 tx=0x7fc24c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc255600640 1 -- 172.21.15.66:0/2115289078 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0b4610 con 0x7fc2580cda90 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc255600640 1 -- 172.21.15.66:0/2115289078 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c0b4790 con 0x7fc2580cda90 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc255600640 1 -- 172.21.15.66:0/2115289078 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0b4b40 con 0x7fc2580cda90 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc260400640 1 -- 172.21.15.66:0/2115289078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25814fd10 con 0x7fc2580cda90 2024-03-30T00:40:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc260400640 1 -- 172.21.15.66:0/2115289078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2580c1b40 con 0x7fc2580cda90 2024-03-30T00:40:43.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.361+0000 7fc260400640 1 -- 172.21.15.66:0/2115289078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc258066c30 con 0x7fc2580cda90 2024-03-30T00:40:43.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.365+0000 7fc255600640 1 -- 172.21.15.66:0/2115289078 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc24c0b2040 con 0x7fc2580cda90 2024-03-30T00:40:43.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.365+0000 7fc255600640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2380666c0 0x7fc238068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.365+0000 7fc255600640 1 -- 172.21.15.66:0/2115289078 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fc24c11e130 con 0x7fc2580cda90 2024-03-30T00:40:43.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.365+0000 7fc25d000640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2380666c0 0x7fc238068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.365+0000 7fc25d000640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2380666c0 0x7fc238068b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc258151090 tx=0x7fc244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:43.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.369+0000 7fc255600640 1 -- 172.21.15.66:0/2115289078 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc24c0eb0a0 con 0x7fc2580cda90 2024-03-30T00:40:43.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.529+0000 7fc260400640 1 -- 172.21.15.66:0/2115289078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2580b3170 con 0x7fc2580cda90 2024-03-30T00:40:43.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.529+0000 7fc255600640 1 -- 172.21.15.66:0/2115289078 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc24c0ee6c0 con 0x7fc2580cda90 2024-03-30T00:40:43.540 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 -- 172.21.15.66:0/2115289078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2380666c0 msgr2=0x7fc238068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2380666c0 0x7fc238068b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc258151090 tx=0x7fc244009040 comp rx=0 tx=0).stop 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 -- 172.21.15.66:0/2115289078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2580cda90 msgr2=0x7fc2581524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2580cda90 0x7fc2581524c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc24c01a040 tx=0x7fc24c0191b0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 -- 172.21.15.66:0/2115289078 shutdown_connections 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc25815a370 0x7fc258164900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2580cde70 0x7fc258157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2380666c0 0x7fc238068b80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 --2- 172.21.15.66:0/2115289078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2580cda90 0x7fc2581524c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 -- 172.21.15.66:0/2115289078 >> 172.21.15.66:0/2115289078 conn(0x7fc258069f60 msgr2=0x7fc25806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 -- 172.21.15.66:0/2115289078 shutdown_connections 2024-03-30T00:40:43.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.533+0000 7fc260400640 1 -- 172.21.15.66:0/2115289078 wait complete. 2024-03-30T00:40:43.561 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f464d600640 1 Processor -- start 2024-03-30T00:40:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f464d600640 1 -- start start 2024-03-30T00:40:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f464d600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f464806cb90 0x7f46480d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f464d600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464815d250 con 0x7f46480cdef0 2024-03-30T00:40:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f464d600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464815d530 con 0x7f464815c6b0 2024-03-30T00:40:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f464d600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464806b1f0 con 0x7f464806cb90 2024-03-30T00:40:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f464cc00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f46480cdef0 0x7f46481604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42474/0 (socket says 172.21.15.66:42474) 2024-03-30T00:40:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f464cc00640 1 -- 172.21.15.66:0/2269960355 learned_addr learned my addr 172.21.15.66:0/2269960355 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f463f400640 1 --2- 172.21.15.66:0/2269960355 >> v2:172.21.15.156:3300/0 conn(0x7f464806cb90 0x7f46480d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f4647400640 1 -- 172.21.15.66:0/2269960355 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426404799 0 0) 0x7f464815d250 con 0x7f46480cdef0 2024-03-30T00:40:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f4647400640 1 -- 172.21.15.66:0/2269960355 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f462c003530 con 0x7f46480cdef0 2024-03-30T00:40:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f4647400640 1 -- 172.21.15.66:0/2269960355 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775496161 0 0) 0x7f464815d530 con 0x7f464815c6b0 2024-03-30T00:40:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f4647400640 1 -- 172.21.15.66:0/2269960355 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f464815d250 con 0x7f464815c6b0 2024-03-30T00:40:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f463f400640 1 -- 172.21.15.66:0/2269960355 >> v1:172.21.15.156:6789/0 conn(0x7f464815c6b0 legacy=0x7f464815ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f463f400640 1 -- 172.21.15.66:0/2269960355 >> v1:172.21.15.66:6789/0 conn(0x7f46480cdef0 legacy=0x7f46481604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.697+0000 7f463f400640 1 -- 172.21.15.66:0/2269960355 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464815d810 con 0x7f464806cb90 2024-03-30T00:40:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.701+0000 7f463f400640 1 --2- 172.21.15.66:0/2269960355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f464806cb90 0x7f46480d0c50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f46300097c0 tx=0x7f463001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87ea0a308ce01a7a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.701+0000 7f4647400640 1 -- 172.21.15.66:0/2269960355 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300ab600 con 0x7f464806cb90 2024-03-30T00:40:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.701+0000 7f4647400640 1 -- 172.21.15.66:0/2269960355 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46300a7050 con 0x7f464806cb90 2024-03-30T00:40:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.701+0000 7f4647400640 1 -- 172.21.15.66:0/2269960355 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300ab980 con 0x7f464806cb90 2024-03-30T00:40:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.701+0000 7f464d600640 1 -- 172.21.15.66:0/2269960355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f464806cb90 msgr2=0x7f46480d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.701+0000 7f464d600640 1 --2- 172.21.15.66:0/2269960355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f464806cb90 0x7f46480d0c50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f46300097c0 tx=0x7f463001c3e0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.701+0000 7f464d600640 1 -- 172.21.15.66:0/2269960355 shutdown_connections 2024-03-30T00:40:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.701+0000 7f464d600640 1 --2- 172.21.15.66:0/2269960355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f464806cb90 0x7f46480d0c50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.701+0000 7f464d600640 1 -- 172.21.15.66:0/2269960355 >> 172.21.15.66:0/2269960355 conn(0x7f4648069d90 msgr2=0x7f464806a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.701+0000 7f464d600640 1 -- 172.21.15.66:0/2269960355 shutdown_connections 2024-03-30T00:40:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.701+0000 7f464d600640 1 -- 172.21.15.66:0/2269960355 wait complete. 2024-03-30T00:40:43.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464d600640 1 Processor -- start 2024-03-30T00:40:43.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464d600640 1 -- start start 2024-03-30T00:40:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464d600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464806cb90 0x7f46480b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464d600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480cdef0 0x7f4648151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815c6b0 0x7f4648151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46481531a0 con 0x7f464815c6b0 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464d600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46480ad290 con 0x7f46480cdef0 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464d600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4648152590 con 0x7f464806cb90 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815c6b0 0x7f4648151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815c6b0 0x7f4648151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60796/0 (socket says 172.21.15.66:60796) 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464cc00640 1 -- 172.21.15.66:0/2358084071 learned_addr learned my addr 172.21.15.66:0/2358084071 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f463f400640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480cdef0 0x7f4648151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f4647e00640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464806cb90 0x7f46480b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464cc00640 1 -- 172.21.15.66:0/2358084071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464806cb90 msgr2=0x7f46480b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464cc00640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464806cb90 0x7f46480b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464cc00640 1 -- 172.21.15.66:0/2358084071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480cdef0 msgr2=0x7f4648151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464cc00640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480cdef0 0x7f4648151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464cc00640 1 -- 172.21.15.66:0/2358084071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4630009420 con 0x7f464815c6b0 2024-03-30T00:40:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464cc00640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815c6b0 0x7f4648151d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f464000cba0 tx=0x7f464000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f4644c00640 1 -- 172.21.15.66:0/2358084071 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464000ace0 con 0x7f464815c6b0 2024-03-30T00:40:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f4644c00640 1 -- 172.21.15.66:0/2358084071 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464000ae40 con 0x7f464815c6b0 2024-03-30T00:40:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464d600640 1 -- 172.21.15.66:0/2358084071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464815e290 con 0x7f464815c6b0 2024-03-30T00:40:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f4644c00640 1 -- 172.21.15.66:0/2358084071 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640012620 con 0x7f464815c6b0 2024-03-30T00:40:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464d600640 1 -- 172.21.15.66:0/2358084071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4648152b50 con 0x7f464815c6b0 2024-03-30T00:40:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.705+0000 7f464d600640 1 -- 172.21.15.66:0/2358084071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46480cf7d0 con 0x7f464815c6b0 2024-03-30T00:40:43.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.709+0000 7f4644c00640 1 -- 172.21.15.66:0/2358084071 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4640007070 con 0x7f464815c6b0 2024-03-30T00:40:43.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.709+0000 7f4644c00640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4624066690 0x7f4624068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:43.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.709+0000 7f4644c00640 1 -- 172.21.15.66:0/2358084071 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f46400845e0 con 0x7f464815c6b0 2024-03-30T00:40:43.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.709+0000 7f4647e00640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4624066690 0x7f4624068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.709+0000 7f4647e00640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4624066690 0x7f4624068b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f46380011f0 tx=0x7f4638009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:43.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.713+0000 7f4644c00640 1 -- 172.21.15.66:0/2358084071 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4640051570 con 0x7f464815c6b0 2024-03-30T00:40:43.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 -- 172.21.15.66:0/2358084071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4648073e00 con 0x7f464815c6b0 2024-03-30T00:40:43.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f4644c00640 1 -- 172.21.15.66:0/2358084071 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4640054b90 con 0x7f464815c6b0 2024-03-30T00:40:43.883 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:43.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 -- 172.21.15.66:0/2358084071 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4624066690 msgr2=0x7f4624068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4624066690 0x7f4624068b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f46380011f0 tx=0x7f4638009040 comp rx=0 tx=0).stop 2024-03-30T00:40:43.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 -- 172.21.15.66:0/2358084071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815c6b0 msgr2=0x7f4648151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:43.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815c6b0 0x7f4648151d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f464000cba0 tx=0x7f464000a430 comp rx=0 tx=0).stop 2024-03-30T00:40:43.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 -- 172.21.15.66:0/2358084071 shutdown_connections 2024-03-30T00:40:43.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4624066690 0x7f4624068b50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815c6b0 0x7f4648151d50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46480cdef0 0x7f4648151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 --2- 172.21.15.66:0/2358084071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464806cb90 0x7f46480b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:43.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 -- 172.21.15.66:0/2358084071 >> 172.21.15.66:0/2358084071 conn(0x7f4648069d90 msgr2=0x7f464815e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:43.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.873+0000 7f464d600640 1 -- 172.21.15.66:0/2358084071 shutdown_connections 2024-03-30T00:40:43.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:43.877+0000 7f464d600640 1 -- 172.21.15.66:0/2358084071 wait complete. 2024-03-30T00:40:43.903 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:44.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04c1600640 1 Processor -- start 2024-03-30T00:40:44.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04c1600640 1 -- start start 2024-03-30T00:40:44.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04c1600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f04bc0ce710 0x7f04bc0ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04c1600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04bc0cb520 con 0x7f04bc15a670 2024-03-30T00:40:44.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04c1600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04bc0cd7d0 con 0x7f04bc0cd330 2024-03-30T00:40:44.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04c1600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04bc0cdab0 con 0x7f04bc0ce710 2024-03-30T00:40:44.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04baa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f04bc15a670 0x7f04bc0b0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42482/0 (socket says 172.21.15.66:42482) 2024-03-30T00:40:44.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04baa00640 1 -- 172.21.15.66:0/565760920 learned_addr learned my addr 172.21.15.66:0/565760920 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:44.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04b3e00640 1 --2- 172.21.15.66:0/565760920 >> v2:172.21.15.66:3300/0 conn(0x7f04bc0ce710 0x7f04bc0ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04ba000640 1 -- 172.21.15.66:0/565760920 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013885785 0 0) 0x7f04bc0cb520 con 0x7f04bc15a670 2024-03-30T00:40:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04ba000640 1 -- 172.21.15.66:0/565760920 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04a0003530 con 0x7f04bc15a670 2024-03-30T00:40:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04b3e00640 1 -- 172.21.15.66:0/565760920 >> v1:172.21.15.156:6789/0 conn(0x7f04bc0cd330 legacy=0x7f04bc0d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04b3e00640 1 -- 172.21.15.66:0/565760920 >> v1:172.21.15.66:6789/0 conn(0x7f04bc15a670 legacy=0x7f04bc0b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.037+0000 7f04b3e00640 1 -- 172.21.15.66:0/565760920 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc0cdd90 con 0x7f04bc0ce710 2024-03-30T00:40:44.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.041+0000 7f04b3e00640 1 --2- 172.21.15.66:0/565760920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0ce710 0x7f04bc0ccdf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f04a40097c0 tx=0x7f04a401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ceb35b3187a7ad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:44.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.041+0000 7f04ba000640 1 -- 172.21.15.66:0/565760920 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a40a8050 con 0x7f04bc0ce710 2024-03-30T00:40:44.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.041+0000 7f04ba000640 1 -- 172.21.15.66:0/565760920 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a40ac5f0 con 0x7f04bc0ce710 2024-03-30T00:40:44.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.041+0000 7f04ba000640 1 -- 172.21.15.66:0/565760920 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a40ac980 con 0x7f04bc0ce710 2024-03-30T00:40:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.041+0000 7f04c1600640 1 -- 172.21.15.66:0/565760920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0ce710 msgr2=0x7f04bc0ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.041+0000 7f04c1600640 1 --2- 172.21.15.66:0/565760920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0ce710 0x7f04bc0ccdf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f04a40097c0 tx=0x7f04a401c470 comp rx=0 tx=0).stop 2024-03-30T00:40:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.041+0000 7f04c1600640 1 -- 172.21.15.66:0/565760920 shutdown_connections 2024-03-30T00:40:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.041+0000 7f04c1600640 1 --2- 172.21.15.66:0/565760920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0ce710 0x7f04bc0ccdf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.041+0000 7f04c1600640 1 -- 172.21.15.66:0/565760920 >> 172.21.15.66:0/565760920 conn(0x7f04bc099870 msgr2=0x7f04bc089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.041+0000 7f04c1600640 1 -- 172.21.15.66:0/565760920 shutdown_connections 2024-03-30T00:40:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.041+0000 7f04c1600640 1 -- 172.21.15.66:0/565760920 wait complete. 2024-03-30T00:40:44.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04c1600640 1 Processor -- start 2024-03-30T00:40:44.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04c1600640 1 -- start start 2024-03-30T00:40:44.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04c1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0cd330 0x7f04bc0c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04c1600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04bc0ce710 0x7f04bc152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04c1600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04bc15a670 0x7f04bc06c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04c1600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc163920 con 0x7f04bc0cd330 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04c1600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc0700a0 con 0x7f04bc0ce710 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04c1600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc153440 con 0x7f04bc15a670 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04baa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0cd330 0x7f04bc0c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04baa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0cd330 0x7f04bc0c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60822/0 (socket says 172.21.15.66:60822) 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04baa00640 1 -- 172.21.15.66:0/3668603193 learned_addr learned my addr 172.21.15.66:0/3668603193 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04bb400640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04bc15a670 0x7f04bc06c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04b3e00640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04bc0ce710 0x7f04bc152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04baa00640 1 -- 172.21.15.66:0/3668603193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04bc15a670 msgr2=0x7f04bc06c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04baa00640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04bc15a670 0x7f04bc06c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04baa00640 1 -- 172.21.15.66:0/3668603193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04bc0ce710 msgr2=0x7f04bc152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04baa00640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04bc0ce710 0x7f04bc152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04baa00640 1 -- 172.21.15.66:0/3668603193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a4009420 con 0x7f04bc0cd330 2024-03-30T00:40:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04bb400640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04bc15a670 0x7f04bc06c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04b3e00640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04bc0ce710 0x7f04bc152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04baa00640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0cd330 0x7f04bc0c40c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c450 tx=0x7f04ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04b2a00640 1 -- 172.21.15.66:0/3668603193 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac005cf0 con 0x7f04bc0cd330 2024-03-30T00:40:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04b2a00640 1 -- 172.21.15.66:0/3668603193 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ac005e50 con 0x7f04bc0cd330 2024-03-30T00:40:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04b2a00640 1 -- 172.21.15.66:0/3668603193 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac007270 con 0x7f04bc0cd330 2024-03-30T00:40:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04c1600640 1 -- 172.21.15.66:0/3668603193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04bc06f800 con 0x7f04bc0cd330 2024-03-30T00:40:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04c1600640 1 -- 172.21.15.66:0/3668603193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc06fdc0 con 0x7f04bc0cd330 2024-03-30T00:40:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.045+0000 7f04c1600640 1 -- 172.21.15.66:0/3668603193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04bc066c30 con 0x7f04bc0cd330 2024-03-30T00:40:44.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.049+0000 7f04b2a00640 1 -- 172.21.15.66:0/3668603193 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f04ac0073d0 con 0x7f04bc0cd330 2024-03-30T00:40:44.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.049+0000 7f04b2a00640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f04980666c0 0x7f0498068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.049+0000 7f04b2a00640 1 -- 172.21.15.66:0/3668603193 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f04ac083a50 con 0x7f04bc0cd330 2024-03-30T00:40:44.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.049+0000 7f04b3e00640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f04980666c0 0x7f0498068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.049+0000 7f04b3e00640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f04980666c0 0x7f0498068b80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f04a40ae000 tx=0x7f04a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:44.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.053+0000 7f04b2a00640 1 -- 172.21.15.66:0/3668603193 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f04ac0509d0 con 0x7f04bc0cd330 2024-03-30T00:40:44.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 -- 172.21.15.66:0/3668603193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04bc06d160 con 0x7f04bc0cd330 2024-03-30T00:40:44.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04b2a00640 1 -- 172.21.15.66:0/3668603193 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04ac0170a0 con 0x7f04bc0cd330 2024-03-30T00:40:44.222 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 -- 172.21.15.66:0/3668603193 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f04980666c0 msgr2=0x7f0498068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f04980666c0 0x7f0498068b80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f04a40ae000 tx=0x7f04a4008040 comp rx=0 tx=0).stop 2024-03-30T00:40:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 -- 172.21.15.66:0/3668603193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0cd330 msgr2=0x7f04bc0c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0cd330 0x7f04bc0c40c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c450 tx=0x7f04ac00c910 comp rx=0 tx=0).stop 2024-03-30T00:40:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 -- 172.21.15.66:0/3668603193 shutdown_connections 2024-03-30T00:40:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04bc15a670 0x7f04bc06c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04bc0ce710 0x7f04bc152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f04980666c0 0x7f0498068b80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 --2- 172.21.15.66:0/3668603193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc0cd330 0x7f04bc0c40c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 -- 172.21.15.66:0/3668603193 >> 172.21.15.66:0/3668603193 conn(0x7f04bc099870 msgr2=0x7f04bc165c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 -- 172.21.15.66:0/3668603193 shutdown_connections 2024-03-30T00:40:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.213+0000 7f04c1600640 1 -- 172.21.15.66:0/3668603193 wait complete. 2024-03-30T00:40:44.244 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:44.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.377+0000 7feb0de00640 1 Processor -- start 2024-03-30T00:40:44.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.377+0000 7feb0de00640 1 -- start start 2024-03-30T00:40:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.377+0000 7feb0de00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7feb080af790 0x7feb08099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.377+0000 7feb0de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb080b0940 con 0x7feb0806bb90 2024-03-30T00:40:44.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.377+0000 7feb0de00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0806b140 con 0x7feb08099130 2024-03-30T00:40:44.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.377+0000 7feb0de00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0806b700 con 0x7feb080af790 2024-03-30T00:40:44.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feaffe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7feb0806bb90 0x7feb08098a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42488/0 (socket says 172.21.15.66:42488) 2024-03-30T00:40:44.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feaffe00640 1 -- 172.21.15.66:0/2576359398 learned_addr learned my addr 172.21.15.66:0/2576359398 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06a00640 1 --2- 172.21.15.66:0/2576359398 >> v2:172.21.15.156:3300/0 conn(0x7feb080af790 0x7feb08099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216581105 0 0) 0x7feb080b0940 con 0x7feb0806bb90 2024-03-30T00:40:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaec003530 con 0x7feb0806bb90 2024-03-30T00:40:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 461968668 0 0) 0x7feaec003530 con 0x7feb0806bb90 2024-03-30T00:40:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 >> v2:172.21.15.156:3300/0 conn(0x7feb080af790 msgr2=0x7feb08099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 --2- 172.21.15.66:0/2576359398 >> v2:172.21.15.156:3300/0 conn(0x7feb080af790 0x7feb08099760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 >> v1:172.21.15.156:6789/0 conn(0x7feb08099130 legacy=0x7feb080ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb080b0b90 con 0x7feb0806bb90 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7feaf0001680 con 0x7feb0806bb90 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 >> v1:172.21.15.66:6789/0 conn(0x7feb0806bb90 legacy=0x7feb08098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 --2- 172.21.15.66:0/2576359398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaec003980 0x7feaec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 --2- 172.21.15.66:0/2576359398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feaec0047d0 0x7feaec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 --2- 172.21.15.66:0/2576359398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feaec005190 0x7feaec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0806b700 con 0x7feaec003980 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec00c030 con 0x7feaec0047d0 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaec00c1f0 con 0x7feaec005190 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06a00640 1 --2- 172.21.15.66:0/2576359398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaec003980 0x7feaec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feaffe00640 1 --2- 172.21.15.66:0/2576359398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feaec0047d0 0x7feaec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06a00640 1 -- 172.21.15.66:0/2576359398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feaec005190 msgr2=0x7feaec00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06a00640 1 --2- 172.21.15.66:0/2576359398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feaec005190 0x7feaec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06a00640 1 -- 172.21.15.66:0/2576359398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feaec0047d0 msgr2=0x7feaec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06a00640 1 --2- 172.21.15.66:0/2576359398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feaec0047d0 0x7feaec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb06a00640 1 -- 172.21.15.66:0/2576359398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec003530 con 0x7feaec003980 2024-03-30T00:40:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.381+0000 7feb07400640 1 -- 172.21.15.66:0/2576359398 reap_dead start 2024-03-30T00:40:44.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb06a00640 1 --2- 172.21.15.66:0/2576359398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaec003980 0x7feaec003da0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009ae0 tx=0x7feaf801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d043894ac4524e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:44.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80a5700 con 0x7feaec003980 2024-03-30T00:40:44.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf80a5860 con 0x7feaec003980 2024-03-30T00:40:44.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb06000640 1 -- 172.21.15.66:0/2576359398 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80a5c10 con 0x7feaec003980 2024-03-30T00:40:44.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb0de00640 1 -- 172.21.15.66:0/2576359398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaec003980 msgr2=0x7feaec003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb0de00640 1 --2- 172.21.15.66:0/2576359398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaec003980 0x7feaec003da0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009ae0 tx=0x7feaf801ca80 comp rx=0 tx=0).stop 2024-03-30T00:40:44.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb0de00640 1 -- 172.21.15.66:0/2576359398 shutdown_connections 2024-03-30T00:40:44.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb0de00640 1 --2- 172.21.15.66:0/2576359398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaec003980 0x7feaec003da0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb0de00640 1 -- 172.21.15.66:0/2576359398 >> 172.21.15.66:0/2576359398 conn(0x7feb080b20c0 msgr2=0x7feb080b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:44.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb0de00640 1 -- 172.21.15.66:0/2576359398 shutdown_connections 2024-03-30T00:40:44.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb0de00640 1 -- 172.21.15.66:0/2576359398 wait complete. 2024-03-30T00:40:44.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb0de00640 1 Processor -- start 2024-03-30T00:40:44.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb0de00640 1 -- start start 2024-03-30T00:40:44.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb0de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feaec003980 0x7feb0815efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.385+0000 7feb0de00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0815e9b0 0x7feb080cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb0de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb081554b0 0x7feb080cf580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb0de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb080c6460 con 0x7feb081554b0 2024-03-30T00:40:44.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb0de00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb080c53c0 con 0x7feb0815e9b0 2024-03-30T00:40:44.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb0de00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb080c56a0 con 0x7feaec003980 2024-03-30T00:40:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb06a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feaec003980 0x7feb0815efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb07400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb081554b0 0x7feb080cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb06a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feaec003980 0x7feb0815efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:47240/0 (socket says 172.21.15.66:47240) 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb06a00640 1 -- 172.21.15.66:0/1464394898 learned_addr learned my addr 172.21.15.66:0/1464394898 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feaffe00640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0815e9b0 0x7feb080cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb07400640 1 -- 172.21.15.66:0/1464394898 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feaec003980 msgr2=0x7feb0815efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb07400640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feaec003980 0x7feb0815efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb07400640 1 -- 172.21.15.66:0/1464394898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0815e9b0 msgr2=0x7feb080cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb07400640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0815e9b0 0x7feb080cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb07400640 1 -- 172.21.15.66:0/1464394898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb080c5980 con 0x7feb081554b0 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb06a00640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feaec003980 0x7feb0815efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb07400640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb081554b0 0x7feb080cf580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb08071310 tx=0x7feb0000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feaff400640 1 -- 172.21.15.66:0/1464394898 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00005d50 con 0x7feb081554b0 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feaff400640 1 -- 172.21.15.66:0/1464394898 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0000ced0 con 0x7feb081554b0 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feaff400640 1 -- 172.21.15.66:0/1464394898 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0000a850 con 0x7feb081554b0 2024-03-30T00:40:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb0de00640 1 -- 172.21.15.66:0/1464394898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb080c5c60 con 0x7feb081554b0 2024-03-30T00:40:44.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb0de00640 1 -- 172.21.15.66:0/1464394898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0815f520 con 0x7feb081554b0 2024-03-30T00:40:44.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feb0de00640 1 -- 172.21.15.66:0/1464394898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb08066c30 con 0x7feb081554b0 2024-03-30T00:40:44.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.389+0000 7feaff400640 1 -- 172.21.15.66:0/1464394898 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7feb0000a9b0 con 0x7feb081554b0 2024-03-30T00:40:44.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.393+0000 7feaff400640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feae4066690 0x7feae4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.393+0000 7feaff400640 1 -- 172.21.15.66:0/1464394898 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7feb00083540 con 0x7feb081554b0 2024-03-30T00:40:44.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.393+0000 7feb06a00640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feae4066690 0x7feae4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.393+0000 7feb06a00640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feae4066690 0x7feae4068b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feaf8000c10 tx=0x7feaf80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:44.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.397+0000 7feaff400640 1 -- 172.21.15.66:0/1464394898 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7feb00050550 con 0x7feb081554b0 2024-03-30T00:40:44.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.553+0000 7feb0de00640 1 -- 172.21.15.66:0/1464394898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feb080c6110 con 0x7feb081554b0 2024-03-30T00:40:44.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.553+0000 7feaff400640 1 -- 172.21.15.66:0/1464394898 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7feb00053b70 con 0x7feb081554b0 2024-03-30T00:40:44.565 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 -- 172.21.15.66:0/1464394898 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feae4066690 msgr2=0x7feae4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feae4066690 0x7feae4068b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feaf8000c10 tx=0x7feaf80046d0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 -- 172.21.15.66:0/1464394898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb081554b0 msgr2=0x7feb080cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb081554b0 0x7feb080cf580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb08071310 tx=0x7feb0000cb90 comp rx=0 tx=0).stop 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 -- 172.21.15.66:0/1464394898 shutdown_connections 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feae4066690 0x7feae4068b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb081554b0 0x7feb080cf580 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0815e9b0 0x7feb080cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 --2- 172.21.15.66:0/1464394898 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feaec003980 0x7feb0815efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 -- 172.21.15.66:0/1464394898 >> 172.21.15.66:0/1464394898 conn(0x7feb080b20c0 msgr2=0x7feb08098a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 -- 172.21.15.66:0/1464394898 shutdown_connections 2024-03-30T00:40:44.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.557+0000 7feb0de00640 1 -- 172.21.15.66:0/1464394898 wait complete. 2024-03-30T00:40:44.614 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-30T00:40:44.615 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.745+0000 7f033f400640 1 Processor -- start 2024-03-30T00:40:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.745+0000 7f033f400640 1 -- start start 2024-03-30T00:40:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.745+0000 7f033f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03380cca40 0x7f03380cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.745+0000 7f033f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0338099960 0x7f033806b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.745+0000 7f033f400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033806bac0 con 0x7f033806b140 2024-03-30T00:40:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.745+0000 7f033f400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0338098ac0 con 0x7f0338099960 2024-03-30T00:40:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.745+0000 7f033f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03380cf3f0 con 0x7f03380cca40 2024-03-30T00:40:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f033d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0338099960 0x7f033806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f0337e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f033806b140 0x7f0338099250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42496/0 (socket says 172.21.15.66:42496) 2024-03-30T00:40:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f0337e00640 1 -- 172.21.15.66:0/2268731541 learned_addr learned my addr 172.21.15.66:0/2268731541 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f033ca00640 1 --2- 172.21.15.66:0/2268731541 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03380cca40 0x7f03380cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f033d400640 1 -- 172.21.15.66:0/2268731541 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03380cca40 msgr2=0x7f03380cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f0337400640 1 -- 172.21.15.66:0/2268731541 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264578831 0 0) 0x7f033806bac0 con 0x7f033806b140 2024-03-30T00:40:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f033d400640 1 --2- 172.21.15.66:0/2268731541 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03380cca40 0x7f03380cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f033d400640 1 -- 172.21.15.66:0/2268731541 >> v1:172.21.15.66:6789/0 conn(0x7f033806b140 legacy=0x7f0338099250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f033d400640 1 -- 172.21.15.66:0/2268731541 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0338099db0 con 0x7f0338099960 2024-03-30T00:40:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f033ca00640 1 --2- 172.21.15.66:0/2268731541 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03380cca40 0x7f03380cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:44.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f033d400640 1 --2- 172.21.15.66:0/2268731541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0338099960 0x7f033806b580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0330009d90 tx=0x7f033001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c46b4ea5b397d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:44.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f0337400640 1 -- 172.21.15.66:0/2268731541 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300a59c0 con 0x7f0338099960 2024-03-30T00:40:44.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f0337400640 1 -- 172.21.15.66:0/2268731541 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03300a5b20 con 0x7f0338099960 2024-03-30T00:40:44.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.749+0000 7f0337400640 1 -- 172.21.15.66:0/2268731541 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300ad440 con 0x7f0338099960 2024-03-30T00:40:44.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 -- 172.21.15.66:0/2268731541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0338099960 msgr2=0x7f033806b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 --2- 172.21.15.66:0/2268731541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0338099960 0x7f033806b580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0330009d90 tx=0x7f033001c900 comp rx=0 tx=0).stop 2024-03-30T00:40:44.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 -- 172.21.15.66:0/2268731541 shutdown_connections 2024-03-30T00:40:44.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 --2- 172.21.15.66:0/2268731541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0338099960 0x7f033806b580 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 --2- 172.21.15.66:0/2268731541 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03380cca40 0x7f03380cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 -- 172.21.15.66:0/2268731541 >> 172.21.15.66:0/2268731541 conn(0x7f03380b2130 msgr2=0x7f03380b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:44.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 -- 172.21.15.66:0/2268731541 shutdown_connections 2024-03-30T00:40:44.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 -- 172.21.15.66:0/2268731541 wait complete. 2024-03-30T00:40:44.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 Processor -- start 2024-03-30T00:40:44.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 -- start start 2024-03-30T00:40:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f033806b140 0x7f03380c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0338099960 0x7f03380c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03380cca40 0x7f03380c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338158a20 con 0x7f033806b140 2024-03-30T00:40:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03380ae240 con 0x7f03380cca40 2024-03-30T00:40:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03380ae520 con 0x7f0338099960 2024-03-30T00:40:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f033806b140 0x7f03380c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f033806b140 0x7f03380c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60858/0 (socket says 172.21.15.66:60858) 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033ca00640 1 -- 172.21.15.66:0/1025868632 learned_addr learned my addr 172.21.15.66:0/1025868632 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f0337e00640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0338099960 0x7f03380c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033d400640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03380cca40 0x7f03380c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033ca00640 1 -- 172.21.15.66:0/1025868632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0338099960 msgr2=0x7f03380c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033ca00640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0338099960 0x7f03380c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033ca00640 1 -- 172.21.15.66:0/1025868632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03380cca40 msgr2=0x7f03380c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033ca00640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03380cca40 0x7f03380c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033ca00640 1 -- 172.21.15.66:0/1025868632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0330009a70 con 0x7f033806b140 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f0337e00640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0338099960 0x7f03380c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033d400640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03380cca40 0x7f03380c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.753+0000 7f033ca00640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f033806b140 0x7f03380c7d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f032c00d800 tx=0x7f032c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.757+0000 7f0334c00640 1 -- 172.21.15.66:0/1025868632 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c0096c0 con 0x7f033806b140 2024-03-30T00:40:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.757+0000 7f0334c00640 1 -- 172.21.15.66:0/1025868632 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c009820 con 0x7f033806b140 2024-03-30T00:40:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.757+0000 7f0334c00640 1 -- 172.21.15.66:0/1025868632 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c009bd0 con 0x7f033806b140 2024-03-30T00:40:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.757+0000 7f033f400640 1 -- 172.21.15.66:0/1025868632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03380ae800 con 0x7f033806b140 2024-03-30T00:40:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.757+0000 7f033f400640 1 -- 172.21.15.66:0/1025868632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03380aedc0 con 0x7f033806b140 2024-03-30T00:40:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.757+0000 7f033f400640 1 -- 172.21.15.66:0/1025868632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03380677b0 con 0x7f033806b140 2024-03-30T00:40:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.757+0000 7f0334c00640 1 -- 172.21.15.66:0/1025868632 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f032c009d30 con 0x7f033806b140 2024-03-30T00:40:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.757+0000 7f0334c00640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0318066690 0x7f0318068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.757+0000 7f0334c00640 1 -- 172.21.15.66:0/1025868632 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f032c084c00 con 0x7f033806b140 2024-03-30T00:40:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.757+0000 7f0337e00640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0318066690 0x7f0318068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:44.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.761+0000 7f0337e00640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0318066690 0x7f0318068b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0324007fd0 tx=0x7f0324007c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.761+0000 7f0334c00640 1 -- 172.21.15.66:0/1025868632 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f032c08b080 con 0x7f033806b140 2024-03-30T00:40:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.925+0000 7f033f400640 1 -- 172.21.15.66:0/1025868632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0338066c30 con 0x7f033806b140 2024-03-30T00:40:44.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.925+0000 7f0334c00640 1 -- 172.21.15.66:0/1025868632 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7f032c051c00 con 0x7f033806b140 2024-03-30T00:40:44.936 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:44.936 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:44.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 -- 172.21.15.66:0/1025868632 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0318066690 msgr2=0x7f0318068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0318066690 0x7f0318068b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0324007fd0 tx=0x7f0324007c20 comp rx=0 tx=0).stop 2024-03-30T00:40:44.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 -- 172.21.15.66:0/1025868632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f033806b140 msgr2=0x7f03380c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:44.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f033806b140 0x7f03380c7d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f032c00d800 tx=0x7f032c00dcc0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 -- 172.21.15.66:0/1025868632 shutdown_connections 2024-03-30T00:40:44.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03380cca40 0x7f03380c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0338099960 0x7f03380c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0318066690 0x7f0318068b50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 --2- 172.21.15.66:0/1025868632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f033806b140 0x7f03380c7d50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:44.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 -- 172.21.15.66:0/1025868632 >> 172.21.15.66:0/1025868632 conn(0x7f03380b2130 msgr2=0x7f03380b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:44.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 -- 172.21.15.66:0/1025868632 shutdown_connections 2024-03-30T00:40:44.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:44.929+0000 7f033f400640 1 -- 172.21.15.66:0/1025868632 wait complete. 2024-03-30T00:40:44.957 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:44.957 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:45.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f0935c00640 1 Processor -- start 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f0935c00640 1 -- start start 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f0935c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300cda90 0x7f09300ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f0935c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f09300cde70 0x7f09300cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f0935c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093015a990 con 0x7f0930159c60 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f0935c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09300b1b80 con 0x7f09300cde70 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f0935c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09300cd3e0 con 0x7f09300cda90 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0930159c60 0x7f093015a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42500/0 (socket says 172.21.15.66:42500) 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092f400640 1 -- 172.21.15.66:0/1868945365 learned_addr learned my addr 172.21.15.66:0/1868945365 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092e000640 1 --2- 172.21.15.66:0/1868945365 >> v2:172.21.15.156:3300/0 conn(0x7f09300cde70 0x7f09300cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092ea00640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300cda90 0x7f09300ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470392118 0 0) 0x7f093015a990 con 0x7f0930159c60 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0914003530 con 0x7f0930159c60 2024-03-30T00:40:45.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 448256545 0 0) 0x7f0914003530 con 0x7f0930159c60 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300cda90 msgr2=0x7f09300ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300cda90 0x7f09300ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 >> v2:172.21.15.156:3300/0 conn(0x7f09300cde70 msgr2=0x7f09300cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 --2- 172.21.15.66:0/1868945365 >> v2:172.21.15.156:3300/0 conn(0x7f09300cde70 0x7f09300cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09300b11e0 con 0x7f0930159c60 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f0928001a00 con 0x7f0930159c60 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 >> v1:172.21.15.66:6789/0 conn(0x7f0930159c60 legacy=0x7f093015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0914003980 0x7f0914003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300cda90 0x7f09140047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0914004d10 0x7f0914005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09300b1b80 con 0x7f0914004d10 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09300cd3e0 con 0x7f0914003980 2024-03-30T00:40:45.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091400b9c0 con 0x7f09300cda90 2024-03-30T00:40:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092f400640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0914004d10 0x7f0914005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092ea00640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0914003980 0x7f0914003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092e000640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300cda90 0x7f09140047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092f400640 1 -- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300cda90 msgr2=0x7f09140047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092f400640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300cda90 0x7f09140047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092f400640 1 -- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0914003980 msgr2=0x7f0914003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092f400640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0914003980 0x7f0914003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092f400640 1 -- 172.21.15.66:0/1868945365 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914003530 con 0x7f0914004d10 2024-03-30T00:40:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.093+0000 7f092e000640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300cda90 0x7f09140047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.097+0000 7f092f400640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0914004d10 0x7f0914005190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f092800a780 tx=0x7f092801fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa46ceb4f4411b2c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.097+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09280aa630 con 0x7f0914004d10 2024-03-30T00:40:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.097+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09280aa790 con 0x7f0914004d10 2024-03-30T00:40:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.097+0000 7f092d600640 1 -- 172.21.15.66:0/1868945365 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09280aab40 con 0x7f0914004d10 2024-03-30T00:40:45.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.097+0000 7f0935c00640 1 -- 172.21.15.66:0/1868945365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0914004d10 msgr2=0x7f0914005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.097+0000 7f0935c00640 1 --2- 172.21.15.66:0/1868945365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0914004d10 0x7f0914005190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f092800a780 tx=0x7f092801fa00 comp rx=0 tx=0).stop 2024-03-30T00:40:45.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.097+0000 7f092f400640 1 -- 172.21.15.66:0/1868945365 reap_dead start 2024-03-30T00:40:45.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.097+0000 7f0935c00640 1 -- 172.21.15.66:0/1868945365 shutdown_connections 2024-03-30T00:40:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.097+0000 7f0935c00640 1 -- 172.21.15.66:0/1868945365 >> 172.21.15.66:0/1868945365 conn(0x7f0930069f60 msgr2=0x7f093006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.097+0000 7f0935c00640 1 -- 172.21.15.66:0/1868945365 shutdown_connections 2024-03-30T00:40:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.097+0000 7f0935c00640 1 -- 172.21.15.66:0/1868945365 wait complete. 2024-03-30T00:40:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0935c00640 1 Processor -- start 2024-03-30T00:40:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0935c00640 1 -- start start 2024-03-30T00:40:45.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0935c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0914004d10 0x7f0930156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0935c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09301566c0 0x7f09300c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0935c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300afdc0 0x7f09300b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0935c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0930156560 con 0x7f09301566c0 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0935c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09300b0780 con 0x7f0914004d10 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0935c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09300af440 con 0x7f09300afdc0 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f092e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09301566c0 0x7f09300c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f092e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09301566c0 0x7f09300c1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60880/0 (socket says 172.21.15.66:60880) 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f092e000640 1 -- 172.21.15.66:0/374796744 learned_addr learned my addr 172.21.15.66:0/374796744 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f092ea00640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0914004d10 0x7f0930156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f092f400640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300afdc0 0x7f09300b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f092e000640 1 -- 172.21.15.66:0/374796744 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300afdc0 msgr2=0x7f09300b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f092e000640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300afdc0 0x7f09300b0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f092e000640 1 -- 172.21.15.66:0/374796744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0914004d10 msgr2=0x7f0930156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f092e000640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0914004d10 0x7f0930156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f092e000640 1 -- 172.21.15.66:0/374796744 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093009ba20 con 0x7f09301566c0 2024-03-30T00:40:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f092e000640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09301566c0 0x7f09300c1e20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f091c011ca0 tx=0x7f091c00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0926a00640 1 -- 172.21.15.66:0/374796744 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c011e50 con 0x7f09301566c0 2024-03-30T00:40:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0926a00640 1 -- 172.21.15.66:0/374796744 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c004020 con 0x7f09301566c0 2024-03-30T00:40:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0926a00640 1 -- 172.21.15.66:0/374796744 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c0043d0 con 0x7f09301566c0 2024-03-30T00:40:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0935c00640 1 -- 172.21.15.66:0/374796744 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093009c190 con 0x7f09301566c0 2024-03-30T00:40:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0935c00640 1 -- 172.21.15.66:0/374796744 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09300af690 con 0x7f09301566c0 2024-03-30T00:40:45.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.101+0000 7f0935c00640 1 -- 172.21.15.66:0/374796744 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09300b1760 con 0x7f09301566c0 2024-03-30T00:40:45.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.105+0000 7f0926a00640 1 -- 172.21.15.66:0/374796744 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f091c003620 con 0x7f09301566c0 2024-03-30T00:40:45.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.105+0000 7f0926a00640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09100666c0 0x7f0910068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.105+0000 7f0926a00640 1 -- 172.21.15.66:0/374796744 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f091c084d20 con 0x7f09301566c0 2024-03-30T00:40:45.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.105+0000 7f092ea00640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09100666c0 0x7f0910068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.105+0000 7f092ea00640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09100666c0 0x7f0910068b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09300c96a0 tx=0x7f0920008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:45.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.109+0000 7f0926a00640 1 -- 172.21.15.66:0/374796744 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f091c051c80 con 0x7f09301566c0 2024-03-30T00:40:45.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 -- 172.21.15.66:0/374796744 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0930066c30 con 0x7f09301566c0 2024-03-30T00:40:45.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0926a00640 1 -- 172.21.15.66:0/374796744 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f091c022020 con 0x7f09301566c0 2024-03-30T00:40:45.281 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:45.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 -- 172.21.15.66:0/374796744 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09100666c0 msgr2=0x7f0910068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09100666c0 0x7f0910068b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09300c96a0 tx=0x7f0920008040 comp rx=0 tx=0).stop 2024-03-30T00:40:45.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 -- 172.21.15.66:0/374796744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09301566c0 msgr2=0x7f09300c1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09301566c0 0x7f09300c1e20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f091c011ca0 tx=0x7f091c00f610 comp rx=0 tx=0).stop 2024-03-30T00:40:45.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 -- 172.21.15.66:0/374796744 shutdown_connections 2024-03-30T00:40:45.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09300afdc0 0x7f09300b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09100666c0 0x7f0910068b80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09301566c0 0x7f09300c1e20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 --2- 172.21.15.66:0/374796744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0914004d10 0x7f0930156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 -- 172.21.15.66:0/374796744 >> 172.21.15.66:0/374796744 conn(0x7f0930069f60 msgr2=0x7f0930159240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:45.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 -- 172.21.15.66:0/374796744 shutdown_connections 2024-03-30T00:40:45.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.273+0000 7f0935c00640 1 -- 172.21.15.66:0/374796744 wait complete. 2024-03-30T00:40:45.302 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:45.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4fe00640 1 Processor -- start 2024-03-30T00:40:45.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4fe00640 1 -- start start 2024-03-30T00:40:45.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbe5015a720 0x7fbe500ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe500d0ed0 0x7fbe500b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4fe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe500cb5b0 con 0x7fbe500b3170 2024-03-30T00:40:45.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4fe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe500cfb00 con 0x7fbe5015a720 2024-03-30T00:40:45.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4fe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5015ab00 con 0x7fbe500d0ed0 2024-03-30T00:40:45.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbe5015a720 0x7fbe500ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe47e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fbe500b3170 0x7fbe500d07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42512/0 (socket says 172.21.15.66:42512) 2024-03-30T00:40:45.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe47e00640 1 -- 172.21.15.66:0/2956047956 learned_addr learned my addr 172.21.15.66:0/2956047956 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:45.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4f400640 1 --2- 172.21.15.66:0/2956047956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe500d0ed0 0x7fbe500b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4ea00640 1 -- 172.21.15.66:0/2956047956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe500d0ed0 msgr2=0x7fbe500b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:45.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4ea00640 1 --2- 172.21.15.66:0/2956047956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe500d0ed0 0x7fbe500b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4ea00640 1 -- 172.21.15.66:0/2956047956 >> v1:172.21.15.66:6789/0 conn(0x7fbe500b3170 legacy=0x7fbe500d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4ea00640 1 -- 172.21.15.66:0/2956047956 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500cb2f0 con 0x7fbe5015a720 2024-03-30T00:40:45.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4f400640 1 --2- 172.21.15.66:0/2956047956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe500d0ed0 0x7fbe500b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:45.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.437+0000 7fbe4e000640 1 -- 172.21.15.66:0/2956047956 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654484877 0 0) 0x7fbe38001120 con 0x7fbe500b3170 2024-03-30T00:40:45.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4ea00640 1 --2- 172.21.15.66:0/2956047956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5015a720 0x7fbe500ca970 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbe400097c0 tx=0x7fbe4001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d7a732b4e8367c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:45.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4e000640 1 -- 172.21.15.66:0/2956047956 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe400ab670 con 0x7fbe5015a720 2024-03-30T00:40:45.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4e000640 1 -- 172.21.15.66:0/2956047956 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe400ab7d0 con 0x7fbe5015a720 2024-03-30T00:40:45.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4e000640 1 -- 172.21.15.66:0/2956047956 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe400abb60 con 0x7fbe5015a720 2024-03-30T00:40:45.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4fe00640 1 -- 172.21.15.66:0/2956047956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5015a720 msgr2=0x7fbe500ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4fe00640 1 --2- 172.21.15.66:0/2956047956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5015a720 0x7fbe500ca970 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbe400097c0 tx=0x7fbe4001c470 comp rx=0 tx=0).stop 2024-03-30T00:40:45.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4fe00640 1 -- 172.21.15.66:0/2956047956 shutdown_connections 2024-03-30T00:40:45.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4fe00640 1 --2- 172.21.15.66:0/2956047956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe500d0ed0 0x7fbe500b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4fe00640 1 --2- 172.21.15.66:0/2956047956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5015a720 0x7fbe500ca970 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4fe00640 1 -- 172.21.15.66:0/2956047956 >> 172.21.15.66:0/2956047956 conn(0x7fbe50069d90 msgr2=0x7fbe5006a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:45.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4fe00640 1 -- 172.21.15.66:0/2956047956 shutdown_connections 2024-03-30T00:40:45.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.441+0000 7fbe4fe00640 1 -- 172.21.15.66:0/2956047956 wait complete. 2024-03-30T00:40:45.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4fe00640 1 Processor -- start 2024-03-30T00:40:45.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4fe00640 1 -- start start 2024-03-30T00:40:45.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe500b3170 0x7fbe5015c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4fe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe500d0ed0 0x7fbe5015e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5015a720 0x7fbe500c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5015d800 con 0x7fbe5015a720 2024-03-30T00:40:45.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4fe00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5015cbf0 con 0x7fbe500d0ed0 2024-03-30T00:40:45.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4fe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5015ced0 con 0x7fbe500b3170 2024-03-30T00:40:45.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5015a720 0x7fbe500c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5015a720 0x7fbe500c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60896/0 (socket says 172.21.15.66:60896) 2024-03-30T00:40:45.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4f400640 1 -- 172.21.15.66:0/3519457049 learned_addr learned my addr 172.21.15.66:0/3519457049 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:45.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4f400640 1 -- 172.21.15.66:0/3519457049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe500b3170 msgr2=0x7fbe5015c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4f400640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe500b3170 0x7fbe5015c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4f400640 1 -- 172.21.15.66:0/3519457049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe500d0ed0 msgr2=0x7fbe5015e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4f400640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe500d0ed0 0x7fbe5015e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4f400640 1 -- 172.21.15.66:0/3519457049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe40009420 con 0x7fbe5015a720 2024-03-30T00:40:45.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4f400640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5015a720 0x7fbe500c4e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800ce70 tx=0x7fbe4800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:45.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe46a00640 1 -- 172.21.15.66:0/3519457049 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48003ba0 con 0x7fbe5015a720 2024-03-30T00:40:45.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe46a00640 1 -- 172.21.15.66:0/3519457049 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe48003d20 con 0x7fbe5015a720 2024-03-30T00:40:45.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe46a00640 1 -- 172.21.15.66:0/3519457049 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4800d880 con 0x7fbe5015a720 2024-03-30T00:40:45.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4fe00640 1 -- 172.21.15.66:0/3519457049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5015d1b0 con 0x7fbe5015a720 2024-03-30T00:40:45.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.445+0000 7fbe4fe00640 1 -- 172.21.15.66:0/3519457049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5015c7e0 con 0x7fbe5015a720 2024-03-30T00:40:45.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.449+0000 7fbe4fe00640 1 -- 172.21.15.66:0/3519457049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe50073e00 con 0x7fbe5015a720 2024-03-30T00:40:45.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.449+0000 7fbe46a00640 1 -- 172.21.15.66:0/3519457049 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbe4800d9e0 con 0x7fbe5015a720 2024-03-30T00:40:45.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.449+0000 7fbe46a00640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe2c066690 0x7fbe2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.449+0000 7fbe46a00640 1 -- 172.21.15.66:0/3519457049 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fbe48084dc0 con 0x7fbe5015a720 2024-03-30T00:40:45.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.449+0000 7fbe4ea00640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe2c066690 0x7fbe2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.449+0000 7fbe4ea00640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe2c066690 0x7fbe2c068b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbe400af000 tx=0x7fbe40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:45.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.453+0000 7fbe46a00640 1 -- 172.21.15.66:0/3519457049 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbe48051dc0 con 0x7fbe5015a720 2024-03-30T00:40:45.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.613+0000 7fbe4fe00640 1 -- 172.21.15.66:0/3519457049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbe5015d940 con 0x7fbe5015a720 2024-03-30T00:40:45.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.613+0000 7fbe46a00640 1 -- 172.21.15.66:0/3519457049 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbe480553e0 con 0x7fbe5015a720 2024-03-30T00:40:45.623 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:45.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 -- 172.21.15.66:0/3519457049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe2c066690 msgr2=0x7fbe2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe2c066690 0x7fbe2c068b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbe400af000 tx=0x7fbe40008040 comp rx=0 tx=0).stop 2024-03-30T00:40:45.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 -- 172.21.15.66:0/3519457049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5015a720 msgr2=0x7fbe500c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5015a720 0x7fbe500c4e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800ce70 tx=0x7fbe4800a710 comp rx=0 tx=0).stop 2024-03-30T00:40:45.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 -- 172.21.15.66:0/3519457049 shutdown_connections 2024-03-30T00:40:45.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe2c066690 0x7fbe2c068b50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5015a720 0x7fbe500c4e80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe500d0ed0 0x7fbe5015e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 --2- 172.21.15.66:0/3519457049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe500b3170 0x7fbe5015c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 -- 172.21.15.66:0/3519457049 >> 172.21.15.66:0/3519457049 conn(0x7fbe50069d90 msgr2=0x7fbe500b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:45.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 -- 172.21.15.66:0/3519457049 shutdown_connections 2024-03-30T00:40:45.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.617+0000 7fbe4fe00640 1 -- 172.21.15.66:0/3519457049 wait complete. 2024-03-30T00:40:45.644 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:45.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.777+0000 7f1a09600640 1 Processor -- start 2024-03-30T00:40:45.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.777+0000 7f1a09600640 1 -- start start 2024-03-30T00:40:45.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a09600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a040acae0 0x7f1a040ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a09600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a040cef50 con 0x7f1a040ceb20 2024-03-30T00:40:45.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a09600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0406ccd0 con 0x7f1a0406d7e0 2024-03-30T00:40:45.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a09600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a040acec0 con 0x7f1a040acae0 2024-03-30T00:40:45.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a08c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1a040ceb20 0x7f1a0406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42514/0 (socket says 172.21.15.66:42514) 2024-03-30T00:40:45.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a08c00640 1 -- 172.21.15.66:0/3610943671 learned_addr learned my addr 172.21.15.66:0/3610943671 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:45.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a03e00640 1 --2- 172.21.15.66:0/3610943671 >> v2:172.21.15.156:3300/0 conn(0x7f1a040acae0 0x7f1a040ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430131341 0 0) 0x7f1a040cef50 con 0x7f1a040ceb20 2024-03-30T00:40:45.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e8003530 con 0x7f1a040ceb20 2024-03-30T00:40:45.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438533264 0 0) 0x7f1a0406ccd0 con 0x7f1a0406d7e0 2024-03-30T00:40:45.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a040cef50 con 0x7f1a0406d7e0 2024-03-30T00:40:45.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4051568020 0 0) 0x7f19e8003530 con 0x7f1a040ceb20 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 >> v2:172.21.15.156:3300/0 conn(0x7f1a040acae0 msgr2=0x7f1a040ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 --2- 172.21.15.66:0/3610943671 >> v2:172.21.15.156:3300/0 conn(0x7f1a040acae0 0x7f1a040ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 >> v1:172.21.15.156:6789/0 conn(0x7f1a0406d7e0 legacy=0x7f1a0406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0415de20 con 0x7f1a040ceb20 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f19fc001a00 con 0x7f1a040ceb20 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 >> v1:172.21.15.66:6789/0 conn(0x7f1a040ceb20 legacy=0x7f1a0406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 --2- 172.21.15.66:0/3610943671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e8003b10 0x7f19e8003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 --2- 172.21.15.66:0/3610943671 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e80047d0 0x7f19e8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 --2- 172.21.15.66:0/3610943671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 0x7f19e800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040acec0 con 0x7f19e8005190 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e800c010 con 0x7f19e8003b10 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e800c1d0 con 0x7f19e80047d0 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a03e00640 1 --2- 172.21.15.66:0/3610943671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e8003b10 0x7f19e8003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a08c00640 1 --2- 172.21.15.66:0/3610943671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 0x7f19e800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a03400640 1 --2- 172.21.15.66:0/3610943671 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e80047d0 0x7f19e8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a08c00640 1 -- 172.21.15.66:0/3610943671 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e80047d0 msgr2=0x7f19e8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:45.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a08c00640 1 --2- 172.21.15.66:0/3610943671 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e80047d0 0x7f19e8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a08c00640 1 -- 172.21.15.66:0/3610943671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e8003b10 msgr2=0x7f19e8003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:45.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a08c00640 1 --2- 172.21.15.66:0/3610943671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e8003b10 0x7f19e8003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.781+0000 7f1a08c00640 1 -- 172.21.15.66:0/3610943671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0415de20 con 0x7f19e8005190 2024-03-30T00:40:45.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a08c00640 1 -- 172.21.15.66:0/3610943671 reap_dead start 2024-03-30T00:40:45.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a08c00640 1 --2- 172.21.15.66:0/3610943671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 0x7f19e800b990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00a870 tx=0x7f19fc01fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab82965966eaabd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:45.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0aa690 con 0x7f19e8005190 2024-03-30T00:40:45.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc0aa810 con 0x7f19e8005190 2024-03-30T00:40:45.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a02a00640 1 -- 172.21.15.66:0/3610943671 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0aabc0 con 0x7f19e8005190 2024-03-30T00:40:45.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a09600640 1 -- 172.21.15.66:0/3610943671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 msgr2=0x7f19e800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a09600640 1 --2- 172.21.15.66:0/3610943671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 0x7f19e800b990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00a870 tx=0x7f19fc01fa60 comp rx=0 tx=0).stop 2024-03-30T00:40:45.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a09600640 1 -- 172.21.15.66:0/3610943671 shutdown_connections 2024-03-30T00:40:45.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a09600640 1 --2- 172.21.15.66:0/3610943671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 0x7f19e800b990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a09600640 1 -- 172.21.15.66:0/3610943671 >> 172.21.15.66:0/3610943671 conn(0x7f1a0406a190 msgr2=0x7f1a0406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:45.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a09600640 1 -- 172.21.15.66:0/3610943671 shutdown_connections 2024-03-30T00:40:45.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a09600640 1 -- 172.21.15.66:0/3610943671 wait complete. 2024-03-30T00:40:45.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a09600640 1 Processor -- start 2024-03-30T00:40:45.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.785+0000 7f1a09600640 1 -- start start 2024-03-30T00:40:45.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a09600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 0x7f1a040976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a09600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a040c4450 0x7f1a040c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a09600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a040b1640 0x7f1a040b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a09600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040b2000 con 0x7f19e8005190 2024-03-30T00:40:45.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a09600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040c4890 con 0x7f1a040b1640 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a09600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040c3030 con 0x7f1a040c4450 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a03e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 0x7f1a040976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a03e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 0x7f1a040976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60906/0 (socket says 172.21.15.66:60906) 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a03e00640 1 -- 172.21.15.66:0/951035501 learned_addr learned my addr 172.21.15.66:0/951035501 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a08c00640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a040b1640 0x7f1a040b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a03400640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a040c4450 0x7f1a040c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a03e00640 1 -- 172.21.15.66:0/951035501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a040c4450 msgr2=0x7f1a040c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a03e00640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a040c4450 0x7f1a040c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a03e00640 1 -- 172.21.15.66:0/951035501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a040b1640 msgr2=0x7f1a040b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a03e00640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a040b1640 0x7f1a040b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a03e00640 1 -- 172.21.15.66:0/951035501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a041f2b90 con 0x7f19e8005190 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a03400640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a040c4450 0x7f1a040c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a03e00640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 0x7f1a040976a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f19f40119c0 tx=0x7f19f4011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:45.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f19fbe00640 1 -- 172.21.15.66:0/951035501 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f400fa90 con 0x7f19e8005190 2024-03-30T00:40:45.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f19fbe00640 1 -- 172.21.15.66:0/951035501 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f400fbf0 con 0x7f19e8005190 2024-03-30T00:40:45.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f19fbe00640 1 -- 172.21.15.66:0/951035501 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4014440 con 0x7f19e8005190 2024-03-30T00:40:45.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a09600640 1 -- 172.21.15.66:0/951035501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a041f1cc0 con 0x7f19e8005190 2024-03-30T00:40:45.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a09600640 1 -- 172.21.15.66:0/951035501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a041f2260 con 0x7f19e8005190 2024-03-30T00:40:45.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f1a09600640 1 -- 172.21.15.66:0/951035501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a04073e00 con 0x7f19e8005190 2024-03-30T00:40:45.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.789+0000 7f19fbe00640 1 -- 172.21.15.66:0/951035501 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f19f40145a0 con 0x7f19e8005190 2024-03-30T00:40:45.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.793+0000 7f19fbe00640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e40666c0 0x7f19e4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:45.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.793+0000 7f19fbe00640 1 -- 172.21.15.66:0/951035501 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f19f4084e20 con 0x7f19e8005190 2024-03-30T00:40:45.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.793+0000 7f1a03400640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e40666c0 0x7f19e4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:45.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.793+0000 7f1a03400640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e40666c0 0x7f19e4068b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a040b2f10 tx=0x7f19f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:45.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.797+0000 7f19fbe00640 1 -- 172.21.15.66:0/951035501 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f19f4051c60 con 0x7f19e8005190 2024-03-30T00:40:45.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.957+0000 7f1a09600640 1 -- 172.21.15.66:0/951035501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a04070d80 con 0x7f19e8005190 2024-03-30T00:40:45.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.957+0000 7f19fbe00640 1 -- 172.21.15.66:0/951035501 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f19f4022070 con 0x7f19e8005190 2024-03-30T00:40:45.967 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:45.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 -- 172.21.15.66:0/951035501 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e40666c0 msgr2=0x7f19e4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e40666c0 0x7f19e4068b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a040b2f10 tx=0x7f19f0009040 comp rx=0 tx=0).stop 2024-03-30T00:40:45.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 -- 172.21.15.66:0/951035501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 msgr2=0x7f1a040976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:45.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 0x7f1a040976a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f19f40119c0 tx=0x7f19f4011e80 comp rx=0 tx=0).stop 2024-03-30T00:40:45.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 -- 172.21.15.66:0/951035501 shutdown_connections 2024-03-30T00:40:45.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a040b1640 0x7f1a040b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a040c4450 0x7f1a040c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e40666c0 0x7f19e4068b80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 --2- 172.21.15.66:0/951035501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e8005190 0x7f1a040976a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:45.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 -- 172.21.15.66:0/951035501 >> 172.21.15.66:0/951035501 conn(0x7f1a0406a190 msgr2=0x7f1a040705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:45.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 -- 172.21.15.66:0/951035501 shutdown_connections 2024-03-30T00:40:45.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:45.961+0000 7f1a09600640 1 -- 172.21.15.66:0/951035501 wait complete. 2024-03-30T00:40:45.970 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:40:45.973 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:40:45.990 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:46.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5193000640 1 Processor -- start 2024-03-30T00:40:46.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5193000640 1 -- start start 2024-03-30T00:40:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5193000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518c0cc560 0x7f518c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5193000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f518c06f8e0 0x7f518c06e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5193000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f518c06e630 0x7f518c0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5193000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c160640 con 0x7f518c06e630 2024-03-30T00:40:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5193000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c06eaa0 con 0x7f518c06f8e0 2024-03-30T00:40:46.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5193000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c06b0b0 con 0x7f518c0cc560 2024-03-30T00:40:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5191000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f518c06e630 0x7f518c0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5191000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f518c06e630 0x7f518c0ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60922/0 (socket says 172.21.15.66:60922) 2024-03-30T00:40:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5191000640 1 -- 172.21.15.66:0/4087124508 learned_addr learned my addr 172.21.15.66:0/4087124508 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:46.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5183400640 1 --2- 172.21.15.66:0/4087124508 >> v2:172.21.15.156:3300/0 conn(0x7f518c06f8e0 0x7f518c06e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f518be00640 1 --2- 172.21.15.66:0/4087124508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518c0cc560 0x7f518c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5191000640 1 -- 172.21.15.66:0/4087124508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518c0cc560 msgr2=0x7f518c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5191000640 1 --2- 172.21.15.66:0/4087124508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518c0cc560 0x7f518c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5191000640 1 -- 172.21.15.66:0/4087124508 >> v2:172.21.15.156:3300/0 conn(0x7f518c06f8e0 msgr2=0x7f518c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5191000640 1 --2- 172.21.15.66:0/4087124508 >> v2:172.21.15.156:3300/0 conn(0x7f518c06f8e0 0x7f518c06e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.201+0000 7f5191000640 1 -- 172.21.15.66:0/4087124508 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c06b390 con 0x7f518c06e630 2024-03-30T00:40:46.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f5191000640 1 --2- 172.21.15.66:0/4087124508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f518c06e630 0x7f518c0ccdc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f518c0b3100 tx=0x7f518401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe2e1fc093ca878 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:46.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f518b400640 1 -- 172.21.15.66:0/4087124508 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51840ab600 con 0x7f518c06e630 2024-03-30T00:40:46.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f518b400640 1 -- 172.21.15.66:0/4087124508 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51840a7050 con 0x7f518c06e630 2024-03-30T00:40:46.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f518b400640 1 -- 172.21.15.66:0/4087124508 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51840ab980 con 0x7f518c06e630 2024-03-30T00:40:46.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f5193000640 1 -- 172.21.15.66:0/4087124508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f518c06e630 msgr2=0x7f518c0ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:46.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f5193000640 1 --2- 172.21.15.66:0/4087124508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f518c06e630 0x7f518c0ccdc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f518c0b3100 tx=0x7f518401c230 comp rx=0 tx=0).stop 2024-03-30T00:40:46.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f5193000640 1 -- 172.21.15.66:0/4087124508 shutdown_connections 2024-03-30T00:40:46.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f5193000640 1 --2- 172.21.15.66:0/4087124508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f518c06e630 0x7f518c0ccdc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f5193000640 1 --2- 172.21.15.66:0/4087124508 >> v2:172.21.15.156:3300/0 conn(0x7f518c06f8e0 0x7f518c06e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f5193000640 1 --2- 172.21.15.66:0/4087124508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518c0cc560 0x7f518c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f5193000640 1 -- 172.21.15.66:0/4087124508 >> 172.21.15.66:0/4087124508 conn(0x7f518c0d0540 msgr2=0x7f518c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:46.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f5193000640 1 -- 172.21.15.66:0/4087124508 shutdown_connections 2024-03-30T00:40:46.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.205+0000 7f5193000640 1 -- 172.21.15.66:0/4087124508 wait complete. 2024-03-30T00:40:46.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5193000640 1 Processor -- start 2024-03-30T00:40:46.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5193000640 1 -- start start 2024-03-30T00:40:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5193000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f518c06e630 0x7f518c0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5193000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f518c06f8e0 0x7f518c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5193000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518c0cc560 0x7f518c0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5193000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c15c3e0 con 0x7f518c06e630 2024-03-30T00:40:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5193000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c15c6c0 con 0x7f518c06f8e0 2024-03-30T00:40:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5193000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c15c9a0 con 0x7f518c0cc560 2024-03-30T00:40:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f518be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f518c06e630 0x7f518c0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f518be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f518c06e630 0x7f518c0cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60926/0 (socket says 172.21.15.66:60926) 2024-03-30T00:40:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f518be00640 1 -- 172.21.15.66:0/348973479 learned_addr learned my addr 172.21.15.66:0/348973479 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5191000640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518c0cc560 0x7f518c0b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5183400640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f518c06f8e0 0x7f518c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f518be00640 1 -- 172.21.15.66:0/348973479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518c0cc560 msgr2=0x7f518c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f518be00640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518c0cc560 0x7f518c0b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f518be00640 1 -- 172.21.15.66:0/348973479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f518c06f8e0 msgr2=0x7f518c0cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f518be00640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f518c06f8e0 0x7f518c0cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f518be00640 1 -- 172.21.15.66:0/348973479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51840094d0 con 0x7f518c06e630 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5191000640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518c0cc560 0x7f518c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5183400640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f518c06f8e0 0x7f518c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f518be00640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f518c06e630 0x7f518c0cb330 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f517c007e40 tx=0x7f517c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5188c00640 1 -- 172.21.15.66:0/348973479 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c014620 con 0x7f518c06e630 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5188c00640 1 -- 172.21.15.66:0/348973479 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c00ee90 con 0x7f518c06e630 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5188c00640 1 -- 172.21.15.66:0/348973479 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c0149c0 con 0x7f518c06e630 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5193000640 1 -- 172.21.15.66:0/348973479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c15cc80 con 0x7f518c06e630 2024-03-30T00:40:46.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.209+0000 7f5193000640 1 -- 172.21.15.66:0/348973479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c15d480 con 0x7f518c06e630 2024-03-30T00:40:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.213+0000 7f5193000640 1 -- 172.21.15.66:0/348973479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518c066c30 con 0x7f518c06e630 2024-03-30T00:40:46.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.213+0000 7f5188c00640 1 -- 172.21.15.66:0/348973479 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f517c014b20 con 0x7f518c06e630 2024-03-30T00:40:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.213+0000 7f5188c00640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 0x7f5168068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.213+0000 7f5188c00640 1 -- 172.21.15.66:0/348973479 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f517c084f10 con 0x7f518c06e630 2024-03-30T00:40:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.213+0000 7f5183400640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 0x7f5168068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.213+0000 7f5183400640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 0x7f5168068b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f518c0af450 tx=0x7f5174008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:46.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.217+0000 7f5188c00640 1 -- 172.21.15.66:0/348973479 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f517c051e80 con 0x7f518c06e630 2024-03-30T00:40:46.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.377+0000 7f5193000640 1 -- 172.21.15.66:0/348973479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f518c06cc90 con 0x7f518c06e630 2024-03-30T00:40:46.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.377+0000 7f5188c00640 1 -- 172.21.15.66:0/348973479 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f517c0554a0 con 0x7f518c06e630 2024-03-30T00:40:46.387 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:46.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.377+0000 7f5193000640 1 -- 172.21.15.66:0/348973479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 msgr2=0x7f5168068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:46.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.377+0000 7f5193000640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 0x7f5168068b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f518c0af450 tx=0x7f5174008040 comp rx=0 tx=0).stop 2024-03-30T00:40:46.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.377+0000 7f5193000640 1 -- 172.21.15.66:0/348973479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f518c06e630 msgr2=0x7f518c0cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:46.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.377+0000 7f5193000640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f518c06e630 0x7f518c0cb330 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f517c007e40 tx=0x7f517c00e4b0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.381+0000 7f5193000640 1 -- 172.21.15.66:0/348973479 shutdown_connections 2024-03-30T00:40:46.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.381+0000 7f5193000640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518c0cc560 0x7f518c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.381+0000 7f5193000640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f518c06f8e0 0x7f518c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.381+0000 7f5193000640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 0x7f5168068b50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.381+0000 7f5193000640 1 --2- 172.21.15.66:0/348973479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f518c06e630 0x7f518c0cb330 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.381+0000 7f5193000640 1 -- 172.21.15.66:0/348973479 >> 172.21.15.66:0/348973479 conn(0x7f518c0d0540 msgr2=0x7f518c0d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:46.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.381+0000 7f5193000640 1 -- 172.21.15.66:0/348973479 shutdown_connections 2024-03-30T00:40:46.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.381+0000 7f5193000640 1 -- 172.21.15.66:0/348973479 wait complete. 2024-03-30T00:40:46.408 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-03-30T00:40:46.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.537+0000 7efe7d600640 1 Processor -- start 2024-03-30T00:40:46.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.537+0000 7efe7d600640 1 -- start start 2024-03-30T00:40:46.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe7d600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efe780cda90 0x7efe780ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe7d600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe780cd3e0 con 0x7efe7815a370 2024-03-30T00:40:46.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe7d600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7815a990 con 0x7efe780cde70 2024-03-30T00:40:46.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe7d600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe780b1b80 con 0x7efe780cda90 2024-03-30T00:40:46.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe76a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efe780cda90 0x7efe780ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe77400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efe7815a370 0x7efe78156910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42518/0 (socket says 172.21.15.66:42518) 2024-03-30T00:40:46.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe77400640 1 -- 172.21.15.66:0/314408398 learned_addr learned my addr 172.21.15.66:0/314408398 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:46.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe76a00640 1 -- 172.21.15.66:0/314408398 >> v1:172.21.15.156:6789/0 conn(0x7efe780cde70 legacy=0x7efe78159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:46.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe76a00640 1 -- 172.21.15.66:0/314408398 >> v1:172.21.15.66:6789/0 conn(0x7efe7815a370 legacy=0x7efe78156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:46.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe76a00640 1 -- 172.21.15.66:0/314408398 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe780b11e0 con 0x7efe780cda90 2024-03-30T00:40:46.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe76000640 1 -- 172.21.15.66:0/314408398 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156441532 0 0) 0x7efe780cd3e0 con 0x7efe7815a370 2024-03-30T00:40:46.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe76a00640 1 --2- 172.21.15.66:0/314408398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe780cda90 0x7efe780ce520 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe680097c0 tx=0x7efe6801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f1abd9f8d00867 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:46.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe76000640 1 -- 172.21.15.66:0/314408398 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe680ab600 con 0x7efe780cda90 2024-03-30T00:40:46.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe76000640 1 -- 172.21.15.66:0/314408398 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe680a7050 con 0x7efe780cda90 2024-03-30T00:40:46.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.541+0000 7efe76000640 1 -- 172.21.15.66:0/314408398 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe680ab980 con 0x7efe780cda90 2024-03-30T00:40:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 -- 172.21.15.66:0/314408398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe780cda90 msgr2=0x7efe780ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 --2- 172.21.15.66:0/314408398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe780cda90 0x7efe780ce520 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe680097c0 tx=0x7efe6801c3d0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 -- 172.21.15.66:0/314408398 shutdown_connections 2024-03-30T00:40:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 --2- 172.21.15.66:0/314408398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe780cda90 0x7efe780ce520 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 -- 172.21.15.66:0/314408398 >> 172.21.15.66:0/314408398 conn(0x7efe78069f60 msgr2=0x7efe7806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:46.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 -- 172.21.15.66:0/314408398 shutdown_connections 2024-03-30T00:40:46.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 -- 172.21.15.66:0/314408398 wait complete. 2024-03-30T00:40:46.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 Processor -- start 2024-03-30T00:40:46.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 -- start start 2024-03-30T00:40:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe780cda90 0x7efe780acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe780cde70 0x7efe78151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7815a370 0x7efe78151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe781f33c0 con 0x7efe780cda90 2024-03-30T00:40:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe78155af0 con 0x7efe7815a370 2024-03-30T00:40:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe7d600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe78155dd0 con 0x7efe780cde70 2024-03-30T00:40:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe76a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe780cda90 0x7efe780acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe76a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe780cda90 0x7efe780acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60948/0 (socket says 172.21.15.66:60948) 2024-03-30T00:40:46.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe76a00640 1 -- 172.21.15.66:0/200977914 learned_addr learned my addr 172.21.15.66:0/200977914 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe6e000640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe780cde70 0x7efe78151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe77400640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7815a370 0x7efe78151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe76a00640 1 -- 172.21.15.66:0/200977914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe780cde70 msgr2=0x7efe78151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe76a00640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe780cde70 0x7efe78151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe76a00640 1 -- 172.21.15.66:0/200977914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7815a370 msgr2=0x7efe78151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe76a00640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7815a370 0x7efe78151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe76a00640 1 -- 172.21.15.66:0/200977914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe68009420 con 0x7efe780cda90 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.545+0000 7efe76a00640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe780cda90 0x7efe780acac0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efe680ae040 tx=0x7efe6801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.549+0000 7efe6f400640 1 -- 172.21.15.66:0/200977914 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe680b7650 con 0x7efe780cda90 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.549+0000 7efe7d600640 1 -- 172.21.15.66:0/200977914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe781560b0 con 0x7efe780cda90 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.549+0000 7efe6f400640 1 -- 172.21.15.66:0/200977914 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe680b77d0 con 0x7efe780cda90 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.549+0000 7efe6f400640 1 -- 172.21.15.66:0/200977914 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe680b7b80 con 0x7efe780cda90 2024-03-30T00:40:46.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.549+0000 7efe7d600640 1 -- 172.21.15.66:0/200977914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe78156670 con 0x7efe780cda90 2024-03-30T00:40:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.549+0000 7efe7d600640 1 -- 172.21.15.66:0/200977914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe78066c30 con 0x7efe780cda90 2024-03-30T00:40:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.549+0000 7efe6f400640 1 -- 172.21.15.66:0/200977914 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efe680b7ce0 con 0x7efe780cda90 2024-03-30T00:40:46.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.549+0000 7efe6f400640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe54066690 0x7efe54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:46.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.549+0000 7efe6f400640 1 -- 172.21.15.66:0/200977914 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7efe6811eeb0 con 0x7efe780cda90 2024-03-30T00:40:46.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.549+0000 7efe6e000640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe54066690 0x7efe54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:46.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.549+0000 7efe6e000640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe54066690 0x7efe54068b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe780723d0 tx=0x7efe60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:46.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.553+0000 7efe6f400640 1 -- 172.21.15.66:0/200977914 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efe680ebd00 con 0x7efe780cda90 2024-03-30T00:40:46.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:46.777+0000 7efe7d600640 1 -- 172.21.15.66:0/200977914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7efe78156860 con 0x7efe780cda90 2024-03-30T00:40:47.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.649+0000 7efe6f400640 1 -- 172.21.15.66:0/200977914 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v74) v1 ==== 145+0+0 (secure 0 0 0) 0x7efe680bb0d0 con 0x7efe780cda90 2024-03-30T00:40:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 -- 172.21.15.66:0/200977914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe54066690 msgr2=0x7efe54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe54066690 0x7efe54068b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe780723d0 tx=0x7efe60009040 comp rx=0 tx=0).stop 2024-03-30T00:40:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 -- 172.21.15.66:0/200977914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe780cda90 msgr2=0x7efe780acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe780cda90 0x7efe780acac0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efe680ae040 tx=0x7efe6801ac10 comp rx=0 tx=0).stop 2024-03-30T00:40:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 -- 172.21.15.66:0/200977914 shutdown_connections 2024-03-30T00:40:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7815a370 0x7efe78151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe780cde70 0x7efe78151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe54066690 0x7efe54068b50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:47.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 --2- 172.21.15.66:0/200977914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe780cda90 0x7efe780acac0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:47.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 -- 172.21.15.66:0/200977914 >> 172.21.15.66:0/200977914 conn(0x7efe78069f60 msgr2=0x7efe7815ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:47.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 -- 172.21.15.66:0/200977914 shutdown_connections 2024-03-30T00:40:47.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.653+0000 7efe7d600640 1 -- 172.21.15.66:0/200977914 wait complete. 2024-03-30T00:40:47.685 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-30T00:40:47.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f158de00640 1 Processor -- start 2024-03-30T00:40:47.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f158de00640 1 -- start start 2024-03-30T00:40:47.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f158de00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f15880cc8d0 0x7f158806c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:47.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f158de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15880ca660 con 0x7f15880cd2d0 2024-03-30T00:40:47.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f158de00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15880ca940 con 0x7f15880cad00 2024-03-30T00:40:47.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f158de00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15880cbec0 con 0x7f15880cc8d0 2024-03-30T00:40:47.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f157e000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f15880cd2d0 0x7f15880cb0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42530/0 (socket says 172.21.15.66:42530) 2024-03-30T00:40:47.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f157e000640 1 -- 172.21.15.66:0/2187103523 learned_addr learned my addr 172.21.15.66:0/2187103523 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:47.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f1587400640 1 --2- 172.21.15.66:0/2187103523 >> v2:172.21.15.156:3300/0 conn(0x7f15880cc8d0 0x7f158806c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:47.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f1586000640 1 -- 172.21.15.66:0/2187103523 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618548901 0 0) 0x7f15880ca660 con 0x7f15880cd2d0 2024-03-30T00:40:47.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f1586000640 1 -- 172.21.15.66:0/2187103523 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f156c003530 con 0x7f15880cd2d0 2024-03-30T00:40:47.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f1586000640 1 -- 172.21.15.66:0/2187103523 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225762554 0 0) 0x7f15880ca940 con 0x7f15880cad00 2024-03-30T00:40:47.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f1586000640 1 -- 172.21.15.66:0/2187103523 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15880ca660 con 0x7f15880cad00 2024-03-30T00:40:47.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f1587400640 1 -- 172.21.15.66:0/2187103523 >> v1:172.21.15.156:6789/0 conn(0x7f15880cad00 legacy=0x7f15880c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:47.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f1586000640 1 -- 172.21.15.66:0/2187103523 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3672136753 0 0) 0x7f156c003530 con 0x7f15880cd2d0 2024-03-30T00:40:47.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f1587400640 1 -- 172.21.15.66:0/2187103523 >> v1:172.21.15.66:6789/0 conn(0x7f15880cd2d0 legacy=0x7f15880cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:47.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.825+0000 7f1587400640 1 -- 172.21.15.66:0/2187103523 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880cc0d0 con 0x7f15880cc8d0 2024-03-30T00:40:47.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f1587400640 1 --2- 172.21.15.66:0/2187103523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15880cc8d0 0x7f158806c030 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f15800097c0 tx=0x7f158001c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0acaefa0f77d864 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:47.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f1586000640 1 -- 172.21.15.66:0/2187103523 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570001680 con 0x7f15880cc8d0 2024-03-30T00:40:47.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f1586000640 1 -- 172.21.15.66:0/2187103523 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15800a7050 con 0x7f15880cc8d0 2024-03-30T00:40:47.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f1586000640 1 -- 172.21.15.66:0/2187103523 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15800ab600 con 0x7f15880cc8d0 2024-03-30T00:40:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f158de00640 1 -- 172.21.15.66:0/2187103523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15880cc8d0 msgr2=0x7f158806c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f158de00640 1 --2- 172.21.15.66:0/2187103523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15880cc8d0 0x7f158806c030 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f15800097c0 tx=0x7f158001c390 comp rx=0 tx=0).stop 2024-03-30T00:40:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f158de00640 1 -- 172.21.15.66:0/2187103523 shutdown_connections 2024-03-30T00:40:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f158de00640 1 --2- 172.21.15.66:0/2187103523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15880cc8d0 0x7f158806c030 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f158de00640 1 -- 172.21.15.66:0/2187103523 >> 172.21.15.66:0/2187103523 conn(0x7f1588069f60 msgr2=0x7f158806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f158de00640 1 -- 172.21.15.66:0/2187103523 shutdown_connections 2024-03-30T00:40:47.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f158de00640 1 -- 172.21.15.66:0/2187103523 wait complete. 2024-03-30T00:40:47.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f158de00640 1 Processor -- start 2024-03-30T00:40:47.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.829+0000 7f158de00640 1 -- start start 2024-03-30T00:40:47.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f158de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15880cad00 0x7f1588159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f158de00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15880cc8d0 0x7f158815bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f158de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15880cd2d0 0x7f15880c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f158de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15881633e0 con 0x7f15880cd2d0 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f158de00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15881636c0 con 0x7f15880cc8d0 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f158de00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1588159b60 con 0x7f15880cad00 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f1587400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15880cd2d0 0x7f15880c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f1587400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15880cd2d0 0x7f15880c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60964/0 (socket says 172.21.15.66:60964) 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f1587400640 1 -- 172.21.15.66:0/1192841989 learned_addr learned my addr 172.21.15.66:0/1192841989 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f157e000640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15880cc8d0 0x7f158815bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f1586a00640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15880cad00 0x7f1588159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f1587400640 1 -- 172.21.15.66:0/1192841989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15880cad00 msgr2=0x7f1588159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f1587400640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15880cad00 0x7f1588159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f1587400640 1 -- 172.21.15.66:0/1192841989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15880cc8d0 msgr2=0x7f158815bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f1587400640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15880cc8d0 0x7f158815bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:47.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f1587400640 1 -- 172.21.15.66:0/1192841989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1580009420 con 0x7f15880cd2d0 2024-03-30T00:40:47.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f1586a00640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15880cad00 0x7f1588159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:47.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f1587400640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15880cd2d0 0x7f15880c37b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f15800097c0 tx=0x7f1580019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:47.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f157fe00640 1 -- 172.21.15.66:0/1192841989 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580006e20 con 0x7f15880cd2d0 2024-03-30T00:40:47.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f157fe00640 1 -- 172.21.15.66:0/1192841989 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15800b4400 con 0x7f15880cd2d0 2024-03-30T00:40:47.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f157fe00640 1 -- 172.21.15.66:0/1192841989 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15800b4770 con 0x7f15880cd2d0 2024-03-30T00:40:47.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f158de00640 1 -- 172.21.15.66:0/1192841989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15881639a0 con 0x7f15880cd2d0 2024-03-30T00:40:47.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f158de00640 1 -- 172.21.15.66:0/1192841989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15880c3ec0 con 0x7f15880cd2d0 2024-03-30T00:40:47.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f158de00640 1 -- 172.21.15.66:0/1192841989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158815c2c0 con 0x7f15880cd2d0 2024-03-30T00:40:47.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.833+0000 7f157fe00640 1 -- 172.21.15.66:0/1192841989 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f15800b2040 con 0x7f15880cd2d0 2024-03-30T00:40:47.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.837+0000 7f157fe00640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1564066690 0x7f1564068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:47.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.837+0000 7f157fe00640 1 -- 172.21.15.66:0/1192841989 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f158011e2c0 con 0x7f15880cd2d0 2024-03-30T00:40:47.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.837+0000 7f1586a00640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1564066690 0x7f1564068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:47.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.837+0000 7f1586a00640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1564066690 0x7f1564068b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1578001570 tx=0x7f1578009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:47.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:47.841+0000 7f157fe00640 1 -- 172.21.15.66:0/1192841989 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f15800eb2c0 con 0x7f15880cd2d0 2024-03-30T00:40:48.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.001+0000 7f158de00640 1 -- 172.21.15.66:0/1192841989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1588163520 con 0x7f15880cd2d0 2024-03-30T00:40:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.005+0000 7f157fe00640 1 -- 172.21.15.66:0/1192841989 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1588163520 con 0x7f15880cd2d0 2024-03-30T00:40:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 -- 172.21.15.66:0/1192841989 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1564066690 msgr2=0x7f1564068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1564066690 0x7f1564068b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1578001570 tx=0x7f1578009040 comp rx=0 tx=0).stop 2024-03-30T00:40:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 -- 172.21.15.66:0/1192841989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15880cd2d0 msgr2=0x7f15880c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15880cd2d0 0x7f15880c37b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f15800097c0 tx=0x7f1580019130 comp rx=0 tx=0).stop 2024-03-30T00:40:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 -- 172.21.15.66:0/1192841989 shutdown_connections 2024-03-30T00:40:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1564066690 0x7f1564068b50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15880cd2d0 0x7f15880c37b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15880cc8d0 0x7f158815bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 --2- 172.21.15.66:0/1192841989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15880cad00 0x7f1588159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 -- 172.21.15.66:0/1192841989 >> 172.21.15.66:0/1192841989 conn(0x7f1588069f60 msgr2=0x7f15880adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 -- 172.21.15.66:0/1192841989 shutdown_connections 2024-03-30T00:40:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.009+0000 7f158de00640 1 -- 172.21.15.66:0/1192841989 wait complete. 2024-03-30T00:40:48.038 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:40:48.038 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:40:48.135 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:40:48.215 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:40:48.217 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:40:48.217 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:40:48.227 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:40:48.227 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:40:48.277 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39bbc00640 1 Processor -- start 2024-03-30T00:40:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39bbc00640 1 -- start start 2024-03-30T00:40:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39bbc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f39b4069d90 0x7f39b406a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39bbc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f39b40cb6e0 0x7f39b40cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39bbc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b406a730 con 0x7f39b415ddd0 2024-03-30T00:40:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39bbc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b415e1b0 con 0x7f39b40cb6e0 2024-03-30T00:40:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39bbc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b40ce300 con 0x7f39b4069d90 2024-03-30T00:40:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39b9c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f39b40cb6e0 0x7f39b40cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39b9c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f39b40cb6e0 0x7f39b40cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60980/0 (socket says 172.21.15.66:60980) 2024-03-30T00:40:48.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39b9c00640 1 -- 172.21.15.66:0/2745843931 learned_addr learned my addr 172.21.15.66:0/2745843931 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:48.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39abe00640 1 --2- 172.21.15.66:0/2745843931 >> v2:172.21.15.156:3300/0 conn(0x7f39b4069d90 0x7f39b406a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:48.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39b9c00640 1 -- 172.21.15.66:0/2745843931 >> v2:172.21.15.156:3300/0 conn(0x7f39b4069d90 msgr2=0x7f39b406a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39b9c00640 1 --2- 172.21.15.66:0/2745843931 >> v2:172.21.15.156:3300/0 conn(0x7f39b4069d90 0x7f39b406a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39b9c00640 1 -- 172.21.15.66:0/2745843931 >> v1:172.21.15.156:6789/0 conn(0x7f39b415ddd0 legacy=0x7f39b40cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.453+0000 7f39b9c00640 1 -- 172.21.15.66:0/2745843931 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b40cd790 con 0x7f39b40cb6e0 2024-03-30T00:40:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39b9c00640 1 --2- 172.21.15.66:0/2745843931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39b40cb6e0 0x7f39b40cbb70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f39b40ccfd0 tx=0x7f39ac01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc8a3203b7ae647 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:48.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39b3e00640 1 -- 172.21.15.66:0/2745843931 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0a5840 con 0x7f39b40cb6e0 2024-03-30T00:40:48.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39b3e00640 1 -- 172.21.15.66:0/2745843931 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac0a59a0 con 0x7f39b40cb6e0 2024-03-30T00:40:48.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39b3e00640 1 -- 172.21.15.66:0/2745843931 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0a5d30 con 0x7f39b40cb6e0 2024-03-30T00:40:48.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39bbc00640 1 -- 172.21.15.66:0/2745843931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39b40cb6e0 msgr2=0x7f39b40cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39bbc00640 1 --2- 172.21.15.66:0/2745843931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39b40cb6e0 0x7f39b40cbb70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f39b40ccfd0 tx=0x7f39ac01c980 comp rx=0 tx=0).stop 2024-03-30T00:40:48.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39bbc00640 1 -- 172.21.15.66:0/2745843931 shutdown_connections 2024-03-30T00:40:48.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39bbc00640 1 --2- 172.21.15.66:0/2745843931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39b40cb6e0 0x7f39b40cbb70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39bbc00640 1 --2- 172.21.15.66:0/2745843931 >> v2:172.21.15.156:3300/0 conn(0x7f39b4069d90 0x7f39b406a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39bbc00640 1 -- 172.21.15.66:0/2745843931 >> 172.21.15.66:0/2745843931 conn(0x7f39b4098a20 msgr2=0x7f39b40b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:48.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39bbc00640 1 -- 172.21.15.66:0/2745843931 shutdown_connections 2024-03-30T00:40:48.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.457+0000 7f39bbc00640 1 -- 172.21.15.66:0/2745843931 wait complete. 2024-03-30T00:40:48.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39bbc00640 1 Processor -- start 2024-03-30T00:40:48.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39bbc00640 1 -- start start 2024-03-30T00:40:48.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39bbc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39b4069d90 0x7f39b41509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:48.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39bbc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39b40cb6e0 0x7f39b4152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:48.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39bbc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39b415ddd0 0x7f39b4152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39bbc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4159cc0 con 0x7f39b4069d90 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39bbc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b41590b0 con 0x7f39b40cb6e0 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39bbc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4159390 con 0x7f39b415ddd0 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39b4069d90 0x7f39b41509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39b4069d90 0x7f39b41509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60990/0 (socket says 172.21.15.66:60990) 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b9200640 1 -- 172.21.15.66:0/243240575 learned_addr learned my addr 172.21.15.66:0/243240575 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39abe00640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39b40cb6e0 0x7f39b4152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b9c00640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39b415ddd0 0x7f39b4152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b9200640 1 -- 172.21.15.66:0/243240575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39b415ddd0 msgr2=0x7f39b4152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b9200640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39b415ddd0 0x7f39b4152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b9200640 1 -- 172.21.15.66:0/243240575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39b40cb6e0 msgr2=0x7f39b4152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b9200640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39b40cb6e0 0x7f39b4152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b9200640 1 -- 172.21.15.66:0/243240575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac009b30 con 0x7f39b4069d90 2024-03-30T00:40:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b9c00640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39b415ddd0 0x7f39b4152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b9200640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39b4069d90 0x7f39b41509c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f39a400b410 tx=0x7f39a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b1600640 1 -- 172.21.15.66:0/243240575 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4004020 con 0x7f39b4069d90 2024-03-30T00:40:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b1600640 1 -- 172.21.15.66:0/243240575 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a4004180 con 0x7f39b4069d90 2024-03-30T00:40:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39b1600640 1 -- 172.21.15.66:0/243240575 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4011630 con 0x7f39b4069d90 2024-03-30T00:40:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39bbc00640 1 -- 172.21.15.66:0/243240575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b4159670 con 0x7f39b4069d90 2024-03-30T00:40:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39bbc00640 1 -- 172.21.15.66:0/243240575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b4150f00 con 0x7f39b4069d90 2024-03-30T00:40:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.461+0000 7f39bbc00640 1 -- 172.21.15.66:0/243240575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b40ca0a0 con 0x7f39b4069d90 2024-03-30T00:40:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.465+0000 7f39b1600640 1 -- 172.21.15.66:0/243240575 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f39a4011790 con 0x7f39b4069d90 2024-03-30T00:40:48.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.465+0000 7f39b1600640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3990066690 0x7f3990068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:48.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.465+0000 7f39b1600640 1 -- 172.21.15.66:0/243240575 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f39a4084320 con 0x7f39b4069d90 2024-03-30T00:40:48.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.465+0000 7f39abe00640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3990066690 0x7f3990068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.465+0000 7f39abe00640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3990066690 0x7f3990068b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f39b40adcd0 tx=0x7f399c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:48.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.469+0000 7f39b1600640 1 -- 172.21.15.66:0/243240575 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f39a40511f0 con 0x7f39b4069d90 2024-03-30T00:40:48.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.629+0000 7f39bbc00640 1 -- 172.21.15.66:0/243240575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39b4066c30 con 0x7f39b4069d90 2024-03-30T00:40:48.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39b1600640 1 -- 172.21.15.66:0/243240575 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7f39a4054810 con 0x7f39b4069d90 2024-03-30T00:40:48.642 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:48.642 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:48.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 -- 172.21.15.66:0/243240575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3990066690 msgr2=0x7f3990068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3990066690 0x7f3990068b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f39b40adcd0 tx=0x7f399c008040 comp rx=0 tx=0).stop 2024-03-30T00:40:48.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 -- 172.21.15.66:0/243240575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39b4069d90 msgr2=0x7f39b41509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39b4069d90 0x7f39b41509c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f39a400b410 tx=0x7f39a400b8d0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 -- 172.21.15.66:0/243240575 shutdown_connections 2024-03-30T00:40:48.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39b415ddd0 0x7f39b4152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39b40cb6e0 0x7f39b4152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3990066690 0x7f3990068b50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 --2- 172.21.15.66:0/243240575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39b4069d90 0x7f39b41509c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 -- 172.21.15.66:0/243240575 >> 172.21.15.66:0/243240575 conn(0x7f39b4098a20 msgr2=0x7f39b40cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:48.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 -- 172.21.15.66:0/243240575 shutdown_connections 2024-03-30T00:40:48.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.633+0000 7f39bbc00640 1 -- 172.21.15.66:0/243240575 wait complete. 2024-03-30T00:40:48.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:48.664 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:48.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.797+0000 7f54b4a00640 1 Processor -- start 2024-03-30T00:40:48.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.797+0000 7f54b4a00640 1 -- start start 2024-03-30T00:40:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54b4a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f54b0069d90 0x7f54b00b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54b4a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b006a350 con 0x7f54b0070ec0 2024-03-30T00:40:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54b4a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b006a630 con 0x7f54b015fda0 2024-03-30T00:40:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54b4a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b006e5f0 con 0x7f54b0069d90 2024-03-30T00:40:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54afe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f54b0069d90 0x7f54b00b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54afe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f54b0069d90 0x7f54b00b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36262/0 (socket says 172.21.15.66:36262) 2024-03-30T00:40:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54afe00640 1 -- 172.21.15.66:0/1559544861 learned_addr learned my addr 172.21.15.66:0/1559544861 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54a7e00640 1 --1- 172.21.15.66:0/1559544861 >> v1:172.21.15.66:6789/0 conn(0x7f54b0070ec0 0x7f54b006efd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33310/0 (socket says 172.21.15.66:33310) 2024-03-30T00:40:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54afe00640 1 -- 172.21.15.66:0/1559544861 >> v1:172.21.15.156:6789/0 conn(0x7f54b015fda0 legacy=0x7f54b00b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54afe00640 1 -- 172.21.15.66:0/1559544861 >> v1:172.21.15.66:6789/0 conn(0x7f54b0070ec0 legacy=0x7f54b006efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54afe00640 1 -- 172.21.15.66:0/1559544861 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b006e7d0 con 0x7f54b0069d90 2024-03-30T00:40:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.801+0000 7f54aea00640 1 -- 172.21.15.66:0/1559544861 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605747378 0 0) 0x7f54b006a350 con 0x7f54b0070ec0 2024-03-30T00:40:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54afe00640 1 --2- 172.21.15.66:0/1559544861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b0069d90 0x7f54b00b3e20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009e50 tx=0x7f54a801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b784103ba07ded21 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54aea00640 1 -- 172.21.15.66:0/1559544861 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a80a5760 con 0x7f54b0069d90 2024-03-30T00:40:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54aea00640 1 -- 172.21.15.66:0/1559544861 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a80a58c0 con 0x7f54b0069d90 2024-03-30T00:40:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54aea00640 1 -- 172.21.15.66:0/1559544861 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a80a5c50 con 0x7f54b0069d90 2024-03-30T00:40:48.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54b4a00640 1 -- 172.21.15.66:0/1559544861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b0069d90 msgr2=0x7f54b00b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54b4a00640 1 --2- 172.21.15.66:0/1559544861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b0069d90 0x7f54b00b3e20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009e50 tx=0x7f54a801c990 comp rx=0 tx=0).stop 2024-03-30T00:40:48.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54b4a00640 1 -- 172.21.15.66:0/1559544861 shutdown_connections 2024-03-30T00:40:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54b4a00640 1 --2- 172.21.15.66:0/1559544861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b0069d90 0x7f54b00b3e20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54b4a00640 1 -- 172.21.15.66:0/1559544861 >> 172.21.15.66:0/1559544861 conn(0x7f54b006cf90 msgr2=0x7f54b006d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54b4a00640 1 -- 172.21.15.66:0/1559544861 shutdown_connections 2024-03-30T00:40:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54b4a00640 1 -- 172.21.15.66:0/1559544861 wait complete. 2024-03-30T00:40:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54b4a00640 1 Processor -- start 2024-03-30T00:40:48.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54b4a00640 1 -- start start 2024-03-30T00:40:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.805+0000 7f54b4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b0069d90 0x7f54b013c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54b4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b0070ec0 0x7f54b01f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54b4a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b015fda0 0x7f54b015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54b4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b006ebb0 con 0x7f54b0069d90 2024-03-30T00:40:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54b4a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0152160 con 0x7f54b015fda0 2024-03-30T00:40:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54b4a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b01f9f60 con 0x7f54b0070ec0 2024-03-30T00:40:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54af400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b0069d90 0x7f54b013c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54af400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b0069d90 0x7f54b013c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36270/0 (socket says 172.21.15.66:36270) 2024-03-30T00:40:48.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54af400640 1 -- 172.21.15.66:0/1422284157 learned_addr learned my addr 172.21.15.66:0/1422284157 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54a7e00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b0070ec0 0x7f54b01f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54af400640 1 -- 172.21.15.66:0/1422284157 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b0070ec0 msgr2=0x7f54b01f88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54afe00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b015fda0 0x7f54b015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54af400640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b0070ec0 0x7f54b01f88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54af400640 1 -- 172.21.15.66:0/1422284157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b015fda0 msgr2=0x7f54b015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54af400640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b015fda0 0x7f54b015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54af400640 1 -- 172.21.15.66:0/1422284157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8009b30 con 0x7f54b0069d90 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54a7e00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b0070ec0 0x7f54b01f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54afe00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b015fda0 0x7f54b015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54af400640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b0069d90 0x7f54b013c630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f54a000c480 tx=0x7f54a000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54a7400640 1 -- 172.21.15.66:0/1422284157 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0012830 con 0x7f54b0069d90 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54a7400640 1 -- 172.21.15.66:0/1422284157 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a00129b0 con 0x7f54b0069d90 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54b4a00640 1 -- 172.21.15.66:0/1422284157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b013cb70 con 0x7f54b0069d90 2024-03-30T00:40:48.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54a7400640 1 -- 172.21.15.66:0/1422284157 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0012d60 con 0x7f54b0069d90 2024-03-30T00:40:48.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54b4a00640 1 -- 172.21.15.66:0/1422284157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b01f95c0 con 0x7f54b0069d90 2024-03-30T00:40:48.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54b4a00640 1 -- 172.21.15.66:0/1422284157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b0073e00 con 0x7f54b0069d90 2024-03-30T00:40:48.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.809+0000 7f54a7400640 1 -- 172.21.15.66:0/1422284157 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f54a001c460 con 0x7f54b0069d90 2024-03-30T00:40:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.813+0000 7f54a7400640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c0666c0 0x7f548c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.813+0000 7f54a7400640 1 -- 172.21.15.66:0/1422284157 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f54a00843e0 con 0x7f54b0069d90 2024-03-30T00:40:48.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.813+0000 7f54a7e00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c0666c0 0x7f548c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:48.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.813+0000 7f54a7e00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c0666c0 0x7f548c068b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5498007fd0 tx=0x7f5498009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:48.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.817+0000 7f54a7400640 1 -- 172.21.15.66:0/1422284157 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f54a00513e0 con 0x7f54b0069d90 2024-03-30T00:40:48.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 -- 172.21.15.66:0/1422284157 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54b00cc570 con 0x7f54b0069d90 2024-03-30T00:40:48.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54a7400640 1 -- 172.21.15.66:0/1422284157 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54a0054a00 con 0x7f54b0069d90 2024-03-30T00:40:48.986 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:48.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 -- 172.21.15.66:0/1422284157 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c0666c0 msgr2=0x7f548c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c0666c0 0x7f548c068b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5498007fd0 tx=0x7f5498009040 comp rx=0 tx=0).stop 2024-03-30T00:40:48.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 -- 172.21.15.66:0/1422284157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b0069d90 msgr2=0x7f54b013c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:48.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b0069d90 0x7f54b013c630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f54a000c480 tx=0x7f54a000c940 comp rx=0 tx=0).stop 2024-03-30T00:40:48.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 -- 172.21.15.66:0/1422284157 shutdown_connections 2024-03-30T00:40:48.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b015fda0 0x7f54b015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b0070ec0 0x7f54b01f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c0666c0 0x7f548c068b80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 --2- 172.21.15.66:0/1422284157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b0069d90 0x7f54b013c630 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:48.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 -- 172.21.15.66:0/1422284157 >> 172.21.15.66:0/1422284157 conn(0x7f54b006cf90 msgr2=0x7f54b00add30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:48.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 -- 172.21.15.66:0/1422284157 shutdown_connections 2024-03-30T00:40:48.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:48.977+0000 7f54b4a00640 1 -- 172.21.15.66:0/1422284157 wait complete. 2024-03-30T00:40:49.008 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.141+0000 7f61a9000640 1 Processor -- start 2024-03-30T00:40:49.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.141+0000 7f61a9000640 1 -- start start 2024-03-30T00:40:49.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.141+0000 7f61a9000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61a40b0fb0 0x7f61a406b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.141+0000 7f61a9000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f61a40b13f0 0x7f61a406b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.141+0000 7f61a9000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a4098ac0 con 0x7f61a40cca40 2024-03-30T00:40:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.141+0000 7f61a9000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a40cf3f0 con 0x7f61a40b13f0 2024-03-30T00:40:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a9000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40cf670 con 0x7f61a40b0fb0 2024-03-30T00:40:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a2800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f61a40b13f0 0x7f61a406b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a2800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f61a40b13f0 0x7f61a406b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36272/0 (socket says 172.21.15.66:36272) 2024-03-30T00:40:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a2800640 1 -- 172.21.15.66:0/2667072056 learned_addr learned my addr 172.21.15.66:0/2667072056 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f619be00640 1 --2- 172.21.15.66:0/2667072056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61a40b0fb0 0x7f61a406b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a2800640 1 -- 172.21.15.66:0/2667072056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61a40b0fb0 msgr2=0x7f61a406b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a2800640 1 --2- 172.21.15.66:0/2667072056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61a40b0fb0 0x7f61a406b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a2800640 1 -- 172.21.15.66:0/2667072056 >> v1:172.21.15.156:6789/0 conn(0x7f61a40cca40 legacy=0x7f61a40cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:49.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a2800640 1 -- 172.21.15.66:0/2667072056 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a40cf950 con 0x7f61a40b13f0 2024-03-30T00:40:49.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a2800640 1 --2- 172.21.15.66:0/2667072056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a40b13f0 0x7f61a406b680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f619c009e50 tx=0x7f619c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e87497b0db6d7c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:49.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a1400640 1 -- 172.21.15.66:0/2667072056 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c0a57d0 con 0x7f61a40b13f0 2024-03-30T00:40:49.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a1400640 1 -- 172.21.15.66:0/2667072056 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c0a5930 con 0x7f61a40b13f0 2024-03-30T00:40:49.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.145+0000 7f61a1400640 1 -- 172.21.15.66:0/2667072056 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c0a5cc0 con 0x7f61a40b13f0 2024-03-30T00:40:49.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 -- 172.21.15.66:0/2667072056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a40b13f0 msgr2=0x7f61a406b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 --2- 172.21.15.66:0/2667072056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a40b13f0 0x7f61a406b680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f619c009e50 tx=0x7f619c01c9c0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 -- 172.21.15.66:0/2667072056 shutdown_connections 2024-03-30T00:40:49.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 --2- 172.21.15.66:0/2667072056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a40b13f0 0x7f61a406b680 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 --2- 172.21.15.66:0/2667072056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61a40b0fb0 0x7f61a406b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 -- 172.21.15.66:0/2667072056 >> 172.21.15.66:0/2667072056 conn(0x7f61a40b2130 msgr2=0x7f61a40b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:49.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 -- 172.21.15.66:0/2667072056 shutdown_connections 2024-03-30T00:40:49.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 -- 172.21.15.66:0/2667072056 wait complete. 2024-03-30T00:40:49.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 Processor -- start 2024-03-30T00:40:49.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 -- start start 2024-03-30T00:40:49.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a40b0fb0 0x7f61a406f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61a40b13f0 0x7f61a406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61a40cca40 0x7f61a40c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c5e30 con 0x7f61a40b0fb0 2024-03-30T00:40:49.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c6110 con 0x7f61a40b13f0 2024-03-30T00:40:49.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a9000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c63f0 con 0x7f61a40cca40 2024-03-30T00:40:49.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a40b0fb0 0x7f61a406f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a40b0fb0 0x7f61a406f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36282/0 (socket says 172.21.15.66:36282) 2024-03-30T00:40:49.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a1e00640 1 -- 172.21.15.66:0/266257569 learned_addr learned my addr 172.21.15.66:0/266257569 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a2800640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61a40cca40 0x7f61a40c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f619be00640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61a40b13f0 0x7f61a406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a1e00640 1 -- 172.21.15.66:0/266257569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61a40cca40 msgr2=0x7f61a40c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a1e00640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61a40cca40 0x7f61a40c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a1e00640 1 -- 172.21.15.66:0/266257569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61a40b13f0 msgr2=0x7f61a406fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a1e00640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61a40b13f0 0x7f61a406fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.149+0000 7f61a1e00640 1 -- 172.21.15.66:0/266257569 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c009b30 con 0x7f61a40b0fb0 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f61a1e00640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a40b0fb0 0x7f61a406f4f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f619400b410 tx=0x7f619400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f619a000640 1 -- 172.21.15.66:0/266257569 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194004020 con 0x7f61a40b0fb0 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f619a000640 1 -- 172.21.15.66:0/266257569 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6194004180 con 0x7f61a40b0fb0 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f619a000640 1 -- 172.21.15.66:0/266257569 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194011630 con 0x7f61a40b0fb0 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f61a9000640 1 -- 172.21.15.66:0/266257569 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a40c66d0 con 0x7f61a40b0fb0 2024-03-30T00:40:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f61a9000640 1 -- 172.21.15.66:0/266257569 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a40c58c0 con 0x7f61a40b0fb0 2024-03-30T00:40:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f61a9000640 1 -- 172.21.15.66:0/266257569 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a40677d0 con 0x7f61a40b0fb0 2024-03-30T00:40:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f619a000640 1 -- 172.21.15.66:0/266257569 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6194011790 con 0x7f61a40b0fb0 2024-03-30T00:40:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f619a000640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6180066690 0x7f6180068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f619a000640 1 -- 172.21.15.66:0/266257569 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f61940843a0 con 0x7f61a40b0fb0 2024-03-30T00:40:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f619be00640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6180066690 0x7f6180068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.153+0000 7f619be00640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6180066690 0x7f6180068b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f61a40b2730 tx=0x7f618c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.157+0000 7f619a000640 1 -- 172.21.15.66:0/266257569 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6194051270 con 0x7f61a40b0fb0 2024-03-30T00:40:49.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.317+0000 7f61a9000640 1 -- 172.21.15.66:0/266257569 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61a4066c30 con 0x7f61a40b0fb0 2024-03-30T00:40:49.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.317+0000 7f619a000640 1 -- 172.21.15.66:0/266257569 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6194054890 con 0x7f61a40b0fb0 2024-03-30T00:40:49.329 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:49.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 -- 172.21.15.66:0/266257569 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6180066690 msgr2=0x7f6180068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6180066690 0x7f6180068b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f61a40b2730 tx=0x7f618c008040 comp rx=0 tx=0).stop 2024-03-30T00:40:49.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 -- 172.21.15.66:0/266257569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a40b0fb0 msgr2=0x7f61a406f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a40b0fb0 0x7f61a406f4f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f619400b410 tx=0x7f619400b8d0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 -- 172.21.15.66:0/266257569 shutdown_connections 2024-03-30T00:40:49.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61a40cca40 0x7f61a40c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61a40b13f0 0x7f61a406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6180066690 0x7f6180068b50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 --2- 172.21.15.66:0/266257569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a40b0fb0 0x7f61a406f4f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 -- 172.21.15.66:0/266257569 >> 172.21.15.66:0/266257569 conn(0x7f61a40b2130 msgr2=0x7f61a406aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:49.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 -- 172.21.15.66:0/266257569 shutdown_connections 2024-03-30T00:40:49.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.321+0000 7f61a9000640 1 -- 172.21.15.66:0/266257569 wait complete. 2024-03-30T00:40:49.349 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.485+0000 7f9f55a00640 1 Processor -- start 2024-03-30T00:40:49.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.485+0000 7f9f55a00640 1 -- start start 2024-03-30T00:40:49.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.485+0000 7f9f55a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9f500cda90 0x7f9f500ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.485+0000 7f9f55a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f500cde70 0x7f9f500cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.485+0000 7f9f55a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5015a990 con 0x7f9f50159c60 2024-03-30T00:40:49.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.485+0000 7f9f55a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f500b1b80 con 0x7f9f500cda90 2024-03-30T00:40:49.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.485+0000 7f9f55a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500cd3e0 con 0x7f9f500cde70 2024-03-30T00:40:49.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.485+0000 7f9f4ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9f500cda90 0x7f9f500ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.489+0000 7f9f4ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9f500cda90 0x7f9f500ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36286/0 (socket says 172.21.15.66:36286) 2024-03-30T00:40:49.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.489+0000 7f9f4ea00640 1 -- 172.21.15.66:0/3789583930 learned_addr learned my addr 172.21.15.66:0/3789583930 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:49.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.489+0000 7f9f46000640 1 --2- 172.21.15.66:0/3789583930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f500cde70 0x7f9f500cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.489+0000 7f9f4ea00640 1 -- 172.21.15.66:0/3789583930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f500cde70 msgr2=0x7f9f500cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:49.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.489+0000 7f9f4ea00640 1 --2- 172.21.15.66:0/3789583930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f500cde70 0x7f9f500cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.489+0000 7f9f4ea00640 1 -- 172.21.15.66:0/3789583930 >> v1:172.21.15.156:6789/0 conn(0x7f9f50159c60 legacy=0x7f9f5015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:49.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.489+0000 7f9f4ea00640 1 -- 172.21.15.66:0/3789583930 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500b11e0 con 0x7f9f500cda90 2024-03-30T00:40:49.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.489+0000 7f9f4ea00640 1 --2- 172.21.15.66:0/3789583930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500cda90 0x7f9f500ce520 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000bb10 tx=0x7f9f4001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373a94a8f1022fcd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.489+0000 7f9f4e000640 1 -- 172.21.15.66:0/3789583930 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f400a7b00 con 0x7f9f500cda90 2024-03-30T00:40:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.489+0000 7f9f4e000640 1 -- 172.21.15.66:0/3789583930 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f400a7c60 con 0x7f9f500cda90 2024-03-30T00:40:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.489+0000 7f9f4e000640 1 -- 172.21.15.66:0/3789583930 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f400af440 con 0x7f9f500cda90 2024-03-30T00:40:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 -- 172.21.15.66:0/3789583930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500cda90 msgr2=0x7f9f500ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 --2- 172.21.15.66:0/3789583930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500cda90 0x7f9f500ce520 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000bb10 tx=0x7f9f4001eb60 comp rx=0 tx=0).stop 2024-03-30T00:40:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 -- 172.21.15.66:0/3789583930 shutdown_connections 2024-03-30T00:40:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 --2- 172.21.15.66:0/3789583930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f500cde70 0x7f9f500cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 --2- 172.21.15.66:0/3789583930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500cda90 0x7f9f500ce520 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 -- 172.21.15.66:0/3789583930 >> 172.21.15.66:0/3789583930 conn(0x7f9f50069f60 msgr2=0x7f9f5006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 -- 172.21.15.66:0/3789583930 shutdown_connections 2024-03-30T00:40:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 -- 172.21.15.66:0/3789583930 wait complete. 2024-03-30T00:40:49.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 Processor -- start 2024-03-30T00:40:49.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 -- start start 2024-03-30T00:40:49.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500cda90 0x7f9f5015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f500cde70 0x7f9f500c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f50159c60 0x7f9f500c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500c2040 con 0x7f9f500cda90 2024-03-30T00:40:49.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f50159340 con 0x7f9f500cde70 2024-03-30T00:40:49.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f55a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5006d6e0 con 0x7f9f50159c60 2024-03-30T00:40:49.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f4ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500cda90 0x7f9f5015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f4ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500cda90 0x7f9f5015d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36300/0 (socket says 172.21.15.66:36300) 2024-03-30T00:40:49.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f4ea00640 1 -- 172.21.15.66:0/4083568155 learned_addr learned my addr 172.21.15.66:0/4083568155 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:49.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f4f400640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f50159c60 0x7f9f500c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f46000640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f500cde70 0x7f9f500c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f4ea00640 1 -- 172.21.15.66:0/4083568155 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f50159c60 msgr2=0x7f9f500c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f4ea00640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f50159c60 0x7f9f500c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f4ea00640 1 -- 172.21.15.66:0/4083568155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f500cde70 msgr2=0x7f9f500c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f4ea00640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f500cde70 0x7f9f500c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f4ea00640 1 -- 172.21.15.66:0/4083568155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4000b770 con 0x7f9f500cda90 2024-03-30T00:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f46000640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f500cde70 0x7f9f500c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.493+0000 7f9f4ea00640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500cda90 0x7f9f5015d7c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b2c0 tx=0x7f9f4001b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.497+0000 7f9f47400640 1 -- 172.21.15.66:0/4083568155 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40006ee0 con 0x7f9f500cda90 2024-03-30T00:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.497+0000 7f9f47400640 1 -- 172.21.15.66:0/4083568155 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f400ba640 con 0x7f9f500cda90 2024-03-30T00:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.497+0000 7f9f47400640 1 -- 172.21.15.66:0/4083568155 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f400ba9f0 con 0x7f9f500cda90 2024-03-30T00:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.497+0000 7f9f55a00640 1 -- 172.21.15.66:0/4083568155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f50158730 con 0x7f9f500cda90 2024-03-30T00:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.497+0000 7f9f55a00640 1 -- 172.21.15.66:0/4083568155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f50158cf0 con 0x7f9f500cda90 2024-03-30T00:40:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.497+0000 7f9f55a00640 1 -- 172.21.15.66:0/4083568155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f50066c30 con 0x7f9f500cda90 2024-03-30T00:40:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.497+0000 7f9f47400640 1 -- 172.21.15.66:0/4083568155 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9f400b6040 con 0x7f9f500cda90 2024-03-30T00:40:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.497+0000 7f9f47400640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f2c066690 0x7f9f2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.497+0000 7f9f47400640 1 -- 172.21.15.66:0/4083568155 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f9f40121980 con 0x7f9f500cda90 2024-03-30T00:40:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.497+0000 7f9f46000640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f2c066690 0x7f9f2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.497+0000 7f9f46000640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f2c066690 0x7f9f2c068b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9f5015dd00 tx=0x7f9f38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.501+0000 7f9f47400640 1 -- 172.21.15.66:0/4083568155 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9f400ee8e0 con 0x7f9f500cda90 2024-03-30T00:40:49.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 -- 172.21.15.66:0/4083568155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f50165280 con 0x7f9f500cda90 2024-03-30T00:40:49.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f47400640 1 -- 172.21.15.66:0/4083568155 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f400f1f00 con 0x7f9f500cda90 2024-03-30T00:40:49.674 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:49.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 -- 172.21.15.66:0/4083568155 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f2c066690 msgr2=0x7f9f2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f2c066690 0x7f9f2c068b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9f5015dd00 tx=0x7f9f38008040 comp rx=0 tx=0).stop 2024-03-30T00:40:49.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 -- 172.21.15.66:0/4083568155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500cda90 msgr2=0x7f9f5015d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500cda90 0x7f9f5015d7c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b2c0 tx=0x7f9f4001b900 comp rx=0 tx=0).stop 2024-03-30T00:40:49.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 -- 172.21.15.66:0/4083568155 shutdown_connections 2024-03-30T00:40:49.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f50159c60 0x7f9f500c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f500cde70 0x7f9f500c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f2c066690 0x7f9f2c068b50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 --2- 172.21.15.66:0/4083568155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f500cda90 0x7f9f5015d7c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 -- 172.21.15.66:0/4083568155 >> 172.21.15.66:0/4083568155 conn(0x7f9f50069f60 msgr2=0x7f9f5015af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 -- 172.21.15.66:0/4083568155 shutdown_connections 2024-03-30T00:40:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.665+0000 7f9f55a00640 1 -- 172.21.15.66:0/4083568155 wait complete. 2024-03-30T00:40:49.696 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:49.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db3a00640 1 Processor -- start 2024-03-30T00:40:49.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db3a00640 1 -- start start 2024-03-30T00:40:49.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db3a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3dac15a640 0x7f3dac06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db3a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3dac0ce520 0x7f3dac0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db3a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dac06d070 con 0x7f3dac0cabb0 2024-03-30T00:40:49.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db3a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dac0cbba0 con 0x7f3dac15a640 2024-03-30T00:40:49.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db3a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dac0cee90 con 0x7f3dac0ce520 2024-03-30T00:40:49.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db1000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3dac15a640 0x7f3dac06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db1000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3dac15a640 0x7f3dac06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36306/0 (socket says 172.21.15.66:36306) 2024-03-30T00:40:49.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db1000640 1 -- 172.21.15.66:0/1642649848 learned_addr learned my addr 172.21.15.66:0/1642649848 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:49.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3da3e00640 1 --2- 172.21.15.66:0/1642649848 >> v2:172.21.15.156:3300/0 conn(0x7f3dac0ce520 0x7f3dac0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db1000640 1 -- 172.21.15.66:0/1642649848 >> v2:172.21.15.156:3300/0 conn(0x7f3dac0ce520 msgr2=0x7f3dac0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:49.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db1000640 1 --2- 172.21.15.66:0/1642649848 >> v2:172.21.15.156:3300/0 conn(0x7f3dac0ce520 0x7f3dac0ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db1000640 1 -- 172.21.15.66:0/1642649848 >> v1:172.21.15.156:6789/0 conn(0x7f3dac0cabb0 legacy=0x7f3dac0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.833+0000 7f3db1000640 1 -- 172.21.15.66:0/1642649848 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac06e2c0 con 0x7f3dac15a640 2024-03-30T00:40:49.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3db1000640 1 --2- 172.21.15.66:0/1642649848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dac15a640 0x7f3dac06dd80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0c88d0 tx=0x7f3d9c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76436779ad7118d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:49.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3dabe00640 1 -- 172.21.15.66:0/1642649848 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0ab610 con 0x7f3dac15a640 2024-03-30T00:40:49.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3dabe00640 1 -- 172.21.15.66:0/1642649848 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c0ab770 con 0x7f3dac15a640 2024-03-30T00:40:49.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3dabe00640 1 -- 172.21.15.66:0/1642649848 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0abb00 con 0x7f3dac15a640 2024-03-30T00:40:49.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3db3a00640 1 -- 172.21.15.66:0/1642649848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dac15a640 msgr2=0x7f3dac06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3db3a00640 1 --2- 172.21.15.66:0/1642649848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dac15a640 0x7f3dac06dd80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0c88d0 tx=0x7f3d9c01c520 comp rx=0 tx=0).stop 2024-03-30T00:40:49.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3db3a00640 1 -- 172.21.15.66:0/1642649848 shutdown_connections 2024-03-30T00:40:49.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3db3a00640 1 --2- 172.21.15.66:0/1642649848 >> v2:172.21.15.156:3300/0 conn(0x7f3dac0ce520 0x7f3dac0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3db3a00640 1 --2- 172.21.15.66:0/1642649848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dac15a640 0x7f3dac06dd80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3db3a00640 1 -- 172.21.15.66:0/1642649848 >> 172.21.15.66:0/1642649848 conn(0x7f3dac069f60 msgr2=0x7f3dac06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:49.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3db3a00640 1 -- 172.21.15.66:0/1642649848 shutdown_connections 2024-03-30T00:40:49.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3db3a00640 1 -- 172.21.15.66:0/1642649848 wait complete. 2024-03-30T00:40:49.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3db3a00640 1 Processor -- start 2024-03-30T00:40:49.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.837+0000 7f3db3a00640 1 -- start start 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db3a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dac0cabb0 0x7f3dac097740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db3a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3dac0ce520 0x7f3dac156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db3a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dac15a640 0x7f3dac163560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db3a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac156380 con 0x7f3dac15a640 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db3a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac156d50 con 0x7f3dac0cabb0 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db3a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0974f0 con 0x7f3dac0ce520 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dac15a640 0x7f3dac163560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dac15a640 0x7f3dac163560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36308/0 (socket says 172.21.15.66:36308) 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db1a00640 1 -- 172.21.15.66:0/301617766 learned_addr learned my addr 172.21.15.66:0/301617766 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db1000640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dac0cabb0 0x7f3dac097740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3da3e00640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3dac0ce520 0x7f3dac156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db1a00640 1 -- 172.21.15.66:0/301617766 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3dac0ce520 msgr2=0x7f3dac156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db1a00640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3dac0ce520 0x7f3dac156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db1a00640 1 -- 172.21.15.66:0/301617766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dac0cabb0 msgr2=0x7f3dac097740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db1a00640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dac0cabb0 0x7f3dac097740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db1a00640 1 -- 172.21.15.66:0/301617766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c0094d0 con 0x7f3dac15a640 2024-03-30T00:40:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db1a00640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dac15a640 0x7f3dac163560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3da400c770 tx=0x7f3da400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3da9600640 1 -- 172.21.15.66:0/301617766 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4003b80 con 0x7f3dac15a640 2024-03-30T00:40:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3da9600640 1 -- 172.21.15.66:0/301617766 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da4003ce0 con 0x7f3dac15a640 2024-03-30T00:40:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3da9600640 1 -- 172.21.15.66:0/301617766 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da400aa90 con 0x7f3dac15a640 2024-03-30T00:40:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db3a00640 1 -- 172.21.15.66:0/301617766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac1642e0 con 0x7f3dac15a640 2024-03-30T00:40:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db3a00640 1 -- 172.21.15.66:0/301617766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dac1649f0 con 0x7f3dac15a640 2024-03-30T00:40:49.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.841+0000 7f3db3a00640 1 -- 172.21.15.66:0/301617766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dac1564f0 con 0x7f3dac15a640 2024-03-30T00:40:49.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.845+0000 7f3da9600640 1 -- 172.21.15.66:0/301617766 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3da400abf0 con 0x7f3dac15a640 2024-03-30T00:40:49.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.845+0000 7f3da9600640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3d88066690 0x7f3d88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:49.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.845+0000 7f3da9600640 1 -- 172.21.15.66:0/301617766 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f3da40842a0 con 0x7f3dac15a640 2024-03-30T00:40:49.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.845+0000 7f3db1000640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3d88066690 0x7f3d88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:49.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.845+0000 7f3db1000640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3d88066690 0x7f3d88068b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0af000 tx=0x7f3d9c0b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:49.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:49.849+0000 7f3da9600640 1 -- 172.21.15.66:0/301617766 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3da40512b0 con 0x7f3dac15a640 2024-03-30T00:40:50.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 -- 172.21.15.66:0/301617766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3dac066c30 con 0x7f3dac15a640 2024-03-30T00:40:50.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3da9600640 1 -- 172.21.15.66:0/301617766 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3da40548d0 con 0x7f3dac15a640 2024-03-30T00:40:50.018 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:50.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 -- 172.21.15.66:0/301617766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3d88066690 msgr2=0x7f3d88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3d88066690 0x7f3d88068b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0af000 tx=0x7f3d9c0b6040 comp rx=0 tx=0).stop 2024-03-30T00:40:50.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 -- 172.21.15.66:0/301617766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dac15a640 msgr2=0x7f3dac163560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dac15a640 0x7f3dac163560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3da400c770 tx=0x7f3da400cc30 comp rx=0 tx=0).stop 2024-03-30T00:40:50.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 -- 172.21.15.66:0/301617766 shutdown_connections 2024-03-30T00:40:50.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3d88066690 0x7f3d88068b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dac15a640 0x7f3dac163560 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3dac0ce520 0x7f3dac156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 --2- 172.21.15.66:0/301617766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dac0cabb0 0x7f3dac097740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 -- 172.21.15.66:0/301617766 >> 172.21.15.66:0/301617766 conn(0x7f3dac069f60 msgr2=0x7f3dac0adbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:50.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 -- 172.21.15.66:0/301617766 shutdown_connections 2024-03-30T00:40:50.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.009+0000 7f3db3a00640 1 -- 172.21.15.66:0/301617766 wait complete. 2024-03-30T00:40:50.039 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:40:50.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.169+0000 7f50a2000640 1 Processor -- start 2024-03-30T00:40:50.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.169+0000 7f50a2000640 1 -- start start 2024-03-30T00:40:50.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f50a2000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f509c15de30 0x7f509c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f50a2000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f509c0af420 0x7f509c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f50a2000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509c0cbb90 con 0x7f509c0ce740 2024-03-30T00:40:50.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f50a2000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c0ce1b0 con 0x7f509c15de30 2024-03-30T00:40:50.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f50a2000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c0afa10 con 0x7f509c0af420 2024-03-30T00:40:50.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f509b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f509c15de30 0x7f509c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f509be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f509c0ce740 0x7f509c06e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33322/0 (socket says 172.21.15.66:33322) 2024-03-30T00:40:50.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f509be00640 1 -- 172.21.15.66:0/2610477095 learned_addr learned my addr 172.21.15.66:0/2610477095 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:50.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f5092a00640 1 --2- 172.21.15.66:0/2610477095 >> v2:172.21.15.156:3300/0 conn(0x7f509c0af420 0x7f509c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f509b400640 1 -- 172.21.15.66:0/2610477095 >> v2:172.21.15.156:3300/0 conn(0x7f509c0af420 msgr2=0x7f509c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:50.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f509b400640 1 --2- 172.21.15.66:0/2610477095 >> v2:172.21.15.156:3300/0 conn(0x7f509c0af420 0x7f509c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f509b400640 1 -- 172.21.15.66:0/2610477095 >> v1:172.21.15.66:6789/0 conn(0x7f509c0ce740 legacy=0x7f509c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.173+0000 7f509b400640 1 -- 172.21.15.66:0/2610477095 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0cf430 con 0x7f509c15de30 2024-03-30T00:40:50.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f509b400640 1 --2- 172.21.15.66:0/2610477095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f509c15de30 0x7f509c0cc040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f509c070080 tx=0x7f508c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3763444bcea55e15 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:50.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f509aa00640 1 -- 172.21.15.66:0/2610477095 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c004230 con 0x7f509c15de30 2024-03-30T00:40:50.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f509aa00640 1 -- 172.21.15.66:0/2610477095 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c0a7050 con 0x7f509c15de30 2024-03-30T00:40:50.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f509aa00640 1 -- 172.21.15.66:0/2610477095 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c003620 con 0x7f509c15de30 2024-03-30T00:40:50.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f50a2000640 1 -- 172.21.15.66:0/2610477095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f509c15de30 msgr2=0x7f509c0cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f50a2000640 1 --2- 172.21.15.66:0/2610477095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f509c15de30 0x7f509c0cc040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f509c070080 tx=0x7f508c01c3f0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f50a2000640 1 -- 172.21.15.66:0/2610477095 shutdown_connections 2024-03-30T00:40:50.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f50a2000640 1 --2- 172.21.15.66:0/2610477095 >> v2:172.21.15.156:3300/0 conn(0x7f509c0af420 0x7f509c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f50a2000640 1 --2- 172.21.15.66:0/2610477095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f509c15de30 0x7f509c0cc040 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f50a2000640 1 -- 172.21.15.66:0/2610477095 >> 172.21.15.66:0/2610477095 conn(0x7f509c0d0540 msgr2=0x7f509c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:50.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f50a2000640 1 -- 172.21.15.66:0/2610477095 shutdown_connections 2024-03-30T00:40:50.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f50a2000640 1 -- 172.21.15.66:0/2610477095 wait complete. 2024-03-30T00:40:50.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f50a2000640 1 Processor -- start 2024-03-30T00:40:50.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.177+0000 7f50a2000640 1 -- start start 2024-03-30T00:40:50.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f50a2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f509c0af420 0x7f509c157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f50a2000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f509c0ce740 0x7f509c0b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f50a2000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f509c15de30 0x7f509c06aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f50a2000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c163ad0 con 0x7f509c0af420 2024-03-30T00:40:50.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f50a2000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c163db0 con 0x7f509c0ce740 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f50a2000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c164090 con 0x7f509c15de30 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f509b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f509c0af420 0x7f509c157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f509b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f509c0af420 0x7f509c157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36338/0 (socket says 172.21.15.66:36338) 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f509b400640 1 -- 172.21.15.66:0/4019562950 learned_addr learned my addr 172.21.15.66:0/4019562950 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f5092a00640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f509c0ce740 0x7f509c0b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f509be00640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f509c15de30 0x7f509c06aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f509b400640 1 -- 172.21.15.66:0/4019562950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f509c15de30 msgr2=0x7f509c06aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f509b400640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f509c15de30 0x7f509c06aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f509b400640 1 -- 172.21.15.66:0/4019562950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f509c0ce740 msgr2=0x7f509c0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f509b400640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f509c0ce740 0x7f509c0b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f509b400640 1 -- 172.21.15.66:0/4019562950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c009420 con 0x7f509c0af420 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f509b400640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f509c0af420 0x7f509c157f00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f508c0ae040 tx=0x7f508c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f5093e00640 1 -- 172.21.15.66:0/4019562950 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c003b30 con 0x7f509c0af420 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f5093e00640 1 -- 172.21.15.66:0/4019562950 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c003c90 con 0x7f509c0af420 2024-03-30T00:40:50.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f50a2000640 1 -- 172.21.15.66:0/4019562950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509c164370 con 0x7f509c0af420 2024-03-30T00:40:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f5093e00640 1 -- 172.21.15.66:0/4019562950 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c0ab440 con 0x7f509c0af420 2024-03-30T00:40:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f50a2000640 1 -- 172.21.15.66:0/4019562950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c0cb6c0 con 0x7f509c0af420 2024-03-30T00:40:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f50a2000640 1 -- 172.21.15.66:0/4019562950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c0cdf60 con 0x7f509c0af420 2024-03-30T00:40:50.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.181+0000 7f5093e00640 1 -- 172.21.15.66:0/4019562950 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f508c0b2040 con 0x7f509c0af420 2024-03-30T00:40:50.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.185+0000 7f5093e00640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5078066690 0x7f5078068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.185+0000 7f5093e00640 1 -- 172.21.15.66:0/4019562950 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f508c11e660 con 0x7f509c0af420 2024-03-30T00:40:50.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.185+0000 7f5092a00640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5078066690 0x7f5078068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.185+0000 7f5092a00640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5078066690 0x7f5078068b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f509c0b3010 tx=0x7f5084008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:50.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.189+0000 7f5093e00640 1 -- 172.21.15.66:0/4019562950 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f508c0eb5c0 con 0x7f509c0af420 2024-03-30T00:40:50.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 -- 172.21.15.66:0/4019562950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f509c163c10 con 0x7f509c0af420 2024-03-30T00:40:50.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f5093e00640 1 -- 172.21.15.66:0/4019562950 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f509c163c10 con 0x7f509c0af420 2024-03-30T00:40:50.373 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:40:50.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 -- 172.21.15.66:0/4019562950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5078066690 msgr2=0x7f5078068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5078066690 0x7f5078068b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f509c0b3010 tx=0x7f5084008040 comp rx=0 tx=0).stop 2024-03-30T00:40:50.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 -- 172.21.15.66:0/4019562950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f509c0af420 msgr2=0x7f509c157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f509c0af420 0x7f509c157f00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f508c0ae040 tx=0x7f508c019190 comp rx=0 tx=0).stop 2024-03-30T00:40:50.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 -- 172.21.15.66:0/4019562950 shutdown_connections 2024-03-30T00:40:50.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f509c15de30 0x7f509c06aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f509c0ce740 0x7f509c0b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5078066690 0x7f5078068b50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 --2- 172.21.15.66:0/4019562950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f509c0af420 0x7f509c157f00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 -- 172.21.15.66:0/4019562950 >> 172.21.15.66:0/4019562950 conn(0x7f509c0d0540 msgr2=0x7f509c15b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:50.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 -- 172.21.15.66:0/4019562950 shutdown_connections 2024-03-30T00:40:50.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.365+0000 7f50a2000640 1 -- 172.21.15.66:0/4019562950 wait complete. 2024-03-30T00:40:50.394 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:40:50.394 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:40:50.403 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:50.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.573+0000 7f3650800640 1 Processor -- start 2024-03-30T00:40:50.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.573+0000 7f3650800640 1 -- start start 2024-03-30T00:40:50.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f3650800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f36480cee50 0x7f36480cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f3650800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f36480af420 0x7f36480d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f3650800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36480cbb90 con 0x7f364815de30 2024-03-30T00:40:50.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f3650800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36480ce1b0 con 0x7f36480af420 2024-03-30T00:40:50.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f3650800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36480afa10 con 0x7f36480cee50 2024-03-30T00:40:50.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f364e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f36480af420 0x7f36480d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f364e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f36480af420 0x7f36480d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36354/0 (socket says 172.21.15.66:36354) 2024-03-30T00:40:50.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f364e800640 1 -- 172.21.15.66:0/3833539839 learned_addr learned my addr 172.21.15.66:0/3833539839 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:50.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f364e800640 1 -- 172.21.15.66:0/3833539839 >> v2:172.21.15.156:3300/0 conn(0x7f36480cee50 msgr2=0x7f36480cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:50.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f364d400640 1 --2- 172.21.15.66:0/3833539839 >> v2:172.21.15.156:3300/0 conn(0x7f36480cee50 0x7f36480cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f364e800640 1 --2- 172.21.15.66:0/3833539839 >> v2:172.21.15.156:3300/0 conn(0x7f36480cee50 0x7f36480cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f364e800640 1 -- 172.21.15.66:0/3833539839 >> v1:172.21.15.156:6789/0 conn(0x7f364815de30 legacy=0x7f36480ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f364e800640 1 -- 172.21.15.66:0/3833539839 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480cf430 con 0x7f36480af420 2024-03-30T00:40:50.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.577+0000 7f364d400640 1 --2- 172.21.15.66:0/3833539839 >> v2:172.21.15.156:3300/0 conn(0x7f36480cee50 0x7f36480cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:50.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f364e800640 1 --2- 172.21.15.66:0/3833539839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36480af420 0x7f36480d1060 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3648070ad0 tx=0x7f364401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287e7938ec97e250 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:50.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f364ca00640 1 -- 172.21.15.66:0/3833539839 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36440a57d0 con 0x7f36480af420 2024-03-30T00:40:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f364ca00640 1 -- 172.21.15.66:0/3833539839 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36440a5930 con 0x7f36480af420 2024-03-30T00:40:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f364ca00640 1 -- 172.21.15.66:0/3833539839 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36440a5cc0 con 0x7f36480af420 2024-03-30T00:40:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 -- 172.21.15.66:0/3833539839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36480af420 msgr2=0x7f36480d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 --2- 172.21.15.66:0/3833539839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36480af420 0x7f36480d1060 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3648070ad0 tx=0x7f364401c9b0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 -- 172.21.15.66:0/3833539839 shutdown_connections 2024-03-30T00:40:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 --2- 172.21.15.66:0/3833539839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36480af420 0x7f36480d1060 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 --2- 172.21.15.66:0/3833539839 >> v2:172.21.15.156:3300/0 conn(0x7f36480cee50 0x7f36480cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 -- 172.21.15.66:0/3833539839 >> 172.21.15.66:0/3833539839 conn(0x7f36480d0540 msgr2=0x7f36480d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 -- 172.21.15.66:0/3833539839 shutdown_connections 2024-03-30T00:40:50.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 -- 172.21.15.66:0/3833539839 wait complete. 2024-03-30T00:40:50.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 Processor -- start 2024-03-30T00:40:50.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 -- start start 2024-03-30T00:40:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36480af420 0x7f3648157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36480cee50 0x7f3648164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f364815de30 0x7f3648164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36480ca460 con 0x7f36480cee50 2024-03-30T00:40:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364806b060 con 0x7f36480af420 2024-03-30T00:40:50.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f3650800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c9bc0 con 0x7f364815de30 2024-03-30T00:40:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.581+0000 7f364d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36480cee50 0x7f3648164990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f364d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36480cee50 0x7f3648164990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36356/0 (socket says 172.21.15.66:36356) 2024-03-30T00:40:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f364d400640 1 -- 172.21.15.66:0/1372322769 learned_addr learned my addr 172.21.15.66:0/1372322769 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f364de00640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36480af420 0x7f3648157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f364e800640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f364815de30 0x7f3648164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f364d400640 1 -- 172.21.15.66:0/1372322769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f364815de30 msgr2=0x7f3648164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f364d400640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f364815de30 0x7f3648164ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f364d400640 1 -- 172.21.15.66:0/1372322769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36480af420 msgr2=0x7f3648157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f364d400640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36480af420 0x7f3648157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f364d400640 1 -- 172.21.15.66:0/1372322769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3644009b30 con 0x7f36480cee50 2024-03-30T00:40:50.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f364d400640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36480cee50 0x7f3648164990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f363800c820 tx=0x7f363800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f3642000640 1 -- 172.21.15.66:0/1372322769 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638007a30 con 0x7f36480cee50 2024-03-30T00:40:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f3642000640 1 -- 172.21.15.66:0/1372322769 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3638007b90 con 0x7f36480cee50 2024-03-30T00:40:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f3650800640 1 -- 172.21.15.66:0/1372322769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36480c9ea0 con 0x7f36480cee50 2024-03-30T00:40:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f3642000640 1 -- 172.21.15.66:0/1372322769 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363800ce10 con 0x7f36480cee50 2024-03-30T00:40:50.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f3650800640 1 -- 172.21.15.66:0/1372322769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36481582a0 con 0x7f36480cee50 2024-03-30T00:40:50.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f3650800640 1 -- 172.21.15.66:0/1372322769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36480cdf60 con 0x7f36480cee50 2024-03-30T00:40:50.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.585+0000 7f3642000640 1 -- 172.21.15.66:0/1372322769 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f36380113f0 con 0x7f36480cee50 2024-03-30T00:40:50.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.589+0000 7f3642000640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f362c066690 0x7f362c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.589+0000 7f3642000640 1 -- 172.21.15.66:0/1372322769 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f36380845e0 con 0x7f36480cee50 2024-03-30T00:40:50.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.589+0000 7f364de00640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f362c066690 0x7f362c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.589+0000 7f364de00640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f362c066690 0x7f362c068b50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f363c007fd0 tx=0x7f363c007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:50.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.593+0000 7f3642000640 1 -- 172.21.15.66:0/1372322769 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f36380515f0 con 0x7f36480cee50 2024-03-30T00:40:50.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.757+0000 7f3650800640 1 -- 172.21.15.66:0/1372322769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3648066c30 con 0x7f36480cee50 2024-03-30T00:40:50.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.757+0000 7f3642000640 1 -- 172.21.15.66:0/1372322769 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7f3638054c10 con 0x7f36480cee50 2024-03-30T00:40:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:50.768 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 -- 172.21.15.66:0/1372322769 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f362c066690 msgr2=0x7f362c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f362c066690 0x7f362c068b50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f363c007fd0 tx=0x7f363c007e40 comp rx=0 tx=0).stop 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 -- 172.21.15.66:0/1372322769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36480cee50 msgr2=0x7f3648164990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36480cee50 0x7f3648164990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f363800c820 tx=0x7f363800cce0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 -- 172.21.15.66:0/1372322769 shutdown_connections 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f364815de30 0x7f3648164ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f362c066690 0x7f362c068b50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36480cee50 0x7f3648164990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 --2- 172.21.15.66:0/1372322769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36480af420 0x7f3648157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 -- 172.21.15.66:0/1372322769 >> 172.21.15.66:0/1372322769 conn(0x7f36480d0540 msgr2=0x7f36480c6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 -- 172.21.15.66:0/1372322769 shutdown_connections 2024-03-30T00:40:50.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.761+0000 7f3650800640 1 -- 172.21.15.66:0/1372322769 wait complete. 2024-03-30T00:40:50.789 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:50.790 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:50.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb13ca00640 1 Processor -- start 2024-03-30T00:40:50.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb13ca00640 1 -- start start 2024-03-30T00:40:50.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb13ca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb13815de30 0x7fb1380cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb13ca00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb1380cee50 0x7fb1380d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb13ca00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1380cbb90 con 0x7fb1380af420 2024-03-30T00:40:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb13ca00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1380ce1b0 con 0x7fb13815de30 2024-03-30T00:40:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb13ca00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1380afa10 con 0x7fb1380cee50 2024-03-30T00:40:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb135e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb13815de30 0x7fb1380cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb12fe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb1380af420 0x7fb1380ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33332/0 (socket says 172.21.15.66:33332) 2024-03-30T00:40:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb12fe00640 1 -- 172.21.15.66:0/1482243569 learned_addr learned my addr 172.21.15.66:0/1482243569 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb136800640 1 --2- 172.21.15.66:0/1482243569 >> v2:172.21.15.156:3300/0 conn(0x7fb1380cee50 0x7fb1380d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb135e00640 1 -- 172.21.15.66:0/1482243569 >> v2:172.21.15.156:3300/0 conn(0x7fb1380cee50 msgr2=0x7fb1380d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb135e00640 1 --2- 172.21.15.66:0/1482243569 >> v2:172.21.15.156:3300/0 conn(0x7fb1380cee50 0x7fb1380d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb135e00640 1 -- 172.21.15.66:0/1482243569 >> v1:172.21.15.66:6789/0 conn(0x7fb1380af420 legacy=0x7fb1380ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb135e00640 1 -- 172.21.15.66:0/1482243569 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1380cf430 con 0x7fb13815de30 2024-03-30T00:40:50.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.925+0000 7fb135400640 1 -- 172.21.15.66:0/1482243569 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167191156 0 0) 0x7fb1380cbb90 con 0x7fb1380af420 2024-03-30T00:40:50.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb135e00640 1 --2- 172.21.15.66:0/1482243569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb13815de30 0x7fb1380cc040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb13808ac40 tx=0x7fb12801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c80ee78b74f460e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:50.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb135400640 1 -- 172.21.15.66:0/1482243569 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128004230 con 0x7fb13815de30 2024-03-30T00:40:50.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb135400640 1 -- 172.21.15.66:0/1482243569 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1280a7050 con 0x7fb13815de30 2024-03-30T00:40:50.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb135400640 1 -- 172.21.15.66:0/1482243569 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128003620 con 0x7fb13815de30 2024-03-30T00:40:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb13ca00640 1 -- 172.21.15.66:0/1482243569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb13815de30 msgr2=0x7fb1380cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb13ca00640 1 --2- 172.21.15.66:0/1482243569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb13815de30 0x7fb1380cc040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb13808ac40 tx=0x7fb12801c3f0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb13ca00640 1 -- 172.21.15.66:0/1482243569 shutdown_connections 2024-03-30T00:40:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb13ca00640 1 --2- 172.21.15.66:0/1482243569 >> v2:172.21.15.156:3300/0 conn(0x7fb1380cee50 0x7fb1380d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb13ca00640 1 --2- 172.21.15.66:0/1482243569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb13815de30 0x7fb1380cc040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb13ca00640 1 -- 172.21.15.66:0/1482243569 >> 172.21.15.66:0/1482243569 conn(0x7fb1380d0540 msgr2=0x7fb1380d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb13ca00640 1 -- 172.21.15.66:0/1482243569 shutdown_connections 2024-03-30T00:40:50.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.929+0000 7fb13ca00640 1 -- 172.21.15.66:0/1482243569 wait complete. 2024-03-30T00:40:50.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb13ca00640 1 Processor -- start 2024-03-30T00:40:50.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb13ca00640 1 -- start start 2024-03-30T00:40:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb13ca00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1380af420 0x7fb138159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb13ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1380cee50 0x7fb1380b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb13ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb13815de30 0x7fb1380b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb13ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380b5510 con 0x7fb13815de30 2024-03-30T00:40:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb13ca00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380b4c70 con 0x7fb1380af420 2024-03-30T00:40:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb13ca00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380b4f50 con 0x7fb1380cee50 2024-03-30T00:40:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb136800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb13815de30 0x7fb1380b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb136800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb13815de30 0x7fb1380b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36384/0 (socket says 172.21.15.66:36384) 2024-03-30T00:40:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb136800640 1 -- 172.21.15.66:0/1781943964 learned_addr learned my addr 172.21.15.66:0/1781943964 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb135e00640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1380af420 0x7fb138159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb12fe00640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1380cee50 0x7fb1380b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb136800640 1 -- 172.21.15.66:0/1781943964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1380cee50 msgr2=0x7fb1380b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb136800640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1380cee50 0x7fb1380b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb136800640 1 -- 172.21.15.66:0/1781943964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1380af420 msgr2=0x7fb138159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb136800640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1380af420 0x7fb138159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb136800640 1 -- 172.21.15.66:0/1781943964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb128009420 con 0x7fb13815de30 2024-03-30T00:40:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb136800640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb13815de30 0x7fb1380b45b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb130007e40 tx=0x7fb13000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb12e000640 1 -- 172.21.15.66:0/1781943964 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130014620 con 0x7fb13815de30 2024-03-30T00:40:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb12e000640 1 -- 172.21.15.66:0/1781943964 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13000ee90 con 0x7fb13815de30 2024-03-30T00:40:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb13ca00640 1 -- 172.21.15.66:0/1781943964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1380b5230 con 0x7fb13815de30 2024-03-30T00:40:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb12e000640 1 -- 172.21.15.66:0/1781943964 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1300149e0 con 0x7fb13815de30 2024-03-30T00:40:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb13ca00640 1 -- 172.21.15.66:0/1781943964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1380b5b10 con 0x7fb13815de30 2024-03-30T00:40:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.933+0000 7fb13ca00640 1 -- 172.21.15.66:0/1781943964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1380cdf60 con 0x7fb13815de30 2024-03-30T00:40:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.937+0000 7fb12e000640 1 -- 172.21.15.66:0/1781943964 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb130014b40 con 0x7fb13815de30 2024-03-30T00:40:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.937+0000 7fb12e000640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb114066690 0x7fb114068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.937+0000 7fb12e000640 1 -- 172.21.15.66:0/1781943964 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fb130084ef0 con 0x7fb13815de30 2024-03-30T00:40:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.937+0000 7fb135e00640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb114066690 0x7fb114068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.937+0000 7fb135e00640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb114066690 0x7fb114068b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1280ad000 tx=0x7fb128008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:50.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:50.941+0000 7fb12e000640 1 -- 172.21.15.66:0/1781943964 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb130051ef0 con 0x7fb13815de30 2024-03-30T00:40:51.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.101+0000 7fb13ca00640 1 -- 172.21.15.66:0/1781943964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb138066c30 con 0x7fb13815de30 2024-03-30T00:40:51.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.101+0000 7fb12e000640 1 -- 172.21.15.66:0/1781943964 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb130055510 con 0x7fb13815de30 2024-03-30T00:40:51.112 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 -- 172.21.15.66:0/1781943964 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb114066690 msgr2=0x7fb114068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb114066690 0x7fb114068b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1280ad000 tx=0x7fb128008040 comp rx=0 tx=0).stop 2024-03-30T00:40:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 -- 172.21.15.66:0/1781943964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb13815de30 msgr2=0x7fb1380b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb13815de30 0x7fb1380b45b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb130007e40 tx=0x7fb13000e4b0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 -- 172.21.15.66:0/1781943964 shutdown_connections 2024-03-30T00:40:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb114066690 0x7fb114068b50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb13815de30 0x7fb1380b45b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1380cee50 0x7fb1380b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 --2- 172.21.15.66:0/1781943964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1380af420 0x7fb138159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 -- 172.21.15.66:0/1781943964 >> 172.21.15.66:0/1781943964 conn(0x7fb1380d0540 msgr2=0x7fb13815a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:51.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 -- 172.21.15.66:0/1781943964 shutdown_connections 2024-03-30T00:40:51.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.105+0000 7fb13ca00640 1 -- 172.21.15.66:0/1781943964 wait complete. 2024-03-30T00:40:51.133 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:51.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.269+0000 7f79bc200640 1 Processor -- start 2024-03-30T00:40:51.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79bc200640 1 -- start start 2024-03-30T00:40:51.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79bc200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f79b40cabb0 0x7f79b40cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79bc200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b406d070 con 0x7f79b40ce520 2024-03-30T00:40:51.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79bc200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b40cbba0 con 0x7f79b415a640 2024-03-30T00:40:51.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79bc200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b40cb8c0 con 0x7f79b40cabb0 2024-03-30T00:40:51.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b8e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f79b40ce520 0x7f79b40ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33340/0 (socket says 172.21.15.66:33340) 2024-03-30T00:40:51.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b8e00640 1 -- 172.21.15.66:0/3566925902 learned_addr learned my addr 172.21.15.66:0/3566925902 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:51.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79ba200640 1 --2- 172.21.15.66:0/3566925902 >> v2:172.21.15.156:3300/0 conn(0x7f79b40cabb0 0x7f79b40cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675193537 0 0) 0x7f79b406d070 con 0x7f79b40ce520 2024-03-30T00:40:51.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799c003530 con 0x7f79b40ce520 2024-03-30T00:40:51.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3097479102 0 0) 0x7f799c003530 con 0x7f79b40ce520 2024-03-30T00:40:51.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 >> v2:172.21.15.156:3300/0 conn(0x7f79b40cabb0 msgr2=0x7f79b40cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:51.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 --2- 172.21.15.66:0/3566925902 >> v2:172.21.15.156:3300/0 conn(0x7f79b40cabb0 0x7f79b40cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 >> v1:172.21.15.156:6789/0 conn(0x7f79b415a640 legacy=0x7f79b406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b406d2a0 con 0x7f79b40ce520 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f79a0001680 con 0x7f79b40ce520 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79ba200640 1 --2- 172.21.15.66:0/3566925902 >> v2:172.21.15.156:3300/0 conn(0x7f79b40cabb0 0x7f79b40cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 >> v1:172.21.15.66:6789/0 conn(0x7f79b40ce520 legacy=0x7f79b40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c003980 0x7f799c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 0x7f799c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f799c005190 0x7f799c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b40cb8c0 con 0x7f799c0047d0 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c00c030 con 0x7f799c003980 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c00c1f0 con 0x7f799c005190 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b8e00640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 0x7f799c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b9800640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c003980 0x7f799c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79ba200640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f799c005190 0x7f799c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b8e00640 1 -- 172.21.15.66:0/3566925902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f799c005190 msgr2=0x7f799c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:51.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b8e00640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f799c005190 0x7f799c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b8e00640 1 -- 172.21.15.66:0/3566925902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c003980 msgr2=0x7f799c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:51.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b8e00640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c003980 0x7f799c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79b8e00640 1 -- 172.21.15.66:0/3566925902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c003530 con 0x7f799c0047d0 2024-03-30T00:40:51.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79ba200640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f799c005190 0x7f799c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:51.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.273+0000 7f79ba200640 1 -- 172.21.15.66:0/3566925902 reap_dead start 2024-03-30T00:40:51.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.277+0000 7f79b8e00640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 0x7f799c004c50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c240 tx=0x7f79a001f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46939238ea7cbe78 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:51.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.277+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0005ae0 con 0x7f799c0047d0 2024-03-30T00:40:51.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.277+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a00ab070 con 0x7f799c0047d0 2024-03-30T00:40:51.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.277+0000 7f79b3e00640 1 -- 172.21.15.66:0/3566925902 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0007070 con 0x7f799c0047d0 2024-03-30T00:40:51.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.277+0000 7f79bc200640 1 -- 172.21.15.66:0/3566925902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 msgr2=0x7f799c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.277+0000 7f79bc200640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 0x7f799c004c50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c240 tx=0x7f79a001f350 comp rx=0 tx=0).stop 2024-03-30T00:40:51.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.277+0000 7f79bc200640 1 -- 172.21.15.66:0/3566925902 shutdown_connections 2024-03-30T00:40:51.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.277+0000 7f79bc200640 1 --2- 172.21.15.66:0/3566925902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 0x7f799c004c50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.277+0000 7f79bc200640 1 -- 172.21.15.66:0/3566925902 >> 172.21.15.66:0/3566925902 conn(0x7f79b4069f60 msgr2=0x7f79b406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:51.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.277+0000 7f79bc200640 1 -- 172.21.15.66:0/3566925902 shutdown_connections 2024-03-30T00:40:51.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.277+0000 7f79bc200640 1 -- 172.21.15.66:0/3566925902 wait complete. 2024-03-30T00:40:51.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79bc200640 1 Processor -- start 2024-03-30T00:40:51.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79bc200640 1 -- start start 2024-03-30T00:40:51.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79bc200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 0x7f79b415aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79bc200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79b415d410 0x7f79b41567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79bc200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b415c150 0x7f79b40c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79bc200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b4156ce0 con 0x7f799c0047d0 2024-03-30T00:40:51.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79bc200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b415b0b0 con 0x7f79b415c150 2024-03-30T00:40:51.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79bc200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b415b390 con 0x7f79b415d410 2024-03-30T00:40:51.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 0x7f79b415aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 0x7f79b415aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36404/0 (socket says 172.21.15.66:36404) 2024-03-30T00:40:51.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b9800640 1 -- 172.21.15.66:0/1917461492 learned_addr learned my addr 172.21.15.66:0/1917461492 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:51.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b8e00640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79b415d410 0x7f79b41567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79ba200640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b415c150 0x7f79b40c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b9800640 1 -- 172.21.15.66:0/1917461492 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79b415d410 msgr2=0x7f79b41567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b9800640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79b415d410 0x7f79b41567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b9800640 1 -- 172.21.15.66:0/1917461492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b415c150 msgr2=0x7f79b40c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b9800640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b415c150 0x7f79b40c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b9800640 1 -- 172.21.15.66:0/1917461492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b415b670 con 0x7f799c0047d0 2024-03-30T00:40:51.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b9800640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 0x7f79b415aa20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f79a800f7f0 tx=0x7f79a800fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:51.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b1600640 1 -- 172.21.15.66:0/1917461492 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a80050d0 con 0x7f799c0047d0 2024-03-30T00:40:51.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b1600640 1 -- 172.21.15.66:0/1917461492 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a8005230 con 0x7f799c0047d0 2024-03-30T00:40:51.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79bc200640 1 -- 172.21.15.66:0/1917461492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b415b950 con 0x7f799c0047d0 2024-03-30T00:40:51.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79b1600640 1 -- 172.21.15.66:0/1917461492 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800d730 con 0x7f799c0047d0 2024-03-30T00:40:51.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79bc200640 1 -- 172.21.15.66:0/1917461492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b415d890 con 0x7f799c0047d0 2024-03-30T00:40:51.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.281+0000 7f79bc200640 1 -- 172.21.15.66:0/1917461492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b4066c30 con 0x7f799c0047d0 2024-03-30T00:40:51.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.285+0000 7f79b1600640 1 -- 172.21.15.66:0/1917461492 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f79a800d890 con 0x7f799c0047d0 2024-03-30T00:40:51.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.285+0000 7f79b1600640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7994066690 0x7f7994068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.285+0000 7f79b1600640 1 -- 172.21.15.66:0/1917461492 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f79a8084500 con 0x7f799c0047d0 2024-03-30T00:40:51.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.285+0000 7f79b8e00640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7994066690 0x7f7994068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.285+0000 7f79b8e00640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7994066690 0x7f7994068b50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f79a00af000 tx=0x7f79a00aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:51.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.289+0000 7f79b1600640 1 -- 172.21.15.66:0/1917461492 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f79a8051470 con 0x7f799c0047d0 2024-03-30T00:40:51.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.449+0000 7f79bc200640 1 -- 172.21.15.66:0/1917461492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79b40adfd0 con 0x7f799c0047d0 2024-03-30T00:40:51.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.449+0000 7f79b1600640 1 -- 172.21.15.66:0/1917461492 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79a8054a90 con 0x7f799c0047d0 2024-03-30T00:40:51.460 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:51.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 -- 172.21.15.66:0/1917461492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7994066690 msgr2=0x7f7994068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7994066690 0x7f7994068b50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f79a00af000 tx=0x7f79a00aa000 comp rx=0 tx=0).stop 2024-03-30T00:40:51.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 -- 172.21.15.66:0/1917461492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 msgr2=0x7f79b415aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 0x7f79b415aa20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f79a800f7f0 tx=0x7f79a800fcb0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 -- 172.21.15.66:0/1917461492 shutdown_connections 2024-03-30T00:40:51.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b415c150 0x7f79b40c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79b415d410 0x7f79b41567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7994066690 0x7f7994068b50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 --2- 172.21.15.66:0/1917461492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f799c0047d0 0x7f79b415aa20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 -- 172.21.15.66:0/1917461492 >> 172.21.15.66:0/1917461492 conn(0x7f79b4069f60 msgr2=0x7f79b40c4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 -- 172.21.15.66:0/1917461492 shutdown_connections 2024-03-30T00:40:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.453+0000 7f79bc200640 1 -- 172.21.15.66:0/1917461492 wait complete. 2024-03-30T00:40:51.482 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:51.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.613+0000 7ff998c00640 1 Processor -- start 2024-03-30T00:40:51.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff998c00640 1 -- start start 2024-03-30T00:40:51.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff998c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9940cda90 0x7ff9940ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff998c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff9940cde70 0x7ff9940cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff998c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff994159c60 0x7ff99415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff998c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9940b1b80 con 0x7ff994159c60 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff998c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99415a990 con 0x7ff9940cde70 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff998c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940b11e0 con 0x7ff9940cda90 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff992800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff994159c60 0x7ff99415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff992800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff994159c60 0x7ff99415a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36414/0 (socket says 172.21.15.66:36414) 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff992800640 1 -- 172.21.15.66:0/617186520 learned_addr learned my addr 172.21.15.66:0/617186520 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff989400640 1 --2- 172.21.15.66:0/617186520 >> v2:172.21.15.156:3300/0 conn(0x7ff9940cde70 0x7ff9940cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff991e00640 1 --2- 172.21.15.66:0/617186520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9940cda90 0x7ff9940ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff992800640 1 -- 172.21.15.66:0/617186520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9940cda90 msgr2=0x7ff9940ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff992800640 1 --2- 172.21.15.66:0/617186520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9940cda90 0x7ff9940ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff992800640 1 -- 172.21.15.66:0/617186520 >> v2:172.21.15.156:3300/0 conn(0x7ff9940cde70 msgr2=0x7ff9940cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff992800640 1 --2- 172.21.15.66:0/617186520 >> v2:172.21.15.156:3300/0 conn(0x7ff9940cde70 0x7ff9940cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.617+0000 7ff992800640 1 -- 172.21.15.66:0/617186520 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9940b14c0 con 0x7ff994159c60 2024-03-30T00:40:51.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff992800640 1 --2- 172.21.15.66:0/617186520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff994159c60 0x7ff99415a0b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff9940cf0d0 tx=0x7ff98c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d4ab5c6d2d4b17 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:51.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff991400640 1 -- 172.21.15.66:0/617186520 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0a5800 con 0x7ff994159c60 2024-03-30T00:40:51.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff991400640 1 -- 172.21.15.66:0/617186520 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c0a5960 con 0x7ff994159c60 2024-03-30T00:40:51.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff991400640 1 -- 172.21.15.66:0/617186520 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0a5cf0 con 0x7ff994159c60 2024-03-30T00:40:51.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff998c00640 1 -- 172.21.15.66:0/617186520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff994159c60 msgr2=0x7ff99415a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff998c00640 1 --2- 172.21.15.66:0/617186520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff994159c60 0x7ff99415a0b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff9940cf0d0 tx=0x7ff98c01c960 comp rx=0 tx=0).stop 2024-03-30T00:40:51.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff998c00640 1 -- 172.21.15.66:0/617186520 shutdown_connections 2024-03-30T00:40:51.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff998c00640 1 --2- 172.21.15.66:0/617186520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff994159c60 0x7ff99415a0b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff998c00640 1 --2- 172.21.15.66:0/617186520 >> v2:172.21.15.156:3300/0 conn(0x7ff9940cde70 0x7ff9940cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff998c00640 1 --2- 172.21.15.66:0/617186520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9940cda90 0x7ff9940ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff998c00640 1 -- 172.21.15.66:0/617186520 >> 172.21.15.66:0/617186520 conn(0x7ff994069f60 msgr2=0x7ff99406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:51.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff998c00640 1 -- 172.21.15.66:0/617186520 shutdown_connections 2024-03-30T00:40:51.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff998c00640 1 -- 172.21.15.66:0/617186520 wait complete. 2024-03-30T00:40:51.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff998c00640 1 Processor -- start 2024-03-30T00:40:51.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.621+0000 7ff998c00640 1 -- start start 2024-03-30T00:40:51.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff998c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9940cda90 0x7ff9941649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff998c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9940cde70 0x7ff99409bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff998c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff994159c60 0x7ff99406fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff998c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940acc10 con 0x7ff9940cde70 2024-03-30T00:40:51.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff998c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99409c190 con 0x7ff9940cda90 2024-03-30T00:40:51.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff998c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940af470 con 0x7ff994159c60 2024-03-30T00:40:51.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff989400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9940cde70 0x7ff99409bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff989400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9940cde70 0x7ff99409bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36426/0 (socket says 172.21.15.66:36426) 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff989400640 1 -- 172.21.15.66:0/3743016971 learned_addr learned my addr 172.21.15.66:0/3743016971 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff991e00640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9940cda90 0x7ff9941649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff992800640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff994159c60 0x7ff99406fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff989400640 1 -- 172.21.15.66:0/3743016971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff994159c60 msgr2=0x7ff99406fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff989400640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff994159c60 0x7ff99406fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff989400640 1 -- 172.21.15.66:0/3743016971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9940cda90 msgr2=0x7ff9941649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff989400640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9940cda90 0x7ff9941649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff989400640 1 -- 172.21.15.66:0/3743016971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c009b30 con 0x7ff9940cde70 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff989400640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9940cde70 0x7ff99409bc50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00e930 tx=0x7ff97c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff98aa00640 1 -- 172.21.15.66:0/3743016971 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c009690 con 0x7ff9940cde70 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff98aa00640 1 -- 172.21.15.66:0/3743016971 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97c00ca00 con 0x7ff9940cde70 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff98aa00640 1 -- 172.21.15.66:0/3743016971 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c00cd70 con 0x7ff9940cde70 2024-03-30T00:40:51.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff998c00640 1 -- 172.21.15.66:0/3743016971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9940af730 con 0x7ff9940cde70 2024-03-30T00:40:51.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff998c00640 1 -- 172.21.15.66:0/3743016971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9940ac8a0 con 0x7ff9940cde70 2024-03-30T00:40:51.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff998c00640 1 -- 172.21.15.66:0/3743016971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff994066c30 con 0x7ff9940cde70 2024-03-30T00:40:51.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.625+0000 7ff98aa00640 1 -- 172.21.15.66:0/3743016971 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff97c013470 con 0x7ff9940cde70 2024-03-30T00:40:51.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.629+0000 7ff98aa00640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9700666c0 0x7ff970068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.629+0000 7ff98aa00640 1 -- 172.21.15.66:0/3743016971 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff97c0847f0 con 0x7ff9940cde70 2024-03-30T00:40:51.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.629+0000 7ff991e00640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9700666c0 0x7ff970068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.629+0000 7ff991e00640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9700666c0 0x7ff970068b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff9940ad7c0 tx=0x7ff984008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:51.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.633+0000 7ff98aa00640 1 -- 172.21.15.66:0/3743016971 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff97c051800 con 0x7ff9940cde70 2024-03-30T00:40:51.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 -- 172.21.15.66:0/3743016971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff9940b3170 con 0x7ff9940cde70 2024-03-30T00:40:51.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff98aa00640 1 -- 172.21.15.66:0/3743016971 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff97c054e20 con 0x7ff9940cde70 2024-03-30T00:40:51.803 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:51.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 -- 172.21.15.66:0/3743016971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9700666c0 msgr2=0x7ff970068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9700666c0 0x7ff970068b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff9940ad7c0 tx=0x7ff984008040 comp rx=0 tx=0).stop 2024-03-30T00:40:51.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 -- 172.21.15.66:0/3743016971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9940cde70 msgr2=0x7ff99409bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9940cde70 0x7ff99409bc50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00e930 tx=0x7ff97c00edf0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 -- 172.21.15.66:0/3743016971 shutdown_connections 2024-03-30T00:40:51.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff994159c60 0x7ff99406fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9700666c0 0x7ff970068b80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9940cde70 0x7ff99409bc50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 --2- 172.21.15.66:0/3743016971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9940cda90 0x7ff9941649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 -- 172.21.15.66:0/3743016971 >> 172.21.15.66:0/3743016971 conn(0x7ff994069f60 msgr2=0x7ff99406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:51.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 -- 172.21.15.66:0/3743016971 shutdown_connections 2024-03-30T00:40:51.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.793+0000 7ff998c00640 1 -- 172.21.15.66:0/3743016971 wait complete. 2024-03-30T00:40:51.824 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.957+0000 7fdd11e00640 1 Processor -- start 2024-03-30T00:40:51.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.957+0000 7fdd11e00640 1 -- start start 2024-03-30T00:40:51.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.957+0000 7fdd11e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd0c0ce820 0x7fdd0c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.957+0000 7fdd11e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c06e7d0 con 0x7fdd0c15ca10 2024-03-30T00:40:51.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.957+0000 7fdd11e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c15cdf0 con 0x7fdd0c0ae4e0 2024-03-30T00:40:51.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.957+0000 7fdd11e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0c0b1f70 con 0x7fdd0c0ce820 2024-03-30T00:40:51.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.957+0000 7fdd0aa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdd0c15ca10 0x7fdd0c06ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33346/0 (socket says 172.21.15.66:33346) 2024-03-30T00:40:51.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.957+0000 7fdd0aa00640 1 -- 172.21.15.66:0/2454612842 learned_addr learned my addr 172.21.15.66:0/2454612842 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:51.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0b400640 1 --2- 172.21.15.66:0/2454612842 >> v2:172.21.15.156:3300/0 conn(0x7fdd0c0ce820 0x7fdd0c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031826462 0 0) 0x7fdd0c06e7d0 con 0x7fdd0c15ca10 2024-03-30T00:40:51.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf0003530 con 0x7fdd0c15ca10 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290636025 0 0) 0x7fdd0c15cdf0 con 0x7fdd0c0ae4e0 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd0c06e7d0 con 0x7fdd0c0ae4e0 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2751537224 0 0) 0x7fdcf0003530 con 0x7fdd0c15ca10 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 >> v2:172.21.15.156:3300/0 conn(0x7fdd0c0ce820 msgr2=0x7fdd0c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 --2- 172.21.15.66:0/2454612842 >> v2:172.21.15.156:3300/0 conn(0x7fdd0c0ce820 0x7fdd0c0cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 >> v1:172.21.15.156:6789/0 conn(0x7fdd0c0ae4e0 legacy=0x7fdd0c0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c0b16d0 con 0x7fdd0c15ca10 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fdcfc001680 con 0x7fdd0c15ca10 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 >> v1:172.21.15.66:6789/0 conn(0x7fdd0c15ca10 legacy=0x7fdd0c06ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 --2- 172.21.15.66:0/2454612842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdcf0003b60 0x7fdcf0003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 --2- 172.21.15.66:0/2454612842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcf00047d0 0x7fdcf0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 --2- 172.21.15.66:0/2454612842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0005190 0x7fdcf000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0b1f70 con 0x7fdcf0005190 2024-03-30T00:40:51.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf000c060 con 0x7fdcf00047d0 2024-03-30T00:40:51.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf000c220 con 0x7fdcf0003b60 2024-03-30T00:40:51.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0b400640 1 --2- 172.21.15.66:0/2454612842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0005190 0x7fdcf000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0aa00640 1 --2- 172.21.15.66:0/2454612842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdcf0003b60 0x7fdcf0003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd02000640 1 --2- 172.21.15.66:0/2454612842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcf00047d0 0x7fdcf0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0b400640 1 -- 172.21.15.66:0/2454612842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdcf0003b60 msgr2=0x7fdcf0003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:51.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0b400640 1 --2- 172.21.15.66:0/2454612842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdcf0003b60 0x7fdcf0003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0b400640 1 -- 172.21.15.66:0/2454612842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcf00047d0 msgr2=0x7fdcf0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:51.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0b400640 1 --2- 172.21.15.66:0/2454612842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcf00047d0 0x7fdcf0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.961+0000 7fdd0b400640 1 -- 172.21.15.66:0/2454612842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf0003530 con 0x7fdcf0005190 2024-03-30T00:40:51.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd0b400640 1 -- 172.21.15.66:0/2454612842 reap_dead start 2024-03-30T00:40:51.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd0b400640 1 --2- 172.21.15.66:0/2454612842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0005190 0x7fdcf000b990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009fc0 tx=0x7fdd0401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e2b220a8b12a5a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:51.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040a5cd0 con 0x7fdcf0005190 2024-03-30T00:40:51.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0400f040 con 0x7fdcf0005190 2024-03-30T00:40:51.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd0a000640 1 -- 172.21.15.66:0/2454612842 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04003be0 con 0x7fdcf0005190 2024-03-30T00:40:51.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 -- 172.21.15.66:0/2454612842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0005190 msgr2=0x7fdcf000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 --2- 172.21.15.66:0/2454612842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0005190 0x7fdcf000b990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009fc0 tx=0x7fdd0401ccc0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 -- 172.21.15.66:0/2454612842 shutdown_connections 2024-03-30T00:40:51.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 --2- 172.21.15.66:0/2454612842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0005190 0x7fdcf000b990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 -- 172.21.15.66:0/2454612842 >> 172.21.15.66:0/2454612842 conn(0x7fdd0c098a20 msgr2=0x7fdd0c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:51.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 -- 172.21.15.66:0/2454612842 shutdown_connections 2024-03-30T00:40:51.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 -- 172.21.15.66:0/2454612842 wait complete. 2024-03-30T00:40:51.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 Processor -- start 2024-03-30T00:40:51.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 -- start start 2024-03-30T00:40:51.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcf0005190 0x7fdd0c0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c146460 0x7fdd0c0b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd0c0b2300 0x7fdd0c0b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0b4f80 con 0x7fdd0c146460 2024-03-30T00:40:51.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0b2ba0 con 0x7fdcf0005190 2024-03-30T00:40:51.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.965+0000 7fdd11e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c156d90 con 0x7fdd0c0b2300 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd02000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c146460 0x7fdd0c0b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd02000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c146460 0x7fdd0c0b4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36446/0 (socket says 172.21.15.66:36446) 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd02000640 1 -- 172.21.15.66:0/2263960883 learned_addr learned my addr 172.21.15.66:0/2263960883 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd0aa00640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcf0005190 0x7fdd0c0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd0b400640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd0c0b2300 0x7fdd0c0b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd02000640 1 -- 172.21.15.66:0/2263960883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd0c0b2300 msgr2=0x7fdd0c0b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd02000640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd0c0b2300 0x7fdd0c0b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd02000640 1 -- 172.21.15.66:0/2263960883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcf0005190 msgr2=0x7fdd0c0b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd02000640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcf0005190 0x7fdd0c0b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd02000640 1 -- 172.21.15.66:0/2263960883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c0b28c0 con 0x7fdd0c146460 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd0aa00640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcf0005190 0x7fdd0c0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd0b400640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd0c0b2300 0x7fdd0c0b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd02000640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c146460 0x7fdd0c0b4500 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40079b0 tx=0x7fdcf400fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd03e00640 1 -- 172.21.15.66:0/2263960883 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4005110 con 0x7fdd0c146460 2024-03-30T00:40:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd03e00640 1 -- 172.21.15.66:0/2263960883 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf4005270 con 0x7fdd0c146460 2024-03-30T00:40:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd11e00640 1 -- 172.21.15.66:0/2263960883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0c0b0ac0 con 0x7fdd0c146460 2024-03-30T00:40:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd03e00640 1 -- 172.21.15.66:0/2263960883 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf40067e0 con 0x7fdd0c146460 2024-03-30T00:40:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd11e00640 1 -- 172.21.15.66:0/2263960883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0c0b1080 con 0x7fdd0c146460 2024-03-30T00:40:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd11e00640 1 -- 172.21.15.66:0/2263960883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0c0b0920 con 0x7fdd0c146460 2024-03-30T00:40:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd03e00640 1 -- 172.21.15.66:0/2263960883 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdcf4006940 con 0x7fdd0c146460 2024-03-30T00:40:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.969+0000 7fdd03e00640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdce8066690 0x7fdce8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.973+0000 7fdd03e00640 1 -- 172.21.15.66:0/2263960883 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fdcf4083bf0 con 0x7fdd0c146460 2024-03-30T00:40:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.973+0000 7fdd0aa00640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdce8066690 0x7fdce8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.973+0000 7fdd0aa00640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdce8066690 0x7fdce8068b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc000c10 tx=0x7fdcfc000ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:51.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:51.973+0000 7fdd03e00640 1 -- 172.21.15.66:0/2263960883 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdd0c0b0920 con 0x7fdd0c146460 2024-03-30T00:40:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.133+0000 7fdd11e00640 1 -- 172.21.15.66:0/2263960883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdd0c0b0920 con 0x7fdd0c146460 2024-03-30T00:40:52.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.133+0000 7fdd03e00640 1 -- 172.21.15.66:0/2263960883 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdd0c0b0920 con 0x7fdd0c146460 2024-03-30T00:40:52.144 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:52.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 -- 172.21.15.66:0/2263960883 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdce8066690 msgr2=0x7fdce8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdce8066690 0x7fdce8068b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc000c10 tx=0x7fdcfc000ee0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 -- 172.21.15.66:0/2263960883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c146460 msgr2=0x7fdd0c0b4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c146460 0x7fdd0c0b4500 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40079b0 tx=0x7fdcf400fb40 comp rx=0 tx=0).stop 2024-03-30T00:40:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 -- 172.21.15.66:0/2263960883 shutdown_connections 2024-03-30T00:40:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd0c0b2300 0x7fdd0c0b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdce8066690 0x7fdce8068b50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd0c146460 0x7fdd0c0b4500 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 --2- 172.21.15.66:0/2263960883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcf0005190 0x7fdd0c0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 -- 172.21.15.66:0/2263960883 >> 172.21.15.66:0/2263960883 conn(0x7fdd0c098a20 msgr2=0x7fdd0c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 -- 172.21.15.66:0/2263960883 shutdown_connections 2024-03-30T00:40:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.137+0000 7fdd11e00640 1 -- 172.21.15.66:0/2263960883 wait complete. 2024-03-30T00:40:52.165 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:40:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.297+0000 7efe03600640 1 Processor -- start 2024-03-30T00:40:52.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.297+0000 7efe03600640 1 -- start start 2024-03-30T00:40:52.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.297+0000 7efe03600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 0x7efdfc06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.297+0000 7efe03600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efdfc0ae4e0 0x7efdfc0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.297+0000 7efe03600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdfc06e7d0 con 0x7efdfc0ce820 2024-03-30T00:40:52.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.297+0000 7efe03600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc0b1f70 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.297+0000 7efe03600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc06f1a0 con 0x7efdfc15ca10 2024-03-30T00:40:52.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.297+0000 7efe00c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 0x7efdfc06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.297+0000 7efdf3e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efdfc0ae4e0 0x7efdfc0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.297+0000 7efe00c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 0x7efdfc06ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:46198/0 (socket says 172.21.15.66:46198) 2024-03-30T00:40:52.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.297+0000 7efe00c00640 1 -- 172.21.15.66:0/424646770 learned_addr learned my addr 172.21.15.66:0/424646770 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:52.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.301+0000 7efe00c00640 1 -- 172.21.15.66:0/424646770 >> v2:172.21.15.156:3300/0 conn(0x7efdfc0ae4e0 msgr2=0x7efdfc0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:52.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.301+0000 7efe00c00640 1 --2- 172.21.15.66:0/424646770 >> v2:172.21.15.156:3300/0 conn(0x7efdfc0ae4e0 0x7efdfc0cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.301+0000 7efe00c00640 1 -- 172.21.15.66:0/424646770 >> v1:172.21.15.156:6789/0 conn(0x7efdfc0ce820 legacy=0x7efdfc160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.301+0000 7efe00c00640 1 -- 172.21.15.66:0/424646770 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc15cdf0 con 0x7efdfc15ca10 2024-03-30T00:40:52.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.301+0000 7efe00c00640 1 --2- 172.21.15.66:0/424646770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 0x7efdfc06ec60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efdec009870 tx=0x7efdec01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59fd7f5ebb75d15e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:52.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.301+0000 7efdfbe00640 1 -- 172.21.15.66:0/424646770 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0ab640 con 0x7efdfc15ca10 2024-03-30T00:40:52.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.301+0000 7efdfbe00640 1 -- 172.21.15.66:0/424646770 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec0a7070 con 0x7efdfc15ca10 2024-03-30T00:40:52.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.301+0000 7efdfbe00640 1 -- 172.21.15.66:0/424646770 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0ab9c0 con 0x7efdfc15ca10 2024-03-30T00:40:52.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 -- 172.21.15.66:0/424646770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 msgr2=0x7efdfc06ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 --2- 172.21.15.66:0/424646770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 0x7efdfc06ec60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efdec009870 tx=0x7efdec01c530 comp rx=0 tx=0).stop 2024-03-30T00:40:52.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 -- 172.21.15.66:0/424646770 shutdown_connections 2024-03-30T00:40:52.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 --2- 172.21.15.66:0/424646770 >> v2:172.21.15.156:3300/0 conn(0x7efdfc0ae4e0 0x7efdfc0cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 --2- 172.21.15.66:0/424646770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 0x7efdfc06ec60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 -- 172.21.15.66:0/424646770 >> 172.21.15.66:0/424646770 conn(0x7efdfc098a20 msgr2=0x7efdfc0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:52.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 -- 172.21.15.66:0/424646770 shutdown_connections 2024-03-30T00:40:52.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 -- 172.21.15.66:0/424646770 wait complete. 2024-03-30T00:40:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 Processor -- start 2024-03-30T00:40:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 -- start start 2024-03-30T00:40:52.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0ae4e0 0x7efdfc0c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0ce820 0x7efdfc0c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 0x7efdfc15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0c90e0 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0c7b00 con 0x7efdfc0ce820 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe03600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0c7de0 con 0x7efdfc15ca10 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe00c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0ae4e0 0x7efdfc0c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe00c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0ae4e0 0x7efdfc0c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36448/0 (socket says 172.21.15.66:36448) 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe00c00640 1 -- 172.21.15.66:0/1951798386 learned_addr learned my addr 172.21.15.66:0/1951798386 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efdf3e00640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0ce820 0x7efdfc0c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe01600640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 0x7efdfc15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe00c00640 1 -- 172.21.15.66:0/1951798386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 msgr2=0x7efdfc15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe00c00640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 0x7efdfc15afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe00c00640 1 -- 172.21.15.66:0/1951798386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0ce820 msgr2=0x7efdfc0c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe00c00640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0ce820 0x7efdfc0c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.305+0000 7efe00c00640 1 -- 172.21.15.66:0/1951798386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec0094d0 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.309+0000 7efe00c00640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0ae4e0 0x7efdfc0c8ba0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efdec0ad040 tx=0x7efdec0192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:52.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.309+0000 7efdf9600640 1 -- 172.21.15.66:0/1951798386 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0bf410 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.309+0000 7efdf9600640 1 -- 172.21.15.66:0/1951798386 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec0bf5b0 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.309+0000 7efdf9600640 1 -- 172.21.15.66:0/1951798386 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0bf960 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.309+0000 7efe03600640 1 -- 172.21.15.66:0/1951798386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc0c80c0 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.309+0000 7efe03600640 1 -- 172.21.15.66:0/1951798386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc0c8680 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.309+0000 7efe03600640 1 -- 172.21.15.66:0/1951798386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc0c8870 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.309+0000 7efdf9600640 1 -- 172.21.15.66:0/1951798386 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efdec0b5400 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.309+0000 7efdf9600640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd80666c0 0x7efdd8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.309+0000 7efdf9600640 1 -- 172.21.15.66:0/1951798386 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7efdec11eac0 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.309+0000 7efdf3e00640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd80666c0 0x7efdd8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.313+0000 7efdf3e00640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd80666c0 0x7efdd8068b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0c76d0 tx=0x7efde4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:52.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.313+0000 7efdf9600640 1 -- 172.21.15.66:0/1951798386 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efdec0ebae0 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.485+0000 7efe03600640 1 -- 172.21.15.66:0/1951798386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efdfc0c7f20 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.485+0000 7efdf9600640 1 -- 172.21.15.66:0/1951798386 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7efdfc0c7f20 con 0x7efdfc0ae4e0 2024-03-30T00:40:52.495 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:40:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.485+0000 7efe03600640 1 -- 172.21.15.66:0/1951798386 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd80666c0 msgr2=0x7efdd8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.485+0000 7efe03600640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd80666c0 0x7efdd8068b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0c76d0 tx=0x7efde4009190 comp rx=0 tx=0).stop 2024-03-30T00:40:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.489+0000 7efe03600640 1 -- 172.21.15.66:0/1951798386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0ae4e0 msgr2=0x7efdfc0c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.489+0000 7efe03600640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0ae4e0 0x7efdfc0c8ba0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efdec0ad040 tx=0x7efdec0192d0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.489+0000 7efe03600640 1 -- 172.21.15.66:0/1951798386 shutdown_connections 2024-03-30T00:40:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.489+0000 7efe03600640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efdfc15ca10 0x7efdfc15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.489+0000 7efe03600640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0ce820 0x7efdfc0c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.489+0000 7efe03600640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efdd80666c0 0x7efdd8068b80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.489+0000 7efe03600640 1 --2- 172.21.15.66:0/1951798386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc0ae4e0 0x7efdfc0c8ba0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.489+0000 7efe03600640 1 -- 172.21.15.66:0/1951798386 >> 172.21.15.66:0/1951798386 conn(0x7efdfc098a20 msgr2=0x7efdfc0c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.489+0000 7efe03600640 1 -- 172.21.15.66:0/1951798386 shutdown_connections 2024-03-30T00:40:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.489+0000 7efe03600640 1 -- 172.21.15.66:0/1951798386 wait complete. 2024-03-30T00:40:52.516 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:40:52.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:52.516 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:40:52.536 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:40:52.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:40:52.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:52.536 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:40:52.593 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:40:52.593 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:40:52.602 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:40:52.602 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:40:52.652 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:52.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747e00640 1 Processor -- start 2024-03-30T00:40:52.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747e00640 1 -- start start 2024-03-30T00:40:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb7480b31f0 0x7fb748069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb74806a2d0 0x7fb7480ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74806f810 con 0x7fb74815c720 2024-03-30T00:40:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74806a740 con 0x7fb74806a2d0 2024-03-30T00:40:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74806ec90 con 0x7fb7480b31f0 2024-03-30T00:40:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb74806a2d0 0x7fb7480ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb74806a2d0 0x7fb7480ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36460/0 (socket says 172.21.15.66:36460) 2024-03-30T00:40:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747400640 1 -- 172.21.15.66:0/3007771414 learned_addr learned my addr 172.21.15.66:0/3007771414 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb73fe00640 1 --2- 172.21.15.66:0/3007771414 >> v2:172.21.15.156:3300/0 conn(0x7fb7480b31f0 0x7fb748069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747400640 1 -- 172.21.15.66:0/3007771414 >> v2:172.21.15.156:3300/0 conn(0x7fb7480b31f0 msgr2=0x7fb748069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747400640 1 --2- 172.21.15.66:0/3007771414 >> v2:172.21.15.156:3300/0 conn(0x7fb7480b31f0 0x7fb748069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747400640 1 -- 172.21.15.66:0/3007771414 >> v1:172.21.15.156:6789/0 conn(0x7fb74815c720 legacy=0x7fb74806e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.785+0000 7fb747400640 1 -- 172.21.15.66:0/3007771414 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74806ef70 con 0x7fb74806a2d0 2024-03-30T00:40:52.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb747400640 1 --2- 172.21.15.66:0/3007771414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74806a2d0 0x7fb7480ce960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb74806dc40 tx=0x7fb74001f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c599342f8400bf68 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb746000640 1 -- 172.21.15.66:0/3007771414 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7400af400 con 0x7fb74806a2d0 2024-03-30T00:40:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb746000640 1 -- 172.21.15.66:0/3007771414 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7400ab050 con 0x7fb74806a2d0 2024-03-30T00:40:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb746000640 1 -- 172.21.15.66:0/3007771414 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7400af780 con 0x7fb74806a2d0 2024-03-30T00:40:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb747e00640 1 -- 172.21.15.66:0/3007771414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74806a2d0 msgr2=0x7fb7480ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb747e00640 1 --2- 172.21.15.66:0/3007771414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74806a2d0 0x7fb7480ce960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb74806dc40 tx=0x7fb74001f1a0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb747e00640 1 -- 172.21.15.66:0/3007771414 shutdown_connections 2024-03-30T00:40:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb747e00640 1 --2- 172.21.15.66:0/3007771414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74806a2d0 0x7fb7480ce960 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb747e00640 1 --2- 172.21.15.66:0/3007771414 >> v2:172.21.15.156:3300/0 conn(0x7fb7480b31f0 0x7fb748069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb747e00640 1 -- 172.21.15.66:0/3007771414 >> 172.21.15.66:0/3007771414 conn(0x7fb74806cf30 msgr2=0x7fb74806d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb747e00640 1 -- 172.21.15.66:0/3007771414 shutdown_connections 2024-03-30T00:40:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.789+0000 7fb747e00640 1 -- 172.21.15.66:0/3007771414 wait complete. 2024-03-30T00:40:52.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb747e00640 1 Processor -- start 2024-03-30T00:40:52.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb747e00640 1 -- start start 2024-03-30T00:40:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb747e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb74806a2d0 0x7fb74815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb747e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7480b31f0 0x7fb7480aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb747e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb74815c720 0x7fb7481f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb747e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7480ab890 con 0x7fb7480b31f0 2024-03-30T00:40:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb747e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7481f2e20 con 0x7fb74806a2d0 2024-03-30T00:40:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb747e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74815e3d0 con 0x7fb74815c720 2024-03-30T00:40:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7480b31f0 0x7fb7480aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7480b31f0 0x7fb7480aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36474/0 (socket says 172.21.15.66:36474) 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73fe00640 1 -- 172.21.15.66:0/777573993 learned_addr learned my addr 172.21.15.66:0/777573993 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb747400640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb74815c720 0x7fb7481f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb746a00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb74806a2d0 0x7fb74815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73fe00640 1 -- 172.21.15.66:0/777573993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb74815c720 msgr2=0x7fb7481f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73fe00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb74815c720 0x7fb7481f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73fe00640 1 -- 172.21.15.66:0/777573993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb74806a2d0 msgr2=0x7fb74815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73fe00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb74806a2d0 0x7fb74815ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73fe00640 1 -- 172.21.15.66:0/777573993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb740009b30 con 0x7fb7480b31f0 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73fe00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7480b31f0 0x7fb7480aed60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb73000d800 tx=0x7fb73000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73f400640 1 -- 172.21.15.66:0/777573993 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73000b6c0 con 0x7fb7480b31f0 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73f400640 1 -- 172.21.15.66:0/777573993 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73000b820 con 0x7fb7480b31f0 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb73f400640 1 -- 172.21.15.66:0/777573993 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73000bb90 con 0x7fb7480b31f0 2024-03-30T00:40:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb747e00640 1 -- 172.21.15.66:0/777573993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74815e6b0 con 0x7fb7480b31f0 2024-03-30T00:40:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.793+0000 7fb747e00640 1 -- 172.21.15.66:0/777573993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74815ec70 con 0x7fb7480b31f0 2024-03-30T00:40:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.797+0000 7fb747e00640 1 -- 172.21.15.66:0/777573993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb748073cf0 con 0x7fb7480b31f0 2024-03-30T00:40:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.797+0000 7fb73f400640 1 -- 172.21.15.66:0/777573993 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb73000bcf0 con 0x7fb7480b31f0 2024-03-30T00:40:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.797+0000 7fb73f400640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7240666e0 0x7fb724068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.797+0000 7fb73f400640 1 -- 172.21.15.66:0/777573993 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fb730084940 con 0x7fb7480b31f0 2024-03-30T00:40:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.797+0000 7fb746a00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7240666e0 0x7fb724068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.797+0000 7fb746a00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7240666e0 0x7fb724068ba0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb738007fd0 tx=0x7fb738007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.801+0000 7fb73f400640 1 -- 172.21.15.66:0/777573993 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb7300519d0 con 0x7fb7480b31f0 2024-03-30T00:40:52.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 -- 172.21.15.66:0/777573993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7480b0d40 con 0x7fb7480b31f0 2024-03-30T00:40:52.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb73f400640 1 -- 172.21.15.66:0/777573993 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7fb730054ff0 con 0x7fb7480b31f0 2024-03-30T00:40:52.974 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:52.974 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:52.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 -- 172.21.15.66:0/777573993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7240666e0 msgr2=0x7fb724068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7240666e0 0x7fb724068ba0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb738007fd0 tx=0x7fb738007e40 comp rx=0 tx=0).stop 2024-03-30T00:40:52.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 -- 172.21.15.66:0/777573993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7480b31f0 msgr2=0x7fb7480aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:52.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7480b31f0 0x7fb7480aed60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb73000d800 tx=0x7fb73000dcc0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 -- 172.21.15.66:0/777573993 shutdown_connections 2024-03-30T00:40:52.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb74815c720 0x7fb7481f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7240666e0 0x7fb724068ba0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7480b31f0 0x7fb7480aed60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 --2- 172.21.15.66:0/777573993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb74806a2d0 0x7fb74815ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:52.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 -- 172.21.15.66:0/777573993 >> 172.21.15.66:0/777573993 conn(0x7fb74806cf30 msgr2=0x7fb7480ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:52.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 -- 172.21.15.66:0/777573993 shutdown_connections 2024-03-30T00:40:52.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:52.965+0000 7fb747e00640 1 -- 172.21.15.66:0/777573993 wait complete. 2024-03-30T00:40:52.996 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:52.997 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.129+0000 7f5296c00640 1 Processor -- start 2024-03-30T00:40:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.129+0000 7f5296c00640 1 -- start start 2024-03-30T00:40:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5296c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5290098c80 0x7f5290099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5296c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52900cf320 0x7f52900ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5296c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5290099ca0 0x7f52900ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5296c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52900cbd90 con 0x7f5290099ca0 2024-03-30T00:40:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5296c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52900cc070 con 0x7f5290098c80 2024-03-30T00:40:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5296c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52900cc350 con 0x7f52900cf320 2024-03-30T00:40:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5294c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5290099ca0 0x7f52900ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5294c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5290099ca0 0x7f52900ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36478/0 (socket says 172.21.15.66:36478) 2024-03-30T00:40:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5294c00640 1 -- 172.21.15.66:0/2878640845 learned_addr learned my addr 172.21.15.66:0/2878640845 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:53.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5287e00640 1 --2- 172.21.15.66:0/2878640845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52900cf320 0x7f52900ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f528fe00640 1 --2- 172.21.15.66:0/2878640845 >> v2:172.21.15.156:3300/0 conn(0x7f5290098c80 0x7f5290099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5294c00640 1 -- 172.21.15.66:0/2878640845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52900cf320 msgr2=0x7f52900ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:53.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5294c00640 1 --2- 172.21.15.66:0/2878640845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52900cf320 0x7f52900ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5294c00640 1 -- 172.21.15.66:0/2878640845 >> v2:172.21.15.156:3300/0 conn(0x7f5290098c80 msgr2=0x7f5290099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:53.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5294c00640 1 --2- 172.21.15.66:0/2878640845 >> v2:172.21.15.156:3300/0 conn(0x7f5290098c80 0x7f5290099760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.133+0000 7f5294c00640 1 -- 172.21.15.66:0/2878640845 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52900b3530 con 0x7f5290099ca0 2024-03-30T00:40:53.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5294c00640 1 --2- 172.21.15.66:0/2878640845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5290099ca0 0x7f52900ce750 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5290071550 tx=0x7f528801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9a51b8126f8466 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:53.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f528f400640 1 -- 172.21.15.66:0/2878640845 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880af400 con 0x7f5290099ca0 2024-03-30T00:40:53.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f528f400640 1 -- 172.21.15.66:0/2878640845 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52880ab050 con 0x7f5290099ca0 2024-03-30T00:40:53.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f528f400640 1 -- 172.21.15.66:0/2878640845 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880af780 con 0x7f5290099ca0 2024-03-30T00:40:53.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 -- 172.21.15.66:0/2878640845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5290099ca0 msgr2=0x7f52900ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 --2- 172.21.15.66:0/2878640845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5290099ca0 0x7f52900ce750 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5290071550 tx=0x7f528801f1e0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 -- 172.21.15.66:0/2878640845 shutdown_connections 2024-03-30T00:40:53.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 --2- 172.21.15.66:0/2878640845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5290099ca0 0x7f52900ce750 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 --2- 172.21.15.66:0/2878640845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52900cf320 0x7f52900ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 --2- 172.21.15.66:0/2878640845 >> v2:172.21.15.156:3300/0 conn(0x7f5290098c80 0x7f5290099760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 -- 172.21.15.66:0/2878640845 >> 172.21.15.66:0/2878640845 conn(0x7f52900b2130 msgr2=0x7f52900b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:53.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 -- 172.21.15.66:0/2878640845 shutdown_connections 2024-03-30T00:40:53.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 -- 172.21.15.66:0/2878640845 wait complete. 2024-03-30T00:40:53.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 Processor -- start 2024-03-30T00:40:53.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 -- start start 2024-03-30T00:40:53.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5290098c80 0x7f5290154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5290099ca0 0x7f52900c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52900cf320 0x7f5290156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52901555b0 con 0x7f52900cf320 2024-03-30T00:40:53.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5290154dc0 con 0x7f5290099ca0 2024-03-30T00:40:53.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.137+0000 7f5296c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5290154ff0 con 0x7f5290098c80 2024-03-30T00:40:53.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5294c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52900cf320 0x7f5290156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f528fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5290098c80 0x7f5290154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5287e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5290099ca0 0x7f52900c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5294c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52900cf320 0x7f5290156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36490/0 (socket says 172.21.15.66:36490) 2024-03-30T00:40:53.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5294c00640 1 -- 172.21.15.66:0/1891440146 learned_addr learned my addr 172.21.15.66:0/1891440146 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:53.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5294c00640 1 -- 172.21.15.66:0/1891440146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5290098c80 msgr2=0x7f5290154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5294c00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5290098c80 0x7f5290154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5294c00640 1 -- 172.21.15.66:0/1891440146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5290099ca0 msgr2=0x7f52900c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5294c00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5290099ca0 0x7f52900c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5294c00640 1 -- 172.21.15.66:0/1891440146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5288009b30 con 0x7f52900cf320 2024-03-30T00:40:53.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5294c00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52900cf320 0x7f5290156e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f52880b2040 tx=0x7f528801bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:53.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f528cc00640 1 -- 172.21.15.66:0/1891440146 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880afde0 con 0x7f52900cf320 2024-03-30T00:40:53.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f528cc00640 1 -- 172.21.15.66:0/1891440146 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288009510 con 0x7f52900cf320 2024-03-30T00:40:53.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f528cc00640 1 -- 172.21.15.66:0/1891440146 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880ba660 con 0x7f52900cf320 2024-03-30T00:40:53.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5296c00640 1 -- 172.21.15.66:0/1891440146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52901552d0 con 0x7f52900cf320 2024-03-30T00:40:53.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5296c00640 1 -- 172.21.15.66:0/1891440146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52901477f0 con 0x7f52900cf320 2024-03-30T00:40:53.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f5296c00640 1 -- 172.21.15.66:0/1891440146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52900677d0 con 0x7f52900cf320 2024-03-30T00:40:53.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.141+0000 7f528cc00640 1 -- 172.21.15.66:0/1891440146 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f52880b6040 con 0x7f52900cf320 2024-03-30T00:40:53.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.145+0000 7f528cc00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f526c0666c0 0x7f526c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.145+0000 7f528cc00640 1 -- 172.21.15.66:0/1891440146 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f5288122740 con 0x7f52900cf320 2024-03-30T00:40:53.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.145+0000 7f528fe00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f526c0666c0 0x7f526c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.145+0000 7f528fe00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f526c0666c0 0x7f526c068b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f52800045f0 tx=0x7f5280008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:53.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.149+0000 7f528cc00640 1 -- 172.21.15.66:0/1891440146 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f52880ef780 con 0x7f52900cf320 2024-03-30T00:40:53.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.309+0000 7f5296c00640 1 -- 172.21.15.66:0/1891440146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5290147370 con 0x7f52900cf320 2024-03-30T00:40:53.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.309+0000 7f528cc00640 1 -- 172.21.15.66:0/1891440146 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5290147370 con 0x7f52900cf320 2024-03-30T00:40:53.320 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 -- 172.21.15.66:0/1891440146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f526c0666c0 msgr2=0x7f526c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f526c0666c0 0x7f526c068b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f52800045f0 tx=0x7f5280008040 comp rx=0 tx=0).stop 2024-03-30T00:40:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 -- 172.21.15.66:0/1891440146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52900cf320 msgr2=0x7f5290156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52900cf320 0x7f5290156e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f52880b2040 tx=0x7f528801bf80 comp rx=0 tx=0).stop 2024-03-30T00:40:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 -- 172.21.15.66:0/1891440146 shutdown_connections 2024-03-30T00:40:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f526c0666c0 0x7f526c068b80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52900cf320 0x7f5290156e20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5290099ca0 0x7f52900c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 --2- 172.21.15.66:0/1891440146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5290098c80 0x7f5290154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 -- 172.21.15.66:0/1891440146 >> 172.21.15.66:0/1891440146 conn(0x7f52900b2130 msgr2=0x7f52900b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 -- 172.21.15.66:0/1891440146 shutdown_connections 2024-03-30T00:40:53.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.313+0000 7f5296c00640 1 -- 172.21.15.66:0/1891440146 wait complete. 2024-03-30T00:40:53.341 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:53.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5ce7a00640 1 Processor -- start 2024-03-30T00:40:53.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5ce7a00640 1 -- start start 2024-03-30T00:40:53.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5ce7a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5ce00cd2d0 0x7f5ce00cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5ce7a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5ce00cc8d0 0x7f5ce006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5ce7a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce00cb620 con 0x7f5ce00cad00 2024-03-30T00:40:53.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5ce7a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce00cbec0 con 0x7f5ce00cd2d0 2024-03-30T00:40:53.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5ce7a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce00ca660 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5cdfe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5ce00cd2d0 0x7f5ce00cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5cdfe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5ce00cd2d0 0x7f5ce00cb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36494/0 (socket says 172.21.15.66:36494) 2024-03-30T00:40:53.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5cdfe00640 1 -- 172.21.15.66:0/132286781 learned_addr learned my addr 172.21.15.66:0/132286781 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:53.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5ce5a00640 1 --2- 172.21.15.66:0/132286781 >> v2:172.21.15.156:3300/0 conn(0x7f5ce00cc8d0 0x7f5ce006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5cdfe00640 1 -- 172.21.15.66:0/132286781 >> v2:172.21.15.156:3300/0 conn(0x7f5ce00cc8d0 msgr2=0x7f5ce006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:53.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5cdfe00640 1 --2- 172.21.15.66:0/132286781 >> v2:172.21.15.156:3300/0 conn(0x7f5ce00cc8d0 0x7f5ce006c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5cdfe00640 1 -- 172.21.15.66:0/132286781 >> v1:172.21.15.156:6789/0 conn(0x7f5ce00cad00 legacy=0x7f5ce00c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.477+0000 7f5cdfe00640 1 -- 172.21.15.66:0/132286781 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce00ca940 con 0x7f5ce00cd2d0 2024-03-30T00:40:53.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5cdfe00640 1 --2- 172.21.15.66:0/132286781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce00cd2d0 0x7f5ce00cb0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5ce006e4b0 tx=0x7f5ccc01e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d811f1f5a66d40 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:53.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5cdf400640 1 -- 172.21.15.66:0/132286781 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc004230 con 0x7f5ce00cd2d0 2024-03-30T00:40:53.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5cdf400640 1 -- 172.21.15.66:0/132286781 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc0ab070 con 0x7f5ce00cd2d0 2024-03-30T00:40:53.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5cdf400640 1 -- 172.21.15.66:0/132286781 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0a7870 con 0x7f5ce00cd2d0 2024-03-30T00:40:53.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5ce7a00640 1 -- 172.21.15.66:0/132286781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce00cd2d0 msgr2=0x7f5ce00cb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5ce7a00640 1 --2- 172.21.15.66:0/132286781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce00cd2d0 0x7f5ce00cb0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5ce006e4b0 tx=0x7f5ccc01e9a0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5ce7a00640 1 -- 172.21.15.66:0/132286781 shutdown_connections 2024-03-30T00:40:53.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5ce7a00640 1 --2- 172.21.15.66:0/132286781 >> v2:172.21.15.156:3300/0 conn(0x7f5ce00cc8d0 0x7f5ce006c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5ce7a00640 1 --2- 172.21.15.66:0/132286781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce00cd2d0 0x7f5ce00cb0e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5ce7a00640 1 -- 172.21.15.66:0/132286781 >> 172.21.15.66:0/132286781 conn(0x7f5ce0069f60 msgr2=0x7f5ce006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:53.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5ce7a00640 1 -- 172.21.15.66:0/132286781 shutdown_connections 2024-03-30T00:40:53.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5ce7a00640 1 -- 172.21.15.66:0/132286781 wait complete. 2024-03-30T00:40:53.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5ce7a00640 1 Processor -- start 2024-03-30T00:40:53.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.481+0000 7f5ce7a00640 1 -- start start 2024-03-30T00:40:53.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5ce7a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ce00cad00 0x7f5ce015a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5ce7a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce00cc8d0 0x7f5ce00c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5ce7a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5ce00cd2d0 0x7f5ce0157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5ce7a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce015aa50 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5ce7a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce015ad30 con 0x7f5ce00cad00 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5ce7a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce015b010 con 0x7f5ce00cd2d0 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce00cc8d0 0x7f5ce00c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce00cc8d0 0x7f5ce00c0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36502/0 (socket says 172.21.15.66:36502) 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdfe00640 1 -- 172.21.15.66:0/2849176940 learned_addr learned my addr 172.21.15.66:0/2849176940 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5ce5000640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ce00cad00 0x7f5ce015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5ce5a00640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5ce00cd2d0 0x7f5ce0157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdfe00640 1 -- 172.21.15.66:0/2849176940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5ce00cd2d0 msgr2=0x7f5ce0157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdfe00640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5ce00cd2d0 0x7f5ce0157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdfe00640 1 -- 172.21.15.66:0/2849176940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ce00cad00 msgr2=0x7f5ce015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdfe00640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ce00cad00 0x7f5ce015a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdfe00640 1 -- 172.21.15.66:0/2849176940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc00b780 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdfe00640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce00cc8d0 0x7f5ce00c0e30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc01c5d0 tx=0x7f5ccc01b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:53.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdcc00640 1 -- 172.21.15.66:0/2849176940 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0a7b80 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdcc00640 1 -- 172.21.15.66:0/2849176940 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc0a7ce0 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdcc00640 1 -- 172.21.15.66:0/2849176940 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc003860 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5ce7a00640 1 -- 172.21.15.66:0/2849176940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce015b2f0 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5ce7a00640 1 -- 172.21.15.66:0/2849176940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce00c1370 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5ce7a00640 1 -- 172.21.15.66:0/2849176940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce00c1560 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.485+0000 7f5cdcc00640 1 -- 172.21.15.66:0/2849176940 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5ccc0039c0 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.489+0000 7f5cdcc00640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5cc0066690 0x7f5cc0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.489+0000 7f5cdcc00640 1 -- 172.21.15.66:0/2849176940 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f5ccc11ea60 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.489+0000 7f5ce5000640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5cc0066690 0x7f5cc0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.489+0000 7f5ce5000640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5cc0066690 0x7f5cc0068b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4007fd0 tx=0x7f5cd4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:53.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.493+0000 7f5cdcc00640 1 -- 172.21.15.66:0/2849176940 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5ccc0eb9e0 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 -- 172.21.15.66:0/2849176940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ce0066c30 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5cdcc00640 1 -- 172.21.15.66:0/2849176940 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ccc0ef000 con 0x7f5ce00cc8d0 2024-03-30T00:40:53.662 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:53.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 -- 172.21.15.66:0/2849176940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5cc0066690 msgr2=0x7f5cc0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5cc0066690 0x7f5cc0068b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4007fd0 tx=0x7f5cd4009040 comp rx=0 tx=0).stop 2024-03-30T00:40:53.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 -- 172.21.15.66:0/2849176940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce00cc8d0 msgr2=0x7f5ce00c0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce00cc8d0 0x7f5ce00c0e30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc01c5d0 tx=0x7f5ccc01b740 comp rx=0 tx=0).stop 2024-03-30T00:40:53.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 -- 172.21.15.66:0/2849176940 shutdown_connections 2024-03-30T00:40:53.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5ce00cd2d0 0x7f5ce0157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5cc0066690 0x7f5cc0068b50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce00cc8d0 0x7f5ce00c0e30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 --2- 172.21.15.66:0/2849176940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ce00cad00 0x7f5ce015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 -- 172.21.15.66:0/2849176940 >> 172.21.15.66:0/2849176940 conn(0x7f5ce0069f60 msgr2=0x7f5ce006d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:53.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 -- 172.21.15.66:0/2849176940 shutdown_connections 2024-03-30T00:40:53.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.653+0000 7f5ce7a00640 1 -- 172.21.15.66:0/2849176940 wait complete. 2024-03-30T00:40:53.684 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.817+0000 7f77b4200640 1 Processor -- start 2024-03-30T00:40:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.817+0000 7f77b4200640 1 -- start start 2024-03-30T00:40:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.817+0000 7f77b4200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 0x7f77ac0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.817+0000 7f77b4200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f77ac0cf290 0x7f77ac0c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.817+0000 7f77b4200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77ac0c4600 con 0x7f77ac099430 2024-03-30T00:40:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.817+0000 7f77b4200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77ac0cc510 con 0x7f77ac0cf290 2024-03-30T00:40:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.817+0000 7f77b4200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac0998c0 con 0x7f77ac15a810 2024-03-30T00:40:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77b1800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 0x7f77ac0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77b2200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f77ac0cf290 0x7f77ac0c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77b2200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f77ac0cf290 0x7f77ac0c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:42608/0 (socket says 172.21.15.66:42608) 2024-03-30T00:40:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77b2200640 1 -- 172.21.15.66:0/38800976 learned_addr learned my addr 172.21.15.66:0/38800976 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77b1800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 0x7f77ac0ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:46250/0 (socket says 172.21.15.66:46250) 2024-03-30T00:40:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77b1800640 1 -- 172.21.15.66:0/38800976 >> v2:172.21.15.156:3300/0 conn(0x7f77ac0cf290 msgr2=0x7f77ac0c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77b1800640 1 --2- 172.21.15.66:0/38800976 >> v2:172.21.15.156:3300/0 conn(0x7f77ac0cf290 0x7f77ac0c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77b1800640 1 -- 172.21.15.66:0/38800976 >> v1:172.21.15.156:6789/0 conn(0x7f77ac099430 legacy=0x7f77ac0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77b1800640 1 -- 172.21.15.66:0/38800976 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac099ba0 con 0x7f77ac15a810 2024-03-30T00:40:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77b2200640 1 --2- 172.21.15.66:0/38800976 >> v2:172.21.15.156:3300/0 conn(0x7f77ac0cf290 0x7f77ac0c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:53.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77b1800640 1 --2- 172.21.15.66:0/38800976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 0x7f77ac0ce380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f77a00097c0 tx=0x7f77a001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf8b40f0e0b7f92 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:53.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.821+0000 7f77abe00640 1 -- 172.21.15.66:0/38800976 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a00a8990 con 0x7f77ac15a810 2024-03-30T00:40:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77abe00640 1 -- 172.21.15.66:0/38800976 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a00a8af0 con 0x7f77ac15a810 2024-03-30T00:40:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77abe00640 1 -- 172.21.15.66:0/38800976 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a00a8e80 con 0x7f77ac15a810 2024-03-30T00:40:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 -- 172.21.15.66:0/38800976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 msgr2=0x7f77ac0ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 --2- 172.21.15.66:0/38800976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 0x7f77ac0ce380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f77a00097c0 tx=0x7f77a001ec90 comp rx=0 tx=0).stop 2024-03-30T00:40:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 -- 172.21.15.66:0/38800976 shutdown_connections 2024-03-30T00:40:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 --2- 172.21.15.66:0/38800976 >> v2:172.21.15.156:3300/0 conn(0x7f77ac0cf290 0x7f77ac0c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 --2- 172.21.15.66:0/38800976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 0x7f77ac0ce380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 -- 172.21.15.66:0/38800976 >> 172.21.15.66:0/38800976 conn(0x7f77ac0b2090 msgr2=0x7f77ac0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 -- 172.21.15.66:0/38800976 shutdown_connections 2024-03-30T00:40:53.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 -- 172.21.15.66:0/38800976 wait complete. 2024-03-30T00:40:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 Processor -- start 2024-03-30T00:40:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 -- start start 2024-03-30T00:40:53.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77ac0cf290 0x7f77ac06aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 0x7f77ac158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77ac15bfa0 0x7f77ac0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac15d210 con 0x7f77ac15bfa0 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac15d4f0 con 0x7f77ac0cf290 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b4200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac15d7d0 con 0x7f77ac15a810 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b2200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77ac15bfa0 0x7f77ac0c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b2200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77ac15bfa0 0x7f77ac0c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36512/0 (socket says 172.21.15.66:36512) 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b2200640 1 -- 172.21.15.66:0/2467657490 learned_addr learned my addr 172.21.15.66:0/2467657490 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b1800640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77ac0cf290 0x7f77ac06aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.825+0000 7f77b0e00640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 0x7f77ac158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77b2200640 1 -- 172.21.15.66:0/2467657490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 msgr2=0x7f77ac158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77b2200640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 0x7f77ac158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77b2200640 1 -- 172.21.15.66:0/2467657490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77ac0cf290 msgr2=0x7f77ac06aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:53.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77b2200640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77ac0cf290 0x7f77ac06aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77b2200640 1 -- 172.21.15.66:0/2467657490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a4009b30 con 0x7f77ac15bfa0 2024-03-30T00:40:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77b2200640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77ac15bfa0 0x7f77ac0c9a10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ef90 tx=0x7f77a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77a9600640 1 -- 172.21.15.66:0/2467657490 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4018070 con 0x7f77ac15bfa0 2024-03-30T00:40:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77a9600640 1 -- 172.21.15.66:0/2467657490 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a400f040 con 0x7f77ac15bfa0 2024-03-30T00:40:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77b4200640 1 -- 172.21.15.66:0/2467657490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a0009420 con 0x7f77ac15bfa0 2024-03-30T00:40:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77a9600640 1 -- 172.21.15.66:0/2467657490 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a400ce00 con 0x7f77ac15bfa0 2024-03-30T00:40:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77b4200640 1 -- 172.21.15.66:0/2467657490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ac158fd0 con 0x7f77ac15bfa0 2024-03-30T00:40:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77b4200640 1 -- 172.21.15.66:0/2467657490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ac06b780 con 0x7f77ac15bfa0 2024-03-30T00:40:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77a9600640 1 -- 172.21.15.66:0/2467657490 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f77a4014480 con 0x7f77ac15bfa0 2024-03-30T00:40:53.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77a9600640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f778c066690 0x7f778c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:53.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77a9600640 1 -- 172.21.15.66:0/2467657490 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f77a4084750 con 0x7f77ac15bfa0 2024-03-30T00:40:53.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.829+0000 7f77b1800640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f778c066690 0x7f778c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:53.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.833+0000 7f77b1800640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f778c066690 0x7f778c068b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77a00b2000 tx=0x7f77a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:53.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.833+0000 7f77a9600640 1 -- 172.21.15.66:0/2467657490 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f77ac06b780 con 0x7f77ac15bfa0 2024-03-30T00:40:54.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.993+0000 7f77b4200640 1 -- 172.21.15.66:0/2467657490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77ac06b780 con 0x7f77ac15bfa0 2024-03-30T00:40:54.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.993+0000 7f77a9600640 1 -- 172.21.15.66:0/2467657490 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77ac06b780 con 0x7f77ac15bfa0 2024-03-30T00:40:54.003 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:54.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 -- 172.21.15.66:0/2467657490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f778c066690 msgr2=0x7f778c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f778c066690 0x7f778c068b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77a00b2000 tx=0x7f77a0008040 comp rx=0 tx=0).stop 2024-03-30T00:40:54.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 -- 172.21.15.66:0/2467657490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77ac15bfa0 msgr2=0x7f77ac0c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77ac15bfa0 0x7f77ac0c9a10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ef90 tx=0x7f77a400c490 comp rx=0 tx=0).stop 2024-03-30T00:40:54.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 -- 172.21.15.66:0/2467657490 shutdown_connections 2024-03-30T00:40:54.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f778c066690 0x7f778c068b50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77ac15bfa0 0x7f77ac0c9a10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f77ac15a810 0x7f77ac158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 --2- 172.21.15.66:0/2467657490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77ac0cf290 0x7f77ac06aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 -- 172.21.15.66:0/2467657490 >> 172.21.15.66:0/2467657490 conn(0x7f77ac0b2090 msgr2=0x7f77ac098a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:54.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 -- 172.21.15.66:0/2467657490 shutdown_connections 2024-03-30T00:40:54.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:53.997+0000 7f77b4200640 1 -- 172.21.15.66:0/2467657490 wait complete. 2024-03-30T00:40:54.024 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.157+0000 7fe2f3e00640 1 Processor -- start 2024-03-30T00:40:54.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.157+0000 7fe2f3e00640 1 -- start start 2024-03-30T00:40:54.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2f3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe2f406c5c0 0x7fe2f406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2f3e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2f4158e30 0x7fe2f415d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2f3e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f415d780 con 0x7fe2f4158a50 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2f3e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f40c8830 con 0x7fe2f406c5c0 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2f3e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40c8b10 con 0x7fe2f4158e30 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2ebe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe2f406c5c0 0x7fe2f406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2ebe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe2f406c5c0 0x7fe2f406d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36516/0 (socket says 172.21.15.66:36516) 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2ebe00640 1 -- 172.21.15.66:0/3502375916 learned_addr learned my addr 172.21.15.66:0/3502375916 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2f3400640 1 --2- 172.21.15.66:0/3502375916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2f4158e30 0x7fe2f415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2ebe00640 1 -- 172.21.15.66:0/3502375916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2f4158e30 msgr2=0x7fe2f415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2ebe00640 1 --2- 172.21.15.66:0/3502375916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2f4158e30 0x7fe2f415d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2ebe00640 1 -- 172.21.15.66:0/3502375916 >> v1:172.21.15.156:6789/0 conn(0x7fe2f4158a50 legacy=0x7fe2f406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2ebe00640 1 -- 172.21.15.66:0/3502375916 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f40cd8b0 con 0x7fe2f406c5c0 2024-03-30T00:40:54.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.161+0000 7fe2ebe00640 1 --2- 172.21.15.66:0/3502375916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2f406c5c0 0x7fe2f406d9e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00aa10 tx=0x7fe2dc01d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f55b22468f09b08 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:54.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f2000640 1 -- 172.21.15.66:0/3502375916 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0a68e0 con 0x7fe2f406c5c0 2024-03-30T00:40:54.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f2000640 1 -- 172.21.15.66:0/3502375916 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc0a6a40 con 0x7fe2f406c5c0 2024-03-30T00:40:54.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f2000640 1 -- 172.21.15.66:0/3502375916 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0a6dd0 con 0x7fe2f406c5c0 2024-03-30T00:40:54.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 -- 172.21.15.66:0/3502375916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2f406c5c0 msgr2=0x7fe2f406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 --2- 172.21.15.66:0/3502375916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2f406c5c0 0x7fe2f406d9e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00aa10 tx=0x7fe2dc01d870 comp rx=0 tx=0).stop 2024-03-30T00:40:54.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 -- 172.21.15.66:0/3502375916 shutdown_connections 2024-03-30T00:40:54.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 --2- 172.21.15.66:0/3502375916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2f4158e30 0x7fe2f415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 --2- 172.21.15.66:0/3502375916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2f406c5c0 0x7fe2f406d9e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 -- 172.21.15.66:0/3502375916 >> 172.21.15.66:0/3502375916 conn(0x7fe2f406a190 msgr2=0x7fe2f406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:54.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 -- 172.21.15.66:0/3502375916 shutdown_connections 2024-03-30T00:40:54.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 -- 172.21.15.66:0/3502375916 wait complete. 2024-03-30T00:40:54.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 Processor -- start 2024-03-30T00:40:54.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 -- start start 2024-03-30T00:40:54.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2f406c5c0 0x7fe2f41f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2f4158a50 0x7fe2f40c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2f4158e30 0x7fe2f40c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f409c230 con 0x7fe2f4158a50 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40b0b50 con 0x7fe2f4158e30 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f409bcb0 con 0x7fe2f406c5c0 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2ebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2f4158a50 0x7fe2f40c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2ebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2f4158a50 0x7fe2f40c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36530/0 (socket says 172.21.15.66:36530) 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2ebe00640 1 -- 172.21.15.66:0/2508405588 learned_addr learned my addr 172.21.15.66:0/2508405588 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f2a00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2f406c5c0 0x7fe2f41f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.165+0000 7fe2f3400640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2f4158e30 0x7fe2f40c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2ebe00640 1 -- 172.21.15.66:0/2508405588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2f406c5c0 msgr2=0x7fe2f41f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2ebe00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2f406c5c0 0x7fe2f41f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2ebe00640 1 -- 172.21.15.66:0/2508405588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2f4158e30 msgr2=0x7fe2f40c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2ebe00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2f4158e30 0x7fe2f40c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2ebe00640 1 -- 172.21.15.66:0/2508405588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc00a670 con 0x7fe2f4158a50 2024-03-30T00:40:54.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2ebe00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2f4158a50 0x7fe2f40c6df0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00a1c0 tx=0x7fe2dc01c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:54.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2eaa00640 1 -- 172.21.15.66:0/2508405588 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0aea10 con 0x7fe2f4158a50 2024-03-30T00:40:54.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2eaa00640 1 -- 172.21.15.66:0/2508405588 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc0aeb70 con 0x7fe2f4158a50 2024-03-30T00:40:54.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2f3e00640 1 -- 172.21.15.66:0/2508405588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f409bf50 con 0x7fe2f4158a50 2024-03-30T00:40:54.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2eaa00640 1 -- 172.21.15.66:0/2508405588 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0b9450 con 0x7fe2f4158a50 2024-03-30T00:40:54.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2f3e00640 1 -- 172.21.15.66:0/2508405588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f41f53c0 con 0x7fe2f4158a50 2024-03-30T00:40:54.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2f3e00640 1 -- 172.21.15.66:0/2508405588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f40af4f0 con 0x7fe2f4158a50 2024-03-30T00:40:54.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2eaa00640 1 -- 172.21.15.66:0/2508405588 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe2dc0b5070 con 0x7fe2f4158a50 2024-03-30T00:40:54.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2eaa00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2d0066690 0x7fe2d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2eaa00640 1 -- 172.21.15.66:0/2508405588 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fe2dc11f080 con 0x7fe2f4158a50 2024-03-30T00:40:54.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.169+0000 7fe2f2a00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2d0066690 0x7fe2d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.173+0000 7fe2f2a00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2d0066690 0x7fe2d0068b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4001810 tx=0x7fe2e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:54.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.173+0000 7fe2eaa00640 1 -- 172.21.15.66:0/2508405588 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe2dc150980 con 0x7fe2f4158a50 2024-03-30T00:40:54.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.333+0000 7fe2f3e00640 1 -- 172.21.15.66:0/2508405588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe2f4073e00 con 0x7fe2f4158a50 2024-03-30T00:40:54.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.333+0000 7fe2eaa00640 1 -- 172.21.15.66:0/2508405588 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe2dc0ec1b0 con 0x7fe2f4158a50 2024-03-30T00:40:54.344 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 -- 172.21.15.66:0/2508405588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2d0066690 msgr2=0x7fe2d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2d0066690 0x7fe2d0068b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4001810 tx=0x7fe2e4009040 comp rx=0 tx=0).stop 2024-03-30T00:40:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 -- 172.21.15.66:0/2508405588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2f4158a50 msgr2=0x7fe2f40c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2f4158a50 0x7fe2f40c6df0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00a1c0 tx=0x7fe2dc01c0b0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 -- 172.21.15.66:0/2508405588 shutdown_connections 2024-03-30T00:40:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2f4158e30 0x7fe2f40c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2d0066690 0x7fe2d0068b50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2f4158a50 0x7fe2f40c6df0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 --2- 172.21.15.66:0/2508405588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2f406c5c0 0x7fe2f41f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 -- 172.21.15.66:0/2508405588 >> 172.21.15.66:0/2508405588 conn(0x7fe2f406a190 msgr2=0x7fe2f4151120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 -- 172.21.15.66:0/2508405588 shutdown_connections 2024-03-30T00:40:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.337+0000 7fe2f3e00640 1 -- 172.21.15.66:0/2508405588 wait complete. 2024-03-30T00:40:54.366 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:40:54.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.497+0000 7f74b3200640 1 Processor -- start 2024-03-30T00:40:54.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.497+0000 7f74b3200640 1 -- start start 2024-03-30T00:40:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.497+0000 7f74b3200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74b40b3170 0x7f74b40b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.497+0000 7f74b3200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b40cb5b0 con 0x7f74b415a720 2024-03-30T00:40:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.497+0000 7f74b3200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b40cb080 con 0x7f74b40d07c0 2024-03-30T00:40:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.497+0000 7f74b3200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b40cfb00 con 0x7f74b40b3170 2024-03-30T00:40:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.497+0000 7f74a9400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74b40b3170 0x7f74b40b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.497+0000 7f74a9400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74b40b3170 0x7f74b40b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36534/0 (socket says 172.21.15.66:36534) 2024-03-30T00:40:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.497+0000 7f74a9400640 1 -- 172.21.15.66:0/2326009325 learned_addr learned my addr 172.21.15.66:0/2326009325 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74a9400640 1 -- 172.21.15.66:0/2326009325 >> v1:172.21.15.156:6789/0 conn(0x7f74b40d07c0 legacy=0x7f74b40ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74a9400640 1 -- 172.21.15.66:0/2326009325 >> v1:172.21.15.66:6789/0 conn(0x7f74b415a720 legacy=0x7f74b40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74a9400640 1 -- 172.21.15.66:0/2326009325 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b415ab00 con 0x7f74b40b3170 2024-03-30T00:40:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74b1400640 1 -- 172.21.15.66:0/2326009325 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007878926 0 0) 0x7f74a40014a0 con 0x7f74b415a720 2024-03-30T00:40:54.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74a9400640 1 --2- 172.21.15.66:0/2326009325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74b40b3170 0x7f74b40b3570 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f749c009870 tx=0x7f749c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea426f50b62e2132 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:54.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74b1400640 1 -- 172.21.15.66:0/2326009325 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c0a9620 con 0x7f74b40b3170 2024-03-30T00:40:54.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74b1400640 1 -- 172.21.15.66:0/2326009325 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c0a9780 con 0x7f74b40b3170 2024-03-30T00:40:54.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74b1400640 1 -- 172.21.15.66:0/2326009325 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c0a9b10 con 0x7f74b40b3170 2024-03-30T00:40:54.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74b3200640 1 -- 172.21.15.66:0/2326009325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74b40b3170 msgr2=0x7f74b40b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74b3200640 1 --2- 172.21.15.66:0/2326009325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74b40b3170 0x7f74b40b3570 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f749c009870 tx=0x7f749c01c560 comp rx=0 tx=0).stop 2024-03-30T00:40:54.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74b3200640 1 -- 172.21.15.66:0/2326009325 shutdown_connections 2024-03-30T00:40:54.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74b3200640 1 --2- 172.21.15.66:0/2326009325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74b40b3170 0x7f74b40b3570 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74b3200640 1 -- 172.21.15.66:0/2326009325 >> 172.21.15.66:0/2326009325 conn(0x7f74b4069d90 msgr2=0x7f74b406a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:54.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74b3200640 1 -- 172.21.15.66:0/2326009325 shutdown_connections 2024-03-30T00:40:54.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.501+0000 7f74b3200640 1 -- 172.21.15.66:0/2326009325 wait complete. 2024-03-30T00:40:54.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b3200640 1 Processor -- start 2024-03-30T00:40:54.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b3200640 1 -- start start 2024-03-30T00:40:54.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74b40b3170 0x7f74b415bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b3200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74b40d07c0 0x7f74b4158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b3200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74b415a720 0x7f74b40c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b3200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b4159840 con 0x7f74b40b3170 2024-03-30T00:40:54.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b3200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b415cc60 con 0x7f74b40d07c0 2024-03-30T00:40:54.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b3200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b41592c0 con 0x7f74b415a720 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74b40b3170 0x7f74b415bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74b40b3170 0x7f74b415bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36538/0 (socket says 172.21.15.66:36538) 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b1e00640 1 -- 172.21.15.66:0/238793585 learned_addr learned my addr 172.21.15.66:0/238793585 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b2800640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74b415a720 0x7f74b40c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b1e00640 1 -- 172.21.15.66:0/238793585 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74b415a720 msgr2=0x7f74b40c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74a9400640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74b40d07c0 0x7f74b4158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b1e00640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74b415a720 0x7f74b40c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b1e00640 1 -- 172.21.15.66:0/238793585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74b40d07c0 msgr2=0x7f74b4158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b1e00640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74b40d07c0 0x7f74b4158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b1e00640 1 -- 172.21.15.66:0/238793585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c0094d0 con 0x7f74b40b3170 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b2800640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74b415a720 0x7f74b40c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74a9400640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74b40d07c0 0x7f74b4158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:54.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b1e00640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74b40b3170 0x7f74b415bce0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f74a400c790 tx=0x7f74a400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:54.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74aaa00640 1 -- 172.21.15.66:0/238793585 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a400cea0 con 0x7f74b40b3170 2024-03-30T00:40:54.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74aaa00640 1 -- 172.21.15.66:0/238793585 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a4003db0 con 0x7f74b40b3170 2024-03-30T00:40:54.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74aaa00640 1 -- 172.21.15.66:0/238793585 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40052b0 con 0x7f74b40b3170 2024-03-30T00:40:54.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b3200640 1 -- 172.21.15.66:0/238793585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b415c3c0 con 0x7f74b40b3170 2024-03-30T00:40:54.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.505+0000 7f74b3200640 1 -- 172.21.15.66:0/238793585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b415c980 con 0x7f74b40b3170 2024-03-30T00:40:54.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.509+0000 7f74b3200640 1 -- 172.21.15.66:0/238793585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b4073e00 con 0x7f74b40b3170 2024-03-30T00:40:54.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.509+0000 7f74aaa00640 1 -- 172.21.15.66:0/238793585 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f74a4005410 con 0x7f74b40b3170 2024-03-30T00:40:54.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.509+0000 7f74aaa00640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7490066690 0x7f7490068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.509+0000 7f74aaa00640 1 -- 172.21.15.66:0/238793585 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f74a4084260 con 0x7f74b40b3170 2024-03-30T00:40:54.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.509+0000 7f74a9400640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7490066690 0x7f7490068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.509+0000 7f74a9400640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7490066690 0x7f7490068b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f749c0ae000 tx=0x7f749c0b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:54.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.513+0000 7f74aaa00640 1 -- 172.21.15.66:0/238793585 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f74a40511c0 con 0x7f74b40b3170 2024-03-30T00:40:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.685+0000 7f74b3200640 1 -- 172.21.15.66:0/238793585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f74b40cb5b0 con 0x7f74b40b3170 2024-03-30T00:40:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.685+0000 7f74aaa00640 1 -- 172.21.15.66:0/238793585 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f74b40cb5b0 con 0x7f74b40b3170 2024-03-30T00:40:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:40:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 -- 172.21.15.66:0/238793585 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7490066690 msgr2=0x7f7490068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7490066690 0x7f7490068b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f749c0ae000 tx=0x7f749c0b3040 comp rx=0 tx=0).stop 2024-03-30T00:40:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 -- 172.21.15.66:0/238793585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74b40b3170 msgr2=0x7f74b415bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74b40b3170 0x7f74b415bce0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f74a400c790 tx=0x7f74a400cc50 comp rx=0 tx=0).stop 2024-03-30T00:40:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 -- 172.21.15.66:0/238793585 shutdown_connections 2024-03-30T00:40:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74b415a720 0x7f74b40c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74b40d07c0 0x7f74b4158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7490066690 0x7f7490068b50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 --2- 172.21.15.66:0/238793585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74b40b3170 0x7f74b415bce0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 -- 172.21.15.66:0/238793585 >> 172.21.15.66:0/238793585 conn(0x7f74b4069d90 msgr2=0x7f74b4165e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 -- 172.21.15.66:0/238793585 shutdown_connections 2024-03-30T00:40:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.689+0000 7f74b3200640 1 -- 172.21.15.66:0/238793585 wait complete. 2024-03-30T00:40:54.717 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:40:54.717 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:40:54.727 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:54.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f1326800640 1 Processor -- start 2024-03-30T00:40:54.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f1326800640 1 -- start start 2024-03-30T00:40:54.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f1326800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f13200cda90 0x7f13200ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f1326800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13200cd3e0 con 0x7f132015a370 2024-03-30T00:40:54.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f1326800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132015a990 con 0x7f13200cde70 2024-03-30T00:40:54.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f1326800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13200b1b80 con 0x7f13200cda90 2024-03-30T00:40:54.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131fe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f132015a370 0x7f1320156910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33376/0 (socket says 172.21.15.66:33376) 2024-03-30T00:40:54.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131fe00640 1 -- 172.21.15.66:0/2688147028 learned_addr learned my addr 172.21.15.66:0/2688147028 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:54.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131f400640 1 --2- 172.21.15.66:0/2688147028 >> v2:172.21.15.156:3300/0 conn(0x7f13200cda90 0x7f13200ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824344218 0 0) 0x7f13200cd3e0 con 0x7f132015a370 2024-03-30T00:40:54.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1304005740 con 0x7f132015a370 2024-03-30T00:40:54.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3872087628 0 0) 0x7f1304005740 con 0x7f132015a370 2024-03-30T00:40:54.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 >> v2:172.21.15.156:3300/0 conn(0x7f13200cda90 msgr2=0x7f13200ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:54.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 --2- 172.21.15.66:0/2688147028 >> v2:172.21.15.156:3300/0 conn(0x7f13200cda90 0x7f13200ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 >> v1:172.21.15.156:6789/0 conn(0x7f13200cde70 legacy=0x7f1320159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13200b11e0 con 0x7f132015a370 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f1318001a00 con 0x7f132015a370 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 >> v1:172.21.15.66:6789/0 conn(0x7f132015a370 legacy=0x7f1320156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 --2- 172.21.15.66:0/2688147028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1304005be0 0x7f1304006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 --2- 172.21.15.66:0/2688147028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 0x7f1304006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 --2- 172.21.15.66:0/2688147028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13040073f0 0x7f130400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13200b1b80 con 0x7f13040069e0 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130400e340 con 0x7f1304005be0 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130400e500 con 0x7f13040073f0 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131ea00640 1 --2- 172.21.15.66:0/2688147028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 0x7f1304006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131f400640 1 --2- 172.21.15.66:0/2688147028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1304005be0 0x7f1304006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131fe00640 1 --2- 172.21.15.66:0/2688147028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13040073f0 0x7f130400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131ea00640 1 -- 172.21.15.66:0/2688147028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13040073f0 msgr2=0x7f130400dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:54.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131ea00640 1 --2- 172.21.15.66:0/2688147028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13040073f0 0x7f130400dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131ea00640 1 -- 172.21.15.66:0/2688147028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1304005be0 msgr2=0x7f1304006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:54.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131ea00640 1 --2- 172.21.15.66:0/2688147028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1304005be0 0x7f1304006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131ea00640 1 -- 172.21.15.66:0/2688147028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1304005740 con 0x7f13040069e0 2024-03-30T00:40:54.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.861+0000 7f131fe00640 1 -- 172.21.15.66:0/2688147028 reap_dead start 2024-03-30T00:40:54.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.865+0000 7f131ea00640 1 --2- 172.21.15.66:0/2688147028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 0x7f1304006e60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f13200ce2c0 tx=0x7f130c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca020c3770db4e3e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:54.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.865+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c0052d0 con 0x7f13040069e0 2024-03-30T00:40:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.865+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130c0a7050 con 0x7f13040069e0 2024-03-30T00:40:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.865+0000 7f131e000640 1 -- 172.21.15.66:0/2688147028 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c003840 con 0x7f13040069e0 2024-03-30T00:40:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.865+0000 7f1326800640 1 -- 172.21.15.66:0/2688147028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 msgr2=0x7f1304006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.865+0000 7f1326800640 1 --2- 172.21.15.66:0/2688147028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 0x7f1304006e60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f13200ce2c0 tx=0x7f130c01c3a0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.865+0000 7f1326800640 1 -- 172.21.15.66:0/2688147028 shutdown_connections 2024-03-30T00:40:54.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.865+0000 7f1326800640 1 --2- 172.21.15.66:0/2688147028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 0x7f1304006e60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.865+0000 7f1326800640 1 -- 172.21.15.66:0/2688147028 >> 172.21.15.66:0/2688147028 conn(0x7f1320069f60 msgr2=0x7f132006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:54.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.865+0000 7f1326800640 1 -- 172.21.15.66:0/2688147028 shutdown_connections 2024-03-30T00:40:54.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.865+0000 7f1326800640 1 -- 172.21.15.66:0/2688147028 wait complete. 2024-03-30T00:40:54.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1326800640 1 Processor -- start 2024-03-30T00:40:54.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1326800640 1 -- start start 2024-03-30T00:40:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1326800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 0x7f1320070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1326800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1320155e10 0x7f13200ac780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1326800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1320155620 0x7f1320157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1326800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132006fed0 con 0x7f13040069e0 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1326800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13200acfb0 con 0x7f1320155e10 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1326800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13200ad290 con 0x7f1320155620 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f131f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 0x7f1320070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f131f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 0x7f1320070920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36570/0 (socket says 172.21.15.66:36570) 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f131f400640 1 -- 172.21.15.66:0/1858606669 learned_addr learned my addr 172.21.15.66:0/1858606669 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f131ea00640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1320155e10 0x7f13200ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f131fe00640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1320155620 0x7f1320157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f131f400640 1 -- 172.21.15.66:0/1858606669 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1320155620 msgr2=0x7f1320157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f131f400640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1320155620 0x7f1320157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f131f400640 1 -- 172.21.15.66:0/1858606669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1320155e10 msgr2=0x7f13200ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f131f400640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1320155e10 0x7f13200ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f131f400640 1 -- 172.21.15.66:0/1858606669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320070120 con 0x7f13040069e0 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f131f400640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 0x7f1320070920 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1310011500 tx=0x7f13100119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:54.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1317400640 1 -- 172.21.15.66:0/1858606669 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310015870 con 0x7f13040069e0 2024-03-30T00:40:54.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1317400640 1 -- 172.21.15.66:0/1858606669 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13100159d0 con 0x7f13040069e0 2024-03-30T00:40:54.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1317400640 1 -- 172.21.15.66:0/1858606669 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310015d40 con 0x7f13040069e0 2024-03-30T00:40:54.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1326800640 1 -- 172.21.15.66:0/1858606669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1320070400 con 0x7f13040069e0 2024-03-30T00:40:54.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.869+0000 7f1326800640 1 -- 172.21.15.66:0/1858606669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1320155aa0 con 0x7f13040069e0 2024-03-30T00:40:54.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.873+0000 7f1326800640 1 -- 172.21.15.66:0/1858606669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1320070590 con 0x7f13040069e0 2024-03-30T00:40:54.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.873+0000 7f1317400640 1 -- 172.21.15.66:0/1858606669 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1310013400 con 0x7f13040069e0 2024-03-30T00:40:54.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.873+0000 7f1317400640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f13000666c0 0x7f1300068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:54.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.873+0000 7f1317400640 1 -- 172.21.15.66:0/1858606669 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f1310084900 con 0x7f13040069e0 2024-03-30T00:40:54.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.873+0000 7f131ea00640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f13000666c0 0x7f1300068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:54.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.873+0000 7f131ea00640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f13000666c0 0x7f1300068b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1304006960 tx=0x7f130c0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:54.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:54.877+0000 7f1317400640 1 -- 172.21.15.66:0/1858606669 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1310051890 con 0x7f13040069e0 2024-03-30T00:40:55.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.041+0000 7f1326800640 1 -- 172.21.15.66:0/1858606669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1320066c30 con 0x7f13040069e0 2024-03-30T00:40:55.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.041+0000 7f1317400640 1 -- 172.21.15.66:0/1858606669 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7f1310054eb0 con 0x7f13040069e0 2024-03-30T00:40:55.051 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:55.051 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:55.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 -- 172.21.15.66:0/1858606669 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f13000666c0 msgr2=0x7f1300068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f13000666c0 0x7f1300068b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1304006960 tx=0x7f130c0011a0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 -- 172.21.15.66:0/1858606669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 msgr2=0x7f1320070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 0x7f1320070920 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1310011500 tx=0x7f13100119c0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 -- 172.21.15.66:0/1858606669 shutdown_connections 2024-03-30T00:40:55.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1320155620 0x7f1320157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1320155e10 0x7f13200ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f13000666c0 0x7f1300068b80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 --2- 172.21.15.66:0/1858606669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13040069e0 0x7f1320070920 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 -- 172.21.15.66:0/1858606669 >> 172.21.15.66:0/1858606669 conn(0x7f1320069f60 msgr2=0x7f132006d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:55.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 -- 172.21.15.66:0/1858606669 shutdown_connections 2024-03-30T00:40:55.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.045+0000 7f1326800640 1 -- 172.21.15.66:0/1858606669 wait complete. 2024-03-30T00:40:55.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:55.073 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:55.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9fe00640 1 Processor -- start 2024-03-30T00:40:55.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9fe00640 1 -- start start 2024-03-30T00:40:55.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9fe00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fda980cec30 0x7fda98159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda980cde70 0x7fda9815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9fe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda9815a6e0 con 0x7fda980cda90 2024-03-30T00:40:55.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9fe00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda980b1b80 con 0x7fda980cec30 2024-03-30T00:40:55.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9fe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda980cd3e0 con 0x7fda980cde70 2024-03-30T00:40:55.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9d400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fda980cda90 0x7fda980ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33390/0 (socket says 172.21.15.66:33390) 2024-03-30T00:40:55.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9d400640 1 -- 172.21.15.66:0/2992635701 learned_addr learned my addr 172.21.15.66:0/2992635701 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:55.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9ca00640 1 --2- 172.21.15.66:0/2992635701 >> v2:172.21.15.156:3300/0 conn(0x7fda980cec30 0x7fda98159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9de00640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda980cde70 0x7fda9815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496881513 0 0) 0x7fda9815a6e0 con 0x7fda980cda90 2024-03-30T00:40:55.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda80003530 con 0x7fda980cda90 2024-03-30T00:40:55.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 527037125 0 0) 0x7fda80003530 con 0x7fda980cda90 2024-03-30T00:40:55.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda980cde70 msgr2=0x7fda9815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:55.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda980cde70 0x7fda9815a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 >> v2:172.21.15.156:3300/0 conn(0x7fda980cec30 msgr2=0x7fda98159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:55.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 --2- 172.21.15.66:0/2992635701 >> v2:172.21.15.156:3300/0 conn(0x7fda980cec30 0x7fda98159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9815a990 con 0x7fda980cda90 2024-03-30T00:40:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fda8c001680 con 0x7fda980cda90 2024-03-30T00:40:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9ca00640 1 --2- 172.21.15.66:0/2992635701 >> v2:172.21.15.156:3300/0 conn(0x7fda980cec30 0x7fda98159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 >> v1:172.21.15.66:6789/0 conn(0x7fda980cda90 legacy=0x7fda980ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda980cde70 0x7fda80003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 0x7fda800047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda80004d10 0x7fda80005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda980b1b80 con 0x7fda80003f00 2024-03-30T00:40:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda980cd3e0 con 0x7fda80004d10 2024-03-30T00:40:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8000b9e0 con 0x7fda980cde70 2024-03-30T00:40:55.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9ca00640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 0x7fda800047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9d400640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda980cde70 0x7fda80003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9de00640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda80004d10 0x7fda80005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9ca00640 1 -- 172.21.15.66:0/2992635701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda980cde70 msgr2=0x7fda80003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9ca00640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda980cde70 0x7fda80003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9ca00640 1 -- 172.21.15.66:0/2992635701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda80004d10 msgr2=0x7fda80005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9ca00640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda80004d10 0x7fda80005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.209+0000 7fda9ca00640 1 -- 172.21.15.66:0/2992635701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda80003530 con 0x7fda80003f00 2024-03-30T00:40:55.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.213+0000 7fda9ca00640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 0x7fda800047d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fda84009cb0 tx=0x7fda8401cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6e7277b083a147 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:55.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.213+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84004020 con 0x7fda80003f00 2024-03-30T00:40:55.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.213+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda84004180 con 0x7fda80003f00 2024-03-30T00:40:55.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.213+0000 7fda97e00640 1 -- 172.21.15.66:0/2992635701 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda840a5800 con 0x7fda80003f00 2024-03-30T00:40:55.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.213+0000 7fda9fe00640 1 -- 172.21.15.66:0/2992635701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 msgr2=0x7fda800047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.213+0000 7fda9fe00640 1 --2- 172.21.15.66:0/2992635701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 0x7fda800047d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fda84009cb0 tx=0x7fda8401cc50 comp rx=0 tx=0).stop 2024-03-30T00:40:55.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.213+0000 7fda9de00640 1 -- 172.21.15.66:0/2992635701 reap_dead start 2024-03-30T00:40:55.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.213+0000 7fda9fe00640 1 -- 172.21.15.66:0/2992635701 shutdown_connections 2024-03-30T00:40:55.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.213+0000 7fda9fe00640 1 -- 172.21.15.66:0/2992635701 >> 172.21.15.66:0/2992635701 conn(0x7fda98069f60 msgr2=0x7fda9806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:55.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.213+0000 7fda9fe00640 1 -- 172.21.15.66:0/2992635701 shutdown_connections 2024-03-30T00:40:55.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.213+0000 7fda9fe00640 1 -- 172.21.15.66:0/2992635701 wait complete. 2024-03-30T00:40:55.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9fe00640 1 Processor -- start 2024-03-30T00:40:55.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9fe00640 1 -- start start 2024-03-30T00:40:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 0x7fda980bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9806d240 0x7fda980cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda98157630 0x7fda98157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda980bf4d0 con 0x7fda80003f00 2024-03-30T00:40:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9fe00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9806d6c0 con 0x7fda9806d240 2024-03-30T00:40:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9fe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9815ac70 con 0x7fda98157630 2024-03-30T00:40:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 0x7fda980bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 0x7fda980bf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36590/0 (socket says 172.21.15.66:36590) 2024-03-30T00:40:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9d400640 1 -- 172.21.15.66:0/1026892929 learned_addr learned my addr 172.21.15.66:0/1026892929 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:55.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9d400640 1 -- 172.21.15.66:0/1026892929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda98157630 msgr2=0x7fda98157ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T00:40:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9d400640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda98157630 0x7fda98157ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9d400640 1 -- 172.21.15.66:0/1026892929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9806d240 msgr2=0x7fda980cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9d400640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9806d240 0x7fda980cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9d400640 1 -- 172.21.15.66:0/1026892929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9815af50 con 0x7fda80003f00 2024-03-30T00:40:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9d400640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 0x7fda980bf6b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00e780 tx=0x7fda8c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda95600640 1 -- 172.21.15.66:0/1026892929 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00ee90 con 0x7fda80003f00 2024-03-30T00:40:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda95600640 1 -- 172.21.15.66:0/1026892929 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c00c7f0 con 0x7fda80003f00 2024-03-30T00:40:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda95600640 1 -- 172.21.15.66:0/1026892929 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00cba0 con 0x7fda80003f00 2024-03-30T00:40:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9fe00640 1 -- 172.21.15.66:0/1026892929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9815b230 con 0x7fda80003f00 2024-03-30T00:40:55.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9fe00640 1 -- 172.21.15.66:0/1026892929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9806cd20 con 0x7fda80003f00 2024-03-30T00:40:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.217+0000 7fda9fe00640 1 -- 172.21.15.66:0/1026892929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9806cf40 con 0x7fda80003f00 2024-03-30T00:40:55.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.221+0000 7fda95600640 1 -- 172.21.15.66:0/1026892929 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fda8c00cd00 con 0x7fda80003f00 2024-03-30T00:40:55.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.221+0000 7fda95600640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066680 0x7fda78068b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.221+0000 7fda95600640 1 -- 172.21.15.66:0/1026892929 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fda8c084580 con 0x7fda80003f00 2024-03-30T00:40:55.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.221+0000 7fda9ca00640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066680 0x7fda78068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.221+0000 7fda9ca00640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066680 0x7fda78068b40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fda840ae000 tx=0x7fda84004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:55.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.225+0000 7fda95600640 1 -- 172.21.15.66:0/1026892929 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fda8c051450 con 0x7fda80003f00 2024-03-30T00:40:55.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 -- 172.21.15.66:0/1026892929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda9815b420 con 0x7fda80003f00 2024-03-30T00:40:55.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda95600640 1 -- 172.21.15.66:0/1026892929 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda8c059e60 con 0x7fda80003f00 2024-03-30T00:40:55.394 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 -- 172.21.15.66:0/1026892929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066680 msgr2=0x7fda78068b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066680 0x7fda78068b40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fda840ae000 tx=0x7fda84004740 comp rx=0 tx=0).stop 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 -- 172.21.15.66:0/1026892929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 msgr2=0x7fda980bf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 0x7fda980bf6b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00e780 tx=0x7fda8c00ec40 comp rx=0 tx=0).stop 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 -- 172.21.15.66:0/1026892929 shutdown_connections 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda98157630 0x7fda98157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9806d240 0x7fda980cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066680 0x7fda78068b40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 --2- 172.21.15.66:0/1026892929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda80003f00 0x7fda980bf6b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 -- 172.21.15.66:0/1026892929 >> 172.21.15.66:0/1026892929 conn(0x7fda98069f60 msgr2=0x7fda9815d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 -- 172.21.15.66:0/1026892929 shutdown_connections 2024-03-30T00:40:55.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.385+0000 7fda9fe00640 1 -- 172.21.15.66:0/1026892929 wait complete. 2024-03-30T00:40:55.415 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:55.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb30e00640 1 Processor -- start 2024-03-30T00:40:55.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb30e00640 1 -- start start 2024-03-30T00:40:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb30e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbb2c160c10 0x7fbb2c0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb30e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb2c0b19a0 0x7fbb2c0c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb30e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c0c9a30 con 0x7fbb2c0c1800 2024-03-30T00:40:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb30e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2c0c1c90 con 0x7fbb2c160c10 2024-03-30T00:40:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb30e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0c9d10 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb29e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbb2c160c10 0x7fbb2c0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb29e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbb2c160c10 0x7fbb2c0c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36604/0 (socket says 172.21.15.66:36604) 2024-03-30T00:40:55.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb29e00640 1 -- 172.21.15.66:0/332044559 learned_addr learned my addr 172.21.15.66:0/332044559 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:55.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb29e00640 1 -- 172.21.15.66:0/332044559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb2c0b19a0 msgr2=0x7fbb2c0c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:55.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb29e00640 1 --2- 172.21.15.66:0/332044559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb2c0b19a0 0x7fbb2c0c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb29e00640 1 -- 172.21.15.66:0/332044559 >> v1:172.21.15.156:6789/0 conn(0x7fbb2c0c1800 legacy=0x7fbb2c0cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:55.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.549+0000 7fbb29e00640 1 -- 172.21.15.66:0/332044559 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c0c9ff0 con 0x7fbb2c160c10 2024-03-30T00:40:55.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb29e00640 1 --2- 172.21.15.66:0/332044559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c160c10 0x7fbb2c0c90e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009870 tx=0x7fbb1c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6135f2bfb4d1d5d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:55.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb29400640 1 -- 172.21.15.66:0/332044559 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0c9830 con 0x7fbb2c160c10 2024-03-30T00:40:55.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb29400640 1 -- 172.21.15.66:0/332044559 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c0aa430 con 0x7fbb2c160c10 2024-03-30T00:40:55.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb29400640 1 -- 172.21.15.66:0/332044559 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0aa7c0 con 0x7fbb2c160c10 2024-03-30T00:40:55.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb30e00640 1 -- 172.21.15.66:0/332044559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c160c10 msgr2=0x7fbb2c0c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb30e00640 1 --2- 172.21.15.66:0/332044559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c160c10 0x7fbb2c0c90e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009870 tx=0x7fbb1c01c4f0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb30e00640 1 -- 172.21.15.66:0/332044559 shutdown_connections 2024-03-30T00:40:55.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb30e00640 1 --2- 172.21.15.66:0/332044559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb2c0b19a0 0x7fbb2c0c12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb30e00640 1 --2- 172.21.15.66:0/332044559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c160c10 0x7fbb2c0c90e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb30e00640 1 -- 172.21.15.66:0/332044559 >> 172.21.15.66:0/332044559 conn(0x7fbb2c069f60 msgr2=0x7fbb2c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:55.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb30e00640 1 -- 172.21.15.66:0/332044559 shutdown_connections 2024-03-30T00:40:55.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.553+0000 7fbb30e00640 1 -- 172.21.15.66:0/332044559 wait complete. 2024-03-30T00:40:55.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb30e00640 1 Processor -- start 2024-03-30T00:40:55.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb30e00640 1 -- start start 2024-03-30T00:40:55.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb30e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c0b19a0 0x7fbb2c15d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb30e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb2c0c1800 0x7fbb2c0c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb30e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb2c160c10 0x7fbb2c0c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb30e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0c50b0 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb30e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c15d5c0 con 0x7fbb2c160c10 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb30e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0c4810 con 0x7fbb2c0c1800 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb29e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c0b19a0 0x7fbb2c15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb29e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c0b19a0 0x7fbb2c15d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36612/0 (socket says 172.21.15.66:36612) 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb29e00640 1 -- 172.21.15.66:0/495418890 learned_addr learned my addr 172.21.15.66:0/495418890 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb2a800640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb2c160c10 0x7fbb2c0c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb23e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb2c0c1800 0x7fbb2c0c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb29e00640 1 -- 172.21.15.66:0/495418890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb2c0c1800 msgr2=0x7fbb2c0c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb29e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb2c0c1800 0x7fbb2c0c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb29e00640 1 -- 172.21.15.66:0/495418890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb2c160c10 msgr2=0x7fbb2c0c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb29e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb2c160c10 0x7fbb2c0c1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb29e00640 1 -- 172.21.15.66:0/495418890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c0094d0 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb2a800640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb2c160c10 0x7fbb2c0c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb23e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb2c0c1800 0x7fbb2c0c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb29e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c0b19a0 0x7fbb2c15d080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0ac040 tx=0x7fbb1c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb22000640 1 -- 172.21.15.66:0/495418890 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0b3800 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb22000640 1 -- 172.21.15.66:0/495418890 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c0b3980 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb22000640 1 -- 172.21.15.66:0/495418890 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0b3d30 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb30e00640 1 -- 172.21.15.66:0/495418890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c0c4af0 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb30e00640 1 -- 172.21.15.66:0/495418890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c161970 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.557+0000 7fbb30e00640 1 -- 172.21.15.66:0/495418890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2c066c30 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.561+0000 7fbb22000640 1 -- 172.21.15.66:0/495418890 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbb1c0b2380 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.561+0000 7fbb22000640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb08066690 0x7fbb08068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.561+0000 7fbb22000640 1 -- 172.21.15.66:0/495418890 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fbb1c11ec60 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.561+0000 7fbb23e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb08066690 0x7fbb08068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.561+0000 7fbb23e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb08066690 0x7fbb08068b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0b2eb0 tx=0x7fbb14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.565+0000 7fbb22000640 1 -- 172.21.15.66:0/495418890 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbb1c0ebc70 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.725+0000 7fbb30e00640 1 -- 172.21.15.66:0/495418890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb2c0ceb90 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.725+0000 7fbb22000640 1 -- 172.21.15.66:0/495418890 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb1c0c3020 con 0x7fbb2c0b19a0 2024-03-30T00:40:55.736 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 -- 172.21.15.66:0/495418890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb08066690 msgr2=0x7fbb08068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb08066690 0x7fbb08068b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0b2eb0 tx=0x7fbb14008040 comp rx=0 tx=0).stop 2024-03-30T00:40:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 -- 172.21.15.66:0/495418890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c0b19a0 msgr2=0x7fbb2c15d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c0b19a0 0x7fbb2c15d080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0ac040 tx=0x7fbb1c01ad30 comp rx=0 tx=0).stop 2024-03-30T00:40:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 -- 172.21.15.66:0/495418890 shutdown_connections 2024-03-30T00:40:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb2c160c10 0x7fbb2c0c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb2c0c1800 0x7fbb2c0c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb08066690 0x7fbb08068b50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 --2- 172.21.15.66:0/495418890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c0b19a0 0x7fbb2c15d080 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 -- 172.21.15.66:0/495418890 >> 172.21.15.66:0/495418890 conn(0x7fbb2c069f60 msgr2=0x7fbb2c070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 -- 172.21.15.66:0/495418890 shutdown_connections 2024-03-30T00:40:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.729+0000 7fbb30e00640 1 -- 172.21.15.66:0/495418890 wait complete. 2024-03-30T00:40:55.757 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:55.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.889+0000 7f8821600640 1 Processor -- start 2024-03-30T00:40:55.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.889+0000 7f8821600640 1 -- start start 2024-03-30T00:40:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.889+0000 7f8821600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f881c15ca80 0x7f881c06b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.889+0000 7f8821600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881c0cb570 0x7f881c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.889+0000 7f8821600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f881c0cb0e0 con 0x7f881c0cf2f0 2024-03-30T00:40:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.889+0000 7f8821600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f881c0cf7f0 con 0x7f881c15ca80 2024-03-30T00:40:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.889+0000 7f8821600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881c0cfad0 con 0x7f881c0cb570 2024-03-30T00:40:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.889+0000 7f881aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f881c15ca80 0x7f881c06b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.889+0000 7f8813e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f881c0cf2f0 0x7f881c06b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33406/0 (socket says 172.21.15.66:33406) 2024-03-30T00:40:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.889+0000 7f8813e00640 1 -- 172.21.15.66:0/2369842298 learned_addr learned my addr 172.21.15.66:0/2369842298 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f881b400640 1 --2- 172.21.15.66:0/2369842298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881c0cb570 0x7f881c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f881aa00640 1 -- 172.21.15.66:0/2369842298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881c0cb570 msgr2=0x7f881c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f881aa00640 1 --2- 172.21.15.66:0/2369842298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881c0cb570 0x7f881c0cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f881aa00640 1 -- 172.21.15.66:0/2369842298 >> v1:172.21.15.66:6789/0 conn(0x7f881c0cf2f0 legacy=0x7f881c06b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f881aa00640 1 -- 172.21.15.66:0/2369842298 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c0cad70 con 0x7f881c15ca80 2024-03-30T00:40:55.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f881a000640 1 -- 172.21.15.66:0/2369842298 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608943186 0 0) 0x7f881c0cb0e0 con 0x7f881c0cf2f0 2024-03-30T00:40:55.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f881aa00640 1 --2- 172.21.15.66:0/2369842298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881c15ca80 0x7f881c06b0b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f880c0097c0 tx=0x7f880c01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64baa905a2745ba8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:55.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f881a000640 1 -- 172.21.15.66:0/2369842298 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880c0a8990 con 0x7f881c15ca80 2024-03-30T00:40:55.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f881a000640 1 -- 172.21.15.66:0/2369842298 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880c0a8af0 con 0x7f881c15ca80 2024-03-30T00:40:55.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f881a000640 1 -- 172.21.15.66:0/2369842298 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880c0a8e80 con 0x7f881c15ca80 2024-03-30T00:40:55.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f8821600640 1 -- 172.21.15.66:0/2369842298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881c15ca80 msgr2=0x7f881c06b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.893+0000 7f8821600640 1 --2- 172.21.15.66:0/2369842298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881c15ca80 0x7f881c06b0b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f880c0097c0 tx=0x7f880c01ec90 comp rx=0 tx=0).stop 2024-03-30T00:40:55.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 -- 172.21.15.66:0/2369842298 shutdown_connections 2024-03-30T00:40:55.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 --2- 172.21.15.66:0/2369842298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881c0cb570 0x7f881c0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 --2- 172.21.15.66:0/2369842298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881c15ca80 0x7f881c06b0b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 -- 172.21.15.66:0/2369842298 >> 172.21.15.66:0/2369842298 conn(0x7f881c0b2090 msgr2=0x7f881c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:55.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 -- 172.21.15.66:0/2369842298 shutdown_connections 2024-03-30T00:40:55.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 -- 172.21.15.66:0/2369842298 wait complete. 2024-03-30T00:40:55.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 Processor -- start 2024-03-30T00:40:55.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 -- start start 2024-03-30T00:40:55.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881c0cb570 0x7f881c0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f881c0cf2f0 0x7f881c0b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881c15ca80 0x7f881c152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c159010 con 0x7f881c15ca80 2024-03-30T00:40:55.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c0b37a0 con 0x7f881c0cf2f0 2024-03-30T00:40:55.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8821600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881c0b3a80 con 0x7f881c0cb570 2024-03-30T00:40:55.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f881b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881c15ca80 0x7f881c152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f881b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881c15ca80 0x7f881c152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36642/0 (socket says 172.21.15.66:36642) 2024-03-30T00:40:55.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f881b400640 1 -- 172.21.15.66:0/4038933739 learned_addr learned my addr 172.21.15.66:0/4038933739 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:55.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f881aa00640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881c0cb570 0x7f881c0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f8813e00640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f881c0cf2f0 0x7f881c0b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f881b400640 1 -- 172.21.15.66:0/4038933739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881c0cb570 msgr2=0x7f881c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f881b400640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881c0cb570 0x7f881c0b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f881b400640 1 -- 172.21.15.66:0/4038933739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f881c0cf2f0 msgr2=0x7f881c0b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f881b400640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f881c0cf2f0 0x7f881c0b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:55.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f881b400640 1 -- 172.21.15.66:0/4038933739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c009420 con 0x7f881c15ca80 2024-03-30T00:40:55.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.897+0000 7f881b400640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881c15ca80 0x7f881c152820 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8814007e40 tx=0x7f881400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:55.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.901+0000 7f8812a00640 1 -- 172.21.15.66:0/4038933739 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814014620 con 0x7f881c15ca80 2024-03-30T00:40:55.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.901+0000 7f8812a00640 1 -- 172.21.15.66:0/4038933739 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881400ee90 con 0x7f881c15ca80 2024-03-30T00:40:55.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.901+0000 7f8812a00640 1 -- 172.21.15.66:0/4038933739 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88140149e0 con 0x7f881c15ca80 2024-03-30T00:40:55.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.901+0000 7f8821600640 1 -- 172.21.15.66:0/4038933739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881c0b3d60 con 0x7f881c15ca80 2024-03-30T00:40:55.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.901+0000 7f8821600640 1 -- 172.21.15.66:0/4038933739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881c0b4320 con 0x7f881c15ca80 2024-03-30T00:40:55.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.901+0000 7f8821600640 1 -- 172.21.15.66:0/4038933739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f881c066c30 con 0x7f881c15ca80 2024-03-30T00:40:55.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.901+0000 7f8812a00640 1 -- 172.21.15.66:0/4038933739 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8814014b40 con 0x7f881c15ca80 2024-03-30T00:40:55.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.901+0000 7f8812a00640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f80666c0 0x7f87f8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:55.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.901+0000 7f8812a00640 1 -- 172.21.15.66:0/4038933739 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f8814084db0 con 0x7f881c15ca80 2024-03-30T00:40:55.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.901+0000 7f881aa00640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f80666c0 0x7f87f8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:55.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.901+0000 7f881aa00640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f80666c0 0x7f87f8068b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f880c0b2000 tx=0x7f880c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:55.905+0000 7f8812a00640 1 -- 172.21.15.66:0/4038933739 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8814051db0 con 0x7f881c15ca80 2024-03-30T00:40:56.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.065+0000 7f8821600640 1 -- 172.21.15.66:0/4038933739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f881c0b2bb0 con 0x7f881c15ca80 2024-03-30T00:40:56.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.065+0000 7f8812a00640 1 -- 172.21.15.66:0/4038933739 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f88140553d0 con 0x7f881c15ca80 2024-03-30T00:40:56.076 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:56.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 -- 172.21.15.66:0/4038933739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f80666c0 msgr2=0x7f87f8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f80666c0 0x7f87f8068b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f880c0b2000 tx=0x7f880c008040 comp rx=0 tx=0).stop 2024-03-30T00:40:56.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 -- 172.21.15.66:0/4038933739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881c15ca80 msgr2=0x7f881c152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881c15ca80 0x7f881c152820 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8814007e40 tx=0x7f881400e4b0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 -- 172.21.15.66:0/4038933739 shutdown_connections 2024-03-30T00:40:56.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f80666c0 0x7f87f8068b80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881c15ca80 0x7f881c152820 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f881c0cf2f0 0x7f881c0b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 --2- 172.21.15.66:0/4038933739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881c0cb570 0x7f881c0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 -- 172.21.15.66:0/4038933739 >> 172.21.15.66:0/4038933739 conn(0x7f881c0b2090 msgr2=0x7f881c0c65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:56.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 -- 172.21.15.66:0/4038933739 shutdown_connections 2024-03-30T00:40:56.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.069+0000 7f8821600640 1 -- 172.21.15.66:0/4038933739 wait complete. 2024-03-30T00:40:56.097 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:56.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.229+0000 7fa987e00640 1 Processor -- start 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.229+0000 7fa987e00640 1 -- start start 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa987e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa98815c890 0x7fa9880cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa987e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa9880ce190 0x7fa9880d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa987e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9880d1290 con 0x7fa9880d0e90 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa987e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9880cf0d0 con 0x7fa98815c890 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa987e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9880cf390 con 0x7fa9880ce190 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa986a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa98815c890 0x7fa9880cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa986a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa98815c890 0x7fa9880cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36644/0 (socket says 172.21.15.66:36644) 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa986a00640 1 -- 172.21.15.66:0/3869406068 learned_addr learned my addr 172.21.15.66:0/3869406068 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa97fe00640 1 --2- 172.21.15.66:0/3869406068 >> v2:172.21.15.156:3300/0 conn(0x7fa9880ce190 0x7fa9880d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa986a00640 1 -- 172.21.15.66:0/3869406068 >> v2:172.21.15.156:3300/0 conn(0x7fa9880ce190 msgr2=0x7fa9880d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa986a00640 1 --2- 172.21.15.66:0/3869406068 >> v2:172.21.15.156:3300/0 conn(0x7fa9880ce190 0x7fa9880d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa986a00640 1 -- 172.21.15.66:0/3869406068 >> v1:172.21.15.156:6789/0 conn(0x7fa9880d0e90 legacy=0x7fa9880b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:56.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa986a00640 1 -- 172.21.15.66:0/3869406068 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9880cf670 con 0x7fa98815c890 2024-03-30T00:40:56.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.233+0000 7fa986a00640 1 --2- 172.21.15.66:0/3869406068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa98815c890 0x7fa9880cdc50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa9880b1fb0 tx=0x7fa97801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d722bdfef76bbcb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:56.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa986000640 1 -- 172.21.15.66:0/3869406068 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9780aa650 con 0x7fa98815c890 2024-03-30T00:40:56.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa986000640 1 -- 172.21.15.66:0/3869406068 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9780aa7b0 con 0x7fa98815c890 2024-03-30T00:40:56.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa986000640 1 -- 172.21.15.66:0/3869406068 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9780aab40 con 0x7fa98815c890 2024-03-30T00:40:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 -- 172.21.15.66:0/3869406068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa98815c890 msgr2=0x7fa9880cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 --2- 172.21.15.66:0/3869406068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa98815c890 0x7fa9880cdc50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa9880b1fb0 tx=0x7fa97801c4f0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 -- 172.21.15.66:0/3869406068 shutdown_connections 2024-03-30T00:40:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 --2- 172.21.15.66:0/3869406068 >> v2:172.21.15.156:3300/0 conn(0x7fa9880ce190 0x7fa9880d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 --2- 172.21.15.66:0/3869406068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa98815c890 0x7fa9880cdc50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 -- 172.21.15.66:0/3869406068 >> 172.21.15.66:0/3869406068 conn(0x7fa98809b8d0 msgr2=0x7fa988089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 -- 172.21.15.66:0/3869406068 shutdown_connections 2024-03-30T00:40:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 -- 172.21.15.66:0/3869406068 wait complete. 2024-03-30T00:40:56.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 Processor -- start 2024-03-30T00:40:56.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 -- start start 2024-03-30T00:40:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9880ce190 0x7fa9880c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9880d0e90 0x7fa9880b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa98815c890 0x7fa9880b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9880c6160 con 0x7fa98815c890 2024-03-30T00:40:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9880c8f90 con 0x7fa9880d0e90 2024-03-30T00:40:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98815a1c0 con 0x7fa9880ce190 2024-03-30T00:40:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa98815c890 0x7fa9880b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa98815c890 0x7fa9880b7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36656/0 (socket says 172.21.15.66:36656) 2024-03-30T00:40:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.237+0000 7fa987400640 1 -- 172.21.15.66:0/1486349381 learned_addr learned my addr 172.21.15.66:0/1486349381 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa986a00640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9880ce190 0x7fa9880c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa97fe00640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9880d0e90 0x7fa9880b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa987400640 1 -- 172.21.15.66:0/1486349381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9880ce190 msgr2=0x7fa9880c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa987400640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9880ce190 0x7fa9880c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa987400640 1 -- 172.21.15.66:0/1486349381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9880d0e90 msgr2=0x7fa9880b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa987400640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9880d0e90 0x7fa9880b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa987400640 1 -- 172.21.15.66:0/1486349381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9780094d0 con 0x7fa98815c890 2024-03-30T00:40:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa987400640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa98815c890 0x7fa9880b7080 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa98000c7a0 tx=0x7fa98000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa97ea00640 1 -- 172.21.15.66:0/1486349381 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980003b80 con 0x7fa98815c890 2024-03-30T00:40:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa97ea00640 1 -- 172.21.15.66:0/1486349381 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa980003ce0 con 0x7fa98815c890 2024-03-30T00:40:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa987e00640 1 -- 172.21.15.66:0/1486349381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9880b7df0 con 0x7fa98815c890 2024-03-30T00:40:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa97ea00640 1 -- 172.21.15.66:0/1486349381 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98000aa90 con 0x7fa98815c890 2024-03-30T00:40:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa987e00640 1 -- 172.21.15.66:0/1486349381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa988159e70 con 0x7fa98815c890 2024-03-30T00:40:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa987e00640 1 -- 172.21.15.66:0/1486349381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98815a350 con 0x7fa98815c890 2024-03-30T00:40:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa97ea00640 1 -- 172.21.15.66:0/1486349381 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa98000abf0 con 0x7fa98815c890 2024-03-30T00:40:56.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa97ea00640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa964066690 0x7fa964068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa97ea00640 1 -- 172.21.15.66:0/1486349381 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fa980084270 con 0x7fa98815c890 2024-03-30T00:40:56.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.241+0000 7fa986a00640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa964066690 0x7fa964068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.245+0000 7fa986a00640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa964066690 0x7fa964068b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa9780ae000 tx=0x7fa9780b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:56.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.245+0000 7fa97ea00640 1 -- 172.21.15.66:0/1486349381 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa980051290 con 0x7fa98815c890 2024-03-30T00:40:56.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.405+0000 7fa987e00640 1 -- 172.21.15.66:0/1486349381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa988073e00 con 0x7fa98815c890 2024-03-30T00:40:56.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa97ea00640 1 -- 172.21.15.66:0/1486349381 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa9800548b0 con 0x7fa98815c890 2024-03-30T00:40:56.417 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 -- 172.21.15.66:0/1486349381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa964066690 msgr2=0x7fa964068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa964066690 0x7fa964068b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa9780ae000 tx=0x7fa9780b5040 comp rx=0 tx=0).stop 2024-03-30T00:40:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 -- 172.21.15.66:0/1486349381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa98815c890 msgr2=0x7fa9880b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa98815c890 0x7fa9880b7080 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa98000c7a0 tx=0x7fa98000cc60 comp rx=0 tx=0).stop 2024-03-30T00:40:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 -- 172.21.15.66:0/1486349381 shutdown_connections 2024-03-30T00:40:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa964066690 0x7fa964068b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa98815c890 0x7fa9880b7080 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9880d0e90 0x7fa9880b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 --2- 172.21.15.66:0/1486349381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9880ce190 0x7fa9880c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 -- 172.21.15.66:0/1486349381 >> 172.21.15.66:0/1486349381 conn(0x7fa98809b8d0 msgr2=0x7fa9880b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 -- 172.21.15.66:0/1486349381 shutdown_connections 2024-03-30T00:40:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.409+0000 7fa987e00640 1 -- 172.21.15.66:0/1486349381 wait complete. 2024-03-30T00:40:56.439 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:40:56.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.569+0000 7f3cd2600640 1 Processor -- start 2024-03-30T00:40:56.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3cd2600640 1 -- start start 2024-03-30T00:40:56.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3cd2600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ccc0b3170 0x7f3ccc0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3cd2600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ccc0d07c0 0x7f3ccc0d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3cd2600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc0cb5b0 con 0x7f3ccc15a720 2024-03-30T00:40:56.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3cd2600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc0cfb00 con 0x7f3ccc0b3170 2024-03-30T00:40:56.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3cd2600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0d11b0 con 0x7f3ccc0d07c0 2024-03-30T00:40:56.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3cd1200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f3ccc15a720 0x7f3ccc0ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33412/0 (socket says 172.21.15.66:33412) 2024-03-30T00:40:56.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3cd1200640 1 -- 172.21.15.66:0/448528335 learned_addr learned my addr 172.21.15.66:0/448528335 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:56.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3ccbe00640 1 --2- 172.21.15.66:0/448528335 >> v2:172.21.15.66:3300/0 conn(0x7f3ccc0b3170 0x7f3ccc0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3cd1c00640 1 --2- 172.21.15.66:0/448528335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ccc0d07c0 0x7f3ccc0d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3ccbe00640 1 -- 172.21.15.66:0/448528335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ccc0d07c0 msgr2=0x7f3ccc0d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:56.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3ccbe00640 1 --2- 172.21.15.66:0/448528335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ccc0d07c0 0x7f3ccc0d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3ccbe00640 1 -- 172.21.15.66:0/448528335 >> v1:172.21.15.66:6789/0 conn(0x7f3ccc15a720 legacy=0x7f3ccc0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3ccbe00640 1 -- 172.21.15.66:0/448528335 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc0cb080 con 0x7f3ccc0b3170 2024-03-30T00:40:56.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.573+0000 7f3ccb400640 1 -- 172.21.15.66:0/448528335 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186296801 0 0) 0x7f3cc0001120 con 0x7f3ccc15a720 2024-03-30T00:40:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3ccbe00640 1 --2- 172.21.15.66:0/448528335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ccc0b3170 0x7f3ccc0b3570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097c0 tx=0x7f3cb801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b194e83087f8161 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3ccb400640 1 -- 172.21.15.66:0/448528335 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80ad640 con 0x7f3ccc0b3170 2024-03-30T00:40:56.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3ccb400640 1 -- 172.21.15.66:0/448528335 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb80ad7a0 con 0x7f3ccc0b3170 2024-03-30T00:40:56.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3ccb400640 1 -- 172.21.15.66:0/448528335 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80adb30 con 0x7f3ccc0b3170 2024-03-30T00:40:56.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3cd2600640 1 -- 172.21.15.66:0/448528335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ccc0b3170 msgr2=0x7f3ccc0b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3cd2600640 1 --2- 172.21.15.66:0/448528335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ccc0b3170 0x7f3ccc0b3570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097c0 tx=0x7f3cb801c4a0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3cd2600640 1 -- 172.21.15.66:0/448528335 shutdown_connections 2024-03-30T00:40:56.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3cd2600640 1 --2- 172.21.15.66:0/448528335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ccc0d07c0 0x7f3ccc0d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3cd2600640 1 --2- 172.21.15.66:0/448528335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ccc0b3170 0x7f3ccc0b3570 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3cd2600640 1 -- 172.21.15.66:0/448528335 >> 172.21.15.66:0/448528335 conn(0x7f3ccc069d90 msgr2=0x7f3ccc06a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:56.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3cd2600640 1 -- 172.21.15.66:0/448528335 shutdown_connections 2024-03-30T00:40:56.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3cd2600640 1 -- 172.21.15.66:0/448528335 wait complete. 2024-03-30T00:40:56.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3cd2600640 1 Processor -- start 2024-03-30T00:40:56.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.577+0000 7f3cd2600640 1 -- start start 2024-03-30T00:40:56.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd2600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ccc0b3170 0x7f3ccc0c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd2600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ccc0d07c0 0x7f3ccc150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd2600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ccc15a720 0x7f3ccc151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd2600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1512d0 con 0x7f3ccc15a720 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd2600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0c5820 con 0x7f3ccc0b3170 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd2600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0c5b00 con 0x7f3ccc0d07c0 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ccc15a720 0x7f3ccc151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ccc15a720 0x7f3ccc151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36686/0 (socket says 172.21.15.66:36686) 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd1c00640 1 -- 172.21.15.66:0/1031412022 learned_addr learned my addr 172.21.15.66:0/1031412022 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd1200640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ccc0b3170 0x7f3ccc0c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3ccbe00640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ccc0d07c0 0x7f3ccc150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd1c00640 1 -- 172.21.15.66:0/1031412022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ccc0d07c0 msgr2=0x7f3ccc150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd1c00640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ccc0d07c0 0x7f3ccc150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd1c00640 1 -- 172.21.15.66:0/1031412022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ccc0b3170 msgr2=0x7f3ccc0c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd1c00640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ccc0b3170 0x7f3ccc0c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd1c00640 1 -- 172.21.15.66:0/1031412022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb8009420 con 0x7f3ccc15a720 2024-03-30T00:40:56.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd1c00640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ccc15a720 0x7f3ccc151ed0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400efc0 tx=0x7f3cc400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:56.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cc8c00640 1 -- 172.21.15.66:0/1031412022 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4014480 con 0x7f3ccc15a720 2024-03-30T00:40:56.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cc8c00640 1 -- 172.21.15.66:0/1031412022 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc400f040 con 0x7f3ccc15a720 2024-03-30T00:40:56.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd2600640 1 -- 172.21.15.66:0/1031412022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc0c5de0 con 0x7f3ccc15a720 2024-03-30T00:40:56.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cc8c00640 1 -- 172.21.15.66:0/1031412022 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4014820 con 0x7f3ccc15a720 2024-03-30T00:40:56.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd2600640 1 -- 172.21.15.66:0/1031412022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc0c63a0 con 0x7f3ccc15a720 2024-03-30T00:40:56.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cd2600640 1 -- 172.21.15.66:0/1031412022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc073e00 con 0x7f3ccc15a720 2024-03-30T00:40:56.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.581+0000 7f3cc8c00640 1 -- 172.21.15.66:0/1031412022 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3cc4014980 con 0x7f3ccc15a720 2024-03-30T00:40:56.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.585+0000 7f3cc8c00640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3cac0666c0 0x7f3cac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:56.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.585+0000 7f3cc8c00640 1 -- 172.21.15.66:0/1031412022 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f3cc4084f80 con 0x7f3ccc15a720 2024-03-30T00:40:56.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.585+0000 7f3cd1200640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3cac0666c0 0x7f3cac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:56.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.585+0000 7f3cd1200640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3cac0666c0 0x7f3cac068b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0000f70 tx=0x7f3cc0007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:56.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.589+0000 7f3cc8c00640 1 -- 172.21.15.66:0/1031412022 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3cc4051f90 con 0x7f3ccc15a720 2024-03-30T00:40:56.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cd2600640 1 -- 172.21.15.66:0/1031412022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3ccc0b2ab0 con 0x7f3ccc15a720 2024-03-30T00:40:56.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cc8c00640 1 -- 172.21.15.66:0/1031412022 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3cc4018020 con 0x7f3ccc15a720 2024-03-30T00:40:56.786 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cd2600640 1 -- 172.21.15.66:0/1031412022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3cac0666c0 msgr2=0x7f3cac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cd2600640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3cac0666c0 0x7f3cac068b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0000f70 tx=0x7f3cc0007df0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cd2600640 1 -- 172.21.15.66:0/1031412022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ccc15a720 msgr2=0x7f3ccc151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cd2600640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ccc15a720 0x7f3ccc151ed0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400efc0 tx=0x7f3cc400c490 comp rx=0 tx=0).stop 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cd2600640 1 -- 172.21.15.66:0/1031412022 shutdown_connections 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cd2600640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3cac0666c0 0x7f3cac068b80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cd2600640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ccc15a720 0x7f3ccc151ed0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cd2600640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ccc0d07c0 0x7f3ccc150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cd2600640 1 --2- 172.21.15.66:0/1031412022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ccc0b3170 0x7f3ccc0c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.777+0000 7f3cd2600640 1 -- 172.21.15.66:0/1031412022 >> 172.21.15.66:0/1031412022 conn(0x7f3ccc069d90 msgr2=0x7f3ccc15e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.781+0000 7f3cd2600640 1 -- 172.21.15.66:0/1031412022 shutdown_connections 2024-03-30T00:40:56.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:56.781+0000 7f3cd2600640 1 -- 172.21.15.66:0/1031412022 wait complete. 2024-03-30T00:40:56.808 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi156.front.sepia.ceph.com... 2024-03-30T00:40:56.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:56.808 DEBUG:teuthology.orchestra.run.smithi156:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-03-30T00:40:56.840 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:40:56.840 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:40:56.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:40:56.841 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T00:40:56.898 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:40:56.899 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:40:56.907 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:40:56.907 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:40:56.956 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:57.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb015e00640 1 Processor -- start 2024-03-30T00:40:57.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb015e00640 1 -- start start 2024-03-30T00:40:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb015e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb01015db90 0x7fb01015e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb015e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb010070b50 0x7fb010070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb015e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01006c410 con 0x7fb0100ce6b0 2024-03-30T00:40:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb015e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb010071520 con 0x7fb01015db90 2024-03-30T00:40:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb015e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01006c6b0 con 0x7fb010070b50 2024-03-30T00:40:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb006000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb01015db90 0x7fb01015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb006000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb01015db90 0x7fb01015e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36690/0 (socket says 172.21.15.66:36690) 2024-03-30T00:40:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb006000640 1 -- 172.21.15.66:0/3803932203 learned_addr learned my addr 172.21.15.66:0/3803932203 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb00f400640 1 --2- 172.21.15.66:0/3803932203 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb010070b50 0x7fb010070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb006000640 1 -- 172.21.15.66:0/3803932203 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb010070b50 msgr2=0x7fb010070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb006000640 1 --2- 172.21.15.66:0/3803932203 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb010070b50 0x7fb010070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb006000640 1 -- 172.21.15.66:0/3803932203 >> v1:172.21.15.156:6789/0 conn(0x7fb0100ce6b0 legacy=0x7fb0100cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.133+0000 7fb006000640 1 -- 172.21.15.66:0/3803932203 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01006c990 con 0x7fb01015db90 2024-03-30T00:40:57.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb006000640 1 --2- 172.21.15.66:0/3803932203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb01015db90 0x7fb01015e070 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce6ea83e4c3b23b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:57.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb00e000640 1 -- 172.21.15.66:0/3803932203 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff80a6040 con 0x7fb01015db90 2024-03-30T00:40:57.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb00e000640 1 -- 172.21.15.66:0/3803932203 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff80aa600 con 0x7fb01015db90 2024-03-30T00:40:57.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb00e000640 1 -- 172.21.15.66:0/3803932203 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff80aa990 con 0x7fb01015db90 2024-03-30T00:40:57.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb015e00640 1 -- 172.21.15.66:0/3803932203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb01015db90 msgr2=0x7fb01015e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb015e00640 1 --2- 172.21.15.66:0/3803932203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb01015db90 0x7fb01015e070 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff801c4a0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb015e00640 1 -- 172.21.15.66:0/3803932203 shutdown_connections 2024-03-30T00:40:57.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb015e00640 1 --2- 172.21.15.66:0/3803932203 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb010070b50 0x7fb010070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb015e00640 1 --2- 172.21.15.66:0/3803932203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb01015db90 0x7fb01015e070 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb015e00640 1 -- 172.21.15.66:0/3803932203 >> 172.21.15.66:0/3803932203 conn(0x7fb010066100 msgr2=0x7fb010069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:57.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb015e00640 1 -- 172.21.15.66:0/3803932203 shutdown_connections 2024-03-30T00:40:57.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.137+0000 7fb015e00640 1 -- 172.21.15.66:0/3803932203 wait complete. 2024-03-30T00:40:57.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb015e00640 1 Processor -- start 2024-03-30T00:40:57.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb015e00640 1 -- start start 2024-03-30T00:40:57.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb015e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb010070b50 0x7fb01014ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb015e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0100ce6b0 0x7fb010150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb015e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb01015db90 0x7fb01014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb015e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01014d660 con 0x7fb010070b50 2024-03-30T00:40:57.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb015e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb010150e40 con 0x7fb01015db90 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb015e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb010155800 con 0x7fb0100ce6b0 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb00ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb010070b50 0x7fb01014ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb00ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb010070b50 0x7fb01014ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36694/0 (socket says 172.21.15.66:36694) 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb00ea00640 1 -- 172.21.15.66:0/1120552044 learned_addr learned my addr 172.21.15.66:0/1120552044 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb006000640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0100ce6b0 0x7fb010150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb00f400640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb01015db90 0x7fb01014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb00ea00640 1 -- 172.21.15.66:0/1120552044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0100ce6b0 msgr2=0x7fb010150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb00ea00640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0100ce6b0 0x7fb010150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb00ea00640 1 -- 172.21.15.66:0/1120552044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb01015db90 msgr2=0x7fb01014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb00ea00640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb01015db90 0x7fb01014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb00ea00640 1 -- 172.21.15.66:0/1120552044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff80094d0 con 0x7fb010070b50 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb006000640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0100ce6b0 0x7fb010150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:57.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb00ea00640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb010070b50 0x7fb01014ce50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb00000c500 tx=0x7fb00000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:57.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb007e00640 1 -- 172.21.15.66:0/1120552044 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000005220 con 0x7fb010070b50 2024-03-30T00:40:57.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb007e00640 1 -- 172.21.15.66:0/1120552044 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb000005380 con 0x7fb010070b50 2024-03-30T00:40:57.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb007e00640 1 -- 172.21.15.66:0/1120552044 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000007270 con 0x7fb010070b50 2024-03-30T00:40:57.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb015e00640 1 -- 172.21.15.66:0/1120552044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb010155ae0 con 0x7fb010070b50 2024-03-30T00:40:57.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.141+0000 7fb015e00640 1 -- 172.21.15.66:0/1120552044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0101560a0 con 0x7fb010070b50 2024-03-30T00:40:57.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.145+0000 7fb015e00640 1 -- 172.21.15.66:0/1120552044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01006e320 con 0x7fb010070b50 2024-03-30T00:40:57.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.145+0000 7fb007e00640 1 -- 172.21.15.66:0/1120552044 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb0000073d0 con 0x7fb010070b50 2024-03-30T00:40:57.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.145+0000 7fb007e00640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafec066690 0x7fafec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.145+0000 7fb007e00640 1 -- 172.21.15.66:0/1120552044 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fb000083f20 con 0x7fb010070b50 2024-03-30T00:40:57.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.145+0000 7fb006000640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafec066690 0x7fafec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.145+0000 7fb006000640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafec066690 0x7fafec068b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faff80ae000 tx=0x7faff80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:57.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.149+0000 7fb007e00640 1 -- 172.21.15.66:0/1120552044 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb000050e00 con 0x7fb010070b50 2024-03-30T00:40:57.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 -- 172.21.15.66:0/1120552044 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb010066c30 con 0x7fb010070b50 2024-03-30T00:40:57.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb007e00640 1 -- 172.21.15.66:0/1120552044 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7fb000054420 con 0x7fb010070b50 2024-03-30T00:40:57.323 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:57.323 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 -- 172.21.15.66:0/1120552044 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafec066690 msgr2=0x7fafec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafec066690 0x7fafec068b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faff80ae000 tx=0x7faff80a7000 comp rx=0 tx=0).stop 2024-03-30T00:40:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 -- 172.21.15.66:0/1120552044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb010070b50 msgr2=0x7fb01014ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb010070b50 0x7fb01014ce50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb00000c500 tx=0x7fb00000c9c0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 -- 172.21.15.66:0/1120552044 shutdown_connections 2024-03-30T00:40:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb01015db90 0x7fb01014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0100ce6b0 0x7fb010150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafec066690 0x7fafec068b50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 --2- 172.21.15.66:0/1120552044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb010070b50 0x7fb01014ce50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 -- 172.21.15.66:0/1120552044 >> 172.21.15.66:0/1120552044 conn(0x7fb010066100 msgr2=0x7fb01015e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:57.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.313+0000 7fb015e00640 1 -- 172.21.15.66:0/1120552044 shutdown_connections 2024-03-30T00:40:57.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.317+0000 7fb015e00640 1 -- 172.21.15.66:0/1120552044 wait complete. 2024-03-30T00:40:57.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:57.345 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:57.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.477+0000 7f1a75200640 1 Processor -- start 2024-03-30T00:40:57.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.477+0000 7f1a75200640 1 -- start start 2024-03-30T00:40:57.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a75200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700cec30 0x7f1a70159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a75200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a700cd3e0 con 0x7f1a700cda90 2024-03-30T00:40:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a75200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7015a990 con 0x7f1a700cde70 2024-03-30T00:40:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a75200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700b1b80 con 0x7f1a700cec30 2024-03-30T00:40:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6de00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1a700cda90 0x7f1a700ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33414/0 (socket says 172.21.15.66:33414) 2024-03-30T00:40:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6de00640 1 -- 172.21.15.66:0/2100575867 learned_addr learned my addr 172.21.15.66:0/2100575867 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a65400640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700cec30 0x7f1a70159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294308464 0 0) 0x7f1a700cd3e0 con 0x7f1a700cda90 2024-03-30T00:40:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a54003530 con 0x7f1a700cda90 2024-03-30T00:40:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3534239744 0 0) 0x7f1a54003530 con 0x7f1a700cda90 2024-03-30T00:40:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700cec30 msgr2=0x7f1a70159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700cec30 0x7f1a70159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 >> v1:172.21.15.156:6789/0 conn(0x7f1a700cde70 legacy=0x7f1a7015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a700b11e0 con 0x7f1a700cda90 2024-03-30T00:40:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f1a60001a00 con 0x7f1a700cda90 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 >> v1:172.21.15.66:6789/0 conn(0x7f1a700cda90 legacy=0x7f1a700ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a540039b0 0x7f1a54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700cec30 0x7f1a540047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a54004d10 0x7f1a54005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700b1b80 con 0x7f1a54004d10 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5400ba70 con 0x7f1a540039b0 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5400de40 con 0x7f1a700cec30 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6e800640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a54004d10 0x7f1a54005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6de00640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a540039b0 0x7f1a54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a65400640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700cec30 0x7f1a540047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6e800640 1 -- 172.21.15.66:0/2100575867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700cec30 msgr2=0x7f1a540047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6e800640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700cec30 0x7f1a540047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6e800640 1 -- 172.21.15.66:0/2100575867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a540039b0 msgr2=0x7f1a54003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:57.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6e800640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a540039b0 0x7f1a54003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.481+0000 7f1a6e800640 1 -- 172.21.15.66:0/2100575867 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a54003530 con 0x7f1a54004d10 2024-03-30T00:40:57.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.485+0000 7f1a6e800640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a54004d10 0x7f1a54005190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1a68001910 tx=0x7f1a6801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04d725f97edb6bb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:57.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.485+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a680a6040 con 0x7f1a54004d10 2024-03-30T00:40:57.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.485+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a680aa620 con 0x7f1a54004d10 2024-03-30T00:40:57.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.485+0000 7f1a6d400640 1 -- 172.21.15.66:0/2100575867 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a680aa9d0 con 0x7f1a54004d10 2024-03-30T00:40:57.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.485+0000 7f1a75200640 1 -- 172.21.15.66:0/2100575867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a54004d10 msgr2=0x7f1a54005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.485+0000 7f1a75200640 1 --2- 172.21.15.66:0/2100575867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a54004d10 0x7f1a54005190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1a68001910 tx=0x7f1a6801c560 comp rx=0 tx=0).stop 2024-03-30T00:40:57.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.485+0000 7f1a6e800640 1 -- 172.21.15.66:0/2100575867 reap_dead start 2024-03-30T00:40:57.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.485+0000 7f1a75200640 1 -- 172.21.15.66:0/2100575867 shutdown_connections 2024-03-30T00:40:57.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.485+0000 7f1a75200640 1 -- 172.21.15.66:0/2100575867 >> 172.21.15.66:0/2100575867 conn(0x7f1a70069f60 msgr2=0x7f1a7006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:57.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.485+0000 7f1a75200640 1 -- 172.21.15.66:0/2100575867 shutdown_connections 2024-03-30T00:40:57.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.485+0000 7f1a75200640 1 -- 172.21.15.66:0/2100575867 wait complete. 2024-03-30T00:40:57.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a75200640 1 Processor -- start 2024-03-30T00:40:57.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a75200640 1 -- start start 2024-03-30T00:40:57.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a75200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54004d10 0x7f1a700c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a75200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7006fed0 0x7f1a70070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a75200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700abbb0 0x7f1a700ac060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a75200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a70070890 con 0x7f1a7006fed0 2024-03-30T00:40:57.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a75200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a70157f40 con 0x7f1a54004d10 2024-03-30T00:40:57.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a75200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700c48a0 con 0x7f1a700abbb0 2024-03-30T00:40:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a65400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7006fed0 0x7f1a70070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a65400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7006fed0 0x7f1a70070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36710/0 (socket says 172.21.15.66:36710) 2024-03-30T00:40:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a65400640 1 -- 172.21.15.66:0/1384597899 learned_addr learned my addr 172.21.15.66:0/1384597899 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a6de00640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54004d10 0x7f1a700c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a6e800640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700abbb0 0x7f1a700ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a65400640 1 -- 172.21.15.66:0/1384597899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700abbb0 msgr2=0x7f1a700ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a65400640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700abbb0 0x7f1a700ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a65400640 1 -- 172.21.15.66:0/1384597899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54004d10 msgr2=0x7f1a700c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a65400640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54004d10 0x7f1a700c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a65400640 1 -- 172.21.15.66:0/1384597899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70157650 con 0x7f1a7006fed0 2024-03-30T00:40:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a65400640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7006fed0 0x7f1a70070350 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1a58011a00 tx=0x7f1a58011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a66a00640 1 -- 172.21.15.66:0/1384597899 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800fa90 con 0x7f1a7006fed0 2024-03-30T00:40:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a66a00640 1 -- 172.21.15.66:0/1384597899 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5800fbf0 con 0x7f1a7006fed0 2024-03-30T00:40:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a66a00640 1 -- 172.21.15.66:0/1384597899 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58014440 con 0x7f1a7006fed0 2024-03-30T00:40:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a75200640 1 -- 172.21.15.66:0/1384597899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a700ac910 con 0x7f1a7006fed0 2024-03-30T00:40:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a75200640 1 -- 172.21.15.66:0/1384597899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a70157bd0 con 0x7f1a7006fed0 2024-03-30T00:40:57.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.489+0000 7f1a75200640 1 -- 172.21.15.66:0/1384597899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a70066c30 con 0x7f1a7006fed0 2024-03-30T00:40:57.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.493+0000 7f1a66a00640 1 -- 172.21.15.66:0/1384597899 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1a580145a0 con 0x7f1a7006fed0 2024-03-30T00:40:57.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.493+0000 7f1a66a00640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a4c0666c0 0x7f1a4c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.493+0000 7f1a66a00640 1 -- 172.21.15.66:0/1384597899 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f1a58084ca0 con 0x7f1a7006fed0 2024-03-30T00:40:57.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.493+0000 7f1a6de00640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a4c0666c0 0x7f1a4c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.493+0000 7f1a6de00640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a4c0666c0 0x7f1a4c068b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1a60001470 tx=0x7f1a60006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:57.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.497+0000 7f1a66a00640 1 -- 172.21.15.66:0/1384597899 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1a58051cd0 con 0x7f1a7006fed0 2024-03-30T00:40:57.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a75200640 1 -- 172.21.15.66:0/1384597899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a70158080 con 0x7f1a7006fed0 2024-03-30T00:40:57.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a66a00640 1 -- 172.21.15.66:0/1384597899 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a580552f0 con 0x7f1a7006fed0 2024-03-30T00:40:57.667 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a75200640 1 -- 172.21.15.66:0/1384597899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a4c0666c0 msgr2=0x7f1a4c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a75200640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a4c0666c0 0x7f1a4c068b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1a60001470 tx=0x7f1a60006040 comp rx=0 tx=0).stop 2024-03-30T00:40:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a75200640 1 -- 172.21.15.66:0/1384597899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7006fed0 msgr2=0x7f1a70070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a75200640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7006fed0 0x7f1a70070350 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1a58011a00 tx=0x7f1a58011e00 comp rx=0 tx=0).stop 2024-03-30T00:40:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a75200640 1 -- 172.21.15.66:0/1384597899 shutdown_connections 2024-03-30T00:40:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a75200640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a700abbb0 0x7f1a700ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a75200640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a4c0666c0 0x7f1a4c068b80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a75200640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a7006fed0 0x7f1a70070350 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a75200640 1 --2- 172.21.15.66:0/1384597899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54004d10 0x7f1a700c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.657+0000 7f1a75200640 1 -- 172.21.15.66:0/1384597899 >> 172.21.15.66:0/1384597899 conn(0x7f1a70069f60 msgr2=0x7f1a7006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:57.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.661+0000 7f1a75200640 1 -- 172.21.15.66:0/1384597899 shutdown_connections 2024-03-30T00:40:57.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.661+0000 7f1a75200640 1 -- 172.21.15.66:0/1384597899 wait complete. 2024-03-30T00:40:57.688 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.821+0000 7f4e19e00640 1 Processor -- start 2024-03-30T00:40:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.821+0000 7f4e19e00640 1 -- start start 2024-03-30T00:40:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e19e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e140d0ed0 0x7f4e140b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e19e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e140cb5b0 con 0x7f4e140b3170 2024-03-30T00:40:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e19e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e140cb080 con 0x7f4e1415a720 2024-03-30T00:40:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e19e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e140cfb00 con 0x7f4e140d0ed0 2024-03-30T00:40:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e13e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f4e140b3170 0x7f4e140d07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33418/0 (socket says 172.21.15.66:33418) 2024-03-30T00:40:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e13e00640 1 -- 172.21.15.66:0/985692863 learned_addr learned my addr 172.21.15.66:0/985692863 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e19400640 1 --2- 172.21.15.66:0/985692863 >> v2:172.21.15.156:3300/0 conn(0x7f4e140d0ed0 0x7f4e140b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e13400640 1 -- 172.21.15.66:0/985692863 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675549032 0 0) 0x7f4e00001120 con 0x7f4e140b3170 2024-03-30T00:40:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e13400640 1 -- 172.21.15.66:0/985692863 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dfc003530 con 0x7f4e140b3170 2024-03-30T00:40:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e13400640 1 -- 172.21.15.66:0/985692863 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699281453 0 0) 0x7f4e140cb080 con 0x7f4e1415a720 2024-03-30T00:40:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e13400640 1 -- 172.21.15.66:0/985692863 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e00001120 con 0x7f4e1415a720 2024-03-30T00:40:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e13400640 1 -- 172.21.15.66:0/985692863 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1478524181 0 0) 0x7f4dfc003530 con 0x7f4e140b3170 2024-03-30T00:40:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e19400640 1 -- 172.21.15.66:0/985692863 >> v1:172.21.15.156:6789/0 conn(0x7f4e1415a720 legacy=0x7f4e140ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e19400640 1 -- 172.21.15.66:0/985692863 >> v1:172.21.15.66:6789/0 conn(0x7f4e140b3170 legacy=0x7f4e140d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.825+0000 7f4e19400640 1 -- 172.21.15.66:0/985692863 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1415ab00 con 0x7f4e140d0ed0 2024-03-30T00:40:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19400640 1 --2- 172.21.15.66:0/985692863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140d0ed0 0x7f4e140b3570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009d90 tx=0x7f4e0c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fec75719a77f70 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e13400640 1 -- 172.21.15.66:0/985692863 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00001810 con 0x7f4e140d0ed0 2024-03-30T00:40:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e13400640 1 -- 172.21.15.66:0/985692863 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c0a56d0 con 0x7f4e140d0ed0 2024-03-30T00:40:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e13400640 1 -- 172.21.15.66:0/985692863 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c0a5a60 con 0x7f4e140d0ed0 2024-03-30T00:40:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 -- 172.21.15.66:0/985692863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140d0ed0 msgr2=0x7f4e140b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 --2- 172.21.15.66:0/985692863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140d0ed0 0x7f4e140b3570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009d90 tx=0x7f4e0c01c890 comp rx=0 tx=0).stop 2024-03-30T00:40:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 -- 172.21.15.66:0/985692863 shutdown_connections 2024-03-30T00:40:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 --2- 172.21.15.66:0/985692863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140d0ed0 0x7f4e140b3570 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 -- 172.21.15.66:0/985692863 >> 172.21.15.66:0/985692863 conn(0x7f4e14069d90 msgr2=0x7f4e1406a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 -- 172.21.15.66:0/985692863 shutdown_connections 2024-03-30T00:40:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 -- 172.21.15.66:0/985692863 wait complete. 2024-03-30T00:40:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 Processor -- start 2024-03-30T00:40:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 -- start start 2024-03-30T00:40:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140b3170 0x7f4e140c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140d0ed0 0x7f4e1414eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e1415a720 0x7f4e1415d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1415dc90 con 0x7f4e140d0ed0 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14158dd0 con 0x7f4e140b3170 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e19e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1415aea0 con 0x7f4e1415a720 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.829+0000 7f4e13e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140d0ed0 0x7f4e1414eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e13e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140d0ed0 0x7f4e1414eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36724/0 (socket says 172.21.15.66:36724) 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e13e00640 1 -- 172.21.15.66:0/44670254 learned_addr learned my addr 172.21.15.66:0/44670254 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e18a00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140b3170 0x7f4e140c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e19400640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e1415a720 0x7f4e1415d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e13e00640 1 -- 172.21.15.66:0/44670254 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e1415a720 msgr2=0x7f4e1415d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e13e00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e1415a720 0x7f4e1415d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e13e00640 1 -- 172.21.15.66:0/44670254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140b3170 msgr2=0x7f4e140c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e13e00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140b3170 0x7f4e140c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e13e00640 1 -- 172.21.15.66:0/44670254 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c009a70 con 0x7f4e140d0ed0 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e19400640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e1415a720 0x7f4e1415d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:57.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e13e00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140d0ed0 0x7f4e1414eec0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000caf0 tx=0x7f4e00001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:57.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e10c00640 1 -- 172.21.15.66:0/44670254 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00005cf0 con 0x7f4e140d0ed0 2024-03-30T00:40:57.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e10c00640 1 -- 172.21.15.66:0/44670254 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e00005e70 con 0x7f4e140d0ed0 2024-03-30T00:40:57.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e10c00640 1 -- 172.21.15.66:0/44670254 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00007290 con 0x7f4e140d0ed0 2024-03-30T00:40:57.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e19e00640 1 -- 172.21.15.66:0/44670254 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e140c4d50 con 0x7f4e140d0ed0 2024-03-30T00:40:57.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e19e00640 1 -- 172.21.15.66:0/44670254 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e140c5290 con 0x7f4e140d0ed0 2024-03-30T00:40:57.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e19e00640 1 -- 172.21.15.66:0/44670254 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e14073e00 con 0x7f4e140d0ed0 2024-03-30T00:40:57.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e10c00640 1 -- 172.21.15.66:0/44670254 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4e000073f0 con 0x7f4e140d0ed0 2024-03-30T00:40:57.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e10c00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 0x7f4df4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:57.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.833+0000 7f4e10c00640 1 -- 172.21.15.66:0/44670254 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f4e00083c50 con 0x7f4e140d0ed0 2024-03-30T00:40:57.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.837+0000 7f4e18a00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 0x7f4df4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:57.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.837+0000 7f4e18a00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 0x7f4df4068b50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e080011f0 tx=0x7f4e08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:57.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:57.837+0000 7f4e10c00640 1 -- 172.21.15.66:0/44670254 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4e00089020 con 0x7f4e140d0ed0 2024-03-30T00:40:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 -- 172.21.15.66:0/44670254 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e140cb5b0 con 0x7f4e140d0ed0 2024-03-30T00:40:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e10c00640 1 -- 172.21.15.66:0/44670254 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e140cb5b0 con 0x7f4e140d0ed0 2024-03-30T00:40:58.009 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:58.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 -- 172.21.15.66:0/44670254 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 msgr2=0x7f4df4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 0x7f4df4068b50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e080011f0 tx=0x7f4e08009040 comp rx=0 tx=0).stop 2024-03-30T00:40:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 -- 172.21.15.66:0/44670254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140d0ed0 msgr2=0x7f4e1414eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140d0ed0 0x7f4e1414eec0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000caf0 tx=0x7f4e00001860 comp rx=0 tx=0).stop 2024-03-30T00:40:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 -- 172.21.15.66:0/44670254 shutdown_connections 2024-03-30T00:40:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e1415a720 0x7f4e1415d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 0x7f4df4068b50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140d0ed0 0x7f4e1414eec0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 --2- 172.21.15.66:0/44670254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140b3170 0x7f4e140c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 -- 172.21.15.66:0/44670254 >> 172.21.15.66:0/44670254 conn(0x7f4e14069d90 msgr2=0x7f4e140ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 -- 172.21.15.66:0/44670254 shutdown_connections 2024-03-30T00:40:58.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.001+0000 7f4e19e00640 1 -- 172.21.15.66:0/44670254 wait complete. 2024-03-30T00:40:58.030 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:40:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb27800640 1 Processor -- start 2024-03-30T00:40:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb27800640 1 -- start start 2024-03-30T00:40:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb27800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7feb200cda90 0x7feb200ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb27800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7feb2015a370 0x7feb200cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb27800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2015a990 con 0x7feb200cde70 2024-03-30T00:40:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb27800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200b1b80 con 0x7feb2015a370 2024-03-30T00:40:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb27800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200cd3e0 con 0x7feb200cda90 2024-03-30T00:40:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb25800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7feb2015a370 0x7feb200cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb25800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7feb2015a370 0x7feb200cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36736/0 (socket says 172.21.15.66:36736) 2024-03-30T00:40:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb25800640 1 -- 172.21.15.66:0/2131541455 learned_addr learned my addr 172.21.15.66:0/2131541455 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb24e00640 1 --2- 172.21.15.66:0/2131541455 >> v2:172.21.15.156:3300/0 conn(0x7feb200cda90 0x7feb200ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb25800640 1 -- 172.21.15.66:0/2131541455 >> v2:172.21.15.156:3300/0 conn(0x7feb200cda90 msgr2=0x7feb200ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb25800640 1 --2- 172.21.15.66:0/2131541455 >> v2:172.21.15.156:3300/0 conn(0x7feb200cda90 0x7feb200ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb25800640 1 -- 172.21.15.66:0/2131541455 >> v1:172.21.15.66:6789/0 conn(0x7feb200cde70 legacy=0x7feb20159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.165+0000 7feb25800640 1 -- 172.21.15.66:0/2131541455 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb200b11e0 con 0x7feb2015a370 2024-03-30T00:40:58.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb25800640 1 --2- 172.21.15.66:0/2131541455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb2015a370 0x7feb200cea60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feb200cf3f0 tx=0x7feb1801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894d05931a202ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:58.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb1fe00640 1 -- 172.21.15.66:0/2131541455 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb180a5730 con 0x7feb2015a370 2024-03-30T00:40:58.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb1fe00640 1 -- 172.21.15.66:0/2131541455 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb180a5890 con 0x7feb2015a370 2024-03-30T00:40:58.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb1fe00640 1 -- 172.21.15.66:0/2131541455 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb180a5c20 con 0x7feb2015a370 2024-03-30T00:40:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb27800640 1 -- 172.21.15.66:0/2131541455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb2015a370 msgr2=0x7feb200cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb27800640 1 --2- 172.21.15.66:0/2131541455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb2015a370 0x7feb200cea60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feb200cf3f0 tx=0x7feb1801c960 comp rx=0 tx=0).stop 2024-03-30T00:40:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb27800640 1 -- 172.21.15.66:0/2131541455 shutdown_connections 2024-03-30T00:40:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb27800640 1 --2- 172.21.15.66:0/2131541455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb2015a370 0x7feb200cea60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb27800640 1 --2- 172.21.15.66:0/2131541455 >> v2:172.21.15.156:3300/0 conn(0x7feb200cda90 0x7feb200ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb27800640 1 -- 172.21.15.66:0/2131541455 >> 172.21.15.66:0/2131541455 conn(0x7feb20069f60 msgr2=0x7feb2006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb27800640 1 -- 172.21.15.66:0/2131541455 shutdown_connections 2024-03-30T00:40:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb27800640 1 -- 172.21.15.66:0/2131541455 wait complete. 2024-03-30T00:40:58.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb27800640 1 Processor -- start 2024-03-30T00:40:58.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.169+0000 7feb27800640 1 -- start start 2024-03-30T00:40:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb27800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb200cda90 0x7feb200c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb27800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb200cde70 0x7feb20156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb27800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb2015a370 0x7feb200cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb27800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb200c3b40 con 0x7feb200cda90 2024-03-30T00:40:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb27800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb200c2c70 con 0x7feb200cde70 2024-03-30T00:40:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb27800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb201566a0 con 0x7feb2015a370 2024-03-30T00:40:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb24e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb200cda90 0x7feb200c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb24e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb200cda90 0x7feb200c2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36750/0 (socket says 172.21.15.66:36750) 2024-03-30T00:40:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb24e00640 1 -- 172.21.15.66:0/3314136546 learned_addr learned my addr 172.21.15.66:0/3314136546 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb25800640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb2015a370 0x7feb200cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb17e00640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb200cde70 0x7feb20156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb24e00640 1 -- 172.21.15.66:0/3314136546 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb2015a370 msgr2=0x7feb200cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb24e00640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb2015a370 0x7feb200cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb24e00640 1 -- 172.21.15.66:0/3314136546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb200cde70 msgr2=0x7feb20156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb24e00640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb200cde70 0x7feb20156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb24e00640 1 -- 172.21.15.66:0/3314136546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb18009b30 con 0x7feb200cda90 2024-03-30T00:40:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb17e00640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb200cde70 0x7feb20156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:40:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb24e00640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb200cda90 0x7feb200c2730 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feb1000d5f0 tx=0x7feb1000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb1d600640 1 -- 172.21.15.66:0/3314136546 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100125f0 con 0x7feb200cda90 2024-03-30T00:40:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb1d600640 1 -- 172.21.15.66:0/3314136546 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb10012750 con 0x7feb200cda90 2024-03-30T00:40:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb1d600640 1 -- 172.21.15.66:0/3314136546 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10012ae0 con 0x7feb200cda90 2024-03-30T00:40:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb27800640 1 -- 172.21.15.66:0/3314136546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb200c2ec0 con 0x7feb200cda90 2024-03-30T00:40:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb27800640 1 -- 172.21.15.66:0/3314136546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb200c3480 con 0x7feb200cda90 2024-03-30T00:40:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb27800640 1 -- 172.21.15.66:0/3314136546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb200ab1b0 con 0x7feb200cda90 2024-03-30T00:40:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.173+0000 7feb1d600640 1 -- 172.21.15.66:0/3314136546 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7feb10012c40 con 0x7feb200cda90 2024-03-30T00:40:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.177+0000 7feb1d600640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feafc066690 0x7feafc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.177+0000 7feb1d600640 1 -- 172.21.15.66:0/3314136546 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7feb10084a00 con 0x7feb200cda90 2024-03-30T00:40:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.177+0000 7feb17e00640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feafc066690 0x7feafc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.177+0000 7feb17e00640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feafc066690 0x7feafc068b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb200c8db0 tx=0x7feb08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.181+0000 7feb1d600640 1 -- 172.21.15.66:0/3314136546 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7feb10051a00 con 0x7feb200cda90 2024-03-30T00:40:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.341+0000 7feb27800640 1 -- 172.21.15.66:0/3314136546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feb20066c30 con 0x7feb200cda90 2024-03-30T00:40:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.341+0000 7feb1d600640 1 -- 172.21.15.66:0/3314136546 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7feb10007380 con 0x7feb200cda90 2024-03-30T00:40:58.351 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:40:58.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 -- 172.21.15.66:0/3314136546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feafc066690 msgr2=0x7feafc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feafc066690 0x7feafc068b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb200c8db0 tx=0x7feb08009040 comp rx=0 tx=0).stop 2024-03-30T00:40:58.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 -- 172.21.15.66:0/3314136546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb200cda90 msgr2=0x7feb200c2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb200cda90 0x7feb200c2730 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feb1000d5f0 tx=0x7feb1000dab0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 -- 172.21.15.66:0/3314136546 shutdown_connections 2024-03-30T00:40:58.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb2015a370 0x7feb200cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb200cde70 0x7feb20156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feafc066690 0x7feafc068b50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 --2- 172.21.15.66:0/3314136546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb200cda90 0x7feb200c2730 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 -- 172.21.15.66:0/3314136546 >> 172.21.15.66:0/3314136546 conn(0x7feb20069f60 msgr2=0x7feb2006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:58.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 -- 172.21.15.66:0/3314136546 shutdown_connections 2024-03-30T00:40:58.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.345+0000 7feb27800640 1 -- 172.21.15.66:0/3314136546 wait complete. 2024-03-30T00:40:58.372 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:40:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.505+0000 7f4895e00640 1 Processor -- start 2024-03-30T00:40:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.505+0000 7f4895e00640 1 -- start start 2024-03-30T00:40:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f489006cb90 0x7f48900d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f48900cdef0 0x7f48900ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48900ce880 con 0x7f489015c6b0 2024-03-30T00:40:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489006b1f0 con 0x7f48900cdef0 2024-03-30T00:40:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489015d250 con 0x7f489006cb90 2024-03-30T00:40:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f48900cdef0 0x7f48900ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f48900cdef0 0x7f48900ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36766/0 (socket says 172.21.15.66:36766) 2024-03-30T00:40:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895400640 1 -- 172.21.15.66:0/1942362958 learned_addr learned my addr 172.21.15.66:0/1942362958 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4887e00640 1 --2- 172.21.15.66:0/1942362958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f489006cb90 0x7f48900d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895400640 1 -- 172.21.15.66:0/1942362958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f489006cb90 msgr2=0x7f48900d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895400640 1 --2- 172.21.15.66:0/1942362958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f489006cb90 0x7f48900d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895400640 1 -- 172.21.15.66:0/1942362958 >> v1:172.21.15.66:6789/0 conn(0x7f489015c6b0 legacy=0x7f489015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.509+0000 7f4895400640 1 -- 172.21.15.66:0/1942362958 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489015d530 con 0x7f48900cdef0 2024-03-30T00:40:58.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895400640 1 --2- 172.21.15.66:0/1942362958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900cdef0 0x7f48900ce340 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4888009e50 tx=0x7f488801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb4e5845dbe8b7e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:58.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f488fe00640 1 -- 172.21.15.66:0/1942362958 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48880a57e0 con 0x7f48900cdef0 2024-03-30T00:40:58.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f488fe00640 1 -- 172.21.15.66:0/1942362958 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48880a5940 con 0x7f48900cdef0 2024-03-30T00:40:58.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f488fe00640 1 -- 172.21.15.66:0/1942362958 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48880a5cd0 con 0x7f48900cdef0 2024-03-30T00:40:58.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 -- 172.21.15.66:0/1942362958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900cdef0 msgr2=0x7f48900ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 --2- 172.21.15.66:0/1942362958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900cdef0 0x7f48900ce340 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4888009e50 tx=0x7f488801ca00 comp rx=0 tx=0).stop 2024-03-30T00:40:58.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 -- 172.21.15.66:0/1942362958 shutdown_connections 2024-03-30T00:40:58.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 --2- 172.21.15.66:0/1942362958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900cdef0 0x7f48900ce340 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 --2- 172.21.15.66:0/1942362958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f489006cb90 0x7f48900d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 -- 172.21.15.66:0/1942362958 >> 172.21.15.66:0/1942362958 conn(0x7f4890069d90 msgr2=0x7f489006a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:58.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 -- 172.21.15.66:0/1942362958 shutdown_connections 2024-03-30T00:40:58.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 -- 172.21.15.66:0/1942362958 wait complete. 2024-03-30T00:40:58.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 Processor -- start 2024-03-30T00:40:58.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 -- start start 2024-03-30T00:40:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f489006cb90 0x7f489015b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900cdef0 0x7f4890154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f489015c6b0 0x7f48900afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4890154690 con 0x7f48900cdef0 2024-03-30T00:40:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48900b30b0 con 0x7f489015c6b0 2024-03-30T00:40:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48900b3390 con 0x7f489006cb90 2024-03-30T00:40:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4887e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900cdef0 0x7f4890154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4887e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900cdef0 0x7f4890154150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36770/0 (socket says 172.21.15.66:36770) 2024-03-30T00:40:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4887e00640 1 -- 172.21.15.66:0/1930243980 learned_addr learned my addr 172.21.15.66:0/1930243980 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:58.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4894a00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f489006cb90 0x7f489015b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.513+0000 7f4895400640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f489015c6b0 0x7f48900afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f4887e00640 1 -- 172.21.15.66:0/1930243980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f489006cb90 msgr2=0x7f489015b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f4887e00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f489006cb90 0x7f489015b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f4887e00640 1 -- 172.21.15.66:0/1930243980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f489015c6b0 msgr2=0x7f48900afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f4887e00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f489015c6b0 0x7f48900afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f4887e00640 1 -- 172.21.15.66:0/1930243980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4888009b30 con 0x7f48900cdef0 2024-03-30T00:40:58.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f4887e00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900cdef0 0x7f4890154150 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f487800d800 tx=0x7f487800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f488d600640 1 -- 172.21.15.66:0/1930243980 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800b6c0 con 0x7f48900cdef0 2024-03-30T00:40:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f488d600640 1 -- 172.21.15.66:0/1930243980 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487800b820 con 0x7f48900cdef0 2024-03-30T00:40:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f488d600640 1 -- 172.21.15.66:0/1930243980 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800bb90 con 0x7f48900cdef0 2024-03-30T00:40:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f4895e00640 1 -- 172.21.15.66:0/1930243980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48900b3670 con 0x7f48900cdef0 2024-03-30T00:40:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f4895e00640 1 -- 172.21.15.66:0/1930243980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48900b3c30 con 0x7f48900cdef0 2024-03-30T00:40:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f4895e00640 1 -- 172.21.15.66:0/1930243980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48900b3dc0 con 0x7f48900cdef0 2024-03-30T00:40:58.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f488d600640 1 -- 172.21.15.66:0/1930243980 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f487800bcf0 con 0x7f48900cdef0 2024-03-30T00:40:58.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f488d600640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c0666c0 0x7f486c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f488d600640 1 -- 172.21.15.66:0/1930243980 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f4878084a20 con 0x7f48900cdef0 2024-03-30T00:40:58.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.517+0000 7f4894a00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c0666c0 0x7f486c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.521+0000 7f4894a00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c0666c0 0x7f486c068b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f48901f8aa0 tx=0x7f4880009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:58.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.521+0000 7f488d600640 1 -- 172.21.15.66:0/1930243980 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f48780519a0 con 0x7f48900cdef0 2024-03-30T00:40:58.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 -- 172.21.15.66:0/1930243980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4890073e00 con 0x7f48900cdef0 2024-03-30T00:40:58.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f488d600640 1 -- 172.21.15.66:0/1930243980 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4878054fc0 con 0x7f48900cdef0 2024-03-30T00:40:58.706 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:40:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 -- 172.21.15.66:0/1930243980 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c0666c0 msgr2=0x7f486c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c0666c0 0x7f486c068b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f48901f8aa0 tx=0x7f4880009040 comp rx=0 tx=0).stop 2024-03-30T00:40:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 -- 172.21.15.66:0/1930243980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900cdef0 msgr2=0x7f4890154150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900cdef0 0x7f4890154150 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f487800d800 tx=0x7f487800dcc0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 -- 172.21.15.66:0/1930243980 shutdown_connections 2024-03-30T00:40:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f489015c6b0 0x7f48900afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c0666c0 0x7f486c068b80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900cdef0 0x7f4890154150 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 --2- 172.21.15.66:0/1930243980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f489006cb90 0x7f489015b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 -- 172.21.15.66:0/1930243980 >> 172.21.15.66:0/1930243980 conn(0x7f4890069d90 msgr2=0x7f4890158fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 -- 172.21.15.66:0/1930243980 shutdown_connections 2024-03-30T00:40:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.697+0000 7f4895e00640 1 -- 172.21.15.66:0/1930243980 wait complete. 2024-03-30T00:40:58.727 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:40:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.857+0000 7f2574000640 1 Processor -- start 2024-03-30T00:40:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2574000640 1 -- start start 2024-03-30T00:40:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2574000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c15a640 0x7f256c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2574000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f256c0cabb0 0x7f256c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2574000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256c06d070 con 0x7f256c0ce520 2024-03-30T00:40:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2574000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c0cbba0 con 0x7f256c0cabb0 2024-03-30T00:40:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2574000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c06e2c0 con 0x7f256c15a640 2024-03-30T00:40:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2570c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f256c0ce520 0x7f256c0ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48546/0 (socket says 172.21.15.66:48546) 2024-03-30T00:40:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2570c00640 1 -- 172.21.15.66:0/730820693 learned_addr learned my addr 172.21.15.66:0/730820693 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:58.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2572000640 1 --2- 172.21.15.66:0/730820693 >> v2:172.21.15.156:3300/0 conn(0x7f256c0cabb0 0x7f256c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2571600640 1 --2- 172.21.15.66:0/730820693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c15a640 0x7f256c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f256be00640 1 -- 172.21.15.66:0/730820693 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115392883 0 0) 0x7f256c06d070 con 0x7f256c0ce520 2024-03-30T00:40:58.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f256be00640 1 -- 172.21.15.66:0/730820693 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2554003530 con 0x7f256c0ce520 2024-03-30T00:40:58.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2572000640 1 -- 172.21.15.66:0/730820693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c15a640 msgr2=0x7f256c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2572000640 1 --2- 172.21.15.66:0/730820693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c15a640 0x7f256c06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2572000640 1 -- 172.21.15.66:0/730820693 >> v1:172.21.15.66:6789/0 conn(0x7f256c0ce520 legacy=0x7f256c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2572000640 1 -- 172.21.15.66:0/730820693 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c0cb8c0 con 0x7f256c0cabb0 2024-03-30T00:40:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.861+0000 7f2571600640 1 --2- 172.21.15.66:0/730820693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c15a640 0x7f256c06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:40:58.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f2572000640 1 --2- 172.21.15.66:0/730820693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f256c0cabb0 0x7f256c0cb040 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2564009d90 tx=0x7f256401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df4fbc76da80d4f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:58.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f256be00640 1 -- 172.21.15.66:0/730820693 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25640a5a00 con 0x7f256c0cabb0 2024-03-30T00:40:58.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f256be00640 1 -- 172.21.15.66:0/730820693 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25640a5b60 con 0x7f256c0cabb0 2024-03-30T00:40:58.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f256be00640 1 -- 172.21.15.66:0/730820693 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25640ad440 con 0x7f256c0cabb0 2024-03-30T00:40:58.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f2574000640 1 -- 172.21.15.66:0/730820693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f256c0cabb0 msgr2=0x7f256c0cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f2574000640 1 --2- 172.21.15.66:0/730820693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f256c0cabb0 0x7f256c0cb040 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2564009d90 tx=0x7f256401c940 comp rx=0 tx=0).stop 2024-03-30T00:40:58.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f2574000640 1 -- 172.21.15.66:0/730820693 shutdown_connections 2024-03-30T00:40:58.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f2574000640 1 --2- 172.21.15.66:0/730820693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f256c0cabb0 0x7f256c0cb040 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f2574000640 1 --2- 172.21.15.66:0/730820693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c15a640 0x7f256c06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f2574000640 1 -- 172.21.15.66:0/730820693 >> 172.21.15.66:0/730820693 conn(0x7f256c069f60 msgr2=0x7f256c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:58.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f2574000640 1 -- 172.21.15.66:0/730820693 shutdown_connections 2024-03-30T00:40:58.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f2574000640 1 -- 172.21.15.66:0/730820693 wait complete. 2024-03-30T00:40:58.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.865+0000 7f2574000640 1 Processor -- start 2024-03-30T00:40:58.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2574000640 1 -- start start 2024-03-30T00:40:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2574000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c0cabb0 0x7f256c1f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2574000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f256c0ce520 0x7f256c0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2574000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f256c15a640 0x7f256c1f8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2574000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c153a70 con 0x7f256c15a640 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2574000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0af1d0 con 0x7f256c0ce520 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2574000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c070580 con 0x7f256c0cabb0 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2572000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f256c15a640 0x7f256c1f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2572000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f256c15a640 0x7f256c1f8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52504/0 (socket says 172.21.15.66:52504) 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2572000640 1 -- 172.21.15.66:0/3007796884 learned_addr learned my addr 172.21.15.66:0/3007796884 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2570c00640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f256c0ce520 0x7f256c0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2571600640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c0cabb0 0x7f256c1f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2572000640 1 -- 172.21.15.66:0/3007796884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c0cabb0 msgr2=0x7f256c1f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2572000640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c0cabb0 0x7f256c1f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2572000640 1 -- 172.21.15.66:0/3007796884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f256c0ce520 msgr2=0x7f256c0aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2572000640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f256c0ce520 0x7f256c0aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2572000640 1 -- 172.21.15.66:0/3007796884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2560009420 con 0x7f256c15a640 2024-03-30T00:40:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2571600640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c0cabb0 0x7f256c1f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:58.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2572000640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f256c15a640 0x7f256c1f8250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2564009d90 tx=0x7f25640196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:58.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2569600640 1 -- 172.21.15.66:0/3007796884 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25640adce0 con 0x7f256c15a640 2024-03-30T00:40:58.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2569600640 1 -- 172.21.15.66:0/3007796884 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25640ade40 con 0x7f256c15a640 2024-03-30T00:40:58.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2569600640 1 -- 172.21.15.66:0/3007796884 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25640b9690 con 0x7f256c15a640 2024-03-30T00:40:58.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2574000640 1 -- 172.21.15.66:0/3007796884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2564009a70 con 0x7f256c15a640 2024-03-30T00:40:58.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2574000640 1 -- 172.21.15.66:0/3007796884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c097270 con 0x7f256c15a640 2024-03-30T00:40:58.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.869+0000 7f2574000640 1 -- 172.21.15.66:0/3007796884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c066c30 con 0x7f256c15a640 2024-03-30T00:40:58.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.873+0000 7f2569600640 1 -- 172.21.15.66:0/3007796884 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f25640b97f0 con 0x7f256c15a640 2024-03-30T00:40:58.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.873+0000 7f2569600640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f254c0666c0 0x7f254c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:58.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.873+0000 7f2569600640 1 -- 172.21.15.66:0/3007796884 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f256411f960 con 0x7f256c15a640 2024-03-30T00:40:58.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.873+0000 7f2571600640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f254c0666c0 0x7f254c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:58.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.873+0000 7f2571600640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f254c0666c0 0x7f254c068b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2560009ca0 tx=0x7f2560008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:58.877+0000 7f2569600640 1 -- 172.21.15.66:0/3007796884 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f25640ec830 con 0x7f256c15a640 2024-03-30T00:40:59.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.049+0000 7f2574000640 1 -- 172.21.15.66:0/3007796884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f256c164440 con 0x7f256c15a640 2024-03-30T00:40:59.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2569600640 1 -- 172.21.15.66:0/3007796884 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2564009450 con 0x7f256c15a640 2024-03-30T00:40:59.061 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:40:59.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 -- 172.21.15.66:0/3007796884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f254c0666c0 msgr2=0x7f254c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f254c0666c0 0x7f254c068b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2560009ca0 tx=0x7f2560008040 comp rx=0 tx=0).stop 2024-03-30T00:40:59.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 -- 172.21.15.66:0/3007796884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f256c15a640 msgr2=0x7f256c1f8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f256c15a640 0x7f256c1f8250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2564009d90 tx=0x7f25640196e0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 -- 172.21.15.66:0/3007796884 shutdown_connections 2024-03-30T00:40:59.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f254c0666c0 0x7f254c068b80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f256c15a640 0x7f256c1f8250 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f256c0ce520 0x7f256c0aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 --2- 172.21.15.66:0/3007796884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f256c0cabb0 0x7f256c1f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 -- 172.21.15.66:0/3007796884 >> 172.21.15.66:0/3007796884 conn(0x7f256c069f60 msgr2=0x7f256c164760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:59.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 -- 172.21.15.66:0/3007796884 shutdown_connections 2024-03-30T00:40:59.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.053+0000 7f2574000640 1 -- 172.21.15.66:0/3007796884 wait complete. 2024-03-30T00:40:59.081 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:40:59.082 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:40:59.091 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:40:59.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.269+0000 7f1797200640 1 Processor -- start 2024-03-30T00:40:59.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.269+0000 7f1797200640 1 -- start start 2024-03-30T00:40:59.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.269+0000 7f1797200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f17900cda90 0x7f17900ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.269+0000 7f1797200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f179015a370 0x7f17900cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.269+0000 7f1797200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f179015a990 con 0x7f17900cde70 2024-03-30T00:40:59.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.269+0000 7f1797200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17900b1b80 con 0x7f17900cda90 2024-03-30T00:40:59.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.269+0000 7f1797200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17900cd3e0 con 0x7f179015a370 2024-03-30T00:40:59.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.269+0000 7f178fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f17900cda90 0x7f17900ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.269+0000 7f178fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f17900cda90 0x7f17900ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52520/0 (socket says 172.21.15.66:52520) 2024-03-30T00:40:59.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.269+0000 7f178fe00640 1 -- 172.21.15.66:0/2594444885 learned_addr learned my addr 172.21.15.66:0/2594444885 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:59.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.273+0000 7f1795200640 1 --2- 172.21.15.66:0/2594444885 >> v2:172.21.15.156:3300/0 conn(0x7f179015a370 0x7f17900cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.273+0000 7f178fe00640 1 -- 172.21.15.66:0/2594444885 >> v2:172.21.15.156:3300/0 conn(0x7f179015a370 msgr2=0x7f17900cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:59.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.273+0000 7f178fe00640 1 --2- 172.21.15.66:0/2594444885 >> v2:172.21.15.156:3300/0 conn(0x7f179015a370 0x7f17900cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.273+0000 7f178fe00640 1 -- 172.21.15.66:0/2594444885 >> v1:172.21.15.66:6789/0 conn(0x7f17900cde70 legacy=0x7f1790159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.273+0000 7f178fe00640 1 -- 172.21.15.66:0/2594444885 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17900b11e0 con 0x7f17900cda90 2024-03-30T00:40:59.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.273+0000 7f178fe00640 1 --2- 172.21.15.66:0/2594444885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17900cda90 0x7f17900ce520 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f17900b3140 tx=0x7f178001e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed197d29146dee31 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:59.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.273+0000 7f178f400640 1 -- 172.21.15.66:0/2594444885 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800ab680 con 0x7f17900cda90 2024-03-30T00:40:59.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.273+0000 7f178f400640 1 -- 172.21.15.66:0/2594444885 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17800ab7e0 con 0x7f17900cda90 2024-03-30T00:40:59.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.273+0000 7f178f400640 1 -- 172.21.15.66:0/2594444885 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800abb70 con 0x7f17900cda90 2024-03-30T00:40:59.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.273+0000 7f1797200640 1 -- 172.21.15.66:0/2594444885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17900cda90 msgr2=0x7f17900ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.273+0000 7f1797200640 1 --2- 172.21.15.66:0/2594444885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17900cda90 0x7f17900ce520 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f17900b3140 tx=0x7f178001e6a0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 -- 172.21.15.66:0/2594444885 shutdown_connections 2024-03-30T00:40:59.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 --2- 172.21.15.66:0/2594444885 >> v2:172.21.15.156:3300/0 conn(0x7f179015a370 0x7f17900cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 --2- 172.21.15.66:0/2594444885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17900cda90 0x7f17900ce520 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 -- 172.21.15.66:0/2594444885 >> 172.21.15.66:0/2594444885 conn(0x7f1790069f60 msgr2=0x7f179006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:59.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 -- 172.21.15.66:0/2594444885 shutdown_connections 2024-03-30T00:40:59.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 -- 172.21.15.66:0/2594444885 wait complete. 2024-03-30T00:40:59.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 Processor -- start 2024-03-30T00:40:59.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 -- start start 2024-03-30T00:40:59.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17900cda90 0x7f17900aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17900cde70 0x7f179009b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f179015a370 0x7f179009bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1790164170 con 0x7f17900cde70 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1790164450 con 0x7f17900cda90 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1797200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1790164730 con 0x7f179015a370 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1787400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17900cde70 0x7f179009b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1787400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17900cde70 0x7f179009b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52524/0 (socket says 172.21.15.66:52524) 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1787400640 1 -- 172.21.15.66:0/2106391473 learned_addr learned my addr 172.21.15.66:0/2106391473 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1795200640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f179015a370 0x7f179009bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f178fe00640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17900cda90 0x7f17900aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1787400640 1 -- 172.21.15.66:0/2106391473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f179015a370 msgr2=0x7f179009bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1787400640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f179015a370 0x7f179009bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1787400640 1 -- 172.21.15.66:0/2106391473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17900cda90 msgr2=0x7f17900aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1787400640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17900cda90 0x7f17900aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1787400640 1 -- 172.21.15.66:0/2106391473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17800094d0 con 0x7f17900cde70 2024-03-30T00:40:59.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f1787400640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17900cde70 0x7f179009b7d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f177800c430 tx=0x7f177800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:59.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.277+0000 7f178cc00640 1 -- 172.21.15.66:0/2106391473 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780052d0 con 0x7f17900cde70 2024-03-30T00:40:59.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.281+0000 7f178cc00640 1 -- 172.21.15.66:0/2106391473 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1778005430 con 0x7f17900cde70 2024-03-30T00:40:59.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.281+0000 7f1797200640 1 -- 172.21.15.66:0/2106391473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1790164a10 con 0x7f17900cde70 2024-03-30T00:40:59.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.281+0000 7f178cc00640 1 -- 172.21.15.66:0/2106391473 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1778007260 con 0x7f17900cde70 2024-03-30T00:40:59.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.281+0000 7f1797200640 1 -- 172.21.15.66:0/2106391473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17900af0a0 con 0x7f17900cde70 2024-03-30T00:40:59.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.281+0000 7f1797200640 1 -- 172.21.15.66:0/2106391473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1790066c30 con 0x7f17900cde70 2024-03-30T00:40:59.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.281+0000 7f178cc00640 1 -- 172.21.15.66:0/2106391473 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f17780073c0 con 0x7f17900cde70 2024-03-30T00:40:59.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.281+0000 7f178cc00640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f176c066690 0x7f176c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.281+0000 7f178cc00640 1 -- 172.21.15.66:0/2106391473 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f1778083910 con 0x7f17900cde70 2024-03-30T00:40:59.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.281+0000 7f178fe00640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f176c066690 0x7f176c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.281+0000 7f178fe00640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f176c066690 0x7f176c068b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f17800b0000 tx=0x7f17800b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:59.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.285+0000 7f178cc00640 1 -- 172.21.15.66:0/2106391473 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1778050920 con 0x7f17900cde70 2024-03-30T00:40:59.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.445+0000 7f1797200640 1 -- 172.21.15.66:0/2106391473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1790164ec0 con 0x7f17900cde70 2024-03-30T00:40:59.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f178cc00640 1 -- 172.21.15.66:0/2106391473 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7f1778053f40 con 0x7f17900cde70 2024-03-30T00:40:59.457 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:40:59.458 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:40:59.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 -- 172.21.15.66:0/2106391473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f176c066690 msgr2=0x7f176c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f176c066690 0x7f176c068b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f17800b0000 tx=0x7f17800b5040 comp rx=0 tx=0).stop 2024-03-30T00:40:59.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 -- 172.21.15.66:0/2106391473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17900cde70 msgr2=0x7f179009b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17900cde70 0x7f179009b7d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f177800c430 tx=0x7f177800c8f0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 -- 172.21.15.66:0/2106391473 shutdown_connections 2024-03-30T00:40:59.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f179015a370 0x7f179009bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f176c066690 0x7f176c068b50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17900cde70 0x7f179009b7d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 --2- 172.21.15.66:0/2106391473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17900cda90 0x7f17900aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 -- 172.21.15.66:0/2106391473 >> 172.21.15.66:0/2106391473 conn(0x7f1790069f60 msgr2=0x7f1790159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:59.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 -- 172.21.15.66:0/2106391473 shutdown_connections 2024-03-30T00:40:59.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.449+0000 7f1797200640 1 -- 172.21.15.66:0/2106391473 wait complete. 2024-03-30T00:40:59.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:40:59.483 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:40:59.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.617+0000 7f0a11400640 1 Processor -- start 2024-03-30T00:40:59.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.617+0000 7f0a11400640 1 -- start start 2024-03-30T00:40:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.617+0000 7f0a11400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c0cc590 0x7f0a0c15c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.617+0000 7f0a11400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c0afdf0 con 0x7f0a0c0b0290 2024-03-30T00:40:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.617+0000 7f0a11400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c0b1010 con 0x7f0a0c160490 2024-03-30T00:40:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.617+0000 7f0a11400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0b0e30 con 0x7f0a0c0cc590 2024-03-30T00:40:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.617+0000 7f0a03e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0a0c0b0290 0x7f0a0c0b0720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48556/0 (socket says 172.21.15.66:48556) 2024-03-30T00:40:59.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.617+0000 7f0a03e00640 1 -- 172.21.15.66:0/529835459 learned_addr learned my addr 172.21.15.66:0/529835459 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:59.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.617+0000 7f0a0b200640 1 --2- 172.21.15.66:0/529835459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c0cc590 0x7f0a0c15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.617+0000 7f0a09e00640 1 -- 172.21.15.66:0/529835459 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308080214 0 0) 0x7f0a0c0afdf0 con 0x7f0a0c0b0290 2024-03-30T00:40:59.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.617+0000 7f0a09e00640 1 -- 172.21.15.66:0/529835459 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f0003530 con 0x7f0a0c0b0290 2024-03-30T00:40:59.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a0b200640 1 -- 172.21.15.66:0/529835459 >> v1:172.21.15.156:6789/0 conn(0x7f0a0c160490 legacy=0x7f0a0c160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a0b200640 1 -- 172.21.15.66:0/529835459 >> v1:172.21.15.66:6789/0 conn(0x7f0a0c0b0290 legacy=0x7f0a0c0b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a0b200640 1 -- 172.21.15.66:0/529835459 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0b12f0 con 0x7f0a0c0cc590 2024-03-30T00:40:59.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a0b200640 1 --2- 172.21.15.66:0/529835459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c0cc590 0x7f0a0c15c200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009dc0 tx=0x7f0a0401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84df73d440df4b1e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:59.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a09e00640 1 -- 172.21.15.66:0/529835459 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a040a5900 con 0x7f0a0c0cc590 2024-03-30T00:40:59.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a09e00640 1 -- 172.21.15.66:0/529835459 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a040a5a60 con 0x7f0a0c0cc590 2024-03-30T00:40:59.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a09e00640 1 -- 172.21.15.66:0/529835459 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a040a5df0 con 0x7f0a0c0cc590 2024-03-30T00:40:59.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a11400640 1 -- 172.21.15.66:0/529835459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c0cc590 msgr2=0x7f0a0c15c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a11400640 1 --2- 172.21.15.66:0/529835459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c0cc590 0x7f0a0c15c200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009dc0 tx=0x7f0a0401c950 comp rx=0 tx=0).stop 2024-03-30T00:40:59.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a11400640 1 -- 172.21.15.66:0/529835459 shutdown_connections 2024-03-30T00:40:59.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a11400640 1 --2- 172.21.15.66:0/529835459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c0cc590 0x7f0a0c15c200 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a11400640 1 -- 172.21.15.66:0/529835459 >> 172.21.15.66:0/529835459 conn(0x7f0a0c099870 msgr2=0x7f0a0c089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:59.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.621+0000 7f0a11400640 1 -- 172.21.15.66:0/529835459 shutdown_connections 2024-03-30T00:40:59.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a11400640 1 -- 172.21.15.66:0/529835459 wait complete. 2024-03-30T00:40:59.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a11400640 1 Processor -- start 2024-03-30T00:40:59.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a11400640 1 -- start start 2024-03-30T00:40:59.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a11400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a0c0b0290 0x7f0a0c0c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a11400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c0cc590 0x7f0a0c0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a11400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c160490 0x7f0a0c15f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a11400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c15cf90 con 0x7f0a0c0b0290 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a11400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c15d830 con 0x7f0a0c0cc590 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a11400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0c6620 con 0x7f0a0c160490 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a0a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a0c0b0290 0x7f0a0c0c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a0a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a0c0b0290 0x7f0a0c0c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52532/0 (socket says 172.21.15.66:52532) 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a0a800640 1 -- 172.21.15.66:0/3718135731 learned_addr learned my addr 172.21.15.66:0/3718135731 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a0b200640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c160490 0x7f0a0c15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a03e00640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c0cc590 0x7f0a0c0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a0a800640 1 -- 172.21.15.66:0/3718135731 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c160490 msgr2=0x7f0a0c15f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a0a800640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c160490 0x7f0a0c15f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a0a800640 1 -- 172.21.15.66:0/3718135731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c0cc590 msgr2=0x7f0a0c0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a0a800640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c0cc590 0x7f0a0c0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a0a800640 1 -- 172.21.15.66:0/3718135731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04009a70 con 0x7f0a0c0b0290 2024-03-30T00:40:59.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a0b200640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c160490 0x7f0a0c15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:59.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a03e00640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c0cc590 0x7f0a0c0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:40:59.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a0a800640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a0c0b0290 0x7f0a0c0c5df0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c370 tx=0x7f09fc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:59.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a02a00640 1 -- 172.21.15.66:0/3718135731 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0052d0 con 0x7f0a0c0b0290 2024-03-30T00:40:59.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a02a00640 1 -- 172.21.15.66:0/3718135731 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc005430 con 0x7f0a0c0b0290 2024-03-30T00:40:59.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a11400640 1 -- 172.21.15.66:0/3718135731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c15d1e0 con 0x7f0a0c0b0290 2024-03-30T00:40:59.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a02a00640 1 -- 172.21.15.66:0/3718135731 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc007260 con 0x7f0a0c0b0290 2024-03-30T00:40:59.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.625+0000 7f0a11400640 1 -- 172.21.15.66:0/3718135731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c0c6330 con 0x7f0a0c0b0290 2024-03-30T00:40:59.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.629+0000 7f0a11400640 1 -- 172.21.15.66:0/3718135731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c066c30 con 0x7f0a0c0b0290 2024-03-30T00:40:59.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.629+0000 7f0a02a00640 1 -- 172.21.15.66:0/3718135731 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f09fc0073c0 con 0x7f0a0c0b0290 2024-03-30T00:40:59.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.629+0000 7f0a02a00640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e8066690 0x7f09e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.629+0000 7f0a02a00640 1 -- 172.21.15.66:0/3718135731 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f09fc083900 con 0x7f0a0c0b0290 2024-03-30T00:40:59.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.629+0000 7f0a03e00640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e8066690 0x7f09e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.629+0000 7f0a03e00640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e8066690 0x7f09e8068b50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c09e360 tx=0x7f09f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:59.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.633+0000 7f0a02a00640 1 -- 172.21.15.66:0/3718135731 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f09fc050870 con 0x7f0a0c0b0290 2024-03-30T00:40:59.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 -- 172.21.15.66:0/3718135731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a0c06af40 con 0x7f0a0c0b0290 2024-03-30T00:40:59.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a02a00640 1 -- 172.21.15.66:0/3718135731 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09fc053e90 con 0x7f0a0c0b0290 2024-03-30T00:40:59.806 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:40:59.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 -- 172.21.15.66:0/3718135731 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e8066690 msgr2=0x7f09e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e8066690 0x7f09e8068b50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c09e360 tx=0x7f09f4009040 comp rx=0 tx=0).stop 2024-03-30T00:40:59.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 -- 172.21.15.66:0/3718135731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a0c0b0290 msgr2=0x7f0a0c0c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a0c0b0290 0x7f0a0c0c5df0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c370 tx=0x7f09fc00c830 comp rx=0 tx=0).stop 2024-03-30T00:40:59.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 -- 172.21.15.66:0/3718135731 shutdown_connections 2024-03-30T00:40:59.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0c160490 0x7f0a0c15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c0cc590 0x7f0a0c0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e8066690 0x7f09e8068b50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 --2- 172.21.15.66:0/3718135731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a0c0b0290 0x7f0a0c0c5df0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 -- 172.21.15.66:0/3718135731 >> 172.21.15.66:0/3718135731 conn(0x7f0a0c099870 msgr2=0x7f0a0c06b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:59.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 -- 172.21.15.66:0/3718135731 shutdown_connections 2024-03-30T00:40:59.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.797+0000 7f0a11400640 1 -- 172.21.15.66:0/3718135731 wait complete. 2024-03-30T00:40:59.827 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:40:59.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.961+0000 7fc06a400640 1 Processor -- start 2024-03-30T00:40:59.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.961+0000 7fc06a400640 1 -- start start 2024-03-30T00:40:59.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.961+0000 7fc06a400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc0640ce820 0x7fc0640cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.961+0000 7fc06a400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06406e7d0 con 0x7fc06415ca10 2024-03-30T00:40:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.961+0000 7fc06a400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06415cdf0 con 0x7fc0640ae4e0 2024-03-30T00:40:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.961+0000 7fc06a400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0640b1f70 con 0x7fc0640ce820 2024-03-30T00:40:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc063400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc06415ca10 0x7fc06406ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48562/0 (socket says 172.21.15.66:48562) 2024-03-30T00:40:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc063400640 1 -- 172.21.15.66:0/1062716883 learned_addr learned my addr 172.21.15.66:0/1062716883 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:40:59.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc063e00640 1 --2- 172.21.15.66:0/1062716883 >> v2:172.21.15.156:3300/0 conn(0x7fc0640ce820 0x7fc0640cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302137957 0 0) 0x7fc06406e7d0 con 0x7fc06415ca10 2024-03-30T00:40:59.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc048003530 con 0x7fc06415ca10 2024-03-30T00:40:59.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 252532607 0 0) 0x7fc048003530 con 0x7fc06415ca10 2024-03-30T00:40:59.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 >> v2:172.21.15.156:3300/0 conn(0x7fc0640ce820 msgr2=0x7fc0640cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:59.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 --2- 172.21.15.66:0/1062716883 >> v2:172.21.15.156:3300/0 conn(0x7fc0640ce820 0x7fc0640cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 >> v1:172.21.15.156:6789/0 conn(0x7fc0640ae4e0 legacy=0x7fc0640cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0640b16d0 con 0x7fc06415ca10 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fc054001680 con 0x7fc06415ca10 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 >> v1:172.21.15.66:6789/0 conn(0x7fc06415ca10 legacy=0x7fc06406ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 --2- 172.21.15.66:0/1062716883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0480039d0 0x7fc048003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 --2- 172.21.15.66:0/1062716883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0480047d0 0x7fc048004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 --2- 172.21.15.66:0/1062716883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc048005190 0x7fc04800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0640b1f70 con 0x7fc0480039d0 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04800c090 con 0x7fc0480047d0 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04800c250 con 0x7fc048005190 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc063400640 1 --2- 172.21.15.66:0/1062716883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0480039d0 0x7fc048003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc05be00640 1 --2- 172.21.15.66:0/1062716883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0480047d0 0x7fc048004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc063e00640 1 --2- 172.21.15.66:0/1062716883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc048005190 0x7fc04800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc063400640 1 -- 172.21.15.66:0/1062716883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc048005190 msgr2=0x7fc04800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc063400640 1 --2- 172.21.15.66:0/1062716883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc048005190 0x7fc04800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc063400640 1 -- 172.21.15.66:0/1062716883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0480047d0 msgr2=0x7fc048004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:40:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc063400640 1 --2- 172.21.15.66:0/1062716883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0480047d0 0x7fc048004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc063400640 1 -- 172.21.15.66:0/1062716883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc048003530 con 0x7fc0480039d0 2024-03-30T00:40:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.965+0000 7fc063e00640 1 -- 172.21.15.66:0/1062716883 reap_dead start 2024-03-30T00:40:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc063400640 1 --2- 172.21.15.66:0/1062716883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0480039d0 0x7fc048003df0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc05400c110 tx=0x7fc05401cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1373377794dca03 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054005ae0 con 0x7fc0480039d0 2024-03-30T00:40:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc054005c40 con 0x7fc0480039d0 2024-03-30T00:40:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc062a00640 1 -- 172.21.15.66:0/1062716883 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054007070 con 0x7fc0480039d0 2024-03-30T00:40:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc06a400640 1 -- 172.21.15.66:0/1062716883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0480039d0 msgr2=0x7fc048003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc06a400640 1 --2- 172.21.15.66:0/1062716883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0480039d0 0x7fc048003df0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc05400c110 tx=0x7fc05401cad0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc06a400640 1 -- 172.21.15.66:0/1062716883 shutdown_connections 2024-03-30T00:40:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc06a400640 1 --2- 172.21.15.66:0/1062716883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0480039d0 0x7fc048003df0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc06a400640 1 -- 172.21.15.66:0/1062716883 >> 172.21.15.66:0/1062716883 conn(0x7fc064098a20 msgr2=0x7fc0640b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:40:59.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc06a400640 1 -- 172.21.15.66:0/1062716883 shutdown_connections 2024-03-30T00:40:59.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc06a400640 1 -- 172.21.15.66:0/1062716883 wait complete. 2024-03-30T00:40:59.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc06a400640 1 Processor -- start 2024-03-30T00:40:59.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc06a400640 1 -- start start 2024-03-30T00:40:59.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.969+0000 7fc06a400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0480039d0 0x7fc064151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc06a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0640ca8f0 0x7fc0640b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc06a400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0640cad70 0x7fc0640b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc06a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06415dc00 con 0x7fc0640ca8f0 2024-03-30T00:40:59.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc06a400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06415d010 con 0x7fc0640cad70 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc06a400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06415d260 con 0x7fc0480039d0 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0640ca8f0 0x7fc0640b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0640ca8f0 0x7fc0640b38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52542/0 (socket says 172.21.15.66:52542) 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05be00640 1 -- 172.21.15.66:0/4253882777 learned_addr learned my addr 172.21.15.66:0/4253882777 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc063e00640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0640cad70 0x7fc0640b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc063400640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0480039d0 0x7fc064151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05be00640 1 -- 172.21.15.66:0/4253882777 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0480039d0 msgr2=0x7fc064151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05be00640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0480039d0 0x7fc064151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05be00640 1 -- 172.21.15.66:0/4253882777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0640cad70 msgr2=0x7fc0640b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05be00640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0640cad70 0x7fc0640b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05be00640 1 -- 172.21.15.66:0/4253882777 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06415d540 con 0x7fc0640ca8f0 2024-03-30T00:40:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05be00640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0640ca8f0 0x7fc0640b38f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00f7a0 tx=0x7fc04c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05b400640 1 -- 172.21.15.66:0/4253882777 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c001350 con 0x7fc0640ca8f0 2024-03-30T00:40:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05b400640 1 -- 172.21.15.66:0/4253882777 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04c0050d0 con 0x7fc0640ca8f0 2024-03-30T00:40:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05b400640 1 -- 172.21.15.66:0/4253882777 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c005480 con 0x7fc0640ca8f0 2024-03-30T00:40:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc06a400640 1 -- 172.21.15.66:0/4253882777 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06415d820 con 0x7fc0640ca8f0 2024-03-30T00:40:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc06a400640 1 -- 172.21.15.66:0/4253882777 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc064155980 con 0x7fc0640ca8f0 2024-03-30T00:40:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc06a400640 1 -- 172.21.15.66:0/4253882777 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06415dd40 con 0x7fc0640ca8f0 2024-03-30T00:40:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.973+0000 7fc05b400640 1 -- 172.21.15.66:0/4253882777 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc04c00d730 con 0x7fc0640ca8f0 2024-03-30T00:40:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.977+0000 7fc05b400640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0400666c0 0x7fc040068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:40:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.977+0000 7fc05b400640 1 -- 172.21.15.66:0/4253882777 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fc04c084310 con 0x7fc0640ca8f0 2024-03-30T00:40:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.977+0000 7fc063400640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0400666c0 0x7fc040068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:40:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.977+0000 7fc063400640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0400666c0 0x7fc040068b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0540ab000 tx=0x7fc0540a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:40:59.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:40:59.981+0000 7fc05b400640 1 -- 172.21.15.66:0/4253882777 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc06415dd40 con 0x7fc0640ca8f0 2024-03-30T00:41:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.137+0000 7fc06a400640 1 -- 172.21.15.66:0/4253882777 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc06415dd40 con 0x7fc0640ca8f0 2024-03-30T00:41:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.137+0000 7fc05b400640 1 -- 172.21.15.66:0/4253882777 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc06415dd40 con 0x7fc0640ca8f0 2024-03-30T00:41:00.148 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:41:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 -- 172.21.15.66:0/4253882777 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0400666c0 msgr2=0x7fc040068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0400666c0 0x7fc040068b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0540ab000 tx=0x7fc0540a7070 comp rx=0 tx=0).stop 2024-03-30T00:41:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 -- 172.21.15.66:0/4253882777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0640ca8f0 msgr2=0x7fc0640b38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0640ca8f0 0x7fc0640b38f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00f7a0 tx=0x7fc04c00fc60 comp rx=0 tx=0).stop 2024-03-30T00:41:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 -- 172.21.15.66:0/4253882777 shutdown_connections 2024-03-30T00:41:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0640cad70 0x7fc0640b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0400666c0 0x7fc040068b80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0640ca8f0 0x7fc0640b38f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 --2- 172.21.15.66:0/4253882777 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0480039d0 0x7fc064151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 -- 172.21.15.66:0/4253882777 >> 172.21.15.66:0/4253882777 conn(0x7fc064098a20 msgr2=0x7fc0640ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 -- 172.21.15.66:0/4253882777 shutdown_connections 2024-03-30T00:41:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.141+0000 7fc06a400640 1 -- 172.21.15.66:0/4253882777 wait complete. 2024-03-30T00:41:00.170 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:41:00.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.301+0000 7fdf78c00640 1 Processor -- start 2024-03-30T00:41:00.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.301+0000 7fdf78c00640 1 -- start start 2024-03-30T00:41:00.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.305+0000 7fdf78c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdf7415ca10 0x7fdf7406ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.305+0000 7fdf78c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7406e7d0 con 0x7fdf740ae4e0 2024-03-30T00:41:00.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.305+0000 7fdf78c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7406f1a0 con 0x7fdf740ce820 2024-03-30T00:41:00.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.305+0000 7fdf78c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf740b1f70 con 0x7fdf7415ca10 2024-03-30T00:41:00.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.305+0000 7fdf71e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdf7415ca10 0x7fdf7406ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:00.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.305+0000 7fdf71e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdf7415ca10 0x7fdf7406ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52548/0 (socket says 172.21.15.66:52548) 2024-03-30T00:41:00.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.305+0000 7fdf71e00640 1 -- 172.21.15.66:0/579328497 learned_addr learned my addr 172.21.15.66:0/579328497 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:00.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.305+0000 7fdf71e00640 1 -- 172.21.15.66:0/579328497 >> v1:172.21.15.156:6789/0 conn(0x7fdf740ce820 legacy=0x7fdf74160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:00.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.305+0000 7fdf71e00640 1 -- 172.21.15.66:0/579328497 >> v1:172.21.15.66:6789/0 conn(0x7fdf740ae4e0 legacy=0x7fdf740cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.305+0000 7fdf71e00640 1 -- 172.21.15.66:0/579328497 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7415cdf0 con 0x7fdf7415ca10 2024-03-30T00:41:00.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf71e00640 1 --2- 172.21.15.66:0/579328497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7415ca10 0x7fdf7406ec60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009870 tx=0x7fdf6401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f26e0c10643c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:00.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf71400640 1 -- 172.21.15.66:0/579328497 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640a9610 con 0x7fdf7415ca10 2024-03-30T00:41:00.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf71400640 1 -- 172.21.15.66:0/579328497 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf640a9770 con 0x7fdf7415ca10 2024-03-30T00:41:00.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf71400640 1 -- 172.21.15.66:0/579328497 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640a9b00 con 0x7fdf7415ca10 2024-03-30T00:41:00.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 -- 172.21.15.66:0/579328497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7415ca10 msgr2=0x7fdf7406ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 --2- 172.21.15.66:0/579328497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7415ca10 0x7fdf7406ec60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009870 tx=0x7fdf6401c530 comp rx=0 tx=0).stop 2024-03-30T00:41:00.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 -- 172.21.15.66:0/579328497 shutdown_connections 2024-03-30T00:41:00.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 --2- 172.21.15.66:0/579328497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7415ca10 0x7fdf7406ec60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 -- 172.21.15.66:0/579328497 >> 172.21.15.66:0/579328497 conn(0x7fdf74098a20 msgr2=0x7fdf740b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:00.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 -- 172.21.15.66:0/579328497 shutdown_connections 2024-03-30T00:41:00.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 -- 172.21.15.66:0/579328497 wait complete. 2024-03-30T00:41:00.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 Processor -- start 2024-03-30T00:41:00.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 -- start start 2024-03-30T00:41:00.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf740ae4e0 0x7fdf740b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf740ce820 0x7fdf740c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7415ca10 0x7fdf740c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7415b450 con 0x7fdf740ce820 2024-03-30T00:41:00.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf740b4fe0 con 0x7fdf7415ca10 2024-03-30T00:41:00.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf78c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7415ae00 con 0x7fdf740ae4e0 2024-03-30T00:41:00.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf69400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf740ce820 0x7fdf740c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:00.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf69400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf740ce820 0x7fdf740c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52552/0 (socket says 172.21.15.66:52552) 2024-03-30T00:41:00.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf69400640 1 -- 172.21.15.66:0/2311543069 learned_addr learned my addr 172.21.15.66:0/2311543069 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:00.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.309+0000 7fdf71e00640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf740ae4e0 0x7fdf740b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf72800640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7415ca10 0x7fdf740c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf69400640 1 -- 172.21.15.66:0/2311543069 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf740ae4e0 msgr2=0x7fdf740b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf69400640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf740ae4e0 0x7fdf740b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf69400640 1 -- 172.21.15.66:0/2311543069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7415ca10 msgr2=0x7fdf740c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf69400640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7415ca10 0x7fdf740c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf69400640 1 -- 172.21.15.66:0/2311543069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640094d0 con 0x7fdf740ce820 2024-03-30T00:41:00.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf69400640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf740ce820 0x7fdf740c9a90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c3c0 tx=0x7fdf5c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:00.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf6aa00640 1 -- 172.21.15.66:0/2311543069 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0052d0 con 0x7fdf740ce820 2024-03-30T00:41:00.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf6aa00640 1 -- 172.21.15.66:0/2311543069 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c005430 con 0x7fdf740ce820 2024-03-30T00:41:00.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf6aa00640 1 -- 172.21.15.66:0/2311543069 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c003840 con 0x7fdf740ce820 2024-03-30T00:41:00.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf78c00640 1 -- 172.21.15.66:0/2311543069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7415b0e0 con 0x7fdf740ce820 2024-03-30T00:41:00.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf78c00640 1 -- 172.21.15.66:0/2311543069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7415b8e0 con 0x7fdf740ce820 2024-03-30T00:41:00.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf78c00640 1 -- 172.21.15.66:0/2311543069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7415b600 con 0x7fdf740ce820 2024-03-30T00:41:00.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf6aa00640 1 -- 172.21.15.66:0/2311543069 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdf5c0039a0 con 0x7fdf740ce820 2024-03-30T00:41:00.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf6aa00640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf500666c0 0x7fdf50068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf6aa00640 1 -- 172.21.15.66:0/2311543069 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fdf5c0839c0 con 0x7fdf740ce820 2024-03-30T00:41:00.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.313+0000 7fdf71e00640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf500666c0 0x7fdf50068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:00.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.317+0000 7fdf71e00640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf500666c0 0x7fdf50068b80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdf640ae000 tx=0x7fdf640b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:00.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.317+0000 7fdf6aa00640 1 -- 172.21.15.66:0/2311543069 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdf7415b600 con 0x7fdf740ce820 2024-03-30T00:41:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.477+0000 7fdf78c00640 1 -- 172.21.15.66:0/2311543069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf74066c30 con 0x7fdf740ce820 2024-03-30T00:41:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf6aa00640 1 -- 172.21.15.66:0/2311543069 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf5c017020 con 0x7fdf740ce820 2024-03-30T00:41:00.489 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:41:00.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 -- 172.21.15.66:0/2311543069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf500666c0 msgr2=0x7fdf50068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf500666c0 0x7fdf50068b80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdf640ae000 tx=0x7fdf640b3040 comp rx=0 tx=0).stop 2024-03-30T00:41:00.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 -- 172.21.15.66:0/2311543069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf740ce820 msgr2=0x7fdf740c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf740ce820 0x7fdf740c9a90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c3c0 tx=0x7fdf5c00c880 comp rx=0 tx=0).stop 2024-03-30T00:41:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 -- 172.21.15.66:0/2311543069 shutdown_connections 2024-03-30T00:41:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7415ca10 0x7fdf740c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf500666c0 0x7fdf50068b80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf740ce820 0x7fdf740c9a90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 --2- 172.21.15.66:0/2311543069 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf740ae4e0 0x7fdf740b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 -- 172.21.15.66:0/2311543069 >> 172.21.15.66:0/2311543069 conn(0x7fdf74098a20 msgr2=0x7fdf7415bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 -- 172.21.15.66:0/2311543069 shutdown_connections 2024-03-30T00:41:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.481+0000 7fdf78c00640 1 -- 172.21.15.66:0/2311543069 wait complete. 2024-03-30T00:41:00.510 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:41:00.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4b6a00640 1 Processor -- start 2024-03-30T00:41:00.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4b6a00640 1 -- start start 2024-03-30T00:41:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4b6a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b015ca80 0x7fb4b006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4b6a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb4b00cf2f0 0x7fb4b006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4b6a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b00cb0e0 con 0x7fb4b006bb30 2024-03-30T00:41:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4b6a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b00cf7f0 con 0x7fb4b00cf2f0 2024-03-30T00:41:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4b6a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00cfad0 con 0x7fb4b015ca80 2024-03-30T00:41:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4b4a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb4b006bb30 0x7fb4b0098a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48588/0 (socket says 172.21.15.66:48588) 2024-03-30T00:41:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4b4a00640 1 -- 172.21.15.66:0/720413793 learned_addr learned my addr 172.21.15.66:0/720413793 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4afe00640 1 --2- 172.21.15.66:0/720413793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b015ca80 0x7fb4b006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4af400640 1 --2- 172.21.15.66:0/720413793 >> v2:172.21.15.156:3300/0 conn(0x7fb4b00cf2f0 0x7fb4b006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4aea00640 1 -- 172.21.15.66:0/720413793 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680582004 0 0) 0x7fb4b00cb0e0 con 0x7fb4b006bb30 2024-03-30T00:41:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4aea00640 1 -- 172.21.15.66:0/720413793 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb494003530 con 0x7fb4b006bb30 2024-03-30T00:41:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4afe00640 1 -- 172.21.15.66:0/720413793 >> v2:172.21.15.156:3300/0 conn(0x7fb4b00cf2f0 msgr2=0x7fb4b006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4afe00640 1 --2- 172.21.15.66:0/720413793 >> v2:172.21.15.156:3300/0 conn(0x7fb4b00cf2f0 0x7fb4b006b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4afe00640 1 -- 172.21.15.66:0/720413793 >> v1:172.21.15.66:6789/0 conn(0x7fb4b006bb30 legacy=0x7fb4b0098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4aea00640 1 -- 172.21.15.66:0/720413793 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 451183844 0 0) 0x7fb494003530 con 0x7fb4b006bb30 2024-03-30T00:41:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.645+0000 7fb4afe00640 1 -- 172.21.15.66:0/720413793 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b00cad70 con 0x7fb4b015ca80 2024-03-30T00:41:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4afe00640 1 --2- 172.21.15.66:0/720413793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b015ca80 0x7fb4b006b0b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00097c0 tx=0x7fb4a001ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89769eb36f8212ef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4aea00640 1 -- 172.21.15.66:0/720413793 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8001a00 con 0x7fb4b015ca80 2024-03-30T00:41:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4aea00640 1 -- 172.21.15.66:0/720413793 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a00a85f0 con 0x7fb4b015ca80 2024-03-30T00:41:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4aea00640 1 -- 172.21.15.66:0/720413793 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00a8980 con 0x7fb4b015ca80 2024-03-30T00:41:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4b6a00640 1 -- 172.21.15.66:0/720413793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b015ca80 msgr2=0x7fb4b006b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4b6a00640 1 --2- 172.21.15.66:0/720413793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b015ca80 0x7fb4b006b0b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00097c0 tx=0x7fb4a001ebb0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4b6a00640 1 -- 172.21.15.66:0/720413793 shutdown_connections 2024-03-30T00:41:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4b6a00640 1 --2- 172.21.15.66:0/720413793 >> v2:172.21.15.156:3300/0 conn(0x7fb4b00cf2f0 0x7fb4b006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4b6a00640 1 --2- 172.21.15.66:0/720413793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b015ca80 0x7fb4b006b0b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4b6a00640 1 -- 172.21.15.66:0/720413793 >> 172.21.15.66:0/720413793 conn(0x7fb4b00b2090 msgr2=0x7fb4b00b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:00.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4b6a00640 1 -- 172.21.15.66:0/720413793 shutdown_connections 2024-03-30T00:41:00.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4b6a00640 1 -- 172.21.15.66:0/720413793 wait complete. 2024-03-30T00:41:00.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.649+0000 7fb4b6a00640 1 Processor -- start 2024-03-30T00:41:00.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4b6a00640 1 -- start start 2024-03-30T00:41:00.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4b6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b006bb30 0x7fb4b00afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4b6a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b00cf2f0 0x7fb4b00c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4b6a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b015ca80 0x7fb4b014a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4b6a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00b4360 con 0x7fb4b006bb30 2024-03-30T00:41:00.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4b6a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00b1000 con 0x7fb4b00cf2f0 2024-03-30T00:41:00.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4b6a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00b0480 con 0x7fb4b015ca80 2024-03-30T00:41:00.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b006bb30 0x7fb4b00afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:00.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b006bb30 0x7fb4b00afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52562/0 (socket says 172.21.15.66:52562) 2024-03-30T00:41:00.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4afe00640 1 -- 172.21.15.66:0/201066558 learned_addr learned my addr 172.21.15.66:0/201066558 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:00.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4afe00640 1 -- 172.21.15.66:0/201066558 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b015ca80 msgr2=0x7fb4b014a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4afe00640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b015ca80 0x7fb4b014a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4afe00640 1 -- 172.21.15.66:0/201066558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b00cf2f0 msgr2=0x7fb4b00c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4afe00640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b00cf2f0 0x7fb4b00c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4afe00640 1 -- 172.21.15.66:0/201066558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a0009420 con 0x7fb4b006bb30 2024-03-30T00:41:00.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4afe00640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b006bb30 0x7fb4b00afb70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00097c0 tx=0x7fb4a001b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:00.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4a7e00640 1 -- 172.21.15.66:0/201066558 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00b0850 con 0x7fb4b006bb30 2024-03-30T00:41:00.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4a7e00640 1 -- 172.21.15.66:0/201066558 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a00b09b0 con 0x7fb4b006bb30 2024-03-30T00:41:00.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4a7e00640 1 -- 172.21.15.66:0/201066558 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00b0d60 con 0x7fb4b006bb30 2024-03-30T00:41:00.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4b6a00640 1 -- 172.21.15.66:0/201066558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b00b0760 con 0x7fb4b006bb30 2024-03-30T00:41:00.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4b6a00640 1 -- 172.21.15.66:0/201066558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b00b0d20 con 0x7fb4b006bb30 2024-03-30T00:41:00.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.653+0000 7fb4b6a00640 1 -- 172.21.15.66:0/201066558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b00ad200 con 0x7fb4b006bb30 2024-03-30T00:41:00.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.657+0000 7fb4a7e00640 1 -- 172.21.15.66:0/201066558 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb4a00b7040 con 0x7fb4b006bb30 2024-03-30T00:41:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.657+0000 7fb4a7e00640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb490066690 0x7fb490068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.657+0000 7fb4a7e00640 1 -- 172.21.15.66:0/201066558 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fb4a0121ea0 con 0x7fb4b006bb30 2024-03-30T00:41:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.657+0000 7fb4af400640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb490066690 0x7fb490068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.657+0000 7fb4af400640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb490066690 0x7fb490068b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb49c006aa0 tx=0x7fb49c006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.661+0000 7fb4a7e00640 1 -- 172.21.15.66:0/201066558 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb4a00eefd0 con 0x7fb4b006bb30 2024-03-30T00:41:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.821+0000 7fb4b6a00640 1 -- 172.21.15.66:0/201066558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb4b0066c30 con 0x7fb4b006bb30 2024-03-30T00:41:00.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.821+0000 7fb4a7e00640 1 -- 172.21.15.66:0/201066558 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb4a00b50a0 con 0x7fb4b006bb30 2024-03-30T00:41:00.831 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:41:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 -- 172.21.15.66:0/201066558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb490066690 msgr2=0x7fb490068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb490066690 0x7fb490068b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb49c006aa0 tx=0x7fb49c006a30 comp rx=0 tx=0).stop 2024-03-30T00:41:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 -- 172.21.15.66:0/201066558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b006bb30 msgr2=0x7fb4b00afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b006bb30 0x7fb4b00afb70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00097c0 tx=0x7fb4a001b950 comp rx=0 tx=0).stop 2024-03-30T00:41:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 -- 172.21.15.66:0/201066558 shutdown_connections 2024-03-30T00:41:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b015ca80 0x7fb4b014a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b00cf2f0 0x7fb4b00c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb490066690 0x7fb490068b50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 --2- 172.21.15.66:0/201066558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b006bb30 0x7fb4b00afb70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:00.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 -- 172.21.15.66:0/201066558 >> 172.21.15.66:0/201066558 conn(0x7fb4b00b2090 msgr2=0x7fb4b00c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:00.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 -- 172.21.15.66:0/201066558 shutdown_connections 2024-03-30T00:41:00.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.825+0000 7fb4b6a00640 1 -- 172.21.15.66:0/201066558 wait complete. 2024-03-30T00:41:00.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:41:00.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc09da00640 1 Processor -- start 2024-03-30T00:41:00.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc09da00640 1 -- start start 2024-03-30T00:41:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc09da00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cec30 0x7fc098159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc09da00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0980cd3e0 con 0x7fc0980cda90 2024-03-30T00:41:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc09da00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09815a990 con 0x7fc0980cde70 2024-03-30T00:41:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc09da00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980b1b80 con 0x7fc0980cec30 2024-03-30T00:41:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc096a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc0980cda90 0x7fc0980ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48592/0 (socket says 172.21.15.66:48592) 2024-03-30T00:41:00.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc096a00640 1 -- 172.21.15.66:0/3884285773 learned_addr learned my addr 172.21.15.66:0/3884285773 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:00.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc08fe00640 1 --2- 172.21.15.66:0/3884285773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cec30 0x7fc098159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:00.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc096000640 1 -- 172.21.15.66:0/3884285773 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735716124 0 0) 0x7fc0980cd3e0 con 0x7fc0980cda90 2024-03-30T00:41:00.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc096000640 1 -- 172.21.15.66:0/3884285773 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc07c003530 con 0x7fc0980cda90 2024-03-30T00:41:00.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc096000640 1 -- 172.21.15.66:0/3884285773 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605658876 0 0) 0x7fc09815a990 con 0x7fc0980cde70 2024-03-30T00:41:00.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc08fe00640 1 -- 172.21.15.66:0/3884285773 >> v1:172.21.15.156:6789/0 conn(0x7fc0980cde70 legacy=0x7fc09815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc08fe00640 1 -- 172.21.15.66:0/3884285773 >> v1:172.21.15.66:6789/0 conn(0x7fc0980cda90 legacy=0x7fc0980ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:00.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.985+0000 7fc08fe00640 1 -- 172.21.15.66:0/3884285773 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980b11e0 con 0x7fc0980cec30 2024-03-30T00:41:00.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.989+0000 7fc08fe00640 1 --2- 172.21.15.66:0/3884285773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cec30 0x7fc098159c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc0800097f0 tx=0x7fc08001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4325c1a3bb2a321 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:00.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.989+0000 7fc096000640 1 -- 172.21.15.66:0/3884285773 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800ab650 con 0x7fc0980cec30 2024-03-30T00:41:00.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.989+0000 7fc096000640 1 -- 172.21.15.66:0/3884285773 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0800ab7b0 con 0x7fc0980cec30 2024-03-30T00:41:00.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.989+0000 7fc096000640 1 -- 172.21.15.66:0/3884285773 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800abb40 con 0x7fc0980cec30 2024-03-30T00:41:01.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.989+0000 7fc09da00640 1 -- 172.21.15.66:0/3884285773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cec30 msgr2=0x7fc098159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.989+0000 7fc09da00640 1 --2- 172.21.15.66:0/3884285773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cec30 0x7fc098159c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc0800097f0 tx=0x7fc08001c450 comp rx=0 tx=0).stop 2024-03-30T00:41:01.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.989+0000 7fc09da00640 1 -- 172.21.15.66:0/3884285773 shutdown_connections 2024-03-30T00:41:01.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.989+0000 7fc09da00640 1 --2- 172.21.15.66:0/3884285773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cec30 0x7fc098159c60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.989+0000 7fc09da00640 1 -- 172.21.15.66:0/3884285773 >> 172.21.15.66:0/3884285773 conn(0x7fc098069f60 msgr2=0x7fc09806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:01.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.989+0000 7fc09da00640 1 -- 172.21.15.66:0/3884285773 shutdown_connections 2024-03-30T00:41:01.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.989+0000 7fc09da00640 1 -- 172.21.15.66:0/3884285773 wait complete. 2024-03-30T00:41:01.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc09da00640 1 Processor -- start 2024-03-30T00:41:01.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc09da00640 1 -- start start 2024-03-30T00:41:01.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc09da00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cda90 0x7fc098163a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc09da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0980cde70 0x7fc0980acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc09da00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0980cec30 0x7fc0981579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc09da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc098163f90 con 0x7fc0980cde70 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc09da00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980abbe0 con 0x7fc0980cec30 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc09da00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980abec0 con 0x7fc0980cda90 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0980cde70 0x7fc0980acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0980cde70 0x7fc0980acc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52572/0 (socket says 172.21.15.66:52572) 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08fe00640 1 -- 172.21.15.66:0/4113641389 learned_addr learned my addr 172.21.15.66:0/4113641389 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc096a00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cda90 0x7fc098163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc097400640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0980cec30 0x7fc0981579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08fe00640 1 -- 172.21.15.66:0/4113641389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cda90 msgr2=0x7fc098163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08fe00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cda90 0x7fc098163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08fe00640 1 -- 172.21.15.66:0/4113641389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0980cec30 msgr2=0x7fc0981579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08fe00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0980cec30 0x7fc0981579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08fe00640 1 -- 172.21.15.66:0/4113641389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080009420 con 0x7fc0980cde70 2024-03-30T00:41:01.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc096a00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cda90 0x7fc098163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:01.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc097400640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0980cec30 0x7fc0981579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:01.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08fe00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0980cde70 0x7fc0980acc80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc080019060 tx=0x7fc0800191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:01.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08ea00640 1 -- 172.21.15.66:0/4113641389 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080004220 con 0x7fc0980cde70 2024-03-30T00:41:01.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08ea00640 1 -- 172.21.15.66:0/4113641389 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc080004380 con 0x7fc0980cde70 2024-03-30T00:41:01.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc08ea00640 1 -- 172.21.15.66:0/4113641389 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800abb40 con 0x7fc0980cde70 2024-03-30T00:41:01.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc09da00640 1 -- 172.21.15.66:0/4113641389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0980ac1a0 con 0x7fc0980cde70 2024-03-30T00:41:01.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.993+0000 7fc09da00640 1 -- 172.21.15.66:0/4113641389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0980ac760 con 0x7fc0980cde70 2024-03-30T00:41:01.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.997+0000 7fc09da00640 1 -- 172.21.15.66:0/4113641389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0980ac950 con 0x7fc0980cde70 2024-03-30T00:41:01.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.997+0000 7fc08ea00640 1 -- 172.21.15.66:0/4113641389 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc0800bc070 con 0x7fc0980cde70 2024-03-30T00:41:01.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.997+0000 7fc08ea00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc074066690 0x7fc074068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.997+0000 7fc08ea00640 1 -- 172.21.15.66:0/4113641389 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fc08011e640 con 0x7fc0980cde70 2024-03-30T00:41:01.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.997+0000 7fc096a00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc074066690 0x7fc074068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:00.997+0000 7fc096a00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc074066690 0x7fc074068b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc088005890 tx=0x7fc088009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.001+0000 7fc08ea00640 1 -- 172.21.15.66:0/4113641389 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc0800eb510 con 0x7fc0980cde70 2024-03-30T00:41:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 -- 172.21.15.66:0/4113641389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc098066c30 con 0x7fc0980cde70 2024-03-30T00:41:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc08ea00640 1 -- 172.21.15.66:0/4113641389 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc0800044f0 con 0x7fc0980cde70 2024-03-30T00:41:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:41:01.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 -- 172.21.15.66:0/4113641389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc074066690 msgr2=0x7fc074068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc074066690 0x7fc074068b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc088005890 tx=0x7fc088009040 comp rx=0 tx=0).stop 2024-03-30T00:41:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 -- 172.21.15.66:0/4113641389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0980cde70 msgr2=0x7fc0980acc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0980cde70 0x7fc0980acc80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc080019060 tx=0x7fc0800191f0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 -- 172.21.15.66:0/4113641389 shutdown_connections 2024-03-30T00:41:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0980cec30 0x7fc0981579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc074066690 0x7fc074068b50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0980cde70 0x7fc0980acc80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 --2- 172.21.15.66:0/4113641389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0980cda90 0x7fc098163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 -- 172.21.15.66:0/4113641389 >> 172.21.15.66:0/4113641389 conn(0x7fc098069f60 msgr2=0x7fc09815da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 -- 172.21.15.66:0/4113641389 shutdown_connections 2024-03-30T00:41:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.173+0000 7fc09da00640 1 -- 172.21.15.66:0/4113641389 wait complete. 2024-03-30T00:41:01.203 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:41:01.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:01.203 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-03-30T00:41:01.223 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:41:01.223 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:41:01.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:01.224 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-30T00:41:01.274 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:41:01.274 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:41:01.284 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:41:01.284 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:41:01.336 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:01.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.465+0000 7ff3b1200640 1 Processor -- start 2024-03-30T00:41:01.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.465+0000 7ff3b1200640 1 -- start start 2024-03-30T00:41:01.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3b1200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff3ac15a640 0x7ff3ac06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3b1200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac0cb2c0 0x7ff3ac0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3b1200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac06d070 con 0x7ff3ac0ce520 2024-03-30T00:41:01.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3b1200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac0cbba0 con 0x7ff3ac15a640 2024-03-30T00:41:01.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3b1200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0cee90 con 0x7ff3ac0cb2c0 2024-03-30T00:41:01.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a1400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff3ac0ce520 0x7ff3ac0cabb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48604/0 (socket says 172.21.15.66:48604) 2024-03-30T00:41:01.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a1400640 1 -- 172.21.15.66:0/1303733608 learned_addr learned my addr 172.21.15.66:0/1303733608 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3aa800640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac0cb2c0 0x7ff3ac0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9e00640 1 --2- 172.21.15.66:0/1303733608 >> v2:172.21.15.156:3300/0 conn(0x7ff3ac15a640 0x7ff3ac06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557234404 0 0) 0x7ff3ac06d070 con 0x7ff3ac0ce520 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff390003530 con 0x7ff3ac0ce520 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3261416849 0 0) 0x7ff390003530 con 0x7ff3ac0ce520 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac0cb2c0 msgr2=0x7ff3ac0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac0cb2c0 0x7ff3ac0ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 >> v2:172.21.15.156:3300/0 conn(0x7ff3ac15a640 msgr2=0x7ff3ac06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 --2- 172.21.15.66:0/1303733608 >> v2:172.21.15.156:3300/0 conn(0x7ff3ac15a640 0x7ff3ac06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac06e2c0 con 0x7ff3ac0ce520 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7ff394001680 con 0x7ff3ac0ce520 2024-03-30T00:41:01.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 >> v1:172.21.15.66:6789/0 conn(0x7ff3ac0ce520 legacy=0x7ff3ac0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff390003980 0x7ff390003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 0x7ff390004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac0cb2c0 0x7ff390005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0cbba0 con 0x7ff3900047d0 2024-03-30T00:41:01.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0cee90 con 0x7ff390003980 2024-03-30T00:41:01.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39000b990 con 0x7ff3ac0cb2c0 2024-03-30T00:41:01.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a1400640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 0x7ff390004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a9e00640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff390003980 0x7ff390003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3aa800640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac0cb2c0 0x7ff390005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a1400640 1 -- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac0cb2c0 msgr2=0x7ff390005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:01.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a1400640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac0cb2c0 0x7ff390005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a1400640 1 -- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff390003980 msgr2=0x7ff390003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:01.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a1400640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff390003980 0x7ff390003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3a1400640 1 -- 172.21.15.66:0/1303733608 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff390003530 con 0x7ff3900047d0 2024-03-30T00:41:01.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.469+0000 7ff3aa800640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac0cb2c0 0x7ff390005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:01.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3a1400640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 0x7ff390004c50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff39400c190 tx=0x7ff39401f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5dcc4a1ea89054 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:01.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940a8c20 con 0x7ff3900047d0 2024-03-30T00:41:01.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3940a8da0 con 0x7ff3900047d0 2024-03-30T00:41:01.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3a9400640 1 -- 172.21.15.66:0/1303733608 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff394005b20 con 0x7ff3900047d0 2024-03-30T00:41:01.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3b1200640 1 -- 172.21.15.66:0/1303733608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 msgr2=0x7ff390004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3b1200640 1 --2- 172.21.15.66:0/1303733608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 0x7ff390004c50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff39400c190 tx=0x7ff39401f390 comp rx=0 tx=0).stop 2024-03-30T00:41:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3aa800640 1 -- 172.21.15.66:0/1303733608 reap_dead start 2024-03-30T00:41:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3b1200640 1 -- 172.21.15.66:0/1303733608 shutdown_connections 2024-03-30T00:41:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3b1200640 1 -- 172.21.15.66:0/1303733608 >> 172.21.15.66:0/1303733608 conn(0x7ff3ac069f60 msgr2=0x7ff3ac06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3b1200640 1 -- 172.21.15.66:0/1303733608 shutdown_connections 2024-03-30T00:41:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3b1200640 1 -- 172.21.15.66:0/1303733608 wait complete. 2024-03-30T00:41:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3b1200640 1 Processor -- start 2024-03-30T00:41:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.473+0000 7ff3b1200640 1 -- start start 2024-03-30T00:41:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3b1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 0x7ff3ac0c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3b1200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3ac149e20 0x7ff3ac1f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3b1200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac15d4d0 0x7ff3ac0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3b1200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1f64a0 con 0x7ff3900047d0 2024-03-30T00:41:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3b1200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0c8e00 con 0x7ff3ac149e20 2024-03-30T00:41:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3b1200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0c90e0 con 0x7ff3ac15d4d0 2024-03-30T00:41:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 0x7ff3ac0c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 0x7ff3ac0c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52576/0 (socket says 172.21.15.66:52576) 2024-03-30T00:41:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a9e00640 1 -- 172.21.15.66:0/1326446124 learned_addr learned my addr 172.21.15.66:0/1326446124 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a1400640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3ac149e20 0x7ff3ac1f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3aa800640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac15d4d0 0x7ff3ac0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a9e00640 1 -- 172.21.15.66:0/1326446124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac15d4d0 msgr2=0x7ff3ac0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a9e00640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac15d4d0 0x7ff3ac0b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a9e00640 1 -- 172.21.15.66:0/1326446124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3ac149e20 msgr2=0x7ff3ac1f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a9e00640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3ac149e20 0x7ff3ac1f5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a9e00640 1 -- 172.21.15.66:0/1326446124 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0c93c0 con 0x7ff3900047d0 2024-03-30T00:41:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a1400640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3ac149e20 0x7ff3ac1f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a9e00640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 0x7ff3ac0c87b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009f90 tx=0x7ff39c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a2a00640 1 -- 172.21.15.66:0/1326446124 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00e040 con 0x7ff3900047d0 2024-03-30T00:41:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a2a00640 1 -- 172.21.15.66:0/1326446124 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c00f040 con 0x7ff3900047d0 2024-03-30T00:41:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3a2a00640 1 -- 172.21.15.66:0/1326446124 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c015870 con 0x7ff3900047d0 2024-03-30T00:41:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3b1200640 1 -- 172.21.15.66:0/1326446124 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac0c96a0 con 0x7ff3900047d0 2024-03-30T00:41:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3b1200640 1 -- 172.21.15.66:0/1326446124 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac15d950 con 0x7ff3900047d0 2024-03-30T00:41:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.477+0000 7ff3b1200640 1 -- 172.21.15.66:0/1326446124 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac066c30 con 0x7ff3900047d0 2024-03-30T00:41:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.481+0000 7ff3a2a00640 1 -- 172.21.15.66:0/1326446124 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff39c0159d0 con 0x7ff3900047d0 2024-03-30T00:41:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.481+0000 7ff3a2a00640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff388066690 0x7ff388068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.481+0000 7ff3a2a00640 1 -- 172.21.15.66:0/1326446124 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff39c0853a0 con 0x7ff3900047d0 2024-03-30T00:41:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.481+0000 7ff3a1400640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff388066690 0x7ff388068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.481+0000 7ff3a1400640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff388066690 0x7ff388068b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff3940ad000 tx=0x7ff394001940 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.485+0000 7ff3a2a00640 1 -- 172.21.15.66:0/1326446124 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff39c052300 con 0x7ff3900047d0 2024-03-30T00:41:01.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.645+0000 7ff3b1200640 1 -- 172.21.15.66:0/1326446124 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3ac071ff0 con 0x7ff3900047d0 2024-03-30T00:41:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.645+0000 7ff3a2a00640 1 -- 172.21.15.66:0/1326446124 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7ff39c00fea0 con 0x7ff3900047d0 2024-03-30T00:41:01.656 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:01.656 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:01.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 -- 172.21.15.66:0/1326446124 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff388066690 msgr2=0x7ff388068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff388066690 0x7ff388068b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff3940ad000 tx=0x7ff394001940 comp rx=0 tx=0).stop 2024-03-30T00:41:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 -- 172.21.15.66:0/1326446124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 msgr2=0x7ff3ac0c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 0x7ff3ac0c87b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009f90 tx=0x7ff39c00f580 comp rx=0 tx=0).stop 2024-03-30T00:41:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 -- 172.21.15.66:0/1326446124 shutdown_connections 2024-03-30T00:41:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3ac15d4d0 0x7ff3ac0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3ac149e20 0x7ff3ac1f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff388066690 0x7ff388068b50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 --2- 172.21.15.66:0/1326446124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3900047d0 0x7ff3ac0c87b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 -- 172.21.15.66:0/1326446124 >> 172.21.15.66:0/1326446124 conn(0x7ff3ac069f60 msgr2=0x7ff3ac0af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 -- 172.21.15.66:0/1326446124 shutdown_connections 2024-03-30T00:41:01.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.649+0000 7ff3b1200640 1 -- 172.21.15.66:0/1326446124 wait complete. 2024-03-30T00:41:01.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:41:01.677 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:41:01.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5593a00640 1 Processor -- start 2024-03-30T00:41:01.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5593a00640 1 -- start start 2024-03-30T00:41:01.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5593a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f558c15a640 0x7f558c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5593a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0ce520 0x7f558c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5593a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f558c06d070 con 0x7f558c0cabb0 2024-03-30T00:41:01.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5593a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558c0cbba0 con 0x7f558c15a640 2024-03-30T00:41:01.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5593a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558c06e2c0 con 0x7f558c0ce520 2024-03-30T00:41:01.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5591000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f558c15a640 0x7f558c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f558be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0ce520 0x7f558c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f558be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0ce520 0x7f558c0ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:39564/0 (socket says 172.21.15.66:39564) 2024-03-30T00:41:01.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f558be00640 1 -- 172.21.15.66:0/3583495426 learned_addr learned my addr 172.21.15.66:0/3583495426 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:01.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5591000640 1 --2- 172.21.15.66:0/3583495426 >> v2:172.21.15.156:3300/0 conn(0x7f558c15a640 0x7f558c06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:41640/0 (socket says 172.21.15.66:41640) 2024-03-30T00:41:01.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5591000640 1 -- 172.21.15.66:0/3583495426 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0ce520 msgr2=0x7f558c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:01.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5591000640 1 --2- 172.21.15.66:0/3583495426 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0ce520 0x7f558c0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5591000640 1 -- 172.21.15.66:0/3583495426 >> v1:172.21.15.156:6789/0 conn(0x7f558c0cabb0 legacy=0x7f558c0cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f5591000640 1 -- 172.21.15.66:0/3583495426 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c0cee60 con 0x7f558c15a640 2024-03-30T00:41:01.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.813+0000 7f558be00640 1 --2- 172.21.15.66:0/3583495426 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0ce520 0x7f558c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:01.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f5591000640 1 --2- 172.21.15.66:0/3583495426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f558c15a640 0x7f558c06dd80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f55800097c0 tx=0x7f558001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c2ce718df3d5940 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:01.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f558b400640 1 -- 172.21.15.66:0/3583495426 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55800ad640 con 0x7f558c15a640 2024-03-30T00:41:01.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f558b400640 1 -- 172.21.15.66:0/3583495426 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55800ad7a0 con 0x7f558c15a640 2024-03-30T00:41:01.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f558b400640 1 -- 172.21.15.66:0/3583495426 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55800adb30 con 0x7f558c15a640 2024-03-30T00:41:01.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f5593a00640 1 -- 172.21.15.66:0/3583495426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f558c15a640 msgr2=0x7f558c06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f5593a00640 1 --2- 172.21.15.66:0/3583495426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f558c15a640 0x7f558c06dd80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f55800097c0 tx=0x7f558001c4a0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f5593a00640 1 -- 172.21.15.66:0/3583495426 shutdown_connections 2024-03-30T00:41:01.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f5593a00640 1 --2- 172.21.15.66:0/3583495426 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0ce520 0x7f558c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f5593a00640 1 --2- 172.21.15.66:0/3583495426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f558c15a640 0x7f558c06dd80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f5593a00640 1 -- 172.21.15.66:0/3583495426 >> 172.21.15.66:0/3583495426 conn(0x7f558c069f60 msgr2=0x7f558c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:01.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f5593a00640 1 -- 172.21.15.66:0/3583495426 shutdown_connections 2024-03-30T00:41:01.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.817+0000 7f5593a00640 1 -- 172.21.15.66:0/3583495426 wait complete. 2024-03-30T00:41:01.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5593a00640 1 Processor -- start 2024-03-30T00:41:01.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5593a00640 1 -- start start 2024-03-30T00:41:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5593a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0cabb0 0x7f558c1f4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5593a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f558c0ce520 0x7f558c0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5593a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f558c15a640 0x7f558c0cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5593a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0afc10 con 0x7f558c15a640 2024-03-30T00:41:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5593a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c1f5a00 con 0x7f558c0ce520 2024-03-30T00:41:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5593a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558c1f4df0 con 0x7f558c0cabb0 2024-03-30T00:41:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5591a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f558c15a640 0x7f558c0cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5591000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0cabb0 0x7f558c1f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5591a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f558c15a640 0x7f558c0cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52586/0 (socket says 172.21.15.66:52586) 2024-03-30T00:41:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f558be00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f558c0ce520 0x7f558c0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5591a00640 1 -- 172.21.15.66:0/791741447 learned_addr learned my addr 172.21.15.66:0/791741447 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:01.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5591a00640 1 -- 172.21.15.66:0/791741447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0cabb0 msgr2=0x7f558c1f4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5591a00640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0cabb0 0x7f558c1f4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5591a00640 1 -- 172.21.15.66:0/791741447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f558c0ce520 msgr2=0x7f558c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:01.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5591a00640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f558c0ce520 0x7f558c0cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:01.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5591a00640 1 -- 172.21.15.66:0/791741447 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55780094d0 con 0x7f558c15a640 2024-03-30T00:41:01.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f558be00640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f558c0ce520 0x7f558c0cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:01.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5591000640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0cabb0 0x7f558c1f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:01.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5591a00640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f558c15a640 0x7f558c0cd410 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f558400c770 tx=0x7f558400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5588c00640 1 -- 172.21.15.66:0/791741447 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584003b80 con 0x7f558c15a640 2024-03-30T00:41:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5588c00640 1 -- 172.21.15.66:0/791741447 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5584003ce0 con 0x7f558c15a640 2024-03-30T00:41:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5588c00640 1 -- 172.21.15.66:0/791741447 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558400aa90 con 0x7f558c15a640 2024-03-30T00:41:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5593a00640 1 -- 172.21.15.66:0/791741447 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5580009420 con 0x7f558c15a640 2024-03-30T00:41:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.821+0000 7f5593a00640 1 -- 172.21.15.66:0/791741447 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558c1f53b0 con 0x7f558c15a640 2024-03-30T00:41:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.825+0000 7f5593a00640 1 -- 172.21.15.66:0/791741447 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558c1f5b90 con 0x7f558c15a640 2024-03-30T00:41:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.825+0000 7f5588c00640 1 -- 172.21.15.66:0/791741447 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f558400abf0 con 0x7f558c15a640 2024-03-30T00:41:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.825+0000 7f5588c00640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f556c066690 0x7f556c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.825+0000 7f5588c00640 1 -- 172.21.15.66:0/791741447 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f5584084290 con 0x7f558c15a640 2024-03-30T00:41:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.825+0000 7f5591000640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f556c066690 0x7f556c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.825+0000 7f5591000640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f556c066690 0x7f556c068b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55800af000 tx=0x7f5580008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:01.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.829+0000 7f5588c00640 1 -- 172.21.15.66:0/791741447 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f55840512b0 con 0x7f558c15a640 2024-03-30T00:41:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.989+0000 7f5593a00640 1 -- 172.21.15.66:0/791741447 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f558c066c30 con 0x7f558c15a640 2024-03-30T00:41:02.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.989+0000 7f5588c00640 1 -- 172.21.15.66:0/791741447 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55840548d0 con 0x7f558c15a640 2024-03-30T00:41:02.001 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:41:02.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 -- 172.21.15.66:0/791741447 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f556c066690 msgr2=0x7f556c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f556c066690 0x7f556c068b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55800af000 tx=0x7f5580008040 comp rx=0 tx=0).stop 2024-03-30T00:41:02.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 -- 172.21.15.66:0/791741447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f558c15a640 msgr2=0x7f558c0cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f558c15a640 0x7f558c0cd410 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f558400c770 tx=0x7f558400cc30 comp rx=0 tx=0).stop 2024-03-30T00:41:02.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 -- 172.21.15.66:0/791741447 shutdown_connections 2024-03-30T00:41:02.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f556c066690 0x7f556c068b50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f558c15a640 0x7f558c0cd410 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f558c0ce520 0x7f558c0cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 --2- 172.21.15.66:0/791741447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558c0cabb0 0x7f558c1f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 -- 172.21.15.66:0/791741447 >> 172.21.15.66:0/791741447 conn(0x7f558c069f60 msgr2=0x7f558c15c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:02.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 -- 172.21.15.66:0/791741447 shutdown_connections 2024-03-30T00:41:02.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:01.993+0000 7f5593a00640 1 -- 172.21.15.66:0/791741447 wait complete. 2024-03-30T00:41:02.022 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:41:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7c9600640 1 Processor -- start 2024-03-30T00:41:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7c9600640 1 -- start start 2024-03-30T00:41:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7c9600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa7c415a640 0x7fa7c406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7c9600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa7c40ce520 0x7fa7c40ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7c9600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c406d070 con 0x7fa7c40cabb0 2024-03-30T00:41:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7c9600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c40cbba0 con 0x7fa7c40ce520 2024-03-30T00:41:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7c9600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c40cee90 con 0x7fa7c415a640 2024-03-30T00:41:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7bbe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa7c40ce520 0x7fa7c40ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7bbe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa7c40ce520 0x7fa7c40ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52588/0 (socket says 172.21.15.66:52588) 2024-03-30T00:41:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7bbe00640 1 -- 172.21.15.66:0/323144138 learned_addr learned my addr 172.21.15.66:0/323144138 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7c2a00640 1 --2- 172.21.15.66:0/323144138 >> v2:172.21.15.156:3300/0 conn(0x7fa7c415a640 0x7fa7c406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7bbe00640 1 -- 172.21.15.66:0/323144138 >> v2:172.21.15.156:3300/0 conn(0x7fa7c415a640 msgr2=0x7fa7c406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7bbe00640 1 --2- 172.21.15.66:0/323144138 >> v2:172.21.15.156:3300/0 conn(0x7fa7c415a640 0x7fa7c406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7bbe00640 1 -- 172.21.15.66:0/323144138 >> v1:172.21.15.156:6789/0 conn(0x7fa7c40cabb0 legacy=0x7fa7c40cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.157+0000 7fa7bbe00640 1 -- 172.21.15.66:0/323144138 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c406e2c0 con 0x7fa7c40ce520 2024-03-30T00:41:02.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7bbe00640 1 --2- 172.21.15.66:0/323144138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c40ce520 0x7fa7c40ce950 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40cf160 tx=0x7fa7ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0617c52decf2b1b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:02.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c2000640 1 -- 172.21.15.66:0/323144138 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac0ab610 con 0x7fa7c40ce520 2024-03-30T00:41:02.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c2000640 1 -- 172.21.15.66:0/323144138 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac0ab770 con 0x7fa7c40ce520 2024-03-30T00:41:02.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c2000640 1 -- 172.21.15.66:0/323144138 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac0abb00 con 0x7fa7c40ce520 2024-03-30T00:41:02.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c9600640 1 -- 172.21.15.66:0/323144138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c40ce520 msgr2=0x7fa7c40ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c9600640 1 --2- 172.21.15.66:0/323144138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c40ce520 0x7fa7c40ce950 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40cf160 tx=0x7fa7ac01c520 comp rx=0 tx=0).stop 2024-03-30T00:41:02.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c9600640 1 -- 172.21.15.66:0/323144138 shutdown_connections 2024-03-30T00:41:02.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c9600640 1 --2- 172.21.15.66:0/323144138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c40ce520 0x7fa7c40ce950 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c9600640 1 --2- 172.21.15.66:0/323144138 >> v2:172.21.15.156:3300/0 conn(0x7fa7c415a640 0x7fa7c406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c9600640 1 -- 172.21.15.66:0/323144138 >> 172.21.15.66:0/323144138 conn(0x7fa7c4069f60 msgr2=0x7fa7c406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c9600640 1 -- 172.21.15.66:0/323144138 shutdown_connections 2024-03-30T00:41:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c9600640 1 -- 172.21.15.66:0/323144138 wait complete. 2024-03-30T00:41:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c9600640 1 Processor -- start 2024-03-30T00:41:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.161+0000 7fa7c9600640 1 -- start start 2024-03-30T00:41:02.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c40cabb0 0x7fa7c4156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c9600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7c40ce520 0x7fa7c4158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c9600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7c415a640 0x7fa7c4159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c9600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40cde90 con 0x7fa7c40cabb0 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c9600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40afb00 con 0x7fa7c40ce520 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c9600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40cbf20 con 0x7fa7c415a640 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c40cabb0 0x7fa7c4156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c40cabb0 0x7fa7c4156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52600/0 (socket says 172.21.15.66:52600) 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c2a00640 1 -- 172.21.15.66:0/3765228097 learned_addr learned my addr 172.21.15.66:0/3765228097 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7bbe00640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7c40ce520 0x7fa7c4158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c3400640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7c415a640 0x7fa7c4159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c2a00640 1 -- 172.21.15.66:0/3765228097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7c415a640 msgr2=0x7fa7c4159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c2a00640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7c415a640 0x7fa7c4159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c2a00640 1 -- 172.21.15.66:0/3765228097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7c40ce520 msgr2=0x7fa7c4158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c2a00640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7c40ce520 0x7fa7c4158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c2a00640 1 -- 172.21.15.66:0/3765228097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac0094d0 con 0x7fa7c40cabb0 2024-03-30T00:41:02.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c2a00640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c40cabb0 0x7fa7c4156f30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e870 tx=0x7fa7b400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:02.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7baa00640 1 -- 172.21.15.66:0/3765228097 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4009510 con 0x7fa7c40cabb0 2024-03-30T00:41:02.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7baa00640 1 -- 172.21.15.66:0/3765228097 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b400eec0 con 0x7fa7c40cabb0 2024-03-30T00:41:02.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c9600640 1 -- 172.21.15.66:0/3765228097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c40cc200 con 0x7fa7c40cabb0 2024-03-30T00:41:02.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7baa00640 1 -- 172.21.15.66:0/3765228097 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400cb40 con 0x7fa7c40cabb0 2024-03-30T00:41:02.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c9600640 1 -- 172.21.15.66:0/3765228097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c40cc7c0 con 0x7fa7c40cabb0 2024-03-30T00:41:02.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7c9600640 1 -- 172.21.15.66:0/3765228097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c4066c30 con 0x7fa7c40cabb0 2024-03-30T00:41:02.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.165+0000 7fa7baa00640 1 -- 172.21.15.66:0/3765228097 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa7b400cca0 con 0x7fa7c40cabb0 2024-03-30T00:41:02.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.169+0000 7fa7baa00640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7a00666c0 0x7fa7a0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.169+0000 7fa7baa00640 1 -- 172.21.15.66:0/3765228097 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fa7b4084620 con 0x7fa7c40cabb0 2024-03-30T00:41:02.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.169+0000 7fa7bbe00640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7a00666c0 0x7fa7a0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.169+0000 7fa7bbe00640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7a00666c0 0x7fa7a0068b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0af000 tx=0x7fa7ac0b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:02.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.173+0000 7fa7baa00640 1 -- 172.21.15.66:0/3765228097 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa7b4051630 con 0x7fa7c40cabb0 2024-03-30T00:41:02.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 -- 172.21.15.66:0/3765228097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7c4157470 con 0x7fa7c40cabb0 2024-03-30T00:41:02.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7baa00640 1 -- 172.21.15.66:0/3765228097 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa7b4054c50 con 0x7fa7c40cabb0 2024-03-30T00:41:02.342 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:41:02.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 -- 172.21.15.66:0/3765228097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7a00666c0 msgr2=0x7fa7a0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7a00666c0 0x7fa7a0068b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0af000 tx=0x7fa7ac0b6040 comp rx=0 tx=0).stop 2024-03-30T00:41:02.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 -- 172.21.15.66:0/3765228097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c40cabb0 msgr2=0x7fa7c4156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c40cabb0 0x7fa7c4156f30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e870 tx=0x7fa7b400ec70 comp rx=0 tx=0).stop 2024-03-30T00:41:02.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 -- 172.21.15.66:0/3765228097 shutdown_connections 2024-03-30T00:41:02.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7c415a640 0x7fa7c4159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7c40ce520 0x7fa7c4158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa7a00666c0 0x7fa7a0068b80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 --2- 172.21.15.66:0/3765228097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c40cabb0 0x7fa7c4156f30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 -- 172.21.15.66:0/3765228097 >> 172.21.15.66:0/3765228097 conn(0x7fa7c4069f60 msgr2=0x7fa7c40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 -- 172.21.15.66:0/3765228097 shutdown_connections 2024-03-30T00:41:02.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.333+0000 7fa7c9600640 1 -- 172.21.15.66:0/3765228097 wait complete. 2024-03-30T00:41:02.364 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:41:02.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.497+0000 7f23d9600640 1 Processor -- start 2024-03-30T00:41:02.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.497+0000 7f23d9600640 1 -- start start 2024-03-30T00:41:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.497+0000 7f23d9600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 0x7f23d40ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.497+0000 7f23d9600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23d40cde70 0x7f23d40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.497+0000 7f23d9600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d415a990 con 0x7f23d4159c60 2024-03-30T00:41:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.497+0000 7f23d9600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d40b1b80 con 0x7f23d40cde70 2024-03-30T00:41:02.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.497+0000 7f23d9600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40cd3e0 con 0x7f23d40cda90 2024-03-30T00:41:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.497+0000 7f23d2a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 0x7f23d40ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.497+0000 7f23ca000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23d40cde70 0x7f23d40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.497+0000 7f23d2a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 0x7f23d40ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:39582/0 (socket says 172.21.15.66:39582) 2024-03-30T00:41:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.497+0000 7f23d2a00640 1 -- 172.21.15.66:0/1680974840 learned_addr learned my addr 172.21.15.66:0/1680974840 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d2a00640 1 -- 172.21.15.66:0/1680974840 >> v2:172.21.15.156:3300/0 conn(0x7f23d40cde70 msgr2=0x7f23d40cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d2a00640 1 --2- 172.21.15.66:0/1680974840 >> v2:172.21.15.156:3300/0 conn(0x7f23d40cde70 0x7f23d40cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d2a00640 1 -- 172.21.15.66:0/1680974840 >> v1:172.21.15.156:6789/0 conn(0x7f23d4159c60 legacy=0x7f23d415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d2a00640 1 -- 172.21.15.66:0/1680974840 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40b11e0 con 0x7f23d40cda90 2024-03-30T00:41:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d2000640 1 -- 172.21.15.66:0/1680974840 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784964102 0 0) 0x7f23d415a990 con 0x7f23d4159c60 2024-03-30T00:41:02.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d2a00640 1 --2- 172.21.15.66:0/1680974840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 0x7f23d40ce520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009870 tx=0x7f23c401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2414175435bf6296 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:02.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d2000640 1 -- 172.21.15.66:0/1680974840 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40ac640 con 0x7f23d40cda90 2024-03-30T00:41:02.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d2000640 1 -- 172.21.15.66:0/1680974840 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c40ac7a0 con 0x7f23d40cda90 2024-03-30T00:41:02.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d2000640 1 -- 172.21.15.66:0/1680974840 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40acb30 con 0x7f23d40cda90 2024-03-30T00:41:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d9600640 1 -- 172.21.15.66:0/1680974840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 msgr2=0x7f23d40ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d9600640 1 --2- 172.21.15.66:0/1680974840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 0x7f23d40ce520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009870 tx=0x7f23c401c4d0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d9600640 1 -- 172.21.15.66:0/1680974840 shutdown_connections 2024-03-30T00:41:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d9600640 1 --2- 172.21.15.66:0/1680974840 >> v2:172.21.15.156:3300/0 conn(0x7f23d40cde70 0x7f23d40cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d9600640 1 --2- 172.21.15.66:0/1680974840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 0x7f23d40ce520 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.501+0000 7f23d9600640 1 -- 172.21.15.66:0/1680974840 >> 172.21.15.66:0/1680974840 conn(0x7f23d4069f60 msgr2=0x7f23d406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 -- 172.21.15.66:0/1680974840 shutdown_connections 2024-03-30T00:41:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 -- 172.21.15.66:0/1680974840 wait complete. 2024-03-30T00:41:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 Processor -- start 2024-03-30T00:41:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 -- start start 2024-03-30T00:41:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 0x7f23d4097960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23d40cde70 0x7f23d40c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23d4159c60 0x7f23d40aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4156720 con 0x7f23d40cde70 2024-03-30T00:41:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4097ea0 con 0x7f23d4159c60 2024-03-30T00:41:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4155680 con 0x7f23d40cda90 2024-03-30T00:41:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23ca000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23d40cde70 0x7f23d40c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23ca000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23d40cde70 0x7f23d40c0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52614/0 (socket says 172.21.15.66:52614) 2024-03-30T00:41:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23ca000640 1 -- 172.21.15.66:0/4018496040 learned_addr learned my addr 172.21.15.66:0/4018496040 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d2a00640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 0x7f23d4097960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d3400640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23d4159c60 0x7f23d40aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23ca000640 1 -- 172.21.15.66:0/4018496040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 msgr2=0x7f23d4097960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23ca000640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 0x7f23d4097960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23ca000640 1 -- 172.21.15.66:0/4018496040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23d4159c60 msgr2=0x7f23d40aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23ca000640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23d4159c60 0x7f23d40aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23ca000640 1 -- 172.21.15.66:0/4018496040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c40094d0 con 0x7f23d40cde70 2024-03-30T00:41:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23ca000640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23d40cde70 0x7f23d40c0dc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00e870 tx=0x7f23bc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23cb400640 1 -- 172.21.15.66:0/4018496040 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc009510 con 0x7f23d40cde70 2024-03-30T00:41:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23cb400640 1 -- 172.21.15.66:0/4018496040 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc00eec0 con 0x7f23d40cde70 2024-03-30T00:41:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 -- 172.21.15.66:0/4018496040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d4155960 con 0x7f23d40cde70 2024-03-30T00:41:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23cb400640 1 -- 172.21.15.66:0/4018496040 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc00cb40 con 0x7f23d40cde70 2024-03-30T00:41:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.505+0000 7f23d9600640 1 -- 172.21.15.66:0/4018496040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d4155f20 con 0x7f23d40cde70 2024-03-30T00:41:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.509+0000 7f23d9600640 1 -- 172.21.15.66:0/4018496040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d4066c30 con 0x7f23d40cde70 2024-03-30T00:41:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.509+0000 7f23cb400640 1 -- 172.21.15.66:0/4018496040 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f23bc00cca0 con 0x7f23d40cde70 2024-03-30T00:41:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.509+0000 7f23cb400640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f23b00666c0 0x7f23b0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.509+0000 7f23cb400640 1 -- 172.21.15.66:0/4018496040 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f23bc084630 con 0x7f23d40cde70 2024-03-30T00:41:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.509+0000 7f23d2a00640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f23b00666c0 0x7f23b0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.509+0000 7f23d2a00640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f23b00666c0 0x7f23b0068b80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f23c40ae000 tx=0x7f23c40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:02.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.513+0000 7f23cb400640 1 -- 172.21.15.66:0/4018496040 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f23bc0515c0 con 0x7f23d40cde70 2024-03-30T00:41:02.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.673+0000 7f23d9600640 1 -- 172.21.15.66:0/4018496040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23d40b3170 con 0x7f23d40cde70 2024-03-30T00:41:02.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.673+0000 7f23cb400640 1 -- 172.21.15.66:0/4018496040 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f23bc054be0 con 0x7f23d40cde70 2024-03-30T00:41:02.684 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:41:02.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 -- 172.21.15.66:0/4018496040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f23b00666c0 msgr2=0x7f23b0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f23b00666c0 0x7f23b0068b80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f23c40ae000 tx=0x7f23c40a7000 comp rx=0 tx=0).stop 2024-03-30T00:41:02.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 -- 172.21.15.66:0/4018496040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23d40cde70 msgr2=0x7f23d40c0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23d40cde70 0x7f23d40c0dc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00e870 tx=0x7f23bc00ec70 comp rx=0 tx=0).stop 2024-03-30T00:41:02.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 -- 172.21.15.66:0/4018496040 shutdown_connections 2024-03-30T00:41:02.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23d4159c60 0x7f23d40aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f23b00666c0 0x7f23b0068b80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23d40cde70 0x7f23d40c0dc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 --2- 172.21.15.66:0/4018496040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23d40cda90 0x7f23d4097960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 -- 172.21.15.66:0/4018496040 >> 172.21.15.66:0/4018496040 conn(0x7f23d4069f60 msgr2=0x7f23d41533e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:02.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 -- 172.21.15.66:0/4018496040 shutdown_connections 2024-03-30T00:41:02.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.677+0000 7f23d9600640 1 -- 172.21.15.66:0/4018496040 wait complete. 2024-03-30T00:41:02.705 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:41:02.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.841+0000 7f3225200640 1 Processor -- start 2024-03-30T00:41:02.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.841+0000 7f3225200640 1 -- start start 2024-03-30T00:41:02.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.841+0000 7f3225200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f322015ca80 0x7f322006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.841+0000 7f3225200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f32200cb570 0x7f32200cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.841+0000 7f3225200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32200cb0e0 con 0x7f32200cf2f0 2024-03-30T00:41:02.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.841+0000 7f3225200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32200cf7f0 con 0x7f32200cb570 2024-03-30T00:41:02.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.841+0000 7f3225200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32200cfad0 con 0x7f322015ca80 2024-03-30T00:41:02.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321e800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f32200cb570 0x7f32200cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f322015ca80 0x7f322006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321e800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f32200cb570 0x7f32200cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:41694/0 (socket says 172.21.15.66:41694) 2024-03-30T00:41:02.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321e800640 1 -- 172.21.15.66:0/3594482444 learned_addr learned my addr 172.21.15.66:0/3594482444 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:02.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321e800640 1 -- 172.21.15.66:0/3594482444 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f322015ca80 msgr2=0x7f322006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:02.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321e800640 1 --2- 172.21.15.66:0/3594482444 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f322015ca80 0x7f322006b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321e800640 1 -- 172.21.15.66:0/3594482444 >> v1:172.21.15.156:6789/0 conn(0x7f32200cf2f0 legacy=0x7f322006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321e800640 1 -- 172.21.15.66:0/3594482444 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200cad70 con 0x7f32200cb570 2024-03-30T00:41:02.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321e800640 1 --2- 172.21.15.66:0/3594482444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32200cb570 0x7f32200cba20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3218009870 tx=0x7f321801ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09d5146ee1343d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:02.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321d400640 1 -- 172.21.15.66:0/3594482444 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32180a8a50 con 0x7f32200cb570 2024-03-30T00:41:02.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321d400640 1 -- 172.21.15.66:0/3594482444 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32180a8bb0 con 0x7f32200cb570 2024-03-30T00:41:02.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.845+0000 7f321d400640 1 -- 172.21.15.66:0/3594482444 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32180b0440 con 0x7f32200cb570 2024-03-30T00:41:02.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 -- 172.21.15.66:0/3594482444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32200cb570 msgr2=0x7f32200cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 --2- 172.21.15.66:0/3594482444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32200cb570 0x7f32200cba20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3218009870 tx=0x7f321801ed50 comp rx=0 tx=0).stop 2024-03-30T00:41:02.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 -- 172.21.15.66:0/3594482444 shutdown_connections 2024-03-30T00:41:02.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 --2- 172.21.15.66:0/3594482444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32200cb570 0x7f32200cba20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 --2- 172.21.15.66:0/3594482444 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f322015ca80 0x7f322006b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 -- 172.21.15.66:0/3594482444 >> 172.21.15.66:0/3594482444 conn(0x7f32200b2090 msgr2=0x7f32200b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:02.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 -- 172.21.15.66:0/3594482444 shutdown_connections 2024-03-30T00:41:02.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 -- 172.21.15.66:0/3594482444 wait complete. 2024-03-30T00:41:02.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 Processor -- start 2024-03-30T00:41:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 -- start start 2024-03-30T00:41:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32200cb570 0x7f32200c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32200cf2f0 0x7f32200ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f322015ca80 0x7f322014c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200c7f50 con 0x7f32200cf2f0 2024-03-30T00:41:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200c8230 con 0x7f322015ca80 2024-03-30T00:41:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3225200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32200c8510 con 0x7f32200cb570 2024-03-30T00:41:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3215400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32200cf2f0 0x7f32200ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3215400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32200cf2f0 0x7f32200ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52622/0 (socket says 172.21.15.66:52622) 2024-03-30T00:41:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3215400640 1 -- 172.21.15.66:0/4152746379 learned_addr learned my addr 172.21.15.66:0/4152746379 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:02.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f321e800640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f322015ca80 0x7f322014c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f321de00640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32200cb570 0x7f32200c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3215400640 1 -- 172.21.15.66:0/4152746379 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32200cb570 msgr2=0x7f32200c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3215400640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32200cb570 0x7f32200c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3215400640 1 -- 172.21.15.66:0/4152746379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f322015ca80 msgr2=0x7f322014c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3215400640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f322015ca80 0x7f322014c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f3215400640 1 -- 172.21.15.66:0/4152746379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32180094d0 con 0x7f32200cf2f0 2024-03-30T00:41:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.849+0000 7f321de00640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32200cb570 0x7f32200c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.853+0000 7f3215400640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32200cf2f0 0x7f32200ce990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f320800c500 tx=0x7f320800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.853+0000 7f3216a00640 1 -- 172.21.15.66:0/4152746379 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208005220 con 0x7f32200cf2f0 2024-03-30T00:41:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.853+0000 7f3216a00640 1 -- 172.21.15.66:0/4152746379 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208005380 con 0x7f32200cf2f0 2024-03-30T00:41:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.853+0000 7f3225200640 1 -- 172.21.15.66:0/4152746379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32200c87f0 con 0x7f32200cf2f0 2024-03-30T00:41:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.853+0000 7f3216a00640 1 -- 172.21.15.66:0/4152746379 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208007270 con 0x7f32200cf2f0 2024-03-30T00:41:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.853+0000 7f3225200640 1 -- 172.21.15.66:0/4152746379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32200c7bb0 con 0x7f32200cf2f0 2024-03-30T00:41:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.853+0000 7f3225200640 1 -- 172.21.15.66:0/4152746379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3220066c30 con 0x7f32200cf2f0 2024-03-30T00:41:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.853+0000 7f3216a00640 1 -- 172.21.15.66:0/4152746379 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f32080073d0 con 0x7f32200cf2f0 2024-03-30T00:41:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.853+0000 7f3216a00640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f31fc066690 0x7f31fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.853+0000 7f3216a00640 1 -- 172.21.15.66:0/4152746379 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f3208083e90 con 0x7f32200cf2f0 2024-03-30T00:41:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.853+0000 7f321de00640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f31fc066690 0x7f31fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.857+0000 7f321de00640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f31fc066690 0x7f31fc068b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3210009fd0 tx=0x7f321000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:02.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:02.857+0000 7f3216a00640 1 -- 172.21.15.66:0/4152746379 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3208050d70 con 0x7f32200cf2f0 2024-03-30T00:41:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.021+0000 7f3225200640 1 -- 172.21.15.66:0/4152746379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f32200b2bb0 con 0x7f32200cf2f0 2024-03-30T00:41:03.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.021+0000 7f3216a00640 1 -- 172.21.15.66:0/4152746379 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3208054390 con 0x7f32200cf2f0 2024-03-30T00:41:03.031 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:41:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 -- 172.21.15.66:0/4152746379 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f31fc066690 msgr2=0x7f31fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f31fc066690 0x7f31fc068b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3210009fd0 tx=0x7f321000b040 comp rx=0 tx=0).stop 2024-03-30T00:41:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 -- 172.21.15.66:0/4152746379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32200cf2f0 msgr2=0x7f32200ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32200cf2f0 0x7f32200ce990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f320800c500 tx=0x7f320800c9c0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 -- 172.21.15.66:0/4152746379 shutdown_connections 2024-03-30T00:41:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f322015ca80 0x7f322014c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f31fc066690 0x7f31fc068b50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32200cf2f0 0x7f32200ce990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 --2- 172.21.15.66:0/4152746379 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32200cb570 0x7f32200c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 -- 172.21.15.66:0/4152746379 >> 172.21.15.66:0/4152746379 conn(0x7f32200b2090 msgr2=0x7f32200b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 -- 172.21.15.66:0/4152746379 shutdown_connections 2024-03-30T00:41:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.025+0000 7f3225200640 1 -- 172.21.15.66:0/4152746379 wait complete. 2024-03-30T00:41:03.053 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:41:03.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f9364e00640 1 Processor -- start 2024-03-30T00:41:03.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f9364e00640 1 -- start start 2024-03-30T00:41:03.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f9364e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f936015ca80 0x7f936006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f9364e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f93600cf2f0 0x7f936006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f9364e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f936006bb30 0x7f9360098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f9364e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93600cb0e0 con 0x7f936006bb30 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f9364e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93600cf7f0 con 0x7f93600cf2f0 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f9364e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93600cfad0 con 0x7f936015ca80 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f935e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f936006bb30 0x7f9360098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f9355400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f93600cf2f0 0x7f936006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f935e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f936006bb30 0x7f9360098a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52626/0 (socket says 172.21.15.66:52626) 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f935e800640 1 -- 172.21.15.66:0/1373925175 learned_addr learned my addr 172.21.15.66:0/1373925175 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f935de00640 1 --2- 172.21.15.66:0/1373925175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f936015ca80 0x7f936006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f935e800640 1 -- 172.21.15.66:0/1373925175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f936015ca80 msgr2=0x7f936006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f935e800640 1 --2- 172.21.15.66:0/1373925175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f936015ca80 0x7f936006b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f935e800640 1 -- 172.21.15.66:0/1373925175 >> v2:172.21.15.156:3300/0 conn(0x7f93600cf2f0 msgr2=0x7f936006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f935e800640 1 --2- 172.21.15.66:0/1373925175 >> v2:172.21.15.156:3300/0 conn(0x7f93600cf2f0 0x7f936006b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.185+0000 7f935e800640 1 -- 172.21.15.66:0/1373925175 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600cad70 con 0x7f936006bb30 2024-03-30T00:41:03.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f935e800640 1 --2- 172.21.15.66:0/1373925175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f936006bb30 0x7f9360098a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f93600711d0 tx=0x7f935801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d2c5835602ab8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f935d400640 1 -- 172.21.15.66:0/1373925175 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580af400 con 0x7f936006bb30 2024-03-30T00:41:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f935d400640 1 -- 172.21.15.66:0/1373925175 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93580ab050 con 0x7f936006bb30 2024-03-30T00:41:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f935d400640 1 -- 172.21.15.66:0/1373925175 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580af780 con 0x7f936006bb30 2024-03-30T00:41:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f9364e00640 1 -- 172.21.15.66:0/1373925175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f936006bb30 msgr2=0x7f9360098a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f9364e00640 1 --2- 172.21.15.66:0/1373925175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f936006bb30 0x7f9360098a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f93600711d0 tx=0x7f935801f1e0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f9364e00640 1 -- 172.21.15.66:0/1373925175 shutdown_connections 2024-03-30T00:41:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f9364e00640 1 --2- 172.21.15.66:0/1373925175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f936006bb30 0x7f9360098a20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f9364e00640 1 --2- 172.21.15.66:0/1373925175 >> v2:172.21.15.156:3300/0 conn(0x7f93600cf2f0 0x7f936006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f9364e00640 1 --2- 172.21.15.66:0/1373925175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f936015ca80 0x7f936006b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f9364e00640 1 -- 172.21.15.66:0/1373925175 >> 172.21.15.66:0/1373925175 conn(0x7f93600b2090 msgr2=0x7f93600b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:03.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f9364e00640 1 -- 172.21.15.66:0/1373925175 shutdown_connections 2024-03-30T00:41:03.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.189+0000 7f9364e00640 1 -- 172.21.15.66:0/1373925175 wait complete. 2024-03-30T00:41:03.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9364e00640 1 Processor -- start 2024-03-30T00:41:03.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9364e00640 1 -- start start 2024-03-30T00:41:03.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9364e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f936006bb30 0x7f936015b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9364e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93600cf2f0 0x7f93600b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9364e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f936015ca80 0x7f93600c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9364e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600b3e30 con 0x7f93600cf2f0 2024-03-30T00:41:03.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9364e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600b31b0 con 0x7f936015ca80 2024-03-30T00:41:03.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9364e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93600b3490 con 0x7f936006bb30 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9355400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93600cf2f0 0x7f93600b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9355400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93600cf2f0 0x7f93600b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52642/0 (socket says 172.21.15.66:52642) 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9355400640 1 -- 172.21.15.66:0/4186020954 learned_addr learned my addr 172.21.15.66:0/4186020954 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f935de00640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f936006bb30 0x7f936015b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f935e800640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f936015ca80 0x7f93600c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9355400640 1 -- 172.21.15.66:0/4186020954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f936006bb30 msgr2=0x7f936015b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9355400640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f936006bb30 0x7f936015b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9355400640 1 -- 172.21.15.66:0/4186020954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f936015ca80 msgr2=0x7f93600c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9355400640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f936015ca80 0x7f93600c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9355400640 1 -- 172.21.15.66:0/4186020954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358009b30 con 0x7f93600cf2f0 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f935de00640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f936006bb30 0x7f936015b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9355400640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93600cf2f0 0x7f93600b4ae0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f934800e8b0 tx=0x7f934800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9356a00640 1 -- 172.21.15.66:0/4186020954 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934800c940 con 0x7f93600cf2f0 2024-03-30T00:41:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9356a00640 1 -- 172.21.15.66:0/4186020954 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934800caa0 con 0x7f93600cf2f0 2024-03-30T00:41:03.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9364e00640 1 -- 172.21.15.66:0/4186020954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93600b3770 con 0x7f93600cf2f0 2024-03-30T00:41:03.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9356a00640 1 -- 172.21.15.66:0/4186020954 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934800ce30 con 0x7f93600cf2f0 2024-03-30T00:41:03.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.193+0000 7f9364e00640 1 -- 172.21.15.66:0/4186020954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936015b660 con 0x7f93600cf2f0 2024-03-30T00:41:03.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.197+0000 7f9364e00640 1 -- 172.21.15.66:0/4186020954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9360066c30 con 0x7f93600cf2f0 2024-03-30T00:41:03.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.197+0000 7f9356a00640 1 -- 172.21.15.66:0/4186020954 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9348013470 con 0x7f93600cf2f0 2024-03-30T00:41:03.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.197+0000 7f9356a00640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f933c066690 0x7f933c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.197+0000 7f9356a00640 1 -- 172.21.15.66:0/4186020954 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f9348084820 con 0x7f93600cf2f0 2024-03-30T00:41:03.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.197+0000 7f935de00640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f933c066690 0x7f933c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.197+0000 7f935de00640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f933c066690 0x7f933c068b50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9350009b10 tx=0x7f9350008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:03.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.201+0000 7f9356a00640 1 -- 172.21.15.66:0/4186020954 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f93480516f0 con 0x7f93600cf2f0 2024-03-30T00:41:03.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 -- 172.21.15.66:0/4186020954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f93600b5c70 con 0x7f93600cf2f0 2024-03-30T00:41:03.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9356a00640 1 -- 172.21.15.66:0/4186020954 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9348054d10 con 0x7f93600cf2f0 2024-03-30T00:41:03.385 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:41:03.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 -- 172.21.15.66:0/4186020954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f933c066690 msgr2=0x7f933c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f933c066690 0x7f933c068b50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9350009b10 tx=0x7f9350008040 comp rx=0 tx=0).stop 2024-03-30T00:41:03.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 -- 172.21.15.66:0/4186020954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93600cf2f0 msgr2=0x7f93600b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93600cf2f0 0x7f93600b4ae0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f934800e8b0 tx=0x7f934800ed70 comp rx=0 tx=0).stop 2024-03-30T00:41:03.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 -- 172.21.15.66:0/4186020954 shutdown_connections 2024-03-30T00:41:03.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f936015ca80 0x7f93600c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f933c066690 0x7f933c068b50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93600cf2f0 0x7f93600b4ae0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 --2- 172.21.15.66:0/4186020954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f936006bb30 0x7f936015b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 -- 172.21.15.66:0/4186020954 >> 172.21.15.66:0/4186020954 conn(0x7f93600b2090 msgr2=0x7f936006f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:03.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 -- 172.21.15.66:0/4186020954 shutdown_connections 2024-03-30T00:41:03.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.377+0000 7f9364e00640 1 -- 172.21.15.66:0/4186020954 wait complete. 2024-03-30T00:41:03.407 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:41:03.407 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:41:03.417 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:03.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a9400640 1 Processor -- start 2024-03-30T00:41:03.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a9400640 1 -- start start 2024-03-30T00:41:03.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a9400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2a40cee50 0x7fa2a40cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a9400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a40cbb90 con 0x7fa2a40af420 2024-03-30T00:41:03.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a9400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a40ce1b0 con 0x7fa2a415de30 2024-03-30T00:41:03.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a9400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a40afa10 con 0x7fa2a40cee50 2024-03-30T00:41:03.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a3200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa2a40af420 0x7fa2a406e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48606/0 (socket says 172.21.15.66:48606) 2024-03-30T00:41:03.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a3200640 1 -- 172.21.15.66:0/2106713105 learned_addr learned my addr 172.21.15.66:0/2106713105 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:03.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a1e00640 1 --2- 172.21.15.66:0/2106713105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2a40cee50 0x7fa2a40cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a1400640 1 -- 172.21.15.66:0/2106713105 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543317208 0 0) 0x7fa2a40cbb90 con 0x7fa2a40af420 2024-03-30T00:41:03.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a1400640 1 -- 172.21.15.66:0/2106713105 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa288003530 con 0x7fa2a40af420 2024-03-30T00:41:03.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a1400640 1 -- 172.21.15.66:0/2106713105 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783428612 0 0) 0x7fa2a40ce1b0 con 0x7fa2a415de30 2024-03-30T00:41:03.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a1400640 1 -- 172.21.15.66:0/2106713105 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a40cbb90 con 0x7fa2a415de30 2024-03-30T00:41:03.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a1e00640 1 -- 172.21.15.66:0/2106713105 >> v1:172.21.15.156:6789/0 conn(0x7fa2a415de30 legacy=0x7fa2a40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a1e00640 1 -- 172.21.15.66:0/2106713105 >> v1:172.21.15.66:6789/0 conn(0x7fa2a40af420 legacy=0x7fa2a406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.549+0000 7fa2a1e00640 1 -- 172.21.15.66:0/2106713105 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a40cf430 con 0x7fa2a40cee50 2024-03-30T00:41:03.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.553+0000 7fa2a1e00640 1 --2- 172.21.15.66:0/2106713105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2a40cee50 0x7fa2a40cc040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2900097f0 tx=0x7fa29001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21653bf1f4f08ddc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:03.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.553+0000 7fa2a1400640 1 -- 172.21.15.66:0/2106713105 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2900ad660 con 0x7fa2a40cee50 2024-03-30T00:41:03.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.553+0000 7fa2a1400640 1 -- 172.21.15.66:0/2106713105 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2900ad7c0 con 0x7fa2a40cee50 2024-03-30T00:41:03.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.553+0000 7fa2a1400640 1 -- 172.21.15.66:0/2106713105 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2900adb30 con 0x7fa2a40cee50 2024-03-30T00:41:03.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.553+0000 7fa2a9400640 1 -- 172.21.15.66:0/2106713105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2a40cee50 msgr2=0x7fa2a40cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.553+0000 7fa2a9400640 1 --2- 172.21.15.66:0/2106713105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2a40cee50 0x7fa2a40cc040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2900097f0 tx=0x7fa29001c450 comp rx=0 tx=0).stop 2024-03-30T00:41:03.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.553+0000 7fa2a9400640 1 -- 172.21.15.66:0/2106713105 shutdown_connections 2024-03-30T00:41:03.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.553+0000 7fa2a9400640 1 --2- 172.21.15.66:0/2106713105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2a40cee50 0x7fa2a40cc040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.553+0000 7fa2a9400640 1 -- 172.21.15.66:0/2106713105 >> 172.21.15.66:0/2106713105 conn(0x7fa2a40d0540 msgr2=0x7fa2a40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:03.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.553+0000 7fa2a9400640 1 -- 172.21.15.66:0/2106713105 shutdown_connections 2024-03-30T00:41:03.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.553+0000 7fa2a9400640 1 -- 172.21.15.66:0/2106713105 wait complete. 2024-03-30T00:41:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a9400640 1 Processor -- start 2024-03-30T00:41:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a9400640 1 -- start start 2024-03-30T00:41:03.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40af420 0x7fa2a41544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a9400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2a40cee50 0x7fa2a40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a9400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2a415de30 0x7fa2a41596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a9400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a40b1fb0 con 0x7fa2a40af420 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a9400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a40b2290 con 0x7fa2a415de30 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a9400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a40b2570 con 0x7fa2a40cee50 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40af420 0x7fa2a41544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40af420 0x7fa2a41544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52658/0 (socket says 172.21.15.66:52658) 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a2800640 1 -- 172.21.15.66:0/4171254919 learned_addr learned my addr 172.21.15.66:0/4171254919 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a1e00640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2a40cee50 0x7fa2a40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a3200640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2a415de30 0x7fa2a41596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a2800640 1 -- 172.21.15.66:0/4171254919 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2a40cee50 msgr2=0x7fa2a40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a2800640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2a40cee50 0x7fa2a40b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a2800640 1 -- 172.21.15.66:0/4171254919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2a415de30 msgr2=0x7fa2a41596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a2800640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2a415de30 0x7fa2a41596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a2800640 1 -- 172.21.15.66:0/4171254919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa290009420 con 0x7fa2a40af420 2024-03-30T00:41:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a3200640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2a415de30 0x7fa2a41596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a2800640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40af420 0x7fa2a41544a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c430 tx=0x7fa29400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa29aa00640 1 -- 172.21.15.66:0/4171254919 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2940052d0 con 0x7fa2a40af420 2024-03-30T00:41:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa29aa00640 1 -- 172.21.15.66:0/4171254919 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa294005450 con 0x7fa2a40af420 2024-03-30T00:41:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa29aa00640 1 -- 172.21.15.66:0/4171254919 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294003840 con 0x7fa2a40af420 2024-03-30T00:41:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a9400640 1 -- 172.21.15.66:0/4171254919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a40b2850 con 0x7fa2a40af420 2024-03-30T00:41:03.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a9400640 1 -- 172.21.15.66:0/4171254919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a40b1bd0 con 0x7fa2a40af420 2024-03-30T00:41:03.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.557+0000 7fa2a9400640 1 -- 172.21.15.66:0/4171254919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a40cdf60 con 0x7fa2a40af420 2024-03-30T00:41:03.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.561+0000 7fa29aa00640 1 -- 172.21.15.66:0/4171254919 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa2940039a0 con 0x7fa2a40af420 2024-03-30T00:41:03.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.561+0000 7fa29aa00640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa284066690 0x7fa284068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.561+0000 7fa29aa00640 1 -- 172.21.15.66:0/4171254919 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fa294083b10 con 0x7fa2a40af420 2024-03-30T00:41:03.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.561+0000 7fa2a1e00640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa284066690 0x7fa284068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.561+0000 7fa2a1e00640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa284066690 0x7fa284068b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa2900af000 tx=0x7fa2900a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:03.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.565+0000 7fa29aa00640 1 -- 172.21.15.66:0/4171254919 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa294050b20 con 0x7fa2a40af420 2024-03-30T00:41:03.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.729+0000 7fa2a9400640 1 -- 172.21.15.66:0/4171254919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2a4066c30 con 0x7fa2a40af420 2024-03-30T00:41:03.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa29aa00640 1 -- 172.21.15.66:0/4171254919 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7fa294054140 con 0x7fa2a40af420 2024-03-30T00:41:03.741 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:03.742 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 -- 172.21.15.66:0/4171254919 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa284066690 msgr2=0x7fa284068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa284066690 0x7fa284068b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa2900af000 tx=0x7fa2900a7000 comp rx=0 tx=0).stop 2024-03-30T00:41:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 -- 172.21.15.66:0/4171254919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40af420 msgr2=0x7fa2a41544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40af420 0x7fa2a41544a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c430 tx=0x7fa29400c8f0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 -- 172.21.15.66:0/4171254919 shutdown_connections 2024-03-30T00:41:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2a415de30 0x7fa2a41596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2a40cee50 0x7fa2a40b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa284066690 0x7fa284068b50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 --2- 172.21.15.66:0/4171254919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40af420 0x7fa2a41544a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 -- 172.21.15.66:0/4171254919 >> 172.21.15.66:0/4171254919 conn(0x7fa2a40d0540 msgr2=0x7fa2a415a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 -- 172.21.15.66:0/4171254919 shutdown_connections 2024-03-30T00:41:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.733+0000 7fa2a9400640 1 -- 172.21.15.66:0/4171254919 wait complete. 2024-03-30T00:41:03.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:41:03.763 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:41:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2648000640 1 Processor -- start 2024-03-30T00:41:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2648000640 1 -- start start 2024-03-30T00:41:03.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2648000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f26400cec30 0x7f2640159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2648000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26400cd3e0 con 0x7f26400cda90 2024-03-30T00:41:03.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2648000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264015a990 con 0x7f26400cde70 2024-03-30T00:41:03.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2648000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26400b1b80 con 0x7f26400cec30 2024-03-30T00:41:03.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2644c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f26400cec30 0x7f2640159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2644c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f26400cec30 0x7f2640159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52664/0 (socket says 172.21.15.66:52664) 2024-03-30T00:41:03.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2644c00640 1 -- 172.21.15.66:0/1860375019 learned_addr learned my addr 172.21.15.66:0/1860375019 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:03.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2644c00640 1 -- 172.21.15.66:0/1860375019 >> v1:172.21.15.156:6789/0 conn(0x7f26400cde70 legacy=0x7f264015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:03.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2644c00640 1 -- 172.21.15.66:0/1860375019 >> v1:172.21.15.66:6789/0 conn(0x7f26400cda90 legacy=0x7f26400ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.897+0000 7f2644c00640 1 -- 172.21.15.66:0/1860375019 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26400b11e0 con 0x7f26400cec30 2024-03-30T00:41:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.901+0000 7f2644c00640 1 --2- 172.21.15.66:0/1860375019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26400cec30 0x7f2640159c60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f262c009870 tx=0x7f262c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e607d37a7f94198 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.901+0000 7f263fe00640 1 -- 172.21.15.66:0/1860375019 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0aa5f0 con 0x7f26400cec30 2024-03-30T00:41:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.901+0000 7f263fe00640 1 -- 172.21.15.66:0/1860375019 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c0aa750 con 0x7f26400cec30 2024-03-30T00:41:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.901+0000 7f263fe00640 1 -- 172.21.15.66:0/1860375019 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0aaae0 con 0x7f26400cec30 2024-03-30T00:41:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.901+0000 7f2648000640 1 -- 172.21.15.66:0/1860375019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26400cec30 msgr2=0x7f2640159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.901+0000 7f2648000640 1 --2- 172.21.15.66:0/1860375019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26400cec30 0x7f2640159c60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f262c009870 tx=0x7f262c01c490 comp rx=0 tx=0).stop 2024-03-30T00:41:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.901+0000 7f2648000640 1 -- 172.21.15.66:0/1860375019 shutdown_connections 2024-03-30T00:41:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.901+0000 7f2648000640 1 --2- 172.21.15.66:0/1860375019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26400cec30 0x7f2640159c60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.901+0000 7f2648000640 1 -- 172.21.15.66:0/1860375019 >> 172.21.15.66:0/1860375019 conn(0x7f2640069f60 msgr2=0x7f264006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:03.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.901+0000 7f2648000640 1 -- 172.21.15.66:0/1860375019 shutdown_connections 2024-03-30T00:41:03.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.901+0000 7f2648000640 1 -- 172.21.15.66:0/1860375019 wait complete. 2024-03-30T00:41:03.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2648000640 1 Processor -- start 2024-03-30T00:41:03.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2648000640 1 -- start start 2024-03-30T00:41:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2648000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26400cda90 0x7f26401560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2648000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26400cde70 0x7f26400ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2648000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26400cec30 0x7f2640158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2648000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2640158900 con 0x7f26400cda90 2024-03-30T00:41:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2648000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2640169180 con 0x7f26400cde70 2024-03-30T00:41:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2648000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26400ac6f0 con 0x7f26400cec30 2024-03-30T00:41:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2645600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26400cda90 0x7f26401560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2645600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26400cda90 0x7f26401560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52666/0 (socket says 172.21.15.66:52666) 2024-03-30T00:41:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2645600640 1 -- 172.21.15.66:0/2269056029 learned_addr learned my addr 172.21.15.66:0/2269056029 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:03.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2644c00640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26400cde70 0x7f26400ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2646000640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26400cec30 0x7f2640158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2645600640 1 -- 172.21.15.66:0/2269056029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26400cec30 msgr2=0x7f2640158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2645600640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26400cec30 0x7f2640158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2645600640 1 -- 172.21.15.66:0/2269056029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26400cde70 msgr2=0x7f26400ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2645600640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26400cde70 0x7f26400ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2645600640 1 -- 172.21.15.66:0/2269056029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c0094d0 con 0x7f26400cda90 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2645600640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26400cda90 0x7f26401560b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f263400c3c0 tx=0x7f263400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f263d600640 1 -- 172.21.15.66:0/2269056029 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26340052d0 con 0x7f26400cda90 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f263d600640 1 -- 172.21.15.66:0/2269056029 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2634005430 con 0x7f26400cda90 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f263d600640 1 -- 172.21.15.66:0/2269056029 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634003840 con 0x7f26400cda90 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2648000640 1 -- 172.21.15.66:0/2269056029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26400ac9d0 con 0x7f26400cda90 2024-03-30T00:41:03.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.905+0000 7f2648000640 1 -- 172.21.15.66:0/2269056029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26400ac380 con 0x7f26400cda90 2024-03-30T00:41:03.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.909+0000 7f2648000640 1 -- 172.21.15.66:0/2269056029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26401586f0 con 0x7f26400cda90 2024-03-30T00:41:03.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.909+0000 7f263d600640 1 -- 172.21.15.66:0/2269056029 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f26340039a0 con 0x7f26400cda90 2024-03-30T00:41:03.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.909+0000 7f263d600640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2620066690 0x7f2620068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:03.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.909+0000 7f263d600640 1 -- 172.21.15.66:0/2269056029 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f2634083b70 con 0x7f26400cda90 2024-03-30T00:41:03.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.909+0000 7f2644c00640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2620066690 0x7f2620068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:03.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.909+0000 7f2644c00640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2620066690 0x7f2620068b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f262c0ae000 tx=0x7f262c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:03.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:03.913+0000 7f263d600640 1 -- 172.21.15.66:0/2269056029 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f26401586f0 con 0x7f26400cda90 2024-03-30T00:41:04.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.073+0000 7f2648000640 1 -- 172.21.15.66:0/2269056029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26401586f0 con 0x7f26400cda90 2024-03-30T00:41:04.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.073+0000 7f263d600640 1 -- 172.21.15.66:0/2269056029 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26401586f0 con 0x7f26400cda90 2024-03-30T00:41:04.083 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:41:04.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 -- 172.21.15.66:0/2269056029 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2620066690 msgr2=0x7f2620068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2620066690 0x7f2620068b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f262c0ae000 tx=0x7f262c0a7000 comp rx=0 tx=0).stop 2024-03-30T00:41:04.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 -- 172.21.15.66:0/2269056029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26400cda90 msgr2=0x7f26401560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26400cda90 0x7f26401560b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f263400c3c0 tx=0x7f263400c880 comp rx=0 tx=0).stop 2024-03-30T00:41:04.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 -- 172.21.15.66:0/2269056029 shutdown_connections 2024-03-30T00:41:04.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26400cec30 0x7f2640158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26400cde70 0x7f26400ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2620066690 0x7f2620068b50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 --2- 172.21.15.66:0/2269056029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26400cda90 0x7f26401560b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 -- 172.21.15.66:0/2269056029 >> 172.21.15.66:0/2269056029 conn(0x7f2640069f60 msgr2=0x7f264006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:04.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 -- 172.21.15.66:0/2269056029 shutdown_connections 2024-03-30T00:41:04.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.077+0000 7f2648000640 1 -- 172.21.15.66:0/2269056029 wait complete. 2024-03-30T00:41:04.104 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:41:04.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.237+0000 7fa2f5000640 1 Processor -- start 2024-03-30T00:41:04.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.237+0000 7fa2f5000640 1 -- start start 2024-03-30T00:41:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.237+0000 7fa2f5000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f015a670 0x7fa2f00b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.237+0000 7fa2f5000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa2f00d01d0 0x7fa2f00d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.237+0000 7fa2f5000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f00cb520 con 0x7fa2f00ce710 2024-03-30T00:41:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.237+0000 7fa2f5000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f00d0ba0 con 0x7fa2f00d01d0 2024-03-30T00:41:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.237+0000 7fa2f5000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f00cd500 con 0x7fa2f015a670 2024-03-30T00:41:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.241+0000 7fa2ee800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa2f00d01d0 0x7fa2f00d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.241+0000 7fa2ee800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa2f00d01d0 0x7fa2f00d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:41742/0 (socket says 172.21.15.66:41742) 2024-03-30T00:41:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.241+0000 7fa2ee800640 1 -- 172.21.15.66:0/3145761442 learned_addr learned my addr 172.21.15.66:0/3145761442 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:04.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.241+0000 7fa2ede00640 1 --2- 172.21.15.66:0/3145761442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f015a670 0x7fa2f00b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.241+0000 7fa2ee800640 1 -- 172.21.15.66:0/3145761442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f015a670 msgr2=0x7fa2f00b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:04.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.241+0000 7fa2ee800640 1 --2- 172.21.15.66:0/3145761442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f015a670 0x7fa2f00b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.241+0000 7fa2ee800640 1 -- 172.21.15.66:0/3145761442 >> v1:172.21.15.156:6789/0 conn(0x7fa2f00ce710 legacy=0x7fa2f00ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.241+0000 7fa2ee800640 1 -- 172.21.15.66:0/3145761442 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f00cd7d0 con 0x7fa2f00d01d0 2024-03-30T00:41:04.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.241+0000 7fa2ee800640 1 --2- 172.21.15.66:0/3145761442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2f00d01d0 0x7fa2f00d0660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009e50 tx=0x7fa2e801ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85cdd8e1837e6a7a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:04.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2ed400640 1 -- 172.21.15.66:0/3145761442 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e80a58d0 con 0x7fa2f00d01d0 2024-03-30T00:41:04.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2ed400640 1 -- 172.21.15.66:0/3145761442 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e80a5a30 con 0x7fa2f00d01d0 2024-03-30T00:41:04.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2ed400640 1 -- 172.21.15.66:0/3145761442 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e80a5dc0 con 0x7fa2f00d01d0 2024-03-30T00:41:04.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 -- 172.21.15.66:0/3145761442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2f00d01d0 msgr2=0x7fa2f00d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 --2- 172.21.15.66:0/3145761442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2f00d01d0 0x7fa2f00d0660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009e50 tx=0x7fa2e801ca00 comp rx=0 tx=0).stop 2024-03-30T00:41:04.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 -- 172.21.15.66:0/3145761442 shutdown_connections 2024-03-30T00:41:04.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 --2- 172.21.15.66:0/3145761442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2f00d01d0 0x7fa2f00d0660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 --2- 172.21.15.66:0/3145761442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f015a670 0x7fa2f00b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 -- 172.21.15.66:0/3145761442 >> 172.21.15.66:0/3145761442 conn(0x7fa2f0099870 msgr2=0x7fa2f0089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:04.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 -- 172.21.15.66:0/3145761442 shutdown_connections 2024-03-30T00:41:04.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 -- 172.21.15.66:0/3145761442 wait complete. 2024-03-30T00:41:04.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 Processor -- start 2024-03-30T00:41:04.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 -- start start 2024-03-30T00:41:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f00ce710 0x7fa2f0158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2f00d01d0 0x7fa2f01513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f015a670 0x7fa2f0164750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f00c4310 con 0x7fa2f015a670 2024-03-30T00:41:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f0158560 con 0x7fa2f00d01d0 2024-03-30T00:41:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2f5000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f0158790 con 0x7fa2f00ce710 2024-03-30T00:41:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2ede00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f00ce710 0x7fa2f0158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2ee800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f015a670 0x7fa2f0164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2ede00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f00ce710 0x7fa2f0158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:39674/0 (socket says 172.21.15.66:39674) 2024-03-30T00:41:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2ede00640 1 -- 172.21.15.66:0/2952938437 learned_addr learned my addr 172.21.15.66:0/2952938437 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.245+0000 7fa2e7e00640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2f00d01d0 0x7fa2f01513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2ede00640 1 -- 172.21.15.66:0/2952938437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2f00d01d0 msgr2=0x7fa2f01513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2ede00640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2f00d01d0 0x7fa2f01513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2ede00640 1 -- 172.21.15.66:0/2952938437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f015a670 msgr2=0x7fa2f0164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2ede00640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f015a670 0x7fa2f0164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2ede00640 1 -- 172.21.15.66:0/2952938437 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e8009b30 con 0x7fa2f00ce710 2024-03-30T00:41:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2ee800640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f015a670 0x7fa2f0164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2ede00640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f00ce710 0x7fa2f0158020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000e870 tx=0x7fa2e000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2e6000640 1 -- 172.21.15.66:0/2952938437 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0009510 con 0x7fa2f00ce710 2024-03-30T00:41:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2e6000640 1 -- 172.21.15.66:0/2952938437 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e000eec0 con 0x7fa2f00ce710 2024-03-30T00:41:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2e6000640 1 -- 172.21.15.66:0/2952938437 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e000cb40 con 0x7fa2f00ce710 2024-03-30T00:41:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2f5000640 1 -- 172.21.15.66:0/2952938437 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f015abd0 con 0x7fa2f00ce710 2024-03-30T00:41:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2f5000640 1 -- 172.21.15.66:0/2952938437 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f006b2b0 con 0x7fa2f00ce710 2024-03-30T00:41:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2f5000640 1 -- 172.21.15.66:0/2952938437 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f006bd90 con 0x7fa2f00ce710 2024-03-30T00:41:04.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2e6000640 1 -- 172.21.15.66:0/2952938437 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa2e000cca0 con 0x7fa2f00ce710 2024-03-30T00:41:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2e6000640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa2cc0666c0 0x7fa2cc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.249+0000 7fa2e6000640 1 -- 172.21.15.66:0/2952938437 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fa2e00845d0 con 0x7fa2f00ce710 2024-03-30T00:41:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.253+0000 7fa2e7e00640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa2cc0666c0 0x7fa2cc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.253+0000 7fa2e7e00640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa2cc0666c0 0x7fa2cc068b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa2f01652e0 tx=0x7fa2d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:04.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.257+0000 7fa2e6000640 1 -- 172.21.15.66:0/2952938437 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa2e00515e0 con 0x7fa2f00ce710 2024-03-30T00:41:04.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.413+0000 7fa2f5000640 1 -- 172.21.15.66:0/2952938437 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2f0066c30 con 0x7fa2f00ce710 2024-03-30T00:41:04.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2e6000640 1 -- 172.21.15.66:0/2952938437 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2e0054c00 con 0x7fa2f00ce710 2024-03-30T00:41:04.425 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:41:04.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 -- 172.21.15.66:0/2952938437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa2cc0666c0 msgr2=0x7fa2cc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa2cc0666c0 0x7fa2cc068b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa2f01652e0 tx=0x7fa2d8009040 comp rx=0 tx=0).stop 2024-03-30T00:41:04.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 -- 172.21.15.66:0/2952938437 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f00ce710 msgr2=0x7fa2f0158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f00ce710 0x7fa2f0158020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000e870 tx=0x7fa2e000ec70 comp rx=0 tx=0).stop 2024-03-30T00:41:04.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 -- 172.21.15.66:0/2952938437 shutdown_connections 2024-03-30T00:41:04.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa2cc0666c0 0x7fa2cc068b80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f015a670 0x7fa2f0164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2f00d01d0 0x7fa2f01513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 --2- 172.21.15.66:0/2952938437 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2f00ce710 0x7fa2f0158020 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 -- 172.21.15.66:0/2952938437 >> 172.21.15.66:0/2952938437 conn(0x7fa2f0099870 msgr2=0x7fa2f0155cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:04.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 -- 172.21.15.66:0/2952938437 shutdown_connections 2024-03-30T00:41:04.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.417+0000 7fa2f5000640 1 -- 172.21.15.66:0/2952938437 wait complete. 2024-03-30T00:41:04.446 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:41:04.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.581+0000 7fed8de00640 1 Processor -- start 2024-03-30T00:41:04.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.581+0000 7fed8de00640 1 -- start start 2024-03-30T00:41:04.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.581+0000 7fed8de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 0x7fed880b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.581+0000 7fed8de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8815cda0 con 0x7fed88160490 2024-03-30T00:41:04.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.581+0000 7fed8de00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed880afdf0 con 0x7fed8815c200 2024-03-30T00:41:04.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.581+0000 7fed8de00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880b15d0 con 0x7fed880b0290 2024-03-30T00:41:04.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.581+0000 7fed86a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fed88160490 0x7fed88160870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48622/0 (socket says 172.21.15.66:48622) 2024-03-30T00:41:04.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.581+0000 7fed86a00640 1 -- 172.21.15.66:0/2158015664 learned_addr learned my addr 172.21.15.66:0/2158015664 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:04.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.581+0000 7fed7fe00640 1 --2- 172.21.15.66:0/2158015664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 0x7fed880b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.581+0000 7fed86000640 1 -- 172.21.15.66:0/2158015664 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406769520 0 0) 0x7fed8815cda0 con 0x7fed88160490 2024-03-30T00:41:04.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.585+0000 7fed86000640 1 -- 172.21.15.66:0/2158015664 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed6c003530 con 0x7fed88160490 2024-03-30T00:41:04.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.585+0000 7fed86000640 1 -- 172.21.15.66:0/2158015664 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2108776295 0 0) 0x7fed880afdf0 con 0x7fed8815c200 2024-03-30T00:41:04.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.585+0000 7fed86000640 1 -- 172.21.15.66:0/2158015664 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed8815cda0 con 0x7fed8815c200 2024-03-30T00:41:04.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.585+0000 7fed7fe00640 1 -- 172.21.15.66:0/2158015664 >> v1:172.21.15.156:6789/0 conn(0x7fed8815c200 legacy=0x7fed8815c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.585+0000 7fed7fe00640 1 -- 172.21.15.66:0/2158015664 >> v1:172.21.15.66:6789/0 conn(0x7fed88160490 legacy=0x7fed88160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.585+0000 7fed7fe00640 1 -- 172.21.15.66:0/2158015664 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed880b1010 con 0x7fed880b0290 2024-03-30T00:41:04.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.585+0000 7fed7fe00640 1 --2- 172.21.15.66:0/2158015664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 0x7fed880b06f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed700097f0 tx=0x7fed7001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73e193856db3b433 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:04.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.585+0000 7fed86000640 1 -- 172.21.15.66:0/2158015664 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed700aa610 con 0x7fed880b0290 2024-03-30T00:41:04.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.585+0000 7fed86000640 1 -- 172.21.15.66:0/2158015664 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed700a6040 con 0x7fed880b0290 2024-03-30T00:41:04.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.585+0000 7fed86000640 1 -- 172.21.15.66:0/2158015664 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed700aa990 con 0x7fed880b0290 2024-03-30T00:41:04.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 -- 172.21.15.66:0/2158015664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 msgr2=0x7fed880b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 --2- 172.21.15.66:0/2158015664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 0x7fed880b06f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed700097f0 tx=0x7fed7001c4b0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 -- 172.21.15.66:0/2158015664 shutdown_connections 2024-03-30T00:41:04.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 --2- 172.21.15.66:0/2158015664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 0x7fed880b06f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 -- 172.21.15.66:0/2158015664 >> 172.21.15.66:0/2158015664 conn(0x7fed88099870 msgr2=0x7fed88089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:04.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 -- 172.21.15.66:0/2158015664 shutdown_connections 2024-03-30T00:41:04.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 -- 172.21.15.66:0/2158015664 wait complete. 2024-03-30T00:41:04.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 Processor -- start 2024-03-30T00:41:04.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 -- start start 2024-03-30T00:41:04.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 0x7fed880ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed8815c200 0x7fed8815f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed88160490 0x7fed880c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c8d90 con 0x7fed8815c200 2024-03-30T00:41:04.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8815fe40 con 0x7fed88160490 2024-03-30T00:41:04.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed8de00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c55a0 con 0x7fed880b0290 2024-03-30T00:41:04.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed7fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed8815c200 0x7fed8815f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed7fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed8815c200 0x7fed8815f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52680/0 (socket says 172.21.15.66:52680) 2024-03-30T00:41:04.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed7fe00640 1 -- 172.21.15.66:0/3789382509 learned_addr learned my addr 172.21.15.66:0/3789382509 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:04.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed86a00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 0x7fed880ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed87400640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed88160490 0x7fed880c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed7fe00640 1 -- 172.21.15.66:0/3789382509 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 msgr2=0x7fed880ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed7fe00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 0x7fed880ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed7fe00640 1 -- 172.21.15.66:0/3789382509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed88160490 msgr2=0x7fed880c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed7fe00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed88160490 0x7fed880c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed7fe00640 1 -- 172.21.15.66:0/3789382509 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70009420 con 0x7fed8815c200 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed86a00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 0x7fed880ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.589+0000 7fed7fe00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed8815c200 0x7fed8815f900 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fed700ac040 tx=0x7fed7001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.593+0000 7fed7f400640 1 -- 172.21.15.66:0/3789382509 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed700be410 con 0x7fed8815c200 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.593+0000 7fed7f400640 1 -- 172.21.15.66:0/3789382509 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed700be5b0 con 0x7fed8815c200 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.593+0000 7fed7f400640 1 -- 172.21.15.66:0/3789382509 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed700be960 con 0x7fed8815c200 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.593+0000 7fed8de00640 1 -- 172.21.15.66:0/3789382509 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed880c8710 con 0x7fed8815c200 2024-03-30T00:41:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.593+0000 7fed8de00640 1 -- 172.21.15.66:0/3789382509 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed880c89b0 con 0x7fed8815c200 2024-03-30T00:41:04.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.593+0000 7fed8de00640 1 -- 172.21.15.66:0/3789382509 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed88066c30 con 0x7fed8815c200 2024-03-30T00:41:04.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.593+0000 7fed7f400640 1 -- 172.21.15.66:0/3789382509 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fed700b4400 con 0x7fed8815c200 2024-03-30T00:41:04.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.593+0000 7fed7f400640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fed64066690 0x7fed64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.593+0000 7fed7f400640 1 -- 172.21.15.66:0/3789382509 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fed7011ec10 con 0x7fed8815c200 2024-03-30T00:41:04.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.593+0000 7fed86a00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fed64066690 0x7fed64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.597+0000 7fed86a00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fed64066690 0x7fed64068b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fed78000f70 tx=0x7fed78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:04.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.597+0000 7fed7f400640 1 -- 172.21.15.66:0/3789382509 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fed700ebae0 con 0x7fed8815c200 2024-03-30T00:41:04.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.761+0000 7fed8de00640 1 -- 172.21.15.66:0/3789382509 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed8806fe10 con 0x7fed8815c200 2024-03-30T00:41:04.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.761+0000 7fed7f400640 1 -- 172.21.15.66:0/3789382509 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed700ef100 con 0x7fed8815c200 2024-03-30T00:41:04.771 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.761+0000 7fed8de00640 1 -- 172.21.15.66:0/3789382509 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fed64066690 msgr2=0x7fed64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.761+0000 7fed8de00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fed64066690 0x7fed64068b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fed78000f70 tx=0x7fed78009040 comp rx=0 tx=0).stop 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.761+0000 7fed8de00640 1 -- 172.21.15.66:0/3789382509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed8815c200 msgr2=0x7fed8815f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.761+0000 7fed8de00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed8815c200 0x7fed8815f900 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fed700ac040 tx=0x7fed7001acf0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.765+0000 7fed8de00640 1 -- 172.21.15.66:0/3789382509 shutdown_connections 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.765+0000 7fed8de00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed88160490 0x7fed880c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.765+0000 7fed8de00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fed64066690 0x7fed64068b50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.765+0000 7fed8de00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed8815c200 0x7fed8815f900 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.765+0000 7fed8de00640 1 --2- 172.21.15.66:0/3789382509 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed880b0290 0x7fed880ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.765+0000 7fed8de00640 1 -- 172.21.15.66:0/3789382509 >> 172.21.15.66:0/3789382509 conn(0x7fed88099870 msgr2=0x7fed8806b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.765+0000 7fed8de00640 1 -- 172.21.15.66:0/3789382509 shutdown_connections 2024-03-30T00:41:04.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.765+0000 7fed8de00640 1 -- 172.21.15.66:0/3789382509 wait complete. 2024-03-30T00:41:04.792 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:41:04.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf84e00640 1 Processor -- start 2024-03-30T00:41:04.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf84e00640 1 -- start start 2024-03-30T00:41:04.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf84e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faf800cf2f0 0x7faf8006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf84e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf800cb570 0x7faf800cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf84e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf800cb0e0 con 0x7faf8015ca80 2024-03-30T00:41:04.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf84e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf800cf7f0 con 0x7faf800cf2f0 2024-03-30T00:41:04.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf84e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf800cfad0 con 0x7faf800cb570 2024-03-30T00:41:04.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf75400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faf800cf2f0 0x7faf8006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf75400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faf800cf2f0 0x7faf8006b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:41770/0 (socket says 172.21.15.66:41770) 2024-03-30T00:41:04.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf75400640 1 -- 172.21.15.66:0/3671721769 learned_addr learned my addr 172.21.15.66:0/3671721769 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:04.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf7e800640 1 --2- 172.21.15.66:0/3671721769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf800cb570 0x7faf800cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.925+0000 7faf7de00640 1 --1- 172.21.15.66:0/3671721769 >> v1:172.21.15.156:6789/0 conn(0x7faf8015ca80 0x7faf8006b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:44508/0 (socket says 172.21.15.66:44508) 2024-03-30T00:41:04.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.929+0000 7faf7d400640 1 -- 172.21.15.66:0/3671721769 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083172027 0 0) 0x7faf800cb0e0 con 0x7faf8015ca80 2024-03-30T00:41:04.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.929+0000 7faf7d400640 1 -- 172.21.15.66:0/3671721769 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf64003530 con 0x7faf8015ca80 2024-03-30T00:41:04.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.929+0000 7faf75400640 1 -- 172.21.15.66:0/3671721769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf800cb570 msgr2=0x7faf800cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:04.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.929+0000 7faf75400640 1 --2- 172.21.15.66:0/3671721769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf800cb570 0x7faf800cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.929+0000 7faf75400640 1 -- 172.21.15.66:0/3671721769 >> v1:172.21.15.156:6789/0 conn(0x7faf8015ca80 legacy=0x7faf8006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.929+0000 7faf75400640 1 -- 172.21.15.66:0/3671721769 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf800cad70 con 0x7faf800cf2f0 2024-03-30T00:41:04.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.929+0000 7faf7e800640 1 --2- 172.21.15.66:0/3671721769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf800cb570 0x7faf800cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:04.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.929+0000 7faf75400640 1 --2- 172.21.15.66:0/3671721769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf800cf2f0 0x7faf8006b7c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faf68009870 tx=0x7faf6801ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4fddbf7b8f44488 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:04.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.929+0000 7faf7d400640 1 -- 172.21.15.66:0/3671721769 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf680a8a50 con 0x7faf800cf2f0 2024-03-30T00:41:04.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.929+0000 7faf7d400640 1 -- 172.21.15.66:0/3671721769 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf680a8bb0 con 0x7faf800cf2f0 2024-03-30T00:41:04.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.929+0000 7faf7d400640 1 -- 172.21.15.66:0/3671721769 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf680b0440 con 0x7faf800cf2f0 2024-03-30T00:41:04.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 -- 172.21.15.66:0/3671721769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf800cf2f0 msgr2=0x7faf8006b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 --2- 172.21.15.66:0/3671721769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf800cf2f0 0x7faf8006b7c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faf68009870 tx=0x7faf6801ed50 comp rx=0 tx=0).stop 2024-03-30T00:41:04.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 -- 172.21.15.66:0/3671721769 shutdown_connections 2024-03-30T00:41:04.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 --2- 172.21.15.66:0/3671721769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf800cb570 0x7faf800cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 --2- 172.21.15.66:0/3671721769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf800cf2f0 0x7faf8006b7c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 -- 172.21.15.66:0/3671721769 >> 172.21.15.66:0/3671721769 conn(0x7faf800b2090 msgr2=0x7faf800b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:04.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 -- 172.21.15.66:0/3671721769 shutdown_connections 2024-03-30T00:41:04.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 -- 172.21.15.66:0/3671721769 wait complete. 2024-03-30T00:41:04.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 Processor -- start 2024-03-30T00:41:04.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 -- start start 2024-03-30T00:41:04.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf800cb570 0x7faf80158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf800cf2f0 0x7faf800b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf8015ca80 0x7faf801f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf800b4d80 con 0x7faf800cb570 2024-03-30T00:41:04.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf800b47a0 con 0x7faf8015ca80 2024-03-30T00:41:04.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf84e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf801590f0 con 0x7faf800cf2f0 2024-03-30T00:41:04.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf7de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf800cb570 0x7faf80158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf7de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf800cb570 0x7faf80158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52688/0 (socket says 172.21.15.66:52688) 2024-03-30T00:41:04.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf7de00640 1 -- 172.21.15.66:0/4169731491 learned_addr learned my addr 172.21.15.66:0/4169731491 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:04.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf7e800640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf8015ca80 0x7faf801f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf75400640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf800cf2f0 0x7faf800b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf7de00640 1 -- 172.21.15.66:0/4169731491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf800cf2f0 msgr2=0x7faf800b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf7de00640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf800cf2f0 0x7faf800b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf7de00640 1 -- 172.21.15.66:0/4169731491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf8015ca80 msgr2=0x7faf801f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:04.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf7de00640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf8015ca80 0x7faf801f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:04.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf7de00640 1 -- 172.21.15.66:0/4169731491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf78009a70 con 0x7faf800cb570 2024-03-30T00:41:04.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.933+0000 7faf7de00640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf800cb570 0x7faf80158bb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c420 tx=0x7faf7000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:04.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.937+0000 7faf76a00640 1 -- 172.21.15.66:0/4169731491 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf700074e0 con 0x7faf800cb570 2024-03-30T00:41:04.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.937+0000 7faf76a00640 1 -- 172.21.15.66:0/4169731491 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf70007660 con 0x7faf800cb570 2024-03-30T00:41:04.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.937+0000 7faf76a00640 1 -- 172.21.15.66:0/4169731491 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70004a30 con 0x7faf800cb570 2024-03-30T00:41:04.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.937+0000 7faf84e00640 1 -- 172.21.15.66:0/4169731491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf680094d0 con 0x7faf800cb570 2024-03-30T00:41:04.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.937+0000 7faf84e00640 1 -- 172.21.15.66:0/4169731491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf800b4a10 con 0x7faf800cb570 2024-03-30T00:41:04.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.937+0000 7faf84e00640 1 -- 172.21.15.66:0/4169731491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf80066c30 con 0x7faf800cb570 2024-03-30T00:41:04.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.937+0000 7faf76a00640 1 -- 172.21.15.66:0/4169731491 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faf70004b90 con 0x7faf800cb570 2024-03-30T00:41:04.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.937+0000 7faf76a00640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faf5c066690 0x7faf5c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:04.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.937+0000 7faf76a00640 1 -- 172.21.15.66:0/4169731491 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7faf70084060 con 0x7faf800cb570 2024-03-30T00:41:04.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.937+0000 7faf75400640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faf5c066690 0x7faf5c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:04.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.937+0000 7faf75400640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faf5c066690 0x7faf5c068b50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf680b2000 tx=0x7faf68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:04.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:04.941+0000 7faf76a00640 1 -- 172.21.15.66:0/4169731491 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faf70050fd0 con 0x7faf800cb570 2024-03-30T00:41:05.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.101+0000 7faf84e00640 1 -- 172.21.15.66:0/4169731491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faf800b2bb0 con 0x7faf800cb570 2024-03-30T00:41:05.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.101+0000 7faf76a00640 1 -- 172.21.15.66:0/4169731491 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7faf700545f0 con 0x7faf800cb570 2024-03-30T00:41:05.111 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:41:05.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 -- 172.21.15.66:0/4169731491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faf5c066690 msgr2=0x7faf5c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faf5c066690 0x7faf5c068b50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf680b2000 tx=0x7faf68008040 comp rx=0 tx=0).stop 2024-03-30T00:41:05.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 -- 172.21.15.66:0/4169731491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf800cb570 msgr2=0x7faf80158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf800cb570 0x7faf80158bb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c420 tx=0x7faf7000c8e0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 -- 172.21.15.66:0/4169731491 shutdown_connections 2024-03-30T00:41:05.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf8015ca80 0x7faf801f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf800cf2f0 0x7faf800b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faf5c066690 0x7faf5c068b50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 --2- 172.21.15.66:0/4169731491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf800cb570 0x7faf80158bb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 -- 172.21.15.66:0/4169731491 >> 172.21.15.66:0/4169731491 conn(0x7faf800b2090 msgr2=0x7faf800cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:05.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 -- 172.21.15.66:0/4169731491 shutdown_connections 2024-03-30T00:41:05.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.105+0000 7faf84e00640 1 -- 172.21.15.66:0/4169731491 wait complete. 2024-03-30T00:41:05.133 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:41:05.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c2a600640 1 Processor -- start 2024-03-30T00:41:05.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c2a600640 1 -- start start 2024-03-30T00:41:05.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c2a600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6c24157060 0x7f6c2406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c2a600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c241610c0 0x7f6c241604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c2a600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2406dc60 con 0x7f6c241609f0 2024-03-30T00:41:05.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c2a600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c24157440 con 0x7f6c241610c0 2024-03-30T00:41:05.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c2a600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c24160e80 con 0x7f6c24157060 2024-03-30T00:41:05.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c1be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c241610c0 0x7f6c241604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:05.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c1be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c241610c0 0x7f6c241604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52700/0 (socket says 172.21.15.66:52700) 2024-03-30T00:41:05.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c1be00640 1 -- 172.21.15.66:0/1499403573 learned_addr learned my addr 172.21.15.66:0/1499403573 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:05.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c23400640 1 --2- 172.21.15.66:0/1499403573 >> v2:172.21.15.156:3300/0 conn(0x7f6c24157060 0x7f6c2406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:05.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c1be00640 1 -- 172.21.15.66:0/1499403573 >> v2:172.21.15.156:3300/0 conn(0x7f6c24157060 msgr2=0x7f6c2406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:05.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c1be00640 1 --2- 172.21.15.66:0/1499403573 >> v2:172.21.15.156:3300/0 conn(0x7f6c24157060 0x7f6c2406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c1be00640 1 -- 172.21.15.66:0/1499403573 >> v1:172.21.15.156:6789/0 conn(0x7f6c241609f0 legacy=0x7f6c2415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:05.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c1be00640 1 -- 172.21.15.66:0/1499403573 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2406df40 con 0x7f6c241610c0 2024-03-30T00:41:05.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.265+0000 7f6c23400640 1 --2- 172.21.15.66:0/1499403573 >> v2:172.21.15.156:3300/0 conn(0x7f6c24157060 0x7f6c2406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:05.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c1be00640 1 --2- 172.21.15.66:0/1499403573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241610c0 0x7f6c241604b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6c24072710 tx=0x7f6c0c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d4823f0c149064 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:05.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c22a00640 1 -- 172.21.15.66:0/1499403573 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2415dde0 con 0x7f6c241610c0 2024-03-30T00:41:05.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c22a00640 1 -- 172.21.15.66:0/1499403573 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c0a7050 con 0x7f6c241610c0 2024-03-30T00:41:05.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c22a00640 1 -- 172.21.15.66:0/1499403573 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0ab820 con 0x7f6c241610c0 2024-03-30T00:41:05.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c2a600640 1 -- 172.21.15.66:0/1499403573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241610c0 msgr2=0x7f6c241604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c2a600640 1 --2- 172.21.15.66:0/1499403573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241610c0 0x7f6c241604b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6c24072710 tx=0x7f6c0c01c3c0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c2a600640 1 -- 172.21.15.66:0/1499403573 shutdown_connections 2024-03-30T00:41:05.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c2a600640 1 --2- 172.21.15.66:0/1499403573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241610c0 0x7f6c241604b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c2a600640 1 --2- 172.21.15.66:0/1499403573 >> v2:172.21.15.156:3300/0 conn(0x7f6c24157060 0x7f6c2406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c2a600640 1 -- 172.21.15.66:0/1499403573 >> 172.21.15.66:0/1499403573 conn(0x7f6c24071750 msgr2=0x7f6c24069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:05.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c2a600640 1 -- 172.21.15.66:0/1499403573 shutdown_connections 2024-03-30T00:41:05.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.269+0000 7f6c2a600640 1 -- 172.21.15.66:0/1499403573 wait complete. 2024-03-30T00:41:05.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c2a600640 1 Processor -- start 2024-03-30T00:41:05.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c2a600640 1 -- start start 2024-03-30T00:41:05.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c2a600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24157060 0x7f6c2414cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c2a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241609f0 0x7f6c2415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c2a600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c241610c0 0x7f6c2415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c2a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240ce450 con 0x7f6c241609f0 2024-03-30T00:41:05.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c2a600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241550c0 con 0x7f6c24157060 2024-03-30T00:41:05.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c2a600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2415f550 con 0x7f6c241610c0 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241609f0 0x7f6c2415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241609f0 0x7f6c2415e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52712/0 (socket says 172.21.15.66:52712) 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1be00640 1 -- 172.21.15.66:0/2788667542 learned_addr learned my addr 172.21.15.66:0/2788667542 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c23400640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24157060 0x7f6c2414cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c23e00640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c241610c0 0x7f6c2415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1be00640 1 -- 172.21.15.66:0/2788667542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c241610c0 msgr2=0x7f6c2415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1be00640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c241610c0 0x7f6c2415ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1be00640 1 -- 172.21.15.66:0/2788667542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24157060 msgr2=0x7f6c2414cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1be00640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24157060 0x7f6c2414cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1be00640 1 -- 172.21.15.66:0/2788667542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c0094d0 con 0x7f6c241609f0 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1be00640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241609f0 0x7f6c2415e9e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c01aa70 tx=0x7f6c0c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:05.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1b400640 1 -- 172.21.15.66:0/2788667542 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0b4870 con 0x7f6c241609f0 2024-03-30T00:41:05.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1b400640 1 -- 172.21.15.66:0/2788667542 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c0b49f0 con 0x7f6c241609f0 2024-03-30T00:41:05.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c1b400640 1 -- 172.21.15.66:0/2788667542 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0b4da0 con 0x7f6c241609f0 2024-03-30T00:41:05.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c2a600640 1 -- 172.21.15.66:0/2788667542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2415f830 con 0x7f6c241609f0 2024-03-30T00:41:05.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c2a600640 1 -- 172.21.15.66:0/2788667542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2415fdf0 con 0x7f6c241609f0 2024-03-30T00:41:05.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.273+0000 7f6c2a600640 1 -- 172.21.15.66:0/2788667542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c24066c30 con 0x7f6c241609f0 2024-03-30T00:41:05.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.277+0000 7f6c1b400640 1 -- 172.21.15.66:0/2788667542 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6c0c0b5440 con 0x7f6c241609f0 2024-03-30T00:41:05.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.277+0000 7f6c1b400640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c00066690 0x7f6c00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.277+0000 7f6c1b400640 1 -- 172.21.15.66:0/2788667542 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f6c0c11ea60 con 0x7f6c241609f0 2024-03-30T00:41:05.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.277+0000 7f6c23400640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c00066690 0x7f6c00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:05.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.277+0000 7f6c23400640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c00066690 0x7f6c00068b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6c14006fd0 tx=0x7f6c14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:05.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.281+0000 7f6c1b400640 1 -- 172.21.15.66:0/2788667542 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6c0c0eba70 con 0x7f6c241609f0 2024-03-30T00:41:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.449+0000 7f6c2a600640 1 -- 172.21.15.66:0/2788667542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6c24071f30 con 0x7f6c241609f0 2024-03-30T00:41:05.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c1b400640 1 -- 172.21.15.66:0/2788667542 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6c0c0ef090 con 0x7f6c241609f0 2024-03-30T00:41:05.461 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:41:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 -- 172.21.15.66:0/2788667542 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c00066690 msgr2=0x7f6c00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c00066690 0x7f6c00068b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6c14006fd0 tx=0x7f6c14008040 comp rx=0 tx=0).stop 2024-03-30T00:41:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 -- 172.21.15.66:0/2788667542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241609f0 msgr2=0x7f6c2415e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241609f0 0x7f6c2415e9e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c01aa70 tx=0x7f6c0c019160 comp rx=0 tx=0).stop 2024-03-30T00:41:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 -- 172.21.15.66:0/2788667542 shutdown_connections 2024-03-30T00:41:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c241610c0 0x7f6c2415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c00066690 0x7f6c00068b50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241609f0 0x7f6c2415e9e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 --2- 172.21.15.66:0/2788667542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c24157060 0x7f6c2414cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 -- 172.21.15.66:0/2788667542 >> 172.21.15.66:0/2788667542 conn(0x7f6c24071750 msgr2=0x7f6c241532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 -- 172.21.15.66:0/2788667542 shutdown_connections 2024-03-30T00:41:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.453+0000 7f6c2a600640 1 -- 172.21.15.66:0/2788667542 wait complete. 2024-03-30T00:41:05.482 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi156.front.sepia.ceph.com... 2024-03-30T00:41:05.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:05.482 DEBUG:teuthology.orchestra.run.smithi156:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-03-30T00:41:05.505 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:41:05.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:41:05.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:05.505 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-30T00:41:05.561 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-30T00:41:05.561 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:05.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cf2400640 1 Processor -- start 2024-03-30T00:41:05.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cf2400640 1 -- start start 2024-03-30T00:41:05.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cf2400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cec06e800 0x7f6cec0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cf2400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6cec06f8e0 0x7f6cec0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cf2400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cec06b0b0 con 0x7f6cec0cc560 2024-03-30T00:41:05.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cf2400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec160640 con 0x7f6cec06f8e0 2024-03-30T00:41:05.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cf2400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec06b390 con 0x7f6cec06e800 2024-03-30T00:41:05.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6ceb400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6cec0cc560 0x7f6cec06e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48630/0 (socket says 172.21.15.66:48630) 2024-03-30T00:41:05.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6ceb400640 1 -- 172.21.15.66:0/2770642178 learned_addr learned my addr 172.21.15.66:0/2770642178 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:05.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cebe00640 1 --2- 172.21.15.66:0/2770642178 >> v2:172.21.15.156:3300/0 conn(0x7f6cec06f8e0 0x7f6cec0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:05.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6ce3e00640 1 --2- 172.21.15.66:0/2770642178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cec06e800 0x7f6cec0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:05.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6ceaa00640 1 -- 172.21.15.66:0/2770642178 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091320638 0 0) 0x7f6cec06b0b0 con 0x7f6cec0cc560 2024-03-30T00:41:05.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6ceaa00640 1 -- 172.21.15.66:0/2770642178 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cd0003530 con 0x7f6cec0cc560 2024-03-30T00:41:05.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cebe00640 1 -- 172.21.15.66:0/2770642178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cec06e800 msgr2=0x7f6cec0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:05.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cebe00640 1 --2- 172.21.15.66:0/2770642178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cec06e800 0x7f6cec0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cebe00640 1 -- 172.21.15.66:0/2770642178 >> v1:172.21.15.66:6789/0 conn(0x7f6cec0cc560 legacy=0x7f6cec06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.693+0000 7f6cebe00640 1 -- 172.21.15.66:0/2770642178 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec06b670 con 0x7f6cec06f8e0 2024-03-30T00:41:05.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6cebe00640 1 --2- 172.21.15.66:0/2770642178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cec06f8e0 0x7f6cec0ce8d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009d90 tx=0x7f6ce401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=def3716e856e0c45 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:05.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6ceaa00640 1 -- 172.21.15.66:0/2770642178 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40a5a00 con 0x7f6cec06f8e0 2024-03-30T00:41:05.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6ceaa00640 1 -- 172.21.15.66:0/2770642178 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce40a5b60 con 0x7f6cec06f8e0 2024-03-30T00:41:05.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6ceaa00640 1 -- 172.21.15.66:0/2770642178 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40ad440 con 0x7f6cec06f8e0 2024-03-30T00:41:05.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6cf2400640 1 -- 172.21.15.66:0/2770642178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cec06f8e0 msgr2=0x7f6cec0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6cf2400640 1 --2- 172.21.15.66:0/2770642178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cec06f8e0 0x7f6cec0ce8d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009d90 tx=0x7f6ce401c940 comp rx=0 tx=0).stop 2024-03-30T00:41:05.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6cf2400640 1 -- 172.21.15.66:0/2770642178 shutdown_connections 2024-03-30T00:41:05.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6cf2400640 1 --2- 172.21.15.66:0/2770642178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cec06f8e0 0x7f6cec0ce8d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6cf2400640 1 --2- 172.21.15.66:0/2770642178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cec06e800 0x7f6cec0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6cf2400640 1 -- 172.21.15.66:0/2770642178 >> 172.21.15.66:0/2770642178 conn(0x7f6cec0d0540 msgr2=0x7f6cec0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:05.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6cf2400640 1 -- 172.21.15.66:0/2770642178 shutdown_connections 2024-03-30T00:41:05.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.697+0000 7f6cf2400640 1 -- 172.21.15.66:0/2770642178 wait complete. 2024-03-30T00:41:05.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6cf2400640 1 Processor -- start 2024-03-30T00:41:05.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6cf2400640 1 -- start start 2024-03-30T00:41:05.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6cf2400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cec06e800 0x7f6cec15fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6cf2400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cec06f8e0 0x7f6cec153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6cf2400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cec0cc560 0x7f6cec0b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6cf2400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec15bfe0 con 0x7f6cec06e800 2024-03-30T00:41:05.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6cf2400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec1536d0 con 0x7f6cec0cc560 2024-03-30T00:41:05.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6cf2400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec15b6b0 con 0x7f6cec06f8e0 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ceb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cec06e800 0x7f6cec15fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ceb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cec06e800 0x7f6cec15fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52728/0 (socket says 172.21.15.66:52728) 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ceb400640 1 -- 172.21.15.66:0/3328591125 learned_addr learned my addr 172.21.15.66:0/3328591125 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ce3e00640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cec06f8e0 0x7f6cec153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ceb400640 1 -- 172.21.15.66:0/3328591125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cec06f8e0 msgr2=0x7f6cec153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ceb400640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cec06f8e0 0x7f6cec153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ceb400640 1 -- 172.21.15.66:0/3328591125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cec0cc560 msgr2=0x7f6cec0b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ceb400640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cec0cc560 0x7f6cec0b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ceb400640 1 -- 172.21.15.66:0/3328591125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4009a70 con 0x7f6cec06e800 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ceb400640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cec06e800 0x7f6cec15fca0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c3c0 tx=0x7f6cdc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ce3400640 1 -- 172.21.15.66:0/3328591125 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc005cf0 con 0x7f6cec06e800 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ce3400640 1 -- 172.21.15.66:0/3328591125 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc005e50 con 0x7f6cec06e800 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6ce3400640 1 -- 172.21.15.66:0/3328591125 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0072f0 con 0x7f6cec06e800 2024-03-30T00:41:05.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6cf2400640 1 -- 172.21.15.66:0/3328591125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec15b990 con 0x7f6cec06e800 2024-03-30T00:41:05.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6cf2400640 1 -- 172.21.15.66:0/3328591125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec15f0e0 con 0x7f6cec06e800 2024-03-30T00:41:05.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.701+0000 7f6cf2400640 1 -- 172.21.15.66:0/3328591125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec066c30 con 0x7f6cec06e800 2024-03-30T00:41:05.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.705+0000 7f6ce3400640 1 -- 172.21.15.66:0/3328591125 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6cdc007450 con 0x7f6cec06e800 2024-03-30T00:41:05.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.705+0000 7f6ce3400640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6cc8066690 0x7f6cc8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:05.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.705+0000 7f6ce3400640 1 -- 172.21.15.66:0/3328591125 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f6cdc083c20 con 0x7f6cec06e800 2024-03-30T00:41:05.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.705+0000 7f6ce3e00640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6cc8066690 0x7f6cc8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:05.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.705+0000 7f6ce3e00640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6cc8066690 0x7f6cc8068b50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6cec1556b0 tx=0x7f6cd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:05.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.709+0000 7f6ce3400640 1 -- 172.21.15.66:0/3328591125 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6cdc050af0 con 0x7f6cec06e800 2024-03-30T00:41:05.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.901+0000 7f6cf2400640 1 -- 172.21.15.66:0/3328591125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cec06e290 con 0x7f6cec06e800 2024-03-30T00:41:05.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6ce3400640 1 -- 172.21.15.66:0/3328591125 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f6cdc017070 con 0x7f6cec06e800 2024-03-30T00:41:05.913 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:05.913 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4122431016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":4122431016},{"type":"v1","addr":"172.21.15.156:6835","nonce":4122431016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3882342907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3882342907},{"type":"v1","addr":"172.21.15.66:6835","nonce":3882342907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:39:51.142575+0000","modified":"2024-03-30T00:40:14.500142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/703895528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":703895528},{"type":"v1","addr":"172.21.15.66:6837","nonce":703895528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:41:05.913 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2024-03-30T00:41:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 -- 172.21.15.66:0/3328591125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6cc8066690 msgr2=0x7f6cc8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6cc8066690 0x7f6cc8068b50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6cec1556b0 tx=0x7f6cd4008040 comp rx=0 tx=0).stop 2024-03-30T00:41:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 -- 172.21.15.66:0/3328591125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cec06e800 msgr2=0x7f6cec15fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cec06e800 0x7f6cec15fca0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c3c0 tx=0x7f6cdc00c880 comp rx=0 tx=0).stop 2024-03-30T00:41:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 -- 172.21.15.66:0/3328591125 shutdown_connections 2024-03-30T00:41:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cec0cc560 0x7f6cec0b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cec06f8e0 0x7f6cec153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6cc8066690 0x7f6cc8068b50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 --2- 172.21.15.66:0/3328591125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cec06e800 0x7f6cec15fca0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 -- 172.21.15.66:0/3328591125 >> 172.21.15.66:0/3328591125 conn(0x7f6cec0d0540 msgr2=0x7f6cec0adec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 -- 172.21.15.66:0/3328591125 shutdown_connections 2024-03-30T00:41:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:05.905+0000 7f6cf2400640 1 -- 172.21.15.66:0/3328591125 wait complete. 2024-03-30T00:41:05.934 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:06.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff11dc00640 1 Processor -- start 2024-03-30T00:41:06.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff11dc00640 1 -- start start 2024-03-30T00:41:06.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff11dc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1180cda90 0x7ff1180ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff11dc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff11815a370 0x7ff1180cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff11dc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11815a990 con 0x7ff1180cde70 2024-03-30T00:41:06.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff11dc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1180b1b80 con 0x7ff11815a370 2024-03-30T00:41:06.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff11dc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1180cd3e0 con 0x7ff1180cda90 2024-03-30T00:41:06.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff10e000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff1180cde70 0x7ff118159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48640/0 (socket says 172.21.15.66:48640) 2024-03-30T00:41:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff10e000640 1 -- 172.21.15.66:0/4015821440 learned_addr learned my addr 172.21.15.66:0/4015821440 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff117400640 1 --2- 172.21.15.66:0/4015821440 >> v2:172.21.15.156:3300/0 conn(0x7ff11815a370 0x7ff1180cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff116a00640 1 --2- 172.21.15.66:0/4015821440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1180cda90 0x7ff1180ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff116000640 1 -- 172.21.15.66:0/4015821440 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875630610 0 0) 0x7ff11815a990 con 0x7ff1180cde70 2024-03-30T00:41:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.065+0000 7ff116000640 1 -- 172.21.15.66:0/4015821440 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0fc003530 con 0x7ff1180cde70 2024-03-30T00:41:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.069+0000 7ff116000640 1 -- 172.21.15.66:0/4015821440 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3152831940 0 0) 0x7ff0fc003530 con 0x7ff1180cde70 2024-03-30T00:41:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.069+0000 7ff116a00640 1 -- 172.21.15.66:0/4015821440 >> v2:172.21.15.156:3300/0 conn(0x7ff11815a370 msgr2=0x7ff1180cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.069+0000 7ff116a00640 1 --2- 172.21.15.66:0/4015821440 >> v2:172.21.15.156:3300/0 conn(0x7ff11815a370 0x7ff1180cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.069+0000 7ff116a00640 1 -- 172.21.15.66:0/4015821440 >> v1:172.21.15.66:6789/0 conn(0x7ff1180cde70 legacy=0x7ff118159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.069+0000 7ff116a00640 1 -- 172.21.15.66:0/4015821440 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1180b11e0 con 0x7ff1180cda90 2024-03-30T00:41:06.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.069+0000 7ff116a00640 1 --2- 172.21.15.66:0/4015821440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1180cda90 0x7ff1180ce520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1080097c0 tx=0x7ff10801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e2efdc8c575a4e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:06.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.069+0000 7ff116000640 1 -- 172.21.15.66:0/4015821440 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100001680 con 0x7ff1180cda90 2024-03-30T00:41:06.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.069+0000 7ff116000640 1 -- 172.21.15.66:0/4015821440 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1080a7050 con 0x7ff1180cda90 2024-03-30T00:41:06.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.069+0000 7ff116000640 1 -- 172.21.15.66:0/4015821440 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1080ab620 con 0x7ff1180cda90 2024-03-30T00:41:06.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 -- 172.21.15.66:0/4015821440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1180cda90 msgr2=0x7ff1180ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 --2- 172.21.15.66:0/4015821440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1180cda90 0x7ff1180ce520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1080097c0 tx=0x7ff10801c330 comp rx=0 tx=0).stop 2024-03-30T00:41:06.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 -- 172.21.15.66:0/4015821440 shutdown_connections 2024-03-30T00:41:06.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 --2- 172.21.15.66:0/4015821440 >> v2:172.21.15.156:3300/0 conn(0x7ff11815a370 0x7ff1180cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 --2- 172.21.15.66:0/4015821440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1180cda90 0x7ff1180ce520 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 -- 172.21.15.66:0/4015821440 >> 172.21.15.66:0/4015821440 conn(0x7ff118069f60 msgr2=0x7ff11806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:06.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 -- 172.21.15.66:0/4015821440 shutdown_connections 2024-03-30T00:41:06.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 -- 172.21.15.66:0/4015821440 wait complete. 2024-03-30T00:41:06.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 Processor -- start 2024-03-30T00:41:06.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 -- start start 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1180cda90 0x7ff1180ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1180cde70 0x7ff118156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff11815a370 0x7ff1180ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff118071560 con 0x7ff11815a370 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1180ab550 con 0x7ff1180cda90 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff11dc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1180ab7f0 con 0x7ff1180cde70 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff117400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff11815a370 0x7ff1180ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff117400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff11815a370 0x7ff1180ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52730/0 (socket says 172.21.15.66:52730) 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff117400640 1 -- 172.21.15.66:0/1631411206 learned_addr learned my addr 172.21.15.66:0/1631411206 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff117400640 1 -- 172.21.15.66:0/1631411206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1180cde70 msgr2=0x7ff118156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff117400640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1180cde70 0x7ff118156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff117400640 1 -- 172.21.15.66:0/1631411206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1180cda90 msgr2=0x7ff1180ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff117400640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1180cda90 0x7ff1180ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff117400640 1 -- 172.21.15.66:0/1631411206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff108009420 con 0x7ff11815a370 2024-03-30T00:41:06.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff117400640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff11815a370 0x7ff1180ac890 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff11000efc0 tx=0x7ff11000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:06.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff10f400640 1 -- 172.21.15.66:0/1631411206 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110009ba0 con 0x7ff11815a370 2024-03-30T00:41:06.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff10f400640 1 -- 172.21.15.66:0/1631411206 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11000f040 con 0x7ff11815a370 2024-03-30T00:41:06.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.073+0000 7ff10f400640 1 -- 172.21.15.66:0/1631411206 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1100146c0 con 0x7ff11815a370 2024-03-30T00:41:06.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.077+0000 7ff11dc00640 1 -- 172.21.15.66:0/1631411206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1180abad0 con 0x7ff11815a370 2024-03-30T00:41:06.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.077+0000 7ff11dc00640 1 -- 172.21.15.66:0/1631411206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1180ac090 con 0x7ff11815a370 2024-03-30T00:41:06.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.077+0000 7ff11dc00640 1 -- 172.21.15.66:0/1631411206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff118066c30 con 0x7ff11815a370 2024-03-30T00:41:06.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.077+0000 7ff10f400640 1 -- 172.21.15.66:0/1631411206 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff110014820 con 0x7ff11815a370 2024-03-30T00:41:06.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.077+0000 7ff10f400640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0f4066690 0x7ff0f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.077+0000 7ff10f400640 1 -- 172.21.15.66:0/1631411206 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff110085380 con 0x7ff11815a370 2024-03-30T00:41:06.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.077+0000 7ff116a00640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0f4066690 0x7ff0f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.077+0000 7ff116a00640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0f4066690 0x7ff0f4068b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff1080ad000 tx=0x7ff108004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:06.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.081+0000 7ff10f400640 1 -- 172.21.15.66:0/1631411206 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff1100522e0 con 0x7ff11815a370 2024-03-30T00:41:06.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.241+0000 7ff11dc00640 1 -- 172.21.15.66:0/1631411206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1180b3170 con 0x7ff11815a370 2024-03-30T00:41:06.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff10f400640 1 -- 172.21.15.66:0/1631411206 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7ff110055900 con 0x7ff11815a370 2024-03-30T00:41:06.253 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:06.254 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:06.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 -- 172.21.15.66:0/1631411206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0f4066690 msgr2=0x7ff0f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0f4066690 0x7ff0f4068b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff1080ad000 tx=0x7ff108004740 comp rx=0 tx=0).stop 2024-03-30T00:41:06.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 -- 172.21.15.66:0/1631411206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff11815a370 msgr2=0x7ff1180ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff11815a370 0x7ff1180ac890 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff11000efc0 tx=0x7ff11000c490 comp rx=0 tx=0).stop 2024-03-30T00:41:06.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 -- 172.21.15.66:0/1631411206 shutdown_connections 2024-03-30T00:41:06.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0f4066690 0x7ff0f4068b50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff11815a370 0x7ff1180ac890 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1180cde70 0x7ff118156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 --2- 172.21.15.66:0/1631411206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1180cda90 0x7ff1180ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 -- 172.21.15.66:0/1631411206 >> 172.21.15.66:0/1631411206 conn(0x7ff118069f60 msgr2=0x7ff1180ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:06.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 -- 172.21.15.66:0/1631411206 shutdown_connections 2024-03-30T00:41:06.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.245+0000 7ff11dc00640 1 -- 172.21.15.66:0/1631411206 wait complete. 2024-03-30T00:41:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:39:49.685024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:39:50.136108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T00:39:53.201055+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T00:41:06.275 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:41:06.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e8a00640 1 Processor -- start 2024-03-30T00:41:06.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e8a00640 1 -- start start 2024-03-30T00:41:06.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e8a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6e40cda90 0x7fa6e40ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e8a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa6e40cde70 0x7fa6e40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e8a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e415a990 con 0x7fa6e4159c60 2024-03-30T00:41:06.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e8a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e40b1b80 con 0x7fa6e40cda90 2024-03-30T00:41:06.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e8a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e40cd3e0 con 0x7fa6e40cde70 2024-03-30T00:41:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e1e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6e40cda90 0x7fa6e40ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e1e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6e40cda90 0x7fa6e40ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52740/0 (socket says 172.21.15.66:52740) 2024-03-30T00:41:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e1e00640 1 -- 172.21.15.66:0/3568035548 learned_addr learned my addr 172.21.15.66:0/3568035548 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6dbe00640 1 --2- 172.21.15.66:0/3568035548 >> v2:172.21.15.156:3300/0 conn(0x7fa6e40cde70 0x7fa6e40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e1e00640 1 -- 172.21.15.66:0/3568035548 >> v2:172.21.15.156:3300/0 conn(0x7fa6e40cde70 msgr2=0x7fa6e40cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e1e00640 1 --2- 172.21.15.66:0/3568035548 >> v2:172.21.15.156:3300/0 conn(0x7fa6e40cde70 0x7fa6e40cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e1e00640 1 -- 172.21.15.66:0/3568035548 >> v1:172.21.15.156:6789/0 conn(0x7fa6e4159c60 legacy=0x7fa6e415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6e1e00640 1 -- 172.21.15.66:0/3568035548 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e40b11e0 con 0x7fa6e40cda90 2024-03-30T00:41:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.409+0000 7fa6dbe00640 1 --2- 172.21.15.66:0/3568035548 >> v2:172.21.15.156:3300/0 conn(0x7fa6e40cde70 0x7fa6e40cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:06.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e1e00640 1 --2- 172.21.15.66:0/3568035548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6e40cda90 0x7fa6e40ce520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40cc320 tx=0x7fa6d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5b7f7afab197ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:06.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e1400640 1 -- 172.21.15.66:0/3568035548 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40ad660 con 0x7fa6e40cda90 2024-03-30T00:41:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e1400640 1 -- 172.21.15.66:0/3568035548 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d40ad7c0 con 0x7fa6e40cda90 2024-03-30T00:41:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e1400640 1 -- 172.21.15.66:0/3568035548 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40adb50 con 0x7fa6e40cda90 2024-03-30T00:41:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e8a00640 1 -- 172.21.15.66:0/3568035548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6e40cda90 msgr2=0x7fa6e40ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e8a00640 1 --2- 172.21.15.66:0/3568035548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6e40cda90 0x7fa6e40ce520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40cc320 tx=0x7fa6d401c450 comp rx=0 tx=0).stop 2024-03-30T00:41:06.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e8a00640 1 -- 172.21.15.66:0/3568035548 shutdown_connections 2024-03-30T00:41:06.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e8a00640 1 --2- 172.21.15.66:0/3568035548 >> v2:172.21.15.156:3300/0 conn(0x7fa6e40cde70 0x7fa6e40cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e8a00640 1 --2- 172.21.15.66:0/3568035548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6e40cda90 0x7fa6e40ce520 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e8a00640 1 -- 172.21.15.66:0/3568035548 >> 172.21.15.66:0/3568035548 conn(0x7fa6e4069f60 msgr2=0x7fa6e406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:06.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e8a00640 1 -- 172.21.15.66:0/3568035548 shutdown_connections 2024-03-30T00:41:06.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.413+0000 7fa6e8a00640 1 -- 172.21.15.66:0/3568035548 wait complete. 2024-03-30T00:41:06.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e8a00640 1 Processor -- start 2024-03-30T00:41:06.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e8a00640 1 -- start start 2024-03-30T00:41:06.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e8a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6e40cda90 0x7fa6e40ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6e40cde70 0x7fa6e41f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e8a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e4159c60 0x7fa6e41648a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e8a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e4158470 con 0x7fa6e40cde70 2024-03-30T00:41:06.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e8a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e41f4d00 con 0x7fa6e4159c60 2024-03-30T00:41:06.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e8a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e4163c80 con 0x7fa6e40cda90 2024-03-30T00:41:06.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6dbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6e40cde70 0x7fa6e41f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6dbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6e40cde70 0x7fa6e41f47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52754/0 (socket says 172.21.15.66:52754) 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6dbe00640 1 -- 172.21.15.66:0/640108184 learned_addr learned my addr 172.21.15.66:0/640108184 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e1e00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6e40cda90 0x7fa6e40ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e2800640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e4159c60 0x7fa6e41648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6dbe00640 1 -- 172.21.15.66:0/640108184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6e40cda90 msgr2=0x7fa6e40ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6dbe00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6e40cda90 0x7fa6e40ab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6dbe00640 1 -- 172.21.15.66:0/640108184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e4159c60 msgr2=0x7fa6e41648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6dbe00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e4159c60 0x7fa6e41648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6dbe00640 1 -- 172.21.15.66:0/640108184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d40094d0 con 0x7fa6e40cde70 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6dbe00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6e40cde70 0x7fa6e41f47c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00c7f0 tx=0x7fa6cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6da000640 1 -- 172.21.15.66:0/640108184 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00cea0 con 0x7fa6e40cde70 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6da000640 1 -- 172.21.15.66:0/640108184 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc007a30 con 0x7fa6e40cde70 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6da000640 1 -- 172.21.15.66:0/640108184 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc007dc0 con 0x7fa6e40cde70 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e8a00640 1 -- 172.21.15.66:0/640108184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e41577f0 con 0x7fa6e40cde70 2024-03-30T00:41:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e8a00640 1 -- 172.21.15.66:0/640108184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e4157db0 con 0x7fa6e40cde70 2024-03-30T00:41:06.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.417+0000 7fa6e8a00640 1 -- 172.21.15.66:0/640108184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e4066c30 con 0x7fa6e40cde70 2024-03-30T00:41:06.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.421+0000 7fa6da000640 1 -- 172.21.15.66:0/640108184 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa6cc0113f0 con 0x7fa6e40cde70 2024-03-30T00:41:06.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.421+0000 7fa6da000640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6c0066690 0x7fa6c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.421+0000 7fa6da000640 1 -- 172.21.15.66:0/640108184 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fa6cc084700 con 0x7fa6e40cde70 2024-03-30T00:41:06.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.421+0000 7fa6e1e00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6c0066690 0x7fa6c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.421+0000 7fa6e1e00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6c0066690 0x7fa6c0068b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40af000 tx=0x7fa6d40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:06.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.425+0000 7fa6da000640 1 -- 172.21.15.66:0/640108184 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa6cc051680 con 0x7fa6e40cde70 2024-03-30T00:41:06.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.585+0000 7fa6e8a00640 1 -- 172.21.15.66:0/640108184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6e40b3170 con 0x7fa6e40cde70 2024-03-30T00:41:06.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.585+0000 7fa6da000640 1 -- 172.21.15.66:0/640108184 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6cc054ca0 con 0x7fa6e40cde70 2024-03-30T00:41:06.595 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:41:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.585+0000 7fa6e8a00640 1 -- 172.21.15.66:0/640108184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6c0066690 msgr2=0x7fa6c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.585+0000 7fa6e8a00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6c0066690 0x7fa6c0068b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40af000 tx=0x7fa6d40a7000 comp rx=0 tx=0).stop 2024-03-30T00:41:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.585+0000 7fa6e8a00640 1 -- 172.21.15.66:0/640108184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6e40cde70 msgr2=0x7fa6e41f47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.585+0000 7fa6e8a00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6e40cde70 0x7fa6e41f47c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00c7f0 tx=0x7fa6cc00ccb0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.589+0000 7fa6e8a00640 1 -- 172.21.15.66:0/640108184 shutdown_connections 2024-03-30T00:41:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.589+0000 7fa6e8a00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e4159c60 0x7fa6e41648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.589+0000 7fa6e8a00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6c0066690 0x7fa6c0068b50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.589+0000 7fa6e8a00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6e40cde70 0x7fa6e41f47c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.589+0000 7fa6e8a00640 1 --2- 172.21.15.66:0/640108184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6e40cda90 0x7fa6e40ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.589+0000 7fa6e8a00640 1 -- 172.21.15.66:0/640108184 >> 172.21.15.66:0/640108184 conn(0x7fa6e4069f60 msgr2=0x7fa6e415c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.589+0000 7fa6e8a00640 1 -- 172.21.15.66:0/640108184 shutdown_connections 2024-03-30T00:41:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.589+0000 7fa6e8a00640 1 -- 172.21.15.66:0/640108184 wait complete. 2024-03-30T00:41:06.616 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:41:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.749+0000 7f9e84c00640 1 Processor -- start 2024-03-30T00:41:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.749+0000 7f9e84c00640 1 -- start start 2024-03-30T00:41:06.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.749+0000 7f9e84c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9e8015ca80 0x7f9e8006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.749+0000 7f9e84c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e800cb0e0 con 0x7f9e800cf2f0 2024-03-30T00:41:06.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.749+0000 7f9e84c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e800cf7f0 con 0x7f9e800cb570 2024-03-30T00:41:06.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.749+0000 7f9e84c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e800cfad0 con 0x7f9e8015ca80 2024-03-30T00:41:06.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e77e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9e800cf2f0 0x7f9e8006b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48644/0 (socket says 172.21.15.66:48644) 2024-03-30T00:41:06.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e77e00640 1 -- 172.21.15.66:0/3837109165 learned_addr learned my addr 172.21.15.66:0/3837109165 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7de00640 1 --2- 172.21.15.66:0/3837109165 >> v2:172.21.15.156:3300/0 conn(0x7f9e8015ca80 0x7f9e8006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055696904 0 0) 0x7f9e800cb0e0 con 0x7f9e800cf2f0 2024-03-30T00:41:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e64003530 con 0x7f9e800cf2f0 2024-03-30T00:41:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3357527758 0 0) 0x7f9e64003530 con 0x7f9e800cf2f0 2024-03-30T00:41:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 >> v2:172.21.15.156:3300/0 conn(0x7f9e8015ca80 msgr2=0x7f9e8006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 --2- 172.21.15.66:0/3837109165 >> v2:172.21.15.156:3300/0 conn(0x7f9e8015ca80 0x7f9e8006b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 >> v1:172.21.15.156:6789/0 conn(0x7f9e800cb570 legacy=0x7f9e800cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e800cad70 con 0x7f9e800cf2f0 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f9e68001680 con 0x7f9e800cf2f0 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 >> v1:172.21.15.66:6789/0 conn(0x7f9e800cf2f0 legacy=0x7f9e8006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e64003980 0x7f9e64003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e64004380 0x7f9e64004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64005050 0x7f9e6400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800cfad0 con 0x7f9e64004380 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6400bf00 con 0x7f9e64005050 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6400c0c0 con 0x7f9e64003980 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e77e00640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e64004380 0x7f9e64004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7de00640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e64003980 0x7f9e64003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7e800640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64005050 0x7f9e6400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e77e00640 1 -- 172.21.15.66:0/3837109165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e64003980 msgr2=0x7f9e64003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e77e00640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e64003980 0x7f9e64003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e77e00640 1 -- 172.21.15.66:0/3837109165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64005050 msgr2=0x7f9e6400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e77e00640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64005050 0x7f9e6400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e77e00640 1 -- 172.21.15.66:0/3837109165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e64003530 con 0x7f9e64004380 2024-03-30T00:41:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7de00640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e64003980 0x7f9e64003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7e800640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e64005050 0x7f9e6400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.753+0000 7f9e7e800640 1 -- 172.21.15.66:0/3837109165 reap_dead start 2024-03-30T00:41:06.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e77e00640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e64004380 0x7f9e64004850 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800c210 tx=0x7f9e6801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcede25cd79cdb76 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:06.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68005ae0 con 0x7f9e64004380 2024-03-30T00:41:06.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e680ab070 con 0x7f9e64004380 2024-03-30T00:41:06.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e7d400640 1 -- 172.21.15.66:0/3837109165 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68007070 con 0x7f9e64004380 2024-03-30T00:41:06.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 -- 172.21.15.66:0/3837109165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e64004380 msgr2=0x7f9e64004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e64004380 0x7f9e64004850 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800c210 tx=0x7f9e6801f3b0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 -- 172.21.15.66:0/3837109165 shutdown_connections 2024-03-30T00:41:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 --2- 172.21.15.66:0/3837109165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e64004380 0x7f9e64004850 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 -- 172.21.15.66:0/3837109165 >> 172.21.15.66:0/3837109165 conn(0x7f9e800b2090 msgr2=0x7f9e800b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 -- 172.21.15.66:0/3837109165 shutdown_connections 2024-03-30T00:41:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 -- 172.21.15.66:0/3837109165 wait complete. 2024-03-30T00:41:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 Processor -- start 2024-03-30T00:41:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 -- start start 2024-03-30T00:41:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e64004380 0x7f9e801563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e800c8760 0x7f9e800b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e800b5c20 0x7f9e8015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800b5790 con 0x7f9e800b5c20 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800b4b80 con 0x7f9e800c8760 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.757+0000 7f9e84c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800b4e60 con 0x7f9e64004380 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e7e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e800b5c20 0x7f9e8015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e7e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e800b5c20 0x7f9e8015a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52776/0 (socket says 172.21.15.66:52776) 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e7e800640 1 -- 172.21.15.66:0/1348840200 learned_addr learned my addr 172.21.15.66:0/1348840200 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e7de00640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e64004380 0x7f9e801563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e77e00640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e800c8760 0x7f9e800b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e7e800640 1 -- 172.21.15.66:0/1348840200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e64004380 msgr2=0x7f9e801563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e7e800640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e64004380 0x7f9e801563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e7e800640 1 -- 172.21.15.66:0/1348840200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e800c8760 msgr2=0x7f9e800b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e7e800640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e800c8760 0x7f9e800b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e7e800640 1 -- 172.21.15.66:0/1348840200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e800b5140 con 0x7f9e800b5c20 2024-03-30T00:41:06.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e7e800640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e800b5c20 0x7f9e8015a510 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9e800b3190 tx=0x7f9e7800eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:06.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e76000640 1 -- 172.21.15.66:0/1348840200 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78003980 con 0x7f9e800b5c20 2024-03-30T00:41:06.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e76000640 1 -- 172.21.15.66:0/1348840200 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e78003ae0 con 0x7f9e800b5c20 2024-03-30T00:41:06.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e76000640 1 -- 172.21.15.66:0/1348840200 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78003e90 con 0x7f9e800b5c20 2024-03-30T00:41:06.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e84c00640 1 -- 172.21.15.66:0/1348840200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e800b5420 con 0x7f9e800b5c20 2024-03-30T00:41:06.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e84c00640 1 -- 172.21.15.66:0/1348840200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e800b4750 con 0x7f9e800b5c20 2024-03-30T00:41:06.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e84c00640 1 -- 172.21.15.66:0/1348840200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e800b4940 con 0x7f9e800b5c20 2024-03-30T00:41:06.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.761+0000 7f9e76000640 1 -- 172.21.15.66:0/1348840200 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9e780050d0 con 0x7f9e800b5c20 2024-03-30T00:41:06.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.765+0000 7f9e76000640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e5c0666c0 0x7f9e5c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:06.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.765+0000 7f9e76000640 1 -- 172.21.15.66:0/1348840200 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f9e78084330 con 0x7f9e800b5c20 2024-03-30T00:41:06.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.765+0000 7f9e7de00640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e5c0666c0 0x7f9e5c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:06.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.765+0000 7f9e7de00640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e5c0666c0 0x7f9e5c068b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9e80068fe0 tx=0x7f9e70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:06.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.769+0000 7f9e76000640 1 -- 172.21.15.66:0/1348840200 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9e800b4940 con 0x7f9e800b5c20 2024-03-30T00:41:06.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 -- 172.21.15.66:0/1348840200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e800b4940 con 0x7f9e800b5c20 2024-03-30T00:41:06.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e76000640 1 -- 172.21.15.66:0/1348840200 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e800b4940 con 0x7f9e800b5c20 2024-03-30T00:41:06.941 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:41:06.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 -- 172.21.15.66:0/1348840200 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e5c0666c0 msgr2=0x7f9e5c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e5c0666c0 0x7f9e5c068b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9e80068fe0 tx=0x7f9e70008040 comp rx=0 tx=0).stop 2024-03-30T00:41:06.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 -- 172.21.15.66:0/1348840200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e800b5c20 msgr2=0x7f9e8015a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:06.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e800b5c20 0x7f9e8015a510 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9e800b3190 tx=0x7f9e7800eb70 comp rx=0 tx=0).stop 2024-03-30T00:41:06.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 -- 172.21.15.66:0/1348840200 shutdown_connections 2024-03-30T00:41:06.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e5c0666c0 0x7f9e5c068b80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e800b5c20 0x7f9e8015a510 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e800c8760 0x7f9e800b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 --2- 172.21.15.66:0/1348840200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e64004380 0x7f9e801563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:06.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 -- 172.21.15.66:0/1348840200 >> 172.21.15.66:0/1348840200 conn(0x7f9e800b2090 msgr2=0x7f9e8006e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:06.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 -- 172.21.15.66:0/1348840200 shutdown_connections 2024-03-30T00:41:06.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:06.933+0000 7f9e84c00640 1 -- 172.21.15.66:0/1348840200 wait complete. 2024-03-30T00:41:06.962 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:41:07.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.093+0000 7f917a600640 1 Processor -- start 2024-03-30T00:41:07.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f917a600640 1 -- start start 2024-03-30T00:41:07.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f917a600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9174159310 0x7f91740cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f917a600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f91740cf950 0x7f9174070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f917a600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917415df10 con 0x7f91740b2620 2024-03-30T00:41:07.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f917a600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91740cccd0 con 0x7f9174159310 2024-03-30T00:41:07.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f917a600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917415d300 con 0x7f91740cf950 2024-03-30T00:41:07.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f9179200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9174159310 0x7f91740cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f9179200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9174159310 0x7f91740cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52790/0 (socket says 172.21.15.66:52790) 2024-03-30T00:41:07.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f9179200640 1 -- 172.21.15.66:0/4229740764 learned_addr learned my addr 172.21.15.66:0/4229740764 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:07.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f9179c00640 1 --2- 172.21.15.66:0/4229740764 >> v2:172.21.15.156:3300/0 conn(0x7f91740cf950 0x7f9174070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f9179200640 1 -- 172.21.15.66:0/4229740764 >> v2:172.21.15.156:3300/0 conn(0x7f91740cf950 msgr2=0x7f9174070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:07.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f9179200640 1 --2- 172.21.15.66:0/4229740764 >> v2:172.21.15.156:3300/0 conn(0x7f91740cf950 0x7f9174070fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f9179200640 1 -- 172.21.15.66:0/4229740764 >> v1:172.21.15.156:6789/0 conn(0x7f91740b2620 legacy=0x7f91740cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:07.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.097+0000 7f9179200640 1 -- 172.21.15.66:0/4229740764 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917415d5e0 con 0x7f9174159310 2024-03-30T00:41:07.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f9179200640 1 --2- 172.21.15.66:0/4229740764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9174159310 0x7f91740cc790 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f91740cd0a0 tx=0x7f916401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2e3ec3657f8fea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:07.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f9173e00640 1 -- 172.21.15.66:0/4229740764 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91640a8a10 con 0x7f9174159310 2024-03-30T00:41:07.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f9173e00640 1 -- 172.21.15.66:0/4229740764 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91640a8b70 con 0x7f9174159310 2024-03-30T00:41:07.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f9173e00640 1 -- 172.21.15.66:0/4229740764 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91640b0440 con 0x7f9174159310 2024-03-30T00:41:07.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f917a600640 1 -- 172.21.15.66:0/4229740764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9174159310 msgr2=0x7f91740cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f917a600640 1 --2- 172.21.15.66:0/4229740764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9174159310 0x7f91740cc790 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f91740cd0a0 tx=0x7f916401ed10 comp rx=0 tx=0).stop 2024-03-30T00:41:07.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f917a600640 1 -- 172.21.15.66:0/4229740764 shutdown_connections 2024-03-30T00:41:07.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f917a600640 1 --2- 172.21.15.66:0/4229740764 >> v2:172.21.15.156:3300/0 conn(0x7f91740cf950 0x7f9174070fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f917a600640 1 --2- 172.21.15.66:0/4229740764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9174159310 0x7f91740cc790 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f917a600640 1 -- 172.21.15.66:0/4229740764 >> 172.21.15.66:0/4229740764 conn(0x7f917406cf90 msgr2=0x7f917406d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:07.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f917a600640 1 -- 172.21.15.66:0/4229740764 shutdown_connections 2024-03-30T00:41:07.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f917a600640 1 -- 172.21.15.66:0/4229740764 wait complete. 2024-03-30T00:41:07.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f917a600640 1 Processor -- start 2024-03-30T00:41:07.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.101+0000 7f917a600640 1 -- start start 2024-03-30T00:41:07.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f917a600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91740b2620 0x7f91740ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f917a600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91740cf950 0x7f91741572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f917a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9174159310 0x7f9174155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f917a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9174157800 con 0x7f9174159310 2024-03-30T00:41:07.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f917a600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91740acf00 con 0x7f91740b2620 2024-03-30T00:41:07.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f917a600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91740ad1e0 con 0x7f91740cf950 2024-03-30T00:41:07.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9179c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9174159310 0x7f9174155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9179c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9174159310 0x7f9174155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52806/0 (socket says 172.21.15.66:52806) 2024-03-30T00:41:07.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9179c00640 1 -- 172.21.15.66:0/1109969950 learned_addr learned my addr 172.21.15.66:0/1109969950 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:07.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9179200640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91740b2620 0x7f91740ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f916be00640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91740cf950 0x7f91741572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9179c00640 1 -- 172.21.15.66:0/1109969950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91740cf950 msgr2=0x7f91741572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9179c00640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91740cf950 0x7f91741572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9179c00640 1 -- 172.21.15.66:0/1109969950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91740b2620 msgr2=0x7f91740ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9179c00640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91740b2620 0x7f91740ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9179c00640 1 -- 172.21.15.66:0/1109969950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91640094d0 con 0x7f9174159310 2024-03-30T00:41:07.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9179200640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91740b2620 0x7f91740ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:07.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9179c00640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9174159310 0x7f9174155cf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f916c007e30 tx=0x7f916c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:07.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9171600640 1 -- 172.21.15.66:0/1109969950 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c014620 con 0x7f9174159310 2024-03-30T00:41:07.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9171600640 1 -- 172.21.15.66:0/1109969950 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c00ee90 con 0x7f9174159310 2024-03-30T00:41:07.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9171600640 1 -- 172.21.15.66:0/1109969950 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0149c0 con 0x7f9174159310 2024-03-30T00:41:07.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f917a600640 1 -- 172.21.15.66:0/1109969950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91740ad4c0 con 0x7f9174159310 2024-03-30T00:41:07.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f917a600640 1 -- 172.21.15.66:0/1109969950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91741513f0 con 0x7f9174159310 2024-03-30T00:41:07.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f917a600640 1 -- 172.21.15.66:0/1109969950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9174073e00 con 0x7f9174159310 2024-03-30T00:41:07.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.105+0000 7f9171600640 1 -- 172.21.15.66:0/1109969950 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f916c014b20 con 0x7f9174159310 2024-03-30T00:41:07.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.109+0000 7f9171600640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9150066690 0x7f9150068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.109+0000 7f9171600640 1 -- 172.21.15.66:0/1109969950 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f916c084ee0 con 0x7f9174159310 2024-03-30T00:41:07.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.109+0000 7f9179200640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9150066690 0x7f9150068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.109+0000 7f9179200640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9150066690 0x7f9150068b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91640b2000 tx=0x7f9164008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:07.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.113+0000 7f9171600640 1 -- 172.21.15.66:0/1109969950 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f916c051ee0 con 0x7f9174159310 2024-03-30T00:41:07.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.277+0000 7f917a600640 1 -- 172.21.15.66:0/1109969950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91740b4750 con 0x7f9174159310 2024-03-30T00:41:07.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.277+0000 7f9171600640 1 -- 172.21.15.66:0/1109969950 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f916c055500 con 0x7f9174159310 2024-03-30T00:41:07.289 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:41:07.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 -- 172.21.15.66:0/1109969950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9150066690 msgr2=0x7f9150068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9150066690 0x7f9150068b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91640b2000 tx=0x7f9164008040 comp rx=0 tx=0).stop 2024-03-30T00:41:07.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 -- 172.21.15.66:0/1109969950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9174159310 msgr2=0x7f9174155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9174159310 0x7f9174155cf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f916c007e30 tx=0x7f916c00e4b0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 -- 172.21.15.66:0/1109969950 shutdown_connections 2024-03-30T00:41:07.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9150066690 0x7f9150068b50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9174159310 0x7f9174155cf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91740cf950 0x7f91741572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 --2- 172.21.15.66:0/1109969950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91740b2620 0x7f91740ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 -- 172.21.15.66:0/1109969950 >> 172.21.15.66:0/1109969950 conn(0x7f917406cf90 msgr2=0x7f91740aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:07.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 -- 172.21.15.66:0/1109969950 shutdown_connections 2024-03-30T00:41:07.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.281+0000 7f917a600640 1 -- 172.21.15.66:0/1109969950 wait complete. 2024-03-30T00:41:07.310 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T00:41:07.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f5520200640 1 Processor -- start 2024-03-30T00:41:07.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f5520200640 1 -- start start 2024-03-30T00:41:07.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f5520200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55180cec30 0x7f5518159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f5520200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f55180cde70 0x7f551815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f5520200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551815a6e0 con 0x7f55180cda90 2024-03-30T00:41:07.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f5520200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55180b1b80 con 0x7f55180cde70 2024-03-30T00:41:07.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f5520200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55180cd3e0 con 0x7f55180cec30 2024-03-30T00:41:07.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f551e200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f55180cde70 0x7f551815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f551e200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f55180cde70 0x7f551815a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52814/0 (socket says 172.21.15.66:52814) 2024-03-30T00:41:07.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f551e200640 1 -- 172.21.15.66:0/1688311848 learned_addr learned my addr 172.21.15.66:0/1688311848 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:07.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f551ce00640 1 --2- 172.21.15.66:0/1688311848 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55180cec30 0x7f5518159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f551e200640 1 -- 172.21.15.66:0/1688311848 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55180cec30 msgr2=0x7f5518159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:07.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f551e200640 1 --2- 172.21.15.66:0/1688311848 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55180cec30 0x7f5518159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f551e200640 1 -- 172.21.15.66:0/1688311848 >> v1:172.21.15.156:6789/0 conn(0x7f55180cda90 legacy=0x7f55180ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.445+0000 7f551e200640 1 -- 172.21.15.66:0/1688311848 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551815a990 con 0x7f55180cde70 2024-03-30T00:41:07.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f551e200640 1 --2- 172.21.15.66:0/1688311848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55180cde70 0x7f551815a1a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f551000d150 tx=0x7f551001eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436f85708f866fca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:07.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5517e00640 1 -- 172.21.15.66:0/1688311848 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551000b2d0 con 0x7f55180cde70 2024-03-30T00:41:07.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5517e00640 1 -- 172.21.15.66:0/1688311848 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551000b430 con 0x7f55180cde70 2024-03-30T00:41:07.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5517e00640 1 -- 172.21.15.66:0/1688311848 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55100a7c90 con 0x7f55180cde70 2024-03-30T00:41:07.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5520200640 1 -- 172.21.15.66:0/1688311848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55180cde70 msgr2=0x7f551815a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5520200640 1 --2- 172.21.15.66:0/1688311848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55180cde70 0x7f551815a1a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f551000d150 tx=0x7f551001eba0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5520200640 1 -- 172.21.15.66:0/1688311848 shutdown_connections 2024-03-30T00:41:07.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5520200640 1 --2- 172.21.15.66:0/1688311848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55180cde70 0x7f551815a1a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5520200640 1 --2- 172.21.15.66:0/1688311848 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55180cec30 0x7f5518159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5520200640 1 -- 172.21.15.66:0/1688311848 >> 172.21.15.66:0/1688311848 conn(0x7f5518069f60 msgr2=0x7f551806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:07.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5520200640 1 -- 172.21.15.66:0/1688311848 shutdown_connections 2024-03-30T00:41:07.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5520200640 1 -- 172.21.15.66:0/1688311848 wait complete. 2024-03-30T00:41:07.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.449+0000 7f5520200640 1 Processor -- start 2024-03-30T00:41:07.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5520200640 1 -- start start 2024-03-30T00:41:07.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5520200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55180cda90 0x7f551806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5520200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55180cde70 0x7f55180b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5520200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55180cec30 0x7f55180ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5520200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55180ac220 con 0x7f55180cec30 2024-03-30T00:41:07.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5520200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5518071ea0 con 0x7f55180cda90 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5520200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5518071600 con 0x7f55180cde70 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f551e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55180cec30 0x7f55180ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f551e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55180cec30 0x7f55180ac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52826/0 (socket says 172.21.15.66:52826) 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f551e200640 1 -- 172.21.15.66:0/778508616 learned_addr learned my addr 172.21.15.66:0/778508616 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f551d800640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55180cda90 0x7f551806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f551ce00640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55180cde70 0x7f55180b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f551e200640 1 -- 172.21.15.66:0/778508616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55180cde70 msgr2=0x7f55180b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f551e200640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55180cde70 0x7f55180b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f551e200640 1 -- 172.21.15.66:0/778508616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55180cda90 msgr2=0x7f551806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f551e200640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55180cda90 0x7f551806d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f551e200640 1 -- 172.21.15.66:0/778508616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551000bd40 con 0x7f55180cec30 2024-03-30T00:41:07.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f551e200640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55180cec30 0x7f55180ac6b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f551001c7d0 tx=0x7f551001b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:07.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5515600640 1 -- 172.21.15.66:0/778508616 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510009800 con 0x7f55180cec30 2024-03-30T00:41:07.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5515600640 1 -- 172.21.15.66:0/778508616 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5510009960 con 0x7f55180cec30 2024-03-30T00:41:07.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5515600640 1 -- 172.21.15.66:0/778508616 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510009d10 con 0x7f55180cec30 2024-03-30T00:41:07.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5520200640 1 -- 172.21.15.66:0/778508616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55180718e0 con 0x7f55180cec30 2024-03-30T00:41:07.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5520200640 1 -- 172.21.15.66:0/778508616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5518071260 con 0x7f55180cec30 2024-03-30T00:41:07.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.453+0000 7f5520200640 1 -- 172.21.15.66:0/778508616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5518066c30 con 0x7f55180cec30 2024-03-30T00:41:07.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.457+0000 7f5515600640 1 -- 172.21.15.66:0/778508616 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f55100ba470 con 0x7f55180cec30 2024-03-30T00:41:07.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.457+0000 7f5515600640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54f8066690 0x7f54f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.457+0000 7f5515600640 1 -- 172.21.15.66:0/778508616 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f5510121d00 con 0x7f55180cec30 2024-03-30T00:41:07.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.457+0000 7f551d800640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54f8066690 0x7f54f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.457+0000 7f551d800640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54f8066690 0x7f54f8068b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f550c000f70 tx=0x7f550c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:07.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.461+0000 7f5515600640 1 -- 172.21.15.66:0/778508616 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f55100eec80 con 0x7f55180cec30 2024-03-30T00:41:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.621+0000 7f5520200640 1 -- 172.21.15.66:0/778508616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f55180ac360 con 0x7f55180cec30 2024-03-30T00:41:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.621+0000 7f5515600640 1 -- 172.21.15.66:0/778508616 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f55100f22a0 con 0x7f55180cec30 2024-03-30T00:41:07.632 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 -- 172.21.15.66:0/778508616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54f8066690 msgr2=0x7f54f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54f8066690 0x7f54f8068b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f550c000f70 tx=0x7f550c009040 comp rx=0 tx=0).stop 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 -- 172.21.15.66:0/778508616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55180cec30 msgr2=0x7f55180ac6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55180cec30 0x7f55180ac6b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f551001c7d0 tx=0x7f551001b940 comp rx=0 tx=0).stop 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 -- 172.21.15.66:0/778508616 shutdown_connections 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54f8066690 0x7f54f8068b50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55180cec30 0x7f55180ac6b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55180cde70 0x7f55180b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 --2- 172.21.15.66:0/778508616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55180cda90 0x7f551806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 -- 172.21.15.66:0/778508616 >> 172.21.15.66:0/778508616 conn(0x7f5518069f60 msgr2=0x7f551815b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 -- 172.21.15.66:0/778508616 shutdown_connections 2024-03-30T00:41:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.625+0000 7f5520200640 1 -- 172.21.15.66:0/778508616 wait complete. 2024-03-30T00:41:07.652 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:07.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff34cc00640 1 Processor -- start 2024-03-30T00:41:07.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff34cc00640 1 -- start start 2024-03-30T00:41:07.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff34cc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff34815a640 0x7ff34806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff34cc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34806d070 con 0x7ff3480cb2c0 2024-03-30T00:41:07.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff34cc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3480cbba0 con 0x7ff3480ce520 2024-03-30T00:41:07.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff34cc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34806e2c0 con 0x7ff34815a640 2024-03-30T00:41:07.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff346800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff3480cb2c0 0x7ff3480ab010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48652/0 (socket says 172.21.15.66:48652) 2024-03-30T00:41:07.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff346800640 1 -- 172.21.15.66:0/2135559006 learned_addr learned my addr 172.21.15.66:0/2135559006 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345e00640 1 --2- 172.21.15.66:0/2135559006 >> v2:172.21.15.156:3300/0 conn(0x7ff34815a640 0x7ff34806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30246014 0 0) 0x7ff34806d070 con 0x7ff3480cb2c0 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff32c003530 con 0x7ff3480cb2c0 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1239352950 0 0) 0x7ff32c003530 con 0x7ff3480cb2c0 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 >> v2:172.21.15.156:3300/0 conn(0x7ff34815a640 msgr2=0x7ff34806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 --2- 172.21.15.66:0/2135559006 >> v2:172.21.15.156:3300/0 conn(0x7ff34815a640 0x7ff34806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 >> v1:172.21.15.156:6789/0 conn(0x7ff3480ce520 legacy=0x7ff3480cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3480cb8c0 con 0x7ff3480cb2c0 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7ff340001a00 con 0x7ff3480cb2c0 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 >> v1:172.21.15.66:6789/0 conn(0x7ff3480cb2c0 legacy=0x7ff3480ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 --2- 172.21.15.66:0/2135559006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32c003980 0x7ff32c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 --2- 172.21.15.66:0/2135559006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff32c004380 0x7ff32c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 --2- 172.21.15.66:0/2135559006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff32c005050 0x7ff32c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34806e2c0 con 0x7ff32c003980 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c00bfb0 con 0x7ff32c004380 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c00c170 con 0x7ff32c005050 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345e00640 1 --2- 172.21.15.66:0/2135559006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32c003980 0x7ff32c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff33d400640 1 --2- 172.21.15.66:0/2135559006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff32c004380 0x7ff32c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff346800640 1 --2- 172.21.15.66:0/2135559006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff32c005050 0x7ff32c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345e00640 1 -- 172.21.15.66:0/2135559006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff32c005050 msgr2=0x7ff32c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345e00640 1 --2- 172.21.15.66:0/2135559006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff32c005050 0x7ff32c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345e00640 1 -- 172.21.15.66:0/2135559006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff32c004380 msgr2=0x7ff32c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345e00640 1 --2- 172.21.15.66:0/2135559006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff32c004380 0x7ff32c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff346800640 1 -- 172.21.15.66:0/2135559006 reap_dead start 2024-03-30T00:41:07.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.785+0000 7ff345e00640 1 -- 172.21.15.66:0/2135559006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c003530 con 0x7ff32c003980 2024-03-30T00:41:07.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.789+0000 7ff345e00640 1 --2- 172.21.15.66:0/2135559006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32c003980 0x7ff32c003d60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff338009ae0 tx=0x7ff33801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6a6b70e1bf4608 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:07.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.789+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3380a5680 con 0x7ff32c003980 2024-03-30T00:41:07.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.789+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3380a57e0 con 0x7ff32c003980 2024-03-30T00:41:07.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.789+0000 7ff345400640 1 -- 172.21.15.66:0/2135559006 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3380a5b90 con 0x7ff32c003980 2024-03-30T00:41:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.789+0000 7ff34cc00640 1 -- 172.21.15.66:0/2135559006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32c003980 msgr2=0x7ff32c003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.789+0000 7ff34cc00640 1 --2- 172.21.15.66:0/2135559006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32c003980 0x7ff32c003d60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff338009ae0 tx=0x7ff33801ca90 comp rx=0 tx=0).stop 2024-03-30T00:41:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.789+0000 7ff34cc00640 1 -- 172.21.15.66:0/2135559006 shutdown_connections 2024-03-30T00:41:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.789+0000 7ff34cc00640 1 --2- 172.21.15.66:0/2135559006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32c003980 0x7ff32c003d60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.789+0000 7ff34cc00640 1 -- 172.21.15.66:0/2135559006 >> 172.21.15.66:0/2135559006 conn(0x7ff348069f60 msgr2=0x7ff34806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.789+0000 7ff34cc00640 1 -- 172.21.15.66:0/2135559006 shutdown_connections 2024-03-30T00:41:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.789+0000 7ff34cc00640 1 -- 172.21.15.66:0/2135559006 wait complete. 2024-03-30T00:41:07.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff34cc00640 1 Processor -- start 2024-03-30T00:41:07.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff34cc00640 1 -- start start 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff34cc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff32c003980 0x7ff3481593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff34cc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff34815bcd0 0x7ff3480c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff34cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34815c150 0x7ff3480c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff34cc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480c83c0 con 0x7ff34815c150 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff34cc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3481f23b0 con 0x7ff32c003980 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff34cc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3481f2690 con 0x7ff34815bcd0 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff346800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34815c150 0x7ff3480c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff346800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34815c150 0x7ff3480c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52832/0 (socket says 172.21.15.66:52832) 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff346800640 1 -- 172.21.15.66:0/3233890574 learned_addr learned my addr 172.21.15.66:0/3233890574 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff33d400640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff34815bcd0 0x7ff3480c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff345e00640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff32c003980 0x7ff3481593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff346800640 1 -- 172.21.15.66:0/3233890574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff34815bcd0 msgr2=0x7ff3480c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff346800640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff34815bcd0 0x7ff3480c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff346800640 1 -- 172.21.15.66:0/3233890574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff32c003980 msgr2=0x7ff3481593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff346800640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff32c003980 0x7ff3481593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff346800640 1 -- 172.21.15.66:0/3233890574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3481f2970 con 0x7ff34815c150 2024-03-30T00:41:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff346800640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34815c150 0x7ff3480c9d40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff340000f90 tx=0x7ff340007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff33ea00640 1 -- 172.21.15.66:0/3233890574 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34000ac80 con 0x7ff34815c150 2024-03-30T00:41:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff33ea00640 1 -- 172.21.15.66:0/3233890574 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34000ae00 con 0x7ff34815c150 2024-03-30T00:41:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff33ea00640 1 -- 172.21.15.66:0/3233890574 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340013630 con 0x7ff34815c150 2024-03-30T00:41:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff34cc00640 1 -- 172.21.15.66:0/3233890574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3481f2c50 con 0x7ff34815c150 2024-03-30T00:41:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.793+0000 7ff34cc00640 1 -- 172.21.15.66:0/3233890574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3481598e0 con 0x7ff34815c150 2024-03-30T00:41:07.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.797+0000 7ff34cc00640 1 -- 172.21.15.66:0/3233890574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff348066c30 con 0x7ff34815c150 2024-03-30T00:41:07.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.797+0000 7ff33ea00640 1 -- 172.21.15.66:0/3233890574 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff340013790 con 0x7ff34815c150 2024-03-30T00:41:07.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.797+0000 7ff33ea00640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 0x7ff324068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:07.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.797+0000 7ff33ea00640 1 -- 172.21.15.66:0/3233890574 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff340084da0 con 0x7ff34815c150 2024-03-30T00:41:07.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.797+0000 7ff345e00640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 0x7ff324068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:07.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.797+0000 7ff345e00640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 0x7ff324068b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff3380af000 tx=0x7ff338008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.801+0000 7ff33ea00640 1 -- 172.21.15.66:0/3233890574 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff340051c70 con 0x7ff34815c150 2024-03-30T00:41:08.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 -- 172.21.15.66:0/3233890574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3480af9a0 con 0x7ff34815c150 2024-03-30T00:41:08.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff33ea00640 1 -- 172.21.15.66:0/3233890574 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4730 (secure 0 0 0) 0x7ff340055290 con 0x7ff34815c150 2024-03-30T00:41:08.006 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:08.007 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4122431016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":4122431016},{"type":"v1","addr":"172.21.15.156:6835","nonce":4122431016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3882342907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3882342907},{"type":"v1","addr":"172.21.15.66:6835","nonce":3882342907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:39:51.142575+0000","modified":"2024-03-30T00:40:14.500142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/703895528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":703895528},{"type":"v1","addr":"172.21.15.66:6837","nonce":703895528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:41:08.007 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2024-03-30T00:41:08.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 -- 172.21.15.66:0/3233890574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 msgr2=0x7ff324068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 0x7ff324068b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff3380af000 tx=0x7ff338008040 comp rx=0 tx=0).stop 2024-03-30T00:41:08.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 -- 172.21.15.66:0/3233890574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34815c150 msgr2=0x7ff3480c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34815c150 0x7ff3480c9d40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff340000f90 tx=0x7ff340007bd0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 -- 172.21.15.66:0/3233890574 shutdown_connections 2024-03-30T00:41:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 0x7ff324068b50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff34815c150 0x7ff3480c9d40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff34815bcd0 0x7ff3480c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 --2- 172.21.15.66:0/3233890574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff32c003980 0x7ff3481593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 -- 172.21.15.66:0/3233890574 >> 172.21.15.66:0/3233890574 conn(0x7ff348069f60 msgr2=0x7ff3480af290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 -- 172.21.15.66:0/3233890574 shutdown_connections 2024-03-30T00:41:08.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:07.997+0000 7ff34cc00640 1 -- 172.21.15.66:0/3233890574 wait complete. 2024-03-30T00:41:08.027 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:08.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.157+0000 7fe536000640 1 Processor -- start 2024-03-30T00:41:08.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.157+0000 7fe536000640 1 -- start start 2024-03-30T00:41:08.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe536000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5300cf2f0 0x7fe53006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe536000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe5300cb570 0x7fe5300cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe536000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5300cb0e0 con 0x7fe53015ca80 2024-03-30T00:41:08.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe536000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5300cf7f0 con 0x7fe5300cb570 2024-03-30T00:41:08.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe536000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300cfad0 con 0x7fe5300cf2f0 2024-03-30T00:41:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe52fe00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe5300cb570 0x7fe5300cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe526a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5300cf2f0 0x7fe53006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe52fe00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe5300cb570 0x7fe5300cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:41922/0 (socket says 172.21.15.66:41922) 2024-03-30T00:41:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe52fe00640 1 -- 172.21.15.66:0/2721119520 learned_addr learned my addr 172.21.15.66:0/2721119520 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe52f400640 1 --1- 172.21.15.66:0/2721119520 >> v1:172.21.15.156:6789/0 conn(0x7fe53015ca80 0x7fe53006b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:44570/0 (socket says 172.21.15.66:44570) 2024-03-30T00:41:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe52fe00640 1 -- 172.21.15.66:0/2721119520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5300cf2f0 msgr2=0x7fe53006b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe52fe00640 1 --2- 172.21.15.66:0/2721119520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5300cf2f0 0x7fe53006b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe52fe00640 1 -- 172.21.15.66:0/2721119520 >> v1:172.21.15.156:6789/0 conn(0x7fe53015ca80 legacy=0x7fe53006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.161+0000 7fe52fe00640 1 -- 172.21.15.66:0/2721119520 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5300cad70 con 0x7fe5300cb570 2024-03-30T00:41:08.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe52fe00640 1 --2- 172.21.15.66:0/2721119520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5300cb570 0x7fe5300cba20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe528009870 tx=0x7fe52801ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96cd58c1abc9544f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:08.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe52ea00640 1 -- 172.21.15.66:0/2721119520 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5280a8a50 con 0x7fe5300cb570 2024-03-30T00:41:08.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe52ea00640 1 -- 172.21.15.66:0/2721119520 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5280a8bb0 con 0x7fe5300cb570 2024-03-30T00:41:08.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe52ea00640 1 -- 172.21.15.66:0/2721119520 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5280b0440 con 0x7fe5300cb570 2024-03-30T00:41:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 -- 172.21.15.66:0/2721119520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5300cb570 msgr2=0x7fe5300cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 --2- 172.21.15.66:0/2721119520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5300cb570 0x7fe5300cba20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe528009870 tx=0x7fe52801ed50 comp rx=0 tx=0).stop 2024-03-30T00:41:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 -- 172.21.15.66:0/2721119520 shutdown_connections 2024-03-30T00:41:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 --2- 172.21.15.66:0/2721119520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5300cb570 0x7fe5300cba20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 --2- 172.21.15.66:0/2721119520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5300cf2f0 0x7fe53006b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 -- 172.21.15.66:0/2721119520 >> 172.21.15.66:0/2721119520 conn(0x7fe5300b2090 msgr2=0x7fe5300b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 -- 172.21.15.66:0/2721119520 shutdown_connections 2024-03-30T00:41:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 -- 172.21.15.66:0/2721119520 wait complete. 2024-03-30T00:41:08.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 Processor -- start 2024-03-30T00:41:08.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 -- start start 2024-03-30T00:41:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5300cb570 0x7fe5300b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5300cf2f0 0x7fe53015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe53015ca80 0x7fe53009bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53009b730 con 0x7fe53015ca80 2024-03-30T00:41:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300c99d0 con 0x7fe5300cb570 2024-03-30T00:41:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe536000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300b1df0 con 0x7fe5300cf2f0 2024-03-30T00:41:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe52fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe53015ca80 0x7fe53009bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe52fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe53015ca80 0x7fe53009bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52836/0 (socket says 172.21.15.66:52836) 2024-03-30T00:41:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe52fe00640 1 -- 172.21.15.66:0/1327945647 learned_addr learned my addr 172.21.15.66:0/1327945647 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe526a00640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5300cf2f0 0x7fe53015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.165+0000 7fe52f400640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5300cb570 0x7fe5300b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe52fe00640 1 -- 172.21.15.66:0/1327945647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5300cf2f0 msgr2=0x7fe53015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe52fe00640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5300cf2f0 0x7fe53015adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe52fe00640 1 -- 172.21.15.66:0/1327945647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5300cb570 msgr2=0x7fe5300b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe52fe00640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5300cb570 0x7fe5300b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe52fe00640 1 -- 172.21.15.66:0/1327945647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5280094d0 con 0x7fe53015ca80 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe52fe00640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe53015ca80 0x7fe53009bd10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe5280b3040 tx=0x7fe52801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe527e00640 1 -- 172.21.15.66:0/1327945647 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5280b0d90 con 0x7fe53015ca80 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe527e00640 1 -- 172.21.15.66:0/1327945647 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5280bb460 con 0x7fe53015ca80 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe527e00640 1 -- 172.21.15.66:0/1327945647 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5280bb7d0 con 0x7fe53015ca80 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe536000640 1 -- 172.21.15.66:0/1327945647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5300c9c20 con 0x7fe53015ca80 2024-03-30T00:41:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe536000640 1 -- 172.21.15.66:0/1327945647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53009b3c0 con 0x7fe53015ca80 2024-03-30T00:41:08.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe536000640 1 -- 172.21.15.66:0/1327945647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53009b8e0 con 0x7fe53015ca80 2024-03-30T00:41:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe527e00640 1 -- 172.21.15.66:0/1327945647 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe5280bb930 con 0x7fe53015ca80 2024-03-30T00:41:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe527e00640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe50c0666c0 0x7fe50c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe527e00640 1 -- 172.21.15.66:0/1327945647 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fe528121d80 con 0x7fe53015ca80 2024-03-30T00:41:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.169+0000 7fe52f400640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe50c0666c0 0x7fe50c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.173+0000 7fe52f400640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe50c0666c0 0x7fe50c068b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe5200012f0 tx=0x7fe520001660 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:08.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.173+0000 7fe527e00640 1 -- 172.21.15.66:0/1327945647 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe53009b8e0 con 0x7fe53015ca80 2024-03-30T00:41:08.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.333+0000 7fe536000640 1 -- 172.21.15.66:0/1327945647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe53009b8e0 con 0x7fe53015ca80 2024-03-30T00:41:08.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe527e00640 1 -- 172.21.15.66:0/1327945647 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7fe53009b8e0 con 0x7fe53015ca80 2024-03-30T00:41:08.345 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:08.346 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:08.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 -- 172.21.15.66:0/1327945647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe50c0666c0 msgr2=0x7fe50c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe50c0666c0 0x7fe50c068b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe5200012f0 tx=0x7fe520001660 comp rx=0 tx=0).stop 2024-03-30T00:41:08.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 -- 172.21.15.66:0/1327945647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe53015ca80 msgr2=0x7fe53009bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe53015ca80 0x7fe53009bd10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe5280b3040 tx=0x7fe52801baf0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 -- 172.21.15.66:0/1327945647 shutdown_connections 2024-03-30T00:41:08.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe50c0666c0 0x7fe50c068b80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe53015ca80 0x7fe53009bd10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5300cf2f0 0x7fe53015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 --2- 172.21.15.66:0/1327945647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5300cb570 0x7fe5300b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 -- 172.21.15.66:0/1327945647 >> 172.21.15.66:0/1327945647 conn(0x7fe5300b2090 msgr2=0x7fe53006ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:08.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 -- 172.21.15.66:0/1327945647 shutdown_connections 2024-03-30T00:41:08.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.337+0000 7fe536000640 1 -- 172.21.15.66:0/1327945647 wait complete. 2024-03-30T00:41:08.367 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T00:41:08.367 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:08.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.497+0000 7f3e75600640 1 Processor -- start 2024-03-30T00:41:08.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.497+0000 7f3e75600640 1 -- start start 2024-03-30T00:41:08.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e75600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7015ca80 0x7f3e7006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e75600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e700cb0e0 con 0x7f3e700cb570 2024-03-30T00:41:08.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e75600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e700cf7f0 con 0x7f3e700cf2f0 2024-03-30T00:41:08.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e75600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700cfad0 con 0x7f3e7015ca80 2024-03-30T00:41:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e6f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f3e700cb570 0x7f3e700cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48660/0 (socket says 172.21.15.66:48660) 2024-03-30T00:41:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e6f400640 1 -- 172.21.15.66:0/1436194629 learned_addr learned my addr 172.21.15.66:0/1436194629 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e6ea00640 1 --2- 172.21.15.66:0/1436194629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7015ca80 0x7f3e7006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e6e000640 1 -- 172.21.15.66:0/1436194629 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897309191 0 0) 0x7f3e700cb0e0 con 0x7f3e700cb570 2024-03-30T00:41:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e6e000640 1 -- 172.21.15.66:0/1436194629 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e54003530 con 0x7f3e700cb570 2024-03-30T00:41:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e6e000640 1 -- 172.21.15.66:0/1436194629 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718256257 0 0) 0x7f3e700cf7f0 con 0x7f3e700cf2f0 2024-03-30T00:41:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e6e000640 1 -- 172.21.15.66:0/1436194629 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e700cb0e0 con 0x7f3e700cf2f0 2024-03-30T00:41:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e6ea00640 1 -- 172.21.15.66:0/1436194629 >> v1:172.21.15.156:6789/0 conn(0x7f3e700cf2f0 legacy=0x7f3e7006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e6ea00640 1 -- 172.21.15.66:0/1436194629 >> v1:172.21.15.66:6789/0 conn(0x7f3e700cb570 legacy=0x7f3e700cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.501+0000 7f3e6ea00640 1 -- 172.21.15.66:0/1436194629 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e700cad70 con 0x7f3e7015ca80 2024-03-30T00:41:08.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e6ea00640 1 --2- 172.21.15.66:0/1436194629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7015ca80 0x7f3e7006b0b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009dc0 tx=0x7f3e6001f140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9badea8efff657 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:08.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e6e000640 1 -- 172.21.15.66:0/1436194629 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e600af600 con 0x7f3e7015ca80 2024-03-30T00:41:08.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e6e000640 1 -- 172.21.15.66:0/1436194629 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e600ab050 con 0x7f3e7015ca80 2024-03-30T00:41:08.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e6e000640 1 -- 172.21.15.66:0/1436194629 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e600af980 con 0x7f3e7015ca80 2024-03-30T00:41:08.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 -- 172.21.15.66:0/1436194629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7015ca80 msgr2=0x7f3e7006b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 --2- 172.21.15.66:0/1436194629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7015ca80 0x7f3e7006b0b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009dc0 tx=0x7f3e6001f140 comp rx=0 tx=0).stop 2024-03-30T00:41:08.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 -- 172.21.15.66:0/1436194629 shutdown_connections 2024-03-30T00:41:08.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 --2- 172.21.15.66:0/1436194629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7015ca80 0x7f3e7006b0b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 -- 172.21.15.66:0/1436194629 >> 172.21.15.66:0/1436194629 conn(0x7f3e700b2090 msgr2=0x7f3e700b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:08.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 -- 172.21.15.66:0/1436194629 shutdown_connections 2024-03-30T00:41:08.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 -- 172.21.15.66:0/1436194629 wait complete. 2024-03-30T00:41:08.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 Processor -- start 2024-03-30T00:41:08.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 -- start start 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e700cb570 0x7f3e7014a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e700cf2f0 0x7f3e701f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7015ca80 0x7f3e700c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7014b410 con 0x7f3e7015ca80 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7014ae50 con 0x7f3e700cf2f0 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.505+0000 7f3e75600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7014b130 con 0x7f3e700cb570 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e6f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7015ca80 0x7f3e700c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e6f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7015ca80 0x7f3e700c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52850/0 (socket says 172.21.15.66:52850) 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e6f400640 1 -- 172.21.15.66:0/2294935612 learned_addr learned my addr 172.21.15.66:0/2294935612 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e6ea00640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e700cb570 0x7f3e7014a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e66000640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e700cf2f0 0x7f3e701f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e6f400640 1 -- 172.21.15.66:0/2294935612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e700cb570 msgr2=0x7f3e7014a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e6f400640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e700cb570 0x7f3e7014a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e6f400640 1 -- 172.21.15.66:0/2294935612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e700cf2f0 msgr2=0x7f3e701f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e6f400640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e700cf2f0 0x7f3e701f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e6f400640 1 -- 172.21.15.66:0/2294935612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e60009a70 con 0x7f3e7015ca80 2024-03-30T00:41:08.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e6f400640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7015ca80 0x7f3e700c9180 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c450 tx=0x7f3e6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:08.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e67400640 1 -- 172.21.15.66:0/2294935612 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e68005cf0 con 0x7f3e7015ca80 2024-03-30T00:41:08.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e67400640 1 -- 172.21.15.66:0/2294935612 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e68005e50 con 0x7f3e7015ca80 2024-03-30T00:41:08.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e67400640 1 -- 172.21.15.66:0/2294935612 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e680072b0 con 0x7f3e7015ca80 2024-03-30T00:41:08.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e75600640 1 -- 172.21.15.66:0/2294935612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e700b4620 con 0x7f3e7015ca80 2024-03-30T00:41:08.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e75600640 1 -- 172.21.15.66:0/2294935612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7015bba0 con 0x7f3e7015ca80 2024-03-30T00:41:08.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e75600640 1 -- 172.21.15.66:0/2294935612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e70066c30 con 0x7f3e7015ca80 2024-03-30T00:41:08.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e67400640 1 -- 172.21.15.66:0/2294935612 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3e68007410 con 0x7f3e7015ca80 2024-03-30T00:41:08.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.509+0000 7f3e67400640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3e4c066690 0x7f3e4c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.513+0000 7f3e67400640 1 -- 172.21.15.66:0/2294935612 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f3e680836d0 con 0x7f3e7015ca80 2024-03-30T00:41:08.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.513+0000 7f3e6ea00640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3e4c066690 0x7f3e4c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.513+0000 7f3e6ea00640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3e4c066690 0x7f3e4c068b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3e600b1000 tx=0x7f3e600aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:08.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.513+0000 7f3e67400640 1 -- 172.21.15.66:0/2294935612 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3e680506e0 con 0x7f3e7015ca80 2024-03-30T00:41:08.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 -- 172.21.15.66:0/2294935612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e700b4ad0 con 0x7f3e7015ca80 2024-03-30T00:41:08.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e67400640 1 -- 172.21.15.66:0/2294935612 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f3e68053d00 con 0x7f3e7015ca80 2024-03-30T00:41:08.717 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:08.717 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4122431016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":4122431016},{"type":"v1","addr":"172.21.15.156:6835","nonce":4122431016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3882342907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3882342907},{"type":"v1","addr":"172.21.15.66:6835","nonce":3882342907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:39:51.142575+0000","modified":"2024-03-30T00:40:14.500142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/703895528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":703895528},{"type":"v1","addr":"172.21.15.66:6837","nonce":703895528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:41:08.718 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2024-03-30T00:41:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 -- 172.21.15.66:0/2294935612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3e4c066690 msgr2=0x7f3e4c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3e4c066690 0x7f3e4c068b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3e600b1000 tx=0x7f3e600aa000 comp rx=0 tx=0).stop 2024-03-30T00:41:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 -- 172.21.15.66:0/2294935612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7015ca80 msgr2=0x7f3e700c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7015ca80 0x7f3e700c9180 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c450 tx=0x7f3e6800c910 comp rx=0 tx=0).stop 2024-03-30T00:41:08.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 -- 172.21.15.66:0/2294935612 shutdown_connections 2024-03-30T00:41:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3e4c066690 0x7f3e4c068b50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7015ca80 0x7f3e700c9180 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e700cf2f0 0x7f3e701f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 --2- 172.21.15.66:0/2294935612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e700cb570 0x7f3e7014a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 -- 172.21.15.66:0/2294935612 >> 172.21.15.66:0/2294935612 conn(0x7f3e700b2090 msgr2=0x7f3e700cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 -- 172.21.15.66:0/2294935612 shutdown_connections 2024-03-30T00:41:08.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.709+0000 7f3e75600640 1 -- 172.21.15.66:0/2294935612 wait complete. 2024-03-30T00:41:08.739 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:08.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.869+0000 7fdbd9a00640 1 Processor -- start 2024-03-30T00:41:08.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.869+0000 7fdbd9a00640 1 -- start start 2024-03-30T00:41:08.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.869+0000 7fdbd9a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdbd40cda90 0x7fdbd40ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.869+0000 7fdbd9a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdbd415a370 0x7fdbd40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.869+0000 7fdbd9a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd415a990 con 0x7fdbd40cde70 2024-03-30T00:41:08.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.869+0000 7fdbd9a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd40b1b80 con 0x7fdbd415a370 2024-03-30T00:41:08.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.869+0000 7fdbd9a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd40cd3e0 con 0x7fdbd40cda90 2024-03-30T00:41:08.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdbd415a370 0x7fdbd40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdbd415a370 0x7fdbd40cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39488/0 (socket says 172.21.15.66:39488) 2024-03-30T00:41:08.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd3400640 1 -- 172.21.15.66:0/811091712 learned_addr learned my addr 172.21.15.66:0/811091712 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:08.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd2a00640 1 --2- 172.21.15.66:0/811091712 >> v2:172.21.15.156:3300/0 conn(0x7fdbd40cda90 0x7fdbd40ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd3400640 1 -- 172.21.15.66:0/811091712 >> v2:172.21.15.156:3300/0 conn(0x7fdbd40cda90 msgr2=0x7fdbd40ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:08.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd3400640 1 --2- 172.21.15.66:0/811091712 >> v2:172.21.15.156:3300/0 conn(0x7fdbd40cda90 0x7fdbd40ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd3400640 1 -- 172.21.15.66:0/811091712 >> v1:172.21.15.156:6789/0 conn(0x7fdbd40cde70 legacy=0x7fdbd4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd3400640 1 -- 172.21.15.66:0/811091712 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd40b11e0 con 0x7fdbd415a370 2024-03-30T00:41:08.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd3400640 1 --2- 172.21.15.66:0/811091712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd415a370 0x7fdbd40cea60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40cf3f0 tx=0x7fdbcc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80616642737da44e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:08.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd2000640 1 -- 172.21.15.66:0/811091712 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0a5730 con 0x7fdbd415a370 2024-03-30T00:41:08.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd2000640 1 -- 172.21.15.66:0/811091712 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc0a5890 con 0x7fdbd415a370 2024-03-30T00:41:08.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.873+0000 7fdbd2000640 1 -- 172.21.15.66:0/811091712 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0a5c20 con 0x7fdbd415a370 2024-03-30T00:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 -- 172.21.15.66:0/811091712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd415a370 msgr2=0x7fdbd40cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 --2- 172.21.15.66:0/811091712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd415a370 0x7fdbd40cea60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40cf3f0 tx=0x7fdbcc01c960 comp rx=0 tx=0).stop 2024-03-30T00:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 -- 172.21.15.66:0/811091712 shutdown_connections 2024-03-30T00:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 --2- 172.21.15.66:0/811091712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd415a370 0x7fdbd40cea60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 --2- 172.21.15.66:0/811091712 >> v2:172.21.15.156:3300/0 conn(0x7fdbd40cda90 0x7fdbd40ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 -- 172.21.15.66:0/811091712 >> 172.21.15.66:0/811091712 conn(0x7fdbd4069f60 msgr2=0x7fdbd406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 -- 172.21.15.66:0/811091712 shutdown_connections 2024-03-30T00:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 -- 172.21.15.66:0/811091712 wait complete. 2024-03-30T00:41:08.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 Processor -- start 2024-03-30T00:41:08.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 -- start start 2024-03-30T00:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbd40cda90 0x7fdbd41524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd40cde70 0x7fdbd4095540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd415a370 0x7fdbd4150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd406c120 con 0x7fdbd40cde70 2024-03-30T00:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd4097560 con 0x7fdbd40cda90 2024-03-30T00:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd9a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd4096ed0 con 0x7fdbd415a370 2024-03-30T00:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbcbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd40cde70 0x7fdbd4095540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbcbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd40cde70 0x7fdbd4095540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39498/0 (socket says 172.21.15.66:39498) 2024-03-30T00:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbcbe00640 1 -- 172.21.15.66:0/1234874393 learned_addr learned my addr 172.21.15.66:0/1234874393 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbd2a00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbd40cda90 0x7fdbd41524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbcbe00640 1 -- 172.21.15.66:0/1234874393 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd415a370 msgr2=0x7fdbd4150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbcbe00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd415a370 0x7fdbd4150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbcbe00640 1 -- 172.21.15.66:0/1234874393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbd40cda90 msgr2=0x7fdbd41524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbcbe00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbd40cda90 0x7fdbd41524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.877+0000 7fdbcbe00640 1 -- 172.21.15.66:0/1234874393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc009b30 con 0x7fdbd40cde70 2024-03-30T00:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbcbe00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd40cde70 0x7fdbd4095540 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c4d0 tx=0x7fdbbc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbcaa00640 1 -- 172.21.15.66:0/1234874393 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc0052d0 con 0x7fdbd40cde70 2024-03-30T00:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbcaa00640 1 -- 172.21.15.66:0/1234874393 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbbc005430 con 0x7fdbd40cde70 2024-03-30T00:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbcaa00640 1 -- 172.21.15.66:0/1234874393 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc012830 con 0x7fdbd40cde70 2024-03-30T00:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbd9a00640 1 -- 172.21.15.66:0/1234874393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd406c370 con 0x7fdbd40cde70 2024-03-30T00:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbd9a00640 1 -- 172.21.15.66:0/1234874393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd4095a80 con 0x7fdbd40cde70 2024-03-30T00:41:08.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbd9a00640 1 -- 172.21.15.66:0/1234874393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd40add10 con 0x7fdbd40cde70 2024-03-30T00:41:08.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbcaa00640 1 -- 172.21.15.66:0/1234874393 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdbbc012990 con 0x7fdbd40cde70 2024-03-30T00:41:08.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbcaa00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbb0066690 0x7fdbb0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:08.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbcaa00640 1 -- 172.21.15.66:0/1234874393 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fdbbc084040 con 0x7fdbd40cde70 2024-03-30T00:41:08.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbd2a00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbb0066690 0x7fdbb0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:08.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.881+0000 7fdbd2a00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbb0066690 0x7fdbb0068b50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40c1c10 tx=0x7fdbc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:08.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:08.885+0000 7fdbcaa00640 1 -- 172.21.15.66:0/1234874393 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdbd40add10 con 0x7fdbd40cde70 2024-03-30T00:41:09.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.077+0000 7fdbd9a00640 1 -- 172.21.15.66:0/1234874393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbd40add10 con 0x7fdbd40cde70 2024-03-30T00:41:09.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbcaa00640 1 -- 172.21.15.66:0/1234874393 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4730 (secure 0 0 0) 0x7fdbd40add10 con 0x7fdbd40cde70 2024-03-30T00:41:09.089 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:09.089 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4122431016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":4122431016},{"type":"v1","addr":"172.21.15.156:6835","nonce":4122431016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/643421946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":643421946},{"type":"v1","addr":"172.21.15.156:6834","nonce":643421946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3882342907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3882342907},{"type":"v1","addr":"172.21.15.66:6835","nonce":3882342907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:39:51.142575+0000","modified":"2024-03-30T00:40:14.500142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/703895528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":703895528},{"type":"v1","addr":"172.21.15.66:6837","nonce":703895528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:41:09.090 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2024-03-30T00:41:09.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 -- 172.21.15.66:0/1234874393 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbb0066690 msgr2=0x7fdbb0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbb0066690 0x7fdbb0068b50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40c1c10 tx=0x7fdbc4008040 comp rx=0 tx=0).stop 2024-03-30T00:41:09.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 -- 172.21.15.66:0/1234874393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd40cde70 msgr2=0x7fdbd4095540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd40cde70 0x7fdbd4095540 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c4d0 tx=0x7fdbbc00c990 comp rx=0 tx=0).stop 2024-03-30T00:41:09.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 -- 172.21.15.66:0/1234874393 shutdown_connections 2024-03-30T00:41:09.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd415a370 0x7fdbd4150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbb0066690 0x7fdbb0068b50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd40cde70 0x7fdbd4095540 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 --2- 172.21.15.66:0/1234874393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbd40cda90 0x7fdbd41524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 -- 172.21.15.66:0/1234874393 >> 172.21.15.66:0/1234874393 conn(0x7fdbd4069f60 msgr2=0x7fdbd415b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:09.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 -- 172.21.15.66:0/1234874393 shutdown_connections 2024-03-30T00:41:09.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.081+0000 7fdbd9a00640 1 -- 172.21.15.66:0/1234874393 wait complete. 2024-03-30T00:41:09.114 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:09.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80bf400640 1 Processor -- start 2024-03-30T00:41:09.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80bf400640 1 -- start start 2024-03-30T00:41:09.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80bf400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80b80cc560 0x7f80b80d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80bf400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f80b806f8e0 0x7f80b806e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80bf400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b806e630 0x7f80b80ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80bf400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b8160640 con 0x7f80b806f8e0 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80bf400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b806eaa0 con 0x7f80b80cc560 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80bf400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b806b0b0 con 0x7f80b806e630 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80afe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f80b806f8e0 0x7f80b806e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80afe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f80b806f8e0 0x7f80b806e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39502/0 (socket says 172.21.15.66:39502) 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80afe00640 1 -- 172.21.15.66:0/2458938357 learned_addr learned my addr 172.21.15.66:0/2458938357 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80bd400640 1 --2- 172.21.15.66:0/2458938357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b806e630 0x7f80b80ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80bca00640 1 --2- 172.21.15.66:0/2458938357 >> v2:172.21.15.156:3300/0 conn(0x7f80b80cc560 0x7f80b80d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80afe00640 1 -- 172.21.15.66:0/2458938357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b806e630 msgr2=0x7f80b80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80afe00640 1 --2- 172.21.15.66:0/2458938357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b806e630 0x7f80b80ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80afe00640 1 -- 172.21.15.66:0/2458938357 >> v2:172.21.15.156:3300/0 conn(0x7f80b80cc560 msgr2=0x7f80b80d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:09.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80afe00640 1 --2- 172.21.15.66:0/2458938357 >> v2:172.21.15.156:3300/0 conn(0x7f80b80cc560 0x7f80b80d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80afe00640 1 -- 172.21.15.66:0/2458938357 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b806b390 con 0x7f80b806f8e0 2024-03-30T00:41:09.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.245+0000 7f80bd400640 1 --2- 172.21.15.66:0/2458938357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b806e630 0x7f80b80ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:09.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80afe00640 1 --2- 172.21.15.66:0/2458938357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b806f8e0 0x7f80b806e0f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f80b8072c50 tx=0x7f80a001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effa26b660225c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:09.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80b7e00640 1 -- 172.21.15.66:0/2458938357 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a00ab600 con 0x7f80b806f8e0 2024-03-30T00:41:09.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80b7e00640 1 -- 172.21.15.66:0/2458938357 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a00a7050 con 0x7f80b806f8e0 2024-03-30T00:41:09.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80b7e00640 1 -- 172.21.15.66:0/2458938357 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a00ab980 con 0x7f80b806f8e0 2024-03-30T00:41:09.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80bf400640 1 -- 172.21.15.66:0/2458938357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b806f8e0 msgr2=0x7f80b806e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80bf400640 1 --2- 172.21.15.66:0/2458938357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b806f8e0 0x7f80b806e0f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f80b8072c50 tx=0x7f80a001c230 comp rx=0 tx=0).stop 2024-03-30T00:41:09.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80bf400640 1 -- 172.21.15.66:0/2458938357 shutdown_connections 2024-03-30T00:41:09.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80bf400640 1 --2- 172.21.15.66:0/2458938357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b806e630 0x7f80b80ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80bf400640 1 --2- 172.21.15.66:0/2458938357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b806f8e0 0x7f80b806e0f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80bf400640 1 --2- 172.21.15.66:0/2458938357 >> v2:172.21.15.156:3300/0 conn(0x7f80b80cc560 0x7f80b80d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80bf400640 1 -- 172.21.15.66:0/2458938357 >> 172.21.15.66:0/2458938357 conn(0x7f80b80d0540 msgr2=0x7f80b80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:09.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80bf400640 1 -- 172.21.15.66:0/2458938357 shutdown_connections 2024-03-30T00:41:09.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.249+0000 7f80bf400640 1 -- 172.21.15.66:0/2458938357 wait complete. 2024-03-30T00:41:09.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bf400640 1 Processor -- start 2024-03-30T00:41:09.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bf400640 1 -- start start 2024-03-30T00:41:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bf400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b806e630 0x7f80b81509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bf400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b806f8e0 0x7f80b81513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80cc560 0x7f80b806ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bf400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b8152a00 con 0x7f80b80cc560 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bf400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b8151df0 con 0x7f80b806e630 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bf400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b81520d0 con 0x7f80b806f8e0 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80cc560 0x7f80b806ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80cc560 0x7f80b806ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39514/0 (socket says 172.21.15.66:39514) 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bd400640 1 -- 172.21.15.66:0/497990018 learned_addr learned my addr 172.21.15.66:0/497990018 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80afe00640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b806f8e0 0x7f80b81513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bca00640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b806e630 0x7f80b81509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bd400640 1 -- 172.21.15.66:0/497990018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b806f8e0 msgr2=0x7f80b81513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bd400640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b806f8e0 0x7f80b81513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bd400640 1 -- 172.21.15.66:0/497990018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b806e630 msgr2=0x7f80b81509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bd400640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b806e630 0x7f80b81509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bd400640 1 -- 172.21.15.66:0/497990018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a00094d0 con 0x7f80b80cc560 2024-03-30T00:41:09.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bd400640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80cc560 0x7f80b806ebe0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f80b0007e10 tx=0x7f80b000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:09.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80b5600640 1 -- 172.21.15.66:0/497990018 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0003ba0 con 0x7f80b80cc560 2024-03-30T00:41:09.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80b5600640 1 -- 172.21.15.66:0/497990018 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b000f040 con 0x7f80b80cc560 2024-03-30T00:41:09.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80b5600640 1 -- 172.21.15.66:0/497990018 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b000de80 con 0x7f80b80cc560 2024-03-30T00:41:09.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bf400640 1 -- 172.21.15.66:0/497990018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b81523b0 con 0x7f80b80cc560 2024-03-30T00:41:09.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bf400640 1 -- 172.21.15.66:0/497990018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b8150ef0 con 0x7f80b80cc560 2024-03-30T00:41:09.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.253+0000 7f80bf400640 1 -- 172.21.15.66:0/497990018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b8154e70 con 0x7f80b80cc560 2024-03-30T00:41:09.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.257+0000 7f80b5600640 1 -- 172.21.15.66:0/497990018 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f80b0004030 con 0x7f80b80cc560 2024-03-30T00:41:09.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.257+0000 7f80b5600640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 0x7f8094068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.257+0000 7f80b5600640 1 -- 172.21.15.66:0/497990018 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f80b0084920 con 0x7f80b80cc560 2024-03-30T00:41:09.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.257+0000 7f80bca00640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 0x7f8094068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.257+0000 7f80bca00640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 0x7f8094068b80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f80a800b930 tx=0x7f80a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:09.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.261+0000 7f80b5600640 1 -- 172.21.15.66:0/497990018 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f80b8154e70 con 0x7f80b80cc560 2024-03-30T00:41:09.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.421+0000 7f80bf400640 1 -- 172.21.15.66:0/497990018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80b8154e70 con 0x7f80b80cc560 2024-03-30T00:41:09.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80b5600640 1 -- 172.21.15.66:0/497990018 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18359 (secure 0 0 0) 0x7f80b8154e70 con 0x7f80b80cc560 2024-03-30T00:41:09.433 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:09.433 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:40:32.587976+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:39:49.685024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:39:50.136108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T00:39:53.201055+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:09.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 -- 172.21.15.66:0/497990018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 msgr2=0x7f8094068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 0x7f8094068b80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f80a800b930 tx=0x7f80a8008040 comp rx=0 tx=0).stop 2024-03-30T00:41:09.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 -- 172.21.15.66:0/497990018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80cc560 msgr2=0x7f80b806ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80cc560 0x7f80b806ebe0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f80b0007e10 tx=0x7f80b000d4a0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 -- 172.21.15.66:0/497990018 shutdown_connections 2024-03-30T00:41:09.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 0x7f8094068b80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80cc560 0x7f80b806ebe0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b806f8e0 0x7f80b81513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 --2- 172.21.15.66:0/497990018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b806e630 0x7f80b81509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 -- 172.21.15.66:0/497990018 >> 172.21.15.66:0/497990018 conn(0x7f80b80d0540 msgr2=0x7f80b80af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:09.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 -- 172.21.15.66:0/497990018 shutdown_connections 2024-03-30T00:41:09.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.425+0000 7f80bf400640 1 -- 172.21.15.66:0/497990018 wait complete. 2024-03-30T00:41:09.454 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:41:09.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd645600640 1 Processor -- start 2024-03-30T00:41:09.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd645600640 1 -- start start 2024-03-30T00:41:09.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd645600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd64015a640 0x7fd64006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd645600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd6400ce520 0x7fd6400ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd645600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64006d070 con 0x7fd6400cabb0 2024-03-30T00:41:09.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd645600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6400cbba0 con 0x7fd6400ce520 2024-03-30T00:41:09.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd645600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64006e2c0 con 0x7fd64015a640 2024-03-30T00:41:09.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd637e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd6400ce520 0x7fd6400ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd637e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd6400ce520 0x7fd6400ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39530/0 (socket says 172.21.15.66:39530) 2024-03-30T00:41:09.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd637e00640 1 -- 172.21.15.66:0/45459452 learned_addr learned my addr 172.21.15.66:0/45459452 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:09.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd63ea00640 1 --2- 172.21.15.66:0/45459452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd64015a640 0x7fd64006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd637e00640 1 -- 172.21.15.66:0/45459452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd64015a640 msgr2=0x7fd64006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:09.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd637e00640 1 --2- 172.21.15.66:0/45459452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd64015a640 0x7fd64006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd637e00640 1 -- 172.21.15.66:0/45459452 >> v1:172.21.15.66:6789/0 conn(0x7fd6400cabb0 legacy=0x7fd6400cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.585+0000 7fd637e00640 1 -- 172.21.15.66:0/45459452 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6400cee60 con 0x7fd6400ce520 2024-03-30T00:41:09.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd637e00640 1 --2- 172.21.15.66:0/45459452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6400ce520 0x7fd6400ce920 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd628009870 tx=0x7fd62801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8be4b1667719668 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:09.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd63e000640 1 -- 172.21.15.66:0/45459452 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280ab650 con 0x7fd6400ce520 2024-03-30T00:41:09.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd63e000640 1 -- 172.21.15.66:0/45459452 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6280ab7b0 con 0x7fd6400ce520 2024-03-30T00:41:09.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd63e000640 1 -- 172.21.15.66:0/45459452 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280abb40 con 0x7fd6400ce520 2024-03-30T00:41:09.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd645600640 1 -- 172.21.15.66:0/45459452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6400ce520 msgr2=0x7fd6400ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd645600640 1 --2- 172.21.15.66:0/45459452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6400ce520 0x7fd6400ce920 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd628009870 tx=0x7fd62801c560 comp rx=0 tx=0).stop 2024-03-30T00:41:09.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd645600640 1 -- 172.21.15.66:0/45459452 shutdown_connections 2024-03-30T00:41:09.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd645600640 1 --2- 172.21.15.66:0/45459452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6400ce520 0x7fd6400ce920 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd645600640 1 --2- 172.21.15.66:0/45459452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd64015a640 0x7fd64006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd645600640 1 -- 172.21.15.66:0/45459452 >> 172.21.15.66:0/45459452 conn(0x7fd640069f60 msgr2=0x7fd64006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:09.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd645600640 1 -- 172.21.15.66:0/45459452 shutdown_connections 2024-03-30T00:41:09.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.589+0000 7fd645600640 1 -- 172.21.15.66:0/45459452 wait complete. 2024-03-30T00:41:09.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd645600640 1 Processor -- start 2024-03-30T00:41:09.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd645600640 1 -- start start 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd645600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6400cabb0 0x7fd640164c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd645600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6400ce520 0x7fd6400708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd645600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd64015a640 0x7fd6400c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd645600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6400c14f0 con 0x7fd64015a640 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd645600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd640070f50 con 0x7fd6400ce520 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd645600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd640071230 con 0x7fd6400cabb0 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd63f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd64015a640 0x7fd6400c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd63f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd64015a640 0x7fd6400c0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39546/0 (socket says 172.21.15.66:39546) 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd63f400640 1 -- 172.21.15.66:0/2881145059 learned_addr learned my addr 172.21.15.66:0/2881145059 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd63ea00640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6400cabb0 0x7fd640164c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd637e00640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6400ce520 0x7fd6400708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd63f400640 1 -- 172.21.15.66:0/2881145059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6400cabb0 msgr2=0x7fd640164c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd63f400640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6400cabb0 0x7fd640164c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd63f400640 1 -- 172.21.15.66:0/2881145059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6400ce520 msgr2=0x7fd6400708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd63f400640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6400ce520 0x7fd6400708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd63f400640 1 -- 172.21.15.66:0/2881145059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6280094d0 con 0x7fd64015a640 2024-03-30T00:41:09.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd63f400640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd64015a640 0x7fd6400c0b70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd63800c770 tx=0x7fd63800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:09.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd636a00640 1 -- 172.21.15.66:0/2881145059 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638003b80 con 0x7fd64015a640 2024-03-30T00:41:09.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd636a00640 1 -- 172.21.15.66:0/2881145059 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd638003ce0 con 0x7fd64015a640 2024-03-30T00:41:09.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd636a00640 1 -- 172.21.15.66:0/2881145059 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63800aa90 con 0x7fd64015a640 2024-03-30T00:41:09.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd645600640 1 -- 172.21.15.66:0/2881145059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd640071510 con 0x7fd64015a640 2024-03-30T00:41:09.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.593+0000 7fd645600640 1 -- 172.21.15.66:0/2881145059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd640071ad0 con 0x7fd64015a640 2024-03-30T00:41:09.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.597+0000 7fd645600640 1 -- 172.21.15.66:0/2881145059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd640066c30 con 0x7fd64015a640 2024-03-30T00:41:09.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.597+0000 7fd636a00640 1 -- 172.21.15.66:0/2881145059 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd63800abf0 con 0x7fd64015a640 2024-03-30T00:41:09.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.597+0000 7fd636a00640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd61c066690 0x7fd61c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.597+0000 7fd636a00640 1 -- 172.21.15.66:0/2881145059 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fd6380842a0 con 0x7fd64015a640 2024-03-30T00:41:09.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.597+0000 7fd63ea00640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd61c066690 0x7fd61c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.597+0000 7fd63ea00640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd61c066690 0x7fd61c068b50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd64006ce20 tx=0x7fd630009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:09.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.601+0000 7fd636a00640 1 -- 172.21.15.66:0/2881145059 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd6380512b0 con 0x7fd64015a640 2024-03-30T00:41:09.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.801+0000 7fd645600640 1 -- 172.21.15.66:0/2881145059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd6400ae7b0 con 0x7fd64015a640 2024-03-30T00:41:09.810 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:41:09.801+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:41:09.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.809+0000 7fd636a00640 1 -- 172.21.15.66:0/2881145059 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd6380548d0 con 0x7fd64015a640 2024-03-30T00:41:09.819 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:41:09.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 -- 172.21.15.66:0/2881145059 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd61c066690 msgr2=0x7fd61c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd61c066690 0x7fd61c068b50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd64006ce20 tx=0x7fd630009190 comp rx=0 tx=0).stop 2024-03-30T00:41:09.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 -- 172.21.15.66:0/2881145059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd64015a640 msgr2=0x7fd6400c0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd64015a640 0x7fd6400c0b70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd63800c770 tx=0x7fd63800cc30 comp rx=0 tx=0).stop 2024-03-30T00:41:09.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 -- 172.21.15.66:0/2881145059 shutdown_connections 2024-03-30T00:41:09.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd61c066690 0x7fd61c068b50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd64015a640 0x7fd6400c0b70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6400ce520 0x7fd6400708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 --2- 172.21.15.66:0/2881145059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6400cabb0 0x7fd640164c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 -- 172.21.15.66:0/2881145059 >> 172.21.15.66:0/2881145059 conn(0x7fd640069f60 msgr2=0x7fd6400ad7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:09.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 -- 172.21.15.66:0/2881145059 shutdown_connections 2024-03-30T00:41:09.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.813+0000 7fd645600640 1 -- 172.21.15.66:0/2881145059 wait complete. 2024-03-30T00:41:09.828 INFO:tasks.ceph.mds.c.smithi066.stderr: -4673> 2024-03-30T00:39:55.973+0000 7fbc06000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:41:09.843 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T00:41:09.877 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:41:09.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f5395c00640 1 Processor -- start 2024-03-30T00:41:09.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f5395c00640 1 -- start start 2024-03-30T00:41:09.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f5395c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f53900ce520 0x7f53900ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f5395c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53900cabb0 0x7f53900cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f5395c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f539006d070 con 0x7f539015a640 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f5395c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53900cbba0 con 0x7f53900cabb0 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f5395c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53900cee70 con 0x7f53900ce520 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f538f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53900cabb0 0x7f53900cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f538f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53900cabb0 0x7f53900cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39576/0 (socket says 172.21.15.66:39576) 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f538f400640 1 -- 172.21.15.66:0/1899525281 learned_addr learned my addr 172.21.15.66:0/1899525281 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f5386000640 1 --2- 172.21.15.66:0/1899525281 >> v2:172.21.15.156:3300/0 conn(0x7f53900ce520 0x7f53900ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f538f400640 1 -- 172.21.15.66:0/1899525281 >> v2:172.21.15.156:3300/0 conn(0x7f53900ce520 msgr2=0x7f53900ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f538f400640 1 --2- 172.21.15.66:0/1899525281 >> v2:172.21.15.156:3300/0 conn(0x7f53900ce520 0x7f53900ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f538f400640 1 -- 172.21.15.66:0/1899525281 >> v1:172.21.15.156:6789/0 conn(0x7f539015a640 legacy=0x7f539006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:09.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.977+0000 7f538f400640 1 -- 172.21.15.66:0/1899525281 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53900cb8c0 con 0x7f53900cabb0 2024-03-30T00:41:09.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f538f400640 1 --2- 172.21.15.66:0/1899525281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53900cabb0 0x7f53900cafe0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f53900c88d0 tx=0x7f538801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec2ee544451a753 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:09.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f538e000640 1 -- 172.21.15.66:0/1899525281 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53880aa610 con 0x7f53900cabb0 2024-03-30T00:41:09.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f538e000640 1 -- 172.21.15.66:0/1899525281 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53880a6040 con 0x7f53900cabb0 2024-03-30T00:41:09.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f538e000640 1 -- 172.21.15.66:0/1899525281 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53880aa990 con 0x7f53900cabb0 2024-03-30T00:41:09.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f5395c00640 1 -- 172.21.15.66:0/1899525281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53900cabb0 msgr2=0x7f53900cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f5395c00640 1 --2- 172.21.15.66:0/1899525281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53900cabb0 0x7f53900cafe0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f53900c88d0 tx=0x7f538801c4b0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f5395c00640 1 -- 172.21.15.66:0/1899525281 shutdown_connections 2024-03-30T00:41:09.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f5395c00640 1 --2- 172.21.15.66:0/1899525281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53900cabb0 0x7f53900cafe0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f5395c00640 1 --2- 172.21.15.66:0/1899525281 >> v2:172.21.15.156:3300/0 conn(0x7f53900ce520 0x7f53900ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f5395c00640 1 -- 172.21.15.66:0/1899525281 >> 172.21.15.66:0/1899525281 conn(0x7f5390069f60 msgr2=0x7f539006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:09.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f5395c00640 1 -- 172.21.15.66:0/1899525281 shutdown_connections 2024-03-30T00:41:09.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.981+0000 7f5395c00640 1 -- 172.21.15.66:0/1899525281 wait complete. 2024-03-30T00:41:09.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5395c00640 1 Processor -- start 2024-03-30T00:41:09.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5395c00640 1 -- start start 2024-03-30T00:41:09.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5395c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53900cabb0 0x7f53900b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5395c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53900ce520 0x7f539014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5395c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f539015a640 0x7f5390162510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5395c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53901f4960 con 0x7f539015a640 2024-03-30T00:41:09.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5395c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5390156150 con 0x7f53900ce520 2024-03-30T00:41:09.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5395c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5390163270 con 0x7f53900cabb0 2024-03-30T00:41:09.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f538f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f539015a640 0x7f5390162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f538f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f539015a640 0x7f5390162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39586/0 (socket says 172.21.15.66:39586) 2024-03-30T00:41:09.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f538f400640 1 -- 172.21.15.66:0/980123108 learned_addr learned my addr 172.21.15.66:0/980123108 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:09.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f538ea00640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53900cabb0 0x7f53900b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5386000640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53900ce520 0x7f539014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f538f400640 1 -- 172.21.15.66:0/980123108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53900cabb0 msgr2=0x7f53900b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f538f400640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53900cabb0 0x7f53900b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f538f400640 1 -- 172.21.15.66:0/980123108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53900ce520 msgr2=0x7f539014ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:09.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f538f400640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53900ce520 0x7f539014ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:09.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f538f400640 1 -- 172.21.15.66:0/980123108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53880094d0 con 0x7f539015a640 2024-03-30T00:41:09.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f538f400640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f539015a640 0x7f5390162510 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f53880ac040 tx=0x7f538801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:09.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5387400640 1 -- 172.21.15.66:0/980123108 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53880bd5f0 con 0x7f539015a640 2024-03-30T00:41:09.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5387400640 1 -- 172.21.15.66:0/980123108 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53880bd750 con 0x7f539015a640 2024-03-30T00:41:09.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5387400640 1 -- 172.21.15.66:0/980123108 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53880bdb00 con 0x7f539015a640 2024-03-30T00:41:09.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5395c00640 1 -- 172.21.15.66:0/980123108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53900ca5b0 con 0x7f539015a640 2024-03-30T00:41:09.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f5395c00640 1 -- 172.21.15.66:0/980123108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5390156640 con 0x7f539015a640 2024-03-30T00:41:09.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.985+0000 7f537fe00640 1 -- 172.21.15.66:0/980123108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5390066c30 con 0x7f539015a640 2024-03-30T00:41:09.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.989+0000 7f5387400640 1 -- 172.21.15.66:0/980123108 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f53880b33f0 con 0x7f539015a640 2024-03-30T00:41:09.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.989+0000 7f5387400640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f536c0666c0 0x7f536c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:09.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.989+0000 7f5387400640 1 -- 172.21.15.66:0/980123108 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6807+0+0 (secure 0 0 0) 0x7f538811f250 con 0x7f539015a640 2024-03-30T00:41:09.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.989+0000 7f538ea00640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f536c0666c0 0x7f536c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:09.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.989+0000 7f538ea00640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f536c0666c0 0x7f536c068b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5390067510 tx=0x7f538000d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:10.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:09.993+0000 7f5387400640 1 -- 172.21.15.66:0/980123108 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f53880ec210 con 0x7f539015a640 2024-03-30T00:41:10.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.189+0000 7f537fe00640 1 -- 172.21.15.66:0/980123108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f539015da90 con 0x7f539015a640 2024-03-30T00:41:10.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f5387400640 1 -- 172.21.15.66:0/980123108 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f53880ef830 con 0x7f539015a640 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 -- 172.21.15.66:0/980123108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f536c0666c0 msgr2=0x7f536c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f536c0666c0 0x7f536c068b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5390067510 tx=0x7f538000d040 comp rx=0 tx=0).stop 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 -- 172.21.15.66:0/980123108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f539015a640 msgr2=0x7f5390162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f539015a640 0x7f5390162510 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f53880ac040 tx=0x7f538801acf0 comp rx=0 tx=0).stop 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 -- 172.21.15.66:0/980123108 shutdown_connections 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f536c0666c0 0x7f536c068b80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f539015a640 0x7f5390162510 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53900ce520 0x7f539014ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 --2- 172.21.15.66:0/980123108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53900cabb0 0x7f53900b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 -- 172.21.15.66:0/980123108 >> 172.21.15.66:0/980123108 conn(0x7f5390069f60 msgr2=0x7f5390158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 -- 172.21.15.66:0/980123108 shutdown_connections 2024-03-30T00:41:10.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:10.813+0000 7f537fe00640 1 -- 172.21.15.66:0/980123108 wait complete. 2024-03-30T00:41:10.846 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T00:41:11.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.029+0000 7f8f68400640 1 Processor -- start 2024-03-30T00:41:11.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.029+0000 7f8f68400640 1 -- start start 2024-03-30T00:41:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.029+0000 7f8f68400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f600acaf0 0x7f8f600b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.029+0000 7f8f68400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8f6015ba00 0x7f8f600ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.029+0000 7f8f68400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f600cbf80 con 0x7f8f600cecf0 2024-03-30T00:41:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.029+0000 7f8f68400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f600cf180 con 0x7f8f6015ba00 2024-03-30T00:41:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.029+0000 7f8f68400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600cc820 con 0x7f8f600acaf0 2024-03-30T00:41:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.029+0000 7f8f65000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8f6015ba00 0x7f8f600ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:11.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.029+0000 7f8f66400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f8f600cecf0 0x7f8f601579e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48852/0 (socket says 172.21.15.66:48852) 2024-03-30T00:41:11.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.029+0000 7f8f66400640 1 -- 172.21.15.66:0/3843164994 learned_addr learned my addr 172.21.15.66:0/3843164994 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:11.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.033+0000 7f8f65a00640 1 --2- 172.21.15.66:0/3843164994 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f600acaf0 0x7f8f600b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:11.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.033+0000 7f8f65000640 1 -- 172.21.15.66:0/3843164994 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f600acaf0 msgr2=0x7f8f600b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:11.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.033+0000 7f8f65000640 1 --2- 172.21.15.66:0/3843164994 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f600acaf0 0x7f8f600b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:11.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.033+0000 7f8f65000640 1 -- 172.21.15.66:0/3843164994 >> v1:172.21.15.66:6789/0 conn(0x7f8f600cecf0 legacy=0x7f8f601579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.033+0000 7f8f65000640 1 -- 172.21.15.66:0/3843164994 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600b2dd0 con 0x7f8f6015ba00 2024-03-30T00:41:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.033+0000 7f8f5fe00640 1 -- 172.21.15.66:0/3843164994 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846442280 0 0) 0x7f8f600cbf80 con 0x7f8f600cecf0 2024-03-30T00:41:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.033+0000 7f8f65a00640 1 --2- 172.21.15.66:0/3843164994 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f600acaf0 0x7f8f600b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.033+0000 7f8f65000640 1 --2- 172.21.15.66:0/3843164994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015ba00 0x7f8f600ce7b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0097c0 tx=0x7f8f4c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88324cfd1c2931d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.033+0000 7f8f5fe00640 1 -- 172.21.15.66:0/3843164994 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0ab600 con 0x7f8f6015ba00 2024-03-30T00:41:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.033+0000 7f8f5fe00640 1 -- 172.21.15.66:0/3843164994 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4c0a7050 con 0x7f8f6015ba00 2024-03-30T00:41:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.033+0000 7f8f5fe00640 1 -- 172.21.15.66:0/3843164994 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0ab980 con 0x7f8f6015ba00 2024-03-30T00:41:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 -- 172.21.15.66:0/3843164994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015ba00 msgr2=0x7f8f600ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 --2- 172.21.15.66:0/3843164994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015ba00 0x7f8f600ce7b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0097c0 tx=0x7f8f4c01c3e0 comp rx=0 tx=0).stop 2024-03-30T00:41:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 -- 172.21.15.66:0/3843164994 shutdown_connections 2024-03-30T00:41:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 --2- 172.21.15.66:0/3843164994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015ba00 0x7f8f600ce7b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 --2- 172.21.15.66:0/3843164994 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f600acaf0 0x7f8f600b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 -- 172.21.15.66:0/3843164994 >> 172.21.15.66:0/3843164994 conn(0x7f8f6006a190 msgr2=0x7f8f6006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 -- 172.21.15.66:0/3843164994 shutdown_connections 2024-03-30T00:41:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 -- 172.21.15.66:0/3843164994 wait complete. 2024-03-30T00:41:11.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 Processor -- start 2024-03-30T00:41:11.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 -- start start 2024-03-30T00:41:11.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f600acaf0 0x7f8f60156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:11.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f600cecf0 0x7f8f600caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:11.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015ba00 0x7f8f600cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:11.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60156990 con 0x7f8f6015ba00 2024-03-30T00:41:11.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600c9770 con 0x7f8f600cecf0 2024-03-30T00:41:11.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f68400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600c9a50 con 0x7f8f600acaf0 2024-03-30T00:41:11.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f66400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015ba00 0x7f8f600cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:11.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f66400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015ba00 0x7f8f600cb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39596/0 (socket says 172.21.15.66:39596) 2024-03-30T00:41:11.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f66400640 1 -- 172.21.15.66:0/1111584032 learned_addr learned my addr 172.21.15.66:0/1111584032 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:11.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f65000640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f600cecf0 0x7f8f600caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:11.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f65a00640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f600acaf0 0x7f8f60156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f66400640 1 -- 172.21.15.66:0/1111584032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f600acaf0 msgr2=0x7f8f60156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f66400640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f600acaf0 0x7f8f60156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f66400640 1 -- 172.21.15.66:0/1111584032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f600cecf0 msgr2=0x7f8f600caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f66400640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f600cecf0 0x7f8f600caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.037+0000 7f8f66400640 1 -- 172.21.15.66:0/1111584032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c009420 con 0x7f8f6015ba00 2024-03-30T00:41:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.041+0000 7f8f66400640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015ba00 0x7f8f600cb030 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800c780 tx=0x7f8f5800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.041+0000 7f8f5d600640 1 -- 172.21.15.66:0/1111584032 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800ce90 con 0x7f8f6015ba00 2024-03-30T00:41:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.041+0000 7f8f5d600640 1 -- 172.21.15.66:0/1111584032 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5800a640 con 0x7f8f6015ba00 2024-03-30T00:41:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.041+0000 7f8f68400640 1 -- 172.21.15.66:0/1111584032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f600c9d30 con 0x7f8f6015ba00 2024-03-30T00:41:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.041+0000 7f8f5d600640 1 -- 172.21.15.66:0/1111584032 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800a9d0 con 0x7f8f6015ba00 2024-03-30T00:41:11.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.041+0000 7f8f68400640 1 -- 172.21.15.66:0/1111584032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f600ca2f0 con 0x7f8f6015ba00 2024-03-30T00:41:11.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.041+0000 7f8f68400640 1 -- 172.21.15.66:0/1111584032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f60066c30 con 0x7f8f6015ba00 2024-03-30T00:41:11.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.041+0000 7f8f5d600640 1 -- 172.21.15.66:0/1111584032 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8f5800ab30 con 0x7f8f6015ba00 2024-03-30T00:41:11.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.041+0000 7f8f5d600640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f40066690 0x7f8f40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:11.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.041+0000 7f8f5d600640 1 -- 172.21.15.66:0/1111584032 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6807+0+0 (secure 0 0 0) 0x7f8f58083d70 con 0x7f8f6015ba00 2024-03-30T00:41:11.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.041+0000 7f8f65a00640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f40066690 0x7f8f40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:11.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.045+0000 7f8f65a00640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f40066690 0x7f8f40068b50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8f601659f0 tx=0x7f8f54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:11.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.049+0000 7f8f5d600640 1 -- 172.21.15.66:0/1111584032 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8f58050ca0 con 0x7f8f6015ba00 2024-03-30T00:41:11.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.213+0000 7f8f68400640 1 -- 172.21.15.66:0/1111584032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8f6006a950 con 0x7f8f6015ba00 2024-03-30T00:41:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.817+0000 7f8f5d600640 1 -- 172.21.15.66:0/1111584032 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8f58018020 con 0x7f8f6015ba00 2024-03-30T00:41:11.826 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-03-30T00:41:11.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 -- 172.21.15.66:0/1111584032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f40066690 msgr2=0x7f8f40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:11.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f40066690 0x7f8f40068b50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8f601659f0 tx=0x7f8f54008040 comp rx=0 tx=0).stop 2024-03-30T00:41:11.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 -- 172.21.15.66:0/1111584032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015ba00 msgr2=0x7f8f600cb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:11.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015ba00 0x7f8f600cb030 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800c780 tx=0x7f8f5800cc40 comp rx=0 tx=0).stop 2024-03-30T00:41:11.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 -- 172.21.15.66:0/1111584032 shutdown_connections 2024-03-30T00:41:11.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f40066690 0x7f8f40068b50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:11.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015ba00 0x7f8f600cb030 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:11.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f600cecf0 0x7f8f600caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:11.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 --2- 172.21.15.66:0/1111584032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f600acaf0 0x7f8f60156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:11.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 -- 172.21.15.66:0/1111584032 >> 172.21.15.66:0/1111584032 conn(0x7f8f6006a190 msgr2=0x7f8f600aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:11.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 -- 172.21.15.66:0/1111584032 shutdown_connections 2024-03-30T00:41:11.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.821+0000 7f8f68400640 1 -- 172.21.15.66:0/1111584032 wait complete. 2024-03-30T00:41:11.854 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T00:41:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff82a800640 1 Processor -- start 2024-03-30T00:41:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff82a800640 1 -- start start 2024-03-30T00:41:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff82a800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8240ceec0 0x7ff8241579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff82a800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8240cbf80 con 0x7ff8240acaf0 2024-03-30T00:41:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff82a800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8240cc820 con 0x7ff82415ba00 2024-03-30T00:41:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff82a800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240b2be0 con 0x7ff8240ceec0 2024-03-30T00:41:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff823400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff8240acaf0 0x7ff8240b24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48858/0 (socket says 172.21.15.66:48858) 2024-03-30T00:41:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff823400640 1 -- 172.21.15.66:0/2465291446 learned_addr learned my addr 172.21.15.66:0/2465291446 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff823e00640 1 --2- 172.21.15.66:0/2465291446 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8240ceec0 0x7ff8241579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff822a00640 1 -- 172.21.15.66:0/2465291446 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496081831 0 0) 0x7ff8240cbf80 con 0x7ff8240acaf0 2024-03-30T00:41:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff822a00640 1 -- 172.21.15.66:0/2465291446 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff808003530 con 0x7ff8240acaf0 2024-03-30T00:41:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff823e00640 1 -- 172.21.15.66:0/2465291446 >> v1:172.21.15.156:6789/0 conn(0x7ff82415ba00 legacy=0x7ff8240ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff823e00640 1 -- 172.21.15.66:0/2465291446 >> v1:172.21.15.66:6789/0 conn(0x7ff8240acaf0 legacy=0x7ff8240b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.993+0000 7ff823e00640 1 -- 172.21.15.66:0/2465291446 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8240b2dd0 con 0x7ff8240ceec0 2024-03-30T00:41:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.997+0000 7ff823e00640 1 --2- 172.21.15.66:0/2465291446 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8240ceec0 0x7ff8241579e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0097f0 tx=0x7ff81c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c11ee9aaa4bed5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:12.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.997+0000 7ff822a00640 1 -- 172.21.15.66:0/2465291446 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0ab620 con 0x7ff8240ceec0 2024-03-30T00:41:12.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.997+0000 7ff822a00640 1 -- 172.21.15.66:0/2465291446 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c0ab780 con 0x7ff8240ceec0 2024-03-30T00:41:12.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.997+0000 7ff822a00640 1 -- 172.21.15.66:0/2465291446 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0abaf0 con 0x7ff8240ceec0 2024-03-30T00:41:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.997+0000 7ff82a800640 1 -- 172.21.15.66:0/2465291446 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8240ceec0 msgr2=0x7ff8241579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.997+0000 7ff82a800640 1 --2- 172.21.15.66:0/2465291446 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8240ceec0 0x7ff8241579e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0097f0 tx=0x7ff81c01c420 comp rx=0 tx=0).stop 2024-03-30T00:41:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.997+0000 7ff82a800640 1 -- 172.21.15.66:0/2465291446 shutdown_connections 2024-03-30T00:41:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.997+0000 7ff82a800640 1 --2- 172.21.15.66:0/2465291446 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8240ceec0 0x7ff8241579e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.997+0000 7ff82a800640 1 -- 172.21.15.66:0/2465291446 >> 172.21.15.66:0/2465291446 conn(0x7ff82406a190 msgr2=0x7ff82406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.997+0000 7ff82a800640 1 -- 172.21.15.66:0/2465291446 shutdown_connections 2024-03-30T00:41:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:11.997+0000 7ff82a800640 1 -- 172.21.15.66:0/2465291446 wait complete. 2024-03-30T00:41:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff82a800640 1 Processor -- start 2024-03-30T00:41:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff82a800640 1 -- start start 2024-03-30T00:41:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff82a800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8240acaf0 0x7ff824152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff82a800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8240ceec0 0x7ff824097960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff82a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82415ba00 0x7ff824156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff82a800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240cb9c0 con 0x7ff82415ba00 2024-03-30T00:41:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff82a800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff824097ea0 con 0x7ff8240acaf0 2024-03-30T00:41:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff82a800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240cb090 con 0x7ff8240ceec0 2024-03-30T00:41:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff823e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82415ba00 0x7ff824156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff823e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82415ba00 0x7ff824156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39610/0 (socket says 172.21.15.66:39610) 2024-03-30T00:41:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff823e00640 1 -- 172.21.15.66:0/3872849619 learned_addr learned my addr 172.21.15.66:0/3872849619 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff823400640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8240acaf0 0x7ff824152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff81aa00640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8240ceec0 0x7ff824097960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff823e00640 1 -- 172.21.15.66:0/3872849619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8240ceec0 msgr2=0x7ff824097960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:12.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff823e00640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8240ceec0 0x7ff824097960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:12.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff823e00640 1 -- 172.21.15.66:0/3872849619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8240acaf0 msgr2=0x7ff824152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:12.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff823e00640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8240acaf0 0x7ff824152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:12.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff823e00640 1 -- 172.21.15.66:0/3872849619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c009420 con 0x7ff82415ba00 2024-03-30T00:41:12.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff823e00640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82415ba00 0x7ff824156870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff81c01aad0 tx=0x7ff81c0191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:12.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.001+0000 7ff81be00640 1 -- 172.21.15.66:0/3872849619 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004510 con 0x7ff82415ba00 2024-03-30T00:41:12.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.005+0000 7ff81be00640 1 -- 172.21.15.66:0/3872849619 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c0ab8f0 con 0x7ff82415ba00 2024-03-30T00:41:12.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.005+0000 7ff81be00640 1 -- 172.21.15.66:0/3872849619 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0abca0 con 0x7ff82415ba00 2024-03-30T00:41:12.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.005+0000 7ff82a800640 1 -- 172.21.15.66:0/3872849619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8240cb370 con 0x7ff82415ba00 2024-03-30T00:41:12.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.005+0000 7ff82a800640 1 -- 172.21.15.66:0/3872849619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82415b4d0 con 0x7ff82415ba00 2024-03-30T00:41:12.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.005+0000 7ff82a800640 1 -- 172.21.15.66:0/3872849619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff824066c30 con 0x7ff82415ba00 2024-03-30T00:41:12.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.005+0000 7ff81be00640 1 -- 172.21.15.66:0/3872849619 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff81c003620 con 0x7ff82415ba00 2024-03-30T00:41:12.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.005+0000 7ff81be00640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8000666c0 0x7ff800068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:12.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.005+0000 7ff81be00640 1 -- 172.21.15.66:0/3872849619 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6369+0+0 (secure 0 0 0) 0x7ff81c11e8c0 con 0x7ff82415ba00 2024-03-30T00:41:12.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.005+0000 7ff823400640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8000666c0 0x7ff800068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:12.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.005+0000 7ff823400640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8000666c0 0x7ff800068b80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff814001570 tx=0x7ff814009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:12.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.009+0000 7ff81be00640 1 -- 172.21.15.66:0/3872849619 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff81c0eba10 con 0x7ff82415ba00 2024-03-30T00:41:12.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.181+0000 7ff82a800640 1 -- 172.21.15.66:0/3872849619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff824153e80 con 0x7ff82415ba00 2024-03-30T00:41:12.830 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-03-30T00:41:12.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.821+0000 7ff81be00640 1 -- 172.21.15.66:0/3872849619 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff81c0c10d0 con 0x7ff82415ba00 2024-03-30T00:41:12.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.821+0000 7ff82a800640 1 -- 172.21.15.66:0/3872849619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8000666c0 msgr2=0x7ff800068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:12.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.821+0000 7ff82a800640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8000666c0 0x7ff800068b80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff814001570 tx=0x7ff814009040 comp rx=0 tx=0).stop 2024-03-30T00:41:12.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.821+0000 7ff82a800640 1 -- 172.21.15.66:0/3872849619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82415ba00 msgr2=0x7ff824156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:12.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.821+0000 7ff82a800640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82415ba00 0x7ff824156870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff81c01aad0 tx=0x7ff81c0191c0 comp rx=0 tx=0).stop 2024-03-30T00:41:12.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.825+0000 7ff82a800640 1 -- 172.21.15.66:0/3872849619 shutdown_connections 2024-03-30T00:41:12.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.825+0000 7ff82a800640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8000666c0 0x7ff800068b80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:12.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.825+0000 7ff82a800640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82415ba00 0x7ff824156870 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:12.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.825+0000 7ff82a800640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8240ceec0 0x7ff824097960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:12.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.825+0000 7ff82a800640 1 --2- 172.21.15.66:0/3872849619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8240acaf0 0x7ff824152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:12.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.825+0000 7ff82a800640 1 -- 172.21.15.66:0/3872849619 >> 172.21.15.66:0/3872849619 conn(0x7ff82406a190 msgr2=0x7ff824157400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:12.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.825+0000 7ff82a800640 1 -- 172.21.15.66:0/3872849619 shutdown_connections 2024-03-30T00:41:12.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.825+0000 7ff82a800640 1 -- 172.21.15.66:0/3872849619 wait complete. 2024-03-30T00:41:12.859 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-30T00:41:13.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a65e00640 1 Processor -- start 2024-03-30T00:41:13.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a65e00640 1 -- start start 2024-03-30T00:41:13.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a65e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2a6006b4b0 0x7f2a600c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:13.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a65e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6006d930 con 0x7f2a6006edd0 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a65e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6015ce90 con 0x7f2a6015cab0 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a65e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600caf10 con 0x7f2a6006b4b0 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a5f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2a6006edd0 0x7f2a600c9750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48874/0 (socket says 172.21.15.66:48874) 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a5f400640 1 -- 172.21.15.66:0/1353719475 learned_addr learned my addr 172.21.15.66:0/1353719475 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a56000640 1 --2- 172.21.15.66:0/1353719475 >> v2:172.21.15.156:3300/0 conn(0x7f2a6006b4b0 0x7f2a600c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a5e000640 1 -- 172.21.15.66:0/1353719475 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630305442 0 0) 0x7f2a6006d930 con 0x7f2a6006edd0 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a5e000640 1 -- 172.21.15.66:0/1353719475 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a44003530 con 0x7f2a6006edd0 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a5e000640 1 -- 172.21.15.66:0/1353719475 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812039414 0 0) 0x7f2a6015ce90 con 0x7f2a6015cab0 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a5e000640 1 -- 172.21.15.66:0/1353719475 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a6006d930 con 0x7f2a6015cab0 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a56000640 1 -- 172.21.15.66:0/1353719475 >> v1:172.21.15.156:6789/0 conn(0x7f2a6015cab0 legacy=0x7f2a6006ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a56000640 1 -- 172.21.15.66:0/1353719475 >> v1:172.21.15.66:6789/0 conn(0x7f2a6006edd0 legacy=0x7f2a600c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a5e000640 1 -- 172.21.15.66:0/1353719475 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1597090763 0 0) 0x7f2a44003530 con 0x7f2a6006edd0 2024-03-30T00:41:13.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:12.997+0000 7f2a56000640 1 -- 172.21.15.66:0/1353719475 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6006d2e0 con 0x7f2a6006b4b0 2024-03-30T00:41:13.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.001+0000 7f2a56000640 1 --2- 172.21.15.66:0/1353719475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6006b4b0 0x7f2a600c9210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2a480097c0 tx=0x7f2a4801c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3e46f8d4530015 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:13.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.001+0000 7f2a5e000640 1 -- 172.21.15.66:0/1353719475 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58001a00 con 0x7f2a6006b4b0 2024-03-30T00:41:13.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.001+0000 7f2a5e000640 1 -- 172.21.15.66:0/1353719475 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a480a7050 con 0x7f2a6006b4b0 2024-03-30T00:41:13.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.001+0000 7f2a5e000640 1 -- 172.21.15.66:0/1353719475 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a480ab620 con 0x7f2a6006b4b0 2024-03-30T00:41:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.001+0000 7f2a65e00640 1 -- 172.21.15.66:0/1353719475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6006b4b0 msgr2=0x7f2a600c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.001+0000 7f2a65e00640 1 --2- 172.21.15.66:0/1353719475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6006b4b0 0x7f2a600c9210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2a480097c0 tx=0x7f2a4801c390 comp rx=0 tx=0).stop 2024-03-30T00:41:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.001+0000 7f2a65e00640 1 -- 172.21.15.66:0/1353719475 shutdown_connections 2024-03-30T00:41:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.001+0000 7f2a65e00640 1 --2- 172.21.15.66:0/1353719475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6006b4b0 0x7f2a600c9210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.001+0000 7f2a65e00640 1 -- 172.21.15.66:0/1353719475 >> 172.21.15.66:0/1353719475 conn(0x7f2a600708d0 msgr2=0x7f2a60070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.001+0000 7f2a65e00640 1 -- 172.21.15.66:0/1353719475 shutdown_connections 2024-03-30T00:41:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.001+0000 7f2a65e00640 1 -- 172.21.15.66:0/1353719475 wait complete. 2024-03-30T00:41:13.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a65e00640 1 Processor -- start 2024-03-30T00:41:13.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a65e00640 1 -- start start 2024-03-30T00:41:13.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a65e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6006b4b0 0x7f2a6014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:13.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a65e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a6006edd0 0x7f2a6013c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:13.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a65e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a6015cab0 0x7f2a600ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:13.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a65e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6013d470 con 0x7f2a6015cab0 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a65e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6013d750 con 0x7f2a6006b4b0 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a65e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6013ab00 con 0x7f2a6006edd0 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a5f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a6015cab0 0x7f2a600ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a5f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a6015cab0 0x7f2a600ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39622/0 (socket says 172.21.15.66:39622) 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a5f400640 1 -- 172.21.15.66:0/1149836398 learned_addr learned my addr 172.21.15.66:0/1149836398 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a5ea00640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6006b4b0 0x7f2a6014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a56000640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a6006edd0 0x7f2a6013c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a5f400640 1 -- 172.21.15.66:0/1149836398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a6006edd0 msgr2=0x7f2a6013c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a5f400640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a6006edd0 0x7f2a6013c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a5f400640 1 -- 172.21.15.66:0/1149836398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6006b4b0 msgr2=0x7f2a6014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a5f400640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6006b4b0 0x7f2a6014e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a5f400640 1 -- 172.21.15.66:0/1149836398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a48009420 con 0x7f2a6015cab0 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a5f400640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a6015cab0 0x7f2a600ca240 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2a58001f50 tx=0x7f2a5800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a57e00640 1 -- 172.21.15.66:0/1149836398 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58012400 con 0x7f2a6015cab0 2024-03-30T00:41:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a57e00640 1 -- 172.21.15.66:0/1149836398 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a58012560 con 0x7f2a6015cab0 2024-03-30T00:41:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a57e00640 1 -- 172.21.15.66:0/1149836398 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58012930 con 0x7f2a6015cab0 2024-03-30T00:41:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a65e00640 1 -- 172.21.15.66:0/1149836398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6013ade0 con 0x7f2a6015cab0 2024-03-30T00:41:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a65e00640 1 -- 172.21.15.66:0/1149836398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6013cec0 con 0x7f2a6015cab0 2024-03-30T00:41:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.005+0000 7f2a65e00640 1 -- 172.21.15.66:0/1149836398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6013a960 con 0x7f2a6015cab0 2024-03-30T00:41:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.009+0000 7f2a57e00640 1 -- 172.21.15.66:0/1149836398 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2a58012a90 con 0x7f2a6015cab0 2024-03-30T00:41:13.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.009+0000 7f2a57e00640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a3c066690 0x7f2a3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:13.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.009+0000 7f2a57e00640 1 -- 172.21.15.66:0/1149836398 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f2a58084850 con 0x7f2a6015cab0 2024-03-30T00:41:13.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.009+0000 7f2a5ea00640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a3c066690 0x7f2a3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:13.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.009+0000 7f2a5ea00640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a3c066690 0x7f2a3c068b50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a500011f0 tx=0x7f2a50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:13.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.013+0000 7f2a57e00640 1 -- 172.21.15.66:0/1149836398 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2a6013a960 con 0x7f2a6015cab0 2024-03-30T00:41:13.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.173+0000 7f2a65e00640 1 -- 172.21.15.66:0/1149836398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2a6013a960 con 0x7f2a6015cab0 2024-03-30T00:41:13.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.833+0000 7f2a57e00640 1 -- 172.21.15.66:0/1149836398 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f2a6013a960 con 0x7f2a6015cab0 2024-03-30T00:41:13.844 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data_ec' removed 2024-03-30T00:41:13.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 -- 172.21.15.66:0/1149836398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a3c066690 msgr2=0x7f2a3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:13.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a3c066690 0x7f2a3c068b50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a500011f0 tx=0x7f2a50009040 comp rx=0 tx=0).stop 2024-03-30T00:41:13.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 -- 172.21.15.66:0/1149836398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a6015cab0 msgr2=0x7f2a600ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:13.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a6015cab0 0x7f2a600ca240 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2a58001f50 tx=0x7f2a5800a430 comp rx=0 tx=0).stop 2024-03-30T00:41:13.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 -- 172.21.15.66:0/1149836398 shutdown_connections 2024-03-30T00:41:13.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a3c066690 0x7f2a3c068b50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:13.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a6015cab0 0x7f2a600ca240 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:13.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a6006edd0 0x7f2a6013c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:13.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 --2- 172.21.15.66:0/1149836398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6006b4b0 0x7f2a6014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:13.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 -- 172.21.15.66:0/1149836398 >> 172.21.15.66:0/1149836398 conn(0x7f2a600708d0 msgr2=0x7f2a600af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:13.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 -- 172.21.15.66:0/1149836398 shutdown_connections 2024-03-30T00:41:13.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:13.837+0000 7f2a65e00640 1 -- 172.21.15.66:0/1149836398 wait complete. 2024-03-30T00:41:13.873 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T00:41:13.873 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-30T00:41:13.873 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T00:41:13.874 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:41:13.874 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T00:41:13.874 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-30T00:41:13.874 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T00:41:13.874 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:41:13.874 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T00:41:13.875 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-30T00:41:13.875 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T00:41:13.875 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:41:13.875 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T00:41:13.875 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-30T00:41:13.875 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T00:41:13.875 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:41:13.887 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T00:41:13.887 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T00:41:13.888 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T00:41:13.888 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T00:41:13.891 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T00:41:13.891 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T00:41:13.891 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T00:41:13.892 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T00:41:13.931 INFO:tasks.ceph.mds.a:Started 2024-03-30T00:41:13.931 DEBUG:teuthology.parallel:result is None 2024-03-30T00:41:13.933 INFO:tasks.ceph.mds.c:Started 2024-03-30T00:41:13.933 DEBUG:teuthology.parallel:result is None 2024-03-30T00:41:13.934 INFO:tasks.ceph.mds.d:Started 2024-03-30T00:41:13.934 DEBUG:teuthology.parallel:result is None 2024-03-30T00:41:13.936 INFO:tasks.ceph.mds.b:Started 2024-03-30T00:41:13.936 DEBUG:teuthology.parallel:result is None 2024-03-30T00:41:13.936 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-30T00:41:14.022 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:41:14.024 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2024-03-30T00:41:14.024 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2024-03-30T00:41:14.025 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:41:14.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.089+0000 7efe84600640 1 Processor -- start 2024-03-30T00:41:14.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.089+0000 7efe84600640 1 -- start start 2024-03-30T00:41:14.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.089+0000 7efe84600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe7c0b0fb0 0x7efe7c06b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:14.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.089+0000 7efe84600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efe7c0b13f0 0x7efe7c06b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:14.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.089+0000 7efe84600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7c098ac0 con 0x7efe7c0cca40 2024-03-30T00:41:14.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.089+0000 7efe84600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7c0cf3f0 con 0x7efe7c0b13f0 2024-03-30T00:41:14.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.089+0000 7efe84600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0cf670 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.089+0000 7efe82600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efe7c0b13f0 0x7efe7c06b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:14.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.089+0000 7efe82600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efe7c0b13f0 0x7efe7c06b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39680/0 (socket says 172.21.15.66:39680) 2024-03-30T00:41:14.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.089+0000 7efe82600640 1 -- 172.21.15.66:0/2915841957 learned_addr learned my addr 172.21.15.66:0/2915841957 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:14.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe81200640 1 --2- 172.21.15.66:0/2915841957 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe7c0b0fb0 0x7efe7c06b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:14.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe82600640 1 -- 172.21.15.66:0/2915841957 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe7c0b0fb0 msgr2=0x7efe7c06b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:14.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe82600640 1 --2- 172.21.15.66:0/2915841957 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe7c0b0fb0 0x7efe7c06b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:14.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe82600640 1 -- 172.21.15.66:0/2915841957 >> v1:172.21.15.156:6789/0 conn(0x7efe7c0cca40 legacy=0x7efe7c0cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:14.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe82600640 1 -- 172.21.15.66:0/2915841957 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c0cf950 con 0x7efe7c0b13f0 2024-03-30T00:41:14.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe7be00640 1 -- 172.21.15.66:0/2915841957 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311852326 0 0) 0x7efe7c098ac0 con 0x7efe7c0cca40 2024-03-30T00:41:14.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe82600640 1 --2- 172.21.15.66:0/2915841957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe7c0b13f0 0x7efe7c06b680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efe74009e50 tx=0x7efe7401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e931e675f15ec172 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:14.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe7be00640 1 -- 172.21.15.66:0/2915841957 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe740a5860 con 0x7efe7c0b13f0 2024-03-30T00:41:14.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe7be00640 1 -- 172.21.15.66:0/2915841957 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe740a59c0 con 0x7efe7c0b13f0 2024-03-30T00:41:14.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe7be00640 1 -- 172.21.15.66:0/2915841957 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe740a5d50 con 0x7efe7c0b13f0 2024-03-30T00:41:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe84600640 1 -- 172.21.15.66:0/2915841957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe7c0b13f0 msgr2=0x7efe7c06b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.093+0000 7efe84600640 1 --2- 172.21.15.66:0/2915841957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe7c0b13f0 0x7efe7c06b680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efe74009e50 tx=0x7efe7401ca00 comp rx=0 tx=0).stop 2024-03-30T00:41:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 -- 172.21.15.66:0/2915841957 shutdown_connections 2024-03-30T00:41:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 --2- 172.21.15.66:0/2915841957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe7c0b13f0 0x7efe7c06b680 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 --2- 172.21.15.66:0/2915841957 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe7c0b0fb0 0x7efe7c06b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 -- 172.21.15.66:0/2915841957 >> 172.21.15.66:0/2915841957 conn(0x7efe7c0b2130 msgr2=0x7efe7c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 -- 172.21.15.66:0/2915841957 shutdown_connections 2024-03-30T00:41:14.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 -- 172.21.15.66:0/2915841957 wait complete. 2024-03-30T00:41:14.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 Processor -- start 2024-03-30T00:41:14.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 -- start start 2024-03-30T00:41:14.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe7c0b0fb0 0x7efe7c153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe7c0b13f0 0x7efe7c1538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7c0cca40 0x7efe7c06f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c153e10 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0b4e60 con 0x7efe7c0cca40 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe84600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0b4530 con 0x7efe7c0b13f0 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe81c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe7c0b0fb0 0x7efe7c153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe81c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe7c0b0fb0 0x7efe7c153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39690/0 (socket says 172.21.15.66:39690) 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe81c00640 1 -- 172.21.15.66:0/2369730850 learned_addr learned my addr 172.21.15.66:0/2369730850 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe81200640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe7c0b13f0 0x7efe7c1538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe82600640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7c0cca40 0x7efe7c06f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe81c00640 1 -- 172.21.15.66:0/2369730850 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe7c0b13f0 msgr2=0x7efe7c1538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe81c00640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe7c0b13f0 0x7efe7c1538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe81c00640 1 -- 172.21.15.66:0/2369730850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7c0cca40 msgr2=0x7efe7c06f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe81c00640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7c0cca40 0x7efe7c06f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe81c00640 1 -- 172.21.15.66:0/2369730850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe74009b30 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe82600640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7c0cca40 0x7efe7c06f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.097+0000 7efe81c00640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe7c0b0fb0 0x7efe7c153f70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efe70001ec0 tx=0x7efe7000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.101+0000 7efe79600640 1 -- 172.21.15.66:0/2369730850 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700054e0 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.101+0000 7efe79600640 1 -- 172.21.15.66:0/2369730850 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe70003600 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.101+0000 7efe79600640 1 -- 172.21.15.66:0/2369730850 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700039b0 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.101+0000 7efe84600640 1 -- 172.21.15.66:0/2369730850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7c0b4810 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.101+0000 7efe84600640 1 -- 172.21.15.66:0/2369730850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7c06f900 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.101+0000 7efe84600640 1 -- 172.21.15.66:0/2369730850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7c0677d0 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.101+0000 7efe79600640 1 -- 172.21.15.66:0/2369730850 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efe70003b10 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.101+0000 7efe79600640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe5c066690 0x7efe5c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:14.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.101+0000 7efe79600640 1 -- 172.21.15.66:0/2369730850 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5263+0+0 (secure 0 0 0) 0x7efe70083660 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.101+0000 7efe81200640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe5c066690 0x7efe5c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:14.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.101+0000 7efe81200640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe5c066690 0x7efe5c068b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efe7c15a560 tx=0x7efe68009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:14.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.105+0000 7efe79600640 1 -- 172.21.15.66:0/2369730850 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efe70050b10 con 0x7efe7c0b0fb0 2024-03-30T00:41:14.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:14.281+0000 7efe84600640 1 -- 172.21.15.66:0/2369730850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7efe7c066c30 con 0x7efe7c0b0fb0 2024-03-30T00:41:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.061+0000 7efe79600640 1 -- 172.21.15.66:0/2369730850 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7efe70017070 con 0x7efe7c0b0fb0 2024-03-30T00:41:15.071 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-03-30T00:41:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 -- 172.21.15.66:0/2369730850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe5c066690 msgr2=0x7efe5c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe5c066690 0x7efe5c068b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efe7c15a560 tx=0x7efe68009190 comp rx=0 tx=0).stop 2024-03-30T00:41:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 -- 172.21.15.66:0/2369730850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe7c0b0fb0 msgr2=0x7efe7c153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe7c0b0fb0 0x7efe7c153f70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efe70001ec0 tx=0x7efe7000c8e0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 -- 172.21.15.66:0/2369730850 shutdown_connections 2024-03-30T00:41:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7c0cca40 0x7efe7c06f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe7c0b13f0 0x7efe7c1538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe5c066690 0x7efe5c068b50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 --2- 172.21.15.66:0/2369730850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe7c0b0fb0 0x7efe7c153f70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 -- 172.21.15.66:0/2369730850 >> 172.21.15.66:0/2369730850 conn(0x7efe7c0b2130 msgr2=0x7efe7c06e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 -- 172.21.15.66:0/2369730850 shutdown_connections 2024-03-30T00:41:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.065+0000 7efe84600640 1 -- 172.21.15.66:0/2369730850 wait complete. 2024-03-30T00:41:15.097 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-30T00:41:15.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fe005200640 1 Processor -- start 2024-03-30T00:41:15.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fe005200640 1 -- start start 2024-03-30T00:41:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fe005200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0000ce6b0 0x7fe00015e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fe005200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe000070b50 0x7fe000070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fe005200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe00006c410 con 0x7fe00015db90 2024-03-30T00:41:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fe005200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe000071520 con 0x7fe000070b50 2024-03-30T00:41:15.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fe005200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00006c6b0 con 0x7fe0000ce6b0 2024-03-30T00:41:15.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fdffde00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0000ce6b0 0x7fe00015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fdffe800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe000070b50 0x7fe000070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fdff5400640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe00015db90 0x7fe0000cea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:43170/0 (socket says 172.21.15.66:43170) 2024-03-30T00:41:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fdffde00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0000ce6b0 0x7fe00015e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:36702/0 (socket says 172.21.15.66:36702) 2024-03-30T00:41:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fdffe800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe000070b50 0x7fe000070fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:60678/0 (socket says 172.21.15.66:60678) 2024-03-30T00:41:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.233+0000 7fdff5400640 1 -- 172.21.15.66:0/889891196 learned_addr learned my addr 172.21.15.66:0/889891196 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.237+0000 7fdffde00640 1 -- 172.21.15.66:0/889891196 >> v2:172.21.15.156:3300/0 conn(0x7fe000070b50 msgr2=0x7fe000070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.237+0000 7fdffde00640 1 --2- 172.21.15.66:0/889891196 >> v2:172.21.15.156:3300/0 conn(0x7fe000070b50 0x7fe000070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.237+0000 7fdffde00640 1 -- 172.21.15.66:0/889891196 >> v1:172.21.15.156:6789/0 conn(0x7fe00015db90 legacy=0x7fe0000cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.237+0000 7fdffde00640 1 -- 172.21.15.66:0/889891196 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00006c990 con 0x7fe0000ce6b0 2024-03-30T00:41:15.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.237+0000 7fdffd400640 1 -- 172.21.15.66:0/889891196 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151093756 0 0) 0x7fe00006c410 con 0x7fe00015db90 2024-03-30T00:41:15.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.237+0000 7fdffde00640 1 --2- 172.21.15.66:0/889891196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0000ce6b0 0x7fe00015e070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdff00097c0 tx=0x7fdff001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e202d126c5dae298 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:15.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.237+0000 7fdffd400640 1 -- 172.21.15.66:0/889891196 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff00ac640 con 0x7fe0000ce6b0 2024-03-30T00:41:15.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.237+0000 7fdffd400640 1 -- 172.21.15.66:0/889891196 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff00ac7a0 con 0x7fe0000ce6b0 2024-03-30T00:41:15.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.237+0000 7fdffd400640 1 -- 172.21.15.66:0/889891196 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff00acb30 con 0x7fe0000ce6b0 2024-03-30T00:41:15.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.237+0000 7fe005200640 1 -- 172.21.15.66:0/889891196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0000ce6b0 msgr2=0x7fe00015e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.237+0000 7fe005200640 1 --2- 172.21.15.66:0/889891196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0000ce6b0 0x7fe00015e070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdff00097c0 tx=0x7fdff001c3e0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 -- 172.21.15.66:0/889891196 shutdown_connections 2024-03-30T00:41:15.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 --2- 172.21.15.66:0/889891196 >> v2:172.21.15.156:3300/0 conn(0x7fe000070b50 0x7fe000070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 --2- 172.21.15.66:0/889891196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0000ce6b0 0x7fe00015e070 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 -- 172.21.15.66:0/889891196 >> 172.21.15.66:0/889891196 conn(0x7fe000066100 msgr2=0x7fe000069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:15.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 -- 172.21.15.66:0/889891196 shutdown_connections 2024-03-30T00:41:15.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 -- 172.21.15.66:0/889891196 wait complete. 2024-03-30T00:41:15.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 Processor -- start 2024-03-30T00:41:15.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 -- start start 2024-03-30T00:41:15.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe000070b50 0x7fe000156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0000ce6b0 0x7fe00014ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe00015db90 0x7fe000159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000155940 con 0x7fe0000ce6b0 2024-03-30T00:41:15.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000156480 con 0x7fe00015db90 2024-03-30T00:41:15.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fe005200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00014b140 con 0x7fe000070b50 2024-03-30T00:41:15.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdff5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0000ce6b0 0x7fe00014ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdffde00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe000070b50 0x7fe000156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdffde00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe000070b50 0x7fe000156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:36714/0 (socket says 172.21.15.66:36714) 2024-03-30T00:41:15.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdffde00640 1 -- 172.21.15.66:0/250177863 learned_addr learned my addr 172.21.15.66:0/250177863 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:15.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdffe800640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe00015db90 0x7fe000159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdffde00640 1 -- 172.21.15.66:0/250177863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe00015db90 msgr2=0x7fe000159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdffde00640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe00015db90 0x7fe000159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdffde00640 1 -- 172.21.15.66:0/250177863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0000ce6b0 msgr2=0x7fe00014ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdffde00640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0000ce6b0 0x7fe00014ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdffde00640 1 -- 172.21.15.66:0/250177863 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff0009420 con 0x7fe000070b50 2024-03-30T00:41:15.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdffde00640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe000070b50 0x7fe000156910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdff001a010 tx=0x7fdff001ac20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:15.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.241+0000 7fdff6a00640 1 -- 172.21.15.66:0/250177863 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff00b8650 con 0x7fe000070b50 2024-03-30T00:41:15.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.245+0000 7fdff6a00640 1 -- 172.21.15.66:0/250177863 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff00b87b0 con 0x7fe000070b50 2024-03-30T00:41:15.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.245+0000 7fdff6a00640 1 -- 172.21.15.66:0/250177863 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff00b8b60 con 0x7fe000070b50 2024-03-30T00:41:15.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.245+0000 7fe005200640 1 -- 172.21.15.66:0/250177863 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00006e190 con 0x7fe000070b50 2024-03-30T00:41:15.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.245+0000 7fe005200640 1 -- 172.21.15.66:0/250177863 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe000155b50 con 0x7fe000070b50 2024-03-30T00:41:15.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.245+0000 7fe005200640 1 -- 172.21.15.66:0/250177863 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00006e320 con 0x7fe000070b50 2024-03-30T00:41:15.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.245+0000 7fdff6a00640 1 -- 172.21.15.66:0/250177863 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdff00b8cc0 con 0x7fe000070b50 2024-03-30T00:41:15.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.245+0000 7fdff6a00640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfdc0666c0 0x7fdfdc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.245+0000 7fdff6a00640 1 -- 172.21.15.66:0/250177863 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fdff011f250 con 0x7fe000070b50 2024-03-30T00:41:15.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.245+0000 7fdff5400640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfdc0666c0 0x7fdfdc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.245+0000 7fdff5400640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfdc0666c0 0x7fdfdc068b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8000f70 tx=0x7fdfe8007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:15.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.249+0000 7fdff6a00640 1 -- 172.21.15.66:0/250177863 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdff00ec8e0 con 0x7fe000070b50 2024-03-30T00:41:15.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.465+0000 7fe005200640 1 -- 172.21.15.66:0/250177863 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe000066c30 con 0x7fe000070b50 2024-03-30T00:41:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.465+0000 7fdff6a00640 1 -- 172.21.15.66:0/250177863 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fdff00efe80 con 0x7fe000070b50 2024-03-30T00:41:15.475 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm857MDRAAu2pldF20deZfoSDDXuTF/A==", 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMmNFGhAAlHS6ZpIoJc7tlYazCISiLw==", 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmuiBsFRAA2kez6ylAylKTWsQ1S2cahA==", 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm/oV3IRAAWn/zG1cLK+Raa2pYPzHd7Q==", 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:41:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAbXwdmf3HtOhAAyRcyb0YUJcJca5fjmZ3Frw==", 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-03-30T00:41:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAfXwdml7cdBRAAs639bBHTPyfhfVZMPRMfYA==", 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAiXwdmeS6KChAAZjLAbKSL7Vswi3qQ4304bA==", 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAlXwdmeuigEBAAiPIPheowKBnn3UZWwpdr+Q==", 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-03-30T00:41:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQApXwdm+bQNHRAAAb/lufdb7Fe+EFevd23gtA==", 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAsXwdmReL1IhAAwCs1cS+P6J8VCOYdrnrPQg==", 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAvXwdmA15vKBAAmpnKrDM/Y0xuBGxdyjrkWA==", 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAyXwdmrl02LhAA6JfEnugoz6l4nsXl6/vq0Q==", 2024-03-30T00:41:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmZjh7JRAA3EFOImM03iOljcwmA7/xWw==", 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:41:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmkZv0LhAAgwCqW25+J1VYO31mWwDtLQ==", 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-03-30T00:41:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmLG1/KxAAmNIZ9ZXdqPRQjCXvz7wC9A==", 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmyJ/nNBAAIgiUoUCbTeLEjL1iaPFw1A==", 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:41:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAXXwdmwWxQLRAAoFyZnFsIo1NvQlzRN+UHmw==", 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-mds", 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA9XwdmtgSeKBAA3GXmvL5YeyXMfO1kgCvbOw==", 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-mds" 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-mgr", 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA9XwdmrQ6eKBAA2S4wFuEcfhJOcDUsE+pn0Q==", 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-osd", 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA9XwdmsBeeKBAA9kRA4cnLNkUyKA7tRq1ZEQ==", 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-osd" 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rbd", 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA9XwdmlSCeKBAAiZTIYicaP2/R4ZFCXXVo3w==", 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-30T00:41:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA9XwdmlimeKBAAEwxp6AJ7HSbuyypNdTdrcA==", 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rgw", 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQA9XwdmcjKeKBAAcRX6FpiffPGJN8aw3XWRnw==", 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-30T00:41:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmEF+xCRAAqRVFguki881+4azP8lsf0g==", 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMg32BRAAUWBF+QbwRGsopcAoBay8wA==", 2024-03-30T00:41:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:41:15.492 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:41:15.492 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:41:15.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:41:15.492 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.492 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:15.492 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-03-30T00:41:15.492 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:41:15.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 -- 172.21.15.66:0/250177863 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfdc0666c0 msgr2=0x7fdfdc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfdc0666c0 0x7fdfdc068b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8000f70 tx=0x7fdfe8007df0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 -- 172.21.15.66:0/250177863 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe000070b50 msgr2=0x7fe000156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe000070b50 0x7fe000156910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdff001a010 tx=0x7fdff001ac20 comp rx=0 tx=0).stop 2024-03-30T00:41:15.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 -- 172.21.15.66:0/250177863 shutdown_connections 2024-03-30T00:41:15.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe00015db90 0x7fe000159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfdc0666c0 0x7fdfdc068b80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0000ce6b0 0x7fe00014ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 --2- 172.21.15.66:0/250177863 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe000070b50 0x7fe000156910 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 -- 172.21.15.66:0/250177863 >> 172.21.15.66:0/250177863 conn(0x7fe000066100 msgr2=0x7fe0000cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:15.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 -- 172.21.15.66:0/250177863 shutdown_connections 2024-03-30T00:41:15.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.469+0000 7fe005200640 1 -- 172.21.15.66:0/250177863 wait complete. 2024-03-30T00:41:15.497 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-30T00:41:15.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd68e00640 1 Processor -- start 2024-03-30T00:41:15.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd68e00640 1 -- start start 2024-03-30T00:41:15.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd68e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efd640ce520 0x7efd640ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd68e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640cabb0 0x7efd640cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd68e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6406d070 con 0x7efd6415a640 2024-03-30T00:41:15.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd68e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd640cbba0 con 0x7efd640ce520 2024-03-30T00:41:15.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd68e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd640cee70 con 0x7efd640cabb0 2024-03-30T00:41:15.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd61e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efd6415a640 0x7efd6406dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48900/0 (socket says 172.21.15.66:48900) 2024-03-30T00:41:15.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd61e00640 1 -- 172.21.15.66:0/3987226718 learned_addr learned my addr 172.21.15.66:0/3987226718 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:15.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd62800640 1 --2- 172.21.15.66:0/3987226718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640cabb0 0x7efd640cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd5be00640 1 --2- 172.21.15.66:0/3987226718 >> v2:172.21.15.156:3300/0 conn(0x7efd640ce520 0x7efd640ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd61400640 1 -- 172.21.15.66:0/3987226718 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686826358 0 0) 0x7efd6406d070 con 0x7efd6415a640 2024-03-30T00:41:15.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd61400640 1 -- 172.21.15.66:0/3987226718 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd48003530 con 0x7efd6415a640 2024-03-30T00:41:15.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd61400640 1 -- 172.21.15.66:0/3987226718 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2459508652 0 0) 0x7efd48003530 con 0x7efd6415a640 2024-03-30T00:41:15.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd62800640 1 -- 172.21.15.66:0/3987226718 >> v2:172.21.15.156:3300/0 conn(0x7efd640ce520 msgr2=0x7efd640ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:15.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd62800640 1 --2- 172.21.15.66:0/3987226718 >> v2:172.21.15.156:3300/0 conn(0x7efd640ce520 0x7efd640ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd62800640 1 -- 172.21.15.66:0/3987226718 >> v1:172.21.15.66:6789/0 conn(0x7efd6415a640 legacy=0x7efd6406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd62800640 1 -- 172.21.15.66:0/3987226718 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd640cb8c0 con 0x7efd640cabb0 2024-03-30T00:41:15.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.629+0000 7efd5be00640 1 --2- 172.21.15.66:0/3987226718 >> v2:172.21.15.156:3300/0 conn(0x7efd640ce520 0x7efd640ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:15.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd62800640 1 --2- 172.21.15.66:0/3987226718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640cabb0 0x7efd640cafe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009d90 tx=0x7efd5c01c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1a6c13f0f1ad2cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:15.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd61400640 1 -- 172.21.15.66:0/3987226718 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54001680 con 0x7efd640cabb0 2024-03-30T00:41:15.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd61400640 1 -- 172.21.15.66:0/3987226718 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c0a5660 con 0x7efd640cabb0 2024-03-30T00:41:15.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd61400640 1 -- 172.21.15.66:0/3987226718 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0a59f0 con 0x7efd640cabb0 2024-03-30T00:41:15.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd68e00640 1 -- 172.21.15.66:0/3987226718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640cabb0 msgr2=0x7efd640cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd68e00640 1 --2- 172.21.15.66:0/3987226718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640cabb0 0x7efd640cafe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009d90 tx=0x7efd5c01c860 comp rx=0 tx=0).stop 2024-03-30T00:41:15.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd68e00640 1 -- 172.21.15.66:0/3987226718 shutdown_connections 2024-03-30T00:41:15.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd68e00640 1 --2- 172.21.15.66:0/3987226718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640cabb0 0x7efd640cafe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd68e00640 1 --2- 172.21.15.66:0/3987226718 >> v2:172.21.15.156:3300/0 conn(0x7efd640ce520 0x7efd640ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd68e00640 1 -- 172.21.15.66:0/3987226718 >> 172.21.15.66:0/3987226718 conn(0x7efd64069f60 msgr2=0x7efd6406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:15.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd68e00640 1 -- 172.21.15.66:0/3987226718 shutdown_connections 2024-03-30T00:41:15.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.633+0000 7efd68e00640 1 -- 172.21.15.66:0/3987226718 wait complete. 2024-03-30T00:41:15.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd68e00640 1 Processor -- start 2024-03-30T00:41:15.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd68e00640 1 -- start start 2024-03-30T00:41:15.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd68e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd640cabb0 0x7efd640aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd68e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640ce520 0x7efd640c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd68e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd6415a640 0x7efd640c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd68e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd640cd980 con 0x7efd640cabb0 2024-03-30T00:41:15.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd68e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd640cd0e0 con 0x7efd6415a640 2024-03-30T00:41:15.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd68e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd640cd3c0 con 0x7efd640ce520 2024-03-30T00:41:15.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd61e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd640cabb0 0x7efd640aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd61e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd640cabb0 0x7efd640aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39708/0 (socket says 172.21.15.66:39708) 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd61e00640 1 -- 172.21.15.66:0/2062818036 learned_addr learned my addr 172.21.15.66:0/2062818036 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd62800640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd6415a640 0x7efd640c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd5be00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640ce520 0x7efd640c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd61e00640 1 -- 172.21.15.66:0/2062818036 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640ce520 msgr2=0x7efd640c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd61e00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640ce520 0x7efd640c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd61e00640 1 -- 172.21.15.66:0/2062818036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd6415a640 msgr2=0x7efd640c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd61e00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd6415a640 0x7efd640c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd61e00640 1 -- 172.21.15.66:0/2062818036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c009420 con 0x7efd640cabb0 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd5be00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640ce520 0x7efd640c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd61e00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd640cabb0 0x7efd640aefc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efd5400c750 tx=0x7efd5400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd5a000640 1 -- 172.21.15.66:0/2062818036 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54005ae0 con 0x7efd640cabb0 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd5a000640 1 -- 172.21.15.66:0/2062818036 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd54005c40 con 0x7efd640cabb0 2024-03-30T00:41:15.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd5a000640 1 -- 172.21.15.66:0/2062818036 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54007070 con 0x7efd640cabb0 2024-03-30T00:41:15.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd68e00640 1 -- 172.21.15.66:0/2062818036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5c009a70 con 0x7efd640cabb0 2024-03-30T00:41:15.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd68e00640 1 -- 172.21.15.66:0/2062818036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6415c490 con 0x7efd640cabb0 2024-03-30T00:41:15.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.637+0000 7efd68e00640 1 -- 172.21.15.66:0/2062818036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd640c52d0 con 0x7efd640cabb0 2024-03-30T00:41:15.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.641+0000 7efd5a000640 1 -- 172.21.15.66:0/2062818036 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efd540071f0 con 0x7efd640cabb0 2024-03-30T00:41:15.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.641+0000 7efd5a000640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efd40066690 0x7efd40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:15.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.641+0000 7efd5a000640 1 -- 172.21.15.66:0/2062818036 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7efd54083d90 con 0x7efd640cabb0 2024-03-30T00:41:15.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.641+0000 7efd5be00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efd40066690 0x7efd40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:15.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.641+0000 7efd5be00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efd40066690 0x7efd40068b50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd640c84c0 tx=0x7efd4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:15.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.645+0000 7efd5a000640 1 -- 172.21.15.66:0/2062818036 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efd54051300 con 0x7efd640cabb0 2024-03-30T00:41:15.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.861+0000 7efd68e00640 1 -- 172.21.15.66:0/2062818036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7efd64066c30 con 0x7efd640cabb0 2024-03-30T00:41:15.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.865+0000 7efd5a000640 1 -- 172.21.15.66:0/2062818036 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7efd54054920 con 0x7efd640cabb0 2024-03-30T00:41:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 -- 172.21.15.66:0/2062818036 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efd40066690 msgr2=0x7efd40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efd40066690 0x7efd40068b50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd640c84c0 tx=0x7efd4c008040 comp rx=0 tx=0).stop 2024-03-30T00:41:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 -- 172.21.15.66:0/2062818036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd640cabb0 msgr2=0x7efd640aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd640cabb0 0x7efd640aefc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efd5400c750 tx=0x7efd5400cc10 comp rx=0 tx=0).stop 2024-03-30T00:41:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 -- 172.21.15.66:0/2062818036 shutdown_connections 2024-03-30T00:41:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd6415a640 0x7efd640c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd640ce520 0x7efd640c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efd40066690 0x7efd40068b50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 --2- 172.21.15.66:0/2062818036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd640cabb0 0x7efd640aefc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 -- 172.21.15.66:0/2062818036 >> 172.21.15.66:0/2062818036 conn(0x7efd64069f60 msgr2=0x7efd640ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 -- 172.21.15.66:0/2062818036 shutdown_connections 2024-03-30T00:41:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:15.869+0000 7efd68e00640 1 -- 172.21.15.66:0/2062818036 wait complete. 2024-03-30T00:41:15.897 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-30T00:41:16.028 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:41:16.031 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:41:16.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe5de00640 1 Processor -- start 2024-03-30T00:41:16.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe5de00640 1 -- start start 2024-03-30T00:41:16.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe5de00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbe580af420 0x7fbe580cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe5de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe580cbb90 con 0x7fbe580cee50 2024-03-30T00:41:16.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe5de00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe580ce1b0 con 0x7fbe5815de30 2024-03-30T00:41:16.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe5de00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe580afa10 con 0x7fbe580af420 2024-03-30T00:41:16.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe56000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fbe580cee50 0x7fbe5806e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48912/0 (socket says 172.21.15.66:48912) 2024-03-30T00:41:16.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe56000640 1 -- 172.21.15.66:0/3692632364 learned_addr learned my addr 172.21.15.66:0/3692632364 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:16.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe57400640 1 --2- 172.21.15.66:0/3692632364 >> v2:172.21.15.156:3300/0 conn(0x7fbe580af420 0x7fbe580cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe55600640 1 -- 172.21.15.66:0/3692632364 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167980251 0 0) 0x7fbe580cbb90 con 0x7fbe580cee50 2024-03-30T00:41:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe55600640 1 -- 172.21.15.66:0/3692632364 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe3c003530 con 0x7fbe580cee50 2024-03-30T00:41:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe55600640 1 -- 172.21.15.66:0/3692632364 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317181939 0 0) 0x7fbe580ce1b0 con 0x7fbe5815de30 2024-03-30T00:41:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe55600640 1 -- 172.21.15.66:0/3692632364 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe580cbb90 con 0x7fbe5815de30 2024-03-30T00:41:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe57400640 1 -- 172.21.15.66:0/3692632364 >> v1:172.21.15.156:6789/0 conn(0x7fbe5815de30 legacy=0x7fbe580ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe57400640 1 -- 172.21.15.66:0/3692632364 >> v1:172.21.15.66:6789/0 conn(0x7fbe580cee50 legacy=0x7fbe5806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe55600640 1 -- 172.21.15.66:0/3692632364 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2031434425 0 0) 0x7fbe3c003530 con 0x7fbe580cee50 2024-03-30T00:41:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.029+0000 7fbe57400640 1 -- 172.21.15.66:0/3692632364 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580cf430 con 0x7fbe580af420 2024-03-30T00:41:16.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.033+0000 7fbe57400640 1 --2- 172.21.15.66:0/3692632364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe580af420 0x7fbe580cc040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009d90 tx=0x7fbe5001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e304eaa3a38138c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:16.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.033+0000 7fbe55600640 1 -- 172.21.15.66:0/3692632364 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe44001680 con 0x7fbe580af420 2024-03-30T00:41:16.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.033+0000 7fbe55600640 1 -- 172.21.15.66:0/3692632364 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe500a55e0 con 0x7fbe580af420 2024-03-30T00:41:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.033+0000 7fbe55600640 1 -- 172.21.15.66:0/3692632364 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe500a5970 con 0x7fbe580af420 2024-03-30T00:41:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.033+0000 7fbe5de00640 1 -- 172.21.15.66:0/3692632364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe580af420 msgr2=0x7fbe580cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.033+0000 7fbe5de00640 1 --2- 172.21.15.66:0/3692632364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe580af420 0x7fbe580cc040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009d90 tx=0x7fbe5001c890 comp rx=0 tx=0).stop 2024-03-30T00:41:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.033+0000 7fbe5de00640 1 -- 172.21.15.66:0/3692632364 shutdown_connections 2024-03-30T00:41:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.033+0000 7fbe5de00640 1 --2- 172.21.15.66:0/3692632364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe580af420 0x7fbe580cc040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.033+0000 7fbe5de00640 1 -- 172.21.15.66:0/3692632364 >> 172.21.15.66:0/3692632364 conn(0x7fbe580d0540 msgr2=0x7fbe580d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.033+0000 7fbe5de00640 1 -- 172.21.15.66:0/3692632364 shutdown_connections 2024-03-30T00:41:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.033+0000 7fbe5de00640 1 -- 172.21.15.66:0/3692632364 wait complete. 2024-03-30T00:41:16.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe5de00640 1 Processor -- start 2024-03-30T00:41:16.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe5de00640 1 -- start start 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe5de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe580af420 0x7fbe580affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe5de00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe580cee50 0x7fbe58155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe5de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5815de30 0x7fbe5815c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe5de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580c8d60 con 0x7fbe5815de30 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe5de00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580b2e40 con 0x7fbe580cee50 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe5de00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580b2880 con 0x7fbe580af420 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe57400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5815de30 0x7fbe5815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe57400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5815de30 0x7fbe5815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39710/0 (socket says 172.21.15.66:39710) 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe57400640 1 -- 172.21.15.66:0/1892386760 learned_addr learned my addr 172.21.15.66:0/1892386760 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe56a00640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe580af420 0x7fbe580affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe56000640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe580cee50 0x7fbe58155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe57400640 1 -- 172.21.15.66:0/1892386760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe580af420 msgr2=0x7fbe580affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe57400640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe580af420 0x7fbe580affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe57400640 1 -- 172.21.15.66:0/1892386760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe580cee50 msgr2=0x7fbe58155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe57400640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe580cee50 0x7fbe58155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe57400640 1 -- 172.21.15.66:0/1892386760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe50009a70 con 0x7fbe5815de30 2024-03-30T00:41:16.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe57400640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5815de30 0x7fbe5815c440 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbe500b0040 tx=0x7fbe50019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:16.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe4f400640 1 -- 172.21.15.66:0/1892386760 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50003ba0 con 0x7fbe5815de30 2024-03-30T00:41:16.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe4f400640 1 -- 172.21.15.66:0/1892386760 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe50003d00 con 0x7fbe5815de30 2024-03-30T00:41:16.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe5de00640 1 -- 172.21.15.66:0/1892386760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe580b2b60 con 0x7fbe5815de30 2024-03-30T00:41:16.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe4f400640 1 -- 172.21.15.66:0/1892386760 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50004260 con 0x7fbe5815de30 2024-03-30T00:41:16.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe5de00640 1 -- 172.21.15.66:0/1892386760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe580c9270 con 0x7fbe5815de30 2024-03-30T00:41:16.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.037+0000 7fbe5de00640 1 -- 172.21.15.66:0/1892386760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe580cdf60 con 0x7fbe5815de30 2024-03-30T00:41:16.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.041+0000 7fbe4f400640 1 -- 172.21.15.66:0/1892386760 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbe500043c0 con 0x7fbe5815de30 2024-03-30T00:41:16.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.041+0000 7fbe4f400640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe38066690 0x7fbe38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.041+0000 7fbe4f400640 1 -- 172.21.15.66:0/1892386760 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fbe5011e550 con 0x7fbe5815de30 2024-03-30T00:41:16.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.041+0000 7fbe56a00640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe38066690 0x7fbe38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.041+0000 7fbe56a00640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe38066690 0x7fbe38068b50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbe480011f0 tx=0x7fbe48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:16.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.045+0000 7fbe4f400640 1 -- 172.21.15.66:0/1892386760 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbe500ebae0 con 0x7fbe5815de30 2024-03-30T00:41:16.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.261+0000 7fbe5de00640 1 -- 172.21.15.66:0/1892386760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fbe58066c30 con 0x7fbe5815de30 2024-03-30T00:41:16.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.265+0000 7fbe4f400640 1 -- 172.21.15.66:0/1892386760 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbe50009450 con 0x7fbe5815de30 2024-03-30T00:41:16.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 -- 172.21.15.66:0/1892386760 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe38066690 msgr2=0x7fbe38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe38066690 0x7fbe38068b50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbe480011f0 tx=0x7fbe48009040 comp rx=0 tx=0).stop 2024-03-30T00:41:16.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 -- 172.21.15.66:0/1892386760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5815de30 msgr2=0x7fbe5815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5815de30 0x7fbe5815c440 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbe500b0040 tx=0x7fbe50019630 comp rx=0 tx=0).stop 2024-03-30T00:41:16.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 -- 172.21.15.66:0/1892386760 shutdown_connections 2024-03-30T00:41:16.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbe38066690 0x7fbe38068b50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe5815de30 0x7fbe5815c440 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe580cee50 0x7fbe58155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 --2- 172.21.15.66:0/1892386760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbe580af420 0x7fbe580affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 -- 172.21.15.66:0/1892386760 >> 172.21.15.66:0/1892386760 conn(0x7fbe580d0540 msgr2=0x7fbe580d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:16.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 -- 172.21.15.66:0/1892386760 shutdown_connections 2024-03-30T00:41:16.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.269+0000 7fbe5de00640 1 -- 172.21.15.66:0/1892386760 wait complete. 2024-03-30T00:41:16.298 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-30T00:41:16.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57f6200640 1 Processor -- start 2024-03-30T00:41:16.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57f6200640 1 -- start start 2024-03-30T00:41:16.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57f6200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f57f00cec30 0x7f57f0159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57f6200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f00cd3e0 con 0x7f57f00cda90 2024-03-30T00:41:16.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57f6200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f015a990 con 0x7f57f00cde70 2024-03-30T00:41:16.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57f6200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f00b1b80 con 0x7f57f00cec30 2024-03-30T00:41:16.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57e6a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f57f00cec30 0x7f57f0159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57e6a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f57f00cec30 0x7f57f0159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39714/0 (socket says 172.21.15.66:39714) 2024-03-30T00:41:16.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57ef400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f57f00cda90 0x7f57f00ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48916/0 (socket says 172.21.15.66:48916) 2024-03-30T00:41:16.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57e6a00640 1 -- 172.21.15.66:0/573026837 learned_addr learned my addr 172.21.15.66:0/573026837 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:16.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57e6a00640 1 -- 172.21.15.66:0/573026837 >> v1:172.21.15.156:6789/0 conn(0x7f57f00cde70 legacy=0x7f57f015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:16.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57e6a00640 1 -- 172.21.15.66:0/573026837 >> v1:172.21.15.66:6789/0 conn(0x7f57f00cda90 legacy=0x7f57f00ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57e6a00640 1 -- 172.21.15.66:0/573026837 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f00b11e0 con 0x7f57f00cec30 2024-03-30T00:41:16.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.429+0000 7f57eea00640 1 -- 172.21.15.66:0/573026837 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235185870 0 0) 0x7f57f00cd3e0 con 0x7f57f00cda90 2024-03-30T00:41:16.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.433+0000 7f57e6a00640 1 --2- 172.21.15.66:0/573026837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f00cec30 0x7f57f0159c60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009870 tx=0x7f57d801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85c1d33ed94d754 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:16.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.433+0000 7f57eea00640 1 -- 172.21.15.66:0/573026837 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d80aa5f0 con 0x7f57f00cec30 2024-03-30T00:41:16.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.433+0000 7f57eea00640 1 -- 172.21.15.66:0/573026837 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d80aa750 con 0x7f57f00cec30 2024-03-30T00:41:16.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.433+0000 7f57eea00640 1 -- 172.21.15.66:0/573026837 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d80aaae0 con 0x7f57f00cec30 2024-03-30T00:41:16.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.433+0000 7f57f6200640 1 -- 172.21.15.66:0/573026837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f00cec30 msgr2=0x7f57f0159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.433+0000 7f57f6200640 1 --2- 172.21.15.66:0/573026837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f00cec30 0x7f57f0159c60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009870 tx=0x7f57d801c490 comp rx=0 tx=0).stop 2024-03-30T00:41:16.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.433+0000 7f57f6200640 1 -- 172.21.15.66:0/573026837 shutdown_connections 2024-03-30T00:41:16.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.433+0000 7f57f6200640 1 --2- 172.21.15.66:0/573026837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f00cec30 0x7f57f0159c60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.433+0000 7f57f6200640 1 -- 172.21.15.66:0/573026837 >> 172.21.15.66:0/573026837 conn(0x7f57f0069f60 msgr2=0x7f57f006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:16.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.433+0000 7f57f6200640 1 -- 172.21.15.66:0/573026837 shutdown_connections 2024-03-30T00:41:16.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.433+0000 7f57f6200640 1 -- 172.21.15.66:0/573026837 wait complete. 2024-03-30T00:41:16.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57f6200640 1 Processor -- start 2024-03-30T00:41:16.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57f6200640 1 -- start start 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57f6200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57f00cda90 0x7f57f0097520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57f6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f00cde70 0x7f57f0156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57f6200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57f00cec30 0x7f57f0095880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57f6200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f0095dc0 con 0x7f57f00cde70 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57f6200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f0096080 con 0x7f57f00cda90 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57f6200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f0096360 con 0x7f57f00cec30 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f00cde70 0x7f57f0156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f00cde70 0x7f57f0156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39718/0 (socket says 172.21.15.66:39718) 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e6a00640 1 -- 172.21.15.66:0/2238065521 learned_addr learned my addr 172.21.15.66:0/2238065521 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57efe00640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57f00cec30 0x7f57f0095880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57ef400640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57f00cda90 0x7f57f0097520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e6a00640 1 -- 172.21.15.66:0/2238065521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57f00cec30 msgr2=0x7f57f0095880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e6a00640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57f00cec30 0x7f57f0095880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e6a00640 1 -- 172.21.15.66:0/2238065521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57f00cda90 msgr2=0x7f57f0097520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e6a00640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57f00cda90 0x7f57f0097520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e6a00640 1 -- 172.21.15.66:0/2238065521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d80094d0 con 0x7f57f00cde70 2024-03-30T00:41:16.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57efe00640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57f00cec30 0x7f57f0095880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:16.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57ef400640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57f00cda90 0x7f57f0097520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:16.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e6a00640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f00cde70 0x7f57f0156350 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f57d80ac040 tx=0x7f57d801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:16.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e7e00640 1 -- 172.21.15.66:0/2238065521 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d80b3800 con 0x7f57f00cde70 2024-03-30T00:41:16.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e7e00640 1 -- 172.21.15.66:0/2238065521 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d80b3960 con 0x7f57f00cde70 2024-03-30T00:41:16.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57e7e00640 1 -- 172.21.15.66:0/2238065521 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d80b3d10 con 0x7f57f00cde70 2024-03-30T00:41:16.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57f6200640 1 -- 172.21.15.66:0/2238065521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f0096640 con 0x7f57f00cde70 2024-03-30T00:41:16.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.437+0000 7f57f6200640 1 -- 172.21.15.66:0/2238065521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f0096c00 con 0x7f57f00cde70 2024-03-30T00:41:16.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.441+0000 7f57f6200640 1 -- 172.21.15.66:0/2238065521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f0096df0 con 0x7f57f00cde70 2024-03-30T00:41:16.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.441+0000 7f57e7e00640 1 -- 172.21.15.66:0/2238065521 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f57d80b2380 con 0x7f57f00cde70 2024-03-30T00:41:16.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.441+0000 7f57e7e00640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f57cc066690 0x7f57cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.441+0000 7f57e7e00640 1 -- 172.21.15.66:0/2238065521 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f57d80a7070 con 0x7f57f00cde70 2024-03-30T00:41:16.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.441+0000 7f57ef400640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f57cc066690 0x7f57cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.441+0000 7f57ef400640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f57cc066690 0x7f57cc068b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f57e0007fd0 tx=0x7f57e0005b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:16.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.445+0000 7f57e7e00640 1 -- 172.21.15.66:0/2238065521 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f57d80ebcb0 con 0x7f57f00cde70 2024-03-30T00:41:16.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.665+0000 7f57f6200640 1 -- 172.21.15.66:0/2238065521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f57f0066c30 con 0x7f57f00cde70 2024-03-30T00:41:16.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.669+0000 7f57e7e00640 1 -- 172.21.15.66:0/2238065521 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f57d80ef2d0 con 0x7f57f00cde70 2024-03-30T00:41:16.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 -- 172.21.15.66:0/2238065521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f57cc066690 msgr2=0x7f57cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f57cc066690 0x7f57cc068b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f57e0007fd0 tx=0x7f57e0005b80 comp rx=0 tx=0).stop 2024-03-30T00:41:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 -- 172.21.15.66:0/2238065521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f00cde70 msgr2=0x7f57f0156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f00cde70 0x7f57f0156350 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f57d80ac040 tx=0x7f57d801acd0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 -- 172.21.15.66:0/2238065521 shutdown_connections 2024-03-30T00:41:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57f00cec30 0x7f57f0095880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f57cc066690 0x7f57cc068b50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57f00cde70 0x7f57f0156350 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 --2- 172.21.15.66:0/2238065521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57f00cda90 0x7f57f0097520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 -- 172.21.15.66:0/2238065521 >> 172.21.15.66:0/2238065521 conn(0x7f57f0069f60 msgr2=0x7f57f015d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 -- 172.21.15.66:0/2238065521 shutdown_connections 2024-03-30T00:41:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.673+0000 7f57f6200640 1 -- 172.21.15.66:0/2238065521 wait complete. 2024-03-30T00:41:16.700 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-30T00:41:16.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d6600640 1 Processor -- start 2024-03-30T00:41:16.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d6600640 1 -- start start 2024-03-30T00:41:16.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d6600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f52d015c6b0 0x7f52d00d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d6600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d015d250 con 0x7f52d00cdef0 2024-03-30T00:41:16.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d6600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d015d530 con 0x7f52d006cb90 2024-03-30T00:41:16.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d6600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d006b1f0 con 0x7f52d015c6b0 2024-03-30T00:41:16.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d5c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f52d00cdef0 0x7f52d01604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48928/0 (socket says 172.21.15.66:48928) 2024-03-30T00:41:16.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d5c00640 1 -- 172.21.15.66:0/2459392847 learned_addr learned my addr 172.21.15.66:0/2459392847 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:16.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d5200640 1 --2- 172.21.15.66:0/2459392847 >> v2:172.21.15.156:3300/0 conn(0x7f52d015c6b0 0x7f52d00d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52cfe00640 1 -- 172.21.15.66:0/2459392847 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903344509 0 0) 0x7f52d015d250 con 0x7f52d00cdef0 2024-03-30T00:41:16.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52cfe00640 1 -- 172.21.15.66:0/2459392847 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b4003530 con 0x7f52d00cdef0 2024-03-30T00:41:16.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d5200640 1 -- 172.21.15.66:0/2459392847 >> v1:172.21.15.156:6789/0 conn(0x7f52d006cb90 legacy=0x7f52d015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d5200640 1 -- 172.21.15.66:0/2459392847 >> v1:172.21.15.66:6789/0 conn(0x7f52d00cdef0 legacy=0x7f52d01604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.833+0000 7f52d5200640 1 -- 172.21.15.66:0/2459392847 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d015d810 con 0x7f52d015c6b0 2024-03-30T00:41:16.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.837+0000 7f52d5200640 1 --2- 172.21.15.66:0/2459392847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52d015c6b0 0x7f52d00d0c50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52c00097c0 tx=0x7f52c001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27263221dcb24da3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:16.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.837+0000 7f52cfe00640 1 -- 172.21.15.66:0/2459392847 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c00ab600 con 0x7f52d015c6b0 2024-03-30T00:41:16.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.837+0000 7f52cfe00640 1 -- 172.21.15.66:0/2459392847 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c00a7050 con 0x7f52d015c6b0 2024-03-30T00:41:16.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.837+0000 7f52cfe00640 1 -- 172.21.15.66:0/2459392847 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c00ab980 con 0x7f52d015c6b0 2024-03-30T00:41:16.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.837+0000 7f52d6600640 1 -- 172.21.15.66:0/2459392847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52d015c6b0 msgr2=0x7f52d00d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.837+0000 7f52d6600640 1 --2- 172.21.15.66:0/2459392847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52d015c6b0 0x7f52d00d0c50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52c00097c0 tx=0x7f52c001c3e0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.837+0000 7f52d6600640 1 -- 172.21.15.66:0/2459392847 shutdown_connections 2024-03-30T00:41:16.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.837+0000 7f52d6600640 1 --2- 172.21.15.66:0/2459392847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52d015c6b0 0x7f52d00d0c50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.837+0000 7f52d6600640 1 -- 172.21.15.66:0/2459392847 >> 172.21.15.66:0/2459392847 conn(0x7f52d0069d90 msgr2=0x7f52d006a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:16.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.837+0000 7f52d6600640 1 -- 172.21.15.66:0/2459392847 shutdown_connections 2024-03-30T00:41:16.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.837+0000 7f52d6600640 1 -- 172.21.15.66:0/2459392847 wait complete. 2024-03-30T00:41:16.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52d6600640 1 Processor -- start 2024-03-30T00:41:16.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52d6600640 1 -- start start 2024-03-30T00:41:16.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52d6600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52d006cb90 0x7f52d0153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52d6600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d00cdef0 0x7f52d0154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52d6600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52d015c6b0 0x7f52d0154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52d6600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d00b3d20 con 0x7f52d00cdef0 2024-03-30T00:41:16.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52d6600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d00b3f70 con 0x7f52d015c6b0 2024-03-30T00:41:16.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52d6600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d00b4250 con 0x7f52d006cb90 2024-03-30T00:41:16.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52c7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d00cdef0 0x7f52d0154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52c7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d00cdef0 0x7f52d0154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39732/0 (socket says 172.21.15.66:39732) 2024-03-30T00:41:16.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52c7e00640 1 -- 172.21.15.66:0/4258718952 learned_addr learned my addr 172.21.15.66:0/4258718952 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:16.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52d5200640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52d006cb90 0x7f52d0153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52d5c00640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52d015c6b0 0x7f52d0154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52c7e00640 1 -- 172.21.15.66:0/4258718952 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52d006cb90 msgr2=0x7f52d0153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52c7e00640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52d006cb90 0x7f52d0153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52c7e00640 1 -- 172.21.15.66:0/4258718952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52d015c6b0 msgr2=0x7f52d0154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:16.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52c7e00640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52d015c6b0 0x7f52d0154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:16.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52c7e00640 1 -- 172.21.15.66:0/4258718952 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c0009420 con 0x7f52d00cdef0 2024-03-30T00:41:16.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.841+0000 7f52c7e00640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d00cdef0 0x7f52d0154590 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f52b800c500 tx=0x7f52b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:16.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.845+0000 7f52cd600640 1 -- 172.21.15.66:0/4258718952 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8005220 con 0x7f52d00cdef0 2024-03-30T00:41:16.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.845+0000 7f52d6600640 1 -- 172.21.15.66:0/4258718952 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d015e290 con 0x7f52d00cdef0 2024-03-30T00:41:16.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.845+0000 7f52cd600640 1 -- 172.21.15.66:0/4258718952 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b8005380 con 0x7f52d00cdef0 2024-03-30T00:41:16.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.845+0000 7f52cd600640 1 -- 172.21.15.66:0/4258718952 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8007270 con 0x7f52d00cdef0 2024-03-30T00:41:16.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.845+0000 7f52d6600640 1 -- 172.21.15.66:0/4258718952 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d00b4810 con 0x7f52d00cdef0 2024-03-30T00:41:16.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.845+0000 7f52d6600640 1 -- 172.21.15.66:0/4258718952 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d0073e00 con 0x7f52d00cdef0 2024-03-30T00:41:16.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.845+0000 7f52cd600640 1 -- 172.21.15.66:0/4258718952 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f52b80073d0 con 0x7f52d00cdef0 2024-03-30T00:41:16.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.849+0000 7f52cd600640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52ac066690 0x7f52ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:16.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.849+0000 7f52cd600640 1 -- 172.21.15.66:0/4258718952 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f52b80838f0 con 0x7f52d00cdef0 2024-03-30T00:41:16.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.849+0000 7f52d5200640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52ac066690 0x7f52ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:16.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.849+0000 7f52d5200640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52ac066690 0x7f52ac068b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f52c00ad000 tx=0x7f52c00a4f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:16.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:16.853+0000 7f52cd600640 1 -- 172.21.15.66:0/4258718952 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f52b8050de0 con 0x7f52d00cdef0 2024-03-30T00:41:17.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.069+0000 7f52d6600640 1 -- 172.21.15.66:0/4258718952 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f52d00b62a0 con 0x7f52d00cdef0 2024-03-30T00:41:17.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52cd600640 1 -- 172.21.15.66:0/4258718952 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f52b8054400 con 0x7f52d00cdef0 2024-03-30T00:41:17.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 -- 172.21.15.66:0/4258718952 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52ac066690 msgr2=0x7f52ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52ac066690 0x7f52ac068b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f52c00ad000 tx=0x7f52c00a4f90 comp rx=0 tx=0).stop 2024-03-30T00:41:17.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 -- 172.21.15.66:0/4258718952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d00cdef0 msgr2=0x7f52d0154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d00cdef0 0x7f52d0154590 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f52b800c500 tx=0x7f52b800c9c0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 -- 172.21.15.66:0/4258718952 shutdown_connections 2024-03-30T00:41:17.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52d015c6b0 0x7f52d0154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52ac066690 0x7f52ac068b50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52d00cdef0 0x7f52d0154590 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 --2- 172.21.15.66:0/4258718952 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52d006cb90 0x7f52d0153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 -- 172.21.15.66:0/4258718952 >> 172.21.15.66:0/4258718952 conn(0x7f52d0069d90 msgr2=0x7f52d00afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:17.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 -- 172.21.15.66:0/4258718952 shutdown_connections 2024-03-30T00:41:17.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.073+0000 7f52d6600640 1 -- 172.21.15.66:0/4258718952 wait complete. 2024-03-30T00:41:17.103 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-30T00:41:17.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.233+0000 7f69e4e00640 1 Processor -- start 2024-03-30T00:41:17.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.233+0000 7f69e4e00640 1 -- start start 2024-03-30T00:41:17.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69e4e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 0x7f69e00cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:17.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69e4e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f69e00ce820 0x7f69e00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:17.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69e4e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e006e7d0 con 0x7f69e015ca10 2024-03-30T00:41:17.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69e4e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e00b1f70 con 0x7f69e00ce820 2024-03-30T00:41:17.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69e4e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e00cf1f0 con 0x7f69e00ae4e0 2024-03-30T00:41:17.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69de800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f69e00ce820 0x7f69e00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:17.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69de800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f69e00ce820 0x7f69e00cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39742/0 (socket says 172.21.15.66:39742) 2024-03-30T00:41:17.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69de800640 1 -- 172.21.15.66:0/3008994494 learned_addr learned my addr 172.21.15.66:0/3008994494 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69d5400640 1 --2- 172.21.15.66:0/3008994494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 0x7f69e00cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69de800640 1 -- 172.21.15.66:0/3008994494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 msgr2=0x7f69e00cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69de800640 1 --2- 172.21.15.66:0/3008994494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 0x7f69e00cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69de800640 1 -- 172.21.15.66:0/3008994494 >> v1:172.21.15.66:6789/0 conn(0x7f69e015ca10 legacy=0x7f69e006ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69de800640 1 -- 172.21.15.66:0/3008994494 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e015cdf0 con 0x7f69e00ce820 2024-03-30T00:41:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69d5400640 1 --2- 172.21.15.66:0/3008994494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 0x7f69e00cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:17.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.237+0000 7f69dd400640 1 -- 172.21.15.66:0/3008994494 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743290914 0 0) 0x7f69e006e7d0 con 0x7f69e015ca10 2024-03-30T00:41:17.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69de800640 1 --2- 172.21.15.66:0/3008994494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69e00ce820 0x7f69e00cecb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f69d80098a0 tx=0x7f69d801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae665266c5db6f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:17.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69dd400640 1 -- 172.21.15.66:0/3008994494 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d80a7070 con 0x7f69e00ce820 2024-03-30T00:41:17.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69dd400640 1 -- 172.21.15.66:0/3008994494 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d80ab430 con 0x7f69e00ce820 2024-03-30T00:41:17.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69dd400640 1 -- 172.21.15.66:0/3008994494 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d80ab7c0 con 0x7f69e00ce820 2024-03-30T00:41:17.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 -- 172.21.15.66:0/3008994494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69e00ce820 msgr2=0x7f69e00cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 --2- 172.21.15.66:0/3008994494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69e00ce820 0x7f69e00cecb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f69d80098a0 tx=0x7f69d801c540 comp rx=0 tx=0).stop 2024-03-30T00:41:17.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 -- 172.21.15.66:0/3008994494 shutdown_connections 2024-03-30T00:41:17.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 --2- 172.21.15.66:0/3008994494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69e00ce820 0x7f69e00cecb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 --2- 172.21.15.66:0/3008994494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 0x7f69e00cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 -- 172.21.15.66:0/3008994494 >> 172.21.15.66:0/3008994494 conn(0x7f69e0098a20 msgr2=0x7f69e00b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:17.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 -- 172.21.15.66:0/3008994494 shutdown_connections 2024-03-30T00:41:17.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 -- 172.21.15.66:0/3008994494 wait complete. 2024-03-30T00:41:17.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 Processor -- start 2024-03-30T00:41:17.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 -- start start 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 0x7f69e00afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69e00ce820 0x7f69e0154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69e015ca10 0x7f69e0154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e00b4db0 con 0x7f69e00ce820 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e015e600 con 0x7f69e015ca10 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.241+0000 7f69e4e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e015e8e0 con 0x7f69e00ae4e0 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69e00ce820 0x7f69e0154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69e00ce820 0x7f69e0154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39756/0 (socket says 172.21.15.66:39756) 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d5400640 1 -- 172.21.15.66:0/1740089757 learned_addr learned my addr 172.21.15.66:0/1740089757 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69dde00640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 0x7f69e00afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69de800640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69e015ca10 0x7f69e0154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:17.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d5400640 1 -- 172.21.15.66:0/1740089757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 msgr2=0x7f69e00afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d5400640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 0x7f69e00afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d5400640 1 -- 172.21.15.66:0/1740089757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69e015ca10 msgr2=0x7f69e0154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d5400640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69e015ca10 0x7f69e0154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d5400640 1 -- 172.21.15.66:0/1740089757 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80094d0 con 0x7f69e00ce820 2024-03-30T00:41:17.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69dde00640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 0x7f69e00afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:17.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d5400640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69e00ce820 0x7f69e0154550 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f69c800c7c0 tx=0x7f69c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:17.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d6a00640 1 -- 172.21.15.66:0/1740089757 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8007a30 con 0x7f69e00ce820 2024-03-30T00:41:17.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d6a00640 1 -- 172.21.15.66:0/1740089757 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c8007b90 con 0x7f69e00ce820 2024-03-30T00:41:17.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d6a00640 1 -- 172.21.15.66:0/1740089757 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c800ce70 con 0x7f69e00ce820 2024-03-30T00:41:17.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69e4e00640 1 -- 172.21.15.66:0/1740089757 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e015ebc0 con 0x7f69e00ce820 2024-03-30T00:41:17.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69e4e00640 1 -- 172.21.15.66:0/1740089757 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e015f180 con 0x7f69e00ce820 2024-03-30T00:41:17.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69e4e00640 1 -- 172.21.15.66:0/1740089757 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e015f3a0 con 0x7f69e00ce820 2024-03-30T00:41:17.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d6a00640 1 -- 172.21.15.66:0/1740089757 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f69c80113f0 con 0x7f69e00ce820 2024-03-30T00:41:17.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.245+0000 7f69d6a00640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f69bc066690 0x7f69bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:17.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.249+0000 7f69d6a00640 1 -- 172.21.15.66:0/1740089757 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f69c80843d0 con 0x7f69e00ce820 2024-03-30T00:41:17.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.249+0000 7f69dde00640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f69bc066690 0x7f69bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:17.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.249+0000 7f69dde00640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f69bc066690 0x7f69bc068b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f69d00012f0 tx=0x7f69d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:17.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.249+0000 7f69d6a00640 1 -- 172.21.15.66:0/1740089757 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f69c8051a00 con 0x7f69e00ce820 2024-03-30T00:41:17.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.465+0000 7f69e4e00640 1 -- 172.21.15.66:0/1740089757 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f69e0066c30 con 0x7f69e00ce820 2024-03-30T00:41:17.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69d6a00640 1 -- 172.21.15.66:0/1740089757 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f69c8055020 con 0x7f69e00ce820 2024-03-30T00:41:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 -- 172.21.15.66:0/1740089757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f69bc066690 msgr2=0x7f69bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f69bc066690 0x7f69bc068b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f69d00012f0 tx=0x7f69d0009040 comp rx=0 tx=0).stop 2024-03-30T00:41:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 -- 172.21.15.66:0/1740089757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69e00ce820 msgr2=0x7f69e0154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69e00ce820 0x7f69e0154550 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f69c800c7c0 tx=0x7f69c800cc80 comp rx=0 tx=0).stop 2024-03-30T00:41:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 -- 172.21.15.66:0/1740089757 shutdown_connections 2024-03-30T00:41:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69e015ca10 0x7f69e0154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f69bc066690 0x7f69bc068b50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69e00ce820 0x7f69e0154550 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 --2- 172.21.15.66:0/1740089757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e00ae4e0 0x7f69e00afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 -- 172.21.15.66:0/1740089757 >> 172.21.15.66:0/1740089757 conn(0x7f69e0098a20 msgr2=0x7f69e015bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 -- 172.21.15.66:0/1740089757 shutdown_connections 2024-03-30T00:41:17.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.473+0000 7f69e4e00640 1 -- 172.21.15.66:0/1740089757 wait complete. 2024-03-30T00:41:17.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-30T00:41:17.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a8200640 1 Processor -- start 2024-03-30T00:41:17.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a8200640 1 -- start start 2024-03-30T00:41:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a8200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f76a00cee50 0x7f76a00cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a8200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a00cbb90 con 0x7f76a015de30 2024-03-30T00:41:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a8200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a00ce1b0 con 0x7f76a00af420 2024-03-30T00:41:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a8200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a00afa10 con 0x7f76a00cee50 2024-03-30T00:41:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a4e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f76a00cee50 0x7f76a00cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a5800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f76a015de30 0x7f76a00ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48944/0 (socket says 172.21.15.66:48944) 2024-03-30T00:41:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a5800640 1 -- 172.21.15.66:0/1975359079 learned_addr learned my addr 172.21.15.66:0/1975359079 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f769fe00640 1 -- 172.21.15.66:0/1975359079 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067631464 0 0) 0x7f76a00cbb90 con 0x7f76a015de30 2024-03-30T00:41:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f769fe00640 1 -- 172.21.15.66:0/1975359079 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7688003530 con 0x7f76a015de30 2024-03-30T00:41:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a4e00640 1 -- 172.21.15.66:0/1975359079 >> v1:172.21.15.156:6789/0 conn(0x7f76a00af420 legacy=0x7f76a006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a4e00640 1 -- 172.21.15.66:0/1975359079 >> v1:172.21.15.66:6789/0 conn(0x7f76a015de30 legacy=0x7f76a00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.633+0000 7f76a4e00640 1 -- 172.21.15.66:0/1975359079 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00cf430 con 0x7f76a00cee50 2024-03-30T00:41:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.637+0000 7f76a4e00640 1 --2- 172.21.15.66:0/1975359079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76a00cee50 0x7f76a00cc040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f768c0097c0 tx=0x7f768c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ee3c4b1dbb7f55 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.637+0000 7f769fe00640 1 -- 172.21.15.66:0/1975359079 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0ac410 con 0x7f76a00cee50 2024-03-30T00:41:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.637+0000 7f769fe00640 1 -- 172.21.15.66:0/1975359079 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c0ac570 con 0x7f76a00cee50 2024-03-30T00:41:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.637+0000 7f769fe00640 1 -- 172.21.15.66:0/1975359079 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0ac900 con 0x7f76a00cee50 2024-03-30T00:41:17.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.637+0000 7f76a8200640 1 -- 172.21.15.66:0/1975359079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76a00cee50 msgr2=0x7f76a00cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.637+0000 7f76a8200640 1 --2- 172.21.15.66:0/1975359079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76a00cee50 0x7f76a00cc040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f768c0097c0 tx=0x7f768c01c4a0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.637+0000 7f76a8200640 1 -- 172.21.15.66:0/1975359079 shutdown_connections 2024-03-30T00:41:17.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.637+0000 7f76a8200640 1 --2- 172.21.15.66:0/1975359079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76a00cee50 0x7f76a00cc040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.637+0000 7f76a8200640 1 -- 172.21.15.66:0/1975359079 >> 172.21.15.66:0/1975359079 conn(0x7f76a00d0540 msgr2=0x7f76a00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:17.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.637+0000 7f76a8200640 1 -- 172.21.15.66:0/1975359079 shutdown_connections 2024-03-30T00:41:17.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.637+0000 7f76a8200640 1 -- 172.21.15.66:0/1975359079 wait complete. 2024-03-30T00:41:17.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a8200640 1 Processor -- start 2024-03-30T00:41:17.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a8200640 1 -- start start 2024-03-30T00:41:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a8200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76a00af420 0x7f76a0154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a8200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76a00cee50 0x7f76a00b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a8200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76a015de30 0x7f76a00b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a8200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0156060 con 0x7f76a00cee50 2024-03-30T00:41:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a8200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01554e0 con 0x7f76a015de30 2024-03-30T00:41:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a8200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01557c0 con 0x7f76a00af420 2024-03-30T00:41:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76a00cee50 0x7f76a00b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76a00cee50 0x7f76a00b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39762/0 (socket says 172.21.15.66:39762) 2024-03-30T00:41:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a4e00640 1 -- 172.21.15.66:0/9876844 learned_addr learned my addr 172.21.15.66:0/9876844 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a5800640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76a00af420 0x7f76a0154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a6200640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76a015de30 0x7f76a00b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a4e00640 1 -- 172.21.15.66:0/9876844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76a00af420 msgr2=0x7f76a0154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a4e00640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76a00af420 0x7f76a0154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a4e00640 1 -- 172.21.15.66:0/9876844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76a015de30 msgr2=0x7f76a00b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a4e00640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76a015de30 0x7f76a00b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a4e00640 1 -- 172.21.15.66:0/9876844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c009420 con 0x7f76a00cee50 2024-03-30T00:41:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a4e00640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76a00cee50 0x7f76a00b0ff0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f768c0190b0 tx=0x7f768c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f769d600640 1 -- 172.21.15.66:0/9876844 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0ac410 con 0x7f76a00cee50 2024-03-30T00:41:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f769d600640 1 -- 172.21.15.66:0/9876844 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c006e20 con 0x7f76a00cee50 2024-03-30T00:41:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f769d600640 1 -- 172.21.15.66:0/9876844 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0b5840 con 0x7f76a00cee50 2024-03-30T00:41:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a8200640 1 -- 172.21.15.66:0/9876844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a0155aa0 con 0x7f76a00cee50 2024-03-30T00:41:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.641+0000 7f76a8200640 1 -- 172.21.15.66:0/9876844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a0155120 con 0x7f76a00cee50 2024-03-30T00:41:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.645+0000 7f76a8200640 1 -- 172.21.15.66:0/9876844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a00cdf60 con 0x7f76a00cee50 2024-03-30T00:41:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.645+0000 7f769d600640 1 -- 172.21.15.66:0/9876844 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f768c0b3040 con 0x7f76a00cee50 2024-03-30T00:41:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.645+0000 7f769d600640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7680066690 0x7f7680068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.645+0000 7f769d600640 1 -- 172.21.15.66:0/9876844 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f768c11def0 con 0x7f76a00cee50 2024-03-30T00:41:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.645+0000 7f76a5800640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7680066690 0x7f7680068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:17.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.645+0000 7f76a5800640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7680066690 0x7f7680068b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f76940011f0 tx=0x7f7694009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:17.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.649+0000 7f769d600640 1 -- 172.21.15.66:0/9876844 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f768c0eb460 con 0x7f76a00cee50 2024-03-30T00:41:17.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.865+0000 7f76a8200640 1 -- 172.21.15.66:0/9876844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f76a0155900 con 0x7f76a00cee50 2024-03-30T00:41:17.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f769d600640 1 -- 172.21.15.66:0/9876844 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f76a0155900 con 0x7f76a00cee50 2024-03-30T00:41:17.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 -- 172.21.15.66:0/9876844 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7680066690 msgr2=0x7f7680068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7680066690 0x7f7680068b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f76940011f0 tx=0x7f7694009040 comp rx=0 tx=0).stop 2024-03-30T00:41:17.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 -- 172.21.15.66:0/9876844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76a00cee50 msgr2=0x7f76a00b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:17.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76a00cee50 0x7f76a00b0ff0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f768c0190b0 tx=0x7f768c019240 comp rx=0 tx=0).stop 2024-03-30T00:41:17.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 -- 172.21.15.66:0/9876844 shutdown_connections 2024-03-30T00:41:17.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76a015de30 0x7f76a00b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7680066690 0x7f7680068b50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76a00cee50 0x7f76a00b0ff0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 --2- 172.21.15.66:0/9876844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76a00af420 0x7f76a0154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:17.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 -- 172.21.15.66:0/9876844 >> 172.21.15.66:0/9876844 conn(0x7f76a00d0540 msgr2=0x7f76a0165d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:17.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 -- 172.21.15.66:0/9876844 shutdown_connections 2024-03-30T00:41:17.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:17.869+0000 7f76a8200640 1 -- 172.21.15.66:0/9876844 wait complete. 2024-03-30T00:41:17.900 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:18.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.029+0000 7f633b400640 1 Processor -- start 2024-03-30T00:41:18.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.029+0000 7f633b400640 1 -- start start 2024-03-30T00:41:18.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f633b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f63340cde70 0x7f633415a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f633b400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340cd3e0 con 0x7f63340cda90 2024-03-30T00:41:18.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f633b400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633415a990 con 0x7f63340cec30 2024-03-30T00:41:18.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f633b400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63340b1b80 con 0x7f63340cde70 2024-03-30T00:41:18.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f6338a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f63340cda90 0x7f63340ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48950/0 (socket says 172.21.15.66:48950) 2024-03-30T00:41:18.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f6338a00640 1 -- 172.21.15.66:0/260748912 learned_addr learned my addr 172.21.15.66:0/260748912 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:18.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f6339400640 1 --2- 172.21.15.66:0/260748912 >> v2:172.21.15.66:3300/0 conn(0x7f63340cde70 0x7f633415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f6333400640 1 -- 172.21.15.66:0/260748912 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557922791 0 0) 0x7f63340cd3e0 con 0x7f63340cda90 2024-03-30T00:41:18.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f6333400640 1 -- 172.21.15.66:0/260748912 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f631c003530 con 0x7f63340cda90 2024-03-30T00:41:18.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f6339400640 1 -- 172.21.15.66:0/260748912 >> v1:172.21.15.156:6789/0 conn(0x7f63340cec30 legacy=0x7f6334159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f6339400640 1 -- 172.21.15.66:0/260748912 >> v1:172.21.15.66:6789/0 conn(0x7f63340cda90 legacy=0x7f63340ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.033+0000 7f6339400640 1 -- 172.21.15.66:0/260748912 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340b11e0 con 0x7f63340cde70 2024-03-30T00:41:18.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f6339400640 1 --2- 172.21.15.66:0/260748912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63340cde70 0x7f633415a370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f632c009d90 tx=0x7f632c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10c230a4373f7de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:18.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f6333400640 1 -- 172.21.15.66:0/260748912 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c0a59a0 con 0x7f63340cde70 2024-03-30T00:41:18.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f6333400640 1 -- 172.21.15.66:0/260748912 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c0a5b00 con 0x7f63340cde70 2024-03-30T00:41:18.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f6333400640 1 -- 172.21.15.66:0/260748912 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c0a5e90 con 0x7f63340cde70 2024-03-30T00:41:18.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 -- 172.21.15.66:0/260748912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63340cde70 msgr2=0x7f633415a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 --2- 172.21.15.66:0/260748912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63340cde70 0x7f633415a370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f632c009d90 tx=0x7f632c01c8e0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 -- 172.21.15.66:0/260748912 shutdown_connections 2024-03-30T00:41:18.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 --2- 172.21.15.66:0/260748912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63340cde70 0x7f633415a370 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 -- 172.21.15.66:0/260748912 >> 172.21.15.66:0/260748912 conn(0x7f6334069f60 msgr2=0x7f633406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:18.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 -- 172.21.15.66:0/260748912 shutdown_connections 2024-03-30T00:41:18.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 -- 172.21.15.66:0/260748912 wait complete. 2024-03-30T00:41:18.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 Processor -- start 2024-03-30T00:41:18.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 -- start start 2024-03-30T00:41:18.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63340cda90 0x7f6334152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63340cde70 0x7f63340ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63340cec30 0x7f63340acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633415b6c0 con 0x7f63340cde70 2024-03-30T00:41:18.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6334156660 con 0x7f63340cec30 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.037+0000 7f633b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6334159c20 con 0x7f63340cda90 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6333e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63340cde70 0x7f63340ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6333e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63340cde70 0x7f63340ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39774/0 (socket says 172.21.15.66:39774) 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6333e00640 1 -- 172.21.15.66:0/4001441647 learned_addr learned my addr 172.21.15.66:0/4001441647 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6338a00640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63340cda90 0x7f6334152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6339400640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63340cec30 0x7f63340acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6333e00640 1 -- 172.21.15.66:0/4001441647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63340cda90 msgr2=0x7f6334152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6333e00640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63340cda90 0x7f6334152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6333e00640 1 -- 172.21.15.66:0/4001441647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63340cec30 msgr2=0x7f63340acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6333e00640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63340cec30 0x7f63340acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6333e00640 1 -- 172.21.15.66:0/4001441647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c009a70 con 0x7f63340cde70 2024-03-30T00:41:18.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6333e00640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63340cde70 0x7f63340ac600 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f632000c4d0 tx=0x7f632000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:18.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6330c00640 1 -- 172.21.15.66:0/4001441647 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63200052d0 con 0x7f63340cde70 2024-03-30T00:41:18.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6330c00640 1 -- 172.21.15.66:0/4001441647 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6320005430 con 0x7f63340cde70 2024-03-30T00:41:18.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6330c00640 1 -- 172.21.15.66:0/4001441647 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320012830 con 0x7f63340cde70 2024-03-30T00:41:18.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f633b400640 1 -- 172.21.15.66:0/4001441647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633415a3e0 con 0x7f63340cde70 2024-03-30T00:41:18.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f633b400640 1 -- 172.21.15.66:0/4001441647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633415b3e0 con 0x7f63340cde70 2024-03-30T00:41:18.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f633b400640 1 -- 172.21.15.66:0/4001441647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6334066c30 con 0x7f63340cde70 2024-03-30T00:41:18.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6330c00640 1 -- 172.21.15.66:0/4001441647 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6320012990 con 0x7f63340cde70 2024-03-30T00:41:18.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.041+0000 7f6330c00640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6314066690 0x7f6314068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.045+0000 7f6330c00640 1 -- 172.21.15.66:0/4001441647 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6320083ab0 con 0x7f63340cde70 2024-03-30T00:41:18.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.045+0000 7f6338a00640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6314066690 0x7f6314068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.045+0000 7f6338a00640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6314066690 0x7f6314068b50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63280011f0 tx=0x7f6328009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:18.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.045+0000 7f6330c00640 1 -- 172.21.15.66:0/4001441647 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6320051020 con 0x7f63340cde70 2024-03-30T00:41:18.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f633b400640 1 -- 172.21.15.66:0/4001441647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6334165280 con 0x7f63340cde70 2024-03-30T00:41:18.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f6330c00640 1 -- 172.21.15.66:0/4001441647 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+12594 (secure 0 0 0) 0x7f6320054640 con 0x7f63340cde70 2024-03-30T00:41:18.219 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:18.219 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":36,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:41:15.061102+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f633b400640 1 -- 172.21.15.66:0/4001441647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6314066690 msgr2=0x7f6314068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f633b400640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6314066690 0x7f6314068b50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63280011f0 tx=0x7f6328009040 comp rx=0 tx=0).stop 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f633b400640 1 -- 172.21.15.66:0/4001441647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63340cde70 msgr2=0x7f63340ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f633b400640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63340cde70 0x7f63340ac600 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f632000c4d0 tx=0x7f632000c990 comp rx=0 tx=0).stop 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f633b400640 1 -- 172.21.15.66:0/4001441647 shutdown_connections 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f633b400640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63340cec30 0x7f63340acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f633b400640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6314066690 0x7f6314068b50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f633b400640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63340cde70 0x7f63340ac600 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f633b400640 1 --2- 172.21.15.66:0/4001441647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63340cda90 0x7f6334152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.209+0000 7f633b400640 1 -- 172.21.15.66:0/4001441647 >> 172.21.15.66:0/4001441647 conn(0x7f6334069f60 msgr2=0x7f633415b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.213+0000 7f633b400640 1 -- 172.21.15.66:0/4001441647 shutdown_connections 2024-03-30T00:41:18.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.213+0000 7f633b400640 1 -- 172.21.15.66:0/4001441647 wait complete. 2024-03-30T00:41:18.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:41:18.241 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:41:18.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95bbe00640 1 Processor -- start 2024-03-30T00:41:18.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95bbe00640 1 -- start start 2024-03-30T00:41:18.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95bbe00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f95b40cf2f0 0x7f95b406b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95bbe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f95b40cb570 0x7f95b40cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95bbe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b40cb0e0 con 0x7f95b415ca80 2024-03-30T00:41:18.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95bbe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b40cf7f0 con 0x7f95b40cb570 2024-03-30T00:41:18.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95bbe00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b40cfad0 con 0x7f95b40cf2f0 2024-03-30T00:41:18.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95b9e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f95b40cb570 0x7f95b40cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95b9400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f95b415ca80 0x7f95b406b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48966/0 (socket says 172.21.15.66:48966) 2024-03-30T00:41:18.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95b9400640 1 -- 172.21.15.66:0/3523023581 learned_addr learned my addr 172.21.15.66:0/3523023581 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:18.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95b8a00640 1 --2- 172.21.15.66:0/3523023581 >> v2:172.21.15.156:3300/0 conn(0x7f95b40cf2f0 0x7f95b406b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95b9e00640 1 -- 172.21.15.66:0/3523023581 >> v2:172.21.15.156:3300/0 conn(0x7f95b40cf2f0 msgr2=0x7f95b406b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:18.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95b9e00640 1 --2- 172.21.15.66:0/3523023581 >> v2:172.21.15.156:3300/0 conn(0x7f95b40cf2f0 0x7f95b406b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95b9e00640 1 -- 172.21.15.66:0/3523023581 >> v1:172.21.15.66:6789/0 conn(0x7f95b415ca80 legacy=0x7f95b406b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95b9e00640 1 -- 172.21.15.66:0/3523023581 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b40cad70 con 0x7f95b40cb570 2024-03-30T00:41:18.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.377+0000 7f95b8a00640 1 --2- 172.21.15.66:0/3523023581 >> v2:172.21.15.156:3300/0 conn(0x7f95b40cf2f0 0x7f95b406b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:18.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95b9e00640 1 --2- 172.21.15.66:0/3523023581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cb570 0x7f95b40cba20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f95b406ced0 tx=0x7f95ac01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4049494c15da5db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:18.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95b3e00640 1 -- 172.21.15.66:0/3523023581 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0af600 con 0x7f95b40cb570 2024-03-30T00:41:18.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95b3e00640 1 -- 172.21.15.66:0/3523023581 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac0ab050 con 0x7f95b40cb570 2024-03-30T00:41:18.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95b3e00640 1 -- 172.21.15.66:0/3523023581 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0af980 con 0x7f95b40cb570 2024-03-30T00:41:18.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95bbe00640 1 -- 172.21.15.66:0/3523023581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cb570 msgr2=0x7f95b40cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95bbe00640 1 --2- 172.21.15.66:0/3523023581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cb570 0x7f95b40cba20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f95b406ced0 tx=0x7f95ac01f110 comp rx=0 tx=0).stop 2024-03-30T00:41:18.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95bbe00640 1 -- 172.21.15.66:0/3523023581 shutdown_connections 2024-03-30T00:41:18.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95bbe00640 1 --2- 172.21.15.66:0/3523023581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cb570 0x7f95b40cba20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95bbe00640 1 --2- 172.21.15.66:0/3523023581 >> v2:172.21.15.156:3300/0 conn(0x7f95b40cf2f0 0x7f95b406b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95bbe00640 1 -- 172.21.15.66:0/3523023581 >> 172.21.15.66:0/3523023581 conn(0x7f95b40b2090 msgr2=0x7f95b40b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:18.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95bbe00640 1 -- 172.21.15.66:0/3523023581 shutdown_connections 2024-03-30T00:41:18.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95bbe00640 1 -- 172.21.15.66:0/3523023581 wait complete. 2024-03-30T00:41:18.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95bbe00640 1 Processor -- start 2024-03-30T00:41:18.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.381+0000 7f95bbe00640 1 -- start start 2024-03-30T00:41:18.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95bbe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95b40cb570 0x7f95b406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cf2f0 0x7f95b40b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95bbe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95b415ca80 0x7f95b40b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95bbe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b409df30 con 0x7f95b40cf2f0 2024-03-30T00:41:18.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95bbe00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b40b5fd0 con 0x7f95b40cb570 2024-03-30T00:41:18.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95bbe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b40b42b0 con 0x7f95b415ca80 2024-03-30T00:41:18.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cf2f0 0x7f95b40b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cf2f0 0x7f95b40b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39790/0 (socket says 172.21.15.66:39790) 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b8a00640 1 -- 172.21.15.66:0/3625220649 learned_addr learned my addr 172.21.15.66:0/3625220649 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b9e00640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95b415ca80 0x7f95b40b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b9400640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95b40cb570 0x7f95b406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b8a00640 1 -- 172.21.15.66:0/3625220649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95b415ca80 msgr2=0x7f95b40b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b8a00640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95b415ca80 0x7f95b40b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b8a00640 1 -- 172.21.15.66:0/3625220649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95b40cb570 msgr2=0x7f95b406e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b8a00640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95b40cb570 0x7f95b406e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b8a00640 1 -- 172.21.15.66:0/3625220649 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac009a70 con 0x7f95b40cf2f0 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b8a00640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cf2f0 0x7f95b40b5460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f95a000c7c0 tx=0x7f95a000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b1600640 1 -- 172.21.15.66:0/3625220649 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0007a30 con 0x7f95b40cf2f0 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b1600640 1 -- 172.21.15.66:0/3625220649 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a0007b90 con 0x7f95b40cf2f0 2024-03-30T00:41:18.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b1600640 1 -- 172.21.15.66:0/3625220649 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a000ce70 con 0x7f95b40cf2f0 2024-03-30T00:41:18.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95bbe00640 1 -- 172.21.15.66:0/3625220649 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b40b4590 con 0x7f95b40cf2f0 2024-03-30T00:41:18.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95bbe00640 1 -- 172.21.15.66:0/3625220649 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b40b4b50 con 0x7f95b40cf2f0 2024-03-30T00:41:18.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95bbe00640 1 -- 172.21.15.66:0/3625220649 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b40b4d40 con 0x7f95b40cf2f0 2024-03-30T00:41:18.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.385+0000 7f95b1600640 1 -- 172.21.15.66:0/3625220649 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f95a00113f0 con 0x7f95b40cf2f0 2024-03-30T00:41:18.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.389+0000 7f95b1600640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9594066690 0x7f9594068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.389+0000 7f95b1600640 1 -- 172.21.15.66:0/3625220649 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f95a0084420 con 0x7f95b40cf2f0 2024-03-30T00:41:18.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.389+0000 7f95b9400640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9594066690 0x7f9594068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.389+0000 7f95b9400640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9594066690 0x7f9594068b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005890 tx=0x7f95a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:18.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.393+0000 7f95b1600640 1 -- 172.21.15.66:0/3625220649 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f95a00519a0 con 0x7f95b40cf2f0 2024-03-30T00:41:18.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.553+0000 7f95bbe00640 1 -- 172.21.15.66:0/3625220649 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95b4066c30 con 0x7f95b40cf2f0 2024-03-30T00:41:18.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95b1600640 1 -- 172.21.15.66:0/3625220649 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95a0054fc0 con 0x7f95b40cf2f0 2024-03-30T00:41:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:41:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 -- 172.21.15.66:0/3625220649 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9594066690 msgr2=0x7f9594068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9594066690 0x7f9594068b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005890 tx=0x7f95a8009040 comp rx=0 tx=0).stop 2024-03-30T00:41:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 -- 172.21.15.66:0/3625220649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cf2f0 msgr2=0x7f95b40b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cf2f0 0x7f95b40b5460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f95a000c7c0 tx=0x7f95a000cc80 comp rx=0 tx=0).stop 2024-03-30T00:41:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 -- 172.21.15.66:0/3625220649 shutdown_connections 2024-03-30T00:41:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95b415ca80 0x7f95b40b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9594066690 0x7f9594068b50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cf2f0 0x7f95b40b5460 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 --2- 172.21.15.66:0/3625220649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95b40cb570 0x7f95b406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 -- 172.21.15.66:0/3625220649 >> 172.21.15.66:0/3625220649 conn(0x7f95b40b2090 msgr2=0x7f95b415acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 -- 172.21.15.66:0/3625220649 shutdown_connections 2024-03-30T00:41:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.557+0000 7f95bbe00640 1 -- 172.21.15.66:0/3625220649 wait complete. 2024-03-30T00:41:18.587 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T00:41:18.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.717+0000 7ff696e00640 1 Processor -- start 2024-03-30T00:41:18.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.717+0000 7ff696e00640 1 -- start start 2024-03-30T00:41:18.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.717+0000 7ff696e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 0x7ff69006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.717+0000 7ff696e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff6900cf2f0 0x7ff69006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.717+0000 7ff696e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6900cb0e0 con 0x7ff69006bb30 2024-03-30T00:41:18.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.717+0000 7ff696e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6900cf7f0 con 0x7ff6900cf2f0 2024-03-30T00:41:18.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.717+0000 7ff696e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6900cfad0 con 0x7ff69015ca80 2024-03-30T00:41:18.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.717+0000 7ff68fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 0x7ff69006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.717+0000 7ff68fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 0x7ff69006b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:36814/0 (socket says 172.21.15.66:36814) 2024-03-30T00:41:18.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.717+0000 7ff68fe00640 1 -- 172.21.15.66:0/358879579 learned_addr learned my addr 172.21.15.66:0/358879579 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:18.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.717+0000 7ff687400640 1 --2- 172.21.15.66:0/358879579 >> v2:172.21.15.156:3300/0 conn(0x7ff6900cf2f0 0x7ff69006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.721+0000 7ff68fe00640 1 -- 172.21.15.66:0/358879579 >> v2:172.21.15.156:3300/0 conn(0x7ff6900cf2f0 msgr2=0x7ff69006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:18.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.721+0000 7ff68fe00640 1 --2- 172.21.15.66:0/358879579 >> v2:172.21.15.156:3300/0 conn(0x7ff6900cf2f0 0x7ff69006b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.721+0000 7ff68fe00640 1 -- 172.21.15.66:0/358879579 >> v1:172.21.15.156:6789/0 conn(0x7ff69006bb30 legacy=0x7ff690098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.721+0000 7ff68fe00640 1 -- 172.21.15.66:0/358879579 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6900cad70 con 0x7ff69015ca80 2024-03-30T00:41:18.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.721+0000 7ff68fe00640 1 --2- 172.21.15.66:0/358879579 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 0x7ff69006b0b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff680009870 tx=0x7ff68001ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09ac0e9bd760256 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:18.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.721+0000 7ff68f400640 1 -- 172.21.15.66:0/358879579 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6800a8a50 con 0x7ff69015ca80 2024-03-30T00:41:18.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.721+0000 7ff68f400640 1 -- 172.21.15.66:0/358879579 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6800a8bb0 con 0x7ff69015ca80 2024-03-30T00:41:18.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.721+0000 7ff68f400640 1 -- 172.21.15.66:0/358879579 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6800b0440 con 0x7ff69015ca80 2024-03-30T00:41:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 -- 172.21.15.66:0/358879579 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 msgr2=0x7ff69006b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 --2- 172.21.15.66:0/358879579 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 0x7ff69006b0b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff680009870 tx=0x7ff68001ed50 comp rx=0 tx=0).stop 2024-03-30T00:41:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 -- 172.21.15.66:0/358879579 shutdown_connections 2024-03-30T00:41:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 --2- 172.21.15.66:0/358879579 >> v2:172.21.15.156:3300/0 conn(0x7ff6900cf2f0 0x7ff69006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 --2- 172.21.15.66:0/358879579 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 0x7ff69006b0b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 -- 172.21.15.66:0/358879579 >> 172.21.15.66:0/358879579 conn(0x7ff6900b2090 msgr2=0x7ff6900b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 -- 172.21.15.66:0/358879579 shutdown_connections 2024-03-30T00:41:18.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 -- 172.21.15.66:0/358879579 wait complete. 2024-03-30T00:41:18.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 Processor -- start 2024-03-30T00:41:18.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 -- start start 2024-03-30T00:41:18.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff69006bb30 0x7ff69009c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6900cf2f0 0x7ff69015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 0x7ff69006f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69009b040 con 0x7ff69006bb30 2024-03-30T00:41:18.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69009b320 con 0x7ff6900cf2f0 2024-03-30T00:41:18.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff696e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69009b600 con 0x7ff69015ca80 2024-03-30T00:41:18.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff68fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff69006bb30 0x7ff69009c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff68fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff69006bb30 0x7ff69009c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39802/0 (socket says 172.21.15.66:39802) 2024-03-30T00:41:18.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff68fe00640 1 -- 172.21.15.66:0/3024306033 learned_addr learned my addr 172.21.15.66:0/3024306033 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:18.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff687400640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6900cf2f0 0x7ff69015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff694e00640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 0x7ff69006f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff68fe00640 1 -- 172.21.15.66:0/3024306033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 msgr2=0x7ff69006f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff68fe00640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 0x7ff69006f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff68fe00640 1 -- 172.21.15.66:0/3024306033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6900cf2f0 msgr2=0x7ff69015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff68fe00640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6900cf2f0 0x7ff69015bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff68fe00640 1 -- 172.21.15.66:0/3024306033 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800094d0 con 0x7ff69006bb30 2024-03-30T00:41:18.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.725+0000 7ff68fe00640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff69006bb30 0x7ff69009c0e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff6800b3040 tx=0x7ff68001baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:18.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.729+0000 7ff68cc00640 1 -- 172.21.15.66:0/3024306033 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6800be070 con 0x7ff69006bb30 2024-03-30T00:41:18.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.729+0000 7ff68cc00640 1 -- 172.21.15.66:0/3024306033 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6800b9420 con 0x7ff69006bb30 2024-03-30T00:41:18.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.729+0000 7ff68cc00640 1 -- 172.21.15.66:0/3024306033 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6800b97f0 con 0x7ff69006bb30 2024-03-30T00:41:18.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.729+0000 7ff696e00640 1 -- 172.21.15.66:0/3024306033 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69009b8e0 con 0x7ff69006bb30 2024-03-30T00:41:18.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.729+0000 7ff696e00640 1 -- 172.21.15.66:0/3024306033 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69015c190 con 0x7ff69006bb30 2024-03-30T00:41:18.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.729+0000 7ff696e00640 1 -- 172.21.15.66:0/3024306033 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69009be00 con 0x7ff69006bb30 2024-03-30T00:41:18.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.729+0000 7ff68cc00640 1 -- 172.21.15.66:0/3024306033 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff6800b7070 con 0x7ff69006bb30 2024-03-30T00:41:18.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.729+0000 7ff68cc00640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff66c066690 0x7ff66c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:18.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.729+0000 7ff68cc00640 1 -- 172.21.15.66:0/3024306033 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff6801215e0 con 0x7ff69006bb30 2024-03-30T00:41:18.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.729+0000 7ff687400640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff66c066690 0x7ff66c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:18.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.733+0000 7ff687400640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff66c066690 0x7ff66c068b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff6900c9490 tx=0x7ff678008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:18.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.733+0000 7ff68cc00640 1 -- 172.21.15.66:0/3024306033 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff69009be00 con 0x7ff69006bb30 2024-03-30T00:41:18.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.897+0000 7ff696e00640 1 -- 172.21.15.66:0/3024306033 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff69009b740 con 0x7ff69006bb30 2024-03-30T00:41:18.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.897+0000 7ff68cc00640 1 -- 172.21.15.66:0/3024306033 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff6800b5060 con 0x7ff69006bb30 2024-03-30T00:41:18.909 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:18.909 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:41:18.909 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:41:18.909 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-03-30T00:41:18.909 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-03-30T00:41:18.909 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:18.909 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:41:18.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 -- 172.21.15.66:0/3024306033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff66c066690 msgr2=0x7ff66c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff66c066690 0x7ff66c068b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff6900c9490 tx=0x7ff678008040 comp rx=0 tx=0).stop 2024-03-30T00:41:18.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 -- 172.21.15.66:0/3024306033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff69006bb30 msgr2=0x7ff69009c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:18.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff69006bb30 0x7ff69009c0e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff6800b3040 tx=0x7ff68001baf0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 -- 172.21.15.66:0/3024306033 shutdown_connections 2024-03-30T00:41:18.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff69015ca80 0x7ff69006f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6900cf2f0 0x7ff69015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff66c066690 0x7ff66c068b50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 --2- 172.21.15.66:0/3024306033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff69006bb30 0x7ff69009c0e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:18.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 -- 172.21.15.66:0/3024306033 >> 172.21.15.66:0/3024306033 conn(0x7ff6900b2090 msgr2=0x7ff6900c44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:18.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 -- 172.21.15.66:0/3024306033 shutdown_connections 2024-03-30T00:41:18.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:18.901+0000 7ff696e00640 1 -- 172.21.15.66:0/3024306033 wait complete. 2024-03-30T00:41:18.930 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T00:41:18.930 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T00:41:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1603800640 1 Processor -- start 2024-03-30T00:41:19.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1603800640 1 -- start start 2024-03-30T00:41:19.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1603800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f15fc098a20 0x7f15fc098e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1603800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15fc0ce930 con 0x7f15fc0b0b20 2024-03-30T00:41:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1603800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15fc099800 con 0x7f15fc0b1280 2024-03-30T00:41:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1603800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15fc099370 con 0x7f15fc098a20 2024-03-30T00:41:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1601800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f15fc098a20 0x7f15fc098e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1601800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f15fc098a20 0x7f15fc098e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57626/0 (socket says 172.21.15.66:57626) 2024-03-30T00:41:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1601800640 1 -- 172.21.15.66:0/3730384584 learned_addr learned my addr 172.21.15.66:0/3730384584 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1600e00640 1 --1- 172.21.15.66:0/3730384584 >> v1:172.21.15.66:6789/0 conn(0x7f15fc0b0b20 0x7f15fc06bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39346/0 (socket says 172.21.15.66:39346) 2024-03-30T00:41:19.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1601800640 1 -- 172.21.15.66:0/3730384584 >> v1:172.21.15.156:6789/0 conn(0x7f15fc0b1280 legacy=0x7f15fc0cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:19.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1601800640 1 -- 172.21.15.66:0/3730384584 >> v1:172.21.15.66:6789/0 conn(0x7f15fc0b0b20 legacy=0x7f15fc06bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:19.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.065+0000 7f1601800640 1 -- 172.21.15.66:0/3730384584 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc099d50 con 0x7f15fc098a20 2024-03-30T00:41:19.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.069+0000 7f1601800640 1 --2- 172.21.15.66:0/3730384584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15fc098a20 0x7f15fc098e30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009e50 tx=0x7f15f401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8633545a0f02c537 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.069+0000 7f15fb400640 1 -- 172.21.15.66:0/3730384584 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f40090d0 con 0x7f15fc098a20 2024-03-30T00:41:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.069+0000 7f15fb400640 1 -- 172.21.15.66:0/3730384584 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f4009230 con 0x7f15fc098a20 2024-03-30T00:41:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.069+0000 7f15fb400640 1 -- 172.21.15.66:0/3730384584 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f40a9620 con 0x7f15fc098a20 2024-03-30T00:41:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.069+0000 7f1603800640 1 -- 172.21.15.66:0/3730384584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15fc098a20 msgr2=0x7f15fc098e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.069+0000 7f1603800640 1 --2- 172.21.15.66:0/3730384584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15fc098a20 0x7f15fc098e30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009e50 tx=0x7f15f401c9f0 comp rx=0 tx=0).stop 2024-03-30T00:41:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.069+0000 7f1603800640 1 -- 172.21.15.66:0/3730384584 shutdown_connections 2024-03-30T00:41:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.069+0000 7f1603800640 1 --2- 172.21.15.66:0/3730384584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15fc098a20 0x7f15fc098e30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.069+0000 7f1603800640 1 -- 172.21.15.66:0/3730384584 >> 172.21.15.66:0/3730384584 conn(0x7f15fc0b2150 msgr2=0x7f15fc0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.069+0000 7f1603800640 1 -- 172.21.15.66:0/3730384584 shutdown_connections 2024-03-30T00:41:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.069+0000 7f1603800640 1 -- 172.21.15.66:0/3730384584 wait complete. 2024-03-30T00:41:19.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1603800640 1 Processor -- start 2024-03-30T00:41:19.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1603800640 1 -- start start 2024-03-30T00:41:19.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1603800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15fc098a20 0x7f15fc161820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:19.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1603800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15fc0b0b20 0x7f15fc0ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:19.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1603800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15fc0b1280 0x7f15fc1620e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:19.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1603800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc0aeab0 con 0x7f15fc098a20 2024-03-30T00:41:19.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1603800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc167890 con 0x7f15fc0b0b20 2024-03-30T00:41:19.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1603800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc166ae0 con 0x7f15fc0b1280 2024-03-30T00:41:19.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1600e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15fc098a20 0x7f15fc161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:19.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1600e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15fc098a20 0x7f15fc161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57630/0 (socket says 172.21.15.66:57630) 2024-03-30T00:41:19.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1600e00640 1 -- 172.21.15.66:0/2610197098 learned_addr learned my addr 172.21.15.66:0/2610197098 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:19.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1601800640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15fc0b1280 0x7f15fc1620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f15fbe00640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15fc0b0b20 0x7f15fc0ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1600e00640 1 -- 172.21.15.66:0/2610197098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15fc0b1280 msgr2=0x7f15fc1620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1600e00640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15fc0b1280 0x7f15fc1620e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1600e00640 1 -- 172.21.15.66:0/2610197098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15fc0b0b20 msgr2=0x7f15fc0ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1600e00640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15fc0b0b20 0x7f15fc0ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1600e00640 1 -- 172.21.15.66:0/2610197098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f4009b30 con 0x7f15fc098a20 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1600e00640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15fc098a20 0x7f15fc161820 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c430 tx=0x7f15f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f15f8c00640 1 -- 172.21.15.66:0/2610197098 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00074e0 con 0x7f15fc098a20 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f15f8c00640 1 -- 172.21.15.66:0/2610197098 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f0007640 con 0x7f15fc098a20 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f15f8c00640 1 -- 172.21.15.66:0/2610197098 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0004a10 con 0x7f15fc098a20 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1603800640 1 -- 172.21.15.66:0/2610197098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc166dc0 con 0x7f15fc098a20 2024-03-30T00:41:19.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.073+0000 7f1603800640 1 -- 172.21.15.66:0/2610197098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc167380 con 0x7f15fc098a20 2024-03-30T00:41:19.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.077+0000 7f1603800640 1 -- 172.21.15.66:0/2610197098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15fc0b1f50 con 0x7f15fc098a20 2024-03-30T00:41:19.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.077+0000 7f15f8c00640 1 -- 172.21.15.66:0/2610197098 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f15f0004b70 con 0x7f15fc098a20 2024-03-30T00:41:19.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.077+0000 7f15f8c00640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15dc066690 0x7f15dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:19.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.077+0000 7f15f8c00640 1 -- 172.21.15.66:0/2610197098 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f15f0083990 con 0x7f15fc098a20 2024-03-30T00:41:19.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.077+0000 7f15fbe00640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15dc066690 0x7f15dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:19.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.077+0000 7f15fbe00640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15dc066690 0x7f15dc068b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0afb10 tx=0x7f15e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:19.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.081+0000 7f15f8c00640 1 -- 172.21.15.66:0/2610197098 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f15f0050f10 con 0x7f15fc098a20 2024-03-30T00:41:19.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.241+0000 7f1603800640 1 -- 172.21.15.66:0/2610197098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f15fc0b1ae0 con 0x7f15fc098a20 2024-03-30T00:41:19.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.857+0000 7f15f8c00640 1 -- 172.21.15.66:0/2610197098 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7f15f0054530 con 0x7f15fc098a20 2024-03-30T00:41:19.868 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-03-30T00:41:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 -- 172.21.15.66:0/2610197098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15dc066690 msgr2=0x7f15dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15dc066690 0x7f15dc068b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0afb10 tx=0x7f15e8008040 comp rx=0 tx=0).stop 2024-03-30T00:41:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 -- 172.21.15.66:0/2610197098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15fc098a20 msgr2=0x7f15fc161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15fc098a20 0x7f15fc161820 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c430 tx=0x7f15f000c8f0 comp rx=0 tx=0).stop 2024-03-30T00:41:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 -- 172.21.15.66:0/2610197098 shutdown_connections 2024-03-30T00:41:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15fc0b1280 0x7f15fc1620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15fc0b0b20 0x7f15fc0ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15dc066690 0x7f15dc068b50 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0afb10 tx=0x7f15e8008040 comp rx=0 tx=0).stop 2024-03-30T00:41:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 --2- 172.21.15.66:0/2610197098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15fc098a20 0x7f15fc161820 secure :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c430 tx=0x7f15f000c8f0 comp rx=0 tx=0).stop 2024-03-30T00:41:19.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 -- 172.21.15.66:0/2610197098 >> 172.21.15.66:0/2610197098 conn(0x7f15fc0b2150 msgr2=0x7f15fc06bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:19.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 -- 172.21.15.66:0/2610197098 shutdown_connections 2024-03-30T00:41:19.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:19.865+0000 7f1603800640 1 -- 172.21.15.66:0/2610197098 wait complete. 2024-03-30T00:41:19.899 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T00:41:20.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bfe000640 1 Processor -- start 2024-03-30T00:41:20.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bfe000640 1 -- start start 2024-03-30T00:41:20.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bfe000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4bf806b190 0x7f4bf8098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:20.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bfe000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bf80cce10 0x7f4bf80cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:20.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bfe000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf8099390 con 0x7f4bf815c9b0 2024-03-30T00:41:20.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bfe000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf80cd800 con 0x7f4bf806b190 2024-03-30T00:41:20.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bfe000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8099670 con 0x7f4bf80cce10 2024-03-30T00:41:20.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bf6a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4bf806b190 0x7f4bf8098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:20.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bf6a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4bf806b190 0x7f4bf8098a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57642/0 (socket says 172.21.15.66:57642) 2024-03-30T00:41:20.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bf6a00640 1 -- 172.21.15.66:0/1321321488 learned_addr learned my addr 172.21.15.66:0/1321321488 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:20.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bf7e00640 1 --2- 172.21.15.66:0/1321321488 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bf80cce10 0x7f4bf80cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:20.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bf6a00640 1 -- 172.21.15.66:0/1321321488 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bf80cce10 msgr2=0x7f4bf80cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:20.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bf6a00640 1 --2- 172.21.15.66:0/1321321488 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bf80cce10 0x7f4bf80cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:20.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bf6a00640 1 -- 172.21.15.66:0/1321321488 >> v1:172.21.15.156:6789/0 conn(0x7f4bf815c9b0 legacy=0x7f4bf806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:20.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bf6a00640 1 -- 172.21.15.66:0/1321321488 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8099950 con 0x7f4bf806b190 2024-03-30T00:41:20.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.041+0000 7f4bf7e00640 1 --2- 172.21.15.66:0/1321321488 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bf80cce10 0x7f4bf80cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:20.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bf6a00640 1 --2- 172.21.15.66:0/1321321488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf806b190 0x7f4bf8098a20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009870 tx=0x7f4be401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1efe5aa720b55 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:20.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bf6000640 1 -- 172.21.15.66:0/1321321488 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40a8a50 con 0x7f4bf806b190 2024-03-30T00:41:20.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bf6000640 1 -- 172.21.15.66:0/1321321488 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be40a8bb0 con 0x7f4bf806b190 2024-03-30T00:41:20.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bf6000640 1 -- 172.21.15.66:0/1321321488 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40b0440 con 0x7f4bf806b190 2024-03-30T00:41:20.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bfe000640 1 -- 172.21.15.66:0/1321321488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf806b190 msgr2=0x7f4bf8098a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:20.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bfe000640 1 --2- 172.21.15.66:0/1321321488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf806b190 0x7f4bf8098a20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009870 tx=0x7f4be401ed50 comp rx=0 tx=0).stop 2024-03-30T00:41:20.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bfe000640 1 -- 172.21.15.66:0/1321321488 shutdown_connections 2024-03-30T00:41:20.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bfe000640 1 --2- 172.21.15.66:0/1321321488 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bf80cce10 0x7f4bf80cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:20.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bfe000640 1 --2- 172.21.15.66:0/1321321488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf806b190 0x7f4bf8098a20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:20.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bfe000640 1 -- 172.21.15.66:0/1321321488 >> 172.21.15.66:0/1321321488 conn(0x7f4bf80d0590 msgr2=0x7f4bf80d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:20.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bfe000640 1 -- 172.21.15.66:0/1321321488 shutdown_connections 2024-03-30T00:41:20.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bfe000640 1 -- 172.21.15.66:0/1321321488 wait complete. 2024-03-30T00:41:20.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.045+0000 7f4bfe000640 1 Processor -- start 2024-03-30T00:41:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bfe000640 1 -- start start 2024-03-30T00:41:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bfe000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bf806b190 0x7f4bf815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bfe000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf80cce10 0x7f4bf81ef980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bfe000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf815c9b0 0x7f4bf80c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bfe000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf81efec0 con 0x7f4bf815c9b0 2024-03-30T00:41:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bfe000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf815e080 con 0x7f4bf80cce10 2024-03-30T00:41:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bfe000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf815e360 con 0x7f4bf806b190 2024-03-30T00:41:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bf7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf815c9b0 0x7f4bf80c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bf7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf815c9b0 0x7f4bf80c6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57658/0 (socket says 172.21.15.66:57658) 2024-03-30T00:41:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bf7e00640 1 -- 172.21.15.66:0/326954575 learned_addr learned my addr 172.21.15.66:0/326954575 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bf6a00640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf80cce10 0x7f4bf81ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bf7400640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bf806b190 0x7f4bf815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bf7e00640 1 -- 172.21.15.66:0/326954575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bf806b190 msgr2=0x7f4bf815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:20.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bf7e00640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bf806b190 0x7f4bf815daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:20.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bf7e00640 1 -- 172.21.15.66:0/326954575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf80cce10 msgr2=0x7f4bf81ef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:20.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bf7e00640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf80cce10 0x7f4bf81ef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:20.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bf7e00640 1 -- 172.21.15.66:0/326954575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40094d0 con 0x7f4bf815c9b0 2024-03-30T00:41:20.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bf7e00640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf815c9b0 0x7f4bf80c6900 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0007ef0 tx=0x7f4bf000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:20.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bef400640 1 -- 172.21.15.66:0/326954575 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0003ba0 con 0x7f4bf815c9b0 2024-03-30T00:41:20.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bef400640 1 -- 172.21.15.66:0/326954575 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf000f040 con 0x7f4bf815c9b0 2024-03-30T00:41:20.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bef400640 1 -- 172.21.15.66:0/326954575 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000de80 con 0x7f4bf815c9b0 2024-03-30T00:41:20.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bfe000640 1 -- 172.21.15.66:0/326954575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf815e640 con 0x7f4bf815c9b0 2024-03-30T00:41:20.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bfe000640 1 -- 172.21.15.66:0/326954575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf80c6e40 con 0x7f4bf815c9b0 2024-03-30T00:41:20.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.049+0000 7f4bfe000640 1 -- 172.21.15.66:0/326954575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf8066c30 con 0x7f4bf815c9b0 2024-03-30T00:41:20.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.053+0000 7f4bef400640 1 -- 172.21.15.66:0/326954575 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4bf0004030 con 0x7f4bf815c9b0 2024-03-30T00:41:20.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.053+0000 7f4bef400640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4bd8066690 0x7f4bd8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:20.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.053+0000 7f4bef400640 1 -- 172.21.15.66:0/326954575 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f4bf00843f0 con 0x7f4bf815c9b0 2024-03-30T00:41:20.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.053+0000 7f4bf7400640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4bd8066690 0x7f4bd8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:20.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.053+0000 7f4bf7400640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4bd8066690 0x7f4bd8068b50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4be8000f70 tx=0x7f4be8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:20.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.057+0000 7f4bef400640 1 -- 172.21.15.66:0/326954575 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4bf0051810 con 0x7f4bf815c9b0 2024-03-30T00:41:20.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.217+0000 7f4bfe000640 1 -- 172.21.15.66:0/326954575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4bf806e250 con 0x7f4bf815c9b0 2024-03-30T00:41:20.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.861+0000 7f4bef400640 1 -- 172.21.15.66:0/326954575 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4bf005b070 con 0x7f4bf815c9b0 2024-03-30T00:41:20.872 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-03-30T00:41:20.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.873+0000 7f4bfe000640 1 -- 172.21.15.66:0/326954575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4bd8066690 msgr2=0x7f4bd8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:20.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.873+0000 7f4bfe000640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4bd8066690 0x7f4bd8068b50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4be8000f70 tx=0x7f4be8009040 comp rx=0 tx=0).stop 2024-03-30T00:41:20.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.873+0000 7f4bfe000640 1 -- 172.21.15.66:0/326954575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf815c9b0 msgr2=0x7f4bf80c6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:20.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.873+0000 7f4bfe000640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf815c9b0 0x7f4bf80c6900 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0007ef0 tx=0x7f4bf000d4a0 comp rx=0 tx=0).stop 2024-03-30T00:41:20.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.873+0000 7f4bfe000640 1 -- 172.21.15.66:0/326954575 shutdown_connections 2024-03-30T00:41:20.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.873+0000 7f4bfe000640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4bd8066690 0x7f4bd8068b50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:20.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.873+0000 7f4bfe000640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf815c9b0 0x7f4bf80c6900 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:20.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.873+0000 7f4bfe000640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf80cce10 0x7f4bf81ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:20.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.873+0000 7f4bfe000640 1 --2- 172.21.15.66:0/326954575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bf806b190 0x7f4bf815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:20.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.873+0000 7f4bfe000640 1 -- 172.21.15.66:0/326954575 >> 172.21.15.66:0/326954575 conn(0x7f4bf80d0590 msgr2=0x7f4bf806fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:20.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.877+0000 7f4bfe000640 1 -- 172.21.15.66:0/326954575 shutdown_connections 2024-03-30T00:41:20.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:20.877+0000 7f4bfe000640 1 -- 172.21.15.66:0/326954575 wait complete. 2024-03-30T00:41:20.915 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T00:41:21.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.049+0000 7fdbbaa00640 1 Processor -- start 2024-03-30T00:41:21.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.049+0000 7fdbbaa00640 1 -- start start 2024-03-30T00:41:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbbaa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdbb415fda0 0x7fdbb40b2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbbaa00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdbb40b2850 0x7fdbb406efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbbaa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb406a350 con 0x7fdbb4070ec0 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbbaa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb406e5f0 con 0x7fdbb415fda0 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbbaa00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb406ebb0 con 0x7fdbb40b2850 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbb9600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdbb415fda0 0x7fdbb40b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbba000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdbb4070ec0 0x7fdbb40b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39360/0 (socket says 172.21.15.66:39360) 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbba000640 1 -- 172.21.15.66:0/3997655240 learned_addr learned my addr 172.21.15.66:0/3997655240 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbb8c00640 1 --2- 172.21.15.66:0/3997655240 >> v2:172.21.15.156:3300/0 conn(0x7fdbb40b2850 0x7fdbb406efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbb9600640 1 -- 172.21.15.66:0/3997655240 >> v2:172.21.15.156:3300/0 conn(0x7fdbb40b2850 msgr2=0x7fdbb406efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbb9600640 1 --2- 172.21.15.66:0/3997655240 >> v2:172.21.15.156:3300/0 conn(0x7fdbb40b2850 0x7fdbb406efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbb9600640 1 -- 172.21.15.66:0/3997655240 >> v1:172.21.15.66:6789/0 conn(0x7fdbb4070ec0 legacy=0x7fdbb40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbb9600640 1 -- 172.21.15.66:0/3997655240 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb406a630 con 0x7fdbb415fda0 2024-03-30T00:41:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.053+0000 7fdbb9600640 1 --2- 172.21.15.66:0/3997655240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb415fda0 0x7fdbb40b2310 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdbb406dc10 tx=0x7fdba801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db8fe28a1898787 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbb3e00640 1 -- 172.21.15.66:0/3997655240 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba80ab610 con 0x7fdbb415fda0 2024-03-30T00:41:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbb3e00640 1 -- 172.21.15.66:0/3997655240 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba80ab770 con 0x7fdbb415fda0 2024-03-30T00:41:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbb3e00640 1 -- 172.21.15.66:0/3997655240 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba80abb00 con 0x7fdbb415fda0 2024-03-30T00:41:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 -- 172.21.15.66:0/3997655240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb415fda0 msgr2=0x7fdbb40b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 --2- 172.21.15.66:0/3997655240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb415fda0 0x7fdbb40b2310 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdbb406dc10 tx=0x7fdba801c450 comp rx=0 tx=0).stop 2024-03-30T00:41:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 -- 172.21.15.66:0/3997655240 shutdown_connections 2024-03-30T00:41:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 --2- 172.21.15.66:0/3997655240 >> v2:172.21.15.156:3300/0 conn(0x7fdbb40b2850 0x7fdbb406efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 --2- 172.21.15.66:0/3997655240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb415fda0 0x7fdbb40b2310 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 -- 172.21.15.66:0/3997655240 >> 172.21.15.66:0/3997655240 conn(0x7fdbb406cf90 msgr2=0x7fdbb406d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 -- 172.21.15.66:0/3997655240 shutdown_connections 2024-03-30T00:41:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 -- 172.21.15.66:0/3997655240 wait complete. 2024-03-30T00:41:21.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 Processor -- start 2024-03-30T00:41:21.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 -- start start 2024-03-30T00:41:21.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbb4070ec0 0x7fdbb415f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb40b2850 0x7fdbb4158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbb415fda0 0x7fdbb41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb415f7b0 con 0x7fdbb40b2850 2024-03-30T00:41:21.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41557b0 con 0x7fdbb415fda0 2024-03-30T00:41:21.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbbaa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb415b640 con 0x7fdbb4070ec0 2024-03-30T00:41:21.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbb8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb40b2850 0x7fdbb4158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.057+0000 7fdbb8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb40b2850 0x7fdbb4158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57678/0 (socket says 172.21.15.66:57678) 2024-03-30T00:41:21.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb8c00640 1 -- 172.21.15.66:0/2863034521 learned_addr learned my addr 172.21.15.66:0/2863034521 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:21.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb8c00640 1 -- 172.21.15.66:0/2863034521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbb4070ec0 msgr2=0x7fdbb415f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb8c00640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbb4070ec0 0x7fdbb415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb8c00640 1 -- 172.21.15.66:0/2863034521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbb415fda0 msgr2=0x7fdbb41526d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T00:41:21.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb8c00640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbb415fda0 0x7fdbb41526d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb8c00640 1 -- 172.21.15.66:0/2863034521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8009420 con 0x7fdbb40b2850 2024-03-30T00:41:21.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb8c00640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb40b2850 0x7fdbb4158fa0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdba000d7d0 tx=0x7fdba000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:21.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb1600640 1 -- 172.21.15.66:0/2863034521 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0018070 con 0x7fdbb40b2850 2024-03-30T00:41:21.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb1600640 1 -- 172.21.15.66:0/2863034521 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba0004020 con 0x7fdbb40b2850 2024-03-30T00:41:21.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbbaa00640 1 -- 172.21.15.66:0/2863034521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb415b920 con 0x7fdbb40b2850 2024-03-30T00:41:21.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb1600640 1 -- 172.21.15.66:0/2863034521 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba00041d0 con 0x7fdbb40b2850 2024-03-30T00:41:21.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbbaa00640 1 -- 172.21.15.66:0/2863034521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb415bee0 con 0x7fdbb40b2850 2024-03-30T00:41:21.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbbaa00640 1 -- 172.21.15.66:0/2863034521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb4073e00 con 0x7fdbb40b2850 2024-03-30T00:41:21.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb1600640 1 -- 172.21.15.66:0/2863034521 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdba0004330 con 0x7fdbb40b2850 2024-03-30T00:41:21.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.061+0000 7fdbb1600640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb94066690 0x7fdb94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.065+0000 7fdbb1600640 1 -- 172.21.15.66:0/2863034521 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fdba00841d0 con 0x7fdbb40b2850 2024-03-30T00:41:21.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.065+0000 7fdbb9600640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb94066690 0x7fdb94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.065+0000 7fdbb9600640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb94066690 0x7fdb94068b50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdba80af000 tx=0x7fdba8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:21.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.069+0000 7fdbb1600640 1 -- 172.21.15.66:0/2863034521 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdba00513c0 con 0x7fdbb40b2850 2024-03-30T00:41:21.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.265+0000 7fdbbaa00640 1 -- 172.21.15.66:0/2863034521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdbb40cc570 con 0x7fdbb40b2850 2024-03-30T00:41:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.273+0000 7fdbb1600640 1 -- 172.21.15.66:0/2863034521 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdba00549e0 con 0x7fdbb40b2850 2024-03-30T00:41:21.283 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 5 and data pool 6 2024-03-30T00:41:21.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.277+0000 7fdbbaa00640 1 -- 172.21.15.66:0/2863034521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb94066690 msgr2=0x7fdb94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.277+0000 7fdbbaa00640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb94066690 0x7fdb94068b50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdba80af000 tx=0x7fdba8008040 comp rx=0 tx=0).stop 2024-03-30T00:41:21.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.277+0000 7fdbbaa00640 1 -- 172.21.15.66:0/2863034521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb40b2850 msgr2=0x7fdbb4158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.277+0000 7fdbbaa00640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb40b2850 0x7fdbb4158fa0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdba000d7d0 tx=0x7fdba000dc90 comp rx=0 tx=0).stop 2024-03-30T00:41:21.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.281+0000 7fdbbaa00640 1 -- 172.21.15.66:0/2863034521 shutdown_connections 2024-03-30T00:41:21.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.281+0000 7fdbbaa00640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbb415fda0 0x7fdbb41526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.281+0000 7fdbbaa00640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb94066690 0x7fdb94068b50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.281+0000 7fdbbaa00640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb40b2850 0x7fdbb4158fa0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.281+0000 7fdbbaa00640 1 --2- 172.21.15.66:0/2863034521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbb4070ec0 0x7fdbb415f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.281+0000 7fdbbaa00640 1 -- 172.21.15.66:0/2863034521 >> 172.21.15.66:0/2863034521 conn(0x7fdbb406cf90 msgr2=0x7fdbb40ab970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:21.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.281+0000 7fdbbaa00640 1 -- 172.21.15.66:0/2863034521 shutdown_connections 2024-03-30T00:41:21.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.281+0000 7fdbbaa00640 1 -- 172.21.15.66:0/2863034521 wait complete. 2024-03-30T00:41:21.325 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:21.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.453+0000 7f0b43e00640 1 Processor -- start 2024-03-30T00:41:21.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.453+0000 7f0b43e00640 1 -- start start 2024-03-30T00:41:21.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b43e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b3c15de30 0x7f0b3c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b43e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0b3c0af420 0x7f0b3c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b43e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b3c0ce740 0x7f0b3c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b43e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c0cf110 con 0x7f0b3c0ce740 2024-03-30T00:41:21.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b43e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c0cbb90 con 0x7f0b3c0af420 2024-03-30T00:41:21.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b43e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0ce1b0 con 0x7f0b3c15de30 2024-03-30T00:41:21.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b41e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b3c0ce740 0x7f0b3c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b41e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b3c0ce740 0x7f0b3c0cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57684/0 (socket says 172.21.15.66:57684) 2024-03-30T00:41:21.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b41e00640 1 -- 172.21.15.66:0/3426111257 learned_addr learned my addr 172.21.15.66:0/3426111257 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:21.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b41400640 1 --2- 172.21.15.66:0/3426111257 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b3c15de30 0x7f0b3c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b40a00640 1 --2- 172.21.15.66:0/3426111257 >> v2:172.21.15.156:3300/0 conn(0x7f0b3c0af420 0x7f0b3c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b41e00640 1 -- 172.21.15.66:0/3426111257 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b3c15de30 msgr2=0x7f0b3c0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:21.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b41e00640 1 --2- 172.21.15.66:0/3426111257 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b3c15de30 0x7f0b3c0cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b41e00640 1 -- 172.21.15.66:0/3426111257 >> v2:172.21.15.156:3300/0 conn(0x7f0b3c0af420 msgr2=0x7f0b3c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:21.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b41e00640 1 --2- 172.21.15.66:0/3426111257 >> v2:172.21.15.156:3300/0 conn(0x7f0b3c0af420 0x7f0b3c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.457+0000 7f0b41e00640 1 -- 172.21.15.66:0/3426111257 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0afa10 con 0x7f0b3c0ce740 2024-03-30T00:41:21.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b41e00640 1 --2- 172.21.15.66:0/3426111257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b3c0ce740 0x7f0b3c0cebd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0b3100 tx=0x7f0b34020d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feca340d9f1a0d89 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b3be00640 1 -- 172.21.15.66:0/3426111257 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340a9aa0 con 0x7f0b3c0ce740 2024-03-30T00:41:21.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b3be00640 1 -- 172.21.15.66:0/3426111257 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b340a9c00 con 0x7f0b3c0ce740 2024-03-30T00:41:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b3be00640 1 -- 172.21.15.66:0/3426111257 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340b13f0 con 0x7f0b3c0ce740 2024-03-30T00:41:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b43e00640 1 -- 172.21.15.66:0/3426111257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b3c0ce740 msgr2=0x7f0b3c0cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b43e00640 1 --2- 172.21.15.66:0/3426111257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b3c0ce740 0x7f0b3c0cebd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0b3100 tx=0x7f0b34020d00 comp rx=0 tx=0).stop 2024-03-30T00:41:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b43e00640 1 -- 172.21.15.66:0/3426111257 shutdown_connections 2024-03-30T00:41:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b43e00640 1 --2- 172.21.15.66:0/3426111257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b3c0ce740 0x7f0b3c0cebd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b43e00640 1 --2- 172.21.15.66:0/3426111257 >> v2:172.21.15.156:3300/0 conn(0x7f0b3c0af420 0x7f0b3c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b43e00640 1 --2- 172.21.15.66:0/3426111257 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b3c15de30 0x7f0b3c0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b43e00640 1 -- 172.21.15.66:0/3426111257 >> 172.21.15.66:0/3426111257 conn(0x7f0b3c0d0540 msgr2=0x7f0b3c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b43e00640 1 -- 172.21.15.66:0/3426111257 shutdown_connections 2024-03-30T00:41:21.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b43e00640 1 -- 172.21.15.66:0/3426111257 wait complete. 2024-03-30T00:41:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b43e00640 1 Processor -- start 2024-03-30T00:41:21.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.461+0000 7f0b43e00640 1 -- start start 2024-03-30T00:41:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b43e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b3c0af420 0x7f0b3c14c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b43e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b3c0ce740 0x7f0b3c14c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b43e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b3c15de30 0x7f0b3c14a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b43e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c14b030 con 0x7f0b3c0af420 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b43e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c14b310 con 0x7f0b3c0ce740 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b43e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c14b5f0 con 0x7f0b3c15de30 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b41400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b3c0af420 0x7f0b3c14c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b41400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b3c0af420 0x7f0b3c14c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57686/0 (socket says 172.21.15.66:57686) 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b41400640 1 -- 172.21.15.66:0/2575034771 learned_addr learned my addr 172.21.15.66:0/2575034771 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b40a00640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b3c0ce740 0x7f0b3c14c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b41e00640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b3c15de30 0x7f0b3c14a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b41400640 1 -- 172.21.15.66:0/2575034771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b3c15de30 msgr2=0x7f0b3c14a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b41400640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b3c15de30 0x7f0b3c14a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b41400640 1 -- 172.21.15.66:0/2575034771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b3c0ce740 msgr2=0x7f0b3c14c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b41400640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b3c0ce740 0x7f0b3c14c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b41400640 1 -- 172.21.15.66:0/2575034771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34009b30 con 0x7f0b3c0af420 2024-03-30T00:41:21.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b41400640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b3c0af420 0x7f0b3c14c0d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000e870 tx=0x7f0b3000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:21.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b39600640 1 -- 172.21.15.66:0/2575034771 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30009510 con 0x7f0b3c0af420 2024-03-30T00:41:21.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b39600640 1 -- 172.21.15.66:0/2575034771 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3000eec0 con 0x7f0b3c0af420 2024-03-30T00:41:21.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b39600640 1 -- 172.21.15.66:0/2575034771 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000cb40 con 0x7f0b3c0af420 2024-03-30T00:41:21.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b43e00640 1 -- 172.21.15.66:0/2575034771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c14b8d0 con 0x7f0b3c0af420 2024-03-30T00:41:21.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b43e00640 1 -- 172.21.15.66:0/2575034771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c14d3f0 con 0x7f0b3c0af420 2024-03-30T00:41:21.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.465+0000 7f0b43e00640 1 -- 172.21.15.66:0/2575034771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c0cdf60 con 0x7f0b3c0af420 2024-03-30T00:41:21.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.469+0000 7f0b39600640 1 -- 172.21.15.66:0/2575034771 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0b3000cca0 con 0x7f0b3c0af420 2024-03-30T00:41:21.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.469+0000 7f0b39600640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b1c066690 0x7f0b1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.469+0000 7f0b39600640 1 -- 172.21.15.66:0/2575034771 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0b30084ad0 con 0x7f0b3c0af420 2024-03-30T00:41:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.469+0000 7f0b40a00640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b1c066690 0x7f0b1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.469+0000 7f0b40a00640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b1c066690 0x7f0b1c068b50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0c7d50 tx=0x7f0b28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:21.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.473+0000 7f0b39600640 1 -- 172.21.15.66:0/2575034771 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0b30051de0 con 0x7f0b3c0af420 2024-03-30T00:41:21.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.633+0000 7f0b43e00640 1 -- 172.21.15.66:0/2575034771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b3c066c30 con 0x7f0b3c0af420 2024-03-30T00:41:21.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.633+0000 7f0b39600640 1 -- 172.21.15.66:0/2575034771 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f0b30017090 con 0x7f0b3c0af420 2024-03-30T00:41:21.644 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:21.644 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:41:21.269606+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:41:19.292693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T00:41:20.267522+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:21.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 -- 172.21.15.66:0/2575034771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b1c066690 msgr2=0x7f0b1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b1c066690 0x7f0b1c068b50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0c7d50 tx=0x7f0b28008040 comp rx=0 tx=0).stop 2024-03-30T00:41:21.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 -- 172.21.15.66:0/2575034771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b3c0af420 msgr2=0x7f0b3c14c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b3c0af420 0x7f0b3c14c0d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000e870 tx=0x7f0b3000ec70 comp rx=0 tx=0).stop 2024-03-30T00:41:21.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 -- 172.21.15.66:0/2575034771 shutdown_connections 2024-03-30T00:41:21.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b3c15de30 0x7f0b3c14a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b3c0ce740 0x7f0b3c14c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b1c066690 0x7f0b1c068b50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 --2- 172.21.15.66:0/2575034771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b3c0af420 0x7f0b3c14c0d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 -- 172.21.15.66:0/2575034771 >> 172.21.15.66:0/2575034771 conn(0x7f0b3c0d0540 msgr2=0x7f0b3c0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:21.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 -- 172.21.15.66:0/2575034771 shutdown_connections 2024-03-30T00:41:21.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.637+0000 7f0b43e00640 1 -- 172.21.15.66:0/2575034771 wait complete. 2024-03-30T00:41:21.665 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:21.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b53000640 1 Processor -- start 2024-03-30T00:41:21.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b53000640 1 -- start start 2024-03-30T00:41:21.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b53000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7b4c06e800 0x7f7b4c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b53000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c06b0b0 con 0x7f7b4c0cc560 2024-03-30T00:41:21.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b53000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c06b390 con 0x7f7b4c06f8e0 2024-03-30T00:41:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b53000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4c160640 con 0x7f7b4c06e800 2024-03-30T00:41:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b43e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7b4c06e800 0x7f7b4c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b4be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7b4c0cc560 0x7f7b4c06e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39364/0 (socket says 172.21.15.66:39364) 2024-03-30T00:41:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b4be00640 1 -- 172.21.15.66:0/116513813 learned_addr learned my addr 172.21.15.66:0/116513813 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b43e00640 1 -- 172.21.15.66:0/116513813 >> v1:172.21.15.156:6789/0 conn(0x7f7b4c06f8e0 legacy=0x7f7b4c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b43e00640 1 -- 172.21.15.66:0/116513813 >> v1:172.21.15.66:6789/0 conn(0x7f7b4c0cc560 legacy=0x7f7b4c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b4b400640 1 -- 172.21.15.66:0/116513813 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968988147 0 0) 0x7f7b4c06b0b0 con 0x7f7b4c0cc560 2024-03-30T00:41:21.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.797+0000 7f7b43e00640 1 -- 172.21.15.66:0/116513813 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c06b670 con 0x7f7b4c06e800 2024-03-30T00:41:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b43e00640 1 --2- 172.21.15.66:0/116513813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c06e800 0x7f7b4c0d1060 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7b340097c0 tx=0x7f7b3401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3df7a17da303f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b4b400640 1 -- 172.21.15.66:0/116513813 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b340aa5f0 con 0x7f7b4c06e800 2024-03-30T00:41:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b4b400640 1 -- 172.21.15.66:0/116513813 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b340aa750 con 0x7f7b4c06e800 2024-03-30T00:41:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b4b400640 1 -- 172.21.15.66:0/116513813 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b340aaae0 con 0x7f7b4c06e800 2024-03-30T00:41:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b53000640 1 -- 172.21.15.66:0/116513813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c06e800 msgr2=0x7f7b4c0d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b53000640 1 --2- 172.21.15.66:0/116513813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c06e800 0x7f7b4c0d1060 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7b340097c0 tx=0x7f7b3401c470 comp rx=0 tx=0).stop 2024-03-30T00:41:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b53000640 1 -- 172.21.15.66:0/116513813 shutdown_connections 2024-03-30T00:41:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b53000640 1 --2- 172.21.15.66:0/116513813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c06e800 0x7f7b4c0d1060 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b53000640 1 -- 172.21.15.66:0/116513813 >> 172.21.15.66:0/116513813 conn(0x7f7b4c0d0540 msgr2=0x7f7b4c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:21.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b53000640 1 -- 172.21.15.66:0/116513813 shutdown_connections 2024-03-30T00:41:21.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b53000640 1 -- 172.21.15.66:0/116513813 wait complete. 2024-03-30T00:41:21.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.801+0000 7f7b53000640 1 Processor -- start 2024-03-30T00:41:21.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b53000640 1 -- start start 2024-03-30T00:41:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b53000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c06e800 0x7f7b4c15c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b53000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c06f8e0 0x7f7b4c0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b53000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c0cc560 0x7f7b4c1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b53000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c0c8c10 con 0x7f7b4c0cc560 2024-03-30T00:41:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b53000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c0b1510 con 0x7f7b4c06f8e0 2024-03-30T00:41:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b53000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c0c8020 con 0x7f7b4c06e800 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b51000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c0cc560 0x7f7b4c1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b51000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c0cc560 0x7f7b4c1515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57698/0 (socket says 172.21.15.66:57698) 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b51000640 1 -- 172.21.15.66:0/2983214854 learned_addr learned my addr 172.21.15.66:0/2983214854 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b43e00640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c06f8e0 0x7f7b4c0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b4be00640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c06e800 0x7f7b4c15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b51000640 1 -- 172.21.15.66:0/2983214854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c06e800 msgr2=0x7f7b4c15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b51000640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c06e800 0x7f7b4c15c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b51000640 1 -- 172.21.15.66:0/2983214854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c06f8e0 msgr2=0x7f7b4c0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b51000640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c06f8e0 0x7f7b4c0b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b51000640 1 -- 172.21.15.66:0/2983214854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b34009420 con 0x7f7b4c0cc560 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b51000640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c0cc560 0x7f7b4c1515a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400c770 tx=0x7f7b4400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b48c00640 1 -- 172.21.15.66:0/2983214854 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44003b80 con 0x7f7b4c0cc560 2024-03-30T00:41:21.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b48c00640 1 -- 172.21.15.66:0/2983214854 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b44003ce0 con 0x7f7b4c0cc560 2024-03-30T00:41:21.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b48c00640 1 -- 172.21.15.66:0/2983214854 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400aa90 con 0x7f7b4c0cc560 2024-03-30T00:41:21.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b53000640 1 -- 172.21.15.66:0/2983214854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4c0c8270 con 0x7f7b4c0cc560 2024-03-30T00:41:21.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b53000640 1 -- 172.21.15.66:0/2983214854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c0c8830 con 0x7f7b4c0cc560 2024-03-30T00:41:21.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.805+0000 7f7b53000640 1 -- 172.21.15.66:0/2983214854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4c0c8d50 con 0x7f7b4c0cc560 2024-03-30T00:41:21.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.809+0000 7f7b48c00640 1 -- 172.21.15.66:0/2983214854 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7b4400abf0 con 0x7f7b4c0cc560 2024-03-30T00:41:21.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.809+0000 7f7b48c00640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 0x7f7b28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:21.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.809+0000 7f7b48c00640 1 -- 172.21.15.66:0/2983214854 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7b44083fe0 con 0x7f7b4c0cc560 2024-03-30T00:41:21.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.809+0000 7f7b4be00640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 0x7f7b28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:21.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.809+0000 7f7b4be00640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 0x7f7b28068b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c000f70 tx=0x7f7b3c007240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:21.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.813+0000 7f7b48c00640 1 -- 172.21.15.66:0/2983214854 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7b4c0c8d50 con 0x7f7b4c0cc560 2024-03-30T00:41:21.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.973+0000 7f7b53000640 1 -- 172.21.15.66:0/2983214854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b4c0c8d50 con 0x7f7b4c0cc560 2024-03-30T00:41:21.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b48c00640 1 -- 172.21.15.66:0/2983214854 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f7b4c0c8d50 con 0x7f7b4c0cc560 2024-03-30T00:41:21.986 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:21.986 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:41:21.269606+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:41:19.292693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T00:41:20.267522+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:21.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 -- 172.21.15.66:0/2983214854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 msgr2=0x7f7b28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 0x7f7b28068b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c000f70 tx=0x7f7b3c007240 comp rx=0 tx=0).stop 2024-03-30T00:41:21.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 -- 172.21.15.66:0/2983214854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c0cc560 msgr2=0x7f7b4c1515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:21.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c0cc560 0x7f7b4c1515a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400c770 tx=0x7f7b4400cc30 comp rx=0 tx=0).stop 2024-03-30T00:41:21.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 -- 172.21.15.66:0/2983214854 shutdown_connections 2024-03-30T00:41:21.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 0x7f7b28068b50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c0cc560 0x7f7b4c1515a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c06f8e0 0x7f7b4c0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 --2- 172.21.15.66:0/2983214854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c06e800 0x7f7b4c15c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:21.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 -- 172.21.15.66:0/2983214854 >> 172.21.15.66:0/2983214854 conn(0x7f7b4c0d0540 msgr2=0x7f7b4c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:21.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 -- 172.21.15.66:0/2983214854 shutdown_connections 2024-03-30T00:41:21.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:21.977+0000 7f7b53000640 1 -- 172.21.15.66:0/2983214854 wait complete. 2024-03-30T00:41:22.007 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T00:41:22.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a8e00640 1 Processor -- start 2024-03-30T00:41:22.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a8e00640 1 -- start start 2024-03-30T00:41:22.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a8e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40cde70 0x7ff8a415a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:22.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a8e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a40cd3e0 con 0x7ff8a40cec30 2024-03-30T00:41:22.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a8e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a415a990 con 0x7ff8a40cda90 2024-03-30T00:41:22.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a8e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40b1b80 con 0x7ff8a40cde70 2024-03-30T00:41:22.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff899400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff8a40cec30 0x7ff8a4159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39380/0 (socket says 172.21.15.66:39380) 2024-03-30T00:41:22.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff899400640 1 -- 172.21.15.66:0/1835468964 learned_addr learned my addr 172.21.15.66:0/1835468964 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:22.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a2800640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40cde70 0x7ff8a415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:22.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696461468 0 0) 0x7ff8a40cd3e0 con 0x7ff8a40cec30 2024-03-30T00:41:22.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff888003530 con 0x7ff8a40cec30 2024-03-30T00:41:22.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 576956359 0 0) 0x7ff888003530 con 0x7ff8a40cec30 2024-03-30T00:41:22.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40cde70 msgr2=0x7ff8a415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:22.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40cde70 0x7ff8a415a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:22.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 >> v1:172.21.15.156:6789/0 conn(0x7ff8a40cda90 legacy=0x7ff8a40ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:22.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a40b11e0 con 0x7ff8a40cec30 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7ff88c0016a0 con 0x7ff8a40cec30 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 >> v1:172.21.15.66:6789/0 conn(0x7ff8a40cec30 legacy=0x7ff8a4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8880039b0 0x7ff888003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40cde70 0x7ff8880047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff888004d10 0x7ff888005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40b1b80 con 0x7ff888004d10 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88800baf0 con 0x7ff8880039b0 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88800dec0 con 0x7ff8a40cde70 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a2800640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff888004d10 0x7ff888005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff8a1e00640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8880039b0 0x7ff888003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.141+0000 7ff899400640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40cde70 0x7ff8880047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a2800640 1 -- 172.21.15.66:0/1835468964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40cde70 msgr2=0x7ff8880047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a2800640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40cde70 0x7ff8880047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a2800640 1 -- 172.21.15.66:0/1835468964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8880039b0 msgr2=0x7ff888003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a2800640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8880039b0 0x7ff888003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a2800640 1 -- 172.21.15.66:0/1835468964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff888003530 con 0x7ff888004d10 2024-03-30T00:41:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a2800640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff888004d10 0x7ff888005190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009fc0 tx=0x7ff89c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad7f63bc570cf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0a5b80 con 0x7ff888004d10 2024-03-30T00:41:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c00f040 con 0x7ff888004d10 2024-03-30T00:41:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a1400640 1 -- 172.21.15.66:0/1835468964 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0ad440 con 0x7ff888004d10 2024-03-30T00:41:22.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1835468964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff888004d10 msgr2=0x7ff888005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:22.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a8e00640 1 --2- 172.21.15.66:0/1835468964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff888004d10 0x7ff888005190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009fc0 tx=0x7ff89c01cc80 comp rx=0 tx=0).stop 2024-03-30T00:41:22.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a2800640 1 -- 172.21.15.66:0/1835468964 reap_dead start 2024-03-30T00:41:22.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1835468964 shutdown_connections 2024-03-30T00:41:22.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1835468964 >> 172.21.15.66:0/1835468964 conn(0x7ff8a4069f60 msgr2=0x7ff8a406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:22.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.145+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1835468964 shutdown_connections 2024-03-30T00:41:22.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1835468964 wait complete. 2024-03-30T00:41:22.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a8e00640 1 Processor -- start 2024-03-30T00:41:22.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a8e00640 1 -- start start 2024-03-30T00:41:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a8e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff888004d10 0x7ff8a41561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a8e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40acea0 0x7ff8a4156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a8e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8a41573c0 0x7ff8a40ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a8e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a4157840 con 0x7ff8a41573c0 2024-03-30T00:41:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a8e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40c4f80 con 0x7ff888004d10 2024-03-30T00:41:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a8e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40c2f00 con 0x7ff8a40acea0 2024-03-30T00:41:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8a41573c0 0x7ff8a40ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8a41573c0 0x7ff8a40ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57710/0 (socket says 172.21.15.66:57710) 2024-03-30T00:41:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a2800640 1 -- 172.21.15.66:0/1359285400 learned_addr learned my addr 172.21.15.66:0/1359285400 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff899400640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40acea0 0x7ff8a4156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:22.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a1e00640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff888004d10 0x7ff8a41561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:22.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a2800640 1 -- 172.21.15.66:0/1359285400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40acea0 msgr2=0x7ff8a4156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:22.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a2800640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40acea0 0x7ff8a4156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:22.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a2800640 1 -- 172.21.15.66:0/1359285400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff888004d10 msgr2=0x7ff8a41561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:22.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a2800640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff888004d10 0x7ff8a41561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:22.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a2800640 1 -- 172.21.15.66:0/1359285400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a41ee030 con 0x7ff8a41573c0 2024-03-30T00:41:22.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a2800640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8a41573c0 0x7ff8a40ab010 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff89c002240 tx=0x7ff89c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:22.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff89aa00640 1 -- 172.21.15.66:0/1359285400 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0039a0 con 0x7ff8a41573c0 2024-03-30T00:41:22.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff89aa00640 1 -- 172.21.15.66:0/1359285400 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c003b00 con 0x7ff8a41573c0 2024-03-30T00:41:22.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff89aa00640 1 -- 172.21.15.66:0/1359285400 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c003e70 con 0x7ff8a41573c0 2024-03-30T00:41:22.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.149+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1359285400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a41ee310 con 0x7ff8a41573c0 2024-03-30T00:41:22.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.153+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1359285400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a41ee8d0 con 0x7ff8a41573c0 2024-03-30T00:41:22.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.153+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1359285400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a40c3090 con 0x7ff8a41573c0 2024-03-30T00:41:22.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.153+0000 7ff89aa00640 1 -- 172.21.15.66:0/1359285400 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff89c0b4040 con 0x7ff8a41573c0 2024-03-30T00:41:22.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.153+0000 7ff89aa00640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff880066690 0x7ff880068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:22.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.153+0000 7ff89aa00640 1 -- 172.21.15.66:0/1359285400 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff89c11dc00 con 0x7ff8a41573c0 2024-03-30T00:41:22.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.153+0000 7ff8a1e00640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff880066690 0x7ff880068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:22.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.153+0000 7ff8a1e00640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff880066690 0x7ff880068b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff894007fd0 tx=0x7ff894009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:22.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.157+0000 7ff89aa00640 1 -- 172.21.15.66:0/1359285400 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff8a40c3090 con 0x7ff8a41573c0 2024-03-30T00:41:22.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:22.353+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1359285400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff8a40c3090 con 0x7ff8a41573c0 2024-03-30T00:41:23.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff89aa00640 1 -- 172.21.15.66:0/1359285400 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff8a40c3090 con 0x7ff8a41573c0 2024-03-30T00:41:23.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1359285400 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff880066690 msgr2=0x7ff880068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:23.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff880066690 0x7ff880068b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff894007fd0 tx=0x7ff894009040 comp rx=0 tx=0).stop 2024-03-30T00:41:23.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1359285400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8a41573c0 msgr2=0x7ff8a40ab010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:23.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8a41573c0 0x7ff8a40ab010 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff89c002240 tx=0x7ff89c009750 comp rx=0 tx=0).stop 2024-03-30T00:41:23.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1359285400 shutdown_connections 2024-03-30T00:41:23.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff880066690 0x7ff880068b50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:23.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8a41573c0 0x7ff8a40ab010 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:23.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8a40acea0 0x7ff8a4156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:23.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 --2- 172.21.15.66:0/1359285400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff888004d10 0x7ff8a41561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:23.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1359285400 >> 172.21.15.66:0/1359285400 conn(0x7ff8a4069f60 msgr2=0x7ff8a406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:23.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1359285400 shutdown_connections 2024-03-30T00:41:23.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.281+0000 7ff8a8e00640 1 -- 172.21.15.66:0/1359285400 wait complete. 2024-03-30T00:41:23.315 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T00:41:23.315 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T00:41:23.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.453+0000 7ff643800640 1 Processor -- start 2024-03-30T00:41:23.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.453+0000 7ff643800640 1 -- start start 2024-03-30T00:41:23.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.453+0000 7ff643800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff63c0cf2f0 0x7ff63c06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:23.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.453+0000 7ff643800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff63c0cb570 0x7ff63c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:23.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.453+0000 7ff643800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63c0cb0e0 con 0x7ff63c15ca80 2024-03-30T00:41:23.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.453+0000 7ff643800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63c0cf7f0 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.453+0000 7ff643800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63c0cfad0 con 0x7ff63c0cb570 2024-03-30T00:41:23.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.453+0000 7ff63be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff63c0cf2f0 0x7ff63c06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:23.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.453+0000 7ff63be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff63c0cf2f0 0x7ff63c06b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57714/0 (socket says 172.21.15.66:57714) 2024-03-30T00:41:23.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.453+0000 7ff63be00640 1 -- 172.21.15.66:0/186591924 learned_addr learned my addr 172.21.15.66:0/186591924 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:23.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.457+0000 7ff641800640 1 --2- 172.21.15.66:0/186591924 >> v2:172.21.15.156:3300/0 conn(0x7ff63c0cb570 0x7ff63c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:23.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.457+0000 7ff63be00640 1 -- 172.21.15.66:0/186591924 >> v2:172.21.15.156:3300/0 conn(0x7ff63c0cb570 msgr2=0x7ff63c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:23.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.457+0000 7ff63be00640 1 --2- 172.21.15.66:0/186591924 >> v2:172.21.15.156:3300/0 conn(0x7ff63c0cb570 0x7ff63c0cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:23.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.457+0000 7ff63be00640 1 -- 172.21.15.66:0/186591924 >> v1:172.21.15.66:6789/0 conn(0x7ff63c15ca80 legacy=0x7ff63c06b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:23.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.457+0000 7ff63be00640 1 -- 172.21.15.66:0/186591924 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c0cad70 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.457+0000 7ff63b400640 1 -- 172.21.15.66:0/186591924 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091281195 0 0) 0x7ff63c0cb0e0 con 0x7ff63c15ca80 2024-03-30T00:41:23.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.457+0000 7ff63be00640 1 --2- 172.21.15.66:0/186591924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff63c0cf2f0 0x7ff63c06b7c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff63c071d80 tx=0x7ff62801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f368fb5114f6e43a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:23.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.457+0000 7ff63b400640 1 -- 172.21.15.66:0/186591924 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280a8a10 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.457+0000 7ff63b400640 1 -- 172.21.15.66:0/186591924 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6280a8b70 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.457+0000 7ff63b400640 1 -- 172.21.15.66:0/186591924 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280b0440 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 -- 172.21.15.66:0/186591924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff63c0cf2f0 msgr2=0x7ff63c06b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 --2- 172.21.15.66:0/186591924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff63c0cf2f0 0x7ff63c06b7c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff63c071d80 tx=0x7ff62801ed10 comp rx=0 tx=0).stop 2024-03-30T00:41:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 -- 172.21.15.66:0/186591924 shutdown_connections 2024-03-30T00:41:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 --2- 172.21.15.66:0/186591924 >> v2:172.21.15.156:3300/0 conn(0x7ff63c0cb570 0x7ff63c0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 --2- 172.21.15.66:0/186591924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff63c0cf2f0 0x7ff63c06b7c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 -- 172.21.15.66:0/186591924 >> 172.21.15.66:0/186591924 conn(0x7ff63c0b2090 msgr2=0x7ff63c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 -- 172.21.15.66:0/186591924 shutdown_connections 2024-03-30T00:41:23.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 -- 172.21.15.66:0/186591924 wait complete. 2024-03-30T00:41:23.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 Processor -- start 2024-03-30T00:41:23.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 -- start start 2024-03-30T00:41:23.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff63c0cb570 0x7ff63c0b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:23.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff63c0cf2f0 0x7ff63c0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:23.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c15ca80 0x7ff63c1f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:23.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c0b4540 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c147730 con 0x7ff63c15ca80 2024-03-30T00:41:23.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff643800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c0b4970 con 0x7ff63c0cb570 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff63be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff63c0cf2f0 0x7ff63c0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff63be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff63c0cf2f0 0x7ff63c0ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57720/0 (socket says 172.21.15.66:57720) 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff63be00640 1 -- 172.21.15.66:0/2833345307 learned_addr learned my addr 172.21.15.66:0/2833345307 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff640e00640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff63c0cb570 0x7ff63c0b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff641800640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c15ca80 0x7ff63c1f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff63be00640 1 -- 172.21.15.66:0/2833345307 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff63c0cb570 msgr2=0x7ff63c0b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff63be00640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff63c0cb570 0x7ff63c0b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff63be00640 1 -- 172.21.15.66:0/2833345307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c15ca80 msgr2=0x7ff63c1f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff63be00640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c15ca80 0x7ff63c1f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff63be00640 1 -- 172.21.15.66:0/2833345307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280094d0 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff640e00640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff63c0cb570 0x7ff63c0b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff641800640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c15ca80 0x7ff63c1f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:23.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.461+0000 7ff63be00640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff63c0cf2f0 0x7ff63c0ca050 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff6280b3040 tx=0x7ff62801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:23.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.465+0000 7ff638c00640 1 -- 172.21.15.66:0/2833345307 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280b0e80 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.465+0000 7ff638c00640 1 -- 172.21.15.66:0/2833345307 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6280bb480 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.465+0000 7ff638c00640 1 -- 172.21.15.66:0/2833345307 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280bb810 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.465+0000 7ff643800640 1 -- 172.21.15.66:0/2833345307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c0ca590 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.465+0000 7ff643800640 1 -- 172.21.15.66:0/2833345307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c0b5df0 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.465+0000 7ff643800640 1 -- 172.21.15.66:0/2833345307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63c066c30 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.465+0000 7ff638c00640 1 -- 172.21.15.66:0/2833345307 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff6280bb970 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.465+0000 7ff638c00640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff61c066690 0x7ff61c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:23.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.465+0000 7ff638c00640 1 -- 172.21.15.66:0/2833345307 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff628122920 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.465+0000 7ff640e00640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff61c066690 0x7ff61c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:23.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.469+0000 7ff640e00640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff61c066690 0x7ff61c068b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f70 tx=0x7ff630009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:23.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.469+0000 7ff638c00640 1 -- 172.21.15.66:0/2833345307 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff6280efb40 con 0x7ff63c0cf2f0 2024-03-30T00:41:23.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:23.669+0000 7ff643800640 1 -- 172.21.15.66:0/2833345307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff63c159030 con 0x7ff63c0cf2f0 2024-03-30T00:41:24.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.281+0000 7ff638c00640 1 -- 172.21.15.66:0/2833345307 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff6280b5020 con 0x7ff63c0cf2f0 2024-03-30T00:41:24.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 -- 172.21.15.66:0/2833345307 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff61c066690 msgr2=0x7ff61c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:24.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff61c066690 0x7ff61c068b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f70 tx=0x7ff630009040 comp rx=0 tx=0).stop 2024-03-30T00:41:24.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 -- 172.21.15.66:0/2833345307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff63c0cf2f0 msgr2=0x7ff63c0ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:24.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff63c0cf2f0 0x7ff63c0ca050 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff6280b3040 tx=0x7ff62801bab0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 -- 172.21.15.66:0/2833345307 shutdown_connections 2024-03-30T00:41:24.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c15ca80 0x7ff63c1f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff61c066690 0x7ff61c068b50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff63c0cf2f0 0x7ff63c0ca050 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 --2- 172.21.15.66:0/2833345307 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff63c0cb570 0x7ff63c0b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 -- 172.21.15.66:0/2833345307 >> 172.21.15.66:0/2833345307 conn(0x7ff63c0b2090 msgr2=0x7ff63c06f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:24.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 -- 172.21.15.66:0/2833345307 shutdown_connections 2024-03-30T00:41:24.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.285+0000 7ff643800640 1 -- 172.21.15.66:0/2833345307 wait complete. 2024-03-30T00:41:24.314 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T00:41:24.314 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:24.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7ec9000640 1 Processor -- start 2024-03-30T00:41:24.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7ec9000640 1 -- start start 2024-03-30T00:41:24.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7ec9000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7ec415a730 0x7f7ec415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7ec9000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7ec40ce330 0x7f7ec40b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7ec9000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec40cbc80 con 0x7f7ec40ce760 2024-03-30T00:41:24.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7ec9000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec40cb100 con 0x7f7ec40ce330 2024-03-30T00:41:24.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7ec9000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec40cb3e0 con 0x7f7ec415a730 2024-03-30T00:41:24.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7eb9400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7ec40ce330 0x7f7ec40b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7ec2800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7ec40ce760 0x7f7ec40cc230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39394/0 (socket says 172.21.15.66:39394) 2024-03-30T00:41:24.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7ec2800640 1 -- 172.21.15.66:0/2961611004 learned_addr learned my addr 172.21.15.66:0/2961611004 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:24.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7ec1e00640 1 --2- 172.21.15.66:0/2961611004 >> v2:172.21.15.156:3300/0 conn(0x7f7ec415a730 0x7f7ec415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7eb9400640 1 -- 172.21.15.66:0/2961611004 >> v2:172.21.15.156:3300/0 conn(0x7f7ec415a730 msgr2=0x7f7ec415e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:24.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7eb9400640 1 --2- 172.21.15.66:0/2961611004 >> v2:172.21.15.156:3300/0 conn(0x7f7ec415a730 0x7f7ec415e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7eb9400640 1 -- 172.21.15.66:0/2961611004 >> v1:172.21.15.66:6789/0 conn(0x7f7ec40ce760 legacy=0x7f7ec40cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:24.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.445+0000 7f7eb9400640 1 -- 172.21.15.66:0/2961611004 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec40cb6c0 con 0x7f7ec40ce330 2024-03-30T00:41:24.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7eb9400640 1 --2- 172.21.15.66:0/2961611004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec40ce330 0x7f7ec40b17b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7ec40cbe90 tx=0x7f7eac01c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54d78d58d3dfbf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:24.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7ec1400640 1 -- 172.21.15.66:0/2961611004 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0ab600 con 0x7f7ec40ce330 2024-03-30T00:41:24.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7ec1400640 1 -- 172.21.15.66:0/2961611004 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac0a7050 con 0x7f7ec40ce330 2024-03-30T00:41:24.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7ec1400640 1 -- 172.21.15.66:0/2961611004 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0ab980 con 0x7f7ec40ce330 2024-03-30T00:41:24.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7ec9000640 1 -- 172.21.15.66:0/2961611004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec40ce330 msgr2=0x7f7ec40b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:24.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7ec9000640 1 --2- 172.21.15.66:0/2961611004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec40ce330 0x7f7ec40b17b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7ec40cbe90 tx=0x7f7eac01c280 comp rx=0 tx=0).stop 2024-03-30T00:41:24.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7ec9000640 1 -- 172.21.15.66:0/2961611004 shutdown_connections 2024-03-30T00:41:24.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7ec9000640 1 --2- 172.21.15.66:0/2961611004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec40ce330 0x7f7ec40b17b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7ec9000640 1 --2- 172.21.15.66:0/2961611004 >> v2:172.21.15.156:3300/0 conn(0x7f7ec415a730 0x7f7ec415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7ec9000640 1 -- 172.21.15.66:0/2961611004 >> 172.21.15.66:0/2961611004 conn(0x7f7ec4098a20 msgr2=0x7f7ec40b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:24.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7ec9000640 1 -- 172.21.15.66:0/2961611004 shutdown_connections 2024-03-30T00:41:24.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.449+0000 7f7ec9000640 1 -- 172.21.15.66:0/2961611004 wait complete. 2024-03-30T00:41:24.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec9000640 1 Processor -- start 2024-03-30T00:41:24.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec9000640 1 -- start start 2024-03-30T00:41:24.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec40ce330 0x7f7ec40b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec9000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec40ce760 0x7f7ec40b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec9000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ec415a730 0x7f7ec41f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec9000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec40b52a0 con 0x7f7ec40ce330 2024-03-30T00:41:24.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec9000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec415d760 con 0x7f7ec415a730 2024-03-30T00:41:24.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec9000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec415da40 con 0x7f7ec40ce760 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec40ce330 0x7f7ec40b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec40ce330 0x7f7ec40b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57730/0 (socket says 172.21.15.66:57730) 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec1e00640 1 -- 172.21.15.66:0/3413066142 learned_addr learned my addr 172.21.15.66:0/3413066142 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec2800640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ec415a730 0x7f7ec41f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7eb9400640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec40ce760 0x7f7ec40b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec1e00640 1 -- 172.21.15.66:0/3413066142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec40ce760 msgr2=0x7f7ec40b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec1e00640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec40ce760 0x7f7ec40b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec1e00640 1 -- 172.21.15.66:0/3413066142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ec415a730 msgr2=0x7f7ec41f5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec1e00640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ec415a730 0x7f7ec41f5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec1e00640 1 -- 172.21.15.66:0/3413066142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac009420 con 0x7f7ec40ce330 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec1e00640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec40ce330 0x7f7ec40b2300 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400d800 tx=0x7f7eb400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ebaa00640 1 -- 172.21.15.66:0/3413066142 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb400b6c0 con 0x7f7ec40ce330 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ebaa00640 1 -- 172.21.15.66:0/3413066142 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb400b820 con 0x7f7ec40ce330 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ebaa00640 1 -- 172.21.15.66:0/3413066142 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb400bb90 con 0x7f7ec40ce330 2024-03-30T00:41:24.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec9000640 1 -- 172.21.15.66:0/3413066142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec40b0150 con 0x7f7ec40ce330 2024-03-30T00:41:24.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.453+0000 7f7ec9000640 1 -- 172.21.15.66:0/3413066142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec40afe40 con 0x7f7ec40ce330 2024-03-30T00:41:24.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.457+0000 7f7ec9000640 1 -- 172.21.15.66:0/3413066142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec40ca350 con 0x7f7ec40ce330 2024-03-30T00:41:24.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.457+0000 7f7ebaa00640 1 -- 172.21.15.66:0/3413066142 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7eb400bcf0 con 0x7f7ec40ce330 2024-03-30T00:41:24.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.457+0000 7f7ebaa00640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea0066690 0x7f7ea0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.457+0000 7f7ebaa00640 1 -- 172.21.15.66:0/3413066142 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7eb40847c0 con 0x7f7ec40ce330 2024-03-30T00:41:24.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.457+0000 7f7eb9400640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea0066690 0x7f7ea0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.457+0000 7f7eb9400640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea0066690 0x7f7ea0068b50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0ad000 tx=0x7f7eac0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:24.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.461+0000 7f7ebaa00640 1 -- 172.21.15.66:0/3413066142 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7eb4051a00 con 0x7f7ec40ce330 2024-03-30T00:41:24.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 -- 172.21.15.66:0/3413066142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ec4066c30 con 0x7f7ec40ce330 2024-03-30T00:41:24.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ebaa00640 1 -- 172.21.15.66:0/3413066142 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f7eb4055020 con 0x7f7ec40ce330 2024-03-30T00:41:24.667 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:24.667 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3895007156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3895007156},{"type":"v1","addr":"172.21.15.156:6835","nonce":3895007156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/65993387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":65993387},{"type":"v1","addr":"172.21.15.156:6834","nonce":65993387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2013605775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2013605775},{"type":"v1","addr":"172.21.15.66:6835","nonce":2013605775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:41:21.269965+0000","modified":"2024-03-30T00:41:24.283171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1694329026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1694329026},{"type":"v1","addr":"172.21.15.66:6837","nonce":1694329026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-30T00:41:24.667 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-03-30T00:41:24.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 -- 172.21.15.66:0/3413066142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea0066690 msgr2=0x7f7ea0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:24.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea0066690 0x7f7ea0068b50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0ad000 tx=0x7f7eac0a6000 comp rx=0 tx=0).stop 2024-03-30T00:41:24.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 -- 172.21.15.66:0/3413066142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec40ce330 msgr2=0x7f7ec40b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:24.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec40ce330 0x7f7ec40b2300 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400d800 tx=0x7f7eb400dcc0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 -- 172.21.15.66:0/3413066142 shutdown_connections 2024-03-30T00:41:24.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ec415a730 0x7f7ec41f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec40ce760 0x7f7ec40b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea0066690 0x7f7ea0068b50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 --2- 172.21.15.66:0/3413066142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec40ce330 0x7f7ec40b2300 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 -- 172.21.15.66:0/3413066142 >> 172.21.15.66:0/3413066142 conn(0x7f7ec4098a20 msgr2=0x7f7ec4159900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:24.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 -- 172.21.15.66:0/3413066142 shutdown_connections 2024-03-30T00:41:24.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.657+0000 7f7ec9000640 1 -- 172.21.15.66:0/3413066142 wait complete. 2024-03-30T00:41:24.688 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:24.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.817+0000 7f9a4c600640 1 Processor -- start 2024-03-30T00:41:24.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.817+0000 7f9a4c600640 1 -- start start 2024-03-30T00:41:24.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.817+0000 7f9a4c600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a4415a810 0x7f9a440ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.817+0000 7f9a4c600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a440cf290 0x7f9a440c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.817+0000 7f9a4c600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a440c4600 con 0x7f9a44099430 2024-03-30T00:41:24.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.817+0000 7f9a4c600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a440cc510 con 0x7f9a4415a810 2024-03-30T00:41:24.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.817+0000 7f9a4c600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a440998c0 con 0x7f9a440cf290 2024-03-30T00:41:24.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a49c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a4415a810 0x7f9a440ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a49c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a4415a810 0x7f9a440ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57742/0 (socket says 172.21.15.66:57742) 2024-03-30T00:41:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a49c00640 1 -- 172.21.15.66:0/777541581 learned_addr learned my addr 172.21.15.66:0/777541581 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a4a600640 1 --2- 172.21.15.66:0/777541581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a440cf290 0x7f9a440c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a49c00640 1 -- 172.21.15.66:0/777541581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a440cf290 msgr2=0x7f9a440c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a49c00640 1 --2- 172.21.15.66:0/777541581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a440cf290 0x7f9a440c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a49c00640 1 -- 172.21.15.66:0/777541581 >> v1:172.21.15.156:6789/0 conn(0x7f9a44099430 legacy=0x7f9a440ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:24.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a49c00640 1 -- 172.21.15.66:0/777541581 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a44099ba0 con 0x7f9a4415a810 2024-03-30T00:41:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a49c00640 1 --2- 172.21.15.66:0/777541581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a4415a810 0x7f9a440ce380 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009870 tx=0x7f9a3801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a462a8b6037447 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a43e00640 1 -- 172.21.15.66:0/777541581 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a380a8a50 con 0x7f9a4415a810 2024-03-30T00:41:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a43e00640 1 -- 172.21.15.66:0/777541581 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a380a8bb0 con 0x7f9a4415a810 2024-03-30T00:41:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.821+0000 7f9a43e00640 1 -- 172.21.15.66:0/777541581 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a380b0440 con 0x7f9a4415a810 2024-03-30T00:41:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 -- 172.21.15.66:0/777541581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a4415a810 msgr2=0x7f9a440ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 --2- 172.21.15.66:0/777541581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a4415a810 0x7f9a440ce380 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009870 tx=0x7f9a3801ed50 comp rx=0 tx=0).stop 2024-03-30T00:41:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 -- 172.21.15.66:0/777541581 shutdown_connections 2024-03-30T00:41:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 --2- 172.21.15.66:0/777541581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a440cf290 0x7f9a440c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 --2- 172.21.15.66:0/777541581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a4415a810 0x7f9a440ce380 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 -- 172.21.15.66:0/777541581 >> 172.21.15.66:0/777541581 conn(0x7f9a440b2090 msgr2=0x7f9a440b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 -- 172.21.15.66:0/777541581 shutdown_connections 2024-03-30T00:41:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 -- 172.21.15.66:0/777541581 wait complete. 2024-03-30T00:41:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 Processor -- start 2024-03-30T00:41:24.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 -- start start 2024-03-30T00:41:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a440cf290 0x7f9a440b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a4415a810 0x7f9a4409cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a4409cff0 0x7f9a4414b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4409e210 con 0x7f9a4415a810 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4409d590 con 0x7f9a440cf290 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4c600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4409d870 con 0x7f9a4409cff0 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a49200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a4415a810 0x7f9a4409cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a49200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a4415a810 0x7f9a4409cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57746/0 (socket says 172.21.15.66:57746) 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a49200640 1 -- 172.21.15.66:0/1411690831 learned_addr learned my addr 172.21.15.66:0/1411690831 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a49c00640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a440cf290 0x7f9a440b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4a600640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a4409cff0 0x7f9a4414b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a49200640 1 -- 172.21.15.66:0/1411690831 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a4409cff0 msgr2=0x7f9a4414b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a49200640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a4409cff0 0x7f9a4414b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a49200640 1 -- 172.21.15.66:0/1411690831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a440cf290 msgr2=0x7f9a440b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a49200640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a440cf290 0x7f9a440b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a49200640 1 -- 172.21.15.66:0/1411690831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a380094d0 con 0x7f9a4415a810 2024-03-30T00:41:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.825+0000 7f9a4a600640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a4409cff0 0x7f9a4414b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.829+0000 7f9a49200640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a4415a810 0x7f9a4409cab0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c500 tx=0x7f9a3000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.829+0000 7f9a41600640 1 -- 172.21.15.66:0/1411690831 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30005220 con 0x7f9a4415a810 2024-03-30T00:41:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.829+0000 7f9a41600640 1 -- 172.21.15.66:0/1411690831 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a30005380 con 0x7f9a4415a810 2024-03-30T00:41:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.829+0000 7f9a41600640 1 -- 172.21.15.66:0/1411690831 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30007270 con 0x7f9a4415a810 2024-03-30T00:41:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.829+0000 7f9a4c600640 1 -- 172.21.15.66:0/1411690831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4409db50 con 0x7f9a4415a810 2024-03-30T00:41:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.829+0000 7f9a4c600640 1 -- 172.21.15.66:0/1411690831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a440b5740 con 0x7f9a4415a810 2024-03-30T00:41:24.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.829+0000 7f9a4c600640 1 -- 172.21.15.66:0/1411690831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a44066c30 con 0x7f9a4415a810 2024-03-30T00:41:24.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.829+0000 7f9a41600640 1 -- 172.21.15.66:0/1411690831 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9a300073d0 con 0x7f9a4415a810 2024-03-30T00:41:24.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.829+0000 7f9a41600640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a240666c0 0x7f9a24068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:24.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.829+0000 7f9a41600640 1 -- 172.21.15.66:0/1411690831 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9a30083af0 con 0x7f9a4415a810 2024-03-30T00:41:24.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.829+0000 7f9a49c00640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a240666c0 0x7f9a24068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:24.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.833+0000 7f9a49c00640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a240666c0 0x7f9a24068b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9a380b2000 tx=0x7f9a38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:24.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:24.833+0000 7f9a41600640 1 -- 172.21.15.66:0/1411690831 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9a30050d30 con 0x7f9a4415a810 2024-03-30T00:41:24.853 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-03-30T00:41:24.845+0000 7f438f400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:41:25.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 -- 172.21.15.66:0/1411690831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a440b3d30 con 0x7f9a4415a810 2024-03-30T00:41:25.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a41600640 1 -- 172.21.15.66:0/1411690831 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f9a30054350 con 0x7f9a4415a810 2024-03-30T00:41:25.011 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:25.011 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":40,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:41:22.272744+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:41:19.292693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T00:41:20.267522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:25.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 -- 172.21.15.66:0/1411690831 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a240666c0 msgr2=0x7f9a24068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a240666c0 0x7f9a24068b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9a380b2000 tx=0x7f9a38009040 comp rx=0 tx=0).stop 2024-03-30T00:41:25.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 -- 172.21.15.66:0/1411690831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a4415a810 msgr2=0x7f9a4409cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a4415a810 0x7f9a4409cab0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c500 tx=0x7f9a3000c9c0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 -- 172.21.15.66:0/1411690831 shutdown_connections 2024-03-30T00:41:25.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a4409cff0 0x7f9a4414b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a240666c0 0x7f9a24068b80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a4415a810 0x7f9a4409cab0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 --2- 172.21.15.66:0/1411690831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a440cf290 0x7f9a440b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 -- 172.21.15.66:0/1411690831 >> 172.21.15.66:0/1411690831 conn(0x7f9a440b2090 msgr2=0x7f9a44098a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:25.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 -- 172.21.15.66:0/1411690831 shutdown_connections 2024-03-30T00:41:25.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.001+0000 7f9a4c600640 1 -- 172.21.15.66:0/1411690831 wait complete. 2024-03-30T00:41:25.032 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T00:41:25.033 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:41:25.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.161+0000 7fce71200640 1 Processor -- start 2024-03-30T00:41:25.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce71200640 1 -- start start 2024-03-30T00:41:25.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce71200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fce6c0cabb0 0x7fce6c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce71200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6c06d070 con 0x7fce6c0ce520 2024-03-30T00:41:25.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce71200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6c0cbba0 con 0x7fce6c15a640 2024-03-30T00:41:25.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce71200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6c0cb8c0 con 0x7fce6c0cabb0 2024-03-30T00:41:25.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce63e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fce6c0ce520 0x7fce6c0ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39406/0 (socket says 172.21.15.66:39406) 2024-03-30T00:41:25.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce63e00640 1 -- 172.21.15.66:0/3109067476 learned_addr learned my addr 172.21.15.66:0/3109067476 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:25.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce6a800640 1 --2- 172.21.15.66:0/3109067476 >> v2:172.21.15.156:3300/0 conn(0x7fce6c0cabb0 0x7fce6c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce69400640 1 -- 172.21.15.66:0/3109067476 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301913032 0 0) 0x7fce6c06d070 con 0x7fce6c0ce520 2024-03-30T00:41:25.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce69400640 1 -- 172.21.15.66:0/3109067476 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce50003530 con 0x7fce6c0ce520 2024-03-30T00:41:25.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce69400640 1 -- 172.21.15.66:0/3109067476 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091871700 0 0) 0x7fce6c0cbba0 con 0x7fce6c15a640 2024-03-30T00:41:25.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce69400640 1 -- 172.21.15.66:0/3109067476 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce6c06d070 con 0x7fce6c15a640 2024-03-30T00:41:25.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce6a800640 1 -- 172.21.15.66:0/3109067476 >> v1:172.21.15.156:6789/0 conn(0x7fce6c15a640 legacy=0x7fce6c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce6a800640 1 -- 172.21.15.66:0/3109067476 >> v1:172.21.15.66:6789/0 conn(0x7fce6c0ce520 legacy=0x7fce6c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.165+0000 7fce6a800640 1 -- 172.21.15.66:0/3109067476 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c06d2a0 con 0x7fce6c0cabb0 2024-03-30T00:41:25.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce6a800640 1 --2- 172.21.15.66:0/3109067476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6c0cabb0 0x7fce6c0cafe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fce64009d90 tx=0x7fce6401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b3f85ddc21645b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:25.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce69400640 1 -- 172.21.15.66:0/3109067476 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce640a5a00 con 0x7fce6c0cabb0 2024-03-30T00:41:25.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce69400640 1 -- 172.21.15.66:0/3109067476 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce640a5b60 con 0x7fce6c0cabb0 2024-03-30T00:41:25.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce69400640 1 -- 172.21.15.66:0/3109067476 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce640ad440 con 0x7fce6c0cabb0 2024-03-30T00:41:25.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce71200640 1 -- 172.21.15.66:0/3109067476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6c0cabb0 msgr2=0x7fce6c0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce71200640 1 --2- 172.21.15.66:0/3109067476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6c0cabb0 0x7fce6c0cafe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fce64009d90 tx=0x7fce6401c940 comp rx=0 tx=0).stop 2024-03-30T00:41:25.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce71200640 1 -- 172.21.15.66:0/3109067476 shutdown_connections 2024-03-30T00:41:25.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce71200640 1 --2- 172.21.15.66:0/3109067476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6c0cabb0 0x7fce6c0cafe0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce71200640 1 -- 172.21.15.66:0/3109067476 >> 172.21.15.66:0/3109067476 conn(0x7fce6c069f60 msgr2=0x7fce6c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:25.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce71200640 1 -- 172.21.15.66:0/3109067476 shutdown_connections 2024-03-30T00:41:25.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce71200640 1 -- 172.21.15.66:0/3109067476 wait complete. 2024-03-30T00:41:25.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.169+0000 7fce71200640 1 Processor -- start 2024-03-30T00:41:25.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce71200640 1 -- start start 2024-03-30T00:41:25.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce71200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6c0cabb0 0x7fce6c09bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce71200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce6c0ce520 0x7fce6c0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce71200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6c15a640 0x7fce6c0c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce71200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c070520 con 0x7fce6c0cabb0 2024-03-30T00:41:25.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce71200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c070240 con 0x7fce6c15a640 2024-03-30T00:41:25.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce71200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0cc3a0 con 0x7fce6c0ce520 2024-03-30T00:41:25.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce69e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6c0cabb0 0x7fce6c09bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce69e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6c0cabb0 0x7fce6c09bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57748/0 (socket says 172.21.15.66:57748) 2024-03-30T00:41:25.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce69e00640 1 -- 172.21.15.66:0/1148337743 learned_addr learned my addr 172.21.15.66:0/1148337743 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:25.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce6a800640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6c15a640 0x7fce6c0c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce69e00640 1 -- 172.21.15.66:0/1148337743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce6c0ce520 msgr2=0x7fce6c0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce63e00640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce6c0ce520 0x7fce6c0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce69e00640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce6c0ce520 0x7fce6c0cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce69e00640 1 -- 172.21.15.66:0/1148337743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6c15a640 msgr2=0x7fce6c0c3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce69e00640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6c15a640 0x7fce6c0c3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce69e00640 1 -- 172.21.15.66:0/1148337743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce64009a70 con 0x7fce6c0cabb0 2024-03-30T00:41:25.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce6a800640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6c15a640 0x7fce6c0c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:25.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce63e00640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce6c0ce520 0x7fce6c0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:25.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce69e00640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6c0cabb0 0x7fce6c09bb10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c430 tx=0x7fce5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:25.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce62000640 1 -- 172.21.15.66:0/1148337743 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c0052d0 con 0x7fce6c0cabb0 2024-03-30T00:41:25.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce62000640 1 -- 172.21.15.66:0/1148337743 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c005450 con 0x7fce6c0cabb0 2024-03-30T00:41:25.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce71200640 1 -- 172.21.15.66:0/1148337743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6c0cc680 con 0x7fce6c0cabb0 2024-03-30T00:41:25.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce62000640 1 -- 172.21.15.66:0/1148337743 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c003840 con 0x7fce6c0cabb0 2024-03-30T00:41:25.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce71200640 1 -- 172.21.15.66:0/1148337743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c06fed0 con 0x7fce6c0cabb0 2024-03-30T00:41:25.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.173+0000 7fce71200640 1 -- 172.21.15.66:0/1148337743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6c066c30 con 0x7fce6c0cabb0 2024-03-30T00:41:25.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.177+0000 7fce62000640 1 -- 172.21.15.66:0/1148337743 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fce5c0039a0 con 0x7fce6c0cabb0 2024-03-30T00:41:25.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.177+0000 7fce62000640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce480666c0 0x7fce48068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.177+0000 7fce62000640 1 -- 172.21.15.66:0/1148337743 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fce5c083a00 con 0x7fce6c0cabb0 2024-03-30T00:41:25.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.177+0000 7fce63e00640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce480666c0 0x7fce48068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.177+0000 7fce63e00640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce480666c0 0x7fce48068b80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0720d0 tx=0x7fce54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:25.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.181+0000 7fce62000640 1 -- 172.21.15.66:0/1148337743 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fce5c050b10 con 0x7fce6c0cabb0 2024-03-30T00:41:25.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.337+0000 7fce71200640 1 -- 172.21.15.66:0/1148337743 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce6c0ab820 con 0x7fce480666c0 2024-03-30T00:41:25.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.345+0000 7fce62000640 1 -- 172.21.15.66:0/1148337743 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289631 (secure 0 0 0) 0x7fce54001400 con 0x7fce480666c0 2024-03-30T00:41:25.359 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:25.361 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 -- 172.21.15.66:0/1148337743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce480666c0 msgr2=0x7fce48068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce480666c0 0x7fce48068b80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0720d0 tx=0x7fce54009040 comp rx=0 tx=0).stop 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 -- 172.21.15.66:0/1148337743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6c0cabb0 msgr2=0x7fce6c09bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6c0cabb0 0x7fce6c09bb10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c430 tx=0x7fce5c00c8f0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 -- 172.21.15.66:0/1148337743 shutdown_connections 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6c15a640 0x7fce6c0c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce6c0ce520 0x7fce6c0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce480666c0 0x7fce48068b80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 --2- 172.21.15.66:0/1148337743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6c0cabb0 0x7fce6c09bb10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 -- 172.21.15.66:0/1148337743 >> 172.21.15.66:0/1148337743 conn(0x7fce6c069f60 msgr2=0x7fce6c06a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 -- 172.21.15.66:0/1148337743 shutdown_connections 2024-03-30T00:41:25.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.349+0000 7fce71200640 1 -- 172.21.15.66:0/1148337743 wait complete. 2024-03-30T00:41:25.380 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-03-30T00:41:24.870224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237352,"kb_used_data":4780,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737368,"statfs":{"total":773094113280,"available":772851064832,"internally_reserved":0,"allocated":4894720,"data_stored":2745304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.009139"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280719+0000","last_change":"2024-03-30T00:41:21.299125+0000","last_active":"2024-03-30T00:41:22.280719+0000","last_peered":"2024-03-30T00:41:22.280719+0000","last_clean":"2024-03-30T00:41:22.280719+0000","last_became_active":"2024-03-30T00:41:21.295699+0000","last_became_peered":"2024-03-30T00:41:21.295699+0000","last_unstale":"2024-03-30T00:41:22.280719+0000","last_undegraded":"2024-03-30T00:41:22.280719+0000","last_fullsized":"2024-03-30T00:41:22.280719+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:09:53.534935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280424+0000","last_change":"2024-03-30T00:41:21.302011+0000","last_active":"2024-03-30T00:41:22.280424+0000","last_peered":"2024-03-30T00:41:22.280424+0000","last_clean":"2024-03-30T00:41:22.280424+0000","last_became_active":"2024-03-30T00:41:21.301698+0000","last_became_peered":"2024-03-30T00:41:21.301698+0000","last_unstale":"2024-03-30T00:41:22.280424+0000","last_undegraded":"2024-03-30T00:41:22.280424+0000","last_fullsized":"2024-03-30T00:41:22.280424+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.993492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.306878+0000","last_change":"2024-03-30T00:41:21.300067+0000","last_active":"2024-03-30T00:41:21.306878+0000","last_peered":"2024-03-30T00:41:21.306878+0000","last_clean":"2024-03-30T00:41:21.306878+0000","last_became_active":"2024-03-30T00:41:21.299822+0000","last_became_peered":"2024-03-30T00:41:21.299822+0000","last_unstale":"2024-03-30T00:41:21.306878+0000","last_undegraded":"2024-03-30T00:41:21.306878+0000","last_fullsized":"2024-03-30T00:41:21.306878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:38.260912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564647+0000","last_change":"2024-03-30T00:41:21.308876+0000","last_active":"2024-03-30T00:41:22.564647+0000","last_peered":"2024-03-30T00:41:22.564647+0000","last_clean":"2024-03-30T00:41:22.564647+0000","last_became_active":"2024-03-30T00:41:21.308646+0000","last_became_peered":"2024-03-30T00:41:21.308646+0000","last_unstale":"2024-03-30T00:41:22.564647+0000","last_undegraded":"2024-03-30T00:41:22.564647+0000","last_fullsized":"2024-03-30T00:41:22.564647+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:16:21.552089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564751+0000","last_change":"2024-03-30T00:41:21.304099+0000","last_active":"2024-03-30T00:41:22.564751+0000","last_peered":"2024-03-30T00:41:22.564751+0000","last_clean":"2024-03-30T00:41:22.564751+0000","last_became_active":"2024-03-30T00:41:21.303866+0000","last_became_peered":"2024-03-30T00:41:21.303866+0000","last_unstale":"2024-03-30T00:41:22.564751+0000","last_undegraded":"2024-03-30T00:41:22.564751+0000","last_fullsized":"2024-03-30T00:41:22.564751+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:58.597464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539688+0000","last_change":"2024-03-30T00:41:21.305574+0000","last_active":"2024-03-30T00:41:22.539688+0000","last_peered":"2024-03-30T00:41:22.539688+0000","last_clean":"2024-03-30T00:41:22.539688+0000","last_became_active":"2024-03-30T00:41:21.304749+0000","last_became_peered":"2024-03-30T00:41:21.304749+0000","last_unstale":"2024-03-30T00:41:22.539688+0000","last_undegraded":"2024-03-30T00:41:22.539688+0000","last_fullsized":"2024-03-30T00:41:22.539688+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:04:31.558432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281029+0000","last_change":"2024-03-30T00:41:21.304184+0000","last_active":"2024-03-30T00:41:22.281029+0000","last_peered":"2024-03-30T00:41:22.281029+0000","last_clean":"2024-03-30T00:41:22.281029+0000","last_became_active":"2024-03-30T00:41:21.303478+0000","last_became_peered":"2024-03-30T00:41:21.303478+0000","last_unstale":"2024-03-30T00:41:22.281029+0000","last_undegraded":"2024-03-30T00:41:22.281029+0000","last_fullsized":"2024-03-30T00:41:22.281029+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:38.260912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.279948+0000","last_change":"2024-03-30T00:41:21.292192+0000","last_active":"2024-03-30T00:41:22.279948+0000","last_peered":"2024-03-30T00:41:22.279948+0000","last_clean":"2024-03-30T00:41:22.279948+0000","last_became_active":"2024-03-30T00:41:21.291955+0000","last_became_peered":"2024-03-30T00:41:21.291955+0000","last_unstale":"2024-03-30T00:41:22.279948+0000","last_undegraded":"2024-03-30T00:41:22.279948+0000","last_fullsized":"2024-03-30T00:41:22.279948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:23:29.452017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.301989+0000","last_change":"2024-03-30T00:41:21.292810+0000","last_active":"2024-03-30T00:41:21.301989+0000","last_peered":"2024-03-30T00:41:21.301989+0000","last_clean":"2024-03-30T00:41:21.301989+0000","last_became_active":"2024-03-30T00:41:21.292574+0000","last_became_peered":"2024-03-30T00:41:21.292574+0000","last_unstale":"2024-03-30T00:41:21.301989+0000","last_undegraded":"2024-03-30T00:41:21.301989+0000","last_fullsized":"2024-03-30T00:41:21.301989+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:48:28.760437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281660+0000","last_change":"2024-03-30T00:41:21.300648+0000","last_active":"2024-03-30T00:41:22.281660+0000","last_peered":"2024-03-30T00:41:22.281660+0000","last_clean":"2024-03-30T00:41:22.281660+0000","last_became_active":"2024-03-30T00:41:21.298814+0000","last_became_peered":"2024-03-30T00:41:21.298814+0000","last_unstale":"2024-03-30T00:41:22.281660+0000","last_undegraded":"2024-03-30T00:41:22.281660+0000","last_fullsized":"2024-03-30T00:41:22.281660+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:47:51.023501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281738+0000","last_change":"2024-03-30T00:41:21.295703+0000","last_active":"2024-03-30T00:41:22.281738+0000","last_peered":"2024-03-30T00:41:22.281738+0000","last_clean":"2024-03-30T00:41:22.281738+0000","last_became_active":"2024-03-30T00:41:21.294095+0000","last_became_peered":"2024-03-30T00:41:21.294095+0000","last_unstale":"2024-03-30T00:41:22.281738+0000","last_undegraded":"2024-03-30T00:41:22.281738+0000","last_fullsized":"2024-03-30T00:41:22.281738+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:09:12.628109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.310174+0000","last_change":"2024-03-30T00:41:21.303726+0000","last_active":"2024-03-30T00:41:21.310174+0000","last_peered":"2024-03-30T00:41:21.310174+0000","last_clean":"2024-03-30T00:41:21.310174+0000","last_became_active":"2024-03-30T00:41:21.303525+0000","last_became_peered":"2024-03-30T00:41:21.303525+0000","last_unstale":"2024-03-30T00:41:21.310174+0000","last_undegraded":"2024-03-30T00:41:21.310174+0000","last_fullsized":"2024-03-30T00:41:21.310174+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:47:51.023501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281345+0000","last_change":"2024-03-30T00:41:21.298639+0000","last_active":"2024-03-30T00:41:22.281345+0000","last_peered":"2024-03-30T00:41:22.281345+0000","last_clean":"2024-03-30T00:41:22.281345+0000","last_became_active":"2024-03-30T00:41:21.297536+0000","last_became_peered":"2024-03-30T00:41:21.297536+0000","last_unstale":"2024-03-30T00:41:22.281345+0000","last_undegraded":"2024-03-30T00:41:22.281345+0000","last_fullsized":"2024-03-30T00:41:22.281345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:56.894917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.306872+0000","last_change":"2024-03-30T00:41:21.298352+0000","last_active":"2024-03-30T00:41:21.306872+0000","last_peered":"2024-03-30T00:41:21.306872+0000","last_clean":"2024-03-30T00:41:21.306872+0000","last_became_active":"2024-03-30T00:41:21.297939+0000","last_became_peered":"2024-03-30T00:41:21.297939+0000","last_unstale":"2024-03-30T00:41:21.306872+0000","last_undegraded":"2024-03-30T00:41:21.306872+0000","last_fullsized":"2024-03-30T00:41:21.306872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:54:01.879198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281902+0000","last_change":"2024-03-30T00:41:21.306401+0000","last_active":"2024-03-30T00:41:22.281902+0000","last_peered":"2024-03-30T00:41:22.281902+0000","last_clean":"2024-03-30T00:41:22.281902+0000","last_became_active":"2024-03-30T00:41:21.306141+0000","last_became_peered":"2024-03-30T00:41:21.306141+0000","last_unstale":"2024-03-30T00:41:22.281902+0000","last_undegraded":"2024-03-30T00:41:22.281902+0000","last_fullsized":"2024-03-30T00:41:22.281902+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:23:29.452017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280824+0000","last_change":"2024-03-30T00:41:21.290189+0000","last_active":"2024-03-30T00:41:22.280824+0000","last_peered":"2024-03-30T00:41:22.280824+0000","last_clean":"2024-03-30T00:41:22.280824+0000","last_became_active":"2024-03-30T00:41:21.289928+0000","last_became_peered":"2024-03-30T00:41:21.289928+0000","last_unstale":"2024-03-30T00:41:22.280824+0000","last_undegraded":"2024-03-30T00:41:22.280824+0000","last_fullsized":"2024-03-30T00:41:22.280824+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:34:18.703475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281764+0000","last_change":"2024-03-30T00:41:21.293202+0000","last_active":"2024-03-30T00:41:22.281764+0000","last_peered":"2024-03-30T00:41:22.281764+0000","last_clean":"2024-03-30T00:41:22.281764+0000","last_became_active":"2024-03-30T00:41:21.292882+0000","last_became_peered":"2024-03-30T00:41:21.292882+0000","last_unstale":"2024-03-30T00:41:22.281764+0000","last_undegraded":"2024-03-30T00:41:22.281764+0000","last_fullsized":"2024-03-30T00:41:22.281764+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:03.515291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.313991+0000","last_change":"2024-03-30T00:41:21.312085+0000","last_active":"2024-03-30T00:41:21.313991+0000","last_peered":"2024-03-30T00:41:21.313991+0000","last_clean":"2024-03-30T00:41:21.313991+0000","last_became_active":"2024-03-30T00:41:21.306396+0000","last_became_peered":"2024-03-30T00:41:21.306396+0000","last_unstale":"2024-03-30T00:41:21.313991+0000","last_undegraded":"2024-03-30T00:41:21.313991+0000","last_fullsized":"2024-03-30T00:41:21.313991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:55.227725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564690+0000","last_change":"2024-03-30T00:41:21.307933+0000","last_active":"2024-03-30T00:41:22.564690+0000","last_peered":"2024-03-30T00:41:22.564690+0000","last_clean":"2024-03-30T00:41:22.564690+0000","last_became_active":"2024-03-30T00:41:21.307551+0000","last_became_peered":"2024-03-30T00:41:21.307551+0000","last_unstale":"2024-03-30T00:41:22.564690+0000","last_undegraded":"2024-03-30T00:41:22.564690+0000","last_fullsized":"2024-03-30T00:41:22.564690+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:02:06.903366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.295928+0000","last_change":"2024-03-30T00:41:21.292507+0000","last_active":"2024-03-30T00:41:21.295928+0000","last_peered":"2024-03-30T00:41:21.295928+0000","last_clean":"2024-03-30T00:41:21.295928+0000","last_became_active":"2024-03-30T00:41:21.292036+0000","last_became_peered":"2024-03-30T00:41:21.292036+0000","last_unstale":"2024-03-30T00:41:21.295928+0000","last_undegraded":"2024-03-30T00:41:21.295928+0000","last_fullsized":"2024-03-30T00:41:21.295928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:58.597464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281335+0000","last_change":"2024-03-30T00:41:21.303867+0000","last_active":"2024-03-30T00:41:22.281335+0000","last_peered":"2024-03-30T00:41:22.281335+0000","last_clean":"2024-03-30T00:41:22.281335+0000","last_became_active":"2024-03-30T00:41:21.301707+0000","last_became_peered":"2024-03-30T00:41:21.301707+0000","last_unstale":"2024-03-30T00:41:22.281335+0000","last_undegraded":"2024-03-30T00:41:22.281335+0000","last_fullsized":"2024-03-30T00:41:22.281335+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:38.690725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.310395+0000","last_change":"2024-03-30T00:41:21.308068+0000","last_active":"2024-03-30T00:41:21.310395+0000","last_peered":"2024-03-30T00:41:21.310395+0000","last_clean":"2024-03-30T00:41:21.310395+0000","last_became_active":"2024-03-30T00:41:21.307511+0000","last_became_peered":"2024-03-30T00:41:21.307511+0000","last_unstale":"2024-03-30T00:41:21.310395+0000","last_undegraded":"2024-03-30T00:41:21.310395+0000","last_fullsized":"2024-03-30T00:41:21.310395+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:53.020603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280942+0000","last_change":"2024-03-30T00:41:21.309056+0000","last_active":"2024-03-30T00:41:22.280942+0000","last_peered":"2024-03-30T00:41:22.280942+0000","last_clean":"2024-03-30T00:41:22.280942+0000","last_became_active":"2024-03-30T00:41:21.308515+0000","last_became_peered":"2024-03-30T00:41:21.308515+0000","last_unstale":"2024-03-30T00:41:22.280942+0000","last_undegraded":"2024-03-30T00:41:22.280942+0000","last_fullsized":"2024-03-30T00:41:22.280942+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:12:15.742797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564531+0000","last_change":"2024-03-30T00:41:21.310278+0000","last_active":"2024-03-30T00:41:22.564531+0000","last_peered":"2024-03-30T00:41:22.564531+0000","last_clean":"2024-03-30T00:41:22.564531+0000","last_became_active":"2024-03-30T00:41:21.310011+0000","last_became_peered":"2024-03-30T00:41:21.310011+0000","last_unstale":"2024-03-30T00:41:22.564531+0000","last_undegraded":"2024-03-30T00:41:22.564531+0000","last_fullsized":"2024-03-30T00:41:22.564531+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:35:02.226502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.313984+0000","last_change":"2024-03-30T00:41:21.312042+0000","last_active":"2024-03-30T00:41:21.313984+0000","last_peered":"2024-03-30T00:41:21.313984+0000","last_clean":"2024-03-30T00:41:21.313984+0000","last_became_active":"2024-03-30T00:41:21.307065+0000","last_became_peered":"2024-03-30T00:41:21.307065+0000","last_unstale":"2024-03-30T00:41:21.313984+0000","last_undegraded":"2024-03-30T00:41:21.313984+0000","last_fullsized":"2024-03-30T00:41:21.313984+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:16:21.552089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539912+0000","last_change":"2024-03-30T00:41:21.297369+0000","last_active":"2024-03-30T00:41:22.539912+0000","last_peered":"2024-03-30T00:41:22.539912+0000","last_clean":"2024-03-30T00:41:22.539912+0000","last_became_active":"2024-03-30T00:41:21.297003+0000","last_became_peered":"2024-03-30T00:41:21.297003+0000","last_unstale":"2024-03-30T00:41:22.539912+0000","last_undegraded":"2024-03-30T00:41:22.539912+0000","last_fullsized":"2024-03-30T00:41:22.539912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:06.394519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.313662+0000","last_change":"2024-03-30T00:41:21.306426+0000","last_active":"2024-03-30T00:41:21.313662+0000","last_peered":"2024-03-30T00:41:21.313662+0000","last_clean":"2024-03-30T00:41:21.313662+0000","last_became_active":"2024-03-30T00:41:21.305854+0000","last_became_peered":"2024-03-30T00:41:21.305854+0000","last_unstale":"2024-03-30T00:41:21.313662+0000","last_undegraded":"2024-03-30T00:41:21.313662+0000","last_fullsized":"2024-03-30T00:41:21.313662+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:35:02.226502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281560+0000","last_change":"2024-03-30T00:41:21.306566+0000","last_active":"2024-03-30T00:41:22.281560+0000","last_peered":"2024-03-30T00:41:22.281560+0000","last_clean":"2024-03-30T00:41:22.281560+0000","last_became_active":"2024-03-30T00:41:21.306309+0000","last_became_peered":"2024-03-30T00:41:21.306309+0000","last_unstale":"2024-03-30T00:41:22.281560+0000","last_undegraded":"2024-03-30T00:41:22.281560+0000","last_fullsized":"2024-03-30T00:41:22.281560+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:53.020603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.306869+0000","last_change":"2024-03-30T00:41:21.305045+0000","last_active":"2024-03-30T00:41:21.306869+0000","last_peered":"2024-03-30T00:41:21.306869+0000","last_clean":"2024-03-30T00:41:21.306869+0000","last_became_active":"2024-03-30T00:41:21.304784+0000","last_became_peered":"2024-03-30T00:41:21.304784+0000","last_unstale":"2024-03-30T00:41:21.306869+0000","last_undegraded":"2024-03-30T00:41:21.306869+0000","last_fullsized":"2024-03-30T00:41:21.306869+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:02:06.903366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.313998+0000","last_change":"2024-03-30T00:41:21.306697+0000","last_active":"2024-03-30T00:41:21.313998+0000","last_peered":"2024-03-30T00:41:21.313998+0000","last_clean":"2024-03-30T00:41:21.313998+0000","last_became_active":"2024-03-30T00:41:21.306126+0000","last_became_peered":"2024-03-30T00:41:21.306126+0000","last_unstale":"2024-03-30T00:41:21.313998+0000","last_undegraded":"2024-03-30T00:41:21.313998+0000","last_fullsized":"2024-03-30T00:41:21.313998+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:35:11.629197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281052+0000","last_change":"2024-03-30T00:41:21.305010+0000","last_active":"2024-03-30T00:41:22.281052+0000","last_peered":"2024-03-30T00:41:22.281052+0000","last_clean":"2024-03-30T00:41:22.281052+0000","last_became_active":"2024-03-30T00:41:21.304743+0000","last_became_peered":"2024-03-30T00:41:21.304743+0000","last_unstale":"2024-03-30T00:41:22.281052+0000","last_undegraded":"2024-03-30T00:41:22.281052+0000","last_fullsized":"2024-03-30T00:41:22.281052+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:08:08.740824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539695+0000","last_change":"2024-03-30T00:41:21.299911+0000","last_active":"2024-03-30T00:41:22.539695+0000","last_peered":"2024-03-30T00:41:22.539695+0000","last_clean":"2024-03-30T00:41:22.539695+0000","last_became_active":"2024-03-30T00:41:21.299722+0000","last_became_peered":"2024-03-30T00:41:21.299722+0000","last_unstale":"2024-03-30T00:41:22.539695+0000","last_undegraded":"2024-03-30T00:41:22.539695+0000","last_fullsized":"2024-03-30T00:41:22.539695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:08:08.740824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281874+0000","last_change":"2024-03-30T00:41:21.289247+0000","last_active":"2024-03-30T00:41:22.281874+0000","last_peered":"2024-03-30T00:41:22.281874+0000","last_clean":"2024-03-30T00:41:22.281874+0000","last_became_active":"2024-03-30T00:41:21.288925+0000","last_became_peered":"2024-03-30T00:41:21.288925+0000","last_unstale":"2024-03-30T00:41:22.281874+0000","last_undegraded":"2024-03-30T00:41:22.281874+0000","last_fullsized":"2024-03-30T00:41:22.281874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:27:46.606824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.313520+0000","last_change":"2024-03-30T00:41:20.888469+0000","last_active":"2024-03-30T00:41:21.313520+0000","last_peered":"2024-03-30T00:41:21.313520+0000","last_clean":"2024-03-30T00:41:21.313520+0000","last_became_active":"2024-03-30T00:41:20.888214+0000","last_became_peered":"2024-03-30T00:41:20.888214+0000","last_unstale":"2024-03-30T00:41:21.313520+0000","last_undegraded":"2024-03-30T00:41:21.313520+0000","last_fullsized":"2024-03-30T00:41:21.313520+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:05:06.481400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539911+0000","last_change":"2024-03-30T00:41:21.306416+0000","last_active":"2024-03-30T00:41:22.539911+0000","last_peered":"2024-03-30T00:41:22.539911+0000","last_clean":"2024-03-30T00:41:22.539911+0000","last_became_active":"2024-03-30T00:41:21.306095+0000","last_became_peered":"2024-03-30T00:41:21.306095+0000","last_unstale":"2024-03-30T00:41:22.539911+0000","last_undegraded":"2024-03-30T00:41:22.539911+0000","last_fullsized":"2024-03-30T00:41:22.539911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:25:48.460169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.302607+0000","last_change":"2024-03-30T00:41:20.888714+0000","last_active":"2024-03-30T00:41:21.302607+0000","last_peered":"2024-03-30T00:41:21.302607+0000","last_clean":"2024-03-30T00:41:21.302607+0000","last_became_active":"2024-03-30T00:41:20.888426+0000","last_became_peered":"2024-03-30T00:41:20.888426+0000","last_unstale":"2024-03-30T00:41:21.302607+0000","last_undegraded":"2024-03-30T00:41:21.302607+0000","last_fullsized":"2024-03-30T00:41:21.302607+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:08:07.735075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.310229+0000","last_change":"2024-03-30T00:41:21.301502+0000","last_active":"2024-03-30T00:41:21.310229+0000","last_peered":"2024-03-30T00:41:21.310229+0000","last_clean":"2024-03-30T00:41:21.310229+0000","last_became_active":"2024-03-30T00:41:21.300775+0000","last_became_peered":"2024-03-30T00:41:21.300775+0000","last_unstale":"2024-03-30T00:41:21.310229+0000","last_undegraded":"2024-03-30T00:41:21.310229+0000","last_fullsized":"2024-03-30T00:41:21.310229+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:38.260912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539985+0000","last_change":"2024-03-30T00:41:20.884939+0000","last_active":"2024-03-30T00:41:22.539985+0000","last_peered":"2024-03-30T00:41:22.539985+0000","last_clean":"2024-03-30T00:41:22.539985+0000","last_became_active":"2024-03-30T00:41:20.884688+0000","last_became_peered":"2024-03-30T00:41:20.884688+0000","last_unstale":"2024-03-30T00:41:22.539985+0000","last_undegraded":"2024-03-30T00:41:22.539985+0000","last_fullsized":"2024-03-30T00:41:22.539985+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:35:10.623449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280634+0000","last_change":"2024-03-30T00:41:21.295354+0000","last_active":"2024-03-30T00:41:22.280634+0000","last_peered":"2024-03-30T00:41:22.280634+0000","last_clean":"2024-03-30T00:41:22.280634+0000","last_became_active":"2024-03-30T00:41:21.294567+0000","last_became_peered":"2024-03-30T00:41:21.294567+0000","last_unstale":"2024-03-30T00:41:22.280634+0000","last_undegraded":"2024-03-30T00:41:22.280634+0000","last_fullsized":"2024-03-30T00:41:22.280634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:12:19.195955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.314018+0000","last_change":"2024-03-30T00:41:20.879477+0000","last_active":"2024-03-30T00:41:21.314018+0000","last_peered":"2024-03-30T00:41:21.314018+0000","last_clean":"2024-03-30T00:41:21.314018+0000","last_became_active":"2024-03-30T00:41:20.879219+0000","last_became_peered":"2024-03-30T00:41:20.879219+0000","last_unstale":"2024-03-30T00:41:21.314018+0000","last_undegraded":"2024-03-30T00:41:21.314018+0000","last_fullsized":"2024-03-30T00:41:21.314018+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:05.388771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.307011+0000","last_change":"2024-03-30T00:41:21.301286+0000","last_active":"2024-03-30T00:41:21.307011+0000","last_peered":"2024-03-30T00:41:21.307011+0000","last_clean":"2024-03-30T00:41:21.307011+0000","last_became_active":"2024-03-30T00:41:21.300413+0000","last_became_peered":"2024-03-30T00:41:21.300413+0000","last_unstale":"2024-03-30T00:41:21.307011+0000","last_undegraded":"2024-03-30T00:41:21.307011+0000","last_fullsized":"2024-03-30T00:41:21.307011+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:54:01.879198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281498+0000","last_change":"2024-03-30T00:41:20.890094+0000","last_active":"2024-03-30T00:41:22.281498+0000","last_peered":"2024-03-30T00:41:22.281498+0000","last_clean":"2024-03-30T00:41:22.281498+0000","last_became_active":"2024-03-30T00:41:20.889838+0000","last_became_peered":"2024-03-30T00:41:20.889838+0000","last_unstale":"2024-03-30T00:41:22.281498+0000","last_undegraded":"2024-03-30T00:41:22.281498+0000","last_fullsized":"2024-03-30T00:41:22.281498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:10:00.616749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281622+0000","last_change":"2024-03-30T00:41:21.305058+0000","last_active":"2024-03-30T00:41:22.281622+0000","last_peered":"2024-03-30T00:41:22.281622+0000","last_clean":"2024-03-30T00:41:22.281622+0000","last_became_active":"2024-03-30T00:41:21.303926+0000","last_became_peered":"2024-03-30T00:41:21.303926+0000","last_unstale":"2024-03-30T00:41:22.281622+0000","last_undegraded":"2024-03-30T00:41:22.281622+0000","last_fullsized":"2024-03-30T00:41:22.281622+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:47.922056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281409+0000","last_change":"2024-03-30T00:41:20.887680+0000","last_active":"2024-03-30T00:41:22.281409+0000","last_peered":"2024-03-30T00:41:22.281409+0000","last_clean":"2024-03-30T00:41:22.281409+0000","last_became_active":"2024-03-30T00:41:20.887417+0000","last_became_peered":"2024-03-30T00:41:20.887417+0000","last_unstale":"2024-03-30T00:41:22.281409+0000","last_undegraded":"2024-03-30T00:41:22.281409+0000","last_fullsized":"2024-03-30T00:41:22.281409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:35:01.220754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.309392+0000","last_change":"2024-03-30T00:41:21.300301+0000","last_active":"2024-03-30T00:41:21.309392+0000","last_peered":"2024-03-30T00:41:21.309392+0000","last_clean":"2024-03-30T00:41:21.309392+0000","last_became_active":"2024-03-30T00:41:21.300026+0000","last_became_peered":"2024-03-30T00:41:21.300026+0000","last_unstale":"2024-03-30T00:41:21.309392+0000","last_undegraded":"2024-03-30T00:41:21.309392+0000","last_fullsized":"2024-03-30T00:41:21.309392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:47:51.023501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539682+0000","last_change":"2024-03-30T00:41:20.887845+0000","last_active":"2024-03-30T00:41:22.539682+0000","last_peered":"2024-03-30T00:41:22.539682+0000","last_clean":"2024-03-30T00:41:22.539682+0000","last_became_active":"2024-03-30T00:41:20.887534+0000","last_became_peered":"2024-03-30T00:41:20.887534+0000","last_unstale":"2024-03-30T00:41:22.539682+0000","last_undegraded":"2024-03-30T00:41:22.539682+0000","last_fullsized":"2024-03-30T00:41:22.539682+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:54.221977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280222+0000","last_change":"2024-03-30T00:41:21.282088+0000","last_active":"2024-03-30T00:41:22.280222+0000","last_peered":"2024-03-30T00:41:22.280222+0000","last_clean":"2024-03-30T00:41:22.280222+0000","last_became_active":"2024-03-30T00:41:21.281824+0000","last_became_peered":"2024-03-30T00:41:21.281824+0000","last_unstale":"2024-03-30T00:41:22.280222+0000","last_undegraded":"2024-03-30T00:41:22.280222+0000","last_fullsized":"2024-03-30T00:41:22.280222+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:49.423423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.278557+0000","last_change":"2024-03-30T00:41:20.889020+0000","last_active":"2024-03-30T00:41:21.278557+0000","last_peered":"2024-03-30T00:41:21.278557+0000","last_clean":"2024-03-30T00:41:21.278557+0000","last_became_active":"2024-03-30T00:41:20.888715+0000","last_became_peered":"2024-03-30T00:41:20.888715+0000","last_unstale":"2024-03-30T00:41:21.278557+0000","last_undegraded":"2024-03-30T00:41:21.278557+0000","last_fullsized":"2024-03-30T00:41:21.278557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:25:47.454420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564708+0000","last_change":"2024-03-30T00:41:21.308399+0000","last_active":"2024-03-30T00:41:22.564708+0000","last_peered":"2024-03-30T00:41:22.564708+0000","last_clean":"2024-03-30T00:41:22.564708+0000","last_became_active":"2024-03-30T00:41:21.308159+0000","last_became_peered":"2024-03-30T00:41:21.308159+0000","last_unstale":"2024-03-30T00:41:22.564708+0000","last_undegraded":"2024-03-30T00:41:22.564708+0000","last_fullsized":"2024-03-30T00:41:22.564708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:55.227725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280798+0000","last_change":"2024-03-30T00:41:20.892397+0000","last_active":"2024-03-30T00:41:22.280798+0000","last_peered":"2024-03-30T00:41:22.280798+0000","last_clean":"2024-03-30T00:41:22.280798+0000","last_became_active":"2024-03-30T00:41:20.891100+0000","last_became_peered":"2024-03-30T00:41:20.891100+0000","last_unstale":"2024-03-30T00:41:22.280798+0000","last_undegraded":"2024-03-30T00:41:22.280798+0000","last_fullsized":"2024-03-30T00:41:22.280798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:14.578964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281718+0000","last_change":"2024-03-30T00:41:21.296856+0000","last_active":"2024-03-30T00:41:22.281718+0000","last_peered":"2024-03-30T00:41:22.281718+0000","last_clean":"2024-03-30T00:41:22.281718+0000","last_became_active":"2024-03-30T00:41:21.296608+0000","last_became_peered":"2024-03-30T00:41:21.296608+0000","last_unstale":"2024-03-30T00:41:22.281718+0000","last_undegraded":"2024-03-30T00:41:22.281718+0000","last_fullsized":"2024-03-30T00:41:22.281718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:33.233464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.282000+0000","last_change":"2024-03-30T00:41:20.882807+0000","last_active":"2024-03-30T00:41:22.282000+0000","last_peered":"2024-03-30T00:41:22.282000+0000","last_clean":"2024-03-30T00:41:22.282000+0000","last_became_active":"2024-03-30T00:41:20.882574+0000","last_became_peered":"2024-03-30T00:41:20.882574+0000","last_unstale":"2024-03-30T00:41:22.282000+0000","last_undegraded":"2024-03-30T00:41:22.282000+0000","last_fullsized":"2024-03-30T00:41:22.282000+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:04:30.552684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.306568+0000","last_change":"2024-03-30T00:41:21.297024+0000","last_active":"2024-03-30T00:41:21.306568+0000","last_peered":"2024-03-30T00:41:21.306568+0000","last_clean":"2024-03-30T00:41:21.306568+0000","last_became_active":"2024-03-30T00:41:21.296624+0000","last_became_peered":"2024-03-30T00:41:21.296624+0000","last_unstale":"2024-03-30T00:41:21.306568+0000","last_undegraded":"2024-03-30T00:41:21.306568+0000","last_fullsized":"2024-03-30T00:41:21.306568+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:56.894917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281855+0000","last_change":"2024-03-30T00:41:20.892928+0000","last_active":"2024-03-30T00:41:22.281855+0000","last_peered":"2024-03-30T00:41:22.281855+0000","last_clean":"2024-03-30T00:41:22.281855+0000","last_became_active":"2024-03-30T00:41:20.892662+0000","last_became_peered":"2024-03-30T00:41:20.892662+0000","last_unstale":"2024-03-30T00:41:22.281855+0000","last_undegraded":"2024-03-30T00:41:22.281855+0000","last_fullsized":"2024-03-30T00:41:22.281855+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:13.057222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.309932+0000","last_change":"2024-03-30T00:41:21.303722+0000","last_active":"2024-03-30T00:41:21.309932+0000","last_peered":"2024-03-30T00:41:21.309932+0000","last_clean":"2024-03-30T00:41:21.309932+0000","last_became_active":"2024-03-30T00:41:21.303414+0000","last_became_peered":"2024-03-30T00:41:21.303414+0000","last_unstale":"2024-03-30T00:41:21.309932+0000","last_undegraded":"2024-03-30T00:41:21.309932+0000","last_fullsized":"2024-03-30T00:41:21.309932+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:31:06.890668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280414+0000","last_change":"2024-03-30T00:41:20.887083+0000","last_active":"2024-03-30T00:41:22.280414+0000","last_peered":"2024-03-30T00:41:22.280414+0000","last_clean":"2024-03-30T00:41:22.280414+0000","last_became_active":"2024-03-30T00:41:20.886886+0000","last_became_peered":"2024-03-30T00:41:20.886886+0000","last_unstale":"2024-03-30T00:41:22.280414+0000","last_undegraded":"2024-03-30T00:41:22.280414+0000","last_fullsized":"2024-03-30T00:41:22.280414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:27:45.601075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.301369+0000","last_change":"2024-03-30T00:41:21.292119+0000","last_active":"2024-03-30T00:41:21.301369+0000","last_peered":"2024-03-30T00:41:21.301369+0000","last_clean":"2024-03-30T00:41:21.301369+0000","last_became_active":"2024-03-30T00:41:21.291836+0000","last_became_peered":"2024-03-30T00:41:21.291836+0000","last_unstale":"2024-03-30T00:41:21.301369+0000","last_undegraded":"2024-03-30T00:41:21.301369+0000","last_fullsized":"2024-03-30T00:41:21.301369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:47.922056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280353+0000","last_change":"2024-03-30T00:41:20.887809+0000","last_active":"2024-03-30T00:41:22.280353+0000","last_peered":"2024-03-30T00:41:22.280353+0000","last_clean":"2024-03-30T00:41:22.280353+0000","last_became_active":"2024-03-30T00:41:20.887603+0000","last_became_peered":"2024-03-30T00:41:20.887603+0000","last_unstale":"2024-03-30T00:41:22.280353+0000","last_undegraded":"2024-03-30T00:41:22.280353+0000","last_fullsized":"2024-03-30T00:41:22.280353+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:32.227716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280081+0000","last_change":"2024-03-30T00:41:21.293830+0000","last_active":"2024-03-30T00:41:22.280081+0000","last_peered":"2024-03-30T00:41:22.280081+0000","last_clean":"2024-03-30T00:41:22.280081+0000","last_became_active":"2024-03-30T00:41:21.293613+0000","last_became_peered":"2024-03-30T00:41:21.293613+0000","last_unstale":"2024-03-30T00:41:22.280081+0000","last_undegraded":"2024-03-30T00:41:22.280081+0000","last_fullsized":"2024-03-30T00:41:22.280081+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:33.066405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539677+0000","last_change":"2024-03-30T00:41:20.887853+0000","last_active":"2024-03-30T00:41:22.539677+0000","last_peered":"2024-03-30T00:41:22.539677+0000","last_clean":"2024-03-30T00:41:22.539677+0000","last_became_active":"2024-03-30T00:41:20.887552+0000","last_became_peered":"2024-03-30T00:41:20.887552+0000","last_unstale":"2024-03-30T00:41:22.539677+0000","last_undegraded":"2024-03-30T00:41:22.539677+0000","last_fullsized":"2024-03-30T00:41:22.539677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:28:56.650073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281072+0000","last_change":"2024-03-30T00:41:21.308393+0000","last_active":"2024-03-30T00:41:22.281072+0000","last_peered":"2024-03-30T00:41:22.281072+0000","last_clean":"2024-03-30T00:41:22.281072+0000","last_became_active":"2024-03-30T00:41:21.308109+0000","last_became_peered":"2024-03-30T00:41:21.308109+0000","last_unstale":"2024-03-30T00:41:22.281072+0000","last_undegraded":"2024-03-30T00:41:22.281072+0000","last_fullsized":"2024-03-30T00:41:22.281072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:06.394519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.292047+0000","last_change":"2024-03-30T00:41:20.883666+0000","last_active":"2024-03-30T00:41:21.292047+0000","last_peered":"2024-03-30T00:41:21.292047+0000","last_clean":"2024-03-30T00:41:21.292047+0000","last_became_active":"2024-03-30T00:41:20.881887+0000","last_became_peered":"2024-03-30T00:41:20.881887+0000","last_unstale":"2024-03-30T00:41:21.292047+0000","last_undegraded":"2024-03-30T00:41:21.292047+0000","last_fullsized":"2024-03-30T00:41:21.292047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:38:16.735996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280216+0000","last_change":"2024-03-30T00:41:21.295992+0000","last_active":"2024-03-30T00:41:22.280216+0000","last_peered":"2024-03-30T00:41:22.280216+0000","last_clean":"2024-03-30T00:41:22.280216+0000","last_became_active":"2024-03-30T00:41:21.295733+0000","last_became_peered":"2024-03-30T00:41:21.295733+0000","last_unstale":"2024-03-30T00:41:22.280216+0000","last_undegraded":"2024-03-30T00:41:22.280216+0000","last_fullsized":"2024-03-30T00:41:22.280216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:25:30.185412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280925+0000","last_change":"2024-03-30T00:41:20.882647+0000","last_active":"2024-03-30T00:41:22.280925+0000","last_peered":"2024-03-30T00:41:22.280925+0000","last_clean":"2024-03-30T00:41:22.280925+0000","last_became_active":"2024-03-30T00:41:20.882388+0000","last_became_peered":"2024-03-30T00:41:20.882388+0000","last_unstale":"2024-03-30T00:41:22.280925+0000","last_undegraded":"2024-03-30T00:41:22.280925+0000","last_fullsized":"2024-03-30T00:41:22.280925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:57.591715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280388+0000","last_change":"2024-03-30T00:41:21.293319+0000","last_active":"2024-03-30T00:41:22.280388+0000","last_peered":"2024-03-30T00:41:22.280388+0000","last_clean":"2024-03-30T00:41:22.280388+0000","last_became_active":"2024-03-30T00:41:21.293089+0000","last_became_peered":"2024-03-30T00:41:21.293089+0000","last_unstale":"2024-03-30T00:41:22.280388+0000","last_undegraded":"2024-03-30T00:41:22.280388+0000","last_fullsized":"2024-03-30T00:41:22.280388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:54:42.854939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281873+0000","last_change":"2024-03-30T00:41:20.891763+0000","last_active":"2024-03-30T00:41:22.281873+0000","last_peered":"2024-03-30T00:41:22.281873+0000","last_clean":"2024-03-30T00:41:22.281873+0000","last_became_active":"2024-03-30T00:41:20.891514+0000","last_became_peered":"2024-03-30T00:41:20.891514+0000","last_unstale":"2024-03-30T00:41:22.281873+0000","last_undegraded":"2024-03-30T00:41:22.281873+0000","last_fullsized":"2024-03-30T00:41:22.281873+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:56:12.781847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281801+0000","last_change":"2024-03-30T00:41:21.305382+0000","last_active":"2024-03-30T00:41:22.281801+0000","last_peered":"2024-03-30T00:41:22.281801+0000","last_clean":"2024-03-30T00:41:22.281801+0000","last_became_active":"2024-03-30T00:41:21.302669+0000","last_became_peered":"2024-03-30T00:41:21.302669+0000","last_unstale":"2024-03-30T00:41:22.281801+0000","last_undegraded":"2024-03-30T00:41:22.281801+0000","last_fullsized":"2024-03-30T00:41:22.281801+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:48:28.760437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.311624+0000","last_change":"2024-03-30T00:41:20.891669+0000","last_active":"2024-03-30T00:41:21.311624+0000","last_peered":"2024-03-30T00:41:21.311624+0000","last_clean":"2024-03-30T00:41:21.311624+0000","last_became_active":"2024-03-30T00:41:20.891408+0000","last_became_peered":"2024-03-30T00:41:20.891408+0000","last_unstale":"2024-03-30T00:41:21.311624+0000","last_undegraded":"2024-03-30T00:41:21.311624+0000","last_fullsized":"2024-03-30T00:41:21.311624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:13.057222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.310247+0000","last_change":"2024-03-30T00:41:21.301706+0000","last_active":"2024-03-30T00:41:21.310247+0000","last_peered":"2024-03-30T00:41:21.310247+0000","last_clean":"2024-03-30T00:41:21.310247+0000","last_became_active":"2024-03-30T00:41:21.301259+0000","last_became_peered":"2024-03-30T00:41:21.301259+0000","last_unstale":"2024-03-30T00:41:21.310247+0000","last_undegraded":"2024-03-30T00:41:21.310247+0000","last_fullsized":"2024-03-30T00:41:21.310247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:56.894917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564812+0000","last_change":"2024-03-30T00:41:20.883861+0000","last_active":"2024-03-30T00:41:22.564812+0000","last_peered":"2024-03-30T00:41:22.564812+0000","last_clean":"2024-03-30T00:41:22.564812+0000","last_became_active":"2024-03-30T00:41:20.882981+0000","last_became_peered":"2024-03-30T00:41:20.882981+0000","last_unstale":"2024-03-30T00:41:22.564812+0000","last_undegraded":"2024-03-30T00:41:22.564812+0000","last_fullsized":"2024-03-30T00:41:22.564812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:16.975551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564840+0000","last_change":"2024-03-30T00:41:21.312004+0000","last_active":"2024-03-30T00:41:22.564840+0000","last_peered":"2024-03-30T00:41:22.564840+0000","last_clean":"2024-03-30T00:41:22.564840+0000","last_became_active":"2024-03-30T00:41:21.311366+0000","last_became_peered":"2024-03-30T00:41:21.311366+0000","last_unstale":"2024-03-30T00:41:22.564840+0000","last_undegraded":"2024-03-30T00:41:22.564840+0000","last_fullsized":"2024-03-30T00:41:22.564840+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:35:11.629197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539817+0000","last_change":"2024-03-30T00:41:20.892029+0000","last_active":"2024-03-30T00:41:22.539817+0000","last_peered":"2024-03-30T00:41:22.539817+0000","last_clean":"2024-03-30T00:41:22.539817+0000","last_became_active":"2024-03-30T00:41:20.891783+0000","last_became_peered":"2024-03-30T00:41:20.891783+0000","last_unstale":"2024-03-30T00:41:22.539817+0000","last_undegraded":"2024-03-30T00:41:22.539817+0000","last_fullsized":"2024-03-30T00:41:22.539817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:16:20.546340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281063+0000","last_change":"2024-03-30T00:41:21.290167+0000","last_active":"2024-03-30T00:41:22.281063+0000","last_peered":"2024-03-30T00:41:22.281063+0000","last_clean":"2024-03-30T00:41:22.281063+0000","last_became_active":"2024-03-30T00:41:21.289571+0000","last_became_peered":"2024-03-30T00:41:21.289571+0000","last_unstale":"2024-03-30T00:41:22.281063+0000","last_undegraded":"2024-03-30T00:41:22.281063+0000","last_fullsized":"2024-03-30T00:41:22.281063+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:25:48.460169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564735+0000","last_change":"2024-03-30T00:41:20.889180+0000","last_active":"2024-03-30T00:41:22.564735+0000","last_peered":"2024-03-30T00:41:22.564735+0000","last_clean":"2024-03-30T00:41:22.564735+0000","last_became_active":"2024-03-30T00:41:20.888914+0000","last_became_peered":"2024-03-30T00:41:20.888914+0000","last_unstale":"2024-03-30T00:41:22.564735+0000","last_undegraded":"2024-03-30T00:41:22.564735+0000","last_fullsized":"2024-03-30T00:41:22.564735+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:05:06.481400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.306378+0000","last_change":"2024-03-30T00:41:21.296641+0000","last_active":"2024-03-30T00:41:21.306378+0000","last_peered":"2024-03-30T00:41:21.306378+0000","last_clean":"2024-03-30T00:41:21.306378+0000","last_became_active":"2024-03-30T00:41:21.296137+0000","last_became_peered":"2024-03-30T00:41:21.296137+0000","last_unstale":"2024-03-30T00:41:21.306378+0000","last_undegraded":"2024-03-30T00:41:21.306378+0000","last_fullsized":"2024-03-30T00:41:21.306378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:14.043465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564566+0000","last_change":"2024-03-30T00:41:20.880206+0000","last_active":"2024-03-30T00:41:22.564566+0000","last_peered":"2024-03-30T00:41:22.564566+0000","last_clean":"2024-03-30T00:41:22.564566+0000","last_became_active":"2024-03-30T00:41:20.879909+0000","last_became_peered":"2024-03-30T00:41:20.879909+0000","last_unstale":"2024-03-30T00:41:22.564566+0000","last_undegraded":"2024-03-30T00:41:22.564566+0000","last_fullsized":"2024-03-30T00:41:22.564566+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:38:16.735996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281336+0000","last_change":"2024-03-30T00:41:21.296861+0000","last_active":"2024-03-30T00:41:22.281336+0000","last_peered":"2024-03-30T00:41:22.281336+0000","last_clean":"2024-03-30T00:41:22.281336+0000","last_became_active":"2024-03-30T00:41:21.296531+0000","last_became_peered":"2024-03-30T00:41:21.296531+0000","last_unstale":"2024-03-30T00:41:22.281336+0000","last_undegraded":"2024-03-30T00:41:22.281336+0000","last_fullsized":"2024-03-30T00:41:22.281336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:54:01.879198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281142+0000","last_change":"2024-03-30T00:41:20.885091+0000","last_active":"2024-03-30T00:41:22.281142+0000","last_peered":"2024-03-30T00:41:22.281142+0000","last_clean":"2024-03-30T00:41:22.281142+0000","last_became_active":"2024-03-30T00:41:20.884826+0000","last_became_peered":"2024-03-30T00:41:20.884826+0000","last_unstale":"2024-03-30T00:41:22.281142+0000","last_undegraded":"2024-03-30T00:41:22.281142+0000","last_fullsized":"2024-03-30T00:41:22.281142+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:02:05.897618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280326+0000","last_change":"2024-03-30T00:41:21.284469+0000","last_active":"2024-03-30T00:41:22.280326+0000","last_peered":"2024-03-30T00:41:22.280326+0000","last_clean":"2024-03-30T00:41:22.280326+0000","last_became_active":"2024-03-30T00:41:21.284184+0000","last_became_peered":"2024-03-30T00:41:21.284184+0000","last_unstale":"2024-03-30T00:41:22.280326+0000","last_undegraded":"2024-03-30T00:41:22.280326+0000","last_fullsized":"2024-03-30T00:41:22.280326+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:38.690725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564634+0000","last_change":"2024-03-30T00:41:20.886308+0000","last_active":"2024-03-30T00:41:22.564634+0000","last_peered":"2024-03-30T00:41:22.564634+0000","last_clean":"2024-03-30T00:41:22.564634+0000","last_became_active":"2024-03-30T00:41:20.886023+0000","last_became_peered":"2024-03-30T00:41:20.886023+0000","last_unstale":"2024-03-30T00:41:22.564634+0000","last_undegraded":"2024-03-30T00:41:22.564634+0000","last_fullsized":"2024-03-30T00:41:22.564634+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:56.730386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539826+0000","last_change":"2024-03-30T00:41:21.298469+0000","last_active":"2024-03-30T00:41:22.539826+0000","last_peered":"2024-03-30T00:41:22.539826+0000","last_clean":"2024-03-30T00:41:22.539826+0000","last_became_active":"2024-03-30T00:41:21.298245+0000","last_became_peered":"2024-03-30T00:41:21.298245+0000","last_unstale":"2024-03-30T00:41:22.539826+0000","last_undegraded":"2024-03-30T00:41:22.539826+0000","last_fullsized":"2024-03-30T00:41:22.539826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:34:18.703475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281836+0000","last_change":"2024-03-30T00:41:20.892935+0000","last_active":"2024-03-30T00:41:22.281836+0000","last_peered":"2024-03-30T00:41:22.281836+0000","last_clean":"2024-03-30T00:41:22.281836+0000","last_became_active":"2024-03-30T00:41:20.892635+0000","last_became_peered":"2024-03-30T00:41:20.892635+0000","last_unstale":"2024-03-30T00:41:22.281836+0000","last_undegraded":"2024-03-30T00:41:22.281836+0000","last_fullsized":"2024-03-30T00:41:22.281836+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:32.524307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281886+0000","last_change":"2024-03-30T00:41:21.294703+0000","last_active":"2024-03-30T00:41:22.281886+0000","last_peered":"2024-03-30T00:41:22.281886+0000","last_clean":"2024-03-30T00:41:22.281886+0000","last_became_active":"2024-03-30T00:41:21.294409+0000","last_became_peered":"2024-03-30T00:41:21.294409+0000","last_unstale":"2024-03-30T00:41:22.281886+0000","last_undegraded":"2024-03-30T00:41:22.281886+0000","last_fullsized":"2024-03-30T00:41:22.281886+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:57:40.390938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564843+0000","last_change":"2024-03-30T00:41:20.883926+0000","last_active":"2024-03-30T00:41:22.564843+0000","last_peered":"2024-03-30T00:41:22.564843+0000","last_clean":"2024-03-30T00:41:22.564843+0000","last_became_active":"2024-03-30T00:41:20.883619+0000","last_became_peered":"2024-03-30T00:41:20.883619+0000","last_unstale":"2024-03-30T00:41:22.564843+0000","last_undegraded":"2024-03-30T00:41:22.564843+0000","last_fullsized":"2024-03-30T00:41:22.564843+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:56:50.718787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280451+0000","last_change":"2024-03-30T00:41:21.291245+0000","last_active":"2024-03-30T00:41:22.280451+0000","last_peered":"2024-03-30T00:41:22.280451+0000","last_clean":"2024-03-30T00:41:22.280451+0000","last_became_active":"2024-03-30T00:41:21.291008+0000","last_became_peered":"2024-03-30T00:41:21.291008+0000","last_unstale":"2024-03-30T00:41:22.280451+0000","last_undegraded":"2024-03-30T00:41:22.280451+0000","last_fullsized":"2024-03-30T00:41:22.280451+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:04.232839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.288602+0000","last_change":"2024-03-30T00:41:20.891163+0000","last_active":"2024-03-30T00:41:21.288602+0000","last_peered":"2024-03-30T00:41:21.288602+0000","last_clean":"2024-03-30T00:41:21.288602+0000","last_became_active":"2024-03-30T00:41:20.890852+0000","last_became_peered":"2024-03-30T00:41:20.890852+0000","last_unstale":"2024-03-30T00:41:21.288602+0000","last_undegraded":"2024-03-30T00:41:21.288602+0000","last_fullsized":"2024-03-30T00:41:21.288602+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:05.388771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.307031+0000","last_change":"2024-03-30T00:41:21.302427+0000","last_active":"2024-03-30T00:41:21.307031+0000","last_peered":"2024-03-30T00:41:21.307031+0000","last_clean":"2024-03-30T00:41:21.307031+0000","last_became_active":"2024-03-30T00:41:21.302208+0000","last_became_peered":"2024-03-30T00:41:21.302208+0000","last_unstale":"2024-03-30T00:41:21.307031+0000","last_undegraded":"2024-03-30T00:41:21.307031+0000","last_fullsized":"2024-03-30T00:41:21.307031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:31:06.890668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.288835+0000","last_change":"2024-03-30T00:41:20.893139+0000","last_active":"2024-03-30T00:41:21.288835+0000","last_peered":"2024-03-30T00:41:21.288835+0000","last_clean":"2024-03-30T00:41:21.288835+0000","last_became_active":"2024-03-30T00:41:20.892041+0000","last_became_peered":"2024-03-30T00:41:20.892041+0000","last_unstale":"2024-03-30T00:41:21.288835+0000","last_undegraded":"2024-03-30T00:41:21.288835+0000","last_fullsized":"2024-03-30T00:41:21.288835+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:25:47.454420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539621+0000","last_change":"2024-03-30T00:41:21.299126+0000","last_active":"2024-03-30T00:41:22.539621+0000","last_peered":"2024-03-30T00:41:22.539621+0000","last_clean":"2024-03-30T00:41:22.539621+0000","last_became_active":"2024-03-30T00:41:21.298853+0000","last_became_peered":"2024-03-30T00:41:21.298853+0000","last_unstale":"2024-03-30T00:41:22.539621+0000","last_undegraded":"2024-03-30T00:41:22.539621+0000","last_fullsized":"2024-03-30T00:41:22.539621+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:03.515291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.291974+0000","last_change":"2024-03-30T00:41:20.886790+0000","last_active":"2024-03-30T00:41:21.291974+0000","last_peered":"2024-03-30T00:41:21.291974+0000","last_clean":"2024-03-30T00:41:21.291974+0000","last_became_active":"2024-03-30T00:41:20.886541+0000","last_became_peered":"2024-03-30T00:41:20.886541+0000","last_unstale":"2024-03-30T00:41:21.291974+0000","last_undegraded":"2024-03-30T00:41:21.291974+0000","last_fullsized":"2024-03-30T00:41:21.291974+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:56.730386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.305175+0000","last_change":"2024-03-30T00:41:21.296348+0000","last_active":"2024-03-30T00:41:21.305175+0000","last_peered":"2024-03-30T00:41:21.305175+0000","last_clean":"2024-03-30T00:41:21.305175+0000","last_became_active":"2024-03-30T00:41:21.295366+0000","last_became_peered":"2024-03-30T00:41:21.295366+0000","last_unstale":"2024-03-30T00:41:21.305175+0000","last_undegraded":"2024-03-30T00:41:21.305175+0000","last_fullsized":"2024-03-30T00:41:21.305175+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:14.043465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.278875+0000","last_change":"2024-03-30T00:41:20.880904+0000","last_active":"2024-03-30T00:41:21.278875+0000","last_peered":"2024-03-30T00:41:21.278875+0000","last_clean":"2024-03-30T00:41:21.278875+0000","last_became_active":"2024-03-30T00:41:20.880463+0000","last_became_peered":"2024-03-30T00:41:20.880463+0000","last_unstale":"2024-03-30T00:41:21.278875+0000","last_undegraded":"2024-03-30T00:41:21.278875+0000","last_fullsized":"2024-03-30T00:41:21.278875+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:08:07.735075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281864+0000","last_change":"2024-03-30T00:41:21.303311+0000","last_active":"2024-03-30T00:41:22.281864+0000","last_peered":"2024-03-30T00:41:22.281864+0000","last_clean":"2024-03-30T00:41:22.281864+0000","last_became_active":"2024-03-30T00:41:21.298110+0000","last_became_peered":"2024-03-30T00:41:21.298110+0000","last_unstale":"2024-03-30T00:41:22.281864+0000","last_undegraded":"2024-03-30T00:41:22.281864+0000","last_fullsized":"2024-03-30T00:41:22.281864+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:15.584712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280007+0000","last_change":"2024-03-30T00:41:20.893451+0000","last_active":"2024-03-30T00:41:22.280007+0000","last_peered":"2024-03-30T00:41:22.280007+0000","last_clean":"2024-03-30T00:41:22.280007+0000","last_became_active":"2024-03-30T00:41:20.892551+0000","last_became_peered":"2024-03-30T00:41:20.892551+0000","last_unstale":"2024-03-30T00:41:22.280007+0000","last_undegraded":"2024-03-30T00:41:22.280007+0000","last_fullsized":"2024-03-30T00:41:22.280007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:57:39.385189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281495+0000","last_change":"2024-03-30T00:41:21.307461+0000","last_active":"2024-03-30T00:41:22.281495+0000","last_peered":"2024-03-30T00:41:22.281495+0000","last_clean":"2024-03-30T00:41:22.281495+0000","last_became_active":"2024-03-30T00:41:21.307219+0000","last_became_peered":"2024-03-30T00:41:21.307219+0000","last_unstale":"2024-03-30T00:41:22.281495+0000","last_undegraded":"2024-03-30T00:41:22.281495+0000","last_fullsized":"2024-03-30T00:41:22.281495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:31:06.890668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281970+0000","last_change":"2024-03-30T00:41:20.884185+0000","last_active":"2024-03-30T00:41:22.281970+0000","last_peered":"2024-03-30T00:41:22.281970+0000","last_clean":"2024-03-30T00:41:22.281970+0000","last_became_active":"2024-03-30T00:41:20.883941+0000","last_became_peered":"2024-03-30T00:41:20.883941+0000","last_unstale":"2024-03-30T00:41:22.281970+0000","last_undegraded":"2024-03-30T00:41:22.281970+0000","last_fullsized":"2024-03-30T00:41:22.281970+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:04:14.843309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.280171+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:41:21.280171+0000","last_peered":"2024-03-30T00:41:21.280171+0000","last_clean":"2024-03-30T00:41:21.280171+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:41:21.280171+0000","last_undegraded":"2024-03-30T00:41:21.280171+0000","last_fullsized":"2024-03-30T00:41:21.280171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1073152,"data_stored":791120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.69699999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.65000000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.60999999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.71499999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67900000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.73299999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.745}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":454656,"data_stored":193080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.0490000000000004,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.66500000000000004}]},{"osd":1,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.3170000000000002,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.4160000000000004,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.63}]},{"osd":2,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":4.375,"5min":4.375,"15min":4.375},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.2960000000000003,"5min":4.2960000000000003,"15min":4.2960000000000003},"last":0.78500000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.3399999999999999,"5min":4.3399999999999999,"15min":4.3399999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.6020000000000003,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.61399999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.82599999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80500000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.65200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":471040,"data_stored":199230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.75700000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.752}]},{"osd":7,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.92800000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":458752,"data_stored":193102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.59299999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.495},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.55200000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.60999999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.68700000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":200716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.29699999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42299999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.76500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":192568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.49199999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.56000000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.127}]},{"osd":5,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.879}]},{"osd":6,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.179}]},{"osd":7,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.01},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.0700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":192588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.501}]},{"osd":2,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.54900000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.61099999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.77100000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.871}]},{"osd":6,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79600000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.82199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":782900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.39600000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.83799999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.78300000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.81100000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.746}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:41:25.386 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:41:25.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4809c00640 1 Processor -- start 2024-03-30T00:41:25.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4809c00640 1 -- start start 2024-03-30T00:41:25.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4809c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f480406bb90 0x7f4804098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4809c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f480406bff0 0x7f4804098f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4809c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48040b0940 con 0x7f48040af790 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4809c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480406b140 con 0x7f480406bff0 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4809c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480406b700 con 0x7f480406bb90 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4803400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f480406bff0 0x7f4804098f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4803400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f480406bff0 0x7f4804098f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57756/0 (socket says 172.21.15.66:57756) 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4803400640 1 -- 172.21.15.66:0/275472591 learned_addr learned my addr 172.21.15.66:0/275472591 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4802000640 1 --2- 172.21.15.66:0/275472591 >> v2:172.21.15.156:3300/0 conn(0x7f480406bb90 0x7f4804098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4803400640 1 -- 172.21.15.66:0/275472591 >> v2:172.21.15.156:3300/0 conn(0x7f480406bb90 msgr2=0x7f4804098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4803400640 1 --2- 172.21.15.66:0/275472591 >> v2:172.21.15.156:3300/0 conn(0x7f480406bb90 0x7f4804098a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4803400640 1 -- 172.21.15.66:0/275472591 >> v1:172.21.15.156:6789/0 conn(0x7f48040af790 legacy=0x7f4804099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.561+0000 7f4803400640 1 -- 172.21.15.66:0/275472591 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48040b0b90 con 0x7f480406bff0 2024-03-30T00:41:25.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4803400640 1 --2- 172.21.15.66:0/275472591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480406bff0 0x7f4804098f60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f48040d05a0 tx=0x7f47fc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e485d79a8c18884 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:25.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4801600640 1 -- 172.21.15.66:0/275472591 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc0a5880 con 0x7f480406bff0 2024-03-30T00:41:25.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4801600640 1 -- 172.21.15.66:0/275472591 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc0a59e0 con 0x7f480406bff0 2024-03-30T00:41:25.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4801600640 1 -- 172.21.15.66:0/275472591 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc0a5d70 con 0x7f480406bff0 2024-03-30T00:41:25.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4809c00640 1 -- 172.21.15.66:0/275472591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480406bff0 msgr2=0x7f4804098f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4809c00640 1 --2- 172.21.15.66:0/275472591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480406bff0 0x7f4804098f60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f48040d05a0 tx=0x7f47fc01c9c0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4809c00640 1 -- 172.21.15.66:0/275472591 shutdown_connections 2024-03-30T00:41:25.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4809c00640 1 --2- 172.21.15.66:0/275472591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480406bff0 0x7f4804098f60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4809c00640 1 --2- 172.21.15.66:0/275472591 >> v2:172.21.15.156:3300/0 conn(0x7f480406bb90 0x7f4804098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4809c00640 1 -- 172.21.15.66:0/275472591 >> 172.21.15.66:0/275472591 conn(0x7f48040b20c0 msgr2=0x7f48040b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:25.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4809c00640 1 -- 172.21.15.66:0/275472591 shutdown_connections 2024-03-30T00:41:25.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4809c00640 1 -- 172.21.15.66:0/275472591 wait complete. 2024-03-30T00:41:25.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4809c00640 1 Processor -- start 2024-03-30T00:41:25.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.565+0000 7f4809c00640 1 -- start start 2024-03-30T00:41:25.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4809c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480406bb90 0x7f480415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4809c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406bff0 0x7f4804157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4809c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48040af790 0x7f4804155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4809c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4804157d40 con 0x7f480406bb90 2024-03-30T00:41:25.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4809c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480414b350 con 0x7f480406bff0 2024-03-30T00:41:25.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4809c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4804157370 con 0x7f48040af790 2024-03-30T00:41:25.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4802a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480406bb90 0x7f480415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4802a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480406bb90 0x7f480415ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57768/0 (socket says 172.21.15.66:57768) 2024-03-30T00:41:25.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4802a00640 1 -- 172.21.15.66:0/3491821746 learned_addr learned my addr 172.21.15.66:0/3491821746 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:25.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4802000640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406bff0 0x7f4804157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4803400640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48040af790 0x7f4804155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4802a00640 1 -- 172.21.15.66:0/3491821746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48040af790 msgr2=0x7f4804155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4802a00640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48040af790 0x7f4804155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4802a00640 1 -- 172.21.15.66:0/3491821746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406bff0 msgr2=0x7f4804157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4802a00640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406bff0 0x7f4804157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4802a00640 1 -- 172.21.15.66:0/3491821746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc009b30 con 0x7f480406bb90 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4803400640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48040af790 0x7f4804155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4802a00640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480406bb90 0x7f480415ae70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f47f400c3c0 tx=0x7f47f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f47faa00640 1 -- 172.21.15.66:0/3491821746 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f40052d0 con 0x7f480406bb90 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f47faa00640 1 -- 172.21.15.66:0/3491821746 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f4005430 con 0x7f480406bb90 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f47faa00640 1 -- 172.21.15.66:0/3491821746 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4003840 con 0x7f480406bb90 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4809c00640 1 -- 172.21.15.66:0/3491821746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4804156760 con 0x7f480406bb90 2024-03-30T00:41:25.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4809c00640 1 -- 172.21.15.66:0/3491821746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4804156d20 con 0x7f480406bb90 2024-03-30T00:41:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f4809c00640 1 -- 172.21.15.66:0/3491821746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4804066c30 con 0x7f480406bb90 2024-03-30T00:41:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.569+0000 7f47faa00640 1 -- 172.21.15.66:0/3491821746 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f47f40039a0 con 0x7f480406bb90 2024-03-30T00:41:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.573+0000 7f47faa00640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47e40666c0 0x7f47e4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.573+0000 7f47faa00640 1 -- 172.21.15.66:0/3491821746 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f47f4083730 con 0x7f480406bb90 2024-03-30T00:41:25.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.573+0000 7f4802000640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47e40666c0 0x7f47e4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.573+0000 7f4802000640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47e40666c0 0x7f47e4068b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f48041f81c0 tx=0x7f47f0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:25.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.577+0000 7f47faa00640 1 -- 172.21.15.66:0/3491821746 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f47f4050a10 con 0x7f480406bb90 2024-03-30T00:41:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.733+0000 7f4809c00640 1 -- 172.21.15.66:0/3491821746 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48040cf110 con 0x7f47e40666c0 2024-03-30T00:41:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.741+0000 7f47faa00640 1 -- 172.21.15.66:0/3491821746 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289631 (secure 0 0 0) 0x7f47f0011a30 con 0x7f47e40666c0 2024-03-30T00:41:25.755 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:41:25.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 -- 172.21.15.66:0/3491821746 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47e40666c0 msgr2=0x7f47e4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47e40666c0 0x7f47e4068b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f48041f81c0 tx=0x7f47f0009190 comp rx=0 tx=0).stop 2024-03-30T00:41:25.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 -- 172.21.15.66:0/3491821746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480406bb90 msgr2=0x7f480415ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480406bb90 0x7f480415ae70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f47f400c3c0 tx=0x7f47f400c880 comp rx=0 tx=0).stop 2024-03-30T00:41:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 -- 172.21.15.66:0/3491821746 shutdown_connections 2024-03-30T00:41:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48040af790 0x7f4804155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406bff0 0x7f4804157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47e40666c0 0x7f47e4068b80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 --2- 172.21.15.66:0/3491821746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480406bb90 0x7f480415ae70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 -- 172.21.15.66:0/3491821746 >> 172.21.15.66:0/3491821746 conn(0x7f48040b20c0 msgr2=0x7f4804098a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 -- 172.21.15.66:0/3491821746 shutdown_connections 2024-03-30T00:41:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.745+0000 7f4809c00640 1 -- 172.21.15.66:0/3491821746 wait complete. 2024-03-30T00:41:25.774 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-03-30T00:41:24.870224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237352,"kb_used_data":4780,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737368,"statfs":{"total":773094113280,"available":772851064832,"internally_reserved":0,"allocated":4894720,"data_stored":2745304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.009139"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280719+0000","last_change":"2024-03-30T00:41:21.299125+0000","last_active":"2024-03-30T00:41:22.280719+0000","last_peered":"2024-03-30T00:41:22.280719+0000","last_clean":"2024-03-30T00:41:22.280719+0000","last_became_active":"2024-03-30T00:41:21.295699+0000","last_became_peered":"2024-03-30T00:41:21.295699+0000","last_unstale":"2024-03-30T00:41:22.280719+0000","last_undegraded":"2024-03-30T00:41:22.280719+0000","last_fullsized":"2024-03-30T00:41:22.280719+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:09:53.534935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280424+0000","last_change":"2024-03-30T00:41:21.302011+0000","last_active":"2024-03-30T00:41:22.280424+0000","last_peered":"2024-03-30T00:41:22.280424+0000","last_clean":"2024-03-30T00:41:22.280424+0000","last_became_active":"2024-03-30T00:41:21.301698+0000","last_became_peered":"2024-03-30T00:41:21.301698+0000","last_unstale":"2024-03-30T00:41:22.280424+0000","last_undegraded":"2024-03-30T00:41:22.280424+0000","last_fullsized":"2024-03-30T00:41:22.280424+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.993492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.306878+0000","last_change":"2024-03-30T00:41:21.300067+0000","last_active":"2024-03-30T00:41:21.306878+0000","last_peered":"2024-03-30T00:41:21.306878+0000","last_clean":"2024-03-30T00:41:21.306878+0000","last_became_active":"2024-03-30T00:41:21.299822+0000","last_became_peered":"2024-03-30T00:41:21.299822+0000","last_unstale":"2024-03-30T00:41:21.306878+0000","last_undegraded":"2024-03-30T00:41:21.306878+0000","last_fullsized":"2024-03-30T00:41:21.306878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:38.260912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564647+0000","last_change":"2024-03-30T00:41:21.308876+0000","last_active":"2024-03-30T00:41:22.564647+0000","last_peered":"2024-03-30T00:41:22.564647+0000","last_clean":"2024-03-30T00:41:22.564647+0000","last_became_active":"2024-03-30T00:41:21.308646+0000","last_became_peered":"2024-03-30T00:41:21.308646+0000","last_unstale":"2024-03-30T00:41:22.564647+0000","last_undegraded":"2024-03-30T00:41:22.564647+0000","last_fullsized":"2024-03-30T00:41:22.564647+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:16:21.552089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564751+0000","last_change":"2024-03-30T00:41:21.304099+0000","last_active":"2024-03-30T00:41:22.564751+0000","last_peered":"2024-03-30T00:41:22.564751+0000","last_clean":"2024-03-30T00:41:22.564751+0000","last_became_active":"2024-03-30T00:41:21.303866+0000","last_became_peered":"2024-03-30T00:41:21.303866+0000","last_unstale":"2024-03-30T00:41:22.564751+0000","last_undegraded":"2024-03-30T00:41:22.564751+0000","last_fullsized":"2024-03-30T00:41:22.564751+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:58.597464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539688+0000","last_change":"2024-03-30T00:41:21.305574+0000","last_active":"2024-03-30T00:41:22.539688+0000","last_peered":"2024-03-30T00:41:22.539688+0000","last_clean":"2024-03-30T00:41:22.539688+0000","last_became_active":"2024-03-30T00:41:21.304749+0000","last_became_peered":"2024-03-30T00:41:21.304749+0000","last_unstale":"2024-03-30T00:41:22.539688+0000","last_undegraded":"2024-03-30T00:41:22.539688+0000","last_fullsized":"2024-03-30T00:41:22.539688+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:04:31.558432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281029+0000","last_change":"2024-03-30T00:41:21.304184+0000","last_active":"2024-03-30T00:41:22.281029+0000","last_peered":"2024-03-30T00:41:22.281029+0000","last_clean":"2024-03-30T00:41:22.281029+0000","last_became_active":"2024-03-30T00:41:21.303478+0000","last_became_peered":"2024-03-30T00:41:21.303478+0000","last_unstale":"2024-03-30T00:41:22.281029+0000","last_undegraded":"2024-03-30T00:41:22.281029+0000","last_fullsized":"2024-03-30T00:41:22.281029+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:38.260912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.279948+0000","last_change":"2024-03-30T00:41:21.292192+0000","last_active":"2024-03-30T00:41:22.279948+0000","last_peered":"2024-03-30T00:41:22.279948+0000","last_clean":"2024-03-30T00:41:22.279948+0000","last_became_active":"2024-03-30T00:41:21.291955+0000","last_became_peered":"2024-03-30T00:41:21.291955+0000","last_unstale":"2024-03-30T00:41:22.279948+0000","last_undegraded":"2024-03-30T00:41:22.279948+0000","last_fullsized":"2024-03-30T00:41:22.279948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:23:29.452017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.301989+0000","last_change":"2024-03-30T00:41:21.292810+0000","last_active":"2024-03-30T00:41:21.301989+0000","last_peered":"2024-03-30T00:41:21.301989+0000","last_clean":"2024-03-30T00:41:21.301989+0000","last_became_active":"2024-03-30T00:41:21.292574+0000","last_became_peered":"2024-03-30T00:41:21.292574+0000","last_unstale":"2024-03-30T00:41:21.301989+0000","last_undegraded":"2024-03-30T00:41:21.301989+0000","last_fullsized":"2024-03-30T00:41:21.301989+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:48:28.760437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281660+0000","last_change":"2024-03-30T00:41:21.300648+0000","last_active":"2024-03-30T00:41:22.281660+0000","last_peered":"2024-03-30T00:41:22.281660+0000","last_clean":"2024-03-30T00:41:22.281660+0000","last_became_active":"2024-03-30T00:41:21.298814+0000","last_became_peered":"2024-03-30T00:41:21.298814+0000","last_unstale":"2024-03-30T00:41:22.281660+0000","last_undegraded":"2024-03-30T00:41:22.281660+0000","last_fullsized":"2024-03-30T00:41:22.281660+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:47:51.023501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281738+0000","last_change":"2024-03-30T00:41:21.295703+0000","last_active":"2024-03-30T00:41:22.281738+0000","last_peered":"2024-03-30T00:41:22.281738+0000","last_clean":"2024-03-30T00:41:22.281738+0000","last_became_active":"2024-03-30T00:41:21.294095+0000","last_became_peered":"2024-03-30T00:41:21.294095+0000","last_unstale":"2024-03-30T00:41:22.281738+0000","last_undegraded":"2024-03-30T00:41:22.281738+0000","last_fullsized":"2024-03-30T00:41:22.281738+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:09:12.628109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.310174+0000","last_change":"2024-03-30T00:41:21.303726+0000","last_active":"2024-03-30T00:41:21.310174+0000","last_peered":"2024-03-30T00:41:21.310174+0000","last_clean":"2024-03-30T00:41:21.310174+0000","last_became_active":"2024-03-30T00:41:21.303525+0000","last_became_peered":"2024-03-30T00:41:21.303525+0000","last_unstale":"2024-03-30T00:41:21.310174+0000","last_undegraded":"2024-03-30T00:41:21.310174+0000","last_fullsized":"2024-03-30T00:41:21.310174+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:47:51.023501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281345+0000","last_change":"2024-03-30T00:41:21.298639+0000","last_active":"2024-03-30T00:41:22.281345+0000","last_peered":"2024-03-30T00:41:22.281345+0000","last_clean":"2024-03-30T00:41:22.281345+0000","last_became_active":"2024-03-30T00:41:21.297536+0000","last_became_peered":"2024-03-30T00:41:21.297536+0000","last_unstale":"2024-03-30T00:41:22.281345+0000","last_undegraded":"2024-03-30T00:41:22.281345+0000","last_fullsized":"2024-03-30T00:41:22.281345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:56.894917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.306872+0000","last_change":"2024-03-30T00:41:21.298352+0000","last_active":"2024-03-30T00:41:21.306872+0000","last_peered":"2024-03-30T00:41:21.306872+0000","last_clean":"2024-03-30T00:41:21.306872+0000","last_became_active":"2024-03-30T00:41:21.297939+0000","last_became_peered":"2024-03-30T00:41:21.297939+0000","last_unstale":"2024-03-30T00:41:21.306872+0000","last_undegraded":"2024-03-30T00:41:21.306872+0000","last_fullsized":"2024-03-30T00:41:21.306872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:54:01.879198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281902+0000","last_change":"2024-03-30T00:41:21.306401+0000","last_active":"2024-03-30T00:41:22.281902+0000","last_peered":"2024-03-30T00:41:22.281902+0000","last_clean":"2024-03-30T00:41:22.281902+0000","last_became_active":"2024-03-30T00:41:21.306141+0000","last_became_peered":"2024-03-30T00:41:21.306141+0000","last_unstale":"2024-03-30T00:41:22.281902+0000","last_undegraded":"2024-03-30T00:41:22.281902+0000","last_fullsized":"2024-03-30T00:41:22.281902+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:23:29.452017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280824+0000","last_change":"2024-03-30T00:41:21.290189+0000","last_active":"2024-03-30T00:41:22.280824+0000","last_peered":"2024-03-30T00:41:22.280824+0000","last_clean":"2024-03-30T00:41:22.280824+0000","last_became_active":"2024-03-30T00:41:21.289928+0000","last_became_peered":"2024-03-30T00:41:21.289928+0000","last_unstale":"2024-03-30T00:41:22.280824+0000","last_undegraded":"2024-03-30T00:41:22.280824+0000","last_fullsized":"2024-03-30T00:41:22.280824+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:34:18.703475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281764+0000","last_change":"2024-03-30T00:41:21.293202+0000","last_active":"2024-03-30T00:41:22.281764+0000","last_peered":"2024-03-30T00:41:22.281764+0000","last_clean":"2024-03-30T00:41:22.281764+0000","last_became_active":"2024-03-30T00:41:21.292882+0000","last_became_peered":"2024-03-30T00:41:21.292882+0000","last_unstale":"2024-03-30T00:41:22.281764+0000","last_undegraded":"2024-03-30T00:41:22.281764+0000","last_fullsized":"2024-03-30T00:41:22.281764+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:03.515291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.313991+0000","last_change":"2024-03-30T00:41:21.312085+0000","last_active":"2024-03-30T00:41:21.313991+0000","last_peered":"2024-03-30T00:41:21.313991+0000","last_clean":"2024-03-30T00:41:21.313991+0000","last_became_active":"2024-03-30T00:41:21.306396+0000","last_became_peered":"2024-03-30T00:41:21.306396+0000","last_unstale":"2024-03-30T00:41:21.313991+0000","last_undegraded":"2024-03-30T00:41:21.313991+0000","last_fullsized":"2024-03-30T00:41:21.313991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:55.227725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564690+0000","last_change":"2024-03-30T00:41:21.307933+0000","last_active":"2024-03-30T00:41:22.564690+0000","last_peered":"2024-03-30T00:41:22.564690+0000","last_clean":"2024-03-30T00:41:22.564690+0000","last_became_active":"2024-03-30T00:41:21.307551+0000","last_became_peered":"2024-03-30T00:41:21.307551+0000","last_unstale":"2024-03-30T00:41:22.564690+0000","last_undegraded":"2024-03-30T00:41:22.564690+0000","last_fullsized":"2024-03-30T00:41:22.564690+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:02:06.903366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.295928+0000","last_change":"2024-03-30T00:41:21.292507+0000","last_active":"2024-03-30T00:41:21.295928+0000","last_peered":"2024-03-30T00:41:21.295928+0000","last_clean":"2024-03-30T00:41:21.295928+0000","last_became_active":"2024-03-30T00:41:21.292036+0000","last_became_peered":"2024-03-30T00:41:21.292036+0000","last_unstale":"2024-03-30T00:41:21.295928+0000","last_undegraded":"2024-03-30T00:41:21.295928+0000","last_fullsized":"2024-03-30T00:41:21.295928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:58.597464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281335+0000","last_change":"2024-03-30T00:41:21.303867+0000","last_active":"2024-03-30T00:41:22.281335+0000","last_peered":"2024-03-30T00:41:22.281335+0000","last_clean":"2024-03-30T00:41:22.281335+0000","last_became_active":"2024-03-30T00:41:21.301707+0000","last_became_peered":"2024-03-30T00:41:21.301707+0000","last_unstale":"2024-03-30T00:41:22.281335+0000","last_undegraded":"2024-03-30T00:41:22.281335+0000","last_fullsized":"2024-03-30T00:41:22.281335+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:38.690725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.310395+0000","last_change":"2024-03-30T00:41:21.308068+0000","last_active":"2024-03-30T00:41:21.310395+0000","last_peered":"2024-03-30T00:41:21.310395+0000","last_clean":"2024-03-30T00:41:21.310395+0000","last_became_active":"2024-03-30T00:41:21.307511+0000","last_became_peered":"2024-03-30T00:41:21.307511+0000","last_unstale":"2024-03-30T00:41:21.310395+0000","last_undegraded":"2024-03-30T00:41:21.310395+0000","last_fullsized":"2024-03-30T00:41:21.310395+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:53.020603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280942+0000","last_change":"2024-03-30T00:41:21.309056+0000","last_active":"2024-03-30T00:41:22.280942+0000","last_peered":"2024-03-30T00:41:22.280942+0000","last_clean":"2024-03-30T00:41:22.280942+0000","last_became_active":"2024-03-30T00:41:21.308515+0000","last_became_peered":"2024-03-30T00:41:21.308515+0000","last_unstale":"2024-03-30T00:41:22.280942+0000","last_undegraded":"2024-03-30T00:41:22.280942+0000","last_fullsized":"2024-03-30T00:41:22.280942+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:12:15.742797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564531+0000","last_change":"2024-03-30T00:41:21.310278+0000","last_active":"2024-03-30T00:41:22.564531+0000","last_peered":"2024-03-30T00:41:22.564531+0000","last_clean":"2024-03-30T00:41:22.564531+0000","last_became_active":"2024-03-30T00:41:21.310011+0000","last_became_peered":"2024-03-30T00:41:21.310011+0000","last_unstale":"2024-03-30T00:41:22.564531+0000","last_undegraded":"2024-03-30T00:41:22.564531+0000","last_fullsized":"2024-03-30T00:41:22.564531+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:35:02.226502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.313984+0000","last_change":"2024-03-30T00:41:21.312042+0000","last_active":"2024-03-30T00:41:21.313984+0000","last_peered":"2024-03-30T00:41:21.313984+0000","last_clean":"2024-03-30T00:41:21.313984+0000","last_became_active":"2024-03-30T00:41:21.307065+0000","last_became_peered":"2024-03-30T00:41:21.307065+0000","last_unstale":"2024-03-30T00:41:21.313984+0000","last_undegraded":"2024-03-30T00:41:21.313984+0000","last_fullsized":"2024-03-30T00:41:21.313984+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:16:21.552089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539912+0000","last_change":"2024-03-30T00:41:21.297369+0000","last_active":"2024-03-30T00:41:22.539912+0000","last_peered":"2024-03-30T00:41:22.539912+0000","last_clean":"2024-03-30T00:41:22.539912+0000","last_became_active":"2024-03-30T00:41:21.297003+0000","last_became_peered":"2024-03-30T00:41:21.297003+0000","last_unstale":"2024-03-30T00:41:22.539912+0000","last_undegraded":"2024-03-30T00:41:22.539912+0000","last_fullsized":"2024-03-30T00:41:22.539912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:06.394519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.313662+0000","last_change":"2024-03-30T00:41:21.306426+0000","last_active":"2024-03-30T00:41:21.313662+0000","last_peered":"2024-03-30T00:41:21.313662+0000","last_clean":"2024-03-30T00:41:21.313662+0000","last_became_active":"2024-03-30T00:41:21.305854+0000","last_became_peered":"2024-03-30T00:41:21.305854+0000","last_unstale":"2024-03-30T00:41:21.313662+0000","last_undegraded":"2024-03-30T00:41:21.313662+0000","last_fullsized":"2024-03-30T00:41:21.313662+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:35:02.226502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281560+0000","last_change":"2024-03-30T00:41:21.306566+0000","last_active":"2024-03-30T00:41:22.281560+0000","last_peered":"2024-03-30T00:41:22.281560+0000","last_clean":"2024-03-30T00:41:22.281560+0000","last_became_active":"2024-03-30T00:41:21.306309+0000","last_became_peered":"2024-03-30T00:41:21.306309+0000","last_unstale":"2024-03-30T00:41:22.281560+0000","last_undegraded":"2024-03-30T00:41:22.281560+0000","last_fullsized":"2024-03-30T00:41:22.281560+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:53.020603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.306869+0000","last_change":"2024-03-30T00:41:21.305045+0000","last_active":"2024-03-30T00:41:21.306869+0000","last_peered":"2024-03-30T00:41:21.306869+0000","last_clean":"2024-03-30T00:41:21.306869+0000","last_became_active":"2024-03-30T00:41:21.304784+0000","last_became_peered":"2024-03-30T00:41:21.304784+0000","last_unstale":"2024-03-30T00:41:21.306869+0000","last_undegraded":"2024-03-30T00:41:21.306869+0000","last_fullsized":"2024-03-30T00:41:21.306869+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:02:06.903366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.313998+0000","last_change":"2024-03-30T00:41:21.306697+0000","last_active":"2024-03-30T00:41:21.313998+0000","last_peered":"2024-03-30T00:41:21.313998+0000","last_clean":"2024-03-30T00:41:21.313998+0000","last_became_active":"2024-03-30T00:41:21.306126+0000","last_became_peered":"2024-03-30T00:41:21.306126+0000","last_unstale":"2024-03-30T00:41:21.313998+0000","last_undegraded":"2024-03-30T00:41:21.313998+0000","last_fullsized":"2024-03-30T00:41:21.313998+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:35:11.629197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281052+0000","last_change":"2024-03-30T00:41:21.305010+0000","last_active":"2024-03-30T00:41:22.281052+0000","last_peered":"2024-03-30T00:41:22.281052+0000","last_clean":"2024-03-30T00:41:22.281052+0000","last_became_active":"2024-03-30T00:41:21.304743+0000","last_became_peered":"2024-03-30T00:41:21.304743+0000","last_unstale":"2024-03-30T00:41:22.281052+0000","last_undegraded":"2024-03-30T00:41:22.281052+0000","last_fullsized":"2024-03-30T00:41:22.281052+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:08:08.740824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539695+0000","last_change":"2024-03-30T00:41:21.299911+0000","last_active":"2024-03-30T00:41:22.539695+0000","last_peered":"2024-03-30T00:41:22.539695+0000","last_clean":"2024-03-30T00:41:22.539695+0000","last_became_active":"2024-03-30T00:41:21.299722+0000","last_became_peered":"2024-03-30T00:41:21.299722+0000","last_unstale":"2024-03-30T00:41:22.539695+0000","last_undegraded":"2024-03-30T00:41:22.539695+0000","last_fullsized":"2024-03-30T00:41:22.539695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:08:08.740824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281874+0000","last_change":"2024-03-30T00:41:21.289247+0000","last_active":"2024-03-30T00:41:22.281874+0000","last_peered":"2024-03-30T00:41:22.281874+0000","last_clean":"2024-03-30T00:41:22.281874+0000","last_became_active":"2024-03-30T00:41:21.288925+0000","last_became_peered":"2024-03-30T00:41:21.288925+0000","last_unstale":"2024-03-30T00:41:22.281874+0000","last_undegraded":"2024-03-30T00:41:22.281874+0000","last_fullsized":"2024-03-30T00:41:22.281874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:27:46.606824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.313520+0000","last_change":"2024-03-30T00:41:20.888469+0000","last_active":"2024-03-30T00:41:21.313520+0000","last_peered":"2024-03-30T00:41:21.313520+0000","last_clean":"2024-03-30T00:41:21.313520+0000","last_became_active":"2024-03-30T00:41:20.888214+0000","last_became_peered":"2024-03-30T00:41:20.888214+0000","last_unstale":"2024-03-30T00:41:21.313520+0000","last_undegraded":"2024-03-30T00:41:21.313520+0000","last_fullsized":"2024-03-30T00:41:21.313520+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:05:06.481400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539911+0000","last_change":"2024-03-30T00:41:21.306416+0000","last_active":"2024-03-30T00:41:22.539911+0000","last_peered":"2024-03-30T00:41:22.539911+0000","last_clean":"2024-03-30T00:41:22.539911+0000","last_became_active":"2024-03-30T00:41:21.306095+0000","last_became_peered":"2024-03-30T00:41:21.306095+0000","last_unstale":"2024-03-30T00:41:22.539911+0000","last_undegraded":"2024-03-30T00:41:22.539911+0000","last_fullsized":"2024-03-30T00:41:22.539911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:25:48.460169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.302607+0000","last_change":"2024-03-30T00:41:20.888714+0000","last_active":"2024-03-30T00:41:21.302607+0000","last_peered":"2024-03-30T00:41:21.302607+0000","last_clean":"2024-03-30T00:41:21.302607+0000","last_became_active":"2024-03-30T00:41:20.888426+0000","last_became_peered":"2024-03-30T00:41:20.888426+0000","last_unstale":"2024-03-30T00:41:21.302607+0000","last_undegraded":"2024-03-30T00:41:21.302607+0000","last_fullsized":"2024-03-30T00:41:21.302607+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:08:07.735075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.310229+0000","last_change":"2024-03-30T00:41:21.301502+0000","last_active":"2024-03-30T00:41:21.310229+0000","last_peered":"2024-03-30T00:41:21.310229+0000","last_clean":"2024-03-30T00:41:21.310229+0000","last_became_active":"2024-03-30T00:41:21.300775+0000","last_became_peered":"2024-03-30T00:41:21.300775+0000","last_unstale":"2024-03-30T00:41:21.310229+0000","last_undegraded":"2024-03-30T00:41:21.310229+0000","last_fullsized":"2024-03-30T00:41:21.310229+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:38.260912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539985+0000","last_change":"2024-03-30T00:41:20.884939+0000","last_active":"2024-03-30T00:41:22.539985+0000","last_peered":"2024-03-30T00:41:22.539985+0000","last_clean":"2024-03-30T00:41:22.539985+0000","last_became_active":"2024-03-30T00:41:20.884688+0000","last_became_peered":"2024-03-30T00:41:20.884688+0000","last_unstale":"2024-03-30T00:41:22.539985+0000","last_undegraded":"2024-03-30T00:41:22.539985+0000","last_fullsized":"2024-03-30T00:41:22.539985+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:35:10.623449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280634+0000","last_change":"2024-03-30T00:41:21.295354+0000","last_active":"2024-03-30T00:41:22.280634+0000","last_peered":"2024-03-30T00:41:22.280634+0000","last_clean":"2024-03-30T00:41:22.280634+0000","last_became_active":"2024-03-30T00:41:21.294567+0000","last_became_peered":"2024-03-30T00:41:21.294567+0000","last_unstale":"2024-03-30T00:41:22.280634+0000","last_undegraded":"2024-03-30T00:41:22.280634+0000","last_fullsized":"2024-03-30T00:41:22.280634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:12:19.195955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.314018+0000","last_change":"2024-03-30T00:41:20.879477+0000","last_active":"2024-03-30T00:41:21.314018+0000","last_peered":"2024-03-30T00:41:21.314018+0000","last_clean":"2024-03-30T00:41:21.314018+0000","last_became_active":"2024-03-30T00:41:20.879219+0000","last_became_peered":"2024-03-30T00:41:20.879219+0000","last_unstale":"2024-03-30T00:41:21.314018+0000","last_undegraded":"2024-03-30T00:41:21.314018+0000","last_fullsized":"2024-03-30T00:41:21.314018+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:05.388771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.307011+0000","last_change":"2024-03-30T00:41:21.301286+0000","last_active":"2024-03-30T00:41:21.307011+0000","last_peered":"2024-03-30T00:41:21.307011+0000","last_clean":"2024-03-30T00:41:21.307011+0000","last_became_active":"2024-03-30T00:41:21.300413+0000","last_became_peered":"2024-03-30T00:41:21.300413+0000","last_unstale":"2024-03-30T00:41:21.307011+0000","last_undegraded":"2024-03-30T00:41:21.307011+0000","last_fullsized":"2024-03-30T00:41:21.307011+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:54:01.879198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281498+0000","last_change":"2024-03-30T00:41:20.890094+0000","last_active":"2024-03-30T00:41:22.281498+0000","last_peered":"2024-03-30T00:41:22.281498+0000","last_clean":"2024-03-30T00:41:22.281498+0000","last_became_active":"2024-03-30T00:41:20.889838+0000","last_became_peered":"2024-03-30T00:41:20.889838+0000","last_unstale":"2024-03-30T00:41:22.281498+0000","last_undegraded":"2024-03-30T00:41:22.281498+0000","last_fullsized":"2024-03-30T00:41:22.281498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:10:00.616749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281622+0000","last_change":"2024-03-30T00:41:21.305058+0000","last_active":"2024-03-30T00:41:22.281622+0000","last_peered":"2024-03-30T00:41:22.281622+0000","last_clean":"2024-03-30T00:41:22.281622+0000","last_became_active":"2024-03-30T00:41:21.303926+0000","last_became_peered":"2024-03-30T00:41:21.303926+0000","last_unstale":"2024-03-30T00:41:22.281622+0000","last_undegraded":"2024-03-30T00:41:22.281622+0000","last_fullsized":"2024-03-30T00:41:22.281622+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:47.922056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281409+0000","last_change":"2024-03-30T00:41:20.887680+0000","last_active":"2024-03-30T00:41:22.281409+0000","last_peered":"2024-03-30T00:41:22.281409+0000","last_clean":"2024-03-30T00:41:22.281409+0000","last_became_active":"2024-03-30T00:41:20.887417+0000","last_became_peered":"2024-03-30T00:41:20.887417+0000","last_unstale":"2024-03-30T00:41:22.281409+0000","last_undegraded":"2024-03-30T00:41:22.281409+0000","last_fullsized":"2024-03-30T00:41:22.281409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:35:01.220754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.309392+0000","last_change":"2024-03-30T00:41:21.300301+0000","last_active":"2024-03-30T00:41:21.309392+0000","last_peered":"2024-03-30T00:41:21.309392+0000","last_clean":"2024-03-30T00:41:21.309392+0000","last_became_active":"2024-03-30T00:41:21.300026+0000","last_became_peered":"2024-03-30T00:41:21.300026+0000","last_unstale":"2024-03-30T00:41:21.309392+0000","last_undegraded":"2024-03-30T00:41:21.309392+0000","last_fullsized":"2024-03-30T00:41:21.309392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:47:51.023501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539682+0000","last_change":"2024-03-30T00:41:20.887845+0000","last_active":"2024-03-30T00:41:22.539682+0000","last_peered":"2024-03-30T00:41:22.539682+0000","last_clean":"2024-03-30T00:41:22.539682+0000","last_became_active":"2024-03-30T00:41:20.887534+0000","last_became_peered":"2024-03-30T00:41:20.887534+0000","last_unstale":"2024-03-30T00:41:22.539682+0000","last_undegraded":"2024-03-30T00:41:22.539682+0000","last_fullsized":"2024-03-30T00:41:22.539682+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:54.221977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280222+0000","last_change":"2024-03-30T00:41:21.282088+0000","last_active":"2024-03-30T00:41:22.280222+0000","last_peered":"2024-03-30T00:41:22.280222+0000","last_clean":"2024-03-30T00:41:22.280222+0000","last_became_active":"2024-03-30T00:41:21.281824+0000","last_became_peered":"2024-03-30T00:41:21.281824+0000","last_unstale":"2024-03-30T00:41:22.280222+0000","last_undegraded":"2024-03-30T00:41:22.280222+0000","last_fullsized":"2024-03-30T00:41:22.280222+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:49.423423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.278557+0000","last_change":"2024-03-30T00:41:20.889020+0000","last_active":"2024-03-30T00:41:21.278557+0000","last_peered":"2024-03-30T00:41:21.278557+0000","last_clean":"2024-03-30T00:41:21.278557+0000","last_became_active":"2024-03-30T00:41:20.888715+0000","last_became_peered":"2024-03-30T00:41:20.888715+0000","last_unstale":"2024-03-30T00:41:21.278557+0000","last_undegraded":"2024-03-30T00:41:21.278557+0000","last_fullsized":"2024-03-30T00:41:21.278557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:25:47.454420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564708+0000","last_change":"2024-03-30T00:41:21.308399+0000","last_active":"2024-03-30T00:41:22.564708+0000","last_peered":"2024-03-30T00:41:22.564708+0000","last_clean":"2024-03-30T00:41:22.564708+0000","last_became_active":"2024-03-30T00:41:21.308159+0000","last_became_peered":"2024-03-30T00:41:21.308159+0000","last_unstale":"2024-03-30T00:41:22.564708+0000","last_undegraded":"2024-03-30T00:41:22.564708+0000","last_fullsized":"2024-03-30T00:41:22.564708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:55.227725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280798+0000","last_change":"2024-03-30T00:41:20.892397+0000","last_active":"2024-03-30T00:41:22.280798+0000","last_peered":"2024-03-30T00:41:22.280798+0000","last_clean":"2024-03-30T00:41:22.280798+0000","last_became_active":"2024-03-30T00:41:20.891100+0000","last_became_peered":"2024-03-30T00:41:20.891100+0000","last_unstale":"2024-03-30T00:41:22.280798+0000","last_undegraded":"2024-03-30T00:41:22.280798+0000","last_fullsized":"2024-03-30T00:41:22.280798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:14.578964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281718+0000","last_change":"2024-03-30T00:41:21.296856+0000","last_active":"2024-03-30T00:41:22.281718+0000","last_peered":"2024-03-30T00:41:22.281718+0000","last_clean":"2024-03-30T00:41:22.281718+0000","last_became_active":"2024-03-30T00:41:21.296608+0000","last_became_peered":"2024-03-30T00:41:21.296608+0000","last_unstale":"2024-03-30T00:41:22.281718+0000","last_undegraded":"2024-03-30T00:41:22.281718+0000","last_fullsized":"2024-03-30T00:41:22.281718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:33.233464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.282000+0000","last_change":"2024-03-30T00:41:20.882807+0000","last_active":"2024-03-30T00:41:22.282000+0000","last_peered":"2024-03-30T00:41:22.282000+0000","last_clean":"2024-03-30T00:41:22.282000+0000","last_became_active":"2024-03-30T00:41:20.882574+0000","last_became_peered":"2024-03-30T00:41:20.882574+0000","last_unstale":"2024-03-30T00:41:22.282000+0000","last_undegraded":"2024-03-30T00:41:22.282000+0000","last_fullsized":"2024-03-30T00:41:22.282000+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:04:30.552684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.306568+0000","last_change":"2024-03-30T00:41:21.297024+0000","last_active":"2024-03-30T00:41:21.306568+0000","last_peered":"2024-03-30T00:41:21.306568+0000","last_clean":"2024-03-30T00:41:21.306568+0000","last_became_active":"2024-03-30T00:41:21.296624+0000","last_became_peered":"2024-03-30T00:41:21.296624+0000","last_unstale":"2024-03-30T00:41:21.306568+0000","last_undegraded":"2024-03-30T00:41:21.306568+0000","last_fullsized":"2024-03-30T00:41:21.306568+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:56.894917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281855+0000","last_change":"2024-03-30T00:41:20.892928+0000","last_active":"2024-03-30T00:41:22.281855+0000","last_peered":"2024-03-30T00:41:22.281855+0000","last_clean":"2024-03-30T00:41:22.281855+0000","last_became_active":"2024-03-30T00:41:20.892662+0000","last_became_peered":"2024-03-30T00:41:20.892662+0000","last_unstale":"2024-03-30T00:41:22.281855+0000","last_undegraded":"2024-03-30T00:41:22.281855+0000","last_fullsized":"2024-03-30T00:41:22.281855+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:13.057222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.309932+0000","last_change":"2024-03-30T00:41:21.303722+0000","last_active":"2024-03-30T00:41:21.309932+0000","last_peered":"2024-03-30T00:41:21.309932+0000","last_clean":"2024-03-30T00:41:21.309932+0000","last_became_active":"2024-03-30T00:41:21.303414+0000","last_became_peered":"2024-03-30T00:41:21.303414+0000","last_unstale":"2024-03-30T00:41:21.309932+0000","last_undegraded":"2024-03-30T00:41:21.309932+0000","last_fullsized":"2024-03-30T00:41:21.309932+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:31:06.890668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280414+0000","last_change":"2024-03-30T00:41:20.887083+0000","last_active":"2024-03-30T00:41:22.280414+0000","last_peered":"2024-03-30T00:41:22.280414+0000","last_clean":"2024-03-30T00:41:22.280414+0000","last_became_active":"2024-03-30T00:41:20.886886+0000","last_became_peered":"2024-03-30T00:41:20.886886+0000","last_unstale":"2024-03-30T00:41:22.280414+0000","last_undegraded":"2024-03-30T00:41:22.280414+0000","last_fullsized":"2024-03-30T00:41:22.280414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:27:45.601075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.301369+0000","last_change":"2024-03-30T00:41:21.292119+0000","last_active":"2024-03-30T00:41:21.301369+0000","last_peered":"2024-03-30T00:41:21.301369+0000","last_clean":"2024-03-30T00:41:21.301369+0000","last_became_active":"2024-03-30T00:41:21.291836+0000","last_became_peered":"2024-03-30T00:41:21.291836+0000","last_unstale":"2024-03-30T00:41:21.301369+0000","last_undegraded":"2024-03-30T00:41:21.301369+0000","last_fullsized":"2024-03-30T00:41:21.301369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:47.922056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280353+0000","last_change":"2024-03-30T00:41:20.887809+0000","last_active":"2024-03-30T00:41:22.280353+0000","last_peered":"2024-03-30T00:41:22.280353+0000","last_clean":"2024-03-30T00:41:22.280353+0000","last_became_active":"2024-03-30T00:41:20.887603+0000","last_became_peered":"2024-03-30T00:41:20.887603+0000","last_unstale":"2024-03-30T00:41:22.280353+0000","last_undegraded":"2024-03-30T00:41:22.280353+0000","last_fullsized":"2024-03-30T00:41:22.280353+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:32.227716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280081+0000","last_change":"2024-03-30T00:41:21.293830+0000","last_active":"2024-03-30T00:41:22.280081+0000","last_peered":"2024-03-30T00:41:22.280081+0000","last_clean":"2024-03-30T00:41:22.280081+0000","last_became_active":"2024-03-30T00:41:21.293613+0000","last_became_peered":"2024-03-30T00:41:21.293613+0000","last_unstale":"2024-03-30T00:41:22.280081+0000","last_undegraded":"2024-03-30T00:41:22.280081+0000","last_fullsized":"2024-03-30T00:41:22.280081+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:33.066405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539677+0000","last_change":"2024-03-30T00:41:20.887853+0000","last_active":"2024-03-30T00:41:22.539677+0000","last_peered":"2024-03-30T00:41:22.539677+0000","last_clean":"2024-03-30T00:41:22.539677+0000","last_became_active":"2024-03-30T00:41:20.887552+0000","last_became_peered":"2024-03-30T00:41:20.887552+0000","last_unstale":"2024-03-30T00:41:22.539677+0000","last_undegraded":"2024-03-30T00:41:22.539677+0000","last_fullsized":"2024-03-30T00:41:22.539677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:28:56.650073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281072+0000","last_change":"2024-03-30T00:41:21.308393+0000","last_active":"2024-03-30T00:41:22.281072+0000","last_peered":"2024-03-30T00:41:22.281072+0000","last_clean":"2024-03-30T00:41:22.281072+0000","last_became_active":"2024-03-30T00:41:21.308109+0000","last_became_peered":"2024-03-30T00:41:21.308109+0000","last_unstale":"2024-03-30T00:41:22.281072+0000","last_undegraded":"2024-03-30T00:41:22.281072+0000","last_fullsized":"2024-03-30T00:41:22.281072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:06.394519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.292047+0000","last_change":"2024-03-30T00:41:20.883666+0000","last_active":"2024-03-30T00:41:21.292047+0000","last_peered":"2024-03-30T00:41:21.292047+0000","last_clean":"2024-03-30T00:41:21.292047+0000","last_became_active":"2024-03-30T00:41:20.881887+0000","last_became_peered":"2024-03-30T00:41:20.881887+0000","last_unstale":"2024-03-30T00:41:21.292047+0000","last_undegraded":"2024-03-30T00:41:21.292047+0000","last_fullsized":"2024-03-30T00:41:21.292047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:38:16.735996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280216+0000","last_change":"2024-03-30T00:41:21.295992+0000","last_active":"2024-03-30T00:41:22.280216+0000","last_peered":"2024-03-30T00:41:22.280216+0000","last_clean":"2024-03-30T00:41:22.280216+0000","last_became_active":"2024-03-30T00:41:21.295733+0000","last_became_peered":"2024-03-30T00:41:21.295733+0000","last_unstale":"2024-03-30T00:41:22.280216+0000","last_undegraded":"2024-03-30T00:41:22.280216+0000","last_fullsized":"2024-03-30T00:41:22.280216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:25:30.185412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280925+0000","last_change":"2024-03-30T00:41:20.882647+0000","last_active":"2024-03-30T00:41:22.280925+0000","last_peered":"2024-03-30T00:41:22.280925+0000","last_clean":"2024-03-30T00:41:22.280925+0000","last_became_active":"2024-03-30T00:41:20.882388+0000","last_became_peered":"2024-03-30T00:41:20.882388+0000","last_unstale":"2024-03-30T00:41:22.280925+0000","last_undegraded":"2024-03-30T00:41:22.280925+0000","last_fullsized":"2024-03-30T00:41:22.280925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:57.591715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280388+0000","last_change":"2024-03-30T00:41:21.293319+0000","last_active":"2024-03-30T00:41:22.280388+0000","last_peered":"2024-03-30T00:41:22.280388+0000","last_clean":"2024-03-30T00:41:22.280388+0000","last_became_active":"2024-03-30T00:41:21.293089+0000","last_became_peered":"2024-03-30T00:41:21.293089+0000","last_unstale":"2024-03-30T00:41:22.280388+0000","last_undegraded":"2024-03-30T00:41:22.280388+0000","last_fullsized":"2024-03-30T00:41:22.280388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:54:42.854939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281873+0000","last_change":"2024-03-30T00:41:20.891763+0000","last_active":"2024-03-30T00:41:22.281873+0000","last_peered":"2024-03-30T00:41:22.281873+0000","last_clean":"2024-03-30T00:41:22.281873+0000","last_became_active":"2024-03-30T00:41:20.891514+0000","last_became_peered":"2024-03-30T00:41:20.891514+0000","last_unstale":"2024-03-30T00:41:22.281873+0000","last_undegraded":"2024-03-30T00:41:22.281873+0000","last_fullsized":"2024-03-30T00:41:22.281873+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:56:12.781847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281801+0000","last_change":"2024-03-30T00:41:21.305382+0000","last_active":"2024-03-30T00:41:22.281801+0000","last_peered":"2024-03-30T00:41:22.281801+0000","last_clean":"2024-03-30T00:41:22.281801+0000","last_became_active":"2024-03-30T00:41:21.302669+0000","last_became_peered":"2024-03-30T00:41:21.302669+0000","last_unstale":"2024-03-30T00:41:22.281801+0000","last_undegraded":"2024-03-30T00:41:22.281801+0000","last_fullsized":"2024-03-30T00:41:22.281801+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:48:28.760437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.311624+0000","last_change":"2024-03-30T00:41:20.891669+0000","last_active":"2024-03-30T00:41:21.311624+0000","last_peered":"2024-03-30T00:41:21.311624+0000","last_clean":"2024-03-30T00:41:21.311624+0000","last_became_active":"2024-03-30T00:41:20.891408+0000","last_became_peered":"2024-03-30T00:41:20.891408+0000","last_unstale":"2024-03-30T00:41:21.311624+0000","last_undegraded":"2024-03-30T00:41:21.311624+0000","last_fullsized":"2024-03-30T00:41:21.311624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:13.057222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.310247+0000","last_change":"2024-03-30T00:41:21.301706+0000","last_active":"2024-03-30T00:41:21.310247+0000","last_peered":"2024-03-30T00:41:21.310247+0000","last_clean":"2024-03-30T00:41:21.310247+0000","last_became_active":"2024-03-30T00:41:21.301259+0000","last_became_peered":"2024-03-30T00:41:21.301259+0000","last_unstale":"2024-03-30T00:41:21.310247+0000","last_undegraded":"2024-03-30T00:41:21.310247+0000","last_fullsized":"2024-03-30T00:41:21.310247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:56.894917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564812+0000","last_change":"2024-03-30T00:41:20.883861+0000","last_active":"2024-03-30T00:41:22.564812+0000","last_peered":"2024-03-30T00:41:22.564812+0000","last_clean":"2024-03-30T00:41:22.564812+0000","last_became_active":"2024-03-30T00:41:20.882981+0000","last_became_peered":"2024-03-30T00:41:20.882981+0000","last_unstale":"2024-03-30T00:41:22.564812+0000","last_undegraded":"2024-03-30T00:41:22.564812+0000","last_fullsized":"2024-03-30T00:41:22.564812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:16.975551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564840+0000","last_change":"2024-03-30T00:41:21.312004+0000","last_active":"2024-03-30T00:41:22.564840+0000","last_peered":"2024-03-30T00:41:22.564840+0000","last_clean":"2024-03-30T00:41:22.564840+0000","last_became_active":"2024-03-30T00:41:21.311366+0000","last_became_peered":"2024-03-30T00:41:21.311366+0000","last_unstale":"2024-03-30T00:41:22.564840+0000","last_undegraded":"2024-03-30T00:41:22.564840+0000","last_fullsized":"2024-03-30T00:41:22.564840+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:35:11.629197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539817+0000","last_change":"2024-03-30T00:41:20.892029+0000","last_active":"2024-03-30T00:41:22.539817+0000","last_peered":"2024-03-30T00:41:22.539817+0000","last_clean":"2024-03-30T00:41:22.539817+0000","last_became_active":"2024-03-30T00:41:20.891783+0000","last_became_peered":"2024-03-30T00:41:20.891783+0000","last_unstale":"2024-03-30T00:41:22.539817+0000","last_undegraded":"2024-03-30T00:41:22.539817+0000","last_fullsized":"2024-03-30T00:41:22.539817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:16:20.546340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281063+0000","last_change":"2024-03-30T00:41:21.290167+0000","last_active":"2024-03-30T00:41:22.281063+0000","last_peered":"2024-03-30T00:41:22.281063+0000","last_clean":"2024-03-30T00:41:22.281063+0000","last_became_active":"2024-03-30T00:41:21.289571+0000","last_became_peered":"2024-03-30T00:41:21.289571+0000","last_unstale":"2024-03-30T00:41:22.281063+0000","last_undegraded":"2024-03-30T00:41:22.281063+0000","last_fullsized":"2024-03-30T00:41:22.281063+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:25:48.460169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564735+0000","last_change":"2024-03-30T00:41:20.889180+0000","last_active":"2024-03-30T00:41:22.564735+0000","last_peered":"2024-03-30T00:41:22.564735+0000","last_clean":"2024-03-30T00:41:22.564735+0000","last_became_active":"2024-03-30T00:41:20.888914+0000","last_became_peered":"2024-03-30T00:41:20.888914+0000","last_unstale":"2024-03-30T00:41:22.564735+0000","last_undegraded":"2024-03-30T00:41:22.564735+0000","last_fullsized":"2024-03-30T00:41:22.564735+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:05:06.481400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.306378+0000","last_change":"2024-03-30T00:41:21.296641+0000","last_active":"2024-03-30T00:41:21.306378+0000","last_peered":"2024-03-30T00:41:21.306378+0000","last_clean":"2024-03-30T00:41:21.306378+0000","last_became_active":"2024-03-30T00:41:21.296137+0000","last_became_peered":"2024-03-30T00:41:21.296137+0000","last_unstale":"2024-03-30T00:41:21.306378+0000","last_undegraded":"2024-03-30T00:41:21.306378+0000","last_fullsized":"2024-03-30T00:41:21.306378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:14.043465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564566+0000","last_change":"2024-03-30T00:41:20.880206+0000","last_active":"2024-03-30T00:41:22.564566+0000","last_peered":"2024-03-30T00:41:22.564566+0000","last_clean":"2024-03-30T00:41:22.564566+0000","last_became_active":"2024-03-30T00:41:20.879909+0000","last_became_peered":"2024-03-30T00:41:20.879909+0000","last_unstale":"2024-03-30T00:41:22.564566+0000","last_undegraded":"2024-03-30T00:41:22.564566+0000","last_fullsized":"2024-03-30T00:41:22.564566+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:38:16.735996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281336+0000","last_change":"2024-03-30T00:41:21.296861+0000","last_active":"2024-03-30T00:41:22.281336+0000","last_peered":"2024-03-30T00:41:22.281336+0000","last_clean":"2024-03-30T00:41:22.281336+0000","last_became_active":"2024-03-30T00:41:21.296531+0000","last_became_peered":"2024-03-30T00:41:21.296531+0000","last_unstale":"2024-03-30T00:41:22.281336+0000","last_undegraded":"2024-03-30T00:41:22.281336+0000","last_fullsized":"2024-03-30T00:41:22.281336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:54:01.879198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281142+0000","last_change":"2024-03-30T00:41:20.885091+0000","last_active":"2024-03-30T00:41:22.281142+0000","last_peered":"2024-03-30T00:41:22.281142+0000","last_clean":"2024-03-30T00:41:22.281142+0000","last_became_active":"2024-03-30T00:41:20.884826+0000","last_became_peered":"2024-03-30T00:41:20.884826+0000","last_unstale":"2024-03-30T00:41:22.281142+0000","last_undegraded":"2024-03-30T00:41:22.281142+0000","last_fullsized":"2024-03-30T00:41:22.281142+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:02:05.897618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280326+0000","last_change":"2024-03-30T00:41:21.284469+0000","last_active":"2024-03-30T00:41:22.280326+0000","last_peered":"2024-03-30T00:41:22.280326+0000","last_clean":"2024-03-30T00:41:22.280326+0000","last_became_active":"2024-03-30T00:41:21.284184+0000","last_became_peered":"2024-03-30T00:41:21.284184+0000","last_unstale":"2024-03-30T00:41:22.280326+0000","last_undegraded":"2024-03-30T00:41:22.280326+0000","last_fullsized":"2024-03-30T00:41:22.280326+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:38.690725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564634+0000","last_change":"2024-03-30T00:41:20.886308+0000","last_active":"2024-03-30T00:41:22.564634+0000","last_peered":"2024-03-30T00:41:22.564634+0000","last_clean":"2024-03-30T00:41:22.564634+0000","last_became_active":"2024-03-30T00:41:20.886023+0000","last_became_peered":"2024-03-30T00:41:20.886023+0000","last_unstale":"2024-03-30T00:41:22.564634+0000","last_undegraded":"2024-03-30T00:41:22.564634+0000","last_fullsized":"2024-03-30T00:41:22.564634+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:56.730386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539826+0000","last_change":"2024-03-30T00:41:21.298469+0000","last_active":"2024-03-30T00:41:22.539826+0000","last_peered":"2024-03-30T00:41:22.539826+0000","last_clean":"2024-03-30T00:41:22.539826+0000","last_became_active":"2024-03-30T00:41:21.298245+0000","last_became_peered":"2024-03-30T00:41:21.298245+0000","last_unstale":"2024-03-30T00:41:22.539826+0000","last_undegraded":"2024-03-30T00:41:22.539826+0000","last_fullsized":"2024-03-30T00:41:22.539826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:34:18.703475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281836+0000","last_change":"2024-03-30T00:41:20.892935+0000","last_active":"2024-03-30T00:41:22.281836+0000","last_peered":"2024-03-30T00:41:22.281836+0000","last_clean":"2024-03-30T00:41:22.281836+0000","last_became_active":"2024-03-30T00:41:20.892635+0000","last_became_peered":"2024-03-30T00:41:20.892635+0000","last_unstale":"2024-03-30T00:41:22.281836+0000","last_undegraded":"2024-03-30T00:41:22.281836+0000","last_fullsized":"2024-03-30T00:41:22.281836+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:32.524307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281886+0000","last_change":"2024-03-30T00:41:21.294703+0000","last_active":"2024-03-30T00:41:22.281886+0000","last_peered":"2024-03-30T00:41:22.281886+0000","last_clean":"2024-03-30T00:41:22.281886+0000","last_became_active":"2024-03-30T00:41:21.294409+0000","last_became_peered":"2024-03-30T00:41:21.294409+0000","last_unstale":"2024-03-30T00:41:22.281886+0000","last_undegraded":"2024-03-30T00:41:22.281886+0000","last_fullsized":"2024-03-30T00:41:22.281886+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:57:40.390938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.564843+0000","last_change":"2024-03-30T00:41:20.883926+0000","last_active":"2024-03-30T00:41:22.564843+0000","last_peered":"2024-03-30T00:41:22.564843+0000","last_clean":"2024-03-30T00:41:22.564843+0000","last_became_active":"2024-03-30T00:41:20.883619+0000","last_became_peered":"2024-03-30T00:41:20.883619+0000","last_unstale":"2024-03-30T00:41:22.564843+0000","last_undegraded":"2024-03-30T00:41:22.564843+0000","last_fullsized":"2024-03-30T00:41:22.564843+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:56:50.718787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280451+0000","last_change":"2024-03-30T00:41:21.291245+0000","last_active":"2024-03-30T00:41:22.280451+0000","last_peered":"2024-03-30T00:41:22.280451+0000","last_clean":"2024-03-30T00:41:22.280451+0000","last_became_active":"2024-03-30T00:41:21.291008+0000","last_became_peered":"2024-03-30T00:41:21.291008+0000","last_unstale":"2024-03-30T00:41:22.280451+0000","last_undegraded":"2024-03-30T00:41:22.280451+0000","last_fullsized":"2024-03-30T00:41:22.280451+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:04.232839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.288602+0000","last_change":"2024-03-30T00:41:20.891163+0000","last_active":"2024-03-30T00:41:21.288602+0000","last_peered":"2024-03-30T00:41:21.288602+0000","last_clean":"2024-03-30T00:41:21.288602+0000","last_became_active":"2024-03-30T00:41:20.890852+0000","last_became_peered":"2024-03-30T00:41:20.890852+0000","last_unstale":"2024-03-30T00:41:21.288602+0000","last_undegraded":"2024-03-30T00:41:21.288602+0000","last_fullsized":"2024-03-30T00:41:21.288602+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:05.388771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.307031+0000","last_change":"2024-03-30T00:41:21.302427+0000","last_active":"2024-03-30T00:41:21.307031+0000","last_peered":"2024-03-30T00:41:21.307031+0000","last_clean":"2024-03-30T00:41:21.307031+0000","last_became_active":"2024-03-30T00:41:21.302208+0000","last_became_peered":"2024-03-30T00:41:21.302208+0000","last_unstale":"2024-03-30T00:41:21.307031+0000","last_undegraded":"2024-03-30T00:41:21.307031+0000","last_fullsized":"2024-03-30T00:41:21.307031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:31:06.890668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.288835+0000","last_change":"2024-03-30T00:41:20.893139+0000","last_active":"2024-03-30T00:41:21.288835+0000","last_peered":"2024-03-30T00:41:21.288835+0000","last_clean":"2024-03-30T00:41:21.288835+0000","last_became_active":"2024-03-30T00:41:20.892041+0000","last_became_peered":"2024-03-30T00:41:20.892041+0000","last_unstale":"2024-03-30T00:41:21.288835+0000","last_undegraded":"2024-03-30T00:41:21.288835+0000","last_fullsized":"2024-03-30T00:41:21.288835+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:25:47.454420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.539621+0000","last_change":"2024-03-30T00:41:21.299126+0000","last_active":"2024-03-30T00:41:22.539621+0000","last_peered":"2024-03-30T00:41:22.539621+0000","last_clean":"2024-03-30T00:41:22.539621+0000","last_became_active":"2024-03-30T00:41:21.298853+0000","last_became_peered":"2024-03-30T00:41:21.298853+0000","last_unstale":"2024-03-30T00:41:22.539621+0000","last_undegraded":"2024-03-30T00:41:22.539621+0000","last_fullsized":"2024-03-30T00:41:22.539621+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:03.515291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.291974+0000","last_change":"2024-03-30T00:41:20.886790+0000","last_active":"2024-03-30T00:41:21.291974+0000","last_peered":"2024-03-30T00:41:21.291974+0000","last_clean":"2024-03-30T00:41:21.291974+0000","last_became_active":"2024-03-30T00:41:20.886541+0000","last_became_peered":"2024-03-30T00:41:20.886541+0000","last_unstale":"2024-03-30T00:41:21.291974+0000","last_undegraded":"2024-03-30T00:41:21.291974+0000","last_fullsized":"2024-03-30T00:41:21.291974+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:56.730386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.305175+0000","last_change":"2024-03-30T00:41:21.296348+0000","last_active":"2024-03-30T00:41:21.305175+0000","last_peered":"2024-03-30T00:41:21.305175+0000","last_clean":"2024-03-30T00:41:21.305175+0000","last_became_active":"2024-03-30T00:41:21.295366+0000","last_became_peered":"2024-03-30T00:41:21.295366+0000","last_unstale":"2024-03-30T00:41:21.305175+0000","last_undegraded":"2024-03-30T00:41:21.305175+0000","last_fullsized":"2024-03-30T00:41:21.305175+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:14.043465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.278875+0000","last_change":"2024-03-30T00:41:20.880904+0000","last_active":"2024-03-30T00:41:21.278875+0000","last_peered":"2024-03-30T00:41:21.278875+0000","last_clean":"2024-03-30T00:41:21.278875+0000","last_became_active":"2024-03-30T00:41:20.880463+0000","last_became_peered":"2024-03-30T00:41:20.880463+0000","last_unstale":"2024-03-30T00:41:21.278875+0000","last_undegraded":"2024-03-30T00:41:21.278875+0000","last_fullsized":"2024-03-30T00:41:21.278875+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:08:07.735075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281864+0000","last_change":"2024-03-30T00:41:21.303311+0000","last_active":"2024-03-30T00:41:22.281864+0000","last_peered":"2024-03-30T00:41:22.281864+0000","last_clean":"2024-03-30T00:41:22.281864+0000","last_became_active":"2024-03-30T00:41:21.298110+0000","last_became_peered":"2024-03-30T00:41:21.298110+0000","last_unstale":"2024-03-30T00:41:22.281864+0000","last_undegraded":"2024-03-30T00:41:22.281864+0000","last_fullsized":"2024-03-30T00:41:22.281864+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:15.584712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.280007+0000","last_change":"2024-03-30T00:41:20.893451+0000","last_active":"2024-03-30T00:41:22.280007+0000","last_peered":"2024-03-30T00:41:22.280007+0000","last_clean":"2024-03-30T00:41:22.280007+0000","last_became_active":"2024-03-30T00:41:20.892551+0000","last_became_peered":"2024-03-30T00:41:20.892551+0000","last_unstale":"2024-03-30T00:41:22.280007+0000","last_undegraded":"2024-03-30T00:41:22.280007+0000","last_fullsized":"2024-03-30T00:41:22.280007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:57:39.385189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281495+0000","last_change":"2024-03-30T00:41:21.307461+0000","last_active":"2024-03-30T00:41:22.281495+0000","last_peered":"2024-03-30T00:41:22.281495+0000","last_clean":"2024-03-30T00:41:22.281495+0000","last_became_active":"2024-03-30T00:41:21.307219+0000","last_became_peered":"2024-03-30T00:41:21.307219+0000","last_unstale":"2024-03-30T00:41:22.281495+0000","last_undegraded":"2024-03-30T00:41:22.281495+0000","last_fullsized":"2024-03-30T00:41:22.281495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:20.860508+0000","last_clean_scrub_stamp":"2024-03-30T00:41:20.860508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:31:06.890668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T00:41:22.281970+0000","last_change":"2024-03-30T00:41:20.884185+0000","last_active":"2024-03-30T00:41:22.281970+0000","last_peered":"2024-03-30T00:41:22.281970+0000","last_clean":"2024-03-30T00:41:22.281970+0000","last_became_active":"2024-03-30T00:41:20.883941+0000","last_became_peered":"2024-03-30T00:41:20.883941+0000","last_unstale":"2024-03-30T00:41:22.281970+0000","last_undegraded":"2024-03-30T00:41:22.281970+0000","last_fullsized":"2024-03-30T00:41:22.281970+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:41:19.854760+0000","last_clean_scrub_stamp":"2024-03-30T00:41:19.854760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:04:14.843309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T00:41:21.280171+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:41:21.280171+0000","last_peered":"2024-03-30T00:41:21.280171+0000","last_clean":"2024-03-30T00:41:21.280171+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:41:21.280171+0000","last_undegraded":"2024-03-30T00:41:21.280171+0000","last_fullsized":"2024-03-30T00:41:21.280171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1073152,"data_stored":791120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.69699999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.65000000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.60999999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.71499999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67900000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.73299999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.745}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":454656,"data_stored":193080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.0490000000000004,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.66500000000000004}]},{"osd":1,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.3170000000000002,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.4160000000000004,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.63}]},{"osd":2,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":4.375,"5min":4.375,"15min":4.375},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.2960000000000003,"5min":4.2960000000000003,"15min":4.2960000000000003},"last":0.78500000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.3399999999999999,"5min":4.3399999999999999,"15min":4.3399999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.6020000000000003,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.61399999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.82599999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80500000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.65200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":471040,"data_stored":199230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.75700000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.752}]},{"osd":7,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.92800000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":458752,"data_stored":193102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.59299999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.495},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.55200000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.60999999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.68700000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":200716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.29699999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42299999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.76500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":192568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.49199999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.56000000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.127}]},{"osd":5,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.879}]},{"osd":6,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.179}]},{"osd":7,"last update":"Sat Mar 30 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.01},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.0700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":192588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.501}]},{"osd":2,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.54900000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.61099999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.77100000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.871}]},{"osd":6,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79600000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.82199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":782900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.39600000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.83799999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.78300000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.81100000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.746}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:41:25.779 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T00:41:25.779 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-30T00:41:25.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f8695a00640 1 Processor -- start 2024-03-30T00:41:25.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f8695a00640 1 -- start start 2024-03-30T00:41:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f8695a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f86900cda60 0x7f869006b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f8695a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86900cf140 con 0x7f869015cba0 2024-03-30T00:41:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f8695a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86900cdee0 con 0x7f86900d0530 2024-03-30T00:41:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f8695a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869006bdc0 con 0x7f86900cda60 2024-03-30T00:41:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f868e000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f86900cda60 0x7f869006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f868ea00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f869015cba0 0x7f86900cf5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39420/0 (socket says 172.21.15.66:39420) 2024-03-30T00:41:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f868ea00640 1 -- 172.21.15.66:0/402135763 learned_addr learned my addr 172.21.15.66:0/402135763 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f868e000640 1 -- 172.21.15.66:0/402135763 >> v1:172.21.15.156:6789/0 conn(0x7f86900d0530 legacy=0x7f86900b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f868e000640 1 -- 172.21.15.66:0/402135763 >> v1:172.21.15.66:6789/0 conn(0x7f869015cba0 legacy=0x7f86900cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.953+0000 7f868e000640 1 -- 172.21.15.66:0/402135763 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869006bfe0 con 0x7f86900cda60 2024-03-30T00:41:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.957+0000 7f868e000640 1 --2- 172.21.15.66:0/402135763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86900cda60 0x7f869006b880 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f867c0097c0 tx=0x7f867c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee9ee06046e3ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:25.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.957+0000 7f868d600640 1 -- 172.21.15.66:0/402135763 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c0ac410 con 0x7f86900cda60 2024-03-30T00:41:25.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.957+0000 7f868d600640 1 -- 172.21.15.66:0/402135763 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c0ac570 con 0x7f86900cda60 2024-03-30T00:41:25.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.957+0000 7f868d600640 1 -- 172.21.15.66:0/402135763 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c0ac900 con 0x7f86900cda60 2024-03-30T00:41:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.957+0000 7f8695a00640 1 -- 172.21.15.66:0/402135763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86900cda60 msgr2=0x7f869006b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.957+0000 7f8695a00640 1 --2- 172.21.15.66:0/402135763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86900cda60 0x7f869006b880 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f867c0097c0 tx=0x7f867c01c4a0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.957+0000 7f8695a00640 1 -- 172.21.15.66:0/402135763 shutdown_connections 2024-03-30T00:41:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.957+0000 7f8695a00640 1 --2- 172.21.15.66:0/402135763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86900cda60 0x7f869006b880 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.957+0000 7f8695a00640 1 -- 172.21.15.66:0/402135763 >> 172.21.15.66:0/402135763 conn(0x7f8690089fb0 msgr2=0x7f869008a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:25.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.957+0000 7f8695a00640 1 -- 172.21.15.66:0/402135763 shutdown_connections 2024-03-30T00:41:25.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.957+0000 7f8695a00640 1 -- 172.21.15.66:0/402135763 wait complete. 2024-03-30T00:41:25.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8695a00640 1 Processor -- start 2024-03-30T00:41:25.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8695a00640 1 -- start start 2024-03-30T00:41:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8695a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86900cda60 0x7f869014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8695a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86900d0530 0x7f869015d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8695a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f869015cba0 0x7f869015f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8695a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869014e980 con 0x7f869015cba0 2024-03-30T00:41:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8695a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86900b4da0 con 0x7f86900cda60 2024-03-30T00:41:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8695a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86900b4190 con 0x7f86900d0530 2024-03-30T00:41:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f868f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f869015cba0 0x7f869015f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f868f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f869015cba0 0x7f869015f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57800/0 (socket says 172.21.15.66:57800) 2024-03-30T00:41:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f868f400640 1 -- 172.21.15.66:0/3119456966 learned_addr learned my addr 172.21.15.66:0/3119456966 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f868ea00640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86900cda60 0x7f869014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f868e000640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86900d0530 0x7f869015d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f868f400640 1 -- 172.21.15.66:0/3119456966 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86900d0530 msgr2=0x7f869015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f868f400640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86900d0530 0x7f869015d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f868f400640 1 -- 172.21.15.66:0/3119456966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86900cda60 msgr2=0x7f869014e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f868f400640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86900cda60 0x7f869014e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f868f400640 1 -- 172.21.15.66:0/3119456966 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c009420 con 0x7f869015cba0 2024-03-30T00:41:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f868f400640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f869015cba0 0x7f869015f510 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f868800c7a0 tx=0x7f868800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8686a00640 1 -- 172.21.15.66:0/3119456966 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688003b80 con 0x7f869015cba0 2024-03-30T00:41:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8686a00640 1 -- 172.21.15.66:0/3119456966 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8688003ce0 con 0x7f869015cba0 2024-03-30T00:41:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8686a00640 1 -- 172.21.15.66:0/3119456966 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800aa90 con 0x7f869015cba0 2024-03-30T00:41:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8695a00640 1 -- 172.21.15.66:0/3119456966 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86900d0ed0 con 0x7f869015cba0 2024-03-30T00:41:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.961+0000 7f8695a00640 1 -- 172.21.15.66:0/3119456966 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86900b4750 con 0x7f869015cba0 2024-03-30T00:41:25.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.965+0000 7f8695a00640 1 -- 172.21.15.66:0/3119456966 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86900b4f30 con 0x7f869015cba0 2024-03-30T00:41:25.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.965+0000 7f8686a00640 1 -- 172.21.15.66:0/3119456966 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f868800abf0 con 0x7f869015cba0 2024-03-30T00:41:25.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.965+0000 7f8686a00640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8670066690 0x7f8670068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.965+0000 7f8686a00640 1 -- 172.21.15.66:0/3119456966 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f86880840e0 con 0x7f869015cba0 2024-03-30T00:41:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.965+0000 7f868ea00640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8670066690 0x7f8670068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.965+0000 7f868ea00640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8670066690 0x7f8670068b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8680005890 tx=0x7f8680009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:25.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:25.969+0000 7f8686a00640 1 -- 172.21.15.66:0/3119456966 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8688051290 con 0x7f869015cba0 2024-03-30T00:41:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.185+0000 7f8695a00640 1 -- 172.21.15.66:0/3119456966 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8690066c30 con 0x7f869015cba0 2024-03-30T00:41:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.189+0000 7f8686a00640 1 -- 172.21.15.66:0/3119456966 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7f86880548b0 con 0x7f869015cba0 2024-03-30T00:41:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.189+0000 7f8695a00640 1 -- 172.21.15.66:0/3119456966 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8670066690 msgr2=0x7f8670068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.189+0000 7f8695a00640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8670066690 0x7f8670068b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8680005890 tx=0x7f8680009040 comp rx=0 tx=0).stop 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.189+0000 7f8695a00640 1 -- 172.21.15.66:0/3119456966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f869015cba0 msgr2=0x7f869015f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.189+0000 7f8695a00640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f869015cba0 0x7f869015f510 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f868800c7a0 tx=0x7f868800cc60 comp rx=0 tx=0).stop 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.189+0000 7f8695a00640 1 -- 172.21.15.66:0/3119456966 shutdown_connections 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.189+0000 7f8695a00640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8670066690 0x7f8670068b50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.189+0000 7f8695a00640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f869015cba0 0x7f869015f510 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.189+0000 7f8695a00640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86900d0530 0x7f869015d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.189+0000 7f8695a00640 1 --2- 172.21.15.66:0/3119456966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86900cda60 0x7f869014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.189+0000 7f8695a00640 1 -- 172.21.15.66:0/3119456966 >> 172.21.15.66:0/3119456966 conn(0x7f8690089fb0 msgr2=0x7f869015bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.193+0000 7f8695a00640 1 -- 172.21.15.66:0/3119456966 shutdown_connections 2024-03-30T00:41:26.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.193+0000 7f8695a00640 1 -- 172.21.15.66:0/3119456966 wait complete. 2024-03-30T00:41:26.220 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:26.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.349+0000 7fb04be00640 1 Processor -- start 2024-03-30T00:41:26.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb04be00640 1 -- start start 2024-03-30T00:41:26.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb04be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c15aa80 0x7fb04c0d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:26.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb04be00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c15eea0 con 0x7fb04c0d0870 2024-03-30T00:41:26.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb04be00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c15f180 con 0x7fb04c15a6a0 2024-03-30T00:41:26.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb04be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0d0d20 con 0x7fb04c15aa80 2024-03-30T00:41:26.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb04b400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb04c0d0870 0x7fb04c0ce9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39430/0 (socket says 172.21.15.66:39430) 2024-03-30T00:41:26.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb04b400640 1 -- 172.21.15.66:0/528522146 learned_addr learned my addr 172.21.15.66:0/528522146 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:26.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb043e00640 1 --2- 172.21.15.66:0/528522146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c15aa80 0x7fb04c0d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:26.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb04a000640 1 -- 172.21.15.66:0/528522146 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99634797 0 0) 0x7fb04c15eea0 con 0x7fb04c0d0870 2024-03-30T00:41:26.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb04a000640 1 -- 172.21.15.66:0/528522146 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb030003530 con 0x7fb04c0d0870 2024-03-30T00:41:26.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb04a000640 1 -- 172.21.15.66:0/528522146 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172775160 0 0) 0x7fb04c15f180 con 0x7fb04c15a6a0 2024-03-30T00:41:26.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb04a000640 1 -- 172.21.15.66:0/528522146 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb04c15eea0 con 0x7fb04c15a6a0 2024-03-30T00:41:26.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb043e00640 1 -- 172.21.15.66:0/528522146 >> v1:172.21.15.156:6789/0 conn(0x7fb04c15a6a0 legacy=0x7fb04c15e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:26.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb043e00640 1 -- 172.21.15.66:0/528522146 >> v1:172.21.15.66:6789/0 conn(0x7fb04c0d0870 legacy=0x7fb04c0ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:26.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.353+0000 7fb043e00640 1 -- 172.21.15.66:0/528522146 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c0ce1a0 con 0x7fb04c15aa80 2024-03-30T00:41:26.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb043e00640 1 --2- 172.21.15.66:0/528522146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c15aa80 0x7fb04c0d0330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0340097f0 tx=0x7fb03401ceb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dca7f877bcb6ad84 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:26.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04a000640 1 -- 172.21.15.66:0/528522146 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340ad660 con 0x7fb04c15aa80 2024-03-30T00:41:26.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04a000640 1 -- 172.21.15.66:0/528522146 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0340ad7c0 con 0x7fb04c15aa80 2024-03-30T00:41:26.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04a000640 1 -- 172.21.15.66:0/528522146 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340adb50 con 0x7fb04c15aa80 2024-03-30T00:41:26.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04be00640 1 -- 172.21.15.66:0/528522146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c15aa80 msgr2=0x7fb04c0d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:26.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04be00640 1 --2- 172.21.15.66:0/528522146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c15aa80 0x7fb04c0d0330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0340097f0 tx=0x7fb03401ceb0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04be00640 1 -- 172.21.15.66:0/528522146 shutdown_connections 2024-03-30T00:41:26.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04be00640 1 --2- 172.21.15.66:0/528522146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c15aa80 0x7fb04c0d0330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04be00640 1 -- 172.21.15.66:0/528522146 >> 172.21.15.66:0/528522146 conn(0x7fb04c06e0f0 msgr2=0x7fb04c06e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:26.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04be00640 1 -- 172.21.15.66:0/528522146 shutdown_connections 2024-03-30T00:41:26.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04be00640 1 -- 172.21.15.66:0/528522146 wait complete. 2024-03-30T00:41:26.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04be00640 1 Processor -- start 2024-03-30T00:41:26.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.357+0000 7fb04be00640 1 -- start start 2024-03-30T00:41:26.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb04be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c0d0870 0x7fb04c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:26.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb04be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c15a6a0 0x7fb04c152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:26.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb04be00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb04c15aa80 0x7fb04c153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:26.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb04be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0c4650 con 0x7fb04c15a6a0 2024-03-30T00:41:26.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb04be00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c14f3b0 con 0x7fb04c15aa80 2024-03-30T00:41:26.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb04be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c15d4b0 con 0x7fb04c0d0870 2024-03-30T00:41:26.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb043e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c15a6a0 0x7fb04c152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:26.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb043e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c15a6a0 0x7fb04c152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57806/0 (socket says 172.21.15.66:57806) 2024-03-30T00:41:26.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb043e00640 1 -- 172.21.15.66:0/1797585300 learned_addr learned my addr 172.21.15.66:0/1797585300 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:26.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb04aa00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c0d0870 0x7fb04c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:26.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb04b400640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb04c15aa80 0x7fb04c153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:26.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb043e00640 1 -- 172.21.15.66:0/1797585300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c0d0870 msgr2=0x7fb04c0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:26.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb043e00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c0d0870 0x7fb04c0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb043e00640 1 -- 172.21.15.66:0/1797585300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb04c15aa80 msgr2=0x7fb04c153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:26.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb043e00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb04c15aa80 0x7fb04c153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb043e00640 1 -- 172.21.15.66:0/1797585300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034009420 con 0x7fb04c15a6a0 2024-03-30T00:41:26.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb043e00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c15a6a0 0x7fb04c152ac0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb034019060 tx=0x7fb0340191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:26.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb042a00640 1 -- 172.21.15.66:0/1797585300 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340b6610 con 0x7fb04c15a6a0 2024-03-30T00:41:26.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb042a00640 1 -- 172.21.15.66:0/1797585300 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0340b6770 con 0x7fb04c15a6a0 2024-03-30T00:41:26.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb042a00640 1 -- 172.21.15.66:0/1797585300 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340b6b20 con 0x7fb04c15a6a0 2024-03-30T00:41:26.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb04be00640 1 -- 172.21.15.66:0/1797585300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c15d790 con 0x7fb04c15a6a0 2024-03-30T00:41:26.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb04be00640 1 -- 172.21.15.66:0/1797585300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c15d120 con 0x7fb04c15a6a0 2024-03-30T00:41:26.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.361+0000 7fb04be00640 1 -- 172.21.15.66:0/1797585300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04c073cf0 con 0x7fb04c15a6a0 2024-03-30T00:41:26.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.365+0000 7fb042a00640 1 -- 172.21.15.66:0/1797585300 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb0340b4040 con 0x7fb04c15a6a0 2024-03-30T00:41:26.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.365+0000 7fb042a00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb028066690 0x7fb028068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:26.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.365+0000 7fb042a00640 1 -- 172.21.15.66:0/1797585300 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb03411e550 con 0x7fb04c15a6a0 2024-03-30T00:41:26.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.365+0000 7fb04aa00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb028066690 0x7fb028068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:26.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.365+0000 7fb04aa00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb028066690 0x7fb028068b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0011f0 tx=0x7fb03c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:26.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.369+0000 7fb042a00640 1 -- 172.21.15.66:0/1797585300 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb0340eb820 con 0x7fb04c15a6a0 2024-03-30T00:41:26.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.565+0000 7fb04be00640 1 -- 172.21.15.66:0/1797585300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb04c0d1230 con 0x7fb04c15a6a0 2024-03-30T00:41:26.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.565+0000 7fb042a00640 1 -- 172.21.15.66:0/1797585300 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fb0340eee40 con 0x7fb04c15a6a0 2024-03-30T00:41:26.575 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:26.576 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3895007156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3895007156},{"type":"v1","addr":"172.21.15.156:6835","nonce":3895007156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/65993387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":65993387},{"type":"v1","addr":"172.21.15.156:6834","nonce":65993387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2013605775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2013605775},{"type":"v1","addr":"172.21.15.66:6835","nonce":2013605775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:41:21.269965+0000","modified":"2024-03-30T00:41:24.283171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1694329026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1694329026},{"type":"v1","addr":"172.21.15.66:6837","nonce":1694329026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-30T00:41:26.576 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-03-30T00:41:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 -- 172.21.15.66:0/1797585300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb028066690 msgr2=0x7fb028068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb028066690 0x7fb028068b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0011f0 tx=0x7fb03c009040 comp rx=0 tx=0).stop 2024-03-30T00:41:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 -- 172.21.15.66:0/1797585300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c15a6a0 msgr2=0x7fb04c152ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c15a6a0 0x7fb04c152ac0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb034019060 tx=0x7fb0340191f0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 -- 172.21.15.66:0/1797585300 shutdown_connections 2024-03-30T00:41:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb04c15aa80 0x7fb04c153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb028066690 0x7fb028068b50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c15a6a0 0x7fb04c152ac0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 --2- 172.21.15.66:0/1797585300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c0d0870 0x7fb04c0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 -- 172.21.15.66:0/1797585300 >> 172.21.15.66:0/1797585300 conn(0x7fb04c06e0f0 msgr2=0x7fb04c06ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:26.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 -- 172.21.15.66:0/1797585300 shutdown_connections 2024-03-30T00:41:26.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:26.569+0000 7fb04be00640 1 -- 172.21.15.66:0/1797585300 wait complete. 2024-03-30T00:41:26.999 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:41:21.269965+0000', 'modified': '2024-03-30T00:41:24.283171+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5490': {'gid': 5490, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6837/1694329026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 1694329026}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 1694329026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:41:26.999 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:41:26.999 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T00:41:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.137+0000 7f86fce00640 1 Processor -- start 2024-03-30T00:41:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.137+0000 7f86fce00640 1 -- start start 2024-03-30T00:41:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.137+0000 7f86fce00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86f80cc500 0x7f86f80975d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.137+0000 7f86fce00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f80abe30 con 0x7f86f80cad00 2024-03-30T00:41:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.137+0000 7f86fce00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f80cb270 con 0x7f86f80cc120 2024-03-30T00:41:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.137+0000 7f86fce00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80cb550 con 0x7f86f80cc500 2024-03-30T00:41:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.137+0000 7f86ed400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f86f80cad00 0x7f86f80ac2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39432/0 (socket says 172.21.15.66:39432) 2024-03-30T00:41:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.137+0000 7f86ed400640 1 -- 172.21.15.66:0/3607760359 learned_addr learned my addr 172.21.15.66:0/3607760359 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.137+0000 7f86f6800640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86f80cc500 0x7f86f80975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.137+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161672564 0 0) 0x7f86f80abe30 con 0x7f86f80cad00 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86dc003510 con 0x7f86f80cad00 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422994588 0 0) 0x7f86f80cb270 con 0x7f86f80cc120 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f80abe30 con 0x7f86f80cc120 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2521265053 0 0) 0x7f86dc003510 con 0x7f86f80cad00 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86f80cc500 msgr2=0x7f86f80975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86f80cc500 0x7f86f80975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 >> v1:172.21.15.156:6789/0 conn(0x7f86f80cc120 legacy=0x7f86f806c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f80cb830 con 0x7f86f80cad00 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f86e0001680 con 0x7f86f80cad00 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 >> v1:172.21.15.66:6789/0 conn(0x7f86f80cad00 legacy=0x7f86f80ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86dc003ae0 0x7f86dc003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86dc008d60 0x7f86dc00b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86f80cc500 0x7f86dc00d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80cb550 con 0x7f86dc008d60 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc00dcc0 con 0x7f86dc003ae0 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc00de80 con 0x7f86f80cc500 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86ed400640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86dc008d60 0x7f86dc00b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5e00640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86dc003ae0 0x7f86dc003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f6800640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86f80cc500 0x7f86dc00d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86ed400640 1 -- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86f80cc500 msgr2=0x7f86dc00d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86ed400640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86f80cc500 0x7f86dc00d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86ed400640 1 -- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86dc003ae0 msgr2=0x7f86dc003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86ed400640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86dc003ae0 0x7f86dc003ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86ed400640 1 -- 172.21.15.66:0/3607760359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc003510 con 0x7f86dc008d60 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f6800640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86f80cc500 0x7f86dc00d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.157+0000 7f86f5e00640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86dc003ae0 0x7f86dc003ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86ed400640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86dc008d60 0x7f86dc00b150 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f86e0001ec0 tx=0x7f86e001cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eba59ca3dd30265 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00058e0 con 0x7f86dc008d60 2024-03-30T00:41:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e0005a60 con 0x7f86dc008d60 2024-03-30T00:41:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86f5400640 1 -- 172.21.15.66:0/3607760359 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0005e10 con 0x7f86dc008d60 2024-03-30T00:41:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86fce00640 1 -- 172.21.15.66:0/3607760359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86dc008d60 msgr2=0x7f86dc00b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86fce00640 1 --2- 172.21.15.66:0/3607760359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86dc008d60 0x7f86dc00b150 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f86e0001ec0 tx=0x7f86e001cb80 comp rx=0 tx=0).stop 2024-03-30T00:41:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86f6800640 1 -- 172.21.15.66:0/3607760359 reap_dead start 2024-03-30T00:41:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86fce00640 1 -- 172.21.15.66:0/3607760359 shutdown_connections 2024-03-30T00:41:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86fce00640 1 -- 172.21.15.66:0/3607760359 >> 172.21.15.66:0/3607760359 conn(0x7f86f80ce2f0 msgr2=0x7f86f80ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86fce00640 1 -- 172.21.15.66:0/3607760359 shutdown_connections 2024-03-30T00:41:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86fce00640 1 -- 172.21.15.66:0/3607760359 wait complete. 2024-03-30T00:41:27.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.161+0000 7f86fce00640 1 Processor -- start 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86fce00640 1 -- start start 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86fce00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86dc008d60 0x7f86f80c4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86fce00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f80b0f50 0x7f86f8091900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86fce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f8093310 0x7f86f81f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86fce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8092ef0 con 0x7f86f8093310 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86fce00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8092270 con 0x7f86f80b0f50 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86fce00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8092550 con 0x7f86dc008d60 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86f6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f8093310 0x7f86f81f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86f6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f8093310 0x7f86f81f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57824/0 (socket says 172.21.15.66:57824) 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86f6800640 1 -- 172.21.15.66:0/2427910466 learned_addr learned my addr 172.21.15.66:0/2427910466 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86ed400640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f80b0f50 0x7f86f8091900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86f5e00640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86dc008d60 0x7f86f80c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86f6800640 1 -- 172.21.15.66:0/2427910466 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86dc008d60 msgr2=0x7f86f80c4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86f6800640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86dc008d60 0x7f86f80c4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86f6800640 1 -- 172.21.15.66:0/2427910466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f80b0f50 msgr2=0x7f86f8091900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86f6800640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f80b0f50 0x7f86f8091900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86f6800640 1 -- 172.21.15.66:0/2427910466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f8092830 con 0x7f86f8093310 2024-03-30T00:41:27.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86f6800640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f8093310 0x7f86f81f4540 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f86f80cef00 tx=0x7f86f000e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:27.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86eea00640 1 -- 172.21.15.66:0/2427910466 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0013790 con 0x7f86f8093310 2024-03-30T00:41:27.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86eea00640 1 -- 172.21.15.66:0/2427910466 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f00138f0 con 0x7f86f8093310 2024-03-30T00:41:27.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86eea00640 1 -- 172.21.15.66:0/2427910466 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0013ca0 con 0x7f86f8093310 2024-03-30T00:41:27.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86fce00640 1 -- 172.21.15.66:0/2427910466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f8092b10 con 0x7f86f8093310 2024-03-30T00:41:27.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.165+0000 7f86fce00640 1 -- 172.21.15.66:0/2427910466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f8091e40 con 0x7f86f8093310 2024-03-30T00:41:27.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86eea00640 1 -- 172.21.15.66:0/2427910466 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f86f001e460 con 0x7f86f8093310 2024-03-30T00:41:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86eea00640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86d4066690 0x7f86d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86eea00640 1 -- 172.21.15.66:0/2427910466 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f86f0083d00 con 0x7f86f8093310 2024-03-30T00:41:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f5e00640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86d4066690 0x7f86d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f8700278000 1 Processor -- start 2024-03-30T00:41:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f8700278000 1 -- start start 2024-03-30T00:41:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f5e00640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86d4066690 0x7f86d4068b50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86f80b2830 tx=0x7f86e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f8700278000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db3c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f8700278000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3cb30 0x55a23daf2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f8700278000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23daf28d0 0x55a23daf2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f8700278000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23d9b65a0 con 0x55a23db3c210 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f8700278000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23d943af0 con 0x55a23daf28d0 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f8700278000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a23d9410c0 con 0x55a23db3cb30 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db3c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db3c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57840/0 (socket says 172.21.15.66:57840) 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f5e00640 1 -- 172.21.15.66:0/2163562950 learned_addr learned my addr 172.21.15.66:0/2163562950 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f6800640 1 --2- 172.21.15.66:0/2163562950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23daf28d0 0x55a23daf2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86ed400640 1 --2- 172.21.15.66:0/2163562950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3cb30 0x55a23daf2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f5e00640 1 -- 172.21.15.66:0/2163562950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3cb30 msgr2=0x55a23daf2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f5e00640 1 --2- 172.21.15.66:0/2163562950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3cb30 0x55a23daf2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f5e00640 1 -- 172.21.15.66:0/2163562950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23daf28d0 msgr2=0x55a23daf2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f5e00640 1 --2- 172.21.15.66:0/2163562950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23daf28d0 0x55a23daf2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f5e00640 1 -- 172.21.15.66:0/2163562950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a23d638890 con 0x55a23db3c210 2024-03-30T00:41:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86ed400640 1 --2- 172.21.15.66:0/2163562950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3cb30 0x55a23daf2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:27.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.169+0000 7f86f6800640 1 --2- 172.21.15.66:0/2163562950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23daf28d0 0x55a23daf2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:27.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86f5e00640 1 --2- 172.21.15.66:0/2163562950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db3c5f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f86e80164a0 tx=0x7f86e8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdb02d427c4b2fa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86e6a00640 1 -- 172.21.15.66:0/2163562950 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e801b040 con 0x55a23db3c210 2024-03-30T00:41:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86e6a00640 1 -- 172.21.15.66:0/2163562950 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e80225c0 con 0x55a23db3c210 2024-03-30T00:41:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86e6a00640 1 -- 172.21.15.66:0/2163562950 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8022930 con 0x55a23db3c210 2024-03-30T00:41:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 -- 172.21.15.66:0/2163562950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 msgr2=0x55a23db3c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 --2- 172.21.15.66:0/2163562950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db3c5f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f86e80164a0 tx=0x7f86e8016000 comp rx=0 tx=0).stop 2024-03-30T00:41:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 -- 172.21.15.66:0/2163562950 shutdown_connections 2024-03-30T00:41:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 --2- 172.21.15.66:0/2163562950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23daf28d0 0x55a23daf2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 --2- 172.21.15.66:0/2163562950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3cb30 0x55a23daf2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 --2- 172.21.15.66:0/2163562950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db3c5f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 -- 172.21.15.66:0/2163562950 >> 172.21.15.66:0/2163562950 conn(0x55a23daf4830 msgr2=0x55a23daec860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:27.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 -- 172.21.15.66:0/2163562950 shutdown_connections 2024-03-30T00:41:27.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 -- 172.21.15.66:0/2163562950 wait complete. 2024-03-30T00:41:27.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 -- start start 2024-03-30T00:41:27.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 Processor -- start 2024-03-30T00:41:27.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23daf28d0 0x55a23db2a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db2a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3cb30 0x55a23db2aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23d93c9b0 con 0x55a23db3c210 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23d8dbc00 con 0x55a23daf28d0 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a23d8fa070 con 0x55a23db3cb30 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f8700278000 10 client.0 _ll_register_callbacks cb 0x55a23dada0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86ed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db2a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86ed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db2a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57848/0 (socket says 172.21.15.66:57848) 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86ed400640 1 -- 172.21.15.66:0/3414298721 learned_addr learned my addr 172.21.15.66:0/3414298721 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86f5e00640 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23daf28d0 0x55a23db2a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86f6800640 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3cb30 0x55a23db2aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86ed400640 1 -- 172.21.15.66:0/3414298721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3cb30 msgr2=0x55a23db2aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86ed400640 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3cb30 0x55a23db2aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86ed400640 1 -- 172.21.15.66:0/3414298721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23daf28d0 msgr2=0x55a23db2a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86ed400640 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23daf28d0 0x55a23db2a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86ed400640 1 -- 172.21.15.66:0/3414298721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e800f030 con 0x55a23db3c210 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86ed400640 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db2a5a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f86e001c9b0 tx=0x7f86e001b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a4d89b707ff26b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86db400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.173+0000 7f86db400640 1 -- 172.21.15.66:0/3414298721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00b3040 con 0x55a23db3c210 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f86db400640 1 -- 172.21.15.66:0/3414298721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f86e00babf0 con 0x55a23db3c210 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 -- 172.21.15.66:0/3414298721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a23d8f95f0 con 0x55a23db3c210 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f86db400640 1 -- 172.21.15.66:0/3414298721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86e0005ea0 con 0x55a23db3c210 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 10 client.5673 fetch_fsmap learned FSMap version 17 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 -- 172.21.15.66:0/3414298721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a23d901ab0 con 0x55a23db3c210 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f86db400640 1 -- 172.21.15.66:0/3414298721 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a23d8f95f0 con 0x55a23db3c210 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f86db400640 1 -- 172.21.15.66:0/3414298721 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f86e00c0020 con 0x55a23db3c210 2024-03-30T00:41:27.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 10 client.5673 fetch_fsmap finished waiting for FSMap version 17 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 10 client.5673 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 20 client.5673 populate_metadata read hostname 'smithi066' 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.66:6836/1694329026,v1:172.21.15.66:6837/1694329026] conn(0x55a23db2b380 0x55a23db4d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 4 client.5673 mds_command: new command op to 5490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 -- 172.21.15.66:0/3414298721 --> [v2:172.21.15.66:6836/1694329026,v1:172.21.15.66:6837/1694329026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a23d939790 con 0x55a23db2b380 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f86f5e00640 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.66:6836/1694329026,v1:172.21.15.66:6837/1694329026] conn(0x55a23db2b380 0x55a23db4d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f86db400640 10 client.5673 ms_handle_connect on v2:172.21.15.66:6836/1694329026 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f86f5e00640 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.66:6836/1694329026,v1:172.21.15.66:6837/1694329026] conn(0x55a23db2b380 0x55a23db4d780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbcbbdee2b2d12f server_cookie=8c2e4ca4369c6ebd in_seq=0 out_seq=0 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f86db400640 1 -- 172.21.15.66:0/3414298721 <== mds.0 v2:172.21.15.66:6836/1694329026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a23d939790 con 0x55a23db2b380 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f86db400640 10 client.5673 handle_command_reply: tid=0 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 client.5673 shutdown 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 -- 172.21.15.66:0/3414298721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 msgr2=0x55a23db2a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db2a5a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f86e001c9b0 tx=0x7f86e001b3c0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 -- 172.21.15.66:0/3414298721 shutdown_connections 2024-03-30T00:41:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.66:6836/1694329026,v1:172.21.15.66:6837/1694329026] conn(0x55a23db2b380 0x55a23db4d780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3cb30 0x55a23db2aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23db2a5a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f86db400640 0 client.5673 ms_handle_reset on v2:172.21.15.66:6836/1694329026 2024-03-30T00:41:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 --2- 172.21.15.66:0/3414298721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23daf28d0 0x55a23db2a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 -- 172.21.15.66:0/3414298721 >> 172.21.15.66:0/3414298721 conn(0x55a23daf4830 msgr2=0x55a23daf3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 -- 172.21.15.66:0/3414298721 shutdown_connections 2024-03-30T00:41:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 1 -- 172.21.15.66:0/3414298721 wait complete. 2024-03-30T00:41:27.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.177+0000 7f8700278000 20 client.5673 trim_cache size 0 max 16384 2024-03-30T00:41:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f8700278000 1 Processor -- start 2024-03-30T00:41:27.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f8700278000 1 -- start start 2024-03-30T00:41:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f8700278000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23daf28d0 0x55a23daec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f8700278000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23daecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f8700278000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23db3cb30 0x55a23db4b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f8700278000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23d5f9940 con 0x55a23db3c210 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f8700278000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23d9409b0 con 0x55a23db3cb30 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f8700278000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a23d93df40 con 0x55a23daf28d0 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86ed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23daecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86ed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23daecc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57864/0 (socket says 172.21.15.66:57864) 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86ed400640 1 -- 172.21.15.66:0/3902439569 learned_addr learned my addr 172.21.15.66:0/3902439569 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86f5e00640 1 --2- 172.21.15.66:0/3902439569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23daf28d0 0x55a23daec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86f6800640 1 --2- 172.21.15.66:0/3902439569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23db3cb30 0x55a23db4b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86ed400640 1 -- 172.21.15.66:0/3902439569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23daf28d0 msgr2=0x55a23daec6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86ed400640 1 --2- 172.21.15.66:0/3902439569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23daf28d0 0x55a23daec6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86ed400640 1 -- 172.21.15.66:0/3902439569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23db3cb30 msgr2=0x55a23db4b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86ed400640 1 --2- 172.21.15.66:0/3902439569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23db3cb30 0x55a23db4b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86ed400640 1 -- 172.21.15.66:0/3902439569 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a23d8e7c00 con 0x55a23db3c210 2024-03-30T00:41:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86f6800640 1 --2- 172.21.15.66:0/3902439569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23db3cb30 0x55a23db4b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.193+0000 7f86ed400640 1 --2- 172.21.15.66:0/3902439569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23daecc20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f86e00018c0 tx=0x7f86e00a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbb528dff663dcc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86e7400640 1 -- 172.21.15.66:0/3902439569 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0005ea0 con 0x55a23db3c210 2024-03-30T00:41:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86e7400640 1 -- 172.21.15.66:0/3902439569 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e0001be0 con 0x55a23db3c210 2024-03-30T00:41:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86e7400640 1 -- 172.21.15.66:0/3902439569 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00a5a80 con 0x55a23db3c210 2024-03-30T00:41:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 -- 172.21.15.66:0/3902439569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 msgr2=0x55a23daecc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 --2- 172.21.15.66:0/3902439569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23daecc20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f86e00018c0 tx=0x7f86e00a7070 comp rx=0 tx=0).stop 2024-03-30T00:41:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 -- 172.21.15.66:0/3902439569 shutdown_connections 2024-03-30T00:41:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 --2- 172.21.15.66:0/3902439569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23db3cb30 0x55a23db4b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 --2- 172.21.15.66:0/3902439569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23db3c210 0x55a23daecc20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 --2- 172.21.15.66:0/3902439569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23daf28d0 0x55a23daec6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 -- 172.21.15.66:0/3902439569 >> 172.21.15.66:0/3902439569 conn(0x55a23daf4830 msgr2=0x55a23daebfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 -- 172.21.15.66:0/3902439569 shutdown_connections 2024-03-30T00:41:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 -- 172.21.15.66:0/3902439569 wait complete. 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 -- start start 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 Processor -- start 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23daf28d0 0x55a23db9ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3c210 0x55a23db9b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23db3cb30 0x55a23db4dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23db3bfb0 con 0x55a23daf28d0 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23dae5cd0 con 0x55a23db3cb30 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a23d8fac80 con 0x55a23db3c210 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 10 client.0 _ll_register_callbacks cb 0x55a23d901590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23daf28d0 0x55a23db9ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23daf28d0 0x55a23db9ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57874/0 (socket says 172.21.15.66:57874) 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86f5e00640 1 -- 172.21.15.66:0/2904595702 learned_addr learned my addr 172.21.15.66:0/2904595702 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86f6800640 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23db3cb30 0x55a23db4dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86ed400640 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3c210 0x55a23db9b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86f5e00640 1 -- 172.21.15.66:0/2904595702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3c210 msgr2=0x55a23db9b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86f5e00640 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3c210 0x55a23db9b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86f5e00640 1 -- 172.21.15.66:0/2904595702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23db3cb30 msgr2=0x55a23db4dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86f5e00640 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23db3cb30 0x55a23db4dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86f5e00640 1 -- 172.21.15.66:0/2904595702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a23d901ab0 con 0x55a23daf28d0 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86f5e00640 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23daf28d0 0x55a23db9ad60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f86e8027040 tx=0x7f86e8001660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b983c47d8da5d042 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86db400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86db400640 1 -- 172.21.15.66:0/2904595702 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8024b60 con 0x55a23daf28d0 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86db400640 1 -- 172.21.15.66:0/2904595702 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f86e8024cc0 con 0x55a23daf28d0 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 -- 172.21.15.66:0/2904595702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a23d90c460 con 0x55a23daf28d0 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f86db400640 1 -- 172.21.15.66:0/2904595702 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86e8024ea0 con 0x55a23daf28d0 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 10 client.5679 fetch_fsmap learned FSMap version 17 2024-03-30T00:41:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.197+0000 7f8700278000 1 -- 172.21.15.66:0/2904595702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a23d638cc0 con 0x55a23daf28d0 2024-03-30T00:41:27.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86db400640 1 -- 172.21.15.66:0/2904595702 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a23d90c460 con 0x55a23daf28d0 2024-03-30T00:41:27.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86db400640 1 -- 172.21.15.66:0/2904595702 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f86e8021040 con 0x55a23daf28d0 2024-03-30T00:41:27.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 10 client.5679 fetch_fsmap finished waiting for FSMap version 17 2024-03-30T00:41:27.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 10 client.5679 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:41:27.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 20 client.5679 populate_metadata read hostname 'smithi066' 2024-03-30T00:41:27.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.66:6836/1694329026,v1:172.21.15.66:6837/1694329026] conn(0x55a23db4e420 0x55a23db4e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 4 client.5679 mds_command: new command op to 5490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:41:27.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 -- 172.21.15.66:0/2904595702 --> [v2:172.21.15.66:6836/1694329026,v1:172.21.15.66:6837/1694329026] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a23d9059c0 con 0x55a23db4e420 2024-03-30T00:41:27.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86ed400640 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.66:6836/1694329026,v1:172.21.15.66:6837/1694329026] conn(0x55a23db4e420 0x55a23db4e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86ed400640 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.66:6836/1694329026,v1:172.21.15.66:6837/1694329026] conn(0x55a23db4e420 0x55a23db4e800 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692820dfee6dd4dc server_cookie=7467e4f3d032ca2e in_seq=0 out_seq=0 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86db400640 10 client.5679 ms_handle_connect on v2:172.21.15.66:6836/1694329026 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86db400640 1 -- 172.21.15.66:0/2904595702 <== mds.0 v2:172.21.15.66:6836/1694329026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a23d9059c0 con 0x55a23db4e420 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86db400640 10 client.5679 handle_command_reply: tid=0 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 client.5679 shutdown 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 -- 172.21.15.66:0/2904595702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23daf28d0 msgr2=0x55a23db9ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23daf28d0 0x55a23db9ad60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f86e8027040 tx=0x7f86e8001660 comp rx=0 tx=0).stop 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 -- 172.21.15.66:0/2904595702 shutdown_connections 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a23db3cb30 0x55a23db4dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.66:6836/1694329026,v1:172.21.15.66:6837/1694329026] conn(0x55a23db4e420 0x55a23db4e800 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55a23db3c210 0x55a23db9b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 --2- 172.21.15.66:0/2904595702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a23daf28d0 0x55a23db9ad60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86db400640 0 client.5679 ms_handle_reset on v2:172.21.15.66:6836/1694329026 2024-03-30T00:41:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 -- 172.21.15.66:0/2904595702 >> 172.21.15.66:0/2904595702 conn(0x55a23daf4830 msgr2=0x55a23daec860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 -- 172.21.15.66:0/2904595702 shutdown_connections 2024-03-30T00:41:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 1 -- 172.21.15.66:0/2904595702 wait complete. 2024-03-30T00:41:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f8700278000 20 client.5679 trim_cache size 0 max 16384 2024-03-30T00:41:27.215 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:41:27.215 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:41:27.215 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:41:27.215 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5490, 2024-03-30T00:41:27.215 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:41:27.215 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:41:27.215 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:41:27.216 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 5.9326510380000004, 2024-03-30T00:41:27.216 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 17, 2024-03-30T00:41:27.216 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 40, 2024-03-30T00:41:27.216 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 40, 2024-03-30T00:41:27.216 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 13.179401459999999 2024-03-30T00:41:27.216 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 -- 172.21.15.66:0/2427910466 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86d4066690 msgr2=0x7f86d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86d4066690 0x7f86d4068b50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86f80b2830 tx=0x7f86e8009040 comp rx=0 tx=0).stop 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 -- 172.21.15.66:0/2427910466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f8093310 msgr2=0x7f86f81f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f8093310 0x7f86f81f4540 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f86f80cef00 tx=0x7f86f000e7a0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 -- 172.21.15.66:0/2427910466 shutdown_connections 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86d4066690 0x7f86d4068b50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86f8093310 0x7f86f81f4540 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f80b0f50 0x7f86f8091900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 --2- 172.21.15.66:0/2427910466 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86dc008d60 0x7f86f80c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 -- 172.21.15.66:0/2427910466 >> 172.21.15.66:0/2427910466 conn(0x7f86f80ce2f0 msgr2=0x7f86f80b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 -- 172.21.15.66:0/2427910466 shutdown_connections 2024-03-30T00:41:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.201+0000 7f86fce00640 1 -- 172.21.15.66:0/2427910466 wait complete. 2024-03-30T00:41:27.230 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:41:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.361+0000 7f512b400640 1 Processor -- start 2024-03-30T00:41:27.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.361+0000 7f512b400640 1 -- start start 2024-03-30T00:41:27.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.361+0000 7f512b400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f51240ce520 0x7f51240ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.361+0000 7f512b400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512406d070 con 0x7f512415a640 2024-03-30T00:41:27.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.361+0000 7f512b400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51240cbba0 con 0x7f51240cabb0 2024-03-30T00:41:27.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.361+0000 7f512b400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51240cee70 con 0x7f51240ce520 2024-03-30T00:41:27.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.361+0000 7f5128a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f512415a640 0x7f512406dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39448/0 (socket says 172.21.15.66:39448) 2024-03-30T00:41:27.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.361+0000 7f5128a00640 1 -- 172.21.15.66:0/1883739473 learned_addr learned my addr 172.21.15.66:0/1883739473 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.361+0000 7f511be00640 1 --2- 172.21.15.66:0/1883739473 >> v2:172.21.15.156:3300/0 conn(0x7f51240ce520 0x7f51240ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.361+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791537079 0 0) 0x7f512406d070 con 0x7f512415a640 2024-03-30T00:41:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.361+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5108003530 con 0x7f512415a640 2024-03-30T00:41:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2938344600 0 0) 0x7f5108003530 con 0x7f512415a640 2024-03-30T00:41:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 >> v2:172.21.15.156:3300/0 conn(0x7f51240ce520 msgr2=0x7f51240ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 --2- 172.21.15.66:0/1883739473 >> v2:172.21.15.156:3300/0 conn(0x7f51240ce520 0x7f51240ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 >> v1:172.21.15.156:6789/0 conn(0x7f51240cabb0 legacy=0x7f51240cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240cb8c0 con 0x7f512415a640 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f5114001680 con 0x7f512415a640 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 >> v1:172.21.15.66:6789/0 conn(0x7f512415a640 legacy=0x7f512406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 --2- 172.21.15.66:0/1883739473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5108003980 0x7f5108003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 --2- 172.21.15.66:0/1883739473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5108004380 0x7f5108004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 --2- 172.21.15.66:0/1883739473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5108005050 0x7f510800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51240cee70 con 0x7f5108005050 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510800bfb0 con 0x7f5108004380 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510800c170 con 0x7f5108003980 2024-03-30T00:41:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5129400640 1 --2- 172.21.15.66:0/1883739473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5108005050 0x7f510800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f511be00640 1 --2- 172.21.15.66:0/1883739473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5108004380 0x7f5108004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5128a00640 1 --2- 172.21.15.66:0/1883739473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5108003980 0x7f5108003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5129400640 1 -- 172.21.15.66:0/1883739473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5108003980 msgr2=0x7f5108003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5129400640 1 --2- 172.21.15.66:0/1883739473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5108003980 0x7f5108003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5129400640 1 -- 172.21.15.66:0/1883739473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5108004380 msgr2=0x7f5108004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5129400640 1 --2- 172.21.15.66:0/1883739473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5108004380 0x7f5108004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5129400640 1 -- 172.21.15.66:0/1883739473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5108003530 con 0x7f5108005050 2024-03-30T00:41:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5129400640 1 -- 172.21.15.66:0/1883739473 reap_dead start 2024-03-30T00:41:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5129400640 1 --2- 172.21.15.66:0/1883739473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5108005050 0x7f510800b990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f511c002010 tx=0x7f511c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7b64ae5431bd0b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c003b80 con 0x7f5108005050 2024-03-30T00:41:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c003d00 con 0x7f5108005050 2024-03-30T00:41:27.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.365+0000 7f5123e00640 1 -- 172.21.15.66:0/1883739473 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c005310 con 0x7f5108005050 2024-03-30T00:41:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 -- 172.21.15.66:0/1883739473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5108005050 msgr2=0x7f510800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 --2- 172.21.15.66:0/1883739473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5108005050 0x7f510800b990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f511c002010 tx=0x7f511c01c8e0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 -- 172.21.15.66:0/1883739473 shutdown_connections 2024-03-30T00:41:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 --2- 172.21.15.66:0/1883739473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5108005050 0x7f510800b990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 -- 172.21.15.66:0/1883739473 >> 172.21.15.66:0/1883739473 conn(0x7f5124069f60 msgr2=0x7f512406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 -- 172.21.15.66:0/1883739473 shutdown_connections 2024-03-30T00:41:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 -- 172.21.15.66:0/1883739473 wait complete. 2024-03-30T00:41:27.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 Processor -- start 2024-03-30T00:41:27.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 -- start start 2024-03-30T00:41:27.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5108005050 0x7f512415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5124149f80 0x7f51241579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5124157ef0 0x7f512415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5124158350 con 0x7f5124149f80 2024-03-30T00:41:27.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512415d960 con 0x7f5108005050 2024-03-30T00:41:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f512b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512415c380 con 0x7f5124157ef0 2024-03-30T00:41:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f511be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5124149f80 0x7f51241579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f511be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5124149f80 0x7f51241579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57888/0 (socket says 172.21.15.66:57888) 2024-03-30T00:41:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f511be00640 1 -- 172.21.15.66:0/655186245 learned_addr learned my addr 172.21.15.66:0/655186245 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f5129400640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5124157ef0 0x7f512415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f511be00640 1 -- 172.21.15.66:0/655186245 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5124157ef0 msgr2=0x7f512415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f511be00640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5124157ef0 0x7f512415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f511be00640 1 -- 172.21.15.66:0/655186245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5108005050 msgr2=0x7f512415d420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T00:41:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f511be00640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5108005050 0x7f512415d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f511be00640 1 -- 172.21.15.66:0/655186245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512415c660 con 0x7f5124149f80 2024-03-30T00:41:27.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.369+0000 7f511be00640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5124149f80 0x7f51241579b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f510c011960 tx=0x7f510c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.373+0000 7f5121600640 1 -- 172.21.15.66:0/655186245 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c00faf0 con 0x7f5124149f80 2024-03-30T00:41:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.373+0000 7f5121600640 1 -- 172.21.15.66:0/655186245 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c00fc50 con 0x7f5124149f80 2024-03-30T00:41:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.373+0000 7f512b400640 1 -- 172.21.15.66:0/655186245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512415c940 con 0x7f5124149f80 2024-03-30T00:41:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.373+0000 7f5121600640 1 -- 172.21.15.66:0/655186245 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c014440 con 0x7f5124149f80 2024-03-30T00:41:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.373+0000 7f512b400640 1 -- 172.21.15.66:0/655186245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512415cf00 con 0x7f5124149f80 2024-03-30T00:41:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.373+0000 7f512b400640 1 -- 172.21.15.66:0/655186245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5124066c30 con 0x7f5124149f80 2024-03-30T00:41:27.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.373+0000 7f5121600640 1 -- 172.21.15.66:0/655186245 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f510c01d400 con 0x7f5124149f80 2024-03-30T00:41:27.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.373+0000 7f5121600640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f51000666c0 0x7f5100068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:27.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.373+0000 7f5121600640 1 -- 172.21.15.66:0/655186245 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f510c084a70 con 0x7f5124149f80 2024-03-30T00:41:27.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.373+0000 7f5128a00640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f51000666c0 0x7f5100068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:27.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.373+0000 7f5128a00640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f51000666c0 0x7f5100068b80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5114001ec0 tx=0x7f5114001160 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:27.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.377+0000 7f5121600640 1 -- 172.21.15.66:0/655186245 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f510c051c90 con 0x7f5124149f80 2024-03-30T00:41:27.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 -- 172.21.15.66:0/655186245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f51240ac460 con 0x7f5124149f80 2024-03-30T00:41:27.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f5121600640 1 -- 172.21.15.66:0/655186245 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f510c0552b0 con 0x7f5124149f80 2024-03-30T00:41:27.587 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:41:27.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 -- 172.21.15.66:0/655186245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f51000666c0 msgr2=0x7f5100068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f51000666c0 0x7f5100068b80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5114001ec0 tx=0x7f5114001160 comp rx=0 tx=0).stop 2024-03-30T00:41:27.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 -- 172.21.15.66:0/655186245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5124149f80 msgr2=0x7f51241579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:27.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5124149f80 0x7f51241579b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f510c011960 tx=0x7f510c011e20 comp rx=0 tx=0).stop 2024-03-30T00:41:27.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 -- 172.21.15.66:0/655186245 shutdown_connections 2024-03-30T00:41:27.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5124157ef0 0x7f512415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f51000666c0 0x7f5100068b80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5124149f80 0x7f51241579b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 --2- 172.21.15.66:0/655186245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5108005050 0x7f512415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:27.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 -- 172.21.15.66:0/655186245 >> 172.21.15.66:0/655186245 conn(0x7f5124069f60 msgr2=0x7f51240c4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:27.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 -- 172.21.15.66:0/655186245 shutdown_connections 2024-03-30T00:41:27.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:27.577+0000 7f512b400640 1 -- 172.21.15.66:0/655186245 wait complete. 2024-03-30T00:41:27.608 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:41:27.608 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:41:27.608 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:41:27.608 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:41:27.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:41:27.608 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:41:27.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:41:27.608 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:41:27.608 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:41:27.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:27.609 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:41:27.618 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:41:27.618 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:41:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42391sec preferred_lft 42391sec 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:41:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:41:27.621 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:41:27.621 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:41:27.621 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:41:27.621 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:41:27.621 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:41:27.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:27.621 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:41:27.678 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:41:27.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:27.680 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:41:27.738 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:41:27.738 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:41:27.738 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:41:27.738 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:41:27.738 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:41:27.738 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:41:27.738 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:41:27.738 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:41:27.738 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:41:27.738 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:41:27.739 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:41:27.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:27.739 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:41:27.739 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:41:27.739 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:41:27.739 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:41:27.739 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:41:27.739 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:41:27.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:27.830 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:41:27.840 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:41:27.840 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:41:27.840 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:41:27.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:27.841 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:41:27.905 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:41:27.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:41:27.906 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:41:27.956 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:41:27.957 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:41:27.957 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:41:28.008 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:41:28.072 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:41:28.072 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:41:28.134 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:41:28.136 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:41:28.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:28.136 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:41:28.188 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:41:28.188 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:41:28.235 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:41:28.257 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:41:28.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:41:28.318 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:41:28.320 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:41:28.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:28.321 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:41:28.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:41:28.321+0000 7fcbeb56e580 -1 init, newargv = 0x5649f6c758f0 newargc=15 2024-03-30T00:41:28.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[39082]: starting ceph client 2024-03-30T00:41:28.338 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[39082]: starting fuse 2024-03-30T00:41:28.339 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:41:28.339 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:41:28.339 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:41:28.339 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:41:28.339 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:41:28.339 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:41:28.340 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:41:28.341 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:41:28.341 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:41:28.341 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:41:28.341 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:41:28.341 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:41:28.341 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:41:28.341 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:41:28.465 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.39082.asok 2024-03-30T00:41:28.649 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39082.asok 2024-03-30T00:41:28.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:28.650 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39082.asok status 2024-03-30T00:41:28.824 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:41:28.824 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:41:28.824 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "39082", 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5691, 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:41:28.825 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 5691 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2860126256 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:41:28.826 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:41:28.827 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2860126256 2024-03-30T00:41:28.827 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:41:28.827 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.5691 192.168.0.1:0/2860126256", 2024-03-30T00:41:28.827 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/2860126256", 2024-03-30T00:41:28.827 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:41:28.827 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 17, 2024-03-30T00:41:28.827 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 40, 2024-03-30T00:41:28.827 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:41:28.827 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:41:28.827 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:41:28.827 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:41:28.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:28.839 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:41:28.839 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:41:28.848 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:41:28.849 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:41:28.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:28.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:41:28.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:28.917 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-03-30T00:41:28.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:28.980 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-03-30T00:41:29.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:29.041 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-03-30T00:41:29.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:29.101 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-03-30T00:41:29.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:29.159 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-03-30T00:41:29.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:29.212 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-30T00:41:29.273 INFO:teuthology.orchestra.run.smithi066.stdout:./ 2024-03-30T00:41:29.273 INFO:teuthology.orchestra.run.smithi066.stdout:./rfile 2024-03-30T00:41:29.273 INFO:teuthology.orchestra.run.smithi066.stdout:./parentdir 2024-03-30T00:41:29.273 INFO:teuthology.orchestra.run.smithi066.stdout:./parentdir/pfile 2024-03-30T00:41:29.273 INFO:teuthology.orchestra.run.smithi066.stdout:./parentdir/childdir 2024-03-30T00:41:29.273 INFO:teuthology.orchestra.run.smithi066.stdout:./parentdir/childdir/cfile 2024-03-30T00:41:29.275 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:41:29.275 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:41:29.275 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:41:29.275 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:41:29.275 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-30T00:41:29.275 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:41:29.385 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:41:29.575 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:41:29.575 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:41:29.575 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:41:29.575 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:41:29.575 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-03-30T00:41:29.575 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:41:29.681 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627778 2024-03-30T00:41:29.870 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:41:29.871 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:41:29.871 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:41:29.871 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:41:29.871 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-03-30T00:41:29.871 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:41:29.977 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627776 2024-03-30T00:41:30.166 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:41:30.166 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:41:30.166 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:41:30.166 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:41:30.166 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-03-30T00:41:30.166 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:41:30.277 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628279 2024-03-30T00:41:30.467 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:41:30.467 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:41:30.467 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:41:30.467 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:41:30.467 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-03-30T00:41:30.467 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:41:30.573 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627777 2024-03-30T00:41:30.763 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:41:30.763 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:41:30.763 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:41:30.763 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:41:30.763 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-03-30T00:41:30.763 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:41:30.869 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628280 2024-03-30T00:41:31.059 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:41:31.059 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:41:31.109 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:31.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.281+0000 7f3693600640 1 Processor -- start 2024-03-30T00:41:31.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.281+0000 7f3693600640 1 -- start start 2024-03-30T00:41:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f3693600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c06e800 0x7f368c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f3693600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f368c06f8e0 0x7f368c0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f3693600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c06b0b0 con 0x7f368c0cc560 2024-03-30T00:41:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f3693600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c160640 con 0x7f368c06f8e0 2024-03-30T00:41:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f3693600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c06b390 con 0x7f368c06e800 2024-03-30T00:41:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f3690c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f368c0cc560 0x7f368c06e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55296/0 (socket says 172.21.15.66:55296) 2024-03-30T00:41:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f3690c00640 1 -- 172.21.15.66:0/1293986933 learned_addr learned my addr 172.21.15.66:0/1293986933 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:31.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368be00640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c06e800 0x7f368c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f3691600640 1 --2- 172.21.15.66:0/1293986933 >> v2:172.21.15.156:3300/0 conn(0x7f368c06f8e0 0x7f368c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839661016 0 0) 0x7f368c06b0b0 con 0x7f368c0cc560 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3674003530 con 0x7f368c0cc560 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3620259780 0 0) 0x7f3674003530 con 0x7f368c0cc560 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c06e800 msgr2=0x7f368c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c06e800 0x7f368c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 >> v2:172.21.15.156:3300/0 conn(0x7f368c06f8e0 msgr2=0x7f368c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 --2- 172.21.15.66:0/1293986933 >> v2:172.21.15.156:3300/0 conn(0x7f368c06f8e0 0x7f368c0ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c06b670 con 0x7f368c0cc560 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f3680001680 con 0x7f368c0cc560 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 >> v1:172.21.15.66:6789/0 conn(0x7f368c0cc560 legacy=0x7f368c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3674003980 0x7f3674003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 0x7f3674004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c06e800 0x7f3674005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c160640 con 0x7f36740047d0 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c06b390 con 0x7f3674003980 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367400b990 con 0x7f368c06e800 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368be00640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 0x7f3674004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f3690c00640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3674003980 0x7f3674003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f3691600640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c06e800 0x7f3674005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368be00640 1 -- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c06e800 msgr2=0x7f3674005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368be00640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c06e800 0x7f3674005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368be00640 1 -- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3674003980 msgr2=0x7f3674003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368be00640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3674003980 0x7f3674003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f368be00640 1 -- 172.21.15.66:0/1293986933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3674003530 con 0x7f36740047d0 2024-03-30T00:41:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.285+0000 7f3691600640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c06e800 0x7f3674005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:31.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f368be00640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 0x7f3674004c50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3678009ab0 tx=0x7f367801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35953193557f8966 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36780a5560 con 0x7f36740047d0 2024-03-30T00:41:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36780a56c0 con 0x7f36740047d0 2024-03-30T00:41:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f368b400640 1 -- 172.21.15.66:0/1293986933 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36780a5a30 con 0x7f36740047d0 2024-03-30T00:41:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f3693600640 1 -- 172.21.15.66:0/1293986933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 msgr2=0x7f3674004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f3693600640 1 --2- 172.21.15.66:0/1293986933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 0x7f3674004c50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3678009ab0 tx=0x7f367801ca50 comp rx=0 tx=0).stop 2024-03-30T00:41:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f3691600640 1 -- 172.21.15.66:0/1293986933 reap_dead start 2024-03-30T00:41:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f3693600640 1 -- 172.21.15.66:0/1293986933 shutdown_connections 2024-03-30T00:41:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f3693600640 1 -- 172.21.15.66:0/1293986933 >> 172.21.15.66:0/1293986933 conn(0x7f368c0d0540 msgr2=0x7f368c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f3693600640 1 -- 172.21.15.66:0/1293986933 shutdown_connections 2024-03-30T00:41:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f3693600640 1 -- 172.21.15.66:0/1293986933 wait complete. 2024-03-30T00:41:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f3693600640 1 Processor -- start 2024-03-30T00:41:31.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.289+0000 7f3693600640 1 -- start start 2024-03-30T00:41:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3693600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 0x7f368c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3693600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c0c3fa0 0x7f368c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3693600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368c0ca240 0x7f368c0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3693600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1f35e0 con 0x7f36740047d0 2024-03-30T00:41:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3693600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1f38c0 con 0x7f368c0ca240 2024-03-30T00:41:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3693600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1f3ba0 con 0x7f368c0c3fa0 2024-03-30T00:41:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3690c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 0x7f368c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3690c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 0x7f368c155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44044/0 (socket says 172.21.15.66:44044) 2024-03-30T00:41:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3690c00640 1 -- 172.21.15.66:0/1294414433 learned_addr learned my addr 172.21.15.66:0/1294414433 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3691600640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368c0ca240 0x7f368c0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f368be00640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c0c3fa0 0x7f368c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3690c00640 1 -- 172.21.15.66:0/1294414433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c0c3fa0 msgr2=0x7f368c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3690c00640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c0c3fa0 0x7f368c0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3690c00640 1 -- 172.21.15.66:0/1294414433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368c0ca240 msgr2=0x7f368c0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3690c00640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368c0ca240 0x7f368c0ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3690c00640 1 -- 172.21.15.66:0/1294414433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c1f3e80 con 0x7f36740047d0 2024-03-30T00:41:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3691600640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368c0ca240 0x7f368c0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f368be00640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c0c3fa0 0x7f368c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3690c00640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 0x7f368c155d20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3680012310 tx=0x7f36800127d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:31.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3688c00640 1 -- 172.21.15.66:0/1294414433 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3680014750 con 0x7f36740047d0 2024-03-30T00:41:31.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3688c00640 1 -- 172.21.15.66:0/1294414433 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36800148b0 con 0x7f36740047d0 2024-03-30T00:41:31.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3688c00640 1 -- 172.21.15.66:0/1294414433 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3680014c60 con 0x7f36740047d0 2024-03-30T00:41:31.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3693600640 1 -- 172.21.15.66:0/1294414433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c1f4160 con 0x7f36740047d0 2024-03-30T00:41:31.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3693600640 1 -- 172.21.15.66:0/1294414433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c1f3350 con 0x7f36740047d0 2024-03-30T00:41:31.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3693600640 1 -- 172.21.15.66:0/1294414433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c066c30 con 0x7f36740047d0 2024-03-30T00:41:31.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.293+0000 7f3688c00640 1 -- 172.21.15.66:0/1294414433 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f36800153e0 con 0x7f36740047d0 2024-03-30T00:41:31.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.297+0000 7f3688c00640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f366c0666c0 0x7f366c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.297+0000 7f3688c00640 1 -- 172.21.15.66:0/1294414433 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3680086be0 con 0x7f36740047d0 2024-03-30T00:41:31.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.297+0000 7f368be00640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f366c0666c0 0x7f366c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.297+0000 7f368be00640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f366c0666c0 0x7f366c068b80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f36780af000 tx=0x7f36780046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:31.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.301+0000 7f3688c00640 1 -- 172.21.15.66:0/1294414433 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3680053d70 con 0x7f36740047d0 2024-03-30T00:41:31.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.461+0000 7f3693600640 1 -- 172.21.15.66:0/1294414433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f368c1600d0 con 0x7f36740047d0 2024-03-30T00:41:31.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.461+0000 7f3688c00640 1 -- 172.21.15.66:0/1294414433 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f368001c020 con 0x7f36740047d0 2024-03-30T00:41:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:31.472 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":40,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:41:22.272744+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:41:19.292693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T00:41:20.267522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:31.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 -- 172.21.15.66:0/1294414433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f366c0666c0 msgr2=0x7f366c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f366c0666c0 0x7f366c068b80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f36780af000 tx=0x7f36780046d0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 -- 172.21.15.66:0/1294414433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 msgr2=0x7f368c155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 0x7f368c155d20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3680012310 tx=0x7f36800127d0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 -- 172.21.15.66:0/1294414433 shutdown_connections 2024-03-30T00:41:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368c0ca240 0x7f368c0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368c0c3fa0 0x7f368c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f366c0666c0 0x7f366c068b80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 --2- 172.21.15.66:0/1294414433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36740047d0 0x7f368c155d20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 -- 172.21.15.66:0/1294414433 >> 172.21.15.66:0/1294414433 conn(0x7f368c0d0540 msgr2=0x7f368c158bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 -- 172.21.15.66:0/1294414433 shutdown_connections 2024-03-30T00:41:31.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.465+0000 7f3693600640 1 -- 172.21.15.66:0/1294414433 wait complete. 2024-03-30T00:41:31.494 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:41:19.292693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:41:20.267522+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:41:31.495 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:41:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.629+0000 7f2668800640 1 Processor -- start 2024-03-30T00:41:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.633+0000 7f2668800640 1 -- start start 2024-03-30T00:41:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.633+0000 7f2668800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2660160bc0 0x7f2660167840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.633+0000 7f2668800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266006da20 con 0x7f26601610c0 2024-03-30T00:41:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.633+0000 7f2668800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266006dc60 con 0x7f2660157060 2024-03-30T00:41:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.633+0000 7f2668800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2660157440 con 0x7f2660160bc0 2024-03-30T00:41:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.633+0000 7f2665400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f26601610c0 0x7f26601604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55304/0 (socket says 172.21.15.66:55304) 2024-03-30T00:41:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.633+0000 7f2666800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2660160bc0 0x7f2660167840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.633+0000 7f2665400640 1 -- 172.21.15.66:0/4115319828 learned_addr learned my addr 172.21.15.66:0/4115319828 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.633+0000 7f2666800640 1 -- 172.21.15.66:0/4115319828 >> v1:172.21.15.156:6789/0 conn(0x7f2660157060 legacy=0x7f266006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.633+0000 7f2666800640 1 -- 172.21.15.66:0/4115319828 >> v1:172.21.15.66:6789/0 conn(0x7f26601610c0 legacy=0x7f26601604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.633+0000 7f2666800640 1 -- 172.21.15.66:0/4115319828 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266006df40 con 0x7f2660160bc0 2024-03-30T00:41:31.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2666800640 1 --2- 172.21.15.66:0/4115319828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2660160bc0 0x7f2660167840 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f265c009d90 tx=0x7f265c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7304bd7eae0fca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:31.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2664a00640 1 -- 172.21.15.66:0/4115319828 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c0a5800 con 0x7f2660160bc0 2024-03-30T00:41:31.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2664a00640 1 -- 172.21.15.66:0/4115319828 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c0a5960 con 0x7f2660160bc0 2024-03-30T00:41:31.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2664a00640 1 -- 172.21.15.66:0/4115319828 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c0a5cf0 con 0x7f2660160bc0 2024-03-30T00:41:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2668800640 1 -- 172.21.15.66:0/4115319828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2660160bc0 msgr2=0x7f2660167840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2668800640 1 --2- 172.21.15.66:0/4115319828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2660160bc0 0x7f2660167840 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f265c009d90 tx=0x7f265c01c850 comp rx=0 tx=0).stop 2024-03-30T00:41:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2668800640 1 -- 172.21.15.66:0/4115319828 shutdown_connections 2024-03-30T00:41:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2668800640 1 --2- 172.21.15.66:0/4115319828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2660160bc0 0x7f2660167840 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2668800640 1 -- 172.21.15.66:0/4115319828 >> 172.21.15.66:0/4115319828 conn(0x7f2660071750 msgr2=0x7f2660069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2668800640 1 -- 172.21.15.66:0/4115319828 shutdown_connections 2024-03-30T00:41:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2668800640 1 -- 172.21.15.66:0/4115319828 wait complete. 2024-03-30T00:41:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2668800640 1 Processor -- start 2024-03-30T00:41:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.637+0000 7f2668800640 1 -- start start 2024-03-30T00:41:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2668800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2660157060 0x7f266014d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2668800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2660160bc0 0x7f2660152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2668800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26601610c0 0x7f266014a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2668800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266014dc50 con 0x7f26601610c0 2024-03-30T00:41:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2668800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2660153470 con 0x7f2660157060 2024-03-30T00:41:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2668800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26601515d0 con 0x7f2660160bc0 2024-03-30T00:41:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2666800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26601610c0 0x7f266014a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2666800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26601610c0 0x7f266014a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44070/0 (socket says 172.21.15.66:44070) 2024-03-30T00:41:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2666800640 1 -- 172.21.15.66:0/606259270 learned_addr learned my addr 172.21.15.66:0/606259270 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:31.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2665e00640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2660157060 0x7f266014d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2665400640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2660160bc0 0x7f2660152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2666800640 1 -- 172.21.15.66:0/606259270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2660160bc0 msgr2=0x7f2660152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2666800640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2660160bc0 0x7f2660152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2666800640 1 -- 172.21.15.66:0/606259270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2660157060 msgr2=0x7f266014d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2666800640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2660157060 0x7f266014d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2666800640 1 -- 172.21.15.66:0/606259270 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c009a70 con 0x7f26601610c0 2024-03-30T00:41:31.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2665e00640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2660157060 0x7f266014d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:31.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2665400640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2660160bc0 0x7f2660152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:31.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2666800640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26601610c0 0x7f266014a740 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f265c0b0040 tx=0x7f265c0195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:31.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f265a000640 1 -- 172.21.15.66:0/606259270 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c0b9860 con 0x7f26601610c0 2024-03-30T00:41:31.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f265a000640 1 -- 172.21.15.66:0/606259270 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c0b99c0 con 0x7f26601610c0 2024-03-30T00:41:31.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f265a000640 1 -- 172.21.15.66:0/606259270 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c0b9d70 con 0x7f26601610c0 2024-03-30T00:41:31.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2668800640 1 -- 172.21.15.66:0/606259270 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26601536c0 con 0x7f26601610c0 2024-03-30T00:41:31.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2668800640 1 -- 172.21.15.66:0/606259270 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2660151b00 con 0x7f26601610c0 2024-03-30T00:41:31.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f2668800640 1 -- 172.21.15.66:0/606259270 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2660066c30 con 0x7f26601610c0 2024-03-30T00:41:31.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.641+0000 7f265a000640 1 -- 172.21.15.66:0/606259270 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f265c0ad440 con 0x7f26601610c0 2024-03-30T00:41:31.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.645+0000 7f265a000640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2644066690 0x7f2644068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.645+0000 7f265a000640 1 -- 172.21.15.66:0/606259270 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f265c11f7e0 con 0x7f26601610c0 2024-03-30T00:41:31.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.645+0000 7f2665e00640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2644066690 0x7f2644068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.645+0000 7f2665e00640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2644066690 0x7f2644068b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f266006aec0 tx=0x7f2654009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:31.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.649+0000 7f265a000640 1 -- 172.21.15.66:0/606259270 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f265c0eca30 con 0x7f26601610c0 2024-03-30T00:41:31.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.809+0000 7f2668800640 1 -- 172.21.15.66:0/606259270 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2660071f30 con 0x7f26601610c0 2024-03-30T00:41:31.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.809+0000 7f265a000640 1 -- 172.21.15.66:0/606259270 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f265c0f0050 con 0x7f26601610c0 2024-03-30T00:41:31.820 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:41:31.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 -- 172.21.15.66:0/606259270 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2644066690 msgr2=0x7f2644068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2644066690 0x7f2644068b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f266006aec0 tx=0x7f2654009040 comp rx=0 tx=0).stop 2024-03-30T00:41:31.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 -- 172.21.15.66:0/606259270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26601610c0 msgr2=0x7f266014a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26601610c0 0x7f266014a740 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f265c0b0040 tx=0x7f265c0195f0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 -- 172.21.15.66:0/606259270 shutdown_connections 2024-03-30T00:41:31.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2644066690 0x7f2644068b50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26601610c0 0x7f266014a740 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2660160bc0 0x7f2660152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 --2- 172.21.15.66:0/606259270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2660157060 0x7f266014d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 -- 172.21.15.66:0/606259270 >> 172.21.15.66:0/606259270 conn(0x7f2660071750 msgr2=0x7f266015cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:31.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 -- 172.21.15.66:0/606259270 shutdown_connections 2024-03-30T00:41:31.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.813+0000 7f2668800640 1 -- 172.21.15.66:0/606259270 wait complete. 2024-03-30T00:41:31.841 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:41:31.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.973+0000 7fc232200640 1 Processor -- start 2024-03-30T00:41:31.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.973+0000 7fc232200640 1 -- start start 2024-03-30T00:41:31.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc232200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc22c0cec30 0x7fc22c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc232200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22c0cde70 0x7fc22c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc232200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c15a6e0 con 0x7fc22c0cda90 2024-03-30T00:41:31.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc232200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22c0b1b80 con 0x7fc22c0cec30 2024-03-30T00:41:31.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc232200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0cd3e0 con 0x7fc22c0cde70 2024-03-30T00:41:31.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22b400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc22c0cda90 0x7fc22c0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55314/0 (socket says 172.21.15.66:55314) 2024-03-30T00:41:31.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22b400640 1 -- 172.21.15.66:0/2802541077 learned_addr learned my addr 172.21.15.66:0/2802541077 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:31.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc223e00640 1 --2- 172.21.15.66:0/2802541077 >> v2:172.21.15.156:3300/0 conn(0x7fc22c0cec30 0x7fc22c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526079357 0 0) 0x7fc22c15a6e0 con 0x7fc22c0cda90 2024-03-30T00:41:31.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22be00640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22c0cde70 0x7fc22c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc210003530 con 0x7fc22c0cda90 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1546768663 0 0) 0x7fc210003530 con 0x7fc22c0cda90 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22c0cde70 msgr2=0x7fc22c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22c0cde70 0x7fc22c15a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 >> v2:172.21.15.156:3300/0 conn(0x7fc22c0cec30 msgr2=0x7fc22c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 --2- 172.21.15.66:0/2802541077 >> v2:172.21.15.156:3300/0 conn(0x7fc22c0cec30 0x7fc22c159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c15a990 con 0x7fc22c0cda90 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fc21c001680 con 0x7fc22c0cda90 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 >> v1:172.21.15.66:6789/0 conn(0x7fc22c0cda90 legacy=0x7fc22c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc210003980 0x7fc210003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2100047d0 0x7fc210004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22c0cde70 0x7fc210005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0b1b80 con 0x7fc2100047d0 2024-03-30T00:41:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0cd3e0 con 0x7fc210003980 2024-03-30T00:41:31.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21000b990 con 0x7fc22c0cde70 2024-03-30T00:41:31.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc223e00640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2100047d0 0x7fc210004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22be00640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22c0cde70 0x7fc210005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc22b400640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc210003980 0x7fc210003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc223e00640 1 -- 172.21.15.66:0/2802541077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22c0cde70 msgr2=0x7fc210005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:31.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc223e00640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22c0cde70 0x7fc210005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc223e00640 1 -- 172.21.15.66:0/2802541077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc210003980 msgr2=0x7fc210003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:31.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc223e00640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc210003980 0x7fc210003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.977+0000 7fc223e00640 1 -- 172.21.15.66:0/2802541077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc210003530 con 0x7fc2100047d0 2024-03-30T00:41:31.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.981+0000 7fc223e00640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2100047d0 0x7fc210004c50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc214009ae0 tx=0x7fc21401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3e64f175b437ca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.981+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140a6480 con 0x7fc2100047d0 2024-03-30T00:41:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.981+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2140a65e0 con 0x7fc2100047d0 2024-03-30T00:41:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.981+0000 7fc22aa00640 1 -- 172.21.15.66:0/2802541077 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140a6950 con 0x7fc2100047d0 2024-03-30T00:41:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.981+0000 7fc232200640 1 -- 172.21.15.66:0/2802541077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2100047d0 msgr2=0x7fc210004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.981+0000 7fc232200640 1 --2- 172.21.15.66:0/2802541077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2100047d0 0x7fc210004c50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc214009ae0 tx=0x7fc21401ca70 comp rx=0 tx=0).stop 2024-03-30T00:41:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.981+0000 7fc22be00640 1 -- 172.21.15.66:0/2802541077 reap_dead start 2024-03-30T00:41:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.981+0000 7fc232200640 1 -- 172.21.15.66:0/2802541077 shutdown_connections 2024-03-30T00:41:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.981+0000 7fc232200640 1 -- 172.21.15.66:0/2802541077 >> 172.21.15.66:0/2802541077 conn(0x7fc22c069f60 msgr2=0x7fc22c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:31.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.981+0000 7fc232200640 1 -- 172.21.15.66:0/2802541077 shutdown_connections 2024-03-30T00:41:31.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.981+0000 7fc232200640 1 -- 172.21.15.66:0/2802541077 wait complete. 2024-03-30T00:41:31.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc232200640 1 Processor -- start 2024-03-30T00:41:31.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc232200640 1 -- start start 2024-03-30T00:41:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc232200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2100047d0 0x7fc22c0ab800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc232200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc22c06cab0 0x7fc22c0aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc232200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc22c1557e0 0x7fc22c155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc232200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c15c4e0 con 0x7fc22c06cab0 2024-03-30T00:41:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc232200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c15be60 con 0x7fc22c1557e0 2024-03-30T00:41:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc232200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0ac030 con 0x7fc2100047d0 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc22c06cab0 0x7fc22c0aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc22c06cab0 0x7fc22c0aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44090/0 (socket says 172.21.15.66:44090) 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223e00640 1 -- 172.21.15.66:0/2543875754 learned_addr learned my addr 172.21.15.66:0/2543875754 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc22b400640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2100047d0 0x7fc22c0ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc22be00640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc22c1557e0 0x7fc22c155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223e00640 1 -- 172.21.15.66:0/2543875754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2100047d0 msgr2=0x7fc22c0ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223e00640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2100047d0 0x7fc22c0ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223e00640 1 -- 172.21.15.66:0/2543875754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc22c1557e0 msgr2=0x7fc22c155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223e00640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc22c1557e0 0x7fc22c155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223e00640 1 -- 172.21.15.66:0/2543875754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c15c100 con 0x7fc22c06cab0 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc22b400640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2100047d0 0x7fc22c0ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:31.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc22be00640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc22c1557e0 0x7fc22c155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223e00640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc22c06cab0 0x7fc22c0aef20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc214000c10 tx=0x7fc210005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223400640 1 -- 172.21.15.66:0/2543875754 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140b5070 con 0x7fc22c06cab0 2024-03-30T00:41:31.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223400640 1 -- 172.21.15.66:0/2543875754 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21400f000 con 0x7fc22c06cab0 2024-03-30T00:41:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc223400640 1 -- 172.21.15.66:0/2543875754 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140a7460 con 0x7fc22c06cab0 2024-03-30T00:41:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc232200640 1 -- 172.21.15.66:0/2543875754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c1561d0 con 0x7fc22c06cab0 2024-03-30T00:41:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc232200640 1 -- 172.21.15.66:0/2543875754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c0ac310 con 0x7fc22c06cab0 2024-03-30T00:41:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.985+0000 7fc232200640 1 -- 172.21.15.66:0/2543875754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c0b0ec0 con 0x7fc22c06cab0 2024-03-30T00:41:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.989+0000 7fc223400640 1 -- 172.21.15.66:0/2543875754 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc2140a75c0 con 0x7fc22c06cab0 2024-03-30T00:41:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.989+0000 7fc223400640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2080666c0 0x7fc208068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.989+0000 7fc223400640 1 -- 172.21.15.66:0/2543875754 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc21411e1c0 con 0x7fc22c06cab0 2024-03-30T00:41:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.989+0000 7fc22b400640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2080666c0 0x7fc208068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:31.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.989+0000 7fc22b400640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2080666c0 0x7fc208068b80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0058c0 tx=0x7fc21c000f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:32.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:31.993+0000 7fc223400640 1 -- 172.21.15.66:0/2543875754 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc2140eb3e0 con 0x7fc22c06cab0 2024-03-30T00:41:32.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.153+0000 7fc232200640 1 -- 172.21.15.66:0/2543875754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc22c066c30 con 0x7fc22c06cab0 2024-03-30T00:41:32.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.153+0000 7fc223400640 1 -- 172.21.15.66:0/2543875754 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2140eea00 con 0x7fc22c06cab0 2024-03-30T00:41:32.163 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:41:32.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 -- 172.21.15.66:0/2543875754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2080666c0 msgr2=0x7fc208068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2080666c0 0x7fc208068b80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0058c0 tx=0x7fc21c000f60 comp rx=0 tx=0).stop 2024-03-30T00:41:32.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 -- 172.21.15.66:0/2543875754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc22c06cab0 msgr2=0x7fc22c0aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc22c06cab0 0x7fc22c0aef20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc214000c10 tx=0x7fc210005800 comp rx=0 tx=0).stop 2024-03-30T00:41:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 -- 172.21.15.66:0/2543875754 shutdown_connections 2024-03-30T00:41:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc22c1557e0 0x7fc22c155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc2080666c0 0x7fc208068b80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc22c06cab0 0x7fc22c0aef20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 --2- 172.21.15.66:0/2543875754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2100047d0 0x7fc22c0ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 -- 172.21.15.66:0/2543875754 >> 172.21.15.66:0/2543875754 conn(0x7fc22c069f60 msgr2=0x7fc22c06d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 -- 172.21.15.66:0/2543875754 shutdown_connections 2024-03-30T00:41:32.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.157+0000 7fc232200640 1 -- 172.21.15.66:0/2543875754 wait complete. 2024-03-30T00:41:32.185 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:41:32.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.317+0000 7f9fe2000640 1 Processor -- start 2024-03-30T00:41:32.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.317+0000 7f9fe2000640 1 -- start start 2024-03-30T00:41:32.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.317+0000 7f9fe2000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fdc0cda90 0x7f9fdc0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.317+0000 7f9fe2000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9fdc0cde70 0x7f9fdc0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.317+0000 7f9fe2000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc15a990 con 0x7f9fdc159c60 2024-03-30T00:41:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.317+0000 7f9fe2000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc0b1b80 con 0x7f9fdc0cde70 2024-03-30T00:41:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.317+0000 7f9fe2000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0cd3e0 con 0x7f9fdc0cda90 2024-03-30T00:41:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.317+0000 7f9fd2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9fdc0cde70 0x7f9fdc0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.317+0000 7f9fdbe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9fdc159c60 0x7f9fdc15a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55322/0 (socket says 172.21.15.66:55322) 2024-03-30T00:41:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.321+0000 7f9fdbe00640 1 -- 172.21.15.66:0/670747272 learned_addr learned my addr 172.21.15.66:0/670747272 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.321+0000 7f9fdb400640 1 --2- 172.21.15.66:0/670747272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fdc0cda90 0x7f9fdc0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.321+0000 7f9fd2a00640 1 -- 172.21.15.66:0/670747272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fdc0cda90 msgr2=0x7f9fdc0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:32.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.321+0000 7f9fd2a00640 1 --2- 172.21.15.66:0/670747272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fdc0cda90 0x7f9fdc0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.321+0000 7f9fd2a00640 1 -- 172.21.15.66:0/670747272 >> v1:172.21.15.66:6789/0 conn(0x7f9fdc159c60 legacy=0x7f9fdc15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.321+0000 7f9fdaa00640 1 -- 172.21.15.66:0/670747272 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632269600 0 0) 0x7f9fdc15a990 con 0x7f9fdc159c60 2024-03-30T00:41:32.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.321+0000 7f9fd2a00640 1 -- 172.21.15.66:0/670747272 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0b11e0 con 0x7f9fdc0cde70 2024-03-30T00:41:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.321+0000 7f9fd2a00640 1 --2- 172.21.15.66:0/670747272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fdc0cde70 0x7f9fdc0cea60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009d90 tx=0x7f9fc401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c5f4700d530de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.321+0000 7f9fdaa00640 1 -- 172.21.15.66:0/670747272 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40a58b0 con 0x7f9fdc0cde70 2024-03-30T00:41:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.321+0000 7f9fdaa00640 1 -- 172.21.15.66:0/670747272 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc40a5a10 con 0x7f9fdc0cde70 2024-03-30T00:41:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.321+0000 7f9fdaa00640 1 -- 172.21.15.66:0/670747272 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40a5da0 con 0x7f9fdc0cde70 2024-03-30T00:41:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 -- 172.21.15.66:0/670747272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fdc0cde70 msgr2=0x7f9fdc0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 --2- 172.21.15.66:0/670747272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fdc0cde70 0x7f9fdc0cea60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009d90 tx=0x7f9fc401c8e0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 -- 172.21.15.66:0/670747272 shutdown_connections 2024-03-30T00:41:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 --2- 172.21.15.66:0/670747272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fdc0cde70 0x7f9fdc0cea60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 --2- 172.21.15.66:0/670747272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fdc0cda90 0x7f9fdc0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 -- 172.21.15.66:0/670747272 >> 172.21.15.66:0/670747272 conn(0x7f9fdc069f60 msgr2=0x7f9fdc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 -- 172.21.15.66:0/670747272 shutdown_connections 2024-03-30T00:41:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 -- 172.21.15.66:0/670747272 wait complete. 2024-03-30T00:41:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 Processor -- start 2024-03-30T00:41:32.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 -- start start 2024-03-30T00:41:32.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fdc0cda90 0x7f9fdc0c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fdc0cde70 0x7f9fdc153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fdc159c60 0x7f9fdc0973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc153fd0 con 0x7f9fdc0cda90 2024-03-30T00:41:32.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0c37e0 con 0x7f9fdc159c60 2024-03-30T00:41:32.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fe2000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0c3ac0 con 0x7f9fdc0cde70 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fdb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fdc0cda90 0x7f9fdc0c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fdb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fdc0cda90 0x7f9fdc0c3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44118/0 (socket says 172.21.15.66:44118) 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fdb400640 1 -- 172.21.15.66:0/1751045864 learned_addr learned my addr 172.21.15.66:0/1751045864 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fdbe00640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fdc159c60 0x7f9fdc0973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fd2a00640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fdc0cde70 0x7f9fdc153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fdb400640 1 -- 172.21.15.66:0/1751045864 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fdc0cde70 msgr2=0x7f9fdc153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fdb400640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fdc0cde70 0x7f9fdc153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fdb400640 1 -- 172.21.15.66:0/1751045864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fdc159c60 msgr2=0x7f9fdc0973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fdb400640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fdc159c60 0x7f9fdc0973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fdb400640 1 -- 172.21.15.66:0/1751045864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc4009a70 con 0x7f9fdc0cda90 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fd2a00640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fdc0cde70 0x7f9fdc153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fdbe00640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fdc159c60 0x7f9fdc0973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:32.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.325+0000 7f9fdb400640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fdc0cda90 0x7f9fdc0c3100 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00d5f0 tx=0x7f9fcc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:32.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.329+0000 7f9fd3e00640 1 -- 172.21.15.66:0/1751045864 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0125f0 con 0x7f9fdc0cda90 2024-03-30T00:41:32.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.329+0000 7f9fd3e00640 1 -- 172.21.15.66:0/1751045864 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc012750 con 0x7f9fdc0cda90 2024-03-30T00:41:32.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.329+0000 7f9fd3e00640 1 -- 172.21.15.66:0/1751045864 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc012ae0 con 0x7f9fdc0cda90 2024-03-30T00:41:32.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.329+0000 7f9fe2000640 1 -- 172.21.15.66:0/1751045864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc0c3da0 con 0x7f9fdc0cda90 2024-03-30T00:41:32.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.329+0000 7f9fe2000640 1 -- 172.21.15.66:0/1751045864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc0c4360 con 0x7f9fdc0cda90 2024-03-30T00:41:32.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.329+0000 7f9fe2000640 1 -- 172.21.15.66:0/1751045864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc066c30 con 0x7f9fdc0cda90 2024-03-30T00:41:32.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.329+0000 7f9fd3e00640 1 -- 172.21.15.66:0/1751045864 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9fcc012c40 con 0x7f9fdc0cda90 2024-03-30T00:41:32.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.329+0000 7f9fd3e00640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9fb80666c0 0x7f9fb8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.329+0000 7f9fd3e00640 1 -- 172.21.15.66:0/1751045864 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9fcc084810 con 0x7f9fdc0cda90 2024-03-30T00:41:32.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.329+0000 7f9fd2a00640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9fb80666c0 0x7f9fb8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.329+0000 7f9fd2a00640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9fb80666c0 0x7f9fb8068b80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40af000 tx=0x7f9fc40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:32.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.333+0000 7f9fd3e00640 1 -- 172.21.15.66:0/1751045864 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9fcc0519a0 con 0x7f9fdc0cda90 2024-03-30T00:41:32.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.493+0000 7f9fe2000640 1 -- 172.21.15.66:0/1751045864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9fdc0b3170 con 0x7f9fdc0cda90 2024-03-30T00:41:32.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.493+0000 7f9fd3e00640 1 -- 172.21.15.66:0/1751045864 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9fcc007380 con 0x7f9fdc0cda90 2024-03-30T00:41:32.504 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:41:32.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 -- 172.21.15.66:0/1751045864 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9fb80666c0 msgr2=0x7f9fb8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9fb80666c0 0x7f9fb8068b80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40af000 tx=0x7f9fc40a7070 comp rx=0 tx=0).stop 2024-03-30T00:41:32.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 -- 172.21.15.66:0/1751045864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fdc0cda90 msgr2=0x7f9fdc0c3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fdc0cda90 0x7f9fdc0c3100 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00d5f0 tx=0x7f9fcc00dab0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 -- 172.21.15.66:0/1751045864 shutdown_connections 2024-03-30T00:41:32.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fdc159c60 0x7f9fdc0973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fdc0cde70 0x7f9fdc153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9fb80666c0 0x7f9fb8068b80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 --2- 172.21.15.66:0/1751045864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fdc0cda90 0x7f9fdc0c3100 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 -- 172.21.15.66:0/1751045864 >> 172.21.15.66:0/1751045864 conn(0x7f9fdc069f60 msgr2=0x7f9fdc06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:32.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 -- 172.21.15.66:0/1751045864 shutdown_connections 2024-03-30T00:41:32.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.497+0000 7f9fe2000640 1 -- 172.21.15.66:0/1751045864 wait complete. 2024-03-30T00:41:32.529 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:41:32.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.661+0000 7f37aae00640 1 Processor -- start 2024-03-30T00:41:32.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.661+0000 7f37aae00640 1 -- start start 2024-03-30T00:41:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.661+0000 7f37aae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37a415ca10 0x7f37a406ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.661+0000 7f37aae00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f37a40ce820 0x7f37a40cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.661+0000 7f37aae00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a406e7d0 con 0x7f37a40ae4e0 2024-03-30T00:41:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.661+0000 7f37aae00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a40b1f70 con 0x7f37a40ce820 2024-03-30T00:41:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.661+0000 7f37aae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a40cf1f0 con 0x7f37a415ca10 2024-03-30T00:41:32.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.661+0000 7f37a8e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f37a40ce820 0x7f37a40cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.661+0000 7f37a8e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f37a40ce820 0x7f37a40cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:32846/0 (socket says 172.21.15.66:32846) 2024-03-30T00:41:32.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.661+0000 7f37a8e00640 1 -- 172.21.15.66:0/3139180359 learned_addr learned my addr 172.21.15.66:0/3139180359 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:32.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.661+0000 7f37a3e00640 1 --2- 172.21.15.66:0/3139180359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37a415ca10 0x7f37a406ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37a8e00640 1 -- 172.21.15.66:0/3139180359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37a415ca10 msgr2=0x7f37a406ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:32.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37a8e00640 1 --2- 172.21.15.66:0/3139180359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37a415ca10 0x7f37a406ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37a8e00640 1 -- 172.21.15.66:0/3139180359 >> v1:172.21.15.156:6789/0 conn(0x7f37a40ae4e0 legacy=0x7f37a40cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37a8e00640 1 -- 172.21.15.66:0/3139180359 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a406f1a0 con 0x7f37a40ce820 2024-03-30T00:41:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37a8e00640 1 --2- 172.21.15.66:0/3139180359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a40ce820 0x7f37a40cecb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f379c009e80 tx=0x7f379c01ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=205eaaf44cd8eb08 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37a3400640 1 -- 172.21.15.66:0/3139180359 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0a57f0 con 0x7f37a40ce820 2024-03-30T00:41:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37a3400640 1 -- 172.21.15.66:0/3139180359 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c0a5950 con 0x7f37a40ce820 2024-03-30T00:41:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37a3400640 1 -- 172.21.15.66:0/3139180359 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0a5ce0 con 0x7f37a40ce820 2024-03-30T00:41:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37aae00640 1 -- 172.21.15.66:0/3139180359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a40ce820 msgr2=0x7f37a40cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37aae00640 1 --2- 172.21.15.66:0/3139180359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a40ce820 0x7f37a40cecb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f379c009e80 tx=0x7f379c01ca10 comp rx=0 tx=0).stop 2024-03-30T00:41:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37aae00640 1 -- 172.21.15.66:0/3139180359 shutdown_connections 2024-03-30T00:41:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37aae00640 1 --2- 172.21.15.66:0/3139180359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a40ce820 0x7f37a40cecb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37aae00640 1 --2- 172.21.15.66:0/3139180359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37a415ca10 0x7f37a406ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.665+0000 7f37aae00640 1 -- 172.21.15.66:0/3139180359 >> 172.21.15.66:0/3139180359 conn(0x7f37a4098a20 msgr2=0x7f37a40b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37aae00640 1 -- 172.21.15.66:0/3139180359 shutdown_connections 2024-03-30T00:41:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37aae00640 1 -- 172.21.15.66:0/3139180359 wait complete. 2024-03-30T00:41:32.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37aae00640 1 Processor -- start 2024-03-30T00:41:32.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37aae00640 1 -- start start 2024-03-30T00:41:32.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37aae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37a40ae4e0 0x7f37a4154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37aae00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a40ce820 0x7f37a415d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37aae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a415ca10 0x7f37a415d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37aae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a415e1c0 con 0x7f37a415ca10 2024-03-30T00:41:32.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37aae00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a415e4a0 con 0x7f37a40ce820 2024-03-30T00:41:32.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37aae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a415e780 con 0x7f37a40ae4e0 2024-03-30T00:41:32.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37a8e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a415ca10 0x7f37a415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37a8e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a415ca10 0x7f37a415d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44132/0 (socket says 172.21.15.66:44132) 2024-03-30T00:41:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37a8e00640 1 -- 172.21.15.66:0/3367455764 learned_addr learned my addr 172.21.15.66:0/3367455764 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37a3e00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37a40ae4e0 0x7f37a4154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f379be00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a40ce820 0x7f37a415d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37a8e00640 1 -- 172.21.15.66:0/3367455764 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37a40ae4e0 msgr2=0x7f37a4154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37a8e00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37a40ae4e0 0x7f37a4154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37a8e00640 1 -- 172.21.15.66:0/3367455764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a40ce820 msgr2=0x7f37a415d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37a8e00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a40ce820 0x7f37a415d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37a8e00640 1 -- 172.21.15.66:0/3367455764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c009b30 con 0x7f37a415ca10 2024-03-30T00:41:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.669+0000 7f37a8e00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a415ca10 0x7f37a415d550 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f379c01a640 tx=0x7f379c0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:32.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.673+0000 7f37a0c00640 1 -- 172.21.15.66:0/3367455764 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0b8880 con 0x7f37a415ca10 2024-03-30T00:41:32.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.673+0000 7f37a0c00640 1 -- 172.21.15.66:0/3367455764 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c0b89e0 con 0x7f37a415ca10 2024-03-30T00:41:32.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.673+0000 7f37a0c00640 1 -- 172.21.15.66:0/3367455764 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0b8d90 con 0x7f37a415ca10 2024-03-30T00:41:32.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.673+0000 7f37aae00640 1 -- 172.21.15.66:0/3367455764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a40c4630 con 0x7f37a415ca10 2024-03-30T00:41:32.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.673+0000 7f37aae00640 1 -- 172.21.15.66:0/3367455764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a415fbd0 con 0x7f37a415ca10 2024-03-30T00:41:32.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.673+0000 7f37aae00640 1 -- 172.21.15.66:0/3367455764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a415e9a0 con 0x7f37a415ca10 2024-03-30T00:41:32.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.673+0000 7f37a0c00640 1 -- 172.21.15.66:0/3367455764 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f379c0c3460 con 0x7f37a415ca10 2024-03-30T00:41:32.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.673+0000 7f37a0c00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3780066690 0x7f3780068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:32.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.673+0000 7f37a0c00640 1 -- 172.21.15.66:0/3367455764 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f379c11f8f0 con 0x7f37a415ca10 2024-03-30T00:41:32.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.673+0000 7f37a3e00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3780066690 0x7f3780068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:32.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.673+0000 7f37a3e00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3780066690 0x7f3780068b50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3794009b60 tx=0x7f3794009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:32.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.677+0000 7f37a0c00640 1 -- 172.21.15.66:0/3367455764 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f379c0ecc40 con 0x7f37a415ca10 2024-03-30T00:41:32.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.853+0000 7f37aae00640 1 -- 172.21.15.66:0/3367455764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f37a4066c30 con 0x7f37a415ca10 2024-03-30T00:41:32.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.853+0000 7f37a0c00640 1 -- 172.21.15.66:0/3367455764 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7f379c009510 con 0x7f37a415ca10 2024-03-30T00:41:32.863 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:41:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 -- 172.21.15.66:0/3367455764 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3780066690 msgr2=0x7f3780068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3780066690 0x7f3780068b50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3794009b60 tx=0x7f3794009190 comp rx=0 tx=0).stop 2024-03-30T00:41:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 -- 172.21.15.66:0/3367455764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a415ca10 msgr2=0x7f37a415d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a415ca10 0x7f37a415d550 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f379c01a640 tx=0x7f379c0197b0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 -- 172.21.15.66:0/3367455764 shutdown_connections 2024-03-30T00:41:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3780066690 0x7f3780068b50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37a415ca10 0x7f37a415d550 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a40ce820 0x7f37a415d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 --2- 172.21.15.66:0/3367455764 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37a40ae4e0 0x7f37a4154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 -- 172.21.15.66:0/3367455764 >> 172.21.15.66:0/3367455764 conn(0x7f37a4098a20 msgr2=0x7f37a40b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 -- 172.21.15.66:0/3367455764 shutdown_connections 2024-03-30T00:41:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:32.857+0000 7f37aae00640 1 -- 172.21.15.66:0/3367455764 wait complete. 2024-03-30T00:41:32.884 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:41:32.884 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:41:32.893 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:33.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.069+0000 7f440ae00640 1 Processor -- start 2024-03-30T00:41:33.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.069+0000 7f440ae00640 1 -- start start 2024-03-30T00:41:33.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440ae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f440406ba80 0x7f440406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440ae00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44040b2b60 0x7f44040caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440ae00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44040ce070 con 0x7f440415c9f0 2024-03-30T00:41:33.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440ae00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440406cf60 con 0x7f44040b2b60 2024-03-30T00:41:33.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440ae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44040ce350 con 0x7f440406ba80 2024-03-30T00:41:33.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440a400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44040b2b60 0x7f44040caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440a400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44040b2b60 0x7f44040caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44138/0 (socket says 172.21.15.66:44138) 2024-03-30T00:41:33.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440a400640 1 -- 172.21.15.66:0/3377084596 learned_addr learned my addr 172.21.15.66:0/3377084596 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:33.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f4409000640 1 --2- 172.21.15.66:0/3377084596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f440406ba80 0x7f440406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440a400640 1 -- 172.21.15.66:0/3377084596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f440406ba80 msgr2=0x7f440406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:33.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440a400640 1 --2- 172.21.15.66:0/3377084596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f440406ba80 0x7f440406ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440a400640 1 -- 172.21.15.66:0/3377084596 >> v1:172.21.15.156:6789/0 conn(0x7f440415c9f0 legacy=0x7f440406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.073+0000 7f440a400640 1 -- 172.21.15.66:0/3377084596 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44040ce630 con 0x7f44040b2b60 2024-03-30T00:41:33.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440a400640 1 --2- 172.21.15.66:0/3377084596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44040b2b60 0x7f44040caf60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009e80 tx=0x7f43fc01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17b051da13045db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:33.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f4403e00640 1 -- 172.21.15.66:0/3377084596 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0a58a0 con 0x7f44040b2b60 2024-03-30T00:41:33.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f4403e00640 1 -- 172.21.15.66:0/3377084596 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc0a5a00 con 0x7f44040b2b60 2024-03-30T00:41:33.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f4403e00640 1 -- 172.21.15.66:0/3377084596 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0a5d90 con 0x7f44040b2b60 2024-03-30T00:41:33.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 -- 172.21.15.66:0/3377084596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44040b2b60 msgr2=0x7f44040caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 --2- 172.21.15.66:0/3377084596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44040b2b60 0x7f44040caf60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009e80 tx=0x7f43fc01c9e0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 -- 172.21.15.66:0/3377084596 shutdown_connections 2024-03-30T00:41:33.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 --2- 172.21.15.66:0/3377084596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44040b2b60 0x7f44040caf60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 --2- 172.21.15.66:0/3377084596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f440406ba80 0x7f440406ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 -- 172.21.15.66:0/3377084596 >> 172.21.15.66:0/3377084596 conn(0x7f440406e0d0 msgr2=0x7f440406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:33.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 -- 172.21.15.66:0/3377084596 shutdown_connections 2024-03-30T00:41:33.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 -- 172.21.15.66:0/3377084596 wait complete. 2024-03-30T00:41:33.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 Processor -- start 2024-03-30T00:41:33.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 -- start start 2024-03-30T00:41:33.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f440406ba80 0x7f44040b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44040b2b60 0x7f4404152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f440415c9f0 0x7f440415e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44040b49a0 con 0x7f440415c9f0 2024-03-30T00:41:33.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44040b4c80 con 0x7f440406ba80 2024-03-30T00:41:33.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.077+0000 7f440ae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44040b4f60 con 0x7f44040b2b60 2024-03-30T00:41:33.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f440415c9f0 0x7f440415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f440415c9f0 0x7f440415e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44152/0 (socket says 172.21.15.66:44152) 2024-03-30T00:41:33.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440a400640 1 -- 172.21.15.66:0/2123304950 learned_addr learned my addr 172.21.15.66:0/2123304950 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:33.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f4409000640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44040b2b60 0x7f4404152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f4409a00640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f440406ba80 0x7f44040b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440a400640 1 -- 172.21.15.66:0/2123304950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44040b2b60 msgr2=0x7f4404152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440a400640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44040b2b60 0x7f4404152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440a400640 1 -- 172.21.15.66:0/2123304950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f440406ba80 msgr2=0x7f44040b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440a400640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f440406ba80 0x7f44040b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440a400640 1 -- 172.21.15.66:0/2123304950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc009b30 con 0x7f440415c9f0 2024-03-30T00:41:33.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f4409a00640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f440406ba80 0x7f44040b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:33.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440a400640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f440415c9f0 0x7f440415e890 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0af040 tx=0x7f43fc019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:33.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f4401600640 1 -- 172.21.15.66:0/2123304950 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0b8670 con 0x7f440415c9f0 2024-03-30T00:41:33.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f4401600640 1 -- 172.21.15.66:0/2123304950 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc0b87f0 con 0x7f440415c9f0 2024-03-30T00:41:33.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f4401600640 1 -- 172.21.15.66:0/2123304950 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0b8ba0 con 0x7f440415c9f0 2024-03-30T00:41:33.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440ae00640 1 -- 172.21.15.66:0/2123304950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44040b5240 con 0x7f440415c9f0 2024-03-30T00:41:33.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440ae00640 1 -- 172.21.15.66:0/2123304950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44040b4650 con 0x7f440415c9f0 2024-03-30T00:41:33.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f440ae00640 1 -- 172.21.15.66:0/2123304950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4404073cf0 con 0x7f440415c9f0 2024-03-30T00:41:33.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f4401600640 1 -- 172.21.15.66:0/2123304950 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f43fc0b8d00 con 0x7f440415c9f0 2024-03-30T00:41:33.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.081+0000 7f4401600640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43e4066690 0x7f43e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.085+0000 7f4401600640 1 -- 172.21.15.66:0/2123304950 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f43fc11fae0 con 0x7f440415c9f0 2024-03-30T00:41:33.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.085+0000 7f4409a00640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43e4066690 0x7f43e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.085+0000 7f4409a00640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43e4066690 0x7f43e4068b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f43f8000f70 tx=0x7f43f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:33.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.089+0000 7f4401600640 1 -- 172.21.15.66:0/2123304950 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f43fc0ecce0 con 0x7f440415c9f0 2024-03-30T00:41:33.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.253+0000 7f440ae00640 1 -- 172.21.15.66:0/2123304950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44040b56f0 con 0x7f440415c9f0 2024-03-30T00:41:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.253+0000 7f4401600640 1 -- 172.21.15.66:0/2123304950 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16355 (secure 0 0 0) 0x7f43fc0f0300 con 0x7f440415c9f0 2024-03-30T00:41:33.263 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:33.263 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":41,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:41:32.502246+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:41:19.292693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T00:41:20.267522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:33.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 -- 172.21.15.66:0/2123304950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43e4066690 msgr2=0x7f43e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43e4066690 0x7f43e4068b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f43f8000f70 tx=0x7f43f8009040 comp rx=0 tx=0).stop 2024-03-30T00:41:33.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 -- 172.21.15.66:0/2123304950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f440415c9f0 msgr2=0x7f440415e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f440415c9f0 0x7f440415e890 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0af040 tx=0x7f43fc019780 comp rx=0 tx=0).stop 2024-03-30T00:41:33.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 -- 172.21.15.66:0/2123304950 shutdown_connections 2024-03-30T00:41:33.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43e4066690 0x7f43e4068b50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f440415c9f0 0x7f440415e890 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44040b2b60 0x7f4404152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 --2- 172.21.15.66:0/2123304950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f440406ba80 0x7f44040b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 -- 172.21.15.66:0/2123304950 >> 172.21.15.66:0/2123304950 conn(0x7f440406e0d0 msgr2=0x7f44040cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:33.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 -- 172.21.15.66:0/2123304950 shutdown_connections 2024-03-30T00:41:33.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.257+0000 7f440ae00640 1 -- 172.21.15.66:0/2123304950 wait complete. 2024-03-30T00:41:33.284 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:41:19.292693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:41:20.267522+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:41:33.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:41:33.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.417+0000 7f7bfae00640 1 Processor -- start 2024-03-30T00:41:33.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.417+0000 7f7bfae00640 1 -- start start 2024-03-30T00:41:33.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bfae00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7bf41610c0 0x7f7bf41604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bfae00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7bf41609f0 0x7f7bf40abc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bfae00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf406da20 con 0x7f7bf4157060 2024-03-30T00:41:33.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bfae00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf4157440 con 0x7f7bf41609f0 2024-03-30T00:41:33.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bfae00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf4160e80 con 0x7f7bf41610c0 2024-03-30T00:41:33.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bf8e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7bf41609f0 0x7f7bf40abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bf8e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7bf41609f0 0x7f7bf40abc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44158/0 (socket says 172.21.15.66:44158) 2024-03-30T00:41:33.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bf8e00640 1 -- 172.21.15.66:0/2805989474 learned_addr learned my addr 172.21.15.66:0/2805989474 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:33.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7beb400640 1 --2- 172.21.15.66:0/2805989474 >> v2:172.21.15.156:3300/0 conn(0x7f7bf41610c0 0x7f7bf41604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bf8e00640 1 -- 172.21.15.66:0/2805989474 >> v2:172.21.15.156:3300/0 conn(0x7f7bf41610c0 msgr2=0x7f7bf41604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:33.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bf8e00640 1 --2- 172.21.15.66:0/2805989474 >> v2:172.21.15.156:3300/0 conn(0x7f7bf41610c0 0x7f7bf41604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bf8e00640 1 -- 172.21.15.66:0/2805989474 >> v1:172.21.15.156:6789/0 conn(0x7f7bf4157060 legacy=0x7f7bf406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:33.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bf8e00640 1 -- 172.21.15.66:0/2805989474 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf406dc60 con 0x7f7bf41609f0 2024-03-30T00:41:33.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.421+0000 7f7bf8e00640 1 --2- 172.21.15.66:0/2805989474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bf41609f0 0x7f7bf40abc00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7bf406f3c0 tx=0x7f7bec01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0d01947993dd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:33.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bf3400640 1 -- 172.21.15.66:0/2805989474 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0ab600 con 0x7f7bf41609f0 2024-03-30T00:41:33.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bf3400640 1 -- 172.21.15.66:0/2805989474 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec0a7050 con 0x7f7bf41609f0 2024-03-30T00:41:33.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bf3400640 1 -- 172.21.15.66:0/2805989474 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0ab980 con 0x7f7bf41609f0 2024-03-30T00:41:33.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 -- 172.21.15.66:0/2805989474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bf41609f0 msgr2=0x7f7bf40abc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 --2- 172.21.15.66:0/2805989474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bf41609f0 0x7f7bf40abc00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7bf406f3c0 tx=0x7f7bec01c3c0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 -- 172.21.15.66:0/2805989474 shutdown_connections 2024-03-30T00:41:33.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 --2- 172.21.15.66:0/2805989474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bf41609f0 0x7f7bf40abc00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 --2- 172.21.15.66:0/2805989474 >> v2:172.21.15.156:3300/0 conn(0x7f7bf41610c0 0x7f7bf41604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 -- 172.21.15.66:0/2805989474 >> 172.21.15.66:0/2805989474 conn(0x7f7bf4071750 msgr2=0x7f7bf4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:33.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 -- 172.21.15.66:0/2805989474 shutdown_connections 2024-03-30T00:41:33.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 -- 172.21.15.66:0/2805989474 wait complete. 2024-03-30T00:41:33.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 Processor -- start 2024-03-30T00:41:33.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 -- start start 2024-03-30T00:41:33.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bf4157060 0x7f7bf4144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bf41609f0 0x7f7bf4154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bf41610c0 0x7f7bf41450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf41559e0 con 0x7f7bf41609f0 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf40b21e0 con 0x7f7bf41610c0 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bfae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf414ed00 con 0x7f7bf4157060 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7beb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bf41609f0 0x7f7bf4154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7beb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bf41609f0 0x7f7bf4154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44170/0 (socket says 172.21.15.66:44170) 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7beb400640 1 -- 172.21.15.66:0/3195362592 learned_addr learned my addr 172.21.15.66:0/3195362592 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bf3e00640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bf4157060 0x7f7bf4144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.425+0000 7f7bf8e00640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bf41610c0 0x7f7bf41450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7beb400640 1 -- 172.21.15.66:0/3195362592 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bf4157060 msgr2=0x7f7bf4144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7beb400640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bf4157060 0x7f7bf4144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7beb400640 1 -- 172.21.15.66:0/3195362592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bf41610c0 msgr2=0x7f7bf41450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7beb400640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bf41610c0 0x7f7bf41450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7beb400640 1 -- 172.21.15.66:0/3195362592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec0094d0 con 0x7f7bf41609f0 2024-03-30T00:41:33.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7beb400640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bf41609f0 0x7f7bf4154ff0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00d7d0 tx=0x7f7bdc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:33.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7bf0c00640 1 -- 172.21.15.66:0/3195362592 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00b6c0 con 0x7f7bf41609f0 2024-03-30T00:41:33.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7bf0c00640 1 -- 172.21.15.66:0/3195362592 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bdc00b820 con 0x7f7bf41609f0 2024-03-30T00:41:33.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7bf0c00640 1 -- 172.21.15.66:0/3195362592 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00bbd0 con 0x7f7bf41609f0 2024-03-30T00:41:33.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7bfae00640 1 -- 172.21.15.66:0/3195362592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf414efe0 con 0x7f7bf41609f0 2024-03-30T00:41:33.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7bfae00640 1 -- 172.21.15.66:0/3195362592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf4144e40 con 0x7f7bf41609f0 2024-03-30T00:41:33.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7bfae00640 1 -- 172.21.15.66:0/3195362592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf4066c30 con 0x7f7bf41609f0 2024-03-30T00:41:33.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7bf0c00640 1 -- 172.21.15.66:0/3195362592 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7bdc00bd30 con 0x7f7bf41609f0 2024-03-30T00:41:33.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7bf0c00640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7bd0066690 0x7f7bd0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7bf0c00640 1 -- 172.21.15.66:0/3195362592 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7bdc084790 con 0x7f7bf41609f0 2024-03-30T00:41:33.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.429+0000 7f7bf3e00640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7bd0066690 0x7f7bd0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.433+0000 7f7bf3e00640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7bd0066690 0x7f7bd0068b50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7be400b450 tx=0x7f7be400d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:33.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.433+0000 7f7bf0c00640 1 -- 172.21.15.66:0/3195362592 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7bdc0519b0 con 0x7f7bf41609f0 2024-03-30T00:41:33.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.601+0000 7f7bfae00640 1 -- 172.21.15.66:0/3195362592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bf4071f30 con 0x7f7bf41609f0 2024-03-30T00:41:33.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.601+0000 7f7bf0c00640 1 -- 172.21.15.66:0/3195362592 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bdc054fd0 con 0x7f7bf41609f0 2024-03-30T00:41:33.612 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:41:33.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 -- 172.21.15.66:0/3195362592 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7bd0066690 msgr2=0x7f7bd0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7bd0066690 0x7f7bd0068b50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7be400b450 tx=0x7f7be400d040 comp rx=0 tx=0).stop 2024-03-30T00:41:33.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 -- 172.21.15.66:0/3195362592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bf41609f0 msgr2=0x7f7bf4154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bf41609f0 0x7f7bf4154ff0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00d7d0 tx=0x7f7bdc00dc90 comp rx=0 tx=0).stop 2024-03-30T00:41:33.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 -- 172.21.15.66:0/3195362592 shutdown_connections 2024-03-30T00:41:33.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bf41610c0 0x7f7bf41450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7bd0066690 0x7f7bd0068b50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bf41609f0 0x7f7bf4154ff0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 --2- 172.21.15.66:0/3195362592 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bf4157060 0x7f7bf4144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 -- 172.21.15.66:0/3195362592 >> 172.21.15.66:0/3195362592 conn(0x7f7bf4071750 msgr2=0x7f7bf41602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:33.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 -- 172.21.15.66:0/3195362592 shutdown_connections 2024-03-30T00:41:33.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.605+0000 7f7bfae00640 1 -- 172.21.15.66:0/3195362592 wait complete. 2024-03-30T00:41:33.634 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:41:33.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f93400640 1 Processor -- start 2024-03-30T00:41:33.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f93400640 1 -- start start 2024-03-30T00:41:33.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f93400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f8c15de30 0x7f0f8c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f93400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0f8c0af420 0x7f0f8c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f93400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f8c0ce740 0x7f0f8c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f93400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c0cf0c0 con 0x7f0f8c0af420 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f93400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c0cbb90 con 0x7f0f8c15de30 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f93400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0ce1b0 con 0x7f0f8c0ce740 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f8be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0f8c0af420 0x7f0f8c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f90a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f8c15de30 0x7f0f8c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f8be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0f8c0af420 0x7f0f8c0d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44186/0 (socket says 172.21.15.66:44186) 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f8be00640 1 -- 172.21.15.66:0/3981921078 learned_addr learned my addr 172.21.15.66:0/3981921078 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f91400640 1 --2- 172.21.15.66:0/3981921078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f8c0ce740 0x7f0f8c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f8be00640 1 -- 172.21.15.66:0/3981921078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f8c0ce740 msgr2=0x7f0f8c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f8be00640 1 --2- 172.21.15.66:0/3981921078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f8c0ce740 0x7f0f8c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f8be00640 1 -- 172.21.15.66:0/3981921078 >> v2:172.21.15.156:3300/0 conn(0x7f0f8c15de30 msgr2=0x7f0f8c0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f8be00640 1 --2- 172.21.15.66:0/3981921078 >> v2:172.21.15.156:3300/0 conn(0x7f0f8c15de30 0x7f0f8c0cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f8be00640 1 -- 172.21.15.66:0/3981921078 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c0afa10 con 0x7f0f8c0af420 2024-03-30T00:41:33.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.769+0000 7f0f90a00640 1 --2- 172.21.15.66:0/3981921078 >> v2:172.21.15.156:3300/0 conn(0x7f0f8c15de30 0x7f0f8c0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:33.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f8be00640 1 --2- 172.21.15.66:0/3981921078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f8c0af420 0x7f0f8c0d1060 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c06db30 tx=0x7f0f7801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c1554800893a21 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:33.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f8b400640 1 -- 172.21.15.66:0/3981921078 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f780ab610 con 0x7f0f8c0af420 2024-03-30T00:41:33.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f8b400640 1 -- 172.21.15.66:0/3981921078 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f780ab770 con 0x7f0f8c0af420 2024-03-30T00:41:33.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f8b400640 1 -- 172.21.15.66:0/3981921078 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f780abb00 con 0x7f0f8c0af420 2024-03-30T00:41:33.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f93400640 1 -- 172.21.15.66:0/3981921078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f8c0af420 msgr2=0x7f0f8c0d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f93400640 1 --2- 172.21.15.66:0/3981921078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f8c0af420 0x7f0f8c0d1060 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c06db30 tx=0x7f0f7801c430 comp rx=0 tx=0).stop 2024-03-30T00:41:33.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f93400640 1 -- 172.21.15.66:0/3981921078 shutdown_connections 2024-03-30T00:41:33.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f93400640 1 --2- 172.21.15.66:0/3981921078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f8c0ce740 0x7f0f8c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f93400640 1 --2- 172.21.15.66:0/3981921078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f8c0af420 0x7f0f8c0d1060 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f93400640 1 --2- 172.21.15.66:0/3981921078 >> v2:172.21.15.156:3300/0 conn(0x7f0f8c15de30 0x7f0f8c0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f93400640 1 -- 172.21.15.66:0/3981921078 >> 172.21.15.66:0/3981921078 conn(0x7f0f8c0d0540 msgr2=0x7f0f8c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:33.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f93400640 1 -- 172.21.15.66:0/3981921078 shutdown_connections 2024-03-30T00:41:33.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.773+0000 7f0f93400640 1 -- 172.21.15.66:0/3981921078 wait complete. 2024-03-30T00:41:33.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f93400640 1 Processor -- start 2024-03-30T00:41:33.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f93400640 1 -- start start 2024-03-30T00:41:33.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f93400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f8c0af420 0x7f0f8c14e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f93400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f8c0ce740 0x7f0f8c0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f93400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f8c15de30 0x7f0f8c0c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f93400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c14f880 con 0x7f0f8c0af420 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f93400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c14ec00 con 0x7f0f8c15de30 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f93400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0c6820 con 0x7f0f8c0ce740 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f90a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f8c0af420 0x7f0f8c14e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f90a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f8c0af420 0x7f0f8c14e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44188/0 (socket says 172.21.15.66:44188) 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f90a00640 1 -- 172.21.15.66:0/160981272 learned_addr learned my addr 172.21.15.66:0/160981272 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f8be00640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f8c0ce740 0x7f0f8c0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f91400640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f8c15de30 0x7f0f8c0c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f90a00640 1 -- 172.21.15.66:0/160981272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f8c0ce740 msgr2=0x7f0f8c0c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f90a00640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f8c0ce740 0x7f0f8c0c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f90a00640 1 -- 172.21.15.66:0/160981272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f8c15de30 msgr2=0x7f0f8c0c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f90a00640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f8c15de30 0x7f0f8c0c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f90a00640 1 -- 172.21.15.66:0/160981272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80009420 con 0x7f0f8c0af420 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f90a00640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f8c0af420 0x7f0f8c14e6c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000e980 tx=0x7f0f8000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f88c00640 1 -- 172.21.15.66:0/160981272 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8000ca00 con 0x7f0f8c0af420 2024-03-30T00:41:33.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f88c00640 1 -- 172.21.15.66:0/160981272 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8000cb60 con 0x7f0f8c0af420 2024-03-30T00:41:33.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f88c00640 1 -- 172.21.15.66:0/160981272 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80013470 con 0x7f0f8c0af420 2024-03-30T00:41:33.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f93400640 1 -- 172.21.15.66:0/160981272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f780094d0 con 0x7f0f8c0af420 2024-03-30T00:41:33.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f93400640 1 -- 172.21.15.66:0/160981272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c150f50 con 0x7f0f8c0af420 2024-03-30T00:41:33.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.777+0000 7f0f93400640 1 -- 172.21.15.66:0/160981272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c14faf0 con 0x7f0f8c0af420 2024-03-30T00:41:33.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.781+0000 7f0f88c00640 1 -- 172.21.15.66:0/160981272 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0f800135d0 con 0x7f0f8c0af420 2024-03-30T00:41:33.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.781+0000 7f0f88c00640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f6c066690 0x7f0f6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:33.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.781+0000 7f0f88c00640 1 -- 172.21.15.66:0/160981272 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0f800844d0 con 0x7f0f8c0af420 2024-03-30T00:41:33.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.781+0000 7f0f8be00640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f6c066690 0x7f0f6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:33.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.781+0000 7f0f8be00640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f6c066690 0x7f0f6c068b50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f780af000 tx=0x7f0f780b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:33.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.785+0000 7f0f88c00640 1 -- 172.21.15.66:0/160981272 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0f80051780 con 0x7f0f8c0af420 2024-03-30T00:41:33.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.945+0000 7f0f93400640 1 -- 172.21.15.66:0/160981272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f8c0cdf60 con 0x7f0f8c0af420 2024-03-30T00:41:33.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.945+0000 7f0f88c00640 1 -- 172.21.15.66:0/160981272 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f80054da0 con 0x7f0f8c0af420 2024-03-30T00:41:33.955 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:41:33.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 -- 172.21.15.66:0/160981272 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f6c066690 msgr2=0x7f0f6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f6c066690 0x7f0f6c068b50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f780af000 tx=0x7f0f780b6040 comp rx=0 tx=0).stop 2024-03-30T00:41:33.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 -- 172.21.15.66:0/160981272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f8c0af420 msgr2=0x7f0f8c14e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:33.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f8c0af420 0x7f0f8c14e6c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000e980 tx=0x7f0f8000ee40 comp rx=0 tx=0).stop 2024-03-30T00:41:33.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 -- 172.21.15.66:0/160981272 shutdown_connections 2024-03-30T00:41:33.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f8c15de30 0x7f0f8c0c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f8c0ce740 0x7f0f8c0c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f6c066690 0x7f0f6c068b50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 --2- 172.21.15.66:0/160981272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f8c0af420 0x7f0f8c14e6c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:33.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 -- 172.21.15.66:0/160981272 >> 172.21.15.66:0/160981272 conn(0x7f0f8c0d0540 msgr2=0x7f0f8c0ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:33.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 -- 172.21.15.66:0/160981272 shutdown_connections 2024-03-30T00:41:33.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:33.949+0000 7f0f93400640 1 -- 172.21.15.66:0/160981272 wait complete. 2024-03-30T00:41:33.977 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:41:34.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.109+0000 7f0dbe800640 1 Processor -- start 2024-03-30T00:41:34.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.109+0000 7f0dbe800640 1 -- start start 2024-03-30T00:41:34.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0dbe800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0db80cec30 0x7f0db8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0dbe800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0db80cde70 0x7f0db815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0dbe800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db815a6e0 con 0x7f0db80cda90 2024-03-30T00:41:34.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0dbe800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db80b1b80 con 0x7f0db80cec30 2024-03-30T00:41:34.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0dbe800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db80cd3e0 con 0x7f0db80cde70 2024-03-30T00:41:34.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0daea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0db80cec30 0x7f0db8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0daea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0db80cec30 0x7f0db8159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44200/0 (socket says 172.21.15.66:44200) 2024-03-30T00:41:34.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0daea00640 1 -- 172.21.15.66:0/2054040937 learned_addr learned my addr 172.21.15.66:0/2054040937 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:34.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0db7e00640 1 --2- 172.21.15.66:0/2054040937 >> v2:172.21.15.156:3300/0 conn(0x7f0db80cde70 0x7f0db815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0daea00640 1 -- 172.21.15.66:0/2054040937 >> v2:172.21.15.156:3300/0 conn(0x7f0db80cde70 msgr2=0x7f0db815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:34.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0daea00640 1 --2- 172.21.15.66:0/2054040937 >> v2:172.21.15.156:3300/0 conn(0x7f0db80cde70 0x7f0db815a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0daea00640 1 -- 172.21.15.66:0/2054040937 >> v1:172.21.15.156:6789/0 conn(0x7f0db80cda90 legacy=0x7f0db80ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.113+0000 7f0daea00640 1 -- 172.21.15.66:0/2054040937 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db815a990 con 0x7f0db80cec30 2024-03-30T00:41:34.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0daea00640 1 --2- 172.21.15.66:0/2054040937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0db80cec30 0x7f0db8159c60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0db80adcf0 tx=0x7f0da001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b100b7890822eb5a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:34.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0db6a00640 1 -- 172.21.15.66:0/2054040937 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da00a7c30 con 0x7f0db80cec30 2024-03-30T00:41:34.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0db6a00640 1 -- 172.21.15.66:0/2054040937 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da00a7d90 con 0x7f0db80cec30 2024-03-30T00:41:34.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0db6a00640 1 -- 172.21.15.66:0/2054040937 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da00af660 con 0x7f0db80cec30 2024-03-30T00:41:34.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 -- 172.21.15.66:0/2054040937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0db80cec30 msgr2=0x7f0db8159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 --2- 172.21.15.66:0/2054040937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0db80cec30 0x7f0db8159c60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0db80adcf0 tx=0x7f0da001eb60 comp rx=0 tx=0).stop 2024-03-30T00:41:34.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 -- 172.21.15.66:0/2054040937 shutdown_connections 2024-03-30T00:41:34.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 --2- 172.21.15.66:0/2054040937 >> v2:172.21.15.156:3300/0 conn(0x7f0db80cde70 0x7f0db815a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 --2- 172.21.15.66:0/2054040937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0db80cec30 0x7f0db8159c60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 -- 172.21.15.66:0/2054040937 >> 172.21.15.66:0/2054040937 conn(0x7f0db8069f60 msgr2=0x7f0db806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:34.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 -- 172.21.15.66:0/2054040937 shutdown_connections 2024-03-30T00:41:34.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 -- 172.21.15.66:0/2054040937 wait complete. 2024-03-30T00:41:34.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 Processor -- start 2024-03-30T00:41:34.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 -- start start 2024-03-30T00:41:34.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0db80cda90 0x7f0db80978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0db80cde70 0x7f0db81f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0db80cec30 0x7f0db80b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db81f5e60 con 0x7f0db80cda90 2024-03-30T00:41:34.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80c57a0 con 0x7f0db80cde70 2024-03-30T00:41:34.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0dbe800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80c71d0 con 0x7f0db80cec30 2024-03-30T00:41:34.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0db7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0db80cda90 0x7f0db80978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0db7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0db80cda90 0x7f0db80978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44202/0 (socket says 172.21.15.66:44202) 2024-03-30T00:41:34.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.117+0000 7f0db7400640 1 -- 172.21.15.66:0/3108739689 learned_addr learned my addr 172.21.15.66:0/3108739689 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:34.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0daea00640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0db80cde70 0x7f0db81f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0db7e00640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0db80cec30 0x7f0db80b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0db7400640 1 -- 172.21.15.66:0/3108739689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0db80cec30 msgr2=0x7f0db80b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0db7400640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0db80cec30 0x7f0db80b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0db7400640 1 -- 172.21.15.66:0/3108739689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0db80cde70 msgr2=0x7f0db81f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0db7400640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0db80cde70 0x7f0db81f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0db7400640 1 -- 172.21.15.66:0/3108739689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da000b770 con 0x7f0db80cda90 2024-03-30T00:41:34.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0db7e00640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0db80cec30 0x7f0db80b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:34.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0db7400640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0db80cda90 0x7f0db80978d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0da800c7f0 tx=0x7f0da800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:34.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0dafe00640 1 -- 172.21.15.66:0/3108739689 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8003b80 con 0x7f0db80cda90 2024-03-30T00:41:34.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0dafe00640 1 -- 172.21.15.66:0/3108739689 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da8003ce0 con 0x7f0db80cda90 2024-03-30T00:41:34.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0dafe00640 1 -- 172.21.15.66:0/3108739689 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da800aa90 con 0x7f0db80cda90 2024-03-30T00:41:34.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0dbe800640 1 -- 172.21.15.66:0/3108739689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db80c74b0 con 0x7f0db80cda90 2024-03-30T00:41:34.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0dbe800640 1 -- 172.21.15.66:0/3108739689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db8097e10 con 0x7f0db80cda90 2024-03-30T00:41:34.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0dbe800640 1 -- 172.21.15.66:0/3108739689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db80c7030 con 0x7f0db80cda90 2024-03-30T00:41:34.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0dafe00640 1 -- 172.21.15.66:0/3108739689 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0da800abf0 con 0x7f0db80cda90 2024-03-30T00:41:34.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.121+0000 7f0dafe00640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0d94066690 0x7f0d94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.125+0000 7f0dafe00640 1 -- 172.21.15.66:0/3108739689 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0da8084150 con 0x7f0db80cda90 2024-03-30T00:41:34.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.125+0000 7f0daea00640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0d94066690 0x7f0d94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.125+0000 7f0daea00640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0d94066690 0x7f0d94068b50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0da00b1000 tx=0x7f0da0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:34.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.125+0000 7f0dafe00640 1 -- 172.21.15.66:0/3108739689 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0db80c7030 con 0x7f0db80cda90 2024-03-30T00:41:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.285+0000 7f0dbe800640 1 -- 172.21.15.66:0/3108739689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0db80c7030 con 0x7f0db80cda90 2024-03-30T00:41:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.285+0000 7f0dafe00640 1 -- 172.21.15.66:0/3108739689 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0db80c7030 con 0x7f0db80cda90 2024-03-30T00:41:34.297 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:41:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 -- 172.21.15.66:0/3108739689 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0d94066690 msgr2=0x7f0d94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0d94066690 0x7f0d94068b50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0da00b1000 tx=0x7f0da0008040 comp rx=0 tx=0).stop 2024-03-30T00:41:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 -- 172.21.15.66:0/3108739689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0db80cda90 msgr2=0x7f0db80978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0db80cda90 0x7f0db80978d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0da800c7f0 tx=0x7f0da800ccb0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 -- 172.21.15.66:0/3108739689 shutdown_connections 2024-03-30T00:41:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0db80cec30 0x7f0db80b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0db80cde70 0x7f0db81f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0d94066690 0x7f0d94068b50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 --2- 172.21.15.66:0/3108739689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0db80cda90 0x7f0db80978d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 -- 172.21.15.66:0/3108739689 >> 172.21.15.66:0/3108739689 conn(0x7f0db8069f60 msgr2=0x7f0db815b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 -- 172.21.15.66:0/3108739689 shutdown_connections 2024-03-30T00:41:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.289+0000 7f0dbe800640 1 -- 172.21.15.66:0/3108739689 wait complete. 2024-03-30T00:41:34.318 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:41:34.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.449+0000 7f0d06400640 1 Processor -- start 2024-03-30T00:41:34.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.449+0000 7f0d06400640 1 -- start start 2024-03-30T00:41:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.449+0000 7f0d06400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d000cda90 0x7f0d000ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.449+0000 7f0d06400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000cd3e0 con 0x7f0d0015a370 2024-03-30T00:41:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.449+0000 7f0d06400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0015a990 con 0x7f0d000cde70 2024-03-30T00:41:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.449+0000 7f0d06400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000b1b80 con 0x7f0d000cda90 2024-03-30T00:41:34.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cffe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0d0015a370 0x7f0d00156910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55328/0 (socket says 172.21.15.66:55328) 2024-03-30T00:41:34.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cffe00640 1 -- 172.21.15.66:0/31328449 learned_addr learned my addr 172.21.15.66:0/31328449 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cff400640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d000cda90 0x7f0d000ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455356165 0 0) 0x7f0d000cd3e0 con 0x7f0d0015a370 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ce4003530 con 0x7f0d0015a370 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679101014 0 0) 0x7f0d0015a990 con 0x7f0d000cde70 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d000cd3e0 con 0x7f0d000cde70 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1059009766 0 0) 0x7f0ce4003530 con 0x7f0d0015a370 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d000cda90 msgr2=0x7f0d000ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d000cda90 0x7f0d000ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 >> v1:172.21.15.156:6789/0 conn(0x7f0d000cde70 legacy=0x7f0d00159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d000b11e0 con 0x7f0d0015a370 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f0cf8001a00 con 0x7f0d0015a370 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 >> v1:172.21.15.66:6789/0 conn(0x7f0d0015a370 legacy=0x7f0d00156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ce4003b40 0x7f0ce4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 0x7f0ce4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ce4005190 0x7f0ce400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000b1b80 con 0x7f0ce40047d0 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce400c060 con 0x7f0ce4005190 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce400c220 con 0x7f0ce4003b40 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cff400640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d000cda90 0x7f0d000ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cf6a00640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 0x7f0ce4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cffe00640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ce4005190 0x7f0ce400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cff400640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ce4003b40 0x7f0ce4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cf6a00640 1 -- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ce4003b40 msgr2=0x7f0ce4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cf6a00640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ce4003b40 0x7f0ce4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cf6a00640 1 -- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ce4005190 msgr2=0x7f0ce400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cf6a00640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ce4005190 0x7f0ce400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.453+0000 7f0cf6a00640 1 -- 172.21.15.66:0/31328449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4003530 con 0x7f0ce40047d0 2024-03-30T00:41:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0cf6a00640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 0x7f0ce4004c50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80018c0 tx=0x7f0ce801c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c791635715e8d4b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80050d0 con 0x7f0ce40047d0 2024-03-30T00:41:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce8005230 con 0x7f0ce40047d0 2024-03-30T00:41:34.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0cfea00640 1 -- 172.21.15.66:0/31328449 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8003600 con 0x7f0ce40047d0 2024-03-30T00:41:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 -- 172.21.15.66:0/31328449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 msgr2=0x7f0ce4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 --2- 172.21.15.66:0/31328449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 0x7f0ce4004c50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80018c0 tx=0x7f0ce801c580 comp rx=0 tx=0).stop 2024-03-30T00:41:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0cffe00640 1 -- 172.21.15.66:0/31328449 reap_dead start 2024-03-30T00:41:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 -- 172.21.15.66:0/31328449 shutdown_connections 2024-03-30T00:41:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 -- 172.21.15.66:0/31328449 >> 172.21.15.66:0/31328449 conn(0x7f0d00069f60 msgr2=0x7f0d0006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 -- 172.21.15.66:0/31328449 shutdown_connections 2024-03-30T00:41:34.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 -- 172.21.15.66:0/31328449 wait complete. 2024-03-30T00:41:34.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 Processor -- start 2024-03-30T00:41:34.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 -- start start 2024-03-30T00:41:34.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 0x7f0d000afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d00158970 0x7f0d000ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d000c5b60 0x7f0d000c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000c6520 con 0x7f0ce40047d0 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.457+0000 7f0d06400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000b01c0 con 0x7f0d000c5b60 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0d06400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001597d0 con 0x7f0d00158970 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cff400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 0x7f0d000afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cff400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 0x7f0d000afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44220/0 (socket says 172.21.15.66:44220) 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cff400640 1 -- 172.21.15.66:0/96444232 learned_addr learned my addr 172.21.15.66:0/96444232 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cffe00640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d000c5b60 0x7f0d000c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cf6a00640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d00158970 0x7f0d000ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cff400640 1 -- 172.21.15.66:0/96444232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d00158970 msgr2=0x7f0d000ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cff400640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d00158970 0x7f0d000ac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cff400640 1 -- 172.21.15.66:0/96444232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d000c5b60 msgr2=0x7f0d000c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cff400640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d000c5b60 0x7f0d000c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cff400640 1 -- 172.21.15.66:0/96444232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d00155e80 con 0x7f0ce40047d0 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cffe00640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d000c5b60 0x7f0d000c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:34.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cff400640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 0x7f0d000afc80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00119d0 tx=0x7f0cf0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:34.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cf7e00640 1 -- 172.21.15.66:0/96444232 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf000fc10 con 0x7f0ce40047d0 2024-03-30T00:41:34.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cf7e00640 1 -- 172.21.15.66:0/96444232 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf000fd70 con 0x7f0ce40047d0 2024-03-30T00:41:34.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cf7e00640 1 -- 172.21.15.66:0/96444232 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf000fee0 con 0x7f0ce40047d0 2024-03-30T00:41:34.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0d06400640 1 -- 172.21.15.66:0/96444232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d00158f30 con 0x7f0ce40047d0 2024-03-30T00:41:34.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0d06400640 1 -- 172.21.15.66:0/96444232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d001594f0 con 0x7f0ce40047d0 2024-03-30T00:41:34.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0d06400640 1 -- 172.21.15.66:0/96444232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d00066c30 con 0x7f0ce40047d0 2024-03-30T00:41:34.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.461+0000 7f0cf7e00640 1 -- 172.21.15.66:0/96444232 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0cf0014640 con 0x7f0ce40047d0 2024-03-30T00:41:34.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.465+0000 7f0cf7e00640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0cdc0666c0 0x7f0cdc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:34.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.465+0000 7f0cf7e00640 1 -- 172.21.15.66:0/96444232 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0cf0084be0 con 0x7f0ce40047d0 2024-03-30T00:41:34.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.465+0000 7f0cf6a00640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0cdc0666c0 0x7f0cdc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:34.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.465+0000 7f0cf6a00640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0cdc0666c0 0x7f0cdc068b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80018c0 tx=0x7f0ce8001350 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:34.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.469+0000 7f0cf7e00640 1 -- 172.21.15.66:0/96444232 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0cf0051de0 con 0x7f0ce40047d0 2024-03-30T00:41:34.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.637+0000 7f0d06400640 1 -- 172.21.15.66:0/96444232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0d000b3170 con 0x7f0ce40047d0 2024-03-30T00:41:34.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0cf7e00640 1 -- 172.21.15.66:0/96444232 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0cf0022020 con 0x7f0ce40047d0 2024-03-30T00:41:34.649 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:41:34.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 -- 172.21.15.66:0/96444232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0cdc0666c0 msgr2=0x7f0cdc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0cdc0666c0 0x7f0cdc068b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80018c0 tx=0x7f0ce8001350 comp rx=0 tx=0).stop 2024-03-30T00:41:34.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 -- 172.21.15.66:0/96444232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 msgr2=0x7f0d000afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:34.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 0x7f0d000afc80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00119d0 tx=0x7f0cf0011e90 comp rx=0 tx=0).stop 2024-03-30T00:41:34.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 -- 172.21.15.66:0/96444232 shutdown_connections 2024-03-30T00:41:34.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d000c5b60 0x7f0d000c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d00158970 0x7f0d000ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0cdc0666c0 0x7f0cdc068b80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 --2- 172.21.15.66:0/96444232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce40047d0 0x7f0d000afc80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 -- 172.21.15.66:0/96444232 >> 172.21.15.66:0/96444232 conn(0x7f0d00069f60 msgr2=0x7f0d000c9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 -- 172.21.15.66:0/96444232 shutdown_connections 2024-03-30T00:41:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:34.641+0000 7f0d06400640 1 -- 172.21.15.66:0/96444232 wait complete. 2024-03-30T00:41:34.670 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:41:34.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:34.670 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:41:34.688 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[39082]: fuse finished with error 0 and tester_r 0 2024-03-30T00:41:34.691 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:41:40.793 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:41:40.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:40.793 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:41:40.804 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:41.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.993+0000 7f059d200640 1 Processor -- start 2024-03-30T00:41:41.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.993+0000 7f059d200640 1 -- start start 2024-03-30T00:41:41.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.993+0000 7f059d200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f05980ae4e0 0x7f05980cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.993+0000 7f059d200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059806e7d0 con 0x7f059815ca10 2024-03-30T00:41:41.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.993+0000 7f059d200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059815cdf0 con 0x7f05980ce820 2024-03-30T00:41:41.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.993+0000 7f059d200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980b1f70 con 0x7f05980ae4e0 2024-03-30T00:41:41.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.993+0000 7f0595e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f059815ca10 0x7f059806ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:38098/0 (socket says 172.21.15.66:38098) 2024-03-30T00:41:41.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.993+0000 7f0595e00640 1 -- 172.21.15.66:0/662726109 learned_addr learned my addr 172.21.15.66:0/662726109 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:41.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f058d400640 1 --2- 172.21.15.66:0/662726109 >> v2:172.21.15.156:3300/0 conn(0x7f05980ae4e0 0x7f05980cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454917904 0 0) 0x7f059806e7d0 con 0x7f059815ca10 2024-03-30T00:41:41.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f057c003530 con 0x7f059815ca10 2024-03-30T00:41:41.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056169547 0 0) 0x7f059815cdf0 con 0x7f05980ce820 2024-03-30T00:41:41.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f059806e7d0 con 0x7f05980ce820 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3416762737 0 0) 0x7f057c003530 con 0x7f059815ca10 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 >> v2:172.21.15.156:3300/0 conn(0x7f05980ae4e0 msgr2=0x7f05980cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 --2- 172.21.15.66:0/662726109 >> v2:172.21.15.156:3300/0 conn(0x7f05980ae4e0 0x7f05980cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 >> v1:172.21.15.156:6789/0 conn(0x7f05980ce820 legacy=0x7f0598160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980b16d0 con 0x7f059815ca10 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f0588001a00 con 0x7f059815ca10 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 >> v1:172.21.15.66:6789/0 conn(0x7f059815ca10 legacy=0x7f059806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 --2- 172.21.15.66:0/662726109 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f057c003b10 0x7f057c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 --2- 172.21.15.66:0/662726109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 0x7f057c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 --2- 172.21.15.66:0/662726109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f057c005190 0x7f057c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05980b1f70 con 0x7f057c0047d0 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c00c010 con 0x7f057c005190 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057c00c1d0 con 0x7f057c003b10 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f058d400640 1 --2- 172.21.15.66:0/662726109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 0x7f057c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0596800640 1 --2- 172.21.15.66:0/662726109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f057c005190 0x7f057c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0595e00640 1 --2- 172.21.15.66:0/662726109 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f057c003b10 0x7f057c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f058d400640 1 -- 172.21.15.66:0/662726109 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f057c003b10 msgr2=0x7f057c003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f058d400640 1 --2- 172.21.15.66:0/662726109 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f057c003b10 0x7f057c003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f058d400640 1 -- 172.21.15.66:0/662726109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f057c005190 msgr2=0x7f057c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f058d400640 1 --2- 172.21.15.66:0/662726109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f057c005190 0x7f057c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f0596800640 1 -- 172.21.15.66:0/662726109 reap_dead start 2024-03-30T00:41:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:40.997+0000 7f058d400640 1 -- 172.21.15.66:0/662726109 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c003530 con 0x7f057c0047d0 2024-03-30T00:41:41.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f058d400640 1 --2- 172.21.15.66:0/662726109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 0x7f057c004c50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0580009ae0 tx=0x7f058001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fb6d6411aa2b2a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:41.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05800a5660 con 0x7f057c0047d0 2024-03-30T00:41:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05800a57c0 con 0x7f057c0047d0 2024-03-30T00:41:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f0595400640 1 -- 172.21.15.66:0/662726109 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05800a5b70 con 0x7f057c0047d0 2024-03-30T00:41:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 -- 172.21.15.66:0/662726109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 msgr2=0x7f057c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 --2- 172.21.15.66:0/662726109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 0x7f057c004c50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0580009ae0 tx=0x7f058001ca70 comp rx=0 tx=0).stop 2024-03-30T00:41:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 -- 172.21.15.66:0/662726109 shutdown_connections 2024-03-30T00:41:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 --2- 172.21.15.66:0/662726109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 0x7f057c004c50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 -- 172.21.15.66:0/662726109 >> 172.21.15.66:0/662726109 conn(0x7f0598098a20 msgr2=0x7f05980b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 -- 172.21.15.66:0/662726109 shutdown_connections 2024-03-30T00:41:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 -- 172.21.15.66:0/662726109 wait complete. 2024-03-30T00:41:41.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 Processor -- start 2024-03-30T00:41:41.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 -- start start 2024-03-30T00:41:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 0x7f0598153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f059815f840 0x7f0598158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0598159270 0x7f05980accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05981599d0 con 0x7f057c0047d0 2024-03-30T00:41:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0598157f20 con 0x7f059815f840 2024-03-30T00:41:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.001+0000 7f059d200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05981596f0 con 0x7f0598159270 2024-03-30T00:41:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f0595e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 0x7f0598153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f0595e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 0x7f0598153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34066/0 (socket says 172.21.15.66:34066) 2024-03-30T00:41:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f0595e00640 1 -- 172.21.15.66:0/2829433854 learned_addr learned my addr 172.21.15.66:0/2829433854 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f058d400640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f059815f840 0x7f0598158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f0596800640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0598159270 0x7f05980accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f0595e00640 1 -- 172.21.15.66:0/2829433854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0598159270 msgr2=0x7f05980accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f0595e00640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0598159270 0x7f05980accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f0595e00640 1 -- 172.21.15.66:0/2829433854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f059815f840 msgr2=0x7f0598158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f0595e00640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f059815f840 0x7f0598158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f0595e00640 1 -- 172.21.15.66:0/2829433854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598157860 con 0x7f057c0047d0 2024-03-30T00:41:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f0595e00640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 0x7f0598153980 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0588007a50 tx=0x7f058800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:41.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f058ea00640 1 -- 172.21.15.66:0/2829433854 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588007a80 con 0x7f057c0047d0 2024-03-30T00:41:41.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f058ea00640 1 -- 172.21.15.66:0/2829433854 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05880133f0 con 0x7f057c0047d0 2024-03-30T00:41:41.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f058ea00640 1 -- 172.21.15.66:0/2829433854 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05880137a0 con 0x7f057c0047d0 2024-03-30T00:41:41.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f059d200640 1 -- 172.21.15.66:0/2829433854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0598157b40 con 0x7f057c0047d0 2024-03-30T00:41:41.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f059d200640 1 -- 172.21.15.66:0/2829433854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0598158340 con 0x7f057c0047d0 2024-03-30T00:41:41.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f059d200640 1 -- 172.21.15.66:0/2829433854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05981576c0 con 0x7f057c0047d0 2024-03-30T00:41:41.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f058ea00640 1 -- 172.21.15.66:0/2829433854 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0588013900 con 0x7f057c0047d0 2024-03-30T00:41:41.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.005+0000 7f058ea00640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0574066690 0x7f0574068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.009+0000 7f058ea00640 1 -- 172.21.15.66:0/2829433854 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0588084e40 con 0x7f057c0047d0 2024-03-30T00:41:41.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.009+0000 7f058d400640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0574066690 0x7f0574068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.009+0000 7f058d400640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0574066690 0x7f0574068b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f05800af000 tx=0x7f05800046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.013+0000 7f058ea00640 1 -- 172.21.15.66:0/2829433854 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f05981576c0 con 0x7f057c0047d0 2024-03-30T00:41:41.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 -- 172.21.15.66:0/2829433854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05981576c0 con 0x7f057c0047d0 2024-03-30T00:41:41.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f058ea00640 1 -- 172.21.15.66:0/2829433854 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f05981576c0 con 0x7f057c0047d0 2024-03-30T00:41:41.218 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:41.218 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3895007156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3895007156},{"type":"v1","addr":"172.21.15.156:6835","nonce":3895007156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/65993387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":65993387},{"type":"v1","addr":"172.21.15.156:6834","nonce":65993387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2013605775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2013605775},{"type":"v1","addr":"172.21.15.66:6835","nonce":2013605775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:41:21.269965+0000","modified":"2024-03-30T00:41:24.283171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1694329026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1694329026},{"type":"v1","addr":"172.21.15.66:6837","nonce":1694329026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-30T00:41:41.219 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-03-30T00:41:41.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 -- 172.21.15.66:0/2829433854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0574066690 msgr2=0x7f0574068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0574066690 0x7f0574068b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f05800af000 tx=0x7f05800046d0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 -- 172.21.15.66:0/2829433854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 msgr2=0x7f0598153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 0x7f0598153980 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0588007a50 tx=0x7f058800a660 comp rx=0 tx=0).stop 2024-03-30T00:41:41.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 -- 172.21.15.66:0/2829433854 shutdown_connections 2024-03-30T00:41:41.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0598159270 0x7f05980accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f059815f840 0x7f0598158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0574066690 0x7f0574068b50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 --2- 172.21.15.66:0/2829433854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f057c0047d0 0x7f0598153980 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 -- 172.21.15.66:0/2829433854 >> 172.21.15.66:0/2829433854 conn(0x7f0598098a20 msgr2=0x7f059815bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:41.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 -- 172.21.15.66:0/2829433854 shutdown_connections 2024-03-30T00:41:41.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.209+0000 7f059d200640 1 -- 172.21.15.66:0/2829433854 wait complete. 2024-03-30T00:41:41.240 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-30T00:41:41.408 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-03-30T00:41:41.409 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-30T00:41:41.409 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:41.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8868400640 1 Processor -- start 2024-03-30T00:41:41.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8868400640 1 -- start start 2024-03-30T00:41:41.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8868400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f88600ce520 0x7f88600ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8868400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f88600cabb0 0x7f88600cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8868400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886006d070 con 0x7f886015a640 2024-03-30T00:41:41.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8868400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88600cbba0 con 0x7f88600cabb0 2024-03-30T00:41:41.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8868400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88600cee40 con 0x7f88600ce520 2024-03-30T00:41:41.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8865a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f886015a640 0x7f886006dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:38104/0 (socket says 172.21.15.66:38104) 2024-03-30T00:41:41.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8865a00640 1 -- 172.21.15.66:0/1077448352 learned_addr learned my addr 172.21.15.66:0/1077448352 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:41.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8866400640 1 --2- 172.21.15.66:0/1077448352 >> v2:172.21.15.66:3300/0 conn(0x7f88600cabb0 0x7f88600cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f885fe00640 1 -- 172.21.15.66:0/1077448352 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145858736 0 0) 0x7f886006d070 con 0x7f886015a640 2024-03-30T00:41:41.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f885fe00640 1 -- 172.21.15.66:0/1077448352 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8848003530 con 0x7f886015a640 2024-03-30T00:41:41.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8866400640 1 -- 172.21.15.66:0/1077448352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f88600ce520 msgr2=0x7f88600ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:41.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8866400640 1 --2- 172.21.15.66:0/1077448352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f88600ce520 0x7f88600ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8866400640 1 -- 172.21.15.66:0/1077448352 >> v1:172.21.15.66:6789/0 conn(0x7f886015a640 legacy=0x7f886006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.541+0000 7f8866400640 1 -- 172.21.15.66:0/1077448352 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88600cb8c0 con 0x7f88600cabb0 2024-03-30T00:41:41.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f8866400640 1 --2- 172.21.15.66:0/1077448352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88600cabb0 0x7f88600cb000 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8858009d90 tx=0x7f885801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39d048dc4bac07a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:41.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f885fe00640 1 -- 172.21.15.66:0/1077448352 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88580a5a00 con 0x7f88600cabb0 2024-03-30T00:41:41.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f885fe00640 1 -- 172.21.15.66:0/1077448352 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88580a5b60 con 0x7f88600cabb0 2024-03-30T00:41:41.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f885fe00640 1 -- 172.21.15.66:0/1077448352 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88580ad440 con 0x7f88600cabb0 2024-03-30T00:41:41.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f8868400640 1 -- 172.21.15.66:0/1077448352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88600cabb0 msgr2=0x7f88600cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f8868400640 1 --2- 172.21.15.66:0/1077448352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88600cabb0 0x7f88600cb000 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8858009d90 tx=0x7f885801c940 comp rx=0 tx=0).stop 2024-03-30T00:41:41.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f8868400640 1 -- 172.21.15.66:0/1077448352 shutdown_connections 2024-03-30T00:41:41.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f8868400640 1 --2- 172.21.15.66:0/1077448352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88600cabb0 0x7f88600cb000 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f8868400640 1 --2- 172.21.15.66:0/1077448352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f88600ce520 0x7f88600ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f8868400640 1 -- 172.21.15.66:0/1077448352 >> 172.21.15.66:0/1077448352 conn(0x7f8860069f60 msgr2=0x7f886006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:41.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f8868400640 1 -- 172.21.15.66:0/1077448352 shutdown_connections 2024-03-30T00:41:41.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f8868400640 1 -- 172.21.15.66:0/1077448352 wait complete. 2024-03-30T00:41:41.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f8868400640 1 Processor -- start 2024-03-30T00:41:41.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.545+0000 7f8868400640 1 -- start start 2024-03-30T00:41:41.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8868400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88600cabb0 0x7f8860164090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8868400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88600ce520 0x7f8860156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8868400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f886015a640 0x7f88600af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8868400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88600c53c0 con 0x7f88600ce520 2024-03-30T00:41:41.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8868400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8860162f20 con 0x7f88600cabb0 2024-03-30T00:41:41.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8868400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8860157090 con 0x7f886015a640 2024-03-30T00:41:41.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8865000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88600ce520 0x7f8860156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8865000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88600ce520 0x7f8860156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34084/0 (socket says 172.21.15.66:34084) 2024-03-30T00:41:41.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8865000640 1 -- 172.21.15.66:0/1300117425 learned_addr learned my addr 172.21.15.66:0/1300117425 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:41.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8866400640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f886015a640 0x7f88600af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8865a00640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88600cabb0 0x7f8860164090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8865000640 1 -- 172.21.15.66:0/1300117425 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f886015a640 msgr2=0x7f88600af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8865000640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f886015a640 0x7f88600af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8865000640 1 -- 172.21.15.66:0/1300117425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88600cabb0 msgr2=0x7f8860164090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8865000640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88600cabb0 0x7f8860164090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8865000640 1 -- 172.21.15.66:0/1300117425 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8858009a70 con 0x7f88600ce520 2024-03-30T00:41:41.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8865000640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88600ce520 0x7f8860156b50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f884c00b550 tx=0x7f884c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:41.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f885d600640 1 -- 172.21.15.66:0/1300117425 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c010870 con 0x7f88600ce520 2024-03-30T00:41:41.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f885d600640 1 -- 172.21.15.66:0/1300117425 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c0109f0 con 0x7f88600ce520 2024-03-30T00:41:41.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8868400640 1 -- 172.21.15.66:0/1300117425 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88600cbec0 con 0x7f88600ce520 2024-03-30T00:41:41.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f885d600640 1 -- 172.21.15.66:0/1300117425 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c010da0 con 0x7f88600ce520 2024-03-30T00:41:41.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8868400640 1 -- 172.21.15.66:0/1300117425 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8860163450 con 0x7f88600ce520 2024-03-30T00:41:41.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.549+0000 7f8868400640 1 -- 172.21.15.66:0/1300117425 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8860072060 con 0x7f88600ce520 2024-03-30T00:41:41.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.553+0000 7f885d600640 1 -- 172.21.15.66:0/1300117425 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f884c00f3c0 con 0x7f88600ce520 2024-03-30T00:41:41.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.553+0000 7f885d600640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8840066690 0x7f8840068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:41.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.553+0000 7f885d600640 1 -- 172.21.15.66:0/1300117425 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f884c0844c0 con 0x7f88600ce520 2024-03-30T00:41:41.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.553+0000 7f8865a00640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8840066690 0x7f8840068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:41.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.553+0000 7f8865a00640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8840066690 0x7f8840068b50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8854005890 tx=0x7f8854009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:41.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.557+0000 7f885d600640 1 -- 172.21.15.66:0/1300117425 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f884c0515a0 con 0x7f88600ce520 2024-03-30T00:41:41.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 -- 172.21.15.66:0/1300117425 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8860066c30 con 0x7f88600ce520 2024-03-30T00:41:41.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f885d600640 1 -- 172.21.15.66:0/1300117425 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f884c054bc0 con 0x7f88600ce520 2024-03-30T00:41:41.771 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:41.771 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3895007156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3895007156},{"type":"v1","addr":"172.21.15.156:6835","nonce":3895007156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/65993387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":65993387},{"type":"v1","addr":"172.21.15.156:6834","nonce":65993387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2013605775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2013605775},{"type":"v1","addr":"172.21.15.66:6835","nonce":2013605775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:41:21.269965+0000","modified":"2024-03-30T00:41:24.283171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1694329026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1694329026},{"type":"v1","addr":"172.21.15.66:6837","nonce":1694329026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-30T00:41:41.771 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-03-30T00:41:41.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 -- 172.21.15.66:0/1300117425 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8840066690 msgr2=0x7f8840068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8840066690 0x7f8840068b50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8854005890 tx=0x7f8854009040 comp rx=0 tx=0).stop 2024-03-30T00:41:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 -- 172.21.15.66:0/1300117425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88600ce520 msgr2=0x7f8860156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88600ce520 0x7f8860156b50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f884c00b550 tx=0x7f884c00ba10 comp rx=0 tx=0).stop 2024-03-30T00:41:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 -- 172.21.15.66:0/1300117425 shutdown_connections 2024-03-30T00:41:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f886015a640 0x7f88600af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8840066690 0x7f8840068b50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88600ce520 0x7f8860156b50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 --2- 172.21.15.66:0/1300117425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88600cabb0 0x7f8860164090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 -- 172.21.15.66:0/1300117425 >> 172.21.15.66:0/1300117425 conn(0x7f8860069f60 msgr2=0x7f88600afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.761+0000 7f8868400640 1 -- 172.21.15.66:0/1300117425 shutdown_connections 2024-03-30T00:41:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:41.765+0000 7f8868400640 1 -- 172.21.15.66:0/1300117425 wait complete. 2024-03-30T00:41:41.792 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-03-30T00:41:41.953 INFO:teuthology.orchestra.run.smithi066.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-03-30T00:41:41.953 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-03-30T00:41:46.082 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:41:46.086 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:41:51.954 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-03-30T00:41:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff99511a080 1 Processor -- start 2024-03-30T00:41:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff99511a080 1 -- start start 2024-03-30T00:41:52.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff99511a080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562769d19b40 0x562769d345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff99511a080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa090 0x562769d2c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff99511a080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562769daa470 0x562769d2c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff99511a080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562769c971e0 con 0x562769daa090 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff99511a080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562769c32960 con 0x562769d19b40 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff99511a080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562769c32600 con 0x562769daa470 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff991a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa090 0x562769d2c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff991a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa090 0x562769d2c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46374/0 (socket says 172.21.15.66:46374) 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff991a00640 1 -- 172.21.15.66:0/3080034112 learned_addr learned my addr 172.21.15.66:0/3080034112 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff992e00640 1 --2- 172.21.15.66:0/3080034112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562769daa470 0x562769d2c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:51.997+0000 7ff992400640 1 --2- 172.21.15.66:0/3080034112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562769d19b40 0x562769d345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff991a00640 1 -- 172.21.15.66:0/3080034112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562769daa470 msgr2=0x562769d2c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff991a00640 1 --2- 172.21.15.66:0/3080034112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562769daa470 0x562769d2c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff991a00640 1 -- 172.21.15.66:0/3080034112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562769d19b40 msgr2=0x562769d345f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff991a00640 1 --2- 172.21.15.66:0/3080034112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562769d19b40 0x562769d345f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff991a00640 1 -- 172.21.15.66:0/3080034112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff984005d30 con 0x562769daa090 2024-03-30T00:41:52.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff992e00640 1 --2- 172.21.15.66:0/3080034112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562769daa470 0x562769d2c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff991a00640 1 --2- 172.21.15.66:0/3080034112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa090 0x562769d2c3e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff98000b280 tx=0x7ff98000b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713bfad1ecb1b319 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff98be00640 1 -- 172.21.15.66:0/3080034112 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980008070 con 0x562769daa090 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff98be00640 1 -- 172.21.15.66:0/3080034112 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98000d4c0 con 0x562769daa090 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff98be00640 1 -- 172.21.15.66:0/3080034112 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98000d690 con 0x562769daa090 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 -- 172.21.15.66:0/3080034112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa090 msgr2=0x562769d2c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 --2- 172.21.15.66:0/3080034112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa090 0x562769d2c3e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff98000b280 tx=0x7ff98000b740 comp rx=0 tx=0).stop 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 -- 172.21.15.66:0/3080034112 shutdown_connections 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 --2- 172.21.15.66:0/3080034112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562769daa470 0x562769d2c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 --2- 172.21.15.66:0/3080034112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa090 0x562769d2c3e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 --2- 172.21.15.66:0/3080034112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562769d19b40 0x562769d345f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 -- 172.21.15.66:0/3080034112 >> 172.21.15.66:0/3080034112 conn(0x562769d15dd0 msgr2=0x562769cee0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 -- 172.21.15.66:0/3080034112 shutdown_connections 2024-03-30T00:41:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 -- 172.21.15.66:0/3080034112 wait complete. 2024-03-30T00:41:52.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 Processor -- start 2024-03-30T00:41:52.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 -- start start 2024-03-30T00:41:52.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562769d19b40 0x562769d307b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562769daa090 0x562769d142d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa470 0x562769d14810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562769c29690 con 0x562769daa470 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562769d14d50 con 0x562769daa090 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff99511a080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562769d92b50 con 0x562769d19b40 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff992e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa470 0x562769d14810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff992e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa470 0x562769d14810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46378/0 (socket says 172.21.15.66:46378) 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff992e00640 1 -- 172.21.15.66:0/3177777378 learned_addr learned my addr 172.21.15.66:0/3177777378 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff992400640 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562769d19b40 0x562769d307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.001+0000 7ff991a00640 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562769daa090 0x562769d142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff992e00640 1 -- 172.21.15.66:0/3177777378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562769d19b40 msgr2=0x562769d307b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff992e00640 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562769d19b40 0x562769d307b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff992e00640 1 -- 172.21.15.66:0/3177777378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562769daa090 msgr2=0x562769d142d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff992e00640 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562769daa090 0x562769d142d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff992e00640 1 -- 172.21.15.66:0/3177777378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff980001b80 con 0x562769daa470 2024-03-30T00:41:52.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff992e00640 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa470 0x562769d14810 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d210 tx=0x7ff98c00d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff98aa00640 1 -- 172.21.15.66:0/3177777378 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00dea0 con 0x562769daa470 2024-03-30T00:41:52.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff98aa00640 1 -- 172.21.15.66:0/3177777378 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c0062e0 con 0x562769daa470 2024-03-30T00:41:52.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff99511a080 1 -- 172.21.15.66:0/3177777378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562769c34d00 con 0x562769daa470 2024-03-30T00:41:52.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff98aa00640 1 -- 172.21.15.66:0/3177777378 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c004730 con 0x562769daa470 2024-03-30T00:41:52.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff99511a080 1 -- 172.21.15.66:0/3177777378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562769c34760 con 0x562769daa470 2024-03-30T00:41:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff98aa00640 1 -- 172.21.15.66:0/3177777378 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff98c004890 con 0x562769daa470 2024-03-30T00:41:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff98aa00640 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9740650a0 0x7ff974067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff98aa00640 1 -- 172.21.15.66:0/3177777378 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff98c0828a0 con 0x562769daa470 2024-03-30T00:41:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.005+0000 7ff992400640 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9740650a0 0x7ff974067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562769d30cf0 0x562769d95310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 -- 172.21.15.66:0/3177777378 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x562769d93a70 con 0x562769d30cf0 2024-03-30T00:41:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff991a00640 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562769d30cf0 0x562769d95310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff992400640 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9740650a0 0x7ff974067560 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff9840ae000 tx=0x7ff9840a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff991a00640 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562769d30cf0 0x562769d95310 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff991a00640 1 -- 172.21.15.66:0/3177777378 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7ff980018040 con 0x562769d30cf0 2024-03-30T00:41:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 -- 172.21.15.66:0/3177777378 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562769d30cf0 msgr2=0x562769d95310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562769d30cf0 0x562769d95310 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 -- 172.21.15.66:0/3177777378 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9740650a0 msgr2=0x7ff974067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9740650a0 0x7ff974067560 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff9840ae000 tx=0x7ff9840a6000 comp rx=0 tx=0).stop 2024-03-30T00:41:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 -- 172.21.15.66:0/3177777378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa470 msgr2=0x562769d14810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 --2- 172.21.15.66:0/3177777378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562769daa470 0x562769d14810 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d210 tx=0x7ff98c00d6d0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff992e00640 1 -- 172.21.15.66:0/3177777378 reap_dead start 2024-03-30T00:41:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 -- 172.21.15.66:0/3177777378 shutdown_connections 2024-03-30T00:41:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 -- 172.21.15.66:0/3177777378 >> 172.21.15.66:0/3177777378 conn(0x562769d15dd0 msgr2=0x562769d18920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 -- 172.21.15.66:0/3177777378 shutdown_connections 2024-03-30T00:41:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.009+0000 7ff99511a080 1 -- 172.21.15.66:0/3177777378 wait complete. 2024-03-30T00:41:52.023 INFO:teuthology.orchestra.run.smithi066.stdout:mytag 2024-03-30T00:41:52.024 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-03-30T00:41:52.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.061+0000 7ff59f50a080 1 Processor -- start 2024-03-30T00:41:52.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.061+0000 7ff59f50a080 1 -- start start 2024-03-30T00:41:52.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56519066d2b0 0x56519070bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565190687910 0x56519066c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565190687cf0 0x56519066c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x565190581960 con 0x565190687910 2024-03-30T00:41:52.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565190581600 con 0x565190687cf0 2024-03-30T00:41:52.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x565190578690 con 0x56519066d2b0 2024-03-30T00:41:52.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565190687910 0x56519066c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565190687910 0x56519066c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46380/0 (socket says 172.21.15.66:46380) 2024-03-30T00:41:52.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59be00640 1 -- 172.21.15.66:0/3344130910 learned_addr learned my addr 172.21.15.66:0/3344130910 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59be00640 1 -- 172.21.15.66:0/3344130910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56519066d2b0 msgr2=0x56519070bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59be00640 1 --2- 172.21.15.66:0/3344130910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56519066d2b0 0x56519070bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59be00640 1 -- 172.21.15.66:0/3344130910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565190687cf0 msgr2=0x56519066c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59be00640 1 --2- 172.21.15.66:0/3344130910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565190687cf0 0x56519066c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59be00640 1 -- 172.21.15.66:0/3344130910 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff588005d30 con 0x565190687910 2024-03-30T00:41:52.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59be00640 1 --2- 172.21.15.66:0/3344130910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565190687910 0x56519066c000 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff58800a450 tx=0x7ff588003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e2b43143c1448 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59aa00640 1 -- 172.21.15.66:0/3344130910 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5880b5070 con 0x565190687910 2024-03-30T00:41:52.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59aa00640 1 -- 172.21.15.66:0/3344130910 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5880a5590 con 0x565190687910 2024-03-30T00:41:52.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59aa00640 1 -- 172.21.15.66:0/3344130910 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5880a5b40 con 0x565190687910 2024-03-30T00:41:52.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 -- 172.21.15.66:0/3344130910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565190687910 msgr2=0x56519066c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 --2- 172.21.15.66:0/3344130910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565190687910 0x56519066c000 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff58800a450 tx=0x7ff588003030 comp rx=0 tx=0).stop 2024-03-30T00:41:52.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 -- 172.21.15.66:0/3344130910 shutdown_connections 2024-03-30T00:41:52.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 --2- 172.21.15.66:0/3344130910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565190687cf0 0x56519066c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 --2- 172.21.15.66:0/3344130910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565190687910 0x56519066c000 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 --2- 172.21.15.66:0/3344130910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56519066d2b0 0x56519070bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 -- 172.21.15.66:0/3344130910 >> 172.21.15.66:0/3344130910 conn(0x565190640830 msgr2=0x5651906838b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:52.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 -- 172.21.15.66:0/3344130910 shutdown_connections 2024-03-30T00:41:52.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 -- 172.21.15.66:0/3344130910 wait complete. 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 Processor -- start 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 -- start start 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.065+0000 7ff59f50a080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56519066d2b0 0x56519070e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59f50a080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565190687910 0x565190706bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59f50a080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x565190687cf0 0x5651907a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59f50a080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x565190681d20 con 0x56519066d2b0 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59f50a080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519071bed0 con 0x565190687910 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59f50a080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x56519071c170 con 0x565190687cf0 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59c800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56519066d2b0 0x56519070e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59c800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56519066d2b0 0x56519070e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46382/0 (socket says 172.21.15.66:46382) 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59c800640 1 -- 172.21.15.66:0/813552343 learned_addr learned my addr 172.21.15.66:0/813552343 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59d200640 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x565190687cf0 0x5651907a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59be00640 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565190687910 0x565190706bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59c800640 1 -- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x565190687cf0 msgr2=0x5651907a9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59c800640 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x565190687cf0 0x5651907a9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59c800640 1 -- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565190687910 msgr2=0x565190706bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59c800640 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565190687910 0x565190706bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59c800640 1 -- 172.21.15.66:0/813552343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651905aaa60 con 0x56519066d2b0 2024-03-30T00:41:52.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59be00640 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565190687910 0x565190706bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:52.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59c800640 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56519066d2b0 0x56519070e020 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff5900030b0 tx=0x7ff590008b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff599600640 1 -- 172.21.15.66:0/813552343 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59000f040 con 0x56519066d2b0 2024-03-30T00:41:52.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff599600640 1 -- 172.21.15.66:0/813552343 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59000d860 con 0x56519066d2b0 2024-03-30T00:41:52.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff599600640 1 -- 172.21.15.66:0/813552343 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59000dc10 con 0x56519066d2b0 2024-03-30T00:41:52.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59f50a080 1 -- 172.21.15.66:0/813552343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5651905908c0 con 0x56519066d2b0 2024-03-30T00:41:52.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff59f50a080 1 -- 172.21.15.66:0/813552343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565190583d00 con 0x56519066d2b0 2024-03-30T00:41:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.069+0000 7ff599600640 1 -- 172.21.15.66:0/813552343 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff59000dd70 con 0x56519066d2b0 2024-03-30T00:41:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff599600640 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff57c0650a0 0x7ff57c067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff599600640 1 -- 172.21.15.66:0/813552343 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff590083740 con 0x56519066d2b0 2024-03-30T00:41:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59be00640 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff57c0650a0 0x7ff57c067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5651907070f0 0x565190680010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 -- 172.21.15.66:0/813552343 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x56519070e560 con 0x5651907070f0 2024-03-30T00:41:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59be00640 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff57c0650a0 0x7ff57c067560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff5880b1000 tx=0x7ff5880a8000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59d200640 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5651907070f0 0x565190680010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59d200640 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5651907070f0 0x565190680010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59d200640 1 -- 172.21.15.66:0/813552343 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7ff594004530 con 0x5651907070f0 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 -- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5651907070f0 msgr2=0x565190680010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5651907070f0 0x565190680010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 -- 172.21.15.66:0/813552343 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff57c0650a0 msgr2=0x7ff57c067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff57c0650a0 0x7ff57c067560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff5880b1000 tx=0x7ff5880a8000 comp rx=0 tx=0).stop 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 -- 172.21.15.66:0/813552343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56519066d2b0 msgr2=0x56519070e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 --2- 172.21.15.66:0/813552343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56519066d2b0 0x56519070e020 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff5900030b0 tx=0x7ff590008b30 comp rx=0 tx=0).stop 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59d200640 1 -- 172.21.15.66:0/813552343 reap_dead start 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 -- 172.21.15.66:0/813552343 shutdown_connections 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 -- 172.21.15.66:0/813552343 >> 172.21.15.66:0/813552343 conn(0x565190640830 msgr2=0x565190708a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 -- 172.21.15.66:0/813552343 shutdown_connections 2024-03-30T00:41:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.073+0000 7ff59f50a080 1 -- 172.21.15.66:0/813552343 wait complete. 2024-03-30T00:41:52.088 INFO:teuthology.orchestra.run.smithi066.stdout:mytag 2024-03-30T00:41:52.089 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-03-30T00:41:52.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb7028080 1 Processor -- start 2024-03-30T00:41:52.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb7028080 1 -- start start 2024-03-30T00:41:52.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb7028080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a84074e30 0x560a8411aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb7028080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8408e910 0x560a841af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb7028080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560a8408ecf0 0x560a841af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb7028080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a83fef1e0 con 0x560a84074e30 2024-03-30T00:41:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb7028080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a83f8a960 con 0x560a8408e910 2024-03-30T00:41:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb7028080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a83f8a600 con 0x560a8408ecf0 2024-03-30T00:41:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb4200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a84074e30 0x560a8411aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb4200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a84074e30 0x560a8411aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46398/0 (socket says 172.21.15.66:46398) 2024-03-30T00:41:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb4200640 1 -- 172.21.15.66:0/612615934 learned_addr learned my addr 172.21.15.66:0/612615934 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb4c00640 1 --2- 172.21.15.66:0/612615934 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560a8408ecf0 0x560a841af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb3800640 1 --2- 172.21.15.66:0/612615934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8408e910 0x560a841af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb4200640 1 -- 172.21.15.66:0/612615934 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560a8408ecf0 msgr2=0x560a841af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb4200640 1 --2- 172.21.15.66:0/612615934 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560a8408ecf0 0x560a841af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb4200640 1 -- 172.21.15.66:0/612615934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8408e910 msgr2=0x560a841af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb4200640 1 --2- 172.21.15.66:0/612615934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8408e910 0x560a841af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.125+0000 7f0bb4200640 1 -- 172.21.15.66:0/612615934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac00a1c0 con 0x560a84074e30 2024-03-30T00:41:52.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb4200640 1 --2- 172.21.15.66:0/612615934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a84074e30 0x560a8411aed0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400b300 tx=0x7f0ba400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c948ca076e6f9ba0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb2400640 1 -- 172.21.15.66:0/612615934 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4014070 con 0x560a84074e30 2024-03-30T00:41:52.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb2400640 1 -- 172.21.15.66:0/612615934 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba400d510 con 0x560a84074e30 2024-03-30T00:41:52.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb2400640 1 -- 172.21.15.66:0/612615934 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba400d8c0 con 0x560a84074e30 2024-03-30T00:41:52.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 -- 172.21.15.66:0/612615934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a84074e30 msgr2=0x560a8411aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 --2- 172.21.15.66:0/612615934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a84074e30 0x560a8411aed0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400b300 tx=0x7f0ba400b7c0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 -- 172.21.15.66:0/612615934 shutdown_connections 2024-03-30T00:41:52.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 --2- 172.21.15.66:0/612615934 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560a8408ecf0 0x560a841af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 --2- 172.21.15.66:0/612615934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8408e910 0x560a841af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 --2- 172.21.15.66:0/612615934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a84074e30 0x560a8411aed0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 -- 172.21.15.66:0/612615934 >> 172.21.15.66:0/612615934 conn(0x560a840732f0 msgr2=0x560a840ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:52.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 -- 172.21.15.66:0/612615934 shutdown_connections 2024-03-30T00:41:52.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 -- 172.21.15.66:0/612615934 wait complete. 2024-03-30T00:41:52.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 Processor -- start 2024-03-30T00:41:52.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 -- start start 2024-03-30T00:41:52.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560a84074e30 0x560a84116ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8408e910 0x560a84086380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a8408ecf0 0x560a840fc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a83f81690 con 0x560a8408ecf0 2024-03-30T00:41:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a840fb2d0 con 0x560a8408e910 2024-03-30T00:41:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a840fb530 con 0x560a84074e30 2024-03-30T00:41:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a8408ecf0 0x560a840fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a8408ecf0 0x560a840fc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46412/0 (socket says 172.21.15.66:46412) 2024-03-30T00:41:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb4c00640 1 -- 172.21.15.66:0/1192395390 learned_addr learned my addr 172.21.15.66:0/1192395390 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb3800640 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8408e910 0x560a84086380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb4200640 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560a84074e30 0x560a84116ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb4c00640 1 -- 172.21.15.66:0/1192395390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560a84074e30 msgr2=0x560a84116ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb4c00640 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560a84074e30 0x560a84116ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb4c00640 1 -- 172.21.15.66:0/1192395390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8408e910 msgr2=0x560a84086380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb4c00640 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8408e910 0x560a84086380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb4c00640 1 -- 172.21.15.66:0/1192395390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba4001bb0 con 0x560a8408ecf0 2024-03-30T00:41:52.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb4c00640 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a8408ecf0 0x560a840fc450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0bac019210 tx=0x7f0bac018380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb1000640 1 -- 172.21.15.66:0/1192395390 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac0b5890 con 0x560a8408ecf0 2024-03-30T00:41:52.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb1000640 1 -- 172.21.15.66:0/1192395390 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bac0b59f0 con 0x560a8408ecf0 2024-03-30T00:41:52.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb1000640 1 -- 172.21.15.66:0/1192395390 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac0b5da0 con 0x560a8408ecf0 2024-03-30T00:41:52.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.129+0000 7f0bb7028080 1 -- 172.21.15.66:0/1192395390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560a83f8cd00 con 0x560a8408ecf0 2024-03-30T00:41:52.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.133+0000 7f0bb7028080 1 -- 172.21.15.66:0/1192395390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a83f8c760 con 0x560a8408ecf0 2024-03-30T00:41:52.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.133+0000 7f0bb1000640 1 -- 172.21.15.66:0/1192395390 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0bac0b1040 con 0x560a8408ecf0 2024-03-30T00:41:52.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.133+0000 7f0bb1000640 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b940650a0 0x7f0b94067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.133+0000 7f0bb1000640 1 -- 172.21.15.66:0/1192395390 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0bac11dfe0 con 0x560a8408ecf0 2024-03-30T00:41:52.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.133+0000 7f0bb4200640 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b940650a0 0x7f0b94067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.133+0000 7f0bb7028080 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x560a84117010 0x560a840fa270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.133+0000 7f0bb7028080 1 -- 172.21.15.66:0/1192395390 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x560a840868c0 con 0x560a84117010 2024-03-30T00:41:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.133+0000 7f0bb3800640 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x560a84117010 0x560a840fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.133+0000 7f0bb4200640 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b940650a0 0x7f0b94067560 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4000fc0 tx=0x7f0ba4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.133+0000 7f0bb3800640 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x560a84117010 0x560a840fa270 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb3800640 1 -- 172.21.15.66:0/1192395390 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f0ba0010630 con 0x560a84117010 2024-03-30T00:41:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb7028080 1 -- 172.21.15.66:0/1192395390 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x560a84117010 msgr2=0x560a840fa270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb7028080 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x560a84117010 0x560a840fa270 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb7028080 1 -- 172.21.15.66:0/1192395390 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b940650a0 msgr2=0x7f0b94067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb7028080 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b940650a0 0x7f0b94067560 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4000fc0 tx=0x7f0ba4009000 comp rx=0 tx=0).stop 2024-03-30T00:41:52.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb7028080 1 -- 172.21.15.66:0/1192395390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a8408ecf0 msgr2=0x560a840fc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb7028080 1 --2- 172.21.15.66:0/1192395390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a8408ecf0 0x560a840fc450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0bac019210 tx=0x7f0bac018380 comp rx=0 tx=0).stop 2024-03-30T00:41:52.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb4c00640 1 -- 172.21.15.66:0/1192395390 reap_dead start 2024-03-30T00:41:52.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb7028080 1 -- 172.21.15.66:0/1192395390 shutdown_connections 2024-03-30T00:41:52.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb7028080 1 -- 172.21.15.66:0/1192395390 >> 172.21.15.66:0/1192395390 conn(0x560a840732f0 msgr2=0x560a840ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:52.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb7028080 1 -- 172.21.15.66:0/1192395390 shutdown_connections 2024-03-30T00:41:52.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.137+0000 7f0bb7028080 1 -- 172.21.15.66:0/1192395390 wait complete. 2024-03-30T00:41:52.150 INFO:teuthology.orchestra.run.smithi066.stdout:mytag 2024-03-30T00:41:52.150 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-03-30T00:41:52.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bdac47080 1 Processor -- start 2024-03-30T00:41:52.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bdac47080 1 -- start start 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bdac47080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5628368a7d40 0x562836957810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bdac47080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628368c1870 0x5628368a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bdac47080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 0x5628368a7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bdac47080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628367c0960 con 0x5628368c1c50 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bdac47080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628367c0600 con 0x5628368c1870 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bdac47080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628367b7690 con 0x5628368a7d40 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bd7c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 0x5628368a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bd7c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 0x5628368a7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46422/0 (socket says 172.21.15.66:46422) 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bd7c00640 1 -- 172.21.15.66:0/3352065080 learned_addr learned my addr 172.21.15.66:0/3352065080 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bd7200640 1 --2- 172.21.15.66:0/3352065080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5628368a7d40 0x562836957810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.229+0000 7f0bd6800640 1 --2- 172.21.15.66:0/3352065080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628368c1870 0x5628368a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 -- 172.21.15.66:0/3352065080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5628368a7d40 msgr2=0x562836957810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 --2- 172.21.15.66:0/3352065080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5628368a7d40 0x562836957810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 -- 172.21.15.66:0/3352065080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628368c1870 msgr2=0x5628368a6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 --2- 172.21.15.66:0/3352065080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628368c1870 0x5628368a6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 -- 172.21.15.66:0/3352065080 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc8005df0 con 0x5628368c1c50 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 --2- 172.21.15.66:0/3352065080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 0x5628368a7170 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00028d0 tx=0x7f0bd0002d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f1d74bad16875f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd5400640 1 -- 172.21.15.66:0/3352065080 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00023c0 con 0x5628368c1c50 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd5400640 1 -- 172.21.15.66:0/3352065080 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd000d960 con 0x5628368c1c50 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd5400640 1 -- 172.21.15.66:0/3352065080 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000dcf0 con 0x5628368c1c50 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 -- 172.21.15.66:0/3352065080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 msgr2=0x5628368a7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 --2- 172.21.15.66:0/3352065080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 0x5628368a7170 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00028d0 tx=0x7f0bd0002d90 comp rx=0 tx=0).stop 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 -- 172.21.15.66:0/3352065080 shutdown_connections 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 --2- 172.21.15.66:0/3352065080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 0x5628368a7170 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 --2- 172.21.15.66:0/3352065080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628368c1870 0x5628368a6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 --2- 172.21.15.66:0/3352065080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5628368a7d40 0x562836957810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 -- 172.21.15.66:0/3352065080 >> 172.21.15.66:0/3352065080 conn(0x5628368a3dd0 msgr2=0x562836950f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 -- 172.21.15.66:0/3352065080 shutdown_connections 2024-03-30T00:41:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 -- 172.21.15.66:0/3352065080 wait complete. 2024-03-30T00:41:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 Processor -- start 2024-03-30T00:41:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 -- start start 2024-03-30T00:41:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628368a7d40 0x562836952fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5628368c1870 0x562836946fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 0x562836945e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562836956190 con 0x5628368c1c50 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562836956430 con 0x5628368a7d40 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bdac47080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628369566d0 con 0x5628368c1870 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 0x562836945e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 0x562836945e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46426/0 (socket says 172.21.15.66:46426) 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 -- 172.21.15.66:0/3179266404 learned_addr learned my addr 172.21.15.66:0/3179266404 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd6800640 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5628368c1870 0x562836946fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7200640 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628368a7d40 0x562836952fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 -- 172.21.15.66:0/3179266404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5628368c1870 msgr2=0x562836946fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5628368c1870 0x562836946fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 -- 172.21.15.66:0/3179266404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628368a7d40 msgr2=0x562836952fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628368a7d40 0x562836952fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 -- 172.21.15.66:0/3179266404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd0002520 con 0x5628368c1c50 2024-03-30T00:41:52.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.233+0000 7f0bd7c00640 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 0x562836945e20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0006fd0 tx=0x7f0bd00033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bcfe00640 1 -- 172.21.15.66:0/3179266404 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0005a50 con 0x5628368c1c50 2024-03-30T00:41:52.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bcfe00640 1 -- 172.21.15.66:0/3179266404 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd001f070 con 0x5628368c1c50 2024-03-30T00:41:52.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bcfe00640 1 -- 172.21.15.66:0/3179266404 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd001b660 con 0x5628368c1c50 2024-03-30T00:41:52.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bdac47080 1 -- 172.21.15.66:0/3179266404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5628367cf8c0 con 0x5628368c1c50 2024-03-30T00:41:52.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bdac47080 1 -- 172.21.15.66:0/3179266404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628367c2d00 con 0x5628368c1c50 2024-03-30T00:41:52.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bcfe00640 1 -- 172.21.15.66:0/3179266404 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0bd001b7c0 con 0x5628368c1c50 2024-03-30T00:41:52.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bcfe00640 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0bb80650a0 0x7f0bb8067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bcfe00640 1 -- 172.21.15.66:0/3179266404 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0bd00848c0 con 0x5628368c1c50 2024-03-30T00:41:52.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bd7200640 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0bb80650a0 0x7f0bb8067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bdac47080 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5628369e67f0 0x5628369e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bdac47080 1 -- 172.21.15.66:0/3179266404 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x5628369e7110 con 0x5628369e67f0 2024-03-30T00:41:52.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bd6800640 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5628369e67f0 0x5628369e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.237+0000 7f0bd7200640 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0bb80650a0 0x7f0bb8067560 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80ad000 tx=0x7f0bc80b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bd6800640 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5628369e67f0 0x5628369e6bd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bd6800640 1 -- 172.21.15.66:0/3179266404 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f0bc4010610 con 0x5628369e67f0 2024-03-30T00:41:52.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bdac47080 1 -- 172.21.15.66:0/3179266404 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5628369e67f0 msgr2=0x5628369e6bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bdac47080 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5628369e67f0 0x5628369e6bd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bdac47080 1 -- 172.21.15.66:0/3179266404 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0bb80650a0 msgr2=0x7f0bb8067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bdac47080 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0bb80650a0 0x7f0bb8067560 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80ad000 tx=0x7f0bc80b2040 comp rx=0 tx=0).stop 2024-03-30T00:41:52.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bdac47080 1 -- 172.21.15.66:0/3179266404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 msgr2=0x562836945e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bdac47080 1 --2- 172.21.15.66:0/3179266404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5628368c1c50 0x562836945e20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0006fd0 tx=0x7f0bd00033b0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bd7c00640 1 -- 172.21.15.66:0/3179266404 reap_dead start 2024-03-30T00:41:52.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bdac47080 1 -- 172.21.15.66:0/3179266404 shutdown_connections 2024-03-30T00:41:52.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bdac47080 1 -- 172.21.15.66:0/3179266404 >> 172.21.15.66:0/3179266404 conn(0x5628368a3dd0 msgr2=0x562836951f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bdac47080 1 -- 172.21.15.66:0/3179266404 shutdown_connections 2024-03-30T00:41:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.241+0000 7f0bdac47080 1 -- 172.21.15.66:0/3179266404 wait complete. 2024-03-30T00:41:52.255 INFO:teuthology.orchestra.run.smithi066.stdout:mytag 2024-03-30T00:41:52.255 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-03-30T00:41:52.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 Processor -- start 2024-03-30T00:41:52.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 -- start start 2024-03-30T00:41:52.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efc4d40 0x56336f065290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56336efde870 0x56336efdd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56336efdec50 0x56336efc3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336eedd960 con 0x56336efc4d40 2024-03-30T00:41:52.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336eedd600 con 0x56336efde870 2024-03-30T00:41:52.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x56336eed4690 con 0x56336efdec50 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efc4d40 0x56336f065290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efc4d40 0x56336f065290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46454/0 (socket says 172.21.15.66:46454) 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c4a00640 1 -- 172.21.15.66:0/2791616408 learned_addr learned my addr 172.21.15.66:0/2791616408 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c5400640 1 --2- 172.21.15.66:0/2791616408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56336efdec50 0x56336efc3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26bfe00640 1 --2- 172.21.15.66:0/2791616408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56336efde870 0x56336efdd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c4a00640 1 -- 172.21.15.66:0/2791616408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56336efdec50 msgr2=0x56336efc3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c4a00640 1 --2- 172.21.15.66:0/2791616408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56336efdec50 0x56336efc3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c4a00640 1 -- 172.21.15.66:0/2791616408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56336efde870 msgr2=0x56336efdd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c4a00640 1 --2- 172.21.15.66:0/2791616408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56336efde870 0x56336efdd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c4a00640 1 -- 172.21.15.66:0/2791616408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b8005d30 con 0x56336efc4d40 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c5400640 1 --2- 172.21.15.66:0/2791616408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56336efdec50 0x56336efc3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c4a00640 1 --2- 172.21.15.66:0/2791616408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efc4d40 0x56336f065290 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f26b800a450 tx=0x7f26b8003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a19776f737cedee server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26bea00640 1 -- 172.21.15.66:0/2791616408 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b80a8b30 con 0x56336efc4d40 2024-03-30T00:41:52.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26bea00640 1 -- 172.21.15.66:0/2791616408 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b80b7040 con 0x56336efc4d40 2024-03-30T00:41:52.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26bea00640 1 -- 172.21.15.66:0/2791616408 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b80b1450 con 0x56336efc4d40 2024-03-30T00:41:52.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 -- 172.21.15.66:0/2791616408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efc4d40 msgr2=0x56336f065290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 --2- 172.21.15.66:0/2791616408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efc4d40 0x56336f065290 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f26b800a450 tx=0x7f26b8003030 comp rx=0 tx=0).stop 2024-03-30T00:41:52.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 -- 172.21.15.66:0/2791616408 shutdown_connections 2024-03-30T00:41:52.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 --2- 172.21.15.66:0/2791616408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56336efdec50 0x56336efc3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 --2- 172.21.15.66:0/2791616408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56336efde870 0x56336efdd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 --2- 172.21.15.66:0/2791616408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efc4d40 0x56336f065290 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 -- 172.21.15.66:0/2791616408 >> 172.21.15.66:0/2791616408 conn(0x56336efc0dd0 msgr2=0x56336f06d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:52.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 -- 172.21.15.66:0/2791616408 shutdown_connections 2024-03-30T00:41:52.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.333+0000 7f26c8449080 1 -- 172.21.15.66:0/2791616408 wait complete. 2024-03-30T00:41:52.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c8449080 1 Processor -- start 2024-03-30T00:41:52.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c8449080 1 -- start start 2024-03-30T00:41:52.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c8449080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56336efc4d40 0x56336f067320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c8449080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56336efde870 0x56336f074920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c8449080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efdec50 0x56336f0623d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c8449080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336f064ef0 con 0x56336efdec50 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c8449080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336f065190 con 0x56336efde870 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c8449080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x56336f065430 con 0x56336efc4d40 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efdec50 0x56336f0623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efdec50 0x56336f0623d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46470/0 (socket says 172.21.15.66:46470) 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c5400640 1 -- 172.21.15.66:0/2410645066 learned_addr learned my addr 172.21.15.66:0/2410645066 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c4a00640 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56336efc4d40 0x56336f067320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26bfe00640 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56336efde870 0x56336f074920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c5400640 1 -- 172.21.15.66:0/2410645066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56336efc4d40 msgr2=0x56336f067320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c5400640 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56336efc4d40 0x56336f067320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c5400640 1 -- 172.21.15.66:0/2410645066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56336efde870 msgr2=0x56336f074920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c5400640 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56336efde870 0x56336f074920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c5400640 1 -- 172.21.15.66:0/2410645066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56336ef06a60 con 0x56336efdec50 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c5400640 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efdec50 0x56336f0623d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f26c000c240 tx=0x7f26c000c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26bd600640 1 -- 172.21.15.66:0/2410645066 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0013670 con 0x56336efdec50 2024-03-30T00:41:52.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26bd600640 1 -- 172.21.15.66:0/2410645066 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c00137d0 con 0x56336efdec50 2024-03-30T00:41:52.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26bd600640 1 -- 172.21.15.66:0/2410645066 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0013b80 con 0x56336efdec50 2024-03-30T00:41:52.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c8449080 1 -- 172.21.15.66:0/2410645066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56336eeec8c0 con 0x56336efdec50 2024-03-30T00:41:52.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.337+0000 7f26c8449080 1 -- 172.21.15.66:0/2410645066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56336eedfd00 con 0x56336efdec50 2024-03-30T00:41:52.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26bd600640 1 -- 172.21.15.66:0/2410645066 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f26c0013ce0 con 0x56336efdec50 2024-03-30T00:41:52.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26bd600640 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26a4065070 0x7f26a4067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26bd600640 1 -- 172.21.15.66:0/2410645066 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26c0083360 con 0x56336efdec50 2024-03-30T00:41:52.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c4a00640 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26a4065070 0x7f26a4067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c8449080 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x56336f105980 0x56336f066080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c8449080 1 -- 172.21.15.66:0/2410645066 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x56336efdc0f0 con 0x56336f105980 2024-03-30T00:41:52.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c4a00640 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26a4065070 0x7f26a4067530 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f26b80bd000 tx=0x7f26b80a8000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26bfe00640 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x56336f105980 0x56336f066080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26bfe00640 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x56336f105980 0x56336f066080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26bfe00640 1 -- 172.21.15.66:0/2410645066 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f26b000e5a0 con 0x56336f105980 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c8449080 1 -- 172.21.15.66:0/2410645066 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x56336f105980 msgr2=0x56336f066080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c8449080 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x56336f105980 0x56336f066080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c8449080 1 -- 172.21.15.66:0/2410645066 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26a4065070 msgr2=0x7f26a4067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c8449080 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26a4065070 0x7f26a4067530 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f26b80bd000 tx=0x7f26b80a8000 comp rx=0 tx=0).stop 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c8449080 1 -- 172.21.15.66:0/2410645066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efdec50 msgr2=0x56336f0623d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c8449080 1 --2- 172.21.15.66:0/2410645066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56336efdec50 0x56336f0623d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f26c000c240 tx=0x7f26c000c700 comp rx=0 tx=0).stop 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c5400640 1 -- 172.21.15.66:0/2410645066 reap_dead start 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c8449080 1 -- 172.21.15.66:0/2410645066 shutdown_connections 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.341+0000 7f26c8449080 1 -- 172.21.15.66:0/2410645066 >> 172.21.15.66:0/2410645066 conn(0x56336efc0dd0 msgr2=0x56336f068c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:52.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.345+0000 7f26c8449080 1 -- 172.21.15.66:0/2410645066 shutdown_connections 2024-03-30T00:41:52.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.345+0000 7f26c8449080 1 -- 172.21.15.66:0/2410645066 wait complete. 2024-03-30T00:41:52.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:41:52.357 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:41:52.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:41:52.487 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:41:52.574 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:41:52.574 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:41:52.574 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:41:52.584 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:41:52.584 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:41:52.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:52.584 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:41:52.632 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-30T00:41:52.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:41:52.633 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:41:52.633 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:41:52.643 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:41:52.643 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:41:52.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:52.643 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T00:41:52.695 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-30T00:41:52.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:41:52.696 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:41:52.696 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:41:52.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:41:52.705 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:41:52.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:52.705 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-30T00:41:52.755 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-30T00:41:52.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:41:52.757 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:41:52.757 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:41:52.766 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:41:52.766 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:41:52.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:41:52.766 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-30T00:41:52.815 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-30T00:41:52.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:41:52.816 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:52.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208b200640 1 Processor -- start 2024-03-30T00:41:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208b200640 1 -- start start 2024-03-30T00:41:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208b200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f208415c740 0x7f208406b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f20840b2520 0x7f20840cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208b200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208415cf90 con 0x7f208406ca90 2024-03-30T00:41:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208415d270 con 0x7f20840b2520 2024-03-30T00:41:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208b200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208415d550 con 0x7f208415c740 2024-03-30T00:41:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f20840b2520 0x7f20840cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f20840b2520 0x7f20840cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46480/0 (socket says 172.21.15.66:46480) 2024-03-30T00:41:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208a800640 1 -- 172.21.15.66:0/2943018901 learned_addr learned my addr 172.21.15.66:0/2943018901 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208a800640 1 -- 172.21.15.66:0/2943018901 >> v2:172.21.15.156:3300/0 conn(0x7f208415c740 msgr2=0x7f208406b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208a800640 1 --2- 172.21.15.66:0/2943018901 >> v2:172.21.15.156:3300/0 conn(0x7f208415c740 0x7f208406b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208a800640 1 -- 172.21.15.66:0/2943018901 >> v1:172.21.15.156:6789/0 conn(0x7f208406ca90 legacy=0x7f20840b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.949+0000 7f208a800640 1 -- 172.21.15.66:0/2943018901 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208415d830 con 0x7f20840b2520 2024-03-30T00:41:52.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f208a800640 1 --2- 172.21.15.66:0/2943018901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20840b2520 0x7f20840cd900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2084071860 tx=0x7f208001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7dfaa1bdae2864 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f2088a00640 1 -- 172.21.15.66:0/2943018901 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800a5860 con 0x7f20840b2520 2024-03-30T00:41:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f2088a00640 1 -- 172.21.15.66:0/2943018901 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20800a59c0 con 0x7f20840b2520 2024-03-30T00:41:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f2088a00640 1 -- 172.21.15.66:0/2943018901 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800a5d50 con 0x7f20840b2520 2024-03-30T00:41:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f208b200640 1 -- 172.21.15.66:0/2943018901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20840b2520 msgr2=0x7f20840cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f208b200640 1 --2- 172.21.15.66:0/2943018901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20840b2520 0x7f20840cd900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2084071860 tx=0x7f208001c980 comp rx=0 tx=0).stop 2024-03-30T00:41:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f208b200640 1 -- 172.21.15.66:0/2943018901 shutdown_connections 2024-03-30T00:41:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f208b200640 1 --2- 172.21.15.66:0/2943018901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20840b2520 0x7f20840cd900 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f208b200640 1 --2- 172.21.15.66:0/2943018901 >> v2:172.21.15.156:3300/0 conn(0x7f208415c740 0x7f208406b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f208b200640 1 -- 172.21.15.66:0/2943018901 >> 172.21.15.66:0/2943018901 conn(0x7f20840cce10 msgr2=0x7f20840cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f208b200640 1 -- 172.21.15.66:0/2943018901 shutdown_connections 2024-03-30T00:41:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.953+0000 7f208b200640 1 -- 172.21.15.66:0/2943018901 wait complete. 2024-03-30T00:41:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f208b200640 1 Processor -- start 2024-03-30T00:41:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f208b200640 1 -- start start 2024-03-30T00:41:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f208b200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f208406ca90 0x7f20840c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f208b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20840b2520 0x7f20840c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f208b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f208415c740 0x7f208414e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f208b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20840cb3d0 con 0x7f20840b2520 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f208b200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20840c6b50 con 0x7f208406ca90 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f208b200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20840cb620 con 0x7f208415c740 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f2089400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20840b2520 0x7f20840c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f2089400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20840b2520 0x7f20840c45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46488/0 (socket says 172.21.15.66:46488) 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f2089400640 1 -- 172.21.15.66:0/3541977135 learned_addr learned my addr 172.21.15.66:0/3541977135 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f208a800640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f208415c740 0x7f208414e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f2089e00640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f208406ca90 0x7f20840c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f2089400640 1 -- 172.21.15.66:0/3541977135 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f208415c740 msgr2=0x7f208414e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f2089400640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f208415c740 0x7f208414e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f2089400640 1 -- 172.21.15.66:0/3541977135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f208406ca90 msgr2=0x7f20840c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f2089400640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f208406ca90 0x7f20840c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f2089400640 1 -- 172.21.15.66:0/3541977135 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2080009b30 con 0x7f20840b2520 2024-03-30T00:41:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f2089400640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20840b2520 0x7f20840c45a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f207400c410 tx=0x7f207400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f207e000640 1 -- 172.21.15.66:0/3541977135 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074003600 con 0x7f20840b2520 2024-03-30T00:41:52.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f207e000640 1 -- 172.21.15.66:0/3541977135 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2074003780 con 0x7f20840b2520 2024-03-30T00:41:52.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f207e000640 1 -- 172.21.15.66:0/3541977135 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074003b30 con 0x7f20840b2520 2024-03-30T00:41:52.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f208b200640 1 -- 172.21.15.66:0/3541977135 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20840cb900 con 0x7f20840b2520 2024-03-30T00:41:52.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.957+0000 7f208b200640 1 -- 172.21.15.66:0/3541977135 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20840c4ae0 con 0x7f20840b2520 2024-03-30T00:41:52.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.961+0000 7f208b200640 1 -- 172.21.15.66:0/3541977135 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2084073cf0 con 0x7f20840b2520 2024-03-30T00:41:52.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.961+0000 7f207e000640 1 -- 172.21.15.66:0/3541977135 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2074003c90 con 0x7f20840b2520 2024-03-30T00:41:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.961+0000 7f207e000640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2068066690 0x7f2068068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.961+0000 7f207e000640 1 -- 172.21.15.66:0/3541977135 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2074083450 con 0x7f20840b2520 2024-03-30T00:41:52.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.961+0000 7f2089e00640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2068066690 0x7f2068068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:52.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.961+0000 7f2089e00640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2068066690 0x7f2068068b50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f20780098f0 tx=0x7f2078006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:52.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:52.965+0000 7f207e000640 1 -- 172.21.15.66:0/3541977135 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2074050710 con 0x7f20840b2520 2024-03-30T00:41:53.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 -- 172.21.15.66:0/3541977135 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20840b6310 con 0x7f20840b2520 2024-03-30T00:41:53.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f207e000640 1 -- 172.21.15.66:0/3541977135 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f2074053d30 con 0x7f20840b2520 2024-03-30T00:41:53.170 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:53.171 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3895007156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3895007156},{"type":"v1","addr":"172.21.15.156:6835","nonce":3895007156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/65993387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":65993387},{"type":"v1","addr":"172.21.15.156:6834","nonce":65993387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2013605775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2013605775},{"type":"v1","addr":"172.21.15.66:6835","nonce":2013605775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:41:21.269965+0000","modified":"2024-03-30T00:41:24.283171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1694329026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1694329026},{"type":"v1","addr":"172.21.15.66:6837","nonce":1694329026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-30T00:41:53.171 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-03-30T00:41:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 -- 172.21.15.66:0/3541977135 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2068066690 msgr2=0x7f2068068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2068066690 0x7f2068068b50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f20780098f0 tx=0x7f2078006f90 comp rx=0 tx=0).stop 2024-03-30T00:41:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 -- 172.21.15.66:0/3541977135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20840b2520 msgr2=0x7f20840c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20840b2520 0x7f20840c45a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f207400c410 tx=0x7f207400c8d0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 -- 172.21.15.66:0/3541977135 shutdown_connections 2024-03-30T00:41:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f208415c740 0x7f208414e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2068066690 0x7f2068068b50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20840b2520 0x7f20840c45a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 --2- 172.21.15.66:0/3541977135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f208406ca90 0x7f20840c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 -- 172.21.15.66:0/3541977135 >> 172.21.15.66:0/3541977135 conn(0x7f20840cce10 msgr2=0x7f208415b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 -- 172.21.15.66:0/3541977135 shutdown_connections 2024-03-30T00:41:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.161+0000 7f208b200640 1 -- 172.21.15.66:0/3541977135 wait complete. 2024-03-30T00:41:53.192 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:53.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a74400640 1 Processor -- start 2024-03-30T00:41:53.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a74400640 1 -- start start 2024-03-30T00:41:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a74400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8a6c15de30 0x7f8a6c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a74400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6c0cbb90 con 0x7f8a6c0cee50 2024-03-30T00:41:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a74400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6c0ce1b0 con 0x7f8a6c0af420 2024-03-30T00:41:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a74400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6c0afa10 con 0x7f8a6c15de30 2024-03-30T00:41:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a71a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8a6c15de30 0x7f8a6c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a71a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8a6c15de30 0x7f8a6c0cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46494/0 (socket says 172.21.15.66:46494) 2024-03-30T00:41:53.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a71a00640 1 -- 172.21.15.66:0/3690146048 learned_addr learned my addr 172.21.15.66:0/3690146048 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:53.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a71a00640 1 -- 172.21.15.66:0/3690146048 >> v1:172.21.15.156:6789/0 conn(0x7f8a6c0af420 legacy=0x7f8a6c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a71a00640 1 -- 172.21.15.66:0/3690146048 >> v1:172.21.15.66:6789/0 conn(0x7f8a6c0cee50 legacy=0x7f8a6c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a71a00640 1 -- 172.21.15.66:0/3690146048 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c0cf430 con 0x7f8a6c15de30 2024-03-30T00:41:53.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.325+0000 7f8a6be00640 1 -- 172.21.15.66:0/3690146048 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010740045 0 0) 0x7f8a6c0cbb90 con 0x7f8a6c0cee50 2024-03-30T00:41:53.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.329+0000 7f8a71a00640 1 --2- 172.21.15.66:0/3690146048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c15de30 0x7f8a6c0cc040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009870 tx=0x7f8a6001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb16d90ea5ba285 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:53.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.329+0000 7f8a6be00640 1 -- 172.21.15.66:0/3690146048 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a600ac640 con 0x7f8a6c15de30 2024-03-30T00:41:53.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.329+0000 7f8a6be00640 1 -- 172.21.15.66:0/3690146048 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a600ac7a0 con 0x7f8a6c15de30 2024-03-30T00:41:53.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.329+0000 7f8a6be00640 1 -- 172.21.15.66:0/3690146048 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a600acb30 con 0x7f8a6c15de30 2024-03-30T00:41:53.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.329+0000 7f8a74400640 1 -- 172.21.15.66:0/3690146048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c15de30 msgr2=0x7f8a6c0cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.329+0000 7f8a74400640 1 --2- 172.21.15.66:0/3690146048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c15de30 0x7f8a6c0cc040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009870 tx=0x7f8a6001c4d0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.329+0000 7f8a74400640 1 -- 172.21.15.66:0/3690146048 shutdown_connections 2024-03-30T00:41:53.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.329+0000 7f8a74400640 1 --2- 172.21.15.66:0/3690146048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c15de30 0x7f8a6c0cc040 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.329+0000 7f8a74400640 1 -- 172.21.15.66:0/3690146048 >> 172.21.15.66:0/3690146048 conn(0x7f8a6c0d0540 msgr2=0x7f8a6c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:53.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.329+0000 7f8a74400640 1 -- 172.21.15.66:0/3690146048 shutdown_connections 2024-03-30T00:41:53.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.329+0000 7f8a74400640 1 -- 172.21.15.66:0/3690146048 wait complete. 2024-03-30T00:41:53.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a74400640 1 Processor -- start 2024-03-30T00:41:53.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a74400640 1 -- start start 2024-03-30T00:41:53.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a74400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0af420 0x7f8a6c157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a74400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a6c0cee50 0x7f8a6c0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a74400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a6c15de30 0x7f8a6c0c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a74400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c158780 con 0x7f8a6c0af420 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a74400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c1584a0 con 0x7f8a6c15de30 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a74400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c0c5b10 con 0x7f8a6c0cee50 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a71a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0af420 0x7f8a6c157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a71a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0af420 0x7f8a6c157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46500/0 (socket says 172.21.15.66:46500) 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a71a00640 1 -- 172.21.15.66:0/2698380411 learned_addr learned my addr 172.21.15.66:0/2698380411 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a71a00640 1 -- 172.21.15.66:0/2698380411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a6c0cee50 msgr2=0x7f8a6c0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a71a00640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a6c0cee50 0x7f8a6c0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a71a00640 1 -- 172.21.15.66:0/2698380411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a6c15de30 msgr2=0x7f8a6c0c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a71a00640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a6c15de30 0x7f8a6c0c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a71a00640 1 -- 172.21.15.66:0/2698380411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a600094d0 con 0x7f8a6c0af420 2024-03-30T00:41:53.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a71a00640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0af420 0x7f8a6c157ec0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8a6001a100 tx=0x7f8a6001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:53.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a69600640 1 -- 172.21.15.66:0/2698380411 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a600b5600 con 0x7f8a6c0af420 2024-03-30T00:41:53.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a69600640 1 -- 172.21.15.66:0/2698380411 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a600b5760 con 0x7f8a6c0af420 2024-03-30T00:41:53.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a69600640 1 -- 172.21.15.66:0/2698380411 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a600b5ad0 con 0x7f8a6c0af420 2024-03-30T00:41:53.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a74400640 1 -- 172.21.15.66:0/2698380411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c0c5df0 con 0x7f8a6c0af420 2024-03-30T00:41:53.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.333+0000 7f8a74400640 1 -- 172.21.15.66:0/2698380411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c156800 con 0x7f8a6c0af420 2024-03-30T00:41:53.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.337+0000 7f8a74400640 1 -- 172.21.15.66:0/2698380411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6c0cdf60 con 0x7f8a6c0af420 2024-03-30T00:41:53.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.337+0000 7f8a69600640 1 -- 172.21.15.66:0/2698380411 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8a600b3040 con 0x7f8a6c0af420 2024-03-30T00:41:53.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.337+0000 7f8a69600640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a4c066690 0x7f8a4c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:53.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.337+0000 7f8a69600640 1 -- 172.21.15.66:0/2698380411 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8a6011eb50 con 0x7f8a6c0af420 2024-03-30T00:41:53.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.337+0000 7f8a71000640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a4c066690 0x7f8a4c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:53.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.337+0000 7f8a71000640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a4c066690 0x7f8a4c068b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a58000f70 tx=0x7f8a580079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:53.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.341+0000 7f8a69600640 1 -- 172.21.15.66:0/2698380411 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8a600ebcc0 con 0x7f8a6c0af420 2024-03-30T00:41:53.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.505+0000 7f8a74400640 1 -- 172.21.15.66:0/2698380411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a6c066c30 con 0x7f8a6c0af420 2024-03-30T00:41:53.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.505+0000 7f8a69600640 1 -- 172.21.15.66:0/2698380411 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16355 (secure 0 0 0) 0x7f8a600b1090 con 0x7f8a6c0af420 2024-03-30T00:41:53.515 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:53.515 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":42,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:41:33.508872+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:41:19.292693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T00:41:20.267522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:53.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 -- 172.21.15.66:0/2698380411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a4c066690 msgr2=0x7f8a4c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a4c066690 0x7f8a4c068b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a58000f70 tx=0x7f8a580079c0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 -- 172.21.15.66:0/2698380411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0af420 msgr2=0x7f8a6c157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0af420 0x7f8a6c157ec0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8a6001a100 tx=0x7f8a6001ad10 comp rx=0 tx=0).stop 2024-03-30T00:41:53.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 -- 172.21.15.66:0/2698380411 shutdown_connections 2024-03-30T00:41:53.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a6c15de30 0x7f8a6c0c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a6c0cee50 0x7f8a6c0b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a4c066690 0x7f8a4c068b50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 --2- 172.21.15.66:0/2698380411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a6c0af420 0x7f8a6c157ec0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 -- 172.21.15.66:0/2698380411 >> 172.21.15.66:0/2698380411 conn(0x7f8a6c0d0540 msgr2=0x7f8a6c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:53.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 -- 172.21.15.66:0/2698380411 shutdown_connections 2024-03-30T00:41:53.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.509+0000 7f8a74400640 1 -- 172.21.15.66:0/2698380411 wait complete. 2024-03-30T00:41:53.536 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:41:19.292693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:41:20.267522+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:41:53.537 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:41:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe887400640 1 Processor -- start 2024-03-30T00:41:53.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe887400640 1 -- start start 2024-03-30T00:41:53.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe887400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe8800cda90 0x7fe8800ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:53.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe887400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe88015a370 0x7fe8800cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:53.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe887400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88015a990 con 0x7fe8800cde70 2024-03-30T00:41:53.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe887400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8800b1b80 con 0x7fe8800cda90 2024-03-30T00:41:53.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe887400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8800cd3e0 con 0x7fe88015a370 2024-03-30T00:41:53.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe884a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe8800cda90 0x7fe8800ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:53.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe884a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe8800cda90 0x7fe8800ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46504/0 (socket says 172.21.15.66:46504) 2024-03-30T00:41:53.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe884a00640 1 -- 172.21.15.66:0/2343781834 learned_addr learned my addr 172.21.15.66:0/2343781834 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:53.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe885400640 1 --2- 172.21.15.66:0/2343781834 >> v2:172.21.15.156:3300/0 conn(0x7fe88015a370 0x7fe8800cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:53.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe884a00640 1 -- 172.21.15.66:0/2343781834 >> v2:172.21.15.156:3300/0 conn(0x7fe88015a370 msgr2=0x7fe8800cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:53.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe884a00640 1 --2- 172.21.15.66:0/2343781834 >> v2:172.21.15.156:3300/0 conn(0x7fe88015a370 0x7fe8800cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe884a00640 1 -- 172.21.15.66:0/2343781834 >> v1:172.21.15.156:6789/0 conn(0x7fe8800cde70 legacy=0x7fe880159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:53.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.673+0000 7fe884a00640 1 -- 172.21.15.66:0/2343781834 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8800b11e0 con 0x7fe8800cda90 2024-03-30T00:41:53.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe884a00640 1 --2- 172.21.15.66:0/2343781834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8800cda90 0x7fe8800ce520 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe8800b3140 tx=0x7fe87401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f3a408ef60be0b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:53.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe87f400640 1 -- 172.21.15.66:0/2343781834 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8740ad660 con 0x7fe8800cda90 2024-03-30T00:41:53.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe87f400640 1 -- 172.21.15.66:0/2343781834 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8740ad7c0 con 0x7fe8800cda90 2024-03-30T00:41:53.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe87f400640 1 -- 172.21.15.66:0/2343781834 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8740adb50 con 0x7fe8800cda90 2024-03-30T00:41:53.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe887400640 1 -- 172.21.15.66:0/2343781834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8800cda90 msgr2=0x7fe8800ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe887400640 1 --2- 172.21.15.66:0/2343781834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8800cda90 0x7fe8800ce520 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe8800b3140 tx=0x7fe87401c450 comp rx=0 tx=0).stop 2024-03-30T00:41:53.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe887400640 1 -- 172.21.15.66:0/2343781834 shutdown_connections 2024-03-30T00:41:53.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe887400640 1 --2- 172.21.15.66:0/2343781834 >> v2:172.21.15.156:3300/0 conn(0x7fe88015a370 0x7fe8800cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe887400640 1 --2- 172.21.15.66:0/2343781834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8800cda90 0x7fe8800ce520 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe887400640 1 -- 172.21.15.66:0/2343781834 >> 172.21.15.66:0/2343781834 conn(0x7fe880069f60 msgr2=0x7fe88006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:53.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe887400640 1 -- 172.21.15.66:0/2343781834 shutdown_connections 2024-03-30T00:41:53.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe887400640 1 -- 172.21.15.66:0/2343781834 wait complete. 2024-03-30T00:41:53.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.677+0000 7fe887400640 1 Processor -- start 2024-03-30T00:41:53.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe887400640 1 -- start start 2024-03-30T00:41:53.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe887400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8800cda90 0x7fe880153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:53.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe887400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8800cde70 0x7fe880071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:53.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe887400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe88015a370 0x7fe880071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:53.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe887400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe880070df0 con 0x7fe88015a370 2024-03-30T00:41:53.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe887400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8801543d0 con 0x7fe8800cde70 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe887400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88006fed0 con 0x7fe8800cda90 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe885400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe88015a370 0x7fe880071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe885400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe88015a370 0x7fe880071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46506/0 (socket says 172.21.15.66:46506) 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe885400640 1 -- 172.21.15.66:0/1122655795 learned_addr learned my addr 172.21.15.66:0/1122655795 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe884a00640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8800cda90 0x7fe880153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe87fe00640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8800cde70 0x7fe880071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe885400640 1 -- 172.21.15.66:0/1122655795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8800cda90 msgr2=0x7fe880153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe885400640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8800cda90 0x7fe880153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe885400640 1 -- 172.21.15.66:0/1122655795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8800cde70 msgr2=0x7fe880071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe885400640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8800cde70 0x7fe880071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe885400640 1 -- 172.21.15.66:0/1122655795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8740094d0 con 0x7fe88015a370 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe87fe00640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8800cde70 0x7fe880071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:53.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe885400640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe88015a370 0x7fe880071e60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe878007e40 tx=0x7fe87800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe87cc00640 1 -- 172.21.15.66:0/1122655795 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe878014620 con 0x7fe88015a370 2024-03-30T00:41:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe87cc00640 1 -- 172.21.15.66:0/1122655795 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87800ee90 con 0x7fe88015a370 2024-03-30T00:41:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe87cc00640 1 -- 172.21.15.66:0/1122655795 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8780149e0 con 0x7fe88015a370 2024-03-30T00:41:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe887400640 1 -- 172.21.15.66:0/1122655795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe880070170 con 0x7fe88015a370 2024-03-30T00:41:53.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe887400640 1 -- 172.21.15.66:0/1122655795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe880070730 con 0x7fe88015a370 2024-03-30T00:41:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.681+0000 7fe887400640 1 -- 172.21.15.66:0/1122655795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe880070f30 con 0x7fe88015a370 2024-03-30T00:41:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.685+0000 7fe87cc00640 1 -- 172.21.15.66:0/1122655795 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe878014b40 con 0x7fe88015a370 2024-03-30T00:41:53.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.685+0000 7fe87cc00640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe860066690 0x7fe860068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.685+0000 7fe87cc00640 1 -- 172.21.15.66:0/1122655795 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe878084b70 con 0x7fe88015a370 2024-03-30T00:41:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.685+0000 7fe884a00640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe860066690 0x7fe860068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:53.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.685+0000 7fe884a00640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe860066690 0x7fe860068b50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe8740af000 tx=0x7fe874008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:53.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.689+0000 7fe87cc00640 1 -- 172.21.15.66:0/1122655795 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe880070f30 con 0x7fe88015a370 2024-03-30T00:41:53.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe887400640 1 -- 172.21.15.66:0/1122655795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe880070f30 con 0x7fe88015a370 2024-03-30T00:41:53.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe87cc00640 1 -- 172.21.15.66:0/1122655795 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe880070f30 con 0x7fe88015a370 2024-03-30T00:41:53.859 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:41:53.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe887400640 1 -- 172.21.15.66:0/1122655795 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe860066690 msgr2=0x7fe860068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe887400640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe860066690 0x7fe860068b50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe8740af000 tx=0x7fe874008040 comp rx=0 tx=0).stop 2024-03-30T00:41:53.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe887400640 1 -- 172.21.15.66:0/1122655795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe88015a370 msgr2=0x7fe880071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:53.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe887400640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe88015a370 0x7fe880071e60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe878007e40 tx=0x7fe87800e4b0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe887400640 1 -- 172.21.15.66:0/1122655795 shutdown_connections 2024-03-30T00:41:53.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe887400640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe860066690 0x7fe860068b50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe887400640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe88015a370 0x7fe880071e60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe887400640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8800cde70 0x7fe880071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe887400640 1 --2- 172.21.15.66:0/1122655795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8800cda90 0x7fe880153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:53.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.849+0000 7fe887400640 1 -- 172.21.15.66:0/1122655795 >> 172.21.15.66:0/1122655795 conn(0x7fe880069f60 msgr2=0x7fe8800ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:53.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.853+0000 7fe887400640 1 -- 172.21.15.66:0/1122655795 shutdown_connections 2024-03-30T00:41:53.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:53.853+0000 7fe887400640 1 -- 172.21.15.66:0/1122655795 wait complete. 2024-03-30T00:41:53.880 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:41:54.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.013+0000 7f41c5200640 1 Processor -- start 2024-03-30T00:41:54.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.013+0000 7f41c5200640 1 -- start start 2024-03-30T00:41:54.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.013+0000 7f41c5200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c00cf2f0 0x7f41c006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.013+0000 7f41c5200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c00cb0e0 con 0x7f41c00cb570 2024-03-30T00:41:54.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.013+0000 7f41c5200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c00cf7f0 con 0x7f41c015ca80 2024-03-30T00:41:54.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.013+0000 7f41c5200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00cfad0 con 0x7f41c00cf2f0 2024-03-30T00:41:54.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.017+0000 7f41be800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f41c00cb570 0x7f41c00cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60452/0 (socket says 172.21.15.66:60452) 2024-03-30T00:41:54.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.017+0000 7f41be800640 1 -- 172.21.15.66:0/2751235065 learned_addr learned my addr 172.21.15.66:0/2751235065 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:54.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.017+0000 7f41b5400640 1 --2- 172.21.15.66:0/2751235065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c00cf2f0 0x7f41c006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.017+0000 7f41bd400640 1 -- 172.21.15.66:0/2751235065 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550123783 0 0) 0x7f41c00cb0e0 con 0x7f41c00cb570 2024-03-30T00:41:54.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.017+0000 7f41bd400640 1 -- 172.21.15.66:0/2751235065 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41a4003530 con 0x7f41c00cb570 2024-03-30T00:41:54.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.017+0000 7f41bd400640 1 -- 172.21.15.66:0/2751235065 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281882073 0 0) 0x7f41c00cf7f0 con 0x7f41c015ca80 2024-03-30T00:41:54.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.017+0000 7f41b5400640 1 -- 172.21.15.66:0/2751235065 >> v1:172.21.15.156:6789/0 conn(0x7f41c015ca80 legacy=0x7f41c006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.017+0000 7f41b5400640 1 -- 172.21.15.66:0/2751235065 >> v1:172.21.15.66:6789/0 conn(0x7f41c00cb570 legacy=0x7f41c00cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.017+0000 7f41b5400640 1 -- 172.21.15.66:0/2751235065 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c00cad70 con 0x7f41c00cf2f0 2024-03-30T00:41:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.017+0000 7f41b5400640 1 --2- 172.21.15.66:0/2751235065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c00cf2f0 0x7f41c006b7c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009dc0 tx=0x7f41a801efc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1abaaf074187cbbd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41bd400640 1 -- 172.21.15.66:0/2751235065 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8001680 con 0x7f41c00cf2f0 2024-03-30T00:41:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41bd400640 1 -- 172.21.15.66:0/2751235065 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a80a8b90 con 0x7f41c00cf2f0 2024-03-30T00:41:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41bd400640 1 -- 172.21.15.66:0/2751235065 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80b0440 con 0x7f41c00cf2f0 2024-03-30T00:41:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 -- 172.21.15.66:0/2751235065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c00cf2f0 msgr2=0x7f41c006b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 --2- 172.21.15.66:0/2751235065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c00cf2f0 0x7f41c006b7c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009dc0 tx=0x7f41a801efc0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 -- 172.21.15.66:0/2751235065 shutdown_connections 2024-03-30T00:41:54.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 --2- 172.21.15.66:0/2751235065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c00cf2f0 0x7f41c006b7c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 -- 172.21.15.66:0/2751235065 >> 172.21.15.66:0/2751235065 conn(0x7f41c00b2090 msgr2=0x7f41c00b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:54.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 -- 172.21.15.66:0/2751235065 shutdown_connections 2024-03-30T00:41:54.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 -- 172.21.15.66:0/2751235065 wait complete. 2024-03-30T00:41:54.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 Processor -- start 2024-03-30T00:41:54.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 -- start start 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c00cb570 0x7f41c00c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c00cf2f0 0x7f41c00b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c015ca80 0x7f41c015a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00c81b0 con 0x7f41c015ca80 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00c8490 con 0x7f41c00cf2f0 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41c5200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c00c8770 con 0x7f41c00cb570 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41be800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c015ca80 0x7f41c015a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41be800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c015ca80 0x7f41c015a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46512/0 (socket says 172.21.15.66:46512) 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41be800640 1 -- 172.21.15.66:0/359800070 learned_addr learned my addr 172.21.15.66:0/359800070 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41bde00640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c00cb570 0x7f41c00c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.021+0000 7f41b5400640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c00cf2f0 0x7f41c00b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41be800640 1 -- 172.21.15.66:0/359800070 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c00cb570 msgr2=0x7f41c00c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41be800640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c00cb570 0x7f41c00c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41be800640 1 -- 172.21.15.66:0/359800070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c00cf2f0 msgr2=0x7f41c00b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41be800640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c00cf2f0 0x7f41c00b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41be800640 1 -- 172.21.15.66:0/359800070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a8009a70 con 0x7f41c015ca80 2024-03-30T00:41:54.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41be800640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c015ca80 0x7f41c015a7a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f41b8000f70 tx=0x7f41b800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41b6a00640 1 -- 172.21.15.66:0/359800070 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b80074e0 con 0x7f41c015ca80 2024-03-30T00:41:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41b6a00640 1 -- 172.21.15.66:0/359800070 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b8007640 con 0x7f41c015ca80 2024-03-30T00:41:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41b6a00640 1 -- 172.21.15.66:0/359800070 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b80049f0 con 0x7f41c015ca80 2024-03-30T00:41:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41c5200640 1 -- 172.21.15.66:0/359800070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c00c8a50 con 0x7f41c015ca80 2024-03-30T00:41:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41c5200640 1 -- 172.21.15.66:0/359800070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c00c9790 con 0x7f41c015ca80 2024-03-30T00:41:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41c5200640 1 -- 172.21.15.66:0/359800070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c00c8f70 con 0x7f41c015ca80 2024-03-30T00:41:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41b6a00640 1 -- 172.21.15.66:0/359800070 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f41b8004b50 con 0x7f41c015ca80 2024-03-30T00:41:54.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41b6a00640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f419c066690 0x7f419c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41b6a00640 1 -- 172.21.15.66:0/359800070 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f41b80837e0 con 0x7f41c015ca80 2024-03-30T00:41:54.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.025+0000 7f41bde00640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f419c066690 0x7f419c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.029+0000 7f41bde00640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f419c066690 0x7f419c068b50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f41b0007fd0 tx=0x7f41b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:54.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.029+0000 7f41b6a00640 1 -- 172.21.15.66:0/359800070 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f41b8050a00 con 0x7f41c015ca80 2024-03-30T00:41:54.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 -- 172.21.15.66:0/359800070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41c0066c30 con 0x7f41c015ca80 2024-03-30T00:41:54.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41b6a00640 1 -- 172.21.15.66:0/359800070 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41b8054020 con 0x7f41c015ca80 2024-03-30T00:41:54.203 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:41:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 -- 172.21.15.66:0/359800070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f419c066690 msgr2=0x7f419c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f419c066690 0x7f419c068b50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f41b0007fd0 tx=0x7f41b0009040 comp rx=0 tx=0).stop 2024-03-30T00:41:54.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 -- 172.21.15.66:0/359800070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c015ca80 msgr2=0x7f41c015a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c015ca80 0x7f41c015a7a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f41b8000f70 tx=0x7f41b800cc20 comp rx=0 tx=0).stop 2024-03-30T00:41:54.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 -- 172.21.15.66:0/359800070 shutdown_connections 2024-03-30T00:41:54.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f419c066690 0x7f419c068b50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c015ca80 0x7f41c015a7a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c00cf2f0 0x7f41c00b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 --2- 172.21.15.66:0/359800070 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c00cb570 0x7f41c00c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 -- 172.21.15.66:0/359800070 >> 172.21.15.66:0/359800070 conn(0x7f41c00b2090 msgr2=0x7f41c006aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:54.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 -- 172.21.15.66:0/359800070 shutdown_connections 2024-03-30T00:41:54.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.193+0000 7f41c5200640 1 -- 172.21.15.66:0/359800070 wait complete. 2024-03-30T00:41:54.224 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:41:54.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.357+0000 7f68a7200640 1 Processor -- start 2024-03-30T00:41:54.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.357+0000 7f68a7200640 1 -- start start 2024-03-30T00:41:54.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.357+0000 7f68a7200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68a00cdf50 0x7f68a00cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.357+0000 7f68a7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f68a00cc420 0x7f68a00c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.357+0000 7f68a7200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a015ada0 con 0x7f68a0159990 2024-03-30T00:41:54.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.357+0000 7f68a7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a00cd450 con 0x7f68a00cc420 2024-03-30T00:41:54.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.357+0000 7f68a7200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a015a910 con 0x7f68a00cdf50 2024-03-30T00:41:54.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.357+0000 7f68a5200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f68a00cc420 0x7f68a00c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.357+0000 7f68a5200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f68a00cc420 0x7f68a00c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46518/0 (socket says 172.21.15.66:46518) 2024-03-30T00:41:54.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.357+0000 7f68a5200640 1 -- 172.21.15.66:0/2047765490 learned_addr learned my addr 172.21.15.66:0/2047765490 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:54.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.357+0000 7f6897400640 1 --2- 172.21.15.66:0/2047765490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68a00cdf50 0x7f68a00cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a5200640 1 -- 172.21.15.66:0/2047765490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68a00cdf50 msgr2=0x7f68a00cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:54.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a5200640 1 --2- 172.21.15.66:0/2047765490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68a00cdf50 0x7f68a00cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a5200640 1 -- 172.21.15.66:0/2047765490 >> v1:172.21.15.156:6789/0 conn(0x7f68a0159990 legacy=0x7f68a00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a5200640 1 -- 172.21.15.66:0/2047765490 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a0159d70 con 0x7f68a00cc420 2024-03-30T00:41:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a5200640 1 --2- 172.21.15.66:0/2047765490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a00cc420 0x7f68a00c9f70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6898009e50 tx=0x7f689801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9cc47a5bdef0c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f689f400640 1 -- 172.21.15.66:0/2047765490 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68980a5750 con 0x7f68a00cc420 2024-03-30T00:41:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f689f400640 1 -- 172.21.15.66:0/2047765490 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68980a58b0 con 0x7f68a00cc420 2024-03-30T00:41:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f689f400640 1 -- 172.21.15.66:0/2047765490 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68980a5c40 con 0x7f68a00cc420 2024-03-30T00:41:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a7200640 1 -- 172.21.15.66:0/2047765490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a00cc420 msgr2=0x7f68a00c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a7200640 1 --2- 172.21.15.66:0/2047765490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a00cc420 0x7f68a00c9f70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6898009e50 tx=0x7f689801c9a0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a7200640 1 -- 172.21.15.66:0/2047765490 shutdown_connections 2024-03-30T00:41:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a7200640 1 --2- 172.21.15.66:0/2047765490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a00cc420 0x7f68a00c9f70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a7200640 1 --2- 172.21.15.66:0/2047765490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68a00cdf50 0x7f68a00cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a7200640 1 -- 172.21.15.66:0/2047765490 >> 172.21.15.66:0/2047765490 conn(0x7f68a0098a20 msgr2=0x7f68a00cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a7200640 1 -- 172.21.15.66:0/2047765490 shutdown_connections 2024-03-30T00:41:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.361+0000 7f68a7200640 1 -- 172.21.15.66:0/2047765490 wait complete. 2024-03-30T00:41:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f68a7200640 1 Processor -- start 2024-03-30T00:41:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f68a7200640 1 -- start start 2024-03-30T00:41:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f68a7200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68a00cc420 0x7f68a01f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f68a7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a00cdf50 0x7f68a01f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f68a7200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68a0159990 0x7f68a00c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f68a7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00c90f0 con 0x7f68a00cdf50 2024-03-30T00:41:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f68a7200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a01f59e0 con 0x7f68a0159990 2024-03-30T00:41:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f68a7200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00c8e10 con 0x7f68a00cc420 2024-03-30T00:41:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f6897400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a00cdf50 0x7f68a01f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f689fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68a00cc420 0x7f68a01f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f689fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68a00cc420 0x7f68a01f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:33434/0 (socket says 172.21.15.66:33434) 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f689fe00640 1 -- 172.21.15.66:0/3306398003 learned_addr learned my addr 172.21.15.66:0/3306398003 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f68a5200640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68a0159990 0x7f68a00c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f6897400640 1 -- 172.21.15.66:0/3306398003 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68a00cc420 msgr2=0x7f68a01f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f6897400640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68a00cc420 0x7f68a01f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f6897400640 1 -- 172.21.15.66:0/3306398003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68a0159990 msgr2=0x7f68a00c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f6897400640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68a0159990 0x7f68a00c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f6897400640 1 -- 172.21.15.66:0/3306398003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6898009b30 con 0x7f68a00cdf50 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f6897400640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a00cdf50 0x7f68a01f4a60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f688800d5f0 tx=0x7f688800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.365+0000 7f689cc00640 1 -- 172.21.15.66:0/3306398003 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880125f0 con 0x7f68a00cdf50 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.369+0000 7f689cc00640 1 -- 172.21.15.66:0/3306398003 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888012750 con 0x7f68a00cdf50 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.369+0000 7f689cc00640 1 -- 172.21.15.66:0/3306398003 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888012ae0 con 0x7f68a00cdf50 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.369+0000 7f68a7200640 1 -- 172.21.15.66:0/3306398003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a01f5390 con 0x7f68a00cdf50 2024-03-30T00:41:54.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.369+0000 7f68a7200640 1 -- 172.21.15.66:0/3306398003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a01f4fa0 con 0x7f68a00cdf50 2024-03-30T00:41:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.369+0000 7f68a7200640 1 -- 172.21.15.66:0/3306398003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a0066c30 con 0x7f68a00cdf50 2024-03-30T00:41:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.369+0000 7f689cc00640 1 -- 172.21.15.66:0/3306398003 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6888012c40 con 0x7f68a00cdf50 2024-03-30T00:41:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.369+0000 7f689cc00640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f687c066690 0x7f687c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.369+0000 7f689cc00640 1 -- 172.21.15.66:0/3306398003 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6888084650 con 0x7f68a00cdf50 2024-03-30T00:41:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.369+0000 7f689fe00640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f687c066690 0x7f687c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.369+0000 7f689fe00640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f687c066690 0x7f687c068b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6890001810 tx=0x7f6890009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:54.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.373+0000 7f689cc00640 1 -- 172.21.15.66:0/3306398003 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6888051910 con 0x7f68a00cdf50 2024-03-30T00:41:54.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f68a7200640 1 -- 172.21.15.66:0/3306398003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68a01f5b20 con 0x7f68a00cdf50 2024-03-30T00:41:54.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f689cc00640 1 -- 172.21.15.66:0/3306398003 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6888054f30 con 0x7f68a00cdf50 2024-03-30T00:41:54.547 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f68a7200640 1 -- 172.21.15.66:0/3306398003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f687c066690 msgr2=0x7f687c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f68a7200640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f687c066690 0x7f687c068b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6890001810 tx=0x7f6890009040 comp rx=0 tx=0).stop 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f68a7200640 1 -- 172.21.15.66:0/3306398003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a00cdf50 msgr2=0x7f68a01f4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f68a7200640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a00cdf50 0x7f68a01f4a60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f688800d5f0 tx=0x7f688800dab0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f68a7200640 1 -- 172.21.15.66:0/3306398003 shutdown_connections 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f68a7200640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68a0159990 0x7f68a00c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f68a7200640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f687c066690 0x7f687c068b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f68a7200640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68a00cdf50 0x7f68a01f4a60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f68a7200640 1 --2- 172.21.15.66:0/3306398003 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68a00cc420 0x7f68a01f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.537+0000 7f68a7200640 1 -- 172.21.15.66:0/3306398003 >> 172.21.15.66:0/3306398003 conn(0x7f68a0098a20 msgr2=0x7f68a006e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.541+0000 7f68a7200640 1 -- 172.21.15.66:0/3306398003 shutdown_connections 2024-03-30T00:41:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.541+0000 7f68a7200640 1 -- 172.21.15.66:0/3306398003 wait complete. 2024-03-30T00:41:54.567 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:54.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.697+0000 7fb2d6600640 1 Processor -- start 2024-03-30T00:41:54.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.697+0000 7fb2d6600640 1 -- start start 2024-03-30T00:41:54.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2d6600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2d00cda90 0x7fb2d00ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2d6600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb2d00cde70 0x7fb2d00cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2d6600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2d0159c60 0x7fb2d015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2d6600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d00b1b80 con 0x7fb2d00cda90 2024-03-30T00:41:54.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2d6600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d015a990 con 0x7fb2d00cde70 2024-03-30T00:41:54.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2d6600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00b11e0 con 0x7fb2d0159c60 2024-03-30T00:41:54.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2cf400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2d00cda90 0x7fb2d00ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2cf400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2d00cda90 0x7fb2d00ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46546/0 (socket says 172.21.15.66:46546) 2024-03-30T00:41:54.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2cf400640 1 -- 172.21.15.66:0/2815557823 learned_addr learned my addr 172.21.15.66:0/2815557823 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:54.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2c7e00640 1 --2- 172.21.15.66:0/2815557823 >> v2:172.21.15.156:3300/0 conn(0x7fb2d00cde70 0x7fb2d00cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2cfe00640 1 --2- 172.21.15.66:0/2815557823 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2d0159c60 0x7fb2d015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2cf400640 1 -- 172.21.15.66:0/2815557823 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2d0159c60 msgr2=0x7fb2d015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:54.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2cf400640 1 --2- 172.21.15.66:0/2815557823 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2d0159c60 0x7fb2d015a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2cf400640 1 -- 172.21.15.66:0/2815557823 >> v2:172.21.15.156:3300/0 conn(0x7fb2d00cde70 msgr2=0x7fb2d00cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:54.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2cf400640 1 --2- 172.21.15.66:0/2815557823 >> v2:172.21.15.156:3300/0 conn(0x7fb2d00cde70 0x7fb2d00cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2cf400640 1 -- 172.21.15.66:0/2815557823 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d00b14c0 con 0x7fb2d00cda90 2024-03-30T00:41:54.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2c7e00640 1 --2- 172.21.15.66:0/2815557823 >> v2:172.21.15.156:3300/0 conn(0x7fb2d00cde70 0x7fb2d00cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:54.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.701+0000 7fb2cf400640 1 --2- 172.21.15.66:0/2815557823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d00cda90 0x7fb2d00ce520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00cc320 tx=0x7fb2c001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d88733c9c976b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:54.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2cea00640 1 -- 172.21.15.66:0/2815557823 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00ab670 con 0x7fb2d00cda90 2024-03-30T00:41:54.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2cea00640 1 -- 172.21.15.66:0/2815557823 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c00ab7d0 con 0x7fb2d00cda90 2024-03-30T00:41:54.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2cea00640 1 -- 172.21.15.66:0/2815557823 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00abb60 con 0x7fb2d00cda90 2024-03-30T00:41:54.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 -- 172.21.15.66:0/2815557823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d00cda90 msgr2=0x7fb2d00ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 --2- 172.21.15.66:0/2815557823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d00cda90 0x7fb2d00ce520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00cc320 tx=0x7fb2c001c490 comp rx=0 tx=0).stop 2024-03-30T00:41:54.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 -- 172.21.15.66:0/2815557823 shutdown_connections 2024-03-30T00:41:54.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 --2- 172.21.15.66:0/2815557823 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2d0159c60 0x7fb2d015a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 --2- 172.21.15.66:0/2815557823 >> v2:172.21.15.156:3300/0 conn(0x7fb2d00cde70 0x7fb2d00cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 --2- 172.21.15.66:0/2815557823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d00cda90 0x7fb2d00ce520 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 -- 172.21.15.66:0/2815557823 >> 172.21.15.66:0/2815557823 conn(0x7fb2d0069f60 msgr2=0x7fb2d006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:54.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 -- 172.21.15.66:0/2815557823 shutdown_connections 2024-03-30T00:41:54.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 -- 172.21.15.66:0/2815557823 wait complete. 2024-03-30T00:41:54.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 Processor -- start 2024-03-30T00:41:54.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 -- start start 2024-03-30T00:41:54.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d00cda90 0x7fb2d01563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d00cde70 0x7fb2d0158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2d0159c60 0x7fb2d00ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00c19b0 con 0x7fb2d00cde70 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0071f00 con 0x7fb2d00cda90 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2d6600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0159420 con 0x7fb2d0159c60 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2c7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d00cde70 0x7fb2d0158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2c7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d00cde70 0x7fb2d0158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46550/0 (socket says 172.21.15.66:46550) 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.705+0000 7fb2c7e00640 1 -- 172.21.15.66:0/3736289487 learned_addr learned my addr 172.21.15.66:0/3736289487 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2cfe00640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2d0159c60 0x7fb2d00ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2cf400640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d00cda90 0x7fb2d01563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7e00640 1 -- 172.21.15.66:0/3736289487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2d0159c60 msgr2=0x7fb2d00ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7e00640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2d0159c60 0x7fb2d00ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7e00640 1 -- 172.21.15.66:0/3736289487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d00cda90 msgr2=0x7fb2d01563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7e00640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d00cda90 0x7fb2d01563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7e00640 1 -- 172.21.15.66:0/3736289487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c00094d0 con 0x7fb2d00cde70 2024-03-30T00:41:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2cfe00640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2d0159c60 0x7fb2d00ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2cf400640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d00cda90 0x7fb2d01563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7e00640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d00cde70 0x7fb2d0158db0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800e7e0 tx=0x7fb2b800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7400640 1 -- 172.21.15.66:0/3736289487 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8004030 con 0x7fb2d00cde70 2024-03-30T00:41:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7400640 1 -- 172.21.15.66:0/3736289487 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b8004190 con 0x7fb2d00cde70 2024-03-30T00:41:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7400640 1 -- 172.21.15.66:0/3736289487 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8004500 con 0x7fb2d00cde70 2024-03-30T00:41:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2d6600640 1 -- 172.21.15.66:0/3736289487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d0159700 con 0x7fb2d00cde70 2024-03-30T00:41:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2d6600640 1 -- 172.21.15.66:0/3736289487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d0151f80 con 0x7fb2d00cde70 2024-03-30T00:41:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2d6600640 1 -- 172.21.15.66:0/3736289487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d0066c30 con 0x7fb2d00cde70 2024-03-30T00:41:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7400640 1 -- 172.21.15.66:0/3736289487 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb2b800c6d0 con 0x7fb2d00cde70 2024-03-30T00:41:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7400640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2ac066690 0x7fb2ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2c7400640 1 -- 172.21.15.66:0/3736289487 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb2b8084130 con 0x7fb2d00cde70 2024-03-30T00:41:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.709+0000 7fb2cf400640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2ac066690 0x7fb2ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:54.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.713+0000 7fb2cf400640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2ac066690 0x7fb2ac068b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00af000 tx=0x7fb2c00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:54.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.713+0000 7fb2c7400640 1 -- 172.21.15.66:0/3736289487 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb2b80513d0 con 0x7fb2d00cde70 2024-03-30T00:41:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.913+0000 7fb2d6600640 1 -- 172.21.15.66:0/3736289487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2d00b3170 con 0x7fb2d00cde70 2024-03-30T00:41:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.913+0000 7fb2c7400640 1 -- 172.21.15.66:0/3736289487 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fb2b80549f0 con 0x7fb2d00cde70 2024-03-30T00:41:54.924 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:54.924 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3895007156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3895007156},{"type":"v1","addr":"172.21.15.156:6835","nonce":3895007156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/65993387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":65993387},{"type":"v1","addr":"172.21.15.156:6834","nonce":65993387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2013605775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2013605775},{"type":"v1","addr":"172.21.15.66:6835","nonce":2013605775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:41:21.269965+0000","modified":"2024-03-30T00:41:24.283171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1694329026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1694329026},{"type":"v1","addr":"172.21.15.66:6837","nonce":1694329026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-30T00:41:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-03-30T00:41:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 -- 172.21.15.66:0/3736289487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2ac066690 msgr2=0x7fb2ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2ac066690 0x7fb2ac068b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00af000 tx=0x7fb2c00a7000 comp rx=0 tx=0).stop 2024-03-30T00:41:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 -- 172.21.15.66:0/3736289487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d00cde70 msgr2=0x7fb2d0158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d00cde70 0x7fb2d0158db0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800e7e0 tx=0x7fb2b800eca0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 -- 172.21.15.66:0/3736289487 shutdown_connections 2024-03-30T00:41:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2d0159c60 0x7fb2d00ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2ac066690 0x7fb2ac068b50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d00cde70 0x7fb2d0158db0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 --2- 172.21.15.66:0/3736289487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d00cda90 0x7fb2d01563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 -- 172.21.15.66:0/3736289487 >> 172.21.15.66:0/3736289487 conn(0x7fb2d0069f60 msgr2=0x7fb2d015bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 -- 172.21.15.66:0/3736289487 shutdown_connections 2024-03-30T00:41:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:54.917+0000 7fb2d6600640 1 -- 172.21.15.66:0/3736289487 wait complete. 2024-03-30T00:41:54.945 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d56c00640 1 Processor -- start 2024-03-30T00:41:55.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d56c00640 1 -- start start 2024-03-30T00:41:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d56c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1d500cda90 0x7f1d500ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d56c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d500cd3e0 con 0x7f1d5015a370 2024-03-30T00:41:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d56c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5015a990 con 0x7f1d500cde70 2024-03-30T00:41:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d56c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d500b1b80 con 0x7f1d500cda90 2024-03-30T00:41:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d4fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1d500cda90 0x7f1d500ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d4fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1d500cda90 0x7f1d500ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46562/0 (socket says 172.21.15.66:46562) 2024-03-30T00:41:55.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d4fe00640 1 -- 172.21.15.66:0/187651661 learned_addr learned my addr 172.21.15.66:0/187651661 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:55.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d4fe00640 1 -- 172.21.15.66:0/187651661 >> v1:172.21.15.156:6789/0 conn(0x7f1d500cde70 legacy=0x7f1d50159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:55.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d4fe00640 1 -- 172.21.15.66:0/187651661 >> v1:172.21.15.66:6789/0 conn(0x7f1d5015a370 legacy=0x7f1d50156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d4fe00640 1 -- 172.21.15.66:0/187651661 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d500b11e0 con 0x7f1d500cda90 2024-03-30T00:41:55.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.076+0000 7f1d4f400640 1 -- 172.21.15.66:0/187651661 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028899018 0 0) 0x7f1d500cd3e0 con 0x7f1d5015a370 2024-03-30T00:41:55.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.080+0000 7f1d4fe00640 1 --2- 172.21.15.66:0/187651661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d500cda90 0x7f1d500ce520 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009870 tx=0x7f1d4001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c0a191445ffae1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:55.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.080+0000 7f1d4f400640 1 -- 172.21.15.66:0/187651661 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d400ac600 con 0x7f1d500cda90 2024-03-30T00:41:55.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.080+0000 7f1d4f400640 1 -- 172.21.15.66:0/187651661 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d400a8050 con 0x7f1d500cda90 2024-03-30T00:41:55.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.080+0000 7f1d4f400640 1 -- 172.21.15.66:0/187651661 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d400ac980 con 0x7f1d500cda90 2024-03-30T00:41:55.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.080+0000 7f1d56c00640 1 -- 172.21.15.66:0/187651661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d500cda90 msgr2=0x7f1d500ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.080+0000 7f1d56c00640 1 --2- 172.21.15.66:0/187651661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d500cda90 0x7f1d500ce520 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009870 tx=0x7f1d4001c490 comp rx=0 tx=0).stop 2024-03-30T00:41:55.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.080+0000 7f1d56c00640 1 -- 172.21.15.66:0/187651661 shutdown_connections 2024-03-30T00:41:55.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.080+0000 7f1d56c00640 1 --2- 172.21.15.66:0/187651661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d500cda90 0x7f1d500ce520 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.080+0000 7f1d56c00640 1 -- 172.21.15.66:0/187651661 >> 172.21.15.66:0/187651661 conn(0x7f1d50069f60 msgr2=0x7f1d5006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:55.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.080+0000 7f1d56c00640 1 -- 172.21.15.66:0/187651661 shutdown_connections 2024-03-30T00:41:55.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.080+0000 7f1d56c00640 1 -- 172.21.15.66:0/187651661 wait complete. 2024-03-30T00:41:55.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d56c00640 1 Processor -- start 2024-03-30T00:41:55.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d56c00640 1 -- start start 2024-03-30T00:41:55.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d56c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d500cda90 0x7f1d501576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d56c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d500cde70 0x7f1d501591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d56c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d5015a370 0x7f1d50151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d56c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d501596e0 con 0x7f1d500cde70 2024-03-30T00:41:55.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d56c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d50148da0 con 0x7f1d500cda90 2024-03-30T00:41:55.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d56c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d50149080 con 0x7f1d5015a370 2024-03-30T00:41:55.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d47400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d500cde70 0x7f1d501591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d47400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d500cde70 0x7f1d501591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46564/0 (socket says 172.21.15.66:46564) 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d47400640 1 -- 172.21.15.66:0/1782779661 learned_addr learned my addr 172.21.15.66:0/1782779661 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d54c00640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d5015a370 0x7f1d50151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d4fe00640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d500cda90 0x7f1d501576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d47400640 1 -- 172.21.15.66:0/1782779661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d5015a370 msgr2=0x7f1d50151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d47400640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d5015a370 0x7f1d50151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d47400640 1 -- 172.21.15.66:0/1782779661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d500cda90 msgr2=0x7f1d501576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d47400640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d500cda90 0x7f1d501576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d47400640 1 -- 172.21.15.66:0/1782779661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d400094d0 con 0x7f1d500cde70 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d47400640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d500cde70 0x7f1d501591a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800c410 tx=0x7f1d3800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d4cc00640 1 -- 172.21.15.66:0/1782779661 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38011790 con 0x7f1d500cde70 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d4cc00640 1 -- 172.21.15.66:0/1782779661 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d380118f0 con 0x7f1d500cde70 2024-03-30T00:41:55.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d4cc00640 1 -- 172.21.15.66:0/1782779661 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38011ce0 con 0x7f1d500cde70 2024-03-30T00:41:55.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d56c00640 1 -- 172.21.15.66:0/1782779661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d50149360 con 0x7f1d500cde70 2024-03-30T00:41:55.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.084+0000 7f1d56c00640 1 -- 172.21.15.66:0/1782779661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d50149920 con 0x7f1d500cde70 2024-03-30T00:41:55.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.088+0000 7f1d56c00640 1 -- 172.21.15.66:0/1782779661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d50066c30 con 0x7f1d500cde70 2024-03-30T00:41:55.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.088+0000 7f1d4cc00640 1 -- 172.21.15.66:0/1782779661 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1d3801c460 con 0x7f1d500cde70 2024-03-30T00:41:55.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.088+0000 7f1d4cc00640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1d2c066690 0x7f1d2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.088+0000 7f1d4cc00640 1 -- 172.21.15.66:0/1782779661 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1d38083f50 con 0x7f1d500cde70 2024-03-30T00:41:55.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.088+0000 7f1d4fe00640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1d2c066690 0x7f1d2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.088+0000 7f1d4fe00640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1d2c066690 0x7f1d2c068b50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1d400ae000 tx=0x7f1d400a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:55.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.092+0000 7f1d4cc00640 1 -- 172.21.15.66:0/1782779661 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1d38051160 con 0x7f1d500cde70 2024-03-30T00:41:55.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.256+0000 7f1d56c00640 1 -- 172.21.15.66:0/1782779661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d500b3170 con 0x7f1d500cde70 2024-03-30T00:41:55.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.256+0000 7f1d4cc00640 1 -- 172.21.15.66:0/1782779661 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16355 (secure 0 0 0) 0x7f1d38054780 con 0x7f1d500cde70 2024-03-30T00:41:55.267 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:55.267 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":42,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:41:33.508872+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:41:19.292693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T00:41:20.267522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:55.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 -- 172.21.15.66:0/1782779661 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1d2c066690 msgr2=0x7f1d2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1d2c066690 0x7f1d2c068b50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1d400ae000 tx=0x7f1d400a7000 comp rx=0 tx=0).stop 2024-03-30T00:41:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 -- 172.21.15.66:0/1782779661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d500cde70 msgr2=0x7f1d501591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d500cde70 0x7f1d501591a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800c410 tx=0x7f1d3800c8d0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 -- 172.21.15.66:0/1782779661 shutdown_connections 2024-03-30T00:41:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d5015a370 0x7f1d50151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1d2c066690 0x7f1d2c068b50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d500cde70 0x7f1d501591a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 --2- 172.21.15.66:0/1782779661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d500cda90 0x7f1d501576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 -- 172.21.15.66:0/1782779661 >> 172.21.15.66:0/1782779661 conn(0x7f1d50069f60 msgr2=0x7f1d5015ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 -- 172.21.15.66:0/1782779661 shutdown_connections 2024-03-30T00:41:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.260+0000 7f1d56c00640 1 -- 172.21.15.66:0/1782779661 wait complete. 2024-03-30T00:41:55.288 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T00:41:55.288 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:55.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007e00640 1 Processor -- start 2024-03-30T00:41:55.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007e00640 1 -- start start 2024-03-30T00:41:55.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f900806ba80 0x7f900806ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 0x7f90080caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90080ce070 con 0x7f900815c9f0 2024-03-30T00:41:55.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900806cf60 con 0x7f900806ba80 2024-03-30T00:41:55.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90080ce350 con 0x7f90080b2b60 2024-03-30T00:41:55.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 0x7f90080caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9006000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f900806ba80 0x7f900806ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9006000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f900806ba80 0x7f900806ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:48220/0 (socket says 172.21.15.66:48220) 2024-03-30T00:41:55.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9006000640 1 -- 172.21.15.66:0/2605674197 learned_addr learned my addr 172.21.15.66:0/2605674197 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 0x7f90080caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:33470/0 (socket says 172.21.15.66:33470) 2024-03-30T00:41:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007400640 1 -- 172.21.15.66:0/2605674197 >> v2:172.21.15.156:3300/0 conn(0x7f900806ba80 msgr2=0x7f900806ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007400640 1 --2- 172.21.15.66:0/2605674197 >> v2:172.21.15.156:3300/0 conn(0x7f900806ba80 0x7f900806ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007400640 1 -- 172.21.15.66:0/2605674197 >> v1:172.21.15.156:6789/0 conn(0x7f900815c9f0 legacy=0x7f900806b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9007400640 1 -- 172.21.15.66:0/2605674197 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90080ce630 con 0x7f90080b2b60 2024-03-30T00:41:55.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.420+0000 7f9006000640 1 --2- 172.21.15.66:0/2605674197 >> v2:172.21.15.156:3300/0 conn(0x7f900806ba80 0x7f900806ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:55.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9007400640 1 --2- 172.21.15.66:0/2605674197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 0x7f90080caf60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9000009d90 tx=0x7f900001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e823f3d4002db789 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:55.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9005600640 1 -- 172.21.15.66:0/2605674197 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000a5a00 con 0x7f90080b2b60 2024-03-30T00:41:55.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9005600640 1 -- 172.21.15.66:0/2605674197 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90000a5b60 con 0x7f90080b2b60 2024-03-30T00:41:55.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9005600640 1 -- 172.21.15.66:0/2605674197 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000ad440 con 0x7f90080b2b60 2024-03-30T00:41:55.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9007e00640 1 -- 172.21.15.66:0/2605674197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 msgr2=0x7f90080caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9007e00640 1 --2- 172.21.15.66:0/2605674197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 0x7f90080caf60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9000009d90 tx=0x7f900001c940 comp rx=0 tx=0).stop 2024-03-30T00:41:55.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9007e00640 1 -- 172.21.15.66:0/2605674197 shutdown_connections 2024-03-30T00:41:55.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9007e00640 1 --2- 172.21.15.66:0/2605674197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 0x7f90080caf60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9007e00640 1 --2- 172.21.15.66:0/2605674197 >> v2:172.21.15.156:3300/0 conn(0x7f900806ba80 0x7f900806ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9007e00640 1 -- 172.21.15.66:0/2605674197 >> 172.21.15.66:0/2605674197 conn(0x7f900806e0d0 msgr2=0x7f900806e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:55.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9007e00640 1 -- 172.21.15.66:0/2605674197 shutdown_connections 2024-03-30T00:41:55.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.424+0000 7f9007e00640 1 -- 172.21.15.66:0/2605674197 wait complete. 2024-03-30T00:41:55.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007e00640 1 Processor -- start 2024-03-30T00:41:55.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007e00640 1 -- start start 2024-03-30T00:41:55.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900806ba80 0x7f90080c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 0x7f90080ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f900815c9f0 0x7f9008159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9008154000 con 0x7f900815c9f0 2024-03-30T00:41:55.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9008155a80 con 0x7f900806ba80 2024-03-30T00:41:55.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90080c9440 con 0x7f90080b2b60 2024-03-30T00:41:55.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f900815c9f0 0x7f9008159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f900815c9f0 0x7f9008159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46570/0 (socket says 172.21.15.66:46570) 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007400640 1 -- 172.21.15.66:0/1392125262 learned_addr learned my addr 172.21.15.66:0/1392125262 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9006000640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 0x7f90080ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9006a00640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900806ba80 0x7f90080c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007400640 1 -- 172.21.15.66:0/1392125262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 msgr2=0x7f90080ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007400640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 0x7f90080ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007400640 1 -- 172.21.15.66:0/1392125262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900806ba80 msgr2=0x7f90080c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007400640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900806ba80 0x7f90080c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007400640 1 -- 172.21.15.66:0/1392125262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff40094d0 con 0x7f900815c9f0 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9006a00640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900806ba80 0x7f90080c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007400640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f900815c9f0 0x7f9008159f70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f900001a570 tx=0x7f90000196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f8ffea00640 1 -- 172.21.15.66:0/1392125262 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000aece0 con 0x7f900815c9f0 2024-03-30T00:41:55.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f8ffea00640 1 -- 172.21.15.66:0/1392125262 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90000aee40 con 0x7f900815c9f0 2024-03-30T00:41:55.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007e00640 1 -- 172.21.15.66:0/1392125262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9000009a70 con 0x7f900815c9f0 2024-03-30T00:41:55.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f8ffea00640 1 -- 172.21.15.66:0/1392125262 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000b7650 con 0x7f900815c9f0 2024-03-30T00:41:55.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007e00640 1 -- 172.21.15.66:0/1392125262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90080c9650 con 0x7f900815c9f0 2024-03-30T00:41:55.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.428+0000 7f9007e00640 1 -- 172.21.15.66:0/1392125262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9008143040 con 0x7f900815c9f0 2024-03-30T00:41:55.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.432+0000 7f8ffea00640 1 -- 172.21.15.66:0/1392125262 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f90000b4050 con 0x7f900815c9f0 2024-03-30T00:41:55.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.432+0000 7f8ffea00640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8fe80666c0 0x7f8fe8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.432+0000 7f8ffea00640 1 -- 172.21.15.66:0/1392125262 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f900011f790 con 0x7f900815c9f0 2024-03-30T00:41:55.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.432+0000 7f9006a00640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8fe80666c0 0x7f8fe8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.432+0000 7f9006a00640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8fe80666c0 0x7f8fe8068b80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8000f70 tx=0x7f8ff8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:55.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.436+0000 7f8ffea00640 1 -- 172.21.15.66:0/1392125262 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f90000ec990 con 0x7f900815c9f0 2024-03-30T00:41:55.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.628+0000 7f9007e00640 1 -- 172.21.15.66:0/1392125262 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9008073cf0 con 0x7f900815c9f0 2024-03-30T00:41:55.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.628+0000 7f8ffea00640 1 -- 172.21.15.66:0/1392125262 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f90000effb0 con 0x7f900815c9f0 2024-03-30T00:41:55.641 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:55.641 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3895007156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3895007156},{"type":"v1","addr":"172.21.15.156:6835","nonce":3895007156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/65993387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":65993387},{"type":"v1","addr":"172.21.15.156:6834","nonce":65993387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2013605775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2013605775},{"type":"v1","addr":"172.21.15.66:6835","nonce":2013605775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:41:21.269965+0000","modified":"2024-03-30T00:41:24.283171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1694329026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1694329026},{"type":"v1","addr":"172.21.15.66:6837","nonce":1694329026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-30T00:41:55.641 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 -- 172.21.15.66:0/1392125262 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8fe80666c0 msgr2=0x7f8fe8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8fe80666c0 0x7f8fe8068b80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8000f70 tx=0x7f8ff8009040 comp rx=0 tx=0).stop 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 -- 172.21.15.66:0/1392125262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f900815c9f0 msgr2=0x7f9008159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f900815c9f0 0x7f9008159f70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f900001a570 tx=0x7f90000196e0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 -- 172.21.15.66:0/1392125262 shutdown_connections 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8fe80666c0 0x7f8fe8068b80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f900815c9f0 0x7f9008159f70 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90080b2b60 0x7f90080ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 --2- 172.21.15.66:0/1392125262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f900806ba80 0x7f90080c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 -- 172.21.15.66:0/1392125262 >> 172.21.15.66:0/1392125262 conn(0x7f900806e0d0 msgr2=0x7f90080b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 -- 172.21.15.66:0/1392125262 shutdown_connections 2024-03-30T00:41:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.632+0000 7f9007e00640 1 -- 172.21.15.66:0/1392125262 wait complete. 2024-03-30T00:41:55.662 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:41:55.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f6185600640 1 Processor -- start 2024-03-30T00:41:55.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f6185600640 1 -- start start 2024-03-30T00:41:55.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f6185600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f618015a640 0x7f618006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f6185600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f61800cabb0 0x7f61800cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f6185600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618006d070 con 0x7f61800ce520 2024-03-30T00:41:55.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f6185600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61800cbba0 con 0x7f61800cabb0 2024-03-30T00:41:55.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f6185600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618006e2c0 con 0x7f618015a640 2024-03-30T00:41:55.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f6176000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f61800ce520 0x7f61800ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60468/0 (socket says 172.21.15.66:60468) 2024-03-30T00:41:55.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f6176000640 1 -- 172.21.15.66:0/1981494941 learned_addr learned my addr 172.21.15.66:0/1981494941 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:55.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f617f400640 1 --2- 172.21.15.66:0/1981494941 >> v2:172.21.15.156:3300/0 conn(0x7f61800cabb0 0x7f61800cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f617ea00640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f618015a640 0x7f618006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.792+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320333719 0 0) 0x7f618006d070 con 0x7f61800ce520 2024-03-30T00:41:55.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6164003530 con 0x7f61800ce520 2024-03-30T00:41:55.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 768882260 0 0) 0x7f6164003530 con 0x7f61800ce520 2024-03-30T00:41:55.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f618015a640 msgr2=0x7f618006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:55.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f618015a640 0x7f618006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 >> v2:172.21.15.156:3300/0 conn(0x7f61800cabb0 msgr2=0x7f61800cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:55.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 --2- 172.21.15.66:0/1981494941 >> v2:172.21.15.156:3300/0 conn(0x7f61800cabb0 0x7f61800cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61800cb8c0 con 0x7f61800ce520 2024-03-30T00:41:55.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f6168001680 con 0x7f61800ce520 2024-03-30T00:41:55.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 >> v1:172.21.15.66:6789/0 conn(0x7f61800ce520 legacy=0x7f61800ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6164003980 0x7f6164003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f618015a640 0x7f61640047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6164004d10 0x7f6164005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61800cbba0 con 0x7f6164003980 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618006e2c0 con 0x7f6164004d10 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616400b990 con 0x7f618015a640 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617ea00640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6164003980 0x7f6164003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617f400640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6164004d10 0x7f6164005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f6176000640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f618015a640 0x7f61640047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617ea00640 1 -- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f618015a640 msgr2=0x7f61640047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617ea00640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f618015a640 0x7f61640047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617ea00640 1 -- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6164004d10 msgr2=0x7f6164005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617ea00640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6164004d10 0x7f6164005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617ea00640 1 -- 172.21.15.66:0/1981494941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6164003530 con 0x7f6164003980 2024-03-30T00:41:55.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f6176000640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f618015a640 0x7f61640047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:55.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.796+0000 7f617ea00640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6164003980 0x7f6164003da0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6170009ae0 tx=0x7f617001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f4ac7f9902d751 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:55.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700a5520 con 0x7f6164003980 2024-03-30T00:41:55.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61700a5680 con 0x7f6164003980 2024-03-30T00:41:55.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f617e000640 1 -- 172.21.15.66:0/1981494941 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700a5a30 con 0x7f6164003980 2024-03-30T00:41:55.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 -- 172.21.15.66:0/1981494941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6164003980 msgr2=0x7f6164003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 --2- 172.21.15.66:0/1981494941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6164003980 0x7f6164003da0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6170009ae0 tx=0x7f617001ca40 comp rx=0 tx=0).stop 2024-03-30T00:41:55.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f617f400640 1 -- 172.21.15.66:0/1981494941 reap_dead start 2024-03-30T00:41:55.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 -- 172.21.15.66:0/1981494941 shutdown_connections 2024-03-30T00:41:55.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 -- 172.21.15.66:0/1981494941 >> 172.21.15.66:0/1981494941 conn(0x7f6180069f60 msgr2=0x7f618006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:55.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 -- 172.21.15.66:0/1981494941 shutdown_connections 2024-03-30T00:41:55.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 -- 172.21.15.66:0/1981494941 wait complete. 2024-03-30T00:41:55.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 Processor -- start 2024-03-30T00:41:55.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 -- start start 2024-03-30T00:41:55.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6164003980 0x7f61800ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61801551f0 0x7f61800c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6180071050 0x7f61800714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6180071a10 con 0x7f61801551f0 2024-03-30T00:41:55.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61800ce150 con 0x7f6164003980 2024-03-30T00:41:55.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6185600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61800cbe80 con 0x7f6180071050 2024-03-30T00:41:55.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6176000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61801551f0 0x7f61800c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6176000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61801551f0 0x7f61800c3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46598/0 (socket says 172.21.15.66:46598) 2024-03-30T00:41:55.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6176000640 1 -- 172.21.15.66:0/1693293911 learned_addr learned my addr 172.21.15.66:0/1693293911 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:55.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f617f400640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6180071050 0x7f61800714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f617ea00640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6164003980 0x7f61800ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6176000640 1 -- 172.21.15.66:0/1693293911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6180071050 msgr2=0x7f61800714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6176000640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6180071050 0x7f61800714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6176000640 1 -- 172.21.15.66:0/1693293911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6164003980 msgr2=0x7f61800ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:55.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6176000640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6164003980 0x7f61800ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:55.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.800+0000 7f6176000640 1 -- 172.21.15.66:0/1693293911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61800cd850 con 0x7f61801551f0 2024-03-30T00:41:55.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f617f400640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6180071050 0x7f61800714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:55.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f6176000640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61801551f0 0x7f61800c3ff0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f616800d800 tx=0x7f616800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:55.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f6177400640 1 -- 172.21.15.66:0/1693293911 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680074e0 con 0x7f61801551f0 2024-03-30T00:41:55.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f6177400640 1 -- 172.21.15.66:0/1693293911 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6168007640 con 0x7f61801551f0 2024-03-30T00:41:55.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f6185600640 1 -- 172.21.15.66:0/1693293911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61800cdad0 con 0x7f61801551f0 2024-03-30T00:41:55.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f6177400640 1 -- 172.21.15.66:0/1693293911 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680049f0 con 0x7f61801551f0 2024-03-30T00:41:55.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f6185600640 1 -- 172.21.15.66:0/1693293911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61800cc0d0 con 0x7f61801551f0 2024-03-30T00:41:55.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f6185600640 1 -- 172.21.15.66:0/1693293911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6180066c30 con 0x7f61801551f0 2024-03-30T00:41:55.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f6177400640 1 -- 172.21.15.66:0/1693293911 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6168004b50 con 0x7f61801551f0 2024-03-30T00:41:55.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f6177400640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f615c066690 0x7f615c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:55.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f6177400640 1 -- 172.21.15.66:0/1693293911 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6168083ef0 con 0x7f61801551f0 2024-03-30T00:41:55.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.804+0000 7f617ea00640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f615c066690 0x7f615c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:55.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.808+0000 7f617ea00640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f615c066690 0x7f615c068b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f61700af000 tx=0x7f61700046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:55.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:55.808+0000 7f6177400640 1 -- 172.21.15.66:0/1693293911 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6168051260 con 0x7f61801551f0 2024-03-30T00:41:56.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.004+0000 7f6185600640 1 -- 172.21.15.66:0/1693293911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61800cb550 con 0x7f61801551f0 2024-03-30T00:41:56.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.004+0000 7f6177400640 1 -- 172.21.15.66:0/1693293911 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f6168019090 con 0x7f61801551f0 2024-03-30T00:41:56.015 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:56.015 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3895007156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3895007156},{"type":"v1","addr":"172.21.15.156:6835","nonce":3895007156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/65993387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":65993387},{"type":"v1","addr":"172.21.15.156:6834","nonce":65993387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2013605775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2013605775},{"type":"v1","addr":"172.21.15.66:6835","nonce":2013605775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:41:21.269965+0000","modified":"2024-03-30T00:41:24.283171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5490":{"gid":5490,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1694329026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1694329026},{"type":"v1","addr":"172.21.15.66:6837","nonce":1694329026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-30T00:41:56.016 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-03-30T00:41:56.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 -- 172.21.15.66:0/1693293911 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f615c066690 msgr2=0x7f615c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f615c066690 0x7f615c068b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f61700af000 tx=0x7f61700046d0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 -- 172.21.15.66:0/1693293911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61801551f0 msgr2=0x7f61800c3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61801551f0 0x7f61800c3ff0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f616800d800 tx=0x7f616800dcc0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 -- 172.21.15.66:0/1693293911 shutdown_connections 2024-03-30T00:41:56.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6180071050 0x7f61800714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f615c066690 0x7f615c068b50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61801551f0 0x7f61800c3ff0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 --2- 172.21.15.66:0/1693293911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6164003980 0x7f61800ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 -- 172.21.15.66:0/1693293911 >> 172.21.15.66:0/1693293911 conn(0x7f6180069f60 msgr2=0x7f61800b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:56.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 -- 172.21.15.66:0/1693293911 shutdown_connections 2024-03-30T00:41:56.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.008+0000 7f6185600640 1 -- 172.21.15.66:0/1693293911 wait complete. 2024-03-30T00:41:56.036 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:41:56.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb31f600640 1 Processor -- start 2024-03-30T00:41:56.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb31f600640 1 -- start start 2024-03-30T00:41:56.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb31f600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb3180ae4e0 0x7fb3180cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb31f600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb3180ce820 0x7fb3180cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb31f600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31806e7d0 con 0x7fb31815ca10 2024-03-30T00:41:56.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb31f600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180b1f70 con 0x7fb3180ae4e0 2024-03-30T00:41:56.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb31f600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180cf1f0 con 0x7fb3180ce820 2024-03-30T00:41:56.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb317e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb3180ae4e0 0x7fb3180cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb317e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb3180ae4e0 0x7fb3180cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46608/0 (socket says 172.21.15.66:46608) 2024-03-30T00:41:56.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb317e00640 1 -- 172.21.15.66:0/1561561241 learned_addr learned my addr 172.21.15.66:0/1561561241 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:56.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb31cc00640 1 --1- 172.21.15.66:0/1561561241 >> v1:172.21.15.66:6789/0 conn(0x7fb31815ca10 0x7fb31806ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60480/0 (socket says 172.21.15.66:60480) 2024-03-30T00:41:56.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb31d600640 1 --2- 172.21.15.66:0/1561561241 >> v2:172.21.15.156:3300/0 conn(0x7fb3180ce820 0x7fb3180cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb317e00640 1 -- 172.21.15.66:0/1561561241 >> v2:172.21.15.156:3300/0 conn(0x7fb3180ce820 msgr2=0x7fb3180cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:56.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb317e00640 1 --2- 172.21.15.66:0/1561561241 >> v2:172.21.15.156:3300/0 conn(0x7fb3180ce820 0x7fb3180cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb317e00640 1 -- 172.21.15.66:0/1561561241 >> v1:172.21.15.66:6789/0 conn(0x7fb31815ca10 legacy=0x7fb31806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb317e00640 1 -- 172.21.15.66:0/1561561241 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31815cdf0 con 0x7fb3180ae4e0 2024-03-30T00:41:56.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.168+0000 7fb317400640 1 -- 172.21.15.66:0/1561561241 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570610576 0 0) 0x7fb31806e7d0 con 0x7fb31815ca10 2024-03-30T00:41:56.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb317e00640 1 --2- 172.21.15.66:0/1561561241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3180ae4e0 0x7fb3180cb620 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb31806ab10 tx=0x7fb30401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e8f376f1ed0419 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:56.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb317400640 1 -- 172.21.15.66:0/1561561241 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3040aa650 con 0x7fb3180ae4e0 2024-03-30T00:41:56.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb317400640 1 -- 172.21.15.66:0/1561561241 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3040aa7b0 con 0x7fb3180ae4e0 2024-03-30T00:41:56.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb317400640 1 -- 172.21.15.66:0/1561561241 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3040aab40 con 0x7fb3180ae4e0 2024-03-30T00:41:56.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb31f600640 1 -- 172.21.15.66:0/1561561241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3180ae4e0 msgr2=0x7fb3180cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb31f600640 1 --2- 172.21.15.66:0/1561561241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3180ae4e0 0x7fb3180cb620 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb31806ab10 tx=0x7fb30401c4f0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb31f600640 1 -- 172.21.15.66:0/1561561241 shutdown_connections 2024-03-30T00:41:56.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb31f600640 1 --2- 172.21.15.66:0/1561561241 >> v2:172.21.15.156:3300/0 conn(0x7fb3180ce820 0x7fb3180cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb31f600640 1 --2- 172.21.15.66:0/1561561241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3180ae4e0 0x7fb3180cb620 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb31f600640 1 -- 172.21.15.66:0/1561561241 >> 172.21.15.66:0/1561561241 conn(0x7fb318098a20 msgr2=0x7fb3180b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:56.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb31f600640 1 -- 172.21.15.66:0/1561561241 shutdown_connections 2024-03-30T00:41:56.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb31f600640 1 -- 172.21.15.66:0/1561561241 wait complete. 2024-03-30T00:41:56.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb31f600640 1 Processor -- start 2024-03-30T00:41:56.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.172+0000 7fb31f600640 1 -- start start 2024-03-30T00:41:56.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31f600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3180ae4e0 0x7fb31815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31f600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3180ce820 0x7fb3180c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31f600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb31815ca10 0x7fb3180b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31f600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb318141ec0 con 0x7fb3180ce820 2024-03-30T00:41:56.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31f600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180b1470 con 0x7fb3180ae4e0 2024-03-30T00:41:56.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31f600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31815a540 con 0x7fb31815ca10 2024-03-30T00:41:56.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb317e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3180ce820 0x7fb3180c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb317e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3180ce820 0x7fb3180c22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46612/0 (socket says 172.21.15.66:46612) 2024-03-30T00:41:56.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb317e00640 1 -- 172.21.15.66:0/2058894101 learned_addr learned my addr 172.21.15.66:0/2058894101 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31cc00640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3180ae4e0 0x7fb31815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31d600640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb31815ca10 0x7fb3180b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb317e00640 1 -- 172.21.15.66:0/2058894101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb31815ca10 msgr2=0x7fb3180b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb317e00640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb31815ca10 0x7fb3180b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb317e00640 1 -- 172.21.15.66:0/2058894101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3180ae4e0 msgr2=0x7fb31815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb317e00640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3180ae4e0 0x7fb31815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb317e00640 1 -- 172.21.15.66:0/2058894101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3040094d0 con 0x7fb3180ce820 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31d600640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb31815ca10 0x7fb3180b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb317e00640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3180ce820 0x7fb3180c22a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb3040ac040 tx=0x7fb30401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb314c00640 1 -- 172.21.15.66:0/2058894101 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3040b3800 con 0x7fb3180ce820 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb314c00640 1 -- 172.21.15.66:0/2058894101 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3040b3960 con 0x7fb3180ce820 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb314c00640 1 -- 172.21.15.66:0/2058894101 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3040b3cd0 con 0x7fb3180ce820 2024-03-30T00:41:56.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31f600640 1 -- 172.21.15.66:0/2058894101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31815a820 con 0x7fb3180ce820 2024-03-30T00:41:56.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31f600640 1 -- 172.21.15.66:0/2058894101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31815ade0 con 0x7fb3180ce820 2024-03-30T00:41:56.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.176+0000 7fb31f600640 1 -- 172.21.15.66:0/2058894101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb318066c30 con 0x7fb3180ce820 2024-03-30T00:41:56.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.180+0000 7fb314c00640 1 -- 172.21.15.66:0/2058894101 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb3040b2380 con 0x7fb3180ce820 2024-03-30T00:41:56.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.180+0000 7fb314c00640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2f8066690 0x7fb2f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.180+0000 7fb314c00640 1 -- 172.21.15.66:0/2058894101 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb30411eb80 con 0x7fb3180ce820 2024-03-30T00:41:56.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.180+0000 7fb31cc00640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2f8066690 0x7fb2f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.180+0000 7fb31cc00640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2f8066690 0x7fb2f8068b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb30c000f70 tx=0x7fb30c007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:56.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.184+0000 7fb314c00640 1 -- 172.21.15.66:0/2058894101 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb3040ebe30 con 0x7fb3180ce820 2024-03-30T00:41:56.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.344+0000 7fb31f600640 1 -- 172.21.15.66:0/2058894101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3180b5430 con 0x7fb3180ce820 2024-03-30T00:41:56.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.344+0000 7fb314c00640 1 -- 172.21.15.66:0/2058894101 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16355 (secure 0 0 0) 0x7fb3040c3090 con 0x7fb3180ce820 2024-03-30T00:41:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:41:56.356 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":42,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:41:33.508872+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:41:19.292693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T00:41:20.267522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:41:56.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 -- 172.21.15.66:0/2058894101 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2f8066690 msgr2=0x7fb2f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2f8066690 0x7fb2f8068b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb30c000f70 tx=0x7fb30c007df0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 -- 172.21.15.66:0/2058894101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3180ce820 msgr2=0x7fb3180c22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3180ce820 0x7fb3180c22a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb3040ac040 tx=0x7fb30401ad30 comp rx=0 tx=0).stop 2024-03-30T00:41:56.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 -- 172.21.15.66:0/2058894101 shutdown_connections 2024-03-30T00:41:56.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb31815ca10 0x7fb3180b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2f8066690 0x7fb2f8068b50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3180ce820 0x7fb3180c22a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 --2- 172.21.15.66:0/2058894101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3180ae4e0 0x7fb31815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 -- 172.21.15.66:0/2058894101 >> 172.21.15.66:0/2058894101 conn(0x7fb318098a20 msgr2=0x7fb318158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:56.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 -- 172.21.15.66:0/2058894101 shutdown_connections 2024-03-30T00:41:56.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.348+0000 7fb31f600640 1 -- 172.21.15.66:0/2058894101 wait complete. 2024-03-30T00:41:56.378 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:41:56.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.508+0000 7fc214000640 1 Processor -- start 2024-03-30T00:41:56.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.508+0000 7fc214000640 1 -- start start 2024-03-30T00:41:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.508+0000 7fc214000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c15a640 0x7fc20c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc214000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc20c0ce520 0x7fc20c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc214000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c06d070 con 0x7fc20c0cabb0 2024-03-30T00:41:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc214000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc20c0cbba0 con 0x7fc20c0ce520 2024-03-30T00:41:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc214000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c06e2c0 con 0x7fc20c15a640 2024-03-30T00:41:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc212000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc20c0cabb0 0x7fc20c0cb000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60492/0 (socket says 172.21.15.66:60492) 2024-03-30T00:41:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc212000640 1 -- 172.21.15.66:0/1228474949 learned_addr learned my addr 172.21.15.66:0/1228474949 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc211600640 1 --2- 172.21.15.66:0/1228474949 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c15a640 0x7fc20c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc210c00640 1 --2- 172.21.15.66:0/1228474949 >> v2:172.21.15.156:3300/0 conn(0x7fc20c0ce520 0x7fc20c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc20be00640 1 -- 172.21.15.66:0/1228474949 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004908668 0 0) 0x7fc20c06d070 con 0x7fc20c0cabb0 2024-03-30T00:41:56.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc20be00640 1 -- 172.21.15.66:0/1228474949 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f4003530 con 0x7fc20c0cabb0 2024-03-30T00:41:56.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc211600640 1 -- 172.21.15.66:0/1228474949 >> v2:172.21.15.156:3300/0 conn(0x7fc20c0ce520 msgr2=0x7fc20c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:56.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc211600640 1 --2- 172.21.15.66:0/1228474949 >> v2:172.21.15.156:3300/0 conn(0x7fc20c0ce520 0x7fc20c0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc211600640 1 -- 172.21.15.66:0/1228474949 >> v1:172.21.15.66:6789/0 conn(0x7fc20c0cabb0 legacy=0x7fc20c0cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc211600640 1 -- 172.21.15.66:0/1228474949 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c0cee60 con 0x7fc20c15a640 2024-03-30T00:41:56.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.512+0000 7fc210c00640 1 --2- 172.21.15.66:0/1228474949 >> v2:172.21.15.156:3300/0 conn(0x7fc20c0ce520 0x7fc20c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:56.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc211600640 1 --2- 172.21.15.66:0/1228474949 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c15a640 0x7fc20c06dd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc2000097c0 tx=0x7fc20001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=569992a89e71fda2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:56.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc20be00640 1 -- 172.21.15.66:0/1228474949 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000ad640 con 0x7fc20c15a640 2024-03-30T00:41:56.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc20be00640 1 -- 172.21.15.66:0/1228474949 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2000ad7a0 con 0x7fc20c15a640 2024-03-30T00:41:56.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc20be00640 1 -- 172.21.15.66:0/1228474949 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000adb30 con 0x7fc20c15a640 2024-03-30T00:41:56.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc214000640 1 -- 172.21.15.66:0/1228474949 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c15a640 msgr2=0x7fc20c06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc214000640 1 --2- 172.21.15.66:0/1228474949 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c15a640 0x7fc20c06dd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc2000097c0 tx=0x7fc20001c4a0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc214000640 1 -- 172.21.15.66:0/1228474949 shutdown_connections 2024-03-30T00:41:56.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc214000640 1 --2- 172.21.15.66:0/1228474949 >> v2:172.21.15.156:3300/0 conn(0x7fc20c0ce520 0x7fc20c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc214000640 1 --2- 172.21.15.66:0/1228474949 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c15a640 0x7fc20c06dd80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc214000640 1 -- 172.21.15.66:0/1228474949 >> 172.21.15.66:0/1228474949 conn(0x7fc20c069f60 msgr2=0x7fc20c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:56.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc214000640 1 -- 172.21.15.66:0/1228474949 shutdown_connections 2024-03-30T00:41:56.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.516+0000 7fc214000640 1 -- 172.21.15.66:0/1228474949 wait complete. 2024-03-30T00:41:56.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc214000640 1 Processor -- start 2024-03-30T00:41:56.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc214000640 1 -- start start 2024-03-30T00:41:56.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc214000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c0cabb0 0x7fc20c0ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc214000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc20c0ce520 0x7fc20c0c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc214000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c15a640 0x7fc20c0ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc214000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0c9740 con 0x7fc20c0ce520 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc214000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0c87b0 con 0x7fc20c0cabb0 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc214000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0cd4f0 con 0x7fc20c15a640 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc210c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc20c0ce520 0x7fc20c0c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc210c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc20c0ce520 0x7fc20c0c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46624/0 (socket says 172.21.15.66:46624) 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc210c00640 1 -- 172.21.15.66:0/244899214 learned_addr learned my addr 172.21.15.66:0/244899214 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc212000640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c15a640 0x7fc20c0ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc211600640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c0cabb0 0x7fc20c0ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc210c00640 1 -- 172.21.15.66:0/244899214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c15a640 msgr2=0x7fc20c0ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc210c00640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c15a640 0x7fc20c0ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc210c00640 1 -- 172.21.15.66:0/244899214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c0cabb0 msgr2=0x7fc20c0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc210c00640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c0cabb0 0x7fc20c0ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc210c00640 1 -- 172.21.15.66:0/244899214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f8009420 con 0x7fc20c0ce520 2024-03-30T00:41:56.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc210c00640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc20c0ce520 0x7fc20c0c9200 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800ebe0 tx=0x7fc1f800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:56.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc209600640 1 -- 172.21.15.66:0/244899214 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800eee0 con 0x7fc20c0ce520 2024-03-30T00:41:56.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc209600640 1 -- 172.21.15.66:0/244899214 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f800cc30 con 0x7fc20c0ce520 2024-03-30T00:41:56.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc209600640 1 -- 172.21.15.66:0/244899214 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8013470 con 0x7fc20c0ce520 2024-03-30T00:41:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc214000640 1 -- 172.21.15.66:0/244899214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc200009420 con 0x7fc20c0ce520 2024-03-30T00:41:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc214000640 1 -- 172.21.15.66:0/244899214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20c0c8a00 con 0x7fc20c0ce520 2024-03-30T00:41:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.520+0000 7fc214000640 1 -- 172.21.15.66:0/244899214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20c066c30 con 0x7fc20c0ce520 2024-03-30T00:41:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.524+0000 7fc209600640 1 -- 172.21.15.66:0/244899214 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc1f80135d0 con 0x7fc20c0ce520 2024-03-30T00:41:56.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.524+0000 7fc209600640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc1ec0666c0 0x7fc1ec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.524+0000 7fc209600640 1 -- 172.21.15.66:0/244899214 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1f8084560 con 0x7fc20c0ce520 2024-03-30T00:41:56.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.524+0000 7fc211600640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc1ec0666c0 0x7fc1ec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.524+0000 7fc211600640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc1ec0666c0 0x7fc1ec068b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc2000af000 tx=0x7fc2000a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:56.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.528+0000 7fc209600640 1 -- 172.21.15.66:0/244899214 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc1f8051770 con 0x7fc20c0ce520 2024-03-30T00:41:56.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.724+0000 7fc214000640 1 -- 172.21.15.66:0/244899214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc20c06dd20 con 0x7fc20c0ce520 2024-03-30T00:41:56.735 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:41:56.724+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:41:56.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.732+0000 7fc209600640 1 -- 172.21.15.66:0/244899214 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc1f8054d90 con 0x7fc20c0ce520 2024-03-30T00:41:56.744 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:41:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 -- 172.21.15.66:0/244899214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc1ec0666c0 msgr2=0x7fc1ec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc1ec0666c0 0x7fc1ec068b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc2000af000 tx=0x7fc2000a7070 comp rx=0 tx=0).stop 2024-03-30T00:41:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 -- 172.21.15.66:0/244899214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc20c0ce520 msgr2=0x7fc20c0c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc20c0ce520 0x7fc20c0c9200 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800ebe0 tx=0x7fc1f800c520 comp rx=0 tx=0).stop 2024-03-30T00:41:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 -- 172.21.15.66:0/244899214 shutdown_connections 2024-03-30T00:41:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c15a640 0x7fc20c0ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc1ec0666c0 0x7fc1ec068b80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc20c0ce520 0x7fc20c0c9200 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 --2- 172.21.15.66:0/244899214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c0cabb0 0x7fc20c0ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 -- 172.21.15.66:0/244899214 >> 172.21.15.66:0/244899214 conn(0x7fc20c069f60 msgr2=0x7fc20c157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 -- 172.21.15.66:0/244899214 shutdown_connections 2024-03-30T00:41:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.736+0000 7fc214000640 1 -- 172.21.15.66:0/244899214 wait complete. 2024-03-30T00:41:56.760 INFO:tasks.ceph.mds.c.smithi066.stderr: -5857> 2024-03-30T00:41:24.845+0000 7f438f400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:41:56.774 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T00:41:56.814 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:41:56.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.908+0000 7f748f200640 1 Processor -- start 2024-03-30T00:41:56.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.908+0000 7f748f200640 1 -- start start 2024-03-30T00:41:56.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.908+0000 7f748f200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7490072650 0x7f749015ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.908+0000 7f748f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749015ab80 con 0x7f7490072ad0 2024-03-30T00:41:56.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.908+0000 7f748f200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749006e140 con 0x7f749006c7f0 2024-03-30T00:41:56.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.908+0000 7f748f200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749015a5c0 con 0x7f7490072650 2024-03-30T00:41:56.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.908+0000 7f748e800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7490072ad0 0x7f749006e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60510/0 (socket says 172.21.15.66:60510) 2024-03-30T00:41:56.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.908+0000 7f748e800640 1 -- 172.21.15.66:0/2801439659 learned_addr learned my addr 172.21.15.66:0/2801439659 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:56.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.908+0000 7f7487e00640 1 --2- 172.21.15.66:0/2801439659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7490072650 0x7f749015ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.912+0000 7f748d400640 1 -- 172.21.15.66:0/2801439659 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949040024 0 0) 0x7f749015ab80 con 0x7f7490072ad0 2024-03-30T00:41:56.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.912+0000 7f748d400640 1 -- 172.21.15.66:0/2801439659 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7474003530 con 0x7f7490072ad0 2024-03-30T00:41:56.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.912+0000 7f748d400640 1 -- 172.21.15.66:0/2801439659 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789144666 0 0) 0x7f749006e140 con 0x7f749006c7f0 2024-03-30T00:41:56.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.912+0000 7f7487e00640 1 -- 172.21.15.66:0/2801439659 >> v1:172.21.15.156:6789/0 conn(0x7f749006c7f0 legacy=0x7f749006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.912+0000 7f7487e00640 1 -- 172.21.15.66:0/2801439659 >> v1:172.21.15.66:6789/0 conn(0x7f7490072ad0 legacy=0x7f749006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.912+0000 7f7487e00640 1 -- 172.21.15.66:0/2801439659 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749015a8a0 con 0x7f7490072650 2024-03-30T00:41:56.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.912+0000 7f7487e00640 1 --2- 172.21.15.66:0/2801439659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7490072650 0x7f749015ae10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f74780097f0 tx=0x7f747801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c777180e99bfbcc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.912+0000 7f748d400640 1 -- 172.21.15.66:0/2801439659 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780aa430 con 0x7f7490072650 2024-03-30T00:41:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.912+0000 7f748d400640 1 -- 172.21.15.66:0/2801439659 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74780aa590 con 0x7f7490072650 2024-03-30T00:41:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.912+0000 7f748d400640 1 -- 172.21.15.66:0/2801439659 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780aa900 con 0x7f7490072650 2024-03-30T00:41:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 -- 172.21.15.66:0/2801439659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7490072650 msgr2=0x7f749015ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 --2- 172.21.15.66:0/2801439659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7490072650 0x7f749015ae10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f74780097f0 tx=0x7f747801c4b0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 -- 172.21.15.66:0/2801439659 shutdown_connections 2024-03-30T00:41:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 --2- 172.21.15.66:0/2801439659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7490072650 0x7f749015ae10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 -- 172.21.15.66:0/2801439659 >> 172.21.15.66:0/2801439659 conn(0x7f74900cea10 msgr2=0x7f74900cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 -- 172.21.15.66:0/2801439659 shutdown_connections 2024-03-30T00:41:56.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 -- 172.21.15.66:0/2801439659 wait complete. 2024-03-30T00:41:56.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 Processor -- start 2024-03-30T00:41:56.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 -- start start 2024-03-30T00:41:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f749006c7f0 0x7f74900ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7490072650 0x7f74901499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7490072ad0 0x7f749014a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7490149ef0 con 0x7f7490072ad0 2024-03-30T00:41:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7490157d20 con 0x7f7490072650 2024-03-30T00:41:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748f200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7490157110 con 0x7f749006c7f0 2024-03-30T00:41:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7490072ad0 0x7f749014a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7490072ad0 0x7f749014a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46668/0 (socket says 172.21.15.66:46668) 2024-03-30T00:41:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748e800640 1 -- 172.21.15.66:0/3468201306 learned_addr learned my addr 172.21.15.66:0/3468201306 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:56.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f7487e00640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7490072650 0x7f74901499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748de00640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f749006c7f0 0x7f74900ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748e800640 1 -- 172.21.15.66:0/3468201306 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f749006c7f0 msgr2=0x7f74900ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748e800640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f749006c7f0 0x7f74900ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748e800640 1 -- 172.21.15.66:0/3468201306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7490072650 msgr2=0x7f74901499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748e800640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7490072650 0x7f74901499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f748e800640 1 -- 172.21.15.66:0/3468201306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7478009420 con 0x7f7490072ad0 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.916+0000 7f7487e00640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7490072650 0x7f74901499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f748e800640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7490072ad0 0x7f749014a5d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f748800cad0 tx=0x7f748800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f7486000640 1 -- 172.21.15.66:0/3468201306 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748800ac30 con 0x7f7490072ad0 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f7486000640 1 -- 172.21.15.66:0/3468201306 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748800ad90 con 0x7f7490072ad0 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f7486000640 1 -- 172.21.15.66:0/3468201306 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488012620 con 0x7f7490072ad0 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f748f200640 1 -- 172.21.15.66:0/3468201306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74901573f0 con 0x7f7490072ad0 2024-03-30T00:41:56.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f748f200640 1 -- 172.21.15.66:0/3468201306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74901579b0 con 0x7f7490072ad0 2024-03-30T00:41:56.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f748f200640 1 -- 172.21.15.66:0/3468201306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7490157eb0 con 0x7f7490072ad0 2024-03-30T00:41:56.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f7486000640 1 -- 172.21.15.66:0/3468201306 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7488007070 con 0x7f7490072ad0 2024-03-30T00:41:56.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f7486000640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f746c066690 0x7f746c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:56.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f7486000640 1 -- 172.21.15.66:0/3468201306 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f74880843a0 con 0x7f7490072ad0 2024-03-30T00:41:56.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f748de00640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f746c066690 0x7f746c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:56.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.920+0000 7f748de00640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f746c066690 0x7f746c068b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7480000f70 tx=0x7f7480009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:56.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:56.924+0000 7f7486000640 1 -- 172.21.15.66:0/3468201306 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f74880514e0 con 0x7f7490072ad0 2024-03-30T00:41:57.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.120+0000 7f748f200640 1 -- 172.21.15.66:0/3468201306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7490073e00 con 0x7f7490072ad0 2024-03-30T00:41:57.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f7486000640 1 -- 172.21.15.66:0/3468201306 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7488054b00 con 0x7f7490072ad0 2024-03-30T00:41:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 -- 172.21.15.66:0/3468201306 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f746c066690 msgr2=0x7f746c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f746c066690 0x7f746c068b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7480000f70 tx=0x7f7480009040 comp rx=0 tx=0).stop 2024-03-30T00:41:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 -- 172.21.15.66:0/3468201306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7490072ad0 msgr2=0x7f749014a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7490072ad0 0x7f749014a5d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f748800cad0 tx=0x7f748800cf90 comp rx=0 tx=0).stop 2024-03-30T00:41:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 -- 172.21.15.66:0/3468201306 shutdown_connections 2024-03-30T00:41:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f746c066690 0x7f746c068b50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7490072ad0 0x7f749014a5d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7490072650 0x7f74901499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 --2- 172.21.15.66:0/3468201306 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f749006c7f0 0x7f74900ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 -- 172.21.15.66:0/3468201306 >> 172.21.15.66:0/3468201306 conn(0x7f74900cea10 msgr2=0x7f7490152d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 -- 172.21.15.66:0/3468201306 shutdown_connections 2024-03-30T00:41:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.740+0000 7f748f200640 1 -- 172.21.15.66:0/3468201306 wait complete. 2024-03-30T00:41:57.774 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T00:41:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f8335c00640 1 Processor -- start 2024-03-30T00:41:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f8335c00640 1 -- start start 2024-03-30T00:41:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f8335c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f833015a410 0x7f83300b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f8335c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83300cdf00 con 0x7f83300ca980 2024-03-30T00:41:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f8335c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83300cd5a0 con 0x7f83300cedf0 2024-03-30T00:41:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f8335c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833015e370 con 0x7f833015a410 2024-03-30T00:41:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f832f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f83300ca980 0x7f83300ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60526/0 (socket says 172.21.15.66:60526) 2024-03-30T00:41:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f832f400640 1 -- 172.21.15.66:0/1214460856 learned_addr learned my addr 172.21.15.66:0/1214460856 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f832fe00640 1 --2- 172.21.15.66:0/1214460856 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f833015a410 0x7f83300b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f832ea00640 1 -- 172.21.15.66:0/1214460856 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249789002 0 0) 0x7f83300cdf00 con 0x7f83300ca980 2024-03-30T00:41:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f832ea00640 1 -- 172.21.15.66:0/1214460856 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8314003530 con 0x7f83300ca980 2024-03-30T00:41:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f832ea00640 1 -- 172.21.15.66:0/1214460856 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053781736 0 0) 0x7f83300cd5a0 con 0x7f83300cedf0 2024-03-30T00:41:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f832ea00640 1 -- 172.21.15.66:0/1214460856 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83300cdf00 con 0x7f83300cedf0 2024-03-30T00:41:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f832ea00640 1 -- 172.21.15.66:0/1214460856 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 570883283 0 0) 0x7f8314003530 con 0x7f83300ca980 2024-03-30T00:41:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f832fe00640 1 -- 172.21.15.66:0/1214460856 >> v1:172.21.15.156:6789/0 conn(0x7f83300cedf0 legacy=0x7f83300cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f832fe00640 1 -- 172.21.15.66:0/1214460856 >> v1:172.21.15.66:6789/0 conn(0x7f83300ca980 legacy=0x7f83300ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.956+0000 7f832fe00640 1 -- 172.21.15.66:0/1214460856 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83300cd880 con 0x7f833015a410 2024-03-30T00:41:57.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.960+0000 7f832fe00640 1 --2- 172.21.15.66:0/1214460856 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f833015a410 0x7f83300b3180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f83200097f0 tx=0x7f832001c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4d53d82c420457d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:57.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.960+0000 7f832ea00640 1 -- 172.21.15.66:0/1214460856 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c001680 con 0x7f833015a410 2024-03-30T00:41:57.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.960+0000 7f832ea00640 1 -- 172.21.15.66:0/1214460856 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83200a7050 con 0x7f833015a410 2024-03-30T00:41:57.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.960+0000 7f832ea00640 1 -- 172.21.15.66:0/1214460856 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83200ab620 con 0x7f833015a410 2024-03-30T00:41:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.960+0000 7f8335c00640 1 -- 172.21.15.66:0/1214460856 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f833015a410 msgr2=0x7f83300b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.960+0000 7f8335c00640 1 --2- 172.21.15.66:0/1214460856 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f833015a410 0x7f83300b3180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f83200097f0 tx=0x7f832001c3d0 comp rx=0 tx=0).stop 2024-03-30T00:41:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.960+0000 7f8335c00640 1 -- 172.21.15.66:0/1214460856 shutdown_connections 2024-03-30T00:41:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.960+0000 7f8335c00640 1 --2- 172.21.15.66:0/1214460856 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f833015a410 0x7f83300b3180 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.960+0000 7f8335c00640 1 -- 172.21.15.66:0/1214460856 >> 172.21.15.66:0/1214460856 conn(0x7f83300b1e80 msgr2=0x7f83300b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:57.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.960+0000 7f8335c00640 1 -- 172.21.15.66:0/1214460856 shutdown_connections 2024-03-30T00:41:57.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.960+0000 7f8335c00640 1 -- 172.21.15.66:0/1214460856 wait complete. 2024-03-30T00:41:57.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335c00640 1 Processor -- start 2024-03-30T00:41:57.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335c00640 1 -- start start 2024-03-30T00:41:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83300ca980 0x7f8330164c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83300cedf0 0x7f8330165530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015a410 0x7f8330165a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833006cbf0 con 0x7f833015a410 2024-03-30T00:41:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83301537c0 con 0x7f83300ca980 2024-03-30T00:41:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833006aea0 con 0x7f83300cedf0 2024-03-30T00:41:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015a410 0x7f8330165a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015a410 0x7f8330165a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46672/0 (socket says 172.21.15.66:46672) 2024-03-30T00:41:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335200640 1 -- 172.21.15.66:0/3415976376 learned_addr learned my addr 172.21.15.66:0/3415976376 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f832fe00640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83300ca980 0x7f8330164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f832f400640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83300cedf0 0x7f8330165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335200640 1 -- 172.21.15.66:0/3415976376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83300cedf0 msgr2=0x7f8330165530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335200640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83300cedf0 0x7f8330165530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335200640 1 -- 172.21.15.66:0/3415976376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83300ca980 msgr2=0x7f8330164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335200640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83300ca980 0x7f8330164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335200640 1 -- 172.21.15.66:0/3415976376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8320009420 con 0x7f833015a410 2024-03-30T00:41:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f832f400640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83300cedf0 0x7f8330165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f832fe00640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83300ca980 0x7f8330164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:41:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335200640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015a410 0x7f8330165a70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f832800c7c0 tx=0x7f832800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8327e00640 1 -- 172.21.15.66:0/3415976376 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328003b80 con 0x7f833015a410 2024-03-30T00:41:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335c00640 1 -- 172.21.15.66:0/3415976376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8330153490 con 0x7f833015a410 2024-03-30T00:41:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8327e00640 1 -- 172.21.15.66:0/3415976376 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328003ce0 con 0x7f833015a410 2024-03-30T00:41:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8327e00640 1 -- 172.21.15.66:0/3415976376 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832800aa50 con 0x7f833015a410 2024-03-30T00:41:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335c00640 1 -- 172.21.15.66:0/3415976376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8330151930 con 0x7f833015a410 2024-03-30T00:41:57.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.964+0000 7f8335c00640 1 -- 172.21.15.66:0/3415976376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8330067150 con 0x7f833015a410 2024-03-30T00:41:57.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.968+0000 7f8327e00640 1 -- 172.21.15.66:0/3415976376 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f832800abb0 con 0x7f833015a410 2024-03-30T00:41:57.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.968+0000 7f8327e00640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8310066690 0x7f8310068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:57.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.968+0000 7f8327e00640 1 -- 172.21.15.66:0/3415976376 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f8328084330 con 0x7f833015a410 2024-03-30T00:41:57.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.968+0000 7f832fe00640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8310066690 0x7f8310068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:57.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.968+0000 7f832fe00640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8310066690 0x7f8310068b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f83200ad000 tx=0x7f8320008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:57.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:57.972+0000 7f8327e00640 1 -- 172.21.15.66:0/3415976376 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f83280513f0 con 0x7f833015a410 2024-03-30T00:41:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.140+0000 7f8335c00640 1 -- 172.21.15.66:0/3415976376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8330154db0 con 0x7f833015a410 2024-03-30T00:41:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.740+0000 7f8327e00640 1 -- 172.21.15.66:0/3415976376 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7f832800ae60 con 0x7f833015a410 2024-03-30T00:41:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-03-30T00:41:58.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 -- 172.21.15.66:0/3415976376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8310066690 msgr2=0x7f8310068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:58.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8310066690 0x7f8310068b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f83200ad000 tx=0x7f8320008040 comp rx=0 tx=0).stop 2024-03-30T00:41:58.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 -- 172.21.15.66:0/3415976376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015a410 msgr2=0x7f8330165a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015a410 0x7f8330165a70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f832800c7c0 tx=0x7f832800cc80 comp rx=0 tx=0).stop 2024-03-30T00:41:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 -- 172.21.15.66:0/3415976376 shutdown_connections 2024-03-30T00:41:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8310066690 0x7f8310068b50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015a410 0x7f8330165a70 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83300cedf0 0x7f8330165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 --2- 172.21.15.66:0/3415976376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83300ca980 0x7f8330164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 -- 172.21.15.66:0/3415976376 >> 172.21.15.66:0/3415976376 conn(0x7f83300b1e80 msgr2=0x7f8330155e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 -- 172.21.15.66:0/3415976376 shutdown_connections 2024-03-30T00:41:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.744+0000 7f8335c00640 1 -- 172.21.15.66:0/3415976376 wait complete. 2024-03-30T00:41:58.779 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T00:41:58.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.916+0000 7fae6a800640 1 Processor -- start 2024-03-30T00:41:58.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.916+0000 7fae6a800640 1 -- start start 2024-03-30T00:41:58.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae6a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae640acaf0 0x7fae640b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:58.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae6a800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fae6415ba00 0x7fae640ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:58.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae6a800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae640cbf80 con 0x7fae640cecf0 2024-03-30T00:41:58.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae6a800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae640cc820 con 0x7fae640acaf0 2024-03-30T00:41:58.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae6a800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae640cf110 con 0x7fae6415ba00 2024-03-30T00:41:58.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae63400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae640acaf0 0x7fae640b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:58.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae63400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae640acaf0 0x7fae640b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35216/0 (socket says 172.21.15.66:35216) 2024-03-30T00:41:58.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae63400640 1 -- 172.21.15.66:0/1644054680 learned_addr learned my addr 172.21.15.66:0/1644054680 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:58.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae63400640 1 -- 172.21.15.66:0/1644054680 >> v2:172.21.15.156:3300/0 conn(0x7fae6415ba00 msgr2=0x7fae640ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:58.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae63400640 1 --2- 172.21.15.66:0/1644054680 >> v2:172.21.15.156:3300/0 conn(0x7fae6415ba00 0x7fae640ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:58.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae63400640 1 -- 172.21.15.66:0/1644054680 >> v1:172.21.15.156:6789/0 conn(0x7fae640cecf0 legacy=0x7fae641579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:58.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.920+0000 7fae63400640 1 -- 172.21.15.66:0/1644054680 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae640b2dd0 con 0x7fae640acaf0 2024-03-30T00:41:58.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae63400640 1 --2- 172.21.15.66:0/1644054680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae640acaf0 0x7fae640b24d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fae640c1e10 tx=0x7fae5401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7480d92b9c61c8d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:58.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae62a00640 1 -- 172.21.15.66:0/1644054680 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540ab620 con 0x7fae640acaf0 2024-03-30T00:41:58.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae62a00640 1 -- 172.21.15.66:0/1644054680 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae540ab780 con 0x7fae640acaf0 2024-03-30T00:41:58.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae62a00640 1 -- 172.21.15.66:0/1644054680 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540abb10 con 0x7fae640acaf0 2024-03-30T00:41:58.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae6a800640 1 -- 172.21.15.66:0/1644054680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae640acaf0 msgr2=0x7fae640b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:58.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae6a800640 1 --2- 172.21.15.66:0/1644054680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae640acaf0 0x7fae640b24d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fae640c1e10 tx=0x7fae5401c420 comp rx=0 tx=0).stop 2024-03-30T00:41:58.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae6a800640 1 -- 172.21.15.66:0/1644054680 shutdown_connections 2024-03-30T00:41:58.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae6a800640 1 --2- 172.21.15.66:0/1644054680 >> v2:172.21.15.156:3300/0 conn(0x7fae6415ba00 0x7fae640ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:58.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae6a800640 1 --2- 172.21.15.66:0/1644054680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae640acaf0 0x7fae640b24d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:58.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae6a800640 1 -- 172.21.15.66:0/1644054680 >> 172.21.15.66:0/1644054680 conn(0x7fae6406a190 msgr2=0x7fae6406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:58.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae6a800640 1 -- 172.21.15.66:0/1644054680 shutdown_connections 2024-03-30T00:41:58.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae6a800640 1 -- 172.21.15.66:0/1644054680 wait complete. 2024-03-30T00:41:58.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae6a800640 1 Processor -- start 2024-03-30T00:41:58.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.924+0000 7fae6a800640 1 -- start start 2024-03-30T00:41:58.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae6a800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae640acaf0 0x7fae640978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:58.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae6a800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae640cecf0 0x7fae64153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:58.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae6a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6415ba00 0x7fae640cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae6a800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6406dd80 con 0x7fae6415ba00 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae6a800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6406cce0 con 0x7fae640cecf0 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae6a800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6406cfc0 con 0x7fae640acaf0 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae63e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6415ba00 0x7fae640cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae63e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6415ba00 0x7fae640cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35232/0 (socket says 172.21.15.66:35232) 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae63e00640 1 -- 172.21.15.66:0/1404497942 learned_addr learned my addr 172.21.15.66:0/1404497942 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae63400640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae640acaf0 0x7fae640978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae5be00640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae640cecf0 0x7fae64153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae63e00640 1 -- 172.21.15.66:0/1404497942 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae640acaf0 msgr2=0x7fae640978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae63e00640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae640acaf0 0x7fae640978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae63e00640 1 -- 172.21.15.66:0/1404497942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae640cecf0 msgr2=0x7fae64153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae63e00640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae640cecf0 0x7fae64153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae63e00640 1 -- 172.21.15.66:0/1404497942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae540094d0 con 0x7fae6415ba00 2024-03-30T00:41:58.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae5be00640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae640cecf0 0x7fae64153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:58.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae63e00640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6415ba00 0x7fae640cad60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00b990 tx=0x7fae5c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:58.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae5b400640 1 -- 172.21.15.66:0/1404497942 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c00cba0 con 0x7fae6415ba00 2024-03-30T00:41:58.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae5b400640 1 -- 172.21.15.66:0/1404497942 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c00cd00 con 0x7fae6415ba00 2024-03-30T00:41:58.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae5b400640 1 -- 172.21.15.66:0/1404497942 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c014680 con 0x7fae6415ba00 2024-03-30T00:41:58.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae6a800640 1 -- 172.21.15.66:0/1404497942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6406d2a0 con 0x7fae6415ba00 2024-03-30T00:41:58.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae6a800640 1 -- 172.21.15.66:0/1404497942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6406d860 con 0x7fae6415ba00 2024-03-30T00:41:58.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae6a800640 1 -- 172.21.15.66:0/1404497942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6406da80 con 0x7fae6415ba00 2024-03-30T00:41:58.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.928+0000 7fae5b400640 1 -- 172.21.15.66:0/1404497942 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fae5c01d3b0 con 0x7fae6415ba00 2024-03-30T00:41:58.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.932+0000 7fae5b400640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae40066690 0x7fae40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:58.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.932+0000 7fae5b400640 1 -- 172.21.15.66:0/1404497942 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fae5c0841a0 con 0x7fae6415ba00 2024-03-30T00:41:58.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.932+0000 7fae63400640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae40066690 0x7fae40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:58.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.932+0000 7fae63400640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae40066690 0x7fae40068b50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fae540af000 tx=0x7fae540a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:58.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:58.936+0000 7fae5b400640 1 -- 172.21.15.66:0/1404497942 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fae5c051470 con 0x7fae6415ba00 2024-03-30T00:41:59.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.096+0000 7fae6a800640 1 -- 172.21.15.66:0/1404497942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fae64066c30 con 0x7fae6415ba00 2024-03-30T00:41:59.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.748+0000 7fae5b400640 1 -- 172.21.15.66:0/1404497942 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7fae5c054a90 con 0x7fae6415ba00 2024-03-30T00:41:59.758 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-03-30T00:41:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 -- 172.21.15.66:0/1404497942 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae40066690 msgr2=0x7fae40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae40066690 0x7fae40068b50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fae540af000 tx=0x7fae540a7000 comp rx=0 tx=0).stop 2024-03-30T00:41:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 -- 172.21.15.66:0/1404497942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6415ba00 msgr2=0x7fae640cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6415ba00 0x7fae640cad60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00b990 tx=0x7fae5c00be50 comp rx=0 tx=0).stop 2024-03-30T00:41:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 -- 172.21.15.66:0/1404497942 shutdown_connections 2024-03-30T00:41:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fae40066690 0x7fae40068b50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6415ba00 0x7fae640cad60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae640cecf0 0x7fae64153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 --2- 172.21.15.66:0/1404497942 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae640acaf0 0x7fae640978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 -- 172.21.15.66:0/1404497942 >> 172.21.15.66:0/1404497942 conn(0x7fae6406a190 msgr2=0x7fae64150830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 -- 172.21.15.66:0/1404497942 shutdown_connections 2024-03-30T00:41:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.752+0000 7fae6a800640 1 -- 172.21.15.66:0/1404497942 wait complete. 2024-03-30T00:41:59.787 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-30T00:41:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81c5400640 1 Processor -- start 2024-03-30T00:41:59.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81c5400640 1 -- start start 2024-03-30T00:41:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81c5400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f81c00ceb60 0x7f81c015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81c5400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c015d360 0x7f81c00cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81c5400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c00cbec0 con 0x7f81c00ce780 2024-03-30T00:41:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81c5400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c015d970 con 0x7f81c00ceb60 2024-03-30T00:41:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81c5400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c015dc50 con 0x7f81c015d360 2024-03-30T00:41:59.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81be800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f81c00ce780 0x7f81c00cc910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36294/0 (socket says 172.21.15.66:36294) 2024-03-30T00:41:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81be800640 1 -- 172.21.15.66:0/1879344207 learned_addr learned my addr 172.21.15.66:0/1879344207 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:41:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81b7e00640 1 --2- 172.21.15.66:0/1879344207 >> v2:172.21.15.156:3300/0 conn(0x7f81c00ceb60 0x7f81c015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81bf200640 1 --2- 172.21.15.66:0/1879344207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c015d360 0x7f81c00cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:59.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81bde00640 1 -- 172.21.15.66:0/1879344207 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128781904 0 0) 0x7f81c00cbec0 con 0x7f81c00ce780 2024-03-30T00:41:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81bde00640 1 -- 172.21.15.66:0/1879344207 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a4003530 con 0x7f81c00ce780 2024-03-30T00:41:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81b7e00640 1 -- 172.21.15.66:0/1879344207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c015d360 msgr2=0x7f81c00cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:41:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81b7e00640 1 --2- 172.21.15.66:0/1879344207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c015d360 0x7f81c00cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81b7e00640 1 -- 172.21.15.66:0/1879344207 >> v1:172.21.15.66:6789/0 conn(0x7f81c00ce780 legacy=0x7f81c00cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81b7e00640 1 -- 172.21.15.66:0/1879344207 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c00cc110 con 0x7f81c00ceb60 2024-03-30T00:41:59.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.928+0000 7f81bf200640 1 --2- 172.21.15.66:0/1879344207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c015d360 0x7f81c00cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:41:59.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81b7e00640 1 --2- 172.21.15.66:0/1879344207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81c00ceb60 0x7f81c015ce20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f81a80097c0 tx=0x7f81a801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d0a6713b7c16739 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:59.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81bde00640 1 -- 172.21.15.66:0/1879344207 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a80ab600 con 0x7f81c00ceb60 2024-03-30T00:41:59.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81bde00640 1 -- 172.21.15.66:0/1879344207 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a80a7050 con 0x7f81c00ceb60 2024-03-30T00:41:59.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81bde00640 1 -- 172.21.15.66:0/1879344207 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a80ab980 con 0x7f81c00ceb60 2024-03-30T00:41:59.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81c5400640 1 -- 172.21.15.66:0/1879344207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81c00ceb60 msgr2=0x7f81c015ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:59.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81c5400640 1 --2- 172.21.15.66:0/1879344207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81c00ceb60 0x7f81c015ce20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f81a80097c0 tx=0x7f81a801c3e0 comp rx=0 tx=0).stop 2024-03-30T00:41:59.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81c5400640 1 -- 172.21.15.66:0/1879344207 shutdown_connections 2024-03-30T00:41:59.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81c5400640 1 --2- 172.21.15.66:0/1879344207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c015d360 0x7f81c00cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:59.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81c5400640 1 --2- 172.21.15.66:0/1879344207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81c00ceb60 0x7f81c015ce20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:59.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81c5400640 1 -- 172.21.15.66:0/1879344207 >> 172.21.15.66:0/1879344207 conn(0x7f81c0069f60 msgr2=0x7f81c006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:41:59.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81c5400640 1 -- 172.21.15.66:0/1879344207 shutdown_connections 2024-03-30T00:41:59.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.932+0000 7f81c5400640 1 -- 172.21.15.66:0/1879344207 wait complete. 2024-03-30T00:41:59.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81c5400640 1 Processor -- start 2024-03-30T00:41:59.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81c5400640 1 -- start start 2024-03-30T00:41:59.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81c5400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81c00ce780 0x7f81c006fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81c5400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c00ceb60 0x7f81c00c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81c5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81c015d360 0x7f81c0156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81c5400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00c53c0 con 0x7f81c015d360 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81c5400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00704b0 con 0x7f81c00ce780 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81c5400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c0070790 con 0x7f81c00ceb60 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81bf200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81c015d360 0x7f81c0156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81bf200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81c015d360 0x7f81c0156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35246/0 (socket says 172.21.15.66:35246) 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81bf200640 1 -- 172.21.15.66:0/3819918697 learned_addr learned my addr 172.21.15.66:0/3819918697 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81bf200640 1 -- 172.21.15.66:0/3819918697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c00ceb60 msgr2=0x7f81c00c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81b7e00640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c00ceb60 0x7f81c00c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81be800640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81c00ce780 0x7f81c006fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81bf200640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c00ceb60 0x7f81c00c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81bf200640 1 -- 172.21.15.66:0/3819918697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81c00ce780 msgr2=0x7f81c006fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:41:59.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81bf200640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81c00ce780 0x7f81c006fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:41:59.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81bf200640 1 -- 172.21.15.66:0/3819918697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b8009a70 con 0x7f81c015d360 2024-03-30T00:41:59.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81b7e00640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c00ceb60 0x7f81c00c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:59.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81be800640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81c00ce780 0x7f81c006fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:41:59.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81bf200640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81c015d360 0x7f81c0156e30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f81b800efc0 tx=0x7f81b8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:59.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81b6a00640 1 -- 172.21.15.66:0/3819918697 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b8009d70 con 0x7f81c015d360 2024-03-30T00:41:59.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81b6a00640 1 -- 172.21.15.66:0/3819918697 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b800f040 con 0x7f81c015d360 2024-03-30T00:41:59.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81c5400640 1 -- 172.21.15.66:0/3819918697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a8009420 con 0x7f81c015d360 2024-03-30T00:41:59.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81b6a00640 1 -- 172.21.15.66:0/3819918697 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b80039a0 con 0x7f81c015d360 2024-03-30T00:41:59.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81c5400640 1 -- 172.21.15.66:0/3819918697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c0070d50 con 0x7f81c015d360 2024-03-30T00:41:59.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.936+0000 7f81c5400640 1 -- 172.21.15.66:0/3819918697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c00c5570 con 0x7f81c015d360 2024-03-30T00:41:59.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.940+0000 7f81b6a00640 1 -- 172.21.15.66:0/3819918697 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f81b80144a0 con 0x7f81c015d360 2024-03-30T00:41:59.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.940+0000 7f81b6a00640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f819c066690 0x7f819c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:41:59.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.940+0000 7f81b6a00640 1 -- 172.21.15.66:0/3819918697 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f81b8084c90 con 0x7f81c015d360 2024-03-30T00:41:59.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.940+0000 7f81be800640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f819c066690 0x7f819c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:41:59.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.940+0000 7f81be800640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f819c066690 0x7f819c068b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f81b00011f0 tx=0x7f81b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:41:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:41:59.944+0000 7f81b6a00640 1 -- 172.21.15.66:0/3819918697 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f81c00c5570 con 0x7f81c015d360 2024-03-30T00:42:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.104+0000 7f81c5400640 1 -- 172.21.15.66:0/3819918697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f81c00c5570 con 0x7f81c015d360 2024-03-30T00:42:00.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.108+0000 7f81b6a00640 1 -- 172.21.15.66:0/3819918697 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f81c00c5570 con 0x7f81c015d360 2024-03-30T00:42:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 -- 172.21.15.66:0/3819918697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f819c066690 msgr2=0x7f819c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f819c066690 0x7f819c068b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f81b00011f0 tx=0x7f81b0009040 comp rx=0 tx=0).stop 2024-03-30T00:42:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 -- 172.21.15.66:0/3819918697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81c015d360 msgr2=0x7f81c0156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81c015d360 0x7f81c0156e30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f81b800efc0 tx=0x7f81b8009f90 comp rx=0 tx=0).stop 2024-03-30T00:42:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 -- 172.21.15.66:0/3819918697 shutdown_connections 2024-03-30T00:42:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f819c066690 0x7f819c068b50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81c015d360 0x7f81c0156e30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81c00ceb60 0x7f81c00c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 --2- 172.21.15.66:0/3819918697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81c00ce780 0x7f81c006fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 -- 172.21.15.66:0/3819918697 >> 172.21.15.66:0/3819918697 conn(0x7f81c0069f60 msgr2=0x7f81c006a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 -- 172.21.15.66:0/3819918697 shutdown_connections 2024-03-30T00:42:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.112+0000 7f81c5400640 1 -- 172.21.15.66:0/3819918697 wait complete. 2024-03-30T00:42:00.140 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-03-30T00:42:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd09800640 1 Processor -- start 2024-03-30T00:42:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd09800640 1 -- start start 2024-03-30T00:42:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd09800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efd0406ba80 0x7efd0406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd09800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd040b2b60 0x7efd040caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd09800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd040ce070 con 0x7efd0415c9f0 2024-03-30T00:42:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd09800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0406cf60 con 0x7efd0406ba80 2024-03-30T00:42:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd09800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd040ce350 con 0x7efd040b2b60 2024-03-30T00:42:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd03e00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7efd0415c9f0 0x7efd0406b370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:60528/0 (socket says 172.21.15.66:60528) 2024-03-30T00:42:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd03e00640 1 -- 172.21.15.66:0/737752160 learned_addr learned my addr 172.21.15.66:0/737752160 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efcfb400640 1 --2- 172.21.15.66:0/737752160 >> v2:172.21.15.156:3300/0 conn(0x7efd0406ba80 0x7efd0406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd08e00640 1 --2- 172.21.15.66:0/737752160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd040b2b60 0x7efd040caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd03400640 1 -- 172.21.15.66:0/737752160 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149648277 0 0) 0x7efd040ce070 con 0x7efd0415c9f0 2024-03-30T00:42:00.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.272+0000 7efd03400640 1 -- 172.21.15.66:0/737752160 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce8003530 con 0x7efd0415c9f0 2024-03-30T00:42:00.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.276+0000 7efcfb400640 1 -- 172.21.15.66:0/737752160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd040b2b60 msgr2=0x7efd040caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:00.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.276+0000 7efcfb400640 1 --2- 172.21.15.66:0/737752160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd040b2b60 0x7efd040caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.276+0000 7efcfb400640 1 -- 172.21.15.66:0/737752160 >> v1:172.21.15.156:6789/0 conn(0x7efd0415c9f0 legacy=0x7efd0406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.276+0000 7efcfb400640 1 -- 172.21.15.66:0/737752160 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd040ce630 con 0x7efd0406ba80 2024-03-30T00:42:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efcfb400640 1 --2- 172.21.15.66:0/737752160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd0406ba80 0x7efd0406ca20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efcec009d90 tx=0x7efcec01c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4da0b0542aaf4a81 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efd03400640 1 -- 172.21.15.66:0/737752160 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0a5a00 con 0x7efd0406ba80 2024-03-30T00:42:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efd03400640 1 -- 172.21.15.66:0/737752160 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec0a5b60 con 0x7efd0406ba80 2024-03-30T00:42:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efd03400640 1 -- 172.21.15.66:0/737752160 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0ad440 con 0x7efd0406ba80 2024-03-30T00:42:00.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efd09800640 1 -- 172.21.15.66:0/737752160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd0406ba80 msgr2=0x7efd0406ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efd09800640 1 --2- 172.21.15.66:0/737752160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd0406ba80 0x7efd0406ca20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efcec009d90 tx=0x7efcec01c980 comp rx=0 tx=0).stop 2024-03-30T00:42:00.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efd09800640 1 -- 172.21.15.66:0/737752160 shutdown_connections 2024-03-30T00:42:00.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efd09800640 1 --2- 172.21.15.66:0/737752160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd040b2b60 0x7efd040caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efd09800640 1 --2- 172.21.15.66:0/737752160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd0406ba80 0x7efd0406ca20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efd09800640 1 -- 172.21.15.66:0/737752160 >> 172.21.15.66:0/737752160 conn(0x7efd0406e0d0 msgr2=0x7efd0406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:00.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efd09800640 1 -- 172.21.15.66:0/737752160 shutdown_connections 2024-03-30T00:42:00.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.280+0000 7efd09800640 1 -- 172.21.15.66:0/737752160 wait complete. 2024-03-30T00:42:00.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efd09800640 1 Processor -- start 2024-03-30T00:42:00.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efd09800640 1 -- start start 2024-03-30T00:42:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efd09800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd0406ba80 0x7efd040c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efd09800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd040b2b60 0x7efd0415f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efd09800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd0415c9f0 0x7efd041556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efd09800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd040c9810 con 0x7efd040b2b60 2024-03-30T00:42:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efd09800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd040c8c00 con 0x7efd0415c9f0 2024-03-30T00:42:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efd09800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd040c8ee0 con 0x7efd0406ba80 2024-03-30T00:42:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efcfb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd040b2b60 0x7efd0415f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efd03e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd0406ba80 0x7efd040c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efcfb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd040b2b60 0x7efd0415f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35260/0 (socket says 172.21.15.66:35260) 2024-03-30T00:42:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efcfb400640 1 -- 172.21.15.66:0/3868211586 learned_addr learned my addr 172.21.15.66:0/3868211586 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efd08e00640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd0415c9f0 0x7efd041556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efcfb400640 1 -- 172.21.15.66:0/3868211586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd0406ba80 msgr2=0x7efd040c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efcfb400640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd0406ba80 0x7efd040c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efcfb400640 1 -- 172.21.15.66:0/3868211586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd0415c9f0 msgr2=0x7efd041556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efcfb400640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd0415c9f0 0x7efd041556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.284+0000 7efcfb400640 1 -- 172.21.15.66:0/3868211586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec009a70 con 0x7efd040b2b60 2024-03-30T00:42:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.288+0000 7efcfb400640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd040b2b60 0x7efd0415f5d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efcec01a5b0 tx=0x7efcec019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.288+0000 7efd00c00640 1 -- 172.21.15.66:0/3868211586 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0add80 con 0x7efd040b2b60 2024-03-30T00:42:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.288+0000 7efd00c00640 1 -- 172.21.15.66:0/3868211586 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec009450 con 0x7efd040b2b60 2024-03-30T00:42:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.288+0000 7efd00c00640 1 -- 172.21.15.66:0/3868211586 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0b9690 con 0x7efd040b2b60 2024-03-30T00:42:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.288+0000 7efd09800640 1 -- 172.21.15.66:0/3868211586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd040c91c0 con 0x7efd040b2b60 2024-03-30T00:42:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.288+0000 7efd09800640 1 -- 172.21.15.66:0/3868211586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0415fb10 con 0x7efd040b2b60 2024-03-30T00:42:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.288+0000 7efd09800640 1 -- 172.21.15.66:0/3868211586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd040c7440 con 0x7efd040b2b60 2024-03-30T00:42:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.288+0000 7efd00c00640 1 -- 172.21.15.66:0/3868211586 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efcec0b97f0 con 0x7efd040b2b60 2024-03-30T00:42:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.288+0000 7efd00c00640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efce0066690 0x7efce0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.288+0000 7efd00c00640 1 -- 172.21.15.66:0/3868211586 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7efcec11f870 con 0x7efd040b2b60 2024-03-30T00:42:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.288+0000 7efd03e00640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efce0066690 0x7efce0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.292+0000 7efd03e00640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efce0066690 0x7efce0068b50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efcf40011f0 tx=0x7efcf4007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.292+0000 7efd00c00640 1 -- 172.21.15.66:0/3868211586 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efcec0eced0 con 0x7efd040b2b60 2024-03-30T00:42:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.512+0000 7efd09800640 1 -- 172.21.15.66:0/3868211586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7efd04073cf0 con 0x7efd040b2b60 2024-03-30T00:42:00.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.756+0000 7efd00c00640 1 -- 172.21.15.66:0/3868211586 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v139) v1 ==== 142+0+0 (secure 0 0 0) 0x7efcec0f04f0 con 0x7efd040b2b60 2024-03-30T00:42:00.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 -- 172.21.15.66:0/3868211586 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efce0066690 msgr2=0x7efce0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efce0066690 0x7efce0068b50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efcf40011f0 tx=0x7efcf4007d40 comp rx=0 tx=0).stop 2024-03-30T00:42:00.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 -- 172.21.15.66:0/3868211586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd040b2b60 msgr2=0x7efd0415f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd040b2b60 0x7efd0415f5d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efcec01a5b0 tx=0x7efcec019720 comp rx=0 tx=0).stop 2024-03-30T00:42:00.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 -- 172.21.15.66:0/3868211586 shutdown_connections 2024-03-30T00:42:00.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd0415c9f0 0x7efd041556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efce0066690 0x7efce0068b50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd040b2b60 0x7efd0415f5d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 --2- 172.21.15.66:0/3868211586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efd0406ba80 0x7efd040c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 -- 172.21.15.66:0/3868211586 >> 172.21.15.66:0/3868211586 conn(0x7efd0406e0d0 msgr2=0x7efd040ccd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:00.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 -- 172.21.15.66:0/3868211586 shutdown_connections 2024-03-30T00:42:00.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.760+0000 7efd09800640 1 -- 172.21.15.66:0/3868211586 wait complete. 2024-03-30T00:42:00.795 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-30T00:42:00.795 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-03-30T00:42:00.795 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-30T00:42:00.795 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:42:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.924+0000 7f6323200640 1 Processor -- start 2024-03-30T00:42:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.924+0000 7f6323200640 1 -- start start 2024-03-30T00:42:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6323200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f632406ba80 0x7f632406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6323200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63240ce070 con 0x7f63240b2b60 2024-03-30T00:42:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6323200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63240ce350 con 0x7f632415c9f0 2024-03-30T00:42:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6323200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632406cf60 con 0x7f632406ba80 2024-03-30T00:42:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6322800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f63240b2b60 0x7f63240caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36308/0 (socket says 172.21.15.66:36308) 2024-03-30T00:42:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6322800640 1 -- 172.21.15.66:0/1705542659 learned_addr learned my addr 172.21.15.66:0/1705542659 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:00.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6319400640 1 --2- 172.21.15.66:0/1705542659 >> v2:172.21.15.156:3300/0 conn(0x7f632406ba80 0x7f632406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880069428 0 0) 0x7f63240ce070 con 0x7f63240b2b60 2024-03-30T00:42:00.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6308003530 con 0x7f63240b2b60 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2717998981 0 0) 0x7f6308003530 con 0x7f63240b2b60 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 >> v2:172.21.15.156:3300/0 conn(0x7f632406ba80 msgr2=0x7f632406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 --2- 172.21.15.66:0/1705542659 >> v2:172.21.15.156:3300/0 conn(0x7f632406ba80 0x7f632406ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 >> v1:172.21.15.156:6789/0 conn(0x7f632415c9f0 legacy=0x7f632406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63240ce630 con 0x7f63240b2b60 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f631c001680 con 0x7f63240b2b60 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 >> v1:172.21.15.66:6789/0 conn(0x7f63240b2b60 legacy=0x7f63240caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 --2- 172.21.15.66:0/1705542659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6308003980 0x7f6308003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 --2- 172.21.15.66:0/1705542659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63080047d0 0x7f6308004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 --2- 172.21.15.66:0/1705542659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6308005190 0x7f630800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630800c100 con 0x7f6308003980 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630800c270 con 0x7f6308005190 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630800c430 con 0x7f63080047d0 2024-03-30T00:42:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321e00640 1 --2- 172.21.15.66:0/1705542659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6308003980 0x7f6308003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6319400640 1 --2- 172.21.15.66:0/1705542659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63080047d0 0x7f6308004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6322800640 1 --2- 172.21.15.66:0/1705542659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6308005190 0x7f630800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321e00640 1 -- 172.21.15.66:0/1705542659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63080047d0 msgr2=0x7f6308004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321e00640 1 --2- 172.21.15.66:0/1705542659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63080047d0 0x7f6308004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321e00640 1 -- 172.21.15.66:0/1705542659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6308005190 msgr2=0x7f630800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321e00640 1 --2- 172.21.15.66:0/1705542659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6308005190 0x7f630800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6321e00640 1 -- 172.21.15.66:0/1705542659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6308003530 con 0x7f6308003980 2024-03-30T00:42:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.928+0000 7f6322800640 1 -- 172.21.15.66:0/1705542659 reap_dead start 2024-03-30T00:42:00.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6321e00640 1 --2- 172.21.15.66:0/1705542659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6308003980 0x7f6308003da0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6314001e70 tx=0x7f631401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d4b28826d78b5b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:00.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314003b80 con 0x7f6308003980 2024-03-30T00:42:00.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6314003ce0 con 0x7f6308003980 2024-03-30T00:42:00.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6321400640 1 -- 172.21.15.66:0/1705542659 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63140052c0 con 0x7f6308003980 2024-03-30T00:42:00.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6323200640 1 -- 172.21.15.66:0/1705542659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6308003980 msgr2=0x7f6308003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6323200640 1 --2- 172.21.15.66:0/1705542659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6308003980 0x7f6308003da0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6314001e70 tx=0x7f631401c730 comp rx=0 tx=0).stop 2024-03-30T00:42:00.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6323200640 1 -- 172.21.15.66:0/1705542659 shutdown_connections 2024-03-30T00:42:00.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6323200640 1 --2- 172.21.15.66:0/1705542659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6308003980 0x7f6308003da0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6323200640 1 -- 172.21.15.66:0/1705542659 >> 172.21.15.66:0/1705542659 conn(0x7f632406e0d0 msgr2=0x7f632406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:00.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6323200640 1 -- 172.21.15.66:0/1705542659 shutdown_connections 2024-03-30T00:42:00.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6323200640 1 -- 172.21.15.66:0/1705542659 wait complete. 2024-03-30T00:42:00.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.932+0000 7f6323200640 1 Processor -- start 2024-03-30T00:42:00.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6323200640 1 -- start start 2024-03-30T00:42:00.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6323200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6308003980 0x7f6324157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6323200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6324147220 0x7f63240b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6323200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63240b51e0 0x7f63240b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6323200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63240b5ba0 con 0x7f63240b51e0 2024-03-30T00:42:00.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6323200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63240b3140 con 0x7f6324147220 2024-03-30T00:42:00.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6323200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63240b3400 con 0x7f6308003980 2024-03-30T00:42:00.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6322800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63240b51e0 0x7f63240b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6322800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63240b51e0 0x7f63240b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35264/0 (socket says 172.21.15.66:35264) 2024-03-30T00:42:00.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6322800640 1 -- 172.21.15.66:0/2998666716 learned_addr learned my addr 172.21.15.66:0/2998666716 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:00.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6321e00640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6308003980 0x7f6324157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6319400640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6324147220 0x7f63240b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6322800640 1 -- 172.21.15.66:0/2998666716 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6308003980 msgr2=0x7f6324157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6322800640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6308003980 0x7f6324157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6322800640 1 -- 172.21.15.66:0/2998666716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6324147220 msgr2=0x7f63240b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6322800640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6324147220 0x7f63240b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6322800640 1 -- 172.21.15.66:0/2998666716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632415ec60 con 0x7f63240b51e0 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6322800640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63240b51e0 0x7f63240b5660 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f63240cf470 tx=0x7f631c007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f631aa00640 1 -- 172.21.15.66:0/2998666716 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c007790 con 0x7f63240b51e0 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f631aa00640 1 -- 172.21.15.66:0/2998666716 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631c001630 con 0x7f63240b51e0 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6323200640 1 -- 172.21.15.66:0/2998666716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632415ef40 con 0x7f63240b51e0 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f631aa00640 1 -- 172.21.15.66:0/2998666716 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c00faf0 con 0x7f63240b51e0 2024-03-30T00:42:00.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6323200640 1 -- 172.21.15.66:0/2998666716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63240b36e0 con 0x7f63240b51e0 2024-03-30T00:42:00.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.936+0000 7f6323200640 1 -- 172.21.15.66:0/2998666716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6324073cf0 con 0x7f63240b51e0 2024-03-30T00:42:00.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.940+0000 7f631aa00640 1 -- 172.21.15.66:0/2998666716 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f631c00fc50 con 0x7f63240b51e0 2024-03-30T00:42:00.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.940+0000 7f631aa00640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63000666c0 0x7f6300068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:00.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.940+0000 7f631aa00640 1 -- 172.21.15.66:0/2998666716 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f631c083cc0 con 0x7f63240b51e0 2024-03-30T00:42:00.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.940+0000 7f6321e00640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63000666c0 0x7f6300068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:00.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.940+0000 7f6321e00640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63000666c0 0x7f6300068b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6314005000 tx=0x7f6314009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:00.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:00.944+0000 7f631aa00640 1 -- 172.21.15.66:0/2998666716 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f631c051280 con 0x7f63240b51e0 2024-03-30T00:42:01.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.104+0000 7f6323200640 1 -- 172.21.15.66:0/2998666716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63240b6420 con 0x7f63240b51e0 2024-03-30T00:42:01.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f631aa00640 1 -- 172.21.15.66:0/2998666716 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+12723 (secure 0 0 0) 0x7f631c0548a0 con 0x7f63240b51e0 2024-03-30T00:42:01.117 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:01.117 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":45,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:41:59.744933+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6837/1694329026":"2024-03-31T00:41:56.730775+0000","172.21.15.66:6836/1694329026":"2024-03-31T00:41:56.730775+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:42:01.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 -- 172.21.15.66:0/2998666716 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63000666c0 msgr2=0x7f6300068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:01.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63000666c0 0x7f6300068b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6314005000 tx=0x7f6314009040 comp rx=0 tx=0).stop 2024-03-30T00:42:01.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 -- 172.21.15.66:0/2998666716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63240b51e0 msgr2=0x7f63240b5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:01.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63240b51e0 0x7f63240b5660 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f63240cf470 tx=0x7f631c007660 comp rx=0 tx=0).stop 2024-03-30T00:42:01.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 -- 172.21.15.66:0/2998666716 shutdown_connections 2024-03-30T00:42:01.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63000666c0 0x7f6300068b80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63240b51e0 0x7f63240b5660 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6324147220 0x7f63240b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 --2- 172.21.15.66:0/2998666716 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6308003980 0x7f6324157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 -- 172.21.15.66:0/2998666716 >> 172.21.15.66:0/2998666716 conn(0x7f632406e0d0 msgr2=0x7f63240cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:01.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 -- 172.21.15.66:0/2998666716 shutdown_connections 2024-03-30T00:42:01.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.108+0000 7f6323200640 1 -- 172.21.15.66:0/2998666716 wait complete. 2024-03-30T00:42:01.138 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:42:01.139 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:42:01.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c65200640 1 Processor -- start 2024-03-30T00:42:01.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c65200640 1 -- start start 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c65200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cec30 0x7f4c60159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c65200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4c600cde70 0x7f4c6015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c65200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6015a6e0 con 0x7f4c600cda90 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c65200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c600b1b80 con 0x7f4c600cde70 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c65200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600cd3e0 con 0x7f4c600cec30 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c5e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4c600cde70 0x7f4c6015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c5e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4c600cde70 0x7f4c6015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35272/0 (socket says 172.21.15.66:35272) 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c5e800640 1 -- 172.21.15.66:0/130519849 learned_addr learned my addr 172.21.15.66:0/130519849 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c55400640 1 --2- 172.21.15.66:0/130519849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cec30 0x7f4c60159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c5e800640 1 -- 172.21.15.66:0/130519849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cec30 msgr2=0x7f4c60159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c5e800640 1 --2- 172.21.15.66:0/130519849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cec30 0x7f4c60159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c5e800640 1 -- 172.21.15.66:0/130519849 >> v1:172.21.15.156:6789/0 conn(0x7f4c600cda90 legacy=0x7f4c600ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:01.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c5e800640 1 -- 172.21.15.66:0/130519849 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6015a990 con 0x7f4c600cde70 2024-03-30T00:42:01.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.272+0000 7f4c55400640 1 --2- 172.21.15.66:0/130519849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cec30 0x7f4c60159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:01.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c5e800640 1 --2- 172.21.15.66:0/130519849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c600cde70 0x7f4c6015a1a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009870 tx=0x7f4c5801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0cd0ae96a7c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:01.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c5d400640 1 -- 172.21.15.66:0/130519849 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c580ab650 con 0x7f4c600cde70 2024-03-30T00:42:01.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c5d400640 1 -- 172.21.15.66:0/130519849 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c580ab7b0 con 0x7f4c600cde70 2024-03-30T00:42:01.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c5d400640 1 -- 172.21.15.66:0/130519849 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c580abb40 con 0x7f4c600cde70 2024-03-30T00:42:01.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c65200640 1 -- 172.21.15.66:0/130519849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c600cde70 msgr2=0x7f4c6015a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:01.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c65200640 1 --2- 172.21.15.66:0/130519849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c600cde70 0x7f4c6015a1a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009870 tx=0x7f4c5801c490 comp rx=0 tx=0).stop 2024-03-30T00:42:01.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c65200640 1 -- 172.21.15.66:0/130519849 shutdown_connections 2024-03-30T00:42:01.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c65200640 1 --2- 172.21.15.66:0/130519849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c600cde70 0x7f4c6015a1a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c65200640 1 --2- 172.21.15.66:0/130519849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cec30 0x7f4c60159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c65200640 1 -- 172.21.15.66:0/130519849 >> 172.21.15.66:0/130519849 conn(0x7f4c60069f60 msgr2=0x7f4c6006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:01.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c65200640 1 -- 172.21.15.66:0/130519849 shutdown_connections 2024-03-30T00:42:01.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c65200640 1 -- 172.21.15.66:0/130519849 wait complete. 2024-03-30T00:42:01.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c65200640 1 Processor -- start 2024-03-30T00:42:01.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.276+0000 7f4c65200640 1 -- start start 2024-03-30T00:42:01.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c65200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cda90 0x7f4c600c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c65200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600cde70 0x7f4c60153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c65200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c600cec30 0x7f4c600abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c65200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c601547e0 con 0x7f4c600cec30 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c65200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c60154ac0 con 0x7f4c600cde70 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c65200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c60154da0 con 0x7f4c600cda90 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c5e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c600cec30 0x7f4c600abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c5e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c600cec30 0x7f4c600abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35284/0 (socket says 172.21.15.66:35284) 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c5e800640 1 -- 172.21.15.66:0/1976634479 learned_addr learned my addr 172.21.15.66:0/1976634479 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c55400640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600cde70 0x7f4c60153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c5de00640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cda90 0x7f4c600c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c5e800640 1 -- 172.21.15.66:0/1976634479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cda90 msgr2=0x7f4c600c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c5e800640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cda90 0x7f4c600c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c5e800640 1 -- 172.21.15.66:0/1976634479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600cde70 msgr2=0x7f4c60153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:01.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c5e800640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600cde70 0x7f4c60153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c5e800640 1 -- 172.21.15.66:0/1976634479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c580094d0 con 0x7f4c600cec30 2024-03-30T00:42:01.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c5de00640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cda90 0x7f4c600c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:01.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c5e800640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c600cec30 0x7f4c600abdf0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4c580ad040 tx=0x7f4c5801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:01.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c56a00640 1 -- 172.21.15.66:0/1976634479 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c580b4800 con 0x7f4c600cec30 2024-03-30T00:42:01.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c56a00640 1 -- 172.21.15.66:0/1976634479 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c580b4980 con 0x7f4c600cec30 2024-03-30T00:42:01.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c56a00640 1 -- 172.21.15.66:0/1976634479 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c580b4d30 con 0x7f4c600cec30 2024-03-30T00:42:01.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c65200640 1 -- 172.21.15.66:0/1976634479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c60155080 con 0x7f4c600cec30 2024-03-30T00:42:01.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c65200640 1 -- 172.21.15.66:0/1976634479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c600c81c0 con 0x7f4c600cec30 2024-03-30T00:42:01.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c65200640 1 -- 172.21.15.66:0/1976634479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c60066c30 con 0x7f4c600cec30 2024-03-30T00:42:01.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.280+0000 7f4c56a00640 1 -- 172.21.15.66:0/1976634479 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4c580c7440 con 0x7f4c600cec30 2024-03-30T00:42:01.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.284+0000 7f4c56a00640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c3c066690 0x7f4c3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.284+0000 7f4c56a00640 1 -- 172.21.15.66:0/1976634479 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4c5811ecf0 con 0x7f4c600cec30 2024-03-30T00:42:01.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.284+0000 7f4c5de00640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c3c066690 0x7f4c3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.284+0000 7f4c5de00640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c3c066690 0x7f4c3c068b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4c500045c0 tx=0x7f4c50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:01.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.288+0000 7f4c56a00640 1 -- 172.21.15.66:0/1976634479 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4c580ec3e0 con 0x7f4c600cec30 2024-03-30T00:42:01.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 -- 172.21.15.66:0/1976634479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c60154920 con 0x7f4c600cec30 2024-03-30T00:42:01.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c56a00640 1 -- 172.21.15.66:0/1976634479 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c60154920 con 0x7f4c600cec30 2024-03-30T00:42:01.458 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 -- 172.21.15.66:0/1976634479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c3c066690 msgr2=0x7f4c3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c3c066690 0x7f4c3c068b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4c500045c0 tx=0x7f4c50009040 comp rx=0 tx=0).stop 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 -- 172.21.15.66:0/1976634479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c600cec30 msgr2=0x7f4c600abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c600cec30 0x7f4c600abdf0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4c580ad040 tx=0x7f4c5801acd0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 -- 172.21.15.66:0/1976634479 shutdown_connections 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c3c066690 0x7f4c3c068b50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c600cec30 0x7f4c600abdf0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600cde70 0x7f4c60153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 --2- 172.21.15.66:0/1976634479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c600cda90 0x7f4c600c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 -- 172.21.15.66:0/1976634479 >> 172.21.15.66:0/1976634479 conn(0x7f4c60069f60 msgr2=0x7f4c6006ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 -- 172.21.15.66:0/1976634479 shutdown_connections 2024-03-30T00:42:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.448+0000 7f4c65200640 1 -- 172.21.15.66:0/1976634479 wait complete. 2024-03-30T00:42:01.478 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-03-30T00:42:01.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3ac7000640 1 Processor -- start 2024-03-30T00:42:01.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3ac7000640 1 -- start start 2024-03-30T00:42:01.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3ac7000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ac00cda90 0x7f3ac00ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3ac7000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3ac00cde70 0x7f3ac00cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3ac7000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac0159c60 0x7f3ac015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3ac7000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac00b1b80 con 0x7f3ac00cda90 2024-03-30T00:42:01.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3ac7000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac015a990 con 0x7f3ac00cde70 2024-03-30T00:42:01.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3ac7000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00b11e0 con 0x7f3ac0159c60 2024-03-30T00:42:01.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3abfe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ac00cda90 0x7f3ac00ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3abfe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ac00cda90 0x7f3ac00ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35292/0 (socket says 172.21.15.66:35292) 2024-03-30T00:42:01.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3abfe00640 1 -- 172.21.15.66:0/186802865 learned_addr learned my addr 172.21.15.66:0/186802865 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:01.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3ac5000640 1 --2- 172.21.15.66:0/186802865 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac0159c60 0x7f3ac015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3ab7e00640 1 --2- 172.21.15.66:0/186802865 >> v2:172.21.15.156:3300/0 conn(0x7f3ac00cde70 0x7f3ac00cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3abfe00640 1 -- 172.21.15.66:0/186802865 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac0159c60 msgr2=0x7f3ac015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3abfe00640 1 --2- 172.21.15.66:0/186802865 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac0159c60 0x7f3ac015a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3abfe00640 1 -- 172.21.15.66:0/186802865 >> v2:172.21.15.156:3300/0 conn(0x7f3ac00cde70 msgr2=0x7f3ac00cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3abfe00640 1 --2- 172.21.15.66:0/186802865 >> v2:172.21.15.156:3300/0 conn(0x7f3ac00cde70 0x7f3ac00cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3abfe00640 1 -- 172.21.15.66:0/186802865 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac00b14c0 con 0x7f3ac00cda90 2024-03-30T00:42:01.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.608+0000 7f3ab7e00640 1 --2- 172.21.15.66:0/186802865 >> v2:172.21.15.156:3300/0 conn(0x7f3ac00cde70 0x7f3ac00cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3abfe00640 1 --2- 172.21.15.66:0/186802865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cda90 0x7f3ac00ce520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00cc320 tx=0x7f3ab001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3d6a4ac1273574 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3abf400640 1 -- 172.21.15.66:0/186802865 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00ab670 con 0x7f3ac00cda90 2024-03-30T00:42:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3abf400640 1 -- 172.21.15.66:0/186802865 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab00ab7d0 con 0x7f3ac00cda90 2024-03-30T00:42:01.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3abf400640 1 -- 172.21.15.66:0/186802865 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00abb60 con 0x7f3ac00cda90 2024-03-30T00:42:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3ac7000640 1 -- 172.21.15.66:0/186802865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cda90 msgr2=0x7f3ac00ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3ac7000640 1 --2- 172.21.15.66:0/186802865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cda90 0x7f3ac00ce520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00cc320 tx=0x7f3ab001c490 comp rx=0 tx=0).stop 2024-03-30T00:42:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3ac7000640 1 -- 172.21.15.66:0/186802865 shutdown_connections 2024-03-30T00:42:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3ac7000640 1 --2- 172.21.15.66:0/186802865 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac0159c60 0x7f3ac015a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3ac7000640 1 --2- 172.21.15.66:0/186802865 >> v2:172.21.15.156:3300/0 conn(0x7f3ac00cde70 0x7f3ac00cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3ac7000640 1 --2- 172.21.15.66:0/186802865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac00cda90 0x7f3ac00ce520 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3ac7000640 1 -- 172.21.15.66:0/186802865 >> 172.21.15.66:0/186802865 conn(0x7f3ac0069f60 msgr2=0x7f3ac006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3ac7000640 1 -- 172.21.15.66:0/186802865 shutdown_connections 2024-03-30T00:42:01.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.612+0000 7f3ac7000640 1 -- 172.21.15.66:0/186802865 wait complete. 2024-03-30T00:42:01.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac7000640 1 Processor -- start 2024-03-30T00:42:01.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac7000640 1 -- start start 2024-03-30T00:42:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac7000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cda90 0x7f3ac00aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac7000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ac00cde70 0x7f3ac0156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac7000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac0159c60 0x7f3ac00ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac7000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00acfd0 con 0x7f3ac0159c60 2024-03-30T00:42:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac7000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00ab9f0 con 0x7f3ac00cde70 2024-03-30T00:42:01.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac7000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00abcd0 con 0x7f3ac00cda90 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac0159c60 0x7f3ac00ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac0159c60 0x7f3ac00ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35302/0 (socket says 172.21.15.66:35302) 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac5000640 1 -- 172.21.15.66:0/2565883042 learned_addr learned my addr 172.21.15.66:0/2565883042 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3abfe00640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cda90 0x7f3ac00aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ab7e00640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ac00cde70 0x7f3ac0156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac5000640 1 -- 172.21.15.66:0/2565883042 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cda90 msgr2=0x7f3ac00aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac5000640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cda90 0x7f3ac00aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac5000640 1 -- 172.21.15.66:0/2565883042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ac00cde70 msgr2=0x7f3ac0156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac5000640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ac00cde70 0x7f3ac0156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac5000640 1 -- 172.21.15.66:0/2565883042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab00094d0 con 0x7f3ac0159c60 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ab7e00640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ac00cde70 0x7f3ac0156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac5000640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac0159c60 0x7f3ac00ab010 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800efc0 tx=0x7f3ab800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3abcc00640 1 -- 172.21.15.66:0/2565883042 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8009ba0 con 0x7f3ac0159c60 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3abcc00640 1 -- 172.21.15.66:0/2565883042 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab800f040 con 0x7f3ac0159c60 2024-03-30T00:42:01.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3abcc00640 1 -- 172.21.15.66:0/2565883042 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80146c0 con 0x7f3ac0159c60 2024-03-30T00:42:01.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac7000640 1 -- 172.21.15.66:0/2565883042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac00abfb0 con 0x7f3ac0159c60 2024-03-30T00:42:01.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.616+0000 7f3ac7000640 1 -- 172.21.15.66:0/2565883042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac00ac570 con 0x7f3ac0159c60 2024-03-30T00:42:01.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.620+0000 7f3ac7000640 1 -- 172.21.15.66:0/2565883042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac0066c30 con 0x7f3ac0159c60 2024-03-30T00:42:01.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.620+0000 7f3abcc00640 1 -- 172.21.15.66:0/2565883042 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3ab8014820 con 0x7f3ac0159c60 2024-03-30T00:42:01.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.620+0000 7f3abcc00640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 0x7f3a9c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:01.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.620+0000 7f3abcc00640 1 -- 172.21.15.66:0/2565883042 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f3ab8084af0 con 0x7f3ac0159c60 2024-03-30T00:42:01.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.620+0000 7f3abfe00640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 0x7f3a9c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:01.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.620+0000 7f3abfe00640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 0x7f3a9c068b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00af000 tx=0x7f3ab00b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:01.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.624+0000 7f3abcc00640 1 -- 172.21.15.66:0/2565883042 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3ab8051f80 con 0x7f3ac0159c60 2024-03-30T00:42:01.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:01.840+0000 7f3ac7000640 1 -- 172.21.15.66:0/2565883042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f3ac0165180 con 0x7f3ac0159c60 2024-03-30T00:42:02.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.764+0000 7f3abcc00640 1 -- 172.21.15.66:0/2565883042 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v141) v1 ==== 152+0+0 (secure 0 0 0) 0x7f3ab80180d0 con 0x7f3ac0159c60 2024-03-30T00:42:02.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 -- 172.21.15.66:0/2565883042 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 msgr2=0x7f3a9c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:02.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 0x7f3a9c068b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00af000 tx=0x7f3ab00b6040 comp rx=0 tx=0).stop 2024-03-30T00:42:02.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 -- 172.21.15.66:0/2565883042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac0159c60 msgr2=0x7f3ac00ab010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:02.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac0159c60 0x7f3ac00ab010 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800efc0 tx=0x7f3ab800c490 comp rx=0 tx=0).stop 2024-03-30T00:42:02.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 -- 172.21.15.66:0/2565883042 shutdown_connections 2024-03-30T00:42:02.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a9c066690 0x7f3a9c068b50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:02.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac0159c60 0x7f3ac00ab010 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:02.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ac00cde70 0x7f3ac0156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:02.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 --2- 172.21.15.66:0/2565883042 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ac00cda90 0x7f3ac00aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:02.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 -- 172.21.15.66:0/2565883042 >> 172.21.15.66:0/2565883042 conn(0x7f3ac0069f60 msgr2=0x7f3ac00ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:02.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 -- 172.21.15.66:0/2565883042 shutdown_connections 2024-03-30T00:42:02.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.768+0000 7f3ac7000640 1 -- 172.21.15.66:0/2565883042 wait complete. 2024-03-30T00:42:02.802 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-30T00:42:02.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9cd400640 1 Processor -- start 2024-03-30T00:42:02.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9cd400640 1 -- start start 2024-03-30T00:42:02.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9cd400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc9c80b4a60 0x7fc9c815f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:02.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9cd400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c8099e70 con 0x7fc9c80cf320 2024-03-30T00:42:02.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9cd400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c80cbd90 con 0x7fc9c8098c80 2024-03-30T00:42:02.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9cd400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c80cc070 con 0x7fc9c80b4a60 2024-03-30T00:42:02.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9c7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc9c80b4a60 0x7fc9c815f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:02.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9bde00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc9c80cf320 0x7fc9c815eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36310/0 (socket says 172.21.15.66:36310) 2024-03-30T00:42:02.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9bde00640 1 -- 172.21.15.66:0/4169691143 learned_addr learned my addr 172.21.15.66:0/4169691143 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:02.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9c7200640 1 -- 172.21.15.66:0/4169691143 >> v1:172.21.15.156:6789/0 conn(0x7fc9c8098c80 legacy=0x7fc9c8099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:02.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9c7200640 1 -- 172.21.15.66:0/4169691143 >> v1:172.21.15.66:6789/0 conn(0x7fc9c80cf320 legacy=0x7fc9c815eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:02.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.940+0000 7fc9c7200640 1 -- 172.21.15.66:0/4169691143 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c80cc350 con 0x7fc9c80b4a60 2024-03-30T00:42:02.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.944+0000 7fc9c7200640 1 --2- 172.21.15.66:0/4169691143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c80b4a60 0x7fc9c815f2a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009d90 tx=0x7fc9c001f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a262c7dcfcd690a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:02.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.944+0000 7fc9c5e00640 1 -- 172.21.15.66:0/4169691143 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00af600 con 0x7fc9c80b4a60 2024-03-30T00:42:02.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.944+0000 7fc9c5e00640 1 -- 172.21.15.66:0/4169691143 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c00ab050 con 0x7fc9c80b4a60 2024-03-30T00:42:02.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.944+0000 7fc9c5e00640 1 -- 172.21.15.66:0/4169691143 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00af980 con 0x7fc9c80b4a60 2024-03-30T00:42:02.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.944+0000 7fc9cd400640 1 -- 172.21.15.66:0/4169691143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c80b4a60 msgr2=0x7fc9c815f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:02.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.944+0000 7fc9cd400640 1 --2- 172.21.15.66:0/4169691143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c80b4a60 0x7fc9c815f2a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009d90 tx=0x7fc9c001f120 comp rx=0 tx=0).stop 2024-03-30T00:42:02.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.944+0000 7fc9cd400640 1 -- 172.21.15.66:0/4169691143 shutdown_connections 2024-03-30T00:42:02.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.944+0000 7fc9cd400640 1 --2- 172.21.15.66:0/4169691143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c80b4a60 0x7fc9c815f2a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.944+0000 7fc9cd400640 1 -- 172.21.15.66:0/4169691143 >> 172.21.15.66:0/4169691143 conn(0x7fc9c80b2130 msgr2=0x7fc9c80b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.944+0000 7fc9cd400640 1 -- 172.21.15.66:0/4169691143 shutdown_connections 2024-03-30T00:42:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.944+0000 7fc9cd400640 1 -- 172.21.15.66:0/4169691143 wait complete. 2024-03-30T00:42:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9cd400640 1 Processor -- start 2024-03-30T00:42:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9cd400640 1 -- start start 2024-03-30T00:42:02.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9cd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c8098c80 0x7fc9c80c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:02.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9cd400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c80b4a60 0x7fc9c81e6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:02.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9cd400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9c80cf320 0x7fc9c81e4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:02.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9cd400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c81e72e0 con 0x7fc9c8098c80 2024-03-30T00:42:02.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9cd400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c81e75c0 con 0x7fc9c80b4a60 2024-03-30T00:42:02.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9cd400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c80c7da0 con 0x7fc9c80cf320 2024-03-30T00:42:02.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9c6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c8098c80 0x7fc9c80c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9c6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c8098c80 0x7fc9c80c63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35310/0 (socket says 172.21.15.66:35310) 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9c6800640 1 -- 172.21.15.66:0/2729567924 learned_addr learned my addr 172.21.15.66:0/2729567924 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9c7200640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9c80cf320 0x7fc9c81e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9c6800640 1 -- 172.21.15.66:0/2729567924 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9c80cf320 msgr2=0x7fc9c81e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9c6800640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9c80cf320 0x7fc9c81e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9c6800640 1 -- 172.21.15.66:0/2729567924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c80b4a60 msgr2=0x7fc9c81e6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9c6800640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c80b4a60 0x7fc9c81e6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9c6800640 1 -- 172.21.15.66:0/2729567924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c0009a70 con 0x7fc9c8098c80 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9c6800640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c8098c80 0x7fc9c80c63c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc9b800b3e0 tx=0x7fc9b800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9bf400640 1 -- 172.21.15.66:0/2729567924 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8011830 con 0x7fc9c8098c80 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9bf400640 1 -- 172.21.15.66:0/2729567924 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b80119b0 con 0x7fc9c8098c80 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9bf400640 1 -- 172.21.15.66:0/2729567924 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8011d60 con 0x7fc9c8098c80 2024-03-30T00:42:02.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9cd400640 1 -- 172.21.15.66:0/2729567924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c80c8080 con 0x7fc9c8098c80 2024-03-30T00:42:02.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9cd400640 1 -- 172.21.15.66:0/2729567924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c814fca0 con 0x7fc9c8098c80 2024-03-30T00:42:02.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.948+0000 7fc9cd400640 1 -- 172.21.15.66:0/2729567924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c80677d0 con 0x7fc9c8098c80 2024-03-30T00:42:02.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.952+0000 7fc9bf400640 1 -- 172.21.15.66:0/2729567924 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc9b801a400 con 0x7fc9c8098c80 2024-03-30T00:42:02.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.952+0000 7fc9bf400640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc9a4066690 0x7fc9a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:02.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.952+0000 7fc9bf400640 1 -- 172.21.15.66:0/2729567924 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fc9b8084220 con 0x7fc9c8098c80 2024-03-30T00:42:02.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.952+0000 7fc9bde00640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc9a4066690 0x7fc9a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:02.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.952+0000 7fc9bde00640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc9a4066690 0x7fc9a4068b50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8144100 tx=0x7fc9b0007a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:02.956+0000 7fc9bf400640 1 -- 172.21.15.66:0/2729567924 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc9b80516c0 con 0x7fc9c8098c80 2024-03-30T00:42:03.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.116+0000 7fc9cd400640 1 -- 172.21.15.66:0/2729567924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc9c8066c30 con 0x7fc9c8098c80 2024-03-30T00:42:03.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.120+0000 7fc9bf400640 1 -- 172.21.15.66:0/2729567924 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc9b80150c0 con 0x7fc9c8098c80 2024-03-30T00:42:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 -- 172.21.15.66:0/2729567924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc9a4066690 msgr2=0x7fc9a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc9a4066690 0x7fc9a4068b50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8144100 tx=0x7fc9b0007a50 comp rx=0 tx=0).stop 2024-03-30T00:42:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 -- 172.21.15.66:0/2729567924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c8098c80 msgr2=0x7fc9c80c63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c8098c80 0x7fc9c80c63c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc9b800b3e0 tx=0x7fc9b800b8a0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 -- 172.21.15.66:0/2729567924 shutdown_connections 2024-03-30T00:42:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9c80cf320 0x7fc9c81e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9c80b4a60 0x7fc9c81e6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc9a4066690 0x7fc9a4068b50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 --2- 172.21.15.66:0/2729567924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c8098c80 0x7fc9c80c63c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 -- 172.21.15.66:0/2729567924 >> 172.21.15.66:0/2729567924 conn(0x7fc9c80b2130 msgr2=0x7fc9c80b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:03.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 -- 172.21.15.66:0/2729567924 shutdown_connections 2024-03-30T00:42:03.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.124+0000 7fc9cd400640 1 -- 172.21.15.66:0/2729567924 wait complete. 2024-03-30T00:42:03.153 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:42:03.153 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:42:03.208 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:42:03.294 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:42:03.294 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:42:03.295 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:42:03.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:42:03.304 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:42:03.304 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:42:03.313 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:42:03.313 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:42:03.313 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:42:03.356 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:42:03.356 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:42:03.356 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:42:03.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:42:03.365 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-30T00:42:03.365 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:42:03.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.524+0000 7fc8f1200640 1 Processor -- start 2024-03-30T00:42:03.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.524+0000 7fc8f1200640 1 -- start start 2024-03-30T00:42:03.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8f1200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc8ec0cd2d0 0x7fc8ec0cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8f1200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8ec0cc8d0 0x7fc8ec06c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8f1200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ec0cb620 con 0x7fc8ec0cad00 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8f1200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ec0cbec0 con 0x7fc8ec0cd2d0 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8f1200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec0ca660 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8e3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc8ec0cd2d0 0x7fc8ec0cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8e3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc8ec0cd2d0 0x7fc8ec0cb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35322/0 (socket says 172.21.15.66:35322) 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8e3e00640 1 -- 172.21.15.66:0/3392237612 learned_addr learned my addr 172.21.15.66:0/3392237612 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8ea800640 1 --2- 172.21.15.66:0/3392237612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8ec0cc8d0 0x7fc8ec06c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8e3e00640 1 -- 172.21.15.66:0/3392237612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8ec0cc8d0 msgr2=0x7fc8ec06c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8e3e00640 1 --2- 172.21.15.66:0/3392237612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8ec0cc8d0 0x7fc8ec06c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8e3e00640 1 -- 172.21.15.66:0/3392237612 >> v1:172.21.15.66:6789/0 conn(0x7fc8ec0cad00 legacy=0x7fc8ec0c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8e3e00640 1 -- 172.21.15.66:0/3392237612 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec0ca940 con 0x7fc8ec0cd2d0 2024-03-30T00:42:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.528+0000 7fc8e9400640 1 -- 172.21.15.66:0/3392237612 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195066403 0 0) 0x7fc8ec0cb620 con 0x7fc8ec0cad00 2024-03-30T00:42:03.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8e3e00640 1 --2- 172.21.15.66:0/3392237612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cd2d0 0x7fc8ec0cb0e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009870 tx=0x7fc8d401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c28f634b9d4b945 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:03.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8e9400640 1 -- 172.21.15.66:0/3392237612 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40a95f0 con 0x7fc8ec0cd2d0 2024-03-30T00:42:03.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8e9400640 1 -- 172.21.15.66:0/3392237612 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d40a9750 con 0x7fc8ec0cd2d0 2024-03-30T00:42:03.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8e9400640 1 -- 172.21.15.66:0/3392237612 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40a9ae0 con 0x7fc8ec0cd2d0 2024-03-30T00:42:03.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 -- 172.21.15.66:0/3392237612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cd2d0 msgr2=0x7fc8ec0cb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 --2- 172.21.15.66:0/3392237612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cd2d0 0x7fc8ec0cb0e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009870 tx=0x7fc8d401c530 comp rx=0 tx=0).stop 2024-03-30T00:42:03.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 -- 172.21.15.66:0/3392237612 shutdown_connections 2024-03-30T00:42:03.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 --2- 172.21.15.66:0/3392237612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8ec0cc8d0 0x7fc8ec06c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 --2- 172.21.15.66:0/3392237612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cd2d0 0x7fc8ec0cb0e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 -- 172.21.15.66:0/3392237612 >> 172.21.15.66:0/3392237612 conn(0x7fc8ec069f60 msgr2=0x7fc8ec06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:03.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 -- 172.21.15.66:0/3392237612 shutdown_connections 2024-03-30T00:42:03.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 -- 172.21.15.66:0/3392237612 wait complete. 2024-03-30T00:42:03.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 Processor -- start 2024-03-30T00:42:03.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 -- start start 2024-03-30T00:42:03.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8ec0cad00 0x7fc8ec0934d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:03.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cc8d0 0x7fc8ec14ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:03.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8ec0cd2d0 0x7fc8ec14f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:03.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec15b5c0 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec0acd60 con 0x7fc8ec0cd2d0 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8f1200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec15b860 con 0x7fc8ec0cad00 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8e3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cc8d0 0x7fc8ec14ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8e3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cc8d0 0x7fc8ec14ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35332/0 (socket says 172.21.15.66:35332) 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8e3e00640 1 -- 172.21.15.66:0/1476792203 learned_addr learned my addr 172.21.15.66:0/1476792203 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.532+0000 7fc8e9e00640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8ec0cad00 0x7fc8ec0934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8ea800640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8ec0cd2d0 0x7fc8ec14f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e3e00640 1 -- 172.21.15.66:0/1476792203 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8ec0cad00 msgr2=0x7fc8ec0934d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e3e00640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8ec0cad00 0x7fc8ec0934d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e3e00640 1 -- 172.21.15.66:0/1476792203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8ec0cd2d0 msgr2=0x7fc8ec14f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e3e00640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8ec0cd2d0 0x7fc8ec14f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e3e00640 1 -- 172.21.15.66:0/1476792203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d40094d0 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e3e00640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cc8d0 0x7fc8ec14ec70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc8d401abe0 tx=0x7fc8d40192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:03.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e2000640 1 -- 172.21.15.66:0/1476792203 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40a95f0 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e2000640 1 -- 172.21.15.66:0/1476792203 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d40b1800 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e2000640 1 -- 172.21.15.66:0/1476792203 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40b1bb0 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8f1200640 1 -- 172.21.15.66:0/1476792203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec15bb40 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8f1200640 1 -- 172.21.15.66:0/1476792203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec093a10 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8f1200640 1 -- 172.21.15.66:0/1476792203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec066c30 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e2000640 1 -- 172.21.15.66:0/1476792203 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc8d40b1d10 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e2000640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc8c8066690 0x7fc8c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:03.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e2000640 1 -- 172.21.15.66:0/1476792203 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fc8d411e570 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.536+0000 7fc8e9e00640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc8c8066690 0x7fc8c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:03.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.540+0000 7fc8e9e00640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc8c8066690 0x7fc8c8068b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000f70 tx=0x7fc8dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:03.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.540+0000 7fc8e2000640 1 -- 172.21.15.66:0/1476792203 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc8d40eba90 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.736+0000 7fc8f1200640 1 -- 172.21.15.66:0/1476792203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8ec06a6d0 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.736+0000 7fc8e2000640 1 -- 172.21.15.66:0/1476792203 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fc8d40ef0b0 con 0x7fc8ec0cc8d0 2024-03-30T00:42:03.748 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:03.748 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3895007156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3895007156},{"type":"v1","addr":"172.21.15.156:6835","nonce":3895007156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/65993387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":65993387},{"type":"v1","addr":"172.21.15.156:6834","nonce":65993387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2013605775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2013605775},{"type":"v1","addr":"172.21.15.66:6835","nonce":2013605775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5898,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1228365362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1228365362},{"type":"v1","addr":"172.21.15.66:6837","nonce":1228365362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2024-03-30T00:42:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 19 2024-03-30T00:42:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 -- 172.21.15.66:0/1476792203 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc8c8066690 msgr2=0x7fc8c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc8c8066690 0x7fc8c8068b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000f70 tx=0x7fc8dc009040 comp rx=0 tx=0).stop 2024-03-30T00:42:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 -- 172.21.15.66:0/1476792203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cc8d0 msgr2=0x7fc8ec14ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cc8d0 0x7fc8ec14ec70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc8d401abe0 tx=0x7fc8d40192d0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 -- 172.21.15.66:0/1476792203 shutdown_connections 2024-03-30T00:42:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8ec0cd2d0 0x7fc8ec14f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc8c8066690 0x7fc8c8068b50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8ec0cc8d0 0x7fc8ec14ec70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 --2- 172.21.15.66:0/1476792203 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8ec0cad00 0x7fc8ec0934d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 -- 172.21.15.66:0/1476792203 >> 172.21.15.66:0/1476792203 conn(0x7fc8ec069f60 msgr2=0x7fc8ec154640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 -- 172.21.15.66:0/1476792203 shutdown_connections 2024-03-30T00:42:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.740+0000 7fc8f1200640 1 -- 172.21.15.66:0/1476792203 wait complete. 2024-03-30T00:42:03.769 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T00:42:03.769 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-30T00:42:03.769 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T00:42:03.769 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:42:03.769 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T00:42:03.770 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-30T00:42:03.770 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T00:42:03.770 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:42:03.770 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T00:42:03.770 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-30T00:42:03.770 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T00:42:03.770 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:42:03.771 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T00:42:03.771 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-30T00:42:03.771 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T00:42:03.771 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:42:03.782 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T00:42:03.782 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T00:42:03.784 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T00:42:03.784 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T00:42:03.785 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T00:42:03.785 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T00:42:03.786 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T00:42:03.786 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T00:42:03.830 INFO:tasks.ceph.mds.a:Started 2024-03-30T00:42:03.831 DEBUG:teuthology.parallel:result is None 2024-03-30T00:42:03.831 INFO:tasks.ceph.mds.d:Started 2024-03-30T00:42:03.831 DEBUG:teuthology.parallel:result is None 2024-03-30T00:42:03.832 INFO:tasks.ceph.mds.b:Started 2024-03-30T00:42:03.832 DEBUG:teuthology.parallel:result is None 2024-03-30T00:42:03.833 INFO:tasks.ceph.mds.c:Started 2024-03-30T00:42:03.833 DEBUG:teuthology.parallel:result is None 2024-03-30T00:42:03.833 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-30T00:42:03.913 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2024-03-30T00:42:03.913 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2024-03-30T00:42:03.915 INFO:tasks.ceph.mds.b.smithi156.stderr:2024-03-30T00:42:03.904+0000 7fcbcce00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-03-30T00:42:03.916 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:42:03.919 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:42:03.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.972+0000 7fe82ce00640 1 Processor -- start 2024-03-30T00:42:03.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.972+0000 7fe82ce00640 1 -- start start 2024-03-30T00:42:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe82ce00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe82806f8e0 0x7fe8280d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe82ce00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82806b0b0 con 0x7fe82806e800 2024-03-30T00:42:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe82ce00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82806b390 con 0x7fe8280cc560 2024-03-30T00:42:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe82ce00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe828160640 con 0x7fe82806f8e0 2024-03-30T00:42:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe81d400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe82806e800 0x7fe8280ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36338/0 (socket says 172.21.15.66:36338) 2024-03-30T00:42:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe81d400640 1 -- 172.21.15.66:0/2639441407 learned_addr learned my addr 172.21.15.66:0/2639441407 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe826800640 1 --2- 172.21.15.66:0/2639441407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe82806f8e0 0x7fe8280d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:03.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe825400640 1 -- 172.21.15.66:0/2639441407 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571391922 0 0) 0x7fe82806b0b0 con 0x7fe82806e800 2024-03-30T00:42:03.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe825400640 1 -- 172.21.15.66:0/2639441407 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe80c003530 con 0x7fe82806e800 2024-03-30T00:42:03.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe825400640 1 -- 172.21.15.66:0/2639441407 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436682131 0 0) 0x7fe82806b390 con 0x7fe8280cc560 2024-03-30T00:42:03.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe825400640 1 -- 172.21.15.66:0/2639441407 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe82806b0b0 con 0x7fe8280cc560 2024-03-30T00:42:03.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe826800640 1 -- 172.21.15.66:0/2639441407 >> v1:172.21.15.156:6789/0 conn(0x7fe8280cc560 legacy=0x7fe82806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe826800640 1 -- 172.21.15.66:0/2639441407 >> v1:172.21.15.66:6789/0 conn(0x7fe82806e800 legacy=0x7fe8280ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe826800640 1 -- 172.21.15.66:0/2639441407 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82806b670 con 0x7fe82806f8e0 2024-03-30T00:42:03.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.976+0000 7fe826800640 1 --2- 172.21.15.66:0/2639441407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe82806f8e0 0x7fe8280d1060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe820009dc0 tx=0x7fe82001c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b1ef357ad78bce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:03.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe825400640 1 -- 172.21.15.66:0/2639441407 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8200a5a20 con 0x7fe82806f8e0 2024-03-30T00:42:03.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe825400640 1 -- 172.21.15.66:0/2639441407 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8200a5b80 con 0x7fe82806f8e0 2024-03-30T00:42:03.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe825400640 1 -- 172.21.15.66:0/2639441407 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8200ad440 con 0x7fe82806f8e0 2024-03-30T00:42:03.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 -- 172.21.15.66:0/2639441407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe82806f8e0 msgr2=0x7fe8280d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 --2- 172.21.15.66:0/2639441407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe82806f8e0 0x7fe8280d1060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe820009dc0 tx=0x7fe82001c960 comp rx=0 tx=0).stop 2024-03-30T00:42:03.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 -- 172.21.15.66:0/2639441407 shutdown_connections 2024-03-30T00:42:03.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 --2- 172.21.15.66:0/2639441407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe82806f8e0 0x7fe8280d1060 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 -- 172.21.15.66:0/2639441407 >> 172.21.15.66:0/2639441407 conn(0x7fe8280d0540 msgr2=0x7fe8280d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:03.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 -- 172.21.15.66:0/2639441407 shutdown_connections 2024-03-30T00:42:03.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 -- 172.21.15.66:0/2639441407 wait complete. 2024-03-30T00:42:03.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 Processor -- start 2024-03-30T00:42:03.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 -- start start 2024-03-30T00:42:03.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe82806e800 0x7fe8280b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:03.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82806f8e0 0x7fe828153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:03.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8280cc560 0x7fe8280add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:03.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82815c350 con 0x7fe82806e800 2024-03-30T00:42:03.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82815d920 con 0x7fe82806f8e0 2024-03-30T00:42:03.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe82ce00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe828153ff0 con 0x7fe8280cc560 2024-03-30T00:42:03.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe825e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe82806e800 0x7fe8280b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:03.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe825e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe82806e800 0x7fe8280b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35380/0 (socket says 172.21.15.66:35380) 2024-03-30T00:42:03.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe825e00640 1 -- 172.21.15.66:0/895644560 learned_addr learned my addr 172.21.15.66:0/895644560 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:03.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe81d400640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82806f8e0 0x7fe828153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:03.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.980+0000 7fe826800640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8280cc560 0x7fe8280add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:03.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe825e00640 1 -- 172.21.15.66:0/895644560 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8280cc560 msgr2=0x7fe8280add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe825e00640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8280cc560 0x7fe8280add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe825e00640 1 -- 172.21.15.66:0/895644560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82806f8e0 msgr2=0x7fe828153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:03.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe825e00640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82806f8e0 0x7fe828153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:03.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe825e00640 1 -- 172.21.15.66:0/895644560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820009a70 con 0x7fe82806e800 2024-03-30T00:42:03.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe825e00640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe82806e800 0x7fe8280b3370 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe81800c430 tx=0x7fe81800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:03.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe81ea00640 1 -- 172.21.15.66:0/895644560 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8180052d0 con 0x7fe82806e800 2024-03-30T00:42:03.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe81ea00640 1 -- 172.21.15.66:0/895644560 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe818005450 con 0x7fe82806e800 2024-03-30T00:42:03.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe81ea00640 1 -- 172.21.15.66:0/895644560 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818003840 con 0x7fe82806e800 2024-03-30T00:42:03.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe82ce00640 1 -- 172.21.15.66:0/895644560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82815d5f0 con 0x7fe82806e800 2024-03-30T00:42:03.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe82ce00640 1 -- 172.21.15.66:0/895644560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8280b3a70 con 0x7fe82806e800 2024-03-30T00:42:03.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe82ce00640 1 -- 172.21.15.66:0/895644560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe828066c30 con 0x7fe82806e800 2024-03-30T00:42:03.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe81ea00640 1 -- 172.21.15.66:0/895644560 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe8180039a0 con 0x7fe82806e800 2024-03-30T00:42:03.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe81ea00640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe804066690 0x7fe804068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:03.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe81ea00640 1 -- 172.21.15.66:0/895644560 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe8180835f0 con 0x7fe82806e800 2024-03-30T00:42:03.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.984+0000 7fe81d400640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe804066690 0x7fe804068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:03.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.988+0000 7fe81d400640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe804066690 0x7fe804068b50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe8280cb750 tx=0x7fe810007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:04.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:03.988+0000 7fe81ea00640 1 -- 172.21.15.66:0/895644560 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe818050b30 con 0x7fe82806e800 2024-03-30T00:42:04.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.160+0000 7fe82ce00640 1 -- 172.21.15.66:0/895644560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe8280cebb0 con 0x7fe82806e800 2024-03-30T00:42:04.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe81ea00640 1 -- 172.21.15.66:0/895644560 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe818054150 con 0x7fe82806e800 2024-03-30T00:42:04.945 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-03-30T00:42:04.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 -- 172.21.15.66:0/895644560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe804066690 msgr2=0x7fe804068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:04.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe804066690 0x7fe804068b50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe8280cb750 tx=0x7fe810007d40 comp rx=0 tx=0).stop 2024-03-30T00:42:04.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 -- 172.21.15.66:0/895644560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe82806e800 msgr2=0x7fe8280b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:04.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe82806e800 0x7fe8280b3370 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe81800c430 tx=0x7fe81800c8f0 comp rx=0 tx=0).stop 2024-03-30T00:42:04.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 -- 172.21.15.66:0/895644560 shutdown_connections 2024-03-30T00:42:04.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8280cc560 0x7fe8280add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:04.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82806f8e0 0x7fe828153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:04.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe804066690 0x7fe804068b50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:04.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 --2- 172.21.15.66:0/895644560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe82806e800 0x7fe8280b3370 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:04.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 -- 172.21.15.66:0/895644560 >> 172.21.15.66:0/895644560 conn(0x7fe8280d0540 msgr2=0x7fe828157470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:04.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 -- 172.21.15.66:0/895644560 shutdown_connections 2024-03-30T00:42:04.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:04.936+0000 7fe82ce00640 1 -- 172.21.15.66:0/895644560 wait complete. 2024-03-30T00:42:04.972 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-30T00:42:05.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb20b600640 1 Processor -- start 2024-03-30T00:42:05.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb20b600640 1 -- start start 2024-03-30T00:42:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb20b600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2040cb7d0 0x7fb204069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb20b600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb2040ccab0 0x7fb20406a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb20b600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2040cc620 con 0x7fb2040cebb0 2024-03-30T00:42:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb20b600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20406ef50 con 0x7fb2040cb7d0 2024-03-30T00:42:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb20b600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2040cbd30 con 0x7fb2040ccab0 2024-03-30T00:42:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb203e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2040cb7d0 0x7fb204069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb203e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2040cb7d0 0x7fb204069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35384/0 (socket says 172.21.15.66:35384) 2024-03-30T00:42:05.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb203e00640 1 -- 172.21.15.66:0/1730357534 learned_addr learned my addr 172.21.15.66:0/1730357534 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:05.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb209600640 1 --2- 172.21.15.66:0/1730357534 >> v2:172.21.15.156:3300/0 conn(0x7fb2040ccab0 0x7fb20406a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb203e00640 1 -- 172.21.15.66:0/1730357534 >> v2:172.21.15.156:3300/0 conn(0x7fb2040ccab0 msgr2=0x7fb20406a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:05.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb203e00640 1 --2- 172.21.15.66:0/1730357534 >> v2:172.21.15.156:3300/0 conn(0x7fb2040ccab0 0x7fb20406a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb203e00640 1 -- 172.21.15.66:0/1730357534 >> v1:172.21.15.156:6789/0 conn(0x7fb2040cebb0 legacy=0x7fb20406e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:05.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.108+0000 7fb203e00640 1 -- 172.21.15.66:0/1730357534 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2040cbfd0 con 0x7fb2040cb7d0 2024-03-30T00:42:05.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb203e00640 1 --2- 172.21.15.66:0/1730357534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2040cb7d0 0x7fb204069d90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb204071ea0 tx=0x7fb1f001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155ccbeca92ed96 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb203400640 1 -- 172.21.15.66:0/1730357534 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00ad660 con 0x7fb2040cb7d0 2024-03-30T00:42:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb203400640 1 -- 172.21.15.66:0/1730357534 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f00ad7c0 con 0x7fb2040cb7d0 2024-03-30T00:42:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb203400640 1 -- 172.21.15.66:0/1730357534 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00adb50 con 0x7fb2040cb7d0 2024-03-30T00:42:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb20b600640 1 -- 172.21.15.66:0/1730357534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2040cb7d0 msgr2=0x7fb204069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb20b600640 1 --2- 172.21.15.66:0/1730357534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2040cb7d0 0x7fb204069d90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb204071ea0 tx=0x7fb1f001c450 comp rx=0 tx=0).stop 2024-03-30T00:42:05.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb20b600640 1 -- 172.21.15.66:0/1730357534 shutdown_connections 2024-03-30T00:42:05.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb20b600640 1 --2- 172.21.15.66:0/1730357534 >> v2:172.21.15.156:3300/0 conn(0x7fb2040ccab0 0x7fb20406a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb20b600640 1 --2- 172.21.15.66:0/1730357534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2040cb7d0 0x7fb204069d90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb20b600640 1 -- 172.21.15.66:0/1730357534 >> 172.21.15.66:0/1730357534 conn(0x7fb204098a20 msgr2=0x7fb2040b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:05.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb20b600640 1 -- 172.21.15.66:0/1730357534 shutdown_connections 2024-03-30T00:42:05.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.112+0000 7fb20b600640 1 -- 172.21.15.66:0/1730357534 wait complete. 2024-03-30T00:42:05.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb20b600640 1 Processor -- start 2024-03-30T00:42:05.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb20b600640 1 -- start start 2024-03-30T00:42:05.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb20b600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2040cb7d0 0x7fb2040c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb20b600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2040ccab0 0x7fb204154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb20b600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2040cebb0 0x7fb20415ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb20b600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040b2f10 con 0x7fb2040ccab0 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb20b600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040c7130 con 0x7fb2040cebb0 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb20b600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040b2300 con 0x7fb2040cb7d0 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb203e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2040ccab0 0x7fb204154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb203e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2040ccab0 0x7fb204154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35396/0 (socket says 172.21.15.66:35396) 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb203e00640 1 -- 172.21.15.66:0/3128972214 learned_addr learned my addr 172.21.15.66:0/3128972214 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb208c00640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2040cb7d0 0x7fb2040c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb209600640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2040cebb0 0x7fb20415ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb203e00640 1 -- 172.21.15.66:0/3128972214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2040cb7d0 msgr2=0x7fb2040c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb203e00640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2040cb7d0 0x7fb2040c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb203e00640 1 -- 172.21.15.66:0/3128972214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2040cebb0 msgr2=0x7fb20415ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb203e00640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2040cebb0 0x7fb20415ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb203e00640 1 -- 172.21.15.66:0/3128972214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f00094d0 con 0x7fb2040ccab0 2024-03-30T00:42:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb209600640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2040cebb0 0x7fb20415ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb203e00640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2040ccab0 0x7fb204154d00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb1f001a0e0 tx=0x7fb1f001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:05.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb200c00640 1 -- 172.21.15.66:0/3128972214 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00b8620 con 0x7fb2040ccab0 2024-03-30T00:42:05.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb200c00640 1 -- 172.21.15.66:0/3128972214 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f00b8780 con 0x7fb2040ccab0 2024-03-30T00:42:05.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb20b600640 1 -- 172.21.15.66:0/3128972214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2040b25e0 con 0x7fb2040ccab0 2024-03-30T00:42:05.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb200c00640 1 -- 172.21.15.66:0/3128972214 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00b8b30 con 0x7fb2040ccab0 2024-03-30T00:42:05.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb20b600640 1 -- 172.21.15.66:0/3128972214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2040b2ba0 con 0x7fb2040ccab0 2024-03-30T00:42:05.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.116+0000 7fb20b600640 1 -- 172.21.15.66:0/3128972214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2040b3080 con 0x7fb2040ccab0 2024-03-30T00:42:05.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.120+0000 7fb200c00640 1 -- 172.21.15.66:0/3128972214 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb1f00b4040 con 0x7fb2040ccab0 2024-03-30T00:42:05.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.120+0000 7fb200c00640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1e4066690 0x7fb1e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.120+0000 7fb200c00640 1 -- 172.21.15.66:0/3128972214 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb1f011e810 con 0x7fb2040ccab0 2024-03-30T00:42:05.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.120+0000 7fb208c00640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1e4066690 0x7fb1e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.120+0000 7fb208c00640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1e4066690 0x7fb1e4068b50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb2040b17f0 tx=0x7fb1f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:05.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.124+0000 7fb200c00640 1 -- 172.21.15.66:0/3128972214 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb1f00ebcf0 con 0x7fb2040ccab0 2024-03-30T00:42:05.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.376+0000 7fb20b600640 1 -- 172.21.15.66:0/3128972214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb204066c30 con 0x7fb2040ccab0 2024-03-30T00:42:05.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.376+0000 7fb200c00640 1 -- 172.21.15.66:0/3128972214 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb1f00ef310 con 0x7fb2040ccab0 2024-03-30T00:42:05.388 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:05.388 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:42:05.388 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-03-30T00:42:05.388 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.388 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-03-30T00:42:05.388 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm857MDRAAu2pldF20deZfoSDDXuTF/A==", 2024-03-30T00:42:05.388 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.388 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:42:05.388 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMmNFGhAAlHS6ZpIoJc7tlYazCISiLw==", 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.389 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmuiBsFRAA2kez6ylAylKTWsQ1S2cahA==", 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm/oV3IRAAWn/zG1cLK+Raa2pYPzHd7Q==", 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:42:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAbXwdmf3HtOhAAyRcyb0YUJcJca5fjmZ3Frw==", 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.391 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAfXwdml7cdBRAAs639bBHTPyfhfVZMPRMfYA==", 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAiXwdmeS6KChAAZjLAbKSL7Vswi3qQ4304bA==", 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAlXwdmeuigEBAAiPIPheowKBnn3UZWwpdr+Q==", 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQApXwdm+bQNHRAAAb/lufdb7Fe+EFevd23gtA==", 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:42:05.393 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAsXwdmReL1IhAAwCs1cS+P6J8VCOYdrnrPQg==", 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.394 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAvXwdmA15vKBAAmpnKrDM/Y0xuBGxdyjrkWA==", 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAyXwdmrl02LhAA6JfEnugoz6l4nsXl6/vq0Q==", 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.395 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmZjh7JRAA3EFOImM03iOljcwmA7/xWw==", 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmkZv0LhAAgwCqW25+J1VYO31mWwDtLQ==", 2024-03-30T00:42:05.396 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmLG1/KxAAmNIZ9ZXdqPRQjCXvz7wC9A==", 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:42:05.397 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmyJ/nNBAAIgiUoUCbTeLEjL1iaPFw1A==", 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAXXwdmwWxQLRAAoFyZnFsIo1NvQlzRN+UHmw==", 2024-03-30T00:42:05.398 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmEF+xCRAAqRVFguki881+4azP8lsf0g==", 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:05.399 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:05.400 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-03-30T00:42:05.400 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMg32BRAAUWBF+QbwRGsopcAoBay8wA==", 2024-03-30T00:42:05.400 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:42:05.400 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:42:05.400 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:42:05.400 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:42:05.400 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.400 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:05.400 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-03-30T00:42:05.400 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:42:05.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 -- 172.21.15.66:0/3128972214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1e4066690 msgr2=0x7fb1e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1e4066690 0x7fb1e4068b50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb2040b17f0 tx=0x7fb1f8009040 comp rx=0 tx=0).stop 2024-03-30T00:42:05.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 -- 172.21.15.66:0/3128972214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2040ccab0 msgr2=0x7fb204154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2040ccab0 0x7fb204154d00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb1f001a0e0 tx=0x7fb1f001acf0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 -- 172.21.15.66:0/3128972214 shutdown_connections 2024-03-30T00:42:05.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2040cebb0 0x7fb20415ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1e4066690 0x7fb1e4068b50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2040ccab0 0x7fb204154d00 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 --2- 172.21.15.66:0/3128972214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2040cb7d0 0x7fb2040c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 -- 172.21.15.66:0/3128972214 >> 172.21.15.66:0/3128972214 conn(0x7fb204098a20 msgr2=0x7fb2040ac230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:05.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 -- 172.21.15.66:0/3128972214 shutdown_connections 2024-03-30T00:42:05.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.380+0000 7fb20b600640 1 -- 172.21.15.66:0/3128972214 wait complete. 2024-03-30T00:42:05.409 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:42:05.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdba200640 1 Processor -- start 2024-03-30T00:42:05.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdba200640 1 -- start start 2024-03-30T00:42:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdba200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb415da90 0x7fbdb40ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdba200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb415ef70 con 0x7fbdb40c9040 2024-03-30T00:42:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdba200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb4070980 con 0x7fbdb40cdac0 2024-03-30T00:42:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdba200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4070c60 con 0x7fbdb415da90 2024-03-30T00:42:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fbdb40c9040 0x7fbdb415e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36352/0 (socket says 172.21.15.66:36352) 2024-03-30T00:42:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3e00640 1 -- 172.21.15.66:0/1359848735 learned_addr learned my addr 172.21.15.66:0/1359848735 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:05.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb8e00640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb415da90 0x7fbdb40ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769677205 0 0) 0x7fbdb415ef70 con 0x7fbdb40c9040 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd9c003530 con 0x7fbdb40c9040 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1048697577 0 0) 0x7fbd9c003530 con 0x7fbdb40c9040 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb415da90 msgr2=0x7fbdb40ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb415da90 0x7fbdb40ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 >> v1:172.21.15.156:6789/0 conn(0x7fbdb40cdac0 legacy=0x7fbdb415e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4070f40 con 0x7fbdb40c9040 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fbda0001680 con 0x7fbdb40c9040 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 >> v1:172.21.15.66:6789/0 conn(0x7fbdb40c9040 legacy=0x7fbdb415e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb415da90 0x7fbd9c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c003ef0 0x7fbd9c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c004d10 0x7fbd9c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4070c60 con 0x7fbd9c003ef0 2024-03-30T00:42:05.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c00b9f0 con 0x7fbd9c004d10 2024-03-30T00:42:05.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c00bbb0 con 0x7fbdb415da90 2024-03-30T00:42:05.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3e00640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c003ef0 0x7fbd9c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb8e00640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb415da90 0x7fbd9c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb9800640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c004d10 0x7fbd9c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb8e00640 1 -- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c004d10 msgr2=0x7fbd9c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:05.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb8e00640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c004d10 0x7fbd9c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb8e00640 1 -- 172.21.15.66:0/1359848735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c003ef0 msgr2=0x7fbd9c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:05.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb8e00640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c003ef0 0x7fbd9c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb8e00640 1 -- 172.21.15.66:0/1359848735 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c003530 con 0x7fbdb415da90 2024-03-30T00:42:05.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.540+0000 7fbdb3e00640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c003ef0 0x7fbd9c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:42:05.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.544+0000 7fbdb8e00640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb415da90 0x7fbd9c0039b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009ae0 tx=0x7fbda801c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad2981444b439a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:05.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.544+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda80a64b0 con 0x7fbdb415da90 2024-03-30T00:42:05.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.544+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda80a6610 con 0x7fbdb415da90 2024-03-30T00:42:05.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.544+0000 7fbdb3400640 1 -- 172.21.15.66:0/1359848735 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda80a69c0 con 0x7fbdb415da90 2024-03-30T00:42:05.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.544+0000 7fbdba200640 1 -- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb415da90 msgr2=0x7fbd9c0039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.544+0000 7fbdba200640 1 --2- 172.21.15.66:0/1359848735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb415da90 0x7fbd9c0039b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009ae0 tx=0x7fbda801c9c0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.544+0000 7fbdb9800640 1 -- 172.21.15.66:0/1359848735 reap_dead start 2024-03-30T00:42:05.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.544+0000 7fbdba200640 1 -- 172.21.15.66:0/1359848735 shutdown_connections 2024-03-30T00:42:05.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.544+0000 7fbdba200640 1 -- 172.21.15.66:0/1359848735 >> 172.21.15.66:0/1359848735 conn(0x7fbdb40ce9f0 msgr2=0x7fbdb40cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:05.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.544+0000 7fbdba200640 1 -- 172.21.15.66:0/1359848735 shutdown_connections 2024-03-30T00:42:05.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.544+0000 7fbdba200640 1 -- 172.21.15.66:0/1359848735 wait complete. 2024-03-30T00:42:05.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdba200640 1 Processor -- start 2024-03-30T00:42:05.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdba200640 1 -- start start 2024-03-30T00:42:05.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdba200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdb415da90 0x7fbdb40c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdba200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdb4146360 0x7fbdb41537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdba200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb4155200 0x7fbdb41556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdba200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4154d70 con 0x7fbdb415da90 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdba200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4154160 con 0x7fbdb4146360 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdba200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4154440 con 0x7fbdb4155200 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb8e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdb415da90 0x7fbdb40c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb8e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdb415da90 0x7fbdb40c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35414/0 (socket says 172.21.15.66:35414) 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb8e00640 1 -- 172.21.15.66:0/1831533899 learned_addr learned my addr 172.21.15.66:0/1831533899 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb9800640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb4155200 0x7fbdb41556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb3e00640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdb4146360 0x7fbdb41537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb8e00640 1 -- 172.21.15.66:0/1831533899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb4155200 msgr2=0x7fbdb41556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb8e00640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb4155200 0x7fbdb41556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb8e00640 1 -- 172.21.15.66:0/1831533899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdb4146360 msgr2=0x7fbdb41537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb8e00640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdb4146360 0x7fbdb41537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb8e00640 1 -- 172.21.15.66:0/1831533899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4154720 con 0x7fbdb415da90 2024-03-30T00:42:05.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb8e00640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdb415da90 0x7fbdb40c4640 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbda801c7f0 tx=0x7fbd9c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:05.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb0c00640 1 -- 172.21.15.66:0/1831533899 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8005d60 con 0x7fbdb415da90 2024-03-30T00:42:05.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb0c00640 1 -- 172.21.15.66:0/1831533899 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda80a6780 con 0x7fbdb415da90 2024-03-30T00:42:05.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdb0c00640 1 -- 172.21.15.66:0/1831533899 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda80a6b30 con 0x7fbdb415da90 2024-03-30T00:42:05.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdba200640 1 -- 172.21.15.66:0/1831533899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb4154a00 con 0x7fbdb415da90 2024-03-30T00:42:05.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.548+0000 7fbdba200640 1 -- 172.21.15.66:0/1831533899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb4153d30 con 0x7fbdb415da90 2024-03-30T00:42:05.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.552+0000 7fbdba200640 1 -- 172.21.15.66:0/1831533899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb4153f20 con 0x7fbdb415da90 2024-03-30T00:42:05.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.552+0000 7fbdb0c00640 1 -- 172.21.15.66:0/1831533899 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbda80a6c90 con 0x7fbdb415da90 2024-03-30T00:42:05.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.552+0000 7fbdb0c00640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbd940666c0 0x7fbd94068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.552+0000 7fbdb0c00640 1 -- 172.21.15.66:0/1831533899 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fbda80aa070 con 0x7fbdb415da90 2024-03-30T00:42:05.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.552+0000 7fbdb3e00640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbd940666c0 0x7fbd94068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.552+0000 7fbdb3e00640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbd940666c0 0x7fbd94068b80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbdb41f8980 tx=0x7fbda0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:05.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.556+0000 7fbdb0c00640 1 -- 172.21.15.66:0/1831533899 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbdb4153f20 con 0x7fbdb415da90 2024-03-30T00:42:05.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdba200640 1 -- 172.21.15.66:0/1831533899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdb4153f20 con 0x7fbdb415da90 2024-03-30T00:42:05.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdb0c00640 1 -- 172.21.15.66:0/1831533899 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+12594 (secure 0 0 0) 0x7fbdb4153f20 con 0x7fbdb415da90 2024-03-30T00:42:05.731 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:05.731 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":50,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:42:04.935640+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:42:05.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdba200640 1 -- 172.21.15.66:0/1831533899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbd940666c0 msgr2=0x7fbd94068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdba200640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbd940666c0 0x7fbd94068b80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbdb41f8980 tx=0x7fbda0009040 comp rx=0 tx=0).stop 2024-03-30T00:42:05.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdba200640 1 -- 172.21.15.66:0/1831533899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdb415da90 msgr2=0x7fbdb40c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdba200640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdb415da90 0x7fbdb40c4640 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbda801c7f0 tx=0x7fbd9c005800 comp rx=0 tx=0).stop 2024-03-30T00:42:05.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdba200640 1 -- 172.21.15.66:0/1831533899 shutdown_connections 2024-03-30T00:42:05.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdba200640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbdb4155200 0x7fbdb41556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdba200640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdb4146360 0x7fbdb41537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdba200640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbd940666c0 0x7fbd94068b80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdba200640 1 --2- 172.21.15.66:0/1831533899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdb415da90 0x7fbdb40c4640 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.720+0000 7fbdba200640 1 -- 172.21.15.66:0/1831533899 >> 172.21.15.66:0/1831533899 conn(0x7fbdb40ce9f0 msgr2=0x7fbdb406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:05.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.724+0000 7fbdba200640 1 -- 172.21.15.66:0/1831533899 shutdown_connections 2024-03-30T00:42:05.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.724+0000 7fbdba200640 1 -- 172.21.15.66:0/1831533899 wait complete. 2024-03-30T00:42:05.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:42:05.753 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:42:05.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.884+0000 7fc40ac00640 1 Processor -- start 2024-03-30T00:42:05.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc40ac00640 1 -- start start 2024-03-30T00:42:05.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc40ac00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc4040cf320 0x7fc40415eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc40ac00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404099060 0x7fc40415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc40ac00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc404099e70 con 0x7fc404098c80 2024-03-30T00:42:05.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc40ac00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4040cbd90 con 0x7fc4040cf320 2024-03-30T00:42:05.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc40ac00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4040cc070 con 0x7fc404099060 2024-03-30T00:42:05.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc3fb400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc4040cf320 0x7fc40415eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc408c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404099060 0x7fc40415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc3fb400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc4040cf320 0x7fc40415eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:53004/0 (socket says 172.21.15.66:53004) 2024-03-30T00:42:05.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc3fb400640 1 -- 172.21.15.66:0/276378994 learned_addr learned my addr 172.21.15.66:0/276378994 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:05.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc408c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404099060 0x7fc40415f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:56180/0 (socket says 172.21.15.66:56180) 2024-03-30T00:42:05.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc403e00640 1 --1- 172.21.15.66:0/276378994 >> v1:172.21.15.156:6789/0 conn(0x7fc404098c80 0x7fc404099760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:60592/0 (socket says 172.21.15.66:60592) 2024-03-30T00:42:05.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc408c00640 1 -- 172.21.15.66:0/276378994 >> v2:172.21.15.156:3300/0 conn(0x7fc4040cf320 msgr2=0x7fc40415eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:05.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc408c00640 1 --2- 172.21.15.66:0/276378994 >> v2:172.21.15.156:3300/0 conn(0x7fc4040cf320 0x7fc40415eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc408c00640 1 -- 172.21.15.66:0/276378994 >> v1:172.21.15.156:6789/0 conn(0x7fc404098c80 legacy=0x7fc404099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.888+0000 7fc408c00640 1 -- 172.21.15.66:0/276378994 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4040cc350 con 0x7fc404099060 2024-03-30T00:42:05.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc408c00640 1 --2- 172.21.15.66:0/276378994 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404099060 0x7fc40415f0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009d90 tx=0x7fc3fc01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aefb78414354e17 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:05.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc403400640 1 -- 172.21.15.66:0/276378994 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0af600 con 0x7fc404099060 2024-03-30T00:42:05.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc403400640 1 -- 172.21.15.66:0/276378994 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc0ab050 con 0x7fc404099060 2024-03-30T00:42:05.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc403400640 1 -- 172.21.15.66:0/276378994 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0af980 con 0x7fc404099060 2024-03-30T00:42:05.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc40ac00640 1 -- 172.21.15.66:0/276378994 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404099060 msgr2=0x7fc40415f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc40ac00640 1 --2- 172.21.15.66:0/276378994 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404099060 0x7fc40415f0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009d90 tx=0x7fc3fc01f150 comp rx=0 tx=0).stop 2024-03-30T00:42:05.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc40ac00640 1 -- 172.21.15.66:0/276378994 shutdown_connections 2024-03-30T00:42:05.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc40ac00640 1 --2- 172.21.15.66:0/276378994 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404099060 0x7fc40415f0d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc40ac00640 1 --2- 172.21.15.66:0/276378994 >> v2:172.21.15.156:3300/0 conn(0x7fc4040cf320 0x7fc40415eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc40ac00640 1 -- 172.21.15.66:0/276378994 >> 172.21.15.66:0/276378994 conn(0x7fc4040b2130 msgr2=0x7fc4040b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:05.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc40ac00640 1 -- 172.21.15.66:0/276378994 shutdown_connections 2024-03-30T00:42:05.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc40ac00640 1 -- 172.21.15.66:0/276378994 wait complete. 2024-03-30T00:42:05.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc40ac00640 1 Processor -- start 2024-03-30T00:42:05.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.892+0000 7fc40ac00640 1 -- start start 2024-03-30T00:42:05.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc40ac00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404098c80 0x7fc404159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc40ac00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc404099060 0x7fc4040c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc40ac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040cf320 0x7fc4040b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc40ac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4041f21b0 con 0x7fc4040cf320 2024-03-30T00:42:05.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc40ac00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc404159c50 con 0x7fc404099060 2024-03-30T00:42:05.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc40ac00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4041f2fb0 con 0x7fc404098c80 2024-03-30T00:42:05.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc408c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040cf320 0x7fc4040b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc408c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040cf320 0x7fc4040b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35428/0 (socket says 172.21.15.66:35428) 2024-03-30T00:42:05.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc408c00640 1 -- 172.21.15.66:0/2787645334 learned_addr learned my addr 172.21.15.66:0/2787645334 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc3fb400640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc404099060 0x7fc4040c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc403e00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404098c80 0x7fc404159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc408c00640 1 -- 172.21.15.66:0/2787645334 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404098c80 msgr2=0x7fc404159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc408c00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404098c80 0x7fc404159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc408c00640 1 -- 172.21.15.66:0/2787645334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc404099060 msgr2=0x7fc4040c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc408c00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc404099060 0x7fc4040c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc408c00640 1 -- 172.21.15.66:0/2787645334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc009a70 con 0x7fc4040cf320 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc403e00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404098c80 0x7fc404159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc3fb400640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc404099060 0x7fc4040c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc408c00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040cf320 0x7fc4040b5a80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0b2040 tx=0x7fc3fc01bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc400c00640 1 -- 172.21.15.66:0/2787645334 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc009450 con 0x7fc4040cf320 2024-03-30T00:42:05.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc400c00640 1 -- 172.21.15.66:0/2787645334 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc0b9410 con 0x7fc4040cf320 2024-03-30T00:42:05.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc400c00640 1 -- 172.21.15.66:0/2787645334 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0b97c0 con 0x7fc4040cf320 2024-03-30T00:42:05.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc40ac00640 1 -- 172.21.15.66:0/2787645334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4041f23a0 con 0x7fc4040cf320 2024-03-30T00:42:05.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc40ac00640 1 -- 172.21.15.66:0/2787645334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4041f2960 con 0x7fc4040cf320 2024-03-30T00:42:05.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.896+0000 7fc40ac00640 1 -- 172.21.15.66:0/2787645334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4040677d0 con 0x7fc4040cf320 2024-03-30T00:42:05.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.900+0000 7fc400c00640 1 -- 172.21.15.66:0/2787645334 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc3fc0b6050 con 0x7fc4040cf320 2024-03-30T00:42:05.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.900+0000 7fc400c00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc3e0066690 0x7fc3e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:05.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.900+0000 7fc400c00640 1 -- 172.21.15.66:0/2787645334 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc3fc1224c0 con 0x7fc4040cf320 2024-03-30T00:42:05.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.900+0000 7fc403e00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc3e0066690 0x7fc3e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:05.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.900+0000 7fc403e00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc3e0066690 0x7fc3e0068b50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4000f70 tx=0x7fc3f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:05.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:05.904+0000 7fc400c00640 1 -- 172.21.15.66:0/2787645334 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc3fc0efa20 con 0x7fc4040cf320 2024-03-30T00:42:06.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc40ac00640 1 -- 172.21.15.66:0/2787645334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4041f3140 con 0x7fc4040cf320 2024-03-30T00:42:06.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc400c00640 1 -- 172.21.15.66:0/2787645334 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3fc0f3040 con 0x7fc4040cf320 2024-03-30T00:42:06.075 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc40ac00640 1 -- 172.21.15.66:0/2787645334 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc3e0066690 msgr2=0x7fc3e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc40ac00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc3e0066690 0x7fc3e0068b50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4000f70 tx=0x7fc3f4009040 comp rx=0 tx=0).stop 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc40ac00640 1 -- 172.21.15.66:0/2787645334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040cf320 msgr2=0x7fc4040b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc40ac00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040cf320 0x7fc4040b5a80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0b2040 tx=0x7fc3fc01bef0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc40ac00640 1 -- 172.21.15.66:0/2787645334 shutdown_connections 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc40ac00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc3e0066690 0x7fc3e0068b50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc40ac00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4040cf320 0x7fc4040b5a80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc40ac00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc404099060 0x7fc4040c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc40ac00640 1 --2- 172.21.15.66:0/2787645334 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc404098c80 0x7fc404159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.064+0000 7fc40ac00640 1 -- 172.21.15.66:0/2787645334 >> 172.21.15.66:0/2787645334 conn(0x7fc4040b2130 msgr2=0x7fc4040ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.068+0000 7fc40ac00640 1 -- 172.21.15.66:0/2787645334 shutdown_connections 2024-03-30T00:42:06.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.068+0000 7fc40ac00640 1 -- 172.21.15.66:0/2787645334 wait complete. 2024-03-30T00:42:06.097 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T00:42:06.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.224+0000 7f9d12600640 1 Processor -- start 2024-03-30T00:42:06.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d12600640 1 -- start start 2024-03-30T00:42:06.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d12600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9d0c15a640 0x7f9d0c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d12600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0cb2c0 0x7f9d0c0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d12600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c06d070 con 0x7f9d0c0ce520 2024-03-30T00:42:06.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d12600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c0cbba0 con 0x7f9d0c15a640 2024-03-30T00:42:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d12600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0cee90 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d0b400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9d0c15a640 0x7f9d0c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d0be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0cb2c0 0x7f9d0c0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d0be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0cb2c0 0x7f9d0c0ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:56202/0 (socket says 172.21.15.66:56202) 2024-03-30T00:42:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d0be00640 1 -- 172.21.15.66:0/2335073175 learned_addr learned my addr 172.21.15.66:0/2335073175 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d0be00640 1 -- 172.21.15.66:0/2335073175 >> v2:172.21.15.156:3300/0 conn(0x7f9d0c15a640 msgr2=0x7f9d0c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d0be00640 1 --2- 172.21.15.66:0/2335073175 >> v2:172.21.15.156:3300/0 conn(0x7f9d0c15a640 0x7f9d0c06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d0be00640 1 -- 172.21.15.66:0/2335073175 >> v1:172.21.15.156:6789/0 conn(0x7f9d0c0ce520 legacy=0x7f9d0c0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d0be00640 1 -- 172.21.15.66:0/2335073175 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c06e2c0 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.228+0000 7f9d0b400640 1 --2- 172.21.15.66:0/2335073175 >> v2:172.21.15.156:3300/0 conn(0x7f9d0c15a640 0x7f9d0c06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:42:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d0be00640 1 --2- 172.21.15.66:0/2335073175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0cb2c0 0x7f9d0c0ce950 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009e50 tx=0x7f9d0401c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd4b1c24471ff179 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d0a000640 1 -- 172.21.15.66:0/2335073175 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d040a5760 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d0a000640 1 -- 172.21.15.66:0/2335073175 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d040a58c0 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d0a000640 1 -- 172.21.15.66:0/2335073175 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d040a5c50 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d12600640 1 -- 172.21.15.66:0/2335073175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0cb2c0 msgr2=0x7f9d0c0ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d12600640 1 --2- 172.21.15.66:0/2335073175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0cb2c0 0x7f9d0c0ce950 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009e50 tx=0x7f9d0401c9f0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d12600640 1 -- 172.21.15.66:0/2335073175 shutdown_connections 2024-03-30T00:42:06.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d12600640 1 --2- 172.21.15.66:0/2335073175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0cb2c0 0x7f9d0c0ce950 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d12600640 1 --2- 172.21.15.66:0/2335073175 >> v2:172.21.15.156:3300/0 conn(0x7f9d0c15a640 0x7f9d0c06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d12600640 1 -- 172.21.15.66:0/2335073175 >> 172.21.15.66:0/2335073175 conn(0x7f9d0c069f60 msgr2=0x7f9d0c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:06.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d12600640 1 -- 172.21.15.66:0/2335073175 shutdown_connections 2024-03-30T00:42:06.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.232+0000 7f9d12600640 1 -- 172.21.15.66:0/2335073175 wait complete. 2024-03-30T00:42:06.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d12600640 1 Processor -- start 2024-03-30T00:42:06.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d12600640 1 -- start start 2024-03-30T00:42:06.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d12600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d0c0cb2c0 0x7f9d0c157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d12600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0ce520 0x7f9d0c158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d12600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d0c15a640 0x7f9d0c159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d12600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c1e00 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d12600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c9ec0 con 0x7f9d0c15a640 2024-03-30T00:42:06.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d12600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0ca1a0 con 0x7f9d0c0ce520 2024-03-30T00:42:06.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d0b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d0c0cb2c0 0x7f9d0c157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d0b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d0c0cb2c0 0x7f9d0c157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35432/0 (socket says 172.21.15.66:35432) 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d0b400640 1 -- 172.21.15.66:0/3125913616 learned_addr learned my addr 172.21.15.66:0/3125913616 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d0be00640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d0c15a640 0x7f9d0c159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d0aa00640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0ce520 0x7f9d0c158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d0b400640 1 -- 172.21.15.66:0/3125913616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0ce520 msgr2=0x7f9d0c158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d0b400640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0ce520 0x7f9d0c158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d0b400640 1 -- 172.21.15.66:0/3125913616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d0c15a640 msgr2=0x7f9d0c159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d0b400640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d0c15a640 0x7f9d0c159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d0b400640 1 -- 172.21.15.66:0/3125913616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc009420 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d0b400640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d0c0cb2c0 0x7f9d0c157160 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00e980 tx=0x7f9cfc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d03400640 1 -- 172.21.15.66:0/3125913616 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc00ca00 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d03400640 1 -- 172.21.15.66:0/3125913616 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc00cb60 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d03400640 1 -- 172.21.15.66:0/3125913616 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc013470 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d12600640 1 -- 172.21.15.66:0/3125913616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d04009b30 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d12600640 1 -- 172.21.15.66:0/3125913616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c0ca760 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.236+0000 7f9d12600640 1 -- 172.21.15.66:0/3125913616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c066c30 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.240+0000 7f9d03400640 1 -- 172.21.15.66:0/3125913616 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9cfc0135d0 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.240+0000 7f9d03400640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9cec066690 0x7f9cec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.240+0000 7f9d03400640 1 -- 172.21.15.66:0/3125913616 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9cfc0842f0 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.240+0000 7f9d0aa00640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9cec066690 0x7f9cec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.240+0000 7f9d0aa00640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9cec066690 0x7f9cec068b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c06ea60 tx=0x7f9cf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:06.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.244+0000 7f9d03400640 1 -- 172.21.15.66:0/3125913616 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9cfc0517d0 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.408+0000 7f9d12600640 1 -- 172.21.15.66:0/3125913616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9d0c06de00 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d03400640 1 -- 172.21.15.66:0/3125913616 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9cfc054df0 con 0x7f9d0c0cb2c0 2024-03-30T00:42:06.421 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:06.421 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:42:06.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:06.421 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-03-30T00:42:06.421 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-03-30T00:42:06.421 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:06.421 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:42:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 -- 172.21.15.66:0/3125913616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9cec066690 msgr2=0x7f9cec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9cec066690 0x7f9cec068b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c06ea60 tx=0x7f9cf8009040 comp rx=0 tx=0).stop 2024-03-30T00:42:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 -- 172.21.15.66:0/3125913616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d0c0cb2c0 msgr2=0x7f9d0c157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d0c0cb2c0 0x7f9d0c157160 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00e980 tx=0x7f9cfc00ee40 comp rx=0 tx=0).stop 2024-03-30T00:42:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 -- 172.21.15.66:0/3125913616 shutdown_connections 2024-03-30T00:42:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d0c15a640 0x7f9d0c159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d0c0ce520 0x7f9d0c158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9cec066690 0x7f9cec068b50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 --2- 172.21.15.66:0/3125913616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d0c0cb2c0 0x7f9d0c157160 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 -- 172.21.15.66:0/3125913616 >> 172.21.15.66:0/3125913616 conn(0x7f9d0c069f60 msgr2=0x7f9d0c06a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:06.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 -- 172.21.15.66:0/3125913616 shutdown_connections 2024-03-30T00:42:06.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.412+0000 7f9d12600640 1 -- 172.21.15.66:0/3125913616 wait complete. 2024-03-30T00:42:06.442 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T00:42:06.442 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T00:42:06.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3430000640 1 Processor -- start 2024-03-30T00:42:06.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3430000640 1 -- start start 2024-03-30T00:42:06.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3430000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3428098a20 0x7f3428098e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3430000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34280ce930 con 0x7f34280b0b20 2024-03-30T00:42:06.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3430000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3428099800 con 0x7f34280b1280 2024-03-30T00:42:06.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3430000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3428099370 con 0x7f3428098a20 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f342d600640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f34280b0b20 0x7f342806bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36360/0 (socket says 172.21.15.66:36360) 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f342d600640 1 -- 172.21.15.66:0/1066381495 learned_addr learned my addr 172.21.15.66:0/1066381495 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455220376 0 0) 0x7f34280ce930 con 0x7f34280b0b20 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3410003530 con 0x7f34280b0b20 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f342e000640 1 --2- 172.21.15.66:0/1066381495 >> v2:172.21.15.156:3300/0 conn(0x7f3428098a20 0x7f3428098e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 738878385 0 0) 0x7f3410003530 con 0x7f34280b0b20 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 >> v2:172.21.15.156:3300/0 conn(0x7f3428098a20 msgr2=0x7f3428098e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3427e00640 1 --2- 172.21.15.66:0/1066381495 >> v2:172.21.15.156:3300/0 conn(0x7f3428098a20 0x7f3428098e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 >> v1:172.21.15.156:6789/0 conn(0x7f34280b1280 legacy=0x7f34280cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3428099d50 con 0x7f34280b0b20 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f341c001680 con 0x7f34280b0b20 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 >> v1:172.21.15.66:6789/0 conn(0x7f34280b0b20 legacy=0x7f342806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.576+0000 7f3427e00640 1 --2- 172.21.15.66:0/1066381495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 0x7f3410003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f3427e00640 1 --2- 172.21.15.66:0/1066381495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34100047d0 0x7f3410004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f3427e00640 1 --2- 172.21.15.66:0/1066381495 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34100051e0 0x7f341000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3428099370 con 0x7f34100039d0 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341000c130 con 0x7f34100047d0 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341000c2f0 con 0x7f34100051e0 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f342d600640 1 --2- 172.21.15.66:0/1066381495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 0x7f3410003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f342e000640 1 --2- 172.21.15.66:0/1066381495 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34100051e0 0x7f341000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f342d600640 1 -- 172.21.15.66:0/1066381495 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34100051e0 msgr2=0x7f341000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f342d600640 1 --2- 172.21.15.66:0/1066381495 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34100051e0 0x7f341000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f342d600640 1 -- 172.21.15.66:0/1066381495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34100047d0 msgr2=0x7f3410004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f342d600640 1 --2- 172.21.15.66:0/1066381495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34100047d0 0x7f3410004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f342d600640 1 -- 172.21.15.66:0/1066381495 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3410003530 con 0x7f34100039d0 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f342e000640 1 --2- 172.21.15.66:0/1066381495 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34100051e0 0x7f341000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f342e000640 1 -- 172.21.15.66:0/1066381495 reap_dead start 2024-03-30T00:42:06.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f342d600640 1 --2- 172.21.15.66:0/1066381495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 0x7f3410003df0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c0e0 tx=0x7f341c01cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42805ad06793faef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:06.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c005ae0 con 0x7f34100039d0 2024-03-30T00:42:06.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c005c40 con 0x7f34100039d0 2024-03-30T00:42:06.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.580+0000 7f3427e00640 1 -- 172.21.15.66:0/1066381495 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c0072b0 con 0x7f34100039d0 2024-03-30T00:42:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 -- 172.21.15.66:0/1066381495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 msgr2=0x7f3410003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 --2- 172.21.15.66:0/1066381495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 0x7f3410003df0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c0e0 tx=0x7f341c01cb00 comp rx=0 tx=0).stop 2024-03-30T00:42:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 -- 172.21.15.66:0/1066381495 shutdown_connections 2024-03-30T00:42:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 --2- 172.21.15.66:0/1066381495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 0x7f3410003df0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 -- 172.21.15.66:0/1066381495 >> 172.21.15.66:0/1066381495 conn(0x7f34280b2150 msgr2=0x7f34280b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 -- 172.21.15.66:0/1066381495 shutdown_connections 2024-03-30T00:42:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 -- 172.21.15.66:0/1066381495 wait complete. 2024-03-30T00:42:06.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 Processor -- start 2024-03-30T00:42:06.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 -- start start 2024-03-30T00:42:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 0x7f34280b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34280c9300 0x7f34280c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f342814d3f0 0x7f342814d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34280c7730 con 0x7f34100039d0 2024-03-30T00:42:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34280c9da0 con 0x7f34280c9300 2024-03-30T00:42:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f3430000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34280c79d0 con 0x7f342814d3f0 2024-03-30T00:42:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f342d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 0x7f34280b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f342d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 0x7f34280b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35456/0 (socket says 172.21.15.66:35456) 2024-03-30T00:42:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f342d600640 1 -- 172.21.15.66:0/1290821434 learned_addr learned my addr 172.21.15.66:0/1290821434 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:06.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f342cc00640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34280c9300 0x7f34280c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f342e000640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f342814d3f0 0x7f342814d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f342d600640 1 -- 172.21.15.66:0/1290821434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f342814d3f0 msgr2=0x7f342814d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f342d600640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f342814d3f0 0x7f342814d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f342d600640 1 -- 172.21.15.66:0/1290821434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34280c9300 msgr2=0x7f34280c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f342d600640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34280c9300 0x7f34280c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f342d600640 1 -- 172.21.15.66:0/1290821434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34280c7cb0 con 0x7f34100039d0 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.584+0000 7f342d600640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 0x7f34280b34e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f341c01c930 tx=0x7f3410005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.588+0000 7f3425600640 1 -- 172.21.15.66:0/1290821434 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c0b4070 con 0x7f34100039d0 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.588+0000 7f3425600640 1 -- 172.21.15.66:0/1290821434 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c0059c0 con 0x7f34100039d0 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.588+0000 7f3425600640 1 -- 172.21.15.66:0/1290821434 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c007270 con 0x7f34100039d0 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.588+0000 7f3430000640 1 -- 172.21.15.66:0/1290821434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34280c7f90 con 0x7f34100039d0 2024-03-30T00:42:06.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.588+0000 7f3430000640 1 -- 172.21.15.66:0/1290821434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34280b3a20 con 0x7f34100039d0 2024-03-30T00:42:06.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.588+0000 7f3430000640 1 -- 172.21.15.66:0/1290821434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34280b1f50 con 0x7f34100039d0 2024-03-30T00:42:06.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.588+0000 7f3425600640 1 -- 172.21.15.66:0/1290821434 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f341c0073d0 con 0x7f34100039d0 2024-03-30T00:42:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.588+0000 7f3425600640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3408066690 0x7f3408068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.588+0000 7f3425600640 1 -- 172.21.15.66:0/1290821434 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f341c11d5c0 con 0x7f34100039d0 2024-03-30T00:42:06.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.588+0000 7f342cc00640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3408066690 0x7f3408068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:06.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.592+0000 7f342cc00640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3408066690 0x7f3408068b50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3414007fd0 tx=0x7f3414009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:06.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.592+0000 7f3425600640 1 -- 172.21.15.66:0/1290821434 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f341c0eab50 con 0x7f34100039d0 2024-03-30T00:42:06.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.756+0000 7f3430000640 1 -- 172.21.15.66:0/1290821434 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f34280b3c10 con 0x7f34100039d0 2024-03-30T00:42:06.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.856+0000 7f3425600640 1 -- 172.21.15.66:0/1290821434 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7f341c0ee170 con 0x7f34100039d0 2024-03-30T00:42:06.866 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-03-30T00:42:06.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 -- 172.21.15.66:0/1290821434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3408066690 msgr2=0x7f3408068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3408066690 0x7f3408068b50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3414007fd0 tx=0x7f3414009040 comp rx=0 tx=0).stop 2024-03-30T00:42:06.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 -- 172.21.15.66:0/1290821434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 msgr2=0x7f34280b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:06.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 0x7f34280b34e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f341c01c930 tx=0x7f3410005800 comp rx=0 tx=0).stop 2024-03-30T00:42:06.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 -- 172.21.15.66:0/1290821434 shutdown_connections 2024-03-30T00:42:06.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f342814d3f0 0x7f342814d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34280c9300 0x7f34280c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3408066690 0x7f3408068b50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 --2- 172.21.15.66:0/1290821434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34100039d0 0x7f34280b34e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:06.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 -- 172.21.15.66:0/1290821434 >> 172.21.15.66:0/1290821434 conn(0x7f34280b2150 msgr2=0x7f34280b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:06.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 -- 172.21.15.66:0/1290821434 shutdown_connections 2024-03-30T00:42:06.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:06.860+0000 7f3430000640 1 -- 172.21.15.66:0/1290821434 wait complete. 2024-03-30T00:42:06.893 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T00:42:07.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.028+0000 7f8184a00640 1 Processor -- start 2024-03-30T00:42:07.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.028+0000 7f8184a00640 1 -- start start 2024-03-30T00:42:07.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f8184a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f818015c960 0x7f81800ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:07.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f8184a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f81800cea10 0x7f81800cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:07.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f8184a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818006b530 con 0x7f818006eb80 2024-03-30T00:42:07.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f8184a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818015cd40 con 0x7f81800cea10 2024-03-30T00:42:07.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f8184a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818006b810 con 0x7f818015c960 2024-03-30T00:42:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f817e800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f818015c960 0x7f81800ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f8175e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f81800cea10 0x7f81800cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f817e800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f818015c960 0x7f81800ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:56236/0 (socket says 172.21.15.66:56236) 2024-03-30T00:42:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f8175e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f81800cea10 0x7f81800cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:53060/0 (socket says 172.21.15.66:53060) 2024-03-30T00:42:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f817f200640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f818006eb80 0x7f818006f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:60626/0 (socket says 172.21.15.66:60626) 2024-03-30T00:42:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f817e800640 1 -- 172.21.15.66:0/1767969749 learned_addr learned my addr 172.21.15.66:0/1767969749 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:07.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f817e800640 1 -- 172.21.15.66:0/1767969749 >> v2:172.21.15.156:3300/0 conn(0x7f81800cea10 msgr2=0x7f81800cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:07.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f817e800640 1 --2- 172.21.15.66:0/1767969749 >> v2:172.21.15.156:3300/0 conn(0x7f81800cea10 0x7f81800cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:07.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f817e800640 1 -- 172.21.15.66:0/1767969749 >> v1:172.21.15.156:6789/0 conn(0x7f818006eb80 legacy=0x7f818006f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:07.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f817de00640 1 -- 172.21.15.66:0/1767969749 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525202149 0 0) 0x7f818006b530 con 0x7f818006eb80 2024-03-30T00:42:07.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.032+0000 7f817e800640 1 -- 172.21.15.66:0/1767969749 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818006baf0 con 0x7f818015c960 2024-03-30T00:42:07.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f817e800640 1 --2- 172.21.15.66:0/1767969749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f818015c960 0x7f81800ccd60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8170009e50 tx=0x7f817001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5493f7c92a92d874 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:07.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f817de00640 1 -- 172.21.15.66:0/1767969749 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81700a5920 con 0x7f818015c960 2024-03-30T00:42:07.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f817de00640 1 -- 172.21.15.66:0/1767969749 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81700a5a80 con 0x7f818015c960 2024-03-30T00:42:07.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f817de00640 1 -- 172.21.15.66:0/1767969749 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81700a5e10 con 0x7f818015c960 2024-03-30T00:42:07.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f8184a00640 1 -- 172.21.15.66:0/1767969749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f818015c960 msgr2=0x7f81800ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:07.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f8184a00640 1 --2- 172.21.15.66:0/1767969749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f818015c960 0x7f81800ccd60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8170009e50 tx=0x7f817001ca30 comp rx=0 tx=0).stop 2024-03-30T00:42:07.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f8184a00640 1 -- 172.21.15.66:0/1767969749 shutdown_connections 2024-03-30T00:42:07.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f8184a00640 1 --2- 172.21.15.66:0/1767969749 >> v2:172.21.15.156:3300/0 conn(0x7f81800cea10 0x7f81800cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:07.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f8184a00640 1 --2- 172.21.15.66:0/1767969749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f818015c960 0x7f81800ccd60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:07.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f8184a00640 1 -- 172.21.15.66:0/1767969749 >> 172.21.15.66:0/1767969749 conn(0x7f818006e060 msgr2=0x7f818006e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:07.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f8184a00640 1 -- 172.21.15.66:0/1767969749 shutdown_connections 2024-03-30T00:42:07.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f8184a00640 1 -- 172.21.15.66:0/1767969749 wait complete. 2024-03-30T00:42:07.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f8184a00640 1 Processor -- start 2024-03-30T00:42:07.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.036+0000 7f8184a00640 1 -- start start 2024-03-30T00:42:07.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8184a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818006eb80 0x7f818013ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:07.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8184a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81800cea10 0x7f81800c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:07.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8184a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f818015c960 0x7f81801549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:07.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8184a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81800b3810 con 0x7f818015c960 2024-03-30T00:42:07.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8184a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81800b4370 con 0x7f818006eb80 2024-03-30T00:42:07.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8184a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81800b3a40 con 0x7f81800cea10 2024-03-30T00:42:07.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f817f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f818015c960 0x7f81801549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:07.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f817e800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818006eb80 0x7f818013ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:07.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f817f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f818015c960 0x7f81801549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35462/0 (socket says 172.21.15.66:35462) 2024-03-30T00:42:07.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f817f200640 1 -- 172.21.15.66:0/575877392 learned_addr learned my addr 172.21.15.66:0/575877392 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:07.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8175e00640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81800cea10 0x7f81800c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:07.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f817f200640 1 -- 172.21.15.66:0/575877392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81800cea10 msgr2=0x7f81800c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:07.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f817f200640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81800cea10 0x7f81800c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:07.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f817f200640 1 -- 172.21.15.66:0/575877392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818006eb80 msgr2=0x7f818013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:07.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f817f200640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818006eb80 0x7f818013ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:07.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f817f200640 1 -- 172.21.15.66:0/575877392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170009b30 con 0x7f818015c960 2024-03-30T00:42:07.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f817f200640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f818015c960 0x7f81801549d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f817800c410 tx=0x7f817800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:07.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8177400640 1 -- 172.21.15.66:0/575877392 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780074e0 con 0x7f818015c960 2024-03-30T00:42:07.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8177400640 1 -- 172.21.15.66:0/575877392 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8178007660 con 0x7f818015c960 2024-03-30T00:42:07.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8184a00640 1 -- 172.21.15.66:0/575877392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81800cabb0 con 0x7f818015c960 2024-03-30T00:42:07.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8177400640 1 -- 172.21.15.66:0/575877392 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178004a30 con 0x7f818015c960 2024-03-30T00:42:07.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8184a00640 1 -- 172.21.15.66:0/575877392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81800b4000 con 0x7f818015c960 2024-03-30T00:42:07.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.040+0000 7f8184a00640 1 -- 172.21.15.66:0/575877392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81800b4500 con 0x7f818015c960 2024-03-30T00:42:07.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.044+0000 7f8177400640 1 -- 172.21.15.66:0/575877392 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8178004c50 con 0x7f818015c960 2024-03-30T00:42:07.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.044+0000 7f8177400640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f815c066690 0x7f815c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:07.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.044+0000 7f8177400640 1 -- 172.21.15.66:0/575877392 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f81780843a0 con 0x7f818015c960 2024-03-30T00:42:07.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.044+0000 7f817e800640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f815c066690 0x7f815c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:07.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.044+0000 7f817e800640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f815c066690 0x7f815c068b50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f81700ae000 tx=0x7f81700a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:07.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.048+0000 7f8177400640 1 -- 172.21.15.66:0/575877392 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8178051840 con 0x7f818015c960 2024-03-30T00:42:07.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.208+0000 7f8184a00640 1 -- 172.21.15.66:0/575877392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8180073d60 con 0x7f818015c960 2024-03-30T00:42:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.864+0000 7f8177400640 1 -- 172.21.15.66:0/575877392 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8178054e60 con 0x7f818015c960 2024-03-30T00:42:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-03-30T00:42:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 -- 172.21.15.66:0/575877392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f815c066690 msgr2=0x7f815c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f815c066690 0x7f815c068b50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f81700ae000 tx=0x7f81700a7070 comp rx=0 tx=0).stop 2024-03-30T00:42:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 -- 172.21.15.66:0/575877392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f818015c960 msgr2=0x7f81801549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f818015c960 0x7f81801549d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f817800c410 tx=0x7f817800c810 comp rx=0 tx=0).stop 2024-03-30T00:42:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 -- 172.21.15.66:0/575877392 shutdown_connections 2024-03-30T00:42:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f815c066690 0x7f815c068b50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f818015c960 0x7f81801549d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81800cea10 0x7f81800c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 --2- 172.21.15.66:0/575877392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f818006eb80 0x7f818013ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 -- 172.21.15.66:0/575877392 >> 172.21.15.66:0/575877392 conn(0x7f818006e060 msgr2=0x7f81800b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 -- 172.21.15.66:0/575877392 shutdown_connections 2024-03-30T00:42:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:07.868+0000 7f8184a00640 1 -- 172.21.15.66:0/575877392 wait complete. 2024-03-30T00:42:07.916 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T00:42:08.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b5c600640 1 Processor -- start 2024-03-30T00:42:08.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b5c600640 1 -- start start 2024-03-30T00:42:08.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b5c600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b54160bc0 0x7f9b54167840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b5c600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5406da20 con 0x7f9b54157060 2024-03-30T00:42:08.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b5c600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5406dc60 con 0x7f9b541610c0 2024-03-30T00:42:08.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b5c600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b54157440 con 0x7f9b54160bc0 2024-03-30T00:42:08.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b59c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9b54157060 0x7f9b5406d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36364/0 (socket says 172.21.15.66:36364) 2024-03-30T00:42:08.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b59c00640 1 -- 172.21.15.66:0/1647465078 learned_addr learned my addr 172.21.15.66:0/1647465078 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:08.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b5a600640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b54160bc0 0x7f9b54167840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534128897 0 0) 0x7f9b48001120 con 0x7f9b54157060 2024-03-30T00:42:08.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b3c003530 con 0x7f9b54157060 2024-03-30T00:42:08.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1038836638 0 0) 0x7f9b3c003530 con 0x7f9b54157060 2024-03-30T00:42:08.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b54160bc0 msgr2=0x7f9b54167840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:08.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b53e00640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b54160bc0 0x7f9b54167840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 >> v1:172.21.15.156:6789/0 conn(0x7f9b541610c0 legacy=0x7f9b541604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5406df40 con 0x7f9b54157060 2024-03-30T00:42:08.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.052+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f9b48001810 con 0x7f9b54157060 2024-03-30T00:42:08.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 >> v1:172.21.15.66:6789/0 conn(0x7f9b54157060 legacy=0x7f9b5406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b53e00640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b54160bc0 0x7f9b3c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b53e00640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c003ef0 0x7f9b3c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b53e00640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b3c004d10 0x7f9b3c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b54157440 con 0x7f9b3c003ef0 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c00bb50 con 0x7f9b3c004d10 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c00bd10 con 0x7f9b54160bc0 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b59200640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c003ef0 0x7f9b3c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b59c00640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b54160bc0 0x7f9b3c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b5a600640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b3c004d10 0x7f9b3c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b59200640 1 -- 172.21.15.66:0/1647465078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b54160bc0 msgr2=0x7f9b3c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b59200640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b54160bc0 0x7f9b3c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b59200640 1 -- 172.21.15.66:0/1647465078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b3c004d10 msgr2=0x7f9b3c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b59200640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b3c004d10 0x7f9b3c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b59200640 1 -- 172.21.15.66:0/1647465078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c003530 con 0x7f9b3c003ef0 2024-03-30T00:42:08.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b59200640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c003ef0 0x7f9b3c0047d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9b541614c0 tx=0x7f9b4001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0101eec247e7601 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.056+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b400ab600 con 0x7f9b3c003ef0 2024-03-30T00:42:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b400a7050 con 0x7f9b3c003ef0 2024-03-30T00:42:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b53e00640 1 -- 172.21.15.66:0/1647465078 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b400ab9e0 con 0x7f9b3c003ef0 2024-03-30T00:42:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 -- 172.21.15.66:0/1647465078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c003ef0 msgr2=0x7f9b3c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 --2- 172.21.15.66:0/1647465078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c003ef0 0x7f9b3c0047d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9b541614c0 tx=0x7f9b4001c4c0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5a600640 1 -- 172.21.15.66:0/1647465078 reap_dead start 2024-03-30T00:42:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 -- 172.21.15.66:0/1647465078 shutdown_connections 2024-03-30T00:42:08.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 -- 172.21.15.66:0/1647465078 >> 172.21.15.66:0/1647465078 conn(0x7f9b54071750 msgr2=0x7f9b54069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:08.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 -- 172.21.15.66:0/1647465078 shutdown_connections 2024-03-30T00:42:08.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 -- 172.21.15.66:0/1647465078 wait complete. 2024-03-30T00:42:08.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 Processor -- start 2024-03-30T00:42:08.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 -- start start 2024-03-30T00:42:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b3c003ef0 0x7f9b54154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b54148420 0x7f9b54069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b54148970 0x7f9b54147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b54154cc0 con 0x7f9b54148970 2024-03-30T00:42:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b541549e0 con 0x7f9b54148420 2024-03-30T00:42:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5c600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b54153550 con 0x7f9b3c003ef0 2024-03-30T00:42:08.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b54148970 0x7f9b54147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b54148970 0x7f9b54147890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35490/0 (socket says 172.21.15.66:35490) 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5a600640 1 -- 172.21.15.66:0/3721966519 learned_addr learned my addr 172.21.15.66:0/3721966519 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b59c00640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b3c003ef0 0x7f9b54154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b59200640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b54148420 0x7f9b54069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5a600640 1 -- 172.21.15.66:0/3721966519 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b3c003ef0 msgr2=0x7f9b54154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.060+0000 7f9b5a600640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b3c003ef0 0x7f9b54154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b5a600640 1 -- 172.21.15.66:0/3721966519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b54148420 msgr2=0x7f9b54069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b5a600640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b54148420 0x7f9b54069ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b5a600640 1 -- 172.21.15.66:0/3721966519 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b54153830 con 0x7f9b54148970 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b59200640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b54148420 0x7f9b54069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b5a600640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b54148970 0x7f9b54147890 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9b540704e0 tx=0x7f9b4c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b51600640 1 -- 172.21.15.66:0/3721966519 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00e040 con 0x7f9b54148970 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b51600640 1 -- 172.21.15.66:0/3721966519 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c00f040 con 0x7f9b54148970 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b5c600640 1 -- 172.21.15.66:0/3721966519 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b54153b10 con 0x7f9b54148970 2024-03-30T00:42:08.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b51600640 1 -- 172.21.15.66:0/3721966519 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c003c00 con 0x7f9b54148970 2024-03-30T00:42:08.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b5c600640 1 -- 172.21.15.66:0/3721966519 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5406a530 con 0x7f9b54148970 2024-03-30T00:42:08.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b5c600640 1 -- 172.21.15.66:0/3721966519 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5406a720 con 0x7f9b54148970 2024-03-30T00:42:08.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b51600640 1 -- 172.21.15.66:0/3721966519 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9b4c004030 con 0x7f9b54148970 2024-03-30T00:42:08.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b51600640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b340666c0 0x7f9b34068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b51600640 1 -- 172.21.15.66:0/3721966519 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f9b4c0855d0 con 0x7f9b54148970 2024-03-30T00:42:08.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.064+0000 7f9b59c00640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b340666c0 0x7f9b34068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.068+0000 7f9b59c00640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b340666c0 0x7f9b34068b80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9b48007fd0 tx=0x7f9b48007e20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:08.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.068+0000 7f9b51600640 1 -- 172.21.15.66:0/3721966519 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9b4c0562d0 con 0x7f9b54148970 2024-03-30T00:42:08.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.264+0000 7f9b5c600640 1 -- 172.21.15.66:0/3721966519 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9b5406a720 con 0x7f9b54148970 2024-03-30T00:42:08.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.272+0000 7f9b51600640 1 -- 172.21.15.66:0/3721966519 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9b5406a720 con 0x7f9b54148970 2024-03-30T00:42:08.283 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 7 and data pool 8 2024-03-30T00:42:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.276+0000 7f9b5c600640 1 -- 172.21.15.66:0/3721966519 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b340666c0 msgr2=0x7f9b34068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.276+0000 7f9b5c600640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b340666c0 0x7f9b34068b80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9b48007fd0 tx=0x7f9b48007e20 comp rx=0 tx=0).stop 2024-03-30T00:42:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.276+0000 7f9b5c600640 1 -- 172.21.15.66:0/3721966519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b54148970 msgr2=0x7f9b54147890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.276+0000 7f9b5c600640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b54148970 0x7f9b54147890 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9b540704e0 tx=0x7f9b4c00f580 comp rx=0 tx=0).stop 2024-03-30T00:42:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.280+0000 7f9b5c600640 1 -- 172.21.15.66:0/3721966519 shutdown_connections 2024-03-30T00:42:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.280+0000 7f9b5c600640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b340666c0 0x7f9b34068b80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.280+0000 7f9b5c600640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b54148970 0x7f9b54147890 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.280+0000 7f9b5c600640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b54148420 0x7f9b54069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.280+0000 7f9b5c600640 1 --2- 172.21.15.66:0/3721966519 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b3c003ef0 0x7f9b54154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.280+0000 7f9b5c600640 1 -- 172.21.15.66:0/3721966519 >> 172.21.15.66:0/3721966519 conn(0x7f9b54071750 msgr2=0x7f9b54072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:08.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.280+0000 7f9b5c600640 1 -- 172.21.15.66:0/3721966519 shutdown_connections 2024-03-30T00:42:08.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.280+0000 7f9b5c600640 1 -- 172.21.15.66:0/3721966519 wait complete. 2024-03-30T00:42:08.330 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:42:08.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f368ca00640 1 Processor -- start 2024-03-30T00:42:08.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f368ca00640 1 -- start start 2024-03-30T00:42:08.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f368ca00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f36880cc560 0x7f36880d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f368ca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f368806f8e0 0x7f368806e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f368ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368806e630 0x7f36880ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f368ca00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3688160640 con 0x7f368806f8e0 2024-03-30T00:42:08.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f368ca00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368806eaa0 con 0x7f36880cc560 2024-03-30T00:42:08.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f368ca00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368806b0b0 con 0x7f368806e630 2024-03-30T00:42:08.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f367d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f368806f8e0 0x7f368806e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f367d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f368806f8e0 0x7f368806e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35494/0 (socket says 172.21.15.66:35494) 2024-03-30T00:42:08.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f367d400640 1 -- 172.21.15.66:0/3327913313 learned_addr learned my addr 172.21.15.66:0/3327913313 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f367d400640 1 -- 172.21.15.66:0/3327913313 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368806e630 msgr2=0x7f36880ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f367d400640 1 --2- 172.21.15.66:0/3327913313 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368806e630 0x7f36880ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f367d400640 1 -- 172.21.15.66:0/3327913313 >> v2:172.21.15.156:3300/0 conn(0x7f36880cc560 msgr2=0x7f36880d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f367d400640 1 --2- 172.21.15.66:0/3327913313 >> v2:172.21.15.156:3300/0 conn(0x7f36880cc560 0x7f36880d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.460+0000 7f367d400640 1 -- 172.21.15.66:0/3327913313 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368806b390 con 0x7f368806f8e0 2024-03-30T00:42:08.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f367d400640 1 --2- 172.21.15.66:0/3327913313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806f8e0 0x7f368806e0f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3688072c50 tx=0x7f367001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968367bc4013d7ec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f3685400640 1 -- 172.21.15.66:0/3327913313 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36700ab600 con 0x7f368806f8e0 2024-03-30T00:42:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f3685400640 1 -- 172.21.15.66:0/3327913313 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36700a7050 con 0x7f368806f8e0 2024-03-30T00:42:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f3685400640 1 -- 172.21.15.66:0/3327913313 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36700ab980 con 0x7f368806f8e0 2024-03-30T00:42:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f368ca00640 1 -- 172.21.15.66:0/3327913313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806f8e0 msgr2=0x7f368806e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f368ca00640 1 --2- 172.21.15.66:0/3327913313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806f8e0 0x7f368806e0f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3688072c50 tx=0x7f367001c230 comp rx=0 tx=0).stop 2024-03-30T00:42:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f368ca00640 1 -- 172.21.15.66:0/3327913313 shutdown_connections 2024-03-30T00:42:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f368ca00640 1 --2- 172.21.15.66:0/3327913313 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368806e630 0x7f36880ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f368ca00640 1 --2- 172.21.15.66:0/3327913313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806f8e0 0x7f368806e0f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f368ca00640 1 --2- 172.21.15.66:0/3327913313 >> v2:172.21.15.156:3300/0 conn(0x7f36880cc560 0x7f36880d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f368ca00640 1 -- 172.21.15.66:0/3327913313 >> 172.21.15.66:0/3327913313 conn(0x7f36880d0540 msgr2=0x7f36880d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:08.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f368ca00640 1 -- 172.21.15.66:0/3327913313 shutdown_connections 2024-03-30T00:42:08.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f368ca00640 1 -- 172.21.15.66:0/3327913313 wait complete. 2024-03-30T00:42:08.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.464+0000 7f368ca00640 1 Processor -- start 2024-03-30T00:42:08.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f368ca00640 1 -- start start 2024-03-30T00:42:08.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f368ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368806e630 0x7f36880b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f368ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806f8e0 0x7f368815f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f368ca00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36880cc560 0x7f3688157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f368ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36880b49b0 con 0x7f368806f8e0 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f368ca00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368815de10 con 0x7f36880cc560 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f368ca00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368815e0f0 con 0x7f368806e630 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806f8e0 0x7f368815f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806f8e0 0x7f368815f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35508/0 (socket says 172.21.15.66:35508) 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367d400640 1 -- 172.21.15.66:0/36121342 learned_addr learned my addr 172.21.15.66:0/36121342 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f3685e00640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368806e630 0x7f36880b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f3686800640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36880cc560 0x7f3688157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367d400640 1 -- 172.21.15.66:0/36121342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368806e630 msgr2=0x7f36880b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367d400640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368806e630 0x7f36880b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367d400640 1 -- 172.21.15.66:0/36121342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36880cc560 msgr2=0x7f3688157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367d400640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36880cc560 0x7f3688157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367d400640 1 -- 172.21.15.66:0/36121342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36700094d0 con 0x7f368806f8e0 2024-03-30T00:42:08.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367d400640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806f8e0 0x7f368815f4a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f36700ae040 tx=0x7f3670018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:08.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367ea00640 1 -- 172.21.15.66:0/36121342 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36700b4640 con 0x7f368806f8e0 2024-03-30T00:42:08.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367ea00640 1 -- 172.21.15.66:0/36121342 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36700b47a0 con 0x7f368806f8e0 2024-03-30T00:42:08.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f367ea00640 1 -- 172.21.15.66:0/36121342 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36700b4b50 con 0x7f368806f8e0 2024-03-30T00:42:08.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f368ca00640 1 -- 172.21.15.66:0/36121342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368815e3d0 con 0x7f368806f8e0 2024-03-30T00:42:08.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f368ca00640 1 -- 172.21.15.66:0/36121342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368815f9e0 con 0x7f368806f8e0 2024-03-30T00:42:08.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.468+0000 7f368ca00640 1 -- 172.21.15.66:0/36121342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3688066c30 con 0x7f368806f8e0 2024-03-30T00:42:08.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.472+0000 7f367ea00640 1 -- 172.21.15.66:0/36121342 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f36700b2070 con 0x7f368806f8e0 2024-03-30T00:42:08.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.472+0000 7f367ea00640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f36640666c0 0x7f3664068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.472+0000 7f367ea00640 1 -- 172.21.15.66:0/36121342 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f367011e190 con 0x7f368806f8e0 2024-03-30T00:42:08.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.472+0000 7f3685e00640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f36640666c0 0x7f3664068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.472+0000 7f3685e00640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f36640666c0 0x7f3664068b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f36780098f0 tx=0x7f3678006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:08.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.476+0000 7f367ea00640 1 -- 172.21.15.66:0/36121342 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f36700eb500 con 0x7f368806f8e0 2024-03-30T00:42:08.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.636+0000 7f368ca00640 1 -- 172.21.15.66:0/36121342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36880ca400 con 0x7f368806f8e0 2024-03-30T00:42:08.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f367ea00640 1 -- 172.21.15.66:0/36121342 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f36700f3e60 con 0x7f368806f8e0 2024-03-30T00:42:08.649 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:08.649 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":53,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:42:08.269378+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:42:06.807254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-30T00:42:07.258973+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 -- 172.21.15.66:0/36121342 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f36640666c0 msgr2=0x7f3664068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f36640666c0 0x7f3664068b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f36780098f0 tx=0x7f3678006f90 comp rx=0 tx=0).stop 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 -- 172.21.15.66:0/36121342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806f8e0 msgr2=0x7f368815f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806f8e0 0x7f368815f4a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f36700ae040 tx=0x7f3670018fd0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 -- 172.21.15.66:0/36121342 shutdown_connections 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36880cc560 0x7f3688157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f36640666c0 0x7f3664068b80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f368806f8e0 0x7f368815f4a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 --2- 172.21.15.66:0/36121342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368806e630 0x7f36880b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 -- 172.21.15.66:0/36121342 >> 172.21.15.66:0/36121342 conn(0x7f36880d0540 msgr2=0x7f36880cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 -- 172.21.15.66:0/36121342 shutdown_connections 2024-03-30T00:42:08.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.640+0000 7f368ca00640 1 -- 172.21.15.66:0/36121342 wait complete. 2024-03-30T00:42:08.669 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:42:08.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f3687800640 1 Processor -- start 2024-03-30T00:42:08.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f3687800640 1 -- start start 2024-03-30T00:42:08.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f3687800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f36800cee50 0x7f36800cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f3687800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36800cbb90 con 0x7f36800af420 2024-03-30T00:42:08.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f3687800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36800ce1b0 con 0x7f368015de30 2024-03-30T00:42:08.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f3687800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36800afa10 con 0x7f36800cee50 2024-03-30T00:42:08.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f367fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f36800cee50 0x7f36800cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f3685800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f36800af420 0x7f368006e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55584/0 (socket says 172.21.15.66:55584) 2024-03-30T00:42:08.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f3685800640 1 -- 172.21.15.66:0/1008873958 learned_addr learned my addr 172.21.15.66:0/1008873958 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:08.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f367fe00640 1 -- 172.21.15.66:0/1008873958 >> v1:172.21.15.156:6789/0 conn(0x7f368015de30 legacy=0x7f36800ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f367fe00640 1 -- 172.21.15.66:0/1008873958 >> v1:172.21.15.66:6789/0 conn(0x7f36800af420 legacy=0x7f368006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f367fe00640 1 -- 172.21.15.66:0/1008873958 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800cf430 con 0x7f36800cee50 2024-03-30T00:42:08.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.800+0000 7f367f400640 1 -- 172.21.15.66:0/1008873958 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916682933 0 0) 0x7f36800cbb90 con 0x7f36800af420 2024-03-30T00:42:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.804+0000 7f367fe00640 1 --2- 172.21.15.66:0/1008873958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36800cee50 0x7f36800cc040 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f366c0097c0 tx=0x7f366c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551f90bf423333c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:08.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.804+0000 7f367f400640 1 -- 172.21.15.66:0/1008873958 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c0ac410 con 0x7f36800cee50 2024-03-30T00:42:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.804+0000 7f367f400640 1 -- 172.21.15.66:0/1008873958 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c0ac570 con 0x7f36800cee50 2024-03-30T00:42:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.804+0000 7f367f400640 1 -- 172.21.15.66:0/1008873958 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c0ac900 con 0x7f36800cee50 2024-03-30T00:42:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.804+0000 7f3687800640 1 -- 172.21.15.66:0/1008873958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36800cee50 msgr2=0x7f36800cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.804+0000 7f3687800640 1 --2- 172.21.15.66:0/1008873958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36800cee50 0x7f36800cc040 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f366c0097c0 tx=0x7f366c01c4a0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.804+0000 7f3687800640 1 -- 172.21.15.66:0/1008873958 shutdown_connections 2024-03-30T00:42:08.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.804+0000 7f3687800640 1 --2- 172.21.15.66:0/1008873958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36800cee50 0x7f36800cc040 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.804+0000 7f3687800640 1 -- 172.21.15.66:0/1008873958 >> 172.21.15.66:0/1008873958 conn(0x7f36800d0540 msgr2=0x7f36800d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:08.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.804+0000 7f3687800640 1 -- 172.21.15.66:0/1008873958 shutdown_connections 2024-03-30T00:42:08.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.804+0000 7f3687800640 1 -- 172.21.15.66:0/1008873958 wait complete. 2024-03-30T00:42:08.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3687800640 1 Processor -- start 2024-03-30T00:42:08.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3687800640 1 -- start start 2024-03-30T00:42:08.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3687800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36800af420 0x7f36801586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3687800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36800cee50 0x7f36800c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3687800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368015de30 0x7f368013f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3687800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368015a060 con 0x7f36800cee50 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3687800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3680158fc0 con 0x7f36800af420 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3687800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36801592a0 con 0x7f368015de30 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36800cee50 0x7f36800c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36800cee50 0x7f36800c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36232/0 (socket says 172.21.15.66:36232) 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367fe00640 1 -- 172.21.15.66:0/3864463751 learned_addr learned my addr 172.21.15.66:0/3864463751 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3684e00640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36800af420 0x7f36801586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3685800640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368015de30 0x7f368013f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367fe00640 1 -- 172.21.15.66:0/3864463751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368015de30 msgr2=0x7f368013f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367fe00640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368015de30 0x7f368013f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367fe00640 1 -- 172.21.15.66:0/3864463751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36800af420 msgr2=0x7f36801586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367fe00640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36800af420 0x7f36801586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367fe00640 1 -- 172.21.15.66:0/3864463751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c009420 con 0x7f36800cee50 2024-03-30T00:42:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3685800640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368015de30 0x7f368013f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367fe00640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36800cee50 0x7f36800c7870 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f366c01a0d0 tx=0x7f366c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:08.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367cc00640 1 -- 172.21.15.66:0/3864463751 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c0ac410 con 0x7f36800cee50 2024-03-30T00:42:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367cc00640 1 -- 172.21.15.66:0/3864463751 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c006e20 con 0x7f36800cee50 2024-03-30T00:42:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f367cc00640 1 -- 172.21.15.66:0/3864463751 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c004230 con 0x7f36800cee50 2024-03-30T00:42:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3687800640 1 -- 172.21.15.66:0/3864463751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3680159580 con 0x7f36800cee50 2024-03-30T00:42:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3687800640 1 -- 172.21.15.66:0/3864463751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3680159b40 con 0x7f36800cee50 2024-03-30T00:42:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.808+0000 7f3687800640 1 -- 172.21.15.66:0/3864463751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36800cdf60 con 0x7f36800cee50 2024-03-30T00:42:08.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.812+0000 7f367cc00640 1 -- 172.21.15.66:0/3864463751 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f366c004390 con 0x7f36800cee50 2024-03-30T00:42:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.812+0000 7f367cc00640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3660066690 0x7f3660068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.812+0000 7f367cc00640 1 -- 172.21.15.66:0/3864463751 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f366c11dbe0 con 0x7f36800cee50 2024-03-30T00:42:08.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.812+0000 7f3684e00640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3660066690 0x7f3660068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:08.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.812+0000 7f3684e00640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3660066690 0x7f3660068b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3674000f70 tx=0x7f3674009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:08.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.816+0000 7f367cc00640 1 -- 172.21.15.66:0/3864463751 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f366c0ead70 con 0x7f36800cee50 2024-03-30T00:42:08.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.980+0000 7f3687800640 1 -- 172.21.15.66:0/3864463751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3680159d30 con 0x7f36800cee50 2024-03-30T00:42:08.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.980+0000 7f367cc00640 1 -- 172.21.15.66:0/3864463751 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f366c0b1090 con 0x7f36800cee50 2024-03-30T00:42:08.992 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:08.992 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":53,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:42:08.269378+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:42:06.807254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-30T00:42:07.258973+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 -- 172.21.15.66:0/3864463751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3660066690 msgr2=0x7f3660068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3660066690 0x7f3660068b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3674000f70 tx=0x7f3674009040 comp rx=0 tx=0).stop 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 -- 172.21.15.66:0/3864463751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36800cee50 msgr2=0x7f36800c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36800cee50 0x7f36800c7870 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f366c01a0d0 tx=0x7f366c01ace0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 -- 172.21.15.66:0/3864463751 shutdown_connections 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f368015de30 0x7f368013f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3660066690 0x7f3660068b50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36800cee50 0x7f36800c7870 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 --2- 172.21.15.66:0/3864463751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36800af420 0x7f36801586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 -- 172.21.15.66:0/3864463751 >> 172.21.15.66:0/3864463751 conn(0x7f36800d0540 msgr2=0x7f36800d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 -- 172.21.15.66:0/3864463751 shutdown_connections 2024-03-30T00:42:08.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:08.984+0000 7f3687800640 1 -- 172.21.15.66:0/3864463751 wait complete. 2024-03-30T00:42:09.013 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T00:42:09.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.144+0000 7f32c0000640 1 Processor -- start 2024-03-30T00:42:09.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.144+0000 7f32c0000640 1 -- start start 2024-03-30T00:42:09.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32c0000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b815a640 0x7f32b806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:09.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32c0000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f32b80ce520 0x7f32b80ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:09.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32c0000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b806d070 con 0x7f32b80cabb0 2024-03-30T00:42:09.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32c0000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b80cbba0 con 0x7f32b80ce520 2024-03-30T00:42:09.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32c0000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b806e2c0 con 0x7f32b815a640 2024-03-30T00:42:09.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32bcc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f32b80ce520 0x7f32b80ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:09.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32be000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f32b80cabb0 0x7f32b80cb000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55598/0 (socket says 172.21.15.66:55598) 2024-03-30T00:42:09.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32be000640 1 -- 172.21.15.66:0/2506072529 learned_addr learned my addr 172.21.15.66:0/2506072529 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:09.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32bd600640 1 --2- 172.21.15.66:0/2506072529 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b815a640 0x7f32b806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:09.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32bcc00640 1 -- 172.21.15.66:0/2506072529 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b815a640 msgr2=0x7f32b806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:09.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32bcc00640 1 --2- 172.21.15.66:0/2506072529 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b815a640 0x7f32b806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:09.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32bcc00640 1 -- 172.21.15.66:0/2506072529 >> v1:172.21.15.66:6789/0 conn(0x7f32b80cabb0 legacy=0x7f32b80cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:09.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32bcc00640 1 -- 172.21.15.66:0/2506072529 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80cee60 con 0x7f32b80ce520 2024-03-30T00:42:09.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.148+0000 7f32b7e00640 1 -- 172.21.15.66:0/2506072529 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209556567 0 0) 0x7f32b806d070 con 0x7f32b80cabb0 2024-03-30T00:42:09.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32bcc00640 1 --2- 172.21.15.66:0/2506072529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32b80ce520 0x7f32b80ce920 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f32a40097c0 tx=0x7f32a401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6d7e78695f8e5a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:09.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32b7e00640 1 -- 172.21.15.66:0/2506072529 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a40ad640 con 0x7f32b80ce520 2024-03-30T00:42:09.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32b7e00640 1 -- 172.21.15.66:0/2506072529 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a40ad7a0 con 0x7f32b80ce520 2024-03-30T00:42:09.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32b7e00640 1 -- 172.21.15.66:0/2506072529 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a40adb30 con 0x7f32b80ce520 2024-03-30T00:42:09.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32c0000640 1 -- 172.21.15.66:0/2506072529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32b80ce520 msgr2=0x7f32b80ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:09.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32c0000640 1 --2- 172.21.15.66:0/2506072529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32b80ce520 0x7f32b80ce920 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f32a40097c0 tx=0x7f32a401c4a0 comp rx=0 tx=0).stop 2024-03-30T00:42:09.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32c0000640 1 -- 172.21.15.66:0/2506072529 shutdown_connections 2024-03-30T00:42:09.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32c0000640 1 --2- 172.21.15.66:0/2506072529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32b80ce520 0x7f32b80ce920 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:09.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32c0000640 1 --2- 172.21.15.66:0/2506072529 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b815a640 0x7f32b806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:09.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32c0000640 1 -- 172.21.15.66:0/2506072529 >> 172.21.15.66:0/2506072529 conn(0x7f32b8069f60 msgr2=0x7f32b806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:09.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32c0000640 1 -- 172.21.15.66:0/2506072529 shutdown_connections 2024-03-30T00:42:09.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32c0000640 1 -- 172.21.15.66:0/2506072529 wait complete. 2024-03-30T00:42:09.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32c0000640 1 Processor -- start 2024-03-30T00:42:09.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.152+0000 7f32c0000640 1 -- start start 2024-03-30T00:42:09.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32c0000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32b80cabb0 0x7f32b80ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:09.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32c0000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32b80ce520 0x7f32b8150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:09.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32c0000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b815a640 0x7f32b80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:09.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32c0000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80cc8d0 con 0x7f32b80cabb0 2024-03-30T00:42:09.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32c0000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80cd260 con 0x7f32b80ce520 2024-03-30T00:42:09.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32c0000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80b07e0 con 0x7f32b815a640 2024-03-30T00:42:09.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32bd600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32b80cabb0 0x7f32b80ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32bd600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32b80cabb0 0x7f32b80ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36252/0 (socket says 172.21.15.66:36252) 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32bd600640 1 -- 172.21.15.66:0/3643733145 learned_addr learned my addr 172.21.15.66:0/3643733145 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32bcc00640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32b80ce520 0x7f32b8150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32be000640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b815a640 0x7f32b80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32bd600640 1 -- 172.21.15.66:0/3643733145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b815a640 msgr2=0x7f32b80713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32bd600640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b815a640 0x7f32b80713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32bd600640 1 -- 172.21.15.66:0/3643733145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32b80ce520 msgr2=0x7f32b8150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32bd600640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32b80ce520 0x7f32b8150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32bd600640 1 -- 172.21.15.66:0/3643733145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4009420 con 0x7f32b80cabb0 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32bcc00640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32b80ce520 0x7f32b8150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32be000640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b815a640 0x7f32b80713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:09.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32bd600640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32b80cabb0 0x7f32b80ccd20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d620 tx=0x7f32ac00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:09.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32b5600640 1 -- 172.21.15.66:0/3643733145 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00b620 con 0x7f32b80cabb0 2024-03-30T00:42:09.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32b5600640 1 -- 172.21.15.66:0/3643733145 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac00b780 con 0x7f32b80cabb0 2024-03-30T00:42:09.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32c0000640 1 -- 172.21.15.66:0/3643733145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b80711a0 con 0x7f32b80cabb0 2024-03-30T00:42:09.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32b5600640 1 -- 172.21.15.66:0/3643733145 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00bb10 con 0x7f32b80cabb0 2024-03-30T00:42:09.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32c0000640 1 -- 172.21.15.66:0/3643733145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b80cda40 con 0x7f32b80cabb0 2024-03-30T00:42:09.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32c0000640 1 -- 172.21.15.66:0/3643733145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b8066c30 con 0x7f32b80cabb0 2024-03-30T00:42:09.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.156+0000 7f32b5600640 1 -- 172.21.15.66:0/3643733145 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f32ac00bc70 con 0x7f32b80cabb0 2024-03-30T00:42:09.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.160+0000 7f32b5600640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f32980666c0 0x7f3298068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:09.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.160+0000 7f32b5600640 1 -- 172.21.15.66:0/3643733145 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f32ac0846f0 con 0x7f32b80cabb0 2024-03-30T00:42:09.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.160+0000 7f32bcc00640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f32980666c0 0x7f3298068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:09.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.160+0000 7f32bcc00640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f32980666c0 0x7f3298068b80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f32a40af000 tx=0x7f32a40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:09.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.164+0000 7f32b5600640 1 -- 172.21.15.66:0/3643733145 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f32ac0519b0 con 0x7f32b80cabb0 2024-03-30T00:42:09.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:09.360+0000 7f32c0000640 1 -- 172.21.15.66:0/3643733145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f32b80ac540 con 0x7f32b80cabb0 2024-03-30T00:42:09.792 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-03-30T00:42:09.784+0000 7f94a6800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:42:10.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.276+0000 7f32b5600640 1 -- 172.21.15.66:0/3643733145 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f32ac054fd0 con 0x7f32b80cabb0 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.276+0000 7f32c0000640 1 -- 172.21.15.66:0/3643733145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f32980666c0 msgr2=0x7f3298068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.276+0000 7f32c0000640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f32980666c0 0x7f3298068b80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f32a40af000 tx=0x7f32a40a7070 comp rx=0 tx=0).stop 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.276+0000 7f32c0000640 1 -- 172.21.15.66:0/3643733145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32b80cabb0 msgr2=0x7f32b80ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.276+0000 7f32c0000640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32b80cabb0 0x7f32b80ccd20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d620 tx=0x7f32ac00dae0 comp rx=0 tx=0).stop 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.276+0000 7f32c0000640 1 -- 172.21.15.66:0/3643733145 shutdown_connections 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.276+0000 7f32c0000640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b815a640 0x7f32b80713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.276+0000 7f32c0000640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32b80ce520 0x7f32b8150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.276+0000 7f32c0000640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f32980666c0 0x7f3298068b80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.276+0000 7f32c0000640 1 --2- 172.21.15.66:0/3643733145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32b80cabb0 0x7f32b80ccd20 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.276+0000 7f32c0000640 1 -- 172.21.15.66:0/3643733145 >> 172.21.15.66:0/3643733145 conn(0x7f32b8069f60 msgr2=0x7f32b8159950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.280+0000 7f32c0000640 1 -- 172.21.15.66:0/3643733145 shutdown_connections 2024-03-30T00:42:10.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.280+0000 7f32c0000640 1 -- 172.21.15.66:0/3643733145 wait complete. 2024-03-30T00:42:10.312 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T00:42:10.312 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T00:42:10.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.448+0000 7f1c2b600640 1 Processor -- start 2024-03-30T00:42:10.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.448+0000 7f1c2b600640 1 -- start start 2024-03-30T00:42:10.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c2b600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1c240ce520 0x7f1c240ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:10.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c2b600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2406d070 con 0x7f1c2415a640 2024-03-30T00:42:10.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c2b600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c240cbba0 con 0x7f1c240cabb0 2024-03-30T00:42:10.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c2b600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c240cee70 con 0x7f1c240ce520 2024-03-30T00:42:10.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c1be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1c240ce520 0x7f1c240ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:10.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c1be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1c240ce520 0x7f1c240ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36256/0 (socket says 172.21.15.66:36256) 2024-03-30T00:42:10.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c1be00640 1 -- 172.21.15.66:0/898732651 learned_addr learned my addr 172.21.15.66:0/898732651 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:10.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c28c00640 1 --1- 172.21.15.66:0/898732651 >> v1:172.21.15.66:6789/0 conn(0x7f1c2415a640 0x7f1c2406dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55614/0 (socket says 172.21.15.66:55614) 2024-03-30T00:42:10.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c1be00640 1 -- 172.21.15.66:0/898732651 >> v1:172.21.15.156:6789/0 conn(0x7f1c240cabb0 legacy=0x7f1c240cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:10.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c1be00640 1 -- 172.21.15.66:0/898732651 >> v1:172.21.15.66:6789/0 conn(0x7f1c2415a640 legacy=0x7f1c2406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:10.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c23e00640 1 -- 172.21.15.66:0/898732651 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585320931 0 0) 0x7f1c2406d070 con 0x7f1c2415a640 2024-03-30T00:42:10.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.452+0000 7f1c1be00640 1 -- 172.21.15.66:0/898732651 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c240cb8c0 con 0x7f1c240ce520 2024-03-30T00:42:10.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c1be00640 1 --2- 172.21.15.66:0/898732651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c240ce520 0x7f1c240ce930 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009870 tx=0x7f1c0c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d920872c6b8564 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:10.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c23e00640 1 -- 172.21.15.66:0/898732651 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0ab650 con 0x7f1c240ce520 2024-03-30T00:42:10.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c23e00640 1 -- 172.21.15.66:0/898732651 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c0ab7b0 con 0x7f1c240ce520 2024-03-30T00:42:10.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c23e00640 1 -- 172.21.15.66:0/898732651 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0abb40 con 0x7f1c240ce520 2024-03-30T00:42:10.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 -- 172.21.15.66:0/898732651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c240ce520 msgr2=0x7f1c240ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:10.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 --2- 172.21.15.66:0/898732651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c240ce520 0x7f1c240ce930 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009870 tx=0x7f1c0c01c560 comp rx=0 tx=0).stop 2024-03-30T00:42:10.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 -- 172.21.15.66:0/898732651 shutdown_connections 2024-03-30T00:42:10.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 --2- 172.21.15.66:0/898732651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c240ce520 0x7f1c240ce930 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:10.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 -- 172.21.15.66:0/898732651 >> 172.21.15.66:0/898732651 conn(0x7f1c24069f60 msgr2=0x7f1c2406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:10.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 -- 172.21.15.66:0/898732651 shutdown_connections 2024-03-30T00:42:10.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 -- 172.21.15.66:0/898732651 wait complete. 2024-03-30T00:42:10.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 Processor -- start 2024-03-30T00:42:10.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 -- start start 2024-03-30T00:42:10.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c240cabb0 0x7f1c2415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c240ce520 0x7f1c240c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c2415a640 0x7f1c240c3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24158610 con 0x7f1c240ce520 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24158330 con 0x7f1c2415a640 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.456+0000 7f1c2b600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c240c93d0 con 0x7f1c240cabb0 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c240ce520 0x7f1c240c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c240ce520 0x7f1c240c2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36264/0 (socket says 172.21.15.66:36264) 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c1be00640 1 -- 172.21.15.66:0/1701707749 learned_addr learned my addr 172.21.15.66:0/1701707749 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c28c00640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c240cabb0 0x7f1c2415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c29600640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c2415a640 0x7f1c240c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c1be00640 1 -- 172.21.15.66:0/1701707749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c240cabb0 msgr2=0x7f1c2415ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:10.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c1be00640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c240cabb0 0x7f1c2415ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c1be00640 1 -- 172.21.15.66:0/1701707749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c2415a640 msgr2=0x7f1c240c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c1be00640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c2415a640 0x7f1c240c3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c1be00640 1 -- 172.21.15.66:0/1701707749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c0094d0 con 0x7f1c240ce520 2024-03-30T00:42:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c28c00640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c240cabb0 0x7f1c2415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c1be00640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c240ce520 0x7f1c240c2cf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c019170 tx=0x7f1c0c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c21600640 1 -- 172.21.15.66:0/1701707749 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0b48c0 con 0x7f1c240ce520 2024-03-30T00:42:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c21600640 1 -- 172.21.15.66:0/1701707749 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c0b4a40 con 0x7f1c240ce520 2024-03-30T00:42:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c21600640 1 -- 172.21.15.66:0/1701707749 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0b4df0 con 0x7f1c240ce520 2024-03-30T00:42:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c2b600640 1 -- 172.21.15.66:0/1701707749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c240c96b0 con 0x7f1c240ce520 2024-03-30T00:42:10.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c2b600640 1 -- 172.21.15.66:0/1701707749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c24157440 con 0x7f1c240ce520 2024-03-30T00:42:10.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c2b600640 1 -- 172.21.15.66:0/1701707749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c240b0c90 con 0x7f1c240ce520 2024-03-30T00:42:10.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.460+0000 7f1c21600640 1 -- 172.21.15.66:0/1701707749 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1c0c0b3380 con 0x7f1c240ce520 2024-03-30T00:42:10.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.464+0000 7f1c21600640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c000666c0 0x7f1c00068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:10.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.464+0000 7f1c21600640 1 -- 172.21.15.66:0/1701707749 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1c0c11eba0 con 0x7f1c240ce520 2024-03-30T00:42:10.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.464+0000 7f1c28c00640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c000666c0 0x7f1c00068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:10.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.464+0000 7f1c28c00640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c000666c0 0x7f1c00068b80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1c14000f70 tx=0x7f1c14007dd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:10.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.464+0000 7f1c21600640 1 -- 172.21.15.66:0/1701707749 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1c0c0ebde0 con 0x7f1c240ce520 2024-03-30T00:42:10.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:10.660+0000 7f1c2b600640 1 -- 172.21.15.66:0/1701707749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1c24066c30 con 0x7f1c240ce520 2024-03-30T00:42:11.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c21600640 1 -- 172.21.15.66:0/1701707749 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1c0c0c40c0 con 0x7f1c240ce520 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 -- 172.21.15.66:0/1701707749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c000666c0 msgr2=0x7f1c00068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c000666c0 0x7f1c00068b80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1c14000f70 tx=0x7f1c14007dd0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 -- 172.21.15.66:0/1701707749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c240ce520 msgr2=0x7f1c240c2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c240ce520 0x7f1c240c2cf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c019170 tx=0x7f1c0c01ada0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 -- 172.21.15.66:0/1701707749 shutdown_connections 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c2415a640 0x7f1c240c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c000666c0 0x7f1c00068b80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c240ce520 0x7f1c240c2cf0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 --2- 172.21.15.66:0/1701707749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c240cabb0 0x7f1c2415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 -- 172.21.15.66:0/1701707749 >> 172.21.15.66:0/1701707749 conn(0x7f1c24069f60 msgr2=0x7f1c240afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 -- 172.21.15.66:0/1701707749 shutdown_connections 2024-03-30T00:42:11.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.492+0000 7f1c2b600640 1 -- 172.21.15.66:0/1701707749 wait complete. 2024-03-30T00:42:11.527 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T00:42:11.528 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:42:11.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97da000640 1 Processor -- start 2024-03-30T00:42:11.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97da000640 1 -- start start 2024-03-30T00:42:11.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97da000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f97d40cc590 0x7f97d415c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:11.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97da000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d40afdf0 con 0x7f97d4160490 2024-03-30T00:42:11.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97da000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d40b1010 con 0x7f97d40b0290 2024-03-30T00:42:11.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97da000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d40b0e30 con 0x7f97d40cc590 2024-03-30T00:42:11.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97d3400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f97d4160490 0x7f97d4160870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55626/0 (socket says 172.21.15.66:55626) 2024-03-30T00:42:11.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97d3400640 1 -- 172.21.15.66:0/486236024 learned_addr learned my addr 172.21.15.66:0/486236024 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:11.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97d3e00640 1 --2- 172.21.15.66:0/486236024 >> v2:172.21.15.66:3300/0 conn(0x7f97d40cc590 0x7f97d415c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:11.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97d2a00640 1 -- 172.21.15.66:0/486236024 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293492526 0 0) 0x7f97d40afdf0 con 0x7f97d4160490 2024-03-30T00:42:11.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97d2a00640 1 -- 172.21.15.66:0/486236024 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b8003530 con 0x7f97d4160490 2024-03-30T00:42:11.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97d3e00640 1 -- 172.21.15.66:0/486236024 >> v1:172.21.15.156:6789/0 conn(0x7f97d40b0290 legacy=0x7f97d40b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:11.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97d3e00640 1 -- 172.21.15.66:0/486236024 >> v1:172.21.15.66:6789/0 conn(0x7f97d4160490 legacy=0x7f97d4160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:11.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.664+0000 7f97d3e00640 1 -- 172.21.15.66:0/486236024 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d40b12f0 con 0x7f97d40cc590 2024-03-30T00:42:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.668+0000 7f97d3e00640 1 --2- 172.21.15.66:0/486236024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97d40cc590 0x7f97d415c200 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009d90 tx=0x7f97cc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74332c30c669989a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.668+0000 7f97d2a00640 1 -- 172.21.15.66:0/486236024 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0a5850 con 0x7f97d40cc590 2024-03-30T00:42:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.668+0000 7f97d2a00640 1 -- 172.21.15.66:0/486236024 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc0a59b0 con 0x7f97d40cc590 2024-03-30T00:42:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.668+0000 7f97d2a00640 1 -- 172.21.15.66:0/486236024 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0a5d20 con 0x7f97d40cc590 2024-03-30T00:42:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.668+0000 7f97da000640 1 -- 172.21.15.66:0/486236024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97d40cc590 msgr2=0x7f97d415c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.668+0000 7f97da000640 1 --2- 172.21.15.66:0/486236024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97d40cc590 0x7f97d415c200 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009d90 tx=0x7f97cc01c8a0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.668+0000 7f97da000640 1 -- 172.21.15.66:0/486236024 shutdown_connections 2024-03-30T00:42:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.668+0000 7f97da000640 1 --2- 172.21.15.66:0/486236024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97d40cc590 0x7f97d415c200 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.668+0000 7f97da000640 1 -- 172.21.15.66:0/486236024 >> 172.21.15.66:0/486236024 conn(0x7f97d4099870 msgr2=0x7f97d4089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.668+0000 7f97da000640 1 -- 172.21.15.66:0/486236024 shutdown_connections 2024-03-30T00:42:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.668+0000 7f97da000640 1 -- 172.21.15.66:0/486236024 wait complete. 2024-03-30T00:42:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97da000640 1 Processor -- start 2024-03-30T00:42:11.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97da000640 1 -- start start 2024-03-30T00:42:11.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97da000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97d40b0290 0x7f97d414d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97da000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97d40cc590 0x7f97d414e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97da000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97d4160490 0x7f97d414ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97da000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40cf710 con 0x7f97d4160490 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97da000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d415d230 con 0x7f97d40cc590 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97da000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40cf190 con 0x7f97d40b0290 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97d3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97d4160490 0x7f97d414ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97d3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97d4160490 0x7f97d414ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36280/0 (socket says 172.21.15.66:36280) 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97d3e00640 1 -- 172.21.15.66:0/3815872309 learned_addr learned my addr 172.21.15.66:0/3815872309 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97d3400640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97d40b0290 0x7f97d414d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97cbe00640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97d40cc590 0x7f97d414e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97d3e00640 1 -- 172.21.15.66:0/3815872309 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97d40b0290 msgr2=0x7f97d414d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97d3e00640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97d40b0290 0x7f97d414d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97d3e00640 1 -- 172.21.15.66:0/3815872309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97d40cc590 msgr2=0x7f97d414e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97d3e00640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97d40cc590 0x7f97d414e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97d3e00640 1 -- 172.21.15.66:0/3815872309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc009a70 con 0x7f97d4160490 2024-03-30T00:42:11.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97d3e00640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97d4160490 0x7f97d414ee00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0194b0 tx=0x7f97cc019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:11.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97cb400640 1 -- 172.21.15.66:0/3815872309 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0ad640 con 0x7f97d4160490 2024-03-30T00:42:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97cb400640 1 -- 172.21.15.66:0/3815872309 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc0ad7a0 con 0x7f97d4160490 2024-03-30T00:42:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97cb400640 1 -- 172.21.15.66:0/3815872309 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0adb10 con 0x7f97d4160490 2024-03-30T00:42:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97da000640 1 -- 172.21.15.66:0/3815872309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d40cf430 con 0x7f97d4160490 2024-03-30T00:42:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97da000640 1 -- 172.21.15.66:0/3815872309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d415fbf0 con 0x7f97d4160490 2024-03-30T00:42:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.672+0000 7f97da000640 1 -- 172.21.15.66:0/3815872309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d415d370 con 0x7f97d4160490 2024-03-30T00:42:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.676+0000 7f97cb400640 1 -- 172.21.15.66:0/3815872309 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f97cc0b4070 con 0x7f97d4160490 2024-03-30T00:42:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.676+0000 7f97cb400640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f97b00666c0 0x7f97b0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.676+0000 7f97cb400640 1 -- 172.21.15.66:0/3815872309 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f97cc11ecc0 con 0x7f97d4160490 2024-03-30T00:42:11.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.676+0000 7f97d3400640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f97b00666c0 0x7f97b0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:11.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.676+0000 7f97d3400640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f97b00666c0 0x7f97b0068b80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f97c40011f0 tx=0x7f97c4007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:11.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.680+0000 7f97cb400640 1 -- 172.21.15.66:0/3815872309 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f97d415d370 con 0x7f97d4160490 2024-03-30T00:42:11.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.876+0000 7f97da000640 1 -- 172.21.15.66:0/3815872309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97d415d370 con 0x7f97d4160490 2024-03-30T00:42:11.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.876+0000 7f97cb400640 1 -- 172.21.15.66:0/3815872309 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f97d415d370 con 0x7f97d4160490 2024-03-30T00:42:11.888 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:11.889 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/465916334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":465916334},{"type":"v1","addr":"172.21.15.156:6835","nonce":465916334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1640047258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1640047258},{"type":"v1","addr":"172.21.15.156:6834","nonce":1640047258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5788,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2112021115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2112021115},{"type":"v1","addr":"172.21.15.66:6835","nonce":2112021115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:42:08.269705+0000","modified":"2024-03-30T00:42:11.492352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5985},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5985":{"gid":5985,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/3557086525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3557086525},{"type":"v1","addr":"172.21.15.66:6837","nonce":3557086525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T00:42:11.889 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-03-30T00:42:11.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 -- 172.21.15.66:0/3815872309 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f97b00666c0 msgr2=0x7f97b0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:11.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f97b00666c0 0x7f97b0068b80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f97c40011f0 tx=0x7f97c4007d40 comp rx=0 tx=0).stop 2024-03-30T00:42:11.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 -- 172.21.15.66:0/3815872309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97d4160490 msgr2=0x7f97d414ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:11.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97d4160490 0x7f97d414ee00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0194b0 tx=0x7f97cc019640 comp rx=0 tx=0).stop 2024-03-30T00:42:11.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 -- 172.21.15.66:0/3815872309 shutdown_connections 2024-03-30T00:42:11.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f97b00666c0 0x7f97b0068b80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97d4160490 0x7f97d414ee00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97d40cc590 0x7f97d414e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 --2- 172.21.15.66:0/3815872309 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97d40b0290 0x7f97d414d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:11.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 -- 172.21.15.66:0/3815872309 >> 172.21.15.66:0/3815872309 conn(0x7f97d4099870 msgr2=0x7f97d40b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:11.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 -- 172.21.15.66:0/3815872309 shutdown_connections 2024-03-30T00:42:11.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:11.880+0000 7f97da000640 1 -- 172.21.15.66:0/3815872309 wait complete. 2024-03-30T00:42:11.910 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:42:12.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8de5800640 1 Processor -- start 2024-03-30T00:42:12.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8de5800640 1 -- start start 2024-03-30T00:42:12.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8de5800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8de00cec30 0x7f8de0159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8de5800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de00cd3e0 con 0x7f8de00cde70 2024-03-30T00:42:12.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8de5800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de015a990 con 0x7f8de00cda90 2024-03-30T00:42:12.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8de5800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00b1b80 con 0x7f8de00cec30 2024-03-30T00:42:12.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8ddf400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f8de00cde70 0x7f8de015a1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55632/0 (socket says 172.21.15.66:55632) 2024-03-30T00:42:12.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8ddf400640 1 -- 172.21.15.66:0/759616497 learned_addr learned my addr 172.21.15.66:0/759616497 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8dd6000640 1 --2- 172.21.15.66:0/759616497 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8de00cec30 0x7f8de0159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8dde000640 1 -- 172.21.15.66:0/759616497 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557753710 0 0) 0x7f8de00cd3e0 con 0x7f8de00cde70 2024-03-30T00:42:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8dde000640 1 -- 172.21.15.66:0/759616497 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dc4003530 con 0x7f8de00cde70 2024-03-30T00:42:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.040+0000 7f8dde000640 1 -- 172.21.15.66:0/759616497 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040239483 0 0) 0x7f8de015a990 con 0x7f8de00cda90 2024-03-30T00:42:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8dde000640 1 -- 172.21.15.66:0/759616497 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8de00cd3e0 con 0x7f8de00cda90 2024-03-30T00:42:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8dd6000640 1 -- 172.21.15.66:0/759616497 >> v1:172.21.15.156:6789/0 conn(0x7f8de00cda90 legacy=0x7f8de00ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8dd6000640 1 -- 172.21.15.66:0/759616497 >> v1:172.21.15.66:6789/0 conn(0x7f8de00cde70 legacy=0x7f8de015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8dd6000640 1 -- 172.21.15.66:0/759616497 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de00b11e0 con 0x7f8de00cec30 2024-03-30T00:42:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8dd6000640 1 --2- 172.21.15.66:0/759616497 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8de00cec30 0x7f8de0159c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80097f0 tx=0x7f8dc801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=289005656c652cbf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8dde000640 1 -- 172.21.15.66:0/759616497 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80ab650 con 0x7f8de00cec30 2024-03-30T00:42:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8dde000640 1 -- 172.21.15.66:0/759616497 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc80ab7b0 con 0x7f8de00cec30 2024-03-30T00:42:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8dde000640 1 -- 172.21.15.66:0/759616497 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80abb40 con 0x7f8de00cec30 2024-03-30T00:42:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8de5800640 1 -- 172.21.15.66:0/759616497 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8de00cec30 msgr2=0x7f8de0159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8de5800640 1 --2- 172.21.15.66:0/759616497 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8de00cec30 0x7f8de0159c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80097f0 tx=0x7f8dc801c450 comp rx=0 tx=0).stop 2024-03-30T00:42:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8de5800640 1 -- 172.21.15.66:0/759616497 shutdown_connections 2024-03-30T00:42:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8de5800640 1 --2- 172.21.15.66:0/759616497 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8de00cec30 0x7f8de0159c60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.044+0000 7f8de5800640 1 -- 172.21.15.66:0/759616497 >> 172.21.15.66:0/759616497 conn(0x7f8de0069f60 msgr2=0x7f8de006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8de5800640 1 -- 172.21.15.66:0/759616497 shutdown_connections 2024-03-30T00:42:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8de5800640 1 -- 172.21.15.66:0/759616497 wait complete. 2024-03-30T00:42:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8de5800640 1 Processor -- start 2024-03-30T00:42:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8de5800640 1 -- start start 2024-03-30T00:42:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8de5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8de00cda90 0x7f8de006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8de5800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8de00cde70 0x7f8de0155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8de5800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8de00cec30 0x7f8de0162830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8de5800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de01562d0 con 0x7f8de00cda90 2024-03-30T00:42:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8de5800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de0162d70 con 0x7f8de00cec30 2024-03-30T00:42:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8de5800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de0162fa0 con 0x7f8de00cde70 2024-03-30T00:42:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8ddea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8de00cda90 0x7f8de006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8ddea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8de00cda90 0x7f8de006d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36294/0 (socket says 172.21.15.66:36294) 2024-03-30T00:42:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8ddea00640 1 -- 172.21.15.66:0/916364684 learned_addr learned my addr 172.21.15.66:0/916364684 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8ddea00640 1 -- 172.21.15.66:0/916364684 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8de00cde70 msgr2=0x7f8de0155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8ddea00640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8de00cde70 0x7f8de0155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8ddea00640 1 -- 172.21.15.66:0/916364684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8de00cec30 msgr2=0x7f8de0162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8ddea00640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8de00cec30 0x7f8de0162830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8ddea00640 1 -- 172.21.15.66:0/916364684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc8009420 con 0x7f8de00cda90 2024-03-30T00:42:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8ddea00640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8de00cda90 0x7f8de006d0d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000c490 tx=0x7f8dd000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:12.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8dd7400640 1 -- 172.21.15.66:0/916364684 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd00052d0 con 0x7f8de00cda90 2024-03-30T00:42:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8dd7400640 1 -- 172.21.15.66:0/916364684 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd0005430 con 0x7f8de00cda90 2024-03-30T00:42:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8dd7400640 1 -- 172.21.15.66:0/916364684 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0012870 con 0x7f8de00cda90 2024-03-30T00:42:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.048+0000 7f8de5800640 1 -- 172.21.15.66:0/916364684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de0163570 con 0x7f8de00cda90 2024-03-30T00:42:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.052+0000 7f8de5800640 1 -- 172.21.15.66:0/916364684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de0163b30 con 0x7f8de00cda90 2024-03-30T00:42:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.052+0000 7f8de5800640 1 -- 172.21.15.66:0/916364684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de01633d0 con 0x7f8de00cda90 2024-03-30T00:42:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.052+0000 7f8dd7400640 1 -- 172.21.15.66:0/916364684 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8dd00129d0 con 0x7f8de00cda90 2024-03-30T00:42:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.052+0000 7f8dd7400640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dbc066690 0x7f8dbc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.052+0000 7f8dd7400640 1 -- 172.21.15.66:0/916364684 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8dd0083f10 con 0x7f8de00cda90 2024-03-30T00:42:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.052+0000 7f8dd6000640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dbc066690 0x7f8dbc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.056+0000 7f8dd6000640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dbc066690 0x7f8dbc068b50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80af000 tx=0x7f8dc8004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:12.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.056+0000 7f8dd7400640 1 -- 172.21.15.66:0/916364684 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8de01633d0 con 0x7f8de00cda90 2024-03-30T00:42:12.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.220+0000 7f8de5800640 1 -- 172.21.15.66:0/916364684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8de01633d0 con 0x7f8de00cda90 2024-03-30T00:42:12.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8dd7400640 1 -- 172.21.15.66:0/916364684 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f8de01633d0 con 0x7f8de00cda90 2024-03-30T00:42:12.233 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:12.233 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":54,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:42:09.272575+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:42:06.807254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-30T00:42:07.258973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:42:12.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 -- 172.21.15.66:0/916364684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dbc066690 msgr2=0x7f8dbc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dbc066690 0x7f8dbc068b50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80af000 tx=0x7f8dc8004740 comp rx=0 tx=0).stop 2024-03-30T00:42:12.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 -- 172.21.15.66:0/916364684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8de00cda90 msgr2=0x7f8de006d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8de00cda90 0x7f8de006d0d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000c490 tx=0x7f8dd000c950 comp rx=0 tx=0).stop 2024-03-30T00:42:12.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 -- 172.21.15.66:0/916364684 shutdown_connections 2024-03-30T00:42:12.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8de00cec30 0x7f8de0162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8de00cde70 0x7f8de0155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8dbc066690 0x7f8dbc068b50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 --2- 172.21.15.66:0/916364684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8de00cda90 0x7f8de006d0d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 -- 172.21.15.66:0/916364684 >> 172.21.15.66:0/916364684 conn(0x7f8de0069f60 msgr2=0x7f8de006d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:12.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 -- 172.21.15.66:0/916364684 shutdown_connections 2024-03-30T00:42:12.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.224+0000 7f8de5800640 1 -- 172.21.15.66:0/916364684 wait complete. 2024-03-30T00:42:12.255 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T00:42:12.255 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:42:12.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.384+0000 7fa0f4600640 1 Processor -- start 2024-03-30T00:42:12.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.384+0000 7fa0f4600640 1 -- start start 2024-03-30T00:42:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.384+0000 7fa0f4600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa0ec0cf290 0x7fa0ec1654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.384+0000 7fa0f4600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec0c4600 con 0x7fa0ec099430 2024-03-30T00:42:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.384+0000 7fa0f4600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec0cc510 con 0x7fa0ec15a810 2024-03-30T00:42:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.384+0000 7fa0f4600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0998c0 con 0x7fa0ec0cf290 2024-03-30T00:42:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.384+0000 7fa0f1200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa0ec099430 0x7fa0ec0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55644/0 (socket says 172.21.15.66:55644) 2024-03-30T00:42:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.384+0000 7fa0f1200640 1 -- 172.21.15.66:0/3898293816 learned_addr learned my addr 172.21.15.66:0/3898293816 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.384+0000 7fa0f2600640 1 --2- 172.21.15.66:0/3898293816 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa0ec0cf290 0x7fa0ec1654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.388+0000 7fa0ebe00640 1 -- 172.21.15.66:0/3898293816 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509158431 0 0) 0x7fa0ec0c4600 con 0x7fa0ec099430 2024-03-30T00:42:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.388+0000 7fa0ebe00640 1 -- 172.21.15.66:0/3898293816 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0d4003530 con 0x7fa0ec099430 2024-03-30T00:42:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.388+0000 7fa0ebe00640 1 -- 172.21.15.66:0/3898293816 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222351061 0 0) 0x7fa0ec0cc510 con 0x7fa0ec15a810 2024-03-30T00:42:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.388+0000 7fa0ebe00640 1 -- 172.21.15.66:0/3898293816 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0ec0c4600 con 0x7fa0ec15a810 2024-03-30T00:42:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.388+0000 7fa0f2600640 1 -- 172.21.15.66:0/3898293816 >> v1:172.21.15.156:6789/0 conn(0x7fa0ec15a810 legacy=0x7fa0ec0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.388+0000 7fa0f2600640 1 -- 172.21.15.66:0/3898293816 >> v1:172.21.15.66:6789/0 conn(0x7fa0ec099430 legacy=0x7fa0ec0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.388+0000 7fa0f2600640 1 -- 172.21.15.66:0/3898293816 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec099ba0 con 0x7fa0ec0cf290 2024-03-30T00:42:12.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.388+0000 7fa0f2600640 1 --2- 172.21.15.66:0/3898293816 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa0ec0cf290 0x7fa0ec1654c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009dc0 tx=0x7fa0e401f170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=798127c1b6af88f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:12.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.388+0000 7fa0ebe00640 1 -- 172.21.15.66:0/3898293816 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40af600 con 0x7fa0ec0cf290 2024-03-30T00:42:12.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.388+0000 7fa0ebe00640 1 -- 172.21.15.66:0/3898293816 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e40ab050 con 0x7fa0ec0cf290 2024-03-30T00:42:12.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.388+0000 7fa0ebe00640 1 -- 172.21.15.66:0/3898293816 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40af980 con 0x7fa0ec0cf290 2024-03-30T00:42:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 -- 172.21.15.66:0/3898293816 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa0ec0cf290 msgr2=0x7fa0ec1654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 --2- 172.21.15.66:0/3898293816 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa0ec0cf290 0x7fa0ec1654c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009dc0 tx=0x7fa0e401f170 comp rx=0 tx=0).stop 2024-03-30T00:42:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 -- 172.21.15.66:0/3898293816 shutdown_connections 2024-03-30T00:42:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 --2- 172.21.15.66:0/3898293816 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa0ec0cf290 0x7fa0ec1654c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 -- 172.21.15.66:0/3898293816 >> 172.21.15.66:0/3898293816 conn(0x7fa0ec0b2090 msgr2=0x7fa0ec0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 -- 172.21.15.66:0/3898293816 shutdown_connections 2024-03-30T00:42:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 -- 172.21.15.66:0/3898293816 wait complete. 2024-03-30T00:42:12.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 Processor -- start 2024-03-30T00:42:12.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 -- start start 2024-03-30T00:42:12.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa0ec0cf290 0x7fa0ec14c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0ec15a810 0x7fa0ec0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec0b4d70 0x7fa0ec09cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec14d180 con 0x7fa0ec0b4d70 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0b5200 con 0x7fa0ec15a810 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f4600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0b1aa0 con 0x7fa0ec0cf290 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f2600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec0b4d70 0x7fa0ec09cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f2600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec0b4d70 0x7fa0ec09cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36300/0 (socket says 172.21.15.66:36300) 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f2600640 1 -- 172.21.15.66:0/1107040166 learned_addr learned my addr 172.21.15.66:0/1107040166 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f1200640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0ec15a810 0x7fa0ec0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f1c00640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa0ec0cf290 0x7fa0ec14c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f2600640 1 -- 172.21.15.66:0/1107040166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa0ec0cf290 msgr2=0x7fa0ec14c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f2600640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa0ec0cf290 0x7fa0ec14c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f2600640 1 -- 172.21.15.66:0/1107040166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0ec15a810 msgr2=0x7fa0ec0b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f2600640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0ec15a810 0x7fa0ec0b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f2600640 1 -- 172.21.15.66:0/1107040166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e4009a70 con 0x7fa0ec0b4d70 2024-03-30T00:42:12.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.392+0000 7fa0f2600640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec0b4d70 0x7fa0ec09cf60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa0e401bd80 tx=0x7fa0e401bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:12.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.396+0000 7fa0e9600640 1 -- 172.21.15.66:0/1107040166 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40afe90 con 0x7fa0ec0b4d70 2024-03-30T00:42:12.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.396+0000 7fa0e9600640 1 -- 172.21.15.66:0/1107040166 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e4009450 con 0x7fa0ec0b4d70 2024-03-30T00:42:12.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.396+0000 7fa0e9600640 1 -- 172.21.15.66:0/1107040166 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40b9650 con 0x7fa0ec0b4d70 2024-03-30T00:42:12.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.396+0000 7fa0f4600640 1 -- 172.21.15.66:0/1107040166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ec1528c0 con 0x7fa0ec0b4d70 2024-03-30T00:42:12.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.396+0000 7fa0f4600640 1 -- 172.21.15.66:0/1107040166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ec0b56d0 con 0x7fa0ec0b4d70 2024-03-30T00:42:12.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.396+0000 7fa0f4600640 1 -- 172.21.15.66:0/1107040166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ec14d2c0 con 0x7fa0ec0b4d70 2024-03-30T00:42:12.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.396+0000 7fa0e9600640 1 -- 172.21.15.66:0/1107040166 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa0e40b6050 con 0x7fa0ec0b4d70 2024-03-30T00:42:12.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.396+0000 7fa0e9600640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa0cc066690 0x7fa0cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.396+0000 7fa0e9600640 1 -- 172.21.15.66:0/1107040166 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa0e4122640 con 0x7fa0ec0b4d70 2024-03-30T00:42:12.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.396+0000 7fa0f1c00640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa0cc066690 0x7fa0cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.400+0000 7fa0f1c00640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa0cc066690 0x7fa0cc068b50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0000f70 tx=0x7fa0e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.400+0000 7fa0e9600640 1 -- 172.21.15.66:0/1107040166 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa0ec14d2c0 con 0x7fa0ec0b4d70 2024-03-30T00:42:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.560+0000 7fa0f4600640 1 -- 172.21.15.66:0/1107040166 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0ec14d2c0 con 0x7fa0cc066690 2024-03-30T00:42:12.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.568+0000 7fa0e9600640 1 -- 172.21.15.66:0/1107040166 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285961 (secure 0 0 0) 0x7fa0ec14d2c0 con 0x7fa0cc066690 2024-03-30T00:42:12.582 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:12.583 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 -- 172.21.15.66:0/1107040166 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa0cc066690 msgr2=0x7fa0cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa0cc066690 0x7fa0cc068b50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0000f70 tx=0x7fa0e0009040 comp rx=0 tx=0).stop 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 -- 172.21.15.66:0/1107040166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec0b4d70 msgr2=0x7fa0ec09cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec0b4d70 0x7fa0ec09cf60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa0e401bd80 tx=0x7fa0e401bf10 comp rx=0 tx=0).stop 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 -- 172.21.15.66:0/1107040166 shutdown_connections 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa0cc066690 0x7fa0cc068b50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec0b4d70 0x7fa0ec09cf60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0ec15a810 0x7fa0ec0b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 --2- 172.21.15.66:0/1107040166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa0ec0cf290 0x7fa0ec14c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 -- 172.21.15.66:0/1107040166 >> 172.21.15.66:0/1107040166 conn(0x7fa0ec0b2090 msgr2=0x7fa0ec0b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 -- 172.21.15.66:0/1107040166 shutdown_connections 2024-03-30T00:42:12.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.572+0000 7fa0f4600640 1 -- 172.21.15.66:0/1107040166 wait complete. 2024-03-30T00:42:12.602 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-03-30T00:42:10.879245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238452,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754736268,"statfs":{"total":773094113280,"available":772849938432,"internally_reserved":0,"allocated":6037504,"data_stored":3291064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018455"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874156+0000","last_change":"2024-03-30T00:42:07.874156+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874156+0000","last_undegraded":"2024-03-30T00:42:07.874156+0000","last_fullsized":"2024-03-30T00:42:07.874156+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873050+0000","last_change":"2024-03-30T00:42:07.873050+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873050+0000","last_undegraded":"2024-03-30T00:42:07.873050+0000","last_fullsized":"2024-03-30T00:42:07.873050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.889268+0000","last_change":"2024-03-30T00:42:07.889268+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.889268+0000","last_undegraded":"2024-03-30T00:42:07.889268+0000","last_fullsized":"2024-03-30T00:42:07.889268+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872559+0000","last_change":"2024-03-30T00:42:07.872559+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872559+0000","last_undegraded":"2024-03-30T00:42:07.872559+0000","last_fullsized":"2024-03-30T00:42:07.872559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874398+0000","last_change":"2024-03-30T00:42:07.874398+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874398+0000","last_undegraded":"2024-03-30T00:42:07.874398+0000","last_fullsized":"2024-03-30T00:42:07.874398+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873733+0000","last_change":"2024-03-30T00:42:07.873733+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873733+0000","last_undegraded":"2024-03-30T00:42:07.873733+0000","last_fullsized":"2024-03-30T00:42:07.873733+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:22.099119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881871+0000","last_change":"2024-03-30T00:42:07.881871+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881871+0000","last_undegraded":"2024-03-30T00:42:07.881871+0000","last_fullsized":"2024-03-30T00:42:07.881871+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:09.745566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877328+0000","last_change":"2024-03-30T00:42:07.877328+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877328+0000","last_undegraded":"2024-03-30T00:42:07.877328+0000","last_fullsized":"2024-03-30T00:42:07.877328+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:01.749453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878241+0000","last_change":"2024-03-30T00:42:07.878241+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878241+0000","last_undegraded":"2024-03-30T00:42:07.878241+0000","last_fullsized":"2024-03-30T00:42:07.878241+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874651+0000","last_change":"2024-03-30T00:42:07.874651+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874651+0000","last_undegraded":"2024-03-30T00:42:07.874651+0000","last_fullsized":"2024-03-30T00:42:07.874651+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879446+0000","last_change":"2024-03-30T00:42:07.879446+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879446+0000","last_undegraded":"2024-03-30T00:42:07.879446+0000","last_fullsized":"2024-03-30T00:42:07.879446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:04.810334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878717+0000","last_change":"2024-03-30T00:42:07.878717+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878717+0000","last_undegraded":"2024-03-30T00:42:07.878717+0000","last_fullsized":"2024-03-30T00:42:07.878717+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874011+0000","last_change":"2024-03-30T00:42:07.874011+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874011+0000","last_undegraded":"2024-03-30T00:42:07.874011+0000","last_fullsized":"2024-03-30T00:42:07.874011+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:03.287560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.882341+0000","last_change":"2024-03-30T00:42:07.882341+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.882341+0000","last_undegraded":"2024-03-30T00:42:07.882341+0000","last_fullsized":"2024-03-30T00:42:07.882341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:54:01.171986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.871624+0000","last_change":"2024-03-30T00:42:07.871624+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.871624+0000","last_undegraded":"2024-03-30T00:42:07.871624+0000","last_fullsized":"2024-03-30T00:42:07.871624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:56.870783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872827+0000","last_change":"2024-03-30T00:42:07.872827+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872827+0000","last_undegraded":"2024-03-30T00:42:07.872827+0000","last_fullsized":"2024-03-30T00:42:07.872827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:24.904667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888103+0000","last_change":"2024-03-30T00:42:07.888103+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888103+0000","last_undegraded":"2024-03-30T00:42:07.888103+0000","last_fullsized":"2024-03-30T00:42:07.888103+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:41:31.279358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872622+0000","last_change":"2024-03-30T00:42:07.872622+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872622+0000","last_undegraded":"2024-03-30T00:42:07.872622+0000","last_fullsized":"2024-03-30T00:42:07.872622+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888796+0000","last_change":"2024-03-30T00:42:07.888796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888796+0000","last_undegraded":"2024-03-30T00:42:07.888796+0000","last_fullsized":"2024-03-30T00:42:07.888796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877961+0000","last_change":"2024-03-30T00:42:07.877961+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877961+0000","last_undegraded":"2024-03-30T00:42:07.877961+0000","last_fullsized":"2024-03-30T00:42:07.877961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:10:20.006391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877522+0000","last_change":"2024-03-30T00:42:07.877522+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877522+0000","last_undegraded":"2024-03-30T00:42:07.877522+0000","last_fullsized":"2024-03-30T00:42:07.877522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:19.899232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877834+0000","last_change":"2024-03-30T00:42:07.877834+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877834+0000","last_undegraded":"2024-03-30T00:42:07.877834+0000","last_fullsized":"2024-03-30T00:42:07.877834+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878875+0000","last_change":"2024-03-30T00:42:07.878875+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878875+0000","last_undegraded":"2024-03-30T00:42:07.878875+0000","last_fullsized":"2024-03-30T00:42:07.878875+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:26.322503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881115+0000","last_change":"2024-03-30T00:42:07.881115+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881115+0000","last_undegraded":"2024-03-30T00:42:07.881115+0000","last_fullsized":"2024-03-30T00:42:07.881115+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:34:13.403177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.887228+0000","last_change":"2024-03-30T00:42:07.887228+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.887228+0000","last_undegraded":"2024-03-30T00:42:07.887228+0000","last_fullsized":"2024-03-30T00:42:07.887228+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880458+0000","last_change":"2024-03-30T00:42:07.880458+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880458+0000","last_undegraded":"2024-03-30T00:42:07.880458+0000","last_fullsized":"2024-03-30T00:42:07.880458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872899+0000","last_change":"2024-03-30T00:42:07.872899+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872899+0000","last_undegraded":"2024-03-30T00:42:07.872899+0000","last_fullsized":"2024-03-30T00:42:07.872899+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:22.536352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877458+0000","last_change":"2024-03-30T00:42:07.877458+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877458+0000","last_undegraded":"2024-03-30T00:42:07.877458+0000","last_fullsized":"2024-03-30T00:42:07.877458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:20.469591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.893998+0000","last_change":"2024-03-30T00:42:07.893998+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.893998+0000","last_undegraded":"2024-03-30T00:42:07.893998+0000","last_fullsized":"2024-03-30T00:42:07.893998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872039+0000","last_change":"2024-03-30T00:42:07.872039+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872039+0000","last_undegraded":"2024-03-30T00:42:07.872039+0000","last_fullsized":"2024-03-30T00:42:07.872039+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881306+0000","last_change":"2024-03-30T00:42:07.881306+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881306+0000","last_undegraded":"2024-03-30T00:42:07.881306+0000","last_fullsized":"2024-03-30T00:42:07.881306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:49.025731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877550+0000","last_change":"2024-03-30T00:42:07.877550+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877550+0000","last_undegraded":"2024-03-30T00:42:07.877550+0000","last_fullsized":"2024-03-30T00:42:07.877550+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.871624+0000","last_change":"2024-03-30T00:42:07.871624+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.871624+0000","last_undegraded":"2024-03-30T00:42:07.871624+0000","last_fullsized":"2024-03-30T00:42:07.871624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:17:22.441495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890610+0000","last_change":"2024-03-30T00:42:07.887914+0000","last_active":"2024-03-30T00:42:07.890610+0000","last_peered":"2024-03-30T00:42:07.890610+0000","last_clean":"2024-03-30T00:42:07.890610+0000","last_became_active":"2024-03-30T00:42:07.887626+0000","last_became_peered":"2024-03-30T00:42:07.887626+0000","last_unstale":"2024-03-30T00:42:07.890610+0000","last_undegraded":"2024-03-30T00:42:07.890610+0000","last_fullsized":"2024-03-30T00:42:07.890610+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:25.319805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876251+0000","last_change":"2024-03-30T00:42:07.876251+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876251+0000","last_undegraded":"2024-03-30T00:42:07.876251+0000","last_fullsized":"2024-03-30T00:42:07.876251+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:29.266461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890823+0000","last_change":"2024-03-30T00:42:07.890823+0000","last_active":"2024-03-30T00:42:07.890823+0000","last_peered":"2024-03-30T00:42:07.890823+0000","last_clean":"2024-03-30T00:42:07.890823+0000","last_became_active":"2024-03-30T00:42:07.888331+0000","last_became_peered":"2024-03-30T00:42:07.888331+0000","last_unstale":"2024-03-30T00:42:07.890823+0000","last_undegraded":"2024-03-30T00:42:07.890823+0000","last_fullsized":"2024-03-30T00:42:07.890823+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:48:23.842037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878821+0000","last_change":"2024-03-30T00:42:07.878821+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878821+0000","last_undegraded":"2024-03-30T00:42:07.878821+0000","last_fullsized":"2024-03-30T00:42:07.878821+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:03.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.884824+0000","last_change":"2024-03-30T00:42:07.884824+0000","last_active":"2024-03-30T00:42:07.884824+0000","last_peered":"2024-03-30T00:42:07.884824+0000","last_clean":"2024-03-30T00:42:07.884824+0000","last_became_active":"2024-03-30T00:42:07.884503+0000","last_became_peered":"2024-03-30T00:42:07.884503+0000","last_unstale":"2024-03-30T00:42:07.884824+0000","last_undegraded":"2024-03-30T00:42:07.884824+0000","last_fullsized":"2024-03-30T00:42:07.884824+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:18:21.768136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877433+0000","last_change":"2024-03-30T00:42:07.877433+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877433+0000","last_undegraded":"2024-03-30T00:42:07.877433+0000","last_fullsized":"2024-03-30T00:42:07.877433+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.892398+0000","last_change":"2024-03-30T00:42:07.892398+0000","last_active":"2024-03-30T00:42:07.892398+0000","last_peered":"2024-03-30T00:42:07.892398+0000","last_clean":"2024-03-30T00:42:07.892398+0000","last_became_active":"2024-03-30T00:42:07.892097+0000","last_became_peered":"2024-03-30T00:42:07.892097+0000","last_unstale":"2024-03-30T00:42:07.892398+0000","last_undegraded":"2024-03-30T00:42:07.892398+0000","last_fullsized":"2024-03-30T00:42:07.892398+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:30:49.532832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874348+0000","last_change":"2024-03-30T00:42:07.874348+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874348+0000","last_undegraded":"2024-03-30T00:42:07.874348+0000","last_fullsized":"2024-03-30T00:42:07.874348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:59.202589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.897983+0000","last_change":"2024-03-30T00:42:07.897983+0000","last_active":"2024-03-30T00:42:07.897983+0000","last_peered":"2024-03-30T00:42:07.897983+0000","last_clean":"2024-03-30T00:42:07.897983+0000","last_became_active":"2024-03-30T00:42:07.897719+0000","last_became_peered":"2024-03-30T00:42:07.897719+0000","last_unstale":"2024-03-30T00:42:07.897983+0000","last_undegraded":"2024-03-30T00:42:07.897983+0000","last_fullsized":"2024-03-30T00:42:07.897983+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:55.868085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873753+0000","last_change":"2024-03-30T00:42:07.873753+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873753+0000","last_undegraded":"2024-03-30T00:42:07.873753+0000","last_fullsized":"2024-03-30T00:42:07.873753+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887534+0000","last_change":"2024-03-30T00:42:07.884431+0000","last_active":"2024-03-30T00:42:07.887534+0000","last_peered":"2024-03-30T00:42:07.887534+0000","last_clean":"2024-03-30T00:42:07.887534+0000","last_became_active":"2024-03-30T00:42:07.884172+0000","last_became_peered":"2024-03-30T00:42:07.884172+0000","last_unstale":"2024-03-30T00:42:07.887534+0000","last_undegraded":"2024-03-30T00:42:07.887534+0000","last_fullsized":"2024-03-30T00:42:07.887534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:24.488599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879706+0000","last_change":"2024-03-30T00:42:07.879706+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879706+0000","last_undegraded":"2024-03-30T00:42:07.879706+0000","last_fullsized":"2024-03-30T00:42:07.879706+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887038+0000","last_change":"2024-03-30T00:42:07.884001+0000","last_active":"2024-03-30T00:42:07.887038+0000","last_peered":"2024-03-30T00:42:07.887038+0000","last_clean":"2024-03-30T00:42:07.887038+0000","last_became_active":"2024-03-30T00:42:07.883690+0000","last_became_peered":"2024-03-30T00:42:07.883690+0000","last_unstale":"2024-03-30T00:42:07.887038+0000","last_undegraded":"2024-03-30T00:42:07.887038+0000","last_fullsized":"2024-03-30T00:42:07.887038+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:26.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873193+0000","last_change":"2024-03-30T00:42:07.873193+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873193+0000","last_undegraded":"2024-03-30T00:42:07.873193+0000","last_fullsized":"2024-03-30T00:42:07.873193+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:22.099119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893642+0000","last_change":"2024-03-30T00:42:07.887955+0000","last_active":"2024-03-30T00:42:07.893642+0000","last_peered":"2024-03-30T00:42:07.893642+0000","last_clean":"2024-03-30T00:42:07.893642+0000","last_became_active":"2024-03-30T00:42:07.887689+0000","last_became_peered":"2024-03-30T00:42:07.887689+0000","last_unstale":"2024-03-30T00:42:07.893642+0000","last_undegraded":"2024-03-30T00:42:07.893642+0000","last_fullsized":"2024-03-30T00:42:07.893642+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:17:21.438797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873054+0000","last_change":"2024-03-30T00:42:07.873054+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873054+0000","last_undegraded":"2024-03-30T00:42:07.873054+0000","last_fullsized":"2024-03-30T00:42:07.873054+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887686+0000","last_change":"2024-03-30T00:42:07.884866+0000","last_active":"2024-03-30T00:42:07.887686+0000","last_peered":"2024-03-30T00:42:07.887686+0000","last_clean":"2024-03-30T00:42:07.887686+0000","last_became_active":"2024-03-30T00:42:07.884550+0000","last_became_peered":"2024-03-30T00:42:07.884550+0000","last_unstale":"2024-03-30T00:42:07.887686+0000","last_undegraded":"2024-03-30T00:42:07.887686+0000","last_fullsized":"2024-03-30T00:42:07.887686+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:10:19.003693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876922+0000","last_change":"2024-03-30T00:42:07.876922+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876922+0000","last_undegraded":"2024-03-30T00:42:07.876922+0000","last_fullsized":"2024-03-30T00:42:07.876922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.874579+0000","last_change":"2024-03-30T00:42:07.874579+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874579+0000","last_undegraded":"2024-03-30T00:42:07.874579+0000","last_fullsized":"2024-03-30T00:42:07.874579+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:17.278798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881886+0000","last_change":"2024-03-30T00:42:07.881886+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881886+0000","last_undegraded":"2024-03-30T00:42:07.881886+0000","last_fullsized":"2024-03-30T00:42:07.881886+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:07.667668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891360+0000","last_change":"2024-03-30T00:42:07.889759+0000","last_active":"2024-03-30T00:42:07.891360+0000","last_peered":"2024-03-30T00:42:07.891360+0000","last_clean":"2024-03-30T00:42:07.891360+0000","last_became_active":"2024-03-30T00:42:07.888856+0000","last_became_peered":"2024-03-30T00:42:07.888856+0000","last_unstale":"2024-03-30T00:42:07.891360+0000","last_undegraded":"2024-03-30T00:42:07.891360+0000","last_fullsized":"2024-03-30T00:42:07.891360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:00:01.121929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873860+0000","last_change":"2024-03-30T00:42:07.873860+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873860+0000","last_undegraded":"2024-03-30T00:42:07.873860+0000","last_fullsized":"2024-03-30T00:42:07.873860+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.878493+0000","last_change":"2024-03-30T00:42:07.878493+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878493+0000","last_undegraded":"2024-03-30T00:42:07.878493+0000","last_fullsized":"2024-03-30T00:42:07.878493+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:48.023033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881591+0000","last_change":"2024-03-30T00:42:07.881591+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881591+0000","last_undegraded":"2024-03-30T00:42:07.881591+0000","last_fullsized":"2024-03-30T00:42:07.881591+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.896028+0000","last_change":"2024-03-30T00:42:07.896028+0000","last_active":"2024-03-30T00:42:07.896028+0000","last_peered":"2024-03-30T00:42:07.896028+0000","last_clean":"2024-03-30T00:42:07.896028+0000","last_became_active":"2024-03-30T00:42:07.895683+0000","last_became_peered":"2024-03-30T00:42:07.895683+0000","last_unstale":"2024-03-30T00:42:07.896028+0000","last_undegraded":"2024-03-30T00:42:07.896028+0000","last_fullsized":"2024-03-30T00:42:07.896028+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:37.405718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888318+0000","last_change":"2024-03-30T00:42:07.888318+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888318+0000","last_undegraded":"2024-03-30T00:42:07.888318+0000","last_fullsized":"2024-03-30T00:42:07.888318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.877016+0000","last_change":"2024-03-30T00:42:07.877016+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877016+0000","last_undegraded":"2024-03-30T00:42:07.877016+0000","last_fullsized":"2024-03-30T00:42:07.877016+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:58:41.813204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880083+0000","last_change":"2024-03-30T00:42:07.880083+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880083+0000","last_undegraded":"2024-03-30T00:42:07.880083+0000","last_fullsized":"2024-03-30T00:42:07.880083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:44.118291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.889095+0000","last_change":"2024-03-30T00:42:07.886045+0000","last_active":"2024-03-30T00:42:07.889095+0000","last_peered":"2024-03-30T00:42:07.889095+0000","last_clean":"2024-03-30T00:42:07.889095+0000","last_became_active":"2024-03-30T00:42:07.885765+0000","last_became_peered":"2024-03-30T00:42:07.885765+0000","last_unstale":"2024-03-30T00:42:07.889095+0000","last_undegraded":"2024-03-30T00:42:07.889095+0000","last_fullsized":"2024-03-30T00:42:07.889095+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:21.533654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888542+0000","last_change":"2024-03-30T00:42:07.888542+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888542+0000","last_undegraded":"2024-03-30T00:42:07.888542+0000","last_fullsized":"2024-03-30T00:42:07.888542+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:01.442897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.876507+0000","last_change":"2024-03-30T00:42:07.876507+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876507+0000","last_undegraded":"2024-03-30T00:42:07.876507+0000","last_fullsized":"2024-03-30T00:42:07.876507+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:16:59.791004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876783+0000","last_change":"2024-03-30T00:42:07.876783+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876783+0000","last_undegraded":"2024-03-30T00:42:07.876783+0000","last_fullsized":"2024-03-30T00:42:07.876783+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.898205+0000","last_change":"2024-03-30T00:42:07.898205+0000","last_active":"2024-03-30T00:42:07.898205+0000","last_peered":"2024-03-30T00:42:07.898205+0000","last_clean":"2024-03-30T00:42:07.898205+0000","last_became_active":"2024-03-30T00:42:07.897873+0000","last_became_peered":"2024-03-30T00:42:07.897873+0000","last_unstale":"2024-03-30T00:42:07.898205+0000","last_undegraded":"2024-03-30T00:42:07.898205+0000","last_fullsized":"2024-03-30T00:42:07.898205+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:28.263763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873297+0000","last_change":"2024-03-30T00:42:07.873297+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873297+0000","last_undegraded":"2024-03-30T00:42:07.873297+0000","last_fullsized":"2024-03-30T00:42:07.873297+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890926+0000","last_change":"2024-03-30T00:42:07.890926+0000","last_active":"2024-03-30T00:42:07.890926+0000","last_peered":"2024-03-30T00:42:07.890926+0000","last_clean":"2024-03-30T00:42:07.890926+0000","last_became_active":"2024-03-30T00:42:07.890676+0000","last_became_peered":"2024-03-30T00:42:07.890676+0000","last_unstale":"2024-03-30T00:42:07.890926+0000","last_undegraded":"2024-03-30T00:42:07.890926+0000","last_fullsized":"2024-03-30T00:42:07.890926+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:24:47.097376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873770+0000","last_change":"2024-03-30T00:42:07.873770+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873770+0000","last_undegraded":"2024-03-30T00:42:07.873770+0000","last_fullsized":"2024-03-30T00:42:07.873770+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890669+0000","last_change":"2024-03-30T00:42:07.888916+0000","last_active":"2024-03-30T00:42:07.890669+0000","last_peered":"2024-03-30T00:42:07.890669+0000","last_clean":"2024-03-30T00:42:07.890669+0000","last_became_active":"2024-03-30T00:42:07.888629+0000","last_became_peered":"2024-03-30T00:42:07.888629+0000","last_unstale":"2024-03-30T00:42:07.890669+0000","last_undegraded":"2024-03-30T00:42:07.890669+0000","last_fullsized":"2024-03-30T00:42:07.890669+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:03.807636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872778+0000","last_change":"2024-03-30T00:42:07.872778+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872778+0000","last_undegraded":"2024-03-30T00:42:07.872778+0000","last_fullsized":"2024-03-30T00:42:07.872778+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890933+0000","last_change":"2024-03-30T00:42:07.889736+0000","last_active":"2024-03-30T00:42:07.890933+0000","last_peered":"2024-03-30T00:42:07.890933+0000","last_clean":"2024-03-30T00:42:07.890933+0000","last_became_active":"2024-03-30T00:42:07.889425+0000","last_became_peered":"2024-03-30T00:42:07.889425+0000","last_unstale":"2024-03-30T00:42:07.890933+0000","last_undegraded":"2024-03-30T00:42:07.890933+0000","last_fullsized":"2024-03-30T00:42:07.890933+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:41.441678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879997+0000","last_change":"2024-03-30T00:42:07.879997+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879997+0000","last_undegraded":"2024-03-30T00:42:07.879997+0000","last_fullsized":"2024-03-30T00:42:07.879997+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:42.969334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.884117+0000","last_change":"2024-03-30T00:42:07.884117+0000","last_active":"2024-03-30T00:42:07.884117+0000","last_peered":"2024-03-30T00:42:07.884117+0000","last_clean":"2024-03-30T00:42:07.884117+0000","last_became_active":"2024-03-30T00:42:07.883887+0000","last_became_peered":"2024-03-30T00:42:07.883887+0000","last_unstale":"2024-03-30T00:42:07.884117+0000","last_undegraded":"2024-03-30T00:42:07.884117+0000","last_fullsized":"2024-03-30T00:42:07.884117+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:18.896534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880105+0000","last_change":"2024-03-30T00:42:07.880105+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880105+0000","last_undegraded":"2024-03-30T00:42:07.880105+0000","last_fullsized":"2024-03-30T00:42:07.880105+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:00:02.124627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893214+0000","last_change":"2024-03-30T00:42:07.893214+0000","last_active":"2024-03-30T00:42:07.893214+0000","last_peered":"2024-03-30T00:42:07.893214+0000","last_clean":"2024-03-30T00:42:07.893214+0000","last_became_active":"2024-03-30T00:42:07.892814+0000","last_became_peered":"2024-03-30T00:42:07.892814+0000","last_unstale":"2024-03-30T00:42:07.893214+0000","last_undegraded":"2024-03-30T00:42:07.893214+0000","last_fullsized":"2024-03-30T00:42:07.893214+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:52.010424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.889733+0000","last_change":"2024-03-30T00:42:07.889733+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.889733+0000","last_undegraded":"2024-03-30T00:42:07.889733+0000","last_fullsized":"2024-03-30T00:42:07.889733+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.896209+0000","last_change":"2024-03-30T00:42:07.896209+0000","last_active":"2024-03-30T00:42:07.896209+0000","last_peered":"2024-03-30T00:42:07.896209+0000","last_clean":"2024-03-30T00:42:07.896209+0000","last_became_active":"2024-03-30T00:42:07.895871+0000","last_became_peered":"2024-03-30T00:42:07.895871+0000","last_unstale":"2024-03-30T00:42:07.896209+0000","last_undegraded":"2024-03-30T00:42:07.896209+0000","last_fullsized":"2024-03-30T00:42:07.896209+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:24.948608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872629+0000","last_change":"2024-03-30T00:42:07.872629+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872629+0000","last_undegraded":"2024-03-30T00:42:07.872629+0000","last_fullsized":"2024-03-30T00:42:07.872629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891909+0000","last_change":"2024-03-30T00:42:07.891909+0000","last_active":"2024-03-30T00:42:07.891909+0000","last_peered":"2024-03-30T00:42:07.891909+0000","last_clean":"2024-03-30T00:42:07.891909+0000","last_became_active":"2024-03-30T00:42:07.891660+0000","last_became_peered":"2024-03-30T00:42:07.891660+0000","last_unstale":"2024-03-30T00:42:07.891909+0000","last_undegraded":"2024-03-30T00:42:07.891909+0000","last_fullsized":"2024-03-30T00:42:07.891909+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:32.488321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876796+0000","last_change":"2024-03-30T00:42:07.876796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876796+0000","last_undegraded":"2024-03-30T00:42:07.876796+0000","last_fullsized":"2024-03-30T00:42:07.876796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:40.974026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.878452+0000","last_change":"2024-03-30T00:42:07.878452+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878452+0000","last_undegraded":"2024-03-30T00:42:07.878452+0000","last_fullsized":"2024-03-30T00:42:07.878452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:30.469686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872796+0000","last_change":"2024-03-30T00:42:07.872796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872796+0000","last_undegraded":"2024-03-30T00:42:07.872796+0000","last_fullsized":"2024-03-30T00:42:07.872796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:30:50.535530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.876204+0000","last_change":"2024-03-30T00:42:07.876204+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876204+0000","last_undegraded":"2024-03-30T00:42:07.876204+0000","last_fullsized":"2024-03-30T00:42:07.876204+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:02:33.633690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876968+0000","last_change":"2024-03-30T00:42:07.876968+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876968+0000","last_undegraded":"2024-03-30T00:42:07.876968+0000","last_fullsized":"2024-03-30T00:42:07.876968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:58.102992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.899036+0000","last_change":"2024-03-30T00:42:07.895012+0000","last_active":"2024-03-30T00:42:07.899036+0000","last_peered":"2024-03-30T00:42:07.899036+0000","last_clean":"2024-03-30T00:42:07.899036+0000","last_became_active":"2024-03-30T00:42:07.894725+0000","last_became_peered":"2024-03-30T00:42:07.894725+0000","last_unstale":"2024-03-30T00:42:07.899036+0000","last_undegraded":"2024-03-30T00:42:07.899036+0000","last_fullsized":"2024-03-30T00:42:07.899036+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:41:59.532828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.892109+0000","last_change":"2024-03-30T00:42:07.892109+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.892109+0000","last_undegraded":"2024-03-30T00:42:07.892109+0000","last_fullsized":"2024-03-30T00:42:07.892109+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893706+0000","last_change":"2024-03-30T00:42:07.893706+0000","last_active":"2024-03-30T00:42:07.893706+0000","last_peered":"2024-03-30T00:42:07.893706+0000","last_clean":"2024-03-30T00:42:07.893706+0000","last_became_active":"2024-03-30T00:42:07.893465+0000","last_became_peered":"2024-03-30T00:42:07.893465+0000","last_unstale":"2024-03-30T00:42:07.893706+0000","last_undegraded":"2024-03-30T00:42:07.893706+0000","last_fullsized":"2024-03-30T00:42:07.893706+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:41.966636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878795+0000","last_change":"2024-03-30T00:42:07.878795+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878795+0000","last_undegraded":"2024-03-30T00:42:07.878795+0000","last_fullsized":"2024-03-30T00:42:07.878795+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893058+0000","last_change":"2024-03-30T00:42:07.893058+0000","last_active":"2024-03-30T00:42:07.893058+0000","last_peered":"2024-03-30T00:42:07.893058+0000","last_clean":"2024-03-30T00:42:07.893058+0000","last_became_active":"2024-03-30T00:42:07.892453+0000","last_became_peered":"2024-03-30T00:42:07.892453+0000","last_unstale":"2024-03-30T00:42:07.893058+0000","last_undegraded":"2024-03-30T00:42:07.893058+0000","last_fullsized":"2024-03-30T00:42:07.893058+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:27.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.890675+0000","last_change":"2024-03-30T00:42:07.890675+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.890675+0000","last_undegraded":"2024-03-30T00:42:07.890675+0000","last_fullsized":"2024-03-30T00:42:07.890675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891397+0000","last_change":"2024-03-30T00:42:07.889478+0000","last_active":"2024-03-30T00:42:07.891397+0000","last_peered":"2024-03-30T00:42:07.891397+0000","last_clean":"2024-03-30T00:42:07.891397+0000","last_became_active":"2024-03-30T00:42:07.889247+0000","last_became_peered":"2024-03-30T00:42:07.889247+0000","last_unstale":"2024-03-30T00:42:07.891397+0000","last_undegraded":"2024-03-30T00:42:07.891397+0000","last_fullsized":"2024-03-30T00:42:07.891397+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:01.336339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877866+0000","last_change":"2024-03-30T00:42:07.877866+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877866+0000","last_undegraded":"2024-03-30T00:42:07.877866+0000","last_fullsized":"2024-03-30T00:42:07.877866+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891637+0000","last_change":"2024-03-30T00:42:07.891209+0000","last_active":"2024-03-30T00:42:07.891637+0000","last_peered":"2024-03-30T00:42:07.891637+0000","last_clean":"2024-03-30T00:42:07.891637+0000","last_became_active":"2024-03-30T00:42:07.890769+0000","last_became_peered":"2024-03-30T00:42:07.890769+0000","last_unstale":"2024-03-30T00:42:07.891637+0000","last_undegraded":"2024-03-30T00:42:07.891637+0000","last_fullsized":"2024-03-30T00:42:07.891637+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:21.096421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879604+0000","last_change":"2024-03-30T00:42:07.879604+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879604+0000","last_undegraded":"2024-03-30T00:42:07.879604+0000","last_fullsized":"2024-03-30T00:42:07.879604+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:02:34.636388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.899549+0000","last_change":"2024-03-30T00:42:07.899549+0000","last_active":"2024-03-30T00:42:07.899549+0000","last_peered":"2024-03-30T00:42:07.899549+0000","last_clean":"2024-03-30T00:42:07.899549+0000","last_became_active":"2024-03-30T00:42:07.896786+0000","last_became_peered":"2024-03-30T00:42:07.896786+0000","last_unstale":"2024-03-30T00:42:07.899549+0000","last_undegraded":"2024-03-30T00:42:07.899549+0000","last_fullsized":"2024-03-30T00:42:07.899549+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:24.488599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-30T00:42:06.865006+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:42:06.865006+0000","last_peered":"2024-03-30T00:42:06.865006+0000","last_clean":"2024-03-30T00:42:06.865006+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:42:06.865006+0000","last_undegraded":"2024-03-30T00:42:06.865006+0000","last_fullsized":"2024-03-30T00:42:06.865006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1200128,"data_stored":854159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.77100000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.62},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.93300000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.626},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.51600000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.747}]},{"osd":2,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78100000000000003,"15min":0.78900000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.194,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.79400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.026,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0840000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82899999999999996},"min":{"1min":0.65100000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0429999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.873}]},{"osd":4,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63500000000000001,"15min":0.629},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.996,"5min":1.026,"15min":1.026},"last":0.97499999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.45400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.89200000000000002,"15min":0.93799999999999994},"min":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90100000000000002,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78300000000000003,"5min":1.47,"15min":1.47},"last":0.70299999999999996}]},{"osd":1,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.90100000000000002,"15min":0.95299999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76900000000000002,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.97299999999999998,"15min":1.0329999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.82399999999999995,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.69299999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.96999999999999997,"15min":1.018},"min":{"1min":0.48699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.82299999999999995,"5min":4.375,"15min":4.375},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.97199999999999998,"15min":1.024},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":4.2960000000000003,"15min":4.2960000000000003},"last":0.79000000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.98799999999999999,"15min":1.038},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97299999999999998,"5min":4.3399999999999999,"15min":4.3399999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.97499999999999998,"15min":1.034},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.73299999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96999999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.42399999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.095},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.78700000000000003,"15min":0.81000000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95099999999999996,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.86299999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.751,"15min":0.76600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77400000000000002,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.72299999999999998,"15min":0.73899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83599999999999997,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90600000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.70899999999999996,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79700000000000004,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.81000000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.79200000000000004,"15min":0.81200000000000006},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98499999999999999,"5min":1.7,"15min":1.7},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.80200000000000005,"5min":1.181,"15min":1.181},"last":0.83499999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63400000000000001,"15min":0.64400000000000002},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91500000000000004,"5min":1.615,"15min":1.615},"last":0.89100000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88100000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.877},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0700000000000001,"5min":1.569,"15min":1.569},"last":0.85199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76000000000000001,"15min":0.76300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89300000000000002,"5min":1.006,"15min":1.006},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.83899999999999997,"15min":0.84899999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.032,"5min":1.383,"15min":1.383},"last":0.92500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77900000000000003,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.44600000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51700000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71499999999999997,"15min":0.72299999999999998},"min":{"1min":0.50700000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.872,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54200000000000004}]},{"osd":2,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.91000000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.755,"15min":0.76300000000000001},"min":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96899999999999997,"5min":1.228,"15min":1.228},"last":0.58499999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.50700000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94899999999999995,"5min":1.27,"15min":1.27},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.751,"5min":0.78300000000000003,"15min":0.78900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91400000000000003,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84899999999999998,"5min":1.329,"15min":1.329},"last":0.438},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93500000000000005,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98899999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95799999999999996,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.497}]},{"osd":7,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.749,"15min":0.76300000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.069,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.747}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.56899999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.53200000000000003,"15min":0.55300000000000005},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.378},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59799999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.623,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.69299999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40500000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.76100000000000001,"15min":0.77600000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98299999999999998,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.872},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.77100000000000002,"15min":0.78500000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0069999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.90200000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.78200000000000003,"15min":0.79300000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.95199999999999996,"5min":1.125,"15min":1.125},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80600000000000005,"15min":0.82099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.024,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.81200000000000006,"15min":0.82699999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0389999999999999,"5min":1.226,"15min":1.226},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.79700000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0109999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.94899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.85099999999999998,"15min":0.86899999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.077,"5min":1.202,"15min":1.202},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.88400000000000001,"15min":0.90800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0740000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.91800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.313,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.627,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":4.0999999999999996},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.51100000000000001,"15min":0.52700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.56200000000000006,"5min":0.747,"15min":0.747},"last":1.9239999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.58199999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":3.1139999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53200000000000003,"15min":0.54400000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.63,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":2.4159999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.495,"15min":0.503},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.53400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76200000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":1.907}]},{"osd":4,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80300000000000005,"15min":0.81499999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.042,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.9470000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.127,"5min":1.528,"15min":1.528},"last":1.972}]},{"osd":5,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83099999999999996,"15min":0.84499999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1240000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":2.4569999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.80600000000000005,"15min":0.82299999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0580000000000001,"5min":1.131,"15min":1.131},"last":2.8039999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.81000000000000005,"15min":0.82099999999999995},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.081,"5min":1.151,"15min":1.151},"last":2.8849999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.179,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.867}]},{"osd":7,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.85999999999999999,"15min":0.88200000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.01,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":2.3690000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0700000000000001,"5min":1.228,"15min":1.228},"last":2.851}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61299999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.46000000000000002,"15min":0.46899999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55700000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.498}]},{"osd":3,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66200000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.626,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92000000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.83199999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77600000000000002,"15min":0.78000000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93400000000000005,"5min":1.097,"15min":1.097},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.90100000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78900000000000003,"15min":0.79500000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95199999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92800000000000005,"5min":0.996,"15min":0.996},"last":0.93400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1200128,"data_stored":854159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.29299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58499999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.61299999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66100000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.623},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":1.006}]},{"osd":3,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71499999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.96899999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79100000000000004,"15min":0.79400000000000004},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.008,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.234},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79900000000000004,"15min":0.80000000000000004},"min":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1279999999999999,"5min":1.153,"15min":1.153},"last":1.252}]},{"osd":5,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.621,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0489999999999999,"5min":1.22,"15min":1.22},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.92300000000000004,"5min":1.462,"15min":1.462},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78900000000000003,"15min":0.79800000000000004},"min":{"1min":0.58299999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.091,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.56299999999999994,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1,"5min":1.085,"15min":1.085},"last":1.2}]},{"osd":7,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.91500000000000004,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.351},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96899999999999997,"5min":1.556,"15min":1.556},"last":1.085}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:42:12.607 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:42:12.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.772+0000 7f9f60600640 1 Processor -- start 2024-03-30T00:42:12.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.772+0000 7f9f60600640 1 -- start start 2024-03-30T00:42:12.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f60600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9f5815ca80 0x7f9f5806b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f60600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9f580cb570 0x7f9f580cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f60600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f580cb0e0 con 0x7f9f580cf2f0 2024-03-30T00:42:12.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f60600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f580cf7f0 con 0x7f9f580cb570 2024-03-30T00:42:12.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f60600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f580cfad0 con 0x7f9f5815ca80 2024-03-30T00:42:12.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f5d200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9f580cf2f0 0x7f9f5806b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55660/0 (socket says 172.21.15.66:55660) 2024-03-30T00:42:12.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f5d200640 1 -- 172.21.15.66:0/4060352341 learned_addr learned my addr 172.21.15.66:0/4060352341 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:12.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f5e600640 1 --2- 172.21.15.66:0/4060352341 >> v2:172.21.15.66:3300/0 conn(0x7f9f580cb570 0x7f9f580cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f5dc00640 1 --2- 172.21.15.66:0/4060352341 >> v2:172.21.15.156:3300/0 conn(0x7f9f5815ca80 0x7f9f5806b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f57e00640 1 -- 172.21.15.66:0/4060352341 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172493686 0 0) 0x7f9f580cb0e0 con 0x7f9f580cf2f0 2024-03-30T00:42:12.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f5e600640 1 -- 172.21.15.66:0/4060352341 >> v2:172.21.15.156:3300/0 conn(0x7f9f5815ca80 msgr2=0x7f9f5806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:12.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f5e600640 1 --2- 172.21.15.66:0/4060352341 >> v2:172.21.15.156:3300/0 conn(0x7f9f5815ca80 0x7f9f5806b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f5e600640 1 -- 172.21.15.66:0/4060352341 >> v1:172.21.15.66:6789/0 conn(0x7f9f580cf2f0 legacy=0x7f9f5806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f5e600640 1 -- 172.21.15.66:0/4060352341 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f580cad70 con 0x7f9f580cb570 2024-03-30T00:42:12.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.776+0000 7f9f5e600640 1 --2- 172.21.15.66:0/4060352341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f580cb570 0x7f9f580cba20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9f58070ed0 tx=0x7f9f5001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84dfa47cd9d3956 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:12.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f57e00640 1 -- 172.21.15.66:0/4060352341 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500af600 con 0x7f9f580cb570 2024-03-30T00:42:12.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f57e00640 1 -- 172.21.15.66:0/4060352341 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f500ab050 con 0x7f9f580cb570 2024-03-30T00:42:12.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f57e00640 1 -- 172.21.15.66:0/4060352341 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500af980 con 0x7f9f580cb570 2024-03-30T00:42:12.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 -- 172.21.15.66:0/4060352341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f580cb570 msgr2=0x7f9f580cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 --2- 172.21.15.66:0/4060352341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f580cb570 0x7f9f580cba20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9f58070ed0 tx=0x7f9f5001f110 comp rx=0 tx=0).stop 2024-03-30T00:42:12.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 -- 172.21.15.66:0/4060352341 shutdown_connections 2024-03-30T00:42:12.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 --2- 172.21.15.66:0/4060352341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f580cb570 0x7f9f580cba20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 --2- 172.21.15.66:0/4060352341 >> v2:172.21.15.156:3300/0 conn(0x7f9f5815ca80 0x7f9f5806b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 -- 172.21.15.66:0/4060352341 >> 172.21.15.66:0/4060352341 conn(0x7f9f580b2090 msgr2=0x7f9f580b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:12.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 -- 172.21.15.66:0/4060352341 shutdown_connections 2024-03-30T00:42:12.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 -- 172.21.15.66:0/4060352341 wait complete. 2024-03-30T00:42:12.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 Processor -- start 2024-03-30T00:42:12.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 -- start start 2024-03-30T00:42:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f580cb570 0x7f9f5815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f580cf2f0 0x7f9f5809cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f5815ca80 0x7f9f5809d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581f21e0 con 0x7f9f580cf2f0 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5815ac30 con 0x7f9f5815ca80 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f60600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5809dad0 con 0x7f9f580cb570 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f5d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f580cf2f0 0x7f9f5809cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f5d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f580cf2f0 0x7f9f5809cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36316/0 (socket says 172.21.15.66:36316) 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f5d200640 1 -- 172.21.15.66:0/3248246378 learned_addr learned my addr 172.21.15.66:0/3248246378 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f5e600640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f5815ca80 0x7f9f5809d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f5dc00640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f580cb570 0x7f9f5815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f5d200640 1 -- 172.21.15.66:0/3248246378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f580cb570 msgr2=0x7f9f5815a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f5d200640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f580cb570 0x7f9f5815a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f5d200640 1 -- 172.21.15.66:0/3248246378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f5815ca80 msgr2=0x7f9f5809d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.780+0000 7f9f5d200640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f5815ca80 0x7f9f5809d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f5d200640 1 -- 172.21.15.66:0/3248246378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009a70 con 0x7f9f580cf2f0 2024-03-30T00:42:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f5e600640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f5815ca80 0x7f9f5809d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f5dc00640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f580cb570 0x7f9f5815a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f5d200640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f580cf2f0 0x7f9f5809cf80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9f440011f0 tx=0x7f9f4400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:12.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f55600640 1 -- 172.21.15.66:0/3248246378 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f440074e0 con 0x7f9f580cf2f0 2024-03-30T00:42:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f55600640 1 -- 172.21.15.66:0/3248246378 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f44007640 con 0x7f9f580cf2f0 2024-03-30T00:42:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f55600640 1 -- 172.21.15.66:0/3248246378 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44004a10 con 0x7f9f580cf2f0 2024-03-30T00:42:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f60600640 1 -- 172.21.15.66:0/3248246378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5809ddb0 con 0x7f9f580cf2f0 2024-03-30T00:42:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f60600640 1 -- 172.21.15.66:0/3248246378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f580c8d20 con 0x7f9f580cf2f0 2024-03-30T00:42:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f60600640 1 -- 172.21.15.66:0/3248246378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5809e2d0 con 0x7f9f580cf2f0 2024-03-30T00:42:12.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f55600640 1 -- 172.21.15.66:0/3248246378 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9f44004b70 con 0x7f9f580cf2f0 2024-03-30T00:42:12.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f55600640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f38066690 0x7f9f38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:12.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f55600640 1 -- 172.21.15.66:0/3248246378 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9f44083520 con 0x7f9f580cf2f0 2024-03-30T00:42:12.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.784+0000 7f9f5dc00640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f38066690 0x7f9f38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:12.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.788+0000 7f9f5dc00640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f38066690 0x7f9f38068b50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9f580b5ee0 tx=0x7f9f4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:12.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.788+0000 7f9f55600640 1 -- 172.21.15.66:0/3248246378 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9f440507e0 con 0x7f9f580cf2f0 2024-03-30T00:42:12.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.952+0000 7f9f60600640 1 -- 172.21.15.66:0/3248246378 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f5806bd00 con 0x7f9f38066690 2024-03-30T00:42:12.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.960+0000 7f9f55600640 1 -- 172.21.15.66:0/3248246378 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285961 (secure 0 0 0) 0x7f9f4c0096d0 con 0x7f9f38066690 2024-03-30T00:42:12.973 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:12.974 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:42:12.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 -- 172.21.15.66:0/3248246378 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f38066690 msgr2=0x7f9f38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f38066690 0x7f9f38068b50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9f580b5ee0 tx=0x7f9f4c008040 comp rx=0 tx=0).stop 2024-03-30T00:42:12.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 -- 172.21.15.66:0/3248246378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f580cf2f0 msgr2=0x7f9f5809cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:12.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f580cf2f0 0x7f9f5809cf80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9f440011f0 tx=0x7f9f4400c9b0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 -- 172.21.15.66:0/3248246378 shutdown_connections 2024-03-30T00:42:12.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f5815ca80 0x7f9f5809d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f38066690 0x7f9f38068b50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f580cf2f0 0x7f9f5809cf80 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 --2- 172.21.15.66:0/3248246378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f580cb570 0x7f9f5815a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:12.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 -- 172.21.15.66:0/3248246378 >> 172.21.15.66:0/3248246378 conn(0x7f9f580b2090 msgr2=0x7f9f580cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:12.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 -- 172.21.15.66:0/3248246378 shutdown_connections 2024-03-30T00:42:12.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:12.964+0000 7f9f60600640 1 -- 172.21.15.66:0/3248246378 wait complete. 2024-03-30T00:42:12.993 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-03-30T00:42:12.879585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238452,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754736268,"statfs":{"total":773094113280,"available":772849938432,"internally_reserved":0,"allocated":6037504,"data_stored":3291064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000958"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874156+0000","last_change":"2024-03-30T00:42:07.874156+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874156+0000","last_undegraded":"2024-03-30T00:42:07.874156+0000","last_fullsized":"2024-03-30T00:42:07.874156+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873050+0000","last_change":"2024-03-30T00:42:07.873050+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873050+0000","last_undegraded":"2024-03-30T00:42:07.873050+0000","last_fullsized":"2024-03-30T00:42:07.873050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.889268+0000","last_change":"2024-03-30T00:42:07.889268+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.889268+0000","last_undegraded":"2024-03-30T00:42:07.889268+0000","last_fullsized":"2024-03-30T00:42:07.889268+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872559+0000","last_change":"2024-03-30T00:42:07.872559+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872559+0000","last_undegraded":"2024-03-30T00:42:07.872559+0000","last_fullsized":"2024-03-30T00:42:07.872559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874398+0000","last_change":"2024-03-30T00:42:07.874398+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874398+0000","last_undegraded":"2024-03-30T00:42:07.874398+0000","last_fullsized":"2024-03-30T00:42:07.874398+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873733+0000","last_change":"2024-03-30T00:42:07.873733+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873733+0000","last_undegraded":"2024-03-30T00:42:07.873733+0000","last_fullsized":"2024-03-30T00:42:07.873733+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:22.099119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881871+0000","last_change":"2024-03-30T00:42:07.881871+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881871+0000","last_undegraded":"2024-03-30T00:42:07.881871+0000","last_fullsized":"2024-03-30T00:42:07.881871+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:09.745566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877328+0000","last_change":"2024-03-30T00:42:07.877328+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877328+0000","last_undegraded":"2024-03-30T00:42:07.877328+0000","last_fullsized":"2024-03-30T00:42:07.877328+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:01.749453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878241+0000","last_change":"2024-03-30T00:42:07.878241+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878241+0000","last_undegraded":"2024-03-30T00:42:07.878241+0000","last_fullsized":"2024-03-30T00:42:07.878241+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874651+0000","last_change":"2024-03-30T00:42:07.874651+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874651+0000","last_undegraded":"2024-03-30T00:42:07.874651+0000","last_fullsized":"2024-03-30T00:42:07.874651+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879446+0000","last_change":"2024-03-30T00:42:07.879446+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879446+0000","last_undegraded":"2024-03-30T00:42:07.879446+0000","last_fullsized":"2024-03-30T00:42:07.879446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:04.810334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878717+0000","last_change":"2024-03-30T00:42:07.878717+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878717+0000","last_undegraded":"2024-03-30T00:42:07.878717+0000","last_fullsized":"2024-03-30T00:42:07.878717+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874011+0000","last_change":"2024-03-30T00:42:07.874011+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874011+0000","last_undegraded":"2024-03-30T00:42:07.874011+0000","last_fullsized":"2024-03-30T00:42:07.874011+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:03.287560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.882341+0000","last_change":"2024-03-30T00:42:07.882341+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.882341+0000","last_undegraded":"2024-03-30T00:42:07.882341+0000","last_fullsized":"2024-03-30T00:42:07.882341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:54:01.171986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.871624+0000","last_change":"2024-03-30T00:42:07.871624+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.871624+0000","last_undegraded":"2024-03-30T00:42:07.871624+0000","last_fullsized":"2024-03-30T00:42:07.871624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:56.870783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872827+0000","last_change":"2024-03-30T00:42:07.872827+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872827+0000","last_undegraded":"2024-03-30T00:42:07.872827+0000","last_fullsized":"2024-03-30T00:42:07.872827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:24.904667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888103+0000","last_change":"2024-03-30T00:42:07.888103+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888103+0000","last_undegraded":"2024-03-30T00:42:07.888103+0000","last_fullsized":"2024-03-30T00:42:07.888103+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:41:31.279358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872622+0000","last_change":"2024-03-30T00:42:07.872622+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872622+0000","last_undegraded":"2024-03-30T00:42:07.872622+0000","last_fullsized":"2024-03-30T00:42:07.872622+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888796+0000","last_change":"2024-03-30T00:42:07.888796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888796+0000","last_undegraded":"2024-03-30T00:42:07.888796+0000","last_fullsized":"2024-03-30T00:42:07.888796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877961+0000","last_change":"2024-03-30T00:42:07.877961+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877961+0000","last_undegraded":"2024-03-30T00:42:07.877961+0000","last_fullsized":"2024-03-30T00:42:07.877961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:10:20.006391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877522+0000","last_change":"2024-03-30T00:42:07.877522+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877522+0000","last_undegraded":"2024-03-30T00:42:07.877522+0000","last_fullsized":"2024-03-30T00:42:07.877522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:19.899232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877834+0000","last_change":"2024-03-30T00:42:07.877834+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877834+0000","last_undegraded":"2024-03-30T00:42:07.877834+0000","last_fullsized":"2024-03-30T00:42:07.877834+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878875+0000","last_change":"2024-03-30T00:42:07.878875+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878875+0000","last_undegraded":"2024-03-30T00:42:07.878875+0000","last_fullsized":"2024-03-30T00:42:07.878875+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:26.322503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881115+0000","last_change":"2024-03-30T00:42:07.881115+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881115+0000","last_undegraded":"2024-03-30T00:42:07.881115+0000","last_fullsized":"2024-03-30T00:42:07.881115+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:34:13.403177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.887228+0000","last_change":"2024-03-30T00:42:07.887228+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.887228+0000","last_undegraded":"2024-03-30T00:42:07.887228+0000","last_fullsized":"2024-03-30T00:42:07.887228+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880458+0000","last_change":"2024-03-30T00:42:07.880458+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880458+0000","last_undegraded":"2024-03-30T00:42:07.880458+0000","last_fullsized":"2024-03-30T00:42:07.880458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872899+0000","last_change":"2024-03-30T00:42:07.872899+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872899+0000","last_undegraded":"2024-03-30T00:42:07.872899+0000","last_fullsized":"2024-03-30T00:42:07.872899+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:22.536352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877458+0000","last_change":"2024-03-30T00:42:07.877458+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877458+0000","last_undegraded":"2024-03-30T00:42:07.877458+0000","last_fullsized":"2024-03-30T00:42:07.877458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:20.469591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.893998+0000","last_change":"2024-03-30T00:42:07.893998+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.893998+0000","last_undegraded":"2024-03-30T00:42:07.893998+0000","last_fullsized":"2024-03-30T00:42:07.893998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872039+0000","last_change":"2024-03-30T00:42:07.872039+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872039+0000","last_undegraded":"2024-03-30T00:42:07.872039+0000","last_fullsized":"2024-03-30T00:42:07.872039+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881306+0000","last_change":"2024-03-30T00:42:07.881306+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881306+0000","last_undegraded":"2024-03-30T00:42:07.881306+0000","last_fullsized":"2024-03-30T00:42:07.881306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:49.025731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877550+0000","last_change":"2024-03-30T00:42:07.877550+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877550+0000","last_undegraded":"2024-03-30T00:42:07.877550+0000","last_fullsized":"2024-03-30T00:42:07.877550+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.871624+0000","last_change":"2024-03-30T00:42:07.871624+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.871624+0000","last_undegraded":"2024-03-30T00:42:07.871624+0000","last_fullsized":"2024-03-30T00:42:07.871624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:17:22.441495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890610+0000","last_change":"2024-03-30T00:42:07.887914+0000","last_active":"2024-03-30T00:42:07.890610+0000","last_peered":"2024-03-30T00:42:07.890610+0000","last_clean":"2024-03-30T00:42:07.890610+0000","last_became_active":"2024-03-30T00:42:07.887626+0000","last_became_peered":"2024-03-30T00:42:07.887626+0000","last_unstale":"2024-03-30T00:42:07.890610+0000","last_undegraded":"2024-03-30T00:42:07.890610+0000","last_fullsized":"2024-03-30T00:42:07.890610+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:25.319805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876251+0000","last_change":"2024-03-30T00:42:07.876251+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876251+0000","last_undegraded":"2024-03-30T00:42:07.876251+0000","last_fullsized":"2024-03-30T00:42:07.876251+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:29.266461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890823+0000","last_change":"2024-03-30T00:42:07.890823+0000","last_active":"2024-03-30T00:42:07.890823+0000","last_peered":"2024-03-30T00:42:07.890823+0000","last_clean":"2024-03-30T00:42:07.890823+0000","last_became_active":"2024-03-30T00:42:07.888331+0000","last_became_peered":"2024-03-30T00:42:07.888331+0000","last_unstale":"2024-03-30T00:42:07.890823+0000","last_undegraded":"2024-03-30T00:42:07.890823+0000","last_fullsized":"2024-03-30T00:42:07.890823+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:48:23.842037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878821+0000","last_change":"2024-03-30T00:42:07.878821+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878821+0000","last_undegraded":"2024-03-30T00:42:07.878821+0000","last_fullsized":"2024-03-30T00:42:07.878821+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:03.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.884824+0000","last_change":"2024-03-30T00:42:07.884824+0000","last_active":"2024-03-30T00:42:07.884824+0000","last_peered":"2024-03-30T00:42:07.884824+0000","last_clean":"2024-03-30T00:42:07.884824+0000","last_became_active":"2024-03-30T00:42:07.884503+0000","last_became_peered":"2024-03-30T00:42:07.884503+0000","last_unstale":"2024-03-30T00:42:07.884824+0000","last_undegraded":"2024-03-30T00:42:07.884824+0000","last_fullsized":"2024-03-30T00:42:07.884824+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:18:21.768136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877433+0000","last_change":"2024-03-30T00:42:07.877433+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877433+0000","last_undegraded":"2024-03-30T00:42:07.877433+0000","last_fullsized":"2024-03-30T00:42:07.877433+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.892398+0000","last_change":"2024-03-30T00:42:07.892398+0000","last_active":"2024-03-30T00:42:07.892398+0000","last_peered":"2024-03-30T00:42:07.892398+0000","last_clean":"2024-03-30T00:42:07.892398+0000","last_became_active":"2024-03-30T00:42:07.892097+0000","last_became_peered":"2024-03-30T00:42:07.892097+0000","last_unstale":"2024-03-30T00:42:07.892398+0000","last_undegraded":"2024-03-30T00:42:07.892398+0000","last_fullsized":"2024-03-30T00:42:07.892398+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:30:49.532832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874348+0000","last_change":"2024-03-30T00:42:07.874348+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874348+0000","last_undegraded":"2024-03-30T00:42:07.874348+0000","last_fullsized":"2024-03-30T00:42:07.874348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:59.202589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.897983+0000","last_change":"2024-03-30T00:42:07.897983+0000","last_active":"2024-03-30T00:42:07.897983+0000","last_peered":"2024-03-30T00:42:07.897983+0000","last_clean":"2024-03-30T00:42:07.897983+0000","last_became_active":"2024-03-30T00:42:07.897719+0000","last_became_peered":"2024-03-30T00:42:07.897719+0000","last_unstale":"2024-03-30T00:42:07.897983+0000","last_undegraded":"2024-03-30T00:42:07.897983+0000","last_fullsized":"2024-03-30T00:42:07.897983+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:55.868085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873753+0000","last_change":"2024-03-30T00:42:07.873753+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873753+0000","last_undegraded":"2024-03-30T00:42:07.873753+0000","last_fullsized":"2024-03-30T00:42:07.873753+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887534+0000","last_change":"2024-03-30T00:42:07.884431+0000","last_active":"2024-03-30T00:42:07.887534+0000","last_peered":"2024-03-30T00:42:07.887534+0000","last_clean":"2024-03-30T00:42:07.887534+0000","last_became_active":"2024-03-30T00:42:07.884172+0000","last_became_peered":"2024-03-30T00:42:07.884172+0000","last_unstale":"2024-03-30T00:42:07.887534+0000","last_undegraded":"2024-03-30T00:42:07.887534+0000","last_fullsized":"2024-03-30T00:42:07.887534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:24.488599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879706+0000","last_change":"2024-03-30T00:42:07.879706+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879706+0000","last_undegraded":"2024-03-30T00:42:07.879706+0000","last_fullsized":"2024-03-30T00:42:07.879706+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887038+0000","last_change":"2024-03-30T00:42:07.884001+0000","last_active":"2024-03-30T00:42:07.887038+0000","last_peered":"2024-03-30T00:42:07.887038+0000","last_clean":"2024-03-30T00:42:07.887038+0000","last_became_active":"2024-03-30T00:42:07.883690+0000","last_became_peered":"2024-03-30T00:42:07.883690+0000","last_unstale":"2024-03-30T00:42:07.887038+0000","last_undegraded":"2024-03-30T00:42:07.887038+0000","last_fullsized":"2024-03-30T00:42:07.887038+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:26.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873193+0000","last_change":"2024-03-30T00:42:07.873193+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873193+0000","last_undegraded":"2024-03-30T00:42:07.873193+0000","last_fullsized":"2024-03-30T00:42:07.873193+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:22.099119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893642+0000","last_change":"2024-03-30T00:42:07.887955+0000","last_active":"2024-03-30T00:42:07.893642+0000","last_peered":"2024-03-30T00:42:07.893642+0000","last_clean":"2024-03-30T00:42:07.893642+0000","last_became_active":"2024-03-30T00:42:07.887689+0000","last_became_peered":"2024-03-30T00:42:07.887689+0000","last_unstale":"2024-03-30T00:42:07.893642+0000","last_undegraded":"2024-03-30T00:42:07.893642+0000","last_fullsized":"2024-03-30T00:42:07.893642+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:17:21.438797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873054+0000","last_change":"2024-03-30T00:42:07.873054+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873054+0000","last_undegraded":"2024-03-30T00:42:07.873054+0000","last_fullsized":"2024-03-30T00:42:07.873054+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887686+0000","last_change":"2024-03-30T00:42:07.884866+0000","last_active":"2024-03-30T00:42:07.887686+0000","last_peered":"2024-03-30T00:42:07.887686+0000","last_clean":"2024-03-30T00:42:07.887686+0000","last_became_active":"2024-03-30T00:42:07.884550+0000","last_became_peered":"2024-03-30T00:42:07.884550+0000","last_unstale":"2024-03-30T00:42:07.887686+0000","last_undegraded":"2024-03-30T00:42:07.887686+0000","last_fullsized":"2024-03-30T00:42:07.887686+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:10:19.003693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876922+0000","last_change":"2024-03-30T00:42:07.876922+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876922+0000","last_undegraded":"2024-03-30T00:42:07.876922+0000","last_fullsized":"2024-03-30T00:42:07.876922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.874579+0000","last_change":"2024-03-30T00:42:07.874579+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874579+0000","last_undegraded":"2024-03-30T00:42:07.874579+0000","last_fullsized":"2024-03-30T00:42:07.874579+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:17.278798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881886+0000","last_change":"2024-03-30T00:42:07.881886+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881886+0000","last_undegraded":"2024-03-30T00:42:07.881886+0000","last_fullsized":"2024-03-30T00:42:07.881886+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:07.667668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891360+0000","last_change":"2024-03-30T00:42:07.889759+0000","last_active":"2024-03-30T00:42:07.891360+0000","last_peered":"2024-03-30T00:42:07.891360+0000","last_clean":"2024-03-30T00:42:07.891360+0000","last_became_active":"2024-03-30T00:42:07.888856+0000","last_became_peered":"2024-03-30T00:42:07.888856+0000","last_unstale":"2024-03-30T00:42:07.891360+0000","last_undegraded":"2024-03-30T00:42:07.891360+0000","last_fullsized":"2024-03-30T00:42:07.891360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:00:01.121929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873860+0000","last_change":"2024-03-30T00:42:07.873860+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873860+0000","last_undegraded":"2024-03-30T00:42:07.873860+0000","last_fullsized":"2024-03-30T00:42:07.873860+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.878493+0000","last_change":"2024-03-30T00:42:07.878493+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878493+0000","last_undegraded":"2024-03-30T00:42:07.878493+0000","last_fullsized":"2024-03-30T00:42:07.878493+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:48.023033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881591+0000","last_change":"2024-03-30T00:42:07.881591+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881591+0000","last_undegraded":"2024-03-30T00:42:07.881591+0000","last_fullsized":"2024-03-30T00:42:07.881591+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.896028+0000","last_change":"2024-03-30T00:42:07.896028+0000","last_active":"2024-03-30T00:42:07.896028+0000","last_peered":"2024-03-30T00:42:07.896028+0000","last_clean":"2024-03-30T00:42:07.896028+0000","last_became_active":"2024-03-30T00:42:07.895683+0000","last_became_peered":"2024-03-30T00:42:07.895683+0000","last_unstale":"2024-03-30T00:42:07.896028+0000","last_undegraded":"2024-03-30T00:42:07.896028+0000","last_fullsized":"2024-03-30T00:42:07.896028+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:37.405718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888318+0000","last_change":"2024-03-30T00:42:07.888318+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888318+0000","last_undegraded":"2024-03-30T00:42:07.888318+0000","last_fullsized":"2024-03-30T00:42:07.888318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.877016+0000","last_change":"2024-03-30T00:42:07.877016+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877016+0000","last_undegraded":"2024-03-30T00:42:07.877016+0000","last_fullsized":"2024-03-30T00:42:07.877016+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:58:41.813204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880083+0000","last_change":"2024-03-30T00:42:07.880083+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880083+0000","last_undegraded":"2024-03-30T00:42:07.880083+0000","last_fullsized":"2024-03-30T00:42:07.880083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:44.118291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.889095+0000","last_change":"2024-03-30T00:42:07.886045+0000","last_active":"2024-03-30T00:42:07.889095+0000","last_peered":"2024-03-30T00:42:07.889095+0000","last_clean":"2024-03-30T00:42:07.889095+0000","last_became_active":"2024-03-30T00:42:07.885765+0000","last_became_peered":"2024-03-30T00:42:07.885765+0000","last_unstale":"2024-03-30T00:42:07.889095+0000","last_undegraded":"2024-03-30T00:42:07.889095+0000","last_fullsized":"2024-03-30T00:42:07.889095+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:21.533654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888542+0000","last_change":"2024-03-30T00:42:07.888542+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888542+0000","last_undegraded":"2024-03-30T00:42:07.888542+0000","last_fullsized":"2024-03-30T00:42:07.888542+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:01.442897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.876507+0000","last_change":"2024-03-30T00:42:07.876507+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876507+0000","last_undegraded":"2024-03-30T00:42:07.876507+0000","last_fullsized":"2024-03-30T00:42:07.876507+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:16:59.791004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876783+0000","last_change":"2024-03-30T00:42:07.876783+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876783+0000","last_undegraded":"2024-03-30T00:42:07.876783+0000","last_fullsized":"2024-03-30T00:42:07.876783+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.898205+0000","last_change":"2024-03-30T00:42:07.898205+0000","last_active":"2024-03-30T00:42:07.898205+0000","last_peered":"2024-03-30T00:42:07.898205+0000","last_clean":"2024-03-30T00:42:07.898205+0000","last_became_active":"2024-03-30T00:42:07.897873+0000","last_became_peered":"2024-03-30T00:42:07.897873+0000","last_unstale":"2024-03-30T00:42:07.898205+0000","last_undegraded":"2024-03-30T00:42:07.898205+0000","last_fullsized":"2024-03-30T00:42:07.898205+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:28.263763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873297+0000","last_change":"2024-03-30T00:42:07.873297+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873297+0000","last_undegraded":"2024-03-30T00:42:07.873297+0000","last_fullsized":"2024-03-30T00:42:07.873297+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890926+0000","last_change":"2024-03-30T00:42:07.890926+0000","last_active":"2024-03-30T00:42:07.890926+0000","last_peered":"2024-03-30T00:42:07.890926+0000","last_clean":"2024-03-30T00:42:07.890926+0000","last_became_active":"2024-03-30T00:42:07.890676+0000","last_became_peered":"2024-03-30T00:42:07.890676+0000","last_unstale":"2024-03-30T00:42:07.890926+0000","last_undegraded":"2024-03-30T00:42:07.890926+0000","last_fullsized":"2024-03-30T00:42:07.890926+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:24:47.097376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873770+0000","last_change":"2024-03-30T00:42:07.873770+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873770+0000","last_undegraded":"2024-03-30T00:42:07.873770+0000","last_fullsized":"2024-03-30T00:42:07.873770+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890669+0000","last_change":"2024-03-30T00:42:07.888916+0000","last_active":"2024-03-30T00:42:07.890669+0000","last_peered":"2024-03-30T00:42:07.890669+0000","last_clean":"2024-03-30T00:42:07.890669+0000","last_became_active":"2024-03-30T00:42:07.888629+0000","last_became_peered":"2024-03-30T00:42:07.888629+0000","last_unstale":"2024-03-30T00:42:07.890669+0000","last_undegraded":"2024-03-30T00:42:07.890669+0000","last_fullsized":"2024-03-30T00:42:07.890669+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:03.807636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872778+0000","last_change":"2024-03-30T00:42:07.872778+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872778+0000","last_undegraded":"2024-03-30T00:42:07.872778+0000","last_fullsized":"2024-03-30T00:42:07.872778+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890933+0000","last_change":"2024-03-30T00:42:07.889736+0000","last_active":"2024-03-30T00:42:07.890933+0000","last_peered":"2024-03-30T00:42:07.890933+0000","last_clean":"2024-03-30T00:42:07.890933+0000","last_became_active":"2024-03-30T00:42:07.889425+0000","last_became_peered":"2024-03-30T00:42:07.889425+0000","last_unstale":"2024-03-30T00:42:07.890933+0000","last_undegraded":"2024-03-30T00:42:07.890933+0000","last_fullsized":"2024-03-30T00:42:07.890933+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:41.441678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879997+0000","last_change":"2024-03-30T00:42:07.879997+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879997+0000","last_undegraded":"2024-03-30T00:42:07.879997+0000","last_fullsized":"2024-03-30T00:42:07.879997+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:42.969334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.884117+0000","last_change":"2024-03-30T00:42:07.884117+0000","last_active":"2024-03-30T00:42:07.884117+0000","last_peered":"2024-03-30T00:42:07.884117+0000","last_clean":"2024-03-30T00:42:07.884117+0000","last_became_active":"2024-03-30T00:42:07.883887+0000","last_became_peered":"2024-03-30T00:42:07.883887+0000","last_unstale":"2024-03-30T00:42:07.884117+0000","last_undegraded":"2024-03-30T00:42:07.884117+0000","last_fullsized":"2024-03-30T00:42:07.884117+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:18.896534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880105+0000","last_change":"2024-03-30T00:42:07.880105+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880105+0000","last_undegraded":"2024-03-30T00:42:07.880105+0000","last_fullsized":"2024-03-30T00:42:07.880105+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:00:02.124627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893214+0000","last_change":"2024-03-30T00:42:07.893214+0000","last_active":"2024-03-30T00:42:07.893214+0000","last_peered":"2024-03-30T00:42:07.893214+0000","last_clean":"2024-03-30T00:42:07.893214+0000","last_became_active":"2024-03-30T00:42:07.892814+0000","last_became_peered":"2024-03-30T00:42:07.892814+0000","last_unstale":"2024-03-30T00:42:07.893214+0000","last_undegraded":"2024-03-30T00:42:07.893214+0000","last_fullsized":"2024-03-30T00:42:07.893214+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:52.010424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.889733+0000","last_change":"2024-03-30T00:42:07.889733+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.889733+0000","last_undegraded":"2024-03-30T00:42:07.889733+0000","last_fullsized":"2024-03-30T00:42:07.889733+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.896209+0000","last_change":"2024-03-30T00:42:07.896209+0000","last_active":"2024-03-30T00:42:07.896209+0000","last_peered":"2024-03-30T00:42:07.896209+0000","last_clean":"2024-03-30T00:42:07.896209+0000","last_became_active":"2024-03-30T00:42:07.895871+0000","last_became_peered":"2024-03-30T00:42:07.895871+0000","last_unstale":"2024-03-30T00:42:07.896209+0000","last_undegraded":"2024-03-30T00:42:07.896209+0000","last_fullsized":"2024-03-30T00:42:07.896209+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:24.948608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872629+0000","last_change":"2024-03-30T00:42:07.872629+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872629+0000","last_undegraded":"2024-03-30T00:42:07.872629+0000","last_fullsized":"2024-03-30T00:42:07.872629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891909+0000","last_change":"2024-03-30T00:42:07.891909+0000","last_active":"2024-03-30T00:42:07.891909+0000","last_peered":"2024-03-30T00:42:07.891909+0000","last_clean":"2024-03-30T00:42:07.891909+0000","last_became_active":"2024-03-30T00:42:07.891660+0000","last_became_peered":"2024-03-30T00:42:07.891660+0000","last_unstale":"2024-03-30T00:42:07.891909+0000","last_undegraded":"2024-03-30T00:42:07.891909+0000","last_fullsized":"2024-03-30T00:42:07.891909+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:32.488321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876796+0000","last_change":"2024-03-30T00:42:07.876796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876796+0000","last_undegraded":"2024-03-30T00:42:07.876796+0000","last_fullsized":"2024-03-30T00:42:07.876796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:40.974026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.878452+0000","last_change":"2024-03-30T00:42:07.878452+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878452+0000","last_undegraded":"2024-03-30T00:42:07.878452+0000","last_fullsized":"2024-03-30T00:42:07.878452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:30.469686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872796+0000","last_change":"2024-03-30T00:42:07.872796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872796+0000","last_undegraded":"2024-03-30T00:42:07.872796+0000","last_fullsized":"2024-03-30T00:42:07.872796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:30:50.535530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.876204+0000","last_change":"2024-03-30T00:42:07.876204+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876204+0000","last_undegraded":"2024-03-30T00:42:07.876204+0000","last_fullsized":"2024-03-30T00:42:07.876204+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:02:33.633690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876968+0000","last_change":"2024-03-30T00:42:07.876968+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876968+0000","last_undegraded":"2024-03-30T00:42:07.876968+0000","last_fullsized":"2024-03-30T00:42:07.876968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:58.102992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.899036+0000","last_change":"2024-03-30T00:42:07.895012+0000","last_active":"2024-03-30T00:42:07.899036+0000","last_peered":"2024-03-30T00:42:07.899036+0000","last_clean":"2024-03-30T00:42:07.899036+0000","last_became_active":"2024-03-30T00:42:07.894725+0000","last_became_peered":"2024-03-30T00:42:07.894725+0000","last_unstale":"2024-03-30T00:42:07.899036+0000","last_undegraded":"2024-03-30T00:42:07.899036+0000","last_fullsized":"2024-03-30T00:42:07.899036+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:41:59.532828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.892109+0000","last_change":"2024-03-30T00:42:07.892109+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.892109+0000","last_undegraded":"2024-03-30T00:42:07.892109+0000","last_fullsized":"2024-03-30T00:42:07.892109+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893706+0000","last_change":"2024-03-30T00:42:07.893706+0000","last_active":"2024-03-30T00:42:07.893706+0000","last_peered":"2024-03-30T00:42:07.893706+0000","last_clean":"2024-03-30T00:42:07.893706+0000","last_became_active":"2024-03-30T00:42:07.893465+0000","last_became_peered":"2024-03-30T00:42:07.893465+0000","last_unstale":"2024-03-30T00:42:07.893706+0000","last_undegraded":"2024-03-30T00:42:07.893706+0000","last_fullsized":"2024-03-30T00:42:07.893706+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:41.966636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878795+0000","last_change":"2024-03-30T00:42:07.878795+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878795+0000","last_undegraded":"2024-03-30T00:42:07.878795+0000","last_fullsized":"2024-03-30T00:42:07.878795+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893058+0000","last_change":"2024-03-30T00:42:07.893058+0000","last_active":"2024-03-30T00:42:07.893058+0000","last_peered":"2024-03-30T00:42:07.893058+0000","last_clean":"2024-03-30T00:42:07.893058+0000","last_became_active":"2024-03-30T00:42:07.892453+0000","last_became_peered":"2024-03-30T00:42:07.892453+0000","last_unstale":"2024-03-30T00:42:07.893058+0000","last_undegraded":"2024-03-30T00:42:07.893058+0000","last_fullsized":"2024-03-30T00:42:07.893058+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:27.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.890675+0000","last_change":"2024-03-30T00:42:07.890675+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.890675+0000","last_undegraded":"2024-03-30T00:42:07.890675+0000","last_fullsized":"2024-03-30T00:42:07.890675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891397+0000","last_change":"2024-03-30T00:42:07.889478+0000","last_active":"2024-03-30T00:42:07.891397+0000","last_peered":"2024-03-30T00:42:07.891397+0000","last_clean":"2024-03-30T00:42:07.891397+0000","last_became_active":"2024-03-30T00:42:07.889247+0000","last_became_peered":"2024-03-30T00:42:07.889247+0000","last_unstale":"2024-03-30T00:42:07.891397+0000","last_undegraded":"2024-03-30T00:42:07.891397+0000","last_fullsized":"2024-03-30T00:42:07.891397+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:01.336339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877866+0000","last_change":"2024-03-30T00:42:07.877866+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877866+0000","last_undegraded":"2024-03-30T00:42:07.877866+0000","last_fullsized":"2024-03-30T00:42:07.877866+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891637+0000","last_change":"2024-03-30T00:42:07.891209+0000","last_active":"2024-03-30T00:42:07.891637+0000","last_peered":"2024-03-30T00:42:07.891637+0000","last_clean":"2024-03-30T00:42:07.891637+0000","last_became_active":"2024-03-30T00:42:07.890769+0000","last_became_peered":"2024-03-30T00:42:07.890769+0000","last_unstale":"2024-03-30T00:42:07.891637+0000","last_undegraded":"2024-03-30T00:42:07.891637+0000","last_fullsized":"2024-03-30T00:42:07.891637+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:21.096421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879604+0000","last_change":"2024-03-30T00:42:07.879604+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879604+0000","last_undegraded":"2024-03-30T00:42:07.879604+0000","last_fullsized":"2024-03-30T00:42:07.879604+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:02:34.636388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.899549+0000","last_change":"2024-03-30T00:42:07.899549+0000","last_active":"2024-03-30T00:42:07.899549+0000","last_peered":"2024-03-30T00:42:07.899549+0000","last_clean":"2024-03-30T00:42:07.899549+0000","last_became_active":"2024-03-30T00:42:07.896786+0000","last_became_peered":"2024-03-30T00:42:07.896786+0000","last_unstale":"2024-03-30T00:42:07.899549+0000","last_undegraded":"2024-03-30T00:42:07.899549+0000","last_fullsized":"2024-03-30T00:42:07.899549+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:24.488599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-30T00:42:06.865006+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:42:06.865006+0000","last_peered":"2024-03-30T00:42:06.865006+0000","last_clean":"2024-03-30T00:42:06.865006+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:42:06.865006+0000","last_undegraded":"2024-03-30T00:42:06.865006+0000","last_fullsized":"2024-03-30T00:42:06.865006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1200128,"data_stored":854159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.77100000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.62},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.93300000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.626},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.51600000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.747}]},{"osd":2,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78100000000000003,"15min":0.78900000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.194,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.79400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.026,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0840000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82899999999999996},"min":{"1min":0.65100000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0429999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.873}]},{"osd":4,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63500000000000001,"15min":0.629},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.996,"5min":1.026,"15min":1.026},"last":0.97499999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.45400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.89200000000000002,"15min":0.93799999999999994},"min":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90100000000000002,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78300000000000003,"5min":1.47,"15min":1.47},"last":0.70299999999999996}]},{"osd":1,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.90100000000000002,"15min":0.95299999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76900000000000002,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.97299999999999998,"15min":1.0329999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.82399999999999995,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.69299999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.96999999999999997,"15min":1.018},"min":{"1min":0.48699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.82299999999999995,"5min":4.375,"15min":4.375},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.97199999999999998,"15min":1.024},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":4.2960000000000003,"15min":4.2960000000000003},"last":0.79000000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.98799999999999999,"15min":1.038},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97299999999999998,"5min":4.3399999999999999,"15min":4.3399999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.97499999999999998,"15min":1.034},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.73299999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96999999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.42399999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.095},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.78700000000000003,"15min":0.81000000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95099999999999996,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.86299999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.751,"15min":0.76600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77400000000000002,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.72299999999999998,"15min":0.73899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83599999999999997,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90600000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.70899999999999996,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79700000000000004,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.81000000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.79200000000000004,"15min":0.81200000000000006},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98499999999999999,"5min":1.7,"15min":1.7},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.80200000000000005,"5min":1.181,"15min":1.181},"last":0.83499999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63400000000000001,"15min":0.64400000000000002},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91500000000000004,"5min":1.615,"15min":1.615},"last":0.89100000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88100000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.877},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0700000000000001,"5min":1.569,"15min":1.569},"last":0.85199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76000000000000001,"15min":0.76300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89300000000000002,"5min":1.006,"15min":1.006},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.83899999999999997,"15min":0.84899999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.032,"5min":1.383,"15min":1.383},"last":0.92500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77900000000000003,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.44600000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51700000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71499999999999997,"15min":0.72299999999999998},"min":{"1min":0.50700000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.872,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54200000000000004}]},{"osd":2,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.91000000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.755,"15min":0.76300000000000001},"min":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96899999999999997,"5min":1.228,"15min":1.228},"last":0.58499999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.50700000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94899999999999995,"5min":1.27,"15min":1.27},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.751,"5min":0.78300000000000003,"15min":0.78900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91400000000000003,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84899999999999998,"5min":1.329,"15min":1.329},"last":0.438},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93500000000000005,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98899999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95799999999999996,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.497}]},{"osd":7,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.749,"15min":0.76300000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.069,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.747}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.56899999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.53200000000000003,"15min":0.55300000000000005},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.378},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59799999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.623,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.69299999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40500000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.76100000000000001,"15min":0.77600000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98299999999999998,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.872},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.77100000000000002,"15min":0.78500000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0069999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.90200000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.78200000000000003,"15min":0.79300000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.95199999999999996,"5min":1.125,"15min":1.125},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80600000000000005,"15min":0.82099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.024,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.81200000000000006,"15min":0.82699999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0389999999999999,"5min":1.226,"15min":1.226},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.79700000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0109999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.94899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.85099999999999998,"15min":0.86899999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.077,"5min":1.202,"15min":1.202},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.88400000000000001,"15min":0.90800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0740000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.91800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.313,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.627,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":4.0999999999999996},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.51100000000000001,"15min":0.52700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.56200000000000006,"5min":0.747,"15min":0.747},"last":1.9239999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.58199999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":3.1139999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53200000000000003,"15min":0.54400000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.63,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":2.4159999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.495,"15min":0.503},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.53400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76200000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":1.907}]},{"osd":4,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80300000000000005,"15min":0.81499999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.042,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.9470000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.127,"5min":1.528,"15min":1.528},"last":1.972}]},{"osd":5,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83099999999999996,"15min":0.84499999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1240000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":2.4569999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.80600000000000005,"15min":0.82299999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0580000000000001,"5min":1.131,"15min":1.131},"last":2.8039999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.81000000000000005,"15min":0.82099999999999995},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.081,"5min":1.151,"15min":1.151},"last":2.8849999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.179,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.867}]},{"osd":7,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.85999999999999999,"15min":0.88200000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.01,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":2.3690000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0700000000000001,"5min":1.228,"15min":1.228},"last":2.851}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61299999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.46000000000000002,"15min":0.46899999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55700000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.498}]},{"osd":3,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66200000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.626,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92000000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.83199999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77600000000000002,"15min":0.78000000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93400000000000005,"5min":1.097,"15min":1.097},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.90100000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78900000000000003,"15min":0.79500000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95199999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92800000000000005,"5min":0.996,"15min":0.996},"last":0.93400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1200128,"data_stored":854159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.29299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58499999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.61299999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66100000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.623},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":1.006}]},{"osd":3,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71499999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.96899999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79100000000000004,"15min":0.79400000000000004},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.008,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.234},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79900000000000004,"15min":0.80000000000000004},"min":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1279999999999999,"5min":1.153,"15min":1.153},"last":1.252}]},{"osd":5,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.621,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0489999999999999,"5min":1.22,"15min":1.22},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.92300000000000004,"5min":1.462,"15min":1.462},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78900000000000003,"15min":0.79800000000000004},"min":{"1min":0.58299999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.091,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.56299999999999994,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1,"5min":1.085,"15min":1.085},"last":1.2}]},{"osd":7,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.91500000000000004,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.351},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96899999999999997,"5min":1.556,"15min":1.556},"last":1.085}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:42:12.997 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:42:13.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.164+0000 7fb2d2000640 1 Processor -- start 2024-03-30T00:42:13.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.164+0000 7fb2d2000640 1 -- start start 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.164+0000 7fb2d2000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb2cc0cd2d0 0x7fb2cc0cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.164+0000 7fb2d2000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2cc0cc8d0 0x7fb2cc06c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.164+0000 7fb2d2000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2cc0cb620 con 0x7fb2cc0cad00 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.164+0000 7fb2d2000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2cc0cbec0 con 0x7fb2cc0cc8d0 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.164+0000 7fb2d2000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2cc0ca660 con 0x7fb2cc0cd2d0 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.164+0000 7fb2cb400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb2cc0cad00 0x7fb2cc0c8940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55668/0 (socket says 172.21.15.66:55668) 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.164+0000 7fb2cb400640 1 -- 172.21.15.66:0/1441698229 learned_addr learned my addr 172.21.15.66:0/1441698229 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.164+0000 7fb2cbe00640 1 --2- 172.21.15.66:0/1441698229 >> v2:172.21.15.66:3300/0 conn(0x7fb2cc0cc8d0 0x7fb2cc06c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2c3e00640 1 --2- 172.21.15.66:0/1441698229 >> v2:172.21.15.156:3300/0 conn(0x7fb2cc0cd2d0 0x7fb2cc0cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2caa00640 1 -- 172.21.15.66:0/1441698229 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200345727 0 0) 0x7fb2cc0cb620 con 0x7fb2cc0cad00 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2cbe00640 1 -- 172.21.15.66:0/1441698229 >> v2:172.21.15.156:3300/0 conn(0x7fb2cc0cd2d0 msgr2=0x7fb2cc0cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2cbe00640 1 --2- 172.21.15.66:0/1441698229 >> v2:172.21.15.156:3300/0 conn(0x7fb2cc0cd2d0 0x7fb2cc0cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2cbe00640 1 -- 172.21.15.66:0/1441698229 >> v1:172.21.15.66:6789/0 conn(0x7fb2cc0cad00 legacy=0x7fb2cc0c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2cbe00640 1 -- 172.21.15.66:0/1441698229 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc0ca940 con 0x7fb2cc0cc8d0 2024-03-30T00:42:13.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2cbe00640 1 --2- 172.21.15.66:0/1441698229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc0cc8d0 0x7fb2cc06c030 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0cba10 tx=0x7fb2c401da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c276d2ed044b9778 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:13.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2caa00640 1 -- 172.21.15.66:0/1441698229 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c400a2d0 con 0x7fb2cc0cc8d0 2024-03-30T00:42:13.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2caa00640 1 -- 172.21.15.66:0/1441698229 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c40a68e0 con 0x7fb2cc0cc8d0 2024-03-30T00:42:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2caa00640 1 -- 172.21.15.66:0/1441698229 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40a6c70 con 0x7fb2cc0cc8d0 2024-03-30T00:42:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2d2000640 1 -- 172.21.15.66:0/1441698229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc0cc8d0 msgr2=0x7fb2cc06c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.168+0000 7fb2d2000640 1 --2- 172.21.15.66:0/1441698229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc0cc8d0 0x7fb2cc06c030 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0cba10 tx=0x7fb2c401da20 comp rx=0 tx=0).stop 2024-03-30T00:42:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 -- 172.21.15.66:0/1441698229 shutdown_connections 2024-03-30T00:42:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 --2- 172.21.15.66:0/1441698229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc0cc8d0 0x7fb2cc06c030 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 --2- 172.21.15.66:0/1441698229 >> v2:172.21.15.156:3300/0 conn(0x7fb2cc0cd2d0 0x7fb2cc0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 -- 172.21.15.66:0/1441698229 >> 172.21.15.66:0/1441698229 conn(0x7fb2cc069f60 msgr2=0x7fb2cc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 -- 172.21.15.66:0/1441698229 shutdown_connections 2024-03-30T00:42:13.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 -- 172.21.15.66:0/1441698229 wait complete. 2024-03-30T00:42:13.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 Processor -- start 2024-03-30T00:42:13.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 -- start start 2024-03-30T00:42:13.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc0cad00 0x7fb2cc097860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2cc0cc8d0 0x7fb2cc1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2cc0cd2d0 0x7fb2cc159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc157010 con 0x7fb2cc0cad00 2024-03-30T00:42:13.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc1572f0 con 0x7fb2cc0cc8d0 2024-03-30T00:42:13.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2d2000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc1575d0 con 0x7fb2cc0cd2d0 2024-03-30T00:42:13.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2cb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc0cad00 0x7fb2cc097860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2cb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc0cad00 0x7fb2cc097860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36332/0 (socket says 172.21.15.66:36332) 2024-03-30T00:42:13.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2cb400640 1 -- 172.21.15.66:0/2787265991 learned_addr learned my addr 172.21.15.66:0/2787265991 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:13.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2c3e00640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2cc0cc8d0 0x7fb2cc1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2cbe00640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2cc0cd2d0 0x7fb2cc159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2cb400640 1 -- 172.21.15.66:0/2787265991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2cc0cd2d0 msgr2=0x7fb2cc159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2cb400640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2cc0cd2d0 0x7fb2cc159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2cb400640 1 -- 172.21.15.66:0/2787265991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2cc0cc8d0 msgr2=0x7fb2cc1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2cb400640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2cc0cc8d0 0x7fb2cc1565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2cb400640 1 -- 172.21.15.66:0/2787265991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c400ab80 con 0x7fb2cc0cad00 2024-03-30T00:42:13.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2cbe00640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2cc0cd2d0 0x7fb2cc159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:13.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2cb400640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc0cad00 0x7fb2cc097860 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c480 tx=0x7fb2bc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:13.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2c3400640 1 -- 172.21.15.66:0/2787265991 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0074e0 con 0x7fb2cc0cad00 2024-03-30T00:42:13.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2c3400640 1 -- 172.21.15.66:0/2787265991 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc007660 con 0x7fb2cc0cad00 2024-03-30T00:42:13.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.172+0000 7fb2c3400640 1 -- 172.21.15.66:0/2787265991 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc004a30 con 0x7fb2cc0cad00 2024-03-30T00:42:13.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.176+0000 7fb2d2000640 1 -- 172.21.15.66:0/2787265991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc1578b0 con 0x7fb2cc0cad00 2024-03-30T00:42:13.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.176+0000 7fb2d2000640 1 -- 172.21.15.66:0/2787265991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2cc1580b0 con 0x7fb2cc0cad00 2024-03-30T00:42:13.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.176+0000 7fb2d2000640 1 -- 172.21.15.66:0/2787265991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2cc066c30 con 0x7fb2cc0cad00 2024-03-30T00:42:13.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.176+0000 7fb2c3400640 1 -- 172.21.15.66:0/2787265991 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb2bc004b90 con 0x7fb2cc0cad00 2024-03-30T00:42:13.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.176+0000 7fb2c3400640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2a80666c0 0x7fb2a8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.176+0000 7fb2c3400640 1 -- 172.21.15.66:0/2787265991 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb2bc0833b0 con 0x7fb2cc0cad00 2024-03-30T00:42:13.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.176+0000 7fb2c3e00640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2a80666c0 0x7fb2a8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.180+0000 7fb2c3e00640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2a80666c0 0x7fb2a8068b80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0c88d0 tx=0x7fb2b4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:13.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.184+0000 7fb2c3400640 1 -- 172.21.15.66:0/2787265991 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb2bc054110 con 0x7fb2cc0cad00 2024-03-30T00:42:13.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.340+0000 7fb2d2000640 1 -- 172.21.15.66:0/2787265991 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2cc157460 con 0x7fb2a80666c0 2024-03-30T00:42:13.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.348+0000 7fb2c3400640 1 -- 172.21.15.66:0/2787265991 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285961 (secure 0 0 0) 0x7fb2cc157460 con 0x7fb2a80666c0 2024-03-30T00:42:13.361 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:13.363 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:42:13.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 -- 172.21.15.66:0/2787265991 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2a80666c0 msgr2=0x7fb2a8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2a80666c0 0x7fb2a8068b80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0c88d0 tx=0x7fb2b4009190 comp rx=0 tx=0).stop 2024-03-30T00:42:13.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 -- 172.21.15.66:0/2787265991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc0cad00 msgr2=0x7fb2cc097860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc0cad00 0x7fb2cc097860 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c480 tx=0x7fb2bc00c940 comp rx=0 tx=0).stop 2024-03-30T00:42:13.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 -- 172.21.15.66:0/2787265991 shutdown_connections 2024-03-30T00:42:13.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2cc0cd2d0 0x7fb2cc159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2cc0cc8d0 0x7fb2cc1565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2a80666c0 0x7fb2a8068b80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 --2- 172.21.15.66:0/2787265991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc0cad00 0x7fb2cc097860 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 -- 172.21.15.66:0/2787265991 >> 172.21.15.66:0/2787265991 conn(0x7fb2cc069f60 msgr2=0x7fb2cc0af170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:13.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 -- 172.21.15.66:0/2787265991 shutdown_connections 2024-03-30T00:42:13.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.352+0000 7fb2d2000640 1 -- 172.21.15.66:0/2787265991 wait complete. 2024-03-30T00:42:13.382 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-03-30T00:42:12.879585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238452,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754736268,"statfs":{"total":773094113280,"available":772849938432,"internally_reserved":0,"allocated":6037504,"data_stored":3291064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000958"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874156+0000","last_change":"2024-03-30T00:42:07.874156+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874156+0000","last_undegraded":"2024-03-30T00:42:07.874156+0000","last_fullsized":"2024-03-30T00:42:07.874156+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873050+0000","last_change":"2024-03-30T00:42:07.873050+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873050+0000","last_undegraded":"2024-03-30T00:42:07.873050+0000","last_fullsized":"2024-03-30T00:42:07.873050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.889268+0000","last_change":"2024-03-30T00:42:07.889268+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.889268+0000","last_undegraded":"2024-03-30T00:42:07.889268+0000","last_fullsized":"2024-03-30T00:42:07.889268+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872559+0000","last_change":"2024-03-30T00:42:07.872559+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872559+0000","last_undegraded":"2024-03-30T00:42:07.872559+0000","last_fullsized":"2024-03-30T00:42:07.872559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874398+0000","last_change":"2024-03-30T00:42:07.874398+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874398+0000","last_undegraded":"2024-03-30T00:42:07.874398+0000","last_fullsized":"2024-03-30T00:42:07.874398+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873733+0000","last_change":"2024-03-30T00:42:07.873733+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873733+0000","last_undegraded":"2024-03-30T00:42:07.873733+0000","last_fullsized":"2024-03-30T00:42:07.873733+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:22.099119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881871+0000","last_change":"2024-03-30T00:42:07.881871+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881871+0000","last_undegraded":"2024-03-30T00:42:07.881871+0000","last_fullsized":"2024-03-30T00:42:07.881871+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:09.745566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877328+0000","last_change":"2024-03-30T00:42:07.877328+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877328+0000","last_undegraded":"2024-03-30T00:42:07.877328+0000","last_fullsized":"2024-03-30T00:42:07.877328+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:01.749453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878241+0000","last_change":"2024-03-30T00:42:07.878241+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878241+0000","last_undegraded":"2024-03-30T00:42:07.878241+0000","last_fullsized":"2024-03-30T00:42:07.878241+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874651+0000","last_change":"2024-03-30T00:42:07.874651+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874651+0000","last_undegraded":"2024-03-30T00:42:07.874651+0000","last_fullsized":"2024-03-30T00:42:07.874651+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879446+0000","last_change":"2024-03-30T00:42:07.879446+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879446+0000","last_undegraded":"2024-03-30T00:42:07.879446+0000","last_fullsized":"2024-03-30T00:42:07.879446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:04.810334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878717+0000","last_change":"2024-03-30T00:42:07.878717+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878717+0000","last_undegraded":"2024-03-30T00:42:07.878717+0000","last_fullsized":"2024-03-30T00:42:07.878717+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874011+0000","last_change":"2024-03-30T00:42:07.874011+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874011+0000","last_undegraded":"2024-03-30T00:42:07.874011+0000","last_fullsized":"2024-03-30T00:42:07.874011+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:03.287560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.882341+0000","last_change":"2024-03-30T00:42:07.882341+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.882341+0000","last_undegraded":"2024-03-30T00:42:07.882341+0000","last_fullsized":"2024-03-30T00:42:07.882341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:54:01.171986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.871624+0000","last_change":"2024-03-30T00:42:07.871624+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.871624+0000","last_undegraded":"2024-03-30T00:42:07.871624+0000","last_fullsized":"2024-03-30T00:42:07.871624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:56.870783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872827+0000","last_change":"2024-03-30T00:42:07.872827+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872827+0000","last_undegraded":"2024-03-30T00:42:07.872827+0000","last_fullsized":"2024-03-30T00:42:07.872827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:24.904667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888103+0000","last_change":"2024-03-30T00:42:07.888103+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888103+0000","last_undegraded":"2024-03-30T00:42:07.888103+0000","last_fullsized":"2024-03-30T00:42:07.888103+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:41:31.279358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872622+0000","last_change":"2024-03-30T00:42:07.872622+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872622+0000","last_undegraded":"2024-03-30T00:42:07.872622+0000","last_fullsized":"2024-03-30T00:42:07.872622+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888796+0000","last_change":"2024-03-30T00:42:07.888796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888796+0000","last_undegraded":"2024-03-30T00:42:07.888796+0000","last_fullsized":"2024-03-30T00:42:07.888796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877961+0000","last_change":"2024-03-30T00:42:07.877961+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877961+0000","last_undegraded":"2024-03-30T00:42:07.877961+0000","last_fullsized":"2024-03-30T00:42:07.877961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:10:20.006391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877522+0000","last_change":"2024-03-30T00:42:07.877522+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877522+0000","last_undegraded":"2024-03-30T00:42:07.877522+0000","last_fullsized":"2024-03-30T00:42:07.877522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:19.899232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877834+0000","last_change":"2024-03-30T00:42:07.877834+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877834+0000","last_undegraded":"2024-03-30T00:42:07.877834+0000","last_fullsized":"2024-03-30T00:42:07.877834+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878875+0000","last_change":"2024-03-30T00:42:07.878875+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878875+0000","last_undegraded":"2024-03-30T00:42:07.878875+0000","last_fullsized":"2024-03-30T00:42:07.878875+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:26.322503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881115+0000","last_change":"2024-03-30T00:42:07.881115+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881115+0000","last_undegraded":"2024-03-30T00:42:07.881115+0000","last_fullsized":"2024-03-30T00:42:07.881115+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:34:13.403177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.887228+0000","last_change":"2024-03-30T00:42:07.887228+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.887228+0000","last_undegraded":"2024-03-30T00:42:07.887228+0000","last_fullsized":"2024-03-30T00:42:07.887228+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880458+0000","last_change":"2024-03-30T00:42:07.880458+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880458+0000","last_undegraded":"2024-03-30T00:42:07.880458+0000","last_fullsized":"2024-03-30T00:42:07.880458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872899+0000","last_change":"2024-03-30T00:42:07.872899+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872899+0000","last_undegraded":"2024-03-30T00:42:07.872899+0000","last_fullsized":"2024-03-30T00:42:07.872899+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:22.536352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877458+0000","last_change":"2024-03-30T00:42:07.877458+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877458+0000","last_undegraded":"2024-03-30T00:42:07.877458+0000","last_fullsized":"2024-03-30T00:42:07.877458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:20.469591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.893998+0000","last_change":"2024-03-30T00:42:07.893998+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.893998+0000","last_undegraded":"2024-03-30T00:42:07.893998+0000","last_fullsized":"2024-03-30T00:42:07.893998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872039+0000","last_change":"2024-03-30T00:42:07.872039+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872039+0000","last_undegraded":"2024-03-30T00:42:07.872039+0000","last_fullsized":"2024-03-30T00:42:07.872039+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881306+0000","last_change":"2024-03-30T00:42:07.881306+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881306+0000","last_undegraded":"2024-03-30T00:42:07.881306+0000","last_fullsized":"2024-03-30T00:42:07.881306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:49.025731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877550+0000","last_change":"2024-03-30T00:42:07.877550+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877550+0000","last_undegraded":"2024-03-30T00:42:07.877550+0000","last_fullsized":"2024-03-30T00:42:07.877550+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.871624+0000","last_change":"2024-03-30T00:42:07.871624+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.871624+0000","last_undegraded":"2024-03-30T00:42:07.871624+0000","last_fullsized":"2024-03-30T00:42:07.871624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:17:22.441495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890610+0000","last_change":"2024-03-30T00:42:07.887914+0000","last_active":"2024-03-30T00:42:07.890610+0000","last_peered":"2024-03-30T00:42:07.890610+0000","last_clean":"2024-03-30T00:42:07.890610+0000","last_became_active":"2024-03-30T00:42:07.887626+0000","last_became_peered":"2024-03-30T00:42:07.887626+0000","last_unstale":"2024-03-30T00:42:07.890610+0000","last_undegraded":"2024-03-30T00:42:07.890610+0000","last_fullsized":"2024-03-30T00:42:07.890610+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:25.319805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876251+0000","last_change":"2024-03-30T00:42:07.876251+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876251+0000","last_undegraded":"2024-03-30T00:42:07.876251+0000","last_fullsized":"2024-03-30T00:42:07.876251+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:29.266461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890823+0000","last_change":"2024-03-30T00:42:07.890823+0000","last_active":"2024-03-30T00:42:07.890823+0000","last_peered":"2024-03-30T00:42:07.890823+0000","last_clean":"2024-03-30T00:42:07.890823+0000","last_became_active":"2024-03-30T00:42:07.888331+0000","last_became_peered":"2024-03-30T00:42:07.888331+0000","last_unstale":"2024-03-30T00:42:07.890823+0000","last_undegraded":"2024-03-30T00:42:07.890823+0000","last_fullsized":"2024-03-30T00:42:07.890823+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:48:23.842037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878821+0000","last_change":"2024-03-30T00:42:07.878821+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878821+0000","last_undegraded":"2024-03-30T00:42:07.878821+0000","last_fullsized":"2024-03-30T00:42:07.878821+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:03.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.884824+0000","last_change":"2024-03-30T00:42:07.884824+0000","last_active":"2024-03-30T00:42:07.884824+0000","last_peered":"2024-03-30T00:42:07.884824+0000","last_clean":"2024-03-30T00:42:07.884824+0000","last_became_active":"2024-03-30T00:42:07.884503+0000","last_became_peered":"2024-03-30T00:42:07.884503+0000","last_unstale":"2024-03-30T00:42:07.884824+0000","last_undegraded":"2024-03-30T00:42:07.884824+0000","last_fullsized":"2024-03-30T00:42:07.884824+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:18:21.768136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877433+0000","last_change":"2024-03-30T00:42:07.877433+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877433+0000","last_undegraded":"2024-03-30T00:42:07.877433+0000","last_fullsized":"2024-03-30T00:42:07.877433+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.892398+0000","last_change":"2024-03-30T00:42:07.892398+0000","last_active":"2024-03-30T00:42:07.892398+0000","last_peered":"2024-03-30T00:42:07.892398+0000","last_clean":"2024-03-30T00:42:07.892398+0000","last_became_active":"2024-03-30T00:42:07.892097+0000","last_became_peered":"2024-03-30T00:42:07.892097+0000","last_unstale":"2024-03-30T00:42:07.892398+0000","last_undegraded":"2024-03-30T00:42:07.892398+0000","last_fullsized":"2024-03-30T00:42:07.892398+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:30:49.532832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874348+0000","last_change":"2024-03-30T00:42:07.874348+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874348+0000","last_undegraded":"2024-03-30T00:42:07.874348+0000","last_fullsized":"2024-03-30T00:42:07.874348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:59.202589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.897983+0000","last_change":"2024-03-30T00:42:07.897983+0000","last_active":"2024-03-30T00:42:07.897983+0000","last_peered":"2024-03-30T00:42:07.897983+0000","last_clean":"2024-03-30T00:42:07.897983+0000","last_became_active":"2024-03-30T00:42:07.897719+0000","last_became_peered":"2024-03-30T00:42:07.897719+0000","last_unstale":"2024-03-30T00:42:07.897983+0000","last_undegraded":"2024-03-30T00:42:07.897983+0000","last_fullsized":"2024-03-30T00:42:07.897983+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:55.868085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873753+0000","last_change":"2024-03-30T00:42:07.873753+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873753+0000","last_undegraded":"2024-03-30T00:42:07.873753+0000","last_fullsized":"2024-03-30T00:42:07.873753+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887534+0000","last_change":"2024-03-30T00:42:07.884431+0000","last_active":"2024-03-30T00:42:07.887534+0000","last_peered":"2024-03-30T00:42:07.887534+0000","last_clean":"2024-03-30T00:42:07.887534+0000","last_became_active":"2024-03-30T00:42:07.884172+0000","last_became_peered":"2024-03-30T00:42:07.884172+0000","last_unstale":"2024-03-30T00:42:07.887534+0000","last_undegraded":"2024-03-30T00:42:07.887534+0000","last_fullsized":"2024-03-30T00:42:07.887534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:24.488599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879706+0000","last_change":"2024-03-30T00:42:07.879706+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879706+0000","last_undegraded":"2024-03-30T00:42:07.879706+0000","last_fullsized":"2024-03-30T00:42:07.879706+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887038+0000","last_change":"2024-03-30T00:42:07.884001+0000","last_active":"2024-03-30T00:42:07.887038+0000","last_peered":"2024-03-30T00:42:07.887038+0000","last_clean":"2024-03-30T00:42:07.887038+0000","last_became_active":"2024-03-30T00:42:07.883690+0000","last_became_peered":"2024-03-30T00:42:07.883690+0000","last_unstale":"2024-03-30T00:42:07.887038+0000","last_undegraded":"2024-03-30T00:42:07.887038+0000","last_fullsized":"2024-03-30T00:42:07.887038+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:26.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873193+0000","last_change":"2024-03-30T00:42:07.873193+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873193+0000","last_undegraded":"2024-03-30T00:42:07.873193+0000","last_fullsized":"2024-03-30T00:42:07.873193+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:22.099119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893642+0000","last_change":"2024-03-30T00:42:07.887955+0000","last_active":"2024-03-30T00:42:07.893642+0000","last_peered":"2024-03-30T00:42:07.893642+0000","last_clean":"2024-03-30T00:42:07.893642+0000","last_became_active":"2024-03-30T00:42:07.887689+0000","last_became_peered":"2024-03-30T00:42:07.887689+0000","last_unstale":"2024-03-30T00:42:07.893642+0000","last_undegraded":"2024-03-30T00:42:07.893642+0000","last_fullsized":"2024-03-30T00:42:07.893642+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:17:21.438797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873054+0000","last_change":"2024-03-30T00:42:07.873054+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873054+0000","last_undegraded":"2024-03-30T00:42:07.873054+0000","last_fullsized":"2024-03-30T00:42:07.873054+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887686+0000","last_change":"2024-03-30T00:42:07.884866+0000","last_active":"2024-03-30T00:42:07.887686+0000","last_peered":"2024-03-30T00:42:07.887686+0000","last_clean":"2024-03-30T00:42:07.887686+0000","last_became_active":"2024-03-30T00:42:07.884550+0000","last_became_peered":"2024-03-30T00:42:07.884550+0000","last_unstale":"2024-03-30T00:42:07.887686+0000","last_undegraded":"2024-03-30T00:42:07.887686+0000","last_fullsized":"2024-03-30T00:42:07.887686+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:10:19.003693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876922+0000","last_change":"2024-03-30T00:42:07.876922+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876922+0000","last_undegraded":"2024-03-30T00:42:07.876922+0000","last_fullsized":"2024-03-30T00:42:07.876922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.874579+0000","last_change":"2024-03-30T00:42:07.874579+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874579+0000","last_undegraded":"2024-03-30T00:42:07.874579+0000","last_fullsized":"2024-03-30T00:42:07.874579+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:17.278798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881886+0000","last_change":"2024-03-30T00:42:07.881886+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881886+0000","last_undegraded":"2024-03-30T00:42:07.881886+0000","last_fullsized":"2024-03-30T00:42:07.881886+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:07.667668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891360+0000","last_change":"2024-03-30T00:42:07.889759+0000","last_active":"2024-03-30T00:42:07.891360+0000","last_peered":"2024-03-30T00:42:07.891360+0000","last_clean":"2024-03-30T00:42:07.891360+0000","last_became_active":"2024-03-30T00:42:07.888856+0000","last_became_peered":"2024-03-30T00:42:07.888856+0000","last_unstale":"2024-03-30T00:42:07.891360+0000","last_undegraded":"2024-03-30T00:42:07.891360+0000","last_fullsized":"2024-03-30T00:42:07.891360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:00:01.121929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873860+0000","last_change":"2024-03-30T00:42:07.873860+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873860+0000","last_undegraded":"2024-03-30T00:42:07.873860+0000","last_fullsized":"2024-03-30T00:42:07.873860+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.878493+0000","last_change":"2024-03-30T00:42:07.878493+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878493+0000","last_undegraded":"2024-03-30T00:42:07.878493+0000","last_fullsized":"2024-03-30T00:42:07.878493+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:48.023033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881591+0000","last_change":"2024-03-30T00:42:07.881591+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881591+0000","last_undegraded":"2024-03-30T00:42:07.881591+0000","last_fullsized":"2024-03-30T00:42:07.881591+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.896028+0000","last_change":"2024-03-30T00:42:07.896028+0000","last_active":"2024-03-30T00:42:07.896028+0000","last_peered":"2024-03-30T00:42:07.896028+0000","last_clean":"2024-03-30T00:42:07.896028+0000","last_became_active":"2024-03-30T00:42:07.895683+0000","last_became_peered":"2024-03-30T00:42:07.895683+0000","last_unstale":"2024-03-30T00:42:07.896028+0000","last_undegraded":"2024-03-30T00:42:07.896028+0000","last_fullsized":"2024-03-30T00:42:07.896028+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:37.405718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888318+0000","last_change":"2024-03-30T00:42:07.888318+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888318+0000","last_undegraded":"2024-03-30T00:42:07.888318+0000","last_fullsized":"2024-03-30T00:42:07.888318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.877016+0000","last_change":"2024-03-30T00:42:07.877016+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877016+0000","last_undegraded":"2024-03-30T00:42:07.877016+0000","last_fullsized":"2024-03-30T00:42:07.877016+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:58:41.813204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880083+0000","last_change":"2024-03-30T00:42:07.880083+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880083+0000","last_undegraded":"2024-03-30T00:42:07.880083+0000","last_fullsized":"2024-03-30T00:42:07.880083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:44.118291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.889095+0000","last_change":"2024-03-30T00:42:07.886045+0000","last_active":"2024-03-30T00:42:07.889095+0000","last_peered":"2024-03-30T00:42:07.889095+0000","last_clean":"2024-03-30T00:42:07.889095+0000","last_became_active":"2024-03-30T00:42:07.885765+0000","last_became_peered":"2024-03-30T00:42:07.885765+0000","last_unstale":"2024-03-30T00:42:07.889095+0000","last_undegraded":"2024-03-30T00:42:07.889095+0000","last_fullsized":"2024-03-30T00:42:07.889095+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:21.533654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888542+0000","last_change":"2024-03-30T00:42:07.888542+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888542+0000","last_undegraded":"2024-03-30T00:42:07.888542+0000","last_fullsized":"2024-03-30T00:42:07.888542+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:01.442897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.876507+0000","last_change":"2024-03-30T00:42:07.876507+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876507+0000","last_undegraded":"2024-03-30T00:42:07.876507+0000","last_fullsized":"2024-03-30T00:42:07.876507+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:16:59.791004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876783+0000","last_change":"2024-03-30T00:42:07.876783+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876783+0000","last_undegraded":"2024-03-30T00:42:07.876783+0000","last_fullsized":"2024-03-30T00:42:07.876783+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.898205+0000","last_change":"2024-03-30T00:42:07.898205+0000","last_active":"2024-03-30T00:42:07.898205+0000","last_peered":"2024-03-30T00:42:07.898205+0000","last_clean":"2024-03-30T00:42:07.898205+0000","last_became_active":"2024-03-30T00:42:07.897873+0000","last_became_peered":"2024-03-30T00:42:07.897873+0000","last_unstale":"2024-03-30T00:42:07.898205+0000","last_undegraded":"2024-03-30T00:42:07.898205+0000","last_fullsized":"2024-03-30T00:42:07.898205+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:28.263763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873297+0000","last_change":"2024-03-30T00:42:07.873297+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873297+0000","last_undegraded":"2024-03-30T00:42:07.873297+0000","last_fullsized":"2024-03-30T00:42:07.873297+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890926+0000","last_change":"2024-03-30T00:42:07.890926+0000","last_active":"2024-03-30T00:42:07.890926+0000","last_peered":"2024-03-30T00:42:07.890926+0000","last_clean":"2024-03-30T00:42:07.890926+0000","last_became_active":"2024-03-30T00:42:07.890676+0000","last_became_peered":"2024-03-30T00:42:07.890676+0000","last_unstale":"2024-03-30T00:42:07.890926+0000","last_undegraded":"2024-03-30T00:42:07.890926+0000","last_fullsized":"2024-03-30T00:42:07.890926+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:24:47.097376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873770+0000","last_change":"2024-03-30T00:42:07.873770+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873770+0000","last_undegraded":"2024-03-30T00:42:07.873770+0000","last_fullsized":"2024-03-30T00:42:07.873770+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890669+0000","last_change":"2024-03-30T00:42:07.888916+0000","last_active":"2024-03-30T00:42:07.890669+0000","last_peered":"2024-03-30T00:42:07.890669+0000","last_clean":"2024-03-30T00:42:07.890669+0000","last_became_active":"2024-03-30T00:42:07.888629+0000","last_became_peered":"2024-03-30T00:42:07.888629+0000","last_unstale":"2024-03-30T00:42:07.890669+0000","last_undegraded":"2024-03-30T00:42:07.890669+0000","last_fullsized":"2024-03-30T00:42:07.890669+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:03.807636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872778+0000","last_change":"2024-03-30T00:42:07.872778+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872778+0000","last_undegraded":"2024-03-30T00:42:07.872778+0000","last_fullsized":"2024-03-30T00:42:07.872778+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890933+0000","last_change":"2024-03-30T00:42:07.889736+0000","last_active":"2024-03-30T00:42:07.890933+0000","last_peered":"2024-03-30T00:42:07.890933+0000","last_clean":"2024-03-30T00:42:07.890933+0000","last_became_active":"2024-03-30T00:42:07.889425+0000","last_became_peered":"2024-03-30T00:42:07.889425+0000","last_unstale":"2024-03-30T00:42:07.890933+0000","last_undegraded":"2024-03-30T00:42:07.890933+0000","last_fullsized":"2024-03-30T00:42:07.890933+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:41.441678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879997+0000","last_change":"2024-03-30T00:42:07.879997+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879997+0000","last_undegraded":"2024-03-30T00:42:07.879997+0000","last_fullsized":"2024-03-30T00:42:07.879997+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:42.969334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.884117+0000","last_change":"2024-03-30T00:42:07.884117+0000","last_active":"2024-03-30T00:42:07.884117+0000","last_peered":"2024-03-30T00:42:07.884117+0000","last_clean":"2024-03-30T00:42:07.884117+0000","last_became_active":"2024-03-30T00:42:07.883887+0000","last_became_peered":"2024-03-30T00:42:07.883887+0000","last_unstale":"2024-03-30T00:42:07.884117+0000","last_undegraded":"2024-03-30T00:42:07.884117+0000","last_fullsized":"2024-03-30T00:42:07.884117+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:18.896534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880105+0000","last_change":"2024-03-30T00:42:07.880105+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880105+0000","last_undegraded":"2024-03-30T00:42:07.880105+0000","last_fullsized":"2024-03-30T00:42:07.880105+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:00:02.124627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893214+0000","last_change":"2024-03-30T00:42:07.893214+0000","last_active":"2024-03-30T00:42:07.893214+0000","last_peered":"2024-03-30T00:42:07.893214+0000","last_clean":"2024-03-30T00:42:07.893214+0000","last_became_active":"2024-03-30T00:42:07.892814+0000","last_became_peered":"2024-03-30T00:42:07.892814+0000","last_unstale":"2024-03-30T00:42:07.893214+0000","last_undegraded":"2024-03-30T00:42:07.893214+0000","last_fullsized":"2024-03-30T00:42:07.893214+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:52.010424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.889733+0000","last_change":"2024-03-30T00:42:07.889733+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.889733+0000","last_undegraded":"2024-03-30T00:42:07.889733+0000","last_fullsized":"2024-03-30T00:42:07.889733+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.896209+0000","last_change":"2024-03-30T00:42:07.896209+0000","last_active":"2024-03-30T00:42:07.896209+0000","last_peered":"2024-03-30T00:42:07.896209+0000","last_clean":"2024-03-30T00:42:07.896209+0000","last_became_active":"2024-03-30T00:42:07.895871+0000","last_became_peered":"2024-03-30T00:42:07.895871+0000","last_unstale":"2024-03-30T00:42:07.896209+0000","last_undegraded":"2024-03-30T00:42:07.896209+0000","last_fullsized":"2024-03-30T00:42:07.896209+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:24.948608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872629+0000","last_change":"2024-03-30T00:42:07.872629+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872629+0000","last_undegraded":"2024-03-30T00:42:07.872629+0000","last_fullsized":"2024-03-30T00:42:07.872629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891909+0000","last_change":"2024-03-30T00:42:07.891909+0000","last_active":"2024-03-30T00:42:07.891909+0000","last_peered":"2024-03-30T00:42:07.891909+0000","last_clean":"2024-03-30T00:42:07.891909+0000","last_became_active":"2024-03-30T00:42:07.891660+0000","last_became_peered":"2024-03-30T00:42:07.891660+0000","last_unstale":"2024-03-30T00:42:07.891909+0000","last_undegraded":"2024-03-30T00:42:07.891909+0000","last_fullsized":"2024-03-30T00:42:07.891909+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:32.488321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876796+0000","last_change":"2024-03-30T00:42:07.876796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876796+0000","last_undegraded":"2024-03-30T00:42:07.876796+0000","last_fullsized":"2024-03-30T00:42:07.876796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:40.974026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.878452+0000","last_change":"2024-03-30T00:42:07.878452+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878452+0000","last_undegraded":"2024-03-30T00:42:07.878452+0000","last_fullsized":"2024-03-30T00:42:07.878452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:30.469686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872796+0000","last_change":"2024-03-30T00:42:07.872796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872796+0000","last_undegraded":"2024-03-30T00:42:07.872796+0000","last_fullsized":"2024-03-30T00:42:07.872796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:30:50.535530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.876204+0000","last_change":"2024-03-30T00:42:07.876204+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876204+0000","last_undegraded":"2024-03-30T00:42:07.876204+0000","last_fullsized":"2024-03-30T00:42:07.876204+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:02:33.633690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876968+0000","last_change":"2024-03-30T00:42:07.876968+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876968+0000","last_undegraded":"2024-03-30T00:42:07.876968+0000","last_fullsized":"2024-03-30T00:42:07.876968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:58.102992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.899036+0000","last_change":"2024-03-30T00:42:07.895012+0000","last_active":"2024-03-30T00:42:07.899036+0000","last_peered":"2024-03-30T00:42:07.899036+0000","last_clean":"2024-03-30T00:42:07.899036+0000","last_became_active":"2024-03-30T00:42:07.894725+0000","last_became_peered":"2024-03-30T00:42:07.894725+0000","last_unstale":"2024-03-30T00:42:07.899036+0000","last_undegraded":"2024-03-30T00:42:07.899036+0000","last_fullsized":"2024-03-30T00:42:07.899036+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:41:59.532828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.892109+0000","last_change":"2024-03-30T00:42:07.892109+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.892109+0000","last_undegraded":"2024-03-30T00:42:07.892109+0000","last_fullsized":"2024-03-30T00:42:07.892109+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893706+0000","last_change":"2024-03-30T00:42:07.893706+0000","last_active":"2024-03-30T00:42:07.893706+0000","last_peered":"2024-03-30T00:42:07.893706+0000","last_clean":"2024-03-30T00:42:07.893706+0000","last_became_active":"2024-03-30T00:42:07.893465+0000","last_became_peered":"2024-03-30T00:42:07.893465+0000","last_unstale":"2024-03-30T00:42:07.893706+0000","last_undegraded":"2024-03-30T00:42:07.893706+0000","last_fullsized":"2024-03-30T00:42:07.893706+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:41.966636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878795+0000","last_change":"2024-03-30T00:42:07.878795+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878795+0000","last_undegraded":"2024-03-30T00:42:07.878795+0000","last_fullsized":"2024-03-30T00:42:07.878795+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893058+0000","last_change":"2024-03-30T00:42:07.893058+0000","last_active":"2024-03-30T00:42:07.893058+0000","last_peered":"2024-03-30T00:42:07.893058+0000","last_clean":"2024-03-30T00:42:07.893058+0000","last_became_active":"2024-03-30T00:42:07.892453+0000","last_became_peered":"2024-03-30T00:42:07.892453+0000","last_unstale":"2024-03-30T00:42:07.893058+0000","last_undegraded":"2024-03-30T00:42:07.893058+0000","last_fullsized":"2024-03-30T00:42:07.893058+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:27.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.890675+0000","last_change":"2024-03-30T00:42:07.890675+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.890675+0000","last_undegraded":"2024-03-30T00:42:07.890675+0000","last_fullsized":"2024-03-30T00:42:07.890675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891397+0000","last_change":"2024-03-30T00:42:07.889478+0000","last_active":"2024-03-30T00:42:07.891397+0000","last_peered":"2024-03-30T00:42:07.891397+0000","last_clean":"2024-03-30T00:42:07.891397+0000","last_became_active":"2024-03-30T00:42:07.889247+0000","last_became_peered":"2024-03-30T00:42:07.889247+0000","last_unstale":"2024-03-30T00:42:07.891397+0000","last_undegraded":"2024-03-30T00:42:07.891397+0000","last_fullsized":"2024-03-30T00:42:07.891397+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:01.336339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877866+0000","last_change":"2024-03-30T00:42:07.877866+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877866+0000","last_undegraded":"2024-03-30T00:42:07.877866+0000","last_fullsized":"2024-03-30T00:42:07.877866+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891637+0000","last_change":"2024-03-30T00:42:07.891209+0000","last_active":"2024-03-30T00:42:07.891637+0000","last_peered":"2024-03-30T00:42:07.891637+0000","last_clean":"2024-03-30T00:42:07.891637+0000","last_became_active":"2024-03-30T00:42:07.890769+0000","last_became_peered":"2024-03-30T00:42:07.890769+0000","last_unstale":"2024-03-30T00:42:07.891637+0000","last_undegraded":"2024-03-30T00:42:07.891637+0000","last_fullsized":"2024-03-30T00:42:07.891637+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:21.096421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879604+0000","last_change":"2024-03-30T00:42:07.879604+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879604+0000","last_undegraded":"2024-03-30T00:42:07.879604+0000","last_fullsized":"2024-03-30T00:42:07.879604+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:02:34.636388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.899549+0000","last_change":"2024-03-30T00:42:07.899549+0000","last_active":"2024-03-30T00:42:07.899549+0000","last_peered":"2024-03-30T00:42:07.899549+0000","last_clean":"2024-03-30T00:42:07.899549+0000","last_became_active":"2024-03-30T00:42:07.896786+0000","last_became_peered":"2024-03-30T00:42:07.896786+0000","last_unstale":"2024-03-30T00:42:07.899549+0000","last_undegraded":"2024-03-30T00:42:07.899549+0000","last_fullsized":"2024-03-30T00:42:07.899549+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:24.488599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-30T00:42:06.865006+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:42:06.865006+0000","last_peered":"2024-03-30T00:42:06.865006+0000","last_clean":"2024-03-30T00:42:06.865006+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:42:06.865006+0000","last_undegraded":"2024-03-30T00:42:06.865006+0000","last_fullsized":"2024-03-30T00:42:06.865006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1200128,"data_stored":854159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.77100000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.62},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.93300000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.626},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.51600000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.747}]},{"osd":2,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78100000000000003,"15min":0.78900000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.194,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.79400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.026,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0840000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82899999999999996},"min":{"1min":0.65100000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0429999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.873}]},{"osd":4,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63500000000000001,"15min":0.629},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.996,"5min":1.026,"15min":1.026},"last":0.97499999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.45400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.89200000000000002,"15min":0.93799999999999994},"min":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90100000000000002,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78300000000000003,"5min":1.47,"15min":1.47},"last":0.70299999999999996}]},{"osd":1,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.90100000000000002,"15min":0.95299999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76900000000000002,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.97299999999999998,"15min":1.0329999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.82399999999999995,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.69299999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.96999999999999997,"15min":1.018},"min":{"1min":0.48699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.82299999999999995,"5min":4.375,"15min":4.375},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.97199999999999998,"15min":1.024},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":4.2960000000000003,"15min":4.2960000000000003},"last":0.79000000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.98799999999999999,"15min":1.038},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97299999999999998,"5min":4.3399999999999999,"15min":4.3399999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.97499999999999998,"15min":1.034},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.73299999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96999999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.42399999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.095},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.78700000000000003,"15min":0.81000000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95099999999999996,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.86299999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.751,"15min":0.76600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77400000000000002,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.72299999999999998,"15min":0.73899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83599999999999997,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90600000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.70899999999999996,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79700000000000004,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.81000000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.79200000000000004,"15min":0.81200000000000006},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98499999999999999,"5min":1.7,"15min":1.7},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.80200000000000005,"5min":1.181,"15min":1.181},"last":0.83499999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63400000000000001,"15min":0.64400000000000002},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91500000000000004,"5min":1.615,"15min":1.615},"last":0.89100000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88100000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.877},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0700000000000001,"5min":1.569,"15min":1.569},"last":0.85199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76000000000000001,"15min":0.76300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89300000000000002,"5min":1.006,"15min":1.006},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.83899999999999997,"15min":0.84899999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.032,"5min":1.383,"15min":1.383},"last":0.92500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77900000000000003,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.44600000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51700000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71499999999999997,"15min":0.72299999999999998},"min":{"1min":0.50700000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.872,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54200000000000004}]},{"osd":2,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.91000000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.755,"15min":0.76300000000000001},"min":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96899999999999997,"5min":1.228,"15min":1.228},"last":0.58499999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.50700000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94899999999999995,"5min":1.27,"15min":1.27},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.751,"5min":0.78300000000000003,"15min":0.78900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91400000000000003,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84899999999999998,"5min":1.329,"15min":1.329},"last":0.438},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93500000000000005,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98899999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95799999999999996,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.497}]},{"osd":7,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.749,"15min":0.76300000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.069,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.747}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.56899999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.53200000000000003,"15min":0.55300000000000005},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.378},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59799999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.623,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.69299999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40500000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.76100000000000001,"15min":0.77600000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98299999999999998,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.872},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.77100000000000002,"15min":0.78500000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0069999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.90200000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.78200000000000003,"15min":0.79300000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.95199999999999996,"5min":1.125,"15min":1.125},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80600000000000005,"15min":0.82099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.024,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.81200000000000006,"15min":0.82699999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0389999999999999,"5min":1.226,"15min":1.226},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.79700000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0109999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.94899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.85099999999999998,"15min":0.86899999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.077,"5min":1.202,"15min":1.202},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.88400000000000001,"15min":0.90800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0740000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.91800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.313,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.627,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":4.0999999999999996},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.51100000000000001,"15min":0.52700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.56200000000000006,"5min":0.747,"15min":0.747},"last":1.9239999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.58199999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":3.1139999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53200000000000003,"15min":0.54400000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.63,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":2.4159999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.495,"15min":0.503},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.53400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76200000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":1.907}]},{"osd":4,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80300000000000005,"15min":0.81499999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.042,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.9470000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.127,"5min":1.528,"15min":1.528},"last":1.972}]},{"osd":5,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83099999999999996,"15min":0.84499999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1240000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":2.4569999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.80600000000000005,"15min":0.82299999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0580000000000001,"5min":1.131,"15min":1.131},"last":2.8039999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.81000000000000005,"15min":0.82099999999999995},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.081,"5min":1.151,"15min":1.151},"last":2.8849999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.179,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.867}]},{"osd":7,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.85999999999999999,"15min":0.88200000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.01,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":2.3690000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0700000000000001,"5min":1.228,"15min":1.228},"last":2.851}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61299999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.46000000000000002,"15min":0.46899999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55700000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.498}]},{"osd":3,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66200000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.626,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92000000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.83199999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77600000000000002,"15min":0.78000000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93400000000000005,"5min":1.097,"15min":1.097},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.90100000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78900000000000003,"15min":0.79500000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95199999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92800000000000005,"5min":0.996,"15min":0.996},"last":0.93400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1200128,"data_stored":854159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.29299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58499999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.61299999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66100000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.623},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":1.006}]},{"osd":3,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71499999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.96899999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79100000000000004,"15min":0.79400000000000004},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.008,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.234},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79900000000000004,"15min":0.80000000000000004},"min":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1279999999999999,"5min":1.153,"15min":1.153},"last":1.252}]},{"osd":5,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.621,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0489999999999999,"5min":1.22,"15min":1.22},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.92300000000000004,"5min":1.462,"15min":1.462},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78900000000000003,"15min":0.79800000000000004},"min":{"1min":0.58299999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.091,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.56299999999999994,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1,"5min":1.085,"15min":1.085},"last":1.2}]},{"osd":7,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.91500000000000004,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.351},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96899999999999997,"5min":1.556,"15min":1.556},"last":1.085}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:42:13.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-03-30T00:42:13.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.874156+0000', 'last_change': '2024-03-30T00:42:07.874156+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.874156+0000', 'last_undegraded': '2024-03-30T00:42:07.874156+0000', 'last_fullsized': '2024-03-30T00:42:07.874156+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:42:13.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2024-03-30T00:42:13.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.873050+0000', 'last_change': '2024-03-30T00:42:07.873050+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.873050+0000', 'last_undegraded': '2024-03-30T00:42:07.873050+0000', 'last_fullsized': '2024-03-30T00:42:07.873050+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:42:13.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.889268+0000', 'last_change': '2024-03-30T00:42:07.889268+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.889268+0000', 'last_undegraded': '2024-03-30T00:42:07.889268+0000', 'last_fullsized': '2024-03-30T00:42:07.889268+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.872559+0000', 'last_change': '2024-03-30T00:42:07.872559+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.872559+0000', 'last_undegraded': '2024-03-30T00:42:07.872559+0000', 'last_fullsized': '2024-03-30T00:42:07.872559+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.874398+0000', 'last_change': '2024-03-30T00:42:07.874398+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.874398+0000', 'last_undegraded': '2024-03-30T00:42:07.874398+0000', 'last_fullsized': '2024-03-30T00:42:07.874398+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.873733+0000', 'last_change': '2024-03-30T00:42:07.873733+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.873733+0000', 'last_undegraded': '2024-03-30T00:42:07.873733+0000', 'last_fullsized': '2024-03-30T00:42:07.873733+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:14:22.099119+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.881871+0000', 'last_change': '2024-03-30T00:42:07.881871+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.881871+0000', 'last_undegraded': '2024-03-30T00:42:07.881871+0000', 'last_fullsized': '2024-03-30T00:42:07.881871+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T02:00:09.745566+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.877328+0000', 'last_change': '2024-03-30T00:42:07.877328+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.877328+0000', 'last_undegraded': '2024-03-30T00:42:07.877328+0000', 'last_fullsized': '2024-03-30T00:42:07.877328+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:20:01.749453+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-03-30T00:42:13.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.878241+0000', 'last_change': '2024-03-30T00:42:07.878241+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.878241+0000', 'last_undegraded': '2024-03-30T00:42:07.878241+0000', 'last_fullsized': '2024-03-30T00:42:07.878241+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.874651+0000', 'last_change': '2024-03-30T00:42:07.874651+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.874651+0000', 'last_undegraded': '2024-03-30T00:42:07.874651+0000', 'last_fullsized': '2024-03-30T00:42:07.874651+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.879446+0000', 'last_change': '2024-03-30T00:42:07.879446+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.879446+0000', 'last_undegraded': '2024-03-30T00:42:07.879446+0000', 'last_fullsized': '2024-03-30T00:42:07.879446+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:12:04.810334+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.878717+0000', 'last_change': '2024-03-30T00:42:07.878717+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.878717+0000', 'last_undegraded': '2024-03-30T00:42:07.878717+0000', 'last_fullsized': '2024-03-30T00:42:07.878717+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.874011+0000', 'last_change': '2024-03-30T00:42:07.874011+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.874011+0000', 'last_undegraded': '2024-03-30T00:42:07.874011+0000', 'last_fullsized': '2024-03-30T00:42:07.874011+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:06:03.287560+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.882341+0000', 'last_change': '2024-03-30T00:42:07.882341+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.882341+0000', 'last_undegraded': '2024-03-30T00:42:07.882341+0000', 'last_fullsized': '2024-03-30T00:42:07.882341+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:54:01.171986+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2024-03-30T00:42:13.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.871624+0000', 'last_change': '2024-03-30T00:42:07.871624+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.871624+0000', 'last_undegraded': '2024-03-30T00:42:07.871624+0000', 'last_fullsized': '2024-03-30T00:42:07.871624+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T08:24:56.870783+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.872827+0000', 'last_change': '2024-03-30T00:42:07.872827+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.872827+0000', 'last_undegraded': '2024-03-30T00:42:07.872827+0000', 'last_fullsized': '2024-03-30T00:42:07.872827+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T05:58:24.904667+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.888103+0000', 'last_change': '2024-03-30T00:42:07.888103+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.888103+0000', 'last_undegraded': '2024-03-30T00:42:07.888103+0000', 'last_fullsized': '2024-03-30T00:42:07.888103+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:41:31.279358+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.872622+0000', 'last_change': '2024-03-30T00:42:07.872622+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.872622+0000', 'last_undegraded': '2024-03-30T00:42:07.872622+0000', 'last_fullsized': '2024-03-30T00:42:07.872622+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.888796+0000', 'last_change': '2024-03-30T00:42:07.888796+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.888796+0000', 'last_undegraded': '2024-03-30T00:42:07.888796+0000', 'last_fullsized': '2024-03-30T00:42:07.888796+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.877961+0000', 'last_change': '2024-03-30T00:42:07.877961+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.877961+0000', 'last_undegraded': '2024-03-30T00:42:07.877961+0000', 'last_fullsized': '2024-03-30T00:42:07.877961+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T12:10:20.006391+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.877522+0000', 'last_change': '2024-03-30T00:42:07.877522+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.877522+0000', 'last_undegraded': '2024-03-30T00:42:07.877522+0000', 'last_fullsized': '2024-03-30T00:42:07.877522+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:53:19.899232+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:42:13.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.877834+0000', 'last_change': '2024-03-30T00:42:07.877834+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.877834+0000', 'last_undegraded': '2024-03-30T00:42:07.877834+0000', 'last_fullsized': '2024-03-30T00:42:07.877834+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.878875+0000', 'last_change': '2024-03-30T00:42:07.878875+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.878875+0000', 'last_undegraded': '2024-03-30T00:42:07.878875+0000', 'last_fullsized': '2024-03-30T00:42:07.878875+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:33:26.322503+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.881115+0000', 'last_change': '2024-03-30T00:42:07.881115+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.881115+0000', 'last_undegraded': '2024-03-30T00:42:07.881115+0000', 'last_fullsized': '2024-03-30T00:42:07.881115+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:34:13.403177+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.887228+0000', 'last_change': '2024-03-30T00:42:07.887228+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.887228+0000', 'last_undegraded': '2024-03-30T00:42:07.887228+0000', 'last_fullsized': '2024-03-30T00:42:07.887228+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.880458+0000', 'last_change': '2024-03-30T00:42:07.880458+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.880458+0000', 'last_undegraded': '2024-03-30T00:42:07.880458+0000', 'last_fullsized': '2024-03-30T00:42:07.880458+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.872899+0000', 'last_change': '2024-03-30T00:42:07.872899+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.872899+0000', 'last_undegraded': '2024-03-30T00:42:07.872899+0000', 'last_fullsized': '2024-03-30T00:42:07.872899+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:52:22.536352+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:42:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.877458+0000', 'last_change': '2024-03-30T00:42:07.877458+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.877458+0000', 'last_undegraded': '2024-03-30T00:42:07.877458+0000', 'last_fullsized': '2024-03-30T00:42:07.877458+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:59:20.469591+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.893998+0000', 'last_change': '2024-03-30T00:42:07.893998+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.893998+0000', 'last_undegraded': '2024-03-30T00:42:07.893998+0000', 'last_fullsized': '2024-03-30T00:42:07.893998+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.872039+0000', 'last_change': '2024-03-30T00:42:07.872039+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.872039+0000', 'last_undegraded': '2024-03-30T00:42:07.872039+0000', 'last_fullsized': '2024-03-30T00:42:07.872039+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.881306+0000', 'last_change': '2024-03-30T00:42:07.881306+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.881306+0000', 'last_undegraded': '2024-03-30T00:42:07.881306+0000', 'last_fullsized': '2024-03-30T00:42:07.881306+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T02:25:49.025731+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.877550+0000', 'last_change': '2024-03-30T00:42:07.877550+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.877550+0000', 'last_undegraded': '2024-03-30T00:42:07.877550+0000', 'last_fullsized': '2024-03-30T00:42:07.877550+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.871624+0000', 'last_change': '2024-03-30T00:42:07.871624+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.871624+0000', 'last_undegraded': '2024-03-30T00:42:07.871624+0000', 'last_fullsized': '2024-03-30T00:42:07.871624+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:17:22.441495+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:42:13.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.876251+0000', 'last_change': '2024-03-30T00:42:07.876251+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.876251+0000', 'last_undegraded': '2024-03-30T00:42:07.876251+0000', 'last_fullsized': '2024-03-30T00:42:07.876251+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:27:29.266461+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.878821+0000', 'last_change': '2024-03-30T00:42:07.878821+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.878821+0000', 'last_undegraded': '2024-03-30T00:42:07.878821+0000', 'last_fullsized': '2024-03-30T00:42:07.878821+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T08:57:03.872764+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.877433+0000', 'last_change': '2024-03-30T00:42:07.877433+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.877433+0000', 'last_undegraded': '2024-03-30T00:42:07.877433+0000', 'last_fullsized': '2024-03-30T00:42:07.877433+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.874348+0000', 'last_change': '2024-03-30T00:42:07.874348+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.874348+0000', 'last_undegraded': '2024-03-30T00:42:07.874348+0000', 'last_fullsized': '2024-03-30T00:42:07.874348+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T02:23:59.202589+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.873753+0000', 'last_change': '2024-03-30T00:42:07.873753+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.873753+0000', 'last_undegraded': '2024-03-30T00:42:07.873753+0000', 'last_fullsized': '2024-03-30T00:42:07.873753+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.879706+0000', 'last_change': '2024-03-30T00:42:07.879706+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.879706+0000', 'last_undegraded': '2024-03-30T00:42:07.879706+0000', 'last_fullsized': '2024-03-30T00:42:07.879706+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-03-30T00:42:13.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.873193+0000', 'last_change': '2024-03-30T00:42:07.873193+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.873193+0000', 'last_undegraded': '2024-03-30T00:42:07.873193+0000', 'last_fullsized': '2024-03-30T00:42:07.873193+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:14:22.099119+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:42:13.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-03-30T00:42:13.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.873054+0000', 'last_change': '2024-03-30T00:42:07.873054+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.873054+0000', 'last_undegraded': '2024-03-30T00:42:07.873054+0000', 'last_fullsized': '2024-03-30T00:42:07.873054+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:42:13.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-03-30T00:42:13.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.876922+0000', 'last_change': '2024-03-30T00:42:07.876922+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.876922+0000', 'last_undegraded': '2024-03-30T00:42:07.876922+0000', 'last_fullsized': '2024-03-30T00:42:07.876922+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:42:13.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-03-30T00:42:13.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 52, 'state': 'creating+activating', 'last_fresh': '2024-03-30T00:42:07.874579+0000', 'last_change': '2024-03-30T00:42:07.874579+0000', 'last_active': '2024-03-30T00:42:06.857438+0000', 'last_peered': '2024-03-30T00:42:06.857438+0000', 'last_clean': '2024-03-30T00:42:06.857438+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.874579+0000', 'last_undegraded': '2024-03-30T00:42:07.874579+0000', 'last_fullsized': '2024-03-30T00:42:07.874579+0000', 'mapping_epoch': 51, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 51, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:13:17.278798+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': ['4', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:42:13.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-03-30T00:42:13.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.881886+0000', 'last_change': '2024-03-30T00:42:07.881886+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.881886+0000', 'last_undegraded': '2024-03-30T00:42:07.881886+0000', 'last_fullsized': '2024-03-30T00:42:07.881886+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:52:07.667668+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-03-30T00:42:13.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.873860+0000', 'last_change': '2024-03-30T00:42:07.873860+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.873860+0000', 'last_undegraded': '2024-03-30T00:42:07.873860+0000', 'last_fullsized': '2024-03-30T00:42:07.873860+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:42:13.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-03-30T00:42:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 52, 'state': 'creating+activating', 'last_fresh': '2024-03-30T00:42:07.878493+0000', 'last_change': '2024-03-30T00:42:07.878493+0000', 'last_active': '2024-03-30T00:42:06.857438+0000', 'last_peered': '2024-03-30T00:42:06.857438+0000', 'last_clean': '2024-03-30T00:42:06.857438+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.878493+0000', 'last_undegraded': '2024-03-30T00:42:07.878493+0000', 'last_fullsized': '2024-03-30T00:42:07.878493+0000', 'mapping_epoch': 51, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 51, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T02:25:48.023033+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['3', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-03-30T00:42:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.881591+0000', 'last_change': '2024-03-30T00:42:07.881591+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.881591+0000', 'last_undegraded': '2024-03-30T00:42:07.881591+0000', 'last_fullsized': '2024-03-30T00:42:07.881591+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:42:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-03-30T00:42:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.888318+0000', 'last_change': '2024-03-30T00:42:07.888318+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.888318+0000', 'last_undegraded': '2024-03-30T00:42:07.888318+0000', 'last_fullsized': '2024-03-30T00:42:07.888318+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-03-30T00:42:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 52, 'state': 'creating+activating', 'last_fresh': '2024-03-30T00:42:07.877016+0000', 'last_change': '2024-03-30T00:42:07.877016+0000', 'last_active': '2024-03-30T00:42:06.857438+0000', 'last_peered': '2024-03-30T00:42:06.857438+0000', 'last_clean': '2024-03-30T00:42:06.857438+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.877016+0000', 'last_undegraded': '2024-03-30T00:42:07.877016+0000', 'last_fullsized': '2024-03-30T00:42:07.877016+0000', 'mapping_epoch': 51, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 51, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:58:41.813204+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': ['6', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:42:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-03-30T00:42:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.880083+0000', 'last_change': '2024-03-30T00:42:07.880083+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.880083+0000', 'last_undegraded': '2024-03-30T00:42:07.880083+0000', 'last_fullsized': '2024-03-30T00:42:07.880083+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:14:44.118291+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-03-30T00:42:13.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.888542+0000', 'last_change': '2024-03-30T00:42:07.888542+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.888542+0000', 'last_undegraded': '2024-03-30T00:42:07.888542+0000', 'last_fullsized': '2024-03-30T00:42:07.888542+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:26:01.442897+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 52, 'state': 'creating+activating', 'last_fresh': '2024-03-30T00:42:07.876507+0000', 'last_change': '2024-03-30T00:42:07.876507+0000', 'last_active': '2024-03-30T00:42:06.857438+0000', 'last_peered': '2024-03-30T00:42:06.857438+0000', 'last_clean': '2024-03-30T00:42:06.857438+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.876507+0000', 'last_undegraded': '2024-03-30T00:42:07.876507+0000', 'last_fullsized': '2024-03-30T00:42:07.876507+0000', 'mapping_epoch': 51, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 51, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:16:59.791004+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['3', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.876783+0000', 'last_change': '2024-03-30T00:42:07.876783+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.876783+0000', 'last_undegraded': '2024-03-30T00:42:07.876783+0000', 'last_fullsized': '2024-03-30T00:42:07.876783+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.873297+0000', 'last_change': '2024-03-30T00:42:07.873297+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.873297+0000', 'last_undegraded': '2024-03-30T00:42:07.873297+0000', 'last_fullsized': '2024-03-30T00:42:07.873297+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.873770+0000', 'last_change': '2024-03-30T00:42:07.873770+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.873770+0000', 'last_undegraded': '2024-03-30T00:42:07.873770+0000', 'last_fullsized': '2024-03-30T00:42:07.873770+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.872778+0000', 'last_change': '2024-03-30T00:42:07.872778+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.872778+0000', 'last_undegraded': '2024-03-30T00:42:07.872778+0000', 'last_fullsized': '2024-03-30T00:42:07.872778+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.879997+0000', 'last_change': '2024-03-30T00:42:07.879997+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.879997+0000', 'last_undegraded': '2024-03-30T00:42:07.879997+0000', 'last_fullsized': '2024-03-30T00:42:07.879997+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:39:42.969334+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:42:13.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-03-30T00:42:13.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.880105+0000', 'last_change': '2024-03-30T00:42:07.880105+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.880105+0000', 'last_undegraded': '2024-03-30T00:42:07.880105+0000', 'last_fullsized': '2024-03-30T00:42:07.880105+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:00:02.124627+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:42:13.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-03-30T00:42:13.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.889733+0000', 'last_change': '2024-03-30T00:42:07.889733+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.889733+0000', 'last_undegraded': '2024-03-30T00:42:07.889733+0000', 'last_fullsized': '2024-03-30T00:42:07.889733+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-03-30T00:42:13.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.872629+0000', 'last_change': '2024-03-30T00:42:07.872629+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.872629+0000', 'last_undegraded': '2024-03-30T00:42:07.872629+0000', 'last_fullsized': '2024-03-30T00:42:07.872629+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:42:13.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-03-30T00:42:13.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.876796+0000', 'last_change': '2024-03-30T00:42:07.876796+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.876796+0000', 'last_undegraded': '2024-03-30T00:42:07.876796+0000', 'last_fullsized': '2024-03-30T00:42:07.876796+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:59:40.974026+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:42:13.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-03-30T00:42:13.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 52, 'state': 'creating+activating', 'last_fresh': '2024-03-30T00:42:07.878452+0000', 'last_change': '2024-03-30T00:42:07.878452+0000', 'last_active': '2024-03-30T00:42:06.857438+0000', 'last_peered': '2024-03-30T00:42:06.857438+0000', 'last_clean': '2024-03-30T00:42:06.857438+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.878452+0000', 'last_undegraded': '2024-03-30T00:42:07.878452+0000', 'last_fullsized': '2024-03-30T00:42:07.878452+0000', 'mapping_epoch': 51, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 51, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:16:30.469686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': ['6', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:42:13.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-03-30T00:42:13.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.872796+0000', 'last_change': '2024-03-30T00:42:07.872796+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.872796+0000', 'last_undegraded': '2024-03-30T00:42:07.872796+0000', 'last_fullsized': '2024-03-30T00:42:07.872796+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:30:50.535530+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 52, 'state': 'creating+activating', 'last_fresh': '2024-03-30T00:42:07.876204+0000', 'last_change': '2024-03-30T00:42:07.876204+0000', 'last_active': '2024-03-30T00:42:06.857438+0000', 'last_peered': '2024-03-30T00:42:06.857438+0000', 'last_clean': '2024-03-30T00:42:06.857438+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.876204+0000', 'last_undegraded': '2024-03-30T00:42:07.876204+0000', 'last_fullsized': '2024-03-30T00:42:07.876204+0000', 'mapping_epoch': 51, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 51, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:06.857438+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:02:33.633690+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.876968+0000', 'last_change': '2024-03-30T00:42:07.876968+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.876968+0000', 'last_undegraded': '2024-03-30T00:42:07.876968+0000', 'last_fullsized': '2024-03-30T00:42:07.876968+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:51:58.102992+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.892109+0000', 'last_change': '2024-03-30T00:42:07.892109+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.892109+0000', 'last_undegraded': '2024-03-30T00:42:07.892109+0000', 'last_fullsized': '2024-03-30T00:42:07.892109+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.878795+0000', 'last_change': '2024-03-30T00:42:07.878795+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.878795+0000', 'last_undegraded': '2024-03-30T00:42:07.878795+0000', 'last_fullsized': '2024-03-30T00:42:07.878795+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.890675+0000', 'last_change': '2024-03-30T00:42:07.890675+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.890675+0000', 'last_undegraded': '2024-03-30T00:42:07.890675+0000', 'last_fullsized': '2024-03-30T00:42:07.890675+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-03-30T00:42:13.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.877866+0000', 'last_change': '2024-03-30T00:42:07.877866+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.877866+0000', 'last_undegraded': '2024-03-30T00:42:07.877866+0000', 'last_fullsized': '2024-03-30T00:42:07.877866+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:42:13.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-03-30T00:42:13.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:42:07.879604+0000', 'last_change': '2024-03-30T00:42:07.879604+0000', 'last_active': '2024-03-30T00:42:07.860136+0000', 'last_peered': '2024-03-30T00:42:07.860136+0000', 'last_clean': '2024-03-30T00:42:07.860136+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:42:07.879604+0000', 'last_undegraded': '2024-03-30T00:42:07.879604+0000', 'last_fullsized': '2024-03-30T00:42:07.879604+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'last_clean_scrub_stamp': '2024-03-30T00:42:07.860136+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:02:34.636388+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:42:13.398 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-30T00:42:13.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.552+0000 7f327a800640 1 Processor -- start 2024-03-30T00:42:13.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.552+0000 7f327a800640 1 -- start start 2024-03-30T00:42:13.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f327a800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f327415a590 0x7f327415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f327a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f32740cb820 0x7f32740b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f327a800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32740b2620 con 0x7f327415a970 2024-03-30T00:42:13.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f327a800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32740b22f0 con 0x7f32740cb820 2024-03-30T00:42:13.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f327a800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327415eb40 con 0x7f327415a590 2024-03-30T00:42:13.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f3273e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f32740cb820 0x7f32740b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f3273e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f32740cb820 0x7f32740b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36338/0 (socket says 172.21.15.66:36338) 2024-03-30T00:42:13.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f3273e00640 1 -- 172.21.15.66:0/984855598 learned_addr learned my addr 172.21.15.66:0/984855598 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:13.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f3273400640 1 --2- 172.21.15.66:0/984855598 >> v2:172.21.15.156:3300/0 conn(0x7f327415a590 0x7f327415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f3273e00640 1 -- 172.21.15.66:0/984855598 >> v2:172.21.15.156:3300/0 conn(0x7f327415a590 msgr2=0x7f327415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:13.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f3273e00640 1 --2- 172.21.15.66:0/984855598 >> v2:172.21.15.156:3300/0 conn(0x7f327415a590 0x7f327415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f3273e00640 1 -- 172.21.15.66:0/984855598 >> v1:172.21.15.156:6789/0 conn(0x7f327415a970 legacy=0x7f32740cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f3273e00640 1 -- 172.21.15.66:0/984855598 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327415ee20 con 0x7f32740cb820 2024-03-30T00:42:13.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.556+0000 7f3273400640 1 --2- 172.21.15.66:0/984855598 >> v2:172.21.15.156:3300/0 conn(0x7f327415a590 0x7f327415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:13.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f3273e00640 1 --2- 172.21.15.66:0/984855598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32740cb820 0x7f32740b2ab0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f326c009e50 tx=0x7f326c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82033a133bd5b3e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:13.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f3272a00640 1 -- 172.21.15.66:0/984855598 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c0af400 con 0x7f32740cb820 2024-03-30T00:42:13.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f3272a00640 1 -- 172.21.15.66:0/984855598 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c0ab050 con 0x7f32740cb820 2024-03-30T00:42:13.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f3272a00640 1 -- 172.21.15.66:0/984855598 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c0af780 con 0x7f32740cb820 2024-03-30T00:42:13.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f327a800640 1 -- 172.21.15.66:0/984855598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32740cb820 msgr2=0x7f32740b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f327a800640 1 --2- 172.21.15.66:0/984855598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32740cb820 0x7f32740b2ab0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f326c009e50 tx=0x7f326c01f1e0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f327a800640 1 -- 172.21.15.66:0/984855598 shutdown_connections 2024-03-30T00:42:13.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f327a800640 1 --2- 172.21.15.66:0/984855598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32740cb820 0x7f32740b2ab0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f327a800640 1 --2- 172.21.15.66:0/984855598 >> v2:172.21.15.156:3300/0 conn(0x7f327415a590 0x7f327415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f327a800640 1 -- 172.21.15.66:0/984855598 >> 172.21.15.66:0/984855598 conn(0x7f32740cf4a0 msgr2=0x7f32740cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:13.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f327a800640 1 -- 172.21.15.66:0/984855598 shutdown_connections 2024-03-30T00:42:13.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f327a800640 1 -- 172.21.15.66:0/984855598 wait complete. 2024-03-30T00:42:13.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f327a800640 1 Processor -- start 2024-03-30T00:42:13.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.560+0000 7f327a800640 1 -- start start 2024-03-30T00:42:13.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f327a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32740cb820 0x7f32740c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f327a800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f327415a590 0x7f327415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f327a800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f327415a970 0x7f327415c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f327a800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32741f2480 con 0x7f32740cb820 2024-03-30T00:42:13.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f327a800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32741f2760 con 0x7f327415a970 2024-03-30T00:42:13.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f327a800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32741f2a40 con 0x7f327415a590 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f3273400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32740cb820 0x7f32740c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f3273400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32740cb820 0x7f32740c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36346/0 (socket says 172.21.15.66:36346) 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f3273400640 1 -- 172.21.15.66:0/1855205999 learned_addr learned my addr 172.21.15.66:0/1855205999 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f3273e00640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f327415a970 0x7f327415c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f326aa00640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f327415a590 0x7f327415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f3273400640 1 -- 172.21.15.66:0/1855205999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f327415a590 msgr2=0x7f327415bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f3273400640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f327415a590 0x7f327415bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f3273400640 1 -- 172.21.15.66:0/1855205999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f327415a970 msgr2=0x7f327415c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f3273400640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f327415a970 0x7f327415c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f3273400640 1 -- 172.21.15.66:0/1855205999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c009b30 con 0x7f32740cb820 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f3273400640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32740cb820 0x7f32740c5400 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f326400e810 tx=0x7f326400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f326be00640 1 -- 172.21.15.66:0/1855205999 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400c8d0 con 0x7f32740cb820 2024-03-30T00:42:13.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f326be00640 1 -- 172.21.15.66:0/1855205999 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326400ca30 con 0x7f32740cb820 2024-03-30T00:42:13.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f326be00640 1 -- 172.21.15.66:0/1855205999 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400cdc0 con 0x7f32740cb820 2024-03-30T00:42:13.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f327a800640 1 -- 172.21.15.66:0/1855205999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32741f2d20 con 0x7f32740cb820 2024-03-30T00:42:13.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f327a800640 1 -- 172.21.15.66:0/1855205999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32741f20c0 con 0x7f32740cb820 2024-03-30T00:42:13.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f327a800640 1 -- 172.21.15.66:0/1855205999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3274066c30 con 0x7f32740cb820 2024-03-30T00:42:13.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.564+0000 7f326be00640 1 -- 172.21.15.66:0/1855205999 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3264013410 con 0x7f32740cb820 2024-03-30T00:42:13.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.568+0000 7f326be00640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3250066690 0x7f3250068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.568+0000 7f326be00640 1 -- 172.21.15.66:0/1855205999 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3264083ff0 con 0x7f32740cb820 2024-03-30T00:42:13.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.568+0000 7f326aa00640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3250066690 0x7f3250068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.568+0000 7f326aa00640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3250066690 0x7f3250068b50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f32740c9660 tx=0x7f325c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:13.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.572+0000 7f326be00640 1 -- 172.21.15.66:0/1855205999 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3264051320 con 0x7f32740cb820 2024-03-30T00:42:13.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 -- 172.21.15.66:0/1855205999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f32740b5420 con 0x7f32740cb820 2024-03-30T00:42:13.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f326be00640 1 -- 172.21.15.66:0/1855205999 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1500 (secure 0 0 0) 0x7f3264054940 con 0x7f32740cb820 2024-03-30T00:42:13.798 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:13.798 INFO:teuthology.orchestra.run.smithi066.stdout:{"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":168,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1711759177,"num_in_osds":8,"osd_in_since":1711759169,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":27},{"state_name":"creating+activating","count":6}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":244174848,"bytes_avail":772849938432,"bytes_total":773094113280,"inactive_pgs_ratio":0.72164946794509888},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5985}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-03-30T00:41:56.877068+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-30T00:42:13.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 -- 172.21.15.66:0/1855205999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3250066690 msgr2=0x7f3250068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3250066690 0x7f3250068b50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f32740c9660 tx=0x7f325c009040 comp rx=0 tx=0).stop 2024-03-30T00:42:13.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 -- 172.21.15.66:0/1855205999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32740cb820 msgr2=0x7f32740c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32740cb820 0x7f32740c5400 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f326400e810 tx=0x7f326400ecd0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 -- 172.21.15.66:0/1855205999 shutdown_connections 2024-03-30T00:42:13.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f327415a970 0x7f327415c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f327415a590 0x7f327415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3250066690 0x7f3250068b50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 --2- 172.21.15.66:0/1855205999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32740cb820 0x7f32740c5400 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 -- 172.21.15.66:0/1855205999 >> 172.21.15.66:0/1855205999 conn(0x7f32740cf4a0 msgr2=0x7f32740b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:13.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 -- 172.21.15.66:0/1855205999 shutdown_connections 2024-03-30T00:42:13.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.788+0000 7f327a800640 1 -- 172.21.15.66:0/1855205999 wait complete. 2024-03-30T00:42:13.818 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-30T00:42:13.818 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:42:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.948+0000 7efcaa200640 1 Processor -- start 2024-03-30T00:42:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.948+0000 7efcaa200640 1 -- start start 2024-03-30T00:42:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.948+0000 7efcaa200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efca406f8e0 0x7efca40d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.948+0000 7efcaa200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca406b0b0 con 0x7efca40cc560 2024-03-30T00:42:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.948+0000 7efcaa200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca406b390 con 0x7efca406e800 2024-03-30T00:42:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.948+0000 7efcaa200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca4160640 con 0x7efca406f8e0 2024-03-30T00:42:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.948+0000 7efca3400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efca40cc560 0x7efca406e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55670/0 (socket says 172.21.15.66:55670) 2024-03-30T00:42:13.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.948+0000 7efca3400640 1 -- 172.21.15.66:0/1401501897 learned_addr learned my addr 172.21.15.66:0/1401501897 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:13.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.948+0000 7efca3e00640 1 --2- 172.21.15.66:0/1401501897 >> v2:172.21.15.156:3300/0 conn(0x7efca406f8e0 0x7efca40d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.948+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291106892 0 0) 0x7efca406b0b0 con 0x7efca40cc560 2024-03-30T00:42:13.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc88003530 con 0x7efca40cc560 2024-03-30T00:42:13.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4081538403 0 0) 0x7efc88003530 con 0x7efca40cc560 2024-03-30T00:42:13.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 >> v2:172.21.15.156:3300/0 conn(0x7efca406f8e0 msgr2=0x7efca40d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:13.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 --2- 172.21.15.66:0/1401501897 >> v2:172.21.15.156:3300/0 conn(0x7efca406f8e0 0x7efca40d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 >> v1:172.21.15.156:6789/0 conn(0x7efca406e800 legacy=0x7efca40ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca406b670 con 0x7efca40cc560 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7efc94001680 con 0x7efca40cc560 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 >> v1:172.21.15.66:6789/0 conn(0x7efca40cc560 legacy=0x7efca406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 0x7efc88003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc880047d0 0x7efc88004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc88005190 0x7efc8800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca4160640 con 0x7efc88003980 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8800c030 con 0x7efc88005190 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8800c1f0 con 0x7efc880047d0 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca3400640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 0x7efc88003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca3e00640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc88005190 0x7efc8800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efc9be00640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc880047d0 0x7efc88004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca3400640 1 -- 172.21.15.66:0/1401501897 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc880047d0 msgr2=0x7efc88004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca3400640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc880047d0 0x7efc88004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca3400640 1 -- 172.21.15.66:0/1401501897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc88005190 msgr2=0x7efc8800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:13.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca3400640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc88005190 0x7efc8800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca3400640 1 -- 172.21.15.66:0/1401501897 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc88003530 con 0x7efc88003980 2024-03-30T00:42:13.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca3e00640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc88005190 0x7efc8800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:13.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca3e00640 1 -- 172.21.15.66:0/1401501897 reap_dead start 2024-03-30T00:42:13.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.952+0000 7efca3400640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 0x7efc88003da0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c240 tx=0x7efc9401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a0e6c1ee9be4f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:13.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94005ae0 con 0x7efc88003980 2024-03-30T00:42:13.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc94005c40 con 0x7efc88003980 2024-03-30T00:42:13.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efca2a00640 1 -- 172.21.15.66:0/1401501897 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc940a59f0 con 0x7efc88003980 2024-03-30T00:42:13.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 -- 172.21.15.66:0/1401501897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 msgr2=0x7efc88003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 0x7efc88003da0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c240 tx=0x7efc9401cbf0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 -- 172.21.15.66:0/1401501897 shutdown_connections 2024-03-30T00:42:13.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 --2- 172.21.15.66:0/1401501897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 0x7efc88003da0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 -- 172.21.15.66:0/1401501897 >> 172.21.15.66:0/1401501897 conn(0x7efca40d0540 msgr2=0x7efca40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:13.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 -- 172.21.15.66:0/1401501897 shutdown_connections 2024-03-30T00:42:13.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 -- 172.21.15.66:0/1401501897 wait complete. 2024-03-30T00:42:13.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 Processor -- start 2024-03-30T00:42:13.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 -- start start 2024-03-30T00:42:13.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 0x7efca40c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efca40c58f0 0x7efca415a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efca40ad2b0 0x7efca40ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40c9fa0 con 0x7efc88003980 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40adc50 con 0x7efca40c58f0 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efcaa200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca40c95b0 con 0x7efca40ad2b0 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efca3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 0x7efca40c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efca3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 0x7efca40c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36378/0 (socket says 172.21.15.66:36378) 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.956+0000 7efca3400640 1 -- 172.21.15.66:0/1849328663 learned_addr learned my addr 172.21.15.66:0/1849328663 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efc9be00640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efca40c58f0 0x7efca415a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efca3400640 1 -- 172.21.15.66:0/1849328663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efca40ad2b0 msgr2=0x7efca40ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efca3400640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efca40ad2b0 0x7efca40ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efca3400640 1 -- 172.21.15.66:0/1849328663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efca40c58f0 msgr2=0x7efca415a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efca3400640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efca40c58f0 0x7efca415a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:13.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efca3400640 1 -- 172.21.15.66:0/1849328663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca40c5e20 con 0x7efc88003980 2024-03-30T00:42:13.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efc9be00640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efca40c58f0 0x7efca415a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:13.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efca3400640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 0x7efca40c9070 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efc9401ca20 tx=0x7efc88005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efc9b400640 1 -- 172.21.15.66:0/1849328663 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc940b4070 con 0x7efc88003980 2024-03-30T00:42:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efc9b400640 1 -- 172.21.15.66:0/1849328663 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc940015c0 con 0x7efc88003980 2024-03-30T00:42:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efcaa200640 1 -- 172.21.15.66:0/1849328663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca40c6100 con 0x7efc88003980 2024-03-30T00:42:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efc9b400640 1 -- 172.21.15.66:0/1849328663 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc940a57b0 con 0x7efc88003980 2024-03-30T00:42:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efcaa200640 1 -- 172.21.15.66:0/1849328663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca40c66c0 con 0x7efc88003980 2024-03-30T00:42:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efcaa200640 1 -- 172.21.15.66:0/1849328663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca40b0e50 con 0x7efc88003980 2024-03-30T00:42:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efc9b400640 1 -- 172.21.15.66:0/1849328663 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efc940a5930 con 0x7efc88003980 2024-03-30T00:42:13.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efc9b400640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc80066690 0x7efc80068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:13.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efc9b400640 1 -- 172.21.15.66:0/1849328663 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efc9411df50 con 0x7efc88003980 2024-03-30T00:42:13.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.960+0000 7efc9be00640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc80066690 0x7efc80068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:13.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.964+0000 7efc9be00640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc80066690 0x7efc80068b50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc8c000c10 tx=0x7efc8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:13.964+0000 7efc9b400640 1 -- 172.21.15.66:0/1849328663 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efc940eb050 con 0x7efc88003980 2024-03-30T00:42:14.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.124+0000 7efcaa200640 1 -- 172.21.15.66:0/1849328663 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efca40cc940 con 0x7efc80066690 2024-03-30T00:42:14.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.132+0000 7efc9b400640 1 -- 172.21.15.66:0/1849328663 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285961 (secure 0 0 0) 0x7efc8c00d050 con 0x7efc80066690 2024-03-30T00:42:14.147 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:14.148 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:42:14.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 -- 172.21.15.66:0/1849328663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc80066690 msgr2=0x7efc80068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:14.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc80066690 0x7efc80068b50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc8c000c10 tx=0x7efc8c009040 comp rx=0 tx=0).stop 2024-03-30T00:42:14.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 -- 172.21.15.66:0/1849328663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 msgr2=0x7efca40c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:14.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 0x7efca40c9070 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efc9401ca20 tx=0x7efc88005800 comp rx=0 tx=0).stop 2024-03-30T00:42:14.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 -- 172.21.15.66:0/1849328663 shutdown_connections 2024-03-30T00:42:14.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efca40ad2b0 0x7efca40ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:14.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efca40c58f0 0x7efca415a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:14.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc80066690 0x7efc80068b50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:14.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 --2- 172.21.15.66:0/1849328663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc88003980 0x7efca40c9070 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:14.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 -- 172.21.15.66:0/1849328663 >> 172.21.15.66:0/1849328663 conn(0x7efca40d0540 msgr2=0x7efca40aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:14.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 -- 172.21.15.66:0/1849328663 shutdown_connections 2024-03-30T00:42:14.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:14.136+0000 7efcaa200640 1 -- 172.21.15.66:0/1849328663 wait complete. 2024-03-30T00:42:14.168 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-03-30T00:42:12.879585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238452,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754736268,"statfs":{"total":773094113280,"available":772849938432,"internally_reserved":0,"allocated":6037504,"data_stored":3291064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000958"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874156+0000","last_change":"2024-03-30T00:42:07.874156+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874156+0000","last_undegraded":"2024-03-30T00:42:07.874156+0000","last_fullsized":"2024-03-30T00:42:07.874156+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873050+0000","last_change":"2024-03-30T00:42:07.873050+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873050+0000","last_undegraded":"2024-03-30T00:42:07.873050+0000","last_fullsized":"2024-03-30T00:42:07.873050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.889268+0000","last_change":"2024-03-30T00:42:07.889268+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.889268+0000","last_undegraded":"2024-03-30T00:42:07.889268+0000","last_fullsized":"2024-03-30T00:42:07.889268+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872559+0000","last_change":"2024-03-30T00:42:07.872559+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872559+0000","last_undegraded":"2024-03-30T00:42:07.872559+0000","last_fullsized":"2024-03-30T00:42:07.872559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874398+0000","last_change":"2024-03-30T00:42:07.874398+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874398+0000","last_undegraded":"2024-03-30T00:42:07.874398+0000","last_fullsized":"2024-03-30T00:42:07.874398+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873733+0000","last_change":"2024-03-30T00:42:07.873733+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873733+0000","last_undegraded":"2024-03-30T00:42:07.873733+0000","last_fullsized":"2024-03-30T00:42:07.873733+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:22.099119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881871+0000","last_change":"2024-03-30T00:42:07.881871+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881871+0000","last_undegraded":"2024-03-30T00:42:07.881871+0000","last_fullsized":"2024-03-30T00:42:07.881871+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:09.745566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877328+0000","last_change":"2024-03-30T00:42:07.877328+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877328+0000","last_undegraded":"2024-03-30T00:42:07.877328+0000","last_fullsized":"2024-03-30T00:42:07.877328+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:01.749453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878241+0000","last_change":"2024-03-30T00:42:07.878241+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878241+0000","last_undegraded":"2024-03-30T00:42:07.878241+0000","last_fullsized":"2024-03-30T00:42:07.878241+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874651+0000","last_change":"2024-03-30T00:42:07.874651+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874651+0000","last_undegraded":"2024-03-30T00:42:07.874651+0000","last_fullsized":"2024-03-30T00:42:07.874651+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879446+0000","last_change":"2024-03-30T00:42:07.879446+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879446+0000","last_undegraded":"2024-03-30T00:42:07.879446+0000","last_fullsized":"2024-03-30T00:42:07.879446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:04.810334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878717+0000","last_change":"2024-03-30T00:42:07.878717+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878717+0000","last_undegraded":"2024-03-30T00:42:07.878717+0000","last_fullsized":"2024-03-30T00:42:07.878717+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874011+0000","last_change":"2024-03-30T00:42:07.874011+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874011+0000","last_undegraded":"2024-03-30T00:42:07.874011+0000","last_fullsized":"2024-03-30T00:42:07.874011+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:03.287560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.882341+0000","last_change":"2024-03-30T00:42:07.882341+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.882341+0000","last_undegraded":"2024-03-30T00:42:07.882341+0000","last_fullsized":"2024-03-30T00:42:07.882341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:54:01.171986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.871624+0000","last_change":"2024-03-30T00:42:07.871624+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.871624+0000","last_undegraded":"2024-03-30T00:42:07.871624+0000","last_fullsized":"2024-03-30T00:42:07.871624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:56.870783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872827+0000","last_change":"2024-03-30T00:42:07.872827+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872827+0000","last_undegraded":"2024-03-30T00:42:07.872827+0000","last_fullsized":"2024-03-30T00:42:07.872827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:24.904667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888103+0000","last_change":"2024-03-30T00:42:07.888103+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888103+0000","last_undegraded":"2024-03-30T00:42:07.888103+0000","last_fullsized":"2024-03-30T00:42:07.888103+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:41:31.279358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872622+0000","last_change":"2024-03-30T00:42:07.872622+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872622+0000","last_undegraded":"2024-03-30T00:42:07.872622+0000","last_fullsized":"2024-03-30T00:42:07.872622+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888796+0000","last_change":"2024-03-30T00:42:07.888796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888796+0000","last_undegraded":"2024-03-30T00:42:07.888796+0000","last_fullsized":"2024-03-30T00:42:07.888796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877961+0000","last_change":"2024-03-30T00:42:07.877961+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877961+0000","last_undegraded":"2024-03-30T00:42:07.877961+0000","last_fullsized":"2024-03-30T00:42:07.877961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:10:20.006391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877522+0000","last_change":"2024-03-30T00:42:07.877522+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877522+0000","last_undegraded":"2024-03-30T00:42:07.877522+0000","last_fullsized":"2024-03-30T00:42:07.877522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:19.899232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877834+0000","last_change":"2024-03-30T00:42:07.877834+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877834+0000","last_undegraded":"2024-03-30T00:42:07.877834+0000","last_fullsized":"2024-03-30T00:42:07.877834+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878875+0000","last_change":"2024-03-30T00:42:07.878875+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878875+0000","last_undegraded":"2024-03-30T00:42:07.878875+0000","last_fullsized":"2024-03-30T00:42:07.878875+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:26.322503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881115+0000","last_change":"2024-03-30T00:42:07.881115+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881115+0000","last_undegraded":"2024-03-30T00:42:07.881115+0000","last_fullsized":"2024-03-30T00:42:07.881115+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:34:13.403177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.887228+0000","last_change":"2024-03-30T00:42:07.887228+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.887228+0000","last_undegraded":"2024-03-30T00:42:07.887228+0000","last_fullsized":"2024-03-30T00:42:07.887228+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880458+0000","last_change":"2024-03-30T00:42:07.880458+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880458+0000","last_undegraded":"2024-03-30T00:42:07.880458+0000","last_fullsized":"2024-03-30T00:42:07.880458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872899+0000","last_change":"2024-03-30T00:42:07.872899+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872899+0000","last_undegraded":"2024-03-30T00:42:07.872899+0000","last_fullsized":"2024-03-30T00:42:07.872899+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:22.536352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877458+0000","last_change":"2024-03-30T00:42:07.877458+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877458+0000","last_undegraded":"2024-03-30T00:42:07.877458+0000","last_fullsized":"2024-03-30T00:42:07.877458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:20.469591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.893998+0000","last_change":"2024-03-30T00:42:07.893998+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.893998+0000","last_undegraded":"2024-03-30T00:42:07.893998+0000","last_fullsized":"2024-03-30T00:42:07.893998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872039+0000","last_change":"2024-03-30T00:42:07.872039+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872039+0000","last_undegraded":"2024-03-30T00:42:07.872039+0000","last_fullsized":"2024-03-30T00:42:07.872039+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881306+0000","last_change":"2024-03-30T00:42:07.881306+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881306+0000","last_undegraded":"2024-03-30T00:42:07.881306+0000","last_fullsized":"2024-03-30T00:42:07.881306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:49.025731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877550+0000","last_change":"2024-03-30T00:42:07.877550+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877550+0000","last_undegraded":"2024-03-30T00:42:07.877550+0000","last_fullsized":"2024-03-30T00:42:07.877550+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.871624+0000","last_change":"2024-03-30T00:42:07.871624+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.871624+0000","last_undegraded":"2024-03-30T00:42:07.871624+0000","last_fullsized":"2024-03-30T00:42:07.871624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:17:22.441495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890610+0000","last_change":"2024-03-30T00:42:07.887914+0000","last_active":"2024-03-30T00:42:07.890610+0000","last_peered":"2024-03-30T00:42:07.890610+0000","last_clean":"2024-03-30T00:42:07.890610+0000","last_became_active":"2024-03-30T00:42:07.887626+0000","last_became_peered":"2024-03-30T00:42:07.887626+0000","last_unstale":"2024-03-30T00:42:07.890610+0000","last_undegraded":"2024-03-30T00:42:07.890610+0000","last_fullsized":"2024-03-30T00:42:07.890610+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:25.319805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876251+0000","last_change":"2024-03-30T00:42:07.876251+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876251+0000","last_undegraded":"2024-03-30T00:42:07.876251+0000","last_fullsized":"2024-03-30T00:42:07.876251+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:29.266461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890823+0000","last_change":"2024-03-30T00:42:07.890823+0000","last_active":"2024-03-30T00:42:07.890823+0000","last_peered":"2024-03-30T00:42:07.890823+0000","last_clean":"2024-03-30T00:42:07.890823+0000","last_became_active":"2024-03-30T00:42:07.888331+0000","last_became_peered":"2024-03-30T00:42:07.888331+0000","last_unstale":"2024-03-30T00:42:07.890823+0000","last_undegraded":"2024-03-30T00:42:07.890823+0000","last_fullsized":"2024-03-30T00:42:07.890823+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:48:23.842037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878821+0000","last_change":"2024-03-30T00:42:07.878821+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878821+0000","last_undegraded":"2024-03-30T00:42:07.878821+0000","last_fullsized":"2024-03-30T00:42:07.878821+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:03.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.884824+0000","last_change":"2024-03-30T00:42:07.884824+0000","last_active":"2024-03-30T00:42:07.884824+0000","last_peered":"2024-03-30T00:42:07.884824+0000","last_clean":"2024-03-30T00:42:07.884824+0000","last_became_active":"2024-03-30T00:42:07.884503+0000","last_became_peered":"2024-03-30T00:42:07.884503+0000","last_unstale":"2024-03-30T00:42:07.884824+0000","last_undegraded":"2024-03-30T00:42:07.884824+0000","last_fullsized":"2024-03-30T00:42:07.884824+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:18:21.768136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877433+0000","last_change":"2024-03-30T00:42:07.877433+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877433+0000","last_undegraded":"2024-03-30T00:42:07.877433+0000","last_fullsized":"2024-03-30T00:42:07.877433+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.892398+0000","last_change":"2024-03-30T00:42:07.892398+0000","last_active":"2024-03-30T00:42:07.892398+0000","last_peered":"2024-03-30T00:42:07.892398+0000","last_clean":"2024-03-30T00:42:07.892398+0000","last_became_active":"2024-03-30T00:42:07.892097+0000","last_became_peered":"2024-03-30T00:42:07.892097+0000","last_unstale":"2024-03-30T00:42:07.892398+0000","last_undegraded":"2024-03-30T00:42:07.892398+0000","last_fullsized":"2024-03-30T00:42:07.892398+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:30:49.532832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.874348+0000","last_change":"2024-03-30T00:42:07.874348+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874348+0000","last_undegraded":"2024-03-30T00:42:07.874348+0000","last_fullsized":"2024-03-30T00:42:07.874348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:59.202589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.897983+0000","last_change":"2024-03-30T00:42:07.897983+0000","last_active":"2024-03-30T00:42:07.897983+0000","last_peered":"2024-03-30T00:42:07.897983+0000","last_clean":"2024-03-30T00:42:07.897983+0000","last_became_active":"2024-03-30T00:42:07.897719+0000","last_became_peered":"2024-03-30T00:42:07.897719+0000","last_unstale":"2024-03-30T00:42:07.897983+0000","last_undegraded":"2024-03-30T00:42:07.897983+0000","last_fullsized":"2024-03-30T00:42:07.897983+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:55.868085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873753+0000","last_change":"2024-03-30T00:42:07.873753+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873753+0000","last_undegraded":"2024-03-30T00:42:07.873753+0000","last_fullsized":"2024-03-30T00:42:07.873753+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887534+0000","last_change":"2024-03-30T00:42:07.884431+0000","last_active":"2024-03-30T00:42:07.887534+0000","last_peered":"2024-03-30T00:42:07.887534+0000","last_clean":"2024-03-30T00:42:07.887534+0000","last_became_active":"2024-03-30T00:42:07.884172+0000","last_became_peered":"2024-03-30T00:42:07.884172+0000","last_unstale":"2024-03-30T00:42:07.887534+0000","last_undegraded":"2024-03-30T00:42:07.887534+0000","last_fullsized":"2024-03-30T00:42:07.887534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:24.488599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879706+0000","last_change":"2024-03-30T00:42:07.879706+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879706+0000","last_undegraded":"2024-03-30T00:42:07.879706+0000","last_fullsized":"2024-03-30T00:42:07.879706+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887038+0000","last_change":"2024-03-30T00:42:07.884001+0000","last_active":"2024-03-30T00:42:07.887038+0000","last_peered":"2024-03-30T00:42:07.887038+0000","last_clean":"2024-03-30T00:42:07.887038+0000","last_became_active":"2024-03-30T00:42:07.883690+0000","last_became_peered":"2024-03-30T00:42:07.883690+0000","last_unstale":"2024-03-30T00:42:07.887038+0000","last_undegraded":"2024-03-30T00:42:07.887038+0000","last_fullsized":"2024-03-30T00:42:07.887038+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:26.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873193+0000","last_change":"2024-03-30T00:42:07.873193+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873193+0000","last_undegraded":"2024-03-30T00:42:07.873193+0000","last_fullsized":"2024-03-30T00:42:07.873193+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:22.099119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893642+0000","last_change":"2024-03-30T00:42:07.887955+0000","last_active":"2024-03-30T00:42:07.893642+0000","last_peered":"2024-03-30T00:42:07.893642+0000","last_clean":"2024-03-30T00:42:07.893642+0000","last_became_active":"2024-03-30T00:42:07.887689+0000","last_became_peered":"2024-03-30T00:42:07.887689+0000","last_unstale":"2024-03-30T00:42:07.893642+0000","last_undegraded":"2024-03-30T00:42:07.893642+0000","last_fullsized":"2024-03-30T00:42:07.893642+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:17:21.438797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873054+0000","last_change":"2024-03-30T00:42:07.873054+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873054+0000","last_undegraded":"2024-03-30T00:42:07.873054+0000","last_fullsized":"2024-03-30T00:42:07.873054+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.887686+0000","last_change":"2024-03-30T00:42:07.884866+0000","last_active":"2024-03-30T00:42:07.887686+0000","last_peered":"2024-03-30T00:42:07.887686+0000","last_clean":"2024-03-30T00:42:07.887686+0000","last_became_active":"2024-03-30T00:42:07.884550+0000","last_became_peered":"2024-03-30T00:42:07.884550+0000","last_unstale":"2024-03-30T00:42:07.887686+0000","last_undegraded":"2024-03-30T00:42:07.887686+0000","last_fullsized":"2024-03-30T00:42:07.887686+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:10:19.003693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876922+0000","last_change":"2024-03-30T00:42:07.876922+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876922+0000","last_undegraded":"2024-03-30T00:42:07.876922+0000","last_fullsized":"2024-03-30T00:42:07.876922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.874579+0000","last_change":"2024-03-30T00:42:07.874579+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.874579+0000","last_undegraded":"2024-03-30T00:42:07.874579+0000","last_fullsized":"2024-03-30T00:42:07.874579+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:17.278798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881886+0000","last_change":"2024-03-30T00:42:07.881886+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881886+0000","last_undegraded":"2024-03-30T00:42:07.881886+0000","last_fullsized":"2024-03-30T00:42:07.881886+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:07.667668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891360+0000","last_change":"2024-03-30T00:42:07.889759+0000","last_active":"2024-03-30T00:42:07.891360+0000","last_peered":"2024-03-30T00:42:07.891360+0000","last_clean":"2024-03-30T00:42:07.891360+0000","last_became_active":"2024-03-30T00:42:07.888856+0000","last_became_peered":"2024-03-30T00:42:07.888856+0000","last_unstale":"2024-03-30T00:42:07.891360+0000","last_undegraded":"2024-03-30T00:42:07.891360+0000","last_fullsized":"2024-03-30T00:42:07.891360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:00:01.121929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873860+0000","last_change":"2024-03-30T00:42:07.873860+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873860+0000","last_undegraded":"2024-03-30T00:42:07.873860+0000","last_fullsized":"2024-03-30T00:42:07.873860+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.878493+0000","last_change":"2024-03-30T00:42:07.878493+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878493+0000","last_undegraded":"2024-03-30T00:42:07.878493+0000","last_fullsized":"2024-03-30T00:42:07.878493+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:48.023033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.881591+0000","last_change":"2024-03-30T00:42:07.881591+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.881591+0000","last_undegraded":"2024-03-30T00:42:07.881591+0000","last_fullsized":"2024-03-30T00:42:07.881591+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.896028+0000","last_change":"2024-03-30T00:42:07.896028+0000","last_active":"2024-03-30T00:42:07.896028+0000","last_peered":"2024-03-30T00:42:07.896028+0000","last_clean":"2024-03-30T00:42:07.896028+0000","last_became_active":"2024-03-30T00:42:07.895683+0000","last_became_peered":"2024-03-30T00:42:07.895683+0000","last_unstale":"2024-03-30T00:42:07.896028+0000","last_undegraded":"2024-03-30T00:42:07.896028+0000","last_fullsized":"2024-03-30T00:42:07.896028+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:37.405718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888318+0000","last_change":"2024-03-30T00:42:07.888318+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888318+0000","last_undegraded":"2024-03-30T00:42:07.888318+0000","last_fullsized":"2024-03-30T00:42:07.888318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.877016+0000","last_change":"2024-03-30T00:42:07.877016+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877016+0000","last_undegraded":"2024-03-30T00:42:07.877016+0000","last_fullsized":"2024-03-30T00:42:07.877016+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:58:41.813204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880083+0000","last_change":"2024-03-30T00:42:07.880083+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880083+0000","last_undegraded":"2024-03-30T00:42:07.880083+0000","last_fullsized":"2024-03-30T00:42:07.880083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:44.118291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.889095+0000","last_change":"2024-03-30T00:42:07.886045+0000","last_active":"2024-03-30T00:42:07.889095+0000","last_peered":"2024-03-30T00:42:07.889095+0000","last_clean":"2024-03-30T00:42:07.889095+0000","last_became_active":"2024-03-30T00:42:07.885765+0000","last_became_peered":"2024-03-30T00:42:07.885765+0000","last_unstale":"2024-03-30T00:42:07.889095+0000","last_undegraded":"2024-03-30T00:42:07.889095+0000","last_fullsized":"2024-03-30T00:42:07.889095+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:21.533654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.888542+0000","last_change":"2024-03-30T00:42:07.888542+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.888542+0000","last_undegraded":"2024-03-30T00:42:07.888542+0000","last_fullsized":"2024-03-30T00:42:07.888542+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:01.442897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.876507+0000","last_change":"2024-03-30T00:42:07.876507+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876507+0000","last_undegraded":"2024-03-30T00:42:07.876507+0000","last_fullsized":"2024-03-30T00:42:07.876507+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:16:59.791004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876783+0000","last_change":"2024-03-30T00:42:07.876783+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876783+0000","last_undegraded":"2024-03-30T00:42:07.876783+0000","last_fullsized":"2024-03-30T00:42:07.876783+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.898205+0000","last_change":"2024-03-30T00:42:07.898205+0000","last_active":"2024-03-30T00:42:07.898205+0000","last_peered":"2024-03-30T00:42:07.898205+0000","last_clean":"2024-03-30T00:42:07.898205+0000","last_became_active":"2024-03-30T00:42:07.897873+0000","last_became_peered":"2024-03-30T00:42:07.897873+0000","last_unstale":"2024-03-30T00:42:07.898205+0000","last_undegraded":"2024-03-30T00:42:07.898205+0000","last_fullsized":"2024-03-30T00:42:07.898205+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:28.263763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873297+0000","last_change":"2024-03-30T00:42:07.873297+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873297+0000","last_undegraded":"2024-03-30T00:42:07.873297+0000","last_fullsized":"2024-03-30T00:42:07.873297+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890926+0000","last_change":"2024-03-30T00:42:07.890926+0000","last_active":"2024-03-30T00:42:07.890926+0000","last_peered":"2024-03-30T00:42:07.890926+0000","last_clean":"2024-03-30T00:42:07.890926+0000","last_became_active":"2024-03-30T00:42:07.890676+0000","last_became_peered":"2024-03-30T00:42:07.890676+0000","last_unstale":"2024-03-30T00:42:07.890926+0000","last_undegraded":"2024-03-30T00:42:07.890926+0000","last_fullsized":"2024-03-30T00:42:07.890926+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:24:47.097376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.873770+0000","last_change":"2024-03-30T00:42:07.873770+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.873770+0000","last_undegraded":"2024-03-30T00:42:07.873770+0000","last_fullsized":"2024-03-30T00:42:07.873770+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890669+0000","last_change":"2024-03-30T00:42:07.888916+0000","last_active":"2024-03-30T00:42:07.890669+0000","last_peered":"2024-03-30T00:42:07.890669+0000","last_clean":"2024-03-30T00:42:07.890669+0000","last_became_active":"2024-03-30T00:42:07.888629+0000","last_became_peered":"2024-03-30T00:42:07.888629+0000","last_unstale":"2024-03-30T00:42:07.890669+0000","last_undegraded":"2024-03-30T00:42:07.890669+0000","last_fullsized":"2024-03-30T00:42:07.890669+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:03.807636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872778+0000","last_change":"2024-03-30T00:42:07.872778+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872778+0000","last_undegraded":"2024-03-30T00:42:07.872778+0000","last_fullsized":"2024-03-30T00:42:07.872778+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.890933+0000","last_change":"2024-03-30T00:42:07.889736+0000","last_active":"2024-03-30T00:42:07.890933+0000","last_peered":"2024-03-30T00:42:07.890933+0000","last_clean":"2024-03-30T00:42:07.890933+0000","last_became_active":"2024-03-30T00:42:07.889425+0000","last_became_peered":"2024-03-30T00:42:07.889425+0000","last_unstale":"2024-03-30T00:42:07.890933+0000","last_undegraded":"2024-03-30T00:42:07.890933+0000","last_fullsized":"2024-03-30T00:42:07.890933+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:41.441678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879997+0000","last_change":"2024-03-30T00:42:07.879997+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879997+0000","last_undegraded":"2024-03-30T00:42:07.879997+0000","last_fullsized":"2024-03-30T00:42:07.879997+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:42.969334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.884117+0000","last_change":"2024-03-30T00:42:07.884117+0000","last_active":"2024-03-30T00:42:07.884117+0000","last_peered":"2024-03-30T00:42:07.884117+0000","last_clean":"2024-03-30T00:42:07.884117+0000","last_became_active":"2024-03-30T00:42:07.883887+0000","last_became_peered":"2024-03-30T00:42:07.883887+0000","last_unstale":"2024-03-30T00:42:07.884117+0000","last_undegraded":"2024-03-30T00:42:07.884117+0000","last_fullsized":"2024-03-30T00:42:07.884117+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:18.896534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.880105+0000","last_change":"2024-03-30T00:42:07.880105+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.880105+0000","last_undegraded":"2024-03-30T00:42:07.880105+0000","last_fullsized":"2024-03-30T00:42:07.880105+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:00:02.124627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893214+0000","last_change":"2024-03-30T00:42:07.893214+0000","last_active":"2024-03-30T00:42:07.893214+0000","last_peered":"2024-03-30T00:42:07.893214+0000","last_clean":"2024-03-30T00:42:07.893214+0000","last_became_active":"2024-03-30T00:42:07.892814+0000","last_became_peered":"2024-03-30T00:42:07.892814+0000","last_unstale":"2024-03-30T00:42:07.893214+0000","last_undegraded":"2024-03-30T00:42:07.893214+0000","last_fullsized":"2024-03-30T00:42:07.893214+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:52.010424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.889733+0000","last_change":"2024-03-30T00:42:07.889733+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.889733+0000","last_undegraded":"2024-03-30T00:42:07.889733+0000","last_fullsized":"2024-03-30T00:42:07.889733+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.896209+0000","last_change":"2024-03-30T00:42:07.896209+0000","last_active":"2024-03-30T00:42:07.896209+0000","last_peered":"2024-03-30T00:42:07.896209+0000","last_clean":"2024-03-30T00:42:07.896209+0000","last_became_active":"2024-03-30T00:42:07.895871+0000","last_became_peered":"2024-03-30T00:42:07.895871+0000","last_unstale":"2024-03-30T00:42:07.896209+0000","last_undegraded":"2024-03-30T00:42:07.896209+0000","last_fullsized":"2024-03-30T00:42:07.896209+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:24.948608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872629+0000","last_change":"2024-03-30T00:42:07.872629+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872629+0000","last_undegraded":"2024-03-30T00:42:07.872629+0000","last_fullsized":"2024-03-30T00:42:07.872629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891909+0000","last_change":"2024-03-30T00:42:07.891909+0000","last_active":"2024-03-30T00:42:07.891909+0000","last_peered":"2024-03-30T00:42:07.891909+0000","last_clean":"2024-03-30T00:42:07.891909+0000","last_became_active":"2024-03-30T00:42:07.891660+0000","last_became_peered":"2024-03-30T00:42:07.891660+0000","last_unstale":"2024-03-30T00:42:07.891909+0000","last_undegraded":"2024-03-30T00:42:07.891909+0000","last_fullsized":"2024-03-30T00:42:07.891909+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:32.488321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876796+0000","last_change":"2024-03-30T00:42:07.876796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876796+0000","last_undegraded":"2024-03-30T00:42:07.876796+0000","last_fullsized":"2024-03-30T00:42:07.876796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:40.974026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.878452+0000","last_change":"2024-03-30T00:42:07.878452+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878452+0000","last_undegraded":"2024-03-30T00:42:07.878452+0000","last_fullsized":"2024-03-30T00:42:07.878452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:30.469686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.872796+0000","last_change":"2024-03-30T00:42:07.872796+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.872796+0000","last_undegraded":"2024-03-30T00:42:07.872796+0000","last_fullsized":"2024-03-30T00:42:07.872796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:30:50.535530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"creating+activating","last_fresh":"2024-03-30T00:42:07.876204+0000","last_change":"2024-03-30T00:42:07.876204+0000","last_active":"2024-03-30T00:42:06.857438+0000","last_peered":"2024-03-30T00:42:06.857438+0000","last_clean":"2024-03-30T00:42:06.857438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876204+0000","last_undegraded":"2024-03-30T00:42:07.876204+0000","last_fullsized":"2024-03-30T00:42:07.876204+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:02:33.633690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.876968+0000","last_change":"2024-03-30T00:42:07.876968+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.876968+0000","last_undegraded":"2024-03-30T00:42:07.876968+0000","last_fullsized":"2024-03-30T00:42:07.876968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:58.102992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.899036+0000","last_change":"2024-03-30T00:42:07.895012+0000","last_active":"2024-03-30T00:42:07.899036+0000","last_peered":"2024-03-30T00:42:07.899036+0000","last_clean":"2024-03-30T00:42:07.899036+0000","last_became_active":"2024-03-30T00:42:07.894725+0000","last_became_peered":"2024-03-30T00:42:07.894725+0000","last_unstale":"2024-03-30T00:42:07.899036+0000","last_undegraded":"2024-03-30T00:42:07.899036+0000","last_fullsized":"2024-03-30T00:42:07.899036+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:41:59.532828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.892109+0000","last_change":"2024-03-30T00:42:07.892109+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.892109+0000","last_undegraded":"2024-03-30T00:42:07.892109+0000","last_fullsized":"2024-03-30T00:42:07.892109+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893706+0000","last_change":"2024-03-30T00:42:07.893706+0000","last_active":"2024-03-30T00:42:07.893706+0000","last_peered":"2024-03-30T00:42:07.893706+0000","last_clean":"2024-03-30T00:42:07.893706+0000","last_became_active":"2024-03-30T00:42:07.893465+0000","last_became_peered":"2024-03-30T00:42:07.893465+0000","last_unstale":"2024-03-30T00:42:07.893706+0000","last_undegraded":"2024-03-30T00:42:07.893706+0000","last_fullsized":"2024-03-30T00:42:07.893706+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:41.966636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.878795+0000","last_change":"2024-03-30T00:42:07.878795+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.878795+0000","last_undegraded":"2024-03-30T00:42:07.878795+0000","last_fullsized":"2024-03-30T00:42:07.878795+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.893058+0000","last_change":"2024-03-30T00:42:07.893058+0000","last_active":"2024-03-30T00:42:07.893058+0000","last_peered":"2024-03-30T00:42:07.893058+0000","last_clean":"2024-03-30T00:42:07.893058+0000","last_became_active":"2024-03-30T00:42:07.892453+0000","last_became_peered":"2024-03-30T00:42:07.892453+0000","last_unstale":"2024-03-30T00:42:07.893058+0000","last_undegraded":"2024-03-30T00:42:07.893058+0000","last_fullsized":"2024-03-30T00:42:07.893058+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:27.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.890675+0000","last_change":"2024-03-30T00:42:07.890675+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.890675+0000","last_undegraded":"2024-03-30T00:42:07.890675+0000","last_fullsized":"2024-03-30T00:42:07.890675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891397+0000","last_change":"2024-03-30T00:42:07.889478+0000","last_active":"2024-03-30T00:42:07.891397+0000","last_peered":"2024-03-30T00:42:07.891397+0000","last_clean":"2024-03-30T00:42:07.891397+0000","last_became_active":"2024-03-30T00:42:07.889247+0000","last_became_peered":"2024-03-30T00:42:07.889247+0000","last_unstale":"2024-03-30T00:42:07.891397+0000","last_undegraded":"2024-03-30T00:42:07.891397+0000","last_fullsized":"2024-03-30T00:42:07.891397+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:01.336339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.877866+0000","last_change":"2024-03-30T00:42:07.877866+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.877866+0000","last_undegraded":"2024-03-30T00:42:07.877866+0000","last_fullsized":"2024-03-30T00:42:07.877866+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.891637+0000","last_change":"2024-03-30T00:42:07.891209+0000","last_active":"2024-03-30T00:42:07.891637+0000","last_peered":"2024-03-30T00:42:07.891637+0000","last_clean":"2024-03-30T00:42:07.891637+0000","last_became_active":"2024-03-30T00:42:07.890769+0000","last_became_peered":"2024-03-30T00:42:07.890769+0000","last_unstale":"2024-03-30T00:42:07.891637+0000","last_undegraded":"2024-03-30T00:42:07.891637+0000","last_fullsized":"2024-03-30T00:42:07.891637+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:21.096421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-30T00:42:07.879604+0000","last_change":"2024-03-30T00:42:07.879604+0000","last_active":"2024-03-30T00:42:07.860136+0000","last_peered":"2024-03-30T00:42:07.860136+0000","last_clean":"2024-03-30T00:42:07.860136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:42:07.879604+0000","last_undegraded":"2024-03-30T00:42:07.879604+0000","last_fullsized":"2024-03-30T00:42:07.879604+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:02:34.636388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-30T00:42:07.899549+0000","last_change":"2024-03-30T00:42:07.899549+0000","last_active":"2024-03-30T00:42:07.899549+0000","last_peered":"2024-03-30T00:42:07.899549+0000","last_clean":"2024-03-30T00:42:07.899549+0000","last_became_active":"2024-03-30T00:42:07.896786+0000","last_became_peered":"2024-03-30T00:42:07.896786+0000","last_unstale":"2024-03-30T00:42:07.899549+0000","last_undegraded":"2024-03-30T00:42:07.899549+0000","last_fullsized":"2024-03-30T00:42:07.899549+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:24.488599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-30T00:42:06.865006+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:42:06.865006+0000","last_peered":"2024-03-30T00:42:06.865006+0000","last_clean":"2024-03-30T00:42:06.865006+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:42:06.865006+0000","last_undegraded":"2024-03-30T00:42:06.865006+0000","last_fullsized":"2024-03-30T00:42:06.865006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1200128,"data_stored":854159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.77100000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.62},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.93300000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.626},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.51600000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.747}]},{"osd":2,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78100000000000003,"15min":0.78900000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.194,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.79400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.026,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0840000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82899999999999996},"min":{"1min":0.65100000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0429999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.873}]},{"osd":4,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63500000000000001,"15min":0.629},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.996,"5min":1.026,"15min":1.026},"last":0.97499999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.45400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.89200000000000002,"15min":0.93799999999999994},"min":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90100000000000002,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78300000000000003,"5min":1.47,"15min":1.47},"last":0.70299999999999996}]},{"osd":1,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.90100000000000002,"15min":0.95299999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76900000000000002,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.97299999999999998,"15min":1.0329999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.82399999999999995,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.69299999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.96999999999999997,"15min":1.018},"min":{"1min":0.48699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.82299999999999995,"5min":4.375,"15min":4.375},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.97199999999999998,"15min":1.024},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":4.2960000000000003,"15min":4.2960000000000003},"last":0.79000000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.98799999999999999,"15min":1.038},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97299999999999998,"5min":4.3399999999999999,"15min":4.3399999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.97499999999999998,"15min":1.034},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.73299999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96999999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.42399999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.095},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.78700000000000003,"15min":0.81000000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95099999999999996,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.86299999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.751,"15min":0.76600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77400000000000002,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.72299999999999998,"15min":0.73899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83599999999999997,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90600000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.70899999999999996,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79700000000000004,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.81000000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.79200000000000004,"15min":0.81200000000000006},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98499999999999999,"5min":1.7,"15min":1.7},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.80200000000000005,"5min":1.181,"15min":1.181},"last":0.83499999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63400000000000001,"15min":0.64400000000000002},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91500000000000004,"5min":1.615,"15min":1.615},"last":0.89100000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88100000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.877},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0700000000000001,"5min":1.569,"15min":1.569},"last":0.85199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76000000000000001,"15min":0.76300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89300000000000002,"5min":1.006,"15min":1.006},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.83899999999999997,"15min":0.84899999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.032,"5min":1.383,"15min":1.383},"last":0.92500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77900000000000003,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.44600000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51700000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71499999999999997,"15min":0.72299999999999998},"min":{"1min":0.50700000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.872,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54200000000000004}]},{"osd":2,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.91000000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.755,"15min":0.76300000000000001},"min":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96899999999999997,"5min":1.228,"15min":1.228},"last":0.58499999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.50700000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94899999999999995,"5min":1.27,"15min":1.27},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.751,"5min":0.78300000000000003,"15min":0.78900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91400000000000003,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84899999999999998,"5min":1.329,"15min":1.329},"last":0.438},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93500000000000005,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98899999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95799999999999996,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.497}]},{"osd":7,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.749,"15min":0.76300000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.069,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.747}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.56899999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.53200000000000003,"15min":0.55300000000000005},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.378},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59799999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.623,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.69299999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40500000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.76100000000000001,"15min":0.77600000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98299999999999998,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.872},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.77100000000000002,"15min":0.78500000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0069999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.90200000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.78200000000000003,"15min":0.79300000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.95199999999999996,"5min":1.125,"15min":1.125},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80600000000000005,"15min":0.82099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.024,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.81200000000000006,"15min":0.82699999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0389999999999999,"5min":1.226,"15min":1.226},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.79700000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0109999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.94899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.85099999999999998,"15min":0.86899999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.077,"5min":1.202,"15min":1.202},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.88400000000000001,"15min":0.90800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0740000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.91800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.313,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.627,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":4.0999999999999996},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.51100000000000001,"15min":0.52700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.56200000000000006,"5min":0.747,"15min":0.747},"last":1.9239999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.58199999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":3.1139999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53200000000000003,"15min":0.54400000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.63,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":2.4159999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.495,"15min":0.503},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.53400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76200000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":1.907}]},{"osd":4,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80300000000000005,"15min":0.81499999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.042,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.9470000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.127,"5min":1.528,"15min":1.528},"last":1.972}]},{"osd":5,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83099999999999996,"15min":0.84499999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1240000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":2.4569999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.80600000000000005,"15min":0.82299999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0580000000000001,"5min":1.131,"15min":1.131},"last":2.8039999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.81000000000000005,"15min":0.82099999999999995},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.081,"5min":1.151,"15min":1.151},"last":2.8849999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.179,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.867}]},{"osd":7,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.85999999999999999,"15min":0.88200000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.01,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":2.3690000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0700000000000001,"5min":1.228,"15min":1.228},"last":2.851}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":263791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61299999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.46000000000000002,"15min":0.46899999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55700000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.498}]},{"osd":3,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66200000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.626,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92000000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.83199999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77600000000000002,"15min":0.78000000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93400000000000005,"5min":1.097,"15min":1.097},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.90100000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78900000000000003,"15min":0.79500000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95199999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92800000000000005,"5min":0.996,"15min":0.996},"last":0.93400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1200128,"data_stored":854159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.29299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58499999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.61299999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66100000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.623},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":1.006}]},{"osd":3,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71499999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.96899999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79100000000000004,"15min":0.79400000000000004},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.008,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.234},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79900000000000004,"15min":0.80000000000000004},"min":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1279999999999999,"5min":1.153,"15min":1.153},"last":1.252}]},{"osd":5,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.621,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0489999999999999,"5min":1.22,"15min":1.22},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.92300000000000004,"5min":1.462,"15min":1.462},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78900000000000003,"15min":0.79800000000000004},"min":{"1min":0.58299999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.091,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.56299999999999994,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1,"5min":1.085,"15min":1.085},"last":1.2}]},{"osd":7,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.91500000000000004,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.351},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96899999999999997,"5min":1.556,"15min":1.556},"last":1.085}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:42:16.162 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:42:16.166 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:42:17.174 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:42:17.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.316+0000 7f7280200640 1 Processor -- start 2024-03-30T00:42:17.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.316+0000 7f7280200640 1 -- start start 2024-03-30T00:42:17.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.316+0000 7f7280200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815ca10 0x7f727806ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.316+0000 7f7280200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727806e7d0 con 0x7f72780ae4e0 2024-03-30T00:42:17.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.316+0000 7f7280200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727806f1a0 con 0x7f72780ce820 2024-03-30T00:42:17.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.316+0000 7f7280200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72780b1f70 con 0x7f727815ca10 2024-03-30T00:42:17.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.316+0000 7f727ce00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f72780ae4e0 0x7f72780cb620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55684/0 (socket says 172.21.15.66:55684) 2024-03-30T00:42:17.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.316+0000 7f727ce00640 1 -- 172.21.15.66:0/3972033403 learned_addr learned my addr 172.21.15.66:0/3972033403 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:17.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.316+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453218692 0 0) 0x7f727806e7d0 con 0x7f72780ae4e0 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.316+0000 7f727d800640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815ca10 0x7f727806ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.316+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7260003530 con 0x7f72780ae4e0 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 691642567 0 0) 0x7f7260003530 con 0x7f72780ae4e0 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815ca10 msgr2=0x7f727806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815ca10 0x7f727806ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 >> v1:172.21.15.156:6789/0 conn(0x7f72780ce820 legacy=0x7f7278160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727815cdf0 con 0x7f72780ae4e0 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f7264001680 con 0x7f72780ae4e0 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 >> v1:172.21.15.66:6789/0 conn(0x7f72780ae4e0 legacy=0x7f72780cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815ca10 0x7f72600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7260003ef0 0x7f72600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7260004d10 0x7f7260005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72780b1f70 con 0x7f7260004d10 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726000ba40 con 0x7f7260003ef0 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726000bc00 con 0x7f727815ca10 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f727d800640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815ca10 0x7f72600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f727e200640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7260004d10 0x7f7260005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f727ce00640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7260003ef0 0x7f72600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f727e200640 1 -- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815ca10 msgr2=0x7f72600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f727e200640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815ca10 0x7f72600039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f727e200640 1 -- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7260003ef0 msgr2=0x7f72600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f727e200640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7260003ef0 0x7f72600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f727e200640 1 -- 172.21.15.66:0/3972033403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7260003530 con 0x7f7260004d10 2024-03-30T00:42:17.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.320+0000 7f727d800640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815ca10 0x7f72600039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:42:17.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f727e200640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7260004d10 0x7f7260005190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7270001f60 tx=0x7f727001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ef61540402d08d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:17.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72700a5770 con 0x7f7260004d10 2024-03-30T00:42:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72700a58d0 con 0x7f7260004d10 2024-03-30T00:42:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7277e00640 1 -- 172.21.15.66:0/3972033403 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72700a5c80 con 0x7f7260004d10 2024-03-30T00:42:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 -- 172.21.15.66:0/3972033403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7260004d10 msgr2=0x7f7260005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 --2- 172.21.15.66:0/3972033403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7260004d10 0x7f7260005190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7270001f60 tx=0x7f727001c880 comp rx=0 tx=0).stop 2024-03-30T00:42:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f727e200640 1 -- 172.21.15.66:0/3972033403 reap_dead start 2024-03-30T00:42:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 -- 172.21.15.66:0/3972033403 shutdown_connections 2024-03-30T00:42:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 -- 172.21.15.66:0/3972033403 >> 172.21.15.66:0/3972033403 conn(0x7f7278098a20 msgr2=0x7f72780b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 -- 172.21.15.66:0/3972033403 shutdown_connections 2024-03-30T00:42:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 -- 172.21.15.66:0/3972033403 wait complete. 2024-03-30T00:42:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 Processor -- start 2024-03-30T00:42:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 -- start start 2024-03-30T00:42:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72781f1a50 0x7f7278157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815d460 0x7f72780ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72780ad960 0x7f7278158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72780adde0 con 0x7f72781f1a50 2024-03-30T00:42:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7278154900 con 0x7f72780ad960 2024-03-30T00:42:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f7280200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727815da30 con 0x7f727815d460 2024-03-30T00:42:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f727d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72781f1a50 0x7f7278157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f727d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72781f1a50 0x7f7278157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36390/0 (socket says 172.21.15.66:36390) 2024-03-30T00:42:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.324+0000 7f727d800640 1 -- 172.21.15.66:0/3786077082 learned_addr learned my addr 172.21.15.66:0/3786077082 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f727e200640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72780ad960 0x7f7278158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f727ce00640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815d460 0x7f72780ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f727d800640 1 -- 172.21.15.66:0/3786077082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815d460 msgr2=0x7f72780ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f727d800640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815d460 0x7f72780ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f727d800640 1 -- 172.21.15.66:0/3786077082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72780ad960 msgr2=0x7f7278158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f727d800640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72780ad960 0x7f7278158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f727d800640 1 -- 172.21.15.66:0/3786077082 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727815dd10 con 0x7f72781f1a50 2024-03-30T00:42:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f727d800640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72781f1a50 0x7f7278157560 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f726c011ca0 tx=0x7f726c00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:17.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f7275600640 1 -- 172.21.15.66:0/3786077082 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c00fd20 con 0x7f72781f1a50 2024-03-30T00:42:17.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f7275600640 1 -- 172.21.15.66:0/3786077082 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c00fe80 con 0x7f72781f1a50 2024-03-30T00:42:17.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f7275600640 1 -- 172.21.15.66:0/3786077082 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c004260 con 0x7f72781f1a50 2024-03-30T00:42:17.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f7280200640 1 -- 172.21.15.66:0/3786077082 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727815dff0 con 0x7f72781f1a50 2024-03-30T00:42:17.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f7280200640 1 -- 172.21.15.66:0/3786077082 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72780b3300 con 0x7f72781f1a50 2024-03-30T00:42:17.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f7280200640 1 -- 172.21.15.66:0/3786077082 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7278066c30 con 0x7f72781f1a50 2024-03-30T00:42:17.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f7275600640 1 -- 172.21.15.66:0/3786077082 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f726c0043c0 con 0x7f72781f1a50 2024-03-30T00:42:17.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f7275600640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7258066690 0x7f7258068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.328+0000 7f7275600640 1 -- 172.21.15.66:0/3786077082 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f726c084b00 con 0x7f72781f1a50 2024-03-30T00:42:17.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.332+0000 7f727ce00640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7258066690 0x7f7258068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:17.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.332+0000 7f727ce00640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7258066690 0x7f7258068b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f72640078f0 tx=0x7f7264001b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:17.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.336+0000 7f7275600640 1 -- 172.21.15.66:0/3786077082 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f726c086e60 con 0x7f72781f1a50 2024-03-30T00:42:17.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.492+0000 7f7280200640 1 -- 172.21.15.66:0/3786077082 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72780b1c90 con 0x7f7258066690 2024-03-30T00:42:17.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.500+0000 7f7275600640 1 -- 172.21.15.66:0/3786077082 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289253 (secure 0 0 0) 0x7f7264001660 con 0x7f7258066690 2024-03-30T00:42:17.513 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:17.515 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:42:17.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 -- 172.21.15.66:0/3786077082 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7258066690 msgr2=0x7f7258068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7258066690 0x7f7258068b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f72640078f0 tx=0x7f7264001b80 comp rx=0 tx=0).stop 2024-03-30T00:42:17.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 -- 172.21.15.66:0/3786077082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72781f1a50 msgr2=0x7f7278157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72781f1a50 0x7f7278157560 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f726c011ca0 tx=0x7f726c00f610 comp rx=0 tx=0).stop 2024-03-30T00:42:17.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 -- 172.21.15.66:0/3786077082 shutdown_connections 2024-03-30T00:42:17.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72780ad960 0x7f7278158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f727815d460 0x7f72780ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7258066690 0x7f7258068b50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 --2- 172.21.15.66:0/3786077082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72781f1a50 0x7f7278157560 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 -- 172.21.15.66:0/3786077082 >> 172.21.15.66:0/3786077082 conn(0x7f7278098a20 msgr2=0x7f727815a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:17.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 -- 172.21.15.66:0/3786077082 shutdown_connections 2024-03-30T00:42:17.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.504+0000 7f7280200640 1 -- 172.21.15.66:0/3786077082 wait complete. 2024-03-30T00:42:17.534 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-03-30T00:42:16.880763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238804,"kb_used_data":6248,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754735916,"statfs":{"total":773094113280,"available":772849577984,"internally_reserved":0,"allocated":6397952,"data_stored":3458268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.607965"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.296896+0000","last_change":"2024-03-30T00:42:08.289714+0000","last_active":"2024-03-30T00:42:08.296896+0000","last_peered":"2024-03-30T00:42:08.296896+0000","last_clean":"2024-03-30T00:42:08.296896+0000","last_became_active":"2024-03-30T00:42:08.289473+0000","last_became_peered":"2024-03-30T00:42:08.289473+0000","last_unstale":"2024-03-30T00:42:08.296896+0000","last_undegraded":"2024-03-30T00:42:08.296896+0000","last_fullsized":"2024-03-30T00:42:08.296896+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.296751+0000","last_change":"2024-03-30T00:42:08.288995+0000","last_active":"2024-03-30T00:42:08.296751+0000","last_peered":"2024-03-30T00:42:08.296751+0000","last_clean":"2024-03-30T00:42:08.296751+0000","last_became_active":"2024-03-30T00:42:08.288479+0000","last_became_peered":"2024-03-30T00:42:08.288479+0000","last_unstale":"2024-03-30T00:42:08.296751+0000","last_undegraded":"2024-03-30T00:42:08.296751+0000","last_fullsized":"2024-03-30T00:42:08.296751+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:38.408416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.300767+0000","last_change":"2024-03-30T00:42:08.293172+0000","last_active":"2024-03-30T00:42:08.300767+0000","last_peered":"2024-03-30T00:42:08.300767+0000","last_clean":"2024-03-30T00:42:08.300767+0000","last_became_active":"2024-03-30T00:42:08.292864+0000","last_became_peered":"2024-03-30T00:42:08.292864+0000","last_unstale":"2024-03-30T00:42:08.300767+0000","last_undegraded":"2024-03-30T00:42:08.300767+0000","last_fullsized":"2024-03-30T00:42:08.300767+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.307530+0000","last_change":"2024-03-30T00:42:08.306659+0000","last_active":"2024-03-30T00:42:08.307530+0000","last_peered":"2024-03-30T00:42:08.307530+0000","last_clean":"2024-03-30T00:42:08.307530+0000","last_became_active":"2024-03-30T00:42:08.306430+0000","last_became_peered":"2024-03-30T00:42:08.306430+0000","last_unstale":"2024-03-30T00:42:08.307530+0000","last_undegraded":"2024-03-30T00:42:08.307530+0000","last_fullsized":"2024-03-30T00:42:08.307530+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.303657+0000","last_change":"2024-03-30T00:42:08.297016+0000","last_active":"2024-03-30T00:42:08.303657+0000","last_peered":"2024-03-30T00:42:08.303657+0000","last_clean":"2024-03-30T00:42:08.303657+0000","last_became_active":"2024-03-30T00:42:08.296092+0000","last_became_peered":"2024-03-30T00:42:08.296092+0000","last_unstale":"2024-03-30T00:42:08.303657+0000","last_undegraded":"2024-03-30T00:42:08.303657+0000","last_fullsized":"2024-03-30T00:42:08.303657+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.287883+0000","last_change":"2024-03-30T00:42:08.281506+0000","last_active":"2024-03-30T00:42:08.287883+0000","last_peered":"2024-03-30T00:42:08.287883+0000","last_clean":"2024-03-30T00:42:08.287883+0000","last_became_active":"2024-03-30T00:42:08.280998+0000","last_became_peered":"2024-03-30T00:42:08.280998+0000","last_unstale":"2024-03-30T00:42:08.287883+0000","last_undegraded":"2024-03-30T00:42:08.287883+0000","last_fullsized":"2024-03-30T00:42:08.287883+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:22.099119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.301283+0000","last_change":"2024-03-30T00:42:08.293914+0000","last_active":"2024-03-30T00:42:08.301283+0000","last_peered":"2024-03-30T00:42:08.301283+0000","last_clean":"2024-03-30T00:42:08.301283+0000","last_became_active":"2024-03-30T00:42:08.293689+0000","last_became_peered":"2024-03-30T00:42:08.293689+0000","last_unstale":"2024-03-30T00:42:08.301283+0000","last_undegraded":"2024-03-30T00:42:08.301283+0000","last_fullsized":"2024-03-30T00:42:08.301283+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:09.745566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.315615+0000","last_change":"2024-03-30T00:42:08.308834+0000","last_active":"2024-03-30T00:42:08.315615+0000","last_peered":"2024-03-30T00:42:08.315615+0000","last_clean":"2024-03-30T00:42:08.315615+0000","last_became_active":"2024-03-30T00:42:08.308541+0000","last_became_peered":"2024-03-30T00:42:08.308541+0000","last_unstale":"2024-03-30T00:42:08.315615+0000","last_undegraded":"2024-03-30T00:42:08.315615+0000","last_fullsized":"2024-03-30T00:42:08.315615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:01.749453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.314685+0000","last_change":"2024-03-30T00:42:08.303146+0000","last_active":"2024-03-30T00:42:08.314685+0000","last_peered":"2024-03-30T00:42:08.314685+0000","last_clean":"2024-03-30T00:42:08.314685+0000","last_became_active":"2024-03-30T00:42:08.302783+0000","last_became_peered":"2024-03-30T00:42:08.302783+0000","last_unstale":"2024-03-30T00:42:08.314685+0000","last_undegraded":"2024-03-30T00:42:08.314685+0000","last_fullsized":"2024-03-30T00:42:08.314685+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.297357+0000","last_change":"2024-03-30T00:42:08.290870+0000","last_active":"2024-03-30T00:42:08.297357+0000","last_peered":"2024-03-30T00:42:08.297357+0000","last_clean":"2024-03-30T00:42:08.297357+0000","last_became_active":"2024-03-30T00:42:08.290574+0000","last_became_peered":"2024-03-30T00:42:08.290574+0000","last_unstale":"2024-03-30T00:42:08.297357+0000","last_undegraded":"2024-03-30T00:42:08.297357+0000","last_fullsized":"2024-03-30T00:42:08.297357+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.295641+0000","last_change":"2024-03-30T00:42:08.293151+0000","last_active":"2024-03-30T00:42:08.295641+0000","last_peered":"2024-03-30T00:42:08.295641+0000","last_clean":"2024-03-30T00:42:08.295641+0000","last_became_active":"2024-03-30T00:42:08.292897+0000","last_became_peered":"2024-03-30T00:42:08.292897+0000","last_unstale":"2024-03-30T00:42:08.295641+0000","last_undegraded":"2024-03-30T00:42:08.295641+0000","last_fullsized":"2024-03-30T00:42:08.295641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:04.810334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.314859+0000","last_change":"2024-03-30T00:42:08.306495+0000","last_active":"2024-03-30T00:42:08.314859+0000","last_peered":"2024-03-30T00:42:08.314859+0000","last_clean":"2024-03-30T00:42:08.314859+0000","last_became_active":"2024-03-30T00:42:08.303606+0000","last_became_peered":"2024-03-30T00:42:08.303606+0000","last_unstale":"2024-03-30T00:42:08.314859+0000","last_undegraded":"2024-03-30T00:42:08.314859+0000","last_fullsized":"2024-03-30T00:42:08.314859+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.301677+0000","last_change":"2024-03-30T00:42:08.291428+0000","last_active":"2024-03-30T00:42:08.301677+0000","last_peered":"2024-03-30T00:42:08.301677+0000","last_clean":"2024-03-30T00:42:08.301677+0000","last_became_active":"2024-03-30T00:42:08.290592+0000","last_became_peered":"2024-03-30T00:42:08.290592+0000","last_unstale":"2024-03-30T00:42:08.301677+0000","last_undegraded":"2024-03-30T00:42:08.301677+0000","last_fullsized":"2024-03-30T00:42:08.301677+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:03.287560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.302977+0000","last_change":"2024-03-30T00:42:08.298794+0000","last_active":"2024-03-30T00:42:08.302977+0000","last_peered":"2024-03-30T00:42:08.302977+0000","last_clean":"2024-03-30T00:42:08.302977+0000","last_became_active":"2024-03-30T00:42:08.298530+0000","last_became_peered":"2024-03-30T00:42:08.298530+0000","last_unstale":"2024-03-30T00:42:08.302977+0000","last_undegraded":"2024-03-30T00:42:08.302977+0000","last_fullsized":"2024-03-30T00:42:08.302977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:54:01.171986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.301401+0000","last_change":"2024-03-30T00:42:08.299295+0000","last_active":"2024-03-30T00:42:08.301401+0000","last_peered":"2024-03-30T00:42:08.301401+0000","last_clean":"2024-03-30T00:42:08.301401+0000","last_became_active":"2024-03-30T00:42:08.298645+0000","last_became_peered":"2024-03-30T00:42:08.298645+0000","last_unstale":"2024-03-30T00:42:08.301401+0000","last_undegraded":"2024-03-30T00:42:08.301401+0000","last_fullsized":"2024-03-30T00:42:08.301401+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:56.870783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.300976+0000","last_change":"2024-03-30T00:42:08.291505+0000","last_active":"2024-03-30T00:42:08.300976+0000","last_peered":"2024-03-30T00:42:08.300976+0000","last_clean":"2024-03-30T00:42:08.300976+0000","last_became_active":"2024-03-30T00:42:08.291105+0000","last_became_peered":"2024-03-30T00:42:08.291105+0000","last_unstale":"2024-03-30T00:42:08.300976+0000","last_undegraded":"2024-03-30T00:42:08.300976+0000","last_fullsized":"2024-03-30T00:42:08.300976+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:24.904667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.292069+0000","last_change":"2024-03-30T00:42:08.287728+0000","last_active":"2024-03-30T00:42:08.292069+0000","last_peered":"2024-03-30T00:42:08.292069+0000","last_clean":"2024-03-30T00:42:08.292069+0000","last_became_active":"2024-03-30T00:42:08.287486+0000","last_became_peered":"2024-03-30T00:42:08.287486+0000","last_unstale":"2024-03-30T00:42:08.292069+0000","last_undegraded":"2024-03-30T00:42:08.292069+0000","last_fullsized":"2024-03-30T00:42:08.292069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:41:31.279358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.296771+0000","last_change":"2024-03-30T00:42:08.289153+0000","last_active":"2024-03-30T00:42:08.296771+0000","last_peered":"2024-03-30T00:42:08.296771+0000","last_clean":"2024-03-30T00:42:08.296771+0000","last_became_active":"2024-03-30T00:42:08.288886+0000","last_became_peered":"2024-03-30T00:42:08.288886+0000","last_unstale":"2024-03-30T00:42:08.296771+0000","last_undegraded":"2024-03-30T00:42:08.296771+0000","last_fullsized":"2024-03-30T00:42:08.296771+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.305268+0000","last_change":"2024-03-30T00:42:08.300883+0000","last_active":"2024-03-30T00:42:08.305268+0000","last_peered":"2024-03-30T00:42:08.305268+0000","last_clean":"2024-03-30T00:42:08.305268+0000","last_became_active":"2024-03-30T00:42:08.300593+0000","last_became_peered":"2024-03-30T00:42:08.300593+0000","last_unstale":"2024-03-30T00:42:08.305268+0000","last_undegraded":"2024-03-30T00:42:08.305268+0000","last_fullsized":"2024-03-30T00:42:08.305268+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.303882+0000","last_change":"2024-03-30T00:42:08.301873+0000","last_active":"2024-03-30T00:42:08.303882+0000","last_peered":"2024-03-30T00:42:08.303882+0000","last_clean":"2024-03-30T00:42:08.303882+0000","last_became_active":"2024-03-30T00:42:08.301595+0000","last_became_peered":"2024-03-30T00:42:08.301595+0000","last_unstale":"2024-03-30T00:42:08.303882+0000","last_undegraded":"2024-03-30T00:42:08.303882+0000","last_fullsized":"2024-03-30T00:42:08.303882+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:10:20.006391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.302845+0000","last_change":"2024-03-30T00:42:08.299126+0000","last_active":"2024-03-30T00:42:08.302845+0000","last_peered":"2024-03-30T00:42:08.302845+0000","last_clean":"2024-03-30T00:42:08.302845+0000","last_became_active":"2024-03-30T00:42:08.298719+0000","last_became_peered":"2024-03-30T00:42:08.298719+0000","last_unstale":"2024-03-30T00:42:08.302845+0000","last_undegraded":"2024-03-30T00:42:08.302845+0000","last_fullsized":"2024-03-30T00:42:08.302845+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:19.899232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.314650+0000","last_change":"2024-03-30T00:42:08.303825+0000","last_active":"2024-03-30T00:42:08.314650+0000","last_peered":"2024-03-30T00:42:08.314650+0000","last_clean":"2024-03-30T00:42:08.314650+0000","last_became_active":"2024-03-30T00:42:08.303565+0000","last_became_peered":"2024-03-30T00:42:08.303565+0000","last_unstale":"2024-03-30T00:42:08.314650+0000","last_undegraded":"2024-03-30T00:42:08.314650+0000","last_fullsized":"2024-03-30T00:42:08.314650+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.295842+0000","last_change":"2024-03-30T00:42:08.293202+0000","last_active":"2024-03-30T00:42:08.295842+0000","last_peered":"2024-03-30T00:42:08.295842+0000","last_clean":"2024-03-30T00:42:08.295842+0000","last_became_active":"2024-03-30T00:42:08.292951+0000","last_became_peered":"2024-03-30T00:42:08.292951+0000","last_unstale":"2024-03-30T00:42:08.295842+0000","last_undegraded":"2024-03-30T00:42:08.295842+0000","last_fullsized":"2024-03-30T00:42:08.295842+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:26.322503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.303515+0000","last_change":"2024-03-30T00:42:08.296600+0000","last_active":"2024-03-30T00:42:08.303515+0000","last_peered":"2024-03-30T00:42:08.303515+0000","last_clean":"2024-03-30T00:42:08.303515+0000","last_became_active":"2024-03-30T00:42:08.296333+0000","last_became_peered":"2024-03-30T00:42:08.296333+0000","last_unstale":"2024-03-30T00:42:08.303515+0000","last_undegraded":"2024-03-30T00:42:08.303515+0000","last_fullsized":"2024-03-30T00:42:08.303515+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:34:13.403177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.311609+0000","last_change":"2024-03-30T00:42:08.305115+0000","last_active":"2024-03-30T00:42:08.311609+0000","last_peered":"2024-03-30T00:42:08.311609+0000","last_clean":"2024-03-30T00:42:08.311609+0000","last_became_active":"2024-03-30T00:42:08.304398+0000","last_became_peered":"2024-03-30T00:42:08.304398+0000","last_unstale":"2024-03-30T00:42:08.311609+0000","last_undegraded":"2024-03-30T00:42:08.311609+0000","last_fullsized":"2024-03-30T00:42:08.311609+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:58.088392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.314951+0000","last_change":"2024-03-30T00:42:08.306989+0000","last_active":"2024-03-30T00:42:08.314951+0000","last_peered":"2024-03-30T00:42:08.314951+0000","last_clean":"2024-03-30T00:42:08.314951+0000","last_became_active":"2024-03-30T00:42:08.306560+0000","last_became_peered":"2024-03-30T00:42:08.306560+0000","last_unstale":"2024-03-30T00:42:08.314951+0000","last_undegraded":"2024-03-30T00:42:08.314951+0000","last_fullsized":"2024-03-30T00:42:08.314951+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.297928+0000","last_change":"2024-03-30T00:42:08.290385+0000","last_active":"2024-03-30T00:42:08.297928+0000","last_peered":"2024-03-30T00:42:08.297928+0000","last_clean":"2024-03-30T00:42:08.297928+0000","last_became_active":"2024-03-30T00:42:08.290104+0000","last_became_peered":"2024-03-30T00:42:08.290104+0000","last_unstale":"2024-03-30T00:42:08.297928+0000","last_undegraded":"2024-03-30T00:42:08.297928+0000","last_fullsized":"2024-03-30T00:42:08.297928+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:17:22.441495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.301763+0000","last_change":"2024-03-30T00:42:08.297805+0000","last_active":"2024-03-30T00:42:08.301763+0000","last_peered":"2024-03-30T00:42:08.301763+0000","last_clean":"2024-03-30T00:42:08.301763+0000","last_became_active":"2024-03-30T00:42:08.297175+0000","last_became_peered":"2024-03-30T00:42:08.297175+0000","last_unstale":"2024-03-30T00:42:08.301763+0000","last_undegraded":"2024-03-30T00:42:08.301763+0000","last_fullsized":"2024-03-30T00:42:08.301763+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:20.469591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.303099+0000","last_change":"2024-03-30T00:42:08.299093+0000","last_active":"2024-03-30T00:42:08.303099+0000","last_peered":"2024-03-30T00:42:08.303099+0000","last_clean":"2024-03-30T00:42:08.303099+0000","last_became_active":"2024-03-30T00:42:08.298848+0000","last_became_peered":"2024-03-30T00:42:08.298848+0000","last_unstale":"2024-03-30T00:42:08.303099+0000","last_undegraded":"2024-03-30T00:42:08.303099+0000","last_fullsized":"2024-03-30T00:42:08.303099+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.300109+0000","last_change":"2024-03-30T00:42:08.293219+0000","last_active":"2024-03-30T00:42:08.300109+0000","last_peered":"2024-03-30T00:42:08.300109+0000","last_clean":"2024-03-30T00:42:08.300109+0000","last_became_active":"2024-03-30T00:42:08.287661+0000","last_became_peered":"2024-03-30T00:42:08.287661+0000","last_unstale":"2024-03-30T00:42:08.300109+0000","last_undegraded":"2024-03-30T00:42:08.300109+0000","last_fullsized":"2024-03-30T00:42:08.300109+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:18.281496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.303454+0000","last_change":"2024-03-30T00:42:08.297873+0000","last_active":"2024-03-30T00:42:08.303454+0000","last_peered":"2024-03-30T00:42:08.303454+0000","last_clean":"2024-03-30T00:42:08.303454+0000","last_became_active":"2024-03-30T00:42:08.297617+0000","last_became_peered":"2024-03-30T00:42:08.297617+0000","last_unstale":"2024-03-30T00:42:08.303454+0000","last_undegraded":"2024-03-30T00:42:08.303454+0000","last_fullsized":"2024-03-30T00:42:08.303454+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:49.025731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.311896+0000","last_change":"2024-03-30T00:42:08.311305+0000","last_active":"2024-03-30T00:42:08.311896+0000","last_peered":"2024-03-30T00:42:08.311896+0000","last_clean":"2024-03-30T00:42:08.311896+0000","last_became_active":"2024-03-30T00:42:08.310751+0000","last_became_peered":"2024-03-30T00:42:08.310751+0000","last_unstale":"2024-03-30T00:42:08.311896+0000","last_undegraded":"2024-03-30T00:42:08.311896+0000","last_fullsized":"2024-03-30T00:42:08.311896+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:43.809122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.289252+0000","last_change":"2024-03-30T00:42:08.283920+0000","last_active":"2024-03-30T00:42:08.289252+0000","last_peered":"2024-03-30T00:42:08.289252+0000","last_clean":"2024-03-30T00:42:08.289252+0000","last_became_active":"2024-03-30T00:42:08.283601+0000","last_became_peered":"2024-03-30T00:42:08.283601+0000","last_unstale":"2024-03-30T00:42:08.289252+0000","last_undegraded":"2024-03-30T00:42:08.289252+0000","last_fullsized":"2024-03-30T00:42:08.289252+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:17:22.441495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.304325+0000","last_change":"2024-03-30T00:42:07.887914+0000","last_active":"2024-03-30T00:42:08.304325+0000","last_peered":"2024-03-30T00:42:08.304325+0000","last_clean":"2024-03-30T00:42:08.304325+0000","last_became_active":"2024-03-30T00:42:07.887626+0000","last_became_peered":"2024-03-30T00:42:07.887626+0000","last_unstale":"2024-03-30T00:42:08.304325+0000","last_undegraded":"2024-03-30T00:42:08.304325+0000","last_fullsized":"2024-03-30T00:42:08.304325+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:25.319805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.298245+0000","last_change":"2024-03-30T00:42:08.289095+0000","last_active":"2024-03-30T00:42:08.298245+0000","last_peered":"2024-03-30T00:42:08.298245+0000","last_clean":"2024-03-30T00:42:08.298245+0000","last_became_active":"2024-03-30T00:42:08.288785+0000","last_became_peered":"2024-03-30T00:42:08.288785+0000","last_unstale":"2024-03-30T00:42:08.298245+0000","last_undegraded":"2024-03-30T00:42:08.298245+0000","last_fullsized":"2024-03-30T00:42:08.298245+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:29.266461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.309064+0000","last_change":"2024-03-30T00:42:07.890823+0000","last_active":"2024-03-30T00:42:08.309064+0000","last_peered":"2024-03-30T00:42:08.309064+0000","last_clean":"2024-03-30T00:42:08.309064+0000","last_became_active":"2024-03-30T00:42:07.888331+0000","last_became_peered":"2024-03-30T00:42:07.888331+0000","last_unstale":"2024-03-30T00:42:08.309064+0000","last_undegraded":"2024-03-30T00:42:08.309064+0000","last_fullsized":"2024-03-30T00:42:08.309064+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:48:23.842037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.305183+0000","last_change":"2024-03-30T00:42:08.303064+0000","last_active":"2024-03-30T00:42:08.305183+0000","last_peered":"2024-03-30T00:42:08.305183+0000","last_clean":"2024-03-30T00:42:08.305183+0000","last_became_active":"2024-03-30T00:42:08.302810+0000","last_became_peered":"2024-03-30T00:42:08.302810+0000","last_unstale":"2024-03-30T00:42:08.305183+0000","last_undegraded":"2024-03-30T00:42:08.305183+0000","last_fullsized":"2024-03-30T00:42:08.305183+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:03.872764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.303339+0000","last_change":"2024-03-30T00:42:07.884824+0000","last_active":"2024-03-30T00:42:08.303339+0000","last_peered":"2024-03-30T00:42:08.303339+0000","last_clean":"2024-03-30T00:42:08.303339+0000","last_became_active":"2024-03-30T00:42:07.884503+0000","last_became_peered":"2024-03-30T00:42:07.884503+0000","last_unstale":"2024-03-30T00:42:08.303339+0000","last_undegraded":"2024-03-30T00:42:08.303339+0000","last_fullsized":"2024-03-30T00:42:08.303339+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:18:21.768136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.314983+0000","last_change":"2024-03-30T00:42:08.306859+0000","last_active":"2024-03-30T00:42:08.314983+0000","last_peered":"2024-03-30T00:42:08.314983+0000","last_clean":"2024-03-30T00:42:08.314983+0000","last_became_active":"2024-03-30T00:42:08.306192+0000","last_became_peered":"2024-03-30T00:42:08.306192+0000","last_unstale":"2024-03-30T00:42:08.314983+0000","last_undegraded":"2024-03-30T00:42:08.314983+0000","last_fullsized":"2024-03-30T00:42:08.314983+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:16.330812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.303978+0000","last_change":"2024-03-30T00:42:07.892398+0000","last_active":"2024-03-30T00:42:08.303978+0000","last_peered":"2024-03-30T00:42:08.303978+0000","last_clean":"2024-03-30T00:42:08.303978+0000","last_became_active":"2024-03-30T00:42:07.892097+0000","last_became_peered":"2024-03-30T00:42:07.892097+0000","last_unstale":"2024-03-30T00:42:08.303978+0000","last_undegraded":"2024-03-30T00:42:08.303978+0000","last_fullsized":"2024-03-30T00:42:08.303978+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:30:49.532832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.304948+0000","last_change":"2024-03-30T00:42:08.303042+0000","last_active":"2024-03-30T00:42:08.304948+0000","last_peered":"2024-03-30T00:42:08.304948+0000","last_clean":"2024-03-30T00:42:08.304948+0000","last_became_active":"2024-03-30T00:42:08.302186+0000","last_became_peered":"2024-03-30T00:42:08.302186+0000","last_unstale":"2024-03-30T00:42:08.304948+0000","last_undegraded":"2024-03-30T00:42:08.304948+0000","last_fullsized":"2024-03-30T00:42:08.304948+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:42.969334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.292783+0000","last_change":"2024-03-30T00:42:07.897983+0000","last_active":"2024-03-30T00:42:08.292783+0000","last_peered":"2024-03-30T00:42:08.292783+0000","last_clean":"2024-03-30T00:42:08.292783+0000","last_became_active":"2024-03-30T00:42:07.897719+0000","last_became_peered":"2024-03-30T00:42:07.897719+0000","last_unstale":"2024-03-30T00:42:08.292783+0000","last_undegraded":"2024-03-30T00:42:08.292783+0000","last_fullsized":"2024-03-30T00:42:08.292783+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:24:55.868085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.300499+0000","last_change":"2024-03-30T00:42:08.293501+0000","last_active":"2024-03-30T00:42:08.300499+0000","last_peered":"2024-03-30T00:42:08.300499+0000","last_clean":"2024-03-30T00:42:08.300499+0000","last_became_active":"2024-03-30T00:42:08.293254+0000","last_became_peered":"2024-03-30T00:42:08.293254+0000","last_unstale":"2024-03-30T00:42:08.300499+0000","last_undegraded":"2024-03-30T00:42:08.300499+0000","last_fullsized":"2024-03-30T00:42:08.300499+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.303455+0000","last_change":"2024-03-30T00:42:07.884431+0000","last_active":"2024-03-30T00:42:08.303455+0000","last_peered":"2024-03-30T00:42:08.303455+0000","last_clean":"2024-03-30T00:42:08.303455+0000","last_became_active":"2024-03-30T00:42:07.884172+0000","last_became_peered":"2024-03-30T00:42:07.884172+0000","last_unstale":"2024-03-30T00:42:08.303455+0000","last_undegraded":"2024-03-30T00:42:08.303455+0000","last_fullsized":"2024-03-30T00:42:08.303455+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:24.488599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.314915+0000","last_change":"2024-03-30T00:42:08.306805+0000","last_active":"2024-03-30T00:42:08.314915+0000","last_peered":"2024-03-30T00:42:08.314915+0000","last_clean":"2024-03-30T00:42:08.314915+0000","last_became_active":"2024-03-30T00:42:08.306148+0000","last_became_peered":"2024-03-30T00:42:08.306148+0000","last_unstale":"2024-03-30T00:42:08.314915+0000","last_undegraded":"2024-03-30T00:42:08.314915+0000","last_fullsized":"2024-03-30T00:42:08.314915+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.292883+0000","last_change":"2024-03-30T00:42:07.884001+0000","last_active":"2024-03-30T00:42:08.292883+0000","last_peered":"2024-03-30T00:42:08.292883+0000","last_clean":"2024-03-30T00:42:08.292883+0000","last_became_active":"2024-03-30T00:42:07.883690+0000","last_became_peered":"2024-03-30T00:42:07.883690+0000","last_unstale":"2024-03-30T00:42:08.292883+0000","last_undegraded":"2024-03-30T00:42:08.292883+0000","last_fullsized":"2024-03-30T00:42:08.292883+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:26.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.299960+0000","last_change":"2024-03-30T00:42:08.292278+0000","last_active":"2024-03-30T00:42:08.299960+0000","last_peered":"2024-03-30T00:42:08.299960+0000","last_clean":"2024-03-30T00:42:08.299960+0000","last_became_active":"2024-03-30T00:42:08.291996+0000","last_became_peered":"2024-03-30T00:42:08.291996+0000","last_unstale":"2024-03-30T00:42:08.299960+0000","last_undegraded":"2024-03-30T00:42:08.299960+0000","last_fullsized":"2024-03-30T00:42:08.299960+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:00.535526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.284891+0000","last_change":"2024-03-30T00:42:07.887955+0000","last_active":"2024-03-30T00:42:08.284891+0000","last_peered":"2024-03-30T00:42:08.284891+0000","last_clean":"2024-03-30T00:42:08.284891+0000","last_became_active":"2024-03-30T00:42:07.887689+0000","last_became_peered":"2024-03-30T00:42:07.887689+0000","last_unstale":"2024-03-30T00:42:08.284891+0000","last_undegraded":"2024-03-30T00:42:08.284891+0000","last_fullsized":"2024-03-30T00:42:08.284891+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:17:21.438797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.310839+0000","last_change":"2024-03-30T00:42:08.309879+0000","last_active":"2024-03-30T00:42:08.310839+0000","last_peered":"2024-03-30T00:42:08.310839+0000","last_clean":"2024-03-30T00:42:08.310839+0000","last_became_active":"2024-03-30T00:42:08.309640+0000","last_became_peered":"2024-03-30T00:42:08.309640+0000","last_unstale":"2024-03-30T00:42:08.310839+0000","last_undegraded":"2024-03-30T00:42:08.310839+0000","last_fullsized":"2024-03-30T00:42:08.310839+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.277824+0000","last_change":"2024-03-30T00:42:07.884866+0000","last_active":"2024-03-30T00:42:08.277824+0000","last_peered":"2024-03-30T00:42:08.277824+0000","last_clean":"2024-03-30T00:42:08.277824+0000","last_became_active":"2024-03-30T00:42:07.884550+0000","last_became_peered":"2024-03-30T00:42:07.884550+0000","last_unstale":"2024-03-30T00:42:08.277824+0000","last_undegraded":"2024-03-30T00:42:08.277824+0000","last_fullsized":"2024-03-30T00:42:08.277824+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:10:19.003693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.300286+0000","last_change":"2024-03-30T00:42:08.292698+0000","last_active":"2024-03-30T00:42:08.300286+0000","last_peered":"2024-03-30T00:42:08.300286+0000","last_clean":"2024-03-30T00:42:08.300286+0000","last_became_active":"2024-03-30T00:42:08.291346+0000","last_became_peered":"2024-03-30T00:42:08.291346+0000","last_unstale":"2024-03-30T00:42:08.300286+0000","last_undegraded":"2024-03-30T00:42:08.300286+0000","last_fullsized":"2024-03-30T00:42:08.300286+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:53.013122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.288959+0000","last_change":"2024-03-30T00:42:07.895223+0000","last_active":"2024-03-30T00:42:08.288959+0000","last_peered":"2024-03-30T00:42:08.288959+0000","last_clean":"2024-03-30T00:42:08.288959+0000","last_became_active":"2024-03-30T00:42:07.894920+0000","last_became_peered":"2024-03-30T00:42:07.894920+0000","last_unstale":"2024-03-30T00:42:08.288959+0000","last_undegraded":"2024-03-30T00:42:08.288959+0000","last_fullsized":"2024-03-30T00:42:08.288959+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:13:17.278798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.302998+0000","last_change":"2024-03-30T00:42:08.298842+0000","last_active":"2024-03-30T00:42:08.302998+0000","last_peered":"2024-03-30T00:42:08.302998+0000","last_clean":"2024-03-30T00:42:08.302998+0000","last_became_active":"2024-03-30T00:42:08.298592+0000","last_became_peered":"2024-03-30T00:42:08.298592+0000","last_unstale":"2024-03-30T00:42:08.302998+0000","last_undegraded":"2024-03-30T00:42:08.302998+0000","last_fullsized":"2024-03-30T00:42:08.302998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:07.667668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.304198+0000","last_change":"2024-03-30T00:42:07.889759+0000","last_active":"2024-03-30T00:42:08.304198+0000","last_peered":"2024-03-30T00:42:08.304198+0000","last_clean":"2024-03-30T00:42:08.304198+0000","last_became_active":"2024-03-30T00:42:07.888856+0000","last_became_peered":"2024-03-30T00:42:07.888856+0000","last_unstale":"2024-03-30T00:42:08.304198+0000","last_undegraded":"2024-03-30T00:42:08.304198+0000","last_fullsized":"2024-03-30T00:42:08.304198+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:00:01.121929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.303637+0000","last_change":"2024-03-30T00:42:08.296874+0000","last_active":"2024-03-30T00:42:08.303637+0000","last_peered":"2024-03-30T00:42:08.303637+0000","last_clean":"2024-03-30T00:42:08.303637+0000","last_became_active":"2024-03-30T00:42:08.295588+0000","last_became_peered":"2024-03-30T00:42:08.295588+0000","last_unstale":"2024-03-30T00:42:08.303637+0000","last_undegraded":"2024-03-30T00:42:08.303637+0000","last_fullsized":"2024-03-30T00:42:08.303637+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.317315+0000","last_change":"2024-03-30T00:42:07.896369+0000","last_active":"2024-03-30T00:42:08.317315+0000","last_peered":"2024-03-30T00:42:08.317315+0000","last_clean":"2024-03-30T00:42:08.317315+0000","last_became_active":"2024-03-30T00:42:07.896085+0000","last_became_peered":"2024-03-30T00:42:07.896085+0000","last_unstale":"2024-03-30T00:42:08.317315+0000","last_undegraded":"2024-03-30T00:42:08.317315+0000","last_fullsized":"2024-03-30T00:42:08.317315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:48.023033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.296564+0000","last_change":"2024-03-30T00:42:08.289048+0000","last_active":"2024-03-30T00:42:08.296564+0000","last_peered":"2024-03-30T00:42:08.296564+0000","last_clean":"2024-03-30T00:42:08.296564+0000","last_became_active":"2024-03-30T00:42:08.288743+0000","last_became_peered":"2024-03-30T00:42:08.288743+0000","last_unstale":"2024-03-30T00:42:08.296564+0000","last_undegraded":"2024-03-30T00:42:08.296564+0000","last_fullsized":"2024-03-30T00:42:08.296564+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.298628+0000","last_change":"2024-03-30T00:42:07.896028+0000","last_active":"2024-03-30T00:42:08.298628+0000","last_peered":"2024-03-30T00:42:08.298628+0000","last_clean":"2024-03-30T00:42:08.298628+0000","last_became_active":"2024-03-30T00:42:07.895683+0000","last_became_peered":"2024-03-30T00:42:07.895683+0000","last_unstale":"2024-03-30T00:42:08.298628+0000","last_undegraded":"2024-03-30T00:42:08.298628+0000","last_fullsized":"2024-03-30T00:42:08.298628+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:33:37.405718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.301291+0000","last_change":"2024-03-30T00:42:08.293781+0000","last_active":"2024-03-30T00:42:08.301291+0000","last_peered":"2024-03-30T00:42:08.301291+0000","last_clean":"2024-03-30T00:42:08.301291+0000","last_became_active":"2024-03-30T00:42:08.293516+0000","last_became_peered":"2024-03-30T00:42:08.293516+0000","last_unstale":"2024-03-30T00:42:08.301291+0000","last_undegraded":"2024-03-30T00:42:08.301291+0000","last_fullsized":"2024-03-30T00:42:08.301291+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:37.737118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.284654+0000","last_change":"2024-03-30T00:42:07.896370+0000","last_active":"2024-03-30T00:42:08.284654+0000","last_peered":"2024-03-30T00:42:08.284654+0000","last_clean":"2024-03-30T00:42:08.284654+0000","last_became_active":"2024-03-30T00:42:07.896086+0000","last_became_peered":"2024-03-30T00:42:07.896086+0000","last_unstale":"2024-03-30T00:42:08.284654+0000","last_undegraded":"2024-03-30T00:42:08.284654+0000","last_fullsized":"2024-03-30T00:42:08.284654+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:58:41.813204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.308135+0000","last_change":"2024-03-30T00:42:08.302981+0000","last_active":"2024-03-30T00:42:08.308135+0000","last_peered":"2024-03-30T00:42:08.308135+0000","last_clean":"2024-03-30T00:42:08.308135+0000","last_became_active":"2024-03-30T00:42:08.302707+0000","last_became_peered":"2024-03-30T00:42:08.302707+0000","last_unstale":"2024-03-30T00:42:08.308135+0000","last_undegraded":"2024-03-30T00:42:08.308135+0000","last_fullsized":"2024-03-30T00:42:08.308135+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:44.118291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.277901+0000","last_change":"2024-03-30T00:42:07.886045+0000","last_active":"2024-03-30T00:42:08.277901+0000","last_peered":"2024-03-30T00:42:08.277901+0000","last_clean":"2024-03-30T00:42:08.277901+0000","last_became_active":"2024-03-30T00:42:07.885765+0000","last_became_peered":"2024-03-30T00:42:07.885765+0000","last_unstale":"2024-03-30T00:42:08.277901+0000","last_undegraded":"2024-03-30T00:42:08.277901+0000","last_fullsized":"2024-03-30T00:42:08.277901+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:21.533654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.305223+0000","last_change":"2024-03-30T00:42:08.299727+0000","last_active":"2024-03-30T00:42:08.305223+0000","last_peered":"2024-03-30T00:42:08.305223+0000","last_clean":"2024-03-30T00:42:08.305223+0000","last_became_active":"2024-03-30T00:42:08.299313+0000","last_became_peered":"2024-03-30T00:42:08.299313+0000","last_unstale":"2024-03-30T00:42:08.305223+0000","last_undegraded":"2024-03-30T00:42:08.305223+0000","last_fullsized":"2024-03-30T00:42:08.305223+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:01.442897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.315566+0000","last_change":"2024-03-30T00:42:07.897520+0000","last_active":"2024-03-30T00:42:08.315566+0000","last_peered":"2024-03-30T00:42:08.315566+0000","last_clean":"2024-03-30T00:42:08.315566+0000","last_became_active":"2024-03-30T00:42:07.897276+0000","last_became_peered":"2024-03-30T00:42:07.897276+0000","last_unstale":"2024-03-30T00:42:08.315566+0000","last_undegraded":"2024-03-30T00:42:08.315566+0000","last_fullsized":"2024-03-30T00:42:08.315566+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:16:59.791004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.315632+0000","last_change":"2024-03-30T00:42:08.309521+0000","last_active":"2024-03-30T00:42:08.315632+0000","last_peered":"2024-03-30T00:42:08.315632+0000","last_clean":"2024-03-30T00:42:08.315632+0000","last_became_active":"2024-03-30T00:42:08.309249+0000","last_became_peered":"2024-03-30T00:42:08.309249+0000","last_unstale":"2024-03-30T00:42:08.315632+0000","last_undegraded":"2024-03-30T00:42:08.315632+0000","last_fullsized":"2024-03-30T00:42:08.315632+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:13:45.518341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.304336+0000","last_change":"2024-03-30T00:42:07.898205+0000","last_active":"2024-03-30T00:42:08.304336+0000","last_peered":"2024-03-30T00:42:08.304336+0000","last_clean":"2024-03-30T00:42:08.304336+0000","last_became_active":"2024-03-30T00:42:07.897873+0000","last_became_peered":"2024-03-30T00:42:07.897873+0000","last_unstale":"2024-03-30T00:42:08.304336+0000","last_undegraded":"2024-03-30T00:42:08.304336+0000","last_fullsized":"2024-03-30T00:42:08.304336+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:28.263763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.292138+0000","last_change":"2024-03-30T00:42:08.288954+0000","last_active":"2024-03-30T00:42:08.292138+0000","last_peered":"2024-03-30T00:42:08.292138+0000","last_clean":"2024-03-30T00:42:08.292138+0000","last_became_active":"2024-03-30T00:42:08.288710+0000","last_became_peered":"2024-03-30T00:42:08.288710+0000","last_unstale":"2024-03-30T00:42:08.292138+0000","last_undegraded":"2024-03-30T00:42:08.292138+0000","last_fullsized":"2024-03-30T00:42:08.292138+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:25.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.300769+0000","last_change":"2024-03-30T00:42:07.890926+0000","last_active":"2024-03-30T00:42:08.300769+0000","last_peered":"2024-03-30T00:42:08.300769+0000","last_clean":"2024-03-30T00:42:08.300769+0000","last_became_active":"2024-03-30T00:42:07.890676+0000","last_became_peered":"2024-03-30T00:42:07.890676+0000","last_unstale":"2024-03-30T00:42:08.300769+0000","last_undegraded":"2024-03-30T00:42:08.300769+0000","last_fullsized":"2024-03-30T00:42:08.300769+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:24:47.097376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.304729+0000","last_change":"2024-03-30T00:42:08.298780+0000","last_active":"2024-03-30T00:42:08.304729+0000","last_peered":"2024-03-30T00:42:08.304729+0000","last_clean":"2024-03-30T00:42:08.304729+0000","last_became_active":"2024-03-30T00:42:08.298527+0000","last_became_peered":"2024-03-30T00:42:08.298527+0000","last_unstale":"2024-03-30T00:42:08.304729+0000","last_undegraded":"2024-03-30T00:42:08.304729+0000","last_fullsized":"2024-03-30T00:42:08.304729+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:17:00.793702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.299511+0000","last_change":"2024-03-30T00:42:07.888916+0000","last_active":"2024-03-30T00:42:08.299511+0000","last_peered":"2024-03-30T00:42:08.299511+0000","last_clean":"2024-03-30T00:42:08.299511+0000","last_became_active":"2024-03-30T00:42:07.888629+0000","last_became_peered":"2024-03-30T00:42:07.888629+0000","last_unstale":"2024-03-30T00:42:08.299511+0000","last_undegraded":"2024-03-30T00:42:08.299511+0000","last_fullsized":"2024-03-30T00:42:08.299511+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:03.807636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.288896+0000","last_change":"2024-03-30T00:42:08.282516+0000","last_active":"2024-03-30T00:42:08.288896+0000","last_peered":"2024-03-30T00:42:08.288896+0000","last_clean":"2024-03-30T00:42:08.288896+0000","last_became_active":"2024-03-30T00:42:08.282273+0000","last_became_peered":"2024-03-30T00:42:08.282273+0000","last_unstale":"2024-03-30T00:42:08.288896+0000","last_undegraded":"2024-03-30T00:42:08.288896+0000","last_fullsized":"2024-03-30T00:42:08.288896+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:25.491297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.286350+0000","last_change":"2024-03-30T00:42:07.889736+0000","last_active":"2024-03-30T00:42:08.286350+0000","last_peered":"2024-03-30T00:42:08.286350+0000","last_clean":"2024-03-30T00:42:08.286350+0000","last_became_active":"2024-03-30T00:42:07.889425+0000","last_became_peered":"2024-03-30T00:42:07.889425+0000","last_unstale":"2024-03-30T00:42:08.286350+0000","last_undegraded":"2024-03-30T00:42:08.286350+0000","last_fullsized":"2024-03-30T00:42:08.286350+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:41.441678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.302277+0000","last_change":"2024-03-30T00:42:08.300637+0000","last_active":"2024-03-30T00:42:08.302277+0000","last_peered":"2024-03-30T00:42:08.302277+0000","last_clean":"2024-03-30T00:42:08.302277+0000","last_became_active":"2024-03-30T00:42:08.300395+0000","last_became_peered":"2024-03-30T00:42:08.300395+0000","last_unstale":"2024-03-30T00:42:08.302277+0000","last_undegraded":"2024-03-30T00:42:08.302277+0000","last_fullsized":"2024-03-30T00:42:08.302277+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:42.969334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.303012+0000","last_change":"2024-03-30T00:42:07.884117+0000","last_active":"2024-03-30T00:42:08.303012+0000","last_peered":"2024-03-30T00:42:08.303012+0000","last_clean":"2024-03-30T00:42:08.303012+0000","last_became_active":"2024-03-30T00:42:07.883887+0000","last_became_peered":"2024-03-30T00:42:07.883887+0000","last_unstale":"2024-03-30T00:42:08.303012+0000","last_undegraded":"2024-03-30T00:42:08.303012+0000","last_fullsized":"2024-03-30T00:42:08.303012+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:18.896534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.305861+0000","last_change":"2024-03-30T00:42:08.304828+0000","last_active":"2024-03-30T00:42:08.305861+0000","last_peered":"2024-03-30T00:42:08.305861+0000","last_clean":"2024-03-30T00:42:08.305861+0000","last_became_active":"2024-03-30T00:42:08.304514+0000","last_became_peered":"2024-03-30T00:42:08.304514+0000","last_unstale":"2024-03-30T00:42:08.305861+0000","last_undegraded":"2024-03-30T00:42:08.305861+0000","last_fullsized":"2024-03-30T00:42:08.305861+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:00:02.124627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.292692+0000","last_change":"2024-03-30T00:42:07.893214+0000","last_active":"2024-03-30T00:42:08.292692+0000","last_peered":"2024-03-30T00:42:08.292692+0000","last_clean":"2024-03-30T00:42:08.292692+0000","last_became_active":"2024-03-30T00:42:07.892814+0000","last_became_peered":"2024-03-30T00:42:07.892814+0000","last_unstale":"2024-03-30T00:42:08.292692+0000","last_undegraded":"2024-03-30T00:42:08.292692+0000","last_fullsized":"2024-03-30T00:42:08.292692+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:52.010424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.303135+0000","last_change":"2024-03-30T00:42:08.298344+0000","last_active":"2024-03-30T00:42:08.303135+0000","last_peered":"2024-03-30T00:42:08.303135+0000","last_clean":"2024-03-30T00:42:08.303135+0000","last_became_active":"2024-03-30T00:42:08.298062+0000","last_became_peered":"2024-03-30T00:42:08.298062+0000","last_unstale":"2024-03-30T00:42:08.303135+0000","last_undegraded":"2024-03-30T00:42:08.303135+0000","last_fullsized":"2024-03-30T00:42:08.303135+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:50.763762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.283254+0000","last_change":"2024-03-30T00:42:07.896209+0000","last_active":"2024-03-30T00:42:08.283254+0000","last_peered":"2024-03-30T00:42:08.283254+0000","last_clean":"2024-03-30T00:42:08.283254+0000","last_became_active":"2024-03-30T00:42:07.895871+0000","last_became_peered":"2024-03-30T00:42:07.895871+0000","last_unstale":"2024-03-30T00:42:08.283254+0000","last_undegraded":"2024-03-30T00:42:08.283254+0000","last_fullsized":"2024-03-30T00:42:08.283254+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:24.948608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.306111+0000","last_change":"2024-03-30T00:42:08.304029+0000","last_active":"2024-03-30T00:42:08.306111+0000","last_peered":"2024-03-30T00:42:08.306111+0000","last_clean":"2024-03-30T00:42:08.306111+0000","last_became_active":"2024-03-30T00:42:08.303765+0000","last_became_peered":"2024-03-30T00:42:08.303765+0000","last_unstale":"2024-03-30T00:42:08.306111+0000","last_undegraded":"2024-03-30T00:42:08.306111+0000","last_fullsized":"2024-03-30T00:42:08.306111+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:02.339037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.278672+0000","last_change":"2024-03-30T00:42:07.891909+0000","last_active":"2024-03-30T00:42:08.278672+0000","last_peered":"2024-03-30T00:42:08.278672+0000","last_clean":"2024-03-30T00:42:08.278672+0000","last_became_active":"2024-03-30T00:42:07.891660+0000","last_became_peered":"2024-03-30T00:42:07.891660+0000","last_unstale":"2024-03-30T00:42:08.278672+0000","last_undegraded":"2024-03-30T00:42:08.278672+0000","last_fullsized":"2024-03-30T00:42:08.278672+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:32.488321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.311885+0000","last_change":"2024-03-30T00:42:08.311181+0000","last_active":"2024-03-30T00:42:08.311885+0000","last_peered":"2024-03-30T00:42:08.311885+0000","last_clean":"2024-03-30T00:42:08.311885+0000","last_became_active":"2024-03-30T00:42:08.310745+0000","last_became_peered":"2024-03-30T00:42:08.310745+0000","last_unstale":"2024-03-30T00:42:08.311885+0000","last_undegraded":"2024-03-30T00:42:08.311885+0000","last_fullsized":"2024-03-30T00:42:08.311885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:40.974026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.302411+0000","last_change":"2024-03-30T00:42:07.896847+0000","last_active":"2024-03-30T00:42:08.302411+0000","last_peered":"2024-03-30T00:42:08.302411+0000","last_clean":"2024-03-30T00:42:08.302411+0000","last_became_active":"2024-03-30T00:42:07.896307+0000","last_became_peered":"2024-03-30T00:42:07.896307+0000","last_unstale":"2024-03-30T00:42:08.302411+0000","last_undegraded":"2024-03-30T00:42:08.302411+0000","last_fullsized":"2024-03-30T00:42:08.302411+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:30.469686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.288042+0000","last_change":"2024-03-30T00:42:08.283030+0000","last_active":"2024-03-30T00:42:08.288042+0000","last_peered":"2024-03-30T00:42:08.288042+0000","last_clean":"2024-03-30T00:42:08.288042+0000","last_became_active":"2024-03-30T00:42:08.282721+0000","last_became_peered":"2024-03-30T00:42:08.282721+0000","last_unstale":"2024-03-30T00:42:08.288042+0000","last_undegraded":"2024-03-30T00:42:08.288042+0000","last_fullsized":"2024-03-30T00:42:08.288042+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:30:50.535530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.277765+0000","last_change":"2024-03-30T00:42:07.894662+0000","last_active":"2024-03-30T00:42:08.277765+0000","last_peered":"2024-03-30T00:42:08.277765+0000","last_clean":"2024-03-30T00:42:08.277765+0000","last_became_active":"2024-03-30T00:42:07.894413+0000","last_became_peered":"2024-03-30T00:42:07.894413+0000","last_unstale":"2024-03-30T00:42:08.277765+0000","last_undegraded":"2024-03-30T00:42:08.277765+0000","last_fullsized":"2024-03-30T00:42:08.277765+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:02:33.633690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.315001+0000","last_change":"2024-03-30T00:42:08.307025+0000","last_active":"2024-03-30T00:42:08.315001+0000","last_peered":"2024-03-30T00:42:08.315001+0000","last_clean":"2024-03-30T00:42:08.315001+0000","last_became_active":"2024-03-30T00:42:08.306593+0000","last_became_peered":"2024-03-30T00:42:08.306593+0000","last_unstale":"2024-03-30T00:42:08.315001+0000","last_undegraded":"2024-03-30T00:42:08.315001+0000","last_fullsized":"2024-03-30T00:42:08.315001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:58.102992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.288450+0000","last_change":"2024-03-30T00:42:07.895012+0000","last_active":"2024-03-30T00:42:08.288450+0000","last_peered":"2024-03-30T00:42:08.288450+0000","last_clean":"2024-03-30T00:42:08.288450+0000","last_became_active":"2024-03-30T00:42:07.894725+0000","last_became_peered":"2024-03-30T00:42:07.894725+0000","last_unstale":"2024-03-30T00:42:08.288450+0000","last_undegraded":"2024-03-30T00:42:08.288450+0000","last_fullsized":"2024-03-30T00:42:08.288450+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:41:59.532828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.314780+0000","last_change":"2024-03-30T00:42:08.311992+0000","last_active":"2024-03-30T00:42:08.314780+0000","last_peered":"2024-03-30T00:42:08.314780+0000","last_clean":"2024-03-30T00:42:08.314780+0000","last_became_active":"2024-03-30T00:42:08.311834+0000","last_became_peered":"2024-03-30T00:42:08.311834+0000","last_unstale":"2024-03-30T00:42:08.314780+0000","last_undegraded":"2024-03-30T00:42:08.314780+0000","last_fullsized":"2024-03-30T00:42:08.314780+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:17.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.292831+0000","last_change":"2024-03-30T00:42:07.893706+0000","last_active":"2024-03-30T00:42:08.292831+0000","last_peered":"2024-03-30T00:42:08.292831+0000","last_clean":"2024-03-30T00:42:08.292831+0000","last_became_active":"2024-03-30T00:42:07.893465+0000","last_became_peered":"2024-03-30T00:42:07.893465+0000","last_unstale":"2024-03-30T00:42:08.292831+0000","last_undegraded":"2024-03-30T00:42:08.292831+0000","last_fullsized":"2024-03-30T00:42:08.292831+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:41.966636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.298152+0000","last_change":"2024-03-30T00:42:08.289749+0000","last_active":"2024-03-30T00:42:08.298152+0000","last_peered":"2024-03-30T00:42:08.298152+0000","last_clean":"2024-03-30T00:42:08.298152+0000","last_became_active":"2024-03-30T00:42:08.289459+0000","last_became_peered":"2024-03-30T00:42:08.289459+0000","last_unstale":"2024-03-30T00:42:08.298152+0000","last_undegraded":"2024-03-30T00:42:08.298152+0000","last_fullsized":"2024-03-30T00:42:08.298152+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:28.058636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.292717+0000","last_change":"2024-03-30T00:42:07.893058+0000","last_active":"2024-03-30T00:42:08.292717+0000","last_peered":"2024-03-30T00:42:08.292717+0000","last_clean":"2024-03-30T00:42:08.292717+0000","last_became_active":"2024-03-30T00:42:07.892453+0000","last_became_peered":"2024-03-30T00:42:07.892453+0000","last_unstale":"2024-03-30T00:42:08.292717+0000","last_undegraded":"2024-03-30T00:42:08.292717+0000","last_fullsized":"2024-03-30T00:42:08.292717+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:27.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.301357+0000","last_change":"2024-03-30T00:42:08.296007+0000","last_active":"2024-03-30T00:42:08.301357+0000","last_peered":"2024-03-30T00:42:08.301357+0000","last_clean":"2024-03-30T00:42:08.301357+0000","last_became_active":"2024-03-30T00:42:08.295768+0000","last_became_peered":"2024-03-30T00:42:08.295768+0000","last_unstale":"2024-03-30T00:42:08.301357+0000","last_undegraded":"2024-03-30T00:42:08.301357+0000","last_fullsized":"2024-03-30T00:42:08.301357+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:39.997242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.283500+0000","last_change":"2024-03-30T00:42:07.889478+0000","last_active":"2024-03-30T00:42:08.283500+0000","last_peered":"2024-03-30T00:42:08.283500+0000","last_clean":"2024-03-30T00:42:08.283500+0000","last_became_active":"2024-03-30T00:42:07.889247+0000","last_became_peered":"2024-03-30T00:42:07.889247+0000","last_unstale":"2024-03-30T00:42:08.283500+0000","last_undegraded":"2024-03-30T00:42:08.283500+0000","last_fullsized":"2024-03-30T00:42:08.283500+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:01.336339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.315602+0000","last_change":"2024-03-30T00:42:08.308713+0000","last_active":"2024-03-30T00:42:08.315602+0000","last_peered":"2024-03-30T00:42:08.315602+0000","last_clean":"2024-03-30T00:42:08.315602+0000","last_became_active":"2024-03-30T00:42:08.308450+0000","last_became_peered":"2024-03-30T00:42:08.308450+0000","last_unstale":"2024-03-30T00:42:08.315602+0000","last_undegraded":"2024-03-30T00:42:08.315602+0000","last_fullsized":"2024-03-30T00:42:08.315602+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:28.651272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.280614+0000","last_change":"2024-03-30T00:42:07.891209+0000","last_active":"2024-03-30T00:42:08.280614+0000","last_peered":"2024-03-30T00:42:08.280614+0000","last_clean":"2024-03-30T00:42:08.280614+0000","last_became_active":"2024-03-30T00:42:07.890769+0000","last_became_peered":"2024-03-30T00:42:07.890769+0000","last_unstale":"2024-03-30T00:42:08.280614+0000","last_undegraded":"2024-03-30T00:42:08.280614+0000","last_fullsized":"2024-03-30T00:42:08.280614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:21.096421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.304203+0000","last_change":"2024-03-30T00:42:08.302091+0000","last_active":"2024-03-30T00:42:08.304203+0000","last_peered":"2024-03-30T00:42:08.304203+0000","last_clean":"2024-03-30T00:42:08.304203+0000","last_became_active":"2024-03-30T00:42:08.301545+0000","last_became_peered":"2024-03-30T00:42:08.301545+0000","last_unstale":"2024-03-30T00:42:08.304203+0000","last_undegraded":"2024-03-30T00:42:08.304203+0000","last_fullsized":"2024-03-30T00:42:08.304203+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:07.860136+0000","last_clean_scrub_stamp":"2024-03-30T00:42:07.860136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:02:34.636388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.284955+0000","last_change":"2024-03-30T00:42:07.899549+0000","last_active":"2024-03-30T00:42:08.284955+0000","last_peered":"2024-03-30T00:42:08.284955+0000","last_clean":"2024-03-30T00:42:08.284955+0000","last_became_active":"2024-03-30T00:42:07.896786+0000","last_became_peered":"2024-03-30T00:42:07.896786+0000","last_unstale":"2024-03-30T00:42:08.284955+0000","last_undegraded":"2024-03-30T00:42:08.284955+0000","last_fullsized":"2024-03-30T00:42:08.284955+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:42:06.857438+0000","last_clean_scrub_stamp":"2024-03-30T00:42:06.857438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:24.488599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-30T00:42:08.277166+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:42:08.277166+0000","last_peered":"2024-03-30T00:42:08.277166+0000","last_clean":"2024-03-30T00:42:08.277166+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:42:08.277166+0000","last_undegraded":"2024-03-30T00:42:08.277166+0000","last_fullsized":"2024-03-30T00:42:08.277166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1245184,"data_stored":874593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.63600000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.62},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.93300000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.51600000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.66400000000000003}]},{"osd":2,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78100000000000003,"15min":0.78900000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.194,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.79400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.026,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.83999999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0840000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82899999999999996},"min":{"1min":0.65100000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0429999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.89500000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82599999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63500000000000001,"15min":0.629},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.97899999999999998}]},{"osd":5,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.996,"5min":1.026,"15min":1.026},"last":1.0049999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.45400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":1.0189999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":651264,"data_stored":284753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.89200000000000002,"15min":0.93799999999999994},"min":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90100000000000002,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78300000000000003,"5min":1.47,"15min":1.47},"last":0.46400000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.90100000000000002,"15min":0.95299999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76900000000000002,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.97299999999999998,"15min":1.0329999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.82399999999999995,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.745}]},{"osd":2,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.96999999999999997,"15min":1.018},"min":{"1min":0.48699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.82299999999999995,"5min":4.375,"15min":4.375},"last":0.501},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.97199999999999998,"15min":1.024},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":4.2960000000000003,"15min":4.2960000000000003},"last":0.622}]},{"osd":3,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.98799999999999999,"15min":1.038},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97299999999999998,"5min":4.3399999999999999,"15min":4.3399999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.97499999999999998,"15min":1.034},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80200000000000005,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.60099999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.56999999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96999999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.55100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":643072,"data_stored":284101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.42399999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.095},"last":0.376},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.78700000000000003,"15min":0.81000000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95099999999999996,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.751,"15min":0.76600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77400000000000002,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.72299999999999998,"15min":0.73899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83599999999999997,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.61099999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90600000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.70899999999999996,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79700000000000004,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.79200000000000004,"15min":0.81200000000000006},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98499999999999999,"5min":1.7,"15min":1.7},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.80200000000000005,"5min":1.181,"15min":1.181},"last":0.71599999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63400000000000001,"15min":0.64400000000000002},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91500000000000004,"5min":1.615,"15min":1.615},"last":0.58499999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88100000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0700000000000001,"5min":1.569,"15min":1.569},"last":0.53400000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76000000000000001,"15min":0.76300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89300000000000002,"5min":1.006,"15min":1.006},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.83899999999999997,"15min":0.84899999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.032,"5min":1.383,"15min":1.383},"last":0.74399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":647168,"data_stored":284683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77900000000000003,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.44600000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.71299999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71499999999999997,"15min":0.72299999999999998},"min":{"1min":0.50700000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.872,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.81200000000000006}]},{"osd":2,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.91000000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.755,"15min":0.76300000000000001},"min":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96899999999999997,"5min":1.228,"15min":1.228},"last":0.68899999999999995}]},{"osd":3,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.50700000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94899999999999995,"5min":1.27,"15min":1.27},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.78300000000000003,"15min":0.78900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91400000000000003,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.80200000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84899999999999998,"5min":1.329,"15min":1.329},"last":0.624},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93500000000000005,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.78800000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98899999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95799999999999996,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.86099999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.749,"15min":0.76300000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.069,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.84499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":285687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.56899999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.53200000000000003,"15min":0.55300000000000005},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.57199999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59799999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46899999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.623,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.69299999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.5}]},{"osd":4,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.76100000000000001,"15min":0.77600000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98299999999999998,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.77100000000000002,"15min":0.78500000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0069999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.70099999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.78200000000000003,"15min":0.79300000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.95199999999999996,"5min":1.125,"15min":1.125},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80600000000000005,"15min":0.82099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.024,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.64100000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.81200000000000006,"15min":0.82699999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0389999999999999,"5min":1.226,"15min":1.226},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.79700000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0109999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:41:40 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.85099999999999998,"15min":0.86899999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.077,"5min":1.202,"15min":1.202},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.88400000000000001,"15min":0.90800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0740000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.74399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":285665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.313,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.627,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.51100000000000001,"15min":0.52700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.56200000000000006,"5min":0.747,"15min":0.747},"last":0.55100000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.58199999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53200000000000003,"15min":0.54400000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.63,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51000000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.495,"15min":0.503},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.53400000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76200000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80300000000000005,"15min":0.81499999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.042,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.127,"5min":1.528,"15min":1.528},"last":0.90600000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83099999999999996,"15min":0.84499999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1240000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.80600000000000005,"15min":0.82299999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0580000000000001,"5min":1.131,"15min":1.131},"last":0.97299999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.81000000000000005,"15min":0.82099999999999995},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.081,"5min":1.151,"15min":1.151},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.179,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.95599999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.85999999999999999,"15min":0.88200000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.01,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0700000000000001,"5min":1.228,"15min":1.228},"last":1.0009999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":284261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61299999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60699999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.46000000000000002,"15min":0.46899999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55700000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.89400000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66200000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.745},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.626,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.81599999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92000000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.95599999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.745},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.995}]},{"osd":6,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77600000000000002,"15min":0.78000000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93400000000000005,"5min":1.097,"15min":1.097},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.90100000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.02}]},{"osd":7,"last update":"Sat Mar 30 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78900000000000003,"15min":0.79500000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95199999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92800000000000005,"5min":0.996,"15min":0.996},"last":1.052}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1245184,"data_stored":874525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.29299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58499999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.61299999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.36299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66100000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.498}]},{"osd":3,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71499999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53600000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79100000000000004,"15min":0.79400000000000004},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.008,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79900000000000004,"15min":0.80000000000000004},"min":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.1279999999999999,"5min":1.153,"15min":1.153},"last":0.73799999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.621,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0489999999999999,"5min":1.22,"15min":1.22},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.92300000000000004,"5min":1.462,"15min":1.462},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78900000000000003,"15min":0.79800000000000004},"min":{"1min":0.58299999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.091,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.56299999999999994,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1,"5min":1.085,"15min":1.085},"last":0.80000000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.91500000000000004,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96899999999999997,"5min":1.556,"15min":1.556},"last":0.93899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:42:17.539 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T00:42:17.539 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-30T00:42:17.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.712+0000 7fd9a7e00640 1 Processor -- start 2024-03-30T00:42:17.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.712+0000 7fd9a7e00640 1 -- start start 2024-03-30T00:42:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.716+0000 7fd9a7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd9a80ce6c0 0x7fd9a815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.716+0000 7fd9a7e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a815d520 con 0x7fd9a80b1db0 2024-03-30T00:42:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.716+0000 7fd9a7e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a815d800 con 0x7fd9a80cd740 2024-03-30T00:42:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.716+0000 7fd9a7e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a815dae0 con 0x7fd9a80ce6c0 2024-03-30T00:42:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.716+0000 7fd9a6a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd9a80ce6c0 0x7fd9a815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.716+0000 7fd9a7400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd9a80b1db0 0x7fd9a80b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55692/0 (socket says 172.21.15.66:55692) 2024-03-30T00:42:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.716+0000 7fd9a7400640 1 -- 172.21.15.66:0/3615206377 learned_addr learned my addr 172.21.15.66:0/3615206377 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:17.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.716+0000 7fd9a6a00640 1 -- 172.21.15.66:0/3615206377 >> v1:172.21.15.156:6789/0 conn(0x7fd9a80cd740 legacy=0x7fd9a80d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:17.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.716+0000 7fd9a6a00640 1 -- 172.21.15.66:0/3615206377 >> v1:172.21.15.66:6789/0 conn(0x7fd9a80b1db0 legacy=0x7fd9a80b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.716+0000 7fd9a6a00640 1 -- 172.21.15.66:0/3615206377 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a806b450 con 0x7fd9a80ce6c0 2024-03-30T00:42:17.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.716+0000 7fd9a6a00640 1 --2- 172.21.15.66:0/3615206377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a80ce6c0 0x7fd9a815cef0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd9980097c0 tx=0x7fd99801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec9fa32a16d8eb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:17.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a6000640 1 -- 172.21.15.66:0/3615206377 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9980ab620 con 0x7fd9a80ce6c0 2024-03-30T00:42:17.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a6000640 1 -- 172.21.15.66:0/3615206377 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9980ab780 con 0x7fd9a80ce6c0 2024-03-30T00:42:17.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a6000640 1 -- 172.21.15.66:0/3615206377 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9980abb10 con 0x7fd9a80ce6c0 2024-03-30T00:42:17.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 -- 172.21.15.66:0/3615206377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a80ce6c0 msgr2=0x7fd9a815cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/3615206377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a80ce6c0 0x7fd9a815cef0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd9980097c0 tx=0x7fd99801c460 comp rx=0 tx=0).stop 2024-03-30T00:42:17.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 -- 172.21.15.66:0/3615206377 shutdown_connections 2024-03-30T00:42:17.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/3615206377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a80ce6c0 0x7fd9a815cef0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 -- 172.21.15.66:0/3615206377 >> 172.21.15.66:0/3615206377 conn(0x7fd9a8069d90 msgr2=0x7fd9a806a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:17.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 -- 172.21.15.66:0/3615206377 shutdown_connections 2024-03-30T00:42:17.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 -- 172.21.15.66:0/3615206377 wait complete. 2024-03-30T00:42:17.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 Processor -- start 2024-03-30T00:42:17.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 -- start start 2024-03-30T00:42:17.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a80b1db0 0x7fd9a815a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a80cd740 0x7fd9a81489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80ce6c0 0x7fd9a815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8156ba0 con 0x7fd9a80cd740 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8156e80 con 0x7fd9a80ce6c0 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd9a7e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80b2ee0 con 0x7fd9a80b1db0 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd99fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a80cd740 0x7fd9a81489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd99fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a80cd740 0x7fd9a81489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36404/0 (socket says 172.21.15.66:36404) 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd99fe00640 1 -- 172.21.15.66:0/2301418894 learned_addr learned my addr 172.21.15.66:0/2301418894 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd99fe00640 1 -- 172.21.15.66:0/2301418894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a80b1db0 msgr2=0x7fd9a815a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd99fe00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a80b1db0 0x7fd9a815a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd99fe00640 1 -- 172.21.15.66:0/2301418894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80ce6c0 msgr2=0x7fd9a815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd99fe00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80ce6c0 0x7fd9a815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.720+0000 7fd99fe00640 1 -- 172.21.15.66:0/2301418894 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd998009420 con 0x7fd9a80cd740 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.724+0000 7fd99fe00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a80cd740 0x7fd9a81489d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c490 tx=0x7fd990001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.724+0000 7fd99ea00640 1 -- 172.21.15.66:0/2301418894 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990003600 con 0x7fd9a80cd740 2024-03-30T00:42:17.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.724+0000 7fd99ea00640 1 -- 172.21.15.66:0/2301418894 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd990003780 con 0x7fd9a80cd740 2024-03-30T00:42:17.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.724+0000 7fd99ea00640 1 -- 172.21.15.66:0/2301418894 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990003b70 con 0x7fd9a80cd740 2024-03-30T00:42:17.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.724+0000 7fd9a7e00640 1 -- 172.21.15.66:0/2301418894 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a80b31c0 con 0x7fd9a80cd740 2024-03-30T00:42:17.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.724+0000 7fd9a7e00640 1 -- 172.21.15.66:0/2301418894 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a8148f10 con 0x7fd9a80cd740 2024-03-30T00:42:17.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.724+0000 7fd9a7e00640 1 -- 172.21.15.66:0/2301418894 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a8073d60 con 0x7fd9a80cd740 2024-03-30T00:42:17.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.724+0000 7fd99ea00640 1 -- 172.21.15.66:0/2301418894 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd990003cd0 con 0x7fd9a80cd740 2024-03-30T00:42:17.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.724+0000 7fd99ea00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 0x7fd984068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:17.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.724+0000 7fd99ea00640 1 -- 172.21.15.66:0/2301418894 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd990083510 con 0x7fd9a80cd740 2024-03-30T00:42:17.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.724+0000 7fd9a6a00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 0x7fd984068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:17.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.728+0000 7fd9a6a00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 0x7fd984068b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd9980af000 tx=0x7fd998008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:17.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.728+0000 7fd99ea00640 1 -- 172.21.15.66:0/2301418894 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd9900506b0 con 0x7fd9a80cd740 2024-03-30T00:42:17.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.948+0000 7fd9a7e00640 1 -- 172.21.15.66:0/2301418894 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd9a80b3670 con 0x7fd9a80cd740 2024-03-30T00:42:17.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.952+0000 7fd99ea00640 1 -- 172.21.15.66:0/2301418894 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd990053cd0 con 0x7fd9a80cd740 2024-03-30T00:42:17.963 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-03-30T00:42:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 -- 172.21.15.66:0/2301418894 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 msgr2=0x7fd984068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 0x7fd984068b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd9980af000 tx=0x7fd998008040 comp rx=0 tx=0).stop 2024-03-30T00:42:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 -- 172.21.15.66:0/2301418894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a80cd740 msgr2=0x7fd9a81489d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a80cd740 0x7fd9a81489d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c490 tx=0x7fd990001e80 comp rx=0 tx=0).stop 2024-03-30T00:42:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 -- 172.21.15.66:0/2301418894 shutdown_connections 2024-03-30T00:42:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80ce6c0 0x7fd9a815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 0x7fd984068b50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a80cd740 0x7fd9a81489d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/2301418894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a80b1db0 0x7fd9a815a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 -- 172.21.15.66:0/2301418894 >> 172.21.15.66:0/2301418894 conn(0x7fd9a8069d90 msgr2=0x7fd9a814ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 -- 172.21.15.66:0/2301418894 shutdown_connections 2024-03-30T00:42:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:17.956+0000 7fd9a7e00640 1 -- 172.21.15.66:0/2301418894 wait complete. 2024-03-30T00:42:17.985 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:42:18.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.112+0000 7fb355800640 1 Processor -- start 2024-03-30T00:42:18.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.112+0000 7fb355800640 1 -- start start 2024-03-30T00:42:18.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb355800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3500ae4e0 0x7fb3500cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb355800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb3500ce820 0x7fb3500cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb355800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35006e7d0 con 0x7fb35015ca10 2024-03-30T00:42:18.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb355800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3500b1f70 con 0x7fb3500ce820 2024-03-30T00:42:18.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb355800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500cf1f0 con 0x7fb3500ae4e0 2024-03-30T00:42:18.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb34f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb3500ce820 0x7fb3500cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb34f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb3500ce820 0x7fb3500cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36410/0 (socket says 172.21.15.66:36410) 2024-03-30T00:42:18.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb34f400640 1 -- 172.21.15.66:0/2346539643 learned_addr learned my addr 172.21.15.66:0/2346539643 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:18.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb347e00640 1 --2- 172.21.15.66:0/2346539643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3500ae4e0 0x7fb3500cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb34f400640 1 -- 172.21.15.66:0/2346539643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3500ae4e0 msgr2=0x7fb3500cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:18.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb34f400640 1 --2- 172.21.15.66:0/2346539643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3500ae4e0 0x7fb3500cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb34f400640 1 -- 172.21.15.66:0/2346539643 >> v1:172.21.15.156:6789/0 conn(0x7fb35015ca10 legacy=0x7fb35006ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.116+0000 7fb34f400640 1 -- 172.21.15.66:0/2346539643 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35015cdf0 con 0x7fb3500ce820 2024-03-30T00:42:18.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb34f400640 1 --2- 172.21.15.66:0/2346539643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500ce820 0x7fb3500cecb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb348009e80 tx=0x7fb34801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f41321f257f5fca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:18.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb34e000640 1 -- 172.21.15.66:0/2346539643 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3480a57d0 con 0x7fb3500ce820 2024-03-30T00:42:18.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb34e000640 1 -- 172.21.15.66:0/2346539643 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3480a5930 con 0x7fb3500ce820 2024-03-30T00:42:18.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb34e000640 1 -- 172.21.15.66:0/2346539643 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3480a5cc0 con 0x7fb3500ce820 2024-03-30T00:42:18.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb355800640 1 -- 172.21.15.66:0/2346539643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500ce820 msgr2=0x7fb3500cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb355800640 1 --2- 172.21.15.66:0/2346539643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500ce820 0x7fb3500cecb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb348009e80 tx=0x7fb34801ca10 comp rx=0 tx=0).stop 2024-03-30T00:42:18.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb355800640 1 -- 172.21.15.66:0/2346539643 shutdown_connections 2024-03-30T00:42:18.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb355800640 1 --2- 172.21.15.66:0/2346539643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500ce820 0x7fb3500cecb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb355800640 1 --2- 172.21.15.66:0/2346539643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3500ae4e0 0x7fb3500cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb355800640 1 -- 172.21.15.66:0/2346539643 >> 172.21.15.66:0/2346539643 conn(0x7fb350098a20 msgr2=0x7fb3500b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:18.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb355800640 1 -- 172.21.15.66:0/2346539643 shutdown_connections 2024-03-30T00:42:18.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb355800640 1 -- 172.21.15.66:0/2346539643 wait complete. 2024-03-30T00:42:18.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb355800640 1 Processor -- start 2024-03-30T00:42:18.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.120+0000 7fb355800640 1 -- start start 2024-03-30T00:42:18.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb355800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500ae4e0 0x7fb3500c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb355800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3500ce820 0x7fb35015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb355800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35015ca10 0x7fb3500b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb355800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500b4590 con 0x7fb3500ae4e0 2024-03-30T00:42:18.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb355800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35015a250 con 0x7fb3500ce820 2024-03-30T00:42:18.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb355800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35015a530 con 0x7fb35015ca10 2024-03-30T00:42:18.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb34ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500ae4e0 0x7fb3500c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb34ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500ae4e0 0x7fb3500c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36420/0 (socket says 172.21.15.66:36420) 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb34ea00640 1 -- 172.21.15.66:0/985268330 learned_addr learned my addr 172.21.15.66:0/985268330 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb347e00640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3500ce820 0x7fb35015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb34ea00640 1 -- 172.21.15.66:0/985268330 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35015ca10 msgr2=0x7fb3500b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb34ea00640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35015ca10 0x7fb3500b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb34ea00640 1 -- 172.21.15.66:0/985268330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3500ce820 msgr2=0x7fb35015f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb34ea00640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3500ce820 0x7fb35015f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb34ea00640 1 -- 172.21.15.66:0/985268330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb348009b30 con 0x7fb3500ae4e0 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb34ea00640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500ae4e0 0x7fb3500c6580 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c3c0 tx=0x7fb34000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb346a00640 1 -- 172.21.15.66:0/985268330 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3400052d0 con 0x7fb3500ae4e0 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb346a00640 1 -- 172.21.15.66:0/985268330 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb340005430 con 0x7fb3500ae4e0 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb346a00640 1 -- 172.21.15.66:0/985268330 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340003840 con 0x7fb3500ae4e0 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb355800640 1 -- 172.21.15.66:0/985268330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35015a810 con 0x7fb3500ae4e0 2024-03-30T00:42:18.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb355800640 1 -- 172.21.15.66:0/985268330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3500b4d60 con 0x7fb3500ae4e0 2024-03-30T00:42:18.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb355800640 1 -- 172.21.15.66:0/985268330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35015ad30 con 0x7fb3500ae4e0 2024-03-30T00:42:18.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.124+0000 7fb346a00640 1 -- 172.21.15.66:0/985268330 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb3400039a0 con 0x7fb3500ae4e0 2024-03-30T00:42:18.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.128+0000 7fb346a00640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb32c066690 0x7fb32c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.128+0000 7fb346a00640 1 -- 172.21.15.66:0/985268330 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb340083760 con 0x7fb3500ae4e0 2024-03-30T00:42:18.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.128+0000 7fb347e00640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb32c066690 0x7fb32c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.128+0000 7fb347e00640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb32c066690 0x7fb32c068b50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb3500b2fc0 tx=0x7fb338008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:18.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.132+0000 7fb346a00640 1 -- 172.21.15.66:0/985268330 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb35015ad30 con 0x7fb3500ae4e0 2024-03-30T00:42:18.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.324+0000 7fb355800640 1 -- 172.21.15.66:0/985268330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb35015ad30 con 0x7fb3500ae4e0 2024-03-30T00:42:18.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.324+0000 7fb346a00640 1 -- 172.21.15.66:0/985268330 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fb340050a30 con 0x7fb3500ae4e0 2024-03-30T00:42:18.336 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:18.336 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/465916334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":465916334},{"type":"v1","addr":"172.21.15.156:6835","nonce":465916334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1640047258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1640047258},{"type":"v1","addr":"172.21.15.156:6834","nonce":1640047258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5788,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2112021115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2112021115},{"type":"v1","addr":"172.21.15.66:6835","nonce":2112021115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:42:08.269705+0000","modified":"2024-03-30T00:42:11.492352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5985},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5985":{"gid":5985,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/3557086525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3557086525},{"type":"v1","addr":"172.21.15.66:6837","nonce":3557086525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T00:42:18.337 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-03-30T00:42:18.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 -- 172.21.15.66:0/985268330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb32c066690 msgr2=0x7fb32c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb32c066690 0x7fb32c068b50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb3500b2fc0 tx=0x7fb338008040 comp rx=0 tx=0).stop 2024-03-30T00:42:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 -- 172.21.15.66:0/985268330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500ae4e0 msgr2=0x7fb3500c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500ae4e0 0x7fb3500c6580 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c3c0 tx=0x7fb34000c880 comp rx=0 tx=0).stop 2024-03-30T00:42:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 -- 172.21.15.66:0/985268330 shutdown_connections 2024-03-30T00:42:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35015ca10 0x7fb3500b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3500ce820 0x7fb35015f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb32c066690 0x7fb32c068b50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 --2- 172.21.15.66:0/985268330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500ae4e0 0x7fb3500c6580 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 -- 172.21.15.66:0/985268330 >> 172.21.15.66:0/985268330 conn(0x7fb350098a20 msgr2=0x7fb3500ac2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 -- 172.21.15.66:0/985268330 shutdown_connections 2024-03-30T00:42:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.328+0000 7fb355800640 1 -- 172.21.15.66:0/985268330 wait complete. 2024-03-30T00:42:18.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:42:08.269705+0000', 'modified': '2024-03-30T00:42:11.492352+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5985}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5985': {'gid': 5985, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6837/3557086525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 3557086525}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 3557086525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:42:18.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:42:18.761 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T00:42:18.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.892+0000 7ff0efe00640 1 Processor -- start 2024-03-30T00:42:18.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.892+0000 7ff0efe00640 1 -- start start 2024-03-30T00:42:18.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0efe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff0e80ce520 0x7ff0e80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0efe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0e80cabb0 0x7ff0e80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0efe00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e806d070 con 0x7ff0e815a640 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0efe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e80cbba0 con 0x7ff0e80ce520 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0efe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e80cee70 con 0x7ff0e80cabb0 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0eca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff0e80ce520 0x7ff0e80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0eca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff0e80ce520 0x7ff0e80ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45668/0 (socket says 172.21.15.66:45668) 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0eca00640 1 -- 172.21.15.66:0/601254125 learned_addr learned my addr 172.21.15.66:0/601254125 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0eca00640 1 -- 172.21.15.66:0/601254125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0e80cabb0 msgr2=0x7ff0e80cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0ede00640 1 --2- 172.21.15.66:0/601254125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0e80cabb0 0x7ff0e80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0eca00640 1 --2- 172.21.15.66:0/601254125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0e80cabb0 0x7ff0e80cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0eca00640 1 -- 172.21.15.66:0/601254125 >> v1:172.21.15.156:6789/0 conn(0x7ff0e815a640 legacy=0x7ff0e806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0eca00640 1 -- 172.21.15.66:0/601254125 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e80cb8c0 con 0x7ff0e80ce520 2024-03-30T00:42:18.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.896+0000 7ff0ede00640 1 --2- 172.21.15.66:0/601254125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0e80cabb0 0x7ff0e80cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:18.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0eca00640 1 --2- 172.21.15.66:0/601254125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e80ce520 0x7ff0e80ce930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009870 tx=0x7ff0d401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7dc5bc368c6902 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0e7e00640 1 -- 172.21.15.66:0/601254125 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40aa610 con 0x7ff0e80ce520 2024-03-30T00:42:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0e7e00640 1 -- 172.21.15.66:0/601254125 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d40a6040 con 0x7ff0e80ce520 2024-03-30T00:42:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0e7e00640 1 -- 172.21.15.66:0/601254125 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40aa990 con 0x7ff0e80ce520 2024-03-30T00:42:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 -- 172.21.15.66:0/601254125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e80ce520 msgr2=0x7ff0e80ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 --2- 172.21.15.66:0/601254125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e80ce520 0x7ff0e80ce930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009870 tx=0x7ff0d401c4f0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 -- 172.21.15.66:0/601254125 shutdown_connections 2024-03-30T00:42:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 --2- 172.21.15.66:0/601254125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0e80cabb0 0x7ff0e80cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 --2- 172.21.15.66:0/601254125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e80ce520 0x7ff0e80ce930 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 -- 172.21.15.66:0/601254125 >> 172.21.15.66:0/601254125 conn(0x7ff0e8069f60 msgr2=0x7ff0e806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 -- 172.21.15.66:0/601254125 shutdown_connections 2024-03-30T00:42:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 -- 172.21.15.66:0/601254125 wait complete. 2024-03-30T00:42:18.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 Processor -- start 2024-03-30T00:42:18.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 -- start start 2024-03-30T00:42:18.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e80cabb0 0x7ff0e815d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e80ce520 0x7ff0e80974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0e815a640 0x7ff0e80979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e8097f30 con 0x7ff0e80ce520 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e80aed60 con 0x7ff0e80cabb0 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0efe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e8070550 con 0x7ff0e815a640 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0eca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e80ce520 0x7ff0e80974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0eca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e80ce520 0x7ff0e80974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45676/0 (socket says 172.21.15.66:45676) 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0eca00640 1 -- 172.21.15.66:0/1503624560 learned_addr learned my addr 172.21.15.66:0/1503624560 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0ede00640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0e815a640 0x7ff0e80979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.900+0000 7ff0ed400640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e80cabb0 0x7ff0e815d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0eca00640 1 -- 172.21.15.66:0/1503624560 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0e815a640 msgr2=0x7ff0e80979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0eca00640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0e815a640 0x7ff0e80979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0eca00640 1 -- 172.21.15.66:0/1503624560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e80cabb0 msgr2=0x7ff0e815d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0eca00640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e80cabb0 0x7ff0e815d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0eca00640 1 -- 172.21.15.66:0/1503624560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d40094d0 con 0x7ff0e80ce520 2024-03-30T00:42:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0ed400640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e80cabb0 0x7ff0e815d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0eca00640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e80ce520 0x7ff0e80974b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40ac040 tx=0x7ff0d401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0e5600640 1 -- 172.21.15.66:0/1503624560 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40b3600 con 0x7ff0e80ce520 2024-03-30T00:42:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0e5600640 1 -- 172.21.15.66:0/1503624560 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d40b3780 con 0x7ff0e80ce520 2024-03-30T00:42:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0e5600640 1 -- 172.21.15.66:0/1503624560 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40b3b30 con 0x7ff0e80ce520 2024-03-30T00:42:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0efe00640 1 -- 172.21.15.66:0/1503624560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e80af430 con 0x7ff0e80ce520 2024-03-30T00:42:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0efe00640 1 -- 172.21.15.66:0/1503624560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e80707e0 con 0x7ff0e80ce520 2024-03-30T00:42:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0e5600640 1 -- 172.21.15.66:0/1503624560 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff0d40b3c90 con 0x7ff0e80ce520 2024-03-30T00:42:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0e5600640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0c80666c0 0x7ff0c8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0e5600640 1 -- 172.21.15.66:0/1503624560 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff0d411e900 con 0x7ff0e80ce520 2024-03-30T00:42:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.904+0000 7ff0ed400640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0c80666c0 0x7ff0c8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 Processor -- start 2024-03-30T00:42:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 -- start start 2024-03-30T00:42:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0ed400640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0c80666c0 0x7ff0c8068b80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0098b0 tx=0x7ff0dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705d210 0x558f7705d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 0x558f77013390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f770138d0 0x558f77013cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f76ed75a0 con 0x558f7705d210 2024-03-30T00:42:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f76e64af0 con 0x558f7705db30 2024-03-30T00:42:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f76e620c0 con 0x558f770138d0 2024-03-30T00:42:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0ed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705d210 0x558f7705d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0ed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705d210 0x558f7705d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45690/0 (socket says 172.21.15.66:45690) 2024-03-30T00:42:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0ed400640 1 -- 172.21.15.66:0/3015259531 learned_addr learned my addr 172.21.15.66:0/3015259531 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0ede00640 1 --2- 172.21.15.66:0/3015259531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f770138d0 0x558f77013cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0eca00640 1 --2- 172.21.15.66:0/3015259531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 0x558f77013390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0ed400640 1 -- 172.21.15.66:0/3015259531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f770138d0 msgr2=0x558f77013cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0ed400640 1 --2- 172.21.15.66:0/3015259531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f770138d0 0x558f77013cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0ed400640 1 -- 172.21.15.66:0/3015259531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 msgr2=0x558f77013390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0ed400640 1 --2- 172.21.15.66:0/3015259531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 0x558f77013390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0ed400640 1 -- 172.21.15.66:0/3015259531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f76b59890 con 0x558f7705d210 2024-03-30T00:42:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0eca00640 1 --2- 172.21.15.66:0/3015259531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 0x558f77013390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0ed400640 1 --2- 172.21.15.66:0/3015259531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705d210 0x558f7705d5f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00d8e0 tx=0x7ff0dc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed43e761d50bcdd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0da000640 1 -- 172.21.15.66:0/3015259531 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00de60 con 0x558f7705d210 2024-03-30T00:42:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0da000640 1 -- 172.21.15.66:0/3015259531 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc0094d0 con 0x558f7705d210 2024-03-30T00:42:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0da000640 1 -- 172.21.15.66:0/3015259531 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc014de0 con 0x558f7705d210 2024-03-30T00:42:18.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 -- 172.21.15.66:0/3015259531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705d210 msgr2=0x558f7705d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3015259531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705d210 0x558f7705d5f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00d8e0 tx=0x7ff0dc010000 comp rx=0 tx=0).stop 2024-03-30T00:42:18.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 -- 172.21.15.66:0/3015259531 shutdown_connections 2024-03-30T00:42:18.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3015259531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f770138d0 0x558f77013cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3015259531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 0x558f77013390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3015259531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705d210 0x558f7705d5f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 -- 172.21.15.66:0/3015259531 >> 172.21.15.66:0/3015259531 conn(0x558f77015830 msgr2=0x558f7700d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:18.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 -- 172.21.15.66:0/3015259531 shutdown_connections 2024-03-30T00:42:18.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 -- 172.21.15.66:0/3015259531 wait complete. 2024-03-30T00:42:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.908+0000 7ff0f3276000 1 -- start start 2024-03-30T00:42:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 1 Processor -- start 2024-03-30T00:42:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f770138d0 0x558f7704b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f7704b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 0x558f7704bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f76e5d9b0 con 0x558f770138d0 2024-03-30T00:42:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f76dfcc00 con 0x558f7705db30 2024-03-30T00:42:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f76e1b070 con 0x558f7705d210 2024-03-30T00:42:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 10 client.0 _ll_register_callbacks cb 0x558f76ffb0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:42:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0ed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f770138d0 0x558f7704b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0ed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f770138d0 0x558f7704b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45706/0 (socket says 172.21.15.66:45706) 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0ed400640 1 -- 172.21.15.66:0/3960873116 learned_addr learned my addr 172.21.15.66:0/3960873116 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0ed400640 1 -- 172.21.15.66:0/3960873116 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 msgr2=0x558f7704b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0eca00640 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f7704b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0ede00640 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 0x558f7704bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0ed400640 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f7704b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0ed400640 1 -- 172.21.15.66:0/3960873116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 msgr2=0x558f7704bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0ed400640 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 0x558f7704bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0ed400640 1 -- 172.21.15.66:0/3960873116 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0dc00e040 con 0x558f770138d0 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0ede00640 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 0x558f7704bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0eca00640 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f7704b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0ed400640 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f770138d0 0x558f7704b060 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc022040 tx=0x7ff0dc004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627b7b3127bd5c6d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0cea00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0cea00640 1 -- 172.21.15.66:0/3960873116 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc028410 con 0x558f770138d0 2024-03-30T00:42:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0cea00640 1 -- 172.21.15.66:0/3960873116 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff0dc028570 con 0x558f770138d0 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 1 -- 172.21.15.66:0/3960873116 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f76e1a5f0 con 0x558f770138d0 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0cea00640 1 -- 172.21.15.66:0/3960873116 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0dc028730 con 0x558f770138d0 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 10 client.6141 fetch_fsmap learned FSMap version 25 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 1 -- 172.21.15.66:0/3960873116 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x558f76e22ab0 con 0x558f770138d0 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0cea00640 1 -- 172.21.15.66:0/3960873116 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f76e1a5f0 con 0x558f770138d0 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0cea00640 1 -- 172.21.15.66:0/3960873116 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff0dc02e020 con 0x558f770138d0 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 10 client.6141 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 10 client.6141 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 20 client.6141 populate_metadata read hostname 'smithi066' 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x558f7704c3b0 0x558f7706e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 4 client.6141 mds_command: new command op to 5985 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0f3276000 1 -- 172.21.15.66:0/3960873116 --> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f76e5a790 con 0x558f7704c3b0 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0eca00640 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x558f7704c3b0 0x558f7706e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0eca00640 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x558f7704c3b0 0x558f7706e780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf258c5017aae418 server_cookie=a40b748bc7f5c848 in_seq=0 out_seq=0 2024-03-30T00:42:18.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.912+0000 7ff0cea00640 10 client.6141 ms_handle_connect on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0cea00640 1 -- 172.21.15.66:0/3960873116 <== mds.0 v2:172.21.15.66:6836/3557086525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558f76e5a790 con 0x558f7704c3b0 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0cea00640 10 client.6141 handle_command_reply: tid=0 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 1 client.6141 shutdown 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 1 -- 172.21.15.66:0/3960873116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f770138d0 msgr2=0x558f7704b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f770138d0 0x558f7704b060 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc022040 tx=0x7ff0dc004690 comp rx=0 tx=0).stop 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 1 -- 172.21.15.66:0/3960873116 shutdown_connections 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x558f7704c3b0 0x558f7706e780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f7705db30 0x558f7704bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f7704b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3960873116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f770138d0 0x558f7704b060 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 1 -- 172.21.15.66:0/3960873116 >> 172.21.15.66:0/3960873116 conn(0x558f77015830 msgr2=0x558f77014c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0cea00640 0 client.6141 ms_handle_reset on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:18.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 1 -- 172.21.15.66:0/3960873116 shutdown_connections 2024-03-30T00:42:18.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 1 -- 172.21.15.66:0/3960873116 wait complete. 2024-03-30T00:42:18.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.916+0000 7ff0f3276000 20 client.6141 trim_cache size 0 max 16384 2024-03-30T00:42:18.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 Processor -- start 2024-03-30T00:42:18.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 -- start start 2024-03-30T00:42:18.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f770138d0 0x558f7700d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f7700dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f77048990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f76b1a940 con 0x558f7705db30 2024-03-30T00:42:18.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f76e5c840 con 0x558f770138d0 2024-03-30T00:42:18.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f7704b960 con 0x558f7705d210 2024-03-30T00:42:18.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0ede00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f77048990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0ede00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f77048990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45716/0 (socket says 172.21.15.66:45716) 2024-03-30T00:42:18.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0ede00640 1 -- 172.21.15.66:0/3028705088 learned_addr learned my addr 172.21.15.66:0/3028705088 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:18.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0ed400640 1 --2- 172.21.15.66:0/3028705088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f770138d0 0x558f7700d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0eca00640 1 --2- 172.21.15.66:0/3028705088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f7700dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0ede00640 1 -- 172.21.15.66:0/3028705088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 msgr2=0x558f7700dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:18.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0ede00640 1 --2- 172.21.15.66:0/3028705088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f7700dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0ede00640 1 -- 172.21.15.66:0/3028705088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f770138d0 msgr2=0x558f7700d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:18.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0ede00640 1 --2- 172.21.15.66:0/3028705088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f770138d0 0x558f7700d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0ede00640 1 -- 172.21.15.66:0/3028705088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f76e08c00 con 0x558f7705db30 2024-03-30T00:42:18.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0eca00640 1 --2- 172.21.15.66:0/3028705088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f7700dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:18.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0ede00640 1 --2- 172.21.15.66:0/3028705088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f77048990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000d480 tx=0x7ff0e000d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7420c45ad5f512ab server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:18.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0daa00640 1 -- 172.21.15.66:0/3028705088 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0007bb0 con 0x558f7705db30 2024-03-30T00:42:18.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0daa00640 1 -- 172.21.15.66:0/3028705088 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e000c910 con 0x558f7705db30 2024-03-30T00:42:18.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0daa00640 1 -- 172.21.15.66:0/3028705088 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000ccc0 con 0x558f7705db30 2024-03-30T00:42:18.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 -- 172.21.15.66:0/3028705088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 msgr2=0x558f77048990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3028705088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f77048990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000d480 tx=0x7ff0e000d940 comp rx=0 tx=0).stop 2024-03-30T00:42:18.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 -- 172.21.15.66:0/3028705088 shutdown_connections 2024-03-30T00:42:18.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3028705088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f77048990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3028705088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f7700dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 --2- 172.21.15.66:0/3028705088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f770138d0 0x558f7700d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 -- 172.21.15.66:0/3028705088 >> 172.21.15.66:0/3028705088 conn(0x558f77015830 msgr2=0x558f7700cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 -- 172.21.15.66:0/3028705088 shutdown_connections 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 -- 172.21.15.66:0/3028705088 wait complete. 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.932+0000 7ff0f3276000 1 -- start start 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 Processor -- start 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f770138d0 0x558f7706fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f770b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f770b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f76de47c0 con 0x558f7705db30 2024-03-30T00:42:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f77006cd0 con 0x558f770138d0 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f76ea0ac0 con 0x558f7705d210 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 10 client.0 _ll_register_callbacks cb 0x558f76e22590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ede00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f770b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ede00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f770b7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45730/0 (socket says 172.21.15.66:45730) 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ede00640 1 -- 172.21.15.66:0/1208473252 learned_addr learned my addr 172.21.15.66:0/1208473252 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0eca00640 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f770b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ed400640 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f770138d0 0x558f7706fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ede00640 1 -- 172.21.15.66:0/1208473252 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 msgr2=0x558f770b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ede00640 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f770b7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ede00640 1 -- 172.21.15.66:0/1208473252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f770138d0 msgr2=0x558f7706fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ede00640 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f770138d0 0x558f7706fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ede00640 1 -- 172.21.15.66:0/1208473252 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e0007d10 con 0x558f7705db30 2024-03-30T00:42:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ede00640 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f770b7e40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff0e001b040 tx=0x7ff0e0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6f6b04aff993da server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0cea00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0cea00640 1 -- 172.21.15.66:0/1208473252 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0003db0 con 0x558f7705db30 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0cea00640 1 -- 172.21.15.66:0/1208473252 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff0e0004040 con 0x558f7705db30 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 -- 172.21.15.66:0/1208473252 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f76e2d460 con 0x558f7705db30 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0cea00640 1 -- 172.21.15.66:0/1208473252 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0e0004200 con 0x558f7705db30 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 10 client.6147 fetch_fsmap learned FSMap version 25 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 -- 172.21.15.66:0/1208473252 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x558f76b59cc0 con 0x558f7705db30 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0cea00640 1 -- 172.21.15.66:0/1208473252 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f76e2d460 con 0x558f7705db30 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0cea00640 1 -- 172.21.15.66:0/1208473252 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff0e0022040 con 0x558f7705db30 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 10 client.6147 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 10 client.6147 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 20 client.6147 populate_metadata read hostname 'smithi066' 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x558f770b8380 0x558f770b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:18.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 4 client.6147 mds_command: new command op to 5985 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 -- 172.21.15.66:0/1208473252 --> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558f76e269c0 con 0x558f770b8380 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ed400640 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x558f770b8380 0x558f770b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0ed400640 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x558f770b8380 0x558f770b8760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af93ba6b593cbb9 server_cookie=1e8eb4f55bda5076 in_seq=0 out_seq=0 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0cea00640 10 client.6147 ms_handle_connect on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0cea00640 1 -- 172.21.15.66:0/1208473252 <== mds.0 v2:172.21.15.66:6836/3557086525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x558f76e269c0 con 0x558f770b8380 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0cea00640 10 client.6147 handle_command_reply: tid=0 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.936+0000 7ff0f3276000 1 client.6147 shutdown 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0f3276000 1 -- 172.21.15.66:0/1208473252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 msgr2=0x558f770b7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0f3276000 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f770b7e40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff0e001b040 tx=0x7ff0e0004ac0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0f3276000 1 -- 172.21.15.66:0/1208473252 shutdown_connections 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0f3276000 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x558f770b8380 0x558f770b8760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0f3276000 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558f7705db30 0x558f770b7e40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0f3276000 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558f7705d210 0x558f770b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0f3276000 1 --2- 172.21.15.66:0/1208473252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558f770138d0 0x558f7706fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0f3276000 1 -- 172.21.15.66:0/1208473252 >> 172.21.15.66:0/1208473252 conn(0x558f77015830 msgr2=0x558f7700d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0cea00640 0 client.6147 ms_handle_reset on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0f3276000 1 -- 172.21.15.66:0/1208473252 shutdown_connections 2024-03-30T00:42:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0f3276000 1 -- 172.21.15.66:0/1208473252 wait complete. 2024-03-30T00:42:18.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0f3276000 20 client.6147 trim_cache size 0 max 16384 2024-03-30T00:42:18.954 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:42:18.954 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:42:18.954 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:42:18.954 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5985, 2024-03-30T00:42:18.954 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:42:18.955 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:42:18.955 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:42:18.955 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 10.671444298999999, 2024-03-30T00:42:18.955 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 25, 2024-03-30T00:42:18.955 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 54, 2024-03-30T00:42:18.955 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 54, 2024-03-30T00:42:18.955 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 15.024814704000001 2024-03-30T00:42:18.955 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:42:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 -- 172.21.15.66:0/1503624560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0c80666c0 msgr2=0x7ff0c8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0c80666c0 0x7ff0c8068b80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0098b0 tx=0x7ff0dc008040 comp rx=0 tx=0).stop 2024-03-30T00:42:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 -- 172.21.15.66:0/1503624560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e80ce520 msgr2=0x7ff0e80974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e80ce520 0x7ff0e80974b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40ac040 tx=0x7ff0d401ad30 comp rx=0 tx=0).stop 2024-03-30T00:42:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 -- 172.21.15.66:0/1503624560 shutdown_connections 2024-03-30T00:42:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0e815a640 0x7ff0e80979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff0c80666c0 0x7ff0c8068b80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0e80ce520 0x7ff0e80974b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 --2- 172.21.15.66:0/1503624560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e80cabb0 0x7ff0e815d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 -- 172.21.15.66:0/1503624560 >> 172.21.15.66:0/1503624560 conn(0x7ff0e8069f60 msgr2=0x7ff0e80ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:18.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 -- 172.21.15.66:0/1503624560 shutdown_connections 2024-03-30T00:42:18.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:18.940+0000 7ff0efe00640 1 -- 172.21.15.66:0/1503624560 wait complete. 2024-03-30T00:42:18.969 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:42:19.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab41400640 1 Processor -- start 2024-03-30T00:42:19.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab41400640 1 -- start start 2024-03-30T00:42:19.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab41400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fab3c0cc560 0x7fab3c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:19.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab41400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c06e800 0x7fab3c0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:19.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab41400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3c06b0b0 con 0x7fab3c06f8e0 2024-03-30T00:42:19.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab41400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3c160640 con 0x7fab3c0cc560 2024-03-30T00:42:19.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab41400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c06b390 con 0x7fab3c06e800 2024-03-30T00:42:19.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab3a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fab3c0cc560 0x7fab3c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:19.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab39e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fab3c06f8e0 0x7fab3c06e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53186/0 (socket says 172.21.15.66:53186) 2024-03-30T00:42:19.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab39e00640 1 -- 172.21.15.66:0/816768683 learned_addr learned my addr 172.21.15.66:0/816768683 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:19.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab3b200640 1 --2- 172.21.15.66:0/816768683 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c06e800 0x7fab3c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:19.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab3a800640 1 -- 172.21.15.66:0/816768683 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c06e800 msgr2=0x7fab3c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:19.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab3a800640 1 --2- 172.21.15.66:0/816768683 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c06e800 0x7fab3c0ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:19.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab3a800640 1 -- 172.21.15.66:0/816768683 >> v1:172.21.15.66:6789/0 conn(0x7fab3c06f8e0 legacy=0x7fab3c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:19.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab3a800640 1 -- 172.21.15.66:0/816768683 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c06b670 con 0x7fab3c0cc560 2024-03-30T00:42:19.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.100+0000 7fab39400640 1 -- 172.21.15.66:0/816768683 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485216176 0 0) 0x7fab3c06b0b0 con 0x7fab3c06f8e0 2024-03-30T00:42:19.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab3a800640 1 --2- 172.21.15.66:0/816768683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab3c0cc560 0x7fab3c0d1060 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0097c0 tx=0x7fab2c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7545e6962b356b8b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:19.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab39400640 1 -- 172.21.15.66:0/816768683 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0ab670 con 0x7fab3c0cc560 2024-03-30T00:42:19.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab39400640 1 -- 172.21.15.66:0/816768683 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c0ab7d0 con 0x7fab3c0cc560 2024-03-30T00:42:19.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab39400640 1 -- 172.21.15.66:0/816768683 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0abb60 con 0x7fab3c0cc560 2024-03-30T00:42:19.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab41400640 1 -- 172.21.15.66:0/816768683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab3c0cc560 msgr2=0x7fab3c0d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:19.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab41400640 1 --2- 172.21.15.66:0/816768683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab3c0cc560 0x7fab3c0d1060 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0097c0 tx=0x7fab2c01c470 comp rx=0 tx=0).stop 2024-03-30T00:42:19.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab41400640 1 -- 172.21.15.66:0/816768683 shutdown_connections 2024-03-30T00:42:19.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab41400640 1 --2- 172.21.15.66:0/816768683 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c06e800 0x7fab3c0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:19.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab41400640 1 --2- 172.21.15.66:0/816768683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab3c0cc560 0x7fab3c0d1060 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:19.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab41400640 1 -- 172.21.15.66:0/816768683 >> 172.21.15.66:0/816768683 conn(0x7fab3c0d0540 msgr2=0x7fab3c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:19.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab41400640 1 -- 172.21.15.66:0/816768683 shutdown_connections 2024-03-30T00:42:19.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab41400640 1 -- 172.21.15.66:0/816768683 wait complete. 2024-03-30T00:42:19.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab41400640 1 Processor -- start 2024-03-30T00:42:19.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.104+0000 7fab41400640 1 -- start start 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab41400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab3c06e800 0x7fab3c14e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab41400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c06f8e0 0x7fab3c14d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab41400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c0cc560 0x7fab3c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab41400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c14fb80 con 0x7fab3c06e800 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab41400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c14ffa0 con 0x7fab3c0cc560 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab41400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c14ef00 con 0x7fab3c06f8e0 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab3a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab3c06e800 0x7fab3c14e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab3a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab3c06e800 0x7fab3c14e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45742/0 (socket says 172.21.15.66:45742) 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab3a800640 1 -- 172.21.15.66:0/641138660 learned_addr learned my addr 172.21.15.66:0/641138660 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab3b200640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c0cc560 0x7fab3c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab39e00640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c06f8e0 0x7fab3c14d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab3a800640 1 -- 172.21.15.66:0/641138660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c06f8e0 msgr2=0x7fab3c14d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:19.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab3a800640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c06f8e0 0x7fab3c14d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab3a800640 1 -- 172.21.15.66:0/641138660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c0cc560 msgr2=0x7fab3c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab3a800640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c0cc560 0x7fab3c0c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab3a800640 1 -- 172.21.15.66:0/641138660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c009420 con 0x7fab3c06e800 2024-03-30T00:42:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab3b200640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c0cc560 0x7fab3c0c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab39e00640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c06f8e0 0x7fab3c14d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab3a800640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab3c06e800 0x7fab3c14e600 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fab2c01a0a0 tx=0x7fab2c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab32a00640 1 -- 172.21.15.66:0/641138660 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0b5810 con 0x7fab3c06e800 2024-03-30T00:42:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab32a00640 1 -- 172.21.15.66:0/641138660 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c0b5990 con 0x7fab3c06e800 2024-03-30T00:42:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab32a00640 1 -- 172.21.15.66:0/641138660 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0b5d40 con 0x7fab3c06e800 2024-03-30T00:42:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab41400640 1 -- 172.21.15.66:0/641138660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c14f1e0 con 0x7fab3c06e800 2024-03-30T00:42:19.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab41400640 1 -- 172.21.15.66:0/641138660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c14f7a0 con 0x7fab3c06e800 2024-03-30T00:42:19.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab41400640 1 -- 172.21.15.66:0/641138660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3c14fcc0 con 0x7fab3c06e800 2024-03-30T00:42:19.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.108+0000 7fab32a00640 1 -- 172.21.15.66:0/641138660 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fab2c0b4390 con 0x7fab3c06e800 2024-03-30T00:42:19.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.112+0000 7fab32a00640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab1c066690 0x7fab1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:19.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.112+0000 7fab32a00640 1 -- 172.21.15.66:0/641138660 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fab2c11ec90 con 0x7fab3c06e800 2024-03-30T00:42:19.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.112+0000 7fab39e00640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab1c066690 0x7fab1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:19.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.112+0000 7fab39e00640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab1c066690 0x7fab1c068b50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab3c15d270 tx=0x7fab28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:19.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.116+0000 7fab32a00640 1 -- 172.21.15.66:0/641138660 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fab3c14fcc0 con 0x7fab3c06e800 2024-03-30T00:42:19.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 -- 172.21.15.66:0/641138660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fab3c066c30 con 0x7fab3c06e800 2024-03-30T00:42:19.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab32a00640 1 -- 172.21.15.66:0/641138660 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fab2c0ebfe0 con 0x7fab3c06e800 2024-03-30T00:42:19.322 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:42:19.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 -- 172.21.15.66:0/641138660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab1c066690 msgr2=0x7fab1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:19.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab1c066690 0x7fab1c068b50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab3c15d270 tx=0x7fab28009040 comp rx=0 tx=0).stop 2024-03-30T00:42:19.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 -- 172.21.15.66:0/641138660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab3c06e800 msgr2=0x7fab3c14e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:19.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab3c06e800 0x7fab3c14e600 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fab2c01a0a0 tx=0x7fab2c01acb0 comp rx=0 tx=0).stop 2024-03-30T00:42:19.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 -- 172.21.15.66:0/641138660 shutdown_connections 2024-03-30T00:42:19.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c0cc560 0x7fab3c0c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:19.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c06f8e0 0x7fab3c14d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:19.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab1c066690 0x7fab1c068b50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:19.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 --2- 172.21.15.66:0/641138660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab3c06e800 0x7fab3c14e600 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:19.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 -- 172.21.15.66:0/641138660 >> 172.21.15.66:0/641138660 conn(0x7fab3c0d0540 msgr2=0x7fab3c06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:19.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 -- 172.21.15.66:0/641138660 shutdown_connections 2024-03-30T00:42:19.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:19.312+0000 7fab41400640 1 -- 172.21.15.66:0/641138660 wait complete. 2024-03-30T00:42:19.343 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:42:19.343 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:42:19.343 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:42:19.343 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:42:19.343 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:42:19.343 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:42:19.343 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:42:19.343 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:42:19.343 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:42:19.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:19.344 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:42:19.353 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:42:19.353 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:42:19.353 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:42:19.353 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:42:19.353 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:42:19.353 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42339sec preferred_lft 42339sec 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:42:19.354 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:42:19.355 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:42:19.356 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:42:19.356 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:42:19.356 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:42:19.356 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:42:19.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:19.356 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:42:19.414 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:42:19.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:19.416 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:42:19.468 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:42:19.468 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:42:19.469 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:42:19.469 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:42:19.469 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:42:19.469 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:42:19.469 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:42:19.469 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:42:19.469 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:42:19.469 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:42:19.469 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:42:19.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:19.470 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:42:19.470 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:42:19.470 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:42:19.470 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:42:19.470 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:42:19.470 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:42:19.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:19.567 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:42:19.576 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:42:19.576 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:42:19.576 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:42:19.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:19.577 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:42:19.644 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:42:19.644 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:42:19.644 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:42:19.692 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:42:19.692 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:42:19.692 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:42:19.744 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:42:19.809 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:42:19.809 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:42:19.870 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:42:19.872 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:42:19.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:19.872 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:42:19.924 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:42:19.924 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:42:19.970 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:42:19.991 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:42:19.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:42:20.051 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:42:20.052 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:42:20.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:20.053 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:42:20.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:42:20.052+0000 7fbabe449580 -1 init, newargv = 0x560c1d0b6c30 newargc=15 2024-03-30T00:42:20.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[40922]: starting ceph client 2024-03-30T00:42:20.072 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[40922]: starting fuse 2024-03-30T00:42:20.074 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:42:20.075 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:42:20.075 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:42:20.075 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:42:20.075 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:42:20.075 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:42:20.075 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:42:20.075 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:42:20.075 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:42:20.075 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:42:20.076 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:42:20.077 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:42:20.077 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:42:20.077 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:42:20.201 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.40922.asok 2024-03-30T00:42:20.385 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40922.asok 2024-03-30T00:42:20.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:20.385 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40922.asok status 2024-03-30T00:42:20.560 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:42:20.560 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "40922", 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6162, 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:42:20.561 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 6162 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1694584418 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:42:20.562 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1694584418 2024-03-30T00:42:20.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:20.563 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.6162 192.168.0.1:0/1694584418", 2024-03-30T00:42:20.563 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/1694584418", 2024-03-30T00:42:20.563 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:42:20.563 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 25, 2024-03-30T00:42:20.563 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 54, 2024-03-30T00:42:20.563 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:42:20.563 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:42:20.563 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:42:20.563 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:42:20.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:20.575 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:42:20.575 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:42:20.587 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:42:20.587 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:42:20.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:20.588 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:42:20.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:20.649 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-03-30T00:42:20.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:20.707 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-03-30T00:42:20.760 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:42:20.760 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:42:20.760 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:42:20.760 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:42:20.760 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-03-30T00:42:20.761 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:42:20.868 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627777 2024-03-30T00:42:21.058 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:42:21.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f7871800640 1 Processor -- start 2024-03-30T00:42:21.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f7871800640 1 -- start start 2024-03-30T00:42:21.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f7871800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f786c15de30 0x7f786c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f7871800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f786c0cee50 0x7f786c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f7871800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c0cbb90 con 0x7f786c0af420 2024-03-30T00:42:21.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f7871800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c0ce1b0 con 0x7f786c0cee50 2024-03-30T00:42:21.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f7871800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0afa10 con 0x7f786c15de30 2024-03-30T00:42:21.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f7862000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f786c0af420 0x7f786c0ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53188/0 (socket says 172.21.15.66:53188) 2024-03-30T00:42:21.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f7862000640 1 -- 172.21.15.66:0/754757094 learned_addr learned my addr 172.21.15.66:0/754757094 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:21.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f786b400640 1 --2- 172.21.15.66:0/754757094 >> v2:172.21.15.156:3300/0 conn(0x7f786c0cee50 0x7f786c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f786aa00640 1 --2- 172.21.15.66:0/754757094 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f786c15de30 0x7f786c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f786a000640 1 -- 172.21.15.66:0/754757094 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209974874 0 0) 0x7f786c0cbb90 con 0x7f786c0af420 2024-03-30T00:42:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f786a000640 1 -- 172.21.15.66:0/754757094 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7850003530 con 0x7f786c0af420 2024-03-30T00:42:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f786b400640 1 -- 172.21.15.66:0/754757094 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f786c15de30 msgr2=0x7f786c0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f786b400640 1 --2- 172.21.15.66:0/754757094 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f786c15de30 0x7f786c0cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f786b400640 1 -- 172.21.15.66:0/754757094 >> v1:172.21.15.66:6789/0 conn(0x7f786c0af420 legacy=0x7f786c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f786a000640 1 -- 172.21.15.66:0/754757094 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4071310292 0 0) 0x7f7850003530 con 0x7f786c0af420 2024-03-30T00:42:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.232+0000 7f786b400640 1 -- 172.21.15.66:0/754757094 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0cf430 con 0x7f786c0cee50 2024-03-30T00:42:21.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f786b400640 1 --2- 172.21.15.66:0/754757094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c0cee50 0x7f786c0d1060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f78640097c0 tx=0x7f786401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f53547a1252313b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:21.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f786a000640 1 -- 172.21.15.66:0/754757094 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640ab600 con 0x7f786c0cee50 2024-03-30T00:42:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f786a000640 1 -- 172.21.15.66:0/754757094 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78640a7050 con 0x7f786c0cee50 2024-03-30T00:42:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f786a000640 1 -- 172.21.15.66:0/754757094 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640ab980 con 0x7f786c0cee50 2024-03-30T00:42:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f7871800640 1 -- 172.21.15.66:0/754757094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c0cee50 msgr2=0x7f786c0d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f7871800640 1 --2- 172.21.15.66:0/754757094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c0cee50 0x7f786c0d1060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f78640097c0 tx=0x7f786401c410 comp rx=0 tx=0).stop 2024-03-30T00:42:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f7871800640 1 -- 172.21.15.66:0/754757094 shutdown_connections 2024-03-30T00:42:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f7871800640 1 --2- 172.21.15.66:0/754757094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c0cee50 0x7f786c0d1060 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f7871800640 1 --2- 172.21.15.66:0/754757094 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f786c15de30 0x7f786c0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f7871800640 1 -- 172.21.15.66:0/754757094 >> 172.21.15.66:0/754757094 conn(0x7f786c0d0540 msgr2=0x7f786c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f7871800640 1 -- 172.21.15.66:0/754757094 shutdown_connections 2024-03-30T00:42:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.236+0000 7f7871800640 1 -- 172.21.15.66:0/754757094 wait complete. 2024-03-30T00:42:21.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7871800640 1 Processor -- start 2024-03-30T00:42:21.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7871800640 1 -- start start 2024-03-30T00:42:21.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7871800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786c0af420 0x7f786c0c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7871800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f786c0cee50 0x7f786c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7871800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c15de30 0x7f786c15a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7871800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c159610 con 0x7f786c0af420 2024-03-30T00:42:21.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7871800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c159860 con 0x7f786c15de30 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7871800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c159b40 con 0x7f786c0cee50 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f786aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786c0af420 0x7f786c0c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f786aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786c0af420 0x7f786c0c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45750/0 (socket says 172.21.15.66:45750) 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f786aa00640 1 -- 172.21.15.66:0/3607013049 learned_addr learned my addr 172.21.15.66:0/3607013049 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7862000640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f786c0cee50 0x7f786c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f786b400640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c15de30 0x7f786c15a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f786aa00640 1 -- 172.21.15.66:0/3607013049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f786c0cee50 msgr2=0x7f786c0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f786aa00640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f786c0cee50 0x7f786c0b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f786aa00640 1 -- 172.21.15.66:0/3607013049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c15de30 msgr2=0x7f786c15a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f786aa00640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c15de30 0x7f786c15a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f786aa00640 1 -- 172.21.15.66:0/3607013049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864009420 con 0x7f786c0af420 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f786aa00640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786c0af420 0x7f786c0c7d50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f785c00efc0 tx=0x7f785c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7863400640 1 -- 172.21.15.66:0/3607013049 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c014480 con 0x7f786c0af420 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7863400640 1 -- 172.21.15.66:0/3607013049 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c00f040 con 0x7f786c0af420 2024-03-30T00:42:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7863400640 1 -- 172.21.15.66:0/3607013049 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c014800 con 0x7f786c0af420 2024-03-30T00:42:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7871800640 1 -- 172.21.15.66:0/3607013049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c159e20 con 0x7f786c0af420 2024-03-30T00:42:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.240+0000 7f7871800640 1 -- 172.21.15.66:0/3607013049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c0c8290 con 0x7f786c0af420 2024-03-30T00:42:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.244+0000 7f7871800640 1 -- 172.21.15.66:0/3607013049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c0cdf60 con 0x7f786c0af420 2024-03-30T00:42:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.244+0000 7f7863400640 1 -- 172.21.15.66:0/3607013049 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f785c014960 con 0x7f786c0af420 2024-03-30T00:42:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.244+0000 7f7863400640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7848066690 0x7f7848068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.244+0000 7f7863400640 1 -- 172.21.15.66:0/3607013049 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f785c084dc0 con 0x7f786c0af420 2024-03-30T00:42:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.244+0000 7f7862000640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7848066690 0x7f7848068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.248+0000 7f7862000640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7848066690 0x7f7848068b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7854007f60 tx=0x7f7854007ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:21.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.248+0000 7f7863400640 1 -- 172.21.15.66:0/3607013049 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f785c052000 con 0x7f786c0af420 2024-03-30T00:42:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 -- 172.21.15.66:0/3607013049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f786c066c30 con 0x7f786c0af420 2024-03-30T00:42:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7863400640 1 -- 172.21.15.66:0/3607013049 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f785c055620 con 0x7f786c0af420 2024-03-30T00:42:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:21.455 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/465916334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":465916334},{"type":"v1","addr":"172.21.15.156:6835","nonce":465916334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1640047258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1640047258},{"type":"v1","addr":"172.21.15.156:6834","nonce":1640047258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5788,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2112021115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2112021115},{"type":"v1","addr":"172.21.15.66:6835","nonce":2112021115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:42:08.269705+0000","modified":"2024-03-30T00:42:11.492352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5985},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5985":{"gid":5985,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/3557086525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3557086525},{"type":"v1","addr":"172.21.15.66:6837","nonce":3557086525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T00:42:21.455 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-03-30T00:42:21.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 -- 172.21.15.66:0/3607013049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7848066690 msgr2=0x7f7848068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7848066690 0x7f7848068b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7854007f60 tx=0x7f7854007ef0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 -- 172.21.15.66:0/3607013049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786c0af420 msgr2=0x7f786c0c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786c0af420 0x7f786c0c7d50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f785c00efc0 tx=0x7f785c00c490 comp rx=0 tx=0).stop 2024-03-30T00:42:21.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 -- 172.21.15.66:0/3607013049 shutdown_connections 2024-03-30T00:42:21.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c15de30 0x7f786c15a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f786c0cee50 0x7f786c0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7848066690 0x7f7848068b50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 --2- 172.21.15.66:0/3607013049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786c0af420 0x7f786c0c7d50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 -- 172.21.15.66:0/3607013049 >> 172.21.15.66:0/3607013049 conn(0x7f786c0d0540 msgr2=0x7f786c157bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:21.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 -- 172.21.15.66:0/3607013049 shutdown_connections 2024-03-30T00:42:21.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.444+0000 7f7871800640 1 -- 172.21.15.66:0/3607013049 wait complete. 2024-03-30T00:42:21.476 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-30T00:42:21.644 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-03-30T00:42:21.645 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-30T00:42:21.645 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-30T00:42:21.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.680+0000 7f7415c21080 1 Processor -- start 2024-03-30T00:42:21.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.680+0000 7f7415c21080 1 -- start start 2024-03-30T00:42:21.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.680+0000 7f7415c21080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73f8002270 0x5618483fed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.680+0000 7f7415c21080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 0x5618483557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.680+0000 7f7415c21080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5618483fd470 0x561848355ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.680+0000 7f7415c21080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618482d31e0 con 0x5618483fd090 2024-03-30T00:42:21.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.680+0000 7f7415c21080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184826e960 con 0x7f73f8002270 2024-03-30T00:42:21.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.680+0000 7f7415c21080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x56184826e600 con 0x5618483fd470 2024-03-30T00:42:21.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 0x5618483557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73f8002270 0x5618483fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 0x5618483557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45770/0 (socket says 172.21.15.66:45770) 2024-03-30T00:42:21.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412400640 1 -- 172.21.15.66:0/1870551765 learned_addr learned my addr 172.21.15.66:0/1870551765 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:21.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7413800640 1 --2- 172.21.15.66:0/1870551765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5618483fd470 0x561848355ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412e00640 1 -- 172.21.15.66:0/1870551765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5618483fd470 msgr2=0x561848355ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:21.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412e00640 1 --2- 172.21.15.66:0/1870551765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5618483fd470 0x561848355ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412e00640 1 -- 172.21.15.66:0/1870551765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 msgr2=0x5618483557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:21.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412e00640 1 --2- 172.21.15.66:0/1870551765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 0x5618483557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412e00640 1 -- 172.21.15.66:0/1870551765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561848270d00 con 0x7f73f8002270 2024-03-30T00:42:21.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412400640 1 --2- 172.21.15.66:0/1870551765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 0x5618483557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:42:21.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412e00640 1 --2- 172.21.15.66:0/1870551765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73f8002270 0x5618483fed00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f73f800a8f0 tx=0x7f7404001130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b32b8cab13b645c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:21.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7411000640 1 -- 172.21.15.66:0/1870551765 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74040aa770 con 0x7f73f8002270 2024-03-30T00:42:21.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7411000640 1 -- 172.21.15.66:0/1870551765 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74040aa8d0 con 0x7f73f8002270 2024-03-30T00:42:21.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7411000640 1 -- 172.21.15.66:0/1870551765 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74040aac80 con 0x7f73f8002270 2024-03-30T00:42:21.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 -- 172.21.15.66:0/1870551765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73f8002270 msgr2=0x5618483fed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 --2- 172.21.15.66:0/1870551765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73f8002270 0x5618483fed00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f73f800a8f0 tx=0x7f7404001130 comp rx=0 tx=0).stop 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 -- 172.21.15.66:0/1870551765 shutdown_connections 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 --2- 172.21.15.66:0/1870551765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5618483fd470 0x561848355ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 --2- 172.21.15.66:0/1870551765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 0x5618483557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 --2- 172.21.15.66:0/1870551765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73f8002270 0x5618483fed00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 -- 172.21.15.66:0/1870551765 >> 172.21.15.66:0/1870551765 conn(0x5618483572f0 msgr2=0x5618483564a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 -- 172.21.15.66:0/1870551765 shutdown_connections 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 -- 172.21.15.66:0/1870551765 wait complete. 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 Processor -- start 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 -- start start 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 0x5618483fb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5618483fd470 0x5618483f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73f8002270 0x5618483f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561848265690 con 0x5618483fd090 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618483fb910 con 0x7f73f8002270 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7415c21080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618483e3ac0 con 0x5618483fd470 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 0x5618483fb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 0x5618483fb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45772/0 (socket says 172.21.15.66:45772) 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412e00640 1 -- 172.21.15.66:0/2737752132 learned_addr learned my addr 172.21.15.66:0/2737752132 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7413800640 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73f8002270 0x5618483f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.684+0000 7f7412400640 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5618483fd470 0x5618483f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7412e00640 1 -- 172.21.15.66:0/2737752132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5618483fd470 msgr2=0x5618483f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7412e00640 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5618483fd470 0x5618483f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7412e00640 1 -- 172.21.15.66:0/2737752132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73f8002270 msgr2=0x5618483f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7412e00640 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73f8002270 0x5618483f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7412e00640 1 -- 172.21.15.66:0/2737752132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7400009550 con 0x5618483fd090 2024-03-30T00:42:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7412e00640 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 0x5618483fb3d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f740401b180 tx=0x7f73f80040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f740b400640 1 -- 172.21.15.66:0/2737752132 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74040c3070 con 0x5618483fd090 2024-03-30T00:42:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f740b400640 1 -- 172.21.15.66:0/2737752132 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7404001240 con 0x5618483fd090 2024-03-30T00:42:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f740b400640 1 -- 172.21.15.66:0/2737752132 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74040be690 con 0x5618483fd090 2024-03-30T00:42:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7415c21080 1 -- 172.21.15.66:0/2737752132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f8001e20 con 0x5618483fd090 2024-03-30T00:42:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7415c21080 1 -- 172.21.15.66:0/2737752132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561848270760 con 0x5618483fd090 2024-03-30T00:42:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f740b400640 1 -- 172.21.15.66:0/2737752132 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f74040be7f0 con 0x5618483fd090 2024-03-30T00:42:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f740b400640 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73f4065070 0x7f73f4067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f740b400640 1 -- 172.21.15.66:0/2737752132 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f740411edc0 con 0x5618483fd090 2024-03-30T00:42:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7412400640 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73f4065070 0x7f73f4067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7415c21080 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x561848497570 0x5618483e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7415c21080 1 -- 172.21.15.66:0/2737752132 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54) v8 -- 0x5618483f9be0 con 0x561848497570 2024-03-30T00:42:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.688+0000 7f7413800640 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x561848497570 0x5618483e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7412400640 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73f4065070 0x7f73f4067530 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7400005360 tx=0x7f740000c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7413800640 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x561848497570 0x5618483e2a50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7413800640 1 -- 172.21.15.66:0/2737752132 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f740c003640 con 0x561848497570 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7415c21080 1 -- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x561848497570 msgr2=0x5618483e2a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7415c21080 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x561848497570 0x5618483e2a50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7415c21080 1 -- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73f4065070 msgr2=0x7f73f4067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7415c21080 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73f4065070 0x7f73f4067530 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7400005360 tx=0x7f740000c040 comp rx=0 tx=0).stop 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7415c21080 1 -- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 msgr2=0x5618483fb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7415c21080 1 --2- 172.21.15.66:0/2737752132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618483fd090 0x5618483fb3d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f740401b180 tx=0x7f73f80040f0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7413800640 1 -- 172.21.15.66:0/2737752132 reap_dead start 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7415c21080 1 -- 172.21.15.66:0/2737752132 shutdown_connections 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7415c21080 1 -- 172.21.15.66:0/2737752132 >> 172.21.15.66:0/2737752132 conn(0x5618483572f0 msgr2=0x5618483e16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7415c21080 1 -- 172.21.15.66:0/2737752132 shutdown_connections 2024-03-30T00:42:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.692+0000 7f7415c21080 1 -- 172.21.15.66:0/2737752132 wait complete. 2024-03-30T00:42:21.706 INFO:teuthology.orchestra.run.smithi066.stdout:Yalphaparent_a 2024-03-30T00:42:21.707 DEBUG:teuthology.orchestra.run.smithi066:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-30T00:42:21.804 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:42:21.804 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627777, 2024-03-30T00:42:21.804 INFO:teuthology.orchestra.run.smithi066.stdout: "ancestors": [ 2024-03-30T00:42:21.804 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:21.804 INFO:teuthology.orchestra.run.smithi066.stdout: "dirino": 1099511627776, 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: "dname": "alpha", 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 2 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: "dirino": 1, 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: "dname": "parent_a", 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 4 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: "pool": 8, 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [] 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:42:21.805 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:21.809 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-03-30T00:42:21.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.888+0000 7f968ab70080 1 Processor -- start 2024-03-30T00:42:21.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.888+0000 7f968ab70080 1 -- start start 2024-03-30T00:42:21.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.888+0000 7f968ab70080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409db8e30 0x558409db76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.888+0000 7f968ab70080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2910 0x558409e5eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.888+0000 7f968ab70080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558409dd2cf0 0x558409ef33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.888+0000 7f968ab70080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409d331e0 con 0x558409dd2910 2024-03-30T00:42:21.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.888+0000 7f968ab70080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409cce960 con 0x558409db8e30 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.888+0000 7f968ab70080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558409cce600 con 0x558409dd2cf0 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.888+0000 7f9687400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2910 0x558409e5eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2910 0x558409e5eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45792/0 (socket says 172.21.15.66:45792) 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687400640 1 -- 172.21.15.66:0/1868718004 learned_addr learned my addr 172.21.15.66:0/1868718004 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687e00640 1 --2- 172.21.15.66:0/1868718004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409db8e30 0x558409db76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9688800640 1 --2- 172.21.15.66:0/1868718004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558409dd2cf0 0x558409ef33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687400640 1 -- 172.21.15.66:0/1868718004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558409dd2cf0 msgr2=0x558409ef33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687400640 1 --2- 172.21.15.66:0/1868718004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558409dd2cf0 0x558409ef33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687400640 1 -- 172.21.15.66:0/1868718004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409db8e30 msgr2=0x558409db76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687400640 1 --2- 172.21.15.66:0/1868718004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409db8e30 0x558409db76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687400640 1 -- 172.21.15.66:0/1868718004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968000a100 con 0x558409dd2910 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687400640 1 --2- 172.21.15.66:0/1868718004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2910 0x558409e5eed0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9674001cf0 tx=0x7f967400b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2a49619bc71f65 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:21.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9686000640 1 -- 172.21.15.66:0/1868718004 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674014070 con 0x558409dd2910 2024-03-30T00:42:21.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9686000640 1 -- 172.21.15.66:0/1868718004 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967400f410 con 0x558409dd2910 2024-03-30T00:42:21.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9686000640 1 -- 172.21.15.66:0/1868718004 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967400f800 con 0x558409dd2910 2024-03-30T00:42:21.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 -- 172.21.15.66:0/1868718004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2910 msgr2=0x558409e5eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 --2- 172.21.15.66:0/1868718004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2910 0x558409e5eed0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9674001cf0 tx=0x7f967400b5c0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 -- 172.21.15.66:0/1868718004 shutdown_connections 2024-03-30T00:42:21.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 --2- 172.21.15.66:0/1868718004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558409dd2cf0 0x558409ef33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 --2- 172.21.15.66:0/1868718004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2910 0x558409e5eed0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 --2- 172.21.15.66:0/1868718004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409db8e30 0x558409db76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 -- 172.21.15.66:0/1868718004 >> 172.21.15.66:0/1868718004 conn(0x558409db72f0 msgr2=0x558409e43e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:21.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 -- 172.21.15.66:0/1868718004 shutdown_connections 2024-03-30T00:42:21.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 -- 172.21.15.66:0/1868718004 wait complete. 2024-03-30T00:42:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 Processor -- start 2024-03-30T00:42:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 -- start start 2024-03-30T00:42:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558409db8e30 0x558409e5a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2cf0 0x558409e3f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409e58f20 0x558409ef9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409cc5690 con 0x558409dd2cf0 2024-03-30T00:42:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409e593c0 con 0x558409e58f20 2024-03-30T00:42:21.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f968ab70080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558409e59660 con 0x558409db8e30 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2cf0 0x558409e3f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2cf0 0x558409e3f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45802/0 (socket says 172.21.15.66:45802) 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687400640 1 -- 172.21.15.66:0/1987721351 learned_addr learned my addr 172.21.15.66:0/1987721351 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9688800640 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409e58f20 0x558409ef9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.892+0000 7f9687e00640 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558409db8e30 0x558409e5a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9687400640 1 -- 172.21.15.66:0/1987721351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558409db8e30 msgr2=0x558409e5a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9687400640 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558409db8e30 0x558409e5a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9687400640 1 -- 172.21.15.66:0/1987721351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409e58f20 msgr2=0x558409ef9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9687400640 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409e58f20 0x558409ef9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9687400640 1 -- 172.21.15.66:0/1987721351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558409cf7a60 con 0x558409dd2cf0 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9687400640 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2cf0 0x558409e3f420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f967400b600 tx=0x7f9674000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9684c00640 1 -- 172.21.15.66:0/1987721351 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674014040 con 0x558409dd2cf0 2024-03-30T00:42:21.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9684c00640 1 -- 172.21.15.66:0/1987721351 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967401b840 con 0x558409dd2cf0 2024-03-30T00:42:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9684c00640 1 -- 172.21.15.66:0/1987721351 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967401bbf0 con 0x558409dd2cf0 2024-03-30T00:42:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f968ab70080 1 -- 172.21.15.66:0/1987721351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558409cd0d00 con 0x558409dd2cf0 2024-03-30T00:42:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f968ab70080 1 -- 172.21.15.66:0/1987721351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558409cd0760 con 0x558409dd2cf0 2024-03-30T00:42:21.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9684c00640 1 -- 172.21.15.66:0/1987721351 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f967401bd50 con 0x558409dd2cf0 2024-03-30T00:42:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9684c00640 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96680650a0 0x7f9668067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9684c00640 1 -- 172.21.15.66:0/1987721351 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9674001e40 con 0x558409dd2cf0 2024-03-30T00:42:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9687e00640 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96680650a0 0x7f9668067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f968ab70080 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x558409e3f960 0x558409f036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f968ab70080 1 -- 172.21.15.66:0/1987721351 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) v8 -- 0x558409ef7e70 con 0x558409e3f960 2024-03-30T00:42:21.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.896+0000 7f9688800640 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x558409e3f960 0x558409f036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:21.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f9687e00640 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96680650a0 0x7f9668067560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9678000c10 tx=0x7f9678010040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:21.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f9688800640 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x558409e3f960 0x558409f036c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f9688800640 1 -- 172.21.15.66:0/1987721351 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v54'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f96800aa890 con 0x558409e3f960 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f968ab70080 1 -- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x558409e3f960 msgr2=0x558409f036c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f968ab70080 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x558409e3f960 0x558409f036c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f968ab70080 1 -- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96680650a0 msgr2=0x7f9668067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f968ab70080 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96680650a0 0x7f9668067560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9678000c10 tx=0x7f9678010040 comp rx=0 tx=0).stop 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f968ab70080 1 -- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2cf0 msgr2=0x558409e3f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f968ab70080 1 --2- 172.21.15.66:0/1987721351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558409dd2cf0 0x558409e3f420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f967400b600 tx=0x7f9674000bd0 comp rx=0 tx=0).stop 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f9688800640 1 -- 172.21.15.66:0/1987721351 reap_dead start 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f968ab70080 1 -- 172.21.15.66:0/1987721351 shutdown_connections 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f968ab70080 1 -- 172.21.15.66:0/1987721351 >> 172.21.15.66:0/1987721351 conn(0x558409db72f0 msgr2=0x558409e5eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f968ab70080 1 -- 172.21.15.66:0/1987721351 shutdown_connections 2024-03-30T00:42:21.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:21.900+0000 7f968ab70080 1 -- 172.21.15.66:0/1987721351 wait complete. 2024-03-30T00:42:21.915 DEBUG:teuthology.orchestra.run.smithi066:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-30T00:42:21.918 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-03-30T00:42:22.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.052+0000 7f0b85800640 1 Processor -- start 2024-03-30T00:42:22.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.052+0000 7f0b85800640 1 -- start start 2024-03-30T00:42:22.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.052+0000 7f0b85800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b800b1280 0x7f0b800cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.052+0000 7f0b85800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b80098a20 0x7f0b80098e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.052+0000 7f0b85800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b800ce930 con 0x7f0b800b0b20 2024-03-30T00:42:22.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.052+0000 7f0b85800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b80099390 con 0x7f0b80098a20 2024-03-30T00:42:22.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.052+0000 7f0b85800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80099800 con 0x7f0b800b1280 2024-03-30T00:42:22.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b7f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b80098a20 0x7f0b80098e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b7f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b80098a20 0x7f0b80098e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45818/0 (socket says 172.21.15.66:45818) 2024-03-30T00:42:22.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b7f400640 1 -- 172.21.15.66:0/2129006290 learned_addr learned my addr 172.21.15.66:0/2129006290 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b76000640 1 --2- 172.21.15.66:0/2129006290 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b800b1280 0x7f0b800cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b7f400640 1 -- 172.21.15.66:0/2129006290 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b800b1280 msgr2=0x7f0b800cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b7f400640 1 --2- 172.21.15.66:0/2129006290 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b800b1280 0x7f0b800cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b7f400640 1 -- 172.21.15.66:0/2129006290 >> v1:172.21.15.156:6789/0 conn(0x7f0b800b0b20 legacy=0x7f0b8006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b7f400640 1 -- 172.21.15.66:0/2129006290 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b80099d50 con 0x7f0b80098a20 2024-03-30T00:42:22.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b7f400640 1 --2- 172.21.15.66:0/2129006290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b80098a20 0x7f0b80098e50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009870 tx=0x7f0b7801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bc0e2d61a0d075 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b7e000640 1 -- 172.21.15.66:0/2129006290 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b780ab600 con 0x7f0b80098a20 2024-03-30T00:42:22.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b7e000640 1 -- 172.21.15.66:0/2129006290 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b780a7050 con 0x7f0b80098a20 2024-03-30T00:42:22.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.056+0000 7f0b7e000640 1 -- 172.21.15.66:0/2129006290 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b780ab980 con 0x7f0b80098a20 2024-03-30T00:42:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 -- 172.21.15.66:0/2129006290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b80098a20 msgr2=0x7f0b80098e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 --2- 172.21.15.66:0/2129006290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b80098a20 0x7f0b80098e50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009870 tx=0x7f0b7801c560 comp rx=0 tx=0).stop 2024-03-30T00:42:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 -- 172.21.15.66:0/2129006290 shutdown_connections 2024-03-30T00:42:22.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 --2- 172.21.15.66:0/2129006290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b80098a20 0x7f0b80098e50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 --2- 172.21.15.66:0/2129006290 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b800b1280 0x7f0b800cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 -- 172.21.15.66:0/2129006290 >> 172.21.15.66:0/2129006290 conn(0x7f0b800b2150 msgr2=0x7f0b800b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 -- 172.21.15.66:0/2129006290 shutdown_connections 2024-03-30T00:42:22.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 -- 172.21.15.66:0/2129006290 wait complete. 2024-03-30T00:42:22.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 Processor -- start 2024-03-30T00:42:22.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 -- start start 2024-03-30T00:42:22.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b80098a20 0x7f0b80154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b800b0b20 0x7f0b800c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b800b1280 0x7f0b800c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800afee0 con 0x7f0b800b1280 2024-03-30T00:42:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80158290 con 0x7f0b800b0b20 2024-03-30T00:42:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b85800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80158570 con 0x7f0b80098a20 2024-03-30T00:42:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b7f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b800b1280 0x7f0b800c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b7f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b800b1280 0x7f0b800c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45832/0 (socket says 172.21.15.66:45832) 2024-03-30T00:42:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b7f400640 1 -- 172.21.15.66:0/2479590129 learned_addr learned my addr 172.21.15.66:0/2479590129 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b76000640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b800b0b20 0x7f0b800c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b7ea00640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b80098a20 0x7f0b80154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b7f400640 1 -- 172.21.15.66:0/2479590129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b80098a20 msgr2=0x7f0b80154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b7f400640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b80098a20 0x7f0b80154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b7f400640 1 -- 172.21.15.66:0/2479590129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b800b0b20 msgr2=0x7f0b800c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b7f400640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b800b0b20 0x7f0b800c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.060+0000 7f0b7f400640 1 -- 172.21.15.66:0/2479590129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780094d0 con 0x7f0b800b1280 2024-03-30T00:42:22.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.064+0000 7f0b7f400640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b800b1280 0x7f0b800c4690 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0b7801a190 tx=0x7f0b7801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.064+0000 7f0b77400640 1 -- 172.21.15.66:0/2479590129 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b780b6620 con 0x7f0b800b1280 2024-03-30T00:42:22.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.064+0000 7f0b77400640 1 -- 172.21.15.66:0/2479590129 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b780b6780 con 0x7f0b800b1280 2024-03-30T00:42:22.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.064+0000 7f0b77400640 1 -- 172.21.15.66:0/2479590129 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b780b6b30 con 0x7f0b800b1280 2024-03-30T00:42:22.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.064+0000 7f0b85800640 1 -- 172.21.15.66:0/2479590129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b80158850 con 0x7f0b800b1280 2024-03-30T00:42:22.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.064+0000 7f0b85800640 1 -- 172.21.15.66:0/2479590129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b80158e10 con 0x7f0b800b1280 2024-03-30T00:42:22.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.064+0000 7f0b77400640 1 -- 172.21.15.66:0/2479590129 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0b780b2040 con 0x7f0b800b1280 2024-03-30T00:42:22.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.064+0000 7f0b77400640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b5c066690 0x7f0b5c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.064+0000 7f0b77400640 1 -- 172.21.15.66:0/2479590129 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0b7811e640 con 0x7f0b800b1280 2024-03-30T00:42:22.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.064+0000 7f0b7ea00640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b5c066690 0x7f0b5c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.064+0000 7f0b88cf2000 1 Processor -- start 2024-03-30T00:42:22.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 -- start start 2024-03-30T00:42:22.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7ea00640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b5c066690 0x7f0b5c068b50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0b70007fd0 tx=0x7f0b70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9194d30 0x5644e9195110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9195650 0x5644e914ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e914c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e9013170 con 0x5644e914bf60 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e8fa0d80 con 0x5644e9195650 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644e8f9e350 con 0x5644e9194d30 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e914c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e914c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45840/0 (socket says 172.21.15.66:45840) 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7f400640 1 -- 172.21.15.66:0/101371418 learned_addr learned my addr 172.21.15.66:0/101371418 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b76000640 1 --2- 172.21.15.66:0/101371418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9195650 0x5644e914ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7ea00640 1 --2- 172.21.15.66:0/101371418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9194d30 0x5644e9195110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7f400640 1 -- 172.21.15.66:0/101371418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9194d30 msgr2=0x5644e9195110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7f400640 1 --2- 172.21.15.66:0/101371418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9194d30 0x5644e9195110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7f400640 1 -- 172.21.15.66:0/101371418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9195650 msgr2=0x5644e914ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7f400640 1 --2- 172.21.15.66:0/101371418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9195650 0x5644e914ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7f400640 1 -- 172.21.15.66:0/101371418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644e8c96070 con 0x5644e914bf60 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7ea00640 1 --2- 172.21.15.66:0/101371418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9194d30 0x5644e9195110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b7f400640 1 --2- 172.21.15.66:0/101371418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e914c340 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0b7801cad0 tx=0x7f0b780ef4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fc9b47ea7ccc18 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b6f400640 1 -- 172.21.15.66:0/101371418 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b780f3120 con 0x5644e914bf60 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b6f400640 1 -- 172.21.15.66:0/101371418 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b780f7910 con 0x5644e914bf60 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b6f400640 1 -- 172.21.15.66:0/101371418 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b780ed970 con 0x5644e914bf60 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 -- 172.21.15.66:0/101371418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 msgr2=0x5644e914c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/101371418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e914c340 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0b7801cad0 tx=0x7f0b780ef4d0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 -- 172.21.15.66:0/101371418 shutdown_connections 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/101371418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e914c340 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/101371418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9195650 0x5644e914ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/101371418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9194d30 0x5644e9195110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 -- 172.21.15.66:0/101371418 >> 172.21.15.66:0/101371418 conn(0x5644e9147f10 msgr2=0x5644e914a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 -- 172.21.15.66:0/101371418 shutdown_connections 2024-03-30T00:42:22.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 -- 172.21.15.66:0/101371418 wait complete. 2024-03-30T00:42:22.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 -- start start 2024-03-30T00:42:22.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.068+0000 7f0b88cf2000 1 Processor -- start 2024-03-30T00:42:22.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e9151940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9194d30 0x5644e9182a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9195650 0x5644e9182f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e8f99c40 con 0x5644e914bf60 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e8f38fa0 con 0x5644e9194d30 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644e8f571c0 con 0x5644e9195650 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 10 client.0 _ll_register_callbacks cb 0x5644e9137060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b7ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e9151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b7ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e9151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45856/0 (socket says 172.21.15.66:45856) 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b7ea00640 1 -- 172.21.15.66:0/2536673791 learned_addr learned my addr 172.21.15.66:0/2536673791 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b76000640 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9194d30 0x5644e9182a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b7f400640 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9195650 0x5644e9182f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b7ea00640 1 -- 172.21.15.66:0/2536673791 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9195650 msgr2=0x5644e9182f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b7ea00640 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9195650 0x5644e9182f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b7ea00640 1 -- 172.21.15.66:0/2536673791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9194d30 msgr2=0x5644e9182a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b7ea00640 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9194d30 0x5644e9182a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b7ea00640 1 -- 172.21.15.66:0/2536673791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b80158e10 con 0x5644e914bf60 2024-03-30T00:42:22.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b76000640 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9194d30 0x5644e9182a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:22.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b7f400640 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9195650 0x5644e9182f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:22.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b7ea00640 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e9151940 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0b700147e0 tx=0x7f0b70001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99882f381189b737 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b63e00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:42:22.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b63e00640 1 -- 172.21.15.66:0/2536673791 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70001d60 con 0x5644e914bf60 2024-03-30T00:42:22.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b63e00640 1 -- 172.21.15.66:0/2536673791 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0b7001c040 con 0x5644e914bf60 2024-03-30T00:42:22.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2536673791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644e8f96a20 con 0x5644e914bf60 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b63e00640 1 -- 172.21.15.66:0/2536673791 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b70001450 con 0x5644e914bf60 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 10 client.6201 fetch_fsmap learned FSMap version 25 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2536673791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5644e8f5eba0 con 0x5644e914bf60 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b63e00640 1 -- 172.21.15.66:0/2536673791 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5644e8f96a20 con 0x5644e914bf60 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b63e00640 1 -- 172.21.15.66:0/2536673791 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0b70012070 con 0x5644e914bf60 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 10 client.6201 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 10 client.6201 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 20 client.6201 populate_metadata read hostname 'smithi066' 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5644e919bca0 0x5644e919c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 4 client.6201 mds_command: new command op to 5985 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2536673791 --> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644e8f98590 con 0x5644e919bca0 2024-03-30T00:42:22.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b76000640 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5644e919bca0 0x5644e919c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b76000640 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5644e919bca0 0x5644e919c0b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c23ffc3ae883bd server_cookie=9119c2aa2012ebd in_seq=0 out_seq=0 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.072+0000 7f0b63e00640 10 client.6201 ms_handle_connect on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b63e00640 1 -- 172.21.15.66:0/2536673791 <== mds.0 v2:172.21.15.66:6836/3557086525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5644e8f98590 con 0x5644e919bca0 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b63e00640 10 client.6201 handle_command_reply: tid=0 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 1 client.6201 shutdown 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2536673791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 msgr2=0x5644e9151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e9151940 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0b700147e0 tx=0x7f0b70001090 comp rx=0 tx=0).stop 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2536673791 shutdown_connections 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5644e919bca0 0x5644e919c0b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e9195650 0x5644e9182f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9194d30 0x5644e9182a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b63e00640 0 client.6201 ms_handle_reset on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:22.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/2536673791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e914bf60 0x5644e9151940 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2536673791 >> 172.21.15.66:0/2536673791 conn(0x5644e9147f10 msgr2=0x5644e914ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2536673791 shutdown_connections 2024-03-30T00:42:22.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2536673791 wait complete. 2024-03-30T00:42:22.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.076+0000 7f0b88cf2000 20 client.6201 trim_cache size 0 max 16384 2024-03-30T00:42:22.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671f400640 1 Processor -- start 2024-03-30T00:42:22.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671f400640 1 -- start start 2024-03-30T00:42:22.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f671815a690 0x7f67180ae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6718071330 0x7f671806ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671f400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f671806b2e0 0x7f671815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671f400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671806b750 con 0x7f671815a690 2024-03-30T00:42:22.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671f400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67180b2450 con 0x7f671806b2e0 2024-03-30T00:42:22.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6718066180 con 0x7f6718071330 2024-03-30T00:42:22.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671ca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f671815a690 0x7f67180ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671ca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f671815a690 0x7f67180ae270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45870/0 (socket says 172.21.15.66:45870) 2024-03-30T00:42:22.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671ca00640 1 -- 172.21.15.66:0/3994579381 learned_addr learned my addr 172.21.15.66:0/3994579381 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671d400640 1 --2- 172.21.15.66:0/3994579381 >> v2:172.21.15.156:3300/0 conn(0x7f671806b2e0 0x7f671815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f6717e00640 1 --2- 172.21.15.66:0/3994579381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6718071330 0x7f671806ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671ca00640 1 -- 172.21.15.66:0/3994579381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6718071330 msgr2=0x7f671806ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671ca00640 1 --2- 172.21.15.66:0/3994579381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6718071330 0x7f671806ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671ca00640 1 -- 172.21.15.66:0/3994579381 >> v2:172.21.15.156:3300/0 conn(0x7f671806b2e0 msgr2=0x7f671815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671ca00640 1 --2- 172.21.15.66:0/3994579381 >> v2:172.21.15.156:3300/0 conn(0x7f671806b2e0 0x7f671815e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.080+0000 7f671ca00640 1 -- 172.21.15.66:0/3994579381 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67180718f0 con 0x7f671815a690 2024-03-30T00:42:22.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f671ca00640 1 --2- 172.21.15.66:0/3994579381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f671815a690 0x7f67180ae270 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6718072790 tx=0x7f670c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb2a9f7cca27e9c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f6717400640 1 -- 172.21.15.66:0/3994579381 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c0ab600 con 0x7f671815a690 2024-03-30T00:42:22.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f6717400640 1 -- 172.21.15.66:0/3994579381 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c0a7050 con 0x7f671815a690 2024-03-30T00:42:22.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f6717400640 1 -- 172.21.15.66:0/3994579381 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c0ab980 con 0x7f671815a690 2024-03-30T00:42:22.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f671f400640 1 -- 172.21.15.66:0/3994579381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f671815a690 msgr2=0x7f67180ae270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f671f400640 1 --2- 172.21.15.66:0/3994579381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f671815a690 0x7f67180ae270 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6718072790 tx=0x7f670c01c260 comp rx=0 tx=0).stop 2024-03-30T00:42:22.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f671f400640 1 -- 172.21.15.66:0/3994579381 shutdown_connections 2024-03-30T00:42:22.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f671f400640 1 --2- 172.21.15.66:0/3994579381 >> v2:172.21.15.156:3300/0 conn(0x7f671806b2e0 0x7f671815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f671f400640 1 --2- 172.21.15.66:0/3994579381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6718071330 0x7f671806ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f671f400640 1 --2- 172.21.15.66:0/3994579381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f671815a690 0x7f67180ae270 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f671f400640 1 -- 172.21.15.66:0/3994579381 >> 172.21.15.66:0/3994579381 conn(0x7f6718070810 msgr2=0x7f6718070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f671f400640 1 -- 172.21.15.66:0/3994579381 shutdown_connections 2024-03-30T00:42:22.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.084+0000 7f671f400640 1 -- 172.21.15.66:0/3994579381 wait complete. 2024-03-30T00:42:22.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671f400640 1 Processor -- start 2024-03-30T00:42:22.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671f400640 1 -- start start 2024-03-30T00:42:22.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f671806b2e0 0x7f6718148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6718071330 0x7f6718158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f671815a690 0x7f6718158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671814d3f0 con 0x7f671815a690 2024-03-30T00:42:22.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671f400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671815c710 con 0x7f671806b2e0 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671815bde0 con 0x7f6718071330 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f671815a690 0x7f6718158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671ca00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f671806b2e0 0x7f6718148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f671815a690 0x7f6718158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45886/0 (socket says 172.21.15.66:45886) 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671d400640 1 -- 172.21.15.66:0/2559199436 learned_addr learned my addr 172.21.15.66:0/2559199436 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f6717e00640 1 --2- 172.21.15.66:0/2559199436 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6718071330 0x7f6718158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671d400640 1 -- 172.21.15.66:0/2559199436 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6718071330 msgr2=0x7f6718158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671d400640 1 --2- 172.21.15.66:0/2559199436 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6718071330 0x7f6718158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671d400640 1 -- 172.21.15.66:0/2559199436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f671806b2e0 msgr2=0x7f6718148a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671d400640 1 --2- 172.21.15.66:0/2559199436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f671806b2e0 0x7f6718148a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671d400640 1 -- 172.21.15.66:0/2559199436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c0094d0 con 0x7f671815a690 2024-03-30T00:42:22.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671d400640 1 --2- 172.21.15.66:0/2559199436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f671815a690 0x7f6718158d50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f671000eed0 tx=0x7f671000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f6714c00640 1 -- 172.21.15.66:0/2559199436 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710010040 con 0x7f671815a690 2024-03-30T00:42:22.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f6714c00640 1 -- 172.21.15.66:0/2559199436 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671000bd20 con 0x7f671815a690 2024-03-30T00:42:22.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671f400640 1 -- 172.21.15.66:0/2559199436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67180b2a60 con 0x7f671815a690 2024-03-30T00:42:22.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f6714c00640 1 -- 172.21.15.66:0/2559199436 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710014640 con 0x7f671815a690 2024-03-30T00:42:22.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671f400640 1 -- 172.21.15.66:0/2559199436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671815c3a0 con 0x7f671815a690 2024-03-30T00:42:22.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.088+0000 7f671f400640 1 -- 172.21.15.66:0/2559199436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f6718066c30 con 0x7f671815a690 2024-03-30T00:42:22.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f6714c00640 1 -- 172.21.15.66:0/2559199436 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f67100147a0 con 0x7f671815a690 2024-03-30T00:42:22.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f0b88cf2000 1 Processor -- start 2024-03-30T00:42:22.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f6714c00640 1 --2- 172.21.15.66:0/2559199436 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f66f8066690 0x7f66f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f0b88cf2000 1 -- start start 2024-03-30T00:42:22.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f6714c00640 1 -- 172.21.15.66:0/2559199436 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6710085480 con 0x7f671815a690 2024-03-30T00:42:22.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f671ca00640 1 --2- 172.21.15.66:0/2559199436 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f66f8066690 0x7f66f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f6714c00640 1 -- 172.21.15.66:0/2559199436 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7f6710085810 con 0x7f671815a690 2024-03-30T00:42:22.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f671f400640 1 -- 172.21.15.66:0/2559199436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f67180b4d00 con 0x7f671815a690 2024-03-30T00:42:22.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f671ca00640 1 --2- 172.21.15.66:0/2559199436 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f66f8066690 0x7f66f8068b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f670c0ad000 tx=0x7f670c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e914bf60 0x5644e91940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9194d30 0x5644e9194620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9195650 0x5644e9197a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e8c56cf0 con 0x5644e9194d30 2024-03-30T00:42:22.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e9151990 con 0x5644e9195650 2024-03-30T00:42:22.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.092+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644e8f9c7a0 con 0x5644e914bf60 2024-03-30T00:42:22.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b76000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9194d30 0x5644e9194620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b76000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9194d30 0x5644e9194620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45892/0 (socket says 172.21.15.66:45892) 2024-03-30T00:42:22.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b76000640 1 -- 172.21.15.66:0/2378142295 learned_addr learned my addr 172.21.15.66:0/2378142295 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b76000640 1 -- 172.21.15.66:0/2378142295 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e914bf60 msgr2=0x5644e91940e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b76000640 1 --2- 172.21.15.66:0/2378142295 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e914bf60 0x5644e91940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b76000640 1 -- 172.21.15.66:0/2378142295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9195650 msgr2=0x5644e9197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b76000640 1 --2- 172.21.15.66:0/2378142295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9195650 0x5644e9197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b76000640 1 -- 172.21.15.66:0/2378142295 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644e8f44ef0 con 0x5644e9194d30 2024-03-30T00:42:22.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b76000640 1 --2- 172.21.15.66:0/2378142295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9194d30 0x5644e9194620 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0b68003c50 tx=0x7f0b68004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e49a1c6af125bb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b6fe00640 1 -- 172.21.15.66:0/2378142295 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6801c070 con 0x5644e9194d30 2024-03-30T00:42:22.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b6fe00640 1 -- 172.21.15.66:0/2378142295 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6801dd10 con 0x5644e9194d30 2024-03-30T00:42:22.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b6fe00640 1 -- 172.21.15.66:0/2378142295 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6801dea0 con 0x5644e9194d30 2024-03-30T00:42:22.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2378142295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9194d30 msgr2=0x5644e9194620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/2378142295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9194d30 0x5644e9194620 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0b68003c50 tx=0x7f0b68004a50 comp rx=0 tx=0).stop 2024-03-30T00:42:22.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2378142295 shutdown_connections 2024-03-30T00:42:22.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/2378142295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9195650 0x5644e9197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/2378142295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9194d30 0x5644e9194620 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/2378142295 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e914bf60 0x5644e91940e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2378142295 >> 172.21.15.66:0/2378142295 conn(0x5644e9147f10 msgr2=0x5644e91939d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2378142295 shutdown_connections 2024-03-30T00:42:22.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 -- 172.21.15.66:0/2378142295 wait complete. 2024-03-30T00:42:22.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 -- start start 2024-03-30T00:42:22.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 Processor -- start 2024-03-30T00:42:22.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e914bf60 0x5644e91f4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9194d30 0x5644e91f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9195650 0x5644e9199d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e8f60420 con 0x5644e9195650 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e91478b0 con 0x5644e9194d30 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644e8f91430 con 0x5644e914bf60 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b88cf2000 10 client.0 _ll_register_callbacks cb 0x5644e90130c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b7f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9195650 0x5644e9199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b7f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9195650 0x5644e9199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45902/0 (socket says 172.21.15.66:45902) 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b7f400640 1 -- 172.21.15.66:0/1085507510 learned_addr learned my addr 172.21.15.66:0/1085507510 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b76000640 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9194d30 0x5644e91f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.096+0000 7f0b7ea00640 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e914bf60 0x5644e91f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b7f400640 1 -- 172.21.15.66:0/1085507510 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e914bf60 msgr2=0x5644e91f4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b7f400640 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e914bf60 0x5644e91f4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b7f400640 1 -- 172.21.15.66:0/1085507510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9194d30 msgr2=0x5644e91f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b7f400640 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9194d30 0x5644e91f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b7f400640 1 -- 172.21.15.66:0/1085507510 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b68011040 con 0x5644e9195650 2024-03-30T00:42:22.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b7f400640 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9195650 0x5644e9199d50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0b7801cad0 tx=0x7f0b78019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3f83fedee85773 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b63e00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:42:22.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b63e00640 1 -- 172.21.15.66:0/1085507510 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78108070 con 0x5644e9195650 2024-03-30T00:42:22.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b63e00640 1 -- 172.21.15.66:0/1085507510 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0b780f3120 con 0x5644e9195650 2024-03-30T00:42:22.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 -- 172.21.15.66:0/1085507510 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644e8f565f0 con 0x5644e9195650 2024-03-30T00:42:22.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b63e00640 1 -- 172.21.15.66:0/1085507510 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b780f7910 con 0x5644e9195650 2024-03-30T00:42:22.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 10 client.6213 fetch_fsmap learned FSMap version 25 2024-03-30T00:42:22.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 -- 172.21.15.66:0/1085507510 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5644e8f38a60 con 0x5644e9195650 2024-03-30T00:42:22.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b63e00640 1 -- 172.21.15.66:0/1085507510 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5644e8f565f0 con 0x5644e9195650 2024-03-30T00:42:22.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b63e00640 1 -- 172.21.15.66:0/1085507510 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0b780ef830 con 0x5644e9195650 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 10 client.6213 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 10 client.6213 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 20 client.6213 populate_metadata read hostname 'smithi066' 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5644e919a290 0x5644e919a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 4 client.6213 mds_command: new command op to 5985 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 -- 172.21.15.66:0/1085507510 --> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x5644e8f58410 con 0x5644e919a290 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b7ea00640 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5644e919a290 0x5644e919a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b63e00640 10 client.6213 ms_handle_connect on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b7ea00640 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5644e919a290 0x5644e919a6a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb6f6dcc339f528 server_cookie=e717cff3fc4ef2d in_seq=0 out_seq=0 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b63e00640 1 -- 172.21.15.66:0/1085507510 <== mds.0 v2:172.21.15.66:6836/3557086525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5644e8f58410 con 0x5644e919a290 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b63e00640 10 client.6213 handle_command_reply: tid=0 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 client.6213 shutdown 2024-03-30T00:42:22.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 -- 172.21.15.66:0/1085507510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9195650 msgr2=0x5644e9199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9195650 0x5644e9199d50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0b7801cad0 tx=0x7f0b78019300 comp rx=0 tx=0).stop 2024-03-30T00:42:22.116 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f94a2200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6162=0-4194304@1} caps={6162=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6162 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x557f2ea86100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:42:22.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 -- 172.21.15.66:0/1085507510 shutdown_connections 2024-03-30T00:42:22.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5644e919a290 0x5644e919a6a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644e9195650 0x5644e9199d50 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644e9194d30 0x5644e91f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 --2- 172.21.15.66:0/1085507510 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5644e914bf60 0x5644e91f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b63e00640 0 client.6213 ms_handle_reset on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:22.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 -- 172.21.15.66:0/1085507510 >> 172.21.15.66:0/1085507510 conn(0x5644e9147f10 msgr2=0x5644e91a27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 -- 172.21.15.66:0/1085507510 shutdown_connections 2024-03-30T00:42:22.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 1 -- 172.21.15.66:0/1085507510 wait complete. 2024-03-30T00:42:22.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.100+0000 7f0b88cf2000 20 client.6213 trim_cache size 0 max 16384 2024-03-30T00:42:22.118 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:42:22.118 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T00:42:22.118 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "f09c2cf9-4f5f-4f60-869c-d29ec2535c71", 2024-03-30T00:42:22.118 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T00:42:22.118 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:42:22.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 -- 172.21.15.66:0/2479590129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b5c066690 msgr2=0x7f0b5c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b5c066690 0x7f0b5c068b50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0b70007fd0 tx=0x7f0b70009040 comp rx=0 tx=0).stop 2024-03-30T00:42:22.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 -- 172.21.15.66:0/2479590129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b800b1280 msgr2=0x7f0b800c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b800b1280 0x7f0b800c4690 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0b7801a190 tx=0x7f0b7801ada0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 -- 172.21.15.66:0/2479590129 shutdown_connections 2024-03-30T00:42:22.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b5c066690 0x7f0b5c068b50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b800b1280 0x7f0b800c4690 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b800b0b20 0x7f0b800c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 --2- 172.21.15.66:0/2479590129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b80098a20 0x7f0b80154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 -- 172.21.15.66:0/2479590129 >> 172.21.15.66:0/2479590129 conn(0x7f0b800b2150 msgr2=0x7f0b8006bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 -- 172.21.15.66:0/2479590129 shutdown_connections 2024-03-30T00:42:22.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.104+0000 7f0b85800640 1 -- 172.21.15.66:0/2479590129 wait complete. 2024-03-30T00:42:22.132 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2024-03-30T00:42:22.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f5563e00640 1 Processor -- start 2024-03-30T00:42:22.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f5563e00640 1 -- start start 2024-03-30T00:42:22.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f5563e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0cabb0 0x7f555c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f5563e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c06d070 con 0x7f555c15a640 2024-03-30T00:42:22.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f5563e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c0cbba0 con 0x7f555c0ce520 2024-03-30T00:42:22.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f5563e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c0cb8c0 con 0x7f555c0cabb0 2024-03-30T00:42:22.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f5561400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f555c15a640 0x7f555c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53212/0 (socket says 172.21.15.66:53212) 2024-03-30T00:42:22.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f5561400640 1 -- 172.21.15.66:0/1945195489 learned_addr learned my addr 172.21.15.66:0/1945195489 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:22.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f5561e00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0cabb0 0x7f555c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358391303 0 0) 0x7f555c06d070 con 0x7f555c15a640 2024-03-30T00:42:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5544003530 con 0x7f555c15a640 2024-03-30T00:42:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148426799 0 0) 0x7f555c0cbba0 con 0x7f555c0ce520 2024-03-30T00:42:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f555c06d070 con 0x7f555c0ce520 2024-03-30T00:42:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2481497328 0 0) 0x7f5544003530 con 0x7f555c15a640 2024-03-30T00:42:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0cabb0 msgr2=0x7f555c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0cabb0 0x7f555c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 >> v1:172.21.15.156:6789/0 conn(0x7f555c0ce520 legacy=0x7f555c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c06d2a0 con 0x7f555c15a640 2024-03-30T00:42:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f5550001680 con 0x7f555c15a640 2024-03-30T00:42:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f5561e00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0cabb0 0x7f555c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:42:22.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 >> v1:172.21.15.66:6789/0 conn(0x7f555c15a640 legacy=0x7f555c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0cabb0 0x7f5544003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.264+0000 7f555be00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55440040d0 0x7f55440047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f555be00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 0x7f5544005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c0cb8c0 con 0x7f5544004d10 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554400baa0 con 0x7f55440040d0 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554400bc60 con 0x7f555c0cabb0 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f5561e00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 0x7f5544005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f5561400640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0cabb0 0x7f5544003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f5560a00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55440040d0 0x7f55440047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f5561e00640 1 -- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0cabb0 msgr2=0x7f5544003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f5561e00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0cabb0 0x7f5544003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f5561e00640 1 -- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55440040d0 msgr2=0x7f55440047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f5561e00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55440040d0 0x7f55440047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f5561e00640 1 -- 172.21.15.66:0/1945195489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5554009a70 con 0x7f5544004d10 2024-03-30T00:42:22.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f5560a00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55440040d0 0x7f55440047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:22.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f5561e00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 0x7f5544005190 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f555400e320 tx=0x7f555401cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13e472c9a7021f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55540a5ac0 con 0x7f5544004d10 2024-03-30T00:42:22.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555400f040 con 0x7f5544004d10 2024-03-30T00:42:22.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.268+0000 7f555be00640 1 -- 172.21.15.66:0/1945195489 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55540a5ea0 con 0x7f5544004d10 2024-03-30T00:42:22.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 -- 172.21.15.66:0/1945195489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 msgr2=0x7f5544005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 --2- 172.21.15.66:0/1945195489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 0x7f5544005190 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f555400e320 tx=0x7f555401cdc0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561e00640 1 -- 172.21.15.66:0/1945195489 reap_dead start 2024-03-30T00:42:22.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 -- 172.21.15.66:0/1945195489 shutdown_connections 2024-03-30T00:42:22.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 -- 172.21.15.66:0/1945195489 >> 172.21.15.66:0/1945195489 conn(0x7f555c069f60 msgr2=0x7f555c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 -- 172.21.15.66:0/1945195489 shutdown_connections 2024-03-30T00:42:22.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 -- 172.21.15.66:0/1945195489 wait complete. 2024-03-30T00:42:22.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 Processor -- start 2024-03-30T00:42:22.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 -- start start 2024-03-30T00:42:22.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 0x7f555c1f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f555c097ba0 0x7f555c0c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0c91d0 0x7f555c157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c0ca310 con 0x7f5544004d10 2024-03-30T00:42:22.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c0c9690 con 0x7f555c097ba0 2024-03-30T00:42:22.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5563e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c0c9970 con 0x7f555c0c91d0 2024-03-30T00:42:22.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 0x7f555c1f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 0x7f555c1f18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45926/0 (socket says 172.21.15.66:45926) 2024-03-30T00:42:22.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561400640 1 -- 172.21.15.66:0/3526419384 learned_addr learned my addr 172.21.15.66:0/3526419384 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5560a00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f555c097ba0 0x7f555c0c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561e00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0c91d0 0x7f555c157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561400640 1 -- 172.21.15.66:0/3526419384 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0c91d0 msgr2=0x7f555c157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561400640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0c91d0 0x7f555c157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561400640 1 -- 172.21.15.66:0/3526419384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f555c097ba0 msgr2=0x7f555c0c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561400640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f555c097ba0 0x7f555c0c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561400640 1 -- 172.21.15.66:0/3526419384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c0c9c50 con 0x7f5544004d10 2024-03-30T00:42:22.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5560a00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f555c097ba0 0x7f555c0c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:22.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561e00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0c91d0 0x7f555c157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:22.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.272+0000 7f5561400640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 0x7f555c1f18d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5550004880 tx=0x7f5550012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f5559600640 1 -- 172.21.15.66:0/3526419384 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550014750 con 0x7f5544004d10 2024-03-30T00:42:22.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f5559600640 1 -- 172.21.15.66:0/3526419384 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55500148b0 con 0x7f5544004d10 2024-03-30T00:42:22.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f5563e00640 1 -- 172.21.15.66:0/3526419384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c0c9f30 con 0x7f5544004d10 2024-03-30T00:42:22.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f5559600640 1 -- 172.21.15.66:0/3526419384 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550014c60 con 0x7f5544004d10 2024-03-30T00:42:22.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f5563e00640 1 -- 172.21.15.66:0/3526419384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c158590 con 0x7f5544004d10 2024-03-30T00:42:22.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f5559600640 1 -- 172.21.15.66:0/3526419384 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f55500153e0 con 0x7f5544004d10 2024-03-30T00:42:22.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f5559600640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f553c066690 0x7f553c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f5559600640 1 -- 172.21.15.66:0/3526419384 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5550086e20 con 0x7f5544004d10 2024-03-30T00:42:22.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f5560a00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f553c066690 0x7f553c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f556729d000 1 Processor -- start 2024-03-30T00:42:22.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f556729d000 1 -- start start 2024-03-30T00:42:22.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.276+0000 7f5560a00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f553c066690 0x7f553c068b50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f555c0b22d0 tx=0x7f5548009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6da08e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da09390 0x556f6d9bc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6d9bca00 0x556f6d9bcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6d887630 con 0x556f6da08a70 2024-03-30T00:42:22.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6d814bd0 con 0x556f6d9bca00 2024-03-30T00:42:22.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f6d8121a0 con 0x556f6da09390 2024-03-30T00:42:22.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f5561400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6da08e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f5561400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6da08e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45938/0 (socket says 172.21.15.66:45938) 2024-03-30T00:42:22.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f5561400640 1 -- 172.21.15.66:0/881110711 learned_addr learned my addr 172.21.15.66:0/881110711 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f5561400640 1 -- 172.21.15.66:0/881110711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da09390 msgr2=0x556f6d9bc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f5561400640 1 --2- 172.21.15.66:0/881110711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da09390 0x556f6d9bc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f5561400640 1 -- 172.21.15.66:0/881110711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6d9bca00 msgr2=0x556f6d9bcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f5561400640 1 --2- 172.21.15.66:0/881110711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6d9bca00 0x556f6d9bcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f5561400640 1 -- 172.21.15.66:0/881110711 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f6d509e20 con 0x556f6da08a70 2024-03-30T00:42:22.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f5561400640 1 --2- 172.21.15.66:0/881110711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6da08e50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5550054200 tx=0x7f5550058400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad904edd0414a21b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f554e000640 1 -- 172.21.15.66:0/881110711 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555006f070 con 0x556f6da08a70 2024-03-30T00:42:22.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f554e000640 1 -- 172.21.15.66:0/881110711 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55500600f0 con 0x556f6da08a70 2024-03-30T00:42:22.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f554e000640 1 -- 172.21.15.66:0/881110711 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550014dc0 con 0x556f6da08a70 2024-03-30T00:42:22.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 -- 172.21.15.66:0/881110711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 msgr2=0x556f6da08e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 --2- 172.21.15.66:0/881110711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6da08e50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5550054200 tx=0x7f5550058400 comp rx=0 tx=0).stop 2024-03-30T00:42:22.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 -- 172.21.15.66:0/881110711 shutdown_connections 2024-03-30T00:42:22.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 --2- 172.21.15.66:0/881110711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6d9bca00 0x556f6d9bcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 --2- 172.21.15.66:0/881110711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da09390 0x556f6d9bc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 --2- 172.21.15.66:0/881110711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6da08e50 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 -- 172.21.15.66:0/881110711 >> 172.21.15.66:0/881110711 conn(0x556f6d7bd160 msgr2=0x556f6d9eb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 -- 172.21.15.66:0/881110711 shutdown_connections 2024-03-30T00:42:22.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 -- 172.21.15.66:0/881110711 wait complete. 2024-03-30T00:42:22.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 -- start start 2024-03-30T00:42:22.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.280+0000 7f556729d000 1 Processor -- start 2024-03-30T00:42:22.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6d9bca00 0x556f6d9c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da08a70 0x556f6d9eca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da09390 0x556f6d9ecf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6d80da90 con 0x556f6d9bca00 2024-03-30T00:42:22.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6d7acd40 con 0x556f6da09390 2024-03-30T00:42:22.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f6d7caf30 con 0x556f6da08a70 2024-03-30T00:42:22.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 10 client.0 _ll_register_callbacks cb 0x556f6d86f810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:42:22.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5561400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6d9bca00 0x556f6d9c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5561400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6d9bca00 0x556f6d9c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45944/0 (socket says 172.21.15.66:45944) 2024-03-30T00:42:22.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5561400640 1 -- 172.21.15.66:0/827667160 learned_addr learned my addr 172.21.15.66:0/827667160 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5561e00640 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da09390 0x556f6d9ecf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5560a00640 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da08a70 0x556f6d9eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5561400640 1 -- 172.21.15.66:0/827667160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da08a70 msgr2=0x556f6d9eca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5561400640 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da08a70 0x556f6d9eca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5561400640 1 -- 172.21.15.66:0/827667160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da09390 msgr2=0x556f6d9ecf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5561400640 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da09390 0x556f6d9ecf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5561400640 1 -- 172.21.15.66:0/827667160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c158590 con 0x556f6d9bca00 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5561e00640 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da09390 0x556f6d9ecf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5561400640 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6d9bca00 0x556f6d9c57d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5550057ec0 tx=0x7f5550012fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173016754f64f7e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5542a00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5542a00640 1 -- 172.21.15.66:0/827667160 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555006f050 con 0x556f6d9bca00 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5542a00640 1 -- 172.21.15.66:0/827667160 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f555005fc60 con 0x556f6d9bca00 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 -- 172.21.15.66:0/827667160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f6d80c3e0 con 0x556f6d9bca00 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5542a00640 1 -- 172.21.15.66:0/827667160 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f555005fe60 con 0x556f6d9bca00 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 10 client.6225 fetch_fsmap learned FSMap version 25 2024-03-30T00:42:22.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 -- 172.21.15.66:0/827667160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x556f6d7d2aa0 con 0x556f6d9bca00 2024-03-30T00:42:22.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5542a00640 1 -- 172.21.15.66:0/827667160 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f6d80c3e0 con 0x556f6d9bca00 2024-03-30T00:42:22.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5542a00640 1 -- 172.21.15.66:0/827667160 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f555006bb40 con 0x556f6d9bca00 2024-03-30T00:42:22.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 10 client.6225 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:42:22.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 10 client.6225 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-30T00:42:22.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 20 client.6225 populate_metadata read hostname 'smithi066' 2024-03-30T00:42:22.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x556f6da102d0 0x556f6da106b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 4 client.6225 mds_command: new command op to 5985 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:42:22.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 -- 172.21.15.66:0/827667160 --> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f6d7ddd00 con 0x556f6da102d0 2024-03-30T00:42:22.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5560a00640 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x556f6da102d0 0x556f6da106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5560a00640 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x556f6da102d0 0x556f6da106b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=647c368e9bd46002 server_cookie=33bcf613a68bd481 in_seq=0 out_seq=0 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5542a00640 10 client.6225 ms_handle_connect on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5542a00640 1 -- 172.21.15.66:0/827667160 <== mds.0 v2:172.21.15.66:6836/3557086525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f6d7ddd00 con 0x556f6da102d0 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5542a00640 10 client.6225 handle_command_reply: tid=0 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 client.6225 shutdown 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 -- 172.21.15.66:0/827667160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6d9bca00 msgr2=0x556f6d9c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6d9bca00 0x556f6d9c57d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5550057ec0 tx=0x7f5550012fd0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 -- 172.21.15.66:0/827667160 shutdown_connections 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x556f6da102d0 0x556f6da106b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da09390 0x556f6d9ecf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da08a70 0x556f6d9eca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 --2- 172.21.15.66:0/827667160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6d9bca00 0x556f6d9c57d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f5542a00640 0 client.6225 ms_handle_reset on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 -- 172.21.15.66:0/827667160 >> 172.21.15.66:0/827667160 conn(0x556f6d7bd160 msgr2=0x556f6d9bbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 -- 172.21.15.66:0/827667160 shutdown_connections 2024-03-30T00:42:22.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.284+0000 7f556729d000 1 -- 172.21.15.66:0/827667160 wait complete. 2024-03-30T00:42:22.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.288+0000 7f556729d000 20 client.6225 trim_cache size 0 max 16384 2024-03-30T00:42:22.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.300+0000 7f556729d000 1 Processor -- start 2024-03-30T00:42:22.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.300+0000 7f556729d000 1 -- start start 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.300+0000 7f556729d000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6d9bca00 0x556f6d9c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.300+0000 7f556729d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6d9bbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.300+0000 7f556729d000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da09390 0x556f6d9bc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.300+0000 7f556729d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6d4caaa0 con 0x556f6da08a70 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.300+0000 7f556729d000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6d811a90 con 0x556f6d9bca00 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.300+0000 7f556729d000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f6da16910 con 0x556f6da09390 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.300+0000 7f5560a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6d9bbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.300+0000 7f5560a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6d9bbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45960/0 (socket says 172.21.15.66:45960) 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.300+0000 7f5560a00640 1 -- 172.21.15.66:0/1745498047 learned_addr learned my addr 172.21.15.66:0/1745498047 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5561400640 1 --2- 172.21.15.66:0/1745498047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6d9bca00 0x556f6d9c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5561e00640 1 --2- 172.21.15.66:0/1745498047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da09390 0x556f6d9bc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5560a00640 1 -- 172.21.15.66:0/1745498047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da09390 msgr2=0x556f6d9bc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5560a00640 1 --2- 172.21.15.66:0/1745498047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da09390 0x556f6d9bc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5560a00640 1 -- 172.21.15.66:0/1745498047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6d9bca00 msgr2=0x556f6d9c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5560a00640 1 --2- 172.21.15.66:0/1745498047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6d9bca00 0x556f6d9c1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5560a00640 1 -- 172.21.15.66:0/1745498047 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f6d7b8df0 con 0x556f6da08a70 2024-03-30T00:42:22.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5561e00640 1 --2- 172.21.15.66:0/1745498047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da09390 0x556f6d9bc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5560a00640 1 --2- 172.21.15.66:0/1745498047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6d9bbc30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f55480053f0 tx=0x7f5548017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db157f92d4828ab5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f554ea00640 1 -- 172.21.15.66:0/1745498047 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554800fcd0 con 0x556f6da08a70 2024-03-30T00:42:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f554ea00640 1 -- 172.21.15.66:0/1745498047 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5548027070 con 0x556f6da08a70 2024-03-30T00:42:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f554ea00640 1 -- 172.21.15.66:0/1745498047 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554800fe30 con 0x556f6da08a70 2024-03-30T00:42:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 -- 172.21.15.66:0/1745498047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 msgr2=0x556f6d9bbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 --2- 172.21.15.66:0/1745498047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6d9bbc30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f55480053f0 tx=0x7f5548017040 comp rx=0 tx=0).stop 2024-03-30T00:42:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 -- 172.21.15.66:0/1745498047 shutdown_connections 2024-03-30T00:42:22.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 --2- 172.21.15.66:0/1745498047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6da09390 0x556f6d9bc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 --2- 172.21.15.66:0/1745498047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da08a70 0x556f6d9bbc30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 --2- 172.21.15.66:0/1745498047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6d9bca00 0x556f6d9c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 -- 172.21.15.66:0/1745498047 >> 172.21.15.66:0/1745498047 conn(0x556f6d7bd160 msgr2=0x556f6d9c4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 -- 172.21.15.66:0/1745498047 shutdown_connections 2024-03-30T00:42:22.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 -- 172.21.15.66:0/1745498047 wait complete. 2024-03-30T00:42:22.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 -- start start 2024-03-30T00:42:22.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 Processor -- start 2024-03-30T00:42:22.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6d9bca00 0x556f6da64430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da08a70 0x556f6d9bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da09390 0x556f6d9bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6d7d7ee0 con 0x556f6da09390 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6d809280 con 0x556f6da08a70 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f6d80c920 con 0x556f6d9bca00 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f556729d000 10 client.0 _ll_register_callbacks cb 0x7f5550054040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5561e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da09390 0x556f6d9bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5561e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da09390 0x556f6d9bfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45976/0 (socket says 172.21.15.66:45976) 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5561e00640 1 -- 172.21.15.66:0/2064297824 learned_addr learned my addr 172.21.15.66:0/2064297824 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5561400640 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6d9bca00 0x556f6da64430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.304+0000 7f5560a00640 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da08a70 0x556f6d9bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5561e00640 1 -- 172.21.15.66:0/2064297824 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6d9bca00 msgr2=0x556f6da64430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5561e00640 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6d9bca00 0x556f6da64430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5561e00640 1 -- 172.21.15.66:0/2064297824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da08a70 msgr2=0x556f6d9bf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5561e00640 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da08a70 0x556f6d9bf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5561e00640 1 -- 172.21.15.66:0/2064297824 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554801d040 con 0x556f6da09390 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5560a00640 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da08a70 0x556f6d9bf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5561e00640 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da09390 0x556f6d9bfbd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f555400e320 tx=0x7f5554009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445d381ff6abe5f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5542a00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5542a00640 1 -- 172.21.15.66:0/2064297824 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55540a5ac0 con 0x556f6da09390 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5542a00640 1 -- 172.21.15.66:0/2064297824 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f55540a5c20 con 0x556f6da09390 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 -- 172.21.15.66:0/2064297824 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f6d7ca970 con 0x556f6da09390 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5542a00640 1 -- 172.21.15.66:0/2064297824 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55540a5de0 con 0x556f6da09390 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 10 client.6231 fetch_fsmap learned FSMap version 25 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 -- 172.21.15.66:0/2064297824 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x556f6d7ac960 con 0x556f6da09390 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5542a00640 1 -- 172.21.15.66:0/2064297824 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f6d7ca970 con 0x556f6da09390 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5542a00640 1 -- 172.21.15.66:0/2064297824 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f55540b5040 con 0x556f6da09390 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 10 client.6231 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 10 client.6231 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 20 client.6231 populate_metadata read hostname 'smithi066' 2024-03-30T00:42:22.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x556f6d9c0110 0x556f6d9c0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:22.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 4 client.6231 mds_command: new command op to 5985 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:42:22.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 -- 172.21.15.66:0/2064297824 --> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556f6d83ca20 con 0x556f6d9c0110 2024-03-30T00:42:22.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5561400640 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x556f6d9c0110 0x556f6d9c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:22.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5542a00640 10 client.6231 ms_handle_connect on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:22.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5561400640 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x556f6d9c0110 0x556f6d9c0520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eebe68ea72d3a4d server_cookie=1041d3f4e27df626 in_seq=0 out_seq=0 2024-03-30T00:42:22.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5542a00640 1 -- 172.21.15.66:0/2064297824 <== mds.0 v2:172.21.15.66:6836/3557086525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x556f6d83ca20 con 0x556f6d9c0110 2024-03-30T00:42:22.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5542a00640 10 client.6231 handle_command_reply: tid=0 2024-03-30T00:42:22.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 client.6231 shutdown 2024-03-30T00:42:22.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 -- 172.21.15.66:0/2064297824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da09390 msgr2=0x556f6d9bfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da09390 0x556f6d9bfbd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f555400e320 tx=0x7f5554009750 comp rx=0 tx=0).stop 2024-03-30T00:42:22.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 -- 172.21.15.66:0/2064297824 shutdown_connections 2024-03-30T00:42:22.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x556f6d9c0110 0x556f6d9c0520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f6da09390 0x556f6d9bfbd0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f6da08a70 0x556f6d9bf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5542a00640 0 client.6231 ms_handle_reset on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:22.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 --2- 172.21.15.66:0/2064297824 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f6d9bca00 0x556f6da64430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 -- 172.21.15.66:0/2064297824 >> 172.21.15.66:0/2064297824 conn(0x556f6d7bd160 msgr2=0x556f6d9bac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 -- 172.21.15.66:0/2064297824 shutdown_connections 2024-03-30T00:42:22.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 1 -- 172.21.15.66:0/2064297824 wait complete. 2024-03-30T00:42:22.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f556729d000 20 client.6231 trim_cache size 0 max 16384 2024-03-30T00:42:22.323 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:42:22.323 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:42:22.323 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-03-30T00:42:22.323 INFO:teuthology.orchestra.run.smithi066.stdout: "f09c2cf9-4f5f-4f60-869c-d29ec2535c71": { 2024-03-30T00:42:22.323 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T00:42:22.324 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "f09c2cf9-4f5f-4f60-869c-d29ec2535c71", 2024-03-30T00:42:22.324 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,repair" 2024-03-30T00:42:22.324 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:22.324 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:22.324 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:42:22.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 -- 172.21.15.66:0/3526419384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f553c066690 msgr2=0x7f553c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f553c066690 0x7f553c068b50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f555c0b22d0 tx=0x7f5548009040 comp rx=0 tx=0).stop 2024-03-30T00:42:22.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 -- 172.21.15.66:0/3526419384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 msgr2=0x7f555c1f18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:22.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 0x7f555c1f18d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5550004880 tx=0x7f5550012920 comp rx=0 tx=0).stop 2024-03-30T00:42:22.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 -- 172.21.15.66:0/3526419384 shutdown_connections 2024-03-30T00:42:22.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f555c0c91d0 0x7f555c157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f555c097ba0 0x7f555c0c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f553c066690 0x7f553c068b50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 --2- 172.21.15.66:0/3526419384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544004d10 0x7f555c1f18d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:22.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 -- 172.21.15.66:0/3526419384 >> 172.21.15.66:0/3526419384 conn(0x7f555c069f60 msgr2=0x7f555c0ab010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:22.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 -- 172.21.15.66:0/3526419384 shutdown_connections 2024-03-30T00:42:22.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:22.308+0000 7f5563e00640 1 -- 172.21.15.66:0/3526419384 wait complete. 2024-03-30T00:42:22.339 INFO:tasks.cephfs.filesystem:scrub status for tag:f09c2cf9-4f5f-4f60-869c-d29ec2535c71 - {'path': '/', 'tag': 'f09c2cf9-4f5f-4f60-869c-d29ec2535c71', 'options': 'recursive,repair'} 2024-03-30T00:42:23.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:23.528+0000 7f6714c00640 1 -- 172.21.15.66:0/2559199436 <== mon.0 v2:172.21.15.66:3300/0 7 ==== log(6 entries from seq 1 at 2024-03-30T00:42:22.106800+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f6710056410 con 0x7f671815a690 2024-03-30T00:42:23.539 INFO:teuthology.orchestra.run.smithi066.stdout: cluster: 2024-03-30T00:42:23.539 INFO:teuthology.orchestra.run.smithi066.stdout: id: 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:42:23.539 INFO:teuthology.orchestra.run.smithi066.stdout: health: HEALTH_OK 2024-03-30T00:42:23.539 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:23.539 INFO:teuthology.orchestra.run.smithi066.stdout: services: 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: mgr: y(active, since 2m), standbys: x 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: mds: 1/1 daemons up, 3 standby 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: data: 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: volumes: 1/1 healthy 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: pools: 3 pools, 97 pgs 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: objects: 24 objects, 579 KiB 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: pgs: 97 active+clean 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: io: 2024-03-30T00:42:23.540 INFO:teuthology.orchestra.run.smithi066.stdout: client: 1.2 KiB/s wr, 0 op/s rd, 3 op/s wr 2024-03-30T00:42:23.541 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:23.541 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:23.541 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:42:22.106800+0000 mds.c [INF] scrub queued for path: / 2024-03-30T00:42:23.541 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:42:22.106805+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-30T00:42:23.541 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:42:22.106812+0000 mds.c [INF] scrub summary: active paths [/] 2024-03-30T00:42:23.541 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:42:22.107804+0000 mds.c [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-03-30T00:42:23.541 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:42:22.107814+0000 mds.c [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-03-30T00:42:23.541 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:42:22.108368+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-30T00:42:25.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:25.536+0000 7f6714c00640 1 -- 172.21.15.66:0/2559199436 <== mon.0 v2:172.21.15.66:3300/0 8 ==== log(1 entries from seq 7 at 2024-03-30T00:42:23.923021+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f671005a190 con 0x7f671815a690 2024-03-30T00:42:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:42:23.923021+0000 mds.c [INF] scrub summary: idle 2024-03-30T00:42:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:32.556+0000 7f6714c00640 1 -- 172.21.15.66:0/2559199436 <== mon.0 v2:172.21.15.66:3300/0 9 ==== log(2 entries from seq 609 at 2024-03-30T00:42:32.514640+0000) v1 ==== 594+0+0 (secure 0 0 0) 0x7f671005e750 con 0x7f671815a690 2024-03-30T00:42:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:33.560+0000 7f6714c00640 1 -- 172.21.15.66:0/2559199436 <== mon.0 v2:172.21.15.66:3300/0 10 ==== log(2 entries from seq 611 at 2024-03-30T00:42:32.558575+0000) v1 ==== 598+0+0 (secure 0 0 0) 0x7f6710054550 con 0x7f671815a690 2024-03-30T00:42:46.227 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:42:46.231 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:42:52.340 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2024-03-30T00:42:52.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f6381c00640 1 Processor -- start 2024-03-30T00:42:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f6381c00640 1 -- start start 2024-03-30T00:42:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f6381c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f637c0cec30 0x7f637c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f6381c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637c0cd3e0 con 0x7f637c0cda90 2024-03-30T00:42:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f6381c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637c15a990 con 0x7f637c0cde70 2024-03-30T00:42:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f6381c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637c0b1b80 con 0x7f637c0cec30 2024-03-30T00:42:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637aa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f637c0cda90 0x7f637c0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56178/0 (socket says 172.21.15.66:56178) 2024-03-30T00:42:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637aa00640 1 -- 172.21.15.66:0/3429440905 learned_addr learned my addr 172.21.15.66:0/3429440905 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f6372000640 1 --2- 172.21.15.66:0/3429440905 >> v2:172.21.15.156:3300/0 conn(0x7f637c0cec30 0x7f637c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857687493 0 0) 0x7f637c0cd3e0 con 0x7f637c0cda90 2024-03-30T00:42:52.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6360003530 con 0x7f637c0cda90 2024-03-30T00:42:52.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 524579435 0 0) 0x7f6360003530 con 0x7f637c0cda90 2024-03-30T00:42:52.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 >> v2:172.21.15.156:3300/0 conn(0x7f637c0cec30 msgr2=0x7f637c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 --2- 172.21.15.66:0/3429440905 >> v2:172.21.15.156:3300/0 conn(0x7f637c0cec30 0x7f637c159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 >> v1:172.21.15.156:6789/0 conn(0x7f637c0cde70 legacy=0x7f637c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c0b11e0 con 0x7f637c0cda90 2024-03-30T00:42:52.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f636c001680 con 0x7f637c0cda90 2024-03-30T00:42:52.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 >> v1:172.21.15.66:6789/0 conn(0x7f637c0cda90 legacy=0x7f637c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 --2- 172.21.15.66:0/3429440905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360003980 0x7f6360003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 --2- 172.21.15.66:0/3429440905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63600047d0 0x7f6360004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 --2- 172.21.15.66:0/3429440905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6360005190 0x7f636000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0b1b80 con 0x7f6360003980 2024-03-30T00:42:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636000c130 con 0x7f63600047d0 2024-03-30T00:42:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636000c2f0 con 0x7f6360005190 2024-03-30T00:42:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637aa00640 1 --2- 172.21.15.66:0/3429440905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360003980 0x7f6360003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f6372000640 1 --2- 172.21.15.66:0/3429440905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63600047d0 0x7f6360004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637b400640 1 --2- 172.21.15.66:0/3429440905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6360005190 0x7f636000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637aa00640 1 -- 172.21.15.66:0/3429440905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6360005190 msgr2=0x7f636000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637aa00640 1 --2- 172.21.15.66:0/3429440905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6360005190 0x7f636000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637aa00640 1 -- 172.21.15.66:0/3429440905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63600047d0 msgr2=0x7f6360004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637aa00640 1 --2- 172.21.15.66:0/3429440905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63600047d0 0x7f6360004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637aa00640 1 -- 172.21.15.66:0/3429440905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6360003530 con 0x7f6360003980 2024-03-30T00:42:52.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637b400640 1 -- 172.21.15.66:0/3429440905 reap_dead start 2024-03-30T00:42:52.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637aa00640 1 --2- 172.21.15.66:0/3429440905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360003980 0x7f6360003da0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f636c00c240 tx=0x7f636c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4e9065f89d5084 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:52.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c005ae0 con 0x7f6360003980 2024-03-30T00:42:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c005c40 con 0x7f6360003980 2024-03-30T00:42:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.488+0000 7f637a000640 1 -- 172.21.15.66:0/3429440905 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0a5b20 con 0x7f6360003980 2024-03-30T00:42:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 -- 172.21.15.66:0/3429440905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360003980 msgr2=0x7f6360003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 --2- 172.21.15.66:0/3429440905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360003980 0x7f6360003da0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f636c00c240 tx=0x7f636c01cc60 comp rx=0 tx=0).stop 2024-03-30T00:42:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 -- 172.21.15.66:0/3429440905 shutdown_connections 2024-03-30T00:42:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 --2- 172.21.15.66:0/3429440905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360003980 0x7f6360003da0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 -- 172.21.15.66:0/3429440905 >> 172.21.15.66:0/3429440905 conn(0x7f637c069f60 msgr2=0x7f637c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 -- 172.21.15.66:0/3429440905 shutdown_connections 2024-03-30T00:42:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 -- 172.21.15.66:0/3429440905 wait complete. 2024-03-30T00:42:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 Processor -- start 2024-03-30T00:42:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 -- start start 2024-03-30T00:42:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6360003980 0x7f637c0aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f637c0c4b40 0x7f637c0c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f637c157e70 0x7f637c158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0ac320 con 0x7f637c0c4b40 2024-03-30T00:42:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0ac600 con 0x7f6360003980 2024-03-30T00:42:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0ac8e0 con 0x7f637c157e70 2024-03-30T00:42:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6372000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f637c0c4b40 0x7f637c0c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6372000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f637c0c4b40 0x7f637c0c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38848/0 (socket says 172.21.15.66:38848) 2024-03-30T00:42:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6372000640 1 -- 172.21.15.66:0/1368530148 learned_addr learned my addr 172.21.15.66:0/1368530148 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f637aa00640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6360003980 0x7f637c0aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f637b400640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f637c157e70 0x7f637c158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6372000640 1 -- 172.21.15.66:0/1368530148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f637c157e70 msgr2=0x7f637c158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6372000640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f637c157e70 0x7f637c158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6372000640 1 -- 172.21.15.66:0/1368530148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6360003980 msgr2=0x7f637c0aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6372000640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6360003980 0x7f637c0aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6372000640 1 -- 172.21.15.66:0/1368530148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c070620 con 0x7f637c0c4b40 2024-03-30T00:42:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f637aa00640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6360003980 0x7f637c0aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f637b400640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f637c157e70 0x7f637c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6372000640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f637c0c4b40 0x7f637c0c26a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f63640119a0 tx=0x7f6364011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6373400640 1 -- 172.21.15.66:0/1368530148 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636400faf0 con 0x7f637c0c4b40 2024-03-30T00:42:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6373400640 1 -- 172.21.15.66:0/1368530148 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636400fc50 con 0x7f637c0c4b40 2024-03-30T00:42:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 -- 172.21.15.66:0/1368530148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c070900 con 0x7f637c0c4b40 2024-03-30T00:42:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6373400640 1 -- 172.21.15.66:0/1368530148 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364014440 con 0x7f637c0c4b40 2024-03-30T00:42:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6381c00640 1 -- 172.21.15.66:0/1368530148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c0abf60 con 0x7f637c0c4b40 2024-03-30T00:42:52.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6373400640 1 -- 172.21.15.66:0/1368530148 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f63640145a0 con 0x7f637c0c4b40 2024-03-30T00:42:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6373400640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63580666c0 0x7f6358068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6373400640 1 -- 172.21.15.66:0/1368530148 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6364084c10 con 0x7f637c0c4b40 2024-03-30T00:42:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f637aa00640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63580666c0 0x7f6358068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6384fa4000 1 Processor -- start 2024-03-30T00:42:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.496+0000 7f6384fa4000 1 -- start start 2024-03-30T00:42:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f637aa00640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63580666c0 0x7f6358068b80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f636c0ab000 tx=0x7f636c0b1040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e59e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e5a390 0x5608e9e0d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e0da50 0x5608e9e0de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e9cd8630 con 0x5608e9e59a70 2024-03-30T00:42:52.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e9c65bd0 con 0x5608e9e0da50 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608e9c631a0 con 0x5608e9e5a390 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f637aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e59e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f637aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e59e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38850/0 (socket says 172.21.15.66:38850) 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f637aa00640 1 -- 172.21.15.66:0/984312699 learned_addr learned my addr 172.21.15.66:0/984312699 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f637b400640 1 --2- 172.21.15.66:0/984312699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e0da50 0x5608e9e0de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6372000640 1 --2- 172.21.15.66:0/984312699 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e5a390 0x5608e9e0d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f637aa00640 1 -- 172.21.15.66:0/984312699 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e5a390 msgr2=0x5608e9e0d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f637aa00640 1 --2- 172.21.15.66:0/984312699 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e5a390 0x5608e9e0d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f637aa00640 1 -- 172.21.15.66:0/984312699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e0da50 msgr2=0x5608e9e0de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f637aa00640 1 --2- 172.21.15.66:0/984312699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e0da50 0x5608e9e0de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f637aa00640 1 -- 172.21.15.66:0/984312699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e995ae20 con 0x5608e9e59a70 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f637aa00640 1 --2- 172.21.15.66:0/984312699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e59e50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f636c098600 tx=0x7f636c001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5cddebcc69519c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f636b400640 1 -- 172.21.15.66:0/984312699 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0c0070 con 0x5608e9e59a70 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f636b400640 1 -- 172.21.15.66:0/984312699 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c001540 con 0x5608e9e59a70 2024-03-30T00:42:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f636b400640 1 -- 172.21.15.66:0/984312699 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0b4040 con 0x5608e9e59a70 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 -- 172.21.15.66:0/984312699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 msgr2=0x5608e9e59e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 --2- 172.21.15.66:0/984312699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e59e50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f636c098600 tx=0x7f636c001a10 comp rx=0 tx=0).stop 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 -- 172.21.15.66:0/984312699 shutdown_connections 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 --2- 172.21.15.66:0/984312699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e0da50 0x5608e9e0de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 --2- 172.21.15.66:0/984312699 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e5a390 0x5608e9e0d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 --2- 172.21.15.66:0/984312699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e59e50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 -- 172.21.15.66:0/984312699 >> 172.21.15.66:0/984312699 conn(0x5608e9c0e160 msgr2=0x5608e9e3c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 -- 172.21.15.66:0/984312699 shutdown_connections 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 -- 172.21.15.66:0/984312699 wait complete. 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 -- start start 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.504+0000 7f6384fa4000 1 Processor -- start 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e0da50 0x5608e9e16820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e59a70 0x5608e9e3da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e5a390 0x5608e9e3df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e9c5ea90 con 0x5608e9e5a390 2024-03-30T00:42:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e9bfdd40 con 0x5608e9e59a70 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608e9c1bf30 con 0x5608e9e0da50 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 10 client.0 _ll_register_callbacks cb 0x5608e9cc0810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e5a390 0x5608e9e3df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e5a390 0x5608e9e3df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38856/0 (socket says 172.21.15.66:38856) 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637b400640 1 -- 172.21.15.66:0/2819543893 learned_addr learned my addr 172.21.15.66:0/2819543893 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6372000640 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e59a70 0x5608e9e3da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637aa00640 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e0da50 0x5608e9e16820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637b400640 1 -- 172.21.15.66:0/2819543893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e0da50 msgr2=0x5608e9e16820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637b400640 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e0da50 0x5608e9e16820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637b400640 1 -- 172.21.15.66:0/2819543893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e59a70 msgr2=0x5608e9e3da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637b400640 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e59a70 0x5608e9e3da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637b400640 1 -- 172.21.15.66:0/2819543893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6360003530 con 0x5608e9e5a390 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637aa00640 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e0da50 0x5608e9e16820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637b400640 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e5a390 0x5608e9e3df90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f637c0b29e0 tx=0x7f63740019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876fbb8d534772a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f635fe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f635fe00640 1 -- 172.21.15.66:0/2819543893 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374005860 con 0x5608e9e5a390 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f635fe00640 1 -- 172.21.15.66:0/2819543893 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6374016480 con 0x5608e9e5a390 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 1 -- 172.21.15.66:0/2819543893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608e9c5d3e0 con 0x5608e9e5a390 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f635fe00640 1 -- 172.21.15.66:0/2819543893 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6374016680 con 0x5608e9e5a390 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 10 client.6243 fetch_fsmap learned FSMap version 25 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 1 -- 172.21.15.66:0/2819543893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5608e9c23aa0 con 0x5608e9e5a390 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f635fe00640 1 -- 172.21.15.66:0/2819543893 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5608e9c5d3e0 con 0x5608e9e5a390 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f635fe00640 1 -- 172.21.15.66:0/2819543893 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f63740167e0 con 0x5608e9e5a390 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 10 client.6243 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 10 client.6243 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 20 client.6243 populate_metadata read hostname 'smithi066' 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5608e9e60e00 0x5608e9e611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 4 client.6243 mds_command: new command op to 5985 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:42:52.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f6384fa4000 1 -- 172.21.15.66:0/2819543893 --> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608e9c2ed00 con 0x5608e9e60e00 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637aa00640 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5608e9e60e00 0x5608e9e611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f635fe00640 10 client.6243 ms_handle_connect on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.508+0000 7f637aa00640 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5608e9e60e00 0x5608e9e611e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4315607355e645 server_cookie=22fcd0217b95f55f in_seq=0 out_seq=0 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f635fe00640 1 -- 172.21.15.66:0/2819543893 <== mds.0 v2:172.21.15.66:6836/3557086525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5608e9c2ed00 con 0x5608e9e60e00 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f635fe00640 10 client.6243 handle_command_reply: tid=0 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 1 client.6243 shutdown 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 1 -- 172.21.15.66:0/2819543893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e5a390 msgr2=0x5608e9e3df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e5a390 0x5608e9e3df90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f637c0b29e0 tx=0x7f63740019f0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 1 -- 172.21.15.66:0/2819543893 shutdown_connections 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5608e9e60e00 0x5608e9e611e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e5a390 0x5608e9e3df90 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e59a70 0x5608e9e3da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 1 --2- 172.21.15.66:0/2819543893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e0da50 0x5608e9e16820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f635fe00640 0 client.6243 ms_handle_reset on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 1 -- 172.21.15.66:0/2819543893 >> 172.21.15.66:0/2819543893 conn(0x5608e9c0e160 msgr2=0x5608e9e0cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:52.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 1 -- 172.21.15.66:0/2819543893 shutdown_connections 2024-03-30T00:42:52.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 1 -- 172.21.15.66:0/2819543893 wait complete. 2024-03-30T00:42:52.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.512+0000 7f6384fa4000 20 client.6243 trim_cache size 0 max 16384 2024-03-30T00:42:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.524+0000 7f6384fa4000 1 Processor -- start 2024-03-30T00:42:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.524+0000 7f6384fa4000 1 -- start start 2024-03-30T00:42:52.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e0da50 0x5608e9e0cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e59a70 0x5608e9e0d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e5a390 0x5608e9e0de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e991baa0 con 0x5608e9e0da50 2024-03-30T00:42:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e9c62a90 con 0x5608e9e5a390 2024-03-30T00:42:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608e9c5a280 con 0x5608e9e59a70 2024-03-30T00:42:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f637aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e0da50 0x5608e9e0cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f637aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e0da50 0x5608e9e0cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38868/0 (socket says 172.21.15.66:38868) 2024-03-30T00:42:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f637aa00640 1 -- 172.21.15.66:0/2228360334 learned_addr learned my addr 172.21.15.66:0/2228360334 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f637aa00640 1 -- 172.21.15.66:0/2228360334 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e59a70 msgr2=0x5608e9e0d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f637aa00640 1 --2- 172.21.15.66:0/2228360334 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e59a70 0x5608e9e0d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f637aa00640 1 -- 172.21.15.66:0/2228360334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e5a390 msgr2=0x5608e9e0de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f637aa00640 1 --2- 172.21.15.66:0/2228360334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e5a390 0x5608e9e0de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f637aa00640 1 -- 172.21.15.66:0/2228360334 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e9c09df0 con 0x5608e9e0da50 2024-03-30T00:42:52.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f637aa00640 1 --2- 172.21.15.66:0/2228360334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e0da50 0x5608e9e0cc80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f636c007f40 tx=0x7f636c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c10738687e1a401 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:52.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f636be00640 1 -- 172.21.15.66:0/2228360334 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0c0070 con 0x5608e9e0da50 2024-03-30T00:42:52.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f636be00640 1 -- 172.21.15.66:0/2228360334 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c0c1040 con 0x5608e9e0da50 2024-03-30T00:42:52.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f636be00640 1 -- 172.21.15.66:0/2228360334 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0cc450 con 0x5608e9e0da50 2024-03-30T00:42:52.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- 172.21.15.66:0/2228360334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e0da50 msgr2=0x5608e9e0cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 --2- 172.21.15.66:0/2228360334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e0da50 0x5608e9e0cc80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f636c007f40 tx=0x7f636c0a7000 comp rx=0 tx=0).stop 2024-03-30T00:42:52.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- 172.21.15.66:0/2228360334 shutdown_connections 2024-03-30T00:42:52.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 --2- 172.21.15.66:0/2228360334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e5a390 0x5608e9e0de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 --2- 172.21.15.66:0/2228360334 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e59a70 0x5608e9e0d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 --2- 172.21.15.66:0/2228360334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e0da50 0x5608e9e0cc80 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- 172.21.15.66:0/2228360334 >> 172.21.15.66:0/2228360334 conn(0x5608e9c0e160 msgr2=0x5608e9e15880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:52.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- 172.21.15.66:0/2228360334 shutdown_connections 2024-03-30T00:42:52.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- 172.21.15.66:0/2228360334 wait complete. 2024-03-30T00:42:52.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- start start 2024-03-30T00:42:52.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 Processor -- start 2024-03-30T00:42:52.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e0da50 0x5608e9e9cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e10bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e5a390 0x5608e9e11130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e9e13100 con 0x5608e9e59a70 2024-03-30T00:42:52.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e9c28640 con 0x5608e9e5a390 2024-03-30T00:42:52.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608e9d17ec0 con 0x5608e9e0da50 2024-03-30T00:42:52.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6384fa4000 10 client.0 _ll_register_callbacks cb 0x5608e9cfb740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:42:52.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6372000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e10bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6372000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e10bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38884/0 (socket says 172.21.15.66:38884) 2024-03-30T00:42:52.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.528+0000 7f6372000640 1 -- 172.21.15.66:0/3398712074 learned_addr learned my addr 172.21.15.66:0/3398712074 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:52.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f637aa00640 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e0da50 0x5608e9e9cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f637b400640 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e5a390 0x5608e9e11130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6372000640 1 -- 172.21.15.66:0/3398712074 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e0da50 msgr2=0x5608e9e9cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6372000640 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e0da50 0x5608e9e9cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6372000640 1 -- 172.21.15.66:0/3398712074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e5a390 msgr2=0x5608e9e11130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6372000640 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e5a390 0x5608e9e11130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6372000640 1 -- 172.21.15.66:0/3398712074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608e995ae20 con 0x5608e9e59a70 2024-03-30T00:42:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f637b400640 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e5a390 0x5608e9e11130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6372000640 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e10bf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6364082ec0 tx=0x7f63640561e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccedcb66ebdf1ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f635fe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:42:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f635fe00640 1 -- 172.21.15.66:0/3398712074 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636406d3d0 con 0x5608e9e59a70 2024-03-30T00:42:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f635fe00640 1 -- 172.21.15.66:0/3398712074 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f636405de10 con 0x5608e9e59a70 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 -- 172.21.15.66:0/3398712074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608e9c1b970 con 0x5608e9e59a70 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f635fe00640 1 -- 172.21.15.66:0/3398712074 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6364054f10 con 0x5608e9e59a70 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 10 client.6249 fetch_fsmap learned FSMap version 25 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 -- 172.21.15.66:0/3398712074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5608e9bfd960 con 0x5608e9e59a70 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f635fe00640 1 -- 172.21.15.66:0/3398712074 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5608e9c1b970 con 0x5608e9e59a70 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f635fe00640 1 -- 172.21.15.66:0/3398712074 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f636401b020 con 0x5608e9e59a70 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 10 client.6249 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 10 client.6249 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 20 client.6249 populate_metadata read hostname 'smithi066' 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5608e9e11670 0x5608e9e11a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 4 client.6249 mds_command: new command op to 5985 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:42:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 -- 172.21.15.66:0/3398712074 --> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5608e9c8da20 con 0x5608e9e11670 2024-03-30T00:42:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f637aa00640 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5608e9e11670 0x5608e9e11a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f637aa00640 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5608e9e11670 0x5608e9e11a50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea5bc96ab6c36ed server_cookie=973cec046464ed7a in_seq=0 out_seq=0 2024-03-30T00:42:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f635fe00640 10 client.6249 ms_handle_connect on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f635fe00640 1 -- 172.21.15.66:0/3398712074 <== mds.0 v2:172.21.15.66:6836/3557086525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5608e9c8da20 con 0x5608e9e11670 2024-03-30T00:42:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f635fe00640 10 client.6249 handle_command_reply: tid=0 2024-03-30T00:42:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 client.6249 shutdown 2024-03-30T00:42:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 -- 172.21.15.66:0/3398712074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 msgr2=0x5608e9e10bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e10bf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6364082ec0 tx=0x7f63640561e0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 -- 172.21.15.66:0/3398712074 shutdown_connections 2024-03-30T00:42:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.66:6836/3557086525,v1:172.21.15.66:6837/3557086525] conn(0x5608e9e11670 0x5608e9e11a50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e9e5a390 0x5608e9e11130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608e9e59a70 0x5608e9e10bf0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 --2- 172.21.15.66:0/3398712074 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608e9e0da50 0x5608e9e9cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f635fe00640 0 client.6249 ms_handle_reset on v2:172.21.15.66:6836/3557086525 2024-03-30T00:42:52.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 -- 172.21.15.66:0/3398712074 >> 172.21.15.66:0/3398712074 conn(0x5608e9c0e160 msgr2=0x5608e9e0bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:52.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 -- 172.21.15.66:0/3398712074 shutdown_connections 2024-03-30T00:42:52.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 1 -- 172.21.15.66:0/3398712074 wait complete. 2024-03-30T00:42:52.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6384fa4000 20 client.6249 trim_cache size 0 max 16384 2024-03-30T00:42:52.546 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:42:52.547 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T00:42:52.547 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-03-30T00:42:52.547 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:42:52.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 -- 172.21.15.66:0/1368530148 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63580666c0 msgr2=0x7f6358068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63580666c0 0x7f6358068b80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f636c0ab000 tx=0x7f636c0b1040 comp rx=0 tx=0).stop 2024-03-30T00:42:52.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 -- 172.21.15.66:0/1368530148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f637c0c4b40 msgr2=0x7f637c0c26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f637c0c4b40 0x7f637c0c26a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f63640119a0 tx=0x7f6364011e60 comp rx=0 tx=0).stop 2024-03-30T00:42:52.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 -- 172.21.15.66:0/1368530148 shutdown_connections 2024-03-30T00:42:52.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f637c157e70 0x7f637c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63580666c0 0x7f6358068b80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f637c0c4b40 0x7f637c0c26a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 --2- 172.21.15.66:0/1368530148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6360003980 0x7f637c0aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 -- 172.21.15.66:0/1368530148 >> 172.21.15.66:0/1368530148 conn(0x7f637c069f60 msgr2=0x7f637c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:52.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 -- 172.21.15.66:0/1368530148 shutdown_connections 2024-03-30T00:42:52.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.532+0000 7f6381c00640 1 -- 172.21.15.66:0/1368530148 wait complete. 2024-03-30T00:42:52.559 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:42:52.660 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:42:52.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e66600640 1 Processor -- start 2024-03-30T00:42:52.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e66600640 1 -- start start 2024-03-30T00:42:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e66600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e600cda90 0x7f7e600ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e66600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7e6015a370 0x7f7e600cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e66600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6015a990 con 0x7f7e600cde70 2024-03-30T00:42:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e66600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e600b1b80 con 0x7f7e6015a370 2024-03-30T00:42:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e66600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600cd3e0 con 0x7f7e600cda90 2024-03-30T00:42:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e5fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7e6015a370 0x7f7e600cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e5fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7e6015a370 0x7f7e600cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38886/0 (socket says 172.21.15.66:38886) 2024-03-30T00:42:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e5fe00640 1 -- 172.21.15.66:0/3825751246 learned_addr learned my addr 172.21.15.66:0/3825751246 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e5f400640 1 --2- 172.21.15.66:0/3825751246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e600cda90 0x7f7e600ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e5fe00640 1 -- 172.21.15.66:0/3825751246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e600cda90 msgr2=0x7f7e600ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e5fe00640 1 --2- 172.21.15.66:0/3825751246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e600cda90 0x7f7e600ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e5fe00640 1 -- 172.21.15.66:0/3825751246 >> v1:172.21.15.156:6789/0 conn(0x7f7e600cde70 legacy=0x7f7e60159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e5fe00640 1 -- 172.21.15.66:0/3825751246 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e600b11e0 con 0x7f7e6015a370 2024-03-30T00:42:52.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.796+0000 7f7e5f400640 1 --2- 172.21.15.66:0/3825751246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e600cda90 0x7f7e600ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e5fe00640 1 --2- 172.21.15.66:0/3825751246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e6015a370 0x7f7e600cea60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009e50 tx=0x7f7e5801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9f56c24377be82 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e5ea00640 1 -- 172.21.15.66:0/3825751246 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580a56f0 con 0x7f7e6015a370 2024-03-30T00:42:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e5ea00640 1 -- 172.21.15.66:0/3825751246 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e580a5850 con 0x7f7e6015a370 2024-03-30T00:42:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e5ea00640 1 -- 172.21.15.66:0/3825751246 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580a5be0 con 0x7f7e6015a370 2024-03-30T00:42:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e66600640 1 -- 172.21.15.66:0/3825751246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e6015a370 msgr2=0x7f7e600cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e66600640 1 --2- 172.21.15.66:0/3825751246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e6015a370 0x7f7e600cea60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009e50 tx=0x7f7e5801c960 comp rx=0 tx=0).stop 2024-03-30T00:42:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e66600640 1 -- 172.21.15.66:0/3825751246 shutdown_connections 2024-03-30T00:42:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e66600640 1 --2- 172.21.15.66:0/3825751246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e6015a370 0x7f7e600cea60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e66600640 1 --2- 172.21.15.66:0/3825751246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e600cda90 0x7f7e600ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e66600640 1 -- 172.21.15.66:0/3825751246 >> 172.21.15.66:0/3825751246 conn(0x7f7e60069f60 msgr2=0x7f7e6006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e66600640 1 -- 172.21.15.66:0/3825751246 shutdown_connections 2024-03-30T00:42:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.800+0000 7f7e66600640 1 -- 172.21.15.66:0/3825751246 wait complete. 2024-03-30T00:42:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e66600640 1 Processor -- start 2024-03-30T00:42:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e66600640 1 -- start start 2024-03-30T00:42:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e66600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e600cda90 0x7f7e6006c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e66600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e600cde70 0x7f7e60097940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e66600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e6015a370 0x7f7e60154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e66600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600ac330 con 0x7f7e600cde70 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e66600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600ac5b0 con 0x7f7e600cda90 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e66600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600ac890 con 0x7f7e6015a370 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e56a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e600cde70 0x7f7e60097940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e56a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e600cde70 0x7f7e60097940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38892/0 (socket says 172.21.15.66:38892) 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e56a00640 1 -- 172.21.15.66:0/3350549692 learned_addr learned my addr 172.21.15.66:0/3350549692 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e5f400640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e600cda90 0x7f7e6006c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e5fe00640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e6015a370 0x7f7e60154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e56a00640 1 -- 172.21.15.66:0/3350549692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e6015a370 msgr2=0x7f7e60154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e56a00640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e6015a370 0x7f7e60154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e56a00640 1 -- 172.21.15.66:0/3350549692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e600cda90 msgr2=0x7f7e6006c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e56a00640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e600cda90 0x7f7e6006c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e56a00640 1 -- 172.21.15.66:0/3350549692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e58009b30 con 0x7f7e600cde70 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e5f400640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e600cda90 0x7f7e6006c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e56a00640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e600cde70 0x7f7e60097940 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800c450 tx=0x7f7e4800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e57e00640 1 -- 172.21.15.66:0/3350549692 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48012830 con 0x7f7e600cde70 2024-03-30T00:42:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e57e00640 1 -- 172.21.15.66:0/3350549692 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e480129b0 con 0x7f7e600cde70 2024-03-30T00:42:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e57e00640 1 -- 172.21.15.66:0/3350549692 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48012d60 con 0x7f7e600cde70 2024-03-30T00:42:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e66600640 1 -- 172.21.15.66:0/3350549692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e600acb70 con 0x7f7e600cde70 2024-03-30T00:42:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.804+0000 7f7e66600640 1 -- 172.21.15.66:0/3350549692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6006d3b0 con 0x7f7e600cde70 2024-03-30T00:42:52.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.808+0000 7f7e66600640 1 -- 172.21.15.66:0/3350549692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e60066c30 con 0x7f7e600cde70 2024-03-30T00:42:52.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.808+0000 7f7e57e00640 1 -- 172.21.15.66:0/3350549692 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7e4801c460 con 0x7f7e600cde70 2024-03-30T00:42:52.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.808+0000 7f7e57e00640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e3c066690 0x7f7e3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:52.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.808+0000 7f7e57e00640 1 -- 172.21.15.66:0/3350549692 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7e48084240 con 0x7f7e600cde70 2024-03-30T00:42:52.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.808+0000 7f7e5f400640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e3c066690 0x7f7e3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:52.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.808+0000 7f7e5f400640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e3c066690 0x7f7e3c068b50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7e50006fd0 tx=0x7f7e50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:52.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:52.812+0000 7f7e57e00640 1 -- 172.21.15.66:0/3350549692 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7e48051390 con 0x7f7e600cde70 2024-03-30T00:42:53.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 -- 172.21.15.66:0/3350549692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e600ad020 con 0x7f7e600cde70 2024-03-30T00:42:53.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e57e00640 1 -- 172.21.15.66:0/3350549692 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f7e480549b0 con 0x7f7e600cde70 2024-03-30T00:42:53.019 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:53.019 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/465916334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":465916334},{"type":"v1","addr":"172.21.15.156:6835","nonce":465916334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1640047258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1640047258},{"type":"v1","addr":"172.21.15.156:6834","nonce":1640047258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5788,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2112021115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2112021115},{"type":"v1","addr":"172.21.15.66:6835","nonce":2112021115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:42:08.269705+0000","modified":"2024-03-30T00:42:11.492352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5985},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5985":{"gid":5985,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/3557086525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3557086525},{"type":"v1","addr":"172.21.15.66:6837","nonce":3557086525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T00:42:53.019 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-03-30T00:42:53.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 -- 172.21.15.66:0/3350549692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e3c066690 msgr2=0x7f7e3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e3c066690 0x7f7e3c068b50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7e50006fd0 tx=0x7f7e50008040 comp rx=0 tx=0).stop 2024-03-30T00:42:53.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 -- 172.21.15.66:0/3350549692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e600cde70 msgr2=0x7f7e60097940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e600cde70 0x7f7e60097940 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800c450 tx=0x7f7e4800c910 comp rx=0 tx=0).stop 2024-03-30T00:42:53.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 -- 172.21.15.66:0/3350549692 shutdown_connections 2024-03-30T00:42:53.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e6015a370 0x7f7e60154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e3c066690 0x7f7e3c068b50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e600cde70 0x7f7e60097940 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 --2- 172.21.15.66:0/3350549692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e600cda90 0x7f7e6006c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 -- 172.21.15.66:0/3350549692 >> 172.21.15.66:0/3350549692 conn(0x7f7e60069f60 msgr2=0x7f7e6015c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:53.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 -- 172.21.15.66:0/3350549692 shutdown_connections 2024-03-30T00:42:53.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.012+0000 7f7e66600640 1 -- 172.21.15.66:0/3350549692 wait complete. 2024-03-30T00:42:53.040 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-30T00:42:53.209 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-03-30T00:42:53.210 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-30T00:42:53.210 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-30T00:42:53.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.248+0000 7fc1bf2d3080 1 Processor -- start 2024-03-30T00:42:53.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.248+0000 7fc1bf2d3080 1 -- start start 2024-03-30T00:42:53.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.248+0000 7fc1bf2d3080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bbab30 0x5561f5c4bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.248+0000 7fc1bf2d3080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561f5bd4860 0x5561f5c4d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.248+0000 7fc1bf2d3080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5561f5bd4c40 0x5561f5b8f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.248+0000 7fc1bf2d3080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f5ad4960 con 0x5561f5bbab30 2024-03-30T00:42:53.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.248+0000 7fc1bf2d3080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f5ad4600 con 0x5561f5bd4860 2024-03-30T00:42:53.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.248+0000 7fc1bf2d3080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561f5acb690 con 0x5561f5bd4c40 2024-03-30T00:42:53.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.248+0000 7fc1bc400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bbab30 0x5561f5c4bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bc400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bbab30 0x5561f5c4bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38902/0 (socket says 172.21.15.66:38902) 2024-03-30T00:42:53.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bc400640 1 -- 172.21.15.66:0/2409856629 learned_addr learned my addr 172.21.15.66:0/2409856629 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:53.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bba00640 1 --2- 172.21.15.66:0/2409856629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561f5bd4860 0x5561f5c4d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bc400640 1 -- 172.21.15.66:0/2409856629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5561f5bd4c40 msgr2=0x5561f5b8f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:53.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bce00640 1 --2- 172.21.15.66:0/2409856629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5561f5bd4c40 0x5561f5b8f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bc400640 1 --2- 172.21.15.66:0/2409856629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5561f5bd4c40 0x5561f5b8f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bc400640 1 -- 172.21.15.66:0/2409856629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561f5bd4860 msgr2=0x5561f5c4d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:53.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bc400640 1 --2- 172.21.15.66:0/2409856629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561f5bd4860 0x5561f5c4d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bc400640 1 -- 172.21.15.66:0/2409856629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b400a1c0 con 0x5561f5bbab30 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bba00640 1 --2- 172.21.15.66:0/2409856629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561f5bd4860 0x5561f5c4d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bce00640 1 --2- 172.21.15.66:0/2409856629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5561f5bd4c40 0x5561f5b8f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bc400640 1 --2- 172.21.15.66:0/2409856629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bbab30 0x5561f5c4bf90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00a750 tx=0x7fc1ac00ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d371dcb662de99a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1ba600640 1 -- 172.21.15.66:0/2409856629 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00f070 con 0x5561f5bbab30 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1ba600640 1 -- 172.21.15.66:0/2409856629 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac008c00 con 0x5561f5bbab30 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1ba600640 1 -- 172.21.15.66:0/2409856629 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0133f0 con 0x5561f5bbab30 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2409856629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bbab30 msgr2=0x5561f5c4bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 --2- 172.21.15.66:0/2409856629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bbab30 0x5561f5c4bf90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00a750 tx=0x7fc1ac00ac10 comp rx=0 tx=0).stop 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2409856629 shutdown_connections 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 --2- 172.21.15.66:0/2409856629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5561f5bd4c40 0x5561f5b8f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 --2- 172.21.15.66:0/2409856629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561f5bd4860 0x5561f5c4d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 --2- 172.21.15.66:0/2409856629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bbab30 0x5561f5c4bf90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2409856629 >> 172.21.15.66:0/2409856629 conn(0x5561f5bb7dd0 msgr2=0x5561f5b8b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:53.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2409856629 shutdown_connections 2024-03-30T00:42:53.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2409856629 wait complete. 2024-03-30T00:42:53.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 Processor -- start 2024-03-30T00:42:53.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 -- start start 2024-03-30T00:42:53.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561f5bbab30 0x5561f5bb6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bd4860 0x5561f5c4d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5561f5bd4c40 0x5561f5c4b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f5cfee70 con 0x5561f5bd4860 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f5cff110 con 0x5561f5bbab30 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bf2d3080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561f5cff3b0 con 0x5561f5bd4c40 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.252+0000 7fc1bba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bd4860 0x5561f5c4d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bd4860 0x5561f5c4d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38906/0 (socket says 172.21.15.66:38906) 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bba00640 1 -- 172.21.15.66:0/2861663032 learned_addr learned my addr 172.21.15.66:0/2861663032 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bc400640 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561f5bbab30 0x5561f5bb6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bce00640 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5561f5bd4c40 0x5561f5c4b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bba00640 1 -- 172.21.15.66:0/2861663032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5561f5bd4c40 msgr2=0x5561f5c4b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bba00640 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5561f5bd4c40 0x5561f5c4b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bba00640 1 -- 172.21.15.66:0/2861663032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561f5bbab30 msgr2=0x5561f5bb6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bba00640 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561f5bbab30 0x5561f5bb6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bba00640 1 -- 172.21.15.66:0/2861663032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac00a430 con 0x5561f5bd4860 2024-03-30T00:42:53.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bce00640 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5561f5bd4c40 0x5561f5c4b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:53.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bba00640 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bd4860 0x5561f5c4d3c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8001ec0 tx=0x7fc1a800a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:53.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1b9200640 1 -- 172.21.15.66:0/2861663032 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8007040 con 0x5561f5bd4860 2024-03-30T00:42:53.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1b9200640 1 -- 172.21.15.66:0/2861663032 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a800ae30 con 0x5561f5bd4860 2024-03-30T00:42:53.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2861663032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5561f5ae38c0 con 0x5561f5bd4860 2024-03-30T00:42:53.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1b9200640 1 -- 172.21.15.66:0/2861663032 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a800f860 con 0x5561f5bd4860 2024-03-30T00:42:53.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2861663032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561f5ad6d00 con 0x5561f5bd4860 2024-03-30T00:42:53.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1b9200640 1 -- 172.21.15.66:0/2861663032 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc1a800fa80 con 0x5561f5bd4860 2024-03-30T00:42:53.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1b9200640 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc19c065070 0x7fc19c067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1b9200640 1 -- 172.21.15.66:0/2861663032 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc1a8083440 con 0x5561f5bd4860 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.256+0000 7fc1bc400640 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc19c065070 0x7fc19c067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5561f5cfe9d0 0x5561f5bb2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2861663032 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56) v8 -- 0x5561f5c4d900 con 0x5561f5cfe9d0 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bce00640 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5561f5cfe9d0 0x5561f5bb2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bc400640 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc19c065070 0x7fc19c067530 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac000c10 tx=0x7fc1ac00c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bce00640 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5561f5cfe9d0 0x5561f5bb2eb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bce00640 1 -- 172.21.15.66:0/2861663032 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fc1b40aa6c0 con 0x5561f5cfe9d0 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5561f5cfe9d0 msgr2=0x5561f5bb2eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5561f5cfe9d0 0x5561f5bb2eb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc19c065070 msgr2=0x7fc19c067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc19c065070 0x7fc19c067530 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac000c10 tx=0x7fc1ac00c000 comp rx=0 tx=0).stop 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bd4860 msgr2=0x5561f5c4d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 --2- 172.21.15.66:0/2861663032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5561f5bd4860 0x5561f5c4d3c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8001ec0 tx=0x7fc1a800a600 comp rx=0 tx=0).stop 2024-03-30T00:42:53.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bce00640 1 -- 172.21.15.66:0/2861663032 reap_dead start 2024-03-30T00:42:53.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2861663032 shutdown_connections 2024-03-30T00:42:53.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2861663032 >> 172.21.15.66:0/2861663032 conn(0x5561f5bb7dd0 msgr2=0x5561f5b8f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:53.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2861663032 shutdown_connections 2024-03-30T00:42:53.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.260+0000 7fc1bf2d3080 1 -- 172.21.15.66:0/2861663032 wait complete. 2024-03-30T00:42:53.271 INFO:teuthology.orchestra.run.smithi066.stdout:Yalphaparent_a 2024-03-30T00:42:53.271 DEBUG:teuthology.orchestra.run.smithi066:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-30T00:42:53.361 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:42:53.361 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627777, 2024-03-30T00:42:53.361 INFO:teuthology.orchestra.run.smithi066.stdout: "ancestors": [ 2024-03-30T00:42:53.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:53.361 INFO:teuthology.orchestra.run.smithi066.stdout: "dirino": 1099511627776, 2024-03-30T00:42:53.361 INFO:teuthology.orchestra.run.smithi066.stdout: "dname": "alpha", 2024-03-30T00:42:53.361 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 4 2024-03-30T00:42:53.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:42:53.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:42:53.362 INFO:teuthology.orchestra.run.smithi066.stdout: "dirino": 1, 2024-03-30T00:42:53.362 INFO:teuthology.orchestra.run.smithi066.stdout: "dname": "parent_a", 2024-03-30T00:42:53.362 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 7 2024-03-30T00:42:53.362 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:42:53.362 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T00:42:53.362 INFO:teuthology.orchestra.run.smithi066.stdout: "pool": 8, 2024-03-30T00:42:53.362 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [] 2024-03-30T00:42:53.362 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:42:53.362 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:53.367 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:42:53.367 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:42:53.495 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:42:53.594 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:42:53.594 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:42:53.594 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:42:53.603 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:42:53.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.780+0000 7f237b400640 1 Processor -- start 2024-03-30T00:42:53.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.780+0000 7f237b400640 1 -- start start 2024-03-30T00:42:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f237b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f237415ca10 0x7f237406ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f237b400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237406e7d0 con 0x7f23740ae4e0 2024-03-30T00:42:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f237b400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237406f1a0 con 0x7f23740ce820 2024-03-30T00:42:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f237b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740b1f70 con 0x7f237415ca10 2024-03-30T00:42:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f2373e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f23740ae4e0 0x7f23740cb620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56196/0 (socket says 172.21.15.66:56196) 2024-03-30T00:42:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f2373e00640 1 -- 172.21.15.66:0/518540117 learned_addr learned my addr 172.21.15.66:0/518540117 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f2378a00640 1 --2- 172.21.15.66:0/518540117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f237415ca10 0x7f237406ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f2373400640 1 -- 172.21.15.66:0/518540117 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014885545 0 0) 0x7f237406e7d0 con 0x7f23740ae4e0 2024-03-30T00:42:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f2373400640 1 -- 172.21.15.66:0/518540117 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f235c003530 con 0x7f23740ae4e0 2024-03-30T00:42:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f2373400640 1 -- 172.21.15.66:0/518540117 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224120763 0 0) 0x7f237406f1a0 con 0x7f23740ce820 2024-03-30T00:42:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f2373400640 1 -- 172.21.15.66:0/518540117 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f237406e7d0 con 0x7f23740ce820 2024-03-30T00:42:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f2378a00640 1 -- 172.21.15.66:0/518540117 >> v1:172.21.15.156:6789/0 conn(0x7f23740ce820 legacy=0x7f2374160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f2378a00640 1 -- 172.21.15.66:0/518540117 >> v1:172.21.15.66:6789/0 conn(0x7f23740ae4e0 legacy=0x7f23740cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.784+0000 7f2378a00640 1 -- 172.21.15.66:0/518540117 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237415cdf0 con 0x7f237415ca10 2024-03-30T00:42:53.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f2378a00640 1 --2- 172.21.15.66:0/518540117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f237415ca10 0x7f237406ec60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f23680097f0 tx=0x7f236801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99d2170ff88a43f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:53.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f2373400640 1 -- 172.21.15.66:0/518540117 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23680ac620 con 0x7f237415ca10 2024-03-30T00:42:53.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f2373400640 1 -- 172.21.15.66:0/518540117 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23680ac780 con 0x7f237415ca10 2024-03-30T00:42:53.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f2373400640 1 -- 172.21.15.66:0/518540117 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23680acb10 con 0x7f237415ca10 2024-03-30T00:42:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f237b400640 1 -- 172.21.15.66:0/518540117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f237415ca10 msgr2=0x7f237406ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f237b400640 1 --2- 172.21.15.66:0/518540117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f237415ca10 0x7f237406ec60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f23680097f0 tx=0x7f236801c4b0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f237b400640 1 -- 172.21.15.66:0/518540117 shutdown_connections 2024-03-30T00:42:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f237b400640 1 --2- 172.21.15.66:0/518540117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f237415ca10 0x7f237406ec60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f237b400640 1 -- 172.21.15.66:0/518540117 >> 172.21.15.66:0/518540117 conn(0x7f2374098a20 msgr2=0x7f23740b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f237b400640 1 -- 172.21.15.66:0/518540117 shutdown_connections 2024-03-30T00:42:53.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f237b400640 1 -- 172.21.15.66:0/518540117 wait complete. 2024-03-30T00:42:53.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f237b400640 1 Processor -- start 2024-03-30T00:42:53.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f237b400640 1 -- start start 2024-03-30T00:42:53.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f237b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23740ae4e0 0x7f2374156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.788+0000 7f237b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23740ce820 0x7f23741576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f237b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237415ca10 0x7f23740c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f237b400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c4440 con 0x7f23740ce820 2024-03-30T00:42:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f237b400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740b3030 con 0x7f237415ca10 2024-03-30T00:42:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f237b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c3d80 con 0x7f23740ae4e0 2024-03-30T00:42:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2373e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23740ce820 0x7f23741576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2373e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23740ce820 0x7f23741576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38916/0 (socket says 172.21.15.66:38916) 2024-03-30T00:42:53.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2373e00640 1 -- 172.21.15.66:0/1429417916 learned_addr learned my addr 172.21.15.66:0/1429417916 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2378a00640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23740ae4e0 0x7f2374156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2379400640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237415ca10 0x7f23740c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2373e00640 1 -- 172.21.15.66:0/1429417916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23740ae4e0 msgr2=0x7f2374156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2373e00640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23740ae4e0 0x7f2374156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2373e00640 1 -- 172.21.15.66:0/1429417916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237415ca10 msgr2=0x7f23740c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2373e00640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237415ca10 0x7f23740c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2373e00640 1 -- 172.21.15.66:0/1429417916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368009420 con 0x7f23740ce820 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2379400640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237415ca10 0x7f23740c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2373e00640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23740ce820 0x7f23741576a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f236000c4d0 tx=0x7f236000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2370c00640 1 -- 172.21.15.66:0/1429417916 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360005cf0 con 0x7f23740ce820 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2370c00640 1 -- 172.21.15.66:0/1429417916 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360005e50 con 0x7f23740ce820 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2370c00640 1 -- 172.21.15.66:0/1429417916 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360007290 con 0x7f23740ce820 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f237b400640 1 -- 172.21.15.66:0/1429417916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23740c4060 con 0x7f23740ce820 2024-03-30T00:42:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f237b400640 1 -- 172.21.15.66:0/1429417916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23740b2c30 con 0x7f23740ce820 2024-03-30T00:42:53.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f237b400640 1 -- 172.21.15.66:0/1429417916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2374066c30 con 0x7f23740ce820 2024-03-30T00:42:53.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.792+0000 7f2370c00640 1 -- 172.21.15.66:0/1429417916 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f23600073f0 con 0x7f23740ce820 2024-03-30T00:42:53.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.796+0000 7f2370c00640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2354066690 0x7f2354068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.796+0000 7f2370c00640 1 -- 172.21.15.66:0/1429417916 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f2360083950 con 0x7f23740ce820 2024-03-30T00:42:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.796+0000 7f2378a00640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2354066690 0x7f2354068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.796+0000 7f2378a00640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2354066690 0x7f2354068b50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f23680ae000 tx=0x7f23680a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.800+0000 7f2370c00640 1 -- 172.21.15.66:0/1429417916 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2360050b30 con 0x7f23740ce820 2024-03-30T00:42:53.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.960+0000 7f237b400640 1 -- 172.21.15.66:0/1429417916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23740cf9d0 con 0x7f23740ce820 2024-03-30T00:42:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.960+0000 7f2370c00640 1 -- 172.21.15.66:0/1429417916 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16402 (secure 0 0 0) 0x7f2360017020 con 0x7f23740ce820 2024-03-30T00:42:53.968 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:53.968 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":56,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:42:33.561663+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:42:06.807254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-30T00:42:07.258973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:42:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 -- 172.21.15.66:0/1429417916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2354066690 msgr2=0x7f2354068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2354066690 0x7f2354068b50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f23680ae000 tx=0x7f23680a7000 comp rx=0 tx=0).stop 2024-03-30T00:42:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 -- 172.21.15.66:0/1429417916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23740ce820 msgr2=0x7f23741576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23740ce820 0x7f23741576a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f236000c4d0 tx=0x7f236000c990 comp rx=0 tx=0).stop 2024-03-30T00:42:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 -- 172.21.15.66:0/1429417916 shutdown_connections 2024-03-30T00:42:53.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237415ca10 0x7f23740c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2354066690 0x7f2354068b50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23740ce820 0x7f23741576a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 --2- 172.21.15.66:0/1429417916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23740ae4e0 0x7f2374156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:53.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 -- 172.21.15.66:0/1429417916 >> 172.21.15.66:0/1429417916 conn(0x7f2374098a20 msgr2=0x7f23740ac2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:53.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 -- 172.21.15.66:0/1429417916 shutdown_connections 2024-03-30T00:42:53.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:53.964+0000 7f237b400640 1 -- 172.21.15.66:0/1429417916 wait complete. 2024-03-30T00:42:53.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:42:06.807254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:42:07.258973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:42:53.990 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:42:54.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.124+0000 7fd628400640 1 Processor -- start 2024-03-30T00:42:54.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.124+0000 7fd628400640 1 -- start start 2024-03-30T00:42:54.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd628400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd6200cec30 0x7fd620159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd628400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd6200cde70 0x7fd62015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd628400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62015a6e0 con 0x7fd6200cda90 2024-03-30T00:42:54.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd628400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6200b1b80 con 0x7fd6200cde70 2024-03-30T00:42:54.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd628400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6200cd3e0 con 0x7fd6200cec30 2024-03-30T00:42:54.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd626400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd6200cde70 0x7fd62015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd626400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd6200cde70 0x7fd62015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38926/0 (socket says 172.21.15.66:38926) 2024-03-30T00:42:54.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd626400640 1 -- 172.21.15.66:0/1962031514 learned_addr learned my addr 172.21.15.66:0/1962031514 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:54.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd625000640 1 --2- 172.21.15.66:0/1962031514 >> v2:172.21.15.156:3300/0 conn(0x7fd6200cec30 0x7fd620159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd626400640 1 -- 172.21.15.66:0/1962031514 >> v2:172.21.15.156:3300/0 conn(0x7fd6200cec30 msgr2=0x7fd620159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:54.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd626400640 1 --2- 172.21.15.66:0/1962031514 >> v2:172.21.15.156:3300/0 conn(0x7fd6200cec30 0x7fd620159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd626400640 1 -- 172.21.15.66:0/1962031514 >> v1:172.21.15.66:6789/0 conn(0x7fd6200cda90 legacy=0x7fd6200ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd626400640 1 -- 172.21.15.66:0/1962031514 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62015a990 con 0x7fd6200cde70 2024-03-30T00:42:54.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.128+0000 7fd61fe00640 1 -- 172.21.15.66:0/1962031514 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159853472 0 0) 0x7fd62015a6e0 con 0x7fd6200cda90 2024-03-30T00:42:54.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd626400640 1 --2- 172.21.15.66:0/1962031514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6200cde70 0x7fd62015a1a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd6200b3110 tx=0x7fd61801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f344e5c49385de6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:54.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd61fe00640 1 -- 172.21.15.66:0/1962031514 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180a5820 con 0x7fd6200cde70 2024-03-30T00:42:54.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd61fe00640 1 -- 172.21.15.66:0/1962031514 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6180a5980 con 0x7fd6200cde70 2024-03-30T00:42:54.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd61fe00640 1 -- 172.21.15.66:0/1962031514 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180a5d10 con 0x7fd6200cde70 2024-03-30T00:42:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 -- 172.21.15.66:0/1962031514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6200cde70 msgr2=0x7fd62015a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 --2- 172.21.15.66:0/1962031514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6200cde70 0x7fd62015a1a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd6200b3110 tx=0x7fd61801c960 comp rx=0 tx=0).stop 2024-03-30T00:42:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 -- 172.21.15.66:0/1962031514 shutdown_connections 2024-03-30T00:42:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 --2- 172.21.15.66:0/1962031514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6200cde70 0x7fd62015a1a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 --2- 172.21.15.66:0/1962031514 >> v2:172.21.15.156:3300/0 conn(0x7fd6200cec30 0x7fd620159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 -- 172.21.15.66:0/1962031514 >> 172.21.15.66:0/1962031514 conn(0x7fd620069f60 msgr2=0x7fd62006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 -- 172.21.15.66:0/1962031514 shutdown_connections 2024-03-30T00:42:54.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 -- 172.21.15.66:0/1962031514 wait complete. 2024-03-30T00:42:54.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 Processor -- start 2024-03-30T00:42:54.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 -- start start 2024-03-30T00:42:54.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6200cda90 0x7fd6200c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6200cde70 0x7fd620071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6200cec30 0x7fd6200ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200c2b40 con 0x7fd6200cec30 2024-03-30T00:42:54.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200af540 con 0x7fd6200cde70 2024-03-30T00:42:54.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd628400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200c2d70 con 0x7fd6200cda90 2024-03-30T00:42:54.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd626400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6200cec30 0x7fd6200ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd626400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6200cec30 0x7fd6200ac480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38936/0 (socket says 172.21.15.66:38936) 2024-03-30T00:42:54.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.132+0000 7fd626400640 1 -- 172.21.15.66:0/3729589888 learned_addr learned my addr 172.21.15.66:0/3729589888 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:54.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd625000640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6200cde70 0x7fd620071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd625a00640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6200cda90 0x7fd6200c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd626400640 1 -- 172.21.15.66:0/3729589888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6200cda90 msgr2=0x7fd6200c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd626400640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6200cda90 0x7fd6200c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd626400640 1 -- 172.21.15.66:0/3729589888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6200cde70 msgr2=0x7fd620071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd626400640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6200cde70 0x7fd620071180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd626400640 1 -- 172.21.15.66:0/3729589888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd618009b30 con 0x7fd6200cec30 2024-03-30T00:42:54.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd626400640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6200cec30 0x7fd6200ac480 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd618019570 tx=0x7fd618019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:54.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd61d600640 1 -- 172.21.15.66:0/3729589888 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180b8670 con 0x7fd6200cec30 2024-03-30T00:42:54.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd61d600640 1 -- 172.21.15.66:0/3729589888 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6180b87d0 con 0x7fd6200cec30 2024-03-30T00:42:54.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd61d600640 1 -- 172.21.15.66:0/3729589888 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180b8b60 con 0x7fd6200cec30 2024-03-30T00:42:54.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd628400640 1 -- 172.21.15.66:0/3729589888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6200c3050 con 0x7fd6200cec30 2024-03-30T00:42:54.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd628400640 1 -- 172.21.15.66:0/3729589888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6200c4070 con 0x7fd6200cec30 2024-03-30T00:42:54.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd628400640 1 -- 172.21.15.66:0/3729589888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6200c3850 con 0x7fd6200cec30 2024-03-30T00:42:54.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd61d600640 1 -- 172.21.15.66:0/3729589888 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd6180b8cc0 con 0x7fd6200cec30 2024-03-30T00:42:54.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd61d600640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd600066690 0x7fd600068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.136+0000 7fd61d600640 1 -- 172.21.15.66:0/3729589888 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fd61811f830 con 0x7fd6200cec30 2024-03-30T00:42:54.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.140+0000 7fd625a00640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd600066690 0x7fd600068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.140+0000 7fd625a00640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd600066690 0x7fd600068b50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd614000f70 tx=0x7fd614009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:54.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.140+0000 7fd61d600640 1 -- 172.21.15.66:0/3729589888 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd6180eca30 con 0x7fd6200cec30 2024-03-30T00:42:54.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.304+0000 7fd628400640 1 -- 172.21.15.66:0/3729589888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd620066c30 con 0x7fd6200cec30 2024-03-30T00:42:54.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.304+0000 7fd61d600640 1 -- 172.21.15.66:0/3729589888 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6180f0050 con 0x7fd6200cec30 2024-03-30T00:42:54.311 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:42:54.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.304+0000 7fd628400640 1 -- 172.21.15.66:0/3729589888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd600066690 msgr2=0x7fd600068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.304+0000 7fd628400640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd600066690 0x7fd600068b50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd614000f70 tx=0x7fd614009040 comp rx=0 tx=0).stop 2024-03-30T00:42:54.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.308+0000 7fd628400640 1 -- 172.21.15.66:0/3729589888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6200cec30 msgr2=0x7fd6200ac480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.308+0000 7fd628400640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6200cec30 0x7fd6200ac480 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd618019570 tx=0x7fd618019700 comp rx=0 tx=0).stop 2024-03-30T00:42:54.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.308+0000 7fd628400640 1 -- 172.21.15.66:0/3729589888 shutdown_connections 2024-03-30T00:42:54.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.308+0000 7fd628400640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd600066690 0x7fd600068b50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.308+0000 7fd628400640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6200cec30 0x7fd6200ac480 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.308+0000 7fd628400640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6200cde70 0x7fd620071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.308+0000 7fd628400640 1 --2- 172.21.15.66:0/3729589888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6200cda90 0x7fd6200c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.308+0000 7fd628400640 1 -- 172.21.15.66:0/3729589888 >> 172.21.15.66:0/3729589888 conn(0x7fd620069f60 msgr2=0x7fd6200cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:54.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.308+0000 7fd628400640 1 -- 172.21.15.66:0/3729589888 shutdown_connections 2024-03-30T00:42:54.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.308+0000 7fd628400640 1 -- 172.21.15.66:0/3729589888 wait complete. 2024-03-30T00:42:54.334 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:42:54.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.468+0000 7f506c200640 1 Processor -- start 2024-03-30T00:42:54.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.468+0000 7f506c200640 1 -- start start 2024-03-30T00:42:54.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f506c200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f50640cda90 0x7f50640ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f506c200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50640cde70 0x7f50640cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f506c200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506415a990 con 0x7f5064159c60 2024-03-30T00:42:54.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f506c200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50640b1b80 con 0x7f50640cda90 2024-03-30T00:42:54.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f506c200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50640cd3e0 con 0x7f50640cde70 2024-03-30T00:42:54.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f5069800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f50640cda90 0x7f50640ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f5069800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f50640cda90 0x7f50640ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38950/0 (socket says 172.21.15.66:38950) 2024-03-30T00:42:54.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f5069800640 1 -- 172.21.15.66:0/1180441895 learned_addr learned my addr 172.21.15.66:0/1180441895 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:54.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f5068e00640 1 --2- 172.21.15.66:0/1180441895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50640cde70 0x7f50640cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f5069800640 1 -- 172.21.15.66:0/1180441895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50640cde70 msgr2=0x7f50640cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:54.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f5069800640 1 --2- 172.21.15.66:0/1180441895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50640cde70 0x7f50640cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f5069800640 1 -- 172.21.15.66:0/1180441895 >> v1:172.21.15.156:6789/0 conn(0x7f5064159c60 legacy=0x7f506415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:54.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f5069800640 1 -- 172.21.15.66:0/1180441895 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50640b11e0 con 0x7f50640cda90 2024-03-30T00:42:54.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.472+0000 7f5068e00640 1 --2- 172.21.15.66:0/1180441895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50640cde70 0x7f50640cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:54.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f5069800640 1 --2- 172.21.15.66:0/1180441895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50640cda90 0x7f50640ce520 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5058009870 tx=0x7f505801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dbdf34c8c8b9a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:54.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f5063e00640 1 -- 172.21.15.66:0/1180441895 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580ad660 con 0x7f50640cda90 2024-03-30T00:42:54.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f5063e00640 1 -- 172.21.15.66:0/1180441895 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50580ad7c0 con 0x7f50640cda90 2024-03-30T00:42:54.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f5063e00640 1 -- 172.21.15.66:0/1180441895 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580adb50 con 0x7f50640cda90 2024-03-30T00:42:54.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f506c200640 1 -- 172.21.15.66:0/1180441895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50640cda90 msgr2=0x7f50640ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f506c200640 1 --2- 172.21.15.66:0/1180441895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50640cda90 0x7f50640ce520 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5058009870 tx=0x7f505801c490 comp rx=0 tx=0).stop 2024-03-30T00:42:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f506c200640 1 -- 172.21.15.66:0/1180441895 shutdown_connections 2024-03-30T00:42:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f506c200640 1 --2- 172.21.15.66:0/1180441895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50640cde70 0x7f50640cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f506c200640 1 --2- 172.21.15.66:0/1180441895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50640cda90 0x7f50640ce520 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f506c200640 1 -- 172.21.15.66:0/1180441895 >> 172.21.15.66:0/1180441895 conn(0x7f5064069f60 msgr2=0x7f506406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f506c200640 1 -- 172.21.15.66:0/1180441895 shutdown_connections 2024-03-30T00:42:54.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f506c200640 1 -- 172.21.15.66:0/1180441895 wait complete. 2024-03-30T00:42:54.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f506c200640 1 Processor -- start 2024-03-30T00:42:54.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.476+0000 7f506c200640 1 -- start start 2024-03-30T00:42:54.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f506c200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50640cda90 0x7f50640b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f506c200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50640cde70 0x7f5064097080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f506c200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5064159c60 0x7f50640ac060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f506c200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5064156310 con 0x7f50640cda90 2024-03-30T00:42:54.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f506c200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5064155790 con 0x7f5064159c60 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f506c200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5064155a70 con 0x7f50640cde70 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5069800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50640cda90 0x7f50640b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5069800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50640cda90 0x7f50640b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38954/0 (socket says 172.21.15.66:38954) 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5069800640 1 -- 172.21.15.66:0/4064713971 learned_addr learned my addr 172.21.15.66:0/4064713971 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f506a200640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5064159c60 0x7f50640ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5068e00640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50640cde70 0x7f5064097080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5069800640 1 -- 172.21.15.66:0/4064713971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50640cde70 msgr2=0x7f5064097080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5069800640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50640cde70 0x7f5064097080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5069800640 1 -- 172.21.15.66:0/4064713971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5064159c60 msgr2=0x7f50640ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5069800640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5064159c60 0x7f50640ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5069800640 1 -- 172.21.15.66:0/4064713971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50580094d0 con 0x7f50640cda90 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5069800640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50640cda90 0x7f50640b0360 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f505801ab40 tx=0x7f5058019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:54.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5061600640 1 -- 172.21.15.66:0/4064713971 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580b6850 con 0x7f50640cda90 2024-03-30T00:42:54.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5061600640 1 -- 172.21.15.66:0/4064713971 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50580b69d0 con 0x7f50640cda90 2024-03-30T00:42:54.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5061600640 1 -- 172.21.15.66:0/4064713971 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580b6d80 con 0x7f50640cda90 2024-03-30T00:42:54.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f506c200640 1 -- 172.21.15.66:0/4064713971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5064155d50 con 0x7f50640cda90 2024-03-30T00:42:54.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f506c200640 1 -- 172.21.15.66:0/4064713971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50640b08a0 con 0x7f50640cda90 2024-03-30T00:42:54.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f506c200640 1 -- 172.21.15.66:0/4064713971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5064157710 con 0x7f50640cda90 2024-03-30T00:42:54.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.480+0000 7f5061600640 1 -- 172.21.15.66:0/4064713971 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f50580b4070 con 0x7f50640cda90 2024-03-30T00:42:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.484+0000 7f5061600640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 0x7f5044068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.484+0000 7f5061600640 1 -- 172.21.15.66:0/4064713971 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f505811e5c0 con 0x7f50640cda90 2024-03-30T00:42:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.484+0000 7f5068e00640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 0x7f5044068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.484+0000 7f5068e00640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 0x7f5044068b50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f50640716d0 tx=0x7f5050008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:54.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.488+0000 7f5061600640 1 -- 172.21.15.66:0/4064713971 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5064157710 con 0x7f50640cda90 2024-03-30T00:42:54.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 -- 172.21.15.66:0/4064713971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5064157710 con 0x7f50640cda90 2024-03-30T00:42:54.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f5061600640 1 -- 172.21.15.66:0/4064713971 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5064157710 con 0x7f50640cda90 2024-03-30T00:42:54.654 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:42:54.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 -- 172.21.15.66:0/4064713971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 msgr2=0x7f5044068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 0x7f5044068b50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f50640716d0 tx=0x7f5050008040 comp rx=0 tx=0).stop 2024-03-30T00:42:54.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 -- 172.21.15.66:0/4064713971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50640cda90 msgr2=0x7f50640b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50640cda90 0x7f50640b0360 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f505801ab40 tx=0x7f5058019230 comp rx=0 tx=0).stop 2024-03-30T00:42:54.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 -- 172.21.15.66:0/4064713971 shutdown_connections 2024-03-30T00:42:54.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5064159c60 0x7f50640ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50640cde70 0x7f5064097080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 0x7f5044068b50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 --2- 172.21.15.66:0/4064713971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50640cda90 0x7f50640b0360 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 -- 172.21.15.66:0/4064713971 >> 172.21.15.66:0/4064713971 conn(0x7f5064069f60 msgr2=0x7f506414f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:54.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 -- 172.21.15.66:0/4064713971 shutdown_connections 2024-03-30T00:42:54.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.648+0000 7f506c200640 1 -- 172.21.15.66:0/4064713971 wait complete. 2024-03-30T00:42:54.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:42:54.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.812+0000 7fe4f1e00640 1 Processor -- start 2024-03-30T00:42:54.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.812+0000 7fe4f1e00640 1 -- start start 2024-03-30T00:42:54.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.812+0000 7fe4f1e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe4ec0b19a0 0x7fe4ec0c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.812+0000 7fe4f1e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe4ec0c1800 0x7fe4ec0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.812+0000 7fe4f1e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ec0c97f0 con 0x7fe4ec160c10 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.812+0000 7fe4f1e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4ec0c1c40 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.812+0000 7fe4f1e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4ec0c9a30 con 0x7fe4ec0c1800 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.812+0000 7fe4e3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe4ec0b19a0 0x7fe4ec0c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.812+0000 7fe4e3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe4ec0b19a0 0x7fe4ec0c12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38968/0 (socket says 172.21.15.66:38968) 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.812+0000 7fe4e3e00640 1 -- 172.21.15.66:0/1368259249 learned_addr learned my addr 172.21.15.66:0/1368259249 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.812+0000 7fe4eaa00640 1 --1- 172.21.15.66:0/1368259249 >> v1:172.21.15.66:6789/0 conn(0x7fe4ec160c10 0x7fe4ec0c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56216/0 (socket says 172.21.15.66:56216) 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4eb400640 1 --2- 172.21.15.66:0/1368259249 >> v2:172.21.15.156:3300/0 conn(0x7fe4ec0c1800 0x7fe4ec0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4e3e00640 1 -- 172.21.15.66:0/1368259249 >> v2:172.21.15.156:3300/0 conn(0x7fe4ec0c1800 msgr2=0x7fe4ec0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4e3e00640 1 --2- 172.21.15.66:0/1368259249 >> v2:172.21.15.156:3300/0 conn(0x7fe4ec0c1800 0x7fe4ec0cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4e3e00640 1 -- 172.21.15.66:0/1368259249 >> v1:172.21.15.66:6789/0 conn(0x7fe4ec160c10 legacy=0x7fe4ec0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4e3e00640 1 -- 172.21.15.66:0/1368259249 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec0c9d10 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4ea000640 1 -- 172.21.15.66:0/1368259249 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461817976 0 0) 0x7fe4dc001120 con 0x7fe4ec160c10 2024-03-30T00:42:54.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4e3e00640 1 --2- 172.21.15.66:0/1368259249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ec0b19a0 0x7fe4ec0c12c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0cf340 tx=0x7fe4d401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c378a479407ce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:54.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4ea000640 1 -- 172.21.15.66:0/1368259249 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40aa650 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4ea000640 1 -- 172.21.15.66:0/1368259249 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d40aa7b0 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4ea000640 1 -- 172.21.15.66:0/1368259249 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40aab40 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1368259249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ec0b19a0 msgr2=0x7fe4ec0c12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4f1e00640 1 --2- 172.21.15.66:0/1368259249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ec0b19a0 0x7fe4ec0c12c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0cf340 tx=0x7fe4d401c4f0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1368259249 shutdown_connections 2024-03-30T00:42:54.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4f1e00640 1 --2- 172.21.15.66:0/1368259249 >> v2:172.21.15.156:3300/0 conn(0x7fe4ec0c1800 0x7fe4ec0cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4f1e00640 1 --2- 172.21.15.66:0/1368259249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ec0b19a0 0x7fe4ec0c12c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.816+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1368259249 >> 172.21.15.66:0/1368259249 conn(0x7fe4ec069f60 msgr2=0x7fe4ec06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:54.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1368259249 shutdown_connections 2024-03-30T00:42:54.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1368259249 wait complete. 2024-03-30T00:42:54.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4f1e00640 1 Processor -- start 2024-03-30T00:42:54.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4f1e00640 1 -- start start 2024-03-30T00:42:54.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4f1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ec0b19a0 0x7fe4ec0ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4f1e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4ec0c1800 0x7fe4ec15b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4f1e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ec160c10 0x7fe4ec15bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4f1e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec0abf50 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4f1e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec15fb60 con 0x7fe4ec160c10 2024-03-30T00:42:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4f1e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec15c660 con 0x7fe4ec0c1800 2024-03-30T00:42:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4eaa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ec0b19a0 0x7fe4ec0ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4eaa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ec0b19a0 0x7fe4ec0ab770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38982/0 (socket says 172.21.15.66:38982) 2024-03-30T00:42:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4eaa00640 1 -- 172.21.15.66:0/1979579853 learned_addr learned my addr 172.21.15.66:0/1979579853 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4eb400640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ec160c10 0x7fe4ec15bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4e3e00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4ec0c1800 0x7fe4ec15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4eaa00640 1 -- 172.21.15.66:0/1979579853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4ec0c1800 msgr2=0x7fe4ec15b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4eaa00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4ec0c1800 0x7fe4ec15b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4eaa00640 1 -- 172.21.15.66:0/1979579853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ec160c10 msgr2=0x7fe4ec15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4eaa00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ec160c10 0x7fe4ec15bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4eaa00640 1 -- 172.21.15.66:0/1979579853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d40094d0 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4eaa00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ec0b19a0 0x7fe4ec0ab770 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00c480 tx=0x7fe4dc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4e3400640 1 -- 172.21.15.66:0/1979579853 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc0074e0 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4e3400640 1 -- 172.21.15.66:0/1979579853 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc007660 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.820+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1979579853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ec0abcb0 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.824+0000 7fe4e3400640 1 -- 172.21.15.66:0/1979579853 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc004a30 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.824+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1979579853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ec15f080 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.824+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1979579853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ec066c30 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.824+0000 7fe4e3400640 1 -- 172.21.15.66:0/1979579853 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe4dc004b90 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.824+0000 7fe4e3400640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4c8066690 0x7fe4c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.824+0000 7fe4e3400640 1 -- 172.21.15.66:0/1979579853 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fe4dc0835d0 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.824+0000 7fe4e3e00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4c8066690 0x7fe4c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.824+0000 7fe4e3e00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4c8066690 0x7fe4c8068b50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40ae000 tx=0x7fe4d40b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:54.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.828+0000 7fe4e3400640 1 -- 172.21.15.66:0/1979579853 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe4dc050860 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.988+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1979579853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4ec0c97f0 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.988+0000 7fe4e3400640 1 -- 172.21.15.66:0/1979579853 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4dc018020 con 0x7fe4ec0b19a0 2024-03-30T00:42:54.996 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:42:54.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1979579853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4c8066690 msgr2=0x7fe4c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4c8066690 0x7fe4c8068b50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40ae000 tx=0x7fe4d40b5040 comp rx=0 tx=0).stop 2024-03-30T00:42:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1979579853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ec0b19a0 msgr2=0x7fe4ec0ab770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ec0b19a0 0x7fe4ec0ab770 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00c480 tx=0x7fe4dc00c940 comp rx=0 tx=0).stop 2024-03-30T00:42:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1979579853 shutdown_connections 2024-03-30T00:42:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ec160c10 0x7fe4ec15bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4ec0c1800 0x7fe4ec15b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4c8066690 0x7fe4c8068b50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 --2- 172.21.15.66:0/1979579853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ec0b19a0 0x7fe4ec0ab770 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1979579853 >> 172.21.15.66:0/1979579853 conn(0x7fe4ec069f60 msgr2=0x7fe4ec06a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1979579853 shutdown_connections 2024-03-30T00:42:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:54.992+0000 7fe4f1e00640 1 -- 172.21.15.66:0/1979579853 wait complete. 2024-03-30T00:42:55.017 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:42:55.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc844400640 1 Processor -- start 2024-03-30T00:42:55.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc844400640 1 -- start start 2024-03-30T00:42:55.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc844400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc83c0cf2f0 0x7fc83c06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc844400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83c0cb0e0 con 0x7fc83c0cb570 2024-03-30T00:42:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc844400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83c0cf7f0 con 0x7fc83c15ca80 2024-03-30T00:42:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc844400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83c0cfad0 con 0x7fc83c0cf2f0 2024-03-30T00:42:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc841000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc83c0cf2f0 0x7fc83c06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc842400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc83c0cb570 0x7fc83c0cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56230/0 (socket says 172.21.15.66:56230) 2024-03-30T00:42:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc842400640 1 -- 172.21.15.66:0/3027737461 learned_addr learned my addr 172.21.15.66:0/3027737461 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:55.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc841000640 1 -- 172.21.15.66:0/3027737461 >> v1:172.21.15.156:6789/0 conn(0x7fc83c15ca80 legacy=0x7fc83c06b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:55.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc841000640 1 -- 172.21.15.66:0/3027737461 >> v1:172.21.15.66:6789/0 conn(0x7fc83c0cb570 legacy=0x7fc83c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:55.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc841000640 1 -- 172.21.15.66:0/3027737461 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c0cad70 con 0x7fc83c0cf2f0 2024-03-30T00:42:55.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.152+0000 7fc83be00640 1 -- 172.21.15.66:0/3027737461 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4274530706 0 0) 0x7fc83c0cb0e0 con 0x7fc83c0cb570 2024-03-30T00:42:55.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc841000640 1 --2- 172.21.15.66:0/3027737461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83c0cf2f0 0x7fc83c06b7c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc8280097c0 tx=0x7fc82801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4840b339fe94d54 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:55.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc83be00640 1 -- 172.21.15.66:0/3027737461 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8280a8950 con 0x7fc83c0cf2f0 2024-03-30T00:42:55.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc83be00640 1 -- 172.21.15.66:0/3027737461 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8280a8ab0 con 0x7fc83c0cf2f0 2024-03-30T00:42:55.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc83be00640 1 -- 172.21.15.66:0/3027737461 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8280a8e40 con 0x7fc83c0cf2f0 2024-03-30T00:42:55.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc844400640 1 -- 172.21.15.66:0/3027737461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83c0cf2f0 msgr2=0x7fc83c06b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:55.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc844400640 1 --2- 172.21.15.66:0/3027737461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83c0cf2f0 0x7fc83c06b7c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc8280097c0 tx=0x7fc82801ec50 comp rx=0 tx=0).stop 2024-03-30T00:42:55.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc844400640 1 -- 172.21.15.66:0/3027737461 shutdown_connections 2024-03-30T00:42:55.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc844400640 1 --2- 172.21.15.66:0/3027737461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83c0cf2f0 0x7fc83c06b7c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:55.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc844400640 1 -- 172.21.15.66:0/3027737461 >> 172.21.15.66:0/3027737461 conn(0x7fc83c0b2090 msgr2=0x7fc83c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:55.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc844400640 1 -- 172.21.15.66:0/3027737461 shutdown_connections 2024-03-30T00:42:55.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc844400640 1 -- 172.21.15.66:0/3027737461 wait complete. 2024-03-30T00:42:55.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.156+0000 7fc844400640 1 Processor -- start 2024-03-30T00:42:55.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc844400640 1 -- start start 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc844400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83c0cb570 0x7fc83c1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc844400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc83c0cf2f0 0x7fc83c09afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc844400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc83c15ca80 0x7fc83c0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc844400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c0b31c0 con 0x7fc83c0cb570 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc844400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c0b34a0 con 0x7fc83c0cf2f0 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc844400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c0b3780 con 0x7fc83c15ca80 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc841a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83c0cb570 0x7fc83c1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc841a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83c0cb570 0x7fc83c1580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38988/0 (socket says 172.21.15.66:38988) 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc841a00640 1 -- 172.21.15.66:0/2592538728 learned_addr learned my addr 172.21.15.66:0/2592538728 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc841000640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc83c0cf2f0 0x7fc83c09afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc842400640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc83c15ca80 0x7fc83c0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:55.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc841a00640 1 -- 172.21.15.66:0/2592538728 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc83c15ca80 msgr2=0x7fc83c0c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:55.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc841a00640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc83c15ca80 0x7fc83c0c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:55.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc841a00640 1 -- 172.21.15.66:0/2592538728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc83c0cf2f0 msgr2=0x7fc83c09afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:55.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc841a00640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc83c0cf2f0 0x7fc83c09afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:55.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc841a00640 1 -- 172.21.15.66:0/2592538728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc828009420 con 0x7fc83c0cb570 2024-03-30T00:42:55.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc842400640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc83c15ca80 0x7fc83c0c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:55.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc841a00640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83c0cb570 0x7fc83c1580f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc83000b410 tx=0x7fc83000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:55.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc839600640 1 -- 172.21.15.66:0/2592538728 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830004020 con 0x7fc83c0cb570 2024-03-30T00:42:55.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc839600640 1 -- 172.21.15.66:0/2592538728 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc830004180 con 0x7fc83c0cb570 2024-03-30T00:42:55.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc839600640 1 -- 172.21.15.66:0/2592538728 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830011630 con 0x7fc83c0cb570 2024-03-30T00:42:55.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc844400640 1 -- 172.21.15.66:0/2592538728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83c0b3a60 con 0x7fc83c0cb570 2024-03-30T00:42:55.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc844400640 1 -- 172.21.15.66:0/2592538728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83c158bf0 con 0x7fc83c0cb570 2024-03-30T00:42:55.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.160+0000 7fc844400640 1 -- 172.21.15.66:0/2592538728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83c066c30 con 0x7fc83c0cb570 2024-03-30T00:42:55.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.164+0000 7fc839600640 1 -- 172.21.15.66:0/2592538728 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc830011790 con 0x7fc83c0cb570 2024-03-30T00:42:55.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.164+0000 7fc839600640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc81c066690 0x7fc81c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:55.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.164+0000 7fc839600640 1 -- 172.21.15.66:0/2592538728 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc830083fc0 con 0x7fc83c0cb570 2024-03-30T00:42:55.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.164+0000 7fc841000640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc81c066690 0x7fc81c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:55.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.164+0000 7fc841000640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc81c066690 0x7fc81c068b50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc8280b2000 tx=0x7fc828008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:55.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.168+0000 7fc839600640 1 -- 172.21.15.66:0/2592538728 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc8300b5980 con 0x7fc83c0cb570 2024-03-30T00:42:55.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.340+0000 7fc844400640 1 -- 172.21.15.66:0/2592538728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc83c0b2bb0 con 0x7fc83c0cb570 2024-03-30T00:42:55.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.340+0000 7fc839600640 1 -- 172.21.15.66:0/2592538728 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc830015080 con 0x7fc83c0cb570 2024-03-30T00:42:55.347 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:42:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 -- 172.21.15.66:0/2592538728 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc81c066690 msgr2=0x7fc81c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc81c066690 0x7fc81c068b50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc8280b2000 tx=0x7fc828008040 comp rx=0 tx=0).stop 2024-03-30T00:42:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 -- 172.21.15.66:0/2592538728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83c0cb570 msgr2=0x7fc83c1580f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83c0cb570 0x7fc83c1580f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc83000b410 tx=0x7fc83000b8d0 comp rx=0 tx=0).stop 2024-03-30T00:42:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 -- 172.21.15.66:0/2592538728 shutdown_connections 2024-03-30T00:42:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc83c15ca80 0x7fc83c0c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc83c0cf2f0 0x7fc83c09afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc81c066690 0x7fc81c068b50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 --2- 172.21.15.66:0/2592538728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83c0cb570 0x7fc83c1580f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 -- 172.21.15.66:0/2592538728 >> 172.21.15.66:0/2592538728 conn(0x7fc83c0b2090 msgr2=0x7fc83c06e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 -- 172.21.15.66:0/2592538728 shutdown_connections 2024-03-30T00:42:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.344+0000 7fc844400640 1 -- 172.21.15.66:0/2592538728 wait complete. 2024-03-30T00:42:55.367 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:42:55.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:55.368 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:42:55.385 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[40922]: fuse finished with error 0 and tester_r 0 2024-03-30T00:42:55.388 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:42:55.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:55.388 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:42:55.498 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:42:55.498 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:42:55.508 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:42:55.508 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:42:55.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:55.508 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T00:42:55.559 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-30T00:42:55.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:42:55.560 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:42:55.560 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:42:55.569 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:42:55.569 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:42:55.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:55.570 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-30T00:42:55.620 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-30T00:42:55.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:42:55.621 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:42:55.621 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:42:55.630 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:42:55.630 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:42:55.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:42:55.631 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-30T00:42:55.679 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-30T00:42:55.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:42:55.680 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:42:55.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.812+0000 7f20a7200640 1 Processor -- start 2024-03-30T00:42:55.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.812+0000 7f20a7200640 1 -- start start 2024-03-30T00:42:55.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f20a7200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f20a006a4a0 0x7f20a00cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:55.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f20a7200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a00cf160 con 0x7f20a015a960 2024-03-30T00:42:55.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f20a7200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a015aed0 con 0x7f20a015e370 2024-03-30T00:42:55.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f20a7200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a015ad40 con 0x7f20a006a4a0 2024-03-30T00:42:55.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f209fe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f20a015a960 0x7f20a00cb020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56234/0 (socket says 172.21.15.66:56234) 2024-03-30T00:42:55.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f209fe00640 1 -- 172.21.15.66:0/3787076840 learned_addr learned my addr 172.21.15.66:0/3787076840 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:55.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f20a5200640 1 --2- 172.21.15.66:0/3787076840 >> v2:172.21.15.156:3300/0 conn(0x7f20a006a4a0 0x7f20a00cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:55.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f209f400640 1 -- 172.21.15.66:0/3787076840 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690789349 0 0) 0x7f2090001120 con 0x7f20a015a960 2024-03-30T00:42:55.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f209f400640 1 -- 172.21.15.66:0/3787076840 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2084003530 con 0x7f20a015a960 2024-03-30T00:42:55.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f209f400640 1 -- 172.21.15.66:0/3787076840 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146835310 0 0) 0x7f20a015aed0 con 0x7f20a015e370 2024-03-30T00:42:55.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f209f400640 1 -- 172.21.15.66:0/3787076840 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2090001120 con 0x7f20a015e370 2024-03-30T00:42:55.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f209f400640 1 -- 172.21.15.66:0/3787076840 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 629756946 0 0) 0x7f2084003530 con 0x7f20a015a960 2024-03-30T00:42:55.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f20a5200640 1 -- 172.21.15.66:0/3787076840 >> v1:172.21.15.156:6789/0 conn(0x7f20a015e370 legacy=0x7f20a0069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:55.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f20a5200640 1 -- 172.21.15.66:0/3787076840 >> v1:172.21.15.66:6789/0 conn(0x7f20a015a960 legacy=0x7f20a00cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:55.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.816+0000 7f20a5200640 1 -- 172.21.15.66:0/3787076840 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00cb730 con 0x7f20a006a4a0 2024-03-30T00:42:55.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f20a5200640 1 --2- 172.21.15.66:0/3787076840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20a006a4a0 0x7f20a00cec20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2098009dc0 tx=0x7f209801f060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8678005246dc632d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:55.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f209f400640 1 -- 172.21.15.66:0/3787076840 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090001810 con 0x7f20a006a4a0 2024-03-30T00:42:55.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f209f400640 1 -- 172.21.15.66:0/3787076840 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20980a8c30 con 0x7f20a006a4a0 2024-03-30T00:42:55.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f209f400640 1 -- 172.21.15.66:0/3787076840 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980b0440 con 0x7f20a006a4a0 2024-03-30T00:42:55.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f20a7200640 1 -- 172.21.15.66:0/3787076840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20a006a4a0 msgr2=0x7f20a00cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:55.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f20a7200640 1 --2- 172.21.15.66:0/3787076840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20a006a4a0 0x7f20a00cec20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2098009dc0 tx=0x7f209801f060 comp rx=0 tx=0).stop 2024-03-30T00:42:55.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f20a7200640 1 -- 172.21.15.66:0/3787076840 shutdown_connections 2024-03-30T00:42:55.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f20a7200640 1 --2- 172.21.15.66:0/3787076840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20a006a4a0 0x7f20a00cec20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:55.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f20a7200640 1 -- 172.21.15.66:0/3787076840 >> 172.21.15.66:0/3787076840 conn(0x7f20a0098a20 msgr2=0x7f20a00b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:55.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f20a7200640 1 -- 172.21.15.66:0/3787076840 shutdown_connections 2024-03-30T00:42:55.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f20a7200640 1 -- 172.21.15.66:0/3787076840 wait complete. 2024-03-30T00:42:55.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f20a7200640 1 Processor -- start 2024-03-30T00:42:55.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.820+0000 7f20a7200640 1 -- start start 2024-03-30T00:42:55.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f20a7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20a006a4a0 0x7f20a014c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:55.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f20a7200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20a015a960 0x7f20a00b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:55.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f20a7200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20a015e370 0x7f20a00c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:55.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f20a7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00c8680 con 0x7f20a006a4a0 2024-03-30T00:42:55.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f20a7200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00c7a70 con 0x7f20a015a960 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f20a7200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00c7d50 con 0x7f20a015e370 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20a006a4a0 0x7f20a014c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20a006a4a0 0x7f20a014c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39000/0 (socket says 172.21.15.66:39000) 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209fe00640 1 -- 172.21.15.66:0/3288803499 learned_addr learned my addr 172.21.15.66:0/3288803499 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f2097400640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20a015a960 0x7f20a00b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f20a5200640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20a015e370 0x7f20a00c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209fe00640 1 -- 172.21.15.66:0/3288803499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20a015e370 msgr2=0x7f20a00c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209fe00640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20a015e370 0x7f20a00c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209fe00640 1 -- 172.21.15.66:0/3288803499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20a015a960 msgr2=0x7f20a00b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209fe00640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20a015a960 0x7f20a00b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209fe00640 1 -- 172.21.15.66:0/3288803499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2098009a70 con 0x7f20a006a4a0 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f2097400640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20a015a960 0x7f20a00b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209fe00640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20a006a4a0 0x7f20a014c750 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f209000cac0 tx=0x7f209000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:55.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209cc00640 1 -- 172.21.15.66:0/3288803499 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090005cf0 con 0x7f20a006a4a0 2024-03-30T00:42:55.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209cc00640 1 -- 172.21.15.66:0/3288803499 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2090005e70 con 0x7f20a006a4a0 2024-03-30T00:42:55.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209cc00640 1 -- 172.21.15.66:0/3288803499 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090007290 con 0x7f20a006a4a0 2024-03-30T00:42:55.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f20a7200640 1 -- 172.21.15.66:0/3288803499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a00c8030 con 0x7f20a006a4a0 2024-03-30T00:42:55.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f20a7200640 1 -- 172.21.15.66:0/3288803499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a014cc90 con 0x7f20a006a4a0 2024-03-30T00:42:55.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f20a7200640 1 -- 172.21.15.66:0/3288803499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a0066c30 con 0x7f20a006a4a0 2024-03-30T00:42:55.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.824+0000 7f209cc00640 1 -- 172.21.15.66:0/3288803499 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f20900073f0 con 0x7f20a006a4a0 2024-03-30T00:42:55.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.828+0000 7f209cc00640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f207c066690 0x7f207c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:55.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.828+0000 7f209cc00640 1 -- 172.21.15.66:0/3288803499 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f2090083a90 con 0x7f20a006a4a0 2024-03-30T00:42:55.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.828+0000 7f2097400640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f207c066690 0x7f207c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:55.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.828+0000 7f2097400640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f207c066690 0x7f207c068b50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f20a015a0c0 tx=0x7f2088009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:55.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:55.832+0000 7f209cc00640 1 -- 172.21.15.66:0/3288803499 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2090050d50 con 0x7f20a006a4a0 2024-03-30T00:42:56.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f20a7200640 1 -- 172.21.15.66:0/3288803499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20a00cf160 con 0x7f20a006a4a0 2024-03-30T00:42:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f209cc00640 1 -- 172.21.15.66:0/3288803499 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f20a00cf160 con 0x7f20a006a4a0 2024-03-30T00:42:56.035 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:56.035 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/465916334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":465916334},{"type":"v1","addr":"172.21.15.156:6835","nonce":465916334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1640047258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1640047258},{"type":"v1","addr":"172.21.15.156:6834","nonce":1640047258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5788,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2112021115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2112021115},{"type":"v1","addr":"172.21.15.66:6835","nonce":2112021115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:42:08.269705+0000","modified":"2024-03-30T00:42:11.492352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5985},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5985":{"gid":5985,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/3557086525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3557086525},{"type":"v1","addr":"172.21.15.66:6837","nonce":3557086525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T00:42:56.035 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-03-30T00:42:56.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f20a7200640 1 -- 172.21.15.66:0/3288803499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f207c066690 msgr2=0x7f207c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f20a7200640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f207c066690 0x7f207c068b50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f20a015a0c0 tx=0x7f2088009040 comp rx=0 tx=0).stop 2024-03-30T00:42:56.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f20a7200640 1 -- 172.21.15.66:0/3288803499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20a006a4a0 msgr2=0x7f20a014c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f20a7200640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20a006a4a0 0x7f20a014c750 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f209000cac0 tx=0x7f209000cf80 comp rx=0 tx=0).stop 2024-03-30T00:42:56.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f20a7200640 1 -- 172.21.15.66:0/3288803499 shutdown_connections 2024-03-30T00:42:56.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f20a7200640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20a015e370 0x7f20a00c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f20a7200640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20a015a960 0x7f20a00b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f20a7200640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f207c066690 0x7f207c068b50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f20a7200640 1 --2- 172.21.15.66:0/3288803499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20a006a4a0 0x7f20a014c750 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.028+0000 7f20a7200640 1 -- 172.21.15.66:0/3288803499 >> 172.21.15.66:0/3288803499 conn(0x7f20a0098a20 msgr2=0x7f20a015da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:56.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.032+0000 7f20a7200640 1 -- 172.21.15.66:0/3288803499 shutdown_connections 2024-03-30T00:42:56.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.032+0000 7f20a7200640 1 -- 172.21.15.66:0/3288803499 wait complete. 2024-03-30T00:42:56.055 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:42:56.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.188+0000 7f54b6200640 1 Processor -- start 2024-03-30T00:42:56.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.188+0000 7f54b6200640 1 -- start start 2024-03-30T00:42:56.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54b6200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f54b00cd2d0 0x7f54b00cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54b6200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b00cc8d0 0x7f54b006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54b6200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b00cb620 con 0x7f54b00cad00 2024-03-30T00:42:56.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54b6200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b00cbec0 con 0x7f54b00cd2d0 2024-03-30T00:42:56.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54b6200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00ca660 con 0x7f54b00cc8d0 2024-03-30T00:42:56.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54af400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f54b00cad00 0x7f54b00c8940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56240/0 (socket says 172.21.15.66:56240) 2024-03-30T00:42:56.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54af400640 1 -- 172.21.15.66:0/2519972228 learned_addr learned my addr 172.21.15.66:0/2519972228 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:56.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54a7e00640 1 --2- 172.21.15.66:0/2519972228 >> v2:172.21.15.66:3300/0 conn(0x7f54b00cd2d0 0x7f54b00cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54aea00640 1 -- 172.21.15.66:0/2519972228 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003695759 0 0) 0x7f54b00cb620 con 0x7f54b00cad00 2024-03-30T00:42:56.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54afe00640 1 --2- 172.21.15.66:0/2519972228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b00cc8d0 0x7f54b006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54a7e00640 1 -- 172.21.15.66:0/2519972228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b00cc8d0 msgr2=0x7f54b006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:56.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54a7e00640 1 --2- 172.21.15.66:0/2519972228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b00cc8d0 0x7f54b006c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54a7e00640 1 -- 172.21.15.66:0/2519972228 >> v1:172.21.15.66:6789/0 conn(0x7f54b00cad00 legacy=0x7f54b00c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54a7e00640 1 -- 172.21.15.66:0/2519972228 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b00ca940 con 0x7f54b00cd2d0 2024-03-30T00:42:56.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54afe00640 1 --2- 172.21.15.66:0/2519972228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b00cc8d0 0x7f54b006c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:56.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54a7e00640 1 --2- 172.21.15.66:0/2519972228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b00cd2d0 0x7f54b00cb0e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f54980097c0 tx=0x7f549801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c93fc32bcd5c12e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:56.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54aea00640 1 -- 172.21.15.66:0/2519972228 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54980ab670 con 0x7f54b00cd2d0 2024-03-30T00:42:56.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.192+0000 7f54aea00640 1 -- 172.21.15.66:0/2519972228 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54980ab7d0 con 0x7f54b00cd2d0 2024-03-30T00:42:56.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54aea00640 1 -- 172.21.15.66:0/2519972228 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54980abb60 con 0x7f54b00cd2d0 2024-03-30T00:42:56.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 -- 172.21.15.66:0/2519972228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b00cd2d0 msgr2=0x7f54b00cb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 --2- 172.21.15.66:0/2519972228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b00cd2d0 0x7f54b00cb0e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f54980097c0 tx=0x7f549801c470 comp rx=0 tx=0).stop 2024-03-30T00:42:56.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 -- 172.21.15.66:0/2519972228 shutdown_connections 2024-03-30T00:42:56.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 --2- 172.21.15.66:0/2519972228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b00cc8d0 0x7f54b006c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 --2- 172.21.15.66:0/2519972228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b00cd2d0 0x7f54b00cb0e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 -- 172.21.15.66:0/2519972228 >> 172.21.15.66:0/2519972228 conn(0x7f54b0069f60 msgr2=0x7f54b006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:56.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 -- 172.21.15.66:0/2519972228 shutdown_connections 2024-03-30T00:42:56.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 -- 172.21.15.66:0/2519972228 wait complete. 2024-03-30T00:42:56.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 Processor -- start 2024-03-30T00:42:56.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 -- start start 2024-03-30T00:42:56.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b00cad00 0x7f54b0156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b00cc8d0 0x7f54b0161940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b00cd2d0 0x7f54b0154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b01582a0 con 0x7f54b00cc8d0 2024-03-30T00:42:56.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0157200 con 0x7f54b00cad00 2024-03-30T00:42:56.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54b6200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b01574e0 con 0x7f54b00cd2d0 2024-03-30T00:42:56.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b00cc8d0 0x7f54b0161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b00cc8d0 0x7f54b0161940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39010/0 (socket says 172.21.15.66:39010) 2024-03-30T00:42:56.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54a7e00640 1 -- 172.21.15.66:0/3660219483 learned_addr learned my addr 172.21.15.66:0/3660219483 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:56.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54af400640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b00cad00 0x7f54b0156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.196+0000 7f54afe00640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b00cd2d0 0x7f54b0154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7e00640 1 -- 172.21.15.66:0/3660219483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b00cd2d0 msgr2=0x7f54b0154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7e00640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b00cd2d0 0x7f54b0154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7e00640 1 -- 172.21.15.66:0/3660219483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b00cad00 msgr2=0x7f54b0156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7e00640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b00cad00 0x7f54b0156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7e00640 1 -- 172.21.15.66:0/3660219483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5498009420 con 0x7f54b00cc8d0 2024-03-30T00:42:56.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7e00640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b00cc8d0 0x7f54b0161940 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f54980097c0 tx=0x7f549801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:56.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7400640 1 -- 172.21.15.66:0/3660219483 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5498004230 con 0x7f54b00cc8d0 2024-03-30T00:42:56.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7400640 1 -- 172.21.15.66:0/3660219483 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54980b9070 con 0x7f54b00cc8d0 2024-03-30T00:42:56.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54b6200640 1 -- 172.21.15.66:0/3660219483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b01577c0 con 0x7f54b00cc8d0 2024-03-30T00:42:56.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7400640 1 -- 172.21.15.66:0/3660219483 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5498003620 con 0x7f54b00cc8d0 2024-03-30T00:42:56.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54b6200640 1 -- 172.21.15.66:0/3660219483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b0157d80 con 0x7f54b00cc8d0 2024-03-30T00:42:56.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54b6200640 1 -- 172.21.15.66:0/3660219483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b0157f70 con 0x7f54b00cc8d0 2024-03-30T00:42:56.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7400640 1 -- 172.21.15.66:0/3660219483 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5498003990 con 0x7f54b00cc8d0 2024-03-30T00:42:56.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7400640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c066690 0x7f548c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.200+0000 7f54a7400640 1 -- 172.21.15.66:0/3660219483 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f54980bb070 con 0x7f54b00cc8d0 2024-03-30T00:42:56.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.204+0000 7f54af400640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c066690 0x7f548c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.204+0000 7f54af400640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c066690 0x7f548c068b50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f54a0007fd0 tx=0x7f54a0007d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:56.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.204+0000 7f54a7400640 1 -- 172.21.15.66:0/3660219483 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f54980ec9f0 con 0x7f54b00cc8d0 2024-03-30T00:42:56.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.372+0000 7f54b6200640 1 -- 172.21.15.66:0/3660219483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54b0066c30 con 0x7f54b00cc8d0 2024-03-30T00:42:56.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.372+0000 7f54a7400640 1 -- 172.21.15.66:0/3660219483 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16402 (secure 0 0 0) 0x7f54980f0010 con 0x7f54b00cc8d0 2024-03-30T00:42:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:56.379 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":56,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:42:33.561663+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:42:06.807254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-30T00:42:07.258973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:42:56.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 -- 172.21.15.66:0/3660219483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c066690 msgr2=0x7f548c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c066690 0x7f548c068b50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f54a0007fd0 tx=0x7f54a0007d60 comp rx=0 tx=0).stop 2024-03-30T00:42:56.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 -- 172.21.15.66:0/3660219483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b00cc8d0 msgr2=0x7f54b0161940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b00cc8d0 0x7f54b0161940 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f54980097c0 tx=0x7f549801acb0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 -- 172.21.15.66:0/3660219483 shutdown_connections 2024-03-30T00:42:56.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54b00cd2d0 0x7f54b0154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f548c066690 0x7f548c068b50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54b00cc8d0 0x7f54b0161940 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 --2- 172.21.15.66:0/3660219483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54b00cad00 0x7f54b0156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 -- 172.21.15.66:0/3660219483 >> 172.21.15.66:0/3660219483 conn(0x7f54b0069f60 msgr2=0x7f54b00c8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:56.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 -- 172.21.15.66:0/3660219483 shutdown_connections 2024-03-30T00:42:56.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.376+0000 7f54b6200640 1 -- 172.21.15.66:0/3660219483 wait complete. 2024-03-30T00:42:56.399 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:42:06.807254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:42:07.258973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:42:56.400 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:42:56.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f847dc00640 1 Processor -- start 2024-03-30T00:42:56.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f847dc00640 1 -- start start 2024-03-30T00:42:56.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f847dc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f84780af420 0x7f84780cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f847dc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84780cbb90 con 0x7f84780cee50 2024-03-30T00:42:56.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f847dc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84780ce1b0 con 0x7f847815de30 2024-03-30T00:42:56.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f847dc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84780afa10 con 0x7f84780af420 2024-03-30T00:42:56.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f846e000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f84780cee50 0x7f847806e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56254/0 (socket says 172.21.15.66:56254) 2024-03-30T00:42:56.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f846e000640 1 -- 172.21.15.66:0/3349922696 learned_addr learned my addr 172.21.15.66:0/3349922696 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:56.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f8477400640 1 --2- 172.21.15.66:0/3349922696 >> v2:172.21.15.156:3300/0 conn(0x7f84780af420 0x7f84780cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f8476000640 1 -- 172.21.15.66:0/3349922696 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804643659 0 0) 0x7f84780cbb90 con 0x7f84780cee50 2024-03-30T00:42:56.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f8477400640 1 -- 172.21.15.66:0/3349922696 >> v1:172.21.15.156:6789/0 conn(0x7f847815de30 legacy=0x7f84780ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f8477400640 1 -- 172.21.15.66:0/3349922696 >> v1:172.21.15.66:6789/0 conn(0x7f84780cee50 legacy=0x7f847806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.536+0000 7f8477400640 1 -- 172.21.15.66:0/3349922696 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84780cf430 con 0x7f84780af420 2024-03-30T00:42:56.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.540+0000 7f8477400640 1 --2- 172.21.15.66:0/3349922696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84780af420 0x7f84780cc040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f84700097c0 tx=0x7f847001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4567f27c415b445 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:56.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.540+0000 7f8476000640 1 -- 172.21.15.66:0/3349922696 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700ac410 con 0x7f84780af420 2024-03-30T00:42:56.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.540+0000 7f8476000640 1 -- 172.21.15.66:0/3349922696 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84700ac570 con 0x7f84780af420 2024-03-30T00:42:56.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.540+0000 7f8476000640 1 -- 172.21.15.66:0/3349922696 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700ac900 con 0x7f84780af420 2024-03-30T00:42:56.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.540+0000 7f847dc00640 1 -- 172.21.15.66:0/3349922696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84780af420 msgr2=0x7f84780cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.540+0000 7f847dc00640 1 --2- 172.21.15.66:0/3349922696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84780af420 0x7f84780cc040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f84700097c0 tx=0x7f847001c4a0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.540+0000 7f847dc00640 1 -- 172.21.15.66:0/3349922696 shutdown_connections 2024-03-30T00:42:56.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.540+0000 7f847dc00640 1 --2- 172.21.15.66:0/3349922696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84780af420 0x7f84780cc040 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.540+0000 7f847dc00640 1 -- 172.21.15.66:0/3349922696 >> 172.21.15.66:0/3349922696 conn(0x7f84780d0540 msgr2=0x7f84780d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:56.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.540+0000 7f847dc00640 1 -- 172.21.15.66:0/3349922696 shutdown_connections 2024-03-30T00:42:56.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.540+0000 7f847dc00640 1 -- 172.21.15.66:0/3349922696 wait complete. 2024-03-30T00:42:56.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f847dc00640 1 Processor -- start 2024-03-30T00:42:56.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f847dc00640 1 -- start start 2024-03-30T00:42:56.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f847dc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84780af420 0x7f8478155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f847dc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84780cee50 0x7f84781577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f847dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815de30 0x7f8478157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f847dc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8478158650 con 0x7f847815de30 2024-03-30T00:42:56.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f847dc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8478158930 con 0x7f84780cee50 2024-03-30T00:42:56.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f847dc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8478158c10 con 0x7f84780af420 2024-03-30T00:42:56.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f8477400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815de30 0x7f8478157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f8477400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815de30 0x7f8478157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39018/0 (socket says 172.21.15.66:39018) 2024-03-30T00:42:56.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f8477400640 1 -- 172.21.15.66:0/1611250735 learned_addr learned my addr 172.21.15.66:0/1611250735 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:56.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f846e000640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84780cee50 0x7f84781577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f8476a00640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84780af420 0x7f8478155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f8477400640 1 -- 172.21.15.66:0/1611250735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84780af420 msgr2=0x7f8478155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f8477400640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84780af420 0x7f8478155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f8477400640 1 -- 172.21.15.66:0/1611250735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84780cee50 msgr2=0x7f84781577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f8477400640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84780cee50 0x7f84781577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f8477400640 1 -- 172.21.15.66:0/1611250735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470009420 con 0x7f847815de30 2024-03-30T00:42:56.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f846e000640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84780cee50 0x7f84781577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:56.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f8477400640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815de30 0x7f8478157cf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f847001a0d0 tx=0x7f847001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:56.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f846f400640 1 -- 172.21.15.66:0/1611250735 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700ac410 con 0x7f847815de30 2024-03-30T00:42:56.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f847dc00640 1 -- 172.21.15.66:0/1611250735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84780b4250 con 0x7f847815de30 2024-03-30T00:42:56.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f846f400640 1 -- 172.21.15.66:0/1611250735 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8470006e20 con 0x7f847815de30 2024-03-30T00:42:56.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f846f400640 1 -- 172.21.15.66:0/1611250735 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470004230 con 0x7f847815de30 2024-03-30T00:42:56.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.544+0000 7f847dc00640 1 -- 172.21.15.66:0/1611250735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8478158230 con 0x7f847815de30 2024-03-30T00:42:56.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.548+0000 7f847dc00640 1 -- 172.21.15.66:0/1611250735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8478158420 con 0x7f847815de30 2024-03-30T00:42:56.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.548+0000 7f846f400640 1 -- 172.21.15.66:0/1611250735 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8470004390 con 0x7f847815de30 2024-03-30T00:42:56.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.548+0000 7f846f400640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f84540666c0 0x7f8454068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.548+0000 7f846f400640 1 -- 172.21.15.66:0/1611250735 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f847011db60 con 0x7f847815de30 2024-03-30T00:42:56.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.548+0000 7f8476a00640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f84540666c0 0x7f8454068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.548+0000 7f8476a00640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f84540666c0 0x7f8454068b80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f847806b150 tx=0x7f8468009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:56.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.552+0000 7f846f400640 1 -- 172.21.15.66:0/1611250735 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f84700eace0 con 0x7f847815de30 2024-03-30T00:42:56.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 -- 172.21.15.66:0/1611250735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84780cdf60 con 0x7f847815de30 2024-03-30T00:42:56.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f846f400640 1 -- 172.21.15.66:0/1611250735 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84700ee300 con 0x7f847815de30 2024-03-30T00:42:56.721 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:42:56.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 -- 172.21.15.66:0/1611250735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f84540666c0 msgr2=0x7f8454068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f84540666c0 0x7f8454068b80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f847806b150 tx=0x7f8468009040 comp rx=0 tx=0).stop 2024-03-30T00:42:56.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 -- 172.21.15.66:0/1611250735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815de30 msgr2=0x7f8478157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815de30 0x7f8478157cf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f847001a0d0 tx=0x7f847001ace0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 -- 172.21.15.66:0/1611250735 shutdown_connections 2024-03-30T00:42:56.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f84540666c0 0x7f8454068b80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f847815de30 0x7f8478157cf0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84780cee50 0x7f84781577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 --2- 172.21.15.66:0/1611250735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84780af420 0x7f8478155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 -- 172.21.15.66:0/1611250735 >> 172.21.15.66:0/1611250735 conn(0x7f84780d0540 msgr2=0x7f84780d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:56.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 -- 172.21.15.66:0/1611250735 shutdown_connections 2024-03-30T00:42:56.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.716+0000 7f847dc00640 1 -- 172.21.15.66:0/1611250735 wait complete. 2024-03-30T00:42:56.742 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:42:56.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa3800640 1 Processor -- start 2024-03-30T00:42:56.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa3800640 1 -- start start 2024-03-30T00:42:56.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa3800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5f9c160bb0 0x7f5f9c1677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa3800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9c0b1d00 con 0x7f5f9c15c780 2024-03-30T00:42:56.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa3800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9c0cd820 con 0x7f5f9c15cb60 2024-03-30T00:42:56.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa3800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c0cdb00 con 0x7f5f9c160bb0 2024-03-30T00:42:56.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa0e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5f9c15c780 0x7f5f9c0cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56264/0 (socket says 172.21.15.66:56264) 2024-03-30T00:42:56.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa0e00640 1 -- 172.21.15.66:0/3944686872 learned_addr learned my addr 172.21.15.66:0/3944686872 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:56.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa1800640 1 --2- 172.21.15.66:0/3944686872 >> v2:172.21.15.156:3300/0 conn(0x7f5f9c160bb0 0x7f5f9c1677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5f9be00640 1 -- 172.21.15.66:0/3944686872 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031400576 0 0) 0x7f5f9c0b1d00 con 0x7f5f9c15c780 2024-03-30T00:42:56.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5f9be00640 1 -- 172.21.15.66:0/3944686872 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f80003530 con 0x7f5f9c15c780 2024-03-30T00:42:56.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa1800640 1 -- 172.21.15.66:0/3944686872 >> v1:172.21.15.156:6789/0 conn(0x7f5f9c15cb60 legacy=0x7f5f9c1604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa1800640 1 -- 172.21.15.66:0/3944686872 >> v1:172.21.15.66:6789/0 conn(0x7f5f9c15c780 legacy=0x7f5f9c0cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.880+0000 7f5fa1800640 1 -- 172.21.15.66:0/3944686872 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0cdde0 con 0x7f5f9c160bb0 2024-03-30T00:42:56.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.884+0000 7f5fa1800640 1 --2- 172.21.15.66:0/3944686872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c160bb0 0x7f5f9c1677d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009d90 tx=0x7f5f9401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7857465572765df server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:56.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.884+0000 7f5f9be00640 1 -- 172.21.15.66:0/3944686872 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f940a5a00 con 0x7f5f9c160bb0 2024-03-30T00:42:56.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.884+0000 7f5f9be00640 1 -- 172.21.15.66:0/3944686872 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f940a5b60 con 0x7f5f9c160bb0 2024-03-30T00:42:56.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.884+0000 7f5f9be00640 1 -- 172.21.15.66:0/3944686872 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f940ad440 con 0x7f5f9c160bb0 2024-03-30T00:42:56.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.884+0000 7f5fa3800640 1 -- 172.21.15.66:0/3944686872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c160bb0 msgr2=0x7f5f9c1677d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.884+0000 7f5fa3800640 1 --2- 172.21.15.66:0/3944686872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c160bb0 0x7f5f9c1677d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009d90 tx=0x7f5f9401c940 comp rx=0 tx=0).stop 2024-03-30T00:42:56.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.884+0000 7f5fa3800640 1 -- 172.21.15.66:0/3944686872 shutdown_connections 2024-03-30T00:42:56.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.884+0000 7f5fa3800640 1 --2- 172.21.15.66:0/3944686872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c160bb0 0x7f5f9c1677d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.884+0000 7f5fa3800640 1 -- 172.21.15.66:0/3944686872 >> 172.21.15.66:0/3944686872 conn(0x7f5f9c09ba30 msgr2=0x7f5f9c089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:56.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.884+0000 7f5fa3800640 1 -- 172.21.15.66:0/3944686872 shutdown_connections 2024-03-30T00:42:56.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.884+0000 7f5fa3800640 1 -- 172.21.15.66:0/3944686872 wait complete. 2024-03-30T00:42:56.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa3800640 1 Processor -- start 2024-03-30T00:42:56.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa3800640 1 -- start start 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa3800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c15c780 0x7f5f9c14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa3800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c15cb60 0x7f5f9c1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa3800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c160bb0 0x7f5f9c154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa3800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c155310 con 0x7f5f9c160bb0 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa3800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c157c30 con 0x7f5f9c15cb60 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa3800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c157f10 con 0x7f5f9c15c780 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c160bb0 0x7f5f9c154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c160bb0 0x7f5f9c154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39034/0 (socket says 172.21.15.66:39034) 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa0e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c15c780 0x7f5f9c14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa1800640 1 -- 172.21.15.66:0/1163931774 learned_addr learned my addr 172.21.15.66:0/1163931774 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5f93e00640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c15cb60 0x7f5f9c1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa1800640 1 -- 172.21.15.66:0/1163931774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c15c780 msgr2=0x7f5f9c14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa1800640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c15c780 0x7f5f9c14fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa1800640 1 -- 172.21.15.66:0/1163931774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c15cb60 msgr2=0x7f5f9c1589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:56.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa1800640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c15cb60 0x7f5f9c1589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:56.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa1800640 1 -- 172.21.15.66:0/1163931774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f94009a70 con 0x7f5f9c160bb0 2024-03-30T00:42:56.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa1800640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c160bb0 0x7f5f9c154930 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009d90 tx=0x7f5f940196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:56.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5f99600640 1 -- 172.21.15.66:0/1163931774 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94004240 con 0x7f5f9c160bb0 2024-03-30T00:42:56.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5f99600640 1 -- 172.21.15.66:0/1163931774 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f940043a0 con 0x7f5f9c160bb0 2024-03-30T00:42:56.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa3800640 1 -- 172.21.15.66:0/1163931774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c1581f0 con 0x7f5f9c160bb0 2024-03-30T00:42:56.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5f99600640 1 -- 172.21.15.66:0/1163931774 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94004730 con 0x7f5f9c160bb0 2024-03-30T00:42:56.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa3800640 1 -- 172.21.15.66:0/1163931774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c1500d0 con 0x7f5f9c160bb0 2024-03-30T00:42:56.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.888+0000 7f5fa3800640 1 -- 172.21.15.66:0/1163931774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c158710 con 0x7f5f9c160bb0 2024-03-30T00:42:56.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.892+0000 7f5f99600640 1 -- 172.21.15.66:0/1163931774 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5f940b4030 con 0x7f5f9c160bb0 2024-03-30T00:42:56.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.892+0000 7f5f99600640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 0x7f5f78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:56.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.892+0000 7f5f99600640 1 -- 172.21.15.66:0/1163931774 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f5f9411f470 con 0x7f5f9c160bb0 2024-03-30T00:42:56.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.892+0000 7f5fa0e00640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 0x7f5f78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:56.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.892+0000 7f5fa0e00640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 0x7f5f78068b50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0011f0 tx=0x7f5f8c007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:56.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:56.896+0000 7f5f99600640 1 -- 172.21.15.66:0/1163931774 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5f9c158710 con 0x7f5f9c160bb0 2024-03-30T00:42:57.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 -- 172.21.15.66:0/1163931774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f9c158710 con 0x7f5f9c160bb0 2024-03-30T00:42:57.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5f99600640 1 -- 172.21.15.66:0/1163931774 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f9c158710 con 0x7f5f9c160bb0 2024-03-30T00:42:57.062 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:42:57.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 -- 172.21.15.66:0/1163931774 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 msgr2=0x7f5f78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 0x7f5f78068b50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0011f0 tx=0x7f5f8c007d40 comp rx=0 tx=0).stop 2024-03-30T00:42:57.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 -- 172.21.15.66:0/1163931774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c160bb0 msgr2=0x7f5f9c154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c160bb0 0x7f5f9c154930 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009d90 tx=0x7f5f940196e0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 -- 172.21.15.66:0/1163931774 shutdown_connections 2024-03-30T00:42:57.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 0x7f5f78068b50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c160bb0 0x7f5f9c154930 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c15cb60 0x7f5f9c1589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 --2- 172.21.15.66:0/1163931774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c15c780 0x7f5f9c14fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 -- 172.21.15.66:0/1163931774 >> 172.21.15.66:0/1163931774 conn(0x7f5f9c09ba30 msgr2=0x7f5f9c0b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:57.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 -- 172.21.15.66:0/1163931774 shutdown_connections 2024-03-30T00:42:57.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.056+0000 7f5fa3800640 1 -- 172.21.15.66:0/1163931774 wait complete. 2024-03-30T00:42:57.082 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:42:57.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb39e000640 1 Processor -- start 2024-03-30T00:42:57.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb39e000640 1 -- start start 2024-03-30T00:42:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb39e000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3980cda90 0x7fb3980ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb39e000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb39815a370 0x7fb3980cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb39e000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39815a990 con 0x7fb3980cde70 2024-03-30T00:42:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb39e000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3980b1b80 con 0x7fb39815a370 2024-03-30T00:42:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb39e000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3980cd3e0 con 0x7fb3980cda90 2024-03-30T00:42:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb397e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb39815a370 0x7fb3980cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb397400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3980cda90 0x7fb3980ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb38fe00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb3980cde70 0x7fb398159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:43606/0 (socket says 172.21.15.66:43606) 2024-03-30T00:42:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb38fe00640 1 -- 172.21.15.66:0/137880717 learned_addr learned my addr 172.21.15.66:0/137880717 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb397e00640 1 -- 172.21.15.66:0/137880717 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3980cda90 msgr2=0x7fb3980ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:57.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb397e00640 1 --2- 172.21.15.66:0/137880717 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3980cda90 0x7fb3980ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb397e00640 1 -- 172.21.15.66:0/137880717 >> v1:172.21.15.156:6789/0 conn(0x7fb3980cde70 legacy=0x7fb398159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb397e00640 1 -- 172.21.15.66:0/137880717 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3980b11e0 con 0x7fb39815a370 2024-03-30T00:42:57.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.220+0000 7fb396a00640 1 -- 172.21.15.66:0/137880717 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914074858 0 0) 0x7fb39815a990 con 0x7fb3980cde70 2024-03-30T00:42:57.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb397e00640 1 --2- 172.21.15.66:0/137880717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39815a370 0x7fb3980cea60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb390009d90 tx=0x7fb39001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bbd153c26883239 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:57.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb396a00640 1 -- 172.21.15.66:0/137880717 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3900a58b0 con 0x7fb39815a370 2024-03-30T00:42:57.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb396a00640 1 -- 172.21.15.66:0/137880717 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3900a5a10 con 0x7fb39815a370 2024-03-30T00:42:57.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb396a00640 1 -- 172.21.15.66:0/137880717 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3900a5da0 con 0x7fb39815a370 2024-03-30T00:42:57.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb39e000640 1 -- 172.21.15.66:0/137880717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39815a370 msgr2=0x7fb3980cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb39e000640 1 --2- 172.21.15.66:0/137880717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39815a370 0x7fb3980cea60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb390009d90 tx=0x7fb39001c8e0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb39e000640 1 -- 172.21.15.66:0/137880717 shutdown_connections 2024-03-30T00:42:57.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb39e000640 1 --2- 172.21.15.66:0/137880717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb39815a370 0x7fb3980cea60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb39e000640 1 --2- 172.21.15.66:0/137880717 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3980cda90 0x7fb3980ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb39e000640 1 -- 172.21.15.66:0/137880717 >> 172.21.15.66:0/137880717 conn(0x7fb398069f60 msgr2=0x7fb39806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:57.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb39e000640 1 -- 172.21.15.66:0/137880717 shutdown_connections 2024-03-30T00:42:57.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.224+0000 7fb39e000640 1 -- 172.21.15.66:0/137880717 wait complete. 2024-03-30T00:42:57.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb39e000640 1 Processor -- start 2024-03-30T00:42:57.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb39e000640 1 -- start start 2024-03-30T00:42:57.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb39e000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3980cda90 0x7fb3980c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb39e000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3980cde70 0x7fb398151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb39e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb39815a370 0x7fb398156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb39e000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb398153a40 con 0x7fb39815a370 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb39e000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3980c0540 con 0x7fb3980cde70 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb39e000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3980c0820 con 0x7fb3980cda90 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb397e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb39815a370 0x7fb398156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb397e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb39815a370 0x7fb398156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39048/0 (socket says 172.21.15.66:39048) 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb397e00640 1 -- 172.21.15.66:0/79916405 learned_addr learned my addr 172.21.15.66:0/79916405 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb397400640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3980cda90 0x7fb3980c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb38fe00640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3980cde70 0x7fb398151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb397e00640 1 -- 172.21.15.66:0/79916405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3980cda90 msgr2=0x7fb3980c15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb397e00640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3980cda90 0x7fb3980c15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb397e00640 1 -- 172.21.15.66:0/79916405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3980cde70 msgr2=0x7fb398151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb397e00640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3980cde70 0x7fb398151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb397e00640 1 -- 172.21.15.66:0/79916405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb390009a70 con 0x7fb39815a370 2024-03-30T00:42:57.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb397400640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3980cda90 0x7fb3980c15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:57.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb38fe00640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3980cde70 0x7fb398151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:57.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb397e00640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb39815a370 0x7fb398156140 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb3900b0040 tx=0x7fb390019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:57.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb38f400640 1 -- 172.21.15.66:0/79916405 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3900b9860 con 0x7fb39815a370 2024-03-30T00:42:57.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb38f400640 1 -- 172.21.15.66:0/79916405 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3900b99c0 con 0x7fb39815a370 2024-03-30T00:42:57.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb38f400640 1 -- 172.21.15.66:0/79916405 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3900b9d70 con 0x7fb39815a370 2024-03-30T00:42:57.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb39e000640 1 -- 172.21.15.66:0/79916405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3980c0b00 con 0x7fb39815a370 2024-03-30T00:42:57.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb39e000640 1 -- 172.21.15.66:0/79916405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3980c10c0 con 0x7fb39815a370 2024-03-30T00:42:57.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.228+0000 7fb39e000640 1 -- 172.21.15.66:0/79916405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3980c12b0 con 0x7fb39815a370 2024-03-30T00:42:57.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.232+0000 7fb38f400640 1 -- 172.21.15.66:0/79916405 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb3900ad440 con 0x7fb39815a370 2024-03-30T00:42:57.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.232+0000 7fb38f400640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb374066690 0x7fb374068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.232+0000 7fb38f400640 1 -- 172.21.15.66:0/79916405 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb39011f620 con 0x7fb39815a370 2024-03-30T00:42:57.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.232+0000 7fb397400640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb374066690 0x7fb374068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.232+0000 7fb397400640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb374066690 0x7fb374068b50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb388009a40 tx=0x7fb388008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:57.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.236+0000 7fb38f400640 1 -- 172.21.15.66:0/79916405 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb3900ec770 con 0x7fb39815a370 2024-03-30T00:42:57.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.408+0000 7fb39e000640 1 -- 172.21.15.66:0/79916405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb398066c30 con 0x7fb39815a370 2024-03-30T00:42:57.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.408+0000 7fb38f400640 1 -- 172.21.15.66:0/79916405 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3900efd90 con 0x7fb39815a370 2024-03-30T00:42:57.416 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:42:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 -- 172.21.15.66:0/79916405 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb374066690 msgr2=0x7fb374068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb374066690 0x7fb374068b50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb388009a40 tx=0x7fb388008040 comp rx=0 tx=0).stop 2024-03-30T00:42:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 -- 172.21.15.66:0/79916405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb39815a370 msgr2=0x7fb398156140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb39815a370 0x7fb398156140 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb3900b0040 tx=0x7fb390019680 comp rx=0 tx=0).stop 2024-03-30T00:42:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 -- 172.21.15.66:0/79916405 shutdown_connections 2024-03-30T00:42:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb374066690 0x7fb374068b50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb39815a370 0x7fb398156140 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3980cde70 0x7fb398151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 --2- 172.21.15.66:0/79916405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3980cda90 0x7fb3980c15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 -- 172.21.15.66:0/79916405 >> 172.21.15.66:0/79916405 conn(0x7fb398069f60 msgr2=0x7fb39806a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 -- 172.21.15.66:0/79916405 shutdown_connections 2024-03-30T00:42:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.412+0000 7fb39e000640 1 -- 172.21.15.66:0/79916405 wait complete. 2024-03-30T00:42:57.437 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:42:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca3f000640 1 Processor -- start 2024-03-30T00:42:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca3f000640 1 -- start start 2024-03-30T00:42:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca3f000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fca380cf2f0 0x7fca3806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca3f000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380cb570 0x7fca380cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca3f000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca380cb0e0 con 0x7fca3815ca80 2024-03-30T00:42:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca3f000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca380cf7f0 con 0x7fca380cf2f0 2024-03-30T00:42:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca3f000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca380cfad0 con 0x7fca380cb570 2024-03-30T00:42:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fca3815ca80 0x7fca3806b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56274/0 (socket says 172.21.15.66:56274) 2024-03-30T00:42:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37e00640 1 -- 172.21.15.66:0/1527160947 learned_addr learned my addr 172.21.15.66:0/1527160947 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca2fe00640 1 --2- 172.21.15.66:0/1527160947 >> v2:172.21.15.156:3300/0 conn(0x7fca380cf2f0 0x7fca3806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca3d000640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380cb570 0x7fca380cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888125292 0 0) 0x7fca380cb0e0 con 0x7fca3815ca80 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca1c003530 con 0x7fca3815ca80 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3667680817 0 0) 0x7fca1c003530 con 0x7fca3815ca80 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380cb570 msgr2=0x7fca380cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380cb570 0x7fca380cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 >> v2:172.21.15.156:3300/0 conn(0x7fca380cf2f0 msgr2=0x7fca3806b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 --2- 172.21.15.66:0/1527160947 >> v2:172.21.15.156:3300/0 conn(0x7fca380cf2f0 0x7fca3806b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380cad70 con 0x7fca3815ca80 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fca28001680 con 0x7fca3815ca80 2024-03-30T00:42:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 >> v1:172.21.15.66:6789/0 conn(0x7fca3815ca80 legacy=0x7fca3806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380cb570 0x7fca1c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca1c003f00 0x7fca1c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca1c004d10 0x7fca1c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca380cf7f0 con 0x7fca1c004d10 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca380cfad0 con 0x7fca1c003f00 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c00b9c0 con 0x7fca380cb570 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca3d000640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca1c004d10 0x7fca1c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca37e00640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380cb570 0x7fca1c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.572+0000 7fca2fe00640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca1c003f00 0x7fca1c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3d000640 1 -- 172.21.15.66:0/1527160947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380cb570 msgr2=0x7fca1c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3d000640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380cb570 0x7fca1c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3d000640 1 -- 172.21.15.66:0/1527160947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca1c003f00 msgr2=0x7fca1c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3d000640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca1c003f00 0x7fca1c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3d000640 1 -- 172.21.15.66:0/1527160947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c003530 con 0x7fca1c004d10 2024-03-30T00:42:57.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3d000640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca1c004d10 0x7fca1c005190 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fca30009fc0 tx=0x7fca3001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff8e3b07a788856 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:57.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300a5c90 con 0x7fca1c004d10 2024-03-30T00:42:57.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3000f040 con 0x7fca1c004d10 2024-03-30T00:42:57.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca37400640 1 -- 172.21.15.66:0/1527160947 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300ad440 con 0x7fca1c004d10 2024-03-30T00:42:57.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3f000640 1 -- 172.21.15.66:0/1527160947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca1c004d10 msgr2=0x7fca1c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3f000640 1 --2- 172.21.15.66:0/1527160947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca1c004d10 0x7fca1c005190 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fca30009fc0 tx=0x7fca3001cc80 comp rx=0 tx=0).stop 2024-03-30T00:42:57.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3d000640 1 -- 172.21.15.66:0/1527160947 reap_dead start 2024-03-30T00:42:57.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3f000640 1 -- 172.21.15.66:0/1527160947 shutdown_connections 2024-03-30T00:42:57.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3f000640 1 -- 172.21.15.66:0/1527160947 >> 172.21.15.66:0/1527160947 conn(0x7fca380b2090 msgr2=0x7fca380b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:57.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.576+0000 7fca3f000640 1 -- 172.21.15.66:0/1527160947 shutdown_connections 2024-03-30T00:42:57.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca3f000640 1 -- 172.21.15.66:0/1527160947 wait complete. 2024-03-30T00:42:57.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca3f000640 1 Processor -- start 2024-03-30T00:42:57.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca3f000640 1 -- start start 2024-03-30T00:42:57.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca3f000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca1c004d10 0x7fca38151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca3f000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca380adb80 0x7fca380ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca3f000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380c8bd0 0x7fca380c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca3f000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca380c9590 con 0x7fca380adb80 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca3f000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca38152270 con 0x7fca1c004d10 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca3f000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca380b06c0 con 0x7fca380c8bd0 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca380adb80 0x7fca380ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca380adb80 0x7fca380ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39066/0 (socket says 172.21.15.66:39066) 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca2fe00640 1 -- 172.21.15.66:0/2436945331 learned_addr learned my addr 172.21.15.66:0/2436945331 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca37e00640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca1c004d10 0x7fca38151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca3d000640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380c8bd0 0x7fca380c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca2fe00640 1 -- 172.21.15.66:0/2436945331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380c8bd0 msgr2=0x7fca380c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca2fe00640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380c8bd0 0x7fca380c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca2fe00640 1 -- 172.21.15.66:0/2436945331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca1c004d10 msgr2=0x7fca38151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca2fe00640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca1c004d10 0x7fca38151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca2fe00640 1 -- 172.21.15.66:0/2436945331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380b09a0 con 0x7fca380adb80 2024-03-30T00:42:57.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca37e00640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca1c004d10 0x7fca38151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca2fe00640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca380adb80 0x7fca380ae000 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fca20011a00 tx=0x7fca20011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca34c00640 1 -- 172.21.15.66:0/2436945331 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000fb50 con 0x7fca380adb80 2024-03-30T00:42:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca34c00640 1 -- 172.21.15.66:0/2436945331 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2000fcb0 con 0x7fca380adb80 2024-03-30T00:42:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca34c00640 1 -- 172.21.15.66:0/2436945331 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20014440 con 0x7fca380adb80 2024-03-30T00:42:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.580+0000 7fca3f000640 1 -- 172.21.15.66:0/2436945331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca380b0c80 con 0x7fca380adb80 2024-03-30T00:42:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.584+0000 7fca3f000640 1 -- 172.21.15.66:0/2436945331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca38151e50 con 0x7fca380adb80 2024-03-30T00:42:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.584+0000 7fca3f000640 1 -- 172.21.15.66:0/2436945331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca38066c30 con 0x7fca380adb80 2024-03-30T00:42:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.584+0000 7fca34c00640 1 -- 172.21.15.66:0/2436945331 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fca200145a0 con 0x7fca380adb80 2024-03-30T00:42:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.584+0000 7fca34c00640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca14066690 0x7fca14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.584+0000 7fca34c00640 1 -- 172.21.15.66:0/2436945331 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fca20084bf0 con 0x7fca380adb80 2024-03-30T00:42:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.584+0000 7fca37e00640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca14066690 0x7fca14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.584+0000 7fca37e00640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca14066690 0x7fca14068b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fca28001060 tx=0x7fca28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:57.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.588+0000 7fca34c00640 1 -- 172.21.15.66:0/2436945331 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fca20051e90 con 0x7fca380adb80 2024-03-30T00:42:57.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.784+0000 7fca3f000640 1 -- 172.21.15.66:0/2436945331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca3815ba80 con 0x7fca380adb80 2024-03-30T00:42:57.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.784+0000 7fca34c00640 1 -- 172.21.15.66:0/2436945331 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fca20022020 con 0x7fca380adb80 2024-03-30T00:42:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:57.792 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/465916334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":465916334},{"type":"v1","addr":"172.21.15.156:6835","nonce":465916334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1640047258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1640047258},{"type":"v1","addr":"172.21.15.156:6834","nonce":1640047258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5788,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2112021115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2112021115},{"type":"v1","addr":"172.21.15.66:6835","nonce":2112021115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:42:08.269705+0000","modified":"2024-03-30T00:42:11.492352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5985},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5985":{"gid":5985,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/3557086525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3557086525},{"type":"v1","addr":"172.21.15.66:6837","nonce":3557086525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T00:42:57.792 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-03-30T00:42:57.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 -- 172.21.15.66:0/2436945331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca14066690 msgr2=0x7fca14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca14066690 0x7fca14068b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fca28001060 tx=0x7fca28009040 comp rx=0 tx=0).stop 2024-03-30T00:42:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 -- 172.21.15.66:0/2436945331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca380adb80 msgr2=0x7fca380ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca380adb80 0x7fca380ae000 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fca20011a00 tx=0x7fca20011ec0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 -- 172.21.15.66:0/2436945331 shutdown_connections 2024-03-30T00:42:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca380c8bd0 0x7fca380c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca14066690 0x7fca14068b50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca380adb80 0x7fca380ae000 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 --2- 172.21.15.66:0/2436945331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca1c004d10 0x7fca38151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 -- 172.21.15.66:0/2436945331 >> 172.21.15.66:0/2436945331 conn(0x7fca380b2090 msgr2=0x7fca380c60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 -- 172.21.15.66:0/2436945331 shutdown_connections 2024-03-30T00:42:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.788+0000 7fca3f000640 1 -- 172.21.15.66:0/2436945331 wait complete. 2024-03-30T00:42:57.815 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:42:57.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.948+0000 7f0017600640 1 Processor -- start 2024-03-30T00:42:57.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.948+0000 7f0017600640 1 -- start start 2024-03-30T00:42:57.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.948+0000 7f0017600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015c780 0x7f00100cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f0017600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00100b1d00 con 0x7f0010160bb0 2024-03-30T00:42:57.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f0017600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00100cff70 con 0x7f001015cb60 2024-03-30T00:42:57.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f0017600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00100cd820 con 0x7f001015c780 2024-03-30T00:42:57.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f0015600640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0010160bb0 0x7f00100d0290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56276/0 (socket says 172.21.15.66:56276) 2024-03-30T00:42:57.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f0015600640 1 -- 172.21.15.66:0/2454933055 learned_addr learned my addr 172.21.15.66:0/2454933055 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:57.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f0014c00640 1 --2- 172.21.15.66:0/2454933055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015c780 0x7f00100cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f000f400640 1 -- 172.21.15.66:0/2454933055 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240464193 0 0) 0x7f00100b1d00 con 0x7f0010160bb0 2024-03-30T00:42:57.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f000f400640 1 -- 172.21.15.66:0/2454933055 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff8003530 con 0x7f0010160bb0 2024-03-30T00:42:57.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f000f400640 1 -- 172.21.15.66:0/2454933055 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765869629 0 0) 0x7f00100cff70 con 0x7f001015cb60 2024-03-30T00:42:57.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f000f400640 1 -- 172.21.15.66:0/2454933055 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00100b1d00 con 0x7f001015cb60 2024-03-30T00:42:57.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f0014c00640 1 -- 172.21.15.66:0/2454933055 >> v1:172.21.15.156:6789/0 conn(0x7f001015cb60 legacy=0x7f00101604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f000f400640 1 -- 172.21.15.66:0/2454933055 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1138906760 0 0) 0x7efff8003530 con 0x7f0010160bb0 2024-03-30T00:42:57.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f0014c00640 1 -- 172.21.15.66:0/2454933055 >> v1:172.21.15.66:6789/0 conn(0x7f0010160bb0 legacy=0x7f00100d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.952+0000 7f0014c00640 1 -- 172.21.15.66:0/2454933055 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00100cdb00 con 0x7f001015c780 2024-03-30T00:42:57.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0014c00640 1 --2- 172.21.15.66:0/2454933055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015c780 0x7f00100cfa30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f00040097f0 tx=0x7f000401c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4d861e9e0aa86de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f000f400640 1 -- 172.21.15.66:0/2454933055 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008001a00 con 0x7f001015c780 2024-03-30T00:42:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f000f400640 1 -- 172.21.15.66:0/2454933055 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00040a7050 con 0x7f001015c780 2024-03-30T00:42:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f000f400640 1 -- 172.21.15.66:0/2454933055 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00040ab620 con 0x7f001015c780 2024-03-30T00:42:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 -- 172.21.15.66:0/2454933055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015c780 msgr2=0x7f00100cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 --2- 172.21.15.66:0/2454933055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015c780 0x7f00100cfa30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f00040097f0 tx=0x7f000401c3d0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 -- 172.21.15.66:0/2454933055 shutdown_connections 2024-03-30T00:42:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 --2- 172.21.15.66:0/2454933055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015c780 0x7f00100cfa30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 -- 172.21.15.66:0/2454933055 >> 172.21.15.66:0/2454933055 conn(0x7f001009ba30 msgr2=0x7f0010089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 -- 172.21.15.66:0/2454933055 shutdown_connections 2024-03-30T00:42:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 -- 172.21.15.66:0/2454933055 wait complete. 2024-03-30T00:42:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 Processor -- start 2024-03-30T00:42:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 -- start start 2024-03-30T00:42:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001015c780 0x7f00100c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015cb60 0x7f00101534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0010160bb0 0x7f00100c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00100b2dc0 con 0x7f001015c780 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001015e860 con 0x7f0010160bb0 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0017600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001015eb40 con 0x7f001015cb60 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0014c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001015c780 0x7f00100c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0014c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001015c780 0x7f00100c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39072/0 (socket says 172.21.15.66:39072) 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.956+0000 7f0014c00640 1 -- 172.21.15.66:0/1554125886 learned_addr learned my addr 172.21.15.66:0/1554125886 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f0014c00640 1 -- 172.21.15.66:0/1554125886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015cb60 msgr2=0x7f00101534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f0015600640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0010160bb0 0x7f00100c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f000fe00640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015cb60 0x7f00101534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f0014c00640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015cb60 0x7f00101534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f0014c00640 1 -- 172.21.15.66:0/1554125886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0010160bb0 msgr2=0x7f00100c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f0014c00640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0010160bb0 0x7f00100c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f0014c00640 1 -- 172.21.15.66:0/1554125886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0004009420 con 0x7f001015c780 2024-03-30T00:42:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f0015600640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0010160bb0 0x7f00100c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f000fe00640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015cb60 0x7f00101534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:42:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f0014c00640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001015c780 0x7f00100c9790 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f00040ae040 tx=0x7f000401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f000cc00640 1 -- 172.21.15.66:0/1554125886 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00040abcd0 con 0x7f001015c780 2024-03-30T00:42:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f000cc00640 1 -- 172.21.15.66:0/1554125886 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00040abe50 con 0x7f001015c780 2024-03-30T00:42:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f0017600640 1 -- 172.21.15.66:0/1554125886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001015ee20 con 0x7f001015c780 2024-03-30T00:42:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f000cc00640 1 -- 172.21.15.66:0/1554125886 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004004260 con 0x7f001015c780 2024-03-30T00:42:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f0017600640 1 -- 172.21.15.66:0/1554125886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00101539f0 con 0x7f001015c780 2024-03-30T00:42:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f0017600640 1 -- 172.21.15.66:0/1554125886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001015e6c0 con 0x7f001015c780 2024-03-30T00:42:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f000cc00640 1 -- 172.21.15.66:0/1554125886 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f00040043c0 con 0x7f001015c780 2024-03-30T00:42:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f000cc00640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efff0066690 0x7efff0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f000cc00640 1 -- 172.21.15.66:0/1554125886 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f000411e490 con 0x7f001015c780 2024-03-30T00:42:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.960+0000 7f000fe00640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efff0066690 0x7efff0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.964+0000 7f000fe00640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efff0066690 0x7efff0068b50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f00101687b0 tx=0x7efffc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:57.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:57.964+0000 7f000cc00640 1 -- 172.21.15.66:0/1554125886 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f001015e6c0 con 0x7f001015c780 2024-03-30T00:42:58.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.128+0000 7f0017600640 1 -- 172.21.15.66:0/1554125886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f001015e6c0 con 0x7f001015c780 2024-03-30T00:42:58.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.128+0000 7f000cc00640 1 -- 172.21.15.66:0/1554125886 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16402 (secure 0 0 0) 0x7f001015e6c0 con 0x7f001015c780 2024-03-30T00:42:58.136 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:58.136 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":56,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:42:33.561663+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:42:06.807254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-30T00:42:07.258973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:42:58.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 -- 172.21.15.66:0/1554125886 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efff0066690 msgr2=0x7efff0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efff0066690 0x7efff0068b50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f00101687b0 tx=0x7efffc009040 comp rx=0 tx=0).stop 2024-03-30T00:42:58.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 -- 172.21.15.66:0/1554125886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001015c780 msgr2=0x7f00100c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001015c780 0x7f00100c9790 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f00040ae040 tx=0x7f000401ac10 comp rx=0 tx=0).stop 2024-03-30T00:42:58.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 -- 172.21.15.66:0/1554125886 shutdown_connections 2024-03-30T00:42:58.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0010160bb0 0x7f00100c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f001015cb60 0x7f00101534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efff0066690 0x7efff0068b50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 --2- 172.21.15.66:0/1554125886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001015c780 0x7f00100c9790 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 -- 172.21.15.66:0/1554125886 >> 172.21.15.66:0/1554125886 conn(0x7f001009ba30 msgr2=0x7f00100b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:58.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 -- 172.21.15.66:0/1554125886 shutdown_connections 2024-03-30T00:42:58.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.132+0000 7f0017600640 1 -- 172.21.15.66:0/1554125886 wait complete. 2024-03-30T00:42:58.158 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T00:42:58.158 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:42:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26d7400640 1 Processor -- start 2024-03-30T00:42:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26d7400640 1 -- start start 2024-03-30T00:42:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26d7400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f26d006e800 0x7f26d00d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26d7400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d006f8e0 0x7f26d00ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26d7400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d006b0b0 con 0x7f26d00cc560 2024-03-30T00:42:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26d7400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d0160640 con 0x7f26d006e800 2024-03-30T00:42:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26d7400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d006b390 con 0x7f26d006f8e0 2024-03-30T00:42:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26d4a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f26d00cc560 0x7f26d006e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56280/0 (socket says 172.21.15.66:56280) 2024-03-30T00:42:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26d4a00640 1 -- 172.21.15.66:0/4034719351 learned_addr learned my addr 172.21.15.66:0/4034719351 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26c7e00640 1 --2- 172.21.15.66:0/4034719351 >> v2:172.21.15.156:3300/0 conn(0x7f26d006e800 0x7f26d00d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26d5400640 1 --2- 172.21.15.66:0/4034719351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d006f8e0 0x7f26d00ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986766272 0 0) 0x7f26d006b0b0 con 0x7f26d00cc560 2024-03-30T00:42:58.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b4003530 con 0x7f26d00cc560 2024-03-30T00:42:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2835057244 0 0) 0x7f26b4003530 con 0x7f26d00cc560 2024-03-30T00:42:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d006f8e0 msgr2=0x7f26d00ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 --2- 172.21.15.66:0/4034719351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d006f8e0 0x7f26d00ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 >> v2:172.21.15.156:3300/0 conn(0x7f26d006e800 msgr2=0x7f26d00d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 --2- 172.21.15.66:0/4034719351 >> v2:172.21.15.156:3300/0 conn(0x7f26d006e800 0x7f26d00d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d006b670 con 0x7f26d00cc560 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f26c00016a0 con 0x7f26d00cc560 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 >> v1:172.21.15.66:6789/0 conn(0x7f26d00cc560 legacy=0x7f26d006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 --2- 172.21.15.66:0/4034719351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26b4003980 0x7f26b4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 --2- 172.21.15.66:0/4034719351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b40047d0 0x7f26b4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 --2- 172.21.15.66:0/4034719351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d006f8e0 0x7f26b4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d0160640 con 0x7f26b4003980 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d006b390 con 0x7f26b40047d0 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b400b990 con 0x7f26d006f8e0 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26d4a00640 1 --2- 172.21.15.66:0/4034719351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26b4003980 0x7f26b4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.292+0000 7f26c7e00640 1 --2- 172.21.15.66:0/4034719351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b40047d0 0x7f26b4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d4a00640 1 -- 172.21.15.66:0/4034719351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d006f8e0 msgr2=0x7f26b4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:58.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d4a00640 1 --2- 172.21.15.66:0/4034719351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d006f8e0 0x7f26b4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d4a00640 1 -- 172.21.15.66:0/4034719351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b40047d0 msgr2=0x7f26b4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:58.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d4a00640 1 --2- 172.21.15.66:0/4034719351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b40047d0 0x7f26b4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d4a00640 1 -- 172.21.15.66:0/4034719351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b4003530 con 0x7f26b4003980 2024-03-30T00:42:58.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d4a00640 1 --2- 172.21.15.66:0/4034719351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26b4003980 0x7f26b4003da0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f26c000c160 tx=0x7f26c001cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb7e44abcfacd2d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:58.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c00058e0 con 0x7f26b4003980 2024-03-30T00:42:58.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c0005a40 con 0x7f26b4003980 2024-03-30T00:42:58.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26cfe00640 1 -- 172.21.15.66:0/4034719351 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0005df0 con 0x7f26b4003980 2024-03-30T00:42:58.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d7400640 1 -- 172.21.15.66:0/4034719351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26b4003980 msgr2=0x7f26b4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d7400640 1 --2- 172.21.15.66:0/4034719351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26b4003980 0x7f26b4003da0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f26c000c160 tx=0x7f26c001cb50 comp rx=0 tx=0).stop 2024-03-30T00:42:58.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d5400640 1 -- 172.21.15.66:0/4034719351 reap_dead start 2024-03-30T00:42:58.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d7400640 1 -- 172.21.15.66:0/4034719351 shutdown_connections 2024-03-30T00:42:58.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d7400640 1 -- 172.21.15.66:0/4034719351 >> 172.21.15.66:0/4034719351 conn(0x7f26d00d0540 msgr2=0x7f26d00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:58.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d7400640 1 -- 172.21.15.66:0/4034719351 shutdown_connections 2024-03-30T00:42:58.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.296+0000 7f26d7400640 1 -- 172.21.15.66:0/4034719351 wait complete. 2024-03-30T00:42:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d7400640 1 Processor -- start 2024-03-30T00:42:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d7400640 1 -- start start 2024-03-30T00:42:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d7400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b4003980 0x7f26d01f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d7400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d00c8e10 0x7f26d00b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d0159820 0x7f26d0159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d7400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d00b1bc0 con 0x7f26d0159820 2024-03-30T00:42:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d7400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d00ca270 con 0x7f26b4003980 2024-03-30T00:42:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d7400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d015a1e0 con 0x7f26d00c8e10 2024-03-30T00:42:58.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d0159820 0x7f26d0159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d0159820 0x7f26d0159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39088/0 (socket says 172.21.15.66:39088) 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d5400640 1 -- 172.21.15.66:0/1910515491 learned_addr learned my addr 172.21.15.66:0/1910515491 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26c7e00640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d00c8e10 0x7f26d00b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d5400640 1 -- 172.21.15.66:0/1910515491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d00c8e10 msgr2=0x7f26d00b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d5400640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d00c8e10 0x7f26d00b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d5400640 1 -- 172.21.15.66:0/1910515491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b4003980 msgr2=0x7f26d01f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d5400640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b4003980 0x7f26d01f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d5400640 1 -- 172.21.15.66:0/1910515491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d00b0d50 con 0x7f26d0159820 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d5400640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d0159820 0x7f26d0159ca0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f26d0072bd0 tx=0x7f26c800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26cd600640 1 -- 172.21.15.66:0/1910515491 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8010040 con 0x7f26d0159820 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26cd600640 1 -- 172.21.15.66:0/1910515491 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c800bce0 con 0x7f26d0159820 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26cd600640 1 -- 172.21.15.66:0/1910515491 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8014640 con 0x7f26d0159820 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d7400640 1 -- 172.21.15.66:0/1910515491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d00b0fb0 con 0x7f26d0159820 2024-03-30T00:42:58.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.300+0000 7f26d7400640 1 -- 172.21.15.66:0/1910515491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d00b1570 con 0x7f26d0159820 2024-03-30T00:42:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.304+0000 7f26d7400640 1 -- 172.21.15.66:0/1910515491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d0066c30 con 0x7f26d0159820 2024-03-30T00:42:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.304+0000 7f26cd600640 1 -- 172.21.15.66:0/1910515491 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f26c80147a0 con 0x7f26d0159820 2024-03-30T00:42:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.304+0000 7f26cd600640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26ac0666c0 0x7f26ac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.304+0000 7f26cd600640 1 -- 172.21.15.66:0/1910515491 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f26c8084a60 con 0x7f26d0159820 2024-03-30T00:42:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.304+0000 7f26d4a00640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26ac0666c0 0x7f26ac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.304+0000 7f26d4a00640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26ac0666c0 0x7f26ac068b80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f26c00aa000 tx=0x7f26c0001960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:58.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.308+0000 7f26cd600640 1 -- 172.21.15.66:0/1910515491 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f26c8051ce0 con 0x7f26d0159820 2024-03-30T00:42:58.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.504+0000 7f26d7400640 1 -- 172.21.15.66:0/1910515491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26d0067340 con 0x7f26d0159820 2024-03-30T00:42:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.504+0000 7f26cd600640 1 -- 172.21.15.66:0/1910515491 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f26c8018090 con 0x7f26d0159820 2024-03-30T00:42:58.512 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:58.512 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/465916334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":465916334},{"type":"v1","addr":"172.21.15.156:6835","nonce":465916334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1640047258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1640047258},{"type":"v1","addr":"172.21.15.156:6834","nonce":1640047258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5788,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2112021115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2112021115},{"type":"v1","addr":"172.21.15.66:6835","nonce":2112021115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:42:08.269705+0000","modified":"2024-03-30T00:42:11.492352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5985},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5985":{"gid":5985,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/3557086525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3557086525},{"type":"v1","addr":"172.21.15.66:6837","nonce":3557086525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T00:42:58.512 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-03-30T00:42:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 -- 172.21.15.66:0/1910515491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26ac0666c0 msgr2=0x7f26ac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26ac0666c0 0x7f26ac068b80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f26c00aa000 tx=0x7f26c0001960 comp rx=0 tx=0).stop 2024-03-30T00:42:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 -- 172.21.15.66:0/1910515491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d0159820 msgr2=0x7f26d0159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d0159820 0x7f26d0159ca0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f26d0072bd0 tx=0x7f26c800b480 comp rx=0 tx=0).stop 2024-03-30T00:42:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 -- 172.21.15.66:0/1910515491 shutdown_connections 2024-03-30T00:42:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26ac0666c0 0x7f26ac068b80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d0159820 0x7f26d0159ca0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d00c8e10 0x7f26d00b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 --2- 172.21.15.66:0/1910515491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b4003980 0x7f26d01f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 -- 172.21.15.66:0/1910515491 >> 172.21.15.66:0/1910515491 conn(0x7f26d00d0540 msgr2=0x7f26d015b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 -- 172.21.15.66:0/1910515491 shutdown_connections 2024-03-30T00:42:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.508+0000 7f26d7400640 1 -- 172.21.15.66:0/1910515491 wait complete. 2024-03-30T00:42:58.532 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:42:58.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b4800640 1 Processor -- start 2024-03-30T00:42:58.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b4800640 1 -- start start 2024-03-30T00:42:58.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b4800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58ac15a640 0x7f58ac06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b4800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58ac0ce520 0x7f58ac0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b4800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac06d070 con 0x7f58ac0cabb0 2024-03-30T00:42:58.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b4800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ac0cbba0 con 0x7f58ac15a640 2024-03-30T00:42:58.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b4800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac06e2c0 con 0x7f58ac0ce520 2024-03-30T00:42:58.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b1e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58ac15a640 0x7f58ac06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b1e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58ac15a640 0x7f58ac06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39092/0 (socket says 172.21.15.66:39092) 2024-03-30T00:42:58.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b1e00640 1 -- 172.21.15.66:0/3201676077 learned_addr learned my addr 172.21.15.66:0/3201676077 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:58.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b1e00640 1 -- 172.21.15.66:0/3201676077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58ac0ce520 msgr2=0x7f58ac0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:58.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b1e00640 1 --2- 172.21.15.66:0/3201676077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58ac0ce520 0x7f58ac0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b1e00640 1 -- 172.21.15.66:0/3201676077 >> v1:172.21.15.156:6789/0 conn(0x7f58ac0cabb0 legacy=0x7f58ac0cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:58.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.668+0000 7f58b1e00640 1 -- 172.21.15.66:0/3201676077 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac0cee60 con 0x7f58ac15a640 2024-03-30T00:42:58.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b1e00640 1 --2- 172.21.15.66:0/3201676077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ac15a640 0x7f58ac06dd80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009870 tx=0x7f58a001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc8f132e38530c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:58.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b0a00640 1 -- 172.21.15.66:0/3201676077 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a00ac630 con 0x7f58ac15a640 2024-03-30T00:42:58.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b0a00640 1 -- 172.21.15.66:0/3201676077 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a00ac790 con 0x7f58ac15a640 2024-03-30T00:42:58.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b0a00640 1 -- 172.21.15.66:0/3201676077 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a00acb20 con 0x7f58ac15a640 2024-03-30T00:42:58.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b4800640 1 -- 172.21.15.66:0/3201676077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ac15a640 msgr2=0x7f58ac06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b4800640 1 --2- 172.21.15.66:0/3201676077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ac15a640 0x7f58ac06dd80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009870 tx=0x7f58a001c520 comp rx=0 tx=0).stop 2024-03-30T00:42:58.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b4800640 1 -- 172.21.15.66:0/3201676077 shutdown_connections 2024-03-30T00:42:58.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b4800640 1 --2- 172.21.15.66:0/3201676077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58ac0ce520 0x7f58ac0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b4800640 1 --2- 172.21.15.66:0/3201676077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ac15a640 0x7f58ac06dd80 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b4800640 1 -- 172.21.15.66:0/3201676077 >> 172.21.15.66:0/3201676077 conn(0x7f58ac069f60 msgr2=0x7f58ac06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:58.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b4800640 1 -- 172.21.15.66:0/3201676077 shutdown_connections 2024-03-30T00:42:58.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.672+0000 7f58b4800640 1 -- 172.21.15.66:0/3201676077 wait complete. 2024-03-30T00:42:58.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b4800640 1 Processor -- start 2024-03-30T00:42:58.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b4800640 1 -- start start 2024-03-30T00:42:58.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b4800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ac0cabb0 0x7f58ac156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b4800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58ac0ce520 0x7f58ac0998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b4800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58ac15a640 0x7f58ac09b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b4800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac09a120 con 0x7f58ac0cabb0 2024-03-30T00:42:58.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b4800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac156ca0 con 0x7f58ac15a640 2024-03-30T00:42:58.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b4800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac09ac30 con 0x7f58ac0ce520 2024-03-30T00:42:58.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ac0cabb0 0x7f58ac156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ac0cabb0 0x7f58ac156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39098/0 (socket says 172.21.15.66:39098) 2024-03-30T00:42:58.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b1e00640 1 -- 172.21.15.66:0/72198131 learned_addr learned my addr 172.21.15.66:0/72198131 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b1400640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58ac0ce520 0x7f58ac0998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b2800640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58ac15a640 0x7f58ac09b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b1e00640 1 -- 172.21.15.66:0/72198131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58ac0ce520 msgr2=0x7f58ac0998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b1e00640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58ac0ce520 0x7f58ac0998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b1e00640 1 -- 172.21.15.66:0/72198131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58ac15a640 msgr2=0x7f58ac09b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b1e00640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58ac15a640 0x7f58ac09b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b1e00640 1 -- 172.21.15.66:0/72198131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00094d0 con 0x7f58ac0cabb0 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b1e00640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ac0cabb0 0x7f58ac156420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f58a00af040 tx=0x7f58a001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58a6000640 1 -- 172.21.15.66:0/72198131 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a00b7830 con 0x7f58ac0cabb0 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58a6000640 1 -- 172.21.15.66:0/72198131 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a00b7990 con 0x7f58ac0cabb0 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58a6000640 1 -- 172.21.15.66:0/72198131 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a00b7d20 con 0x7f58ac0cabb0 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b4800640 1 -- 172.21.15.66:0/72198131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac09a300 con 0x7f58ac0cabb0 2024-03-30T00:42:58.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b4800640 1 -- 172.21.15.66:0/72198131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac09a8c0 con 0x7f58ac0cabb0 2024-03-30T00:42:58.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.676+0000 7f58b4800640 1 -- 172.21.15.66:0/72198131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac066c30 con 0x7f58ac0cabb0 2024-03-30T00:42:58.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.680+0000 7f58a6000640 1 -- 172.21.15.66:0/72198131 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f58a00b3040 con 0x7f58ac0cabb0 2024-03-30T00:42:58.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.680+0000 7f58a6000640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5890066690 0x7f5890068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:58.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.680+0000 7f58a6000640 1 -- 172.21.15.66:0/72198131 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f58a011e7a0 con 0x7f58ac0cabb0 2024-03-30T00:42:58.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.680+0000 7f58b1400640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5890066690 0x7f5890068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.680+0000 7f58b1400640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5890066690 0x7f5890068b50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f58ac1578b0 tx=0x7f589c009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:58.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.684+0000 7f58a6000640 1 -- 172.21.15.66:0/72198131 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f58a00eb9b0 con 0x7f58ac0cabb0 2024-03-30T00:42:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.880+0000 7f58b4800640 1 -- 172.21.15.66:0/72198131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58ac15c070 con 0x7f58ac0cabb0 2024-03-30T00:42:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.880+0000 7f58a6000640 1 -- 172.21.15.66:0/72198131 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f58a00eefd0 con 0x7f58ac0cabb0 2024-03-30T00:42:58.887 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:58.887 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/465916334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":465916334},{"type":"v1","addr":"172.21.15.156:6835","nonce":465916334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1640047258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1640047258},{"type":"v1","addr":"172.21.15.156:6834","nonce":1640047258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5788,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2112021115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2112021115},{"type":"v1","addr":"172.21.15.66:6835","nonce":2112021115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:42:08.269705+0000","modified":"2024-03-30T00:42:11.492352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5985},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5985":{"gid":5985,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/3557086525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3557086525},{"type":"v1","addr":"172.21.15.66:6837","nonce":3557086525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T00:42:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2024-03-30T00:42:58.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.880+0000 7f58b4800640 1 -- 172.21.15.66:0/72198131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5890066690 msgr2=0x7f5890068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.880+0000 7f58b4800640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5890066690 0x7f5890068b50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f58ac1578b0 tx=0x7f589c009110 comp rx=0 tx=0).stop 2024-03-30T00:42:58.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.880+0000 7f58b4800640 1 -- 172.21.15.66:0/72198131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ac0cabb0 msgr2=0x7f58ac156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:58.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.880+0000 7f58b4800640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ac0cabb0 0x7f58ac156420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f58a00af040 tx=0x7f58a001ad60 comp rx=0 tx=0).stop 2024-03-30T00:42:58.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.880+0000 7f58b4800640 1 -- 172.21.15.66:0/72198131 shutdown_connections 2024-03-30T00:42:58.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.880+0000 7f58b4800640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58ac15a640 0x7f58ac09b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.880+0000 7f58b4800640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58ac0ce520 0x7f58ac0998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.880+0000 7f58b4800640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5890066690 0x7f5890068b50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.880+0000 7f58b4800640 1 --2- 172.21.15.66:0/72198131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58ac0cabb0 0x7f58ac156420 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:58.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.884+0000 7f58b4800640 1 -- 172.21.15.66:0/72198131 >> 172.21.15.66:0/72198131 conn(0x7f58ac069f60 msgr2=0x7f58ac0cafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:58.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.884+0000 7f58b4800640 1 -- 172.21.15.66:0/72198131 shutdown_connections 2024-03-30T00:42:58.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:58.884+0000 7f58b4800640 1 -- 172.21.15.66:0/72198131 wait complete. 2024-03-30T00:42:58.909 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:42:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc239400640 1 Processor -- start 2024-03-30T00:42:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc239400640 1 -- start start 2024-03-30T00:42:59.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc239400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc23406e800 0x7fc2340d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc239400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc23406f8e0 0x7fc2340ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc239400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc23406b0b0 con 0x7fc2340cc560 2024-03-30T00:42:59.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc239400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc234160640 con 0x7fc23406e800 2024-03-30T00:42:59.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc239400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23406b390 con 0x7fc23406f8e0 2024-03-30T00:42:59.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc229e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc23406e800 0x7fc2340d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc229e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc23406e800 0x7fc2340d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34518/0 (socket says 172.21.15.66:34518) 2024-03-30T00:42:59.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc229e00640 1 -- 172.21.15.66:0/2824444191 learned_addr learned my addr 172.21.15.66:0/2824444191 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:59.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc233200640 1 --2- 172.21.15.66:0/2824444191 >> v2:172.21.15.156:3300/0 conn(0x7fc23406f8e0 0x7fc2340ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc229e00640 1 -- 172.21.15.66:0/2824444191 >> v2:172.21.15.156:3300/0 conn(0x7fc23406f8e0 msgr2=0x7fc2340ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc229e00640 1 --2- 172.21.15.66:0/2824444191 >> v2:172.21.15.156:3300/0 conn(0x7fc23406f8e0 0x7fc2340ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc229e00640 1 -- 172.21.15.66:0/2824444191 >> v1:172.21.15.156:6789/0 conn(0x7fc2340cc560 legacy=0x7fc23406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.044+0000 7fc229e00640 1 -- 172.21.15.66:0/2824444191 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23406b670 con 0x7fc23406e800 2024-03-30T00:42:59.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc229e00640 1 --2- 172.21.15.66:0/2824444191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc23406e800 0x7fc2340d1060 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc234070b00 tx=0x7fc21c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c902d09b0b92fc7b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:59.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc231e00640 1 -- 172.21.15.66:0/2824444191 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0aa670 con 0x7fc23406e800 2024-03-30T00:42:59.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc231e00640 1 -- 172.21.15.66:0/2824444191 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c0aa7d0 con 0x7fc23406e800 2024-03-30T00:42:59.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc231e00640 1 -- 172.21.15.66:0/2824444191 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0aab60 con 0x7fc23406e800 2024-03-30T00:42:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc239400640 1 -- 172.21.15.66:0/2824444191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc23406e800 msgr2=0x7fc2340d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc239400640 1 --2- 172.21.15.66:0/2824444191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc23406e800 0x7fc2340d1060 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc234070b00 tx=0x7fc21c01c4f0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc239400640 1 -- 172.21.15.66:0/2824444191 shutdown_connections 2024-03-30T00:42:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc239400640 1 --2- 172.21.15.66:0/2824444191 >> v2:172.21.15.156:3300/0 conn(0x7fc23406f8e0 0x7fc2340ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc239400640 1 --2- 172.21.15.66:0/2824444191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc23406e800 0x7fc2340d1060 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc239400640 1 -- 172.21.15.66:0/2824444191 >> 172.21.15.66:0/2824444191 conn(0x7fc2340d0540 msgr2=0x7fc2340d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc239400640 1 -- 172.21.15.66:0/2824444191 shutdown_connections 2024-03-30T00:42:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc239400640 1 -- 172.21.15.66:0/2824444191 wait complete. 2024-03-30T00:42:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.048+0000 7fc239400640 1 Processor -- start 2024-03-30T00:42:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc239400640 1 -- start start 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc239400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc23406e800 0x7fc234152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc239400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc23406f8e0 0x7fc2340c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc239400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cc560 0x7fc2340d0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc239400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2340c7980 con 0x7fc2340cc560 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc239400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2340c7c60 con 0x7fc23406e800 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc239400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2340c7f40 con 0x7fc23406f8e0 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc233200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cc560 0x7fc2340d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc233200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cc560 0x7fc2340d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34534/0 (socket says 172.21.15.66:34534) 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc233200640 1 -- 172.21.15.66:0/1349403857 learned_addr learned my addr 172.21.15.66:0/1349403857 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc229e00640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc23406f8e0 0x7fc2340c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc232800640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc23406e800 0x7fc234152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc233200640 1 -- 172.21.15.66:0/1349403857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc23406f8e0 msgr2=0x7fc2340c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc233200640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc23406f8e0 0x7fc2340c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc233200640 1 -- 172.21.15.66:0/1349403857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc23406e800 msgr2=0x7fc234152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc233200640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc23406e800 0x7fc234152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc233200640 1 -- 172.21.15.66:0/1349403857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c0094d0 con 0x7fc2340cc560 2024-03-30T00:42:59.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc233200640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cc560 0x7fc2340d0920 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00efc0 tx=0x7fc22c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:59.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc22b400640 1 -- 172.21.15.66:0/1349403857 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c014480 con 0x7fc2340cc560 2024-03-30T00:42:59.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc22b400640 1 -- 172.21.15.66:0/1349403857 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22c00f040 con 0x7fc2340cc560 2024-03-30T00:42:59.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc239400640 1 -- 172.21.15.66:0/1349403857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2340c8220 con 0x7fc2340cc560 2024-03-30T00:42:59.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc22b400640 1 -- 172.21.15.66:0/1349403857 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c014820 con 0x7fc2340cc560 2024-03-30T00:42:59.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc239400640 1 -- 172.21.15.66:0/1349403857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc234152ea0 con 0x7fc2340cc560 2024-03-30T00:42:59.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.052+0000 7fc239400640 1 -- 172.21.15.66:0/1349403857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2340c8460 con 0x7fc2340cc560 2024-03-30T00:42:59.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.056+0000 7fc22b400640 1 -- 172.21.15.66:0/1349403857 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc22c014980 con 0x7fc2340cc560 2024-03-30T00:42:59.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.056+0000 7fc22b400640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 0x7fc210068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.056+0000 7fc22b400640 1 -- 172.21.15.66:0/1349403857 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc22c084ef0 con 0x7fc2340cc560 2024-03-30T00:42:59.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.056+0000 7fc232800640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 0x7fc210068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.056+0000 7fc232800640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 0x7fc210068b50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc224000f70 tx=0x7fc224009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:59.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.060+0000 7fc22b400640 1 -- 172.21.15.66:0/1349403857 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc22c052040 con 0x7fc2340cc560 2024-03-30T00:42:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.220+0000 7fc239400640 1 -- 172.21.15.66:0/1349403857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc234066c30 con 0x7fc2340cc560 2024-03-30T00:42:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc22b400640 1 -- 172.21.15.66:0/1349403857 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16402 (secure 0 0 0) 0x7fc22c018020 con 0x7fc2340cc560 2024-03-30T00:42:59.229 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:42:59.230 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":56,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:42:33.561663+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:42:06.807254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-30T00:42:07.258973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:42:59.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 -- 172.21.15.66:0/1349403857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 msgr2=0x7fc210068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 0x7fc210068b50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc224000f70 tx=0x7fc224009040 comp rx=0 tx=0).stop 2024-03-30T00:42:59.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 -- 172.21.15.66:0/1349403857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cc560 msgr2=0x7fc2340d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cc560 0x7fc2340d0920 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00efc0 tx=0x7fc22c00c490 comp rx=0 tx=0).stop 2024-03-30T00:42:59.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 -- 172.21.15.66:0/1349403857 shutdown_connections 2024-03-30T00:42:59.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 0x7fc210068b50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cc560 0x7fc2340d0920 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc23406f8e0 0x7fc2340c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 --2- 172.21.15.66:0/1349403857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc23406e800 0x7fc234152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 -- 172.21.15.66:0/1349403857 >> 172.21.15.66:0/1349403857 conn(0x7fc2340d0540 msgr2=0x7fc2340ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:59.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 -- 172.21.15.66:0/1349403857 shutdown_connections 2024-03-30T00:42:59.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.224+0000 7fc239400640 1 -- 172.21.15.66:0/1349403857 wait complete. 2024-03-30T00:42:59.251 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:42:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.384+0000 7fbaeba00640 1 Processor -- start 2024-03-30T00:42:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.384+0000 7fbaeba00640 1 -- start start 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbaeba00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbae40cf2f0 0x7fbae406b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbaeba00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbae40cb570 0x7fbae40cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbaeba00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae40cb0e0 con 0x7fbae415ca80 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbaeba00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae40cf7f0 con 0x7fbae40cf2f0 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbaeba00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae40cfad0 con 0x7fbae40cb570 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbae3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbae40cf2f0 0x7fbae406b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbae9000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fbae415ca80 0x7fbae406b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57208/0 (socket says 172.21.15.66:57208) 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbae9000640 1 -- 172.21.15.66:0/3691286513 learned_addr learned my addr 172.21.15.66:0/3691286513 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbae9a00640 1 --2- 172.21.15.66:0/3691286513 >> v2:172.21.15.156:3300/0 conn(0x7fbae40cb570 0x7fbae40cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbae3e00640 1 -- 172.21.15.66:0/3691286513 >> v2:172.21.15.156:3300/0 conn(0x7fbae40cb570 msgr2=0x7fbae40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbae3e00640 1 --2- 172.21.15.66:0/3691286513 >> v2:172.21.15.156:3300/0 conn(0x7fbae40cb570 0x7fbae40cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbae3e00640 1 -- 172.21.15.66:0/3691286513 >> v1:172.21.15.66:6789/0 conn(0x7fbae415ca80 legacy=0x7fbae406b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbae3400640 1 -- 172.21.15.66:0/3691286513 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247987417 0 0) 0x7fbae40cb0e0 con 0x7fbae415ca80 2024-03-30T00:42:59.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.388+0000 7fbae3e00640 1 -- 172.21.15.66:0/3691286513 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae40cad70 con 0x7fbae40cf2f0 2024-03-30T00:42:59.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbae3e00640 1 --2- 172.21.15.66:0/3691286513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbae40cf2f0 0x7fbae406b7c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbae4071d80 tx=0x7fbad001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ad368eaf007285 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:59.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbae3400640 1 -- 172.21.15.66:0/3691286513 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad00a8970 con 0x7fbae40cf2f0 2024-03-30T00:42:59.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbae3400640 1 -- 172.21.15.66:0/3691286513 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad00a8ad0 con 0x7fbae40cf2f0 2024-03-30T00:42:59.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbae3400640 1 -- 172.21.15.66:0/3691286513 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad00a8e60 con 0x7fbae40cf2f0 2024-03-30T00:42:59.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 -- 172.21.15.66:0/3691286513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbae40cf2f0 msgr2=0x7fbae406b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 --2- 172.21.15.66:0/3691286513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbae40cf2f0 0x7fbae406b7c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbae4071d80 tx=0x7fbad001ec70 comp rx=0 tx=0).stop 2024-03-30T00:42:59.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 -- 172.21.15.66:0/3691286513 shutdown_connections 2024-03-30T00:42:59.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 --2- 172.21.15.66:0/3691286513 >> v2:172.21.15.156:3300/0 conn(0x7fbae40cb570 0x7fbae40cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 --2- 172.21.15.66:0/3691286513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbae40cf2f0 0x7fbae406b7c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 -- 172.21.15.66:0/3691286513 >> 172.21.15.66:0/3691286513 conn(0x7fbae40b2090 msgr2=0x7fbae40b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:59.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 -- 172.21.15.66:0/3691286513 shutdown_connections 2024-03-30T00:42:59.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 -- 172.21.15.66:0/3691286513 wait complete. 2024-03-30T00:42:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 Processor -- start 2024-03-30T00:42:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 -- start start 2024-03-30T00:42:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbae40cb570 0x7fbae40b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbae40cf2f0 0x7fbae40ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbae415ca80 0x7fbae41f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40b4540 con 0x7fbae40cb570 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4147730 con 0x7fbae415ca80 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbaeba00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40b4970 con 0x7fbae40cf2f0 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.392+0000 7fbae9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbae40cb570 0x7fbae40b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbae40cb570 0x7fbae40b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34548/0 (socket says 172.21.15.66:34548) 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae9000640 1 -- 172.21.15.66:0/1509100637 learned_addr learned my addr 172.21.15.66:0/1509100637 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae9a00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbae415ca80 0x7fbae41f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae3e00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbae40cf2f0 0x7fbae40ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae9000640 1 -- 172.21.15.66:0/1509100637 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbae40cf2f0 msgr2=0x7fbae40ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae9000640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbae40cf2f0 0x7fbae40ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae9000640 1 -- 172.21.15.66:0/1509100637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbae415ca80 msgr2=0x7fbae41f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae9000640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbae415ca80 0x7fbae41f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae9000640 1 -- 172.21.15.66:0/1509100637 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad0009420 con 0x7fbae40cb570 2024-03-30T00:42:59.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae3e00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbae40cf2f0 0x7fbae40ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:59.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae9000640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbae40cb570 0x7fbae40b54e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c480 tx=0x7fbad800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:59.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae0c00640 1 -- 172.21.15.66:0/1509100637 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad80074e0 con 0x7fbae40cb570 2024-03-30T00:42:59.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae0c00640 1 -- 172.21.15.66:0/1509100637 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad8007660 con 0x7fbae40cb570 2024-03-30T00:42:59.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbaeba00640 1 -- 172.21.15.66:0/1509100637 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae40ca590 con 0x7fbae40cb570 2024-03-30T00:42:59.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae0c00640 1 -- 172.21.15.66:0/1509100637 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad8004a30 con 0x7fbae40cb570 2024-03-30T00:42:59.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbaeba00640 1 -- 172.21.15.66:0/1509100637 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae40b5df0 con 0x7fbae40cb570 2024-03-30T00:42:59.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbaeba00640 1 -- 172.21.15.66:0/1509100637 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae4066c30 con 0x7fbae40cb570 2024-03-30T00:42:59.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae0c00640 1 -- 172.21.15.66:0/1509100637 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbad8004b90 con 0x7fbae40cb570 2024-03-30T00:42:59.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae0c00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbac4066690 0x7fbac4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.396+0000 7fbae0c00640 1 -- 172.21.15.66:0/1509100637 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fbad8083530 con 0x7fbae40cb570 2024-03-30T00:42:59.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.400+0000 7fbae3e00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbac4066690 0x7fbac4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.400+0000 7fbae3e00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbac4066690 0x7fbac4068b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbad00b2000 tx=0x7fbad0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:59.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.400+0000 7fbae0c00640 1 -- 172.21.15.66:0/1509100637 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbad80541a0 con 0x7fbae40cb570 2024-03-30T00:42:59.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.596+0000 7fbaeba00640 1 -- 172.21.15.66:0/1509100637 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbae4159030 con 0x7fbae40cb570 2024-03-30T00:42:59.603 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:42:59.596+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:42:59.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.604+0000 7fbae0c00640 1 -- 172.21.15.66:0/1509100637 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbad8053d30 con 0x7fbae40cb570 2024-03-30T00:42:59.612 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:42:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 -- 172.21.15.66:0/1509100637 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbac4066690 msgr2=0x7fbac4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbac4066690 0x7fbac4068b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbad00b2000 tx=0x7fbad0008040 comp rx=0 tx=0).stop 2024-03-30T00:42:59.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 -- 172.21.15.66:0/1509100637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbae40cb570 msgr2=0x7fbae40b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbae40cb570 0x7fbae40b54e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c480 tx=0x7fbad800c940 comp rx=0 tx=0).stop 2024-03-30T00:42:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 -- 172.21.15.66:0/1509100637 shutdown_connections 2024-03-30T00:42:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbae415ca80 0x7fbae41f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbae40cf2f0 0x7fbae40ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbac4066690 0x7fbac4068b50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 --2- 172.21.15.66:0/1509100637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbae40cb570 0x7fbae40b54e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 -- 172.21.15.66:0/1509100637 >> 172.21.15.66:0/1509100637 conn(0x7fbae40b2090 msgr2=0x7fbae406f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 -- 172.21.15.66:0/1509100637 shutdown_connections 2024-03-30T00:42:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.608+0000 7fbaeba00640 1 -- 172.21.15.66:0/1509100637 wait complete. 2024-03-30T00:42:59.621 INFO:tasks.ceph.mds.c.smithi066.stderr: -4853> 2024-03-30T00:42:09.784+0000 7f94a6800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:42:59.624 INFO:tasks.ceph.mds.c.smithi066.stderr: -3062> 2024-03-30T00:42:22.100+0000 7f94a2200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6162=0-4194304@1} caps={6162=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6162 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x557f2ea86100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:42:59.635 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T00:42:59.669 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:42:59.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f25c00640 1 Processor -- start 2024-03-30T00:42:59.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f25c00640 1 -- start start 2024-03-30T00:42:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f25c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f200cda90 0x7f5f200ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f25c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f200cde70 0x7f5f200cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f25c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5f20159c60 0x7f5f2015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f25c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f200b1b80 con 0x7f5f200cde70 2024-03-30T00:42:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f25c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2015a990 con 0x7f5f20159c60 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f25c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200b11e0 con 0x7f5f200cda90 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f16000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f200cde70 0x7f5f200cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f16000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f200cde70 0x7f5f200cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34572/0 (socket says 172.21.15.66:34572) 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f16000640 1 -- 172.21.15.66:0/2983929959 learned_addr learned my addr 172.21.15.66:0/2983929959 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f1f400640 1 --2- 172.21.15.66:0/2983929959 >> v2:172.21.15.156:3300/0 conn(0x7f5f20159c60 0x7f5f2015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f1ea00640 1 --2- 172.21.15.66:0/2983929959 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f200cda90 0x7f5f200ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f16000640 1 -- 172.21.15.66:0/2983929959 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f200cda90 msgr2=0x7f5f200ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f16000640 1 --2- 172.21.15.66:0/2983929959 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f200cda90 0x7f5f200ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f16000640 1 -- 172.21.15.66:0/2983929959 >> v2:172.21.15.156:3300/0 conn(0x7f5f20159c60 msgr2=0x7f5f2015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f16000640 1 --2- 172.21.15.66:0/2983929959 >> v2:172.21.15.156:3300/0 conn(0x7f5f20159c60 0x7f5f2015a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.772+0000 7f5f16000640 1 -- 172.21.15.66:0/2983929959 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f200b14c0 con 0x7f5f200cde70 2024-03-30T00:42:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f1f400640 1 --2- 172.21.15.66:0/2983929959 >> v2:172.21.15.156:3300/0 conn(0x7f5f20159c60 0x7f5f2015a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:42:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f16000640 1 --2- 172.21.15.66:0/2983929959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f200cde70 0x7f5f200cea60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5f200724d0 tx=0x7f5f0801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2eb26d8acdd5c82 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f1e000640 1 -- 172.21.15.66:0/2983929959 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080ab670 con 0x7f5f200cde70 2024-03-30T00:42:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f1e000640 1 -- 172.21.15.66:0/2983929959 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f080ab7d0 con 0x7f5f200cde70 2024-03-30T00:42:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f1e000640 1 -- 172.21.15.66:0/2983929959 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080abb60 con 0x7f5f200cde70 2024-03-30T00:42:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f25c00640 1 -- 172.21.15.66:0/2983929959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f200cde70 msgr2=0x7f5f200cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f25c00640 1 --2- 172.21.15.66:0/2983929959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f200cde70 0x7f5f200cea60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5f200724d0 tx=0x7f5f0801c490 comp rx=0 tx=0).stop 2024-03-30T00:42:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f25c00640 1 -- 172.21.15.66:0/2983929959 shutdown_connections 2024-03-30T00:42:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f25c00640 1 --2- 172.21.15.66:0/2983929959 >> v2:172.21.15.156:3300/0 conn(0x7f5f20159c60 0x7f5f2015a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f25c00640 1 --2- 172.21.15.66:0/2983929959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f200cde70 0x7f5f200cea60 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f25c00640 1 --2- 172.21.15.66:0/2983929959 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f200cda90 0x7f5f200ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f25c00640 1 -- 172.21.15.66:0/2983929959 >> 172.21.15.66:0/2983929959 conn(0x7f5f20069f60 msgr2=0x7f5f2006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:42:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f25c00640 1 -- 172.21.15.66:0/2983929959 shutdown_connections 2024-03-30T00:42:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.776+0000 7f5f25c00640 1 -- 172.21.15.66:0/2983929959 wait complete. 2024-03-30T00:42:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f25c00640 1 Processor -- start 2024-03-30T00:42:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f25c00640 1 -- start start 2024-03-30T00:42:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f25c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f200cda90 0x7f5f20159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f25c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f200cde70 0x7f5f201561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f25c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f20159c60 0x7f5f20070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f25c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f20159880 con 0x7f5f200cde70 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f25c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f20156700 con 0x7f5f200cda90 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f25c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200ad720 con 0x7f5f20159c60 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f16000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f200cde70 0x7f5f201561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f16000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f200cde70 0x7f5f201561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34586/0 (socket says 172.21.15.66:34586) 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f16000640 1 -- 172.21.15.66:0/1071879485 learned_addr learned my addr 172.21.15.66:0/1071879485 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f1f400640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f20159c60 0x7f5f20070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f1ea00640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f200cda90 0x7f5f20159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f16000640 1 -- 172.21.15.66:0/1071879485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f20159c60 msgr2=0x7f5f20070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f16000640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f20159c60 0x7f5f20070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f16000640 1 -- 172.21.15.66:0/1071879485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f200cda90 msgr2=0x7f5f20159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:42:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f16000640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f200cda90 0x7f5f20159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:42:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f16000640 1 -- 172.21.15.66:0/1071879485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f080094d0 con 0x7f5f200cde70 2024-03-30T00:42:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f16000640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f200cde70 0x7f5f201561c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5f080ad040 tx=0x7f5f0801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f17400640 1 -- 172.21.15.66:0/1071879485 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080b4800 con 0x7f5f200cde70 2024-03-30T00:42:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f17400640 1 -- 172.21.15.66:0/1071879485 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f080b4960 con 0x7f5f200cde70 2024-03-30T00:42:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f25c00640 1 -- 172.21.15.66:0/1071879485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f200ada00 con 0x7f5f200cde70 2024-03-30T00:42:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f17400640 1 -- 172.21.15.66:0/1071879485 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080b4d10 con 0x7f5f200cde70 2024-03-30T00:42:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.780+0000 7f5f25c00640 1 -- 172.21.15.66:0/1071879485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f20163ec0 con 0x7f5f200cde70 2024-03-30T00:42:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.784+0000 7f5f25c00640 1 -- 172.21.15.66:0/1071879485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f201576a0 con 0x7f5f200cde70 2024-03-30T00:42:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.784+0000 7f5f17400640 1 -- 172.21.15.66:0/1071879485 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5f080b3380 con 0x7f5f200cde70 2024-03-30T00:42:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.784+0000 7f5f17400640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5efc066690 0x7f5efc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:42:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.784+0000 7f5f17400640 1 -- 172.21.15.66:0/1071879485 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f5f0811e940 con 0x7f5f200cde70 2024-03-30T00:42:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.784+0000 7f5f1ea00640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5efc066690 0x7f5efc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:42:59.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.784+0000 7f5f1ea00640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5efc066690 0x7f5efc068b50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5f100039f0 tx=0x7f5f10009790 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:42:59.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.788+0000 7f5f17400640 1 -- 172.21.15.66:0/1071879485 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5f080ebcb0 con 0x7f5f200cde70 2024-03-30T00:42:59.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:42:59.984+0000 7f5f25c00640 1 -- 172.21.15.66:0/1071879485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5f20066c30 con 0x7f5f200cde70 2024-03-30T00:43:00.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.636+0000 7f5f17400640 1 -- 172.21.15.66:0/1071879485 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5f080ef2d0 con 0x7f5f200cde70 2024-03-30T00:43:00.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.636+0000 7f5f25c00640 1 -- 172.21.15.66:0/1071879485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5efc066690 msgr2=0x7f5efc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:00.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.636+0000 7f5f25c00640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5efc066690 0x7f5efc068b50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5f100039f0 tx=0x7f5f10009790 comp rx=0 tx=0).stop 2024-03-30T00:43:00.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.636+0000 7f5f25c00640 1 -- 172.21.15.66:0/1071879485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f200cde70 msgr2=0x7f5f201561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:00.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.636+0000 7f5f25c00640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f200cde70 0x7f5f201561c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5f080ad040 tx=0x7f5f0801acd0 comp rx=0 tx=0).stop 2024-03-30T00:43:00.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.636+0000 7f5f25c00640 1 -- 172.21.15.66:0/1071879485 shutdown_connections 2024-03-30T00:43:00.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.636+0000 7f5f25c00640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f20159c60 0x7f5f20070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:00.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.636+0000 7f5f25c00640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5efc066690 0x7f5efc068b50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:00.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.636+0000 7f5f25c00640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f200cde70 0x7f5f201561c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:00.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.636+0000 7f5f25c00640 1 --2- 172.21.15.66:0/1071879485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f200cda90 0x7f5f20159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:00.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.636+0000 7f5f25c00640 1 -- 172.21.15.66:0/1071879485 >> 172.21.15.66:0/1071879485 conn(0x7f5f20069f60 msgr2=0x7f5f200cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:00.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.640+0000 7f5f25c00640 1 -- 172.21.15.66:0/1071879485 shutdown_connections 2024-03-30T00:43:00.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.640+0000 7f5f25c00640 1 -- 172.21.15.66:0/1071879485 wait complete. 2024-03-30T00:43:00.668 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T00:43:00.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4ff0800640 1 Processor -- start 2024-03-30T00:43:00.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4ff0800640 1 -- start start 2024-03-30T00:43:00.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4ff0800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4fe80acaf0 0x7f4fe80b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:00.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4ff0800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4fe815ba00 0x7f4fe80ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:00.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4ff0800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe80cbf80 con 0x7f4fe80cecf0 2024-03-30T00:43:00.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4ff0800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe80cc820 con 0x7f4fe80acaf0 2024-03-30T00:43:00.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4ff0800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe80cf110 con 0x7f4fe815ba00 2024-03-30T00:43:00.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4fede00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4fe80acaf0 0x7f4fe80b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:00.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4fede00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4fe80acaf0 0x7f4fe80b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34602/0 (socket says 172.21.15.66:34602) 2024-03-30T00:43:00.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4fede00640 1 -- 172.21.15.66:0/1723985151 learned_addr learned my addr 172.21.15.66:0/1723985151 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:00.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4fed400640 1 --2- 172.21.15.66:0/1723985151 >> v2:172.21.15.156:3300/0 conn(0x7f4fe815ba00 0x7f4fe80ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:00.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4fede00640 1 -- 172.21.15.66:0/1723985151 >> v2:172.21.15.156:3300/0 conn(0x7f4fe815ba00 msgr2=0x7f4fe80ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:00.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4fede00640 1 --2- 172.21.15.66:0/1723985151 >> v2:172.21.15.156:3300/0 conn(0x7f4fe815ba00 0x7f4fe80ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:00.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4fede00640 1 -- 172.21.15.66:0/1723985151 >> v1:172.21.15.66:6789/0 conn(0x7f4fe80cecf0 legacy=0x7f4fe81579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:00.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.856+0000 7f4fede00640 1 -- 172.21.15.66:0/1723985151 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80b2dd0 con 0x7f4fe80acaf0 2024-03-30T00:43:00.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4fede00640 1 --2- 172.21.15.66:0/1723985151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe80acaf0 0x7f4fe80b24d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80c1e10 tx=0x7f4fdc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3912f778b41f31d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:00.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4feca00640 1 -- 172.21.15.66:0/1723985151 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0ab660 con 0x7f4fe80acaf0 2024-03-30T00:43:00.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4feca00640 1 -- 172.21.15.66:0/1723985151 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc0ab7c0 con 0x7f4fe80acaf0 2024-03-30T00:43:00.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4feca00640 1 -- 172.21.15.66:0/1723985151 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0abb50 con 0x7f4fe80acaf0 2024-03-30T00:43:00.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4ff0800640 1 -- 172.21.15.66:0/1723985151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe80acaf0 msgr2=0x7f4fe80b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:00.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4ff0800640 1 --2- 172.21.15.66:0/1723985151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe80acaf0 0x7f4fe80b24d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80c1e10 tx=0x7f4fdc01c460 comp rx=0 tx=0).stop 2024-03-30T00:43:00.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4ff0800640 1 -- 172.21.15.66:0/1723985151 shutdown_connections 2024-03-30T00:43:00.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4ff0800640 1 --2- 172.21.15.66:0/1723985151 >> v2:172.21.15.156:3300/0 conn(0x7f4fe815ba00 0x7f4fe80ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4ff0800640 1 --2- 172.21.15.66:0/1723985151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe80acaf0 0x7f4fe80b24d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4ff0800640 1 -- 172.21.15.66:0/1723985151 >> 172.21.15.66:0/1723985151 conn(0x7f4fe806a190 msgr2=0x7f4fe806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4ff0800640 1 -- 172.21.15.66:0/1723985151 shutdown_connections 2024-03-30T00:43:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4ff0800640 1 -- 172.21.15.66:0/1723985151 wait complete. 2024-03-30T00:43:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4ff0800640 1 Processor -- start 2024-03-30T00:43:00.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.860+0000 7f4ff0800640 1 -- start start 2024-03-30T00:43:00.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4ff0800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fe80acaf0 0x7f4fe8153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:00.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4ff0800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe80cecf0 0x7f4fe8097830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:00.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4ff0800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe815ba00 0x7f4fe80ab550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:00.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4ff0800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80c2870 con 0x7f4fe815ba00 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4ff0800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80c1f80 con 0x7f4fe80cecf0 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4ff0800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80c2220 con 0x7f4fe80acaf0 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fee800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe815ba00 0x7f4fe80ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fede00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fe80acaf0 0x7f4fe8153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fed400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe80cecf0 0x7f4fe8097830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fede00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fe80acaf0 0x7f4fe8153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:35446/0 (socket says 172.21.15.66:35446) 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fede00640 1 -- 172.21.15.66:0/1937746444 learned_addr learned my addr 172.21.15.66:0/1937746444 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fee800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe815ba00 0x7f4fe80ab550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34606/0 (socket says 172.21.15.66:34606) 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fee800640 1 -- 172.21.15.66:0/1937746444 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fe80acaf0 msgr2=0x7f4fe8153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fee800640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fe80acaf0 0x7f4fe8153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fee800640 1 -- 172.21.15.66:0/1937746444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe80cecf0 msgr2=0x7f4fe8097830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fee800640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe80cecf0 0x7f4fe8097830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:00.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fee800640 1 -- 172.21.15.66:0/1937746444 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc0094d0 con 0x7f4fe815ba00 2024-03-30T00:43:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fede00640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fe80acaf0 0x7f4fe8153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fee800640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe815ba00 0x7f4fe80ab550 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4001dc0 tx=0x7f4fe4010420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fe2000640 1 -- 172.21.15.66:0/1937746444 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4013070 con 0x7f4fe815ba00 2024-03-30T00:43:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fe2000640 1 -- 172.21.15.66:0/1937746444 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe400e2e0 con 0x7f4fe815ba00 2024-03-30T00:43:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4ff0800640 1 -- 172.21.15.66:0/1937746444 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe80c2500 con 0x7f4fe815ba00 2024-03-30T00:43:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4fe2000640 1 -- 172.21.15.66:0/1937746444 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40020d0 con 0x7f4fe815ba00 2024-03-30T00:43:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4ff0800640 1 -- 172.21.15.66:0/1937746444 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe8154470 con 0x7f4fe815ba00 2024-03-30T00:43:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.864+0000 7f4ff0800640 1 -- 172.21.15.66:0/1937746444 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe8066c30 con 0x7f4fe815ba00 2024-03-30T00:43:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.868+0000 7f4fe2000640 1 -- 172.21.15.66:0/1937746444 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4fe4010b30 con 0x7f4fe815ba00 2024-03-30T00:43:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.868+0000 7f4fe2000640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4fcc066690 0x7f4fcc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.868+0000 7f4fe2000640 1 -- 172.21.15.66:0/1937746444 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f4fe4087d10 con 0x7f4fe815ba00 2024-03-30T00:43:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.868+0000 7f4fede00640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4fcc066690 0x7f4fcc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.868+0000 7f4fede00640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4fcc066690 0x7f4fcc068b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc01c4a0 tx=0x7f4fdc0b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:00.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:00.872+0000 7f4fe2000640 1 -- 172.21.15.66:0/1937746444 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4fe408d020 con 0x7f4fe815ba00 2024-03-30T00:43:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.036+0000 7f4ff0800640 1 -- 172.21.15.66:0/1937746444 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4fe80c29b0 con 0x7f4fe815ba00 2024-03-30T00:43:01.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.644+0000 7f4fe2000640 1 -- 172.21.15.66:0/1937746444 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4fe4054ec0 con 0x7f4fe815ba00 2024-03-30T00:43:01.651 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-03-30T00:43:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 -- 172.21.15.66:0/1937746444 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4fcc066690 msgr2=0x7f4fcc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4fcc066690 0x7f4fcc068b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc01c4a0 tx=0x7f4fdc0b6040 comp rx=0 tx=0).stop 2024-03-30T00:43:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 -- 172.21.15.66:0/1937746444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe815ba00 msgr2=0x7f4fe80ab550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe815ba00 0x7f4fe80ab550 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4001dc0 tx=0x7f4fe4010420 comp rx=0 tx=0).stop 2024-03-30T00:43:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 -- 172.21.15.66:0/1937746444 shutdown_connections 2024-03-30T00:43:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4fcc066690 0x7f4fcc068b50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe815ba00 0x7f4fe80ab550 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe80cecf0 0x7f4fe8097830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:01.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 --2- 172.21.15.66:0/1937746444 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fe80acaf0 0x7f4fe8153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:01.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 -- 172.21.15.66:0/1937746444 >> 172.21.15.66:0/1937746444 conn(0x7f4fe806a190 msgr2=0x7f4fe80b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:01.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 -- 172.21.15.66:0/1937746444 shutdown_connections 2024-03-30T00:43:01.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.648+0000 7f4ff0800640 1 -- 172.21.15.66:0/1937746444 wait complete. 2024-03-30T00:43:01.677 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T00:43:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.824+0000 7ff858400640 1 Processor -- start 2024-03-30T00:43:01.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.824+0000 7ff858400640 1 -- start start 2024-03-30T00:43:01.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.824+0000 7ff858400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff8500b1e80 0x7ff850159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.824+0000 7ff858400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8500b13c0 con 0x7ff8501594f0 2024-03-30T00:43:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.824+0000 7ff858400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85015a810 con 0x7ff8500ceec0 2024-03-30T00:43:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.824+0000 7ff858400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8500b1a60 con 0x7ff8500b1e80 2024-03-30T00:43:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.824+0000 7ff855a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff8501594f0 0x7ff8500ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57220/0 (socket says 172.21.15.66:57220) 2024-03-30T00:43:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.824+0000 7ff855a00640 1 -- 172.21.15.66:0/2179286665 learned_addr learned my addr 172.21.15.66:0/2179286665 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.824+0000 7ff856400640 1 --2- 172.21.15.66:0/2179286665 >> v2:172.21.15.156:3300/0 conn(0x7ff8500b1e80 0x7ff850159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.824+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306227389 0 0) 0x7ff8500b13c0 con 0x7ff8501594f0 2024-03-30T00:43:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.824+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff838003530 con 0x7ff8501594f0 2024-03-30T00:43:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 89081731 0 0) 0x7ff838003530 con 0x7ff8501594f0 2024-03-30T00:43:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 >> v2:172.21.15.156:3300/0 conn(0x7ff8500b1e80 msgr2=0x7ff850159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:01.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 --2- 172.21.15.66:0/2179286665 >> v2:172.21.15.156:3300/0 conn(0x7ff8500b1e80 0x7ff850159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 >> v1:172.21.15.156:6789/0 conn(0x7ff8500ceec0 legacy=0x7ff8501598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8500b1680 con 0x7ff8501594f0 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7ff844001680 con 0x7ff8501594f0 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 >> v1:172.21.15.66:6789/0 conn(0x7ff8501594f0 legacy=0x7ff8500ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 --2- 172.21.15.66:0/2179286665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff838003980 0x7ff838003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 --2- 172.21.15.66:0/2179286665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff838004380 0x7ff838004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 --2- 172.21.15.66:0/2179286665 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff838005050 0x7ff83800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b1a60 con 0x7ff838004380 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83800e1c0 con 0x7ff838003980 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83800e380 con 0x7ff838005050 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff855000640 1 --2- 172.21.15.66:0/2179286665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff838004380 0x7ff838004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:01.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff855a00640 1 --2- 172.21.15.66:0/2179286665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff838003980 0x7ff838003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:01.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff856400640 1 --2- 172.21.15.66:0/2179286665 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff838005050 0x7ff83800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:01.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff855a00640 1 -- 172.21.15.66:0/2179286665 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff838005050 msgr2=0x7ff83800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:01.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff855a00640 1 --2- 172.21.15.66:0/2179286665 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff838005050 0x7ff83800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:01.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff855a00640 1 -- 172.21.15.66:0/2179286665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff838004380 msgr2=0x7ff838004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:01.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff855a00640 1 --2- 172.21.15.66:0/2179286665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff838004380 0x7ff838004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:01.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff855a00640 1 -- 172.21.15.66:0/2179286665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff838003530 con 0x7ff838003980 2024-03-30T00:43:01.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff856400640 1 -- 172.21.15.66:0/2179286665 reap_dead start 2024-03-30T00:43:01.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.828+0000 7ff855a00640 1 --2- 172.21.15.66:0/2179286665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff838003980 0x7ff838003d60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff84400c190 tx=0x7ff84401cb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c558a036659d4c86 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:01.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844005ae0 con 0x7ff838003980 2024-03-30T00:43:01.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff844005c60 con 0x7ff838003980 2024-03-30T00:43:01.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff84fe00640 1 -- 172.21.15.66:0/2179286665 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8440a5ac0 con 0x7ff838003980 2024-03-30T00:43:01.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 -- 172.21.15.66:0/2179286665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff838003980 msgr2=0x7ff838003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:01.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 --2- 172.21.15.66:0/2179286665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff838003980 0x7ff838003d60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff84400c190 tx=0x7ff84401cb80 comp rx=0 tx=0).stop 2024-03-30T00:43:01.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 -- 172.21.15.66:0/2179286665 shutdown_connections 2024-03-30T00:43:01.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 --2- 172.21.15.66:0/2179286665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff838003980 0x7ff838003d60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:01.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 -- 172.21.15.66:0/2179286665 >> 172.21.15.66:0/2179286665 conn(0x7ff85006a190 msgr2=0x7ff85006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:01.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 -- 172.21.15.66:0/2179286665 shutdown_connections 2024-03-30T00:43:01.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 -- 172.21.15.66:0/2179286665 wait complete. 2024-03-30T00:43:01.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 Processor -- start 2024-03-30T00:43:01.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 -- start start 2024-03-30T00:43:01.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff838003980 0x7ff850071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8501569c0 0x7ff8501f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8500ac270 0x7ff8500ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b2280 con 0x7ff8500ac270 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500acc30 con 0x7ff838003980 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff858400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff850157b90 con 0x7ff8501569c0 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff856400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8500ac270 0x7ff8500ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff856400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8500ac270 0x7ff8500ac6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34632/0 (socket says 172.21.15.66:34632) 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.832+0000 7ff856400640 1 -- 172.21.15.66:0/2254258534 learned_addr learned my addr 172.21.15.66:0/2254258534 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff855a00640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff838003980 0x7ff850071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff855000640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8501569c0 0x7ff8501f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff856400640 1 -- 172.21.15.66:0/2254258534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8501569c0 msgr2=0x7ff8501f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:01.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff856400640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8501569c0 0x7ff8501f1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:01.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff856400640 1 -- 172.21.15.66:0/2254258534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff838003980 msgr2=0x7ff850071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:01.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff856400640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff838003980 0x7ff850071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:01.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff856400640 1 -- 172.21.15.66:0/2254258534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff850156f80 con 0x7ff8500ac270 2024-03-30T00:43:01.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff856400640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8500ac270 0x7ff8500ac6f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff8500ade60 tx=0x7ff84800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:01.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff84d600640 1 -- 172.21.15.66:0/2254258534 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84800e040 con 0x7ff8500ac270 2024-03-30T00:43:01.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff84d600640 1 -- 172.21.15.66:0/2254258534 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84800f040 con 0x7ff8500ac270 2024-03-30T00:43:01.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff858400640 1 -- 172.21.15.66:0/2254258534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff850157260 con 0x7ff8500ac270 2024-03-30T00:43:01.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff84d600640 1 -- 172.21.15.66:0/2254258534 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848015430 con 0x7ff8500ac270 2024-03-30T00:43:01.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff858400640 1 -- 172.21.15.66:0/2254258534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff850157820 con 0x7ff8500ac270 2024-03-30T00:43:01.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff858400640 1 -- 172.21.15.66:0/2254258534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8500c3900 con 0x7ff8500ac270 2024-03-30T00:43:01.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff84d600640 1 -- 172.21.15.66:0/2254258534 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff848015590 con 0x7ff8500ac270 2024-03-30T00:43:01.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff84d600640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff830066690 0x7ff830068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:01.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff84d600640 1 -- 172.21.15.66:0/2254258534 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5735+0+0 (secure 0 0 0) 0x7ff848084910 con 0x7ff8500ac270 2024-03-30T00:43:01.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.836+0000 7ff855a00640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff830066690 0x7ff830068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:01.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.840+0000 7ff855a00640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff830066690 0x7ff830068b50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff8440ab000 tx=0x7ff8440b1040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:01.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:01.840+0000 7ff84d600640 1 -- 172.21.15.66:0/2254258534 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff848051bc0 con 0x7ff8500ac270 2024-03-30T00:43:02.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.004+0000 7ff858400640 1 -- 172.21.15.66:0/2254258534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff850066c30 con 0x7ff8500ac270 2024-03-30T00:43:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.648+0000 7ff84d600640 1 -- 172.21.15.66:0/2254258534 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff8480551e0 con 0x7ff8500ac270 2024-03-30T00:43:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-03-30T00:43:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 -- 172.21.15.66:0/2254258534 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff830066690 msgr2=0x7ff830068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff830066690 0x7ff830068b50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff8440ab000 tx=0x7ff8440b1040 comp rx=0 tx=0).stop 2024-03-30T00:43:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 -- 172.21.15.66:0/2254258534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8500ac270 msgr2=0x7ff8500ac6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8500ac270 0x7ff8500ac6f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff8500ade60 tx=0x7ff84800f580 comp rx=0 tx=0).stop 2024-03-30T00:43:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 -- 172.21.15.66:0/2254258534 shutdown_connections 2024-03-30T00:43:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff830066690 0x7ff830068b50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8500ac270 0x7ff8500ac6f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8501569c0 0x7ff8501f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 --2- 172.21.15.66:0/2254258534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff838003980 0x7ff850071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:02.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 -- 172.21.15.66:0/2254258534 >> 172.21.15.66:0/2254258534 conn(0x7ff85006a190 msgr2=0x7ff8500c8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:02.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 -- 172.21.15.66:0/2254258534 shutdown_connections 2024-03-30T00:43:02.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.652+0000 7ff858400640 1 -- 172.21.15.66:0/2254258534 wait complete. 2024-03-30T00:43:02.685 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-30T00:43:02.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.824+0000 7ff736a00640 1 Processor -- start 2024-03-30T00:43:02.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.824+0000 7ff736a00640 1 -- start start 2024-03-30T00:43:02.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.824+0000 7ff736a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7301610c0 0x7ff7301604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:02.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.824+0000 7ff736a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73006da20 con 0x7ff7301609f0 2024-03-30T00:43:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.824+0000 7ff736a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73006dc60 con 0x7ff730157060 2024-03-30T00:43:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.824+0000 7ff736a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff730157440 con 0x7ff7301610c0 2024-03-30T00:43:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.828+0000 7ff727400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7301610c0 0x7ff7301604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.828+0000 7ff727400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7301610c0 0x7ff7301604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34638/0 (socket says 172.21.15.66:34638) 2024-03-30T00:43:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.828+0000 7ff734a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff7301609f0 0x7ff7300abc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57228/0 (socket says 172.21.15.66:57228) 2024-03-30T00:43:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.828+0000 7ff727400640 1 -- 172.21.15.66:0/4011992471 learned_addr learned my addr 172.21.15.66:0/4011992471 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.828+0000 7ff727400640 1 -- 172.21.15.66:0/4011992471 >> v1:172.21.15.156:6789/0 conn(0x7ff730157060 legacy=0x7ff73006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.828+0000 7ff727400640 1 -- 172.21.15.66:0/4011992471 >> v1:172.21.15.66:6789/0 conn(0x7ff7301609f0 legacy=0x7ff7300abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:02.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.828+0000 7ff727400640 1 -- 172.21.15.66:0/4011992471 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73006df40 con 0x7ff7301610c0 2024-03-30T00:43:02.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.828+0000 7ff727400640 1 --2- 172.21.15.66:0/4011992471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7301610c0 0x7ff7301604b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff718009870 tx=0x7ff71801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3b42a4d03c75d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:02.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.828+0000 7ff72f400640 1 -- 172.21.15.66:0/4011992471 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7180ab600 con 0x7ff7301610c0 2024-03-30T00:43:02.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.828+0000 7ff72f400640 1 -- 172.21.15.66:0/4011992471 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7180a7050 con 0x7ff7301610c0 2024-03-30T00:43:02.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.828+0000 7ff72f400640 1 -- 172.21.15.66:0/4011992471 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7180ab980 con 0x7ff7301610c0 2024-03-30T00:43:02.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 -- 172.21.15.66:0/4011992471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7301610c0 msgr2=0x7ff7301604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:02.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 --2- 172.21.15.66:0/4011992471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7301610c0 0x7ff7301604b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff718009870 tx=0x7ff71801c400 comp rx=0 tx=0).stop 2024-03-30T00:43:02.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 -- 172.21.15.66:0/4011992471 shutdown_connections 2024-03-30T00:43:02.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 --2- 172.21.15.66:0/4011992471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7301610c0 0x7ff7301604b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:02.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 -- 172.21.15.66:0/4011992471 >> 172.21.15.66:0/4011992471 conn(0x7ff730071750 msgr2=0x7ff730069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:02.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 -- 172.21.15.66:0/4011992471 shutdown_connections 2024-03-30T00:43:02.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 -- 172.21.15.66:0/4011992471 wait complete. 2024-03-30T00:43:02.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 Processor -- start 2024-03-30T00:43:02.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 -- start start 2024-03-30T00:43:02.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff730157060 0x7ff73015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:02.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7301609f0 0x7ff73014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:02.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7301610c0 0x7ff73014dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:02.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73014d910 con 0x7ff730157060 2024-03-30T00:43:02.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73015eea0 con 0x7ff7301610c0 2024-03-30T00:43:02.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff736a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73015f180 con 0x7ff7301609f0 2024-03-30T00:43:02.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff72fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff730157060 0x7ff73015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:02.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff72fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff730157060 0x7ff73015c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34652/0 (socket says 172.21.15.66:34652) 2024-03-30T00:43:02.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff72fe00640 1 -- 172.21.15.66:0/3493394619 learned_addr learned my addr 172.21.15.66:0/3493394619 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:02.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff734a00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7301610c0 0x7ff73014dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:02.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff727400640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7301609f0 0x7ff73014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff72fe00640 1 -- 172.21.15.66:0/3493394619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7301609f0 msgr2=0x7ff73014e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff72fe00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7301609f0 0x7ff73014e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff72fe00640 1 -- 172.21.15.66:0/3493394619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7301610c0 msgr2=0x7ff73014dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff72fe00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7301610c0 0x7ff73014dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff72fe00640 1 -- 172.21.15.66:0/3493394619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7180094d0 con 0x7ff730157060 2024-03-30T00:43:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff734a00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7301610c0 0x7ff73014dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff727400640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7301609f0 0x7ff73014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.832+0000 7ff72fe00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff730157060 0x7ff73015c1a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff72000b410 tx=0x7ff72000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.836+0000 7ff72cc00640 1 -- 172.21.15.66:0/3493394619 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720004020 con 0x7ff730157060 2024-03-30T00:43:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.836+0000 7ff72cc00640 1 -- 172.21.15.66:0/3493394619 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff720004180 con 0x7ff730157060 2024-03-30T00:43:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.836+0000 7ff72cc00640 1 -- 172.21.15.66:0/3493394619 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720011630 con 0x7ff730157060 2024-03-30T00:43:02.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.836+0000 7ff736a00640 1 -- 172.21.15.66:0/3493394619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff730151f10 con 0x7ff730157060 2024-03-30T00:43:02.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.836+0000 7ff736a00640 1 -- 172.21.15.66:0/3493394619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7301514a0 con 0x7ff730157060 2024-03-30T00:43:02.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.836+0000 7ff736a00640 1 -- 172.21.15.66:0/3493394619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73014da80 con 0x7ff730157060 2024-03-30T00:43:02.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.836+0000 7ff72cc00640 1 -- 172.21.15.66:0/3493394619 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff720011790 con 0x7ff730157060 2024-03-30T00:43:02.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.836+0000 7ff72cc00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff70c066690 0x7ff70c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:02.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.836+0000 7ff72cc00640 1 -- 172.21.15.66:0/3493394619 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff720083de0 con 0x7ff730157060 2024-03-30T00:43:02.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.836+0000 7ff727400640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff70c066690 0x7ff70c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:02.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.836+0000 7ff727400640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff70c066690 0x7ff70c068b50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff7180ad000 tx=0x7ff718008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:02.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:02.840+0000 7ff72cc00640 1 -- 172.21.15.66:0/3493394619 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff720051270 con 0x7ff730157060 2024-03-30T00:43:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.004+0000 7ff736a00640 1 -- 172.21.15.66:0/3493394619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff730066c30 con 0x7ff730157060 2024-03-30T00:43:03.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff72cc00640 1 -- 172.21.15.66:0/3493394619 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff720054890 con 0x7ff730157060 2024-03-30T00:43:03.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 -- 172.21.15.66:0/3493394619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff70c066690 msgr2=0x7ff70c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:03.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff70c066690 0x7ff70c068b50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff7180ad000 tx=0x7ff718008040 comp rx=0 tx=0).stop 2024-03-30T00:43:03.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 -- 172.21.15.66:0/3493394619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff730157060 msgr2=0x7ff73015c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:03.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff730157060 0x7ff73015c1a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff72000b410 tx=0x7ff72000b8d0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 -- 172.21.15.66:0/3493394619 shutdown_connections 2024-03-30T00:43:03.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7301610c0 0x7ff73014dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7301609f0 0x7ff73014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff70c066690 0x7ff70c068b50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 --2- 172.21.15.66:0/3493394619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff730157060 0x7ff73015c1a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 -- 172.21.15.66:0/3493394619 >> 172.21.15.66:0/3493394619 conn(0x7ff730071750 msgr2=0x7ff7300ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:03.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 -- 172.21.15.66:0/3493394619 shutdown_connections 2024-03-30T00:43:03.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.008+0000 7ff736a00640 1 -- 172.21.15.66:0/3493394619 wait complete. 2024-03-30T00:43:03.035 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-03-30T00:43:03.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.168+0000 7f448b800640 1 Processor -- start 2024-03-30T00:43:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.168+0000 7f448b800640 1 -- start start 2024-03-30T00:43:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.168+0000 7f448b800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cda90 0x7f44840ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.168+0000 7f448b800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f44840cde70 0x7f44840cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.168+0000 7f448b800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4484159c60 0x7f448415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.168+0000 7f448b800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44840b1b80 con 0x7f4484159c60 2024-03-30T00:43:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.168+0000 7f448b800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448415a990 con 0x7f44840cde70 2024-03-30T00:43:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.168+0000 7f448b800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44840b11e0 con 0x7f44840cda90 2024-03-30T00:43:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4489800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4484159c60 0x7f448415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4489800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4484159c60 0x7f448415a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34656/0 (socket says 172.21.15.66:34656) 2024-03-30T00:43:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4489800640 1 -- 172.21.15.66:0/2473102956 learned_addr learned my addr 172.21.15.66:0/2473102956 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f447be00640 1 --2- 172.21.15.66:0/2473102956 >> v2:172.21.15.156:3300/0 conn(0x7f44840cde70 0x7f44840cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4488e00640 1 --2- 172.21.15.66:0/2473102956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cda90 0x7f44840ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4489800640 1 -- 172.21.15.66:0/2473102956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cda90 msgr2=0x7f44840ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4489800640 1 --2- 172.21.15.66:0/2473102956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cda90 0x7f44840ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4489800640 1 -- 172.21.15.66:0/2473102956 >> v2:172.21.15.156:3300/0 conn(0x7f44840cde70 msgr2=0x7f44840cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4489800640 1 --2- 172.21.15.66:0/2473102956 >> v2:172.21.15.156:3300/0 conn(0x7f44840cde70 0x7f44840cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4489800640 1 -- 172.21.15.66:0/2473102956 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44840b14c0 con 0x7f4484159c60 2024-03-30T00:43:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4488e00640 1 --2- 172.21.15.66:0/2473102956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cda90 0x7f44840ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4489800640 1 --2- 172.21.15.66:0/2473102956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4484159c60 0x7f448415a0b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f44840cf0d0 tx=0x7f447c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a751776f0f0c5f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4483e00640 1 -- 172.21.15.66:0/2473102956 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0a5800 con 0x7f4484159c60 2024-03-30T00:43:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4483e00640 1 -- 172.21.15.66:0/2473102956 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c0a5960 con 0x7f4484159c60 2024-03-30T00:43:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.172+0000 7f4483e00640 1 -- 172.21.15.66:0/2473102956 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0a5cf0 con 0x7f4484159c60 2024-03-30T00:43:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 -- 172.21.15.66:0/2473102956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4484159c60 msgr2=0x7f448415a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 --2- 172.21.15.66:0/2473102956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4484159c60 0x7f448415a0b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f44840cf0d0 tx=0x7f447c01c960 comp rx=0 tx=0).stop 2024-03-30T00:43:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 -- 172.21.15.66:0/2473102956 shutdown_connections 2024-03-30T00:43:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 --2- 172.21.15.66:0/2473102956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4484159c60 0x7f448415a0b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 --2- 172.21.15.66:0/2473102956 >> v2:172.21.15.156:3300/0 conn(0x7f44840cde70 0x7f44840cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 --2- 172.21.15.66:0/2473102956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cda90 0x7f44840ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 -- 172.21.15.66:0/2473102956 >> 172.21.15.66:0/2473102956 conn(0x7f4484069f60 msgr2=0x7f448406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 -- 172.21.15.66:0/2473102956 shutdown_connections 2024-03-30T00:43:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 -- 172.21.15.66:0/2473102956 wait complete. 2024-03-30T00:43:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 Processor -- start 2024-03-30T00:43:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 -- start start 2024-03-30T00:43:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44840cda90 0x7f4484156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cde70 0x7f448406c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4484159c60 0x7f448406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44840aea00 con 0x7f4484159c60 2024-03-30T00:43:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44840addf0 con 0x7f44840cda90 2024-03-30T00:43:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f448b800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44840ae0d0 con 0x7f44840cde70 2024-03-30T00:43:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f4489800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4484159c60 0x7f448406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f4489800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4484159c60 0x7f448406ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34666/0 (socket says 172.21.15.66:34666) 2024-03-30T00:43:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f4489800640 1 -- 172.21.15.66:0/3914014907 learned_addr learned my addr 172.21.15.66:0/3914014907 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f4488e00640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44840cda90 0x7f4484156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f447be00640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cde70 0x7f448406c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f4489800640 1 -- 172.21.15.66:0/3914014907 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cde70 msgr2=0x7f448406c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f4489800640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cde70 0x7f448406c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f4489800640 1 -- 172.21.15.66:0/3914014907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44840cda90 msgr2=0x7f4484156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f4489800640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44840cda90 0x7f4484156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.176+0000 7f4489800640 1 -- 172.21.15.66:0/3914014907 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c009b30 con 0x7f4484159c60 2024-03-30T00:43:03.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.180+0000 7f4489800640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4484159c60 0x7f448406ca40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f447c0af040 tx=0x7f447c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:03.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.180+0000 7f4481600640 1 -- 172.21.15.66:0/3914014907 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0b8640 con 0x7f4484159c60 2024-03-30T00:43:03.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.180+0000 7f4481600640 1 -- 172.21.15.66:0/3914014907 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c0b87a0 con 0x7f4484159c60 2024-03-30T00:43:03.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.180+0000 7f4481600640 1 -- 172.21.15.66:0/3914014907 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0b8b50 con 0x7f4484159c60 2024-03-30T00:43:03.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.180+0000 7f448b800640 1 -- 172.21.15.66:0/3914014907 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44840ae3b0 con 0x7f4484159c60 2024-03-30T00:43:03.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.180+0000 7f448b800640 1 -- 172.21.15.66:0/3914014907 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4484156690 con 0x7f4484159c60 2024-03-30T00:43:03.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.180+0000 7f448b800640 1 -- 172.21.15.66:0/3914014907 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44840aeb70 con 0x7f4484159c60 2024-03-30T00:43:03.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.180+0000 7f4481600640 1 -- 172.21.15.66:0/3914014907 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f447c0b8cb0 con 0x7f4484159c60 2024-03-30T00:43:03.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.180+0000 7f4481600640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4460066690 0x7f4460068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.180+0000 7f4481600640 1 -- 172.21.15.66:0/3914014907 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f447c11f1a0 con 0x7f4484159c60 2024-03-30T00:43:03.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.180+0000 7f4488e00640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4460066690 0x7f4460068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.184+0000 7f4488e00640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4460066690 0x7f4460068b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f447400aaf0 tx=0x7f4474008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:03.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.184+0000 7f4481600640 1 -- 172.21.15.66:0/3914014907 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f447c0ec810 con 0x7f4484159c60 2024-03-30T00:43:03.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.404+0000 7f448b800640 1 -- 172.21.15.66:0/3914014907 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f4484066c30 con 0x7f4484159c60 2024-03-30T00:43:03.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f4481600640 1 -- 172.21.15.66:0/3914014907 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v188) v1 ==== 149+0+0 (secure 0 0 0) 0x7f447c0efe30 con 0x7f4484159c60 2024-03-30T00:43:03.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f448b800640 1 -- 172.21.15.66:0/3914014907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4460066690 msgr2=0x7f4460068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:03.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f448b800640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4460066690 0x7f4460068b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f447400aaf0 tx=0x7f4474008040 comp rx=0 tx=0).stop 2024-03-30T00:43:03.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f448b800640 1 -- 172.21.15.66:0/3914014907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4484159c60 msgr2=0x7f448406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:03.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f448b800640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4484159c60 0x7f448406ca40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f447c0af040 tx=0x7f447c019700 comp rx=0 tx=0).stop 2024-03-30T00:43:03.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f448b800640 1 -- 172.21.15.66:0/3914014907 shutdown_connections 2024-03-30T00:43:03.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f448b800640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4460066690 0x7f4460068b50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f448b800640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4484159c60 0x7f448406ca40 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f448b800640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44840cde70 0x7f448406c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f448b800640 1 --2- 172.21.15.66:0/3914014907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44840cda90 0x7f4484156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f448b800640 1 -- 172.21.15.66:0/3914014907 >> 172.21.15.66:0/3914014907 conn(0x7f4484069f60 msgr2=0x7f44840af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:03.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.660+0000 7f448b800640 1 -- 172.21.15.66:0/3914014907 shutdown_connections 2024-03-30T00:43:03.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.664+0000 7f448b800640 1 -- 172.21.15.66:0/3914014907 wait complete. 2024-03-30T00:43:03.689 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-03-30T00:43:03.689 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-30T00:43:03.690 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-30T00:43:03.690 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:43:03.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc989800640 1 Processor -- start 2024-03-30T00:43:03.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc989800640 1 -- start start 2024-03-30T00:43:03.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc989800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc9841610c0 0x7fc9841604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc989800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9841609f0 0x7fc9840abc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc989800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98406da20 con 0x7fc984157060 2024-03-30T00:43:03.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc989800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc984157440 con 0x7fc9841610c0 2024-03-30T00:43:03.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc989800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc984160ea0 con 0x7fc9841609f0 2024-03-30T00:43:03.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc97a000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc9841610c0 0x7fc9841604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc983400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9841609f0 0x7fc9840abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc982a00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc984157060 0x7fc98406d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:33302/0 (socket says 172.21.15.66:33302) 2024-03-30T00:43:03.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc97a000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc9841610c0 0x7fc9841604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:46424/0 (socket says 172.21.15.66:46424) 2024-03-30T00:43:03.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.824+0000 7fc982a00640 1 -- 172.21.15.66:0/370820001 learned_addr learned my addr 172.21.15.66:0/370820001 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:03.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc97a000640 1 -- 172.21.15.66:0/370820001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9841609f0 msgr2=0x7fc9840abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:03.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc97a000640 1 --2- 172.21.15.66:0/370820001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9841609f0 0x7fc9840abc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc97a000640 1 -- 172.21.15.66:0/370820001 >> v1:172.21.15.156:6789/0 conn(0x7fc984157060 legacy=0x7fc98406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:03.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc982000640 1 -- 172.21.15.66:0/370820001 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477787399 0 0) 0x7fc974001120 con 0x7fc984157060 2024-03-30T00:43:03.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc97a000640 1 -- 172.21.15.66:0/370820001 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98406dc60 con 0x7fc9841610c0 2024-03-30T00:43:03.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc97a000640 1 --2- 172.21.15.66:0/370820001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9841610c0 0x7fc9841604b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0097c0 tx=0x7fc96c01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dc86d7dabf780a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:03.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc982000640 1 -- 172.21.15.66:0/370820001 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0ab600 con 0x7fc9841610c0 2024-03-30T00:43:03.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc982000640 1 -- 172.21.15.66:0/370820001 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c0a7050 con 0x7fc9841610c0 2024-03-30T00:43:03.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc982000640 1 -- 172.21.15.66:0/370820001 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0ab980 con 0x7fc9841610c0 2024-03-30T00:43:03.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc989800640 1 -- 172.21.15.66:0/370820001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9841610c0 msgr2=0x7fc9841604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:03.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc989800640 1 --2- 172.21.15.66:0/370820001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9841610c0 0x7fc9841604b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0097c0 tx=0x7fc96c01c380 comp rx=0 tx=0).stop 2024-03-30T00:43:03.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc989800640 1 -- 172.21.15.66:0/370820001 shutdown_connections 2024-03-30T00:43:03.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc989800640 1 --2- 172.21.15.66:0/370820001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9841609f0 0x7fc9840abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc989800640 1 --2- 172.21.15.66:0/370820001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9841610c0 0x7fc9841604b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc989800640 1 -- 172.21.15.66:0/370820001 >> 172.21.15.66:0/370820001 conn(0x7fc984071750 msgr2=0x7fc984069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:03.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc989800640 1 -- 172.21.15.66:0/370820001 shutdown_connections 2024-03-30T00:43:03.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.828+0000 7fc989800640 1 -- 172.21.15.66:0/370820001 wait complete. 2024-03-30T00:43:03.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc989800640 1 Processor -- start 2024-03-30T00:43:03.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc989800640 1 -- start start 2024-03-30T00:43:03.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc989800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984157060 0x7fc9840b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc989800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9841609f0 0x7fc9840b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc989800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9841610c0 0x7fc984145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc989800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98415e860 con 0x7fc984157060 2024-03-30T00:43:03.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc989800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9841567b0 con 0x7fc9841609f0 2024-03-30T00:43:03.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc989800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc984156a90 con 0x7fc9841610c0 2024-03-30T00:43:03.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc982a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984157060 0x7fc9840b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc982a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984157060 0x7fc9840b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34674/0 (socket says 172.21.15.66:34674) 2024-03-30T00:43:03.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc982a00640 1 -- 172.21.15.66:0/1093280406 learned_addr learned my addr 172.21.15.66:0/1093280406 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:03.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc97a000640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9841609f0 0x7fc9840b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc983400640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9841610c0 0x7fc984145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc982a00640 1 -- 172.21.15.66:0/1093280406 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9841610c0 msgr2=0x7fc984145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:03.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc982a00640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9841610c0 0x7fc984145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc982a00640 1 -- 172.21.15.66:0/1093280406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9841609f0 msgr2=0x7fc9840b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:03.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc982a00640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9841609f0 0x7fc9840b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:03.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc982a00640 1 -- 172.21.15.66:0/1093280406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c009420 con 0x7fc984157060 2024-03-30T00:43:03.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc982a00640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984157060 0x7fc9840b21e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc97400c480 tx=0x7fc97400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:03.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc97b400640 1 -- 172.21.15.66:0/1093280406 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9740074e0 con 0x7fc984157060 2024-03-30T00:43:03.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc97b400640 1 -- 172.21.15.66:0/1093280406 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc974007660 con 0x7fc984157060 2024-03-30T00:43:03.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc97b400640 1 -- 172.21.15.66:0/1093280406 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974004a30 con 0x7fc984157060 2024-03-30T00:43:03.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc989800640 1 -- 172.21.15.66:0/1093280406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98415ead0 con 0x7fc984157060 2024-03-30T00:43:03.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.832+0000 7fc989800640 1 -- 172.21.15.66:0/1093280406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc984156d70 con 0x7fc984157060 2024-03-30T00:43:03.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.836+0000 7fc989800640 1 -- 172.21.15.66:0/1093280406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc984066c30 con 0x7fc984157060 2024-03-30T00:43:03.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.836+0000 7fc97b400640 1 -- 172.21.15.66:0/1093280406 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc974004b90 con 0x7fc984157060 2024-03-30T00:43:03.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.836+0000 7fc97b400640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc960066690 0x7fc960068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:03.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.836+0000 7fc97b400640 1 -- 172.21.15.66:0/1093280406 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fc974082fe0 con 0x7fc984157060 2024-03-30T00:43:03.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.836+0000 7fc97a000640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc960066690 0x7fc960068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:03.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.836+0000 7fc97a000640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc960066690 0x7fc960068b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0ad000 tx=0x7fc96c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:03.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:03.840+0000 7fc97b400640 1 -- 172.21.15.66:0/1093280406 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc974053fd0 con 0x7fc984157060 2024-03-30T00:43:04.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.004+0000 7fc989800640 1 -- 172.21.15.66:0/1093280406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc98406da20 con 0x7fc984157060 2024-03-30T00:43:04.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc97b400640 1 -- 172.21.15.66:0/1093280406 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+12723 (secure 0 0 0) 0x7fc98406da20 con 0x7fc984157060 2024-03-30T00:43:04.013 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:04.013 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":59,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:43:02.644650+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6837/3557086525":"2024-03-31T00:42:59.598807+0000","172.21.15.66:6836/3557086525":"2024-03-31T00:42:59.598807+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:43:04.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 -- 172.21.15.66:0/1093280406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc960066690 msgr2=0x7fc960068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:04.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc960066690 0x7fc960068b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0ad000 tx=0x7fc96c0a6000 comp rx=0 tx=0).stop 2024-03-30T00:43:04.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 -- 172.21.15.66:0/1093280406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984157060 msgr2=0x7fc9840b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:04.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984157060 0x7fc9840b21e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc97400c480 tx=0x7fc97400c940 comp rx=0 tx=0).stop 2024-03-30T00:43:04.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 -- 172.21.15.66:0/1093280406 shutdown_connections 2024-03-30T00:43:04.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9841610c0 0x7fc984145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9841609f0 0x7fc9840b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc960066690 0x7fc960068b50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 --2- 172.21.15.66:0/1093280406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc984157060 0x7fc9840b21e0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 -- 172.21.15.66:0/1093280406 >> 172.21.15.66:0/1093280406 conn(0x7fc984071750 msgr2=0x7fc9840af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:04.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 -- 172.21.15.66:0/1093280406 shutdown_connections 2024-03-30T00:43:04.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.008+0000 7fc989800640 1 -- 172.21.15.66:0/1093280406 wait complete. 2024-03-30T00:43:04.036 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:43:04.037 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:43:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.172+0000 7f1b63c00640 1 Processor -- start 2024-03-30T00:43:04.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.172+0000 7f1b63c00640 1 -- start start 2024-03-30T00:43:04.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b63c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1b5c15a810 0x7f1b5c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b63c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1b5c099430 0x7f1b5c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b63c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0cf0c0 0x7f1b5c0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b63c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c0cf510 con 0x7f1b5c099430 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b63c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c0c4600 con 0x7f1b5c15a810 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b63c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0cc510 con 0x7f1b5c0cf0c0 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b53e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1b5c099430 0x7f1b5c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b53e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1b5c099430 0x7f1b5c0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34682/0 (socket says 172.21.15.66:34682) 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b53e00640 1 -- 172.21.15.66:0/2468892080 learned_addr learned my addr 172.21.15.66:0/2468892080 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b61200640 1 --2- 172.21.15.66:0/2468892080 >> v2:172.21.15.156:3300/0 conn(0x7f1b5c15a810 0x7f1b5c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b61c00640 1 --2- 172.21.15.66:0/2468892080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0cf0c0 0x7f1b5c0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b53e00640 1 -- 172.21.15.66:0/2468892080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0cf0c0 msgr2=0x7f1b5c0cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b53e00640 1 --2- 172.21.15.66:0/2468892080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0cf0c0 0x7f1b5c0cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b53e00640 1 -- 172.21.15.66:0/2468892080 >> v2:172.21.15.156:3300/0 conn(0x7f1b5c15a810 msgr2=0x7f1b5c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b53e00640 1 --2- 172.21.15.66:0/2468892080 >> v2:172.21.15.156:3300/0 conn(0x7f1b5c15a810 0x7f1b5c0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b53e00640 1 -- 172.21.15.66:0/2468892080 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c0998c0 con 0x7f1b5c099430 2024-03-30T00:43:04.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.176+0000 7f1b61200640 1 --2- 172.21.15.66:0/2468892080 >> v2:172.21.15.156:3300/0 conn(0x7f1b5c15a810 0x7f1b5c0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:04.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b53e00640 1 --2- 172.21.15.66:0/2468892080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b5c099430 0x7f1b5c0ceb80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c072da0 tx=0x7f1b4401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d35384aad31064b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:04.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b5be00640 1 -- 172.21.15.66:0/2468892080 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440a89d0 con 0x7f1b5c099430 2024-03-30T00:43:04.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b5be00640 1 -- 172.21.15.66:0/2468892080 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b440a8b30 con 0x7f1b5c099430 2024-03-30T00:43:04.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b5be00640 1 -- 172.21.15.66:0/2468892080 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440b0440 con 0x7f1b5c099430 2024-03-30T00:43:04.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b63c00640 1 -- 172.21.15.66:0/2468892080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b5c099430 msgr2=0x7f1b5c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:04.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b63c00640 1 --2- 172.21.15.66:0/2468892080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b5c099430 0x7f1b5c0ceb80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c072da0 tx=0x7f1b4401ed10 comp rx=0 tx=0).stop 2024-03-30T00:43:04.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b63c00640 1 -- 172.21.15.66:0/2468892080 shutdown_connections 2024-03-30T00:43:04.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b63c00640 1 --2- 172.21.15.66:0/2468892080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0cf0c0 0x7f1b5c0cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b63c00640 1 --2- 172.21.15.66:0/2468892080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b5c099430 0x7f1b5c0ceb80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b63c00640 1 --2- 172.21.15.66:0/2468892080 >> v2:172.21.15.156:3300/0 conn(0x7f1b5c15a810 0x7f1b5c0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b63c00640 1 -- 172.21.15.66:0/2468892080 >> 172.21.15.66:0/2468892080 conn(0x7f1b5c0b2090 msgr2=0x7f1b5c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:04.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b63c00640 1 -- 172.21.15.66:0/2468892080 shutdown_connections 2024-03-30T00:43:04.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b63c00640 1 -- 172.21.15.66:0/2468892080 wait complete. 2024-03-30T00:43:04.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b63c00640 1 Processor -- start 2024-03-30T00:43:04.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.180+0000 7f1b63c00640 1 -- start start 2024-03-30T00:43:04.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b63c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0cf0c0 0x7f1b5c0c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b63c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c15a810 0x7f1b5c1597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b63c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b5c0c86c0 0x7f1b5c1e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b63c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0c8b80 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b63c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0c8e60 con 0x7f1b5c15a810 2024-03-30T00:43:04.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b63c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0c9140 con 0x7f1b5c0cf0c0 2024-03-30T00:43:04.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b61c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b5c0c86c0 0x7f1b5c1e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:04.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b61c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b5c0c86c0 0x7f1b5c1e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34694/0 (socket says 172.21.15.66:34694) 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b61c00640 1 -- 172.21.15.66:0/2757215452 learned_addr learned my addr 172.21.15.66:0/2757215452 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b61200640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0cf0c0 0x7f1b5c0c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b53e00640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c15a810 0x7f1b5c1597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b61c00640 1 -- 172.21.15.66:0/2757215452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0cf0c0 msgr2=0x7f1b5c0c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b61c00640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0cf0c0 0x7f1b5c0c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b61c00640 1 -- 172.21.15.66:0/2757215452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c15a810 msgr2=0x7f1b5c1597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b61c00640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c15a810 0x7f1b5c1597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b61c00640 1 -- 172.21.15.66:0/2757215452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b440094d0 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b61200640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0cf0c0 0x7f1b5c0c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b53e00640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c15a810 0x7f1b5c1597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b61c00640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b5c0c86c0 0x7f1b5c1e4f70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1b54007d20 tx=0x7f1b5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b59600640 1 -- 172.21.15.66:0/2757215452 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54014420 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b59600640 1 -- 172.21.15.66:0/2757215452 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b540145a0 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b59600640 1 -- 172.21.15.66:0/2757215452 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54014950 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b63c00640 1 -- 172.21.15.66:0/2757215452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c0c9420 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b63c00640 1 -- 172.21.15.66:0/2757215452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c159d20 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b63c00640 1 -- 172.21.15.66:0/2757215452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c066c30 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.184+0000 7f1b59600640 1 -- 172.21.15.66:0/2757215452 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1b54014ab0 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.188+0000 7f1b59600640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b38066690 0x7f1b38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.188+0000 7f1b59600640 1 -- 172.21.15.66:0/2757215452 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1b54084810 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.188+0000 7f1b61200640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b38066690 0x7f1b38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:04.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.188+0000 7f1b61200640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b38066690 0x7f1b38068b50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009980 tx=0x7f1b4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:04.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.192+0000 7f1b59600640 1 -- 172.21.15.66:0/2757215452 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1b54051dd0 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.352+0000 7f1b63c00640 1 -- 172.21.15.66:0/2757215452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b5c0c98d0 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b59600640 1 -- 172.21.15.66:0/2757215452 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b540553f0 con 0x7f1b5c0c86c0 2024-03-30T00:43:04.361 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:43:04.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 -- 172.21.15.66:0/2757215452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b38066690 msgr2=0x7f1b38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:04.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b38066690 0x7f1b38068b50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009980 tx=0x7f1b4c008040 comp rx=0 tx=0).stop 2024-03-30T00:43:04.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 -- 172.21.15.66:0/2757215452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b5c0c86c0 msgr2=0x7f1b5c1e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:04.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b5c0c86c0 0x7f1b5c1e4f70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1b54007d20 tx=0x7f1b5400e4b0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 -- 172.21.15.66:0/2757215452 shutdown_connections 2024-03-30T00:43:04.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b38066690 0x7f1b38068b50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b5c0c86c0 0x7f1b5c1e4f70 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c15a810 0x7f1b5c1597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 --2- 172.21.15.66:0/2757215452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0cf0c0 0x7f1b5c0c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 -- 172.21.15.66:0/2757215452 >> 172.21.15.66:0/2757215452 conn(0x7f1b5c0b2090 msgr2=0x7f1b5c06e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:04.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 -- 172.21.15.66:0/2757215452 shutdown_connections 2024-03-30T00:43:04.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.356+0000 7f1b63c00640 1 -- 172.21.15.66:0/2757215452 wait complete. 2024-03-30T00:43:04.383 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-03-30T00:43:04.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe801800640 1 Processor -- start 2024-03-30T00:43:04.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe801800640 1 -- start start 2024-03-30T00:43:04.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe801800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe7fc0ca540 0x7fe7fc0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe801800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe7fc070ae0 0x7fe7fc0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe801800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc070f30 con 0x7fe7fc0ade30 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe801800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc0cb290 con 0x7fe7fc0ca540 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe801800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc0ca9f0 con 0x7fe7fc070ae0 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe7fb400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe7fc0ca540 0x7fe7fc0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe7fb400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe7fc0ca540 0x7fe7fc0705a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34704/0 (socket says 172.21.15.66:34704) 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe7fb400640 1 -- 172.21.15.66:0/115556474 learned_addr learned my addr 172.21.15.66:0/115556474 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe800e00640 1 --2- 172.21.15.66:0/115556474 >> v2:172.21.15.156:3300/0 conn(0x7fe7fc070ae0 0x7fe7fc0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe7fb400640 1 -- 172.21.15.66:0/115556474 >> v2:172.21.15.156:3300/0 conn(0x7fe7fc070ae0 msgr2=0x7fe7fc0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe7fb400640 1 --2- 172.21.15.66:0/115556474 >> v2:172.21.15.156:3300/0 conn(0x7fe7fc070ae0 0x7fe7fc0ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe7fb400640 1 -- 172.21.15.66:0/115556474 >> v1:172.21.15.66:6789/0 conn(0x7fe7fc0ade30 legacy=0x7fe7fc0c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.520+0000 7fe7fb400640 1 -- 172.21.15.66:0/115556474 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc0cacd0 con 0x7fe7fc0ca540 2024-03-30T00:43:04.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe7fb400640 1 --2- 172.21.15.66:0/115556474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0ca540 0x7fe7fc0705a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc06e610 tx=0x7fe7e801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbe451e625ef46d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:04.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe7faa00640 1 -- 172.21.15.66:0/115556474 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80ac630 con 0x7fe7fc0ca540 2024-03-30T00:43:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe7faa00640 1 -- 172.21.15.66:0/115556474 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e80ac790 con 0x7fe7fc0ca540 2024-03-30T00:43:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe7faa00640 1 -- 172.21.15.66:0/115556474 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80acb20 con 0x7fe7fc0ca540 2024-03-30T00:43:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe801800640 1 -- 172.21.15.66:0/115556474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0ca540 msgr2=0x7fe7fc0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe801800640 1 --2- 172.21.15.66:0/115556474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0ca540 0x7fe7fc0705a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc06e610 tx=0x7fe7e801c3d0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe801800640 1 -- 172.21.15.66:0/115556474 shutdown_connections 2024-03-30T00:43:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe801800640 1 --2- 172.21.15.66:0/115556474 >> v2:172.21.15.156:3300/0 conn(0x7fe7fc070ae0 0x7fe7fc0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe801800640 1 --2- 172.21.15.66:0/115556474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0ca540 0x7fe7fc0705a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe801800640 1 -- 172.21.15.66:0/115556474 >> 172.21.15.66:0/115556474 conn(0x7fe7fc0ce9f0 msgr2=0x7fe7fc0cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe801800640 1 -- 172.21.15.66:0/115556474 shutdown_connections 2024-03-30T00:43:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe801800640 1 -- 172.21.15.66:0/115556474 wait complete. 2024-03-30T00:43:04.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe801800640 1 Processor -- start 2024-03-30T00:43:04.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.524+0000 7fe801800640 1 -- start start 2024-03-30T00:43:04.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe801800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7fc070ae0 0x7fe7fc15b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe801800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0ade30 0x7fe7fc14b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe801800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7fc0ca540 0x7fe7fc152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe801800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0c2a50 con 0x7fe7fc0ade30 2024-03-30T00:43:04.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe801800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc152d30 con 0x7fe7fc070ae0 2024-03-30T00:43:04.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe801800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc153010 con 0x7fe7fc0ca540 2024-03-30T00:43:04.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0ade30 0x7fe7fc14b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:04.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0ade30 0x7fe7fc14b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34718/0 (socket says 172.21.15.66:34718) 2024-03-30T00:43:04.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7fb400640 1 -- 172.21.15.66:0/757066780 learned_addr learned my addr 172.21.15.66:0/757066780 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:04.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7fb400640 1 -- 172.21.15.66:0/757066780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7fc0ca540 msgr2=0x7fe7fc152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:04.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7fb400640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7fc0ca540 0x7fe7fc152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7fb400640 1 -- 172.21.15.66:0/757066780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7fc070ae0 msgr2=0x7fe7fc15b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:04.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7fb400640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7fc070ae0 0x7fe7fc15b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:04.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7fb400640 1 -- 172.21.15.66:0/757066780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e80094d0 con 0x7fe7fc0ade30 2024-03-30T00:43:04.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7fb400640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0ade30 0x7fe7fc14b060 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8018fe0 tx=0x7fe7e8019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:04.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7f3e00640 1 -- 172.21.15.66:0/757066780 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80b5640 con 0x7fe7fc0ade30 2024-03-30T00:43:04.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7f3e00640 1 -- 172.21.15.66:0/757066780 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e80b57a0 con 0x7fe7fc0ade30 2024-03-30T00:43:04.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe801800640 1 -- 172.21.15.66:0/757066780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc1532f0 con 0x7fe7fc0ade30 2024-03-30T00:43:04.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe7f3e00640 1 -- 172.21.15.66:0/757066780 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80b5b50 con 0x7fe7fc0ade30 2024-03-30T00:43:04.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe801800640 1 -- 172.21.15.66:0/757066780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc14b5a0 con 0x7fe7fc0ade30 2024-03-30T00:43:04.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.528+0000 7fe801800640 1 -- 172.21.15.66:0/757066780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc067110 con 0x7fe7fc0ade30 2024-03-30T00:43:04.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.532+0000 7fe7f3e00640 1 -- 172.21.15.66:0/757066780 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe7e80b3070 con 0x7fe7fc0ade30 2024-03-30T00:43:04.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.532+0000 7fe7f3e00640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe7dc066690 0x7fe7dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:04.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.532+0000 7fe7f3e00640 1 -- 172.21.15.66:0/757066780 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe7e811e080 con 0x7fe7fc0ade30 2024-03-30T00:43:04.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.532+0000 7fe7fbe00640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe7dc066690 0x7fe7dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:04.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.532+0000 7fe7fbe00640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe7dc066690 0x7fe7dc068b50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000f70 tx=0x7fe7ec0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:04.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.536+0000 7fe7f3e00640 1 -- 172.21.15.66:0/757066780 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe7e814f980 con 0x7fe7fc0ade30 2024-03-30T00:43:04.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:04.760+0000 7fe801800640 1 -- 172.21.15.66:0/757066780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fe7fc073cf0 con 0x7fe7fc0ade30 2024-03-30T00:43:05.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe7f3e00640 1 -- 172.21.15.66:0/757066780 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v190) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe7e80ef030 con 0x7fe7fc0ade30 2024-03-30T00:43:05.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 -- 172.21.15.66:0/757066780 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe7dc066690 msgr2=0x7fe7dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:05.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe7dc066690 0x7fe7dc068b50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000f70 tx=0x7fe7ec0079c0 comp rx=0 tx=0).stop 2024-03-30T00:43:05.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 -- 172.21.15.66:0/757066780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0ade30 msgr2=0x7fe7fc14b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:05.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0ade30 0x7fe7fc14b060 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8018fe0 tx=0x7fe7e8019170 comp rx=0 tx=0).stop 2024-03-30T00:43:05.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 -- 172.21.15.66:0/757066780 shutdown_connections 2024-03-30T00:43:05.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7fc0ca540 0x7fe7fc152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:05.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe7dc066690 0x7fe7dc068b50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:05.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7fc0ade30 0x7fe7fc14b060 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:05.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 --2- 172.21.15.66:0/757066780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7fc070ae0 0x7fe7fc15b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:05.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 -- 172.21.15.66:0/757066780 >> 172.21.15.66:0/757066780 conn(0x7fe7fc0ce9f0 msgr2=0x7fe7fc06a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:05.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 -- 172.21.15.66:0/757066780 shutdown_connections 2024-03-30T00:43:05.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.668+0000 7fe801800640 1 -- 172.21.15.66:0/757066780 wait complete. 2024-03-30T00:43:05.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-30T00:43:05.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.840+0000 7f9c34a00640 1 Processor -- start 2024-03-30T00:43:05.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.840+0000 7f9c34a00640 1 -- start start 2024-03-30T00:43:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c34a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9c300ce520 0x7f9c300ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c34a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9c300cabb0 0x7f9c300cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c34a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3006d070 con 0x7f9c3015a640 2024-03-30T00:43:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c34a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c300cbba0 con 0x7f9c300cabb0 2024-03-30T00:43:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c34a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c300cee70 con 0x7f9c300ce520 2024-03-30T00:43:05.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c2de00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9c3015a640 0x7f9c3006dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57246/0 (socket says 172.21.15.66:57246) 2024-03-30T00:43:05.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c2de00640 1 -- 172.21.15.66:0/2946555981 learned_addr learned my addr 172.21.15.66:0/2946555981 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:05.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c2e800640 1 --2- 172.21.15.66:0/2946555981 >> v2:172.21.15.66:3300/0 conn(0x7f9c300cabb0 0x7f9c300cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:05.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c25400640 1 --2- 172.21.15.66:0/2946555981 >> v2:172.21.15.156:3300/0 conn(0x7f9c300ce520 0x7f9c300ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:05.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c2d400640 1 -- 172.21.15.66:0/2946555981 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294688833 0 0) 0x7f9c3006d070 con 0x7f9c3015a640 2024-03-30T00:43:05.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c2d400640 1 -- 172.21.15.66:0/2946555981 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c14003530 con 0x7f9c3015a640 2024-03-30T00:43:05.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c2e800640 1 -- 172.21.15.66:0/2946555981 >> v2:172.21.15.156:3300/0 conn(0x7f9c300ce520 msgr2=0x7f9c300ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:05.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c2e800640 1 --2- 172.21.15.66:0/2946555981 >> v2:172.21.15.156:3300/0 conn(0x7f9c300ce520 0x7f9c300ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:05.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c2e800640 1 -- 172.21.15.66:0/2946555981 >> v1:172.21.15.66:6789/0 conn(0x7f9c3015a640 legacy=0x7f9c3006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:05.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c2e800640 1 -- 172.21.15.66:0/2946555981 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c300cb8c0 con 0x7f9c300cabb0 2024-03-30T00:43:05.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.844+0000 7f9c25400640 1 --2- 172.21.15.66:0/2946555981 >> v2:172.21.15.156:3300/0 conn(0x7f9c300ce520 0x7f9c300ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:05.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c2e800640 1 --2- 172.21.15.66:0/2946555981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c300cabb0 0x7f9c300cafe0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9c300c88d0 tx=0x7f9c2801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e91a75a7ebe62ba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:05.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c2d400640 1 -- 172.21.15.66:0/2946555981 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c280ab650 con 0x7f9c300cabb0 2024-03-30T00:43:05.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c2d400640 1 -- 172.21.15.66:0/2946555981 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c280ab7b0 con 0x7f9c300cabb0 2024-03-30T00:43:05.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c2d400640 1 -- 172.21.15.66:0/2946555981 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c280abb40 con 0x7f9c300cabb0 2024-03-30T00:43:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 -- 172.21.15.66:0/2946555981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c300cabb0 msgr2=0x7f9c300cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 --2- 172.21.15.66:0/2946555981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c300cabb0 0x7f9c300cafe0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9c300c88d0 tx=0x7f9c2801c450 comp rx=0 tx=0).stop 2024-03-30T00:43:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 -- 172.21.15.66:0/2946555981 shutdown_connections 2024-03-30T00:43:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 --2- 172.21.15.66:0/2946555981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c300cabb0 0x7f9c300cafe0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 --2- 172.21.15.66:0/2946555981 >> v2:172.21.15.156:3300/0 conn(0x7f9c300ce520 0x7f9c300ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 -- 172.21.15.66:0/2946555981 >> 172.21.15.66:0/2946555981 conn(0x7f9c30069f60 msgr2=0x7f9c3006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 -- 172.21.15.66:0/2946555981 shutdown_connections 2024-03-30T00:43:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 -- 172.21.15.66:0/2946555981 wait complete. 2024-03-30T00:43:05.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 Processor -- start 2024-03-30T00:43:05.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 -- start start 2024-03-30T00:43:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c300cabb0 0x7f9c30153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c300ce520 0x7f9c300c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c3015a640 0x7f9c300c3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c30153d40 con 0x7f9c300ce520 2024-03-30T00:43:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c300ae700 con 0x7f9c3015a640 2024-03-30T00:43:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c34a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c30154020 con 0x7f9c300cabb0 2024-03-30T00:43:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c25400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c300ce520 0x7f9c300c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c25400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c300ce520 0x7f9c300c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34736/0 (socket says 172.21.15.66:34736) 2024-03-30T00:43:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.848+0000 7f9c25400640 1 -- 172.21.15.66:0/3936668211 learned_addr learned my addr 172.21.15.66:0/3936668211 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c2de00640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c300cabb0 0x7f9c30153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c2e800640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c3015a640 0x7f9c300c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c25400640 1 -- 172.21.15.66:0/3936668211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c300cabb0 msgr2=0x7f9c30153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c25400640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c300cabb0 0x7f9c30153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c25400640 1 -- 172.21.15.66:0/3936668211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c3015a640 msgr2=0x7f9c300c3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c25400640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c3015a640 0x7f9c300c3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c25400640 1 -- 172.21.15.66:0/3936668211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c28009420 con 0x7f9c300ce520 2024-03-30T00:43:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c25400640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c300ce520 0x7f9c300c6780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e840 tx=0x7f9c1800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c26a00640 1 -- 172.21.15.66:0/3936668211 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1800c8d0 con 0x7f9c300ce520 2024-03-30T00:43:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c26a00640 1 -- 172.21.15.66:0/3936668211 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1800ca30 con 0x7f9c300ce520 2024-03-30T00:43:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c34a00640 1 -- 172.21.15.66:0/3936668211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c30154300 con 0x7f9c300ce520 2024-03-30T00:43:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c26a00640 1 -- 172.21.15.66:0/3936668211 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1800cdc0 con 0x7f9c300ce520 2024-03-30T00:43:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c34a00640 1 -- 172.21.15.66:0/3936668211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c301545e0 con 0x7f9c300ce520 2024-03-30T00:43:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c34a00640 1 -- 172.21.15.66:0/3936668211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c30066c30 con 0x7f9c300ce520 2024-03-30T00:43:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c26a00640 1 -- 172.21.15.66:0/3936668211 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9c18013410 con 0x7f9c300ce520 2024-03-30T00:43:05.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.852+0000 7f9c26a00640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9c0c066690 0x7f9c0c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:05.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.856+0000 7f9c26a00640 1 -- 172.21.15.66:0/3936668211 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f9c18083ed0 con 0x7f9c300ce520 2024-03-30T00:43:05.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.856+0000 7f9c2de00640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9c0c066690 0x7f9c0c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:05.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.856+0000 7f9c2de00640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9c0c066690 0x7f9c0c068b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9c20001570 tx=0x7f9c20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:05.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:05.860+0000 7f9c26a00640 1 -- 172.21.15.66:0/3936668211 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9c18051360 con 0x7f9c300ce520 2024-03-30T00:43:06.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.020+0000 7f9c34a00640 1 -- 172.21.15.66:0/3936668211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9c301549b0 con 0x7f9c300ce520 2024-03-30T00:43:06.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.024+0000 7f9c26a00640 1 -- 172.21.15.66:0/3936668211 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9c18054980 con 0x7f9c300ce520 2024-03-30T00:43:06.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 -- 172.21.15.66:0/3936668211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9c0c066690 msgr2=0x7f9c0c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9c0c066690 0x7f9c0c068b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9c20001570 tx=0x7f9c20009040 comp rx=0 tx=0).stop 2024-03-30T00:43:06.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 -- 172.21.15.66:0/3936668211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c300ce520 msgr2=0x7f9c300c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c300ce520 0x7f9c300c6780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e840 tx=0x7f9c1800ed00 comp rx=0 tx=0).stop 2024-03-30T00:43:06.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 -- 172.21.15.66:0/3936668211 shutdown_connections 2024-03-30T00:43:06.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c3015a640 0x7f9c300c3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9c0c066690 0x7f9c0c068b50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c300ce520 0x7f9c300c6780 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 --2- 172.21.15.66:0/3936668211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c300cabb0 0x7f9c30153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 -- 172.21.15.66:0/3936668211 >> 172.21.15.66:0/3936668211 conn(0x7f9c30069f60 msgr2=0x7f9c300ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:06.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 -- 172.21.15.66:0/3936668211 shutdown_connections 2024-03-30T00:43:06.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.028+0000 7f9c34a00640 1 -- 172.21.15.66:0/3936668211 wait complete. 2024-03-30T00:43:06.053 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:43:06.053 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:43:06.107 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:43:06.210 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:43:06.210 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:43:06.210 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:43:06.219 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:43:06.219 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:43:06.219 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:43:06.229 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:43:06.229 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:43:06.229 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:43:06.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:43:06.268 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:43:06.268 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:43:06.283 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:43:06.283 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-30T00:43:06.283 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:43:06.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.444+0000 7f34e3600640 1 Processor -- start 2024-03-30T00:43:06.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.444+0000 7f34e3600640 1 -- start start 2024-03-30T00:43:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34e3600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f34dc0cda90 0x7f34dc0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34e3600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34dc15a370 0x7f34dc0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34e3600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34dc15a990 con 0x7f34dc0cde70 2024-03-30T00:43:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34e3600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34dc0b1b80 con 0x7f34dc0cda90 2024-03-30T00:43:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34e3600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc0cd3e0 con 0x7f34dc15a370 2024-03-30T00:43:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34d3e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f34dc0cde70 0x7f34dc159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57252/0 (socket says 172.21.15.66:57252) 2024-03-30T00:43:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34d3e00640 1 -- 172.21.15.66:0/4235454100 learned_addr learned my addr 172.21.15.66:0/4235454100 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:06.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34e0c00640 1 --2- 172.21.15.66:0/4235454100 >> v2:172.21.15.66:3300/0 conn(0x7f34dc0cda90 0x7f34dc0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34e1600640 1 --2- 172.21.15.66:0/4235454100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34dc15a370 0x7f34dc0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34e0c00640 1 -- 172.21.15.66:0/4235454100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34dc15a370 msgr2=0x7f34dc0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:06.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34e0c00640 1 --2- 172.21.15.66:0/4235454100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34dc15a370 0x7f34dc0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34e0c00640 1 -- 172.21.15.66:0/4235454100 >> v1:172.21.15.66:6789/0 conn(0x7f34dc0cde70 legacy=0x7f34dc159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34e0c00640 1 -- 172.21.15.66:0/4235454100 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc0b11e0 con 0x7f34dc0cda90 2024-03-30T00:43:06.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.448+0000 7f34dbe00640 1 -- 172.21.15.66:0/4235454100 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567592779 0 0) 0x7f34dc15a990 con 0x7f34dc0cde70 2024-03-30T00:43:06.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e0c00640 1 --2- 172.21.15.66:0/4235454100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc0cda90 0x7f34dc0ce520 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0097c0 tx=0x7f34cc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ea6683063902da server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:06.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34dbe00640 1 -- 172.21.15.66:0/4235454100 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0ab600 con 0x7f34dc0cda90 2024-03-30T00:43:06.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34dbe00640 1 -- 172.21.15.66:0/4235454100 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc0a7050 con 0x7f34dc0cda90 2024-03-30T00:43:06.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34dbe00640 1 -- 172.21.15.66:0/4235454100 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0ab980 con 0x7f34dc0cda90 2024-03-30T00:43:06.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 -- 172.21.15.66:0/4235454100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc0cda90 msgr2=0x7f34dc0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 --2- 172.21.15.66:0/4235454100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc0cda90 0x7f34dc0ce520 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0097c0 tx=0x7f34cc01c410 comp rx=0 tx=0).stop 2024-03-30T00:43:06.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 -- 172.21.15.66:0/4235454100 shutdown_connections 2024-03-30T00:43:06.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 --2- 172.21.15.66:0/4235454100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34dc15a370 0x7f34dc0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 --2- 172.21.15.66:0/4235454100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc0cda90 0x7f34dc0ce520 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 -- 172.21.15.66:0/4235454100 >> 172.21.15.66:0/4235454100 conn(0x7f34dc069f60 msgr2=0x7f34dc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:06.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 -- 172.21.15.66:0/4235454100 shutdown_connections 2024-03-30T00:43:06.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 -- 172.21.15.66:0/4235454100 wait complete. 2024-03-30T00:43:06.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 Processor -- start 2024-03-30T00:43:06.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 -- start start 2024-03-30T00:43:06.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc0cda90 0x7f34dc156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34dc0cde70 0x7f34dc159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34dc15a370 0x7f34dc162f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc156790 con 0x7f34dc0cda90 2024-03-30T00:43:06.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc1644c0 con 0x7f34dc15a370 2024-03-30T00:43:06.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e3600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc163f00 con 0x7f34dc0cde70 2024-03-30T00:43:06.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc0cda90 0x7f34dc156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc0cda90 0x7f34dc156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34744/0 (socket says 172.21.15.66:34744) 2024-03-30T00:43:06.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.452+0000 7f34e0c00640 1 -- 172.21.15.66:0/3448769656 learned_addr learned my addr 172.21.15.66:0/3448769656 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:06.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34d3e00640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34dc0cde70 0x7f34dc159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34e1600640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34dc15a370 0x7f34dc162f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34e0c00640 1 -- 172.21.15.66:0/3448769656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34dc0cde70 msgr2=0x7f34dc159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34e0c00640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34dc0cde70 0x7f34dc159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34e0c00640 1 -- 172.21.15.66:0/3448769656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34dc15a370 msgr2=0x7f34dc162f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34e0c00640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34dc15a370 0x7f34dc162f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34e0c00640 1 -- 172.21.15.66:0/3448769656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc009420 con 0x7f34dc0cda90 2024-03-30T00:43:06.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34e0c00640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc0cda90 0x7f34dc156250 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0097c0 tx=0x7f34cc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:06.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34d9600640 1 -- 172.21.15.66:0/3448769656 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0b4610 con 0x7f34dc0cda90 2024-03-30T00:43:06.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34d9600640 1 -- 172.21.15.66:0/3448769656 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc0abe90 con 0x7f34dc0cda90 2024-03-30T00:43:06.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34d9600640 1 -- 172.21.15.66:0/3448769656 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0b49d0 con 0x7f34dc0cda90 2024-03-30T00:43:06.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34e3600640 1 -- 172.21.15.66:0/3448769656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc1641e0 con 0x7f34dc0cda90 2024-03-30T00:43:06.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34e3600640 1 -- 172.21.15.66:0/3448769656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc162a30 con 0x7f34dc0cda90 2024-03-30T00:43:06.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34e3600640 1 -- 172.21.15.66:0/3448769656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc066c30 con 0x7f34dc0cda90 2024-03-30T00:43:06.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.456+0000 7f34d9600640 1 -- 172.21.15.66:0/3448769656 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f34cc0b2040 con 0x7f34dc0cda90 2024-03-30T00:43:06.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.460+0000 7f34d9600640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34b8066690 0x7f34b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.460+0000 7f34d9600640 1 -- 172.21.15.66:0/3448769656 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f34cc11dfd0 con 0x7f34dc0cda90 2024-03-30T00:43:06.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.460+0000 7f34d3e00640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34b8066690 0x7f34b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.460+0000 7f34d3e00640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34b8066690 0x7f34b8068b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0b0c60 tx=0x7f34c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:06.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.464+0000 7f34d9600640 1 -- 172.21.15.66:0/3448769656 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f34cc0eb4f0 con 0x7f34dc0cda90 2024-03-30T00:43:06.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.656+0000 7f34e3600640 1 -- 172.21.15.66:0/3448769656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34dc162c20 con 0x7f34dc0cda90 2024-03-30T00:43:06.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.656+0000 7f34d9600640 1 -- 172.21.15.66:0/3448769656 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f34cc0b0de0 con 0x7f34dc0cda90 2024-03-30T00:43:06.664 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:06.664 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/465916334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":465916334},{"type":"v1","addr":"172.21.15.156:6835","nonce":465916334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1640047258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1640047258},{"type":"v1","addr":"172.21.15.156:6834","nonce":1640047258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5788,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2112021115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2112021115},{"type":"v1","addr":"172.21.15.66:6835","nonce":2112021115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2228447330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2228447330},{"type":"v1","addr":"172.21.15.66:6837","nonce":2228447330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[]} 2024-03-30T00:43:06.664 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2024-03-30T00:43:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 -- 172.21.15.66:0/3448769656 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34b8066690 msgr2=0x7f34b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34b8066690 0x7f34b8068b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0b0c60 tx=0x7f34c4009040 comp rx=0 tx=0).stop 2024-03-30T00:43:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 -- 172.21.15.66:0/3448769656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc0cda90 msgr2=0x7f34dc156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc0cda90 0x7f34dc156250 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0097c0 tx=0x7f34cc0191b0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 -- 172.21.15.66:0/3448769656 shutdown_connections 2024-03-30T00:43:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34dc15a370 0x7f34dc162f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34dc0cde70 0x7f34dc159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34b8066690 0x7f34b8068b50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 --2- 172.21.15.66:0/3448769656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc0cda90 0x7f34dc156250 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 -- 172.21.15.66:0/3448769656 >> 172.21.15.66:0/3448769656 conn(0x7f34dc069f60 msgr2=0x7f34dc06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:06.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 -- 172.21.15.66:0/3448769656 shutdown_connections 2024-03-30T00:43:06.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.660+0000 7f34e3600640 1 -- 172.21.15.66:0/3448769656 wait complete. 2024-03-30T00:43:06.686 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T00:43:06.686 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-30T00:43:06.686 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T00:43:06.686 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:43:06.686 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T00:43:06.687 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-30T00:43:06.687 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T00:43:06.687 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:43:06.687 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T00:43:06.687 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-30T00:43:06.687 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T00:43:06.687 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:43:06.688 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T00:43:06.688 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-30T00:43:06.688 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T00:43:06.688 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:43:06.700 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T00:43:06.700 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T00:43:06.701 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T00:43:06.702 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T00:43:06.706 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T00:43:06.706 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T00:43:06.707 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T00:43:06.708 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T00:43:06.748 INFO:tasks.ceph.mds.c:Started 2024-03-30T00:43:06.748 DEBUG:teuthology.parallel:result is None 2024-03-30T00:43:06.750 INFO:tasks.ceph.mds.d:Started 2024-03-30T00:43:06.750 DEBUG:teuthology.parallel:result is None 2024-03-30T00:43:06.751 INFO:tasks.ceph.mds.a:Started 2024-03-30T00:43:06.751 DEBUG:teuthology.parallel:result is None 2024-03-30T00:43:06.751 INFO:tasks.ceph.mds.b:Started 2024-03-30T00:43:06.751 DEBUG:teuthology.parallel:result is None 2024-03-30T00:43:06.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-30T00:43:06.835 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:43:06.837 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:43:06.839 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2024-03-30T00:43:06.840 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2024-03-30T00:43:06.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.900+0000 7f22c6e00640 1 Processor -- start 2024-03-30T00:43:06.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.900+0000 7f22c6e00640 1 -- start start 2024-03-30T00:43:06.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22c6e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f22c015de30 0x7f22c00cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22c6e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f22c00af420 0x7f22c00d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22c6e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c00cbb90 con 0x7f22c00ce740 2024-03-30T00:43:06.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22c6e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c00ce1b0 con 0x7f22c015de30 2024-03-30T00:43:06.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22c6e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c00afa10 con 0x7f22c00af420 2024-03-30T00:43:06.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22c4e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f22c00ce740 0x7f22c006e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57266/0 (socket says 172.21.15.66:57266) 2024-03-30T00:43:06.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22c4e00640 1 -- 172.21.15.66:0/2567596261 learned_addr learned my addr 172.21.15.66:0/2567596261 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:06.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22bfe00640 1 --2- 172.21.15.66:0/2567596261 >> v2:172.21.15.66:3300/0 conn(0x7f22c015de30 0x7f22c00cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22b7400640 1 --2- 172.21.15.66:0/2567596261 >> v2:172.21.15.156:3300/0 conn(0x7f22c00af420 0x7f22c00d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22bf400640 1 -- 172.21.15.66:0/2567596261 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533249695 0 0) 0x7f22c00cbb90 con 0x7f22c00ce740 2024-03-30T00:43:06.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22bf400640 1 -- 172.21.15.66:0/2567596261 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22a4003530 con 0x7f22c00ce740 2024-03-30T00:43:06.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22bfe00640 1 -- 172.21.15.66:0/2567596261 >> v2:172.21.15.156:3300/0 conn(0x7f22c00af420 msgr2=0x7f22c00d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:06.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22bfe00640 1 --2- 172.21.15.66:0/2567596261 >> v2:172.21.15.156:3300/0 conn(0x7f22c00af420 0x7f22c00d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22bfe00640 1 -- 172.21.15.66:0/2567596261 >> v1:172.21.15.66:6789/0 conn(0x7f22c00ce740 legacy=0x7f22c006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22bfe00640 1 -- 172.21.15.66:0/2567596261 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c00cf430 con 0x7f22c015de30 2024-03-30T00:43:06.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.904+0000 7f22bfe00640 1 --2- 172.21.15.66:0/2567596261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22c015de30 0x7f22c00cc040 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f22c0070080 tx=0x7f22b001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb7c93fd91c4b9c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:06.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22bf400640 1 -- 172.21.15.66:0/2567596261 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0004230 con 0x7f22c015de30 2024-03-30T00:43:06.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22bf400640 1 -- 172.21.15.66:0/2567596261 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b00a7050 con 0x7f22c015de30 2024-03-30T00:43:06.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22bf400640 1 -- 172.21.15.66:0/2567596261 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0003620 con 0x7f22c015de30 2024-03-30T00:43:06.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 -- 172.21.15.66:0/2567596261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22c015de30 msgr2=0x7f22c00cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 --2- 172.21.15.66:0/2567596261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22c015de30 0x7f22c00cc040 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f22c0070080 tx=0x7f22b001c3f0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 -- 172.21.15.66:0/2567596261 shutdown_connections 2024-03-30T00:43:06.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 --2- 172.21.15.66:0/2567596261 >> v2:172.21.15.156:3300/0 conn(0x7f22c00af420 0x7f22c00d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 --2- 172.21.15.66:0/2567596261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22c015de30 0x7f22c00cc040 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 -- 172.21.15.66:0/2567596261 >> 172.21.15.66:0/2567596261 conn(0x7f22c00d0540 msgr2=0x7f22c00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:06.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 -- 172.21.15.66:0/2567596261 shutdown_connections 2024-03-30T00:43:06.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 -- 172.21.15.66:0/2567596261 wait complete. 2024-03-30T00:43:06.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 Processor -- start 2024-03-30T00:43:06.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 -- start start 2024-03-30T00:43:06.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f22c00af420 0x7f22c00affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22c00ce740 0x7f22c00c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22c015de30 0x7f22c00ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c00b0830 con 0x7f22c00ce740 2024-03-30T00:43:06.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c00b0b10 con 0x7f22c015de30 2024-03-30T00:43:06.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c6e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c00b0df0 con 0x7f22c00af420 2024-03-30T00:43:06.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22b7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22c00ce740 0x7f22c00c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22b7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22c00ce740 0x7f22c00c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34788/0 (socket says 172.21.15.66:34788) 2024-03-30T00:43:06.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22b7400640 1 -- 172.21.15.66:0/4230505145 learned_addr learned my addr 172.21.15.66:0/4230505145 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:06.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22c4e00640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22c015de30 0x7f22c00ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22bfe00640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f22c00af420 0x7f22c00affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22b7400640 1 -- 172.21.15.66:0/4230505145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f22c00af420 msgr2=0x7f22c00affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22b7400640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f22c00af420 0x7f22c00affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.908+0000 7f22b7400640 1 -- 172.21.15.66:0/4230505145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22c015de30 msgr2=0x7f22c00ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:06.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22b7400640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22c015de30 0x7f22c00ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:06.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22b7400640 1 -- 172.21.15.66:0/4230505145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b0009420 con 0x7f22c00ce740 2024-03-30T00:43:06.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22b7400640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22c00ce740 0x7f22c00c6810 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f22a800e870 tx=0x7f22a800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:06.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22bcc00640 1 -- 172.21.15.66:0/4230505145 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8009510 con 0x7f22c00ce740 2024-03-30T00:43:06.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22bcc00640 1 -- 172.21.15.66:0/4230505145 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a800eec0 con 0x7f22c00ce740 2024-03-30T00:43:06.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22bcc00640 1 -- 172.21.15.66:0/4230505145 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a800cb40 con 0x7f22c00ce740 2024-03-30T00:43:06.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22c6e00640 1 -- 172.21.15.66:0/4230505145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c00b10d0 con 0x7f22c00ce740 2024-03-30T00:43:06.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22c6e00640 1 -- 172.21.15.66:0/4230505145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c00b04e0 con 0x7f22c00ce740 2024-03-30T00:43:06.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22c6e00640 1 -- 172.21.15.66:0/4230505145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c00cdf60 con 0x7f22c00ce740 2024-03-30T00:43:06.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22bcc00640 1 -- 172.21.15.66:0/4230505145 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f22a800ccc0 con 0x7f22c00ce740 2024-03-30T00:43:06.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22bcc00640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f229c066690 0x7f229c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:06.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22bcc00640 1 -- 172.21.15.66:0/4230505145 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f22a8083fe0 con 0x7f22c00ce740 2024-03-30T00:43:06.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.912+0000 7f22bfe00640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f229c066690 0x7f229c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:06.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.916+0000 7f22bfe00640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f229c066690 0x7f229c068b50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f22b00ad000 tx=0x7f22b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:06.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:06.916+0000 7f22bcc00640 1 -- 172.21.15.66:0/4230505145 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f22a80515c0 con 0x7f22c00ce740 2024-03-30T00:43:07.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.092+0000 7f22c6e00640 1 -- 172.21.15.66:0/4230505145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f22c00c4980 con 0x7f22c00ce740 2024-03-30T00:43:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.868+0000 7f22bcc00640 1 -- 172.21.15.66:0/4230505145 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) v1 ==== 97+0+0 (secure 0 0 0) 0x7f22a8054be0 con 0x7f22c00ce740 2024-03-30T00:43:07.876 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-03-30T00:43:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 -- 172.21.15.66:0/4230505145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f229c066690 msgr2=0x7f229c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f229c066690 0x7f229c068b50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f22b00ad000 tx=0x7f22b0008040 comp rx=0 tx=0).stop 2024-03-30T00:43:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 -- 172.21.15.66:0/4230505145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22c00ce740 msgr2=0x7f22c00c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22c00ce740 0x7f22c00c6810 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f22a800e870 tx=0x7f22a800ec70 comp rx=0 tx=0).stop 2024-03-30T00:43:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 -- 172.21.15.66:0/4230505145 shutdown_connections 2024-03-30T00:43:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22c015de30 0x7f22c00ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f229c066690 0x7f229c068b50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22c00ce740 0x7f22c00c6810 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 --2- 172.21.15.66:0/4230505145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f22c00af420 0x7f22c00affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 -- 172.21.15.66:0/4230505145 >> 172.21.15.66:0/4230505145 conn(0x7f22c00d0540 msgr2=0x7f22c015aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 -- 172.21.15.66:0/4230505145 shutdown_connections 2024-03-30T00:43:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:07.872+0000 7f22c6e00640 1 -- 172.21.15.66:0/4230505145 wait complete. 2024-03-30T00:43:07.899 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-30T00:43:08.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.036+0000 7f08a8c00640 1 Processor -- start 2024-03-30T00:43:08.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.036+0000 7f08a8c00640 1 -- start start 2024-03-30T00:43:08.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.036+0000 7f08a8c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f08a40cda90 0x7f08a40ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.036+0000 7f08a8c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a40cd3e0 con 0x7f08a40cde70 2024-03-30T00:43:08.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.036+0000 7f08a8c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a415a990 con 0x7f08a415a370 2024-03-30T00:43:08.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.036+0000 7f08a8c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a40b1b80 con 0x7f08a40cda90 2024-03-30T00:43:08.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.036+0000 7f0899400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f08a40cde70 0x7f08a4159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57278/0 (socket says 172.21.15.66:57278) 2024-03-30T00:43:08.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.036+0000 7f08a1e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f08a40cda90 0x7f08a40ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.036+0000 7f0899400640 1 -- 172.21.15.66:0/3417567904 learned_addr learned my addr 172.21.15.66:0/3417567904 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:08.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a1e00640 1 -- 172.21.15.66:0/3417567904 >> v1:172.21.15.156:6789/0 conn(0x7f08a415a370 legacy=0x7f08a4156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a1e00640 1 -- 172.21.15.66:0/3417567904 >> v1:172.21.15.66:6789/0 conn(0x7f08a40cde70 legacy=0x7f08a4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a1e00640 1 -- 172.21.15.66:0/3417567904 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a40b11e0 con 0x7f08a40cda90 2024-03-30T00:43:08.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a1400640 1 -- 172.21.15.66:0/3417567904 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344485437 0 0) 0x7f08a40cd3e0 con 0x7f08a40cde70 2024-03-30T00:43:08.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a1e00640 1 --2- 172.21.15.66:0/3417567904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40cda90 0x7f08a40ce520 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f08940097c0 tx=0x7f089401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76184e32930c337a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:08.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a1400640 1 -- 172.21.15.66:0/3417567904 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08940ab600 con 0x7f08a40cda90 2024-03-30T00:43:08.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a1400640 1 -- 172.21.15.66:0/3417567904 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08940a7050 con 0x7f08a40cda90 2024-03-30T00:43:08.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a1400640 1 -- 172.21.15.66:0/3417567904 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08940ab980 con 0x7f08a40cda90 2024-03-30T00:43:08.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a8c00640 1 -- 172.21.15.66:0/3417567904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40cda90 msgr2=0x7f08a40ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a8c00640 1 --2- 172.21.15.66:0/3417567904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40cda90 0x7f08a40ce520 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f08940097c0 tx=0x7f089401c410 comp rx=0 tx=0).stop 2024-03-30T00:43:08.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a8c00640 1 -- 172.21.15.66:0/3417567904 shutdown_connections 2024-03-30T00:43:08.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a8c00640 1 --2- 172.21.15.66:0/3417567904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40cda90 0x7f08a40ce520 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a8c00640 1 -- 172.21.15.66:0/3417567904 >> 172.21.15.66:0/3417567904 conn(0x7f08a4069f60 msgr2=0x7f08a406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:08.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a8c00640 1 -- 172.21.15.66:0/3417567904 shutdown_connections 2024-03-30T00:43:08.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.040+0000 7f08a8c00640 1 -- 172.21.15.66:0/3417567904 wait complete. 2024-03-30T00:43:08.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a8c00640 1 Processor -- start 2024-03-30T00:43:08.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a8c00640 1 -- start start 2024-03-30T00:43:08.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a8c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08a40cda90 0x7f08a40ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a8c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08a40cde70 0x7f08a4097a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a415a370 0x7f08a40cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a8c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40af2a0 con 0x7f08a415a370 2024-03-30T00:43:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a8c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a4070b90 con 0x7f08a40cda90 2024-03-30T00:43:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a8c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a4162e50 con 0x7f08a40cde70 2024-03-30T00:43:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a415a370 0x7f08a40cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a415a370 0x7f08a40cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34812/0 (socket says 172.21.15.66:34812) 2024-03-30T00:43:08.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a2800640 1 -- 172.21.15.66:0/4109566104 learned_addr learned my addr 172.21.15.66:0/4109566104 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a1e00640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08a40cda90 0x7f08a40ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f0899400640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08a40cde70 0x7f08a4097a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a2800640 1 -- 172.21.15.66:0/4109566104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08a40cde70 msgr2=0x7f08a4097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a2800640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08a40cde70 0x7f08a4097a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a2800640 1 -- 172.21.15.66:0/4109566104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08a40cda90 msgr2=0x7f08a40ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a2800640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08a40cda90 0x7f08a40ae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a2800640 1 -- 172.21.15.66:0/4109566104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0894009420 con 0x7f08a415a370 2024-03-30T00:43:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f0899400640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08a40cde70 0x7f08a4097a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a2800640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a415a370 0x7f08a40cb310 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f089c00c430 tx=0x7f089c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f089aa00640 1 -- 172.21.15.66:0/4109566104 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c005220 con 0x7f08a415a370 2024-03-30T00:43:08.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f089aa00640 1 -- 172.21.15.66:0/4109566104 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c005380 con 0x7f08a415a370 2024-03-30T00:43:08.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f089aa00640 1 -- 172.21.15.66:0/4109566104 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c007270 con 0x7f08a415a370 2024-03-30T00:43:08.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a8c00640 1 -- 172.21.15.66:0/4109566104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a40aea00 con 0x7f08a415a370 2024-03-30T00:43:08.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.044+0000 7f08a8c00640 1 -- 172.21.15.66:0/4109566104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a40aefc0 con 0x7f08a415a370 2024-03-30T00:43:08.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.048+0000 7f08a8c00640 1 -- 172.21.15.66:0/4109566104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a40708f0 con 0x7f08a415a370 2024-03-30T00:43:08.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.048+0000 7f089aa00640 1 -- 172.21.15.66:0/4109566104 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f089c0073d0 con 0x7f08a415a370 2024-03-30T00:43:08.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.048+0000 7f089aa00640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0880066690 0x7f0880068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.048+0000 7f089aa00640 1 -- 172.21.15.66:0/4109566104 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f089c083520 con 0x7f08a415a370 2024-03-30T00:43:08.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.048+0000 7f08a1e00640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0880066690 0x7f0880068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.048+0000 7f08a1e00640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0880066690 0x7f0880068b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f08940ad000 tx=0x7f0894008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:08.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.052+0000 7f089aa00640 1 -- 172.21.15.66:0/4109566104 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f08a40708f0 con 0x7f08a415a370 2024-03-30T00:43:08.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.264+0000 7f08a8c00640 1 -- 172.21.15.66:0/4109566104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f08a4070cd0 con 0x7f08a415a370 2024-03-30T00:43:08.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f089aa00640 1 -- 172.21.15.66:0/4109566104 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f08a4070cd0 con 0x7f08a415a370 2024-03-30T00:43:08.273 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:08.273 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:43:08.273 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-03-30T00:43:08.273 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.273 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-03-30T00:43:08.273 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm857MDRAAu2pldF20deZfoSDDXuTF/A==", 2024-03-30T00:43:08.273 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.273 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMmNFGhAAlHS6ZpIoJc7tlYazCISiLw==", 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmuiBsFRAA2kez6ylAylKTWsQ1S2cahA==", 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm/oV3IRAAWn/zG1cLK+Raa2pYPzHd7Q==", 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.275 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAbXwdmf3HtOhAAyRcyb0YUJcJca5fjmZ3Frw==", 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.276 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAfXwdml7cdBRAAs639bBHTPyfhfVZMPRMfYA==", 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAiXwdmeS6KChAAZjLAbKSL7Vswi3qQ4304bA==", 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:43:08.277 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAlXwdmeuigEBAAiPIPheowKBnn3UZWwpdr+Q==", 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-03-30T00:43:08.278 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQApXwdm+bQNHRAAAb/lufdb7Fe+EFevd23gtA==", 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAsXwdmReL1IhAAwCs1cS+P6J8VCOYdrnrPQg==", 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAvXwdmA15vKBAAmpnKrDM/Y0xuBGxdyjrkWA==", 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAyXwdmrl02LhAA6JfEnugoz6l4nsXl6/vq0Q==", 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:43:08.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmZjh7JRAA3EFOImM03iOljcwmA7/xWw==", 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.281 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmkZv0LhAAgwCqW25+J1VYO31mWwDtLQ==", 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmLG1/KxAAmNIZ9ZXdqPRQjCXvz7wC9A==", 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:43:08.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmyJ/nNBAAIgiUoUCbTeLEjL1iaPFw1A==", 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:43:08.283 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAXXwdmwWxQLRAAoFyZnFsIo1NvQlzRN+UHmw==", 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmEF+xCRAAqRVFguki881+4azP8lsf0g==", 2024-03-30T00:43:08.284 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMg32BRAAUWBF+QbwRGsopcAoBay8wA==", 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:08.285 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-03-30T00:43:08.286 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:43:08.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 -- 172.21.15.66:0/4109566104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0880066690 msgr2=0x7f0880068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0880066690 0x7f0880068b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f08940ad000 tx=0x7f0894008040 comp rx=0 tx=0).stop 2024-03-30T00:43:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 -- 172.21.15.66:0/4109566104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a415a370 msgr2=0x7f08a40cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a415a370 0x7f08a40cb310 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f089c00c430 tx=0x7f089c00c8f0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 -- 172.21.15.66:0/4109566104 shutdown_connections 2024-03-30T00:43:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0880066690 0x7f0880068b50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a415a370 0x7f08a40cb310 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08a40cde70 0x7f08a4097a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 --2- 172.21.15.66:0/4109566104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08a40cda90 0x7f08a40ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 -- 172.21.15.66:0/4109566104 >> 172.21.15.66:0/4109566104 conn(0x7f08a4069f60 msgr2=0x7f08a415d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 -- 172.21.15.66:0/4109566104 shutdown_connections 2024-03-30T00:43:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.268+0000 7f08a8c00640 1 -- 172.21.15.66:0/4109566104 wait complete. 2024-03-30T00:43:08.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:43:08.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.428+0000 7f8a52a00640 1 Processor -- start 2024-03-30T00:43:08.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.428+0000 7f8a52a00640 1 -- start start 2024-03-30T00:43:08.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.428+0000 7f8a52a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8a4c0cda90 0x7f8a4c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.428+0000 7f8a52a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a4c0cde70 0x7f8a4c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.428+0000 7f8a52a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4c15a990 con 0x7f8a4c159c60 2024-03-30T00:43:08.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.428+0000 7f8a52a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a4c0b1b80 con 0x7f8a4c0cda90 2024-03-30T00:43:08.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.428+0000 7f8a52a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c0cd3e0 con 0x7f8a4c0cde70 2024-03-30T00:43:08.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a43400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a4c0cde70 0x7f8a4c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a4be00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8a4c0cda90 0x7f8a4c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a43400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a4c0cde70 0x7f8a4c0cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:35616/0 (socket says 172.21.15.66:35616) 2024-03-30T00:43:08.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a43400640 1 -- 172.21.15.66:0/3267141499 learned_addr learned my addr 172.21.15.66:0/3267141499 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:08.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a4b400640 1 -- 172.21.15.66:0/3267141499 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439803475 0 0) 0x7f8a4c15a990 con 0x7f8a4c159c60 2024-03-30T00:43:08.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a4be00640 1 -- 172.21.15.66:0/3267141499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a4c0cde70 msgr2=0x7f8a4c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:08.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a4be00640 1 --2- 172.21.15.66:0/3267141499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a4c0cde70 0x7f8a4c0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a4be00640 1 -- 172.21.15.66:0/3267141499 >> v1:172.21.15.156:6789/0 conn(0x7f8a4c159c60 legacy=0x7f8a4c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a4be00640 1 -- 172.21.15.66:0/3267141499 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c0b11e0 con 0x7f8a4c0cda90 2024-03-30T00:43:08.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a4be00640 1 --2- 172.21.15.66:0/3267141499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4c0cda90 0x7f8a4c0ce520 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0097c0 tx=0x7f8a3c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9833db0c88171ec7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:08.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a4b400640 1 -- 172.21.15.66:0/3267141499 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0ab600 con 0x7f8a4c0cda90 2024-03-30T00:43:08.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a4b400640 1 -- 172.21.15.66:0/3267141499 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c0a7050 con 0x7f8a4c0cda90 2024-03-30T00:43:08.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.432+0000 7f8a4b400640 1 -- 172.21.15.66:0/3267141499 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0ab980 con 0x7f8a4c0cda90 2024-03-30T00:43:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 -- 172.21.15.66:0/3267141499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4c0cda90 msgr2=0x7f8a4c0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 --2- 172.21.15.66:0/3267141499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4c0cda90 0x7f8a4c0ce520 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0097c0 tx=0x7f8a3c01c410 comp rx=0 tx=0).stop 2024-03-30T00:43:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 -- 172.21.15.66:0/3267141499 shutdown_connections 2024-03-30T00:43:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 --2- 172.21.15.66:0/3267141499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a4c0cde70 0x7f8a4c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 --2- 172.21.15.66:0/3267141499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4c0cda90 0x7f8a4c0ce520 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 -- 172.21.15.66:0/3267141499 >> 172.21.15.66:0/3267141499 conn(0x7f8a4c069f60 msgr2=0x7f8a4c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 -- 172.21.15.66:0/3267141499 shutdown_connections 2024-03-30T00:43:08.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 -- 172.21.15.66:0/3267141499 wait complete. 2024-03-30T00:43:08.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 Processor -- start 2024-03-30T00:43:08.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 -- start start 2024-03-30T00:43:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4c0cda90 0x7f8a4c071b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a4c0cde70 0x7f8a4c0cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a4c159c60 0x7f8a4c0cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c158710 con 0x7f8a4c0cde70 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c072060 con 0x7f8a4c0cda90 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a52a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c156670 con 0x7f8a4c159c60 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a43400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a4c0cde70 0x7f8a4c0cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a43400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a4c0cde70 0x7f8a4c0cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34828/0 (socket says 172.21.15.66:34828) 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a43400640 1 -- 172.21.15.66:0/1448647363 learned_addr learned my addr 172.21.15.66:0/1448647363 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a4be00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4c0cda90 0x7f8a4c071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a50a00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a4c159c60 0x7f8a4c0cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a43400640 1 -- 172.21.15.66:0/1448647363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a4c159c60 msgr2=0x7f8a4c0cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a43400640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a4c159c60 0x7f8a4c0cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a43400640 1 -- 172.21.15.66:0/1448647363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4c0cda90 msgr2=0x7f8a4c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a43400640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4c0cda90 0x7f8a4c071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a43400640 1 -- 172.21.15.66:0/1448647363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c009420 con 0x7f8a4c0cde70 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.436+0000 7f8a4be00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4c0cda90 0x7f8a4c071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:08.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a43400640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a4c0cde70 0x7f8a4c0cb310 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400b240 tx=0x7f8a3400d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:08.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a48c00640 1 -- 172.21.15.66:0/1448647363 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34010440 con 0x7f8a4c0cde70 2024-03-30T00:43:08.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a48c00640 1 -- 172.21.15.66:0/1448647363 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a340105a0 con 0x7f8a4c0cde70 2024-03-30T00:43:08.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a48c00640 1 -- 172.21.15.66:0/1448647363 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34010950 con 0x7f8a4c0cde70 2024-03-30T00:43:08.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a52a00640 1 -- 172.21.15.66:0/1448647363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4c158900 con 0x7f8a4c0cde70 2024-03-30T00:43:08.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a52a00640 1 -- 172.21.15.66:0/1448647363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4c158ba0 con 0x7f8a4c0cde70 2024-03-30T00:43:08.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a52a00640 1 -- 172.21.15.66:0/1448647363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4c066c30 con 0x7f8a4c0cde70 2024-03-30T00:43:08.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a48c00640 1 -- 172.21.15.66:0/1448647363 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8a34010ab0 con 0x7f8a4c0cde70 2024-03-30T00:43:08.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a48c00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a28066690 0x7f8a28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a48c00640 1 -- 172.21.15.66:0/1448647363 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8a340848c0 con 0x7f8a4c0cde70 2024-03-30T00:43:08.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a4be00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a28066690 0x7f8a28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.440+0000 7f8a4be00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a28066690 0x7f8a28068b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0ad000 tx=0x7f8a3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:08.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.444+0000 7f8a48c00640 1 -- 172.21.15.66:0/1448647363 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8a34051da0 con 0x7f8a4c0cde70 2024-03-30T00:43:08.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.604+0000 7f8a52a00640 1 -- 172.21.15.66:0/1448647363 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a4c0b2710 con 0x7f8a4c0cde70 2024-03-30T00:43:08.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a48c00640 1 -- 172.21.15.66:0/1448647363 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+12594 (secure 0 0 0) 0x7f8a3407ce60 con 0x7f8a4c0cde70 2024-03-30T00:43:08.614 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:08.614 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":64,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:43:07.863971+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:43:08.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 -- 172.21.15.66:0/1448647363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a28066690 msgr2=0x7f8a28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a28066690 0x7f8a28068b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0ad000 tx=0x7f8a3c008040 comp rx=0 tx=0).stop 2024-03-30T00:43:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 -- 172.21.15.66:0/1448647363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a4c0cde70 msgr2=0x7f8a4c0cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a4c0cde70 0x7f8a4c0cb310 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400b240 tx=0x7f8a3400d350 comp rx=0 tx=0).stop 2024-03-30T00:43:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 -- 172.21.15.66:0/1448647363 shutdown_connections 2024-03-30T00:43:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a4c159c60 0x7f8a4c0cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a28066690 0x7f8a28068b50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a4c0cde70 0x7f8a4c0cb310 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 --2- 172.21.15.66:0/1448647363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4c0cda90 0x7f8a4c071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 -- 172.21.15.66:0/1448647363 >> 172.21.15.66:0/1448647363 conn(0x7f8a4c069f60 msgr2=0x7f8a4c15d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 -- 172.21.15.66:0/1448647363 shutdown_connections 2024-03-30T00:43:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.608+0000 7f8a52a00640 1 -- 172.21.15.66:0/1448647363 wait complete. 2024-03-30T00:43:08.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:43:08.635 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:43:08.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.772+0000 7f4d0aa00640 1 Processor -- start 2024-03-30T00:43:08.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.772+0000 7f4d0aa00640 1 -- start start 2024-03-30T00:43:08.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.772+0000 7f4d0aa00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4d0415ca80 0x7f4d0406b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.772+0000 7f4d0aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cf2f0 0x7f4d0406b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.772+0000 7f4d0aa00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d040cb0e0 con 0x7f4d0406bb30 2024-03-30T00:43:08.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.772+0000 7f4d0aa00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d040cf7f0 con 0x7f4d0415ca80 2024-03-30T00:43:08.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.772+0000 7f4d0aa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040cfad0 con 0x7f4d040cf2f0 2024-03-30T00:43:08.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.776+0000 7f4d03e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4d0415ca80 0x7f4d0406b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.776+0000 7f4cfb400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cf2f0 0x7f4d0406b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.776+0000 7f4d08a00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4d0406bb30 0x7f4d04098a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:45028/0 (socket says 172.21.15.66:45028) 2024-03-30T00:43:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.776+0000 7f4d08a00640 1 -- 172.21.15.66:0/910390370 learned_addr learned my addr 172.21.15.66:0/910390370 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.776+0000 7f4d03400640 1 -- 172.21.15.66:0/910390370 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894635064 0 0) 0x7f4d040cb0e0 con 0x7f4d0406bb30 2024-03-30T00:43:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.776+0000 7f4d03400640 1 -- 172.21.15.66:0/910390370 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce8003530 con 0x7f4d0406bb30 2024-03-30T00:43:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.776+0000 7f4cfb400640 1 -- 172.21.15.66:0/910390370 >> v2:172.21.15.156:3300/0 conn(0x7f4d0415ca80 msgr2=0x7f4d0406b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.776+0000 7f4cfb400640 1 --2- 172.21.15.66:0/910390370 >> v2:172.21.15.156:3300/0 conn(0x7f4d0415ca80 0x7f4d0406b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.776+0000 7f4cfb400640 1 -- 172.21.15.66:0/910390370 >> v1:172.21.15.156:6789/0 conn(0x7f4d0406bb30 legacy=0x7f4d04098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.776+0000 7f4cfb400640 1 -- 172.21.15.66:0/910390370 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040cad70 con 0x7f4d040cf2f0 2024-03-30T00:43:08.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.776+0000 7f4d03e00640 1 --2- 172.21.15.66:0/910390370 >> v2:172.21.15.156:3300/0 conn(0x7f4d0415ca80 0x7f4d0406b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:08.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4cfb400640 1 --2- 172.21.15.66:0/910390370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cf2f0 0x7f4d0406b5f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0097c0 tx=0x7f4cec01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=278a8f50556408fb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:08.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d03400640 1 -- 172.21.15.66:0/910390370 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec0a8990 con 0x7f4d040cf2f0 2024-03-30T00:43:08.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d03400640 1 -- 172.21.15.66:0/910390370 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cec0a8af0 con 0x7f4d040cf2f0 2024-03-30T00:43:08.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d03400640 1 -- 172.21.15.66:0/910390370 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec0a8e80 con 0x7f4d040cf2f0 2024-03-30T00:43:08.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 -- 172.21.15.66:0/910390370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cf2f0 msgr2=0x7f4d0406b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 --2- 172.21.15.66:0/910390370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cf2f0 0x7f4d0406b5f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0097c0 tx=0x7f4cec01ec90 comp rx=0 tx=0).stop 2024-03-30T00:43:08.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 -- 172.21.15.66:0/910390370 shutdown_connections 2024-03-30T00:43:08.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 --2- 172.21.15.66:0/910390370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cf2f0 0x7f4d0406b5f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 --2- 172.21.15.66:0/910390370 >> v2:172.21.15.156:3300/0 conn(0x7f4d0415ca80 0x7f4d0406b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 -- 172.21.15.66:0/910390370 >> 172.21.15.66:0/910390370 conn(0x7f4d040b2090 msgr2=0x7f4d040b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:08.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 -- 172.21.15.66:0/910390370 shutdown_connections 2024-03-30T00:43:08.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 -- 172.21.15.66:0/910390370 wait complete. 2024-03-30T00:43:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 Processor -- start 2024-03-30T00:43:08.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 -- start start 2024-03-30T00:43:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0406bb30 0x7f4d0409b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cf2f0 0x7f4d0409a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d0415ca80 0x7f4d0415a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d041f6330 con 0x7f4d0415ca80 2024-03-30T00:43:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040c8890 con 0x7f4d0406bb30 2024-03-30T00:43:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d0aa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d041f5460 con 0x7f4d040cf2f0 2024-03-30T00:43:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d08a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d0415ca80 0x7f4d0415a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d08a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d0415ca80 0x7f4d0415a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44074/0 (socket says 172.21.15.66:44074) 2024-03-30T00:43:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d03e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0406bb30 0x7f4d0409b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4d08a00640 1 -- 172.21.15.66:0/2724364335 learned_addr learned my addr 172.21.15.66:0/2724364335 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:08.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.780+0000 7f4cfb400640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cf2f0 0x7f4d0409a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d08a00640 1 -- 172.21.15.66:0/2724364335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cf2f0 msgr2=0x7f4d0409a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d08a00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cf2f0 0x7f4d0409a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d08a00640 1 -- 172.21.15.66:0/2724364335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0406bb30 msgr2=0x7f4d0409b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d08a00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0406bb30 0x7f4d0409b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d08a00640 1 -- 172.21.15.66:0/2724364335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf4009420 con 0x7f4d0415ca80 2024-03-30T00:43:08.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d08a00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d0415ca80 0x7f4d0415a120 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00cb70 tx=0x7f4cfc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:08.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d00c00640 1 -- 172.21.15.66:0/2724364335 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00ad40 con 0x7f4d0415ca80 2024-03-30T00:43:08.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d00c00640 1 -- 172.21.15.66:0/2724364335 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc012400 con 0x7f4d0415ca80 2024-03-30T00:43:08.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d00c00640 1 -- 172.21.15.66:0/2724364335 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00aea0 con 0x7f4d0415ca80 2024-03-30T00:43:08.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d0aa00640 1 -- 172.21.15.66:0/2724364335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cec009420 con 0x7f4d0415ca80 2024-03-30T00:43:08.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d0aa00640 1 -- 172.21.15.66:0/2724364335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d041f56b0 con 0x7f4d0415ca80 2024-03-30T00:43:08.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d0aa00640 1 -- 172.21.15.66:0/2724364335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d04066c30 con 0x7f4d0415ca80 2024-03-30T00:43:08.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d00c00640 1 -- 172.21.15.66:0/2724364335 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4cfc007070 con 0x7f4d0415ca80 2024-03-30T00:43:08.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d00c00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4ce0066690 0x7f4ce0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:08.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.784+0000 7f4d00c00640 1 -- 172.21.15.66:0/2724364335 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4cfc083f30 con 0x7f4d0415ca80 2024-03-30T00:43:08.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.788+0000 7f4d03e00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4ce0066690 0x7f4ce0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:08.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.788+0000 7f4d03e00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4ce0066690 0x7f4ce0068b50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009c50 tx=0x7f4cf4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:08.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.788+0000 7f4d00c00640 1 -- 172.21.15.66:0/2724364335 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4cfc051550 con 0x7f4d0415ca80 2024-03-30T00:43:08.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.948+0000 7f4d0aa00640 1 -- 172.21.15.66:0/2724364335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d040b2bb0 con 0x7f4d0415ca80 2024-03-30T00:43:08.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d00c00640 1 -- 172.21.15.66:0/2724364335 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4cfc0170a0 con 0x7f4d0415ca80 2024-03-30T00:43:08.957 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:43:08.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 -- 172.21.15.66:0/2724364335 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4ce0066690 msgr2=0x7f4ce0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4ce0066690 0x7f4ce0068b50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009c50 tx=0x7f4cf4009190 comp rx=0 tx=0).stop 2024-03-30T00:43:08.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 -- 172.21.15.66:0/2724364335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d0415ca80 msgr2=0x7f4d0415a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:08.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d0415ca80 0x7f4d0415a120 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00cb70 tx=0x7f4cfc00a430 comp rx=0 tx=0).stop 2024-03-30T00:43:08.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 -- 172.21.15.66:0/2724364335 shutdown_connections 2024-03-30T00:43:08.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4ce0066690 0x7f4ce0068b50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d0415ca80 0x7f4d0415a120 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cf2f0 0x7f4d0409a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 --2- 172.21.15.66:0/2724364335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0406bb30 0x7f4d0409b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:08.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 -- 172.21.15.66:0/2724364335 >> 172.21.15.66:0/2724364335 conn(0x7f4d040b2090 msgr2=0x7f4d040ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:08.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 -- 172.21.15.66:0/2724364335 shutdown_connections 2024-03-30T00:43:08.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:08.952+0000 7f4d0aa00640 1 -- 172.21.15.66:0/2724364335 wait complete. 2024-03-30T00:43:08.977 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T00:43:09.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6ac400640 1 Processor -- start 2024-03-30T00:43:09.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6ac400640 1 -- start start 2024-03-30T00:43:09.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6ac400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd6a40ae4e0 0x7fd6a40cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6ac400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a40ce820 0x7fd6a40cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6ac400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a406e7d0 con 0x7fd6a415ca10 2024-03-30T00:43:09.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6ac400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a40b1f70 con 0x7fd6a40ae4e0 2024-03-30T00:43:09.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6ac400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a40cf1f0 con 0x7fd6a40ce820 2024-03-30T00:43:09.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6a9000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd6a40ae4e0 0x7fd6a40cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6aa400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a40ce820 0x7fd6a40cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6a9000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd6a40ae4e0 0x7fd6a40cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:33800/0 (socket says 172.21.15.66:33800) 2024-03-30T00:43:09.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6aa400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a40ce820 0x7fd6a40cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:42826/0 (socket says 172.21.15.66:42826) 2024-03-30T00:43:09.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6a9000640 1 -- 172.21.15.66:0/1928379329 learned_addr learned my addr 172.21.15.66:0/1928379329 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:09.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.112+0000 7fd6a9a00640 1 --1- 172.21.15.66:0/1928379329 >> v1:172.21.15.156:6789/0 conn(0x7fd6a415ca10 0x7fd6a406ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:45030/0 (socket says 172.21.15.66:45030) 2024-03-30T00:43:09.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6aa400640 1 -- 172.21.15.66:0/1928379329 >> v2:172.21.15.156:3300/0 conn(0x7fd6a40ae4e0 msgr2=0x7fd6a40cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:09.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6aa400640 1 --2- 172.21.15.66:0/1928379329 >> v2:172.21.15.156:3300/0 conn(0x7fd6a40ae4e0 0x7fd6a40cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6aa400640 1 -- 172.21.15.66:0/1928379329 >> v1:172.21.15.156:6789/0 conn(0x7fd6a415ca10 legacy=0x7fd6a406ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6a3e00640 1 -- 172.21.15.66:0/1928379329 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690702154 0 0) 0x7fd6a406e7d0 con 0x7fd6a415ca10 2024-03-30T00:43:09.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6aa400640 1 -- 172.21.15.66:0/1928379329 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a415cdf0 con 0x7fd6a40ce820 2024-03-30T00:43:09.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6a9000640 1 --2- 172.21.15.66:0/1928379329 >> v2:172.21.15.156:3300/0 conn(0x7fd6a40ae4e0 0x7fd6a40cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:09.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6aa400640 1 --2- 172.21.15.66:0/1928379329 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a40ce820 0x7fd6a40cecb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009d90 tx=0x7fd69c01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b27f1580a0bfbced server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:09.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6a3e00640 1 -- 172.21.15.66:0/1928379329 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0a5a20 con 0x7fd6a40ce820 2024-03-30T00:43:09.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6a3e00640 1 -- 172.21.15.66:0/1928379329 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c0a5b80 con 0x7fd6a40ce820 2024-03-30T00:43:09.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6a3e00640 1 -- 172.21.15.66:0/1928379329 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0ad440 con 0x7fd6a40ce820 2024-03-30T00:43:09.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6ac400640 1 -- 172.21.15.66:0/1928379329 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a40ce820 msgr2=0x7fd6a40cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6ac400640 1 --2- 172.21.15.66:0/1928379329 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a40ce820 0x7fd6a40cecb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009d90 tx=0x7fd69c01c940 comp rx=0 tx=0).stop 2024-03-30T00:43:09.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6ac400640 1 -- 172.21.15.66:0/1928379329 shutdown_connections 2024-03-30T00:43:09.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6ac400640 1 --2- 172.21.15.66:0/1928379329 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a40ce820 0x7fd6a40cecb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6ac400640 1 --2- 172.21.15.66:0/1928379329 >> v2:172.21.15.156:3300/0 conn(0x7fd6a40ae4e0 0x7fd6a40cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6ac400640 1 -- 172.21.15.66:0/1928379329 >> 172.21.15.66:0/1928379329 conn(0x7fd6a4098a20 msgr2=0x7fd6a40b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:09.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6ac400640 1 -- 172.21.15.66:0/1928379329 shutdown_connections 2024-03-30T00:43:09.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.116+0000 7fd6ac400640 1 -- 172.21.15.66:0/1928379329 wait complete. 2024-03-30T00:43:09.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6ac400640 1 Processor -- start 2024-03-30T00:43:09.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6ac400640 1 -- start start 2024-03-30T00:43:09.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6ac400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a40ae4e0 0x7fd6a4152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6ac400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6a40ce820 0x7fd6a41573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6ac400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a415ca10 0x7fd6a4152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6ac400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a40c3180 con 0x7fd6a40ce820 2024-03-30T00:43:09.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6ac400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a4158480 con 0x7fd6a40ae4e0 2024-03-30T00:43:09.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6ac400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a4157be0 con 0x7fd6a415ca10 2024-03-30T00:43:09.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6a40ce820 0x7fd6a41573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6a40ce820 0x7fd6a41573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44086/0 (socket says 172.21.15.66:44086) 2024-03-30T00:43:09.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a9000640 1 -- 172.21.15.66:0/4158539854 learned_addr learned my addr 172.21.15.66:0/4158539854 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:09.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a9a00640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a40ae4e0 0x7fd6a4152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a9000640 1 -- 172.21.15.66:0/4158539854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a415ca10 msgr2=0x7fd6a4152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6aa400640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a415ca10 0x7fd6a4152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a9000640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a415ca10 0x7fd6a4152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a9000640 1 -- 172.21.15.66:0/4158539854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a40ae4e0 msgr2=0x7fd6a4152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a9000640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a40ae4e0 0x7fd6a4152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a9000640 1 -- 172.21.15.66:0/4158539854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6900094d0 con 0x7fd6a40ce820 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a9a00640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a40ae4e0 0x7fd6a4152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6aa400640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a415ca10 0x7fd6a4152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a9000640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6a40ce820 0x7fd6a41573b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd69000e9d0 tx=0x7fd69000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a1600640 1 -- 172.21.15.66:0/4158539854 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69000ca60 con 0x7fd6a40ce820 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a1600640 1 -- 172.21.15.66:0/4158539854 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69000cbc0 con 0x7fd6a40ce820 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6a1600640 1 -- 172.21.15.66:0/4158539854 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690013470 con 0x7fd6a40ce820 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6ac400640 1 -- 172.21.15.66:0/4158539854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c009a70 con 0x7fd6a40ce820 2024-03-30T00:43:09.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.120+0000 7fd6ac400640 1 -- 172.21.15.66:0/4158539854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a41581a0 con 0x7fd6a40ce820 2024-03-30T00:43:09.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.124+0000 7fd6ac400640 1 -- 172.21.15.66:0/4158539854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a4066c30 con 0x7fd6a40ce820 2024-03-30T00:43:09.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.124+0000 7fd6a1600640 1 -- 172.21.15.66:0/4158539854 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd6900135d0 con 0x7fd6a40ce820 2024-03-30T00:43:09.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.124+0000 7fd6a1600640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd684066690 0x7fd684068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.124+0000 7fd6a1600640 1 -- 172.21.15.66:0/4158539854 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd690084240 con 0x7fd6a40ce820 2024-03-30T00:43:09.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.124+0000 7fd6a9a00640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd684066690 0x7fd684068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.124+0000 7fd6a9a00640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd684066690 0x7fd684068b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd698007fd0 tx=0x7fd698009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:09.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.128+0000 7fd6a1600640 1 -- 172.21.15.66:0/4158539854 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd690051850 con 0x7fd6a40ce820 2024-03-30T00:43:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 -- 172.21.15.66:0/4158539854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd6a40b5430 con 0x7fd6a40ce820 2024-03-30T00:43:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6a1600640 1 -- 172.21.15.66:0/4158539854 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd690054e70 con 0x7fd6a40ce820 2024-03-30T00:43:09.297 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:09.298 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:43:09.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:09.298 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-03-30T00:43:09.298 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-03-30T00:43:09.298 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:09.298 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:43:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 -- 172.21.15.66:0/4158539854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd684066690 msgr2=0x7fd684068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd684066690 0x7fd684068b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd698007fd0 tx=0x7fd698009040 comp rx=0 tx=0).stop 2024-03-30T00:43:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 -- 172.21.15.66:0/4158539854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6a40ce820 msgr2=0x7fd6a41573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6a40ce820 0x7fd6a41573b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd69000e9d0 tx=0x7fd69000ee90 comp rx=0 tx=0).stop 2024-03-30T00:43:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 -- 172.21.15.66:0/4158539854 shutdown_connections 2024-03-30T00:43:09.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6a415ca10 0x7fd6a4152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd684066690 0x7fd684068b50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6a40ce820 0x7fd6a41573b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 --2- 172.21.15.66:0/4158539854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6a40ae4e0 0x7fd6a4152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 -- 172.21.15.66:0/4158539854 >> 172.21.15.66:0/4158539854 conn(0x7fd6a4098a20 msgr2=0x7fd6a40ad100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:09.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 -- 172.21.15.66:0/4158539854 shutdown_connections 2024-03-30T00:43:09.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.292+0000 7fd6ac400640 1 -- 172.21.15.66:0/4158539854 wait complete. 2024-03-30T00:43:09.318 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T00:43:09.318 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T00:43:09.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1591400640 1 Processor -- start 2024-03-30T00:43:09.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1591400640 1 -- start start 2024-03-30T00:43:09.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1591400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f158c0cbd30 0x7f158c15c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1591400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f158c0cb650 0x7f158c15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1591400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c0cbb50 con 0x7f158c06e7f0 2024-03-30T00:43:09.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1591400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c06a860 con 0x7f158c0cbd30 2024-03-30T00:43:09.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1591400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0cd340 con 0x7f158c0cb650 2024-03-30T00:43:09.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1583e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f158c0cbd30 0x7f158c15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1583e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f158c0cbd30 0x7f158c15c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44096/0 (socket says 172.21.15.66:44096) 2024-03-30T00:43:09.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1583e00640 1 -- 172.21.15.66:0/2011269689 learned_addr learned my addr 172.21.15.66:0/2011269689 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:09.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f158b200640 1 --2- 172.21.15.66:0/2011269689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f158c0cb650 0x7f158c15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1583e00640 1 -- 172.21.15.66:0/2011269689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f158c0cb650 msgr2=0x7f158c15c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:09.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1583e00640 1 --2- 172.21.15.66:0/2011269689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f158c0cb650 0x7f158c15c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1583e00640 1 -- 172.21.15.66:0/2011269689 >> v1:172.21.15.156:6789/0 conn(0x7f158c06e7f0 legacy=0x7f158c06ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:09.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.456+0000 7f1583e00640 1 -- 172.21.15.66:0/2011269689 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c069ec0 con 0x7f158c0cbd30 2024-03-30T00:43:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1583e00640 1 --2- 172.21.15.66:0/2011269689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f158c0cbd30 0x7f158c15c160 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1574009870 tx=0x7f157401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afb97d26b6cf474 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1589e00640 1 -- 172.21.15.66:0/2011269689 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15740ab600 con 0x7f158c0cbd30 2024-03-30T00:43:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1589e00640 1 -- 172.21.15.66:0/2011269689 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15740ab760 con 0x7f158c0cbd30 2024-03-30T00:43:09.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1589e00640 1 -- 172.21.15.66:0/2011269689 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15740abaf0 con 0x7f158c0cbd30 2024-03-30T00:43:09.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1591400640 1 -- 172.21.15.66:0/2011269689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f158c0cbd30 msgr2=0x7f158c15c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1591400640 1 --2- 172.21.15.66:0/2011269689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f158c0cbd30 0x7f158c15c160 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1574009870 tx=0x7f157401c460 comp rx=0 tx=0).stop 2024-03-30T00:43:09.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1591400640 1 -- 172.21.15.66:0/2011269689 shutdown_connections 2024-03-30T00:43:09.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1591400640 1 --2- 172.21.15.66:0/2011269689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f158c0cb650 0x7f158c15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1591400640 1 --2- 172.21.15.66:0/2011269689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f158c0cbd30 0x7f158c15c160 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1591400640 1 -- 172.21.15.66:0/2011269689 >> 172.21.15.66:0/2011269689 conn(0x7f158c0b1030 msgr2=0x7f158c0b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:09.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1591400640 1 -- 172.21.15.66:0/2011269689 shutdown_connections 2024-03-30T00:43:09.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.460+0000 7f1591400640 1 -- 172.21.15.66:0/2011269689 wait complete. 2024-03-30T00:43:09.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1591400640 1 Processor -- start 2024-03-30T00:43:09.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1591400640 1 -- start start 2024-03-30T00:43:09.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1591400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f158c06e7f0 0x7f158c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1591400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f158c0cb650 0x7f158c0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1591400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f158c0cbd30 0x7f158c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1591400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0b3410 con 0x7f158c0cb650 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1591400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0b3770 con 0x7f158c06e7f0 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1591400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0b3a10 con 0x7f158c0cbd30 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1583e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f158c0cb650 0x7f158c0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1583e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f158c0cb650 0x7f158c0b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44108/0 (socket says 172.21.15.66:44108) 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1583e00640 1 -- 172.21.15.66:0/1861307039 learned_addr learned my addr 172.21.15.66:0/1861307039 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f158a800640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f158c06e7f0 0x7f158c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f158b200640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f158c0cbd30 0x7f158c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1583e00640 1 -- 172.21.15.66:0/1861307039 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f158c0cbd30 msgr2=0x7f158c06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1583e00640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f158c0cbd30 0x7f158c06d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1583e00640 1 -- 172.21.15.66:0/1861307039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f158c06e7f0 msgr2=0x7f158c0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1583e00640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f158c06e7f0 0x7f158c0b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1583e00640 1 -- 172.21.15.66:0/1861307039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15740094d0 con 0x7f158c0cb650 2024-03-30T00:43:09.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f158b200640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f158c0cbd30 0x7f158c06d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:09.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1583e00640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f158c0cb650 0x7f158c0b2ed0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f15740ad040 tx=0x7f1574019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:09.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1582a00640 1 -- 172.21.15.66:0/1861307039 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15740b5810 con 0x7f158c0cb650 2024-03-30T00:43:09.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1582a00640 1 -- 172.21.15.66:0/1861307039 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15740b59b0 con 0x7f158c0cb650 2024-03-30T00:43:09.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1582a00640 1 -- 172.21.15.66:0/1861307039 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15740b5d60 con 0x7f158c0cb650 2024-03-30T00:43:09.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1591400640 1 -- 172.21.15.66:0/1861307039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158c0b3cf0 con 0x7f158c0cb650 2024-03-30T00:43:09.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.464+0000 7f1591400640 1 -- 172.21.15.66:0/1861307039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c0b42b0 con 0x7f158c0cb650 2024-03-30T00:43:09.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.468+0000 7f1591400640 1 -- 172.21.15.66:0/1861307039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c066c30 con 0x7f158c0cb650 2024-03-30T00:43:09.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.468+0000 7f1582a00640 1 -- 172.21.15.66:0/1861307039 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f15740c93f0 con 0x7f158c0cb650 2024-03-30T00:43:09.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.468+0000 7f1582a00640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1568066690 0x7f1568068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.468+0000 7f1582a00640 1 -- 172.21.15.66:0/1861307039 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f157411eb50 con 0x7f158c0cb650 2024-03-30T00:43:09.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.468+0000 7f158a800640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1568066690 0x7f1568068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.468+0000 7f158a800640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1568066690 0x7f1568068b50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f157c0098e0 tx=0x7f157c009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:09.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.472+0000 7f1582a00640 1 -- 172.21.15.66:0/1861307039 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f15740ec290 con 0x7f158c0cb650 2024-03-30T00:43:09.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.636+0000 7f1591400640 1 -- 172.21.15.66:0/1861307039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f158c0b5460 con 0x7f158c0cb650 2024-03-30T00:43:09.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.732+0000 7f1582a00640 1 -- 172.21.15.66:0/1861307039 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f15740c40b0 con 0x7f158c0cb650 2024-03-30T00:43:09.741 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-03-30T00:43:09.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 -- 172.21.15.66:0/1861307039 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1568066690 msgr2=0x7f1568068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1568066690 0x7f1568068b50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f157c0098e0 tx=0x7f157c009060 comp rx=0 tx=0).stop 2024-03-30T00:43:09.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 -- 172.21.15.66:0/1861307039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f158c0cb650 msgr2=0x7f158c0b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f158c0cb650 0x7f158c0b2ed0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f15740ad040 tx=0x7f1574019200 comp rx=0 tx=0).stop 2024-03-30T00:43:09.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 -- 172.21.15.66:0/1861307039 shutdown_connections 2024-03-30T00:43:09.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f158c0cbd30 0x7f158c06d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1568066690 0x7f1568068b50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f158c0cb650 0x7f158c0b2ed0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 --2- 172.21.15.66:0/1861307039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f158c06e7f0 0x7f158c0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 -- 172.21.15.66:0/1861307039 >> 172.21.15.66:0/1861307039 conn(0x7f158c0b1030 msgr2=0x7f158c0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:09.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 -- 172.21.15.66:0/1861307039 shutdown_connections 2024-03-30T00:43:09.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.736+0000 7f1591400640 1 -- 172.21.15.66:0/1861307039 wait complete. 2024-03-30T00:43:09.767 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T00:43:09.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a7400640 1 Processor -- start 2024-03-30T00:43:09.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a7400640 1 -- start start 2024-03-30T00:43:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a7400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00af960 0x7fe1a00b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a7400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe1a00b0430 0x7fe1a00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a7400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a00affa0 con 0x7fe1a0098a20 2024-03-30T00:43:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a7400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a0099bd0 con 0x7fe1a00b0430 2024-03-30T00:43:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a7400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00afd40 con 0x7fe1a00af960 2024-03-30T00:43:09.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00af960 0x7fe1a00b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a5400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe1a00b0430 0x7fe1a00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00af960 0x7fe1a00b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:42846/0 (socket says 172.21.15.66:42846) 2024-03-30T00:43:09.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a4a00640 1 -- 172.21.15.66:0/2675568169 learned_addr learned my addr 172.21.15.66:0/2675568169 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:09.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe197e00640 1 --1- 172.21.15.66:0/2675568169 >> v1:172.21.15.156:6789/0 conn(0x7fe1a0098a20 0x7fe1a0098e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:45036/0 (socket says 172.21.15.66:45036) 2024-03-30T00:43:09.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a4a00640 1 -- 172.21.15.66:0/2675568169 >> v2:172.21.15.156:3300/0 conn(0x7fe1a00b0430 msgr2=0x7fe1a00b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:09.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a4a00640 1 --2- 172.21.15.66:0/2675568169 >> v2:172.21.15.156:3300/0 conn(0x7fe1a00b0430 0x7fe1a00b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe19fe00640 1 -- 172.21.15.66:0/2675568169 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221549565 0 0) 0x7fe1a00affa0 con 0x7fe1a0098a20 2024-03-30T00:43:09.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a4a00640 1 -- 172.21.15.66:0/2675568169 >> v1:172.21.15.156:6789/0 conn(0x7fe1a0098a20 legacy=0x7fe1a0098e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a4a00640 1 -- 172.21.15.66:0/2675568169 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00cf5b0 con 0x7fe1a00af960 2024-03-30T00:43:09.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.908+0000 7fe1a5400640 1 --2- 172.21.15.66:0/2675568169 >> v2:172.21.15.156:3300/0 conn(0x7fe1a00b0430 0x7fe1a00b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe1a4a00640 1 --2- 172.21.15.66:0/2675568169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00af960 0x7fe1a00b2c50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe190006fd0 tx=0x7fe19001ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6690251169eea9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe19fe00640 1 -- 172.21.15.66:0/2675568169 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1900a8a30 con 0x7fe1a00af960 2024-03-30T00:43:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe19fe00640 1 -- 172.21.15.66:0/2675568169 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1900a8b90 con 0x7fe1a00af960 2024-03-30T00:43:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe19fe00640 1 -- 172.21.15.66:0/2675568169 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1900b0440 con 0x7fe1a00af960 2024-03-30T00:43:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe1a7400640 1 -- 172.21.15.66:0/2675568169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00af960 msgr2=0x7fe1a00b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe1a7400640 1 --2- 172.21.15.66:0/2675568169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00af960 0x7fe1a00b2c50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe190006fd0 tx=0x7fe19001ed30 comp rx=0 tx=0).stop 2024-03-30T00:43:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe1a7400640 1 -- 172.21.15.66:0/2675568169 shutdown_connections 2024-03-30T00:43:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe1a7400640 1 --2- 172.21.15.66:0/2675568169 >> v2:172.21.15.156:3300/0 conn(0x7fe1a00b0430 0x7fe1a00b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe1a7400640 1 --2- 172.21.15.66:0/2675568169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00af960 0x7fe1a00b2c50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe1a7400640 1 -- 172.21.15.66:0/2675568169 >> 172.21.15.66:0/2675568169 conn(0x7fe1a00b2130 msgr2=0x7fe1a00b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe1a7400640 1 -- 172.21.15.66:0/2675568169 shutdown_connections 2024-03-30T00:43:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.912+0000 7fe1a7400640 1 -- 172.21.15.66:0/2675568169 wait complete. 2024-03-30T00:43:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a7400640 1 Processor -- start 2024-03-30T00:43:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a7400640 1 -- start start 2024-03-30T00:43:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1a0098a20 0x7fe1a00ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a7400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a00af960 0x7fe1a015d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a7400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00b0430 0x7fe1a015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a7400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00c8810 con 0x7fe1a0098a20 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a7400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00c77b0 con 0x7fe1a00af960 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a7400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00c7a50 con 0x7fe1a00b0430 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1a0098a20 0x7fe1a00ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1a0098a20 0x7fe1a00ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44110/0 (socket says 172.21.15.66:44110) 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a4a00640 1 -- 172.21.15.66:0/793707433 learned_addr learned my addr 172.21.15.66:0/793707433 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a5400640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00b0430 0x7fe1a015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe197e00640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a00af960 0x7fe1a015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a4a00640 1 -- 172.21.15.66:0/793707433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00b0430 msgr2=0x7fe1a015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a4a00640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00b0430 0x7fe1a015dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a4a00640 1 -- 172.21.15.66:0/793707433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a00af960 msgr2=0x7fe1a015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a4a00640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a00af960 0x7fe1a015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a4a00640 1 -- 172.21.15.66:0/793707433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe198009a70 con 0x7fe1a0098a20 2024-03-30T00:43:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a5400640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00b0430 0x7fe1a015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe197e00640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a00af960 0x7fe1a015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a4a00640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1a0098a20 0x7fe1a00ca460 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe190009930 tx=0x7fe19001bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe19d600640 1 -- 172.21.15.66:0/793707433 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190004230 con 0x7fe1a0098a20 2024-03-30T00:43:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe19d600640 1 -- 172.21.15.66:0/793707433 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe190004390 con 0x7fe1a0098a20 2024-03-30T00:43:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe19d600640 1 -- 172.21.15.66:0/793707433 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190003820 con 0x7fe1a0098a20 2024-03-30T00:43:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a7400640 1 -- 172.21.15.66:0/793707433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe190009480 con 0x7fe1a0098a20 2024-03-30T00:43:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.916+0000 7fe1a7400640 1 -- 172.21.15.66:0/793707433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a00c8010 con 0x7fe1a0098a20 2024-03-30T00:43:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.920+0000 7fe1a7400640 1 -- 172.21.15.66:0/793707433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a0066c30 con 0x7fe1a0098a20 2024-03-30T00:43:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.920+0000 7fe19d600640 1 -- 172.21.15.66:0/793707433 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe190003980 con 0x7fe1a0098a20 2024-03-30T00:43:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.920+0000 7fe19d600640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe17c066690 0x7fe17c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.920+0000 7fe19d600640 1 -- 172.21.15.66:0/793707433 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fe1901220f0 con 0x7fe1a0098a20 2024-03-30T00:43:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.920+0000 7fe197e00640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe17c066690 0x7fe17c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.920+0000 7fe197e00640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe17c066690 0x7fe17c068b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00ce180 tx=0x7fe188007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:09.924+0000 7fe19d600640 1 -- 172.21.15.66:0/793707433 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe1900ef4f0 con 0x7fe1a0098a20 2024-03-30T00:43:10.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.088+0000 7fe1a7400640 1 -- 172.21.15.66:0/793707433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe1a00cded0 con 0x7fe1a0098a20 2024-03-30T00:43:10.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.736+0000 7fe19d600640 1 -- 172.21.15.66:0/793707433 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v66) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe1900f2b10 con 0x7fe1a0098a20 2024-03-30T00:43:10.745 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-03-30T00:43:10.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.744+0000 7fe1a7400640 1 -- 172.21.15.66:0/793707433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe17c066690 msgr2=0x7fe17c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:10.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.744+0000 7fe1a7400640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe17c066690 0x7fe17c068b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00ce180 tx=0x7fe188007cb0 comp rx=0 tx=0).stop 2024-03-30T00:43:10.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.744+0000 7fe1a7400640 1 -- 172.21.15.66:0/793707433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1a0098a20 msgr2=0x7fe1a00ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:10.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.744+0000 7fe1a7400640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1a0098a20 0x7fe1a00ca460 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe190009930 tx=0x7fe19001bad0 comp rx=0 tx=0).stop 2024-03-30T00:43:10.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.748+0000 7fe1a7400640 1 -- 172.21.15.66:0/793707433 shutdown_connections 2024-03-30T00:43:10.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.748+0000 7fe1a7400640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1a00b0430 0x7fe1a015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:10.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.748+0000 7fe1a7400640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a00af960 0x7fe1a015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:10.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.748+0000 7fe1a7400640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe17c066690 0x7fe17c068b50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:10.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.748+0000 7fe1a7400640 1 --2- 172.21.15.66:0/793707433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1a0098a20 0x7fe1a00ca460 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:10.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.748+0000 7fe1a7400640 1 -- 172.21.15.66:0/793707433 >> 172.21.15.66:0/793707433 conn(0x7fe1a00b2130 msgr2=0x7fe1a0155690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:10.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.748+0000 7fe1a7400640 1 -- 172.21.15.66:0/793707433 shutdown_connections 2024-03-30T00:43:10.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.748+0000 7fe1a7400640 1 -- 172.21.15.66:0/793707433 wait complete. 2024-03-30T00:43:10.787 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T00:43:10.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.924+0000 7fb263a00640 1 Processor -- start 2024-03-30T00:43:10.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.924+0000 7fb263a00640 1 -- start start 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb263a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb25c157060 0x7fb25c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb263a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb25c1610c0 0x7fb25c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb263a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25c06dc60 con 0x7fb25c1609f0 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb263a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c157440 con 0x7fb25c157060 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb263a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c160ea0 con 0x7fb25c1610c0 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb261000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb25c157060 0x7fb25c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb261000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb25c157060 0x7fb25c06d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44118/0 (socket says 172.21.15.66:44118) 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb261000640 1 -- 172.21.15.66:0/2836845282 learned_addr learned my addr 172.21.15.66:0/2836845282 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb253e00640 1 --2- 172.21.15.66:0/2836845282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb25c1610c0 0x7fb25c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb261000640 1 -- 172.21.15.66:0/2836845282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb25c1610c0 msgr2=0x7fb25c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb261000640 1 --2- 172.21.15.66:0/2836845282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb25c1610c0 0x7fb25c1604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb261000640 1 -- 172.21.15.66:0/2836845282 >> v1:172.21.15.156:6789/0 conn(0x7fb25c1609f0 legacy=0x7fb25c15cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:10.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.928+0000 7fb261000640 1 -- 172.21.15.66:0/2836845282 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c06df40 con 0x7fb25c157060 2024-03-30T00:43:10.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb261000640 1 --2- 172.21.15.66:0/2836845282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c157060 0x7fb25c06d310 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009870 tx=0x7fb24c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb02e4f28a06f89 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:10.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb25be00640 1 -- 172.21.15.66:0/2836845282 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c15dde0 con 0x7fb25c157060 2024-03-30T00:43:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb25be00640 1 -- 172.21.15.66:0/2836845282 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c0a7050 con 0x7fb25c157060 2024-03-30T00:43:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb25be00640 1 -- 172.21.15.66:0/2836845282 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c0ab820 con 0x7fb25c157060 2024-03-30T00:43:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 -- 172.21.15.66:0/2836845282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c157060 msgr2=0x7fb25c06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 --2- 172.21.15.66:0/2836845282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c157060 0x7fb25c06d310 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009870 tx=0x7fb24c01c400 comp rx=0 tx=0).stop 2024-03-30T00:43:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 -- 172.21.15.66:0/2836845282 shutdown_connections 2024-03-30T00:43:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 --2- 172.21.15.66:0/2836845282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb25c1610c0 0x7fb25c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 --2- 172.21.15.66:0/2836845282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c157060 0x7fb25c06d310 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 -- 172.21.15.66:0/2836845282 >> 172.21.15.66:0/2836845282 conn(0x7fb25c071750 msgr2=0x7fb25c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 -- 172.21.15.66:0/2836845282 shutdown_connections 2024-03-30T00:43:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 -- 172.21.15.66:0/2836845282 wait complete. 2024-03-30T00:43:10.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 Processor -- start 2024-03-30T00:43:10.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 -- start start 2024-03-30T00:43:10.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c157060 0x7fb25c159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:10.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb25c1609f0 0x7fb25c152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:10.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb25c1610c0 0x7fb25c157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:10.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b2a60 con 0x7fb25c157060 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b2c90 con 0x7fb25c1609f0 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb263a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0b2f70 con 0x7fb25c1610c0 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb261000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c157060 0x7fb25c159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb261000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c157060 0x7fb25c159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44126/0 (socket says 172.21.15.66:44126) 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.932+0000 7fb261000640 1 -- 172.21.15.66:0/3565429999 learned_addr learned my addr 172.21.15.66:0/3565429999 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb261000640 1 -- 172.21.15.66:0/3565429999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb25c1610c0 msgr2=0x7fb25c157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb261000640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb25c1610c0 0x7fb25c157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb261000640 1 -- 172.21.15.66:0/3565429999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb25c1609f0 msgr2=0x7fb25c152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb253e00640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb25c1609f0 0x7fb25c152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb261000640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb25c1609f0 0x7fb25c152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb261000640 1 -- 172.21.15.66:0/3565429999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c0094d0 con 0x7fb25c157060 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb253e00640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb25c1609f0 0x7fb25c152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb261000640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c157060 0x7fb25c159f20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb24c01a030 tx=0x7fb24c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:10.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb259600640 1 -- 172.21.15.66:0/3565429999 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c0b5400 con 0x7fb25c157060 2024-03-30T00:43:10.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb259600640 1 -- 172.21.15.66:0/3565429999 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c0b5580 con 0x7fb25c157060 2024-03-30T00:43:10.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb259600640 1 -- 172.21.15.66:0/3565429999 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c0b5930 con 0x7fb25c157060 2024-03-30T00:43:10.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb263a00640 1 -- 172.21.15.66:0/3565429999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25c14f250 con 0x7fb25c157060 2024-03-30T00:43:10.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb263a00640 1 -- 172.21.15.66:0/3565429999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c14f810 con 0x7fb25c157060 2024-03-30T00:43:10.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb263a00640 1 -- 172.21.15.66:0/3565429999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c0b3140 con 0x7fb25c157060 2024-03-30T00:43:10.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb259600640 1 -- 172.21.15.66:0/3565429999 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb24c0b5a90 con 0x7fb25c157060 2024-03-30T00:43:10.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.936+0000 7fb259600640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb238066690 0x7fb238068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:10.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.940+0000 7fb259600640 1 -- 172.21.15.66:0/3565429999 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fb24c11e2f0 con 0x7fb25c157060 2024-03-30T00:43:10.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.940+0000 7fb253e00640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb238066690 0x7fb238068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:10.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.940+0000 7fb253e00640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb238066690 0x7fb238068b50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb25c167770 tx=0x7fb244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:10.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:10.944+0000 7fb259600640 1 -- 172.21.15.66:0/3565429999 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb24c0eb570 con 0x7fb25c157060 2024-03-30T00:43:11.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.136+0000 7fb263a00640 1 -- 172.21.15.66:0/3565429999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb25c066c30 con 0x7fb25c157060 2024-03-30T00:43:11.151 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 9 and data pool 10 2024-03-30T00:43:11.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.144+0000 7fb259600640 1 -- 172.21.15.66:0/3565429999 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v29) v1 ==== 173+0+0 (secure 0 0 0) 0x7fb24c0eeb90 con 0x7fb25c157060 2024-03-30T00:43:11.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.148+0000 7fb263a00640 1 -- 172.21.15.66:0/3565429999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb238066690 msgr2=0x7fb238068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.152+0000 7fb263a00640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb238066690 0x7fb238068b50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb25c167770 tx=0x7fb244008040 comp rx=0 tx=0).stop 2024-03-30T00:43:11.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.152+0000 7fb263a00640 1 -- 172.21.15.66:0/3565429999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c157060 msgr2=0x7fb25c159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.152+0000 7fb263a00640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c157060 0x7fb25c159f20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb24c01a030 tx=0x7fb24c0191a0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.152+0000 7fb263a00640 1 -- 172.21.15.66:0/3565429999 shutdown_connections 2024-03-30T00:43:11.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.152+0000 7fb263a00640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb25c1610c0 0x7fb25c157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.152+0000 7fb263a00640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb25c1609f0 0x7fb25c152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.152+0000 7fb263a00640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb238066690 0x7fb238068b50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.152+0000 7fb263a00640 1 --2- 172.21.15.66:0/3565429999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb25c157060 0x7fb25c159f20 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.152+0000 7fb263a00640 1 -- 172.21.15.66:0/3565429999 >> 172.21.15.66:0/3565429999 conn(0x7fb25c071750 msgr2=0x7fb25c150790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:11.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.152+0000 7fb263a00640 1 -- 172.21.15.66:0/3565429999 shutdown_connections 2024-03-30T00:43:11.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.152+0000 7fb263a00640 1 -- 172.21.15.66:0/3565429999 wait complete. 2024-03-30T00:43:11.193 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:43:11.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8157e00640 1 Processor -- start 2024-03-30T00:43:11.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8157e00640 1 -- start start 2024-03-30T00:43:11.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8157e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f815815c9f0 0x7f815806ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8157e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 0x7f815806b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8157e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81580ce070 con 0x7f815806b8b0 2024-03-30T00:43:11.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8157e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f815806bd60 con 0x7f815815c9f0 2024-03-30T00:43:11.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8157e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815806cf60 con 0x7f81580b2b60 2024-03-30T00:43:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8156000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 0x7f815806b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8156a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f815815c9f0 0x7f815806ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8156000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 0x7f815806b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:42868/0 (socket says 172.21.15.66:42868) 2024-03-30T00:43:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8156000640 1 -- 172.21.15.66:0/3593436038 learned_addr learned my addr 172.21.15.66:0/3593436038 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8156a00640 1 -- 172.21.15.66:0/3593436038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 msgr2=0x7f815806b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8156a00640 1 --2- 172.21.15.66:0/3593436038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 0x7f815806b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8156a00640 1 -- 172.21.15.66:0/3593436038 >> v1:172.21.15.156:6789/0 conn(0x7f815806b8b0 legacy=0x7f8158160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8155600640 1 -- 172.21.15.66:0/3593436038 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2354776817 0 0) 0x7f81580ce070 con 0x7f815806b8b0 2024-03-30T00:43:11.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8156a00640 1 -- 172.21.15.66:0/3593436038 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81580ce350 con 0x7f815815c9f0 2024-03-30T00:43:11.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.328+0000 7f8156000640 1 --2- 172.21.15.66:0/3593436038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 0x7f815806b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:11.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8156a00640 1 --2- 172.21.15.66:0/3593436038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815815c9f0 0x7f815806ca20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f81480097f0 tx=0x7f814801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=306768a96873b272 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:11.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8155600640 1 -- 172.21.15.66:0/3593436038 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81480ab670 con 0x7f815815c9f0 2024-03-30T00:43:11.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8155600640 1 -- 172.21.15.66:0/3593436038 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81480ab7d0 con 0x7f815815c9f0 2024-03-30T00:43:11.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8155600640 1 -- 172.21.15.66:0/3593436038 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81480abb60 con 0x7f815815c9f0 2024-03-30T00:43:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8157e00640 1 -- 172.21.15.66:0/3593436038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815815c9f0 msgr2=0x7f815806ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8157e00640 1 --2- 172.21.15.66:0/3593436038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815815c9f0 0x7f815806ca20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f81480097f0 tx=0x7f814801c470 comp rx=0 tx=0).stop 2024-03-30T00:43:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8157e00640 1 -- 172.21.15.66:0/3593436038 shutdown_connections 2024-03-30T00:43:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8157e00640 1 --2- 172.21.15.66:0/3593436038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 0x7f815806b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8157e00640 1 --2- 172.21.15.66:0/3593436038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815815c9f0 0x7f815806ca20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8157e00640 1 -- 172.21.15.66:0/3593436038 >> 172.21.15.66:0/3593436038 conn(0x7f815806e0d0 msgr2=0x7f815806e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:11.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8157e00640 1 -- 172.21.15.66:0/3593436038 shutdown_connections 2024-03-30T00:43:11.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.332+0000 7f8157e00640 1 -- 172.21.15.66:0/3593436038 wait complete. 2024-03-30T00:43:11.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157e00640 1 Processor -- start 2024-03-30T00:43:11.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157e00640 1 -- start start 2024-03-30T00:43:11.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815806b8b0 0x7f815815b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 0x7f81580b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f815815c9f0 0x7f8158154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81580c6430 con 0x7f815815c9f0 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815815bf10 con 0x7f815806b8b0 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81580b5750 con 0x7f81580b2b60 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f815815c9f0 0x7f8158154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f815815c9f0 0x7f8158154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44134/0 (socket says 172.21.15.66:44134) 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157400640 1 -- 172.21.15.66:0/2652982161 learned_addr learned my addr 172.21.15.66:0/2652982161 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8156000640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 0x7f81580b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8156a00640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815806b8b0 0x7f815815b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157400640 1 -- 172.21.15.66:0/2652982161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 msgr2=0x7f81580b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157400640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 0x7f81580b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157400640 1 -- 172.21.15.66:0/2652982161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815806b8b0 msgr2=0x7f815815b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157400640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815806b8b0 0x7f815815b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157400640 1 -- 172.21.15.66:0/2652982161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81440094d0 con 0x7f815815c9f0 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8156000640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 0x7f81580b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:11.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157400640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f815815c9f0 0x7f8158154930 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f815000c7b0 tx=0x7f815000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:11.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f814ea00640 1 -- 172.21.15.66:0/2652982161 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150003db0 con 0x7f815815c9f0 2024-03-30T00:43:11.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f814ea00640 1 -- 172.21.15.66:0/2652982161 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8150005070 con 0x7f815815c9f0 2024-03-30T00:43:11.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f814ea00640 1 -- 172.21.15.66:0/2652982161 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150005420 con 0x7f815815c9f0 2024-03-30T00:43:11.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157e00640 1 -- 172.21.15.66:0/2652982161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8148009420 con 0x7f815815c9f0 2024-03-30T00:43:11.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157e00640 1 -- 172.21.15.66:0/2652982161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81580c6610 con 0x7f815815c9f0 2024-03-30T00:43:11.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.336+0000 7f8157e00640 1 -- 172.21.15.66:0/2652982161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8158073cf0 con 0x7f815815c9f0 2024-03-30T00:43:11.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.340+0000 7f814ea00640 1 -- 172.21.15.66:0/2652982161 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8150005580 con 0x7f815815c9f0 2024-03-30T00:43:11.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.340+0000 7f814ea00640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8138066690 0x7f8138068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.340+0000 7f814ea00640 1 -- 172.21.15.66:0/2652982161 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f81500840d0 con 0x7f815815c9f0 2024-03-30T00:43:11.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.340+0000 7f8156a00640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8138066690 0x7f8138068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.340+0000 7f8156a00640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8138066690 0x7f8138068b50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f81480af000 tx=0x7f81480a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:11.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.344+0000 7f814ea00640 1 -- 172.21.15.66:0/2652982161 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8150051270 con 0x7f815815c9f0 2024-03-30T00:43:11.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.504+0000 7f8157e00640 1 -- 172.21.15.66:0/2652982161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81580cda50 con 0x7f815815c9f0 2024-03-30T00:43:11.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.504+0000 7f814ea00640 1 -- 172.21.15.66:0/2652982161 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16261 (secure 0 0 0) 0x7f8150054890 con 0x7f815815c9f0 2024-03-30T00:43:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:11.512 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":67,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:43:11.140707+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:43:09.681968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T00:43:10.133411+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:43:11.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 -- 172.21.15.66:0/2652982161 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8138066690 msgr2=0x7f8138068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8138066690 0x7f8138068b50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f81480af000 tx=0x7f81480a7000 comp rx=0 tx=0).stop 2024-03-30T00:43:11.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 -- 172.21.15.66:0/2652982161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f815815c9f0 msgr2=0x7f8158154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f815815c9f0 0x7f8158154930 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f815000c7b0 tx=0x7f815000cc70 comp rx=0 tx=0).stop 2024-03-30T00:43:11.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 -- 172.21.15.66:0/2652982161 shutdown_connections 2024-03-30T00:43:11.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8138066690 0x7f8138068b50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f815815c9f0 0x7f8158154930 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81580b2b60 0x7f81580b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 --2- 172.21.15.66:0/2652982161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815806b8b0 0x7f815815b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 -- 172.21.15.66:0/2652982161 >> 172.21.15.66:0/2652982161 conn(0x7f815806e0d0 msgr2=0x7f815806e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:11.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 -- 172.21.15.66:0/2652982161 shutdown_connections 2024-03-30T00:43:11.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.508+0000 7f8157e00640 1 -- 172.21.15.66:0/2652982161 wait complete. 2024-03-30T00:43:11.534 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:43:11.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe59f000640 1 Processor -- start 2024-03-30T00:43:11.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe59f000640 1 -- start start 2024-03-30T00:43:11.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe59f000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59815de30 0x7fe5980cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe59f000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe5980cee50 0x7fe5980d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe59f000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5980cbb90 con 0x7fe5980af420 2024-03-30T00:43:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe59f000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5980ce1b0 con 0x7fe5980cee50 2024-03-30T00:43:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe59f000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980afa10 con 0x7fe59815de30 2024-03-30T00:43:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe58fe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe5980af420 0x7fe5980ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60898/0 (socket says 172.21.15.66:60898) 2024-03-30T00:43:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe58fe00640 1 -- 172.21.15.66:0/3203904774 learned_addr learned my addr 172.21.15.66:0/3203904774 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597e00640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59815de30 0x7fe5980cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe59d000640 1 --2- 172.21.15.66:0/3203904774 >> v2:172.21.15.156:3300/0 conn(0x7fe5980cee50 0x7fe5980d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251542647 0 0) 0x7fe5980cbb90 con 0x7fe5980af420 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe57c003530 con 0x7fe5980af420 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2015305679 0 0) 0x7fe57c003530 con 0x7fe5980af420 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59815de30 msgr2=0x7fe5980cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59815de30 0x7fe5980cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 >> v2:172.21.15.156:3300/0 conn(0x7fe5980cee50 msgr2=0x7fe5980d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 --2- 172.21.15.66:0/3203904774 >> v2:172.21.15.156:3300/0 conn(0x7fe5980cee50 0x7fe5980d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5980cf430 con 0x7fe5980af420 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fe5800016a0 con 0x7fe5980af420 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 >> v1:172.21.15.66:6789/0 conn(0x7fe5980af420 legacy=0x7fe5980ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe57c003980 0x7fe57c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0047d0 0x7fe57c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59815de30 0x7fe57c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980ce1b0 con 0x7fe57c0047d0 2024-03-30T00:43:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980afa10 con 0x7fe57c003980 2024-03-30T00:43:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.668+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c00b9e0 con 0x7fe59815de30 2024-03-30T00:43:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe58fe00640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0047d0 0x7fe57c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe597e00640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe57c003980 0x7fe57c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe59d000640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59815de30 0x7fe57c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe58fe00640 1 -- 172.21.15.66:0/3203904774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59815de30 msgr2=0x7fe57c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe58fe00640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59815de30 0x7fe57c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe58fe00640 1 -- 172.21.15.66:0/3203904774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe57c003980 msgr2=0x7fe57c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe58fe00640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe57c003980 0x7fe57c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe58fe00640 1 -- 172.21.15.66:0/3203904774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c003530 con 0x7fe57c0047d0 2024-03-30T00:43:11.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe58fe00640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0047d0 0x7fe57c004c50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe58000c190 tx=0x7fe58001cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401c3589f78e1e9a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:11.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5800058e0 con 0x7fe57c0047d0 2024-03-30T00:43:11.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe580005a40 con 0x7fe57c0047d0 2024-03-30T00:43:11.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.672+0000 7fe597400640 1 -- 172.21.15.66:0/3203904774 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580005df0 con 0x7fe57c0047d0 2024-03-30T00:43:11.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 -- 172.21.15.66:0/3203904774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0047d0 msgr2=0x7fe57c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 --2- 172.21.15.66:0/3203904774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0047d0 0x7fe57c004c50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe58000c190 tx=0x7fe58001cb80 comp rx=0 tx=0).stop 2024-03-30T00:43:11.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59d000640 1 -- 172.21.15.66:0/3203904774 reap_dead start 2024-03-30T00:43:11.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 -- 172.21.15.66:0/3203904774 shutdown_connections 2024-03-30T00:43:11.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 -- 172.21.15.66:0/3203904774 >> 172.21.15.66:0/3203904774 conn(0x7fe5980d0540 msgr2=0x7fe5980d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:11.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 -- 172.21.15.66:0/3203904774 shutdown_connections 2024-03-30T00:43:11.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 -- 172.21.15.66:0/3203904774 wait complete. 2024-03-30T00:43:11.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 Processor -- start 2024-03-30T00:43:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 -- start start 2024-03-30T00:43:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe57c0047d0 0x7fe5981f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59814b0f0 0x7fe598147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5981f3810 0x7fe598158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598148130 con 0x7fe5981f3810 2024-03-30T00:43:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598149050 con 0x7fe57c0047d0 2024-03-30T00:43:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59f000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe598148440 con 0x7fe59814b0f0 2024-03-30T00:43:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5981f3810 0x7fe598158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5981f3810 0x7fe598158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44156/0 (socket says 172.21.15.66:44156) 2024-03-30T00:43:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59d000640 1 -- 172.21.15.66:0/1778114657 learned_addr learned my addr 172.21.15.66:0/1778114657 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe597e00640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe57c0047d0 0x7fe5981f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe58fe00640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59814b0f0 0x7fe598147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59d000640 1 -- 172.21.15.66:0/1778114657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59814b0f0 msgr2=0x7fe598147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59d000640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59814b0f0 0x7fe598147bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59d000640 1 -- 172.21.15.66:0/1778114657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe57c0047d0 msgr2=0x7fe5981f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59d000640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe57c0047d0 0x7fe5981f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59d000640 1 -- 172.21.15.66:0/1778114657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe598148720 con 0x7fe5981f3810 2024-03-30T00:43:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.676+0000 7fe59d000640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5981f3810 0x7fe598158a40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe5980700a0 tx=0x7fe59000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.680+0000 7fe594c00640 1 -- 172.21.15.66:0/1778114657 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000e040 con 0x7fe5981f3810 2024-03-30T00:43:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.680+0000 7fe594c00640 1 -- 172.21.15.66:0/1778114657 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59000f040 con 0x7fe5981f3810 2024-03-30T00:43:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.680+0000 7fe59f000640 1 -- 172.21.15.66:0/1778114657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe598148a00 con 0x7fe5981f3810 2024-03-30T00:43:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.680+0000 7fe594c00640 1 -- 172.21.15.66:0/1778114657 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590015670 con 0x7fe5981f3810 2024-03-30T00:43:11.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.680+0000 7fe59f000640 1 -- 172.21.15.66:0/1778114657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5981f3cc0 con 0x7fe5981f3810 2024-03-30T00:43:11.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.680+0000 7fe59f000640 1 -- 172.21.15.66:0/1778114657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5980cdf60 con 0x7fe5981f3810 2024-03-30T00:43:11.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.680+0000 7fe594c00640 1 -- 172.21.15.66:0/1778114657 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe5900157d0 con 0x7fe5981f3810 2024-03-30T00:43:11.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.680+0000 7fe594c00640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe574066690 0x7fe574068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:11.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.680+0000 7fe594c00640 1 -- 172.21.15.66:0/1778114657 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe590084b30 con 0x7fe5981f3810 2024-03-30T00:43:11.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.680+0000 7fe597e00640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe574066690 0x7fe574068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:11.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.680+0000 7fe597e00640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe574066690 0x7fe574068b50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe59809e5a0 tx=0x7fe588008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:11.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.684+0000 7fe594c00640 1 -- 172.21.15.66:0/1778114657 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe590051cc0 con 0x7fe5981f3810 2024-03-30T00:43:11.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.848+0000 7fe59f000640 1 -- 172.21.15.66:0/1778114657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5981f3f10 con 0x7fe5981f3810 2024-03-30T00:43:11.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.848+0000 7fe594c00640 1 -- 172.21.15.66:0/1778114657 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16261 (secure 0 0 0) 0x7fe5900552e0 con 0x7fe5981f3810 2024-03-30T00:43:11.855 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:11.855 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":67,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:43:11.140707+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:43:09.681968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T00:43:10.133411+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:43:11.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 -- 172.21.15.66:0/1778114657 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe574066690 msgr2=0x7fe574068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe574066690 0x7fe574068b50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe59809e5a0 tx=0x7fe588008040 comp rx=0 tx=0).stop 2024-03-30T00:43:11.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 -- 172.21.15.66:0/1778114657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5981f3810 msgr2=0x7fe598158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:11.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5981f3810 0x7fe598158a40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe5980700a0 tx=0x7fe59000f580 comp rx=0 tx=0).stop 2024-03-30T00:43:11.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 -- 172.21.15.66:0/1778114657 shutdown_connections 2024-03-30T00:43:11.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe574066690 0x7fe574068b50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5981f3810 0x7fe598158a40 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe59814b0f0 0x7fe598147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 --2- 172.21.15.66:0/1778114657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe57c0047d0 0x7fe5981f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:11.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 -- 172.21.15.66:0/1778114657 >> 172.21.15.66:0/1778114657 conn(0x7fe5980d0540 msgr2=0x7fe59815c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:11.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 -- 172.21.15.66:0/1778114657 shutdown_connections 2024-03-30T00:43:11.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:11.852+0000 7fe59f000640 1 -- 172.21.15.66:0/1778114657 wait complete. 2024-03-30T00:43:11.878 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T00:43:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.012+0000 7f19e9400640 1 Processor -- start 2024-03-30T00:43:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.012+0000 7f19e9400640 1 -- start start 2024-03-30T00:43:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e9400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f19e415a640 0x7f19e406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e9400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e40cb2c0 0x7f19e40ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e9400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e406d070 con 0x7f19e40ce520 2024-03-30T00:43:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e9400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e40cbba0 con 0x7f19e415a640 2024-03-30T00:43:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e9400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e40cee90 con 0x7f19e40cb2c0 2024-03-30T00:43:12.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e3200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e40cb2c0 0x7f19e40ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:12.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19d9e00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f19e40ce520 0x7f19e40cabb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:45080/0 (socket says 172.21.15.66:45080) 2024-03-30T00:43:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19d9e00640 1 -- 172.21.15.66:0/1642261300 learned_addr learned my addr 172.21.15.66:0/1642261300 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e2800640 1 --2- 172.21.15.66:0/1642261300 >> v2:172.21.15.156:3300/0 conn(0x7f19e415a640 0x7f19e406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e1e00640 1 -- 172.21.15.66:0/1642261300 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164688159 0 0) 0x7f19e406d070 con 0x7f19e40ce520 2024-03-30T00:43:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e1e00640 1 -- 172.21.15.66:0/1642261300 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19c8003530 con 0x7f19e40ce520 2024-03-30T00:43:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e3200640 1 -- 172.21.15.66:0/1642261300 >> v2:172.21.15.156:3300/0 conn(0x7f19e415a640 msgr2=0x7f19e406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e3200640 1 --2- 172.21.15.66:0/1642261300 >> v2:172.21.15.156:3300/0 conn(0x7f19e415a640 0x7f19e406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e3200640 1 -- 172.21.15.66:0/1642261300 >> v1:172.21.15.156:6789/0 conn(0x7f19e40ce520 legacy=0x7f19e40cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e3200640 1 -- 172.21.15.66:0/1642261300 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e406e2c0 con 0x7f19e40cb2c0 2024-03-30T00:43:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.016+0000 7f19e2800640 1 --2- 172.21.15.66:0/1642261300 >> v2:172.21.15.156:3300/0 conn(0x7f19e415a640 0x7f19e406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:12.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e3200640 1 --2- 172.21.15.66:0/1642261300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e40cb2c0 0x7f19e40ce950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009d90 tx=0x7f19dc01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f271b43045aef3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:12.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e1e00640 1 -- 172.21.15.66:0/1642261300 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0a5920 con 0x7f19e40cb2c0 2024-03-30T00:43:12.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e1e00640 1 -- 172.21.15.66:0/1642261300 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc0a5a80 con 0x7f19e40cb2c0 2024-03-30T00:43:12.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e1e00640 1 -- 172.21.15.66:0/1642261300 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0a5df0 con 0x7f19e40cb2c0 2024-03-30T00:43:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e9400640 1 -- 172.21.15.66:0/1642261300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e40cb2c0 msgr2=0x7f19e40ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e9400640 1 --2- 172.21.15.66:0/1642261300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e40cb2c0 0x7f19e40ce950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009d90 tx=0x7f19dc01c970 comp rx=0 tx=0).stop 2024-03-30T00:43:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e9400640 1 -- 172.21.15.66:0/1642261300 shutdown_connections 2024-03-30T00:43:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e9400640 1 --2- 172.21.15.66:0/1642261300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e40cb2c0 0x7f19e40ce950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e9400640 1 --2- 172.21.15.66:0/1642261300 >> v2:172.21.15.156:3300/0 conn(0x7f19e415a640 0x7f19e406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e9400640 1 -- 172.21.15.66:0/1642261300 >> 172.21.15.66:0/1642261300 conn(0x7f19e4069f60 msgr2=0x7f19e406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e9400640 1 -- 172.21.15.66:0/1642261300 shutdown_connections 2024-03-30T00:43:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e9400640 1 -- 172.21.15.66:0/1642261300 wait complete. 2024-03-30T00:43:12.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e9400640 1 Processor -- start 2024-03-30T00:43:12.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.020+0000 7f19e9400640 1 -- start start 2024-03-30T00:43:12.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19e9400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e40cb2c0 0x7f19e4156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:12.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19e9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ce520 0x7f19e40c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:12.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19e9400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e415a640 0x7f19e4070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:12.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19e9400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e40cb0e0 con 0x7f19e40ce520 2024-03-30T00:43:12.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19e9400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e40ca3d0 con 0x7f19e415a640 2024-03-30T00:43:12.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19e9400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e40cdef0 con 0x7f19e40cb2c0 2024-03-30T00:43:12.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19d9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ce520 0x7f19e40c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:12.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19e2800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e40cb2c0 0x7f19e4156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:12.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19d9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ce520 0x7f19e40c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44170/0 (socket says 172.21.15.66:44170) 2024-03-30T00:43:12.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19d9e00640 1 -- 172.21.15.66:0/2536827146 learned_addr learned my addr 172.21.15.66:0/2536827146 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:12.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19e3200640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e415a640 0x7f19e4070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19d9e00640 1 -- 172.21.15.66:0/2536827146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e40cb2c0 msgr2=0x7f19e4156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19d9e00640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e40cb2c0 0x7f19e4156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19d9e00640 1 -- 172.21.15.66:0/2536827146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e415a640 msgr2=0x7f19e4070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19d9e00640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e415a640 0x7f19e4070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19d9e00640 1 -- 172.21.15.66:0/2536827146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4009420 con 0x7f19e40ce520 2024-03-30T00:43:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19d9e00640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ce520 0x7f19e40c8900 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c4d0 tx=0x7f19cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19db400640 1 -- 172.21.15.66:0/2536827146 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc005cf0 con 0x7f19e40ce520 2024-03-30T00:43:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19db400640 1 -- 172.21.15.66:0/2536827146 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc005e50 con 0x7f19e40ce520 2024-03-30T00:43:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19e9400640 1 -- 172.21.15.66:0/2536827146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19dc009a70 con 0x7f19e40ce520 2024-03-30T00:43:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19db400640 1 -- 172.21.15.66:0/2536827146 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc007270 con 0x7f19e40ce520 2024-03-30T00:43:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19e9400640 1 -- 172.21.15.66:0/2536827146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e40ca600 con 0x7f19e40ce520 2024-03-30T00:43:12.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19e9400640 1 -- 172.21.15.66:0/2536827146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e4066c30 con 0x7f19e40ce520 2024-03-30T00:43:12.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.024+0000 7f19db400640 1 -- 172.21.15.66:0/2536827146 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f19cc0073d0 con 0x7f19e40ce520 2024-03-30T00:43:12.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.028+0000 7f19db400640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19c0066690 0x7f19c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:12.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.028+0000 7f19db400640 1 -- 172.21.15.66:0/2536827146 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f19cc0837b0 con 0x7f19e40ce520 2024-03-30T00:43:12.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.028+0000 7f19e2800640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19c0066690 0x7f19c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:12.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.028+0000 7f19e2800640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19c0066690 0x7f19c0068b50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009c50 tx=0x7f19d4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:12.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.032+0000 7f19db400640 1 -- 172.21.15.66:0/2536827146 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f19cc050a70 con 0x7f19e40ce520 2024-03-30T00:43:12.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:12.228+0000 7f19e9400640 1 -- 172.21.15.66:0/2536827146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f19e406dbd0 con 0x7f19e40ce520 2024-03-30T00:43:12.687 INFO:tasks.ceph.mds.d.smithi156.stderr:2024-03-30T00:43:12.676+0000 7f6209e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:43:13.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.152+0000 7f19db400640 1 -- 172.21.15.66:0/2536827146 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f19cc054090 con 0x7f19e40ce520 2024-03-30T00:43:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 -- 172.21.15.66:0/2536827146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19c0066690 msgr2=0x7f19c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19c0066690 0x7f19c0068b50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009c50 tx=0x7f19d4009190 comp rx=0 tx=0).stop 2024-03-30T00:43:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 -- 172.21.15.66:0/2536827146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ce520 msgr2=0x7f19e40c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ce520 0x7f19e40c8900 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c4d0 tx=0x7f19cc00c990 comp rx=0 tx=0).stop 2024-03-30T00:43:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 -- 172.21.15.66:0/2536827146 shutdown_connections 2024-03-30T00:43:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19e415a640 0x7f19e4070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19c0066690 0x7f19c0068b50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e40ce520 0x7f19e40c8900 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 --2- 172.21.15.66:0/2536827146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19e40cb2c0 0x7f19e4156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 -- 172.21.15.66:0/2536827146 >> 172.21.15.66:0/2536827146 conn(0x7f19e4069f60 msgr2=0x7f19e406a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 -- 172.21.15.66:0/2536827146 shutdown_connections 2024-03-30T00:43:13.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.156+0000 7f19e9400640 1 -- 172.21.15.66:0/2536827146 wait complete. 2024-03-30T00:43:13.185 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T00:43:13.185 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T00:43:13.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.324+0000 7f643ae00640 1 Processor -- start 2024-03-30T00:43:13.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f643ae00640 1 -- start start 2024-03-30T00:43:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f643ae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6434157060 0x7f643406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f643ae00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f64341610c0 0x7f64341604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f643ae00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643406dc60 con 0x7f64341609f0 2024-03-30T00:43:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f643ae00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6434157440 con 0x7f64341610c0 2024-03-30T00:43:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f643ae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6434160ea0 con 0x7f6434157060 2024-03-30T00:43:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6438e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f64341609f0 0x7f643415cec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60906/0 (socket says 172.21.15.66:60906) 2024-03-30T00:43:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6438e00640 1 -- 172.21.15.66:0/3390220873 learned_addr learned my addr 172.21.15.66:0/3390220873 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433e00640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6434157060 0x7f643406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f642b400640 1 --2- 172.21.15.66:0/3390220873 >> v2:172.21.15.156:3300/0 conn(0x7f64341610c0 0x7f64341604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173743417 0 0) 0x7f643406dc60 con 0x7f64341609f0 2024-03-30T00:43:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6418003530 con 0x7f64341609f0 2024-03-30T00:43:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2171610541 0 0) 0x7f6418003530 con 0x7f64341609f0 2024-03-30T00:43:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6434157060 msgr2=0x7f643406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6434157060 0x7f643406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 >> v2:172.21.15.156:3300/0 conn(0x7f64341610c0 msgr2=0x7f64341604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 --2- 172.21.15.66:0/3390220873 >> v2:172.21.15.156:3300/0 conn(0x7f64341610c0 0x7f64341604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643406df40 con 0x7f64341609f0 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f642b400640 1 --2- 172.21.15.66:0/3390220873 >> v2:172.21.15.156:3300/0 conn(0x7f64341610c0 0x7f64341604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f642c001680 con 0x7f64341609f0 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 >> v1:172.21.15.66:6789/0 conn(0x7f64341609f0 legacy=0x7f643415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6434157060 0x7f6418003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6418003f50 0x7f64180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6418004d10 0x7f6418005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6434157440 con 0x7f6418004d10 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6434160ea0 con 0x7f6418003f50 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641800bac0 con 0x7f6434157060 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6438e00640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6418004d10 0x7f6418005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433e00640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6434157060 0x7f6418003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f642b400640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6418003f50 0x7f64180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6438e00640 1 -- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6434157060 msgr2=0x7f6418003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6438e00640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6434157060 0x7f6418003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6438e00640 1 -- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6418003f50 msgr2=0x7f64180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6438e00640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6418003f50 0x7f64180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6438e00640 1 -- 172.21.15.66:0/3390220873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6418003530 con 0x7f6418004d10 2024-03-30T00:43:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.328+0000 7f6433e00640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6434157060 0x7f6418003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.332+0000 7f6438e00640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6418004d10 0x7f6418005190 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c190 tx=0x7f642c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceeee9f192c9990f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.332+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0058e0 con 0x7f6418004d10 2024-03-30T00:43:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.332+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c005a60 con 0x7f6418004d10 2024-03-30T00:43:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.332+0000 7f6433400640 1 -- 172.21.15.66:0/3390220873 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c005e50 con 0x7f6418004d10 2024-03-30T00:43:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.332+0000 7f643ae00640 1 -- 172.21.15.66:0/3390220873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6418004d10 msgr2=0x7f6418005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.332+0000 7f643ae00640 1 --2- 172.21.15.66:0/3390220873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6418004d10 0x7f6418005190 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c190 tx=0x7f642c01cc40 comp rx=0 tx=0).stop 2024-03-30T00:43:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.332+0000 7f6438e00640 1 -- 172.21.15.66:0/3390220873 reap_dead start 2024-03-30T00:43:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.332+0000 7f643ae00640 1 -- 172.21.15.66:0/3390220873 shutdown_connections 2024-03-30T00:43:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.332+0000 7f643ae00640 1 -- 172.21.15.66:0/3390220873 >> 172.21.15.66:0/3390220873 conn(0x7f6434071750 msgr2=0x7f6434069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.332+0000 7f643ae00640 1 -- 172.21.15.66:0/3390220873 shutdown_connections 2024-03-30T00:43:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.332+0000 7f643ae00640 1 -- 172.21.15.66:0/3390220873 wait complete. 2024-03-30T00:43:13.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f643ae00640 1 Processor -- start 2024-03-30T00:43:13.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f643ae00640 1 -- start start 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f643ae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64341454c0 0x7f6434154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f643ae00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6434146050 0x7f643415bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f643ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6434157580 0x7f643415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f643ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6434158d80 con 0x7f6434157580 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f643ae00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64340b3220 con 0x7f6434146050 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f643ae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64340b2910 con 0x7f64341454c0 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6438e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6434157580 0x7f643415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6438e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6434157580 0x7f643415c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44192/0 (socket says 172.21.15.66:44192) 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6438e00640 1 -- 172.21.15.66:0/3744853467 learned_addr learned my addr 172.21.15.66:0/3744853467 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f642b400640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6434146050 0x7f643415bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6433e00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64341454c0 0x7f6434154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6438e00640 1 -- 172.21.15.66:0/3744853467 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64341454c0 msgr2=0x7f6434154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:13.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6438e00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64341454c0 0x7f6434154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:13.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6438e00640 1 -- 172.21.15.66:0/3744853467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6434146050 msgr2=0x7f643415bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:13.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6438e00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6434146050 0x7f643415bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:13.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6438e00640 1 -- 172.21.15.66:0/3744853467 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6434157a80 con 0x7f6434157580 2024-03-30T00:43:13.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6438e00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6434157580 0x7f643415c460 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f642c0ab040 tx=0x7f6418005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:13.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6430c00640 1 -- 172.21.15.66:0/3744853467 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c001640 con 0x7f6434157580 2024-03-30T00:43:13.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6430c00640 1 -- 172.21.15.66:0/3744853467 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c001b80 con 0x7f6434157580 2024-03-30T00:43:13.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f6430c00640 1 -- 172.21.15.66:0/3744853467 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c004ab0 con 0x7f6434157580 2024-03-30T00:43:13.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f643ae00640 1 -- 172.21.15.66:0/3744853467 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64340b2b60 con 0x7f6434157580 2024-03-30T00:43:13.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f643ae00640 1 -- 172.21.15.66:0/3744853467 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6434148e20 con 0x7f6434157580 2024-03-30T00:43:13.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.336+0000 7f643ae00640 1 -- 172.21.15.66:0/3744853467 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64340b3360 con 0x7f6434157580 2024-03-30T00:43:13.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.340+0000 7f6430c00640 1 -- 172.21.15.66:0/3744853467 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f642c004c10 con 0x7f6434157580 2024-03-30T00:43:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.340+0000 7f6430c00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6410066690 0x7f6410068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:13.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.340+0000 7f6430c00640 1 -- 172.21.15.66:0/3744853467 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f642c11d5b0 con 0x7f6434157580 2024-03-30T00:43:13.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.340+0000 7f6433e00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6410066690 0x7f6410068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:13.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.340+0000 7f6433e00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6410066690 0x7f6410068b50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f64240144d0 tx=0x7f642400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:13.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.344+0000 7f6430c00640 1 -- 172.21.15.66:0/3744853467 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f642c0ea880 con 0x7f6434157580 2024-03-30T00:43:13.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:13.536+0000 7f643ae00640 1 -- 172.21.15.66:0/3744853467 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6434066c30 con 0x7f6434157580 2024-03-30T00:43:14.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f6430c00640 1 -- 172.21.15.66:0/3744853467 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f642c0edea0 con 0x7f6434157580 2024-03-30T00:43:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 -- 172.21.15.66:0/3744853467 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6410066690 msgr2=0x7f6410068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6410066690 0x7f6410068b50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f64240144d0 tx=0x7f642400b040 comp rx=0 tx=0).stop 2024-03-30T00:43:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 -- 172.21.15.66:0/3744853467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6434157580 msgr2=0x7f643415c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6434157580 0x7f643415c460 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f642c0ab040 tx=0x7f6418005800 comp rx=0 tx=0).stop 2024-03-30T00:43:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 -- 172.21.15.66:0/3744853467 shutdown_connections 2024-03-30T00:43:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6410066690 0x7f6410068b50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6434157580 0x7f643415c460 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6434146050 0x7f643415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 --2- 172.21.15.66:0/3744853467 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64341454c0 0x7f6434154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 -- 172.21.15.66:0/3744853467 >> 172.21.15.66:0/3744853467 conn(0x7f6434071750 msgr2=0x7f6434069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:14.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 -- 172.21.15.66:0/3744853467 shutdown_connections 2024-03-30T00:43:14.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.160+0000 7f643ae00640 1 -- 172.21.15.66:0/3744853467 wait complete. 2024-03-30T00:43:14.188 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T00:43:14.188 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:43:14.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.320+0000 7f90baa00640 1 Processor -- start 2024-03-30T00:43:14.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90baa00640 1 -- start start 2024-03-30T00:43:14.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90baa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90b40cee50 0x7f90b40cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:14.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90baa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b40cbb90 con 0x7f90b40af420 2024-03-30T00:43:14.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90baa00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b40ce1b0 con 0x7f90b415de30 2024-03-30T00:43:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90baa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40afa10 con 0x7f90b40cee50 2024-03-30T00:43:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90b8a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f90b40af420 0x7f90b406e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60918/0 (socket says 172.21.15.66:60918) 2024-03-30T00:43:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90b8a00640 1 -- 172.21.15.66:0/2380993209 learned_addr learned my addr 172.21.15.66:0/2380993209 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90abe00640 1 --2- 172.21.15.66:0/2380993209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90b40cee50 0x7f90b40cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90b3400640 1 -- 172.21.15.66:0/2380993209 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764852660 0 0) 0x7f90b40cbb90 con 0x7f90b40af420 2024-03-30T00:43:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90b3400640 1 -- 172.21.15.66:0/2380993209 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9098003530 con 0x7f90b40af420 2024-03-30T00:43:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90b3400640 1 -- 172.21.15.66:0/2380993209 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672536092 0 0) 0x7f90b40ce1b0 con 0x7f90b415de30 2024-03-30T00:43:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90abe00640 1 -- 172.21.15.66:0/2380993209 >> v1:172.21.15.156:6789/0 conn(0x7f90b415de30 legacy=0x7f90b40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90abe00640 1 -- 172.21.15.66:0/2380993209 >> v1:172.21.15.66:6789/0 conn(0x7f90b40af420 legacy=0x7f90b406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.324+0000 7f90abe00640 1 -- 172.21.15.66:0/2380993209 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b40cf430 con 0x7f90b40cee50 2024-03-30T00:43:14.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90abe00640 1 --2- 172.21.15.66:0/2380993209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90b40cee50 0x7f90b40cc040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f909c0097f0 tx=0x7f909c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ef03da1a0ff4ce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90b3400640 1 -- 172.21.15.66:0/2380993209 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c0ad660 con 0x7f90b40cee50 2024-03-30T00:43:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90b3400640 1 -- 172.21.15.66:0/2380993209 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c0ad7c0 con 0x7f90b40cee50 2024-03-30T00:43:14.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90b3400640 1 -- 172.21.15.66:0/2380993209 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c0adb30 con 0x7f90b40cee50 2024-03-30T00:43:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90baa00640 1 -- 172.21.15.66:0/2380993209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90b40cee50 msgr2=0x7f90b40cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90baa00640 1 --2- 172.21.15.66:0/2380993209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90b40cee50 0x7f90b40cc040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f909c0097f0 tx=0x7f909c01c450 comp rx=0 tx=0).stop 2024-03-30T00:43:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90baa00640 1 -- 172.21.15.66:0/2380993209 shutdown_connections 2024-03-30T00:43:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90baa00640 1 --2- 172.21.15.66:0/2380993209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90b40cee50 0x7f90b40cc040 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90baa00640 1 -- 172.21.15.66:0/2380993209 >> 172.21.15.66:0/2380993209 conn(0x7f90b40d0540 msgr2=0x7f90b40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90baa00640 1 -- 172.21.15.66:0/2380993209 shutdown_connections 2024-03-30T00:43:14.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90baa00640 1 -- 172.21.15.66:0/2380993209 wait complete. 2024-03-30T00:43:14.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90baa00640 1 Processor -- start 2024-03-30T00:43:14.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.328+0000 7f90baa00640 1 -- start start 2024-03-30T00:43:14.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90baa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90b40af420 0x7f90b40ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:14.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90baa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90b40cee50 0x7f90b40b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:14.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90baa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90b415de30 0x7f90b41f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:14.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90baa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40c8ff0 con 0x7f90b40cee50 2024-03-30T00:43:14.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90baa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40b2370 con 0x7f90b415de30 2024-03-30T00:43:14.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90baa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40b2610 con 0x7f90b40af420 2024-03-30T00:43:14.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90b40cee50 0x7f90b40b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:14.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90b40cee50 0x7f90b40b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44196/0 (socket says 172.21.15.66:44196) 2024-03-30T00:43:14.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90abe00640 1 -- 172.21.15.66:0/1114485064 learned_addr learned my addr 172.21.15.66:0/1114485064 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:14.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90b8a00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90b415de30 0x7f90b41f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:14.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90b3e00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90b40af420 0x7f90b40ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:14.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90abe00640 1 -- 172.21.15.66:0/1114485064 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90b40af420 msgr2=0x7f90b40ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90abe00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90b40af420 0x7f90b40ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90abe00640 1 -- 172.21.15.66:0/1114485064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90b415de30 msgr2=0x7f90b41f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90abe00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90b415de30 0x7f90b41f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90abe00640 1 -- 172.21.15.66:0/1114485064 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c009420 con 0x7f90b40cee50 2024-03-30T00:43:14.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90abe00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90b40cee50 0x7f90b40b5430 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f909c000c10 tx=0x7f909c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:14.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90b0c00640 1 -- 172.21.15.66:0/1114485064 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c0adb30 con 0x7f90b40cee50 2024-03-30T00:43:14.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90b0c00640 1 -- 172.21.15.66:0/1114485064 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c0adc90 con 0x7f90b40cee50 2024-03-30T00:43:14.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90b0c00640 1 -- 172.21.15.66:0/1114485064 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c003620 con 0x7f90b40cee50 2024-03-30T00:43:14.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90baa00640 1 -- 172.21.15.66:0/1114485064 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b40b28f0 con 0x7f90b40cee50 2024-03-30T00:43:14.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90baa00640 1 -- 172.21.15.66:0/1114485064 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b40ca560 con 0x7f90b40cee50 2024-03-30T00:43:14.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.332+0000 7f90baa00640 1 -- 172.21.15.66:0/1114485064 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b40cdf60 con 0x7f90b40cee50 2024-03-30T00:43:14.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.336+0000 7f90b0c00640 1 -- 172.21.15.66:0/1114485064 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f909c0b4050 con 0x7f90b40cee50 2024-03-30T00:43:14.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.336+0000 7f90b0c00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9090066690 0x7f9090068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:14.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.336+0000 7f90b0c00640 1 -- 172.21.15.66:0/1114485064 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f909c11e9a0 con 0x7f90b40cee50 2024-03-30T00:43:14.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.336+0000 7f90b3e00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9090066690 0x7f9090068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:14.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.336+0000 7f90b3e00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9090066690 0x7f9090068b50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f90b415d850 tx=0x7f90a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:14.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.340+0000 7f90b0c00640 1 -- 172.21.15.66:0/1114485064 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f909c0ebbc0 con 0x7f90b40cee50 2024-03-30T00:43:14.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.532+0000 7f90baa00640 1 -- 172.21.15.66:0/1114485064 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90b4066c30 con 0x7f90b40cee50 2024-03-30T00:43:14.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.532+0000 7f90b0c00640 1 -- 172.21.15.66:0/1114485064 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f909c0b2090 con 0x7f90b40cee50 2024-03-30T00:43:14.540 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:14.540 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4272961638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4272961638},{"type":"v1","addr":"172.21.15.156:6834","nonce":4272961638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2144186622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2144186622},{"type":"v1","addr":"172.21.15.66:6835","nonce":2144186622}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/750308788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":750308788},{"type":"v1","addr":"172.21.15.66:6837","nonce":750308788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:43:11.141056+0000","modified":"2024-03-30T00:43:14.154117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6468":{"gid":6468,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/1137593870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1137593870},{"type":"v1","addr":"172.21.15.156:6835","nonce":1137593870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-30T00:43:14.540 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 33 2024-03-30T00:43:14.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 -- 172.21.15.66:0/1114485064 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9090066690 msgr2=0x7f9090068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9090066690 0x7f9090068b50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f90b415d850 tx=0x7f90a4009040 comp rx=0 tx=0).stop 2024-03-30T00:43:14.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 -- 172.21.15.66:0/1114485064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90b40cee50 msgr2=0x7f90b40b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90b40cee50 0x7f90b40b5430 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f909c000c10 tx=0x7f909c0191f0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 -- 172.21.15.66:0/1114485064 shutdown_connections 2024-03-30T00:43:14.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90b415de30 0x7f90b41f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9090066690 0x7f9090068b50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90b40cee50 0x7f90b40b5430 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 --2- 172.21.15.66:0/1114485064 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90b40af420 0x7f90b40ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 -- 172.21.15.66:0/1114485064 >> 172.21.15.66:0/1114485064 conn(0x7f90b40d0540 msgr2=0x7f90b40ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:14.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 -- 172.21.15.66:0/1114485064 shutdown_connections 2024-03-30T00:43:14.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.536+0000 7f90baa00640 1 -- 172.21.15.66:0/1114485064 wait complete. 2024-03-30T00:43:14.562 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:43:14.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe65d800640 1 Processor -- start 2024-03-30T00:43:14.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe65d800640 1 -- start start 2024-03-30T00:43:14.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe65d800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe65815de30 0x7fe6580cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:14.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe65d800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6580cee50 0x7fe6580d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:14.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe65d800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6580cbb90 con 0x7fe6580af420 2024-03-30T00:43:14.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe65d800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6580ce1b0 con 0x7fe65815de30 2024-03-30T00:43:14.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe65d800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580afa10 con 0x7fe6580cee50 2024-03-30T00:43:14.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe656a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe65815de30 0x7fe6580cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:14.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe656a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe65815de30 0x7fe6580cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44208/0 (socket says 172.21.15.66:44208) 2024-03-30T00:43:14.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe656a00640 1 -- 172.21.15.66:0/880796801 learned_addr learned my addr 172.21.15.66:0/880796801 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:14.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe657400640 1 --2- 172.21.15.66:0/880796801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6580cee50 0x7fe6580d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:14.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe656a00640 1 -- 172.21.15.66:0/880796801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6580cee50 msgr2=0x7fe6580d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:14.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe656a00640 1 --2- 172.21.15.66:0/880796801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6580cee50 0x7fe6580d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe656a00640 1 -- 172.21.15.66:0/880796801 >> v1:172.21.15.156:6789/0 conn(0x7fe6580af420 legacy=0x7fe6580ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:14.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe656a00640 1 -- 172.21.15.66:0/880796801 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6580cf430 con 0x7fe65815de30 2024-03-30T00:43:14.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.696+0000 7fe657400640 1 --2- 172.21.15.66:0/880796801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6580cee50 0x7fe6580d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:14.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe656a00640 1 --2- 172.21.15.66:0/880796801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe65815de30 0x7fe6580cc040 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe648009870 tx=0x7fe64801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4151fb0bd553004 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:14.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe655600640 1 -- 172.21.15.66:0/880796801 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6480a7070 con 0x7fe65815de30 2024-03-30T00:43:14.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe655600640 1 -- 172.21.15.66:0/880796801 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6480ab430 con 0x7fe65815de30 2024-03-30T00:43:14.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe655600640 1 -- 172.21.15.66:0/880796801 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6480ab7c0 con 0x7fe65815de30 2024-03-30T00:43:14.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe65d800640 1 -- 172.21.15.66:0/880796801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe65815de30 msgr2=0x7fe6580cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe65d800640 1 --2- 172.21.15.66:0/880796801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe65815de30 0x7fe6580cc040 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe648009870 tx=0x7fe64801c4d0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe65d800640 1 -- 172.21.15.66:0/880796801 shutdown_connections 2024-03-30T00:43:14.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe65d800640 1 --2- 172.21.15.66:0/880796801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6580cee50 0x7fe6580d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe65d800640 1 --2- 172.21.15.66:0/880796801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe65815de30 0x7fe6580cc040 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe65d800640 1 -- 172.21.15.66:0/880796801 >> 172.21.15.66:0/880796801 conn(0x7fe6580d0540 msgr2=0x7fe6580d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:14.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe65d800640 1 -- 172.21.15.66:0/880796801 shutdown_connections 2024-03-30T00:43:14.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.700+0000 7fe65d800640 1 -- 172.21.15.66:0/880796801 wait complete. 2024-03-30T00:43:14.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe65d800640 1 Processor -- start 2024-03-30T00:43:14.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe65d800640 1 -- start start 2024-03-30T00:43:14.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe65d800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6580af420 0x7fe658153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe65d800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6580cee50 0x7fe6581588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe65d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe65815de30 0x7fe658158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe65d800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580c4d90 con 0x7fe65815de30 2024-03-30T00:43:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe65d800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe658163370 con 0x7fe6580af420 2024-03-30T00:43:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe65d800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe658163650 con 0x7fe6580cee50 2024-03-30T00:43:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe657400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe65815de30 0x7fe658158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe657400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe65815de30 0x7fe658158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44220/0 (socket says 172.21.15.66:44220) 2024-03-30T00:43:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe657400640 1 -- 172.21.15.66:0/732600300 learned_addr learned my addr 172.21.15.66:0/732600300 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe656a00640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6580af420 0x7fe658153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe656000640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6580cee50 0x7fe6581588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:14.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe657400640 1 -- 172.21.15.66:0/732600300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6580cee50 msgr2=0x7fe6581588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe657400640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6580cee50 0x7fe6581588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe657400640 1 -- 172.21.15.66:0/732600300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6580af420 msgr2=0x7fe658153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe657400640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6580af420 0x7fe658153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe657400640 1 -- 172.21.15.66:0/732600300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6480094d0 con 0x7fe65815de30 2024-03-30T00:43:14.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe656a00640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6580af420 0x7fe658153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:14.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe657400640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe65815de30 0x7fe658158e20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe650007e70 tx=0x7fe65000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:14.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe64ea00640 1 -- 172.21.15.66:0/732600300 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650015600 con 0x7fe65815de30 2024-03-30T00:43:14.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe64ea00640 1 -- 172.21.15.66:0/732600300 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65000f040 con 0x7fe65815de30 2024-03-30T00:43:14.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe64ea00640 1 -- 172.21.15.66:0/732600300 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6500159a0 con 0x7fe65815de30 2024-03-30T00:43:14.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe65d800640 1 -- 172.21.15.66:0/732600300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe658163930 con 0x7fe65815de30 2024-03-30T00:43:14.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe65d800640 1 -- 172.21.15.66:0/732600300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6580c5050 con 0x7fe65815de30 2024-03-30T00:43:14.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.704+0000 7fe65d800640 1 -- 172.21.15.66:0/732600300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6580cdf60 con 0x7fe65815de30 2024-03-30T00:43:14.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.708+0000 7fe64ea00640 1 -- 172.21.15.66:0/732600300 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe65001e3b0 con 0x7fe65815de30 2024-03-30T00:43:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.708+0000 7fe64ea00640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe638066690 0x7fe638068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.708+0000 7fe64ea00640 1 -- 172.21.15.66:0/732600300 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe650084630 con 0x7fe65815de30 2024-03-30T00:43:14.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.708+0000 7fe656a00640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe638066690 0x7fe638068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:14.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.708+0000 7fe656a00640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe638066690 0x7fe638068b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6480af000 tx=0x7fe648008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:14.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.712+0000 7fe64ea00640 1 -- 172.21.15.66:0/732600300 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe650051850 con 0x7fe65815de30 2024-03-30T00:43:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.872+0000 7fe65d800640 1 -- 172.21.15.66:0/732600300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe658163b20 con 0x7fe65815de30 2024-03-30T00:43:14.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe64ea00640 1 -- 172.21.15.66:0/732600300 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16248 (secure 0 0 0) 0x7fe650054e70 con 0x7fe65815de30 2024-03-30T00:43:14.881 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:14.881 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":68,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:43:12.143735+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:43:09.681968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T00:43:10.133411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 -- 172.21.15.66:0/732600300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe638066690 msgr2=0x7fe638068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe638066690 0x7fe638068b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6480af000 tx=0x7fe648008040 comp rx=0 tx=0).stop 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 -- 172.21.15.66:0/732600300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe65815de30 msgr2=0x7fe658158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe65815de30 0x7fe658158e20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe650007e70 tx=0x7fe65000d4a0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 -- 172.21.15.66:0/732600300 shutdown_connections 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe638066690 0x7fe638068b50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe65815de30 0x7fe658158e20 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6580cee50 0x7fe6581588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 --2- 172.21.15.66:0/732600300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6580af420 0x7fe658153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 -- 172.21.15.66:0/732600300 >> 172.21.15.66:0/732600300 conn(0x7fe6580d0540 msgr2=0x7fe65815d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 -- 172.21.15.66:0/732600300 shutdown_connections 2024-03-30T00:43:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:14.876+0000 7fe65d800640 1 -- 172.21.15.66:0/732600300 wait complete. 2024-03-30T00:43:14.903 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T00:43:14.903 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:43:15.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.036+0000 7f6749800640 1 Processor -- start 2024-03-30T00:43:15.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.036+0000 7f6749800640 1 -- start start 2024-03-30T00:43:15.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6749800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f674415e5d0 0x7f67440cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6749800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674415eab0 con 0x7f674415ae20 2024-03-30T00:43:15.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6749800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674415ed90 con 0x7f674415aa40 2024-03-30T00:43:15.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6749800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67440cec50 con 0x7f674415e5d0 2024-03-30T00:43:15.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6748e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f674415ae20 0x7f67440ccde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60926/0 (socket says 172.21.15.66:60926) 2024-03-30T00:43:15.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6748e00640 1 -- 172.21.15.66:0/1626821981 learned_addr learned my addr 172.21.15.66:0/1626821981 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6743400640 1 --2- 172.21.15.66:0/1626821981 >> v2:172.21.15.156:3300/0 conn(0x7f674415e5d0 0x7f67440cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131988186 0 0) 0x7f674415eab0 con 0x7f674415ae20 2024-03-30T00:43:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6728003530 con 0x7f674415ae20 2024-03-30T00:43:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3252644480 0 0) 0x7f6728003530 con 0x7f674415ae20 2024-03-30T00:43:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 >> v2:172.21.15.156:3300/0 conn(0x7f674415e5d0 msgr2=0x7f67440cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 --2- 172.21.15.66:0/1626821981 >> v2:172.21.15.156:3300/0 conn(0x7f674415e5d0 0x7f67440cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 >> v1:172.21.15.156:6789/0 conn(0x7f674415aa40 legacy=0x7f67440ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674415f070 con 0x7f674415ae20 2024-03-30T00:43:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f673c001a00 con 0x7f674415ae20 2024-03-30T00:43:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 >> v1:172.21.15.66:6789/0 conn(0x7f674415ae20 legacy=0x7f67440ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 --2- 172.21.15.66:0/1626821981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67280039d0 0x7f6728003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 --2- 172.21.15.66:0/1626821981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f67280047d0 0x7f6728004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 --2- 172.21.15.66:0/1626821981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67280051e0 0x7f672800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67440cec50 con 0x7f67280051e0 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672800c130 con 0x7f67280039d0 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672800c2f0 con 0x7f67280047d0 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6748e00640 1 --2- 172.21.15.66:0/1626821981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67280051e0 0x7f672800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6743400640 1 --2- 172.21.15.66:0/1626821981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f67280047d0 0x7f6728004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6743e00640 1 --2- 172.21.15.66:0/1626821981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67280039d0 0x7f6728003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6748e00640 1 -- 172.21.15.66:0/1626821981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f67280047d0 msgr2=0x7f6728004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6748e00640 1 --2- 172.21.15.66:0/1626821981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f67280047d0 0x7f6728004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6748e00640 1 -- 172.21.15.66:0/1626821981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67280039d0 msgr2=0x7f6728003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6748e00640 1 --2- 172.21.15.66:0/1626821981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67280039d0 0x7f6728003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.040+0000 7f6748e00640 1 -- 172.21.15.66:0/1626821981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6728003530 con 0x7f67280051e0 2024-03-30T00:43:15.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6748e00640 1 -- 172.21.15.66:0/1626821981 reap_dead start 2024-03-30T00:43:15.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6748e00640 1 --2- 172.21.15.66:0/1626821981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67280051e0 0x7f672800b990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f673c002240 tx=0x7f673c01f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890cb5ca9eb58428 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:15.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0b0070 con 0x7f67280051e0 2024-03-30T00:43:15.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c0ac410 con 0x7f67280051e0 2024-03-30T00:43:15.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6742a00640 1 -- 172.21.15.66:0/1626821981 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0ac7c0 con 0x7f67280051e0 2024-03-30T00:43:15.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6749800640 1 -- 172.21.15.66:0/1626821981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67280051e0 msgr2=0x7f672800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6749800640 1 --2- 172.21.15.66:0/1626821981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67280051e0 0x7f672800b990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f673c002240 tx=0x7f673c01f960 comp rx=0 tx=0).stop 2024-03-30T00:43:15.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6749800640 1 -- 172.21.15.66:0/1626821981 shutdown_connections 2024-03-30T00:43:15.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6749800640 1 --2- 172.21.15.66:0/1626821981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67280051e0 0x7f672800b990 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6749800640 1 -- 172.21.15.66:0/1626821981 >> 172.21.15.66:0/1626821981 conn(0x7f674406e0d0 msgr2=0x7f674406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:15.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6749800640 1 -- 172.21.15.66:0/1626821981 shutdown_connections 2024-03-30T00:43:15.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6749800640 1 -- 172.21.15.66:0/1626821981 wait complete. 2024-03-30T00:43:15.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6749800640 1 Processor -- start 2024-03-30T00:43:15.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.044+0000 7f6749800640 1 -- start start 2024-03-30T00:43:15.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6749800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67280051e0 0x7f67440b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6749800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f67441f17f0 0x7f67441f1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6749800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67441f25f0 0x7f67441f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6749800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67441f3360 con 0x7f67441f25f0 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6749800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67440b1d70 con 0x7f67280051e0 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6749800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67441f21b0 con 0x7f67441f17f0 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6748e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67441f25f0 0x7f67441f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6748e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67441f25f0 0x7f67441f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44242/0 (socket says 172.21.15.66:44242) 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6748e00640 1 -- 172.21.15.66:0/3379785558 learned_addr learned my addr 172.21.15.66:0/3379785558 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6743e00640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67280051e0 0x7f67440b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6743400640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f67441f17f0 0x7f67441f1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6748e00640 1 -- 172.21.15.66:0/3379785558 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f67441f17f0 msgr2=0x7f67441f1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6748e00640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f67441f17f0 0x7f67441f1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6748e00640 1 -- 172.21.15.66:0/3379785558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67280051e0 msgr2=0x7f67440b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6748e00640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67280051e0 0x7f67440b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6748e00640 1 -- 172.21.15.66:0/3379785558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440b2010 con 0x7f67441f25f0 2024-03-30T00:43:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6743400640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f67441f17f0 0x7f67441f1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6743e00640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67280051e0 0x7f67440b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6748e00640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67441f25f0 0x7f67441f2a70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f673c01f790 tx=0x7f6728005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f673be00640 1 -- 172.21.15.66:0/3379785558 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0b0040 con 0x7f67441f25f0 2024-03-30T00:43:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f673be00640 1 -- 172.21.15.66:0/3379785558 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c001260 con 0x7f67441f25f0 2024-03-30T00:43:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f673be00640 1 -- 172.21.15.66:0/3379785558 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c001a00 con 0x7f67441f25f0 2024-03-30T00:43:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6749800640 1 -- 172.21.15.66:0/3379785558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67440b22f0 con 0x7f67441f25f0 2024-03-30T00:43:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6749800640 1 -- 172.21.15.66:0/3379785558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67440b28b0 con 0x7f67441f25f0 2024-03-30T00:43:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f6749800640 1 -- 172.21.15.66:0/3379785558 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67440b2aa0 con 0x7f67441f25f0 2024-03-30T00:43:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.048+0000 7f673be00640 1 -- 172.21.15.66:0/3379785558 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f673c0c1460 con 0x7f67441f25f0 2024-03-30T00:43:15.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.052+0000 7f673be00640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6724066710 0x7f6724068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.052+0000 7f673be00640 1 -- 172.21.15.66:0/3379785558 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f673c122250 con 0x7f67441f25f0 2024-03-30T00:43:15.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.052+0000 7f6743e00640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6724066710 0x7f6724068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.052+0000 7f6743e00640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6724066710 0x7f6724068bd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6734007fd0 tx=0x7f6734009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:15.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.056+0000 7f673be00640 1 -- 172.21.15.66:0/3379785558 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f673c0ef460 con 0x7f67441f25f0 2024-03-30T00:43:15.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.212+0000 7f6749800640 1 -- 172.21.15.66:0/3379785558 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67440cb500 con 0x7f6724066710 2024-03-30T00:43:15.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.220+0000 7f673be00640 1 -- 172.21.15.66:0/3379785558 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289301 (secure 0 0 0) 0x7f67340014a0 con 0x7f6724066710 2024-03-30T00:43:15.229 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:15.231 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:43:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 -- 172.21.15.66:0/3379785558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6724066710 msgr2=0x7f6724068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6724066710 0x7f6724068bd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6734007fd0 tx=0x7f6734009040 comp rx=0 tx=0).stop 2024-03-30T00:43:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 -- 172.21.15.66:0/3379785558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67441f25f0 msgr2=0x7f67441f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67441f25f0 0x7f67441f2a70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f673c01f790 tx=0x7f6728005800 comp rx=0 tx=0).stop 2024-03-30T00:43:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 -- 172.21.15.66:0/3379785558 shutdown_connections 2024-03-30T00:43:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6724066710 0x7f6724068bd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67441f25f0 0x7f67441f2a70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f67441f17f0 0x7f67441f1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 --2- 172.21.15.66:0/3379785558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67280051e0 0x7f67440b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 -- 172.21.15.66:0/3379785558 >> 172.21.15.66:0/3379785558 conn(0x7f674406e0d0 msgr2=0x7f67440cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:15.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 -- 172.21.15.66:0/3379785558 shutdown_connections 2024-03-30T00:43:15.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.224+0000 7f6749800640 1 -- 172.21.15.66:0/3379785558 wait complete. 2024-03-30T00:43:15.250 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-03-30T00:43:14.893717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240200,"kb_used_data":7596,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754734520,"statfs":{"total":773094113280,"available":772848148480,"internally_reserved":0,"allocated":7778304,"data_stored":4106264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.158722"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.177106+0000","last_change":"2024-03-30T00:43:11.174174+0000","last_active":"2024-03-30T00:43:11.177106+0000","last_peered":"2024-03-30T00:43:11.177106+0000","last_clean":"2024-03-30T00:43:11.177106+0000","last_became_active":"2024-03-30T00:43:11.173862+0000","last_became_peered":"2024-03-30T00:43:11.173862+0000","last_unstale":"2024-03-30T00:43:11.177106+0000","last_undegraded":"2024-03-30T00:43:11.177106+0000","last_fullsized":"2024-03-30T00:43:11.177106+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:31.136461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.172319+0000","last_change":"2024-03-30T00:43:11.163085+0000","last_active":"2024-03-30T00:43:11.172319+0000","last_peered":"2024-03-30T00:43:11.172319+0000","last_clean":"2024-03-30T00:43:11.172319+0000","last_became_active":"2024-03-30T00:43:11.162810+0000","last_became_peered":"2024-03-30T00:43:11.162810+0000","last_unstale":"2024-03-30T00:43:11.172319+0000","last_undegraded":"2024-03-30T00:43:11.172319+0000","last_fullsized":"2024-03-30T00:43:11.172319+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:54.701307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155746+0000","last_change":"2024-03-30T00:43:11.174928+0000","last_active":"2024-03-30T00:43:12.155746+0000","last_peered":"2024-03-30T00:43:12.155746+0000","last_clean":"2024-03-30T00:43:12.155746+0000","last_became_active":"2024-03-30T00:43:11.174463+0000","last_became_peered":"2024-03-30T00:43:11.174463+0000","last_unstale":"2024-03-30T00:43:12.155746+0000","last_undegraded":"2024-03-30T00:43:12.155746+0000","last_fullsized":"2024-03-30T00:43:12.155746+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:49:10.790473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156558+0000","last_change":"2024-03-30T00:43:11.164044+0000","last_active":"2024-03-30T00:43:12.156558+0000","last_peered":"2024-03-30T00:43:12.156558+0000","last_clean":"2024-03-30T00:43:12.156558+0000","last_became_active":"2024-03-30T00:43:11.163769+0000","last_became_peered":"2024-03-30T00:43:11.163769+0000","last_unstale":"2024-03-30T00:43:12.156558+0000","last_undegraded":"2024-03-30T00:43:12.156558+0000","last_fullsized":"2024-03-30T00:43:12.156558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:12.337025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902067+0000","last_change":"2024-03-30T00:43:11.171880+0000","last_active":"2024-03-30T00:43:12.902067+0000","last_peered":"2024-03-30T00:43:12.902067+0000","last_clean":"2024-03-30T00:43:12.902067+0000","last_became_active":"2024-03-30T00:43:11.168977+0000","last_became_peered":"2024-03-30T00:43:11.168977+0000","last_unstale":"2024-03-30T00:43:12.902067+0000","last_undegraded":"2024-03-30T00:43:12.902067+0000","last_fullsized":"2024-03-30T00:43:12.902067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:19:28.439620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783770+0000","last_change":"2024-03-30T00:43:11.163581+0000","last_active":"2024-03-30T00:43:12.783770+0000","last_peered":"2024-03-30T00:43:12.783770+0000","last_clean":"2024-03-30T00:43:12.783770+0000","last_became_active":"2024-03-30T00:43:11.163332+0000","last_became_peered":"2024-03-30T00:43:11.163332+0000","last_unstale":"2024-03-30T00:43:12.783770+0000","last_undegraded":"2024-03-30T00:43:12.783770+0000","last_fullsized":"2024-03-30T00:43:12.783770+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:08.769384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155837+0000","last_change":"2024-03-30T00:43:11.162297+0000","last_active":"2024-03-30T00:43:12.155837+0000","last_peered":"2024-03-30T00:43:12.155837+0000","last_clean":"2024-03-30T00:43:12.155837+0000","last_became_active":"2024-03-30T00:43:11.161755+0000","last_became_peered":"2024-03-30T00:43:11.161755+0000","last_unstale":"2024-03-30T00:43:12.155837+0000","last_undegraded":"2024-03-30T00:43:12.155837+0000","last_fullsized":"2024-03-30T00:43:12.155837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:12:55.124790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.181281+0000","last_change":"2024-03-30T00:43:11.176478+0000","last_active":"2024-03-30T00:43:11.181281+0000","last_peered":"2024-03-30T00:43:11.181281+0000","last_clean":"2024-03-30T00:43:11.181281+0000","last_became_active":"2024-03-30T00:43:11.176039+0000","last_became_peered":"2024-03-30T00:43:11.176039+0000","last_unstale":"2024-03-30T00:43:11.181281+0000","last_undegraded":"2024-03-30T00:43:11.181281+0000","last_fullsized":"2024-03-30T00:43:11.181281+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:54.701307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155755+0000","last_change":"2024-03-30T00:43:11.185700+0000","last_active":"2024-03-30T00:43:12.155755+0000","last_peered":"2024-03-30T00:43:12.155755+0000","last_clean":"2024-03-30T00:43:12.155755+0000","last_became_active":"2024-03-30T00:43:11.185441+0000","last_became_peered":"2024-03-30T00:43:11.185441+0000","last_unstale":"2024-03-30T00:43:12.155755+0000","last_undegraded":"2024-03-30T00:43:12.155755+0000","last_fullsized":"2024-03-30T00:43:12.155755+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:12.015584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155873+0000","last_change":"2024-03-30T00:43:11.169320+0000","last_active":"2024-03-30T00:43:12.155873+0000","last_peered":"2024-03-30T00:43:12.155873+0000","last_clean":"2024-03-30T00:43:12.155873+0000","last_became_active":"2024-03-30T00:43:11.169029+0000","last_became_peered":"2024-03-30T00:43:11.169029+0000","last_unstale":"2024-03-30T00:43:12.155873+0000","last_undegraded":"2024-03-30T00:43:12.155873+0000","last_fullsized":"2024-03-30T00:43:12.155873+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:07:44.367842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156118+0000","last_change":"2024-03-30T00:43:11.165561+0000","last_active":"2024-03-30T00:43:12.156118+0000","last_peered":"2024-03-30T00:43:12.156118+0000","last_clean":"2024-03-30T00:43:12.156118+0000","last_became_active":"2024-03-30T00:43:11.165233+0000","last_became_peered":"2024-03-30T00:43:11.165233+0000","last_unstale":"2024-03-30T00:43:12.156118+0000","last_undegraded":"2024-03-30T00:43:12.156118+0000","last_fullsized":"2024-03-30T00:43:12.156118+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:49:10.790473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.181283+0000","last_change":"2024-03-30T00:43:11.176652+0000","last_active":"2024-03-30T00:43:11.181283+0000","last_peered":"2024-03-30T00:43:11.181283+0000","last_clean":"2024-03-30T00:43:11.181283+0000","last_became_active":"2024-03-30T00:43:11.175998+0000","last_became_peered":"2024-03-30T00:43:11.175998+0000","last_unstale":"2024-03-30T00:43:11.181283+0000","last_undegraded":"2024-03-30T00:43:11.181283+0000","last_fullsized":"2024-03-30T00:43:11.181283+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:10:52.232984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.186391+0000","last_change":"2024-03-30T00:43:11.185626+0000","last_active":"2024-03-30T00:43:11.186391+0000","last_peered":"2024-03-30T00:43:11.186391+0000","last_clean":"2024-03-30T00:43:11.186391+0000","last_became_active":"2024-03-30T00:43:11.185344+0000","last_became_peered":"2024-03-30T00:43:11.185344+0000","last_unstale":"2024-03-30T00:43:11.186391+0000","last_undegraded":"2024-03-30T00:43:11.186391+0000","last_fullsized":"2024-03-30T00:43:11.186391+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:46:13.402751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155683+0000","last_change":"2024-03-30T00:43:11.174882+0000","last_active":"2024-03-30T00:43:12.155683+0000","last_peered":"2024-03-30T00:43:12.155683+0000","last_clean":"2024-03-30T00:43:12.155683+0000","last_became_active":"2024-03-30T00:43:11.174572+0000","last_became_peered":"2024-03-30T00:43:11.174572+0000","last_unstale":"2024-03-30T00:43:12.155683+0000","last_undegraded":"2024-03-30T00:43:12.155683+0000","last_fullsized":"2024-03-30T00:43:12.155683+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:56:57.944571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155590+0000","last_change":"2024-03-30T00:43:11.160639+0000","last_active":"2024-03-30T00:43:12.155590+0000","last_peered":"2024-03-30T00:43:12.155590+0000","last_clean":"2024-03-30T00:43:12.155590+0000","last_became_active":"2024-03-30T00:43:11.159795+0000","last_became_peered":"2024-03-30T00:43:11.159795+0000","last_unstale":"2024-03-30T00:43:12.155590+0000","last_undegraded":"2024-03-30T00:43:12.155590+0000","last_fullsized":"2024-03-30T00:43:12.155590+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:31.136461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155692+0000","last_change":"2024-03-30T00:43:11.169526+0000","last_active":"2024-03-30T00:43:12.155692+0000","last_peered":"2024-03-30T00:43:12.155692+0000","last_clean":"2024-03-30T00:43:12.155692+0000","last_became_active":"2024-03-30T00:43:11.169278+0000","last_became_peered":"2024-03-30T00:43:11.169278+0000","last_unstale":"2024-03-30T00:43:12.155692+0000","last_undegraded":"2024-03-30T00:43:12.155692+0000","last_fullsized":"2024-03-30T00:43:12.155692+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:02:45.675408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.173315+0000","last_change":"2024-03-30T00:43:11.166428+0000","last_active":"2024-03-30T00:43:11.173315+0000","last_peered":"2024-03-30T00:43:11.173315+0000","last_clean":"2024-03-30T00:43:11.173315+0000","last_became_active":"2024-03-30T00:43:11.166191+0000","last_became_peered":"2024-03-30T00:43:11.166191+0000","last_unstale":"2024-03-30T00:43:11.173315+0000","last_undegraded":"2024-03-30T00:43:11.173315+0000","last_fullsized":"2024-03-30T00:43:11.173315+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:56:57.944571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156437+0000","last_change":"2024-03-30T00:43:11.165896+0000","last_active":"2024-03-30T00:43:12.156437+0000","last_peered":"2024-03-30T00:43:12.156437+0000","last_clean":"2024-03-30T00:43:12.156437+0000","last_became_active":"2024-03-30T00:43:11.165593+0000","last_became_peered":"2024-03-30T00:43:11.165593+0000","last_unstale":"2024-03-30T00:43:12.156437+0000","last_undegraded":"2024-03-30T00:43:12.156437+0000","last_fullsized":"2024-03-30T00:43:12.156437+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:07:44.367842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902411+0000","last_change":"2024-03-30T00:43:11.173869+0000","last_active":"2024-03-30T00:43:12.902411+0000","last_peered":"2024-03-30T00:43:12.902411+0000","last_clean":"2024-03-30T00:43:12.902411+0000","last_became_active":"2024-03-30T00:43:11.173545+0000","last_became_peered":"2024-03-30T00:43:11.173545+0000","last_unstale":"2024-03-30T00:43:12.902411+0000","last_undegraded":"2024-03-30T00:43:12.902411+0000","last_fullsized":"2024-03-30T00:43:12.902411+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:25.419335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.176946+0000","last_change":"2024-03-30T00:43:11.172346+0000","last_active":"2024-03-30T00:43:11.176946+0000","last_peered":"2024-03-30T00:43:11.176946+0000","last_clean":"2024-03-30T00:43:11.176946+0000","last_became_active":"2024-03-30T00:43:11.170651+0000","last_became_peered":"2024-03-30T00:43:11.170651+0000","last_unstale":"2024-03-30T00:43:11.176946+0000","last_undegraded":"2024-03-30T00:43:11.176946+0000","last_fullsized":"2024-03-30T00:43:11.176946+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:25:29.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.179067+0000","last_change":"2024-03-30T00:43:11.174359+0000","last_active":"2024-03-30T00:43:11.179067+0000","last_peered":"2024-03-30T00:43:11.179067+0000","last_clean":"2024-03-30T00:43:11.179067+0000","last_became_active":"2024-03-30T00:43:11.173824+0000","last_became_peered":"2024-03-30T00:43:11.173824+0000","last_unstale":"2024-03-30T00:43:11.179067+0000","last_undegraded":"2024-03-30T00:43:11.179067+0000","last_fullsized":"2024-03-30T00:43:11.179067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:49:10.790473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.172459+0000","last_change":"2024-03-30T00:43:11.163873+0000","last_active":"2024-03-30T00:43:11.172459+0000","last_peered":"2024-03-30T00:43:11.172459+0000","last_clean":"2024-03-30T00:43:11.172459+0000","last_became_active":"2024-03-30T00:43:11.163609+0000","last_became_peered":"2024-03-30T00:43:11.163609+0000","last_unstale":"2024-03-30T00:43:11.172459+0000","last_undegraded":"2024-03-30T00:43:11.172459+0000","last_fullsized":"2024-03-30T00:43:11.172459+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:46.420657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.171129+0000","last_change":"2024-03-30T00:43:11.164162+0000","last_active":"2024-03-30T00:43:11.171129+0000","last_peered":"2024-03-30T00:43:11.171129+0000","last_clean":"2024-03-30T00:43:11.171129+0000","last_became_active":"2024-03-30T00:43:11.163911+0000","last_became_peered":"2024-03-30T00:43:11.163911+0000","last_unstale":"2024-03-30T00:43:11.171129+0000","last_undegraded":"2024-03-30T00:43:11.171129+0000","last_fullsized":"2024-03-30T00:43:11.171129+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:08.769384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156069+0000","last_change":"2024-03-30T00:43:11.173235+0000","last_active":"2024-03-30T00:43:12.156069+0000","last_peered":"2024-03-30T00:43:12.156069+0000","last_clean":"2024-03-30T00:43:12.156069+0000","last_became_active":"2024-03-30T00:43:11.172991+0000","last_became_peered":"2024-03-30T00:43:11.172991+0000","last_unstale":"2024-03-30T00:43:12.156069+0000","last_undegraded":"2024-03-30T00:43:12.156069+0000","last_fullsized":"2024-03-30T00:43:12.156069+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:12.337025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.181303+0000","last_change":"2024-03-30T00:43:11.178069+0000","last_active":"2024-03-30T00:43:11.181303+0000","last_peered":"2024-03-30T00:43:11.181303+0000","last_clean":"2024-03-30T00:43:11.181303+0000","last_became_active":"2024-03-30T00:43:11.177741+0000","last_became_peered":"2024-03-30T00:43:11.177741+0000","last_unstale":"2024-03-30T00:43:11.181303+0000","last_undegraded":"2024-03-30T00:43:11.181303+0000","last_fullsized":"2024-03-30T00:43:11.181303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:12.015584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.181080+0000","last_change":"2024-03-30T00:43:11.176359+0000","last_active":"2024-03-30T00:43:11.181080+0000","last_peered":"2024-03-30T00:43:11.181080+0000","last_clean":"2024-03-30T00:43:11.181080+0000","last_became_active":"2024-03-30T00:43:11.175167+0000","last_became_peered":"2024-03-30T00:43:11.175167+0000","last_unstale":"2024-03-30T00:43:11.181080+0000","last_undegraded":"2024-03-30T00:43:11.181080+0000","last_fullsized":"2024-03-30T00:43:11.181080+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:34.053522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156242+0000","last_change":"2024-03-30T00:43:11.164962+0000","last_active":"2024-03-30T00:43:12.156242+0000","last_peered":"2024-03-30T00:43:12.156242+0000","last_clean":"2024-03-30T00:43:12.156242+0000","last_became_active":"2024-03-30T00:43:11.164132+0000","last_became_peered":"2024-03-30T00:43:11.164132+0000","last_unstale":"2024-03-30T00:43:12.156242+0000","last_undegraded":"2024-03-30T00:43:12.156242+0000","last_fullsized":"2024-03-30T00:43:12.156242+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:12.015584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.174996+0000","last_change":"2024-03-30T00:43:11.171783+0000","last_active":"2024-03-30T00:43:11.174996+0000","last_peered":"2024-03-30T00:43:11.174996+0000","last_clean":"2024-03-30T00:43:11.174996+0000","last_became_active":"2024-03-30T00:43:11.171553+0000","last_became_peered":"2024-03-30T00:43:11.171553+0000","last_unstale":"2024-03-30T00:43:11.174996+0000","last_undegraded":"2024-03-30T00:43:11.174996+0000","last_fullsized":"2024-03-30T00:43:11.174996+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:46.189391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155354+0000","last_change":"2024-03-30T00:43:11.183818+0000","last_active":"2024-03-30T00:43:12.155354+0000","last_peered":"2024-03-30T00:43:12.155354+0000","last_clean":"2024-03-30T00:43:12.155354+0000","last_became_active":"2024-03-30T00:43:11.183410+0000","last_became_peered":"2024-03-30T00:43:11.183410+0000","last_unstale":"2024-03-30T00:43:12.155354+0000","last_undegraded":"2024-03-30T00:43:12.155354+0000","last_fullsized":"2024-03-30T00:43:12.155354+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:46.189391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.180338+0000","last_change":"2024-03-30T00:43:11.176234+0000","last_active":"2024-03-30T00:43:11.180338+0000","last_peered":"2024-03-30T00:43:11.180338+0000","last_clean":"2024-03-30T00:43:11.180338+0000","last_became_active":"2024-03-30T00:43:11.175884+0000","last_became_peered":"2024-03-30T00:43:11.175884+0000","last_unstale":"2024-03-30T00:43:11.180338+0000","last_undegraded":"2024-03-30T00:43:11.180338+0000","last_fullsized":"2024-03-30T00:43:11.180338+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:07:44.367842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156199+0000","last_change":"2024-03-30T00:43:11.158808+0000","last_active":"2024-03-30T00:43:12.156199+0000","last_peered":"2024-03-30T00:43:12.156199+0000","last_clean":"2024-03-30T00:43:12.156199+0000","last_became_active":"2024-03-30T00:43:11.158339+0000","last_became_peered":"2024-03-30T00:43:11.158339+0000","last_unstale":"2024-03-30T00:43:12.156199+0000","last_undegraded":"2024-03-30T00:43:12.156199+0000","last_fullsized":"2024-03-30T00:43:12.156199+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:46.189391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783546+0000","last_change":"2024-03-30T00:43:11.169952+0000","last_active":"2024-03-30T00:43:12.783546+0000","last_peered":"2024-03-30T00:43:12.783546+0000","last_clean":"2024-03-30T00:43:12.783546+0000","last_became_active":"2024-03-30T00:43:11.169511+0000","last_became_peered":"2024-03-30T00:43:11.169511+0000","last_unstale":"2024-03-30T00:43:12.783546+0000","last_undegraded":"2024-03-30T00:43:12.783546+0000","last_fullsized":"2024-03-30T00:43:12.783546+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:02:50.870716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902283+0000","last_change":"2024-03-30T00:43:11.166814+0000","last_active":"2024-03-30T00:43:12.902283+0000","last_peered":"2024-03-30T00:43:12.902283+0000","last_clean":"2024-03-30T00:43:12.902283+0000","last_became_active":"2024-03-30T00:43:11.166408+0000","last_became_peered":"2024-03-30T00:43:11.166408+0000","last_unstale":"2024-03-30T00:43:12.902283+0000","last_undegraded":"2024-03-30T00:43:12.902283+0000","last_fullsized":"2024-03-30T00:43:12.902283+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:53.098797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"67'4","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.175146+0000","last_change":"2024-03-30T00:43:10.752801+0000","last_active":"2024-03-30T00:43:11.175146+0000","last_peered":"2024-03-30T00:43:11.175146+0000","last_clean":"2024-03-30T00:43:11.175146+0000","last_became_active":"2024-03-30T00:43:10.751722+0000","last_became_peered":"2024-03-30T00:43:10.751722+0000","last_unstale":"2024-03-30T00:43:11.175146+0000","last_undegraded":"2024-03-30T00:43:11.175146+0000","last_fullsized":"2024-03-30T00:43:11.175146+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:28.430262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156775+0000","last_change":"2024-03-30T00:43:11.182061+0000","last_active":"2024-03-30T00:43:12.156775+0000","last_peered":"2024-03-30T00:43:12.156775+0000","last_clean":"2024-03-30T00:43:12.156775+0000","last_became_active":"2024-03-30T00:43:11.181781+0000","last_became_peered":"2024-03-30T00:43:11.181781+0000","last_unstale":"2024-03-30T00:43:12.156775+0000","last_undegraded":"2024-03-30T00:43:12.156775+0000","last_fullsized":"2024-03-30T00:43:12.156775+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:41:03.894477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"67'1","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783938+0000","last_change":"2024-03-30T00:43:10.762465+0000","last_active":"2024-03-30T00:43:12.783938+0000","last_peered":"2024-03-30T00:43:12.783938+0000","last_clean":"2024-03-30T00:43:12.783938+0000","last_became_active":"2024-03-30T00:43:10.762238+0000","last_became_peered":"2024-03-30T00:43:10.762238+0000","last_unstale":"2024-03-30T00:43:12.783938+0000","last_undegraded":"2024-03-30T00:43:12.783938+0000","last_fullsized":"2024-03-30T00:43:12.783938+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:33.051174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783884+0000","last_change":"2024-03-30T00:43:11.169336+0000","last_active":"2024-03-30T00:43:12.783884+0000","last_peered":"2024-03-30T00:43:12.783884+0000","last_clean":"2024-03-30T00:43:12.783884+0000","last_became_active":"2024-03-30T00:43:11.169091+0000","last_became_peered":"2024-03-30T00:43:11.169091+0000","last_unstale":"2024-03-30T00:43:12.783884+0000","last_undegraded":"2024-03-30T00:43:12.783884+0000","last_fullsized":"2024-03-30T00:43:12.783884+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:46:41.582112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"67'2","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783846+0000","last_change":"2024-03-30T00:43:10.762845+0000","last_active":"2024-03-30T00:43:12.783846+0000","last_peered":"2024-03-30T00:43:12.783846+0000","last_clean":"2024-03-30T00:43:12.783846+0000","last_became_active":"2024-03-30T00:43:10.762548+0000","last_became_peered":"2024-03-30T00:43:10.762548+0000","last_unstale":"2024-03-30T00:43:12.783846+0000","last_undegraded":"2024-03-30T00:43:12.783846+0000","last_fullsized":"2024-03-30T00:43:12.783846+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:38:16.808375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155710+0000","last_change":"2024-03-30T00:43:11.171048+0000","last_active":"2024-03-30T00:43:12.155710+0000","last_peered":"2024-03-30T00:43:12.155710+0000","last_clean":"2024-03-30T00:43:12.155710+0000","last_became_active":"2024-03-30T00:43:11.169752+0000","last_became_peered":"2024-03-30T00:43:11.169752+0000","last_unstale":"2024-03-30T00:43:12.155710+0000","last_undegraded":"2024-03-30T00:43:12.155710+0000","last_fullsized":"2024-03-30T00:43:12.155710+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:54.701307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"67'2","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783731+0000","last_change":"2024-03-30T00:43:10.756202+0000","last_active":"2024-03-30T00:43:12.783731+0000","last_peered":"2024-03-30T00:43:12.783731+0000","last_clean":"2024-03-30T00:43:12.783731+0000","last_became_active":"2024-03-30T00:43:10.755940+0000","last_became_peered":"2024-03-30T00:43:10.755940+0000","last_unstale":"2024-03-30T00:43:12.783731+0000","last_undegraded":"2024-03-30T00:43:12.783731+0000","last_fullsized":"2024-03-30T00:43:12.783731+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:02.108234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155688+0000","last_change":"2024-03-30T00:43:11.156922+0000","last_active":"2024-03-30T00:43:12.155688+0000","last_peered":"2024-03-30T00:43:12.155688+0000","last_clean":"2024-03-30T00:43:12.155688+0000","last_became_active":"2024-03-30T00:43:11.156635+0000","last_became_peered":"2024-03-30T00:43:11.156635+0000","last_unstale":"2024-03-30T00:43:12.155688+0000","last_undegraded":"2024-03-30T00:43:12.155688+0000","last_fullsized":"2024-03-30T00:43:12.155688+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:50:53.954307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.165079+0000","last_change":"2024-03-30T00:43:10.752090+0000","last_active":"2024-03-30T00:43:11.165079+0000","last_peered":"2024-03-30T00:43:11.165079+0000","last_clean":"2024-03-30T00:43:11.165079+0000","last_became_active":"2024-03-30T00:43:10.751830+0000","last_became_peered":"2024-03-30T00:43:10.751830+0000","last_unstale":"2024-03-30T00:43:11.165079+0000","last_undegraded":"2024-03-30T00:43:11.165079+0000","last_fullsized":"2024-03-30T00:43:11.165079+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:26:27.085243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.172920+0000","last_change":"2024-03-30T00:43:11.165524+0000","last_active":"2024-03-30T00:43:11.172920+0000","last_peered":"2024-03-30T00:43:11.172920+0000","last_clean":"2024-03-30T00:43:11.172920+0000","last_became_active":"2024-03-30T00:43:11.165047+0000","last_became_peered":"2024-03-30T00:43:11.165047+0000","last_unstale":"2024-03-30T00:43:11.172920+0000","last_undegraded":"2024-03-30T00:43:11.172920+0000","last_fullsized":"2024-03-30T00:43:11.172920+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:49:10.790473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.168794+0000","last_change":"2024-03-30T00:43:10.758940+0000","last_active":"2024-03-30T00:43:11.168794+0000","last_peered":"2024-03-30T00:43:11.168794+0000","last_clean":"2024-03-30T00:43:11.168794+0000","last_became_active":"2024-03-30T00:43:10.758632+0000","last_became_peered":"2024-03-30T00:43:10.758632+0000","last_unstale":"2024-03-30T00:43:11.168794+0000","last_undegraded":"2024-03-30T00:43:11.168794+0000","last_fullsized":"2024-03-30T00:43:11.168794+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:38:16.808375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902015+0000","last_change":"2024-03-30T00:43:11.170525+0000","last_active":"2024-03-30T00:43:12.902015+0000","last_peered":"2024-03-30T00:43:12.902015+0000","last_clean":"2024-03-30T00:43:12.902015+0000","last_became_active":"2024-03-30T00:43:11.170246+0000","last_became_peered":"2024-03-30T00:43:11.170246+0000","last_unstale":"2024-03-30T00:43:12.902015+0000","last_undegraded":"2024-03-30T00:43:12.902015+0000","last_fullsized":"2024-03-30T00:43:12.902015+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:57:49.883141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.150004+0000","last_change":"2024-03-30T00:43:10.765925+0000","last_active":"2024-03-30T00:43:11.150004+0000","last_peered":"2024-03-30T00:43:11.150004+0000","last_clean":"2024-03-30T00:43:11.150004+0000","last_became_active":"2024-03-30T00:43:10.765642+0000","last_became_peered":"2024-03-30T00:43:10.765642+0000","last_unstale":"2024-03-30T00:43:11.150004+0000","last_undegraded":"2024-03-30T00:43:11.150004+0000","last_fullsized":"2024-03-30T00:43:11.150004+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:33.051174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.161966+0000","last_change":"2024-03-30T00:43:11.158686+0000","last_active":"2024-03-30T00:43:11.161966+0000","last_peered":"2024-03-30T00:43:11.161966+0000","last_clean":"2024-03-30T00:43:11.161966+0000","last_became_active":"2024-03-30T00:43:11.158427+0000","last_became_peered":"2024-03-30T00:43:11.158427+0000","last_unstale":"2024-03-30T00:43:11.161966+0000","last_undegraded":"2024-03-30T00:43:11.161966+0000","last_fullsized":"2024-03-30T00:43:11.161966+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:26:05.722702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902223+0000","last_change":"2024-03-30T00:43:10.755378+0000","last_active":"2024-03-30T00:43:12.902223+0000","last_peered":"2024-03-30T00:43:12.902223+0000","last_clean":"2024-03-30T00:43:12.902223+0000","last_became_active":"2024-03-30T00:43:10.754308+0000","last_became_peered":"2024-03-30T00:43:10.754308+0000","last_unstale":"2024-03-30T00:43:12.902223+0000","last_undegraded":"2024-03-30T00:43:12.902223+0000","last_fullsized":"2024-03-30T00:43:12.902223+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:17:07.832744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.159612+0000","last_change":"2024-03-30T00:43:11.157364+0000","last_active":"2024-03-30T00:43:11.159612+0000","last_peered":"2024-03-30T00:43:11.159612+0000","last_clean":"2024-03-30T00:43:11.159612+0000","last_became_active":"2024-03-30T00:43:11.156978+0000","last_became_peered":"2024-03-30T00:43:11.156978+0000","last_unstale":"2024-03-30T00:43:11.159612+0000","last_undegraded":"2024-03-30T00:43:11.159612+0000","last_fullsized":"2024-03-30T00:43:11.159612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:07:44.367842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155861+0000","last_change":"2024-03-30T00:43:10.757052+0000","last_active":"2024-03-30T00:43:12.155861+0000","last_peered":"2024-03-30T00:43:12.155861+0000","last_clean":"2024-03-30T00:43:12.155861+0000","last_became_active":"2024-03-30T00:43:10.756696+0000","last_became_peered":"2024-03-30T00:43:10.756696+0000","last_unstale":"2024-03-30T00:43:12.155861+0000","last_undegraded":"2024-03-30T00:43:12.155861+0000","last_fullsized":"2024-03-30T00:43:12.155861+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:57:53.489388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.172343+0000","last_change":"2024-03-30T00:43:11.168077+0000","last_active":"2024-03-30T00:43:11.172343+0000","last_peered":"2024-03-30T00:43:11.172343+0000","last_clean":"2024-03-30T00:43:11.172343+0000","last_became_active":"2024-03-30T00:43:11.167797+0000","last_became_peered":"2024-03-30T00:43:11.167797+0000","last_unstale":"2024-03-30T00:43:11.172343+0000","last_undegraded":"2024-03-30T00:43:11.172343+0000","last_fullsized":"2024-03-30T00:43:11.172343+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:46:41.582112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156549+0000","last_change":"2024-03-30T00:43:10.750107+0000","last_active":"2024-03-30T00:43:12.156549+0000","last_peered":"2024-03-30T00:43:12.156549+0000","last_clean":"2024-03-30T00:43:12.156549+0000","last_became_active":"2024-03-30T00:43:10.749738+0000","last_became_peered":"2024-03-30T00:43:10.749738+0000","last_unstale":"2024-03-30T00:43:12.156549+0000","last_undegraded":"2024-03-30T00:43:12.156549+0000","last_fullsized":"2024-03-30T00:43:12.156549+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:23.758029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902443+0000","last_change":"2024-03-30T00:43:11.178139+0000","last_active":"2024-03-30T00:43:12.902443+0000","last_peered":"2024-03-30T00:43:12.902443+0000","last_clean":"2024-03-30T00:43:12.902443+0000","last_became_active":"2024-03-30T00:43:11.177884+0000","last_became_peered":"2024-03-30T00:43:11.177884+0000","last_unstale":"2024-03-30T00:43:12.902443+0000","last_undegraded":"2024-03-30T00:43:12.902443+0000","last_fullsized":"2024-03-30T00:43:12.902443+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:22.378638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"67'3","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155824+0000","last_change":"2024-03-30T00:43:10.754587+0000","last_active":"2024-03-30T00:43:12.155824+0000","last_peered":"2024-03-30T00:43:12.155824+0000","last_clean":"2024-03-30T00:43:12.155824+0000","last_became_active":"2024-03-30T00:43:10.754294+0000","last_became_peered":"2024-03-30T00:43:10.754294+0000","last_unstale":"2024-03-30T00:43:12.155824+0000","last_undegraded":"2024-03-30T00:43:12.155824+0000","last_fullsized":"2024-03-30T00:43:12.155824+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:23.758029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156273+0000","last_change":"2024-03-30T00:43:11.162508+0000","last_active":"2024-03-30T00:43:12.156273+0000","last_peered":"2024-03-30T00:43:12.156273+0000","last_clean":"2024-03-30T00:43:12.156273+0000","last_became_active":"2024-03-30T00:43:11.162238+0000","last_became_peered":"2024-03-30T00:43:11.162238+0000","last_unstale":"2024-03-30T00:43:12.156273+0000","last_undegraded":"2024-03-30T00:43:12.156273+0000","last_fullsized":"2024-03-30T00:43:12.156273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:51.250839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"67'6","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.187674+0000","last_change":"2024-03-30T00:43:10.750590+0000","last_active":"2024-03-30T00:43:11.187674+0000","last_peered":"2024-03-30T00:43:11.187674+0000","last_clean":"2024-03-30T00:43:11.187674+0000","last_became_active":"2024-03-30T00:43:10.749733+0000","last_became_peered":"2024-03-30T00:43:10.749733+0000","last_unstale":"2024-03-30T00:43:11.187674+0000","last_undegraded":"2024-03-30T00:43:11.187674+0000","last_fullsized":"2024-03-30T00:43:11.187674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:23.758029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.154966+0000","last_change":"2024-03-30T00:43:11.165002+0000","last_active":"2024-03-30T00:43:12.154966+0000","last_peered":"2024-03-30T00:43:12.154966+0000","last_clean":"2024-03-30T00:43:12.154966+0000","last_became_active":"2024-03-30T00:43:11.164463+0000","last_became_peered":"2024-03-30T00:43:11.164463+0000","last_unstale":"2024-03-30T00:43:12.154966+0000","last_undegraded":"2024-03-30T00:43:12.154966+0000","last_fullsized":"2024-03-30T00:43:12.154966+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:31.136461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"67'2","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155407+0000","last_change":"2024-03-30T00:43:10.764960+0000","last_active":"2024-03-30T00:43:12.155407+0000","last_peered":"2024-03-30T00:43:12.155407+0000","last_clean":"2024-03-30T00:43:12.155407+0000","last_became_active":"2024-03-30T00:43:10.764461+0000","last_became_peered":"2024-03-30T00:43:10.764461+0000","last_unstale":"2024-03-30T00:43:12.155407+0000","last_undegraded":"2024-03-30T00:43:12.155407+0000","last_fullsized":"2024-03-30T00:43:12.155407+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:28.430262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.180002+0000","last_change":"2024-03-30T00:43:11.175247+0000","last_active":"2024-03-30T00:43:11.180002+0000","last_peered":"2024-03-30T00:43:11.180002+0000","last_clean":"2024-03-30T00:43:11.180002+0000","last_became_active":"2024-03-30T00:43:11.174854+0000","last_became_peered":"2024-03-30T00:43:11.174854+0000","last_unstale":"2024-03-30T00:43:11.180002+0000","last_undegraded":"2024-03-30T00:43:11.180002+0000","last_fullsized":"2024-03-30T00:43:11.180002+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:41:03.894477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.158819+0000","last_change":"2024-03-30T00:43:10.760606+0000","last_active":"2024-03-30T00:43:11.158819+0000","last_peered":"2024-03-30T00:43:11.158819+0000","last_clean":"2024-03-30T00:43:11.158819+0000","last_became_active":"2024-03-30T00:43:10.759487+0000","last_became_peered":"2024-03-30T00:43:10.759487+0000","last_unstale":"2024-03-30T00:43:11.158819+0000","last_undegraded":"2024-03-30T00:43:11.158819+0000","last_fullsized":"2024-03-30T00:43:11.158819+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:21:59.823030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902083+0000","last_change":"2024-03-30T00:43:11.172357+0000","last_active":"2024-03-30T00:43:12.902083+0000","last_peered":"2024-03-30T00:43:12.902083+0000","last_clean":"2024-03-30T00:43:12.902083+0000","last_became_active":"2024-03-30T00:43:11.172092+0000","last_became_peered":"2024-03-30T00:43:11.172092+0000","last_unstale":"2024-03-30T00:43:12.902083+0000","last_undegraded":"2024-03-30T00:43:12.902083+0000","last_fullsized":"2024-03-30T00:43:12.902083+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:12:08.266211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155852+0000","last_change":"2024-03-30T00:43:10.755458+0000","last_active":"2024-03-30T00:43:12.155852+0000","last_peered":"2024-03-30T00:43:12.155852+0000","last_clean":"2024-03-30T00:43:12.155852+0000","last_became_active":"2024-03-30T00:43:10.755186+0000","last_became_peered":"2024-03-30T00:43:10.755186+0000","last_unstale":"2024-03-30T00:43:12.155852+0000","last_undegraded":"2024-03-30T00:43:12.155852+0000","last_fullsized":"2024-03-30T00:43:12.155852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:29:21.858586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.159667+0000","last_change":"2024-03-30T00:43:11.157247+0000","last_active":"2024-03-30T00:43:11.159667+0000","last_peered":"2024-03-30T00:43:11.159667+0000","last_clean":"2024-03-30T00:43:11.159667+0000","last_became_active":"2024-03-30T00:43:11.156989+0000","last_became_peered":"2024-03-30T00:43:11.156989+0000","last_unstale":"2024-03-30T00:43:11.159667+0000","last_undegraded":"2024-03-30T00:43:11.159667+0000","last_fullsized":"2024-03-30T00:43:11.159667+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:12.337025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"67'8","reported_seq":19,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155820+0000","last_change":"2024-03-30T00:43:10.763891+0000","last_active":"2024-03-30T00:43:12.155820+0000","last_peered":"2024-03-30T00:43:12.155820+0000","last_clean":"2024-03-30T00:43:12.155820+0000","last_became_active":"2024-03-30T00:43:10.760736+0000","last_became_peered":"2024-03-30T00:43:10.760736+0000","last_unstale":"2024-03-30T00:43:12.155820+0000","last_undegraded":"2024-03-30T00:43:12.155820+0000","last_fullsized":"2024-03-30T00:43:12.155820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:28.430262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156480+0000","last_change":"2024-03-30T00:43:11.185385+0000","last_active":"2024-03-30T00:43:12.156480+0000","last_peered":"2024-03-30T00:43:12.156480+0000","last_clean":"2024-03-30T00:43:12.156480+0000","last_became_active":"2024-03-30T00:43:11.185144+0000","last_became_peered":"2024-03-30T00:43:11.185144+0000","last_unstale":"2024-03-30T00:43:12.156480+0000","last_undegraded":"2024-03-30T00:43:12.156480+0000","last_fullsized":"2024-03-30T00:43:12.156480+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:10:52.232984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.178218+0000","last_change":"2024-03-30T00:43:10.763317+0000","last_active":"2024-03-30T00:43:11.178218+0000","last_peered":"2024-03-30T00:43:11.178218+0000","last_clean":"2024-03-30T00:43:11.178218+0000","last_became_active":"2024-03-30T00:43:10.763065+0000","last_became_peered":"2024-03-30T00:43:10.763065+0000","last_unstale":"2024-03-30T00:43:11.178218+0000","last_undegraded":"2024-03-30T00:43:11.178218+0000","last_fullsized":"2024-03-30T00:43:11.178218+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:10:43.217235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902399+0000","last_change":"2024-03-30T00:43:11.181397+0000","last_active":"2024-03-30T00:43:12.902399+0000","last_peered":"2024-03-30T00:43:12.902399+0000","last_clean":"2024-03-30T00:43:12.902399+0000","last_became_active":"2024-03-30T00:43:11.181130+0000","last_became_peered":"2024-03-30T00:43:11.181130+0000","last_unstale":"2024-03-30T00:43:12.902399+0000","last_undegraded":"2024-03-30T00:43:12.902399+0000","last_fullsized":"2024-03-30T00:43:12.902399+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:57:25.787445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"67'2","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155809+0000","last_change":"2024-03-30T00:43:10.764643+0000","last_active":"2024-03-30T00:43:12.155809+0000","last_peered":"2024-03-30T00:43:12.155809+0000","last_clean":"2024-03-30T00:43:12.155809+0000","last_became_active":"2024-03-30T00:43:10.763645+0000","last_became_peered":"2024-03-30T00:43:10.763645+0000","last_unstale":"2024-03-30T00:43:12.155809+0000","last_undegraded":"2024-03-30T00:43:12.155809+0000","last_fullsized":"2024-03-30T00:43:12.155809+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:05:32.100565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156565+0000","last_change":"2024-03-30T00:43:11.174140+0000","last_active":"2024-03-30T00:43:12.156565+0000","last_peered":"2024-03-30T00:43:12.156565+0000","last_clean":"2024-03-30T00:43:12.156565+0000","last_became_active":"2024-03-30T00:43:11.173894+0000","last_became_peered":"2024-03-30T00:43:11.173894+0000","last_unstale":"2024-03-30T00:43:12.156565+0000","last_undegraded":"2024-03-30T00:43:12.156565+0000","last_fullsized":"2024-03-30T00:43:12.156565+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:10:52.232984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"67'1","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156024+0000","last_change":"2024-03-30T00:43:10.758412+0000","last_active":"2024-03-30T00:43:12.156024+0000","last_peered":"2024-03-30T00:43:12.156024+0000","last_clean":"2024-03-30T00:43:12.156024+0000","last_became_active":"2024-03-30T00:43:10.758104+0000","last_became_peered":"2024-03-30T00:43:10.758104+0000","last_unstale":"2024-03-30T00:43:12.156024+0000","last_undegraded":"2024-03-30T00:43:12.156024+0000","last_fullsized":"2024-03-30T00:43:12.156024+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:24.416987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902436+0000","last_change":"2024-03-30T00:43:11.179456+0000","last_active":"2024-03-30T00:43:12.902436+0000","last_peered":"2024-03-30T00:43:12.902436+0000","last_clean":"2024-03-30T00:43:12.902436+0000","last_became_active":"2024-03-30T00:43:11.179197+0000","last_became_peered":"2024-03-30T00:43:11.179197+0000","last_unstale":"2024-03-30T00:43:12.902436+0000","last_undegraded":"2024-03-30T00:43:12.902436+0000","last_fullsized":"2024-03-30T00:43:12.902436+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:25:40.893560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.150457+0000","last_change":"2024-03-30T00:43:10.755679+0000","last_active":"2024-03-30T00:43:11.150457+0000","last_peered":"2024-03-30T00:43:11.150457+0000","last_clean":"2024-03-30T00:43:11.150457+0000","last_became_active":"2024-03-30T00:43:10.755401+0000","last_became_peered":"2024-03-30T00:43:10.755401+0000","last_unstale":"2024-03-30T00:43:11.150457+0000","last_undegraded":"2024-03-30T00:43:11.150457+0000","last_fullsized":"2024-03-30T00:43:11.150457+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:41:02.892129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783621+0000","last_change":"2024-03-30T00:43:11.169750+0000","last_active":"2024-03-30T00:43:12.783621+0000","last_peered":"2024-03-30T00:43:12.783621+0000","last_clean":"2024-03-30T00:43:12.783621+0000","last_became_active":"2024-03-30T00:43:11.169185+0000","last_became_peered":"2024-03-30T00:43:11.169185+0000","last_unstale":"2024-03-30T00:43:12.783621+0000","last_undegraded":"2024-03-30T00:43:12.783621+0000","last_fullsized":"2024-03-30T00:43:12.783621+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:25:29.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"67'2","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783768+0000","last_change":"2024-03-30T00:43:10.765327+0000","last_active":"2024-03-30T00:43:12.783768+0000","last_peered":"2024-03-30T00:43:12.783768+0000","last_clean":"2024-03-30T00:43:12.783768+0000","last_became_active":"2024-03-30T00:43:10.765054+0000","last_became_peered":"2024-03-30T00:43:10.765054+0000","last_unstale":"2024-03-30T00:43:12.783768+0000","last_undegraded":"2024-03-30T00:43:12.783768+0000","last_fullsized":"2024-03-30T00:43:12.783768+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:46:12.400403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.179261+0000","last_change":"2024-03-30T00:43:11.174993+0000","last_active":"2024-03-30T00:43:11.179261+0000","last_peered":"2024-03-30T00:43:11.179261+0000","last_clean":"2024-03-30T00:43:11.179261+0000","last_became_active":"2024-03-30T00:43:11.174629+0000","last_became_peered":"2024-03-30T00:43:11.174629+0000","last_unstale":"2024-03-30T00:43:11.179261+0000","last_undegraded":"2024-03-30T00:43:11.179261+0000","last_fullsized":"2024-03-30T00:43:11.179261+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:46.189391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.158921+0000","last_change":"2024-03-30T00:43:10.758820+0000","last_active":"2024-03-30T00:43:11.158921+0000","last_peered":"2024-03-30T00:43:11.158921+0000","last_clean":"2024-03-30T00:43:11.158921+0000","last_became_active":"2024-03-30T00:43:10.757306+0000","last_became_peered":"2024-03-30T00:43:10.757306+0000","last_unstale":"2024-03-30T00:43:11.158921+0000","last_undegraded":"2024-03-30T00:43:11.158921+0000","last_fullsized":"2024-03-30T00:43:11.158921+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:28.430262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902182+0000","last_change":"2024-03-30T00:43:11.170005+0000","last_active":"2024-03-30T00:43:12.902182+0000","last_peered":"2024-03-30T00:43:12.902182+0000","last_clean":"2024-03-30T00:43:12.902182+0000","last_became_active":"2024-03-30T00:43:11.169646+0000","last_became_peered":"2024-03-30T00:43:11.169646+0000","last_unstale":"2024-03-30T00:43:12.902182+0000","last_undegraded":"2024-03-30T00:43:12.902182+0000","last_fullsized":"2024-03-30T00:43:12.902182+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:04:38.382257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783552+0000","last_change":"2024-03-30T00:43:10.753839+0000","last_active":"2024-03-30T00:43:12.783552+0000","last_peered":"2024-03-30T00:43:12.783552+0000","last_clean":"2024-03-30T00:43:12.783552+0000","last_became_active":"2024-03-30T00:43:10.753578+0000","last_became_peered":"2024-03-30T00:43:10.753578+0000","last_unstale":"2024-03-30T00:43:12.783552+0000","last_undegraded":"2024-03-30T00:43:12.783552+0000","last_fullsized":"2024-03-30T00:43:12.783552+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:41:02.892129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783692+0000","last_change":"2024-03-30T00:43:11.164616+0000","last_active":"2024-03-30T00:43:12.783692+0000","last_peered":"2024-03-30T00:43:12.783692+0000","last_clean":"2024-03-30T00:43:12.783692+0000","last_became_active":"2024-03-30T00:43:11.164309+0000","last_became_peered":"2024-03-30T00:43:11.164309+0000","last_unstale":"2024-03-30T00:43:12.783692+0000","last_undegraded":"2024-03-30T00:43:12.783692+0000","last_fullsized":"2024-03-30T00:43:12.783692+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:46.420657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.178768+0000","last_change":"2024-03-30T00:43:10.758910+0000","last_active":"2024-03-30T00:43:11.178768+0000","last_peered":"2024-03-30T00:43:11.178768+0000","last_clean":"2024-03-30T00:43:11.178768+0000","last_became_active":"2024-03-30T00:43:10.758600+0000","last_became_peered":"2024-03-30T00:43:10.758600+0000","last_unstale":"2024-03-30T00:43:11.178768+0000","last_undegraded":"2024-03-30T00:43:11.178768+0000","last_fullsized":"2024-03-30T00:43:11.178768+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:29:21.858586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156037+0000","last_change":"2024-03-30T00:43:11.166129+0000","last_active":"2024-03-30T00:43:12.156037+0000","last_peered":"2024-03-30T00:43:12.156037+0000","last_clean":"2024-03-30T00:43:12.156037+0000","last_became_active":"2024-03-30T00:43:11.165827+0000","last_became_peered":"2024-03-30T00:43:11.165827+0000","last_unstale":"2024-03-30T00:43:12.156037+0000","last_undegraded":"2024-03-30T00:43:12.156037+0000","last_fullsized":"2024-03-30T00:43:12.156037+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:17:08.835092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.161855+0000","last_change":"2024-03-30T00:43:10.762929+0000","last_active":"2024-03-30T00:43:11.161855+0000","last_peered":"2024-03-30T00:43:11.161855+0000","last_clean":"2024-03-30T00:43:11.161855+0000","last_became_active":"2024-03-30T00:43:10.762592+0000","last_became_peered":"2024-03-30T00:43:10.762592+0000","last_unstale":"2024-03-30T00:43:11.161855+0000","last_undegraded":"2024-03-30T00:43:11.161855+0000","last_fullsized":"2024-03-30T00:43:11.161855+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:35:16.490841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.180079+0000","last_change":"2024-03-30T00:43:11.175500+0000","last_active":"2024-03-30T00:43:11.180079+0000","last_peered":"2024-03-30T00:43:11.180079+0000","last_clean":"2024-03-30T00:43:11.180079+0000","last_became_active":"2024-03-30T00:43:11.174800+0000","last_became_peered":"2024-03-30T00:43:11.174800+0000","last_unstale":"2024-03-30T00:43:11.180079+0000","last_undegraded":"2024-03-30T00:43:11.180079+0000","last_fullsized":"2024-03-30T00:43:11.180079+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:29:23.961976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783931+0000","last_change":"2024-03-30T00:43:10.761241+0000","last_active":"2024-03-30T00:43:12.783931+0000","last_peered":"2024-03-30T00:43:12.783931+0000","last_clean":"2024-03-30T00:43:12.783931+0000","last_became_active":"2024-03-30T00:43:10.760880+0000","last_became_peered":"2024-03-30T00:43:10.760880+0000","last_unstale":"2024-03-30T00:43:12.783931+0000","last_undegraded":"2024-03-30T00:43:12.783931+0000","last_fullsized":"2024-03-30T00:43:12.783931+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:26:27.085243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902429+0000","last_change":"2024-03-30T00:43:11.170993+0000","last_active":"2024-03-30T00:43:12.902429+0000","last_peered":"2024-03-30T00:43:12.902429+0000","last_clean":"2024-03-30T00:43:12.902429+0000","last_became_active":"2024-03-30T00:43:11.170693+0000","last_became_peered":"2024-03-30T00:43:11.170693+0000","last_unstale":"2024-03-30T00:43:12.902429+0000","last_undegraded":"2024-03-30T00:43:12.902429+0000","last_fullsized":"2024-03-30T00:43:12.902429+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:31.904282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.146864+0000","last_change":"2024-03-30T00:43:10.750485+0000","last_active":"2024-03-30T00:43:11.146864+0000","last_peered":"2024-03-30T00:43:11.146864+0000","last_clean":"2024-03-30T00:43:11.146864+0000","last_became_active":"2024-03-30T00:43:10.749356+0000","last_became_peered":"2024-03-30T00:43:10.749356+0000","last_unstale":"2024-03-30T00:43:11.146864+0000","last_undegraded":"2024-03-30T00:43:11.146864+0000","last_fullsized":"2024-03-30T00:43:11.146864+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:05:32.100565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902001+0000","last_change":"2024-03-30T00:43:11.162930+0000","last_active":"2024-03-30T00:43:12.902001+0000","last_peered":"2024-03-30T00:43:12.902001+0000","last_clean":"2024-03-30T00:43:12.902001+0000","last_became_active":"2024-03-30T00:43:11.162665+0000","last_became_peered":"2024-03-30T00:43:11.162665+0000","last_unstale":"2024-03-30T00:43:12.902001+0000","last_undegraded":"2024-03-30T00:43:12.902001+0000","last_fullsized":"2024-03-30T00:43:12.902001+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:53:40.511494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155531+0000","last_change":"2024-03-30T00:43:10.757217+0000","last_active":"2024-03-30T00:43:12.155531+0000","last_peered":"2024-03-30T00:43:12.155531+0000","last_clean":"2024-03-30T00:43:12.155531+0000","last_became_active":"2024-03-30T00:43:10.756873+0000","last_became_peered":"2024-03-30T00:43:10.756873+0000","last_unstale":"2024-03-30T00:43:12.155531+0000","last_undegraded":"2024-03-30T00:43:12.155531+0000","last_fullsized":"2024-03-30T00:43:12.155531+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:35:16.490841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156519+0000","last_change":"2024-03-30T00:43:11.179302+0000","last_active":"2024-03-30T00:43:12.156519+0000","last_peered":"2024-03-30T00:43:12.156519+0000","last_clean":"2024-03-30T00:43:12.156519+0000","last_became_active":"2024-03-30T00:43:11.177661+0000","last_became_peered":"2024-03-30T00:43:11.177661+0000","last_unstale":"2024-03-30T00:43:12.156519+0000","last_undegraded":"2024-03-30T00:43:12.156519+0000","last_fullsized":"2024-03-30T00:43:12.156519+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:56:57.944571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.149847+0000","last_change":"2024-03-30T00:43:10.752788+0000","last_active":"2024-03-30T00:43:11.149847+0000","last_peered":"2024-03-30T00:43:11.149847+0000","last_clean":"2024-03-30T00:43:11.149847+0000","last_became_active":"2024-03-30T00:43:10.752487+0000","last_became_peered":"2024-03-30T00:43:10.752487+0000","last_unstale":"2024-03-30T00:43:11.149847+0000","last_undegraded":"2024-03-30T00:43:11.149847+0000","last_fullsized":"2024-03-30T00:43:11.149847+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:29:22.959628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.158024+0000","last_change":"2024-03-30T00:43:11.155067+0000","last_active":"2024-03-30T00:43:11.158024+0000","last_peered":"2024-03-30T00:43:11.158024+0000","last_clean":"2024-03-30T00:43:11.158024+0000","last_became_active":"2024-03-30T00:43:11.154706+0000","last_became_peered":"2024-03-30T00:43:11.154706+0000","last_unstale":"2024-03-30T00:43:11.158024+0000","last_undegraded":"2024-03-30T00:43:11.158024+0000","last_fullsized":"2024-03-30T00:43:11.158024+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:51.250839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.150101+0000","last_change":"2024-03-30T00:43:10.762271+0000","last_active":"2024-03-30T00:43:11.150101+0000","last_peered":"2024-03-30T00:43:11.150101+0000","last_clean":"2024-03-30T00:43:11.150101+0000","last_became_active":"2024-03-30T00:43:10.761991+0000","last_became_peered":"2024-03-30T00:43:10.761991+0000","last_unstale":"2024-03-30T00:43:11.150101+0000","last_undegraded":"2024-03-30T00:43:11.150101+0000","last_fullsized":"2024-03-30T00:43:11.150101+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:46:12.400403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155906+0000","last_change":"2024-03-30T00:43:11.168893+0000","last_active":"2024-03-30T00:43:12.155906+0000","last_peered":"2024-03-30T00:43:12.155906+0000","last_clean":"2024-03-30T00:43:12.155906+0000","last_became_active":"2024-03-30T00:43:11.167502+0000","last_became_peered":"2024-03-30T00:43:11.167502+0000","last_unstale":"2024-03-30T00:43:12.155906+0000","last_undegraded":"2024-03-30T00:43:12.155906+0000","last_fullsized":"2024-03-30T00:43:12.155906+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:15:10.075537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.145553+0000","last_change":"2024-03-30T00:43:10.753455+0000","last_active":"2024-03-30T00:43:11.145553+0000","last_peered":"2024-03-30T00:43:11.145553+0000","last_clean":"2024-03-30T00:43:11.145553+0000","last_became_active":"2024-03-30T00:43:10.753160+0000","last_became_peered":"2024-03-30T00:43:10.753160+0000","last_unstale":"2024-03-30T00:43:11.145553+0000","last_undegraded":"2024-03-30T00:43:11.145553+0000","last_fullsized":"2024-03-30T00:43:11.145553+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:35:16.490841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902108+0000","last_change":"2024-03-30T00:43:11.169291+0000","last_active":"2024-03-30T00:43:12.902108+0000","last_peered":"2024-03-30T00:43:12.902108+0000","last_clean":"2024-03-30T00:43:12.902108+0000","last_became_active":"2024-03-30T00:43:11.168708+0000","last_became_peered":"2024-03-30T00:43:11.168708+0000","last_unstale":"2024-03-30T00:43:12.902108+0000","last_undegraded":"2024-03-30T00:43:12.902108+0000","last_fullsized":"2024-03-30T00:43:12.902108+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:43:22.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.149484+0000","last_change":"2024-03-30T00:43:10.755218+0000","last_active":"2024-03-30T00:43:11.149484+0000","last_peered":"2024-03-30T00:43:11.149484+0000","last_clean":"2024-03-30T00:43:11.149484+0000","last_became_active":"2024-03-30T00:43:10.754957+0000","last_became_peered":"2024-03-30T00:43:10.754957+0000","last_unstale":"2024-03-30T00:43:11.149484+0000","last_undegraded":"2024-03-30T00:43:11.149484+0000","last_fullsized":"2024-03-30T00:43:11.149484+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:02.108234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.150035+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:43:11.150035+0000","last_peered":"2024-03-30T00:43:11.150035+0000","last_clean":"2024-03-30T00:43:11.150035+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:43:11.150035+0000","last_undegraded":"2024-03-30T00:43:11.150035+0000","last_fullsized":"2024-03-30T00:43:11.150035+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1433600,"data_stored":960382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65200000000000002,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.501},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.63600000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1459999999999999,"5min":1.149,"15min":1.149},"last":0.71099999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.64100000000000001,"15min":0.626},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92900000000000005,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2450000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76600000000000001,"15min":0.78800000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.91600000000000004,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77800000000000002,"15min":0.79600000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2270000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.81399999999999995,"15min":0.82599999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.996,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.79500000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.49099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.65800000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0609999999999999,"5min":1.121,"15min":1.121},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.504,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0049999999999999,"5min":1.026,"15min":1.026},"last":0.81699999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47999999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":1.214,"15min":1.214},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.0189999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.83799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":815104,"data_stored":362652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.83299999999999996,"15min":0.91800000000000004},"min":{"1min":0.46700000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86199999999999999,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":1.2450000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91400000000000003,"5min":1.47,"15min":1.47},"last":0.71999999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.83999999999999997,"15min":0.93200000000000005},"min":{"1min":0.48199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86899999999999999,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.90700000000000003,"15min":1.0109999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93700000000000006,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":1.018}]},{"osd":2,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.90100000000000002,"15min":0.995},"min":{"1min":0.501,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90000000000000002,"5min":4.375,"15min":4.375},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.90100000000000002,"15min":1},"min":{"1min":0.50800000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84799999999999998,"5min":4.2960000000000003,"15min":4.2960000000000003},"last":0.76900000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.92000000000000004,"15min":1.016},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92700000000000005,"5min":4.3399999999999999,"15min":4.3399999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.89800000000000002,"15min":1.008},"min":{"1min":0.505,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81599999999999995,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.747}]},{"osd":4,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.40400000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.86399999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.53000000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90200000000000002,"5min":1.036,"15min":1.036},"last":0.88300000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.67900000000000005,"15min":0.69699999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.91900000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":1.101}]},{"osd":7,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71999999999999997,"15min":0.72299999999999998},"min":{"1min":0.49299999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.52800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89100000000000001,"5min":1.036,"15min":1.036},"last":0.84799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":815104,"data_stored":362640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65000000000000002,"15min":0.66600000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":1.095,"15min":1.095},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.86299999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.73299999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88400000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70499999999999996,"15min":0.73299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.87,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69299999999999995,"15min":0.71799999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92900000000000005,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.442}]},{"osd":3,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.76200000000000001,"15min":0.80200000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":1.7,"15min":1.7},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73999999999999999,"15min":0.76300000000000001},"min":{"1min":0.53200000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93500000000000005,"5min":1.181,"15min":1.181},"last":0.57299999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.76600000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.75,"15min":0.78000000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89300000000000002,"5min":1.615,"15min":1.615},"last":0.78000000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66200000000000003,"15min":0.67900000000000005},"min":{"1min":0.40699999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.877,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97899999999999998,"5min":1.569,"15min":1.569},"last":0.66100000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.745,"15min":0.75800000000000001},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.82299999999999995,"15min":0.84399999999999997},"min":{"1min":0.54300000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.03,"5min":1.383,"15min":1.383},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":835584,"data_stored":370802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.68600000000000005},"min":{"1min":0.441,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1699999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.64300000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90200000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71799999999999997,"15min":0.72999999999999998},"min":{"1min":0.47499999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94299999999999995,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71399999999999997,"15min":0.72299999999999998},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97999999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52100000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.92200000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.76200000000000001},"min":{"1min":0.53700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2190000000000001,"5min":1.228,"15min":1.228},"last":0.84399999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.196,"5min":1.27,"15min":1.27},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77800000000000002,"15min":0.78700000000000003},"min":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.95999999999999996,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.99299999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67800000000000005,"15min":0.70099999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80300000000000005,"5min":1.329,"15min":1.329},"last":0.873},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.71999999999999997,"15min":0.75800000000000001},"min":{"1min":0.44,"5min":0.373,"15min":0.373},"max":{"1min":0.83099999999999996,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.90000000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.68300000000000005,"15min":0.71899999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89100000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.73699999999999999,"15min":0.77000000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.93600000000000005,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.69099999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.72399999999999998,"15min":0.754},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96899999999999997,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.81100000000000005,"15min":0.83999999999999997},"min":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93000000000000005,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.85599999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":362618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47199999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.437},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51700000000000002,"15min":0.54800000000000004},"min":{"1min":0.35699999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.65800000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45300000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.53500000000000003}]},{"osd":2,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.5,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68600000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57399999999999995,"15min":0.59199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70399999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47399999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.75700000000000001,"15min":0.77500000000000002},"min":{"1min":0.60499999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.76300000000000001,"15min":0.78300000000000003},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98099999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.61199999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1060000000000001,"5min":1.125,"15min":1.125},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78900000000000003,"15min":0.81499999999999995},"min":{"1min":0.55200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93300000000000005,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80300000000000005,"15min":0.82399999999999995},"min":{"1min":0.624,"5min":0.499,"15min":0.499},"max":{"1min":1.1699999999999999,"5min":1.226,"15min":1.226},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0129999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.621}]},{"osd":7,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.86299999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0700000000000001,"5min":1.202,"15min":1.202},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.871,"15min":0.90400000000000003},"min":{"1min":0.74399999999999999,"5min":0.497,"15min":0.497},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.71699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":823296,"data_stored":369238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.46600000000000003,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.0999999999999996,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.45500000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.49099999999999999,"15min":0.501},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.437}]},{"osd":4,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.80800000000000005,"15min":0.81699999999999995},"min":{"1min":0.60099999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.85799999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.53400000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81200000000000006,"15min":0.82499999999999996},"min":{"1min":0.56699999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.56399999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.85799999999999998,"15min":0.88100000000000001},"min":{"1min":0.60099999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.495},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.57099999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":839680,"data_stored":364946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65600000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.32800000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45600000000000002,"15min":0.46800000000000003},"min":{"1min":0.38,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.48899999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.745,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.48599999999999999,"15min":0.46999999999999997},"min":{"1min":0.41499999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.374}]},{"osd":4,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.72899999999999998,"15min":0.71599999999999997},"min":{"1min":0.56899999999999995,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.61499999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.84799999999999998}]},{"osd":5,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.69899999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.014,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.67300000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.0669999999999999,"5min":1.097,"15min":1.097},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.78300000000000003,"15min":0.77400000000000002},"min":{"1min":0.69299999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.86199999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.80200000000000005,"15min":0.80000000000000004},"min":{"1min":0.65000000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.79600000000000004,"15min":0.78000000000000003},"min":{"1min":0.74299999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.877}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28396,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343444,"statfs":{"total":96636764160,"available":96607686656,"internally_reserved":0,"allocated":1404928,"data_stored":952986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46700000000000003,"15min":0.48099999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75800000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.34499999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.50600000000000001,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.497},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.56100000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.505,"15min":0.51400000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53400000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79700000000000004,"15min":0.79600000000000004},"min":{"1min":0.504,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.234,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.81200000000000006,"15min":0.80400000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.64500000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2609999999999999,"5min":1.462,"15min":1.462},"last":0.57399999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80000000000000004,"15min":0.80200000000000005},"min":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.196,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78100000000000003,"15min":0.78000000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.60699999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.351,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.80100000000000005,"15min":0.80000000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1220000000000001,"5min":1.556,"15min":1.556},"last":0.79600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:43:15.255 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:43:15.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458ee00640 1 Processor -- start 2024-03-30T00:43:15.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458ee00640 1 -- start start 2024-03-30T00:43:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458815aa40 0x7f45880cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458ee00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f458815e5d0 0x7f45880ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458ee00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f458815ae20 0x7f45880ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458ee00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458815eab0 con 0x7f458815ae20 2024-03-30T00:43:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458ee00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458815ed90 con 0x7f458815e5d0 2024-03-30T00:43:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458815f070 con 0x7f458815aa40 2024-03-30T00:43:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f458815ae20 0x7f45880ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f458815ae20 0x7f45880ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44256/0 (socket says 172.21.15.66:44256) 2024-03-30T00:43:15.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458e400640 1 -- 172.21.15.66:0/2651383441 learned_addr learned my addr 172.21.15.66:0/2651383441 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:15.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458d000640 1 --2- 172.21.15.66:0/2651383441 >> v2:172.21.15.156:3300/0 conn(0x7f458815e5d0 0x7f45880ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458da00640 1 --2- 172.21.15.66:0/2651383441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458815aa40 0x7f45880cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458e400640 1 -- 172.21.15.66:0/2651383441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458815aa40 msgr2=0x7f45880cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:15.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458e400640 1 --2- 172.21.15.66:0/2651383441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458815aa40 0x7f45880cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458e400640 1 -- 172.21.15.66:0/2651383441 >> v2:172.21.15.156:3300/0 conn(0x7f458815e5d0 msgr2=0x7f45880ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:15.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458e400640 1 --2- 172.21.15.66:0/2651383441 >> v2:172.21.15.156:3300/0 conn(0x7f458815e5d0 0x7f45880ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458e400640 1 -- 172.21.15.66:0/2651383441 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458815f350 con 0x7f458815ae20 2024-03-30T00:43:15.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.432+0000 7f458d000640 1 --2- 172.21.15.66:0/2651383441 >> v2:172.21.15.156:3300/0 conn(0x7f458815e5d0 0x7f45880ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:15.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458e400640 1 --2- 172.21.15.66:0/2651383441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458815ae20 0x7f45880ccde0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f45880d14e0 tx=0x7f458001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365adc43b4fb32b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:15.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f4587e00640 1 -- 172.21.15.66:0/2651383441 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45800a57c0 con 0x7f458815ae20 2024-03-30T00:43:15.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f4587e00640 1 -- 172.21.15.66:0/2651383441 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45800a5920 con 0x7f458815ae20 2024-03-30T00:43:15.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f4587e00640 1 -- 172.21.15.66:0/2651383441 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45800a5cb0 con 0x7f458815ae20 2024-03-30T00:43:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458ee00640 1 -- 172.21.15.66:0/2651383441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458815ae20 msgr2=0x7f45880ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458ee00640 1 --2- 172.21.15.66:0/2651383441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458815ae20 0x7f45880ccde0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f45880d14e0 tx=0x7f458001c960 comp rx=0 tx=0).stop 2024-03-30T00:43:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458ee00640 1 -- 172.21.15.66:0/2651383441 shutdown_connections 2024-03-30T00:43:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458ee00640 1 --2- 172.21.15.66:0/2651383441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458815ae20 0x7f45880ccde0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458ee00640 1 --2- 172.21.15.66:0/2651383441 >> v2:172.21.15.156:3300/0 conn(0x7f458815e5d0 0x7f45880ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458ee00640 1 --2- 172.21.15.66:0/2651383441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458815aa40 0x7f45880cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458ee00640 1 -- 172.21.15.66:0/2651383441 >> 172.21.15.66:0/2651383441 conn(0x7f458806e0d0 msgr2=0x7f458806e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458ee00640 1 -- 172.21.15.66:0/2651383441 shutdown_connections 2024-03-30T00:43:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458ee00640 1 -- 172.21.15.66:0/2651383441 wait complete. 2024-03-30T00:43:15.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458ee00640 1 Processor -- start 2024-03-30T00:43:15.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.436+0000 7f458ee00640 1 -- start start 2024-03-30T00:43:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458ee00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f458815aa40 0x7f4588156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458815ae20 0x7f4588163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458815e5d0 0x7f4588159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4588150d60 con 0x7f458815ae20 2024-03-30T00:43:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458ee00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458806af40 con 0x7f458815aa40 2024-03-30T00:43:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458806b220 con 0x7f458815e5d0 2024-03-30T00:43:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458815ae20 0x7f4588163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458815ae20 0x7f4588163ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44268/0 (socket says 172.21.15.66:44268) 2024-03-30T00:43:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458d000640 1 -- 172.21.15.66:0/4122746131 learned_addr learned my addr 172.21.15.66:0/4122746131 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458da00640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f458815aa40 0x7f4588156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458e400640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458815e5d0 0x7f4588159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458d000640 1 -- 172.21.15.66:0/4122746131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458815e5d0 msgr2=0x7f4588159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458d000640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458815e5d0 0x7f4588159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458d000640 1 -- 172.21.15.66:0/4122746131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f458815aa40 msgr2=0x7f4588156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458d000640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f458815aa40 0x7f4588156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458d000640 1 -- 172.21.15.66:0/4122746131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4580009b30 con 0x7f458815ae20 2024-03-30T00:43:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458d000640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458815ae20 0x7f4588163ad0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f457400d800 tx=0x7f457400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f4585600640 1 -- 172.21.15.66:0/4122746131 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45740096c0 con 0x7f458815ae20 2024-03-30T00:43:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f4585600640 1 -- 172.21.15.66:0/4122746131 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4574009820 con 0x7f458815ae20 2024-03-30T00:43:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f4585600640 1 -- 172.21.15.66:0/4122746131 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574009bd0 con 0x7f458815ae20 2024-03-30T00:43:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458ee00640 1 -- 172.21.15.66:0/4122746131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458806b500 con 0x7f458815ae20 2024-03-30T00:43:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458ee00640 1 -- 172.21.15.66:0/4122746131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4588159e80 con 0x7f458815ae20 2024-03-30T00:43:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.440+0000 7f458ee00640 1 -- 172.21.15.66:0/4122746131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4588073cf0 con 0x7f458815ae20 2024-03-30T00:43:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.444+0000 7f4585600640 1 -- 172.21.15.66:0/4122746131 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4574009d30 con 0x7f458815ae20 2024-03-30T00:43:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.444+0000 7f4585600640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4568066690 0x7f4568068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.444+0000 7f4585600640 1 -- 172.21.15.66:0/4122746131 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f45740848f0 con 0x7f458815ae20 2024-03-30T00:43:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.444+0000 7f458da00640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4568066690 0x7f4568068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.444+0000 7f458da00640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4568066690 0x7f4568068b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4588154700 tx=0x7f457c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:15.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.448+0000 7f4585600640 1 -- 172.21.15.66:0/4122746131 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4574051b30 con 0x7f458815ae20 2024-03-30T00:43:15.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.604+0000 7f458ee00640 1 -- 172.21.15.66:0/4122746131 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45880b1f20 con 0x7f4568066690 2024-03-30T00:43:15.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.612+0000 7f4585600640 1 -- 172.21.15.66:0/4122746131 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289301 (secure 0 0 0) 0x7f457c00b720 con 0x7f4568066690 2024-03-30T00:43:15.622 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:15.624 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:43:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 -- 172.21.15.66:0/4122746131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4568066690 msgr2=0x7f4568068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4568066690 0x7f4568068b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4588154700 tx=0x7f457c008040 comp rx=0 tx=0).stop 2024-03-30T00:43:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 -- 172.21.15.66:0/4122746131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458815ae20 msgr2=0x7f4588163ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458815ae20 0x7f4588163ad0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f457400d800 tx=0x7f457400dcc0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 -- 172.21.15.66:0/4122746131 shutdown_connections 2024-03-30T00:43:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458815e5d0 0x7f4588159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4568066690 0x7f4568068b50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458815ae20 0x7f4588163ad0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 --2- 172.21.15.66:0/4122746131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f458815aa40 0x7f4588156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 -- 172.21.15.66:0/4122746131 >> 172.21.15.66:0/4122746131 conn(0x7f458806e0d0 msgr2=0x7f45880b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 -- 172.21.15.66:0/4122746131 shutdown_connections 2024-03-30T00:43:15.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.616+0000 7f458ee00640 1 -- 172.21.15.66:0/4122746131 wait complete. 2024-03-30T00:43:15.642 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-03-30T00:43:14.893717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240200,"kb_used_data":7596,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754734520,"statfs":{"total":773094113280,"available":772848148480,"internally_reserved":0,"allocated":7778304,"data_stored":4106264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.158722"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.177106+0000","last_change":"2024-03-30T00:43:11.174174+0000","last_active":"2024-03-30T00:43:11.177106+0000","last_peered":"2024-03-30T00:43:11.177106+0000","last_clean":"2024-03-30T00:43:11.177106+0000","last_became_active":"2024-03-30T00:43:11.173862+0000","last_became_peered":"2024-03-30T00:43:11.173862+0000","last_unstale":"2024-03-30T00:43:11.177106+0000","last_undegraded":"2024-03-30T00:43:11.177106+0000","last_fullsized":"2024-03-30T00:43:11.177106+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:31.136461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.172319+0000","last_change":"2024-03-30T00:43:11.163085+0000","last_active":"2024-03-30T00:43:11.172319+0000","last_peered":"2024-03-30T00:43:11.172319+0000","last_clean":"2024-03-30T00:43:11.172319+0000","last_became_active":"2024-03-30T00:43:11.162810+0000","last_became_peered":"2024-03-30T00:43:11.162810+0000","last_unstale":"2024-03-30T00:43:11.172319+0000","last_undegraded":"2024-03-30T00:43:11.172319+0000","last_fullsized":"2024-03-30T00:43:11.172319+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:54.701307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155746+0000","last_change":"2024-03-30T00:43:11.174928+0000","last_active":"2024-03-30T00:43:12.155746+0000","last_peered":"2024-03-30T00:43:12.155746+0000","last_clean":"2024-03-30T00:43:12.155746+0000","last_became_active":"2024-03-30T00:43:11.174463+0000","last_became_peered":"2024-03-30T00:43:11.174463+0000","last_unstale":"2024-03-30T00:43:12.155746+0000","last_undegraded":"2024-03-30T00:43:12.155746+0000","last_fullsized":"2024-03-30T00:43:12.155746+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:49:10.790473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156558+0000","last_change":"2024-03-30T00:43:11.164044+0000","last_active":"2024-03-30T00:43:12.156558+0000","last_peered":"2024-03-30T00:43:12.156558+0000","last_clean":"2024-03-30T00:43:12.156558+0000","last_became_active":"2024-03-30T00:43:11.163769+0000","last_became_peered":"2024-03-30T00:43:11.163769+0000","last_unstale":"2024-03-30T00:43:12.156558+0000","last_undegraded":"2024-03-30T00:43:12.156558+0000","last_fullsized":"2024-03-30T00:43:12.156558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:12.337025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902067+0000","last_change":"2024-03-30T00:43:11.171880+0000","last_active":"2024-03-30T00:43:12.902067+0000","last_peered":"2024-03-30T00:43:12.902067+0000","last_clean":"2024-03-30T00:43:12.902067+0000","last_became_active":"2024-03-30T00:43:11.168977+0000","last_became_peered":"2024-03-30T00:43:11.168977+0000","last_unstale":"2024-03-30T00:43:12.902067+0000","last_undegraded":"2024-03-30T00:43:12.902067+0000","last_fullsized":"2024-03-30T00:43:12.902067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:19:28.439620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783770+0000","last_change":"2024-03-30T00:43:11.163581+0000","last_active":"2024-03-30T00:43:12.783770+0000","last_peered":"2024-03-30T00:43:12.783770+0000","last_clean":"2024-03-30T00:43:12.783770+0000","last_became_active":"2024-03-30T00:43:11.163332+0000","last_became_peered":"2024-03-30T00:43:11.163332+0000","last_unstale":"2024-03-30T00:43:12.783770+0000","last_undegraded":"2024-03-30T00:43:12.783770+0000","last_fullsized":"2024-03-30T00:43:12.783770+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:08.769384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155837+0000","last_change":"2024-03-30T00:43:11.162297+0000","last_active":"2024-03-30T00:43:12.155837+0000","last_peered":"2024-03-30T00:43:12.155837+0000","last_clean":"2024-03-30T00:43:12.155837+0000","last_became_active":"2024-03-30T00:43:11.161755+0000","last_became_peered":"2024-03-30T00:43:11.161755+0000","last_unstale":"2024-03-30T00:43:12.155837+0000","last_undegraded":"2024-03-30T00:43:12.155837+0000","last_fullsized":"2024-03-30T00:43:12.155837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:12:55.124790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.181281+0000","last_change":"2024-03-30T00:43:11.176478+0000","last_active":"2024-03-30T00:43:11.181281+0000","last_peered":"2024-03-30T00:43:11.181281+0000","last_clean":"2024-03-30T00:43:11.181281+0000","last_became_active":"2024-03-30T00:43:11.176039+0000","last_became_peered":"2024-03-30T00:43:11.176039+0000","last_unstale":"2024-03-30T00:43:11.181281+0000","last_undegraded":"2024-03-30T00:43:11.181281+0000","last_fullsized":"2024-03-30T00:43:11.181281+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:54.701307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155755+0000","last_change":"2024-03-30T00:43:11.185700+0000","last_active":"2024-03-30T00:43:12.155755+0000","last_peered":"2024-03-30T00:43:12.155755+0000","last_clean":"2024-03-30T00:43:12.155755+0000","last_became_active":"2024-03-30T00:43:11.185441+0000","last_became_peered":"2024-03-30T00:43:11.185441+0000","last_unstale":"2024-03-30T00:43:12.155755+0000","last_undegraded":"2024-03-30T00:43:12.155755+0000","last_fullsized":"2024-03-30T00:43:12.155755+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:12.015584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155873+0000","last_change":"2024-03-30T00:43:11.169320+0000","last_active":"2024-03-30T00:43:12.155873+0000","last_peered":"2024-03-30T00:43:12.155873+0000","last_clean":"2024-03-30T00:43:12.155873+0000","last_became_active":"2024-03-30T00:43:11.169029+0000","last_became_peered":"2024-03-30T00:43:11.169029+0000","last_unstale":"2024-03-30T00:43:12.155873+0000","last_undegraded":"2024-03-30T00:43:12.155873+0000","last_fullsized":"2024-03-30T00:43:12.155873+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:07:44.367842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156118+0000","last_change":"2024-03-30T00:43:11.165561+0000","last_active":"2024-03-30T00:43:12.156118+0000","last_peered":"2024-03-30T00:43:12.156118+0000","last_clean":"2024-03-30T00:43:12.156118+0000","last_became_active":"2024-03-30T00:43:11.165233+0000","last_became_peered":"2024-03-30T00:43:11.165233+0000","last_unstale":"2024-03-30T00:43:12.156118+0000","last_undegraded":"2024-03-30T00:43:12.156118+0000","last_fullsized":"2024-03-30T00:43:12.156118+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:49:10.790473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.181283+0000","last_change":"2024-03-30T00:43:11.176652+0000","last_active":"2024-03-30T00:43:11.181283+0000","last_peered":"2024-03-30T00:43:11.181283+0000","last_clean":"2024-03-30T00:43:11.181283+0000","last_became_active":"2024-03-30T00:43:11.175998+0000","last_became_peered":"2024-03-30T00:43:11.175998+0000","last_unstale":"2024-03-30T00:43:11.181283+0000","last_undegraded":"2024-03-30T00:43:11.181283+0000","last_fullsized":"2024-03-30T00:43:11.181283+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:10:52.232984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.186391+0000","last_change":"2024-03-30T00:43:11.185626+0000","last_active":"2024-03-30T00:43:11.186391+0000","last_peered":"2024-03-30T00:43:11.186391+0000","last_clean":"2024-03-30T00:43:11.186391+0000","last_became_active":"2024-03-30T00:43:11.185344+0000","last_became_peered":"2024-03-30T00:43:11.185344+0000","last_unstale":"2024-03-30T00:43:11.186391+0000","last_undegraded":"2024-03-30T00:43:11.186391+0000","last_fullsized":"2024-03-30T00:43:11.186391+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:46:13.402751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155683+0000","last_change":"2024-03-30T00:43:11.174882+0000","last_active":"2024-03-30T00:43:12.155683+0000","last_peered":"2024-03-30T00:43:12.155683+0000","last_clean":"2024-03-30T00:43:12.155683+0000","last_became_active":"2024-03-30T00:43:11.174572+0000","last_became_peered":"2024-03-30T00:43:11.174572+0000","last_unstale":"2024-03-30T00:43:12.155683+0000","last_undegraded":"2024-03-30T00:43:12.155683+0000","last_fullsized":"2024-03-30T00:43:12.155683+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:56:57.944571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155590+0000","last_change":"2024-03-30T00:43:11.160639+0000","last_active":"2024-03-30T00:43:12.155590+0000","last_peered":"2024-03-30T00:43:12.155590+0000","last_clean":"2024-03-30T00:43:12.155590+0000","last_became_active":"2024-03-30T00:43:11.159795+0000","last_became_peered":"2024-03-30T00:43:11.159795+0000","last_unstale":"2024-03-30T00:43:12.155590+0000","last_undegraded":"2024-03-30T00:43:12.155590+0000","last_fullsized":"2024-03-30T00:43:12.155590+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:31.136461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155692+0000","last_change":"2024-03-30T00:43:11.169526+0000","last_active":"2024-03-30T00:43:12.155692+0000","last_peered":"2024-03-30T00:43:12.155692+0000","last_clean":"2024-03-30T00:43:12.155692+0000","last_became_active":"2024-03-30T00:43:11.169278+0000","last_became_peered":"2024-03-30T00:43:11.169278+0000","last_unstale":"2024-03-30T00:43:12.155692+0000","last_undegraded":"2024-03-30T00:43:12.155692+0000","last_fullsized":"2024-03-30T00:43:12.155692+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:02:45.675408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.173315+0000","last_change":"2024-03-30T00:43:11.166428+0000","last_active":"2024-03-30T00:43:11.173315+0000","last_peered":"2024-03-30T00:43:11.173315+0000","last_clean":"2024-03-30T00:43:11.173315+0000","last_became_active":"2024-03-30T00:43:11.166191+0000","last_became_peered":"2024-03-30T00:43:11.166191+0000","last_unstale":"2024-03-30T00:43:11.173315+0000","last_undegraded":"2024-03-30T00:43:11.173315+0000","last_fullsized":"2024-03-30T00:43:11.173315+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:56:57.944571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156437+0000","last_change":"2024-03-30T00:43:11.165896+0000","last_active":"2024-03-30T00:43:12.156437+0000","last_peered":"2024-03-30T00:43:12.156437+0000","last_clean":"2024-03-30T00:43:12.156437+0000","last_became_active":"2024-03-30T00:43:11.165593+0000","last_became_peered":"2024-03-30T00:43:11.165593+0000","last_unstale":"2024-03-30T00:43:12.156437+0000","last_undegraded":"2024-03-30T00:43:12.156437+0000","last_fullsized":"2024-03-30T00:43:12.156437+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:07:44.367842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902411+0000","last_change":"2024-03-30T00:43:11.173869+0000","last_active":"2024-03-30T00:43:12.902411+0000","last_peered":"2024-03-30T00:43:12.902411+0000","last_clean":"2024-03-30T00:43:12.902411+0000","last_became_active":"2024-03-30T00:43:11.173545+0000","last_became_peered":"2024-03-30T00:43:11.173545+0000","last_unstale":"2024-03-30T00:43:12.902411+0000","last_undegraded":"2024-03-30T00:43:12.902411+0000","last_fullsized":"2024-03-30T00:43:12.902411+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:25.419335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.176946+0000","last_change":"2024-03-30T00:43:11.172346+0000","last_active":"2024-03-30T00:43:11.176946+0000","last_peered":"2024-03-30T00:43:11.176946+0000","last_clean":"2024-03-30T00:43:11.176946+0000","last_became_active":"2024-03-30T00:43:11.170651+0000","last_became_peered":"2024-03-30T00:43:11.170651+0000","last_unstale":"2024-03-30T00:43:11.176946+0000","last_undegraded":"2024-03-30T00:43:11.176946+0000","last_fullsized":"2024-03-30T00:43:11.176946+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:25:29.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.179067+0000","last_change":"2024-03-30T00:43:11.174359+0000","last_active":"2024-03-30T00:43:11.179067+0000","last_peered":"2024-03-30T00:43:11.179067+0000","last_clean":"2024-03-30T00:43:11.179067+0000","last_became_active":"2024-03-30T00:43:11.173824+0000","last_became_peered":"2024-03-30T00:43:11.173824+0000","last_unstale":"2024-03-30T00:43:11.179067+0000","last_undegraded":"2024-03-30T00:43:11.179067+0000","last_fullsized":"2024-03-30T00:43:11.179067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:49:10.790473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.172459+0000","last_change":"2024-03-30T00:43:11.163873+0000","last_active":"2024-03-30T00:43:11.172459+0000","last_peered":"2024-03-30T00:43:11.172459+0000","last_clean":"2024-03-30T00:43:11.172459+0000","last_became_active":"2024-03-30T00:43:11.163609+0000","last_became_peered":"2024-03-30T00:43:11.163609+0000","last_unstale":"2024-03-30T00:43:11.172459+0000","last_undegraded":"2024-03-30T00:43:11.172459+0000","last_fullsized":"2024-03-30T00:43:11.172459+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:46.420657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.171129+0000","last_change":"2024-03-30T00:43:11.164162+0000","last_active":"2024-03-30T00:43:11.171129+0000","last_peered":"2024-03-30T00:43:11.171129+0000","last_clean":"2024-03-30T00:43:11.171129+0000","last_became_active":"2024-03-30T00:43:11.163911+0000","last_became_peered":"2024-03-30T00:43:11.163911+0000","last_unstale":"2024-03-30T00:43:11.171129+0000","last_undegraded":"2024-03-30T00:43:11.171129+0000","last_fullsized":"2024-03-30T00:43:11.171129+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:08.769384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156069+0000","last_change":"2024-03-30T00:43:11.173235+0000","last_active":"2024-03-30T00:43:12.156069+0000","last_peered":"2024-03-30T00:43:12.156069+0000","last_clean":"2024-03-30T00:43:12.156069+0000","last_became_active":"2024-03-30T00:43:11.172991+0000","last_became_peered":"2024-03-30T00:43:11.172991+0000","last_unstale":"2024-03-30T00:43:12.156069+0000","last_undegraded":"2024-03-30T00:43:12.156069+0000","last_fullsized":"2024-03-30T00:43:12.156069+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:12.337025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.181303+0000","last_change":"2024-03-30T00:43:11.178069+0000","last_active":"2024-03-30T00:43:11.181303+0000","last_peered":"2024-03-30T00:43:11.181303+0000","last_clean":"2024-03-30T00:43:11.181303+0000","last_became_active":"2024-03-30T00:43:11.177741+0000","last_became_peered":"2024-03-30T00:43:11.177741+0000","last_unstale":"2024-03-30T00:43:11.181303+0000","last_undegraded":"2024-03-30T00:43:11.181303+0000","last_fullsized":"2024-03-30T00:43:11.181303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:12.015584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.181080+0000","last_change":"2024-03-30T00:43:11.176359+0000","last_active":"2024-03-30T00:43:11.181080+0000","last_peered":"2024-03-30T00:43:11.181080+0000","last_clean":"2024-03-30T00:43:11.181080+0000","last_became_active":"2024-03-30T00:43:11.175167+0000","last_became_peered":"2024-03-30T00:43:11.175167+0000","last_unstale":"2024-03-30T00:43:11.181080+0000","last_undegraded":"2024-03-30T00:43:11.181080+0000","last_fullsized":"2024-03-30T00:43:11.181080+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:34.053522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156242+0000","last_change":"2024-03-30T00:43:11.164962+0000","last_active":"2024-03-30T00:43:12.156242+0000","last_peered":"2024-03-30T00:43:12.156242+0000","last_clean":"2024-03-30T00:43:12.156242+0000","last_became_active":"2024-03-30T00:43:11.164132+0000","last_became_peered":"2024-03-30T00:43:11.164132+0000","last_unstale":"2024-03-30T00:43:12.156242+0000","last_undegraded":"2024-03-30T00:43:12.156242+0000","last_fullsized":"2024-03-30T00:43:12.156242+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:12.015584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.174996+0000","last_change":"2024-03-30T00:43:11.171783+0000","last_active":"2024-03-30T00:43:11.174996+0000","last_peered":"2024-03-30T00:43:11.174996+0000","last_clean":"2024-03-30T00:43:11.174996+0000","last_became_active":"2024-03-30T00:43:11.171553+0000","last_became_peered":"2024-03-30T00:43:11.171553+0000","last_unstale":"2024-03-30T00:43:11.174996+0000","last_undegraded":"2024-03-30T00:43:11.174996+0000","last_fullsized":"2024-03-30T00:43:11.174996+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:46.189391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155354+0000","last_change":"2024-03-30T00:43:11.183818+0000","last_active":"2024-03-30T00:43:12.155354+0000","last_peered":"2024-03-30T00:43:12.155354+0000","last_clean":"2024-03-30T00:43:12.155354+0000","last_became_active":"2024-03-30T00:43:11.183410+0000","last_became_peered":"2024-03-30T00:43:11.183410+0000","last_unstale":"2024-03-30T00:43:12.155354+0000","last_undegraded":"2024-03-30T00:43:12.155354+0000","last_fullsized":"2024-03-30T00:43:12.155354+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:46.189391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.180338+0000","last_change":"2024-03-30T00:43:11.176234+0000","last_active":"2024-03-30T00:43:11.180338+0000","last_peered":"2024-03-30T00:43:11.180338+0000","last_clean":"2024-03-30T00:43:11.180338+0000","last_became_active":"2024-03-30T00:43:11.175884+0000","last_became_peered":"2024-03-30T00:43:11.175884+0000","last_unstale":"2024-03-30T00:43:11.180338+0000","last_undegraded":"2024-03-30T00:43:11.180338+0000","last_fullsized":"2024-03-30T00:43:11.180338+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:07:44.367842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156199+0000","last_change":"2024-03-30T00:43:11.158808+0000","last_active":"2024-03-30T00:43:12.156199+0000","last_peered":"2024-03-30T00:43:12.156199+0000","last_clean":"2024-03-30T00:43:12.156199+0000","last_became_active":"2024-03-30T00:43:11.158339+0000","last_became_peered":"2024-03-30T00:43:11.158339+0000","last_unstale":"2024-03-30T00:43:12.156199+0000","last_undegraded":"2024-03-30T00:43:12.156199+0000","last_fullsized":"2024-03-30T00:43:12.156199+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:46.189391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783546+0000","last_change":"2024-03-30T00:43:11.169952+0000","last_active":"2024-03-30T00:43:12.783546+0000","last_peered":"2024-03-30T00:43:12.783546+0000","last_clean":"2024-03-30T00:43:12.783546+0000","last_became_active":"2024-03-30T00:43:11.169511+0000","last_became_peered":"2024-03-30T00:43:11.169511+0000","last_unstale":"2024-03-30T00:43:12.783546+0000","last_undegraded":"2024-03-30T00:43:12.783546+0000","last_fullsized":"2024-03-30T00:43:12.783546+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:02:50.870716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902283+0000","last_change":"2024-03-30T00:43:11.166814+0000","last_active":"2024-03-30T00:43:12.902283+0000","last_peered":"2024-03-30T00:43:12.902283+0000","last_clean":"2024-03-30T00:43:12.902283+0000","last_became_active":"2024-03-30T00:43:11.166408+0000","last_became_peered":"2024-03-30T00:43:11.166408+0000","last_unstale":"2024-03-30T00:43:12.902283+0000","last_undegraded":"2024-03-30T00:43:12.902283+0000","last_fullsized":"2024-03-30T00:43:12.902283+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:53.098797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"67'4","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.175146+0000","last_change":"2024-03-30T00:43:10.752801+0000","last_active":"2024-03-30T00:43:11.175146+0000","last_peered":"2024-03-30T00:43:11.175146+0000","last_clean":"2024-03-30T00:43:11.175146+0000","last_became_active":"2024-03-30T00:43:10.751722+0000","last_became_peered":"2024-03-30T00:43:10.751722+0000","last_unstale":"2024-03-30T00:43:11.175146+0000","last_undegraded":"2024-03-30T00:43:11.175146+0000","last_fullsized":"2024-03-30T00:43:11.175146+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:28.430262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156775+0000","last_change":"2024-03-30T00:43:11.182061+0000","last_active":"2024-03-30T00:43:12.156775+0000","last_peered":"2024-03-30T00:43:12.156775+0000","last_clean":"2024-03-30T00:43:12.156775+0000","last_became_active":"2024-03-30T00:43:11.181781+0000","last_became_peered":"2024-03-30T00:43:11.181781+0000","last_unstale":"2024-03-30T00:43:12.156775+0000","last_undegraded":"2024-03-30T00:43:12.156775+0000","last_fullsized":"2024-03-30T00:43:12.156775+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:41:03.894477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"67'1","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783938+0000","last_change":"2024-03-30T00:43:10.762465+0000","last_active":"2024-03-30T00:43:12.783938+0000","last_peered":"2024-03-30T00:43:12.783938+0000","last_clean":"2024-03-30T00:43:12.783938+0000","last_became_active":"2024-03-30T00:43:10.762238+0000","last_became_peered":"2024-03-30T00:43:10.762238+0000","last_unstale":"2024-03-30T00:43:12.783938+0000","last_undegraded":"2024-03-30T00:43:12.783938+0000","last_fullsized":"2024-03-30T00:43:12.783938+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:33.051174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783884+0000","last_change":"2024-03-30T00:43:11.169336+0000","last_active":"2024-03-30T00:43:12.783884+0000","last_peered":"2024-03-30T00:43:12.783884+0000","last_clean":"2024-03-30T00:43:12.783884+0000","last_became_active":"2024-03-30T00:43:11.169091+0000","last_became_peered":"2024-03-30T00:43:11.169091+0000","last_unstale":"2024-03-30T00:43:12.783884+0000","last_undegraded":"2024-03-30T00:43:12.783884+0000","last_fullsized":"2024-03-30T00:43:12.783884+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:46:41.582112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"67'2","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783846+0000","last_change":"2024-03-30T00:43:10.762845+0000","last_active":"2024-03-30T00:43:12.783846+0000","last_peered":"2024-03-30T00:43:12.783846+0000","last_clean":"2024-03-30T00:43:12.783846+0000","last_became_active":"2024-03-30T00:43:10.762548+0000","last_became_peered":"2024-03-30T00:43:10.762548+0000","last_unstale":"2024-03-30T00:43:12.783846+0000","last_undegraded":"2024-03-30T00:43:12.783846+0000","last_fullsized":"2024-03-30T00:43:12.783846+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:38:16.808375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155710+0000","last_change":"2024-03-30T00:43:11.171048+0000","last_active":"2024-03-30T00:43:12.155710+0000","last_peered":"2024-03-30T00:43:12.155710+0000","last_clean":"2024-03-30T00:43:12.155710+0000","last_became_active":"2024-03-30T00:43:11.169752+0000","last_became_peered":"2024-03-30T00:43:11.169752+0000","last_unstale":"2024-03-30T00:43:12.155710+0000","last_undegraded":"2024-03-30T00:43:12.155710+0000","last_fullsized":"2024-03-30T00:43:12.155710+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:54.701307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"67'2","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783731+0000","last_change":"2024-03-30T00:43:10.756202+0000","last_active":"2024-03-30T00:43:12.783731+0000","last_peered":"2024-03-30T00:43:12.783731+0000","last_clean":"2024-03-30T00:43:12.783731+0000","last_became_active":"2024-03-30T00:43:10.755940+0000","last_became_peered":"2024-03-30T00:43:10.755940+0000","last_unstale":"2024-03-30T00:43:12.783731+0000","last_undegraded":"2024-03-30T00:43:12.783731+0000","last_fullsized":"2024-03-30T00:43:12.783731+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:02.108234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155688+0000","last_change":"2024-03-30T00:43:11.156922+0000","last_active":"2024-03-30T00:43:12.155688+0000","last_peered":"2024-03-30T00:43:12.155688+0000","last_clean":"2024-03-30T00:43:12.155688+0000","last_became_active":"2024-03-30T00:43:11.156635+0000","last_became_peered":"2024-03-30T00:43:11.156635+0000","last_unstale":"2024-03-30T00:43:12.155688+0000","last_undegraded":"2024-03-30T00:43:12.155688+0000","last_fullsized":"2024-03-30T00:43:12.155688+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:50:53.954307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.165079+0000","last_change":"2024-03-30T00:43:10.752090+0000","last_active":"2024-03-30T00:43:11.165079+0000","last_peered":"2024-03-30T00:43:11.165079+0000","last_clean":"2024-03-30T00:43:11.165079+0000","last_became_active":"2024-03-30T00:43:10.751830+0000","last_became_peered":"2024-03-30T00:43:10.751830+0000","last_unstale":"2024-03-30T00:43:11.165079+0000","last_undegraded":"2024-03-30T00:43:11.165079+0000","last_fullsized":"2024-03-30T00:43:11.165079+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:26:27.085243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.172920+0000","last_change":"2024-03-30T00:43:11.165524+0000","last_active":"2024-03-30T00:43:11.172920+0000","last_peered":"2024-03-30T00:43:11.172920+0000","last_clean":"2024-03-30T00:43:11.172920+0000","last_became_active":"2024-03-30T00:43:11.165047+0000","last_became_peered":"2024-03-30T00:43:11.165047+0000","last_unstale":"2024-03-30T00:43:11.172920+0000","last_undegraded":"2024-03-30T00:43:11.172920+0000","last_fullsized":"2024-03-30T00:43:11.172920+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:49:10.790473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.168794+0000","last_change":"2024-03-30T00:43:10.758940+0000","last_active":"2024-03-30T00:43:11.168794+0000","last_peered":"2024-03-30T00:43:11.168794+0000","last_clean":"2024-03-30T00:43:11.168794+0000","last_became_active":"2024-03-30T00:43:10.758632+0000","last_became_peered":"2024-03-30T00:43:10.758632+0000","last_unstale":"2024-03-30T00:43:11.168794+0000","last_undegraded":"2024-03-30T00:43:11.168794+0000","last_fullsized":"2024-03-30T00:43:11.168794+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:38:16.808375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902015+0000","last_change":"2024-03-30T00:43:11.170525+0000","last_active":"2024-03-30T00:43:12.902015+0000","last_peered":"2024-03-30T00:43:12.902015+0000","last_clean":"2024-03-30T00:43:12.902015+0000","last_became_active":"2024-03-30T00:43:11.170246+0000","last_became_peered":"2024-03-30T00:43:11.170246+0000","last_unstale":"2024-03-30T00:43:12.902015+0000","last_undegraded":"2024-03-30T00:43:12.902015+0000","last_fullsized":"2024-03-30T00:43:12.902015+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:57:49.883141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.150004+0000","last_change":"2024-03-30T00:43:10.765925+0000","last_active":"2024-03-30T00:43:11.150004+0000","last_peered":"2024-03-30T00:43:11.150004+0000","last_clean":"2024-03-30T00:43:11.150004+0000","last_became_active":"2024-03-30T00:43:10.765642+0000","last_became_peered":"2024-03-30T00:43:10.765642+0000","last_unstale":"2024-03-30T00:43:11.150004+0000","last_undegraded":"2024-03-30T00:43:11.150004+0000","last_fullsized":"2024-03-30T00:43:11.150004+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:33.051174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.161966+0000","last_change":"2024-03-30T00:43:11.158686+0000","last_active":"2024-03-30T00:43:11.161966+0000","last_peered":"2024-03-30T00:43:11.161966+0000","last_clean":"2024-03-30T00:43:11.161966+0000","last_became_active":"2024-03-30T00:43:11.158427+0000","last_became_peered":"2024-03-30T00:43:11.158427+0000","last_unstale":"2024-03-30T00:43:11.161966+0000","last_undegraded":"2024-03-30T00:43:11.161966+0000","last_fullsized":"2024-03-30T00:43:11.161966+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:26:05.722702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902223+0000","last_change":"2024-03-30T00:43:10.755378+0000","last_active":"2024-03-30T00:43:12.902223+0000","last_peered":"2024-03-30T00:43:12.902223+0000","last_clean":"2024-03-30T00:43:12.902223+0000","last_became_active":"2024-03-30T00:43:10.754308+0000","last_became_peered":"2024-03-30T00:43:10.754308+0000","last_unstale":"2024-03-30T00:43:12.902223+0000","last_undegraded":"2024-03-30T00:43:12.902223+0000","last_fullsized":"2024-03-30T00:43:12.902223+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:17:07.832744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.159612+0000","last_change":"2024-03-30T00:43:11.157364+0000","last_active":"2024-03-30T00:43:11.159612+0000","last_peered":"2024-03-30T00:43:11.159612+0000","last_clean":"2024-03-30T00:43:11.159612+0000","last_became_active":"2024-03-30T00:43:11.156978+0000","last_became_peered":"2024-03-30T00:43:11.156978+0000","last_unstale":"2024-03-30T00:43:11.159612+0000","last_undegraded":"2024-03-30T00:43:11.159612+0000","last_fullsized":"2024-03-30T00:43:11.159612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:07:44.367842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155861+0000","last_change":"2024-03-30T00:43:10.757052+0000","last_active":"2024-03-30T00:43:12.155861+0000","last_peered":"2024-03-30T00:43:12.155861+0000","last_clean":"2024-03-30T00:43:12.155861+0000","last_became_active":"2024-03-30T00:43:10.756696+0000","last_became_peered":"2024-03-30T00:43:10.756696+0000","last_unstale":"2024-03-30T00:43:12.155861+0000","last_undegraded":"2024-03-30T00:43:12.155861+0000","last_fullsized":"2024-03-30T00:43:12.155861+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:57:53.489388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.172343+0000","last_change":"2024-03-30T00:43:11.168077+0000","last_active":"2024-03-30T00:43:11.172343+0000","last_peered":"2024-03-30T00:43:11.172343+0000","last_clean":"2024-03-30T00:43:11.172343+0000","last_became_active":"2024-03-30T00:43:11.167797+0000","last_became_peered":"2024-03-30T00:43:11.167797+0000","last_unstale":"2024-03-30T00:43:11.172343+0000","last_undegraded":"2024-03-30T00:43:11.172343+0000","last_fullsized":"2024-03-30T00:43:11.172343+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:46:41.582112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156549+0000","last_change":"2024-03-30T00:43:10.750107+0000","last_active":"2024-03-30T00:43:12.156549+0000","last_peered":"2024-03-30T00:43:12.156549+0000","last_clean":"2024-03-30T00:43:12.156549+0000","last_became_active":"2024-03-30T00:43:10.749738+0000","last_became_peered":"2024-03-30T00:43:10.749738+0000","last_unstale":"2024-03-30T00:43:12.156549+0000","last_undegraded":"2024-03-30T00:43:12.156549+0000","last_fullsized":"2024-03-30T00:43:12.156549+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:23.758029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902443+0000","last_change":"2024-03-30T00:43:11.178139+0000","last_active":"2024-03-30T00:43:12.902443+0000","last_peered":"2024-03-30T00:43:12.902443+0000","last_clean":"2024-03-30T00:43:12.902443+0000","last_became_active":"2024-03-30T00:43:11.177884+0000","last_became_peered":"2024-03-30T00:43:11.177884+0000","last_unstale":"2024-03-30T00:43:12.902443+0000","last_undegraded":"2024-03-30T00:43:12.902443+0000","last_fullsized":"2024-03-30T00:43:12.902443+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:22.378638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"67'3","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155824+0000","last_change":"2024-03-30T00:43:10.754587+0000","last_active":"2024-03-30T00:43:12.155824+0000","last_peered":"2024-03-30T00:43:12.155824+0000","last_clean":"2024-03-30T00:43:12.155824+0000","last_became_active":"2024-03-30T00:43:10.754294+0000","last_became_peered":"2024-03-30T00:43:10.754294+0000","last_unstale":"2024-03-30T00:43:12.155824+0000","last_undegraded":"2024-03-30T00:43:12.155824+0000","last_fullsized":"2024-03-30T00:43:12.155824+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:23.758029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156273+0000","last_change":"2024-03-30T00:43:11.162508+0000","last_active":"2024-03-30T00:43:12.156273+0000","last_peered":"2024-03-30T00:43:12.156273+0000","last_clean":"2024-03-30T00:43:12.156273+0000","last_became_active":"2024-03-30T00:43:11.162238+0000","last_became_peered":"2024-03-30T00:43:11.162238+0000","last_unstale":"2024-03-30T00:43:12.156273+0000","last_undegraded":"2024-03-30T00:43:12.156273+0000","last_fullsized":"2024-03-30T00:43:12.156273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:51.250839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"67'6","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.187674+0000","last_change":"2024-03-30T00:43:10.750590+0000","last_active":"2024-03-30T00:43:11.187674+0000","last_peered":"2024-03-30T00:43:11.187674+0000","last_clean":"2024-03-30T00:43:11.187674+0000","last_became_active":"2024-03-30T00:43:10.749733+0000","last_became_peered":"2024-03-30T00:43:10.749733+0000","last_unstale":"2024-03-30T00:43:11.187674+0000","last_undegraded":"2024-03-30T00:43:11.187674+0000","last_fullsized":"2024-03-30T00:43:11.187674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:23.758029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.154966+0000","last_change":"2024-03-30T00:43:11.165002+0000","last_active":"2024-03-30T00:43:12.154966+0000","last_peered":"2024-03-30T00:43:12.154966+0000","last_clean":"2024-03-30T00:43:12.154966+0000","last_became_active":"2024-03-30T00:43:11.164463+0000","last_became_peered":"2024-03-30T00:43:11.164463+0000","last_unstale":"2024-03-30T00:43:12.154966+0000","last_undegraded":"2024-03-30T00:43:12.154966+0000","last_fullsized":"2024-03-30T00:43:12.154966+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:31.136461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"67'2","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155407+0000","last_change":"2024-03-30T00:43:10.764960+0000","last_active":"2024-03-30T00:43:12.155407+0000","last_peered":"2024-03-30T00:43:12.155407+0000","last_clean":"2024-03-30T00:43:12.155407+0000","last_became_active":"2024-03-30T00:43:10.764461+0000","last_became_peered":"2024-03-30T00:43:10.764461+0000","last_unstale":"2024-03-30T00:43:12.155407+0000","last_undegraded":"2024-03-30T00:43:12.155407+0000","last_fullsized":"2024-03-30T00:43:12.155407+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:28.430262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.180002+0000","last_change":"2024-03-30T00:43:11.175247+0000","last_active":"2024-03-30T00:43:11.180002+0000","last_peered":"2024-03-30T00:43:11.180002+0000","last_clean":"2024-03-30T00:43:11.180002+0000","last_became_active":"2024-03-30T00:43:11.174854+0000","last_became_peered":"2024-03-30T00:43:11.174854+0000","last_unstale":"2024-03-30T00:43:11.180002+0000","last_undegraded":"2024-03-30T00:43:11.180002+0000","last_fullsized":"2024-03-30T00:43:11.180002+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:41:03.894477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.158819+0000","last_change":"2024-03-30T00:43:10.760606+0000","last_active":"2024-03-30T00:43:11.158819+0000","last_peered":"2024-03-30T00:43:11.158819+0000","last_clean":"2024-03-30T00:43:11.158819+0000","last_became_active":"2024-03-30T00:43:10.759487+0000","last_became_peered":"2024-03-30T00:43:10.759487+0000","last_unstale":"2024-03-30T00:43:11.158819+0000","last_undegraded":"2024-03-30T00:43:11.158819+0000","last_fullsized":"2024-03-30T00:43:11.158819+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:21:59.823030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902083+0000","last_change":"2024-03-30T00:43:11.172357+0000","last_active":"2024-03-30T00:43:12.902083+0000","last_peered":"2024-03-30T00:43:12.902083+0000","last_clean":"2024-03-30T00:43:12.902083+0000","last_became_active":"2024-03-30T00:43:11.172092+0000","last_became_peered":"2024-03-30T00:43:11.172092+0000","last_unstale":"2024-03-30T00:43:12.902083+0000","last_undegraded":"2024-03-30T00:43:12.902083+0000","last_fullsized":"2024-03-30T00:43:12.902083+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:12:08.266211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155852+0000","last_change":"2024-03-30T00:43:10.755458+0000","last_active":"2024-03-30T00:43:12.155852+0000","last_peered":"2024-03-30T00:43:12.155852+0000","last_clean":"2024-03-30T00:43:12.155852+0000","last_became_active":"2024-03-30T00:43:10.755186+0000","last_became_peered":"2024-03-30T00:43:10.755186+0000","last_unstale":"2024-03-30T00:43:12.155852+0000","last_undegraded":"2024-03-30T00:43:12.155852+0000","last_fullsized":"2024-03-30T00:43:12.155852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:29:21.858586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.159667+0000","last_change":"2024-03-30T00:43:11.157247+0000","last_active":"2024-03-30T00:43:11.159667+0000","last_peered":"2024-03-30T00:43:11.159667+0000","last_clean":"2024-03-30T00:43:11.159667+0000","last_became_active":"2024-03-30T00:43:11.156989+0000","last_became_peered":"2024-03-30T00:43:11.156989+0000","last_unstale":"2024-03-30T00:43:11.159667+0000","last_undegraded":"2024-03-30T00:43:11.159667+0000","last_fullsized":"2024-03-30T00:43:11.159667+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:12.337025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"67'8","reported_seq":19,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155820+0000","last_change":"2024-03-30T00:43:10.763891+0000","last_active":"2024-03-30T00:43:12.155820+0000","last_peered":"2024-03-30T00:43:12.155820+0000","last_clean":"2024-03-30T00:43:12.155820+0000","last_became_active":"2024-03-30T00:43:10.760736+0000","last_became_peered":"2024-03-30T00:43:10.760736+0000","last_unstale":"2024-03-30T00:43:12.155820+0000","last_undegraded":"2024-03-30T00:43:12.155820+0000","last_fullsized":"2024-03-30T00:43:12.155820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:28.430262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156480+0000","last_change":"2024-03-30T00:43:11.185385+0000","last_active":"2024-03-30T00:43:12.156480+0000","last_peered":"2024-03-30T00:43:12.156480+0000","last_clean":"2024-03-30T00:43:12.156480+0000","last_became_active":"2024-03-30T00:43:11.185144+0000","last_became_peered":"2024-03-30T00:43:11.185144+0000","last_unstale":"2024-03-30T00:43:12.156480+0000","last_undegraded":"2024-03-30T00:43:12.156480+0000","last_fullsized":"2024-03-30T00:43:12.156480+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:10:52.232984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.178218+0000","last_change":"2024-03-30T00:43:10.763317+0000","last_active":"2024-03-30T00:43:11.178218+0000","last_peered":"2024-03-30T00:43:11.178218+0000","last_clean":"2024-03-30T00:43:11.178218+0000","last_became_active":"2024-03-30T00:43:10.763065+0000","last_became_peered":"2024-03-30T00:43:10.763065+0000","last_unstale":"2024-03-30T00:43:11.178218+0000","last_undegraded":"2024-03-30T00:43:11.178218+0000","last_fullsized":"2024-03-30T00:43:11.178218+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:10:43.217235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902399+0000","last_change":"2024-03-30T00:43:11.181397+0000","last_active":"2024-03-30T00:43:12.902399+0000","last_peered":"2024-03-30T00:43:12.902399+0000","last_clean":"2024-03-30T00:43:12.902399+0000","last_became_active":"2024-03-30T00:43:11.181130+0000","last_became_peered":"2024-03-30T00:43:11.181130+0000","last_unstale":"2024-03-30T00:43:12.902399+0000","last_undegraded":"2024-03-30T00:43:12.902399+0000","last_fullsized":"2024-03-30T00:43:12.902399+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:57:25.787445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"67'2","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155809+0000","last_change":"2024-03-30T00:43:10.764643+0000","last_active":"2024-03-30T00:43:12.155809+0000","last_peered":"2024-03-30T00:43:12.155809+0000","last_clean":"2024-03-30T00:43:12.155809+0000","last_became_active":"2024-03-30T00:43:10.763645+0000","last_became_peered":"2024-03-30T00:43:10.763645+0000","last_unstale":"2024-03-30T00:43:12.155809+0000","last_undegraded":"2024-03-30T00:43:12.155809+0000","last_fullsized":"2024-03-30T00:43:12.155809+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:05:32.100565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156565+0000","last_change":"2024-03-30T00:43:11.174140+0000","last_active":"2024-03-30T00:43:12.156565+0000","last_peered":"2024-03-30T00:43:12.156565+0000","last_clean":"2024-03-30T00:43:12.156565+0000","last_became_active":"2024-03-30T00:43:11.173894+0000","last_became_peered":"2024-03-30T00:43:11.173894+0000","last_unstale":"2024-03-30T00:43:12.156565+0000","last_undegraded":"2024-03-30T00:43:12.156565+0000","last_fullsized":"2024-03-30T00:43:12.156565+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:10:52.232984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"67'1","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156024+0000","last_change":"2024-03-30T00:43:10.758412+0000","last_active":"2024-03-30T00:43:12.156024+0000","last_peered":"2024-03-30T00:43:12.156024+0000","last_clean":"2024-03-30T00:43:12.156024+0000","last_became_active":"2024-03-30T00:43:10.758104+0000","last_became_peered":"2024-03-30T00:43:10.758104+0000","last_unstale":"2024-03-30T00:43:12.156024+0000","last_undegraded":"2024-03-30T00:43:12.156024+0000","last_fullsized":"2024-03-30T00:43:12.156024+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:24.416987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902436+0000","last_change":"2024-03-30T00:43:11.179456+0000","last_active":"2024-03-30T00:43:12.902436+0000","last_peered":"2024-03-30T00:43:12.902436+0000","last_clean":"2024-03-30T00:43:12.902436+0000","last_became_active":"2024-03-30T00:43:11.179197+0000","last_became_peered":"2024-03-30T00:43:11.179197+0000","last_unstale":"2024-03-30T00:43:12.902436+0000","last_undegraded":"2024-03-30T00:43:12.902436+0000","last_fullsized":"2024-03-30T00:43:12.902436+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:25:40.893560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.150457+0000","last_change":"2024-03-30T00:43:10.755679+0000","last_active":"2024-03-30T00:43:11.150457+0000","last_peered":"2024-03-30T00:43:11.150457+0000","last_clean":"2024-03-30T00:43:11.150457+0000","last_became_active":"2024-03-30T00:43:10.755401+0000","last_became_peered":"2024-03-30T00:43:10.755401+0000","last_unstale":"2024-03-30T00:43:11.150457+0000","last_undegraded":"2024-03-30T00:43:11.150457+0000","last_fullsized":"2024-03-30T00:43:11.150457+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:41:02.892129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783621+0000","last_change":"2024-03-30T00:43:11.169750+0000","last_active":"2024-03-30T00:43:12.783621+0000","last_peered":"2024-03-30T00:43:12.783621+0000","last_clean":"2024-03-30T00:43:12.783621+0000","last_became_active":"2024-03-30T00:43:11.169185+0000","last_became_peered":"2024-03-30T00:43:11.169185+0000","last_unstale":"2024-03-30T00:43:12.783621+0000","last_undegraded":"2024-03-30T00:43:12.783621+0000","last_fullsized":"2024-03-30T00:43:12.783621+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:25:29.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"67'2","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783768+0000","last_change":"2024-03-30T00:43:10.765327+0000","last_active":"2024-03-30T00:43:12.783768+0000","last_peered":"2024-03-30T00:43:12.783768+0000","last_clean":"2024-03-30T00:43:12.783768+0000","last_became_active":"2024-03-30T00:43:10.765054+0000","last_became_peered":"2024-03-30T00:43:10.765054+0000","last_unstale":"2024-03-30T00:43:12.783768+0000","last_undegraded":"2024-03-30T00:43:12.783768+0000","last_fullsized":"2024-03-30T00:43:12.783768+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:46:12.400403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.179261+0000","last_change":"2024-03-30T00:43:11.174993+0000","last_active":"2024-03-30T00:43:11.179261+0000","last_peered":"2024-03-30T00:43:11.179261+0000","last_clean":"2024-03-30T00:43:11.179261+0000","last_became_active":"2024-03-30T00:43:11.174629+0000","last_became_peered":"2024-03-30T00:43:11.174629+0000","last_unstale":"2024-03-30T00:43:11.179261+0000","last_undegraded":"2024-03-30T00:43:11.179261+0000","last_fullsized":"2024-03-30T00:43:11.179261+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:46.189391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.158921+0000","last_change":"2024-03-30T00:43:10.758820+0000","last_active":"2024-03-30T00:43:11.158921+0000","last_peered":"2024-03-30T00:43:11.158921+0000","last_clean":"2024-03-30T00:43:11.158921+0000","last_became_active":"2024-03-30T00:43:10.757306+0000","last_became_peered":"2024-03-30T00:43:10.757306+0000","last_unstale":"2024-03-30T00:43:11.158921+0000","last_undegraded":"2024-03-30T00:43:11.158921+0000","last_fullsized":"2024-03-30T00:43:11.158921+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:17:28.430262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902182+0000","last_change":"2024-03-30T00:43:11.170005+0000","last_active":"2024-03-30T00:43:12.902182+0000","last_peered":"2024-03-30T00:43:12.902182+0000","last_clean":"2024-03-30T00:43:12.902182+0000","last_became_active":"2024-03-30T00:43:11.169646+0000","last_became_peered":"2024-03-30T00:43:11.169646+0000","last_unstale":"2024-03-30T00:43:12.902182+0000","last_undegraded":"2024-03-30T00:43:12.902182+0000","last_fullsized":"2024-03-30T00:43:12.902182+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:04:38.382257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783552+0000","last_change":"2024-03-30T00:43:10.753839+0000","last_active":"2024-03-30T00:43:12.783552+0000","last_peered":"2024-03-30T00:43:12.783552+0000","last_clean":"2024-03-30T00:43:12.783552+0000","last_became_active":"2024-03-30T00:43:10.753578+0000","last_became_peered":"2024-03-30T00:43:10.753578+0000","last_unstale":"2024-03-30T00:43:12.783552+0000","last_undegraded":"2024-03-30T00:43:12.783552+0000","last_fullsized":"2024-03-30T00:43:12.783552+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:41:02.892129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783692+0000","last_change":"2024-03-30T00:43:11.164616+0000","last_active":"2024-03-30T00:43:12.783692+0000","last_peered":"2024-03-30T00:43:12.783692+0000","last_clean":"2024-03-30T00:43:12.783692+0000","last_became_active":"2024-03-30T00:43:11.164309+0000","last_became_peered":"2024-03-30T00:43:11.164309+0000","last_unstale":"2024-03-30T00:43:12.783692+0000","last_undegraded":"2024-03-30T00:43:12.783692+0000","last_fullsized":"2024-03-30T00:43:12.783692+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:46.420657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.178768+0000","last_change":"2024-03-30T00:43:10.758910+0000","last_active":"2024-03-30T00:43:11.178768+0000","last_peered":"2024-03-30T00:43:11.178768+0000","last_clean":"2024-03-30T00:43:11.178768+0000","last_became_active":"2024-03-30T00:43:10.758600+0000","last_became_peered":"2024-03-30T00:43:10.758600+0000","last_unstale":"2024-03-30T00:43:11.178768+0000","last_undegraded":"2024-03-30T00:43:11.178768+0000","last_fullsized":"2024-03-30T00:43:11.178768+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:29:21.858586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156037+0000","last_change":"2024-03-30T00:43:11.166129+0000","last_active":"2024-03-30T00:43:12.156037+0000","last_peered":"2024-03-30T00:43:12.156037+0000","last_clean":"2024-03-30T00:43:12.156037+0000","last_became_active":"2024-03-30T00:43:11.165827+0000","last_became_peered":"2024-03-30T00:43:11.165827+0000","last_unstale":"2024-03-30T00:43:12.156037+0000","last_undegraded":"2024-03-30T00:43:12.156037+0000","last_fullsized":"2024-03-30T00:43:12.156037+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:17:08.835092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.161855+0000","last_change":"2024-03-30T00:43:10.762929+0000","last_active":"2024-03-30T00:43:11.161855+0000","last_peered":"2024-03-30T00:43:11.161855+0000","last_clean":"2024-03-30T00:43:11.161855+0000","last_became_active":"2024-03-30T00:43:10.762592+0000","last_became_peered":"2024-03-30T00:43:10.762592+0000","last_unstale":"2024-03-30T00:43:11.161855+0000","last_undegraded":"2024-03-30T00:43:11.161855+0000","last_fullsized":"2024-03-30T00:43:11.161855+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:35:16.490841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.180079+0000","last_change":"2024-03-30T00:43:11.175500+0000","last_active":"2024-03-30T00:43:11.180079+0000","last_peered":"2024-03-30T00:43:11.180079+0000","last_clean":"2024-03-30T00:43:11.180079+0000","last_became_active":"2024-03-30T00:43:11.174800+0000","last_became_peered":"2024-03-30T00:43:11.174800+0000","last_unstale":"2024-03-30T00:43:11.180079+0000","last_undegraded":"2024-03-30T00:43:11.180079+0000","last_fullsized":"2024-03-30T00:43:11.180079+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:29:23.961976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.783931+0000","last_change":"2024-03-30T00:43:10.761241+0000","last_active":"2024-03-30T00:43:12.783931+0000","last_peered":"2024-03-30T00:43:12.783931+0000","last_clean":"2024-03-30T00:43:12.783931+0000","last_became_active":"2024-03-30T00:43:10.760880+0000","last_became_peered":"2024-03-30T00:43:10.760880+0000","last_unstale":"2024-03-30T00:43:12.783931+0000","last_undegraded":"2024-03-30T00:43:12.783931+0000","last_fullsized":"2024-03-30T00:43:12.783931+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:26:27.085243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902429+0000","last_change":"2024-03-30T00:43:11.170993+0000","last_active":"2024-03-30T00:43:12.902429+0000","last_peered":"2024-03-30T00:43:12.902429+0000","last_clean":"2024-03-30T00:43:12.902429+0000","last_became_active":"2024-03-30T00:43:11.170693+0000","last_became_peered":"2024-03-30T00:43:11.170693+0000","last_unstale":"2024-03-30T00:43:12.902429+0000","last_undegraded":"2024-03-30T00:43:12.902429+0000","last_fullsized":"2024-03-30T00:43:12.902429+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:31.904282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.146864+0000","last_change":"2024-03-30T00:43:10.750485+0000","last_active":"2024-03-30T00:43:11.146864+0000","last_peered":"2024-03-30T00:43:11.146864+0000","last_clean":"2024-03-30T00:43:11.146864+0000","last_became_active":"2024-03-30T00:43:10.749356+0000","last_became_peered":"2024-03-30T00:43:10.749356+0000","last_unstale":"2024-03-30T00:43:11.146864+0000","last_undegraded":"2024-03-30T00:43:11.146864+0000","last_fullsized":"2024-03-30T00:43:11.146864+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:05:32.100565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902001+0000","last_change":"2024-03-30T00:43:11.162930+0000","last_active":"2024-03-30T00:43:12.902001+0000","last_peered":"2024-03-30T00:43:12.902001+0000","last_clean":"2024-03-30T00:43:12.902001+0000","last_became_active":"2024-03-30T00:43:11.162665+0000","last_became_peered":"2024-03-30T00:43:11.162665+0000","last_unstale":"2024-03-30T00:43:12.902001+0000","last_undegraded":"2024-03-30T00:43:12.902001+0000","last_fullsized":"2024-03-30T00:43:12.902001+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:53:40.511494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155531+0000","last_change":"2024-03-30T00:43:10.757217+0000","last_active":"2024-03-30T00:43:12.155531+0000","last_peered":"2024-03-30T00:43:12.155531+0000","last_clean":"2024-03-30T00:43:12.155531+0000","last_became_active":"2024-03-30T00:43:10.756873+0000","last_became_peered":"2024-03-30T00:43:10.756873+0000","last_unstale":"2024-03-30T00:43:12.155531+0000","last_undegraded":"2024-03-30T00:43:12.155531+0000","last_fullsized":"2024-03-30T00:43:12.155531+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:35:16.490841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.156519+0000","last_change":"2024-03-30T00:43:11.179302+0000","last_active":"2024-03-30T00:43:12.156519+0000","last_peered":"2024-03-30T00:43:12.156519+0000","last_clean":"2024-03-30T00:43:12.156519+0000","last_became_active":"2024-03-30T00:43:11.177661+0000","last_became_peered":"2024-03-30T00:43:11.177661+0000","last_unstale":"2024-03-30T00:43:12.156519+0000","last_undegraded":"2024-03-30T00:43:12.156519+0000","last_fullsized":"2024-03-30T00:43:12.156519+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:56:57.944571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.149847+0000","last_change":"2024-03-30T00:43:10.752788+0000","last_active":"2024-03-30T00:43:11.149847+0000","last_peered":"2024-03-30T00:43:11.149847+0000","last_clean":"2024-03-30T00:43:11.149847+0000","last_became_active":"2024-03-30T00:43:10.752487+0000","last_became_peered":"2024-03-30T00:43:10.752487+0000","last_unstale":"2024-03-30T00:43:11.149847+0000","last_undegraded":"2024-03-30T00:43:11.149847+0000","last_fullsized":"2024-03-30T00:43:11.149847+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:29:22.959628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.158024+0000","last_change":"2024-03-30T00:43:11.155067+0000","last_active":"2024-03-30T00:43:11.158024+0000","last_peered":"2024-03-30T00:43:11.158024+0000","last_clean":"2024-03-30T00:43:11.158024+0000","last_became_active":"2024-03-30T00:43:11.154706+0000","last_became_peered":"2024-03-30T00:43:11.154706+0000","last_unstale":"2024-03-30T00:43:11.158024+0000","last_undegraded":"2024-03-30T00:43:11.158024+0000","last_fullsized":"2024-03-30T00:43:11.158024+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:51.250839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.150101+0000","last_change":"2024-03-30T00:43:10.762271+0000","last_active":"2024-03-30T00:43:11.150101+0000","last_peered":"2024-03-30T00:43:11.150101+0000","last_clean":"2024-03-30T00:43:11.150101+0000","last_became_active":"2024-03-30T00:43:10.761991+0000","last_became_peered":"2024-03-30T00:43:10.761991+0000","last_unstale":"2024-03-30T00:43:11.150101+0000","last_undegraded":"2024-03-30T00:43:11.150101+0000","last_fullsized":"2024-03-30T00:43:11.150101+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:46:12.400403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.155906+0000","last_change":"2024-03-30T00:43:11.168893+0000","last_active":"2024-03-30T00:43:12.155906+0000","last_peered":"2024-03-30T00:43:12.155906+0000","last_clean":"2024-03-30T00:43:12.155906+0000","last_became_active":"2024-03-30T00:43:11.167502+0000","last_became_peered":"2024-03-30T00:43:11.167502+0000","last_unstale":"2024-03-30T00:43:12.155906+0000","last_undegraded":"2024-03-30T00:43:12.155906+0000","last_fullsized":"2024-03-30T00:43:12.155906+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:15:10.075537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.145553+0000","last_change":"2024-03-30T00:43:10.753455+0000","last_active":"2024-03-30T00:43:11.145553+0000","last_peered":"2024-03-30T00:43:11.145553+0000","last_clean":"2024-03-30T00:43:11.145553+0000","last_became_active":"2024-03-30T00:43:10.753160+0000","last_became_peered":"2024-03-30T00:43:10.753160+0000","last_unstale":"2024-03-30T00:43:11.145553+0000","last_undegraded":"2024-03-30T00:43:11.145553+0000","last_fullsized":"2024-03-30T00:43:11.145553+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:35:16.490841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-30T00:43:12.902108+0000","last_change":"2024-03-30T00:43:11.169291+0000","last_active":"2024-03-30T00:43:12.902108+0000","last_peered":"2024-03-30T00:43:12.902108+0000","last_clean":"2024-03-30T00:43:12.902108+0000","last_became_active":"2024-03-30T00:43:11.168708+0000","last_became_peered":"2024-03-30T00:43:11.168708+0000","last_unstale":"2024-03-30T00:43:12.902108+0000","last_undegraded":"2024-03-30T00:43:12.902108+0000","last_fullsized":"2024-03-30T00:43:12.902108+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:10.734515+0000","last_clean_scrub_stamp":"2024-03-30T00:43:10.734515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:43:22.680886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.149484+0000","last_change":"2024-03-30T00:43:10.755218+0000","last_active":"2024-03-30T00:43:11.149484+0000","last_peered":"2024-03-30T00:43:11.149484+0000","last_clean":"2024-03-30T00:43:11.149484+0000","last_became_active":"2024-03-30T00:43:10.754957+0000","last_became_peered":"2024-03-30T00:43:10.754957+0000","last_unstale":"2024-03-30T00:43:11.149484+0000","last_undegraded":"2024-03-30T00:43:11.149484+0000","last_fullsized":"2024-03-30T00:43:11.149484+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:43:09.732167+0000","last_clean_scrub_stamp":"2024-03-30T00:43:09.732167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:02.108234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-30T00:43:11.150035+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:43:11.150035+0000","last_peered":"2024-03-30T00:43:11.150035+0000","last_clean":"2024-03-30T00:43:11.150035+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:43:11.150035+0000","last_undegraded":"2024-03-30T00:43:11.150035+0000","last_fullsized":"2024-03-30T00:43:11.150035+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1433600,"data_stored":960382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65200000000000002,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.501},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.63600000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1459999999999999,"5min":1.149,"15min":1.149},"last":0.71099999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.64100000000000001,"15min":0.626},"min":{"1min":0.54500000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92900000000000005,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2450000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76600000000000001,"15min":0.78800000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.91600000000000004,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77800000000000002,"15min":0.79600000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2270000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.81399999999999995,"15min":0.82599999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.996,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.79500000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.49099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.65800000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0609999999999999,"5min":1.121,"15min":1.121},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.504,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0049999999999999,"5min":1.026,"15min":1.026},"last":0.81699999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47999999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":1.214,"15min":1.214},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.0189999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.83799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":815104,"data_stored":362652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.83299999999999996,"15min":0.91800000000000004},"min":{"1min":0.46700000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86199999999999999,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":1.2450000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91400000000000003,"5min":1.47,"15min":1.47},"last":0.71999999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.83999999999999997,"15min":0.93200000000000005},"min":{"1min":0.48199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86899999999999999,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.90700000000000003,"15min":1.0109999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93700000000000006,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":1.018}]},{"osd":2,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.90100000000000002,"15min":0.995},"min":{"1min":0.501,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90000000000000002,"5min":4.375,"15min":4.375},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.90100000000000002,"15min":1},"min":{"1min":0.50800000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84799999999999998,"5min":4.2960000000000003,"15min":4.2960000000000003},"last":0.76900000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.92000000000000004,"15min":1.016},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92700000000000005,"5min":4.3399999999999999,"15min":4.3399999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.89800000000000002,"15min":1.008},"min":{"1min":0.505,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81599999999999995,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.747}]},{"osd":4,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.40400000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.86399999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.53000000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90200000000000002,"5min":1.036,"15min":1.036},"last":0.88300000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.67900000000000005,"15min":0.69699999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.91900000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":1.101}]},{"osd":7,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71999999999999997,"15min":0.72299999999999998},"min":{"1min":0.49299999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.52800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89100000000000001,"5min":1.036,"15min":1.036},"last":0.84799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":815104,"data_stored":362640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65000000000000002,"15min":0.66600000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":1.095,"15min":1.095},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.86299999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.73299999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88400000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70499999999999996,"15min":0.73299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.87,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69299999999999995,"15min":0.71799999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92900000000000005,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.442}]},{"osd":3,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.76200000000000001,"15min":0.80200000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":1.7,"15min":1.7},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73999999999999999,"15min":0.76300000000000001},"min":{"1min":0.53200000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93500000000000005,"5min":1.181,"15min":1.181},"last":0.57299999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.76600000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.75,"15min":0.78000000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89300000000000002,"5min":1.615,"15min":1.615},"last":0.78000000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66200000000000003,"15min":0.67900000000000005},"min":{"1min":0.40699999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.877,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97899999999999998,"5min":1.569,"15min":1.569},"last":0.66100000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.745,"15min":0.75800000000000001},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.82299999999999995,"15min":0.84399999999999997},"min":{"1min":0.54300000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.03,"5min":1.383,"15min":1.383},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":835584,"data_stored":370802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.68600000000000005},"min":{"1min":0.441,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1699999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.64300000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90200000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71799999999999997,"15min":0.72999999999999998},"min":{"1min":0.47499999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94299999999999995,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71399999999999997,"15min":0.72299999999999998},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97999999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52100000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.92200000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.76200000000000001},"min":{"1min":0.53700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2190000000000001,"5min":1.228,"15min":1.228},"last":0.84399999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.196,"5min":1.27,"15min":1.27},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77800000000000002,"15min":0.78700000000000003},"min":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.95999999999999996,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.99299999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67800000000000005,"15min":0.70099999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80300000000000005,"5min":1.329,"15min":1.329},"last":0.873},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.71999999999999997,"15min":0.75800000000000001},"min":{"1min":0.44,"5min":0.373,"15min":0.373},"max":{"1min":0.83099999999999996,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.90000000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.68300000000000005,"15min":0.71899999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89100000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.73699999999999999,"15min":0.77000000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.93600000000000005,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.69099999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.72399999999999998,"15min":0.754},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96899999999999997,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.81100000000000005,"15min":0.83999999999999997},"min":{"1min":0.56499999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93000000000000005,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.85599999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":362618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47199999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.437},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51700000000000002,"15min":0.54800000000000004},"min":{"1min":0.35699999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.65800000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45300000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.53500000000000003}]},{"osd":2,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.5,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68600000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57399999999999995,"15min":0.59199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70399999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47399999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.75700000000000001,"15min":0.77500000000000002},"min":{"1min":0.60499999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.76300000000000001,"15min":0.78300000000000003},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98099999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.61199999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1060000000000001,"5min":1.125,"15min":1.125},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78900000000000003,"15min":0.81499999999999995},"min":{"1min":0.55200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93300000000000005,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80300000000000005,"15min":0.82399999999999995},"min":{"1min":0.624,"5min":0.499,"15min":0.499},"max":{"1min":1.1699999999999999,"5min":1.226,"15min":1.226},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0129999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.621}]},{"osd":7,"last update":"Sat Mar 30 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.86299999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0700000000000001,"5min":1.202,"15min":1.202},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.871,"15min":0.90400000000000003},"min":{"1min":0.74399999999999999,"5min":0.497,"15min":0.497},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.71699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":823296,"data_stored":369238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.46600000000000003,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.0999999999999996,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52000000000000002,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.45500000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.49099999999999999,"15min":0.501},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.437}]},{"osd":4,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.80800000000000005,"15min":0.81699999999999995},"min":{"1min":0.60099999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.85799999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.53400000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81200000000000006,"15min":0.82499999999999996},"min":{"1min":0.56699999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.56399999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.85799999999999998,"15min":0.88100000000000001},"min":{"1min":0.60099999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.495},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.57099999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":839680,"data_stored":364946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65600000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.32800000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45600000000000002,"15min":0.46800000000000003},"min":{"1min":0.38,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.48899999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.745,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.48599999999999999,"15min":0.46999999999999997},"min":{"1min":0.41499999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.374}]},{"osd":4,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.72899999999999998,"15min":0.71599999999999997},"min":{"1min":0.56899999999999995,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.61499999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.84799999999999998}]},{"osd":5,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.69899999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.014,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.67300000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.0669999999999999,"5min":1.097,"15min":1.097},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.78300000000000003,"15min":0.77400000000000002},"min":{"1min":0.69299999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.86199999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:42:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.80200000000000005,"15min":0.80000000000000004},"min":{"1min":0.65000000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.79600000000000004,"15min":0.78000000000000003},"min":{"1min":0.74299999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.877}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28396,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343444,"statfs":{"total":96636764160,"available":96607686656,"internally_reserved":0,"allocated":1404928,"data_stored":952986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46700000000000003,"15min":0.48099999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75800000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.34499999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.50600000000000001,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.497},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.56100000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.505,"15min":0.51400000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53400000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79700000000000004,"15min":0.79600000000000004},"min":{"1min":0.504,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.234,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.81200000000000006,"15min":0.80400000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.64500000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2609999999999999,"5min":1.462,"15min":1.462},"last":0.57399999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80000000000000004,"15min":0.80200000000000005},"min":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.196,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78100000000000003,"15min":0.78000000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.60699999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:42:39 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.351,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.80100000000000005,"15min":0.80000000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1220000000000001,"5min":1.556,"15min":1.556},"last":0.79600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:43:15.646 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T00:43:15.646 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-30T00:43:15.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.824+0000 7fa6ae400640 1 Processor -- start 2024-03-30T00:43:15.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.824+0000 7fa6ae400640 1 -- start start 2024-03-30T00:43:15.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6ae400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6a815a770 0x7fa6a80c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6ae400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa6a80b1db0 0x7fa6a80cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6ae400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a80ca130 con 0x7fa6a80ce780 2024-03-30T00:43:15.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6ae400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a80cb120 con 0x7fa6a815a770 2024-03-30T00:43:15.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6ae400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a815ab50 con 0x7fa6a80b1db0 2024-03-30T00:43:15.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6a7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6a815a770 0x7fa6a80c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6a7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6a815a770 0x7fa6a80c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44270/0 (socket says 172.21.15.66:44270) 2024-03-30T00:43:15.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6a7400640 1 -- 172.21.15.66:0/1918879184 learned_addr learned my addr 172.21.15.66:0/1918879184 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:15.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6a7400640 1 -- 172.21.15.66:0/1918879184 >> v2:172.21.15.156:3300/0 conn(0x7fa6a80b1db0 msgr2=0x7fa6a80cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:15.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa69fe00640 1 --2- 172.21.15.66:0/1918879184 >> v2:172.21.15.156:3300/0 conn(0x7fa6a80b1db0 0x7fa6a80cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6a7400640 1 --2- 172.21.15.66:0/1918879184 >> v2:172.21.15.156:3300/0 conn(0x7fa6a80b1db0 0x7fa6a80cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6a7400640 1 -- 172.21.15.66:0/1918879184 >> v1:172.21.15.156:6789/0 conn(0x7fa6a80ce780 legacy=0x7fa6a80710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:15.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa6a7400640 1 -- 172.21.15.66:0/1918879184 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a80ca400 con 0x7fa6a815a770 2024-03-30T00:43:15.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.828+0000 7fa69fe00640 1 --2- 172.21.15.66:0/1918879184 >> v2:172.21.15.156:3300/0 conn(0x7fa6a80b1db0 0x7fa6a80cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:15.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6a7400640 1 --2- 172.21.15.66:0/1918879184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a815a770 0x7fa6a80c9bf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80cbe90 tx=0x7fa69801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c9152cdbf73391 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:15.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6a6a00640 1 -- 172.21.15.66:0/1918879184 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6980ac620 con 0x7fa6a815a770 2024-03-30T00:43:15.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6a6a00640 1 -- 172.21.15.66:0/1918879184 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6980ac780 con 0x7fa6a815a770 2024-03-30T00:43:15.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6a6a00640 1 -- 172.21.15.66:0/1918879184 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6980acb10 con 0x7fa6a815a770 2024-03-30T00:43:15.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 -- 172.21.15.66:0/1918879184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a815a770 msgr2=0x7fa6a80c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 --2- 172.21.15.66:0/1918879184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a815a770 0x7fa6a80c9bf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80cbe90 tx=0x7fa69801c4b0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 -- 172.21.15.66:0/1918879184 shutdown_connections 2024-03-30T00:43:15.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 --2- 172.21.15.66:0/1918879184 >> v2:172.21.15.156:3300/0 conn(0x7fa6a80b1db0 0x7fa6a80cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 --2- 172.21.15.66:0/1918879184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a815a770 0x7fa6a80c9bf0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 -- 172.21.15.66:0/1918879184 >> 172.21.15.66:0/1918879184 conn(0x7fa6a806a190 msgr2=0x7fa6a806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:15.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 -- 172.21.15.66:0/1918879184 shutdown_connections 2024-03-30T00:43:15.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 -- 172.21.15.66:0/1918879184 wait complete. 2024-03-30T00:43:15.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 Processor -- start 2024-03-30T00:43:15.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 -- start start 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6a80b1db0 0x7fa6a80977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a80ce780 0x7fa6a8156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a815a770 0x7fa6a80c1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a8158590 con 0x7fa6a815a770 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a8158d40 con 0x7fa6a80ce780 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6ae400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a806dbb0 con 0x7fa6a80b1db0 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.832+0000 7fa6a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a815a770 0x7fa6a80c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a815a770 0x7fa6a80c1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44286/0 (socket says 172.21.15.66:44286) 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6a7e00640 1 -- 172.21.15.66:0/1076292905 learned_addr learned my addr 172.21.15.66:0/1076292905 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6a7400640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6a80b1db0 0x7fa6a80977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa69fe00640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a80ce780 0x7fa6a8156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6a7e00640 1 -- 172.21.15.66:0/1076292905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6a80b1db0 msgr2=0x7fa6a80977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6a7e00640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6a80b1db0 0x7fa6a80977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6a7e00640 1 -- 172.21.15.66:0/1076292905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a80ce780 msgr2=0x7fa6a8156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:15.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6a7e00640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a80ce780 0x7fa6a8156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:15.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6a7e00640 1 -- 172.21.15.66:0/1076292905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6980094d0 con 0x7fa6a815a770 2024-03-30T00:43:15.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6a7e00640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a815a770 0x7fa6a80c1270 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c780 tx=0x7fa6a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:15.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa69f400640 1 -- 172.21.15.66:0/1076292905 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0003d90 con 0x7fa6a815a770 2024-03-30T00:43:15.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa69f400640 1 -- 172.21.15.66:0/1076292905 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a000ce30 con 0x7fa6a815a770 2024-03-30T00:43:15.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa69f400640 1 -- 172.21.15.66:0/1076292905 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0005290 con 0x7fa6a815a770 2024-03-30T00:43:15.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6ae400640 1 -- 172.21.15.66:0/1076292905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a81587c0 con 0x7fa6a815a770 2024-03-30T00:43:15.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6ae400640 1 -- 172.21.15.66:0/1076292905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a8158a60 con 0x7fa6a815a770 2024-03-30T00:43:15.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa6ae400640 1 -- 172.21.15.66:0/1076292905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a8156940 con 0x7fa6a815a770 2024-03-30T00:43:15.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa69f400640 1 -- 172.21.15.66:0/1076292905 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa6a00053f0 con 0x7fa6a815a770 2024-03-30T00:43:15.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.836+0000 7fa69f400640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa684066690 0x7fa684068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:15.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.840+0000 7fa69f400640 1 -- 172.21.15.66:0/1076292905 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa6a0083660 con 0x7fa6a815a770 2024-03-30T00:43:15.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.840+0000 7fa6a7400640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa684066690 0x7fa684068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:15.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.840+0000 7fa6a7400640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa684066690 0x7fa684068b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa6980ae000 tx=0x7fa6980a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:15.844+0000 7fa69f400640 1 -- 172.21.15.66:0/1076292905 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa6a0050930 con 0x7fa6a815a770 2024-03-30T00:43:16.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.060+0000 7fa6ae400640 1 -- 172.21.15.66:0/1076292905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa6a8066c30 con 0x7fa6a815a770 2024-03-30T00:43:16.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa69f400640 1 -- 172.21.15.66:0/1076292905 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa6a0053f50 con 0x7fa6a815a770 2024-03-30T00:43:16.070 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-03-30T00:43:16.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 -- 172.21.15.66:0/1076292905 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa684066690 msgr2=0x7fa684068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:16.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa684066690 0x7fa684068b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa6980ae000 tx=0x7fa6980a7000 comp rx=0 tx=0).stop 2024-03-30T00:43:16.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 -- 172.21.15.66:0/1076292905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a815a770 msgr2=0x7fa6a80c1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:16.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a815a770 0x7fa6a80c1270 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c780 tx=0x7fa6a000cc40 comp rx=0 tx=0).stop 2024-03-30T00:43:16.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 -- 172.21.15.66:0/1076292905 shutdown_connections 2024-03-30T00:43:16.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa684066690 0x7fa684068b50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a815a770 0x7fa6a80c1270 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a80ce780 0x7fa6a8156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 --2- 172.21.15.66:0/1076292905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6a80b1db0 0x7fa6a80977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 -- 172.21.15.66:0/1076292905 >> 172.21.15.66:0/1076292905 conn(0x7fa6a806a190 msgr2=0x7fa6a81527a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:16.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 -- 172.21.15.66:0/1076292905 shutdown_connections 2024-03-30T00:43:16.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.064+0000 7fa6ae400640 1 -- 172.21.15.66:0/1076292905 wait complete. 2024-03-30T00:43:16.091 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:43:16.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.224+0000 7fb063e00640 1 Processor -- start 2024-03-30T00:43:16.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.224+0000 7fb063e00640 1 -- start start 2024-03-30T00:43:16.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.224+0000 7fb063e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb05c15ca80 0x7fb05c06b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:16.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.224+0000 7fb063e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb05c0cf2f0 0x7fb05c06b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:16.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.224+0000 7fb063e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb05c0cb0e0 con 0x7fb05c06bb30 2024-03-30T00:43:16.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.224+0000 7fb063e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05c0cf7f0 con 0x7fb05c0cf2f0 2024-03-30T00:43:16.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.224+0000 7fb063e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05c0cfad0 con 0x7fb05c15ca80 2024-03-30T00:43:16.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.224+0000 7fb060a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb05c0cf2f0 0x7fb05c06b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:16.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.224+0000 7fb060a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb05c0cf2f0 0x7fb05c06b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44296/0 (socket says 172.21.15.66:44296) 2024-03-30T00:43:16.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.224+0000 7fb060a00640 1 -- 172.21.15.66:0/2088203065 learned_addr learned my addr 172.21.15.66:0/2088203065 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:16.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.228+0000 7fb061400640 1 --2- 172.21.15.66:0/2088203065 >> v2:172.21.15.156:3300/0 conn(0x7fb05c15ca80 0x7fb05c06b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:16.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.228+0000 7fb060a00640 1 -- 172.21.15.66:0/2088203065 >> v2:172.21.15.156:3300/0 conn(0x7fb05c15ca80 msgr2=0x7fb05c06b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:16.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.228+0000 7fb060a00640 1 --2- 172.21.15.66:0/2088203065 >> v2:172.21.15.156:3300/0 conn(0x7fb05c15ca80 0x7fb05c06b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.228+0000 7fb060a00640 1 -- 172.21.15.66:0/2088203065 >> v1:172.21.15.156:6789/0 conn(0x7fb05c06bb30 legacy=0x7fb05c098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:16.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.228+0000 7fb060a00640 1 -- 172.21.15.66:0/2088203065 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c0cad70 con 0x7fb05c0cf2f0 2024-03-30T00:43:16.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.228+0000 7fb060a00640 1 --2- 172.21.15.66:0/2088203065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb05c0cf2f0 0x7fb05c06b5f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb05c070ed0 tx=0x7fb04801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8729512236783003 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:16.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.228+0000 7fb05be00640 1 -- 172.21.15.66:0/2088203065 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0480a89d0 con 0x7fb05c0cf2f0 2024-03-30T00:43:16.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.228+0000 7fb05be00640 1 -- 172.21.15.66:0/2088203065 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0480a8b30 con 0x7fb05c0cf2f0 2024-03-30T00:43:16.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.228+0000 7fb05be00640 1 -- 172.21.15.66:0/2088203065 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0480b0440 con 0x7fb05c0cf2f0 2024-03-30T00:43:16.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.228+0000 7fb063e00640 1 -- 172.21.15.66:0/2088203065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb05c0cf2f0 msgr2=0x7fb05c06b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:16.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.228+0000 7fb063e00640 1 --2- 172.21.15.66:0/2088203065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb05c0cf2f0 0x7fb05c06b5f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb05c070ed0 tx=0x7fb04801ecd0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 -- 172.21.15.66:0/2088203065 shutdown_connections 2024-03-30T00:43:16.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 --2- 172.21.15.66:0/2088203065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb05c0cf2f0 0x7fb05c06b5f0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 --2- 172.21.15.66:0/2088203065 >> v2:172.21.15.156:3300/0 conn(0x7fb05c15ca80 0x7fb05c06b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 -- 172.21.15.66:0/2088203065 >> 172.21.15.66:0/2088203065 conn(0x7fb05c0b2090 msgr2=0x7fb05c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:16.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 -- 172.21.15.66:0/2088203065 shutdown_connections 2024-03-30T00:43:16.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 -- 172.21.15.66:0/2088203065 wait complete. 2024-03-30T00:43:16.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 Processor -- start 2024-03-30T00:43:16.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 -- start start 2024-03-30T00:43:16.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb05c06bb30 0x7fb05c0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:16.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb05c0cf2f0 0x7fb05c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:16.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb05c15ca80 0x7fb05c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:16.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c0b41e0 con 0x7fb05c06bb30 2024-03-30T00:43:16.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c0b4490 con 0x7fb05c15ca80 2024-03-30T00:43:16.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb063e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c0b4770 con 0x7fb05c0cf2f0 2024-03-30T00:43:16.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb061400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb05c06bb30 0x7fb05c0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:16.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb061400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb05c06bb30 0x7fb05c0b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44302/0 (socket says 172.21.15.66:44302) 2024-03-30T00:43:16.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb061400640 1 -- 172.21.15.66:0/3215583437 learned_addr learned my addr 172.21.15.66:0/3215583437 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:16.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb061e00640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb05c15ca80 0x7fb05c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:16.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb060a00640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb05c0cf2f0 0x7fb05c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:16.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb061400640 1 -- 172.21.15.66:0/3215583437 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb05c0cf2f0 msgr2=0x7fb05c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:16.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb061400640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb05c0cf2f0 0x7fb05c0b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb061400640 1 -- 172.21.15.66:0/3215583437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb05c15ca80 msgr2=0x7fb05c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:16.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb061400640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb05c15ca80 0x7fb05c0b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb061400640 1 -- 172.21.15.66:0/3215583437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0480094d0 con 0x7fb05c06bb30 2024-03-30T00:43:16.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.232+0000 7fb061400640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb05c06bb30 0x7fb05c0b3760 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb05000d5f0 tx=0x7fb05000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:16.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.236+0000 7fb059600640 1 -- 172.21.15.66:0/3215583437 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0500125f0 con 0x7fb05c06bb30 2024-03-30T00:43:16.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.236+0000 7fb059600640 1 -- 172.21.15.66:0/3215583437 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb050012750 con 0x7fb05c06bb30 2024-03-30T00:43:16.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.236+0000 7fb063e00640 1 -- 172.21.15.66:0/3215583437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05c0b4a50 con 0x7fb05c06bb30 2024-03-30T00:43:16.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.236+0000 7fb059600640 1 -- 172.21.15.66:0/3215583437 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050012ae0 con 0x7fb05c06bb30 2024-03-30T00:43:16.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.236+0000 7fb063e00640 1 -- 172.21.15.66:0/3215583437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05c0b5010 con 0x7fb05c06bb30 2024-03-30T00:43:16.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.236+0000 7fb063e00640 1 -- 172.21.15.66:0/3215583437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05c0b51d0 con 0x7fb05c06bb30 2024-03-30T00:43:16.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.236+0000 7fb059600640 1 -- 172.21.15.66:0/3215583437 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb050012c40 con 0x7fb05c06bb30 2024-03-30T00:43:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.236+0000 7fb059600640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb03c0666c0 0x7fb03c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.236+0000 7fb059600640 1 -- 172.21.15.66:0/3215583437 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb0500846c0 con 0x7fb05c06bb30 2024-03-30T00:43:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.236+0000 7fb060a00640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb03c0666c0 0x7fb03c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.240+0000 7fb060a00640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb03c0666c0 0x7fb03c068b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb0480b2000 tx=0x7fb048009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.240+0000 7fb059600640 1 -- 172.21.15.66:0/3215583437 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb050051900 con 0x7fb05c06bb30 2024-03-30T00:43:16.282 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:43:16.285 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:43:16.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.436+0000 7fb063e00640 1 -- 172.21.15.66:0/3215583437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb05c066c30 con 0x7fb05c06bb30 2024-03-30T00:43:16.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.436+0000 7fb059600640 1 -- 172.21.15.66:0/3215583437 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fb050007380 con 0x7fb05c06bb30 2024-03-30T00:43:16.443 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:16.443 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4272961638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4272961638},{"type":"v1","addr":"172.21.15.156:6834","nonce":4272961638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2144186622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2144186622},{"type":"v1","addr":"172.21.15.66:6835","nonce":2144186622}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/750308788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":750308788},{"type":"v1","addr":"172.21.15.66:6837","nonce":750308788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:43:11.141056+0000","modified":"2024-03-30T00:43:14.154117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6468":{"gid":6468,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/1137593870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1137593870},{"type":"v1","addr":"172.21.15.156:6835","nonce":1137593870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-30T00:43:16.443 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 33 2024-03-30T00:43:16.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 -- 172.21.15.66:0/3215583437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb03c0666c0 msgr2=0x7fb03c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:16.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb03c0666c0 0x7fb03c068b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb0480b2000 tx=0x7fb048009040 comp rx=0 tx=0).stop 2024-03-30T00:43:16.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 -- 172.21.15.66:0/3215583437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb05c06bb30 msgr2=0x7fb05c0b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:16.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb05c06bb30 0x7fb05c0b3760 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb05000d5f0 tx=0x7fb05000dab0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 -- 172.21.15.66:0/3215583437 shutdown_connections 2024-03-30T00:43:16.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb05c15ca80 0x7fb05c0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb05c0cf2f0 0x7fb05c0b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb03c0666c0 0x7fb03c068b80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 --2- 172.21.15.66:0/3215583437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb05c06bb30 0x7fb05c0b3760 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:16.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 -- 172.21.15.66:0/3215583437 >> 172.21.15.66:0/3215583437 conn(0x7fb05c0b2090 msgr2=0x7fb05c0cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:16.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 -- 172.21.15.66:0/3215583437 shutdown_connections 2024-03-30T00:43:16.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:16.440+0000 7fb063e00640 1 -- 172.21.15.66:0/3215583437 wait complete. 2024-03-30T00:43:16.866 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:43:11.141056+0000', 'modified': '2024-03-30T00:43:14.154117+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6468}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6468': {'gid': 6468, 'name': 'd', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6835/1137593870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6833', 'nonce': 1137593870}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 1137593870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:43:16.867 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:43:16.867 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-30T00:43:17.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2d7200640 1 Processor -- start 2024-03-30T00:43:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2d7200640 1 -- start start 2024-03-30T00:43:17.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2d7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff2d015ca80 0x7ff2d006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2d7200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d00cb0e0 con 0x7ff2d00cf2f0 2024-03-30T00:43:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2d7200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d00cf7f0 con 0x7ff2d00cb570 2024-03-30T00:43:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2d7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d00cfad0 con 0x7ff2d015ca80 2024-03-30T00:43:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2cfe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff2d015ca80 0x7ff2d006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2c7400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff2d00cf2f0 0x7ff2d006b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60928/0 (socket says 172.21.15.66:60928) 2024-03-30T00:43:17.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2c7400640 1 -- 172.21.15.66:0/2674781613 learned_addr learned my addr 172.21.15.66:0/2674781613 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:17.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2cfe00640 1 -- 172.21.15.66:0/2674781613 >> v1:172.21.15.156:6789/0 conn(0x7ff2d00cb570 legacy=0x7ff2d00cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2cfe00640 1 -- 172.21.15.66:0/2674781613 >> v1:172.21.15.66:6789/0 conn(0x7ff2d00cf2f0 legacy=0x7ff2d006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2cfe00640 1 -- 172.21.15.66:0/2674781613 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d00cad70 con 0x7ff2d015ca80 2024-03-30T00:43:17.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.004+0000 7ff2cf400640 1 -- 172.21.15.66:0/2674781613 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562098399 0 0) 0x7ff2d00cb0e0 con 0x7ff2d00cf2f0 2024-03-30T00:43:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/2674781613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2d015ca80 0x7ff2d006b0b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00097c0 tx=0x7ff2c001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d11a3e6edfb026e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2cf400640 1 -- 172.21.15.66:0/2674781613 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00a8990 con 0x7ff2d015ca80 2024-03-30T00:43:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2cf400640 1 -- 172.21.15.66:0/2674781613 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c00a8af0 con 0x7ff2d015ca80 2024-03-30T00:43:17.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2cf400640 1 -- 172.21.15.66:0/2674781613 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00a8e80 con 0x7ff2d015ca80 2024-03-30T00:43:17.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2d7200640 1 -- 172.21.15.66:0/2674781613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2d015ca80 msgr2=0x7ff2d006b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2d7200640 1 --2- 172.21.15.66:0/2674781613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2d015ca80 0x7ff2d006b0b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00097c0 tx=0x7ff2c001ec90 comp rx=0 tx=0).stop 2024-03-30T00:43:17.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2d7200640 1 -- 172.21.15.66:0/2674781613 shutdown_connections 2024-03-30T00:43:17.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2d7200640 1 --2- 172.21.15.66:0/2674781613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2d015ca80 0x7ff2d006b0b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2d7200640 1 -- 172.21.15.66:0/2674781613 >> 172.21.15.66:0/2674781613 conn(0x7ff2d00b2090 msgr2=0x7ff2d00b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:17.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2d7200640 1 -- 172.21.15.66:0/2674781613 shutdown_connections 2024-03-30T00:43:17.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2d7200640 1 -- 172.21.15.66:0/2674781613 wait complete. 2024-03-30T00:43:17.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.008+0000 7ff2d7200640 1 Processor -- start 2024-03-30T00:43:17.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d7200640 1 -- start start 2024-03-30T00:43:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d7200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2d00cb570 0x7ff2d00b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d7200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2d00cf2f0 0x7ff2d014b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2d015ca80 0x7ff2d00c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00b3700 con 0x7ff2d015ca80 2024-03-30T00:43:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d7200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00b39a0 con 0x7ff2d00cf2f0 2024-03-30T00:43:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d7200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00b3c80 con 0x7ff2d00cb570 2024-03-30T00:43:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2d015ca80 0x7ff2d00c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2d015ca80 0x7ff2d00c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44326/0 (socket says 172.21.15.66:44326) 2024-03-30T00:43:17.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d5200640 1 -- 172.21.15.66:0/2058542915 learned_addr learned my addr 172.21.15.66:0/2058542915 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:17.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2c7400640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2d00cf2f0 0x7ff2d014b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2d00cb570 0x7ff2d00b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d5200640 1 -- 172.21.15.66:0/2058542915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2d00cb570 msgr2=0x7ff2d00b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d5200640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2d00cb570 0x7ff2d00b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d5200640 1 -- 172.21.15.66:0/2058542915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2d00cf2f0 msgr2=0x7ff2d014b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d5200640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2d00cf2f0 0x7ff2d014b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d5200640 1 -- 172.21.15.66:0/2058542915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c0009420 con 0x7ff2d015ca80 2024-03-30T00:43:17.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2d00cb570 0x7ff2d00b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:17.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d5200640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2d015ca80 0x7ff2d00c9330 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800c430 tx=0x7ff2c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:17.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2ccc00640 1 -- 172.21.15.66:0/2058542915 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8005220 con 0x7ff2d015ca80 2024-03-30T00:43:17.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2ccc00640 1 -- 172.21.15.66:0/2058542915 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c8005380 con 0x7ff2d015ca80 2024-03-30T00:43:17.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2ccc00640 1 -- 172.21.15.66:0/2058542915 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8007270 con 0x7ff2d015ca80 2024-03-30T00:43:17.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d7200640 1 -- 172.21.15.66:0/2058542915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d00b3f60 con 0x7ff2d015ca80 2024-03-30T00:43:17.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.012+0000 7ff2d7200640 1 -- 172.21.15.66:0/2058542915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d00b4520 con 0x7ff2d015ca80 2024-03-30T00:43:17.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2ccc00640 1 -- 172.21.15.66:0/2058542915 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff2c80073d0 con 0x7ff2d015ca80 2024-03-30T00:43:17.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2ccc00640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff2ac066690 0x7ff2ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2ccc00640 1 -- 172.21.15.66:0/2058542915 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff2c8083840 con 0x7ff2d015ca80 2024-03-30T00:43:17.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff2ac066690 0x7ff2ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2da6ea000 1 Processor -- start 2024-03-30T00:43:17.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2da6ea000 1 -- start start 2024-03-30T00:43:17.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff2ac066690 0x7ff2ac068b50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00b2000 tx=0x7ff2c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:17.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd435f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd4368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bd2fe5a0 con 0x5559bd47fdb0 2024-03-30T00:43:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bd28baf0 con 0x5559bd4806d0 2024-03-30T00:43:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559bd2890c0 con 0x5559bd4364c0 2024-03-30T00:43:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44336/0 (socket says 172.21.15.66:44336) 2024-03-30T00:43:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2cfe00640 1 -- 172.21.15.66:0/3034047075 learned_addr learned my addr 172.21.15.66:0/3034047075 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2c7400640 1 --2- 172.21.15.66:0/3034047075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd435f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.016+0000 7ff2d5200640 1 --2- 172.21.15.66:0/3034047075 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd4368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2cfe00640 1 -- 172.21.15.66:0/3034047075 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 msgr2=0x5559bd4368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/3034047075 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd4368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2cfe00640 1 -- 172.21.15.66:0/3034047075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 msgr2=0x5559bd435f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/3034047075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd435f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2cfe00640 1 -- 172.21.15.66:0/3034047075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559bcf80890 con 0x5559bd47fdb0 2024-03-30T00:43:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c7400640 1 --2- 172.21.15.66:0/3034047075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd435f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/3034047075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480190 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff2c001b8a0 tx=0x7ff2c00aa070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b410555a688f7a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c4c00640 1 -- 172.21.15.66:0/3034047075 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00c7430 con 0x5559bd47fdb0 2024-03-30T00:43:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c4c00640 1 -- 172.21.15.66:0/3034047075 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c00c1070 con 0x5559bd47fdb0 2024-03-30T00:43:17.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c4c00640 1 -- 172.21.15.66:0/3034047075 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00c7790 con 0x5559bd47fdb0 2024-03-30T00:43:17.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 -- 172.21.15.66:0/3034047075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 msgr2=0x5559bd480190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/3034047075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480190 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff2c001b8a0 tx=0x7ff2c00aa070 comp rx=0 tx=0).stop 2024-03-30T00:43:17.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 -- 172.21.15.66:0/3034047075 shutdown_connections 2024-03-30T00:43:17.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/3034047075 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd4368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/3034047075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd435f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/3034047075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480190 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 -- 172.21.15.66:0/3034047075 >> 172.21.15.66:0/3034047075 conn(0x5559bd43c830 msgr2=0x5559bd434860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:17.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 -- 172.21.15.66:0/3034047075 shutdown_connections 2024-03-30T00:43:17.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 -- 172.21.15.66:0/3034047075 wait complete. 2024-03-30T00:43:17.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 -- start start 2024-03-30T00:43:17.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 Processor -- start 2024-03-30T00:43:17.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd43b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd480ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bd2849b0 con 0x5559bd47fdb0 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bd223c00 con 0x5559bd4806d0 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559bd242070 con 0x5559bd4364c0 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2da6ea000 10 client.0 _ll_register_callbacks cb 0x5559bd4220e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44338/0 (socket says 172.21.15.66:44338) 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c7400640 1 -- 172.21.15.66:0/2896127811 learned_addr learned my addr 172.21.15.66:0/2896127811 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2d5200640 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd480ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd43b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c7400640 1 -- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 msgr2=0x5559bd43b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c7400640 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd43b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c7400640 1 -- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 msgr2=0x5559bd480ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c7400640 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd480ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2c7400640 1 -- 172.21.15.66:0/2896127811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d00cad70 con 0x5559bd47fdb0 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.020+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd43b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2c7400640 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480ab0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8007090 tx=0x7ff2b8007550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3667c212d24e8d99 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2bd600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2bd600640 1 -- 172.21.15.66:0/2896127811 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8011660 con 0x5559bd47fdb0 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 -- 172.21.15.66:0/2896127811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559bd2415f0 con 0x5559bd47fdb0 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2bd600640 1 -- 172.21.15.66:0/2896127811 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff2b80117c0 con 0x5559bd47fdb0 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2bd600640 1 -- 172.21.15.66:0/2896127811 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b8011980 con 0x5559bd47fdb0 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 10 client.6579 fetch_fsmap learned FSMap version 33 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 -- 172.21.15.66:0/2896127811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5559bd249ab0 con 0x5559bd47fdb0 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2bd600640 1 -- 172.21.15.66:0/2896127811 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5559bd2415f0 con 0x5559bd47fdb0 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2bd600640 1 -- 172.21.15.66:0/2896127811 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff2b8011ae0 con 0x5559bd47fdb0 2024-03-30T00:43:17.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 10 client.6579 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T00:43:17.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 10 client.6579 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T00:43:17.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 20 client.6579 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:17.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5559bd4865d0 0x5559bd4869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 4 client.6579 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:43:17.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 -- 172.21.15.66:0/2896127811 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559bd281790 con 0x5559bd4865d0 2024-03-30T00:43:17.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5559bd4865d0 0x5559bd4869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5559bd4865d0 0x5559bd4869b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ffb77caed065a9 server_cookie=720db5747bd2ce69 in_seq=0 out_seq=0 2024-03-30T00:43:17.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2bd600640 10 client.6579 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:17.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2bd600640 1 -- 172.21.15.66:0/2896127811 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5559bd281790 con 0x5559bd4865d0 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2bd600640 10 client.6579 handle_command_reply: tid=0 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 client.6579 shutdown 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 -- 172.21.15.66:0/2896127811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 msgr2=0x5559bd480ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480ab0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8007090 tx=0x7ff2b8007550 comp rx=0 tx=0).stop 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 -- 172.21.15.66:0/2896127811 shutdown_connections 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5559bd4865d0 0x5559bd4869b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd480ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd480ab0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/2896127811 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd43b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2bd600640 0 client.6579 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 -- 172.21.15.66:0/2896127811 >> 172.21.15.66:0/2896127811 conn(0x5559bd43c830 msgr2=0x5559bd434f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:17.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 -- 172.21.15.66:0/2896127811 shutdown_connections 2024-03-30T00:43:17.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 1 -- 172.21.15.66:0/2896127811 wait complete. 2024-03-30T00:43:17.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.024+0000 7ff2da6ea000 20 client.6579 trim_cache size 0 max 16384 2024-03-30T00:43:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.040+0000 7ff2da6ea000 1 Processor -- start 2024-03-30T00:43:17.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.040+0000 7ff2da6ea000 1 -- start start 2024-03-30T00:43:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.040+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd47eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.040+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd47f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.040+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd481890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.040+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bcf41940 con 0x5559bd47fdb0 2024-03-30T00:43:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.040+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bd480ce0 con 0x5559bd4806d0 2024-03-30T00:43:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.040+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559bd2c7ac0 con 0x5559bd4364c0 2024-03-30T00:43:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd47f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd47f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44346/0 (socket says 172.21.15.66:44346) 2024-03-30T00:43:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 -- 172.21.15.66:0/3588285337 learned_addr learned my addr 172.21.15.66:0/3588285337 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:17.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/3588285337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd47eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2d5200640 1 --2- 172.21.15.66:0/3588285337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd481890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 -- 172.21.15.66:0/3588285337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 msgr2=0x5559bd47eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 --2- 172.21.15.66:0/3588285337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd47eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 -- 172.21.15.66:0/3588285337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 msgr2=0x5559bd481890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 --2- 172.21.15.66:0/3588285337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd481890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 -- 172.21.15.66:0/3588285337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559bd22fc00 con 0x5559bd47fdb0 2024-03-30T00:43:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 --2- 172.21.15.66:0/3588285337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd47f3f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80112a0 tx=0x7ff2b800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe7dd333f3968e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c5600640 1 -- 172.21.15.66:0/3588285337 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b802b070 con 0x5559bd47fdb0 2024-03-30T00:43:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c5600640 1 -- 172.21.15.66:0/3588285337 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b8002090 con 0x5559bd47fdb0 2024-03-30T00:43:17.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c5600640 1 -- 172.21.15.66:0/3588285337 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8015b10 con 0x5559bd47fdb0 2024-03-30T00:43:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 -- 172.21.15.66:0/3588285337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 msgr2=0x5559bd47f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/3588285337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd47f3f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80112a0 tx=0x7ff2b800c000 comp rx=0 tx=0).stop 2024-03-30T00:43:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 -- 172.21.15.66:0/3588285337 shutdown_connections 2024-03-30T00:43:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/3588285337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd481890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/3588285337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd47f3f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/3588285337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd47eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 -- 172.21.15.66:0/3588285337 >> 172.21.15.66:0/3588285337 conn(0x5559bd43c830 msgr2=0x5559bd47d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 -- 172.21.15.66:0/3588285337 shutdown_connections 2024-03-30T00:43:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 -- 172.21.15.66:0/3588285337 wait complete. 2024-03-30T00:43:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 -- start start 2024-03-30T00:43:17.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 Processor -- start 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd4c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd484510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd484a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bd221c10 con 0x5559bd47fdb0 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bd2537c0 con 0x5559bd4806d0 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559bd439b10 con 0x5559bd4364c0 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2da6ea000 10 client.0 _ll_register_callbacks cb 0x5559bd249590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd484510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd484510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44356/0 (socket says 172.21.15.66:44356) 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.044+0000 7ff2c7400640 1 -- 172.21.15.66:0/1412972193 learned_addr learned my addr 172.21.15.66:0/1412972193 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd4c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2d5200640 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd484a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2c7400640 1 -- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 msgr2=0x5559bd4c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2c7400640 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd4c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2c7400640 1 -- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 msgr2=0x5559bd484a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2c7400640 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd484a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2c7400640 1 -- 172.21.15.66:0/1412972193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559bd249ab0 con 0x5559bd47fdb0 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2c7400640 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd484510 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8004cd0 tx=0x7ff2b8001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44ea8c73f1e5ad8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2bd600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2bd600640 1 -- 172.21.15.66:0/1412972193 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8015880 con 0x5559bd47fdb0 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2bd600640 1 -- 172.21.15.66:0/1412972193 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff2b8001390 con 0x5559bd47fdb0 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 -- 172.21.15.66:0/1412972193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559bd43a5f0 con 0x5559bd47fdb0 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2bd600640 1 -- 172.21.15.66:0/1412972193 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b80159e0 con 0x5559bd47fdb0 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 10 client.6585 fetch_fsmap learned FSMap version 33 2024-03-30T00:43:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 -- 172.21.15.66:0/1412972193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5559bcf80cc0 con 0x5559bd47fdb0 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2bd600640 1 -- 172.21.15.66:0/1412972193 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5559bd43a5f0 con 0x5559bd47fdb0 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2bd600640 1 -- 172.21.15.66:0/1412972193 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff2b8015b40 con 0x5559bd47fdb0 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 10 client.6585 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 10 client.6585 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 20 client.6585 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5559bd484f90 0x5559bd485370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 4 client.6585 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 -- 172.21.15.66:0/1412972193 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5559bd254460 con 0x5559bd484f90 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5559bd484f90 0x5559bd485370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2cfe00640 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5559bd484f90 0x5559bd485370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84d5f7a0e39309b1 server_cookie=d1c9b78f5d40e5a9 in_seq=0 out_seq=0 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2bd600640 10 client.6585 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2bd600640 1 -- 172.21.15.66:0/1412972193 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5559bd254460 con 0x5559bd484f90 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2bd600640 10 client.6585 handle_command_reply: tid=0 2024-03-30T00:43:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 client.6585 shutdown 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 -- 172.21.15.66:0/1412972193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 msgr2=0x5559bd484510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd484510 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8004cd0 tx=0x7ff2b8001260 comp rx=0 tx=0).stop 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 -- 172.21.15.66:0/1412972193 shutdown_connections 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5559bd484f90 0x5559bd485370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559bd4806d0 0x5559bd484a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559bd47fdb0 0x5559bd484510 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 --2- 172.21.15.66:0/1412972193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559bd4364c0 0x5559bd4c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2bd600640 0 client.6585 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 -- 172.21.15.66:0/1412972193 >> 172.21.15.66:0/1412972193 conn(0x5559bd43c830 msgr2=0x5559bd43bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 -- 172.21.15.66:0/1412972193 shutdown_connections 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 1 -- 172.21.15.66:0/1412972193 wait complete. 2024-03-30T00:43:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2da6ea000 20 client.6585 trim_cache size 0 max 16384 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6468, 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 5.907094259, 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 33, 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 68, 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 68, 2024-03-30T00:43:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 10.211223089000001 2024-03-30T00:43:17.061 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:43:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2d7200640 1 -- 172.21.15.66:0/2058542915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff2ac066690 msgr2=0x7ff2ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2d7200640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff2ac066690 0x7ff2ac068b50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00b2000 tx=0x7ff2c0008040 comp rx=0 tx=0).stop 2024-03-30T00:43:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2d7200640 1 -- 172.21.15.66:0/2058542915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2d015ca80 msgr2=0x7ff2d00c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.048+0000 7ff2d7200640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2d015ca80 0x7ff2d00c9330 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800c430 tx=0x7ff2c800c830 comp rx=0 tx=0).stop 2024-03-30T00:43:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.052+0000 7ff2d7200640 1 -- 172.21.15.66:0/2058542915 shutdown_connections 2024-03-30T00:43:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.052+0000 7ff2d7200640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff2ac066690 0x7ff2ac068b50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.052+0000 7ff2d7200640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2d015ca80 0x7ff2d00c9330 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.052+0000 7ff2d7200640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2d00cf2f0 0x7ff2d014b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.052+0000 7ff2d7200640 1 --2- 172.21.15.66:0/2058542915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2d00cb570 0x7ff2d00b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.052+0000 7ff2d7200640 1 -- 172.21.15.66:0/2058542915 >> 172.21.15.66:0/2058542915 conn(0x7ff2d00b2090 msgr2=0x7ff2d00cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.052+0000 7ff2d7200640 1 -- 172.21.15.66:0/2058542915 shutdown_connections 2024-03-30T00:43:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.052+0000 7ff2d7200640 1 -- 172.21.15.66:0/2058542915 wait complete. 2024-03-30T00:43:17.076 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:43:17.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.208+0000 7f473ec00640 1 Processor -- start 2024-03-30T00:43:17.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f473ec00640 1 -- start start 2024-03-30T00:43:17.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f473ec00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f473806e800 0x7f47380d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f473ec00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f473806f8e0 0x7f47380ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f473ec00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473806b0b0 con 0x7f47380cc560 2024-03-30T00:43:17.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f473ec00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4738160640 con 0x7f473806e800 2024-03-30T00:43:17.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f473ec00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473806b390 con 0x7f473806f8e0 2024-03-30T00:43:17.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f47380cc560 0x7f473806e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60940/0 (socket says 172.21.15.66:60940) 2024-03-30T00:43:17.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737e00640 1 -- 172.21.15.66:0/1188106854 learned_addr learned my addr 172.21.15.66:0/1188106854 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:17.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f473cc00640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f473806f8e0 0x7f47380ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f472f400640 1 --2- 172.21.15.66:0/1188106854 >> v2:172.21.15.156:3300/0 conn(0x7f473806e800 0x7f47380d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804978518 0 0) 0x7f473806b0b0 con 0x7f47380cc560 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f471c003530 con 0x7f47380cc560 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2552413773 0 0) 0x7f471c003530 con 0x7f47380cc560 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f473806f8e0 msgr2=0x7f47380ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f473806f8e0 0x7f47380ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 >> v2:172.21.15.156:3300/0 conn(0x7f473806e800 msgr2=0x7f47380d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 --2- 172.21.15.66:0/1188106854 >> v2:172.21.15.156:3300/0 conn(0x7f473806e800 0x7f47380d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473806b670 con 0x7f47380cc560 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f47280016a0 con 0x7f47380cc560 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f472f400640 1 --2- 172.21.15.66:0/1188106854 >> v2:172.21.15.156:3300/0 conn(0x7f473806e800 0x7f47380d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 >> v1:172.21.15.66:6789/0 conn(0x7f47380cc560 legacy=0x7f473806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 0x7f471c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f473806f8e0 0x7f471c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f471c004d10 0x7f471c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4738160640 con 0x7f471c003980 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473806b390 con 0x7f471c004d10 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471c00b990 con 0x7f473806f8e0 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737e00640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 0x7f471c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f472f400640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f473806f8e0 0x7f471c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f473cc00640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f471c004d10 0x7f471c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737e00640 1 -- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f473806f8e0 msgr2=0x7f471c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737e00640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f473806f8e0 0x7f471c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737e00640 1 -- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f471c004d10 msgr2=0x7f471c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737e00640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f471c004d10 0x7f471c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f4737e00640 1 -- 172.21.15.66:0/1188106854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c003530 con 0x7f471c003980 2024-03-30T00:43:17.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.212+0000 7f473cc00640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f471c004d10 0x7f471c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:17.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.216+0000 7f4737e00640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 0x7f471c003da0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f472800c240 tx=0x7f472801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04dea63f57a0101 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:17.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.216+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47280a5910 con 0x7f471c003980 2024-03-30T00:43:17.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.216+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47280a5a90 con 0x7f471c003980 2024-03-30T00:43:17.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.216+0000 7f4737400640 1 -- 172.21.15.66:0/1188106854 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47280a5e40 con 0x7f471c003980 2024-03-30T00:43:17.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.216+0000 7f473ec00640 1 -- 172.21.15.66:0/1188106854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 msgr2=0x7f471c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.216+0000 7f473ec00640 1 --2- 172.21.15.66:0/1188106854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 0x7f471c003da0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f472800c240 tx=0x7f472801cc10 comp rx=0 tx=0).stop 2024-03-30T00:43:17.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.216+0000 7f473cc00640 1 -- 172.21.15.66:0/1188106854 reap_dead start 2024-03-30T00:43:17.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.216+0000 7f473ec00640 1 -- 172.21.15.66:0/1188106854 shutdown_connections 2024-03-30T00:43:17.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.216+0000 7f473ec00640 1 -- 172.21.15.66:0/1188106854 >> 172.21.15.66:0/1188106854 conn(0x7f47380d0540 msgr2=0x7f47380d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:17.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.216+0000 7f473ec00640 1 -- 172.21.15.66:0/1188106854 shutdown_connections 2024-03-30T00:43:17.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.216+0000 7f473ec00640 1 -- 172.21.15.66:0/1188106854 wait complete. 2024-03-30T00:43:17.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473ec00640 1 Processor -- start 2024-03-30T00:43:17.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473ec00640 1 -- start start 2024-03-30T00:43:17.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 0x7f47380b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473ec00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47380c9a20 0x7f47380c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473ec00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47381f4600 0x7f47381f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473815bb30 con 0x7f471c003980 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473ec00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473815b200 con 0x7f47381f4600 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473ec00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473815b4e0 con 0x7f47380c9a20 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4737e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 0x7f47380b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4737e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 0x7f47380b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44370/0 (socket says 172.21.15.66:44370) 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4737e00640 1 -- 172.21.15.66:0/1471381436 learned_addr learned my addr 172.21.15.66:0/1471381436 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473cc00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47381f4600 0x7f47381f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f472f400640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47380c9a20 0x7f47380c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4737e00640 1 -- 172.21.15.66:0/1471381436 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47380c9a20 msgr2=0x7f47380c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4737e00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47380c9a20 0x7f47380c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4737e00640 1 -- 172.21.15.66:0/1471381436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47381f4600 msgr2=0x7f47381f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4737e00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47381f4600 0x7f47381f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4737e00640 1 -- 172.21.15.66:0/1471381436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473815b7c0 con 0x7f471c003980 2024-03-30T00:43:17.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473cc00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47381f4600 0x7f47381f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:17.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4737e00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 0x7f47380b5290 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4728007040 tx=0x7f471c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:17.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4734c00640 1 -- 172.21.15.66:0/1471381436 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728004c80 con 0x7f471c003980 2024-03-30T00:43:17.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4734c00640 1 -- 172.21.15.66:0/1471381436 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47280b3070 con 0x7f471c003980 2024-03-30T00:43:17.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f4734c00640 1 -- 172.21.15.66:0/1471381436 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47280070d0 con 0x7f471c003980 2024-03-30T00:43:17.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473ec00640 1 -- 172.21.15.66:0/1471381436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47380ca3c0 con 0x7f471c003980 2024-03-30T00:43:17.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473ec00640 1 -- 172.21.15.66:0/1471381436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47380b57d0 con 0x7f471c003980 2024-03-30T00:43:17.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.220+0000 7f473ec00640 1 -- 172.21.15.66:0/1471381436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47380b59c0 con 0x7f471c003980 2024-03-30T00:43:17.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.224+0000 7f4734c00640 1 -- 172.21.15.66:0/1471381436 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4728007250 con 0x7f471c003980 2024-03-30T00:43:17.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.224+0000 7f4734c00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4714066690 0x7f4714068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:17.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.224+0000 7f4734c00640 1 -- 172.21.15.66:0/1471381436 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f47280b5070 con 0x7f471c003980 2024-03-30T00:43:17.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.224+0000 7f472f400640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4714066690 0x7f4714068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:17.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.224+0000 7f472f400640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4714066690 0x7f4714068b50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f473815e0f0 tx=0x7f4720008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:17.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.228+0000 7f4734c00640 1 -- 172.21.15.66:0/1471381436 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f47380b59c0 con 0x7f471c003980 2024-03-30T00:43:17.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.424+0000 7f473ec00640 1 -- 172.21.15.66:0/1471381436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f47380b59c0 con 0x7f471c003980 2024-03-30T00:43:17.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.424+0000 7f4734c00640 1 -- 172.21.15.66:0/1471381436 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7f47380b59c0 con 0x7f471c003980 2024-03-30T00:43:17.433 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:43:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 -- 172.21.15.66:0/1471381436 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4714066690 msgr2=0x7f4714068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4714066690 0x7f4714068b50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f473815e0f0 tx=0x7f4720008040 comp rx=0 tx=0).stop 2024-03-30T00:43:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 -- 172.21.15.66:0/1471381436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 msgr2=0x7f47380b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 0x7f47380b5290 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4728007040 tx=0x7f471c005800 comp rx=0 tx=0).stop 2024-03-30T00:43:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 -- 172.21.15.66:0/1471381436 shutdown_connections 2024-03-30T00:43:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47381f4600 0x7f47381f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47380c9a20 0x7f47380c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4714066690 0x7f4714068b50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 --2- 172.21.15.66:0/1471381436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f471c003980 0x7f47380b5290 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:17.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 -- 172.21.15.66:0/1471381436 >> 172.21.15.66:0/1471381436 conn(0x7f47380d0540 msgr2=0x7f473815a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:17.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 -- 172.21.15.66:0/1471381436 shutdown_connections 2024-03-30T00:43:17.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:17.428+0000 7f473ec00640 1 -- 172.21.15.66:0/1471381436 wait complete. 2024-03-30T00:43:17.454 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:43:17.454 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:43:17.454 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:43:17.454 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:43:17.454 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:43:17.454 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:43:17.454 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:43:17.454 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:43:17.455 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:43:17.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:17.455 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:43:17.464 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:43:17.465 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42281sec preferred_lft 42281sec 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:43:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:43:17.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:17.467 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:43:17.527 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:43:17.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:17.529 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:43:17.583 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:43:17.584 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:43:17.584 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:43:17.584 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:43:17.584 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:43:17.584 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:43:17.584 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:43:17.584 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:43:17.584 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:43:17.584 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:43:17.584 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:43:17.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:17.585 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:43:17.585 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:43:17.585 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:43:17.585 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:43:17.585 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:43:17.585 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:43:17.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:17.674 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:43:17.683 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:43:17.683 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:43:17.684 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:43:17.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:17.684 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:43:17.744 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:43:17.744 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:43:17.744 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:43:17.796 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:43:17.796 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:43:17.797 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:43:17.851 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:43:17.913 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:43:17.913 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:43:17.974 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:43:17.976 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:43:17.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:17.976 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:43:18.028 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:43:18.028 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:43:18.074 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:43:18.095 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:43:18.095 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:43:18.154 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:43:18.156 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:43:18.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:18.156 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:43:18.169 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:43:18.164+0000 7f23e1aa8580 -1 init, newargv = 0x55bccec20900 newargc=15 2024-03-30T00:43:18.169 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[42625]: starting ceph client 2024-03-30T00:43:18.177 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:43:18.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[42625]: starting fuse 2024-03-30T00:43:18.178 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:43:18.178 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:43:18.179 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:43:18.301 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.42625.asok 2024-03-30T00:43:18.485 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42625.asok 2024-03-30T00:43:18.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:18.485 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42625.asok status 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "42625", 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:43:18.664 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6597, 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 6597 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2759610883 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:43:18.665 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2759610883 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.6597 192.168.0.1:0/2759610883", 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/2759610883", 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 33, 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 68, 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:43:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:43:18.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:18.679 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:43:18.679 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:43:18.689 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:43:18.689 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:43:18.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:18.689 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:43:18.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:18.753 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-03-30T00:43:18.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:43:18.814 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-03-30T00:43:18.873 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:43:18.873 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:43:18.873 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:43:18.873 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:43:18.873 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-03-30T00:43:18.873 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:43:18.980 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627777 2024-03-30T00:43:19.170 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:43:19.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.344+0000 7f7ecee00640 1 Processor -- start 2024-03-30T00:43:19.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.344+0000 7f7ecee00640 1 -- start start 2024-03-30T00:43:19.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ecee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec815a640 0x7f7ec806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ecee00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7ec80cabb0 0x7f7ec80cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ecee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec806d070 con 0x7f7ec80ce520 2024-03-30T00:43:19.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ecee00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec80cbba0 con 0x7f7ec80cabb0 2024-03-30T00:43:19.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ecee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec806e2c0 con 0x7f7ec815a640 2024-03-30T00:43:19.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ebf400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7ec80ce520 0x7f7ec80ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:54396/0 (socket says 172.21.15.66:54396) 2024-03-30T00:43:19.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ebf400640 1 -- 172.21.15.66:0/1574456934 learned_addr learned my addr 172.21.15.66:0/1574456934 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7e00640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec815a640 0x7f7ec806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ecce00640 1 --2- 172.21.15.66:0/1574456934 >> v2:172.21.15.156:3300/0 conn(0x7f7ec80cabb0 0x7f7ec80cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387850449 0 0) 0x7f7ec806d070 con 0x7f7ec80ce520 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eac003530 con 0x7f7ec80ce520 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 93593584 0 0) 0x7f7eac003530 con 0x7f7ec80ce520 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec815a640 msgr2=0x7f7ec806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec815a640 0x7f7ec806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 >> v2:172.21.15.156:3300/0 conn(0x7f7ec80cabb0 msgr2=0x7f7ec80cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 --2- 172.21.15.66:0/1574456934 >> v2:172.21.15.156:3300/0 conn(0x7f7ec80cabb0 0x7f7ec80cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec80cb8c0 con 0x7f7ec80ce520 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f7eb0001680 con 0x7f7ec80ce520 2024-03-30T00:43:19.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 >> v1:172.21.15.66:6789/0 conn(0x7f7ec80ce520 legacy=0x7f7ec80ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac003980 0x7f7eac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec815a640 0x7f7eac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7eac004d10 0x7f7eac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80cbba0 con 0x7f7eac003980 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec806e2c0 con 0x7f7eac004d10 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac00b990 con 0x7f7ec815a640 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7e00640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac003980 0x7f7eac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ebf400640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec815a640 0x7f7eac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ecce00640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7eac004d10 0x7f7eac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7e00640 1 -- 172.21.15.66:0/1574456934 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec815a640 msgr2=0x7f7eac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7e00640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec815a640 0x7f7eac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7e00640 1 -- 172.21.15.66:0/1574456934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7eac004d10 msgr2=0x7f7eac005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7e00640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7eac004d10 0x7f7eac005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.348+0000 7f7ec7e00640 1 -- 172.21.15.66:0/1574456934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac003530 con 0x7f7eac003980 2024-03-30T00:43:19.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.352+0000 7f7ec7e00640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac003980 0x7f7eac003da0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009fc0 tx=0x7f7eb801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfd5074eb68e713 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:19.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.352+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80a5b50 con 0x7f7eac003980 2024-03-30T00:43:19.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.352+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb800f040 con 0x7f7eac003980 2024-03-30T00:43:19.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.352+0000 7f7ec7400640 1 -- 172.21.15.66:0/1574456934 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80ad440 con 0x7f7eac003980 2024-03-30T00:43:19.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.352+0000 7f7ecee00640 1 -- 172.21.15.66:0/1574456934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac003980 msgr2=0x7f7eac003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.352+0000 7f7ecee00640 1 --2- 172.21.15.66:0/1574456934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eac003980 0x7f7eac003da0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009fc0 tx=0x7f7eb801cc50 comp rx=0 tx=0).stop 2024-03-30T00:43:19.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.352+0000 7f7ecce00640 1 -- 172.21.15.66:0/1574456934 reap_dead start 2024-03-30T00:43:19.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.352+0000 7f7ecee00640 1 -- 172.21.15.66:0/1574456934 shutdown_connections 2024-03-30T00:43:19.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.352+0000 7f7ecee00640 1 -- 172.21.15.66:0/1574456934 >> 172.21.15.66:0/1574456934 conn(0x7f7ec8069f60 msgr2=0x7f7ec806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:19.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.352+0000 7f7ecee00640 1 -- 172.21.15.66:0/1574456934 shutdown_connections 2024-03-30T00:43:19.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.352+0000 7f7ecee00640 1 -- 172.21.15.66:0/1574456934 wait complete. 2024-03-30T00:43:19.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecee00640 1 Processor -- start 2024-03-30T00:43:19.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecee00640 1 -- start start 2024-03-30T00:43:19.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecee00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7eac003980 0x7f7ec81f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec80c9aa0 0x7f7ec80c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec8158920 0x7f7ec8158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80ca460 con 0x7f7ec8158920 2024-03-30T00:43:19.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecee00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec81f87d0 con 0x7f7eac003980 2024-03-30T00:43:19.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec81592e0 con 0x7f7ec80c9aa0 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec8158920 0x7f7ec8158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ec7e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7eac003980 0x7f7ec81f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec8158920 0x7f7ec8158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58414/0 (socket says 172.21.15.66:58414) 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecce00640 1 -- 172.21.15.66:0/4198353350 learned_addr learned my addr 172.21.15.66:0/4198353350 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ebf400640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec80c9aa0 0x7f7ec80c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecce00640 1 -- 172.21.15.66:0/4198353350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec80c9aa0 msgr2=0x7f7ec80c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecce00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec80c9aa0 0x7f7ec80c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecce00640 1 -- 172.21.15.66:0/4198353350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7eac003980 msgr2=0x7f7ec81f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecce00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7eac003980 0x7f7ec81f8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecce00640 1 -- 172.21.15.66:0/4198353350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec80ccde0 con 0x7f7ec8158920 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ec7e00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7eac003980 0x7f7ec81f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecce00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec8158920 0x7f7ec8158da0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80b17e0 tx=0x7f7ec0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:19.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ec4c00640 1 -- 172.21.15.66:0/4198353350 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0004500 con 0x7f7ec8158920 2024-03-30T00:43:19.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ec4c00640 1 -- 172.21.15.66:0/4198353350 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec0003620 con 0x7f7ec8158920 2024-03-30T00:43:19.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ec4c00640 1 -- 172.21.15.66:0/4198353350 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0003a10 con 0x7f7ec8158920 2024-03-30T00:43:19.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecee00640 1 -- 172.21.15.66:0/4198353350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec80cd060 con 0x7f7ec8158920 2024-03-30T00:43:19.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecee00640 1 -- 172.21.15.66:0/4198353350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec80cd620 con 0x7f7ec8158920 2024-03-30T00:43:19.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.356+0000 7f7ecee00640 1 -- 172.21.15.66:0/4198353350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec8066c30 con 0x7f7ec8158920 2024-03-30T00:43:19.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.360+0000 7f7ec4c00640 1 -- 172.21.15.66:0/4198353350 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7ec0003b70 con 0x7f7ec8158920 2024-03-30T00:43:19.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.360+0000 7f7ec4c00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea40666c0 0x7f7ea4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.360+0000 7f7ec4c00640 1 -- 172.21.15.66:0/4198353350 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7ec0084580 con 0x7f7ec8158920 2024-03-30T00:43:19.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.360+0000 7f7ec7e00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea40666c0 0x7f7ea4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.360+0000 7f7ec7e00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea40666c0 0x7f7ea4068b80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80af000 tx=0x7f7eb80a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:19.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.364+0000 7f7ec4c00640 1 -- 172.21.15.66:0/4198353350 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7ec0051850 con 0x7f7ec8158920 2024-03-30T00:43:19.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.560+0000 7f7ecee00640 1 -- 172.21.15.66:0/4198353350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ec80cb550 con 0x7f7ec8158920 2024-03-30T00:43:19.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.560+0000 7f7ec4c00640 1 -- 172.21.15.66:0/4198353350 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f7ec0054e70 con 0x7f7ec8158920 2024-03-30T00:43:19.567 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:19.567 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4272961638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4272961638},{"type":"v1","addr":"172.21.15.156:6834","nonce":4272961638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2144186622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2144186622},{"type":"v1","addr":"172.21.15.66:6835","nonce":2144186622}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/750308788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":750308788},{"type":"v1","addr":"172.21.15.66:6837","nonce":750308788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:43:11.141056+0000","modified":"2024-03-30T00:43:14.154117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6468":{"gid":6468,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/1137593870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1137593870},{"type":"v1","addr":"172.21.15.156:6835","nonce":1137593870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-30T00:43:19.567 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 33 2024-03-30T00:43:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 -- 172.21.15.66:0/4198353350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea40666c0 msgr2=0x7f7ea4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea40666c0 0x7f7ea4068b80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80af000 tx=0x7f7eb80a7040 comp rx=0 tx=0).stop 2024-03-30T00:43:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 -- 172.21.15.66:0/4198353350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec8158920 msgr2=0x7f7ec8158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec8158920 0x7f7ec8158da0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80b17e0 tx=0x7f7ec0011920 comp rx=0 tx=0).stop 2024-03-30T00:43:19.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 -- 172.21.15.66:0/4198353350 shutdown_connections 2024-03-30T00:43:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ea40666c0 0x7f7ea4068b80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec8158920 0x7f7ec8158da0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ec80c9aa0 0x7f7ec80c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 --2- 172.21.15.66:0/4198353350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7eac003980 0x7f7ec81f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 -- 172.21.15.66:0/4198353350 >> 172.21.15.66:0/4198353350 conn(0x7f7ec8069f60 msgr2=0x7f7ec80aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 -- 172.21.15.66:0/4198353350 shutdown_connections 2024-03-30T00:43:19.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.564+0000 7f7ecee00640 1 -- 172.21.15.66:0/4198353350 wait complete. 2024-03-30T00:43:19.588 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-03-30T00:43:19.758 INFO:teuthology.orchestra.run.smithi156.stdout:{"message":"","return_code":0} 2024-03-30T00:43:19.759 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-30T00:43:19.759 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-30T00:43:19.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 Processor -- start 2024-03-30T00:43:19.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 -- start start 2024-03-30T00:43:19.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024910 0x560f3a0b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3a024cf0 0x560f3a093540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560f3a094290 0x560f3a007cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f39f20960 con 0x560f3a024910 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f39f20600 con 0x560f3a024cf0 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f39f17690 con 0x560f3a094290 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c45200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024910 0x560f3a0b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c45200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024910 0x560f3a0b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58432/0 (socket says 172.21.15.66:58432) 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c45200640 1 -- 172.21.15.66:0/4159338169 learned_addr learned my addr 172.21.15.66:0/4159338169 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c45c00640 1 --2- 172.21.15.66:0/4159338169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560f3a094290 0x560f3a007cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c3fe00640 1 --2- 172.21.15.66:0/4159338169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3a024cf0 0x560f3a093540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c45200640 1 -- 172.21.15.66:0/4159338169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560f3a094290 msgr2=0x560f3a007cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c45200640 1 --2- 172.21.15.66:0/4159338169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560f3a094290 0x560f3a007cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c45200640 1 -- 172.21.15.66:0/4159338169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3a024cf0 msgr2=0x560f3a093540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c45200640 1 --2- 172.21.15.66:0/4159338169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3a024cf0 0x560f3a093540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c45200640 1 -- 172.21.15.66:0/4159338169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4000a100 con 0x560f3a024910 2024-03-30T00:43:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c45c00640 1 --2- 172.21.15.66:0/4159338169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560f3a094290 0x560f3a007cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c45200640 1 --2- 172.21.15.66:0/4159338169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024910 0x560f3a0b0100 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0c38001ce0 tx=0x7f0c3800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239b92ef67296de8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c3ea00640 1 -- 172.21.15.66:0/4159338169 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800e480 con 0x560f3a024910 2024-03-30T00:43:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c3ea00640 1 -- 172.21.15.66:0/4159338169 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3800e5e0 con 0x560f3a024910 2024-03-30T00:43:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c3ea00640 1 -- 172.21.15.66:0/4159338169 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800e970 con 0x560f3a024910 2024-03-30T00:43:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 -- 172.21.15.66:0/4159338169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024910 msgr2=0x560f3a0b0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 --2- 172.21.15.66:0/4159338169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024910 0x560f3a0b0100 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0c38001ce0 tx=0x7f0c3800b6d0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 -- 172.21.15.66:0/4159338169 shutdown_connections 2024-03-30T00:43:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 --2- 172.21.15.66:0/4159338169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560f3a094290 0x560f3a007cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 --2- 172.21.15.66:0/4159338169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3a024cf0 0x560f3a093540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 --2- 172.21.15.66:0/4159338169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024910 0x560f3a0b0100 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 -- 172.21.15.66:0/4159338169 >> 172.21.15.66:0/4159338169 conn(0x560f3a0092f0 msgr2=0x560f3a0af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:19.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 -- 172.21.15.66:0/4159338169 shutdown_connections 2024-03-30T00:43:19.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 -- 172.21.15.66:0/4159338169 wait complete. 2024-03-30T00:43:19.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 Processor -- start 2024-03-30T00:43:19.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.800+0000 7f0c48021080 1 -- start start 2024-03-30T00:43:19.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c48021080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3a024910 0x560f3a0b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c48021080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024cf0 0x560f3a09c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c48021080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560f3a094290 0x560f3a09c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c48021080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3a0b1780 con 0x560f3a024cf0 2024-03-30T00:43:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c48021080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3a09b180 con 0x560f3a024910 2024-03-30T00:43:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c48021080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f3a09b420 con 0x560f3a094290 2024-03-30T00:43:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024cf0 0x560f3a09c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024cf0 0x560f3a09c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58442/0 (socket says 172.21.15.66:58442) 2024-03-30T00:43:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3fe00640 1 -- 172.21.15.66:0/2133924488 learned_addr learned my addr 172.21.15.66:0/2133924488 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c45200640 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3a024910 0x560f3a0b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c45c00640 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560f3a094290 0x560f3a09c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3fe00640 1 -- 172.21.15.66:0/2133924488 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560f3a094290 msgr2=0x560f3a09c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3fe00640 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560f3a094290 0x560f3a09c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3fe00640 1 -- 172.21.15.66:0/2133924488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3a024910 msgr2=0x560f3a0b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3fe00640 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f3a024910 0x560f3a0b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3fe00640 1 -- 172.21.15.66:0/2133924488 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f39f22970 con 0x560f3a024cf0 2024-03-30T00:43:19.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3fe00640 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024cf0 0x560f3a09c0a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0c300086f0 tx=0x7f0c30008bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:19.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3d600640 1 -- 172.21.15.66:0/2133924488 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30006b80 con 0x560f3a024cf0 2024-03-30T00:43:19.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3d600640 1 -- 172.21.15.66:0/2133924488 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c30010040 con 0x560f3a024cf0 2024-03-30T00:43:19.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c48021080 1 -- 172.21.15.66:0/2133924488 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f39f2f8c0 con 0x560f3a024cf0 2024-03-30T00:43:19.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3d600640 1 -- 172.21.15.66:0/2133924488 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c300143f0 con 0x560f3a024cf0 2024-03-30T00:43:19.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c48021080 1 -- 172.21.15.66:0/2133924488 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f39f22d00 con 0x560f3a024cf0 2024-03-30T00:43:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.804+0000 7f0c3d600640 1 -- 172.21.15.66:0/2133924488 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0c30014550 con 0x560f3a024cf0 2024-03-30T00:43:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c3d600640 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0c240650a0 0x7f0c24067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c3d600640 1 -- 172.21.15.66:0/2133924488 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0c30082c20 con 0x560f3a024cf0 2024-03-30T00:43:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c45200640 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0c240650a0 0x7f0c24067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x560f3a09ac00 0x560f3a0b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 -- 172.21.15.66:0/2133924488 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68) v8 -- 0x560f3a0b30c0 con 0x560f3a09ac00 2024-03-30T00:43:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c45200640 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0c240650a0 0x7f0c24067560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0c38001280 tx=0x7f0c3801b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c45c00640 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x560f3a09ac00 0x560f3a0b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c45c00640 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x560f3a09ac00 0x560f3a0b2b80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:19.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c45c00640 1 -- 172.21.15.66:0/2133924488 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f0c400af520 con 0x560f3a09ac00 2024-03-30T00:43:19.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 -- 172.21.15.66:0/2133924488 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x560f3a09ac00 msgr2=0x560f3a0b2b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x560f3a09ac00 0x560f3a0b2b80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 -- 172.21.15.66:0/2133924488 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0c240650a0 msgr2=0x7f0c24067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0c240650a0 0x7f0c24067560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0c38001280 tx=0x7f0c3801b040 comp rx=0 tx=0).stop 2024-03-30T00:43:19.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 -- 172.21.15.66:0/2133924488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024cf0 msgr2=0x560f3a09c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 --2- 172.21.15.66:0/2133924488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f3a024cf0 0x560f3a09c0a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0c300086f0 tx=0x7f0c30008bb0 comp rx=0 tx=0).stop 2024-03-30T00:43:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c45c00640 1 -- 172.21.15.66:0/2133924488 reap_dead start 2024-03-30T00:43:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 -- 172.21.15.66:0/2133924488 shutdown_connections 2024-03-30T00:43:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 -- 172.21.15.66:0/2133924488 >> 172.21.15.66:0/2133924488 conn(0x560f3a0092f0 msgr2=0x560f3a09a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 -- 172.21.15.66:0/2133924488 shutdown_connections 2024-03-30T00:43:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:19.808+0000 7f0c48021080 1 -- 172.21.15.66:0/2133924488 wait complete. 2024-03-30T00:43:19.821 INFO:teuthology.orchestra.run.smithi066.stdout:Xfiledir_test 2024-03-30T00:43:19.821 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:19.822 DEBUG:teuthology.orchestra.run.smithi066:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-30T00:43:19.913 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:43:19.913 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627777, 2024-03-30T00:43:19.913 INFO:teuthology.orchestra.run.smithi066.stdout: "ancestors": [ 2024-03-30T00:43:19.913 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:19.913 INFO:teuthology.orchestra.run.smithi066.stdout: "dirino": 1099511627776, 2024-03-30T00:43:19.913 INFO:teuthology.orchestra.run.smithi066.stdout: "dname": "file", 2024-03-30T00:43:19.913 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 2 2024-03-30T00:43:19.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:43:19.914 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:19.914 INFO:teuthology.orchestra.run.smithi066.stdout: "dirino": 1, 2024-03-30T00:43:19.914 INFO:teuthology.orchestra.run.smithi066.stdout: "dname": "dir_test", 2024-03-30T00:43:19.914 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 4 2024-03-30T00:43:19.914 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:19.914 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T00:43:19.914 INFO:teuthology.orchestra.run.smithi066.stdout: "pool": 10, 2024-03-30T00:43:19.914 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [] 2024-03-30T00:43:19.914 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:43:19.914 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:19.917 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-03-30T00:43:20.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.000+0000 7f7f4a536080 1 Processor -- start 2024-03-30T00:43:20.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.000+0000 7f7f4a536080 1 -- start start 2024-03-30T00:43:20.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc1816e40 0x555cc17e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555cc18a6100 0x555cc17e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cc18a64e0 0x555cc182bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc17931e0 con 0x555cc1816e40 2024-03-30T00:43:20.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc172e960 con 0x555cc18a64e0 2024-03-30T00:43:20.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cc172e600 con 0x555cc18a6100 2024-03-30T00:43:20.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f47800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc1816e40 0x555cc17e7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f47800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc1816e40 0x555cc17e7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58456/0 (socket says 172.21.15.66:58456) 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f47800640 1 -- 172.21.15.66:0/1857873874 learned_addr learned my addr 172.21.15.66:0/1857873874 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f48200640 1 --2- 172.21.15.66:0/1857873874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cc18a64e0 0x555cc182bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f46e00640 1 --2- 172.21.15.66:0/1857873874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555cc18a6100 0x555cc17e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f47800640 1 -- 172.21.15.66:0/1857873874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555cc18a6100 msgr2=0x555cc17e9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f47800640 1 --2- 172.21.15.66:0/1857873874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555cc18a6100 0x555cc17e9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f47800640 1 -- 172.21.15.66:0/1857873874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cc18a64e0 msgr2=0x555cc182bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f47800640 1 --2- 172.21.15.66:0/1857873874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cc18a64e0 0x555cc182bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f47800640 1 -- 172.21.15.66:0/1857873874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f34005df0 con 0x555cc1816e40 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f46e00640 1 --2- 172.21.15.66:0/1857873874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555cc18a6100 0x555cc17e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f47800640 1 --2- 172.21.15.66:0/1857873874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc1816e40 0x555cc17e7fb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7f38001df0 tx=0x7f7f3800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f777357b88ec7f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f45a00640 1 -- 172.21.15.66:0/1857873874 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38014070 con 0x555cc1816e40 2024-03-30T00:43:20.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f45a00640 1 -- 172.21.15.66:0/1857873874 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3800f410 con 0x555cc1816e40 2024-03-30T00:43:20.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f45a00640 1 -- 172.21.15.66:0/1857873874 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800f7a0 con 0x555cc1816e40 2024-03-30T00:43:20.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 -- 172.21.15.66:0/1857873874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc1816e40 msgr2=0x555cc17e7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 --2- 172.21.15.66:0/1857873874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc1816e40 0x555cc17e7fb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7f38001df0 tx=0x7f7f3800b5c0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 -- 172.21.15.66:0/1857873874 shutdown_connections 2024-03-30T00:43:20.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 --2- 172.21.15.66:0/1857873874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cc18a64e0 0x555cc182bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 --2- 172.21.15.66:0/1857873874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555cc18a6100 0x555cc17e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 --2- 172.21.15.66:0/1857873874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc1816e40 0x555cc17e7fb0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 -- 172.21.15.66:0/1857873874 >> 172.21.15.66:0/1857873874 conn(0x555cc1811dd0 msgr2=0x555cc1815f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 -- 172.21.15.66:0/1857873874 shutdown_connections 2024-03-30T00:43:20.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 -- 172.21.15.66:0/1857873874 wait complete. 2024-03-30T00:43:20.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 Processor -- start 2024-03-30T00:43:20.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.004+0000 7f7f4a536080 1 -- start start 2024-03-30T00:43:20.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f4a536080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cc1816e40 0x555cc1829860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f4a536080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc18a6100 0x555cc18a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f4a536080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555cc18a64e0 0x555cc18a3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f4a536080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc1725690 con 0x555cc18a6100 2024-03-30T00:43:20.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f4a536080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc182d690 con 0x555cc1816e40 2024-03-30T00:43:20.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f4a536080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cc18a3880 con 0x555cc18a64e0 2024-03-30T00:43:20.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f46e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc18a6100 0x555cc18a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f47800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cc1816e40 0x555cc1829860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f46e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc18a6100 0x555cc18a2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58466/0 (socket says 172.21.15.66:58466) 2024-03-30T00:43:20.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f46e00640 1 -- 172.21.15.66:0/1530990169 learned_addr learned my addr 172.21.15.66:0/1530990169 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f48200640 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555cc18a64e0 0x555cc18a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f46e00640 1 -- 172.21.15.66:0/1530990169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555cc18a64e0 msgr2=0x555cc18a3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f46e00640 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555cc18a64e0 0x555cc18a3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f46e00640 1 -- 172.21.15.66:0/1530990169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cc1816e40 msgr2=0x555cc1829860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f46e00640 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cc1816e40 0x555cc1829860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f46e00640 1 -- 172.21.15.66:0/1530990169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cc1730970 con 0x555cc18a6100 2024-03-30T00:43:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f46e00640 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc18a6100 0x555cc18a2d00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7f34019480 tx=0x7f7f340185f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f3fe00640 1 -- 172.21.15.66:0/1530990169 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f340ac990 con 0x555cc18a6100 2024-03-30T00:43:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f3fe00640 1 -- 172.21.15.66:0/1530990169 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f340acaf0 con 0x555cc18a6100 2024-03-30T00:43:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f4a536080 1 -- 172.21.15.66:0/1530990169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555cc1730d00 con 0x555cc18a6100 2024-03-30T00:43:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f3fe00640 1 -- 172.21.15.66:0/1530990169 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f340ace60 con 0x555cc18a6100 2024-03-30T00:43:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f4a536080 1 -- 172.21.15.66:0/1530990169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cc1730760 con 0x555cc18a6100 2024-03-30T00:43:20.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.008+0000 7f7f3fe00640 1 -- 172.21.15.66:0/1530990169 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7f340b3050 con 0x555cc18a6100 2024-03-30T00:43:20.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f3fe00640 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7f280650a0 0x7f7f28067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f3fe00640 1 -- 172.21.15.66:0/1530990169 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7f3411e320 con 0x555cc18a6100 2024-03-30T00:43:20.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f47800640 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7f280650a0 0x7f7f28067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x555cc1829da0 0x555cc1961340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 -- 172.21.15.66:0/1530990169 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68) v8 -- 0x555cc18c1400 con 0x555cc1829da0 2024-03-30T00:43:20.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f48200640 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x555cc1829da0 0x555cc1961340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f47800640 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7f280650a0 0x7f7f28067560 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800b600 tx=0x7f7f38009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f48200640 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x555cc1829da0 0x555cc1961340 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f48200640 1 -- 172.21.15.66:0/1530990169 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v68'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7f4000d850 con 0x555cc1829da0 2024-03-30T00:43:20.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 -- 172.21.15.66:0/1530990169 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x555cc1829da0 msgr2=0x555cc1961340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x555cc1829da0 0x555cc1961340 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 -- 172.21.15.66:0/1530990169 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7f280650a0 msgr2=0x7f7f28067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7f280650a0 0x7f7f28067560 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800b600 tx=0x7f7f38009000 comp rx=0 tx=0).stop 2024-03-30T00:43:20.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 -- 172.21.15.66:0/1530990169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc18a6100 msgr2=0x555cc18a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 --2- 172.21.15.66:0/1530990169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc18a6100 0x555cc18a2d00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7f34019480 tx=0x7f7f340185f0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f48200640 1 -- 172.21.15.66:0/1530990169 reap_dead start 2024-03-30T00:43:20.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 -- 172.21.15.66:0/1530990169 shutdown_connections 2024-03-30T00:43:20.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 -- 172.21.15.66:0/1530990169 >> 172.21.15.66:0/1530990169 conn(0x555cc1811dd0 msgr2=0x555cc18a0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 -- 172.21.15.66:0/1530990169 shutdown_connections 2024-03-30T00:43:20.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.012+0000 7f7f4a536080 1 -- 172.21.15.66:0/1530990169 wait complete. 2024-03-30T00:43:20.024 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-03-30T00:43:20.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.204+0000 7f1026200640 1 Processor -- start 2024-03-30T00:43:20.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.204+0000 7f1026200640 1 -- start start 2024-03-30T00:43:20.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f1026200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f102015ba00 0x7f10200ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f1026200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f10200cecf0 0x7f10201579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f1026200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10200cbf80 con 0x7f10200acaf0 2024-03-30T00:43:20.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f1026200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10200b2be0 con 0x7f10200cecf0 2024-03-30T00:43:20.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f1026200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10200cc820 con 0x7f102015ba00 2024-03-30T00:43:20.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f101f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f10200acaf0 0x7f10200b24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:54432/0 (socket says 172.21.15.66:54432) 2024-03-30T00:43:20.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f101f400640 1 -- 172.21.15.66:0/2931031766 learned_addr learned my addr 172.21.15.66:0/2931031766 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:20.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f101fe00640 1 --2- 172.21.15.66:0/2931031766 >> v2:172.21.15.66:3300/0 conn(0x7f10200cecf0 0x7f10201579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f1017e00640 1 --2- 172.21.15.66:0/2931031766 >> v2:172.21.15.156:3300/0 conn(0x7f102015ba00 0x7f10200ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f101ea00640 1 -- 172.21.15.66:0/2931031766 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200091601 0 0) 0x7f10200cbf80 con 0x7f10200acaf0 2024-03-30T00:43:20.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f101ea00640 1 -- 172.21.15.66:0/2931031766 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1004003530 con 0x7f10200acaf0 2024-03-30T00:43:20.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f101fe00640 1 -- 172.21.15.66:0/2931031766 >> v2:172.21.15.156:3300/0 conn(0x7f102015ba00 msgr2=0x7f10200ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f101fe00640 1 --2- 172.21.15.66:0/2931031766 >> v2:172.21.15.156:3300/0 conn(0x7f102015ba00 0x7f10200ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f101fe00640 1 -- 172.21.15.66:0/2931031766 >> v1:172.21.15.66:6789/0 conn(0x7f10200acaf0 legacy=0x7f10200b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f101fe00640 1 -- 172.21.15.66:0/2931031766 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10200cf110 con 0x7f10200cecf0 2024-03-30T00:43:20.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.208+0000 7f101fe00640 1 --2- 172.21.15.66:0/2931031766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10200cecf0 0x7f10201579e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f10200c1e10 tx=0x7f101801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e316ec645f9b3bfa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f101ea00640 1 -- 172.21.15.66:0/2931031766 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180a5930 con 0x7f10200cecf0 2024-03-30T00:43:20.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f101ea00640 1 -- 172.21.15.66:0/2931031766 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10180a5a90 con 0x7f10200cecf0 2024-03-30T00:43:20.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f101ea00640 1 -- 172.21.15.66:0/2931031766 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180a5e20 con 0x7f10200cecf0 2024-03-30T00:43:20.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 -- 172.21.15.66:0/2931031766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10200cecf0 msgr2=0x7f10201579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 --2- 172.21.15.66:0/2931031766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10200cecf0 0x7f10201579e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f10200c1e10 tx=0x7f101801c870 comp rx=0 tx=0).stop 2024-03-30T00:43:20.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 -- 172.21.15.66:0/2931031766 shutdown_connections 2024-03-30T00:43:20.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 --2- 172.21.15.66:0/2931031766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10200cecf0 0x7f10201579e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 --2- 172.21.15.66:0/2931031766 >> v2:172.21.15.156:3300/0 conn(0x7f102015ba00 0x7f10200ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 -- 172.21.15.66:0/2931031766 >> 172.21.15.66:0/2931031766 conn(0x7f102006a190 msgr2=0x7f102006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 -- 172.21.15.66:0/2931031766 shutdown_connections 2024-03-30T00:43:20.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 -- 172.21.15.66:0/2931031766 wait complete. 2024-03-30T00:43:20.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 Processor -- start 2024-03-30T00:43:20.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 -- start start 2024-03-30T00:43:20.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10200acaf0 0x7f1020157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10200cecf0 0x7f10200ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f102015ba00 0x7f10200aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10200c9910 con 0x7f10200cecf0 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10200c7280 con 0x7f10200acaf0 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1026200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10200c7560 con 0x7f102015ba00 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1017e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10200cecf0 0x7f10200ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1017e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10200cecf0 0x7f10200ae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58482/0 (socket says 172.21.15.66:58482) 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1017e00640 1 -- 172.21.15.66:0/2864383969 learned_addr learned my addr 172.21.15.66:0/2864383969 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f101fe00640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f102015ba00 0x7f10200aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f101f400640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10200acaf0 0x7f1020157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1017e00640 1 -- 172.21.15.66:0/2864383969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f102015ba00 msgr2=0x7f10200aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1017e00640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f102015ba00 0x7f10200aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1017e00640 1 -- 172.21.15.66:0/2864383969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10200acaf0 msgr2=0x7f1020157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.212+0000 7f1017e00640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10200acaf0 0x7f1020157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1017e00640 1 -- 172.21.15.66:0/2864383969 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1018009a70 con 0x7f10200cecf0 2024-03-30T00:43:20.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1017e00640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10200cecf0 0x7f10200ae570 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f100800e900 tx=0x7f100800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1017400640 1 -- 172.21.15.66:0/2864383969 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008009690 con 0x7f10200cecf0 2024-03-30T00:43:20.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1017400640 1 -- 172.21.15.66:0/2864383969 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100800ca00 con 0x7f10200cecf0 2024-03-30T00:43:20.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1026200640 1 -- 172.21.15.66:0/2864383969 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10200c7840 con 0x7f10200cecf0 2024-03-30T00:43:20.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1017400640 1 -- 172.21.15.66:0/2864383969 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100800cdb0 con 0x7f10200cecf0 2024-03-30T00:43:20.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1026200640 1 -- 172.21.15.66:0/2864383969 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10200c64c0 con 0x7f10200cecf0 2024-03-30T00:43:20.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1017400640 1 -- 172.21.15.66:0/2864383969 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1008013470 con 0x7f10200cecf0 2024-03-30T00:43:20.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1017400640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ffc066690 0x7f0ffc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1017400640 1 -- 172.21.15.66:0/2864383969 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f10080844e0 con 0x7f10200cecf0 2024-03-30T00:43:20.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f101f400640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ffc066690 0x7f0ffc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1029634000 1 Processor -- start 2024-03-30T00:43:20.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.216+0000 7f1029634000 1 -- start start 2024-03-30T00:43:20.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f101f400640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ffc066690 0x7f0ffc068b50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f10100011f0 tx=0x7f1010007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab5820 0x5649faab5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab6140 0x5649faa6c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faa6ce50 0x5649faa6d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fa935100 con 0x5649faab5820 2024-03-30T00:43:20.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fa8c2d10 con 0x5649faab6140 2024-03-30T00:43:20.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649fa8c02e0 con 0x5649faa6ce50 2024-03-30T00:43:20.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f101f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab5820 0x5649faab5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f101f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab5820 0x5649faab5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58492/0 (socket says 172.21.15.66:58492) 2024-03-30T00:43:20.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f101f400640 1 -- 172.21.15.66:0/3654699894 learned_addr learned my addr 172.21.15.66:0/3654699894 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f101fe00640 1 --2- 172.21.15.66:0/3654699894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faa6ce50 0x5649faa6d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1017e00640 1 --2- 172.21.15.66:0/3654699894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab6140 0x5649faa6c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f101f400640 1 -- 172.21.15.66:0/3654699894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faa6ce50 msgr2=0x5649faa6d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f101f400640 1 --2- 172.21.15.66:0/3654699894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faa6ce50 0x5649faa6d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f101f400640 1 -- 172.21.15.66:0/3654699894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab6140 msgr2=0x5649faa6c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f101f400640 1 --2- 172.21.15.66:0/3654699894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab6140 0x5649faa6c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f101f400640 1 -- 172.21.15.66:0/3654699894 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649fa5b7fb0 con 0x5649faab5820 2024-03-30T00:43:20.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f101f400640 1 --2- 172.21.15.66:0/3654699894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab5820 0x5649faab5c00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1010006a00 tx=0x7f1010012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2c51d5beb2938c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f100fe00640 1 -- 172.21.15.66:0/3654699894 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010001450 con 0x5649faab5820 2024-03-30T00:43:20.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f100fe00640 1 -- 172.21.15.66:0/3654699894 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10100015b0 con 0x5649faab5820 2024-03-30T00:43:20.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f100fe00640 1 -- 172.21.15.66:0/3654699894 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010001960 con 0x5649faab5820 2024-03-30T00:43:20.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 -- 172.21.15.66:0/3654699894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab5820 msgr2=0x5649faab5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 --2- 172.21.15.66:0/3654699894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab5820 0x5649faab5c00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1010006a00 tx=0x7f1010012000 comp rx=0 tx=0).stop 2024-03-30T00:43:20.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 -- 172.21.15.66:0/3654699894 shutdown_connections 2024-03-30T00:43:20.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 --2- 172.21.15.66:0/3654699894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faa6ce50 0x5649faa6d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 --2- 172.21.15.66:0/3654699894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab6140 0x5649faa6c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 --2- 172.21.15.66:0/3654699894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab5820 0x5649faab5c00 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 -- 172.21.15.66:0/3654699894 >> 172.21.15.66:0/3654699894 conn(0x5649fa86b280 msgr2=0x5649faa6b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 -- 172.21.15.66:0/3654699894 shutdown_connections 2024-03-30T00:43:20.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 -- 172.21.15.66:0/3654699894 wait complete. 2024-03-30T00:43:20.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 -- start start 2024-03-30T00:43:20.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.220+0000 7f1029634000 1 Processor -- start 2024-03-30T00:43:20.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faa779d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab5820 0x5649faabdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab6140 0x5649faabe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fa8bbbd0 con 0x5649faa6ce50 2024-03-30T00:43:20.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fa85af60 con 0x5649faab5820 2024-03-30T00:43:20.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649fa879180 con 0x5649faab6140 2024-03-30T00:43:20.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 10 client.0 _ll_register_callbacks cb 0x5649fa8b5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:20.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f101f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faa779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f101f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faa779d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58494/0 (socket says 172.21.15.66:58494) 2024-03-30T00:43:20.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f101f400640 1 -- 172.21.15.66:0/4100840140 learned_addr learned my addr 172.21.15.66:0/4100840140 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1017e00640 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab5820 0x5649faabdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f101f400640 1 -- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab6140 msgr2=0x5649faabe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f101f400640 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab6140 0x5649faabe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f101f400640 1 -- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab5820 msgr2=0x5649faabdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f101f400640 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab5820 0x5649faabdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f101f400640 1 -- 172.21.15.66:0/4100840140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1004003530 con 0x5649faa6ce50 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1017e00640 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab5820 0x5649faabdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f101f400640 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faa779d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f10100069d0 tx=0x7f1010000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8130199ef530e85 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f100cc00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f100cc00640 1 -- 172.21.15.66:0/4100840140 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010001d00 con 0x5649faa6ce50 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f100cc00640 1 -- 172.21.15.66:0/4100840140 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f101001b050 con 0x5649faa6ce50 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 1 -- 172.21.15.66:0/4100840140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649fa88b850 con 0x5649faa6ce50 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f100cc00640 1 -- 172.21.15.66:0/4100840140 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f101002d430 con 0x5649faa6ce50 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 10 client.6642 fetch_fsmap learned FSMap version 33 2024-03-30T00:43:20.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 1 -- 172.21.15.66:0/4100840140 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5649fa880bc0 con 0x5649faa6ce50 2024-03-30T00:43:20.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f100cc00640 1 -- 172.21.15.66:0/4100840140 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5649fa88b850 con 0x5649faa6ce50 2024-03-30T00:43:20.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f100cc00640 1 -- 172.21.15.66:0/4100840140 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1010028070 con 0x5649faa6ce50 2024-03-30T00:43:20.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 10 client.6642 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T00:43:20.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 10 client.6642 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-30T00:43:20.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 20 client.6642 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:20.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5649faa72ae0 0x5649faa72ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 4 client.6642 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:43:20.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1029634000 1 -- 172.21.15.66:0/4100840140 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649fa8b89b0 con 0x5649faa72ae0 2024-03-30T00:43:20.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1017e00640 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5649faa72ae0 0x5649faa72ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f1017e00640 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5649faa72ae0 0x5649faa72ec0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a875f9cf1105dd7 server_cookie=7596a4c393dd6bdf in_seq=0 out_seq=0 2024-03-30T00:43:20.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.224+0000 7f100cc00640 10 client.6642 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:20.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f100cc00640 1 -- 172.21.15.66:0/4100840140 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5649fa8b89b0 con 0x5649faa72ae0 2024-03-30T00:43:20.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f100cc00640 10 client.6642 handle_command_reply: tid=0 2024-03-30T00:43:20.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 1 client.6642 shutdown 2024-03-30T00:43:20.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 1 -- 172.21.15.66:0/4100840140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 msgr2=0x5649faa779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faa779d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f10100069d0 tx=0x7f1010000ee0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 1 -- 172.21.15.66:0/4100840140 shutdown_connections 2024-03-30T00:43:20.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5649faa72ae0 0x5649faa72ec0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab6140 0x5649faabe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab5820 0x5649faabdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 1 --2- 172.21.15.66:0/4100840140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faa779d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f100cc00640 0 client.6642 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:20.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 1 -- 172.21.15.66:0/4100840140 >> 172.21.15.66:0/4100840140 conn(0x5649fa86b280 msgr2=0x5649faa6b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 1 -- 172.21.15.66:0/4100840140 shutdown_connections 2024-03-30T00:43:20.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 1 -- 172.21.15.66:0/4100840140 wait complete. 2024-03-30T00:43:20.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.228+0000 7f1029634000 20 client.6642 trim_cache size 0 max 16384 2024-03-30T00:43:20.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.240+0000 7f1029634000 1 Processor -- start 2024-03-30T00:43:20.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.240+0000 7f1029634000 1 -- start start 2024-03-30T00:43:20.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faa6ce50 0x5649fab0f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab5820 0x5649fab0ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab6140 0x5649fab10440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fa578c30 con 0x5649faab6140 2024-03-30T00:43:20.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fa88b350 con 0x5649faa6ce50 2024-03-30T00:43:20.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649fa8ff2b0 con 0x5649faab5820 2024-03-30T00:43:20.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f101fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab6140 0x5649fab10440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f101fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab6140 0x5649fab10440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58500/0 (socket says 172.21.15.66:58500) 2024-03-30T00:43:20.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f101fe00640 1 -- 172.21.15.66:0/403214328 learned_addr learned my addr 172.21.15.66:0/403214328 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1017e00640 1 --2- 172.21.15.66:0/403214328 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab5820 0x5649fab0ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f101f400640 1 --2- 172.21.15.66:0/403214328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faa6ce50 0x5649fab0f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f101fe00640 1 -- 172.21.15.66:0/403214328 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab5820 msgr2=0x5649fab0ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f101fe00640 1 --2- 172.21.15.66:0/403214328 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab5820 0x5649fab0ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f101fe00640 1 -- 172.21.15.66:0/403214328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faa6ce50 msgr2=0x5649fab0f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f101fe00640 1 --2- 172.21.15.66:0/403214328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faa6ce50 0x5649fab0f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f101fe00640 1 -- 172.21.15.66:0/403214328 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649fa866f10 con 0x5649faab6140 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f101fe00640 1 --2- 172.21.15.66:0/403214328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab6140 0x5649fab10440 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f101801cd50 tx=0x7f1018004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6bd0488cc0a478 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1014c00640 1 -- 172.21.15.66:0/403214328 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180ad640 con 0x5649faab6140 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1014c00640 1 -- 172.21.15.66:0/403214328 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10180ad7a0 con 0x5649faab6140 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1014c00640 1 -- 172.21.15.66:0/403214328 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180adb30 con 0x5649faab6140 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- 172.21.15.66:0/403214328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab6140 msgr2=0x5649fab10440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 --2- 172.21.15.66:0/403214328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab6140 0x5649fab10440 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f101801cd50 tx=0x7f1018004a50 comp rx=0 tx=0).stop 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- 172.21.15.66:0/403214328 shutdown_connections 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 --2- 172.21.15.66:0/403214328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faab6140 0x5649fab10440 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 --2- 172.21.15.66:0/403214328 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab5820 0x5649fab0ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 --2- 172.21.15.66:0/403214328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faa6ce50 0x5649fab0f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- 172.21.15.66:0/403214328 >> 172.21.15.66:0/403214328 conn(0x5649fa86b280 msgr2=0x5649faaa2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- 172.21.15.66:0/403214328 shutdown_connections 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- 172.21.15.66:0/403214328 wait complete. 2024-03-30T00:43:20.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- start start 2024-03-30T00:43:20.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 Processor -- start 2024-03-30T00:43:20.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faafc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab5820 0x5649fab12880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab6140 0x5649fab12dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fa8beb30 con 0x5649faa6ce50 2024-03-30T00:43:20.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649fa8b5df0 con 0x5649faab5820 2024-03-30T00:43:20.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.244+0000 7f1029634000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649fa8b35e0 con 0x5649faab6140 2024-03-30T00:43:20.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 10 client.0 _ll_register_callbacks cb 0x5649fa86c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:20.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f101f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faafc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f101f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faafc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58504/0 (socket says 172.21.15.66:58504) 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f101f400640 1 -- 172.21.15.66:0/895416606 learned_addr learned my addr 172.21.15.66:0/895416606 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1017e00640 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab5820 0x5649fab12880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f101fe00640 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab6140 0x5649fab12dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f101f400640 1 -- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab6140 msgr2=0x5649fab12dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f101f400640 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab6140 0x5649fab12dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f101f400640 1 -- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab5820 msgr2=0x5649fab12880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f101f400640 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab5820 0x5649fab12880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f101f400640 1 -- 172.21.15.66:0/895416606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10200cf110 con 0x5649faa6ce50 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f101f400640 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faafc9c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1010000f70 tx=0x7f101000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6a8b20fbffe267 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f100cc00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f100cc00640 1 -- 172.21.15.66:0/895416606 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010001d00 con 0x5649faa6ce50 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f100cc00640 1 -- 172.21.15.66:0/895416606 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f101001b040 con 0x5649faa6ce50 2024-03-30T00:43:20.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 -- 172.21.15.66:0/895416606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649faa6faf0 con 0x5649faa6ce50 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f100cc00640 1 -- 172.21.15.66:0/895416606 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1010006400 con 0x5649faa6ce50 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 10 client.6648 fetch_fsmap learned FSMap version 33 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 -- 172.21.15.66:0/895416606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5649fa85aa80 con 0x5649faa6ce50 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f100cc00640 1 -- 172.21.15.66:0/895416606 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5649faa6faf0 con 0x5649faa6ce50 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f100cc00640 1 -- 172.21.15.66:0/895416606 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1010028040 con 0x5649faa6ce50 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 10 client.6648 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 10 client.6648 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 20 client.6648 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5649fab15690 0x5649fab13cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 4 client.6648 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 -- 172.21.15.66:0/895416606 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x5649fa88b570 con 0x5649fab15690 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1017e00640 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5649fab15690 0x5649fab13cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1017e00640 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5649fab15690 0x5649fab13cb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a88fc3064a76d0b server_cookie=5ca6b9441901c5f in_seq=0 out_seq=0 2024-03-30T00:43:20.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f100cc00640 10 client.6648 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:20.258 INFO:tasks.ceph.mds.d.smithi156.stderr:2024-03-30T00:43:20.248+0000 7f6205800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6597=0-4194304@1} caps={6597=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6597 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5628a73afb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:43:20.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f100cc00640 1 -- 172.21.15.66:0/895416606 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5649fa88b570 con 0x5649fab15690 2024-03-30T00:43:20.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f100cc00640 10 client.6648 handle_command_reply: tid=0 2024-03-30T00:43:20.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 client.6648 shutdown 2024-03-30T00:43:20.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 -- 172.21.15.66:0/895416606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 msgr2=0x5649faafc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faafc9c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1010000f70 tx=0x7f101000a000 comp rx=0 tx=0).stop 2024-03-30T00:43:20.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 -- 172.21.15.66:0/895416606 shutdown_connections 2024-03-30T00:43:20.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5649fab15690 0x5649fab13cb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5649faab6140 0x5649fab12dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649faab5820 0x5649fab12880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 --2- 172.21.15.66:0/895416606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5649faa6ce50 0x5649faafc9c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 -- 172.21.15.66:0/895416606 >> 172.21.15.66:0/895416606 conn(0x5649fa86b280 msgr2=0x5649faaa2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f100cc00640 0 client.6648 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:20.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 -- 172.21.15.66:0/895416606 shutdown_connections 2024-03-30T00:43:20.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.248+0000 7f1029634000 1 -- 172.21.15.66:0/895416606 wait complete. 2024-03-30T00:43:20.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1029634000 20 client.6648 trim_cache size 0 max 16384 2024-03-30T00:43:20.260 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:43:20.260 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T00:43:20.260 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "06692700-0919-453f-9686-2b6770b6102b", 2024-03-30T00:43:20.260 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T00:43:20.261 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:43:20.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 -- 172.21.15.66:0/2864383969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ffc066690 msgr2=0x7f0ffc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ffc066690 0x7f0ffc068b50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f10100011f0 tx=0x7f1010007d40 comp rx=0 tx=0).stop 2024-03-30T00:43:20.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 -- 172.21.15.66:0/2864383969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10200cecf0 msgr2=0x7f10200ae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10200cecf0 0x7f10200ae570 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f100800e900 tx=0x7f100800edc0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 -- 172.21.15.66:0/2864383969 shutdown_connections 2024-03-30T00:43:20.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f102015ba00 0x7f10200aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ffc066690 0x7f0ffc068b50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10200cecf0 0x7f10200ae570 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 --2- 172.21.15.66:0/2864383969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10200acaf0 0x7f1020157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 -- 172.21.15.66:0/2864383969 >> 172.21.15.66:0/2864383969 conn(0x7f102006a190 msgr2=0x7f10200ad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 -- 172.21.15.66:0/2864383969 shutdown_connections 2024-03-30T00:43:20.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.252+0000 7f1026200640 1 -- 172.21.15.66:0/2864383969 wait complete. 2024-03-30T00:43:20.275 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-30T00:43:20.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb788000640 1 Processor -- start 2024-03-30T00:43:20.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb788000640 1 -- start start 2024-03-30T00:43:20.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb788000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7800cde70 0x7fb78015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb788000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7800cd3e0 con 0x7fb7800cec30 2024-03-30T00:43:20.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb788000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78015a990 con 0x7fb7800cda90 2024-03-30T00:43:20.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb788000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800b1b80 con 0x7fb7800cde70 2024-03-30T00:43:20.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb786000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7800cde70 0x7fb78015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb786000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7800cde70 0x7fb78015a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58512/0 (socket says 172.21.15.66:58512) 2024-03-30T00:43:20.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb786000640 1 -- 172.21.15.66:0/1802669843 learned_addr learned my addr 172.21.15.66:0/1802669843 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb784c00640 1 --1- 172.21.15.66:0/1802669843 >> v1:172.21.15.66:6789/0 conn(0x7fb7800cec30 0x7fb780159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:54444/0 (socket says 172.21.15.66:54444) 2024-03-30T00:43:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb786000640 1 -- 172.21.15.66:0/1802669843 >> v1:172.21.15.156:6789/0 conn(0x7fb7800cda90 legacy=0x7fb7800ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb786000640 1 -- 172.21.15.66:0/1802669843 >> v1:172.21.15.66:6789/0 conn(0x7fb7800cec30 legacy=0x7fb780159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.412+0000 7fb786000640 1 -- 172.21.15.66:0/1802669843 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7800b11e0 con 0x7fb7800cde70 2024-03-30T00:43:20.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb786000640 1 --2- 172.21.15.66:0/1802669843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7800cde70 0x7fb78015a370 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb77800d150 tx=0x7fb77801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba3ac1a7e620fa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb77fe00640 1 -- 172.21.15.66:0/1802669843 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77800b2d0 con 0x7fb7800cde70 2024-03-30T00:43:20.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb77fe00640 1 -- 172.21.15.66:0/1802669843 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77800b430 con 0x7fb7800cde70 2024-03-30T00:43:20.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb77fe00640 1 -- 172.21.15.66:0/1802669843 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7780a7c90 con 0x7fb7800cde70 2024-03-30T00:43:20.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb788000640 1 -- 172.21.15.66:0/1802669843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7800cde70 msgr2=0x7fb78015a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb788000640 1 --2- 172.21.15.66:0/1802669843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7800cde70 0x7fb78015a370 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb77800d150 tx=0x7fb77801eba0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb788000640 1 -- 172.21.15.66:0/1802669843 shutdown_connections 2024-03-30T00:43:20.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb788000640 1 --2- 172.21.15.66:0/1802669843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7800cde70 0x7fb78015a370 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb788000640 1 -- 172.21.15.66:0/1802669843 >> 172.21.15.66:0/1802669843 conn(0x7fb780069f60 msgr2=0x7fb78006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb788000640 1 -- 172.21.15.66:0/1802669843 shutdown_connections 2024-03-30T00:43:20.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb788000640 1 -- 172.21.15.66:0/1802669843 wait complete. 2024-03-30T00:43:20.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb788000640 1 Processor -- start 2024-03-30T00:43:20.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.416+0000 7fb788000640 1 -- start start 2024-03-30T00:43:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb788000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7800cda90 0x7fb7801558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb788000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7800cde70 0x7fb7800c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb788000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7800cec30 0x7fb78006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb788000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb780155df0 con 0x7fb7800cec30 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb788000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb780092260 con 0x7fb7800cde70 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb788000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800c78c0 con 0x7fb7800cda90 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb786000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7800cec30 0x7fb78006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb786000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7800cec30 0x7fb78006d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58526/0 (socket says 172.21.15.66:58526) 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb786000640 1 -- 172.21.15.66:0/43109276 learned_addr learned my addr 172.21.15.66:0/43109276 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb784c00640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7800cde70 0x7fb7800c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb785600640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7800cda90 0x7fb7801558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb786000640 1 -- 172.21.15.66:0/43109276 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7800cda90 msgr2=0x7fb7801558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb786000640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7800cda90 0x7fb7801558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb786000640 1 -- 172.21.15.66:0/43109276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7800cde70 msgr2=0x7fb7800c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb786000640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7800cde70 0x7fb7800c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb786000640 1 -- 172.21.15.66:0/43109276 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77800bd40 con 0x7fb7800cec30 2024-03-30T00:43:20.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb786000640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7800cec30 0x7fb78006d510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb77801b7b0 tx=0x7fb77801b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb77d600640 1 -- 172.21.15.66:0/43109276 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778009800 con 0x7fb7800cec30 2024-03-30T00:43:20.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb77d600640 1 -- 172.21.15.66:0/43109276 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb778009960 con 0x7fb7800cec30 2024-03-30T00:43:20.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb77d600640 1 -- 172.21.15.66:0/43109276 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778009d10 con 0x7fb7800cec30 2024-03-30T00:43:20.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb788000640 1 -- 172.21.15.66:0/43109276 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7800c7ba0 con 0x7fb7800cec30 2024-03-30T00:43:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb788000640 1 -- 172.21.15.66:0/43109276 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7800c8160 con 0x7fb7800cec30 2024-03-30T00:43:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.420+0000 7fb77d600640 1 -- 172.21.15.66:0/43109276 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb7780ba470 con 0x7fb7800cec30 2024-03-30T00:43:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb77d600640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7600666c0 0x7fb760068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb77d600640 1 -- 172.21.15.66:0/43109276 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb7781219f0 con 0x7fb7800cec30 2024-03-30T00:43:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb785600640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7600666c0 0x7fb760068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb78b555000 1 Processor -- start 2024-03-30T00:43:20.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb785600640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7600666c0 0x7fb760068b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb7801f4670 tx=0x7fb774009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb78b555000 1 -- start start 2024-03-30T00:43:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb78b555000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec21200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb78b555000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec21205e0 0x561ec20d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb78b555000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec20d5e20 0x561ec20d6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb78b555000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1f9a630 con 0x561ec21205e0 2024-03-30T00:43:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb78b555000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1f27bd0 con 0x561ec211fcc0 2024-03-30T00:43:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb78b555000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ec1f251a0 con 0x561ec20d5e20 2024-03-30T00:43:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb784c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec21205e0 0x561ec20d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb784c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec21205e0 0x561ec20d58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58532/0 (socket says 172.21.15.66:58532) 2024-03-30T00:43:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb784c00640 1 -- 172.21.15.66:0/2948029908 learned_addr learned my addr 172.21.15.66:0/2948029908 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb785600640 1 --2- 172.21.15.66:0/2948029908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec21200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb786000640 1 --2- 172.21.15.66:0/2948029908 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec20d5e20 0x561ec20d6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb784c00640 1 -- 172.21.15.66:0/2948029908 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec20d5e20 msgr2=0x561ec20d6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb784c00640 1 --2- 172.21.15.66:0/2948029908 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec20d5e20 0x561ec20d6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb784c00640 1 -- 172.21.15.66:0/2948029908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 msgr2=0x561ec21200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb784c00640 1 --2- 172.21.15.66:0/2948029908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec21200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb784c00640 1 -- 172.21.15.66:0/2948029908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ec1c1ce20 con 0x561ec21205e0 2024-03-30T00:43:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb785600640 1 --2- 172.21.15.66:0/2948029908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec21200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb784c00640 1 --2- 172.21.15.66:0/2948029908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec21205e0 0x561ec20d58e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb76c004c80 tx=0x7fb76c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93af38dadac0e356 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb772000640 1 -- 172.21.15.66:0/2948029908 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c004e90 con 0x561ec21205e0 2024-03-30T00:43:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb772000640 1 -- 172.21.15.66:0/2948029908 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c001840 con 0x561ec21205e0 2024-03-30T00:43:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.424+0000 7fb772000640 1 -- 172.21.15.66:0/2948029908 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c001bf0 con 0x561ec21205e0 2024-03-30T00:43:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 -- 172.21.15.66:0/2948029908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec21205e0 msgr2=0x561ec20d58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 --2- 172.21.15.66:0/2948029908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec21205e0 0x561ec20d58e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb76c004c80 tx=0x7fb76c007430 comp rx=0 tx=0).stop 2024-03-30T00:43:20.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 -- 172.21.15.66:0/2948029908 shutdown_connections 2024-03-30T00:43:20.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 --2- 172.21.15.66:0/2948029908 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec20d5e20 0x561ec20d6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 --2- 172.21.15.66:0/2948029908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec21205e0 0x561ec20d58e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 --2- 172.21.15.66:0/2948029908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec21200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 -- 172.21.15.66:0/2948029908 >> 172.21.15.66:0/2948029908 conn(0x561ec1ed0160 msgr2=0x561ec20d8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 -- 172.21.15.66:0/2948029908 shutdown_connections 2024-03-30T00:43:20.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 -- 172.21.15.66:0/2948029908 wait complete. 2024-03-30T00:43:20.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 -- start start 2024-03-30T00:43:20.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 Processor -- start 2024-03-30T00:43:20.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec20e1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec20e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec21205e0 0x561ec20e1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1f20a90 con 0x561ec20d5e20 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1ebfd40 con 0x561ec211fcc0 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ec1eddf30 con 0x561ec21205e0 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 10 client.0 _ll_register_callbacks cb 0x561ec1f82810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb785600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec20e1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb785600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec20e1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58536/0 (socket says 172.21.15.66:58536) 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb785600640 1 -- 172.21.15.66:0/2498115382 learned_addr learned my addr 172.21.15.66:0/2498115382 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb784c00640 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec20e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb786000640 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec21205e0 0x561ec20e1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb785600640 1 -- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec21205e0 msgr2=0x561ec20e1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb785600640 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec21205e0 0x561ec20e1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb785600640 1 -- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 msgr2=0x561ec20e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb785600640 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec20e1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb785600640 1 -- 172.21.15.66:0/2498115382 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78015a990 con 0x561ec20d5e20 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb784c00640 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec20e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb785600640 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec20e1230 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb774014700 tx=0x7fb774001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cf155c23cdf11f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb766a00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb766a00640 1 -- 172.21.15.66:0/2498115382 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774013680 con 0x561ec20d5e20 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb766a00640 1 -- 172.21.15.66:0/2498115382 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb77401c040 con 0x561ec20d5e20 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 -- 172.21.15.66:0/2498115382 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ec1f1f3e0 con 0x561ec20d5e20 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb766a00640 1 -- 172.21.15.66:0/2498115382 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7740137e0 con 0x561ec20d5e20 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 10 client.6663 fetch_fsmap learned FSMap version 33 2024-03-30T00:43:20.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 1 -- 172.21.15.66:0/2498115382 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x7fb7800cd3e0 con 0x561ec20d5e20 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb766a00640 1 -- 172.21.15.66:0/2498115382 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561ec1f1f3e0 con 0x561ec20d5e20 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb766a00640 1 -- 172.21.15.66:0/2498115382 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb774012070 con 0x561ec20d5e20 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 10 client.6663 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 10 client.6663 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.428+0000 7fb78b555000 20 client.6663 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x561ec20e26c0 0x561ec2131d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 4 client.6663 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 -- 172.21.15.66:0/2498115382 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ec1ef0d00 con 0x561ec20e26c0 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb784c00640 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x561ec20e26c0 0x561ec2131d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb784c00640 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x561ec20e26c0 0x561ec2131d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30618931e566f361 server_cookie=46496c73e3bb0ae in_seq=0 out_seq=0 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb766a00640 10 client.6663 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:20.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb766a00640 1 -- 172.21.15.66:0/2498115382 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561ec1ef0d00 con 0x561ec20e26c0 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb766a00640 10 client.6663 handle_command_reply: tid=0 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 client.6663 shutdown 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 -- 172.21.15.66:0/2498115382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 msgr2=0x561ec20e1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec20e1230 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb774014700 tx=0x7fb774001090 comp rx=0 tx=0).stop 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 -- 172.21.15.66:0/2498115382 shutdown_connections 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x561ec20e26c0 0x561ec2131d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec21205e0 0x561ec20e1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec20e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 --2- 172.21.15.66:0/2498115382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec20e1230 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 -- 172.21.15.66:0/2498115382 >> 172.21.15.66:0/2498115382 conn(0x561ec1ed0160 msgr2=0x561ec20d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb766a00640 0 client.6663 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:20.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 -- 172.21.15.66:0/2498115382 shutdown_connections 2024-03-30T00:43:20.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 1 -- 172.21.15.66:0/2498115382 wait complete. 2024-03-30T00:43:20.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.432+0000 7fb78b555000 20 client.6663 trim_cache size 0 max 16384 2024-03-30T00:43:20.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 Processor -- start 2024-03-30T00:43:20.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 -- start start 2024-03-30T00:43:20.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec2133ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec2132d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec21205e0 0x561ec21332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1bddaa0 con 0x561ec20d5e20 2024-03-30T00:43:20.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1fd9ec0 con 0x561ec211fcc0 2024-03-30T00:43:20.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ec1ff1470 con 0x561ec21205e0 2024-03-30T00:43:20.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb785600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec2133ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb785600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec2133ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58552/0 (socket says 172.21.15.66:58552) 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb785600640 1 -- 172.21.15.66:0/3259888777 learned_addr learned my addr 172.21.15.66:0/3259888777 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb786000640 1 --2- 172.21.15.66:0/3259888777 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec21205e0 0x561ec21332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb784c00640 1 --2- 172.21.15.66:0/3259888777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec2132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb785600640 1 -- 172.21.15.66:0/3259888777 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec21205e0 msgr2=0x561ec21332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb785600640 1 --2- 172.21.15.66:0/3259888777 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec21205e0 0x561ec21332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb785600640 1 -- 172.21.15.66:0/3259888777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 msgr2=0x561ec2132d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb785600640 1 --2- 172.21.15.66:0/3259888777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec2132d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb785600640 1 -- 172.21.15.66:0/3259888777 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ec1ee5aa0 con 0x561ec20d5e20 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb786000640 1 --2- 172.21.15.66:0/3259888777 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec21205e0 0x561ec21332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb784c00640 1 --2- 172.21.15.66:0/3259888777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec2132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb785600640 1 --2- 172.21.15.66:0/3259888777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec2133ee0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb774014fd0 tx=0x7fb774021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bfa41f4662ec1c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb772a00640 1 -- 172.21.15.66:0/3259888777 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774021480 con 0x561ec20d5e20 2024-03-30T00:43:20.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb772a00640 1 -- 172.21.15.66:0/3259888777 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb774013680 con 0x561ec20d5e20 2024-03-30T00:43:20.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb772a00640 1 -- 172.21.15.66:0/3259888777 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774013e50 con 0x561ec20d5e20 2024-03-30T00:43:20.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 -- 172.21.15.66:0/3259888777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 msgr2=0x561ec2133ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 --2- 172.21.15.66:0/3259888777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec2133ee0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb774014fd0 tx=0x7fb774021000 comp rx=0 tx=0).stop 2024-03-30T00:43:20.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 -- 172.21.15.66:0/3259888777 shutdown_connections 2024-03-30T00:43:20.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 --2- 172.21.15.66:0/3259888777 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec21205e0 0x561ec21332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 --2- 172.21.15.66:0/3259888777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec211fcc0 0x561ec2132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 --2- 172.21.15.66:0/3259888777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec20d5e20 0x561ec2133ee0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 -- 172.21.15.66:0/3259888777 >> 172.21.15.66:0/3259888777 conn(0x561ec1ed0160 msgr2=0x561ec212d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 -- 172.21.15.66:0/3259888777 shutdown_connections 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.448+0000 7fb78b555000 1 -- 172.21.15.66:0/3259888777 wait complete. 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 -- start start 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 Processor -- start 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec20d5e20 0x561ec217f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec211fcc0 0x561ec217fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec21205e0 0x561ec211d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1f64170 con 0x561ec211fcc0 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec20dd870 con 0x561ec21205e0 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ec1f24a90 con 0x561ec20d5e20 2024-03-30T00:43:20.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 10 client.0 _ll_register_callbacks cb 0x7fb76c004890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb784c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec211fcc0 0x561ec217fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb784c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec211fcc0 0x561ec217fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58554/0 (socket says 172.21.15.66:58554) 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb784c00640 1 -- 172.21.15.66:0/710769830 learned_addr learned my addr 172.21.15.66:0/710769830 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb785600640 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec20d5e20 0x561ec217f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb786000640 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec21205e0 0x561ec211d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb784c00640 1 -- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec20d5e20 msgr2=0x561ec217f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb784c00640 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec20d5e20 0x561ec217f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb784c00640 1 -- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec21205e0 msgr2=0x561ec211d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb784c00640 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec21205e0 0x561ec211d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb784c00640 1 -- 172.21.15.66:0/710769830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7800cd3e0 con 0x561ec211fcc0 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb786000640 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec21205e0 0x561ec211d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:20.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb784c00640 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec211fcc0 0x561ec217fd20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb76c015040 tx=0x7fb76c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d28b17da0d4619 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb766a00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb766a00640 1 -- 172.21.15.66:0/710769830 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c024070 con 0x561ec211fcc0 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb766a00640 1 -- 172.21.15.66:0/710769830 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb76c007b10 con 0x561ec211fcc0 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 -- 172.21.15.66:0/710769830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ec1edd970 con 0x561ec211fcc0 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb766a00640 1 -- 172.21.15.66:0/710769830 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb76c028730 con 0x561ec211fcc0 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 10 client.6669 fetch_fsmap learned FSMap version 33 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 -- 172.21.15.66:0/710769830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x561ec1ecbdf0 con 0x561ec211fcc0 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb766a00640 1 -- 172.21.15.66:0/710769830 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561ec1edd970 con 0x561ec211fcc0 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb766a00640 1 -- 172.21.15.66:0/710769830 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb76c012070 con 0x561ec211fcc0 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 10 client.6669 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 10 client.6669 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-30T00:43:20.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 20 client.6669 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:20.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x561ec211d5e0 0x561ec211d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:20.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 4 client.6669 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:43:20.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb78b555000 1 -- 172.21.15.66:0/710769830 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561ec1f4fa20 con 0x561ec211d5e0 2024-03-30T00:43:20.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.452+0000 7fb785600640 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x561ec211d5e0 0x561ec211d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:20.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb785600640 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x561ec211d5e0 0x561ec211d9c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349c29331e39bb01 server_cookie=ea8017e8554663bc in_seq=0 out_seq=0 2024-03-30T00:43:20.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb766a00640 10 client.6669 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:20.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb766a00640 1 -- 172.21.15.66:0/710769830 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x561ec1f4fa20 con 0x561ec211d5e0 2024-03-30T00:43:20.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb766a00640 10 client.6669 handle_command_reply: tid=0 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 1 client.6669 shutdown 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 1 -- 172.21.15.66:0/710769830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec211fcc0 msgr2=0x561ec217fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec211fcc0 0x561ec217fd20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb76c015040 tx=0x7fb76c000ee0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 1 -- 172.21.15.66:0/710769830 shutdown_connections 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x561ec211d5e0 0x561ec211d9c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ec21205e0 0x561ec211d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb766a00640 0 client.6669 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ec211fcc0 0x561ec217fd20 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 1 --2- 172.21.15.66:0/710769830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561ec20d5e20 0x561ec217f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 1 -- 172.21.15.66:0/710769830 >> 172.21.15.66:0/710769830 conn(0x561ec1ed0160 msgr2=0x561ec2131d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 1 -- 172.21.15.66:0/710769830 shutdown_connections 2024-03-30T00:43:20.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 1 -- 172.21.15.66:0/710769830 wait complete. 2024-03-30T00:43:20.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb78b555000 20 client.6669 trim_cache size 0 max 16384 2024-03-30T00:43:20.467 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:43:20.467 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:43:20.467 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-03-30T00:43:20.467 INFO:teuthology.orchestra.run.smithi066.stdout: "06692700-0919-453f-9686-2b6770b6102b": { 2024-03-30T00:43:20.467 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T00:43:20.467 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "06692700-0919-453f-9686-2b6770b6102b", 2024-03-30T00:43:20.467 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive" 2024-03-30T00:43:20.467 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:20.468 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:20.468 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:43:20.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 -- 172.21.15.66:0/43109276 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7600666c0 msgr2=0x7fb760068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7600666c0 0x7fb760068b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb7801f4670 tx=0x7fb774009040 comp rx=0 tx=0).stop 2024-03-30T00:43:20.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 -- 172.21.15.66:0/43109276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7800cec30 msgr2=0x7fb78006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:20.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7800cec30 0x7fb78006d510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb77801b7b0 tx=0x7fb77801b940 comp rx=0 tx=0).stop 2024-03-30T00:43:20.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 -- 172.21.15.66:0/43109276 shutdown_connections 2024-03-30T00:43:20.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb7600666c0 0x7fb760068b80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7800cec30 0x7fb78006d510 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7800cde70 0x7fb7800c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 --2- 172.21.15.66:0/43109276 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7800cda90 0x7fb7801558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:20.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 -- 172.21.15.66:0/43109276 >> 172.21.15.66:0/43109276 conn(0x7fb780069f60 msgr2=0x7fb7800cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:20.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 -- 172.21.15.66:0/43109276 shutdown_connections 2024-03-30T00:43:20.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:20.456+0000 7fb788000640 1 -- 172.21.15.66:0/43109276 wait complete. 2024-03-30T00:43:20.481 INFO:tasks.cephfs.filesystem:scrub status for tag:06692700-0919-453f-9686-2b6770b6102b - {'path': '/', 'tag': '06692700-0919-453f-9686-2b6770b6102b', 'options': 'recursive'} 2024-03-30T00:43:23.533 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:43:23.528+0000 7f1c7fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-30T00:43:46.337 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:43:46.342 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:43:50.482 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-30T00:43:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f191d600640 1 Processor -- start 2024-03-30T00:43:50.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f191d600640 1 -- start start 2024-03-30T00:43:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f191d600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f19181610c0 0x7f19181604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f191d600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191806da20 con 0x7f19181609f0 2024-03-30T00:43:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f191d600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191806dc60 con 0x7f1918157060 2024-03-30T00:43:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f191d600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1918157440 con 0x7f19181610c0 2024-03-30T00:43:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f1917400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f19181609f0 0x7f19180abc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33972/0 (socket says 172.21.15.66:33972) 2024-03-30T00:43:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f1917400640 1 -- 172.21.15.66:0/1824056039 learned_addr learned my addr 172.21.15.66:0/1824056039 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f190e000640 1 --2- 172.21.15.66:0/1824056039 >> v2:172.21.15.156:3300/0 conn(0x7f19181610c0 0x7f19181604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f1916000640 1 -- 172.21.15.66:0/1824056039 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918373817 0 0) 0x7f1910001120 con 0x7f19181609f0 2024-03-30T00:43:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f1916000640 1 -- 172.21.15.66:0/1824056039 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18fc003530 con 0x7f19181609f0 2024-03-30T00:43:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f190e000640 1 -- 172.21.15.66:0/1824056039 >> v1:172.21.15.156:6789/0 conn(0x7f1918157060 legacy=0x7f191806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f190e000640 1 -- 172.21.15.66:0/1824056039 >> v1:172.21.15.66:6789/0 conn(0x7f19181609f0 legacy=0x7f19180abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.620+0000 7f190e000640 1 -- 172.21.15.66:0/1824056039 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191806df40 con 0x7f19181610c0 2024-03-30T00:43:50.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.624+0000 7f190e000640 1 --2- 172.21.15.66:0/1824056039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19181610c0 0x7f19181604b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1900009d90 tx=0x7f190001c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e6957ce12632293 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:50.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.624+0000 7f1916000640 1 -- 172.21.15.66:0/1824056039 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000a5710 con 0x7f19181610c0 2024-03-30T00:43:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.624+0000 7f1916000640 1 -- 172.21.15.66:0/1824056039 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19000a5870 con 0x7f19181610c0 2024-03-30T00:43:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.624+0000 7f1916000640 1 -- 172.21.15.66:0/1824056039 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000a5c00 con 0x7f19181610c0 2024-03-30T00:43:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.624+0000 7f191d600640 1 -- 172.21.15.66:0/1824056039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19181610c0 msgr2=0x7f19181604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.624+0000 7f191d600640 1 --2- 172.21.15.66:0/1824056039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19181610c0 0x7f19181604b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1900009d90 tx=0x7f190001c850 comp rx=0 tx=0).stop 2024-03-30T00:43:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.624+0000 7f191d600640 1 -- 172.21.15.66:0/1824056039 shutdown_connections 2024-03-30T00:43:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.624+0000 7f191d600640 1 --2- 172.21.15.66:0/1824056039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19181610c0 0x7f19181604b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.624+0000 7f191d600640 1 -- 172.21.15.66:0/1824056039 >> 172.21.15.66:0/1824056039 conn(0x7f1918071750 msgr2=0x7f1918069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.624+0000 7f191d600640 1 -- 172.21.15.66:0/1824056039 shutdown_connections 2024-03-30T00:43:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.624+0000 7f191d600640 1 -- 172.21.15.66:0/1824056039 wait complete. 2024-03-30T00:43:50.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f191d600640 1 Processor -- start 2024-03-30T00:43:50.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f191d600640 1 -- start start 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f191d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1918157060 0x7f191814bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f191d600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19181609f0 0x7f19180b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f191d600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19181610c0 0x7f1918155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f191d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1918153660 con 0x7f1918157060 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f191d600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1918155e50 con 0x7f19181610c0 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f191d600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19181538b0 con 0x7f19181609f0 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f1916a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1918157060 0x7f191814bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f1916a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1918157060 0x7f191814bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36818/0 (socket says 172.21.15.66:36818) 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f1916a00640 1 -- 172.21.15.66:0/1109731861 learned_addr learned my addr 172.21.15.66:0/1109731861 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f190e000640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19181609f0 0x7f19180b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f1917400640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19181610c0 0x7f1918155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f1916a00640 1 -- 172.21.15.66:0/1109731861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19181609f0 msgr2=0x7f19180b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f1916a00640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19181609f0 0x7f19180b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f1916a00640 1 -- 172.21.15.66:0/1109731861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19181610c0 msgr2=0x7f1918155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f1916a00640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19181610c0 0x7f1918155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f1916a00640 1 -- 172.21.15.66:0/1109731861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1900009a70 con 0x7f1918157060 2024-03-30T00:43:50.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f190e000640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19181609f0 0x7f19180b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:50.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f1916a00640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1918157060 0x7f191814bac0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f190800c3c0 tx=0x7f190800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:50.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f190f400640 1 -- 172.21.15.66:0/1109731861 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080052d0 con 0x7f1918157060 2024-03-30T00:43:50.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f190f400640 1 -- 172.21.15.66:0/1109731861 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1908005430 con 0x7f1918157060 2024-03-30T00:43:50.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f191d600640 1 -- 172.21.15.66:0/1109731861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1918153b90 con 0x7f1918157060 2024-03-30T00:43:50.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f190f400640 1 -- 172.21.15.66:0/1109731861 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908003840 con 0x7f1918157060 2024-03-30T00:43:50.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.628+0000 7f191d600640 1 -- 172.21.15.66:0/1109731861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191814c000 con 0x7f1918157060 2024-03-30T00:43:50.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f190f400640 1 -- 172.21.15.66:0/1109731861 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f19080039a0 con 0x7f1918157060 2024-03-30T00:43:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f190f400640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18f4066690 0x7f18f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f190f400640 1 -- 172.21.15.66:0/1109731861 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f19080838a0 con 0x7f1918157060 2024-03-30T00:43:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f190e000640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18f4066690 0x7f18f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f1920b63000 1 Processor -- start 2024-03-30T00:43:50.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f1920b63000 1 -- start start 2024-03-30T00:43:50.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f190e000640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18f4066690 0x7f18f4068b50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f19000af000 tx=0x7f19000a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f1920b63000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba197ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f1920b63000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197d390 0x555ba19304c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f1920b63000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba1930a00 0x555ba1930de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f1920b63000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba17fb630 con 0x555ba197ca70 2024-03-30T00:43:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f1920b63000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba1788bd0 con 0x555ba1930a00 2024-03-30T00:43:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.632+0000 7f1920b63000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ba17861a0 con 0x555ba197d390 2024-03-30T00:43:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1916a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba197ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1916a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba197ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36834/0 (socket says 172.21.15.66:36834) 2024-03-30T00:43:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1916a00640 1 -- 172.21.15.66:0/881878052 learned_addr learned my addr 172.21.15.66:0/881878052 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f190e000640 1 --2- 172.21.15.66:0/881878052 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197d390 0x555ba19304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1917400640 1 --2- 172.21.15.66:0/881878052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba1930a00 0x555ba1930de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1916a00640 1 -- 172.21.15.66:0/881878052 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197d390 msgr2=0x555ba19304c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:50.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1916a00640 1 --2- 172.21.15.66:0/881878052 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197d390 0x555ba19304c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1916a00640 1 -- 172.21.15.66:0/881878052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba1930a00 msgr2=0x555ba1930de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:50.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1916a00640 1 --2- 172.21.15.66:0/881878052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba1930a00 0x555ba1930de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1916a00640 1 -- 172.21.15.66:0/881878052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ba147de20 con 0x555ba197ca70 2024-03-30T00:43:50.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1917400640 1 --2- 172.21.15.66:0/881878052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba1930a00 0x555ba1930de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:50.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1916a00640 1 --2- 172.21.15.66:0/881878052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba197ce50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f190805c5b0 tx=0x7f1908054150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149007f525b50290 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:50.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1907400640 1 -- 172.21.15.66:0/881878052 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190805cae0 con 0x555ba197ca70 2024-03-30T00:43:50.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1907400640 1 -- 172.21.15.66:0/881878052 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1908052a90 con 0x555ba197ca70 2024-03-30T00:43:50.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1907400640 1 -- 172.21.15.66:0/881878052 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908058120 con 0x555ba197ca70 2024-03-30T00:43:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 -- 172.21.15.66:0/881878052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 msgr2=0x555ba197ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 --2- 172.21.15.66:0/881878052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba197ce50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f190805c5b0 tx=0x7f1908054150 comp rx=0 tx=0).stop 2024-03-30T00:43:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 -- 172.21.15.66:0/881878052 shutdown_connections 2024-03-30T00:43:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 --2- 172.21.15.66:0/881878052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba1930a00 0x555ba1930de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 --2- 172.21.15.66:0/881878052 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197d390 0x555ba19304c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 --2- 172.21.15.66:0/881878052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba197ce50 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 -- 172.21.15.66:0/881878052 >> 172.21.15.66:0/881878052 conn(0x555ba1731160 msgr2=0x555ba195f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 -- 172.21.15.66:0/881878052 shutdown_connections 2024-03-30T00:43:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 -- 172.21.15.66:0/881878052 wait complete. 2024-03-30T00:43:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 -- start start 2024-03-30T00:43:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 Processor -- start 2024-03-30T00:43:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba1930a00 0x555ba19397d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba1960a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba197d390 0x555ba1960f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba1781a90 con 0x555ba197ca70 2024-03-30T00:43:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba1720d40 con 0x555ba197d390 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ba173ef30 con 0x555ba1930a00 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1920b63000 10 client.0 _ll_register_callbacks cb 0x555ba17e3810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f190e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba1960a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f190e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba1960a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36844/0 (socket says 172.21.15.66:36844) 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f190e000640 1 -- 172.21.15.66:0/3855418377 learned_addr learned my addr 172.21.15.66:0/3855418377 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1916a00640 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba1930a00 0x555ba19397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.636+0000 7f1917400640 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba197d390 0x555ba1960f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f190e000640 1 -- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba1930a00 msgr2=0x555ba19397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f190e000640 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba1930a00 0x555ba19397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f190e000640 1 -- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba197d390 msgr2=0x555ba1960f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f190e000640 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba197d390 0x555ba1960f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f190e000640 1 -- 172.21.15.66:0/3855418377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191814c000 con 0x555ba197ca70 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f190e000640 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba1960a50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f190001cd30 tx=0x7f190001b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c64eb5c399e7825 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:50.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f18fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f18fbe00640 1 -- 172.21.15.66:0/3855418377 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000b8d90 con 0x555ba197ca70 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 -- 172.21.15.66:0/3855418377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ba17803e0 con 0x555ba197ca70 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f18fbe00640 1 -- 172.21.15.66:0/3855418377 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f19000ad440 con 0x555ba197ca70 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f18fbe00640 1 -- 172.21.15.66:0/3855418377 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19000ad600 con 0x555ba197ca70 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 10 client.6681 fetch_fsmap learned FSMap version 34 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 -- 172.21.15.66:0/3855418377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x555ba1746aa0 con 0x555ba197ca70 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f18fbe00640 1 -- 172.21.15.66:0/3855418377 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555ba17803e0 con 0x555ba197ca70 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f18fbe00640 1 -- 172.21.15.66:0/3855418377 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f19000ad760 con 0x555ba197ca70 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 10 client.6681 fetch_fsmap finished waiting for FSMap version 34 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 10 client.6681 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 20 client.6681 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x555ba19842d0 0x555ba19846b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 4 client.6681 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:43:50.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 -- 172.21.15.66:0/3855418377 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ba1751d00 con 0x555ba19842d0 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1916a00640 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x555ba19842d0 0x555ba19846b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1916a00640 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x555ba19842d0 0x555ba19846b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc097b05d4e9d72 server_cookie=8d8b8f07d48acd0c in_seq=0 out_seq=0 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f18fbe00640 10 client.6681 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f18fbe00640 1 -- 172.21.15.66:0/3855418377 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555ba1751d00 con 0x555ba19842d0 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f18fbe00640 10 client.6681 handle_command_reply: tid=0 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 client.6681 shutdown 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 -- 172.21.15.66:0/3855418377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 msgr2=0x555ba1960a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba1960a50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f190001cd30 tx=0x7f190001b090 comp rx=0 tx=0).stop 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 -- 172.21.15.66:0/3855418377 shutdown_connections 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x555ba19842d0 0x555ba19846b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba197d390 0x555ba1960f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba1960a50 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f18fbe00640 0 client.6681 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 --2- 172.21.15.66:0/3855418377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba1930a00 0x555ba19397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 -- 172.21.15.66:0/3855418377 >> 172.21.15.66:0/3855418377 conn(0x555ba1731160 msgr2=0x555ba192fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:50.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 -- 172.21.15.66:0/3855418377 shutdown_connections 2024-03-30T00:43:50.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 1 -- 172.21.15.66:0/3855418377 wait complete. 2024-03-30T00:43:50.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.640+0000 7f1920b63000 20 client.6681 trim_cache size 0 max 16384 2024-03-30T00:43:50.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1920b63000 1 Processor -- start 2024-03-30T00:43:50.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1920b63000 1 -- start start 2024-03-30T00:43:50.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1920b63000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba1930a00 0x555ba192fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1920b63000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197ca70 0x555ba1930170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1920b63000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba197d390 0x555ba1930de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1920b63000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba143eaa0 con 0x555ba1930a00 2024-03-30T00:43:50.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1920b63000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba1789e60 con 0x555ba197d390 2024-03-30T00:43:50.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1920b63000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ba192a930 con 0x555ba197ca70 2024-03-30T00:43:50.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1916a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba1930a00 0x555ba192fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1916a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba1930a00 0x555ba192fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36848/0 (socket says 172.21.15.66:36848) 2024-03-30T00:43:50.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1916a00640 1 -- 172.21.15.66:0/3994431100 learned_addr learned my addr 172.21.15.66:0/3994431100 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:50.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f190e000640 1 --2- 172.21.15.66:0/3994431100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197ca70 0x555ba1930170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1917400640 1 --2- 172.21.15.66:0/3994431100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba197d390 0x555ba1930de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1916a00640 1 -- 172.21.15.66:0/3994431100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197ca70 msgr2=0x555ba1930170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:50.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1916a00640 1 --2- 172.21.15.66:0/3994431100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197ca70 0x555ba1930170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1916a00640 1 -- 172.21.15.66:0/3994431100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba197d390 msgr2=0x555ba1930de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:50.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1916a00640 1 --2- 172.21.15.66:0/3994431100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba197d390 0x555ba1930de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.656+0000 7f1916a00640 1 -- 172.21.15.66:0/3994431100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ba172cdf0 con 0x555ba1930a00 2024-03-30T00:43:50.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1916a00640 1 --2- 172.21.15.66:0/3994431100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba1930a00 0x555ba192fc30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f190800c390 tx=0x7f1908014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a3cf4ca26f6aed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1907e00640 1 -- 172.21.15.66:0/3994431100 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190805cae0 con 0x555ba1930a00 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1907e00640 1 -- 172.21.15.66:0/3994431100 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1908050e00 con 0x555ba1930a00 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1907e00640 1 -- 172.21.15.66:0/3994431100 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908058360 con 0x555ba1930a00 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 -- 172.21.15.66:0/3994431100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba1930a00 msgr2=0x555ba192fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 --2- 172.21.15.66:0/3994431100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba1930a00 0x555ba192fc30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f190800c390 tx=0x7f1908014000 comp rx=0 tx=0).stop 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 -- 172.21.15.66:0/3994431100 shutdown_connections 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 --2- 172.21.15.66:0/3994431100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba197d390 0x555ba1930de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 --2- 172.21.15.66:0/3994431100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197ca70 0x555ba1930170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 --2- 172.21.15.66:0/3994431100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba1930a00 0x555ba192fc30 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 -- 172.21.15.66:0/3994431100 >> 172.21.15.66:0/3994431100 conn(0x555ba1731160 msgr2=0x555ba1938830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 -- 172.21.15.66:0/3994431100 shutdown_connections 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 -- 172.21.15.66:0/3994431100 wait complete. 2024-03-30T00:43:50.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 -- start start 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 Processor -- start 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba1930a00 0x555ba1933f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba19d8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197d390 0x555ba19d8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba177ee90 con 0x555ba197ca70 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba17c5170 con 0x555ba1930a00 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ba1852470 con 0x555ba197d390 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 10 client.0 _ll_register_callbacks cb 0x7f19080011b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f190e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba19d8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f190e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba19d8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36860/0 (socket says 172.21.15.66:36860) 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f190e000640 1 -- 172.21.15.66:0/2253103997 learned_addr learned my addr 172.21.15.66:0/2253103997 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1916a00640 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba1930a00 0x555ba1933f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1917400640 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197d390 0x555ba19d8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f190e000640 1 -- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197d390 msgr2=0x555ba19d8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f190e000640 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197d390 0x555ba19d8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f190e000640 1 -- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba1930a00 msgr2=0x555ba1933f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f190e000640 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba1930a00 0x555ba1933f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f190e000640 1 -- 172.21.15.66:0/2253103997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ba147de20 con 0x555ba197ca70 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f190e000640 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba19d8440 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1900009d90 tx=0x7f1900009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b441cdc5c7549353 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f18fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f18fbe00640 1 -- 172.21.15.66:0/2253103997 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000adcc0 con 0x555ba197ca70 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f18fbe00640 1 -- 172.21.15.66:0/2253103997 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f19000bc030 con 0x555ba197ca70 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.660+0000 7f1920b63000 1 -- 172.21.15.66:0/2253103997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ba173e970 con 0x555ba197ca70 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f18fbe00640 1 -- 172.21.15.66:0/2253103997 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19000ad440 con 0x555ba197ca70 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 10 client.6687 fetch_fsmap learned FSMap version 34 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 -- 172.21.15.66:0/2253103997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x555ba1720960 con 0x555ba197ca70 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f18fbe00640 1 -- 172.21.15.66:0/2253103997 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555ba173e970 con 0x555ba197ca70 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f18fbe00640 1 -- 172.21.15.66:0/2253103997 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1900005d20 con 0x555ba197ca70 2024-03-30T00:43:50.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 10 client.6687 fetch_fsmap finished waiting for FSMap version 34 2024-03-30T00:43:50.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 10 client.6687 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-30T00:43:50.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 20 client.6687 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:50.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x555ba19d8ec0 0x555ba19d92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 4 client.6687 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:43:50.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 -- 172.21.15.66:0/2253103997 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555ba17b0a20 con 0x555ba19d8ec0 2024-03-30T00:43:50.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1916a00640 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x555ba19d8ec0 0x555ba19d92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1916a00640 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x555ba19d8ec0 0x555ba19d92d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc84d05e751aa11 server_cookie=1ebc7aaed54364c4 in_seq=0 out_seq=0 2024-03-30T00:43:50.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f18fbe00640 10 client.6687 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:50.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f18fbe00640 1 -- 172.21.15.66:0/2253103997 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555ba17b0a20 con 0x555ba19d8ec0 2024-03-30T00:43:50.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f18fbe00640 10 client.6687 handle_command_reply: tid=0 2024-03-30T00:43:50.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 client.6687 shutdown 2024-03-30T00:43:50.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 -- 172.21.15.66:0/2253103997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 msgr2=0x555ba19d8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba19d8440 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1900009d90 tx=0x7f1900009750 comp rx=0 tx=0).stop 2024-03-30T00:43:50.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 -- 172.21.15.66:0/2253103997 shutdown_connections 2024-03-30T00:43:50.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x555ba19d8ec0 0x555ba19d92d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555ba197d390 0x555ba19d8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ba197ca70 0x555ba19d8440 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f18fbe00640 0 client.6687 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:50.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 --2- 172.21.15.66:0/2253103997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555ba1930a00 0x555ba1933f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 -- 172.21.15.66:0/2253103997 >> 172.21.15.66:0/2253103997 conn(0x555ba1731160 msgr2=0x555ba192ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:50.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 -- 172.21.15.66:0/2253103997 shutdown_connections 2024-03-30T00:43:50.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 1 -- 172.21.15.66:0/2253103997 wait complete. 2024-03-30T00:43:50.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f1920b63000 20 client.6687 trim_cache size 0 max 16384 2024-03-30T00:43:50.674 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:43:50.675 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T00:43:50.675 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-03-30T00:43:50.675 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:43:50.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 -- 172.21.15.66:0/1109731861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18f4066690 msgr2=0x7f18f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18f4066690 0x7f18f4068b50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f19000af000 tx=0x7f19000a7070 comp rx=0 tx=0).stop 2024-03-30T00:43:50.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 -- 172.21.15.66:0/1109731861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1918157060 msgr2=0x7f191814bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1918157060 0x7f191814bac0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f190800c3c0 tx=0x7f190800c880 comp rx=0 tx=0).stop 2024-03-30T00:43:50.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 -- 172.21.15.66:0/1109731861 shutdown_connections 2024-03-30T00:43:50.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19181610c0 0x7f1918155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19181609f0 0x7f19180b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18f4066690 0x7f18f4068b50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 --2- 172.21.15.66:0/1109731861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1918157060 0x7f191814bac0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 -- 172.21.15.66:0/1109731861 >> 172.21.15.66:0/1109731861 conn(0x7f1918071750 msgr2=0x7f19180af840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:50.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 -- 172.21.15.66:0/1109731861 shutdown_connections 2024-03-30T00:43:50.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.664+0000 7f191d600640 1 -- 172.21.15.66:0/1109731861 wait complete. 2024-03-30T00:43:50.690 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:43:50.690 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:43:50.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3c600640 1 Processor -- start 2024-03-30T00:43:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3c600640 1 -- start start 2024-03-30T00:43:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3c600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab34161140 0x7fab340cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3c600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fab340d1060 0x7fab340c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3c600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab340cb3e0 con 0x7fab34160d60 2024-03-30T00:43:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3c600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab340cb6c0 con 0x7fab340d1060 2024-03-30T00:43:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3c600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab340cb9a0 con 0x7fab34161140 2024-03-30T00:43:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab39c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fab34160d60 0x7fab340cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33978/0 (socket says 172.21.15.66:33978) 2024-03-30T00:43:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab39c00640 1 -- 172.21.15.66:0/4057866280 learned_addr learned my addr 172.21.15.66:0/4057866280 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab39200640 1 --2- 172.21.15.66:0/4057866280 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab34161140 0x7fab340cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3a600640 1 --2- 172.21.15.66:0/4057866280 >> v2:172.21.15.156:3300/0 conn(0x7fab340d1060 0x7fab340c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab33e00640 1 -- 172.21.15.66:0/4057866280 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332949467 0 0) 0x7fab340cb3e0 con 0x7fab34160d60 2024-03-30T00:43:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab33e00640 1 -- 172.21.15.66:0/4057866280 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab1c003530 con 0x7fab34160d60 2024-03-30T00:43:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3a600640 1 -- 172.21.15.66:0/4057866280 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab34161140 msgr2=0x7fab340cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:50.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3a600640 1 --2- 172.21.15.66:0/4057866280 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab34161140 0x7fab340cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3a600640 1 -- 172.21.15.66:0/4057866280 >> v1:172.21.15.66:6789/0 conn(0x7fab34160d60 legacy=0x7fab340cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.828+0000 7fab3a600640 1 -- 172.21.15.66:0/4057866280 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340cbc80 con 0x7fab340d1060 2024-03-30T00:43:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab3a600640 1 --2- 172.21.15.66:0/4057866280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab340d1060 0x7fab340c3510 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009d90 tx=0x7fab2c01f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76279df86d4e255 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab33e00640 1 -- 172.21.15.66:0/4057866280 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0af600 con 0x7fab340d1060 2024-03-30T00:43:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab33e00640 1 -- 172.21.15.66:0/4057866280 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c0ab050 con 0x7fab340d1060 2024-03-30T00:43:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab33e00640 1 -- 172.21.15.66:0/4057866280 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0af980 con 0x7fab340d1060 2024-03-30T00:43:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab3c600640 1 -- 172.21.15.66:0/4057866280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab340d1060 msgr2=0x7fab340c3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab3c600640 1 --2- 172.21.15.66:0/4057866280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab340d1060 0x7fab340c3510 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009d90 tx=0x7fab2c01f150 comp rx=0 tx=0).stop 2024-03-30T00:43:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab3c600640 1 -- 172.21.15.66:0/4057866280 shutdown_connections 2024-03-30T00:43:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab3c600640 1 --2- 172.21.15.66:0/4057866280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab340d1060 0x7fab340c3510 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab3c600640 1 --2- 172.21.15.66:0/4057866280 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab34161140 0x7fab340cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab3c600640 1 -- 172.21.15.66:0/4057866280 >> 172.21.15.66:0/4057866280 conn(0x7fab340d0540 msgr2=0x7fab340d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab3c600640 1 -- 172.21.15.66:0/4057866280 shutdown_connections 2024-03-30T00:43:50.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.832+0000 7fab3c600640 1 -- 172.21.15.66:0/4057866280 wait complete. 2024-03-30T00:43:50.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3c600640 1 Processor -- start 2024-03-30T00:43:50.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3c600640 1 -- start start 2024-03-30T00:43:50.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3c600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab340d1060 0x7fab340b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3c600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab34160d60 0x7fab3415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3c600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab34161140 0x7fab340c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3c600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab340c6690 con 0x7fab340d1060 2024-03-30T00:43:50.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3c600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab340c6b20 con 0x7fab34161140 2024-03-30T00:43:50.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3c600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab340c5d60 con 0x7fab34160d60 2024-03-30T00:43:50.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab39c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab340d1060 0x7fab340b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab39c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab340d1060 0x7fab340b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36876/0 (socket says 172.21.15.66:36876) 2024-03-30T00:43:50.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab39c00640 1 -- 172.21.15.66:0/4034946418 learned_addr learned my addr 172.21.15.66:0/4034946418 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:50.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab39200640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab34160d60 0x7fab3415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3a600640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab34161140 0x7fab340c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab39c00640 1 -- 172.21.15.66:0/4034946418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab34160d60 msgr2=0x7fab3415ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab39c00640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab34160d60 0x7fab3415ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab39c00640 1 -- 172.21.15.66:0/4034946418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab34161140 msgr2=0x7fab340c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:50.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab39c00640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab34161140 0x7fab340c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:50.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab39c00640 1 -- 172.21.15.66:0/4034946418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c009a70 con 0x7fab340d1060 2024-03-30T00:43:50.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab39c00640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab340d1060 0x7fab340b29e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fab2800c4d0 tx=0x7fab2800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:50.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab31600640 1 -- 172.21.15.66:0/4034946418 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28005cf0 con 0x7fab340d1060 2024-03-30T00:43:50.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab31600640 1 -- 172.21.15.66:0/4034946418 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab28005e50 con 0x7fab340d1060 2024-03-30T00:43:50.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab31600640 1 -- 172.21.15.66:0/4034946418 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28007270 con 0x7fab340d1060 2024-03-30T00:43:50.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3c600640 1 -- 172.21.15.66:0/4034946418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab340c6040 con 0x7fab340d1060 2024-03-30T00:43:50.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3c600640 1 -- 172.21.15.66:0/4034946418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3415cf40 con 0x7fab340d1060 2024-03-30T00:43:50.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.836+0000 7fab3c600640 1 -- 172.21.15.66:0/4034946418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab34066c30 con 0x7fab340d1060 2024-03-30T00:43:50.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.840+0000 7fab31600640 1 -- 172.21.15.66:0/4034946418 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fab280073d0 con 0x7fab340d1060 2024-03-30T00:43:50.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.840+0000 7fab31600640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab140666c0 0x7fab14068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:50.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.840+0000 7fab31600640 1 -- 172.21.15.66:0/4034946418 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fab280838c0 con 0x7fab340d1060 2024-03-30T00:43:50.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.840+0000 7fab39200640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab140666c0 0x7fab14068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:50.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.840+0000 7fab39200640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab140666c0 0x7fab14068b80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fab340b27b0 tx=0x7fab20009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:50.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:50.844+0000 7fab31600640 1 -- 172.21.15.66:0/4034946418 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fab28050a70 con 0x7fab340d1060 2024-03-30T00:43:51.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.036+0000 7fab3c600640 1 -- 172.21.15.66:0/4034946418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab340cb0a0 con 0x7fab340d1060 2024-03-30T00:43:51.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab31600640 1 -- 172.21.15.66:0/4034946418 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fab28017090 con 0x7fab340d1060 2024-03-30T00:43:51.046 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:51.047 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4272961638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4272961638},{"type":"v1","addr":"172.21.15.156:6834","nonce":4272961638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2144186622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2144186622},{"type":"v1","addr":"172.21.15.66:6835","nonce":2144186622}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/750308788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":750308788},{"type":"v1","addr":"172.21.15.66:6837","nonce":750308788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:43:11.141056+0000","modified":"2024-03-30T00:43:23.528635+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6468":{"gid":6468,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6835/1137593870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1137593870},{"type":"v1","addr":"172.21.15.156:6835","nonce":1137593870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-30T00:43:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 34 2024-03-30T00:43:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 -- 172.21.15.66:0/4034946418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab140666c0 msgr2=0x7fab14068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab140666c0 0x7fab14068b80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fab340b27b0 tx=0x7fab20009190 comp rx=0 tx=0).stop 2024-03-30T00:43:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 -- 172.21.15.66:0/4034946418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab340d1060 msgr2=0x7fab340b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab340d1060 0x7fab340b29e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fab2800c4d0 tx=0x7fab2800c990 comp rx=0 tx=0).stop 2024-03-30T00:43:51.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 -- 172.21.15.66:0/4034946418 shutdown_connections 2024-03-30T00:43:51.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab34161140 0x7fab340c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab34160d60 0x7fab3415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab140666c0 0x7fab14068b80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 --2- 172.21.15.66:0/4034946418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab340d1060 0x7fab340b29e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 -- 172.21.15.66:0/4034946418 >> 172.21.15.66:0/4034946418 conn(0x7fab340d0540 msgr2=0x7fab340d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 -- 172.21.15.66:0/4034946418 shutdown_connections 2024-03-30T00:43:51.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.040+0000 7fab3c600640 1 -- 172.21.15.66:0/4034946418 wait complete. 2024-03-30T00:43:51.067 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-03-30T00:43:51.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.204+0000 7f37cba00640 1 Processor -- start 2024-03-30T00:43:51.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.204+0000 7f37cba00640 1 -- start start 2024-03-30T00:43:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37cba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37c40cc580 0x7f37c4167630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37cba00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c40cb0e0 con 0x7f37c415ca80 2024-03-30T00:43:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37cba00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c406bed0 con 0x7f37c40cf2f0 2024-03-30T00:43:51.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37cba00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40cf7f0 con 0x7f37c40cc580 2024-03-30T00:43:51.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37c9000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f37c415ca80 0x7f37c406b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33992/0 (socket says 172.21.15.66:33992) 2024-03-30T00:43:51.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37c9000640 1 -- 172.21.15.66:0/677205569 learned_addr learned my addr 172.21.15.66:0/677205569 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:51.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37c9a00640 1 --2- 172.21.15.66:0/677205569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37c40cc580 0x7f37c4167630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37c3e00640 1 -- 172.21.15.66:0/677205569 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900267568 0 0) 0x7f37c40cb0e0 con 0x7f37c415ca80 2024-03-30T00:43:51.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37c3e00640 1 -- 172.21.15.66:0/677205569 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a8003530 con 0x7f37c415ca80 2024-03-30T00:43:51.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37c9a00640 1 -- 172.21.15.66:0/677205569 >> v1:172.21.15.156:6789/0 conn(0x7f37c40cf2f0 legacy=0x7f37c406b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37c3e00640 1 -- 172.21.15.66:0/677205569 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3747113524 0 0) 0x7f37a8003530 con 0x7f37c415ca80 2024-03-30T00:43:51.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37c9a00640 1 -- 172.21.15.66:0/677205569 >> v1:172.21.15.66:6789/0 conn(0x7f37c415ca80 legacy=0x7f37c406b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.208+0000 7f37c9a00640 1 -- 172.21.15.66:0/677205569 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c40cfad0 con 0x7f37c40cc580 2024-03-30T00:43:51.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37c9a00640 1 --2- 172.21.15.66:0/677205569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37c40cc580 0x7f37c4167630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009dc0 tx=0x7f37bc01c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=447ab1b9146a5751 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37c3e00640 1 -- 172.21.15.66:0/677205569 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4001680 con 0x7f37c40cc580 2024-03-30T00:43:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37c3e00640 1 -- 172.21.15.66:0/677205569 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc0a5650 con 0x7f37c40cc580 2024-03-30T00:43:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37c3e00640 1 -- 172.21.15.66:0/677205569 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0a59e0 con 0x7f37c40cc580 2024-03-30T00:43:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 -- 172.21.15.66:0/677205569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37c40cc580 msgr2=0x7f37c4167630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 --2- 172.21.15.66:0/677205569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37c40cc580 0x7f37c4167630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009dc0 tx=0x7f37bc01c7f0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 -- 172.21.15.66:0/677205569 shutdown_connections 2024-03-30T00:43:51.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 --2- 172.21.15.66:0/677205569 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37c40cc580 0x7f37c4167630 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 -- 172.21.15.66:0/677205569 >> 172.21.15.66:0/677205569 conn(0x7f37c40b2090 msgr2=0x7f37c40b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 -- 172.21.15.66:0/677205569 shutdown_connections 2024-03-30T00:43:51.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 -- 172.21.15.66:0/677205569 wait complete. 2024-03-30T00:43:51.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 Processor -- start 2024-03-30T00:43:51.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 -- start start 2024-03-30T00:43:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37c40cc580 0x7f37c40c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c40cf2f0 0x7f37c415f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37c415ca80 0x7f37c41f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c41f5cd0 con 0x7f37c40cf2f0 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4159370 con 0x7f37c415ca80 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.212+0000 7f37cba00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c41f5980 con 0x7f37c40cc580 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c40cf2f0 0x7f37c415f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c40cf2f0 0x7f37c415f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36878/0 (socket says 172.21.15.66:36878) 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37bbe00640 1 -- 172.21.15.66:0/2960283015 learned_addr learned my addr 172.21.15.66:0/2960283015 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37c9000640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37c40cc580 0x7f37c40c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37bbe00640 1 -- 172.21.15.66:0/2960283015 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37c40cc580 msgr2=0x7f37c40c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37bbe00640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37c40cc580 0x7f37c40c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37bbe00640 1 -- 172.21.15.66:0/2960283015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37c415ca80 msgr2=0x7f37c41f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37bbe00640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37c415ca80 0x7f37c41f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37bbe00640 1 -- 172.21.15.66:0/2960283015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc009a70 con 0x7f37c40cf2f0 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37bbe00640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c40cf2f0 0x7f37c415f5e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009e70 tx=0x7f37ac00c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37c1600640 1 -- 172.21.15.66:0/2960283015 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac018070 con 0x7f37c40cf2f0 2024-03-30T00:43:51.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37c1600640 1 -- 172.21.15.66:0/2960283015 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00cdb0 con 0x7f37c40cf2f0 2024-03-30T00:43:51.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37c1600640 1 -- 172.21.15.66:0/2960283015 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac004260 con 0x7f37c40cf2f0 2024-03-30T00:43:51.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37cba00640 1 -- 172.21.15.66:0/2960283015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c41f28d0 con 0x7f37c40cf2f0 2024-03-30T00:43:51.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37cba00640 1 -- 172.21.15.66:0/2960283015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c41f2e50 con 0x7f37c40cf2f0 2024-03-30T00:43:51.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.216+0000 7f37c1600640 1 -- 172.21.15.66:0/2960283015 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f37ac0087f0 con 0x7f37c40cf2f0 2024-03-30T00:43:51.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c1600640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37a0066690 0x7f37a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c1600640 1 -- 172.21.15.66:0/2960283015 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f37ac0865e0 con 0x7f37c40cf2f0 2024-03-30T00:43:51.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c9000640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37a0066690 0x7f37a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 Processor -- start 2024-03-30T00:43:51.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 -- start start 2024-03-30T00:43:51.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c9000640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37a0066690 0x7f37a0068b50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f37b40017b0 tx=0x7f37b4001640 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 0x5565bb2b7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7bb0 0x5565bb26dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bb136340 con 0x5565bb26e1b0 2024-03-30T00:43:51.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bb0c2cd0 con 0x5565bb2b7290 2024-03-30T00:43:51.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565bb0c02a0 con 0x5565bb2b7bb0 2024-03-30T00:43:51.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36880/0 (socket says 172.21.15.66:36880) 2024-03-30T00:43:51.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c9a00640 1 -- 172.21.15.66:0/2187507178 learned_addr learned my addr 172.21.15.66:0/2187507178 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c9a00640 1 -- 172.21.15.66:0/2187507178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7bb0 msgr2=0x5565bb26dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c9a00640 1 --2- 172.21.15.66:0/2187507178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7bb0 0x5565bb26dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c9a00640 1 -- 172.21.15.66:0/2187507178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 msgr2=0x5565bb2b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c9a00640 1 --2- 172.21.15.66:0/2187507178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 0x5565bb2b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c9a00640 1 -- 172.21.15.66:0/2187507178 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565badae4b0 con 0x5565bb26e1b0 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37c9a00640 1 --2- 172.21.15.66:0/2187507178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26e590 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0b0040 tx=0x7f37bc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689499c245a46b98 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37b9600640 1 -- 172.21.15.66:0/2187507178 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0a5a40 con 0x5565bb26e1b0 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37b9600640 1 -- 172.21.15.66:0/2187507178 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc0a5ba0 con 0x5565bb26e1b0 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37b9600640 1 -- 172.21.15.66:0/2187507178 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0ad440 con 0x5565bb26e1b0 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 -- 172.21.15.66:0/2187507178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 msgr2=0x5565bb26e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 --2- 172.21.15.66:0/2187507178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26e590 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0b0040 tx=0x7f37bc004a50 comp rx=0 tx=0).stop 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 -- 172.21.15.66:0/2187507178 shutdown_connections 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 --2- 172.21.15.66:0/2187507178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26e590 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 --2- 172.21.15.66:0/2187507178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7bb0 0x5565bb26dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 --2- 172.21.15.66:0/2187507178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 0x5565bb2b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 -- 172.21.15.66:0/2187507178 >> 172.21.15.66:0/2187507178 conn(0x5565bb26a160 msgr2=0x5565bb26c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 -- 172.21.15.66:0/2187507178 shutdown_connections 2024-03-30T00:43:51.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.220+0000 7f37cedff000 1 -- 172.21.15.66:0/2187507178 wait complete. 2024-03-30T00:43:51.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 -- start start 2024-03-30T00:43:51.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 Processor -- start 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb273680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7290 0x5565bb273bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7bb0 0x5565bb278e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bb0bbb90 con 0x5565bb26e1b0 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bb0790a0 con 0x5565bb2b7bb0 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565bb078d40 con 0x5565bb2b7290 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 10 client.0 _ll_register_callbacks cb 0x5565bb170c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37c9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb273680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37c9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb273680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36884/0 (socket says 172.21.15.66:36884) 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37c9000640 1 -- 172.21.15.66:0/1508731976 learned_addr learned my addr 172.21.15.66:0/1508731976 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37c9a00640 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7bb0 0x5565bb278e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37bbe00640 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7290 0x5565bb273bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37c9000640 1 -- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7290 msgr2=0x5565bb273bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37c9000640 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7290 0x5565bb273bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37c9000640 1 -- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7bb0 msgr2=0x5565bb278e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37c9000640 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7bb0 0x5565bb278e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37c9000640 1 -- 172.21.15.66:0/1508731976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c40cfad0 con 0x5565bb26e1b0 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37c9000640 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb273680 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f37b400ee30 tx=0x7f37b4001240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b244c9e23b340706 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37b2000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37b2000640 1 -- 172.21.15.66:0/1508731976 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4015110 con 0x5565bb26e1b0 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37b2000640 1 -- 172.21.15.66:0/1508731976 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f37b4015270 con 0x5565bb26e1b0 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 -- 172.21.15.66:0/1508731976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565bb08b550 con 0x5565bb26e1b0 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37b2000640 1 -- 172.21.15.66:0/1508731976 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37b4013420 con 0x5565bb26e1b0 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 10 client.6705 fetch_fsmap learned FSMap version 34 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 -- 172.21.15.66:0/1508731976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x5565bb080ba0 con 0x5565bb26e1b0 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37b2000640 1 -- 172.21.15.66:0/1508731976 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5565bb08b550 con 0x5565bb26e1b0 2024-03-30T00:43:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37b2000640 1 -- 172.21.15.66:0/1508731976 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f37b4013580 con 0x5565bb26e1b0 2024-03-30T00:43:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 10 client.6705 fetch_fsmap finished waiting for FSMap version 34 2024-03-30T00:43:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 10 client.6705 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T00:43:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 20 client.6705 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5565bb2c6320 0x5565bb2c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 4 client.6705 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:43:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37cedff000 1 -- 172.21.15.66:0/1508731976 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565bb0b8970 con 0x5565bb2c6320 2024-03-30T00:43:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.224+0000 7f37bbe00640 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5565bb2c6320 0x5565bb2c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37bbe00640 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5565bb2c6320 0x5565bb2c8710 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46acff02f4818ef6 server_cookie=cadfdfb9b435f8b5 in_seq=0 out_seq=0 2024-03-30T00:43:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37b2000640 10 client.6705 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:51.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37b2000640 1 -- 172.21.15.66:0/1508731976 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5565bb0b8970 con 0x5565bb2c6320 2024-03-30T00:43:51.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37b2000640 10 client.6705 handle_command_reply: tid=0 2024-03-30T00:43:51.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 1 client.6705 shutdown 2024-03-30T00:43:51.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 1 -- 172.21.15.66:0/1508731976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 msgr2=0x5565bb273680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb273680 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f37b400ee30 tx=0x7f37b4001240 comp rx=0 tx=0).stop 2024-03-30T00:43:51.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 1 -- 172.21.15.66:0/1508731976 shutdown_connections 2024-03-30T00:43:51.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5565bb2c6320 0x5565bb2c8710 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7bb0 0x5565bb278e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7290 0x5565bb273bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 1 --2- 172.21.15.66:0/1508731976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb273680 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 1 -- 172.21.15.66:0/1508731976 >> 172.21.15.66:0/1508731976 conn(0x5565bb26a160 msgr2=0x5565bb26cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37b2000640 0 client.6705 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:51.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 1 -- 172.21.15.66:0/1508731976 shutdown_connections 2024-03-30T00:43:51.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 1 -- 172.21.15.66:0/1508731976 wait complete. 2024-03-30T00:43:51.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.228+0000 7f37cedff000 20 client.6705 trim_cache size 0 max 16384 2024-03-30T00:43:51.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37cedff000 1 Processor -- start 2024-03-30T00:43:51.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37cedff000 1 -- start start 2024-03-30T00:43:51.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37cedff000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37cedff000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 0x5565bb26daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37cedff000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7bb0 0x5565bb2c6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37cedff000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bad78c40 con 0x5565bb26e1b0 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37cedff000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bb086ad0 con 0x5565bb2b7290 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37cedff000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565bb0baa20 con 0x5565bb2b7bb0 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37c9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37c9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36900/0 (socket says 172.21.15.66:36900) 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37c9000640 1 -- 172.21.15.66:0/1068049502 learned_addr learned my addr 172.21.15.66:0/1068049502 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37bbe00640 1 --2- 172.21.15.66:0/1068049502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 0x5565bb26daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37c9a00640 1 --2- 172.21.15.66:0/1068049502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7bb0 0x5565bb2c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37c9000640 1 -- 172.21.15.66:0/1068049502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7bb0 msgr2=0x5565bb2c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37c9000640 1 --2- 172.21.15.66:0/1068049502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7bb0 0x5565bb2c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37c9000640 1 -- 172.21.15.66:0/1068049502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 msgr2=0x5565bb26daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37c9000640 1 --2- 172.21.15.66:0/1068049502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 0x5565bb26daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37c9000640 1 -- 172.21.15.66:0/1068049502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565bb066ef0 con 0x5565bb26e1b0 2024-03-30T00:43:51.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37c9a00640 1 --2- 172.21.15.66:0/1068049502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7bb0 0x5565bb2c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37c9000640 1 --2- 172.21.15.66:0/1068049502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26d560 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f37b4001b10 tx=0x7f37b401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21b9d249d68e487 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37ba000640 1 -- 172.21.15.66:0/1068049502 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4013ae0 con 0x5565bb26e1b0 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37ba000640 1 -- 172.21.15.66:0/1068049502 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b400b070 con 0x5565bb26e1b0 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.240+0000 7f37ba000640 1 -- 172.21.15.66:0/1068049502 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4015110 con 0x5565bb26e1b0 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 -- 172.21.15.66:0/1068049502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 msgr2=0x5565bb26d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 --2- 172.21.15.66:0/1068049502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26d560 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f37b4001b10 tx=0x7f37b401d000 comp rx=0 tx=0).stop 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 -- 172.21.15.66:0/1068049502 shutdown_connections 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 --2- 172.21.15.66:0/1068049502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb2b7bb0 0x5565bb2c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 --2- 172.21.15.66:0/1068049502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 0x5565bb26daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 --2- 172.21.15.66:0/1068049502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb26e1b0 0x5565bb26d560 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 -- 172.21.15.66:0/1068049502 >> 172.21.15.66:0/1068049502 conn(0x5565bb26a160 msgr2=0x5565bb273540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 -- 172.21.15.66:0/1068049502 shutdown_connections 2024-03-30T00:43:51.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 -- 172.21.15.66:0/1068049502 wait complete. 2024-03-30T00:43:51.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 -- start start 2024-03-30T00:43:51.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 Processor -- start 2024-03-30T00:43:51.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb26e1b0 0x5565bb2c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 0x5565bb315de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb2b7bb0 0x5565bb316320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bb058a40 con 0x5565bb2b7bb0 2024-03-30T00:43:51.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bb0b93a0 con 0x5565bb2b7290 2024-03-30T00:43:51.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565bb0be6f0 con 0x5565bb26e1b0 2024-03-30T00:43:51.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 10 client.0 _ll_register_callbacks cb 0x5565bb159870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:51.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37c9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb2b7bb0 0x5565bb316320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37c9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb2b7bb0 0x5565bb316320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36912/0 (socket says 172.21.15.66:36912) 2024-03-30T00:43:51.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37c9a00640 1 -- 172.21.15.66:0/1219221272 learned_addr learned my addr 172.21.15.66:0/1219221272 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37bbe00640 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 0x5565bb315de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37c9000640 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb26e1b0 0x5565bb2c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37c9a00640 1 -- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb26e1b0 msgr2=0x5565bb2c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37c9a00640 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb26e1b0 0x5565bb2c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37c9a00640 1 -- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 msgr2=0x5565bb315de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37c9a00640 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 0x5565bb315de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37c9a00640 1 -- 172.21.15.66:0/1219221272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565bb080ba0 con 0x5565bb2b7bb0 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37c9a00640 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb2b7bb0 0x5565bb316320 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f37bc01ccd0 tx=0x7f37bc01b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0651c013a93290 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37b2000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37b2000640 1 -- 172.21.15.66:0/1219221272 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0c7070 con 0x5565bb2b7bb0 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37b2000640 1 -- 172.21.15.66:0/1219221272 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f37bc0a5d00 con 0x5565bb2b7bb0 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 -- 172.21.15.66:0/1219221272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565bb06b620 con 0x5565bb2b7bb0 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37b2000640 1 -- 172.21.15.66:0/1219221272 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37bc004040 con 0x5565bb2b7bb0 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 10 client.6711 fetch_fsmap learned FSMap version 34 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37cedff000 1 -- 172.21.15.66:0/1219221272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x5565badb7fc0 con 0x5565bb2b7bb0 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37b2000640 1 -- 172.21.15.66:0/1219221272 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5565bb06b620 con 0x5565bb2b7bb0 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.244+0000 7f37b2000640 1 -- 172.21.15.66:0/1219221272 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f37bc0041a0 con 0x5565bb2b7bb0 2024-03-30T00:43:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 10 client.6711 fetch_fsmap finished waiting for FSMap version 34 2024-03-30T00:43:51.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 10 client.6711 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T00:43:51.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 20 client.6711 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:51.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5565bb316860 0x5565bb312870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 4 client.6711 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-30T00:43:51.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 -- 172.21.15.66:0/1219221272 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5565bb0ba4e0 con 0x5565bb316860 2024-03-30T00:43:51.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37c9000640 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5565bb316860 0x5565bb312870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37c9000640 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5565bb316860 0x5565bb312870 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5678e474dc132f8 server_cookie=1c912c508ba45fb9 in_seq=0 out_seq=0 2024-03-30T00:43:51.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37b2000640 10 client.6711 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:51.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37b2000640 1 -- 172.21.15.66:0/1219221272 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+140 (crc 0 0 0) 0x5565bb0ba4e0 con 0x5565bb316860 2024-03-30T00:43:51.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37b2000640 10 client.6711 handle_command_reply: tid=0 2024-03-30T00:43:51.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 client.6711 shutdown 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 -- 172.21.15.66:0/1219221272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb2b7bb0 msgr2=0x5565bb316320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb2b7bb0 0x5565bb316320 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f37bc01ccd0 tx=0x7f37bc01b030 comp rx=0 tx=0).stop 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 -- 172.21.15.66:0/1219221272 shutdown_connections 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5565bb316860 0x5565bb312870 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565bb2b7bb0 0x5565bb316320 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565bb2b7290 0x5565bb315de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 --2- 172.21.15.66:0/1219221272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5565bb26e1b0 0x5565bb2c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 -- 172.21.15.66:0/1219221272 >> 172.21.15.66:0/1219221272 conn(0x5565bb26a160 msgr2=0x5565bb26c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37b2000640 0 client.6711 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 -- 172.21.15.66:0/1219221272 shutdown_connections 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 1 -- 172.21.15.66:0/1219221272 wait complete. 2024-03-30T00:43:51.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cedff000 20 client.6711 trim_cache size 0 max 16384 2024-03-30T00:43:51.259 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:51.259 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:43:51.259 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:43:51.259 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2024-03-30T00:43:51.259 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 574781027, 2024-03-30T00:43:51.259 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627777, 2024-03-30T00:43:51.259 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/dir_test/file" 2024-03-30T00:43:51.259 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:43:51.259 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:43:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 -- 172.21.15.66:0/2960283015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37a0066690 msgr2=0x7f37a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37a0066690 0x7f37a0068b50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f37b40017b0 tx=0x7f37b4001640 comp rx=0 tx=0).stop 2024-03-30T00:43:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 -- 172.21.15.66:0/2960283015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c40cf2f0 msgr2=0x7f37c415f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c40cf2f0 0x7f37c415f5e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009e70 tx=0x7f37ac00c430 comp rx=0 tx=0).stop 2024-03-30T00:43:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 -- 172.21.15.66:0/2960283015 shutdown_connections 2024-03-30T00:43:51.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37c415ca80 0x7f37c41f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37a0066690 0x7f37a0068b50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c40cf2f0 0x7f37c415f5e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 --2- 172.21.15.66:0/2960283015 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37c40cc580 0x7f37c40c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 -- 172.21.15.66:0/2960283015 >> 172.21.15.66:0/2960283015 conn(0x7f37c40b2090 msgr2=0x7f37c4098a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 -- 172.21.15.66:0/2960283015 shutdown_connections 2024-03-30T00:43:51.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.248+0000 7f37cba00640 1 -- 172.21.15.66:0/2960283015 wait complete. 2024-03-30T00:43:51.274 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-30T00:43:51.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbf4a00640 1 Processor -- start 2024-03-30T00:43:51.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbf4a00640 1 -- start start 2024-03-30T00:43:51.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbf4a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcbf006e800 0x7fcbf00d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf006f8e0 0x7fcbf00ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbf4a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf006b0b0 con 0x7fcbf00cc560 2024-03-30T00:43:51.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbf4a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf0160640 con 0x7fcbf006e800 2024-03-30T00:43:51.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbf4a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf006b390 con 0x7fcbf006f8e0 2024-03-30T00:43:51.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbee800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf006f8e0 0x7fcbf00ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbed400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcbf006e800 0x7fcbf00d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbee800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf006f8e0 0x7fcbf00ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:50954/0 (socket says 172.21.15.66:50954) 2024-03-30T00:43:51.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbee800640 1 -- 172.21.15.66:0/506632082 learned_addr learned my addr 172.21.15.66:0/506632082 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbee800640 1 -- 172.21.15.66:0/506632082 >> v2:172.21.15.156:3300/0 conn(0x7fcbf006e800 msgr2=0x7fcbf00d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbee800640 1 --2- 172.21.15.66:0/506632082 >> v2:172.21.15.156:3300/0 conn(0x7fcbf006e800 0x7fcbf00d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbee800640 1 -- 172.21.15.66:0/506632082 >> v1:172.21.15.156:6789/0 conn(0x7fcbf00cc560 legacy=0x7fcbf006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.408+0000 7fcbee800640 1 -- 172.21.15.66:0/506632082 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf006b670 con 0x7fcbf006f8e0 2024-03-30T00:43:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbee800640 1 --2- 172.21.15.66:0/506632082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf006f8e0 0x7fcbf00ce8d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009e50 tx=0x7fcbe801ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7120abf7eb7359f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbeca00640 1 -- 172.21.15.66:0/506632082 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80a58d0 con 0x7fcbf006f8e0 2024-03-30T00:43:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbeca00640 1 -- 172.21.15.66:0/506632082 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe80a5a30 con 0x7fcbf006f8e0 2024-03-30T00:43:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbeca00640 1 -- 172.21.15.66:0/506632082 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80a5dc0 con 0x7fcbf006f8e0 2024-03-30T00:43:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbf4a00640 1 -- 172.21.15.66:0/506632082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf006f8e0 msgr2=0x7fcbf00ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbf4a00640 1 --2- 172.21.15.66:0/506632082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf006f8e0 0x7fcbf00ce8d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009e50 tx=0x7fcbe801ca00 comp rx=0 tx=0).stop 2024-03-30T00:43:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbf4a00640 1 -- 172.21.15.66:0/506632082 shutdown_connections 2024-03-30T00:43:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbf4a00640 1 --2- 172.21.15.66:0/506632082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf006f8e0 0x7fcbf00ce8d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbf4a00640 1 --2- 172.21.15.66:0/506632082 >> v2:172.21.15.156:3300/0 conn(0x7fcbf006e800 0x7fcbf00d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbf4a00640 1 -- 172.21.15.66:0/506632082 >> 172.21.15.66:0/506632082 conn(0x7fcbf00d0540 msgr2=0x7fcbf00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbf4a00640 1 -- 172.21.15.66:0/506632082 shutdown_connections 2024-03-30T00:43:51.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.412+0000 7fcbf4a00640 1 -- 172.21.15.66:0/506632082 wait complete. 2024-03-30T00:43:51.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbf4a00640 1 Processor -- start 2024-03-30T00:43:51.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbf4a00640 1 -- start start 2024-03-30T00:43:51.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbf006e800 0x7fcbf014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbf006f8e0 0x7fcbf014fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf00cc560 0x7fcbf01516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbf4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf014dfe0 con 0x7fcbf006f8e0 2024-03-30T00:43:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbf4a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00ca470 con 0x7fcbf006e800 2024-03-30T00:43:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbf4a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00c98f0 con 0x7fcbf00cc560 2024-03-30T00:43:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbf006f8e0 0x7fcbf014fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbf006f8e0 0x7fcbf014fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36924/0 (socket says 172.21.15.66:36924) 2024-03-30T00:43:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbed400640 1 -- 172.21.15.66:0/3834421171 learned_addr learned my addr 172.21.15.66:0/3834421171 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbee800640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf00cc560 0x7fcbf01516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbede00640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbf006e800 0x7fcbf014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbed400640 1 -- 172.21.15.66:0/3834421171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf00cc560 msgr2=0x7fcbf01516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbed400640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf00cc560 0x7fcbf01516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbed400640 1 -- 172.21.15.66:0/3834421171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbf006e800 msgr2=0x7fcbf014daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbed400640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbf006e800 0x7fcbf014daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbed400640 1 -- 172.21.15.66:0/3834421171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe8009b30 con 0x7fcbf006f8e0 2024-03-30T00:43:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbed400640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbf006f8e0 0x7fcbf014fbf0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00e930 tx=0x7fcbdc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbe6000640 1 -- 172.21.15.66:0/3834421171 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc009690 con 0x7fcbf006f8e0 2024-03-30T00:43:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbe6000640 1 -- 172.21.15.66:0/3834421171 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc00ca00 con 0x7fcbf006f8e0 2024-03-30T00:43:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbe6000640 1 -- 172.21.15.66:0/3834421171 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc00cd90 con 0x7fcbf006f8e0 2024-03-30T00:43:51.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbf4a00640 1 -- 172.21.15.66:0/3834421171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf00c9bd0 con 0x7fcbf006f8e0 2024-03-30T00:43:51.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbf4a00640 1 -- 172.21.15.66:0/3834421171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf00ca190 con 0x7fcbf006f8e0 2024-03-30T00:43:51.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.416+0000 7fcbf4a00640 1 -- 172.21.15.66:0/3834421171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf0066c30 con 0x7fcbf006f8e0 2024-03-30T00:43:51.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.420+0000 7fcbe6000640 1 -- 172.21.15.66:0/3834421171 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcbdc013470 con 0x7fcbf006f8e0 2024-03-30T00:43:51.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.420+0000 7fcbe6000640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbd0066690 0x7fcbd0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.420+0000 7fcbe6000640 1 -- 172.21.15.66:0/3834421171 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fcbdc084560 con 0x7fcbf006f8e0 2024-03-30T00:43:51.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.420+0000 7fcbede00640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbd0066690 0x7fcbd0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.420+0000 7fcbede00640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbd0066690 0x7fcbd0068b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0007fd0 tx=0x7fcbe0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.424+0000 7fcbe6000640 1 -- 172.21.15.66:0/3834421171 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcbdc051860 con 0x7fcbf006f8e0 2024-03-30T00:43:51.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.616+0000 7fcbf4a00640 1 -- 172.21.15.66:0/3834421171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcbf00aeb60 con 0x7fcbf006f8e0 2024-03-30T00:43:51.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.616+0000 7fcbe6000640 1 -- 172.21.15.66:0/3834421171 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fcbdc054e80 con 0x7fcbf006f8e0 2024-03-30T00:43:51.625 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:51.625 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-30T00:43:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 -- 172.21.15.66:0/3834421171 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbd0066690 msgr2=0x7fcbd0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbd0066690 0x7fcbd0068b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0007fd0 tx=0x7fcbe0009040 comp rx=0 tx=0).stop 2024-03-30T00:43:51.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 -- 172.21.15.66:0/3834421171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbf006f8e0 msgr2=0x7fcbf014fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbf006f8e0 0x7fcbf014fbf0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00e930 tx=0x7fcbdc00edf0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 -- 172.21.15.66:0/3834421171 shutdown_connections 2024-03-30T00:43:51.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbf00cc560 0x7fcbf01516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbd0066690 0x7fcbd0068b50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbf006f8e0 0x7fcbf014fbf0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 --2- 172.21.15.66:0/3834421171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbf006e800 0x7fcbf014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 -- 172.21.15.66:0/3834421171 >> 172.21.15.66:0/3834421171 conn(0x7fcbf00d0540 msgr2=0x7fcbf015bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 -- 172.21.15.66:0/3834421171 shutdown_connections 2024-03-30T00:43:51.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.620+0000 7fcbf4a00640 1 -- 172.21.15.66:0/3834421171 wait complete. 2024-03-30T00:43:51.645 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-30T00:43:51.646 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-03-30T00:43:51.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b52800640 1 Processor -- start 2024-03-30T00:43:51.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b52800640 1 -- start start 2024-03-30T00:43:51.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b52800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1b4c1594f0 0x7f1b4c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b52800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1b4c0cec60 0x7f1b4c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b52800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4c0b13c0 con 0x7f1b4c0ce7b0 2024-03-30T00:43:51.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b52800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c0b1a60 con 0x7f1b4c0cec60 2024-03-30T00:43:51.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b52800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c0cda90 con 0x7f1b4c1594f0 2024-03-30T00:43:51.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b4be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1b4c0cec60 0x7f1b4c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b4be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1b4c0cec60 0x7f1b4c159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36932/0 (socket says 172.21.15.66:36932) 2024-03-30T00:43:51.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b4be00640 1 -- 172.21.15.66:0/2976279599 learned_addr learned my addr 172.21.15.66:0/2976279599 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b4b400640 1 --2- 172.21.15.66:0/2976279599 >> v2:172.21.15.156:3300/0 conn(0x7f1b4c1594f0 0x7f1b4c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b4be00640 1 -- 172.21.15.66:0/2976279599 >> v2:172.21.15.156:3300/0 conn(0x7f1b4c1594f0 msgr2=0x7f1b4c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b4be00640 1 --2- 172.21.15.66:0/2976279599 >> v2:172.21.15.156:3300/0 conn(0x7f1b4c1594f0 0x7f1b4c0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b4be00640 1 -- 172.21.15.66:0/2976279599 >> v1:172.21.15.66:6789/0 conn(0x7f1b4c0ce7b0 legacy=0x7f1b4c1598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b4be00640 1 -- 172.21.15.66:0/2976279599 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c15a810 con 0x7f1b4c0cec60 2024-03-30T00:43:51.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.784+0000 7f1b4aa00640 1 -- 172.21.15.66:0/2976279599 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903476969 0 0) 0x7f1b4c0b13c0 con 0x7f1b4c0ce7b0 2024-03-30T00:43:51.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b4be00640 1 --2- 172.21.15.66:0/2976279599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b4c0cec60 0x7f1b4c159fe0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0cf3a0 tx=0x7f1b4401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91576a8bbf03704 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b4aa00640 1 -- 172.21.15.66:0/2976279599 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440a9400 con 0x7f1b4c0cec60 2024-03-30T00:43:51.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b4aa00640 1 -- 172.21.15.66:0/2976279599 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b440a9560 con 0x7f1b4c0cec60 2024-03-30T00:43:51.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b4aa00640 1 -- 172.21.15.66:0/2976279599 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440a98f0 con 0x7f1b4c0cec60 2024-03-30T00:43:51.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b52800640 1 -- 172.21.15.66:0/2976279599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b4c0cec60 msgr2=0x7f1b4c159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b52800640 1 --2- 172.21.15.66:0/2976279599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b4c0cec60 0x7f1b4c159fe0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0cf3a0 tx=0x7f1b4401c520 comp rx=0 tx=0).stop 2024-03-30T00:43:51.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b52800640 1 -- 172.21.15.66:0/2976279599 shutdown_connections 2024-03-30T00:43:51.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b52800640 1 --2- 172.21.15.66:0/2976279599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b4c0cec60 0x7f1b4c159fe0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b52800640 1 --2- 172.21.15.66:0/2976279599 >> v2:172.21.15.156:3300/0 conn(0x7f1b4c1594f0 0x7f1b4c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b52800640 1 -- 172.21.15.66:0/2976279599 >> 172.21.15.66:0/2976279599 conn(0x7f1b4c06a190 msgr2=0x7f1b4c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b52800640 1 -- 172.21.15.66:0/2976279599 shutdown_connections 2024-03-30T00:43:51.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.788+0000 7f1b52800640 1 -- 172.21.15.66:0/2976279599 wait complete. 2024-03-30T00:43:51.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b52800640 1 Processor -- start 2024-03-30T00:43:51.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b52800640 1 -- start start 2024-03-30T00:43:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b52800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b4c0ce7b0 0x7f1b4c0c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b52800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b4c0cec60 0x7f1b4c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b52800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b4c1594f0 0x7f1b4c0c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b52800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0c8ff0 con 0x7f1b4c0ce7b0 2024-03-30T00:43:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b52800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0cbec0 con 0x7f1b4c0cec60 2024-03-30T00:43:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b52800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0cb290 con 0x7f1b4c1594f0 2024-03-30T00:43:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b4b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b4c0ce7b0 0x7f1b4c0c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b4b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b4c0ce7b0 0x7f1b4c0c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36938/0 (socket says 172.21.15.66:36938) 2024-03-30T00:43:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b4b400640 1 -- 172.21.15.66:0/644366891 learned_addr learned my addr 172.21.15.66:0/644366891 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b4b400640 1 -- 172.21.15.66:0/644366891 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b4c1594f0 msgr2=0x7f1b4c0c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b4b400640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b4c1594f0 0x7f1b4c0c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b4b400640 1 -- 172.21.15.66:0/644366891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b4c0cec60 msgr2=0x7f1b4c15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b4b400640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b4c0cec60 0x7f1b4c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b4b400640 1 -- 172.21.15.66:0/644366891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b440094d0 con 0x7f1b4c0ce7b0 2024-03-30T00:43:51.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b4b400640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b4c0ce7b0 0x7f1b4c0c99d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c007e40 tx=0x7f1b3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b43e00640 1 -- 172.21.15.66:0/644366891 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c014620 con 0x7f1b4c0ce7b0 2024-03-30T00:43:51.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b43e00640 1 -- 172.21.15.66:0/644366891 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c00ee90 con 0x7f1b4c0ce7b0 2024-03-30T00:43:51.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b52800640 1 -- 172.21.15.66:0/644366891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c0c9540 con 0x7f1b4c0ce7b0 2024-03-30T00:43:51.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b43e00640 1 -- 172.21.15.66:0/644366891 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c0149c0 con 0x7f1b4c0ce7b0 2024-03-30T00:43:51.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.792+0000 7f1b52800640 1 -- 172.21.15.66:0/644366891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c06dcb0 con 0x7f1b4c0ce7b0 2024-03-30T00:43:51.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.796+0000 7f1b43e00640 1 -- 172.21.15.66:0/644366891 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1b3c014b20 con 0x7f1b4c0ce7b0 2024-03-30T00:43:51.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.796+0000 7f1b43e00640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b280666c0 0x7f1b28068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.796+0000 7f1b43e00640 1 -- 172.21.15.66:0/644366891 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f1b3c084c00 con 0x7f1b4c0ce7b0 2024-03-30T00:43:51.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.796+0000 7f1b42a00640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b280666c0 0x7f1b28068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.796+0000 7f1b55bde000 1 Processor -- start 2024-03-30T00:43:51.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.796+0000 7f1b55bde000 1 -- start start 2024-03-30T00:43:51.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.796+0000 7f1b42a00640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b280666c0 0x7f1b28068b80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1b34007fd0 tx=0x7f1b34007920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e4b20 0x55cbfd4e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd4e5440 0x55cbfd49a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd49ae80 0x55cbfd49b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd3635c0 con 0x55cbfd4e4b20 2024-03-30T00:43:51.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd2efa00 con 0x55cbfd4e5440 2024-03-30T00:43:51.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd2ecfd0 con 0x55cbfd49ae80 2024-03-30T00:43:51.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b4b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e4b20 0x55cbfd4e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b4b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e4b20 0x55cbfd4e4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36944/0 (socket says 172.21.15.66:36944) 2024-03-30T00:43:51.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b4b400640 1 -- 172.21.15.66:0/228913696 learned_addr learned my addr 172.21.15.66:0/228913696 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b42a00640 1 --2- 172.21.15.66:0/228913696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd4e5440 0x55cbfd49a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b4be00640 1 --2- 172.21.15.66:0/228913696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd49ae80 0x55cbfd49b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b4b400640 1 -- 172.21.15.66:0/228913696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd49ae80 msgr2=0x55cbfd49b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b4b400640 1 --2- 172.21.15.66:0/228913696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd49ae80 0x55cbfd49b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b4b400640 1 -- 172.21.15.66:0/228913696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd4e5440 msgr2=0x55cbfd49a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b4b400640 1 --2- 172.21.15.66:0/228913696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd4e5440 0x55cbfd49a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b4b400640 1 -- 172.21.15.66:0/228913696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbfd2898c0 con 0x55cbfd4e4b20 2024-03-30T00:43:51.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b4b400640 1 --2- 172.21.15.66:0/228913696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e4b20 0x55cbfd4e4f00 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c05d9b0 tx=0x7f1b3c0561f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccf963ecde46ce3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b3be00640 1 -- 172.21.15.66:0/228913696 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c05dee0 con 0x55cbfd4e4b20 2024-03-30T00:43:51.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b3be00640 1 -- 172.21.15.66:0/228913696 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c053e90 con 0x55cbfd4e4b20 2024-03-30T00:43:51.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b3be00640 1 -- 172.21.15.66:0/228913696 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c054fe0 con 0x55cbfd4e4b20 2024-03-30T00:43:51.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 -- 172.21.15.66:0/228913696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e4b20 msgr2=0x55cbfd4e4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 --2- 172.21.15.66:0/228913696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e4b20 0x55cbfd4e4f00 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c05d9b0 tx=0x7f1b3c0561f0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 -- 172.21.15.66:0/228913696 shutdown_connections 2024-03-30T00:43:51.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 --2- 172.21.15.66:0/228913696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd49ae80 0x55cbfd49b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 --2- 172.21.15.66:0/228913696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd4e5440 0x55cbfd49a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 --2- 172.21.15.66:0/228913696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e4b20 0x55cbfd4e4f00 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 -- 172.21.15.66:0/228913696 >> 172.21.15.66:0/228913696 conn(0x55cbfd4a1400 msgr2=0x55cbfd499220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 -- 172.21.15.66:0/228913696 shutdown_connections 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 -- 172.21.15.66:0/228913696 wait complete. 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 -- start start 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.800+0000 7f1b55bde000 1 Processor -- start 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd49ae80 0x55cbfd4a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd4a6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4a6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd2e88c0 con 0x55cbfd4e5440 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd288160 con 0x55cbfd49ae80 2024-03-30T00:43:51.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd287e00 con 0x55cbfd4e4b20 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 10 client.0 _ll_register_callbacks cb 0x55cbfd486820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36956/0 (socket says 172.21.15.66:36956) 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd49ae80 0x55cbfd4a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4be00640 1 -- 172.21.15.66:0/3145811584 learned_addr learned my addr 172.21.15.66:0/3145811584 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b42a00640 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd4a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4be00640 1 -- 172.21.15.66:0/3145811584 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 msgr2=0x55cbfd4a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4be00640 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd4a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4be00640 1 -- 172.21.15.66:0/3145811584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd49ae80 msgr2=0x55cbfd4a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4be00640 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd49ae80 0x55cbfd4a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4be00640 1 -- 172.21.15.66:0/3145811584 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c06dcb0 con 0x55cbfd4e5440 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4be00640 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4a6770 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1b4401ca00 tx=0x7f1b440046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157cbbba0a1ec975 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b38c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:51.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b38c00640 1 -- 172.21.15.66:0/3145811584 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440a9ae0 con 0x55cbfd4e5440 2024-03-30T00:43:51.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b38c00640 1 -- 172.21.15.66:0/3145811584 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f1b440b6070 con 0x55cbfd4e5440 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 1 -- 172.21.15.66:0/3145811584 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbfd2e7210 con 0x55cbfd4e5440 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b38c00640 1 -- 172.21.15.66:0/3145811584 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b440a9c40 con 0x55cbfd4e5440 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 10 client.6729 fetch_fsmap learned FSMap version 34 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 1 -- 172.21.15.66:0/3145811584 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55cbfcfe4d50 con 0x55cbfd4e5440 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b38c00640 1 -- 172.21.15.66:0/3145811584 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbfd2e7210 con 0x55cbfd4e5440 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b38c00640 1 -- 172.21.15.66:0/3145811584 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1b440bc430 con 0x55cbfd4e5440 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 10 client.6729 fetch_fsmap finished waiting for FSMap version 34 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 10 client.6729 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 20 client.6729 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55cbfd4f3dc0 0x55cbfd4f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 4 client.6729 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b55bde000 1 -- 172.21.15.66:0/3145811584 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbfd2a6aa0 con 0x55cbfd4f3dc0 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4b400640 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55cbfd4f3dc0 0x55cbfd4f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b4b400640 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55cbfd4f3dc0 0x55cbfd4f61b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf08c82e17317ef server_cookie=e451dbffe34112b0 in_seq=0 out_seq=0 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b38c00640 10 client.6729 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b38c00640 1 -- 172.21.15.66:0/3145811584 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cbfd2a6aa0 con 0x55cbfd4f3dc0 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.804+0000 7f1b38c00640 10 client.6729 handle_command_reply: tid=0 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 1 client.6729 shutdown 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 1 -- 172.21.15.66:0/3145811584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 msgr2=0x55cbfd4a6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4a6770 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1b4401ca00 tx=0x7f1b440046d0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 1 -- 172.21.15.66:0/3145811584 shutdown_connections 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55cbfd4f3dc0 0x55cbfd4f61b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4a6770 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd4a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b38c00640 0 client.6729 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 1 --2- 172.21.15.66:0/3145811584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd49ae80 0x55cbfd4a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 1 -- 172.21.15.66:0/3145811584 >> 172.21.15.66:0/3145811584 conn(0x55cbfd4a1400 msgr2=0x55cbfd499920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 1 -- 172.21.15.66:0/3145811584 shutdown_connections 2024-03-30T00:43:51.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 1 -- 172.21.15.66:0/3145811584 wait complete. 2024-03-30T00:43:51.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.808+0000 7f1b55bde000 20 client.6729 trim_cache size 0 max 16384 2024-03-30T00:43:51.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.820+0000 7f1b55bde000 1 Processor -- start 2024-03-30T00:43:51.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.820+0000 7f1b55bde000 1 -- start start 2024-03-30T00:43:51.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd49ae80 0x55cbfd4e3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd4e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4ff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfcfa59d0 con 0x55cbfd4e5440 2024-03-30T00:43:51.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd2b0750 con 0x55cbfd49ae80 2024-03-30T00:43:51.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd2afef0 con 0x55cbfd4e4b20 2024-03-30T00:43:51.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4ff570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36962/0 (socket says 172.21.15.66:36962) 2024-03-30T00:43:51.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4be00640 1 -- 172.21.15.66:0/3508286634 learned_addr learned my addr 172.21.15.66:0/3508286634 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4b400640 1 --2- 172.21.15.66:0/3508286634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd49ae80 0x55cbfd4e3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b42a00640 1 --2- 172.21.15.66:0/3508286634 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd4e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4be00640 1 -- 172.21.15.66:0/3508286634 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 msgr2=0x55cbfd4e3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4be00640 1 --2- 172.21.15.66:0/3508286634 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd4e3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4be00640 1 -- 172.21.15.66:0/3508286634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd49ae80 msgr2=0x55cbfd4e3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4be00640 1 --2- 172.21.15.66:0/3508286634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd49ae80 0x55cbfd4e3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4be00640 1 -- 172.21.15.66:0/3508286634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbfd2adba0 con 0x55cbfd4e5440 2024-03-30T00:43:51.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4b400640 1 --2- 172.21.15.66:0/3508286634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd49ae80 0x55cbfd4e3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:51.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4be00640 1 --2- 172.21.15.66:0/3508286634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4ff570 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1b44001ec0 tx=0x7f1b440a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5262ea54ee8296ad server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b40c00640 1 -- 172.21.15.66:0/3508286634 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440bcb30 con 0x55cbfd4e5440 2024-03-30T00:43:51.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b40c00640 1 -- 172.21.15.66:0/3508286634 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b440bcc90 con 0x55cbfd4e5440 2024-03-30T00:43:51.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b40c00640 1 -- 172.21.15.66:0/3508286634 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44004260 con 0x55cbfd4e5440 2024-03-30T00:43:51.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- 172.21.15.66:0/3508286634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 msgr2=0x55cbfd4ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 --2- 172.21.15.66:0/3508286634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4ff570 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1b44001ec0 tx=0x7f1b440a9000 comp rx=0 tx=0).stop 2024-03-30T00:43:51.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- 172.21.15.66:0/3508286634 shutdown_connections 2024-03-30T00:43:51.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 --2- 172.21.15.66:0/3508286634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd4e5440 0x55cbfd4ff570 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 --2- 172.21.15.66:0/3508286634 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd4e3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 --2- 172.21.15.66:0/3508286634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd49ae80 0x55cbfd4e3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- 172.21.15.66:0/3508286634 >> 172.21.15.66:0/3508286634 conn(0x55cbfd4a1400 msgr2=0x55cbfd4f6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- 172.21.15.66:0/3508286634 shutdown_connections 2024-03-30T00:43:51.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- 172.21.15.66:0/3508286634 wait complete. 2024-03-30T00:43:51.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- start start 2024-03-30T00:43:51.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 Processor -- start 2024-03-30T00:43:51.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd49ae80 0x55cbfd547000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd547540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd4e5440 0x55cbfd501d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd2e60d0 con 0x55cbfd49ae80 2024-03-30T00:43:51.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd49f650 con 0x55cbfd4e5440 2024-03-30T00:43:51.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbfd49dca0 con 0x55cbfd4e4b20 2024-03-30T00:43:51.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b55bde000 10 client.0 _ll_register_callbacks cb 0x55cbfd296f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:51.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd49ae80 0x55cbfd547000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd49ae80 0x55cbfd547000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36974/0 (socket says 172.21.15.66:36974) 2024-03-30T00:43:51.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.824+0000 7f1b4b400640 1 -- 172.21.15.66:0/2462866021 learned_addr learned my addr 172.21.15.66:0/2462866021 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:51.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b42a00640 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd547540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b4be00640 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd4e5440 0x55cbfd501d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b4b400640 1 -- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 msgr2=0x55cbfd547540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b4b400640 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd547540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b4b400640 1 -- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd4e5440 msgr2=0x55cbfd501d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:51.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b4b400640 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd4e5440 0x55cbfd501d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b4b400640 1 -- 172.21.15.66:0/2462866021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbfcfe4d50 con 0x55cbfd49ae80 2024-03-30T00:43:51.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b4be00640 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd4e5440 0x55cbfd501d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:51.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b4b400640 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd49ae80 0x55cbfd547000 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c051fa0 tx=0x7f1b3c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb9f276c8d5cc3a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:51.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b38c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:51.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b38c00640 1 -- 172.21.15.66:0/2462866021 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c05dee0 con 0x55cbfd49ae80 2024-03-30T00:43:51.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b38c00640 1 -- 172.21.15.66:0/2462866021 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f1b3c081040 con 0x55cbfd49ae80 2024-03-30T00:43:51.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 -- 172.21.15.66:0/2462866021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbfd49e630 con 0x55cbfd49ae80 2024-03-30T00:43:51.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b38c00640 1 -- 172.21.15.66:0/2462866021 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b3c05f050 con 0x55cbfd49ae80 2024-03-30T00:43:51.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 10 client.6735 fetch_fsmap learned FSMap version 34 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 -- 172.21.15.66:0/2462866021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55cbfd293cf0 con 0x55cbfd49ae80 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b38c00640 1 -- 172.21.15.66:0/2462866021 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbfd49e630 con 0x55cbfd49ae80 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b38c00640 1 -- 172.21.15.66:0/2462866021 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1b3c062080 con 0x55cbfd49ae80 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 10 client.6735 fetch_fsmap finished waiting for FSMap version 34 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 10 client.6735 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 20 client.6735 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55cbfd502720 0x55cbfd502b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 4 client.6735 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 -- 172.21.15.66:0/2462866021 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x55cbfd2e56a0 con 0x55cbfd502720 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b42a00640 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55cbfd502720 0x55cbfd502b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b42a00640 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55cbfd502720 0x55cbfd502b30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed096732d48ab305 server_cookie=5d4ee298d6c33189 in_seq=0 out_seq=0 2024-03-30T00:43:51.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b38c00640 10 client.6735 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:51.840 INFO:tasks.ceph.mds.d.smithi156.stderr:2024-03-30T00:43:51.827+0000 7f6205800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-03-30T00:43:18.868087+0000 1=1+0) (iversion lock) caps={6597=pAsLsXsFs/-@2},l=6597(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x5628a73afb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:43:51.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b38c00640 1 -- 172.21.15.66:0/2462866021 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cbfd2e56a0 con 0x55cbfd502720 2024-03-30T00:43:51.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b38c00640 10 client.6735 handle_command_reply: tid=0 2024-03-30T00:43:51.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 client.6735 shutdown 2024-03-30T00:43:51.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 -- 172.21.15.66:0/2462866021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd49ae80 msgr2=0x55cbfd547000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd49ae80 0x55cbfd547000 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c051fa0 tx=0x7f1b3c015000 comp rx=0 tx=0).stop 2024-03-30T00:43:51.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 -- 172.21.15.66:0/2462866021 shutdown_connections 2024-03-30T00:43:51.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55cbfd502720 0x55cbfd502b30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbfd4e5440 0x55cbfd501d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55cbfd4e4b20 0x55cbfd547540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 --2- 172.21.15.66:0/2462866021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbfd49ae80 0x55cbfd547000 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b38c00640 0 client.6735 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:51.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 -- 172.21.15.66:0/2462866021 >> 172.21.15.66:0/2462866021 conn(0x55cbfd4a1400 msgr2=0x55cbfd4f6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 -- 172.21.15.66:0/2462866021 shutdown_connections 2024-03-30T00:43:51.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.828+0000 7f1b55bde000 1 -- 172.21.15.66:0/2462866021 wait complete. 2024-03-30T00:43:51.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b55bde000 20 client.6735 trim_cache size 0 max 16384 2024-03-30T00:43:51.843 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:43:51.843 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T00:43:51.843 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "118af35d-6b0e-4d94-95ce-3748db1290d5", 2024-03-30T00:43:51.843 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T00:43:51.843 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:43:51.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 -- 172.21.15.66:0/644366891 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b280666c0 msgr2=0x7f1b28068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b280666c0 0x7f1b28068b80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1b34007fd0 tx=0x7f1b34007920 comp rx=0 tx=0).stop 2024-03-30T00:43:51.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 -- 172.21.15.66:0/644366891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b4c0ce7b0 msgr2=0x7f1b4c0c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:51.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b4c0ce7b0 0x7f1b4c0c99d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c007e40 tx=0x7f1b3c00e4b0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 -- 172.21.15.66:0/644366891 shutdown_connections 2024-03-30T00:43:51.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b4c1594f0 0x7f1b4c0c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b4c0cec60 0x7f1b4c15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1b280666c0 0x7f1b28068b80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 --2- 172.21.15.66:0/644366891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b4c0ce7b0 0x7f1b4c0c99d0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:51.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 -- 172.21.15.66:0/644366891 >> 172.21.15.66:0/644366891 conn(0x7f1b4c06a190 msgr2=0x7f1b4c15ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:51.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 -- 172.21.15.66:0/644366891 shutdown_connections 2024-03-30T00:43:51.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.832+0000 7f1b52800640 1 -- 172.21.15.66:0/644366891 wait complete. 2024-03-30T00:43:51.855 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-30T00:43:51.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f68200640 1 Processor -- start 2024-03-30T00:43:51.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f68200640 1 -- start start 2024-03-30T00:43:51.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f68200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2f6015ca10 0x7f2f6006ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f68200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2f600ae4e0 0x7f2f600cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:51.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f68200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6006e7d0 con 0x7f2f600ce820 2024-03-30T00:43:51.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f68200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f600b1f70 con 0x7f2f600ae4e0 2024-03-30T00:43:51.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f68200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6006f1a0 con 0x7f2f6015ca10 2024-03-30T00:43:51.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f64e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2f600ae4e0 0x7f2f600cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:51.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f66200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2f600ce820 0x7f2f60160510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34004/0 (socket says 172.21.15.66:34004) 2024-03-30T00:43:51.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f64e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2f600ae4e0 0x7f2f600cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36984/0 (socket says 172.21.15.66:36984) 2024-03-30T00:43:52.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f66200640 1 -- 172.21.15.66:0/932200397 learned_addr learned my addr 172.21.15.66:0/932200397 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:52.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f65800640 1 --2- 172.21.15.66:0/932200397 >> v2:172.21.15.156:3300/0 conn(0x7f2f6015ca10 0x7f2f6006ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f64e00640 1 -- 172.21.15.66:0/932200397 >> v2:172.21.15.156:3300/0 conn(0x7f2f6015ca10 msgr2=0x7f2f6006ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f64e00640 1 --2- 172.21.15.66:0/932200397 >> v2:172.21.15.156:3300/0 conn(0x7f2f6015ca10 0x7f2f6006ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f64e00640 1 -- 172.21.15.66:0/932200397 >> v1:172.21.15.66:6789/0 conn(0x7f2f600ce820 legacy=0x7f2f60160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f64e00640 1 -- 172.21.15.66:0/932200397 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6015cdf0 con 0x7f2f600ae4e0 2024-03-30T00:43:52.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.992+0000 7f2f5fe00640 1 -- 172.21.15.66:0/932200397 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673249692 0 0) 0x7f2f6006e7d0 con 0x7f2f600ce820 2024-03-30T00:43:52.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f64e00640 1 --2- 172.21.15.66:0/932200397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f600ae4e0 0x7f2f600cb620 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2f6006f9f0 tx=0x7f2f4c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b56f672a75974b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f5fe00640 1 -- 172.21.15.66:0/932200397 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0ab650 con 0x7f2f600ae4e0 2024-03-30T00:43:52.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f5fe00640 1 -- 172.21.15.66:0/932200397 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4c0ab7b0 con 0x7f2f600ae4e0 2024-03-30T00:43:52.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f5fe00640 1 -- 172.21.15.66:0/932200397 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0abb40 con 0x7f2f600ae4e0 2024-03-30T00:43:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f68200640 1 -- 172.21.15.66:0/932200397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f600ae4e0 msgr2=0x7f2f600cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f68200640 1 --2- 172.21.15.66:0/932200397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f600ae4e0 0x7f2f600cb620 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2f6006f9f0 tx=0x7f2f4c01c450 comp rx=0 tx=0).stop 2024-03-30T00:43:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f68200640 1 -- 172.21.15.66:0/932200397 shutdown_connections 2024-03-30T00:43:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f68200640 1 --2- 172.21.15.66:0/932200397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f600ae4e0 0x7f2f600cb620 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f68200640 1 --2- 172.21.15.66:0/932200397 >> v2:172.21.15.156:3300/0 conn(0x7f2f6015ca10 0x7f2f6006ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f68200640 1 -- 172.21.15.66:0/932200397 >> 172.21.15.66:0/932200397 conn(0x7f2f60098a20 msgr2=0x7f2f600b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f68200640 1 -- 172.21.15.66:0/932200397 shutdown_connections 2024-03-30T00:43:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:51.996+0000 7f2f68200640 1 -- 172.21.15.66:0/932200397 wait complete. 2024-03-30T00:43:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f68200640 1 Processor -- start 2024-03-30T00:43:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f68200640 1 -- start start 2024-03-30T00:43:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f68200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f600ae4e0 0x7f2f6006dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f68200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f600ce820 0x7f2f600c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f68200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f6015ca10 0x7f2f60155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f68200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600c42b0 con 0x7f2f6015ca10 2024-03-30T00:43:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f68200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600c36a0 con 0x7f2f600ae4e0 2024-03-30T00:43:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f68200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600c3980 con 0x7f2f600ce820 2024-03-30T00:43:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f66200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f6015ca10 0x7f2f60155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f66200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f6015ca10 0x7f2f60155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36992/0 (socket says 172.21.15.66:36992) 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f66200640 1 -- 172.21.15.66:0/1079440597 learned_addr learned my addr 172.21.15.66:0/1079440597 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f64e00640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f600ce820 0x7f2f600c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f65800640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f600ae4e0 0x7f2f6006dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f66200640 1 -- 172.21.15.66:0/1079440597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f600ce820 msgr2=0x7f2f600c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f66200640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f600ce820 0x7f2f600c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f66200640 1 -- 172.21.15.66:0/1079440597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f600ae4e0 msgr2=0x7f2f6006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f66200640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f600ae4e0 0x7f2f6006dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f66200640 1 -- 172.21.15.66:0/1079440597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c009420 con 0x7f2f6015ca10 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f64e00640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f600ce820 0x7f2f600c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f66200640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f6015ca10 0x7f2f60155810 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2f58001570 tx=0x7f2f5800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f5d600640 1 -- 172.21.15.66:0/1079440597 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800a640 con 0x7f2f6015ca10 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f5d600640 1 -- 172.21.15.66:0/1079440597 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5800a7c0 con 0x7f2f6015ca10 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f5d600640 1 -- 172.21.15.66:0/1079440597 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800ab70 con 0x7f2f6015ca10 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f68200640 1 -- 172.21.15.66:0/1079440597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f600c3c60 con 0x7f2f6015ca10 2024-03-30T00:43:52.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.000+0000 7f2f68200640 1 -- 172.21.15.66:0/1079440597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f60155d50 con 0x7f2f6015ca10 2024-03-30T00:43:52.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f5d600640 1 -- 172.21.15.66:0/1079440597 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2f5800acd0 con 0x7f2f6015ca10 2024-03-30T00:43:52.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f5d600640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2f40066690 0x7f2f40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f5d600640 1 -- 172.21.15.66:0/1079440597 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f2f58083890 con 0x7f2f6015ca10 2024-03-30T00:43:52.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f65800640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2f40066690 0x7f2f40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f6b751000 1 Processor -- start 2024-03-30T00:43:52.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f6b751000 1 -- start start 2024-03-30T00:43:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f65800640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2f40066690 0x7f2f40068b50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009b60 tx=0x7f2f54009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fc22a70 0x55789fc22e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc23390 0x55789fbd64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fbd6a00 0x55789fbd6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f6b751000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789faa1630 con 0x55789fbd6a00 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f6b751000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789fa2ebd0 con 0x55789fc22a70 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f6b751000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55789fa2c1a0 con 0x55789fc23390 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f66200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fbd6a00 0x55789fbd6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f66200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fbd6a00 0x55789fbd6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37000/0 (socket says 172.21.15.66:37000) 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f66200640 1 -- 172.21.15.66:0/2750025842 learned_addr learned my addr 172.21.15.66:0/2750025842 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f65800640 1 --2- 172.21.15.66:0/2750025842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fc22a70 0x55789fc22e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f64e00640 1 --2- 172.21.15.66:0/2750025842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc23390 0x55789fbd64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f66200640 1 -- 172.21.15.66:0/2750025842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc23390 msgr2=0x55789fbd64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f66200640 1 --2- 172.21.15.66:0/2750025842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc23390 0x55789fbd64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f66200640 1 -- 172.21.15.66:0/2750025842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fc22a70 msgr2=0x55789fc22e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f66200640 1 --2- 172.21.15.66:0/2750025842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fc22a70 0x55789fc22e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f66200640 1 -- 172.21.15.66:0/2750025842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55789f723e20 con 0x55789fbd6a00 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f66200640 1 --2- 172.21.15.66:0/2750025842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fbd6a00 0x55789fbd6de0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2f58081b80 tx=0x7f2f58054400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d16c2906e2584f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f52000640 1 -- 172.21.15.66:0/2750025842 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5805cad0 con 0x55789fbd6a00 2024-03-30T00:43:52.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f52000640 1 -- 172.21.15.66:0/2750025842 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f58052b30 con 0x55789fbd6a00 2024-03-30T00:43:52.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.004+0000 7f2f52000640 1 -- 172.21.15.66:0/2750025842 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58058170 con 0x55789fbd6a00 2024-03-30T00:43:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 -- 172.21.15.66:0/2750025842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fbd6a00 msgr2=0x55789fbd6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2750025842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fbd6a00 0x55789fbd6de0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2f58081b80 tx=0x7f2f58054400 comp rx=0 tx=0).stop 2024-03-30T00:43:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 -- 172.21.15.66:0/2750025842 shutdown_connections 2024-03-30T00:43:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2750025842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fbd6a00 0x55789fbd6de0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2750025842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc23390 0x55789fbd64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2750025842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fc22a70 0x55789fc22e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 -- 172.21.15.66:0/2750025842 >> 172.21.15.66:0/2750025842 conn(0x55789f9d7160 msgr2=0x55789fc05820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 -- 172.21.15.66:0/2750025842 shutdown_connections 2024-03-30T00:43:52.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 -- 172.21.15.66:0/2750025842 wait complete. 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 -- start start 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 Processor -- start 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fbd6a00 0x55789fbdead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc22a70 0x55789fc06a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fc06f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789fa27a90 con 0x55789fc23390 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789f9c6d40 con 0x55789fbd6a00 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55789f9e4f30 con 0x55789fc22a70 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 10 client.0 _ll_register_callbacks cb 0x55789fa89810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f66200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fc06f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f64e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc22a70 0x55789fc06a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f65800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fbd6a00 0x55789fbdead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f66200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fc06f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37006/0 (socket says 172.21.15.66:37006) 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f66200640 1 -- 172.21.15.66:0/2410038377 learned_addr learned my addr 172.21.15.66:0/2410038377 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f66200640 1 -- 172.21.15.66:0/2410038377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc22a70 msgr2=0x55789fc06a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f66200640 1 --2- 172.21.15.66:0/2410038377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc22a70 0x55789fc06a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f66200640 1 -- 172.21.15.66:0/2410038377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fbd6a00 msgr2=0x55789fbdead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f66200640 1 --2- 172.21.15.66:0/2410038377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fbd6a00 0x55789fbdead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f66200640 1 -- 172.21.15.66:0/2410038377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f60155d50 con 0x55789fc23390 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f66200640 1 --2- 172.21.15.66:0/2410038377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fc06f90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2f58050c40 tx=0x7f2f5801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cdef4c66fe6740 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f46a00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f46a00640 1 -- 172.21.15.66:0/2410038377 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5805cad0 con 0x55789fc23390 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 -- 172.21.15.66:0/2410038377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55789fa263e0 con 0x55789fc23390 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f46a00640 1 -- 172.21.15.66:0/2410038377 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f2f5805c640 con 0x55789fc23390 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f46a00640 1 -- 172.21.15.66:0/2410038377 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f5805c800 con 0x55789fc23390 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 10 client.6750 fetch_fsmap learned FSMap version 34 2024-03-30T00:43:52.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.008+0000 7f2f6b751000 1 -- 172.21.15.66:0/2410038377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55789f9ecaa0 con 0x55789fc23390 2024-03-30T00:43:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f46a00640 1 -- 172.21.15.66:0/2410038377 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55789fa263e0 con 0x55789fc23390 2024-03-30T00:43:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f46a00640 1 -- 172.21.15.66:0/2410038377 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2f5807bae0 con 0x55789fc23390 2024-03-30T00:43:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 10 client.6750 fetch_fsmap finished waiting for FSMap version 34 2024-03-30T00:43:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 10 client.6750 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-30T00:43:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 20 client.6750 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2410038377 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55789fc321e0 0x55789fc345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 4 client.6750 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:43:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 -- 172.21.15.66:0/2410038377 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55789f9f7d00 con 0x55789fc321e0 2024-03-30T00:43:52.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f65800640 1 --2- 172.21.15.66:0/2410038377 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55789fc321e0 0x55789fc345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f46a00640 10 client.6750 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:52.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f65800640 1 --2- 172.21.15.66:0/2410038377 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55789fc321e0 0x55789fc345d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2a5e8d36a1bfa04 server_cookie=bb6c481a1635349a in_seq=0 out_seq=0 2024-03-30T00:43:52.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f46a00640 1 -- 172.21.15.66:0/2410038377 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55789f9f7d00 con 0x55789fc321e0 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f46a00640 10 client.6750 handle_command_reply: tid=0 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 client.6750 shutdown 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 -- 172.21.15.66:0/2410038377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 msgr2=0x55789fc06f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2410038377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fc06f90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2f58050c40 tx=0x7f2f5801e040 comp rx=0 tx=0).stop 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 -- 172.21.15.66:0/2410038377 shutdown_connections 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2410038377 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55789fc321e0 0x55789fc345d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2410038377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fc06f90 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2410038377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc22a70 0x55789fc06a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2410038377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fbd6a00 0x55789fbdead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f46a00640 0 client.6750 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 -- 172.21.15.66:0/2410038377 >> 172.21.15.66:0/2410038377 conn(0x55789f9d7160 msgr2=0x55789fbd5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 -- 172.21.15.66:0/2410038377 shutdown_connections 2024-03-30T00:43:52.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 1 -- 172.21.15.66:0/2410038377 wait complete. 2024-03-30T00:43:52.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.012+0000 7f2f6b751000 20 client.6750 trim_cache size 0 max 16384 2024-03-30T00:43:52.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f6b751000 1 Processor -- start 2024-03-30T00:43:52.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f6b751000 1 -- start start 2024-03-30T00:43:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fbd6a00 0x55789fbd5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc22a70 0x55789fbd6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fbd6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f6b751000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789f6e4aa0 con 0x55789fc23390 2024-03-30T00:43:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f6b751000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789fa6b170 con 0x55789fbd6a00 2024-03-30T00:43:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f6b751000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55789f9f70b0 con 0x55789fc22a70 2024-03-30T00:43:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f66200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fbd6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f66200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fbd6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37008/0 (socket says 172.21.15.66:37008) 2024-03-30T00:43:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f66200640 1 -- 172.21.15.66:0/2337888122 learned_addr learned my addr 172.21.15.66:0/2337888122 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f64e00640 1 --2- 172.21.15.66:0/2337888122 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc22a70 0x55789fbd6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f65800640 1 --2- 172.21.15.66:0/2337888122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fbd6a00 0x55789fbd5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f66200640 1 -- 172.21.15.66:0/2337888122 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc22a70 msgr2=0x55789fbd6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f66200640 1 --2- 172.21.15.66:0/2337888122 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc22a70 0x55789fbd6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f66200640 1 -- 172.21.15.66:0/2337888122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fbd6a00 msgr2=0x55789fbd5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f66200640 1 --2- 172.21.15.66:0/2337888122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fbd6a00 0x55789fbd5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f66200640 1 -- 172.21.15.66:0/2337888122 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55789f9d2df0 con 0x55789fc23390 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f65800640 1 --2- 172.21.15.66:0/2337888122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fbd6a00 0x55789fbd5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f66200640 1 --2- 172.21.15.66:0/2337888122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fbd6de0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2f5806c240 tx=0x7f2f58083000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9035e85945a5da5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f52a00640 1 -- 172.21.15.66:0/2337888122 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5805cad0 con 0x55789fc23390 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f52a00640 1 -- 172.21.15.66:0/2337888122 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f58058170 con 0x55789fc23390 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f52a00640 1 -- 172.21.15.66:0/2337888122 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5805c880 con 0x55789fc23390 2024-03-30T00:43:52.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f6b751000 1 -- 172.21.15.66:0/2337888122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 msgr2=0x55789fbd6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.028+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2337888122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fbd6de0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2f5806c240 tx=0x7f2f58083000 comp rx=0 tx=0).stop 2024-03-30T00:43:52.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 -- 172.21.15.66:0/2337888122 shutdown_connections 2024-03-30T00:43:52.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2337888122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc23390 0x55789fbd6de0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2337888122 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fc22a70 0x55789fbd6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2337888122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fbd6a00 0x55789fbd5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 -- 172.21.15.66:0/2337888122 >> 172.21.15.66:0/2337888122 conn(0x55789f9d7160 msgr2=0x55789fbd54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 -- 172.21.15.66:0/2337888122 shutdown_connections 2024-03-30T00:43:52.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 -- 172.21.15.66:0/2337888122 wait complete. 2024-03-30T00:43:52.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 -- start start 2024-03-30T00:43:52.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 Processor -- start 2024-03-30T00:43:52.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fbd6a00 0x55789fbd9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc22a70 0x55789fbd9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fc23390 0x55789fbd9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789f9f1ee0 con 0x55789fc22a70 2024-03-30T00:43:52.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789f9f1640 con 0x55789fc23390 2024-03-30T00:43:52.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55789fc21b30 con 0x55789fbd6a00 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 10 client.0 _ll_register_callbacks cb 0x7f2f580018e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f64e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc22a70 0x55789fbd9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f64e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc22a70 0x55789fbd9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37016/0 (socket says 172.21.15.66:37016) 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f64e00640 1 -- 172.21.15.66:0/2986105855 learned_addr learned my addr 172.21.15.66:0/2986105855 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f65800640 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fbd6a00 0x55789fbd9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f66200640 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fc23390 0x55789fbd9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f64e00640 1 -- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fbd6a00 msgr2=0x55789fbd9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f64e00640 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fbd6a00 0x55789fbd9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f64e00640 1 -- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fc23390 msgr2=0x55789fbd9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f64e00640 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fc23390 0x55789fbd9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f64e00640 1 -- 172.21.15.66:0/2986105855 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55789f9ecaa0 con 0x55789fc22a70 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f66200640 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fc23390 0x55789fbd9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f64e00640 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc22a70 0x55789fbd9a90 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0ad040 tx=0x7f2f4c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd05d333defbf67 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f46a00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f46a00640 1 -- 172.21.15.66:0/2986105855 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0b9040 con 0x55789fc22a70 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f46a00640 1 -- 172.21.15.66:0/2986105855 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f2f4c0c1480 con 0x55789fc22a70 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 -- 172.21.15.66:0/2986105855 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55789f9e4970 con 0x55789fc22a70 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f46a00640 1 -- 172.21.15.66:0/2986105855 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f4c0abea0 con 0x55789fc22a70 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 10 client.6756 fetch_fsmap learned FSMap version 34 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f6b751000 1 -- 172.21.15.66:0/2986105855 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55789f9c6960 con 0x55789fc22a70 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f46a00640 1 -- 172.21.15.66:0/2986105855 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55789f9e4970 con 0x55789fc22a70 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.032+0000 7f2f46a00640 1 -- 172.21.15.66:0/2986105855 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2f4c0c1640 con 0x55789fc22a70 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 10 client.6756 fetch_fsmap finished waiting for FSMap version 34 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 10 client.6756 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 20 client.6756 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55789fbda510 0x55789fbda920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 4 client.6756 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:43:52.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 -- 172.21.15.66:0/2986105855 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55789fa56a20 con 0x55789fbda510 2024-03-30T00:43:52.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f65800640 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55789fbda510 0x55789fbda920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f65800640 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55789fbda510 0x55789fbda920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68a16434d517062 server_cookie=2aa947d774b2b9da in_seq=0 out_seq=0 2024-03-30T00:43:52.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f46a00640 10 client.6756 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:52.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f46a00640 1 -- 172.21.15.66:0/2986105855 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55789fa56a20 con 0x55789fbda510 2024-03-30T00:43:52.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f46a00640 10 client.6756 handle_command_reply: tid=0 2024-03-30T00:43:52.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 client.6756 shutdown 2024-03-30T00:43:52.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 -- 172.21.15.66:0/2986105855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc22a70 msgr2=0x55789fbd9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc22a70 0x55789fbd9a90 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0ad040 tx=0x7f2f4c01ac90 comp rx=0 tx=0).stop 2024-03-30T00:43:52.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 -- 172.21.15.66:0/2986105855 shutdown_connections 2024-03-30T00:43:52.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x55789fbda510 0x55789fbda920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55789fc23390 0x55789fbd9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f46a00640 0 client.6756 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:52.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55789fc22a70 0x55789fbd9a90 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 --2- 172.21.15.66:0/2986105855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55789fbd6a00 0x55789fbd9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 -- 172.21.15.66:0/2986105855 >> 172.21.15.66:0/2986105855 conn(0x55789f9d7160 msgr2=0x55789fbdd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 -- 172.21.15.66:0/2986105855 shutdown_connections 2024-03-30T00:43:52.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 1 -- 172.21.15.66:0/2986105855 wait complete. 2024-03-30T00:43:52.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f6b751000 20 client.6756 trim_cache size 0 max 16384 2024-03-30T00:43:52.046 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:43:52.047 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T00:43:52.047 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-03-30T00:43:52.047 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:43:52.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 -- 172.21.15.66:0/1079440597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2f40066690 msgr2=0x7f2f40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2f40066690 0x7f2f40068b50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009b60 tx=0x7f2f54009190 comp rx=0 tx=0).stop 2024-03-30T00:43:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 -- 172.21.15.66:0/1079440597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f6015ca10 msgr2=0x7f2f60155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f6015ca10 0x7f2f60155810 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2f58001570 tx=0x7f2f5800cca0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 -- 172.21.15.66:0/1079440597 shutdown_connections 2024-03-30T00:43:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2f40066690 0x7f2f40068b50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f6015ca10 0x7f2f60155810 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f600ce820 0x7f2f600c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 --2- 172.21.15.66:0/1079440597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f600ae4e0 0x7f2f6006dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 -- 172.21.15.66:0/1079440597 >> 172.21.15.66:0/1079440597 conn(0x7f2f60098a20 msgr2=0x7f2f60069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 -- 172.21.15.66:0/1079440597 shutdown_connections 2024-03-30T00:43:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.036+0000 7f2f68200640 1 -- 172.21.15.66:0/1079440597 wait complete. 2024-03-30T00:43:52.061 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:43:52.061 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:43:52.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfac000640 1 Processor -- start 2024-03-30T00:43:52.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfac000640 1 -- start start 2024-03-30T00:43:52.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfac000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa415de30 0x7fbfa40cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfac000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbfa40cee50 0x7fbfa40d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfac000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa40cbb90 con 0x7fbfa40af420 2024-03-30T00:43:52.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfac000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa40ce1b0 con 0x7fbfa40cee50 2024-03-30T00:43:52.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfac000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40afa10 con 0x7fbfa415de30 2024-03-30T00:43:52.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfa9600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa415de30 0x7fbfa40cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfaa000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbfa40cee50 0x7fbfa40d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfa9600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa415de30 0x7fbfa40cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:51062/0 (socket says 172.21.15.66:51062) 2024-03-30T00:43:52.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfa9600640 1 -- 172.21.15.66:0/1524315659 learned_addr learned my addr 172.21.15.66:0/1524315659 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfa8c00640 1 --1- 172.21.15.66:0/1524315659 >> v1:172.21.15.156:6789/0 conn(0x7fbfa40af420 0x7fbfa40ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:56034/0 (socket says 172.21.15.66:56034) 2024-03-30T00:43:52.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfa3e00640 1 -- 172.21.15.66:0/1524315659 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817649680 0 0) 0x7fbfa40cbb90 con 0x7fbfa40af420 2024-03-30T00:43:52.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfa9600640 1 -- 172.21.15.66:0/1524315659 >> v2:172.21.15.156:3300/0 conn(0x7fbfa40cee50 msgr2=0x7fbfa40d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfa9600640 1 --2- 172.21.15.66:0/1524315659 >> v2:172.21.15.156:3300/0 conn(0x7fbfa40cee50 0x7fbfa40d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfa9600640 1 -- 172.21.15.66:0/1524315659 >> v1:172.21.15.156:6789/0 conn(0x7fbfa40af420 legacy=0x7fbfa40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfa9600640 1 -- 172.21.15.66:0/1524315659 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa40cf430 con 0x7fbfa415de30 2024-03-30T00:43:52.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.195+0000 7fbfaa000640 1 --2- 172.21.15.66:0/1524315659 >> v2:172.21.15.156:3300/0 conn(0x7fbfa40cee50 0x7fbfa40d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:52.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfa9600640 1 --2- 172.21.15.66:0/1524315659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa415de30 0x7fbfa40cc040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009870 tx=0x7fbf9801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3406079c24e20 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfa3e00640 1 -- 172.21.15.66:0/1524315659 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf980a7070 con 0x7fbfa415de30 2024-03-30T00:43:52.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfa3e00640 1 -- 172.21.15.66:0/1524315659 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf980ab430 con 0x7fbfa415de30 2024-03-30T00:43:52.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfa3e00640 1 -- 172.21.15.66:0/1524315659 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf980ab7c0 con 0x7fbfa415de30 2024-03-30T00:43:52.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfac000640 1 -- 172.21.15.66:0/1524315659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa415de30 msgr2=0x7fbfa40cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfac000640 1 --2- 172.21.15.66:0/1524315659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa415de30 0x7fbfa40cc040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009870 tx=0x7fbf9801c4d0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfac000640 1 -- 172.21.15.66:0/1524315659 shutdown_connections 2024-03-30T00:43:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfac000640 1 --2- 172.21.15.66:0/1524315659 >> v2:172.21.15.156:3300/0 conn(0x7fbfa40cee50 0x7fbfa40d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfac000640 1 --2- 172.21.15.66:0/1524315659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa415de30 0x7fbfa40cc040 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfac000640 1 -- 172.21.15.66:0/1524315659 >> 172.21.15.66:0/1524315659 conn(0x7fbfa40d0540 msgr2=0x7fbfa40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfac000640 1 -- 172.21.15.66:0/1524315659 shutdown_connections 2024-03-30T00:43:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.199+0000 7fbfac000640 1 -- 172.21.15.66:0/1524315659 wait complete. 2024-03-30T00:43:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfac000640 1 Processor -- start 2024-03-30T00:43:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfac000640 1 -- start start 2024-03-30T00:43:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfac000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa40af420 0x7fbfa414bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfac000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa40cee50 0x7fbfa40b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfac000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfa415de30 0x7fbfa40b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfac000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40b6000 con 0x7fbfa40af420 2024-03-30T00:43:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfac000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa414c4b0 con 0x7fbfa415de30 2024-03-30T00:43:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfac000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40b59f0 con 0x7fbfa40cee50 2024-03-30T00:43:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa40af420 0x7fbfa414bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa40af420 0x7fbfa414bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37032/0 (socket says 172.21.15.66:37032) 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa9600640 1 -- 172.21.15.66:0/843825267 learned_addr learned my addr 172.21.15.66:0/843825267 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa9600640 1 -- 172.21.15.66:0/843825267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa40cee50 msgr2=0x7fbfa40b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa8c00640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa40cee50 0x7fbfa40b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfaa000640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfa415de30 0x7fbfa40b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa9600640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa40cee50 0x7fbfa40b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa9600640 1 -- 172.21.15.66:0/843825267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfa415de30 msgr2=0x7fbfa40b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa9600640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfa415de30 0x7fbfa40b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa9600640 1 -- 172.21.15.66:0/843825267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c009a70 con 0x7fbfa40af420 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfaa000640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfa415de30 0x7fbfa40b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa8c00640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa40cee50 0x7fbfa40b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa9600640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa40af420 0x7fbfa414bd10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbf980ad040 tx=0x7fbf98019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa1600640 1 -- 172.21.15.66:0/843825267 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf980b4800 con 0x7fbfa40af420 2024-03-30T00:43:52.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa1600640 1 -- 172.21.15.66:0/843825267 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf980b4980 con 0x7fbfa40af420 2024-03-30T00:43:52.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfac000640 1 -- 172.21.15.66:0/843825267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf980094d0 con 0x7fbfa40af420 2024-03-30T00:43:52.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfa1600640 1 -- 172.21.15.66:0/843825267 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf980b4d30 con 0x7fbfa40af420 2024-03-30T00:43:52.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.203+0000 7fbfac000640 1 -- 172.21.15.66:0/843825267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa40b5c90 con 0x7fbfa40af420 2024-03-30T00:43:52.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.207+0000 7fbfac000640 1 -- 172.21.15.66:0/843825267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa40cdf60 con 0x7fbfa40af420 2024-03-30T00:43:52.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.207+0000 7fbfa1600640 1 -- 172.21.15.66:0/843825267 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbf980b3380 con 0x7fbfa40af420 2024-03-30T00:43:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.207+0000 7fbfa1600640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbf84066690 0x7fbf84068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.207+0000 7fbfa1600640 1 -- 172.21.15.66:0/843825267 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fbf9811e9d0 con 0x7fbfa40af420 2024-03-30T00:43:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.207+0000 7fbfa8c00640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbf84066690 0x7fbf84068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.207+0000 7fbfa8c00640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbf84066690 0x7fbf84068b50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbf90000f70 tx=0x7fbf90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.211+0000 7fbfa1600640 1 -- 172.21.15.66:0/843825267 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbf980ebb80 con 0x7fbfa40af420 2024-03-30T00:43:52.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 -- 172.21.15.66:0/843825267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfa4066c30 con 0x7fbfa40af420 2024-03-30T00:43:52.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfa1600640 1 -- 172.21.15.66:0/843825267 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fbf980c4070 con 0x7fbfa40af420 2024-03-30T00:43:52.413 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:52.413 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4272961638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4272961638},{"type":"v1","addr":"172.21.15.156:6834","nonce":4272961638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2144186622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2144186622},{"type":"v1","addr":"172.21.15.66:6835","nonce":2144186622}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/750308788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":750308788},{"type":"v1","addr":"172.21.15.66:6837","nonce":750308788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:43:11.141056+0000","modified":"2024-03-30T00:43:23.528635+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6468":{"gid":6468,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6835/1137593870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1137593870},{"type":"v1","addr":"172.21.15.156:6835","nonce":1137593870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-30T00:43:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 34 2024-03-30T00:43:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 -- 172.21.15.66:0/843825267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbf84066690 msgr2=0x7fbf84068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbf84066690 0x7fbf84068b50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbf90000f70 tx=0x7fbf90009040 comp rx=0 tx=0).stop 2024-03-30T00:43:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 -- 172.21.15.66:0/843825267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa40af420 msgr2=0x7fbfa414bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa40af420 0x7fbfa414bd10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbf980ad040 tx=0x7fbf98019270 comp rx=0 tx=0).stop 2024-03-30T00:43:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 -- 172.21.15.66:0/843825267 shutdown_connections 2024-03-30T00:43:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbfa415de30 0x7fbfa40b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbfa40cee50 0x7fbfa40b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbf84066690 0x7fbf84068b50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 --2- 172.21.15.66:0/843825267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa40af420 0x7fbfa414bd10 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 -- 172.21.15.66:0/843825267 >> 172.21.15.66:0/843825267 conn(0x7fbfa40d0540 msgr2=0x7fbfa406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 -- 172.21.15.66:0/843825267 shutdown_connections 2024-03-30T00:43:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.407+0000 7fbfac000640 1 -- 172.21.15.66:0/843825267 wait complete. 2024-03-30T00:43:52.434 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-03-30T00:43:52.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.567+0000 7f7bc7000640 1 Processor -- start 2024-03-30T00:43:52.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.567+0000 7f7bc7000640 1 -- start start 2024-03-30T00:43:52.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.567+0000 7f7bc7000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7bc00ceb60 0x7f7bc015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.567+0000 7f7bc7000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc00cbec0 con 0x7f7bc015d360 2024-03-30T00:43:52.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.567+0000 7f7bc7000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc015d970 con 0x7f7bc00ce780 2024-03-30T00:43:52.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.567+0000 7f7bc7000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc015dc50 con 0x7f7bc00ceb60 2024-03-30T00:43:52.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.571+0000 7f7bc5000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7bc015d360 0x7f7bc00cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34018/0 (socket says 172.21.15.66:34018) 2024-03-30T00:43:52.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.571+0000 7f7bc5000640 1 -- 172.21.15.66:0/3954814709 learned_addr learned my addr 172.21.15.66:0/3954814709 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.571+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/3954814709 >> v2:172.21.15.156:3300/0 conn(0x7f7bc00ceb60 0x7f7bc015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.571+0000 7f7bbf400640 1 -- 172.21.15.66:0/3954814709 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549921244 0 0) 0x7f7bc00cbec0 con 0x7f7bc015d360 2024-03-30T00:43:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.571+0000 7f7bbf400640 1 -- 172.21.15.66:0/3954814709 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba4003530 con 0x7f7bc015d360 2024-03-30T00:43:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.571+0000 7f7bb7e00640 1 -- 172.21.15.66:0/3954814709 >> v1:172.21.15.156:6789/0 conn(0x7f7bc00ce780 legacy=0x7f7bc00cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.571+0000 7f7bb7e00640 1 -- 172.21.15.66:0/3954814709 >> v1:172.21.15.66:6789/0 conn(0x7f7bc015d360 legacy=0x7f7bc00cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.571+0000 7f7bbf400640 1 -- 172.21.15.66:0/3954814709 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 175573932 0 0) 0x7f7ba4003530 con 0x7f7bc015d360 2024-03-30T00:43:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.571+0000 7f7bb7e00640 1 -- 172.21.15.66:0/3954814709 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc00cc110 con 0x7f7bc00ceb60 2024-03-30T00:43:52.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.571+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/3954814709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bc00ceb60 0x7f7bc015ce20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80097c0 tx=0x7f7ba801c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35066d379dbfb23b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bbf400640 1 -- 172.21.15.66:0/3954814709 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8001a00 con 0x7f7bc00ceb60 2024-03-30T00:43:52.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bbf400640 1 -- 172.21.15.66:0/3954814709 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba80a7050 con 0x7f7bc00ceb60 2024-03-30T00:43:52.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bbf400640 1 -- 172.21.15.66:0/3954814709 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80ab600 con 0x7f7bc00ceb60 2024-03-30T00:43:52.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 -- 172.21.15.66:0/3954814709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bc00ceb60 msgr2=0x7f7bc015ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 --2- 172.21.15.66:0/3954814709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bc00ceb60 0x7f7bc015ce20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80097c0 tx=0x7f7ba801c300 comp rx=0 tx=0).stop 2024-03-30T00:43:52.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 -- 172.21.15.66:0/3954814709 shutdown_connections 2024-03-30T00:43:52.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 --2- 172.21.15.66:0/3954814709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bc00ceb60 0x7f7bc015ce20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 -- 172.21.15.66:0/3954814709 >> 172.21.15.66:0/3954814709 conn(0x7f7bc0069f60 msgr2=0x7f7bc006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 -- 172.21.15.66:0/3954814709 shutdown_connections 2024-03-30T00:43:52.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 -- 172.21.15.66:0/3954814709 wait complete. 2024-03-30T00:43:52.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 Processor -- start 2024-03-30T00:43:52.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 -- start start 2024-03-30T00:43:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc00ce780 0x7f7bc0152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bc00ceb60 0x7f7bc0154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bc015d360 0x7f7bc0154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc0155b10 con 0x7f7bc00ce780 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc006c1f0 con 0x7f7bc00ceb60 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bc7000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc015a080 con 0x7f7bc015d360 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.575+0000 7f7bbfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc00ce780 0x7f7bc0152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc00ce780 0x7f7bc0152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37034/0 (socket says 172.21.15.66:37034) 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbfe00640 1 -- 172.21.15.66:0/2895533188 learned_addr learned my addr 172.21.15.66:0/2895533188 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bc00ceb60 0x7f7bc0154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bc5000640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bc015d360 0x7f7bc0154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbfe00640 1 -- 172.21.15.66:0/2895533188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bc015d360 msgr2=0x7f7bc0154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bc015d360 0x7f7bc0154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbfe00640 1 -- 172.21.15.66:0/2895533188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bc00ceb60 msgr2=0x7f7bc0154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bc00ceb60 0x7f7bc0154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbfe00640 1 -- 172.21.15.66:0/2895533188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba8009420 con 0x7f7bc00ce780 2024-03-30T00:43:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bc5000640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bc015d360 0x7f7bc0154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc00ce780 0x7f7bc0152680 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000c500 tx=0x7f7bb000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbcc00640 1 -- 172.21.15.66:0/2895533188 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0005220 con 0x7f7bc00ce780 2024-03-30T00:43:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbcc00640 1 -- 172.21.15.66:0/2895533188 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb0005380 con 0x7f7bc00ce780 2024-03-30T00:43:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbcc00640 1 -- 172.21.15.66:0/2895533188 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0007270 con 0x7f7bc00ce780 2024-03-30T00:43:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bc7000640 1 -- 172.21.15.66:0/2895533188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc0159ac0 con 0x7f7bc00ce780 2024-03-30T00:43:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bc7000640 1 -- 172.21.15.66:0/2895533188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc0152bc0 con 0x7f7bc00ce780 2024-03-30T00:43:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.579+0000 7f7bbcc00640 1 -- 172.21.15.66:0/2895533188 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7bb00073d0 con 0x7f7bc00ce780 2024-03-30T00:43:52.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bbcc00640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b9c066710 0x7f7b9c068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bbcc00640 1 -- 172.21.15.66:0/2895533188 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f7bb00835d0 con 0x7f7bc00ce780 2024-03-30T00:43:52.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b9c066710 0x7f7b9c068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bca487000 1 Processor -- start 2024-03-30T00:43:52.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bca487000 1 -- start start 2024-03-30T00:43:52.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b9c066710 0x7f7b9c068bd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80ad000 tx=0x7f7ba80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bca487000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46290 0x5581cda46670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bca487000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cd9fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bca487000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cd9fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bca487000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cd8c5340 con 0x5581cda46bb0 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bca487000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cd851cd0 con 0x5581cd9fd1b0 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bca487000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581cd84f2a0 con 0x5581cda46290 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cd9fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cd9fcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37048/0 (socket says 172.21.15.66:37048) 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb7e00640 1 -- 172.21.15.66:0/2444226205 learned_addr learned my addr 172.21.15.66:0/2444226205 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bc5000640 1 --2- 172.21.15.66:0/2444226205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cd9fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/2444226205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46290 0x5581cda46670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb7e00640 1 -- 172.21.15.66:0/2444226205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46290 msgr2=0x5581cda46670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/2444226205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46290 0x5581cda46670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb7e00640 1 -- 172.21.15.66:0/2444226205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 msgr2=0x5581cd9fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/2444226205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cd9fd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb7e00640 1 -- 172.21.15.66:0/2444226205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581cd53d4b0 con 0x5581cda46bb0 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bc5000640 1 --2- 172.21.15.66:0/2444226205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cd9fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/2444226205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cd9fcc70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8097df0 tx=0x7f7ba8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c814abc66dc7dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb4c00640 1 -- 172.21.15.66:0/2444226205 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8003d70 con 0x5581cda46bb0 2024-03-30T00:43:52.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb4c00640 1 -- 172.21.15.66:0/2444226205 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8003ed0 con 0x5581cda46bb0 2024-03-30T00:43:52.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bb4c00640 1 -- 172.21.15.66:0/2444226205 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80c2670 con 0x5581cda46bb0 2024-03-30T00:43:52.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bca487000 1 -- 172.21.15.66:0/2444226205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 msgr2=0x5581cd9fcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.583+0000 7f7bca487000 1 --2- 172.21.15.66:0/2444226205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cd9fcc70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8097df0 tx=0x7f7ba8004740 comp rx=0 tx=0).stop 2024-03-30T00:43:52.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 -- 172.21.15.66:0/2444226205 shutdown_connections 2024-03-30T00:43:52.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 --2- 172.21.15.66:0/2444226205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cd9fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 --2- 172.21.15.66:0/2444226205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cd9fcc70 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 --2- 172.21.15.66:0/2444226205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46290 0x5581cda46670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 -- 172.21.15.66:0/2444226205 >> 172.21.15.66:0/2444226205 conn(0x5581cd9f9160 msgr2=0x5581cd9fb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 -- 172.21.15.66:0/2444226205 shutdown_connections 2024-03-30T00:43:52.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 -- 172.21.15.66:0/2444226205 wait complete. 2024-03-30T00:43:52.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 -- start start 2024-03-30T00:43:52.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 Processor -- start 2024-03-30T00:43:52.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cda026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cda02c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46bb0 0x5581cda07e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cd84ab90 con 0x5581cda46290 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cd8080a0 con 0x5581cd9fd1b0 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581cd807d40 con 0x5581cda46bb0 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 10 client.0 _ll_register_callbacks cb 0x5581cd8ffc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bb7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cda02c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bb7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cda02c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37062/0 (socket says 172.21.15.66:37062) 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bb7e00640 1 -- 172.21.15.66:0/1839903517 learned_addr learned my addr 172.21.15.66:0/1839903517 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cda026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bc5000640 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46bb0 0x5581cda07e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bb7e00640 1 -- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46bb0 msgr2=0x5581cda07e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46bb0 0x5581cda07e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bb7e00640 1 -- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 msgr2=0x5581cda026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cda026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bb7e00640 1 -- 172.21.15.66:0/1839903517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc00cc110 con 0x5581cda46290 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cda02c20 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8000c10 tx=0x7f7ba801ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7529565705233a92 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bad600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bad600640 1 -- 172.21.15.66:0/1839903517 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80c26d0 con 0x5581cda46290 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bad600640 1 -- 172.21.15.66:0/1839903517 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f7ba80d5cc0 con 0x5581cda46290 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 -- 172.21.15.66:0/1839903517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581cd81a550 con 0x5581cda46290 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bad600640 1 -- 172.21.15.66:0/1839903517 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ba80d5e80 con 0x5581cda46290 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 10 client.6771 fetch_fsmap learned FSMap version 34 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 -- 172.21.15.66:0/1839903517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x5581cd80fba0 con 0x5581cda46290 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bad600640 1 -- 172.21.15.66:0/1839903517 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581cd81a550 con 0x5581cda46290 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bad600640 1 -- 172.21.15.66:0/1839903517 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7ba80c2830 con 0x5581cda46290 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 10 client.6771 fetch_fsmap finished waiting for FSMap version 34 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 10 client.6771 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 20 client.6771 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:52.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5581cda55320 0x5581cda57710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 4 client.6771 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.587+0000 7f7bca487000 1 -- 172.21.15.66:0/1839903517 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581cd847970 con 0x5581cda55320 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5581cda55320 0x5581cda57710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5581cda55320 0x5581cda57710 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45b9f0184dd87b8 server_cookie=91c30a02f082ceed in_seq=0 out_seq=0 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bad600640 10 client.6771 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bad600640 1 -- 172.21.15.66:0/1839903517 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5581cd847970 con 0x5581cda55320 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bad600640 10 client.6771 handle_command_reply: tid=0 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 1 client.6771 shutdown 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 1 -- 172.21.15.66:0/1839903517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 msgr2=0x5581cda02c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cda02c20 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8000c10 tx=0x7f7ba801ab40 comp rx=0 tx=0).stop 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 1 -- 172.21.15.66:0/1839903517 shutdown_connections 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5581cda55320 0x5581cda57710 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46bb0 0x5581cda07e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cda02c20 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bad600640 0 client.6771 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 1 --2- 172.21.15.66:0/1839903517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cda026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 1 -- 172.21.15.66:0/1839903517 >> 172.21.15.66:0/1839903517 conn(0x5581cd9f9160 msgr2=0x5581cd9fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 1 -- 172.21.15.66:0/1839903517 shutdown_connections 2024-03-30T00:43:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 1 -- 172.21.15.66:0/1839903517 wait complete. 2024-03-30T00:43:52.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.591+0000 7f7bca487000 20 client.6771 trim_cache size 0 max 16384 2024-03-30T00:43:52.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bca487000 1 Processor -- start 2024-03-30T00:43:52.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bca487000 1 -- start start 2024-03-30T00:43:52.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bca487000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cd9fc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bca487000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cd9fcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bca487000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46bb0 0x5581cda55700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bca487000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cd507c40 con 0x5581cda46290 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bca487000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cd846380 con 0x5581cd9fd1b0 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bca487000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581cd847f90 con 0x5581cda46bb0 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bb7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cd9fcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bb7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cd9fcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37078/0 (socket says 172.21.15.66:37078) 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bb7e00640 1 -- 172.21.15.66:0/937333581 learned_addr learned my addr 172.21.15.66:0/937333581 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/937333581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cd9fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bc5000640 1 --2- 172.21.15.66:0/937333581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46bb0 0x5581cda55700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bb7e00640 1 -- 172.21.15.66:0/937333581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46bb0 msgr2=0x5581cda55700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/937333581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46bb0 0x5581cda55700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bb7e00640 1 -- 172.21.15.66:0/937333581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 msgr2=0x5581cd9fc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/937333581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cd9fc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bb7e00640 1 -- 172.21.15.66:0/937333581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581cd7f5ef0 con 0x5581cda46290 2024-03-30T00:43:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/937333581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cd9fc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.603+0000 7f7bc5000640 1 --2- 172.21.15.66:0/937333581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46bb0 0x5581cda55700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/937333581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cd9fcaa0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7ba801c870 tx=0x7f7ba80cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a5bd68e9d6d5b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bb5600640 1 -- 172.21.15.66:0/937333581 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80cb650 con 0x5581cda46290 2024-03-30T00:43:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bb5600640 1 -- 172.21.15.66:0/937333581 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba80cb7b0 con 0x5581cda46290 2024-03-30T00:43:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bb5600640 1 -- 172.21.15.66:0/937333581 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80cbb90 con 0x5581cda46290 2024-03-30T00:43:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 -- 172.21.15.66:0/937333581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 msgr2=0x5581cd9fcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 --2- 172.21.15.66:0/937333581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cd9fcaa0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7ba801c870 tx=0x7f7ba80cd040 comp rx=0 tx=0).stop 2024-03-30T00:43:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 -- 172.21.15.66:0/937333581 shutdown_connections 2024-03-30T00:43:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 --2- 172.21.15.66:0/937333581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46bb0 0x5581cda55700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 --2- 172.21.15.66:0/937333581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46290 0x5581cd9fcaa0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 --2- 172.21.15.66:0/937333581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cd9fc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 -- 172.21.15.66:0/937333581 >> 172.21.15.66:0/937333581 conn(0x5581cd9f9160 msgr2=0x5581cda44dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 -- 172.21.15.66:0/937333581 shutdown_connections 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 -- 172.21.15.66:0/937333581 wait complete. 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 -- start start 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 Processor -- start 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cda58d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46290 0x5581cdaa3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cdaa40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cd9f45e0 con 0x5581cda46bb0 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cd849a20 con 0x5581cd9fd1b0 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581cd7e7a40 con 0x5581cda46290 2024-03-30T00:43:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 10 client.0 _ll_register_callbacks cb 0x5581cd8e8870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bc5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cdaa40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bc5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cdaa40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37090/0 (socket says 172.21.15.66:37090) 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bc5000640 1 -- 172.21.15.66:0/2040664628 learned_addr learned my addr 172.21.15.66:0/2040664628 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bb7e00640 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46290 0x5581cdaa3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cda58d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bc5000640 1 -- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46290 msgr2=0x5581cdaa3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bc5000640 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46290 0x5581cdaa3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bc5000640 1 -- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 msgr2=0x5581cda58d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bc5000640 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cda58d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bc5000640 1 -- 172.21.15.66:0/2040664628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581cd80fba0 con 0x5581cda46bb0 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bc5000640 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cdaa40c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8002180 tx=0x7f7bb80078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2abc085492d5fad server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bad600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bad600640 1 -- 172.21.15.66:0/2040664628 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8014a50 con 0x5581cda46bb0 2024-03-30T00:43:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bad600640 1 -- 172.21.15.66:0/2040664628 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f7bb8014bb0 con 0x5581cda46bb0 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.607+0000 7f7bca487000 1 -- 172.21.15.66:0/2040664628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581cda00070 con 0x5581cda46bb0 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bad600640 1 -- 172.21.15.66:0/2040664628 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bb8014d70 con 0x5581cda46bb0 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 10 client.6777 fetch_fsmap learned FSMap version 34 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 -- 172.21.15.66:0/2040664628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x5581cd546fc0 con 0x5581cda46bb0 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bad600640 1 -- 172.21.15.66:0/2040664628 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581cda00070 con 0x5581cda46bb0 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bad600640 1 -- 172.21.15.66:0/2040664628 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7bb8017460 con 0x5581cda46bb0 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 10 client.6777 fetch_fsmap finished waiting for FSMap version 34 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 10 client.6777 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 20 client.6777 populate_metadata read hostname 'smithi066' 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5581cda60ad0 0x5581cda60eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 4 client.6777 mds_command: new command op to 6468 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 -- 172.21.15.66:0/2040664628 --> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5581cd7fa620 con 0x5581cda60ad0 2024-03-30T00:43:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5581cda60ad0 0x5581cda60eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bbfe00640 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5581cda60ad0 0x5581cda60eb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ab54321d9e9e73 server_cookie=21d8ceb4ba72c3b1 in_seq=0 out_seq=0 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bad600640 10 client.6777 ms_handle_connect on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bad600640 1 -- 172.21.15.66:0/2040664628 <== mds.0 v2:172.21.15.156:6833/1137593870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581cd7fa620 con 0x5581cda60ad0 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bad600640 10 client.6777 handle_command_reply: tid=0 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 client.6777 shutdown 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 -- 172.21.15.66:0/2040664628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 msgr2=0x5581cdaa40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cdaa40c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8002180 tx=0x7f7bb80078e0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 -- 172.21.15.66:0/2040664628 shutdown_connections 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:6833/1137593870,v1:172.21.15.156:6835/1137593870] conn(0x5581cda60ad0 0x5581cda60eb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5581cda46bb0 0x5581cdaa40c0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5581cda46290 0x5581cdaa3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 --2- 172.21.15.66:0/2040664628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581cd9fd1b0 0x5581cda58d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bad600640 0 client.6777 ms_handle_reset on v2:172.21.15.156:6833/1137593870 2024-03-30T00:43:52.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 -- 172.21.15.66:0/2040664628 >> 172.21.15.66:0/2040664628 conn(0x5581cd9f9160 msgr2=0x5581cda44dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 -- 172.21.15.66:0/2040664628 shutdown_connections 2024-03-30T00:43:52.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 1 -- 172.21.15.66:0/2040664628 wait complete. 2024-03-30T00:43:52.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bca487000 20 client.6777 trim_cache size 0 max 16384 2024-03-30T00:43:52.623 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:52.623 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-03-30T00:43:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 -- 172.21.15.66:0/2895533188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b9c066710 msgr2=0x7f7b9c068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b9c066710 0x7f7b9c068bd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80ad000 tx=0x7f7ba80a6000 comp rx=0 tx=0).stop 2024-03-30T00:43:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 -- 172.21.15.66:0/2895533188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc00ce780 msgr2=0x7f7bc0152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc00ce780 0x7f7bc0152680 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000c500 tx=0x7f7bb000c9c0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 -- 172.21.15.66:0/2895533188 shutdown_connections 2024-03-30T00:43:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bc015d360 0x7f7bc0154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bc00ceb60 0x7f7bc0154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b9c066710 0x7f7b9c068bd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 --2- 172.21.15.66:0/2895533188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc00ce780 0x7f7bc0152680 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 -- 172.21.15.66:0/2895533188 >> 172.21.15.66:0/2895533188 conn(0x7f7bc0069f60 msgr2=0x7f7bc00ab2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 -- 172.21.15.66:0/2895533188 shutdown_connections 2024-03-30T00:43:52.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.611+0000 7f7bc7000640 1 -- 172.21.15.66:0/2895533188 wait complete. 2024-03-30T00:43:52.638 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-30T00:43:52.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae2c00640 1 Processor -- start 2024-03-30T00:43:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae2c00640 1 -- start start 2024-03-30T00:43:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae2c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8adc099060 0x7f8adc0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae2c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8adc0b0470 0x7f8adc0cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae2c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc099760 con 0x7f8adc098c80 2024-03-30T00:43:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae2c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc0cb8f0 con 0x7f8adc0b0470 2024-03-30T00:43:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae2c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0999b0 con 0x7f8adc099060 2024-03-30T00:43:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae0c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8adc0b0470 0x7f8adc0cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae0c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8adc0b0470 0x7f8adc0cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37106/0 (socket says 172.21.15.66:37106) 2024-03-30T00:43:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae0c00640 1 -- 172.21.15.66:0/3772759307 learned_addr learned my addr 172.21.15.66:0/3772759307 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8adbe00640 1 --1- 172.21.15.66:0/3772759307 >> v1:172.21.15.66:6789/0 conn(0x7f8adc098c80 0x7f8adc0ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34030/0 (socket says 172.21.15.66:34030) 2024-03-30T00:43:52.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ad3e00640 1 --2- 172.21.15.66:0/3772759307 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8adc099060 0x7f8adc0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae0c00640 1 -- 172.21.15.66:0/3772759307 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8adc099060 msgr2=0x7f8adc0aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:52.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae0c00640 1 --2- 172.21.15.66:0/3772759307 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8adc099060 0x7f8adc0aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae0c00640 1 -- 172.21.15.66:0/3772759307 >> v1:172.21.15.66:6789/0 conn(0x7f8adc098c80 legacy=0x7f8adc0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ae0c00640 1 -- 172.21.15.66:0/3772759307 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc099c90 con 0x7f8adc0b0470 2024-03-30T00:43:52.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8adb400640 1 -- 172.21.15.66:0/3772759307 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622791787 0 0) 0x7f8adc099760 con 0x7f8adc098c80 2024-03-30T00:43:52.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.771+0000 7f8ad3e00640 1 --2- 172.21.15.66:0/3772759307 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8adc099060 0x7f8adc0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:43:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8ae0c00640 1 --2- 172.21.15.66:0/3772759307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8adc0b0470 0x7f8adc0cbd80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009e50 tx=0x7f8ad401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a350c4138d72c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8adb400640 1 -- 172.21.15.66:0/3772759307 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40a5740 con 0x7f8adc0b0470 2024-03-30T00:43:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8adb400640 1 -- 172.21.15.66:0/3772759307 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad40a58a0 con 0x7f8adc0b0470 2024-03-30T00:43:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8adb400640 1 -- 172.21.15.66:0/3772759307 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40a5c30 con 0x7f8adc0b0470 2024-03-30T00:43:52.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8ae2c00640 1 -- 172.21.15.66:0/3772759307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8adc0b0470 msgr2=0x7f8adc0cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8ae2c00640 1 --2- 172.21.15.66:0/3772759307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8adc0b0470 0x7f8adc0cbd80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009e50 tx=0x7f8ad401c970 comp rx=0 tx=0).stop 2024-03-30T00:43:52.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8ae2c00640 1 -- 172.21.15.66:0/3772759307 shutdown_connections 2024-03-30T00:43:52.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8ae2c00640 1 --2- 172.21.15.66:0/3772759307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8adc0b0470 0x7f8adc0cbd80 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8ae2c00640 1 --2- 172.21.15.66:0/3772759307 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8adc099060 0x7f8adc0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8ae2c00640 1 -- 172.21.15.66:0/3772759307 >> 172.21.15.66:0/3772759307 conn(0x7f8adc0b20c0 msgr2=0x7f8adc0b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8ae2c00640 1 -- 172.21.15.66:0/3772759307 shutdown_connections 2024-03-30T00:43:52.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.775+0000 7f8ae2c00640 1 -- 172.21.15.66:0/3772759307 wait complete. 2024-03-30T00:43:52.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae2c00640 1 Processor -- start 2024-03-30T00:43:52.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae2c00640 1 -- start start 2024-03-30T00:43:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae2c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8adc098c80 0x7f8adc158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae2c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8adc099060 0x7f8adc0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae2c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8adc0b0470 0x7f8adc0c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae2c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0c9e30 con 0x7f8adc099060 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae2c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0c8850 con 0x7f8adc0b0470 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae2c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0c8b30 con 0x7f8adc098c80 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8adc099060 0x7f8adc0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8adc099060 0x7f8adc0c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37116/0 (socket says 172.21.15.66:37116) 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad3e00640 1 -- 172.21.15.66:0/421794459 learned_addr learned my addr 172.21.15.66:0/421794459 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae0c00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8adc0b0470 0x7f8adc0c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8adbe00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8adc098c80 0x7f8adc158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad3e00640 1 -- 172.21.15.66:0/421794459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8adc098c80 msgr2=0x7f8adc158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad3e00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8adc098c80 0x7f8adc158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad3e00640 1 -- 172.21.15.66:0/421794459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8adc0b0470 msgr2=0x7f8adc0c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad3e00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8adc0b0470 0x7f8adc0c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad3e00640 1 -- 172.21.15.66:0/421794459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4009b30 con 0x7f8adc099060 2024-03-30T00:43:52.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad3e00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8adc099060 0x7f8adc0c98f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400c7f0 tx=0x7f8ac400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad8c00640 1 -- 172.21.15.66:0/421794459 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4007820 con 0x7f8adc099060 2024-03-30T00:43:52.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad8c00640 1 -- 172.21.15.66:0/421794459 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac4007980 con 0x7f8adc099060 2024-03-30T00:43:52.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ad8c00640 1 -- 172.21.15.66:0/421794459 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4007d30 con 0x7f8adc099060 2024-03-30T00:43:52.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae2c00640 1 -- 172.21.15.66:0/421794459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc0c8e10 con 0x7f8adc099060 2024-03-30T00:43:52.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae2c00640 1 -- 172.21.15.66:0/421794459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc0c93d0 con 0x7f8adc099060 2024-03-30T00:43:52.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.779+0000 7f8ae2c00640 1 -- 172.21.15.66:0/421794459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc066c30 con 0x7f8adc099060 2024-03-30T00:43:52.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.783+0000 7f8ad8c00640 1 -- 172.21.15.66:0/421794459 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8ac40113f0 con 0x7f8adc099060 2024-03-30T00:43:52.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.783+0000 7f8ad8c00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ab8066690 0x7f8ab8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:52.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.783+0000 7f8ad8c00640 1 -- 172.21.15.66:0/421794459 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f8ac4084640 con 0x7f8adc099060 2024-03-30T00:43:52.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.783+0000 7f8adbe00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ab8066690 0x7f8ab8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:52.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.783+0000 7f8adbe00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ab8066690 0x7f8ab8068b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8acc000f70 tx=0x7f8acc007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:52.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.787+0000 7f8ad8c00640 1 -- 172.21.15.66:0/421794459 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8ac4051880 con 0x7f8adc099060 2024-03-30T00:43:52.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.979+0000 7f8ae2c00640 1 -- 172.21.15.66:0/421794459 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8adc0b2d20 con 0x7f8adc099060 2024-03-30T00:43:52.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.979+0000 7f8ad8c00640 1 -- 172.21.15.66:0/421794459 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8ac4054ea0 con 0x7f8adc099060 2024-03-30T00:43:52.988 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:52.988 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-30T00:43:52.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 -- 172.21.15.66:0/421794459 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ab8066690 msgr2=0x7f8ab8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ab8066690 0x7f8ab8068b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8acc000f70 tx=0x7f8acc007df0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 -- 172.21.15.66:0/421794459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8adc099060 msgr2=0x7f8adc0c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:52.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8adc099060 0x7f8adc0c98f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400c7f0 tx=0x7f8ac400ccb0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 -- 172.21.15.66:0/421794459 shutdown_connections 2024-03-30T00:43:52.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8adc0b0470 0x7f8adc0c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ab8066690 0x7f8ab8068b50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8adc099060 0x7f8adc0c98f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 --2- 172.21.15.66:0/421794459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8adc098c80 0x7f8adc158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:52.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 -- 172.21.15.66:0/421794459 >> 172.21.15.66:0/421794459 conn(0x7f8adc0b20c0 msgr2=0x7f8adc0ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:52.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 -- 172.21.15.66:0/421794459 shutdown_connections 2024-03-30T00:43:52.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:52.983+0000 7f8ae2c00640 1 -- 172.21.15.66:0/421794459 wait complete. 2024-03-30T00:43:53.010 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2024-03-30T00:43:58.011 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-30T00:43:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f4270000640 1 Processor -- start 2024-03-30T00:43:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f4270000640 1 -- start start 2024-03-30T00:43:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f4270000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4268158340 0x7f426815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f4270000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42680cc8b0 0x7f426815be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f4270000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426815bc10 con 0x7f42680cc450 2024-03-30T00:43:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f4270000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4268158720 con 0x7f4268158340 2024-03-30T00:43:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f4270000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42680cb3b0 con 0x7f42680cc8b0 2024-03-30T00:43:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f426d600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4268158340 0x7f426815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f426e000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42680cc8b0 0x7f426815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f426d600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4268158340 0x7f426815b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:55784/0 (socket says 172.21.15.66:55784) 2024-03-30T00:43:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f426d600640 1 -- 172.21.15.66:0/2437226405 learned_addr learned my addr 172.21.15.66:0/2437226405 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f426cc00640 1 --1- 172.21.15.66:0/2437226405 >> v1:172.21.15.156:6789/0 conn(0x7f42680cc450 0x7f42680ce870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:56048/0 (socket says 172.21.15.66:56048) 2024-03-30T00:43:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f426e000640 1 -- 172.21.15.66:0/2437226405 >> v2:172.21.15.156:3300/0 conn(0x7f4268158340 msgr2=0x7f426815b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f426e000640 1 --2- 172.21.15.66:0/2437226405 >> v2:172.21.15.156:3300/0 conn(0x7f4268158340 0x7f426815b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f426e000640 1 -- 172.21.15.66:0/2437226405 >> v1:172.21.15.156:6789/0 conn(0x7f42680cc450 legacy=0x7f42680ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f426e000640 1 -- 172.21.15.66:0/2437226405 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42680cb690 con 0x7f42680cc8b0 2024-03-30T00:43:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.155+0000 7f426d600640 1 --2- 172.21.15.66:0/2437226405 >> v2:172.21.15.156:3300/0 conn(0x7f4268158340 0x7f426815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:43:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f426e000640 1 --2- 172.21.15.66:0/2437226405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42680cc8b0 0x7f426815be10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4260009d90 tx=0x7f426001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6060cbaf643718b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f4267e00640 1 -- 172.21.15.66:0/2437226405 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600a5a00 con 0x7f42680cc8b0 2024-03-30T00:43:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f4267e00640 1 -- 172.21.15.66:0/2437226405 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42600a5b60 con 0x7f42680cc8b0 2024-03-30T00:43:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f4267e00640 1 -- 172.21.15.66:0/2437226405 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600ad440 con 0x7f42680cc8b0 2024-03-30T00:43:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f4270000640 1 -- 172.21.15.66:0/2437226405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42680cc8b0 msgr2=0x7f426815be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f4270000640 1 --2- 172.21.15.66:0/2437226405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42680cc8b0 0x7f426815be10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4260009d90 tx=0x7f426001c940 comp rx=0 tx=0).stop 2024-03-30T00:43:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f4270000640 1 -- 172.21.15.66:0/2437226405 shutdown_connections 2024-03-30T00:43:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f4270000640 1 --2- 172.21.15.66:0/2437226405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42680cc8b0 0x7f426815be10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f4270000640 1 --2- 172.21.15.66:0/2437226405 >> v2:172.21.15.156:3300/0 conn(0x7f4268158340 0x7f426815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f4270000640 1 -- 172.21.15.66:0/2437226405 >> 172.21.15.66:0/2437226405 conn(0x7f4268089fb0 msgr2=0x7f426808a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f4270000640 1 -- 172.21.15.66:0/2437226405 shutdown_connections 2024-03-30T00:43:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.159+0000 7f4270000640 1 -- 172.21.15.66:0/2437226405 wait complete. 2024-03-30T00:43:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4270000640 1 Processor -- start 2024-03-30T00:43:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4270000640 1 -- start start 2024-03-30T00:43:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4270000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42680cc450 0x7f4268152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4270000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42680cc8b0 0x7f42681455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4270000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4268158340 0x7f42680c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4270000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4268151e20 con 0x7f42680cc450 2024-03-30T00:43:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4270000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42680c89a0 con 0x7f4268158340 2024-03-30T00:43:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4270000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42680c86c0 con 0x7f42680cc8b0 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f426d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42680cc450 0x7f4268152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f426d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42680cc450 0x7f4268152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37118/0 (socket says 172.21.15.66:37118) 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f426d600640 1 -- 172.21.15.66:0/931563442 learned_addr learned my addr 172.21.15.66:0/931563442 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f426cc00640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42680cc8b0 0x7f42681455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f426d600640 1 -- 172.21.15.66:0/931563442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42680cc8b0 msgr2=0x7f42681455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f426d600640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42680cc8b0 0x7f42681455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f426d600640 1 -- 172.21.15.66:0/931563442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4268158340 msgr2=0x7f42680c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f426d600640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4268158340 0x7f42680c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f426d600640 1 -- 172.21.15.66:0/931563442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0094d0 con 0x7f42680cc450 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f426d600640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42680cc450 0x7f4268152240 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f425c00e930 tx=0x7f425c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4265600640 1 -- 172.21.15.66:0/931563442 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c00ca00 con 0x7f42680cc450 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4265600640 1 -- 172.21.15.66:0/931563442 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c00cb60 con 0x7f42680cc450 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4270000640 1 -- 172.21.15.66:0/931563442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4260009a70 con 0x7f42680cc450 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4265600640 1 -- 172.21.15.66:0/931563442 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c013470 con 0x7f42680cc450 2024-03-30T00:43:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.163+0000 7f4270000640 1 -- 172.21.15.66:0/931563442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4268151a40 con 0x7f42680cc450 2024-03-30T00:43:58.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.167+0000 7f4270000640 1 -- 172.21.15.66:0/931563442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4268066c30 con 0x7f42680cc450 2024-03-30T00:43:58.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.167+0000 7f4265600640 1 -- 172.21.15.66:0/931563442 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f425c0135d0 con 0x7f42680cc450 2024-03-30T00:43:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.167+0000 7f4265600640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4248066690 0x7f4248068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.167+0000 7f4265600640 1 -- 172.21.15.66:0/931563442 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f425c0849e0 con 0x7f42680cc450 2024-03-30T00:43:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.167+0000 7f426cc00640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4248066690 0x7f4248068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.167+0000 7f426cc00640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4248066690 0x7f4248068b50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4268149ad0 tx=0x7f4254007e50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:58.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.171+0000 7f4265600640 1 -- 172.21.15.66:0/931563442 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f425c051b00 con 0x7f42680cc450 2024-03-30T00:43:58.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.363+0000 7f4270000640 1 -- 172.21.15.66:0/931563442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f42680d0900 con 0x7f42680cc450 2024-03-30T00:43:58.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.363+0000 7f4265600640 1 -- 172.21.15.66:0/931563442 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f425c021040 con 0x7f42680cc450 2024-03-30T00:43:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:58.372 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-30T00:43:58.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 -- 172.21.15.66:0/931563442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4248066690 msgr2=0x7f4248068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:58.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4248066690 0x7f4248068b50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4268149ad0 tx=0x7f4254007e50 comp rx=0 tx=0).stop 2024-03-30T00:43:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 -- 172.21.15.66:0/931563442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42680cc450 msgr2=0x7f4268152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42680cc450 0x7f4268152240 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f425c00e930 tx=0x7f425c00edf0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 -- 172.21.15.66:0/931563442 shutdown_connections 2024-03-30T00:43:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4268158340 0x7f42680c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42680cc8b0 0x7f42681455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4248066690 0x7f4248068b50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 --2- 172.21.15.66:0/931563442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42680cc450 0x7f4268152240 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 -- 172.21.15.66:0/931563442 >> 172.21.15.66:0/931563442 conn(0x7f4268089fb0 msgr2=0x7f426806f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 -- 172.21.15.66:0/931563442 shutdown_connections 2024-03-30T00:43:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.367+0000 7f4270000640 1 -- 172.21.15.66:0/931563442 wait complete. 2024-03-30T00:43:58.393 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-30T00:43:58.393 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:43:58.394 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:43:58.487 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:43:58.590 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:43:58.590 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:43:58.590 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:43:58.599 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:43:58.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ec2e00640 1 Processor -- start 2024-03-30T00:43:58.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ec2e00640 1 -- start start 2024-03-30T00:43:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ec2e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ebc0cda90 0x7f9ebc0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ec2e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9ebc0cde70 0x7f9ebc0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ec2e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ebc159c60 0x7f9ebc15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ec2e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ebc0b1b80 con 0x7f9ebc0cda90 2024-03-30T00:43:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ec2e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ebc15a990 con 0x7f9ebc0cde70 2024-03-30T00:43:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ec2e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0b11e0 con 0x7f9ebc159c60 2024-03-30T00:43:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ebbe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ebc0cda90 0x7f9ebc0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ebbe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ebc0cda90 0x7f9ebc0ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33536/0 (socket says 172.21.15.66:33536) 2024-03-30T00:43:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ebbe00640 1 -- 172.21.15.66:0/2977834718 learned_addr learned my addr 172.21.15.66:0/2977834718 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ec0e00640 1 --2- 172.21.15.66:0/2977834718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ebc159c60 0x7f9ebc15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9eb3e00640 1 --2- 172.21.15.66:0/2977834718 >> v2:172.21.15.156:3300/0 conn(0x7f9ebc0cde70 0x7f9ebc0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ebbe00640 1 -- 172.21.15.66:0/2977834718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ebc159c60 msgr2=0x7f9ebc15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ebbe00640 1 --2- 172.21.15.66:0/2977834718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ebc159c60 0x7f9ebc15a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ebbe00640 1 -- 172.21.15.66:0/2977834718 >> v2:172.21.15.156:3300/0 conn(0x7f9ebc0cde70 msgr2=0x7f9ebc0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ebbe00640 1 --2- 172.21.15.66:0/2977834718 >> v2:172.21.15.156:3300/0 conn(0x7f9ebc0cde70 0x7f9ebc0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.779+0000 7f9ebbe00640 1 -- 172.21.15.66:0/2977834718 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc0b14c0 con 0x7f9ebc0cda90 2024-03-30T00:43:58.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ebbe00640 1 --2- 172.21.15.66:0/2977834718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ebc0cda90 0x7f9ebc0ce520 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0cc320 tx=0x7f9eac01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74fcfb0545d2a56 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:58.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ebb400640 1 -- 172.21.15.66:0/2977834718 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0ab670 con 0x7f9ebc0cda90 2024-03-30T00:43:58.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ebb400640 1 -- 172.21.15.66:0/2977834718 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac0ab7d0 con 0x7f9ebc0cda90 2024-03-30T00:43:58.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ebb400640 1 -- 172.21.15.66:0/2977834718 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0abb60 con 0x7f9ebc0cda90 2024-03-30T00:43:58.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ec2e00640 1 -- 172.21.15.66:0/2977834718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ebc0cda90 msgr2=0x7f9ebc0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:58.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ec2e00640 1 --2- 172.21.15.66:0/2977834718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ebc0cda90 0x7f9ebc0ce520 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0cc320 tx=0x7f9eac01c490 comp rx=0 tx=0).stop 2024-03-30T00:43:58.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ec2e00640 1 -- 172.21.15.66:0/2977834718 shutdown_connections 2024-03-30T00:43:58.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ec2e00640 1 --2- 172.21.15.66:0/2977834718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ebc159c60 0x7f9ebc15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ec2e00640 1 --2- 172.21.15.66:0/2977834718 >> v2:172.21.15.156:3300/0 conn(0x7f9ebc0cde70 0x7f9ebc0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ec2e00640 1 --2- 172.21.15.66:0/2977834718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ebc0cda90 0x7f9ebc0ce520 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ec2e00640 1 -- 172.21.15.66:0/2977834718 >> 172.21.15.66:0/2977834718 conn(0x7f9ebc069f60 msgr2=0x7f9ebc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:58.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ec2e00640 1 -- 172.21.15.66:0/2977834718 shutdown_connections 2024-03-30T00:43:58.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.783+0000 7f9ec2e00640 1 -- 172.21.15.66:0/2977834718 wait complete. 2024-03-30T00:43:58.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ec2e00640 1 Processor -- start 2024-03-30T00:43:58.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ec2e00640 1 -- start start 2024-03-30T00:43:58.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ec2e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ebc0cda90 0x7f9ebc151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ec2e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ebc0cde70 0x7f9ebc0c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ec2e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ebc159c60 0x7f9ebc06cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ec2e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc159910 con 0x7f9ebc0cde70 2024-03-30T00:43:58.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ec2e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc159630 con 0x7f9ebc159c60 2024-03-30T00:43:58.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ec2e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc06c4b0 con 0x7f9ebc0cda90 2024-03-30T00:43:58.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ebc0cde70 0x7f9ebc0c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ebc0cde70 0x7f9ebc0c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33548/0 (socket says 172.21.15.66:33548) 2024-03-30T00:43:58.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb3e00640 1 -- 172.21.15.66:0/57413561 learned_addr learned my addr 172.21.15.66:0/57413561 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:58.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ebbe00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ebc0cda90 0x7f9ebc151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ec0e00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ebc159c60 0x7f9ebc06cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb3e00640 1 -- 172.21.15.66:0/57413561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ebc0cda90 msgr2=0x7f9ebc151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb3e00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ebc0cda90 0x7f9ebc151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb3e00640 1 -- 172.21.15.66:0/57413561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ebc159c60 msgr2=0x7f9ebc06cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb3e00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ebc159c60 0x7f9ebc06cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb3e00640 1 -- 172.21.15.66:0/57413561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac0094d0 con 0x7f9ebc0cde70 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb3e00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ebc0cde70 0x7f9ebc0c3530 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400d7d0 tx=0x7f9ea400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb8c00640 1 -- 172.21.15.66:0/57413561 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40096a0 con 0x7f9ebc0cde70 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb8c00640 1 -- 172.21.15.66:0/57413561 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea4009800 con 0x7f9ebc0cde70 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9eb8c00640 1 -- 172.21.15.66:0/57413561 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4009bb0 con 0x7f9ebc0cde70 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ec2e00640 1 -- 172.21.15.66:0/57413561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ebc06c790 con 0x7f9ebc0cde70 2024-03-30T00:43:58.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.787+0000 7f9ec2e00640 1 -- 172.21.15.66:0/57413561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc0c3a70 con 0x7f9ebc0cde70 2024-03-30T00:43:58.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.791+0000 7f9ec2e00640 1 -- 172.21.15.66:0/57413561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ebc06c310 con 0x7f9ebc0cde70 2024-03-30T00:43:58.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.791+0000 7f9eb8c00640 1 -- 172.21.15.66:0/57413561 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9ea4009d10 con 0x7f9ebc0cde70 2024-03-30T00:43:58.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.791+0000 7f9eb8c00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e98066690 0x7f9e98068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:58.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.791+0000 7f9eb8c00640 1 -- 172.21.15.66:0/57413561 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f9ea4084760 con 0x7f9ebc0cde70 2024-03-30T00:43:58.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.791+0000 7f9ebbe00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e98066690 0x7f9e98068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:58.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.791+0000 7f9ebbe00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e98066690 0x7f9e98068b50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0af000 tx=0x7f9eac0b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:58.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.795+0000 7f9eb8c00640 1 -- 172.21.15.66:0/57413561 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9ebc06c310 con 0x7f9ebc0cde70 2024-03-30T00:43:58.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.959+0000 7f9ec2e00640 1 -- 172.21.15.66:0/57413561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ebc06c310 con 0x7f9ebc0cde70 2024-03-30T00:43:58.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.959+0000 7f9eb8c00640 1 -- 172.21.15.66:0/57413561 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16487 (secure 0 0 0) 0x7f9ebc06c310 con 0x7f9ebc0cde70 2024-03-30T00:43:58.969 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:43:58.969 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":70,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:43:34.562474+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:43:09.681968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T00:43:10.133411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.38","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 -- 172.21.15.66:0/57413561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e98066690 msgr2=0x7f9e98068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e98066690 0x7f9e98068b50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0af000 tx=0x7f9eac0b6040 comp rx=0 tx=0).stop 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 -- 172.21.15.66:0/57413561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ebc0cde70 msgr2=0x7f9ebc0c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ebc0cde70 0x7f9ebc0c3530 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400d7d0 tx=0x7f9ea400dc90 comp rx=0 tx=0).stop 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 -- 172.21.15.66:0/57413561 shutdown_connections 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ebc159c60 0x7f9ebc06cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9e98066690 0x7f9e98068b50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ebc0cde70 0x7f9ebc0c3530 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 --2- 172.21.15.66:0/57413561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ebc0cda90 0x7f9ebc151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 -- 172.21.15.66:0/57413561 >> 172.21.15.66:0/57413561 conn(0x7f9ebc069f60 msgr2=0x7f9ebc15ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 -- 172.21.15.66:0/57413561 shutdown_connections 2024-03-30T00:43:58.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:58.963+0000 7f9ec2e00640 1 -- 172.21.15.66:0/57413561 wait complete. 2024-03-30T00:43:58.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:43:09.681968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:43:10.133411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:43:58.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:43:59.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa722e00640 1 Processor -- start 2024-03-30T00:43:59.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa722e00640 1 -- start start 2024-03-30T00:43:59.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa722e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa71c0ceb60 0x7fa71c15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa722e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa71c15d360 0x7fa71c0cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa722e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71c0cbec0 con 0x7fa71c0ce780 2024-03-30T00:43:59.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa722e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71c15d970 con 0x7fa71c15d360 2024-03-30T00:43:59.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa722e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71c15dc50 con 0x7fa71c0ceb60 2024-03-30T00:43:59.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa71b400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa71c0ceb60 0x7fa71c15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa71be00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa71c0ce780 0x7fa71c0cc910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:55642/0 (socket says 172.21.15.66:55642) 2024-03-30T00:43:59.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa71be00640 1 -- 172.21.15.66:0/2172591275 learned_addr learned my addr 172.21.15.66:0/2172591275 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:59.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa720e00640 1 --2- 172.21.15.66:0/2172591275 >> v2:172.21.15.66:3300/0 conn(0x7fa71c15d360 0x7fa71c0cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa720e00640 1 -- 172.21.15.66:0/2172591275 >> v2:172.21.15.156:3300/0 conn(0x7fa71c0ceb60 msgr2=0x7fa71c15ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:59.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa720e00640 1 --2- 172.21.15.66:0/2172591275 >> v2:172.21.15.156:3300/0 conn(0x7fa71c0ceb60 0x7fa71c15ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa720e00640 1 -- 172.21.15.66:0/2172591275 >> v1:172.21.15.156:6789/0 conn(0x7fa71c0ce780 legacy=0x7fa71c0cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.127+0000 7fa720e00640 1 -- 172.21.15.66:0/2172591275 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c0cc110 con 0x7fa71c15d360 2024-03-30T00:43:59.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa720e00640 1 --2- 172.21.15.66:0/2172591275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa71c15d360 0x7fa71c0cd5e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0cd020 tx=0x7fa71401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffe7b7cb8c16e13 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa71aa00640 1 -- 172.21.15.66:0/2172591275 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7140a5930 con 0x7fa71c15d360 2024-03-30T00:43:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa71aa00640 1 -- 172.21.15.66:0/2172591275 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7140a5a90 con 0x7fa71c15d360 2024-03-30T00:43:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa71aa00640 1 -- 172.21.15.66:0/2172591275 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7140a5e20 con 0x7fa71c15d360 2024-03-30T00:43:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa722e00640 1 -- 172.21.15.66:0/2172591275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa71c15d360 msgr2=0x7fa71c0cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa722e00640 1 --2- 172.21.15.66:0/2172591275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa71c15d360 0x7fa71c0cd5e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0cd020 tx=0x7fa71401c870 comp rx=0 tx=0).stop 2024-03-30T00:43:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa722e00640 1 -- 172.21.15.66:0/2172591275 shutdown_connections 2024-03-30T00:43:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa722e00640 1 --2- 172.21.15.66:0/2172591275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa71c15d360 0x7fa71c0cd5e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa722e00640 1 --2- 172.21.15.66:0/2172591275 >> v2:172.21.15.156:3300/0 conn(0x7fa71c0ceb60 0x7fa71c15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa722e00640 1 -- 172.21.15.66:0/2172591275 >> 172.21.15.66:0/2172591275 conn(0x7fa71c069f60 msgr2=0x7fa71c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa722e00640 1 -- 172.21.15.66:0/2172591275 shutdown_connections 2024-03-30T00:43:59.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa722e00640 1 -- 172.21.15.66:0/2172591275 wait complete. 2024-03-30T00:43:59.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.131+0000 7fa722e00640 1 Processor -- start 2024-03-30T00:43:59.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa722e00640 1 -- start start 2024-03-30T00:43:59.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa722e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa71c0ce780 0x7fa71c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa722e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa71c0ceb60 0x7fa71c0c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa722e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa71c15d360 0x7fa71c0af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa722e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c070840 con 0x7fa71c0ce780 2024-03-30T00:43:59.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa722e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c0c7c50 con 0x7fa71c15d360 2024-03-30T00:43:59.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa722e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c0c7f30 con 0x7fa71c0ceb60 2024-03-30T00:43:59.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa71be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa71c0ce780 0x7fa71c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa71be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa71c0ce780 0x7fa71c152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33566/0 (socket says 172.21.15.66:33566) 2024-03-30T00:43:59.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa71be00640 1 -- 172.21.15.66:0/1741310976 learned_addr learned my addr 172.21.15.66:0/1741310976 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:59.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa720e00640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa71c15d360 0x7fa71c0af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa71b400640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa71c0ceb60 0x7fa71c0c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa71be00640 1 -- 172.21.15.66:0/1741310976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa71c0ceb60 msgr2=0x7fa71c0c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa71be00640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa71c0ceb60 0x7fa71c0c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa71be00640 1 -- 172.21.15.66:0/1741310976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa71c15d360 msgr2=0x7fa71c0af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa71be00640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa71c15d360 0x7fa71c0af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa71be00640 1 -- 172.21.15.66:0/1741310976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa714009a70 con 0x7fa71c0ce780 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa71b400640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa71c0ceb60 0x7fa71c0c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa71be00640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa71c0ce780 0x7fa71c152ce0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00c500 tx=0x7fa70c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa713e00640 1 -- 172.21.15.66:0/1741310976 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c005cf0 con 0x7fa71c0ce780 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa713e00640 1 -- 172.21.15.66:0/1741310976 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c005e50 con 0x7fa71c0ce780 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa713e00640 1 -- 172.21.15.66:0/1741310976 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0072f0 con 0x7fa71c0ce780 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa722e00640 1 -- 172.21.15.66:0/1741310976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71c0c8210 con 0x7fa71c0ce780 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa722e00640 1 -- 172.21.15.66:0/1741310976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71c0c5ac0 con 0x7fa71c0ce780 2024-03-30T00:43:59.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.135+0000 7fa722e00640 1 -- 172.21.15.66:0/1741310976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa71c066c30 con 0x7fa71c0ce780 2024-03-30T00:43:59.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.139+0000 7fa713e00640 1 -- 172.21.15.66:0/1741310976 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa70c007450 con 0x7fa71c0ce780 2024-03-30T00:43:59.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.139+0000 7fa713e00640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6fc0666c0 0x7fa6fc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.139+0000 7fa713e00640 1 -- 172.21.15.66:0/1741310976 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fa70c083870 con 0x7fa71c0ce780 2024-03-30T00:43:59.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.139+0000 7fa71b400640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6fc0666c0 0x7fa6fc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.139+0000 7fa71b400640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6fc0666c0 0x7fa6fc068b80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa71c09c210 tx=0x7fa708008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:59.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.143+0000 7fa713e00640 1 -- 172.21.15.66:0/1741310976 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa70c050ab0 con 0x7fa71c0ce780 2024-03-30T00:43:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.307+0000 7fa722e00640 1 -- 172.21.15.66:0/1741310976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa71c0b2090 con 0x7fa71c0ce780 2024-03-30T00:43:59.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa713e00640 1 -- 172.21.15.66:0/1741310976 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa70c0540d0 con 0x7fa71c0ce780 2024-03-30T00:43:59.317 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:43:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 -- 172.21.15.66:0/1741310976 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6fc0666c0 msgr2=0x7fa6fc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6fc0666c0 0x7fa6fc068b80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa71c09c210 tx=0x7fa708008040 comp rx=0 tx=0).stop 2024-03-30T00:43:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 -- 172.21.15.66:0/1741310976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa71c0ce780 msgr2=0x7fa71c152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa71c0ce780 0x7fa71c152ce0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00c500 tx=0x7fa70c00c9c0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 -- 172.21.15.66:0/1741310976 shutdown_connections 2024-03-30T00:43:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa71c15d360 0x7fa71c0af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa71c0ceb60 0x7fa71c0c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa6fc0666c0 0x7fa6fc068b80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 --2- 172.21.15.66:0/1741310976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa71c0ce780 0x7fa71c152ce0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 -- 172.21.15.66:0/1741310976 >> 172.21.15.66:0/1741310976 conn(0x7fa71c069f60 msgr2=0x7fa71c071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 -- 172.21.15.66:0/1741310976 shutdown_connections 2024-03-30T00:43:59.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.311+0000 7fa722e00640 1 -- 172.21.15.66:0/1741310976 wait complete. 2024-03-30T00:43:59.341 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:43:59.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.475+0000 7fe787c00640 1 Processor -- start 2024-03-30T00:43:59.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.475+0000 7fe787c00640 1 -- start start 2024-03-30T00:43:59.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe787c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe7800cda90 0x7fe7800ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe787c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe7800cde70 0x7fe7800cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe787c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78015a990 con 0x7fe780159c60 2024-03-30T00:43:59.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe787c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7800b1b80 con 0x7fe7800cda90 2024-03-30T00:43:59.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe787c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7800cd3e0 con 0x7fe7800cde70 2024-03-30T00:43:59.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe785200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe7800cda90 0x7fe7800ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe785200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe7800cda90 0x7fe7800ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33568/0 (socket says 172.21.15.66:33568) 2024-03-30T00:43:59.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe785200640 1 -- 172.21.15.66:0/3077833821 learned_addr learned my addr 172.21.15.66:0/3077833821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:59.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe77fe00640 1 --2- 172.21.15.66:0/3077833821 >> v2:172.21.15.156:3300/0 conn(0x7fe7800cde70 0x7fe7800cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe785200640 1 -- 172.21.15.66:0/3077833821 >> v2:172.21.15.156:3300/0 conn(0x7fe7800cde70 msgr2=0x7fe7800cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:59.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe785200640 1 --2- 172.21.15.66:0/3077833821 >> v2:172.21.15.156:3300/0 conn(0x7fe7800cde70 0x7fe7800cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe785200640 1 -- 172.21.15.66:0/3077833821 >> v1:172.21.15.156:6789/0 conn(0x7fe780159c60 legacy=0x7fe78015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:59.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.479+0000 7fe785200640 1 -- 172.21.15.66:0/3077833821 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7800b11e0 con 0x7fe7800cda90 2024-03-30T00:43:59.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe785200640 1 --2- 172.21.15.66:0/3077833821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cda90 0x7fe7800ce520 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe7800cc320 tx=0x7fe77401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b9380ce33a9dcf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:59.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe77f400640 1 -- 172.21.15.66:0/3077833821 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7740ad660 con 0x7fe7800cda90 2024-03-30T00:43:59.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe77f400640 1 -- 172.21.15.66:0/3077833821 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7740ad7c0 con 0x7fe7800cda90 2024-03-30T00:43:59.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe77f400640 1 -- 172.21.15.66:0/3077833821 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7740adb50 con 0x7fe7800cda90 2024-03-30T00:43:59.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe787c00640 1 -- 172.21.15.66:0/3077833821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cda90 msgr2=0x7fe7800ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe787c00640 1 --2- 172.21.15.66:0/3077833821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cda90 0x7fe7800ce520 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe7800cc320 tx=0x7fe77401c450 comp rx=0 tx=0).stop 2024-03-30T00:43:59.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe787c00640 1 -- 172.21.15.66:0/3077833821 shutdown_connections 2024-03-30T00:43:59.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe787c00640 1 --2- 172.21.15.66:0/3077833821 >> v2:172.21.15.156:3300/0 conn(0x7fe7800cde70 0x7fe7800cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe787c00640 1 --2- 172.21.15.66:0/3077833821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cda90 0x7fe7800ce520 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe787c00640 1 -- 172.21.15.66:0/3077833821 >> 172.21.15.66:0/3077833821 conn(0x7fe780069f60 msgr2=0x7fe78006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:59.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe787c00640 1 -- 172.21.15.66:0/3077833821 shutdown_connections 2024-03-30T00:43:59.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe787c00640 1 -- 172.21.15.66:0/3077833821 wait complete. 2024-03-30T00:43:59.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe787c00640 1 Processor -- start 2024-03-30T00:43:59.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.483+0000 7fe787c00640 1 -- start start 2024-03-30T00:43:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe787c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cda90 0x7fe7801f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe787c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7800cde70 0x7fe7800abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe787c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe780159c60 0x7fe7800aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe787c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800ad370 con 0x7fe7800cda90 2024-03-30T00:43:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe787c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800ac760 con 0x7fe780159c60 2024-03-30T00:43:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe787c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800aca40 con 0x7fe7800cde70 2024-03-30T00:43:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe785200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cda90 0x7fe7801f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe785200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cda90 0x7fe7801f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33580/0 (socket says 172.21.15.66:33580) 2024-03-30T00:43:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe785200640 1 -- 172.21.15.66:0/3008626131 learned_addr learned my addr 172.21.15.66:0/3008626131 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe785200640 1 -- 172.21.15.66:0/3008626131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7800cde70 msgr2=0x7fe7800abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe785200640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7800cde70 0x7fe7800abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe785200640 1 -- 172.21.15.66:0/3008626131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe780159c60 msgr2=0x7fe7800aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe785200640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe780159c60 0x7fe7800aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe785200640 1 -- 172.21.15.66:0/3008626131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740094d0 con 0x7fe7800cda90 2024-03-30T00:43:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe785200640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cda90 0x7fe7801f5a80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe77401a080 tx=0x7fe7740191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe77cc00640 1 -- 172.21.15.66:0/3008626131 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7740b6640 con 0x7fe7800cda90 2024-03-30T00:43:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe77cc00640 1 -- 172.21.15.66:0/3008626131 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7740b67a0 con 0x7fe7800cda90 2024-03-30T00:43:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe787c00640 1 -- 172.21.15.66:0/3008626131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7800acd20 con 0x7fe7800cda90 2024-03-30T00:43:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe77cc00640 1 -- 172.21.15.66:0/3008626131 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7740b6b50 con 0x7fe7800cda90 2024-03-30T00:43:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe787c00640 1 -- 172.21.15.66:0/3008626131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7800ac3b0 con 0x7fe7800cda90 2024-03-30T00:43:59.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe787c00640 1 -- 172.21.15.66:0/3008626131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe780066c30 con 0x7fe7800cda90 2024-03-30T00:43:59.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.487+0000 7fe77cc00640 1 -- 172.21.15.66:0/3008626131 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe7740b4070 con 0x7fe7800cda90 2024-03-30T00:43:59.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.491+0000 7fe77cc00640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe760066690 0x7fe760068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.491+0000 7fe77cc00640 1 -- 172.21.15.66:0/3008626131 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fe77411e500 con 0x7fe7800cda90 2024-03-30T00:43:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.491+0000 7fe77fe00640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe760066690 0x7fe760068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.491+0000 7fe77fe00640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe760066690 0x7fe760068b50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe7800c8820 tx=0x7fe76c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.495+0000 7fe77cc00640 1 -- 172.21.15.66:0/3008626131 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe7740eb720 con 0x7fe7800cda90 2024-03-30T00:43:59.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 -- 172.21.15.66:0/3008626131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe7800c1bc0 con 0x7fe7800cda90 2024-03-30T00:43:59.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe77cc00640 1 -- 172.21.15.66:0/3008626131 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe7740eed40 con 0x7fe7800cda90 2024-03-30T00:43:59.662 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:43:59.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 -- 172.21.15.66:0/3008626131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe760066690 msgr2=0x7fe760068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe760066690 0x7fe760068b50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe7800c8820 tx=0x7fe76c008040 comp rx=0 tx=0).stop 2024-03-30T00:43:59.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 -- 172.21.15.66:0/3008626131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cda90 msgr2=0x7fe7801f5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cda90 0x7fe7801f5a80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe77401a080 tx=0x7fe7740191f0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 -- 172.21.15.66:0/3008626131 shutdown_connections 2024-03-30T00:43:59.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe780159c60 0x7fe7800aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7800cde70 0x7fe7800abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe760066690 0x7fe760068b50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 --2- 172.21.15.66:0/3008626131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cda90 0x7fe7801f5a80 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 -- 172.21.15.66:0/3008626131 >> 172.21.15.66:0/3008626131 conn(0x7fe780069f60 msgr2=0x7fe78015d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:59.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 -- 172.21.15.66:0/3008626131 shutdown_connections 2024-03-30T00:43:59.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.655+0000 7fe787c00640 1 -- 172.21.15.66:0/3008626131 wait complete. 2024-03-30T00:43:59.684 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:43:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.819+0000 7fdaa2600640 1 Processor -- start 2024-03-30T00:43:59.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.819+0000 7fdaa2600640 1 -- start start 2024-03-30T00:43:59.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.819+0000 7fdaa2600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9c0cda90 0x7fda9c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fdaa2600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fda9c15a370 0x7fda9c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fdaa2600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda9c15a990 con 0x7fda9c0cde70 2024-03-30T00:43:59.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fdaa2600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9c0b1b80 con 0x7fda9c15a370 2024-03-30T00:43:59.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fdaa2600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c0cd3e0 con 0x7fda9c0cda90 2024-03-30T00:43:59.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fda93e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fda9c0cde70 0x7fda9c159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46194/0 (socket says 172.21.15.66:46194) 2024-03-30T00:43:59.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fda93e00640 1 -- 172.21.15.66:0/217381284 learned_addr learned my addr 172.21.15.66:0/217381284 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:43:59.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fda9be00640 1 --2- 172.21.15.66:0/217381284 >> v2:172.21.15.66:3300/0 conn(0x7fda9c15a370 0x7fda9c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fda9b400640 1 --2- 172.21.15.66:0/217381284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9c0cda90 0x7fda9c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fda9aa00640 1 -- 172.21.15.66:0/217381284 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219247996 0 0) 0x7fda9c15a990 con 0x7fda9c0cde70 2024-03-30T00:43:59.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fda9aa00640 1 -- 172.21.15.66:0/217381284 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda80003530 con 0x7fda9c0cde70 2024-03-30T00:43:59.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fda9be00640 1 -- 172.21.15.66:0/217381284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9c0cda90 msgr2=0x7fda9c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:43:59.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fda9be00640 1 --2- 172.21.15.66:0/217381284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9c0cda90 0x7fda9c0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fda9be00640 1 -- 172.21.15.66:0/217381284 >> v1:172.21.15.66:6789/0 conn(0x7fda9c0cde70 legacy=0x7fda9c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fda9be00640 1 -- 172.21.15.66:0/217381284 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c0b11e0 con 0x7fda9c15a370 2024-03-30T00:43:59.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.823+0000 7fda9be00640 1 --2- 172.21.15.66:0/217381284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda9c15a370 0x7fda9c0cea60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fda94009d90 tx=0x7fda9401eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16fd324e3626f6d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:59.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fda9aa00640 1 -- 172.21.15.66:0/217381284 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda940a7ad0 con 0x7fda9c15a370 2024-03-30T00:43:59.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fda9aa00640 1 -- 172.21.15.66:0/217381284 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda940a7c30 con 0x7fda9c15a370 2024-03-30T00:43:59.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fda9aa00640 1 -- 172.21.15.66:0/217381284 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda940af440 con 0x7fda9c15a370 2024-03-30T00:43:59.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 -- 172.21.15.66:0/217381284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda9c15a370 msgr2=0x7fda9c0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 --2- 172.21.15.66:0/217381284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda9c15a370 0x7fda9c0cea60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fda94009d90 tx=0x7fda9401eaf0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 -- 172.21.15.66:0/217381284 shutdown_connections 2024-03-30T00:43:59.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 --2- 172.21.15.66:0/217381284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda9c15a370 0x7fda9c0cea60 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 --2- 172.21.15.66:0/217381284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9c0cda90 0x7fda9c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 -- 172.21.15.66:0/217381284 >> 172.21.15.66:0/217381284 conn(0x7fda9c069f60 msgr2=0x7fda9c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:43:59.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 -- 172.21.15.66:0/217381284 shutdown_connections 2024-03-30T00:43:59.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 -- 172.21.15.66:0/217381284 wait complete. 2024-03-30T00:43:59.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 Processor -- start 2024-03-30T00:43:59.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 -- start start 2024-03-30T00:43:59.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9c0cda90 0x7fda9c0acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda9c0cde70 0x7fda9c162620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9c15a370 0x7fda9c15bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c163000 con 0x7fda9c0cde70 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c1632e0 con 0x7fda9c15a370 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fdaa2600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c1635c0 con 0x7fda9c0cda90 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fda93e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda9c0cde70 0x7fda9c162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fda93e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda9c0cde70 0x7fda9c162620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33600/0 (socket says 172.21.15.66:33600) 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fda93e00640 1 -- 172.21.15.66:0/3392888452 learned_addr learned my addr 172.21.15.66:0/3392888452 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fda9b400640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9c0cda90 0x7fda9c0acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.827+0000 7fda9be00640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9c15a370 0x7fda9c15bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93e00640 1 -- 172.21.15.66:0/3392888452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9c0cda90 msgr2=0x7fda9c0acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93e00640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9c0cda90 0x7fda9c0acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93e00640 1 -- 172.21.15.66:0/3392888452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9c15a370 msgr2=0x7fda9c15bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:43:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93e00640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9c15a370 0x7fda9c15bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:43:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93e00640 1 -- 172.21.15.66:0/3392888452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda94009a70 con 0x7fda9c0cde70 2024-03-30T00:43:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93e00640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda9c0cde70 0x7fda9c162620 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fda8400c450 tx=0x7fda8400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93400640 1 -- 172.21.15.66:0/3392888452 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84005cf0 con 0x7fda9c0cde70 2024-03-30T00:43:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93400640 1 -- 172.21.15.66:0/3392888452 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda84005e50 con 0x7fda9c0cde70 2024-03-30T00:43:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fdaa2600640 1 -- 172.21.15.66:0/3392888452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c1638a0 con 0x7fda9c0cde70 2024-03-30T00:43:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93400640 1 -- 172.21.15.66:0/3392888452 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84007290 con 0x7fda9c0cde70 2024-03-30T00:43:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fdaa2600640 1 -- 172.21.15.66:0/3392888452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c15b830 con 0x7fda9c0cde70 2024-03-30T00:43:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fdaa2600640 1 -- 172.21.15.66:0/3392888452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9c066c30 con 0x7fda9c0cde70 2024-03-30T00:43:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93400640 1 -- 172.21.15.66:0/3392888452 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fda840073f0 con 0x7fda9c0cde70 2024-03-30T00:43:59.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93400640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066690 0x7fda78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:43:59.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda93400640 1 -- 172.21.15.66:0/3392888452 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fda840837d0 con 0x7fda9c0cde70 2024-03-30T00:43:59.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.831+0000 7fda9b400640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066690 0x7fda78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:43:59.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.835+0000 7fda9b400640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066690 0x7fda78068b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fda8c007a60 tx=0x7fda8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:43:59.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.835+0000 7fda93400640 1 -- 172.21.15.66:0/3392888452 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fda84050ab0 con 0x7fda9c0cde70 2024-03-30T00:44:00.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.995+0000 7fdaa2600640 1 -- 172.21.15.66:0/3392888452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda9c163d50 con 0x7fda9c0cde70 2024-03-30T00:44:00.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.995+0000 7fda93400640 1 -- 172.21.15.66:0/3392888452 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda84017020 con 0x7fda9c0cde70 2024-03-30T00:44:00.004 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:44:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 -- 172.21.15.66:0/3392888452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066690 msgr2=0x7fda78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066690 0x7fda78068b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fda8c007a60 tx=0x7fda8c008040 comp rx=0 tx=0).stop 2024-03-30T00:44:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 -- 172.21.15.66:0/3392888452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda9c0cde70 msgr2=0x7fda9c162620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda9c0cde70 0x7fda9c162620 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fda8400c450 tx=0x7fda8400c910 comp rx=0 tx=0).stop 2024-03-30T00:44:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 -- 172.21.15.66:0/3392888452 shutdown_connections 2024-03-30T00:44:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9c15a370 0x7fda9c15bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fda78066690 0x7fda78068b50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda9c0cde70 0x7fda9c162620 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 --2- 172.21.15.66:0/3392888452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9c0cda90 0x7fda9c0acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 -- 172.21.15.66:0/3392888452 >> 172.21.15.66:0/3392888452 conn(0x7fda9c069f60 msgr2=0x7fda9c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 -- 172.21.15.66:0/3392888452 shutdown_connections 2024-03-30T00:44:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:43:59.999+0000 7fdaa2600640 1 -- 172.21.15.66:0/3392888452 wait complete. 2024-03-30T00:44:00.025 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:44:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c09e00640 1 Processor -- start 2024-03-30T00:44:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c09e00640 1 -- start start 2024-03-30T00:44:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c09e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4c0415c750 0x7f4c040b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c09e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4c0415cb30 0x7f4c0406a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c09e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c040b1800 con 0x7f4c040ce5d0 2024-03-30T00:44:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c09e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c040afcf0 con 0x7f4c0415c750 2024-03-30T00:44:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c09e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c040b06a0 con 0x7f4c0415cb30 2024-03-30T00:44:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c08a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4c0415c750 0x7f4c040b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c09400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f4c040ce5d0 0x7f4c040cea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46206/0 (socket says 172.21.15.66:46206) 2024-03-30T00:44:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c08a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4c0415c750 0x7f4c040b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33616/0 (socket says 172.21.15.66:33616) 2024-03-30T00:44:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c09400640 1 -- 172.21.15.66:0/3271136646 learned_addr learned my addr 172.21.15.66:0/3271136646 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4bfbe00640 1 --2- 172.21.15.66:0/3271136646 >> v2:172.21.15.156:3300/0 conn(0x7f4c0415cb30 0x7f4c0406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c08a00640 1 -- 172.21.15.66:0/3271136646 >> v2:172.21.15.156:3300/0 conn(0x7f4c0415cb30 msgr2=0x7f4c0406a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c08a00640 1 --2- 172.21.15.66:0/3271136646 >> v2:172.21.15.156:3300/0 conn(0x7f4c0415cb30 0x7f4c0406a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c08a00640 1 -- 172.21.15.66:0/3271136646 >> v1:172.21.15.66:6789/0 conn(0x7f4c040ce5d0 legacy=0x7f4c040cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c08a00640 1 -- 172.21.15.66:0/3271136646 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c040b0980 con 0x7f4c0415c750 2024-03-30T00:44:00.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.159+0000 7f4c03e00640 1 -- 172.21.15.66:0/3271136646 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51484519 0 0) 0x7f4c040b1800 con 0x7f4c040ce5d0 2024-03-30T00:44:00.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c08a00640 1 --2- 172.21.15.66:0/3271136646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c0415c750 0x7f4c040b00c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4c04070550 tx=0x7f4bf401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ccbd1889f3df25 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:00.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c03e00640 1 -- 172.21.15.66:0/3271136646 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40ab600 con 0x7f4c0415c750 2024-03-30T00:44:00.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c03e00640 1 -- 172.21.15.66:0/3271136646 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf40a7050 con 0x7f4c0415c750 2024-03-30T00:44:00.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c03e00640 1 -- 172.21.15.66:0/3271136646 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40ab980 con 0x7f4c0415c750 2024-03-30T00:44:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c09e00640 1 -- 172.21.15.66:0/3271136646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c0415c750 msgr2=0x7f4c040b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c09e00640 1 --2- 172.21.15.66:0/3271136646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c0415c750 0x7f4c040b00c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4c04070550 tx=0x7f4bf401c3f0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c09e00640 1 -- 172.21.15.66:0/3271136646 shutdown_connections 2024-03-30T00:44:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c09e00640 1 --2- 172.21.15.66:0/3271136646 >> v2:172.21.15.156:3300/0 conn(0x7f4c0415cb30 0x7f4c0406a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c09e00640 1 --2- 172.21.15.66:0/3271136646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c0415c750 0x7f4c040b00c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c09e00640 1 -- 172.21.15.66:0/3271136646 >> 172.21.15.66:0/3271136646 conn(0x7f4c04098a20 msgr2=0x7f4c040b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c09e00640 1 -- 172.21.15.66:0/3271136646 shutdown_connections 2024-03-30T00:44:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.163+0000 7f4c09e00640 1 -- 172.21.15.66:0/3271136646 wait complete. 2024-03-30T00:44:00.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09e00640 1 Processor -- start 2024-03-30T00:44:00.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09e00640 1 -- start start 2024-03-30T00:44:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c040ce5d0 0x7f4c04156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c0415c750 0x7f4c041f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c0415cb30 0x7f4c040cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c041580e0 con 0x7f4c040ce5d0 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040c30f0 con 0x7f4c0415cb30 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c041f9740 con 0x7f4c0415c750 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c08a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c040ce5d0 0x7f4c04156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c08a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c040ce5d0 0x7f4c04156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33630/0 (socket says 172.21.15.66:33630) 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c08a00640 1 -- 172.21.15.66:0/596026546 learned_addr learned my addr 172.21.15.66:0/596026546 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09400640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c0415cb30 0x7f4c040cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4bfbe00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c0415c750 0x7f4c041f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c08a00640 1 -- 172.21.15.66:0/596026546 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c0415c750 msgr2=0x7f4c041f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c08a00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c0415c750 0x7f4c041f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c08a00640 1 -- 172.21.15.66:0/596026546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c0415cb30 msgr2=0x7f4c040cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c08a00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c0415cb30 0x7f4c040cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c08a00640 1 -- 172.21.15.66:0/596026546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4009420 con 0x7f4c040ce5d0 2024-03-30T00:44:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4bfbe00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c0415c750 0x7f4c041f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c08a00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c040ce5d0 0x7f4c04156e80 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40ae040 tx=0x7f4bf401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c01600640 1 -- 172.21.15.66:0/596026546 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40abe90 con 0x7f4c040ce5d0 2024-03-30T00:44:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c01600640 1 -- 172.21.15.66:0/596026546 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf40b7450 con 0x7f4c040ce5d0 2024-03-30T00:44:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c01600640 1 -- 172.21.15.66:0/596026546 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40b7820 con 0x7f4c040ce5d0 2024-03-30T00:44:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09e00640 1 -- 172.21.15.66:0/596026546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c041574d0 con 0x7f4c040ce5d0 2024-03-30T00:44:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09e00640 1 -- 172.21.15.66:0/596026546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c04157a90 con 0x7f4c040ce5d0 2024-03-30T00:44:00.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.167+0000 7f4c09e00640 1 -- 172.21.15.66:0/596026546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c04073e00 con 0x7f4c040ce5d0 2024-03-30T00:44:00.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.171+0000 7f4c01600640 1 -- 172.21.15.66:0/596026546 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4bf40b7980 con 0x7f4c040ce5d0 2024-03-30T00:44:00.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.171+0000 7f4c01600640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4be0066690 0x7f4be0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:00.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.171+0000 7f4c01600640 1 -- 172.21.15.66:0/596026546 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f4bf411ec60 con 0x7f4c040ce5d0 2024-03-30T00:44:00.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.171+0000 7f4bfbe00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4be0066690 0x7f4be0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:00.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.171+0000 7f4bfbe00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4be0066690 0x7f4be0068b50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4c04167780 tx=0x7f4bec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:00.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.175+0000 7f4c01600640 1 -- 172.21.15.66:0/596026546 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4bf40ebd60 con 0x7f4c040ce5d0 2024-03-30T00:44:00.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c09e00640 1 -- 172.21.15.66:0/596026546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4c04150050 con 0x7f4c040ce5d0 2024-03-30T00:44:00.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c01600640 1 -- 172.21.15.66:0/596026546 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4bf40bb080 con 0x7f4c040ce5d0 2024-03-30T00:44:00.355 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c09e00640 1 -- 172.21.15.66:0/596026546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4be0066690 msgr2=0x7f4be0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c09e00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4be0066690 0x7f4be0068b50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4c04167780 tx=0x7f4bec008040 comp rx=0 tx=0).stop 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c09e00640 1 -- 172.21.15.66:0/596026546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c040ce5d0 msgr2=0x7f4c04156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c09e00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c040ce5d0 0x7f4c04156e80 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40ae040 tx=0x7f4bf401ac30 comp rx=0 tx=0).stop 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c09e00640 1 -- 172.21.15.66:0/596026546 shutdown_connections 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c09e00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c0415cb30 0x7f4c040cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c09e00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c0415c750 0x7f4c041f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c09e00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4be0066690 0x7f4be0068b50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c09e00640 1 --2- 172.21.15.66:0/596026546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c040ce5d0 0x7f4c04156e80 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.347+0000 7f4c09e00640 1 -- 172.21.15.66:0/596026546 >> 172.21.15.66:0/596026546 conn(0x7f4c04098a20 msgr2=0x7f4c040ac4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.351+0000 7f4c09e00640 1 -- 172.21.15.66:0/596026546 shutdown_connections 2024-03-30T00:44:00.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.351+0000 7f4c09e00640 1 -- 172.21.15.66:0/596026546 wait complete. 2024-03-30T00:44:00.375 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:44:00.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:00.375 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:00.393 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[42625]: fuse finished with error 0 and tester_r 0 2024-03-30T00:44:00.396 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:44:00.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:00.396 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:44:00.561 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:44:00.562 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:44:00.573 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:44:00.574 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:44:00.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:00.574 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T00:44:00.622 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-30T00:44:00.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:44:00.623 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:44:00.623 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:44:00.632 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:44:00.632 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:44:00.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:00.632 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-30T00:44:00.684 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-30T00:44:00.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:44:00.685 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:44:00.685 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:44:00.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:44:00.693 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:44:00.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:00.694 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-30T00:44:00.742 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-30T00:44:00.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:44:00.743 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:44:00.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b52600640 1 Processor -- start 2024-03-30T00:44:00.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b52600640 1 -- start start 2024-03-30T00:44:00.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b52600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c0cda90 0x7f7b4c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:00.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b52600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c0cd3e0 con 0x7f7b4c15a370 2024-03-30T00:44:00.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b52600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c15a990 con 0x7f7b4c0cde70 2024-03-30T00:44:00.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b52600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c0b1b80 con 0x7f7b4c0cda90 2024-03-30T00:44:00.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b4be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7b4c15a370 0x7f7b4c156910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46210/0 (socket says 172.21.15.66:46210) 2024-03-30T00:44:00.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b4be00640 1 -- 172.21.15.66:0/3859621694 learned_addr learned my addr 172.21.15.66:0/3859621694 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:00.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b4b400640 1 --2- 172.21.15.66:0/3859621694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c0cda90 0x7f7b4c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:00.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b4aa00640 1 -- 172.21.15.66:0/3859621694 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520023143 0 0) 0x7f7b4c0cd3e0 con 0x7f7b4c15a370 2024-03-30T00:44:00.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b4aa00640 1 -- 172.21.15.66:0/3859621694 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b30003530 con 0x7f7b4c15a370 2024-03-30T00:44:00.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b4aa00640 1 -- 172.21.15.66:0/3859621694 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893566993 0 0) 0x7f7b4c15a990 con 0x7f7b4c0cde70 2024-03-30T00:44:00.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b4b400640 1 -- 172.21.15.66:0/3859621694 >> v1:172.21.15.156:6789/0 conn(0x7f7b4c0cde70 legacy=0x7f7b4c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b4b400640 1 -- 172.21.15.66:0/3859621694 >> v1:172.21.15.66:6789/0 conn(0x7f7b4c15a370 legacy=0x7f7b4c156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.875+0000 7f7b4b400640 1 -- 172.21.15.66:0/3859621694 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c0b11e0 con 0x7f7b4c0cda90 2024-03-30T00:44:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.879+0000 7f7b4b400640 1 --2- 172.21.15.66:0/3859621694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c0cda90 0x7f7b4c0ce520 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0097f0 tx=0x7f7b3c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb1618b5630ad20f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.879+0000 7f7b4aa00640 1 -- 172.21.15.66:0/3859621694 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0ad660 con 0x7f7b4c0cda90 2024-03-30T00:44:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.879+0000 7f7b4aa00640 1 -- 172.21.15.66:0/3859621694 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c0ad7c0 con 0x7f7b4c0cda90 2024-03-30T00:44:00.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.879+0000 7f7b4aa00640 1 -- 172.21.15.66:0/3859621694 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0adb30 con 0x7f7b4c0cda90 2024-03-30T00:44:00.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.879+0000 7f7b52600640 1 -- 172.21.15.66:0/3859621694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c0cda90 msgr2=0x7f7b4c0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.879+0000 7f7b52600640 1 --2- 172.21.15.66:0/3859621694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c0cda90 0x7f7b4c0ce520 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0097f0 tx=0x7f7b3c01c450 comp rx=0 tx=0).stop 2024-03-30T00:44:00.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.879+0000 7f7b52600640 1 -- 172.21.15.66:0/3859621694 shutdown_connections 2024-03-30T00:44:00.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.879+0000 7f7b52600640 1 --2- 172.21.15.66:0/3859621694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c0cda90 0x7f7b4c0ce520 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.879+0000 7f7b52600640 1 -- 172.21.15.66:0/3859621694 >> 172.21.15.66:0/3859621694 conn(0x7f7b4c069f60 msgr2=0x7f7b4c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:00.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.879+0000 7f7b52600640 1 -- 172.21.15.66:0/3859621694 shutdown_connections 2024-03-30T00:44:00.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.879+0000 7f7b52600640 1 -- 172.21.15.66:0/3859621694 wait complete. 2024-03-30T00:44:00.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b52600640 1 Processor -- start 2024-03-30T00:44:00.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b52600640 1 -- start start 2024-03-30T00:44:00.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b52600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c0cda90 0x7f7b4c06fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:00.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b52600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c0cde70 0x7f7b4c06d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:00.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b52600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c15a370 0x7f7b4c0aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:00.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b52600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c0acf70 con 0x7f7b4c15a370 2024-03-30T00:44:00.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b52600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c0ad1d0 con 0x7f7b4c0cda90 2024-03-30T00:44:00.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b52600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c162750 con 0x7f7b4c0cde70 2024-03-30T00:44:00.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c15a370 0x7f7b4c0aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:00.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c15a370 0x7f7b4c0aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33632/0 (socket says 172.21.15.66:33632) 2024-03-30T00:44:00.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b4be00640 1 -- 172.21.15.66:0/765734016 learned_addr learned my addr 172.21.15.66:0/765734016 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:00.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b4b400640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c0cda90 0x7f7b4c06fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b42a00640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c0cde70 0x7f7b4c06d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b4be00640 1 -- 172.21.15.66:0/765734016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c0cde70 msgr2=0x7f7b4c06d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b4be00640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c0cde70 0x7f7b4c06d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b4be00640 1 -- 172.21.15.66:0/765734016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c0cda90 msgr2=0x7f7b4c06fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b4be00640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c0cda90 0x7f7b4c06fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b4be00640 1 -- 172.21.15.66:0/765734016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c009420 con 0x7f7b4c15a370 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b4be00640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c15a370 0x7f7b4c0aca30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400caa0 tx=0x7f7b4400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b43e00640 1 -- 172.21.15.66:0/765734016 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400ac00 con 0x7f7b4c15a370 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b43e00640 1 -- 172.21.15.66:0/765734016 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4400ad60 con 0x7f7b4c15a370 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b52600640 1 -- 172.21.15.66:0/765734016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4c162a30 con 0x7f7b4c15a370 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b43e00640 1 -- 172.21.15.66:0/765734016 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44012620 con 0x7f7b4c15a370 2024-03-30T00:44:00.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.883+0000 7f7b52600640 1 -- 172.21.15.66:0/765734016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c162ff0 con 0x7f7b4c15a370 2024-03-30T00:44:00.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.887+0000 7f7b52600640 1 -- 172.21.15.66:0/765734016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4c1631e0 con 0x7f7b4c15a370 2024-03-30T00:44:00.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.887+0000 7f7b43e00640 1 -- 172.21.15.66:0/765734016 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7b44007070 con 0x7f7b4c15a370 2024-03-30T00:44:00.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.887+0000 7f7b43e00640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 0x7f7b28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:00.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.887+0000 7f7b43e00640 1 -- 172.21.15.66:0/765734016 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f7b440842a0 con 0x7f7b4c15a370 2024-03-30T00:44:00.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.887+0000 7f7b4b400640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 0x7f7b28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:00.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.887+0000 7f7b4b400640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 0x7f7b28068b50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0af000 tx=0x7f7b3c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:00.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:00.891+0000 7f7b43e00640 1 -- 172.21.15.66:0/765734016 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7b440514e0 con 0x7f7b4c15a370 2024-03-30T00:44:01.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 -- 172.21.15.66:0/765734016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b4c066c30 con 0x7f7b4c15a370 2024-03-30T00:44:01.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b43e00640 1 -- 172.21.15.66:0/765734016 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f7b44054b00 con 0x7f7b4c15a370 2024-03-30T00:44:01.094 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:01.095 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4272961638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4272961638},{"type":"v1","addr":"172.21.15.156:6834","nonce":4272961638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2144186622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2144186622},{"type":"v1","addr":"172.21.15.66:6835","nonce":2144186622}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/750308788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":750308788},{"type":"v1","addr":"172.21.15.66:6837","nonce":750308788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:43:11.141056+0000","modified":"2024-03-30T00:43:55.622856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6468":{"gid":6468,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":14,"addr":"172.21.15.156:6835/1137593870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1137593870},{"type":"v1","addr":"172.21.15.156:6835","nonce":1137593870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-30T00:44:01.095 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 35 2024-03-30T00:44:01.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 -- 172.21.15.66:0/765734016 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 msgr2=0x7f7b28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 0x7f7b28068b50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0af000 tx=0x7f7b3c0a7000 comp rx=0 tx=0).stop 2024-03-30T00:44:01.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 -- 172.21.15.66:0/765734016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c15a370 msgr2=0x7f7b4c0aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c15a370 0x7f7b4c0aca30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400caa0 tx=0x7f7b4400cf60 comp rx=0 tx=0).stop 2024-03-30T00:44:01.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 -- 172.21.15.66:0/765734016 shutdown_connections 2024-03-30T00:44:01.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b28066690 0x7f7b28068b50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c15a370 0x7f7b4c0aca30 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b4c0cde70 0x7f7b4c06d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 --2- 172.21.15.66:0/765734016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c0cda90 0x7f7b4c06fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 -- 172.21.15.66:0/765734016 >> 172.21.15.66:0/765734016 conn(0x7f7b4c069f60 msgr2=0x7f7b4c0cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:01.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 -- 172.21.15.66:0/765734016 shutdown_connections 2024-03-30T00:44:01.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.087+0000 7f7b52600640 1 -- 172.21.15.66:0/765734016 wait complete. 2024-03-30T00:44:01.115 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:01.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.247+0000 7f47d1400640 1 Processor -- start 2024-03-30T00:44:01.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.247+0000 7f47d1400640 1 -- start start 2024-03-30T00:44:01.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.247+0000 7f47d1400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47cc0aca40 0x7f47cc0ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.247+0000 7f47d1400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f47cc0cea00 0x7f47cc06cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.247+0000 7f47d1400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47cc0b1580 con 0x7f47cc06cff0 2024-03-30T00:44:01.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.247+0000 7f47d1400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47cc0c8fa0 con 0x7f47cc0cea00 2024-03-30T00:44:01.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.247+0000 7f47d1400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc0cee40 con 0x7f47cc0aca40 2024-03-30T00:44:01.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47c1e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f47cc0cea00 0x7f47cc06cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47ca800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47cc0aca40 0x7f47cc0ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47cb200640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f47cc06cff0 0x7f47cc15acb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:55670/0 (socket says 172.21.15.66:55670) 2024-03-30T00:44:01.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47cb200640 1 -- 172.21.15.66:0/1672226895 learned_addr learned my addr 172.21.15.66:0/1672226895 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:01.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47c1e00640 1 -- 172.21.15.66:0/1672226895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47cc0aca40 msgr2=0x7f47cc0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:01.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47c1e00640 1 --2- 172.21.15.66:0/1672226895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47cc0aca40 0x7f47cc0ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47c1e00640 1 -- 172.21.15.66:0/1672226895 >> v1:172.21.15.156:6789/0 conn(0x7f47cc06cff0 legacy=0x7f47cc15acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47c9e00640 1 -- 172.21.15.66:0/1672226895 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629027511 0 0) 0x7f47cc0b1580 con 0x7f47cc06cff0 2024-03-30T00:44:01.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47c1e00640 1 -- 172.21.15.66:0/1672226895 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc06ddc0 con 0x7f47cc0cea00 2024-03-30T00:44:01.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47c1e00640 1 --2- 172.21.15.66:0/1672226895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc0cea00 0x7f47cc06cab0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009d90 tx=0x7f47b401c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c64988a972339961 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:01.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47c9e00640 1 -- 172.21.15.66:0/1672226895 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b40a5970 con 0x7f47cc0cea00 2024-03-30T00:44:01.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47c9e00640 1 -- 172.21.15.66:0/1672226895 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b40a5ad0 con 0x7f47cc0cea00 2024-03-30T00:44:01.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.251+0000 7f47c9e00640 1 -- 172.21.15.66:0/1672226895 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b40a5e60 con 0x7f47cc0cea00 2024-03-30T00:44:01.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 -- 172.21.15.66:0/1672226895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc0cea00 msgr2=0x7f47cc06cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 --2- 172.21.15.66:0/1672226895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc0cea00 0x7f47cc06cab0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009d90 tx=0x7f47b401c8b0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 -- 172.21.15.66:0/1672226895 shutdown_connections 2024-03-30T00:44:01.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 --2- 172.21.15.66:0/1672226895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc0cea00 0x7f47cc06cab0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 --2- 172.21.15.66:0/1672226895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47cc0aca40 0x7f47cc0ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 -- 172.21.15.66:0/1672226895 >> 172.21.15.66:0/1672226895 conn(0x7f47cc069f60 msgr2=0x7f47cc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:01.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 -- 172.21.15.66:0/1672226895 shutdown_connections 2024-03-30T00:44:01.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 -- 172.21.15.66:0/1672226895 wait complete. 2024-03-30T00:44:01.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 Processor -- start 2024-03-30T00:44:01.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 -- start start 2024-03-30T00:44:01.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47cc06cff0 0x7f47cc156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47cc0aca40 0x7f47cc0c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc0cea00 0x7f47cc1540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc0c58a0 con 0x7f47cc06cff0 2024-03-30T00:44:01.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc0adfc0 con 0x7f47cc0cea00 2024-03-30T00:44:01.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47d1400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc0c6370 con 0x7f47cc0aca40 2024-03-30T00:44:01.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47ca800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47cc06cff0 0x7f47cc156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47ca800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47cc06cff0 0x7f47cc156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33642/0 (socket says 172.21.15.66:33642) 2024-03-30T00:44:01.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47ca800640 1 -- 172.21.15.66:0/1005057684 learned_addr learned my addr 172.21.15.66:0/1005057684 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:01.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47c1e00640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47cc0aca40 0x7f47cc0c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47cb200640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc0cea00 0x7f47cc1540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47ca800640 1 -- 172.21.15.66:0/1005057684 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47cc0aca40 msgr2=0x7f47cc0c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47ca800640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47cc0aca40 0x7f47cc0c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47ca800640 1 -- 172.21.15.66:0/1005057684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc0cea00 msgr2=0x7f47cc1540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47ca800640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc0cea00 0x7f47cc1540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47ca800640 1 -- 172.21.15.66:0/1005057684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b4009a70 con 0x7f47cc06cff0 2024-03-30T00:44:01.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.255+0000 7f47cb200640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc0cea00 0x7f47cc1540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:01.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.259+0000 7f47ca800640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47cc06cff0 0x7f47cc156660 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00e930 tx=0x7f47bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:01.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.259+0000 7f47c3400640 1 -- 172.21.15.66:0/1005057684 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc009690 con 0x7f47cc06cff0 2024-03-30T00:44:01.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.259+0000 7f47c3400640 1 -- 172.21.15.66:0/1005057684 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc00ca00 con 0x7f47cc06cff0 2024-03-30T00:44:01.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.259+0000 7f47c3400640 1 -- 172.21.15.66:0/1005057684 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00cd70 con 0x7f47cc06cff0 2024-03-30T00:44:01.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.259+0000 7f47d1400640 1 -- 172.21.15.66:0/1005057684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47cc0c5ad0 con 0x7f47cc06cff0 2024-03-30T00:44:01.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.259+0000 7f47d1400640 1 -- 172.21.15.66:0/1005057684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47cc0c6090 con 0x7f47cc06cff0 2024-03-30T00:44:01.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.259+0000 7f47d1400640 1 -- 172.21.15.66:0/1005057684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47cc066c30 con 0x7f47cc06cff0 2024-03-30T00:44:01.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.259+0000 7f47c3400640 1 -- 172.21.15.66:0/1005057684 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f47bc013470 con 0x7f47cc06cff0 2024-03-30T00:44:01.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.259+0000 7f47c3400640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47a8066690 0x7f47a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.259+0000 7f47c3400640 1 -- 172.21.15.66:0/1005057684 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f47bc084850 con 0x7f47cc06cff0 2024-03-30T00:44:01.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.259+0000 7f47c1e00640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47a8066690 0x7f47a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.263+0000 7f47c1e00640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47a8066690 0x7f47a8068b50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f47b40af000 tx=0x7f47b40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:01.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.263+0000 7f47c3400640 1 -- 172.21.15.66:0/1005057684 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f47bc051bf0 con 0x7f47cc06cff0 2024-03-30T00:44:01.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.423+0000 7f47d1400640 1 -- 172.21.15.66:0/1005057684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47cc06fe50 con 0x7f47cc06cff0 2024-03-30T00:44:01.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47c3400640 1 -- 172.21.15.66:0/1005057684 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16487 (secure 0 0 0) 0x7f47bc055210 con 0x7f47cc06cff0 2024-03-30T00:44:01.434 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:01.434 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":70,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:43:34.562474+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:43:09.681968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T00:43:10.133411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.38","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:01.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 -- 172.21.15.66:0/1005057684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47a8066690 msgr2=0x7f47a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47a8066690 0x7f47a8068b50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f47b40af000 tx=0x7f47b40a7070 comp rx=0 tx=0).stop 2024-03-30T00:44:01.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 -- 172.21.15.66:0/1005057684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47cc06cff0 msgr2=0x7f47cc156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47cc06cff0 0x7f47cc156660 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00e930 tx=0x7f47bc00edf0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 -- 172.21.15.66:0/1005057684 shutdown_connections 2024-03-30T00:44:01.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47cc0cea00 0x7f47cc1540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47cc0aca40 0x7f47cc0c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47a8066690 0x7f47a8068b50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 --2- 172.21.15.66:0/1005057684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47cc06cff0 0x7f47cc156660 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 -- 172.21.15.66:0/1005057684 >> 172.21.15.66:0/1005057684 conn(0x7f47cc069f60 msgr2=0x7f47cc15e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:01.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 -- 172.21.15.66:0/1005057684 shutdown_connections 2024-03-30T00:44:01.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.427+0000 7f47d1400640 1 -- 172.21.15.66:0/1005057684 wait complete. 2024-03-30T00:44:01.456 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:43:09.681968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:43:10.133411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:44:01.456 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:44:01.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.591+0000 7f5589400640 1 Processor -- start 2024-03-30T00:44:01.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.591+0000 7f5589400640 1 -- start start 2024-03-30T00:44:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.591+0000 7f5589400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f55840ce520 0x7f55840ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.591+0000 7f5589400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f558406d070 con 0x7f55840cabb0 2024-03-30T00:44:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.591+0000 7f5589400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55840cbba0 con 0x7f558415a640 2024-03-30T00:44:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.591+0000 7f5589400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55840cee70 con 0x7f55840ce520 2024-03-30T00:44:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.591+0000 7f5579e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f55840ce520 0x7f55840ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.591+0000 7f5583200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f55840cabb0 0x7f55840cafe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46226/0 (socket says 172.21.15.66:46226) 2024-03-30T00:44:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.591+0000 7f5583200640 1 -- 172.21.15.66:0/3567554317 learned_addr learned my addr 172.21.15.66:0/3567554317 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5579e00640 1 -- 172.21.15.66:0/3567554317 >> v1:172.21.15.156:6789/0 conn(0x7f558415a640 legacy=0x7f558406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5579e00640 1 -- 172.21.15.66:0/3567554317 >> v1:172.21.15.66:6789/0 conn(0x7f55840cabb0 legacy=0x7f55840cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5579e00640 1 -- 172.21.15.66:0/3567554317 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55840cb8c0 con 0x7f55840ce520 2024-03-30T00:44:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5581e00640 1 -- 172.21.15.66:0/3567554317 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079280220 0 0) 0x7f558406d070 con 0x7f55840cabb0 2024-03-30T00:44:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5579e00640 1 --2- 172.21.15.66:0/3567554317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55840ce520 0x7f55840ce930 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f556c0097c0 tx=0x7f556c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9053ef872bd7b19c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5581e00640 1 -- 172.21.15.66:0/3567554317 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c0a8050 con 0x7f55840ce520 2024-03-30T00:44:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5581e00640 1 -- 172.21.15.66:0/3567554317 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c0ac5f0 con 0x7f55840ce520 2024-03-30T00:44:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5581e00640 1 -- 172.21.15.66:0/3567554317 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c0ac980 con 0x7f55840ce520 2024-03-30T00:44:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5589400640 1 -- 172.21.15.66:0/3567554317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55840ce520 msgr2=0x7f55840ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5589400640 1 --2- 172.21.15.66:0/3567554317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55840ce520 0x7f55840ce930 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f556c0097c0 tx=0x7f556c01c4a0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5589400640 1 -- 172.21.15.66:0/3567554317 shutdown_connections 2024-03-30T00:44:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5589400640 1 --2- 172.21.15.66:0/3567554317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55840ce520 0x7f55840ce930 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.595+0000 7f5589400640 1 -- 172.21.15.66:0/3567554317 >> 172.21.15.66:0/3567554317 conn(0x7f5584069f60 msgr2=0x7f558406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5589400640 1 -- 172.21.15.66:0/3567554317 shutdown_connections 2024-03-30T00:44:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5589400640 1 -- 172.21.15.66:0/3567554317 wait complete. 2024-03-30T00:44:01.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5589400640 1 Processor -- start 2024-03-30T00:44:01.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5589400640 1 -- start start 2024-03-30T00:44:01.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5589400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840cabb0 0x7f5584096ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5589400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55840ce520 0x7f558415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5589400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558415a640 0x7f558415c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5589400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558415dc60 con 0x7f55840ce520 2024-03-30T00:44:01.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5589400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55840c0ec0 con 0x7f55840cabb0 2024-03-30T00:44:01.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5589400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55840c0be0 con 0x7f558415a640 2024-03-30T00:44:01.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5579e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55840ce520 0x7f558415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5579e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55840ce520 0x7f558415bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33654/0 (socket says 172.21.15.66:33654) 2024-03-30T00:44:01.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5579e00640 1 -- 172.21.15.66:0/2990889311 learned_addr learned my addr 172.21.15.66:0/2990889311 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:01.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5583200640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558415a640 0x7f558415c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5579e00640 1 -- 172.21.15.66:0/2990889311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558415a640 msgr2=0x7f558415c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5579e00640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558415a640 0x7f558415c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5579e00640 1 -- 172.21.15.66:0/2990889311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840cabb0 msgr2=0x7f5584096ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5582800640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840cabb0 0x7f5584096ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5579e00640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840cabb0 0x7f5584096ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5579e00640 1 -- 172.21.15.66:0/2990889311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c009420 con 0x7f55840ce520 2024-03-30T00:44:01.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5582800640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840cabb0 0x7f5584096ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:01.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f5579e00640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55840ce520 0x7f558415bbb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f556c0af040 tx=0x7f556c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.599+0000 7f557b400640 1 -- 172.21.15.66:0/2990889311 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c0b5620 con 0x7f55840ce520 2024-03-30T00:44:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.603+0000 7f557b400640 1 -- 172.21.15.66:0/2990889311 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c0b5780 con 0x7f55840ce520 2024-03-30T00:44:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.603+0000 7f557b400640 1 -- 172.21.15.66:0/2990889311 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c0b5b30 con 0x7f55840ce520 2024-03-30T00:44:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.603+0000 7f5589400640 1 -- 172.21.15.66:0/2990889311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558406ff70 con 0x7f55840ce520 2024-03-30T00:44:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.603+0000 7f5589400640 1 -- 172.21.15.66:0/2990889311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55840973e0 con 0x7f55840ce520 2024-03-30T00:44:01.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.603+0000 7f5589400640 1 -- 172.21.15.66:0/2990889311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5584066c30 con 0x7f55840ce520 2024-03-30T00:44:01.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.603+0000 7f557b400640 1 -- 172.21.15.66:0/2990889311 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f556c0b3040 con 0x7f55840ce520 2024-03-30T00:44:01.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.603+0000 7f557b400640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55600666c0 0x7f5560068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.603+0000 7f557b400640 1 -- 172.21.15.66:0/2990889311 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f556c11e130 con 0x7f55840ce520 2024-03-30T00:44:01.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.603+0000 7f5582800640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55600666c0 0x7f5560068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.603+0000 7f5582800640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55600666c0 0x7f5560068b80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f55840b2390 tx=0x7f5574009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:01.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.607+0000 7f557b400640 1 -- 172.21.15.66:0/2990889311 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f556c0eb230 con 0x7f55840ce520 2024-03-30T00:44:01.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 -- 172.21.15.66:0/2990889311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5584070420 con 0x7f55840ce520 2024-03-30T00:44:01.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f557b400640 1 -- 172.21.15.66:0/2990889311 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f556c0ee850 con 0x7f55840ce520 2024-03-30T00:44:01.778 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:44:01.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 -- 172.21.15.66:0/2990889311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55600666c0 msgr2=0x7f5560068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55600666c0 0x7f5560068b80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f55840b2390 tx=0x7f5574009040 comp rx=0 tx=0).stop 2024-03-30T00:44:01.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 -- 172.21.15.66:0/2990889311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55840ce520 msgr2=0x7f558415bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55840ce520 0x7f558415bbb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f556c0af040 tx=0x7f556c019240 comp rx=0 tx=0).stop 2024-03-30T00:44:01.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 -- 172.21.15.66:0/2990889311 shutdown_connections 2024-03-30T00:44:01.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f558415a640 0x7f558415c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55600666c0 0x7f5560068b80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55840ce520 0x7f558415bbb0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 --2- 172.21.15.66:0/2990889311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840cabb0 0x7f5584096ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 -- 172.21.15.66:0/2990889311 >> 172.21.15.66:0/2990889311 conn(0x7f5584069f60 msgr2=0x7f55840afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:01.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 -- 172.21.15.66:0/2990889311 shutdown_connections 2024-03-30T00:44:01.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.771+0000 7f5589400640 1 -- 172.21.15.66:0/2990889311 wait complete. 2024-03-30T00:44:01.800 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:44:01.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.935+0000 7fb2c5a00640 1 Processor -- start 2024-03-30T00:44:01.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.935+0000 7fb2c5a00640 1 -- start start 2024-03-30T00:44:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2c5a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2c015c6b0 0x7fb2c00d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2c5a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c015d250 con 0x7fb2c00cdef0 2024-03-30T00:44:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2c5a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c015d530 con 0x7fb2c006cb90 2024-03-30T00:44:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2c5a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c006b1f0 con 0x7fb2c015c6b0 2024-03-30T00:44:01.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2c5000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb2c00cdef0 0x7fb2c01604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46240/0 (socket says 172.21.15.66:46240) 2024-03-30T00:44:01.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2c5000640 1 -- 172.21.15.66:0/3270226310 learned_addr learned my addr 172.21.15.66:0/3270226310 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:01.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2bfe00640 1 --2- 172.21.15.66:0/3270226310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2c015c6b0 0x7fb2c00d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2bf400640 1 -- 172.21.15.66:0/3270226310 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123716894 0 0) 0x7fb2c015d250 con 0x7fb2c00cdef0 2024-03-30T00:44:01.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2bf400640 1 -- 172.21.15.66:0/3270226310 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a4003530 con 0x7fb2c00cdef0 2024-03-30T00:44:01.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2bf400640 1 -- 172.21.15.66:0/3270226310 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165931212 0 0) 0x7fb2c015d530 con 0x7fb2c006cb90 2024-03-30T00:44:01.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2bf400640 1 -- 172.21.15.66:0/3270226310 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c015d250 con 0x7fb2c006cb90 2024-03-30T00:44:01.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2bfe00640 1 -- 172.21.15.66:0/3270226310 >> v1:172.21.15.156:6789/0 conn(0x7fb2c006cb90 legacy=0x7fb2c015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2bfe00640 1 -- 172.21.15.66:0/3270226310 >> v1:172.21.15.66:6789/0 conn(0x7fb2c00cdef0 legacy=0x7fb2c01604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.939+0000 7fb2bfe00640 1 -- 172.21.15.66:0/3270226310 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c015d810 con 0x7fb2c015c6b0 2024-03-30T00:44:01.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2bfe00640 1 --2- 172.21.15.66:0/3270226310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2c015c6b0 0x7fb2c00d0c50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00097f0 tx=0x7fb2b001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e341d93666ddfa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:01.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2bf400640 1 -- 172.21.15.66:0/3270226310 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b00ac620 con 0x7fb2c015c6b0 2024-03-30T00:44:01.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2bf400640 1 -- 172.21.15.66:0/3270226310 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b00ac780 con 0x7fb2c015c6b0 2024-03-30T00:44:01.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2bf400640 1 -- 172.21.15.66:0/3270226310 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b00acaf0 con 0x7fb2c015c6b0 2024-03-30T00:44:01.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2c5a00640 1 -- 172.21.15.66:0/3270226310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2c015c6b0 msgr2=0x7fb2c00d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2c5a00640 1 --2- 172.21.15.66:0/3270226310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2c015c6b0 0x7fb2c00d0c50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00097f0 tx=0x7fb2b001c4b0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2c5a00640 1 -- 172.21.15.66:0/3270226310 shutdown_connections 2024-03-30T00:44:01.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2c5a00640 1 --2- 172.21.15.66:0/3270226310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2c015c6b0 0x7fb2c00d0c50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2c5a00640 1 -- 172.21.15.66:0/3270226310 >> 172.21.15.66:0/3270226310 conn(0x7fb2c0069d90 msgr2=0x7fb2c006a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:01.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2c5a00640 1 -- 172.21.15.66:0/3270226310 shutdown_connections 2024-03-30T00:44:01.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2c5a00640 1 -- 172.21.15.66:0/3270226310 wait complete. 2024-03-30T00:44:01.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2c5a00640 1 Processor -- start 2024-03-30T00:44:01.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.943+0000 7fb2c5a00640 1 -- start start 2024-03-30T00:44:01.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2c5a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2c006cb90 0x7fb2c01f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2c5a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c00cdef0 0x7fb2c00b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2c5a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2c015c6b0 0x7fb2c015a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2c5a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c00b4350 con 0x7fb2c006cb90 2024-03-30T00:44:01.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2c5a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c00b3780 con 0x7fb2c00cdef0 2024-03-30T00:44:01.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2c5a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c00b3a20 con 0x7fb2c015c6b0 2024-03-30T00:44:01.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2c006cb90 0x7fb2c01f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2c006cb90 0x7fb2c01f8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33670/0 (socket says 172.21.15.66:33670) 2024-03-30T00:44:01.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bfe00640 1 -- 172.21.15.66:0/1135029852 learned_addr learned my addr 172.21.15.66:0/1135029852 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:01.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2b7e00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c00cdef0 0x7fb2c00b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2c5000640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2c015c6b0 0x7fb2c015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bfe00640 1 -- 172.21.15.66:0/1135029852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2c015c6b0 msgr2=0x7fb2c015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bfe00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2c015c6b0 0x7fb2c015a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bfe00640 1 -- 172.21.15.66:0/1135029852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c00cdef0 msgr2=0x7fb2c00b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:01.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bfe00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c00cdef0 0x7fb2c00b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:01.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bfe00640 1 -- 172.21.15.66:0/1135029852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0009420 con 0x7fb2c006cb90 2024-03-30T00:44:01.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bfe00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2c006cb90 0x7fb2c01f8d70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00097f0 tx=0x7fb2b0019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:01.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bcc00640 1 -- 172.21.15.66:0/1135029852 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b00ac620 con 0x7fb2c006cb90 2024-03-30T00:44:01.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bcc00640 1 -- 172.21.15.66:0/1135029852 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b00b5400 con 0x7fb2c006cb90 2024-03-30T00:44:01.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bcc00640 1 -- 172.21.15.66:0/1135029852 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b00b5790 con 0x7fb2c006cb90 2024-03-30T00:44:01.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2c5a00640 1 -- 172.21.15.66:0/1135029852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c00b3d00 con 0x7fb2c006cb90 2024-03-30T00:44:01.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2c5a00640 1 -- 172.21.15.66:0/1135029852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c00b5b70 con 0x7fb2c006cb90 2024-03-30T00:44:01.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2c5a00640 1 -- 172.21.15.66:0/1135029852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c0073e00 con 0x7fb2c006cb90 2024-03-30T00:44:01.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.947+0000 7fb2bcc00640 1 -- 172.21.15.66:0/1135029852 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb2b00b3040 con 0x7fb2c006cb90 2024-03-30T00:44:01.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.951+0000 7fb2bcc00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb29c0666c0 0x7fb29c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:01.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.951+0000 7fb2bcc00640 1 -- 172.21.15.66:0/1135029852 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fb2b011dd70 con 0x7fb2c006cb90 2024-03-30T00:44:01.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.951+0000 7fb2b7e00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb29c0666c0 0x7fb29c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:01.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.951+0000 7fb2b7e00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb29c0666c0 0x7fb29c068b80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8007fd0 tx=0x7fb2a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:01.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:01.955+0000 7fb2bcc00640 1 -- 172.21.15.66:0/1135029852 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb2b00eaf80 con 0x7fb2c006cb90 2024-03-30T00:44:02.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.111+0000 7fb2c5a00640 1 -- 172.21.15.66:0/1135029852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2c0073e00 con 0x7fb2c006cb90 2024-03-30T00:44:02.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2bcc00640 1 -- 172.21.15.66:0/1135029852 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2b00ee5a0 con 0x7fb2c006cb90 2024-03-30T00:44:02.121 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:44:02.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 -- 172.21.15.66:0/1135029852 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb29c0666c0 msgr2=0x7fb29c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb29c0666c0 0x7fb29c068b80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8007fd0 tx=0x7fb2a8009040 comp rx=0 tx=0).stop 2024-03-30T00:44:02.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 -- 172.21.15.66:0/1135029852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2c006cb90 msgr2=0x7fb2c01f8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2c006cb90 0x7fb2c01f8d70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00097f0 tx=0x7fb2b0019250 comp rx=0 tx=0).stop 2024-03-30T00:44:02.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 -- 172.21.15.66:0/1135029852 shutdown_connections 2024-03-30T00:44:02.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2c015c6b0 0x7fb2c015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c00cdef0 0x7fb2c00b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb29c0666c0 0x7fb29c068b80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 --2- 172.21.15.66:0/1135029852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2c006cb90 0x7fb2c01f8d70 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 -- 172.21.15.66:0/1135029852 >> 172.21.15.66:0/1135029852 conn(0x7fb2c0069d90 msgr2=0x7fb2c006a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:02.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 -- 172.21.15.66:0/1135029852 shutdown_connections 2024-03-30T00:44:02.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.115+0000 7fb2c5a00640 1 -- 172.21.15.66:0/1135029852 wait complete. 2024-03-30T00:44:02.143 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:44:02.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f4424c00640 1 Processor -- start 2024-03-30T00:44:02.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f4424c00640 1 -- start start 2024-03-30T00:44:02.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f4424c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f44200cd2d0 0x7f44200cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f4424c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44200cc8d0 0x7f442006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f4424c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44200cb620 con 0x7f44200cad00 2024-03-30T00:44:02.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f4424c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44200cbec0 con 0x7f44200cc8d0 2024-03-30T00:44:02.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f4424c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44200ca660 con 0x7f44200cd2d0 2024-03-30T00:44:02.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f441de00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f44200cad00 0x7f44200c8940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46250/0 (socket says 172.21.15.66:46250) 2024-03-30T00:44:02.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f441de00640 1 -- 172.21.15.66:0/2765305964 learned_addr learned my addr 172.21.15.66:0/2765305964 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:02.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f441e800640 1 --2- 172.21.15.66:0/2765305964 >> v2:172.21.15.66:3300/0 conn(0x7f44200cc8d0 0x7f442006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f4417e00640 1 --2- 172.21.15.66:0/2765305964 >> v2:172.21.15.156:3300/0 conn(0x7f44200cd2d0 0x7f44200cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f441d400640 1 -- 172.21.15.66:0/2765305964 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535040843 0 0) 0x7f44200cb620 con 0x7f44200cad00 2024-03-30T00:44:02.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f441d400640 1 -- 172.21.15.66:0/2765305964 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4404003530 con 0x7f44200cad00 2024-03-30T00:44:02.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f441e800640 1 -- 172.21.15.66:0/2765305964 >> v2:172.21.15.156:3300/0 conn(0x7f44200cd2d0 msgr2=0x7f44200cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:02.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f441e800640 1 --2- 172.21.15.66:0/2765305964 >> v2:172.21.15.156:3300/0 conn(0x7f44200cd2d0 0x7f44200cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f441e800640 1 -- 172.21.15.66:0/2765305964 >> v1:172.21.15.66:6789/0 conn(0x7f44200cad00 legacy=0x7f44200c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.279+0000 7f441e800640 1 -- 172.21.15.66:0/2765305964 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200ca940 con 0x7f44200cc8d0 2024-03-30T00:44:02.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f441e800640 1 --2- 172.21.15.66:0/2765305964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44200cc8d0 0x7f442006c030 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f44200cba10 tx=0x7f441801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3cd1b1fce679f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:02.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f441d400640 1 -- 172.21.15.66:0/2765305964 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44180a59c0 con 0x7f44200cc8d0 2024-03-30T00:44:02.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f441d400640 1 -- 172.21.15.66:0/2765305964 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44180a5b20 con 0x7f44200cc8d0 2024-03-30T00:44:02.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f441d400640 1 -- 172.21.15.66:0/2765305964 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44180ad440 con 0x7f44200cc8d0 2024-03-30T00:44:02.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f4424c00640 1 -- 172.21.15.66:0/2765305964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44200cc8d0 msgr2=0x7f442006c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f4424c00640 1 --2- 172.21.15.66:0/2765305964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44200cc8d0 0x7f442006c030 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f44200cba10 tx=0x7f441801c900 comp rx=0 tx=0).stop 2024-03-30T00:44:02.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f4424c00640 1 -- 172.21.15.66:0/2765305964 shutdown_connections 2024-03-30T00:44:02.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f4424c00640 1 --2- 172.21.15.66:0/2765305964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44200cc8d0 0x7f442006c030 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f4424c00640 1 --2- 172.21.15.66:0/2765305964 >> v2:172.21.15.156:3300/0 conn(0x7f44200cd2d0 0x7f44200cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f4424c00640 1 -- 172.21.15.66:0/2765305964 >> 172.21.15.66:0/2765305964 conn(0x7f4420069f60 msgr2=0x7f442006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:02.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f4424c00640 1 -- 172.21.15.66:0/2765305964 shutdown_connections 2024-03-30T00:44:02.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.283+0000 7f4424c00640 1 -- 172.21.15.66:0/2765305964 wait complete. 2024-03-30T00:44:02.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4424c00640 1 Processor -- start 2024-03-30T00:44:02.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4424c00640 1 -- start start 2024-03-30T00:44:02.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4424c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44200cad00 0x7f4420150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4424c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44200cc8d0 0x7f4420153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4424c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44200cd2d0 0x7f4420097540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4424c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4420151010 con 0x7f44200cad00 2024-03-30T00:44:02.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4424c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200718b0 con 0x7f44200cc8d0 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4424c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4420071b90 con 0x7f44200cd2d0 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f441de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44200cad00 0x7f4420150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f441de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44200cad00 0x7f4420150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33692/0 (socket says 172.21.15.66:33692) 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f441de00640 1 -- 172.21.15.66:0/1303156857 learned_addr learned my addr 172.21.15.66:0/1303156857 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4417e00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44200cc8d0 0x7f4420153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f441e800640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44200cd2d0 0x7f4420097540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f441de00640 1 -- 172.21.15.66:0/1303156857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44200cd2d0 msgr2=0x7f4420097540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f441de00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44200cd2d0 0x7f4420097540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f441de00640 1 -- 172.21.15.66:0/1303156857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44200cc8d0 msgr2=0x7f4420153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f441de00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44200cc8d0 0x7f4420153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f441de00640 1 -- 172.21.15.66:0/1303156857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4418009a70 con 0x7f44200cad00 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f441de00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44200cad00 0x7f4420150ad0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f441000c4d0 tx=0x7f441000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4416000640 1 -- 172.21.15.66:0/1303156857 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410005cf0 con 0x7f44200cad00 2024-03-30T00:44:02.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4416000640 1 -- 172.21.15.66:0/1303156857 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4410005e50 con 0x7f44200cad00 2024-03-30T00:44:02.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4416000640 1 -- 172.21.15.66:0/1303156857 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410007270 con 0x7f44200cad00 2024-03-30T00:44:02.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4424c00640 1 -- 172.21.15.66:0/1303156857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4420071e70 con 0x7f44200cad00 2024-03-30T00:44:02.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4424c00640 1 -- 172.21.15.66:0/1303156857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44201520b0 con 0x7f44200cad00 2024-03-30T00:44:02.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.287+0000 7f4424c00640 1 -- 172.21.15.66:0/1303156857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4420151c00 con 0x7f44200cad00 2024-03-30T00:44:02.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.291+0000 7f4416000640 1 -- 172.21.15.66:0/1303156857 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f44100073d0 con 0x7f44200cad00 2024-03-30T00:44:02.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.291+0000 7f4416000640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43fc066690 0x7f43fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.291+0000 7f4416000640 1 -- 172.21.15.66:0/1303156857 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f4410083970 con 0x7f44200cad00 2024-03-30T00:44:02.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.291+0000 7f4417e00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43fc066690 0x7f43fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.291+0000 7f4417e00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43fc066690 0x7f43fc068b50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f44200cd6d0 tx=0x7f4408008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:02.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.295+0000 7f4416000640 1 -- 172.21.15.66:0/1303156857 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4410050b20 con 0x7f44200cad00 2024-03-30T00:44:02.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 -- 172.21.15.66:0/1303156857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44201522a0 con 0x7f44200cad00 2024-03-30T00:44:02.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4416000640 1 -- 172.21.15.66:0/1303156857 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4410017020 con 0x7f44200cad00 2024-03-30T00:44:02.462 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:44:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 -- 172.21.15.66:0/1303156857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43fc066690 msgr2=0x7f43fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43fc066690 0x7f43fc068b50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f44200cd6d0 tx=0x7f4408008040 comp rx=0 tx=0).stop 2024-03-30T00:44:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 -- 172.21.15.66:0/1303156857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44200cad00 msgr2=0x7f4420150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44200cad00 0x7f4420150ad0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f441000c4d0 tx=0x7f441000c990 comp rx=0 tx=0).stop 2024-03-30T00:44:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 -- 172.21.15.66:0/1303156857 shutdown_connections 2024-03-30T00:44:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44200cd2d0 0x7f4420097540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44200cc8d0 0x7f4420153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f43fc066690 0x7f43fc068b50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 --2- 172.21.15.66:0/1303156857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44200cad00 0x7f4420150ad0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 -- 172.21.15.66:0/1303156857 >> 172.21.15.66:0/1303156857 conn(0x7f4420069f60 msgr2=0x7f44200aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 -- 172.21.15.66:0/1303156857 shutdown_connections 2024-03-30T00:44:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.455+0000 7f4424c00640 1 -- 172.21.15.66:0/1303156857 wait complete. 2024-03-30T00:44:02.483 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:44:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.615+0000 7fc23ac00640 1 Processor -- start 2024-03-30T00:44:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.615+0000 7fc23ac00640 1 -- start start 2024-03-30T00:44:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.615+0000 7fc23ac00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc2340cda90 0x7fc2340ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.615+0000 7fc23ac00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2340cde70 0x7fc2340cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.615+0000 7fc23ac00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc23415a990 con 0x7fc234159c60 2024-03-30T00:44:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.615+0000 7fc23ac00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2340b1b80 con 0x7fc2340cde70 2024-03-30T00:44:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.615+0000 7fc23ac00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2340cd3e0 con 0x7fc2340cda90 2024-03-30T00:44:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.615+0000 7fc22b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2340cde70 0x7fc2340cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.619+0000 7fc22b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2340cde70 0x7fc2340cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33694/0 (socket says 172.21.15.66:33694) 2024-03-30T00:44:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.619+0000 7fc233e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc2340cda90 0x7fc2340ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.619+0000 7fc22b400640 1 -- 172.21.15.66:0/2279459969 learned_addr learned my addr 172.21.15.66:0/2279459969 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.619+0000 7fc22b400640 1 -- 172.21.15.66:0/2279459969 >> v2:172.21.15.156:3300/0 conn(0x7fc2340cda90 msgr2=0x7fc2340ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.619+0000 7fc22b400640 1 --2- 172.21.15.66:0/2279459969 >> v2:172.21.15.156:3300/0 conn(0x7fc2340cda90 0x7fc2340ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.619+0000 7fc22b400640 1 -- 172.21.15.66:0/2279459969 >> v1:172.21.15.156:6789/0 conn(0x7fc234159c60 legacy=0x7fc23415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.619+0000 7fc22b400640 1 -- 172.21.15.66:0/2279459969 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2340b11e0 con 0x7fc2340cde70 2024-03-30T00:44:02.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.619+0000 7fc22b400640 1 --2- 172.21.15.66:0/2279459969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cde70 0x7fc2340cea60 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc2340cf3f0 tx=0x7fc21c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3c3a22272b41b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:02.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.619+0000 7fc233400640 1 -- 172.21.15.66:0/2279459969 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0ac600 con 0x7fc2340cde70 2024-03-30T00:44:02.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.619+0000 7fc233400640 1 -- 172.21.15.66:0/2279459969 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c0a8050 con 0x7fc2340cde70 2024-03-30T00:44:02.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.619+0000 7fc233400640 1 -- 172.21.15.66:0/2279459969 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0ac980 con 0x7fc2340cde70 2024-03-30T00:44:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 -- 172.21.15.66:0/2279459969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cde70 msgr2=0x7fc2340cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 --2- 172.21.15.66:0/2279459969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cde70 0x7fc2340cea60 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc2340cf3f0 tx=0x7fc21c01c490 comp rx=0 tx=0).stop 2024-03-30T00:44:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 -- 172.21.15.66:0/2279459969 shutdown_connections 2024-03-30T00:44:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 --2- 172.21.15.66:0/2279459969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cde70 0x7fc2340cea60 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 --2- 172.21.15.66:0/2279459969 >> v2:172.21.15.156:3300/0 conn(0x7fc2340cda90 0x7fc2340ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 -- 172.21.15.66:0/2279459969 >> 172.21.15.66:0/2279459969 conn(0x7fc234069f60 msgr2=0x7fc23406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 -- 172.21.15.66:0/2279459969 shutdown_connections 2024-03-30T00:44:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 -- 172.21.15.66:0/2279459969 wait complete. 2024-03-30T00:44:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 Processor -- start 2024-03-30T00:44:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 -- start start 2024-03-30T00:44:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cda90 0x7fc234156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2340cde70 0x7fc2340c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc234159c60 0x7fc2340c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2341576d0 con 0x7fc2340cda90 2024-03-30T00:44:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2340c5da0 con 0x7fc234159c60 2024-03-30T00:44:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc23ac00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc234157940 con 0x7fc2340cde70 2024-03-30T00:44:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc233e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cda90 0x7fc234156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc233e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cda90 0x7fc234156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33710/0 (socket says 172.21.15.66:33710) 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc233e00640 1 -- 172.21.15.66:0/3028433888 learned_addr learned my addr 172.21.15.66:0/3028433888 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc22b400640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2340cde70 0x7fc2340c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc238c00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc234159c60 0x7fc2340c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc233e00640 1 -- 172.21.15.66:0/3028433888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2340cde70 msgr2=0x7fc2340c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc233e00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2340cde70 0x7fc2340c1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc233e00640 1 -- 172.21.15.66:0/3028433888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc234159c60 msgr2=0x7fc2340c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc233e00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc234159c60 0x7fc2340c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc233e00640 1 -- 172.21.15.66:0/3028433888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c0094d0 con 0x7fc2340cda90 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc238c00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc234159c60 0x7fc2340c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.623+0000 7fc233e00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cda90 0x7fc234156370 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc22400e870 tx=0x7fc22400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.627+0000 7fc230c00640 1 -- 172.21.15.66:0/3028433888 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224009510 con 0x7fc2340cda90 2024-03-30T00:44:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.627+0000 7fc230c00640 1 -- 172.21.15.66:0/3028433888 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22400eec0 con 0x7fc2340cda90 2024-03-30T00:44:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.627+0000 7fc230c00640 1 -- 172.21.15.66:0/3028433888 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22400cb40 con 0x7fc2340cda90 2024-03-30T00:44:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.627+0000 7fc23ac00640 1 -- 172.21.15.66:0/3028433888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc234157c20 con 0x7fc2340cda90 2024-03-30T00:44:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.627+0000 7fc23ac00640 1 -- 172.21.15.66:0/3028433888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc234159510 con 0x7fc2340cda90 2024-03-30T00:44:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.627+0000 7fc23ac00640 1 -- 172.21.15.66:0/3028433888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc234066c30 con 0x7fc2340cda90 2024-03-30T00:44:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.627+0000 7fc230c00640 1 -- 172.21.15.66:0/3028433888 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc22400cca0 con 0x7fc2340cda90 2024-03-30T00:44:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.627+0000 7fc230c00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 0x7fc210068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.627+0000 7fc230c00640 1 -- 172.21.15.66:0/3028433888 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fc224084450 con 0x7fc2340cda90 2024-03-30T00:44:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.627+0000 7fc22b400640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 0x7fc210068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.627+0000 7fc22b400640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 0x7fc210068b50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0ae000 tx=0x7fc21c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:02.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.631+0000 7fc230c00640 1 -- 172.21.15.66:0/3028433888 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc224051600 con 0x7fc2340cda90 2024-03-30T00:44:02.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.831+0000 7fc23ac00640 1 -- 172.21.15.66:0/3028433888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2340b3170 con 0x7fc2340cda90 2024-03-30T00:44:02.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.831+0000 7fc230c00640 1 -- 172.21.15.66:0/3028433888 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fc224054c20 con 0x7fc2340cda90 2024-03-30T00:44:02.840 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:02.840 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4272961638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4272961638},{"type":"v1","addr":"172.21.15.156:6834","nonce":4272961638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2144186622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2144186622},{"type":"v1","addr":"172.21.15.66:6835","nonce":2144186622}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/750308788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":750308788},{"type":"v1","addr":"172.21.15.66:6837","nonce":750308788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:43:11.141056+0000","modified":"2024-03-30T00:43:55.622856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6468":{"gid":6468,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":14,"addr":"172.21.15.156:6835/1137593870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1137593870},{"type":"v1","addr":"172.21.15.156:6835","nonce":1137593870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-30T00:44:02.840 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 35 2024-03-30T00:44:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 -- 172.21.15.66:0/3028433888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 msgr2=0x7fc210068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 0x7fc210068b50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0ae000 tx=0x7fc21c0a7000 comp rx=0 tx=0).stop 2024-03-30T00:44:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 -- 172.21.15.66:0/3028433888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cda90 msgr2=0x7fc234156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cda90 0x7fc234156370 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc22400e870 tx=0x7fc22400ec70 comp rx=0 tx=0).stop 2024-03-30T00:44:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 -- 172.21.15.66:0/3028433888 shutdown_connections 2024-03-30T00:44:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc234159c60 0x7fc2340c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2340cde70 0x7fc2340c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc210066690 0x7fc210068b50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 --2- 172.21.15.66:0/3028433888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2340cda90 0x7fc234156370 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:02.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 -- 172.21.15.66:0/3028433888 >> 172.21.15.66:0/3028433888 conn(0x7fc234069f60 msgr2=0x7fc23415a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:02.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 -- 172.21.15.66:0/3028433888 shutdown_connections 2024-03-30T00:44:02.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.835+0000 7fc23ac00640 1 -- 172.21.15.66:0/3028433888 wait complete. 2024-03-30T00:44:02.860 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:03.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efbe00640 1 Processor -- start 2024-03-30T00:44:03.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efbe00640 1 -- start start 2024-03-30T00:44:03.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efbe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1efc06a1f0 0x7f1efc06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efbe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc0cd300 con 0x7f1efc160d70 2024-03-30T00:44:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efbe00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc1656d0 con 0x7f1efc069d90 2024-03-30T00:44:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efbe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1efc0ae130 con 0x7f1efc06a1f0 2024-03-30T00:44:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efaa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1efc160d70 0x7f1efc0ada20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46264/0 (socket says 172.21.15.66:46264) 2024-03-30T00:44:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efaa00640 1 -- 172.21.15.66:0/4046265790 learned_addr learned my addr 172.21.15.66:0/4046265790 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efb400640 1 --2- 172.21.15.66:0/4046265790 >> v2:172.21.15.66:3300/0 conn(0x7f1efc06a1f0 0x7f1efc06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1ef9600640 1 -- 172.21.15.66:0/4046265790 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37586317 0 0) 0x7f1efc0cd300 con 0x7f1efc160d70 2024-03-30T00:44:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1ef9600640 1 -- 172.21.15.66:0/4046265790 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ee0003530 con 0x7f1efc160d70 2024-03-30T00:44:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efb400640 1 -- 172.21.15.66:0/4046265790 >> v1:172.21.15.156:6789/0 conn(0x7f1efc069d90 legacy=0x7f1efc06cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efb400640 1 -- 172.21.15.66:0/4046265790 >> v1:172.21.15.66:6789/0 conn(0x7f1efc160d70 legacy=0x7f1efc0ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.995+0000 7f1efb400640 1 -- 172.21.15.66:0/4046265790 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc164b50 con 0x7f1efc06a1f0 2024-03-30T00:44:03.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1efb400640 1 --2- 172.21.15.66:0/4046265790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1efc06a1f0 0x7f1efc06d2b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009d90 tx=0x7f1ef401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2606ac1669abf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:03.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1ef9600640 1 -- 172.21.15.66:0/4046265790 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40a5960 con 0x7f1efc06a1f0 2024-03-30T00:44:03.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1ef9600640 1 -- 172.21.15.66:0/4046265790 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef40a5ac0 con 0x7f1efc06a1f0 2024-03-30T00:44:03.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1ef9600640 1 -- 172.21.15.66:0/4046265790 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40a5e50 con 0x7f1efc06a1f0 2024-03-30T00:44:03.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1efbe00640 1 -- 172.21.15.66:0/4046265790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1efc06a1f0 msgr2=0x7f1efc06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1efbe00640 1 --2- 172.21.15.66:0/4046265790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1efc06a1f0 0x7f1efc06d2b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009d90 tx=0x7f1ef401c8a0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1efbe00640 1 -- 172.21.15.66:0/4046265790 shutdown_connections 2024-03-30T00:44:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1efbe00640 1 --2- 172.21.15.66:0/4046265790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1efc06a1f0 0x7f1efc06d2b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1efbe00640 1 -- 172.21.15.66:0/4046265790 >> 172.21.15.66:0/4046265790 conn(0x7f1efc0ce9f0 msgr2=0x7f1efc0cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1efbe00640 1 -- 172.21.15.66:0/4046265790 shutdown_connections 2024-03-30T00:44:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1efbe00640 1 -- 172.21.15.66:0/4046265790 wait complete. 2024-03-30T00:44:03.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1efbe00640 1 Processor -- start 2024-03-30T00:44:03.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:02.999+0000 7f1efbe00640 1 -- start start 2024-03-30T00:44:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efbe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1efc069d90 0x7f1efc0c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efbe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1efc06a1f0 0x7f1efc1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1efc160d70 0x7f1efc0cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efbe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0c5560 con 0x7f1efc160d70 2024-03-30T00:44:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efbe00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc1564e0 con 0x7f1efc06a1f0 2024-03-30T00:44:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efbe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0c26c0 con 0x7f1efc069d90 2024-03-30T00:44:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1efc160d70 0x7f1efc0cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1efc160d70 0x7f1efc0cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33724/0 (socket says 172.21.15.66:33724) 2024-03-30T00:44:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efb400640 1 -- 172.21.15.66:0/1679675577 learned_addr learned my addr 172.21.15.66:0/1679675577 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efaa00640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1efc069d90 0x7f1efc0c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efa000640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1efc06a1f0 0x7f1efc1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efb400640 1 -- 172.21.15.66:0/1679675577 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1efc069d90 msgr2=0x7f1efc0c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efb400640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1efc069d90 0x7f1efc0c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efb400640 1 -- 172.21.15.66:0/1679675577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1efc06a1f0 msgr2=0x7f1efc1580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efb400640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1efc06a1f0 0x7f1efc1580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efb400640 1 -- 172.21.15.66:0/1679675577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef4009a70 con 0x7f1efc160d70 2024-03-30T00:44:03.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efb400640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1efc160d70 0x7f1efc0cbb00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009d90 tx=0x7f1efc06e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:03.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1ef2a00640 1 -- 172.21.15.66:0/1679675577 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40add70 con 0x7f1efc160d70 2024-03-30T00:44:03.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1ef2a00640 1 -- 172.21.15.66:0/1679675577 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef4009450 con 0x7f1efc160d70 2024-03-30T00:44:03.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efbe00640 1 -- 172.21.15.66:0/1679675577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc0c29a0 con 0x7f1efc160d70 2024-03-30T00:44:03.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1ef2a00640 1 -- 172.21.15.66:0/1679675577 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40b9670 con 0x7f1efc160d70 2024-03-30T00:44:03.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efbe00640 1 -- 172.21.15.66:0/1679675577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc0c5170 con 0x7f1efc160d70 2024-03-30T00:44:03.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.003+0000 7f1efbe00640 1 -- 172.21.15.66:0/1679675577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc067110 con 0x7f1efc160d70 2024-03-30T00:44:03.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.007+0000 7f1ef2a00640 1 -- 172.21.15.66:0/1679675577 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1ef40b97d0 con 0x7f1efc160d70 2024-03-30T00:44:03.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.007+0000 7f1ef2a00640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1edc066690 0x7f1edc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.007+0000 7f1ef2a00640 1 -- 172.21.15.66:0/1679675577 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f1ef411f960 con 0x7f1efc160d70 2024-03-30T00:44:03.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.007+0000 7f1efaa00640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1edc066690 0x7f1edc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.007+0000 7f1efaa00640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1edc066690 0x7f1edc068b50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1eec007fd0 tx=0x7f1eec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:03.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.011+0000 7f1ef2a00640 1 -- 172.21.15.66:0/1679675577 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1ef40ecb10 con 0x7f1efc160d70 2024-03-30T00:44:03.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.171+0000 7f1efbe00640 1 -- 172.21.15.66:0/1679675577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1efc1566a0 con 0x7f1efc160d70 2024-03-30T00:44:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.171+0000 7f1ef2a00640 1 -- 172.21.15.66:0/1679675577 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16487 (secure 0 0 0) 0x7f1ef40f0130 con 0x7f1efc160d70 2024-03-30T00:44:03.180 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:03.180 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":70,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:43:34.562474+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:43:09.681968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T00:43:10.133411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.38","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 -- 172.21.15.66:0/1679675577 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1edc066690 msgr2=0x7f1edc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1edc066690 0x7f1edc068b50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1eec007fd0 tx=0x7f1eec009040 comp rx=0 tx=0).stop 2024-03-30T00:44:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 -- 172.21.15.66:0/1679675577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1efc160d70 msgr2=0x7f1efc0cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1efc160d70 0x7f1efc0cbb00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009d90 tx=0x7f1efc06e000 comp rx=0 tx=0).stop 2024-03-30T00:44:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 -- 172.21.15.66:0/1679675577 shutdown_connections 2024-03-30T00:44:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1edc066690 0x7f1edc068b50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1efc160d70 0x7f1efc0cbb00 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1efc06a1f0 0x7f1efc1580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 --2- 172.21.15.66:0/1679675577 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1efc069d90 0x7f1efc0c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 -- 172.21.15.66:0/1679675577 >> 172.21.15.66:0/1679675577 conn(0x7f1efc0ce9f0 msgr2=0x7f1efc0c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 -- 172.21.15.66:0/1679675577 shutdown_connections 2024-03-30T00:44:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.175+0000 7f1efbe00640 1 -- 172.21.15.66:0/1679675577 wait complete. 2024-03-30T00:44:03.202 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T00:44:03.202 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:44:03.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f9196400640 1 Processor -- start 2024-03-30T00:44:03.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f9196400640 1 -- start start 2024-03-30T00:44:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f9196400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f919006bf30 0x7f919006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f9196400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f919015f410 0x7f919015f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f9196400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91900af990 con 0x7f91900d0be0 2024-03-30T00:44:03.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f9196400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91900af380 con 0x7f919006bf30 2024-03-30T00:44:03.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f9196400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919006c980 con 0x7f919015f410 2024-03-30T00:44:03.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f918f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f91900d0be0 0x7f91900d0fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46274/0 (socket says 172.21.15.66:46274) 2024-03-30T00:44:03.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f918f400640 1 -- 172.21.15.66:0/3710911445 learned_addr learned my addr 172.21.15.66:0/3710911445 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f918fe00640 1 --2- 172.21.15.66:0/3710911445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f919015f410 0x7f919015f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f9186a00640 1 --2- 172.21.15.66:0/3710911445 >> v2:172.21.15.156:3300/0 conn(0x7f919006bf30 0x7f919006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f918ea00640 1 -- 172.21.15.66:0/3710911445 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219634913 0 0) 0x7f91900af990 con 0x7f91900d0be0 2024-03-30T00:44:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f918ea00640 1 -- 172.21.15.66:0/3710911445 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9174003530 con 0x7f91900d0be0 2024-03-30T00:44:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f918fe00640 1 -- 172.21.15.66:0/3710911445 >> v2:172.21.15.156:3300/0 conn(0x7f919006bf30 msgr2=0x7f919006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f918ea00640 1 -- 172.21.15.66:0/3710911445 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2942673185 0 0) 0x7f9174003530 con 0x7f91900d0be0 2024-03-30T00:44:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f918fe00640 1 --2- 172.21.15.66:0/3710911445 >> v2:172.21.15.156:3300/0 conn(0x7f919006bf30 0x7f919006c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f918fe00640 1 -- 172.21.15.66:0/3710911445 >> v1:172.21.15.66:6789/0 conn(0x7f91900d0be0 legacy=0x7f91900d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.335+0000 7f918fe00640 1 -- 172.21.15.66:0/3710911445 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919006cc60 con 0x7f919015f410 2024-03-30T00:44:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f918fe00640 1 --2- 172.21.15.66:0/3710911445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f919015f410 0x7f919015f8a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91880097c0 tx=0x7f918801c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb7f0d7cd6a7523 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f918ea00640 1 -- 172.21.15.66:0/3710911445 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180001a00 con 0x7f919015f410 2024-03-30T00:44:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f918ea00640 1 -- 172.21.15.66:0/3710911445 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91880a7050 con 0x7f919015f410 2024-03-30T00:44:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f918ea00640 1 -- 172.21.15.66:0/3710911445 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91880ab600 con 0x7f919015f410 2024-03-30T00:44:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f9196400640 1 -- 172.21.15.66:0/3710911445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f919015f410 msgr2=0x7f919015f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f9196400640 1 --2- 172.21.15.66:0/3710911445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f919015f410 0x7f919015f8a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91880097c0 tx=0x7f918801c390 comp rx=0 tx=0).stop 2024-03-30T00:44:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f9196400640 1 -- 172.21.15.66:0/3710911445 shutdown_connections 2024-03-30T00:44:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f9196400640 1 --2- 172.21.15.66:0/3710911445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f919015f410 0x7f919015f8a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f9196400640 1 --2- 172.21.15.66:0/3710911445 >> v2:172.21.15.156:3300/0 conn(0x7f919006bf30 0x7f919006c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f9196400640 1 -- 172.21.15.66:0/3710911445 >> 172.21.15.66:0/3710911445 conn(0x7f919009ba90 msgr2=0x7f9190089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f9196400640 1 -- 172.21.15.66:0/3710911445 shutdown_connections 2024-03-30T00:44:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.339+0000 7f9196400640 1 -- 172.21.15.66:0/3710911445 wait complete. 2024-03-30T00:44:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9196400640 1 Processor -- start 2024-03-30T00:44:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9196400640 1 -- start start 2024-03-30T00:44:03.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9196400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919006bf30 0x7f91900cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9196400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91900d0be0 0x7f9190152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9196400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f919015f410 0x7f91900c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9196400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91900cb240 con 0x7f919006bf30 2024-03-30T00:44:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9196400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91900ca5c0 con 0x7f91900d0be0 2024-03-30T00:44:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9196400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91900ca8a0 con 0x7f919015f410 2024-03-30T00:44:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f918f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919006bf30 0x7f91900cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f918f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919006bf30 0x7f91900cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33730/0 (socket says 172.21.15.66:33730) 2024-03-30T00:44:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f918f400640 1 -- 172.21.15.66:0/3227301559 learned_addr learned my addr 172.21.15.66:0/3227301559 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f918fe00640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f919015f410 0x7f91900c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9186a00640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91900d0be0 0x7f9190152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f918f400640 1 -- 172.21.15.66:0/3227301559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f919015f410 msgr2=0x7f91900c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f918f400640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f919015f410 0x7f91900c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f918f400640 1 -- 172.21.15.66:0/3227301559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91900d0be0 msgr2=0x7f9190152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f918f400640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91900d0be0 0x7f9190152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f918f400640 1 -- 172.21.15.66:0/3227301559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9188009420 con 0x7f919006bf30 2024-03-30T00:44:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f918f400640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919006bf30 0x7f91900cb660 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f91800018e0 tx=0x7f918000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:03.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9187e00640 1 -- 172.21.15.66:0/3227301559 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180012400 con 0x7f919006bf30 2024-03-30T00:44:03.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9187e00640 1 -- 172.21.15.66:0/3227301559 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9180012560 con 0x7f919006bf30 2024-03-30T00:44:03.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9196400640 1 -- 172.21.15.66:0/3227301559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91900cab80 con 0x7f919006bf30 2024-03-30T00:44:03.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9187e00640 1 -- 172.21.15.66:0/3227301559 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180012950 con 0x7f919006bf30 2024-03-30T00:44:03.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9196400640 1 -- 172.21.15.66:0/3227301559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91900c8cf0 con 0x7f919006bf30 2024-03-30T00:44:03.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.343+0000 7f9196400640 1 -- 172.21.15.66:0/3227301559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91900cb380 con 0x7f919006bf30 2024-03-30T00:44:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.347+0000 7f9187e00640 1 -- 172.21.15.66:0/3227301559 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9180012ab0 con 0x7f919006bf30 2024-03-30T00:44:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.347+0000 7f9187e00640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f916c0666c0 0x7f916c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.347+0000 7f9187e00640 1 -- 172.21.15.66:0/3227301559 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f9180084a80 con 0x7f919006bf30 2024-03-30T00:44:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.347+0000 7f9186a00640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f916c0666c0 0x7f916c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.347+0000 7f9186a00640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f916c0666c0 0x7f916c068b80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f919015f270 tx=0x7f9178006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:03.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.351+0000 7f9187e00640 1 -- 172.21.15.66:0/3227301559 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9180051c30 con 0x7f919006bf30 2024-03-30T00:44:03.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 -- 172.21.15.66:0/3227301559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91900b84d0 con 0x7f919006bf30 2024-03-30T00:44:03.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9187e00640 1 -- 172.21.15.66:0/3227301559 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f9180055250 con 0x7f919006bf30 2024-03-30T00:44:03.554 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:03.554 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4272961638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4272961638},{"type":"v1","addr":"172.21.15.156:6834","nonce":4272961638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2144186622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2144186622},{"type":"v1","addr":"172.21.15.66:6835","nonce":2144186622}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/750308788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":750308788},{"type":"v1","addr":"172.21.15.66:6837","nonce":750308788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:43:11.141056+0000","modified":"2024-03-30T00:43:55.622856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6468":{"gid":6468,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":14,"addr":"172.21.15.156:6835/1137593870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1137593870},{"type":"v1","addr":"172.21.15.156:6835","nonce":1137593870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-30T00:44:03.555 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 35 2024-03-30T00:44:03.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 -- 172.21.15.66:0/3227301559 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f916c0666c0 msgr2=0x7f916c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f916c0666c0 0x7f916c068b80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f919015f270 tx=0x7f9178006f90 comp rx=0 tx=0).stop 2024-03-30T00:44:03.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 -- 172.21.15.66:0/3227301559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919006bf30 msgr2=0x7f91900cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919006bf30 0x7f91900cb660 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f91800018e0 tx=0x7f918000a430 comp rx=0 tx=0).stop 2024-03-30T00:44:03.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 -- 172.21.15.66:0/3227301559 shutdown_connections 2024-03-30T00:44:03.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f919015f410 0x7f91900c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91900d0be0 0x7f9190152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f916c0666c0 0x7f916c068b80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 --2- 172.21.15.66:0/3227301559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919006bf30 0x7f91900cb660 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 -- 172.21.15.66:0/3227301559 >> 172.21.15.66:0/3227301559 conn(0x7f919009ba90 msgr2=0x7f91900d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:03.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 -- 172.21.15.66:0/3227301559 shutdown_connections 2024-03-30T00:44:03.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.547+0000 7f9196400640 1 -- 172.21.15.66:0/3227301559 wait complete. 2024-03-30T00:44:03.576 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:44:03.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.707+0000 7fdd54a00640 1 Processor -- start 2024-03-30T00:44:03.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.707+0000 7fdd54a00640 1 -- start start 2024-03-30T00:44:03.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd54a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd500cc560 0x7fdd500d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd54a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdd5006e800 0x7fdd500ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd54a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5006b0b0 con 0x7fdd5006f8e0 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd54a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd50160640 con 0x7fdd5006e800 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd54a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5006b390 con 0x7fdd500cc560 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd47e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdd5006f8e0 0x7fdd5006e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46276/0 (socket says 172.21.15.66:46276) 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd47e00640 1 -- 172.21.15.66:0/3928862351 learned_addr learned my addr 172.21.15.66:0/3928862351 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd4e800640 1 --2- 172.21.15.66:0/3928862351 >> v2:172.21.15.66:3300/0 conn(0x7fdd5006e800 0x7fdd500ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd4de00640 1 --2- 172.21.15.66:0/3928862351 >> v2:172.21.15.156:3300/0 conn(0x7fdd500cc560 0x7fdd500d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd4d400640 1 -- 172.21.15.66:0/3928862351 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115661348 0 0) 0x7fdd5006b0b0 con 0x7fdd5006f8e0 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd4d400640 1 -- 172.21.15.66:0/3928862351 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd34003530 con 0x7fdd5006f8e0 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd4e800640 1 -- 172.21.15.66:0/3928862351 >> v2:172.21.15.156:3300/0 conn(0x7fdd500cc560 msgr2=0x7fdd500d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd4e800640 1 --2- 172.21.15.66:0/3928862351 >> v2:172.21.15.156:3300/0 conn(0x7fdd500cc560 0x7fdd500d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd4e800640 1 -- 172.21.15.66:0/3928862351 >> v1:172.21.15.66:6789/0 conn(0x7fdd5006f8e0 legacy=0x7fdd5006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd4e800640 1 -- 172.21.15.66:0/3928862351 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5006b670 con 0x7fdd5006e800 2024-03-30T00:44:03.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.711+0000 7fdd4de00640 1 --2- 172.21.15.66:0/3928862351 >> v2:172.21.15.156:3300/0 conn(0x7fdd500cc560 0x7fdd500d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:03.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd4e800640 1 --2- 172.21.15.66:0/3928862351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd5006e800 0x7fdd500ce8d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdd50072c50 tx=0x7fdd4801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5accac4dfb571c48 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:03.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd4d400640 1 -- 172.21.15.66:0/3928862351 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480a59e0 con 0x7fdd5006e800 2024-03-30T00:44:03.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd4d400640 1 -- 172.21.15.66:0/3928862351 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd480a5b40 con 0x7fdd5006e800 2024-03-30T00:44:03.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd4d400640 1 -- 172.21.15.66:0/3928862351 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480ad440 con 0x7fdd5006e800 2024-03-30T00:44:03.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd54a00640 1 -- 172.21.15.66:0/3928862351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd5006e800 msgr2=0x7fdd500ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd54a00640 1 --2- 172.21.15.66:0/3928862351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd5006e800 0x7fdd500ce8d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdd50072c50 tx=0x7fdd4801c900 comp rx=0 tx=0).stop 2024-03-30T00:44:03.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd54a00640 1 -- 172.21.15.66:0/3928862351 shutdown_connections 2024-03-30T00:44:03.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd54a00640 1 --2- 172.21.15.66:0/3928862351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd5006e800 0x7fdd500ce8d0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd54a00640 1 --2- 172.21.15.66:0/3928862351 >> v2:172.21.15.156:3300/0 conn(0x7fdd500cc560 0x7fdd500d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd54a00640 1 -- 172.21.15.66:0/3928862351 >> 172.21.15.66:0/3928862351 conn(0x7fdd500d0540 msgr2=0x7fdd500d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:03.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd54a00640 1 -- 172.21.15.66:0/3928862351 shutdown_connections 2024-03-30T00:44:03.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd54a00640 1 -- 172.21.15.66:0/3928862351 wait complete. 2024-03-30T00:44:03.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd54a00640 1 Processor -- start 2024-03-30T00:44:03.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.715+0000 7fdd54a00640 1 -- start start 2024-03-30T00:44:03.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd54a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd5006e800 0x7fdd50155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd54a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5006f8e0 0x7fdd500b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd54a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cc560 0x7fdd5015d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd54a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500b0140 con 0x7fdd5006e800 2024-03-30T00:44:03.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd54a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500b0390 con 0x7fdd5006f8e0 2024-03-30T00:44:03.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd54a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500b0670 con 0x7fdd500cc560 2024-03-30T00:44:03.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd4de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd5006e800 0x7fdd50155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd4de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd5006e800 0x7fdd50155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33752/0 (socket says 172.21.15.66:33752) 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd4de00640 1 -- 172.21.15.66:0/1164898530 learned_addr learned my addr 172.21.15.66:0/1164898530 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd47e00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5006f8e0 0x7fdd500b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd4e800640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cc560 0x7fdd5015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd4de00640 1 -- 172.21.15.66:0/1164898530 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cc560 msgr2=0x7fdd5015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd4de00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cc560 0x7fdd5015d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd4de00640 1 -- 172.21.15.66:0/1164898530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5006f8e0 msgr2=0x7fdd500b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd4de00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5006f8e0 0x7fdd500b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd4de00640 1 -- 172.21.15.66:0/1164898530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48009a70 con 0x7fdd5006e800 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd4e800640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cc560 0x7fdd5015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd47e00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5006f8e0 0x7fdd500b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd4de00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd5006e800 0x7fdd50155780 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000d800 tx=0x7fdd4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd46000640 1 -- 172.21.15.66:0/1164898530 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd400096c0 con 0x7fdd5006e800 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd46000640 1 -- 172.21.15.66:0/1164898530 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd40009820 con 0x7fdd5006e800 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd46000640 1 -- 172.21.15.66:0/1164898530 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40009bd0 con 0x7fdd5006e800 2024-03-30T00:44:03.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd54a00640 1 -- 172.21.15.66:0/1164898530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd500b0950 con 0x7fdd5006e800 2024-03-30T00:44:03.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd54a00640 1 -- 172.21.15.66:0/1164898530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd50155cc0 con 0x7fdd5006e800 2024-03-30T00:44:03.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd54a00640 1 -- 172.21.15.66:0/1164898530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd50066c30 con 0x7fdd5006e800 2024-03-30T00:44:03.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.719+0000 7fdd46000640 1 -- 172.21.15.66:0/1164898530 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdd40009d30 con 0x7fdd5006e800 2024-03-30T00:44:03.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.723+0000 7fdd46000640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 0x7fdd2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:03.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.723+0000 7fdd46000640 1 -- 172.21.15.66:0/1164898530 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fdd40084a60 con 0x7fdd5006e800 2024-03-30T00:44:03.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.723+0000 7fdd47e00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 0x7fdd2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:03.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.723+0000 7fdd47e00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 0x7fdd2c068b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdd500c97d0 tx=0x7fdd38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:03.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.727+0000 7fdd46000640 1 -- 172.21.15.66:0/1164898530 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdd40051c00 con 0x7fdd5006e800 2024-03-30T00:44:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.919+0000 7fdd54a00640 1 -- 172.21.15.66:0/1164898530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd5006e430 con 0x7fdd5006e800 2024-03-30T00:44:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.919+0000 7fdd46000640 1 -- 172.21.15.66:0/1164898530 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fdd40055220 con 0x7fdd5006e800 2024-03-30T00:44:03.927 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:03.927 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4272961638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4272961638},{"type":"v1","addr":"172.21.15.156:6834","nonce":4272961638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2144186622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2144186622},{"type":"v1","addr":"172.21.15.66:6835","nonce":2144186622}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/750308788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":750308788},{"type":"v1","addr":"172.21.15.66:6837","nonce":750308788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:43:11.141056+0000","modified":"2024-03-30T00:43:55.622856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6468":{"gid":6468,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":14,"addr":"172.21.15.156:6835/1137593870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1137593870},{"type":"v1","addr":"172.21.15.156:6835","nonce":1137593870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-30T00:44:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 35 2024-03-30T00:44:03.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 -- 172.21.15.66:0/1164898530 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 msgr2=0x7fdd2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 0x7fdd2c068b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdd500c97d0 tx=0x7fdd38009040 comp rx=0 tx=0).stop 2024-03-30T00:44:03.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 -- 172.21.15.66:0/1164898530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd5006e800 msgr2=0x7fdd50155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:03.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd5006e800 0x7fdd50155780 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000d800 tx=0x7fdd4000dcc0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 -- 172.21.15.66:0/1164898530 shutdown_connections 2024-03-30T00:44:03.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd500cc560 0x7fdd5015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd5006f8e0 0x7fdd500b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd2c066690 0x7fdd2c068b50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 --2- 172.21.15.66:0/1164898530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd5006e800 0x7fdd50155780 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:03.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 -- 172.21.15.66:0/1164898530 >> 172.21.15.66:0/1164898530 conn(0x7fdd500d0540 msgr2=0x7fdd500ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:03.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 -- 172.21.15.66:0/1164898530 shutdown_connections 2024-03-30T00:44:03.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:03.923+0000 7fdd54a00640 1 -- 172.21.15.66:0/1164898530 wait complete. 2024-03-30T00:44:03.948 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:04.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd5a400640 1 Processor -- start 2024-03-30T00:44:04.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd5a400640 1 -- start start 2024-03-30T00:44:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd5a400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5406de40 0x7fdd5415ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd5a400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5415e250 con 0x7fdd5406da60 2024-03-30T00:44:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd5a400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5415e530 con 0x7fdd540c92d0 2024-03-30T00:44:04.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd5a400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5415ebf0 con 0x7fdd5406de40 2024-03-30T00:44:04.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd59000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdd5406da60 0x7fdd540c8bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46286/0 (socket says 172.21.15.66:46286) 2024-03-30T00:44:04.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd59000640 1 -- 172.21.15.66:0/499631372 learned_addr learned my addr 172.21.15.66:0/499631372 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd4be00640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5406de40 0x7fdd5415ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559862190 0 0) 0x7fdd5415e250 con 0x7fdd5406da60 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd38003530 con 0x7fdd5406da60 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2849616446 0 0) 0x7fdd38003530 con 0x7fdd5406da60 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5406de40 msgr2=0x7fdd5415ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5406de40 0x7fdd5415ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 >> v1:172.21.15.156:6789/0 conn(0x7fdd540c92d0 legacy=0x7fdd5415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5415e810 con 0x7fdd5406da60 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fdd44001680 con 0x7fdd5406da60 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 >> v1:172.21.15.66:6789/0 conn(0x7fdd5406da60 legacy=0x7fdd540c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd380039b0 0x7fdd38003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5406de40 0x7fdd380047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd38004d10 0x7fdd38005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5415ebf0 con 0x7fdd38004d10 2024-03-30T00:44:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3800ba40 con 0x7fdd380039b0 2024-03-30T00:44:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3800bc00 con 0x7fdd5406de40 2024-03-30T00:44:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd59a00640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd38004d10 0x7fdd38005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd59000640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd380039b0 0x7fdd38003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd4be00640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5406de40 0x7fdd380047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd59a00640 1 -- 172.21.15.66:0/499631372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5406de40 msgr2=0x7fdd380047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd59a00640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd5406de40 0x7fdd380047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd59a00640 1 -- 172.21.15.66:0/499631372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd380039b0 msgr2=0x7fdd38003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd59a00640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd380039b0 0x7fdd38003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.083+0000 7fdd59a00640 1 -- 172.21.15.66:0/499631372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38003530 con 0x7fdd38004d10 2024-03-30T00:44:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.087+0000 7fdd59a00640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd38004d10 0x7fdd38005190 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c001f60 tx=0x7fdd4c01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595000644154ecc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.087+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0a6670 con 0x7fdd38004d10 2024-03-30T00:44:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.087+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c0a67d0 con 0x7fdd38004d10 2024-03-30T00:44:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.087+0000 7fdd53e00640 1 -- 172.21.15.66:0/499631372 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0a6c00 con 0x7fdd38004d10 2024-03-30T00:44:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.087+0000 7fdd5a400640 1 -- 172.21.15.66:0/499631372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd38004d10 msgr2=0x7fdd38005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.087+0000 7fdd5a400640 1 --2- 172.21.15.66:0/499631372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd38004d10 0x7fdd38005190 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c001f60 tx=0x7fdd4c01c880 comp rx=0 tx=0).stop 2024-03-30T00:44:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.087+0000 7fdd59a00640 1 -- 172.21.15.66:0/499631372 reap_dead start 2024-03-30T00:44:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.087+0000 7fdd5a400640 1 -- 172.21.15.66:0/499631372 shutdown_connections 2024-03-30T00:44:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.087+0000 7fdd5a400640 1 -- 172.21.15.66:0/499631372 >> 172.21.15.66:0/499631372 conn(0x7fdd540b00e0 msgr2=0x7fdd540b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.087+0000 7fdd5a400640 1 -- 172.21.15.66:0/499631372 shutdown_connections 2024-03-30T00:44:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.087+0000 7fdd5a400640 1 -- 172.21.15.66:0/499631372 wait complete. 2024-03-30T00:44:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd5a400640 1 Processor -- start 2024-03-30T00:44:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd5a400640 1 -- start start 2024-03-30T00:44:04.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd5a400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd38004d10 0x7fdd540c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd5a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd540b3830 0x7fdd54159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd5a400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd540b3cb0 0x7fdd54159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd5a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540c6bd0 con 0x7fdd540b3830 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd5a400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54159bc0 con 0x7fdd38004d10 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd5a400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540c6330 con 0x7fdd540b3cb0 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd540b3830 0x7fdd54159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd540b3830 0x7fdd54159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33768/0 (socket says 172.21.15.66:33768) 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd4be00640 1 -- 172.21.15.66:0/3188426255 learned_addr learned my addr 172.21.15.66:0/3188426255 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd59a00640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd540b3cb0 0x7fdd54159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd59000640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd38004d10 0x7fdd540c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd4be00640 1 -- 172.21.15.66:0/3188426255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd540b3cb0 msgr2=0x7fdd54159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd4be00640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd540b3cb0 0x7fdd54159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd4be00640 1 -- 172.21.15.66:0/3188426255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd38004d10 msgr2=0x7fdd540c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd4be00640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd38004d10 0x7fdd540c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd4be00640 1 -- 172.21.15.66:0/3188426255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd540c6610 con 0x7fdd540b3830 2024-03-30T00:44:04.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd59a00640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd540b3cb0 0x7fdd54159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:04.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd4be00640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd540b3830 0x7fdd54159140 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0119d0 tx=0x7fdd3c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:04.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd51600640 1 -- 172.21.15.66:0/3188426255 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c00faf0 con 0x7fdd540b3830 2024-03-30T00:44:04.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd51600640 1 -- 172.21.15.66:0/3188426255 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c00fc50 con 0x7fdd540b3830 2024-03-30T00:44:04.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd51600640 1 -- 172.21.15.66:0/3188426255 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c014440 con 0x7fdd540b3830 2024-03-30T00:44:04.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd5a400640 1 -- 172.21.15.66:0/3188426255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd540c68f0 con 0x7fdd540b3830 2024-03-30T00:44:04.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.091+0000 7fdd5a400640 1 -- 172.21.15.66:0/3188426255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd540c5ff0 con 0x7fdd540b3830 2024-03-30T00:44:04.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.095+0000 7fdd5a400640 1 -- 172.21.15.66:0/3188426255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd54073cf0 con 0x7fdd540b3830 2024-03-30T00:44:04.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.095+0000 7fdd51600640 1 -- 172.21.15.66:0/3188426255 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdd3c0145a0 con 0x7fdd540b3830 2024-03-30T00:44:04.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.095+0000 7fdd51600640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd30066690 0x7fdd30068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.095+0000 7fdd51600640 1 -- 172.21.15.66:0/3188426255 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fdd3c084a80 con 0x7fdd540b3830 2024-03-30T00:44:04.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.095+0000 7fdd59000640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd30066690 0x7fdd30068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.095+0000 7fdd59000640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd30066690 0x7fdd30068b50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdd540c4e10 tx=0x7fdd44001160 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:04.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.099+0000 7fdd51600640 1 -- 172.21.15.66:0/3188426255 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdd3c051d70 con 0x7fdd540b3830 2024-03-30T00:44:04.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.267+0000 7fdd5a400640 1 -- 172.21.15.66:0/3188426255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd540c7470 con 0x7fdd540b3830 2024-03-30T00:44:04.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.267+0000 7fdd51600640 1 -- 172.21.15.66:0/3188426255 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16487 (secure 0 0 0) 0x7fdd3c055390 con 0x7fdd540b3830 2024-03-30T00:44:04.274 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:04.275 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":70,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:43:34.562474+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:43:09.681968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T00:43:10.133411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.38","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 -- 172.21.15.66:0/3188426255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd30066690 msgr2=0x7fdd30068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd30066690 0x7fdd30068b50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdd540c4e10 tx=0x7fdd44001160 comp rx=0 tx=0).stop 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 -- 172.21.15.66:0/3188426255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd540b3830 msgr2=0x7fdd54159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd540b3830 0x7fdd54159140 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0119d0 tx=0x7fdd3c011dd0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 -- 172.21.15.66:0/3188426255 shutdown_connections 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd540b3cb0 0x7fdd54159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdd30066690 0x7fdd30068b50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd540b3830 0x7fdd54159140 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 --2- 172.21.15.66:0/3188426255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd38004d10 0x7fdd540c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 -- 172.21.15.66:0/3188426255 >> 172.21.15.66:0/3188426255 conn(0x7fdd540b00e0 msgr2=0x7fdd5415b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 -- 172.21.15.66:0/3188426255 shutdown_connections 2024-03-30T00:44:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.271+0000 7fdd5a400640 1 -- 172.21.15.66:0/3188426255 wait complete. 2024-03-30T00:44:04.297 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:44:04.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.431+0000 7fb15ee00640 1 Processor -- start 2024-03-30T00:44:04.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.431+0000 7fb15ee00640 1 -- start start 2024-03-30T00:44:04.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.431+0000 7fb15ee00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb158098c80 0x7fb158099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.431+0000 7fb15ee00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb158099ca0 0x7fb1580ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.431+0000 7fb15ee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1580cbd90 con 0x7fb1580cf320 2024-03-30T00:44:04.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.431+0000 7fb15ee00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1580cc070 con 0x7fb158099ca0 2024-03-30T00:44:04.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.431+0000 7fb15ee00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1580cc350 con 0x7fb158098c80 2024-03-30T00:44:04.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb14f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb1580cf320 0x7fb1580ce210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46298/0 (socket says 172.21.15.66:46298) 2024-03-30T00:44:04.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb14f400640 1 -- 172.21.15.66:0/3712001223 learned_addr learned my addr 172.21.15.66:0/3712001223 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:04.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb15ce00640 1 --2- 172.21.15.66:0/3712001223 >> v2:172.21.15.66:3300/0 conn(0x7fb158099ca0 0x7fb1580ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb157e00640 1 --2- 172.21.15.66:0/3712001223 >> v2:172.21.15.156:3300/0 conn(0x7fb158098c80 0x7fb158099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb157400640 1 -- 172.21.15.66:0/3712001223 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269002566 0 0) 0x7fb1580cbd90 con 0x7fb1580cf320 2024-03-30T00:44:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb157400640 1 -- 172.21.15.66:0/3712001223 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb13c003530 con 0x7fb1580cf320 2024-03-30T00:44:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb15ce00640 1 -- 172.21.15.66:0/3712001223 >> v2:172.21.15.156:3300/0 conn(0x7fb158098c80 msgr2=0x7fb158099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb15ce00640 1 --2- 172.21.15.66:0/3712001223 >> v2:172.21.15.156:3300/0 conn(0x7fb158098c80 0x7fb158099760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb15ce00640 1 -- 172.21.15.66:0/3712001223 >> v1:172.21.15.66:6789/0 conn(0x7fb1580cf320 legacy=0x7fb1580ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb15ce00640 1 -- 172.21.15.66:0/3712001223 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1580b3530 con 0x7fb158099ca0 2024-03-30T00:44:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb157e00640 1 --2- 172.21.15.66:0/3712001223 >> v2:172.21.15.156:3300/0 conn(0x7fb158098c80 0x7fb158099760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:04.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb15ce00640 1 --2- 172.21.15.66:0/3712001223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb158099ca0 0x7fb1580ce920 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb158071550 tx=0x7fb15001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a0ac8ea6bf7175 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:04.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb157400640 1 -- 172.21.15.66:0/3712001223 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1500af600 con 0x7fb158099ca0 2024-03-30T00:44:04.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb157400640 1 -- 172.21.15.66:0/3712001223 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1500ab050 con 0x7fb158099ca0 2024-03-30T00:44:04.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.435+0000 7fb157400640 1 -- 172.21.15.66:0/3712001223 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1500af980 con 0x7fb158099ca0 2024-03-30T00:44:04.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 -- 172.21.15.66:0/3712001223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb158099ca0 msgr2=0x7fb1580ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 --2- 172.21.15.66:0/3712001223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb158099ca0 0x7fb1580ce920 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb158071550 tx=0x7fb15001f110 comp rx=0 tx=0).stop 2024-03-30T00:44:04.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 -- 172.21.15.66:0/3712001223 shutdown_connections 2024-03-30T00:44:04.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 --2- 172.21.15.66:0/3712001223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb158099ca0 0x7fb1580ce920 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 --2- 172.21.15.66:0/3712001223 >> v2:172.21.15.156:3300/0 conn(0x7fb158098c80 0x7fb158099760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 -- 172.21.15.66:0/3712001223 >> 172.21.15.66:0/3712001223 conn(0x7fb1580b2130 msgr2=0x7fb1580b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:04.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 -- 172.21.15.66:0/3712001223 shutdown_connections 2024-03-30T00:44:04.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 -- 172.21.15.66:0/3712001223 wait complete. 2024-03-30T00:44:04.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 Processor -- start 2024-03-30T00:44:04.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 -- start start 2024-03-30T00:44:04.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb158098c80 0x7fb15806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb158099ca0 0x7fb1580c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1580cf320 0x7fb1580b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1580c7c50 con 0x7fb158098c80 2024-03-30T00:44:04.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1580c7e80 con 0x7fb1580cf320 2024-03-30T00:44:04.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb15ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1580c8160 con 0x7fb158099ca0 2024-03-30T00:44:04.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb157e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb158098c80 0x7fb15806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb157e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb158098c80 0x7fb15806f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33784/0 (socket says 172.21.15.66:33784) 2024-03-30T00:44:04.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb157e00640 1 -- 172.21.15.66:0/3987436773 learned_addr learned my addr 172.21.15.66:0/3987436773 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:04.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb157e00640 1 -- 172.21.15.66:0/3987436773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb158099ca0 msgr2=0x7fb1580c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb157e00640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb158099ca0 0x7fb1580c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb157e00640 1 -- 172.21.15.66:0/3987436773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1580cf320 msgr2=0x7fb1580b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb157e00640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1580cf320 0x7fb1580b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb157e00640 1 -- 172.21.15.66:0/3987436773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb150009a70 con 0x7fb158098c80 2024-03-30T00:44:04.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.439+0000 7fb157e00640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb158098c80 0x7fb15806f4c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb14800e810 tx=0x7fb14800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:04.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.443+0000 7fb154c00640 1 -- 172.21.15.66:0/3987436773 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14800c8d0 con 0x7fb158098c80 2024-03-30T00:44:04.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.443+0000 7fb154c00640 1 -- 172.21.15.66:0/3987436773 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14800ca30 con 0x7fb158098c80 2024-03-30T00:44:04.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.443+0000 7fb15ee00640 1 -- 172.21.15.66:0/3987436773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1580c8440 con 0x7fb158098c80 2024-03-30T00:44:04.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.443+0000 7fb154c00640 1 -- 172.21.15.66:0/3987436773 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14800cdc0 con 0x7fb158098c80 2024-03-30T00:44:04.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.443+0000 7fb15ee00640 1 -- 172.21.15.66:0/3987436773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1580c4210 con 0x7fb158098c80 2024-03-30T00:44:04.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.443+0000 7fb15ee00640 1 -- 172.21.15.66:0/3987436773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1580677d0 con 0x7fb158098c80 2024-03-30T00:44:04.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.443+0000 7fb154c00640 1 -- 172.21.15.66:0/3987436773 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb148013410 con 0x7fb158098c80 2024-03-30T00:44:04.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.443+0000 7fb154c00640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb134066690 0x7fb134068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.443+0000 7fb154c00640 1 -- 172.21.15.66:0/3987436773 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fb148084240 con 0x7fb158098c80 2024-03-30T00:44:04.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.443+0000 7fb14f400640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb134066690 0x7fb134068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.443+0000 7fb14f400640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb134066690 0x7fb134068b50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb140007fd0 tx=0x7fb140007940 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:04.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.447+0000 7fb154c00640 1 -- 172.21.15.66:0/3987436773 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb148051350 con 0x7fb158098c80 2024-03-30T00:44:04.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.643+0000 7fb15ee00640 1 -- 172.21.15.66:0/3987436773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb158067340 con 0x7fb158098c80 2024-03-30T00:44:04.650 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:44:04.643+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:44:04.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.651+0000 7fb154c00640 1 -- 172.21.15.66:0/3987436773 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb148054970 con 0x7fb158098c80 2024-03-30T00:44:04.660 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:44:04.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 -- 172.21.15.66:0/3987436773 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb134066690 msgr2=0x7fb134068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb134066690 0x7fb134068b50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb140007fd0 tx=0x7fb140007940 comp rx=0 tx=0).stop 2024-03-30T00:44:04.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 -- 172.21.15.66:0/3987436773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb158098c80 msgr2=0x7fb15806f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb158098c80 0x7fb15806f4c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb14800e810 tx=0x7fb14800ecd0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 -- 172.21.15.66:0/3987436773 shutdown_connections 2024-03-30T00:44:04.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1580cf320 0x7fb1580b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb158099ca0 0x7fb1580c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb134066690 0x7fb134068b50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 --2- 172.21.15.66:0/3987436773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb158098c80 0x7fb15806f4c0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 -- 172.21.15.66:0/3987436773 >> 172.21.15.66:0/3987436773 conn(0x7fb1580b2130 msgr2=0x7fb15806c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:04.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 -- 172.21.15.66:0/3987436773 shutdown_connections 2024-03-30T00:44:04.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.655+0000 7fb15ee00640 1 -- 172.21.15.66:0/3987436773 wait complete. 2024-03-30T00:44:04.670 INFO:tasks.ceph.mds.d.smithi156.stderr: -5221> 2024-03-30T00:43:12.676+0000 7f6209e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:44:04.674 INFO:tasks.ceph.mds.d.smithi156.stderr: -3535> 2024-03-30T00:43:20.248+0000 7f6205800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6597=0-4194304@1} caps={6597=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6597 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5628a73afb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:44:04.676 INFO:tasks.ceph.mds.d.smithi156.stderr: -1741> 2024-03-30T00:43:51.827+0000 7f6205800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-03-30T00:43:18.868087+0000 1=1+0) (iversion lock) caps={6597=pAsLsXsFs/-@2},l=6597(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x5628a73afb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:44:04.681 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T00:44:04.722 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2024-03-30T00:44:04.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f7200640 1 Processor -- start 2024-03-30T00:44:04.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f7200640 1 -- start start 2024-03-30T00:44:04.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f7200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2f815fda0 0x7fb2f80b2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f7200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb2f80b2850 0x7fb2f806efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2f8070ec0 0x7fb2f806f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f806e5f0 con 0x7fb2f8070ec0 2024-03-30T00:44:04.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f7200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f806ebb0 con 0x7fb2f80b2850 2024-03-30T00:44:04.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f7200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f806a350 con 0x7fb2f815fda0 2024-03-30T00:44:04.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f6800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2f8070ec0 0x7fb2f806f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f6800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2f8070ec0 0x7fb2f806f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33788/0 (socket says 172.21.15.66:33788) 2024-03-30T00:44:04.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f6800640 1 -- 172.21.15.66:0/649113553 learned_addr learned my addr 172.21.15.66:0/649113553 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:04.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2ed400640 1 --2- 172.21.15.66:0/649113553 >> v2:172.21.15.156:3300/0 conn(0x7fb2f80b2850 0x7fb2f806efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.819+0000 7fb2f5e00640 1 --2- 172.21.15.66:0/649113553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2f815fda0 0x7fb2f80b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f6800640 1 -- 172.21.15.66:0/649113553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2f815fda0 msgr2=0x7fb2f80b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:04.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f6800640 1 --2- 172.21.15.66:0/649113553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2f815fda0 0x7fb2f80b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f6800640 1 -- 172.21.15.66:0/649113553 >> v2:172.21.15.156:3300/0 conn(0x7fb2f80b2850 msgr2=0x7fb2f806efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:04.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f6800640 1 --2- 172.21.15.66:0/649113553 >> v2:172.21.15.156:3300/0 conn(0x7fb2f80b2850 0x7fb2f806efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f6800640 1 -- 172.21.15.66:0/649113553 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f806a630 con 0x7fb2f8070ec0 2024-03-30T00:44:04.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f6800640 1 --2- 172.21.15.66:0/649113553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2f8070ec0 0x7fb2f806f510 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80d0490 tx=0x7fb2f001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b48fb673bda2ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:04.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f5400640 1 -- 172.21.15.66:0/649113553 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f00a5750 con 0x7fb2f8070ec0 2024-03-30T00:44:04.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f5400640 1 -- 172.21.15.66:0/649113553 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f00a58b0 con 0x7fb2f8070ec0 2024-03-30T00:44:04.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f5400640 1 -- 172.21.15.66:0/649113553 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f00a5c40 con 0x7fb2f8070ec0 2024-03-30T00:44:04.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f7200640 1 -- 172.21.15.66:0/649113553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2f8070ec0 msgr2=0x7fb2f806f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f7200640 1 --2- 172.21.15.66:0/649113553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2f8070ec0 0x7fb2f806f510 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80d0490 tx=0x7fb2f001c9c0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f7200640 1 -- 172.21.15.66:0/649113553 shutdown_connections 2024-03-30T00:44:04.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f7200640 1 --2- 172.21.15.66:0/649113553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2f8070ec0 0x7fb2f806f510 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f7200640 1 --2- 172.21.15.66:0/649113553 >> v2:172.21.15.156:3300/0 conn(0x7fb2f80b2850 0x7fb2f806efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f7200640 1 --2- 172.21.15.66:0/649113553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2f815fda0 0x7fb2f80b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f7200640 1 -- 172.21.15.66:0/649113553 >> 172.21.15.66:0/649113553 conn(0x7fb2f806cf90 msgr2=0x7fb2f806d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:04.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f7200640 1 -- 172.21.15.66:0/649113553 shutdown_connections 2024-03-30T00:44:04.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.823+0000 7fb2f7200640 1 -- 172.21.15.66:0/649113553 wait complete. 2024-03-30T00:44:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f7200640 1 Processor -- start 2024-03-30T00:44:04.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f7200640 1 -- start start 2024-03-30T00:44:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f7200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2f8070ec0 0x7fb2f8156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2f80b2850 0x7fb2f814f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f7200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f815fda0 0x7fb2f8150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f8152fa0 con 0x7fb2f80b2850 2024-03-30T00:44:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f7200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f8151c60 con 0x7fb2f815fda0 2024-03-30T00:44:04.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f7200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f8150fe0 con 0x7fb2f8070ec0 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2ed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2f80b2850 0x7fb2f814f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2ed400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2f80b2850 0x7fb2f814f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33796/0 (socket says 172.21.15.66:33796) 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2ed400640 1 -- 172.21.15.66:0/2018347663 learned_addr learned my addr 172.21.15.66:0/2018347663 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f5e00640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2f8070ec0 0x7fb2f8156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f6800640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f815fda0 0x7fb2f8150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2ed400640 1 -- 172.21.15.66:0/2018347663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2f8070ec0 msgr2=0x7fb2f8156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2ed400640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2f8070ec0 0x7fb2f8156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2ed400640 1 -- 172.21.15.66:0/2018347663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f815fda0 msgr2=0x7fb2f8150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2ed400640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f815fda0 0x7fb2f8150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2ed400640 1 -- 172.21.15.66:0/2018347663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f0009b30 con 0x7fb2f80b2850 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f5e00640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2f8070ec0 0x7fb2f8156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2ed400640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2f80b2850 0x7fb2f814f270 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000e8b0 tx=0x7fb2e000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2eea00640 1 -- 172.21.15.66:0/2018347663 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e000c940 con 0x7fb2f80b2850 2024-03-30T00:44:04.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2eea00640 1 -- 172.21.15.66:0/2018347663 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e000caa0 con 0x7fb2f80b2850 2024-03-30T00:44:04.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f7200640 1 -- 172.21.15.66:0/2018347663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f81512c0 con 0x7fb2f80b2850 2024-03-30T00:44:04.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2eea00640 1 -- 172.21.15.66:0/2018347663 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e000ce30 con 0x7fb2f80b2850 2024-03-30T00:44:04.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.827+0000 7fb2f7200640 1 -- 172.21.15.66:0/2018347663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f8151880 con 0x7fb2f80b2850 2024-03-30T00:44:04.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.831+0000 7fb2f7200640 1 -- 172.21.15.66:0/2018347663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f8151da0 con 0x7fb2f80b2850 2024-03-30T00:44:04.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.831+0000 7fb2eea00640 1 -- 172.21.15.66:0/2018347663 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb2e0013470 con 0x7fb2f80b2850 2024-03-30T00:44:04.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.831+0000 7fb2eea00640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2d4066690 0x7fb2d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:04.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.831+0000 7fb2eea00640 1 -- 172.21.15.66:0/2018347663 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fb2e0084430 con 0x7fb2f80b2850 2024-03-30T00:44:04.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.831+0000 7fb2f5e00640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2d4066690 0x7fb2d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:04.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.831+0000 7fb2f5e00640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2d4066690 0x7fb2d4068b50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009b10 tx=0x7fb2e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:04.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:04.835+0000 7fb2eea00640 1 -- 172.21.15.66:0/2018347663 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb2f8151da0 con 0x7fb2f80b2850 2024-03-30T00:44:05.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.035+0000 7fb2f7200640 1 -- 172.21.15.66:0/2018347663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb2f8151da0 con 0x7fb2f80b2850 2024-03-30T00:44:05.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.663+0000 7fb2eea00640 1 -- 172.21.15.66:0/2018347663 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb2f8151da0 con 0x7fb2f80b2850 2024-03-30T00:44:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 -- 172.21.15.66:0/2018347663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2d4066690 msgr2=0x7fb2d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2d4066690 0x7fb2d4068b50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009b10 tx=0x7fb2e8008040 comp rx=0 tx=0).stop 2024-03-30T00:44:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 -- 172.21.15.66:0/2018347663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2f80b2850 msgr2=0x7fb2f814f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2f80b2850 0x7fb2f814f270 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000e8b0 tx=0x7fb2e000ed70 comp rx=0 tx=0).stop 2024-03-30T00:44:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 -- 172.21.15.66:0/2018347663 shutdown_connections 2024-03-30T00:44:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f815fda0 0x7fb2f8150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2d4066690 0x7fb2d4068b50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2f80b2850 0x7fb2f814f270 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 --2- 172.21.15.66:0/2018347663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2f8070ec0 0x7fb2f8156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 -- 172.21.15.66:0/2018347663 >> 172.21.15.66:0/2018347663 conn(0x7fb2f806cf90 msgr2=0x7fb2f806ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:05.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 -- 172.21.15.66:0/2018347663 shutdown_connections 2024-03-30T00:44:05.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.667+0000 7fb2f7200640 1 -- 172.21.15.66:0/2018347663 wait complete. 2024-03-30T00:44:05.698 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T00:44:05.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.839+0000 7fd58ee00640 1 Processor -- start 2024-03-30T00:44:05.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.839+0000 7fd58ee00640 1 -- start start 2024-03-30T00:44:05.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.839+0000 7fd58ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 0x7fd58815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:05.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd58ee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5880d0970 con 0x7fd5880cdf00 2024-03-30T00:44:05.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd58ee00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5880d0c50 con 0x7fd58806cb20 2024-03-30T00:44:05.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd58ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880d0f30 con 0x7fd58815c730 2024-03-30T00:44:05.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd58e400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd5880cdf00 0x7fd58806b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46306/0 (socket says 172.21.15.66:46306) 2024-03-30T00:44:05.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd58e400640 1 -- 172.21.15.66:0/3603430398 learned_addr learned my addr 172.21.15.66:0/3603430398 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:05.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd58da00640 1 --2- 172.21.15.66:0/3603430398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 0x7fd58815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd587e00640 1 -- 172.21.15.66:0/3603430398 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412358795 0 0) 0x7fd5880d0970 con 0x7fd5880cdf00 2024-03-30T00:44:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd587e00640 1 -- 172.21.15.66:0/3603430398 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd570003530 con 0x7fd5880cdf00 2024-03-30T00:44:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd587e00640 1 -- 172.21.15.66:0/3603430398 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398472889 0 0) 0x7fd5880d0c50 con 0x7fd58806cb20 2024-03-30T00:44:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd587e00640 1 -- 172.21.15.66:0/3603430398 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5880d0970 con 0x7fd58806cb20 2024-03-30T00:44:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd58da00640 1 -- 172.21.15.66:0/3603430398 >> v1:172.21.15.156:6789/0 conn(0x7fd58806cb20 legacy=0x7fd5880cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd58da00640 1 -- 172.21.15.66:0/3603430398 >> v1:172.21.15.66:6789/0 conn(0x7fd5880cdf00 legacy=0x7fd58806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:05.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.843+0000 7fd58da00640 1 -- 172.21.15.66:0/3603430398 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58806aea0 con 0x7fd58815c730 2024-03-30T00:44:05.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58da00640 1 --2- 172.21.15.66:0/3603430398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 0x7fd58815dc80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0097f0 tx=0x7fd57c01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2664de52b6ed2607 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:05.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd587e00640 1 -- 172.21.15.66:0/3603430398 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0ac620 con 0x7fd58815c730 2024-03-30T00:44:05.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd587e00640 1 -- 172.21.15.66:0/3603430398 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c0ac780 con 0x7fd58815c730 2024-03-30T00:44:05.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd587e00640 1 -- 172.21.15.66:0/3603430398 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0acb10 con 0x7fd58815c730 2024-03-30T00:44:05.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 -- 172.21.15.66:0/3603430398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 msgr2=0x7fd58815dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 --2- 172.21.15.66:0/3603430398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 0x7fd58815dc80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0097f0 tx=0x7fd57c01c4b0 comp rx=0 tx=0).stop 2024-03-30T00:44:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 -- 172.21.15.66:0/3603430398 shutdown_connections 2024-03-30T00:44:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 --2- 172.21.15.66:0/3603430398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 0x7fd58815dc80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 -- 172.21.15.66:0/3603430398 >> 172.21.15.66:0/3603430398 conn(0x7fd5880b1e80 msgr2=0x7fd5880b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 -- 172.21.15.66:0/3603430398 shutdown_connections 2024-03-30T00:44:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 -- 172.21.15.66:0/3603430398 wait complete. 2024-03-30T00:44:05.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 Processor -- start 2024-03-30T00:44:05.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 -- start start 2024-03-30T00:44:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58806cb20 0x7fd58814b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880cdf00 0x7fd588148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 0x7fd5880c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:05.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd588148e80 con 0x7fd5880cdf00 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880b2ec0 con 0x7fd58806cb20 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880b31a0 con 0x7fd58815c730 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880cdf00 0x7fd588148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880cdf00 0x7fd588148940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33810/0 (socket says 172.21.15.66:33810) 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58d000640 1 -- 172.21.15.66:0/1238042095 learned_addr learned my addr 172.21.15.66:0/1238042095 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58da00640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58806cb20 0x7fd58814b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.847+0000 7fd58e400640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 0x7fd5880c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd58d000640 1 -- 172.21.15.66:0/1238042095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 msgr2=0x7fd5880c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd58d000640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 0x7fd5880c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd58d000640 1 -- 172.21.15.66:0/1238042095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58806cb20 msgr2=0x7fd58814b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd58d000640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58806cb20 0x7fd58814b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd58d000640 1 -- 172.21.15.66:0/1238042095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c009420 con 0x7fd5880cdf00 2024-03-30T00:44:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd58e400640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 0x7fd5880c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd58d000640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880cdf00 0x7fd588148940 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c490 tx=0x7fd57400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd585600640 1 -- 172.21.15.66:0/1238042095 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5740052d0 con 0x7fd5880cdf00 2024-03-30T00:44:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd585600640 1 -- 172.21.15.66:0/1238042095 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd574005430 con 0x7fd5880cdf00 2024-03-30T00:44:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd58ee00640 1 -- 172.21.15.66:0/1238042095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5880ce8a0 con 0x7fd5880cdf00 2024-03-30T00:44:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd585600640 1 -- 172.21.15.66:0/1238042095 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574012870 con 0x7fd5880cdf00 2024-03-30T00:44:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd58ee00640 1 -- 172.21.15.66:0/1238042095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd588151da0 con 0x7fd5880cdf00 2024-03-30T00:44:05.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd58ee00640 1 -- 172.21.15.66:0/1238042095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd588067150 con 0x7fd5880cdf00 2024-03-30T00:44:05.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd585600640 1 -- 172.21.15.66:0/1238042095 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd5740129d0 con 0x7fd5880cdf00 2024-03-30T00:44:05.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd585600640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd568066690 0x7fd568068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:05.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd585600640 1 -- 172.21.15.66:0/1238042095 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fd574083f50 con 0x7fd5880cdf00 2024-03-30T00:44:05.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.851+0000 7fd58da00640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd568066690 0x7fd568068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:05.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.855+0000 7fd58da00640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd568066690 0x7fd568068b50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0ae000 tx=0x7fd57c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:05.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:05.855+0000 7fd585600640 1 -- 172.21.15.66:0/1238042095 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd574050ff0 con 0x7fd5880cdf00 2024-03-30T00:44:06.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.019+0000 7fd58ee00640 1 -- 172.21.15.66:0/1238042095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd588073cf0 con 0x7fd5880cdf00 2024-03-30T00:44:06.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.667+0000 7fd585600640 1 -- 172.21.15.66:0/1238042095 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd574054610 con 0x7fd5880cdf00 2024-03-30T00:44:06.674 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-03-30T00:44:06.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.667+0000 7fd58ee00640 1 -- 172.21.15.66:0/1238042095 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd568066690 msgr2=0x7fd568068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:06.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.667+0000 7fd58ee00640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd568066690 0x7fd568068b50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0ae000 tx=0x7fd57c0a7000 comp rx=0 tx=0).stop 2024-03-30T00:44:06.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.667+0000 7fd58ee00640 1 -- 172.21.15.66:0/1238042095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880cdf00 msgr2=0x7fd588148940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:06.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.667+0000 7fd58ee00640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880cdf00 0x7fd588148940 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c490 tx=0x7fd57400c950 comp rx=0 tx=0).stop 2024-03-30T00:44:06.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.671+0000 7fd58ee00640 1 -- 172.21.15.66:0/1238042095 shutdown_connections 2024-03-30T00:44:06.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.671+0000 7fd58ee00640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd58815c730 0x7fd5880c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:06.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.671+0000 7fd58ee00640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd568066690 0x7fd568068b50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:06.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.671+0000 7fd58ee00640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5880cdf00 0x7fd588148940 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:06.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.671+0000 7fd58ee00640 1 --2- 172.21.15.66:0/1238042095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58806cb20 0x7fd58814b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:06.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.671+0000 7fd58ee00640 1 -- 172.21.15.66:0/1238042095 >> 172.21.15.66:0/1238042095 conn(0x7fd5880b1e80 msgr2=0x7fd5880b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:06.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.671+0000 7fd58ee00640 1 -- 172.21.15.66:0/1238042095 shutdown_connections 2024-03-30T00:44:06.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.671+0000 7fd58ee00640 1 -- 172.21.15.66:0/1238042095 wait complete. 2024-03-30T00:44:06.701 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T00:44:06.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.839+0000 7eff7d600640 1 Processor -- start 2024-03-30T00:44:06.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.839+0000 7eff7d600640 1 -- start start 2024-03-30T00:44:06.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff7d600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7eff7806b190 0x7eff78098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff7d600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7eff780cce10 0x7eff780cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff7d600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff78099390 con 0x7eff7815c9b0 2024-03-30T00:44:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff7d600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff780cd800 con 0x7eff7806b190 2024-03-30T00:44:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff7d600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff78099670 con 0x7eff780cce10 2024-03-30T00:44:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff6e000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7eff7806b190 0x7eff78098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff6e000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7eff7806b190 0x7eff78098a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33816/0 (socket says 172.21.15.66:33816) 2024-03-30T00:44:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff6e000640 1 -- 172.21.15.66:0/2936848004 learned_addr learned my addr 172.21.15.66:0/2936848004 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff77400640 1 --2- 172.21.15.66:0/2936848004 >> v2:172.21.15.156:3300/0 conn(0x7eff780cce10 0x7eff780cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff6e000640 1 -- 172.21.15.66:0/2936848004 >> v2:172.21.15.156:3300/0 conn(0x7eff780cce10 msgr2=0x7eff780cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff6e000640 1 --2- 172.21.15.66:0/2936848004 >> v2:172.21.15.156:3300/0 conn(0x7eff780cce10 0x7eff780cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff6e000640 1 -- 172.21.15.66:0/2936848004 >> v1:172.21.15.66:6789/0 conn(0x7eff7815c9b0 legacy=0x7eff7806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:06.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff6e000640 1 -- 172.21.15.66:0/2936848004 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78099950 con 0x7eff7806b190 2024-03-30T00:44:06.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.843+0000 7eff77400640 1 --2- 172.21.15.66:0/2936848004 >> v2:172.21.15.156:3300/0 conn(0x7eff780cce10 0x7eff780cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:06.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff6e000640 1 --2- 172.21.15.66:0/2936848004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7806b190 0x7eff78098a20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7eff7809a090 tx=0x7eff6001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d4bffe59bf7388 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:06.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff76000640 1 -- 172.21.15.66:0/2936848004 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600a8a10 con 0x7eff7806b190 2024-03-30T00:44:06.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff76000640 1 -- 172.21.15.66:0/2936848004 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff600a8b70 con 0x7eff7806b190 2024-03-30T00:44:06.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff76000640 1 -- 172.21.15.66:0/2936848004 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600b0440 con 0x7eff7806b190 2024-03-30T00:44:06.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 -- 172.21.15.66:0/2936848004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7806b190 msgr2=0x7eff78098a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:06.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 --2- 172.21.15.66:0/2936848004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7806b190 0x7eff78098a20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7eff7809a090 tx=0x7eff6001ed10 comp rx=0 tx=0).stop 2024-03-30T00:44:06.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 -- 172.21.15.66:0/2936848004 shutdown_connections 2024-03-30T00:44:06.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 --2- 172.21.15.66:0/2936848004 >> v2:172.21.15.156:3300/0 conn(0x7eff780cce10 0x7eff780cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:06.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 --2- 172.21.15.66:0/2936848004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7806b190 0x7eff78098a20 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:06.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 -- 172.21.15.66:0/2936848004 >> 172.21.15.66:0/2936848004 conn(0x7eff780d0590 msgr2=0x7eff780d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:06.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 -- 172.21.15.66:0/2936848004 shutdown_connections 2024-03-30T00:44:06.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 -- 172.21.15.66:0/2936848004 wait complete. 2024-03-30T00:44:06.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 Processor -- start 2024-03-30T00:44:06.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 -- start start 2024-03-30T00:44:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7806b190 0x7eff7815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff780cce10 0x7eff7815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff7815c9b0 0x7eff7814d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff780c5880 con 0x7eff7806b190 2024-03-30T00:44:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff781f23c0 con 0x7eff780cce10 2024-03-30T00:44:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff7d600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff780c4c70 con 0x7eff7815c9b0 2024-03-30T00:44:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.847+0000 7eff76a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7806b190 0x7eff7815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff76a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7806b190 0x7eff7815fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33822/0 (socket says 172.21.15.66:33822) 2024-03-30T00:44:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff76a00640 1 -- 172.21.15.66:0/1715871097 learned_addr learned my addr 172.21.15.66:0/1715871097 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff77400640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff7815c9b0 0x7eff7814d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:06.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff6e000640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff780cce10 0x7eff7815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:06.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff76a00640 1 -- 172.21.15.66:0/1715871097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff7815c9b0 msgr2=0x7eff7814d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:06.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff76a00640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff7815c9b0 0x7eff7814d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:06.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff76a00640 1 -- 172.21.15.66:0/1715871097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff780cce10 msgr2=0x7eff7815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:06.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff76a00640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff780cce10 0x7eff7815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:06.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff76a00640 1 -- 172.21.15.66:0/1715871097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600094d0 con 0x7eff7806b190 2024-03-30T00:44:06.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff76a00640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7806b190 0x7eff7815fc90 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7eff6800c3c0 tx=0x7eff6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:06.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff6f400640 1 -- 172.21.15.66:0/1715871097 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680052d0 con 0x7eff7806b190 2024-03-30T00:44:06.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff6f400640 1 -- 172.21.15.66:0/1715871097 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff68005430 con 0x7eff7806b190 2024-03-30T00:44:06.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff7d600640 1 -- 172.21.15.66:0/1715871097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff780c4f50 con 0x7eff7806b190 2024-03-30T00:44:06.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff6f400640 1 -- 172.21.15.66:0/1715871097 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68003840 con 0x7eff7806b190 2024-03-30T00:44:06.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff7d600640 1 -- 172.21.15.66:0/1715871097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff780c5510 con 0x7eff7806b190 2024-03-30T00:44:06.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff7d600640 1 -- 172.21.15.66:0/1715871097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff780b1890 con 0x7eff7806b190 2024-03-30T00:44:06.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.851+0000 7eff6f400640 1 -- 172.21.15.66:0/1715871097 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7eff680039a0 con 0x7eff7806b190 2024-03-30T00:44:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.855+0000 7eff6f400640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7eff54066690 0x7eff54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.855+0000 7eff6f400640 1 -- 172.21.15.66:0/1715871097 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5793+0+0 (secure 0 0 0) 0x7eff68083600 con 0x7eff7806b190 2024-03-30T00:44:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.855+0000 7eff6e000640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7eff54066690 0x7eff54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.855+0000 7eff6e000640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7eff54066690 0x7eff54068b50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7eff600b2000 tx=0x7eff60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:06.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:06.859+0000 7eff6f400640 1 -- 172.21.15.66:0/1715871097 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7eff68050a30 con 0x7eff7806b190 2024-03-30T00:44:07.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.019+0000 7eff7d600640 1 -- 172.21.15.66:0/1715871097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7eff78066c30 con 0x7eff7806b190 2024-03-30T00:44:07.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.675+0000 7eff6f400640 1 -- 172.21.15.66:0/1715871097 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) v1 ==== 169+0+0 (secure 0 0 0) 0x7eff68054050 con 0x7eff7806b190 2024-03-30T00:44:07.684 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-03-30T00:44:07.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 -- 172.21.15.66:0/1715871097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7eff54066690 msgr2=0x7eff54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:07.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7eff54066690 0x7eff54068b50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7eff600b2000 tx=0x7eff60009040 comp rx=0 tx=0).stop 2024-03-30T00:44:07.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 -- 172.21.15.66:0/1715871097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7806b190 msgr2=0x7eff7815fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:07.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7806b190 0x7eff7815fc90 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7eff6800c3c0 tx=0x7eff6800c880 comp rx=0 tx=0).stop 2024-03-30T00:44:07.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 -- 172.21.15.66:0/1715871097 shutdown_connections 2024-03-30T00:44:07.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff7815c9b0 0x7eff7814d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:07.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff780cce10 0x7eff7815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:07.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7eff54066690 0x7eff54068b50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:07.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 --2- 172.21.15.66:0/1715871097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7806b190 0x7eff7815fc90 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:07.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 -- 172.21.15.66:0/1715871097 >> 172.21.15.66:0/1715871097 conn(0x7eff780d0590 msgr2=0x7eff780cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:07.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 -- 172.21.15.66:0/1715871097 shutdown_connections 2024-03-30T00:44:07.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.679+0000 7eff7d600640 1 -- 172.21.15.66:0/1715871097 wait complete. 2024-03-30T00:44:07.709 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-30T00:44:07.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c03800640 1 Processor -- start 2024-03-30T00:44:07.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c03800640 1 -- start start 2024-03-30T00:44:07.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c03800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6bfc15ca80 0x7f6bfc06b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:07.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c03800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bfc0cf2f0 0x7f6bfc06b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:07.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c03800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc0cb0e0 con 0x7f6bfc06bb30 2024-03-30T00:44:07.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c03800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc0cf7f0 con 0x7f6bfc15ca80 2024-03-30T00:44:07.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c03800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0cfad0 con 0x7f6bfc0cf2f0 2024-03-30T00:44:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c00e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6bfc15ca80 0x7f6bfc06b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6bfbe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bfc0cf2f0 0x7f6bfc06b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c01800640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f6bfc06bb30 0x7f6bfc098a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:55722/0 (socket says 172.21.15.66:55722) 2024-03-30T00:44:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c01800640 1 -- 172.21.15.66:0/2715109156 learned_addr learned my addr 172.21.15.66:0/2715109156 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6bfb400640 1 -- 172.21.15.66:0/2715109156 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991417809 0 0) 0x7f6bfc0cb0e0 con 0x7f6bfc06bb30 2024-03-30T00:44:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6bfb400640 1 -- 172.21.15.66:0/2715109156 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6be4003530 con 0x7f6bfc06bb30 2024-03-30T00:44:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c00e00640 1 -- 172.21.15.66:0/2715109156 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bfc0cf2f0 msgr2=0x7f6bfc06b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c00e00640 1 --2- 172.21.15.66:0/2715109156 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bfc0cf2f0 0x7f6bfc06b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c00e00640 1 -- 172.21.15.66:0/2715109156 >> v1:172.21.15.156:6789/0 conn(0x7f6bfc06bb30 legacy=0x7f6bfc098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6c00e00640 1 -- 172.21.15.66:0/2715109156 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0cad70 con 0x7f6bfc15ca80 2024-03-30T00:44:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.847+0000 7f6bfbe00640 1 --2- 172.21.15.66:0/2715109156 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bfc0cf2f0 0x7f6bfc06b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6c00e00640 1 --2- 172.21.15.66:0/2715109156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc15ca80 0x7f6bfc06b0b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00097c0 tx=0x7f6bf001ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29124b063bae6f29 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:07.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6bfb400640 1 -- 172.21.15.66:0/2715109156 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00a8990 con 0x7f6bfc15ca80 2024-03-30T00:44:07.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6bfb400640 1 -- 172.21.15.66:0/2715109156 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf00a8af0 con 0x7f6bfc15ca80 2024-03-30T00:44:07.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6bfb400640 1 -- 172.21.15.66:0/2715109156 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00a8e80 con 0x7f6bfc15ca80 2024-03-30T00:44:07.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6c03800640 1 -- 172.21.15.66:0/2715109156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc15ca80 msgr2=0x7f6bfc06b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:07.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6c03800640 1 --2- 172.21.15.66:0/2715109156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc15ca80 0x7f6bfc06b0b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00097c0 tx=0x7f6bf001ec90 comp rx=0 tx=0).stop 2024-03-30T00:44:07.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6c03800640 1 -- 172.21.15.66:0/2715109156 shutdown_connections 2024-03-30T00:44:07.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6c03800640 1 --2- 172.21.15.66:0/2715109156 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bfc0cf2f0 0x7f6bfc06b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:07.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6c03800640 1 --2- 172.21.15.66:0/2715109156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc15ca80 0x7f6bfc06b0b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:07.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6c03800640 1 -- 172.21.15.66:0/2715109156 >> 172.21.15.66:0/2715109156 conn(0x7f6bfc0b2090 msgr2=0x7f6bfc0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:07.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6c03800640 1 -- 172.21.15.66:0/2715109156 shutdown_connections 2024-03-30T00:44:07.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.851+0000 7f6c03800640 1 -- 172.21.15.66:0/2715109156 wait complete. 2024-03-30T00:44:07.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c03800640 1 Processor -- start 2024-03-30T00:44:07.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c03800640 1 -- start start 2024-03-30T00:44:07.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c03800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bfc06bb30 0x7f6bfc0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:07.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c03800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bfc0cf2f0 0x7f6bfc1f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:07.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c03800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc15ca80 0x7f6bfc15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:07.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c03800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c8860 con 0x7f6bfc06bb30 2024-03-30T00:44:07.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c03800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc1f2e50 con 0x7f6bfc15ca80 2024-03-30T00:44:07.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c03800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c7c50 con 0x7f6bfc0cf2f0 2024-03-30T00:44:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c00e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bfc06bb30 0x7f6bfc0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c00e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bfc06bb30 0x7f6bfc0c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33832/0 (socket says 172.21.15.66:33832) 2024-03-30T00:44:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c00e00640 1 -- 172.21.15.66:0/534877882 learned_addr learned my addr 172.21.15.66:0/534877882 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6bfbe00640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bfc0cf2f0 0x7f6bfc1f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c01800640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc15ca80 0x7f6bfc15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c00e00640 1 -- 172.21.15.66:0/534877882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bfc0cf2f0 msgr2=0x7f6bfc1f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c00e00640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bfc0cf2f0 0x7f6bfc1f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c00e00640 1 -- 172.21.15.66:0/534877882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc15ca80 msgr2=0x7f6bfc15a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c00e00640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc15ca80 0x7f6bfc15a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c00e00640 1 -- 172.21.15.66:0/534877882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be8009420 con 0x7f6bfc06bb30 2024-03-30T00:44:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c00e00640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bfc06bb30 0x7f6bfc0c7670 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00b3040 tx=0x7f6bf001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6bf8c00640 1 -- 172.21.15.66:0/534877882 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00b0d80 con 0x7f6bfc06bb30 2024-03-30T00:44:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6bf8c00640 1 -- 172.21.15.66:0/534877882 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf00b9430 con 0x7f6bfc06bb30 2024-03-30T00:44:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6bf8c00640 1 -- 172.21.15.66:0/534877882 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00b97e0 con 0x7f6bfc06bb30 2024-03-30T00:44:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c03800640 1 -- 172.21.15.66:0/534877882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf0009420 con 0x7f6bfc06bb30 2024-03-30T00:44:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.855+0000 7f6c03800640 1 -- 172.21.15.66:0/534877882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc0c8210 con 0x7f6bfc06bb30 2024-03-30T00:44:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.859+0000 7f6c03800640 1 -- 172.21.15.66:0/534877882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc066c30 con 0x7f6bfc06bb30 2024-03-30T00:44:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.859+0000 7f6bf8c00640 1 -- 172.21.15.66:0/534877882 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6bf00b7040 con 0x7f6bfc06bb30 2024-03-30T00:44:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.859+0000 7f6bf8c00640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bdc0666c0 0x7f6bdc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.859+0000 7f6bf8c00640 1 -- 172.21.15.66:0/534877882 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f6bf01218c0 con 0x7f6bfc06bb30 2024-03-30T00:44:07.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.859+0000 7f6bfbe00640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bdc0666c0 0x7f6bdc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:07.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.859+0000 7f6bfbe00640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bdc0666c0 0x7f6bdc068b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009cf0 tx=0x7f6be8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:07.863+0000 7f6bf8c00640 1 -- 172.21.15.66:0/534877882 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6bf00eee00 con 0x7f6bfc06bb30 2024-03-30T00:44:08.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.027+0000 7f6c03800640 1 -- 172.21.15.66:0/534877882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6bfc0b2bb0 con 0x7f6bfc06bb30 2024-03-30T00:44:08.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.031+0000 7f6bf8c00640 1 -- 172.21.15.66:0/534877882 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6bf00b50a0 con 0x7f6bfc06bb30 2024-03-30T00:44:08.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 -- 172.21.15.66:0/534877882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bdc0666c0 msgr2=0x7f6bdc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bdc0666c0 0x7f6bdc068b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009cf0 tx=0x7f6be8009190 comp rx=0 tx=0).stop 2024-03-30T00:44:08.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 -- 172.21.15.66:0/534877882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bfc06bb30 msgr2=0x7f6bfc0c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bfc06bb30 0x7f6bfc0c7670 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00b3040 tx=0x7f6bf001ba30 comp rx=0 tx=0).stop 2024-03-30T00:44:08.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 -- 172.21.15.66:0/534877882 shutdown_connections 2024-03-30T00:44:08.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc15ca80 0x7f6bfc15a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bfc0cf2f0 0x7f6bfc1f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bdc0666c0 0x7f6bdc068b80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 --2- 172.21.15.66:0/534877882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bfc06bb30 0x7f6bfc0c7670 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 -- 172.21.15.66:0/534877882 >> 172.21.15.66:0/534877882 conn(0x7f6bfc0b2090 msgr2=0x7f6bfc15b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:08.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 -- 172.21.15.66:0/534877882 shutdown_connections 2024-03-30T00:44:08.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.035+0000 7f6c03800640 1 -- 172.21.15.66:0/534877882 wait complete. 2024-03-30T00:44:08.061 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-03-30T00:44:08.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c4400640 1 Processor -- start 2024-03-30T00:44:08.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c4400640 1 -- start start 2024-03-30T00:44:08.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c4400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa7bc0cec30 0x7fa7bc159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c4400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0cd3e0 con 0x7fa7bc0cda90 2024-03-30T00:44:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c4400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc15a990 con 0x7fa7bc0cde70 2024-03-30T00:44:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c4400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7bc0b1b80 con 0x7fa7bc0cec30 2024-03-30T00:44:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c1a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa7bc0cda90 0x7fa7bc0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46324/0 (socket says 172.21.15.66:46324) 2024-03-30T00:44:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c1a00640 1 -- 172.21.15.66:0/3380158430 learned_addr learned my addr 172.21.15.66:0/3380158430 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c1000640 1 --2- 172.21.15.66:0/3380158430 >> v2:172.21.15.156:3300/0 conn(0x7fa7bc0cec30 0x7fa7bc159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249048696 0 0) 0x7fa7bc0cd3e0 con 0x7fa7bc0cda90 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a4003530 con 0x7fa7bc0cda90 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906942536 0 0) 0x7fa7bc15a990 con 0x7fa7bc0cde70 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7bc0cd3e0 con 0x7fa7bc0cde70 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3404381210 0 0) 0x7fa7a4003530 con 0x7fa7bc0cda90 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 >> v2:172.21.15.156:3300/0 conn(0x7fa7bc0cec30 msgr2=0x7fa7bc159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 --2- 172.21.15.66:0/3380158430 >> v2:172.21.15.156:3300/0 conn(0x7fa7bc0cec30 0x7fa7bc159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 >> v1:172.21.15.156:6789/0 conn(0x7fa7bc0cde70 legacy=0x7fa7bc15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc0b11e0 con 0x7fa7bc0cda90 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fa7b0001680 con 0x7fa7bc0cda90 2024-03-30T00:44:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c1000640 1 --2- 172.21.15.66:0/3380158430 >> v2:172.21.15.156:3300/0 conn(0x7fa7bc0cec30 0x7fa7bc159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 >> v1:172.21.15.66:6789/0 conn(0x7fa7bc0cda90 legacy=0x7fa7bc0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 --2- 172.21.15.66:0/3380158430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7a4003b10 0x7fa7a4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 --2- 172.21.15.66:0/3380158430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a40047d0 0x7fa7a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 --2- 172.21.15.66:0/3380158430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a4005190 0x7fa7a400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0b1b80 con 0x7fa7a4005190 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a400e2d0 con 0x7fa7a40047d0 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a400e490 con 0x7fa7a4003b10 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c2400640 1 --2- 172.21.15.66:0/3380158430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a4005190 0x7fa7a400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c1a00640 1 --2- 172.21.15.66:0/3380158430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7a4003b10 0x7fa7a4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c1000640 1 --2- 172.21.15.66:0/3380158430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a40047d0 0x7fa7a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c2400640 1 -- 172.21.15.66:0/3380158430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7a4003b10 msgr2=0x7fa7a4003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c2400640 1 --2- 172.21.15.66:0/3380158430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7a4003b10 0x7fa7a4003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c2400640 1 -- 172.21.15.66:0/3380158430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a40047d0 msgr2=0x7fa7a4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c2400640 1 --2- 172.21.15.66:0/3380158430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a40047d0 0x7fa7a4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.195+0000 7fa7c2400640 1 -- 172.21.15.66:0/3380158430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a4003530 con 0x7fa7a4005190 2024-03-30T00:44:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7c2400640 1 -- 172.21.15.66:0/3380158430 reap_dead start 2024-03-30T00:44:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7c2400640 1 --2- 172.21.15.66:0/3380158430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a4005190 0x7fa7a400b990 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4001fd0 tx=0x7fa7b401f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6871b56760b419cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40a8c90 con 0x7fa7a4005190 2024-03-30T00:44:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b40a8df0 con 0x7fa7a4005190 2024-03-30T00:44:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7bbe00640 1 -- 172.21.15.66:0/3380158430 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4003bc0 con 0x7fa7a4005190 2024-03-30T00:44:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7c4400640 1 -- 172.21.15.66:0/3380158430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a4005190 msgr2=0x7fa7a400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7c4400640 1 --2- 172.21.15.66:0/3380158430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a4005190 0x7fa7a400b990 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4001fd0 tx=0x7fa7b401f070 comp rx=0 tx=0).stop 2024-03-30T00:44:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7c4400640 1 -- 172.21.15.66:0/3380158430 shutdown_connections 2024-03-30T00:44:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7c4400640 1 --2- 172.21.15.66:0/3380158430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7a4005190 0x7fa7a400b990 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7c4400640 1 -- 172.21.15.66:0/3380158430 >> 172.21.15.66:0/3380158430 conn(0x7fa7bc069f60 msgr2=0x7fa7bc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7c4400640 1 -- 172.21.15.66:0/3380158430 shutdown_connections 2024-03-30T00:44:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.199+0000 7fa7c4400640 1 -- 172.21.15.66:0/3380158430 wait complete. 2024-03-30T00:44:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c4400640 1 Processor -- start 2024-03-30T00:44:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c4400640 1 -- start start 2024-03-30T00:44:08.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c4400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a4005190 0x7fa7bc155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c4400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7bc06fed0 0x7fa7bc0c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c4400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7bc071600 0x7fa7bc071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c4400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc071010 con 0x7fa7bc06fed0 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c4400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc071ff0 con 0x7fa7a4005190 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c4400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc071430 con 0x7fa7bc071600 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7bc06fed0 0x7fa7bc0c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7bc06fed0 0x7fa7bc0c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33856/0 (socket says 172.21.15.66:33856) 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c1000640 1 -- 172.21.15.66:0/1945868370 learned_addr learned my addr 172.21.15.66:0/1945868370 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c2400640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7bc071600 0x7fa7bc071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c1a00640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a4005190 0x7fa7bc155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c1000640 1 -- 172.21.15.66:0/1945868370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7bc071600 msgr2=0x7fa7bc071ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c1000640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7bc071600 0x7fa7bc071ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c1000640 1 -- 172.21.15.66:0/1945868370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a4005190 msgr2=0x7fa7bc155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c1000640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a4005190 0x7fa7bc155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c1000640 1 -- 172.21.15.66:0/1945868370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc156660 con 0x7fa7bc06fed0 2024-03-30T00:44:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c1000640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7bc06fed0 0x7fa7bc0c4530 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8011a80 tx=0x7fa7a8011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7b9600640 1 -- 172.21.15.66:0/1945868370 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800fc70 con 0x7fa7bc06fed0 2024-03-30T00:44:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7b9600640 1 -- 172.21.15.66:0/1945868370 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a800fdd0 con 0x7fa7bc06fed0 2024-03-30T00:44:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7b9600640 1 -- 172.21.15.66:0/1945868370 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8014680 con 0x7fa7bc06fed0 2024-03-30T00:44:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c4400640 1 -- 172.21.15.66:0/1945868370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc0c4a70 con 0x7fa7bc06fed0 2024-03-30T00:44:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.203+0000 7fa7c4400640 1 -- 172.21.15.66:0/1945868370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc070670 con 0x7fa7bc06fed0 2024-03-30T00:44:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.207+0000 7fa7c4400640 1 -- 172.21.15.66:0/1945868370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc066c30 con 0x7fa7bc06fed0 2024-03-30T00:44:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.207+0000 7fa7b9600640 1 -- 172.21.15.66:0/1945868370 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa7a80147e0 con 0x7fa7bc06fed0 2024-03-30T00:44:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.207+0000 7fa7b9600640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa79c066690 0x7fa79c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.207+0000 7fa7b9600640 1 -- 172.21.15.66:0/1945868370 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa7a80847e0 con 0x7fa7bc06fed0 2024-03-30T00:44:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.207+0000 7fa7c1a00640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa79c066690 0x7fa79c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.207+0000 7fa7c1a00640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa79c066690 0x7fa79c068b50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0ca830 tx=0x7fa7b0001e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.211+0000 7fa7b9600640 1 -- 172.21.15.66:0/1945868370 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa7a8051db0 con 0x7fa7bc06fed0 2024-03-30T00:44:08.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.431+0000 7fa7c4400640 1 -- 172.21.15.66:0/1945868370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fa7bc15cb40 con 0x7fa7bc06fed0 2024-03-30T00:44:08.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.675+0000 7fa7b9600640 1 -- 172.21.15.66:0/1945868370 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v239) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa7a80220f0 con 0x7fa7bc06fed0 2024-03-30T00:44:08.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 -- 172.21.15.66:0/1945868370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa79c066690 msgr2=0x7fa79c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa79c066690 0x7fa79c068b50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0ca830 tx=0x7fa7b0001e10 comp rx=0 tx=0).stop 2024-03-30T00:44:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 -- 172.21.15.66:0/1945868370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7bc06fed0 msgr2=0x7fa7bc0c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7bc06fed0 0x7fa7bc0c4530 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8011a80 tx=0x7fa7a8011f40 comp rx=0 tx=0).stop 2024-03-30T00:44:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 -- 172.21.15.66:0/1945868370 shutdown_connections 2024-03-30T00:44:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7bc071600 0x7fa7bc071ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa79c066690 0x7fa79c068b50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7bc06fed0 0x7fa7bc0c4530 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 --2- 172.21.15.66:0/1945868370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a4005190 0x7fa7bc155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 -- 172.21.15.66:0/1945868370 >> 172.21.15.66:0/1945868370 conn(0x7fa7bc069f60 msgr2=0x7fa7bc0c0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 -- 172.21.15.66:0/1945868370 shutdown_connections 2024-03-30T00:44:08.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.679+0000 7fa7c4400640 1 -- 172.21.15.66:0/1945868370 wait complete. 2024-03-30T00:44:08.707 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-30T00:44:08.707 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-30T00:44:08.707 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-30T00:44:08.707 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:08.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.839+0000 7f971e400640 1 Processor -- start 2024-03-30T00:44:08.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.839+0000 7f971e400640 1 -- start start 2024-03-30T00:44:08.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f971815cca0 0x7f97180cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971e400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f97180cdf90 0x7f97180d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971e400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97180d0840 0x7f9718160720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971e400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971815d080 con 0x7f971815cca0 2024-03-30T00:44:08.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971e400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97180d0c90 con 0x7f97180cdf90 2024-03-30T00:44:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971e400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97180cb4a0 con 0x7f97180d0840 2024-03-30T00:44:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971d000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f971815cca0 0x7f97180cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971d000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f971815cca0 0x7f97180cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60488/0 (socket says 172.21.15.66:60488) 2024-03-30T00:44:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971d000640 1 -- 172.21.15.66:0/1167175657 learned_addr learned my addr 172.21.15.66:0/1167175657 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971da00640 1 --2- 172.21.15.66:0/1167175657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97180d0840 0x7f9718160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f9717e00640 1 --2- 172.21.15.66:0/1167175657 >> v2:172.21.15.156:3300/0 conn(0x7f97180cdf90 0x7f97180d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971d000640 1 -- 172.21.15.66:0/1167175657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97180d0840 msgr2=0x7f9718160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971d000640 1 --2- 172.21.15.66:0/1167175657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97180d0840 0x7f9718160720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971d000640 1 -- 172.21.15.66:0/1167175657 >> v2:172.21.15.156:3300/0 conn(0x7f97180cdf90 msgr2=0x7f97180d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971d000640 1 --2- 172.21.15.66:0/1167175657 >> v2:172.21.15.156:3300/0 conn(0x7f97180cdf90 0x7f97180d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971d000640 1 -- 172.21.15.66:0/1167175657 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97180cb780 con 0x7f971815cca0 2024-03-30T00:44:08.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f971d000640 1 --2- 172.21.15.66:0/1167175657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f971815cca0 0x7f97180cab50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9718071f10 tx=0x7f970c01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6716881c5468a81 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:08.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f9717400640 1 -- 172.21.15.66:0/1167175657 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c0ab600 con 0x7f971815cca0 2024-03-30T00:44:08.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f9717400640 1 -- 172.21.15.66:0/1167175657 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c0a7050 con 0x7f971815cca0 2024-03-30T00:44:08.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.843+0000 7f9717400640 1 -- 172.21.15.66:0/1167175657 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c0ab980 con 0x7f971815cca0 2024-03-30T00:44:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 -- 172.21.15.66:0/1167175657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f971815cca0 msgr2=0x7f97180cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 --2- 172.21.15.66:0/1167175657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f971815cca0 0x7f97180cab50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9718071f10 tx=0x7f970c01c230 comp rx=0 tx=0).stop 2024-03-30T00:44:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 -- 172.21.15.66:0/1167175657 shutdown_connections 2024-03-30T00:44:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 --2- 172.21.15.66:0/1167175657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97180d0840 0x7f9718160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 --2- 172.21.15.66:0/1167175657 >> v2:172.21.15.156:3300/0 conn(0x7f97180cdf90 0x7f97180d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 --2- 172.21.15.66:0/1167175657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f971815cca0 0x7f97180cab50 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 -- 172.21.15.66:0/1167175657 >> 172.21.15.66:0/1167175657 conn(0x7f971806e0f0 msgr2=0x7f971806e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:08.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 -- 172.21.15.66:0/1167175657 shutdown_connections 2024-03-30T00:44:08.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 -- 172.21.15.66:0/1167175657 wait complete. 2024-03-30T00:44:08.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 Processor -- start 2024-03-30T00:44:08.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 -- start start 2024-03-30T00:44:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97180cdf90 0x7f97180b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97180d0840 0x7f971815d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971815cca0 0x7f971815d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97180b38f0 con 0x7f97180d0840 2024-03-30T00:44:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97180b3b70 con 0x7f97180cdf90 2024-03-30T00:44:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971e400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97180b3e50 con 0x7f971815cca0 2024-03-30T00:44:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f9717e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97180d0840 0x7f971815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f9717e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97180d0840 0x7f971815d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60492/0 (socket says 172.21.15.66:60492) 2024-03-30T00:44:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f9717e00640 1 -- 172.21.15.66:0/339345723 learned_addr learned my addr 172.21.15.66:0/339345723 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971da00640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971815cca0 0x7f971815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971d000640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97180cdf90 0x7f97180b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f9717e00640 1 -- 172.21.15.66:0/339345723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971815cca0 msgr2=0x7f971815d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f9717e00640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971815cca0 0x7f971815d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f9717e00640 1 -- 172.21.15.66:0/339345723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97180cdf90 msgr2=0x7f97180b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f9717e00640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97180cdf90 0x7f97180b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f9717e00640 1 -- 172.21.15.66:0/339345723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c0094d0 con 0x7f97180d0840 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.847+0000 7f971d000640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97180cdf90 0x7f97180b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.851+0000 7f9717e00640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97180d0840 0x7f971815d220 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f970400d800 tx=0x7f970400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.851+0000 7f9714c00640 1 -- 172.21.15.66:0/339345723 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97040096a0 con 0x7f97180d0840 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.851+0000 7f9714c00640 1 -- 172.21.15.66:0/339345723 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9704009800 con 0x7f97180d0840 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.851+0000 7f9714c00640 1 -- 172.21.15.66:0/339345723 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704009b70 con 0x7f97180d0840 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.851+0000 7f971e400640 1 -- 172.21.15.66:0/339345723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97180b4130 con 0x7f97180d0840 2024-03-30T00:44:08.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.851+0000 7f971e400640 1 -- 172.21.15.66:0/339345723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97180b46f0 con 0x7f97180d0840 2024-03-30T00:44:08.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.851+0000 7f971e400640 1 -- 172.21.15.66:0/339345723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9718073cf0 con 0x7f97180d0840 2024-03-30T00:44:08.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.851+0000 7f9714c00640 1 -- 172.21.15.66:0/339345723 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9704009cd0 con 0x7f97180d0840 2024-03-30T00:44:08.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.851+0000 7f9714c00640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96f8066690 0x7f96f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:08.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.851+0000 7f9714c00640 1 -- 172.21.15.66:0/339345723 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f97040848f0 con 0x7f97180d0840 2024-03-30T00:44:08.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.851+0000 7f971d000640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96f8066690 0x7f96f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:08.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.855+0000 7f971d000640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96f8066690 0x7f96f8068b50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f970c0ad000 tx=0x7f970c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:08.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:08.855+0000 7f9714c00640 1 -- 172.21.15.66:0/339345723 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9704051cf0 con 0x7f97180d0840 2024-03-30T00:44:09.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 -- 172.21.15.66:0/339345723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97180b6340 con 0x7f97180d0840 2024-03-30T00:44:09.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f9714c00640 1 -- 172.21.15.66:0/339345723 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+12726 (secure 0 0 0) 0x7f9704055310 con 0x7f97180d0840 2024-03-30T00:44:09.026 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:09.026 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":73,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:07.667780+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.156:6835/1137593870":"2024-03-31T00:44:04.645758+0000","172.21.15.156:6833/1137593870":"2024-03-31T00:44:04.645758+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:09.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 -- 172.21.15.66:0/339345723 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96f8066690 msgr2=0x7f96f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96f8066690 0x7f96f8068b50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f970c0ad000 tx=0x7f970c008040 comp rx=0 tx=0).stop 2024-03-30T00:44:09.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 -- 172.21.15.66:0/339345723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97180d0840 msgr2=0x7f971815d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97180d0840 0x7f971815d220 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f970400d800 tx=0x7f970400dcc0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 -- 172.21.15.66:0/339345723 shutdown_connections 2024-03-30T00:44:09.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971815cca0 0x7f971815d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96f8066690 0x7f96f8068b50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97180d0840 0x7f971815d220 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 --2- 172.21.15.66:0/339345723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97180cdf90 0x7f97180b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 -- 172.21.15.66:0/339345723 >> 172.21.15.66:0/339345723 conn(0x7f971806e0f0 msgr2=0x7f971806c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:09.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 -- 172.21.15.66:0/339345723 shutdown_connections 2024-03-30T00:44:09.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.019+0000 7f971e400640 1 -- 172.21.15.66:0/339345723 wait complete. 2024-03-30T00:44:09.047 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:44:09.047 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:44:09.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e17c00640 1 Processor -- start 2024-03-30T00:44:09.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e17c00640 1 -- start start 2024-03-30T00:44:09.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e17c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2e100ce520 0x7f2e100ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e17c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1006d070 con 0x7f2e100cabb0 2024-03-30T00:44:09.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e17c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e100cbba0 con 0x7f2e1015a640 2024-03-30T00:44:09.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e17c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e100cee70 con 0x7f2e100ce520 2024-03-30T00:44:09.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e15c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2e100cabb0 0x7f2e100cafe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:38636/0 (socket says 172.21.15.66:38636) 2024-03-30T00:44:09.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e15c00640 1 -- 172.21.15.66:0/3704295538 learned_addr learned my addr 172.21.15.66:0/3704295538 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:09.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e07e00640 1 --2- 172.21.15.66:0/3704295538 >> v2:172.21.15.156:3300/0 conn(0x7f2e100ce520 0x7f2e100ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097689419 0 0) 0x7f2e1006d070 con 0x7f2e100cabb0 2024-03-30T00:44:09.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2df4003530 con 0x7f2e100cabb0 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3479463823 0 0) 0x7f2df4003530 con 0x7f2e100cabb0 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 >> v2:172.21.15.156:3300/0 conn(0x7f2e100ce520 msgr2=0x7f2e100ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 --2- 172.21.15.66:0/3704295538 >> v2:172.21.15.156:3300/0 conn(0x7f2e100ce520 0x7f2e100ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 >> v1:172.21.15.156:6789/0 conn(0x7f2e1015a640 legacy=0x7f2e1006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e100cb8c0 con 0x7f2e100cabb0 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f2e08001a00 con 0x7f2e100cabb0 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 >> v1:172.21.15.66:6789/0 conn(0x7f2e100cabb0 legacy=0x7f2e100cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df4003980 0x7f2df4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df40047d0 0x7f2df4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2df4005190 0x7f2df400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e100cee70 con 0x7f2df40047d0 2024-03-30T00:44:09.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df400c030 con 0x7f2df4003980 2024-03-30T00:44:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df400c1f0 con 0x7f2df4005190 2024-03-30T00:44:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.183+0000 7f2e07e00640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df40047d0 0x7f2df4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e15200640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df4003980 0x7f2df4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e15c00640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2df4005190 0x7f2df400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e07e00640 1 -- 172.21.15.66:0/3704295538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2df4005190 msgr2=0x7f2df400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e07e00640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2df4005190 0x7f2df400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e07e00640 1 -- 172.21.15.66:0/3704295538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df4003980 msgr2=0x7f2df4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e07e00640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df4003980 0x7f2df4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e07e00640 1 -- 172.21.15.66:0/3704295538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df4003530 con 0x7f2df40047d0 2024-03-30T00:44:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e15c00640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2df4005190 0x7f2df400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e15c00640 1 -- 172.21.15.66:0/3704295538 reap_dead start 2024-03-30T00:44:09.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e07e00640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df40047d0 0x7f2df4004c50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009ae0 tx=0x7f2df801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9321b782c8e6c15 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:09.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80a5660 con 0x7f2df40047d0 2024-03-30T00:44:09.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df80a57c0 con 0x7f2df40047d0 2024-03-30T00:44:09.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e0fe00640 1 -- 172.21.15.66:0/3704295538 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80a5b70 con 0x7f2df40047d0 2024-03-30T00:44:09.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e17c00640 1 -- 172.21.15.66:0/3704295538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df40047d0 msgr2=0x7f2df4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.187+0000 7f2e17c00640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df40047d0 0x7f2df4004c50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009ae0 tx=0x7f2df801ca70 comp rx=0 tx=0).stop 2024-03-30T00:44:09.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 -- 172.21.15.66:0/3704295538 shutdown_connections 2024-03-30T00:44:09.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 --2- 172.21.15.66:0/3704295538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df40047d0 0x7f2df4004c50 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 -- 172.21.15.66:0/3704295538 >> 172.21.15.66:0/3704295538 conn(0x7f2e10069f60 msgr2=0x7f2e1006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:09.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 -- 172.21.15.66:0/3704295538 shutdown_connections 2024-03-30T00:44:09.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 -- 172.21.15.66:0/3704295538 wait complete. 2024-03-30T00:44:09.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 Processor -- start 2024-03-30T00:44:09.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 -- start start 2024-03-30T00:44:09.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df40047d0 0x7f2e101f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e100c7e70 0x7f2e1009b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e100ad5f0 0x7f2e100ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e101f43d0 con 0x7f2e100ad5f0 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1015c120 con 0x7f2df40047d0 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e17c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e100adfb0 con 0x7f2e100c7e70 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e15c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e100ad5f0 0x7f2e100ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e15c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e100ad5f0 0x7f2e100ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60506/0 (socket says 172.21.15.66:60506) 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e15c00640 1 -- 172.21.15.66:0/347965617 learned_addr learned my addr 172.21.15.66:0/347965617 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e07e00640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e100c7e70 0x7f2e1009b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e15200640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df40047d0 0x7f2e101f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e15c00640 1 -- 172.21.15.66:0/347965617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e100c7e70 msgr2=0x7f2e1009b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e15c00640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e100c7e70 0x7f2e1009b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e15c00640 1 -- 172.21.15.66:0/347965617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df40047d0 msgr2=0x7f2e101f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e15c00640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df40047d0 0x7f2e101f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e15c00640 1 -- 172.21.15.66:0/347965617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e100c82f0 con 0x7f2e100ad5f0 2024-03-30T00:44:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e07e00640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e100c7e70 0x7f2e1009b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e15200640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df40047d0 0x7f2e101f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e15c00640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e100ad5f0 0x7f2e100ada70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2e100acfe0 tx=0x7f2e08011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e0d600640 1 -- 172.21.15.66:0/347965617 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800f8a0 con 0x7f2e100ad5f0 2024-03-30T00:44:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.191+0000 7f2e0d600640 1 -- 172.21.15.66:0/347965617 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0800fa00 con 0x7f2e100ad5f0 2024-03-30T00:44:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.195+0000 7f2e0d600640 1 -- 172.21.15.66:0/347965617 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800fd70 con 0x7f2e100ad5f0 2024-03-30T00:44:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.195+0000 7f2e17c00640 1 -- 172.21.15.66:0/347965617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e101f3b30 con 0x7f2e100ad5f0 2024-03-30T00:44:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.195+0000 7f2e17c00640 1 -- 172.21.15.66:0/347965617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e101f40f0 con 0x7f2e100ad5f0 2024-03-30T00:44:09.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.195+0000 7f2e17c00640 1 -- 172.21.15.66:0/347965617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1015be80 con 0x7f2e100ad5f0 2024-03-30T00:44:09.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.195+0000 7f2e0d600640 1 -- 172.21.15.66:0/347965617 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2e0801f460 con 0x7f2e100ad5f0 2024-03-30T00:44:09.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.195+0000 7f2e0d600640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2dec0666c0 0x7f2dec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.195+0000 7f2e0d600640 1 -- 172.21.15.66:0/347965617 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2e08086f80 con 0x7f2e100ad5f0 2024-03-30T00:44:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.195+0000 7f2e15200640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2dec0666c0 0x7f2dec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.195+0000 7f2e15200640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2dec0666c0 0x7f2dec068b80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e00007fd0 tx=0x7f2e00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.199+0000 7f2e0d600640 1 -- 172.21.15.66:0/347965617 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2e1015be80 con 0x7f2e100ad5f0 2024-03-30T00:44:09.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 -- 172.21.15.66:0/347965617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e1015c260 con 0x7f2e100ad5f0 2024-03-30T00:44:09.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e0d600640 1 -- 172.21.15.66:0/347965617 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e1015c260 con 0x7f2e100ad5f0 2024-03-30T00:44:09.370 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:44:09.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 -- 172.21.15.66:0/347965617 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2dec0666c0 msgr2=0x7f2dec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2dec0666c0 0x7f2dec068b80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e00007fd0 tx=0x7f2e00009040 comp rx=0 tx=0).stop 2024-03-30T00:44:09.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 -- 172.21.15.66:0/347965617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e100ad5f0 msgr2=0x7f2e100ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e100ad5f0 0x7f2e100ada70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2e100acfe0 tx=0x7f2e08011c20 comp rx=0 tx=0).stop 2024-03-30T00:44:09.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 -- 172.21.15.66:0/347965617 shutdown_connections 2024-03-30T00:44:09.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2dec0666c0 0x7f2dec068b80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e100ad5f0 0x7f2e100ada70 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e100c7e70 0x7f2e1009b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 --2- 172.21.15.66:0/347965617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df40047d0 0x7f2e101f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 -- 172.21.15.66:0/347965617 >> 172.21.15.66:0/347965617 conn(0x7f2e10069f60 msgr2=0x7f2e100ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:09.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 -- 172.21.15.66:0/347965617 shutdown_connections 2024-03-30T00:44:09.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.363+0000 7f2e17c00640 1 -- 172.21.15.66:0/347965617 wait complete. 2024-03-30T00:44:09.392 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-03-30T00:44:09.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f3471000640 1 Processor -- start 2024-03-30T00:44:09.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f3471000640 1 -- start start 2024-03-30T00:44:09.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f3471000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f346c0cd2d0 0x7f346c0c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f3471000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f346c0ce980 0x7f346c0cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f3471000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346c0ccfe0 con 0x7f346c0ce520 2024-03-30T00:44:09.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f3471000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346c0cbec0 con 0x7f346c0ce980 2024-03-30T00:44:09.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f3471000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c0c8e80 con 0x7f346c0cd2d0 2024-03-30T00:44:09.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f346a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f346c0ce980 0x7f346c0cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f346a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f346c0ce980 0x7f346c0cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60518/0 (socket says 172.21.15.66:60518) 2024-03-30T00:44:09.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f346a800640 1 -- 172.21.15.66:0/3814385647 learned_addr learned my addr 172.21.15.66:0/3814385647 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:09.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f346a800640 1 -- 172.21.15.66:0/3814385647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f346c0cd2d0 msgr2=0x7f346c0c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:09.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f3469e00640 1 --2- 172.21.15.66:0/3814385647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f346c0cd2d0 0x7f346c0c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f346a800640 1 --2- 172.21.15.66:0/3814385647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f346c0cd2d0 0x7f346c0c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f346a800640 1 -- 172.21.15.66:0/3814385647 >> v1:172.21.15.156:6789/0 conn(0x7f346c0ce520 legacy=0x7f346c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:09.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f346a800640 1 -- 172.21.15.66:0/3814385647 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c0ca660 con 0x7f346c0ce980 2024-03-30T00:44:09.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.527+0000 7f3469e00640 1 --2- 172.21.15.66:0/3814385647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f346c0cd2d0 0x7f346c0c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:09.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f346a800640 1 --2- 172.21.15.66:0/3814385647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f346c0ce980 0x7f346c0cad00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f346400d150 tx=0x7f346401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a19e0b0e9773b3e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:09.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f3469400640 1 -- 172.21.15.66:0/3814385647 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346400b2d0 con 0x7f346c0ce980 2024-03-30T00:44:09.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f3469400640 1 -- 172.21.15.66:0/3814385647 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346400b430 con 0x7f346c0ce980 2024-03-30T00:44:09.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f3469400640 1 -- 172.21.15.66:0/3814385647 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640a7b80 con 0x7f346c0ce980 2024-03-30T00:44:09.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f3471000640 1 -- 172.21.15.66:0/3814385647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f346c0ce980 msgr2=0x7f346c0cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f3471000640 1 --2- 172.21.15.66:0/3814385647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f346c0ce980 0x7f346c0cad00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f346400d150 tx=0x7f346401ebc0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f3471000640 1 -- 172.21.15.66:0/3814385647 shutdown_connections 2024-03-30T00:44:09.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f3471000640 1 --2- 172.21.15.66:0/3814385647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f346c0ce980 0x7f346c0cad00 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f3471000640 1 --2- 172.21.15.66:0/3814385647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f346c0cd2d0 0x7f346c0c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f3471000640 1 -- 172.21.15.66:0/3814385647 >> 172.21.15.66:0/3814385647 conn(0x7f346c069f60 msgr2=0x7f346c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:09.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f3471000640 1 -- 172.21.15.66:0/3814385647 shutdown_connections 2024-03-30T00:44:09.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.531+0000 7f3471000640 1 -- 172.21.15.66:0/3814385647 wait complete. 2024-03-30T00:44:09.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3471000640 1 Processor -- start 2024-03-30T00:44:09.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3471000640 1 -- start start 2024-03-30T00:44:09.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3471000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f346c0cd2d0 0x7f346c0c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3471000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f346c0ce520 0x7f346c0c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3471000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f346c0ce980 0x7f346c0c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3471000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c158290 con 0x7f346c0ce980 2024-03-30T00:44:09.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3471000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c162270 con 0x7f346c0ce520 2024-03-30T00:44:09.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3471000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c161c90 con 0x7f346c0cd2d0 2024-03-30T00:44:09.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f346a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f346c0ce980 0x7f346c0c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f346a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f346c0ce980 0x7f346c0c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60528/0 (socket says 172.21.15.66:60528) 2024-03-30T00:44:09.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f346a800640 1 -- 172.21.15.66:0/1525317724 learned_addr learned my addr 172.21.15.66:0/1525317724 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:09.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3461400640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f346c0ce520 0x7f346c0c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3469e00640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f346c0cd2d0 0x7f346c0c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f346a800640 1 -- 172.21.15.66:0/1525317724 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f346c0cd2d0 msgr2=0x7f346c0c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f346a800640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f346c0cd2d0 0x7f346c0c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f346a800640 1 -- 172.21.15.66:0/1525317724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f346c0ce520 msgr2=0x7f346c0c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f346a800640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f346c0ce520 0x7f346c0c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f346a800640 1 -- 172.21.15.66:0/1525317724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346400bd40 con 0x7f346c0ce980 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f346a800640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f346c0ce980 0x7f346c0c46a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f34640b1040 tx=0x7f346401b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3462a00640 1 -- 172.21.15.66:0/1525317724 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3464009800 con 0x7f346c0ce980 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3462a00640 1 -- 172.21.15.66:0/1525317724 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3464009980 con 0x7f346c0ce980 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3462a00640 1 -- 172.21.15.66:0/1525317724 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3464009d30 con 0x7f346c0ce980 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3471000640 1 -- 172.21.15.66:0/1525317724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346c1584a0 con 0x7f346c0ce980 2024-03-30T00:44:09.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3471000640 1 -- 172.21.15.66:0/1525317724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346c158780 con 0x7f346c0ce980 2024-03-30T00:44:09.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.535+0000 7f3471000640 1 -- 172.21.15.66:0/1525317724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346c162420 con 0x7f346c0ce980 2024-03-30T00:44:09.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.539+0000 7f3462a00640 1 -- 172.21.15.66:0/1525317724 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f34640c3460 con 0x7f346c0ce980 2024-03-30T00:44:09.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.539+0000 7f3462a00640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34480666c0 0x7f3448068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:09.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.539+0000 7f3462a00640 1 -- 172.21.15.66:0/1525317724 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f346411f740 con 0x7f346c0ce980 2024-03-30T00:44:09.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.539+0000 7f3469e00640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34480666c0 0x7f3448068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:09.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.539+0000 7f3469e00640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34480666c0 0x7f3448068b80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f345c00ab60 tx=0x7f345c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:09.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.543+0000 7f3462a00640 1 -- 172.21.15.66:0/1525317724 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f346c162420 con 0x7f346c0ce980 2024-03-30T00:44:09.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:09.759+0000 7f3471000640 1 -- 172.21.15.66:0/1525317724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f346c162420 con 0x7f346c0ce980 2024-03-30T00:44:10.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.683+0000 7f3462a00640 1 -- 172.21.15.66:0/1525317724 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v241) v1 ==== 169+0+0 (secure 0 0 0) 0x7f346c162420 con 0x7f346c0ce980 2024-03-30T00:44:10.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 -- 172.21.15.66:0/1525317724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34480666c0 msgr2=0x7f3448068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:10.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34480666c0 0x7f3448068b80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f345c00ab60 tx=0x7f345c008040 comp rx=0 tx=0).stop 2024-03-30T00:44:10.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 -- 172.21.15.66:0/1525317724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f346c0ce980 msgr2=0x7f346c0c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:10.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f346c0ce980 0x7f346c0c46a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f34640b1040 tx=0x7f346401b960 comp rx=0 tx=0).stop 2024-03-30T00:44:10.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 -- 172.21.15.66:0/1525317724 shutdown_connections 2024-03-30T00:44:10.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f34480666c0 0x7f3448068b80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:10.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f346c0ce980 0x7f346c0c46a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:10.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f346c0ce520 0x7f346c0c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:10.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 --2- 172.21.15.66:0/1525317724 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f346c0cd2d0 0x7f346c0c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:10.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 -- 172.21.15.66:0/1525317724 >> 172.21.15.66:0/1525317724 conn(0x7f346c069f60 msgr2=0x7f346c06cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:10.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 -- 172.21.15.66:0/1525317724 shutdown_connections 2024-03-30T00:44:10.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.687+0000 7f3471000640 1 -- 172.21.15.66:0/1525317724 wait complete. 2024-03-30T00:44:10.718 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-30T00:44:10.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.859+0000 7f5fa2c00640 1 Processor -- start 2024-03-30T00:44:10.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.859+0000 7f5fa2c00640 1 -- start start 2024-03-30T00:44:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa2c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5f9c099430 0x7f5f9c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa2c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f9c0cf0c0 0x7f5f9c0c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa2c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9c0c4600 con 0x7f5f9c15a810 2024-03-30T00:44:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa2c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c0cc510 con 0x7f5f9c0cf0c0 2024-03-30T00:44:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa2c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c0cf4c0 con 0x7f5f9c099430 2024-03-30T00:44:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa0c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f9c0cf0c0 0x7f5f9c0c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa0c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f9c0cf0c0 0x7f5f9c0c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60532/0 (socket says 172.21.15.66:60532) 2024-03-30T00:44:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa0c00640 1 -- 172.21.15.66:0/707550889 learned_addr learned my addr 172.21.15.66:0/707550889 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5f93400640 1 --2- 172.21.15.66:0/707550889 >> v2:172.21.15.156:3300/0 conn(0x7f5f9c099430 0x7f5f9c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa0c00640 1 -- 172.21.15.66:0/707550889 >> v2:172.21.15.156:3300/0 conn(0x7f5f9c099430 msgr2=0x7f5f9c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa0c00640 1 --2- 172.21.15.66:0/707550889 >> v2:172.21.15.156:3300/0 conn(0x7f5f9c099430 0x7f5f9c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:10.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa0c00640 1 -- 172.21.15.66:0/707550889 >> v1:172.21.15.156:6789/0 conn(0x7f5f9c15a810 legacy=0x7f5f9c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:10.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.863+0000 7f5fa0c00640 1 -- 172.21.15.66:0/707550889 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0998c0 con 0x7f5f9c0cf0c0 2024-03-30T00:44:10.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa0c00640 1 --2- 172.21.15.66:0/707550889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c0cf0c0 0x7f5f9c0c4a90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0cab80 tx=0x7f5f9401f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3b6c553d9dd03f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:10.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5f9b400640 1 -- 172.21.15.66:0/707550889 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f940af400 con 0x7f5f9c0cf0c0 2024-03-30T00:44:10.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5f9b400640 1 -- 172.21.15.66:0/707550889 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f940ab050 con 0x7f5f9c0cf0c0 2024-03-30T00:44:10.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5f9b400640 1 -- 172.21.15.66:0/707550889 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f940af780 con 0x7f5f9c0cf0c0 2024-03-30T00:44:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 -- 172.21.15.66:0/707550889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c0cf0c0 msgr2=0x7f5f9c0c4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 --2- 172.21.15.66:0/707550889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c0cf0c0 0x7f5f9c0c4a90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0cab80 tx=0x7f5f9401f1a0 comp rx=0 tx=0).stop 2024-03-30T00:44:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 -- 172.21.15.66:0/707550889 shutdown_connections 2024-03-30T00:44:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 --2- 172.21.15.66:0/707550889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c0cf0c0 0x7f5f9c0c4a90 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 --2- 172.21.15.66:0/707550889 >> v2:172.21.15.156:3300/0 conn(0x7f5f9c099430 0x7f5f9c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 -- 172.21.15.66:0/707550889 >> 172.21.15.66:0/707550889 conn(0x7f5f9c0b2090 msgr2=0x7f5f9c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 -- 172.21.15.66:0/707550889 shutdown_connections 2024-03-30T00:44:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 -- 172.21.15.66:0/707550889 wait complete. 2024-03-30T00:44:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 Processor -- start 2024-03-30T00:44:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 -- start start 2024-03-30T00:44:10.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0cf0c0 0x7f5f9c1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c15a810 0x7f5f9c0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c0b3720 0x7f5f9c0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0c87a0 con 0x7f5f9c15a810 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0b5d10 con 0x7f5f9c0b3720 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.867+0000 7f5fa2c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0b5ff0 con 0x7f5f9c0cf0c0 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f93400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c15a810 0x7f5f9c0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f93400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c15a810 0x7f5f9c0b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60546/0 (socket says 172.21.15.66:60546) 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f93400640 1 -- 172.21.15.66:0/1399685376 learned_addr learned my addr 172.21.15.66:0/1399685376 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f9be00640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0cf0c0 0x7f5f9c1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5fa0c00640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c0b3720 0x7f5f9c0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f93400640 1 -- 172.21.15.66:0/1399685376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0cf0c0 msgr2=0x7f5f9c1555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f93400640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0cf0c0 0x7f5f9c1555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f93400640 1 -- 172.21.15.66:0/1399685376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c0b3720 msgr2=0x7f5f9c0c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f93400640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c0b3720 0x7f5f9c0c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f93400640 1 -- 172.21.15.66:0/1399685376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f94009b30 con 0x7f5f9c15a810 2024-03-30T00:44:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f93400640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c15a810 0x7f5f9c0b54c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400d5f0 tx=0x7f5f8400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f98c00640 1 -- 172.21.15.66:0/1399685376 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f840125f0 con 0x7f5f9c15a810 2024-03-30T00:44:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f98c00640 1 -- 172.21.15.66:0/1399685376 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f84012750 con 0x7f5f9c15a810 2024-03-30T00:44:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5fa2c00640 1 -- 172.21.15.66:0/1399685376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c0c8a20 con 0x7f5f9c15a810 2024-03-30T00:44:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f98c00640 1 -- 172.21.15.66:0/1399685376 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84012ae0 con 0x7f5f9c15a810 2024-03-30T00:44:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5fa2c00640 1 -- 172.21.15.66:0/1399685376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c0b5a00 con 0x7f5f9c15a810 2024-03-30T00:44:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5fa2c00640 1 -- 172.21.15.66:0/1399685376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c066c30 con 0x7f5f9c15a810 2024-03-30T00:44:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f98c00640 1 -- 172.21.15.66:0/1399685376 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5f84012c40 con 0x7f5f9c15a810 2024-03-30T00:44:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.871+0000 7f5f98c00640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 0x7f5f78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.875+0000 7f5f98c00640 1 -- 172.21.15.66:0/1399685376 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f5f84084550 con 0x7f5f9c15a810 2024-03-30T00:44:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.875+0000 7f5f9be00640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 0x7f5f78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.875+0000 7f5f9be00640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 0x7f5f78068b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0b2af0 tx=0x7f5f8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:10.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:10.879+0000 7f5f98c00640 1 -- 172.21.15.66:0/1399685376 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5f840519e0 con 0x7f5f9c15a810 2024-03-30T00:44:11.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.039+0000 7f5fa2c00640 1 -- 172.21.15.66:0/1399685376 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5f9c0b2bb0 con 0x7f5f9c15a810 2024-03-30T00:44:11.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5f98c00640 1 -- 172.21.15.66:0/1399685376 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5f84007380 con 0x7f5f9c15a810 2024-03-30T00:44:11.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 -- 172.21.15.66:0/1399685376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 msgr2=0x7f5f78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:11.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 0x7f5f78068b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0b2af0 tx=0x7f5f8c009040 comp rx=0 tx=0).stop 2024-03-30T00:44:11.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 -- 172.21.15.66:0/1399685376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c15a810 msgr2=0x7f5f9c0b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:11.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c15a810 0x7f5f9c0b54c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400d5f0 tx=0x7f5f8400dab0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 -- 172.21.15.66:0/1399685376 shutdown_connections 2024-03-30T00:44:11.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c0b3720 0x7f5f9c0c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f78066690 0x7f5f78068b50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f9c15a810 0x7f5f9c0b54c0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 --2- 172.21.15.66:0/1399685376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0cf0c0 0x7f5f9c1555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 -- 172.21.15.66:0/1399685376 >> 172.21.15.66:0/1399685376 conn(0x7f5f9c0b2090 msgr2=0x7f5f9c0c4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:11.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 -- 172.21.15.66:0/1399685376 shutdown_connections 2024-03-30T00:44:11.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.043+0000 7f5fa2c00640 1 -- 172.21.15.66:0/1399685376 wait complete. 2024-03-30T00:44:11.071 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:44:11.071 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:44:11.151 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:44:11.210 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:44:11.210 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:44:11.210 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:44:11.219 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:44:11.219 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:44:11.219 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:44:11.229 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:44:11.229 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:44:11.229 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:44:11.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:44:11.268 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:44:11.268 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:44:11.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:44:11.279 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-30T00:44:11.279 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:44:11.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f600e400640 1 Processor -- start 2024-03-30T00:44:11.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f600e400640 1 -- start start 2024-03-30T00:44:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f600e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f600806e800 0x7f60080d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f600e400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600806f8e0 0x7f60080ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f600e400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600806b0b0 con 0x7f60080cc560 2024-03-30T00:44:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f600e400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6008160640 con 0x7f600806e800 2024-03-30T00:44:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f600e400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600806b390 con 0x7f600806f8e0 2024-03-30T00:44:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f6007400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f60080cc560 0x7f600806e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:38646/0 (socket says 172.21.15.66:38646) 2024-03-30T00:44:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f6007400640 1 -- 172.21.15.66:0/1513187110 learned_addr learned my addr 172.21.15.66:0/1513187110 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f5ffea00640 1 --2- 172.21.15.66:0/1513187110 >> v2:172.21.15.66:3300/0 conn(0x7f600806e800 0x7f60080d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f6006a00640 1 -- 172.21.15.66:0/1513187110 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617789236 0 0) 0x7f600806b0b0 con 0x7f60080cc560 2024-03-30T00:44:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f6006a00640 1 -- 172.21.15.66:0/1513187110 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fec003530 con 0x7f60080cc560 2024-03-30T00:44:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f5ffea00640 1 -- 172.21.15.66:0/1513187110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600806f8e0 msgr2=0x7f60080ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f5ffea00640 1 --2- 172.21.15.66:0/1513187110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600806f8e0 0x7f60080ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f5ffea00640 1 -- 172.21.15.66:0/1513187110 >> v1:172.21.15.66:6789/0 conn(0x7f60080cc560 legacy=0x7f600806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.447+0000 7f5ffea00640 1 -- 172.21.15.66:0/1513187110 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600806b670 con 0x7f600806e800 2024-03-30T00:44:11.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f5ffea00640 1 --2- 172.21.15.66:0/1513187110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f600806e800 0x7f60080d1060 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00097c0 tx=0x7f5ff001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee69f40522b6edd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:11.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f6006a00640 1 -- 172.21.15.66:0/1513187110 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00aa5f0 con 0x7f600806e800 2024-03-30T00:44:11.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f6006a00640 1 -- 172.21.15.66:0/1513187110 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff00aa750 con 0x7f600806e800 2024-03-30T00:44:11.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f6006a00640 1 -- 172.21.15.66:0/1513187110 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00aaae0 con 0x7f600806e800 2024-03-30T00:44:11.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f600e400640 1 -- 172.21.15.66:0/1513187110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f600806e800 msgr2=0x7f60080d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:11.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f600e400640 1 --2- 172.21.15.66:0/1513187110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f600806e800 0x7f60080d1060 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00097c0 tx=0x7f5ff001c470 comp rx=0 tx=0).stop 2024-03-30T00:44:11.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f600e400640 1 -- 172.21.15.66:0/1513187110 shutdown_connections 2024-03-30T00:44:11.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f600e400640 1 --2- 172.21.15.66:0/1513187110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600806f8e0 0x7f60080ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f600e400640 1 --2- 172.21.15.66:0/1513187110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f600806e800 0x7f60080d1060 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f600e400640 1 -- 172.21.15.66:0/1513187110 >> 172.21.15.66:0/1513187110 conn(0x7f60080d0540 msgr2=0x7f60080d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:11.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f600e400640 1 -- 172.21.15.66:0/1513187110 shutdown_connections 2024-03-30T00:44:11.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.451+0000 7f600e400640 1 -- 172.21.15.66:0/1513187110 wait complete. 2024-03-30T00:44:11.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f600e400640 1 Processor -- start 2024-03-30T00:44:11.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f600e400640 1 -- start start 2024-03-30T00:44:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f600e400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f600806e800 0x7f60080ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f600e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f600806f8e0 0x7f600815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f600e400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60080cc560 0x7f600815eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f600e400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600815d890 con 0x7f600806f8e0 2024-03-30T00:44:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f600e400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600815f060 con 0x7f600806e800 2024-03-30T00:44:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f600e400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600815da80 con 0x7f60080cc560 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5ffea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f600806f8e0 0x7f600815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5ffea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f600806f8e0 0x7f600815d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60560/0 (socket says 172.21.15.66:60560) 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5ffea00640 1 -- 172.21.15.66:0/508670087 learned_addr learned my addr 172.21.15.66:0/508670087 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f6007400640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f600806e800 0x7f60080ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f6007e00640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60080cc560 0x7f600815eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5ffea00640 1 -- 172.21.15.66:0/508670087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60080cc560 msgr2=0x7f600815eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5ffea00640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60080cc560 0x7f600815eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5ffea00640 1 -- 172.21.15.66:0/508670087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f600806e800 msgr2=0x7f60080ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5ffea00640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f600806e800 0x7f60080ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5ffea00640 1 -- 172.21.15.66:0/508670087 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff0009420 con 0x7f600806f8e0 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f6007e00640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60080cc560 0x7f600815eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f6007400640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f600806e800 0x7f60080ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5ffea00640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f600806f8e0 0x7f600815d350 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00ac040 tx=0x7f5ff001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5fffe00640 1 -- 172.21.15.66:0/508670087 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00be610 con 0x7f600806f8e0 2024-03-30T00:44:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5fffe00640 1 -- 172.21.15.66:0/508670087 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff00be790 con 0x7f600806f8e0 2024-03-30T00:44:11.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f600e400640 1 -- 172.21.15.66:0/508670087 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600815dd60 con 0x7f600806f8e0 2024-03-30T00:44:11.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f5fffe00640 1 -- 172.21.15.66:0/508670087 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00beb40 con 0x7f600806f8e0 2024-03-30T00:44:11.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f600e400640 1 -- 172.21.15.66:0/508670087 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600815e320 con 0x7f600806f8e0 2024-03-30T00:44:11.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.455+0000 7f600e400640 1 -- 172.21.15.66:0/508670087 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6008066c30 con 0x7f600806f8e0 2024-03-30T00:44:11.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.459+0000 7f5fffe00640 1 -- 172.21.15.66:0/508670087 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5ff00b4400 con 0x7f600806f8e0 2024-03-30T00:44:11.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.459+0000 7f5fffe00640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5fe4066690 0x7f5fe4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.459+0000 7f5fffe00640 1 -- 172.21.15.66:0/508670087 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f5ff00a7070 con 0x7f600806f8e0 2024-03-30T00:44:11.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.459+0000 7f6007400640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5fe4066690 0x7f5fe4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:11.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.459+0000 7f6007400640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5fe4066690 0x7f5fe4068b50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8005890 tx=0x7f5ff8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:11.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.463+0000 7f5fffe00640 1 -- 172.21.15.66:0/508670087 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5ff00ebc70 con 0x7f600806f8e0 2024-03-30T00:44:11.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.655+0000 7f600e400640 1 -- 172.21.15.66:0/508670087 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60080ad2f0 con 0x7f600806f8e0 2024-03-30T00:44:11.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f5fffe00640 1 -- 172.21.15.66:0/508670087 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f5ff00ef290 con 0x7f600806f8e0 2024-03-30T00:44:11.665 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:11.666 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4272961638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4272961638},{"type":"v1","addr":"172.21.15.156:6834","nonce":4272961638}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2144186622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2144186622},{"type":"v1","addr":"172.21.15.66:6835","nonce":2144186622}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/750308788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":750308788},{"type":"v1","addr":"172.21.15.66:6837","nonce":750308788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3239730573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3239730573},{"type":"v1","addr":"172.21.15.156:6835","nonce":3239730573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[]} 2024-03-30T00:44:11.666 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 37 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 -- 172.21.15.66:0/508670087 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5fe4066690 msgr2=0x7f5fe4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5fe4066690 0x7f5fe4068b50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8005890 tx=0x7f5ff8009040 comp rx=0 tx=0).stop 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 -- 172.21.15.66:0/508670087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f600806f8e0 msgr2=0x7f600815d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f600806f8e0 0x7f600815d350 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00ac040 tx=0x7f5ff001acb0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 -- 172.21.15.66:0/508670087 shutdown_connections 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60080cc560 0x7f600815eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5fe4066690 0x7f5fe4068b50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f600806f8e0 0x7f600815d350 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 --2- 172.21.15.66:0/508670087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f600806e800 0x7f60080ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 -- 172.21.15.66:0/508670087 >> 172.21.15.66:0/508670087 conn(0x7f60080d0540 msgr2=0x7f60080d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 -- 172.21.15.66:0/508670087 shutdown_connections 2024-03-30T00:44:11.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.659+0000 7f600e400640 1 -- 172.21.15.66:0/508670087 wait complete. 2024-03-30T00:44:11.686 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T00:44:11.686 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-30T00:44:11.687 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T00:44:11.687 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:44:11.687 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T00:44:11.687 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-30T00:44:11.687 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T00:44:11.687 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:44:11.687 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T00:44:11.688 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-30T00:44:11.688 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T00:44:11.688 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:44:11.688 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T00:44:11.688 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-30T00:44:11.688 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T00:44:11.688 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:44:11.700 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T00:44:11.700 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T00:44:11.702 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T00:44:11.702 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T00:44:11.703 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T00:44:11.704 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T00:44:11.705 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T00:44:11.705 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T00:44:11.750 INFO:tasks.ceph.mds.d:Started 2024-03-30T00:44:11.750 DEBUG:teuthology.parallel:result is None 2024-03-30T00:44:11.751 INFO:tasks.ceph.mds.a:Started 2024-03-30T00:44:11.751 DEBUG:teuthology.parallel:result is None 2024-03-30T00:44:11.752 INFO:tasks.ceph.mds.b:Started 2024-03-30T00:44:11.752 DEBUG:teuthology.parallel:result is None 2024-03-30T00:44:11.753 INFO:tasks.ceph.mds.c:Started 2024-03-30T00:44:11.753 DEBUG:teuthology.parallel:result is None 2024-03-30T00:44:11.753 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-30T00:44:11.837 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:44:11.839 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:44:11.840 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2024-03-30T00:44:11.840 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2024-03-30T00:44:11.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f0c200640 1 Processor -- start 2024-03-30T00:44:11.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f0c200640 1 -- start start 2024-03-30T00:44:11.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f0c200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0f0406e800 0x7f0f040d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f0c200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f0406f8e0 0x7f0f040ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f0c200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0406b0b0 con 0x7f0f040cc560 2024-03-30T00:44:11.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f0c200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f04160640 con 0x7f0f0406e800 2024-03-30T00:44:11.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f0c200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0406b390 con 0x7f0f0406f8e0 2024-03-30T00:44:11.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f08e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0f0406e800 0x7f0f040d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:11.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f08e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0f0406e800 0x7f0f040d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60602/0 (socket says 172.21.15.66:60602) 2024-03-30T00:44:11.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f08e00640 1 -- 172.21.15.66:0/2444276346 learned_addr learned my addr 172.21.15.66:0/2444276346 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:11.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f0a200640 1 --2- 172.21.15.66:0/2444276346 >> v2:172.21.15.156:3300/0 conn(0x7f0f0406f8e0 0x7f0f040ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:11.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f08e00640 1 -- 172.21.15.66:0/2444276346 >> v2:172.21.15.156:3300/0 conn(0x7f0f0406f8e0 msgr2=0x7f0f040ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:11.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f08e00640 1 --2- 172.21.15.66:0/2444276346 >> v2:172.21.15.156:3300/0 conn(0x7f0f0406f8e0 0x7f0f040ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f08e00640 1 -- 172.21.15.66:0/2444276346 >> v1:172.21.15.156:6789/0 conn(0x7f0f040cc560 legacy=0x7f0f0406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:11.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f08e00640 1 -- 172.21.15.66:0/2444276346 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0406b670 con 0x7f0f0406e800 2024-03-30T00:44:11.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.895+0000 7f0f0a200640 1 --2- 172.21.15.66:0/2444276346 >> v2:172.21.15.156:3300/0 conn(0x7f0f0406f8e0 0x7f0f040ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:11.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f08e00640 1 --2- 172.21.15.66:0/2444276346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f0406e800 0x7f0f040d1060 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0f04070b00 tx=0x7f0ef001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cdefdd634867d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:11.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f03e00640 1 -- 172.21.15.66:0/2444276346 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00aa670 con 0x7f0f0406e800 2024-03-30T00:44:11.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f03e00640 1 -- 172.21.15.66:0/2444276346 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef00aa7d0 con 0x7f0f0406e800 2024-03-30T00:44:11.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f03e00640 1 -- 172.21.15.66:0/2444276346 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00aab60 con 0x7f0f0406e800 2024-03-30T00:44:11.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f0c200640 1 -- 172.21.15.66:0/2444276346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f0406e800 msgr2=0x7f0f040d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:11.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f0c200640 1 --2- 172.21.15.66:0/2444276346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f0406e800 0x7f0f040d1060 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0f04070b00 tx=0x7f0ef001c4f0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f0c200640 1 -- 172.21.15.66:0/2444276346 shutdown_connections 2024-03-30T00:44:11.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f0c200640 1 --2- 172.21.15.66:0/2444276346 >> v2:172.21.15.156:3300/0 conn(0x7f0f0406f8e0 0x7f0f040ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f0c200640 1 --2- 172.21.15.66:0/2444276346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f0406e800 0x7f0f040d1060 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f0c200640 1 -- 172.21.15.66:0/2444276346 >> 172.21.15.66:0/2444276346 conn(0x7f0f040d0540 msgr2=0x7f0f040d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:11.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f0c200640 1 -- 172.21.15.66:0/2444276346 shutdown_connections 2024-03-30T00:44:11.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.899+0000 7f0f0c200640 1 -- 172.21.15.66:0/2444276346 wait complete. 2024-03-30T00:44:11.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f0c200640 1 Processor -- start 2024-03-30T00:44:11.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f0c200640 1 -- start start 2024-03-30T00:44:11.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f0c200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f0406e800 0x7f0f0415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f0c200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f0406f8e0 0x7f0f0406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f0c200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f040cc560 0x7f0f040c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f0c200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f04151580 con 0x7f0f0406e800 2024-03-30T00:44:11.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f0c200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0415e6a0 con 0x7f0f040cc560 2024-03-30T00:44:11.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f0c200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f04150f50 con 0x7f0f0406f8e0 2024-03-30T00:44:11.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f09800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f0406e800 0x7f0f0415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:11.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f09800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f0406e800 0x7f0f0415fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60612/0 (socket says 172.21.15.66:60612) 2024-03-30T00:44:11.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f09800640 1 -- 172.21.15.66:0/948796663 learned_addr learned my addr 172.21.15.66:0/948796663 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:11.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f08e00640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f0406f8e0 0x7f0f0406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:11.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f0a200640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f040cc560 0x7f0f040c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:11.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f09800640 1 -- 172.21.15.66:0/948796663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f0406f8e0 msgr2=0x7f0f0406ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:11.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f09800640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f0406f8e0 0x7f0f0406ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f09800640 1 -- 172.21.15.66:0/948796663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f040cc560 msgr2=0x7f0f040c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:11.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f09800640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f040cc560 0x7f0f040c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:11.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f09800640 1 -- 172.21.15.66:0/948796663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef00094d0 con 0x7f0f0406e800 2024-03-30T00:44:11.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f09800640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f0406e800 0x7f0f0415fa40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800c3c0 tx=0x7f0ef800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:11.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f01600640 1 -- 172.21.15.66:0/948796663 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef80052d0 con 0x7f0f0406e800 2024-03-30T00:44:11.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f01600640 1 -- 172.21.15.66:0/948796663 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef8005430 con 0x7f0f0406e800 2024-03-30T00:44:11.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f01600640 1 -- 172.21.15.66:0/948796663 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8003840 con 0x7f0f0406e800 2024-03-30T00:44:11.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f0c200640 1 -- 172.21.15.66:0/948796663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0415e940 con 0x7f0f0406e800 2024-03-30T00:44:11.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.903+0000 7f0f0c200640 1 -- 172.21.15.66:0/948796663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f04151210 con 0x7f0f0406e800 2024-03-30T00:44:11.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.907+0000 7f0f0c200640 1 -- 172.21.15.66:0/948796663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0415ee60 con 0x7f0f0406e800 2024-03-30T00:44:11.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.907+0000 7f0f01600640 1 -- 172.21.15.66:0/948796663 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0ef80039a0 con 0x7f0f0406e800 2024-03-30T00:44:11.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.907+0000 7f0f01600640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ee40666c0 0x7f0ee4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:11.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.907+0000 7f0f01600640 1 -- 172.21.15.66:0/948796663 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0ef8083440 con 0x7f0f0406e800 2024-03-30T00:44:11.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.907+0000 7f0f08e00640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ee40666c0 0x7f0ee4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:11.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.907+0000 7f0f08e00640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ee40666c0 0x7f0ee4068b80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00ae000 tx=0x7f0ef00b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:11.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:11.911+0000 7f0f01600640 1 -- 172.21.15.66:0/948796663 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0ef8050a10 con 0x7f0f0406e800 2024-03-30T00:44:12.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.087+0000 7f0f0c200640 1 -- 172.21.15.66:0/948796663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0f04066c30 con 0x7f0f0406e800 2024-03-30T00:44:12.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.859+0000 7f0f01600640 1 -- 172.21.15.66:0/948796663 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v78) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0ef8054030 con 0x7f0f0406e800 2024-03-30T00:44:12.866 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-03-30T00:44:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.859+0000 7f0f0c200640 1 -- 172.21.15.66:0/948796663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ee40666c0 msgr2=0x7f0ee4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:12.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.859+0000 7f0f0c200640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ee40666c0 0x7f0ee4068b80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00ae000 tx=0x7f0ef00b5040 comp rx=0 tx=0).stop 2024-03-30T00:44:12.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.859+0000 7f0f0c200640 1 -- 172.21.15.66:0/948796663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f0406e800 msgr2=0x7f0f0415fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:12.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.859+0000 7f0f0c200640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f0406e800 0x7f0f0415fa40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800c3c0 tx=0x7f0ef800c880 comp rx=0 tx=0).stop 2024-03-30T00:44:12.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.859+0000 7f0f0c200640 1 -- 172.21.15.66:0/948796663 shutdown_connections 2024-03-30T00:44:12.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.859+0000 7f0f0c200640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f040cc560 0x7f0f040c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:12.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.859+0000 7f0f0c200640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f0406f8e0 0x7f0f0406ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:12.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.859+0000 7f0f0c200640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ee40666c0 0x7f0ee4068b80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:12.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.859+0000 7f0f0c200640 1 --2- 172.21.15.66:0/948796663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f0406e800 0x7f0f0415fa40 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:12.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.859+0000 7f0f0c200640 1 -- 172.21.15.66:0/948796663 >> 172.21.15.66:0/948796663 conn(0x7f0f040d0540 msgr2=0x7f0f040ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:12.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.863+0000 7f0f0c200640 1 -- 172.21.15.66:0/948796663 shutdown_connections 2024-03-30T00:44:12.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:12.863+0000 7f0f0c200640 1 -- 172.21.15.66:0/948796663 wait complete. 2024-03-30T00:44:12.892 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-30T00:44:13.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff34c200640 1 Processor -- start 2024-03-30T00:44:13.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff34c200640 1 -- start start 2024-03-30T00:44:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff34c200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff3440cca40 0x7ff3440cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff34c200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff34406b140 0x7ff34406b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff34c200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34406bac0 con 0x7ff344099250 2024-03-30T00:44:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff34c200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff344098ac0 con 0x7ff3440cca40 2024-03-30T00:44:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff34c200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440cf3f0 con 0x7ff34406b140 2024-03-30T00:44:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff349800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff3440cca40 0x7ff3440cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff349800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff3440cca40 0x7ff3440cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60618/0 (socket says 172.21.15.66:60618) 2024-03-30T00:44:13.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff349800640 1 -- 172.21.15.66:0/1736426035 learned_addr learned my addr 172.21.15.66:0/1736426035 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:13.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff348e00640 1 --2- 172.21.15.66:0/1736426035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff34406b140 0x7ff34406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff349800640 1 -- 172.21.15.66:0/1736426035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff34406b140 msgr2=0x7ff34406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:13.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff349800640 1 --2- 172.21.15.66:0/1736426035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff34406b140 0x7ff34406b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff349800640 1 -- 172.21.15.66:0/1736426035 >> v1:172.21.15.156:6789/0 conn(0x7ff344099250 legacy=0x7ff3440996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:13.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff349800640 1 -- 172.21.15.66:0/1736426035 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff344099db0 con 0x7ff3440cca40 2024-03-30T00:44:13.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.031+0000 7ff348e00640 1 --2- 172.21.15.66:0/1736426035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff34406b140 0x7ff34406b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:13.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff349800640 1 --2- 172.21.15.66:0/1736426035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3440cca40 0x7ff3440cac90 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff338009870 tx=0x7ff33801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bb683c10849346 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:13.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff343e00640 1 -- 172.21.15.66:0/1736426035 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3380ab670 con 0x7ff3440cca40 2024-03-30T00:44:13.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff343e00640 1 -- 172.21.15.66:0/1736426035 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3380ab7d0 con 0x7ff3440cca40 2024-03-30T00:44:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff343e00640 1 -- 172.21.15.66:0/1736426035 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3380abb60 con 0x7ff3440cca40 2024-03-30T00:44:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff34c200640 1 -- 172.21.15.66:0/1736426035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3440cca40 msgr2=0x7ff3440cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff34c200640 1 --2- 172.21.15.66:0/1736426035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3440cca40 0x7ff3440cac90 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff338009870 tx=0x7ff33801c4b0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff34c200640 1 -- 172.21.15.66:0/1736426035 shutdown_connections 2024-03-30T00:44:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff34c200640 1 --2- 172.21.15.66:0/1736426035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff34406b140 0x7ff34406b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff34c200640 1 --2- 172.21.15.66:0/1736426035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3440cca40 0x7ff3440cac90 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff34c200640 1 -- 172.21.15.66:0/1736426035 >> 172.21.15.66:0/1736426035 conn(0x7ff3440b2130 msgr2=0x7ff3440b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff34c200640 1 -- 172.21.15.66:0/1736426035 shutdown_connections 2024-03-30T00:44:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.035+0000 7ff34c200640 1 -- 172.21.15.66:0/1736426035 wait complete. 2024-03-30T00:44:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34c200640 1 Processor -- start 2024-03-30T00:44:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34c200640 1 -- start start 2024-03-30T00:44:13.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34c200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff34406b140 0x7ff344155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34c200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff344099250 0x7ff34414e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34c200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3440cca40 0x7ff3440adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34c200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3441564d0 con 0x7ff3440cca40 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34c200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff344155950 con 0x7ff34406b140 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34c200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff344155c30 con 0x7ff344099250 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3440cca40 0x7ff3440adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3440cca40 0x7ff3440adc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60626/0 (socket says 172.21.15.66:60626) 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34a200640 1 -- 172.21.15.66:0/140755813 learned_addr learned my addr 172.21.15.66:0/140755813 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff349800640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff34406b140 0x7ff344155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff348e00640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff344099250 0x7ff34414e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34a200640 1 -- 172.21.15.66:0/140755813 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff344099250 msgr2=0x7ff34414e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34a200640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff344099250 0x7ff34414e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34a200640 1 -- 172.21.15.66:0/140755813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff34406b140 msgr2=0x7ff344155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34a200640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff34406b140 0x7ff344155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34a200640 1 -- 172.21.15.66:0/140755813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3380094d0 con 0x7ff3440cca40 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34a200640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3440cca40 0x7ff3440adc20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00c780 tx=0x7ff33c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff341600640 1 -- 172.21.15.66:0/140755813 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c004b70 con 0x7ff3440cca40 2024-03-30T00:44:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff341600640 1 -- 172.21.15.66:0/140755813 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c00ce90 con 0x7ff3440cca40 2024-03-30T00:44:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff341600640 1 -- 172.21.15.66:0/140755813 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00a850 con 0x7ff3440cca40 2024-03-30T00:44:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.039+0000 7ff34c200640 1 -- 172.21.15.66:0/140755813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff344155f10 con 0x7ff3440cca40 2024-03-30T00:44:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.043+0000 7ff34c200640 1 -- 172.21.15.66:0/140755813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34414e970 con 0x7ff3440cca40 2024-03-30T00:44:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.043+0000 7ff34c200640 1 -- 172.21.15.66:0/140755813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3440677d0 con 0x7ff3440cca40 2024-03-30T00:44:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.043+0000 7ff341600640 1 -- 172.21.15.66:0/140755813 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff33c00a9b0 con 0x7ff3440cca40 2024-03-30T00:44:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.043+0000 7ff341600640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 0x7ff324068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.043+0000 7ff341600640 1 -- 172.21.15.66:0/140755813 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff33c084040 con 0x7ff3440cca40 2024-03-30T00:44:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.043+0000 7ff349800640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 0x7ff324068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.043+0000 7ff349800640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 0x7ff324068b50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff3380af000 tx=0x7ff3380b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:13.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.047+0000 7ff341600640 1 -- 172.21.15.66:0/140755813 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff33c0b5980 con 0x7ff3440cca40 2024-03-30T00:44:13.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 -- 172.21.15.66:0/140755813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff34414ec70 con 0x7ff3440cca40 2024-03-30T00:44:13.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff341600640 1 -- 172.21.15.66:0/140755813 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff34414ec70 con 0x7ff3440cca40 2024-03-30T00:44:13.274 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:13.274 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:44:13.274 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-03-30T00:44:13.274 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm857MDRAAu2pldF20deZfoSDDXuTF/A==", 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMmNFGhAAlHS6ZpIoJc7tlYazCISiLw==", 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:44:13.275 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmuiBsFRAA2kez6ylAylKTWsQ1S2cahA==", 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm/oV3IRAAWn/zG1cLK+Raa2pYPzHd7Q==", 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAbXwdmf3HtOhAAyRcyb0YUJcJca5fjmZ3Frw==", 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.277 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAfXwdml7cdBRAAs639bBHTPyfhfVZMPRMfYA==", 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAiXwdmeS6KChAAZjLAbKSL7Vswi3qQ4304bA==", 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAlXwdmeuigEBAAiPIPheowKBnn3UZWwpdr+Q==", 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:44:13.279 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQApXwdm+bQNHRAAAb/lufdb7Fe+EFevd23gtA==", 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-03-30T00:44:13.280 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAsXwdmReL1IhAAwCs1cS+P6J8VCOYdrnrPQg==", 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAvXwdmA15vKBAAmpnKrDM/Y0xuBGxdyjrkWA==", 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAyXwdmrl02LhAA6JfEnugoz6l4nsXl6/vq0Q==", 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmZjh7JRAA3EFOImM03iOljcwmA7/xWw==", 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmkZv0LhAAgwCqW25+J1VYO31mWwDtLQ==", 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmLG1/KxAAmNIZ9ZXdqPRQjCXvz7wC9A==", 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-03-30T00:44:13.284 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmyJ/nNBAAIgiUoUCbTeLEjL1iaPFw1A==", 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAXXwdmwWxQLRAAoFyZnFsIo1NvQlzRN+UHmw==", 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:44:13.285 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmEF+xCRAAqRVFguki881+4azP8lsf0g==", 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-03-30T00:44:13.286 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMg32BRAAUWBF+QbwRGsopcAoBay8wA==", 2024-03-30T00:44:13.287 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:44:13.287 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:44:13.287 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:44:13.287 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:44:13.287 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.287 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:13.287 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-03-30T00:44:13.287 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:44:13.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 -- 172.21.15.66:0/140755813 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 msgr2=0x7ff324068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 0x7ff324068b50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff3380af000 tx=0x7ff3380b6040 comp rx=0 tx=0).stop 2024-03-30T00:44:13.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 -- 172.21.15.66:0/140755813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3440cca40 msgr2=0x7ff3440adc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3440cca40 0x7ff3440adc20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00c780 tx=0x7ff33c00cc40 comp rx=0 tx=0).stop 2024-03-30T00:44:13.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 -- 172.21.15.66:0/140755813 shutdown_connections 2024-03-30T00:44:13.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff324066690 0x7ff324068b50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3440cca40 0x7ff3440adc20 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff344099250 0x7ff34414e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 --2- 172.21.15.66:0/140755813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff34406b140 0x7ff344155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 -- 172.21.15.66:0/140755813 >> 172.21.15.66:0/140755813 conn(0x7ff3440b2130 msgr2=0x7ff3440ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:13.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 -- 172.21.15.66:0/140755813 shutdown_connections 2024-03-30T00:44:13.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.267+0000 7ff34c200640 1 -- 172.21.15.66:0/140755813 wait complete. 2024-03-30T00:44:13.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.427+0000 7f99f5000640 1 Processor -- start 2024-03-30T00:44:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99f5000640 1 -- start start 2024-03-30T00:44:13.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99f5000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f99f015ca10 0x7f99f006ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99f5000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f99f00ae4e0 0x7f99f00cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99f5000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99f00ce820 0x7f99f00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99f5000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f00b1f70 con 0x7f99f015ca10 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99f5000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f00cf1f0 con 0x7f99f00ae4e0 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99f5000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f006e7d0 con 0x7f99f00ce820 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99ede00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f99f015ca10 0x7f99f006ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99ede00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f99f015ca10 0x7f99f006ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60634/0 (socket says 172.21.15.66:60634) 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99ede00640 1 -- 172.21.15.66:0/3417999208 learned_addr learned my addr 172.21.15.66:0/3417999208 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99e7e00640 1 --2- 172.21.15.66:0/3417999208 >> v2:172.21.15.156:3300/0 conn(0x7f99f00ae4e0 0x7f99f00cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99ee800640 1 --2- 172.21.15.66:0/3417999208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99f00ce820 0x7f99f00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99ede00640 1 -- 172.21.15.66:0/3417999208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99f00ce820 msgr2=0x7f99f00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99ede00640 1 --2- 172.21.15.66:0/3417999208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99f00ce820 0x7f99f00cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99ede00640 1 -- 172.21.15.66:0/3417999208 >> v2:172.21.15.156:3300/0 conn(0x7f99f00ae4e0 msgr2=0x7f99f00cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99ede00640 1 --2- 172.21.15.66:0/3417999208 >> v2:172.21.15.156:3300/0 conn(0x7f99f00ae4e0 0x7f99f00cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.431+0000 7f99ede00640 1 -- 172.21.15.66:0/3417999208 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f006f1a0 con 0x7f99f015ca10 2024-03-30T00:44:13.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99ede00640 1 --2- 172.21.15.66:0/3417999208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f015ca10 0x7f99f006ec60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f99f00ae490 tx=0x7f99e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9effa69cc82aef05 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99ed400640 1 -- 172.21.15.66:0/3417999208 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e00aa660 con 0x7f99f015ca10 2024-03-30T00:44:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99ed400640 1 -- 172.21.15.66:0/3417999208 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e00aa7c0 con 0x7f99f015ca10 2024-03-30T00:44:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99ed400640 1 -- 172.21.15.66:0/3417999208 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e00aab50 con 0x7f99f015ca10 2024-03-30T00:44:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99f5000640 1 -- 172.21.15.66:0/3417999208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f015ca10 msgr2=0x7f99f006ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99f5000640 1 --2- 172.21.15.66:0/3417999208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f015ca10 0x7f99f006ec60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f99f00ae490 tx=0x7f99e001c520 comp rx=0 tx=0).stop 2024-03-30T00:44:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99f5000640 1 -- 172.21.15.66:0/3417999208 shutdown_connections 2024-03-30T00:44:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99f5000640 1 --2- 172.21.15.66:0/3417999208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99f00ce820 0x7f99f00cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99f5000640 1 --2- 172.21.15.66:0/3417999208 >> v2:172.21.15.156:3300/0 conn(0x7f99f00ae4e0 0x7f99f00cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99f5000640 1 --2- 172.21.15.66:0/3417999208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f015ca10 0x7f99f006ec60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99f5000640 1 -- 172.21.15.66:0/3417999208 >> 172.21.15.66:0/3417999208 conn(0x7f99f0098a20 msgr2=0x7f99f00b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99f5000640 1 -- 172.21.15.66:0/3417999208 shutdown_connections 2024-03-30T00:44:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99f5000640 1 -- 172.21.15.66:0/3417999208 wait complete. 2024-03-30T00:44:13.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99f5000640 1 Processor -- start 2024-03-30T00:44:13.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.435+0000 7f99f5000640 1 -- start start 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99f5000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f00ae4e0 0x7f99f0156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99f5000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99f00ce820 0x7f99f00b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99f5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f015ca10 0x7f99f01555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99f5000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00c4350 con 0x7f99f015ca10 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99f5000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00c45a0 con 0x7f99f00ae4e0 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99f5000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00c4880 con 0x7f99f00ce820 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99ee800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f015ca10 0x7f99f01555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99ee800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f015ca10 0x7f99f01555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60640/0 (socket says 172.21.15.66:60640) 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99ee800640 1 -- 172.21.15.66:0/1098544947 learned_addr learned my addr 172.21.15.66:0/1098544947 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99e7e00640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99f00ce820 0x7f99f00b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99ede00640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f00ae4e0 0x7f99f0156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99ee800640 1 -- 172.21.15.66:0/1098544947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99f00ce820 msgr2=0x7f99f00b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99ee800640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99f00ce820 0x7f99f00b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99ee800640 1 -- 172.21.15.66:0/1098544947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f00ae4e0 msgr2=0x7f99f0156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99ee800640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f00ae4e0 0x7f99f0156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99ee800640 1 -- 172.21.15.66:0/1098544947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00094d0 con 0x7f99f015ca10 2024-03-30T00:44:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99ede00640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f00ae4e0 0x7f99f0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99ee800640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f015ca10 0x7f99f01555a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f99e8007e40 tx=0x7f99e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99e6000640 1 -- 172.21.15.66:0/1098544947 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8014620 con 0x7f99f015ca10 2024-03-30T00:44:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99e6000640 1 -- 172.21.15.66:0/1098544947 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e800ee90 con 0x7f99f015ca10 2024-03-30T00:44:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99e6000640 1 -- 172.21.15.66:0/1098544947 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80149e0 con 0x7f99f015ca10 2024-03-30T00:44:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99f5000640 1 -- 172.21.15.66:0/1098544947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f00c4b60 con 0x7f99f015ca10 2024-03-30T00:44:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99f5000640 1 -- 172.21.15.66:0/1098544947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f01566a0 con 0x7f99f015ca10 2024-03-30T00:44:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99f5000640 1 -- 172.21.15.66:0/1098544947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f00c5080 con 0x7f99f015ca10 2024-03-30T00:44:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.439+0000 7f99e6000640 1 -- 172.21.15.66:0/1098544947 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f99e8014b40 con 0x7f99f015ca10 2024-03-30T00:44:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.443+0000 7f99e6000640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f99cc066690 0x7f99cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.443+0000 7f99e6000640 1 -- 172.21.15.66:0/1098544947 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f99e80846d0 con 0x7f99f015ca10 2024-03-30T00:44:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.443+0000 7f99ede00640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f99cc066690 0x7f99cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.443+0000 7f99ede00640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f99cc066690 0x7f99cc068b50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f99e00ae000 tx=0x7f99e00b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.447+0000 7f99e6000640 1 -- 172.21.15.66:0/1098544947 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f99f00c5080 con 0x7f99f015ca10 2024-03-30T00:44:13.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.607+0000 7f99f5000640 1 -- 172.21.15.66:0/1098544947 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99f00c5080 con 0x7f99f015ca10 2024-03-30T00:44:13.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.607+0000 7f99e6000640 1 -- 172.21.15.66:0/1098544947 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+12595 (secure 0 0 0) 0x7f99f00c5080 con 0x7f99f015ca10 2024-03-30T00:44:13.615 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:13.616 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":78,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:12.857237+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:13.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 -- 172.21.15.66:0/1098544947 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f99cc066690 msgr2=0x7f99cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f99cc066690 0x7f99cc068b50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f99e00ae000 tx=0x7f99e00b5040 comp rx=0 tx=0).stop 2024-03-30T00:44:13.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 -- 172.21.15.66:0/1098544947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f015ca10 msgr2=0x7f99f01555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f015ca10 0x7f99f01555a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f99e8007e40 tx=0x7f99e800e4b0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 -- 172.21.15.66:0/1098544947 shutdown_connections 2024-03-30T00:44:13.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f99cc066690 0x7f99cc068b50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f015ca10 0x7f99f01555a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99f00ce820 0x7f99f00b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 --2- 172.21.15.66:0/1098544947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f00ae4e0 0x7f99f0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 -- 172.21.15.66:0/1098544947 >> 172.21.15.66:0/1098544947 conn(0x7f99f0098a20 msgr2=0x7f99f015b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:13.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 -- 172.21.15.66:0/1098544947 shutdown_connections 2024-03-30T00:44:13.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.611+0000 7f99f5000640 1 -- 172.21.15.66:0/1098544947 wait complete. 2024-03-30T00:44:13.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:44:13.637 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:44:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.771+0000 7f4e92200640 1 Processor -- start 2024-03-30T00:44:13.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e92200640 1 -- start start 2024-03-30T00:44:13.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e92200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 0x7f4e8c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e92200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e8c15a370 0x7f4e8c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e92200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8c15a990 con 0x7f4e8c0cde70 2024-03-30T00:44:13.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e92200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8c0b1b80 con 0x7f4e8c15a370 2024-03-30T00:44:13.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e92200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0cd3e0 con 0x7f4e8c0cda90 2024-03-30T00:44:13.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e8b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 0x7f4e8c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e8be00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e8c15a370 0x7f4e8c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e83e00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4e8c0cde70 0x7f4e8c159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:34712/0 (socket says 172.21.15.66:34712) 2024-03-30T00:44:13.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e83e00640 1 -- 172.21.15.66:0/2669257875 learned_addr learned my addr 172.21.15.66:0/2669257875 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:13.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e8aa00640 1 -- 172.21.15.66:0/2669257875 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925977102 0 0) 0x7f4e8c15a990 con 0x7f4e8c0cde70 2024-03-30T00:44:13.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e8aa00640 1 -- 172.21.15.66:0/2669257875 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e70003530 con 0x7f4e8c0cde70 2024-03-30T00:44:13.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e8b400640 1 -- 172.21.15.66:0/2669257875 >> v2:172.21.15.156:3300/0 conn(0x7f4e8c15a370 msgr2=0x7f4e8c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:13.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e8b400640 1 --2- 172.21.15.66:0/2669257875 >> v2:172.21.15.156:3300/0 conn(0x7f4e8c15a370 0x7f4e8c0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e8b400640 1 -- 172.21.15.66:0/2669257875 >> v1:172.21.15.156:6789/0 conn(0x7f4e8c0cde70 legacy=0x7f4e8c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e8b400640 1 -- 172.21.15.66:0/2669257875 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c0b11e0 con 0x7f4e8c0cda90 2024-03-30T00:44:13.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.775+0000 7f4e8be00640 1 --2- 172.21.15.66:0/2669257875 >> v2:172.21.15.156:3300/0 conn(0x7f4e8c15a370 0x7f4e8c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:13.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e8b400640 1 --2- 172.21.15.66:0/2669257875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 0x7f4e8c0ce520 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0097c0 tx=0x7f4e7c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94ebc2cfb0d9246d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:13.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e8aa00640 1 -- 172.21.15.66:0/2669257875 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0ab600 con 0x7f4e8c0cda90 2024-03-30T00:44:13.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e8aa00640 1 -- 172.21.15.66:0/2669257875 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c0a7050 con 0x7f4e8c0cda90 2024-03-30T00:44:13.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e8aa00640 1 -- 172.21.15.66:0/2669257875 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0ab960 con 0x7f4e8c0cda90 2024-03-30T00:44:13.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e92200640 1 -- 172.21.15.66:0/2669257875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 msgr2=0x7f4e8c0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e92200640 1 --2- 172.21.15.66:0/2669257875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 0x7f4e8c0ce520 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0097c0 tx=0x7f4e7c01c410 comp rx=0 tx=0).stop 2024-03-30T00:44:13.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e92200640 1 -- 172.21.15.66:0/2669257875 shutdown_connections 2024-03-30T00:44:13.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e92200640 1 --2- 172.21.15.66:0/2669257875 >> v2:172.21.15.156:3300/0 conn(0x7f4e8c15a370 0x7f4e8c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e92200640 1 --2- 172.21.15.66:0/2669257875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 0x7f4e8c0ce520 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e92200640 1 -- 172.21.15.66:0/2669257875 >> 172.21.15.66:0/2669257875 conn(0x7f4e8c069f60 msgr2=0x7f4e8c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:13.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e92200640 1 -- 172.21.15.66:0/2669257875 shutdown_connections 2024-03-30T00:44:13.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e92200640 1 -- 172.21.15.66:0/2669257875 wait complete. 2024-03-30T00:44:13.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e92200640 1 Processor -- start 2024-03-30T00:44:13.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.779+0000 7f4e92200640 1 -- start start 2024-03-30T00:44:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e92200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 0x7f4e8c154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e92200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e8c0cde70 0x7f4e8c1538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e92200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c15a370 0x7f4e8c0abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e92200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c06de30 con 0x7f4e8c0cde70 2024-03-30T00:44:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e92200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0acb00 con 0x7f4e8c15a370 2024-03-30T00:44:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e92200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0acde0 con 0x7f4e8c0cda90 2024-03-30T00:44:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e8c0cde70 0x7f4e8c1538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e8c0cde70 0x7f4e8c1538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60650/0 (socket says 172.21.15.66:60650) 2024-03-30T00:44:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83e00640 1 -- 172.21.15.66:0/2421014647 learned_addr learned my addr 172.21.15.66:0/2421014647 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e8b400640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 0x7f4e8c154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e8be00640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c15a370 0x7f4e8c0abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83e00640 1 -- 172.21.15.66:0/2421014647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 msgr2=0x7f4e8c154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83e00640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 0x7f4e8c154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83e00640 1 -- 172.21.15.66:0/2421014647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c15a370 msgr2=0x7f4e8c0abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83e00640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c15a370 0x7f4e8c0abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83e00640 1 -- 172.21.15.66:0/2421014647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e84009a70 con 0x7f4e8c0cde70 2024-03-30T00:44:13.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e8b400640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 0x7f4e8c154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83e00640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e8c0cde70 0x7f4e8c1538e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400c450 tx=0x7f4e7400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83400640 1 -- 172.21.15.66:0/2421014647 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e74005cf0 con 0x7f4e8c0cde70 2024-03-30T00:44:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83400640 1 -- 172.21.15.66:0/2421014647 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e74005e50 con 0x7f4e8c0cde70 2024-03-30T00:44:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e92200640 1 -- 172.21.15.66:0/2421014647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7c009420 con 0x7f4e8c0cde70 2024-03-30T00:44:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e83400640 1 -- 172.21.15.66:0/2421014647 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e74007290 con 0x7f4e8c0cde70 2024-03-30T00:44:13.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e92200640 1 -- 172.21.15.66:0/2421014647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8c0ad3a0 con 0x7f4e8c0cde70 2024-03-30T00:44:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.783+0000 7f4e92200640 1 -- 172.21.15.66:0/2421014647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8c0ad590 con 0x7f4e8c0cde70 2024-03-30T00:44:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.787+0000 7f4e83400640 1 -- 172.21.15.66:0/2421014647 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4e740073f0 con 0x7f4e8c0cde70 2024-03-30T00:44:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.787+0000 7f4e83400640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4e68066690 0x7f4e68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.787+0000 7f4e83400640 1 -- 172.21.15.66:0/2421014647 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4e74083540 con 0x7f4e8c0cde70 2024-03-30T00:44:13.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.787+0000 7f4e8b400640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4e68066690 0x7f4e68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:13.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.787+0000 7f4e8b400640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4e68066690 0x7f4e68068b50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0ad000 tx=0x7f4e7c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:13.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.791+0000 7f4e83400640 1 -- 172.21.15.66:0/2421014647 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4e74050ab0 con 0x7f4e8c0cde70 2024-03-30T00:44:13.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 -- 172.21.15.66:0/2421014647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e8c066c30 con 0x7f4e8c0cde70 2024-03-30T00:44:13.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e83400640 1 -- 172.21.15.66:0/2421014647 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e740170a0 con 0x7f4e8c0cde70 2024-03-30T00:44:13.959 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:44:13.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 -- 172.21.15.66:0/2421014647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4e68066690 msgr2=0x7f4e68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4e68066690 0x7f4e68068b50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0ad000 tx=0x7f4e7c0a6000 comp rx=0 tx=0).stop 2024-03-30T00:44:13.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 -- 172.21.15.66:0/2421014647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e8c0cde70 msgr2=0x7f4e8c1538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:13.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e8c0cde70 0x7f4e8c1538e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400c450 tx=0x7f4e7400c910 comp rx=0 tx=0).stop 2024-03-30T00:44:13.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 -- 172.21.15.66:0/2421014647 shutdown_connections 2024-03-30T00:44:13.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c15a370 0x7f4e8c0abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4e68066690 0x7f4e68068b50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e8c0cde70 0x7f4e8c1538e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 --2- 172.21.15.66:0/2421014647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e8c0cda90 0x7f4e8c154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:13.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 -- 172.21.15.66:0/2421014647 >> 172.21.15.66:0/2421014647 conn(0x7f4e8c069f60 msgr2=0x7f4e8c0c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:13.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 -- 172.21.15.66:0/2421014647 shutdown_connections 2024-03-30T00:44:13.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:13.951+0000 7f4e92200640 1 -- 172.21.15.66:0/2421014647 wait complete. 2024-03-30T00:44:13.979 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T00:44:14.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.111+0000 7fa6c5200640 1 Processor -- start 2024-03-30T00:44:14.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.111+0000 7fa6c5200640 1 -- start start 2024-03-30T00:44:14.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6c5200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa6c015ca10 0x7fa6c006ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6c5200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c00ae4e0 0x7fa6c00cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6c5200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c006e7d0 con 0x7fa6c00ce820 2024-03-30T00:44:14.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6c5200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c00b1f70 con 0x7fa6c015ca10 2024-03-30T00:44:14.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6c5200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c006f1a0 con 0x7fa6c00ae4e0 2024-03-30T00:44:14.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6be800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa6c00ce820 0x7fa6c0160510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:38668/0 (socket says 172.21.15.66:38668) 2024-03-30T00:44:14.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6be800640 1 -- 172.21.15.66:0/3564857074 learned_addr learned my addr 172.21.15.66:0/3564857074 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:14.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6b7e00640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c00ae4e0 0x7fa6c00cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bde00640 1 --2- 172.21.15.66:0/3564857074 >> v2:172.21.15.156:3300/0 conn(0x7fa6c015ca10 0x7fa6c006ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106174006 0 0) 0x7fa6c006e7d0 con 0x7fa6c00ce820 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a4003530 con 0x7fa6c00ce820 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1634929684 0 0) 0x7fa6a4003530 con 0x7fa6c00ce820 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c00ae4e0 msgr2=0x7fa6c00cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c00ae4e0 0x7fa6c00cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 >> v2:172.21.15.156:3300/0 conn(0x7fa6c015ca10 msgr2=0x7fa6c006ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 --2- 172.21.15.66:0/3564857074 >> v2:172.21.15.156:3300/0 conn(0x7fa6c015ca10 0x7fa6c006ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c015cdf0 con 0x7fa6c00ce820 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fa6b8001a00 con 0x7fa6c00ce820 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 >> v1:172.21.15.66:6789/0 conn(0x7fa6c00ce820 legacy=0x7fa6c0160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a4003980 0x7fa6a4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c00ae4e0 0x7fa6a40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a4004d10 0x7fa6a4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c00b1f70 con 0x7fa6a4003980 2024-03-30T00:44:14.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c006f1a0 con 0x7fa6a4004d10 2024-03-30T00:44:14.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a400b990 con 0x7fa6c00ae4e0 2024-03-30T00:44:14.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bde00640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a4003980 0x7fa6a4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6b7e00640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c00ae4e0 0x7fa6a40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6be800640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a4004d10 0x7fa6a4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bde00640 1 -- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c00ae4e0 msgr2=0x7fa6a40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:14.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bde00640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c00ae4e0 0x7fa6a40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bde00640 1 -- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a4004d10 msgr2=0x7fa6a4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:14.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bde00640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a4004d10 0x7fa6a4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6bde00640 1 -- 172.21.15.66:0/3564857074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a4003530 con 0x7fa6a4003980 2024-03-30T00:44:14.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.115+0000 7fa6be800640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a4004d10 0x7fa6a4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:14.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.119+0000 7fa6bde00640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a4003980 0x7fa6a4003da0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009ae0 tx=0x7fa6b001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f223d303d359d5ab server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:14.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.119+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00a5560 con 0x7fa6a4003980 2024-03-30T00:44:14.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.119+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b00a56c0 con 0x7fa6a4003980 2024-03-30T00:44:14.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.119+0000 7fa6bd400640 1 -- 172.21.15.66:0/3564857074 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00a5a70 con 0x7fa6a4003980 2024-03-30T00:44:14.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.119+0000 7fa6c5200640 1 -- 172.21.15.66:0/3564857074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a4003980 msgr2=0x7fa6a4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.119+0000 7fa6c5200640 1 --2- 172.21.15.66:0/3564857074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a4003980 0x7fa6a4003da0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009ae0 tx=0x7fa6b001ca80 comp rx=0 tx=0).stop 2024-03-30T00:44:14.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.119+0000 7fa6be800640 1 -- 172.21.15.66:0/3564857074 reap_dead start 2024-03-30T00:44:14.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.119+0000 7fa6c5200640 1 -- 172.21.15.66:0/3564857074 shutdown_connections 2024-03-30T00:44:14.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.119+0000 7fa6c5200640 1 -- 172.21.15.66:0/3564857074 >> 172.21.15.66:0/3564857074 conn(0x7fa6c0098a20 msgr2=0x7fa6c00b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:14.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.119+0000 7fa6c5200640 1 -- 172.21.15.66:0/3564857074 shutdown_connections 2024-03-30T00:44:14.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.119+0000 7fa6c5200640 1 -- 172.21.15.66:0/3564857074 wait complete. 2024-03-30T00:44:14.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6c5200640 1 Processor -- start 2024-03-30T00:44:14.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6c5200640 1 -- start start 2024-03-30T00:44:14.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6c5200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a4003980 0x7fa6c00c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6c5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c01529e0 0x7fa6c0151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6c5200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c0152490 0x7fa6c0166670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6c5200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c00b1020 con 0x7fa6c01529e0 2024-03-30T00:44:14.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6c5200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c0152250 con 0x7fa6a4003980 2024-03-30T00:44:14.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6c5200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c00adf70 con 0x7fa6c0152490 2024-03-30T00:44:14.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c01529e0 0x7fa6c0151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c01529e0 0x7fa6c0151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60666/0 (socket says 172.21.15.66:60666) 2024-03-30T00:44:14.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b7e00640 1 -- 172.21.15.66:0/956449443 learned_addr learned my addr 172.21.15.66:0/956449443 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:14.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6be800640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c0152490 0x7fa6c0166670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6bde00640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a4003980 0x7fa6c00c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b7e00640 1 -- 172.21.15.66:0/956449443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c0152490 msgr2=0x7fa6c0166670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b7e00640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c0152490 0x7fa6c0166670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b7e00640 1 -- 172.21.15.66:0/956449443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a4003980 msgr2=0x7fa6c00c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b7e00640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a4003980 0x7fa6c00c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b7e00640 1 -- 172.21.15.66:0/956449443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c00b12c0 con 0x7fa6c01529e0 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b7e00640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c01529e0 0x7fa6c0151d10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80119d0 tx=0x7fa6a8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b6000640 1 -- 172.21.15.66:0/956449443 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800fb50 con 0x7fa6c01529e0 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b6000640 1 -- 172.21.15.66:0/956449443 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a800fcb0 con 0x7fa6c01529e0 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6c5200640 1 -- 172.21.15.66:0/956449443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c00ae250 con 0x7fa6c01529e0 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6b6000640 1 -- 172.21.15.66:0/956449443 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8014440 con 0x7fa6c01529e0 2024-03-30T00:44:14.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6c5200640 1 -- 172.21.15.66:0/956449443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c0166bb0 con 0x7fa6c01529e0 2024-03-30T00:44:14.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.123+0000 7fa6c5200640 1 -- 172.21.15.66:0/956449443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c00b0e80 con 0x7fa6c01529e0 2024-03-30T00:44:14.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.127+0000 7fa6b6000640 1 -- 172.21.15.66:0/956449443 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa6a80145a0 con 0x7fa6c01529e0 2024-03-30T00:44:14.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.127+0000 7fa6b6000640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa69c0666c0 0x7fa69c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.127+0000 7fa6b6000640 1 -- 172.21.15.66:0/956449443 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa6a8084880 con 0x7fa6c01529e0 2024-03-30T00:44:14.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.127+0000 7fa6bde00640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa69c0666c0 0x7fa69c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.127+0000 7fa6bde00640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa69c0666c0 0x7fa69c068b80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00af000 tx=0x7fa6b00046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:14.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.131+0000 7fa6b6000640 1 -- 172.21.15.66:0/956449443 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa6c00b0e80 con 0x7fa6c01529e0 2024-03-30T00:44:14.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 -- 172.21.15.66:0/956449443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa6c00b0e80 con 0x7fa6c01529e0 2024-03-30T00:44:14.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6b6000640 1 -- 172.21.15.66:0/956449443 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa6c00b0e80 con 0x7fa6c01529e0 2024-03-30T00:44:14.302 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:14.302 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:44:14.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:44:14.302 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-03-30T00:44:14.302 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-03-30T00:44:14.302 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:14.302 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:44:14.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 -- 172.21.15.66:0/956449443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa69c0666c0 msgr2=0x7fa69c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa69c0666c0 0x7fa69c068b80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00af000 tx=0x7fa6b00046d0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 -- 172.21.15.66:0/956449443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c01529e0 msgr2=0x7fa6c0151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c01529e0 0x7fa6c0151d10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80119d0 tx=0x7fa6a8011e90 comp rx=0 tx=0).stop 2024-03-30T00:44:14.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 -- 172.21.15.66:0/956449443 shutdown_connections 2024-03-30T00:44:14.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6c0152490 0x7fa6c0166670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa69c0666c0 0x7fa69c068b80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c01529e0 0x7fa6c0151d10 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 --2- 172.21.15.66:0/956449443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a4003980 0x7fa6c00c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 -- 172.21.15.66:0/956449443 >> 172.21.15.66:0/956449443 conn(0x7fa6c0098a20 msgr2=0x7fa6c0156a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:14.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 -- 172.21.15.66:0/956449443 shutdown_connections 2024-03-30T00:44:14.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.295+0000 7fa6c5200640 1 -- 172.21.15.66:0/956449443 wait complete. 2024-03-30T00:44:14.323 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T00:44:14.323 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T00:44:14.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.459+0000 7f7808600640 1 Processor -- start 2024-03-30T00:44:14.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.459+0000 7f7808600640 1 -- start start 2024-03-30T00:44:14.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.459+0000 7f7808600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f780015cca0 0x7f780015df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.459+0000 7f7808600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7800099850 0x7f7800098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.459+0000 7f7808600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780006bdf0 con 0x7f7800098f60 2024-03-30T00:44:14.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.459+0000 7f7808600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78000993f0 con 0x7f780015cca0 2024-03-30T00:44:14.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.459+0000 7f7808600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780006b1e0 con 0x7f7800099850 2024-03-30T00:44:14.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f7805c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f780015cca0 0x7f780015df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f7806600640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7800098f60 0x7f7800160940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:38678/0 (socket says 172.21.15.66:38678) 2024-03-30T00:44:14.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f7806600640 1 -- 172.21.15.66:0/2369752916 learned_addr learned my addr 172.21.15.66:0/2369752916 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:14.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f77ffe00640 1 -- 172.21.15.66:0/2369752916 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335506858 0 0) 0x7f780006bdf0 con 0x7f7800098f60 2024-03-30T00:44:14.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f77ffe00640 1 -- 172.21.15.66:0/2369752916 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77e8003530 con 0x7f7800098f60 2024-03-30T00:44:14.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f7805c00640 1 -- 172.21.15.66:0/2369752916 >> v2:172.21.15.156:3300/0 conn(0x7f7800099850 msgr2=0x7f7800098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:14.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f7805c00640 1 --2- 172.21.15.66:0/2369752916 >> v2:172.21.15.156:3300/0 conn(0x7f7800099850 0x7f7800098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f7805c00640 1 -- 172.21.15.66:0/2369752916 >> v1:172.21.15.66:6789/0 conn(0x7f7800098f60 legacy=0x7f7800160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f7805c00640 1 -- 172.21.15.66:0/2369752916 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780006b4c0 con 0x7f780015cca0 2024-03-30T00:44:14.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f7805c00640 1 --2- 172.21.15.66:0/2369752916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f780015cca0 0x7f780015df00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f78000ce6f0 tx=0x7f77f401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d906b5c92b116de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:14.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f77ffe00640 1 -- 172.21.15.66:0/2369752916 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f40ab600 con 0x7f780015cca0 2024-03-30T00:44:14.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f77ffe00640 1 -- 172.21.15.66:0/2369752916 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f40a7050 con 0x7f780015cca0 2024-03-30T00:44:14.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.463+0000 7f77ffe00640 1 -- 172.21.15.66:0/2369752916 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f40ab980 con 0x7f780015cca0 2024-03-30T00:44:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 -- 172.21.15.66:0/2369752916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f780015cca0 msgr2=0x7f780015df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 --2- 172.21.15.66:0/2369752916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f780015cca0 0x7f780015df00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f78000ce6f0 tx=0x7f77f401c450 comp rx=0 tx=0).stop 2024-03-30T00:44:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 -- 172.21.15.66:0/2369752916 shutdown_connections 2024-03-30T00:44:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 --2- 172.21.15.66:0/2369752916 >> v2:172.21.15.156:3300/0 conn(0x7f7800099850 0x7f7800098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 --2- 172.21.15.66:0/2369752916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f780015cca0 0x7f780015df00 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 -- 172.21.15.66:0/2369752916 >> 172.21.15.66:0/2369752916 conn(0x7f78000b2140 msgr2=0x7f78000b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:14.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 -- 172.21.15.66:0/2369752916 shutdown_connections 2024-03-30T00:44:14.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 -- 172.21.15.66:0/2369752916 wait complete. 2024-03-30T00:44:14.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 Processor -- start 2024-03-30T00:44:14.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 -- start start 2024-03-30T00:44:14.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7800098f60 0x7f78000b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7800099850 0x7f7800160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f780015cca0 0x7f78000b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780015a540 con 0x7f7800099850 2024-03-30T00:44:14.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780015a820 con 0x7f780015cca0 2024-03-30T00:44:14.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7808600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780015ab00 con 0x7f7800098f60 2024-03-30T00:44:14.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7805200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7800099850 0x7f7800160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7805200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7800099850 0x7f7800160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60672/0 (socket says 172.21.15.66:60672) 2024-03-30T00:44:14.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7805200640 1 -- 172.21.15.66:0/891729150 learned_addr learned my addr 172.21.15.66:0/891729150 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:14.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7805c00640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7800098f60 0x7f78000b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7806600640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f780015cca0 0x7f78000b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7805200640 1 -- 172.21.15.66:0/891729150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7800098f60 msgr2=0x7f78000b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7805200640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7800098f60 0x7f78000b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7805200640 1 -- 172.21.15.66:0/891729150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f780015cca0 msgr2=0x7f78000b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7805200640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f780015cca0 0x7f78000b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.467+0000 7f7805200640 1 -- 172.21.15.66:0/891729150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4009420 con 0x7f7800099850 2024-03-30T00:44:14.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.471+0000 7f7805200640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7800099850 0x7f7800160890 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00b550 tx=0x7f77ec00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:14.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.471+0000 7f77fd600640 1 -- 172.21.15.66:0/891729150 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec010870 con 0x7f7800099850 2024-03-30T00:44:14.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.471+0000 7f77fd600640 1 -- 172.21.15.66:0/891729150 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec0109f0 con 0x7f7800099850 2024-03-30T00:44:14.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.471+0000 7f77fd600640 1 -- 172.21.15.66:0/891729150 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec010da0 con 0x7f7800099850 2024-03-30T00:44:14.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.471+0000 7f7808600640 1 -- 172.21.15.66:0/891729150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780006c400 con 0x7f7800099850 2024-03-30T00:44:14.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.471+0000 7f7808600640 1 -- 172.21.15.66:0/891729150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78000b4af0 con 0x7f7800099850 2024-03-30T00:44:14.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.471+0000 7f7808600640 1 -- 172.21.15.66:0/891729150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780006c260 con 0x7f7800099850 2024-03-30T00:44:14.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.471+0000 7f77fd600640 1 -- 172.21.15.66:0/891729150 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f77ec00f3c0 con 0x7f7800099850 2024-03-30T00:44:14.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.471+0000 7f77fd600640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77e0066690 0x7f77e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:14.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.471+0000 7f77fd600640 1 -- 172.21.15.66:0/891729150 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f77ec083ef0 con 0x7f7800099850 2024-03-30T00:44:14.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.471+0000 7f7805c00640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77e0066690 0x7f77e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:14.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.475+0000 7f7805c00640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77e0066690 0x7f77e0068b50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f77f40ad000 tx=0x7f77f40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:14.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.475+0000 7f77fd600640 1 -- 172.21.15.66:0/891729150 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f780006c260 con 0x7f7800099850 2024-03-30T00:44:14.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.639+0000 7f7808600640 1 -- 172.21.15.66:0/891729150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f780006c260 con 0x7f7800099850 2024-03-30T00:44:14.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f77fd600640 1 -- 172.21.15.66:0/891729150 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f780006c260 con 0x7f7800099850 2024-03-30T00:44:14.869 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-03-30T00:44:14.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f7808600640 1 -- 172.21.15.66:0/891729150 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77e0066690 msgr2=0x7f77e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f7808600640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77e0066690 0x7f77e0068b50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f77f40ad000 tx=0x7f77f40a6000 comp rx=0 tx=0).stop 2024-03-30T00:44:14.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f7808600640 1 -- 172.21.15.66:0/891729150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7800099850 msgr2=0x7f7800160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f7808600640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7800099850 0x7f7800160890 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00b550 tx=0x7f77ec00ba10 comp rx=0 tx=0).stop 2024-03-30T00:44:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f7808600640 1 -- 172.21.15.66:0/891729150 shutdown_connections 2024-03-30T00:44:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f7808600640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f780015cca0 0x7f78000b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f7808600640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f77e0066690 0x7f77e0068b50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f7808600640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7800099850 0x7f7800160890 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f7808600640 1 --2- 172.21.15.66:0/891729150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7800098f60 0x7f78000b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f7808600640 1 -- 172.21.15.66:0/891729150 >> 172.21.15.66:0/891729150 conn(0x7f78000b2140 msgr2=0x7f78000cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.863+0000 7f7808600640 1 -- 172.21.15.66:0/891729150 shutdown_connections 2024-03-30T00:44:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:14.867+0000 7f7808600640 1 -- 172.21.15.66:0/891729150 wait complete. 2024-03-30T00:44:14.895 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T00:44:15.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e51800640 1 Processor -- start 2024-03-30T00:44:15.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e51800640 1 -- start start 2024-03-30T00:44:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e51800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0e4c0d05e0 0x7f0e4c0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e51800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0e4c0caff0 0x7f0e4c0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e51800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4c0b0800 con 0x7f0e4c15a950 2024-03-30T00:44:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e51800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4c0d0f90 con 0x7f0e4c0caff0 2024-03-30T00:44:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e51800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4c0b0ae0 con 0x7f0e4c0d05e0 2024-03-30T00:44:15.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e4aa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0e4c15a950 0x7f0e4c0afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:38694/0 (socket says 172.21.15.66:38694) 2024-03-30T00:44:15.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e4b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0e4c0caff0 0x7f0e4c0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:15.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e4aa00640 1 -- 172.21.15.66:0/48812392 learned_addr learned my addr 172.21.15.66:0/48812392 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:15.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e43e00640 1 --2- 172.21.15.66:0/48812392 >> v2:172.21.15.156:3300/0 conn(0x7f0e4c0d05e0 0x7f0e4c0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:15.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e4b400640 1 -- 172.21.15.66:0/48812392 >> v2:172.21.15.156:3300/0 conn(0x7f0e4c0d05e0 msgr2=0x7f0e4c0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:15.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e4b400640 1 --2- 172.21.15.66:0/48812392 >> v2:172.21.15.156:3300/0 conn(0x7f0e4c0d05e0 0x7f0e4c0d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:15.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e4b400640 1 -- 172.21.15.66:0/48812392 >> v1:172.21.15.66:6789/0 conn(0x7f0e4c15a950 legacy=0x7f0e4c0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:15.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e4b400640 1 -- 172.21.15.66:0/48812392 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c0b0dc0 con 0x7f0e4c0caff0 2024-03-30T00:44:15.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e43e00640 1 --2- 172.21.15.66:0/48812392 >> v2:172.21.15.156:3300/0 conn(0x7f0e4c0d05e0 0x7f0e4c0d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:15.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.035+0000 7f0e4a000640 1 -- 172.21.15.66:0/48812392 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446495189 0 0) 0x7f0e4c0b0800 con 0x7f0e4c15a950 2024-03-30T00:44:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e4b400640 1 --2- 172.21.15.66:0/48812392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4c0caff0 0x7f0e4c0cb400 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c06e1b0 tx=0x7f0e4401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592305b3b00c2410 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e4a000640 1 -- 172.21.15.66:0/48812392 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e440a59c0 con 0x7f0e4c0caff0 2024-03-30T00:44:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e4a000640 1 -- 172.21.15.66:0/48812392 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e440a5b20 con 0x7f0e4c0caff0 2024-03-30T00:44:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e4a000640 1 -- 172.21.15.66:0/48812392 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e440ad440 con 0x7f0e4c0caff0 2024-03-30T00:44:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e51800640 1 -- 172.21.15.66:0/48812392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4c0caff0 msgr2=0x7f0e4c0cb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:15.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e51800640 1 --2- 172.21.15.66:0/48812392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4c0caff0 0x7f0e4c0cb400 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c06e1b0 tx=0x7f0e4401c900 comp rx=0 tx=0).stop 2024-03-30T00:44:15.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e51800640 1 -- 172.21.15.66:0/48812392 shutdown_connections 2024-03-30T00:44:15.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e51800640 1 --2- 172.21.15.66:0/48812392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4c0caff0 0x7f0e4c0cb400 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:15.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e51800640 1 --2- 172.21.15.66:0/48812392 >> v2:172.21.15.156:3300/0 conn(0x7f0e4c0d05e0 0x7f0e4c0d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:15.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e51800640 1 -- 172.21.15.66:0/48812392 >> 172.21.15.66:0/48812392 conn(0x7f0e4c089fb0 msgr2=0x7f0e4c08a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:15.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e51800640 1 -- 172.21.15.66:0/48812392 shutdown_connections 2024-03-30T00:44:15.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.039+0000 7f0e51800640 1 -- 172.21.15.66:0/48812392 wait complete. 2024-03-30T00:44:15.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e51800640 1 Processor -- start 2024-03-30T00:44:15.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e51800640 1 -- start start 2024-03-30T00:44:15.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e51800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4c0caff0 0x7f0e4c155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:15.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e51800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e4c0d05e0 0x7f0e4c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:15.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e51800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e4c15a950 0x7f0e4c153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:15.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e51800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c159a30 con 0x7f0e4c0caff0 2024-03-30T00:44:15.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e51800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c158db0 con 0x7f0e4c15a950 2024-03-30T00:44:15.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e51800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c159090 con 0x7f0e4c0d05e0 2024-03-30T00:44:15.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e4aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4c0caff0 0x7f0e4c155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:15.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e4aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4c0caff0 0x7f0e4c155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60684/0 (socket says 172.21.15.66:60684) 2024-03-30T00:44:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e4aa00640 1 -- 172.21.15.66:0/1300936472 learned_addr learned my addr 172.21.15.66:0/1300936472 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e43e00640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e4c0d05e0 0x7f0e4c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e4b400640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e4c15a950 0x7f0e4c153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e4aa00640 1 -- 172.21.15.66:0/1300936472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e4c0d05e0 msgr2=0x7f0e4c153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e4aa00640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e4c0d05e0 0x7f0e4c153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e4aa00640 1 -- 172.21.15.66:0/1300936472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e4c15a950 msgr2=0x7f0e4c153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e4aa00640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e4c15a950 0x7f0e4c153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:15.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e4aa00640 1 -- 172.21.15.66:0/1300936472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e44009a70 con 0x7f0e4c0caff0 2024-03-30T00:44:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e4b400640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e4c15a950 0x7f0e4c153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e4aa00640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4c0caff0 0x7f0e4c155270 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00c480 tx=0x7f0e3c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e42a00640 1 -- 172.21.15.66:0/1300936472 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0074e0 con 0x7f0e4c0caff0 2024-03-30T00:44:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e42a00640 1 -- 172.21.15.66:0/1300936472 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c007660 con 0x7f0e4c0caff0 2024-03-30T00:44:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e51800640 1 -- 172.21.15.66:0/1300936472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4c159370 con 0x7f0e4c0caff0 2024-03-30T00:44:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e42a00640 1 -- 172.21.15.66:0/1300936472 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c004a30 con 0x7f0e4c0caff0 2024-03-30T00:44:15.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.043+0000 7f0e51800640 1 -- 172.21.15.66:0/1300936472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c1557b0 con 0x7f0e4c0caff0 2024-03-30T00:44:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.047+0000 7f0e51800640 1 -- 172.21.15.66:0/1300936472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4c066c30 con 0x7f0e4c0caff0 2024-03-30T00:44:15.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.047+0000 7f0e42a00640 1 -- 172.21.15.66:0/1300936472 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0e3c01c3b0 con 0x7f0e4c0caff0 2024-03-30T00:44:15.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.047+0000 7f0e42a00640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0e28066690 0x7f0e28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:15.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.047+0000 7f0e42a00640 1 -- 172.21.15.66:0/1300936472 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f0e3c083890 con 0x7f0e4c0caff0 2024-03-30T00:44:15.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.051+0000 7f0e43e00640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0e28066690 0x7f0e28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:15.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.051+0000 7f0e43e00640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0e28066690 0x7f0e28068b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c159e70 tx=0x7f0e34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.051+0000 7f0e42a00640 1 -- 172.21.15.66:0/1300936472 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0e3c054850 con 0x7f0e4c0caff0 2024-03-30T00:44:15.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.211+0000 7f0e51800640 1 -- 172.21.15.66:0/1300936472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0e4c15df70 con 0x7f0e4c0caff0 2024-03-30T00:44:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.871+0000 7f0e42a00640 1 -- 172.21.15.66:0/1300936472 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0e3c0180d0 con 0x7f0e4c0caff0 2024-03-30T00:44:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-03-30T00:44:15.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.879+0000 7f0e51800640 1 -- 172.21.15.66:0/1300936472 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0e28066690 msgr2=0x7f0e28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:15.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.879+0000 7f0e51800640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0e28066690 0x7f0e28068b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c159e70 tx=0x7f0e34008040 comp rx=0 tx=0).stop 2024-03-30T00:44:15.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.879+0000 7f0e51800640 1 -- 172.21.15.66:0/1300936472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4c0caff0 msgr2=0x7f0e4c155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:15.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.879+0000 7f0e51800640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4c0caff0 0x7f0e4c155270 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00c480 tx=0x7f0e3c00c940 comp rx=0 tx=0).stop 2024-03-30T00:44:15.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.879+0000 7f0e51800640 1 -- 172.21.15.66:0/1300936472 shutdown_connections 2024-03-30T00:44:15.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.879+0000 7f0e51800640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e4c15a950 0x7f0e4c153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:15.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.879+0000 7f0e51800640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e4c0d05e0 0x7f0e4c153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:15.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.879+0000 7f0e51800640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0e28066690 0x7f0e28068b50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:15.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.879+0000 7f0e51800640 1 --2- 172.21.15.66:0/1300936472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4c0caff0 0x7f0e4c155270 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:15.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.879+0000 7f0e51800640 1 -- 172.21.15.66:0/1300936472 >> 172.21.15.66:0/1300936472 conn(0x7f0e4c089fb0 msgr2=0x7f0e4c0aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:15.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.883+0000 7f0e51800640 1 -- 172.21.15.66:0/1300936472 shutdown_connections 2024-03-30T00:44:15.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:15.883+0000 7f0e51800640 1 -- 172.21.15.66:0/1300936472 wait complete. 2024-03-30T00:44:15.921 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T00:44:16.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d8800640 1 Processor -- start 2024-03-30T00:44:16.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d8800640 1 -- start start 2024-03-30T00:44:16.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d8800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f26d00b0290 0x7f26d00b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d8800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f26d015c200 0x7f26d015c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d8800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d00afdf0 con 0x7f26d0160490 2024-03-30T00:44:16.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d8800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d00b15d0 con 0x7f26d015c200 2024-03-30T00:44:16.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d8800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d00b1010 con 0x7f26d00b0290 2024-03-30T00:44:16.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d6800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f26d015c200 0x7f26d015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d5e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f26d0160490 0x7f26d0160870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:38704/0 (socket says 172.21.15.66:38704) 2024-03-30T00:44:16.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d5e00640 1 -- 172.21.15.66:0/2629243781 learned_addr learned my addr 172.21.15.66:0/2629243781 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:16.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d5400640 1 --2- 172.21.15.66:0/2629243781 >> v2:172.21.15.156:3300/0 conn(0x7f26d00b0290 0x7f26d00b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d6800640 1 -- 172.21.15.66:0/2629243781 >> v2:172.21.15.156:3300/0 conn(0x7f26d00b0290 msgr2=0x7f26d00b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:16.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d6800640 1 --2- 172.21.15.66:0/2629243781 >> v2:172.21.15.156:3300/0 conn(0x7f26d00b0290 0x7f26d00b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d4a00640 1 -- 172.21.15.66:0/2629243781 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436591746 0 0) 0x7f26d00afdf0 con 0x7f26d0160490 2024-03-30T00:44:16.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d6800640 1 -- 172.21.15.66:0/2629243781 >> v1:172.21.15.66:6789/0 conn(0x7f26d0160490 legacy=0x7f26d0160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.059+0000 7f26d6800640 1 -- 172.21.15.66:0/2629243781 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d00b12f0 con 0x7f26d015c200 2024-03-30T00:44:16.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d6800640 1 --2- 172.21.15.66:0/2629243781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d015c200 0x7f26d015c630 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f26d0071f40 tx=0x7f26cc01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a6c3c0a063767a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:16.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d4a00640 1 -- 172.21.15.66:0/2629243781 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0a59c0 con 0x7f26d015c200 2024-03-30T00:44:16.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d4a00640 1 -- 172.21.15.66:0/2629243781 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26cc0a5b20 con 0x7f26d015c200 2024-03-30T00:44:16.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d4a00640 1 -- 172.21.15.66:0/2629243781 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0ad440 con 0x7f26d015c200 2024-03-30T00:44:16.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d8800640 1 -- 172.21.15.66:0/2629243781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d015c200 msgr2=0x7f26d015c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d8800640 1 --2- 172.21.15.66:0/2629243781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d015c200 0x7f26d015c630 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f26d0071f40 tx=0x7f26cc01c900 comp rx=0 tx=0).stop 2024-03-30T00:44:16.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d8800640 1 -- 172.21.15.66:0/2629243781 shutdown_connections 2024-03-30T00:44:16.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d8800640 1 --2- 172.21.15.66:0/2629243781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d015c200 0x7f26d015c630 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d8800640 1 --2- 172.21.15.66:0/2629243781 >> v2:172.21.15.156:3300/0 conn(0x7f26d00b0290 0x7f26d00b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d8800640 1 -- 172.21.15.66:0/2629243781 >> 172.21.15.66:0/2629243781 conn(0x7f26d0099870 msgr2=0x7f26d0089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:16.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d8800640 1 -- 172.21.15.66:0/2629243781 shutdown_connections 2024-03-30T00:44:16.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d8800640 1 -- 172.21.15.66:0/2629243781 wait complete. 2024-03-30T00:44:16.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.063+0000 7f26d8800640 1 Processor -- start 2024-03-30T00:44:16.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d8800640 1 -- start start 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d8800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d00b0290 0x7f26d01f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d8800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26d015c200 0x7f26d01f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d8800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d0160490 0x7f26d01f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d8800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d015fdd0 con 0x7f26d00b0290 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d8800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d01f5940 con 0x7f26d015c200 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d8800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d015faf0 con 0x7f26d0160490 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d00b0290 0x7f26d01f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d00b0290 0x7f26d01f62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60698/0 (socket says 172.21.15.66:60698) 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d5e00640 1 -- 172.21.15.66:0/3340255146 learned_addr learned my addr 172.21.15.66:0/3340255146 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d5400640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26d015c200 0x7f26d01f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d6800640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d0160490 0x7f26d01f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d5e00640 1 -- 172.21.15.66:0/3340255146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d0160490 msgr2=0x7f26d01f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d5e00640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d0160490 0x7f26d01f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d5e00640 1 -- 172.21.15.66:0/3340255146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26d015c200 msgr2=0x7f26d01f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d5e00640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26d015c200 0x7f26d01f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d5e00640 1 -- 172.21.15.66:0/3340255146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc009a70 con 0x7f26d00b0290 2024-03-30T00:44:16.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d5400640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26d015c200 0x7f26d01f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:16.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d5e00640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d00b0290 0x7f26d01f62f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c480 tx=0x7f26c400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:16.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26ca000640 1 -- 172.21.15.66:0/3340255146 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c40074e0 con 0x7f26d00b0290 2024-03-30T00:44:16.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26ca000640 1 -- 172.21.15.66:0/3340255146 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c4007660 con 0x7f26d00b0290 2024-03-30T00:44:16.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26ca000640 1 -- 172.21.15.66:0/3340255146 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4004a30 con 0x7f26d00b0290 2024-03-30T00:44:16.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d8800640 1 -- 172.21.15.66:0/3340255146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d01f5bc0 con 0x7f26d00b0290 2024-03-30T00:44:16.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d8800640 1 -- 172.21.15.66:0/3340255146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d00ceff0 con 0x7f26d00b0290 2024-03-30T00:44:16.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.067+0000 7f26d8800640 1 -- 172.21.15.66:0/3340255146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d00cf240 con 0x7f26d00b0290 2024-03-30T00:44:16.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.071+0000 7f26ca000640 1 -- 172.21.15.66:0/3340255146 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f26c4004b90 con 0x7f26d00b0290 2024-03-30T00:44:16.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.071+0000 7f26ca000640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26b40666c0 0x7f26b4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.071+0000 7f26ca000640 1 -- 172.21.15.66:0/3340255146 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f26c40833f0 con 0x7f26d00b0290 2024-03-30T00:44:16.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.071+0000 7f26d5400640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26b40666c0 0x7f26b4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.071+0000 7f26d5400640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26b40666c0 0x7f26b4068b80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f26d009e4b0 tx=0x7f26c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:16.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.075+0000 7f26ca000640 1 -- 172.21.15.66:0/3340255146 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f26c4050670 con 0x7f26d00b0290 2024-03-30T00:44:16.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.271+0000 7f26d8800640 1 -- 172.21.15.66:0/3340255146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f26d0066c30 con 0x7f26d00b0290 2024-03-30T00:44:16.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.275+0000 7f26ca000640 1 -- 172.21.15.66:0/3340255146 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v39) v1 ==== 174+0+0 (secure 0 0 0) 0x7f26c4053c90 con 0x7f26d00b0290 2024-03-30T00:44:16.285 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 11 and data pool 12 2024-03-30T00:44:16.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 -- 172.21.15.66:0/3340255146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26b40666c0 msgr2=0x7f26b4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26b40666c0 0x7f26b4068b80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f26d009e4b0 tx=0x7f26c0008040 comp rx=0 tx=0).stop 2024-03-30T00:44:16.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 -- 172.21.15.66:0/3340255146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d00b0290 msgr2=0x7f26d01f62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d00b0290 0x7f26d01f62f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c480 tx=0x7f26c400c940 comp rx=0 tx=0).stop 2024-03-30T00:44:16.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 -- 172.21.15.66:0/3340255146 shutdown_connections 2024-03-30T00:44:16.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26d0160490 0x7f26d01f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26d015c200 0x7f26d01f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f26b40666c0 0x7f26b4068b80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 --2- 172.21.15.66:0/3340255146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26d00b0290 0x7f26d01f62f0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 -- 172.21.15.66:0/3340255146 >> 172.21.15.66:0/3340255146 conn(0x7f26d0099870 msgr2=0x7f26d0167790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:16.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 -- 172.21.15.66:0/3340255146 shutdown_connections 2024-03-30T00:44:16.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.283+0000 7f26d8800640 1 -- 172.21.15.66:0/3340255146 wait complete. 2024-03-30T00:44:16.327 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:16.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.459+0000 7fdae8400640 1 Processor -- start 2024-03-30T00:44:16.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.459+0000 7fdae8400640 1 -- start start 2024-03-30T00:44:16.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.459+0000 7fdae8400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdae01610c0 0x7fdae01604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.459+0000 7fdae8400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdae01609f0 0x7fdae00abc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.459+0000 7fdae8400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae006da20 con 0x7fdae0157060 2024-03-30T00:44:16.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.459+0000 7fdae8400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae0157440 con 0x7fdae01610c0 2024-03-30T00:44:16.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.459+0000 7fdae8400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae0160ea0 con 0x7fdae01609f0 2024-03-30T00:44:16.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdae5000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdae01610c0 0x7fdae01604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdae5000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdae01610c0 0x7fdae01604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:42104/0 (socket says 172.21.15.66:42104) 2024-03-30T00:44:16.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdae6400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdae01609f0 0x7fdae00abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdae5000640 1 -- 172.21.15.66:0/3836911463 learned_addr learned my addr 172.21.15.66:0/3836911463 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:16.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdae5a00640 1 --1- 172.21.15.66:0/3836911463 >> v1:172.21.15.156:6789/0 conn(0x7fdae0157060 0x7fdae006d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:34722/0 (socket says 172.21.15.66:34722) 2024-03-30T00:44:16.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdae5000640 1 -- 172.21.15.66:0/3836911463 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdae01609f0 msgr2=0x7fdae00abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:16.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdae5000640 1 --2- 172.21.15.66:0/3836911463 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdae01609f0 0x7fdae00abc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdae5000640 1 -- 172.21.15.66:0/3836911463 >> v1:172.21.15.156:6789/0 conn(0x7fdae0157060 legacy=0x7fdae006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdae5000640 1 -- 172.21.15.66:0/3836911463 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae006dc60 con 0x7fdae01610c0 2024-03-30T00:44:16.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdae5000640 1 --2- 172.21.15.66:0/3836911463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae01610c0 0x7fdae01604b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009870 tx=0x7fdacc01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7679318e43a0fba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:16.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdadfe00640 1 -- 172.21.15.66:0/3836911463 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0ab600 con 0x7fdae01610c0 2024-03-30T00:44:16.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdadfe00640 1 -- 172.21.15.66:0/3836911463 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc0a7050 con 0x7fdae01610c0 2024-03-30T00:44:16.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.463+0000 7fdadfe00640 1 -- 172.21.15.66:0/3836911463 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0ab980 con 0x7fdae01610c0 2024-03-30T00:44:16.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 -- 172.21.15.66:0/3836911463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae01610c0 msgr2=0x7fdae01604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 --2- 172.21.15.66:0/3836911463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae01610c0 0x7fdae01604b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009870 tx=0x7fdacc01c440 comp rx=0 tx=0).stop 2024-03-30T00:44:16.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 -- 172.21.15.66:0/3836911463 shutdown_connections 2024-03-30T00:44:16.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 --2- 172.21.15.66:0/3836911463 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdae01609f0 0x7fdae00abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 --2- 172.21.15.66:0/3836911463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae01610c0 0x7fdae01604b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 -- 172.21.15.66:0/3836911463 >> 172.21.15.66:0/3836911463 conn(0x7fdae0071750 msgr2=0x7fdae0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:16.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 -- 172.21.15.66:0/3836911463 shutdown_connections 2024-03-30T00:44:16.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 -- 172.21.15.66:0/3836911463 wait complete. 2024-03-30T00:44:16.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 Processor -- start 2024-03-30T00:44:16.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 -- start start 2024-03-30T00:44:16.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdae0157060 0x7fdae01594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdae01609f0 0x7fdae0069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae01610c0 0x7fdae0146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae0154c20 con 0x7fdae0157060 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae0159d00 con 0x7fdae01610c0 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae8400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae0159fe0 con 0x7fdae01609f0 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae5a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdae0157060 0x7fdae01594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae5a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdae0157060 0x7fdae01594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60706/0 (socket says 172.21.15.66:60706) 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae5a00640 1 -- 172.21.15.66:0/4128049344 learned_addr learned my addr 172.21.15.66:0/4128049344 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae5000640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdae01609f0 0x7fdae0069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.467+0000 7fdae6400640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae01610c0 0x7fdae0146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdae5a00640 1 -- 172.21.15.66:0/4128049344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdae01609f0 msgr2=0x7fdae0069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdae5a00640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdae01609f0 0x7fdae0069ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdae5a00640 1 -- 172.21.15.66:0/4128049344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae01610c0 msgr2=0x7fdae0146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdae5a00640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae01610c0 0x7fdae0146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdae5a00640 1 -- 172.21.15.66:0/4128049344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc0094d0 con 0x7fdae0157060 2024-03-30T00:44:16.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdae5000640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdae01609f0 0x7fdae0069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:16.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdae5a00640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdae0157060 0x7fdae01594b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c430 tx=0x7fdad400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:16.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdadd600640 1 -- 172.21.15.66:0/4128049344 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad40074e0 con 0x7fdae0157060 2024-03-30T00:44:16.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdadd600640 1 -- 172.21.15.66:0/4128049344 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad4007640 con 0x7fdae0157060 2024-03-30T00:44:16.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdadd600640 1 -- 172.21.15.66:0/4128049344 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad4004a10 con 0x7fdae0157060 2024-03-30T00:44:16.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdae8400640 1 -- 172.21.15.66:0/4128049344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae01545d0 con 0x7fdae0157060 2024-03-30T00:44:16.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdae8400640 1 -- 172.21.15.66:0/4128049344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae01599f0 con 0x7fdae0157060 2024-03-30T00:44:16.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdae8400640 1 -- 172.21.15.66:0/4128049344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae0066c30 con 0x7fdae0157060 2024-03-30T00:44:16.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.471+0000 7fdadd600640 1 -- 172.21.15.66:0/4128049344 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdad4004b70 con 0x7fdae0157060 2024-03-30T00:44:16.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.475+0000 7fdadd600640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdac0066690 0x7fdac0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.475+0000 7fdadd600640 1 -- 172.21.15.66:0/4128049344 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdad4083aa0 con 0x7fdae0157060 2024-03-30T00:44:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.475+0000 7fdae5000640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdac0066690 0x7fdac0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.475+0000 7fdae5000640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdac0066690 0x7fdac0068b50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0ad000 tx=0x7fdacc0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:16.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.479+0000 7fdadd600640 1 -- 172.21.15.66:0/4128049344 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdad4050ce0 con 0x7fdae0157060 2024-03-30T00:44:16.493 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:44:16.496 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:44:16.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.639+0000 7fdae8400640 1 -- 172.21.15.66:0/4128049344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdae006da20 con 0x7fdae0157060 2024-03-30T00:44:16.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdadd600640 1 -- 172.21.15.66:0/4128049344 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16322 (secure 0 0 0) 0x7fdae006da20 con 0x7fdae0157060 2024-03-30T00:44:16.650 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:16.650 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":81,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:16.273876+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:44:14.685880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-30T00:44:15.260373+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 -- 172.21.15.66:0/4128049344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdac0066690 msgr2=0x7fdac0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdac0066690 0x7fdac0068b50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0ad000 tx=0x7fdacc0a6000 comp rx=0 tx=0).stop 2024-03-30T00:44:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 -- 172.21.15.66:0/4128049344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdae0157060 msgr2=0x7fdae01594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdae0157060 0x7fdae01594b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c430 tx=0x7fdad400c8f0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 -- 172.21.15.66:0/4128049344 shutdown_connections 2024-03-30T00:44:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdae01610c0 0x7fdae0146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdae01609f0 0x7fdae0069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdac0066690 0x7fdac0068b50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 --2- 172.21.15.66:0/4128049344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdae0157060 0x7fdae01594b0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 -- 172.21.15.66:0/4128049344 >> 172.21.15.66:0/4128049344 conn(0x7fdae0071750 msgr2=0x7fdae00aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 -- 172.21.15.66:0/4128049344 shutdown_connections 2024-03-30T00:44:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.643+0000 7fdae8400640 1 -- 172.21.15.66:0/4128049344 wait complete. 2024-03-30T00:44:16.670 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:16.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd491400640 1 Processor -- start 2024-03-30T00:44:16.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd491400640 1 -- start start 2024-03-30T00:44:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd491400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd48c157060 0x7fd48c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd491400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd48c1610c0 0x7fd48c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd491400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd48c1609f0 0x7fd48c15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd491400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c157440 con 0x7fd48c157060 2024-03-30T00:44:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd491400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c160ea0 con 0x7fd48c1609f0 2024-03-30T00:44:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd491400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c06dc60 con 0x7fd48c1610c0 2024-03-30T00:44:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd48a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd48c157060 0x7fd48c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd48a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd48c157060 0x7fd48c06d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60716/0 (socket says 172.21.15.66:60716) 2024-03-30T00:44:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd48a800640 1 -- 172.21.15.66:0/814122430 learned_addr learned my addr 172.21.15.66:0/814122430 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd481e00640 1 --2- 172.21.15.66:0/814122430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd48c1610c0 0x7fd48c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.803+0000 7fd48b200640 1 --2- 172.21.15.66:0/814122430 >> v2:172.21.15.156:3300/0 conn(0x7fd48c1609f0 0x7fd48c15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd48a800640 1 -- 172.21.15.66:0/814122430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd48c1610c0 msgr2=0x7fd48c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd48a800640 1 --2- 172.21.15.66:0/814122430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd48c1610c0 0x7fd48c1604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd48a800640 1 -- 172.21.15.66:0/814122430 >> v2:172.21.15.156:3300/0 conn(0x7fd48c1609f0 msgr2=0x7fd48c15cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd48a800640 1 --2- 172.21.15.66:0/814122430 >> v2:172.21.15.156:3300/0 conn(0x7fd48c1609f0 0x7fd48c15cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd48a800640 1 -- 172.21.15.66:0/814122430 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c06df40 con 0x7fd48c157060 2024-03-30T00:44:16.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd48a800640 1 --2- 172.21.15.66:0/814122430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd48c157060 0x7fd48c06d310 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd48c161580 tx=0x7fd47c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e7e800d6c407fc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd489e00640 1 -- 172.21.15.66:0/814122430 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c0a5640 con 0x7fd48c157060 2024-03-30T00:44:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd489e00640 1 -- 172.21.15.66:0/814122430 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47c0a57a0 con 0x7fd48c157060 2024-03-30T00:44:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd489e00640 1 -- 172.21.15.66:0/814122430 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c0a5b30 con 0x7fd48c157060 2024-03-30T00:44:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd491400640 1 -- 172.21.15.66:0/814122430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd48c157060 msgr2=0x7fd48c06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd491400640 1 --2- 172.21.15.66:0/814122430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd48c157060 0x7fd48c06d310 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd48c161580 tx=0x7fd47c01c8d0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd491400640 1 -- 172.21.15.66:0/814122430 shutdown_connections 2024-03-30T00:44:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd491400640 1 --2- 172.21.15.66:0/814122430 >> v2:172.21.15.156:3300/0 conn(0x7fd48c1609f0 0x7fd48c15cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd491400640 1 --2- 172.21.15.66:0/814122430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd48c1610c0 0x7fd48c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd491400640 1 --2- 172.21.15.66:0/814122430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd48c157060 0x7fd48c06d310 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.807+0000 7fd491400640 1 -- 172.21.15.66:0/814122430 >> 172.21.15.66:0/814122430 conn(0x7fd48c071750 msgr2=0x7fd48c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 -- 172.21.15.66:0/814122430 shutdown_connections 2024-03-30T00:44:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 -- 172.21.15.66:0/814122430 wait complete. 2024-03-30T00:44:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 Processor -- start 2024-03-30T00:44:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 -- start start 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd48c157060 0x7fd48c14f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd48c1609f0 0x7fd48c14d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd48c1610c0 0x7fd48c154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c151c60 con 0x7fd48c1610c0 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c1534b0 con 0x7fd48c1609f0 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c152490 con 0x7fd48c157060 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd48b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd48c1610c0 0x7fd48c154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd48b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd48c1610c0 0x7fd48c154810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60730/0 (socket says 172.21.15.66:60730) 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd48b200640 1 -- 172.21.15.66:0/3205594914 learned_addr learned my addr 172.21.15.66:0/3205594914 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd481e00640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd48c1609f0 0x7fd48c14d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd48a800640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd48c157060 0x7fd48c14f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd48b200640 1 -- 172.21.15.66:0/3205594914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd48c157060 msgr2=0x7fd48c14f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd48b200640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd48c157060 0x7fd48c14f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd48b200640 1 -- 172.21.15.66:0/3205594914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd48c1609f0 msgr2=0x7fd48c14d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd48b200640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd48c1609f0 0x7fd48c14d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd48b200640 1 -- 172.21.15.66:0/3205594914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c009b30 con 0x7fd48c1610c0 2024-03-30T00:44:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd48a800640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd48c157060 0x7fd48c14f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd481e00640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd48c1609f0 0x7fd48c14d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd48b200640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd48c1610c0 0x7fd48c154810 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd48400d830 tx=0x7fd48400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd483400640 1 -- 172.21.15.66:0/3205594914 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd484009660 con 0x7fd48c1610c0 2024-03-30T00:44:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd483400640 1 -- 172.21.15.66:0/3205594914 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4840097c0 con 0x7fd48c1610c0 2024-03-30T00:44:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 -- 172.21.15.66:0/3205594914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48c1531d0 con 0x7fd48c1610c0 2024-03-30T00:44:16.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd483400640 1 -- 172.21.15.66:0/3205594914 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd484009b70 con 0x7fd48c1610c0 2024-03-30T00:44:16.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.811+0000 7fd491400640 1 -- 172.21.15.66:0/3205594914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c152120 con 0x7fd48c1610c0 2024-03-30T00:44:16.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.815+0000 7fd491400640 1 -- 172.21.15.66:0/3205594914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c066c30 con 0x7fd48c1610c0 2024-03-30T00:44:16.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.815+0000 7fd483400640 1 -- 172.21.15.66:0/3205594914 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd484009cd0 con 0x7fd48c1610c0 2024-03-30T00:44:16.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.815+0000 7fd483400640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd468066690 0x7fd468068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:16.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.815+0000 7fd483400640 1 -- 172.21.15.66:0/3205594914 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd484084a40 con 0x7fd48c1610c0 2024-03-30T00:44:16.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.815+0000 7fd48a800640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd468066690 0x7fd468068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:16.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.815+0000 7fd48a800640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd468066690 0x7fd468068b50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0ae000 tx=0x7fd47c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:16.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.819+0000 7fd483400640 1 -- 172.21.15.66:0/3205594914 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd484051bd0 con 0x7fd48c1610c0 2024-03-30T00:44:16.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.983+0000 7fd491400640 1 -- 172.21.15.66:0/3205594914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd48c1525d0 con 0x7fd48c1610c0 2024-03-30T00:44:16.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd483400640 1 -- 172.21.15.66:0/3205594914 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16322 (secure 0 0 0) 0x7fd484018090 con 0x7fd48c1610c0 2024-03-30T00:44:16.993 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:16.994 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":81,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:16.273876+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:44:14.685880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-30T00:44:15.260373+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:16.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 -- 172.21.15.66:0/3205594914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd468066690 msgr2=0x7fd468068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd468066690 0x7fd468068b50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0ae000 tx=0x7fd47c0a7070 comp rx=0 tx=0).stop 2024-03-30T00:44:16.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 -- 172.21.15.66:0/3205594914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd48c1610c0 msgr2=0x7fd48c154810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:16.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd48c1610c0 0x7fd48c154810 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd48400d830 tx=0x7fd48400dcf0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 -- 172.21.15.66:0/3205594914 shutdown_connections 2024-03-30T00:44:16.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd468066690 0x7fd468068b50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd48c1610c0 0x7fd48c154810 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd48c1609f0 0x7fd48c14d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 --2- 172.21.15.66:0/3205594914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd48c157060 0x7fd48c14f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:16.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 -- 172.21.15.66:0/3205594914 >> 172.21.15.66:0/3205594914 conn(0x7fd48c071750 msgr2=0x7fd48c0b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:16.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 -- 172.21.15.66:0/3205594914 shutdown_connections 2024-03-30T00:44:16.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:16.987+0000 7fd491400640 1 -- 172.21.15.66:0/3205594914 wait complete. 2024-03-30T00:44:17.015 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T00:44:17.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e14400640 1 Processor -- start 2024-03-30T00:44:17.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e14400640 1 -- start start 2024-03-30T00:44:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e14400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1e0c0ce520 0x7f1e0c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e14400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0c0cabb0 0x7f1e0c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e14400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c06d070 con 0x7f1e0c15a640 2024-03-30T00:44:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e14400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e0c0cbba0 con 0x7f1e0c0ce520 2024-03-30T00:44:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e14400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0cee70 con 0x7f1e0c0cabb0 2024-03-30T00:44:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e11a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1e0c15a640 0x7f1e0c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:38712/0 (socket says 172.21.15.66:38712) 2024-03-30T00:44:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e11a00640 1 -- 172.21.15.66:0/1737869392 learned_addr learned my addr 172.21.15.66:0/1737869392 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e11000640 1 --2- 172.21.15.66:0/1737869392 >> v2:172.21.15.66:3300/0 conn(0x7f1e0c0ce520 0x7f1e0c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e12400640 1 --2- 172.21.15.66:0/1737869392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0c0cabb0 0x7f1e0c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:17.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e0be00640 1 -- 172.21.15.66:0/1737869392 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558930490 0 0) 0x7f1e0c06d070 con 0x7f1e0c15a640 2024-03-30T00:44:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e0be00640 1 -- 172.21.15.66:0/1737869392 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df4003530 con 0x7f1e0c15a640 2024-03-30T00:44:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e11000640 1 -- 172.21.15.66:0/1737869392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0c0cabb0 msgr2=0x7f1e0c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e11000640 1 --2- 172.21.15.66:0/1737869392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0c0cabb0 0x7f1e0c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e11000640 1 -- 172.21.15.66:0/1737869392 >> v1:172.21.15.66:6789/0 conn(0x7f1e0c15a640 legacy=0x7f1e0c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:17.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.151+0000 7f1e11000640 1 -- 172.21.15.66:0/1737869392 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c0cb8c0 con 0x7f1e0c0ce520 2024-03-30T00:44:17.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e11000640 1 --2- 172.21.15.66:0/1737869392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c0ce520 0x7f1e0c0ce930 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1df80097c0 tx=0x7f1df801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533de0d8b9f365b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e0be00640 1 -- 172.21.15.66:0/1737869392 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df80ab670 con 0x7f1e0c0ce520 2024-03-30T00:44:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e0be00640 1 -- 172.21.15.66:0/1737869392 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df80ab7d0 con 0x7f1e0c0ce520 2024-03-30T00:44:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e0be00640 1 -- 172.21.15.66:0/1737869392 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df80abb60 con 0x7f1e0c0ce520 2024-03-30T00:44:17.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e14400640 1 -- 172.21.15.66:0/1737869392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c0ce520 msgr2=0x7f1e0c0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e14400640 1 --2- 172.21.15.66:0/1737869392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c0ce520 0x7f1e0c0ce930 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1df80097c0 tx=0x7f1df801c470 comp rx=0 tx=0).stop 2024-03-30T00:44:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e14400640 1 -- 172.21.15.66:0/1737869392 shutdown_connections 2024-03-30T00:44:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e14400640 1 --2- 172.21.15.66:0/1737869392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0c0cabb0 0x7f1e0c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e14400640 1 --2- 172.21.15.66:0/1737869392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c0ce520 0x7f1e0c0ce930 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e14400640 1 -- 172.21.15.66:0/1737869392 >> 172.21.15.66:0/1737869392 conn(0x7f1e0c069f60 msgr2=0x7f1e0c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e14400640 1 -- 172.21.15.66:0/1737869392 shutdown_connections 2024-03-30T00:44:17.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e14400640 1 -- 172.21.15.66:0/1737869392 wait complete. 2024-03-30T00:44:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e14400640 1 Processor -- start 2024-03-30T00:44:17.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.155+0000 7f1e14400640 1 -- start start 2024-03-30T00:44:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e14400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e0c0cabb0 0x7f1e0c0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e14400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0c0ce520 0x7f1e0c0aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e14400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c15a640 0x7f1e0c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e14400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c15dc30 con 0x7f1e0c15a640 2024-03-30T00:44:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e14400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0cdda0 con 0x7f1e0c0cabb0 2024-03-30T00:44:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e14400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0ae4b0 con 0x7f1e0c0ce520 2024-03-30T00:44:17.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e12400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c15a640 0x7f1e0c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e12400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c15a640 0x7f1e0c15d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60748/0 (socket says 172.21.15.66:60748) 2024-03-30T00:44:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e12400640 1 -- 172.21.15.66:0/1306165129 learned_addr learned my addr 172.21.15.66:0/1306165129 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e11a00640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e0c0cabb0 0x7f1e0c0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e11000640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0c0ce520 0x7f1e0c0aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e12400640 1 -- 172.21.15.66:0/1306165129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0c0ce520 msgr2=0x7f1e0c0aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e12400640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0c0ce520 0x7f1e0c0aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e12400640 1 -- 172.21.15.66:0/1306165129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e0c0cabb0 msgr2=0x7f1e0c0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e12400640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e0c0cabb0 0x7f1e0c0cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e12400640 1 -- 172.21.15.66:0/1306165129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8009420 con 0x7f1e0c15a640 2024-03-30T00:44:17.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e12400640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c15a640 0x7f1e0c15d6f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1e04007e40 tx=0x7f1e0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:17.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e09600640 1 -- 172.21.15.66:0/1306165129 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04014620 con 0x7f1e0c15a640 2024-03-30T00:44:17.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e09600640 1 -- 172.21.15.66:0/1306165129 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0400ee90 con 0x7f1e0c15a640 2024-03-30T00:44:17.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e09600640 1 -- 172.21.15.66:0/1306165129 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e040149e0 con 0x7f1e0c15a640 2024-03-30T00:44:17.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e14400640 1 -- 172.21.15.66:0/1306165129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c0ae790 con 0x7f1e0c15a640 2024-03-30T00:44:17.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e14400640 1 -- 172.21.15.66:0/1306165129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c0ae1a0 con 0x7f1e0c15a640 2024-03-30T00:44:17.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.159+0000 7f1e14400640 1 -- 172.21.15.66:0/1306165129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c066c30 con 0x7f1e0c15a640 2024-03-30T00:44:17.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.163+0000 7f1e09600640 1 -- 172.21.15.66:0/1306165129 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1e04014b40 con 0x7f1e0c15a640 2024-03-30T00:44:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.163+0000 7f1e09600640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1dec0666c0 0x7f1dec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.163+0000 7f1e09600640 1 -- 172.21.15.66:0/1306165129 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1e04084b30 con 0x7f1e0c15a640 2024-03-30T00:44:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.163+0000 7f1e11a00640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1dec0666c0 0x7f1dec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:17.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.163+0000 7f1e11a00640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1dec0666c0 0x7f1dec068b80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1e000011f0 tx=0x7f1e00007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:17.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.167+0000 7f1e09600640 1 -- 172.21.15.66:0/1306165129 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1e04051e10 con 0x7f1e0c15a640 2024-03-30T00:44:17.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:17.367+0000 7f1e14400640 1 -- 172.21.15.66:0/1306165129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1e0c0ac3f0 con 0x7f1e0c15a640 2024-03-30T00:44:17.708 INFO:tasks.ceph.mds.b.smithi156.stderr:2024-03-30T00:44:17.703+0000 7fe86a200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:44:18.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.283+0000 7f1e09600640 1 -- 172.21.15.66:0/1306165129 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1e04055430 con 0x7f1e0c15a640 2024-03-30T00:44:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 -- 172.21.15.66:0/1306165129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1dec0666c0 msgr2=0x7f1dec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1dec0666c0 0x7f1dec068b80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1e000011f0 tx=0x7f1e00007d40 comp rx=0 tx=0).stop 2024-03-30T00:44:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 -- 172.21.15.66:0/1306165129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c15a640 msgr2=0x7f1e0c15d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c15a640 0x7f1e0c15d6f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1e04007e40 tx=0x7f1e0400e4b0 comp rx=0 tx=0).stop 2024-03-30T00:44:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 -- 172.21.15.66:0/1306165129 shutdown_connections 2024-03-30T00:44:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1dec0666c0 0x7f1dec068b80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e0c15a640 0x7f1e0c15d6f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:18.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0c0ce520 0x7f1e0c0aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:18.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 --2- 172.21.15.66:0/1306165129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e0c0cabb0 0x7f1e0c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:18.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 -- 172.21.15.66:0/1306165129 >> 172.21.15.66:0/1306165129 conn(0x7f1e0c069f60 msgr2=0x7f1e0c0c9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:18.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 -- 172.21.15.66:0/1306165129 shutdown_connections 2024-03-30T00:44:18.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.287+0000 7f1e14400640 1 -- 172.21.15.66:0/1306165129 wait complete. 2024-03-30T00:44:18.318 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T00:44:18.318 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T00:44:18.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc3ca00640 1 Processor -- start 2024-03-30T00:44:18.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc3ca00640 1 -- start start 2024-03-30T00:44:18.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc3ca00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdc3815db90 0x7fdc3815e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:18.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc3ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 0x7fdc38070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:18.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc3ca00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3806c410 con 0x7fdc380ce6b0 2024-03-30T00:44:18.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc3ca00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc38071520 con 0x7fdc3815db90 2024-03-30T00:44:18.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc3ca00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3806c6b0 con 0x7fdc38070b50 2024-03-30T00:44:18.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc36800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 0x7fdc38070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:18.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc36800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 0x7fdc38070fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:38788/0 (socket says 172.21.15.66:38788) 2024-03-30T00:44:18.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc36800640 1 -- 172.21.15.66:0/851407044 learned_addr learned my addr 172.21.15.66:0/851407044 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:18.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc2d400640 1 --2- 172.21.15.66:0/851407044 >> v2:172.21.15.156:3300/0 conn(0x7fdc3815db90 0x7fdc3815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:18.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc36800640 1 -- 172.21.15.66:0/851407044 >> v2:172.21.15.156:3300/0 conn(0x7fdc3815db90 msgr2=0x7fdc3815e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:18.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc36800640 1 --2- 172.21.15.66:0/851407044 >> v2:172.21.15.156:3300/0 conn(0x7fdc3815db90 0x7fdc3815e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:18.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc36800640 1 -- 172.21.15.66:0/851407044 >> v1:172.21.15.156:6789/0 conn(0x7fdc380ce6b0 legacy=0x7fdc380cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:18.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.459+0000 7fdc36800640 1 -- 172.21.15.66:0/851407044 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3806c990 con 0x7fdc38070b50 2024-03-30T00:44:18.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc36800640 1 --2- 172.21.15.66:0/851407044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 0x7fdc38070fe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009870 tx=0x7fdc3001c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=631255b28528caf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:18.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc35400640 1 -- 172.21.15.66:0/851407044 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc300ab620 con 0x7fdc38070b50 2024-03-30T00:44:18.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc35400640 1 -- 172.21.15.66:0/851407044 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc300ab780 con 0x7fdc38070b50 2024-03-30T00:44:18.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc35400640 1 -- 172.21.15.66:0/851407044 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc300abb10 con 0x7fdc38070b50 2024-03-30T00:44:18.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc3ca00640 1 -- 172.21.15.66:0/851407044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 msgr2=0x7fdc38070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:18.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc3ca00640 1 --2- 172.21.15.66:0/851407044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 0x7fdc38070fe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009870 tx=0x7fdc3001c460 comp rx=0 tx=0).stop 2024-03-30T00:44:18.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc3ca00640 1 -- 172.21.15.66:0/851407044 shutdown_connections 2024-03-30T00:44:18.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc3ca00640 1 --2- 172.21.15.66:0/851407044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 0x7fdc38070fe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:18.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc3ca00640 1 --2- 172.21.15.66:0/851407044 >> v2:172.21.15.156:3300/0 conn(0x7fdc3815db90 0x7fdc3815e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:18.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc3ca00640 1 -- 172.21.15.66:0/851407044 >> 172.21.15.66:0/851407044 conn(0x7fdc38066100 msgr2=0x7fdc38069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:18.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc3ca00640 1 -- 172.21.15.66:0/851407044 shutdown_connections 2024-03-30T00:44:18.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.463+0000 7fdc3ca00640 1 -- 172.21.15.66:0/851407044 wait complete. 2024-03-30T00:44:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc3ca00640 1 Processor -- start 2024-03-30T00:44:18.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc3ca00640 1 -- start start 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc3ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 0x7fdc38154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc3ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc380ce6b0 0x7fdc38149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc3ca00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc3815db90 0x7fdc38159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc3ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381551f0 con 0x7fdc380ce6b0 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc3ca00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381554d0 con 0x7fdc3815db90 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc3ca00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381557b0 con 0x7fdc38070b50 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc380ce6b0 0x7fdc38149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc380ce6b0 0x7fdc38149550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60764/0 (socket says 172.21.15.66:60764) 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2d400640 1 -- 172.21.15.66:0/2196262067 learned_addr learned my addr 172.21.15.66:0/2196262067 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc36800640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc3815db90 0x7fdc38159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc35e00640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 0x7fdc38154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2d400640 1 -- 172.21.15.66:0/2196262067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 msgr2=0x7fdc38154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:18.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2d400640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 0x7fdc38154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:18.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2d400640 1 -- 172.21.15.66:0/2196262067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc3815db90 msgr2=0x7fdc38159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:18.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2d400640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc3815db90 0x7fdc38159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:18.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2d400640 1 -- 172.21.15.66:0/2196262067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc300094d0 con 0x7fdc380ce6b0 2024-03-30T00:44:18.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc35e00640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 0x7fdc38154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:18.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2d400640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc380ce6b0 0x7fdc38149550 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000efc0 tx=0x7fdc2000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:18.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2ea00640 1 -- 172.21.15.66:0/2196262067 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20009b80 con 0x7fdc380ce6b0 2024-03-30T00:44:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2ea00640 1 -- 172.21.15.66:0/2196262067 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2000f040 con 0x7fdc380ce6b0 2024-03-30T00:44:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc2ea00640 1 -- 172.21.15.66:0/2196262067 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc200146a0 con 0x7fdc380ce6b0 2024-03-30T00:44:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.467+0000 7fdc3ca00640 1 -- 172.21.15.66:0/2196262067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc38155a90 con 0x7fdc380ce6b0 2024-03-30T00:44:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.471+0000 7fdc3ca00640 1 -- 172.21.15.66:0/2196262067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc38156290 con 0x7fdc380ce6b0 2024-03-30T00:44:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.471+0000 7fdc3ca00640 1 -- 172.21.15.66:0/2196262067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3806e320 con 0x7fdc380ce6b0 2024-03-30T00:44:18.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.471+0000 7fdc2ea00640 1 -- 172.21.15.66:0/2196262067 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdc20014800 con 0x7fdc380ce6b0 2024-03-30T00:44:18.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.471+0000 7fdc2ea00640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc140666c0 0x7fdc14068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:18.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.471+0000 7fdc2ea00640 1 -- 172.21.15.66:0/2196262067 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdc200850c0 con 0x7fdc380ce6b0 2024-03-30T00:44:18.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.471+0000 7fdc35e00640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc140666c0 0x7fdc14068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:18.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.471+0000 7fdc35e00640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc140666c0 0x7fdc14068b80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdc38151960 tx=0x7fdc28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:18.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.475+0000 7fdc2ea00640 1 -- 172.21.15.66:0/2196262067 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdc20052250 con 0x7fdc380ce6b0 2024-03-30T00:44:18.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:18.671+0000 7fdc3ca00640 1 -- 172.21.15.66:0/2196262067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdc38066c30 con 0x7fdc380ce6b0 2024-03-30T00:44:19.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc2ea00640 1 -- 172.21.15.66:0/2196262067 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdc20055870 con 0x7fdc380ce6b0 2024-03-30T00:44:19.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 -- 172.21.15.66:0/2196262067 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc140666c0 msgr2=0x7fdc14068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc140666c0 0x7fdc14068b80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdc38151960 tx=0x7fdc28009040 comp rx=0 tx=0).stop 2024-03-30T00:44:19.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 -- 172.21.15.66:0/2196262067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc380ce6b0 msgr2=0x7fdc38149550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc380ce6b0 0x7fdc38149550 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000efc0 tx=0x7fdc2000c490 comp rx=0 tx=0).stop 2024-03-30T00:44:19.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 -- 172.21.15.66:0/2196262067 shutdown_connections 2024-03-30T00:44:19.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc3815db90 0x7fdc38159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc140666c0 0x7fdc14068b80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc380ce6b0 0x7fdc38149550 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 --2- 172.21.15.66:0/2196262067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc38070b50 0x7fdc38154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 -- 172.21.15.66:0/2196262067 >> 172.21.15.66:0/2196262067 conn(0x7fdc38066100 msgr2=0x7fdc38150480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:19.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 -- 172.21.15.66:0/2196262067 shutdown_connections 2024-03-30T00:44:19.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.287+0000 7fdc3ca00640 1 -- 172.21.15.66:0/2196262067 wait complete. 2024-03-30T00:44:19.317 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T00:44:19.317 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:44:19.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.451+0000 7f1aa0200640 1 Processor -- start 2024-03-30T00:44:19.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.451+0000 7f1aa0200640 1 -- start start 2024-03-30T00:44:19.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1aa0200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a980cf2f0 0x7f1a9806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1aa0200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a980cb570 0x7f1a980cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1aa0200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a980cb0e0 con 0x7f1a9815ca80 2024-03-30T00:44:19.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1aa0200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a980cf7f0 con 0x7f1a980cb570 2024-03-30T00:44:19.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1aa0200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980cfad0 con 0x7f1a980cf2f0 2024-03-30T00:44:19.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1a9ce00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a980cf2f0 0x7f1a9806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1a9ce00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a980cf2f0 0x7f1a9806b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:52476/0 (socket says 172.21.15.66:52476) 2024-03-30T00:44:19.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1a9ce00640 1 -- 172.21.15.66:0/231572305 learned_addr learned my addr 172.21.15.66:0/231572305 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:19.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1a9e200640 1 --2- 172.21.15.66:0/231572305 >> v2:172.21.15.156:3300/0 conn(0x7f1a980cb570 0x7f1a980cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1a9ce00640 1 -- 172.21.15.66:0/231572305 >> v2:172.21.15.156:3300/0 conn(0x7f1a980cb570 msgr2=0x7f1a980cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:19.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1a9ce00640 1 --2- 172.21.15.66:0/231572305 >> v2:172.21.15.156:3300/0 conn(0x7f1a980cb570 0x7f1a980cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1a9ce00640 1 -- 172.21.15.66:0/231572305 >> v1:172.21.15.156:6789/0 conn(0x7f1a9815ca80 legacy=0x7f1a9806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1a9ce00640 1 -- 172.21.15.66:0/231572305 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a980cad70 con 0x7f1a980cf2f0 2024-03-30T00:44:19.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.455+0000 7f1a9e200640 1 --2- 172.21.15.66:0/231572305 >> v2:172.21.15.156:3300/0 conn(0x7f1a980cb570 0x7f1a980cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:19.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1a9ce00640 1 --2- 172.21.15.66:0/231572305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a980cf2f0 0x7f1a9806b7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009870 tx=0x7f1a8401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3632bbf5a0d9b05b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:19.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1a97e00640 1 -- 172.21.15.66:0/231572305 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a840a8a50 con 0x7f1a980cf2f0 2024-03-30T00:44:19.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1a97e00640 1 -- 172.21.15.66:0/231572305 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a840a8bb0 con 0x7f1a980cf2f0 2024-03-30T00:44:19.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1a97e00640 1 -- 172.21.15.66:0/231572305 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a840b0440 con 0x7f1a980cf2f0 2024-03-30T00:44:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1aa0200640 1 -- 172.21.15.66:0/231572305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a980cf2f0 msgr2=0x7f1a9806b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1aa0200640 1 --2- 172.21.15.66:0/231572305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a980cf2f0 0x7f1a9806b7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009870 tx=0x7f1a8401ed50 comp rx=0 tx=0).stop 2024-03-30T00:44:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1aa0200640 1 -- 172.21.15.66:0/231572305 shutdown_connections 2024-03-30T00:44:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1aa0200640 1 --2- 172.21.15.66:0/231572305 >> v2:172.21.15.156:3300/0 conn(0x7f1a980cb570 0x7f1a980cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1aa0200640 1 --2- 172.21.15.66:0/231572305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a980cf2f0 0x7f1a9806b7c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1aa0200640 1 -- 172.21.15.66:0/231572305 >> 172.21.15.66:0/231572305 conn(0x7f1a980b2090 msgr2=0x7f1a980b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1aa0200640 1 -- 172.21.15.66:0/231572305 shutdown_connections 2024-03-30T00:44:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1aa0200640 1 -- 172.21.15.66:0/231572305 wait complete. 2024-03-30T00:44:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1aa0200640 1 Processor -- start 2024-03-30T00:44:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.459+0000 7f1aa0200640 1 -- start start 2024-03-30T00:44:19.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1aa0200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a980cb570 0x7f1a9814f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1aa0200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a980cf2f0 0x7f1a980c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1aa0200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a9815ca80 0x7f1a980c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1aa0200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980b31c0 con 0x7f1a980cb570 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1aa0200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980c8bd0 con 0x7f1a980cf2f0 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1aa0200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980c8eb0 con 0x7f1a9815ca80 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a9d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a980cb570 0x7f1a9814f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a9d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a980cb570 0x7f1a9814f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53206/0 (socket says 172.21.15.66:53206) 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a9d800640 1 -- 172.21.15.66:0/1282776961 learned_addr learned my addr 172.21.15.66:0/1282776961 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a9ce00640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a980cf2f0 0x7f1a980c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a9e200640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a9815ca80 0x7f1a980c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a9d800640 1 -- 172.21.15.66:0/1282776961 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a9815ca80 msgr2=0x7f1a980c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a9d800640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a9815ca80 0x7f1a980c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a9d800640 1 -- 172.21.15.66:0/1282776961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a980cf2f0 msgr2=0x7f1a980c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a9d800640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a980cf2f0 0x7f1a980c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a9d800640 1 -- 172.21.15.66:0/1282776961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a90009a70 con 0x7f1a980cb570 2024-03-30T00:44:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a9d800640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a980cb570 0x7f1a9814f8f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00c3c0 tx=0x7f1a8c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a95600640 1 -- 172.21.15.66:0/1282776961 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0052d0 con 0x7f1a980cb570 2024-03-30T00:44:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a95600640 1 -- 172.21.15.66:0/1282776961 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c005430 con 0x7f1a980cb570 2024-03-30T00:44:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a95600640 1 -- 172.21.15.66:0/1282776961 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c003840 con 0x7f1a980cb570 2024-03-30T00:44:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1aa0200640 1 -- 172.21.15.66:0/1282776961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a840094d0 con 0x7f1a980cb570 2024-03-30T00:44:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1aa0200640 1 -- 172.21.15.66:0/1282776961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a980b3740 con 0x7f1a980cb570 2024-03-30T00:44:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1aa0200640 1 -- 172.21.15.66:0/1282776961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a980b3980 con 0x7f1a980cb570 2024-03-30T00:44:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.463+0000 7f1a95600640 1 -- 172.21.15.66:0/1282776961 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1a8c0039a0 con 0x7f1a980cb570 2024-03-30T00:44:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.467+0000 7f1a95600640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a78066690 0x7f1a78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.467+0000 7f1a95600640 1 -- 172.21.15.66:0/1282776961 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1a8c083860 con 0x7f1a980cb570 2024-03-30T00:44:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.467+0000 7f1a9ce00640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a78066690 0x7f1a78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.467+0000 7f1a9ce00640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a78066690 0x7f1a78068b50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1a840b2000 tx=0x7f1a84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:19.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.471+0000 7f1a95600640 1 -- 172.21.15.66:0/1282776961 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1a980b3980 con 0x7f1a980cb570 2024-03-30T00:44:19.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.663+0000 7f1aa0200640 1 -- 172.21.15.66:0/1282776961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a980b3980 con 0x7f1a980cb570 2024-03-30T00:44:19.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1a95600640 1 -- 172.21.15.66:0/1282776961 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4742 (secure 0 0 0) 0x7f1a980b3980 con 0x7f1a980cb570 2024-03-30T00:44:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:19.673 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:44:19.285136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/2089872680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":2089872680},{"type":"v1","addr":"172.21.15.156:6835","nonce":2089872680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:44:19.674 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 43 2024-03-30T00:44:19.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 -- 172.21.15.66:0/1282776961 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a78066690 msgr2=0x7f1a78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a78066690 0x7f1a78068b50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1a840b2000 tx=0x7f1a84009040 comp rx=0 tx=0).stop 2024-03-30T00:44:19.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 -- 172.21.15.66:0/1282776961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a980cb570 msgr2=0x7f1a9814f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a980cb570 0x7f1a9814f8f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00c3c0 tx=0x7f1a8c00c7c0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 -- 172.21.15.66:0/1282776961 shutdown_connections 2024-03-30T00:44:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a9815ca80 0x7f1a980c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a980cf2f0 0x7f1a980c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1a78066690 0x7f1a78068b50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 --2- 172.21.15.66:0/1282776961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a980cb570 0x7f1a9814f8f0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 -- 172.21.15.66:0/1282776961 >> 172.21.15.66:0/1282776961 conn(0x7f1a980b2090 msgr2=0x7f1a980cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 -- 172.21.15.66:0/1282776961 shutdown_connections 2024-03-30T00:44:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.667+0000 7f1aa0200640 1 -- 172.21.15.66:0/1282776961 wait complete. 2024-03-30T00:44:19.694 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:19.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.827+0000 7f982a000640 1 Processor -- start 2024-03-30T00:44:19.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.827+0000 7f982a000640 1 -- start start 2024-03-30T00:44:19.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.827+0000 7f982a000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cf2f0 0x7f982406b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.827+0000 7f982a000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f98240cb570 0x7f98240cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.827+0000 7f982a000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98240cb0e0 con 0x7f982415ca80 2024-03-30T00:44:19.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.827+0000 7f982a000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98240cf7f0 con 0x7f98240cb570 2024-03-30T00:44:19.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.827+0000 7f982a000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98240cfad0 con 0x7f98240cf2f0 2024-03-30T00:44:19.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.827+0000 7f9823e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f98240cb570 0x7f98240cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.827+0000 7f981be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cf2f0 0x7f982406b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.827+0000 7f9823e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f98240cb570 0x7f98240cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:42302/0 (socket says 172.21.15.66:42302) 2024-03-30T00:44:19.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.827+0000 7f9823e00640 1 -- 172.21.15.66:0/2130871407 learned_addr learned my addr 172.21.15.66:0/2130871407 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:19.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.831+0000 7f9823e00640 1 -- 172.21.15.66:0/2130871407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cf2f0 msgr2=0x7f982406b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:19.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.831+0000 7f9823e00640 1 --2- 172.21.15.66:0/2130871407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cf2f0 0x7f982406b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.831+0000 7f9823e00640 1 -- 172.21.15.66:0/2130871407 >> v1:172.21.15.156:6789/0 conn(0x7f982415ca80 legacy=0x7f982406b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.831+0000 7f9823e00640 1 -- 172.21.15.66:0/2130871407 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98240cad70 con 0x7f98240cb570 2024-03-30T00:44:19.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.831+0000 7f981be00640 1 --2- 172.21.15.66:0/2130871407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cf2f0 0x7f982406b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:19.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.831+0000 7f9823e00640 1 --2- 172.21.15.66:0/2130871407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98240cb570 0x7f98240cba20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f981c009e50 tx=0x7f981c01f210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ee72cb07ab306ef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:19.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.831+0000 7f9822a00640 1 -- 172.21.15.66:0/2130871407 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c0af400 con 0x7f98240cb570 2024-03-30T00:44:19.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.831+0000 7f9822a00640 1 -- 172.21.15.66:0/2130871407 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981c0ab050 con 0x7f98240cb570 2024-03-30T00:44:19.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.831+0000 7f9822a00640 1 -- 172.21.15.66:0/2130871407 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c0af780 con 0x7f98240cb570 2024-03-30T00:44:19.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.831+0000 7f982a000640 1 -- 172.21.15.66:0/2130871407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98240cb570 msgr2=0x7f98240cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.831+0000 7f982a000640 1 --2- 172.21.15.66:0/2130871407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98240cb570 0x7f98240cba20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f981c009e50 tx=0x7f981c01f210 comp rx=0 tx=0).stop 2024-03-30T00:44:19.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 -- 172.21.15.66:0/2130871407 shutdown_connections 2024-03-30T00:44:19.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 --2- 172.21.15.66:0/2130871407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98240cb570 0x7f98240cba20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 --2- 172.21.15.66:0/2130871407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cf2f0 0x7f982406b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 -- 172.21.15.66:0/2130871407 >> 172.21.15.66:0/2130871407 conn(0x7f98240b2090 msgr2=0x7f98240b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:19.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 -- 172.21.15.66:0/2130871407 shutdown_connections 2024-03-30T00:44:19.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 -- 172.21.15.66:0/2130871407 wait complete. 2024-03-30T00:44:19.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 Processor -- start 2024-03-30T00:44:19.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 -- start start 2024-03-30T00:44:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cb570 0x7f98240b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98240cf2f0 0x7f9824153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f982415ca80 0x7f98240c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98240c8350 con 0x7f982415ca80 2024-03-30T00:44:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98241542c0 con 0x7f98240cf2f0 2024-03-30T00:44:19.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f982a000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98240c52a0 con 0x7f98240cb570 2024-03-30T00:44:19.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f9823e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f982415ca80 0x7f98240c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f9823e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f982415ca80 0x7f98240c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53216/0 (socket says 172.21.15.66:53216) 2024-03-30T00:44:19.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f9823e00640 1 -- 172.21.15.66:0/407313985 learned_addr learned my addr 172.21.15.66:0/407313985 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:19.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f981be00640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98240cf2f0 0x7f9824153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f9823400640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cb570 0x7f98240b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f9823e00640 1 -- 172.21.15.66:0/407313985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cb570 msgr2=0x7f98240b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f9823e00640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cb570 0x7f98240b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f9823e00640 1 -- 172.21.15.66:0/407313985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98240cf2f0 msgr2=0x7f9824153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f9823e00640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98240cf2f0 0x7f9824153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f9823e00640 1 -- 172.21.15.66:0/407313985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c009420 con 0x7f982415ca80 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f981be00640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98240cf2f0 0x7f9824153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f9823400640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cb570 0x7f98240b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.835+0000 7f9823e00640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f982415ca80 0x7f98240c6320 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f981c0b2040 tx=0x7f981c01bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.839+0000 7f981b400640 1 -- 172.21.15.66:0/407313985 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c0afd30 con 0x7f982415ca80 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.839+0000 7f981b400640 1 -- 172.21.15.66:0/407313985 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981c009510 con 0x7f982415ca80 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.839+0000 7f981b400640 1 -- 172.21.15.66:0/407313985 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c0ba660 con 0x7f982415ca80 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.839+0000 7f982a000640 1 -- 172.21.15.66:0/407313985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c009b30 con 0x7f982415ca80 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.839+0000 7f982a000640 1 -- 172.21.15.66:0/407313985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98240c5840 con 0x7f982415ca80 2024-03-30T00:44:19.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.839+0000 7f982a000640 1 -- 172.21.15.66:0/407313985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9824066c30 con 0x7f982415ca80 2024-03-30T00:44:19.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.839+0000 7f981b400640 1 -- 172.21.15.66:0/407313985 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f981c0b6040 con 0x7f982415ca80 2024-03-30T00:44:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.839+0000 7f981b400640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9800066690 0x7f9800068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.839+0000 7f981b400640 1 -- 172.21.15.66:0/407313985 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f981c122470 con 0x7f982415ca80 2024-03-30T00:44:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.839+0000 7f9823400640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9800066690 0x7f9800068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.843+0000 7f9823400640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9800066690 0x7f9800068b50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9814000f70 tx=0x7f9814009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:19.843+0000 7f981b400640 1 -- 172.21.15.66:0/407313985 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f981c0ef6c0 con 0x7f982415ca80 2024-03-30T00:44:20.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.007+0000 7f982a000640 1 -- 172.21.15.66:0/407313985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98240b2bb0 con 0x7f982415ca80 2024-03-30T00:44:20.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.007+0000 7f981b400640 1 -- 172.21.15.66:0/407313985 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16309 (secure 0 0 0) 0x7f981c0f2ce0 con 0x7f982415ca80 2024-03-30T00:44:20.015 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:20.015 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":82,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:17.276711+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:44:14.685880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-30T00:44:15.260373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:20.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 -- 172.21.15.66:0/407313985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9800066690 msgr2=0x7f9800068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9800066690 0x7f9800068b50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9814000f70 tx=0x7f9814009040 comp rx=0 tx=0).stop 2024-03-30T00:44:20.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 -- 172.21.15.66:0/407313985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f982415ca80 msgr2=0x7f98240c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f982415ca80 0x7f98240c6320 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f981c0b2040 tx=0x7f981c01bfb0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 -- 172.21.15.66:0/407313985 shutdown_connections 2024-03-30T00:44:20.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9800066690 0x7f9800068b50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f982415ca80 0x7f98240c6320 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98240cf2f0 0x7f9824153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 --2- 172.21.15.66:0/407313985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98240cb570 0x7f98240b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 -- 172.21.15.66:0/407313985 >> 172.21.15.66:0/407313985 conn(0x7f98240b2090 msgr2=0x7f982406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:20.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 -- 172.21.15.66:0/407313985 shutdown_connections 2024-03-30T00:44:20.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.011+0000 7f982a000640 1 -- 172.21.15.66:0/407313985 wait complete. 2024-03-30T00:44:20.036 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T00:44:20.037 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:44:20.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f4462e00640 1 Processor -- start 2024-03-30T00:44:20.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f4462e00640 1 -- start start 2024-03-30T00:44:20.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f4462e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f445c15de30 0x7f445c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f4462e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f445c0af420 0x7f445c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f4462e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445c0cbb90 con 0x7f445c0ce740 2024-03-30T00:44:20.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f4462e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445c0ce1b0 con 0x7f445c15de30 2024-03-30T00:44:20.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f4462e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445c0afa10 con 0x7f445c0af420 2024-03-30T00:44:20.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f445be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f445c15de30 0x7f445c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f445be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f445c15de30 0x7f445c0cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53220/0 (socket says 172.21.15.66:53220) 2024-03-30T00:44:20.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f445be00640 1 -- 172.21.15.66:0/368298252 learned_addr learned my addr 172.21.15.66:0/368298252 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:20.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f4453400640 1 --2- 172.21.15.66:0/368298252 >> v2:172.21.15.156:3300/0 conn(0x7f445c0af420 0x7f445c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f445be00640 1 -- 172.21.15.66:0/368298252 >> v2:172.21.15.156:3300/0 conn(0x7f445c0af420 msgr2=0x7f445c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:20.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f445be00640 1 --2- 172.21.15.66:0/368298252 >> v2:172.21.15.156:3300/0 conn(0x7f445c0af420 0x7f445c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f445be00640 1 -- 172.21.15.66:0/368298252 >> v1:172.21.15.156:6789/0 conn(0x7f445c0ce740 legacy=0x7f445c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:20.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.171+0000 7f445be00640 1 -- 172.21.15.66:0/368298252 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c0cf430 con 0x7f445c15de30 2024-03-30T00:44:20.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f445be00640 1 --2- 172.21.15.66:0/368298252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f445c15de30 0x7f445c0cc040 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f445c070080 tx=0x7f444c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fef4c69af50341 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f445b400640 1 -- 172.21.15.66:0/368298252 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0a5780 con 0x7f445c15de30 2024-03-30T00:44:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f445b400640 1 -- 172.21.15.66:0/368298252 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c0a58e0 con 0x7f445c15de30 2024-03-30T00:44:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f445b400640 1 -- 172.21.15.66:0/368298252 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0a5c70 con 0x7f445c15de30 2024-03-30T00:44:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f4462e00640 1 -- 172.21.15.66:0/368298252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f445c15de30 msgr2=0x7f445c0cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f4462e00640 1 --2- 172.21.15.66:0/368298252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f445c15de30 0x7f445c0cc040 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f445c070080 tx=0x7f444c01c960 comp rx=0 tx=0).stop 2024-03-30T00:44:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f4462e00640 1 -- 172.21.15.66:0/368298252 shutdown_connections 2024-03-30T00:44:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f4462e00640 1 --2- 172.21.15.66:0/368298252 >> v2:172.21.15.156:3300/0 conn(0x7f445c0af420 0x7f445c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f4462e00640 1 --2- 172.21.15.66:0/368298252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f445c15de30 0x7f445c0cc040 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f4462e00640 1 -- 172.21.15.66:0/368298252 >> 172.21.15.66:0/368298252 conn(0x7f445c0d0540 msgr2=0x7f445c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:20.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f4462e00640 1 -- 172.21.15.66:0/368298252 shutdown_connections 2024-03-30T00:44:20.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.175+0000 7f4462e00640 1 -- 172.21.15.66:0/368298252 wait complete. 2024-03-30T00:44:20.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4462e00640 1 Processor -- start 2024-03-30T00:44:20.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4462e00640 1 -- start start 2024-03-30T00:44:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4462e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f445c0af420 0x7f445c157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4462e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f445c0ce740 0x7f445c0c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4462e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f445c15de30 0x7f445c0c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4462e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0ca0f0 con 0x7f445c0af420 2024-03-30T00:44:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4462e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0c94e0 con 0x7f445c0ce740 2024-03-30T00:44:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4462e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0c97c0 con 0x7f445c15de30 2024-03-30T00:44:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f445be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f445c0af420 0x7f445c157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f445be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f445c0af420 0x7f445c157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53232/0 (socket says 172.21.15.66:53232) 2024-03-30T00:44:20.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f445be00640 1 -- 172.21.15.66:0/3362700928 learned_addr learned my addr 172.21.15.66:0/3362700928 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4453400640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f445c0ce740 0x7f445c0c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4460e00640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f445c15de30 0x7f445c0c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f445be00640 1 -- 172.21.15.66:0/3362700928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f445c15de30 msgr2=0x7f445c0c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f445be00640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f445c15de30 0x7f445c0c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f445be00640 1 -- 172.21.15.66:0/3362700928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f445c0ce740 msgr2=0x7f445c0c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f445be00640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f445c0ce740 0x7f445c0c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f445be00640 1 -- 172.21.15.66:0/3362700928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c009b30 con 0x7f445c0af420 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f445be00640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f445c0af420 0x7f445c157b20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f444c0af040 tx=0x7f444c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4458c00640 1 -- 172.21.15.66:0/3362700928 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0b8670 con 0x7f445c0af420 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4458c00640 1 -- 172.21.15.66:0/3362700928 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c0b87f0 con 0x7f445c0af420 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4458c00640 1 -- 172.21.15.66:0/3362700928 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0b8ba0 con 0x7f445c0af420 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4462e00640 1 -- 172.21.15.66:0/3362700928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c0c9aa0 con 0x7f445c0af420 2024-03-30T00:44:20.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4462e00640 1 -- 172.21.15.66:0/3362700928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c158060 con 0x7f445c0af420 2024-03-30T00:44:20.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.179+0000 7f4462e00640 1 -- 172.21.15.66:0/3362700928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445c0cdf60 con 0x7f445c0af420 2024-03-30T00:44:20.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.183+0000 7f4458c00640 1 -- 172.21.15.66:0/3362700928 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f444c0b8d00 con 0x7f445c0af420 2024-03-30T00:44:20.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.183+0000 7f4458c00640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44380666c0 0x7f4438068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.183+0000 7f4458c00640 1 -- 172.21.15.66:0/3362700928 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f444c11f2a0 con 0x7f445c0af420 2024-03-30T00:44:20.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.183+0000 7f4453400640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44380666c0 0x7f4438068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.183+0000 7f4453400640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44380666c0 0x7f4438068b80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f445c09e5a0 tx=0x7f4444008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:20.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.187+0000 7f4458c00640 1 -- 172.21.15.66:0/3362700928 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f444c0ec430 con 0x7f445c0af420 2024-03-30T00:44:20.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.351+0000 7f4462e00640 1 -- 172.21.15.66:0/3362700928 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f445c06ec10 con 0x7f44380666c0 2024-03-30T00:44:20.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.359+0000 7f4458c00640 1 -- 172.21.15.66:0/3362700928 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289123 (secure 0 0 0) 0x7f444400b670 con 0x7f44380666c0 2024-03-30T00:44:20.369 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:20.371 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:44:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 -- 172.21.15.66:0/3362700928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44380666c0 msgr2=0x7f4438068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44380666c0 0x7f4438068b80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f445c09e5a0 tx=0x7f4444008040 comp rx=0 tx=0).stop 2024-03-30T00:44:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 -- 172.21.15.66:0/3362700928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f445c0af420 msgr2=0x7f445c157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f445c0af420 0x7f445c157b20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f444c0af040 tx=0x7f444c019700 comp rx=0 tx=0).stop 2024-03-30T00:44:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 -- 172.21.15.66:0/3362700928 shutdown_connections 2024-03-30T00:44:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f445c15de30 0x7f445c0c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f445c0ce740 0x7f445c0c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44380666c0 0x7f4438068b80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 --2- 172.21.15.66:0/3362700928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f445c0af420 0x7f445c157b20 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 -- 172.21.15.66:0/3362700928 >> 172.21.15.66:0/3362700928 conn(0x7f445c0d0540 msgr2=0x7f445c0ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 -- 172.21.15.66:0/3362700928 shutdown_connections 2024-03-30T00:44:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.363+0000 7f4462e00640 1 -- 172.21.15.66:0/3362700928 wait complete. 2024-03-30T00:44:20.388 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-03-30T00:44:18.907951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249780,"kb_used_data":9004,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754724940,"statfs":{"total":773094113280,"available":772838338560,"internally_reserved":0,"allocated":9220096,"data_stored":4791336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":246402616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.044336"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288047+0000","last_change":"2024-03-30T00:44:16.304877+0000","last_active":"2024-03-30T00:44:17.288047+0000","last_peered":"2024-03-30T00:44:17.288047+0000","last_clean":"2024-03-30T00:44:17.288047+0000","last_became_active":"2024-03-30T00:44:16.304512+0000","last_became_peered":"2024-03-30T00:44:16.304512+0000","last_unstale":"2024-03-30T00:44:17.288047+0000","last_undegraded":"2024-03-30T00:44:17.288047+0000","last_fullsized":"2024-03-30T00:44:17.288047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:29.166147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575509+0000","last_change":"2024-03-30T00:44:16.313300+0000","last_active":"2024-03-30T00:44:17.575509+0000","last_peered":"2024-03-30T00:44:17.575509+0000","last_clean":"2024-03-30T00:44:17.575509+0000","last_became_active":"2024-03-30T00:44:16.312331+0000","last_became_peered":"2024-03-30T00:44:16.312331+0000","last_unstale":"2024-03-30T00:44:17.575509+0000","last_undegraded":"2024-03-30T00:44:17.575509+0000","last_fullsized":"2024-03-30T00:44:17.575509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:03:22.521314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287866+0000","last_change":"2024-03-30T00:44:16.296703+0000","last_active":"2024-03-30T00:44:17.287866+0000","last_peered":"2024-03-30T00:44:17.287866+0000","last_clean":"2024-03-30T00:44:17.287866+0000","last_became_active":"2024-03-30T00:44:16.296422+0000","last_became_peered":"2024-03-30T00:44:16.296422+0000","last_unstale":"2024-03-30T00:44:17.287866+0000","last_undegraded":"2024-03-30T00:44:17.287866+0000","last_fullsized":"2024-03-30T00:44:17.287866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:29.166147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572642+0000","last_change":"2024-03-30T00:44:16.300501+0000","last_active":"2024-03-30T00:44:17.572642+0000","last_peered":"2024-03-30T00:44:17.572642+0000","last_clean":"2024-03-30T00:44:17.572642+0000","last_became_active":"2024-03-30T00:44:16.300244+0000","last_became_peered":"2024-03-30T00:44:16.300244+0000","last_unstale":"2024-03-30T00:44:17.572642+0000","last_undegraded":"2024-03-30T00:44:17.572642+0000","last_fullsized":"2024-03-30T00:44:17.572642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:21:39.077425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574430+0000","last_change":"2024-03-30T00:44:16.293733+0000","last_active":"2024-03-30T00:44:17.574430+0000","last_peered":"2024-03-30T00:44:17.574430+0000","last_clean":"2024-03-30T00:44:17.574430+0000","last_became_active":"2024-03-30T00:44:16.293498+0000","last_became_peered":"2024-03-30T00:44:16.293498+0000","last_unstale":"2024-03-30T00:44:17.574430+0000","last_undegraded":"2024-03-30T00:44:17.574430+0000","last_fullsized":"2024-03-30T00:44:17.574430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:44:29.962689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574916+0000","last_change":"2024-03-30T00:44:16.307279+0000","last_active":"2024-03-30T00:44:17.574916+0000","last_peered":"2024-03-30T00:44:17.574916+0000","last_clean":"2024-03-30T00:44:17.574916+0000","last_became_active":"2024-03-30T00:44:16.306977+0000","last_became_peered":"2024-03-30T00:44:16.306977+0000","last_unstale":"2024-03-30T00:44:17.574916+0000","last_undegraded":"2024-03-30T00:44:17.574916+0000","last_fullsized":"2024-03-30T00:44:17.574916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:15:23.555013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573138+0000","last_change":"2024-03-30T00:44:16.298397+0000","last_active":"2024-03-30T00:44:17.573138+0000","last_peered":"2024-03-30T00:44:17.573138+0000","last_clean":"2024-03-30T00:44:17.573138+0000","last_became_active":"2024-03-30T00:44:16.298067+0000","last_became_peered":"2024-03-30T00:44:16.298067+0000","last_unstale":"2024-03-30T00:44:17.573138+0000","last_undegraded":"2024-03-30T00:44:17.573138+0000","last_fullsized":"2024-03-30T00:44:17.573138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:26:21.001769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574616+0000","last_change":"2024-03-30T00:44:16.307545+0000","last_active":"2024-03-30T00:44:17.574616+0000","last_peered":"2024-03-30T00:44:17.574616+0000","last_clean":"2024-03-30T00:44:17.574616+0000","last_became_active":"2024-03-30T00:44:16.307262+0000","last_became_peered":"2024-03-30T00:44:16.307262+0000","last_unstale":"2024-03-30T00:44:17.574616+0000","last_undegraded":"2024-03-30T00:44:17.574616+0000","last_fullsized":"2024-03-30T00:44:17.574616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:22.545875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288591+0000","last_change":"2024-03-30T00:44:16.310142+0000","last_active":"2024-03-30T00:44:17.288591+0000","last_peered":"2024-03-30T00:44:17.288591+0000","last_clean":"2024-03-30T00:44:17.288591+0000","last_became_active":"2024-03-30T00:44:16.309876+0000","last_became_peered":"2024-03-30T00:44:16.309876+0000","last_unstale":"2024-03-30T00:44:17.288591+0000","last_undegraded":"2024-03-30T00:44:17.288591+0000","last_fullsized":"2024-03-30T00:44:17.288591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:26.136845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572426+0000","last_change":"2024-03-30T00:44:16.297834+0000","last_active":"2024-03-30T00:44:17.572426+0000","last_peered":"2024-03-30T00:44:17.572426+0000","last_clean":"2024-03-30T00:44:17.572426+0000","last_became_active":"2024-03-30T00:44:16.296724+0000","last_became_peered":"2024-03-30T00:44:16.296724+0000","last_unstale":"2024-03-30T00:44:17.572426+0000","last_undegraded":"2024-03-30T00:44:17.572426+0000","last_fullsized":"2024-03-30T00:44:17.572426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:09.608809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575385+0000","last_change":"2024-03-30T00:44:16.312271+0000","last_active":"2024-03-30T00:44:17.575385+0000","last_peered":"2024-03-30T00:44:17.575385+0000","last_clean":"2024-03-30T00:44:17.575385+0000","last_became_active":"2024-03-30T00:44:16.312004+0000","last_became_peered":"2024-03-30T00:44:16.312004+0000","last_unstale":"2024-03-30T00:44:17.575385+0000","last_undegraded":"2024-03-30T00:44:17.575385+0000","last_fullsized":"2024-03-30T00:44:17.575385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:20.925778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572745+0000","last_change":"2024-03-30T00:44:16.299208+0000","last_active":"2024-03-30T00:44:17.572745+0000","last_peered":"2024-03-30T00:44:17.572745+0000","last_clean":"2024-03-30T00:44:17.572745+0000","last_became_active":"2024-03-30T00:44:16.298954+0000","last_became_peered":"2024-03-30T00:44:16.298954+0000","last_unstale":"2024-03-30T00:44:17.572745+0000","last_undegraded":"2024-03-30T00:44:17.572745+0000","last_fullsized":"2024-03-30T00:44:17.572745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:01:33.097514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574553+0000","last_change":"2024-03-30T00:44:16.296847+0000","last_active":"2024-03-30T00:44:17.574553+0000","last_peered":"2024-03-30T00:44:17.574553+0000","last_clean":"2024-03-30T00:44:17.574553+0000","last_became_active":"2024-03-30T00:44:16.296611+0000","last_became_peered":"2024-03-30T00:44:16.296611+0000","last_unstale":"2024-03-30T00:44:17.574553+0000","last_undegraded":"2024-03-30T00:44:17.574553+0000","last_fullsized":"2024-03-30T00:44:17.574553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:45:23.669464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574548+0000","last_change":"2024-03-30T00:44:16.300954+0000","last_active":"2024-03-30T00:44:17.574548+0000","last_peered":"2024-03-30T00:44:17.574548+0000","last_clean":"2024-03-30T00:44:17.574548+0000","last_became_active":"2024-03-30T00:44:16.300685+0000","last_became_peered":"2024-03-30T00:44:16.300685+0000","last_unstale":"2024-03-30T00:44:17.574548+0000","last_undegraded":"2024-03-30T00:44:17.574548+0000","last_fullsized":"2024-03-30T00:44:17.574548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:21.550411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574036+0000","last_change":"2024-03-30T00:44:16.298641+0000","last_active":"2024-03-30T00:44:17.574036+0000","last_peered":"2024-03-30T00:44:17.574036+0000","last_clean":"2024-03-30T00:44:17.574036+0000","last_became_active":"2024-03-30T00:44:16.295092+0000","last_became_peered":"2024-03-30T00:44:16.295092+0000","last_unstale":"2024-03-30T00:44:17.574036+0000","last_undegraded":"2024-03-30T00:44:17.574036+0000","last_fullsized":"2024-03-30T00:44:17.574036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:27.180701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288520+0000","last_change":"2024-03-30T00:44:16.301865+0000","last_active":"2024-03-30T00:44:17.288520+0000","last_peered":"2024-03-30T00:44:17.288520+0000","last_clean":"2024-03-30T00:44:17.288520+0000","last_became_active":"2024-03-30T00:44:16.301592+0000","last_became_peered":"2024-03-30T00:44:16.301592+0000","last_unstale":"2024-03-30T00:44:17.288520+0000","last_undegraded":"2024-03-30T00:44:17.288520+0000","last_fullsized":"2024-03-30T00:44:17.288520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:27:29.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574639+0000","last_change":"2024-03-30T00:44:16.299617+0000","last_active":"2024-03-30T00:44:17.574639+0000","last_peered":"2024-03-30T00:44:17.574639+0000","last_clean":"2024-03-30T00:44:17.574639+0000","last_became_active":"2024-03-30T00:44:16.299181+0000","last_became_peered":"2024-03-30T00:44:16.299181+0000","last_unstale":"2024-03-30T00:44:17.574639+0000","last_undegraded":"2024-03-30T00:44:17.574639+0000","last_fullsized":"2024-03-30T00:44:17.574639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:45.560695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287875+0000","last_change":"2024-03-30T00:44:16.303169+0000","last_active":"2024-03-30T00:44:17.287875+0000","last_peered":"2024-03-30T00:44:17.287875+0000","last_clean":"2024-03-30T00:44:17.287875+0000","last_became_active":"2024-03-30T00:44:16.302922+0000","last_became_peered":"2024-03-30T00:44:16.302922+0000","last_unstale":"2024-03-30T00:44:17.287875+0000","last_undegraded":"2024-03-30T00:44:17.287875+0000","last_fullsized":"2024-03-30T00:44:17.287875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:27:29.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574956+0000","last_change":"2024-03-30T00:44:16.297451+0000","last_active":"2024-03-30T00:44:17.574956+0000","last_peered":"2024-03-30T00:44:17.574956+0000","last_clean":"2024-03-30T00:44:17.574956+0000","last_became_active":"2024-03-30T00:44:16.297073+0000","last_became_peered":"2024-03-30T00:44:16.297073+0000","last_unstale":"2024-03-30T00:44:17.574956+0000","last_undegraded":"2024-03-30T00:44:17.574956+0000","last_fullsized":"2024-03-30T00:44:17.574956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:09:53.712612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574640+0000","last_change":"2024-03-30T00:44:16.291770+0000","last_active":"2024-03-30T00:44:17.574640+0000","last_peered":"2024-03-30T00:44:17.574640+0000","last_clean":"2024-03-30T00:44:17.574640+0000","last_became_active":"2024-03-30T00:44:16.291500+0000","last_became_peered":"2024-03-30T00:44:16.291500+0000","last_unstale":"2024-03-30T00:44:17.574640+0000","last_undegraded":"2024-03-30T00:44:17.574640+0000","last_fullsized":"2024-03-30T00:44:17.574640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:34.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288071+0000","last_change":"2024-03-30T00:44:16.306532+0000","last_active":"2024-03-30T00:44:17.288071+0000","last_peered":"2024-03-30T00:44:17.288071+0000","last_clean":"2024-03-30T00:44:17.288071+0000","last_became_active":"2024-03-30T00:44:16.306132+0000","last_became_peered":"2024-03-30T00:44:16.306132+0000","last_unstale":"2024-03-30T00:44:17.288071+0000","last_undegraded":"2024-03-30T00:44:17.288071+0000","last_fullsized":"2024-03-30T00:44:17.288071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:49.013320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574268+0000","last_change":"2024-03-30T00:44:16.295820+0000","last_active":"2024-03-30T00:44:17.574268+0000","last_peered":"2024-03-30T00:44:17.574268+0000","last_clean":"2024-03-30T00:44:17.574268+0000","last_became_active":"2024-03-30T00:44:16.295585+0000","last_became_peered":"2024-03-30T00:44:16.295585+0000","last_unstale":"2024-03-30T00:44:17.574268+0000","last_undegraded":"2024-03-30T00:44:17.574268+0000","last_fullsized":"2024-03-30T00:44:17.574268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:35:09.835107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287908+0000","last_change":"2024-03-30T00:44:16.303265+0000","last_active":"2024-03-30T00:44:17.287908+0000","last_peered":"2024-03-30T00:44:17.287908+0000","last_clean":"2024-03-30T00:44:17.287908+0000","last_became_active":"2024-03-30T00:44:16.302982+0000","last_became_peered":"2024-03-30T00:44:16.302982+0000","last_unstale":"2024-03-30T00:44:17.287908+0000","last_undegraded":"2024-03-30T00:44:17.287908+0000","last_fullsized":"2024-03-30T00:44:17.287908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:34:00.433221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574015+0000","last_change":"2024-03-30T00:44:16.301715+0000","last_active":"2024-03-30T00:44:17.574015+0000","last_peered":"2024-03-30T00:44:17.574015+0000","last_clean":"2024-03-30T00:44:17.574015+0000","last_became_active":"2024-03-30T00:44:16.301099+0000","last_became_peered":"2024-03-30T00:44:16.301099+0000","last_unstale":"2024-03-30T00:44:17.574015+0000","last_undegraded":"2024-03-30T00:44:17.574015+0000","last_fullsized":"2024-03-30T00:44:17.574015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:07.180780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575431+0000","last_change":"2024-03-30T00:44:16.300902+0000","last_active":"2024-03-30T00:44:17.575431+0000","last_peered":"2024-03-30T00:44:17.575431+0000","last_clean":"2024-03-30T00:44:17.575431+0000","last_became_active":"2024-03-30T00:44:16.300615+0000","last_became_peered":"2024-03-30T00:44:16.300615+0000","last_unstale":"2024-03-30T00:44:17.575431+0000","last_undegraded":"2024-03-30T00:44:17.575431+0000","last_fullsized":"2024-03-30T00:44:17.575431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:15:23.555013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572596+0000","last_change":"2024-03-30T00:44:16.301499+0000","last_active":"2024-03-30T00:44:17.572596+0000","last_peered":"2024-03-30T00:44:17.572596+0000","last_clean":"2024-03-30T00:44:17.572596+0000","last_became_active":"2024-03-30T00:44:16.301152+0000","last_became_peered":"2024-03-30T00:44:16.301152+0000","last_unstale":"2024-03-30T00:44:17.572596+0000","last_undegraded":"2024-03-30T00:44:17.572596+0000","last_fullsized":"2024-03-30T00:44:17.572596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:51.271671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288119+0000","last_change":"2024-03-30T00:44:16.304149+0000","last_active":"2024-03-30T00:44:17.288119+0000","last_peered":"2024-03-30T00:44:17.288119+0000","last_clean":"2024-03-30T00:44:17.288119+0000","last_became_active":"2024-03-30T00:44:16.303898+0000","last_became_peered":"2024-03-30T00:44:16.303898+0000","last_unstale":"2024-03-30T00:44:17.288119+0000","last_undegraded":"2024-03-30T00:44:17.288119+0000","last_fullsized":"2024-03-30T00:44:17.288119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:49.013320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574636+0000","last_change":"2024-03-30T00:44:16.297947+0000","last_active":"2024-03-30T00:44:17.574636+0000","last_peered":"2024-03-30T00:44:17.574636+0000","last_clean":"2024-03-30T00:44:17.574636+0000","last_became_active":"2024-03-30T00:44:16.295260+0000","last_became_peered":"2024-03-30T00:44:16.295260+0000","last_unstale":"2024-03-30T00:44:17.574636+0000","last_undegraded":"2024-03-30T00:44:17.574636+0000","last_fullsized":"2024-03-30T00:44:17.574636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:30:33.280971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575149+0000","last_change":"2024-03-30T00:44:16.300685+0000","last_active":"2024-03-30T00:44:17.575149+0000","last_peered":"2024-03-30T00:44:17.575149+0000","last_clean":"2024-03-30T00:44:17.575149+0000","last_became_active":"2024-03-30T00:44:16.300332+0000","last_became_peered":"2024-03-30T00:44:16.300332+0000","last_unstale":"2024-03-30T00:44:17.575149+0000","last_undegraded":"2024-03-30T00:44:17.575149+0000","last_fullsized":"2024-03-30T00:44:17.575149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:09:16.752883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287531+0000","last_change":"2024-03-30T00:44:16.303630+0000","last_active":"2024-03-30T00:44:17.287531+0000","last_peered":"2024-03-30T00:44:17.287531+0000","last_clean":"2024-03-30T00:44:17.287531+0000","last_became_active":"2024-03-30T00:44:16.303239+0000","last_became_peered":"2024-03-30T00:44:16.303239+0000","last_unstale":"2024-03-30T00:44:17.287531+0000","last_undegraded":"2024-03-30T00:44:17.287531+0000","last_fullsized":"2024-03-30T00:44:17.287531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:27.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287485+0000","last_change":"2024-03-30T00:44:16.299032+0000","last_active":"2024-03-30T00:44:17.287485+0000","last_peered":"2024-03-30T00:44:17.287485+0000","last_clean":"2024-03-30T00:44:17.287485+0000","last_became_active":"2024-03-30T00:44:16.298800+0000","last_became_peered":"2024-03-30T00:44:16.298800+0000","last_unstale":"2024-03-30T00:44:17.287485+0000","last_undegraded":"2024-03-30T00:44:17.287485+0000","last_fullsized":"2024-03-30T00:44:17.287485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:34:00.433221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288424+0000","last_change":"2024-03-30T00:44:16.301931+0000","last_active":"2024-03-30T00:44:17.288424+0000","last_peered":"2024-03-30T00:44:17.288424+0000","last_clean":"2024-03-30T00:44:17.288424+0000","last_became_active":"2024-03-30T00:44:16.301664+0000","last_became_peered":"2024-03-30T00:44:16.301664+0000","last_unstale":"2024-03-30T00:44:17.288424+0000","last_undegraded":"2024-03-30T00:44:17.288424+0000","last_fullsized":"2024-03-30T00:44:17.288424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:08:21.384792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287405+0000","last_change":"2024-03-30T00:44:16.303876+0000","last_active":"2024-03-30T00:44:17.287405+0000","last_peered":"2024-03-30T00:44:17.287405+0000","last_clean":"2024-03-30T00:44:17.287405+0000","last_became_active":"2024-03-30T00:44:16.303629+0000","last_became_peered":"2024-03-30T00:44:16.303629+0000","last_unstale":"2024-03-30T00:44:17.287405+0000","last_undegraded":"2024-03-30T00:44:17.287405+0000","last_fullsized":"2024-03-30T00:44:17.287405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:08:43.430473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574725+0000","last_change":"2024-03-30T00:44:15.894920+0000","last_active":"2024-03-30T00:44:17.574725+0000","last_peered":"2024-03-30T00:44:17.574725+0000","last_clean":"2024-03-30T00:44:17.574725+0000","last_became_active":"2024-03-30T00:44:15.893636+0000","last_became_peered":"2024-03-30T00:44:15.893636+0000","last_unstale":"2024-03-30T00:44:17.574725+0000","last_undegraded":"2024-03-30T00:44:17.574725+0000","last_fullsized":"2024-03-30T00:44:17.574725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:19:37.486774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574131+0000","last_change":"2024-03-30T00:44:16.296368+0000","last_active":"2024-03-30T00:44:17.574131+0000","last_peered":"2024-03-30T00:44:17.574131+0000","last_clean":"2024-03-30T00:44:17.574131+0000","last_became_active":"2024-03-30T00:44:16.295470+0000","last_became_peered":"2024-03-30T00:44:16.295470+0000","last_unstale":"2024-03-30T00:44:17.574131+0000","last_undegraded":"2024-03-30T00:44:17.574131+0000","last_fullsized":"2024-03-30T00:44:17.574131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:42:13.878266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287807+0000","last_change":"2024-03-30T00:44:15.889822+0000","last_active":"2024-03-30T00:44:17.287807+0000","last_peered":"2024-03-30T00:44:17.287807+0000","last_clean":"2024-03-30T00:44:17.287807+0000","last_became_active":"2024-03-30T00:44:15.889544+0000","last_became_peered":"2024-03-30T00:44:15.889544+0000","last_unstale":"2024-03-30T00:44:17.287807+0000","last_undegraded":"2024-03-30T00:44:17.287807+0000","last_fullsized":"2024-03-30T00:44:17.287807+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:09:08.918197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288256+0000","last_change":"2024-03-30T00:44:16.309609+0000","last_active":"2024-03-30T00:44:17.288256+0000","last_peered":"2024-03-30T00:44:17.288256+0000","last_clean":"2024-03-30T00:44:17.288256+0000","last_became_active":"2024-03-30T00:44:16.309122+0000","last_became_peered":"2024-03-30T00:44:16.309122+0000","last_unstale":"2024-03-30T00:44:17.288256+0000","last_undegraded":"2024-03-30T00:44:17.288256+0000","last_fullsized":"2024-03-30T00:44:17.288256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:46:29.847764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287729+0000","last_change":"2024-03-30T00:44:15.894077+0000","last_active":"2024-03-30T00:44:17.287729+0000","last_peered":"2024-03-30T00:44:17.287729+0000","last_clean":"2024-03-30T00:44:17.287729+0000","last_became_active":"2024-03-30T00:44:15.893798+0000","last_became_peered":"2024-03-30T00:44:15.893798+0000","last_unstale":"2024-03-30T00:44:17.287729+0000","last_undegraded":"2024-03-30T00:44:17.287729+0000","last_fullsized":"2024-03-30T00:44:17.287729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:28.329099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288139+0000","last_change":"2024-03-30T00:44:16.305280+0000","last_active":"2024-03-30T00:44:17.288139+0000","last_peered":"2024-03-30T00:44:17.288139+0000","last_clean":"2024-03-30T00:44:17.288139+0000","last_became_active":"2024-03-30T00:44:16.305037+0000","last_became_peered":"2024-03-30T00:44:16.305037+0000","last_unstale":"2024-03-30T00:44:17.288139+0000","last_undegraded":"2024-03-30T00:44:17.288139+0000","last_fullsized":"2024-03-30T00:44:17.288139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:27:29.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573387+0000","last_change":"2024-03-30T00:44:15.891108+0000","last_active":"2024-03-30T00:44:17.573387+0000","last_peered":"2024-03-30T00:44:17.573387+0000","last_clean":"2024-03-30T00:44:17.573387+0000","last_became_active":"2024-03-30T00:44:15.890534+0000","last_became_peered":"2024-03-30T00:44:15.890534+0000","last_unstale":"2024-03-30T00:44:17.573387+0000","last_undegraded":"2024-03-30T00:44:17.573387+0000","last_fullsized":"2024-03-30T00:44:17.573387+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:19.831013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575171+0000","last_change":"2024-03-30T00:44:16.296714+0000","last_active":"2024-03-30T00:44:17.575171+0000","last_peered":"2024-03-30T00:44:17.575171+0000","last_clean":"2024-03-30T00:44:17.575171+0000","last_became_active":"2024-03-30T00:44:16.296118+0000","last_became_peered":"2024-03-30T00:44:16.296118+0000","last_unstale":"2024-03-30T00:44:17.575171+0000","last_undegraded":"2024-03-30T00:44:17.575171+0000","last_fullsized":"2024-03-30T00:44:17.575171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:04.876527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288250+0000","last_change":"2024-03-30T00:44:15.893199+0000","last_active":"2024-03-30T00:44:17.288250+0000","last_peered":"2024-03-30T00:44:17.288250+0000","last_clean":"2024-03-30T00:44:17.288250+0000","last_became_active":"2024-03-30T00:44:15.892791+0000","last_became_peered":"2024-03-30T00:44:15.892791+0000","last_unstale":"2024-03-30T00:44:17.288250+0000","last_undegraded":"2024-03-30T00:44:17.288250+0000","last_fullsized":"2024-03-30T00:44:17.288250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:44.963645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575406+0000","last_change":"2024-03-30T00:44:16.297214+0000","last_active":"2024-03-30T00:44:17.575406+0000","last_peered":"2024-03-30T00:44:17.575406+0000","last_clean":"2024-03-30T00:44:17.575406+0000","last_became_active":"2024-03-30T00:44:16.296958+0000","last_became_peered":"2024-03-30T00:44:16.296958+0000","last_unstale":"2024-03-30T00:44:17.575406+0000","last_undegraded":"2024-03-30T00:44:17.575406+0000","last_fullsized":"2024-03-30T00:44:17.575406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:40:45.735026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572309+0000","last_change":"2024-03-30T00:44:15.895373+0000","last_active":"2024-03-30T00:44:17.572309+0000","last_peered":"2024-03-30T00:44:17.572309+0000","last_clean":"2024-03-30T00:44:17.572309+0000","last_became_active":"2024-03-30T00:44:15.895102+0000","last_became_peered":"2024-03-30T00:44:15.895102+0000","last_unstale":"2024-03-30T00:44:17.572309+0000","last_undegraded":"2024-03-30T00:44:17.572309+0000","last_fullsized":"2024-03-30T00:44:17.572309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:33:54.146938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288649+0000","last_change":"2024-03-30T00:44:16.308709+0000","last_active":"2024-03-30T00:44:17.288649+0000","last_peered":"2024-03-30T00:44:17.288649+0000","last_clean":"2024-03-30T00:44:17.288649+0000","last_became_active":"2024-03-30T00:44:16.308398+0000","last_became_peered":"2024-03-30T00:44:16.308398+0000","last_unstale":"2024-03-30T00:44:17.288649+0000","last_undegraded":"2024-03-30T00:44:17.288649+0000","last_fullsized":"2024-03-30T00:44:17.288649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:31.152253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574631+0000","last_change":"2024-03-30T00:44:15.889303+0000","last_active":"2024-03-30T00:44:17.574631+0000","last_peered":"2024-03-30T00:44:17.574631+0000","last_clean":"2024-03-30T00:44:17.574631+0000","last_became_active":"2024-03-30T00:44:15.889035+0000","last_became_peered":"2024-03-30T00:44:15.889035+0000","last_unstale":"2024-03-30T00:44:17.574631+0000","last_undegraded":"2024-03-30T00:44:17.574631+0000","last_fullsized":"2024-03-30T00:44:17.574631+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:26:27.776857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573604+0000","last_change":"2024-03-30T00:44:16.298589+0000","last_active":"2024-03-30T00:44:17.573604+0000","last_peered":"2024-03-30T00:44:17.573604+0000","last_clean":"2024-03-30T00:44:17.573604+0000","last_became_active":"2024-03-30T00:44:16.297920+0000","last_became_peered":"2024-03-30T00:44:16.297920+0000","last_unstale":"2024-03-30T00:44:17.573604+0000","last_undegraded":"2024-03-30T00:44:17.573604+0000","last_fullsized":"2024-03-30T00:44:17.573604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:51.509235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574118+0000","last_change":"2024-03-30T00:44:15.895431+0000","last_active":"2024-03-30T00:44:17.574118+0000","last_peered":"2024-03-30T00:44:17.574118+0000","last_clean":"2024-03-30T00:44:17.574118+0000","last_became_active":"2024-03-30T00:44:15.894494+0000","last_became_peered":"2024-03-30T00:44:15.894494+0000","last_unstale":"2024-03-30T00:44:17.574118+0000","last_undegraded":"2024-03-30T00:44:17.574118+0000","last_fullsized":"2024-03-30T00:44:17.574118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:34:02.924271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574105+0000","last_change":"2024-03-30T00:44:16.301930+0000","last_active":"2024-03-30T00:44:17.574105+0000","last_peered":"2024-03-30T00:44:17.574105+0000","last_clean":"2024-03-30T00:44:17.574105+0000","last_became_active":"2024-03-30T00:44:16.301424+0000","last_became_peered":"2024-03-30T00:44:16.301424+0000","last_unstale":"2024-03-30T00:44:17.574105+0000","last_undegraded":"2024-03-30T00:44:17.574105+0000","last_fullsized":"2024-03-30T00:44:17.574105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:49.433494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287984+0000","last_change":"2024-03-30T00:44:15.895696+0000","last_active":"2024-03-30T00:44:17.287984+0000","last_peered":"2024-03-30T00:44:17.287984+0000","last_clean":"2024-03-30T00:44:17.287984+0000","last_became_active":"2024-03-30T00:44:15.894153+0000","last_became_peered":"2024-03-30T00:44:15.894153+0000","last_unstale":"2024-03-30T00:44:17.287984+0000","last_undegraded":"2024-03-30T00:44:17.287984+0000","last_fullsized":"2024-03-30T00:44:17.287984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:47:07.113229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572419+0000","last_change":"2024-03-30T00:44:16.295860+0000","last_active":"2024-03-30T00:44:17.572419+0000","last_peered":"2024-03-30T00:44:17.572419+0000","last_clean":"2024-03-30T00:44:17.572419+0000","last_became_active":"2024-03-30T00:44:16.295577+0000","last_became_peered":"2024-03-30T00:44:16.295577+0000","last_unstale":"2024-03-30T00:44:17.572419+0000","last_undegraded":"2024-03-30T00:44:17.572419+0000","last_fullsized":"2024-03-30T00:44:17.572419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:34:43.997849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572640+0000","last_change":"2024-03-30T00:44:15.893771+0000","last_active":"2024-03-30T00:44:17.572640+0000","last_peered":"2024-03-30T00:44:17.572640+0000","last_clean":"2024-03-30T00:44:17.572640+0000","last_became_active":"2024-03-30T00:44:15.893503+0000","last_became_peered":"2024-03-30T00:44:15.893503+0000","last_unstale":"2024-03-30T00:44:17.572640+0000","last_undegraded":"2024-03-30T00:44:17.572640+0000","last_fullsized":"2024-03-30T00:44:17.572640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:28.329099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574701+0000","last_change":"2024-03-30T00:44:16.294343+0000","last_active":"2024-03-30T00:44:17.574701+0000","last_peered":"2024-03-30T00:44:17.574701+0000","last_clean":"2024-03-30T00:44:17.574701+0000","last_became_active":"2024-03-30T00:44:16.293730+0000","last_became_peered":"2024-03-30T00:44:16.293730+0000","last_unstale":"2024-03-30T00:44:17.574701+0000","last_undegraded":"2024-03-30T00:44:17.574701+0000","last_fullsized":"2024-03-30T00:44:17.574701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:11:23.497004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574691+0000","last_change":"2024-03-30T00:44:15.894288+0000","last_active":"2024-03-30T00:44:17.574691+0000","last_peered":"2024-03-30T00:44:17.574691+0000","last_clean":"2024-03-30T00:44:17.574691+0000","last_became_active":"2024-03-30T00:44:15.894035+0000","last_became_peered":"2024-03-30T00:44:15.894035+0000","last_unstale":"2024-03-30T00:44:17.574691+0000","last_undegraded":"2024-03-30T00:44:17.574691+0000","last_fullsized":"2024-03-30T00:44:17.574691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:23.741176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287691+0000","last_change":"2024-03-30T00:44:16.297732+0000","last_active":"2024-03-30T00:44:17.287691+0000","last_peered":"2024-03-30T00:44:17.287691+0000","last_clean":"2024-03-30T00:44:17.287691+0000","last_became_active":"2024-03-30T00:44:16.297478+0000","last_became_peered":"2024-03-30T00:44:16.297478+0000","last_unstale":"2024-03-30T00:44:17.287691+0000","last_undegraded":"2024-03-30T00:44:17.287691+0000","last_fullsized":"2024-03-30T00:44:17.287691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:49.013320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573552+0000","last_change":"2024-03-30T00:44:15.899251+0000","last_active":"2024-03-30T00:44:17.573552+0000","last_peered":"2024-03-30T00:44:17.573552+0000","last_clean":"2024-03-30T00:44:17.573552+0000","last_became_active":"2024-03-30T00:44:15.898675+0000","last_became_peered":"2024-03-30T00:44:15.898675+0000","last_unstale":"2024-03-30T00:44:17.573552+0000","last_undegraded":"2024-03-30T00:44:17.573552+0000","last_fullsized":"2024-03-30T00:44:17.573552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:34:17.023248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572745+0000","last_change":"2024-03-30T00:44:16.295026+0000","last_active":"2024-03-30T00:44:17.572745+0000","last_peered":"2024-03-30T00:44:17.572745+0000","last_clean":"2024-03-30T00:44:17.572745+0000","last_became_active":"2024-03-30T00:44:16.292104+0000","last_became_peered":"2024-03-30T00:44:16.292104+0000","last_unstale":"2024-03-30T00:44:17.572745+0000","last_undegraded":"2024-03-30T00:44:17.572745+0000","last_fullsized":"2024-03-30T00:44:17.572745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:27.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572303+0000","last_change":"2024-03-30T00:44:15.897855+0000","last_active":"2024-03-30T00:44:17.572303+0000","last_peered":"2024-03-30T00:44:17.572303+0000","last_clean":"2024-03-30T00:44:17.572303+0000","last_became_active":"2024-03-30T00:44:15.897514+0000","last_became_peered":"2024-03-30T00:44:15.897514+0000","last_unstale":"2024-03-30T00:44:17.572303+0000","last_undegraded":"2024-03-30T00:44:17.572303+0000","last_fullsized":"2024-03-30T00:44:17.572303+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:30.149492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573534+0000","last_change":"2024-03-30T00:44:16.298473+0000","last_active":"2024-03-30T00:44:17.573534+0000","last_peered":"2024-03-30T00:44:17.573534+0000","last_clean":"2024-03-30T00:44:17.573534+0000","last_became_active":"2024-03-30T00:44:16.298155+0000","last_became_peered":"2024-03-30T00:44:16.298155+0000","last_unstale":"2024-03-30T00:44:17.573534+0000","last_undegraded":"2024-03-30T00:44:17.573534+0000","last_fullsized":"2024-03-30T00:44:17.573534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:31:21.221158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287534+0000","last_change":"2024-03-30T00:44:15.896610+0000","last_active":"2024-03-30T00:44:17.287534+0000","last_peered":"2024-03-30T00:44:17.287534+0000","last_clean":"2024-03-30T00:44:17.287534+0000","last_became_active":"2024-03-30T00:44:15.896357+0000","last_became_peered":"2024-03-30T00:44:15.896357+0000","last_unstale":"2024-03-30T00:44:17.287534+0000","last_undegraded":"2024-03-30T00:44:17.287534+0000","last_fullsized":"2024-03-30T00:44:17.287534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:45.617202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573548+0000","last_change":"2024-03-30T00:44:16.305318+0000","last_active":"2024-03-30T00:44:17.573548+0000","last_peered":"2024-03-30T00:44:17.573548+0000","last_clean":"2024-03-30T00:44:17.573548+0000","last_became_active":"2024-03-30T00:44:16.304906+0000","last_became_peered":"2024-03-30T00:44:16.304906+0000","last_unstale":"2024-03-30T00:44:17.573548+0000","last_undegraded":"2024-03-30T00:44:17.573548+0000","last_fullsized":"2024-03-30T00:44:17.573548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:50.698731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574324+0000","last_change":"2024-03-30T00:44:15.890743+0000","last_active":"2024-03-30T00:44:17.574324+0000","last_peered":"2024-03-30T00:44:17.574324+0000","last_clean":"2024-03-30T00:44:17.574324+0000","last_became_active":"2024-03-30T00:44:15.890487+0000","last_became_peered":"2024-03-30T00:44:15.890487+0000","last_unstale":"2024-03-30T00:44:17.574324+0000","last_undegraded":"2024-03-30T00:44:17.574324+0000","last_fullsized":"2024-03-30T00:44:17.574324+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:39.171876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573710+0000","last_change":"2024-03-30T00:44:16.302640+0000","last_active":"2024-03-30T00:44:17.573710+0000","last_peered":"2024-03-30T00:44:17.573710+0000","last_clean":"2024-03-30T00:44:17.573710+0000","last_became_active":"2024-03-30T00:44:16.302357+0000","last_became_peered":"2024-03-30T00:44:16.302357+0000","last_unstale":"2024-03-30T00:44:17.573710+0000","last_undegraded":"2024-03-30T00:44:17.573710+0000","last_fullsized":"2024-03-30T00:44:17.573710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:35:57.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574192+0000","last_change":"2024-03-30T00:44:15.890728+0000","last_active":"2024-03-30T00:44:17.574192+0000","last_peered":"2024-03-30T00:44:17.574192+0000","last_clean":"2024-03-30T00:44:17.574192+0000","last_became_active":"2024-03-30T00:44:15.890428+0000","last_became_peered":"2024-03-30T00:44:15.890428+0000","last_unstale":"2024-03-30T00:44:17.574192+0000","last_undegraded":"2024-03-30T00:44:17.574192+0000","last_fullsized":"2024-03-30T00:44:17.574192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:39:42.688699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574094+0000","last_change":"2024-03-30T00:44:16.289471+0000","last_active":"2024-03-30T00:44:17.574094+0000","last_peered":"2024-03-30T00:44:17.574094+0000","last_clean":"2024-03-30T00:44:17.574094+0000","last_became_active":"2024-03-30T00:44:16.289235+0000","last_became_peered":"2024-03-30T00:44:16.289235+0000","last_unstale":"2024-03-30T00:44:17.574094+0000","last_undegraded":"2024-03-30T00:44:17.574094+0000","last_fullsized":"2024-03-30T00:44:17.574094+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:58:31.868890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288086+0000","last_change":"2024-03-30T00:44:15.893511+0000","last_active":"2024-03-30T00:44:17.288086+0000","last_peered":"2024-03-30T00:44:17.288086+0000","last_clean":"2024-03-30T00:44:17.288086+0000","last_became_active":"2024-03-30T00:44:15.893221+0000","last_became_peered":"2024-03-30T00:44:15.893221+0000","last_unstale":"2024-03-30T00:44:17.288086+0000","last_undegraded":"2024-03-30T00:44:17.288086+0000","last_fullsized":"2024-03-30T00:44:17.288086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:38.935040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574277+0000","last_change":"2024-03-30T00:44:16.297406+0000","last_active":"2024-03-30T00:44:17.574277+0000","last_peered":"2024-03-30T00:44:17.574277+0000","last_clean":"2024-03-30T00:44:17.574277+0000","last_became_active":"2024-03-30T00:44:16.297157+0000","last_became_peered":"2024-03-30T00:44:16.297157+0000","last_unstale":"2024-03-30T00:44:17.574277+0000","last_undegraded":"2024-03-30T00:44:17.574277+0000","last_fullsized":"2024-03-30T00:44:17.574277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:52:51.815643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288731+0000","last_change":"2024-03-30T00:44:15.895444+0000","last_active":"2024-03-30T00:44:17.288731+0000","last_peered":"2024-03-30T00:44:17.288731+0000","last_clean":"2024-03-30T00:44:17.288731+0000","last_became_active":"2024-03-30T00:44:15.895116+0000","last_became_peered":"2024-03-30T00:44:15.895116+0000","last_unstale":"2024-03-30T00:44:17.288731+0000","last_undegraded":"2024-03-30T00:44:17.288731+0000","last_fullsized":"2024-03-30T00:44:17.288731+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:50.506474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574369+0000","last_change":"2024-03-30T00:44:16.302258+0000","last_active":"2024-03-30T00:44:17.574369+0000","last_peered":"2024-03-30T00:44:17.574369+0000","last_clean":"2024-03-30T00:44:17.574369+0000","last_became_active":"2024-03-30T00:44:16.302019+0000","last_became_peered":"2024-03-30T00:44:16.302019+0000","last_unstale":"2024-03-30T00:44:17.574369+0000","last_undegraded":"2024-03-30T00:44:17.574369+0000","last_fullsized":"2024-03-30T00:44:17.574369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:34:39.338907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574782+0000","last_change":"2024-03-30T00:44:15.898203+0000","last_active":"2024-03-30T00:44:17.574782+0000","last_peered":"2024-03-30T00:44:17.574782+0000","last_clean":"2024-03-30T00:44:17.574782+0000","last_became_active":"2024-03-30T00:44:15.897951+0000","last_became_peered":"2024-03-30T00:44:15.897951+0000","last_unstale":"2024-03-30T00:44:17.574782+0000","last_undegraded":"2024-03-30T00:44:17.574782+0000","last_fullsized":"2024-03-30T00:44:17.574782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:12:39.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574320+0000","last_change":"2024-03-30T00:44:16.290687+0000","last_active":"2024-03-30T00:44:17.574320+0000","last_peered":"2024-03-30T00:44:17.574320+0000","last_clean":"2024-03-30T00:44:17.574320+0000","last_became_active":"2024-03-30T00:44:16.289250+0000","last_became_peered":"2024-03-30T00:44:16.289250+0000","last_unstale":"2024-03-30T00:44:17.574320+0000","last_undegraded":"2024-03-30T00:44:17.574320+0000","last_fullsized":"2024-03-30T00:44:17.574320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:07.180780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287624+0000","last_change":"2024-03-30T00:44:15.889708+0000","last_active":"2024-03-30T00:44:17.287624+0000","last_peered":"2024-03-30T00:44:17.287624+0000","last_clean":"2024-03-30T00:44:17.287624+0000","last_became_active":"2024-03-30T00:44:15.889450+0000","last_became_peered":"2024-03-30T00:44:15.889450+0000","last_unstale":"2024-03-30T00:44:17.287624+0000","last_undegraded":"2024-03-30T00:44:17.287624+0000","last_fullsized":"2024-03-30T00:44:17.287624+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:47:07.113229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575289+0000","last_change":"2024-03-30T00:44:16.305471+0000","last_active":"2024-03-30T00:44:17.575289+0000","last_peered":"2024-03-30T00:44:17.575289+0000","last_clean":"2024-03-30T00:44:17.575289+0000","last_became_active":"2024-03-30T00:44:16.305163+0000","last_became_peered":"2024-03-30T00:44:16.305163+0000","last_unstale":"2024-03-30T00:44:17.575289+0000","last_undegraded":"2024-03-30T00:44:17.575289+0000","last_fullsized":"2024-03-30T00:44:17.575289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:33.986685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574414+0000","last_change":"2024-03-30T00:44:15.888585+0000","last_active":"2024-03-30T00:44:17.574414+0000","last_peered":"2024-03-30T00:44:17.574414+0000","last_clean":"2024-03-30T00:44:17.574414+0000","last_became_active":"2024-03-30T00:44:15.888266+0000","last_became_peered":"2024-03-30T00:44:15.888266+0000","last_unstale":"2024-03-30T00:44:17.574414+0000","last_undegraded":"2024-03-30T00:44:17.574414+0000","last_fullsized":"2024-03-30T00:44:17.574414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:08.606048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575616+0000","last_change":"2024-03-30T00:44:16.307577+0000","last_active":"2024-03-30T00:44:17.575616+0000","last_peered":"2024-03-30T00:44:17.575616+0000","last_clean":"2024-03-30T00:44:17.575616+0000","last_became_active":"2024-03-30T00:44:16.307344+0000","last_became_peered":"2024-03-30T00:44:16.307344+0000","last_unstale":"2024-03-30T00:44:17.575616+0000","last_undegraded":"2024-03-30T00:44:17.575616+0000","last_fullsized":"2024-03-30T00:44:17.575616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:06:31.726939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574334+0000","last_change":"2024-03-30T00:44:15.890129+0000","last_active":"2024-03-30T00:44:17.574334+0000","last_peered":"2024-03-30T00:44:17.574334+0000","last_clean":"2024-03-30T00:44:17.574334+0000","last_became_active":"2024-03-30T00:44:15.889889+0000","last_became_peered":"2024-03-30T00:44:15.889889+0000","last_unstale":"2024-03-30T00:44:17.574334+0000","last_undegraded":"2024-03-30T00:44:17.574334+0000","last_fullsized":"2024-03-30T00:44:17.574334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:34:42.995087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573687+0000","last_change":"2024-03-30T00:44:16.298769+0000","last_active":"2024-03-30T00:44:17.573687+0000","last_peered":"2024-03-30T00:44:17.573687+0000","last_clean":"2024-03-30T00:44:17.573687+0000","last_became_active":"2024-03-30T00:44:16.298288+0000","last_became_peered":"2024-03-30T00:44:16.298288+0000","last_unstale":"2024-03-30T00:44:17.573687+0000","last_undegraded":"2024-03-30T00:44:17.573687+0000","last_fullsized":"2024-03-30T00:44:17.573687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:45.966406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574178+0000","last_change":"2024-03-30T00:44:15.891005+0000","last_active":"2024-03-30T00:44:17.574178+0000","last_peered":"2024-03-30T00:44:17.574178+0000","last_clean":"2024-03-30T00:44:17.574178+0000","last_became_active":"2024-03-30T00:44:15.890769+0000","last_became_peered":"2024-03-30T00:44:15.890769+0000","last_unstale":"2024-03-30T00:44:17.574178+0000","last_undegraded":"2024-03-30T00:44:17.574178+0000","last_fullsized":"2024-03-30T00:44:17.574178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:50.268910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287679+0000","last_change":"2024-03-30T00:44:16.300595+0000","last_active":"2024-03-30T00:44:17.287679+0000","last_peered":"2024-03-30T00:44:17.287679+0000","last_clean":"2024-03-30T00:44:17.287679+0000","last_became_active":"2024-03-30T00:44:16.300218+0000","last_became_peered":"2024-03-30T00:44:16.300218+0000","last_unstale":"2024-03-30T00:44:17.287679+0000","last_undegraded":"2024-03-30T00:44:17.287679+0000","last_fullsized":"2024-03-30T00:44:17.287679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:08:43.430473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572646+0000","last_change":"2024-03-30T00:44:15.887464+0000","last_active":"2024-03-30T00:44:17.572646+0000","last_peered":"2024-03-30T00:44:17.572646+0000","last_clean":"2024-03-30T00:44:17.572646+0000","last_became_active":"2024-03-30T00:44:15.886447+0000","last_became_peered":"2024-03-30T00:44:15.886447+0000","last_unstale":"2024-03-30T00:44:17.572646+0000","last_undegraded":"2024-03-30T00:44:17.572646+0000","last_fullsized":"2024-03-30T00:44:17.572646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:25.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573225+0000","last_change":"2024-03-30T00:44:16.303260+0000","last_active":"2024-03-30T00:44:17.573225+0000","last_peered":"2024-03-30T00:44:17.573225+0000","last_clean":"2024-03-30T00:44:17.573225+0000","last_became_active":"2024-03-30T00:44:16.302811+0000","last_became_peered":"2024-03-30T00:44:16.302811+0000","last_unstale":"2024-03-30T00:44:17.573225+0000","last_undegraded":"2024-03-30T00:44:17.573225+0000","last_fullsized":"2024-03-30T00:44:17.573225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:49:15.557324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575606+0000","last_change":"2024-03-30T00:44:15.891396+0000","last_active":"2024-03-30T00:44:17.575606+0000","last_peered":"2024-03-30T00:44:17.575606+0000","last_clean":"2024-03-30T00:44:17.575606+0000","last_became_active":"2024-03-30T00:44:15.891155+0000","last_became_peered":"2024-03-30T00:44:15.891155+0000","last_unstale":"2024-03-30T00:44:17.575606+0000","last_undegraded":"2024-03-30T00:44:17.575606+0000","last_fullsized":"2024-03-30T00:44:17.575606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:50.268910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288257+0000","last_change":"2024-03-30T00:44:16.309424+0000","last_active":"2024-03-30T00:44:17.288257+0000","last_peered":"2024-03-30T00:44:17.288257+0000","last_clean":"2024-03-30T00:44:17.288257+0000","last_became_active":"2024-03-30T00:44:16.308766+0000","last_became_peered":"2024-03-30T00:44:16.308766+0000","last_unstale":"2024-03-30T00:44:17.288257+0000","last_undegraded":"2024-03-30T00:44:17.288257+0000","last_fullsized":"2024-03-30T00:44:17.288257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:33:55.149700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287964+0000","last_change":"2024-03-30T00:44:15.893935+0000","last_active":"2024-03-30T00:44:17.287964+0000","last_peered":"2024-03-30T00:44:17.287964+0000","last_clean":"2024-03-30T00:44:17.287964+0000","last_became_active":"2024-03-30T00:44:15.893673+0000","last_became_peered":"2024-03-30T00:44:15.893673+0000","last_unstale":"2024-03-30T00:44:17.287964+0000","last_undegraded":"2024-03-30T00:44:17.287964+0000","last_fullsized":"2024-03-30T00:44:17.287964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:14.918980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575185+0000","last_change":"2024-03-30T00:44:16.302037+0000","last_active":"2024-03-30T00:44:17.575185+0000","last_peered":"2024-03-30T00:44:17.575185+0000","last_clean":"2024-03-30T00:44:17.575185+0000","last_became_active":"2024-03-30T00:44:16.301516+0000","last_became_peered":"2024-03-30T00:44:16.301516+0000","last_unstale":"2024-03-30T00:44:17.575185+0000","last_undegraded":"2024-03-30T00:44:17.575185+0000","last_fullsized":"2024-03-30T00:44:17.575185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:39:17.000234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287693+0000","last_change":"2024-03-30T00:44:15.888482+0000","last_active":"2024-03-30T00:44:17.287693+0000","last_peered":"2024-03-30T00:44:17.287693+0000","last_clean":"2024-03-30T00:44:17.287693+0000","last_became_active":"2024-03-30T00:44:15.888165+0000","last_became_peered":"2024-03-30T00:44:15.888165+0000","last_unstale":"2024-03-30T00:44:17.287693+0000","last_undegraded":"2024-03-30T00:44:17.287693+0000","last_fullsized":"2024-03-30T00:44:17.287693+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:09:23.017153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573340+0000","last_change":"2024-03-30T00:44:16.303356+0000","last_active":"2024-03-30T00:44:17.573340+0000","last_peered":"2024-03-30T00:44:17.573340+0000","last_clean":"2024-03-30T00:44:17.573340+0000","last_became_active":"2024-03-30T00:44:16.303052+0000","last_became_peered":"2024-03-30T00:44:16.303052+0000","last_unstale":"2024-03-30T00:44:17.573340+0000","last_undegraded":"2024-03-30T00:44:17.573340+0000","last_fullsized":"2024-03-30T00:44:17.573340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:57:20.784137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573301+0000","last_change":"2024-03-30T00:44:15.891617+0000","last_active":"2024-03-30T00:44:17.573301+0000","last_peered":"2024-03-30T00:44:17.573301+0000","last_clean":"2024-03-30T00:44:17.573301+0000","last_became_active":"2024-03-30T00:44:15.891335+0000","last_became_peered":"2024-03-30T00:44:15.891335+0000","last_unstale":"2024-03-30T00:44:17.573301+0000","last_undegraded":"2024-03-30T00:44:17.573301+0000","last_fullsized":"2024-03-30T00:44:17.573301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:43:30.408270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573478+0000","last_change":"2024-03-30T00:44:16.299191+0000","last_active":"2024-03-30T00:44:17.573478+0000","last_peered":"2024-03-30T00:44:17.573478+0000","last_clean":"2024-03-30T00:44:17.573478+0000","last_became_active":"2024-03-30T00:44:16.297785+0000","last_became_peered":"2024-03-30T00:44:16.297785+0000","last_unstale":"2024-03-30T00:44:17.573478+0000","last_undegraded":"2024-03-30T00:44:17.573478+0000","last_fullsized":"2024-03-30T00:44:17.573478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:09:11.545797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574469+0000","last_change":"2024-03-30T00:44:15.895307+0000","last_active":"2024-03-30T00:44:17.574469+0000","last_peered":"2024-03-30T00:44:17.574469+0000","last_clean":"2024-03-30T00:44:17.574469+0000","last_became_active":"2024-03-30T00:44:15.894810+0000","last_became_peered":"2024-03-30T00:44:15.894810+0000","last_unstale":"2024-03-30T00:44:17.574469+0000","last_undegraded":"2024-03-30T00:44:17.574469+0000","last_fullsized":"2024-03-30T00:44:17.574469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:55:34.160063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287965+0000","last_change":"2024-03-30T00:44:16.298000+0000","last_active":"2024-03-30T00:44:17.287965+0000","last_peered":"2024-03-30T00:44:17.287965+0000","last_clean":"2024-03-30T00:44:17.287965+0000","last_became_active":"2024-03-30T00:44:16.297792+0000","last_became_peered":"2024-03-30T00:44:16.297792+0000","last_unstale":"2024-03-30T00:44:17.287965+0000","last_undegraded":"2024-03-30T00:44:17.287965+0000","last_fullsized":"2024-03-30T00:44:17.287965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:08:21.384792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573250+0000","last_change":"2024-03-30T00:44:15.894525+0000","last_active":"2024-03-30T00:44:17.573250+0000","last_peered":"2024-03-30T00:44:17.573250+0000","last_clean":"2024-03-30T00:44:17.573250+0000","last_became_active":"2024-03-30T00:44:15.894273+0000","last_became_peered":"2024-03-30T00:44:15.894273+0000","last_unstale":"2024-03-30T00:44:17.573250+0000","last_undegraded":"2024-03-30T00:44:17.573250+0000","last_fullsized":"2024-03-30T00:44:17.573250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:36:40.678790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288032+0000","last_change":"2024-03-30T00:44:16.292522+0000","last_active":"2024-03-30T00:44:17.288032+0000","last_peered":"2024-03-30T00:44:17.288032+0000","last_clean":"2024-03-30T00:44:17.288032+0000","last_became_active":"2024-03-30T00:44:16.292255+0000","last_became_peered":"2024-03-30T00:44:16.292255+0000","last_unstale":"2024-03-30T00:44:17.288032+0000","last_undegraded":"2024-03-30T00:44:17.288032+0000","last_fullsized":"2024-03-30T00:44:17.288032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:46:29.847764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287391+0000","last_change":"2024-03-30T00:44:15.882801+0000","last_active":"2024-03-30T00:44:17.287391+0000","last_peered":"2024-03-30T00:44:17.287391+0000","last_clean":"2024-03-30T00:44:17.287391+0000","last_became_active":"2024-03-30T00:44:15.882526+0000","last_became_peered":"2024-03-30T00:44:15.882526+0000","last_unstale":"2024-03-30T00:44:17.287391+0000","last_undegraded":"2024-03-30T00:44:17.287391+0000","last_fullsized":"2024-03-30T00:44:17.287391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:45.617202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287865+0000","last_change":"2024-03-30T00:44:16.308219+0000","last_active":"2024-03-30T00:44:17.287865+0000","last_peered":"2024-03-30T00:44:17.287865+0000","last_clean":"2024-03-30T00:44:17.287865+0000","last_became_active":"2024-03-30T00:44:16.307891+0000","last_became_peered":"2024-03-30T00:44:16.307891+0000","last_unstale":"2024-03-30T00:44:17.287865+0000","last_undegraded":"2024-03-30T00:44:17.287865+0000","last_fullsized":"2024-03-30T00:44:17.287865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:08:43.430473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574975+0000","last_change":"2024-03-30T00:44:15.894627+0000","last_active":"2024-03-30T00:44:17.574975+0000","last_peered":"2024-03-30T00:44:17.574975+0000","last_clean":"2024-03-30T00:44:17.574975+0000","last_became_active":"2024-03-30T00:44:15.894365+0000","last_became_peered":"2024-03-30T00:44:15.894365+0000","last_unstale":"2024-03-30T00:44:17.574975+0000","last_undegraded":"2024-03-30T00:44:17.574975+0000","last_fullsized":"2024-03-30T00:44:17.574975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:01:32.094752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":227,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-30T00:44:16.285847+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:44:16.285847+0000","last_peered":"2024-03-30T00:44:16.285847+0000","last_clean":"2024-03-30T00:44:16.285847+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:44:16.285847+0000","last_undegraded":"2024-03-30T00:44:16.285847+0000","last_fullsized":"2024-03-30T00:44:16.285847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1601536,"data_stored":1045262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.501,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88800000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95599999999999996,"5min":1.149,"15min":1.149},"last":0.72199999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65600000000000003,"15min":0.63100000000000001},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.92200000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91500000000000004,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.81100000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77000000000000002,"15min":0.78500000000000003},"min":{"1min":0.46200000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96199999999999997,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.873},{"interface":"front","average":{"1min":0.753,"5min":0.75600000000000001,"15min":0.78400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.84199999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76900000000000002,"15min":0.79300000000000004},"min":{"1min":0.502,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80600000000000005,"15min":0.82299999999999995},"min":{"1min":0.60299999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.95399999999999996,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.93999999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60999999999999999,"15min":0.63},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.77100000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.48699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.89000000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.89900000000000002,"5min":1.121,"15min":1.121},"last":1.032},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.745,"15min":0.76500000000000001},"min":{"1min":0.52300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83799999999999997,"5min":1.026,"15min":1.026},"last":0.95799999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.49299999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.86599999999999999,"5min":1.214,"15min":1.214},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.50800000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.89800000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.98399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":449816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.78500000000000003,"15min":0.90200000000000002},"min":{"1min":0.443,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.2450000000000001,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.751},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.94599999999999995,"5min":1.47,"15min":1.47},"last":0.63900000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.78700000000000003,"15min":0.91500000000000004},"min":{"1min":0.51500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0249999999999999,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.83999999999999997,"15min":0.98799999999999999},"min":{"1min":0.55600000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.018,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.72299999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.83299999999999996,"15min":0.97299999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1499999999999999,"5min":4.375,"15min":4.375},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.83299999999999996,"15min":0.97699999999999998},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93500000000000005,"5min":4.2960000000000003,"15min":4.2960000000000003},"last":0.73199999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.85499999999999998,"15min":0.99399999999999999},"min":{"1min":0.57799999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89400000000000002,"5min":4.3399999999999999,"15min":4.3399999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.82899999999999996,"15min":0.98499999999999999},"min":{"1min":0.50800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98899999999999999,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.81000000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72599999999999998,"15min":0.75800000000000001},"min":{"1min":0.5,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94499999999999995,"5min":1.036,"15min":1.036},"last":0.41799999999999998}]},{"osd":5,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.65200000000000002,"15min":0.68799999999999994},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92100000000000004,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.69799999999999995,"15min":0.70899999999999996},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.51900000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.52500000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.99299999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71199999999999997,"15min":0.73299999999999998},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88,"5min":1.036,"15min":1.036},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1007616,"data_stored":454906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64500000000000002,"15min":0.66500000000000004},"min":{"1min":0.46600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.78300000000000003,"5min":1.095,"15min":1.095},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71299999999999997,"15min":0.78500000000000003},"min":{"1min":0.49399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92400000000000004,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.627}]},{"osd":1,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71699999999999997,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91800000000000004,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.72699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.47599999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84799999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68300000000000005,"15min":0.71499999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.90700000000000003,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.68400000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73899999999999999,"15min":0.79400000000000004},"min":{"1min":0.46000000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.92400000000000004,"5min":1.7,"15min":1.7},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72999999999999998,"15min":0.76000000000000001},"min":{"1min":0.51900000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91700000000000004,"5min":1.181,"15min":1.181},"last":0.65000000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59899999999999998,"15min":0.63200000000000001},"min":{"1min":0.34999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.82499999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73399999999999999,"15min":0.77500000000000002},"min":{"1min":0.45400000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":1.615,"15min":1.615},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65300000000000002,"15min":0.67600000000000005},"min":{"1min":0.49299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88200000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67800000000000005,"15min":0.71499999999999997},"min":{"1min":0.42899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.872,"5min":1.569,"15min":1.569},"last":0.57199999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72399999999999998,"15min":0.751},"min":{"1min":0.49299999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.95599999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96599999999999997,"5min":1.383,"15min":1.383},"last":0.73199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":1024000,"data_stored":456540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.49399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98699999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.48899999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.876,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.752}]},{"osd":1,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.54100000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93999999999999995,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0169999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73299999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.629,"5min":0.495,"15min":0.495},"max":{"1min":0.93500000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.626},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.502,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97899999999999998,"5min":1.228,"15min":1.228},"last":0.67000000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71399999999999997,"15min":0.70999999999999996},"min":{"1min":0.54300000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.996,"5min":1.27,"15min":1.27},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.78500000000000003},"min":{"1min":0.59199999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.99299999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65900000000000003,"15min":0.69499999999999995},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92100000000000004,"5min":1.329,"15min":1.329},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69899999999999995,"15min":0.751},"min":{"1min":0.379,"5min":0.373,"15min":0.373},"max":{"1min":0.90000000000000002,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.69599999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65900000000000003,"15min":0.71099999999999997},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84099999999999997,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71199999999999997,"15min":0.76200000000000001},"min":{"1min":0.46300000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.42899999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70399999999999996,"15min":0.748},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78600000000000003,"15min":0.83199999999999996},"min":{"1min":0.53000000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98899999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.46600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":448834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.69699999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50800000000000001,"15min":0.54500000000000004},"min":{"1min":0.29999999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51600000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.503,"15min":0.51700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.63500000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.439},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.56499999999999995},"min":{"1min":0.34200000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67500000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.55600000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72199999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53300000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.753,"15min":0.77300000000000002},"min":{"1min":0.53600000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.101,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76700000000000002,"15min":0.78400000000000003},"min":{"1min":0.495,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0760000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.026}]},{"osd":5,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.56499999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0620000000000001,"5min":1.125,"15min":1.125},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78300000000000003,"15min":0.81299999999999994},"min":{"1min":0.58999999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0629999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.878}]},{"osd":6,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79700000000000004,"15min":0.82199999999999995},"min":{"1min":0.57399999999999995,"5min":0.499,"15min":0.499},"max":{"1min":1.157,"5min":1.226,"15min":1.226},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":1.004}]},{"osd":7,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.82799999999999996,"15min":0.86099999999999999},"min":{"1min":0.61799999999999999,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1799999999999999,"5min":1.202,"15min":1.202},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.85899999999999999,"15min":0.90000000000000002},"min":{"1min":0.55700000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.2010000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.86799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":448994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.46300000000000002,"15min":0.441},"min":{"1min":0.31,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.64900000000000002,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66600000000000004,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.42399999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76900000000000002,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68000000000000005,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.38600000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69599999999999995,"5min":0.752,"15min":0.752},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.437,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84999999999999998,"5min":1.907,"15min":1.907},"last":0.441}]},{"osd":4,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.60299999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.55,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.82299999999999995,"15min":0.85399999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.292,"5min":1.972,"15min":1.972},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.57899999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2250000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5820000000000001,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.74399999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.82499999999999996,"15min":0.82599999999999996},"min":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.645,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.56399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6140000000000001,"5min":1.867,"15min":1.867},"last":0.76600000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84499999999999997,"15min":0.877},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2649999999999999,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.63400000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.278,"5min":2.851,"15min":2.851},"last":0.82499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":448364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42199999999999999,"15min":0.42799999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56000000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66500000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51800000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44700000000000001,"15min":0.46500000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.496,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69399999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66900000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.499,"15min":0.47399999999999998},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.63700000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.53700000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88900000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95099999999999996,"5min":1.2,"15min":1.2},"last":0.85899999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.96799999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97899999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77500000000000002,"15min":0.78000000000000003},"min":{"1min":0.57799999999999996,"5min":0.503,"15min":0.503},"max":{"1min":0.89100000000000001,"5min":1.097,"15min":1.097},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":0.77200000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0209999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.79000000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.79900000000000004},"min":{"1min":0.60199999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0529999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.877},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79500000000000004,"15min":0.78000000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98099999999999998,"5min":1.236,"15min":1.236},"last":0.89500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1585152,"data_stored":1038620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45500000000000002,"15min":0.47699999999999998},"min":{"1min":0.307,"5min":0.245,"15min":0.245},"max":{"1min":0.69699999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.34499999999999997,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76700000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40699999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.503,"15min":0.52100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.40999999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83099999999999996,"5min":1.006,"15min":1.006},"last":0.45200000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.502,"15min":0.51300000000000001},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71599999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56499999999999995,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73899999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42999999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.80500000000000005,"15min":0.79900000000000004},"min":{"1min":0.55700000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.137,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81599999999999995,"15min":0.80600000000000005},"min":{"1min":0.57199999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.252,"15min":1.252},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.58799999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1919999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.78000000000000003,"15min":0.77800000000000002},"min":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.085,"5min":1.462,"15min":1.462},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80100000000000005,"15min":0.80200000000000005},"min":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1060000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1659999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81499999999999995,"15min":0.81299999999999994},"min":{"1min":0.61199999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0409999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.80400000000000005,"15min":0.80100000000000005},"min":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2070000000000001,"5min":1.556,"15min":1.556},"last":0.622}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:44:20.394 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:44:20.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.567+0000 7f87fc200640 1 Processor -- start 2024-03-30T00:44:20.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.567+0000 7f87fc200640 1 -- start start 2024-03-30T00:44:20.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87fc200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f87f4099430 0x7f87f40ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87fc200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87f40cf0c0 0x7f87f40c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87fc200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f40cf510 con 0x7f87f415a810 2024-03-30T00:44:20.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87fc200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f40c4600 con 0x7f87f4099430 2024-03-30T00:44:20.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87fc200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f40cc510 con 0x7f87f40cf0c0 2024-03-30T00:44:20.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87f8e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f87f4099430 0x7f87f40ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87f8e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f87f4099430 0x7f87f40ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53240/0 (socket says 172.21.15.66:53240) 2024-03-30T00:44:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87f8e00640 1 -- 172.21.15.66:0/2977820098 learned_addr learned my addr 172.21.15.66:0/2977820098 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87fa200640 1 --2- 172.21.15.66:0/2977820098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87f40cf0c0 0x7f87f40c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87f8e00640 1 -- 172.21.15.66:0/2977820098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87f40cf0c0 msgr2=0x7f87f40c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87f8e00640 1 --2- 172.21.15.66:0/2977820098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87f40cf0c0 0x7f87f40c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87f8e00640 1 -- 172.21.15.66:0/2977820098 >> v1:172.21.15.156:6789/0 conn(0x7f87f415a810 legacy=0x7f87f40ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87f8e00640 1 -- 172.21.15.66:0/2977820098 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f40998c0 con 0x7f87f4099430 2024-03-30T00:44:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87f8e00640 1 --2- 172.21.15.66:0/2977820098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87f4099430 0x7f87f40ceb80 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009870 tx=0x7f87e001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c72222920ae0d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87f3e00640 1 -- 172.21.15.66:0/2977820098 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e00a8a50 con 0x7f87f4099430 2024-03-30T00:44:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87f3e00640 1 -- 172.21.15.66:0/2977820098 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e00a8bb0 con 0x7f87f4099430 2024-03-30T00:44:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.571+0000 7f87f3e00640 1 -- 172.21.15.66:0/2977820098 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e00b0440 con 0x7f87f4099430 2024-03-30T00:44:20.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 -- 172.21.15.66:0/2977820098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87f4099430 msgr2=0x7f87f40ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 --2- 172.21.15.66:0/2977820098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87f4099430 0x7f87f40ceb80 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009870 tx=0x7f87e001ed50 comp rx=0 tx=0).stop 2024-03-30T00:44:20.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 -- 172.21.15.66:0/2977820098 shutdown_connections 2024-03-30T00:44:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 --2- 172.21.15.66:0/2977820098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87f40cf0c0 0x7f87f40c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 --2- 172.21.15.66:0/2977820098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87f4099430 0x7f87f40ceb80 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 -- 172.21.15.66:0/2977820098 >> 172.21.15.66:0/2977820098 conn(0x7f87f40b2090 msgr2=0x7f87f40b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 -- 172.21.15.66:0/2977820098 shutdown_connections 2024-03-30T00:44:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 -- 172.21.15.66:0/2977820098 wait complete. 2024-03-30T00:44:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 Processor -- start 2024-03-30T00:44:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 -- start start 2024-03-30T00:44:20.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87f40cf0c0 0x7f87f40c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87f415a810 0x7f87f4159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87f4159b50 0x7f87f40b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f4153090 con 0x7f87f40cf0c0 2024-03-30T00:44:20.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f40ca480 con 0x7f87f415a810 2024-03-30T00:44:20.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fc200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f4152760 con 0x7f87f4159b50 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87f9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87f40cf0c0 0x7f87f40c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87f9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87f40cf0c0 0x7f87f40c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53252/0 (socket says 172.21.15.66:53252) 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87f9800640 1 -- 172.21.15.66:0/68896005 learned_addr learned my addr 172.21.15.66:0/68896005 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87f8e00640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87f415a810 0x7f87f4159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87fa200640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87f4159b50 0x7f87f40b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87f9800640 1 -- 172.21.15.66:0/68896005 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87f4159b50 msgr2=0x7f87f40b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87f9800640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87f4159b50 0x7f87f40b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87f9800640 1 -- 172.21.15.66:0/68896005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87f415a810 msgr2=0x7f87f4159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87f9800640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87f415a810 0x7f87f4159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87f9800640 1 -- 172.21.15.66:0/68896005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e00094d0 con 0x7f87f40cf0c0 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.575+0000 7f87f8e00640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87f415a810 0x7f87f4159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.579+0000 7f87f9800640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87f40cf0c0 0x7f87f40c9f40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f87e800c3a0 tx=0x7f87e800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:20.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.579+0000 7f87f1600640 1 -- 172.21.15.66:0/68896005 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e80052d0 con 0x7f87f40cf0c0 2024-03-30T00:44:20.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.579+0000 7f87f1600640 1 -- 172.21.15.66:0/68896005 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e8005450 con 0x7f87f40cf0c0 2024-03-30T00:44:20.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.579+0000 7f87f1600640 1 -- 172.21.15.66:0/68896005 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8003860 con 0x7f87f40cf0c0 2024-03-30T00:44:20.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.579+0000 7f87fc200640 1 -- 172.21.15.66:0/68896005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f4152a40 con 0x7f87f40cf0c0 2024-03-30T00:44:20.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.579+0000 7f87fc200640 1 -- 172.21.15.66:0/68896005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f4155530 con 0x7f87f40cf0c0 2024-03-30T00:44:20.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.579+0000 7f87fc200640 1 -- 172.21.15.66:0/68896005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f4066c30 con 0x7f87f40cf0c0 2024-03-30T00:44:20.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.579+0000 7f87f1600640 1 -- 172.21.15.66:0/68896005 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f87e80039c0 con 0x7f87f40cf0c0 2024-03-30T00:44:20.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.579+0000 7f87f1600640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87d40666e0 0x7f87d4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.579+0000 7f87f1600640 1 -- 172.21.15.66:0/68896005 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f87e80836f0 con 0x7f87f40cf0c0 2024-03-30T00:44:20.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.579+0000 7f87f8e00640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87d40666e0 0x7f87d4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.583+0000 7f87f8e00640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87d40666e0 0x7f87d4068ba0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f87e00b2000 tx=0x7f87e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:20.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.583+0000 7f87f1600640 1 -- 172.21.15.66:0/68896005 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f87e8050900 con 0x7f87f40cf0c0 2024-03-30T00:44:20.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.739+0000 7f87fc200640 1 -- 172.21.15.66:0/68896005 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87f40c4290 con 0x7f87d40666e0 2024-03-30T00:44:20.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.747+0000 7f87f1600640 1 -- 172.21.15.66:0/68896005 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289123 (secure 0 0 0) 0x7f87e00b0540 con 0x7f87d40666e0 2024-03-30T00:44:20.759 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:20.760 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:44:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 -- 172.21.15.66:0/68896005 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87d40666e0 msgr2=0x7f87d4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87d40666e0 0x7f87d4068ba0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f87e00b2000 tx=0x7f87e0008040 comp rx=0 tx=0).stop 2024-03-30T00:44:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 -- 172.21.15.66:0/68896005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87f40cf0c0 msgr2=0x7f87f40c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87f40cf0c0 0x7f87f40c9f40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f87e800c3a0 tx=0x7f87e800c860 comp rx=0 tx=0).stop 2024-03-30T00:44:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 -- 172.21.15.66:0/68896005 shutdown_connections 2024-03-30T00:44:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87f4159b50 0x7f87f40b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87f415a810 0x7f87f4159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87d40666e0 0x7f87d4068ba0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 --2- 172.21.15.66:0/68896005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87f40cf0c0 0x7f87f40c9f40 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 -- 172.21.15.66:0/68896005 >> 172.21.15.66:0/68896005 conn(0x7f87f40b2090 msgr2=0x7f87f40cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 -- 172.21.15.66:0/68896005 shutdown_connections 2024-03-30T00:44:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.751+0000 7f87fc200640 1 -- 172.21.15.66:0/68896005 wait complete. 2024-03-30T00:44:20.779 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-03-30T00:44:18.907951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249780,"kb_used_data":9004,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754724940,"statfs":{"total":773094113280,"available":772838338560,"internally_reserved":0,"allocated":9220096,"data_stored":4791336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":246402616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.044336"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288047+0000","last_change":"2024-03-30T00:44:16.304877+0000","last_active":"2024-03-30T00:44:17.288047+0000","last_peered":"2024-03-30T00:44:17.288047+0000","last_clean":"2024-03-30T00:44:17.288047+0000","last_became_active":"2024-03-30T00:44:16.304512+0000","last_became_peered":"2024-03-30T00:44:16.304512+0000","last_unstale":"2024-03-30T00:44:17.288047+0000","last_undegraded":"2024-03-30T00:44:17.288047+0000","last_fullsized":"2024-03-30T00:44:17.288047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:29.166147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575509+0000","last_change":"2024-03-30T00:44:16.313300+0000","last_active":"2024-03-30T00:44:17.575509+0000","last_peered":"2024-03-30T00:44:17.575509+0000","last_clean":"2024-03-30T00:44:17.575509+0000","last_became_active":"2024-03-30T00:44:16.312331+0000","last_became_peered":"2024-03-30T00:44:16.312331+0000","last_unstale":"2024-03-30T00:44:17.575509+0000","last_undegraded":"2024-03-30T00:44:17.575509+0000","last_fullsized":"2024-03-30T00:44:17.575509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:03:22.521314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287866+0000","last_change":"2024-03-30T00:44:16.296703+0000","last_active":"2024-03-30T00:44:17.287866+0000","last_peered":"2024-03-30T00:44:17.287866+0000","last_clean":"2024-03-30T00:44:17.287866+0000","last_became_active":"2024-03-30T00:44:16.296422+0000","last_became_peered":"2024-03-30T00:44:16.296422+0000","last_unstale":"2024-03-30T00:44:17.287866+0000","last_undegraded":"2024-03-30T00:44:17.287866+0000","last_fullsized":"2024-03-30T00:44:17.287866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:29.166147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572642+0000","last_change":"2024-03-30T00:44:16.300501+0000","last_active":"2024-03-30T00:44:17.572642+0000","last_peered":"2024-03-30T00:44:17.572642+0000","last_clean":"2024-03-30T00:44:17.572642+0000","last_became_active":"2024-03-30T00:44:16.300244+0000","last_became_peered":"2024-03-30T00:44:16.300244+0000","last_unstale":"2024-03-30T00:44:17.572642+0000","last_undegraded":"2024-03-30T00:44:17.572642+0000","last_fullsized":"2024-03-30T00:44:17.572642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:21:39.077425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574430+0000","last_change":"2024-03-30T00:44:16.293733+0000","last_active":"2024-03-30T00:44:17.574430+0000","last_peered":"2024-03-30T00:44:17.574430+0000","last_clean":"2024-03-30T00:44:17.574430+0000","last_became_active":"2024-03-30T00:44:16.293498+0000","last_became_peered":"2024-03-30T00:44:16.293498+0000","last_unstale":"2024-03-30T00:44:17.574430+0000","last_undegraded":"2024-03-30T00:44:17.574430+0000","last_fullsized":"2024-03-30T00:44:17.574430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:44:29.962689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574916+0000","last_change":"2024-03-30T00:44:16.307279+0000","last_active":"2024-03-30T00:44:17.574916+0000","last_peered":"2024-03-30T00:44:17.574916+0000","last_clean":"2024-03-30T00:44:17.574916+0000","last_became_active":"2024-03-30T00:44:16.306977+0000","last_became_peered":"2024-03-30T00:44:16.306977+0000","last_unstale":"2024-03-30T00:44:17.574916+0000","last_undegraded":"2024-03-30T00:44:17.574916+0000","last_fullsized":"2024-03-30T00:44:17.574916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:15:23.555013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573138+0000","last_change":"2024-03-30T00:44:16.298397+0000","last_active":"2024-03-30T00:44:17.573138+0000","last_peered":"2024-03-30T00:44:17.573138+0000","last_clean":"2024-03-30T00:44:17.573138+0000","last_became_active":"2024-03-30T00:44:16.298067+0000","last_became_peered":"2024-03-30T00:44:16.298067+0000","last_unstale":"2024-03-30T00:44:17.573138+0000","last_undegraded":"2024-03-30T00:44:17.573138+0000","last_fullsized":"2024-03-30T00:44:17.573138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:26:21.001769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574616+0000","last_change":"2024-03-30T00:44:16.307545+0000","last_active":"2024-03-30T00:44:17.574616+0000","last_peered":"2024-03-30T00:44:17.574616+0000","last_clean":"2024-03-30T00:44:17.574616+0000","last_became_active":"2024-03-30T00:44:16.307262+0000","last_became_peered":"2024-03-30T00:44:16.307262+0000","last_unstale":"2024-03-30T00:44:17.574616+0000","last_undegraded":"2024-03-30T00:44:17.574616+0000","last_fullsized":"2024-03-30T00:44:17.574616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:22.545875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288591+0000","last_change":"2024-03-30T00:44:16.310142+0000","last_active":"2024-03-30T00:44:17.288591+0000","last_peered":"2024-03-30T00:44:17.288591+0000","last_clean":"2024-03-30T00:44:17.288591+0000","last_became_active":"2024-03-30T00:44:16.309876+0000","last_became_peered":"2024-03-30T00:44:16.309876+0000","last_unstale":"2024-03-30T00:44:17.288591+0000","last_undegraded":"2024-03-30T00:44:17.288591+0000","last_fullsized":"2024-03-30T00:44:17.288591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:26.136845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572426+0000","last_change":"2024-03-30T00:44:16.297834+0000","last_active":"2024-03-30T00:44:17.572426+0000","last_peered":"2024-03-30T00:44:17.572426+0000","last_clean":"2024-03-30T00:44:17.572426+0000","last_became_active":"2024-03-30T00:44:16.296724+0000","last_became_peered":"2024-03-30T00:44:16.296724+0000","last_unstale":"2024-03-30T00:44:17.572426+0000","last_undegraded":"2024-03-30T00:44:17.572426+0000","last_fullsized":"2024-03-30T00:44:17.572426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:09.608809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575385+0000","last_change":"2024-03-30T00:44:16.312271+0000","last_active":"2024-03-30T00:44:17.575385+0000","last_peered":"2024-03-30T00:44:17.575385+0000","last_clean":"2024-03-30T00:44:17.575385+0000","last_became_active":"2024-03-30T00:44:16.312004+0000","last_became_peered":"2024-03-30T00:44:16.312004+0000","last_unstale":"2024-03-30T00:44:17.575385+0000","last_undegraded":"2024-03-30T00:44:17.575385+0000","last_fullsized":"2024-03-30T00:44:17.575385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:20.925778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572745+0000","last_change":"2024-03-30T00:44:16.299208+0000","last_active":"2024-03-30T00:44:17.572745+0000","last_peered":"2024-03-30T00:44:17.572745+0000","last_clean":"2024-03-30T00:44:17.572745+0000","last_became_active":"2024-03-30T00:44:16.298954+0000","last_became_peered":"2024-03-30T00:44:16.298954+0000","last_unstale":"2024-03-30T00:44:17.572745+0000","last_undegraded":"2024-03-30T00:44:17.572745+0000","last_fullsized":"2024-03-30T00:44:17.572745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:01:33.097514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574553+0000","last_change":"2024-03-30T00:44:16.296847+0000","last_active":"2024-03-30T00:44:17.574553+0000","last_peered":"2024-03-30T00:44:17.574553+0000","last_clean":"2024-03-30T00:44:17.574553+0000","last_became_active":"2024-03-30T00:44:16.296611+0000","last_became_peered":"2024-03-30T00:44:16.296611+0000","last_unstale":"2024-03-30T00:44:17.574553+0000","last_undegraded":"2024-03-30T00:44:17.574553+0000","last_fullsized":"2024-03-30T00:44:17.574553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:45:23.669464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574548+0000","last_change":"2024-03-30T00:44:16.300954+0000","last_active":"2024-03-30T00:44:17.574548+0000","last_peered":"2024-03-30T00:44:17.574548+0000","last_clean":"2024-03-30T00:44:17.574548+0000","last_became_active":"2024-03-30T00:44:16.300685+0000","last_became_peered":"2024-03-30T00:44:16.300685+0000","last_unstale":"2024-03-30T00:44:17.574548+0000","last_undegraded":"2024-03-30T00:44:17.574548+0000","last_fullsized":"2024-03-30T00:44:17.574548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:43:21.550411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574036+0000","last_change":"2024-03-30T00:44:16.298641+0000","last_active":"2024-03-30T00:44:17.574036+0000","last_peered":"2024-03-30T00:44:17.574036+0000","last_clean":"2024-03-30T00:44:17.574036+0000","last_became_active":"2024-03-30T00:44:16.295092+0000","last_became_peered":"2024-03-30T00:44:16.295092+0000","last_unstale":"2024-03-30T00:44:17.574036+0000","last_undegraded":"2024-03-30T00:44:17.574036+0000","last_fullsized":"2024-03-30T00:44:17.574036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:27.180701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288520+0000","last_change":"2024-03-30T00:44:16.301865+0000","last_active":"2024-03-30T00:44:17.288520+0000","last_peered":"2024-03-30T00:44:17.288520+0000","last_clean":"2024-03-30T00:44:17.288520+0000","last_became_active":"2024-03-30T00:44:16.301592+0000","last_became_peered":"2024-03-30T00:44:16.301592+0000","last_unstale":"2024-03-30T00:44:17.288520+0000","last_undegraded":"2024-03-30T00:44:17.288520+0000","last_fullsized":"2024-03-30T00:44:17.288520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:27:29.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574639+0000","last_change":"2024-03-30T00:44:16.299617+0000","last_active":"2024-03-30T00:44:17.574639+0000","last_peered":"2024-03-30T00:44:17.574639+0000","last_clean":"2024-03-30T00:44:17.574639+0000","last_became_active":"2024-03-30T00:44:16.299181+0000","last_became_peered":"2024-03-30T00:44:16.299181+0000","last_unstale":"2024-03-30T00:44:17.574639+0000","last_undegraded":"2024-03-30T00:44:17.574639+0000","last_fullsized":"2024-03-30T00:44:17.574639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:18:45.560695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287875+0000","last_change":"2024-03-30T00:44:16.303169+0000","last_active":"2024-03-30T00:44:17.287875+0000","last_peered":"2024-03-30T00:44:17.287875+0000","last_clean":"2024-03-30T00:44:17.287875+0000","last_became_active":"2024-03-30T00:44:16.302922+0000","last_became_peered":"2024-03-30T00:44:16.302922+0000","last_unstale":"2024-03-30T00:44:17.287875+0000","last_undegraded":"2024-03-30T00:44:17.287875+0000","last_fullsized":"2024-03-30T00:44:17.287875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:27:29.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574956+0000","last_change":"2024-03-30T00:44:16.297451+0000","last_active":"2024-03-30T00:44:17.574956+0000","last_peered":"2024-03-30T00:44:17.574956+0000","last_clean":"2024-03-30T00:44:17.574956+0000","last_became_active":"2024-03-30T00:44:16.297073+0000","last_became_peered":"2024-03-30T00:44:16.297073+0000","last_unstale":"2024-03-30T00:44:17.574956+0000","last_undegraded":"2024-03-30T00:44:17.574956+0000","last_fullsized":"2024-03-30T00:44:17.574956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:09:53.712612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574640+0000","last_change":"2024-03-30T00:44:16.291770+0000","last_active":"2024-03-30T00:44:17.574640+0000","last_peered":"2024-03-30T00:44:17.574640+0000","last_clean":"2024-03-30T00:44:17.574640+0000","last_became_active":"2024-03-30T00:44:16.291500+0000","last_became_peered":"2024-03-30T00:44:16.291500+0000","last_unstale":"2024-03-30T00:44:17.574640+0000","last_undegraded":"2024-03-30T00:44:17.574640+0000","last_fullsized":"2024-03-30T00:44:17.574640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:34.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288071+0000","last_change":"2024-03-30T00:44:16.306532+0000","last_active":"2024-03-30T00:44:17.288071+0000","last_peered":"2024-03-30T00:44:17.288071+0000","last_clean":"2024-03-30T00:44:17.288071+0000","last_became_active":"2024-03-30T00:44:16.306132+0000","last_became_peered":"2024-03-30T00:44:16.306132+0000","last_unstale":"2024-03-30T00:44:17.288071+0000","last_undegraded":"2024-03-30T00:44:17.288071+0000","last_fullsized":"2024-03-30T00:44:17.288071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:49.013320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574268+0000","last_change":"2024-03-30T00:44:16.295820+0000","last_active":"2024-03-30T00:44:17.574268+0000","last_peered":"2024-03-30T00:44:17.574268+0000","last_clean":"2024-03-30T00:44:17.574268+0000","last_became_active":"2024-03-30T00:44:16.295585+0000","last_became_peered":"2024-03-30T00:44:16.295585+0000","last_unstale":"2024-03-30T00:44:17.574268+0000","last_undegraded":"2024-03-30T00:44:17.574268+0000","last_fullsized":"2024-03-30T00:44:17.574268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:35:09.835107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287908+0000","last_change":"2024-03-30T00:44:16.303265+0000","last_active":"2024-03-30T00:44:17.287908+0000","last_peered":"2024-03-30T00:44:17.287908+0000","last_clean":"2024-03-30T00:44:17.287908+0000","last_became_active":"2024-03-30T00:44:16.302982+0000","last_became_peered":"2024-03-30T00:44:16.302982+0000","last_unstale":"2024-03-30T00:44:17.287908+0000","last_undegraded":"2024-03-30T00:44:17.287908+0000","last_fullsized":"2024-03-30T00:44:17.287908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:34:00.433221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574015+0000","last_change":"2024-03-30T00:44:16.301715+0000","last_active":"2024-03-30T00:44:17.574015+0000","last_peered":"2024-03-30T00:44:17.574015+0000","last_clean":"2024-03-30T00:44:17.574015+0000","last_became_active":"2024-03-30T00:44:16.301099+0000","last_became_peered":"2024-03-30T00:44:16.301099+0000","last_unstale":"2024-03-30T00:44:17.574015+0000","last_undegraded":"2024-03-30T00:44:17.574015+0000","last_fullsized":"2024-03-30T00:44:17.574015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:07.180780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575431+0000","last_change":"2024-03-30T00:44:16.300902+0000","last_active":"2024-03-30T00:44:17.575431+0000","last_peered":"2024-03-30T00:44:17.575431+0000","last_clean":"2024-03-30T00:44:17.575431+0000","last_became_active":"2024-03-30T00:44:16.300615+0000","last_became_peered":"2024-03-30T00:44:16.300615+0000","last_unstale":"2024-03-30T00:44:17.575431+0000","last_undegraded":"2024-03-30T00:44:17.575431+0000","last_fullsized":"2024-03-30T00:44:17.575431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:15:23.555013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572596+0000","last_change":"2024-03-30T00:44:16.301499+0000","last_active":"2024-03-30T00:44:17.572596+0000","last_peered":"2024-03-30T00:44:17.572596+0000","last_clean":"2024-03-30T00:44:17.572596+0000","last_became_active":"2024-03-30T00:44:16.301152+0000","last_became_peered":"2024-03-30T00:44:16.301152+0000","last_unstale":"2024-03-30T00:44:17.572596+0000","last_undegraded":"2024-03-30T00:44:17.572596+0000","last_fullsized":"2024-03-30T00:44:17.572596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:51.271671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288119+0000","last_change":"2024-03-30T00:44:16.304149+0000","last_active":"2024-03-30T00:44:17.288119+0000","last_peered":"2024-03-30T00:44:17.288119+0000","last_clean":"2024-03-30T00:44:17.288119+0000","last_became_active":"2024-03-30T00:44:16.303898+0000","last_became_peered":"2024-03-30T00:44:16.303898+0000","last_unstale":"2024-03-30T00:44:17.288119+0000","last_undegraded":"2024-03-30T00:44:17.288119+0000","last_fullsized":"2024-03-30T00:44:17.288119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:49.013320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574636+0000","last_change":"2024-03-30T00:44:16.297947+0000","last_active":"2024-03-30T00:44:17.574636+0000","last_peered":"2024-03-30T00:44:17.574636+0000","last_clean":"2024-03-30T00:44:17.574636+0000","last_became_active":"2024-03-30T00:44:16.295260+0000","last_became_peered":"2024-03-30T00:44:16.295260+0000","last_unstale":"2024-03-30T00:44:17.574636+0000","last_undegraded":"2024-03-30T00:44:17.574636+0000","last_fullsized":"2024-03-30T00:44:17.574636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:30:33.280971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575149+0000","last_change":"2024-03-30T00:44:16.300685+0000","last_active":"2024-03-30T00:44:17.575149+0000","last_peered":"2024-03-30T00:44:17.575149+0000","last_clean":"2024-03-30T00:44:17.575149+0000","last_became_active":"2024-03-30T00:44:16.300332+0000","last_became_peered":"2024-03-30T00:44:16.300332+0000","last_unstale":"2024-03-30T00:44:17.575149+0000","last_undegraded":"2024-03-30T00:44:17.575149+0000","last_fullsized":"2024-03-30T00:44:17.575149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:09:16.752883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287531+0000","last_change":"2024-03-30T00:44:16.303630+0000","last_active":"2024-03-30T00:44:17.287531+0000","last_peered":"2024-03-30T00:44:17.287531+0000","last_clean":"2024-03-30T00:44:17.287531+0000","last_became_active":"2024-03-30T00:44:16.303239+0000","last_became_peered":"2024-03-30T00:44:16.303239+0000","last_unstale":"2024-03-30T00:44:17.287531+0000","last_undegraded":"2024-03-30T00:44:17.287531+0000","last_fullsized":"2024-03-30T00:44:17.287531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:27.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287485+0000","last_change":"2024-03-30T00:44:16.299032+0000","last_active":"2024-03-30T00:44:17.287485+0000","last_peered":"2024-03-30T00:44:17.287485+0000","last_clean":"2024-03-30T00:44:17.287485+0000","last_became_active":"2024-03-30T00:44:16.298800+0000","last_became_peered":"2024-03-30T00:44:16.298800+0000","last_unstale":"2024-03-30T00:44:17.287485+0000","last_undegraded":"2024-03-30T00:44:17.287485+0000","last_fullsized":"2024-03-30T00:44:17.287485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:34:00.433221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288424+0000","last_change":"2024-03-30T00:44:16.301931+0000","last_active":"2024-03-30T00:44:17.288424+0000","last_peered":"2024-03-30T00:44:17.288424+0000","last_clean":"2024-03-30T00:44:17.288424+0000","last_became_active":"2024-03-30T00:44:16.301664+0000","last_became_peered":"2024-03-30T00:44:16.301664+0000","last_unstale":"2024-03-30T00:44:17.288424+0000","last_undegraded":"2024-03-30T00:44:17.288424+0000","last_fullsized":"2024-03-30T00:44:17.288424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:08:21.384792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287405+0000","last_change":"2024-03-30T00:44:16.303876+0000","last_active":"2024-03-30T00:44:17.287405+0000","last_peered":"2024-03-30T00:44:17.287405+0000","last_clean":"2024-03-30T00:44:17.287405+0000","last_became_active":"2024-03-30T00:44:16.303629+0000","last_became_peered":"2024-03-30T00:44:16.303629+0000","last_unstale":"2024-03-30T00:44:17.287405+0000","last_undegraded":"2024-03-30T00:44:17.287405+0000","last_fullsized":"2024-03-30T00:44:17.287405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:08:43.430473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574725+0000","last_change":"2024-03-30T00:44:15.894920+0000","last_active":"2024-03-30T00:44:17.574725+0000","last_peered":"2024-03-30T00:44:17.574725+0000","last_clean":"2024-03-30T00:44:17.574725+0000","last_became_active":"2024-03-30T00:44:15.893636+0000","last_became_peered":"2024-03-30T00:44:15.893636+0000","last_unstale":"2024-03-30T00:44:17.574725+0000","last_undegraded":"2024-03-30T00:44:17.574725+0000","last_fullsized":"2024-03-30T00:44:17.574725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:19:37.486774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574131+0000","last_change":"2024-03-30T00:44:16.296368+0000","last_active":"2024-03-30T00:44:17.574131+0000","last_peered":"2024-03-30T00:44:17.574131+0000","last_clean":"2024-03-30T00:44:17.574131+0000","last_became_active":"2024-03-30T00:44:16.295470+0000","last_became_peered":"2024-03-30T00:44:16.295470+0000","last_unstale":"2024-03-30T00:44:17.574131+0000","last_undegraded":"2024-03-30T00:44:17.574131+0000","last_fullsized":"2024-03-30T00:44:17.574131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:42:13.878266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287807+0000","last_change":"2024-03-30T00:44:15.889822+0000","last_active":"2024-03-30T00:44:17.287807+0000","last_peered":"2024-03-30T00:44:17.287807+0000","last_clean":"2024-03-30T00:44:17.287807+0000","last_became_active":"2024-03-30T00:44:15.889544+0000","last_became_peered":"2024-03-30T00:44:15.889544+0000","last_unstale":"2024-03-30T00:44:17.287807+0000","last_undegraded":"2024-03-30T00:44:17.287807+0000","last_fullsized":"2024-03-30T00:44:17.287807+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:09:08.918197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288256+0000","last_change":"2024-03-30T00:44:16.309609+0000","last_active":"2024-03-30T00:44:17.288256+0000","last_peered":"2024-03-30T00:44:17.288256+0000","last_clean":"2024-03-30T00:44:17.288256+0000","last_became_active":"2024-03-30T00:44:16.309122+0000","last_became_peered":"2024-03-30T00:44:16.309122+0000","last_unstale":"2024-03-30T00:44:17.288256+0000","last_undegraded":"2024-03-30T00:44:17.288256+0000","last_fullsized":"2024-03-30T00:44:17.288256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:46:29.847764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287729+0000","last_change":"2024-03-30T00:44:15.894077+0000","last_active":"2024-03-30T00:44:17.287729+0000","last_peered":"2024-03-30T00:44:17.287729+0000","last_clean":"2024-03-30T00:44:17.287729+0000","last_became_active":"2024-03-30T00:44:15.893798+0000","last_became_peered":"2024-03-30T00:44:15.893798+0000","last_unstale":"2024-03-30T00:44:17.287729+0000","last_undegraded":"2024-03-30T00:44:17.287729+0000","last_fullsized":"2024-03-30T00:44:17.287729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:28.329099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288139+0000","last_change":"2024-03-30T00:44:16.305280+0000","last_active":"2024-03-30T00:44:17.288139+0000","last_peered":"2024-03-30T00:44:17.288139+0000","last_clean":"2024-03-30T00:44:17.288139+0000","last_became_active":"2024-03-30T00:44:16.305037+0000","last_became_peered":"2024-03-30T00:44:16.305037+0000","last_unstale":"2024-03-30T00:44:17.288139+0000","last_undegraded":"2024-03-30T00:44:17.288139+0000","last_fullsized":"2024-03-30T00:44:17.288139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:27:29.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573387+0000","last_change":"2024-03-30T00:44:15.891108+0000","last_active":"2024-03-30T00:44:17.573387+0000","last_peered":"2024-03-30T00:44:17.573387+0000","last_clean":"2024-03-30T00:44:17.573387+0000","last_became_active":"2024-03-30T00:44:15.890534+0000","last_became_peered":"2024-03-30T00:44:15.890534+0000","last_unstale":"2024-03-30T00:44:17.573387+0000","last_undegraded":"2024-03-30T00:44:17.573387+0000","last_fullsized":"2024-03-30T00:44:17.573387+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:47:19.831013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575171+0000","last_change":"2024-03-30T00:44:16.296714+0000","last_active":"2024-03-30T00:44:17.575171+0000","last_peered":"2024-03-30T00:44:17.575171+0000","last_clean":"2024-03-30T00:44:17.575171+0000","last_became_active":"2024-03-30T00:44:16.296118+0000","last_became_peered":"2024-03-30T00:44:16.296118+0000","last_unstale":"2024-03-30T00:44:17.575171+0000","last_undegraded":"2024-03-30T00:44:17.575171+0000","last_fullsized":"2024-03-30T00:44:17.575171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:39:04.876527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288250+0000","last_change":"2024-03-30T00:44:15.893199+0000","last_active":"2024-03-30T00:44:17.288250+0000","last_peered":"2024-03-30T00:44:17.288250+0000","last_clean":"2024-03-30T00:44:17.288250+0000","last_became_active":"2024-03-30T00:44:15.892791+0000","last_became_peered":"2024-03-30T00:44:15.892791+0000","last_unstale":"2024-03-30T00:44:17.288250+0000","last_undegraded":"2024-03-30T00:44:17.288250+0000","last_fullsized":"2024-03-30T00:44:17.288250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:44.963645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575406+0000","last_change":"2024-03-30T00:44:16.297214+0000","last_active":"2024-03-30T00:44:17.575406+0000","last_peered":"2024-03-30T00:44:17.575406+0000","last_clean":"2024-03-30T00:44:17.575406+0000","last_became_active":"2024-03-30T00:44:16.296958+0000","last_became_peered":"2024-03-30T00:44:16.296958+0000","last_unstale":"2024-03-30T00:44:17.575406+0000","last_undegraded":"2024-03-30T00:44:17.575406+0000","last_fullsized":"2024-03-30T00:44:17.575406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:40:45.735026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572309+0000","last_change":"2024-03-30T00:44:15.895373+0000","last_active":"2024-03-30T00:44:17.572309+0000","last_peered":"2024-03-30T00:44:17.572309+0000","last_clean":"2024-03-30T00:44:17.572309+0000","last_became_active":"2024-03-30T00:44:15.895102+0000","last_became_peered":"2024-03-30T00:44:15.895102+0000","last_unstale":"2024-03-30T00:44:17.572309+0000","last_undegraded":"2024-03-30T00:44:17.572309+0000","last_fullsized":"2024-03-30T00:44:17.572309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:33:54.146938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288649+0000","last_change":"2024-03-30T00:44:16.308709+0000","last_active":"2024-03-30T00:44:17.288649+0000","last_peered":"2024-03-30T00:44:17.288649+0000","last_clean":"2024-03-30T00:44:17.288649+0000","last_became_active":"2024-03-30T00:44:16.308398+0000","last_became_peered":"2024-03-30T00:44:16.308398+0000","last_unstale":"2024-03-30T00:44:17.288649+0000","last_undegraded":"2024-03-30T00:44:17.288649+0000","last_fullsized":"2024-03-30T00:44:17.288649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:31.152253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574631+0000","last_change":"2024-03-30T00:44:15.889303+0000","last_active":"2024-03-30T00:44:17.574631+0000","last_peered":"2024-03-30T00:44:17.574631+0000","last_clean":"2024-03-30T00:44:17.574631+0000","last_became_active":"2024-03-30T00:44:15.889035+0000","last_became_peered":"2024-03-30T00:44:15.889035+0000","last_unstale":"2024-03-30T00:44:17.574631+0000","last_undegraded":"2024-03-30T00:44:17.574631+0000","last_fullsized":"2024-03-30T00:44:17.574631+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:26:27.776857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573604+0000","last_change":"2024-03-30T00:44:16.298589+0000","last_active":"2024-03-30T00:44:17.573604+0000","last_peered":"2024-03-30T00:44:17.573604+0000","last_clean":"2024-03-30T00:44:17.573604+0000","last_became_active":"2024-03-30T00:44:16.297920+0000","last_became_peered":"2024-03-30T00:44:16.297920+0000","last_unstale":"2024-03-30T00:44:17.573604+0000","last_undegraded":"2024-03-30T00:44:17.573604+0000","last_fullsized":"2024-03-30T00:44:17.573604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:51.509235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574118+0000","last_change":"2024-03-30T00:44:15.895431+0000","last_active":"2024-03-30T00:44:17.574118+0000","last_peered":"2024-03-30T00:44:17.574118+0000","last_clean":"2024-03-30T00:44:17.574118+0000","last_became_active":"2024-03-30T00:44:15.894494+0000","last_became_peered":"2024-03-30T00:44:15.894494+0000","last_unstale":"2024-03-30T00:44:17.574118+0000","last_undegraded":"2024-03-30T00:44:17.574118+0000","last_fullsized":"2024-03-30T00:44:17.574118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:34:02.924271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574105+0000","last_change":"2024-03-30T00:44:16.301930+0000","last_active":"2024-03-30T00:44:17.574105+0000","last_peered":"2024-03-30T00:44:17.574105+0000","last_clean":"2024-03-30T00:44:17.574105+0000","last_became_active":"2024-03-30T00:44:16.301424+0000","last_became_peered":"2024-03-30T00:44:16.301424+0000","last_unstale":"2024-03-30T00:44:17.574105+0000","last_undegraded":"2024-03-30T00:44:17.574105+0000","last_fullsized":"2024-03-30T00:44:17.574105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:49.433494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287984+0000","last_change":"2024-03-30T00:44:15.895696+0000","last_active":"2024-03-30T00:44:17.287984+0000","last_peered":"2024-03-30T00:44:17.287984+0000","last_clean":"2024-03-30T00:44:17.287984+0000","last_became_active":"2024-03-30T00:44:15.894153+0000","last_became_peered":"2024-03-30T00:44:15.894153+0000","last_unstale":"2024-03-30T00:44:17.287984+0000","last_undegraded":"2024-03-30T00:44:17.287984+0000","last_fullsized":"2024-03-30T00:44:17.287984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:47:07.113229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572419+0000","last_change":"2024-03-30T00:44:16.295860+0000","last_active":"2024-03-30T00:44:17.572419+0000","last_peered":"2024-03-30T00:44:17.572419+0000","last_clean":"2024-03-30T00:44:17.572419+0000","last_became_active":"2024-03-30T00:44:16.295577+0000","last_became_peered":"2024-03-30T00:44:16.295577+0000","last_unstale":"2024-03-30T00:44:17.572419+0000","last_undegraded":"2024-03-30T00:44:17.572419+0000","last_fullsized":"2024-03-30T00:44:17.572419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:34:43.997849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572640+0000","last_change":"2024-03-30T00:44:15.893771+0000","last_active":"2024-03-30T00:44:17.572640+0000","last_peered":"2024-03-30T00:44:17.572640+0000","last_clean":"2024-03-30T00:44:17.572640+0000","last_became_active":"2024-03-30T00:44:15.893503+0000","last_became_peered":"2024-03-30T00:44:15.893503+0000","last_unstale":"2024-03-30T00:44:17.572640+0000","last_undegraded":"2024-03-30T00:44:17.572640+0000","last_fullsized":"2024-03-30T00:44:17.572640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:28.329099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574701+0000","last_change":"2024-03-30T00:44:16.294343+0000","last_active":"2024-03-30T00:44:17.574701+0000","last_peered":"2024-03-30T00:44:17.574701+0000","last_clean":"2024-03-30T00:44:17.574701+0000","last_became_active":"2024-03-30T00:44:16.293730+0000","last_became_peered":"2024-03-30T00:44:16.293730+0000","last_unstale":"2024-03-30T00:44:17.574701+0000","last_undegraded":"2024-03-30T00:44:17.574701+0000","last_fullsized":"2024-03-30T00:44:17.574701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:11:23.497004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574691+0000","last_change":"2024-03-30T00:44:15.894288+0000","last_active":"2024-03-30T00:44:17.574691+0000","last_peered":"2024-03-30T00:44:17.574691+0000","last_clean":"2024-03-30T00:44:17.574691+0000","last_became_active":"2024-03-30T00:44:15.894035+0000","last_became_peered":"2024-03-30T00:44:15.894035+0000","last_unstale":"2024-03-30T00:44:17.574691+0000","last_undegraded":"2024-03-30T00:44:17.574691+0000","last_fullsized":"2024-03-30T00:44:17.574691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:23.741176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287691+0000","last_change":"2024-03-30T00:44:16.297732+0000","last_active":"2024-03-30T00:44:17.287691+0000","last_peered":"2024-03-30T00:44:17.287691+0000","last_clean":"2024-03-30T00:44:17.287691+0000","last_became_active":"2024-03-30T00:44:16.297478+0000","last_became_peered":"2024-03-30T00:44:16.297478+0000","last_unstale":"2024-03-30T00:44:17.287691+0000","last_undegraded":"2024-03-30T00:44:17.287691+0000","last_fullsized":"2024-03-30T00:44:17.287691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:49.013320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573552+0000","last_change":"2024-03-30T00:44:15.899251+0000","last_active":"2024-03-30T00:44:17.573552+0000","last_peered":"2024-03-30T00:44:17.573552+0000","last_clean":"2024-03-30T00:44:17.573552+0000","last_became_active":"2024-03-30T00:44:15.898675+0000","last_became_peered":"2024-03-30T00:44:15.898675+0000","last_unstale":"2024-03-30T00:44:17.573552+0000","last_undegraded":"2024-03-30T00:44:17.573552+0000","last_fullsized":"2024-03-30T00:44:17.573552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:34:17.023248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572745+0000","last_change":"2024-03-30T00:44:16.295026+0000","last_active":"2024-03-30T00:44:17.572745+0000","last_peered":"2024-03-30T00:44:17.572745+0000","last_clean":"2024-03-30T00:44:17.572745+0000","last_became_active":"2024-03-30T00:44:16.292104+0000","last_became_peered":"2024-03-30T00:44:16.292104+0000","last_unstale":"2024-03-30T00:44:17.572745+0000","last_undegraded":"2024-03-30T00:44:17.572745+0000","last_fullsized":"2024-03-30T00:44:17.572745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:27.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572303+0000","last_change":"2024-03-30T00:44:15.897855+0000","last_active":"2024-03-30T00:44:17.572303+0000","last_peered":"2024-03-30T00:44:17.572303+0000","last_clean":"2024-03-30T00:44:17.572303+0000","last_became_active":"2024-03-30T00:44:15.897514+0000","last_became_peered":"2024-03-30T00:44:15.897514+0000","last_unstale":"2024-03-30T00:44:17.572303+0000","last_undegraded":"2024-03-30T00:44:17.572303+0000","last_fullsized":"2024-03-30T00:44:17.572303+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:30.149492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573534+0000","last_change":"2024-03-30T00:44:16.298473+0000","last_active":"2024-03-30T00:44:17.573534+0000","last_peered":"2024-03-30T00:44:17.573534+0000","last_clean":"2024-03-30T00:44:17.573534+0000","last_became_active":"2024-03-30T00:44:16.298155+0000","last_became_peered":"2024-03-30T00:44:16.298155+0000","last_unstale":"2024-03-30T00:44:17.573534+0000","last_undegraded":"2024-03-30T00:44:17.573534+0000","last_fullsized":"2024-03-30T00:44:17.573534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:31:21.221158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287534+0000","last_change":"2024-03-30T00:44:15.896610+0000","last_active":"2024-03-30T00:44:17.287534+0000","last_peered":"2024-03-30T00:44:17.287534+0000","last_clean":"2024-03-30T00:44:17.287534+0000","last_became_active":"2024-03-30T00:44:15.896357+0000","last_became_peered":"2024-03-30T00:44:15.896357+0000","last_unstale":"2024-03-30T00:44:17.287534+0000","last_undegraded":"2024-03-30T00:44:17.287534+0000","last_fullsized":"2024-03-30T00:44:17.287534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:45.617202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573548+0000","last_change":"2024-03-30T00:44:16.305318+0000","last_active":"2024-03-30T00:44:17.573548+0000","last_peered":"2024-03-30T00:44:17.573548+0000","last_clean":"2024-03-30T00:44:17.573548+0000","last_became_active":"2024-03-30T00:44:16.304906+0000","last_became_peered":"2024-03-30T00:44:16.304906+0000","last_unstale":"2024-03-30T00:44:17.573548+0000","last_undegraded":"2024-03-30T00:44:17.573548+0000","last_fullsized":"2024-03-30T00:44:17.573548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:50.698731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574324+0000","last_change":"2024-03-30T00:44:15.890743+0000","last_active":"2024-03-30T00:44:17.574324+0000","last_peered":"2024-03-30T00:44:17.574324+0000","last_clean":"2024-03-30T00:44:17.574324+0000","last_became_active":"2024-03-30T00:44:15.890487+0000","last_became_peered":"2024-03-30T00:44:15.890487+0000","last_unstale":"2024-03-30T00:44:17.574324+0000","last_undegraded":"2024-03-30T00:44:17.574324+0000","last_fullsized":"2024-03-30T00:44:17.574324+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:39.171876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573710+0000","last_change":"2024-03-30T00:44:16.302640+0000","last_active":"2024-03-30T00:44:17.573710+0000","last_peered":"2024-03-30T00:44:17.573710+0000","last_clean":"2024-03-30T00:44:17.573710+0000","last_became_active":"2024-03-30T00:44:16.302357+0000","last_became_peered":"2024-03-30T00:44:16.302357+0000","last_unstale":"2024-03-30T00:44:17.573710+0000","last_undegraded":"2024-03-30T00:44:17.573710+0000","last_fullsized":"2024-03-30T00:44:17.573710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:35:57.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574192+0000","last_change":"2024-03-30T00:44:15.890728+0000","last_active":"2024-03-30T00:44:17.574192+0000","last_peered":"2024-03-30T00:44:17.574192+0000","last_clean":"2024-03-30T00:44:17.574192+0000","last_became_active":"2024-03-30T00:44:15.890428+0000","last_became_peered":"2024-03-30T00:44:15.890428+0000","last_unstale":"2024-03-30T00:44:17.574192+0000","last_undegraded":"2024-03-30T00:44:17.574192+0000","last_fullsized":"2024-03-30T00:44:17.574192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:39:42.688699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574094+0000","last_change":"2024-03-30T00:44:16.289471+0000","last_active":"2024-03-30T00:44:17.574094+0000","last_peered":"2024-03-30T00:44:17.574094+0000","last_clean":"2024-03-30T00:44:17.574094+0000","last_became_active":"2024-03-30T00:44:16.289235+0000","last_became_peered":"2024-03-30T00:44:16.289235+0000","last_unstale":"2024-03-30T00:44:17.574094+0000","last_undegraded":"2024-03-30T00:44:17.574094+0000","last_fullsized":"2024-03-30T00:44:17.574094+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:58:31.868890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288086+0000","last_change":"2024-03-30T00:44:15.893511+0000","last_active":"2024-03-30T00:44:17.288086+0000","last_peered":"2024-03-30T00:44:17.288086+0000","last_clean":"2024-03-30T00:44:17.288086+0000","last_became_active":"2024-03-30T00:44:15.893221+0000","last_became_peered":"2024-03-30T00:44:15.893221+0000","last_unstale":"2024-03-30T00:44:17.288086+0000","last_undegraded":"2024-03-30T00:44:17.288086+0000","last_fullsized":"2024-03-30T00:44:17.288086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:38.935040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574277+0000","last_change":"2024-03-30T00:44:16.297406+0000","last_active":"2024-03-30T00:44:17.574277+0000","last_peered":"2024-03-30T00:44:17.574277+0000","last_clean":"2024-03-30T00:44:17.574277+0000","last_became_active":"2024-03-30T00:44:16.297157+0000","last_became_peered":"2024-03-30T00:44:16.297157+0000","last_unstale":"2024-03-30T00:44:17.574277+0000","last_undegraded":"2024-03-30T00:44:17.574277+0000","last_fullsized":"2024-03-30T00:44:17.574277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:52:51.815643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288731+0000","last_change":"2024-03-30T00:44:15.895444+0000","last_active":"2024-03-30T00:44:17.288731+0000","last_peered":"2024-03-30T00:44:17.288731+0000","last_clean":"2024-03-30T00:44:17.288731+0000","last_became_active":"2024-03-30T00:44:15.895116+0000","last_became_peered":"2024-03-30T00:44:15.895116+0000","last_unstale":"2024-03-30T00:44:17.288731+0000","last_undegraded":"2024-03-30T00:44:17.288731+0000","last_fullsized":"2024-03-30T00:44:17.288731+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:50.506474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574369+0000","last_change":"2024-03-30T00:44:16.302258+0000","last_active":"2024-03-30T00:44:17.574369+0000","last_peered":"2024-03-30T00:44:17.574369+0000","last_clean":"2024-03-30T00:44:17.574369+0000","last_became_active":"2024-03-30T00:44:16.302019+0000","last_became_peered":"2024-03-30T00:44:16.302019+0000","last_unstale":"2024-03-30T00:44:17.574369+0000","last_undegraded":"2024-03-30T00:44:17.574369+0000","last_fullsized":"2024-03-30T00:44:17.574369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:34:39.338907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574782+0000","last_change":"2024-03-30T00:44:15.898203+0000","last_active":"2024-03-30T00:44:17.574782+0000","last_peered":"2024-03-30T00:44:17.574782+0000","last_clean":"2024-03-30T00:44:17.574782+0000","last_became_active":"2024-03-30T00:44:15.897951+0000","last_became_peered":"2024-03-30T00:44:15.897951+0000","last_unstale":"2024-03-30T00:44:17.574782+0000","last_undegraded":"2024-03-30T00:44:17.574782+0000","last_fullsized":"2024-03-30T00:44:17.574782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:12:39.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574320+0000","last_change":"2024-03-30T00:44:16.290687+0000","last_active":"2024-03-30T00:44:17.574320+0000","last_peered":"2024-03-30T00:44:17.574320+0000","last_clean":"2024-03-30T00:44:17.574320+0000","last_became_active":"2024-03-30T00:44:16.289250+0000","last_became_peered":"2024-03-30T00:44:16.289250+0000","last_unstale":"2024-03-30T00:44:17.574320+0000","last_undegraded":"2024-03-30T00:44:17.574320+0000","last_fullsized":"2024-03-30T00:44:17.574320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:07.180780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287624+0000","last_change":"2024-03-30T00:44:15.889708+0000","last_active":"2024-03-30T00:44:17.287624+0000","last_peered":"2024-03-30T00:44:17.287624+0000","last_clean":"2024-03-30T00:44:17.287624+0000","last_became_active":"2024-03-30T00:44:15.889450+0000","last_became_peered":"2024-03-30T00:44:15.889450+0000","last_unstale":"2024-03-30T00:44:17.287624+0000","last_undegraded":"2024-03-30T00:44:17.287624+0000","last_fullsized":"2024-03-30T00:44:17.287624+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:47:07.113229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575289+0000","last_change":"2024-03-30T00:44:16.305471+0000","last_active":"2024-03-30T00:44:17.575289+0000","last_peered":"2024-03-30T00:44:17.575289+0000","last_clean":"2024-03-30T00:44:17.575289+0000","last_became_active":"2024-03-30T00:44:16.305163+0000","last_became_peered":"2024-03-30T00:44:16.305163+0000","last_unstale":"2024-03-30T00:44:17.575289+0000","last_undegraded":"2024-03-30T00:44:17.575289+0000","last_fullsized":"2024-03-30T00:44:17.575289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:33.986685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574414+0000","last_change":"2024-03-30T00:44:15.888585+0000","last_active":"2024-03-30T00:44:17.574414+0000","last_peered":"2024-03-30T00:44:17.574414+0000","last_clean":"2024-03-30T00:44:17.574414+0000","last_became_active":"2024-03-30T00:44:15.888266+0000","last_became_peered":"2024-03-30T00:44:15.888266+0000","last_unstale":"2024-03-30T00:44:17.574414+0000","last_undegraded":"2024-03-30T00:44:17.574414+0000","last_fullsized":"2024-03-30T00:44:17.574414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:08.606048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575616+0000","last_change":"2024-03-30T00:44:16.307577+0000","last_active":"2024-03-30T00:44:17.575616+0000","last_peered":"2024-03-30T00:44:17.575616+0000","last_clean":"2024-03-30T00:44:17.575616+0000","last_became_active":"2024-03-30T00:44:16.307344+0000","last_became_peered":"2024-03-30T00:44:16.307344+0000","last_unstale":"2024-03-30T00:44:17.575616+0000","last_undegraded":"2024-03-30T00:44:17.575616+0000","last_fullsized":"2024-03-30T00:44:17.575616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:06:31.726939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574334+0000","last_change":"2024-03-30T00:44:15.890129+0000","last_active":"2024-03-30T00:44:17.574334+0000","last_peered":"2024-03-30T00:44:17.574334+0000","last_clean":"2024-03-30T00:44:17.574334+0000","last_became_active":"2024-03-30T00:44:15.889889+0000","last_became_peered":"2024-03-30T00:44:15.889889+0000","last_unstale":"2024-03-30T00:44:17.574334+0000","last_undegraded":"2024-03-30T00:44:17.574334+0000","last_fullsized":"2024-03-30T00:44:17.574334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:34:42.995087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573687+0000","last_change":"2024-03-30T00:44:16.298769+0000","last_active":"2024-03-30T00:44:17.573687+0000","last_peered":"2024-03-30T00:44:17.573687+0000","last_clean":"2024-03-30T00:44:17.573687+0000","last_became_active":"2024-03-30T00:44:16.298288+0000","last_became_peered":"2024-03-30T00:44:16.298288+0000","last_unstale":"2024-03-30T00:44:17.573687+0000","last_undegraded":"2024-03-30T00:44:17.573687+0000","last_fullsized":"2024-03-30T00:44:17.573687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:45.966406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574178+0000","last_change":"2024-03-30T00:44:15.891005+0000","last_active":"2024-03-30T00:44:17.574178+0000","last_peered":"2024-03-30T00:44:17.574178+0000","last_clean":"2024-03-30T00:44:17.574178+0000","last_became_active":"2024-03-30T00:44:15.890769+0000","last_became_peered":"2024-03-30T00:44:15.890769+0000","last_unstale":"2024-03-30T00:44:17.574178+0000","last_undegraded":"2024-03-30T00:44:17.574178+0000","last_fullsized":"2024-03-30T00:44:17.574178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:50.268910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287679+0000","last_change":"2024-03-30T00:44:16.300595+0000","last_active":"2024-03-30T00:44:17.287679+0000","last_peered":"2024-03-30T00:44:17.287679+0000","last_clean":"2024-03-30T00:44:17.287679+0000","last_became_active":"2024-03-30T00:44:16.300218+0000","last_became_peered":"2024-03-30T00:44:16.300218+0000","last_unstale":"2024-03-30T00:44:17.287679+0000","last_undegraded":"2024-03-30T00:44:17.287679+0000","last_fullsized":"2024-03-30T00:44:17.287679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:08:43.430473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.572646+0000","last_change":"2024-03-30T00:44:15.887464+0000","last_active":"2024-03-30T00:44:17.572646+0000","last_peered":"2024-03-30T00:44:17.572646+0000","last_clean":"2024-03-30T00:44:17.572646+0000","last_became_active":"2024-03-30T00:44:15.886447+0000","last_became_peered":"2024-03-30T00:44:15.886447+0000","last_unstale":"2024-03-30T00:44:17.572646+0000","last_undegraded":"2024-03-30T00:44:17.572646+0000","last_fullsized":"2024-03-30T00:44:17.572646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:40:25.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573225+0000","last_change":"2024-03-30T00:44:16.303260+0000","last_active":"2024-03-30T00:44:17.573225+0000","last_peered":"2024-03-30T00:44:17.573225+0000","last_clean":"2024-03-30T00:44:17.573225+0000","last_became_active":"2024-03-30T00:44:16.302811+0000","last_became_peered":"2024-03-30T00:44:16.302811+0000","last_unstale":"2024-03-30T00:44:17.573225+0000","last_undegraded":"2024-03-30T00:44:17.573225+0000","last_fullsized":"2024-03-30T00:44:17.573225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:49:15.557324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575606+0000","last_change":"2024-03-30T00:44:15.891396+0000","last_active":"2024-03-30T00:44:17.575606+0000","last_peered":"2024-03-30T00:44:17.575606+0000","last_clean":"2024-03-30T00:44:17.575606+0000","last_became_active":"2024-03-30T00:44:15.891155+0000","last_became_peered":"2024-03-30T00:44:15.891155+0000","last_unstale":"2024-03-30T00:44:17.575606+0000","last_undegraded":"2024-03-30T00:44:17.575606+0000","last_fullsized":"2024-03-30T00:44:17.575606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:50.268910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288257+0000","last_change":"2024-03-30T00:44:16.309424+0000","last_active":"2024-03-30T00:44:17.288257+0000","last_peered":"2024-03-30T00:44:17.288257+0000","last_clean":"2024-03-30T00:44:17.288257+0000","last_became_active":"2024-03-30T00:44:16.308766+0000","last_became_peered":"2024-03-30T00:44:16.308766+0000","last_unstale":"2024-03-30T00:44:17.288257+0000","last_undegraded":"2024-03-30T00:44:17.288257+0000","last_fullsized":"2024-03-30T00:44:17.288257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:33:55.149700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287964+0000","last_change":"2024-03-30T00:44:15.893935+0000","last_active":"2024-03-30T00:44:17.287964+0000","last_peered":"2024-03-30T00:44:17.287964+0000","last_clean":"2024-03-30T00:44:17.287964+0000","last_became_active":"2024-03-30T00:44:15.893673+0000","last_became_peered":"2024-03-30T00:44:15.893673+0000","last_unstale":"2024-03-30T00:44:17.287964+0000","last_undegraded":"2024-03-30T00:44:17.287964+0000","last_fullsized":"2024-03-30T00:44:17.287964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:14.918980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.575185+0000","last_change":"2024-03-30T00:44:16.302037+0000","last_active":"2024-03-30T00:44:17.575185+0000","last_peered":"2024-03-30T00:44:17.575185+0000","last_clean":"2024-03-30T00:44:17.575185+0000","last_became_active":"2024-03-30T00:44:16.301516+0000","last_became_peered":"2024-03-30T00:44:16.301516+0000","last_unstale":"2024-03-30T00:44:17.575185+0000","last_undegraded":"2024-03-30T00:44:17.575185+0000","last_fullsized":"2024-03-30T00:44:17.575185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:39:17.000234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287693+0000","last_change":"2024-03-30T00:44:15.888482+0000","last_active":"2024-03-30T00:44:17.287693+0000","last_peered":"2024-03-30T00:44:17.287693+0000","last_clean":"2024-03-30T00:44:17.287693+0000","last_became_active":"2024-03-30T00:44:15.888165+0000","last_became_peered":"2024-03-30T00:44:15.888165+0000","last_unstale":"2024-03-30T00:44:17.287693+0000","last_undegraded":"2024-03-30T00:44:17.287693+0000","last_fullsized":"2024-03-30T00:44:17.287693+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:09:23.017153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573340+0000","last_change":"2024-03-30T00:44:16.303356+0000","last_active":"2024-03-30T00:44:17.573340+0000","last_peered":"2024-03-30T00:44:17.573340+0000","last_clean":"2024-03-30T00:44:17.573340+0000","last_became_active":"2024-03-30T00:44:16.303052+0000","last_became_peered":"2024-03-30T00:44:16.303052+0000","last_unstale":"2024-03-30T00:44:17.573340+0000","last_undegraded":"2024-03-30T00:44:17.573340+0000","last_fullsized":"2024-03-30T00:44:17.573340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:57:20.784137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573301+0000","last_change":"2024-03-30T00:44:15.891617+0000","last_active":"2024-03-30T00:44:17.573301+0000","last_peered":"2024-03-30T00:44:17.573301+0000","last_clean":"2024-03-30T00:44:17.573301+0000","last_became_active":"2024-03-30T00:44:15.891335+0000","last_became_peered":"2024-03-30T00:44:15.891335+0000","last_unstale":"2024-03-30T00:44:17.573301+0000","last_undegraded":"2024-03-30T00:44:17.573301+0000","last_fullsized":"2024-03-30T00:44:17.573301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:43:30.408270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573478+0000","last_change":"2024-03-30T00:44:16.299191+0000","last_active":"2024-03-30T00:44:17.573478+0000","last_peered":"2024-03-30T00:44:17.573478+0000","last_clean":"2024-03-30T00:44:17.573478+0000","last_became_active":"2024-03-30T00:44:16.297785+0000","last_became_peered":"2024-03-30T00:44:16.297785+0000","last_unstale":"2024-03-30T00:44:17.573478+0000","last_undegraded":"2024-03-30T00:44:17.573478+0000","last_fullsized":"2024-03-30T00:44:17.573478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:09:11.545797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574469+0000","last_change":"2024-03-30T00:44:15.895307+0000","last_active":"2024-03-30T00:44:17.574469+0000","last_peered":"2024-03-30T00:44:17.574469+0000","last_clean":"2024-03-30T00:44:17.574469+0000","last_became_active":"2024-03-30T00:44:15.894810+0000","last_became_peered":"2024-03-30T00:44:15.894810+0000","last_unstale":"2024-03-30T00:44:17.574469+0000","last_undegraded":"2024-03-30T00:44:17.574469+0000","last_fullsized":"2024-03-30T00:44:17.574469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:55:34.160063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287965+0000","last_change":"2024-03-30T00:44:16.298000+0000","last_active":"2024-03-30T00:44:17.287965+0000","last_peered":"2024-03-30T00:44:17.287965+0000","last_clean":"2024-03-30T00:44:17.287965+0000","last_became_active":"2024-03-30T00:44:16.297792+0000","last_became_peered":"2024-03-30T00:44:16.297792+0000","last_unstale":"2024-03-30T00:44:17.287965+0000","last_undegraded":"2024-03-30T00:44:17.287965+0000","last_fullsized":"2024-03-30T00:44:17.287965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:08:21.384792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.573250+0000","last_change":"2024-03-30T00:44:15.894525+0000","last_active":"2024-03-30T00:44:17.573250+0000","last_peered":"2024-03-30T00:44:17.573250+0000","last_clean":"2024-03-30T00:44:17.573250+0000","last_became_active":"2024-03-30T00:44:15.894273+0000","last_became_peered":"2024-03-30T00:44:15.894273+0000","last_unstale":"2024-03-30T00:44:17.573250+0000","last_undegraded":"2024-03-30T00:44:17.573250+0000","last_fullsized":"2024-03-30T00:44:17.573250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:36:40.678790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.288032+0000","last_change":"2024-03-30T00:44:16.292522+0000","last_active":"2024-03-30T00:44:17.288032+0000","last_peered":"2024-03-30T00:44:17.288032+0000","last_clean":"2024-03-30T00:44:17.288032+0000","last_became_active":"2024-03-30T00:44:16.292255+0000","last_became_peered":"2024-03-30T00:44:16.292255+0000","last_unstale":"2024-03-30T00:44:17.288032+0000","last_undegraded":"2024-03-30T00:44:17.288032+0000","last_fullsized":"2024-03-30T00:44:17.288032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:46:29.847764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287391+0000","last_change":"2024-03-30T00:44:15.882801+0000","last_active":"2024-03-30T00:44:17.287391+0000","last_peered":"2024-03-30T00:44:17.287391+0000","last_clean":"2024-03-30T00:44:17.287391+0000","last_became_active":"2024-03-30T00:44:15.882526+0000","last_became_peered":"2024-03-30T00:44:15.882526+0000","last_unstale":"2024-03-30T00:44:17.287391+0000","last_undegraded":"2024-03-30T00:44:17.287391+0000","last_fullsized":"2024-03-30T00:44:17.287391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:45.617202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.287865+0000","last_change":"2024-03-30T00:44:16.308219+0000","last_active":"2024-03-30T00:44:17.287865+0000","last_peered":"2024-03-30T00:44:17.287865+0000","last_clean":"2024-03-30T00:44:17.287865+0000","last_became_active":"2024-03-30T00:44:16.307891+0000","last_became_peered":"2024-03-30T00:44:16.307891+0000","last_unstale":"2024-03-30T00:44:17.287865+0000","last_undegraded":"2024-03-30T00:44:17.287865+0000","last_fullsized":"2024-03-30T00:44:17.287865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:15.863712+0000","last_clean_scrub_stamp":"2024-03-30T00:44:15.863712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:08:43.430473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:44:17.574975+0000","last_change":"2024-03-30T00:44:15.894627+0000","last_active":"2024-03-30T00:44:17.574975+0000","last_peered":"2024-03-30T00:44:17.574975+0000","last_clean":"2024-03-30T00:44:17.574975+0000","last_became_active":"2024-03-30T00:44:15.894365+0000","last_became_peered":"2024-03-30T00:44:15.894365+0000","last_unstale":"2024-03-30T00:44:17.574975+0000","last_undegraded":"2024-03-30T00:44:17.574975+0000","last_fullsized":"2024-03-30T00:44:17.574975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:44:14.860951+0000","last_clean_scrub_stamp":"2024-03-30T00:44:14.860951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:01:32.094752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":227,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-30T00:44:16.285847+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:44:16.285847+0000","last_peered":"2024-03-30T00:44:16.285847+0000","last_clean":"2024-03-30T00:44:16.285847+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:44:16.285847+0000","last_undegraded":"2024-03-30T00:44:16.285847+0000","last_fullsized":"2024-03-30T00:44:16.285847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1601536,"data_stored":1045262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.501,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88800000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95599999999999996,"5min":1.149,"15min":1.149},"last":0.72199999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65600000000000003,"15min":0.63100000000000001},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.92200000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91500000000000004,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.81100000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77000000000000002,"15min":0.78500000000000003},"min":{"1min":0.46200000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96199999999999997,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.873},{"interface":"front","average":{"1min":0.753,"5min":0.75600000000000001,"15min":0.78400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.84199999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76900000000000002,"15min":0.79300000000000004},"min":{"1min":0.502,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80600000000000005,"15min":0.82299999999999995},"min":{"1min":0.60299999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.95399999999999996,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.93999999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60999999999999999,"15min":0.63},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.77100000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.48699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.89000000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.89900000000000002,"5min":1.121,"15min":1.121},"last":1.032},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.745,"15min":0.76500000000000001},"min":{"1min":0.52300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83799999999999997,"5min":1.026,"15min":1.026},"last":0.95799999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.49299999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.86599999999999999,"5min":1.214,"15min":1.214},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.50800000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.89800000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.98399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":449816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.78500000000000003,"15min":0.90200000000000002},"min":{"1min":0.443,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.2450000000000001,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.751},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.94599999999999995,"5min":1.47,"15min":1.47},"last":0.63900000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.78700000000000003,"15min":0.91500000000000004},"min":{"1min":0.51500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0249999999999999,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.83999999999999997,"15min":0.98799999999999999},"min":{"1min":0.55600000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.018,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.72299999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.83299999999999996,"15min":0.97299999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1499999999999999,"5min":4.375,"15min":4.375},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.83299999999999996,"15min":0.97699999999999998},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93500000000000005,"5min":4.2960000000000003,"15min":4.2960000000000003},"last":0.73199999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.85499999999999998,"15min":0.99399999999999999},"min":{"1min":0.57799999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89400000000000002,"5min":4.3399999999999999,"15min":4.3399999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.82899999999999996,"15min":0.98499999999999999},"min":{"1min":0.50800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98899999999999999,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.81000000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72599999999999998,"15min":0.75800000000000001},"min":{"1min":0.5,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94499999999999995,"5min":1.036,"15min":1.036},"last":0.41799999999999998}]},{"osd":5,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.65200000000000002,"15min":0.68799999999999994},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92100000000000004,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.69799999999999995,"15min":0.70899999999999996},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.51900000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.52500000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.99299999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71199999999999997,"15min":0.73299999999999998},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88,"5min":1.036,"15min":1.036},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1007616,"data_stored":454906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64500000000000002,"15min":0.66500000000000004},"min":{"1min":0.46600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.78300000000000003,"5min":1.095,"15min":1.095},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71299999999999997,"15min":0.78500000000000003},"min":{"1min":0.49399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92400000000000004,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.627}]},{"osd":1,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71699999999999997,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91800000000000004,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.72699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.47599999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84799999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68300000000000005,"15min":0.71499999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.90700000000000003,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.68400000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73899999999999999,"15min":0.79400000000000004},"min":{"1min":0.46000000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.92400000000000004,"5min":1.7,"15min":1.7},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72999999999999998,"15min":0.76000000000000001},"min":{"1min":0.51900000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91700000000000004,"5min":1.181,"15min":1.181},"last":0.65000000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59899999999999998,"15min":0.63200000000000001},"min":{"1min":0.34999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.82499999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73399999999999999,"15min":0.77500000000000002},"min":{"1min":0.45400000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":1.615,"15min":1.615},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65300000000000002,"15min":0.67600000000000005},"min":{"1min":0.49299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88200000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67800000000000005,"15min":0.71499999999999997},"min":{"1min":0.42899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.872,"5min":1.569,"15min":1.569},"last":0.57199999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72399999999999998,"15min":0.751},"min":{"1min":0.49299999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.95599999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96599999999999997,"5min":1.383,"15min":1.383},"last":0.73199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":1024000,"data_stored":456540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.49399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98699999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.48899999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.876,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.752}]},{"osd":1,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.54100000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93999999999999995,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0169999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73299999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.629,"5min":0.495,"15min":0.495},"max":{"1min":0.93500000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.626},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.502,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97899999999999998,"5min":1.228,"15min":1.228},"last":0.67000000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71399999999999997,"15min":0.70999999999999996},"min":{"1min":0.54300000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.996,"5min":1.27,"15min":1.27},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.78500000000000003},"min":{"1min":0.59199999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.99299999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65900000000000003,"15min":0.69499999999999995},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92100000000000004,"5min":1.329,"15min":1.329},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69899999999999995,"15min":0.751},"min":{"1min":0.379,"5min":0.373,"15min":0.373},"max":{"1min":0.90000000000000002,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.69599999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65900000000000003,"15min":0.71099999999999997},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84099999999999997,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71199999999999997,"15min":0.76200000000000001},"min":{"1min":0.46300000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.42899999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70399999999999996,"15min":0.748},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78600000000000003,"15min":0.83199999999999996},"min":{"1min":0.53000000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98899999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.46600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":448834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.69699999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50800000000000001,"15min":0.54500000000000004},"min":{"1min":0.29999999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51600000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.503,"15min":0.51700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.63500000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.439},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.56499999999999995},"min":{"1min":0.34200000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67500000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.55600000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72199999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53300000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.753,"15min":0.77300000000000002},"min":{"1min":0.53600000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.101,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76700000000000002,"15min":0.78400000000000003},"min":{"1min":0.495,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0760000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.026}]},{"osd":5,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.56499999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0620000000000001,"5min":1.125,"15min":1.125},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78300000000000003,"15min":0.81299999999999994},"min":{"1min":0.58999999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0629999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.878}]},{"osd":6,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79700000000000004,"15min":0.82199999999999995},"min":{"1min":0.57399999999999995,"5min":0.499,"15min":0.499},"max":{"1min":1.157,"5min":1.226,"15min":1.226},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":1.004}]},{"osd":7,"last update":"Sat Mar 30 00:43:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.82799999999999996,"15min":0.86099999999999999},"min":{"1min":0.61799999999999999,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1799999999999999,"5min":1.202,"15min":1.202},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.85899999999999999,"15min":0.90000000000000002},"min":{"1min":0.55700000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.2010000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.86799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":448994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.46300000000000002,"15min":0.441},"min":{"1min":0.31,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.64900000000000002,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66600000000000004,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.42399999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76900000000000002,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68000000000000005,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.38600000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69599999999999995,"5min":0.752,"15min":0.752},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.437,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84999999999999998,"5min":1.907,"15min":1.907},"last":0.441}]},{"osd":4,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.60299999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.55,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.82299999999999995,"15min":0.85399999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.292,"5min":1.972,"15min":1.972},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.57899999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2250000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5820000000000001,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.74399999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.82499999999999996,"15min":0.82599999999999996},"min":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.645,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.56399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6140000000000001,"5min":1.867,"15min":1.867},"last":0.76600000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84499999999999997,"15min":0.877},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2649999999999999,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.63400000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.278,"5min":2.851,"15min":2.851},"last":0.82499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":448364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42199999999999999,"15min":0.42799999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56000000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66500000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51800000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44700000000000001,"15min":0.46500000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.496,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69399999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66900000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.499,"15min":0.47399999999999998},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.63700000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.53700000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88900000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95099999999999996,"5min":1.2,"15min":1.2},"last":0.85899999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.96799999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76700000000000002,"15min":0.76300000000000001},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97899999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77500000000000002,"15min":0.78000000000000003},"min":{"1min":0.57799999999999996,"5min":0.503,"15min":0.503},"max":{"1min":0.89100000000000001,"5min":1.097,"15min":1.097},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":0.77200000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0209999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.79000000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:43:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.79900000000000004},"min":{"1min":0.60199999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0529999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.877},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79500000000000004,"15min":0.78000000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98099999999999998,"5min":1.236,"15min":1.236},"last":0.89500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1585152,"data_stored":1038620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45500000000000002,"15min":0.47699999999999998},"min":{"1min":0.307,"5min":0.245,"15min":0.245},"max":{"1min":0.69699999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.34499999999999997,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76700000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40699999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.503,"15min":0.52100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.40999999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83099999999999996,"5min":1.006,"15min":1.006},"last":0.45200000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.502,"15min":0.51300000000000001},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71599999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56499999999999995,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73899999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42999999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.80500000000000005,"15min":0.79900000000000004},"min":{"1min":0.55700000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.137,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81599999999999995,"15min":0.80600000000000005},"min":{"1min":0.57199999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.252,"15min":1.252},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.58799999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1919999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.78000000000000003,"15min":0.77800000000000002},"min":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.085,"5min":1.462,"15min":1.462},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80100000000000005,"15min":0.80200000000000005},"min":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1060000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1659999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:43:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81499999999999995,"15min":0.81299999999999994},"min":{"1min":0.61199999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0409999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.80400000000000005,"15min":0.80100000000000005},"min":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2070000000000001,"5min":1.556,"15min":1.556},"last":0.622}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:44:20.784 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T00:44:20.785 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-30T00:44:20.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.963+0000 7fe94f800640 1 Processor -- start 2024-03-30T00:44:20.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.963+0000 7fe94f800640 1 -- start start 2024-03-30T00:44:20.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.963+0000 7fe94f800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe94815ca30 0x7fe9480b2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.963+0000 7fe94f800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe9480b0bc0 0x7fe9480caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.963+0000 7fe94f800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9480cf200 con 0x7fe9480ce6d0 2024-03-30T00:44:20.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.963+0000 7fe94f800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94815ce10 con 0x7fe9480b0bc0 2024-03-30T00:44:20.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.963+0000 7fe94f800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9480cec40 con 0x7fe94815ca30 2024-03-30T00:44:20.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.963+0000 7fe94d800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe9480b0bc0 0x7fe9480caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe947e00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe9480ce6d0 0x7fe9480b04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:59268/0 (socket says 172.21.15.66:59268) 2024-03-30T00:44:20.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe947e00640 1 -- 172.21.15.66:0/3637862790 learned_addr learned my addr 172.21.15.66:0/3637862790 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:20.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe94d800640 1 --2- 172.21.15.66:0/3637862790 >> v2:172.21.15.156:3300/0 conn(0x7fe9480b0bc0 0x7fe9480caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:42352/0 (socket says 172.21.15.66:42352) 2024-03-30T00:44:20.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe94ce00640 1 --2- 172.21.15.66:0/3637862790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe94815ca30 0x7fe9480b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe947400640 1 -- 172.21.15.66:0/3637862790 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738446821 0 0) 0x7fe9480cf200 con 0x7fe9480ce6d0 2024-03-30T00:44:20.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe947400640 1 -- 172.21.15.66:0/3637862790 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe930003530 con 0x7fe9480ce6d0 2024-03-30T00:44:20.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe94d800640 1 -- 172.21.15.66:0/3637862790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe94815ca30 msgr2=0x7fe9480b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:20.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe94d800640 1 --2- 172.21.15.66:0/3637862790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe94815ca30 0x7fe9480b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe94d800640 1 -- 172.21.15.66:0/3637862790 >> v1:172.21.15.156:6789/0 conn(0x7fe9480ce6d0 legacy=0x7fe9480b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe94d800640 1 -- 172.21.15.66:0/3637862790 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9480cef20 con 0x7fe9480b0bc0 2024-03-30T00:44:20.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe94ce00640 1 --2- 172.21.15.66:0/3637862790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe94815ca30 0x7fe9480b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:20.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe94d800640 1 --2- 172.21.15.66:0/3637862790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9480b0bc0 0x7fe9480caaa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe940009d90 tx=0x7fe94001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d476407ded55ec6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:20.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe947400640 1 -- 172.21.15.66:0/3637862790 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9400a5a00 con 0x7fe9480b0bc0 2024-03-30T00:44:20.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe947400640 1 -- 172.21.15.66:0/3637862790 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9400a5b60 con 0x7fe9480b0bc0 2024-03-30T00:44:20.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.967+0000 7fe947400640 1 -- 172.21.15.66:0/3637862790 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9400ad440 con 0x7fe9480b0bc0 2024-03-30T00:44:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 -- 172.21.15.66:0/3637862790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9480b0bc0 msgr2=0x7fe9480caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 --2- 172.21.15.66:0/3637862790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9480b0bc0 0x7fe9480caaa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe940009d90 tx=0x7fe94001c940 comp rx=0 tx=0).stop 2024-03-30T00:44:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 -- 172.21.15.66:0/3637862790 shutdown_connections 2024-03-30T00:44:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 --2- 172.21.15.66:0/3637862790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9480b0bc0 0x7fe9480caaa0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 --2- 172.21.15.66:0/3637862790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe94815ca30 0x7fe9480b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 -- 172.21.15.66:0/3637862790 >> 172.21.15.66:0/3637862790 conn(0x7fe948098a20 msgr2=0x7fe9480b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:20.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 -- 172.21.15.66:0/3637862790 shutdown_connections 2024-03-30T00:44:20.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 -- 172.21.15.66:0/3637862790 wait complete. 2024-03-30T00:44:20.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 Processor -- start 2024-03-30T00:44:20.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 -- start start 2024-03-30T00:44:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9480b0bc0 0x7fe94814bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9480ce6d0 0x7fe94814c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe94815ca30 0x7fe9480b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94814ca40 con 0x7fe9480b0bc0 2024-03-30T00:44:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9481539f0 con 0x7fe94815ca30 2024-03-30T00:44:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94f800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe948153cd0 con 0x7fe9480ce6d0 2024-03-30T00:44:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9480b0bc0 0x7fe94814bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9480b0bc0 0x7fe94814bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53256/0 (socket says 172.21.15.66:53256) 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94ce00640 1 -- 172.21.15.66:0/2241534588 learned_addr learned my addr 172.21.15.66:0/2241534588 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94d800640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe94815ca30 0x7fe9480b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94ce00640 1 -- 172.21.15.66:0/2241534588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9480ce6d0 msgr2=0x7fe94814c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe947e00640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9480ce6d0 0x7fe94814c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94ce00640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9480ce6d0 0x7fe94814c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94ce00640 1 -- 172.21.15.66:0/2241534588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe94815ca30 msgr2=0x7fe9480b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94ce00640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe94815ca30 0x7fe9480b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94ce00640 1 -- 172.21.15.66:0/2241534588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c009420 con 0x7fe9480b0bc0 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe94d800640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe94815ca30 0x7fe9480b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.971+0000 7fe947e00640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9480ce6d0 0x7fe94814c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe94ce00640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9480b0bc0 0x7fe94814bfc0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00e980 tx=0x7fe93c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe944c00640 1 -- 172.21.15.66:0/2241534588 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c00ca00 con 0x7fe9480b0bc0 2024-03-30T00:44:20.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe944c00640 1 -- 172.21.15.66:0/2241534588 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c00cb60 con 0x7fe9480b0bc0 2024-03-30T00:44:20.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe944c00640 1 -- 172.21.15.66:0/2241534588 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c013470 con 0x7fe9480b0bc0 2024-03-30T00:44:20.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe94f800640 1 -- 172.21.15.66:0/2241534588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe940009a70 con 0x7fe9480b0bc0 2024-03-30T00:44:20.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe94f800640 1 -- 172.21.15.66:0/2241534588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94814f080 con 0x7fe9480b0bc0 2024-03-30T00:44:20.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe94f800640 1 -- 172.21.15.66:0/2241534588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe948066c30 con 0x7fe9480b0bc0 2024-03-30T00:44:20.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe944c00640 1 -- 172.21.15.66:0/2241534588 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe93c0135d0 con 0x7fe9480b0bc0 2024-03-30T00:44:20.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe944c00640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe928066690 0x7fe928068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:20.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe944c00640 1 -- 172.21.15.66:0/2241534588 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe93c0846c0 con 0x7fe9480b0bc0 2024-03-30T00:44:20.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe947e00640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe928066690 0x7fe928068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:20.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.975+0000 7fe947e00640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe928066690 0x7fe928068b50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe9481514a0 tx=0x7fe934009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:20.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:20.979+0000 7fe944c00640 1 -- 172.21.15.66:0/2241534588 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe93c0517c0 con 0x7fe9480b0bc0 2024-03-30T00:44:21.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.203+0000 7fe94f800640 1 -- 172.21.15.66:0/2241534588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe948158070 con 0x7fe9480b0bc0 2024-03-30T00:44:21.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.207+0000 7fe944c00640 1 -- 172.21.15.66:0/2241534588 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe93c054de0 con 0x7fe9480b0bc0 2024-03-30T00:44:21.214 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-03-30T00:44:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 -- 172.21.15.66:0/2241534588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe928066690 msgr2=0x7fe928068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe928066690 0x7fe928068b50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe9481514a0 tx=0x7fe934009040 comp rx=0 tx=0).stop 2024-03-30T00:44:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 -- 172.21.15.66:0/2241534588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9480b0bc0 msgr2=0x7fe94814bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9480b0bc0 0x7fe94814bfc0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00e980 tx=0x7fe93c00ee40 comp rx=0 tx=0).stop 2024-03-30T00:44:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 -- 172.21.15.66:0/2241534588 shutdown_connections 2024-03-30T00:44:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe94815ca30 0x7fe9480b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9480ce6d0 0x7fe94814c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe928066690 0x7fe928068b50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 --2- 172.21.15.66:0/2241534588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9480b0bc0 0x7fe94814bfc0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 -- 172.21.15.66:0/2241534588 >> 172.21.15.66:0/2241534588 conn(0x7fe948098a20 msgr2=0x7fe948154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:21.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 -- 172.21.15.66:0/2241534588 shutdown_connections 2024-03-30T00:44:21.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.211+0000 7fe94f800640 1 -- 172.21.15.66:0/2241534588 wait complete. 2024-03-30T00:44:21.236 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:44:21.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.367+0000 7f131d200640 1 Processor -- start 2024-03-30T00:44:21.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.367+0000 7f131d200640 1 -- start start 2024-03-30T00:44:21.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.367+0000 7f131d200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f13180cf2f0 0x7f131806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:21.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.367+0000 7f131d200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f13180cb570 0x7f13180cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:21.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.367+0000 7f131d200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13180cb0e0 con 0x7f131815ca80 2024-03-30T00:44:21.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.367+0000 7f131d200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13180cf7f0 con 0x7f13180cb570 2024-03-30T00:44:21.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.367+0000 7f131d200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13180cfad0 con 0x7f13180cf2f0 2024-03-30T00:44:21.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.367+0000 7f1315e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f131815ca80 0x7f131806b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55086/0 (socket says 172.21.15.66:55086) 2024-03-30T00:44:21.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.367+0000 7f1315e00640 1 -- 172.21.15.66:0/3475170685 learned_addr learned my addr 172.21.15.66:0/3475170685 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:21.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.367+0000 7f1316800640 1 --2- 172.21.15.66:0/3475170685 >> v2:172.21.15.66:3300/0 conn(0x7f13180cb570 0x7f13180cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:21.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f130d400640 1 --2- 172.21.15.66:0/3475170685 >> v2:172.21.15.156:3300/0 conn(0x7f13180cf2f0 0x7f131806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:21.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f1315400640 1 -- 172.21.15.66:0/3475170685 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052863166 0 0) 0x7f13180cb0e0 con 0x7f131815ca80 2024-03-30T00:44:21.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f1316800640 1 -- 172.21.15.66:0/3475170685 >> v2:172.21.15.156:3300/0 conn(0x7f13180cf2f0 msgr2=0x7f131806b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:21.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f1316800640 1 --2- 172.21.15.66:0/3475170685 >> v2:172.21.15.156:3300/0 conn(0x7f13180cf2f0 0x7f131806b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f1316800640 1 -- 172.21.15.66:0/3475170685 >> v1:172.21.15.66:6789/0 conn(0x7f131815ca80 legacy=0x7f131806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:21.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f1316800640 1 -- 172.21.15.66:0/3475170685 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13180cad70 con 0x7f13180cb570 2024-03-30T00:44:21.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f1316800640 1 --2- 172.21.15.66:0/3475170685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13180cb570 0x7f13180cba20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f131806ced0 tx=0x7f131001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b7e05b7cf77105 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:21.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f1315400640 1 -- 172.21.15.66:0/3475170685 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100a8970 con 0x7f13180cb570 2024-03-30T00:44:21.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f1315400640 1 -- 172.21.15.66:0/3475170685 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13100a8ad0 con 0x7f13180cb570 2024-03-30T00:44:21.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f1315400640 1 -- 172.21.15.66:0/3475170685 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100a8e60 con 0x7f13180cb570 2024-03-30T00:44:21.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f131d200640 1 -- 172.21.15.66:0/3475170685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13180cb570 msgr2=0x7f13180cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:21.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f131d200640 1 --2- 172.21.15.66:0/3475170685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13180cb570 0x7f13180cba20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f131806ced0 tx=0x7f131001ec70 comp rx=0 tx=0).stop 2024-03-30T00:44:21.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f131d200640 1 -- 172.21.15.66:0/3475170685 shutdown_connections 2024-03-30T00:44:21.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f131d200640 1 --2- 172.21.15.66:0/3475170685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13180cb570 0x7f13180cba20 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f131d200640 1 --2- 172.21.15.66:0/3475170685 >> v2:172.21.15.156:3300/0 conn(0x7f13180cf2f0 0x7f131806b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.371+0000 7f131d200640 1 -- 172.21.15.66:0/3475170685 >> 172.21.15.66:0/3475170685 conn(0x7f13180b2090 msgr2=0x7f13180b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:21.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f131d200640 1 -- 172.21.15.66:0/3475170685 shutdown_connections 2024-03-30T00:44:21.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f131d200640 1 -- 172.21.15.66:0/3475170685 wait complete. 2024-03-30T00:44:21.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f131d200640 1 Processor -- start 2024-03-30T00:44:21.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f131d200640 1 -- start start 2024-03-30T00:44:21.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f131d200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13180cb570 0x7f13180b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:21.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f131d200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13180cf2f0 0x7f13181f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:21.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f131d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f131815ca80 0x7f13180c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:21.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f131d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13181f5ca0 con 0x7f131815ca80 2024-03-30T00:44:21.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f131d200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13181f61d0 con 0x7f13180cf2f0 2024-03-30T00:44:21.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f131d200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13180b32f0 con 0x7f13180cb570 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f1316800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f131815ca80 0x7f13180c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f1316800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f131815ca80 0x7f13180c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53274/0 (socket says 172.21.15.66:53274) 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f1316800640 1 -- 172.21.15.66:0/1692387829 learned_addr learned my addr 172.21.15.66:0/1692387829 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f1315e00640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13180cb570 0x7f13180b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f130d400640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13180cf2f0 0x7f13181f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f1316800640 1 -- 172.21.15.66:0/1692387829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13180cb570 msgr2=0x7f13180b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f1316800640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13180cb570 0x7f13180b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f1316800640 1 -- 172.21.15.66:0/1692387829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13180cf2f0 msgr2=0x7f13181f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f1316800640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13180cf2f0 0x7f13181f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f1316800640 1 -- 172.21.15.66:0/1692387829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1310009420 con 0x7f131815ca80 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f1316800640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f131815ca80 0x7f13180c78d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f13100b3040 tx=0x7f131001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:21.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f130ea00640 1 -- 172.21.15.66:0/1692387829 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100b0e20 con 0x7f131815ca80 2024-03-30T00:44:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f130ea00640 1 -- 172.21.15.66:0/1692387829 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13100b9410 con 0x7f131815ca80 2024-03-30T00:44:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f131d200640 1 -- 172.21.15.66:0/1692387829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13181f5ef0 con 0x7f131815ca80 2024-03-30T00:44:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.375+0000 7f130ea00640 1 -- 172.21.15.66:0/1692387829 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100b97c0 con 0x7f131815ca80 2024-03-30T00:44:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.379+0000 7f131d200640 1 -- 172.21.15.66:0/1692387829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13180b3870 con 0x7f131815ca80 2024-03-30T00:44:21.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.379+0000 7f131d200640 1 -- 172.21.15.66:0/1692387829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13180b3a60 con 0x7f131815ca80 2024-03-30T00:44:21.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.379+0000 7f130ea00640 1 -- 172.21.15.66:0/1692387829 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f13100b7040 con 0x7f131815ca80 2024-03-30T00:44:21.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.379+0000 7f130ea00640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12f4066690 0x7f12f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:21.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.379+0000 7f130ea00640 1 -- 172.21.15.66:0/1692387829 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1310121e20 con 0x7f131815ca80 2024-03-30T00:44:21.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.379+0000 7f1315e00640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12f4066690 0x7f12f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:21.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.379+0000 7f1315e00640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12f4066690 0x7f12f4068b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1308000f70 tx=0x7f1308007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:21.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.383+0000 7f130ea00640 1 -- 172.21.15.66:0/1692387829 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f13100ef0f0 con 0x7f131815ca80 2024-03-30T00:44:21.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.579+0000 7f131d200640 1 -- 172.21.15.66:0/1692387829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1318066c30 con 0x7f131815ca80 2024-03-30T00:44:21.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.579+0000 7f130ea00640 1 -- 172.21.15.66:0/1692387829 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4742 (secure 0 0 0) 0x7f13100b5090 con 0x7f131815ca80 2024-03-30T00:44:21.588 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:21.588 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:44:19.285136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/2089872680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":2089872680},{"type":"v1","addr":"172.21.15.156:6835","nonce":2089872680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:44:21.588 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 43 2024-03-30T00:44:21.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 -- 172.21.15.66:0/1692387829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12f4066690 msgr2=0x7f12f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:21.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12f4066690 0x7f12f4068b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1308000f70 tx=0x7f1308007df0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 -- 172.21.15.66:0/1692387829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f131815ca80 msgr2=0x7f13180c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:21.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f131815ca80 0x7f13180c78d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f13100b3040 tx=0x7f131001ba10 comp rx=0 tx=0).stop 2024-03-30T00:44:21.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 -- 172.21.15.66:0/1692387829 shutdown_connections 2024-03-30T00:44:21.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12f4066690 0x7f12f4068b50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f131815ca80 0x7f13180c78d0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13180cf2f0 0x7f13181f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 --2- 172.21.15.66:0/1692387829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13180cb570 0x7f13180b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:21.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 -- 172.21.15.66:0/1692387829 >> 172.21.15.66:0/1692387829 conn(0x7f13180b2090 msgr2=0x7f13180ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:21.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 -- 172.21.15.66:0/1692387829 shutdown_connections 2024-03-30T00:44:21.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:21.583+0000 7f131d200640 1 -- 172.21.15.66:0/1692387829 wait complete. 2024-03-30T00:44:22.011 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:44:16.274202+0000', 'modified': '2024-03-30T00:44:19.285136+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7002}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7002': {'gid': 7002, 'name': 'b', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6835/2089872680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6833', 'nonce': 2089872680}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 2089872680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:44:22.011 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:44:22.012 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-30T00:44:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe697400640 1 Processor -- start 2024-03-30T00:44:22.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe697400640 1 -- start start 2024-03-30T00:44:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe697400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6900cabb0 0x7fe6900cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe697400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69006d070 con 0x7fe6900ce520 2024-03-30T00:44:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe697400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6900cbba0 con 0x7fe69015a640 2024-03-30T00:44:22.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe697400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6900cb8c0 con 0x7fe6900cabb0 2024-03-30T00:44:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe687e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe6900ce520 0x7fe6900ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55102/0 (socket says 172.21.15.66:55102) 2024-03-30T00:44:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe695400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6900cabb0 0x7fe6900cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe687e00640 1 -- 172.21.15.66:0/323057420 learned_addr learned my addr 172.21.15.66:0/323057420 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe695400640 1 -- 172.21.15.66:0/323057420 >> v1:172.21.15.156:6789/0 conn(0x7fe69015a640 legacy=0x7fe69006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe695400640 1 -- 172.21.15.66:0/323057420 >> v1:172.21.15.66:6789/0 conn(0x7fe6900ce520 legacy=0x7fe6900ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe695400640 1 -- 172.21.15.66:0/323057420 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69006d2a0 con 0x7fe6900cabb0 2024-03-30T00:44:22.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.147+0000 7fe68fe00640 1 -- 172.21.15.66:0/323057420 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963130161 0 0) 0x7fe69006d070 con 0x7fe6900ce520 2024-03-30T00:44:22.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.151+0000 7fe695400640 1 --2- 172.21.15.66:0/323057420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6900cabb0 0x7fe6900cafe0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe688009d90 tx=0x7fe68801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727b5b6b37722d40 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:22.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.151+0000 7fe68fe00640 1 -- 172.21.15.66:0/323057420 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880a5a00 con 0x7fe6900cabb0 2024-03-30T00:44:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.151+0000 7fe68fe00640 1 -- 172.21.15.66:0/323057420 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6880a5b60 con 0x7fe6900cabb0 2024-03-30T00:44:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.151+0000 7fe68fe00640 1 -- 172.21.15.66:0/323057420 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880ad440 con 0x7fe6900cabb0 2024-03-30T00:44:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.151+0000 7fe697400640 1 -- 172.21.15.66:0/323057420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6900cabb0 msgr2=0x7fe6900cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.151+0000 7fe697400640 1 --2- 172.21.15.66:0/323057420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6900cabb0 0x7fe6900cafe0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe688009d90 tx=0x7fe68801c940 comp rx=0 tx=0).stop 2024-03-30T00:44:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.151+0000 7fe697400640 1 -- 172.21.15.66:0/323057420 shutdown_connections 2024-03-30T00:44:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.151+0000 7fe697400640 1 --2- 172.21.15.66:0/323057420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6900cabb0 0x7fe6900cafe0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.151+0000 7fe697400640 1 -- 172.21.15.66:0/323057420 >> 172.21.15.66:0/323057420 conn(0x7fe690069f60 msgr2=0x7fe69006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.151+0000 7fe697400640 1 -- 172.21.15.66:0/323057420 shutdown_connections 2024-03-30T00:44:22.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.151+0000 7fe697400640 1 -- 172.21.15.66:0/323057420 wait complete. 2024-03-30T00:44:22.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe697400640 1 Processor -- start 2024-03-30T00:44:22.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe697400640 1 -- start start 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe697400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6900cabb0 0x7fe690151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe697400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6900ce520 0x7fe6900adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe697400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69015a640 0x7fe690071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe697400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe690096280 con 0x7fe69015a640 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe697400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900c1040 con 0x7fe6900cabb0 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe697400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe690096490 con 0x7fe6900ce520 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe695400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69015a640 0x7fe690071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe695400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69015a640 0x7fe690071520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53292/0 (socket says 172.21.15.66:53292) 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe695400640 1 -- 172.21.15.66:0/270901352 learned_addr learned my addr 172.21.15.66:0/270901352 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe694a00640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6900cabb0 0x7fe690151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe687e00640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6900ce520 0x7fe6900adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe695400640 1 -- 172.21.15.66:0/270901352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6900ce520 msgr2=0x7fe6900adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe695400640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6900ce520 0x7fe6900adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe695400640 1 -- 172.21.15.66:0/270901352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6900cabb0 msgr2=0x7fe690151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe695400640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6900cabb0 0x7fe690151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe695400640 1 -- 172.21.15.66:0/270901352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe688009a70 con 0x7fe69015a640 2024-03-30T00:44:22.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe695400640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69015a640 0x7fe690071520 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe68801a570 tx=0x7fe6880196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:22.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe68d600640 1 -- 172.21.15.66:0/270901352 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880a5a00 con 0x7fe69015a640 2024-03-30T00:44:22.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe68d600640 1 -- 172.21.15.66:0/270901352 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6880039a0 con 0x7fe69015a640 2024-03-30T00:44:22.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe68d600640 1 -- 172.21.15.66:0/270901352 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688003d10 con 0x7fe69015a640 2024-03-30T00:44:22.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe697400640 1 -- 172.21.15.66:0/270901352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe690096770 con 0x7fe69015a640 2024-03-30T00:44:22.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.155+0000 7fe697400640 1 -- 172.21.15.66:0/270901352 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe690096d30 con 0x7fe69015a640 2024-03-30T00:44:22.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe68d600640 1 -- 172.21.15.66:0/270901352 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe6880b4050 con 0x7fe69015a640 2024-03-30T00:44:22.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe68d600640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe66c066690 0x7fe66c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe68d600640 1 -- 172.21.15.66:0/270901352 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe68811f6b0 con 0x7fe69015a640 2024-03-30T00:44:22.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe694a00640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe66c066690 0x7fe66c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe69a899000 1 Processor -- start 2024-03-30T00:44:22.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe69a899000 1 -- start start 2024-03-30T00:44:22.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe694a00640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe66c066690 0x7fe66c068b50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe6900afd40 tx=0x7fe680009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:22.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe69a899000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde93d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe69a899000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde93db30 0x558cde8f3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe69a899000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde8f38d0 0x558cde8f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe69a899000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cde7b75a0 con 0x558cde93d210 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe69a899000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cde744af0 con 0x558cde93db30 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe69a899000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cde7420c0 con 0x558cde8f38d0 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe694a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde93d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe694a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde93d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53306/0 (socket says 172.21.15.66:53306) 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe694a00640 1 -- 172.21.15.66:0/2596873679 learned_addr learned my addr 172.21.15.66:0/2596873679 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe695400640 1 --2- 172.21.15.66:0/2596873679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde8f38d0 0x558cde8f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.159+0000 7fe687e00640 1 --2- 172.21.15.66:0/2596873679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde93db30 0x558cde8f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe694a00640 1 -- 172.21.15.66:0/2596873679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde8f38d0 msgr2=0x558cde8f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe694a00640 1 --2- 172.21.15.66:0/2596873679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde8f38d0 0x558cde8f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe694a00640 1 -- 172.21.15.66:0/2596873679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde93db30 msgr2=0x558cde8f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:22.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe694a00640 1 --2- 172.21.15.66:0/2596873679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde93db30 0x558cde8f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe694a00640 1 -- 172.21.15.66:0/2596873679 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cde439890 con 0x558cde93d210 2024-03-30T00:44:22.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe694a00640 1 --2- 172.21.15.66:0/2596873679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde93d5f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe6800052f0 tx=0x7fe680011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc27bc4738f1f98 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:22.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe685600640 1 -- 172.21.15.66:0/2596873679 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680005460 con 0x558cde93d210 2024-03-30T00:44:22.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe685600640 1 -- 172.21.15.66:0/2596873679 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68000e6b0 con 0x558cde93d210 2024-03-30T00:44:22.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe685600640 1 -- 172.21.15.66:0/2596873679 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68000ea20 con 0x558cde93d210 2024-03-30T00:44:22.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 -- 172.21.15.66:0/2596873679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 msgr2=0x558cde93d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 --2- 172.21.15.66:0/2596873679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde93d5f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe6800052f0 tx=0x7fe680011000 comp rx=0 tx=0).stop 2024-03-30T00:44:22.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 -- 172.21.15.66:0/2596873679 shutdown_connections 2024-03-30T00:44:22.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 --2- 172.21.15.66:0/2596873679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde8f38d0 0x558cde8f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 --2- 172.21.15.66:0/2596873679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde93db30 0x558cde8f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 --2- 172.21.15.66:0/2596873679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde93d5f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 -- 172.21.15.66:0/2596873679 >> 172.21.15.66:0/2596873679 conn(0x558cde8f5c00 msgr2=0x558cde8f1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:22.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 -- 172.21.15.66:0/2596873679 shutdown_connections 2024-03-30T00:44:22.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 -- 172.21.15.66:0/2596873679 wait complete. 2024-03-30T00:44:22.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 -- start start 2024-03-30T00:44:22.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 Processor -- start 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 0x558cde929340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde93d210 0x558cde93df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde93e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cde73d9b0 con 0x558cde93db30 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cde6dcc00 con 0x558cde8f38d0 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cde6fb070 con 0x558cde93d210 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe69a899000 10 client.0 _ll_register_callbacks cb 0x558cde8db0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe695400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde93e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe695400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde93e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53322/0 (socket says 172.21.15.66:53322) 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe695400640 1 -- 172.21.15.66:0/830999642 learned_addr learned my addr 172.21.15.66:0/830999642 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe687e00640 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde93d210 0x558cde93df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe694a00640 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 0x558cde929340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe695400640 1 -- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde93d210 msgr2=0x558cde93df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe695400640 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde93d210 0x558cde93df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe695400640 1 -- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 msgr2=0x558cde929340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe695400640 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 0x558cde929340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe695400640 1 -- 172.21.15.66:0/830999642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68000f040 con 0x558cde93db30 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe687e00640 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde93d210 0x558cde93df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.163+0000 7fe694a00640 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 0x558cde929340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe695400640 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde93e450 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fe688019550 tx=0x7fe6880f0830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e951a5b63983b09 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe67e000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe67e000640 1 -- 172.21.15.66:0/830999642 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880f4360 con 0x558cde93db30 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe67e000640 1 -- 172.21.15.66:0/830999642 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe6880f09d0 con 0x558cde93db30 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 -- 172.21.15.66:0/830999642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cde6fa5f0 con 0x558cde93db30 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe67e000640 1 -- 172.21.15.66:0/830999642 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6880f0b90 con 0x558cde93db30 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 10 client.7122 fetch_fsmap learned FSMap version 43 2024-03-30T00:44:22.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 -- 172.21.15.66:0/830999642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x558cde702ab0 con 0x558cde93db30 2024-03-30T00:44:22.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe67e000640 1 -- 172.21.15.66:0/830999642 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558cde6fa5f0 con 0x558cde93db30 2024-03-30T00:44:22.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe67e000640 1 -- 172.21.15.66:0/830999642 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe688005d20 con 0x558cde93db30 2024-03-30T00:44:22.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 10 client.7122 fetch_fsmap finished waiting for FSMap version 43 2024-03-30T00:44:22.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 10 client.7122 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T00:44:22.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 20 client.7122 populate_metadata read hostname 'smithi066' 2024-03-30T00:44:22.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:6833/2089872680,v1:172.21.15.156:6835/2089872680] conn(0x558cde93ee60 0x558cde93f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 4 client.7122 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:44:22.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 -- 172.21.15.66:0/830999642 --> [v2:172.21.15.156:6833/2089872680,v1:172.21.15.156:6835/2089872680] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cde73a790 con 0x558cde93ee60 2024-03-30T00:44:22.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe694a00640 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:6833/2089872680,v1:172.21.15.156:6835/2089872680] conn(0x558cde93ee60 0x558cde93f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe694a00640 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:6833/2089872680,v1:172.21.15.156:6835/2089872680] conn(0x558cde93ee60 0x558cde93f240 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ced11484c7ab6f server_cookie=1c82617ef594cc5 in_seq=0 out_seq=0 2024-03-30T00:44:22.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe67e000640 10 client.7122 ms_handle_connect on v2:172.21.15.156:6833/2089872680 2024-03-30T00:44:22.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe67e000640 1 -- 172.21.15.66:0/830999642 <== mds.0 v2:172.21.15.156:6833/2089872680 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558cde73a790 con 0x558cde93ee60 2024-03-30T00:44:22.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe67e000640 10 client.7122 handle_command_reply: tid=0 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 client.7122 shutdown 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 -- 172.21.15.66:0/830999642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 msgr2=0x558cde93e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde93e450 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fe688019550 tx=0x7fe6880f0830 comp rx=0 tx=0).stop 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 -- 172.21.15.66:0/830999642 shutdown_connections 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:6833/2089872680,v1:172.21.15.156:6835/2089872680] conn(0x558cde93ee60 0x558cde93f240 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde93e450 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde93d210 0x558cde93df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 --2- 172.21.15.66:0/830999642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 0x558cde929340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe67e000640 0 client.7122 ms_handle_reset on v2:172.21.15.156:6833/2089872680 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.167+0000 7fe69a899000 1 -- 172.21.15.66:0/830999642 >> 172.21.15.66:0/830999642 conn(0x558cde8f5c00 msgr2=0x558cde8f4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.171+0000 7fe69a899000 1 -- 172.21.15.66:0/830999642 shutdown_connections 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.171+0000 7fe69a899000 1 -- 172.21.15.66:0/830999642 wait complete. 2024-03-30T00:44:22.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.171+0000 7fe69a899000 20 client.7122 trim_cache size 0 max 16384 2024-03-30T00:44:22.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.183+0000 7fe69a899000 1 Processor -- start 2024-03-30T00:44:22.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.183+0000 7fe69a899000 1 -- start start 2024-03-30T00:44:22.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde8f38d0 0x558cde952830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde93d210 0x558cde94e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde94ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cde3fa940 con 0x558cde93db30 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cde731530 con 0x558cde93d210 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cde73b1c0 con 0x558cde8f38d0 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe695400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde94ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe695400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde94ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53336/0 (socket says 172.21.15.66:53336) 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe695400640 1 -- 172.21.15.66:0/938277196 learned_addr learned my addr 172.21.15.66:0/938277196 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe687e00640 1 --2- 172.21.15.66:0/938277196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde93d210 0x558cde94e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe694a00640 1 --2- 172.21.15.66:0/938277196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde8f38d0 0x558cde952830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe695400640 1 -- 172.21.15.66:0/938277196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde8f38d0 msgr2=0x558cde952830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe695400640 1 --2- 172.21.15.66:0/938277196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde8f38d0 0x558cde952830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe695400640 1 -- 172.21.15.66:0/938277196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde93d210 msgr2=0x558cde94e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe695400640 1 --2- 172.21.15.66:0/938277196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde93d210 0x558cde94e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe695400640 1 -- 172.21.15.66:0/938277196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cde6e8c00 con 0x558cde93db30 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe687e00640 1 --2- 172.21.15.66:0/938277196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde93d210 0x558cde94e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe694a00640 1 --2- 172.21.15.66:0/938277196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde8f38d0 0x558cde952830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe695400640 1 --2- 172.21.15.66:0/938277196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde94ebe0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fe6880ecbb0 tx=0x7fe6880f3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edc0a8885019519 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe686000640 1 -- 172.21.15.66:0/938277196 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880f0e70 con 0x558cde93db30 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe686000640 1 -- 172.21.15.66:0/938277196 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6880efd00 con 0x558cde93db30 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe686000640 1 -- 172.21.15.66:0/938277196 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880f4160 con 0x558cde93db30 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- 172.21.15.66:0/938277196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 msgr2=0x558cde94ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 --2- 172.21.15.66:0/938277196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde94ebe0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fe6880ecbb0 tx=0x7fe6880f3000 comp rx=0 tx=0).stop 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- 172.21.15.66:0/938277196 shutdown_connections 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 --2- 172.21.15.66:0/938277196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93db30 0x558cde94ebe0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 --2- 172.21.15.66:0/938277196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde93d210 0x558cde94e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 --2- 172.21.15.66:0/938277196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde8f38d0 0x558cde952830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- 172.21.15.66:0/938277196 >> 172.21.15.66:0/938277196 conn(0x558cde8f5c00 msgr2=0x558cde8f2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:22.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- 172.21.15.66:0/938277196 shutdown_connections 2024-03-30T00:44:22.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- 172.21.15.66:0/938277196 wait complete. 2024-03-30T00:44:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- start start 2024-03-30T00:44:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 Processor -- start 2024-03-30T00:44:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 0x558cde951cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde99ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde93db30 0x558cde99b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cde780ac0 con 0x558cde93d210 2024-03-30T00:44:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cde9494c0 con 0x558cde8f38d0 2024-03-30T00:44:22.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cde73ef40 con 0x558cde93db30 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe69a899000 10 client.0 _ll_register_callbacks cb 0x558cde702590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe687e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde99ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe694a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 0x558cde951cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe687e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde99ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53348/0 (socket says 172.21.15.66:53348) 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.187+0000 7fe687e00640 1 -- 172.21.15.66:0/1032127236 learned_addr learned my addr 172.21.15.66:0/1032127236 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe695400640 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde93db30 0x558cde99b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe694a00640 1 -- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde93db30 msgr2=0x558cde99b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe694a00640 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde93db30 0x558cde99b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe694a00640 1 -- 172.21.15.66:0/1032127236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 msgr2=0x558cde99ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe694a00640 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde99ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe694a00640 1 -- 172.21.15.66:0/1032127236 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cde702ab0 con 0x558cde8f38d0 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe687e00640 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde99ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe694a00640 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 0x558cde951cb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe680020000 tx=0x7fe680000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1284a325b9d3efa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:22.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe67e000640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe67e000640 1 -- 172.21.15.66:0/1032127236 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680033070 con 0x558cde8f38d0 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 -- 172.21.15.66:0/1032127236 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cde70d460 con 0x558cde8f38d0 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe67e000640 1 -- 172.21.15.66:0/1032127236 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe68002e970 con 0x558cde8f38d0 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe67e000640 1 -- 172.21.15.66:0/1032127236 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe680026070 con 0x558cde8f38d0 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 10 client.6787 fetch_fsmap learned FSMap version 43 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 -- 172.21.15.66:0/1032127236 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x558cde439cc0 con 0x558cde8f38d0 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe67e000640 1 -- 172.21.15.66:0/1032127236 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558cde70d460 con 0x558cde8f38d0 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe67e000640 1 -- 172.21.15.66:0/1032127236 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 43) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe680036020 con 0x558cde8f38d0 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 10 client.6787 fetch_fsmap finished waiting for FSMap version 43 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 10 client.6787 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 20 client.6787 populate_metadata read hostname 'smithi066' 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:6833/2089872680,v1:172.21.15.156:6835/2089872680] conn(0x558cde957520 0x558cde957900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 4 client.6787 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:44:22.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 -- 172.21.15.66:0/1032127236 --> [v2:172.21.15.156:6833/2089872680,v1:172.21.15.156:6835/2089872680] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558cde7069c0 con 0x558cde957520 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe687e00640 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:6833/2089872680,v1:172.21.15.156:6835/2089872680] conn(0x558cde957520 0x558cde957900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe687e00640 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:6833/2089872680,v1:172.21.15.156:6835/2089872680] conn(0x558cde957520 0x558cde957900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ea0ba747a08585 server_cookie=b8c11572b5ccd6f2 in_seq=0 out_seq=0 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe67e000640 10 client.6787 ms_handle_connect on v2:172.21.15.156:6833/2089872680 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe67e000640 1 -- 172.21.15.66:0/1032127236 <== mds.0 v2:172.21.15.156:6833/2089872680 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x558cde7069c0 con 0x558cde957520 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe67e000640 10 client.6787 handle_command_reply: tid=0 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 client.6787 shutdown 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 -- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 msgr2=0x558cde951cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 0x558cde951cb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe680020000 tx=0x7fe680000ee0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 -- 172.21.15.66:0/1032127236 shutdown_connections 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:6833/2089872680,v1:172.21.15.156:6835/2089872680] conn(0x558cde957520 0x558cde957900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558cde93db30 0x558cde99b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558cde93d210 0x558cde99ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 --2- 172.21.15.66:0/1032127236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cde8f38d0 0x558cde951cb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe67e000640 0 client.6787 ms_handle_reset on v2:172.21.15.156:6833/2089872680 2024-03-30T00:44:22.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 -- 172.21.15.66:0/1032127236 >> 172.21.15.66:0/1032127236 conn(0x558cde8f5c00 msgr2=0x558cde8f3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:22.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 -- 172.21.15.66:0/1032127236 shutdown_connections 2024-03-30T00:44:22.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 1 -- 172.21.15.66:0/1032127236 wait complete. 2024-03-30T00:44:22.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.191+0000 7fe69a899000 20 client.6787 trim_cache size 0 max 16384 2024-03-30T00:44:22.204 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:44:22.204 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:44:22.204 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:44:22.204 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 7002, 2024-03-30T00:44:22.204 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:44:22.204 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:44:22.205 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:44:22.205 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 5.9179595110000003, 2024-03-30T00:44:22.205 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 43, 2024-03-30T00:44:22.205 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 82, 2024-03-30T00:44:22.205 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 82, 2024-03-30T00:44:22.205 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 10.355691783999999 2024-03-30T00:44:22.205 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:44:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 -- 172.21.15.66:0/270901352 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe66c066690 msgr2=0x7fe66c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe66c066690 0x7fe66c068b50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe6900afd40 tx=0x7fe680009040 comp rx=0 tx=0).stop 2024-03-30T00:44:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 -- 172.21.15.66:0/270901352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69015a640 msgr2=0x7fe690071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69015a640 0x7fe690071520 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe68801a570 tx=0x7fe6880196e0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 -- 172.21.15.66:0/270901352 shutdown_connections 2024-03-30T00:44:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe66c066690 0x7fe66c068b50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe69015a640 0x7fe690071520 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6900ce520 0x7fe6900adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 --2- 172.21.15.66:0/270901352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6900cabb0 0x7fe690151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 -- 172.21.15.66:0/270901352 >> 172.21.15.66:0/270901352 conn(0x7fe690069f60 msgr2=0x7fe6900afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:22.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 -- 172.21.15.66:0/270901352 shutdown_connections 2024-03-30T00:44:22.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.195+0000 7fe697400640 1 -- 172.21.15.66:0/270901352 wait complete. 2024-03-30T00:44:22.220 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:44:22.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.351+0000 7f2cd8800640 1 Processor -- start 2024-03-30T00:44:22.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.351+0000 7f2cd8800640 1 -- start start 2024-03-30T00:44:22.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.351+0000 7f2cd8800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cd015a810 0x7f2cd00ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.351+0000 7f2cd8800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd00c4600 con 0x7f2cd00cf290 2024-03-30T00:44:22.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.351+0000 7f2cd8800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd00cc510 con 0x7f2cd0099430 2024-03-30T00:44:22.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.355+0000 7f2cd8800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd00998c0 con 0x7f2cd015a810 2024-03-30T00:44:22.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.355+0000 7f2cd6800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2cd00cf290 0x7f2cd00c4a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55106/0 (socket says 172.21.15.66:55106) 2024-03-30T00:44:22.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.355+0000 7f2cd6800640 1 -- 172.21.15.66:0/2546822927 learned_addr learned my addr 172.21.15.66:0/2546822927 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:22.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.355+0000 7f2cd5e00640 1 --2- 172.21.15.66:0/2546822927 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cd015a810 0x7f2cd00ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.355+0000 7f2cd4a00640 1 -- 172.21.15.66:0/2546822927 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391271240 0 0) 0x7f2cd00c4600 con 0x7f2cd00cf290 2024-03-30T00:44:22.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.355+0000 7f2cd5e00640 1 -- 172.21.15.66:0/2546822927 >> v1:172.21.15.156:6789/0 conn(0x7f2cd0099430 legacy=0x7f2cd00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.355+0000 7f2cd5e00640 1 -- 172.21.15.66:0/2546822927 >> v1:172.21.15.66:6789/0 conn(0x7f2cd00cf290 legacy=0x7f2cd00c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.355+0000 7f2cd5e00640 1 -- 172.21.15.66:0/2546822927 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0099ba0 con 0x7f2cd015a810 2024-03-30T00:44:22.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.355+0000 7f2cd5e00640 1 --2- 172.21.15.66:0/2546822927 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cd015a810 0x7f2cd00ce380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009770 tx=0x7f2cc401c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f3b505bc1466b69 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:22.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd4a00640 1 -- 172.21.15.66:0/2546822927 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40ac620 con 0x7f2cd015a810 2024-03-30T00:44:22.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd4a00640 1 -- 172.21.15.66:0/2546822927 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc40ac780 con 0x7f2cd015a810 2024-03-30T00:44:22.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd4a00640 1 -- 172.21.15.66:0/2546822927 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40acb10 con 0x7f2cd015a810 2024-03-30T00:44:22.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 -- 172.21.15.66:0/2546822927 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cd015a810 msgr2=0x7f2cd00ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 --2- 172.21.15.66:0/2546822927 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cd015a810 0x7f2cd00ce380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009770 tx=0x7f2cc401c3a0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 -- 172.21.15.66:0/2546822927 shutdown_connections 2024-03-30T00:44:22.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 --2- 172.21.15.66:0/2546822927 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cd015a810 0x7f2cd00ce380 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 -- 172.21.15.66:0/2546822927 >> 172.21.15.66:0/2546822927 conn(0x7f2cd00b2090 msgr2=0x7f2cd00b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:22.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 -- 172.21.15.66:0/2546822927 shutdown_connections 2024-03-30T00:44:22.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 -- 172.21.15.66:0/2546822927 wait complete. 2024-03-30T00:44:22.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 Processor -- start 2024-03-30T00:44:22.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 -- start start 2024-03-30T00:44:22.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd00cf290 0x7f2cd00c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cd015a810 0x7f2cd0159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cd00c87a0 0x7f2cd015d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd00c8df0 con 0x7f2cd00cf290 2024-03-30T00:44:22.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd00c9970 con 0x7f2cd015a810 2024-03-30T00:44:22.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd8800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd00ca340 con 0x7f2cd00c87a0 2024-03-30T00:44:22.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd00cf290 0x7f2cd00c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd00cf290 0x7f2cd00c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53356/0 (socket says 172.21.15.66:53356) 2024-03-30T00:44:22.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.359+0000 7f2cd5e00640 1 -- 172.21.15.66:0/86533583 learned_addr learned my addr 172.21.15.66:0/86533583 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:22.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cd6800640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cd00c87a0 0x7f2cd015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cd5400640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cd015a810 0x7f2cd0159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cd5e00640 1 -- 172.21.15.66:0/86533583 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cd00c87a0 msgr2=0x7f2cd015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cd5e00640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cd00c87a0 0x7f2cd015d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cd5e00640 1 -- 172.21.15.66:0/86533583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cd015a810 msgr2=0x7f2cd0159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cd5e00640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cd015a810 0x7f2cd0159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cd5e00640 1 -- 172.21.15.66:0/86533583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc4009420 con 0x7f2cd00cf290 2024-03-30T00:44:22.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cd5e00640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd00cf290 0x7f2cd00c9e00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401c880 tx=0x7f2cc4019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:22.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cca000640 1 -- 172.21.15.66:0/86533583 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4004230 con 0x7f2cd00cf290 2024-03-30T00:44:22.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cca000640 1 -- 172.21.15.66:0/86533583 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc4004390 con 0x7f2cd00cf290 2024-03-30T00:44:22.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cca000640 1 -- 172.21.15.66:0/86533583 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40b7890 con 0x7f2cd00cf290 2024-03-30T00:44:22.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cd8800640 1 -- 172.21.15.66:0/86533583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd00c9040 con 0x7f2cd00cf290 2024-03-30T00:44:22.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cd8800640 1 -- 172.21.15.66:0/86533583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd00c9600 con 0x7f2cd00cf290 2024-03-30T00:44:22.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cd8800640 1 -- 172.21.15.66:0/86533583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd0066c30 con 0x7f2cd00cf290 2024-03-30T00:44:22.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.363+0000 7f2cca000640 1 -- 172.21.15.66:0/86533583 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2cc40b3040 con 0x7f2cd00cf290 2024-03-30T00:44:22.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.367+0000 7f2cca000640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2cb4066690 0x7f2cb4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:22.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.367+0000 7f2cca000640 1 -- 172.21.15.66:0/86533583 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2cc411e1d0 con 0x7f2cd00cf290 2024-03-30T00:44:22.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.367+0000 7f2cd5400640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2cb4066690 0x7f2cb4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:22.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.367+0000 7f2cd5400640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2cb4066690 0x7f2cb4068b50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0000c10 tx=0x7f2cc000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:22.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.371+0000 7f2cca000640 1 -- 172.21.15.66:0/86533583 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2cc40eb3f0 con 0x7f2cd00cf290 2024-03-30T00:44:22.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 -- 172.21.15.66:0/86533583 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2cd00b2bb0 con 0x7f2cd00cf290 2024-03-30T00:44:22.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cca000640 1 -- 172.21.15.66:0/86533583 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2cc40eea10 con 0x7f2cd00cf290 2024-03-30T00:44:22.574 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:44:22.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 -- 172.21.15.66:0/86533583 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2cb4066690 msgr2=0x7f2cb4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2cb4066690 0x7f2cb4068b50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0000c10 tx=0x7f2cc000b040 comp rx=0 tx=0).stop 2024-03-30T00:44:22.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 -- 172.21.15.66:0/86533583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd00cf290 msgr2=0x7f2cd00c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:22.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd00cf290 0x7f2cd00c9e00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401c880 tx=0x7f2cc4019140 comp rx=0 tx=0).stop 2024-03-30T00:44:22.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 -- 172.21.15.66:0/86533583 shutdown_connections 2024-03-30T00:44:22.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cd00c87a0 0x7f2cd015d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cd015a810 0x7f2cd0159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2cb4066690 0x7f2cb4068b50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 --2- 172.21.15.66:0/86533583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cd00cf290 0x7f2cd00c9e00 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:22.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 -- 172.21.15.66:0/86533583 >> 172.21.15.66:0/86533583 conn(0x7f2cd00b2090 msgr2=0x7f2cd00b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:22.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 -- 172.21.15.66:0/86533583 shutdown_connections 2024-03-30T00:44:22.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:22.567+0000 7f2cd8800640 1 -- 172.21.15.66:0/86533583 wait complete. 2024-03-30T00:44:22.596 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:44:22.596 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:44:22.596 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:44:22.596 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:44:22.596 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:22.596 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:44:22.596 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:44:22.596 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:44:22.596 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:44:22.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:22.597 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:44:22.606 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:44:22.606 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:44:22.606 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:44:22.606 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:44:22.607 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42216sec preferred_lft 42216sec 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:44:22.608 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:44:22.609 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:44:22.609 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:22.609 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:44:22.609 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:44:22.609 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:44:22.609 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:22.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:22.609 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:44:22.667 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:44:22.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:22.670 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:44:22.723 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:44:22.724 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:44:22.724 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:44:22.724 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:44:22.724 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:44:22.724 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:44:22.724 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:44:22.724 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:44:22.724 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:44:22.724 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:44:22.724 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:44:22.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:22.725 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:44:22.725 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:44:22.725 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:44:22.725 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:44:22.725 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:44:22.725 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:44:22.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:22.814 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:44:22.827 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:44:22.827 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:44:22.827 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:44:22.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:22.828 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:44:22.892 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:44:22.893 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:44:22.893 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:44:22.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:44:22.949 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:23.000 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:44:23.065 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:44:23.065 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:44:23.130 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:44:23.132 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:44:23.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:23.133 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:44:23.184 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:44:23.184 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:44:23.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:44:23.251 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:44:23.252 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:44:23.314 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:44:23.316 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:44:23.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:23.316 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:44:23.326 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:44:23.319+0000 7fbb956ed580 -1 init, newargv = 0x556bed08bd70 newargc=15 2024-03-30T00:44:23.327 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[44555]: starting ceph client 2024-03-30T00:44:23.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[44555]: starting fuse 2024-03-30T00:44:23.336 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:44:23.336 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:44:23.336 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:44:23.337 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:44:23.338 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:44:23.461 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.44555.asok 2024-03-30T00:44:23.645 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44555.asok 2024-03-30T00:44:23.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:23.646 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44555.asok status 2024-03-30T00:44:23.820 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "44555", 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:44:23.821 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 7140, 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 7140 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2044719730 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:23.822 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2044719730 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.7140 192.168.0.1:0/2044719730", 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/2044719730", 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 43, 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 82, 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:44:23.823 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:44:23.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:23.836 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:44:23.836 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:23.845 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:44:23.845 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:23.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:23.845 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:23.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:23.909 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-03-30T00:44:23.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:23.969 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-03-30T00:44:24.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:24.024 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-03-30T00:44:24.083 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:44:24.083 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:44:24.083 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:44:24.083 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:44:24.083 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-03-30T00:44:24.083 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:44:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627776 2024-03-30T00:44:24.381 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:44:24.381 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:44:24.437 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:24.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.611+0000 7f09efc00640 1 Processor -- start 2024-03-30T00:44:24.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.611+0000 7f09efc00640 1 -- start start 2024-03-30T00:44:24.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09efc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e80cd5e0 0x7f09e80cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:24.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09efc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e80cbec0 con 0x7f09e80ceb60 2024-03-30T00:44:24.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09efc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e80cdfa0 con 0x7f09e80ce780 2024-03-30T00:44:24.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09efc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e815d970 con 0x7f09e80cd5e0 2024-03-30T00:44:24.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09e7e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f09e80ceb60 0x7f09e815ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55108/0 (socket says 172.21.15.66:55108) 2024-03-30T00:44:24.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09e7e00640 1 -- 172.21.15.66:0/3499342489 learned_addr learned my addr 172.21.15.66:0/3499342489 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:24.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09edc00640 1 --2- 172.21.15.66:0/3499342489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e80cd5e0 0x7f09e80cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:24.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09e7400640 1 -- 172.21.15.66:0/3499342489 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520705591 0 0) 0x7f09e80cbec0 con 0x7f09e80ceb60 2024-03-30T00:44:24.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09e7400640 1 -- 172.21.15.66:0/3499342489 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09d0003530 con 0x7f09e80ceb60 2024-03-30T00:44:24.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09e7400640 1 -- 172.21.15.66:0/3499342489 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638313592 0 0) 0x7f09e80cdfa0 con 0x7f09e80ce780 2024-03-30T00:44:24.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09e7400640 1 -- 172.21.15.66:0/3499342489 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09e80cbec0 con 0x7f09e80ce780 2024-03-30T00:44:24.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09edc00640 1 -- 172.21.15.66:0/3499342489 >> v1:172.21.15.156:6789/0 conn(0x7f09e80ce780 legacy=0x7f09e80cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:24.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09edc00640 1 -- 172.21.15.66:0/3499342489 >> v1:172.21.15.66:6789/0 conn(0x7f09e80ceb60 legacy=0x7f09e815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:24.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.615+0000 7f09edc00640 1 -- 172.21.15.66:0/3499342489 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e815dc50 con 0x7f09e80cd5e0 2024-03-30T00:44:24.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09edc00640 1 --2- 172.21.15.66:0/3499342489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e80cd5e0 0x7f09e80cda60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009dc0 tx=0x7f09e001c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75311379fd2a9b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:24.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09e7400640 1 -- 172.21.15.66:0/3499342489 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e00a5990 con 0x7f09e80cd5e0 2024-03-30T00:44:24.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09e7400640 1 -- 172.21.15.66:0/3499342489 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e00a5af0 con 0x7f09e80cd5e0 2024-03-30T00:44:24.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09e7400640 1 -- 172.21.15.66:0/3499342489 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e00a5e80 con 0x7f09e80cd5e0 2024-03-30T00:44:24.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 -- 172.21.15.66:0/3499342489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e80cd5e0 msgr2=0x7f09e80cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:24.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 --2- 172.21.15.66:0/3499342489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e80cd5e0 0x7f09e80cda60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009dc0 tx=0x7f09e001c8d0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 -- 172.21.15.66:0/3499342489 shutdown_connections 2024-03-30T00:44:24.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 --2- 172.21.15.66:0/3499342489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e80cd5e0 0x7f09e80cda60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 -- 172.21.15.66:0/3499342489 >> 172.21.15.66:0/3499342489 conn(0x7f09e8069f60 msgr2=0x7f09e806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:24.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 -- 172.21.15.66:0/3499342489 shutdown_connections 2024-03-30T00:44:24.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 -- 172.21.15.66:0/3499342489 wait complete. 2024-03-30T00:44:24.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 Processor -- start 2024-03-30T00:44:24.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 -- start start 2024-03-30T00:44:24.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e80cd5e0 0x7f09e80c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e80ce780 0x7f09e80c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09e80ceb60 0x7f09e8155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80984c0 con 0x7f09e80ce780 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e81564a0 con 0x7f09e80ceb60 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09efc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80c6460 con 0x7f09e80cd5e0 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e80ce780 0x7f09e80c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e80ce780 0x7f09e80c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53362/0 (socket says 172.21.15.66:53362) 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09e7e00640 1 -- 172.21.15.66:0/3349979944 learned_addr learned my addr 172.21.15.66:0/3349979944 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.619+0000 7f09ed200640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e80cd5e0 0x7f09e80c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09edc00640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09e80ceb60 0x7f09e8155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09e7e00640 1 -- 172.21.15.66:0/3349979944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e80cd5e0 msgr2=0x7f09e80c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09e7e00640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e80cd5e0 0x7f09e80c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09e7e00640 1 -- 172.21.15.66:0/3349979944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09e80ceb60 msgr2=0x7f09e8155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:24.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09e7e00640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09e80ceb60 0x7f09e8155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09e7e00640 1 -- 172.21.15.66:0/3349979944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e0009a70 con 0x7f09e80ce780 2024-03-30T00:44:24.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09e7e00640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e80ce780 0x7f09e80c5dd0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f09d400c420 tx=0x7f09d400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:24.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09e4c00640 1 -- 172.21.15.66:0/3349979944 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4005cf0 con 0x7f09e80ce780 2024-03-30T00:44:24.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09e4c00640 1 -- 172.21.15.66:0/3349979944 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d4005e50 con 0x7f09e80ce780 2024-03-30T00:44:24.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09efc00640 1 -- 172.21.15.66:0/3349979944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e80c6740 con 0x7f09e80ce780 2024-03-30T00:44:24.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09e4c00640 1 -- 172.21.15.66:0/3349979944 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d40072d0 con 0x7f09e80ce780 2024-03-30T00:44:24.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09efc00640 1 -- 172.21.15.66:0/3349979944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e80c6d00 con 0x7f09e80ce780 2024-03-30T00:44:24.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09efc00640 1 -- 172.21.15.66:0/3349979944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e8066c30 con 0x7f09e80ce780 2024-03-30T00:44:24.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09e4c00640 1 -- 172.21.15.66:0/3349979944 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f09d4007430 con 0x7f09e80ce780 2024-03-30T00:44:24.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.623+0000 7f09e4c00640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09c8066690 0x7f09c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:24.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.627+0000 7f09e4c00640 1 -- 172.21.15.66:0/3349979944 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f09d4083880 con 0x7f09e80ce780 2024-03-30T00:44:24.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.627+0000 7f09ed200640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09c8066690 0x7f09c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:24.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.627+0000 7f09ed200640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09c8066690 0x7f09c8068b50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0011f0 tx=0x7f09dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:24.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.631+0000 7f09e4c00640 1 -- 172.21.15.66:0/3349979944 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f09d4050ac0 con 0x7f09e80ce780 2024-03-30T00:44:24.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.791+0000 7f09efc00640 1 -- 172.21.15.66:0/3349979944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09e80b2090 con 0x7f09e80ce780 2024-03-30T00:44:24.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.791+0000 7f09e4c00640 1 -- 172.21.15.66:0/3349979944 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16309 (secure 0 0 0) 0x7f09d4017090 con 0x7f09e80ce780 2024-03-30T00:44:24.799 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:24.799 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":82,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:17.276711+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:44:14.685880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-30T00:44:15.260373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:24.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 -- 172.21.15.66:0/3349979944 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09c8066690 msgr2=0x7f09c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:24.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09c8066690 0x7f09c8068b50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0011f0 tx=0x7f09dc009040 comp rx=0 tx=0).stop 2024-03-30T00:44:24.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 -- 172.21.15.66:0/3349979944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e80ce780 msgr2=0x7f09e80c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:24.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e80ce780 0x7f09e80c5dd0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f09d400c420 tx=0x7f09d400c8e0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 -- 172.21.15.66:0/3349979944 shutdown_connections 2024-03-30T00:44:24.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09e80ceb60 0x7f09e8155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09c8066690 0x7f09c8068b50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e80ce780 0x7f09e80c5dd0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 --2- 172.21.15.66:0/3349979944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e80cd5e0 0x7f09e80c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 -- 172.21.15.66:0/3349979944 >> 172.21.15.66:0/3349979944 conn(0x7f09e8069f60 msgr2=0x7f09e806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:24.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 -- 172.21.15.66:0/3349979944 shutdown_connections 2024-03-30T00:44:24.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.795+0000 7f09efc00640 1 -- 172.21.15.66:0/3349979944 wait complete. 2024-03-30T00:44:24.821 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:44:14.685880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:44:15.260373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:44:24.821 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:44:24.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.955+0000 7f1a0b200640 1 Processor -- start 2024-03-30T00:44:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a0b200640 1 -- start start 2024-03-30T00:44:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a0b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a0c0d0360 0x7f1a0c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:24.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a0b200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a0c15cc40 0x7f1a0c160490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:24.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a0b200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0c06cde0 con 0x7f1a0c0d0740 2024-03-30T00:44:24.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a0b200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0c1613f0 con 0x7f1a0c15cc40 2024-03-30T00:44:24.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a0b200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c160b50 con 0x7f1a0c0d0360 2024-03-30T00:44:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a01400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a0c15cc40 0x7f1a0c160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a09e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a0c0d0360 0x7f1a0c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a01400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a0c15cc40 0x7f1a0c160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:42428/0 (socket says 172.21.15.66:42428) 2024-03-30T00:44:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a01400640 1 -- 172.21.15.66:0/3539152875 learned_addr learned my addr 172.21.15.66:0/3539152875 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a09e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a0c0d0360 0x7f1a0c0cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:52650/0 (socket says 172.21.15.66:52650) 2024-03-30T00:44:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a01400640 1 -- 172.21.15.66:0/3539152875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a0c0d0360 msgr2=0x7f1a0c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a01400640 1 --2- 172.21.15.66:0/3539152875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a0c0d0360 0x7f1a0c0cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a01400640 1 -- 172.21.15.66:0/3539152875 >> v1:172.21.15.156:6789/0 conn(0x7f1a0c0d0740 legacy=0x7f1a0c15d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a01400640 1 -- 172.21.15.66:0/3539152875 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c160e30 con 0x7f1a0c15cc40 2024-03-30T00:44:24.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.959+0000 7f1a09400640 1 -- 172.21.15.66:0/3539152875 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713279347 0 0) 0x7f1a0c06cde0 con 0x7f1a0c0d0740 2024-03-30T00:44:24.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a01400640 1 --2- 172.21.15.66:0/3539152875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a0c15cc40 0x7f1a0c160490 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009870 tx=0x7f19f401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37cf172f1aba7d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:24.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a09400640 1 -- 172.21.15.66:0/3539152875 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f40a95f0 con 0x7f1a0c15cc40 2024-03-30T00:44:24.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a09400640 1 -- 172.21.15.66:0/3539152875 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f40a9750 con 0x7f1a0c15cc40 2024-03-30T00:44:24.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a09400640 1 -- 172.21.15.66:0/3539152875 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f40a9ae0 con 0x7f1a0c15cc40 2024-03-30T00:44:24.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a0b200640 1 -- 172.21.15.66:0/3539152875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a0c15cc40 msgr2=0x7f1a0c160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:24.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a0b200640 1 --2- 172.21.15.66:0/3539152875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a0c15cc40 0x7f1a0c160490 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009870 tx=0x7f19f401c530 comp rx=0 tx=0).stop 2024-03-30T00:44:24.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a0b200640 1 -- 172.21.15.66:0/3539152875 shutdown_connections 2024-03-30T00:44:24.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a0b200640 1 --2- 172.21.15.66:0/3539152875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a0c15cc40 0x7f1a0c160490 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a0b200640 1 --2- 172.21.15.66:0/3539152875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a0c0d0360 0x7f1a0c0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a0b200640 1 -- 172.21.15.66:0/3539152875 >> 172.21.15.66:0/3539152875 conn(0x7f1a0c06e150 msgr2=0x7f1a0c06e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:24.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a0b200640 1 -- 172.21.15.66:0/3539152875 shutdown_connections 2024-03-30T00:44:24.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a0b200640 1 -- 172.21.15.66:0/3539152875 wait complete. 2024-03-30T00:44:24.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a0b200640 1 Processor -- start 2024-03-30T00:44:24.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.963+0000 7f1a0b200640 1 -- start start 2024-03-30T00:44:24.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a0b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a0c0d0360 0x7f1a0c1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:24.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a0b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a0c0d0740 0x7f1a0c14e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:24.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a0b200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a0c15cc40 0x7f1a0c1494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:24.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a0b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c0c61d0 con 0x7f1a0c0d0740 2024-03-30T00:44:24.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a0b200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c0c55c0 con 0x7f1a0c15cc40 2024-03-30T00:44:24.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a0b200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c0c58a0 con 0x7f1a0c0d0360 2024-03-30T00:44:24.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a01400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a0c0d0740 0x7f1a0c14e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:24.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a01400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a0c0d0740 0x7f1a0c14e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53374/0 (socket says 172.21.15.66:53374) 2024-03-30T00:44:24.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a01400640 1 -- 172.21.15.66:0/1934115069 learned_addr learned my addr 172.21.15.66:0/1934115069 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:24.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a09e00640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a0c0d0360 0x7f1a0c1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:24.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a0a800640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a0c15cc40 0x7f1a0c1494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a01400640 1 -- 172.21.15.66:0/1934115069 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a0c0d0360 msgr2=0x7f1a0c1558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a01400640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a0c0d0360 0x7f1a0c1558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a01400640 1 -- 172.21.15.66:0/1934115069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a0c15cc40 msgr2=0x7f1a0c1494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a01400640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a0c15cc40 0x7f1a0c1494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a01400640 1 -- 172.21.15.66:0/1934115069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40094d0 con 0x7f1a0c0d0740 2024-03-30T00:44:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a01400640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a0c0d0740 0x7f1a0c14e550 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f19f40ab040 tx=0x7f19f401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a02a00640 1 -- 172.21.15.66:0/1934115069 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f40bc650 con 0x7f1a0c0d0740 2024-03-30T00:44:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a02a00640 1 -- 172.21.15.66:0/1934115069 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f40bc7b0 con 0x7f1a0c0d0740 2024-03-30T00:44:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a02a00640 1 -- 172.21.15.66:0/1934115069 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f40bcb60 con 0x7f1a0c0d0740 2024-03-30T00:44:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a0b200640 1 -- 172.21.15.66:0/1934115069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c0c5b80 con 0x7f1a0c0d0740 2024-03-30T00:44:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a0b200640 1 -- 172.21.15.66:0/1934115069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c155e20 con 0x7f1a0c0d0740 2024-03-30T00:44:24.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.967+0000 7f1a0b200640 1 -- 172.21.15.66:0/1934115069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c0c6340 con 0x7f1a0c0d0740 2024-03-30T00:44:24.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.971+0000 7f1a02a00640 1 -- 172.21.15.66:0/1934115069 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f19f40bccc0 con 0x7f1a0c0d0740 2024-03-30T00:44:24.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.971+0000 7f1a02a00640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e8066690 0x7f19e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:24.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.971+0000 7f1a02a00640 1 -- 172.21.15.66:0/1934115069 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f19f411eb20 con 0x7f1a0c0d0740 2024-03-30T00:44:24.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.971+0000 7f1a09e00640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e8066690 0x7f19e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:24.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.971+0000 7f1a09e00640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e8066690 0x7f19e8068b50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009ac0 tx=0x7f19fc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:24.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:24.975+0000 7f1a02a00640 1 -- 172.21.15.66:0/1934115069 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f19f40ebd60 con 0x7f1a0c0d0740 2024-03-30T00:44:25.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.135+0000 7f1a0b200640 1 -- 172.21.15.66:0/1934115069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a0c0c59e0 con 0x7f1a0c0d0740 2024-03-30T00:44:25.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.135+0000 7f1a02a00640 1 -- 172.21.15.66:0/1934115069 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a0c0c59e0 con 0x7f1a0c0d0740 2024-03-30T00:44:25.143 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:44:25.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 -- 172.21.15.66:0/1934115069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e8066690 msgr2=0x7f19e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e8066690 0x7f19e8068b50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009ac0 tx=0x7f19fc009190 comp rx=0 tx=0).stop 2024-03-30T00:44:25.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 -- 172.21.15.66:0/1934115069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a0c0d0740 msgr2=0x7f1a0c14e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a0c0d0740 0x7f1a0c14e550 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f19f40ab040 tx=0x7f19f401ad70 comp rx=0 tx=0).stop 2024-03-30T00:44:25.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 -- 172.21.15.66:0/1934115069 shutdown_connections 2024-03-30T00:44:25.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a0c15cc40 0x7f1a0c1494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f19e8066690 0x7f19e8068b50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a0c0d0740 0x7f1a0c14e550 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 --2- 172.21.15.66:0/1934115069 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a0c0d0360 0x7f1a0c1558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 -- 172.21.15.66:0/1934115069 >> 172.21.15.66:0/1934115069 conn(0x7f1a0c06e150 msgr2=0x7f1a0c15ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:25.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 -- 172.21.15.66:0/1934115069 shutdown_connections 2024-03-30T00:44:25.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.139+0000 7f1a0b200640 1 -- 172.21.15.66:0/1934115069 wait complete. 2024-03-30T00:44:25.164 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:44:25.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.299+0000 7f92f1c00640 1 Processor -- start 2024-03-30T00:44:25.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.299+0000 7f92f1c00640 1 -- start start 2024-03-30T00:44:25.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92f1c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92ec15ca80 0x7f92ec06b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92f1c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f92ec0cf2f0 0x7f92ec06b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92f1c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f92ec06bb30 0x7f92ec098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92f1c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ec0cb0e0 con 0x7f92ec06bb30 2024-03-30T00:44:25.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92f1c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ec0cf7f0 con 0x7f92ec0cf2f0 2024-03-30T00:44:25.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92f1c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec0cfad0 con 0x7f92ec15ca80 2024-03-30T00:44:25.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92eb400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f92ec06bb30 0x7f92ec098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92eb400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f92ec06bb30 0x7f92ec098a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53386/0 (socket says 172.21.15.66:53386) 2024-03-30T00:44:25.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92eb400640 1 -- 172.21.15.66:0/4106367368 learned_addr learned my addr 172.21.15.66:0/4106367368 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:25.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92eaa00640 1 --2- 172.21.15.66:0/4106367368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92ec15ca80 0x7f92ec06b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92e2000640 1 --2- 172.21.15.66:0/4106367368 >> v2:172.21.15.156:3300/0 conn(0x7f92ec0cf2f0 0x7f92ec06b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92eb400640 1 -- 172.21.15.66:0/4106367368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92ec15ca80 msgr2=0x7f92ec06b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:25.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92eb400640 1 --2- 172.21.15.66:0/4106367368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92ec15ca80 0x7f92ec06b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92eb400640 1 -- 172.21.15.66:0/4106367368 >> v2:172.21.15.156:3300/0 conn(0x7f92ec0cf2f0 msgr2=0x7f92ec06b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:25.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92eb400640 1 --2- 172.21.15.66:0/4106367368 >> v2:172.21.15.156:3300/0 conn(0x7f92ec0cf2f0 0x7f92ec06b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.303+0000 7f92eb400640 1 -- 172.21.15.66:0/4106367368 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec0cad70 con 0x7f92ec06bb30 2024-03-30T00:44:25.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92eb400640 1 --2- 172.21.15.66:0/4106367368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92ec06bb30 0x7f92ec098a20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0711d0 tx=0x7f92e401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b19715c7a5ed057 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:25.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92ea000640 1 -- 172.21.15.66:0/4106367368 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40a89d0 con 0x7f92ec06bb30 2024-03-30T00:44:25.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92ea000640 1 -- 172.21.15.66:0/4106367368 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e40a8b30 con 0x7f92ec06bb30 2024-03-30T00:44:25.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92ea000640 1 -- 172.21.15.66:0/4106367368 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40b0440 con 0x7f92ec06bb30 2024-03-30T00:44:25.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 -- 172.21.15.66:0/4106367368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92ec06bb30 msgr2=0x7f92ec098a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 --2- 172.21.15.66:0/4106367368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92ec06bb30 0x7f92ec098a20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0711d0 tx=0x7f92e401ed10 comp rx=0 tx=0).stop 2024-03-30T00:44:25.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 -- 172.21.15.66:0/4106367368 shutdown_connections 2024-03-30T00:44:25.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 --2- 172.21.15.66:0/4106367368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92ec06bb30 0x7f92ec098a20 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 --2- 172.21.15.66:0/4106367368 >> v2:172.21.15.156:3300/0 conn(0x7f92ec0cf2f0 0x7f92ec06b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 --2- 172.21.15.66:0/4106367368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92ec15ca80 0x7f92ec06b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 -- 172.21.15.66:0/4106367368 >> 172.21.15.66:0/4106367368 conn(0x7f92ec0b2090 msgr2=0x7f92ec0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:25.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 -- 172.21.15.66:0/4106367368 shutdown_connections 2024-03-30T00:44:25.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 -- 172.21.15.66:0/4106367368 wait complete. 2024-03-30T00:44:25.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 Processor -- start 2024-03-30T00:44:25.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 -- start start 2024-03-30T00:44:25.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92ec06bb30 0x7f92ec0b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ec0cf2f0 0x7f92ec0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92ec15ca80 0x7f92ec0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec0ca520 con 0x7f92ec15ca80 2024-03-30T00:44:25.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec0ca800 con 0x7f92ec0cf2f0 2024-03-30T00:44:25.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.307+0000 7f92f1c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec146b70 con 0x7f92ec06bb30 2024-03-30T00:44:25.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92eb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92ec15ca80 0x7f92ec0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92eaa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92ec06bb30 0x7f92ec0b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92eb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92ec15ca80 0x7f92ec0b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53390/0 (socket says 172.21.15.66:53390) 2024-03-30T00:44:25.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92eb400640 1 -- 172.21.15.66:0/1845919660 learned_addr learned my addr 172.21.15.66:0/1845919660 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:25.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92e2000640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ec0cf2f0 0x7f92ec0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92eb400640 1 -- 172.21.15.66:0/1845919660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92ec06bb30 msgr2=0x7f92ec0b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92eb400640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92ec06bb30 0x7f92ec0b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92eb400640 1 -- 172.21.15.66:0/1845919660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ec0cf2f0 msgr2=0x7f92ec0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92eb400640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ec0cf2f0 0x7f92ec0b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92eb400640 1 -- 172.21.15.66:0/1845919660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40094d0 con 0x7f92ec15ca80 2024-03-30T00:44:25.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92eb400640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92ec15ca80 0x7f92ec0b4360 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f92e40097c0 tx=0x7f92e401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:25.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92e3400640 1 -- 172.21.15.66:0/1845919660 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40b0cc0 con 0x7f92ec15ca80 2024-03-30T00:44:25.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92e3400640 1 -- 172.21.15.66:0/1845919660 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e40b0e20 con 0x7f92ec15ca80 2024-03-30T00:44:25.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92e3400640 1 -- 172.21.15.66:0/1845919660 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40bb660 con 0x7f92ec15ca80 2024-03-30T00:44:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92f1c00640 1 -- 172.21.15.66:0/1845919660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ec146e50 con 0x7f92ec15ca80 2024-03-30T00:44:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92f1c00640 1 -- 172.21.15.66:0/1845919660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec0b56a0 con 0x7f92ec15ca80 2024-03-30T00:44:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92f1c00640 1 -- 172.21.15.66:0/1845919660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ec066c30 con 0x7f92ec15ca80 2024-03-30T00:44:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.311+0000 7f92e3400640 1 -- 172.21.15.66:0/1845919660 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f92e40bb7c0 con 0x7f92ec15ca80 2024-03-30T00:44:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.315+0000 7f92e3400640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f92c8066690 0x7f92c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.315+0000 7f92e3400640 1 -- 172.21.15.66:0/1845919660 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f92e4121a40 con 0x7f92ec15ca80 2024-03-30T00:44:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.315+0000 7f92eaa00640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f92c8066690 0x7f92c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.315+0000 7f92eaa00640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f92c8066690 0x7f92c8068b50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f92dc011340 tx=0x7f92dc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:25.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.319+0000 7f92e3400640 1 -- 172.21.15.66:0/1845919660 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f92e40eed10 con 0x7f92ec15ca80 2024-03-30T00:44:25.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 -- 172.21.15.66:0/1845919660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92ec147300 con 0x7f92ec15ca80 2024-03-30T00:44:25.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92e3400640 1 -- 172.21.15.66:0/1845919660 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f92e40b50a0 con 0x7f92ec15ca80 2024-03-30T00:44:25.487 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:44:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 -- 172.21.15.66:0/1845919660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f92c8066690 msgr2=0x7f92c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f92c8066690 0x7f92c8068b50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f92dc011340 tx=0x7f92dc00b040 comp rx=0 tx=0).stop 2024-03-30T00:44:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 -- 172.21.15.66:0/1845919660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92ec15ca80 msgr2=0x7f92ec0b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92ec15ca80 0x7f92ec0b4360 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f92e40097c0 tx=0x7f92e401bab0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 -- 172.21.15.66:0/1845919660 shutdown_connections 2024-03-30T00:44:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f92c8066690 0x7f92c8068b50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92ec15ca80 0x7f92ec0b4360 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ec0cf2f0 0x7f92ec0b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 --2- 172.21.15.66:0/1845919660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92ec06bb30 0x7f92ec0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 -- 172.21.15.66:0/1845919660 >> 172.21.15.66:0/1845919660 conn(0x7f92ec0b2090 msgr2=0x7f92ec0ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 -- 172.21.15.66:0/1845919660 shutdown_connections 2024-03-30T00:44:25.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.479+0000 7f92f1c00640 1 -- 172.21.15.66:0/1845919660 wait complete. 2024-03-30T00:44:25.509 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:44:25.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.647+0000 7fde7bc00640 1 Processor -- start 2024-03-30T00:44:25.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.647+0000 7fde7bc00640 1 -- start start 2024-03-30T00:44:25.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde7bc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fde740ae4e0 0x7fde740cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde7bc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fde740ce820 0x7fde740cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde7bc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde7406e7d0 con 0x7fde7415ca10 2024-03-30T00:44:25.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde7bc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde740b1f70 con 0x7fde740ce820 2024-03-30T00:44:25.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde7bc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde740cf1f0 con 0x7fde740ae4e0 2024-03-30T00:44:25.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde79c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fde740ce820 0x7fde740cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde79c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fde740ce820 0x7fde740cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53394/0 (socket says 172.21.15.66:53394) 2024-03-30T00:44:25.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde79200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fde7415ca10 0x7fde7406ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55112/0 (socket says 172.21.15.66:55112) 2024-03-30T00:44:25.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde79c00640 1 -- 172.21.15.66:0/2142032388 learned_addr learned my addr 172.21.15.66:0/2142032388 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:25.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde6be00640 1 --2- 172.21.15.66:0/2142032388 >> v2:172.21.15.156:3300/0 conn(0x7fde740ae4e0 0x7fde740cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde79c00640 1 -- 172.21.15.66:0/2142032388 >> v2:172.21.15.156:3300/0 conn(0x7fde740ae4e0 msgr2=0x7fde740cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:25.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde79c00640 1 --2- 172.21.15.66:0/2142032388 >> v2:172.21.15.156:3300/0 conn(0x7fde740ae4e0 0x7fde740cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde79c00640 1 -- 172.21.15.66:0/2142032388 >> v1:172.21.15.66:6789/0 conn(0x7fde7415ca10 legacy=0x7fde7406ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde79c00640 1 -- 172.21.15.66:0/2142032388 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7415cdf0 con 0x7fde740ce820 2024-03-30T00:44:25.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde73e00640 1 -- 172.21.15.66:0/2142032388 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265897115 0 0) 0x7fde7406e7d0 con 0x7fde7415ca10 2024-03-30T00:44:25.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.651+0000 7fde79c00640 1 --2- 172.21.15.66:0/2142032388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde740ce820 0x7fde740cecb0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fde740ae490 tx=0x7fde6c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7766bd8996b1e8f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:25.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde73e00640 1 -- 172.21.15.66:0/2142032388 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0a58a0 con 0x7fde740ce820 2024-03-30T00:44:25.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde73e00640 1 -- 172.21.15.66:0/2142032388 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c0a5a00 con 0x7fde740ce820 2024-03-30T00:44:25.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde73e00640 1 -- 172.21.15.66:0/2142032388 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0a5d90 con 0x7fde740ce820 2024-03-30T00:44:25.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 -- 172.21.15.66:0/2142032388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde740ce820 msgr2=0x7fde740cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 --2- 172.21.15.66:0/2142032388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde740ce820 0x7fde740cecb0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fde740ae490 tx=0x7fde6c01c9c0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 -- 172.21.15.66:0/2142032388 shutdown_connections 2024-03-30T00:44:25.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 --2- 172.21.15.66:0/2142032388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde740ce820 0x7fde740cecb0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 --2- 172.21.15.66:0/2142032388 >> v2:172.21.15.156:3300/0 conn(0x7fde740ae4e0 0x7fde740cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 -- 172.21.15.66:0/2142032388 >> 172.21.15.66:0/2142032388 conn(0x7fde74098a20 msgr2=0x7fde740b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:25.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 -- 172.21.15.66:0/2142032388 shutdown_connections 2024-03-30T00:44:25.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 -- 172.21.15.66:0/2142032388 wait complete. 2024-03-30T00:44:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 Processor -- start 2024-03-30T00:44:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 -- start start 2024-03-30T00:44:25.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde740ae4e0 0x7fde7415fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde740ce820 0x7fde74156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde7415ca10 0x7fde741f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7415f890 con 0x7fde740ae4e0 2024-03-30T00:44:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde74156aa0 con 0x7fde7415ca10 2024-03-30T00:44:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde7bc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7415d580 con 0x7fde740ce820 2024-03-30T00:44:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde79200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde740ae4e0 0x7fde7415fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde79200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde740ae4e0 0x7fde7415fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53398/0 (socket says 172.21.15.66:53398) 2024-03-30T00:44:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde79200640 1 -- 172.21.15.66:0/281069553 learned_addr learned my addr 172.21.15.66:0/281069553 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.655+0000 7fde79c00640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde7415ca10 0x7fde741f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde6be00640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde740ce820 0x7fde74156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde79200640 1 -- 172.21.15.66:0/281069553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde740ce820 msgr2=0x7fde74156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde79200640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde740ce820 0x7fde74156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde79200640 1 -- 172.21.15.66:0/281069553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde7415ca10 msgr2=0x7fde741f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde79200640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde7415ca10 0x7fde741f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde79200640 1 -- 172.21.15.66:0/281069553 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c009b30 con 0x7fde740ae4e0 2024-03-30T00:44:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde6be00640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde740ce820 0x7fde74156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde79200640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde740ae4e0 0x7fde7415fd20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fde6400c480 tx=0x7fde6400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde71600640 1 -- 172.21.15.66:0/281069553 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde640074e0 con 0x7fde740ae4e0 2024-03-30T00:44:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde71600640 1 -- 172.21.15.66:0/281069553 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde64007660 con 0x7fde740ae4e0 2024-03-30T00:44:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde71600640 1 -- 172.21.15.66:0/281069553 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64004a30 con 0x7fde740ae4e0 2024-03-30T00:44:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde7bc00640 1 -- 172.21.15.66:0/281069553 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7415f240 con 0x7fde740ae4e0 2024-03-30T00:44:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde7bc00640 1 -- 172.21.15.66:0/281069553 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7415d220 con 0x7fde740ae4e0 2024-03-30T00:44:25.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde7bc00640 1 -- 172.21.15.66:0/281069553 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde74066c30 con 0x7fde740ae4e0 2024-03-30T00:44:25.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde71600640 1 -- 172.21.15.66:0/281069553 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fde64004b90 con 0x7fde740ae4e0 2024-03-30T00:44:25.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde71600640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde50066690 0x7fde50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde71600640 1 -- 172.21.15.66:0/281069553 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fde64083320 con 0x7fde740ae4e0 2024-03-30T00:44:25.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.659+0000 7fde6be00640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde50066690 0x7fde50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.663+0000 7fde6be00640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde50066690 0x7fde50068b50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fde740cba30 tx=0x7fde5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:25.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.663+0000 7fde71600640 1 -- 172.21.15.66:0/281069553 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fde64054100 con 0x7fde740ae4e0 2024-03-30T00:44:25.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 -- 172.21.15.66:0/281069553 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde740b5430 con 0x7fde740ae4e0 2024-03-30T00:44:25.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde71600640 1 -- 172.21.15.66:0/281069553 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde64053c90 con 0x7fde740ae4e0 2024-03-30T00:44:25.833 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:44:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 -- 172.21.15.66:0/281069553 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde50066690 msgr2=0x7fde50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde50066690 0x7fde50068b50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fde740cba30 tx=0x7fde5c008040 comp rx=0 tx=0).stop 2024-03-30T00:44:25.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 -- 172.21.15.66:0/281069553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde740ae4e0 msgr2=0x7fde7415fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde740ae4e0 0x7fde7415fd20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fde6400c480 tx=0x7fde6400c940 comp rx=0 tx=0).stop 2024-03-30T00:44:25.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 -- 172.21.15.66:0/281069553 shutdown_connections 2024-03-30T00:44:25.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde7415ca10 0x7fde741f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde740ce820 0x7fde74156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde50066690 0x7fde50068b50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 --2- 172.21.15.66:0/281069553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde740ae4e0 0x7fde7415fd20 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:25.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 -- 172.21.15.66:0/281069553 >> 172.21.15.66:0/281069553 conn(0x7fde74098a20 msgr2=0x7fde7406d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:25.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 -- 172.21.15.66:0/281069553 shutdown_connections 2024-03-30T00:44:25.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.827+0000 7fde7bc00640 1 -- 172.21.15.66:0/281069553 wait complete. 2024-03-30T00:44:25.855 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:44:25.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ecd000640 1 Processor -- start 2024-03-30T00:44:25.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ecd000640 1 -- start start 2024-03-30T00:44:25.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ecd000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1ec80cec30 0x7f1ec8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:25.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ecd000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec80cd3e0 con 0x7f1ec80cda90 2024-03-30T00:44:25.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ecd000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec815a990 con 0x7f1ec80cde70 2024-03-30T00:44:25.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ecd000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec80b1b80 con 0x7f1ec80cec30 2024-03-30T00:44:25.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ec5e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1ec80cda90 0x7f1ec80ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55124/0 (socket says 172.21.15.66:55124) 2024-03-30T00:44:25.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ec5e00640 1 -- 172.21.15.66:0/2587045855 learned_addr learned my addr 172.21.15.66:0/2587045855 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:25.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ebd400640 1 --2- 172.21.15.66:0/2587045855 >> v2:172.21.15.156:3300/0 conn(0x7f1ec80cec30 0x7f1ec8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:25.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ec5400640 1 -- 172.21.15.66:0/2587045855 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215053467 0 0) 0x7f1ec80cd3e0 con 0x7f1ec80cda90 2024-03-30T00:44:25.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ec5400640 1 -- 172.21.15.66:0/2587045855 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eac003530 con 0x7f1ec80cda90 2024-03-30T00:44:25.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ebd400640 1 -- 172.21.15.66:0/2587045855 >> v1:172.21.15.156:6789/0 conn(0x7f1ec80cde70 legacy=0x7f1ec815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ebd400640 1 -- 172.21.15.66:0/2587045855 >> v1:172.21.15.66:6789/0 conn(0x7f1ec80cda90 legacy=0x7f1ec80ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:25.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.991+0000 7f1ebd400640 1 -- 172.21.15.66:0/2587045855 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80b11e0 con 0x7f1ec80cec30 2024-03-30T00:44:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ebd400640 1 --2- 172.21.15.66:0/2587045855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec80cec30 0x7f1ec8159c60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00097c0 tx=0x7f1eb001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fae647625d1bdd78 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ec5400640 1 -- 172.21.15.66:0/2587045855 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00ab610 con 0x7f1ec80cec30 2024-03-30T00:44:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ec5400640 1 -- 172.21.15.66:0/2587045855 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb00ab770 con 0x7f1ec80cec30 2024-03-30T00:44:26.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ec5400640 1 -- 172.21.15.66:0/2587045855 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00abae0 con 0x7f1ec80cec30 2024-03-30T00:44:26.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ecd000640 1 -- 172.21.15.66:0/2587045855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec80cec30 msgr2=0x7f1ec8159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ecd000640 1 --2- 172.21.15.66:0/2587045855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec80cec30 0x7f1ec8159c60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00097c0 tx=0x7f1eb001c410 comp rx=0 tx=0).stop 2024-03-30T00:44:26.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ecd000640 1 -- 172.21.15.66:0/2587045855 shutdown_connections 2024-03-30T00:44:26.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ecd000640 1 --2- 172.21.15.66:0/2587045855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec80cec30 0x7f1ec8159c60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ecd000640 1 -- 172.21.15.66:0/2587045855 >> 172.21.15.66:0/2587045855 conn(0x7f1ec8069f60 msgr2=0x7f1ec806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:26.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ecd000640 1 -- 172.21.15.66:0/2587045855 shutdown_connections 2024-03-30T00:44:26.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ecd000640 1 -- 172.21.15.66:0/2587045855 wait complete. 2024-03-30T00:44:26.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ecd000640 1 Processor -- start 2024-03-30T00:44:26.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.995+0000 7f1ecd000640 1 -- start start 2024-03-30T00:44:26.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ecd000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec80cda90 0x7f1ec80c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ecd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ec80cde70 0x7f1ec806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ecd000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ec80cec30 0x7f1ec8158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ecd000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec81636e0 con 0x7f1ec80cde70 2024-03-30T00:44:26.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ecd000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c27e0 con 0x7f1ec80cda90 2024-03-30T00:44:26.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ecd000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c2a60 con 0x7f1ec80cec30 2024-03-30T00:44:26.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ec80cde70 0x7f1ec806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ec80cde70 0x7f1ec806d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53406/0 (socket says 172.21.15.66:53406) 2024-03-30T00:44:26.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebd400640 1 -- 172.21.15.66:0/3674162113 learned_addr learned my addr 172.21.15.66:0/3674162113 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:26.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ec5e00640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec80cda90 0x7f1ec80c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ec6800640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ec80cec30 0x7f1ec8158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebd400640 1 -- 172.21.15.66:0/3674162113 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ec80cec30 msgr2=0x7f1ec8158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebd400640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ec80cec30 0x7f1ec8158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebd400640 1 -- 172.21.15.66:0/3674162113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec80cda90 msgr2=0x7f1ec80c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebd400640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec80cda90 0x7f1ec80c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebd400640 1 -- 172.21.15.66:0/3674162113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb0009420 con 0x7f1ec80cde70 2024-03-30T00:44:26.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebd400640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ec80cde70 0x7f1ec806d560 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00ad040 tx=0x7f1eb001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:26.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebea00640 1 -- 172.21.15.66:0/3674162113 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0004220 con 0x7f1ec80cde70 2024-03-30T00:44:26.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebea00640 1 -- 172.21.15.66:0/3674162113 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb00b9070 con 0x7f1ec80cde70 2024-03-30T00:44:26.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ebea00640 1 -- 172.21.15.66:0/3674162113 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0003620 con 0x7f1ec80cde70 2024-03-30T00:44:26.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ecd000640 1 -- 172.21.15.66:0/3674162113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec8163090 con 0x7f1ec80cde70 2024-03-30T00:44:26.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ecd000640 1 -- 172.21.15.66:0/3674162113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec806daa0 con 0x7f1ec80cde70 2024-03-30T00:44:26.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:25.999+0000 7f1ecd000640 1 -- 172.21.15.66:0/3674162113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec8162ef0 con 0x7f1ec80cde70 2024-03-30T00:44:26.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.003+0000 7f1ebea00640 1 -- 172.21.15.66:0/3674162113 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1eb0003780 con 0x7f1ec80cde70 2024-03-30T00:44:26.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.003+0000 7f1ebea00640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ea4066690 0x7f1ea4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.003+0000 7f1ebea00640 1 -- 172.21.15.66:0/3674162113 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1eb00bb070 con 0x7f1ec80cde70 2024-03-30T00:44:26.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.003+0000 7f1ec5e00640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ea4066690 0x7f1ea4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.003+0000 7f1ec5e00640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ea4066690 0x7f1ea4068b50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8001570 tx=0x7f1eb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:26.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.007+0000 7f1ebea00640 1 -- 172.21.15.66:0/3674162113 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1ec8162ef0 con 0x7f1ec80cde70 2024-03-30T00:44:26.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 -- 172.21.15.66:0/3674162113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1ec8163890 con 0x7f1ec80cde70 2024-03-30T00:44:26.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ebea00640 1 -- 172.21.15.66:0/3674162113 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v82) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1ec8163890 con 0x7f1ec80cde70 2024-03-30T00:44:26.189 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:44:26.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 -- 172.21.15.66:0/3674162113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ea4066690 msgr2=0x7f1ea4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ea4066690 0x7f1ea4068b50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8001570 tx=0x7f1eb8009040 comp rx=0 tx=0).stop 2024-03-30T00:44:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 -- 172.21.15.66:0/3674162113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ec80cde70 msgr2=0x7f1ec806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ec80cde70 0x7f1ec806d560 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00ad040 tx=0x7f1eb001ac50 comp rx=0 tx=0).stop 2024-03-30T00:44:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 -- 172.21.15.66:0/3674162113 shutdown_connections 2024-03-30T00:44:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ec80cec30 0x7f1ec8158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ea4066690 0x7f1ea4068b50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ec80cde70 0x7f1ec806d560 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 --2- 172.21.15.66:0/3674162113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec80cda90 0x7f1ec80c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 -- 172.21.15.66:0/3674162113 >> 172.21.15.66:0/3674162113 conn(0x7f1ec8069f60 msgr2=0x7f1ec815adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 -- 172.21.15.66:0/3674162113 shutdown_connections 2024-03-30T00:44:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.183+0000 7f1ecd000640 1 -- 172.21.15.66:0/3674162113 wait complete. 2024-03-30T00:44:26.211 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:44:26.211 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:44:26.220 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:26.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.395+0000 7f3fd3400640 1 Processor -- start 2024-03-30T00:44:26.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.395+0000 7f3fd3400640 1 -- start start 2024-03-30T00:44:26.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd3400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fcc0cf2f0 0x7f3fcc06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3fcc0cb570 0x7f3fcc0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd3400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fcc0cb0e0 con 0x7f3fcc15ca80 2024-03-30T00:44:26.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd3400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fcc0cf7f0 con 0x7f3fcc0cb570 2024-03-30T00:44:26.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd3400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0cfad0 con 0x7f3fcc0cf2f0 2024-03-30T00:44:26.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd1400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3fcc0cb570 0x7f3fcc0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd1400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3fcc0cb570 0x7f3fcc0cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53420/0 (socket says 172.21.15.66:53420) 2024-03-30T00:44:26.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd1400640 1 -- 172.21.15.66:0/390588456 learned_addr learned my addr 172.21.15.66:0/390588456 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:26.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd0a00640 1 --1- 172.21.15.66:0/390588456 >> v1:172.21.15.66:6789/0 conn(0x7f3fcc15ca80 0x7f3fcc06b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55130/0 (socket says 172.21.15.66:55130) 2024-03-30T00:44:26.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fcbe00640 1 --2- 172.21.15.66:0/390588456 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fcc0cf2f0 0x7f3fcc06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd1400640 1 -- 172.21.15.66:0/390588456 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fcc0cf2f0 msgr2=0x7f3fcc06b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:26.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd1400640 1 --2- 172.21.15.66:0/390588456 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fcc0cf2f0 0x7f3fcc06b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd1400640 1 -- 172.21.15.66:0/390588456 >> v1:172.21.15.66:6789/0 conn(0x7f3fcc15ca80 legacy=0x7f3fcc06b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd1400640 1 -- 172.21.15.66:0/390588456 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc0cad70 con 0x7f3fcc0cb570 2024-03-30T00:44:26.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fd1400640 1 --2- 172.21.15.66:0/390588456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0cb570 0x7f3fcc0cba20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009e50 tx=0x7f3fc401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2150a577e973a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:26.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fcb400640 1 -- 172.21.15.66:0/390588456 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40af400 con 0x7f3fcc0cb570 2024-03-30T00:44:26.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fcb400640 1 -- 172.21.15.66:0/390588456 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc40ab050 con 0x7f3fcc0cb570 2024-03-30T00:44:26.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.399+0000 7f3fcb400640 1 -- 172.21.15.66:0/390588456 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40af780 con 0x7f3fcc0cb570 2024-03-30T00:44:26.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 -- 172.21.15.66:0/390588456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0cb570 msgr2=0x7f3fcc0cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 --2- 172.21.15.66:0/390588456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0cb570 0x7f3fcc0cba20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009e50 tx=0x7f3fc401f210 comp rx=0 tx=0).stop 2024-03-30T00:44:26.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 -- 172.21.15.66:0/390588456 shutdown_connections 2024-03-30T00:44:26.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 --2- 172.21.15.66:0/390588456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0cb570 0x7f3fcc0cba20 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 --2- 172.21.15.66:0/390588456 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fcc0cf2f0 0x7f3fcc06b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 -- 172.21.15.66:0/390588456 >> 172.21.15.66:0/390588456 conn(0x7f3fcc0b2090 msgr2=0x7f3fcc0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:26.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 -- 172.21.15.66:0/390588456 shutdown_connections 2024-03-30T00:44:26.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 -- 172.21.15.66:0/390588456 wait complete. 2024-03-30T00:44:26.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 Processor -- start 2024-03-30T00:44:26.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 -- start start 2024-03-30T00:44:26.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fcc0cb570 0x7f3fcc0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fcc0cf2f0 0x7f3fcc15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc15ca80 0x7f3fcc09bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc09b730 con 0x7f3fcc15ca80 2024-03-30T00:44:26.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0c99d0 con 0x7f3fcc0cf2f0 2024-03-30T00:44:26.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd3400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0b1df0 con 0x7f3fcc0cb570 2024-03-30T00:44:26.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc15ca80 0x7f3fcc09bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc15ca80 0x7f3fcc09bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53428/0 (socket says 172.21.15.66:53428) 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd1400640 1 -- 172.21.15.66:0/2879974571 learned_addr learned my addr 172.21.15.66:0/2879974571 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fcbe00640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fcc0cf2f0 0x7f3fcc15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd0a00640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fcc0cb570 0x7f3fcc0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd1400640 1 -- 172.21.15.66:0/2879974571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fcc0cb570 msgr2=0x7f3fcc0b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd1400640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fcc0cb570 0x7f3fcc0b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd1400640 1 -- 172.21.15.66:0/2879974571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fcc0cf2f0 msgr2=0x7f3fcc15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd1400640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fcc0cf2f0 0x7f3fcc15adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.403+0000 7f3fd1400640 1 -- 172.21.15.66:0/2879974571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc4009b30 con 0x7f3fcc15ca80 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.407+0000 7f3fd1400640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc15ca80 0x7f3fcc09bd10 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40b2040 tx=0x7f3fc401bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.407+0000 7f3fc8c00640 1 -- 172.21.15.66:0/2879974571 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40afd30 con 0x7f3fcc15ca80 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.407+0000 7f3fc8c00640 1 -- 172.21.15.66:0/2879974571 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc4009510 con 0x7f3fcc15ca80 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.407+0000 7f3fc8c00640 1 -- 172.21.15.66:0/2879974571 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40ba660 con 0x7f3fcc15ca80 2024-03-30T00:44:26.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.407+0000 7f3fd3400640 1 -- 172.21.15.66:0/2879974571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc0c9c20 con 0x7f3fcc15ca80 2024-03-30T00:44:26.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.407+0000 7f3fd3400640 1 -- 172.21.15.66:0/2879974571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc09b3c0 con 0x7f3fcc15ca80 2024-03-30T00:44:26.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.407+0000 7f3fd3400640 1 -- 172.21.15.66:0/2879974571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fcc09b8e0 con 0x7f3fcc15ca80 2024-03-30T00:44:26.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.407+0000 7f3fc8c00640 1 -- 172.21.15.66:0/2879974571 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3fc40b6040 con 0x7f3fcc15ca80 2024-03-30T00:44:26.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.407+0000 7f3fc8c00640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3fac0666c0 0x7f3fac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.407+0000 7f3fc8c00640 1 -- 172.21.15.66:0/2879974571 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3fc4122460 con 0x7f3fcc15ca80 2024-03-30T00:44:26.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.407+0000 7f3fd0a00640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3fac0666c0 0x7f3fac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.411+0000 7f3fd0a00640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3fac0666c0 0x7f3fac068b80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0000f70 tx=0x7f3fc0007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:26.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.411+0000 7f3fc8c00640 1 -- 172.21.15.66:0/2879974571 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3fcc09b8e0 con 0x7f3fcc15ca80 2024-03-30T00:44:26.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 -- 172.21.15.66:0/2879974571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fcc09b8e0 con 0x7f3fcc15ca80 2024-03-30T00:44:26.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fc8c00640 1 -- 172.21.15.66:0/2879974571 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16309 (secure 0 0 0) 0x7f3fcc09b8e0 con 0x7f3fcc15ca80 2024-03-30T00:44:26.582 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:26.582 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":82,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:17.276711+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:44:14.685880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-30T00:44:15.260373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 -- 172.21.15.66:0/2879974571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3fac0666c0 msgr2=0x7f3fac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3fac0666c0 0x7f3fac068b80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0000f70 tx=0x7f3fc0007cb0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 -- 172.21.15.66:0/2879974571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc15ca80 msgr2=0x7f3fcc09bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc15ca80 0x7f3fcc09bd10 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40b2040 tx=0x7f3fc401bfb0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 -- 172.21.15.66:0/2879974571 shutdown_connections 2024-03-30T00:44:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3fac0666c0 0x7f3fac068b80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc15ca80 0x7f3fcc09bd10 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fcc0cf2f0 0x7f3fcc15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 --2- 172.21.15.66:0/2879974571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fcc0cb570 0x7f3fcc0b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 -- 172.21.15.66:0/2879974571 >> 172.21.15.66:0/2879974571 conn(0x7f3fcc0b2090 msgr2=0x7f3fcc06ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 -- 172.21.15.66:0/2879974571 shutdown_connections 2024-03-30T00:44:26.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.575+0000 7f3fd3400640 1 -- 172.21.15.66:0/2879974571 wait complete. 2024-03-30T00:44:26.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:44:14.685880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:44:15.260373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:44:26.605 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:44:26.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.739+0000 7f16d7800640 1 Processor -- start 2024-03-30T00:44:26.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d7800640 1 -- start start 2024-03-30T00:44:26.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d7800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f16d0160bc0 0x7f16d0167840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d7800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d006da20 con 0x7f16d01610c0 2024-03-30T00:44:26.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d7800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d006dc60 con 0x7f16d0157060 2024-03-30T00:44:26.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d7800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d0157440 con 0x7f16d0160bc0 2024-03-30T00:44:26.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16c7e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f16d01610c0 0x7f16d01604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55136/0 (socket says 172.21.15.66:55136) 2024-03-30T00:44:26.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16c7e00640 1 -- 172.21.15.66:0/2770639662 learned_addr learned my addr 172.21.15.66:0/2770639662 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:26.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d5800640 1 --2- 172.21.15.66:0/2770639662 >> v2:172.21.15.156:3300/0 conn(0x7f16d0160bc0 0x7f16d0167840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520657041 0 0) 0x7f16b8001120 con 0x7f16d01610c0 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16b4003530 con 0x7f16d01610c0 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1488654971 0 0) 0x7f16b4003530 con 0x7f16d01610c0 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 >> v2:172.21.15.156:3300/0 conn(0x7f16d0160bc0 msgr2=0x7f16d0167840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 --2- 172.21.15.66:0/2770639662 >> v2:172.21.15.156:3300/0 conn(0x7f16d0160bc0 0x7f16d0167840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 >> v1:172.21.15.156:6789/0 conn(0x7f16d0157060 legacy=0x7f16d006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d006df40 con 0x7f16d01610c0 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f16b8001810 con 0x7f16d01610c0 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 >> v1:172.21.15.66:6789/0 conn(0x7f16d01610c0 legacy=0x7f16d01604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 --2- 172.21.15.66:0/2770639662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16b40039d0 0x7f16b4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 --2- 172.21.15.66:0/2770639662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16b40047d0 0x7f16b4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 --2- 172.21.15.66:0/2770639662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b40052b0 0x7f16b400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d0157440 con 0x7f16b40052b0 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b400c1b0 con 0x7f16b40047d0 2024-03-30T00:44:26.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b400c370 con 0x7f16b40039d0 2024-03-30T00:44:26.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d5800640 1 --2- 172.21.15.66:0/2770639662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b40052b0 0x7f16b400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d4e00640 1 --2- 172.21.15.66:0/2770639662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16b40039d0 0x7f16b4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16c7e00640 1 --2- 172.21.15.66:0/2770639662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16b40047d0 0x7f16b4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d5800640 1 -- 172.21.15.66:0/2770639662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16b40039d0 msgr2=0x7f16b4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:26.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d5800640 1 --2- 172.21.15.66:0/2770639662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16b40039d0 0x7f16b4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d5800640 1 -- 172.21.15.66:0/2770639662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16b40047d0 msgr2=0x7f16b4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:26.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d5800640 1 --2- 172.21.15.66:0/2770639662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16b40047d0 0x7f16b4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.743+0000 7f16d5800640 1 -- 172.21.15.66:0/2770639662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b4003530 con 0x7f16b40052b0 2024-03-30T00:44:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16d5800640 1 -- 172.21.15.66:0/2770639662 reap_dead start 2024-03-30T00:44:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16d5800640 1 --2- 172.21.15.66:0/2770639662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b40052b0 0x7f16b400b990 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f16d00726e0 tx=0x7f16c801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d027fe08a2a2dfc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8003ba0 con 0x7f16b40052b0 2024-03-30T00:44:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c800f040 con 0x7f16b40052b0 2024-03-30T00:44:26.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16cfe00640 1 -- 172.21.15.66:0/2770639662 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c80a5940 con 0x7f16b40052b0 2024-03-30T00:44:26.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16d7800640 1 -- 172.21.15.66:0/2770639662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b40052b0 msgr2=0x7f16b400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16d7800640 1 --2- 172.21.15.66:0/2770639662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b40052b0 0x7f16b400b990 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f16d00726e0 tx=0x7f16c801cc60 comp rx=0 tx=0).stop 2024-03-30T00:44:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16d7800640 1 -- 172.21.15.66:0/2770639662 shutdown_connections 2024-03-30T00:44:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16d7800640 1 --2- 172.21.15.66:0/2770639662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16b40052b0 0x7f16b400b990 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16d7800640 1 -- 172.21.15.66:0/2770639662 >> 172.21.15.66:0/2770639662 conn(0x7f16d0071750 msgr2=0x7f16d0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16d7800640 1 -- 172.21.15.66:0/2770639662 shutdown_connections 2024-03-30T00:44:26.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.747+0000 7f16d7800640 1 -- 172.21.15.66:0/2770639662 wait complete. 2024-03-30T00:44:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d7800640 1 Processor -- start 2024-03-30T00:44:26.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d7800640 1 -- start start 2024-03-30T00:44:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d7800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16b40052b0 0x7f16d0149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d7800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16d014e980 0x7f16d014bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d7800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16d014ede0 0x7f16d014c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d7800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d0149cc0 con 0x7f16d014e980 2024-03-30T00:44:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d7800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d0153830 con 0x7f16d014ede0 2024-03-30T00:44:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d7800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d0153d30 con 0x7f16b40052b0 2024-03-30T00:44:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16c7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16d014e980 0x7f16d014bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16c7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16d014e980 0x7f16d014bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53446/0 (socket says 172.21.15.66:53446) 2024-03-30T00:44:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16c7e00640 1 -- 172.21.15.66:0/2486288119 learned_addr learned my addr 172.21.15.66:0/2486288119 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d4e00640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16b40052b0 0x7f16d0149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d5800640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16d014ede0 0x7f16d014c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16c7e00640 1 -- 172.21.15.66:0/2486288119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16b40052b0 msgr2=0x7f16d0149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16c7e00640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16b40052b0 0x7f16d0149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16c7e00640 1 -- 172.21.15.66:0/2486288119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16d014ede0 msgr2=0x7f16d014c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16c7e00640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16d014ede0 0x7f16d014c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16c7e00640 1 -- 172.21.15.66:0/2486288119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d0154010 con 0x7f16d014e980 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16c7e00640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16d014e980 0x7f16d014bfc0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f16b80072c0 tx=0x7f16b8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16cd600640 1 -- 172.21.15.66:0/2486288119 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8001780 con 0x7f16d014e980 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16cd600640 1 -- 172.21.15.66:0/2486288119 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b800f9c0 con 0x7f16d014e980 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d7800640 1 -- 172.21.15.66:0/2486288119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d01542f0 con 0x7f16d014e980 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16cd600640 1 -- 172.21.15.66:0/2486288119 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800fd70 con 0x7f16d014e980 2024-03-30T00:44:26.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.751+0000 7f16d7800640 1 -- 172.21.15.66:0/2486288119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d01548b0 con 0x7f16d014e980 2024-03-30T00:44:26.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.755+0000 7f16d7800640 1 -- 172.21.15.66:0/2486288119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d0154a70 con 0x7f16d014e980 2024-03-30T00:44:26.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.755+0000 7f16cd600640 1 -- 172.21.15.66:0/2486288119 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f16b801d460 con 0x7f16d014e980 2024-03-30T00:44:26.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.755+0000 7f16cd600640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f16ac066690 0x7f16ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.755+0000 7f16cd600640 1 -- 172.21.15.66:0/2486288119 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f16b80841c0 con 0x7f16d014e980 2024-03-30T00:44:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.755+0000 7f16d4e00640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f16ac066690 0x7f16ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.755+0000 7f16d4e00640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f16ac066690 0x7f16ac068b50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f16c0001100 tx=0x7f16c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.759+0000 7f16cd600640 1 -- 172.21.15.66:0/2486288119 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f16b8051400 con 0x7f16d014e980 2024-03-30T00:44:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.919+0000 7f16d7800640 1 -- 172.21.15.66:0/2486288119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16d006da20 con 0x7f16d014e980 2024-03-30T00:44:26.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.919+0000 7f16cd600640 1 -- 172.21.15.66:0/2486288119 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16d006da20 con 0x7f16d014e980 2024-03-30T00:44:26.928 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:44:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 -- 172.21.15.66:0/2486288119 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f16ac066690 msgr2=0x7f16ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f16ac066690 0x7f16ac068b50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f16c0001100 tx=0x7f16c0009040 comp rx=0 tx=0).stop 2024-03-30T00:44:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 -- 172.21.15.66:0/2486288119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16d014e980 msgr2=0x7f16d014bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:26.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16d014e980 0x7f16d014bfc0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f16b80072c0 tx=0x7f16b8007780 comp rx=0 tx=0).stop 2024-03-30T00:44:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 -- 172.21.15.66:0/2486288119 shutdown_connections 2024-03-30T00:44:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16d014ede0 0x7f16d014c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f16ac066690 0x7f16ac068b50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16d014e980 0x7f16d014bfc0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 --2- 172.21.15.66:0/2486288119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16b40052b0 0x7f16d0149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 -- 172.21.15.66:0/2486288119 >> 172.21.15.66:0/2486288119 conn(0x7f16d0071750 msgr2=0x7f16d015bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 -- 172.21.15.66:0/2486288119 shutdown_connections 2024-03-30T00:44:26.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:26.923+0000 7f16d7800640 1 -- 172.21.15.66:0/2486288119 wait complete. 2024-03-30T00:44:26.949 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:44:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.083+0000 7f0255a00640 1 Processor -- start 2024-03-30T00:44:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.083+0000 7f0255a00640 1 -- start start 2024-03-30T00:44:27.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f0255a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f025015a640 0x7f025006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f0255a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f02500cabb0 0x7f02500cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f0255a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025006d070 con 0x7f02500ce520 2024-03-30T00:44:27.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f0255a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02500cbba0 con 0x7f02500cabb0 2024-03-30T00:44:27.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f0255a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025006e2c0 con 0x7f025015a640 2024-03-30T00:44:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f024ea00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f025015a640 0x7f025006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f024f400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f02500cabb0 0x7f02500cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f024ea00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f025015a640 0x7f025006dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:52740/0 (socket says 172.21.15.66:52740) 2024-03-30T00:44:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f024ea00640 1 -- 172.21.15.66:0/3404851872 learned_addr learned my addr 172.21.15.66:0/3404851872 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f024f400640 1 --2- 172.21.15.66:0/3404851872 >> v2:172.21.15.156:3300/0 conn(0x7f02500cabb0 0x7f02500cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:42534/0 (socket says 172.21.15.66:42534) 2024-03-30T00:44:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f024ea00640 1 -- 172.21.15.66:0/3404851872 >> v2:172.21.15.156:3300/0 conn(0x7f02500cabb0 msgr2=0x7f02500cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f024ea00640 1 --2- 172.21.15.66:0/3404851872 >> v2:172.21.15.156:3300/0 conn(0x7f02500cabb0 0x7f02500cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f024ea00640 1 -- 172.21.15.66:0/3404851872 >> v1:172.21.15.156:6789/0 conn(0x7f02500ce520 legacy=0x7f02500ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f024ea00640 1 -- 172.21.15.66:0/3404851872 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02500cb8c0 con 0x7f025015a640 2024-03-30T00:44:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.087+0000 7f024f400640 1 --2- 172.21.15.66:0/3404851872 >> v2:172.21.15.156:3300/0 conn(0x7f02500cabb0 0x7f02500cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f024ea00640 1 --2- 172.21.15.66:0/3404851872 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f025015a640 0x7f025006dd80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0240009870 tx=0x7f024001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b0f1e51cee60a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f024e000640 1 -- 172.21.15.66:0/3404851872 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400a95f0 con 0x7f025015a640 2024-03-30T00:44:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f024e000640 1 -- 172.21.15.66:0/3404851872 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400a9750 con 0x7f025015a640 2024-03-30T00:44:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f024e000640 1 -- 172.21.15.66:0/3404851872 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400a9ae0 con 0x7f025015a640 2024-03-30T00:44:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f0255a00640 1 -- 172.21.15.66:0/3404851872 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f025015a640 msgr2=0x7f025006dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f0255a00640 1 --2- 172.21.15.66:0/3404851872 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f025015a640 0x7f025006dd80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0240009870 tx=0x7f024001c530 comp rx=0 tx=0).stop 2024-03-30T00:44:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f0255a00640 1 -- 172.21.15.66:0/3404851872 shutdown_connections 2024-03-30T00:44:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f0255a00640 1 --2- 172.21.15.66:0/3404851872 >> v2:172.21.15.156:3300/0 conn(0x7f02500cabb0 0x7f02500cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f0255a00640 1 --2- 172.21.15.66:0/3404851872 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f025015a640 0x7f025006dd80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f0255a00640 1 -- 172.21.15.66:0/3404851872 >> 172.21.15.66:0/3404851872 conn(0x7f0250069f60 msgr2=0x7f025006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f0255a00640 1 -- 172.21.15.66:0/3404851872 shutdown_connections 2024-03-30T00:44:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f0255a00640 1 -- 172.21.15.66:0/3404851872 wait complete. 2024-03-30T00:44:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f0255a00640 1 Processor -- start 2024-03-30T00:44:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.091+0000 7f0255a00640 1 -- start start 2024-03-30T00:44:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0255a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02500cabb0 0x7f02500972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0255a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02500ce520 0x7f02500cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0255a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025015a640 0x7f02500cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0255a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02500ca0e0 con 0x7f025015a640 2024-03-30T00:44:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0255a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0250097820 con 0x7f02500cabb0 2024-03-30T00:44:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0255a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02500c9460 con 0x7f02500ce520 2024-03-30T00:44:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f024f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025015a640 0x7f02500cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f024f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025015a640 0x7f02500cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53454/0 (socket says 172.21.15.66:53454) 2024-03-30T00:44:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f024f400640 1 -- 172.21.15.66:0/1554948803 learned_addr learned my addr 172.21.15.66:0/1554948803 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f024ea00640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02500cabb0 0x7f02500972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0247e00640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02500ce520 0x7f02500cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f024f400640 1 -- 172.21.15.66:0/1554948803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02500ce520 msgr2=0x7f02500cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f024f400640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02500ce520 0x7f02500cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f024f400640 1 -- 172.21.15.66:0/1554948803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02500cabb0 msgr2=0x7f02500972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f024f400640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02500cabb0 0x7f02500972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f024f400640 1 -- 172.21.15.66:0/1554948803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0248009a70 con 0x7f025015a640 2024-03-30T00:44:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f024f400640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025015a640 0x7f02500cc400 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f024800efc0 tx=0x7f0248009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0246a00640 1 -- 172.21.15.66:0/1554948803 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248009d70 con 0x7f025015a640 2024-03-30T00:44:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0246a00640 1 -- 172.21.15.66:0/1554948803 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024800f040 con 0x7f025015a640 2024-03-30T00:44:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0246a00640 1 -- 172.21.15.66:0/1554948803 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480039a0 con 0x7f025015a640 2024-03-30T00:44:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0255a00640 1 -- 172.21.15.66:0/1554948803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02400094d0 con 0x7f025015a640 2024-03-30T00:44:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0255a00640 1 -- 172.21.15.66:0/1554948803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02500c9a20 con 0x7f025015a640 2024-03-30T00:44:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0255a00640 1 -- 172.21.15.66:0/1554948803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0250072c00 con 0x7f025015a640 2024-03-30T00:44:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.095+0000 7f0246a00640 1 -- 172.21.15.66:0/1554948803 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f02480144a0 con 0x7f025015a640 2024-03-30T00:44:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.099+0000 7f0246a00640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f022c066690 0x7f022c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.099+0000 7f0246a00640 1 -- 172.21.15.66:0/1554948803 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0248084d60 con 0x7f025015a640 2024-03-30T00:44:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.099+0000 7f024ea00640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f022c066690 0x7f022c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.099+0000 7f024ea00640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f022c066690 0x7f022c068b50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f02400ae000 tx=0x7f0240009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:27.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.103+0000 7f0246a00640 1 -- 172.21.15.66:0/1554948803 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f02480520d0 con 0x7f025015a640 2024-03-30T00:44:27.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.259+0000 7f0255a00640 1 -- 172.21.15.66:0/1554948803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f02500ca220 con 0x7f025015a640 2024-03-30T00:44:27.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.259+0000 7f0246a00640 1 -- 172.21.15.66:0/1554948803 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0248018020 con 0x7f025015a640 2024-03-30T00:44:27.268 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:44:27.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 -- 172.21.15.66:0/1554948803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f022c066690 msgr2=0x7f022c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f022c066690 0x7f022c068b50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f02400ae000 tx=0x7f0240009040 comp rx=0 tx=0).stop 2024-03-30T00:44:27.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 -- 172.21.15.66:0/1554948803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025015a640 msgr2=0x7f02500cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025015a640 0x7f02500cc400 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f024800efc0 tx=0x7f0248009f90 comp rx=0 tx=0).stop 2024-03-30T00:44:27.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 -- 172.21.15.66:0/1554948803 shutdown_connections 2024-03-30T00:44:27.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f022c066690 0x7f022c068b50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025015a640 0x7f02500cc400 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02500ce520 0x7f02500cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 --2- 172.21.15.66:0/1554948803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02500cabb0 0x7f02500972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 -- 172.21.15.66:0/1554948803 >> 172.21.15.66:0/1554948803 conn(0x7f0250069f60 msgr2=0x7f02500afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:27.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 -- 172.21.15.66:0/1554948803 shutdown_connections 2024-03-30T00:44:27.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.263+0000 7f0255a00640 1 -- 172.21.15.66:0/1554948803 wait complete. 2024-03-30T00:44:27.289 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:44:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd02f400640 1 Processor -- start 2024-03-30T00:44:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd02f400640 1 -- start start 2024-03-30T00:44:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd02f400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd02815ca80 0x7fd02806b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd02f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd0280cf2f0 0x7fd02806b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd02f400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0280cb0e0 con 0x7fd02806bb30 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd02f400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0280cf7f0 con 0x7fd0280cf2f0 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd02f400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0280cfad0 con 0x7fd02815ca80 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd01fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd0280cf2f0 0x7fd02806b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd01fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd0280cf2f0 0x7fd02806b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53460/0 (socket says 172.21.15.66:53460) 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd01fe00640 1 -- 172.21.15.66:0/1195088696 learned_addr learned my addr 172.21.15.66:0/1195088696 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd02d400640 1 --1- 172.21.15.66:0/1195088696 >> v1:172.21.15.66:6789/0 conn(0x7fd02806bb30 0x7fd028098a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55148/0 (socket says 172.21.15.66:55148) 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd02ca00640 1 --2- 172.21.15.66:0/1195088696 >> v2:172.21.15.156:3300/0 conn(0x7fd02815ca80 0x7fd02806b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd01fe00640 1 -- 172.21.15.66:0/1195088696 >> v2:172.21.15.156:3300/0 conn(0x7fd02815ca80 msgr2=0x7fd02806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd01fe00640 1 --2- 172.21.15.66:0/1195088696 >> v2:172.21.15.156:3300/0 conn(0x7fd02815ca80 0x7fd02806b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd01fe00640 1 -- 172.21.15.66:0/1195088696 >> v1:172.21.15.66:6789/0 conn(0x7fd02806bb30 legacy=0x7fd028098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd01fe00640 1 -- 172.21.15.66:0/1195088696 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0280cad70 con 0x7fd0280cf2f0 2024-03-30T00:44:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.427+0000 7fd027e00640 1 -- 172.21.15.66:0/1195088696 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139667394 0 0) 0x7fd0280cb0e0 con 0x7fd02806bb30 2024-03-30T00:44:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd01fe00640 1 --2- 172.21.15.66:0/1195088696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0280cf2f0 0x7fd02806b5f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd028070ed0 tx=0x7fd01001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c564675771be7e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd027e00640 1 -- 172.21.15.66:0/1195088696 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0100a8a10 con 0x7fd0280cf2f0 2024-03-30T00:44:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd027e00640 1 -- 172.21.15.66:0/1195088696 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0100a8b70 con 0x7fd0280cf2f0 2024-03-30T00:44:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd027e00640 1 -- 172.21.15.66:0/1195088696 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0100b0440 con 0x7fd0280cf2f0 2024-03-30T00:44:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd02f400640 1 -- 172.21.15.66:0/1195088696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0280cf2f0 msgr2=0x7fd02806b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd02f400640 1 --2- 172.21.15.66:0/1195088696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0280cf2f0 0x7fd02806b5f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd028070ed0 tx=0x7fd01001ed10 comp rx=0 tx=0).stop 2024-03-30T00:44:27.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd02f400640 1 -- 172.21.15.66:0/1195088696 shutdown_connections 2024-03-30T00:44:27.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd02f400640 1 --2- 172.21.15.66:0/1195088696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0280cf2f0 0x7fd02806b5f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd02f400640 1 --2- 172.21.15.66:0/1195088696 >> v2:172.21.15.156:3300/0 conn(0x7fd02815ca80 0x7fd02806b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd02f400640 1 -- 172.21.15.66:0/1195088696 >> 172.21.15.66:0/1195088696 conn(0x7fd0280b2090 msgr2=0x7fd0280b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:27.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd02f400640 1 -- 172.21.15.66:0/1195088696 shutdown_connections 2024-03-30T00:44:27.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.431+0000 7fd02f400640 1 -- 172.21.15.66:0/1195088696 wait complete. 2024-03-30T00:44:27.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02f400640 1 Processor -- start 2024-03-30T00:44:27.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02f400640 1 -- start start 2024-03-30T00:44:27.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd02806bb30 0x7fd028158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0280cf2f0 0x7fd0280b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd02815ca80 0x7fd0280c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02815d1a0 con 0x7fd0280cf2f0 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02f400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02815d480 con 0x7fd02815ca80 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02815d760 con 0x7fd02806bb30 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd01fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0280cf2f0 0x7fd0280b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd01fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0280cf2f0 0x7fd0280b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53476/0 (socket says 172.21.15.66:53476) 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd01fe00640 1 -- 172.21.15.66:0/3514719696 learned_addr learned my addr 172.21.15.66:0/3514719696 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02ca00640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd02806bb30 0x7fd028158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02d400640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd02815ca80 0x7fd0280c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd01fe00640 1 -- 172.21.15.66:0/3514719696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd02806bb30 msgr2=0x7fd028158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd01fe00640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd02806bb30 0x7fd028158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd01fe00640 1 -- 172.21.15.66:0/3514719696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd02815ca80 msgr2=0x7fd0280c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd01fe00640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd02815ca80 0x7fd0280c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd01fe00640 1 -- 172.21.15.66:0/3514719696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0100094d0 con 0x7fd0280cf2f0 2024-03-30T00:44:27.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd01fe00640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0280cf2f0 0x7fd0280b3350 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd0100b3040 tx=0x7fd01001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:27.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd025600640 1 -- 172.21.15.66:0/3514719696 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0100b0e80 con 0x7fd0280cf2f0 2024-03-30T00:44:27.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd025600640 1 -- 172.21.15.66:0/3514719696 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0100bb480 con 0x7fd0280cf2f0 2024-03-30T00:44:27.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd025600640 1 -- 172.21.15.66:0/3514719696 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0100bb810 con 0x7fd0280cf2f0 2024-03-30T00:44:27.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02f400640 1 -- 172.21.15.66:0/3514719696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02815da40 con 0x7fd0280cf2f0 2024-03-30T00:44:27.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02f400640 1 -- 172.21.15.66:0/3514719696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd028158fe0 con 0x7fd0280cf2f0 2024-03-30T00:44:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.435+0000 7fd02f400640 1 -- 172.21.15.66:0/3514719696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd028066c30 con 0x7fd0280cf2f0 2024-03-30T00:44:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.439+0000 7fd025600640 1 -- 172.21.15.66:0/3514719696 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd0100bb970 con 0x7fd0280cf2f0 2024-03-30T00:44:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.439+0000 7fd025600640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd004066690 0x7fd004068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.439+0000 7fd025600640 1 -- 172.21.15.66:0/3514719696 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd010122860 con 0x7fd0280cf2f0 2024-03-30T00:44:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.439+0000 7fd02ca00640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd004066690 0x7fd004068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.439+0000 7fd02ca00640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd004066690 0x7fd004068b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd0180079c0 tx=0x7fd018008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:27.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.443+0000 7fd025600640 1 -- 172.21.15.66:0/3514719696 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd0100efa80 con 0x7fd0280cf2f0 2024-03-30T00:44:27.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.603+0000 7fd02f400640 1 -- 172.21.15.66:0/3514719696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd02815def0 con 0x7fd0280cf2f0 2024-03-30T00:44:27.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.603+0000 7fd025600640 1 -- 172.21.15.66:0/3514719696 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0100f30a0 con 0x7fd0280cf2f0 2024-03-30T00:44:27.612 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:44:27.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 -- 172.21.15.66:0/3514719696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd004066690 msgr2=0x7fd004068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd004066690 0x7fd004068b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd0180079c0 tx=0x7fd018008040 comp rx=0 tx=0).stop 2024-03-30T00:44:27.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 -- 172.21.15.66:0/3514719696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0280cf2f0 msgr2=0x7fd0280b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0280cf2f0 0x7fd0280b3350 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd0100b3040 tx=0x7fd01001bab0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 -- 172.21.15.66:0/3514719696 shutdown_connections 2024-03-30T00:44:27.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd02815ca80 0x7fd0280c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd004066690 0x7fd004068b50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0280cf2f0 0x7fd0280b3350 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 --2- 172.21.15.66:0/3514719696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd02806bb30 0x7fd028158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 -- 172.21.15.66:0/3514719696 >> 172.21.15.66:0/3514719696 conn(0x7fd0280b2090 msgr2=0x7fd0280c6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:27.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 -- 172.21.15.66:0/3514719696 shutdown_connections 2024-03-30T00:44:27.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.607+0000 7fd02f400640 1 -- 172.21.15.66:0/3514719696 wait complete. 2024-03-30T00:44:27.633 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:44:27.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f44200640 1 Processor -- start 2024-03-30T00:44:27.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f44200640 1 -- start start 2024-03-30T00:44:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f44200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c15ca80 0x7f8f3c06b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f44200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8f3c0cf2f0 0x7f8f3c06b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f44200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c0cb0e0 con 0x7f8f3c06bb30 2024-03-30T00:44:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f44200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3c0cf7f0 con 0x7f8f3c0cf2f0 2024-03-30T00:44:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f44200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0cfad0 con 0x7f8f3c15ca80 2024-03-30T00:44:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f41800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c15ca80 0x7f8f3c06b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f40e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8f3c0cf2f0 0x7f8f3c06b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f42200640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f8f3c06bb30 0x7f8f3c098a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:59326/0 (socket says 172.21.15.66:59326) 2024-03-30T00:44:27.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f42200640 1 -- 172.21.15.66:0/2288683584 learned_addr learned my addr 172.21.15.66:0/2288683584 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:27.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f40e00640 1 -- 172.21.15.66:0/2288683584 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c15ca80 msgr2=0x7f8f3c06b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:27.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f40e00640 1 --2- 172.21.15.66:0/2288683584 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c15ca80 0x7f8f3c06b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.767+0000 7f8f40e00640 1 -- 172.21.15.66:0/2288683584 >> v1:172.21.15.156:6789/0 conn(0x7f8f3c06bb30 legacy=0x7f8f3c098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f40e00640 1 -- 172.21.15.66:0/2288683584 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c0cad70 con 0x7f8f3c0cf2f0 2024-03-30T00:44:27.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f41800640 1 --2- 172.21.15.66:0/2288683584 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c15ca80 0x7f8f3c06b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:27.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f40e00640 1 --2- 172.21.15.66:0/2288683584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c0cf2f0 0x7f8f3c06b5f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8f280097c0 tx=0x7f8f2801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d21e35e03269bc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:27.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f3be00640 1 -- 172.21.15.66:0/2288683584 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f280a8990 con 0x7f8f3c0cf2f0 2024-03-30T00:44:27.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f3be00640 1 -- 172.21.15.66:0/2288683584 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f280a8af0 con 0x7f8f3c0cf2f0 2024-03-30T00:44:27.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f3be00640 1 -- 172.21.15.66:0/2288683584 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f280a8e80 con 0x7f8f3c0cf2f0 2024-03-30T00:44:27.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f44200640 1 -- 172.21.15.66:0/2288683584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c0cf2f0 msgr2=0x7f8f3c06b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f44200640 1 --2- 172.21.15.66:0/2288683584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c0cf2f0 0x7f8f3c06b5f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8f280097c0 tx=0x7f8f2801ec90 comp rx=0 tx=0).stop 2024-03-30T00:44:27.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f44200640 1 -- 172.21.15.66:0/2288683584 shutdown_connections 2024-03-30T00:44:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f44200640 1 --2- 172.21.15.66:0/2288683584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c0cf2f0 0x7f8f3c06b5f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f44200640 1 --2- 172.21.15.66:0/2288683584 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c15ca80 0x7f8f3c06b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f44200640 1 -- 172.21.15.66:0/2288683584 >> 172.21.15.66:0/2288683584 conn(0x7f8f3c0b2090 msgr2=0x7f8f3c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f44200640 1 -- 172.21.15.66:0/2288683584 shutdown_connections 2024-03-30T00:44:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.771+0000 7f8f44200640 1 -- 172.21.15.66:0/2288683584 wait complete. 2024-03-30T00:44:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f44200640 1 Processor -- start 2024-03-30T00:44:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f44200640 1 -- start start 2024-03-30T00:44:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f44200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f3c06bb30 0x7f8f3c149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f44200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c0cf2f0 0x7f8f3c09be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f44200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c15ca80 0x7f8f3c09a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f44200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c09c3b0 con 0x7f8f3c06bb30 2024-03-30T00:44:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f44200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c09a850 con 0x7f8f3c15ca80 2024-03-30T00:44:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f44200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c146ae0 con 0x7f8f3c0cf2f0 2024-03-30T00:44:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f41800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f3c06bb30 0x7f8f3c149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f41800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f3c06bb30 0x7f8f3c149710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53488/0 (socket says 172.21.15.66:53488) 2024-03-30T00:44:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f40e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c0cf2f0 0x7f8f3c09be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f41800640 1 -- 172.21.15.66:0/628628302 learned_addr learned my addr 172.21.15.66:0/628628302 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f42200640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c15ca80 0x7f8f3c09a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f41800640 1 -- 172.21.15.66:0/628628302 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c0cf2f0 msgr2=0x7f8f3c09be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f41800640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c0cf2f0 0x7f8f3c09be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f41800640 1 -- 172.21.15.66:0/628628302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c15ca80 msgr2=0x7f8f3c09a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f41800640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c15ca80 0x7f8f3c09a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f41800640 1 -- 172.21.15.66:0/628628302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f30009420 con 0x7f8f3c06bb30 2024-03-30T00:44:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f40e00640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c0cf2f0 0x7f8f3c09be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f41800640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f3c06bb30 0x7f8f3c149710 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e980 tx=0x7f8f3000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f39600640 1 -- 172.21.15.66:0/628628302 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3000ca00 con 0x7f8f3c06bb30 2024-03-30T00:44:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f39600640 1 -- 172.21.15.66:0/628628302 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3000cb60 con 0x7f8f3c06bb30 2024-03-30T00:44:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.775+0000 7f8f44200640 1 -- 172.21.15.66:0/628628302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f28009420 con 0x7f8f3c06bb30 2024-03-30T00:44:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.779+0000 7f8f39600640 1 -- 172.21.15.66:0/628628302 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30013470 con 0x7f8f3c06bb30 2024-03-30T00:44:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.779+0000 7f8f44200640 1 -- 172.21.15.66:0/628628302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c1470a0 con 0x7f8f3c06bb30 2024-03-30T00:44:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.779+0000 7f8f44200640 1 -- 172.21.15.66:0/628628302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c066c30 con 0x7f8f3c06bb30 2024-03-30T00:44:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.779+0000 7f8f39600640 1 -- 172.21.15.66:0/628628302 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8f300135d0 con 0x7f8f3c06bb30 2024-03-30T00:44:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.779+0000 7f8f39600640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f1c066690 0x7f8f1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.779+0000 7f8f39600640 1 -- 172.21.15.66:0/628628302 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8f30084560 con 0x7f8f3c06bb30 2024-03-30T00:44:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.779+0000 7f8f40e00640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f1c066690 0x7f8f1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:27.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.779+0000 7f8f40e00640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f1c066690 0x7f8f1c068b50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8f280b2000 tx=0x7f8f28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:27.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.783+0000 7f8f39600640 1 -- 172.21.15.66:0/628628302 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8f300517a0 con 0x7f8f3c06bb30 2024-03-30T00:44:27.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.959+0000 7f8f44200640 1 -- 172.21.15.66:0/628628302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8f3c15c160 con 0x7f8f3c06bb30 2024-03-30T00:44:27.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.959+0000 7f8f39600640 1 -- 172.21.15.66:0/628628302 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v82) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8f30054dc0 con 0x7f8f3c06bb30 2024-03-30T00:44:27.967 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:44:27.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 -- 172.21.15.66:0/628628302 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f1c066690 msgr2=0x7f8f1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f1c066690 0x7f8f1c068b50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8f280b2000 tx=0x7f8f28008040 comp rx=0 tx=0).stop 2024-03-30T00:44:27.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 -- 172.21.15.66:0/628628302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f3c06bb30 msgr2=0x7f8f3c149710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:27.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f3c06bb30 0x7f8f3c149710 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e980 tx=0x7f8f3000ee40 comp rx=0 tx=0).stop 2024-03-30T00:44:27.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 -- 172.21.15.66:0/628628302 shutdown_connections 2024-03-30T00:44:27.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c15ca80 0x7f8f3c09a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c0cf2f0 0x7f8f3c09be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f1c066690 0x7f8f1c068b50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 --2- 172.21.15.66:0/628628302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f3c06bb30 0x7f8f3c149710 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:27.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 -- 172.21.15.66:0/628628302 >> 172.21.15.66:0/628628302 conn(0x7f8f3c0b2090 msgr2=0x7f8f3c06f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:27.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 -- 172.21.15.66:0/628628302 shutdown_connections 2024-03-30T00:44:27.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:27.963+0000 7f8f44200640 1 -- 172.21.15.66:0/628628302 wait complete. 2024-03-30T00:44:27.989 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:44:27.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:27.990 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:28.008 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[44555]: fuse finished with error 0 and tester_r 0 2024-03-30T00:44:28.011 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:44:34.112 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:44:34.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:34.113 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:44:34.124 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:44:34.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.315+0000 7fd40f400640 1 Processor -- start 2024-03-30T00:44:34.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.315+0000 7fd40f400640 1 -- start start 2024-03-30T00:44:34.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.315+0000 7fd40f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd4080ce520 0x7fd4080ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:34.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.315+0000 7fd40f400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40806d070 con 0x7fd40815a640 2024-03-30T00:44:34.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.315+0000 7fd40f400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4080cbba0 con 0x7fd4080cabb0 2024-03-30T00:44:34.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.315+0000 7fd40f400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4080cee70 con 0x7fd4080ce520 2024-03-30T00:44:34.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.315+0000 7fd40ca00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd40815a640 0x7fd40806dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52778/0 (socket says 172.21.15.66:52778) 2024-03-30T00:44:34.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.315+0000 7fd40ca00640 1 -- 172.21.15.66:0/3193545117 learned_addr learned my addr 172.21.15.66:0/3193545117 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:34.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.315+0000 7fd3ffe00640 1 --2- 172.21.15.66:0/3193545117 >> v2:172.21.15.66:3300/0 conn(0x7fd4080ce520 0x7fd4080ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:34.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.319+0000 7fd3ffe00640 1 -- 172.21.15.66:0/3193545117 >> v1:172.21.15.156:6789/0 conn(0x7fd4080cabb0 legacy=0x7fd4080cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:34.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.319+0000 7fd3ffe00640 1 -- 172.21.15.66:0/3193545117 >> v1:172.21.15.66:6789/0 conn(0x7fd40815a640 legacy=0x7fd40806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:34.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.319+0000 7fd3ffe00640 1 -- 172.21.15.66:0/3193545117 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4080cb8c0 con 0x7fd4080ce520 2024-03-30T00:44:34.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.319+0000 7fd3ffe00640 1 --2- 172.21.15.66:0/3193545117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4080ce520 0x7fd4080ce930 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00097c0 tx=0x7fd3f001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7570d3feaf553ef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:34.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.319+0000 7fd407e00640 1 -- 172.21.15.66:0/3193545117 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0004230 con 0x7fd4080ce520 2024-03-30T00:44:34.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.319+0000 7fd407e00640 1 -- 172.21.15.66:0/3193545117 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f00a7050 con 0x7fd4080ce520 2024-03-30T00:44:34.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.319+0000 7fd407e00640 1 -- 172.21.15.66:0/3193545117 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0003620 con 0x7fd4080ce520 2024-03-30T00:44:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.319+0000 7fd40f400640 1 -- 172.21.15.66:0/3193545117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4080ce520 msgr2=0x7fd4080ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.319+0000 7fd40f400640 1 --2- 172.21.15.66:0/3193545117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4080ce520 0x7fd4080ce930 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00097c0 tx=0x7fd3f001c460 comp rx=0 tx=0).stop 2024-03-30T00:44:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 -- 172.21.15.66:0/3193545117 shutdown_connections 2024-03-30T00:44:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 --2- 172.21.15.66:0/3193545117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4080ce520 0x7fd4080ce930 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 -- 172.21.15.66:0/3193545117 >> 172.21.15.66:0/3193545117 conn(0x7fd408069f60 msgr2=0x7fd40806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 -- 172.21.15.66:0/3193545117 shutdown_connections 2024-03-30T00:44:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 -- 172.21.15.66:0/3193545117 wait complete. 2024-03-30T00:44:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 Processor -- start 2024-03-30T00:44:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 -- start start 2024-03-30T00:44:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4080cabb0 0x7fd4080c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4080ce520 0x7fd4081f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd40815a640 0x7fd4081f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4080c46f0 con 0x7fd4080ce520 2024-03-30T00:44:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd408158e50 con 0x7fd40815a640 2024-03-30T00:44:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4081590f0 con 0x7fd4080cabb0 2024-03-30T00:44:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd3ffe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4080ce520 0x7fd4081f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd3ffe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4080ce520 0x7fd4081f6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39454/0 (socket says 172.21.15.66:39454) 2024-03-30T00:44:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd3ffe00640 1 -- 172.21.15.66:0/2688171377 learned_addr learned my addr 172.21.15.66:0/2688171377 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40d400640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd40815a640 0x7fd4081f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd40ca00640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4080cabb0 0x7fd4080c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd3ffe00640 1 -- 172.21.15.66:0/2688171377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4080cabb0 msgr2=0x7fd4080c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd3ffe00640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4080cabb0 0x7fd4080c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd3ffe00640 1 -- 172.21.15.66:0/2688171377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd40815a640 msgr2=0x7fd4081f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd3ffe00640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd40815a640 0x7fd4081f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd3ffe00640 1 -- 172.21.15.66:0/2688171377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f0009420 con 0x7fd4080ce520 2024-03-30T00:44:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd3ffe00640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4080ce520 0x7fd4081f6600 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0019070 tx=0x7fd3f0019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd405600640 1 -- 172.21.15.66:0/2688171377 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0003cf0 con 0x7fd4080ce520 2024-03-30T00:44:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.323+0000 7fd405600640 1 -- 172.21.15.66:0/2688171377 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f0003e50 con 0x7fd4080ce520 2024-03-30T00:44:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.327+0000 7fd405600640 1 -- 172.21.15.66:0/2688171377 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f00ab680 con 0x7fd4080ce520 2024-03-30T00:44:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.327+0000 7fd40f400640 1 -- 172.21.15.66:0/2688171377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd408159370 con 0x7fd4080ce520 2024-03-30T00:44:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.327+0000 7fd40f400640 1 -- 172.21.15.66:0/2688171377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd408159650 con 0x7fd4080ce520 2024-03-30T00:44:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.327+0000 7fd40f400640 1 -- 172.21.15.66:0/2688171377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4080c4860 con 0x7fd4080ce520 2024-03-30T00:44:34.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.327+0000 7fd405600640 1 -- 172.21.15.66:0/2688171377 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd3f00b2040 con 0x7fd4080ce520 2024-03-30T00:44:34.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.327+0000 7fd405600640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd3e4066690 0x7fd3e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:34.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.327+0000 7fd405600640 1 -- 172.21.15.66:0/2688171377 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd3f011e360 con 0x7fd4080ce520 2024-03-30T00:44:34.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.327+0000 7fd40ca00640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd3e4066690 0x7fd3e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:34.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.327+0000 7fd40ca00640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd3e4066690 0x7fd3e4068b50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8005890 tx=0x7fd3f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:34.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.331+0000 7fd405600640 1 -- 172.21.15.66:0/2688171377 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd3f00eb4f0 con 0x7fd4080ce520 2024-03-30T00:44:34.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.527+0000 7fd40f400640 1 -- 172.21.15.66:0/2688171377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd408066c30 con 0x7fd4080ce520 2024-03-30T00:44:34.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.527+0000 7fd405600640 1 -- 172.21.15.66:0/2688171377 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4742 (secure 0 0 0) 0x7fd3f00eeb10 con 0x7fd4080ce520 2024-03-30T00:44:34.535 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:34.536 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:44:19.285136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/2089872680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":2089872680},{"type":"v1","addr":"172.21.15.156:6835","nonce":2089872680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:44:34.536 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 43 2024-03-30T00:44:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 -- 172.21.15.66:0/2688171377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd3e4066690 msgr2=0x7fd3e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd3e4066690 0x7fd3e4068b50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8005890 tx=0x7fd3f8009040 comp rx=0 tx=0).stop 2024-03-30T00:44:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 -- 172.21.15.66:0/2688171377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4080ce520 msgr2=0x7fd4081f6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4080ce520 0x7fd4081f6600 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0019070 tx=0x7fd3f0019200 comp rx=0 tx=0).stop 2024-03-30T00:44:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 -- 172.21.15.66:0/2688171377 shutdown_connections 2024-03-30T00:44:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd40815a640 0x7fd4081f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd3e4066690 0x7fd3e4068b50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4080ce520 0x7fd4081f6600 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 --2- 172.21.15.66:0/2688171377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4080cabb0 0x7fd4080c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 -- 172.21.15.66:0/2688171377 >> 172.21.15.66:0/2688171377 conn(0x7fd408069f60 msgr2=0x7fd4081502e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:34.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 -- 172.21.15.66:0/2688171377 shutdown_connections 2024-03-30T00:44:34.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.531+0000 7fd40f400640 1 -- 172.21.15.66:0/2688171377 wait complete. 2024-03-30T00:44:34.557 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-03-30T00:44:34.729 INFO:teuthology.orchestra.run.smithi156.stdout:{"message":"","return_code":0} 2024-03-30T00:44:34.730 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-30T00:44:34.730 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:44:34.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.863+0000 7fc932600640 1 Processor -- start 2024-03-30T00:44:34.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.863+0000 7fc932600640 1 -- start start 2024-03-30T00:44:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.863+0000 7fc932600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c157060 0x7fc92c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.863+0000 7fc932600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc92c1610c0 0x7fc92c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.863+0000 7fc932600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92c06dc60 con 0x7fc92c1609f0 2024-03-30T00:44:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.863+0000 7fc932600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c157440 con 0x7fc92c1610c0 2024-03-30T00:44:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.863+0000 7fc932600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c160ea0 con 0x7fc92c157060 2024-03-30T00:44:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.863+0000 7fc92be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc92c1609f0 0x7fc92c15cec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52786/0 (socket says 172.21.15.66:52786) 2024-03-30T00:44:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.863+0000 7fc92be00640 1 -- 172.21.15.66:0/2754846213 learned_addr learned my addr 172.21.15.66:0/2754846213 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc922a00640 1 --2- 172.21.15.66:0/2754846213 >> v2:172.21.15.156:3300/0 conn(0x7fc92c1610c0 0x7fc92c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:34.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc92b400640 1 --2- 172.21.15.66:0/2754846213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c157060 0x7fc92c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:34.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc92aa00640 1 -- 172.21.15.66:0/2754846213 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130823683 0 0) 0x7fc92c06dc60 con 0x7fc92c1609f0 2024-03-30T00:44:34.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc92aa00640 1 -- 172.21.15.66:0/2754846213 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc910003530 con 0x7fc92c1609f0 2024-03-30T00:44:34.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc92b400640 1 -- 172.21.15.66:0/2754846213 >> v2:172.21.15.156:3300/0 conn(0x7fc92c1610c0 msgr2=0x7fc92c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:34.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc92b400640 1 --2- 172.21.15.66:0/2754846213 >> v2:172.21.15.156:3300/0 conn(0x7fc92c1610c0 0x7fc92c1604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc92b400640 1 -- 172.21.15.66:0/2754846213 >> v1:172.21.15.66:6789/0 conn(0x7fc92c1609f0 legacy=0x7fc92c15cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:34.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc92b400640 1 -- 172.21.15.66:0/2754846213 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c06df40 con 0x7fc92c157060 2024-03-30T00:44:34.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc922a00640 1 --2- 172.21.15.66:0/2754846213 >> v2:172.21.15.156:3300/0 conn(0x7fc92c1610c0 0x7fc92c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:34.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc92b400640 1 --2- 172.21.15.66:0/2754846213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c157060 0x7fc92c06d310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0097c0 tx=0x7fc91c01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71d9018bf486d37e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:34.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc92aa00640 1 -- 172.21.15.66:0/2754846213 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c15dde0 con 0x7fc92c157060 2024-03-30T00:44:34.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc92aa00640 1 -- 172.21.15.66:0/2754846213 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91c0a7050 con 0x7fc92c157060 2024-03-30T00:44:34.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.867+0000 7fc92aa00640 1 -- 172.21.15.66:0/2754846213 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c0ab820 con 0x7fc92c157060 2024-03-30T00:44:34.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 -- 172.21.15.66:0/2754846213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c157060 msgr2=0x7fc92c06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:34.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 --2- 172.21.15.66:0/2754846213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c157060 0x7fc92c06d310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0097c0 tx=0x7fc91c01c380 comp rx=0 tx=0).stop 2024-03-30T00:44:34.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 -- 172.21.15.66:0/2754846213 shutdown_connections 2024-03-30T00:44:34.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 --2- 172.21.15.66:0/2754846213 >> v2:172.21.15.156:3300/0 conn(0x7fc92c1610c0 0x7fc92c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 --2- 172.21.15.66:0/2754846213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c157060 0x7fc92c06d310 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 -- 172.21.15.66:0/2754846213 >> 172.21.15.66:0/2754846213 conn(0x7fc92c071750 msgr2=0x7fc92c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:34.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 -- 172.21.15.66:0/2754846213 shutdown_connections 2024-03-30T00:44:34.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 -- 172.21.15.66:0/2754846213 wait complete. 2024-03-30T00:44:34.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 Processor -- start 2024-03-30T00:44:34.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 -- start start 2024-03-30T00:44:34.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc92c157060 0x7fc92c06cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:34.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c1609f0 0x7fc92c14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:34.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c1610c0 0x7fc92c0af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:34.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0b3710 con 0x7fc92c157060 2024-03-30T00:44:34.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0b3a80 con 0x7fc92c1610c0 2024-03-30T00:44:34.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc932600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c14f6a0 con 0x7fc92c1609f0 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc92b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc92c157060 0x7fc92c06cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc92b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc92c157060 0x7fc92c06cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39466/0 (socket says 172.21.15.66:39466) 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc92b400640 1 -- 172.21.15.66:0/1751442385 learned_addr learned my addr 172.21.15.66:0/1751442385 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc92be00640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c1610c0 0x7fc92c0af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc922a00640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c1609f0 0x7fc92c14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc92b400640 1 -- 172.21.15.66:0/1751442385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c1609f0 msgr2=0x7fc92c14fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc92b400640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c1609f0 0x7fc92c14fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc92b400640 1 -- 172.21.15.66:0/1751442385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c1610c0 msgr2=0x7fc92c0af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc92b400640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c1610c0 0x7fc92c0af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.871+0000 7fc92b400640 1 -- 172.21.15.66:0/1751442385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc914009420 con 0x7fc92c157060 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc922a00640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c1609f0 0x7fc92c14fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc92b400640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc92c157060 0x7fc92c06cf60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0ae040 tx=0x7fc91c019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc923e00640 1 -- 172.21.15.66:0/1751442385 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c0b4430 con 0x7fc92c157060 2024-03-30T00:44:34.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc923e00640 1 -- 172.21.15.66:0/1751442385 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91c0b45b0 con 0x7fc92c157060 2024-03-30T00:44:34.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc923e00640 1 -- 172.21.15.66:0/1751442385 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c0b4960 con 0x7fc92c157060 2024-03-30T00:44:34.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc932600640 1 -- 172.21.15.66:0/1751442385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91c009420 con 0x7fc92c157060 2024-03-30T00:44:34.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc932600640 1 -- 172.21.15.66:0/1751442385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c0b3fb0 con 0x7fc92c157060 2024-03-30T00:44:34.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc932600640 1 -- 172.21.15.66:0/1751442385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c066c30 con 0x7fc92c157060 2024-03-30T00:44:34.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc923e00640 1 -- 172.21.15.66:0/1751442385 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc91c0b2040 con 0x7fc92c157060 2024-03-30T00:44:34.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc923e00640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc908066690 0x7fc908068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:34.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc923e00640 1 -- 172.21.15.66:0/1751442385 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc91c11e180 con 0x7fc92c157060 2024-03-30T00:44:34.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.875+0000 7fc922a00640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc908066690 0x7fc908068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:34.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.879+0000 7fc922a00640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc908066690 0x7fc908068b50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc92c15be50 tx=0x7fc914008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:34.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:34.879+0000 7fc923e00640 1 -- 172.21.15.66:0/1751442385 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc91c0eb3a0 con 0x7fc92c157060 2024-03-30T00:44:35.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.075+0000 7fc932600640 1 -- 172.21.15.66:0/1751442385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc92c071f30 con 0x7fc92c157060 2024-03-30T00:44:35.084 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:44:35.075+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:44:35.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.083+0000 7fc923e00640 1 -- 172.21.15.66:0/1751442385 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc91c0ee9c0 con 0x7fc92c157060 2024-03-30T00:44:35.092 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:44:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 -- 172.21.15.66:0/1751442385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc908066690 msgr2=0x7fc908068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc908066690 0x7fc908068b50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc92c15be50 tx=0x7fc914008040 comp rx=0 tx=0).stop 2024-03-30T00:44:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 -- 172.21.15.66:0/1751442385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc92c157060 msgr2=0x7fc92c06cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc92c157060 0x7fc92c06cf60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0ae040 tx=0x7fc91c019120 comp rx=0 tx=0).stop 2024-03-30T00:44:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 -- 172.21.15.66:0/1751442385 shutdown_connections 2024-03-30T00:44:35.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c1610c0 0x7fc92c0af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c1609f0 0x7fc92c14fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc908066690 0x7fc908068b50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 --2- 172.21.15.66:0/1751442385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc92c157060 0x7fc92c06cf60 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 -- 172.21.15.66:0/1751442385 >> 172.21.15.66:0/1751442385 conn(0x7fc92c071750 msgr2=0x7fc92c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 -- 172.21.15.66:0/1751442385 shutdown_connections 2024-03-30T00:44:35.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.087+0000 7fc932600640 1 -- 172.21.15.66:0/1751442385 wait complete. 2024-03-30T00:44:35.098 INFO:tasks.ceph.mds.b.smithi156.stderr: -3472> 2024-03-30T00:44:17.703+0000 7fe86a200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:44:35.113 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-03-30T00:44:35.146 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2024-03-30T00:44:35.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.151+0000 7f078c447080 1 Processor -- start 2024-03-30T00:44:35.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.151+0000 7f078c447080 1 -- start start 2024-03-30T00:44:35.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557016f70e40 0x557016f82080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557016f8ad00 0x557016f701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557017000100 0x557017018cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557016e88960 con 0x557017000100 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557016e88600 con 0x557016f70e40 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x557016e7f690 con 0x557016f8ad00 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557017000100 0x557017018cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557017000100 0x557017018cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39472/0 (socket says 172.21.15.66:39472) 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078a000640 1 -- 172.21.15.66:0/3539705424 learned_addr learned my addr 172.21.15.66:0/3539705424 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f0789600640 1 --2- 172.21.15.66:0/3539705424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557016f70e40 0x557016f82080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f0788c00640 1 --2- 172.21.15.66:0/3539705424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557016f8ad00 0x557016f701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078a000640 1 -- 172.21.15.66:0/3539705424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557016f8ad00 msgr2=0x557016f701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078a000640 1 --2- 172.21.15.66:0/3539705424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557016f8ad00 0x557016f701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078a000640 1 -- 172.21.15.66:0/3539705424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557016f70e40 msgr2=0x557016f82080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078a000640 1 --2- 172.21.15.66:0/3539705424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557016f70e40 0x557016f82080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078a000640 1 -- 172.21.15.66:0/3539705424 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078400a1c0 con 0x557017000100 2024-03-30T00:44:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078a000640 1 --2- 172.21.15.66:0/3539705424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557017000100 0x557017018cf0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f07840ac040 tx=0x7f07840033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3655bd0194aaeeb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f0783400640 1 -- 172.21.15.66:0/3539705424 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840a9ac0 con 0x557017000100 2024-03-30T00:44:35.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f0783400640 1 -- 172.21.15.66:0/3539705424 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07840a9c20 con 0x557017000100 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f0783400640 1 -- 172.21.15.66:0/3539705424 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840a95b0 con 0x557017000100 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- 172.21.15.66:0/3539705424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557017000100 msgr2=0x557017018cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 --2- 172.21.15.66:0/3539705424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557017000100 0x557017018cf0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f07840ac040 tx=0x7f07840033b0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- 172.21.15.66:0/3539705424 shutdown_connections 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 --2- 172.21.15.66:0/3539705424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557017000100 0x557017018cf0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 --2- 172.21.15.66:0/3539705424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557016f8ad00 0x557016f701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 --2- 172.21.15.66:0/3539705424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557016f70e40 0x557016f82080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- 172.21.15.66:0/3539705424 >> 172.21.15.66:0/3539705424 conn(0x557016f6bdd0 msgr2=0x557016f440c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- 172.21.15.66:0/3539705424 shutdown_connections 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- 172.21.15.66:0/3539705424 wait complete. 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 Processor -- start 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- start start 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557016f70e40 0x5570170208a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557016f8ad00 0x55701701f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557017000100 0x557016f44060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701701f670 con 0x557016f70e40 2024-03-30T00:44:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557016f43140 con 0x557016f8ad00 2024-03-30T00:44:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.155+0000 7f078c447080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x557016f433e0 con 0x557017000100 2024-03-30T00:44:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f0789600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557016f70e40 0x5570170208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f0789600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557016f70e40 0x5570170208a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39482/0 (socket says 172.21.15.66:39482) 2024-03-30T00:44:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f078a000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557017000100 0x557016f44060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f0788c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557016f8ad00 0x55701701f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:35.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f0789600640 1 -- 172.21.15.66:0/713359982 learned_addr learned my addr 172.21.15.66:0/713359982 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:35.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f078a000640 1 -- 172.21.15.66:0/713359982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557016f8ad00 msgr2=0x55701701f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f078a000640 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557016f8ad00 0x55701701f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f078a000640 1 -- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557016f70e40 msgr2=0x5570170208a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f078a000640 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557016f70e40 0x5570170208a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f078a000640 1 -- 172.21.15.66:0/713359982 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557016eb1a60 con 0x557017000100 2024-03-30T00:44:35.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f0788c00640 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557016f8ad00 0x55701701f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:35.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f0789600640 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557016f70e40 0x5570170208a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:35.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f078a000640 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557017000100 0x557016f44060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f078400a560 tx=0x7f0784018340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:35.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f0782000640 1 -- 172.21.15.66:0/713359982 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840a9ac0 con 0x557017000100 2024-03-30T00:44:35.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f0782000640 1 -- 172.21.15.66:0/713359982 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07840b3aa0 con 0x557017000100 2024-03-30T00:44:35.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f0782000640 1 -- 172.21.15.66:0/713359982 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840b3e10 con 0x557017000100 2024-03-30T00:44:35.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f078c447080 1 -- 172.21.15.66:0/713359982 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557016e978c0 con 0x557017000100 2024-03-30T00:44:35.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f078c447080 1 -- 172.21.15.66:0/713359982 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557016e8ad00 con 0x557017000100 2024-03-30T00:44:35.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f0782000640 1 -- 172.21.15.66:0/713359982 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f07840b0050 con 0x557017000100 2024-03-30T00:44:35.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.159+0000 7f0782000640 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f076c0650a0 0x7f076c067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.163+0000 7f0782000640 1 -- 172.21.15.66:0/713359982 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f078411eae0 con 0x557017000100 2024-03-30T00:44:35.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.163+0000 7f0789600640 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f076c0650a0 0x7f076c067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:35.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.163+0000 7f078c447080 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5570170aa500 0x557016ffbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.163+0000 7f078c447080 1 -- 172.21.15.66:0/713359982 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v8 -- 0x557016ffc290 con 0x5570170aa500 2024-03-30T00:44:35.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.163+0000 7f0788c00640 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5570170aa500 0x557016ffbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:35.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.163+0000 7f0789600640 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f076c0650a0 0x7f076c067560 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f077c009ca0 tx=0x7f077c00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:35.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.163+0000 7f0788c00640 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5570170aa500 0x557016ffbd50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f0788c00640 1 -- 172.21.15.66:0/713359982 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v83'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f077800eb20 con 0x5570170aa500 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f078c447080 1 -- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5570170aa500 msgr2=0x557016ffbd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f078c447080 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5570170aa500 0x557016ffbd50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f078c447080 1 -- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f076c0650a0 msgr2=0x7f076c067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f078c447080 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f076c0650a0 0x7f076c067560 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f077c009ca0 tx=0x7f077c00d040 comp rx=0 tx=0).stop 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f078c447080 1 -- 172.21.15.66:0/713359982 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557017000100 msgr2=0x557016f44060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f078c447080 1 --2- 172.21.15.66:0/713359982 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557017000100 0x557016f44060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f078400a560 tx=0x7f0784018340 comp rx=0 tx=0).stop 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f078a000640 1 -- 172.21.15.66:0/713359982 reap_dead start 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f078c447080 1 -- 172.21.15.66:0/713359982 shutdown_connections 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f078c447080 1 -- 172.21.15.66:0/713359982 >> 172.21.15.66:0/713359982 conn(0x557016f6bdd0 msgr2=0x557016f701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f078c447080 1 -- 172.21.15.66:0/713359982 shutdown_connections 2024-03-30T00:44:35.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.167+0000 7f078c447080 1 -- 172.21.15.66:0/713359982 wait complete. 2024-03-30T00:44:35.179 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-30T00:44:35.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f1ee00640 1 Processor -- start 2024-03-30T00:44:35.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f1ee00640 1 -- start start 2024-03-30T00:44:35.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f1ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f180cec30 0x7f5f18159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f1ee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f180cd3e0 con 0x7f5f180cda90 2024-03-30T00:44:35.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f1ee00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1815a990 con 0x7f5f180cde70 2024-03-30T00:44:35.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f1ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180b1b80 con 0x7f5f180cec30 2024-03-30T00:44:35.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f17e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5f180cda90 0x7f5f180ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52792/0 (socket says 172.21.15.66:52792) 2024-03-30T00:44:35.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f17e00640 1 -- 172.21.15.66:0/1557688946 learned_addr learned my addr 172.21.15.66:0/1557688946 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:35.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f0fe00640 1 --2- 172.21.15.66:0/1557688946 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f180cec30 0x7f5f18159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:35.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f17400640 1 -- 172.21.15.66:0/1557688946 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130962076 0 0) 0x7f5f180cd3e0 con 0x7f5f180cda90 2024-03-30T00:44:35.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f17400640 1 -- 172.21.15.66:0/1557688946 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5efc003530 con 0x7f5f180cda90 2024-03-30T00:44:35.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f17400640 1 -- 172.21.15.66:0/1557688946 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027585821 0 0) 0x7f5f1815a990 con 0x7f5f180cde70 2024-03-30T00:44:35.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f17400640 1 -- 172.21.15.66:0/1557688946 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f180cd3e0 con 0x7f5f180cde70 2024-03-30T00:44:35.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f0fe00640 1 -- 172.21.15.66:0/1557688946 >> v1:172.21.15.156:6789/0 conn(0x7f5f180cde70 legacy=0x7f5f1815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.355+0000 7f5f0fe00640 1 -- 172.21.15.66:0/1557688946 >> v1:172.21.15.66:6789/0 conn(0x7f5f180cda90 legacy=0x7f5f180ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.359+0000 7f5f0fe00640 1 -- 172.21.15.66:0/1557688946 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f180b11e0 con 0x7f5f180cec30 2024-03-30T00:44:35.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.359+0000 7f5f0fe00640 1 --2- 172.21.15.66:0/1557688946 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f180cec30 0x7f5f18159c60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5f000097f0 tx=0x7f5f0001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a9e898e325f792 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:35.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.359+0000 7f5f17400640 1 -- 172.21.15.66:0/1557688946 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f000ab650 con 0x7f5f180cec30 2024-03-30T00:44:35.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.359+0000 7f5f17400640 1 -- 172.21.15.66:0/1557688946 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f000ab7b0 con 0x7f5f180cec30 2024-03-30T00:44:35.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.359+0000 7f5f17400640 1 -- 172.21.15.66:0/1557688946 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f000abb20 con 0x7f5f180cec30 2024-03-30T00:44:35.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 -- 172.21.15.66:0/1557688946 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f180cec30 msgr2=0x7f5f18159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 --2- 172.21.15.66:0/1557688946 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f180cec30 0x7f5f18159c60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5f000097f0 tx=0x7f5f0001c450 comp rx=0 tx=0).stop 2024-03-30T00:44:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 -- 172.21.15.66:0/1557688946 shutdown_connections 2024-03-30T00:44:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 --2- 172.21.15.66:0/1557688946 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f180cec30 0x7f5f18159c60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 -- 172.21.15.66:0/1557688946 >> 172.21.15.66:0/1557688946 conn(0x7f5f18069f60 msgr2=0x7f5f1806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 -- 172.21.15.66:0/1557688946 shutdown_connections 2024-03-30T00:44:35.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 -- 172.21.15.66:0/1557688946 wait complete. 2024-03-30T00:44:35.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 Processor -- start 2024-03-30T00:44:35.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 -- start start 2024-03-30T00:44:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f180cda90 0x7f5f1806fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f180cde70 0x7f5f18163670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f180cec30 0x7f5f18163bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180ac9d0 con 0x7f5f180cda90 2024-03-30T00:44:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f18158250 con 0x7f5f180cde70 2024-03-30T00:44:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f18157f20 con 0x7f5f180cec30 2024-03-30T00:44:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f180cda90 0x7f5f1806fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f180cda90 0x7f5f1806fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39496/0 (socket says 172.21.15.66:39496) 2024-03-30T00:44:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f17e00640 1 -- 172.21.15.66:0/3458121201 learned_addr learned my addr 172.21.15.66:0/3458121201 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f17e00640 1 -- 172.21.15.66:0/3458121201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f180cec30 msgr2=0x7f5f18163bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f17e00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f180cec30 0x7f5f18163bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f17e00640 1 -- 172.21.15.66:0/3458121201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f180cde70 msgr2=0x7f5f18163670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f17e00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f180cde70 0x7f5f18163670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f17e00640 1 -- 172.21.15.66:0/3458121201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f00009420 con 0x7f5f180cda90 2024-03-30T00:44:35.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f17e00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f180cda90 0x7f5f1806fed0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c4f0 tx=0x7f5f0800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:35.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f14c00640 1 -- 172.21.15.66:0/3458121201 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08005cf0 con 0x7f5f180cda90 2024-03-30T00:44:35.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f14c00640 1 -- 172.21.15.66:0/3458121201 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08005e50 con 0x7f5f180cda90 2024-03-30T00:44:35.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f14c00640 1 -- 172.21.15.66:0/3458121201 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08007270 con 0x7f5f180cda90 2024-03-30T00:44:35.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.363+0000 7f5f1ee00640 1 -- 172.21.15.66:0/3458121201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f18070410 con 0x7f5f180cda90 2024-03-30T00:44:35.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.367+0000 7f5f1ee00640 1 -- 172.21.15.66:0/3458121201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f180acf00 con 0x7f5f180cda90 2024-03-30T00:44:35.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.367+0000 7f5f1ee00640 1 -- 172.21.15.66:0/3458121201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f18066c30 con 0x7f5f180cda90 2024-03-30T00:44:35.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.367+0000 7f5f14c00640 1 -- 172.21.15.66:0/3458121201 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5f080073d0 con 0x7f5f180cda90 2024-03-30T00:44:35.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.367+0000 7f5f14c00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ef4066690 0x7f5ef4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:35.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.367+0000 7f5f14c00640 1 -- 172.21.15.66:0/3458121201 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5f08083870 con 0x7f5f180cda90 2024-03-30T00:44:35.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.367+0000 7f5f0fe00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ef4066690 0x7f5ef4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:35.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.367+0000 7f5f0fe00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ef4066690 0x7f5ef4068b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5f000af000 tx=0x7f5f00004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:35.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.371+0000 7f5f14c00640 1 -- 172.21.15.66:0/3458121201 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5f08050ae0 con 0x7f5f180cda90 2024-03-30T00:44:35.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:35.567+0000 7f5f1ee00640 1 -- 172.21.15.66:0/3458121201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f5f18158090 con 0x7f5f180cda90 2024-03-30T00:44:36.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f14c00640 1 -- 172.21.15.66:0/3458121201 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v45) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5f18158090 con 0x7f5f180cda90 2024-03-30T00:44:36.093 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-30T00:44:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 -- 172.21.15.66:0/3458121201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ef4066690 msgr2=0x7f5ef4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ef4066690 0x7f5ef4068b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5f000af000 tx=0x7f5f00004740 comp rx=0 tx=0).stop 2024-03-30T00:44:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 -- 172.21.15.66:0/3458121201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f180cda90 msgr2=0x7f5f1806fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f180cda90 0x7f5f1806fed0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c4f0 tx=0x7f5f0800c9b0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 -- 172.21.15.66:0/3458121201 shutdown_connections 2024-03-30T00:44:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f180cec30 0x7f5f18163bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f180cde70 0x7f5f18163670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5ef4066690 0x7f5ef4068b50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 --2- 172.21.15.66:0/3458121201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f180cda90 0x7f5f1806fed0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 -- 172.21.15.66:0/3458121201 >> 172.21.15.66:0/3458121201 conn(0x7f5f18069f60 msgr2=0x7f5f1815d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:36.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 -- 172.21.15.66:0/3458121201 shutdown_connections 2024-03-30T00:44:36.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.087+0000 7f5f1ee00640 1 -- 172.21.15.66:0/3458121201 wait complete. 2024-03-30T00:44:36.123 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:44:36.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f43200640 1 Processor -- start 2024-03-30T00:44:36.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f43200640 1 -- start start 2024-03-30T00:44:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f43200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f3c0cc560 0x7f5f3c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f43200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5f3c06e800 0x7f5f3c0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f43200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3c06b0b0 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f43200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3c160640 con 0x7f5f3c0cc560 2024-03-30T00:44:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f43200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3c06b390 con 0x7f5f3c06e800 2024-03-30T00:44:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f3be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f3c0cc560 0x7f5f3c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:36.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f3be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f3c0cc560 0x7f5f3c0d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39502/0 (socket says 172.21.15.66:39502) 2024-03-30T00:44:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f3be00640 1 -- 172.21.15.66:0/3286436017 learned_addr learned my addr 172.21.15.66:0/3286436017 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f33400640 1 --1- 172.21.15.66:0/3286436017 >> v1:172.21.15.66:6789/0 conn(0x7f5f3c06f8e0 0x7f5f3c06e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52804/0 (socket says 172.21.15.66:52804) 2024-03-30T00:44:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f41200640 1 --2- 172.21.15.66:0/3286436017 >> v2:172.21.15.156:3300/0 conn(0x7f5f3c06e800 0x7f5f3c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f3be00640 1 -- 172.21.15.66:0/3286436017 >> v2:172.21.15.156:3300/0 conn(0x7f5f3c06e800 msgr2=0x7f5f3c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f3be00640 1 --2- 172.21.15.66:0/3286436017 >> v2:172.21.15.156:3300/0 conn(0x7f5f3c06e800 0x7f5f3c0ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f3be00640 1 -- 172.21.15.66:0/3286436017 >> v1:172.21.15.66:6789/0 conn(0x7f5f3c06f8e0 legacy=0x7f5f3c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:36.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.259+0000 7f5f3be00640 1 -- 172.21.15.66:0/3286436017 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c06b670 con 0x7f5f3c0cc560 2024-03-30T00:44:36.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f3be00640 1 --2- 172.21.15.66:0/3286436017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f3c0cc560 0x7f5f3c0d1060 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0ce1a0 tx=0x7f5f2c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ba7a09c25cc4e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:36.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f3b400640 1 -- 172.21.15.66:0/3286436017 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0aa670 con 0x7f5f3c0cc560 2024-03-30T00:44:36.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f3b400640 1 -- 172.21.15.66:0/3286436017 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c0aa7d0 con 0x7f5f3c0cc560 2024-03-30T00:44:36.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f3b400640 1 -- 172.21.15.66:0/3286436017 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0aab60 con 0x7f5f3c0cc560 2024-03-30T00:44:36.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f43200640 1 -- 172.21.15.66:0/3286436017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f3c0cc560 msgr2=0x7f5f3c0d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:36.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f43200640 1 --2- 172.21.15.66:0/3286436017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f3c0cc560 0x7f5f3c0d1060 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0ce1a0 tx=0x7f5f2c01c4f0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f43200640 1 -- 172.21.15.66:0/3286436017 shutdown_connections 2024-03-30T00:44:36.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f43200640 1 --2- 172.21.15.66:0/3286436017 >> v2:172.21.15.156:3300/0 conn(0x7f5f3c06e800 0x7f5f3c0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f43200640 1 --2- 172.21.15.66:0/3286436017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f3c0cc560 0x7f5f3c0d1060 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f43200640 1 -- 172.21.15.66:0/3286436017 >> 172.21.15.66:0/3286436017 conn(0x7f5f3c0d0540 msgr2=0x7f5f3c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:36.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f43200640 1 -- 172.21.15.66:0/3286436017 shutdown_connections 2024-03-30T00:44:36.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f43200640 1 -- 172.21.15.66:0/3286436017 wait complete. 2024-03-30T00:44:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.263+0000 7f5f43200640 1 Processor -- start 2024-03-30T00:44:36.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f43200640 1 -- start start 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f43200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f3c06e800 0x7f5f3c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f43200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f3c06f8e0 0x7f5f3c06ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f43200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3c0cc560 0x7f5f3c06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f43200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0ca080 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f43200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0c9470 con 0x7f5f3c0cc560 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f43200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0c9750 con 0x7f5f3c06e800 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f33400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f3c06f8e0 0x7f5f3c06ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f33400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f3c06f8e0 0x7f5f3c06ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39512/0 (socket says 172.21.15.66:39512) 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f33400640 1 -- 172.21.15.66:0/3025721885 learned_addr learned my addr 172.21.15.66:0/3025721885 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f41200640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3c0cc560 0x7f5f3c06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f3be00640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f3c06e800 0x7f5f3c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:36.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f33400640 1 -- 172.21.15.66:0/3025721885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f3c06e800 msgr2=0x7f5f3c1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:36.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f33400640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f3c06e800 0x7f5f3c1536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f33400640 1 -- 172.21.15.66:0/3025721885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3c0cc560 msgr2=0x7f5f3c06f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:36.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f33400640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3c0cc560 0x7f5f3c06f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f33400640 1 -- 172.21.15.66:0/3025721885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0094d0 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f33400640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f3c06f8e0 0x7f5f3c06ebe0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400c420 tx=0x7f5f2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:36.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f38c00640 1 -- 172.21.15.66:0/3025721885 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240074e0 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f38c00640 1 -- 172.21.15.66:0/3025721885 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f24007660 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f43200640 1 -- 172.21.15.66:0/3025721885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c0c9a30 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f38c00640 1 -- 172.21.15.66:0/3025721885 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24004a30 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f43200640 1 -- 172.21.15.66:0/3025721885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c153c10 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.267+0000 7f5f43200640 1 -- 172.21.15.66:0/3025721885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3c066c30 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.271+0000 7f5f38c00640 1 -- 172.21.15.66:0/3025721885 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5f24004c50 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.271+0000 7f5f38c00640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f18066690 0x7f5f18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:36.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.271+0000 7f5f38c00640 1 -- 172.21.15.66:0/3025721885 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5f24083d60 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.271+0000 7f5f3be00640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f18066690 0x7f5f18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:36.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.271+0000 7f5f3be00640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f18066690 0x7f5f18068b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0ae000 tx=0x7f5f2c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:36.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.275+0000 7f5f38c00640 1 -- 172.21.15.66:0/3025721885 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5f24050e90 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.471+0000 7f5f43200640 1 -- 172.21.15.66:0/3025721885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f3c0cb0f0 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.471+0000 7f5f38c00640 1 -- 172.21.15.66:0/3025721885 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f5f240544b0 con 0x7f5f3c06f8e0 2024-03-30T00:44:36.479 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:36.479 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/254323195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":254323195},{"type":"v1","addr":"172.21.15.156:6835","nonce":254323195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:44:36.089041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":46,"state":"up:replay","state_seq":1,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:44:36.480 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 46 2024-03-30T00:44:36.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 -- 172.21.15.66:0/3025721885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f18066690 msgr2=0x7f5f18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:36.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f18066690 0x7f5f18068b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0ae000 tx=0x7f5f2c0a7000 comp rx=0 tx=0).stop 2024-03-30T00:44:36.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 -- 172.21.15.66:0/3025721885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f3c06f8e0 msgr2=0x7f5f3c06ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:36.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f3c06f8e0 0x7f5f3c06ebe0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400c420 tx=0x7f5f2400c8e0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 -- 172.21.15.66:0/3025721885 shutdown_connections 2024-03-30T00:44:36.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3c0cc560 0x7f5f3c06f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f18066690 0x7f5f18068b50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f3c06f8e0 0x7f5f3c06ebe0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 --2- 172.21.15.66:0/3025721885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f3c06e800 0x7f5f3c1536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:36.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 -- 172.21.15.66:0/3025721885 >> 172.21.15.66:0/3025721885 conn(0x7f5f3c0d0540 msgr2=0x7f5f3c0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:36.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 -- 172.21.15.66:0/3025721885 shutdown_connections 2024-03-30T00:44:36.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:36.475+0000 7f5f43200640 1 -- 172.21.15.66:0/3025721885 wait complete. 2024-03-30T00:44:36.903 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:44:16.274202+0000', 'modified': '2024-03-30T00:44:36.089041+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6996}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6996': {'gid': 6996, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.66:6837/2050987583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 2050987583}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 2050987583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:44:36.903 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6996:up:replay 2024-03-30T00:44:37.904 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:44:38.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c51200640 1 Processor -- start 2024-03-30T00:44:38.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c51200640 1 -- start start 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c51200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1c4c15e370 0x7f1c4c0cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c51200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1c4c069d90 0x7f1c4c06a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c51200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4c0cf160 con 0x7f1c4c15a960 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c51200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c15ad40 con 0x7f1c4c15e370 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c51200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c06a720 con 0x7f1c4c069d90 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c43e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1c4c15e370 0x7f1c4c0cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c43e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1c4c15e370 0x7f1c4c0cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39528/0 (socket says 172.21.15.66:39528) 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c43e00640 1 -- 172.21.15.66:0/491955432 learned_addr learned my addr 172.21.15.66:0/491955432 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c4a800640 1 --2- 172.21.15.66:0/491955432 >> v2:172.21.15.156:3300/0 conn(0x7f1c4c069d90 0x7f1c4c06a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c43e00640 1 -- 172.21.15.66:0/491955432 >> v2:172.21.15.156:3300/0 conn(0x7f1c4c069d90 msgr2=0x7f1c4c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c43e00640 1 --2- 172.21.15.66:0/491955432 >> v2:172.21.15.156:3300/0 conn(0x7f1c4c069d90 0x7f1c4c06a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c43e00640 1 -- 172.21.15.66:0/491955432 >> v1:172.21.15.156:6789/0 conn(0x7f1c4c15a960 legacy=0x7f1c4c0cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:38.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.051+0000 7f1c43e00640 1 -- 172.21.15.66:0/491955432 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c15aed0 con 0x7f1c4c15e370 2024-03-30T00:44:38.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c43e00640 1 --2- 172.21.15.66:0/491955432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c4c15e370 0x7f1c4c0cec20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009870 tx=0x7f1c3401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170b6904800a0217 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c49400640 1 -- 172.21.15.66:0/491955432 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c340a8a10 con 0x7f1c4c15e370 2024-03-30T00:44:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c49400640 1 -- 172.21.15.66:0/491955432 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c340a8b70 con 0x7f1c4c15e370 2024-03-30T00:44:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c49400640 1 -- 172.21.15.66:0/491955432 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c340b0440 con 0x7f1c4c15e370 2024-03-30T00:44:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c51200640 1 -- 172.21.15.66:0/491955432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c4c15e370 msgr2=0x7f1c4c0cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c51200640 1 --2- 172.21.15.66:0/491955432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c4c15e370 0x7f1c4c0cec20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009870 tx=0x7f1c3401ed10 comp rx=0 tx=0).stop 2024-03-30T00:44:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c51200640 1 -- 172.21.15.66:0/491955432 shutdown_connections 2024-03-30T00:44:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c51200640 1 --2- 172.21.15.66:0/491955432 >> v2:172.21.15.156:3300/0 conn(0x7f1c4c069d90 0x7f1c4c06a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c51200640 1 --2- 172.21.15.66:0/491955432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c4c15e370 0x7f1c4c0cec20 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c51200640 1 -- 172.21.15.66:0/491955432 >> 172.21.15.66:0/491955432 conn(0x7f1c4c098a20 msgr2=0x7f1c4c0b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c51200640 1 -- 172.21.15.66:0/491955432 shutdown_connections 2024-03-30T00:44:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c51200640 1 -- 172.21.15.66:0/491955432 wait complete. 2024-03-30T00:44:38.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.055+0000 7f1c51200640 1 Processor -- start 2024-03-30T00:44:38.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c51200640 1 -- start start 2024-03-30T00:44:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c51200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c4c069d90 0x7f1c4c151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c51200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c4c15a960 0x7f1c4c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c51200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c15e370 0x7f1c4c157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c51200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0c82b0 con 0x7f1c4c15a960 2024-03-30T00:44:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c51200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0c7670 con 0x7f1c4c15e370 2024-03-30T00:44:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c51200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c151650 con 0x7f1c4c069d90 2024-03-30T00:44:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c43e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c4c15a960 0x7f1c4c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c43e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c4c15a960 0x7f1c4c157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39540/0 (socket says 172.21.15.66:39540) 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c43e00640 1 -- 172.21.15.66:0/2460809123 learned_addr learned my addr 172.21.15.66:0/2460809123 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c49e00640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c4c069d90 0x7f1c4c151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c4a800640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c15e370 0x7f1c4c157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c43e00640 1 -- 172.21.15.66:0/2460809123 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c4c069d90 msgr2=0x7f1c4c151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c43e00640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c4c069d90 0x7f1c4c151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c43e00640 1 -- 172.21.15.66:0/2460809123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c15e370 msgr2=0x7f1c4c157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c43e00640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c15e370 0x7f1c4c157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c43e00640 1 -- 172.21.15.66:0/2460809123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c340094d0 con 0x7f1c4c15a960 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c43e00640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c4c15a960 0x7f1c4c157780 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1c340b3040 tx=0x7f1c3401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c42000640 1 -- 172.21.15.66:0/2460809123 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c340b9620 con 0x7f1c4c15a960 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c42000640 1 -- 172.21.15.66:0/2460809123 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c340b9780 con 0x7f1c4c15a960 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c42000640 1 -- 172.21.15.66:0/2460809123 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c340b9b30 con 0x7f1c4c15a960 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c51200640 1 -- 172.21.15.66:0/2460809123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c0c7910 con 0x7f1c4c15a960 2024-03-30T00:44:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c51200640 1 -- 172.21.15.66:0/2460809123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c0c7ed0 con 0x7f1c4c15a960 2024-03-30T00:44:38.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.059+0000 7f1c51200640 1 -- 172.21.15.66:0/2460809123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c0c83f0 con 0x7f1c4c15a960 2024-03-30T00:44:38.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.063+0000 7f1c42000640 1 -- 172.21.15.66:0/2460809123 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1c340b7070 con 0x7f1c4c15a960 2024-03-30T00:44:38.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.063+0000 7f1c42000640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c28066690 0x7f1c28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:38.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.063+0000 7f1c42000640 1 -- 172.21.15.66:0/2460809123 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f1c34122160 con 0x7f1c4c15a960 2024-03-30T00:44:38.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.063+0000 7f1c49e00640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c28066690 0x7f1c28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:38.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.063+0000 7f1c49e00640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c28066690 0x7f1c28068b50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c007fd0 tx=0x7f1c3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:38.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.067+0000 7f1c42000640 1 -- 172.21.15.66:0/2460809123 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1c4c0c83f0 con 0x7f1c4c15a960 2024-03-30T00:44:38.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.263+0000 7f1c51200640 1 -- 172.21.15.66:0/2460809123 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c4c0c83f0 con 0x7f1c4c15a960 2024-03-30T00:44:38.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.263+0000 7f1c42000640 1 -- 172.21.15.66:0/2460809123 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f1c4c0c83f0 con 0x7f1c4c15a960 2024-03-30T00:44:38.271 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:38.271 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/254323195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":254323195},{"type":"v1","addr":"172.21.15.156:6835","nonce":254323195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:44:38.092519+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":46,"state":"up:rejoin","state_seq":9,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:44:38.271 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 48 2024-03-30T00:44:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.263+0000 7f1c51200640 1 -- 172.21.15.66:0/2460809123 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c28066690 msgr2=0x7f1c28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.263+0000 7f1c51200640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c28066690 0x7f1c28068b50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c007fd0 tx=0x7f1c3c009040 comp rx=0 tx=0).stop 2024-03-30T00:44:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.267+0000 7f1c51200640 1 -- 172.21.15.66:0/2460809123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c4c15a960 msgr2=0x7f1c4c157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.267+0000 7f1c51200640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c4c15a960 0x7f1c4c157780 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1c340b3040 tx=0x7f1c3401bab0 comp rx=0 tx=0).stop 2024-03-30T00:44:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.267+0000 7f1c51200640 1 -- 172.21.15.66:0/2460809123 shutdown_connections 2024-03-30T00:44:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.267+0000 7f1c51200640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c15e370 0x7f1c4c157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.267+0000 7f1c51200640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1c28066690 0x7f1c28068b50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.267+0000 7f1c51200640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c4c15a960 0x7f1c4c157780 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.267+0000 7f1c51200640 1 --2- 172.21.15.66:0/2460809123 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c4c069d90 0x7f1c4c151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:38.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.267+0000 7f1c51200640 1 -- 172.21.15.66:0/2460809123 >> 172.21.15.66:0/2460809123 conn(0x7f1c4c098a20 msgr2=0x7f1c4c0ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:38.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.267+0000 7f1c51200640 1 -- 172.21.15.66:0/2460809123 shutdown_connections 2024-03-30T00:44:38.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:38.267+0000 7f1c51200640 1 -- 172.21.15.66:0/2460809123 wait complete. 2024-03-30T00:44:38.695 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:44:16.274202+0000', 'modified': '2024-03-30T00:44:38.092519+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6996}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6996': {'gid': 6996, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:rejoin', 'state_seq': 9, 'addr': '172.21.15.66:6837/2050987583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 2050987583}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 2050987583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:44:38.695 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6996:up:rejoin 2024-03-30T00:44:39.696 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:44:39.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10dd000640 1 Processor -- start 2024-03-30T00:44:39.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10dd000640 1 -- start start 2024-03-30T00:44:39.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10dd000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f10d815a960 0x7f10d815e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:39.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10dd000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10d8069d90 0x7f10d806a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:39.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10dd000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d815aed0 con 0x7f10d80cec20 2024-03-30T00:44:39.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10dd000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d806a760 con 0x7f10d815a960 2024-03-30T00:44:39.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10dd000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d815ad40 con 0x7f10d8069d90 2024-03-30T00:44:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10d5e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f10d815a960 0x7f10d815e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10d6800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10d8069d90 0x7f10d806a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10d5e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f10d815a960 0x7f10d815e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:53444/0 (socket says 172.21.15.66:53444) 2024-03-30T00:44:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10d5e00640 1 -- 172.21.15.66:0/3807342310 learned_addr learned my addr 172.21.15.66:0/3807342310 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10d5e00640 1 -- 172.21.15.66:0/3807342310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10d8069d90 msgr2=0x7f10d806a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10d5e00640 1 --2- 172.21.15.66:0/3807342310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10d8069d90 0x7f10d806a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10d5e00640 1 -- 172.21.15.66:0/3807342310 >> v1:172.21.15.156:6789/0 conn(0x7f10d80cec20 legacy=0x7f10d80cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10d5e00640 1 -- 172.21.15.66:0/3807342310 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d80cb730 con 0x7f10d815a960 2024-03-30T00:44:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10d5400640 1 -- 172.21.15.66:0/3807342310 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 600312497 0 0) 0x7f10d815aed0 con 0x7f10d80cec20 2024-03-30T00:44:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.843+0000 7f10d6800640 1 --2- 172.21.15.66:0/3807342310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10d8069d90 0x7f10d806a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:39.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10d5e00640 1 --2- 172.21.15.66:0/3807342310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d815a960 0x7f10d815e370 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009870 tx=0x7f10c801ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40352057bee96840 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:39.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10d5400640 1 -- 172.21.15.66:0/3807342310 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c80a8a50 con 0x7f10d815a960 2024-03-30T00:44:39.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10d5400640 1 -- 172.21.15.66:0/3807342310 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c80a8bb0 con 0x7f10d815a960 2024-03-30T00:44:39.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10d5400640 1 -- 172.21.15.66:0/3807342310 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c80b0440 con 0x7f10d815a960 2024-03-30T00:44:39.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10dd000640 1 -- 172.21.15.66:0/3807342310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d815a960 msgr2=0x7f10d815e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:39.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10dd000640 1 --2- 172.21.15.66:0/3807342310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d815a960 0x7f10d815e370 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009870 tx=0x7f10c801ed50 comp rx=0 tx=0).stop 2024-03-30T00:44:39.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10dd000640 1 -- 172.21.15.66:0/3807342310 shutdown_connections 2024-03-30T00:44:39.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10dd000640 1 --2- 172.21.15.66:0/3807342310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10d8069d90 0x7f10d806a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:39.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10dd000640 1 --2- 172.21.15.66:0/3807342310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d815a960 0x7f10d815e370 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:39.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10dd000640 1 -- 172.21.15.66:0/3807342310 >> 172.21.15.66:0/3807342310 conn(0x7f10d8098a20 msgr2=0x7f10d80b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:39.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10dd000640 1 -- 172.21.15.66:0/3807342310 shutdown_connections 2024-03-30T00:44:39.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.847+0000 7f10dd000640 1 -- 172.21.15.66:0/3807342310 wait complete. 2024-03-30T00:44:39.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10dd000640 1 Processor -- start 2024-03-30T00:44:39.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10dd000640 1 -- start start 2024-03-30T00:44:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10dd000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8069d90 0x7f10d80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10dd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80cec20 0x7f10d80ac610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10dd000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10d815a960 0x7f10d81551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10dd000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80ad650 con 0x7f10d80cec20 2024-03-30T00:44:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10dd000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80c72a0 con 0x7f10d8069d90 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10dd000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80acb50 con 0x7f10d815a960 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80cec20 0x7f10d80ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80cec20 0x7f10d80ac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49256/0 (socket says 172.21.15.66:49256) 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10cfe00640 1 -- 172.21.15.66:0/1786514825 learned_addr learned my addr 172.21.15.66:0/1786514825 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10d5e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8069d90 0x7f10d80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10d6800640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10d815a960 0x7f10d81551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10cfe00640 1 -- 172.21.15.66:0/1786514825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10d815a960 msgr2=0x7f10d81551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10cfe00640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10d815a960 0x7f10d81551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10cfe00640 1 -- 172.21.15.66:0/1786514825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8069d90 msgr2=0x7f10d80c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10cfe00640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8069d90 0x7f10d80c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10cfe00640 1 -- 172.21.15.66:0/1786514825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d0009a70 con 0x7f10d80cec20 2024-03-30T00:44:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10cfe00640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80cec20 0x7f10d80ac610 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c4d0 tx=0x7f10c000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10ce000640 1 -- 172.21.15.66:0/1786514825 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c00054e0 con 0x7f10d80cec20 2024-03-30T00:44:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10ce000640 1 -- 172.21.15.66:0/1786514825 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c0003600 con 0x7f10d80cec20 2024-03-30T00:44:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10dd000640 1 -- 172.21.15.66:0/1786514825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c80094d0 con 0x7f10d80cec20 2024-03-30T00:44:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10ce000640 1 -- 172.21.15.66:0/1786514825 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0003990 con 0x7f10d80cec20 2024-03-30T00:44:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.851+0000 7f10dd000640 1 -- 172.21.15.66:0/1786514825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d80ad090 con 0x7f10d80cec20 2024-03-30T00:44:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.855+0000 7f10dd000640 1 -- 172.21.15.66:0/1786514825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d8066c30 con 0x7f10d80cec20 2024-03-30T00:44:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.855+0000 7f10ce000640 1 -- 172.21.15.66:0/1786514825 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f10c0003b30 con 0x7f10d80cec20 2024-03-30T00:44:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.855+0000 7f10ce000640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f10b4066690 0x7f10b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.855+0000 7f10ce000640 1 -- 172.21.15.66:0/1786514825 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f10c0083950 con 0x7f10d80cec20 2024-03-30T00:44:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.855+0000 7f10d5e00640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f10b4066690 0x7f10b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.855+0000 7f10d5e00640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f10b4066690 0x7f10b4068b50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f10c80b2000 tx=0x7f10c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:39.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:39.859+0000 7f10ce000640 1 -- 172.21.15.66:0/1786514825 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f10c0054680 con 0x7f10d80cec20 2024-03-30T00:44:40.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.055+0000 7f10dd000640 1 -- 172.21.15.66:0/1786514825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10d80c47c0 con 0x7f10d80cec20 2024-03-30T00:44:40.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.055+0000 7f10ce000640 1 -- 172.21.15.66:0/1786514825 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f10c0017020 con 0x7f10d80cec20 2024-03-30T00:44:40.064 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:40.064 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/254323195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":254323195},{"type":"v1","addr":"172.21.15.156:6835","nonce":254323195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:44:39.098104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":10,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:44:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 49 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 -- 172.21.15.66:0/1786514825 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f10b4066690 msgr2=0x7f10b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f10b4066690 0x7f10b4068b50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f10c80b2000 tx=0x7f10c8009040 comp rx=0 tx=0).stop 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 -- 172.21.15.66:0/1786514825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80cec20 msgr2=0x7f10d80ac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80cec20 0x7f10d80ac610 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c4d0 tx=0x7f10c000c8d0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 -- 172.21.15.66:0/1786514825 shutdown_connections 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10d815a960 0x7f10d81551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f10b4066690 0x7f10b4068b50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80cec20 0x7f10d80ac610 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 --2- 172.21.15.66:0/1786514825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8069d90 0x7f10d80c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 -- 172.21.15.66:0/1786514825 >> 172.21.15.66:0/1786514825 conn(0x7f10d8098a20 msgr2=0x7f10d80b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 -- 172.21.15.66:0/1786514825 shutdown_connections 2024-03-30T00:44:40.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.059+0000 7f10dd000640 1 -- 172.21.15.66:0/1786514825 wait complete. 2024-03-30T00:44:40.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:44:16.274202+0000', 'modified': '2024-03-30T00:44:39.098104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6996}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6996': {'gid': 6996, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.66:6837/2050987583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 2050987583}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 2050987583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:44:40.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:44:40.487 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T00:44:40.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc770c00640 1 Processor -- start 2024-03-30T00:44:40.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc770c00640 1 -- start start 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc770c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc76c0cda90 0x7fc76c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc770c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc76c0cde70 0x7fc76c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc770c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76c15a990 con 0x7fc76c159c60 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc770c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76c0b1b80 con 0x7fc76c0cde70 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc770c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76c0cd3e0 con 0x7fc76c0cda90 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc763e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc76c0cde70 0x7fc76c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc763e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc76c0cde70 0x7fc76c0cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49266/0 (socket says 172.21.15.66:49266) 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc763e00640 1 -- 172.21.15.66:0/2365267144 learned_addr learned my addr 172.21.15.66:0/2365267144 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc769e00640 1 --2- 172.21.15.66:0/2365267144 >> v2:172.21.15.156:3300/0 conn(0x7fc76c0cda90 0x7fc76c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc763e00640 1 -- 172.21.15.66:0/2365267144 >> v2:172.21.15.156:3300/0 conn(0x7fc76c0cda90 msgr2=0x7fc76c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc763e00640 1 --2- 172.21.15.66:0/2365267144 >> v2:172.21.15.156:3300/0 conn(0x7fc76c0cda90 0x7fc76c0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc763e00640 1 -- 172.21.15.66:0/2365267144 >> v1:172.21.15.156:6789/0 conn(0x7fc76c159c60 legacy=0x7fc76c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc763e00640 1 -- 172.21.15.66:0/2365267144 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c0b11e0 con 0x7fc76c0cde70 2024-03-30T00:44:40.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.623+0000 7fc769e00640 1 --2- 172.21.15.66:0/2365267144 >> v2:172.21.15.156:3300/0 conn(0x7fc76c0cda90 0x7fc76c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:40.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc763e00640 1 --2- 172.21.15.66:0/2365267144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc76c0cde70 0x7fc76c0cea60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0cf3f0 tx=0x7fc75401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8e49aa0ed3fd38 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:40.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc769400640 1 -- 172.21.15.66:0/2365267144 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7540ad660 con 0x7fc76c0cde70 2024-03-30T00:44:40.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc769400640 1 -- 172.21.15.66:0/2365267144 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7540ad7c0 con 0x7fc76c0cde70 2024-03-30T00:44:40.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc769400640 1 -- 172.21.15.66:0/2365267144 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7540adb50 con 0x7fc76c0cde70 2024-03-30T00:44:40.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc770c00640 1 -- 172.21.15.66:0/2365267144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc76c0cde70 msgr2=0x7fc76c0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc770c00640 1 --2- 172.21.15.66:0/2365267144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc76c0cde70 0x7fc76c0cea60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0cf3f0 tx=0x7fc75401c450 comp rx=0 tx=0).stop 2024-03-30T00:44:40.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc770c00640 1 -- 172.21.15.66:0/2365267144 shutdown_connections 2024-03-30T00:44:40.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc770c00640 1 --2- 172.21.15.66:0/2365267144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc76c0cde70 0x7fc76c0cea60 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc770c00640 1 --2- 172.21.15.66:0/2365267144 >> v2:172.21.15.156:3300/0 conn(0x7fc76c0cda90 0x7fc76c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc770c00640 1 -- 172.21.15.66:0/2365267144 >> 172.21.15.66:0/2365267144 conn(0x7fc76c069f60 msgr2=0x7fc76c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:40.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc770c00640 1 -- 172.21.15.66:0/2365267144 shutdown_connections 2024-03-30T00:44:40.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc770c00640 1 -- 172.21.15.66:0/2365267144 wait complete. 2024-03-30T00:44:40.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc770c00640 1 Processor -- start 2024-03-30T00:44:40.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.627+0000 7fc770c00640 1 -- start start 2024-03-30T00:44:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc770c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc76c0cda90 0x7fc76c0c1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc770c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc76c0cde70 0x7fc76c0c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc770c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc76c159c60 0x7fc76c0ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc770c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c0c4030 con 0x7fc76c159c60 2024-03-30T00:44:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc770c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c0c3760 con 0x7fc76c0cde70 2024-03-30T00:44:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc770c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c097a70 con 0x7fc76c0cda90 2024-03-30T00:44:40.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc76a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc76c159c60 0x7fc76c0ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc76a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc76c159c60 0x7fc76c0ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49268/0 (socket says 172.21.15.66:49268) 2024-03-30T00:44:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc76a800640 1 -- 172.21.15.66:0/4097904344 learned_addr learned my addr 172.21.15.66:0/4097904344 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc763e00640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc76c0cde70 0x7fc76c0c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc769e00640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc76c0cda90 0x7fc76c0c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc76a800640 1 -- 172.21.15.66:0/4097904344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc76c0cda90 msgr2=0x7fc76c0c1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc76a800640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc76c0cda90 0x7fc76c0c1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc76a800640 1 -- 172.21.15.66:0/4097904344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc76c0cde70 msgr2=0x7fc76c0c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc76a800640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc76c0cde70 0x7fc76c0c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc76a800640 1 -- 172.21.15.66:0/4097904344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7540094d0 con 0x7fc76c159c60 2024-03-30T00:44:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc769e00640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc76c0cda90 0x7fc76c0c1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc76a800640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc76c159c60 0x7fc76c0ab010 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc76400b9c0 tx=0x7fc76400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:40.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc762000640 1 -- 172.21.15.66:0/4097904344 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76400cba0 con 0x7fc76c159c60 2024-03-30T00:44:40.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc762000640 1 -- 172.21.15.66:0/4097904344 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76400cd00 con 0x7fc76c159c60 2024-03-30T00:44:40.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc762000640 1 -- 172.21.15.66:0/4097904344 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764014680 con 0x7fc76c159c60 2024-03-30T00:44:40.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc770c00640 1 -- 172.21.15.66:0/4097904344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76c0c39e0 con 0x7fc76c159c60 2024-03-30T00:44:40.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.631+0000 7fc770c00640 1 -- 172.21.15.66:0/4097904344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c156640 con 0x7fc76c159c60 2024-03-30T00:44:40.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc762000640 1 -- 172.21.15.66:0/4097904344 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc764007370 con 0x7fc76c159c60 2024-03-30T00:44:40.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc762000640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc7480666c0 0x7fc748068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc762000640 1 -- 172.21.15.66:0/4097904344 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fc764083dd0 con 0x7fc76c159c60 2024-03-30T00:44:40.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc769e00640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc7480666c0 0x7fc748068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc77405c000 1 Processor -- start 2024-03-30T00:44:40.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc77405c000 1 -- start start 2024-03-30T00:44:40.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc769e00640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc7480666c0 0x7fc748068b80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc75c006fd0 tx=0x7fc75c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:40.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc77405c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b27d210 0x55c03b27d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc77405c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b233390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc77405c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 0x55c03b233cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc77405c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03b0f75a0 con 0x55c03b27db30 2024-03-30T00:44:40.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc77405c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03b084af0 con 0x55c03b27d210 2024-03-30T00:44:40.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc77405c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c03b0820c0 con 0x55c03b2338d0 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc763e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b233390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc763e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b233390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49278/0 (socket says 172.21.15.66:49278) 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc763e00640 1 -- 172.21.15.66:0/4217921183 learned_addr learned my addr 172.21.15.66:0/4217921183 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc76a800640 1 --2- 172.21.15.66:0/4217921183 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 0x55c03b233cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc769e00640 1 --2- 172.21.15.66:0/4217921183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b27d210 0x55c03b27d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc763e00640 1 -- 172.21.15.66:0/4217921183 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 msgr2=0x55c03b233cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc763e00640 1 --2- 172.21.15.66:0/4217921183 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 0x55c03b233cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc763e00640 1 -- 172.21.15.66:0/4217921183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b27d210 msgr2=0x55c03b27d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc763e00640 1 --2- 172.21.15.66:0/4217921183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b27d210 0x55c03b27d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc763e00640 1 -- 172.21.15.66:0/4217921183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c03ad79890 con 0x55c03b27db30 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc76a800640 1 --2- 172.21.15.66:0/4217921183 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 0x55c03b233cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc763e00640 1 --2- 172.21.15.66:0/4217921183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b233390 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc75401a080 tx=0x7fc7540a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2822dc9973948e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc75aa00640 1 -- 172.21.15.66:0/4217921183 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7540adbf0 con 0x55c03b27db30 2024-03-30T00:44:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc75aa00640 1 -- 172.21.15.66:0/4217921183 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7540add50 con 0x55c03b27db30 2024-03-30T00:44:40.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.635+0000 7fc75aa00640 1 -- 172.21.15.66:0/4217921183 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7540b9690 con 0x55c03b27db30 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 -- 172.21.15.66:0/4217921183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 msgr2=0x55c03b233390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 --2- 172.21.15.66:0/4217921183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b233390 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc75401a080 tx=0x7fc7540a7000 comp rx=0 tx=0).stop 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 -- 172.21.15.66:0/4217921183 shutdown_connections 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 --2- 172.21.15.66:0/4217921183 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 0x55c03b233cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 --2- 172.21.15.66:0/4217921183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b233390 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 --2- 172.21.15.66:0/4217921183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b27d210 0x55c03b27d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 -- 172.21.15.66:0/4217921183 >> 172.21.15.66:0/4217921183 conn(0x55c03b235830 msgr2=0x55c03b22d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 -- 172.21.15.66:0/4217921183 shutdown_connections 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 -- 172.21.15.66:0/4217921183 wait complete. 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 -- start start 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 Processor -- start 2024-03-30T00:44:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b2338d0 0x55c03b26b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b26b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b27db30 0x55c03b26bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03b07d9b0 con 0x55c03b27d210 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03b01cc00 con 0x55c03b2338d0 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c03b03b070 con 0x55c03b27db30 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 10 client.0 _ll_register_callbacks cb 0x55c03b21b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc763e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b26b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc763e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b26b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49290/0 (socket says 172.21.15.66:49290) 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc763e00640 1 -- 172.21.15.66:0/300413368 learned_addr learned my addr 172.21.15.66:0/300413368 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc769e00640 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b2338d0 0x55c03b26b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc76a800640 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b27db30 0x55c03b26bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc763e00640 1 -- 172.21.15.66:0/300413368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b27db30 msgr2=0x55c03b26bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc763e00640 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b27db30 0x55c03b26bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc763e00640 1 -- 172.21.15.66:0/300413368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b2338d0 msgr2=0x55c03b26b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc763e00640 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b2338d0 0x55c03b26b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc763e00640 1 -- 172.21.15.66:0/300413368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c0b11e0 con 0x55c03b27d210 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc763e00640 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b26b690 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc754097e30 tx=0x7fc7540046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8206d88a96cbc874 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc74f400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc74f400640 1 -- 172.21.15.66:0/300413368 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc754004240 con 0x55c03b27d210 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc74f400640 1 -- 172.21.15.66:0/300413368 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fc7540ade40 con 0x55c03b27d210 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 -- 172.21.15.66:0/300413368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c03b03a5f0 con 0x55c03b27d210 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc74f400640 1 -- 172.21.15.66:0/300413368 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7540043a0 con 0x55c03b27d210 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 10 client.7266 fetch_fsmap learned FSMap version 49 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.639+0000 7fc77405c000 1 -- 172.21.15.66:0/300413368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55c03b042ab0 con 0x55c03b27d210 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc74f400640 1 -- 172.21.15.66:0/300413368 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c03b03a5f0 con 0x55c03b27d210 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc74f400640 1 -- 172.21.15.66:0/300413368 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc754003620 con 0x55c03b27d210 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 10 client.7266 fetch_fsmap finished waiting for FSMap version 49 2024-03-30T00:44:40.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 10 client.7266 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:44:40.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 20 client.7266 populate_metadata read hostname 'smithi066' 2024-03-30T00:44:40.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55c03b26c4a0 0x55c03b28e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 4 client.7266 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:44:40.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 -- 172.21.15.66:0/300413368 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c03b07a790 con 0x55c03b26c4a0 2024-03-30T00:44:40.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc769e00640 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55c03b26c4a0 0x55c03b28e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc769e00640 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55c03b26c4a0 0x55c03b28e780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa28b9df271fd9d server_cookie=5e5ae3cec8da2e38 in_seq=0 out_seq=0 2024-03-30T00:44:40.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc74f400640 10 client.7266 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:40.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc74f400640 1 -- 172.21.15.66:0/300413368 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c03b07a790 con 0x55c03b26c4a0 2024-03-30T00:44:40.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc74f400640 10 client.7266 handle_command_reply: tid=0 2024-03-30T00:44:40.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 client.7266 shutdown 2024-03-30T00:44:40.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 -- 172.21.15.66:0/300413368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 msgr2=0x55c03b26b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b26b690 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc754097e30 tx=0x7fc7540046d0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 -- 172.21.15.66:0/300413368 shutdown_connections 2024-03-30T00:44:40.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55c03b26c4a0 0x55c03b28e780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b27db30 0x55c03b26bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b26b690 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 --2- 172.21.15.66:0/300413368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b2338d0 0x55c03b26b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc74f400640 0 client.7266 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:40.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 -- 172.21.15.66:0/300413368 >> 172.21.15.66:0/300413368 conn(0x55c03b235830 msgr2=0x55c03b234e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:40.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 -- 172.21.15.66:0/300413368 shutdown_connections 2024-03-30T00:44:40.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 1 -- 172.21.15.66:0/300413368 wait complete. 2024-03-30T00:44:40.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.643+0000 7fc77405c000 20 client.7266 trim_cache size 0 max 16384 2024-03-30T00:44:40.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc77405c000 1 Processor -- start 2024-03-30T00:44:40.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc77405c000 1 -- start start 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc77405c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b2338d0 0x55c03b22d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc77405c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b27d210 0x55c03b22dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc77405c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b28c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc77405c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03ad3a940 con 0x55c03b27db30 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc77405c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03b044c60 con 0x55c03b2338d0 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc77405c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c03b0c0ac0 con 0x55c03b27d210 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc76a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b28c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc76a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b28c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49304/0 (socket says 172.21.15.66:49304) 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc76a800640 1 -- 172.21.15.66:0/2021444556 learned_addr learned my addr 172.21.15.66:0/2021444556 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc769e00640 1 --2- 172.21.15.66:0/2021444556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b2338d0 0x55c03b22d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc763e00640 1 --2- 172.21.15.66:0/2021444556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b27d210 0x55c03b22dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc76a800640 1 -- 172.21.15.66:0/2021444556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b27d210 msgr2=0x55c03b22dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc76a800640 1 --2- 172.21.15.66:0/2021444556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b27d210 0x55c03b22dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc76a800640 1 -- 172.21.15.66:0/2021444556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b2338d0 msgr2=0x55c03b22d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc76a800640 1 --2- 172.21.15.66:0/2021444556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b2338d0 0x55c03b22d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc76a800640 1 -- 172.21.15.66:0/2021444556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c03b028c00 con 0x55c03b27db30 2024-03-30T00:44:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc763e00640 1 --2- 172.21.15.66:0/2021444556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b27d210 0x55c03b22dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc76a800640 1 --2- 172.21.15.66:0/2021444556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b28c770 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc76400b990 tx=0x7fc764058ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2576356e523c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc75b400640 1 -- 172.21.15.66:0/2021444556 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764051380 con 0x55c03b27db30 2024-03-30T00:44:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc75b400640 1 -- 172.21.15.66:0/2021444556 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7640514e0 con 0x55c03b27db30 2024-03-30T00:44:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc75b400640 1 -- 172.21.15.66:0/2021444556 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640543a0 con 0x55c03b27db30 2024-03-30T00:44:40.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc77405c000 1 -- 172.21.15.66:0/2021444556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 msgr2=0x55c03b28c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.659+0000 7fc77405c000 1 --2- 172.21.15.66:0/2021444556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b28c770 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc76400b990 tx=0x7fc764058ab0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 -- 172.21.15.66:0/2021444556 shutdown_connections 2024-03-30T00:44:40.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 --2- 172.21.15.66:0/2021444556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27db30 0x55c03b28c770 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 --2- 172.21.15.66:0/2021444556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b27d210 0x55c03b22dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 --2- 172.21.15.66:0/2021444556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b2338d0 0x55c03b22d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 -- 172.21.15.66:0/2021444556 >> 172.21.15.66:0/2021444556 conn(0x55c03b235830 msgr2=0x55c03b22cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:40.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 -- 172.21.15.66:0/2021444556 shutdown_connections 2024-03-30T00:44:40.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 -- 172.21.15.66:0/2021444556 wait complete. 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 -- start start 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 Processor -- start 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 0x55c03b2db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b2dbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b27db30 0x55c03b28eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03b07adb0 con 0x55c03b27d210 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03b046ef0 con 0x55c03b27db30 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c03b07a440 con 0x55c03b2338d0 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 10 client.0 _ll_register_callbacks cb 0x55c03b042590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc763e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b2dbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc763e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b2dbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49310/0 (socket says 172.21.15.66:49310) 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc763e00640 1 -- 172.21.15.66:0/1323826020 learned_addr learned my addr 172.21.15.66:0/1323826020 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc76a800640 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b27db30 0x55c03b28eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc769e00640 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 0x55c03b2db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc763e00640 1 -- 172.21.15.66:0/1323826020 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 msgr2=0x55c03b2db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc763e00640 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 0x55c03b2db7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc763e00640 1 -- 172.21.15.66:0/1323826020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b27db30 msgr2=0x55c03b28eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc763e00640 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b27db30 0x55c03b28eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc763e00640 1 -- 172.21.15.66:0/1323826020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c156640 con 0x55c03b27d210 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc769e00640 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 0x55c03b2db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc76a800640 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b27db30 0x55c03b28eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc763e00640 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b2dbcf0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc75401ab00 tx=0x7fc75401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18e90db2758a336 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc74f400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc74f400640 1 -- 172.21.15.66:0/1323826020 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7540cf070 con 0x55c03b27d210 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc74f400640 1 -- 172.21.15.66:0/1323826020 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fc754003ae0 con 0x55c03b27d210 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 -- 172.21.15.66:0/1323826020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c03b04d460 con 0x55c03b27d210 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc74f400640 1 -- 172.21.15.66:0/1323826020 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7540a4e70 con 0x55c03b27d210 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 10 client.7272 fetch_fsmap learned FSMap version 49 2024-03-30T00:44:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 -- 172.21.15.66:0/1323826020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55c03ad79cc0 con 0x55c03b27d210 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc74f400640 1 -- 172.21.15.66:0/1323826020 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c03b04d460 con 0x55c03b27d210 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc74f400640 1 -- 172.21.15.66:0/1323826020 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc7540d2020 con 0x55c03b27d210 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 10 client.7272 fetch_fsmap finished waiting for FSMap version 49 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 10 client.7272 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 20 client.7272 populate_metadata read hostname 'smithi066' 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55c03b28f870 0x55c03b28fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 4 client.7272 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc77405c000 1 -- 172.21.15.66:0/1323826020 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c03b0469c0 con 0x55c03b28f870 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.663+0000 7fc769e00640 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55c03b28f870 0x55c03b28fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc769e00640 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55c03b28f870 0x55c03b28fc50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f34cbc4a188c1717 server_cookie=74480636f801b06b in_seq=0 out_seq=0 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc74f400640 10 client.7272 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc74f400640 1 -- 172.21.15.66:0/1323826020 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c03b0469c0 con 0x55c03b28f870 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc74f400640 10 client.7272 handle_command_reply: tid=0 2024-03-30T00:44:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 1 client.7272 shutdown 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 1 -- 172.21.15.66:0/1323826020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 msgr2=0x55c03b2dbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b2dbcf0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc75401ab00 tx=0x7fc75401ac90 comp rx=0 tx=0).stop 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 1 -- 172.21.15.66:0/1323826020 shutdown_connections 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55c03b28f870 0x55c03b28fc50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c03b27db30 0x55c03b28eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c03b27d210 0x55c03b2dbcf0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc74f400640 0 client.7272 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 1 --2- 172.21.15.66:0/1323826020 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c03b2338d0 0x55c03b2db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 1 -- 172.21.15.66:0/1323826020 >> 172.21.15.66:0/1323826020 conn(0x55c03b235830 msgr2=0x55c03b22d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 1 -- 172.21.15.66:0/1323826020 shutdown_connections 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 1 -- 172.21.15.66:0/1323826020 wait complete. 2024-03-30T00:44:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc77405c000 20 client.7272 trim_cache size 0 max 16384 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 6996, 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 4.5777707989999996, 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 49, 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 83, 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 28.830251835999999 2024-03-30T00:44:40.679 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:44:40.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 -- 172.21.15.66:0/4097904344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc7480666c0 msgr2=0x7fc748068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc7480666c0 0x7fc748068b80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc75c006fd0 tx=0x7fc75c008040 comp rx=0 tx=0).stop 2024-03-30T00:44:40.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 -- 172.21.15.66:0/4097904344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc76c159c60 msgr2=0x7fc76c0ab010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc76c159c60 0x7fc76c0ab010 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc76400b9c0 tx=0x7fc76400be80 comp rx=0 tx=0).stop 2024-03-30T00:44:40.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 -- 172.21.15.66:0/4097904344 shutdown_connections 2024-03-30T00:44:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc7480666c0 0x7fc748068b80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc76c159c60 0x7fc76c0ab010 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc76c0cde70 0x7fc76c0c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 --2- 172.21.15.66:0/4097904344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc76c0cda90 0x7fc76c0c1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 -- 172.21.15.66:0/4097904344 >> 172.21.15.66:0/4097904344 conn(0x7fc76c069f60 msgr2=0x7fc76c15b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 -- 172.21.15.66:0/4097904344 shutdown_connections 2024-03-30T00:44:40.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.667+0000 7fc770c00640 1 -- 172.21.15.66:0/4097904344 wait complete. 2024-03-30T00:44:40.693 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:44:40.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9c5e00640 1 Processor -- start 2024-03-30T00:44:40.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9c5e00640 1 -- start start 2024-03-30T00:44:40.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9c5e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9c015a640 0x7fe9c006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9c5e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe9c00ce520 0x7fe9c00ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9c5e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe9c00cabb0 0x7fe9c00cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9c5e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c006d070 con 0x7fe9c00cabb0 2024-03-30T00:44:40.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9c5e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c00cbba0 con 0x7fe9c00ce520 2024-03-30T00:44:40.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9c5e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c006e2c0 con 0x7fe9c015a640 2024-03-30T00:44:40.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9bf400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe9c00cabb0 0x7fe9c00cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9bf400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe9c00cabb0 0x7fe9c00cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49322/0 (socket says 172.21.15.66:49322) 2024-03-30T00:44:40.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9bf400640 1 -- 172.21.15.66:0/1242485068 learned_addr learned my addr 172.21.15.66:0/1242485068 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:40.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9b6000640 1 --2- 172.21.15.66:0/1242485068 >> v2:172.21.15.156:3300/0 conn(0x7fe9c00ce520 0x7fe9c00ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9bea00640 1 --2- 172.21.15.66:0/1242485068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9c015a640 0x7fe9c006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9bf400640 1 -- 172.21.15.66:0/1242485068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9c015a640 msgr2=0x7fe9c006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9bf400640 1 --2- 172.21.15.66:0/1242485068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9c015a640 0x7fe9c006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9bf400640 1 -- 172.21.15.66:0/1242485068 >> v2:172.21.15.156:3300/0 conn(0x7fe9c00ce520 msgr2=0x7fe9c00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:40.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9bf400640 1 --2- 172.21.15.66:0/1242485068 >> v2:172.21.15.156:3300/0 conn(0x7fe9c00ce520 0x7fe9c00ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.827+0000 7fe9bf400640 1 -- 172.21.15.66:0/1242485068 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00cee60 con 0x7fe9c00cabb0 2024-03-30T00:44:40.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9bf400640 1 --2- 172.21.15.66:0/1242485068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9c00cabb0 0x7fe9c00cb000 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00cbe00 tx=0x7fe9b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56421500bfdaf45e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:40.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9be000640 1 -- 172.21.15.66:0/1242485068 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80aa610 con 0x7fe9c00cabb0 2024-03-30T00:44:40.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9be000640 1 -- 172.21.15.66:0/1242485068 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b80a6040 con 0x7fe9c00cabb0 2024-03-30T00:44:40.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9be000640 1 -- 172.21.15.66:0/1242485068 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80aa990 con 0x7fe9c00cabb0 2024-03-30T00:44:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9c5e00640 1 -- 172.21.15.66:0/1242485068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9c00cabb0 msgr2=0x7fe9c00cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9c5e00640 1 --2- 172.21.15.66:0/1242485068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9c00cabb0 0x7fe9c00cb000 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00cbe00 tx=0x7fe9b801c4f0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9c5e00640 1 -- 172.21.15.66:0/1242485068 shutdown_connections 2024-03-30T00:44:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9c5e00640 1 --2- 172.21.15.66:0/1242485068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9c00cabb0 0x7fe9c00cb000 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9c5e00640 1 --2- 172.21.15.66:0/1242485068 >> v2:172.21.15.156:3300/0 conn(0x7fe9c00ce520 0x7fe9c00ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9c5e00640 1 --2- 172.21.15.66:0/1242485068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9c015a640 0x7fe9c006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9c5e00640 1 -- 172.21.15.66:0/1242485068 >> 172.21.15.66:0/1242485068 conn(0x7fe9c0069f60 msgr2=0x7fe9c006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9c5e00640 1 -- 172.21.15.66:0/1242485068 shutdown_connections 2024-03-30T00:44:40.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.831+0000 7fe9c5e00640 1 -- 172.21.15.66:0/1242485068 wait complete. 2024-03-30T00:44:40.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9c5e00640 1 Processor -- start 2024-03-30T00:44:40.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9c5e00640 1 -- start start 2024-03-30T00:44:40.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9c5e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9c00cabb0 0x7fe9c00cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9c5e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c00ce520 0x7fe9c0070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9c5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9c015a640 0x7fe9c0071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9c5e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0092f90 con 0x7fe9c015a640 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9c5e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00cc9c0 con 0x7fe9c00ce520 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9c5e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0092310 con 0x7fe9c00cabb0 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9bf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9c015a640 0x7fe9c0071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9bf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9c015a640 0x7fe9c0071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49332/0 (socket says 172.21.15.66:49332) 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9bf400640 1 -- 172.21.15.66:0/3328993211 learned_addr learned my addr 172.21.15.66:0/3328993211 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9bea00640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9c00cabb0 0x7fe9c00cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9b6000640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c00ce520 0x7fe9c0070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9bf400640 1 -- 172.21.15.66:0/3328993211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9c00cabb0 msgr2=0x7fe9c00cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9bf400640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9c00cabb0 0x7fe9c00cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9bf400640 1 -- 172.21.15.66:0/3328993211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c00ce520 msgr2=0x7fe9c0070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9bf400640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c00ce520 0x7fe9c0070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9bf400640 1 -- 172.21.15.66:0/3328993211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b80094d0 con 0x7fe9c015a640 2024-03-30T00:44:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9b6000640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c00ce520 0x7fe9c0070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9bf400640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9c015a640 0x7fe9c0071460 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80ac040 tx=0x7fe9b801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9b7e00640 1 -- 172.21.15.66:0/3328993211 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80bd5f0 con 0x7fe9c015a640 2024-03-30T00:44:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9b7e00640 1 -- 172.21.15.66:0/3328993211 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b80bd750 con 0x7fe9c015a640 2024-03-30T00:44:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9b7e00640 1 -- 172.21.15.66:0/3328993211 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80bdb00 con 0x7fe9c015a640 2024-03-30T00:44:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9c5e00640 1 -- 172.21.15.66:0/3328993211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c00925f0 con 0x7fe9c015a640 2024-03-30T00:44:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9c5e00640 1 -- 172.21.15.66:0/3328993211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c0092bb0 con 0x7fe9c015a640 2024-03-30T00:44:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.835+0000 7fe9c5e00640 1 -- 172.21.15.66:0/3328993211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c0066c30 con 0x7fe9c015a640 2024-03-30T00:44:40.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.839+0000 7fe9b7e00640 1 -- 172.21.15.66:0/3328993211 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe9b80b33f0 con 0x7fe9c015a640 2024-03-30T00:44:40.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.839+0000 7fe9b7e00640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe99c066690 0x7fe99c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:40.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.839+0000 7fe9b7e00640 1 -- 172.21.15.66:0/3328993211 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fe9b811efc0 con 0x7fe9c015a640 2024-03-30T00:44:40.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.839+0000 7fe9bea00640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe99c066690 0x7fe99c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:40.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.839+0000 7fe9bea00640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe99c066690 0x7fe99c068b50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0003ab0 tx=0x7fe9b0009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:40.843+0000 7fe9b7e00640 1 -- 172.21.15.66:0/3328993211 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe9b80ec2b0 con 0x7fe9c015a640 2024-03-30T00:44:41.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 -- 172.21.15.66:0/3328993211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe9c00af340 con 0x7fe9c015a640 2024-03-30T00:44:41.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9b7e00640 1 -- 172.21.15.66:0/3328993211 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe9b80ef8d0 con 0x7fe9c015a640 2024-03-30T00:44:41.046 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 -- 172.21.15.66:0/3328993211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe99c066690 msgr2=0x7fe99c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe99c066690 0x7fe99c068b50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0003ab0 tx=0x7fe9b0009830 comp rx=0 tx=0).stop 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 -- 172.21.15.66:0/3328993211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9c015a640 msgr2=0x7fe9c0071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9c015a640 0x7fe9c0071460 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80ac040 tx=0x7fe9b801ad30 comp rx=0 tx=0).stop 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 -- 172.21.15.66:0/3328993211 shutdown_connections 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe99c066690 0x7fe99c068b50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9c015a640 0x7fe9c0071460 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c00ce520 0x7fe9c0070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 --2- 172.21.15.66:0/3328993211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9c00cabb0 0x7fe9c00cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 -- 172.21.15.66:0/3328993211 >> 172.21.15.66:0/3328993211 conn(0x7fe9c0069f60 msgr2=0x7fe9c00abc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 -- 172.21.15.66:0/3328993211 shutdown_connections 2024-03-30T00:44:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:41.039+0000 7fe9c5e00640 1 -- 172.21.15.66:0/3328993211 wait complete. 2024-03-30T00:44:41.067 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:44:41.067 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:44:41.067 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:44:41.067 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:44:41.067 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:41.067 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:44:41.067 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:44:41.067 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:44:41.068 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:44:41.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:41.068 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:44:41.077 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:44:41.077 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:44:41.077 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:44:41.077 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:44:41.078 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42197sec preferred_lft 42197sec 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:44:41.079 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:44:41.080 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:44:41.080 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:41.080 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:44:41.080 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:44:41.080 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:44:41.080 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:44:41.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:41.080 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:44:41.138 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:44:41.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:41.140 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:44:41.196 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:44:41.196 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:44:41.196 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:44:41.196 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:44:41.196 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:44:41.197 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:44:41.197 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:44:41.197 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:44:41.197 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:44:41.197 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:44:41.197 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:44:41.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:41.197 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:44:41.197 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:44:41.197 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:44:41.197 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:44:41.198 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:44:41.198 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:44:41.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:41.290 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:44:41.299 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:44:41.299 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:44:41.300 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:44:41.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:41.300 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:44:41.360 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:44:41.360 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:44:41.360 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:41.412 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:44:41.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:44:41.412 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:41.463 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:44:41.524 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:44:41.524 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:44:41.586 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:44:41.588 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:44:41.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:41.589 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:44:41.640 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:44:41.640 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:44:41.687 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:44:41.707 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:44:41.707 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:44:41.766 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:44:41.768 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:44:41.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:41.768 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:44:41.782 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:44:41.775+0000 7f985dc47580 -1 init, newargv = 0x55c5b5759e80 newargc=15 2024-03-30T00:44:41.782 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[45175]: starting ceph client 2024-03-30T00:44:41.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[45175]: starting fuse 2024-03-30T00:44:41.790 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:44:41.791 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:44:41.791 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:44:41.791 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:44:41.791 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:44:41.791 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:44:41.791 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:44:41.791 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:44:41.791 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:44:41.791 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:44:41.791 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:44:41.791 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:44:41.792 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:44:41.793 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:44:41.913 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.45175.asok 2024-03-30T00:44:42.097 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45175.asok 2024-03-30T00:44:42.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:42.097 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45175.asok status 2024-03-30T00:44:42.273 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:44:42.273 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:44:42.273 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:44:42.273 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:44:42.273 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:44:42.273 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:44:42.273 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:44:42.273 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "45175", 2024-03-30T00:44:42.273 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:44:42.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 7284, 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 7284 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1411658041 2024-03-30T00:44:42.274 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1411658041 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.7284 192.168.0.1:0/1411658041", 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/1411658041", 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 49, 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 83, 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:44:42.275 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:44:42.276 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:44:42.276 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:44:42.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:42.288 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:44:42.288 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:42.297 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:44:42.297 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:42.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:42.297 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:42.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:42.359 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-03-30T00:44:42.413 INFO:teuthology.orchestra.run.smithi066.stdout:file_undamaged 2024-03-30T00:44:42.414 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:44:42.414 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:44:42.414 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:44:42.414 DEBUG:teuthology.orchestra.run.smithi066:> import json 2024-03-30T00:44:42.414 DEBUG:teuthology.orchestra.run.smithi066:> import sys 2024-03-30T00:44:42.415 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:44:42.415 DEBUG:teuthology.orchestra.run.smithi066:> try: 2024-03-30T00:44:42.415 DEBUG:teuthology.orchestra.run.smithi066:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-03-30T00:44:42.415 DEBUG:teuthology.orchestra.run.smithi066:> except OSError as e: 2024-03-30T00:44:42.415 DEBUG:teuthology.orchestra.run.smithi066:> sys.exit(e.errno) 2024-03-30T00:44:42.415 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:44:42.415 DEBUG:teuthology.orchestra.run.smithi066:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-03-30T00:44:42.415 DEBUG:teuthology.orchestra.run.smithi066:> print(json.dumps( 2024-03-30T00:44:42.415 DEBUG:teuthology.orchestra.run.smithi066:> dict([(a, getattr(s, a)) for a in attrs]), 2024-03-30T00:44:42.415 DEBUG:teuthology.orchestra.run.smithi066:> indent=2)) 2024-03-30T00:44:42.415 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:44:42.707 INFO:teuthology.orchestra.run.smithi066.stderr:daemon-helper: command failed with exit status 2 2024-03-30T00:44:42.714 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-30T00:44:42.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:42.715 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-03-30T00:44:42.771 INFO:teuthology.orchestra.run.smithi066.stdout:2 2024-03-30T00:44:42.772 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:44:42.772 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:44:42.821 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:43.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:42.999+0000 7f6808a00640 1 Processor -- start 2024-03-30T00:44:43.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6808a00640 1 -- start start 2024-03-30T00:44:43.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6808a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6804159520 0x7f68040ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6808a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68040cdc40 0x7f68040b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6808a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f68040ce630 0x7f68040b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6808a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68040ccf10 con 0x7f68040ce630 2024-03-30T00:44:43.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6808a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68040cebb0 con 0x7f6804159520 2024-03-30T00:44:43.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6808a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68040cd160 con 0x7f68040cdc40 2024-03-30T00:44:43.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6803200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f68040ce630 0x7f68040b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6803200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f68040ce630 0x7f68040b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49342/0 (socket says 172.21.15.66:49342) 2024-03-30T00:44:43.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6803200640 1 -- 172.21.15.66:0/3908604099 learned_addr learned my addr 172.21.15.66:0/3908604099 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:43.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6802800640 1 --2- 172.21.15.66:0/3908604099 >> v2:172.21.15.156:3300/0 conn(0x7f6804159520 0x7f68040ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f67f9e00640 1 --2- 172.21.15.66:0/3908604099 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68040cdc40 0x7f68040b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6803200640 1 -- 172.21.15.66:0/3908604099 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68040cdc40 msgr2=0x7f68040b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:43.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6803200640 1 --2- 172.21.15.66:0/3908604099 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68040cdc40 0x7f68040b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6803200640 1 -- 172.21.15.66:0/3908604099 >> v2:172.21.15.156:3300/0 conn(0x7f6804159520 msgr2=0x7f68040ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:43.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6803200640 1 --2- 172.21.15.66:0/3908604099 >> v2:172.21.15.156:3300/0 conn(0x7f6804159520 0x7f68040ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.003+0000 7f6803200640 1 -- 172.21.15.66:0/3908604099 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68040cd440 con 0x7f68040ce630 2024-03-30T00:44:43.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6803200640 1 --2- 172.21.15.66:0/3908604099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68040ce630 0x7f68040b1300 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6804070940 tx=0x7f67fc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfb48ac40d5b4f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:43.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6801e00640 1 -- 172.21.15.66:0/3908604099 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0ab5f0 con 0x7f68040ce630 2024-03-30T00:44:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6801e00640 1 -- 172.21.15.66:0/3908604099 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc0ab750 con 0x7f68040ce630 2024-03-30T00:44:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6801e00640 1 -- 172.21.15.66:0/3908604099 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0abae0 con 0x7f68040ce630 2024-03-30T00:44:43.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6808a00640 1 -- 172.21.15.66:0/3908604099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68040ce630 msgr2=0x7f68040b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6808a00640 1 --2- 172.21.15.66:0/3908604099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68040ce630 0x7f68040b1300 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6804070940 tx=0x7f67fc01c520 comp rx=0 tx=0).stop 2024-03-30T00:44:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6808a00640 1 -- 172.21.15.66:0/3908604099 shutdown_connections 2024-03-30T00:44:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6808a00640 1 --2- 172.21.15.66:0/3908604099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68040ce630 0x7f68040b1300 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6808a00640 1 --2- 172.21.15.66:0/3908604099 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68040cdc40 0x7f68040b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6808a00640 1 --2- 172.21.15.66:0/3908604099 >> v2:172.21.15.156:3300/0 conn(0x7f6804159520 0x7f68040ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6808a00640 1 -- 172.21.15.66:0/3908604099 >> 172.21.15.66:0/3908604099 conn(0x7f68040cbdc0 msgr2=0x7f68040cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6808a00640 1 -- 172.21.15.66:0/3908604099 shutdown_connections 2024-03-30T00:44:43.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6808a00640 1 -- 172.21.15.66:0/3908604099 wait complete. 2024-03-30T00:44:43.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6808a00640 1 Processor -- start 2024-03-30T00:44:43.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.007+0000 7f6808a00640 1 -- start start 2024-03-30T00:44:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6808a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68040cdc40 0x7f6804155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6808a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68040ce630 0x7f6804151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6808a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6804159520 0x7f6804150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6808a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68040b4a10 con 0x7f68040cdc40 2024-03-30T00:44:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6808a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68040b4580 con 0x7f6804159520 2024-03-30T00:44:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6808a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6804155c10 con 0x7f68040ce630 2024-03-30T00:44:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6802800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68040cdc40 0x7f6804155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6802800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68040cdc40 0x7f6804155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49346/0 (socket says 172.21.15.66:49346) 2024-03-30T00:44:43.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6802800640 1 -- 172.21.15.66:0/624717143 learned_addr learned my addr 172.21.15.66:0/624717143 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6803200640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6804159520 0x7f6804150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f67f9e00640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68040ce630 0x7f6804151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6802800640 1 -- 172.21.15.66:0/624717143 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68040ce630 msgr2=0x7f6804151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6802800640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68040ce630 0x7f6804151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6802800640 1 -- 172.21.15.66:0/624717143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6804159520 msgr2=0x7f6804150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6802800640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6804159520 0x7f6804150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6802800640 1 -- 172.21.15.66:0/624717143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc0094d0 con 0x7f68040cdc40 2024-03-30T00:44:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6802800640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68040cdc40 0x7f6804155630 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f67f400d620 tx=0x7f67f400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:43.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f67fb400640 1 -- 172.21.15.66:0/624717143 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f40125f0 con 0x7f68040cdc40 2024-03-30T00:44:43.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f67fb400640 1 -- 172.21.15.66:0/624717143 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f4012750 con 0x7f68040cdc40 2024-03-30T00:44:43.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6808a00640 1 -- 172.21.15.66:0/624717143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68040b3f30 con 0x7f68040cdc40 2024-03-30T00:44:43.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f67fb400640 1 -- 172.21.15.66:0/624717143 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4012ae0 con 0x7f68040cdc40 2024-03-30T00:44:43.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6808a00640 1 -- 172.21.15.66:0/624717143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6804151b50 con 0x7f68040cdc40 2024-03-30T00:44:43.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f6808a00640 1 -- 172.21.15.66:0/624717143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68040b3d90 con 0x7f68040cdc40 2024-03-30T00:44:43.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.011+0000 7f67fb400640 1 -- 172.21.15.66:0/624717143 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f67f4012c40 con 0x7f68040cdc40 2024-03-30T00:44:43.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.015+0000 7f67fb400640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67e00666c0 0x7f67e0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.015+0000 7f67fb400640 1 -- 172.21.15.66:0/624717143 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f67f4084750 con 0x7f68040cdc40 2024-03-30T00:44:43.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.015+0000 7f67f9e00640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67e00666c0 0x7f67e0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.015+0000 7f67f9e00640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67e00666c0 0x7f67e0068b80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f68040717d0 tx=0x7f67ec003a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:43.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.019+0000 7f67fb400640 1 -- 172.21.15.66:0/624717143 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f68040b3d90 con 0x7f68040cdc40 2024-03-30T00:44:43.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 -- 172.21.15.66:0/624717143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68040b3d90 con 0x7f68040cdc40 2024-03-30T00:44:43.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f67fb400640 1 -- 172.21.15.66:0/624717143 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16440 (secure 0 0 0) 0x7f68040b3d90 con 0x7f68040cdc40 2024-03-30T00:44:43.190 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:43.190 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":83,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:35.079642+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:44:14.685880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-30T00:44:15.260373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.156:6835/2089872680":"2024-03-31T00:44:35.079618+0000","172.21.15.156:6833/2089872680":"2024-03-31T00:44:35.079618+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:43.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 -- 172.21.15.66:0/624717143 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67e00666c0 msgr2=0x7f67e0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67e00666c0 0x7f67e0068b80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f68040717d0 tx=0x7f67ec003a20 comp rx=0 tx=0).stop 2024-03-30T00:44:43.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 -- 172.21.15.66:0/624717143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68040cdc40 msgr2=0x7f6804155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68040cdc40 0x7f6804155630 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f67f400d620 tx=0x7f67f400dae0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 -- 172.21.15.66:0/624717143 shutdown_connections 2024-03-30T00:44:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6804159520 0x7f6804150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68040ce630 0x7f6804151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67e00666c0 0x7f67e0068b80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 --2- 172.21.15.66:0/624717143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68040cdc40 0x7f6804155630 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 -- 172.21.15.66:0/624717143 >> 172.21.15.66:0/624717143 conn(0x7f68040cbdc0 msgr2=0x7f68040cc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 -- 172.21.15.66:0/624717143 shutdown_connections 2024-03-30T00:44:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.183+0000 7f6808a00640 1 -- 172.21.15.66:0/624717143 wait complete. 2024-03-30T00:44:43.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:44:14.685880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:44:15.260373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:44:43.212 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:44:43.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.347+0000 7f21b1600640 1 Processor -- start 2024-03-30T00:44:43.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.347+0000 7f21b1600640 1 -- start start 2024-03-30T00:44:43.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.347+0000 7f21b1600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f21ac0cad00 0x7f21ac0c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.347+0000 7f21b1600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0cd2d0 0x7f21ac0cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.347+0000 7f21b1600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac0c8e80 con 0x7f21ac0cc8d0 2024-03-30T00:44:43.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.347+0000 7f21b1600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac0cbec0 con 0x7f21ac0cad00 2024-03-30T00:44:43.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.347+0000 7f21b1600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0cb620 con 0x7f21ac0cd2d0 2024-03-30T00:44:43.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.347+0000 7f21ab400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f21ac0cc8d0 0x7f21ac06c030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:44302/0 (socket says 172.21.15.66:44302) 2024-03-30T00:44:43.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.347+0000 7f21ab400640 1 -- 172.21.15.66:0/3303279319 learned_addr learned my addr 172.21.15.66:0/3303279319 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:43.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213719278 0 0) 0x7f21ac0c8e80 con 0x7f21ac0cc8d0 2024-03-30T00:44:43.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2190003530 con 0x7f21ac0cc8d0 2024-03-30T00:44:43.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aaa00640 1 --2- 172.21.15.66:0/3303279319 >> v2:172.21.15.156:3300/0 conn(0x7f21ac0cad00 0x7f21ac0c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21a3e00640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0cd2d0 0x7f21ac0cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4081975739 0 0) 0x7f2190003530 con 0x7f21ac0cc8d0 2024-03-30T00:44:43.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0cd2d0 msgr2=0x7f21ac0cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:43.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0cd2d0 0x7f21ac0cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 >> v2:172.21.15.156:3300/0 conn(0x7f21ac0cad00 msgr2=0x7f21ac0c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 --2- 172.21.15.66:0/3303279319 >> v2:172.21.15.156:3300/0 conn(0x7f21ac0cad00 0x7f21ac0c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0ca660 con 0x7f21ac0cc8d0 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f21a4001a00 con 0x7f21ac0cc8d0 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 >> v1:172.21.15.66:6789/0 conn(0x7f21ac0cc8d0 legacy=0x7f21ac06c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0cd2d0 0x7f2190003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2190003fb0 0x7f21900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2190004d10 0x7f2190005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0cbec0 con 0x7f2190004d10 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0cb620 con 0x7f2190003fb0 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219000ba90 con 0x7f21ac0cd2d0 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21ab400640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2190004d10 0x7f2190005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21aaa00640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0cd2d0 0x7f2190003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21a3e00640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2190003fb0 0x7f21900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21ab400640 1 -- 172.21.15.66:0/3303279319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0cd2d0 msgr2=0x7f2190003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:43.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21ab400640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0cd2d0 0x7f2190003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21ab400640 1 -- 172.21.15.66:0/3303279319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2190003fb0 msgr2=0x7f21900047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:43.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21ab400640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2190003fb0 0x7f21900047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.351+0000 7f21ab400640 1 -- 172.21.15.66:0/3303279319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2190003530 con 0x7f2190004d10 2024-03-30T00:44:43.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21ab400640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2190004d10 0x7f2190005190 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f21a400cb00 tx=0x7f21a401fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb61362cc90146e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:43.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a40a8a60 con 0x7f2190004d10 2024-03-30T00:44:43.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a40ac070 con 0x7f2190004d10 2024-03-30T00:44:43.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21aa000640 1 -- 172.21.15.66:0/3303279319 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a40a8e60 con 0x7f2190004d10 2024-03-30T00:44:43.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 -- 172.21.15.66:0/3303279319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2190004d10 msgr2=0x7f2190005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 --2- 172.21.15.66:0/3303279319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2190004d10 0x7f2190005190 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f21a400cb00 tx=0x7f21a401fa50 comp rx=0 tx=0).stop 2024-03-30T00:44:43.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21ab400640 1 -- 172.21.15.66:0/3303279319 reap_dead start 2024-03-30T00:44:43.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 -- 172.21.15.66:0/3303279319 shutdown_connections 2024-03-30T00:44:43.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 -- 172.21.15.66:0/3303279319 >> 172.21.15.66:0/3303279319 conn(0x7f21ac069f60 msgr2=0x7f21ac06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:43.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 -- 172.21.15.66:0/3303279319 shutdown_connections 2024-03-30T00:44:43.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 -- 172.21.15.66:0/3303279319 wait complete. 2024-03-30T00:44:43.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 Processor -- start 2024-03-30T00:44:43.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 -- start start 2024-03-30T00:44:43.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21ac1f1d90 0x7f21ac0cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0c1cc0 0x7f21ac0c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21ac1f2170 0x7f21ac0c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0ac060 con 0x7f21ac1f2170 2024-03-30T00:44:43.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0ac300 con 0x7f21ac1f1d90 2024-03-30T00:44:43.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.355+0000 7f21b1600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0c79c0 con 0x7f21ac0c1cc0 2024-03-30T00:44:43.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21ab400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21ac1f2170 0x7f21ac0c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21ab400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21ac1f2170 0x7f21ac0c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49350/0 (socket says 172.21.15.66:49350) 2024-03-30T00:44:43.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21ab400640 1 -- 172.21.15.66:0/414191842 learned_addr learned my addr 172.21.15.66:0/414191842 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:43.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21aaa00640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21ac1f1d90 0x7f21ac0cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21a3e00640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0c1cc0 0x7f21ac0c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21ab400640 1 -- 172.21.15.66:0/414191842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0c1cc0 msgr2=0x7f21ac0c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21ab400640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0c1cc0 0x7f21ac0c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21ab400640 1 -- 172.21.15.66:0/414191842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21ac1f1d90 msgr2=0x7f21ac0cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21ab400640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21ac1f1d90 0x7f21ac0cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21ab400640 1 -- 172.21.15.66:0/414191842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0c7ca0 con 0x7f21ac1f2170 2024-03-30T00:44:43.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21ab400640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21ac1f2170 0x7f21ac0c4750 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f21a400cb00 tx=0x7f2190005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21a2a00640 1 -- 172.21.15.66:0/414191842 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a40a8a60 con 0x7f21ac1f2170 2024-03-30T00:44:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21a2a00640 1 -- 172.21.15.66:0/414191842 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a40a8bc0 con 0x7f21ac1f2170 2024-03-30T00:44:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21a2a00640 1 -- 172.21.15.66:0/414191842 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a40019d0 con 0x7f21ac1f2170 2024-03-30T00:44:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21b1600640 1 -- 172.21.15.66:0/414191842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac0c7f80 con 0x7f21ac1f2170 2024-03-30T00:44:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21b1600640 1 -- 172.21.15.66:0/414191842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac0c7590 con 0x7f21ac1f2170 2024-03-30T00:44:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21b1600640 1 -- 172.21.15.66:0/414191842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac0c7780 con 0x7f21ac1f2170 2024-03-30T00:44:43.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21a2a00640 1 -- 172.21.15.66:0/414191842 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f21a40070d0 con 0x7f21ac1f2170 2024-03-30T00:44:43.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.359+0000 7f21a2a00640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f21880666c0 0x7f2188068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.363+0000 7f21a2a00640 1 -- 172.21.15.66:0/414191842 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f21a411e770 con 0x7f21ac1f2170 2024-03-30T00:44:43.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.363+0000 7f21aaa00640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f21880666c0 0x7f2188068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.363+0000 7f21aaa00640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f21880666c0 0x7f2188068b80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f219c003730 tx=0x7f219c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:43.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.367+0000 7f21a2a00640 1 -- 172.21.15.66:0/414191842 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f21ac0c7780 con 0x7f21ac1f2170 2024-03-30T00:44:43.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.527+0000 7f21b1600640 1 -- 172.21.15.66:0/414191842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21ac0c8170 con 0x7f21ac1f2170 2024-03-30T00:44:43.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.527+0000 7f21a2a00640 1 -- 172.21.15.66:0/414191842 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21ac0c8170 con 0x7f21ac1f2170 2024-03-30T00:44:43.536 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:44:43.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 -- 172.21.15.66:0/414191842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f21880666c0 msgr2=0x7f2188068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f21880666c0 0x7f2188068b80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f219c003730 tx=0x7f219c008040 comp rx=0 tx=0).stop 2024-03-30T00:44:43.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 -- 172.21.15.66:0/414191842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21ac1f2170 msgr2=0x7f21ac0c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21ac1f2170 0x7f21ac0c4750 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f21a400cb00 tx=0x7f2190005800 comp rx=0 tx=0).stop 2024-03-30T00:44:43.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 -- 172.21.15.66:0/414191842 shutdown_connections 2024-03-30T00:44:43.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f21880666c0 0x7f2188068b80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21ac1f2170 0x7f21ac0c4750 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21ac0c1cc0 0x7f21ac0c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 --2- 172.21.15.66:0/414191842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21ac1f1d90 0x7f21ac0cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 -- 172.21.15.66:0/414191842 >> 172.21.15.66:0/414191842 conn(0x7f21ac069f60 msgr2=0x7f21ac0b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:43.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 -- 172.21.15.66:0/414191842 shutdown_connections 2024-03-30T00:44:43.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.531+0000 7f21b1600640 1 -- 172.21.15.66:0/414191842 wait complete. 2024-03-30T00:44:43.559 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:44:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f4618e00640 1 Processor -- start 2024-03-30T00:44:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f4618e00640 1 -- start start 2024-03-30T00:44:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f4618e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46141610c0 0x7f46141604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f4618e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46141609f0 0x7f46140abc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f4618e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461406da20 con 0x7f4614157060 2024-03-30T00:44:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f4618e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4614157440 con 0x7f46141610c0 2024-03-30T00:44:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f4618e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4614160ea0 con 0x7f46141609f0 2024-03-30T00:44:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f460be00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46141610c0 0x7f46141604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f4612800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46141609f0 0x7f46140abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f460be00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46141610c0 0x7f46141604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:53606/0 (socket says 172.21.15.66:53606) 2024-03-30T00:44:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f4611e00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4614157060 0x7f461406d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:59470/0 (socket says 172.21.15.66:59470) 2024-03-30T00:44:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f460be00640 1 -- 172.21.15.66:0/3267403997 learned_addr learned my addr 172.21.15.66:0/3267403997 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f460be00640 1 -- 172.21.15.66:0/3267403997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46141609f0 msgr2=0x7f46140abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f460be00640 1 --2- 172.21.15.66:0/3267403997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46141609f0 0x7f46140abc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f460be00640 1 -- 172.21.15.66:0/3267403997 >> v1:172.21.15.156:6789/0 conn(0x7f4614157060 legacy=0x7f461406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f4611400640 1 -- 172.21.15.66:0/3267403997 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231171375 0 0) 0x7f4604001120 con 0x7f4614157060 2024-03-30T00:44:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.695+0000 7f460be00640 1 -- 172.21.15.66:0/3267403997 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461406dc60 con 0x7f46141610c0 2024-03-30T00:44:43.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f460be00640 1 --2- 172.21.15.66:0/3267403997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141610c0 0x7f46141604b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009870 tx=0x7f45fc01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6ae4a7bde9da7b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f4611400640 1 -- 172.21.15.66:0/3267403997 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc0ab600 con 0x7f46141610c0 2024-03-30T00:44:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f4611400640 1 -- 172.21.15.66:0/3267403997 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc0a7050 con 0x7f46141610c0 2024-03-30T00:44:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f4611400640 1 -- 172.21.15.66:0/3267403997 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc0ab980 con 0x7f46141610c0 2024-03-30T00:44:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f4618e00640 1 -- 172.21.15.66:0/3267403997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141610c0 msgr2=0x7f46141604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f4618e00640 1 --2- 172.21.15.66:0/3267403997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141610c0 0x7f46141604b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009870 tx=0x7f45fc01c440 comp rx=0 tx=0).stop 2024-03-30T00:44:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f4618e00640 1 -- 172.21.15.66:0/3267403997 shutdown_connections 2024-03-30T00:44:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f4618e00640 1 --2- 172.21.15.66:0/3267403997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46141609f0 0x7f46140abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f4618e00640 1 --2- 172.21.15.66:0/3267403997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46141610c0 0x7f46141604b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f4618e00640 1 -- 172.21.15.66:0/3267403997 >> 172.21.15.66:0/3267403997 conn(0x7f4614071750 msgr2=0x7f4614069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f4618e00640 1 -- 172.21.15.66:0/3267403997 shutdown_connections 2024-03-30T00:44:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.699+0000 7f4618e00640 1 -- 172.21.15.66:0/3267403997 wait complete. 2024-03-30T00:44:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4618e00640 1 Processor -- start 2024-03-30T00:44:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4618e00640 1 -- start start 2024-03-30T00:44:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4618e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4614157060 0x7f461414dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4618e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46141609f0 0x7f4614145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4618e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46141610c0 0x7f4614145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4618e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4614156e10 con 0x7f46141610c0 2024-03-30T00:44:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4618e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4614156890 con 0x7f4614157060 2024-03-30T00:44:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4618e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4614156b30 con 0x7f46141609f0 2024-03-30T00:44:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4612800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46141610c0 0x7f4614145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4612800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46141610c0 0x7f4614145c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49364/0 (socket says 172.21.15.66:49364) 2024-03-30T00:44:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4612800640 1 -- 172.21.15.66:0/2826195575 learned_addr learned my addr 172.21.15.66:0/2826195575 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4611e00640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4614157060 0x7f461414dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f460be00640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46141609f0 0x7f4614145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4612800640 1 -- 172.21.15.66:0/2826195575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46141609f0 msgr2=0x7f4614145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4612800640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46141609f0 0x7f4614145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4612800640 1 -- 172.21.15.66:0/2826195575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4614157060 msgr2=0x7f461414dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4612800640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4614157060 0x7f461414dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4612800640 1 -- 172.21.15.66:0/2826195575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc0094d0 con 0x7f46141610c0 2024-03-30T00:44:43.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4612800640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46141610c0 0x7f4614145c60 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f460c00efc0 tx=0x7f460c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:43.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f460a000640 1 -- 172.21.15.66:0/2826195575 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c014480 con 0x7f46141610c0 2024-03-30T00:44:43.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f460a000640 1 -- 172.21.15.66:0/2826195575 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c00f040 con 0x7f46141610c0 2024-03-30T00:44:43.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f460a000640 1 -- 172.21.15.66:0/2826195575 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c014820 con 0x7f46141610c0 2024-03-30T00:44:43.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4618e00640 1 -- 172.21.15.66:0/2826195575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461415e7b0 con 0x7f46141610c0 2024-03-30T00:44:43.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.703+0000 7f4618e00640 1 -- 172.21.15.66:0/2826195575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4614146910 con 0x7f46141610c0 2024-03-30T00:44:43.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.707+0000 7f4618e00640 1 -- 172.21.15.66:0/2826195575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461415e610 con 0x7f46141610c0 2024-03-30T00:44:43.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.707+0000 7f460a000640 1 -- 172.21.15.66:0/2826195575 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f460c014980 con 0x7f46141610c0 2024-03-30T00:44:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.707+0000 7f460a000640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f45f0066690 0x7f45f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.707+0000 7f460a000640 1 -- 172.21.15.66:0/2826195575 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f460c085080 con 0x7f46141610c0 2024-03-30T00:44:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.707+0000 7f4611e00640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f45f0066690 0x7f45f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.707+0000 7f4611e00640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f45f0066690 0x7f45f0068b50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4604000f70 tx=0x7f4604007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:43.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.711+0000 7f460a000640 1 -- 172.21.15.66:0/2826195575 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f460c0b6980 con 0x7f46141610c0 2024-03-30T00:44:43.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.871+0000 7f4618e00640 1 -- 172.21.15.66:0/2826195575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4614066c30 con 0x7f46141610c0 2024-03-30T00:44:43.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f460a000640 1 -- 172.21.15.66:0/2826195575 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f460c018020 con 0x7f46141610c0 2024-03-30T00:44:43.881 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:44:43.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 -- 172.21.15.66:0/2826195575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f45f0066690 msgr2=0x7f45f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f45f0066690 0x7f45f0068b50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4604000f70 tx=0x7f4604007df0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 -- 172.21.15.66:0/2826195575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46141610c0 msgr2=0x7f4614145c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:43.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46141610c0 0x7f4614145c60 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f460c00efc0 tx=0x7f460c00c490 comp rx=0 tx=0).stop 2024-03-30T00:44:43.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 -- 172.21.15.66:0/2826195575 shutdown_connections 2024-03-30T00:44:43.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f45f0066690 0x7f45f0068b50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46141610c0 0x7f4614145c60 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46141609f0 0x7f4614145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 --2- 172.21.15.66:0/2826195575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4614157060 0x7f461414dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:43.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 -- 172.21.15.66:0/2826195575 >> 172.21.15.66:0/2826195575 conn(0x7f4614071750 msgr2=0x7f46140afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:43.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 -- 172.21.15.66:0/2826195575 shutdown_connections 2024-03-30T00:44:43.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:43.875+0000 7f4618e00640 1 -- 172.21.15.66:0/2826195575 wait complete. 2024-03-30T00:44:43.902 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:44:44.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2516a00640 1 Processor -- start 2024-03-30T00:44:44.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2516a00640 1 -- start start 2024-03-30T00:44:44.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2516a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f251015a640 0x7f251006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2516a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f25100ce520 0x7f25100ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2516a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f25100cabb0 0x7f25100cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2516a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251006d070 con 0x7f25100ce520 2024-03-30T00:44:44.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2516a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25100cbba0 con 0x7f25100cabb0 2024-03-30T00:44:44.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2516a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251006e2c0 con 0x7f251015a640 2024-03-30T00:44:44.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2507400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f25100ce520 0x7f25100ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2507400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f25100ce520 0x7f25100ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49366/0 (socket says 172.21.15.66:49366) 2024-03-30T00:44:44.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2507400640 1 -- 172.21.15.66:0/3063637262 learned_addr learned my addr 172.21.15.66:0/3063637262 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2514a00640 1 --2- 172.21.15.66:0/3063637262 >> v2:172.21.15.156:3300/0 conn(0x7f25100cabb0 0x7f25100cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f250fe00640 1 --2- 172.21.15.66:0/3063637262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f251015a640 0x7f251006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2507400640 1 -- 172.21.15.66:0/3063637262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f251015a640 msgr2=0x7f251006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2507400640 1 --2- 172.21.15.66:0/3063637262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f251015a640 0x7f251006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2507400640 1 -- 172.21.15.66:0/3063637262 >> v2:172.21.15.156:3300/0 conn(0x7f25100cabb0 msgr2=0x7f25100cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2507400640 1 --2- 172.21.15.66:0/3063637262 >> v2:172.21.15.156:3300/0 conn(0x7f25100cabb0 0x7f25100cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.039+0000 7f2507400640 1 -- 172.21.15.66:0/3063637262 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25100cee60 con 0x7f25100ce520 2024-03-30T00:44:44.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f2507400640 1 --2- 172.21.15.66:0/3063637262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25100ce520 0x7f25100ce920 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f25100acfe0 tx=0x7f24f801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1369fa56d937fb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:44.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f250f400640 1 -- 172.21.15.66:0/3063637262 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80aa610 con 0x7f25100ce520 2024-03-30T00:44:44.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f250f400640 1 -- 172.21.15.66:0/3063637262 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f80a6040 con 0x7f25100ce520 2024-03-30T00:44:44.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f250f400640 1 -- 172.21.15.66:0/3063637262 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80aa990 con 0x7f25100ce520 2024-03-30T00:44:44.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f2516a00640 1 -- 172.21.15.66:0/3063637262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25100ce520 msgr2=0x7f25100ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f2516a00640 1 --2- 172.21.15.66:0/3063637262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25100ce520 0x7f25100ce920 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f25100acfe0 tx=0x7f24f801c4f0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f2516a00640 1 -- 172.21.15.66:0/3063637262 shutdown_connections 2024-03-30T00:44:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f2516a00640 1 --2- 172.21.15.66:0/3063637262 >> v2:172.21.15.156:3300/0 conn(0x7f25100cabb0 0x7f25100cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f2516a00640 1 --2- 172.21.15.66:0/3063637262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25100ce520 0x7f25100ce920 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f2516a00640 1 --2- 172.21.15.66:0/3063637262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f251015a640 0x7f251006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f2516a00640 1 -- 172.21.15.66:0/3063637262 >> 172.21.15.66:0/3063637262 conn(0x7f2510069f60 msgr2=0x7f251006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f2516a00640 1 -- 172.21.15.66:0/3063637262 shutdown_connections 2024-03-30T00:44:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.043+0000 7f2516a00640 1 -- 172.21.15.66:0/3063637262 wait complete. 2024-03-30T00:44:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2516a00640 1 Processor -- start 2024-03-30T00:44:44.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2516a00640 1 -- start start 2024-03-30T00:44:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2516a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25100cabb0 0x7f25100c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2516a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25100ce520 0x7f25100977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2516a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f251015a640 0x7f251015c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2516a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251015dbf0 con 0x7f25100ce520 2024-03-30T00:44:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2516a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100c4ca0 con 0x7f251015a640 2024-03-30T00:44:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2516a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251015b430 con 0x7f25100cabb0 2024-03-30T00:44:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2507400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25100ce520 0x7f25100977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2507400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25100ce520 0x7f25100977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49374/0 (socket says 172.21.15.66:49374) 2024-03-30T00:44:44.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2507400640 1 -- 172.21.15.66:0/4146809784 learned_addr learned my addr 172.21.15.66:0/4146809784 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:44.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f250fe00640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25100cabb0 0x7f25100c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2507400640 1 -- 172.21.15.66:0/4146809784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25100cabb0 msgr2=0x7f25100c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2507400640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25100cabb0 0x7f25100c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2507400640 1 -- 172.21.15.66:0/4146809784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f251015a640 msgr2=0x7f251015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2507400640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f251015a640 0x7f251015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2507400640 1 -- 172.21.15.66:0/4146809784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f80094d0 con 0x7f25100ce520 2024-03-30T00:44:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2507400640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25100ce520 0x7f25100977a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f24f801aba0 tx=0x7f24f801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f250cc00640 1 -- 172.21.15.66:0/4146809784 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80bd3f0 con 0x7f25100ce520 2024-03-30T00:44:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f250cc00640 1 -- 172.21.15.66:0/4146809784 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f80bd550 con 0x7f25100ce520 2024-03-30T00:44:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f250cc00640 1 -- 172.21.15.66:0/4146809784 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80bd8c0 con 0x7f25100ce520 2024-03-30T00:44:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2516a00640 1 -- 172.21.15.66:0/4146809784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251015bfb0 con 0x7f25100ce520 2024-03-30T00:44:44.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2516a00640 1 -- 172.21.15.66:0/4146809784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251015b680 con 0x7f25100ce520 2024-03-30T00:44:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.047+0000 7f2516a00640 1 -- 172.21.15.66:0/4146809784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2510066c30 con 0x7f25100ce520 2024-03-30T00:44:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.051+0000 7f250cc00640 1 -- 172.21.15.66:0/4146809784 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f24f80b33f0 con 0x7f25100ce520 2024-03-30T00:44:44.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.051+0000 7f250cc00640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f24ec066690 0x7f24ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.051+0000 7f250cc00640 1 -- 172.21.15.66:0/4146809784 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f24f811e720 con 0x7f25100ce520 2024-03-30T00:44:44.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.051+0000 7f250fe00640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f24ec066690 0x7f24ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.051+0000 7f250fe00640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f24ec066690 0x7f24ec068b50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2500009990 tx=0x7f2500008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:44.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.055+0000 7f250cc00640 1 -- 172.21.15.66:0/4146809784 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f24f80eb900 con 0x7f25100ce520 2024-03-30T00:44:44.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 -- 172.21.15.66:0/4146809784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2510071cc0 con 0x7f25100ce520 2024-03-30T00:44:44.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f250cc00640 1 -- 172.21.15.66:0/4146809784 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f24f80eef20 con 0x7f25100ce520 2024-03-30T00:44:44.222 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:44:44.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 -- 172.21.15.66:0/4146809784 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f24ec066690 msgr2=0x7f24ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f24ec066690 0x7f24ec068b50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2500009990 tx=0x7f2500008040 comp rx=0 tx=0).stop 2024-03-30T00:44:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 -- 172.21.15.66:0/4146809784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25100ce520 msgr2=0x7f25100977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25100ce520 0x7f25100977a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f24f801aba0 tx=0x7f24f801ad30 comp rx=0 tx=0).stop 2024-03-30T00:44:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 -- 172.21.15.66:0/4146809784 shutdown_connections 2024-03-30T00:44:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f251015a640 0x7f251015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f24ec066690 0x7f24ec068b50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25100ce520 0x7f25100977a0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 --2- 172.21.15.66:0/4146809784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25100cabb0 0x7f25100c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 -- 172.21.15.66:0/4146809784 >> 172.21.15.66:0/4146809784 conn(0x7f2510069f60 msgr2=0x7f25100aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 -- 172.21.15.66:0/4146809784 shutdown_connections 2024-03-30T00:44:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.215+0000 7f2516a00640 1 -- 172.21.15.66:0/4146809784 wait complete. 2024-03-30T00:44:44.242 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:44:44.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f1305000640 1 Processor -- start 2024-03-30T00:44:44.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f1305000640 1 -- start start 2024-03-30T00:44:44.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f1305000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f130006f8e0 0x7f13000d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f1305000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130006b0b0 con 0x7f130006e800 2024-03-30T00:44:44.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f1305000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130006b390 con 0x7f13000cc560 2024-03-30T00:44:44.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f1305000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1300160640 con 0x7f130006f8e0 2024-03-30T00:44:44.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f12f5400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f130006e800 0x7f13000ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:44318/0 (socket says 172.21.15.66:44318) 2024-03-30T00:44:44.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f12f5400640 1 -- 172.21.15.66:0/3101385760 learned_addr learned my addr 172.21.15.66:0/3101385760 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:44.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f12fe800640 1 --2- 172.21.15.66:0/3101385760 >> v2:172.21.15.66:3300/0 conn(0x7f130006f8e0 0x7f13000d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f12fd400640 1 -- 172.21.15.66:0/3101385760 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696406988 0 0) 0x7f130006b0b0 con 0x7f130006e800 2024-03-30T00:44:44.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f12fd400640 1 -- 172.21.15.66:0/3101385760 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e4003530 con 0x7f130006e800 2024-03-30T00:44:44.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f12fe800640 1 -- 172.21.15.66:0/3101385760 >> v1:172.21.15.156:6789/0 conn(0x7f13000cc560 legacy=0x7f130006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f12fe800640 1 -- 172.21.15.66:0/3101385760 >> v1:172.21.15.66:6789/0 conn(0x7f130006e800 legacy=0x7f13000ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.375+0000 7f12fe800640 1 -- 172.21.15.66:0/3101385760 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130006b670 con 0x7f130006f8e0 2024-03-30T00:44:44.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.379+0000 7f12fe800640 1 --2- 172.21.15.66:0/3101385760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f130006f8e0 0x7f13000d1060 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009d90 tx=0x7f12f801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d7da096efa391b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.379+0000 7f12fd400640 1 -- 172.21.15.66:0/3101385760 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f80a5a00 con 0x7f130006f8e0 2024-03-30T00:44:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.379+0000 7f12fd400640 1 -- 172.21.15.66:0/3101385760 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f80a5b60 con 0x7f130006f8e0 2024-03-30T00:44:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.379+0000 7f12fd400640 1 -- 172.21.15.66:0/3101385760 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f80ad440 con 0x7f130006f8e0 2024-03-30T00:44:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.379+0000 7f1305000640 1 -- 172.21.15.66:0/3101385760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f130006f8e0 msgr2=0x7f13000d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.379+0000 7f1305000640 1 --2- 172.21.15.66:0/3101385760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f130006f8e0 0x7f13000d1060 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009d90 tx=0x7f12f801c940 comp rx=0 tx=0).stop 2024-03-30T00:44:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.379+0000 7f1305000640 1 -- 172.21.15.66:0/3101385760 shutdown_connections 2024-03-30T00:44:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.379+0000 7f1305000640 1 --2- 172.21.15.66:0/3101385760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f130006f8e0 0x7f13000d1060 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.379+0000 7f1305000640 1 -- 172.21.15.66:0/3101385760 >> 172.21.15.66:0/3101385760 conn(0x7f13000d0540 msgr2=0x7f13000d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.379+0000 7f1305000640 1 -- 172.21.15.66:0/3101385760 shutdown_connections 2024-03-30T00:44:44.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.379+0000 7f1305000640 1 -- 172.21.15.66:0/3101385760 wait complete. 2024-03-30T00:44:44.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f1305000640 1 Processor -- start 2024-03-30T00:44:44.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f1305000640 1 -- start start 2024-03-30T00:44:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f1305000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f130006e800 0x7f13000aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f1305000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f130006f8e0 0x7f1300151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f1305000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13000cc560 0x7f130006ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f1305000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13000cbcd0 con 0x7f13000cc560 2024-03-30T00:44:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f1305000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13000c97a0 con 0x7f130006e800 2024-03-30T00:44:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f1305000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13000ca320 con 0x7f130006f8e0 2024-03-30T00:44:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12fe800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13000cc560 0x7f130006ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12fe800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13000cc560 0x7f130006ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49386/0 (socket says 172.21.15.66:49386) 2024-03-30T00:44:44.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12fe800640 1 -- 172.21.15.66:0/3530895322 learned_addr learned my addr 172.21.15.66:0/3530895322 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12fde00640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f130006e800 0x7f13000aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12f5400640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f130006f8e0 0x7f1300151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12fe800640 1 -- 172.21.15.66:0/3530895322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f130006f8e0 msgr2=0x7f1300151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12fe800640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f130006f8e0 0x7f1300151b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12fe800640 1 -- 172.21.15.66:0/3530895322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f130006e800 msgr2=0x7f13000aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12fe800640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f130006e800 0x7f13000aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12fe800640 1 -- 172.21.15.66:0/3530895322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f8009a70 con 0x7f13000cc560 2024-03-30T00:44:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12fde00640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f130006e800 0x7f13000aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12fe800640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13000cc560 0x7f130006ebe0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f12f8019550 tx=0x7f12f80196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12f6a00640 1 -- 172.21.15.66:0/3530895322 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f80bf070 con 0x7f13000cc560 2024-03-30T00:44:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12f6a00640 1 -- 172.21.15.66:0/3530895322 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f8009450 con 0x7f13000cc560 2024-03-30T00:44:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f1305000640 1 -- 172.21.15.66:0/3530895322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13000c99f0 con 0x7f13000cc560 2024-03-30T00:44:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f12f6a00640 1 -- 172.21.15.66:0/3530895322 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f80aed80 con 0x7f13000cc560 2024-03-30T00:44:44.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.383+0000 7f1305000640 1 -- 172.21.15.66:0/3530895322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13000c9fb0 con 0x7f13000cc560 2024-03-30T00:44:44.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.387+0000 7f1305000640 1 -- 172.21.15.66:0/3530895322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1300066c30 con 0x7f13000cc560 2024-03-30T00:44:44.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.387+0000 7f12f6a00640 1 -- 172.21.15.66:0/3530895322 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f12f80b4050 con 0x7f13000cc560 2024-03-30T00:44:44.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.387+0000 7f12f6a00640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12dc0666c0 0x7f12dc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.387+0000 7f12f6a00640 1 -- 172.21.15.66:0/3530895322 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f12f811eed0 con 0x7f13000cc560 2024-03-30T00:44:44.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.387+0000 7f12fde00640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12dc0666c0 0x7f12dc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.387+0000 7f12fde00640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12dc0666c0 0x7f12dc068b80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f12f0000f70 tx=0x7f12f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.391+0000 7f12f6a00640 1 -- 172.21.15.66:0/3530895322 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f12f80ec000 con 0x7f13000cc560 2024-03-30T00:44:44.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.563+0000 7f1305000640 1 -- 172.21.15.66:0/3530895322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1300158920 con 0x7f13000cc560 2024-03-30T00:44:44.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.563+0000 7f12f6a00640 1 -- 172.21.15.66:0/3530895322 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v83) v1 ==== 80+0+124 (secure 0 0 0) 0x7f12f80b2020 con 0x7f13000cc560 2024-03-30T00:44:44.571 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.156:6835/2089872680 2024-03-31T00:44:35.079618+0000 2024-03-30T00:44:44.572 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.156:6833/2089872680 2024-03-31T00:44:35.079618+0000 2024-03-30T00:44:44.572 INFO:teuthology.orchestra.run.smithi066.stderr:listed 2 entries 2024-03-30T00:44:44.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 -- 172.21.15.66:0/3530895322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12dc0666c0 msgr2=0x7f12dc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12dc0666c0 0x7f12dc068b80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f12f0000f70 tx=0x7f12f0009040 comp rx=0 tx=0).stop 2024-03-30T00:44:44.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 -- 172.21.15.66:0/3530895322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13000cc560 msgr2=0x7f130006ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13000cc560 0x7f130006ebe0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f12f8019550 tx=0x7f12f80196e0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 -- 172.21.15.66:0/3530895322 shutdown_connections 2024-03-30T00:44:44.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12dc0666c0 0x7f12dc068b80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13000cc560 0x7f130006ebe0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f130006f8e0 0x7f1300151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 --2- 172.21.15.66:0/3530895322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f130006e800 0x7f13000aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 -- 172.21.15.66:0/3530895322 >> 172.21.15.66:0/3530895322 conn(0x7f13000d0540 msgr2=0x7f13000d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:44.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 -- 172.21.15.66:0/3530895322 shutdown_connections 2024-03-30T00:44:44.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.567+0000 7f1305000640 1 -- 172.21.15.66:0/3530895322 wait complete. 2024-03-30T00:44:44.592 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:44:44.592 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:44:44.601 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:44:44.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.779+0000 7fd243e00640 1 Processor -- start 2024-03-30T00:44:44.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.779+0000 7fd243e00640 1 -- start start 2024-03-30T00:44:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd243e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0cca40 0x7fd23c0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd243e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd23c099960 0x7fd23c06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd243e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23c06bac0 con 0x7fd23c06b140 2024-03-30T00:44:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd243e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23c098ac0 con 0x7fd23c099960 2024-03-30T00:44:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd243e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0cf3f0 con 0x7fd23c0cca40 2024-03-30T00:44:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd240a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd23c06b140 0x7fd23c099250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:44332/0 (socket says 172.21.15.66:44332) 2024-03-30T00:44:44.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd240a00640 1 -- 172.21.15.66:0/2800324093 learned_addr learned my addr 172.21.15.66:0/2800324093 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:44.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd241400640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0cca40 0x7fd23c0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd241e00640 1 --2- 172.21.15.66:0/2800324093 >> v2:172.21.15.156:3300/0 conn(0x7fd23c099960 0x7fd23c06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067215106 0 0) 0x7fd23c06bac0 con 0x7fd23c06b140 2024-03-30T00:44:44.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd224003530 con 0x7fd23c06b140 2024-03-30T00:44:44.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2113327521 0 0) 0x7fd224003530 con 0x7fd23c06b140 2024-03-30T00:44:44.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0cca40 msgr2=0x7fd23c0cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:44.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0cca40 0x7fd23c0cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 >> v2:172.21.15.156:3300/0 conn(0x7fd23c099960 msgr2=0x7fd23c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 --2- 172.21.15.66:0/2800324093 >> v2:172.21.15.156:3300/0 conn(0x7fd23c099960 0x7fd23c06b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c099db0 con 0x7fd23c06b140 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fd228001680 con 0x7fd23c06b140 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 >> v1:172.21.15.66:6789/0 conn(0x7fd23c06b140 legacy=0x7fd23c099250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd224003980 0x7fd224003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0cca40 0x7fd2240047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd224004d10 0x7fd224005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c098ac0 con 0x7fd224003980 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0cf3f0 con 0x7fd224004d10 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22400b990 con 0x7fd23c0cca40 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd241400640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd224003980 0x7fd224003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd240a00640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0cca40 0x7fd2240047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd241e00640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd224004d10 0x7fd224005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd241400640 1 -- 172.21.15.66:0/2800324093 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0cca40 msgr2=0x7fd2240047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd241400640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0cca40 0x7fd2240047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd241400640 1 -- 172.21.15.66:0/2800324093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd224004d10 msgr2=0x7fd224005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:44.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd241400640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd224004d10 0x7fd224005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.783+0000 7fd241400640 1 -- 172.21.15.66:0/2800324093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd224003530 con 0x7fd224003980 2024-03-30T00:44:44.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd241400640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd224003980 0x7fd224003da0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd230009ae0 tx=0x7fd23001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d786f8e0de62819b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:44.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300a55d0 con 0x7fd224003980 2024-03-30T00:44:44.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2300a5730 con 0x7fd224003980 2024-03-30T00:44:44.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd23be00640 1 -- 172.21.15.66:0/2800324093 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300a5ae0 con 0x7fd224003980 2024-03-30T00:44:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd243e00640 1 -- 172.21.15.66:0/2800324093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd224003980 msgr2=0x7fd224003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd243e00640 1 --2- 172.21.15.66:0/2800324093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd224003980 0x7fd224003da0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd230009ae0 tx=0x7fd23001ca40 comp rx=0 tx=0).stop 2024-03-30T00:44:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd241e00640 1 -- 172.21.15.66:0/2800324093 reap_dead start 2024-03-30T00:44:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd243e00640 1 -- 172.21.15.66:0/2800324093 shutdown_connections 2024-03-30T00:44:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd243e00640 1 -- 172.21.15.66:0/2800324093 >> 172.21.15.66:0/2800324093 conn(0x7fd23c0b2130 msgr2=0x7fd23c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd243e00640 1 -- 172.21.15.66:0/2800324093 shutdown_connections 2024-03-30T00:44:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd243e00640 1 -- 172.21.15.66:0/2800324093 wait complete. 2024-03-30T00:44:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd243e00640 1 Processor -- start 2024-03-30T00:44:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.787+0000 7fd243e00640 1 -- start start 2024-03-30T00:44:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd243e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd224003980 0x7fd23c159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd243e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0c4b60 0x7fd23c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd243e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd23c15c4e0 0x7fd23c158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd243e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c15a0e0 con 0x7fd23c15c4e0 2024-03-30T00:44:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd243e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c15a3c0 con 0x7fd224003980 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd243e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c15a6a0 con 0x7fd23c0c4b60 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd241e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd23c15c4e0 0x7fd23c158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd241e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd23c15c4e0 0x7fd23c158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49404/0 (socket says 172.21.15.66:49404) 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd241400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd224003980 0x7fd23c159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd241e00640 1 -- 172.21.15.66:0/2213129103 learned_addr learned my addr 172.21.15.66:0/2213129103 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd240a00640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0c4b60 0x7fd23c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd241e00640 1 -- 172.21.15.66:0/2213129103 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0c4b60 msgr2=0x7fd23c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd241e00640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0c4b60 0x7fd23c15d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd241e00640 1 -- 172.21.15.66:0/2213129103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd224003980 msgr2=0x7fd23c159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd241e00640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd224003980 0x7fd23c159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd241e00640 1 -- 172.21.15.66:0/2213129103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c15a980 con 0x7fd23c15c4e0 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd240a00640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0c4b60 0x7fd23c15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd241e00640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd23c15c4e0 0x7fd23c158ca0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd23c071550 tx=0x7fd2340115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd239600640 1 -- 172.21.15.66:0/2213129103 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400e070 con 0x7fd23c15c4e0 2024-03-30T00:44:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd239600640 1 -- 172.21.15.66:0/2213129103 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd234011e10 con 0x7fd23c15c4e0 2024-03-30T00:44:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd239600640 1 -- 172.21.15.66:0/2213129103 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340039a0 con 0x7fd23c15c4e0 2024-03-30T00:44:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd243e00640 1 -- 172.21.15.66:0/2213129103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c15ac60 con 0x7fd23c15c4e0 2024-03-30T00:44:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd243e00640 1 -- 172.21.15.66:0/2213129103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23c0c4fe0 con 0x7fd23c15c4e0 2024-03-30T00:44:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.791+0000 7fd243e00640 1 -- 172.21.15.66:0/2213129103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23c0677d0 con 0x7fd23c15c4e0 2024-03-30T00:44:44.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.795+0000 7fd239600640 1 -- 172.21.15.66:0/2213129103 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd234003b00 con 0x7fd23c15c4e0 2024-03-30T00:44:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.795+0000 7fd239600640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd21c0666c0 0x7fd21c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.795+0000 7fd239600640 1 -- 172.21.15.66:0/2213129103 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd2340847f0 con 0x7fd23c15c4e0 2024-03-30T00:44:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.795+0000 7fd241400640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd21c0666c0 0x7fd21c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.795+0000 7fd241400640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd21c0666c0 0x7fd21c068b80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd2300af000 tx=0x7fd2300046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:44.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.799+0000 7fd239600640 1 -- 172.21.15.66:0/2213129103 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd2340519b0 con 0x7fd23c15c4e0 2024-03-30T00:44:44.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.963+0000 7fd243e00640 1 -- 172.21.15.66:0/2213129103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd23c066c30 con 0x7fd23c15c4e0 2024-03-30T00:44:44.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.963+0000 7fd239600640 1 -- 172.21.15.66:0/2213129103 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16440 (secure 0 0 0) 0x7fd234054fd0 con 0x7fd23c15c4e0 2024-03-30T00:44:44.972 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:44:44.972 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":83,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:35.079642+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:44:14.685880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-30T00:44:15.260373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.156:6835/2089872680":"2024-03-31T00:44:35.079618+0000","172.21.15.156:6833/2089872680":"2024-03-31T00:44:35.079618+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:44:44.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 -- 172.21.15.66:0/2213129103 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd21c0666c0 msgr2=0x7fd21c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd21c0666c0 0x7fd21c068b80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd2300af000 tx=0x7fd2300046d0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 -- 172.21.15.66:0/2213129103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd23c15c4e0 msgr2=0x7fd23c158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:44.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd23c15c4e0 0x7fd23c158ca0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd23c071550 tx=0x7fd2340115a0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 -- 172.21.15.66:0/2213129103 shutdown_connections 2024-03-30T00:44:44.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd21c0666c0 0x7fd21c068b80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd23c15c4e0 0x7fd23c158ca0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd23c0c4b60 0x7fd23c15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 --2- 172.21.15.66:0/2213129103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd224003980 0x7fd23c159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:44.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 -- 172.21.15.66:0/2213129103 >> 172.21.15.66:0/2213129103 conn(0x7fd23c0b2130 msgr2=0x7fd23c0cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:44.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 -- 172.21.15.66:0/2213129103 shutdown_connections 2024-03-30T00:44:44.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:44.967+0000 7fd243e00640 1 -- 172.21.15.66:0/2213129103 wait complete. 2024-03-30T00:44:44.994 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:44:14.685880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:44:15.260373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:44:44.994 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:44:45.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a69200640 1 Processor -- start 2024-03-30T00:44:45.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a69200640 1 -- start start 2024-03-30T00:44:45.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a69200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5a6415ca80 0x7f5a6406b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a69200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 0x7f5a640cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a69200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a640cb0e0 con 0x7f5a640cf2f0 2024-03-30T00:44:45.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a69200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640cf7f0 con 0x7f5a6415ca80 2024-03-30T00:44:45.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a69200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640cfad0 con 0x7f5a640cb570 2024-03-30T00:44:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a62800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 0x7f5a640cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a61e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5a6415ca80 0x7f5a6406b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a62800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 0x7f5a640cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:41310/0 (socket says 172.21.15.66:41310) 2024-03-30T00:44:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a62800640 1 -- 172.21.15.66:0/1468422882 learned_addr learned my addr 172.21.15.66:0/1468422882 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:45.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a62800640 1 -- 172.21.15.66:0/1468422882 >> v2:172.21.15.156:3300/0 conn(0x7f5a6415ca80 msgr2=0x7f5a6406b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:45.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a62800640 1 --2- 172.21.15.66:0/1468422882 >> v2:172.21.15.156:3300/0 conn(0x7f5a6415ca80 0x7f5a6406b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a62800640 1 -- 172.21.15.66:0/1468422882 >> v1:172.21.15.156:6789/0 conn(0x7f5a640cf2f0 legacy=0x7f5a6406b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a62800640 1 -- 172.21.15.66:0/1468422882 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640cad70 con 0x7f5a640cb570 2024-03-30T00:44:45.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.131+0000 7f5a61e00640 1 --2- 172.21.15.66:0/1468422882 >> v2:172.21.15.156:3300/0 conn(0x7f5a6415ca80 0x7f5a6406b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a62800640 1 --2- 172.21.15.66:0/1468422882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 0x7f5a640cba20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009e50 tx=0x7f5a5c01f210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c403626265c192 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a61400640 1 -- 172.21.15.66:0/1468422882 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0af400 con 0x7f5a640cb570 2024-03-30T00:44:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a61400640 1 -- 172.21.15.66:0/1468422882 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c0ab050 con 0x7f5a640cb570 2024-03-30T00:44:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a61400640 1 -- 172.21.15.66:0/1468422882 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0af780 con 0x7f5a640cb570 2024-03-30T00:44:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a69200640 1 -- 172.21.15.66:0/1468422882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 msgr2=0x7f5a640cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a69200640 1 --2- 172.21.15.66:0/1468422882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 0x7f5a640cba20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009e50 tx=0x7f5a5c01f210 comp rx=0 tx=0).stop 2024-03-30T00:44:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a69200640 1 -- 172.21.15.66:0/1468422882 shutdown_connections 2024-03-30T00:44:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a69200640 1 --2- 172.21.15.66:0/1468422882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 0x7f5a640cba20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a69200640 1 --2- 172.21.15.66:0/1468422882 >> v2:172.21.15.156:3300/0 conn(0x7f5a6415ca80 0x7f5a6406b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a69200640 1 -- 172.21.15.66:0/1468422882 >> 172.21.15.66:0/1468422882 conn(0x7f5a640b2090 msgr2=0x7f5a640b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a69200640 1 -- 172.21.15.66:0/1468422882 shutdown_connections 2024-03-30T00:44:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.135+0000 7f5a69200640 1 -- 172.21.15.66:0/1468422882 wait complete. 2024-03-30T00:44:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a69200640 1 Processor -- start 2024-03-30T00:44:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a69200640 1 -- start start 2024-03-30T00:44:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a69200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 0x7f5a64158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a69200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a640cf2f0 0x7f5a640c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a69200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a6415ca80 0x7f5a640c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a69200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a641f6540 con 0x7f5a640cf2f0 2024-03-30T00:44:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a69200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640b6330 con 0x7f5a6415ca80 2024-03-30T00:44:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a69200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64159060 con 0x7f5a640cb570 2024-03-30T00:44:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a59400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a640cf2f0 0x7f5a640c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a59400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a640cf2f0 0x7f5a640c8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49412/0 (socket says 172.21.15.66:49412) 2024-03-30T00:44:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a59400640 1 -- 172.21.15.66:0/678854032 learned_addr learned my addr 172.21.15.66:0/678854032 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a61e00640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 0x7f5a64158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a62800640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a6415ca80 0x7f5a640c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a59400640 1 -- 172.21.15.66:0/678854032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 msgr2=0x7f5a64158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a59400640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 0x7f5a64158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a59400640 1 -- 172.21.15.66:0/678854032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a6415ca80 msgr2=0x7f5a640c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a59400640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a6415ca80 0x7f5a640c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a59400640 1 -- 172.21.15.66:0/678854032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a54009420 con 0x7f5a640cf2f0 2024-03-30T00:44:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a59400640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a640cf2f0 0x7f5a640c8f20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c420 tx=0x7f5a4c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:45.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a5aa00640 1 -- 172.21.15.66:0/678854032 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c0052d0 con 0x7f5a640cf2f0 2024-03-30T00:44:45.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a5aa00640 1 -- 172.21.15.66:0/678854032 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c005430 con 0x7f5a640cf2f0 2024-03-30T00:44:45.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a5aa00640 1 -- 172.21.15.66:0/678854032 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c007260 con 0x7f5a640cf2f0 2024-03-30T00:44:45.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a69200640 1 -- 172.21.15.66:0/678854032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c009b30 con 0x7f5a640cf2f0 2024-03-30T00:44:45.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a69200640 1 -- 172.21.15.66:0/678854032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a640b5850 con 0x7f5a640cf2f0 2024-03-30T00:44:45.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.139+0000 7f5a69200640 1 -- 172.21.15.66:0/678854032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a64066c30 con 0x7f5a640cf2f0 2024-03-30T00:44:45.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.143+0000 7f5a5aa00640 1 -- 172.21.15.66:0/678854032 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5a4c0073c0 con 0x7f5a640cf2f0 2024-03-30T00:44:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.143+0000 7f5a5aa00640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5a40066690 0x7f5a40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.143+0000 7f5a5aa00640 1 -- 172.21.15.66:0/678854032 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5a4c0836f0 con 0x7f5a640cf2f0 2024-03-30T00:44:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.143+0000 7f5a61e00640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5a40066690 0x7f5a40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.143+0000 7f5a61e00640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5a40066690 0x7f5a40068b50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009c50 tx=0x7f5a54009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.147+0000 7f5a5aa00640 1 -- 172.21.15.66:0/678854032 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5a4c0508d0 con 0x7f5a640cf2f0 2024-03-30T00:44:45.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.307+0000 7f5a69200640 1 -- 172.21.15.66:0/678854032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a640c8d00 con 0x7f5a640cf2f0 2024-03-30T00:44:45.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.307+0000 7f5a5aa00640 1 -- 172.21.15.66:0/678854032 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a4c053ef0 con 0x7f5a640cf2f0 2024-03-30T00:44:45.315 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:44:45.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 -- 172.21.15.66:0/678854032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5a40066690 msgr2=0x7f5a40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5a40066690 0x7f5a40068b50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009c50 tx=0x7f5a54009190 comp rx=0 tx=0).stop 2024-03-30T00:44:45.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 -- 172.21.15.66:0/678854032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a640cf2f0 msgr2=0x7f5a640c8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a640cf2f0 0x7f5a640c8f20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c420 tx=0x7f5a4c00c8e0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 -- 172.21.15.66:0/678854032 shutdown_connections 2024-03-30T00:44:45.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a6415ca80 0x7f5a640c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5a40066690 0x7f5a40068b50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a640cf2f0 0x7f5a640c8f20 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 --2- 172.21.15.66:0/678854032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a640cb570 0x7f5a64158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 -- 172.21.15.66:0/678854032 >> 172.21.15.66:0/678854032 conn(0x7f5a640b2090 msgr2=0x7f5a6406f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:45.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 -- 172.21.15.66:0/678854032 shutdown_connections 2024-03-30T00:44:45.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.311+0000 7f5a69200640 1 -- 172.21.15.66:0/678854032 wait complete. 2024-03-30T00:44:45.336 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:44:45.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd476600640 1 Processor -- start 2024-03-30T00:44:45.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd476600640 1 -- start start 2024-03-30T00:44:45.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd476600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4700cc560 0x7fd4700d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd476600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd47006f8e0 0x7fd47006e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd476600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd47006e630 0x7fd4700ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd476600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd470160640 con 0x7fd47006f8e0 2024-03-30T00:44:45.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd476600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47006eaa0 con 0x7fd47006e630 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd476600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47006b0b0 con 0x7fd4700cc560 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd466a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd47006f8e0 0x7fd47006e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd466a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd47006f8e0 0x7fd47006e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49420/0 (socket says 172.21.15.66:49420) 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd466a00640 1 -- 172.21.15.66:0/3970309624 learned_addr learned my addr 172.21.15.66:0/3970309624 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd46fe00640 1 --2- 172.21.15.66:0/3970309624 >> v2:172.21.15.156:3300/0 conn(0x7fd47006e630 0x7fd4700ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd46f400640 1 --2- 172.21.15.66:0/3970309624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4700cc560 0x7fd4700d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd466a00640 1 -- 172.21.15.66:0/3970309624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4700cc560 msgr2=0x7fd4700d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd466a00640 1 --2- 172.21.15.66:0/3970309624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4700cc560 0x7fd4700d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd466a00640 1 -- 172.21.15.66:0/3970309624 >> v2:172.21.15.156:3300/0 conn(0x7fd47006e630 msgr2=0x7fd4700ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd466a00640 1 --2- 172.21.15.66:0/3970309624 >> v2:172.21.15.156:3300/0 conn(0x7fd47006e630 0x7fd4700ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd466a00640 1 -- 172.21.15.66:0/3970309624 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47006b390 con 0x7fd47006f8e0 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd46f400640 1 --2- 172.21.15.66:0/3970309624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4700cc560 0x7fd4700d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:45.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.471+0000 7fd46fe00640 1 --2- 172.21.15.66:0/3970309624 >> v2:172.21.15.156:3300/0 conn(0x7fd47006e630 0x7fd4700ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:45.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd466a00640 1 --2- 172.21.15.66:0/3970309624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47006f8e0 0x7fd47006e0f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd47006dac0 tx=0x7fd45801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974b36fd38e40483 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:45.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd46ea00640 1 -- 172.21.15.66:0/3970309624 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4580ab600 con 0x7fd47006f8e0 2024-03-30T00:44:45.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd46ea00640 1 -- 172.21.15.66:0/3970309624 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4580a7050 con 0x7fd47006f8e0 2024-03-30T00:44:45.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd46ea00640 1 -- 172.21.15.66:0/3970309624 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4580ab980 con 0x7fd47006f8e0 2024-03-30T00:44:45.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd476600640 1 -- 172.21.15.66:0/3970309624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47006f8e0 msgr2=0x7fd47006e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd476600640 1 --2- 172.21.15.66:0/3970309624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47006f8e0 0x7fd47006e0f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd47006dac0 tx=0x7fd45801c230 comp rx=0 tx=0).stop 2024-03-30T00:44:45.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd476600640 1 -- 172.21.15.66:0/3970309624 shutdown_connections 2024-03-30T00:44:45.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd476600640 1 --2- 172.21.15.66:0/3970309624 >> v2:172.21.15.156:3300/0 conn(0x7fd47006e630 0x7fd4700ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd476600640 1 --2- 172.21.15.66:0/3970309624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47006f8e0 0x7fd47006e0f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd476600640 1 --2- 172.21.15.66:0/3970309624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4700cc560 0x7fd4700d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd476600640 1 -- 172.21.15.66:0/3970309624 >> 172.21.15.66:0/3970309624 conn(0x7fd4700d0540 msgr2=0x7fd4700d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:45.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd476600640 1 -- 172.21.15.66:0/3970309624 shutdown_connections 2024-03-30T00:44:45.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.475+0000 7fd476600640 1 -- 172.21.15.66:0/3970309624 wait complete. 2024-03-30T00:44:45.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd476600640 1 Processor -- start 2024-03-30T00:44:45.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd476600640 1 -- start start 2024-03-30T00:44:45.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd476600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47006e630 0x7fd4700c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd476600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd47006f8e0 0x7fd47015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd476600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4700cc560 0x7fd470155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd476600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700c7610 con 0x7fd47006e630 2024-03-30T00:44:45.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd476600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700c78b0 con 0x7fd4700cc560 2024-03-30T00:44:45.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd476600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700c7b90 con 0x7fd47006f8e0 2024-03-30T00:44:45.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd46f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47006e630 0x7fd4700c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd46f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47006e630 0x7fd4700c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49424/0 (socket says 172.21.15.66:49424) 2024-03-30T00:44:45.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd46f400640 1 -- 172.21.15.66:0/687318574 learned_addr learned my addr 172.21.15.66:0/687318574 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:45.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd46fe00640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4700cc560 0x7fd470155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd466a00640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd47006f8e0 0x7fd47015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd46f400640 1 -- 172.21.15.66:0/687318574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd47006f8e0 msgr2=0x7fd47015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd46f400640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd47006f8e0 0x7fd47015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd46f400640 1 -- 172.21.15.66:0/687318574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4700cc560 msgr2=0x7fd470155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd46f400640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4700cc560 0x7fd470155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd46f400640 1 -- 172.21.15.66:0/687318574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4580094d0 con 0x7fd47006e630 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd46fe00640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4700cc560 0x7fd470155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd466a00640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd47006f8e0 0x7fd47015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd46f400640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47006e630 0x7fd4700c94f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd460007ef0 tx=0x7fd46000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd467e00640 1 -- 172.21.15.66:0/687318574 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460003ba0 con 0x7fd47006e630 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd467e00640 1 -- 172.21.15.66:0/687318574 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46000f040 con 0x7fd47006e630 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd476600640 1 -- 172.21.15.66:0/687318574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4700c7e70 con 0x7fd47006e630 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd467e00640 1 -- 172.21.15.66:0/687318574 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46000de80 con 0x7fd47006e630 2024-03-30T00:44:45.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.479+0000 7fd476600640 1 -- 172.21.15.66:0/687318574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4700c8430 con 0x7fd47006e630 2024-03-30T00:44:45.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.483+0000 7fd476600640 1 -- 172.21.15.66:0/687318574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd470066c30 con 0x7fd47006e630 2024-03-30T00:44:45.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.483+0000 7fd467e00640 1 -- 172.21.15.66:0/687318574 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd460004030 con 0x7fd47006e630 2024-03-30T00:44:45.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.483+0000 7fd467e00640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 0x7fd44c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.483+0000 7fd467e00640 1 -- 172.21.15.66:0/687318574 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd4600845b0 con 0x7fd47006e630 2024-03-30T00:44:45.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.483+0000 7fd466a00640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 0x7fd44c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.483+0000 7fd466a00640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 0x7fd44c068b50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd4580ad000 tx=0x7fd458008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:45.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.487+0000 7fd467e00640 1 -- 172.21.15.66:0/687318574 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd460051820 con 0x7fd47006e630 2024-03-30T00:44:45.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.647+0000 7fd476600640 1 -- 172.21.15.66:0/687318574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4700b06d0 con 0x7fd47006e630 2024-03-30T00:44:45.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.647+0000 7fd467e00640 1 -- 172.21.15.66:0/687318574 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd460054e40 con 0x7fd47006e630 2024-03-30T00:44:45.657 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:44:45.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 -- 172.21.15.66:0/687318574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 msgr2=0x7fd44c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 0x7fd44c068b50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd4580ad000 tx=0x7fd458008040 comp rx=0 tx=0).stop 2024-03-30T00:44:45.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 -- 172.21.15.66:0/687318574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47006e630 msgr2=0x7fd4700c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47006e630 0x7fd4700c94f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd460007ef0 tx=0x7fd46000d4a0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 -- 172.21.15.66:0/687318574 shutdown_connections 2024-03-30T00:44:45.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4700cc560 0x7fd470155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd47006f8e0 0x7fd47015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 0x7fd44c068b50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 --2- 172.21.15.66:0/687318574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47006e630 0x7fd4700c94f0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 -- 172.21.15.66:0/687318574 >> 172.21.15.66:0/687318574 conn(0x7fd4700d0540 msgr2=0x7fd47015f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:45.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 -- 172.21.15.66:0/687318574 shutdown_connections 2024-03-30T00:44:45.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.651+0000 7fd476600640 1 -- 172.21.15.66:0/687318574 wait complete. 2024-03-30T00:44:45.678 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:44:45.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8db1600640 1 Processor -- start 2024-03-30T00:44:45.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8db1600640 1 -- start start 2024-03-30T00:44:45.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8db1600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dac0cda90 0x7f8dac0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8db1600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8dac15a370 0x7f8dac0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8db1600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dac15a990 con 0x7f8dac0cde70 2024-03-30T00:44:45.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8db1600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dac0b1b80 con 0x7f8dac15a370 2024-03-30T00:44:45.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8db1600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0cd3e0 con 0x7f8dac0cda90 2024-03-30T00:44:45.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8dab400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8dac15a370 0x7f8dac0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8daaa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dac0cda90 0x7f8dac0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8da2000640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f8dac0cde70 0x7f8dac159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:59510/0 (socket says 172.21.15.66:59510) 2024-03-30T00:44:45.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8da2000640 1 -- 172.21.15.66:0/282466619 learned_addr learned my addr 172.21.15.66:0/282466619 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:45.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8dab400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8dac15a370 0x7f8dac0cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:53676/0 (socket says 172.21.15.66:53676) 2024-03-30T00:44:45.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8daa000640 1 -- 172.21.15.66:0/282466619 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898687993 0 0) 0x7f8dac15a990 con 0x7f8dac0cde70 2024-03-30T00:44:45.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8daaa00640 1 -- 172.21.15.66:0/282466619 >> v2:172.21.15.156:3300/0 conn(0x7f8dac15a370 msgr2=0x7f8dac0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:45.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8daaa00640 1 --2- 172.21.15.66:0/282466619 >> v2:172.21.15.156:3300/0 conn(0x7f8dac15a370 0x7f8dac0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8daaa00640 1 -- 172.21.15.66:0/282466619 >> v1:172.21.15.156:6789/0 conn(0x7f8dac0cde70 legacy=0x7f8dac159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8daaa00640 1 -- 172.21.15.66:0/282466619 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac0b11e0 con 0x7f8dac0cda90 2024-03-30T00:44:45.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.815+0000 7f8dab400640 1 --2- 172.21.15.66:0/282466619 >> v2:172.21.15.156:3300/0 conn(0x7f8dac15a370 0x7f8dac0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:45.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8daaa00640 1 --2- 172.21.15.66:0/282466619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dac0cda90 0x7f8dac0ce520 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009d90 tx=0x7f8d9c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=794341c0f818896b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:45.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8daa000640 1 -- 172.21.15.66:0/282466619 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0a58b0 con 0x7f8dac0cda90 2024-03-30T00:44:45.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8daa000640 1 -- 172.21.15.66:0/282466619 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9c0a5a10 con 0x7f8dac0cda90 2024-03-30T00:44:45.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8daa000640 1 -- 172.21.15.66:0/282466619 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0a5da0 con 0x7f8dac0cda90 2024-03-30T00:44:45.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8db1600640 1 -- 172.21.15.66:0/282466619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dac0cda90 msgr2=0x7f8dac0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8db1600640 1 --2- 172.21.15.66:0/282466619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dac0cda90 0x7f8dac0ce520 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009d90 tx=0x7f8d9c01c8e0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8db1600640 1 -- 172.21.15.66:0/282466619 shutdown_connections 2024-03-30T00:44:45.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8db1600640 1 --2- 172.21.15.66:0/282466619 >> v2:172.21.15.156:3300/0 conn(0x7f8dac15a370 0x7f8dac0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8db1600640 1 --2- 172.21.15.66:0/282466619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dac0cda90 0x7f8dac0ce520 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8db1600640 1 -- 172.21.15.66:0/282466619 >> 172.21.15.66:0/282466619 conn(0x7f8dac069f60 msgr2=0x7f8dac06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:45.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8db1600640 1 -- 172.21.15.66:0/282466619 shutdown_connections 2024-03-30T00:44:45.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.819+0000 7f8db1600640 1 -- 172.21.15.66:0/282466619 wait complete. 2024-03-30T00:44:45.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8db1600640 1 Processor -- start 2024-03-30T00:44:45.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8db1600640 1 -- start start 2024-03-30T00:44:45.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8db1600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac0cda90 0x7f8dac0979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8db1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dac0cde70 0x7f8dac0af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8db1600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dac15a370 0x7f8dac0ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8db1600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac097f10 con 0x7f8dac0cde70 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8db1600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0abe20 con 0x7f8dac0cda90 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8db1600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0ac100 con 0x7f8dac15a370 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dac0cde70 0x7f8dac0af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dac0cde70 0x7f8dac0af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49426/0 (socket says 172.21.15.66:49426) 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da2000640 1 -- 172.21.15.66:0/745559414 learned_addr learned my addr 172.21.15.66:0/745559414 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8daaa00640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac0cda90 0x7f8dac0979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8dab400640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dac15a370 0x7f8dac0ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da2000640 1 -- 172.21.15.66:0/745559414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dac15a370 msgr2=0x7f8dac0ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da2000640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dac15a370 0x7f8dac0ab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da2000640 1 -- 172.21.15.66:0/745559414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac0cda90 msgr2=0x7f8dac0979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:45.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da2000640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac0cda90 0x7f8dac0979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:45.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da2000640 1 -- 172.21.15.66:0/745559414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da4009420 con 0x7f8dac0cde70 2024-03-30T00:44:45.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da2000640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dac0cde70 0x7f8dac0af530 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400c480 tx=0x7f8d9400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:45.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da3400640 1 -- 172.21.15.66:0/745559414 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d940074e0 con 0x7f8dac0cde70 2024-03-30T00:44:45.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da3400640 1 -- 172.21.15.66:0/745559414 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d94007660 con 0x7f8dac0cde70 2024-03-30T00:44:45.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8da3400640 1 -- 172.21.15.66:0/745559414 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94004a30 con 0x7f8dac0cde70 2024-03-30T00:44:45.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8db1600640 1 -- 172.21.15.66:0/745559414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c009a70 con 0x7f8dac0cde70 2024-03-30T00:44:45.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8db1600640 1 -- 172.21.15.66:0/745559414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac0ac6c0 con 0x7f8dac0cde70 2024-03-30T00:44:45.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.823+0000 7f8db1600640 1 -- 172.21.15.66:0/745559414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac066c30 con 0x7f8dac0cde70 2024-03-30T00:44:45.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.827+0000 7f8da3400640 1 -- 172.21.15.66:0/745559414 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8d94004b90 con 0x7f8dac0cde70 2024-03-30T00:44:45.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.827+0000 7f8da3400640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d88066690 0x7f8d88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:45.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.827+0000 7f8da3400640 1 -- 172.21.15.66:0/745559414 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f8d94083520 con 0x7f8dac0cde70 2024-03-30T00:44:45.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.827+0000 7f8daaa00640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d88066690 0x7f8d88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:45.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.827+0000 7f8daaa00640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d88066690 0x7f8d88068b50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0af000 tx=0x7f8d9c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:45.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.831+0000 7f8da3400640 1 -- 172.21.15.66:0/745559414 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8d94054170 con 0x7f8dac0cde70 2024-03-30T00:44:45.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.991+0000 7f8db1600640 1 -- 172.21.15.66:0/745559414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8dac1673c0 con 0x7f8dac0cde70 2024-03-30T00:44:46.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.991+0000 7f8da3400640 1 -- 172.21.15.66:0/745559414 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d94018020 con 0x7f8dac0cde70 2024-03-30T00:44:46.000 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:44:46.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 -- 172.21.15.66:0/745559414 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d88066690 msgr2=0x7f8d88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:46.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d88066690 0x7f8d88068b50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0af000 tx=0x7f8d9c0a7070 comp rx=0 tx=0).stop 2024-03-30T00:44:46.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 -- 172.21.15.66:0/745559414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dac0cde70 msgr2=0x7f8dac0af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:46.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dac0cde70 0x7f8dac0af530 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400c480 tx=0x7f8d9400c940 comp rx=0 tx=0).stop 2024-03-30T00:44:46.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 -- 172.21.15.66:0/745559414 shutdown_connections 2024-03-30T00:44:46.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dac15a370 0x7f8dac0ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d88066690 0x7f8d88068b50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dac0cde70 0x7f8dac0af530 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 --2- 172.21.15.66:0/745559414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac0cda90 0x7f8dac0979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 -- 172.21.15.66:0/745559414 >> 172.21.15.66:0/745559414 conn(0x7f8dac069f60 msgr2=0x7f8dac14eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:46.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 -- 172.21.15.66:0/745559414 shutdown_connections 2024-03-30T00:44:46.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:45.995+0000 7f8db1600640 1 -- 172.21.15.66:0/745559414 wait complete. 2024-03-30T00:44:46.021 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:44:46.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.155+0000 7f9aa4e00640 1 Processor -- start 2024-03-30T00:44:46.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.155+0000 7f9aa4e00640 1 -- start start 2024-03-30T00:44:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.155+0000 7f9aa4e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9aa00ce6b0 0x7f9aa015e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.155+0000 7f9aa4e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9aa015db90 0x7f9aa00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.155+0000 7f9aa4e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa006c410 con 0x7f9aa0070b50 2024-03-30T00:44:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.155+0000 7f9aa4e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa00cefd0 con 0x7f9aa015db90 2024-03-30T00:44:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.155+0000 7f9aa4e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa006c6b0 con 0x7f9aa00ce6b0 2024-03-30T00:44:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.159+0000 7f9a97e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9aa015db90 0x7f9aa00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.159+0000 7f9a9e800640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f9aa0070b50 0x7f9aa006d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:59514/0 (socket says 172.21.15.66:59514) 2024-03-30T00:44:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.159+0000 7f9a9de00640 1 --2- 172.21.15.66:0/2662977391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9aa00ce6b0 0x7f9aa015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.159+0000 7f9a9e800640 1 -- 172.21.15.66:0/2662977391 learned_addr learned my addr 172.21.15.66:0/2662977391 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:44:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.159+0000 7f9a97e00640 1 -- 172.21.15.66:0/2662977391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9aa00ce6b0 msgr2=0x7f9aa015e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.159+0000 7f9a97e00640 1 --2- 172.21.15.66:0/2662977391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9aa00ce6b0 0x7f9aa015e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.159+0000 7f9a97e00640 1 -- 172.21.15.66:0/2662977391 >> v1:172.21.15.156:6789/0 conn(0x7f9aa0070b50 legacy=0x7f9aa006d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.159+0000 7f9a97e00640 1 -- 172.21.15.66:0/2662977391 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa006c990 con 0x7f9aa015db90 2024-03-30T00:44:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.159+0000 7f9a9d400640 1 -- 172.21.15.66:0/2662977391 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014018882 0 0) 0x7f9aa006c410 con 0x7f9aa0070b50 2024-03-30T00:44:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.159+0000 7f9a9de00640 1 --2- 172.21.15.66:0/2662977391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9aa00ce6b0 0x7f9aa015e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:44:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.159+0000 7f9a97e00640 1 --2- 172.21.15.66:0/2662977391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9aa015db90 0x7f9aa00cea90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9a880097c0 tx=0x7f9a8801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=560b6a90b6d65fc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:46.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9a9d400640 1 -- 172.21.15.66:0/2662977391 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a880ac640 con 0x7f9aa015db90 2024-03-30T00:44:46.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9a9d400640 1 -- 172.21.15.66:0/2662977391 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a880ac7a0 con 0x7f9aa015db90 2024-03-30T00:44:46.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9a9d400640 1 -- 172.21.15.66:0/2662977391 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a880acb30 con 0x7f9aa015db90 2024-03-30T00:44:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 -- 172.21.15.66:0/2662977391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9aa015db90 msgr2=0x7f9aa00cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 --2- 172.21.15.66:0/2662977391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9aa015db90 0x7f9aa00cea90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9a880097c0 tx=0x7f9a8801c3e0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 -- 172.21.15.66:0/2662977391 shutdown_connections 2024-03-30T00:44:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 --2- 172.21.15.66:0/2662977391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9aa015db90 0x7f9aa00cea90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 --2- 172.21.15.66:0/2662977391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9aa00ce6b0 0x7f9aa015e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 -- 172.21.15.66:0/2662977391 >> 172.21.15.66:0/2662977391 conn(0x7f9aa0066100 msgr2=0x7f9aa0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 -- 172.21.15.66:0/2662977391 shutdown_connections 2024-03-30T00:44:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 -- 172.21.15.66:0/2662977391 wait complete. 2024-03-30T00:44:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 Processor -- start 2024-03-30T00:44:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 -- start start 2024-03-30T00:44:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9aa0070b50 0x7f9aa01448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9aa00ce6b0 0x7f9aa014f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9aa015db90 0x7f9aa014fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa01566f0 con 0x7f9aa0070b50 2024-03-30T00:44:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa014eed0 con 0x7f9aa00ce6b0 2024-03-30T00:44:46.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9aa4e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa01500d0 con 0x7f9aa015db90 2024-03-30T00:44:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9a9de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9aa0070b50 0x7f9aa01448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9a9de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9aa0070b50 0x7f9aa01448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49442/0 (socket says 172.21.15.66:49442) 2024-03-30T00:44:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9a9de00640 1 -- 172.21.15.66:0/457382739 learned_addr learned my addr 172.21.15.66:0/457382739 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9a9e800640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9aa015db90 0x7f9aa014fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.163+0000 7f9a97e00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9aa00ce6b0 0x7f9aa014f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a9de00640 1 -- 172.21.15.66:0/457382739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9aa015db90 msgr2=0x7f9aa014fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a9de00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9aa015db90 0x7f9aa014fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a9de00640 1 -- 172.21.15.66:0/457382739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9aa00ce6b0 msgr2=0x7f9aa014f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a9de00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9aa00ce6b0 0x7f9aa014f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a9de00640 1 -- 172.21.15.66:0/457382739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a90009420 con 0x7f9aa0070b50 2024-03-30T00:44:46.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a97e00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9aa00ce6b0 0x7f9aa014f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:46.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a9de00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9aa0070b50 0x7f9aa01448c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000ebe0 tx=0x7f9a9000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:46.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a96000640 1 -- 172.21.15.66:0/457382739 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000eee0 con 0x7f9aa0070b50 2024-03-30T00:44:46.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a96000640 1 -- 172.21.15.66:0/457382739 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9000cc30 con 0x7f9aa0070b50 2024-03-30T00:44:46.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9aa4e00640 1 -- 172.21.15.66:0/457382739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a88009420 con 0x7f9aa0070b50 2024-03-30T00:44:46.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a96000640 1 -- 172.21.15.66:0/457382739 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90013470 con 0x7f9aa0070b50 2024-03-30T00:44:46.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9aa4e00640 1 -- 172.21.15.66:0/457382739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa0156030 con 0x7f9aa0070b50 2024-03-30T00:44:46.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9aa4e00640 1 -- 172.21.15.66:0/457382739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa006e320 con 0x7f9aa0070b50 2024-03-30T00:44:46.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a96000640 1 -- 172.21.15.66:0/457382739 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9a900135d0 con 0x7f9aa0070b50 2024-03-30T00:44:46.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a96000640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a7c066690 0x7f9a7c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:46.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a96000640 1 -- 172.21.15.66:0/457382739 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f9a90084690 con 0x7f9aa0070b50 2024-03-30T00:44:46.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.167+0000 7f9a97e00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a7c066690 0x7f9a7c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:46.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.171+0000 7f9a97e00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a7c066690 0x7f9a7c068b50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9a880ae000 tx=0x7f9a880a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.171+0000 7f9a96000640 1 -- 172.21.15.66:0/457382739 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9a90051870 con 0x7f9aa0070b50 2024-03-30T00:44:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.343+0000 7f9aa4e00640 1 -- 172.21.15.66:0/457382739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9aa0066c30 con 0x7f9aa0070b50 2024-03-30T00:44:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.343+0000 7f9a96000640 1 -- 172.21.15.66:0/457382739 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v83) v1 ==== 80+0+124 (secure 0 0 0) 0x7f9a90054e90 con 0x7f9aa0070b50 2024-03-30T00:44:46.353 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.156:6835/2089872680 2024-03-31T00:44:35.079618+0000 2024-03-30T00:44:46.353 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.156:6833/2089872680 2024-03-31T00:44:35.079618+0000 2024-03-30T00:44:46.353 INFO:teuthology.orchestra.run.smithi066.stderr:listed 2 entries 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 -- 172.21.15.66:0/457382739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a7c066690 msgr2=0x7f9a7c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a7c066690 0x7f9a7c068b50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9a880ae000 tx=0x7f9a880a7000 comp rx=0 tx=0).stop 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 -- 172.21.15.66:0/457382739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9aa0070b50 msgr2=0x7f9aa01448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9aa0070b50 0x7f9aa01448c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000ebe0 tx=0x7f9a9000c520 comp rx=0 tx=0).stop 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 -- 172.21.15.66:0/457382739 shutdown_connections 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9aa015db90 0x7f9aa014fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9aa00ce6b0 0x7f9aa014f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a7c066690 0x7f9a7c068b50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 --2- 172.21.15.66:0/457382739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9aa0070b50 0x7f9aa01448c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 -- 172.21.15.66:0/457382739 >> 172.21.15.66:0/457382739 conn(0x7f9aa0066100 msgr2=0x7f9aa0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 -- 172.21.15.66:0/457382739 shutdown_connections 2024-03-30T00:44:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:46.347+0000 7f9aa4e00640 1 -- 172.21.15.66:0/457382739 wait complete. 2024-03-30T00:44:46.374 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:44:46.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:46.375 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:44:46.393 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[45175]: fuse finished with error 0 and tester_r 0 2024-03-30T00:44:46.395 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:44:46.541 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:44:46.545 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:44:50.558 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:44:50.551+0000 7f1c7fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-30T00:44:52.497 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:44:52.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:44:52.497 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:44:52.508 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-03-30T00:44:52.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.703+0000 7fca86a00640 1 Processor -- start 2024-03-30T00:44:52.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.703+0000 7fca86a00640 1 -- start start 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca86a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fca8006e7f0 0x7fca8006ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca86a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca800cbd30 0x7fca8006d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca86a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fca800cb650 0x7fca8006d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca86a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8006a860 con 0x7fca800cb650 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca86a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca800cd340 con 0x7fca8006e7f0 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca86a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca800cbb50 con 0x7fca800cbd30 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca84a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fca800cb650 0x7fca8006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca84a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fca800cb650 0x7fca8006d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58996/0 (socket says 172.21.15.66:58996) 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca84a00640 1 -- 172.21.15.66:0/2128383868 learned_addr learned my addr 172.21.15.66:0/2128383868 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca77400640 1 --2- 172.21.15.66:0/2128383868 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca800cbd30 0x7fca8006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca7fe00640 1 --2- 172.21.15.66:0/2128383868 >> v2:172.21.15.156:3300/0 conn(0x7fca8006e7f0 0x7fca8006ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca84a00640 1 -- 172.21.15.66:0/2128383868 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca800cbd30 msgr2=0x7fca8006d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca84a00640 1 --2- 172.21.15.66:0/2128383868 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca800cbd30 0x7fca8006d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca84a00640 1 -- 172.21.15.66:0/2128383868 >> v2:172.21.15.156:3300/0 conn(0x7fca8006e7f0 msgr2=0x7fca8006ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca84a00640 1 --2- 172.21.15.66:0/2128383868 >> v2:172.21.15.156:3300/0 conn(0x7fca8006e7f0 0x7fca8006ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.707+0000 7fca84a00640 1 -- 172.21.15.66:0/2128383868 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8006f110 con 0x7fca800cb650 2024-03-30T00:44:52.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca84a00640 1 --2- 172.21.15.66:0/2128383868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca800cb650 0x7fca8006d760 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fca8006fb50 tx=0x7fca7801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94025dec7ec18f3a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca7f400640 1 -- 172.21.15.66:0/2128383868 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780a57d0 con 0x7fca800cb650 2024-03-30T00:44:52.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca7f400640 1 -- 172.21.15.66:0/2128383868 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca780a5930 con 0x7fca800cb650 2024-03-30T00:44:52.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca7f400640 1 -- 172.21.15.66:0/2128383868 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780a5cc0 con 0x7fca800cb650 2024-03-30T00:44:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 -- 172.21.15.66:0/2128383868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca800cb650 msgr2=0x7fca8006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 --2- 172.21.15.66:0/2128383868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca800cb650 0x7fca8006d760 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fca8006fb50 tx=0x7fca7801c8f0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 -- 172.21.15.66:0/2128383868 shutdown_connections 2024-03-30T00:44:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 --2- 172.21.15.66:0/2128383868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca800cb650 0x7fca8006d760 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 --2- 172.21.15.66:0/2128383868 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca800cbd30 0x7fca8006d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 --2- 172.21.15.66:0/2128383868 >> v2:172.21.15.156:3300/0 conn(0x7fca8006e7f0 0x7fca8006ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 -- 172.21.15.66:0/2128383868 >> 172.21.15.66:0/2128383868 conn(0x7fca800b1030 msgr2=0x7fca800b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 -- 172.21.15.66:0/2128383868 shutdown_connections 2024-03-30T00:44:52.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 -- 172.21.15.66:0/2128383868 wait complete. 2024-03-30T00:44:52.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 Processor -- start 2024-03-30T00:44:52.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 -- start start 2024-03-30T00:44:52.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca8006e7f0 0x7fca800b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca800cb650 0x7fca800b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca800cbd30 0x7fca800c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800b5210 con 0x7fca800cbd30 2024-03-30T00:44:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800b28b0 con 0x7fca8006e7f0 2024-03-30T00:44:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.711+0000 7fca86a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca800b4f30 con 0x7fca800cb650 2024-03-30T00:44:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca84a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca800cbd30 0x7fca800c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca84a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca800cbd30 0x7fca800c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59012/0 (socket says 172.21.15.66:59012) 2024-03-30T00:44:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca84a00640 1 -- 172.21.15.66:0/97954924 learned_addr learned my addr 172.21.15.66:0/97954924 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca7fe00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca8006e7f0 0x7fca800b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca77400640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca800cb650 0x7fca800b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca84a00640 1 -- 172.21.15.66:0/97954924 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca800cb650 msgr2=0x7fca800b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca84a00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca800cb650 0x7fca800b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca84a00640 1 -- 172.21.15.66:0/97954924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca8006e7f0 msgr2=0x7fca800b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca84a00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca8006e7f0 0x7fca800b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca84a00640 1 -- 172.21.15.66:0/97954924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca78009b30 con 0x7fca800cbd30 2024-03-30T00:44:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca84a00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca800cbd30 0x7fca800c8c40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fca7801afa0 tx=0x7fca78019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca7cc00640 1 -- 172.21.15.66:0/97954924 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780b8670 con 0x7fca800cbd30 2024-03-30T00:44:52.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca7cc00640 1 -- 172.21.15.66:0/97954924 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca780b87d0 con 0x7fca800cbd30 2024-03-30T00:44:52.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca7cc00640 1 -- 172.21.15.66:0/97954924 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780b8b80 con 0x7fca800cbd30 2024-03-30T00:44:52.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca86a00640 1 -- 172.21.15.66:0/97954924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca800aedc0 con 0x7fca800cbd30 2024-03-30T00:44:52.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca86a00640 1 -- 172.21.15.66:0/97954924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca80156810 con 0x7fca800cbd30 2024-03-30T00:44:52.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.715+0000 7fca7cc00640 1 -- 172.21.15.66:0/97954924 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fca780b8ce0 con 0x7fca800cbd30 2024-03-30T00:44:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca7cc00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca5c0666c0 0x7fca5c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca7cc00640 1 -- 172.21.15.66:0/97954924 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fca7811f100 con 0x7fca800cbd30 2024-03-30T00:44:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca7fe00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca5c0666c0 0x7fca5c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca89f20000 1 Processor -- start 2024-03-30T00:44:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca89f20000 1 -- start start 2024-03-30T00:44:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca7fe00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca5c0666c0 0x7fca5c068b80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fca70009a40 tx=0x7fca70009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca89f20000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x561768fc0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca89f20000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x561768f76cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca89f20000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x561768f77610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca89f20000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768e3f3a0 con 0x561768fc11c0 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca89f20000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768dccab0 con 0x561768f77230 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca89f20000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561768dca080 con 0x561768fc08a0 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca77400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x561768f76cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca77400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x561768f76cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59024/0 (socket says 172.21.15.66:59024) 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca77400640 1 -- 172.21.15.66:0/3669235455 learned_addr learned my addr 172.21.15.66:0/3669235455 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca7fe00640 1 --2- 172.21.15.66:0/3669235455 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x561768fc0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca84a00640 1 --2- 172.21.15.66:0/3669235455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x561768f77610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca77400640 1 -- 172.21.15.66:0/3669235455 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 msgr2=0x561768fc0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca77400640 1 --2- 172.21.15.66:0/3669235455 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x561768fc0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca77400640 1 -- 172.21.15.66:0/3669235455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 msgr2=0x561768f77610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca77400640 1 --2- 172.21.15.66:0/3669235455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x561768f77610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca77400640 1 -- 172.21.15.66:0/3669235455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561768ac2000 con 0x561768fc11c0 2024-03-30T00:44:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca77400640 1 --2- 172.21.15.66:0/3669235455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x561768f76cf0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fca68009be0 tx=0x7fca68011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c0ada3c28f3080 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca74c00640 1 -- 172.21.15.66:0/3669235455 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68004030 con 0x561768fc11c0 2024-03-30T00:44:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca74c00640 1 -- 172.21.15.66:0/3669235455 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68004190 con 0x561768fc11c0 2024-03-30T00:44:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.719+0000 7fca74c00640 1 -- 172.21.15.66:0/3669235455 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68004520 con 0x561768fc11c0 2024-03-30T00:44:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 -- 172.21.15.66:0/3669235455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 msgr2=0x561768f76cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 --2- 172.21.15.66:0/3669235455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x561768f76cf0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fca68009be0 tx=0x7fca68011690 comp rx=0 tx=0).stop 2024-03-30T00:44:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 -- 172.21.15.66:0/3669235455 shutdown_connections 2024-03-30T00:44:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 --2- 172.21.15.66:0/3669235455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x561768f77610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 --2- 172.21.15.66:0/3669235455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x561768f76cf0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 --2- 172.21.15.66:0/3669235455 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x561768fc0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 -- 172.21.15.66:0/3669235455 >> 172.21.15.66:0/3669235455 conn(0x561768f7d5a0 msgr2=0x561768f755d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 -- 172.21.15.66:0/3669235455 shutdown_connections 2024-03-30T00:44:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 -- 172.21.15.66:0/3669235455 wait complete. 2024-03-30T00:44:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 -- start start 2024-03-30T00:44:52.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 Processor -- start 2024-03-30T00:44:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768f77230 0x561768f7bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768fc08a0 0x561768f7c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc11c0 0x561768f820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768dc5970 con 0x561768f77230 2024-03-30T00:44:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768d65a00 con 0x561768fc08a0 2024-03-30T00:44:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561768d656a0 con 0x561768fc11c0 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 10 client.0 _ll_register_callbacks cb 0x561768f62bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca7fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768f77230 0x561768f7bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca7fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768f77230 0x561768f7bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59034/0 (socket says 172.21.15.66:59034) 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca7fe00640 1 -- 172.21.15.66:0/3249133865 learned_addr learned my addr 172.21.15.66:0/3249133865 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca77400640 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768fc08a0 0x561768f7c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca84a00640 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc11c0 0x561768f820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca7fe00640 1 -- 172.21.15.66:0/3249133865 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc11c0 msgr2=0x561768f820a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca7fe00640 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc11c0 0x561768f820a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca7fe00640 1 -- 172.21.15.66:0/3249133865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768fc08a0 msgr2=0x561768f7c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca7fe00640 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768fc08a0 0x561768f7c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca7fe00640 1 -- 172.21.15.66:0/3249133865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca68009840 con 0x561768f77230 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca7fe00640 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768f77230 0x561768f7bd00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fca70004620 tx=0x7fca700046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23beca94f6d17609 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca6d600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:44:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca6d600640 1 -- 172.21.15.66:0/3249133865 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7001c040 con 0x561768f77230 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca6d600640 1 -- 172.21.15.66:0/3249133865 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fca70014440 con 0x561768f77230 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 -- 172.21.15.66:0/3249133865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561768d75370 con 0x561768f77230 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca6d600640 1 -- 172.21.15.66:0/3249133865 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca70003e70 con 0x561768f77230 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 10 client.7347 fetch_fsmap learned FSMap version 50 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca89f20000 1 -- 172.21.15.66:0/3249133865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x561768d8a8f0 con 0x561768f77230 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.723+0000 7fca6d600640 1 -- 172.21.15.66:0/3249133865 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561768d75370 con 0x561768f77230 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca6d600640 1 -- 172.21.15.66:0/3249133865 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fca70014600 con 0x561768f77230 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 10 client.7347 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 10 client.7347 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 20 client.7347 populate_metadata read hostname 'smithi066' 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x561768fcf930 0x561768fd1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 4 client.7347 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 -- 172.21.15.66:0/3249133865 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561768dc42c0 con 0x561768fcf930 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca77400640 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x561768fcf930 0x561768fd1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca77400640 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x561768fcf930 0x561768fd1d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4676f4843d838908 server_cookie=590e28a87ce75c27 in_seq=0 out_seq=0 2024-03-30T00:44:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca6d600640 10 client.7347 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca6d600640 1 -- 172.21.15.66:0/3249133865 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561768dc42c0 con 0x561768fcf930 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca6d600640 10 client.7347 handle_command_reply: tid=0 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 client.7347 shutdown 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 -- 172.21.15.66:0/3249133865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768f77230 msgr2=0x561768f7bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768f77230 0x561768f7bd00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fca70004620 tx=0x7fca700046d0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 -- 172.21.15.66:0/3249133865 shutdown_connections 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x561768fcf930 0x561768fd1d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc11c0 0x561768f820a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768fc08a0 0x561768f7c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 --2- 172.21.15.66:0/3249133865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768f77230 0x561768f7bd00 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca6d600640 0 client.7347 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 -- 172.21.15.66:0/3249133865 >> 172.21.15.66:0/3249133865 conn(0x561768f7d5a0 msgr2=0x561768f75cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 -- 172.21.15.66:0/3249133865 shutdown_connections 2024-03-30T00:44:52.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 1 -- 172.21.15.66:0/3249133865 wait complete. 2024-03-30T00:44:52.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.727+0000 7fca89f20000 20 client.7347 trim_cache size 0 max 16384 2024-03-30T00:44:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 Processor -- start 2024-03-30T00:44:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 -- start start 2024-03-30T00:44:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x56176901ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x56176901b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768a82c80 con 0x561768fc11c0 2024-03-30T00:44:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768dbf740 con 0x561768f77230 2024-03-30T00:44:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561768dc3180 con 0x561768fc08a0 2024-03-30T00:44:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca84a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca84a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59044/0 (socket says 172.21.15.66:59044) 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca84a00640 1 -- 172.21.15.66:0/1827196618 learned_addr learned my addr 172.21.15.66:0/1827196618 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca7fe00640 1 --2- 172.21.15.66:0/1827196618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x56176901ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca77400640 1 --2- 172.21.15.66:0/1827196618 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x56176901b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca84a00640 1 -- 172.21.15.66:0/1827196618 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 msgr2=0x56176901b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca84a00640 1 --2- 172.21.15.66:0/1827196618 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x56176901b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca84a00640 1 -- 172.21.15.66:0/1827196618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 msgr2=0x56176901ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca84a00640 1 --2- 172.21.15.66:0/1827196618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x56176901ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca84a00640 1 -- 172.21.15.66:0/1827196618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561768d70c40 con 0x561768fc11c0 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca84a00640 1 --2- 172.21.15.66:0/1827196618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901b790 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fca7801d190 tx=0x7fca780f6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6271adb3a0c0bd9c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca75600640 1 -- 172.21.15.66:0/1827196618 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780f3dc0 con 0x561768fc11c0 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca75600640 1 -- 172.21.15.66:0/1827196618 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca780f8380 con 0x561768fc11c0 2024-03-30T00:44:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca75600640 1 -- 172.21.15.66:0/1827196618 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780ec750 con 0x561768fc11c0 2024-03-30T00:44:52.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 -- 172.21.15.66:0/1827196618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 msgr2=0x56176901b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 --2- 172.21.15.66:0/1827196618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901b790 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fca7801d190 tx=0x7fca780f6040 comp rx=0 tx=0).stop 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 -- 172.21.15.66:0/1827196618 shutdown_connections 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 --2- 172.21.15.66:0/1827196618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901b790 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 --2- 172.21.15.66:0/1827196618 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x56176901b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 --2- 172.21.15.66:0/1827196618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x56176901ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 -- 172.21.15.66:0/1827196618 >> 172.21.15.66:0/1827196618 conn(0x561768f7d5a0 msgr2=0x561768fd2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 -- 172.21.15.66:0/1827196618 shutdown_connections 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 -- 172.21.15.66:0/1827196618 wait complete. 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.743+0000 7fca89f20000 1 -- start start 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 1 Processor -- start 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x561768fbf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x56176901ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768dbfb90 con 0x561768fc11c0 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768dc9970 con 0x561768f77230 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561768d6e270 con 0x561768fc08a0 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 10 client.0 _ll_register_callbacks cb 0x561768d64110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca84a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca84a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59058/0 (socket says 172.21.15.66:59058) 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca84a00640 1 -- 172.21.15.66:0/3702118182 learned_addr learned my addr 172.21.15.66:0/3702118182 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca77400640 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x56176901ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca7fe00640 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x561768fbf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca84a00640 1 -- 172.21.15.66:0/3702118182 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 msgr2=0x56176901ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca84a00640 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x56176901ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca84a00640 1 -- 172.21.15.66:0/3702118182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 msgr2=0x561768fbf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca84a00640 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x561768fbf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca84a00640 1 -- 172.21.15.66:0/3702118182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca80156810 con 0x561768fc11c0 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca84a00640 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901f240 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fca780ec540 tx=0x7fca7811d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468ed5c4fcaee8eb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca6d600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:44:52.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca6d600640 1 -- 172.21.15.66:0/3702118182 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780f04a0 con 0x561768fc11c0 2024-03-30T00:44:52.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca6d600640 1 -- 172.21.15.66:0/3702118182 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fca7811d170 con 0x561768fc11c0 2024-03-30T00:44:52.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 1 -- 172.21.15.66:0/3702118182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561768dc2750 con 0x561768fc11c0 2024-03-30T00:44:52.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca6d600640 1 -- 172.21.15.66:0/3702118182 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca780ef530 con 0x561768fc11c0 2024-03-30T00:44:52.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 10 client.7353 fetch_fsmap learned FSMap version 50 2024-03-30T00:44:52.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 1 -- 172.21.15.66:0/3702118182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x561768d64970 con 0x561768fc11c0 2024-03-30T00:44:52.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca6d600640 1 -- 172.21.15.66:0/3702118182 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561768dc2750 con 0x561768fc11c0 2024-03-30T00:44:52.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca6d600640 1 -- 172.21.15.66:0/3702118182 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fca78005d20 con 0x561768fc11c0 2024-03-30T00:44:52.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 10 client.7353 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:44:52.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 10 client.7353 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:44:52.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 20 client.7353 populate_metadata read hostname 'smithi066' 2024-03-30T00:44:52.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x561768fdb0e0 0x561768fdb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 4 client.7353 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-03-30T00:44:52.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca89f20000 1 -- 172.21.15.66:0/3702118182 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x561768d8ded0 con 0x561768fdb0e0 2024-03-30T00:44:52.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca7fe00640 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x561768fdb0e0 0x561768fdb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca6d600640 10 client.7353 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:52.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca7fe00640 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x561768fdb0e0 0x561768fdb4f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a912e41c0a32feb6 server_cookie=5a5724c5350314db in_seq=0 out_seq=0 2024-03-30T00:44:52.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.747+0000 7fca6d600640 1 -- 172.21.15.66:0/3702118182 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561768d8ded0 con 0x561768fdb0e0 2024-03-30T00:44:52.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca6d600640 10 client.7353 handle_command_reply: tid=0 2024-03-30T00:44:52.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 1 client.7353 shutdown 2024-03-30T00:44:52.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 1 -- 172.21.15.66:0/3702118182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 msgr2=0x56176901f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901f240 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fca780ec540 tx=0x7fca7811d360 comp rx=0 tx=0).stop 2024-03-30T00:44:52.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 1 -- 172.21.15.66:0/3702118182 shutdown_connections 2024-03-30T00:44:52.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x561768fdb0e0 0x561768fdb4f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561768fc11c0 0x56176901f240 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561768fc08a0 0x56176901ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca6d600640 0 client.7353 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 1 --2- 172.21.15.66:0/3702118182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561768f77230 0x561768fbf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 1 -- 172.21.15.66:0/3702118182 >> 172.21.15.66:0/3702118182 conn(0x561768f7d5a0 msgr2=0x561768fd3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 1 -- 172.21.15.66:0/3702118182 shutdown_connections 2024-03-30T00:44:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 1 -- 172.21.15.66:0/3702118182 wait complete. 2024-03-30T00:44:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca89f20000 20 client.7353 trim_cache size 0 max 16384 2024-03-30T00:44:52.761 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:44:52.761 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T00:44:52.761 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "c7f0db39-80be-4287-9367-db6af790e250", 2024-03-30T00:44:52.761 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T00:44:52.761 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:44:52.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 -- 172.21.15.66:0/97954924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca5c0666c0 msgr2=0x7fca5c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca5c0666c0 0x7fca5c068b80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fca70009a40 tx=0x7fca70009190 comp rx=0 tx=0).stop 2024-03-30T00:44:52.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 -- 172.21.15.66:0/97954924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca800cbd30 msgr2=0x7fca800c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca800cbd30 0x7fca800c8c40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fca7801afa0 tx=0x7fca78019690 comp rx=0 tx=0).stop 2024-03-30T00:44:52.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 -- 172.21.15.66:0/97954924 shutdown_connections 2024-03-30T00:44:52.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca5c0666c0 0x7fca5c068b80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca800cbd30 0x7fca800c8c40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca800cb650 0x7fca800b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 --2- 172.21.15.66:0/97954924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca8006e7f0 0x7fca800b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 -- 172.21.15.66:0/97954924 >> 172.21.15.66:0/97954924 conn(0x7fca800b1030 msgr2=0x7fca800ad5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 -- 172.21.15.66:0/97954924 shutdown_connections 2024-03-30T00:44:52.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.751+0000 7fca86a00640 1 -- 172.21.15.66:0/97954924 wait complete. 2024-03-30T00:44:52.777 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-30T00:44:52.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.911+0000 7f5667000640 1 Processor -- start 2024-03-30T00:44:52.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.911+0000 7f5667000640 1 -- start start 2024-03-30T00:44:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f5667000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015a650 0x7f56600cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f5667000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f566015aa30 0x7f566006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f5667000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566015b260 con 0x7f566015b6f0 2024-03-30T00:44:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f5667000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56600ccd40 con 0x7f566015aa30 2024-03-30T00:44:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f5667000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566006ace0 con 0x7f566015a650 2024-03-30T00:44:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f565fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015a650 0x7f56600cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f565fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015a650 0x7f56600cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:59024/0 (socket says 172.21.15.66:59024) 2024-03-30T00:44:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f565fe00640 1 -- 172.21.15.66:0/78612888 learned_addr learned my addr 172.21.15.66:0/78612888 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f565f400640 1 --2- 172.21.15.66:0/78612888 >> v2:172.21.15.156:3300/0 conn(0x7f566015aa30 0x7f566006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f565fe00640 1 -- 172.21.15.66:0/78612888 >> v2:172.21.15.156:3300/0 conn(0x7f566015aa30 msgr2=0x7f566006a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f565fe00640 1 --2- 172.21.15.66:0/78612888 >> v2:172.21.15.156:3300/0 conn(0x7f566015aa30 0x7f566006a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f565fe00640 1 -- 172.21.15.66:0/78612888 >> v1:172.21.15.156:6789/0 conn(0x7f566015b6f0 legacy=0x7f56600b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.915+0000 7f565fe00640 1 -- 172.21.15.66:0/78612888 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56600d0230 con 0x7f566015a650 2024-03-30T00:44:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f565fe00640 1 --2- 172.21.15.66:0/78612888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015a650 0x7f56600cb170 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5650009870 tx=0x7f565001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91be3386755e2d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f565ea00640 1 -- 172.21.15.66:0/78612888 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56500ab650 con 0x7f566015a650 2024-03-30T00:44:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f565ea00640 1 -- 172.21.15.66:0/78612888 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56500ab7b0 con 0x7f566015a650 2024-03-30T00:44:52.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f565ea00640 1 -- 172.21.15.66:0/78612888 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56500abb40 con 0x7f566015a650 2024-03-30T00:44:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f5667000640 1 -- 172.21.15.66:0/78612888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015a650 msgr2=0x7f56600cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f5667000640 1 --2- 172.21.15.66:0/78612888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015a650 0x7f56600cb170 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5650009870 tx=0x7f565001c560 comp rx=0 tx=0).stop 2024-03-30T00:44:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f5667000640 1 -- 172.21.15.66:0/78612888 shutdown_connections 2024-03-30T00:44:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f5667000640 1 --2- 172.21.15.66:0/78612888 >> v2:172.21.15.156:3300/0 conn(0x7f566015aa30 0x7f566006a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f5667000640 1 --2- 172.21.15.66:0/78612888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015a650 0x7f56600cb170 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f5667000640 1 -- 172.21.15.66:0/78612888 >> 172.21.15.66:0/78612888 conn(0x7f5660098a20 msgr2=0x7f56600cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f5667000640 1 -- 172.21.15.66:0/78612888 shutdown_connections 2024-03-30T00:44:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f5667000640 1 -- 172.21.15.66:0/78612888 wait complete. 2024-03-30T00:44:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f5667000640 1 Processor -- start 2024-03-30T00:44:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.919+0000 7f5667000640 1 -- start start 2024-03-30T00:44:52.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5667000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f566015a650 0x7f5660163b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5667000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f566015aa30 0x7f56601595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5667000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015b6f0 0x7f56601f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5667000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5660164df0 con 0x7f566015aa30 2024-03-30T00:44:52.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5667000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56601f5000 con 0x7f566015a650 2024-03-30T00:44:52.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5667000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56601641e0 con 0x7f566015b6f0 2024-03-30T00:44:52.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f565f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f566015aa30 0x7f56601595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f565f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f566015aa30 0x7f56601595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59062/0 (socket says 172.21.15.66:59062) 2024-03-30T00:44:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f565f400640 1 -- 172.21.15.66:0/1740141948 learned_addr learned my addr 172.21.15.66:0/1740141948 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5665000640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015b6f0 0x7f56601f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f565fe00640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f566015a650 0x7f5660163b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f565f400640 1 -- 172.21.15.66:0/1740141948 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015b6f0 msgr2=0x7f56601f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f565f400640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015b6f0 0x7f56601f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f565f400640 1 -- 172.21.15.66:0/1740141948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f566015a650 msgr2=0x7f5660163b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f565f400640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f566015a650 0x7f5660163b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f565f400640 1 -- 172.21.15.66:0/1740141948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500094d0 con 0x7f566015aa30 2024-03-30T00:44:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f565f400640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f566015aa30 0x7f56601595b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f564c00e960 tx=0x7f564c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5657e00640 1 -- 172.21.15.66:0/1740141948 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c009690 con 0x7f566015aa30 2024-03-30T00:44:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5657e00640 1 -- 172.21.15.66:0/1740141948 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c00ca00 con 0x7f566015aa30 2024-03-30T00:44:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5657e00640 1 -- 172.21.15.66:0/1740141948 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c00cd70 con 0x7f566015aa30 2024-03-30T00:44:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5667000640 1 -- 172.21.15.66:0/1740141948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56601644c0 con 0x7f566015aa30 2024-03-30T00:44:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5667000640 1 -- 172.21.15.66:0/1740141948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5660164a80 con 0x7f566015aa30 2024-03-30T00:44:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.923+0000 7f5657e00640 1 -- 172.21.15.66:0/1740141948 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f564c013470 con 0x7f566015aa30 2024-03-30T00:44:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f5657e00640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5640066690 0x7f5640068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f5657e00640 1 -- 172.21.15.66:0/1740141948 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f564c0845d0 con 0x7f566015aa30 2024-03-30T00:44:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565fe00640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5640066690 0x7f5640068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f566a422000 1 Processor -- start 2024-03-30T00:44:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f566a422000 1 -- start start 2024-03-30T00:44:52.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565fe00640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5640066690 0x7f5640068b50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f56500af000 tx=0x7f56500a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f566a422000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde736a70 0x55fcde736e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f566a422000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f566a422000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f566a422000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcde5b5630 con 0x55fcde737390 2024-03-30T00:44:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f566a422000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcde542bd0 con 0x55fcde736a70 2024-03-30T00:44:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f566a422000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fcde5401a0 con 0x55fcde6eaa00 2024-03-30T00:44:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6ea4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59066/0 (socket says 172.21.15.66:59066) 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565f400640 1 -- 172.21.15.66:0/2158956571 learned_addr learned my addr 172.21.15.66:0/2158956571 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565fe00640 1 --2- 172.21.15.66:0/2158956571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde736a70 0x55fcde736e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f5665000640 1 --2- 172.21.15.66:0/2158956571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565f400640 1 -- 172.21.15.66:0/2158956571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 msgr2=0x55fcde6eade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565f400640 1 --2- 172.21.15.66:0/2158956571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6eade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565f400640 1 -- 172.21.15.66:0/2158956571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde736a70 msgr2=0x55fcde736e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565f400640 1 --2- 172.21.15.66:0/2158956571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde736a70 0x55fcde736e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565f400640 1 -- 172.21.15.66:0/2158956571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcde237e20 con 0x55fcde737390 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f565f400640 1 --2- 172.21.15.66:0/2158956571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6ea4c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f564c00c520 tx=0x7f564c0550c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5b581cc680cab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f5654c00640 1 -- 172.21.15.66:0/2158956571 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c05d840 con 0x55fcde737390 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f5654c00640 1 -- 172.21.15.66:0/2158956571 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c053590 con 0x55fcde737390 2024-03-30T00:44:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.927+0000 7f5654c00640 1 -- 172.21.15.66:0/2158956571 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c051b60 con 0x55fcde737390 2024-03-30T00:44:52.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 -- 172.21.15.66:0/2158956571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 msgr2=0x55fcde6ea4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 --2- 172.21.15.66:0/2158956571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6ea4c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f564c00c520 tx=0x7f564c0550c0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 -- 172.21.15.66:0/2158956571 shutdown_connections 2024-03-30T00:44:52.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 --2- 172.21.15.66:0/2158956571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6eade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 --2- 172.21.15.66:0/2158956571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6ea4c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 --2- 172.21.15.66:0/2158956571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde736a70 0x55fcde736e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 -- 172.21.15.66:0/2158956571 >> 172.21.15.66:0/2158956571 conn(0x55fcde4eb160 msgr2=0x55fcde719820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 -- 172.21.15.66:0/2158956571 shutdown_connections 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 -- 172.21.15.66:0/2158956571 wait complete. 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 -- start start 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 Processor -- start 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6f3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde71aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde737390 0x55fcde71af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcde53ba90 con 0x55fcde736a70 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcde4dad40 con 0x55fcde737390 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fcde4f8f30 con 0x55fcde6eaa00 2024-03-30T00:44:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 10 client.0 _ll_register_callbacks cb 0x55fcde59d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f565f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde71aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f565f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde71aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59082/0 (socket says 172.21.15.66:59082) 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f565f400640 1 -- 172.21.15.66:0/2509813886 learned_addr learned my addr 172.21.15.66:0/2509813886 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f565fe00640 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f5665000640 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde737390 0x55fcde71af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f565f400640 1 -- 172.21.15.66:0/2509813886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 msgr2=0x55fcde6f3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f565f400640 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6f3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f565f400640 1 -- 172.21.15.66:0/2509813886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde737390 msgr2=0x55fcde71af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f565f400640 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde737390 0x55fcde71af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f565f400640 1 -- 172.21.15.66:0/2509813886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5660164a80 con 0x55fcde736a70 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f565f400640 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde71aa50 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f564c00c520 tx=0x7f564c00efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb0aecf548a7b67 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f5649600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f5649600640 1 -- 172.21.15.66:0/2509813886 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c05d8c0 con 0x55fcde736a70 2024-03-30T00:44:52.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f5649600640 1 -- 172.21.15.66:0/2509813886 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f564c0537f0 con 0x55fcde736a70 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 -- 172.21.15.66:0/2509813886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcde53a3e0 con 0x55fcde736a70 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f5649600640 1 -- 172.21.15.66:0/2509813886 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f564c073260 con 0x55fcde736a70 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 10 client.7362 fetch_fsmap learned FSMap version 50 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 -- 172.21.15.66:0/2509813886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55fcde500aa0 con 0x55fcde736a70 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f5649600640 1 -- 172.21.15.66:0/2509813886 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c078070 con 0x55fcde736a70 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f5649600640 1 -- 172.21.15.66:0/2509813886 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f564c082d20 con 0x55fcde736a70 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 10 client.7362 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 10 client.7362 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 20 client.7362 populate_metadata read hostname 'smithi066' 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.931+0000 7f566a422000 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fcde73e2d0 0x55fcde73e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 4 client.7362 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 -- 172.21.15.66:0/2509813886 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcde50bd00 con 0x55fcde73e2d0 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f565fe00640 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fcde73e2d0 0x55fcde73e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f565fe00640 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fcde73e2d0 0x55fcde73e6b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9eee3e11dad61a9 server_cookie=7f73b5428d357959 in_seq=0 out_seq=0 2024-03-30T00:44:52.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f5649600640 10 client.7362 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:52.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f5649600640 1 -- 172.21.15.66:0/2509813886 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fcde50bd00 con 0x55fcde73e2d0 2024-03-30T00:44:52.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f5649600640 10 client.7362 handle_command_reply: tid=0 2024-03-30T00:44:52.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 client.7362 shutdown 2024-03-30T00:44:52.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 -- 172.21.15.66:0/2509813886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 msgr2=0x55fcde71aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde71aa50 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f564c00c520 tx=0x7f564c00efd0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 -- 172.21.15.66:0/2509813886 shutdown_connections 2024-03-30T00:44:52.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fcde73e2d0 0x55fcde73e6b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde737390 0x55fcde71af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde71aa50 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 --2- 172.21.15.66:0/2509813886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 -- 172.21.15.66:0/2509813886 >> 172.21.15.66:0/2509813886 conn(0x55fcde4eb160 msgr2=0x55fcde6e9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f5649600640 0 client.7362 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:52.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 -- 172.21.15.66:0/2509813886 shutdown_connections 2024-03-30T00:44:52.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 1 -- 172.21.15.66:0/2509813886 wait complete. 2024-03-30T00:44:52.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.935+0000 7f566a422000 20 client.7362 trim_cache size 0 max 16384 2024-03-30T00:44:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 Processor -- start 2024-03-30T00:44:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 -- start start 2024-03-30T00:44:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6e9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde736a70 0x55fcde6ea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcde1f8aa0 con 0x55fcde737390 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcde50b0b0 con 0x55fcde736a70 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fcde507390 con 0x55fcde6eaa00 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5665000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5665000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6eade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59098/0 (socket says 172.21.15.66:59098) 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5665000640 1 -- 172.21.15.66:0/3107336827 learned_addr learned my addr 172.21.15.66:0/3107336827 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f565f400640 1 --2- 172.21.15.66:0/3107336827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde736a70 0x55fcde6ea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f565fe00640 1 --2- 172.21.15.66:0/3107336827 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6e9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5665000640 1 -- 172.21.15.66:0/3107336827 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 msgr2=0x55fcde6e9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5665000640 1 --2- 172.21.15.66:0/3107336827 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6e9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5665000640 1 -- 172.21.15.66:0/3107336827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde736a70 msgr2=0x55fcde6ea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5665000640 1 --2- 172.21.15.66:0/3107336827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde736a70 0x55fcde6ea170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5665000640 1 -- 172.21.15.66:0/3107336827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcde4e6df0 con 0x55fcde737390 2024-03-30T00:44:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f565f400640 1 --2- 172.21.15.66:0/3107336827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde736a70 0x55fcde6ea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f565fe00640 1 --2- 172.21.15.66:0/3107336827 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6e9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5665000640 1 --2- 172.21.15.66:0/3107336827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6eade0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f5658005220 tx=0x7f56580056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c652ab13a173ca0e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5655600640 1 -- 172.21.15.66:0/3107336827 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658006b60 con 0x55fcde737390 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5655600640 1 -- 172.21.15.66:0/3107336827 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5658006cc0 con 0x55fcde737390 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f5655600640 1 -- 172.21.15.66:0/3107336827 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658006e30 con 0x55fcde737390 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 -- 172.21.15.66:0/3107336827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 msgr2=0x55fcde6eade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 --2- 172.21.15.66:0/3107336827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6eade0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f5658005220 tx=0x7f56580056e0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 -- 172.21.15.66:0/3107336827 shutdown_connections 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 --2- 172.21.15.66:0/3107336827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde737390 0x55fcde6eade0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 --2- 172.21.15.66:0/3107336827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde736a70 0x55fcde6ea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 --2- 172.21.15.66:0/3107336827 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde6eaa00 0x55fcde6e9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 -- 172.21.15.66:0/3107336827 >> 172.21.15.66:0/3107336827 conn(0x55fcde4eb160 msgr2=0x55fcde6f2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 -- 172.21.15.66:0/3107336827 shutdown_connections 2024-03-30T00:44:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 -- 172.21.15.66:0/3107336827 wait complete. 2024-03-30T00:44:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 -- start start 2024-03-30T00:44:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.951+0000 7f566a422000 1 Processor -- start 2024-03-30T00:44:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde6eaa00 0x55fcde6ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde6ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde737390 0x55fcde792ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcde53fa90 con 0x55fcde736a70 2024-03-30T00:44:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcde543e60 con 0x55fcde6eaa00 2024-03-30T00:44:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fcde57f170 con 0x55fcde737390 2024-03-30T00:44:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 10 client.0 _ll_register_callbacks cb 0x7f564c051790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde6ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde6ee5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59114/0 (socket says 172.21.15.66:59114) 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565f400640 1 -- 172.21.15.66:0/82414059 learned_addr learned my addr 172.21.15.66:0/82414059 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f5665000640 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde737390 0x55fcde792ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565fe00640 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde6eaa00 0x55fcde6ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565f400640 1 -- 172.21.15.66:0/82414059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde737390 msgr2=0x55fcde792ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565f400640 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde737390 0x55fcde792ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565f400640 1 -- 172.21.15.66:0/82414059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde6eaa00 msgr2=0x55fcde6ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565f400640 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde6eaa00 0x55fcde6ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565f400640 1 -- 172.21.15.66:0/82414059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565800f040 con 0x55fcde736a70 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565fe00640 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde6eaa00 0x55fcde6ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565f400640 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde6ee5d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f564c053a20 tx=0x7f564c055220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bb4b7c73c543f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:44:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f5649600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f5649600640 1 -- 172.21.15.66:0/82414059 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c084390 con 0x55fcde736a70 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f5649600640 1 -- 172.21.15.66:0/82414059 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f564c0537f0 con 0x55fcde736a70 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 -- 172.21.15.66:0/82414059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcde4f8970 con 0x55fcde736a70 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f5649600640 1 -- 172.21.15.66:0/82414059 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f564c00e030 con 0x55fcde736a70 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 10 client.7368 fetch_fsmap learned FSMap version 50 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 -- 172.21.15.66:0/82414059 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55fcde4da960 con 0x55fcde736a70 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f5649600640 1 -- 172.21.15.66:0/82414059 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c061020 con 0x55fcde736a70 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f5649600640 1 -- 172.21.15.66:0/82414059 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f564c011070 con 0x55fcde736a70 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 10 client.7368 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 10 client.7368 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 20 client.7368 populate_metadata read hostname 'smithi066' 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fcde7933e0 0x55fcde7937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 4 client.7368 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:44:52.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 -- 172.21.15.66:0/82414059 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fcde56aa20 con 0x55fcde7933e0 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565fe00640 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fcde7933e0 0x55fcde7937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f565fe00640 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fcde7933e0 0x55fcde7937c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d47ccb6d4c008b server_cookie=8bdb8562c2b850f9 in_seq=0 out_seq=0 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f5649600640 10 client.7368 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f5649600640 1 -- 172.21.15.66:0/82414059 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fcde56aa20 con 0x55fcde7933e0 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f5649600640 10 client.7368 handle_command_reply: tid=0 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 client.7368 shutdown 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 -- 172.21.15.66:0/82414059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 msgr2=0x55fcde6ee5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde6ee5d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f564c053a20 tx=0x7f564c055220 comp rx=0 tx=0).stop 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 -- 172.21.15.66:0/82414059 shutdown_connections 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fcde7933e0 0x55fcde7937c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fcde737390 0x55fcde792ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fcde736a70 0x55fcde6ee5d0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 --2- 172.21.15.66:0/82414059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fcde6eaa00 0x55fcde6ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f5649600640 0 client.7368 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:44:52.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 -- 172.21.15.66:0/82414059 >> 172.21.15.66:0/82414059 conn(0x55fcde4eb160 msgr2=0x55fcde6e8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 -- 172.21.15.66:0/82414059 shutdown_connections 2024-03-30T00:44:52.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.955+0000 7f566a422000 1 -- 172.21.15.66:0/82414059 wait complete. 2024-03-30T00:44:52.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f566a422000 20 client.7368 trim_cache size 0 max 16384 2024-03-30T00:44:52.969 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:44:52.969 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:44:52.969 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-03-30T00:44:52.969 INFO:teuthology.orchestra.run.smithi066.stdout: "c7f0db39-80be-4287-9367-db6af790e250": { 2024-03-30T00:44:52.969 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/subdir", 2024-03-30T00:44:52.969 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "c7f0db39-80be-4287-9367-db6af790e250", 2024-03-30T00:44:52.970 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive" 2024-03-30T00:44:52.970 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:52.970 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:44:52.970 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 -- 172.21.15.66:0/1740141948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5640066690 msgr2=0x7f5640068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5640066690 0x7f5640068b50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f56500af000 tx=0x7f56500a7000 comp rx=0 tx=0).stop 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 -- 172.21.15.66:0/1740141948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f566015aa30 msgr2=0x7f56601595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f566015aa30 0x7f56601595b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f564c00e960 tx=0x7f564c00ee20 comp rx=0 tx=0).stop 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 -- 172.21.15.66:0/1740141948 shutdown_connections 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f566015b6f0 0x7f56601f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5640066690 0x7f5640068b50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f566015aa30 0x7f56601595b0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 --2- 172.21.15.66:0/1740141948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f566015a650 0x7f5660163b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 -- 172.21.15.66:0/1740141948 >> 172.21.15.66:0/1740141948 conn(0x7f5660098a20 msgr2=0x7f56600aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 -- 172.21.15.66:0/1740141948 shutdown_connections 2024-03-30T00:44:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:44:52.959+0000 7f5667000640 1 -- 172.21.15.66:0/1740141948 wait complete. 2024-03-30T00:44:52.984 INFO:tasks.cephfs.filesystem:scrub status for tag:c7f0db39-80be-4287-9367-db6af790e250 - {'path': '/subdir', 'tag': 'c7f0db39-80be-4287-9367-db6af790e250', 'options': 'recursive'} 2024-03-30T00:45:16.597 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:45:16.601 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:45:22.984 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-30T00:45:23.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc9200640 1 Processor -- start 2024-03-30T00:45:23.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc9200640 1 -- start start 2024-03-30T00:45:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc9200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415a6e0 0x7f2dc40b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc9200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc406abb0 con 0x7f2dc40cff80 2024-03-30T00:45:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc9200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc40cc110 con 0x7f2dc415e2b0 2024-03-30T00:45:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc9200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40aecc0 con 0x7f2dc415a6e0 2024-03-30T00:45:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2db9400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2dc40cff80 0x7f2dc4069d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57792/0 (socket says 172.21.15.66:57792) 2024-03-30T00:45:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2db9400640 1 -- 172.21.15.66:0/3157737283 learned_addr learned my addr 172.21.15.66:0/3157737283 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:45:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805270854 0 0) 0x7f2dc406abb0 con 0x7f2dc40cff80 2024-03-30T00:45:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2da8003530 con 0x7f2dc40cff80 2024-03-30T00:45:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415a6e0 0x7f2dc40b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 755000150 0 0) 0x7f2da8003530 con 0x7f2dc40cff80 2024-03-30T00:45:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415a6e0 msgr2=0x7f2dc40b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc1400640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415a6e0 0x7f2dc40b03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.131+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 >> v1:172.21.15.156:6789/0 conn(0x7f2dc415e2b0 legacy=0x7f2dc406a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc406ce40 con 0x7f2dc40cff80 2024-03-30T00:45:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f2dac001680 con 0x7f2dc40cff80 2024-03-30T00:45:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 >> v1:172.21.15.66:6789/0 conn(0x7f2dc40cff80 legacy=0x7f2dc4069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc1400640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2da80039b0 0x7f2da8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc1400640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415a6e0 0x7f2da80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc1400640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 0x7f2da8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40aecc0 con 0x7f2da8004d10 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da800baa0 con 0x7f2da80039b0 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da800bc60 con 0x7f2dc415a6e0 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2da80039b0 0x7f2da8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc2800640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 0x7f2da8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2db9400640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415a6e0 0x7f2da80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc2800640 1 -- 172.21.15.66:0/3157737283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415a6e0 msgr2=0x7f2da80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc2800640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415a6e0 0x7f2da80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc2800640 1 -- 172.21.15.66:0/3157737283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2da80039b0 msgr2=0x7f2da8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc2800640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2da80039b0 0x7f2da8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc2800640 1 -- 172.21.15.66:0/3157737283 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da8003530 con 0x7f2da8004d10 2024-03-30T00:45:23.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.135+0000 7f2dc2800640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 0x7f2da8005190 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc007fd0 tx=0x7f2dbc01c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bfa044646adb67 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0a5730 con 0x7f2da8004d10 2024-03-30T00:45:23.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc0a5890 con 0x7f2da8004d10 2024-03-30T00:45:23.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc1400640 1 -- 172.21.15.66:0/3157737283 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0a5c40 con 0x7f2da8004d10 2024-03-30T00:45:23.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 -- 172.21.15.66:0/3157737283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 msgr2=0x7f2da8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 --2- 172.21.15.66:0/3157737283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 0x7f2da8005190 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc007fd0 tx=0x7f2dbc01c840 comp rx=0 tx=0).stop 2024-03-30T00:45:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc2800640 1 -- 172.21.15.66:0/3157737283 reap_dead start 2024-03-30T00:45:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 -- 172.21.15.66:0/3157737283 shutdown_connections 2024-03-30T00:45:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 -- 172.21.15.66:0/3157737283 >> 172.21.15.66:0/3157737283 conn(0x7f2dc4098a20 msgr2=0x7f2dc40cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 -- 172.21.15.66:0/3157737283 shutdown_connections 2024-03-30T00:45:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 -- 172.21.15.66:0/3157737283 wait complete. 2024-03-30T00:45:23.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 Processor -- start 2024-03-30T00:45:23.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 -- start start 2024-03-30T00:45:23.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 0x7f2dc40c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415daf0 0x7f2dc40ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dc40b29c0 0x7f2dc40b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40b31d0 con 0x7f2da8004d10 2024-03-30T00:45:23.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc41574c0 con 0x7f2dc40b29c0 2024-03-30T00:45:23.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc9200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40b2390 con 0x7f2dc415daf0 2024-03-30T00:45:23.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 0x7f2dc40c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 0x7f2dc40c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57542/0 (socket says 172.21.15.66:57542) 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc1e00640 1 -- 172.21.15.66:0/1802268285 learned_addr learned my addr 172.21.15.66:0/1802268285 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc1e00640 1 -- 172.21.15.66:0/1802268285 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415daf0 msgr2=0x7f2dc40ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415daf0 0x7f2dc40ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc1e00640 1 -- 172.21.15.66:0/1802268285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dc40b29c0 msgr2=0x7f2dc40b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.139+0000 7f2dc2800640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dc40b29c0 0x7f2dc40b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.143+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dc40b29c0 0x7f2dc40b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.143+0000 7f2dc1e00640 1 -- 172.21.15.66:0/1802268285 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc40b2670 con 0x7f2da8004d10 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.143+0000 7f2dc2800640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dc40b29c0 0x7f2dc40b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.143+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 0x7f2dc40c5270 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2db400b7f0 tx=0x7f2db400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.143+0000 7f2dbaa00640 1 -- 172.21.15.66:0/1802268285 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4004170 con 0x7f2da8004d10 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.143+0000 7f2dbaa00640 1 -- 172.21.15.66:0/1802268285 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4010830 con 0x7f2da8004d10 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.143+0000 7f2dbaa00640 1 -- 172.21.15.66:0/1802268285 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4010be0 con 0x7f2da8004d10 2024-03-30T00:45:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.143+0000 7f2dc9200640 1 -- 172.21.15.66:0/1802268285 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc40cdab0 con 0x7f2da8004d10 2024-03-30T00:45:23.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.143+0000 7f2dc9200640 1 -- 172.21.15.66:0/1802268285 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc40ce070 con 0x7f2da8004d10 2024-03-30T00:45:23.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.143+0000 7f2dbaa00640 1 -- 172.21.15.66:0/1802268285 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2db4010d40 con 0x7f2da8004d10 2024-03-30T00:45:23.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dbaa00640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2da00666c0 0x7f2da0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dbaa00640 1 -- 172.21.15.66:0/1802268285 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f2db4084950 con 0x7f2da8004d10 2024-03-30T00:45:23.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2db9400640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2da00666c0 0x7f2da0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 Processor -- start 2024-03-30T00:45:23.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 -- start start 2024-03-30T00:45:23.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2db9400640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2da00666c0 0x7f2da0068b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0011f0 tx=0x7f2dac001160 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x560860801e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860802390 0x5608607b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608607b5a00 0x5608607b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560860680630 con 0x560860801a70 2024-03-30T00:45:23.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086060dbd0 con 0x5608607b5a00 2024-03-30T00:45:23.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x56086060b1a0 con 0x560860802390 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dc1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x560860801e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dc1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x560860801e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57546/0 (socket says 172.21.15.66:57546) 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dc1e00640 1 -- 172.21.15.66:0/3767752680 learned_addr learned my addr 172.21.15.66:0/3767752680 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2db9400640 1 --2- 172.21.15.66:0/3767752680 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860802390 0x5608607b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dc2800640 1 --2- 172.21.15.66:0/3767752680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608607b5a00 0x5608607b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dc1e00640 1 -- 172.21.15.66:0/3767752680 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860802390 msgr2=0x5608607b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/3767752680 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860802390 0x5608607b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dc1e00640 1 -- 172.21.15.66:0/3767752680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608607b5a00 msgr2=0x5608607b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/3767752680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608607b5a00 0x5608607b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dc1e00640 1 -- 172.21.15.66:0/3767752680 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560860302e20 con 0x560860801a70 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/3767752680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x560860801e50 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2db405d690 tx=0x7f2db4055260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb034c373ef73b90 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2db2a00640 1 -- 172.21.15.66:0/3767752680 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db406d1c0 con 0x560860801a70 2024-03-30T00:45:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2db2a00640 1 -- 172.21.15.66:0/3767752680 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4051ee0 con 0x560860801a70 2024-03-30T00:45:23.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2db2a00640 1 -- 172.21.15.66:0/3767752680 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40593b0 con 0x560860801a70 2024-03-30T00:45:23.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 -- 172.21.15.66:0/3767752680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 msgr2=0x560860801e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 --2- 172.21.15.66:0/3767752680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x560860801e50 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2db405d690 tx=0x7f2db4055260 comp rx=0 tx=0).stop 2024-03-30T00:45:23.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 -- 172.21.15.66:0/3767752680 shutdown_connections 2024-03-30T00:45:23.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 --2- 172.21.15.66:0/3767752680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608607b5a00 0x5608607b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 --2- 172.21.15.66:0/3767752680 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860802390 0x5608607b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 --2- 172.21.15.66:0/3767752680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x560860801e50 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 -- 172.21.15.66:0/3767752680 >> 172.21.15.66:0/3767752680 conn(0x5608605b6160 msgr2=0x5608607e4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 -- 172.21.15.66:0/3767752680 shutdown_connections 2024-03-30T00:45:23.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.147+0000 7f2dcc665000 1 -- 172.21.15.66:0/3767752680 wait complete. 2024-03-30T00:45:23.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 -- start start 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 Processor -- start 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608607b5a00 0x5608607be810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860801a70 0x5608607e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860802390 0x5608607e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560860606a90 con 0x560860802390 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608605a5d40 con 0x560860801a70 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608605c3f30 con 0x5608607b5a00 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 10 client.0 _ll_register_callbacks cb 0x560860668810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dc2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860802390 0x5608607e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dc2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860802390 0x5608607e5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57558/0 (socket says 172.21.15.66:57558) 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dc2800640 1 -- 172.21.15.66:0/2456990628 learned_addr learned my addr 172.21.15.66:0/2456990628 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608607b5a00 0x5608607be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2db9400640 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860801a70 0x5608607e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dc2800640 1 -- 172.21.15.66:0/2456990628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608607b5a00 msgr2=0x5608607be810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dc2800640 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608607b5a00 0x5608607be810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dc2800640 1 -- 172.21.15.66:0/2456990628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860801a70 msgr2=0x5608607e5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dc2800640 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860801a70 0x5608607e5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dc2800640 1 -- 172.21.15.66:0/2456990628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc40ce070 con 0x560860802390 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2db9400640 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860801a70 0x5608607e5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dc2800640 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860802390 0x5608607e5f90 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc01c670 tx=0x7f2dbc0195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59caedbb4c510d67 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2da7400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2da7400640 1 -- 172.21.15.66:0/2456990628 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc006e20 con 0x560860802390 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2da7400640 1 -- 172.21.15.66:0/2456990628 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f2dbc0a5a00 con 0x560860802390 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 -- 172.21.15.66:0/2456990628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608606053e0 con 0x560860802390 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2da7400640 1 -- 172.21.15.66:0/2456990628 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dbc0a5bc0 con 0x560860802390 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 10 client.7380 fetch_fsmap learned FSMap version 50 2024-03-30T00:45:23.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 -- 172.21.15.66:0/2456990628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x5608605cbaa0 con 0x560860802390 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2da7400640 1 -- 172.21.15.66:0/2456990628 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5608606053e0 con 0x560860802390 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2da7400640 1 -- 172.21.15.66:0/2456990628 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2dbc0a5d20 con 0x560860802390 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 10 client.7380 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 10 client.7380 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 20 client.7380 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x5608608092d0 0x5608608096b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 4 client.7380 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dcc665000 1 -- 172.21.15.66:0/2456990628 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608605d6d00 con 0x5608608092d0 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.151+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x5608608092d0 0x5608608096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2da7400640 10 client.7380 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x5608608092d0 0x5608608096b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dd5f5f19cb65b66 server_cookie=904bf03391e028f1 in_seq=0 out_seq=0 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2da7400640 1 -- 172.21.15.66:0/2456990628 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5608605d6d00 con 0x5608608092d0 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2da7400640 10 client.7380 handle_command_reply: tid=0 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 1 client.7380 shutdown 2024-03-30T00:45:23.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 1 -- 172.21.15.66:0/2456990628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860802390 msgr2=0x5608607e5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860802390 0x5608607e5f90 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc01c670 tx=0x7f2dbc0195e0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 1 -- 172.21.15.66:0/2456990628 shutdown_connections 2024-03-30T00:45:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x5608608092d0 0x5608608096b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860802390 0x5608607e5f90 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860801a70 0x5608607e5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 1 --2- 172.21.15.66:0/2456990628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5608607b5a00 0x5608607be810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2da7400640 0 client.7380 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 1 -- 172.21.15.66:0/2456990628 >> 172.21.15.66:0/2456990628 conn(0x5608605b6160 msgr2=0x5608607b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 1 -- 172.21.15.66:0/2456990628 shutdown_connections 2024-03-30T00:45:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 1 -- 172.21.15.66:0/2456990628 wait complete. 2024-03-30T00:45:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.155+0000 7f2dcc665000 20 client.7380 trim_cache size 0 max 16384 2024-03-30T00:45:23.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.167+0000 7f2dcc665000 1 Processor -- start 2024-03-30T00:45:23.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.167+0000 7f2dcc665000 1 -- start start 2024-03-30T00:45:23.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608607b5a00 0x5608607b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x5608607b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860802390 0x5608607b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608602c3aa0 con 0x560860801a70 2024-03-30T00:45:23.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608605d60b0 con 0x5608607b5a00 2024-03-30T00:45:23.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608606042a0 con 0x560860802390 2024-03-30T00:45:23.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x5608607b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x5608607b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57574/0 (socket says 172.21.15.66:57574) 2024-03-30T00:45:23.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db9400640 1 -- 172.21.15.66:0/2218035283 learned_addr learned my addr 172.21.15.66:0/2218035283 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dc2800640 1 --2- 172.21.15.66:0/2218035283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860802390 0x5608607b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/2218035283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608607b5a00 0x5608607b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db9400640 1 -- 172.21.15.66:0/2218035283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860802390 msgr2=0x5608607b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db9400640 1 --2- 172.21.15.66:0/2218035283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860802390 0x5608607b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db9400640 1 -- 172.21.15.66:0/2218035283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608607b5a00 msgr2=0x5608607b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db9400640 1 --2- 172.21.15.66:0/2218035283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608607b5a00 0x5608607b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db9400640 1 -- 172.21.15.66:0/2218035283 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608605b1df0 con 0x560860801a70 2024-03-30T00:45:23.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dc2800640 1 --2- 172.21.15.66:0/2218035283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860802390 0x5608607b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:23.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db9400640 1 --2- 172.21.15.66:0/2218035283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x5608607b5170 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40d03c0 tx=0x7f2dac015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee91c2090c707873 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db3400640 1 -- 172.21.15.66:0/2218035283 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac016780 con 0x560860801a70 2024-03-30T00:45:23.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db3400640 1 -- 172.21.15.66:0/2218035283 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac0168e0 con 0x560860801a70 2024-03-30T00:45:23.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2db3400640 1 -- 172.21.15.66:0/2218035283 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac016c50 con 0x560860801a70 2024-03-30T00:45:23.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 -- 172.21.15.66:0/2218035283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 msgr2=0x5608607b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 --2- 172.21.15.66:0/2218035283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x5608607b5170 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40d03c0 tx=0x7f2dac015000 comp rx=0 tx=0).stop 2024-03-30T00:45:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 -- 172.21.15.66:0/2218035283 shutdown_connections 2024-03-30T00:45:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 --2- 172.21.15.66:0/2218035283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860802390 0x5608607b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 --2- 172.21.15.66:0/2218035283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560860801a70 0x5608607b5170 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 --2- 172.21.15.66:0/2218035283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608607b5a00 0x5608607b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 -- 172.21.15.66:0/2218035283 >> 172.21.15.66:0/2218035283 conn(0x5608605b6160 msgr2=0x5608607bd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 -- 172.21.15.66:0/2218035283 shutdown_connections 2024-03-30T00:45:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 -- 172.21.15.66:0/2218035283 wait complete. 2024-03-30T00:45:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 -- start start 2024-03-30T00:45:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.171+0000 7f2dcc665000 1 Processor -- start 2024-03-30T00:45:23.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608607b5a00 0x5608607b9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860801a70 0x56086081d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860802390 0x56086081d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608605d0ee0 con 0x5608607b5a00 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608607becc0 con 0x560860802390 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608606095f0 con 0x560860801a70 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 10 client.0 _ll_register_callbacks cb 0x7f2db40555a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608607b5a00 0x5608607b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608607b5a00 0x5608607b9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57586/0 (socket says 172.21.15.66:57586) 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc1e00640 1 -- 172.21.15.66:0/204254655 learned_addr learned my addr 172.21.15.66:0/204254655 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2db9400640 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860801a70 0x56086081d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc2800640 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860802390 0x56086081d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc1e00640 1 -- 172.21.15.66:0/204254655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860801a70 msgr2=0x56086081d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860801a70 0x56086081d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc1e00640 1 -- 172.21.15.66:0/204254655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860802390 msgr2=0x56086081d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860802390 0x56086081d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc1e00640 1 -- 172.21.15.66:0/204254655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc406ce40 con 0x5608607b5a00 2024-03-30T00:45:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc2800640 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860802390 0x56086081d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc1e00640 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608607b5a00 0x5608607b9530 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2db405d690 tx=0x7f2db401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b1596cc409bd36 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2da7400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2da7400640 1 -- 172.21.15.66:0/204254655 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4059170 con 0x5608607b5a00 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2da7400640 1 -- 172.21.15.66:0/204254655 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f2db40592d0 con 0x5608607b5a00 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 -- 172.21.15.66:0/204254655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608605c3970 con 0x5608607b5a00 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2da7400640 1 -- 172.21.15.66:0/204254655 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2db407cbd0 con 0x5608607b5a00 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 10 client.7386 fetch_fsmap learned FSMap version 50 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 -- 172.21.15.66:0/204254655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x5608605a5960 con 0x5608607b5a00 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2da7400640 1 -- 172.21.15.66:0/204254655 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5608605c3970 con 0x5608607b5a00 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2da7400640 1 -- 172.21.15.66:0/204254655 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2db405d730 con 0x5608607b5a00 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 10 client.7386 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 10 client.7386 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 20 client.7386 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x5608607b9a70 0x56086081dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 4 client.7386 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 -- 172.21.15.66:0/204254655 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560860635a20 con 0x5608607b9a70 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2db9400640 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x5608607b9a70 0x56086081dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2db9400640 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x5608607b9a70 0x56086081dc60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8490645e351b70b1 server_cookie=9a1c96302d20a525 in_seq=0 out_seq=0 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2da7400640 10 client.7386 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2da7400640 1 -- 172.21.15.66:0/204254655 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560860635a20 con 0x5608607b9a70 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2da7400640 10 client.7386 handle_command_reply: tid=0 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 client.7386 shutdown 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 -- 172.21.15.66:0/204254655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608607b5a00 msgr2=0x5608607b9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608607b5a00 0x5608607b9530 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2db405d690 tx=0x7f2db401e000 comp rx=0 tx=0).stop 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 -- 172.21.15.66:0/204254655 shutdown_connections 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x5608607b9a70 0x56086081dc60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560860802390 0x56086081d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x560860801a70 0x56086081d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2da7400640 0 client.7386 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:23.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 --2- 172.21.15.66:0/204254655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5608607b5a00 0x5608607b9530 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 -- 172.21.15.66:0/204254655 >> 172.21.15.66:0/204254655 conn(0x5608605b6160 msgr2=0x5608607b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 -- 172.21.15.66:0/204254655 shutdown_connections 2024-03-30T00:45:23.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 1 -- 172.21.15.66:0/204254655 wait complete. 2024-03-30T00:45:23.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dcc665000 20 client.7386 trim_cache size 0 max 16384 2024-03-30T00:45:23.189 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:45:23.189 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T00:45:23.189 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-03-30T00:45:23.189 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:45:23.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc9200640 1 -- 172.21.15.66:0/1802268285 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2da00666c0 msgr2=0x7f2da0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc9200640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2da00666c0 0x7f2da0068b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0011f0 tx=0x7f2dac001160 comp rx=0 tx=0).stop 2024-03-30T00:45:23.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc9200640 1 -- 172.21.15.66:0/1802268285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 msgr2=0x7f2dc40c5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.175+0000 7f2dc9200640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 0x7f2dc40c5270 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2db400b7f0 tx=0x7f2db400bcb0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.179+0000 7f2dc9200640 1 -- 172.21.15.66:0/1802268285 shutdown_connections 2024-03-30T00:45:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.179+0000 7f2dc9200640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dc40b29c0 0x7f2dc40b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.179+0000 7f2dc9200640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2dc415daf0 0x7f2dc40ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.179+0000 7f2dc9200640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2da00666c0 0x7f2da0068b80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.179+0000 7f2dc9200640 1 --2- 172.21.15.66:0/1802268285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2da8004d10 0x7f2dc40c5270 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.179+0000 7f2dc9200640 1 -- 172.21.15.66:0/1802268285 >> 172.21.15.66:0/1802268285 conn(0x7f2dc4098a20 msgr2=0x7f2dc4157b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.179+0000 7f2dc9200640 1 -- 172.21.15.66:0/1802268285 shutdown_connections 2024-03-30T00:45:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.179+0000 7f2dc9200640 1 -- 172.21.15.66:0/1802268285 wait complete. 2024-03-30T00:45:23.203 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:45:23.204 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:45:23.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.335+0000 7f481da00640 1 Processor -- start 2024-03-30T00:45:23.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.335+0000 7f481da00640 1 -- start start 2024-03-30T00:45:23.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.335+0000 7f481da00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f48180ae4e0 0x7f48180cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.335+0000 7f481da00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 0x7f48180cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.339+0000 7f481da00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481806e7d0 con 0x7f481815ca10 2024-03-30T00:45:23.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.339+0000 7f481da00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48180b1f70 con 0x7f48180ae4e0 2024-03-30T00:45:23.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.339+0000 7f481da00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48180cf1f0 con 0x7f48180ce820 2024-03-30T00:45:23.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.339+0000 7f4817400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 0x7f48180cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.339+0000 7f480e000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f48180ae4e0 0x7f48180cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.339+0000 7f4817400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 0x7f48180cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:51972/0 (socket says 172.21.15.66:51972) 2024-03-30T00:45:23.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.339+0000 7f4817400640 1 -- 172.21.15.66:0/3555569762 learned_addr learned my addr 172.21.15.66:0/3555569762 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.339+0000 7f4817400640 1 -- 172.21.15.66:0/3555569762 >> v2:172.21.15.156:3300/0 conn(0x7f48180ae4e0 msgr2=0x7f48180cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.339+0000 7f4817400640 1 --2- 172.21.15.66:0/3555569762 >> v2:172.21.15.156:3300/0 conn(0x7f48180ae4e0 0x7f48180cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.339+0000 7f4817400640 1 -- 172.21.15.66:0/3555569762 >> v1:172.21.15.156:6789/0 conn(0x7f481815ca10 legacy=0x7f481806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.339+0000 7f4817400640 1 -- 172.21.15.66:0/3555569762 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481815cdf0 con 0x7f48180ce820 2024-03-30T00:45:23.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f4817400640 1 --2- 172.21.15.66:0/3555569762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 0x7f48180cecb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4810009870 tx=0x7f481001c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe753f9e8d76981c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f4816000640 1 -- 172.21.15.66:0/3555569762 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48100aa430 con 0x7f48180ce820 2024-03-30T00:45:23.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f4816000640 1 -- 172.21.15.66:0/3555569762 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48100aa590 con 0x7f48180ce820 2024-03-30T00:45:23.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f4816000640 1 -- 172.21.15.66:0/3555569762 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48100aa920 con 0x7f48180ce820 2024-03-30T00:45:23.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 -- 172.21.15.66:0/3555569762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 msgr2=0x7f48180cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 --2- 172.21.15.66:0/3555569762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 0x7f48180cecb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4810009870 tx=0x7f481001c4f0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 -- 172.21.15.66:0/3555569762 shutdown_connections 2024-03-30T00:45:23.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 --2- 172.21.15.66:0/3555569762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 0x7f48180cecb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 --2- 172.21.15.66:0/3555569762 >> v2:172.21.15.156:3300/0 conn(0x7f48180ae4e0 0x7f48180cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 -- 172.21.15.66:0/3555569762 >> 172.21.15.66:0/3555569762 conn(0x7f4818098a20 msgr2=0x7f48180b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 -- 172.21.15.66:0/3555569762 shutdown_connections 2024-03-30T00:45:23.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 -- 172.21.15.66:0/3555569762 wait complete. 2024-03-30T00:45:23.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 Processor -- start 2024-03-30T00:45:23.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 -- start start 2024-03-30T00:45:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48180ae4e0 0x7f48181529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 0x7f48180c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f481815ca10 0x7f48180b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4818153730 con 0x7f48180ae4e0 2024-03-30T00:45:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4818153a10 con 0x7f481815ca10 2024-03-30T00:45:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.343+0000 7f481da00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4818153cf0 con 0x7f48180ce820 2024-03-30T00:45:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f4816a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48180ae4e0 0x7f48181529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f4816a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48180ae4e0 0x7f48181529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57598/0 (socket says 172.21.15.66:57598) 2024-03-30T00:45:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f4816a00640 1 -- 172.21.15.66:0/625495567 learned_addr learned my addr 172.21.15.66:0/625495567 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f480e000640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 0x7f48180c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f4817400640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f481815ca10 0x7f48180b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f4816a00640 1 -- 172.21.15.66:0/625495567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 msgr2=0x7f48180c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f4816a00640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 0x7f48180c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f4816a00640 1 -- 172.21.15.66:0/625495567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f481815ca10 msgr2=0x7f48180b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f4816a00640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f481815ca10 0x7f48180b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f4816a00640 1 -- 172.21.15.66:0/625495567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100094d0 con 0x7f48180ae4e0 2024-03-30T00:45:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f4816a00640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48180ae4e0 0x7f48181529e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f480800ba50 tx=0x7f480800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f480f400640 1 -- 172.21.15.66:0/625495567 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480800cbe0 con 0x7f48180ae4e0 2024-03-30T00:45:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f480f400640 1 -- 172.21.15.66:0/625495567 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480800cd80 con 0x7f48180ae4e0 2024-03-30T00:45:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f481da00640 1 -- 172.21.15.66:0/625495567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4818153fd0 con 0x7f48180ae4e0 2024-03-30T00:45:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f480f400640 1 -- 172.21.15.66:0/625495567 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808014680 con 0x7f48180ae4e0 2024-03-30T00:45:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f481da00640 1 -- 172.21.15.66:0/625495567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4818152f20 con 0x7f48180ae4e0 2024-03-30T00:45:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f481da00640 1 -- 172.21.15.66:0/625495567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4818066c30 con 0x7f48180ae4e0 2024-03-30T00:45:23.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f480f400640 1 -- 172.21.15.66:0/625495567 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f48080147e0 con 0x7f48180ae4e0 2024-03-30T00:45:23.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.347+0000 7f480f400640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47f40666c0 0x7f47f4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.351+0000 7f480f400640 1 -- 172.21.15.66:0/625495567 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f4808084150 con 0x7f48180ae4e0 2024-03-30T00:45:23.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.351+0000 7f480e000640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47f40666c0 0x7f47f4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.351+0000 7f480e000640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47f40666c0 0x7f47f4068b80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f48180c9f70 tx=0x7f4800009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.355+0000 7f480f400640 1 -- 172.21.15.66:0/625495567 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f48080513c0 con 0x7f48180ae4e0 2024-03-30T00:45:23.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 -- 172.21.15.66:0/625495567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48180b5430 con 0x7f48180ae4e0 2024-03-30T00:45:23.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f480f400640 1 -- 172.21.15.66:0/625495567 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f48080549e0 con 0x7f48180ae4e0 2024-03-30T00:45:23.555 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:45:23.555 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/254323195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":254323195},{"type":"v1","addr":"172.21.15.156:6835","nonce":254323195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:44:50.553952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":13,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:45:23.555 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 50 2024-03-30T00:45:23.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 -- 172.21.15.66:0/625495567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47f40666c0 msgr2=0x7f47f4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47f40666c0 0x7f47f4068b80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f48180c9f70 tx=0x7f4800009190 comp rx=0 tx=0).stop 2024-03-30T00:45:23.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 -- 172.21.15.66:0/625495567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48180ae4e0 msgr2=0x7f48181529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48180ae4e0 0x7f48181529e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f480800ba50 tx=0x7f480800bf10 comp rx=0 tx=0).stop 2024-03-30T00:45:23.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 -- 172.21.15.66:0/625495567 shutdown_connections 2024-03-30T00:45:23.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f481815ca10 0x7f48180b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48180ce820 0x7f48180c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47f40666c0 0x7f47f4068b80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 --2- 172.21.15.66:0/625495567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48180ae4e0 0x7f48181529e0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 -- 172.21.15.66:0/625495567 >> 172.21.15.66:0/625495567 conn(0x7f4818098a20 msgr2=0x7f48180ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 -- 172.21.15.66:0/625495567 shutdown_connections 2024-03-30T00:45:23.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.547+0000 7f481da00640 1 -- 172.21.15.66:0/625495567 wait complete. 2024-03-30T00:45:23.577 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-30T00:45:23.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.711+0000 7fd9c8000640 1 Processor -- start 2024-03-30T00:45:23.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c8000640 1 -- start start 2024-03-30T00:45:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c8000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9c015a640 0x7fd9c006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c8000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd9c00ce520 0x7fd9c00ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c8000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd9c00cabb0 0x7fd9c00cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c8000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c006d070 con 0x7fd9c00ce520 2024-03-30T00:45:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c8000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c00cbba0 con 0x7fd9c00cabb0 2024-03-30T00:45:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c8000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c006e2c0 con 0x7fd9c015a640 2024-03-30T00:45:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c4c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd9c00ce520 0x7fd9c00ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c4c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd9c00ce520 0x7fd9c00ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57604/0 (socket says 172.21.15.66:57604) 2024-03-30T00:45:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c4c00640 1 -- 172.21.15.66:0/655231285 learned_addr learned my addr 172.21.15.66:0/655231285 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c5600640 1 --2- 172.21.15.66:0/655231285 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9c015a640 0x7fd9c006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c6000640 1 --2- 172.21.15.66:0/655231285 >> v2:172.21.15.156:3300/0 conn(0x7fd9c00cabb0 0x7fd9c00cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c4c00640 1 -- 172.21.15.66:0/655231285 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9c015a640 msgr2=0x7fd9c006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/655231285 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9c015a640 0x7fd9c006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c4c00640 1 -- 172.21.15.66:0/655231285 >> v2:172.21.15.156:3300/0 conn(0x7fd9c00cabb0 msgr2=0x7fd9c00cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/655231285 >> v2:172.21.15.156:3300/0 conn(0x7fd9c00cabb0 0x7fd9c00cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.715+0000 7fd9c4c00640 1 -- 172.21.15.66:0/655231285 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00cee60 con 0x7fd9c00ce520 2024-03-30T00:45:23.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/655231285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9c00ce520 0x7fd9c00ce920 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00acfe0 tx=0x7fd9ac01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42d4f9bc4d9a3b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9bfe00640 1 -- 172.21.15.66:0/655231285 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0aa610 con 0x7fd9c00ce520 2024-03-30T00:45:23.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9bfe00640 1 -- 172.21.15.66:0/655231285 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac0a6040 con 0x7fd9c00ce520 2024-03-30T00:45:23.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9bfe00640 1 -- 172.21.15.66:0/655231285 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0aa990 con 0x7fd9c00ce520 2024-03-30T00:45:23.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c8000640 1 -- 172.21.15.66:0/655231285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9c00ce520 msgr2=0x7fd9c00ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c8000640 1 --2- 172.21.15.66:0/655231285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9c00ce520 0x7fd9c00ce920 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00acfe0 tx=0x7fd9ac01c4f0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c8000640 1 -- 172.21.15.66:0/655231285 shutdown_connections 2024-03-30T00:45:23.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c8000640 1 --2- 172.21.15.66:0/655231285 >> v2:172.21.15.156:3300/0 conn(0x7fd9c00cabb0 0x7fd9c00cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c8000640 1 --2- 172.21.15.66:0/655231285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9c00ce520 0x7fd9c00ce920 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c8000640 1 --2- 172.21.15.66:0/655231285 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9c015a640 0x7fd9c006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c8000640 1 -- 172.21.15.66:0/655231285 >> 172.21.15.66:0/655231285 conn(0x7fd9c0069f60 msgr2=0x7fd9c006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c8000640 1 -- 172.21.15.66:0/655231285 shutdown_connections 2024-03-30T00:45:23.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c8000640 1 -- 172.21.15.66:0/655231285 wait complete. 2024-03-30T00:45:23.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c8000640 1 Processor -- start 2024-03-30T00:45:23.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.719+0000 7fd9c8000640 1 -- start start 2024-03-30T00:45:23.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c8000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9c00cabb0 0x7fd9c0159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c8000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9c00ce520 0x7fd9c00ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c8000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9c015a640 0x7fd9c01530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c8000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0070f20 con 0x7fd9c00cabb0 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c8000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c006ddd0 con 0x7fd9c015a640 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c8000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0071170 con 0x7fd9c00ce520 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9c00cabb0 0x7fd9c0159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9c00cabb0 0x7fd9c0159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57610/0 (socket says 172.21.15.66:57610) 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c5600640 1 -- 172.21.15.66:0/4134879664 learned_addr learned my addr 172.21.15.66:0/4134879664 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c5600640 1 -- 172.21.15.66:0/4134879664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9c00ce520 msgr2=0x7fd9c00ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9c00ce520 0x7fd9c00ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c6000640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9c015a640 0x7fd9c01530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c5600640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9c00ce520 0x7fd9c00ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c5600640 1 -- 172.21.15.66:0/4134879664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9c015a640 msgr2=0x7fd9c01530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c5600640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9c015a640 0x7fd9c01530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c5600640 1 -- 172.21.15.66:0/4134879664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac0094d0 con 0x7fd9c00cabb0 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9c00ce520 0x7fd9c00ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c6000640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9c015a640 0x7fd9c01530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:45:23.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c5600640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9c00cabb0 0x7fd9c0159b30 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400d620 tx=0x7fd9b400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9bd600640 1 -- 172.21.15.66:0/4134879664 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400b620 con 0x7fd9c00cabb0 2024-03-30T00:45:23.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9bd600640 1 -- 172.21.15.66:0/4134879664 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b400b780 con 0x7fd9c00cabb0 2024-03-30T00:45:23.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9bd600640 1 -- 172.21.15.66:0/4134879664 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400bb10 con 0x7fd9c00cabb0 2024-03-30T00:45:23.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c8000640 1 -- 172.21.15.66:0/4134879664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c0071450 con 0x7fd9c00cabb0 2024-03-30T00:45:23.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9c8000640 1 -- 172.21.15.66:0/4134879664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c0071a10 con 0x7fd9c00cabb0 2024-03-30T00:45:23.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.723+0000 7fd9bd600640 1 -- 172.21.15.66:0/4134879664 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd9b400bc70 con 0x7fd9c00cabb0 2024-03-30T00:45:23.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9bd600640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9a00666c0 0x7fd9a0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9bd600640 1 -- 172.21.15.66:0/4134879664 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd9b4084880 con 0x7fd9c00cabb0 2024-03-30T00:45:23.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9a00666c0 0x7fd9a0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9cb3a7000 1 Processor -- start 2024-03-30T00:45:23.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9cb3a7000 1 -- start start 2024-03-30T00:45:23.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9a00666c0 0x7fd9a0068b80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0ae000 tx=0x7fd9ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2dc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2dcbb0 0x55fc7e292c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2931b0 0x55fc7e293590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7e15b340 con 0x55fc7e2dc290 2024-03-30T00:45:23.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7e0e7cd0 con 0x55fc7e2931b0 2024-03-30T00:45:23.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc7e0e52a0 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2dc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2dc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57620/0 (socket says 172.21.15.66:57620) 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c5600640 1 -- 172.21.15.66:0/1522686511 learned_addr learned my addr 172.21.15.66:0/1522686511 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/1522686511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2dcbb0 0x55fc7e292c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c5600640 1 -- 172.21.15.66:0/1522686511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2dcbb0 msgr2=0x55fc7e292c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c6000640 1 --2- 172.21.15.66:0/1522686511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2931b0 0x55fc7e293590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c5600640 1 --2- 172.21.15.66:0/1522686511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2dcbb0 0x55fc7e292c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c5600640 1 -- 172.21.15.66:0/1522686511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2931b0 msgr2=0x55fc7e293590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c5600640 1 --2- 172.21.15.66:0/1522686511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2931b0 0x55fc7e293590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c5600640 1 -- 172.21.15.66:0/1522686511 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc7ddd34b0 con 0x55fc7e2dc290 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/1522686511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2dcbb0 0x55fc7e292c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c6000640 1 --2- 172.21.15.66:0/1522686511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2931b0 0x55fc7e293590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9c5600640 1 --2- 172.21.15.66:0/1522686511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2dc670 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd9b405d530 tx=0x7fd9b4055d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345b5beae2728bef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9b2000640 1 -- 172.21.15.66:0/1522686511 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b405da60 con 0x55fc7e2dc290 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9b2000640 1 -- 172.21.15.66:0/1522686511 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b4053a10 con 0x55fc7e2dc290 2024-03-30T00:45:23.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.727+0000 7fd9b2000640 1 -- 172.21.15.66:0/1522686511 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4051d80 con 0x55fc7e2dc290 2024-03-30T00:45:23.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/1522686511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 msgr2=0x55fc7e2dc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/1522686511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2dc670 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd9b405d530 tx=0x7fd9b4055d70 comp rx=0 tx=0).stop 2024-03-30T00:45:23.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/1522686511 shutdown_connections 2024-03-30T00:45:23.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/1522686511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2931b0 0x55fc7e293590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/1522686511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2dcbb0 0x55fc7e292c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/1522686511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2dc670 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/1522686511 >> 172.21.15.66:0/1522686511 conn(0x55fc7e28f160 msgr2=0x55fc7e291550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/1522686511 shutdown_connections 2024-03-30T00:45:23.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/1522686511 wait complete. 2024-03-30T00:45:23.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 -- start start 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 Processor -- start 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2931b0 0x55fc7e2986b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2dc290 0x55fc7e298bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dcbb0 0x55fc7e29de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7e0e0b90 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7e09e0a0 con 0x55fc7e2dc290 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc7e09dd40 con 0x55fc7e2931b0 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 10 client.0 _ll_register_callbacks cb 0x55fc7e195c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9c6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dcbb0 0x55fc7e29de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9c6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dcbb0 0x55fc7e29de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57626/0 (socket says 172.21.15.66:57626) 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9c6000640 1 -- 172.21.15.66:0/2395173310 learned_addr learned my addr 172.21.15.66:0/2395173310 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9c5600640 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2931b0 0x55fc7e2986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2dc290 0x55fc7e298bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9c6000640 1 -- 172.21.15.66:0/2395173310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2931b0 msgr2=0x55fc7e2986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9c6000640 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2931b0 0x55fc7e2986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9c6000640 1 -- 172.21.15.66:0/2395173310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2dc290 msgr2=0x55fc7e298bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9c6000640 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2dc290 0x55fc7e298bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9c6000640 1 -- 172.21.15.66:0/2395173310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c0071a10 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9c6000640 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dcbb0 0x55fc7e29de40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8007a30 tx=0x7fd9b8007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d72dcaf0a2219ea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:23.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9a6a00640 1 -- 172.21.15.66:0/2395173310 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8010a70 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9a6a00640 1 -- 172.21.15.66:0/2395173310 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd9b8010bd0 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/2395173310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc7e0b0550 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9a6a00640 1 -- 172.21.15.66:0/2395173310 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b8010d90 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 10 client.7401 fetch_fsmap learned FSMap version 50 2024-03-30T00:45:23.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/2395173310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55fc7e0a5ba0 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.731+0000 7fd9a6a00640 1 -- 172.21.15.66:0/2395173310 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fc7e0b0550 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9a6a00640 1 -- 172.21.15.66:0/2395173310 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd9b80039a0 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 10 client.7401 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:45:23.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 10 client.7401 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:45:23.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 20 client.7401 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:23.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fc7e2eb320 0x55fc7e2ed710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 4 client.7401 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:45:23.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/2395173310 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc7e0dd970 con 0x55fc7e2eb320 2024-03-30T00:45:23.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9c5600640 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fc7e2eb320 0x55fc7e2ed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9a6a00640 10 client.7401 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:23.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9c5600640 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fc7e2eb320 0x55fc7e2ed710 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a5924005b3c1da server_cookie=eac4d7af1748a889 in_seq=0 out_seq=0 2024-03-30T00:45:23.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9a6a00640 1 -- 172.21.15.66:0/2395173310 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fc7e0dd970 con 0x55fc7e2eb320 2024-03-30T00:45:23.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9a6a00640 10 client.7401 handle_command_reply: tid=0 2024-03-30T00:45:23.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 client.7401 shutdown 2024-03-30T00:45:23.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/2395173310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dcbb0 msgr2=0x55fc7e29de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dcbb0 0x55fc7e29de40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8007a30 tx=0x7fd9b8007e30 comp rx=0 tx=0).stop 2024-03-30T00:45:23.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/2395173310 shutdown_connections 2024-03-30T00:45:23.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fc7e2eb320 0x55fc7e2ed710 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dcbb0 0x55fc7e29de40 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2dc290 0x55fc7e298bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/2395173310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2931b0 0x55fc7e2986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9a6a00640 0 client.7401 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/2395173310 >> 172.21.15.66:0/2395173310 conn(0x55fc7e28f160 msgr2=0x55fc7e291de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/2395173310 shutdown_connections 2024-03-30T00:45:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/2395173310 wait complete. 2024-03-30T00:45:23.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.735+0000 7fd9cb3a7000 20 client.7401 trim_cache size 0 max 16384 2024-03-30T00:45:23.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9cb3a7000 1 Processor -- start 2024-03-30T00:45:23.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9cb3a7000 1 -- start start 2024-03-30T00:45:23.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e28f160 0x55fc7e2ef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2931b0 0x55fc7e2ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7dd9dc40 con 0x55fc7e2dc290 2024-03-30T00:45:23.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7e09a300 con 0x55fc7e28f160 2024-03-30T00:45:23.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc7e0ddf90 con 0x55fc7e2931b0 2024-03-30T00:45:23.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2eb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57642/0 (socket says 172.21.15.66:57642) 2024-03-30T00:45:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c6000640 1 -- 172.21.15.66:0/127648538 learned_addr learned my addr 172.21.15.66:0/127648538 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c5600640 1 --2- 172.21.15.66:0/127648538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e28f160 0x55fc7e2ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/127648538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2931b0 0x55fc7e2ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c6000640 1 -- 172.21.15.66:0/127648538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2931b0 msgr2=0x55fc7e2ef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c6000640 1 --2- 172.21.15.66:0/127648538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2931b0 0x55fc7e2ef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c6000640 1 -- 172.21.15.66:0/127648538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e28f160 msgr2=0x55fc7e2ef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c6000640 1 --2- 172.21.15.66:0/127648538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e28f160 0x55fc7e2ef260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c6000640 1 -- 172.21.15.66:0/127648538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc7e08bef0 con 0x55fc7e2dc290 2024-03-30T00:45:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c5600640 1 --2- 172.21.15.66:0/127648538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e28f160 0x55fc7e2ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:45:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/127648538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2931b0 0x55fc7e2ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:45:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.747+0000 7fd9c6000640 1 --2- 172.21.15.66:0/127648538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2eb700 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8000f90 tx=0x7fd9b8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3eb429d654f31c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9b2a00640 1 -- 172.21.15.66:0/127648538 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8010590 con 0x55fc7e2dc290 2024-03-30T00:45:23.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9b2a00640 1 -- 172.21.15.66:0/127648538 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b800f040 con 0x55fc7e2dc290 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9b2a00640 1 -- 172.21.15.66:0/127648538 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8010a70 con 0x55fc7e2dc290 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/127648538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 msgr2=0x55fc7e2eb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/127648538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2eb700 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8000f90 tx=0x7fd9b8015000 comp rx=0 tx=0).stop 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/127648538 shutdown_connections 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/127648538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e2dc290 0x55fc7e2eb700 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/127648538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2931b0 0x55fc7e2ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/127648538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e28f160 0x55fc7e2ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/127648538 >> 172.21.15.66:0/127648538 conn(0x55fc7e2db7b0 msgr2=0x55fc7e292460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/127648538 shutdown_connections 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/127648538 wait complete. 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 -- start start 2024-03-30T00:45:23.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 Processor -- start 2024-03-30T00:45:23.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e28f160 0x55fc7e2eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2931b0 0x55fc7e3366f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2dc290 0x55fc7e336c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7e28a5e0 con 0x55fc7e28f160 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7e089520 con 0x55fc7e2931b0 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc7e0de3a0 con 0x55fc7e2dc290 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 10 client.0 _ll_register_callbacks cb 0x55fc7e17e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9c5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e28f160 0x55fc7e2eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9c5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e28f160 0x55fc7e2eee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57652/0 (socket says 172.21.15.66:57652) 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9c5600640 1 -- 172.21.15.66:0/4043211443 learned_addr learned my addr 172.21.15.66:0/4043211443 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9c6000640 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2dc290 0x55fc7e336c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2931b0 0x55fc7e3366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9c5600640 1 -- 172.21.15.66:0/4043211443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2dc290 msgr2=0x55fc7e336c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9c5600640 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2dc290 0x55fc7e336c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9c5600640 1 -- 172.21.15.66:0/4043211443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2931b0 msgr2=0x55fc7e3366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9c5600640 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2931b0 0x55fc7e3366f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9c5600640 1 -- 172.21.15.66:0/4043211443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc7e0a5ba0 con 0x55fc7e28f160 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9c5600640 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e28f160 0x55fc7e2eee50 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd9b405d530 tx=0x7fd9b400b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a96c3a9d5cdca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9a6a00640 1 -- 172.21.15.66:0/4043211443 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4053870 con 0x55fc7e28f160 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9a6a00640 1 -- 172.21.15.66:0/4043211443 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd9b4081040 con 0x55fc7e28f160 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.751+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/4043211443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc7e090620 con 0x55fc7e28f160 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9a6a00640 1 -- 172.21.15.66:0/4043211443 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b4051d80 con 0x55fc7e28f160 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 10 client.7407 fetch_fsmap learned FSMap version 50 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/4043211443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55fc7dddcfc0 con 0x55fc7e28f160 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9a6a00640 1 -- 172.21.15.66:0/4043211443 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fc7e090620 con 0x55fc7e28f160 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9a6a00640 1 -- 172.21.15.66:0/4043211443 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd9b4054b60 con 0x55fc7e28f160 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 10 client.7407 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 10 client.7407 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:45:23.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 20 client.7407 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fc7e2dcbb0 0x55fc7e337170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 4 client.7407 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/4043211443 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55fc7e0df4e0 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fc7e2dcbb0 0x55fc7e337170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c4c00640 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fc7e2dcbb0 0x55fc7e337170 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81349f3dc6b4d05a server_cookie=553422394e96af1e in_seq=0 out_seq=0 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9a6a00640 10 client.7407 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9a6a00640 1 -- 172.21.15.66:0/4043211443 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+237 (crc 0 0 0) 0x55fc7e0df4e0 con 0x55fc7e2dcbb0 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9a6a00640 10 client.7407 handle_command_reply: tid=0 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 client.7407 shutdown 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/4043211443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e28f160 msgr2=0x55fc7e2eee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e28f160 0x55fc7e2eee50 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd9b405d530 tx=0x7fd9b400b290 comp rx=0 tx=0).stop 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/4043211443 shutdown_connections 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55fc7e2dcbb0 0x55fc7e337170 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55fc7e2dc290 0x55fc7e336c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc7e2931b0 0x55fc7e3366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 --2- 172.21.15.66:0/4043211443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc7e28f160 0x55fc7e2eee50 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9a6a00640 0 client.7407 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:23.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/4043211443 >> 172.21.15.66:0/4043211443 conn(0x55fc7e2db7b0 msgr2=0x55fc7e297660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/4043211443 shutdown_connections 2024-03-30T00:45:23.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 1 -- 172.21.15.66:0/4043211443 wait complete. 2024-03-30T00:45:23.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9cb3a7000 20 client.7407 trim_cache size 0 max 16384 2024-03-30T00:45:23.766 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:45:23.766 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:45:23.766 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:45:23.767 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "dentry", 2024-03-30T00:45:23.767 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 3582010763, 2024-03-30T00:45:23.767 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627776, 2024-03-30T00:45:23.767 INFO:teuthology.orchestra.run.smithi066.stdout: "frag": "*", 2024-03-30T00:45:23.767 INFO:teuthology.orchestra.run.smithi066.stdout: "dname": "file_to_be_damaged", 2024-03-30T00:45:23.767 INFO:teuthology.orchestra.run.smithi066.stdout: "snap_id": "head", 2024-03-30T00:45:23.767 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/subdir/file_to_be_damaged" 2024-03-30T00:45:23.767 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:45:23.767 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:45:23.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 -- 172.21.15.66:0/4134879664 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9a00666c0 msgr2=0x7fd9a0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9a00666c0 0x7fd9a0068b80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0ae000 tx=0x7fd9ac008040 comp rx=0 tx=0).stop 2024-03-30T00:45:23.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 -- 172.21.15.66:0/4134879664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9c00cabb0 msgr2=0x7fd9c0159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9c00cabb0 0x7fd9c0159b30 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400d620 tx=0x7fd9b400dae0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 -- 172.21.15.66:0/4134879664 shutdown_connections 2024-03-30T00:45:23.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9c015a640 0x7fd9c01530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9c00ce520 0x7fd9c00ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9a00666c0 0x7fd9a0068b80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 --2- 172.21.15.66:0/4134879664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9c00cabb0 0x7fd9c0159b30 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 -- 172.21.15.66:0/4134879664 >> 172.21.15.66:0/4134879664 conn(0x7fd9c0069f60 msgr2=0x7fd9c00cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 -- 172.21.15.66:0/4134879664 shutdown_connections 2024-03-30T00:45:23.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.755+0000 7fd9c8000640 1 -- 172.21.15.66:0/4134879664 wait complete. 2024-03-30T00:45:23.782 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-30T00:45:23.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9eedc00640 1 Processor -- start 2024-03-30T00:45:23.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9eedc00640 1 -- start start 2024-03-30T00:45:23.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9eedc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ee815a640 0x7f9ee806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9eedc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ee80ce520 0x7f9ee80ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9eedc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee806d070 con 0x7f9ee80cabb0 2024-03-30T00:45:23.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9eedc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee80cbba0 con 0x7f9ee80ce520 2024-03-30T00:45:23.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9eedc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee806e2c0 con 0x7f9ee815a640 2024-03-30T00:45:23.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9ede000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ee80ce520 0x7f9ee80ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9ede000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ee80ce520 0x7f9ee80ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57662/0 (socket says 172.21.15.66:57662) 2024-03-30T00:45:23.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9ede000640 1 -- 172.21.15.66:0/3871372773 learned_addr learned my addr 172.21.15.66:0/3871372773 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9ee6a00640 1 --2- 172.21.15.66:0/3871372773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ee815a640 0x7f9ee806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9ede000640 1 -- 172.21.15.66:0/3871372773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ee815a640 msgr2=0x7f9ee806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:23.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9ede000640 1 --2- 172.21.15.66:0/3871372773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ee815a640 0x7f9ee806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9ede000640 1 -- 172.21.15.66:0/3871372773 >> v1:172.21.15.156:6789/0 conn(0x7f9ee80cabb0 legacy=0x7f9ee80cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9ede000640 1 -- 172.21.15.66:0/3871372773 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee80cee60 con 0x7f9ee80ce520 2024-03-30T00:45:23.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.915+0000 7f9ee6a00640 1 --2- 172.21.15.66:0/3871372773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ee815a640 0x7f9ee806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:23.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9ede000640 1 --2- 172.21.15.66:0/3871372773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee80ce520 0x7f9ee80ce920 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009870 tx=0x7f9ed001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e336af7da99fadfa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9ee6000640 1 -- 172.21.15.66:0/3871372773 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00ab650 con 0x7f9ee80ce520 2024-03-30T00:45:23.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9ee6000640 1 -- 172.21.15.66:0/3871372773 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed00ab7b0 con 0x7f9ee80ce520 2024-03-30T00:45:23.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9ee6000640 1 -- 172.21.15.66:0/3871372773 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00abb40 con 0x7f9ee80ce520 2024-03-30T00:45:23.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9eedc00640 1 -- 172.21.15.66:0/3871372773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee80ce520 msgr2=0x7f9ee80ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9eedc00640 1 --2- 172.21.15.66:0/3871372773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee80ce520 0x7f9ee80ce920 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009870 tx=0x7f9ed001c560 comp rx=0 tx=0).stop 2024-03-30T00:45:23.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9eedc00640 1 -- 172.21.15.66:0/3871372773 shutdown_connections 2024-03-30T00:45:23.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9eedc00640 1 --2- 172.21.15.66:0/3871372773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee80ce520 0x7f9ee80ce920 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9eedc00640 1 --2- 172.21.15.66:0/3871372773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ee815a640 0x7f9ee806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9eedc00640 1 -- 172.21.15.66:0/3871372773 >> 172.21.15.66:0/3871372773 conn(0x7f9ee8069f60 msgr2=0x7f9ee806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:23.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9eedc00640 1 -- 172.21.15.66:0/3871372773 shutdown_connections 2024-03-30T00:45:23.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9eedc00640 1 -- 172.21.15.66:0/3871372773 wait complete. 2024-03-30T00:45:23.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.919+0000 7f9eedc00640 1 Processor -- start 2024-03-30T00:45:23.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9eedc00640 1 -- start start 2024-03-30T00:45:23.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9eedc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee80cabb0 0x7f9ee80979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9eedc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ee80ce520 0x7f9ee815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9eedc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee815a640 0x7f9ee815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9eedc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee815d600 con 0x7f9ee815a640 2024-03-30T00:45:23.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9eedc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee8097f10 con 0x7f9ee80cabb0 2024-03-30T00:45:23.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9eedc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee815c980 con 0x7f9ee80ce520 2024-03-30T00:45:23.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9ee7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee815a640 0x7f9ee815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9ee6a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee80cabb0 0x7f9ee80979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9ee6a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee80cabb0 0x7f9ee80979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:56620/0 (socket says 172.21.15.66:56620) 2024-03-30T00:45:23.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9ee6a00640 1 -- 172.21.15.66:0/837645490 learned_addr learned my addr 172.21.15.66:0/837645490 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:23.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9ede000640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ee80ce520 0x7f9ee815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9ee7400640 1 -- 172.21.15.66:0/837645490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ee80ce520 msgr2=0x7f9ee815bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9ee7400640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ee80ce520 0x7f9ee815bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9ee7400640 1 -- 172.21.15.66:0/837645490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee80cabb0 msgr2=0x7f9ee80979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:23.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9ee7400640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee80cabb0 0x7f9ee80979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:23.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9ee7400640 1 -- 172.21.15.66:0/837645490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed00094d0 con 0x7f9ee815a640 2024-03-30T00:45:23.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9ee7400640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee815a640 0x7f9ee815c0f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000c730 tx=0x7f9ee000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9edf400640 1 -- 172.21.15.66:0/837645490 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0003b80 con 0x7f9ee815a640 2024-03-30T00:45:23.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9edf400640 1 -- 172.21.15.66:0/837645490 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee0003ce0 con 0x7f9ee815a640 2024-03-30T00:45:23.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9edf400640 1 -- 172.21.15.66:0/837645490 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee000aa30 con 0x7f9ee815a640 2024-03-30T00:45:23.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9eedc00640 1 -- 172.21.15.66:0/837645490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee815cc60 con 0x7f9ee815a640 2024-03-30T00:45:23.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9eedc00640 1 -- 172.21.15.66:0/837645490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee815d220 con 0x7f9ee815a640 2024-03-30T00:45:23.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.923+0000 7f9eedc00640 1 -- 172.21.15.66:0/837645490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee80cbec0 con 0x7f9ee815a640 2024-03-30T00:45:23.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.927+0000 7f9edf400640 1 -- 172.21.15.66:0/837645490 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9ee000ab90 con 0x7f9ee815a640 2024-03-30T00:45:23.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.927+0000 7f9edf400640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ec4066690 0x7f9ec4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:23.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.927+0000 7f9edf400640 1 -- 172.21.15.66:0/837645490 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f9ee0083ff0 con 0x7f9ee815a640 2024-03-30T00:45:23.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.927+0000 7f9ee6a00640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ec4066690 0x7f9ec4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:23.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.927+0000 7f9ee6a00640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ec4066690 0x7f9ec4068b50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800bb70 tx=0x7f9ed8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:23.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:23.931+0000 7f9edf400640 1 -- 172.21.15.66:0/837645490 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9ee00b5980 con 0x7f9ee815a640 2024-03-30T00:45:24.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.123+0000 7f9eedc00640 1 -- 172.21.15.66:0/837645490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9ee8159e60 con 0x7f9ee815a640 2024-03-30T00:45:24.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9edf400640 1 -- 172.21.15.66:0/837645490 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9ee80cbec0 con 0x7f9ee815a640 2024-03-30T00:45:24.133 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:45:24.133 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-30T00:45:24.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 -- 172.21.15.66:0/837645490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ec4066690 msgr2=0x7f9ec4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ec4066690 0x7f9ec4068b50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800bb70 tx=0x7f9ed8008040 comp rx=0 tx=0).stop 2024-03-30T00:45:24.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 -- 172.21.15.66:0/837645490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee815a640 msgr2=0x7f9ee815c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee815a640 0x7f9ee815c0f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000c730 tx=0x7f9ee000cbf0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 -- 172.21.15.66:0/837645490 shutdown_connections 2024-03-30T00:45:24.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ec4066690 0x7f9ec4068b50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee815a640 0x7f9ee815c0f0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ee80ce520 0x7f9ee815bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 --2- 172.21.15.66:0/837645490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee80cabb0 0x7f9ee80979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 -- 172.21.15.66:0/837645490 >> 172.21.15.66:0/837645490 conn(0x7f9ee8069f60 msgr2=0x7f9ee8158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 -- 172.21.15.66:0/837645490 shutdown_connections 2024-03-30T00:45:24.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.127+0000 7f9eedc00640 1 -- 172.21.15.66:0/837645490 wait complete. 2024-03-30T00:45:24.154 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-30T00:45:24.155 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-03-30T00:45:24.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.291+0000 7ff8de200640 1 Processor -- start 2024-03-30T00:45:24.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.291+0000 7ff8de200640 1 -- start start 2024-03-30T00:45:24.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.291+0000 7ff8de200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d80acaf0 0x7ff8d80b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.291+0000 7ff8de200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d80cbf80 con 0x7ff8d80ceec0 2024-03-30T00:45:24.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.291+0000 7ff8de200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d80cc820 con 0x7ff8d815ba00 2024-03-30T00:45:24.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.291+0000 7ff8de200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80b2dd0 con 0x7ff8d80acaf0 2024-03-30T00:45:24.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.291+0000 7ff8d7e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff8d80ceec0 0x7ff8d81579e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57808/0 (socket says 172.21.15.66:57808) 2024-03-30T00:45:24.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.291+0000 7ff8d7e00640 1 -- 172.21.15.66:0/4163504974 learned_addr learned my addr 172.21.15.66:0/4163504974 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:45:24.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.291+0000 7ff8d7400640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d80acaf0 0x7ff8d80b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.291+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974708564 0 0) 0x7ff8d80cbf80 con 0x7ff8d80ceec0 2024-03-30T00:45:24.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.291+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8bc003530 con 0x7ff8d80ceec0 2024-03-30T00:45:24.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543216467 0 0) 0x7ff8d80cc820 con 0x7ff8d815ba00 2024-03-30T00:45:24.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d80cbf80 con 0x7ff8d815ba00 2024-03-30T00:45:24.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783187659 0 0) 0x7ff8bc003530 con 0x7ff8d80ceec0 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d80acaf0 msgr2=0x7ff8d80b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d80acaf0 0x7ff8d80b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 >> v1:172.21.15.156:6789/0 conn(0x7ff8d815ba00 legacy=0x7ff8d80ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d80cc1d0 con 0x7ff8d80ceec0 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7ff8d0001a00 con 0x7ff8d80ceec0 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d7400640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d80acaf0 0x7ff8d80b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 >> v1:172.21.15.66:6789/0 conn(0x7ff8d80ceec0 legacy=0x7ff8d81579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8bc003b40 0x7ff8bc003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d80acaf0 0x7ff8bc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc004d10 0x7ff8bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80b2dd0 con 0x7ff8bc003b40 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc00ba70 con 0x7ff8bc004d10 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc00bc30 con 0x7ff8d80acaf0 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d7400640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8bc003b40 0x7ff8bc003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d7e00640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc004d10 0x7ff8bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d80acaf0 0x7ff8bc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d7400640 1 -- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d80acaf0 msgr2=0x7ff8bc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d7400640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d80acaf0 0x7ff8bc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d7400640 1 -- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc004d10 msgr2=0x7ff8bc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d7400640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc004d10 0x7ff8bc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d7400640 1 -- 172.21.15.66:0/4163504974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c8009420 con 0x7ff8bc003b40 2024-03-30T00:45:24.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.295+0000 7ff8d7400640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8bc003b40 0x7ff8bc003f40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009cb0 tx=0x7ff8c801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89eb593d4cb45ba5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80ad0e0 con 0x7ff8bc003b40 2024-03-30T00:45:24.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c80a58a0 con 0x7ff8bc003b40 2024-03-30T00:45:24.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8d6a00640 1 -- 172.21.15.66:0/4163504974 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80a5c50 con 0x7ff8bc003b40 2024-03-30T00:45:24.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 -- 172.21.15.66:0/4163504974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8bc003b40 msgr2=0x7ff8bc003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 --2- 172.21.15.66:0/4163504974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8bc003b40 0x7ff8bc003f40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009cb0 tx=0x7ff8c801cbe0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8d7e00640 1 -- 172.21.15.66:0/4163504974 reap_dead start 2024-03-30T00:45:24.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 -- 172.21.15.66:0/4163504974 shutdown_connections 2024-03-30T00:45:24.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 -- 172.21.15.66:0/4163504974 >> 172.21.15.66:0/4163504974 conn(0x7ff8d806a190 msgr2=0x7ff8d806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 -- 172.21.15.66:0/4163504974 shutdown_connections 2024-03-30T00:45:24.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 -- 172.21.15.66:0/4163504974 wait complete. 2024-03-30T00:45:24.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 Processor -- start 2024-03-30T00:45:24.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 -- start start 2024-03-30T00:45:24.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc003b40 0x7ff8d80c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d8156bd0 0x7ff8d81562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d8162400 0x7ff8d80cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80c6d40 con 0x7ff8d8156bd0 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80c8780 con 0x7ff8bc003b40 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8de200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80c6a60 con 0x7ff8d8162400 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d8156bd0 0x7ff8d81562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d8156bd0 0x7ff8d81562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57678/0 (socket says 172.21.15.66:57678) 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8cfe00640 1 -- 172.21.15.66:0/2732951199 learned_addr learned my addr 172.21.15.66:0/2732951199 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8d7400640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc003b40 0x7ff8d80c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.299+0000 7ff8d7e00640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d8162400 0x7ff8d80cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cfe00640 1 -- 172.21.15.66:0/2732951199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d8162400 msgr2=0x7ff8d80cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d8162400 0x7ff8d80cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cfe00640 1 -- 172.21.15.66:0/2732951199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc003b40 msgr2=0x7ff8d80c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc003b40 0x7ff8d80c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cfe00640 1 -- 172.21.15.66:0/2732951199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d80ab430 con 0x7ff8d8156bd0 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8d7e00640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d8162400 0x7ff8d80cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:24.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d8156bd0 0x7ff8d81562a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000e7b0 tx=0x7ff8c000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cf400640 1 -- 172.21.15.66:0/2732951199 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000ee60 con 0x7ff8d8156bd0 2024-03-30T00:45:24.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cf400640 1 -- 172.21.15.66:0/2732951199 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c00052e0 con 0x7ff8d8156bd0 2024-03-30T00:45:24.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8de200640 1 -- 172.21.15.66:0/2732951199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d80c33f0 con 0x7ff8d8156bd0 2024-03-30T00:45:24.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cf400640 1 -- 172.21.15.66:0/2732951199 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000c900 con 0x7ff8d8156bd0 2024-03-30T00:45:24.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8de200640 1 -- 172.21.15.66:0/2732951199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d80c39b0 con 0x7ff8d8156bd0 2024-03-30T00:45:24.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cf400640 1 -- 172.21.15.66:0/2732951199 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff8c000ca60 con 0x7ff8d8156bd0 2024-03-30T00:45:24.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cf400640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8b40666c0 0x7ff8b4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.303+0000 7ff8cf400640 1 -- 172.21.15.66:0/2732951199 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff8c00841a0 con 0x7ff8d8156bd0 2024-03-30T00:45:24.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8d7400640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8b40666c0 0x7ff8b4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 Processor -- start 2024-03-30T00:45:24.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 -- start start 2024-03-30T00:45:24.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8d7400640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8b40666c0 0x7ff8b4068b80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80af000 tx=0x7ff8c80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb9a0dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb99c3d70 0x55bfb99c4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb988b820 con 0x55bfb9a0e320 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb9818a20 con 0x55bfb99c3d70 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb9815ff0 con 0x55bfb9a0da00 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57692/0 (socket says 172.21.15.66:57692) 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8cfe00640 1 -- 172.21.15.66:0/2724883263 learned_addr learned my addr 172.21.15.66:0/2724883263 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8d7e00640 1 --2- 172.21.15.66:0/2724883263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb99c3d70 0x55bfb99c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8d7400640 1 --2- 172.21.15.66:0/2724883263 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb9a0dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8cfe00640 1 -- 172.21.15.66:0/2724883263 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 msgr2=0x55bfb9a0dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/2724883263 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb9a0dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8cfe00640 1 -- 172.21.15.66:0/2724883263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb99c3d70 msgr2=0x55bfb99c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/2724883263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb99c3d70 0x55bfb99c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8cfe00640 1 -- 172.21.15.66:0/2724883263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb950dc90 con 0x55bfb9a0e320 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/2724883263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c3830 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00050a0 tx=0x7ff8c0054980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a9cbbf2859f3c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8c7e00640 1 -- 172.21.15.66:0/2724883263 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0058e50 con 0x55bfb9a0e320 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8c7e00640 1 -- 172.21.15.66:0/2724883263 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c005d410 con 0x55bfb9a0e320 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8c7e00640 1 -- 172.21.15.66:0/2724883263 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0005450 con 0x55bfb9a0e320 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 -- 172.21.15.66:0/2724883263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 msgr2=0x55bfb99c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/2724883263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c3830 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00050a0 tx=0x7ff8c0054980 comp rx=0 tx=0).stop 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 -- 172.21.15.66:0/2724883263 shutdown_connections 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/2724883263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb99c3d70 0x55bfb99c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/2724883263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c3830 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/2724883263 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb9a0dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 -- 172.21.15.66:0/2724883263 >> 172.21.15.66:0/2724883263 conn(0x55bfb99ca2f0 msgr2=0x55bfb99c2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 -- 172.21.15.66:0/2724883263 shutdown_connections 2024-03-30T00:45:24.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 -- 172.21.15.66:0/2724883263 wait complete. 2024-03-30T00:45:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.307+0000 7ff8e16b3000 1 -- start start 2024-03-30T00:45:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 1 Processor -- start 2024-03-30T00:45:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb99c3d70 0x55bfb9a0d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb99c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb98118e0 con 0x55bfb9a0e320 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb97b1230 con 0x55bfb99c3d70 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb97b0ed0 con 0x55bfb9a0da00 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 10 client.0 _ll_register_callbacks cb 0x55bfb99af700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57700/0 (socket says 172.21.15.66:57700) 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7e00640 1 -- 172.21.15.66:0/3422511719 learned_addr learned my addr 172.21.15.66:0/3422511719 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb99c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7400640 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb99c3d70 0x55bfb9a0d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7e00640 1 -- 172.21.15.66:0/3422511719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 msgr2=0x55bfb99c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7e00640 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb99c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7e00640 1 -- 172.21.15.66:0/3422511719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb99c3d70 msgr2=0x55bfb9a0d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7e00640 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb99c3d70 0x55bfb9a0d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7e00640 1 -- 172.21.15.66:0/3422511719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d80c39b0 con 0x55bfb9a0e320 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7e00640 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c9440 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80c1e30 tx=0x7ff8d00078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e7483f02d5e90a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3422511719 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d000a9b0 con 0x55bfb9a0e320 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3422511719 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff8d000ab10 con 0x55bfb9a0e320 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3422511719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfb97ca6c0 con 0x55bfb9a0e320 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3422511719 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8d000acd0 con 0x55bfb9a0e320 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 10 client.7425 fetch_fsmap learned FSMap version 50 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3422511719 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55bfb97d6be0 con 0x55bfb9a0e320 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3422511719 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bfb97ca6c0 con 0x55bfb9a0e320 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3422511719 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff8d0012050 con 0x55bfb9a0e320 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 10 client.7425 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 10 client.7425 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 20 client.7425 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55bfb99d0480 0x55bfb9a1fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 4 client.7425 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3422511719 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfb97dab40 con 0x55bfb99d0480 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7400640 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55bfb99d0480 0x55bfb9a1fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8c4c00640 10 client.7425 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.311+0000 7ff8d7400640 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55bfb99d0480 0x55bfb9a1fcc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59aabb52c9370191 server_cookie=c91adbad6030a76d in_seq=0 out_seq=0 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3422511719 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bfb97dab40 con 0x55bfb99d0480 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8c4c00640 10 client.7425 handle_command_reply: tid=0 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 1 client.7425 shutdown 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3422511719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 msgr2=0x55bfb99c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c9440 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80c1e30 tx=0x7ff8d00078c0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3422511719 shutdown_connections 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55bfb99d0480 0x55bfb9a1fcc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb9a0e320 0x55bfb99c9440 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb99c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8c4c00640 0 client.7425 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:24.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3422511719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb99c3d70 0x55bfb9a0d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3422511719 >> 172.21.15.66:0/3422511719 conn(0x55bfb99ca2f0 msgr2=0x55bfb99c2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3422511719 shutdown_connections 2024-03-30T00:45:24.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3422511719 wait complete. 2024-03-30T00:45:24.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.315+0000 7ff8e16b3000 20 client.7425 trim_cache size 0 max 16384 2024-03-30T00:45:24.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.327+0000 7ff8e16b3000 1 Processor -- start 2024-03-30T00:45:24.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.327+0000 7ff8e16b3000 1 -- start start 2024-03-30T00:45:24.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a0c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb9a0da00 0x55bfb9a0c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0e320 0x55bfb9a1dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb99cf820 con 0x55bfb99c3d70 2024-03-30T00:45:24.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb94ced40 con 0x55bfb9a0da00 2024-03-30T00:45:24.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb99c7430 con 0x55bfb9a0e320 2024-03-30T00:45:24.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a0c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a0c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57714/0 (socket says 172.21.15.66:57714) 2024-03-30T00:45:24.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8d7400640 1 -- 172.21.15.66:0/1230073294 learned_addr learned my addr 172.21.15.66:0/1230073294 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:24.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8d7e00640 1 --2- 172.21.15.66:0/1230073294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0e320 0x55bfb9a1dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/1230073294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb9a0da00 0x55bfb9a0c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8d7400640 1 -- 172.21.15.66:0/1230073294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0e320 msgr2=0x55bfb9a1dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8d7400640 1 --2- 172.21.15.66:0/1230073294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0e320 0x55bfb9a1dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8d7400640 1 -- 172.21.15.66:0/1230073294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb9a0da00 msgr2=0x55bfb9a0c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8d7400640 1 --2- 172.21.15.66:0/1230073294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb9a0da00 0x55bfb9a0c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8d7400640 1 -- 172.21.15.66:0/1230073294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb97bcd30 con 0x55bfb99c3d70 2024-03-30T00:45:24.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8d7400640 1 --2- 172.21.15.66:0/1230073294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a0c180 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80b0040 tx=0x7ff8c80a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e403a601c75817 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8ccc00640 1 -- 172.21.15.66:0/1230073294 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80a5eb0 con 0x55bfb99c3d70 2024-03-30T00:45:24.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8ccc00640 1 -- 172.21.15.66:0/1230073294 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c80cb370 con 0x55bfb99c3d70 2024-03-30T00:45:24.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8ccc00640 1 -- 172.21.15.66:0/1230073294 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8004020 con 0x55bfb99c3d70 2024-03-30T00:45:24.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 -- 172.21.15.66:0/1230073294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 msgr2=0x55bfb9a0c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/1230073294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a0c180 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80b0040 tx=0x7ff8c80a7070 comp rx=0 tx=0).stop 2024-03-30T00:45:24.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 -- 172.21.15.66:0/1230073294 shutdown_connections 2024-03-30T00:45:24.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/1230073294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0e320 0x55bfb9a1dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/1230073294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb9a0da00 0x55bfb9a0c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/1230073294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a0c180 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 -- 172.21.15.66:0/1230073294 >> 172.21.15.66:0/1230073294 conn(0x55bfb99ca2f0 msgr2=0x55bfb9a0aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 -- 172.21.15.66:0/1230073294 shutdown_connections 2024-03-30T00:45:24.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 -- 172.21.15.66:0/1230073294 wait complete. 2024-03-30T00:45:24.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 -- start start 2024-03-30T00:45:24.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.331+0000 7ff8e16b3000 1 Processor -- start 2024-03-30T00:45:24.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a21860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb9a29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb9a0e320 0x55bfb9a295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb980d0d0 con 0x55bfb99c3d70 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb97db590 con 0x55bfb9a0e320 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb99c7f60 con 0x55bfb9a0da00 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 10 client.0 _ll_register_callbacks cb 0x55bfb998fb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a21860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8d7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a21860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57726/0 (socket says 172.21.15.66:57726) 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8d7400640 1 -- 172.21.15.66:0/3421655610 learned_addr learned my addr 172.21.15.66:0/3421655610 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb9a29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8d7e00640 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb9a0e320 0x55bfb9a295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8d7400640 1 -- 172.21.15.66:0/3421655610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 msgr2=0x55bfb9a29080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8d7400640 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb9a29080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8d7400640 1 -- 172.21.15.66:0/3421655610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb9a0e320 msgr2=0x55bfb9a295c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8d7400640 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb9a0e320 0x55bfb9a295c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8d7400640 1 -- 172.21.15.66:0/3421655610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c8009420 con 0x55bfb99c3d70 2024-03-30T00:45:24.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8d7400640 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a21860 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80b0040 tx=0x7ff8c8019980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2957c31fe03fdb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3421655610 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8004020 con 0x55bfb99c3d70 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3421655610 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff8c8004180 con 0x55bfb99c3d70 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3421655610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfb97b7fa0 con 0x55bfb99c3d70 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3421655610 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8c8004340 con 0x55bfb99c3d70 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 10 client.7431 fetch_fsmap learned FSMap version 50 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3421655610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55bfb950e0c0 con 0x55bfb99c3d70 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3421655610 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bfb97b7fa0 con 0x55bfb99c3d70 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3421655610 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff8c80bf040 con 0x55bfb99c3d70 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 10 client.7431 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 10 client.7431 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 20 client.7431 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55bfb9a6c7a0 0x55bfb9a6cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 4 client.7431 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-03-30T00:45:24.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3421655610 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x55bfb980e6c0 con 0x55bfb9a6c7a0 2024-03-30T00:45:24.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55bfb9a6c7a0 0x55bfb9a6cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8cfe00640 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55bfb9a6c7a0 0x55bfb9a6cbb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49369cb69bfc01f3 server_cookie=57ee6109e6918baf in_seq=0 out_seq=0 2024-03-30T00:45:24.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8c4c00640 10 client.7431 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8c4c00640 1 -- 172.21.15.66:0/3421655610 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bfb980e6c0 con 0x55bfb9a6c7a0 2024-03-30T00:45:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8c4c00640 10 client.7431 handle_command_reply: tid=0 2024-03-30T00:45:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 client.7431 shutdown 2024-03-30T00:45:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3421655610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 msgr2=0x55bfb9a21860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.335+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a21860 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80b0040 tx=0x7ff8c8019980 comp rx=0 tx=0).stop 2024-03-30T00:45:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3421655610 shutdown_connections 2024-03-30T00:45:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55bfb9a6c7a0 0x55bfb9a6cbb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfb9a0e320 0x55bfb9a295c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55bfb9a0da00 0x55bfb9a29080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8e16b3000 1 --2- 172.21.15.66:0/3421655610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfb99c3d70 0x55bfb9a21860 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8c4c00640 0 client.7431 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3421655610 >> 172.21.15.66:0/3421655610 conn(0x55bfb99ca2f0 msgr2=0x55bfb9a0b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3421655610 shutdown_connections 2024-03-30T00:45:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8e16b3000 1 -- 172.21.15.66:0/3421655610 wait complete. 2024-03-30T00:45:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8e16b3000 20 client.7431 trim_cache size 0 max 16384 2024-03-30T00:45:24.348 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:45:24.348 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T00:45:24.348 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "a9395b65-dabf-4e2d-922d-1fe73f8adc07", 2024-03-30T00:45:24.348 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T00:45:24.348 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:45:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 -- 172.21.15.66:0/2732951199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8b40666c0 msgr2=0x7ff8b4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8b40666c0 0x7ff8b4068b80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80af000 tx=0x7ff8c80046d0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 -- 172.21.15.66:0/2732951199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d8156bd0 msgr2=0x7ff8d81562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d8156bd0 0x7ff8d81562a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000e7b0 tx=0x7ff8c000ec70 comp rx=0 tx=0).stop 2024-03-30T00:45:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 -- 172.21.15.66:0/2732951199 shutdown_connections 2024-03-30T00:45:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8d8162400 0x7ff8d80cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8b40666c0 0x7ff8b4068b80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d8156bd0 0x7ff8d81562a0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 --2- 172.21.15.66:0/2732951199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc003b40 0x7ff8d80c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 -- 172.21.15.66:0/2732951199 >> 172.21.15.66:0/2732951199 conn(0x7ff8d806a190 msgr2=0x7ff8d8070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 -- 172.21.15.66:0/2732951199 shutdown_connections 2024-03-30T00:45:24.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.339+0000 7ff8de200640 1 -- 172.21.15.66:0/2732951199 wait complete. 2024-03-30T00:45:24.364 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-30T00:45:24.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb81200640 1 Processor -- start 2024-03-30T00:45:24.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb81200640 1 -- start start 2024-03-30T00:45:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb81200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c0cda90 0x7fbb7c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb81200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbb7c15a370 0x7fbb7c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb81200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c15a990 con 0x7fbb7c0cde70 2024-03-30T00:45:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb81200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b1b80 con 0x7fbb7c15a370 2024-03-30T00:45:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb81200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0cd3e0 con 0x7fbb7c0cda90 2024-03-30T00:45:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb73e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fbb7c0cde70 0x7fbb7c159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57820/0 (socket says 172.21.15.66:57820) 2024-03-30T00:45:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb73e00640 1 -- 172.21.15.66:0/661952889 learned_addr learned my addr 172.21.15.66:0/661952889 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:45:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb79e00640 1 --2- 172.21.15.66:0/661952889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c0cda90 0x7fbb7c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb7a800640 1 --2- 172.21.15.66:0/661952889 >> v2:172.21.15.156:3300/0 conn(0x7fbb7c15a370 0x7fbb7c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb79400640 1 -- 172.21.15.66:0/661952889 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584731868 0 0) 0x7fbb7c15a990 con 0x7fbb7c0cde70 2024-03-30T00:45:24.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb79400640 1 -- 172.21.15.66:0/661952889 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb60003530 con 0x7fbb7c0cde70 2024-03-30T00:45:24.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb79e00640 1 -- 172.21.15.66:0/661952889 >> v2:172.21.15.156:3300/0 conn(0x7fbb7c15a370 msgr2=0x7fbb7c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb79e00640 1 --2- 172.21.15.66:0/661952889 >> v2:172.21.15.156:3300/0 conn(0x7fbb7c15a370 0x7fbb7c0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb79e00640 1 -- 172.21.15.66:0/661952889 >> v1:172.21.15.66:6789/0 conn(0x7fbb7c0cde70 legacy=0x7fbb7c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb79e00640 1 -- 172.21.15.66:0/661952889 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c0b11e0 con 0x7fbb7c0cda90 2024-03-30T00:45:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.499+0000 7fbb7a800640 1 --2- 172.21.15.66:0/661952889 >> v2:172.21.15.156:3300/0 conn(0x7fbb7c15a370 0x7fbb7c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:45:24.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb79e00640 1 --2- 172.21.15.66:0/661952889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c0cda90 0x7fbb7c0ce520 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0097c0 tx=0x7fbb6c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c44a1d5f2a77744b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb79400640 1 -- 172.21.15.66:0/661952889 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0ab600 con 0x7fbb7c0cda90 2024-03-30T00:45:24.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb79400640 1 -- 172.21.15.66:0/661952889 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c0a7050 con 0x7fbb7c0cda90 2024-03-30T00:45:24.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb79400640 1 -- 172.21.15.66:0/661952889 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0ab980 con 0x7fbb7c0cda90 2024-03-30T00:45:24.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb81200640 1 -- 172.21.15.66:0/661952889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c0cda90 msgr2=0x7fbb7c0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb81200640 1 --2- 172.21.15.66:0/661952889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c0cda90 0x7fbb7c0ce520 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0097c0 tx=0x7fbb6c01c410 comp rx=0 tx=0).stop 2024-03-30T00:45:24.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb81200640 1 -- 172.21.15.66:0/661952889 shutdown_connections 2024-03-30T00:45:24.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb81200640 1 --2- 172.21.15.66:0/661952889 >> v2:172.21.15.156:3300/0 conn(0x7fbb7c15a370 0x7fbb7c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb81200640 1 --2- 172.21.15.66:0/661952889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c0cda90 0x7fbb7c0ce520 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb81200640 1 -- 172.21.15.66:0/661952889 >> 172.21.15.66:0/661952889 conn(0x7fbb7c069f60 msgr2=0x7fbb7c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb81200640 1 -- 172.21.15.66:0/661952889 shutdown_connections 2024-03-30T00:45:24.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.503+0000 7fbb81200640 1 -- 172.21.15.66:0/661952889 wait complete. 2024-03-30T00:45:24.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb81200640 1 Processor -- start 2024-03-30T00:45:24.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb81200640 1 -- start start 2024-03-30T00:45:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb81200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c0cda90 0x7fbb7c156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb81200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb7c0cde70 0x7fbb7c0af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb81200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c15a370 0x7fbb7c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb81200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c06fed0 con 0x7fbb7c0cda90 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb81200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c158810 con 0x7fbb7c0cde70 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb81200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c158af0 con 0x7fbb7c15a370 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb79e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c0cda90 0x7fbb7c156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb79e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c0cda90 0x7fbb7c156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57740/0 (socket says 172.21.15.66:57740) 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb79e00640 1 -- 172.21.15.66:0/986913529 learned_addr learned my addr 172.21.15.66:0/986913529 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb73e00640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb7c0cde70 0x7fbb7c0af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb7a800640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c15a370 0x7fbb7c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb79e00640 1 -- 172.21.15.66:0/986913529 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c15a370 msgr2=0x7fbb7c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb79e00640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c15a370 0x7fbb7c06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb79e00640 1 -- 172.21.15.66:0/986913529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb7c0cde70 msgr2=0x7fbb7c0af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb79e00640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb7c0cde70 0x7fbb7c0af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb79e00640 1 -- 172.21.15.66:0/986913529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb74009a70 con 0x7fbb7c0cda90 2024-03-30T00:45:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb7a800640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c15a370 0x7fbb7c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb79e00640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c0cda90 0x7fbb7c156370 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c01a040 tx=0x7fbb6c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb72000640 1 -- 172.21.15.66:0/986913529 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0b4410 con 0x7fbb7c0cda90 2024-03-30T00:45:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb72000640 1 -- 172.21.15.66:0/986913529 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c0b4590 con 0x7fbb7c0cda90 2024-03-30T00:45:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb81200640 1 -- 172.21.15.66:0/986913529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c009420 con 0x7fbb7c0cda90 2024-03-30T00:45:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb72000640 1 -- 172.21.15.66:0/986913529 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0b4940 con 0x7fbb7c0cda90 2024-03-30T00:45:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.507+0000 7fbb81200640 1 -- 172.21.15.66:0/986913529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c070400 con 0x7fbb7c0cda90 2024-03-30T00:45:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb72000640 1 -- 172.21.15.66:0/986913529 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbb6c0b2040 con 0x7fbb7c0cda90 2024-03-30T00:45:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb72000640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb58066690 0x7fbb58068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb72000640 1 -- 172.21.15.66:0/986913529 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fbb6c11e310 con 0x7fbb7c0cda90 2024-03-30T00:45:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb73e00640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb58066690 0x7fbb58068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb8465b000 1 Processor -- start 2024-03-30T00:45:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb8465b000 1 -- start start 2024-03-30T00:45:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb73e00640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb58066690 0x7fbb58068b50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c15df10 tx=0x7fbb64007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62ba70 0x55640f62be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f5df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f5dfa00 0x55640f5dfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb8465b000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640f4aa630 con 0x55640f62c390 2024-03-30T00:45:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb8465b000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640f437bd0 con 0x55640f5dfa00 2024-03-30T00:45:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.511+0000 7fbb8465b000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55640f4351a0 con 0x55640f62ba70 2024-03-30T00:45:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb73e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f5df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb73e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f5df4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57748/0 (socket says 172.21.15.66:57748) 2024-03-30T00:45:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb73e00640 1 -- 172.21.15.66:0/4077080769 learned_addr learned my addr 172.21.15.66:0/4077080769 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb79e00640 1 --2- 172.21.15.66:0/4077080769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62ba70 0x55640f62be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb7a800640 1 --2- 172.21.15.66:0/4077080769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f5dfa00 0x55640f5dfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb73e00640 1 -- 172.21.15.66:0/4077080769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62ba70 msgr2=0x55640f62be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb73e00640 1 --2- 172.21.15.66:0/4077080769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62ba70 0x55640f62be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb73e00640 1 -- 172.21.15.66:0/4077080769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f5dfa00 msgr2=0x55640f5dfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb73e00640 1 --2- 172.21.15.66:0/4077080769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f5dfa00 0x55640f5dfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb73e00640 1 -- 172.21.15.66:0/4077080769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55640f12ce20 con 0x55640f62c390 2024-03-30T00:45:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb7a800640 1 --2- 172.21.15.66:0/4077080769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f5dfa00 0x55640f5dfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb73e00640 1 --2- 172.21.15.66:0/4077080769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f5df4c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fbb64005a80 tx=0x7fbb64012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72dfbeb9b347cc32 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb6aa00640 1 -- 172.21.15.66:0/4077080769 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640014f0 con 0x55640f62c390 2024-03-30T00:45:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb6aa00640 1 -- 172.21.15.66:0/4077080769 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb64001650 con 0x55640f62c390 2024-03-30T00:45:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb6aa00640 1 -- 172.21.15.66:0/4077080769 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640019c0 con 0x55640f62c390 2024-03-30T00:45:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 -- 172.21.15.66:0/4077080769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 msgr2=0x55640f5df4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 --2- 172.21.15.66:0/4077080769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f5df4c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fbb64005a80 tx=0x7fbb64012000 comp rx=0 tx=0).stop 2024-03-30T00:45:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 -- 172.21.15.66:0/4077080769 shutdown_connections 2024-03-30T00:45:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 --2- 172.21.15.66:0/4077080769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f5dfa00 0x55640f5dfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 --2- 172.21.15.66:0/4077080769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f5df4c0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 --2- 172.21.15.66:0/4077080769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62ba70 0x55640f62be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 -- 172.21.15.66:0/4077080769 >> 172.21.15.66:0/4077080769 conn(0x55640f3e0160 msgr2=0x55640f60e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 -- 172.21.15.66:0/4077080769 shutdown_connections 2024-03-30T00:45:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 -- 172.21.15.66:0/4077080769 wait complete. 2024-03-30T00:45:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 -- start start 2024-03-30T00:45:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 Processor -- start 2024-03-30T00:45:24.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f5dfa00 0x55640f5e8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62ba70 0x55640f60fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f60ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640f430a90 con 0x55640f62c390 2024-03-30T00:45:24.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640f3cfd40 con 0x55640f62ba70 2024-03-30T00:45:24.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55640f3edf30 con 0x55640f5dfa00 2024-03-30T00:45:24.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb8465b000 10 client.0 _ll_register_callbacks cb 0x55640f492810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:24.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb7a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f60ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb79e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f5dfa00 0x55640f5e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb73e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62ba70 0x55640f60fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb7a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f60ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57752/0 (socket says 172.21.15.66:57752) 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb7a800640 1 -- 172.21.15.66:0/2116846971 learned_addr learned my addr 172.21.15.66:0/2116846971 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.515+0000 7fbb79e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f5dfa00 0x55640f5e8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:52172/0 (socket says 172.21.15.66:52172) 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb7a800640 1 -- 172.21.15.66:0/2116846971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f5dfa00 msgr2=0x55640f5e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb7a800640 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f5dfa00 0x55640f5e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb7a800640 1 -- 172.21.15.66:0/2116846971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62ba70 msgr2=0x55640f60fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb7a800640 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62ba70 0x55640f60fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb7a800640 1 -- 172.21.15.66:0/2116846971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb60003530 con 0x55640f62c390 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb79e00640 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f5dfa00 0x55640f5e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb73e00640 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62ba70 0x55640f60fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb7a800640 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f60ff90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fbb74007dd0 tx=0x7fbb74012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1734d91d4061347c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb5f400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb5f400640 1 -- 172.21.15.66:0/2116846971 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7400e040 con 0x55640f62c390 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb5f400640 1 -- 172.21.15.66:0/2116846971 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fbb74018670 con 0x55640f62c390 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 -- 172.21.15.66:0/2116846971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55640f42f3e0 con 0x55640f62c390 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb5f400640 1 -- 172.21.15.66:0/2116846971 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb74018830 con 0x55640f62c390 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 10 client.7443 fetch_fsmap learned FSMap version 50 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 -- 172.21.15.66:0/2116846971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55640f3f5aa0 con 0x55640f62c390 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb5f400640 1 -- 172.21.15.66:0/2116846971 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55640f42f3e0 con 0x55640f62c390 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb5f400640 1 -- 172.21.15.66:0/2116846971 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbb74014040 con 0x55640f62c390 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 10 client.7443 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 10 client.7443 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 20 client.7443 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55640f6332d0 0x55640f6336e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 4 client.7443 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 -- 172.21.15.66:0/2116846971 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55640f400d00 con 0x55640f6332d0 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb79e00640 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55640f6332d0 0x55640f6336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb5f400640 10 client.7443 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:24.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb79e00640 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55640f6332d0 0x55640f6336e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8549fd4de404352a server_cookie=4091a6f5b28b2c4b in_seq=0 out_seq=0 2024-03-30T00:45:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb5f400640 1 -- 172.21.15.66:0/2116846971 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55640f400d00 con 0x55640f6332d0 2024-03-30T00:45:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb5f400640 10 client.7443 handle_command_reply: tid=0 2024-03-30T00:45:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 client.7443 shutdown 2024-03-30T00:45:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 -- 172.21.15.66:0/2116846971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 msgr2=0x55640f60ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f60ff90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fbb74007dd0 tx=0x7fbb74012610 comp rx=0 tx=0).stop 2024-03-30T00:45:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 -- 172.21.15.66:0/2116846971 shutdown_connections 2024-03-30T00:45:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55640f6332d0 0x55640f6336e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f62c390 0x55640f60ff90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62ba70 0x55640f60fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 --2- 172.21.15.66:0/2116846971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f5dfa00 0x55640f5e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb5f400640 0 client.7443 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 -- 172.21.15.66:0/2116846971 >> 172.21.15.66:0/2116846971 conn(0x55640f3e0160 msgr2=0x55640f5debb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 -- 172.21.15.66:0/2116846971 shutdown_connections 2024-03-30T00:45:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 1 -- 172.21.15.66:0/2116846971 wait complete. 2024-03-30T00:45:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.519+0000 7fbb8465b000 20 client.7443 trim_cache size 0 max 16384 2024-03-30T00:45:24.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb8465b000 1 Processor -- start 2024-03-30T00:45:24.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb8465b000 1 -- start start 2024-03-30T00:45:24.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5dec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62ba70 0x55640f5df170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62c390 0x55640f5dfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb8465b000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640f0edaa0 con 0x55640f5dfa00 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb8465b000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640f5d9930 con 0x55640f62ba70 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb8465b000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55640f42c280 con 0x55640f62c390 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb79e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5dec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb79e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5dec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57760/0 (socket says 172.21.15.66:57760) 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb79e00640 1 -- 172.21.15.66:0/2729017721 learned_addr learned my addr 172.21.15.66:0/2729017721 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb7a800640 1 --2- 172.21.15.66:0/2729017721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62c390 0x55640f5dfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb73e00640 1 --2- 172.21.15.66:0/2729017721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62ba70 0x55640f5df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb79e00640 1 -- 172.21.15.66:0/2729017721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62c390 msgr2=0x55640f5dfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb79e00640 1 --2- 172.21.15.66:0/2729017721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62c390 0x55640f5dfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb79e00640 1 -- 172.21.15.66:0/2729017721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62ba70 msgr2=0x55640f5df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb79e00640 1 --2- 172.21.15.66:0/2729017721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62ba70 0x55640f5df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb79e00640 1 -- 172.21.15.66:0/2729017721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55640f3dbdf0 con 0x55640f5dfa00 2024-03-30T00:45:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.535+0000 7fbb79e00640 1 --2- 172.21.15.66:0/2729017721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5dec30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c097df0 tx=0x7fbb6c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9a33f78d2d07cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb6b400640 1 -- 172.21.15.66:0/2729017721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0f2fe0 con 0x55640f5dfa00 2024-03-30T00:45:24.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb6b400640 1 -- 172.21.15.66:0/2729017721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c0f75a0 con 0x55640f5dfa00 2024-03-30T00:45:24.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb6b400640 1 -- 172.21.15.66:0/2729017721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c11e0b0 con 0x55640f5dfa00 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 -- 172.21.15.66:0/2729017721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 msgr2=0x55640f5dec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 --2- 172.21.15.66:0/2729017721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5dec30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c097df0 tx=0x7fbb6c009190 comp rx=0 tx=0).stop 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 -- 172.21.15.66:0/2729017721 shutdown_connections 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 --2- 172.21.15.66:0/2729017721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62c390 0x55640f5dfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 --2- 172.21.15.66:0/2729017721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62ba70 0x55640f5df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 --2- 172.21.15.66:0/2729017721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5dec30 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 -- 172.21.15.66:0/2729017721 >> 172.21.15.66:0/2729017721 conn(0x55640f3e0160 msgr2=0x55640f5e7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 -- 172.21.15.66:0/2729017721 shutdown_connections 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 -- 172.21.15.66:0/2729017721 wait complete. 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 -- start start 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 Processor -- start 2024-03-30T00:45:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62ba70 0x55640f6471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62c390 0x55640f647720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640f60fc80 con 0x55640f5dfa00 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640f4e9ec0 con 0x55640f62c390 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55640f5e8cc0 con 0x55640f62ba70 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 10 client.0 _ll_register_callbacks cb 0x7fbb64006b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb79e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb79e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5e2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57770/0 (socket says 172.21.15.66:57770) 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb79e00640 1 -- 172.21.15.66:0/1757776286 learned_addr learned my addr 172.21.15.66:0/1757776286 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb73e00640 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62ba70 0x55640f6471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb7a800640 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62c390 0x55640f647720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb79e00640 1 -- 172.21.15.66:0/1757776286 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62ba70 msgr2=0x55640f6471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb79e00640 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62ba70 0x55640f6471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb79e00640 1 -- 172.21.15.66:0/1757776286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62c390 msgr2=0x55640f647720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb79e00640 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62c390 0x55640f647720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb79e00640 1 -- 172.21.15.66:0/1757776286 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c070400 con 0x55640f5dfa00 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb79e00640 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5e2f90 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c11c700 tx=0x7fbb6c11c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33679d02614c6240 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb5f400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb5f400640 1 -- 172.21.15.66:0/1757776286 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0ed550 con 0x55640f5dfa00 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb5f400640 1 -- 172.21.15.66:0/1757776286 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fbb6c10d0b0 con 0x55640f5dfa00 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 -- 172.21.15.66:0/1757776286 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55640f3ed970 con 0x55640f5dfa00 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb5f400640 1 -- 172.21.15.66:0/1757776286 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb6c10d270 con 0x55640f5dfa00 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 10 client.7449 fetch_fsmap learned FSMap version 50 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.539+0000 7fbb8465b000 1 -- 172.21.15.66:0/1757776286 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55640f3cf960 con 0x55640f5dfa00 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb5f400640 1 -- 172.21.15.66:0/1757776286 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55640f3ed970 con 0x55640f5dfa00 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb5f400640 1 -- 172.21.15.66:0/1757776286 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbb6c0ba020 con 0x55640f5dfa00 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 10 client.7449 fetch_fsmap finished waiting for FSMap version 50 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 10 client.7449 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:45:24.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 20 client.7449 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:24.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55640f647c60 0x55640f68bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:24.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 4 client.7449 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:45:24.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 -- 172.21.15.66:0/1757776286 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55640f45fa20 con 0x55640f647c60 2024-03-30T00:45:24.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb73e00640 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55640f647c60 0x55640f68bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:24.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb73e00640 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55640f647c60 0x55640f68bda0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebdbbbea9eb1155a server_cookie=b35b366e053acaf6 in_seq=0 out_seq=0 2024-03-30T00:45:24.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb5f400640 10 client.7449 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:24.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb5f400640 1 -- 172.21.15.66:0/1757776286 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55640f45fa20 con 0x55640f647c60 2024-03-30T00:45:24.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb5f400640 10 client.7449 handle_command_reply: tid=0 2024-03-30T00:45:24.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 client.7449 shutdown 2024-03-30T00:45:24.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 -- 172.21.15.66:0/1757776286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 msgr2=0x55640f5e2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5e2f90 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c11c700 tx=0x7fbb6c11c600 comp rx=0 tx=0).stop 2024-03-30T00:45:24.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 -- 172.21.15.66:0/1757776286 shutdown_connections 2024-03-30T00:45:24.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55640f647c60 0x55640f68bda0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55640f62c390 0x55640f647720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55640f62ba70 0x55640f6471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 --2- 172.21.15.66:0/1757776286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55640f5dfa00 0x55640f5e2f90 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb5f400640 0 client.7449 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:24.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 -- 172.21.15.66:0/1757776286 >> 172.21.15.66:0/1757776286 conn(0x55640f3e0160 msgr2=0x55640f5ddc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 -- 172.21.15.66:0/1757776286 shutdown_connections 2024-03-30T00:45:24.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 1 -- 172.21.15.66:0/1757776286 wait complete. 2024-03-30T00:45:24.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb8465b000 20 client.7449 trim_cache size 0 max 16384 2024-03-30T00:45:24.554 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:45:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:45:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-03-30T00:45:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: "a9395b65-dabf-4e2d-922d-1fe73f8adc07": { 2024-03-30T00:45:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/subdir", 2024-03-30T00:45:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "a9395b65-dabf-4e2d-922d-1fe73f8adc07", 2024-03-30T00:45:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,repair" 2024-03-30T00:45:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:45:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:45:24.555 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:45:24.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 -- 172.21.15.66:0/986913529 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb58066690 msgr2=0x7fbb58068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb58066690 0x7fbb58068b50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c15df10 tx=0x7fbb64007d40 comp rx=0 tx=0).stop 2024-03-30T00:45:24.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 -- 172.21.15.66:0/986913529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c0cda90 msgr2=0x7fbb7c156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:24.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c0cda90 0x7fbb7c156370 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c01a040 tx=0x7fbb6c0191b0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 -- 172.21.15.66:0/986913529 shutdown_connections 2024-03-30T00:45:24.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb7c15a370 0x7fbb7c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb7c0cde70 0x7fbb7c0af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb58066690 0x7fbb58068b50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 --2- 172.21.15.66:0/986913529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c0cda90 0x7fbb7c156370 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:24.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 -- 172.21.15.66:0/986913529 >> 172.21.15.66:0/986913529 conn(0x7fbb7c069f60 msgr2=0x7fbb7c164b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:24.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 -- 172.21.15.66:0/986913529 shutdown_connections 2024-03-30T00:45:24.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:24.543+0000 7fbb81200640 1 -- 172.21.15.66:0/986913529 wait complete. 2024-03-30T00:45:24.570 INFO:tasks.cephfs.filesystem:scrub status for tag:a9395b65-dabf-4e2d-922d-1fe73f8adc07 - {'path': '/subdir', 'tag': 'a9395b65-dabf-4e2d-922d-1fe73f8adc07', 'options': 'recursive,repair'} 2024-03-30T00:45:46.783 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:45:46.787 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:45:54.571 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-30T00:45:54.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.718+0000 7f7201e00640 1 Processor -- start 2024-03-30T00:45:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.718+0000 7f7201e00640 1 -- start start 2024-03-30T00:45:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.718+0000 7f7201e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71fc0cda90 0x7f71fc0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.718+0000 7f7201e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f71fc0cde70 0x7f71fc0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.718+0000 7f7201e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc15a990 con 0x7f71fc159c60 2024-03-30T00:45:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.718+0000 7f7201e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc0b1b80 con 0x7f71fc0cde70 2024-03-30T00:45:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.718+0000 7f7201e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0cd3e0 con 0x7f71fc0cda90 2024-03-30T00:45:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.718+0000 7f71f2000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f71fc0cde70 0x7f71fc0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.718+0000 7f71f2000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f71fc0cde70 0x7f71fc0cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:33994/0 (socket says 172.21.15.66:33994) 2024-03-30T00:45:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.718+0000 7f71f2000640 1 -- 172.21.15.66:0/1362810928 learned_addr learned my addr 172.21.15.66:0/1362810928 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.718+0000 7f71faa00640 1 --2- 172.21.15.66:0/1362810928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71fc0cda90 0x7f71fc0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f71f2000640 1 -- 172.21.15.66:0/1362810928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71fc0cda90 msgr2=0x7f71fc0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f71f2000640 1 --2- 172.21.15.66:0/1362810928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71fc0cda90 0x7f71fc0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f71f2000640 1 -- 172.21.15.66:0/1362810928 >> v1:172.21.15.156:6789/0 conn(0x7f71fc159c60 legacy=0x7f71fc15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f71f2000640 1 -- 172.21.15.66:0/1362810928 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc0b11e0 con 0x7f71fc0cde70 2024-03-30T00:45:54.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f71f2000640 1 --2- 172.21.15.66:0/1362810928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71fc0cde70 0x7f71fc0cea60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f71e400d150 tx=0x7f71e401eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca4ac981bebee3ec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:54.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f71fa000640 1 -- 172.21.15.66:0/1362810928 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e400b2d0 con 0x7f71fc0cde70 2024-03-30T00:45:54.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f71fa000640 1 -- 172.21.15.66:0/1362810928 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e400b430 con 0x7f71fc0cde70 2024-03-30T00:45:54.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f71fa000640 1 -- 172.21.15.66:0/1362810928 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40a7ba0 con 0x7f71fc0cde70 2024-03-30T00:45:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f7201e00640 1 -- 172.21.15.66:0/1362810928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71fc0cde70 msgr2=0x7f71fc0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f7201e00640 1 --2- 172.21.15.66:0/1362810928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71fc0cde70 0x7f71fc0cea60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f71e400d150 tx=0x7f71e401eba0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f7201e00640 1 -- 172.21.15.66:0/1362810928 shutdown_connections 2024-03-30T00:45:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f7201e00640 1 --2- 172.21.15.66:0/1362810928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71fc0cde70 0x7f71fc0cea60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f7201e00640 1 --2- 172.21.15.66:0/1362810928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71fc0cda90 0x7f71fc0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f7201e00640 1 -- 172.21.15.66:0/1362810928 >> 172.21.15.66:0/1362810928 conn(0x7f71fc069f60 msgr2=0x7f71fc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.722+0000 7f7201e00640 1 -- 172.21.15.66:0/1362810928 shutdown_connections 2024-03-30T00:45:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f7201e00640 1 -- 172.21.15.66:0/1362810928 wait complete. 2024-03-30T00:45:54.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f7201e00640 1 Processor -- start 2024-03-30T00:45:54.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f7201e00640 1 -- start start 2024-03-30T00:45:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f7201e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71fc0cda90 0x7f71fc1511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f7201e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71fc0cde70 0x7f71fc0ac980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f7201e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71fc159c60 0x7f71fc0acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f7201e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0ac050 con 0x7f71fc159c60 2024-03-30T00:45:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f7201e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc165200 con 0x7f71fc0cda90 2024-03-30T00:45:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f7201e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0aba20 con 0x7f71fc0cde70 2024-03-30T00:45:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71fc159c60 0x7f71fc0acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71fc159c60 0x7f71fc0acec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38524/0 (socket says 172.21.15.66:38524) 2024-03-30T00:45:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71fb400640 1 -- 172.21.15.66:0/2261547195 learned_addr learned my addr 172.21.15.66:0/2261547195 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71f2000640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71fc0cde70 0x7f71fc0ac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71faa00640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71fc0cda90 0x7f71fc1511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71fb400640 1 -- 172.21.15.66:0/2261547195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71fc0cde70 msgr2=0x7f71fc0ac980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71fb400640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71fc0cde70 0x7f71fc0ac980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71fb400640 1 -- 172.21.15.66:0/2261547195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71fc0cda90 msgr2=0x7f71fc1511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71fb400640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71fc0cda90 0x7f71fc1511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71fb400640 1 -- 172.21.15.66:0/2261547195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e400bd40 con 0x7f71fc159c60 2024-03-30T00:45:54.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71faa00640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71fc0cda90 0x7f71fc1511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:45:54.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.726+0000 7f71fb400640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71fc159c60 0x7f71fc0acec0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f71f400c460 tx=0x7f71f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:54.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.730+0000 7f71f3e00640 1 -- 172.21.15.66:0/2261547195 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4005220 con 0x7f71fc159c60 2024-03-30T00:45:54.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.730+0000 7f71f3e00640 1 -- 172.21.15.66:0/2261547195 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f4005380 con 0x7f71fc159c60 2024-03-30T00:45:54.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.730+0000 7f7201e00640 1 -- 172.21.15.66:0/2261547195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc070f80 con 0x7f71fc159c60 2024-03-30T00:45:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.730+0000 7f71f3e00640 1 -- 172.21.15.66:0/2261547195 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4007270 con 0x7f71fc159c60 2024-03-30T00:45:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.730+0000 7f7201e00640 1 -- 172.21.15.66:0/2261547195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc0abc70 con 0x7f71fc159c60 2024-03-30T00:45:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.730+0000 7f71f3e00640 1 -- 172.21.15.66:0/2261547195 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f71f40073d0 con 0x7f71fc159c60 2024-03-30T00:45:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.730+0000 7f71f3e00640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f71d8066690 0x7f71d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.730+0000 7f71f3e00640 1 -- 172.21.15.66:0/2261547195 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f71f4083a90 con 0x7f71fc159c60 2024-03-30T00:45:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.730+0000 7f71faa00640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f71d8066690 0x7f71d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 Processor -- start 2024-03-30T00:45:54.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 -- start start 2024-03-30T00:45:54.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71faa00640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f71d8066690 0x7f71d8068b50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009eb0 tx=0x7f71ec009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c397d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c397854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39785de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c39650630 con 0x559c39785a00 2024-03-30T00:45:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c395ddbd0 con 0x559c397d1a70 2024-03-30T00:45:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c395db1a0 con 0x559c397d2390 2024-03-30T00:45:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39785de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39785de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38530/0 (socket says 172.21.15.66:38530) 2024-03-30T00:45:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71fb400640 1 -- 172.21.15.66:0/295884247 learned_addr learned my addr 172.21.15.66:0/295884247 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71faa00640 1 --2- 172.21.15.66:0/295884247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c397d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71f2000640 1 --2- 172.21.15.66:0/295884247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c397854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71fb400640 1 -- 172.21.15.66:0/295884247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 msgr2=0x559c397854c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71fb400640 1 --2- 172.21.15.66:0/295884247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c397854c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71fb400640 1 -- 172.21.15.66:0/295884247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 msgr2=0x559c397d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71fb400640 1 --2- 172.21.15.66:0/295884247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c397d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71fb400640 1 -- 172.21.15.66:0/295884247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c392d2e20 con 0x559c39785a00 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71fb400640 1 --2- 172.21.15.66:0/295884247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39785de0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f71f405c7d0 tx=0x7f71f40543e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed446a7c9862ee7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71ebe00640 1 -- 172.21.15.66:0/295884247 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f405cd00 con 0x559c39785a00 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71ebe00640 1 -- 172.21.15.66:0/295884247 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f4052b00 con 0x559c39785a00 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f71ebe00640 1 -- 172.21.15.66:0/295884247 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40510d0 con 0x559c39785a00 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 -- 172.21.15.66:0/295884247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 msgr2=0x559c39785de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 --2- 172.21.15.66:0/295884247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39785de0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f71f405c7d0 tx=0x7f71f40543e0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 -- 172.21.15.66:0/295884247 shutdown_connections 2024-03-30T00:45:54.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 --2- 172.21.15.66:0/295884247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39785de0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 --2- 172.21.15.66:0/295884247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c397854c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 --2- 172.21.15.66:0/295884247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c397d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 -- 172.21.15.66:0/295884247 >> 172.21.15.66:0/295884247 conn(0x559c39586160 msgr2=0x559c397b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:54.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 -- 172.21.15.66:0/295884247 shutdown_connections 2024-03-30T00:45:54.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 -- 172.21.15.66:0/295884247 wait complete. 2024-03-30T00:45:54.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 -- start start 2024-03-30T00:45:54.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.734+0000 7f72051d4000 1 Processor -- start 2024-03-30T00:45:54.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c39785a00 0x559c3978e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d1a70 0x559c397b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c397d2390 0x559c397b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c395d6a90 con 0x559c397d2390 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c39575d40 con 0x559c39785a00 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c39593f30 con 0x559c397d1a70 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 10 client.0 _ll_register_callbacks cb 0x559c39638810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c397d2390 0x559c397b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c397d2390 0x559c397b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38534/0 (socket says 172.21.15.66:38534) 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71fb400640 1 -- 172.21.15.66:0/495674814 learned_addr learned my addr 172.21.15.66:0/495674814 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71faa00640 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c39785a00 0x559c3978e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71f2000640 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d1a70 0x559c397b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71fb400640 1 -- 172.21.15.66:0/495674814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d1a70 msgr2=0x559c397b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71fb400640 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d1a70 0x559c397b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71fb400640 1 -- 172.21.15.66:0/495674814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c39785a00 msgr2=0x559c3978e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71fb400640 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c39785a00 0x559c3978e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71fb400640 1 -- 172.21.15.66:0/495674814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc0abc70 con 0x559c397d2390 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71fb400640 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c397d2390 0x559c397b5f90 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f71f400ce20 tx=0x7f71f4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b82ccc342d0f9c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71e8c00640 1 -- 172.21.15.66:0/495674814 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f405cd00 con 0x559c397d2390 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71e8c00640 1 -- 172.21.15.66:0/495674814 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f71f4052d60 con 0x559c397d2390 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 1 -- 172.21.15.66:0/495674814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c395d53e0 con 0x559c397d2390 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71e8c00640 1 -- 172.21.15.66:0/495674814 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71f40510d0 con 0x559c397d2390 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 10 client.7458 fetch_fsmap learned FSMap version 51 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 1 -- 172.21.15.66:0/495674814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x559c3959baa0 con 0x559c397d2390 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71e8c00640 1 -- 172.21.15.66:0/495674814 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c395d53e0 con 0x559c397d2390 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71e8c00640 1 -- 172.21.15.66:0/495674814 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 51) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f71f4019030 con 0x559c397d2390 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 10 client.7458 fetch_fsmap finished waiting for FSMap version 51 2024-03-30T00:45:54.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 10 client.7458 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 20 client.7458 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x559c397d8e00 0x559c397d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 4 client.7458 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 1 -- 172.21.15.66:0/495674814 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c395a6d00 con 0x559c397d8e00 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71faa00640 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x559c397d8e00 0x559c397d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71faa00640 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x559c397d8e00 0x559c397d91e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1aadbed49f1e52e server_cookie=5b9d42dbf4ad4c2a in_seq=0 out_seq=0 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71e8c00640 10 client.7458 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71e8c00640 1 -- 172.21.15.66:0/495674814 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559c395a6d00 con 0x559c397d8e00 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f71e8c00640 10 client.7458 handle_command_reply: tid=0 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.738+0000 7f72051d4000 1 client.7458 shutdown 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f72051d4000 1 -- 172.21.15.66:0/495674814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c397d2390 msgr2=0x559c397b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f72051d4000 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c397d2390 0x559c397b5f90 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f71f400ce20 tx=0x7f71f4000ee0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f72051d4000 1 -- 172.21.15.66:0/495674814 shutdown_connections 2024-03-30T00:45:54.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f72051d4000 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x559c397d8e00 0x559c397d91e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f72051d4000 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c397d2390 0x559c397b5f90 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f72051d4000 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d1a70 0x559c397b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f72051d4000 1 --2- 172.21.15.66:0/495674814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c39785a00 0x559c3978e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f71e8c00640 0 client.7458 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:54.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f72051d4000 1 -- 172.21.15.66:0/495674814 >> 172.21.15.66:0/495674814 conn(0x559c39586160 msgr2=0x559c39784bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:54.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f72051d4000 1 -- 172.21.15.66:0/495674814 shutdown_connections 2024-03-30T00:45:54.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f72051d4000 1 -- 172.21.15.66:0/495674814 wait complete. 2024-03-30T00:45:54.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.742+0000 7f72051d4000 20 client.7458 trim_cache size 0 max 16384 2024-03-30T00:45:54.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.754+0000 7f72051d4000 1 Processor -- start 2024-03-30T00:45:54.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.754+0000 7f72051d4000 1 -- start start 2024-03-30T00:45:54.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c3978d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c3978dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c39785de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c39293aa0 con 0x559c39785a00 2024-03-30T00:45:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c395d42a0 con 0x559c397d1a70 2024-03-30T00:45:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c395d5920 con 0x559c397d2390 2024-03-30T00:45:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71faa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c3978d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71faa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c3978d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38544/0 (socket says 172.21.15.66:38544) 2024-03-30T00:45:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71faa00640 1 -- 172.21.15.66:0/3824298690 learned_addr learned my addr 172.21.15.66:0/3824298690 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71f2000640 1 --2- 172.21.15.66:0/3824298690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c3978dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71fb400640 1 --2- 172.21.15.66:0/3824298690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c39785de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71faa00640 1 -- 172.21.15.66:0/3824298690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 msgr2=0x559c39785de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71faa00640 1 --2- 172.21.15.66:0/3824298690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c39785de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71faa00640 1 -- 172.21.15.66:0/3824298690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 msgr2=0x559c3978dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71faa00640 1 --2- 172.21.15.66:0/3824298690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c3978dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71faa00640 1 -- 172.21.15.66:0/3824298690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c39581df0 con 0x559c39785a00 2024-03-30T00:45:54.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71f2000640 1 --2- 172.21.15.66:0/3824298690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c3978dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:45:54.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71fb400640 1 --2- 172.21.15.66:0/3824298690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c39785de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:45:54.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71faa00640 1 --2- 172.21.15.66:0/3824298690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c3978d830 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f71ec004630 tx=0x7f71ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d651506f8fb6b4c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:54.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71f0c00640 1 -- 172.21.15.66:0/3824298690 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec02a070 con 0x559c39785a00 2024-03-30T00:45:54.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71f0c00640 1 -- 172.21.15.66:0/3824298690 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec02bc30 con 0x559c39785a00 2024-03-30T00:45:54.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f71f0c00640 1 -- 172.21.15.66:0/3824298690 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec02bda0 con 0x559c39785a00 2024-03-30T00:45:54.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- 172.21.15.66:0/3824298690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 msgr2=0x559c3978d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 --2- 172.21.15.66:0/3824298690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c3978d830 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f71ec004630 tx=0x7f71ec018000 comp rx=0 tx=0).stop 2024-03-30T00:45:54.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- 172.21.15.66:0/3824298690 shutdown_connections 2024-03-30T00:45:54.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 --2- 172.21.15.66:0/3824298690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c39785de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 --2- 172.21.15.66:0/3824298690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c3978dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 --2- 172.21.15.66:0/3824298690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c3978d830 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- 172.21.15.66:0/3824298690 >> 172.21.15.66:0/3824298690 conn(0x559c39586160 msgr2=0x559c397df890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- 172.21.15.66:0/3824298690 shutdown_connections 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- 172.21.15.66:0/3824298690 wait complete. 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- start start 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 Processor -- start 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39788600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c39788b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c39789080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c3962e920 con 0x559c39785a00 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c395daa90 con 0x559c397d1a70 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c395a6290 con 0x559c397d2390 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.758+0000 7f72051d4000 10 client.0 _ll_register_callbacks cb 0x7f71f4050d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:54.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71faa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39788600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71faa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39788600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38548/0 (socket says 172.21.15.66:38548) 2024-03-30T00:45:54.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71faa00640 1 -- 172.21.15.66:0/1584431346 learned_addr learned my addr 172.21.15.66:0/1584431346 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:54.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71fb400640 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c39789080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71f2000640 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c39788b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71faa00640 1 -- 172.21.15.66:0/1584431346 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 msgr2=0x559c39789080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71faa00640 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c39789080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71faa00640 1 -- 172.21.15.66:0/1584431346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 msgr2=0x559c39788b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71faa00640 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c39788b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71faa00640 1 -- 172.21.15.66:0/1584431346 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec004410 con 0x559c39785a00 2024-03-30T00:45:54.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71f2000640 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c39788b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:54.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71faa00640 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39788600 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f71ec011b20 tx=0x7f71ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63e1881748128a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71e8c00640 1 -- 172.21.15.66:0/1584431346 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec003820 con 0x559c39785a00 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71e8c00640 1 -- 172.21.15.66:0/1584431346 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f71ec003980 con 0x559c39785a00 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 -- 172.21.15.66:0/1584431346 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c39593970 con 0x559c39785a00 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71e8c00640 1 -- 172.21.15.66:0/1584431346 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71ec003b40 con 0x559c39785a00 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 10 client.7464 fetch_fsmap learned FSMap version 51 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 -- 172.21.15.66:0/1584431346 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x559c39575960 con 0x559c39785a00 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71e8c00640 1 -- 172.21.15.66:0/1584431346 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c39593970 con 0x559c39785a00 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71e8c00640 1 -- 172.21.15.66:0/1584431346 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 51) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f71ec033030 con 0x559c39785a00 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 10 client.7464 fetch_fsmap finished waiting for FSMap version 51 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 10 client.7464 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-30T00:45:54.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 20 client.7464 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x559c397895c0 0x559c397899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 4 client.7464 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 -- 172.21.15.66:0/1584431346 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559c39605a20 con 0x559c397895c0 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71f2000640 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x559c397895c0 0x559c397899a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71f2000640 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x559c397895c0 0x559c397899a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a549575a4ee904bf server_cookie=e3263eda3b3b7426 in_seq=0 out_seq=0 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71e8c00640 10 client.7464 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71e8c00640 1 -- 172.21.15.66:0/1584431346 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559c39605a20 con 0x559c397895c0 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71e8c00640 10 client.7464 handle_command_reply: tid=0 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 client.7464 shutdown 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 -- 172.21.15.66:0/1584431346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 msgr2=0x559c39788600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39788600 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f71ec011b20 tx=0x7f71ec014000 comp rx=0 tx=0).stop 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 -- 172.21.15.66:0/1584431346 shutdown_connections 2024-03-30T00:45:54.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x559c397895c0 0x559c397899a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c397d2390 0x559c39789080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c397d1a70 0x559c39788b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 --2- 172.21.15.66:0/1584431346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559c39785a00 0x559c39788600 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f71e8c00640 0 client.7464 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:54.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 -- 172.21.15.66:0/1584431346 >> 172.21.15.66:0/1584431346 conn(0x559c39586160 msgr2=0x559c3978d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:54.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 -- 172.21.15.66:0/1584431346 shutdown_connections 2024-03-30T00:45:54.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 1 -- 172.21.15.66:0/1584431346 wait complete. 2024-03-30T00:45:54.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f72051d4000 20 client.7464 trim_cache size 0 max 16384 2024-03-30T00:45:54.777 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:45:54.778 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T00:45:54.778 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-03-30T00:45:54.778 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:45:54.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 -- 172.21.15.66:0/2261547195 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f71d8066690 msgr2=0x7f71d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f71d8066690 0x7f71d8068b50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009eb0 tx=0x7f71ec009190 comp rx=0 tx=0).stop 2024-03-30T00:45:54.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 -- 172.21.15.66:0/2261547195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71fc159c60 msgr2=0x7f71fc0acec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71fc159c60 0x7f71fc0acec0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f71f400c460 tx=0x7f71f400c920 comp rx=0 tx=0).stop 2024-03-30T00:45:54.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 -- 172.21.15.66:0/2261547195 shutdown_connections 2024-03-30T00:45:54.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f71d8066690 0x7f71d8068b50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71fc159c60 0x7f71fc0acec0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71fc0cde70 0x7f71fc0ac980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 --2- 172.21.15.66:0/2261547195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71fc0cda90 0x7f71fc1511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 -- 172.21.15.66:0/2261547195 >> 172.21.15.66:0/2261547195 conn(0x7f71fc069f60 msgr2=0x7f71fc06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:54.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 -- 172.21.15.66:0/2261547195 shutdown_connections 2024-03-30T00:45:54.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.762+0000 7f7201e00640 1 -- 172.21.15.66:0/2261547195 wait complete. 2024-03-30T00:45:54.792 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:45:54.792 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:45:54.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.922+0000 7fdb4fa00640 1 Processor -- start 2024-03-30T00:45:54.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.922+0000 7fdb4fa00640 1 -- start start 2024-03-30T00:45:54.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4fa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480cec30 0x7fdb48159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4fa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb480cd3e0 con 0x7fdb480cda90 2024-03-30T00:45:54.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4fa00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4815a990 con 0x7fdb480cde70 2024-03-30T00:45:54.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4fa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480b1b80 con 0x7fdb480cec30 2024-03-30T00:45:54.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4d000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdb480cda90 0x7fdb480ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:49890/0 (socket says 172.21.15.66:49890) 2024-03-30T00:45:54.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4d000640 1 -- 172.21.15.66:0/4272394176 learned_addr learned my addr 172.21.15.66:0/4272394176 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:45:54.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275474572 0 0) 0x7fdb480cd3e0 con 0x7fdb480cda90 2024-03-30T00:45:54.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb2c003530 con 0x7fdb480cda90 2024-03-30T00:45:54.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb3fe00640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480cec30 0x7fdb48159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1561312609 0 0) 0x7fdb2c003530 con 0x7fdb480cda90 2024-03-30T00:45:54.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480cec30 msgr2=0x7fdb48159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480cec30 0x7fdb48159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 >> v1:172.21.15.156:6789/0 conn(0x7fdb480cde70 legacy=0x7fdb4815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb480b11e0 con 0x7fdb480cda90 2024-03-30T00:45:54.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fdb38001680 con 0x7fdb480cda90 2024-03-30T00:45:54.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 >> v1:172.21.15.66:6789/0 conn(0x7fdb480cda90 legacy=0x7fdb480ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0039b0 0x7fdb2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480cec30 0x7fdb2c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c004d10 0x7fdb2c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480b1b80 con 0x7fdb2c0039b0 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c00baf0 con 0x7fdb2c004d10 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c00dec0 con 0x7fdb480cec30 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4d000640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0039b0 0x7fdb2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb3fe00640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480cec30 0x7fdb2c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4da00640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c004d10 0x7fdb2c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4d000640 1 -- 172.21.15.66:0/4272394176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480cec30 msgr2=0x7fdb2c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4d000640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480cec30 0x7fdb2c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4d000640 1 -- 172.21.15.66:0/4272394176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c004d10 msgr2=0x7fdb2c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4d000640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c004d10 0x7fdb2c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.926+0000 7fdb4d000640 1 -- 172.21.15.66:0/4272394176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c003530 con 0x7fdb2c0039b0 2024-03-30T00:45:54.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb4d000640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0039b0 0x7fdb2c003dd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c240 tx=0x7fdb3801cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8416256383c814a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:54.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38005ae0 con 0x7fdb2c0039b0 2024-03-30T00:45:54.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb38005c40 con 0x7fdb2c0039b0 2024-03-30T00:45:54.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb47e00640 1 -- 172.21.15.66:0/4272394176 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380a5820 con 0x7fdb2c0039b0 2024-03-30T00:45:54.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb4fa00640 1 -- 172.21.15.66:0/4272394176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0039b0 msgr2=0x7fdb2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb4fa00640 1 --2- 172.21.15.66:0/4272394176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0039b0 0x7fdb2c003dd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c240 tx=0x7fdb3801cba0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb4da00640 1 -- 172.21.15.66:0/4272394176 reap_dead start 2024-03-30T00:45:54.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb4fa00640 1 -- 172.21.15.66:0/4272394176 shutdown_connections 2024-03-30T00:45:54.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb4fa00640 1 -- 172.21.15.66:0/4272394176 >> 172.21.15.66:0/4272394176 conn(0x7fdb48069f60 msgr2=0x7fdb4806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:54.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb4fa00640 1 -- 172.21.15.66:0/4272394176 shutdown_connections 2024-03-30T00:45:54.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb4fa00640 1 -- 172.21.15.66:0/4272394176 wait complete. 2024-03-30T00:45:54.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb4fa00640 1 Processor -- start 2024-03-30T00:45:54.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.930+0000 7fdb4fa00640 1 -- start start 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb4fa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0039b0 0x7fdb48158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb4fa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb480b0790 0x7fdb480ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb4fa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480974d0 0x7fdb48097950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb4fa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48158f90 con 0x7fdb480b0790 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb4fa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48152d20 con 0x7fdb2c0039b0 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb4fa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480b0c10 con 0x7fdb480974d0 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb3fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb480b0790 0x7fdb480ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb3fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb480b0790 0x7fdb480ab850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38558/0 (socket says 172.21.15.66:38558) 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb3fe00640 1 -- 172.21.15.66:0/3985345081 learned_addr learned my addr 172.21.15.66:0/3985345081 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb4d000640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0039b0 0x7fdb48158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb4da00640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480974d0 0x7fdb48097950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb3fe00640 1 -- 172.21.15.66:0/3985345081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480974d0 msgr2=0x7fdb48097950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb3fe00640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480974d0 0x7fdb48097950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb3fe00640 1 -- 172.21.15.66:0/3985345081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0039b0 msgr2=0x7fdb48158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb3fe00640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0039b0 0x7fdb48158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb3fe00640 1 -- 172.21.15.66:0/3985345081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb48097e90 con 0x7fdb480b0790 2024-03-30T00:45:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb3fe00640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb480b0790 0x7fdb480ab850 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fdb30011a00 tx=0x7fdb30011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb45600640 1 -- 172.21.15.66:0/3985345081 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000fb50 con 0x7fdb480b0790 2024-03-30T00:45:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb45600640 1 -- 172.21.15.66:0/3985345081 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3000fcb0 con 0x7fdb480b0790 2024-03-30T00:45:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb45600640 1 -- 172.21.15.66:0/3985345081 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30014440 con 0x7fdb480b0790 2024-03-30T00:45:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb4fa00640 1 -- 172.21.15.66:0/3985345081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb481586f0 con 0x7fdb480b0790 2024-03-30T00:45:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb4fa00640 1 -- 172.21.15.66:0/3985345081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb48158cb0 con 0x7fdb480b0790 2024-03-30T00:45:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb4fa00640 1 -- 172.21.15.66:0/3985345081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb48066c30 con 0x7fdb480b0790 2024-03-30T00:45:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.934+0000 7fdb45600640 1 -- 172.21.15.66:0/3985345081 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdb300145a0 con 0x7fdb480b0790 2024-03-30T00:45:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.938+0000 7fdb45600640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb24066690 0x7fdb24068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.938+0000 7fdb45600640 1 -- 172.21.15.66:0/3985345081 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fdb30084b70 con 0x7fdb480b0790 2024-03-30T00:45:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.938+0000 7fdb4d000640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb24066690 0x7fdb24068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.938+0000 7fdb4d000640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb24066690 0x7fdb24068b50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdb380ab000 tx=0x7fdb380a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:54.942+0000 7fdb45600640 1 -- 172.21.15.66:0/3985345081 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdb30051df0 con 0x7fdb480b0790 2024-03-30T00:45:55.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.138+0000 7fdb4fa00640 1 -- 172.21.15.66:0/3985345081 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb48152e60 con 0x7fdb480b0790 2024-03-30T00:45:55.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.138+0000 7fdb45600640 1 -- 172.21.15.66:0/3985345081 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4741 (secure 0 0 0) 0x7fdb30022020 con 0x7fdb480b0790 2024-03-30T00:45:55.147 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:45:55.148 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/254323195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":254323195},{"type":"v1","addr":"172.21.15.156:6835","nonce":254323195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:45:30.650828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":23,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:45:55.148 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 51 2024-03-30T00:45:55.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 -- 172.21.15.66:0/3985345081 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb24066690 msgr2=0x7fdb24068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb24066690 0x7fdb24068b50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdb380ab000 tx=0x7fdb380a7070 comp rx=0 tx=0).stop 2024-03-30T00:45:55.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 -- 172.21.15.66:0/3985345081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb480b0790 msgr2=0x7fdb480ab850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb480b0790 0x7fdb480ab850 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fdb30011a00 tx=0x7fdb30011ec0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 -- 172.21.15.66:0/3985345081 shutdown_connections 2024-03-30T00:45:55.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb480974d0 0x7fdb48097950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdb24066690 0x7fdb24068b50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb480b0790 0x7fdb480ab850 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 --2- 172.21.15.66:0/3985345081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0039b0 0x7fdb48158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 -- 172.21.15.66:0/3985345081 >> 172.21.15.66:0/3985345081 conn(0x7fdb48069f60 msgr2=0x7fdb4815c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:55.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 -- 172.21.15.66:0/3985345081 shutdown_connections 2024-03-30T00:45:55.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.142+0000 7fdb4fa00640 1 -- 172.21.15.66:0/3985345081 wait complete. 2024-03-30T00:45:55.168 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-30T00:45:55.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.302+0000 7fee2b000640 1 Processor -- start 2024-03-30T00:45:55.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.302+0000 7fee2b000640 1 -- start start 2024-03-30T00:45:55.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.302+0000 7fee2b000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fee240b0290 0x7fee240b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.302+0000 7fee2b000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fee2415c200 0x7fee2415c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.302+0000 7fee2b000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee240afdf0 con 0x7fee24160490 2024-03-30T00:45:55.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.302+0000 7fee2b000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee240b15d0 con 0x7fee2415c200 2024-03-30T00:45:55.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.302+0000 7fee2b000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee240b1010 con 0x7fee240b0290 2024-03-30T00:45:55.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.302+0000 7fee23e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fee24160490 0x7fee24160870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:49906/0 (socket says 172.21.15.66:49906) 2024-03-30T00:45:55.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.302+0000 7fee23e00640 1 -- 172.21.15.66:0/3129172669 learned_addr learned my addr 172.21.15.66:0/3129172669 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:45:55.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.302+0000 7fee29000640 1 --2- 172.21.15.66:0/3129172669 >> v2:172.21.15.66:3300/0 conn(0x7fee2415c200 0x7fee2415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee1be00640 1 --2- 172.21.15.66:0/3129172669 >> v2:172.21.15.156:3300/0 conn(0x7fee240b0290 0x7fee240b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee29000640 1 -- 172.21.15.66:0/3129172669 >> v2:172.21.15.156:3300/0 conn(0x7fee240b0290 msgr2=0x7fee240b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:55.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee29000640 1 --2- 172.21.15.66:0/3129172669 >> v2:172.21.15.156:3300/0 conn(0x7fee240b0290 0x7fee240b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee29000640 1 -- 172.21.15.66:0/3129172669 >> v1:172.21.15.66:6789/0 conn(0x7fee24160490 legacy=0x7fee24160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee23400640 1 -- 172.21.15.66:0/3129172669 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605264295 0 0) 0x7fee240afdf0 con 0x7fee24160490 2024-03-30T00:45:55.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee29000640 1 -- 172.21.15.66:0/3129172669 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee240b12f0 con 0x7fee2415c200 2024-03-30T00:45:55.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee29000640 1 --2- 172.21.15.66:0/3129172669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2415c200 0x7fee2415c630 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fee24071f40 tx=0x7fee1c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de56b182a1ff5e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee23400640 1 -- 172.21.15.66:0/3129172669 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0a59c0 con 0x7fee2415c200 2024-03-30T00:45:55.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee23400640 1 -- 172.21.15.66:0/3129172669 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c0a5b20 con 0x7fee2415c200 2024-03-30T00:45:55.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee23400640 1 -- 172.21.15.66:0/3129172669 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0ad440 con 0x7fee2415c200 2024-03-30T00:45:55.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee2b000640 1 -- 172.21.15.66:0/3129172669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2415c200 msgr2=0x7fee2415c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.306+0000 7fee2b000640 1 --2- 172.21.15.66:0/3129172669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2415c200 0x7fee2415c630 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fee24071f40 tx=0x7fee1c01c900 comp rx=0 tx=0).stop 2024-03-30T00:45:55.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 -- 172.21.15.66:0/3129172669 shutdown_connections 2024-03-30T00:45:55.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 --2- 172.21.15.66:0/3129172669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2415c200 0x7fee2415c630 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 --2- 172.21.15.66:0/3129172669 >> v2:172.21.15.156:3300/0 conn(0x7fee240b0290 0x7fee240b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 -- 172.21.15.66:0/3129172669 >> 172.21.15.66:0/3129172669 conn(0x7fee24099870 msgr2=0x7fee24089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:55.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 -- 172.21.15.66:0/3129172669 shutdown_connections 2024-03-30T00:45:55.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 -- 172.21.15.66:0/3129172669 wait complete. 2024-03-30T00:45:55.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 Processor -- start 2024-03-30T00:45:55.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 -- start start 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee240b0290 0x7fee241f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2415c200 0x7fee241f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee24160490 0x7fee241f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2415fdd0 con 0x7fee2415c200 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee241f5940 con 0x7fee24160490 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee2b000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2415faf0 con 0x7fee240b0290 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2415c200 0x7fee241f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2415c200 0x7fee241f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38568/0 (socket says 172.21.15.66:38568) 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee1be00640 1 -- 172.21.15.66:0/1543599411 learned_addr learned my addr 172.21.15.66:0/1543599411 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee23e00640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee240b0290 0x7fee241f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee29000640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee24160490 0x7fee241f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee1be00640 1 -- 172.21.15.66:0/1543599411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee240b0290 msgr2=0x7fee241f62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee1be00640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee240b0290 0x7fee241f62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee1be00640 1 -- 172.21.15.66:0/1543599411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee24160490 msgr2=0x7fee241f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee1be00640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee24160490 0x7fee241f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee1be00640 1 -- 172.21.15.66:0/1543599411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c009a70 con 0x7fee2415c200 2024-03-30T00:45:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.310+0000 7fee1be00640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2415c200 0x7fee241f4ec0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00e930 tx=0x7fee0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee20c00640 1 -- 172.21.15.66:0/1543599411 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c009690 con 0x7fee2415c200 2024-03-30T00:45:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee20c00640 1 -- 172.21.15.66:0/1543599411 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c00ca00 con 0x7fee2415c200 2024-03-30T00:45:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee20c00640 1 -- 172.21.15.66:0/1543599411 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00cd70 con 0x7fee2415c200 2024-03-30T00:45:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee2b000640 1 -- 172.21.15.66:0/1543599411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee241f5bc0 con 0x7fee2415c200 2024-03-30T00:45:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee2b000640 1 -- 172.21.15.66:0/1543599411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee240ceff0 con 0x7fee2415c200 2024-03-30T00:45:55.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee20c00640 1 -- 172.21.15.66:0/1543599411 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fee0c013470 con 0x7fee2415c200 2024-03-30T00:45:55.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee20c00640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee000666c0 0x7fee00068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee20c00640 1 -- 172.21.15.66:0/1543599411 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fee0c084530 con 0x7fee2415c200 2024-03-30T00:45:55.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee23e00640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee000666c0 0x7fee00068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee2e420000 1 Processor -- start 2024-03-30T00:45:55.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee2e420000 1 -- start start 2024-03-30T00:45:55.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.314+0000 7fee23e00640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee000666c0 0x7fee00068b80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fee14000f70 tx=0x7fee14007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51a290 0x55d4fe51a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe51abb0 0x55d4fe4d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe4d11b0 0x55d4fe4d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fe399340 con 0x55d4fe4d11b0 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fe325cd0 con 0x55d4fe51a290 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4fe3232a0 con 0x55d4fe51abb0 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee29000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe4d11b0 0x55d4fe4d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee29000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe4d11b0 0x55d4fe4d1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38582/0 (socket says 172.21.15.66:38582) 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee29000640 1 -- 172.21.15.66:0/3801201663 learned_addr learned my addr 172.21.15.66:0/3801201663 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee23e00640 1 --2- 172.21.15.66:0/3801201663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51a290 0x55d4fe51a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee1be00640 1 --2- 172.21.15.66:0/3801201663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe51abb0 0x55d4fe4d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee29000640 1 -- 172.21.15.66:0/3801201663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe51abb0 msgr2=0x55d4fe4d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee29000640 1 --2- 172.21.15.66:0/3801201663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe51abb0 0x55d4fe4d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee29000640 1 -- 172.21.15.66:0/3801201663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51a290 msgr2=0x55d4fe51a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee29000640 1 --2- 172.21.15.66:0/3801201663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51a290 0x55d4fe51a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee29000640 1 -- 172.21.15.66:0/3801201663 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4fe0114b0 con 0x55d4fe4d11b0 2024-03-30T00:45:55.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee29000640 1 --2- 172.21.15.66:0/3801201663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe4d11b0 0x55d4fe4d1590 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fee1c01a530 tx=0x7fee1c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791f4fb608d8f118 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee18c00640 1 -- 172.21.15.66:0/3801201663 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0b8650 con 0x55d4fe4d11b0 2024-03-30T00:45:55.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee18c00640 1 -- 172.21.15.66:0/3801201663 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c0b87b0 con 0x55d4fe4d11b0 2024-03-30T00:45:55.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee18c00640 1 -- 172.21.15.66:0/3801201663 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0b8b20 con 0x55d4fe4d11b0 2024-03-30T00:45:55.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 -- 172.21.15.66:0/3801201663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe4d11b0 msgr2=0x55d4fe4d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 --2- 172.21.15.66:0/3801201663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe4d11b0 0x55d4fe4d1590 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fee1c01a530 tx=0x7fee1c004ac0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 -- 172.21.15.66:0/3801201663 shutdown_connections 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 --2- 172.21.15.66:0/3801201663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe4d11b0 0x55d4fe4d1590 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 --2- 172.21.15.66:0/3801201663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe51abb0 0x55d4fe4d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 --2- 172.21.15.66:0/3801201663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51a290 0x55d4fe51a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 -- 172.21.15.66:0/3801201663 >> 172.21.15.66:0/3801201663 conn(0x55d4fe4cd160 msgr2=0x55d4fe4cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 -- 172.21.15.66:0/3801201663 shutdown_connections 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 -- 172.21.15.66:0/3801201663 wait complete. 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 -- start start 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.318+0000 7fee2e420000 1 Processor -- start 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe4d11b0 0x55d4fe4d6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe51a290 0x55d4fe4d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51abb0 0x55d4fe4dbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fe31eb90 con 0x55d4fe51abb0 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fe2dc0a0 con 0x55d4fe4d11b0 2024-03-30T00:45:55.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4fe2dbd40 con 0x55d4fe51a290 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 10 client.0 _ll_register_callbacks cb 0x55d4fe3d3c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee29000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51abb0 0x55d4fe4dbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee29000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51abb0 0x55d4fe4dbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38584/0 (socket says 172.21.15.66:38584) 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee29000640 1 -- 172.21.15.66:0/1692647624 learned_addr learned my addr 172.21.15.66:0/1692647624 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee1be00640 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe51a290 0x55d4fe4d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee23e00640 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe4d11b0 0x55d4fe4d6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee29000640 1 -- 172.21.15.66:0/1692647624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe51a290 msgr2=0x55d4fe4d6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee29000640 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe51a290 0x55d4fe4d6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee29000640 1 -- 172.21.15.66:0/1692647624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe4d11b0 msgr2=0x55d4fe4d6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee29000640 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe4d11b0 0x55d4fe4d6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee29000640 1 -- 172.21.15.66:0/1692647624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee240b12f0 con 0x55d4fe51abb0 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee23e00640 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe4d11b0 0x55d4fe4d6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee29000640 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51abb0 0x55d4fe4dbe40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fee1c01cde0 tx=0x7fee1c01b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f131bb0bab1abd5d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee11600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee11600640 1 -- 172.21.15.66:0/1692647624 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c004240 con 0x55d4fe51abb0 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee11600640 1 -- 172.21.15.66:0/1692647624 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fee1c0043a0 con 0x55d4fe51abb0 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 -- 172.21.15.66:0/1692647624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4fe2ee550 con 0x55d4fe51abb0 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee11600640 1 -- 172.21.15.66:0/1692647624 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee1c003e90 con 0x55d4fe51abb0 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 10 client.7485 fetch_fsmap learned FSMap version 51 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 -- 172.21.15.66:0/1692647624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55d4fe2e3ba0 con 0x55d4fe51abb0 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee11600640 1 -- 172.21.15.66:0/1692647624 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d4fe2ee550 con 0x55d4fe51abb0 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee11600640 1 -- 172.21.15.66:0/1692647624 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 51) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fee1c004560 con 0x55d4fe51abb0 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 10 client.7485 fetch_fsmap finished waiting for FSMap version 51 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 10 client.7485 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 20 client.7485 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55d4fe529320 0x55d4fe52b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 4 client.7485 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:45:55.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 -- 172.21.15.66:0/1692647624 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4fe31b970 con 0x55d4fe529320 2024-03-30T00:45:55.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee23e00640 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55d4fe529320 0x55d4fe52b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee23e00640 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55d4fe529320 0x55d4fe52b710 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4133fd6bf3b68d0d server_cookie=ab10980170977198 in_seq=0 out_seq=0 2024-03-30T00:45:55.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee11600640 10 client.7485 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:55.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee11600640 1 -- 172.21.15.66:0/1692647624 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d4fe31b970 con 0x55d4fe529320 2024-03-30T00:45:55.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee11600640 10 client.7485 handle_command_reply: tid=0 2024-03-30T00:45:55.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 client.7485 shutdown 2024-03-30T00:45:55.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 -- 172.21.15.66:0/1692647624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51abb0 msgr2=0x55d4fe4dbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.322+0000 7fee2e420000 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51abb0 0x55d4fe4dbe40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fee1c01cde0 tx=0x7fee1c01b140 comp rx=0 tx=0).stop 2024-03-30T00:45:55.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.326+0000 7fee2e420000 1 -- 172.21.15.66:0/1692647624 shutdown_connections 2024-03-30T00:45:55.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.326+0000 7fee2e420000 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55d4fe529320 0x55d4fe52b710 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.326+0000 7fee2e420000 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51abb0 0x55d4fe4dbe40 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.326+0000 7fee2e420000 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe51a290 0x55d4fe4d6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.326+0000 7fee2e420000 1 --2- 172.21.15.66:0/1692647624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe4d11b0 0x55d4fe4d6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.326+0000 7fee11600640 0 client.7485 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:55.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.326+0000 7fee2e420000 1 -- 172.21.15.66:0/1692647624 >> 172.21.15.66:0/1692647624 conn(0x55d4fe4cd160 msgr2=0x55d4fe4cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:55.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.326+0000 7fee2e420000 1 -- 172.21.15.66:0/1692647624 shutdown_connections 2024-03-30T00:45:55.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.326+0000 7fee2e420000 1 -- 172.21.15.66:0/1692647624 wait complete. 2024-03-30T00:45:55.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.326+0000 7fee2e420000 20 client.7485 trim_cache size 0 max 16384 2024-03-30T00:45:55.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 Processor -- start 2024-03-30T00:45:55.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 -- start start 2024-03-30T00:45:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe4d11b0 0x55d4fe5736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe573c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51abb0 0x55d4fe574170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fdfdbc40 con 0x55d4fe51a290 2024-03-30T00:45:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fe2e9ad0 con 0x55d4fe51abb0 2024-03-30T00:45:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4fe31eb90 con 0x55d4fe4d11b0 2024-03-30T00:45:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe573c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe573c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38586/0 (socket says 172.21.15.66:38586) 2024-03-30T00:45:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee1be00640 1 -- 172.21.15.66:0/1635406564 learned_addr learned my addr 172.21.15.66:0/1635406564 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee23e00640 1 --2- 172.21.15.66:0/1635406564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe4d11b0 0x55d4fe5736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee29000640 1 --2- 172.21.15.66:0/1635406564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51abb0 0x55d4fe574170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee1be00640 1 -- 172.21.15.66:0/1635406564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe4d11b0 msgr2=0x55d4fe5736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee1be00640 1 --2- 172.21.15.66:0/1635406564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe4d11b0 0x55d4fe5736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee1be00640 1 -- 172.21.15.66:0/1635406564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51abb0 msgr2=0x55d4fe574170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee1be00640 1 --2- 172.21.15.66:0/1635406564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51abb0 0x55d4fe574170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee1be00640 1 -- 172.21.15.66:0/1635406564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4fe2c9ef0 con 0x55d4fe51a290 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee29000640 1 --2- 172.21.15.66:0/1635406564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51abb0 0x55d4fe574170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee1be00640 1 --2- 172.21.15.66:0/1635406564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe573c30 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00e900 tx=0x7fee0c0550c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5403be02b351d2a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee19600640 1 -- 172.21.15.66:0/1635406564 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c05d860 con 0x55d4fe51a290 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee19600640 1 -- 172.21.15.66:0/1635406564 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c053810 con 0x55d4fe51a290 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee19600640 1 -- 172.21.15.66:0/1635406564 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c054960 con 0x55d4fe51a290 2024-03-30T00:45:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 -- 172.21.15.66:0/1635406564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 msgr2=0x55d4fe573c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 --2- 172.21.15.66:0/1635406564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe573c30 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00e900 tx=0x7fee0c0550c0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 -- 172.21.15.66:0/1635406564 shutdown_connections 2024-03-30T00:45:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 --2- 172.21.15.66:0/1635406564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51abb0 0x55d4fe574170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 --2- 172.21.15.66:0/1635406564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe573c30 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 --2- 172.21.15.66:0/1635406564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe4d11b0 0x55d4fe5736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 -- 172.21.15.66:0/1635406564 >> 172.21.15.66:0/1635406564 conn(0x55d4fe4cd160 msgr2=0x55d4fe4d6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 -- 172.21.15.66:0/1635406564 shutdown_connections 2024-03-30T00:45:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 -- 172.21.15.66:0/1635406564 wait complete. 2024-03-30T00:45:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 -- start start 2024-03-30T00:45:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.338+0000 7fee2e420000 1 Processor -- start 2024-03-30T00:45:55.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe4d11b0 0x55d4fe574110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe52be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51abb0 0x55d4fe52c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fe326f60 con 0x55d4fe51a290 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fe322b90 con 0x55d4fe51abb0 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4fe31c3a0 con 0x55d4fe4d11b0 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 10 client.0 _ll_register_callbacks cb 0x55d4fe3bc870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe52be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe52be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38588/0 (socket says 172.21.15.66:38588) 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee1be00640 1 -- 172.21.15.66:0/381093095 learned_addr learned my addr 172.21.15.66:0/381093095 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee23e00640 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe4d11b0 0x55d4fe574110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee29000640 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51abb0 0x55d4fe52c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee1be00640 1 -- 172.21.15.66:0/381093095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe4d11b0 msgr2=0x55d4fe574110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee1be00640 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe4d11b0 0x55d4fe574110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee1be00640 1 -- 172.21.15.66:0/381093095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51abb0 msgr2=0x55d4fe52c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee1be00640 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51abb0 0x55d4fe52c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee1be00640 1 -- 172.21.15.66:0/381093095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee240ceff0 con 0x55d4fe51a290 2024-03-30T00:45:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee23e00640 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe4d11b0 0x55d4fe574110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee1be00640 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe52be70 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00c520 tx=0x7fee0c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee68edfe9c354b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee11600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:45:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee11600640 1 -- 172.21.15.66:0/381093095 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c077070 con 0x55d4fe51a290 2024-03-30T00:45:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee11600640 1 -- 172.21.15.66:0/381093095 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fee0c05d860 con 0x55d4fe51a290 2024-03-30T00:45:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 -- 172.21.15.66:0/381093095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4fe2ce620 con 0x55d4fe51a290 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee11600640 1 -- 172.21.15.66:0/381093095 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee0c054960 con 0x55d4fe51a290 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 10 client.7491 fetch_fsmap learned FSMap version 51 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 -- 172.21.15.66:0/381093095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55d4fe01afc0 con 0x55d4fe51a290 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee11600640 1 -- 172.21.15.66:0/381093095 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d4fe2ce620 con 0x55d4fe51a290 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee11600640 1 -- 172.21.15.66:0/381093095 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 51) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fee0c011070 con 0x55d4fe51a290 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 10 client.7491 fetch_fsmap finished waiting for FSMap version 51 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 10 client.7491 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 20 client.7491 populate_metadata read hostname 'smithi066' 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55d4fe52c8f0 0x55d4fe52ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 4 client.7491 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 -- 172.21.15.66:0/381093095 --> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55d4fe31d4e0 con 0x55d4fe52c8f0 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee23e00640 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55d4fe52c8f0 0x55d4fe52ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee11600640 10 client.7491 ms_handle_connect on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee23e00640 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55d4fe52c8f0 0x55d4fe52ccd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de2724003ee33fe server_cookie=6464f96530d0756b in_seq=0 out_seq=0 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee11600640 1 -- 172.21.15.66:0/381093095 <== mds.0 v2:172.21.15.66:6835/2050987583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d4fe31d4e0 con 0x55d4fe52c8f0 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee11600640 10 client.7491 handle_command_reply: tid=0 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 client.7491 shutdown 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 -- 172.21.15.66:0/381093095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 msgr2=0x55d4fe52be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe52be70 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00c520 tx=0x7fee0c000bd0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 -- 172.21.15.66:0/381093095 shutdown_connections 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.66:6835/2050987583,v1:172.21.15.66:6837/2050987583] conn(0x55d4fe52c8f0 0x55d4fe52ccd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4fe51abb0 0x55d4fe52c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4fe51a290 0x55d4fe52be70 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 --2- 172.21.15.66:0/381093095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d4fe4d11b0 0x55d4fe574110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 -- 172.21.15.66:0/381093095 >> 172.21.15.66:0/381093095 conn(0x55d4fe4cd160 msgr2=0x55d4fe518dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee11600640 0 client.7491 ms_handle_reset on v2:172.21.15.66:6835/2050987583 2024-03-30T00:45:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 -- 172.21.15.66:0/381093095 shutdown_connections 2024-03-30T00:45:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 1 -- 172.21.15.66:0/381093095 wait complete. 2024-03-30T00:45:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.342+0000 7fee2e420000 20 client.7491 trim_cache size 0 max 16384 2024-03-30T00:45:55.359 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:45:55.359 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-03-30T00:45:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 -- 172.21.15.66:0/1543599411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee000666c0 msgr2=0x7fee00068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee000666c0 0x7fee00068b80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fee14000f70 tx=0x7fee14007c10 comp rx=0 tx=0).stop 2024-03-30T00:45:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 -- 172.21.15.66:0/1543599411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2415c200 msgr2=0x7fee241f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2415c200 0x7fee241f4ec0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00e930 tx=0x7fee0c00edf0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 -- 172.21.15.66:0/1543599411 shutdown_connections 2024-03-30T00:45:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee24160490 0x7fee241f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fee000666c0 0x7fee00068b80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee2415c200 0x7fee241f4ec0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 --2- 172.21.15.66:0/1543599411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fee240b0290 0x7fee241f62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 -- 172.21.15.66:0/1543599411 >> 172.21.15.66:0/1543599411 conn(0x7fee24099870 msgr2=0x7fee24167790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:55.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 -- 172.21.15.66:0/1543599411 shutdown_connections 2024-03-30T00:45:55.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.346+0000 7fee2b000640 1 -- 172.21.15.66:0/1543599411 wait complete. 2024-03-30T00:45:55.371 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-30T00:45:55.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.502+0000 7f5faf000640 1 Processor -- start 2024-03-30T00:45:55.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.502+0000 7f5faf000640 1 -- start start 2024-03-30T00:45:55.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5faf000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5fa80cee50 0x7f5fa80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5faf000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5fa80af420 0x7f5fa80d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5faf000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa80cbb90 con 0x7f5fa815de30 2024-03-30T00:45:55.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5faf000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa80ce1b0 con 0x7f5fa80af420 2024-03-30T00:45:55.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5faf000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80afa10 con 0x7f5fa80cee50 2024-03-30T00:45:55.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5fad000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5fa80af420 0x7f5fa80d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5fad000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5fa80af420 0x7f5fa80d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38598/0 (socket says 172.21.15.66:38598) 2024-03-30T00:45:55.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5fad000640 1 -- 172.21.15.66:0/2410778262 learned_addr learned my addr 172.21.15.66:0/2410778262 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:55.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5f9fe00640 1 --2- 172.21.15.66:0/2410778262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5fa80cee50 0x7f5fa80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5fad000640 1 -- 172.21.15.66:0/2410778262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5fa80cee50 msgr2=0x7f5fa80cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:55.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5fad000640 1 --2- 172.21.15.66:0/2410778262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5fa80cee50 0x7f5fa80cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5fad000640 1 -- 172.21.15.66:0/2410778262 >> v1:172.21.15.156:6789/0 conn(0x7f5fa815de30 legacy=0x7f5fa80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5fad000640 1 -- 172.21.15.66:0/2410778262 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa80cf430 con 0x7f5fa80af420 2024-03-30T00:45:55.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5fad000640 1 --2- 172.21.15.66:0/2410778262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa80af420 0x7f5fa80d1060 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009e50 tx=0x7f5fa001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7705010c76fd7571 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.506+0000 7f5fa7400640 1 -- 172.21.15.66:0/2410778262 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00a57c0 con 0x7f5fa80af420 2024-03-30T00:45:55.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5fa7400640 1 -- 172.21.15.66:0/2410778262 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa00a5920 con 0x7f5fa80af420 2024-03-30T00:45:55.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5fa7400640 1 -- 172.21.15.66:0/2410778262 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00a5cb0 con 0x7f5fa80af420 2024-03-30T00:45:55.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 -- 172.21.15.66:0/2410778262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa80af420 msgr2=0x7f5fa80d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 --2- 172.21.15.66:0/2410778262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa80af420 0x7f5fa80d1060 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009e50 tx=0x7f5fa001c9a0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 -- 172.21.15.66:0/2410778262 shutdown_connections 2024-03-30T00:45:55.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 --2- 172.21.15.66:0/2410778262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa80af420 0x7f5fa80d1060 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 --2- 172.21.15.66:0/2410778262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5fa80cee50 0x7f5fa80cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 -- 172.21.15.66:0/2410778262 >> 172.21.15.66:0/2410778262 conn(0x7f5fa80d0540 msgr2=0x7f5fa80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:55.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 -- 172.21.15.66:0/2410778262 shutdown_connections 2024-03-30T00:45:55.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 -- 172.21.15.66:0/2410778262 wait complete. 2024-03-30T00:45:55.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 Processor -- start 2024-03-30T00:45:55.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 -- start start 2024-03-30T00:45:55.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fa80af420 0x7f5fa8153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa80cee50 0x7f5fa80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5fa815de30 0x7f5fa80b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81538c0 con 0x7f5fa80cee50 2024-03-30T00:45:55.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80b20d0 con 0x7f5fa80af420 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5faf000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80b3cd0 con 0x7f5fa815de30 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5f9fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa80cee50 0x7f5fa80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5f9fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa80cee50 0x7f5fa80b4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38608/0 (socket says 172.21.15.66:38608) 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5f9fe00640 1 -- 172.21.15.66:0/140042567 learned_addr learned my addr 172.21.15.66:0/140042567 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5fa7e00640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fa80af420 0x7f5fa8153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.510+0000 7f5fad000640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5fa815de30 0x7f5fa80b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5f9fe00640 1 -- 172.21.15.66:0/140042567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5fa815de30 msgr2=0x7f5fa80b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5f9fe00640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5fa815de30 0x7f5fa80b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5f9fe00640 1 -- 172.21.15.66:0/140042567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fa80af420 msgr2=0x7f5fa8153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5f9fe00640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fa80af420 0x7f5fa8153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5f9fe00640 1 -- 172.21.15.66:0/140042567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa0009b30 con 0x7f5fa80cee50 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5fad000640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5fa815de30 0x7f5fa80b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:55.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5f9fe00640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa80cee50 0x7f5fa80b4d70 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000e930 tx=0x7f5f9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5fa4c00640 1 -- 172.21.15.66:0/140042567 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90009690 con 0x7f5fa80cee50 2024-03-30T00:45:55.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5fa4c00640 1 -- 172.21.15.66:0/140042567 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9000ca00 con 0x7f5fa80cee50 2024-03-30T00:45:55.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5fa4c00640 1 -- 172.21.15.66:0/140042567 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000cd70 con 0x7f5fa80cee50 2024-03-30T00:45:55.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5faf000640 1 -- 172.21.15.66:0/140042567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa80b3fb0 con 0x7f5fa80cee50 2024-03-30T00:45:55.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5faf000640 1 -- 172.21.15.66:0/140042567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa80b4570 con 0x7f5fa80cee50 2024-03-30T00:45:55.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5faf000640 1 -- 172.21.15.66:0/140042567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa80cdf60 con 0x7f5fa80cee50 2024-03-30T00:45:55.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5fa4c00640 1 -- 172.21.15.66:0/140042567 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5f90013470 con 0x7f5fa80cee50 2024-03-30T00:45:55.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5fa4c00640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f84066690 0x7f5f84068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5fa4c00640 1 -- 172.21.15.66:0/140042567 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5f900844f0 con 0x7f5fa80cee50 2024-03-30T00:45:55.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.514+0000 7f5fa7e00640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f84066690 0x7f5f84068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.518+0000 7f5fa7e00640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f84066690 0x7f5f84068b50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5f98007fd0 tx=0x7f5f98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.518+0000 7f5fa4c00640 1 -- 172.21.15.66:0/140042567 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5f90051820 con 0x7f5fa80cee50 2024-03-30T00:45:55.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.718+0000 7f5faf000640 1 -- 172.21.15.66:0/140042567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5fa8066c30 con 0x7f5fa80cee50 2024-03-30T00:45:55.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.718+0000 7f5fa4c00640 1 -- 172.21.15.66:0/140042567 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5f90054e40 con 0x7f5fa80cee50 2024-03-30T00:45:55.728 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:45:55.728 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-30T00:45:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 -- 172.21.15.66:0/140042567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f84066690 msgr2=0x7f5f84068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f84066690 0x7f5f84068b50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5f98007fd0 tx=0x7f5f98009040 comp rx=0 tx=0).stop 2024-03-30T00:45:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 -- 172.21.15.66:0/140042567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa80cee50 msgr2=0x7f5fa80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa80cee50 0x7f5fa80b4d70 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000e930 tx=0x7f5f9000edf0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 -- 172.21.15.66:0/140042567 shutdown_connections 2024-03-30T00:45:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5fa815de30 0x7f5fa80b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5f84066690 0x7f5f84068b50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa80cee50 0x7f5fa80b4d70 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 --2- 172.21.15.66:0/140042567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fa80af420 0x7f5fa8153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 -- 172.21.15.66:0/140042567 >> 172.21.15.66:0/140042567 conn(0x7f5fa80d0540 msgr2=0x7f5fa815a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 -- 172.21.15.66:0/140042567 shutdown_connections 2024-03-30T00:45:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.722+0000 7f5faf000640 1 -- 172.21.15.66:0/140042567 wait complete. 2024-03-30T00:45:55.750 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-30T00:45:55.750 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:45:55.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a4800640 1 Processor -- start 2024-03-30T00:45:55.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a4800640 1 -- start start 2024-03-30T00:45:55.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a4800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f009c15de30 0x7f009c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a4800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f009c0af420 0x7f009c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a4800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009c0cbb90 con 0x7f009c0ce740 2024-03-30T00:45:55.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a4800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009c0ce1b0 con 0x7f009c0af420 2024-03-30T00:45:55.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a4800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0afa10 con 0x7f009c15de30 2024-03-30T00:45:55.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a1400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f009c0af420 0x7f009c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a1400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f009c0af420 0x7f009c0d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38616/0 (socket says 172.21.15.66:38616) 2024-03-30T00:45:55.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a1400640 1 -- 172.21.15.66:0/2072162075 learned_addr learned my addr 172.21.15.66:0/2072162075 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:55.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a1e00640 1 --2- 172.21.15.66:0/2072162075 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f009c15de30 0x7f009c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a1400640 1 -- 172.21.15.66:0/2072162075 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f009c15de30 msgr2=0x7f009c0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:55.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a1400640 1 --2- 172.21.15.66:0/2072162075 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f009c15de30 0x7f009c0cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a1400640 1 -- 172.21.15.66:0/2072162075 >> v1:172.21.15.156:6789/0 conn(0x7f009c0ce740 legacy=0x7f009c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.882+0000 7f00a1400640 1 -- 172.21.15.66:0/2072162075 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c0cf430 con 0x7f009c0af420 2024-03-30T00:45:55.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a1400640 1 --2- 172.21.15.66:0/2072162075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009c0af420 0x7f009c0d1060 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f008c009870 tx=0x7f008c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124e87a37a1e417 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a0a00640 1 -- 172.21.15.66:0/2072162075 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c0a7070 con 0x7f009c0af420 2024-03-30T00:45:55.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a0a00640 1 -- 172.21.15.66:0/2072162075 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008c0ab430 con 0x7f009c0af420 2024-03-30T00:45:55.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a0a00640 1 -- 172.21.15.66:0/2072162075 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c0ab7c0 con 0x7f009c0af420 2024-03-30T00:45:55.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a4800640 1 -- 172.21.15.66:0/2072162075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009c0af420 msgr2=0x7f009c0d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a4800640 1 --2- 172.21.15.66:0/2072162075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009c0af420 0x7f009c0d1060 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f008c009870 tx=0x7f008c01c4d0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a4800640 1 -- 172.21.15.66:0/2072162075 shutdown_connections 2024-03-30T00:45:55.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a4800640 1 --2- 172.21.15.66:0/2072162075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009c0af420 0x7f009c0d1060 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a4800640 1 --2- 172.21.15.66:0/2072162075 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f009c15de30 0x7f009c0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a4800640 1 -- 172.21.15.66:0/2072162075 >> 172.21.15.66:0/2072162075 conn(0x7f009c0d0540 msgr2=0x7f009c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:55.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a4800640 1 -- 172.21.15.66:0/2072162075 shutdown_connections 2024-03-30T00:45:55.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.886+0000 7f00a4800640 1 -- 172.21.15.66:0/2072162075 wait complete. 2024-03-30T00:45:55.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a4800640 1 Processor -- start 2024-03-30T00:45:55.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a4800640 1 -- start start 2024-03-30T00:45:55.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a4800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f009c0af420 0x7f009c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a4800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f009c0ce740 0x7f009c153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a4800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009c15de30 0x7f009c159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a4800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c1594e0 con 0x7f009c15de30 2024-03-30T00:45:55.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a4800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0b3e40 con 0x7f009c0ce740 2024-03-30T00:45:55.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a4800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c1585c0 con 0x7f009c0af420 2024-03-30T00:45:55.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009c15de30 0x7f009c159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009c15de30 0x7f009c159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38632/0 (socket says 172.21.15.66:38632) 2024-03-30T00:45:55.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a2800640 1 -- 172.21.15.66:0/3089472079 learned_addr learned my addr 172.21.15.66:0/3089472079 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:55.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a1400640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f009c0ce740 0x7f009c153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a1e00640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f009c0af420 0x7f009c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a2800640 1 -- 172.21.15.66:0/3089472079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f009c0af420 msgr2=0x7f009c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a2800640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f009c0af420 0x7f009c0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a2800640 1 -- 172.21.15.66:0/3089472079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f009c0ce740 msgr2=0x7f009c153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:55.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a2800640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f009c0ce740 0x7f009c153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:55.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a2800640 1 -- 172.21.15.66:0/3089472079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c0094d0 con 0x7f009c15de30 2024-03-30T00:45:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a2800640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009c15de30 0x7f009c159810 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f009800c770 tx=0x7f009800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f0096000640 1 -- 172.21.15.66:0/3089472079 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098003b80 con 0x7f009c15de30 2024-03-30T00:45:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f0096000640 1 -- 172.21.15.66:0/3089472079 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0098003ce0 con 0x7f009c15de30 2024-03-30T00:45:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f0096000640 1 -- 172.21.15.66:0/3089472079 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800aa90 con 0x7f009c15de30 2024-03-30T00:45:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a4800640 1 -- 172.21.15.66:0/3089472079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009c15a650 con 0x7f009c15de30 2024-03-30T00:45:55.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.890+0000 7f00a4800640 1 -- 172.21.15.66:0/3089472079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009c158b40 con 0x7f009c15de30 2024-03-30T00:45:55.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.894+0000 7f00a4800640 1 -- 172.21.15.66:0/3089472079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009c0cdf60 con 0x7f009c15de30 2024-03-30T00:45:55.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.894+0000 7f0096000640 1 -- 172.21.15.66:0/3089472079 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f009800abf0 con 0x7f009c15de30 2024-03-30T00:45:55.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.894+0000 7f0096000640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0080066690 0x7f0080068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:55.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.894+0000 7f0096000640 1 -- 172.21.15.66:0/3089472079 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f00980840d0 con 0x7f009c15de30 2024-03-30T00:45:55.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.894+0000 7f00a1e00640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0080066690 0x7f0080068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:55.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.894+0000 7f00a1e00640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0080066690 0x7f0080068b50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f00900079c0 tx=0x7f0090008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:55.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:55.898+0000 7f0096000640 1 -- 172.21.15.66:0/3089472079 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f00980512b0 con 0x7f009c15de30 2024-03-30T00:45:56.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.094+0000 7f00a4800640 1 -- 172.21.15.66:0/3089472079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f009c066c30 con 0x7f009c15de30 2024-03-30T00:45:56.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.094+0000 7f0096000640 1 -- 172.21.15.66:0/3089472079 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) v1 ==== 53+0+73 (secure 0 0 0) 0x7f00980548d0 con 0x7f009c15de30 2024-03-30T00:45:56.103 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:45:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 -- 172.21.15.66:0/3089472079 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0080066690 msgr2=0x7f0080068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0080066690 0x7f0080068b50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f00900079c0 tx=0x7f0090008040 comp rx=0 tx=0).stop 2024-03-30T00:45:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 -- 172.21.15.66:0/3089472079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009c15de30 msgr2=0x7f009c159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009c15de30 0x7f009c159810 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f009800c770 tx=0x7f009800cc30 comp rx=0 tx=0).stop 2024-03-30T00:45:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 -- 172.21.15.66:0/3089472079 shutdown_connections 2024-03-30T00:45:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0080066690 0x7f0080068b50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009c15de30 0x7f009c159810 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f009c0ce740 0x7f009c153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 --2- 172.21.15.66:0/3089472079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f009c0af420 0x7f009c0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 -- 172.21.15.66:0/3089472079 >> 172.21.15.66:0/3089472079 conn(0x7f009c0d0540 msgr2=0x7f009c0ad6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 -- 172.21.15.66:0/3089472079 shutdown_connections 2024-03-30T00:45:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:56.098+0000 7f00a4800640 1 -- 172.21.15.66:0/3089472079 wait complete. 2024-03-30T00:45:56.125 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:45:56.125 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:45:56.125 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:45:56.125 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:45:56.125 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:45:56.125 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:45:56.125 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:45:56.125 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:45:56.125 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:45:56.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:56.126 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:45:56.135 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:45:56.135 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:45:56.136 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42122sec preferred_lft 42122sec 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:45:56.137 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:45:56.138 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:45:56.138 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:45:56.138 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:45:56.138 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:45:56.138 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:45:56.138 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:45:56.138 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:45:56.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:56.138 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:45:56.194 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:45:56.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:56.196 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:45:56.247 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:45:56.248 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:45:56.248 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:45:56.248 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:45:56.248 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:45:56.248 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:45:56.248 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:45:56.248 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:45:56.248 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:45:56.248 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:45:56.248 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:45:56.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:56.249 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:45:56.249 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:45:56.249 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:45:56.249 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:45:56.249 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:45:56.249 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:45:56.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:56.337 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:45:56.346 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:45:56.346 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:45:56.347 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:45:56.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:56.347 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:45:56.407 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:45:56.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:45:56.408 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:45:56.456 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:45:56.456 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:45:56.456 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:45:56.511 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:45:56.572 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:45:56.572 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:45:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:45:56.639 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:45:56.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:56.640 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:45:56.688 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:45:56.688 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:45:56.735 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:45:56.756 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:45:56.756 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:45:56.814 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:45:56.816 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:45:56.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:56.817 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:45:56.832 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:45:56.822+0000 7f213c84b580 -1 init, newargv = 0x559e18861180 newargc=15 2024-03-30T00:45:56.832 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[46066]: starting ceph client 2024-03-30T00:45:56.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[46066]: starting fuse 2024-03-30T00:45:56.841 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:45:56.841 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:45:56.841 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:45:56.841 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:45:56.841 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:45:56.842 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:45:56.843 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:45:56.843 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:45:56.843 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:45:56.843 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:45:56.843 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:45:56.843 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:45:56.843 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:45:56.843 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:45:56.843 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:45:56.843 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:45:56.965 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.46066.asok 2024-03-30T00:45:57.149 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46066.asok 2024-03-30T00:45:57.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:57.150 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46066.asok status 2024-03-30T00:45:57.327 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:45:57.327 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:45:57.327 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "46066", 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 7506, 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:45:57.328 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 7506 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2406992301 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2406992301 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.7506 192.168.0.1:0/2406992301", 2024-03-30T00:45:57.329 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/2406992301", 2024-03-30T00:45:57.330 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:45:57.330 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 51, 2024-03-30T00:45:57.330 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 83, 2024-03-30T00:45:57.330 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:45:57.330 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:45:57.330 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:45:57.330 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:45:57.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:57.343 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:45:57.343 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:45:57.352 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:45:57.352 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:45:57.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:57.353 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:45:57.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:57.415 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-03-30T00:45:57.467 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:45:57.467 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-03-30T00:45:57.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd9323080 1 Processor -- start 2024-03-30T00:45:57.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd9323080 1 -- start start 2024-03-30T00:45:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd9323080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592df9d4910 0x5592dfa60fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd9323080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4cf0 0x5592dfa5d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd9323080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5592dfa44290 0x5592dfa5d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd9323080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df8d0960 con 0x5592df9d4cf0 2024-03-30T00:45:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd9323080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df8d0600 con 0x5592df9d4910 2024-03-30T00:45:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd9323080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592df8c7690 con 0x5592dfa44290 2024-03-30T00:45:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd5c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4cf0 0x5592dfa5d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd5c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4cf0 0x5592dfa5d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38650/0 (socket says 172.21.15.66:38650) 2024-03-30T00:45:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd5c00640 1 -- 172.21.15.66:0/292453049 learned_addr learned my addr 172.21.15.66:0/292453049 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:57.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd7000640 1 --2- 172.21.15.66:0/292453049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5592dfa44290 0x5592dfa5d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.546+0000 7f6bd6600640 1 --2- 172.21.15.66:0/292453049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592df9d4910 0x5592dfa60fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd5c00640 1 -- 172.21.15.66:0/292453049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5592dfa44290 msgr2=0x5592dfa5d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd5c00640 1 --2- 172.21.15.66:0/292453049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5592dfa44290 0x5592dfa5d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd5c00640 1 -- 172.21.15.66:0/292453049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592df9d4910 msgr2=0x5592dfa60fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd5c00640 1 --2- 172.21.15.66:0/292453049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592df9d4910 0x5592dfa60fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd5c00640 1 -- 172.21.15.66:0/292453049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd000a1c0 con 0x5592df9d4cf0 2024-03-30T00:45:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd5c00640 1 --2- 172.21.15.66:0/292453049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4cf0 0x5592dfa5d020 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400a6f0 tx=0x7f6bc400abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265bb3a0036d76e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bcfe00640 1 -- 172.21.15.66:0/292453049 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400f070 con 0x5592df9d4cf0 2024-03-30T00:45:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bcfe00640 1 -- 172.21.15.66:0/292453049 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc400ae00 con 0x5592df9d4cf0 2024-03-30T00:45:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bcfe00640 1 -- 172.21.15.66:0/292453049 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4008c10 con 0x5592df9d4cf0 2024-03-30T00:45:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 -- 172.21.15.66:0/292453049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4cf0 msgr2=0x5592dfa5d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 --2- 172.21.15.66:0/292453049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4cf0 0x5592dfa5d020 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400a6f0 tx=0x7f6bc400abb0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 -- 172.21.15.66:0/292453049 shutdown_connections 2024-03-30T00:45:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 --2- 172.21.15.66:0/292453049 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5592dfa44290 0x5592dfa5d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 --2- 172.21.15.66:0/292453049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4cf0 0x5592dfa5d020 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 --2- 172.21.15.66:0/292453049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592df9d4910 0x5592dfa60fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 -- 172.21.15.66:0/292453049 >> 172.21.15.66:0/292453049 conn(0x5592df9b92f0 msgr2=0x5592df9b7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 -- 172.21.15.66:0/292453049 shutdown_connections 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 -- 172.21.15.66:0/292453049 wait complete. 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 Processor -- start 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 -- start start 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4910 0x5592dfa5cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592df9d4cf0 0x5592df9ba720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5592dfa44290 0x5592df9bac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dfafbfd0 con 0x5592df9d4910 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dfafb7f0 con 0x5592df9d4cf0 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd9323080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592dfafba90 con 0x5592dfa44290 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd6600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4910 0x5592dfa5cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd6600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4910 0x5592dfa5cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38664/0 (socket says 172.21.15.66:38664) 2024-03-30T00:45:57.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd6600640 1 -- 172.21.15.66:0/1700575848 learned_addr learned my addr 172.21.15.66:0/1700575848 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd5c00640 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592df9d4cf0 0x5592df9ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd7000640 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5592dfa44290 0x5592df9bac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd6600640 1 -- 172.21.15.66:0/1700575848 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5592dfa44290 msgr2=0x5592df9bac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd6600640 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5592dfa44290 0x5592df9bac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd6600640 1 -- 172.21.15.66:0/1700575848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592df9d4cf0 msgr2=0x5592df9ba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd6600640 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592df9d4cf0 0x5592df9ba720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd6600640 1 -- 172.21.15.66:0/1700575848 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc400a430 con 0x5592df9d4910 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bd6600640 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4910 0x5592dfa5cbd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800dee0 tx=0x7f6bc800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.550+0000 7f6bcea00640 1 -- 172.21.15.66:0/1700575848 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80032d0 con 0x5592df9d4910 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bcea00640 1 -- 172.21.15.66:0/1700575848 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc8001460 con 0x5592df9d4910 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bcea00640 1 -- 172.21.15.66:0/1700575848 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8001810 con 0x5592df9d4910 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bd9323080 1 -- 172.21.15.66:0/1700575848 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5592df8df8c0 con 0x5592df9d4910 2024-03-30T00:45:57.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bd9323080 1 -- 172.21.15.66:0/1700575848 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592df8d2d00 con 0x5592df9d4910 2024-03-30T00:45:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bcea00640 1 -- 172.21.15.66:0/1700575848 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6bc8001970 con 0x5592df9d4910 2024-03-30T00:45:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bcea00640 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bb80650a0 0x7f6bb8067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bcea00640 1 -- 172.21.15.66:0/1700575848 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f6bc8082ff0 con 0x5592df9d4910 2024-03-30T00:45:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bd5c00640 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bb80650a0 0x7f6bb8067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bd9323080 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5592dfa66660 0x5592dfa66a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bd9323080 1 -- 172.21.15.66:0/1700575848 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v8 -- 0x5592dfa66fb0 con 0x5592dfa66660 2024-03-30T00:45:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bd7000640 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5592dfa66660 0x5592dfa66a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.554+0000 7f6bd5c00640 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bb80650a0 0x7f6bb8067560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400afd0 tx=0x7f6bc4012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd7000640 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5592dfa66660 0x5592dfa66a70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd7000640 1 -- 172.21.15.66:0/1700575848 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v83'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6bd00ab900 con 0x5592dfa66660 2024-03-30T00:45:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd9323080 1 -- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5592dfa66660 msgr2=0x5592dfa66a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd9323080 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5592dfa66660 0x5592dfa66a70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd9323080 1 -- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bb80650a0 msgr2=0x7f6bb8067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd9323080 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bb80650a0 0x7f6bb8067560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400afd0 tx=0x7f6bc4012000 comp rx=0 tx=0).stop 2024-03-30T00:45:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd9323080 1 -- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4910 msgr2=0x5592dfa5cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd9323080 1 --2- 172.21.15.66:0/1700575848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592df9d4910 0x5592dfa5cbd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800dee0 tx=0x7f6bc800b5c0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd7000640 1 -- 172.21.15.66:0/1700575848 reap_dead start 2024-03-30T00:45:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd9323080 1 -- 172.21.15.66:0/1700575848 shutdown_connections 2024-03-30T00:45:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd9323080 1 -- 172.21.15.66:0/1700575848 >> 172.21.15.66:0/1700575848 conn(0x5592df9b92f0 msgr2=0x5592df9b7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd9323080 1 -- 172.21.15.66:0/1700575848 shutdown_connections 2024-03-30T00:45:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.558+0000 7f6bd9323080 1 -- 172.21.15.66:0/1700575848 wait complete. 2024-03-30T00:45:57.571 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:45:57.571 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:45:57.691 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:45:57.773 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:45:57.773 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:45:57.774 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:45:57.783 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:45:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.954+0000 7fe64a600640 1 Processor -- start 2024-03-30T00:45:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.954+0000 7fe64a600640 1 -- start start 2024-03-30T00:45:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe64a600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6440cec30 0x7fe644159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe64a600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe6440cde70 0x7fe64415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe64a600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64415a6e0 con 0x7fe6440cda90 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe64a600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6440b1b80 con 0x7fe6440cec30 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe64a600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6440cd3e0 con 0x7fe6440cde70 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe63aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6440cec30 0x7fe644159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe63aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6440cec30 0x7fe644159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38670/0 (socket says 172.21.15.66:38670) 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe63aa00640 1 -- 172.21.15.66:0/44441711 learned_addr learned my addr 172.21.15.66:0/44441711 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe643e00640 1 --2- 172.21.15.66:0/44441711 >> v2:172.21.15.156:3300/0 conn(0x7fe6440cde70 0x7fe64415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe63aa00640 1 -- 172.21.15.66:0/44441711 >> v2:172.21.15.156:3300/0 conn(0x7fe6440cde70 msgr2=0x7fe64415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe63aa00640 1 --2- 172.21.15.66:0/44441711 >> v2:172.21.15.156:3300/0 conn(0x7fe6440cde70 0x7fe64415a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe63aa00640 1 -- 172.21.15.66:0/44441711 >> v1:172.21.15.156:6789/0 conn(0x7fe6440cda90 legacy=0x7fe6440ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.958+0000 7fe63aa00640 1 -- 172.21.15.66:0/44441711 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64415a990 con 0x7fe6440cec30 2024-03-30T00:45:57.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe63aa00640 1 --2- 172.21.15.66:0/44441711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6440cec30 0x7fe644159c60 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe6440adcf0 tx=0x7fe62c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edc5fd130bd4ef6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:57.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe642a00640 1 -- 172.21.15.66:0/44441711 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0a57e0 con 0x7fe6440cec30 2024-03-30T00:45:57.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe642a00640 1 -- 172.21.15.66:0/44441711 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c0a5940 con 0x7fe6440cec30 2024-03-30T00:45:57.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe642a00640 1 -- 172.21.15.66:0/44441711 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0a5cd0 con 0x7fe6440cec30 2024-03-30T00:45:57.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe64a600640 1 -- 172.21.15.66:0/44441711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6440cec30 msgr2=0x7fe644159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:57.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe64a600640 1 --2- 172.21.15.66:0/44441711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6440cec30 0x7fe644159c60 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe6440adcf0 tx=0x7fe62c01c920 comp rx=0 tx=0).stop 2024-03-30T00:45:57.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe64a600640 1 -- 172.21.15.66:0/44441711 shutdown_connections 2024-03-30T00:45:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe64a600640 1 --2- 172.21.15.66:0/44441711 >> v2:172.21.15.156:3300/0 conn(0x7fe6440cde70 0x7fe64415a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe64a600640 1 --2- 172.21.15.66:0/44441711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6440cec30 0x7fe644159c60 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe64a600640 1 -- 172.21.15.66:0/44441711 >> 172.21.15.66:0/44441711 conn(0x7fe644069f60 msgr2=0x7fe64406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe64a600640 1 -- 172.21.15.66:0/44441711 shutdown_connections 2024-03-30T00:45:57.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe64a600640 1 -- 172.21.15.66:0/44441711 wait complete. 2024-03-30T00:45:57.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe64a600640 1 Processor -- start 2024-03-30T00:45:57.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.962+0000 7fe64a600640 1 -- start start 2024-03-30T00:45:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe64a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6440cda90 0x7fe6440c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe64a600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6440cde70 0x7fe6440c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe64a600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6440cec30 0x7fe644163670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe64a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe644158590 con 0x7fe6440cda90 2024-03-30T00:45:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe64a600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64406de90 con 0x7fe6440cde70 2024-03-30T00:45:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe64a600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64406d8d0 con 0x7fe6440cec30 2024-03-30T00:45:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe643400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6440cda90 0x7fe6440c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe643400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6440cda90 0x7fe6440c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38672/0 (socket says 172.21.15.66:38672) 2024-03-30T00:45:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe643400640 1 -- 172.21.15.66:0/3443378861 learned_addr learned my addr 172.21.15.66:0/3443378861 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe63aa00640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6440cde70 0x7fe6440c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe643e00640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6440cec30 0x7fe644163670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe643400640 1 -- 172.21.15.66:0/3443378861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6440cec30 msgr2=0x7fe644163670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe643400640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6440cec30 0x7fe644163670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe643400640 1 -- 172.21.15.66:0/3443378861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6440cde70 msgr2=0x7fe6440c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:57.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe643400640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6440cde70 0x7fe6440c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe643400640 1 -- 172.21.15.66:0/3443378861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c009b30 con 0x7fe6440cda90 2024-03-30T00:45:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe643e00640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6440cec30 0x7fe644163670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe643400640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6440cda90 0x7fe6440c53c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe63400c450 tx=0x7fe63400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe63be00640 1 -- 172.21.15.66:0/3443378861 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634012830 con 0x7fe6440cda90 2024-03-30T00:45:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe63be00640 1 -- 172.21.15.66:0/3443378861 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6340129b0 con 0x7fe6440cda90 2024-03-30T00:45:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe64a600640 1 -- 172.21.15.66:0/3443378861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64406dbb0 con 0x7fe6440cda90 2024-03-30T00:45:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe63be00640 1 -- 172.21.15.66:0/3443378861 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634012d60 con 0x7fe6440cda90 2024-03-30T00:45:57.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe64a600640 1 -- 172.21.15.66:0/3443378861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64415b8e0 con 0x7fe6440cda90 2024-03-30T00:45:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe64a600640 1 -- 172.21.15.66:0/3443378861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe644158080 con 0x7fe6440cda90 2024-03-30T00:45:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.966+0000 7fe63be00640 1 -- 172.21.15.66:0/3443378861 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe63401c460 con 0x7fe6440cda90 2024-03-30T00:45:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.970+0000 7fe63be00640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6200666c0 0x7fe620068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:57.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.970+0000 7fe63be00640 1 -- 172.21.15.66:0/3443378861 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fe634084160 con 0x7fe6440cda90 2024-03-30T00:45:57.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.970+0000 7fe63aa00640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6200666c0 0x7fe620068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:57.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.970+0000 7fe63aa00640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6200666c0 0x7fe620068b80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0ae000 tx=0x7fe62c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:57.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:57.974+0000 7fe63be00640 1 -- 172.21.15.66:0/3443378861 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe644158080 con 0x7fe6440cda90 2024-03-30T00:45:58.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.134+0000 7fe64a600640 1 -- 172.21.15.66:0/3443378861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6441583f0 con 0x7fe6440cda90 2024-03-30T00:45:58.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.134+0000 7fe63be00640 1 -- 172.21.15.66:0/3443378861 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16440 (secure 0 0 0) 0x7fe6441583f0 con 0x7fe6440cda90 2024-03-30T00:45:58.144 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:45:58.144 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":83,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:35.079642+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:44:14.685880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-30T00:44:15.260373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.156:6835/2089872680":"2024-03-31T00:44:35.079618+0000","172.21.15.156:6833/2089872680":"2024-03-31T00:44:35.079618+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:45:58.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 -- 172.21.15.66:0/3443378861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6200666c0 msgr2=0x7fe620068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6200666c0 0x7fe620068b80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0ae000 tx=0x7fe62c0a7070 comp rx=0 tx=0).stop 2024-03-30T00:45:58.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 -- 172.21.15.66:0/3443378861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6440cda90 msgr2=0x7fe6440c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6440cda90 0x7fe6440c53c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe63400c450 tx=0x7fe63400c910 comp rx=0 tx=0).stop 2024-03-30T00:45:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 -- 172.21.15.66:0/3443378861 shutdown_connections 2024-03-30T00:45:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6440cec30 0x7fe644163670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6440cde70 0x7fe6440c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6200666c0 0x7fe620068b80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 --2- 172.21.15.66:0/3443378861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6440cda90 0x7fe6440c53c0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 -- 172.21.15.66:0/3443378861 >> 172.21.15.66:0/3443378861 conn(0x7fe644069f60 msgr2=0x7fe644164c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 -- 172.21.15.66:0/3443378861 shutdown_connections 2024-03-30T00:45:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.138+0000 7fe64a600640 1 -- 172.21.15.66:0/3443378861 wait complete. 2024-03-30T00:45:58.166 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:44:14.685880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:44:15.260373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:45:58.167 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:45:58.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb277000640 1 Processor -- start 2024-03-30T00:45:58.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb277000640 1 -- start start 2024-03-30T00:45:58.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb277000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb27015de30 0x7fb2700cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb277000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2700af420 0x7fb2700d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb277000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2700cbb90 con 0x7fb2700ce740 2024-03-30T00:45:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb277000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2700ce1b0 con 0x7fb2700af420 2024-03-30T00:45:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb277000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700afa10 con 0x7fb27015de30 2024-03-30T00:45:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb267400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2700af420 0x7fb2700d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb267400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2700af420 0x7fb2700d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38686/0 (socket says 172.21.15.66:38686) 2024-03-30T00:45:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb267400640 1 -- 172.21.15.66:0/3046828148 learned_addr learned my addr 172.21.15.66:0/3046828148 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb267400640 1 -- 172.21.15.66:0/3046828148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb27015de30 msgr2=0x7fb2700cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb267400640 1 --2- 172.21.15.66:0/3046828148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb27015de30 0x7fb2700cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb267400640 1 -- 172.21.15.66:0/3046828148 >> v1:172.21.15.156:6789/0 conn(0x7fb2700ce740 legacy=0x7fb27006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:58.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.302+0000 7fb267400640 1 -- 172.21.15.66:0/3046828148 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2700cf430 con 0x7fb2700af420 2024-03-30T00:45:58.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb267400640 1 --2- 172.21.15.66:0/3046828148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2700af420 0x7fb2700d1060 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb258009870 tx=0x7fb25801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49011200a6bad5d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:58.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb26f400640 1 -- 172.21.15.66:0/3046828148 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2580ab5f0 con 0x7fb2700af420 2024-03-30T00:45:58.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb26f400640 1 -- 172.21.15.66:0/3046828148 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2580ab750 con 0x7fb2700af420 2024-03-30T00:45:58.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb26f400640 1 -- 172.21.15.66:0/3046828148 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2580abae0 con 0x7fb2700af420 2024-03-30T00:45:58.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb277000640 1 -- 172.21.15.66:0/3046828148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2700af420 msgr2=0x7fb2700d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb277000640 1 --2- 172.21.15.66:0/3046828148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2700af420 0x7fb2700d1060 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb258009870 tx=0x7fb25801c490 comp rx=0 tx=0).stop 2024-03-30T00:45:58.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb277000640 1 -- 172.21.15.66:0/3046828148 shutdown_connections 2024-03-30T00:45:58.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb277000640 1 --2- 172.21.15.66:0/3046828148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2700af420 0x7fb2700d1060 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb277000640 1 --2- 172.21.15.66:0/3046828148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb27015de30 0x7fb2700cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb277000640 1 -- 172.21.15.66:0/3046828148 >> 172.21.15.66:0/3046828148 conn(0x7fb2700d0540 msgr2=0x7fb2700d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:58.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb277000640 1 -- 172.21.15.66:0/3046828148 shutdown_connections 2024-03-30T00:45:58.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.306+0000 7fb277000640 1 -- 172.21.15.66:0/3046828148 wait complete. 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb277000640 1 Processor -- start 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb277000640 1 -- start start 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb277000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2700af420 0x7fb27014fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb277000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2700ce740 0x7fb2701593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb277000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb27015de30 0x7fb2701526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb277000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb270150090 con 0x7fb27015de30 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb277000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb270151d60 con 0x7fb2700af420 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb277000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb270151a80 con 0x7fb2700ce740 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb275000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb27015de30 0x7fb2701526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb275000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb27015de30 0x7fb2701526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38702/0 (socket says 172.21.15.66:38702) 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb275000640 1 -- 172.21.15.66:0/1903505915 learned_addr learned my addr 172.21.15.66:0/1903505915 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb26fe00640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2700af420 0x7fb27014fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb267400640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2700ce740 0x7fb2701593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb275000640 1 -- 172.21.15.66:0/1903505915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2700ce740 msgr2=0x7fb2701593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb275000640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2700ce740 0x7fb2701593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb275000640 1 -- 172.21.15.66:0/1903505915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2700af420 msgr2=0x7fb27014fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb275000640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2700af420 0x7fb27014fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb275000640 1 -- 172.21.15.66:0/1903505915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2580094d0 con 0x7fb27015de30 2024-03-30T00:45:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb275000640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb27015de30 0x7fb2701526b0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c780 tx=0x7fb26800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb26cc00640 1 -- 172.21.15.66:0/1903505915 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268004b70 con 0x7fb27015de30 2024-03-30T00:45:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb26cc00640 1 -- 172.21.15.66:0/1903505915 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26800ce90 con 0x7fb27015de30 2024-03-30T00:45:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb26cc00640 1 -- 172.21.15.66:0/1903505915 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26800a850 con 0x7fb27015de30 2024-03-30T00:45:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb277000640 1 -- 172.21.15.66:0/1903505915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2701588f0 con 0x7fb27015de30 2024-03-30T00:45:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.310+0000 7fb277000640 1 -- 172.21.15.66:0/1903505915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb270158eb0 con 0x7fb27015de30 2024-03-30T00:45:58.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.314+0000 7fb277000640 1 -- 172.21.15.66:0/1903505915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb270158750 con 0x7fb27015de30 2024-03-30T00:45:58.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.314+0000 7fb26cc00640 1 -- 172.21.15.66:0/1903505915 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb26800a9b0 con 0x7fb27015de30 2024-03-30T00:45:58.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.314+0000 7fb26cc00640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb24c066690 0x7fb24c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.314+0000 7fb26cc00640 1 -- 172.21.15.66:0/1903505915 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fb268084400 con 0x7fb27015de30 2024-03-30T00:45:58.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.314+0000 7fb26fe00640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb24c066690 0x7fb24c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.314+0000 7fb26fe00640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb24c066690 0x7fb24c068b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb2600098f0 tx=0x7fb260006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.318+0000 7fb26cc00640 1 -- 172.21.15.66:0/1903505915 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb270158750 con 0x7fb27015de30 2024-03-30T00:45:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.478+0000 7fb277000640 1 -- 172.21.15.66:0/1903505915 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb270158750 con 0x7fb27015de30 2024-03-30T00:45:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.478+0000 7fb26cc00640 1 -- 172.21.15.66:0/1903505915 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb270158750 con 0x7fb27015de30 2024-03-30T00:45:58.488 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 -- 172.21.15.66:0/1903505915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb24c066690 msgr2=0x7fb24c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb24c066690 0x7fb24c068b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb2600098f0 tx=0x7fb260006f90 comp rx=0 tx=0).stop 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 -- 172.21.15.66:0/1903505915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb27015de30 msgr2=0x7fb2701526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb27015de30 0x7fb2701526b0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c780 tx=0x7fb26800cc40 comp rx=0 tx=0).stop 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 -- 172.21.15.66:0/1903505915 shutdown_connections 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb24c066690 0x7fb24c068b50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb27015de30 0x7fb2701526b0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2700ce740 0x7fb2701593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 --2- 172.21.15.66:0/1903505915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2700af420 0x7fb27014fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 -- 172.21.15.66:0/1903505915 >> 172.21.15.66:0/1903505915 conn(0x7fb2700d0540 msgr2=0x7fb2700cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 -- 172.21.15.66:0/1903505915 shutdown_connections 2024-03-30T00:45:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.482+0000 7fb277000640 1 -- 172.21.15.66:0/1903505915 wait complete. 2024-03-30T00:45:58.508 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:45:58.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.642+0000 7efc97200640 1 Processor -- start 2024-03-30T00:45:58.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.642+0000 7efc97200640 1 -- start start 2024-03-30T00:45:58.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.642+0000 7efc97200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efc900cd2d0 0x7efc900cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.642+0000 7efc97200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc900cc8d0 0x7efc9006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.642+0000 7efc97200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc900cb620 con 0x7efc900cad00 2024-03-30T00:45:58.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.642+0000 7efc97200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc900cbec0 con 0x7efc900cd2d0 2024-03-30T00:45:58.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.642+0000 7efc97200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc900ca660 con 0x7efc900cc8d0 2024-03-30T00:45:58.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.642+0000 7efc8fe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efc900cad00 0x7efc900c8940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:49918/0 (socket says 172.21.15.66:49918) 2024-03-30T00:45:58.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.642+0000 7efc8fe00640 1 -- 172.21.15.66:0/964123796 learned_addr learned my addr 172.21.15.66:0/964123796 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:45:58.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.642+0000 7efc87400640 1 --2- 172.21.15.66:0/964123796 >> v2:172.21.15.66:3300/0 conn(0x7efc900cd2d0 0x7efc900cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.646+0000 7efc87400640 1 -- 172.21.15.66:0/964123796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc900cc8d0 msgr2=0x7efc9006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:58.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.646+0000 7efc87400640 1 --2- 172.21.15.66:0/964123796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc900cc8d0 0x7efc9006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.646+0000 7efc87400640 1 -- 172.21.15.66:0/964123796 >> v1:172.21.15.66:6789/0 conn(0x7efc900cad00 legacy=0x7efc900c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.646+0000 7efc87400640 1 -- 172.21.15.66:0/964123796 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc900ca940 con 0x7efc900cd2d0 2024-03-30T00:45:58.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.646+0000 7efc8f400640 1 -- 172.21.15.66:0/964123796 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148834466 0 0) 0x7efc900cb620 con 0x7efc900cad00 2024-03-30T00:45:58.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.646+0000 7efc87400640 1 --2- 172.21.15.66:0/964123796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc900cd2d0 0x7efc900cb0e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7efc7800a960 tx=0x7efc7801d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9e46abe3eb22c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:58.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.646+0000 7efc8f400640 1 -- 172.21.15.66:0/964123796 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780a68f0 con 0x7efc900cd2d0 2024-03-30T00:45:58.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.646+0000 7efc8f400640 1 -- 172.21.15.66:0/964123796 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc780a6a50 con 0x7efc900cd2d0 2024-03-30T00:45:58.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.646+0000 7efc8f400640 1 -- 172.21.15.66:0/964123796 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780a6de0 con 0x7efc900cd2d0 2024-03-30T00:45:58.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.646+0000 7efc97200640 1 -- 172.21.15.66:0/964123796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc900cd2d0 msgr2=0x7efc900cb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.646+0000 7efc97200640 1 --2- 172.21.15.66:0/964123796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc900cd2d0 0x7efc900cb0e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7efc7800a960 tx=0x7efc7801d840 comp rx=0 tx=0).stop 2024-03-30T00:45:58.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 -- 172.21.15.66:0/964123796 shutdown_connections 2024-03-30T00:45:58.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 --2- 172.21.15.66:0/964123796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc900cc8d0 0x7efc9006c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 --2- 172.21.15.66:0/964123796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc900cd2d0 0x7efc900cb0e0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 -- 172.21.15.66:0/964123796 >> 172.21.15.66:0/964123796 conn(0x7efc90069f60 msgr2=0x7efc9006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:58.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 -- 172.21.15.66:0/964123796 shutdown_connections 2024-03-30T00:45:58.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 -- 172.21.15.66:0/964123796 wait complete. 2024-03-30T00:45:58.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 Processor -- start 2024-03-30T00:45:58.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 -- start start 2024-03-30T00:45:58.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc900cad00 0x7efc90162950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc900cc8d0 0x7efc901eb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc900cd2d0 0x7efc901577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc901586d0 con 0x7efc900cd2d0 2024-03-30T00:45:58.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc901596e0 con 0x7efc900cc8d0 2024-03-30T00:45:58.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc97200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc901599c0 con 0x7efc900cad00 2024-03-30T00:45:58.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc95200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc900cd2d0 0x7efc901577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc8fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc900cad00 0x7efc90162950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc8fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc900cad00 0x7efc90162950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:54554/0 (socket says 172.21.15.66:54554) 2024-03-30T00:45:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc8fe00640 1 -- 172.21.15.66:0/2853088058 learned_addr learned my addr 172.21.15.66:0/2853088058 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc87400640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc900cc8d0 0x7efc901eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc95200640 1 -- 172.21.15.66:0/2853088058 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc900cad00 msgr2=0x7efc90162950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc95200640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc900cad00 0x7efc90162950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc95200640 1 -- 172.21.15.66:0/2853088058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc900cc8d0 msgr2=0x7efc901eb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc95200640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc900cc8d0 0x7efc901eb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc95200640 1 -- 172.21.15.66:0/2853088058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7800a5c0 con 0x7efc900cd2d0 2024-03-30T00:45:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc8fe00640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc900cad00 0x7efc90162950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:45:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.650+0000 7efc95200640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc900cd2d0 0x7efc901577e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7efc8800be20 tx=0x7efc8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:58.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.654+0000 7efc8cc00640 1 -- 172.21.15.66:0/2853088058 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8800ce70 con 0x7efc900cd2d0 2024-03-30T00:45:58.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.654+0000 7efc8cc00640 1 -- 172.21.15.66:0/2853088058 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc88014440 con 0x7efc900cd2d0 2024-03-30T00:45:58.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.654+0000 7efc8cc00640 1 -- 172.21.15.66:0/2853088058 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88014810 con 0x7efc900cd2d0 2024-03-30T00:45:58.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.654+0000 7efc97200640 1 -- 172.21.15.66:0/2853088058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc90159ca0 con 0x7efc900cd2d0 2024-03-30T00:45:58.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.654+0000 7efc97200640 1 -- 172.21.15.66:0/2853088058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9015a260 con 0x7efc900cd2d0 2024-03-30T00:45:58.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.654+0000 7efc97200640 1 -- 172.21.15.66:0/2853088058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc900c6120 con 0x7efc900cd2d0 2024-03-30T00:45:58.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.654+0000 7efc8cc00640 1 -- 172.21.15.66:0/2853088058 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efc8801d3b0 con 0x7efc900cd2d0 2024-03-30T00:45:58.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.654+0000 7efc8cc00640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc6c066690 0x7efc6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.654+0000 7efc8cc00640 1 -- 172.21.15.66:0/2853088058 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7efc88084450 con 0x7efc900cd2d0 2024-03-30T00:45:58.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.654+0000 7efc8fe00640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc6c066690 0x7efc6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.654+0000 7efc8fe00640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc6c066690 0x7efc6c068b50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efc80000f70 tx=0x7efc80007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:58.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.658+0000 7efc8cc00640 1 -- 172.21.15.66:0/2853088058 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efc880516c0 con 0x7efc900cd2d0 2024-03-30T00:45:58.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.818+0000 7efc97200640 1 -- 172.21.15.66:0/2853088058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efc90066c30 con 0x7efc900cd2d0 2024-03-30T00:45:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.818+0000 7efc8cc00640 1 -- 172.21.15.66:0/2853088058 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc88054ce0 con 0x7efc900cd2d0 2024-03-30T00:45:58.829 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 -- 172.21.15.66:0/2853088058 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc6c066690 msgr2=0x7efc6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc6c066690 0x7efc6c068b50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efc80000f70 tx=0x7efc80007eb0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 -- 172.21.15.66:0/2853088058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc900cd2d0 msgr2=0x7efc901577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc900cd2d0 0x7efc901577e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7efc8800be20 tx=0x7efc8800bf10 comp rx=0 tx=0).stop 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 -- 172.21.15.66:0/2853088058 shutdown_connections 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc6c066690 0x7efc6c068b50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc900cd2d0 0x7efc901577e0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc900cc8d0 0x7efc901eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 --2- 172.21.15.66:0/2853088058 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc900cad00 0x7efc90162950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 -- 172.21.15.66:0/2853088058 >> 172.21.15.66:0/2853088058 conn(0x7efc90069f60 msgr2=0x7efc900b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 -- 172.21.15.66:0/2853088058 shutdown_connections 2024-03-30T00:45:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.822+0000 7efc97200640 1 -- 172.21.15.66:0/2853088058 wait complete. 2024-03-30T00:45:58.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:45:58.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.982+0000 7f6fe1c00640 1 Processor -- start 2024-03-30T00:45:58.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.982+0000 7f6fe1c00640 1 -- start start 2024-03-30T00:45:58.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fe1c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6fdc0b4a60 0x7f6fdc15f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fe1c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc099e70 con 0x7f6fdc098c80 2024-03-30T00:45:58.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fe1c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc0cbd90 con 0x7f6fdc0cf320 2024-03-30T00:45:58.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fe1c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fdc0cc070 con 0x7f6fdc0b4a60 2024-03-30T00:45:58.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fdaa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6fdc098c80 0x7f6fdc099760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52890/0 (socket says 172.21.15.66:52890) 2024-03-30T00:45:58.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fdaa00640 1 -- 172.21.15.66:0/2761280755 learned_addr learned my addr 172.21.15.66:0/2761280755 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:45:58.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fdb400640 1 --2- 172.21.15.66:0/2761280755 >> v2:172.21.15.156:3300/0 conn(0x7f6fdc0b4a60 0x7f6fdc15f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005368089 0 0) 0x7f6fdc099e70 con 0x7f6fdc098c80 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc0003530 con 0x7f6fdc098c80 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1431254558 0 0) 0x7f6fc0003530 con 0x7f6fdc098c80 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 >> v2:172.21.15.156:3300/0 conn(0x7f6fdc0b4a60 msgr2=0x7f6fdc15f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 --2- 172.21.15.66:0/2761280755 >> v2:172.21.15.156:3300/0 conn(0x7f6fdc0b4a60 0x7f6fdc15f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 >> v1:172.21.15.156:6789/0 conn(0x7f6fdc0cf320 legacy=0x7f6fdc15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc0cc350 con 0x7f6fdc098c80 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f6fcc001a00 con 0x7f6fdc098c80 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 >> v1:172.21.15.66:6789/0 conn(0x7f6fdc098c80 legacy=0x7f6fdc099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 --2- 172.21.15.66:0/2761280755 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fc0003980 0x7f6fc0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 --2- 172.21.15.66:0/2761280755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fc00047d0 0x7f6fc0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 --2- 172.21.15.66:0/2761280755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fc0005190 0x7f6fc000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:58.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc0cc070 con 0x7f6fc0005190 2024-03-30T00:45:58.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc000c040 con 0x7f6fc00047d0 2024-03-30T00:45:58.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc000c200 con 0x7f6fc0003980 2024-03-30T00:45:58.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fdb400640 1 --2- 172.21.15.66:0/2761280755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fc0005190 0x7f6fc000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fdaa00640 1 --2- 172.21.15.66:0/2761280755 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fc0003980 0x7f6fc0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fd2000640 1 --2- 172.21.15.66:0/2761280755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fc00047d0 0x7f6fc0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:58.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fdb400640 1 -- 172.21.15.66:0/2761280755 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fc0003980 msgr2=0x7f6fc0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:58.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fdb400640 1 --2- 172.21.15.66:0/2761280755 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fc0003980 0x7f6fc0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fdb400640 1 -- 172.21.15.66:0/2761280755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fc00047d0 msgr2=0x7f6fc0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:58.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fdb400640 1 --2- 172.21.15.66:0/2761280755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fc00047d0 0x7f6fc0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.986+0000 7f6fdb400640 1 -- 172.21.15.66:0/2761280755 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc0003530 con 0x7f6fc0005190 2024-03-30T00:45:58.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fdb400640 1 -- 172.21.15.66:0/2761280755 reap_dead start 2024-03-30T00:45:58.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fdb400640 1 --2- 172.21.15.66:0/2761280755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fc0005190 0x7f6fc000b990 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009ae0 tx=0x7f6fd401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fb38163e55355f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:58.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4004020 con 0x7f6fc0005190 2024-03-30T00:45:58.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd40041c0 con 0x7f6fc0005190 2024-03-30T00:45:58.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fda000640 1 -- 172.21.15.66:0/2761280755 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd40a56a0 con 0x7f6fc0005190 2024-03-30T00:45:58.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2761280755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fc0005190 msgr2=0x7f6fc000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:58.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fe1c00640 1 --2- 172.21.15.66:0/2761280755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fc0005190 0x7f6fc000b990 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009ae0 tx=0x7f6fd401cab0 comp rx=0 tx=0).stop 2024-03-30T00:45:58.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2761280755 shutdown_connections 2024-03-30T00:45:58.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fe1c00640 1 --2- 172.21.15.66:0/2761280755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fc0005190 0x7f6fc000b990 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2761280755 >> 172.21.15.66:0/2761280755 conn(0x7f6fdc0b2130 msgr2=0x7f6fdc0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:59.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2761280755 shutdown_connections 2024-03-30T00:45:59.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.990+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2761280755 wait complete. 2024-03-30T00:45:59.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fe1c00640 1 Processor -- start 2024-03-30T00:45:59.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fe1c00640 1 -- start start 2024-03-30T00:45:59.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fe1c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fc0005190 0x7f6fdc1f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fe1c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fdc15b110 0x7f6fdc15b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fe1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fdc15cdf0 0x7f6fdc15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fe1c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc15dac0 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fe1c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc15dda0 con 0x7f6fc0005190 2024-03-30T00:45:59.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fe1c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc15e080 con 0x7f6fdc15b110 2024-03-30T00:45:59.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fdb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fdc15cdf0 0x7f6fdc15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fdb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fdc15cdf0 0x7f6fdc15d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37784/0 (socket says 172.21.15.66:37784) 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fdb400640 1 -- 172.21.15.66:0/2617008993 learned_addr learned my addr 172.21.15.66:0/2617008993 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fd2000640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fdc15b110 0x7f6fdc15b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fdaa00640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fc0005190 0x7f6fdc1f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fdb400640 1 -- 172.21.15.66:0/2617008993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fdc15b110 msgr2=0x7f6fdc15b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fdb400640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fdc15b110 0x7f6fdc15b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fdb400640 1 -- 172.21.15.66:0/2617008993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fc0005190 msgr2=0x7f6fdc1f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fdb400640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fc0005190 0x7f6fdc1f1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fdb400640 1 -- 172.21.15.66:0/2617008993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc15e360 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fdb400640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fdc15cdf0 0x7f6fdc15d270 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6fd401c8e0 tx=0x7f6fd4019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fd3400640 1 -- 172.21.15.66:0/2617008993 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4004530 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fd3400640 1 -- 172.21.15.66:0/2617008993 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd40b2040 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2617008993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc0c46f0 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fd3400640 1 -- 172.21.15.66:0/2617008993 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd40ad6c0 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2617008993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc14ecb0 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.994+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2617008993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fdc0677d0 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.998+0000 7f6fd3400640 1 -- 172.21.15.66:0/2617008993 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6fd40ad820 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.998+0000 7f6fd3400640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6fb8066690 0x7f6fb8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.998+0000 7f6fd3400640 1 -- 172.21.15.66:0/2617008993 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f6fd411e050 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.998+0000 7f6fdaa00640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6fb8066690 0x7f6fb8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:58.998+0000 7f6fdaa00640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6fb8066690 0x7f6fb8068b50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc000f90 tx=0x7f6fcc006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:59.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.002+0000 7f6fd3400640 1 -- 172.21.15.66:0/2617008993 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6fd40eb230 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.162+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2617008993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6fdc066c30 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.162+0000 7f6fd3400640 1 -- 172.21.15.66:0/2617008993 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6fd40ee850 con 0x7f6fdc15cdf0 2024-03-30T00:45:59.171 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:45:59.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2617008993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6fb8066690 msgr2=0x7f6fb8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6fb8066690 0x7f6fb8068b50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc000f90 tx=0x7f6fcc006040 comp rx=0 tx=0).stop 2024-03-30T00:45:59.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2617008993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fdc15cdf0 msgr2=0x7f6fdc15d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fdc15cdf0 0x7f6fdc15d270 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6fd401c8e0 tx=0x7f6fd4019850 comp rx=0 tx=0).stop 2024-03-30T00:45:59.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2617008993 shutdown_connections 2024-03-30T00:45:59.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6fb8066690 0x7f6fb8068b50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fdc15cdf0 0x7f6fdc15d270 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fdc15b110 0x7f6fdc15b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 --2- 172.21.15.66:0/2617008993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fc0005190 0x7f6fdc1f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2617008993 >> 172.21.15.66:0/2617008993 conn(0x7f6fdc0b2130 msgr2=0x7f6fdc0b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:59.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2617008993 shutdown_connections 2024-03-30T00:45:59.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.166+0000 7f6fe1c00640 1 -- 172.21.15.66:0/2617008993 wait complete. 2024-03-30T00:45:59.193 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:45:59.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00fc400640 1 Processor -- start 2024-03-30T00:45:59.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00fc400640 1 -- start start 2024-03-30T00:45:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00fc400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f00f40cec30 0x7f00f4159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00fc400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f40cd3e0 con 0x7f00f40cde70 2024-03-30T00:45:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00fc400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f415a990 con 0x7f00f40cda90 2024-03-30T00:45:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00fc400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f40b1b80 con 0x7f00f40cec30 2024-03-30T00:45:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00fa400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f00f40cde70 0x7f00f415a1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52896/0 (socket says 172.21.15.66:52896) 2024-03-30T00:45:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00fa400640 1 -- 172.21.15.66:0/2943055923 learned_addr learned my addr 172.21.15.66:0/2943055923 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:45:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f9000640 1 --2- 172.21.15.66:0/2943055923 >> v2:172.21.15.156:3300/0 conn(0x7f00f40cec30 0x7f00f4159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549001513 0 0) 0x7f00f40cd3e0 con 0x7f00f40cde70 2024-03-30T00:45:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00dc003530 con 0x7f00f40cde70 2024-03-30T00:45:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 916105911 0 0) 0x7f00dc003530 con 0x7f00f40cde70 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 >> v2:172.21.15.156:3300/0 conn(0x7f00f40cec30 msgr2=0x7f00f4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 --2- 172.21.15.66:0/2943055923 >> v2:172.21.15.156:3300/0 conn(0x7f00f40cec30 0x7f00f4159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 >> v1:172.21.15.156:6789/0 conn(0x7f00f40cda90 legacy=0x7f00f40ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f40b11e0 con 0x7f00f40cde70 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f00ec001a00 con 0x7f00f40cde70 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 >> v1:172.21.15.66:6789/0 conn(0x7f00f40cde70 legacy=0x7f00f415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00dc003980 0x7f00dc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00dc0047d0 0x7f00dc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00dc005190 0x7f00dc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40b1b80 con 0x7f00dc003980 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc00c130 con 0x7f00dc0047d0 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc00c2f0 con 0x7f00dc005190 2024-03-30T00:45:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f9a00640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00dc003980 0x7f00dc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f9000640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00dc0047d0 0x7f00dc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00fa400640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00dc005190 0x7f00dc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f9a00640 1 -- 172.21.15.66:0/2943055923 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00dc005190 msgr2=0x7f00dc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f9a00640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00dc005190 0x7f00dc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f9a00640 1 -- 172.21.15.66:0/2943055923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00dc0047d0 msgr2=0x7f00dc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f9a00640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00dc0047d0 0x7f00dc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00f9a00640 1 -- 172.21.15.66:0/2943055923 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc003530 con 0x7f00dc003980 2024-03-30T00:45:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00fa400640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00dc005190 0x7f00dc00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.326+0000 7f00fa400640 1 -- 172.21.15.66:0/2943055923 reap_dead start 2024-03-30T00:45:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.330+0000 7f00f9a00640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00dc003980 0x7f00dc003da0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f00e8001780 tx=0x7f00e801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388055fa011b161 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.330+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e80052d0 con 0x7f00dc003980 2024-03-30T00:45:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.330+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e80a7050 con 0x7f00dc003980 2024-03-30T00:45:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.330+0000 7f00f3e00640 1 -- 172.21.15.66:0/2943055923 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e8003600 con 0x7f00dc003980 2024-03-30T00:45:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.330+0000 7f00fc400640 1 -- 172.21.15.66:0/2943055923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00dc003980 msgr2=0x7f00dc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.330+0000 7f00fc400640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00dc003980 0x7f00dc003da0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f00e8001780 tx=0x7f00e801c400 comp rx=0 tx=0).stop 2024-03-30T00:45:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.330+0000 7f00fc400640 1 -- 172.21.15.66:0/2943055923 shutdown_connections 2024-03-30T00:45:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.330+0000 7f00fc400640 1 --2- 172.21.15.66:0/2943055923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00dc003980 0x7f00dc003da0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.330+0000 7f00fc400640 1 -- 172.21.15.66:0/2943055923 >> 172.21.15.66:0/2943055923 conn(0x7f00f4069f60 msgr2=0x7f00f406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.330+0000 7f00fc400640 1 -- 172.21.15.66:0/2943055923 shutdown_connections 2024-03-30T00:45:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.330+0000 7f00fc400640 1 -- 172.21.15.66:0/2943055923 wait complete. 2024-03-30T00:45:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fc400640 1 Processor -- start 2024-03-30T00:45:59.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fc400640 1 -- start start 2024-03-30T00:45:59.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fc400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00dc003980 0x7f00f41f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fc400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f406d770 0x7f00f40acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fc400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f40c0d90 0x7f00f4153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fc400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40aca90 con 0x7f00f40c0d90 2024-03-30T00:45:59.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fc400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40ac160 con 0x7f00f406d770 2024-03-30T00:45:59.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fc400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40ac440 con 0x7f00dc003980 2024-03-30T00:45:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fa400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f40c0d90 0x7f00f4153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fa400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f40c0d90 0x7f00f4153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37798/0 (socket says 172.21.15.66:37798) 2024-03-30T00:45:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fa400640 1 -- 172.21.15.66:0/3537289220 learned_addr learned my addr 172.21.15.66:0/3537289220 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00f9a00640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00dc003980 0x7f00f41f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00f9000640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f406d770 0x7f00f40acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fa400640 1 -- 172.21.15.66:0/3537289220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00dc003980 msgr2=0x7f00f41f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fa400640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00dc003980 0x7f00f41f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fa400640 1 -- 172.21.15.66:0/3537289220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f406d770 msgr2=0x7f00f40acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fa400640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f406d770 0x7f00f40acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fa400640 1 -- 172.21.15.66:0/3537289220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f40ac720 con 0x7f00f40c0d90 2024-03-30T00:45:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fa400640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f40c0d90 0x7f00f4153f60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f00ec002020 tx=0x7f00ec00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00f1600640 1 -- 172.21.15.66:0/3537289220 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec00eb90 con 0x7f00f40c0d90 2024-03-30T00:45:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00f1600640 1 -- 172.21.15.66:0/3537289220 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec00ecf0 con 0x7f00f40c0d90 2024-03-30T00:45:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00f1600640 1 -- 172.21.15.66:0/3537289220 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec013640 con 0x7f00f40c0d90 2024-03-30T00:45:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fc400640 1 -- 172.21.15.66:0/3537289220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f40abe70 con 0x7f00f40c0d90 2024-03-30T00:45:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.334+0000 7f00fc400640 1 -- 172.21.15.66:0/3537289220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f40ad460 con 0x7f00f40c0d90 2024-03-30T00:45:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.338+0000 7f00fc400640 1 -- 172.21.15.66:0/3537289220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f4066c30 con 0x7f00f40c0d90 2024-03-30T00:45:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.338+0000 7f00f1600640 1 -- 172.21.15.66:0/3537289220 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f00ec0137a0 con 0x7f00f40c0d90 2024-03-30T00:45:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.338+0000 7f00f1600640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f00d40666c0 0x7f00d4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.338+0000 7f00f1600640 1 -- 172.21.15.66:0/3537289220 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f00ec0847f0 con 0x7f00f40c0d90 2024-03-30T00:45:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.338+0000 7f00f9a00640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f00d40666c0 0x7f00d4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.338+0000 7f00f9a00640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f00d40666c0 0x7f00d4068b80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f00e8001750 tx=0x7f00e80011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:59.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.342+0000 7f00f1600640 1 -- 172.21.15.66:0/3537289220 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f00ec051920 con 0x7f00f40c0d90 2024-03-30T00:45:59.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.514+0000 7f00fc400640 1 -- 172.21.15.66:0/3537289220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f00f40b3170 con 0x7f00f40c0d90 2024-03-30T00:45:59.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.514+0000 7f00f1600640 1 -- 172.21.15.66:0/3537289220 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v83) v1 ==== 80+0+124 (secure 0 0 0) 0x7f00ec054f40 con 0x7f00f40c0d90 2024-03-30T00:45:59.523 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.156:6835/2089872680 2024-03-31T00:44:35.079618+0000 2024-03-30T00:45:59.523 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.156:6833/2089872680 2024-03-31T00:44:35.079618+0000 2024-03-30T00:45:59.523 INFO:teuthology.orchestra.run.smithi066.stderr:listed 2 entries 2024-03-30T00:45:59.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 -- 172.21.15.66:0/3537289220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f00d40666c0 msgr2=0x7f00d4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f00d40666c0 0x7f00d4068b80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f00e8001750 tx=0x7f00e80011a0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 -- 172.21.15.66:0/3537289220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f40c0d90 msgr2=0x7f00f4153f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f40c0d90 0x7f00f4153f60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f00ec002020 tx=0x7f00ec00e530 comp rx=0 tx=0).stop 2024-03-30T00:45:59.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 -- 172.21.15.66:0/3537289220 shutdown_connections 2024-03-30T00:45:59.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f00d40666c0 0x7f00d4068b80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00f40c0d90 0x7f00f4153f60 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f406d770 0x7f00f40acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 --2- 172.21.15.66:0/3537289220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00dc003980 0x7f00f41f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 -- 172.21.15.66:0/3537289220 >> 172.21.15.66:0/3537289220 conn(0x7f00f4069f60 msgr2=0x7f00f40cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:59.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 -- 172.21.15.66:0/3537289220 shutdown_connections 2024-03-30T00:45:59.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.518+0000 7f00fc400640 1 -- 172.21.15.66:0/3537289220 wait complete. 2024-03-30T00:45:59.544 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:45:59.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:59.544 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:45:59.563 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[46066]: fuse finished with error 0 and tester_r 0 2024-03-30T00:45:59.564 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:45:59.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:59.565 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:45:59.621 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:45:59.622 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:45:59.632 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:45:59.632 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:45:59.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:59.633 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T00:45:59.682 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-30T00:45:59.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:45:59.684 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:45:59.684 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:45:59.694 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:45:59.694 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:45:59.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:59.695 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-30T00:45:59.744 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-30T00:45:59.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:45:59.746 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:45:59.746 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:45:59.757 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:45:59.758 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:45:59.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:45:59.758 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-30T00:45:59.806 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-30T00:45:59.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:45:59.808 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:45:59.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cc0400640 1 Processor -- start 2024-03-30T00:45:59.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cc0400640 1 -- start start 2024-03-30T00:45:59.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cc0400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8cb80cda90 0x7f8cb80ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cc0400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb80cde70 0x7f8cb80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cc0400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb815a990 con 0x7f8cb8159c60 2024-03-30T00:45:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cc0400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb80b1b80 con 0x7f8cb80cda90 2024-03-30T00:45:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cc0400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb80cd3e0 con 0x7f8cb80cde70 2024-03-30T00:45:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cbda00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8cb80cda90 0x7f8cb80ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cbda00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8cb80cda90 0x7f8cb80ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37802/0 (socket says 172.21.15.66:37802) 2024-03-30T00:45:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cbda00640 1 -- 172.21.15.66:0/3225234506 learned_addr learned my addr 172.21.15.66:0/3225234506 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cbd000640 1 --2- 172.21.15.66:0/3225234506 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb80cde70 0x7f8cb80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cbda00640 1 -- 172.21.15.66:0/3225234506 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb80cde70 msgr2=0x7f8cb80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:45:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cbda00640 1 --2- 172.21.15.66:0/3225234506 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb80cde70 0x7f8cb80cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cbda00640 1 -- 172.21.15.66:0/3225234506 >> v1:172.21.15.156:6789/0 conn(0x7f8cb8159c60 legacy=0x7f8cb815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.942+0000 7f8cbda00640 1 -- 172.21.15.66:0/3225234506 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb80b11e0 con 0x7f8cb80cda90 2024-03-30T00:45:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cbda00640 1 --2- 172.21.15.66:0/3225234506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb80cda90 0x7f8cb80ce520 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009870 tx=0x7f8cac01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b7c1a55738c818 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:59.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cb7e00640 1 -- 172.21.15.66:0/3225234506 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac0ac640 con 0x7f8cb80cda90 2024-03-30T00:45:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cb7e00640 1 -- 172.21.15.66:0/3225234506 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac0ac7a0 con 0x7f8cb80cda90 2024-03-30T00:45:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cb7e00640 1 -- 172.21.15.66:0/3225234506 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac0acb30 con 0x7f8cb80cda90 2024-03-30T00:45:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cc0400640 1 -- 172.21.15.66:0/3225234506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb80cda90 msgr2=0x7f8cb80ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cc0400640 1 --2- 172.21.15.66:0/3225234506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb80cda90 0x7f8cb80ce520 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009870 tx=0x7f8cac01c4d0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cc0400640 1 -- 172.21.15.66:0/3225234506 shutdown_connections 2024-03-30T00:45:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cc0400640 1 --2- 172.21.15.66:0/3225234506 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb80cde70 0x7f8cb80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cc0400640 1 --2- 172.21.15.66:0/3225234506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb80cda90 0x7f8cb80ce520 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cc0400640 1 -- 172.21.15.66:0/3225234506 >> 172.21.15.66:0/3225234506 conn(0x7f8cb8069f60 msgr2=0x7f8cb806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:45:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cc0400640 1 -- 172.21.15.66:0/3225234506 shutdown_connections 2024-03-30T00:45:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.946+0000 7f8cc0400640 1 -- 172.21.15.66:0/3225234506 wait complete. 2024-03-30T00:45:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cc0400640 1 Processor -- start 2024-03-30T00:45:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cc0400640 1 -- start start 2024-03-30T00:45:59.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cc0400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb80cda90 0x7f8cb8097860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cc0400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb80cde70 0x7f8cb80abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cc0400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb8159c60 0x7f8cb8168530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cc0400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb80ac7c0 con 0x7f8cb80cde70 2024-03-30T00:45:59.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cc0400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb80acaa0 con 0x7f8cb8159c60 2024-03-30T00:45:59.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cc0400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb80acd80 con 0x7f8cb80cda90 2024-03-30T00:45:59.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb80cde70 0x7f8cb80abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb80cde70 0x7f8cb80abe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37816/0 (socket says 172.21.15.66:37816) 2024-03-30T00:45:59.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbd000640 1 -- 172.21.15.66:0/1656636693 learned_addr learned my addr 172.21.15.66:0/1656636693 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:45:59.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbda00640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb80cda90 0x7f8cb8097860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbe400640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb8159c60 0x7f8cb8168530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbd000640 1 -- 172.21.15.66:0/1656636693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb80cda90 msgr2=0x7f8cb8097860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbd000640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb80cda90 0x7f8cb8097860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbd000640 1 -- 172.21.15.66:0/1656636693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb8159c60 msgr2=0x7f8cb8168530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:45:59.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbd000640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb8159c60 0x7f8cb8168530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:45:59.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbd000640 1 -- 172.21.15.66:0/1656636693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0094d0 con 0x7f8cb80cde70 2024-03-30T00:45:59.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbda00640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb80cda90 0x7f8cb8097860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:45:59.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cbd000640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb80cde70 0x7f8cb80abe70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400d800 tx=0x7f8ca400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:59.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cb5600640 1 -- 172.21.15.66:0/1656636693 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400b6c0 con 0x7f8cb80cde70 2024-03-30T00:45:59.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cb5600640 1 -- 172.21.15.66:0/1656636693 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca400b860 con 0x7f8cb80cde70 2024-03-30T00:45:59.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cc0400640 1 -- 172.21.15.66:0/1656636693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb814eb30 con 0x7f8cb80cde70 2024-03-30T00:45:59.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cb5600640 1 -- 172.21.15.66:0/1656636693 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400bc10 con 0x7f8cb80cde70 2024-03-30T00:45:59.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cc0400640 1 -- 172.21.15.66:0/1656636693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb80ac580 con 0x7f8cb80cde70 2024-03-30T00:45:59.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.950+0000 7f8cc0400640 1 -- 172.21.15.66:0/1656636693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb814ed20 con 0x7f8cb80cde70 2024-03-30T00:45:59.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.954+0000 7f8cb5600640 1 -- 172.21.15.66:0/1656636693 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8ca401e460 con 0x7f8cb80cde70 2024-03-30T00:45:59.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.954+0000 7f8cb5600640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c98066690 0x7f8c98068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:45:59.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.954+0000 7f8cb5600640 1 -- 172.21.15.66:0/1656636693 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f8ca40847d0 con 0x7f8cb80cde70 2024-03-30T00:45:59.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.954+0000 7f8cbda00640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c98066690 0x7f8c98068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:45:59.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.954+0000 7f8cbda00640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c98066690 0x7f8c98068b50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0ae000 tx=0x7f8cac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:45:59.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:45:59.958+0000 7f8cb5600640 1 -- 172.21.15.66:0/1656636693 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8ca4051900 con 0x7f8cb80cde70 2024-03-30T00:46:00.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.150+0000 7f8cc0400640 1 -- 172.21.15.66:0/1656636693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cb80ac580 con 0x7f8cb80cde70 2024-03-30T00:46:00.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.150+0000 7f8cb5600640 1 -- 172.21.15.66:0/1656636693 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f8cb80ac580 con 0x7f8cb80cde70 2024-03-30T00:46:00.160 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:00.160 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/254323195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":254323195},{"type":"v1","addr":"172.21.15.156:6835","nonce":254323195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:45:30.650828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":23,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:46:00.161 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 51 2024-03-30T00:46:00.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 -- 172.21.15.66:0/1656636693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c98066690 msgr2=0x7f8c98068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c98066690 0x7f8c98068b50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0ae000 tx=0x7f8cac008040 comp rx=0 tx=0).stop 2024-03-30T00:46:00.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 -- 172.21.15.66:0/1656636693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb80cde70 msgr2=0x7f8cb80abe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb80cde70 0x7f8cb80abe70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400d800 tx=0x7f8ca400dcc0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 -- 172.21.15.66:0/1656636693 shutdown_connections 2024-03-30T00:46:00.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb8159c60 0x7f8cb8168530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c98066690 0x7f8c98068b50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb80cde70 0x7f8cb80abe70 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 --2- 172.21.15.66:0/1656636693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb80cda90 0x7f8cb8097860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 -- 172.21.15.66:0/1656636693 >> 172.21.15.66:0/1656636693 conn(0x7f8cb8069f60 msgr2=0x7f8cb80afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:00.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 -- 172.21.15.66:0/1656636693 shutdown_connections 2024-03-30T00:46:00.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.154+0000 7f8cc0400640 1 -- 172.21.15.66:0/1656636693 wait complete. 2024-03-30T00:46:00.181 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:46:00.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.314+0000 7f1e9de00640 1 Processor -- start 2024-03-30T00:46:00.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.314+0000 7f1e9de00640 1 -- start start 2024-03-30T00:46:00.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e9de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1e980ce520 0x7f1e980ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:00.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e9de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9806d070 con 0x7f1e980cabb0 2024-03-30T00:46:00.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e9de00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e980cbba0 con 0x7f1e9815a640 2024-03-30T00:46:00.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e9de00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e980cee70 con 0x7f1e980ce520 2024-03-30T00:46:00.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e97400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1e980cabb0 0x7f1e980cafe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52902/0 (socket says 172.21.15.66:52902) 2024-03-30T00:46:00.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e97400640 1 -- 172.21.15.66:0/446489590 learned_addr learned my addr 172.21.15.66:0/446489590 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:00.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e96000640 1 --2- 172.21.15.66:0/446489590 >> v2:172.21.15.66:3300/0 conn(0x7f1e980ce520 0x7f1e980ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:00.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e96000640 1 -- 172.21.15.66:0/446489590 >> v1:172.21.15.156:6789/0 conn(0x7f1e9815a640 legacy=0x7f1e9806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:00.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e96000640 1 -- 172.21.15.66:0/446489590 >> v1:172.21.15.66:6789/0 conn(0x7f1e980cabb0 legacy=0x7f1e980cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e96000640 1 -- 172.21.15.66:0/446489590 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e980cb8c0 con 0x7f1e980ce520 2024-03-30T00:46:00.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e95600640 1 -- 172.21.15.66:0/446489590 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439679121 0 0) 0x7f1e9806d070 con 0x7f1e980cabb0 2024-03-30T00:46:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e96000640 1 --2- 172.21.15.66:0/446489590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e980ce520 0x7f1e980ce930 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1e840097c0 tx=0x7f1e8401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f18a5de1a012f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e95600640 1 -- 172.21.15.66:0/446489590 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84004230 con 0x7f1e980ce520 2024-03-30T00:46:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e95600640 1 -- 172.21.15.66:0/446489590 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e840a7050 con 0x7f1e980ce520 2024-03-30T00:46:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.318+0000 7f1e95600640 1 -- 172.21.15.66:0/446489590 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84003620 con 0x7f1e980ce520 2024-03-30T00:46:00.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 -- 172.21.15.66:0/446489590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e980ce520 msgr2=0x7f1e980ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 --2- 172.21.15.66:0/446489590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e980ce520 0x7f1e980ce930 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1e840097c0 tx=0x7f1e8401c460 comp rx=0 tx=0).stop 2024-03-30T00:46:00.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 -- 172.21.15.66:0/446489590 shutdown_connections 2024-03-30T00:46:00.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 --2- 172.21.15.66:0/446489590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e980ce520 0x7f1e980ce930 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 -- 172.21.15.66:0/446489590 >> 172.21.15.66:0/446489590 conn(0x7f1e98069f60 msgr2=0x7f1e9806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:00.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 -- 172.21.15.66:0/446489590 shutdown_connections 2024-03-30T00:46:00.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 -- 172.21.15.66:0/446489590 wait complete. 2024-03-30T00:46:00.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 Processor -- start 2024-03-30T00:46:00.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 -- start start 2024-03-30T00:46:00.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e980cabb0 0x7f1e980cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:00.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e980ce520 0x7f1e98157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:00.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e9815a640 0x7f1e98070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:00.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980c53c0 con 0x7f1e980ce520 2024-03-30T00:46:00.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e98157780 con 0x7f1e9815a640 2024-03-30T00:46:00.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e9de00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980c4d90 con 0x7f1e980cabb0 2024-03-30T00:46:00.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e96000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e980ce520 0x7f1e98157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:00.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e96000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e980ce520 0x7f1e98157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37824/0 (socket says 172.21.15.66:37824) 2024-03-30T00:46:00.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e96000640 1 -- 172.21.15.66:0/2618729681 learned_addr learned my addr 172.21.15.66:0/2618729681 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:00.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e96a00640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e980cabb0 0x7f1e980cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:00.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e97400640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e9815a640 0x7f1e98070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:00.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e96000640 1 -- 172.21.15.66:0/2618729681 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e980cabb0 msgr2=0x7f1e980cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e96000640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e980cabb0 0x7f1e980cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e96000640 1 -- 172.21.15.66:0/2618729681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e9815a640 msgr2=0x7f1e98070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e96000640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e9815a640 0x7f1e98070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e96000640 1 -- 172.21.15.66:0/2618729681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e84009420 con 0x7f1e980ce520 2024-03-30T00:46:00.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.322+0000 7f1e96a00640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e980cabb0 0x7f1e980cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:00.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.326+0000 7f1e96000640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e980ce520 0x7f1e98157240 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1e840ae040 tx=0x7f1e84019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:00.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.326+0000 7f1e8f400640 1 -- 172.21.15.66:0/2618729681 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84003cf0 con 0x7f1e980ce520 2024-03-30T00:46:00.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.326+0000 7f1e8f400640 1 -- 172.21.15.66:0/2618729681 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e84003e50 con 0x7f1e980ce520 2024-03-30T00:46:00.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.326+0000 7f1e9de00640 1 -- 172.21.15.66:0/2618729681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e980c5030 con 0x7f1e980ce520 2024-03-30T00:46:00.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.326+0000 7f1e8f400640 1 -- 172.21.15.66:0/2618729681 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840ab660 con 0x7f1e980ce520 2024-03-30T00:46:00.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.326+0000 7f1e9de00640 1 -- 172.21.15.66:0/2618729681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e980c58f0 con 0x7f1e980ce520 2024-03-30T00:46:00.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.326+0000 7f1e9de00640 1 -- 172.21.15.66:0/2618729681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e98066c30 con 0x7f1e980ce520 2024-03-30T00:46:00.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.326+0000 7f1e8f400640 1 -- 172.21.15.66:0/2618729681 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1e840b2040 con 0x7f1e980ce520 2024-03-30T00:46:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.326+0000 7f1e8f400640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e78066690 0x7f1e78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.326+0000 7f1e8f400640 1 -- 172.21.15.66:0/2618729681 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f1e8411e430 con 0x7f1e980ce520 2024-03-30T00:46:00.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.326+0000 7f1e96a00640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e78066690 0x7f1e78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:00.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.330+0000 7f1e96a00640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e78066690 0x7f1e78068b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1e880098b0 tx=0x7f1e88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:00.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.330+0000 7f1e8f400640 1 -- 172.21.15.66:0/2618729681 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1e840eb710 con 0x7f1e980ce520 2024-03-30T00:46:00.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 -- 172.21.15.66:0/2618729681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e980b2f10 con 0x7f1e980ce520 2024-03-30T00:46:00.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e8f400640 1 -- 172.21.15.66:0/2618729681 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16440 (secure 0 0 0) 0x7f1e840eed30 con 0x7f1e980ce520 2024-03-30T00:46:00.502 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:00.502 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":83,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:35.079642+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:44:14.685880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-30T00:44:15.260373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.156:6835/2089872680":"2024-03-31T00:44:35.079618+0000","172.21.15.156:6833/2089872680":"2024-03-31T00:44:35.079618+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:46:00.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 -- 172.21.15.66:0/2618729681 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e78066690 msgr2=0x7f1e78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e78066690 0x7f1e78068b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1e880098b0 tx=0x7f1e88008040 comp rx=0 tx=0).stop 2024-03-30T00:46:00.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 -- 172.21.15.66:0/2618729681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e980ce520 msgr2=0x7f1e98157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e980ce520 0x7f1e98157240 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1e840ae040 tx=0x7f1e84019200 comp rx=0 tx=0).stop 2024-03-30T00:46:00.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 -- 172.21.15.66:0/2618729681 shutdown_connections 2024-03-30T00:46:00.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e9815a640 0x7f1e98070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e78066690 0x7f1e78068b50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e980ce520 0x7f1e98157240 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 --2- 172.21.15.66:0/2618729681 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e980cabb0 0x7f1e980cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 -- 172.21.15.66:0/2618729681 >> 172.21.15.66:0/2618729681 conn(0x7f1e98069f60 msgr2=0x7f1e980af570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:00.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 -- 172.21.15.66:0/2618729681 shutdown_connections 2024-03-30T00:46:00.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.494+0000 7f1e9de00640 1 -- 172.21.15.66:0/2618729681 wait complete. 2024-03-30T00:46:00.523 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:44:14.685880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:44:15.260373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:46:00.523 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:46:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.658+0000 7fca5aa00640 1 Processor -- start 2024-03-30T00:46:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.658+0000 7fca5aa00640 1 -- start start 2024-03-30T00:46:00.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.658+0000 7fca5aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca5415a640 0x7fca5406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:00.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.658+0000 7fca5aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fca540ce520 0x7fca540ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:00.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.658+0000 7fca5aa00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5406d070 con 0x7fca540cabb0 2024-03-30T00:46:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.658+0000 7fca5aa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca540cbba0 con 0x7fca540ce520 2024-03-30T00:46:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.658+0000 7fca5aa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5406e2c0 con 0x7fca5415a640 2024-03-30T00:46:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.658+0000 7fca4b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fca540ce520 0x7fca540ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.662+0000 7fca4b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fca540ce520 0x7fca540ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37830/0 (socket says 172.21.15.66:37830) 2024-03-30T00:46:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.662+0000 7fca4b400640 1 -- 172.21.15.66:0/1873821104 learned_addr learned my addr 172.21.15.66:0/1873821104 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.662+0000 7fca53e00640 1 --2- 172.21.15.66:0/1873821104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca5415a640 0x7fca5406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.662+0000 7fca4b400640 1 -- 172.21.15.66:0/1873821104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca5415a640 msgr2=0x7fca5406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.662+0000 7fca4b400640 1 --2- 172.21.15.66:0/1873821104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca5415a640 0x7fca5406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.662+0000 7fca4b400640 1 -- 172.21.15.66:0/1873821104 >> v1:172.21.15.156:6789/0 conn(0x7fca540cabb0 legacy=0x7fca540cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.662+0000 7fca4b400640 1 -- 172.21.15.66:0/1873821104 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca540cee60 con 0x7fca540ce520 2024-03-30T00:46:00.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.662+0000 7fca4b400640 1 --2- 172.21.15.66:0/1873821104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca540ce520 0x7fca540ce920 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009870 tx=0x7fca3c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a685d954a0f4ced server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:00.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.662+0000 7fca53400640 1 -- 172.21.15.66:0/1873821104 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0ab650 con 0x7fca540ce520 2024-03-30T00:46:00.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.662+0000 7fca53400640 1 -- 172.21.15.66:0/1873821104 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c0ab7b0 con 0x7fca540ce520 2024-03-30T00:46:00.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.662+0000 7fca53400640 1 -- 172.21.15.66:0/1873821104 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0abb40 con 0x7fca540ce520 2024-03-30T00:46:00.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 -- 172.21.15.66:0/1873821104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca540ce520 msgr2=0x7fca540ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 --2- 172.21.15.66:0/1873821104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca540ce520 0x7fca540ce920 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009870 tx=0x7fca3c01c560 comp rx=0 tx=0).stop 2024-03-30T00:46:00.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 -- 172.21.15.66:0/1873821104 shutdown_connections 2024-03-30T00:46:00.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 --2- 172.21.15.66:0/1873821104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca540ce520 0x7fca540ce920 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 --2- 172.21.15.66:0/1873821104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca5415a640 0x7fca5406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 -- 172.21.15.66:0/1873821104 >> 172.21.15.66:0/1873821104 conn(0x7fca54069f60 msgr2=0x7fca5406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:00.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 -- 172.21.15.66:0/1873821104 shutdown_connections 2024-03-30T00:46:00.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 -- 172.21.15.66:0/1873821104 wait complete. 2024-03-30T00:46:00.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 Processor -- start 2024-03-30T00:46:00.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 -- start start 2024-03-30T00:46:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca540cabb0 0x7fca54155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca540ce520 0x7fca54097960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca5415a640 0x7fca540ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca541582d0 con 0x7fca540ce520 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca541587c0 con 0x7fca540cabb0 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca5aa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca540b08f0 con 0x7fca5415a640 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca4b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca540ce520 0x7fca54097960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca4b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca540ce520 0x7fca54097960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37842/0 (socket says 172.21.15.66:37842) 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca4b400640 1 -- 172.21.15.66:0/1577403220 learned_addr learned my addr 172.21.15.66:0/1577403220 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca53e00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca540cabb0 0x7fca54155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca58a00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca5415a640 0x7fca540ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca4b400640 1 -- 172.21.15.66:0/1577403220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca5415a640 msgr2=0x7fca540ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca4b400640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca5415a640 0x7fca540ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca4b400640 1 -- 172.21.15.66:0/1577403220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca540cabb0 msgr2=0x7fca54155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca4b400640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca540cabb0 0x7fca54155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca4b400640 1 -- 172.21.15.66:0/1577403220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0094d0 con 0x7fca540ce520 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca53e00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca540cabb0 0x7fca54155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:00.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.666+0000 7fca4b400640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca540ce520 0x7fca54097960 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fca3c01ac10 tx=0x7fca3c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:00.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.670+0000 7fca50c00640 1 -- 172.21.15.66:0/1577403220 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0b4800 con 0x7fca540ce520 2024-03-30T00:46:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.670+0000 7fca50c00640 1 -- 172.21.15.66:0/1577403220 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c0b4980 con 0x7fca540ce520 2024-03-30T00:46:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.670+0000 7fca50c00640 1 -- 172.21.15.66:0/1577403220 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0b4d30 con 0x7fca540ce520 2024-03-30T00:46:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.670+0000 7fca5aa00640 1 -- 172.21.15.66:0/1577403220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca54097ea0 con 0x7fca540ce520 2024-03-30T00:46:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.670+0000 7fca5aa00640 1 -- 172.21.15.66:0/1577403220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca54153af0 con 0x7fca540ce520 2024-03-30T00:46:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.670+0000 7fca5aa00640 1 -- 172.21.15.66:0/1577403220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca54152e60 con 0x7fca540ce520 2024-03-30T00:46:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.670+0000 7fca50c00640 1 -- 172.21.15.66:0/1577403220 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fca3c0c7440 con 0x7fca540ce520 2024-03-30T00:46:00.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.670+0000 7fca50c00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca30066690 0x7fca30068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.670+0000 7fca50c00640 1 -- 172.21.15.66:0/1577403220 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fca3c11eff0 con 0x7fca540ce520 2024-03-30T00:46:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.670+0000 7fca53e00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca30066690 0x7fca30068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:00.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.670+0000 7fca53e00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca30066690 0x7fca30068b50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fca44009b10 tx=0x7fca44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:00.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.674+0000 7fca50c00640 1 -- 172.21.15.66:0/1577403220 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fca3c0ec1f0 con 0x7fca540ce520 2024-03-30T00:46:00.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 -- 172.21.15.66:0/1577403220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca54066c30 con 0x7fca540ce520 2024-03-30T00:46:00.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca50c00640 1 -- 172.21.15.66:0/1577403220 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca3c0ef810 con 0x7fca540ce520 2024-03-30T00:46:00.845 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:46:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 -- 172.21.15.66:0/1577403220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca30066690 msgr2=0x7fca30068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca30066690 0x7fca30068b50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fca44009b10 tx=0x7fca44008040 comp rx=0 tx=0).stop 2024-03-30T00:46:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 -- 172.21.15.66:0/1577403220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca540ce520 msgr2=0x7fca54097960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca540ce520 0x7fca54097960 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fca3c01ac10 tx=0x7fca3c01ada0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 -- 172.21.15.66:0/1577403220 shutdown_connections 2024-03-30T00:46:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca5415a640 0x7fca540ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fca30066690 0x7fca30068b50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca540ce520 0x7fca54097960 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 --2- 172.21.15.66:0/1577403220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca540cabb0 0x7fca54155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:00.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 -- 172.21.15.66:0/1577403220 >> 172.21.15.66:0/1577403220 conn(0x7fca54069f60 msgr2=0x7fca541579b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:00.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 -- 172.21.15.66:0/1577403220 shutdown_connections 2024-03-30T00:46:00.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:00.838+0000 7fca5aa00640 1 -- 172.21.15.66:0/1577403220 wait complete. 2024-03-30T00:46:00.866 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:46:01.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcde0c00640 1 Processor -- start 2024-03-30T00:46:01.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcde0c00640 1 -- start start 2024-03-30T00:46:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcde0c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcddc0ce780 0x7fcddc0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcde0c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 0x7fcddc15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcde0c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc0cbec0 con 0x7fcddc15d360 2024-03-30T00:46:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcde0c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc0cce50 con 0x7fcddc0ce780 2024-03-30T00:46:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcde0c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc15d970 con 0x7fcddc0ceb60 2024-03-30T00:46:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcdd9e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcddc0ce780 0x7fcddc0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcdda800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fcddc15d360 0x7fcddc0cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52908/0 (socket says 172.21.15.66:52908) 2024-03-30T00:46:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcdda800640 1 -- 172.21.15.66:0/1154868542 learned_addr learned my addr 172.21.15.66:0/1154868542 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcdd3e00640 1 --2- 172.21.15.66:0/1154868542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 0x7fcddc15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcdd9400640 1 -- 172.21.15.66:0/1154868542 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117440898 0 0) 0x7fcddc0cbec0 con 0x7fcddc15d360 2024-03-30T00:46:01.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcdd9e00640 1 -- 172.21.15.66:0/1154868542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 msgr2=0x7fcddc15ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:01.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcdd9e00640 1 --2- 172.21.15.66:0/1154868542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 0x7fcddc15ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcdd9e00640 1 -- 172.21.15.66:0/1154868542 >> v1:172.21.15.66:6789/0 conn(0x7fcddc15d360 legacy=0x7fcddc0cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcdd9e00640 1 -- 172.21.15.66:0/1154868542 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc15dc50 con 0x7fcddc0ce780 2024-03-30T00:46:01.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.002+0000 7fcdd3e00640 1 --2- 172.21.15.66:0/1154868542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 0x7fcddc15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:01.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcdd9e00640 1 --2- 172.21.15.66:0/1154868542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcddc0ce780 0x7fcddc0cc910 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0097c0 tx=0x7fcdcc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d370e58ee41df1b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:01.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcdd9400640 1 -- 172.21.15.66:0/1154868542 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0ab600 con 0x7fcddc0ce780 2024-03-30T00:46:01.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcdd9400640 1 -- 172.21.15.66:0/1154868542 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc0a7050 con 0x7fcddc0ce780 2024-03-30T00:46:01.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcdd9400640 1 -- 172.21.15.66:0/1154868542 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0ab980 con 0x7fcddc0ce780 2024-03-30T00:46:01.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcde0c00640 1 -- 172.21.15.66:0/1154868542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcddc0ce780 msgr2=0x7fcddc0cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcde0c00640 1 --2- 172.21.15.66:0/1154868542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcddc0ce780 0x7fcddc0cc910 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0097c0 tx=0x7fcdcc01c3e0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcde0c00640 1 -- 172.21.15.66:0/1154868542 shutdown_connections 2024-03-30T00:46:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcde0c00640 1 --2- 172.21.15.66:0/1154868542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 0x7fcddc15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcde0c00640 1 --2- 172.21.15.66:0/1154868542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcddc0ce780 0x7fcddc0cc910 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcde0c00640 1 -- 172.21.15.66:0/1154868542 >> 172.21.15.66:0/1154868542 conn(0x7fcddc069f60 msgr2=0x7fcddc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcde0c00640 1 -- 172.21.15.66:0/1154868542 shutdown_connections 2024-03-30T00:46:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.006+0000 7fcde0c00640 1 -- 172.21.15.66:0/1154868542 wait complete. 2024-03-30T00:46:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcde0c00640 1 Processor -- start 2024-03-30T00:46:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcde0c00640 1 -- start start 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcde0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcddc0ce780 0x7fcddc0c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcde0c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 0x7fcddc097080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcde0c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcddc15d360 0x7fcddc070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcde0c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0c53c0 con 0x7fcddc0ce780 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcde0c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0c5f40 con 0x7fcddc15d360 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcde0c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0c5610 con 0x7fcddc0ceb60 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcddc0ce780 0x7fcddc0c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcddc0ce780 0x7fcddc0c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37856/0 (socket says 172.21.15.66:37856) 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd9e00640 1 -- 172.21.15.66:0/557503129 learned_addr learned my addr 172.21.15.66:0/557503129 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdda800640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcddc15d360 0x7fcddc070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd3e00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 0x7fcddc097080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd9e00640 1 -- 172.21.15.66:0/557503129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 msgr2=0x7fcddc097080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd9e00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 0x7fcddc097080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd9e00640 1 -- 172.21.15.66:0/557503129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcddc15d360 msgr2=0x7fcddc070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd9e00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcddc15d360 0x7fcddc070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd9e00640 1 -- 172.21.15.66:0/557503129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc009420 con 0x7fcddc0ce780 2024-03-30T00:46:01.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd3e00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 0x7fcddc097080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:01.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdda800640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcddc15d360 0x7fcddc070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:01.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd9e00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcddc0ce780 0x7fcddc0c4780 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc01a010 tx=0x7fcdcc01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:01.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd2000640 1 -- 172.21.15.66:0/557503129 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc006e20 con 0x7fcddc0ce780 2024-03-30T00:46:01.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd2000640 1 -- 172.21.15.66:0/557503129 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc004030 con 0x7fcddc0ce780 2024-03-30T00:46:01.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcdd2000640 1 -- 172.21.15.66:0/557503129 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0043a0 con 0x7fcddc0ce780 2024-03-30T00:46:01.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcde0c00640 1 -- 172.21.15.66:0/557503129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc0c58f0 con 0x7fcddc0ce780 2024-03-30T00:46:01.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcde0c00640 1 -- 172.21.15.66:0/557503129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc0c4cc0 con 0x7fcddc0ce780 2024-03-30T00:46:01.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.010+0000 7fcde0c00640 1 -- 172.21.15.66:0/557503129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc066c30 con 0x7fcddc0ce780 2024-03-30T00:46:01.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.014+0000 7fcdd2000640 1 -- 172.21.15.66:0/557503129 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcdcc003620 con 0x7fcddc0ce780 2024-03-30T00:46:01.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.014+0000 7fcdd2000640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcdb80666c0 0x7fcdb8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.014+0000 7fcdd2000640 1 -- 172.21.15.66:0/557503129 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fcdcc11ea20 con 0x7fcddc0ce780 2024-03-30T00:46:01.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.014+0000 7fcdd3e00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcdb80666c0 0x7fcdb8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.014+0000 7fcdd3e00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcdb80666c0 0x7fcdb8068b80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcddc067400 tx=0x7fcdc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:01.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.018+0000 7fcdd2000640 1 -- 172.21.15.66:0/557503129 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcdcc0ebda0 con 0x7fcddc0ce780 2024-03-30T00:46:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.178+0000 7fcde0c00640 1 -- 172.21.15.66:0/557503129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcddc06ca50 con 0x7fcddc0ce780 2024-03-30T00:46:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.178+0000 7fcdd2000640 1 -- 172.21.15.66:0/557503129 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcdcc0b0020 con 0x7fcddc0ce780 2024-03-30T00:46:01.187 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:46:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 -- 172.21.15.66:0/557503129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcdb80666c0 msgr2=0x7fcdb8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcdb80666c0 0x7fcdb8068b80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcddc067400 tx=0x7fcdc4008040 comp rx=0 tx=0).stop 2024-03-30T00:46:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 -- 172.21.15.66:0/557503129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcddc0ce780 msgr2=0x7fcddc0c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcddc0ce780 0x7fcddc0c4780 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc01a010 tx=0x7fcdcc01ac20 comp rx=0 tx=0).stop 2024-03-30T00:46:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 -- 172.21.15.66:0/557503129 shutdown_connections 2024-03-30T00:46:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcddc15d360 0x7fcddc070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcddc0ceb60 0x7fcddc097080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcdb80666c0 0x7fcdb8068b80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 --2- 172.21.15.66:0/557503129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcddc0ce780 0x7fcddc0c4780 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 -- 172.21.15.66:0/557503129 >> 172.21.15.66:0/557503129 conn(0x7fcddc069f60 msgr2=0x7fcddc06d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:01.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 -- 172.21.15.66:0/557503129 shutdown_connections 2024-03-30T00:46:01.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.182+0000 7fcde0c00640 1 -- 172.21.15.66:0/557503129 wait complete. 2024-03-30T00:46:01.208 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:46:01.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.342+0000 7efe28a00640 1 Processor -- start 2024-03-30T00:46:01.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.342+0000 7efe28a00640 1 -- start start 2024-03-30T00:46:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.342+0000 7efe28a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe240b3170 0x7efe240b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.342+0000 7efe28a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe240cb5b0 con 0x7efe2415a720 2024-03-30T00:46:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.342+0000 7efe28a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe240cb080 con 0x7efe240d07c0 2024-03-30T00:46:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.342+0000 7efe28a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe240cfb00 con 0x7efe240b3170 2024-03-30T00:46:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.342+0000 7efe22800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efe2415a720 0x7efe240ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52920/0 (socket says 172.21.15.66:52920) 2024-03-30T00:46:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.342+0000 7efe22800640 1 -- 172.21.15.66:0/1828075007 learned_addr learned my addr 172.21.15.66:0/1828075007 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.342+0000 7efe1be00640 1 --2- 172.21.15.66:0/1828075007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe240b3170 0x7efe240b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.342+0000 7efe21e00640 1 -- 172.21.15.66:0/1828075007 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950118981 0 0) 0x7efe14001120 con 0x7efe2415a720 2024-03-30T00:46:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.342+0000 7efe21e00640 1 -- 172.21.15.66:0/1828075007 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe08003530 con 0x7efe2415a720 2024-03-30T00:46:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.346+0000 7efe21e00640 1 -- 172.21.15.66:0/1828075007 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180835303 0 0) 0x7efe240cb080 con 0x7efe240d07c0 2024-03-30T00:46:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.346+0000 7efe21e00640 1 -- 172.21.15.66:0/1828075007 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe14001120 con 0x7efe240d07c0 2024-03-30T00:46:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.346+0000 7efe1be00640 1 -- 172.21.15.66:0/1828075007 >> v1:172.21.15.156:6789/0 conn(0x7efe240d07c0 legacy=0x7efe240ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.346+0000 7efe1be00640 1 -- 172.21.15.66:0/1828075007 >> v1:172.21.15.66:6789/0 conn(0x7efe2415a720 legacy=0x7efe240ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.346+0000 7efe1be00640 1 -- 172.21.15.66:0/1828075007 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2415ab00 con 0x7efe240b3170 2024-03-30T00:46:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.346+0000 7efe1be00640 1 --2- 172.21.15.66:0/1828075007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe240b3170 0x7efe240b3570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0097f0 tx=0x7efe0c01c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355bc150e0730e72 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.346+0000 7efe21e00640 1 -- 172.21.15.66:0/1828075007 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14001810 con 0x7efe240b3170 2024-03-30T00:46:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.346+0000 7efe21e00640 1 -- 172.21.15.66:0/1828075007 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c0a7050 con 0x7efe240b3170 2024-03-30T00:46:01.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.346+0000 7efe21e00640 1 -- 172.21.15.66:0/1828075007 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0ab620 con 0x7efe240b3170 2024-03-30T00:46:01.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.346+0000 7efe28a00640 1 -- 172.21.15.66:0/1828075007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe240b3170 msgr2=0x7efe240b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.346+0000 7efe28a00640 1 --2- 172.21.15.66:0/1828075007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe240b3170 0x7efe240b3570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0097f0 tx=0x7efe0c01c320 comp rx=0 tx=0).stop 2024-03-30T00:46:01.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 -- 172.21.15.66:0/1828075007 shutdown_connections 2024-03-30T00:46:01.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 --2- 172.21.15.66:0/1828075007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe240b3170 0x7efe240b3570 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 -- 172.21.15.66:0/1828075007 >> 172.21.15.66:0/1828075007 conn(0x7efe24069d90 msgr2=0x7efe2406a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:01.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 -- 172.21.15.66:0/1828075007 shutdown_connections 2024-03-30T00:46:01.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 -- 172.21.15.66:0/1828075007 wait complete. 2024-03-30T00:46:01.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 Processor -- start 2024-03-30T00:46:01.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 -- start start 2024-03-30T00:46:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe240b3170 0x7efe241f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe240d07c0 0x7efe24148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe2415a720 0x7efe2414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2414c200 con 0x7efe240b3170 2024-03-30T00:46:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe241f6fa0 con 0x7efe240d07c0 2024-03-30T00:46:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2414b450 con 0x7efe2415a720 2024-03-30T00:46:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe22800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe240b3170 0x7efe241f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe22800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe240b3170 0x7efe241f7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37866/0 (socket says 172.21.15.66:37866) 2024-03-30T00:46:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe22800640 1 -- 172.21.15.66:0/2324539633 learned_addr learned my addr 172.21.15.66:0/2324539633 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:01.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe23200640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe2415a720 0x7efe2414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe1be00640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe240d07c0 0x7efe24148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe22800640 1 -- 172.21.15.66:0/2324539633 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe2415a720 msgr2=0x7efe2414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe22800640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe2415a720 0x7efe2414bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe22800640 1 -- 172.21.15.66:0/2324539633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe240d07c0 msgr2=0x7efe24148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe22800640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe240d07c0 0x7efe24148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe22800640 1 -- 172.21.15.66:0/2324539633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c009420 con 0x7efe240b3170 2024-03-30T00:46:01.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe22800640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe240b3170 0x7efe241f7a40 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efe1400caa0 tx=0x7efe1400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:01.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe1aa00640 1 -- 172.21.15.66:0/2324539633 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140075a0 con 0x7efe240b3170 2024-03-30T00:46:01.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe1aa00640 1 -- 172.21.15.66:0/2324539633 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe14007700 con 0x7efe240b3170 2024-03-30T00:46:01.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 -- 172.21.15.66:0/2324539633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe241f7240 con 0x7efe240b3170 2024-03-30T00:46:01.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe1aa00640 1 -- 172.21.15.66:0/2324539633 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14004a10 con 0x7efe240b3170 2024-03-30T00:46:01.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.350+0000 7efe28a00640 1 -- 172.21.15.66:0/2324539633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2414b730 con 0x7efe240b3170 2024-03-30T00:46:01.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.354+0000 7efe28a00640 1 -- 172.21.15.66:0/2324539633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe24073e00 con 0x7efe240b3170 2024-03-30T00:46:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.354+0000 7efe1aa00640 1 -- 172.21.15.66:0/2324539633 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efe14004b70 con 0x7efe240b3170 2024-03-30T00:46:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.354+0000 7efe1aa00640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe00066690 0x7efe00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.354+0000 7efe1aa00640 1 -- 172.21.15.66:0/2324539633 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7efe14083af0 con 0x7efe240b3170 2024-03-30T00:46:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.354+0000 7efe1be00640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe00066690 0x7efe00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.354+0000 7efe1be00640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe00066690 0x7efe00068b50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0ad000 tx=0x7efe0c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:01.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.358+0000 7efe1aa00640 1 -- 172.21.15.66:0/2324539633 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efe14050ce0 con 0x7efe240b3170 2024-03-30T00:46:01.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.530+0000 7efe28a00640 1 -- 172.21.15.66:0/2324539633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe240cb5b0 con 0x7efe240b3170 2024-03-30T00:46:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe1aa00640 1 -- 172.21.15.66:0/2324539633 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe240cb5b0 con 0x7efe240b3170 2024-03-30T00:46:01.541 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:46:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 -- 172.21.15.66:0/2324539633 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe00066690 msgr2=0x7efe00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe00066690 0x7efe00068b50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0ad000 tx=0x7efe0c0a6000 comp rx=0 tx=0).stop 2024-03-30T00:46:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 -- 172.21.15.66:0/2324539633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe240b3170 msgr2=0x7efe241f7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe240b3170 0x7efe241f7a40 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efe1400caa0 tx=0x7efe1400cf60 comp rx=0 tx=0).stop 2024-03-30T00:46:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 -- 172.21.15.66:0/2324539633 shutdown_connections 2024-03-30T00:46:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe2415a720 0x7efe2414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe240d07c0 0x7efe24148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efe00066690 0x7efe00068b50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 --2- 172.21.15.66:0/2324539633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe240b3170 0x7efe241f7a40 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 -- 172.21.15.66:0/2324539633 >> 172.21.15.66:0/2324539633 conn(0x7efe24069d90 msgr2=0x7efe240b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 -- 172.21.15.66:0/2324539633 shutdown_connections 2024-03-30T00:46:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.534+0000 7efe28a00640 1 -- 172.21.15.66:0/2324539633 wait complete. 2024-03-30T00:46:01.562 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:46:01.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b7bc00640 1 Processor -- start 2024-03-30T00:46:01.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b7bc00640 1 -- start start 2024-03-30T00:46:01.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b7bc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b7415a640 0x7f9b7406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b7bc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9b740ce520 0x7f9b740ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b7bc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b7406d070 con 0x7f9b740cabb0 2024-03-30T00:46:01.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b7bc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b740cbba0 con 0x7f9b740ce520 2024-03-30T00:46:01.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b7bc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7406e2c0 con 0x7f9b7415a640 2024-03-30T00:46:01.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b6be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9b740ce520 0x7f9b740ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b6be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9b740ce520 0x7f9b740ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37874/0 (socket says 172.21.15.66:37874) 2024-03-30T00:46:01.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b6be00640 1 -- 172.21.15.66:0/2760030742 learned_addr learned my addr 172.21.15.66:0/2760030742 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:01.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b79200640 1 --2- 172.21.15.66:0/2760030742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b7415a640 0x7f9b7406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b6be00640 1 -- 172.21.15.66:0/2760030742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b7415a640 msgr2=0x7f9b7406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:01.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b6be00640 1 --2- 172.21.15.66:0/2760030742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b7415a640 0x7f9b7406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b6be00640 1 -- 172.21.15.66:0/2760030742 >> v1:172.21.15.156:6789/0 conn(0x7f9b740cabb0 legacy=0x7f9b740cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b6be00640 1 -- 172.21.15.66:0/2760030742 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b740cee60 con 0x7f9b740ce520 2024-03-30T00:46:01.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.694+0000 7f9b79200640 1 --2- 172.21.15.66:0/2760030742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b7415a640 0x7f9b7406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:01.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b6be00640 1 --2- 172.21.15.66:0/2760030742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b740ce520 0x7f9b740ce920 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009870 tx=0x7f9b5c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826bd7a062858592 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:01.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b73e00640 1 -- 172.21.15.66:0/2760030742 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c0ab650 con 0x7f9b740ce520 2024-03-30T00:46:01.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b73e00640 1 -- 172.21.15.66:0/2760030742 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5c0ab7b0 con 0x7f9b740ce520 2024-03-30T00:46:01.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b73e00640 1 -- 172.21.15.66:0/2760030742 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c0abb40 con 0x7f9b740ce520 2024-03-30T00:46:01.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b7bc00640 1 -- 172.21.15.66:0/2760030742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b740ce520 msgr2=0x7f9b740ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b7bc00640 1 --2- 172.21.15.66:0/2760030742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b740ce520 0x7f9b740ce920 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009870 tx=0x7f9b5c01c560 comp rx=0 tx=0).stop 2024-03-30T00:46:01.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b7bc00640 1 -- 172.21.15.66:0/2760030742 shutdown_connections 2024-03-30T00:46:01.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b7bc00640 1 --2- 172.21.15.66:0/2760030742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b740ce520 0x7f9b740ce920 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b7bc00640 1 --2- 172.21.15.66:0/2760030742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b7415a640 0x7f9b7406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b7bc00640 1 -- 172.21.15.66:0/2760030742 >> 172.21.15.66:0/2760030742 conn(0x7f9b74069f60 msgr2=0x7f9b7406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b7bc00640 1 -- 172.21.15.66:0/2760030742 shutdown_connections 2024-03-30T00:46:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.698+0000 7f9b7bc00640 1 -- 172.21.15.66:0/2760030742 wait complete. 2024-03-30T00:46:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b7bc00640 1 Processor -- start 2024-03-30T00:46:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b7bc00640 1 -- start start 2024-03-30T00:46:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b7bc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b740cabb0 0x7f9b74158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b7bc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b740ce520 0x7f9b740c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b7bc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7415a640 0x7f9b740c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b7bc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b74164280 con 0x7f9b7415a640 2024-03-30T00:46:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b7bc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b741f4cf0 con 0x7f9b740cabb0 2024-03-30T00:46:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b7bc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b741f9210 con 0x7f9b740ce520 2024-03-30T00:46:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b79c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7415a640 0x7f9b740c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b79c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7415a640 0x7f9b740c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37886/0 (socket says 172.21.15.66:37886) 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b79c00640 1 -- 172.21.15.66:0/4267955113 learned_addr learned my addr 172.21.15.66:0/4267955113 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b79200640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b740cabb0 0x7f9b74158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b79c00640 1 -- 172.21.15.66:0/4267955113 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b740ce520 msgr2=0x7f9b740c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b6be00640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b740ce520 0x7f9b740c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b79c00640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b740ce520 0x7f9b740c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b79c00640 1 -- 172.21.15.66:0/4267955113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b740cabb0 msgr2=0x7f9b74158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b79c00640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b740cabb0 0x7f9b74158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b79c00640 1 -- 172.21.15.66:0/4267955113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c0094d0 con 0x7f9b7415a640 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b79c00640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7415a640 0x7f9b740c9e70 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c770 tx=0x7f9b6c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b71600640 1 -- 172.21.15.66:0/4267955113 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c003b80 con 0x7f9b7415a640 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b71600640 1 -- 172.21.15.66:0/4267955113 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c003ce0 con 0x7f9b7415a640 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b7bc00640 1 -- 172.21.15.66:0/4267955113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b741f8ce0 con 0x7f9b7415a640 2024-03-30T00:46:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b71600640 1 -- 172.21.15.66:0/4267955113 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c00aa90 con 0x7f9b7415a640 2024-03-30T00:46:01.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.702+0000 7f9b7bc00640 1 -- 172.21.15.66:0/4267955113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b741f4ef0 con 0x7f9b7415a640 2024-03-30T00:46:01.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.706+0000 7f9b7bc00640 1 -- 172.21.15.66:0/4267955113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b74072c00 con 0x7f9b7415a640 2024-03-30T00:46:01.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.706+0000 7f9b71600640 1 -- 172.21.15.66:0/4267955113 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9b6c00abf0 con 0x7f9b7415a640 2024-03-30T00:46:01.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.706+0000 7f9b71600640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b500666c0 0x7f9b50068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:01.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.706+0000 7f9b71600640 1 -- 172.21.15.66:0/4267955113 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f9b6c084160 con 0x7f9b7415a640 2024-03-30T00:46:01.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.706+0000 7f9b79200640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b500666c0 0x7f9b50068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:01.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.706+0000 7f9b79200640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b500666c0 0x7f9b50068b80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9b64006fd0 tx=0x7f9b64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:01.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.710+0000 7f9b71600640 1 -- 172.21.15.66:0/4267955113 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9b6c051320 con 0x7f9b7415a640 2024-03-30T00:46:01.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.906+0000 7f9b7bc00640 1 -- 172.21.15.66:0/4267955113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b74066c30 con 0x7f9b7415a640 2024-03-30T00:46:01.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.906+0000 7f9b71600640 1 -- 172.21.15.66:0/4267955113 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f9b6c054940 con 0x7f9b7415a640 2024-03-30T00:46:01.916 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:01.916 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/254323195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":254323195},{"type":"v1","addr":"172.21.15.156:6835","nonce":254323195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:45:30.650828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":23,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:46:01.916 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 51 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 -- 172.21.15.66:0/4267955113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b500666c0 msgr2=0x7f9b50068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b500666c0 0x7f9b50068b80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9b64006fd0 tx=0x7f9b64008040 comp rx=0 tx=0).stop 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 -- 172.21.15.66:0/4267955113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7415a640 msgr2=0x7f9b740c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7415a640 0x7f9b740c9e70 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c770 tx=0x7f9b6c00cc30 comp rx=0 tx=0).stop 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 -- 172.21.15.66:0/4267955113 shutdown_connections 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9b500666c0 0x7f9b50068b80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b7415a640 0x7f9b740c9e70 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9b740ce520 0x7f9b740c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 --2- 172.21.15.66:0/4267955113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b740cabb0 0x7f9b74158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 -- 172.21.15.66:0/4267955113 >> 172.21.15.66:0/4267955113 conn(0x7f9b74069f60 msgr2=0x7f9b740ac6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 -- 172.21.15.66:0/4267955113 shutdown_connections 2024-03-30T00:46:01.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:01.910+0000 7f9b7bc00640 1 -- 172.21.15.66:0/4267955113 wait complete. 2024-03-30T00:46:01.937 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:46:02.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec57e00640 1 Processor -- start 2024-03-30T00:46:02.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec57e00640 1 -- start start 2024-03-30T00:46:02.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec57e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500cebb0 0x7fec5006e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec57e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec500cc620 con 0x7fec5006a4a0 2024-03-30T00:46:02.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec57e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec500cbd30 con 0x7fec500cb7d0 2024-03-30T00:46:02.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec57e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec500cbfd0 con 0x7fec500cebb0 2024-03-30T00:46:02.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec55e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fec5006a4a0 0x7fec50160730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52926/0 (socket says 172.21.15.66:52926) 2024-03-30T00:46:02.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec55e00640 1 -- 172.21.15.66:0/3956548787 learned_addr learned my addr 172.21.15.66:0/3956548787 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:02.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec55400640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500cebb0 0x7fec5006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807592029 0 0) 0x7fec500cc620 con 0x7fec5006a4a0 2024-03-30T00:46:02.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec38003530 con 0x7fec5006a4a0 2024-03-30T00:46:02.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3770269929 0 0) 0x7fec38003530 con 0x7fec5006a4a0 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500cebb0 msgr2=0x7fec5006e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500cebb0 0x7fec5006e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 >> v1:172.21.15.156:6789/0 conn(0x7fec500cb7d0 legacy=0x7fec50069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec500cc2b0 con 0x7fec5006a4a0 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fec48001a00 con 0x7fec5006a4a0 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 >> v1:172.21.15.66:6789/0 conn(0x7fec5006a4a0 legacy=0x7fec50160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec380039b0 0x7fec38003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500cebb0 0x7fec380047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 0x7fec38005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec500cbfd0 con 0x7fec38004d10 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3800ba40 con 0x7fec380039b0 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3800bbb0 con 0x7fec500cebb0 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec55e00640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 0x7fec38005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec55400640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec380039b0 0x7fec38003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec54a00640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500cebb0 0x7fec380047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec55e00640 1 -- 172.21.15.66:0/3956548787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500cebb0 msgr2=0x7fec380047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:02.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec55e00640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500cebb0 0x7fec380047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec55e00640 1 -- 172.21.15.66:0/3956548787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec380039b0 msgr2=0x7fec38003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:02.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec55e00640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec380039b0 0x7fec38003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.070+0000 7fec55e00640 1 -- 172.21.15.66:0/3956548787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec38003530 con 0x7fec38004d10 2024-03-30T00:46:02.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.074+0000 7fec55e00640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 0x7fec38005190 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fec4800a810 tx=0x7fec4801fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e79f983d9f15f21 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:02.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.074+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec480aa620 con 0x7fec38004d10 2024-03-30T00:46:02.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.074+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec480aa780 con 0x7fec38004d10 2024-03-30T00:46:02.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.074+0000 7fec4fe00640 1 -- 172.21.15.66:0/3956548787 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec480aab30 con 0x7fec38004d10 2024-03-30T00:46:02.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.074+0000 7fec57e00640 1 -- 172.21.15.66:0/3956548787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 msgr2=0x7fec38005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.074+0000 7fec57e00640 1 --2- 172.21.15.66:0/3956548787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 0x7fec38005190 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fec4800a810 tx=0x7fec4801fa80 comp rx=0 tx=0).stop 2024-03-30T00:46:02.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.074+0000 7fec55e00640 1 -- 172.21.15.66:0/3956548787 reap_dead start 2024-03-30T00:46:02.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.074+0000 7fec57e00640 1 -- 172.21.15.66:0/3956548787 shutdown_connections 2024-03-30T00:46:02.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.074+0000 7fec57e00640 1 -- 172.21.15.66:0/3956548787 >> 172.21.15.66:0/3956548787 conn(0x7fec50098a20 msgr2=0x7fec500b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:02.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.074+0000 7fec57e00640 1 -- 172.21.15.66:0/3956548787 shutdown_connections 2024-03-30T00:46:02.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.074+0000 7fec57e00640 1 -- 172.21.15.66:0/3956548787 wait complete. 2024-03-30T00:46:02.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec57e00640 1 Processor -- start 2024-03-30T00:46:02.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec57e00640 1 -- start start 2024-03-30T00:46:02.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec57e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 0x7fec500b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec57e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500acd60 0x7fec500ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec57e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec5014b5b0 0x7fec5014ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec57e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec50149bd0 con 0x7fec38004d10 2024-03-30T00:46:02.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec57e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec500ad720 con 0x7fec5014b5b0 2024-03-30T00:46:02.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec57e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec500b5290 con 0x7fec500acd60 2024-03-30T00:46:02.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec55400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 0x7fec500b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec55400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 0x7fec500b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37898/0 (socket says 172.21.15.66:37898) 2024-03-30T00:46:02.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec55400640 1 -- 172.21.15.66:0/4207095714 learned_addr learned my addr 172.21.15.66:0/4207095714 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:02.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec55400640 1 -- 172.21.15.66:0/4207095714 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500acd60 msgr2=0x7fec500ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec55400640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500acd60 0x7fec500ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec55400640 1 -- 172.21.15.66:0/4207095714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec5014b5b0 msgr2=0x7fec5014ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec55400640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec5014b5b0 0x7fec5014ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec55400640 1 -- 172.21.15.66:0/4207095714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec500ccdb0 con 0x7fec38004d10 2024-03-30T00:46:02.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec55400640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 0x7fec500b4aa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fec440119d0 tx=0x7fec44011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:02.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec4d600640 1 -- 172.21.15.66:0/4207095714 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4400fad0 con 0x7fec38004d10 2024-03-30T00:46:02.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec4d600640 1 -- 172.21.15.66:0/4207095714 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4400fc30 con 0x7fec38004d10 2024-03-30T00:46:02.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec4d600640 1 -- 172.21.15.66:0/4207095714 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec44014440 con 0x7fec38004d10 2024-03-30T00:46:02.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec57e00640 1 -- 172.21.15.66:0/4207095714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec500b4fe0 con 0x7fec38004d10 2024-03-30T00:46:02.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.078+0000 7fec57e00640 1 -- 172.21.15.66:0/4207095714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec50149510 con 0x7fec38004d10 2024-03-30T00:46:02.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.082+0000 7fec57e00640 1 -- 172.21.15.66:0/4207095714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec50066c30 con 0x7fec38004d10 2024-03-30T00:46:02.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.082+0000 7fec4d600640 1 -- 172.21.15.66:0/4207095714 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fec440145a0 con 0x7fec38004d10 2024-03-30T00:46:02.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.082+0000 7fec4d600640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec300666c0 0x7fec30068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.082+0000 7fec4d600640 1 -- 172.21.15.66:0/4207095714 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fec44084bc0 con 0x7fec38004d10 2024-03-30T00:46:02.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.082+0000 7fec54a00640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec300666c0 0x7fec30068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.082+0000 7fec54a00640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec300666c0 0x7fec30068b80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fec500cfd80 tx=0x7fec3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:02.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.086+0000 7fec4d600640 1 -- 172.21.15.66:0/4207095714 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fec44051e40 con 0x7fec38004d10 2024-03-30T00:46:02.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 -- 172.21.15.66:0/4207095714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec5006a8c0 con 0x7fec38004d10 2024-03-30T00:46:02.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec4d600640 1 -- 172.21.15.66:0/4207095714 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16440 (secure 0 0 0) 0x7fec44022020 con 0x7fec38004d10 2024-03-30T00:46:02.258 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:02.258 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":83,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:35.079642+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:44:14.685880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-30T00:44:15.260373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.156:6835/2089872680":"2024-03-31T00:44:35.079618+0000","172.21.15.156:6833/2089872680":"2024-03-31T00:44:35.079618+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:46:02.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 -- 172.21.15.66:0/4207095714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec300666c0 msgr2=0x7fec30068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec300666c0 0x7fec30068b80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fec500cfd80 tx=0x7fec3c009040 comp rx=0 tx=0).stop 2024-03-30T00:46:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 -- 172.21.15.66:0/4207095714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 msgr2=0x7fec500b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 0x7fec500b4aa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fec440119d0 tx=0x7fec44011e90 comp rx=0 tx=0).stop 2024-03-30T00:46:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 -- 172.21.15.66:0/4207095714 shutdown_connections 2024-03-30T00:46:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec5014b5b0 0x7fec5014ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec500acd60 0x7fec500ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec300666c0 0x7fec30068b80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 --2- 172.21.15.66:0/4207095714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec38004d10 0x7fec500b4aa0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 -- 172.21.15.66:0/4207095714 >> 172.21.15.66:0/4207095714 conn(0x7fec50098a20 msgr2=0x7fec500cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 -- 172.21.15.66:0/4207095714 shutdown_connections 2024-03-30T00:46:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.250+0000 7fec57e00640 1 -- 172.21.15.66:0/4207095714 wait complete. 2024-03-30T00:46:02.280 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T00:46:02.280 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:46:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.410+0000 7fdf18e00640 1 Processor -- start 2024-03-30T00:46:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.410+0000 7fdf18e00640 1 -- start start 2024-03-30T00:46:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf18e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdf140ce410 0x7fdf1406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf18e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdf1406dfa0 0x7fdf1415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf18e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf140cd370 con 0x7fdf141506a0 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf18e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf140cd150 con 0x7fdf1406dfa0 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf18e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1406e410 con 0x7fdf140ce410 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf11e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdf141506a0 0x7fdf140cca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52928/0 (socket says 172.21.15.66:52928) 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf11e00640 1 -- 172.21.15.66:0/1985276963 learned_addr learned my addr 172.21.15.66:0/1985276963 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf12800640 1 --2- 172.21.15.66:0/1985276963 >> v2:172.21.15.66:3300/0 conn(0x7fdf1406dfa0 0x7fdf1415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf0be00640 1 --2- 172.21.15.66:0/1985276963 >> v2:172.21.15.156:3300/0 conn(0x7fdf140ce410 0x7fdf1406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf12800640 1 -- 172.21.15.66:0/1985276963 >> v2:172.21.15.156:3300/0 conn(0x7fdf140ce410 msgr2=0x7fdf1406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf12800640 1 --2- 172.21.15.66:0/1985276963 >> v2:172.21.15.156:3300/0 conn(0x7fdf140ce410 0x7fdf1406da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf12800640 1 -- 172.21.15.66:0/1985276963 >> v1:172.21.15.66:6789/0 conn(0x7fdf141506a0 legacy=0x7fdf140cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf12800640 1 -- 172.21.15.66:0/1985276963 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf140cd650 con 0x7fdf1406dfa0 2024-03-30T00:46:02.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf11400640 1 -- 172.21.15.66:0/1985276963 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13723621 0 0) 0x7fdf140cd370 con 0x7fdf141506a0 2024-03-30T00:46:02.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.414+0000 7fdf0be00640 1 --2- 172.21.15.66:0/1985276963 >> v2:172.21.15.156:3300/0 conn(0x7fdf140ce410 0x7fdf1406da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:02.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf12800640 1 --2- 172.21.15.66:0/1985276963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf1406dfa0 0x7fdf1415a6a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fdf1406ff40 tx=0x7fdf0c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d79fac57096f412 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:02.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf11400640 1 -- 172.21.15.66:0/1985276963 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0a5850 con 0x7fdf1406dfa0 2024-03-30T00:46:02.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf11400640 1 -- 172.21.15.66:0/1985276963 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c0a59b0 con 0x7fdf1406dfa0 2024-03-30T00:46:02.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf11400640 1 -- 172.21.15.66:0/1985276963 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0a5d40 con 0x7fdf1406dfa0 2024-03-30T00:46:02.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 -- 172.21.15.66:0/1985276963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf1406dfa0 msgr2=0x7fdf1415a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 --2- 172.21.15.66:0/1985276963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf1406dfa0 0x7fdf1415a6a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fdf1406ff40 tx=0x7fdf0c01c8a0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 -- 172.21.15.66:0/1985276963 shutdown_connections 2024-03-30T00:46:02.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 --2- 172.21.15.66:0/1985276963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf1406dfa0 0x7fdf1415a6a0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 --2- 172.21.15.66:0/1985276963 >> v2:172.21.15.156:3300/0 conn(0x7fdf140ce410 0x7fdf1406da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 -- 172.21.15.66:0/1985276963 >> 172.21.15.66:0/1985276963 conn(0x7fdf14071750 msgr2=0x7fdf14069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:02.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 -- 172.21.15.66:0/1985276963 shutdown_connections 2024-03-30T00:46:02.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 -- 172.21.15.66:0/1985276963 wait complete. 2024-03-30T00:46:02.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 Processor -- start 2024-03-30T00:46:02.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 -- start start 2024-03-30T00:46:02.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf1406dfa0 0x7fdf1414b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf140ce410 0x7fdf14158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf141506a0 0x7fdf14158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1414bdb0 con 0x7fdf141506a0 2024-03-30T00:46:02.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140b2f40 con 0x7fdf1406dfa0 2024-03-30T00:46:02.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf18e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1414b7f0 con 0x7fdf140ce410 2024-03-30T00:46:02.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf12800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf141506a0 0x7fdf14158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf12800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf141506a0 0x7fdf14158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37918/0 (socket says 172.21.15.66:37918) 2024-03-30T00:46:02.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.418+0000 7fdf12800640 1 -- 172.21.15.66:0/3641096220 learned_addr learned my addr 172.21.15.66:0/3641096220 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:02.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf0be00640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf140ce410 0x7fdf14158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf11e00640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf1406dfa0 0x7fdf1414b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf12800640 1 -- 172.21.15.66:0/3641096220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf140ce410 msgr2=0x7fdf14158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf12800640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf140ce410 0x7fdf14158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf12800640 1 -- 172.21.15.66:0/3641096220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf1406dfa0 msgr2=0x7fdf1414b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf12800640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf1406dfa0 0x7fdf1414b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf12800640 1 -- 172.21.15.66:0/3641096220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c009a70 con 0x7fdf141506a0 2024-03-30T00:46:02.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf12800640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf141506a0 0x7fdf14158a60 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0b0040 tx=0x7fdf0c019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:02.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf0a000640 1 -- 172.21.15.66:0/3641096220 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c003ba0 con 0x7fdf141506a0 2024-03-30T00:46:02.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf0a000640 1 -- 172.21.15.66:0/3641096220 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c003d00 con 0x7fdf141506a0 2024-03-30T00:46:02.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf18e00640 1 -- 172.21.15.66:0/3641096220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1414bad0 con 0x7fdf141506a0 2024-03-30T00:46:02.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf0a000640 1 -- 172.21.15.66:0/3641096220 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0ad890 con 0x7fdf141506a0 2024-03-30T00:46:02.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf18e00640 1 -- 172.21.15.66:0/3641096220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf14166c00 con 0x7fdf141506a0 2024-03-30T00:46:02.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf18e00640 1 -- 172.21.15.66:0/3641096220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf140b30b0 con 0x7fdf141506a0 2024-03-30T00:46:02.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf0a000640 1 -- 172.21.15.66:0/3641096220 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdf0c0b4050 con 0x7fdf141506a0 2024-03-30T00:46:02.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf0a000640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdef0066690 0x7fdef0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.422+0000 7fdf0a000640 1 -- 172.21.15.66:0/3641096220 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fdf0c11efd0 con 0x7fdf141506a0 2024-03-30T00:46:02.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.426+0000 7fdf11e00640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdef0066690 0x7fdef0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.426+0000 7fdf11e00640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdef0066690 0x7fdef0068b50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdf04000f70 tx=0x7fdf04007d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:02.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.426+0000 7fdf0a000640 1 -- 172.21.15.66:0/3641096220 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdf0c0ec190 con 0x7fdf141506a0 2024-03-30T00:46:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 -- 172.21.15.66:0/3641096220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf14066c30 con 0x7fdf141506a0 2024-03-30T00:46:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf0a000640 1 -- 172.21.15.66:0/3641096220 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4741 (secure 0 0 0) 0x7fdf0c0b2090 con 0x7fdf141506a0 2024-03-30T00:46:02.634 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:02.634 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/254323195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":254323195},{"type":"v1","addr":"172.21.15.156:6835","nonce":254323195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:45:30.650828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":23,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:46:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 51 2024-03-30T00:46:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 -- 172.21.15.66:0/3641096220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdef0066690 msgr2=0x7fdef0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdef0066690 0x7fdef0068b50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdf04000f70 tx=0x7fdf04007d10 comp rx=0 tx=0).stop 2024-03-30T00:46:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 -- 172.21.15.66:0/3641096220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf141506a0 msgr2=0x7fdf14158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf141506a0 0x7fdf14158a60 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0b0040 tx=0x7fdf0c019640 comp rx=0 tx=0).stop 2024-03-30T00:46:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 -- 172.21.15.66:0/3641096220 shutdown_connections 2024-03-30T00:46:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdef0066690 0x7fdef0068b50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf141506a0 0x7fdf14158a60 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf140ce410 0x7fdf14158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 --2- 172.21.15.66:0/3641096220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf1406dfa0 0x7fdf1414b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 -- 172.21.15.66:0/3641096220 >> 172.21.15.66:0/3641096220 conn(0x7fdf14071750 msgr2=0x7fdf1406ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:02.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 -- 172.21.15.66:0/3641096220 shutdown_connections 2024-03-30T00:46:02.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.626+0000 7fdf18e00640 1 -- 172.21.15.66:0/3641096220 wait complete. 2024-03-30T00:46:02.654 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:46:02.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.786+0000 7ff9e6400640 1 Processor -- start 2024-03-30T00:46:02.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.786+0000 7ff9e6400640 1 -- start start 2024-03-30T00:46:02.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.786+0000 7ff9e6400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff9e00cf2f0 0x7ff9e006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.786+0000 7ff9e6400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9e00cb570 0x7ff9e00cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.786+0000 7ff9e6400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e00cb0e0 con 0x7ff9e015ca80 2024-03-30T00:46:02.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.786+0000 7ff9e6400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e00cf7f0 con 0x7ff9e00cf2f0 2024-03-30T00:46:02.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.786+0000 7ff9e6400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00cfad0 con 0x7ff9e00cb570 2024-03-30T00:46:02.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.790+0000 7ff9d6a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff9e00cf2f0 0x7ff9e006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.790+0000 7ff9dfe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9e00cb570 0x7ff9e00cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.790+0000 7ff9df400640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff9e015ca80 0x7ff9e006b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:36968/0 (socket says 172.21.15.66:36968) 2024-03-30T00:46:02.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.790+0000 7ff9df400640 1 -- 172.21.15.66:0/1233151104 learned_addr learned my addr 172.21.15.66:0/1233151104 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:02.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.790+0000 7ff9dea00640 1 -- 172.21.15.66:0/1233151104 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373444256 0 0) 0x7ff9e00cb0e0 con 0x7ff9e015ca80 2024-03-30T00:46:02.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.790+0000 7ff9dfe00640 1 -- 172.21.15.66:0/1233151104 >> v2:172.21.15.156:3300/0 conn(0x7ff9e00cf2f0 msgr2=0x7ff9e006b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:02.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.790+0000 7ff9dfe00640 1 --2- 172.21.15.66:0/1233151104 >> v2:172.21.15.156:3300/0 conn(0x7ff9e00cf2f0 0x7ff9e006b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.790+0000 7ff9dfe00640 1 -- 172.21.15.66:0/1233151104 >> v1:172.21.15.156:6789/0 conn(0x7ff9e015ca80 legacy=0x7ff9e006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.790+0000 7ff9dfe00640 1 -- 172.21.15.66:0/1233151104 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00cad70 con 0x7ff9e00cb570 2024-03-30T00:46:02.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.790+0000 7ff9d6a00640 1 --2- 172.21.15.66:0/1233151104 >> v2:172.21.15.156:3300/0 conn(0x7ff9e00cf2f0 0x7ff9e006b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:46:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.790+0000 7ff9dfe00640 1 --2- 172.21.15.66:0/1233151104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9e00cb570 0x7ff9e00cba20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009d90 tx=0x7ff9d801f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36817060fdecda3f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9dea00640 1 -- 172.21.15.66:0/1233151104 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80af600 con 0x7ff9e00cb570 2024-03-30T00:46:02.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9dea00640 1 -- 172.21.15.66:0/1233151104 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d80ab050 con 0x7ff9e00cb570 2024-03-30T00:46:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9dea00640 1 -- 172.21.15.66:0/1233151104 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80af980 con 0x7ff9e00cb570 2024-03-30T00:46:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 -- 172.21.15.66:0/1233151104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9e00cb570 msgr2=0x7ff9e00cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 --2- 172.21.15.66:0/1233151104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9e00cb570 0x7ff9e00cba20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009d90 tx=0x7ff9d801f150 comp rx=0 tx=0).stop 2024-03-30T00:46:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 -- 172.21.15.66:0/1233151104 shutdown_connections 2024-03-30T00:46:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 --2- 172.21.15.66:0/1233151104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9e00cb570 0x7ff9e00cba20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 --2- 172.21.15.66:0/1233151104 >> v2:172.21.15.156:3300/0 conn(0x7ff9e00cf2f0 0x7ff9e006b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 -- 172.21.15.66:0/1233151104 >> 172.21.15.66:0/1233151104 conn(0x7ff9e00b2090 msgr2=0x7ff9e00b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 -- 172.21.15.66:0/1233151104 shutdown_connections 2024-03-30T00:46:02.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 -- 172.21.15.66:0/1233151104 wait complete. 2024-03-30T00:46:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 Processor -- start 2024-03-30T00:46:02.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 -- start start 2024-03-30T00:46:02.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e00cb570 0x7ff9e015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9e00cf2f0 0x7ff9e01f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9e015ca80 0x7ff9e0158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c9d00 con 0x7ff9e015ca80 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c9130 con 0x7ff9e00cb570 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9e6400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c93d0 con 0x7ff9e00cf2f0 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9dfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9e015ca80 0x7ff9e0158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9dfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9e015ca80 0x7ff9e0158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37922/0 (socket says 172.21.15.66:37922) 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9dfe00640 1 -- 172.21.15.66:0/2284648792 learned_addr learned my addr 172.21.15.66:0/2284648792 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9df400640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e00cb570 0x7ff9e015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.794+0000 7ff9d6a00640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9e00cf2f0 0x7ff9e01f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9dfe00640 1 -- 172.21.15.66:0/2284648792 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9e00cf2f0 msgr2=0x7ff9e01f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9dfe00640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9e00cf2f0 0x7ff9e01f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9dfe00640 1 -- 172.21.15.66:0/2284648792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e00cb570 msgr2=0x7ff9e015f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9dfe00640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e00cb570 0x7ff9e015f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9dfe00640 1 -- 172.21.15.66:0/2284648792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c8009420 con 0x7ff9e015ca80 2024-03-30T00:46:02.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9dfe00640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9e015ca80 0x7ff9e0158700 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff9d801cd80 tx=0x7ff9d801bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:02.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9d7e00640 1 -- 172.21.15.66:0/2284648792 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8009450 con 0x7ff9e015ca80 2024-03-30T00:46:02.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9d7e00640 1 -- 172.21.15.66:0/2284648792 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d80b9410 con 0x7ff9e015ca80 2024-03-30T00:46:02.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9d7e00640 1 -- 172.21.15.66:0/2284648792 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80b97a0 con 0x7ff9e015ca80 2024-03-30T00:46:02.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9e6400640 1 -- 172.21.15.66:0/2284648792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d8009a70 con 0x7ff9e015ca80 2024-03-30T00:46:02.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9e6400640 1 -- 172.21.15.66:0/2284648792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e00c9990 con 0x7ff9e015ca80 2024-03-30T00:46:02.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9e6400640 1 -- 172.21.15.66:0/2284648792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e00b2f90 con 0x7ff9e015ca80 2024-03-30T00:46:02.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9d7e00640 1 -- 172.21.15.66:0/2284648792 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff9d80b6050 con 0x7ff9e015ca80 2024-03-30T00:46:02.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9d7e00640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9bc066690 0x7ff9bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:02.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9d7e00640 1 -- 172.21.15.66:0/2284648792 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff9d8122630 con 0x7ff9e015ca80 2024-03-30T00:46:02.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.798+0000 7ff9df400640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9bc066690 0x7ff9bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:02.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.802+0000 7ff9df400640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9bc066690 0x7ff9bc068b50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000f70 tx=0x7ff9d0007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:02.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.802+0000 7ff9d7e00640 1 -- 172.21.15.66:0/2284648792 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff9d80ef940 con 0x7ff9e015ca80 2024-03-30T00:46:03.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:02.998+0000 7ff9e6400640 1 -- 172.21.15.66:0/2284648792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9e0066c30 con 0x7ff9e015ca80 2024-03-30T00:46:03.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9d7e00640 1 -- 172.21.15.66:0/2284648792 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4741 (secure 0 0 0) 0x7ff9d80f2f60 con 0x7ff9e015ca80 2024-03-30T00:46:03.009 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:03.009 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/254323195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":254323195},{"type":"v1","addr":"172.21.15.156:6835","nonce":254323195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:44:16.274202+0000","modified":"2024-03-30T00:45:30.650828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":23,"addr":"172.21.15.66:6837/2050987583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2050987583},{"type":"v1","addr":"172.21.15.66:6837","nonce":2050987583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T00:46:03.009 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 51 2024-03-30T00:46:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 -- 172.21.15.66:0/2284648792 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9bc066690 msgr2=0x7ff9bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9bc066690 0x7ff9bc068b50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000f70 tx=0x7ff9d0007df0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 -- 172.21.15.66:0/2284648792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9e015ca80 msgr2=0x7ff9e0158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9e015ca80 0x7ff9e0158700 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff9d801cd80 tx=0x7ff9d801bef0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 -- 172.21.15.66:0/2284648792 shutdown_connections 2024-03-30T00:46:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff9bc066690 0x7ff9bc068b50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9e015ca80 0x7ff9e0158700 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9e00cf2f0 0x7ff9e01f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 --2- 172.21.15.66:0/2284648792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9e00cb570 0x7ff9e015f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 -- 172.21.15.66:0/2284648792 >> 172.21.15.66:0/2284648792 conn(0x7ff9e00b2090 msgr2=0x7ff9e00cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:03.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 -- 172.21.15.66:0/2284648792 shutdown_connections 2024-03-30T00:46:03.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.002+0000 7ff9e6400640 1 -- 172.21.15.66:0/2284648792 wait complete. 2024-03-30T00:46:03.030 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:46:03.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.162+0000 7f5687e00640 1 Processor -- start 2024-03-30T00:46:03.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.162+0000 7f5687e00640 1 -- start start 2024-03-30T00:46:03.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.162+0000 7f5687e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f568806da10 0x7f56880b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.162+0000 7f5687e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568815afd0 con 0x7f568815a750 2024-03-30T00:46:03.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.162+0000 7f5687e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56880ca4b0 con 0x7f568815e2e0 2024-03-30T00:46:03.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.162+0000 7f5687e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880b1cd0 con 0x7f568806da10 2024-03-30T00:46:03.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.162+0000 7f5686a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f568815a750 0x7f568815b460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52936/0 (socket says 172.21.15.66:52936) 2024-03-30T00:46:03.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.162+0000 7f5686a00640 1 -- 172.21.15.66:0/165532648 learned_addr learned my addr 172.21.15.66:0/165532648 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:03.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.162+0000 7f567fe00640 1 --2- 172.21.15.66:0/165532648 >> v2:172.21.15.156:3300/0 conn(0x7f568806da10 0x7f56880b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.162+0000 7f5686000640 1 -- 172.21.15.66:0/165532648 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818534480 0 0) 0x7f568815afd0 con 0x7f568815a750 2024-03-30T00:46:03.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.162+0000 7f5686000640 1 -- 172.21.15.66:0/165532648 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f566c003530 con 0x7f568815a750 2024-03-30T00:46:03.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f5686000640 1 -- 172.21.15.66:0/165532648 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1424717647 0 0) 0x7f566c003530 con 0x7f568815a750 2024-03-30T00:46:03.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f567fe00640 1 -- 172.21.15.66:0/165532648 >> v1:172.21.15.156:6789/0 conn(0x7f568815e2e0 legacy=0x7f5688069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f567fe00640 1 -- 172.21.15.66:0/165532648 >> v1:172.21.15.66:6789/0 conn(0x7f568815a750 legacy=0x7f568815b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f567fe00640 1 -- 172.21.15.66:0/165532648 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880b1430 con 0x7f568806da10 2024-03-30T00:46:03.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f567fe00640 1 --2- 172.21.15.66:0/165532648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f568806da10 0x7f56880b0dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f56700097c0 tx=0x7f567001c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc2038777e5b798 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f5686000640 1 -- 172.21.15.66:0/165532648 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678001680 con 0x7f568806da10 2024-03-30T00:46:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f5686000640 1 -- 172.21.15.66:0/165532648 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56700a7050 con 0x7f568806da10 2024-03-30T00:46:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f5686000640 1 -- 172.21.15.66:0/165532648 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56700ab620 con 0x7f568806da10 2024-03-30T00:46:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f5687e00640 1 -- 172.21.15.66:0/165532648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f568806da10 msgr2=0x7f56880b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f5687e00640 1 --2- 172.21.15.66:0/165532648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f568806da10 0x7f56880b0dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f56700097c0 tx=0x7f567001c300 comp rx=0 tx=0).stop 2024-03-30T00:46:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f5687e00640 1 -- 172.21.15.66:0/165532648 shutdown_connections 2024-03-30T00:46:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.166+0000 7f5687e00640 1 --2- 172.21.15.66:0/165532648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f568806da10 0x7f56880b0dc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 -- 172.21.15.66:0/165532648 >> 172.21.15.66:0/165532648 conn(0x7f56880cbdc0 msgr2=0x7f56880cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 -- 172.21.15.66:0/165532648 shutdown_connections 2024-03-30T00:46:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 -- 172.21.15.66:0/165532648 wait complete. 2024-03-30T00:46:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 Processor -- start 2024-03-30T00:46:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 -- start start 2024-03-30T00:46:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f568806da10 0x7f56880c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f568815a750 0x7f56880b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f568815e2e0 0x7f56880b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880b3150 con 0x7f568815e2e0 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5688072010 con 0x7f568806da10 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56880c9350 con 0x7f568815a750 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f568815e2e0 0x7f56880b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f568815e2e0 0x7f56880b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37934/0 (socket says 172.21.15.66:37934) 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687400640 1 -- 172.21.15.66:0/914143982 learned_addr learned my addr 172.21.15.66:0/914143982 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5686a00640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f568806da10 0x7f56880c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f567fe00640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f568815a750 0x7f56880b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687400640 1 -- 172.21.15.66:0/914143982 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f568815a750 msgr2=0x7f56880b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687400640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f568815a750 0x7f56880b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687400640 1 -- 172.21.15.66:0/914143982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f568806da10 msgr2=0x7f56880c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687400640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f568806da10 0x7f56880c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687400640 1 -- 172.21.15.66:0/914143982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5670009420 con 0x7f568815e2e0 2024-03-30T00:46:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687400640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f568815e2e0 0x7f56880b35e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f568000c730 tx=0x7f568000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f567ea00640 1 -- 172.21.15.66:0/914143982 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680003b80 con 0x7f568815e2e0 2024-03-30T00:46:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f567ea00640 1 -- 172.21.15.66:0/914143982 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5680003ce0 con 0x7f568815e2e0 2024-03-30T00:46:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f5687e00640 1 -- 172.21.15.66:0/914143982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5688071770 con 0x7f568815e2e0 2024-03-30T00:46:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.170+0000 7f567ea00640 1 -- 172.21.15.66:0/914143982 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568000aa30 con 0x7f568815e2e0 2024-03-30T00:46:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.174+0000 7f5687e00640 1 -- 172.21.15.66:0/914143982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5688071d30 con 0x7f568815e2e0 2024-03-30T00:46:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.174+0000 7f5687e00640 1 -- 172.21.15.66:0/914143982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5688067110 con 0x7f568815e2e0 2024-03-30T00:46:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.174+0000 7f567ea00640 1 -- 172.21.15.66:0/914143982 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f568000ab90 con 0x7f568815e2e0 2024-03-30T00:46:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.174+0000 7f567ea00640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56640666c0 0x7f5664068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.174+0000 7f567ea00640 1 -- 172.21.15.66:0/914143982 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5680084000 con 0x7f568815e2e0 2024-03-30T00:46:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.174+0000 7f5686a00640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56640666c0 0x7f5664068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.174+0000 7f5686a00640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56640666c0 0x7f5664068b80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5678006b60 tx=0x7f5678001dd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:03.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.178+0000 7f567ea00640 1 -- 172.21.15.66:0/914143982 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f56800511d0 con 0x7f568815e2e0 2024-03-30T00:46:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.342+0000 7f5687e00640 1 -- 172.21.15.66:0/914143982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5688073cf0 con 0x7f568815e2e0 2024-03-30T00:46:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.342+0000 7f567ea00640 1 -- 172.21.15.66:0/914143982 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16440 (secure 0 0 0) 0x7f56800547f0 con 0x7f568815e2e0 2024-03-30T00:46:03.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:03.352 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":83,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:44:35.079642+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:44:14.685880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-30T00:44:15.260373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.156:6835/2089872680":"2024-03-31T00:44:35.079618+0000","172.21.15.156:6833/2089872680":"2024-03-31T00:44:35.079618+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 -- 172.21.15.66:0/914143982 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56640666c0 msgr2=0x7f5664068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56640666c0 0x7f5664068b80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5678006b60 tx=0x7f5678001dd0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 -- 172.21.15.66:0/914143982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f568815e2e0 msgr2=0x7f56880b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f568815e2e0 0x7f56880b35e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f568000c730 tx=0x7f568000cbf0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 -- 172.21.15.66:0/914143982 shutdown_connections 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f56640666c0 0x7f5664068b80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f568815e2e0 0x7f56880b35e0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f568815a750 0x7f56880b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 --2- 172.21.15.66:0/914143982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f568806da10 0x7f56880c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 -- 172.21.15.66:0/914143982 >> 172.21.15.66:0/914143982 conn(0x7f56880cbdc0 msgr2=0x7f56880ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 -- 172.21.15.66:0/914143982 shutdown_connections 2024-03-30T00:46:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.346+0000 7f5687e00640 1 -- 172.21.15.66:0/914143982 wait complete. 2024-03-30T00:46:03.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:46:03.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8af200640 1 Processor -- start 2024-03-30T00:46:03.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8af200640 1 -- start start 2024-03-30T00:46:03.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8af200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd8b015c770 0x7fd8b00b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8af200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8b0160c80 0x7fd8b006ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8af200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b015cb50 con 0x7fd8b00b4880 2024-03-30T00:46:03.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8af200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b0161240 con 0x7fd8b015c770 2024-03-30T00:46:03.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8af200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00ac340 con 0x7fd8b0160c80 2024-03-30T00:46:03.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8ade00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd8b015c770 0x7fd8b00b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8ae800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd8b00b4880 0x7fd8b00ac7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52938/0 (socket says 172.21.15.66:52938) 2024-03-30T00:46:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8ae800640 1 -- 172.21.15.66:0/4186917835 learned_addr learned my addr 172.21.15.66:0/4186917835 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8a5400640 1 --2- 172.21.15.66:0/4186917835 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8b0160c80 0x7fd8b006ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8ade00640 1 -- 172.21.15.66:0/4186917835 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8b0160c80 msgr2=0x7fd8b006ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8ade00640 1 --2- 172.21.15.66:0/4186917835 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8b0160c80 0x7fd8b006ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8ade00640 1 -- 172.21.15.66:0/4186917835 >> v1:172.21.15.66:6789/0 conn(0x7fd8b00b4880 legacy=0x7fd8b00ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8ade00640 1 -- 172.21.15.66:0/4186917835 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b00b2520 con 0x7fd8b015c770 2024-03-30T00:46:03.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8ad400640 1 -- 172.21.15.66:0/4186917835 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594213965 0 0) 0x7fd8a8001120 con 0x7fd8b00b4880 2024-03-30T00:46:03.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.506+0000 7fd8a5400640 1 --2- 172.21.15.66:0/4186917835 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8b0160c80 0x7fd8b006ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8ade00640 1 --2- 172.21.15.66:0/4186917835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b015c770 0x7fd8b00b1fe0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009d90 tx=0x7fd8a001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c832bbb1d4d5eaa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8ad400640 1 -- 172.21.15.66:0/4186917835 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00a5880 con 0x7fd8b015c770 2024-03-30T00:46:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8ad400640 1 -- 172.21.15.66:0/4186917835 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a00a59e0 con 0x7fd8b015c770 2024-03-30T00:46:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8ad400640 1 -- 172.21.15.66:0/4186917835 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00a5d70 con 0x7fd8b015c770 2024-03-30T00:46:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8af200640 1 -- 172.21.15.66:0/4186917835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b015c770 msgr2=0x7fd8b00b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8af200640 1 --2- 172.21.15.66:0/4186917835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b015c770 0x7fd8b00b1fe0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009d90 tx=0x7fd8a001c8b0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8af200640 1 -- 172.21.15.66:0/4186917835 shutdown_connections 2024-03-30T00:46:03.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8af200640 1 --2- 172.21.15.66:0/4186917835 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8b0160c80 0x7fd8b006ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8af200640 1 --2- 172.21.15.66:0/4186917835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b015c770 0x7fd8b00b1fe0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8af200640 1 -- 172.21.15.66:0/4186917835 >> 172.21.15.66:0/4186917835 conn(0x7fd8b0070730 msgr2=0x7fd8b0070b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:03.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8af200640 1 -- 172.21.15.66:0/4186917835 shutdown_connections 2024-03-30T00:46:03.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.510+0000 7fd8af200640 1 -- 172.21.15.66:0/4186917835 wait complete. 2024-03-30T00:46:03.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8af200640 1 Processor -- start 2024-03-30T00:46:03.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8af200640 1 -- start start 2024-03-30T00:46:03.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8af200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00b4880 0x7fd8b0069d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8af200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8b015c770 0x7fd8b006a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8af200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b0160c80 0x7fd8b0141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8af200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0142750 con 0x7fd8b00b4880 2024-03-30T00:46:03.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8af200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b013fb50 con 0x7fd8b0160c80 2024-03-30T00:46:03.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8af200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b013fe30 con 0x7fd8b015c770 2024-03-30T00:46:03.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8ade00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00b4880 0x7fd8b0069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8ade00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00b4880 0x7fd8b0069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37954/0 (socket says 172.21.15.66:37954) 2024-03-30T00:46:03.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8ade00640 1 -- 172.21.15.66:0/4214549751 learned_addr learned my addr 172.21.15.66:0/4214549751 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:03.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8a5400640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8b015c770 0x7fd8b006a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8ae800640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b0160c80 0x7fd8b0141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8ade00640 1 -- 172.21.15.66:0/4214549751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8b015c770 msgr2=0x7fd8b006a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8ade00640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8b015c770 0x7fd8b006a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8ade00640 1 -- 172.21.15.66:0/4214549751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b0160c80 msgr2=0x7fd8b0141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8ade00640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b0160c80 0x7fd8b0141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8ade00640 1 -- 172.21.15.66:0/4214549751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a0009a70 con 0x7fd8b00b4880 2024-03-30T00:46:03.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8ae800640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b0160c80 0x7fd8b0141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:03.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8ade00640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00b4880 0x7fd8b0069d90 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00b0040 tx=0x7fd8a0019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:03.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8a6a00640 1 -- 172.21.15.66:0/4214549751 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00ad850 con 0x7fd8b00b4880 2024-03-30T00:46:03.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8a6a00640 1 -- 172.21.15.66:0/4214549751 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a00ad9b0 con 0x7fd8b00b4880 2024-03-30T00:46:03.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8a6a00640 1 -- 172.21.15.66:0/4214549751 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00add60 con 0x7fd8b00b4880 2024-03-30T00:46:03.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8af200640 1 -- 172.21.15.66:0/4214549751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b0140110 con 0x7fd8b00b4880 2024-03-30T00:46:03.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8af200640 1 -- 172.21.15.66:0/4214549751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b013f690 con 0x7fd8b00b4880 2024-03-30T00:46:03.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.514+0000 7fd8af200640 1 -- 172.21.15.66:0/4214549751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b0073e00 con 0x7fd8b00b4880 2024-03-30T00:46:03.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.518+0000 7fd8a6a00640 1 -- 172.21.15.66:0/4214549751 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd8a00b4050 con 0x7fd8b00b4880 2024-03-30T00:46:03.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.518+0000 7fd8a6a00640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd88c066690 0x7fd88c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.518+0000 7fd8a6a00640 1 -- 172.21.15.66:0/4214549751 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd8a011f750 con 0x7fd8b00b4880 2024-03-30T00:46:03.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.518+0000 7fd8a5400640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd88c066690 0x7fd88c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.518+0000 7fd8a5400640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd88c066690 0x7fd88c068b50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd898006fd0 tx=0x7fd898008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:03.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.522+0000 7fd8a6a00640 1 -- 172.21.15.66:0/4214549751 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd8a00eca40 con 0x7fd8b00b4880 2024-03-30T00:46:03.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.714+0000 7fd8af200640 1 -- 172.21.15.66:0/4214549751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd8b006dbf0 con 0x7fd8b00b4880 2024-03-30T00:46:03.723 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:46:03.714+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:46:03.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.722+0000 7fd8a6a00640 1 -- 172.21.15.66:0/4214549751 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd8a00f0060 con 0x7fd8b00b4880 2024-03-30T00:46:03.731 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 -- 172.21.15.66:0/4214549751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd88c066690 msgr2=0x7fd88c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd88c066690 0x7fd88c068b50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd898006fd0 tx=0x7fd898008040 comp rx=0 tx=0).stop 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 -- 172.21.15.66:0/4214549751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00b4880 msgr2=0x7fd8b0069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00b4880 0x7fd8b0069d90 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00b0040 tx=0x7fd8a0019650 comp rx=0 tx=0).stop 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 -- 172.21.15.66:0/4214549751 shutdown_connections 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b0160c80 0x7fd8b0141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8b015c770 0x7fd8b006a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd88c066690 0x7fd88c068b50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 --2- 172.21.15.66:0/4214549751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b00b4880 0x7fd8b0069d90 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 -- 172.21.15.66:0/4214549751 >> 172.21.15.66:0/4214549751 conn(0x7fd8b0070730 msgr2=0x7fd8b015ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 -- 172.21.15.66:0/4214549751 shutdown_connections 2024-03-30T00:46:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.726+0000 7fd8af200640 1 -- 172.21.15.66:0/4214549751 wait complete. 2024-03-30T00:46:03.755 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T00:46:03.793 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:46:03.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7e00640 1 Processor -- start 2024-03-30T00:46:03.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7e00640 1 -- start start 2024-03-30T00:46:03.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f07e80b3170 0x7f07e80b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f07e80d07c0 0x7f07e80d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e80cb5b0 con 0x7f07e815a720 2024-03-30T00:46:03.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e80cfb00 con 0x7f07e80d07c0 2024-03-30T00:46:03.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e80d1170 con 0x7f07e80b3170 2024-03-30T00:46:03.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f07e80d07c0 0x7f07e80d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f07e80d07c0 0x7f07e80d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37986/0 (socket says 172.21.15.66:37986) 2024-03-30T00:46:03.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7400640 1 -- 172.21.15.66:0/1939498122 learned_addr learned my addr 172.21.15.66:0/1939498122 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:03.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07de000640 1 --2- 172.21.15.66:0/1939498122 >> v2:172.21.15.156:3300/0 conn(0x7f07e80b3170 0x7f07e80b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7400640 1 -- 172.21.15.66:0/1939498122 >> v2:172.21.15.156:3300/0 conn(0x7f07e80b3170 msgr2=0x7f07e80b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:03.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7400640 1 --2- 172.21.15.66:0/1939498122 >> v2:172.21.15.156:3300/0 conn(0x7f07e80b3170 0x7f07e80b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7400640 1 -- 172.21.15.66:0/1939498122 >> v1:172.21.15.156:6789/0 conn(0x7f07e815a720 legacy=0x7f07e80ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.894+0000 7f07e7400640 1 -- 172.21.15.66:0/1939498122 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80cb080 con 0x7f07e80d07c0 2024-03-30T00:46:03.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e7400640 1 --2- 172.21.15.66:0/1939498122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e80d07c0 0x7f07e80d0c30 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f07e80cc740 tx=0x7f07e001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d055d8672690a8f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e6000640 1 -- 172.21.15.66:0/1939498122 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00a57a0 con 0x7f07e80d07c0 2024-03-30T00:46:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e6000640 1 -- 172.21.15.66:0/1939498122 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e00a5900 con 0x7f07e80d07c0 2024-03-30T00:46:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e6000640 1 -- 172.21.15.66:0/1939498122 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00a5c90 con 0x7f07e80d07c0 2024-03-30T00:46:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e7e00640 1 -- 172.21.15.66:0/1939498122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e80d07c0 msgr2=0x7f07e80d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e7e00640 1 --2- 172.21.15.66:0/1939498122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e80d07c0 0x7f07e80d0c30 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f07e80cc740 tx=0x7f07e001c9f0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e7e00640 1 -- 172.21.15.66:0/1939498122 shutdown_connections 2024-03-30T00:46:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e7e00640 1 --2- 172.21.15.66:0/1939498122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e80d07c0 0x7f07e80d0c30 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e7e00640 1 --2- 172.21.15.66:0/1939498122 >> v2:172.21.15.156:3300/0 conn(0x7f07e80b3170 0x7f07e80b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e7e00640 1 -- 172.21.15.66:0/1939498122 >> 172.21.15.66:0/1939498122 conn(0x7f07e8069d90 msgr2=0x7f07e806a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e7e00640 1 -- 172.21.15.66:0/1939498122 shutdown_connections 2024-03-30T00:46:03.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e7e00640 1 -- 172.21.15.66:0/1939498122 wait complete. 2024-03-30T00:46:03.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e7e00640 1 Processor -- start 2024-03-30T00:46:03.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.898+0000 7f07e7e00640 1 -- start start 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07e80b3170 0x7f07e81509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07e80d07c0 0x7f07e814d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e815a720 0x7f07e80b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e8152e80 con 0x7f07e815a720 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80b0930 con 0x7f07e80d07c0 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80b0c10 con 0x7f07e80b3170 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e815a720 0x7f07e80b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e815a720 0x7f07e80b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38000/0 (socket says 172.21.15.66:38000) 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7400640 1 -- 172.21.15.66:0/3848285017 learned_addr learned my addr 172.21.15.66:0/3848285017 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e6a00640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07e80b3170 0x7f07e81509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07de000640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07e80d07c0 0x7f07e814d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7400640 1 -- 172.21.15.66:0/3848285017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07e80b3170 msgr2=0x7f07e81509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7400640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07e80b3170 0x7f07e81509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7400640 1 -- 172.21.15.66:0/3848285017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07e80d07c0 msgr2=0x7f07e814d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7400640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07e80d07c0 0x7f07e814d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7400640 1 -- 172.21.15.66:0/3848285017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e0009b30 con 0x7f07e815a720 2024-03-30T00:46:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e6a00640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07e80b3170 0x7f07e81509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7400640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e815a720 0x7f07e80b5090 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f07e00af040 tx=0x7f07e0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07df400640 1 -- 172.21.15.66:0/3848285017 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00a57a0 con 0x7f07e815a720 2024-03-30T00:46:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07df400640 1 -- 172.21.15.66:0/3848285017 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e00b8880 con 0x7f07e815a720 2024-03-30T00:46:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07df400640 1 -- 172.21.15.66:0/3848285017 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00b8c30 con 0x7f07e815a720 2024-03-30T00:46:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7e00640 1 -- 172.21.15.66:0/3848285017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e80b0ef0 con 0x7f07e815a720 2024-03-30T00:46:03.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7e00640 1 -- 172.21.15.66:0/3848285017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e8159e70 con 0x7f07e815a720 2024-03-30T00:46:03.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.902+0000 7f07e7e00640 1 -- 172.21.15.66:0/3848285017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e8073e00 con 0x7f07e815a720 2024-03-30T00:46:03.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.906+0000 7f07df400640 1 -- 172.21.15.66:0/3848285017 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f07e00c3460 con 0x7f07e815a720 2024-03-30T00:46:03.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.906+0000 7f07df400640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07c4066690 0x7f07c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:03.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.906+0000 7f07df400640 1 -- 172.21.15.66:0/3848285017 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f07e011faa0 con 0x7f07e815a720 2024-03-30T00:46:03.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.906+0000 7f07e6a00640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07c4066690 0x7f07c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:03.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.906+0000 7f07e6a00640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07c4066690 0x7f07c4068b50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f07d8000f70 tx=0x7f07d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:03.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:03.910+0000 7f07df400640 1 -- 172.21.15.66:0/3848285017 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f07e00ecd30 con 0x7f07e815a720 2024-03-30T00:46:04.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.106+0000 7f07e7e00640 1 -- 172.21.15.66:0/3848285017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f07e80b20a0 con 0x7f07e815a720 2024-03-30T00:46:04.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07df400640 1 -- 172.21.15.66:0/3848285017 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) v1 ==== 104+0+0 (secure 0 0 0) 0x7f07e00f0350 con 0x7f07e815a720 2024-03-30T00:46:04.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 -- 172.21.15.66:0/3848285017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07c4066690 msgr2=0x7f07c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:04.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07c4066690 0x7f07c4068b50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f07d8000f70 tx=0x7f07d8009040 comp rx=0 tx=0).stop 2024-03-30T00:46:04.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 -- 172.21.15.66:0/3848285017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e815a720 msgr2=0x7f07e80b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:04.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e815a720 0x7f07e80b5090 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f07e00af040 tx=0x7f07e0019790 comp rx=0 tx=0).stop 2024-03-30T00:46:04.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 -- 172.21.15.66:0/3848285017 shutdown_connections 2024-03-30T00:46:04.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07c4066690 0x7f07c4068b50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:04.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07e815a720 0x7f07e80b5090 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:04.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07e80d07c0 0x7f07e814d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:04.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 --2- 172.21.15.66:0/3848285017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07e80b3170 0x7f07e81509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:04.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 -- 172.21.15.66:0/3848285017 >> 172.21.15.66:0/3848285017 conn(0x7f07e8069d90 msgr2=0x7f07e80b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:04.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 -- 172.21.15.66:0/3848285017 shutdown_connections 2024-03-30T00:46:04.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.734+0000 7f07e7e00640 1 -- 172.21.15.66:0/3848285017 wait complete. 2024-03-30T00:46:04.768 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T00:46:04.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2e4a00640 1 Processor -- start 2024-03-30T00:46:04.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2e4a00640 1 -- start start 2024-03-30T00:46:04.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2e4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2e0159090 0x7fe2e0159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:04.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2e4a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e015d210 con 0x7fe2e006dfd0 2024-03-30T00:46:04.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2e4a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e00b13e0 con 0x7fe2e00b1e30 2024-03-30T00:46:04.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2e4a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e006e460 con 0x7fe2e0159090 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2dfe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe2e006dfd0 0x7fe2e00ce3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52952/0 (socket says 172.21.15.66:52952) 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2dfe00640 1 -- 172.21.15.66:0/1451407879 learned_addr learned my addr 172.21.15.66:0/1451407879 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2df400640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2e0159090 0x7fe2e0159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028064977 0 0) 0x7fe2e015d210 con 0x7fe2e006dfd0 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c4005740 con 0x7fe2e006dfd0 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575649986 0 0) 0x7fe2e00b13e0 con 0x7fe2e00b1e30 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2e015d210 con 0x7fe2e00b1e30 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3570117042 0 0) 0x7fe2c4005740 con 0x7fe2e006dfd0 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2e0159090 msgr2=0x7fe2e0159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2de000640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2e0159090 0x7fe2e0159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 >> v1:172.21.15.156:6789/0 conn(0x7fe2e00b1e30 legacy=0x7fe2e006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e006e740 con 0x7fe2e006dfd0 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fe2d8001a00 con 0x7fe2e006dfd0 2024-03-30T00:46:04.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2df400640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2e0159090 0x7fe2e0159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.950+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 >> v1:172.21.15.66:6789/0 conn(0x7fe2e006dfd0 legacy=0x7fe2e00ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2de000640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2e0159090 0x7fe2c4005d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2de000640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c4006290 0x7fe2c40069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2de000640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2c4006f20 0x7fe2c40073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e006e460 con 0x7fe2c4006f20 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c400dc50 con 0x7fe2c4006290 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c400de10 con 0x7fe2e0159090 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2dfe00640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2c4006f20 0x7fe2c40073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2df400640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2e0159090 0x7fe2c4005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2dea00640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c4006290 0x7fe2c40069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2dfe00640 1 -- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2e0159090 msgr2=0x7fe2c4005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2dfe00640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2e0159090 0x7fe2c4005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2dfe00640 1 -- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c4006290 msgr2=0x7fe2c40069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2dfe00640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c4006290 0x7fe2c40069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:04.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2dfe00640 1 -- 172.21.15.66:0/1451407879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d0009420 con 0x7fe2c4006f20 2024-03-30T00:46:04.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2dfe00640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2c4006f20 0x7fe2c40073a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe2e006ea60 tx=0x7fe2d801d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c8d30b720ea867 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:04.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d80ad630 con 0x7fe2c4006f20 2024-03-30T00:46:04.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d80a8070 con 0x7fe2c4006f20 2024-03-30T00:46:04.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.954+0000 7fe2de000640 1 -- 172.21.15.66:0/1451407879 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d80ad9f0 con 0x7fe2c4006f20 2024-03-30T00:46:04.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 -- 172.21.15.66:0/1451407879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2c4006f20 msgr2=0x7fe2c40073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:04.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 --2- 172.21.15.66:0/1451407879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2c4006f20 0x7fe2c40073a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe2e006ea60 tx=0x7fe2d801d2a0 comp rx=0 tx=0).stop 2024-03-30T00:46:04.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2dfe00640 1 -- 172.21.15.66:0/1451407879 reap_dead start 2024-03-30T00:46:04.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 -- 172.21.15.66:0/1451407879 shutdown_connections 2024-03-30T00:46:04.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 -- 172.21.15.66:0/1451407879 >> 172.21.15.66:0/1451407879 conn(0x7fe2e0069f60 msgr2=0x7fe2e006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:04.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 -- 172.21.15.66:0/1451407879 shutdown_connections 2024-03-30T00:46:04.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 -- 172.21.15.66:0/1451407879 wait complete. 2024-03-30T00:46:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 Processor -- start 2024-03-30T00:46:04.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 -- start start 2024-03-30T00:46:04.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2c4006f20 0x7fe2e00cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:04.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2e01f1fa0 0x7fe2e00ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e00accf0 0x7fe2e014ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0159510 con 0x7fe2e00accf0 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e01597f0 con 0x7fe2e01f1fa0 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2e4a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0159ad0 con 0x7fe2c4006f20 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2dfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e00accf0 0x7fe2e014ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2dfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e00accf0 0x7fe2e014ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38018/0 (socket says 172.21.15.66:38018) 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2dfe00640 1 -- 172.21.15.66:0/628352597 learned_addr learned my addr 172.21.15.66:0/628352597 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2df400640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2c4006f20 0x7fe2e00cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2dea00640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2e01f1fa0 0x7fe2e00ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2dfe00640 1 -- 172.21.15.66:0/628352597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2c4006f20 msgr2=0x7fe2e00cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2dfe00640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2c4006f20 0x7fe2e00cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2dfe00640 1 -- 172.21.15.66:0/628352597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2e01f1fa0 msgr2=0x7fe2e00ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2dfe00640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2e01f1fa0 0x7fe2e00ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:04.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2dfe00640 1 -- 172.21.15.66:0/628352597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0159db0 con 0x7fe2e00accf0 2024-03-30T00:46:04.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.958+0000 7fe2dea00640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2e01f1fa0 0x7fe2e00ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:04.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.962+0000 7fe2dfe00640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e00accf0 0x7fe2e014ba50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe2d801d0d0 tx=0x7fe2c4007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:04.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.962+0000 7fe2d7400640 1 -- 172.21.15.66:0/628352597 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8001f60 con 0x7fe2e00accf0 2024-03-30T00:46:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.962+0000 7fe2d7400640 1 -- 172.21.15.66:0/628352597 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d8001800 con 0x7fe2e00accf0 2024-03-30T00:46:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.962+0000 7fe2d7400640 1 -- 172.21.15.66:0/628352597 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8004d70 con 0x7fe2e00accf0 2024-03-30T00:46:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.962+0000 7fe2e4a00640 1 -- 172.21.15.66:0/628352597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e015a090 con 0x7fe2e00accf0 2024-03-30T00:46:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.962+0000 7fe2e4a00640 1 -- 172.21.15.66:0/628352597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e00cd540 con 0x7fe2e00accf0 2024-03-30T00:46:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.962+0000 7fe2e4a00640 1 -- 172.21.15.66:0/628352597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e00cd730 con 0x7fe2e00accf0 2024-03-30T00:46:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.962+0000 7fe2d7400640 1 -- 172.21.15.66:0/628352597 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe2d80aa030 con 0x7fe2e00accf0 2024-03-30T00:46:04.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.962+0000 7fe2d7400640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2c0066690 0x7fe2c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:04.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.962+0000 7fe2d7400640 1 -- 172.21.15.66:0/628352597 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fe2d811e460 con 0x7fe2e00accf0 2024-03-30T00:46:04.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.962+0000 7fe2df400640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2c0066690 0x7fe2c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:04.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.966+0000 7fe2df400640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2c0066690 0x7fe2c0068b50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009d60 tx=0x7fe2d0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:04.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:04.966+0000 7fe2d7400640 1 -- 172.21.15.66:0/628352597 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe2d80eb560 con 0x7fe2e00accf0 2024-03-30T00:46:05.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.126+0000 7fe2e4a00640 1 -- 172.21.15.66:0/628352597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2e015a280 con 0x7fe2e00accf0 2024-03-30T00:46:05.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.742+0000 7fe2d7400640 1 -- 172.21.15.66:0/628352597 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe2d80eeb80 con 0x7fe2e00accf0 2024-03-30T00:46:05.756 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-03-30T00:46:05.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 -- 172.21.15.66:0/628352597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2c0066690 msgr2=0x7fe2c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:05.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2c0066690 0x7fe2c0068b50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009d60 tx=0x7fe2d0009190 comp rx=0 tx=0).stop 2024-03-30T00:46:05.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 -- 172.21.15.66:0/628352597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e00accf0 msgr2=0x7fe2e014ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:05.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e00accf0 0x7fe2e014ba50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe2d801d0d0 tx=0x7fe2c4007a10 comp rx=0 tx=0).stop 2024-03-30T00:46:05.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 -- 172.21.15.66:0/628352597 shutdown_connections 2024-03-30T00:46:05.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe2c0066690 0x7fe2c0068b50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:05.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e00accf0 0x7fe2e014ba50 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:05.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2e01f1fa0 0x7fe2e00ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:05.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 --2- 172.21.15.66:0/628352597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2c4006f20 0x7fe2e00cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:05.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 -- 172.21.15.66:0/628352597 >> 172.21.15.66:0/628352597 conn(0x7fe2e0069f60 msgr2=0x7fe2e006c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:05.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 -- 172.21.15.66:0/628352597 shutdown_connections 2024-03-30T00:46:05.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.746+0000 7fe2e4a00640 1 -- 172.21.15.66:0/628352597 wait complete. 2024-03-30T00:46:05.777 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T00:46:05.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.918+0000 7f948ee00640 1 Processor -- start 2024-03-30T00:46:05.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.918+0000 7f948ee00640 1 -- start start 2024-03-30T00:46:05.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94880aca30 0x7f94880b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:05.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948ee00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f948806ce40 0x7f94880b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:05.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948ee00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948815cb80 con 0x7f948806d580 2024-03-30T00:46:05.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948ee00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94880cf810 con 0x7f948806ce40 2024-03-30T00:46:05.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948806c790 con 0x7f94880aca30 2024-03-30T00:46:05.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948d000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94880aca30 0x7f94880b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:05.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948e400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f948806ce40 0x7f94880b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:05.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948da00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f948806d580 0x7f94880b1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:37012/0 (socket says 172.21.15.66:37012) 2024-03-30T00:46:05.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948da00640 1 -- 172.21.15.66:0/1550032975 learned_addr learned my addr 172.21.15.66:0/1550032975 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:05.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948e400640 1 -- 172.21.15.66:0/1550032975 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94880aca30 msgr2=0x7f94880b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:05.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948e400640 1 --2- 172.21.15.66:0/1550032975 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94880aca30 0x7f94880b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:05.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948e400640 1 -- 172.21.15.66:0/1550032975 >> v1:172.21.15.156:6789/0 conn(0x7f948806d580 legacy=0x7f94880b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:05.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.922+0000 7f948e400640 1 -- 172.21.15.66:0/1550032975 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948806ca70 con 0x7f948806ce40 2024-03-30T00:46:05.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f948e400640 1 --2- 172.21.15.66:0/1550032975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f948806ce40 0x7f94880b2030 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9480009d90 tx=0x7f948001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b583a3b4d4d5b8b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:05.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f9487e00640 1 -- 172.21.15.66:0/1550032975 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94800a5a00 con 0x7f948806ce40 2024-03-30T00:46:05.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f9487e00640 1 -- 172.21.15.66:0/1550032975 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94800a5b60 con 0x7f948806ce40 2024-03-30T00:46:05.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f9487e00640 1 -- 172.21.15.66:0/1550032975 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94800ad440 con 0x7f948806ce40 2024-03-30T00:46:05.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f948ee00640 1 -- 172.21.15.66:0/1550032975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f948806ce40 msgr2=0x7f94880b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:05.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f948ee00640 1 --2- 172.21.15.66:0/1550032975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f948806ce40 0x7f94880b2030 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9480009d90 tx=0x7f948001c940 comp rx=0 tx=0).stop 2024-03-30T00:46:05.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f948ee00640 1 -- 172.21.15.66:0/1550032975 shutdown_connections 2024-03-30T00:46:05.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f948ee00640 1 --2- 172.21.15.66:0/1550032975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f948806ce40 0x7f94880b2030 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:05.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f948ee00640 1 --2- 172.21.15.66:0/1550032975 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94880aca30 0x7f94880b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:05.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f948ee00640 1 -- 172.21.15.66:0/1550032975 >> 172.21.15.66:0/1550032975 conn(0x7f94880ce930 msgr2=0x7f94880ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:05.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f948ee00640 1 -- 172.21.15.66:0/1550032975 shutdown_connections 2024-03-30T00:46:05.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f948ee00640 1 -- 172.21.15.66:0/1550032975 wait complete. 2024-03-30T00:46:05.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f948ee00640 1 Processor -- start 2024-03-30T00:46:05.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.926+0000 7f948ee00640 1 -- start start 2024-03-30T00:46:05.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948ee00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f948806ce40 0x7f94881457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:05.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f948806d580 0x7f9488143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:05.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94880aca30 0x7f94880abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:05.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9488145d00 con 0x7f94880aca30 2024-03-30T00:46:05.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948ee00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9488149440 con 0x7f948806ce40 2024-03-30T00:46:05.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9488149fc0 con 0x7f948806d580 2024-03-30T00:46:05.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94880aca30 0x7f94880abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:05.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94880aca30 0x7f94880abac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38034/0 (socket says 172.21.15.66:38034) 2024-03-30T00:46:05.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948e400640 1 -- 172.21.15.66:0/3671712294 learned_addr learned my addr 172.21.15.66:0/3671712294 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:05.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948d000640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f948806d580 0x7f9488143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:05.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948da00640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f948806ce40 0x7f94881457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:05.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948e400640 1 -- 172.21.15.66:0/3671712294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f948806d580 msgr2=0x7f9488143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:05.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948e400640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f948806d580 0x7f9488143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:05.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948e400640 1 -- 172.21.15.66:0/3671712294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f948806ce40 msgr2=0x7f94881457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:05.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948e400640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f948806ce40 0x7f94881457c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:05.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948e400640 1 -- 172.21.15.66:0/3671712294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9480009a70 con 0x7f94880aca30 2024-03-30T00:46:05.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948e400640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94880aca30 0x7f94880abac0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f9480009d90 tx=0x7f94800196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:05.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f9485600640 1 -- 172.21.15.66:0/3671712294 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94800aede0 con 0x7f94880aca30 2024-03-30T00:46:05.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f9485600640 1 -- 172.21.15.66:0/3671712294 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9480009450 con 0x7f94880aca30 2024-03-30T00:46:05.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f9485600640 1 -- 172.21.15.66:0/3671712294 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94800b7650 con 0x7f94880aca30 2024-03-30T00:46:05.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948ee00640 1 -- 172.21.15.66:0/3671712294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9488149690 con 0x7f94880aca30 2024-03-30T00:46:05.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948ee00640 1 -- 172.21.15.66:0/3671712294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9488149c50 con 0x7f94880aca30 2024-03-30T00:46:05.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f948ee00640 1 -- 172.21.15.66:0/3671712294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948814a150 con 0x7f94880aca30 2024-03-30T00:46:05.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.930+0000 7f9485600640 1 -- 172.21.15.66:0/3671712294 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f94800b4050 con 0x7f94880aca30 2024-03-30T00:46:05.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.934+0000 7f9485600640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9468066690 0x7f9468068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:05.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.934+0000 7f9485600640 1 -- 172.21.15.66:0/3671712294 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f948011f450 con 0x7f94880aca30 2024-03-30T00:46:05.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.934+0000 7f948da00640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9468066690 0x7f9468068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:05.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.934+0000 7f948da00640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9468066690 0x7f9468068b50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f947c000f70 tx=0x7f947c007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:05.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:05.938+0000 7f9485600640 1 -- 172.21.15.66:0/3671712294 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f94800ec820 con 0x7f94880aca30 2024-03-30T00:46:06.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.098+0000 7f948ee00640 1 -- 172.21.15.66:0/3671712294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9488073cf0 con 0x7f94880aca30 2024-03-30T00:46:06.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.746+0000 7f9485600640 1 -- 172.21.15.66:0/3671712294 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) v1 ==== 169+0+0 (secure 0 0 0) 0x7f94800efe40 con 0x7f94880aca30 2024-03-30T00:46:06.755 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 -- 172.21.15.66:0/3671712294 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9468066690 msgr2=0x7f9468068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9468066690 0x7f9468068b50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f947c000f70 tx=0x7f947c007e90 comp rx=0 tx=0).stop 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 -- 172.21.15.66:0/3671712294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94880aca30 msgr2=0x7f94880abac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94880aca30 0x7f94880abac0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f9480009d90 tx=0x7f94800196e0 comp rx=0 tx=0).stop 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 -- 172.21.15.66:0/3671712294 shutdown_connections 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9468066690 0x7f9468068b50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94880aca30 0x7f94880abac0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f948806d580 0x7f9488143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 --2- 172.21.15.66:0/3671712294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f948806ce40 0x7f94881457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 -- 172.21.15.66:0/3671712294 >> 172.21.15.66:0/3671712294 conn(0x7f94880ce930 msgr2=0x7f94880ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 -- 172.21.15.66:0/3671712294 shutdown_connections 2024-03-30T00:46:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.750+0000 7f948ee00640 1 -- 172.21.15.66:0/3671712294 wait complete. 2024-03-30T00:46:06.782 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-30T00:46:06.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3570e00640 1 Processor -- start 2024-03-30T00:46:06.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3570e00640 1 -- start start 2024-03-30T00:46:06.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3570e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f356c15ca10 0x7f356c06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3570e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f356c0ae4e0 0x7f356c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3570e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356c06e7d0 con 0x7f356c0ce820 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3570e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356c0b1f70 con 0x7f356c0ae4e0 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3570e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356c06f1a0 con 0x7f356c15ca10 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3563e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f356c0ae4e0 0x7f356c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3563e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f356c0ae4e0 0x7f356c0cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38042/0 (socket says 172.21.15.66:38042) 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3563e00640 1 -- 172.21.15.66:0/3447600691 learned_addr learned my addr 172.21.15.66:0/3447600691 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f356a800640 1 --1- 172.21.15.66:0/3447600691 >> v1:172.21.15.66:6789/0 conn(0x7f356c0ce820 0x7f356c160510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52960/0 (socket says 172.21.15.66:52960) 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3569e00640 1 --2- 172.21.15.66:0/3447600691 >> v2:172.21.15.156:3300/0 conn(0x7f356c15ca10 0x7f356c06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3563e00640 1 -- 172.21.15.66:0/3447600691 >> v2:172.21.15.156:3300/0 conn(0x7f356c15ca10 msgr2=0x7f356c06ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3563e00640 1 --2- 172.21.15.66:0/3447600691 >> v2:172.21.15.156:3300/0 conn(0x7f356c15ca10 0x7f356c06ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:06.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3563e00640 1 -- 172.21.15.66:0/3447600691 >> v1:172.21.15.66:6789/0 conn(0x7f356c0ce820 legacy=0x7f356c160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:06.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3563e00640 1 -- 172.21.15.66:0/3447600691 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c15cdf0 con 0x7f356c0ae4e0 2024-03-30T00:46:06.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.918+0000 7f3569400640 1 -- 172.21.15.66:0/3447600691 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197888438 0 0) 0x7f356c06e7d0 con 0x7f356c0ce820 2024-03-30T00:46:06.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3563e00640 1 --2- 172.21.15.66:0/3447600691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae4e0 0x7f356c0cb620 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f356c06f9f0 tx=0x7f355401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244d463c8947eb86 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3569400640 1 -- 172.21.15.66:0/3447600691 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35540aa650 con 0x7f356c0ae4e0 2024-03-30T00:46:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3569400640 1 -- 172.21.15.66:0/3447600691 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35540aa7b0 con 0x7f356c0ae4e0 2024-03-30T00:46:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3569400640 1 -- 172.21.15.66:0/3447600691 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35540aab40 con 0x7f356c0ae4e0 2024-03-30T00:46:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3570e00640 1 -- 172.21.15.66:0/3447600691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae4e0 msgr2=0x7f356c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3570e00640 1 --2- 172.21.15.66:0/3447600691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae4e0 0x7f356c0cb620 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f356c06f9f0 tx=0x7f355401c4f0 comp rx=0 tx=0).stop 2024-03-30T00:46:06.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3570e00640 1 -- 172.21.15.66:0/3447600691 shutdown_connections 2024-03-30T00:46:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3570e00640 1 --2- 172.21.15.66:0/3447600691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae4e0 0x7f356c0cb620 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3570e00640 1 --2- 172.21.15.66:0/3447600691 >> v2:172.21.15.156:3300/0 conn(0x7f356c15ca10 0x7f356c06ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3570e00640 1 -- 172.21.15.66:0/3447600691 >> 172.21.15.66:0/3447600691 conn(0x7f356c098a20 msgr2=0x7f356c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3570e00640 1 -- 172.21.15.66:0/3447600691 shutdown_connections 2024-03-30T00:46:06.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3570e00640 1 -- 172.21.15.66:0/3447600691 wait complete. 2024-03-30T00:46:06.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3570e00640 1 Processor -- start 2024-03-30T00:46:06.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.922+0000 7f3570e00640 1 -- start start 2024-03-30T00:46:06.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3570e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae4e0 0x7f356c06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:06.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3570e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f356c0ce820 0x7f356c0c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:06.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3570e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356c15ca10 0x7f356c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:06.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3570e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0c42b0 con 0x7f356c0ae4e0 2024-03-30T00:46:06.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3570e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0c36a0 con 0x7f356c15ca10 2024-03-30T00:46:06.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3570e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0c3980 con 0x7f356c0ce820 2024-03-30T00:46:06.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3569e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae4e0 0x7f356c06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:06.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3569e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae4e0 0x7f356c06dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38052/0 (socket says 172.21.15.66:38052) 2024-03-30T00:46:06.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3569e00640 1 -- 172.21.15.66:0/3936743480 learned_addr learned my addr 172.21.15.66:0/3936743480 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:06.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3569e00640 1 -- 172.21.15.66:0/3936743480 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f356c0ce820 msgr2=0x7f356c0c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:06.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3569e00640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f356c0ce820 0x7f356c0c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:06.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3569e00640 1 -- 172.21.15.66:0/3936743480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356c15ca10 msgr2=0x7f356c155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:06.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3569e00640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356c15ca10 0x7f356c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:06.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3569e00640 1 -- 172.21.15.66:0/3936743480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35540094d0 con 0x7f356c0ae4e0 2024-03-30T00:46:06.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3569e00640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae4e0 0x7f356c06dd70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f355c00d800 tx=0x7f355c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:06.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3562000640 1 -- 172.21.15.66:0/3936743480 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c00b6c0 con 0x7f356c0ae4e0 2024-03-30T00:46:06.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3562000640 1 -- 172.21.15.66:0/3936743480 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c00b820 con 0x7f356c0ae4e0 2024-03-30T00:46:06.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3570e00640 1 -- 172.21.15.66:0/3936743480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c0c3c60 con 0x7f356c0ae4e0 2024-03-30T00:46:06.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3562000640 1 -- 172.21.15.66:0/3936743480 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c00bb90 con 0x7f356c0ae4e0 2024-03-30T00:46:06.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3570e00640 1 -- 172.21.15.66:0/3936743480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356c155d50 con 0x7f356c0ae4e0 2024-03-30T00:46:06.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3570e00640 1 -- 172.21.15.66:0/3936743480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356c155fa0 con 0x7f356c0ae4e0 2024-03-30T00:46:06.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.926+0000 7f3562000640 1 -- 172.21.15.66:0/3936743480 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f355c00bcf0 con 0x7f356c0ae4e0 2024-03-30T00:46:06.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.930+0000 7f3562000640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3548066690 0x7f3548068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:06.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.930+0000 7f3562000640 1 -- 172.21.15.66:0/3936743480 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f355c084610 con 0x7f356c0ae4e0 2024-03-30T00:46:06.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.930+0000 7f3563e00640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3548066690 0x7f3548068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:06.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.930+0000 7f3563e00640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3548066690 0x7f3548068b50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f35540ae000 tx=0x7f3554008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:06.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:06.934+0000 7f3562000640 1 -- 172.21.15.66:0/3936743480 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f356c155fa0 con 0x7f356c0ae4e0 2024-03-30T00:46:07.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.094+0000 7f3570e00640 1 -- 172.21.15.66:0/3936743480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f356c155fa0 con 0x7f356c0ae4e0 2024-03-30T00:46:07.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.102+0000 7f3562000640 1 -- 172.21.15.66:0/3936743480 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f356c155fa0 con 0x7f356c0ae4e0 2024-03-30T00:46:07.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 -- 172.21.15.66:0/3936743480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3548066690 msgr2=0x7f3548068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3548066690 0x7f3548068b50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f35540ae000 tx=0x7f3554008040 comp rx=0 tx=0).stop 2024-03-30T00:46:07.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 -- 172.21.15.66:0/3936743480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae4e0 msgr2=0x7f356c06dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae4e0 0x7f356c06dd70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f355c00d800 tx=0x7f355c00dcc0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 -- 172.21.15.66:0/3936743480 shutdown_connections 2024-03-30T00:46:07.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356c15ca10 0x7f356c155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f356c0ce820 0x7f356c0c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3548066690 0x7f3548068b50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 --2- 172.21.15.66:0/3936743480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f356c0ae4e0 0x7f356c06dd70 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 -- 172.21.15.66:0/3936743480 >> 172.21.15.66:0/3936743480 conn(0x7f356c098a20 msgr2=0x7f356c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:07.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 -- 172.21.15.66:0/3936743480 shutdown_connections 2024-03-30T00:46:07.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.110+0000 7f3570e00640 1 -- 172.21.15.66:0/3936743480 wait complete. 2024-03-30T00:46:07.143 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-03-30T00:46:07.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a5a800640 1 Processor -- start 2024-03-30T00:46:07.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a5a800640 1 -- start start 2024-03-30T00:46:07.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a5a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2a540cda90 0x7f2a540ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:07.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a5a800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a540cde70 0x7f2a540cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a5a800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5415a990 con 0x7f2a54159c60 2024-03-30T00:46:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a5a800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a540b1b80 con 0x7f2a540cda90 2024-03-30T00:46:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a5a800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540cd3e0 con 0x7f2a540cde70 2024-03-30T00:46:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a53400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2a540cda90 0x7f2a540ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a53400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2a540cda90 0x7f2a540ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38064/0 (socket says 172.21.15.66:38064) 2024-03-30T00:46:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a53400640 1 -- 172.21.15.66:0/3036616659 learned_addr learned my addr 172.21.15.66:0/3036616659 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a4be00640 1 --2- 172.21.15.66:0/3036616659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a540cde70 0x7f2a540cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a53400640 1 -- 172.21.15.66:0/3036616659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a540cde70 msgr2=0x7f2a540cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:07.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a53400640 1 --2- 172.21.15.66:0/3036616659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a540cde70 0x7f2a540cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a53400640 1 -- 172.21.15.66:0/3036616659 >> v1:172.21.15.156:6789/0 conn(0x7f2a54159c60 legacy=0x7f2a5415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a53400640 1 -- 172.21.15.66:0/3036616659 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540b11e0 con 0x7f2a540cda90 2024-03-30T00:46:07.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.274+0000 7f2a4be00640 1 --2- 172.21.15.66:0/3036616659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a540cde70 0x7f2a540cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:07.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a53400640 1 --2- 172.21.15.66:0/3036616659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a540cda90 0x7f2a540ce520 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400bb10 tx=0x7f2a4401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8396285de047b04 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:07.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a52a00640 1 -- 172.21.15.66:0/3036616659 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a440a7b80 con 0x7f2a540cda90 2024-03-30T00:46:07.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a52a00640 1 -- 172.21.15.66:0/3036616659 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a440a7ce0 con 0x7f2a540cda90 2024-03-30T00:46:07.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a52a00640 1 -- 172.21.15.66:0/3036616659 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a440af660 con 0x7f2a540cda90 2024-03-30T00:46:07.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a5a800640 1 -- 172.21.15.66:0/3036616659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a540cda90 msgr2=0x7f2a540ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a5a800640 1 --2- 172.21.15.66:0/3036616659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a540cda90 0x7f2a540ce520 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400bb10 tx=0x7f2a4401eba0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a5a800640 1 -- 172.21.15.66:0/3036616659 shutdown_connections 2024-03-30T00:46:07.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a5a800640 1 --2- 172.21.15.66:0/3036616659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a540cde70 0x7f2a540cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a5a800640 1 --2- 172.21.15.66:0/3036616659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a540cda90 0x7f2a540ce520 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a5a800640 1 -- 172.21.15.66:0/3036616659 >> 172.21.15.66:0/3036616659 conn(0x7f2a54069f60 msgr2=0x7f2a5406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:07.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a5a800640 1 -- 172.21.15.66:0/3036616659 shutdown_connections 2024-03-30T00:46:07.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.278+0000 7f2a5a800640 1 -- 172.21.15.66:0/3036616659 wait complete. 2024-03-30T00:46:07.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a5a800640 1 Processor -- start 2024-03-30T00:46:07.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a5a800640 1 -- start start 2024-03-30T00:46:07.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a5a800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a540cda90 0x7f2a54071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:07.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a5a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a540cde70 0x7f2a540c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:07.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a5a800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a54159c60 0x7f2a540c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:07.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a5a800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5414fd90 con 0x7f2a540cde70 2024-03-30T00:46:07.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a5a800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540c1b30 con 0x7f2a540cda90 2024-03-30T00:46:07.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a5a800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540ab240 con 0x7f2a54159c60 2024-03-30T00:46:07.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a540cde70 0x7f2a540c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:07.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a540cde70 0x7f2a540c10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38068/0 (socket says 172.21.15.66:38068) 2024-03-30T00:46:07.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4be00640 1 -- 172.21.15.66:0/3114811492 learned_addr learned my addr 172.21.15.66:0/3114811492 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:07.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a53400640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a540cda90 0x7f2a54071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:07.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a53e00640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a54159c60 0x7f2a540c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:07.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4be00640 1 -- 172.21.15.66:0/3114811492 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a54159c60 msgr2=0x7f2a540c15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4be00640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a54159c60 0x7f2a540c15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4be00640 1 -- 172.21.15.66:0/3114811492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a540cda90 msgr2=0x7f2a54071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4be00640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a540cda90 0x7f2a54071180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4be00640 1 -- 172.21.15.66:0/3114811492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4400b770 con 0x7f2a540cde70 2024-03-30T00:46:07.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a53e00640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a54159c60 0x7f2a540c15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:07.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4be00640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a540cde70 0x7f2a540c10b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e840 tx=0x7f2a3c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:07.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4b400640 1 -- 172.21.15.66:0/3114811492 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c00c8d0 con 0x7f2a540cde70 2024-03-30T00:46:07.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4b400640 1 -- 172.21.15.66:0/3114811492 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c00ca30 con 0x7f2a540cde70 2024-03-30T00:46:07.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a4b400640 1 -- 172.21.15.66:0/3114811492 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c00cdc0 con 0x7f2a540cde70 2024-03-30T00:46:07.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a5a800640 1 -- 172.21.15.66:0/3114811492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5414f4f0 con 0x7f2a540cde70 2024-03-30T00:46:07.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.282+0000 7f2a5a800640 1 -- 172.21.15.66:0/3114811492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5414fab0 con 0x7f2a540cde70 2024-03-30T00:46:07.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.286+0000 7f2a5a800640 1 -- 172.21.15.66:0/3114811492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a540ab3d0 con 0x7f2a540cde70 2024-03-30T00:46:07.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.286+0000 7f2a4b400640 1 -- 172.21.15.66:0/3114811492 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2a3c013410 con 0x7f2a540cde70 2024-03-30T00:46:07.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.286+0000 7f2a4b400640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a30066690 0x7f2a30068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:07.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.286+0000 7f2a4b400640 1 -- 172.21.15.66:0/3114811492 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f2a3c083d30 con 0x7f2a540cde70 2024-03-30T00:46:07.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.286+0000 7f2a53400640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a30066690 0x7f2a30068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:07.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.286+0000 7f2a53400640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a30066690 0x7f2a30068b50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2a440b1000 tx=0x7f2a440a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:07.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.290+0000 7f2a4b400640 1 -- 172.21.15.66:0/3114811492 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2a3c051340 con 0x7f2a540cde70 2024-03-30T00:46:07.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.506+0000 7f2a5a800640 1 -- 172.21.15.66:0/3114811492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f2a54066c30 con 0x7f2a540cde70 2024-03-30T00:46:07.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.762+0000 7f2a4b400640 1 -- 172.21.15.66:0/3114811492 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v324) v1 ==== 166+0+0 (secure 0 0 0) 0x7f2a3c054960 con 0x7f2a540cde70 2024-03-30T00:46:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 -- 172.21.15.66:0/3114811492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a30066690 msgr2=0x7f2a30068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a30066690 0x7f2a30068b50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2a440b1000 tx=0x7f2a440a9070 comp rx=0 tx=0).stop 2024-03-30T00:46:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 -- 172.21.15.66:0/3114811492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a540cde70 msgr2=0x7f2a540c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a540cde70 0x7f2a540c10b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e840 tx=0x7f2a3c00ed00 comp rx=0 tx=0).stop 2024-03-30T00:46:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 -- 172.21.15.66:0/3114811492 shutdown_connections 2024-03-30T00:46:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a54159c60 0x7f2a540c15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a30066690 0x7f2a30068b50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a540cde70 0x7f2a540c10b0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 --2- 172.21.15.66:0/3114811492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a540cda90 0x7f2a54071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 -- 172.21.15.66:0/3114811492 >> 172.21.15.66:0/3114811492 conn(0x7f2a54069f60 msgr2=0x7f2a540afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:07.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 -- 172.21.15.66:0/3114811492 shutdown_connections 2024-03-30T00:46:07.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.766+0000 7f2a5a800640 1 -- 172.21.15.66:0/3114811492 wait complete. 2024-03-30T00:46:07.795 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-30T00:46:07.795 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-30T00:46:07.795 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-30T00:46:07.796 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:46:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f5375800640 1 Processor -- start 2024-03-30T00:46:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f5375800640 1 -- start start 2024-03-30T00:46:07.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f5375800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f537015aa40 0x7f53700cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:07.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f5375800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537015eab0 con 0x7f537015e5d0 2024-03-30T00:46:07.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f5375800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537015ed90 con 0x7f537015ae20 2024-03-30T00:46:07.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f5375800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53700cec50 con 0x7f537015aa40 2024-03-30T00:46:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f536f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f537015e5d0 0x7f53700ce540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52970/0 (socket says 172.21.15.66:52970) 2024-03-30T00:46:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f536f400640 1 -- 172.21.15.66:0/1747043608 learned_addr learned my addr 172.21.15.66:0/1747043608 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f536fe00640 1 --2- 172.21.15.66:0/1747043608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f537015aa40 0x7f53700cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f536ea00640 1 -- 172.21.15.66:0/1747043608 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671306320 0 0) 0x7f537015eab0 con 0x7f537015e5d0 2024-03-30T00:46:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f536ea00640 1 -- 172.21.15.66:0/1747043608 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5354003530 con 0x7f537015e5d0 2024-03-30T00:46:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f536ea00640 1 -- 172.21.15.66:0/1747043608 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891619121 0 0) 0x7f537015ed90 con 0x7f537015ae20 2024-03-30T00:46:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f536ea00640 1 -- 172.21.15.66:0/1747043608 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f537015eab0 con 0x7f537015ae20 2024-03-30T00:46:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f536fe00640 1 -- 172.21.15.66:0/1747043608 >> v1:172.21.15.156:6789/0 conn(0x7f537015ae20 legacy=0x7f53700ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f536fe00640 1 -- 172.21.15.66:0/1747043608 >> v1:172.21.15.66:6789/0 conn(0x7f537015e5d0 legacy=0x7f53700ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.930+0000 7f536fe00640 1 -- 172.21.15.66:0/1747043608 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537015f070 con 0x7f537015aa40 2024-03-30T00:46:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f536fe00640 1 --2- 172.21.15.66:0/1747043608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f537015aa40 0x7f53700cb500 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53600097f0 tx=0x7f536001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe3af02025ede1cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f536ea00640 1 -- 172.21.15.66:0/1747043608 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53600ad680 con 0x7f537015aa40 2024-03-30T00:46:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f536ea00640 1 -- 172.21.15.66:0/1747043608 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53600ad7e0 con 0x7f537015aa40 2024-03-30T00:46:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f536ea00640 1 -- 172.21.15.66:0/1747043608 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53600adb50 con 0x7f537015aa40 2024-03-30T00:46:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f5375800640 1 -- 172.21.15.66:0/1747043608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f537015aa40 msgr2=0x7f53700cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f5375800640 1 --2- 172.21.15.66:0/1747043608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f537015aa40 0x7f53700cb500 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53600097f0 tx=0x7f536001c450 comp rx=0 tx=0).stop 2024-03-30T00:46:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f5375800640 1 -- 172.21.15.66:0/1747043608 shutdown_connections 2024-03-30T00:46:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f5375800640 1 --2- 172.21.15.66:0/1747043608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f537015aa40 0x7f53700cb500 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f5375800640 1 -- 172.21.15.66:0/1747043608 >> 172.21.15.66:0/1747043608 conn(0x7f537006e0d0 msgr2=0x7f537006e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f5375800640 1 -- 172.21.15.66:0/1747043608 shutdown_connections 2024-03-30T00:46:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f5375800640 1 -- 172.21.15.66:0/1747043608 wait complete. 2024-03-30T00:46:07.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.934+0000 7f5375800640 1 Processor -- start 2024-03-30T00:46:07.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5375800640 1 -- start start 2024-03-30T00:46:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5375800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f537015aa40 0x7f53700c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5375800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f537015ae20 0x7f5370151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5375800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537015e5d0 0x7f53701556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5375800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53700c8ee0 con 0x7f537015aa40 2024-03-30T00:46:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5375800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53700c9840 con 0x7f537015e5d0 2024-03-30T00:46:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5375800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53700c8260 con 0x7f537015ae20 2024-03-30T00:46:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f536fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f537015aa40 0x7f53700c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f536fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f537015aa40 0x7f53700c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38072/0 (socket says 172.21.15.66:38072) 2024-03-30T00:46:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f536fe00640 1 -- 172.21.15.66:0/836888384 learned_addr learned my addr 172.21.15.66:0/836888384 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f536f400640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f537015ae20 0x7f5370151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5374e00640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537015e5d0 0x7f53701556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f536fe00640 1 -- 172.21.15.66:0/836888384 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f537015ae20 msgr2=0x7f5370151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f536fe00640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f537015ae20 0x7f5370151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f536fe00640 1 -- 172.21.15.66:0/836888384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537015e5d0 msgr2=0x7f53701556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f536fe00640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537015e5d0 0x7f53701556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f536fe00640 1 -- 172.21.15.66:0/836888384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5360009420 con 0x7f537015aa40 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5374e00640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537015e5d0 0x7f53701556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f536fe00640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f537015aa40 0x7f53700c9300 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f53600097f0 tx=0x7f53600191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5367e00640 1 -- 172.21.15.66:0/836888384 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53600add60 con 0x7f537015aa40 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5367e00640 1 -- 172.21.15.66:0/836888384 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53600ad640 con 0x7f537015aa40 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5367e00640 1 -- 172.21.15.66:0/836888384 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53600b6600 con 0x7f537015aa40 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5375800640 1 -- 172.21.15.66:0/836888384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53700c8540 con 0x7f537015aa40 2024-03-30T00:46:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5375800640 1 -- 172.21.15.66:0/836888384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53700c8b00 con 0x7f537015aa40 2024-03-30T00:46:07.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.938+0000 7f5375800640 1 -- 172.21.15.66:0/836888384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53700c9150 con 0x7f537015aa40 2024-03-30T00:46:07.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.942+0000 7f5367e00640 1 -- 172.21.15.66:0/836888384 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f53600b4040 con 0x7f537015aa40 2024-03-30T00:46:07.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.942+0000 7f5367e00640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53500666c0 0x7f5350068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:07.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.942+0000 7f5367e00640 1 -- 172.21.15.66:0/836888384 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f536011d300 con 0x7f537015aa40 2024-03-30T00:46:07.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.942+0000 7f536f400640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53500666c0 0x7f5350068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:07.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.942+0000 7f536f400640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53500666c0 0x7f5350068b80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5370159b60 tx=0x7f535c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:07.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:07.946+0000 7f5367e00640 1 -- 172.21.15.66:0/836888384 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f53600ea6a0 con 0x7f537015aa40 2024-03-30T00:46:08.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.106+0000 7f5375800640 1 -- 172.21.15.66:0/836888384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5370073cf0 con 0x7f537015aa40 2024-03-30T00:46:08.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.106+0000 7f5367e00640 1 -- 172.21.15.66:0/836888384 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+12856 (secure 0 0 0) 0x7f53600edcc0 con 0x7f537015aa40 2024-03-30T00:46:08.117 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:08.117 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":86,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:46:06.744583+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6837/2050987583":"2024-03-31T00:46:03.718234+0000","172.21.15.66:6835/2050987583":"2024-03-31T00:46:03.718234+0000","172.21.15.156:6835/2089872680":"2024-03-31T00:44:35.079618+0000","172.21.15.156:6833/2089872680":"2024-03-31T00:44:35.079618+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:46:08.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 -- 172.21.15.66:0/836888384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53500666c0 msgr2=0x7f5350068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53500666c0 0x7f5350068b80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5370159b60 tx=0x7f535c009040 comp rx=0 tx=0).stop 2024-03-30T00:46:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 -- 172.21.15.66:0/836888384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f537015aa40 msgr2=0x7f53700c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f537015aa40 0x7f53700c9300 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f53600097f0 tx=0x7f53600191f0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 -- 172.21.15.66:0/836888384 shutdown_connections 2024-03-30T00:46:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f537015e5d0 0x7f53701556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f537015ae20 0x7f5370151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53500666c0 0x7f5350068b80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 --2- 172.21.15.66:0/836888384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f537015aa40 0x7f53700c9300 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 -- 172.21.15.66:0/836888384 >> 172.21.15.66:0/836888384 conn(0x7f537006e0d0 msgr2=0x7f5370159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 -- 172.21.15.66:0/836888384 shutdown_connections 2024-03-30T00:46:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.110+0000 7f5375800640 1 -- 172.21.15.66:0/836888384 wait complete. 2024-03-30T00:46:08.137 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:46:08.138 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:46:08.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a08c00640 1 Processor -- start 2024-03-30T00:46:08.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a08c00640 1 -- start start 2024-03-30T00:46:08.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a08c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a04099430 0x7f4a040ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a08c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4a040cf0c0 0x7f4a040c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a08c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a040c4600 con 0x7f4a0415a810 2024-03-30T00:46:08.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a08c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a040cf550 con 0x7f4a040cf0c0 2024-03-30T00:46:08.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a08c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040cc510 con 0x7f4a04099430 2024-03-30T00:46:08.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a02800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4a040cf0c0 0x7f4a040c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:08.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a01e00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4a0415a810 0x7f4a040ce380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:37052/0 (socket says 172.21.15.66:37052) 2024-03-30T00:46:08.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f49f9400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a04099430 0x7f4a040ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:08.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a02800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4a040cf0c0 0x7f4a040c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:46110/0 (socket says 172.21.15.66:46110) 2024-03-30T00:46:08.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a01e00640 1 -- 172.21.15.66:0/180749282 learned_addr learned my addr 172.21.15.66:0/180749282 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:08.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f49f9400640 1 -- 172.21.15.66:0/180749282 >> v2:172.21.15.156:3300/0 conn(0x7f4a040cf0c0 msgr2=0x7f4a040c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:08.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f49f9400640 1 --2- 172.21.15.66:0/180749282 >> v2:172.21.15.156:3300/0 conn(0x7f4a040cf0c0 0x7f4a040c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f49f9400640 1 -- 172.21.15.66:0/180749282 >> v1:172.21.15.156:6789/0 conn(0x7f4a0415a810 legacy=0x7f4a040ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f49f9400640 1 -- 172.21.15.66:0/180749282 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a040998c0 con 0x7f4a04099430 2024-03-30T00:46:08.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.274+0000 7f4a02800640 1 --2- 172.21.15.66:0/180749282 >> v2:172.21.15.156:3300/0 conn(0x7f4a040cf0c0 0x7f4a040c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:46:08.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f49f9400640 1 --2- 172.21.15.66:0/180749282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a04099430 0x7f4a040ceb80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0097c0 tx=0x7f49ec01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02148ff91fd2dec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f4a01400640 1 -- 172.21.15.66:0/180749282 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0a8990 con 0x7f4a04099430 2024-03-30T00:46:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f4a01400640 1 -- 172.21.15.66:0/180749282 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec0a8af0 con 0x7f4a04099430 2024-03-30T00:46:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f4a01400640 1 -- 172.21.15.66:0/180749282 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0a8e80 con 0x7f4a04099430 2024-03-30T00:46:08.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f4a08c00640 1 -- 172.21.15.66:0/180749282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a04099430 msgr2=0x7f4a040ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f4a08c00640 1 --2- 172.21.15.66:0/180749282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a04099430 0x7f4a040ceb80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0097c0 tx=0x7f49ec01ec90 comp rx=0 tx=0).stop 2024-03-30T00:46:08.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f4a08c00640 1 -- 172.21.15.66:0/180749282 shutdown_connections 2024-03-30T00:46:08.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f4a08c00640 1 --2- 172.21.15.66:0/180749282 >> v2:172.21.15.156:3300/0 conn(0x7f4a040cf0c0 0x7f4a040c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f4a08c00640 1 --2- 172.21.15.66:0/180749282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a04099430 0x7f4a040ceb80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f4a08c00640 1 -- 172.21.15.66:0/180749282 >> 172.21.15.66:0/180749282 conn(0x7f4a040b2090 msgr2=0x7f4a040b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:08.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f4a08c00640 1 -- 172.21.15.66:0/180749282 shutdown_connections 2024-03-30T00:46:08.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.278+0000 7f4a08c00640 1 -- 172.21.15.66:0/180749282 wait complete. 2024-03-30T00:46:08.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f4a08c00640 1 Processor -- start 2024-03-30T00:46:08.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f4a08c00640 1 -- start start 2024-03-30T00:46:08.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f4a08c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a040cf0c0 0x7f4a040c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f4a08c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a0415a810 0x7f4a040c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f4a08c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a04154020 0x7f4a041f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f4a08c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0415c920 con 0x7f4a0415a810 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f4a08c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040700f0 con 0x7f4a040cf0c0 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f4a08c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040c7b40 con 0x7f4a04154020 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49f9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a0415a810 0x7f4a040c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49f9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a0415a810 0x7f4a040c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38088/0 (socket says 172.21.15.66:38088) 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49f9400640 1 -- 172.21.15.66:0/1469102830 learned_addr learned my addr 172.21.15.66:0/1469102830 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49f9400640 1 -- 172.21.15.66:0/1469102830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a04154020 msgr2=0x7f4a041f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49f9400640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a04154020 0x7f4a041f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49f9400640 1 -- 172.21.15.66:0/1469102830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a040cf0c0 msgr2=0x7f4a040c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49f9400640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a040cf0c0 0x7f4a040c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49f9400640 1 -- 172.21.15.66:0/1469102830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc009a70 con 0x7f4a0415a810 2024-03-30T00:46:08.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49f9400640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a0415a810 0x7f4a040c9d30 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f49ec01b8a0 tx=0x7f49ec01ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:08.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49faa00640 1 -- 172.21.15.66:0/1469102830 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0b1d40 con 0x7f4a0415a810 2024-03-30T00:46:08.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49faa00640 1 -- 172.21.15.66:0/1469102830 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec0b9410 con 0x7f4a0415a810 2024-03-30T00:46:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f49faa00640 1 -- 172.21.15.66:0/1469102830 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0b1ea0 con 0x7f4a0415a810 2024-03-30T00:46:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f4a08c00640 1 -- 172.21.15.66:0/1469102830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec009420 con 0x7f4a0415a810 2024-03-30T00:46:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.282+0000 7f4a08c00640 1 -- 172.21.15.66:0/1469102830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0415cdf0 con 0x7f4a0415a810 2024-03-30T00:46:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.286+0000 7f4a08c00640 1 -- 172.21.15.66:0/1469102830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a04066c30 con 0x7f4a0415a810 2024-03-30T00:46:08.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.286+0000 7f49faa00640 1 -- 172.21.15.66:0/1469102830 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f49ec0b7040 con 0x7f4a0415a810 2024-03-30T00:46:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.286+0000 7f49faa00640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f49e0066690 0x7f49e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.286+0000 7f49faa00640 1 -- 172.21.15.66:0/1469102830 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f49ec121920 con 0x7f4a0415a810 2024-03-30T00:46:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.286+0000 7f4a01e00640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f49e0066690 0x7f49e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:08.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.286+0000 7f4a01e00640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f49e0066690 0x7f49e0068b50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f49f4000f70 tx=0x7f49f4007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:08.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.290+0000 7f49faa00640 1 -- 172.21.15.66:0/1469102830 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f49ec0eed50 con 0x7f4a0415a810 2024-03-30T00:46:08.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.450+0000 7f4a08c00640 1 -- 172.21.15.66:0/1469102830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a040c7cf0 con 0x7f4a0415a810 2024-03-30T00:46:08.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.450+0000 7f49faa00640 1 -- 172.21.15.66:0/1469102830 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49ec0b50a0 con 0x7f4a0415a810 2024-03-30T00:46:08.460 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 -- 172.21.15.66:0/1469102830 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f49e0066690 msgr2=0x7f49e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f49e0066690 0x7f49e0068b50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f49f4000f70 tx=0x7f49f4007a90 comp rx=0 tx=0).stop 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 -- 172.21.15.66:0/1469102830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a0415a810 msgr2=0x7f4a040c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a0415a810 0x7f4a040c9d30 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f49ec01b8a0 tx=0x7f49ec01ba30 comp rx=0 tx=0).stop 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 -- 172.21.15.66:0/1469102830 shutdown_connections 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a04154020 0x7f4a041f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f49e0066690 0x7f49e0068b50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a0415a810 0x7f4a040c9d30 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 --2- 172.21.15.66:0/1469102830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a040cf0c0 0x7f4a040c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 -- 172.21.15.66:0/1469102830 >> 172.21.15.66:0/1469102830 conn(0x7f4a040b2090 msgr2=0x7f4a040c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 -- 172.21.15.66:0/1469102830 shutdown_connections 2024-03-30T00:46:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.454+0000 7f4a08c00640 1 -- 172.21.15.66:0/1469102830 wait complete. 2024-03-30T00:46:08.481 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-03-30T00:46:08.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f1441e00640 1 Processor -- start 2024-03-30T00:46:08.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f1441e00640 1 -- start start 2024-03-30T00:46:08.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f1441e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c0cc560 0x7f143c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f1441e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f143c06f8e0 0x7f143c06e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f1441e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143c06b0b0 con 0x7f143c06e630 2024-03-30T00:46:08.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f1441e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143c160640 con 0x7f143c06f8e0 2024-03-30T00:46:08.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f1441e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c06eaa0 con 0x7f143c0cc560 2024-03-30T00:46:08.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f143aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c0cc560 0x7f143c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:08.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f1433e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f143c06f8e0 0x7f143c06e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:08.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f143aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c0cc560 0x7f143c0d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:59928/0 (socket says 172.21.15.66:59928) 2024-03-30T00:46:08.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f143aa00640 1 -- 172.21.15.66:0/211254898 learned_addr learned my addr 172.21.15.66:0/211254898 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:08.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f143aa00640 1 -- 172.21.15.66:0/211254898 >> v2:172.21.15.156:3300/0 conn(0x7f143c06f8e0 msgr2=0x7f143c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:08.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f143aa00640 1 --2- 172.21.15.66:0/211254898 >> v2:172.21.15.156:3300/0 conn(0x7f143c06f8e0 0x7f143c06e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f143aa00640 1 -- 172.21.15.66:0/211254898 >> v1:172.21.15.156:6789/0 conn(0x7f143c06e630 legacy=0x7f143c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.614+0000 7f143aa00640 1 -- 172.21.15.66:0/211254898 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c06b390 con 0x7f143c0cc560 2024-03-30T00:46:08.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f143aa00640 1 --2- 172.21.15.66:0/211254898 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c0cc560 0x7f143c0d1060 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f142c009870 tx=0x7f142c01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47e7fce67f7a5e7c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:08.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f143a000640 1 -- 172.21.15.66:0/211254898 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0ad660 con 0x7f143c0cc560 2024-03-30T00:46:08.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f143a000640 1 -- 172.21.15.66:0/211254898 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c0ad7c0 con 0x7f143c0cc560 2024-03-30T00:46:08.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f143a000640 1 -- 172.21.15.66:0/211254898 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0adb50 con 0x7f143c0cc560 2024-03-30T00:46:08.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f1441e00640 1 -- 172.21.15.66:0/211254898 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c0cc560 msgr2=0x7f143c0d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f1441e00640 1 --2- 172.21.15.66:0/211254898 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c0cc560 0x7f143c0d1060 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f142c009870 tx=0x7f142c01c530 comp rx=0 tx=0).stop 2024-03-30T00:46:08.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f1441e00640 1 -- 172.21.15.66:0/211254898 shutdown_connections 2024-03-30T00:46:08.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f1441e00640 1 --2- 172.21.15.66:0/211254898 >> v2:172.21.15.156:3300/0 conn(0x7f143c06f8e0 0x7f143c06e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f1441e00640 1 --2- 172.21.15.66:0/211254898 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c0cc560 0x7f143c0d1060 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f1441e00640 1 -- 172.21.15.66:0/211254898 >> 172.21.15.66:0/211254898 conn(0x7f143c0d0540 msgr2=0x7f143c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:08.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f1441e00640 1 -- 172.21.15.66:0/211254898 shutdown_connections 2024-03-30T00:46:08.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.618+0000 7f1441e00640 1 -- 172.21.15.66:0/211254898 wait complete. 2024-03-30T00:46:08.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1441e00640 1 Processor -- start 2024-03-30T00:46:08.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1441e00640 1 -- start start 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1441e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06e630 0x7f143c155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1441e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c06f8e0 0x7f143c1590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1441e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f143c0cc560 0x7f143c0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1441e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c15a7f0 con 0x7f143c0cc560 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1441e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c159b70 con 0x7f143c06e630 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1441e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c159e50 con 0x7f143c06f8e0 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f143b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f143c0cc560 0x7f143c0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f143b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f143c0cc560 0x7f143c0cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38096/0 (socket says 172.21.15.66:38096) 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f143aa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06e630 0x7f143c155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f143b400640 1 -- 172.21.15.66:0/3104616463 learned_addr learned my addr 172.21.15.66:0/3104616463 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1433e00640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c06f8e0 0x7f143c1590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f143b400640 1 -- 172.21.15.66:0/3104616463 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c06f8e0 msgr2=0x7f143c1590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f143b400640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c06f8e0 0x7f143c1590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f143b400640 1 -- 172.21.15.66:0/3104616463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06e630 msgr2=0x7f143c155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:08.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f143b400640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06e630 0x7f143c155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:08.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f143b400640 1 -- 172.21.15.66:0/3104616463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c0094d0 con 0x7f143c0cc560 2024-03-30T00:46:08.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f143aa00640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06e630 0x7f143c155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:46:08.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f143b400640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f143c0cc560 0x7f143c0cab70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f143400c730 tx=0x7f143400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:08.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1433400640 1 -- 172.21.15.66:0/3104616463 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434003b80 con 0x7f143c0cc560 2024-03-30T00:46:08.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1433400640 1 -- 172.21.15.66:0/3104616463 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1434003ce0 con 0x7f143c0cc560 2024-03-30T00:46:08.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1441e00640 1 -- 172.21.15.66:0/3104616463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143c15a130 con 0x7f143c0cc560 2024-03-30T00:46:08.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1433400640 1 -- 172.21.15.66:0/3104616463 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143400aa30 con 0x7f143c0cc560 2024-03-30T00:46:08.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.622+0000 7f1441e00640 1 -- 172.21.15.66:0/3104616463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143c1560f0 con 0x7f143c0cc560 2024-03-30T00:46:08.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.626+0000 7f1441e00640 1 -- 172.21.15.66:0/3104616463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143c15a930 con 0x7f143c0cc560 2024-03-30T00:46:08.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.626+0000 7f1433400640 1 -- 172.21.15.66:0/3104616463 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f143400ab90 con 0x7f143c0cc560 2024-03-30T00:46:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.626+0000 7f1433400640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1418066690 0x7f1418068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.626+0000 7f1433400640 1 -- 172.21.15.66:0/3104616463 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f1434083cd0 con 0x7f143c0cc560 2024-03-30T00:46:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.626+0000 7f143aa00640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1418066690 0x7f1418068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.626+0000 7f143aa00640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1418066690 0x7f1418068b50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f142c0af000 tx=0x7f142c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:08.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.630+0000 7f1433400640 1 -- 172.21.15.66:0/3104616463 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f143c15a930 con 0x7f143c0cc560 2024-03-30T00:46:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:08.850+0000 7f1441e00640 1 -- 172.21.15.66:0/3104616463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f143c15a930 con 0x7f143c0cc560 2024-03-30T00:46:09.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.770+0000 7f1433400640 1 -- 172.21.15.66:0/3104616463 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v326) v1 ==== 170+0+0 (secure 0 0 0) 0x7f143c15a930 con 0x7f143c0cc560 2024-03-30T00:46:09.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 -- 172.21.15.66:0/3104616463 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1418066690 msgr2=0x7f1418068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:09.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1418066690 0x7f1418068b50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f142c0af000 tx=0x7f142c0a7000 comp rx=0 tx=0).stop 2024-03-30T00:46:09.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 -- 172.21.15.66:0/3104616463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f143c0cc560 msgr2=0x7f143c0cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:09.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f143c0cc560 0x7f143c0cab70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f143400c730 tx=0x7f143400cbf0 comp rx=0 tx=0).stop 2024-03-30T00:46:09.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 -- 172.21.15.66:0/3104616463 shutdown_connections 2024-03-30T00:46:09.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1418066690 0x7f1418068b50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:09.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f143c0cc560 0x7f143c0cab70 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:09.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f143c06f8e0 0x7f143c1590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:09.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 --2- 172.21.15.66:0/3104616463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f143c06e630 0x7f143c155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:09.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 -- 172.21.15.66:0/3104616463 >> 172.21.15.66:0/3104616463 conn(0x7f143c0d0540 msgr2=0x7f143c0ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:09.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 -- 172.21.15.66:0/3104616463 shutdown_connections 2024-03-30T00:46:09.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.774+0000 7f1441e00640 1 -- 172.21.15.66:0/3104616463 wait complete. 2024-03-30T00:46:09.805 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-30T00:46:09.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.942+0000 7fcfdd000640 1 Processor -- start 2024-03-30T00:46:09.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfdd000640 1 -- start start 2024-03-30T00:46:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfdd000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcfd80ce520 0x7fcfd80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfdd000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcfd80cabb0 0x7fcfd80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfdd000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd806d070 con 0x7fcfd815a640 2024-03-30T00:46:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfdd000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd80cbba0 con 0x7fcfd80cabb0 2024-03-30T00:46:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfdd000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd80cee70 con 0x7fcfd80ce520 2024-03-30T00:46:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfd6800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcfd80cabb0 0x7fcfd80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfd6800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcfd80cabb0 0x7fcfd80cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44136/0 (socket says 172.21.15.66:44136) 2024-03-30T00:46:09.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfd6800640 1 -- 172.21.15.66:0/4161348583 learned_addr learned my addr 172.21.15.66:0/4161348583 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:09.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfcfe00640 1 --2- 172.21.15.66:0/4161348583 >> v2:172.21.15.156:3300/0 conn(0x7fcfd80ce520 0x7fcfd80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:09.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfd6800640 1 -- 172.21.15.66:0/4161348583 >> v2:172.21.15.156:3300/0 conn(0x7fcfd80ce520 msgr2=0x7fcfd80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:09.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfd6800640 1 --2- 172.21.15.66:0/4161348583 >> v2:172.21.15.156:3300/0 conn(0x7fcfd80ce520 0x7fcfd80ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:09.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfd6800640 1 -- 172.21.15.66:0/4161348583 >> v1:172.21.15.156:6789/0 conn(0x7fcfd815a640 legacy=0x7fcfd806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:09.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.946+0000 7fcfd6800640 1 -- 172.21.15.66:0/4161348583 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80cb8c0 con 0x7fcfd80cabb0 2024-03-30T00:46:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfd6800640 1 --2- 172.21.15.66:0/4161348583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd80cabb0 0x7fcfd80cafe0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80c88d0 tx=0x7fcfd001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957ef7bbaf7b353c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfd5400640 1 -- 172.21.15.66:0/4161348583 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00a5840 con 0x7fcfd80cabb0 2024-03-30T00:46:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfd5400640 1 -- 172.21.15.66:0/4161348583 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd00a59a0 con 0x7fcfd80cabb0 2024-03-30T00:46:09.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfd5400640 1 -- 172.21.15.66:0/4161348583 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00a5d30 con 0x7fcfd80cabb0 2024-03-30T00:46:09.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfdd000640 1 -- 172.21.15.66:0/4161348583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd80cabb0 msgr2=0x7fcfd80cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:09.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfdd000640 1 --2- 172.21.15.66:0/4161348583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd80cabb0 0x7fcfd80cafe0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80c88d0 tx=0x7fcfd001c980 comp rx=0 tx=0).stop 2024-03-30T00:46:09.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfdd000640 1 -- 172.21.15.66:0/4161348583 shutdown_connections 2024-03-30T00:46:09.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfdd000640 1 --2- 172.21.15.66:0/4161348583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd80cabb0 0x7fcfd80cafe0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:09.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfdd000640 1 --2- 172.21.15.66:0/4161348583 >> v2:172.21.15.156:3300/0 conn(0x7fcfd80ce520 0x7fcfd80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:09.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfdd000640 1 -- 172.21.15.66:0/4161348583 >> 172.21.15.66:0/4161348583 conn(0x7fcfd8069f60 msgr2=0x7fcfd806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:09.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfdd000640 1 -- 172.21.15.66:0/4161348583 shutdown_connections 2024-03-30T00:46:09.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfdd000640 1 -- 172.21.15.66:0/4161348583 wait complete. 2024-03-30T00:46:09.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfdd000640 1 Processor -- start 2024-03-30T00:46:09.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.950+0000 7fcfdd000640 1 -- start start 2024-03-30T00:46:09.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfdd000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd80cabb0 0x7fcfd8155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:09.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfdd000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd80ce520 0x7fcfd8156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:09.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfdd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd815a640 0x7fcfd80cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:09.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfdd000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8156cd0 con 0x7fcfd815a640 2024-03-30T00:46:09.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfdd000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8156150 con 0x7fcfd80ce520 2024-03-30T00:46:09.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfdd000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8156430 con 0x7fcfd80cabb0 2024-03-30T00:46:09.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfd6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd815a640 0x7fcfd80cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:09.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfd6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd815a640 0x7fcfd80cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44140/0 (socket says 172.21.15.66:44140) 2024-03-30T00:46:09.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfd6800640 1 -- 172.21.15.66:0/1051150604 learned_addr learned my addr 172.21.15.66:0/1051150604 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:09.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfcfe00640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd80ce520 0x7fcfd8156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:09.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfd5e00640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd80cabb0 0x7fcfd8155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:09.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfd6800640 1 -- 172.21.15.66:0/1051150604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd80cabb0 msgr2=0x7fcfd8155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:09.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfd6800640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd80cabb0 0x7fcfd8155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:09.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfd6800640 1 -- 172.21.15.66:0/1051150604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd80ce520 msgr2=0x7fcfd8156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:09.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfd6800640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd80ce520 0x7fcfd8156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:09.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfd6800640 1 -- 172.21.15.66:0/1051150604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd0009b30 con 0x7fcfd815a640 2024-03-30T00:46:09.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfd6800640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd815a640 0x7fcfd80cbec0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00af040 tx=0x7fcfd0019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:09.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfce000640 1 -- 172.21.15.66:0/1051150604 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00b8670 con 0x7fcfd815a640 2024-03-30T00:46:09.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfce000640 1 -- 172.21.15.66:0/1051150604 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd00b87d0 con 0x7fcfd815a640 2024-03-30T00:46:09.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfce000640 1 -- 172.21.15.66:0/1051150604 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00b8b80 con 0x7fcfd815a640 2024-03-30T00:46:09.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfdd000640 1 -- 172.21.15.66:0/1051150604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd8156710 con 0x7fcfd815a640 2024-03-30T00:46:09.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfdd000640 1 -- 172.21.15.66:0/1051150604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd80c9a70 con 0x7fcfd815a640 2024-03-30T00:46:09.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfdd000640 1 -- 172.21.15.66:0/1051150604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd81625a0 con 0x7fcfd815a640 2024-03-30T00:46:09.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.954+0000 7fcfce000640 1 -- 172.21.15.66:0/1051150604 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcfd00b8ce0 con 0x7fcfd815a640 2024-03-30T00:46:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.958+0000 7fcfce000640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcfb40666c0 0x7fcfb4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.958+0000 7fcfce000640 1 -- 172.21.15.66:0/1051150604 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fcfd011f6a0 con 0x7fcfd815a640 2024-03-30T00:46:09.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.958+0000 7fcfd5e00640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcfb40666c0 0x7fcfb4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:09.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.958+0000 7fcfd5e00640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcfb40666c0 0x7fcfb4068b80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80098e0 tx=0x7fcfc8009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:09.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:09.962+0000 7fcfce000640 1 -- 172.21.15.66:0/1051150604 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcfd81625a0 con 0x7fcfd815a640 2024-03-30T00:46:10.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.126+0000 7fcfdd000640 1 -- 172.21.15.66:0/1051150604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcfd81625a0 con 0x7fcfd815a640 2024-03-30T00:46:10.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.130+0000 7fcfce000640 1 -- 172.21.15.66:0/1051150604 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcfd81625a0 con 0x7fcfd815a640 2024-03-30T00:46:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 -- 172.21.15.66:0/1051150604 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcfb40666c0 msgr2=0x7fcfb4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcfb40666c0 0x7fcfb4068b80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80098e0 tx=0x7fcfc8009060 comp rx=0 tx=0).stop 2024-03-30T00:46:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 -- 172.21.15.66:0/1051150604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd815a640 msgr2=0x7fcfd80cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd815a640 0x7fcfd80cbec0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00af040 tx=0x7fcfd0019720 comp rx=0 tx=0).stop 2024-03-30T00:46:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 -- 172.21.15.66:0/1051150604 shutdown_connections 2024-03-30T00:46:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcfb40666c0 0x7fcfb4068b80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd815a640 0x7fcfd80cbec0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd80ce520 0x7fcfd8156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 --2- 172.21.15.66:0/1051150604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd80cabb0 0x7fcfd8155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 -- 172.21.15.66:0/1051150604 >> 172.21.15.66:0/1051150604 conn(0x7fcfd8069f60 msgr2=0x7fcfd80b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 -- 172.21.15.66:0/1051150604 shutdown_connections 2024-03-30T00:46:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.134+0000 7fcfdd000640 1 -- 172.21.15.66:0/1051150604 wait complete. 2024-03-30T00:46:10.160 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:46:10.160 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:46:10.251 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:46:10.329 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:46:10.329 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:46:10.329 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:46:10.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:46:10.338 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:46:10.338 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:46:10.347 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:46:10.348 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:46:10.348 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:46:10.388 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:46:10.388 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:46:10.388 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:46:10.402 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:46:10.402 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-30T00:46:10.402 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:46:10.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb343400640 1 Processor -- start 2024-03-30T00:46:10.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb343400640 1 -- start start 2024-03-30T00:46:10.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb343400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c15de30 0x7fb33c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:10.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb343400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb33c0af420 0x7fb33c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:10.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb343400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb33c0ce740 0x7fb33c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:10.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb343400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c0cf110 con 0x7fb33c0ce740 2024-03-30T00:46:10.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb343400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c0cbb90 con 0x7fb33c0af420 2024-03-30T00:46:10.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb343400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0ce1b0 con 0x7fb33c15de30 2024-03-30T00:46:10.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb341400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb33c0ce740 0x7fb33c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb341400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb33c0ce740 0x7fb33c0cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44142/0 (socket says 172.21.15.66:44142) 2024-03-30T00:46:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb341400640 1 -- 172.21.15.66:0/4009901831 learned_addr learned my addr 172.21.15.66:0/4009901831 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb340a00640 1 --2- 172.21.15.66:0/4009901831 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c15de30 0x7fb33c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb333e00640 1 --2- 172.21.15.66:0/4009901831 >> v2:172.21.15.156:3300/0 conn(0x7fb33c0af420 0x7fb33c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb341400640 1 -- 172.21.15.66:0/4009901831 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c15de30 msgr2=0x7fb33c0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb341400640 1 --2- 172.21.15.66:0/4009901831 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c15de30 0x7fb33c0cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb341400640 1 -- 172.21.15.66:0/4009901831 >> v2:172.21.15.156:3300/0 conn(0x7fb33c0af420 msgr2=0x7fb33c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb341400640 1 --2- 172.21.15.66:0/4009901831 >> v2:172.21.15.156:3300/0 conn(0x7fb33c0af420 0x7fb33c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb341400640 1 -- 172.21.15.66:0/4009901831 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c0afa10 con 0x7fb33c0ce740 2024-03-30T00:46:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb333e00640 1 --2- 172.21.15.66:0/4009901831 >> v2:172.21.15.156:3300/0 conn(0x7fb33c0af420 0x7fb33c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:10.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.566+0000 7fb340a00640 1 --2- 172.21.15.66:0/4009901831 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c15de30 0x7fb33c0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:10.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb341400640 1 --2- 172.21.15.66:0/4009901831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33c0ce740 0x7fb33c0cebd0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0b3100 tx=0x7fb33401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36378c971900a38f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:10.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb33be00640 1 -- 172.21.15.66:0/4009901831 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340a57a0 con 0x7fb33c0ce740 2024-03-30T00:46:10.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb33be00640 1 -- 172.21.15.66:0/4009901831 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3340a5900 con 0x7fb33c0ce740 2024-03-30T00:46:10.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb33be00640 1 -- 172.21.15.66:0/4009901831 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340a5c90 con 0x7fb33c0ce740 2024-03-30T00:46:10.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb343400640 1 -- 172.21.15.66:0/4009901831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33c0ce740 msgr2=0x7fb33c0cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:10.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb343400640 1 --2- 172.21.15.66:0/4009901831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33c0ce740 0x7fb33c0cebd0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0b3100 tx=0x7fb33401c900 comp rx=0 tx=0).stop 2024-03-30T00:46:10.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb343400640 1 -- 172.21.15.66:0/4009901831 shutdown_connections 2024-03-30T00:46:10.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb343400640 1 --2- 172.21.15.66:0/4009901831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33c0ce740 0x7fb33c0cebd0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb343400640 1 --2- 172.21.15.66:0/4009901831 >> v2:172.21.15.156:3300/0 conn(0x7fb33c0af420 0x7fb33c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb343400640 1 --2- 172.21.15.66:0/4009901831 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c15de30 0x7fb33c0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb343400640 1 -- 172.21.15.66:0/4009901831 >> 172.21.15.66:0/4009901831 conn(0x7fb33c0d0540 msgr2=0x7fb33c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:10.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb343400640 1 -- 172.21.15.66:0/4009901831 shutdown_connections 2024-03-30T00:46:10.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb343400640 1 -- 172.21.15.66:0/4009901831 wait complete. 2024-03-30T00:46:10.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb343400640 1 Processor -- start 2024-03-30T00:46:10.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.570+0000 7fb343400640 1 -- start start 2024-03-30T00:46:10.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb343400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0af420 0x7fb33c0b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:10.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb343400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0ce740 0x7fb33c15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:10.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb343400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33c15de30 0x7fb33c14b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:10.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb343400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0b2aa0 con 0x7fb33c15de30 2024-03-30T00:46:10.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb343400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0b2ec0 con 0x7fb33c0ce740 2024-03-30T00:46:10.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb343400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0b1e20 con 0x7fb33c0af420 2024-03-30T00:46:10.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb341400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33c15de30 0x7fb33c14b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:10.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb341400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33c15de30 0x7fb33c14b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44150/0 (socket says 172.21.15.66:44150) 2024-03-30T00:46:10.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb341400640 1 -- 172.21.15.66:0/3874834666 learned_addr learned my addr 172.21.15.66:0/3874834666 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:10.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb333e00640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0ce740 0x7fb33c15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:10.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb340a00640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0af420 0x7fb33c0b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:10.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb341400640 1 -- 172.21.15.66:0/3874834666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0af420 msgr2=0x7fb33c0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:10.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb341400640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0af420 0x7fb33c0b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb341400640 1 -- 172.21.15.66:0/3874834666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0ce740 msgr2=0x7fb33c15a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:10.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb341400640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0ce740 0x7fb33c15a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb341400640 1 -- 172.21.15.66:0/3874834666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb334009b30 con 0x7fb33c15de30 2024-03-30T00:46:10.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb340a00640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0af420 0x7fb33c0b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:10.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb341400640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33c15de30 0x7fb33c14b940 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb33401d1a0 tx=0x7fb334019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:10.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb339600640 1 -- 172.21.15.66:0/3874834666 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340b8670 con 0x7fb33c15de30 2024-03-30T00:46:10.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb339600640 1 -- 172.21.15.66:0/3874834666 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3340b87d0 con 0x7fb33c15de30 2024-03-30T00:46:10.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb339600640 1 -- 172.21.15.66:0/3874834666 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340b8b80 con 0x7fb33c15de30 2024-03-30T00:46:10.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb343400640 1 -- 172.21.15.66:0/3874834666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c0b2100 con 0x7fb33c15de30 2024-03-30T00:46:10.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb343400640 1 -- 172.21.15.66:0/3874834666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c0b26c0 con 0x7fb33c15de30 2024-03-30T00:46:10.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.574+0000 7fb343400640 1 -- 172.21.15.66:0/3874834666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c0cdf60 con 0x7fb33c15de30 2024-03-30T00:46:10.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.578+0000 7fb339600640 1 -- 172.21.15.66:0/3874834666 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb3340b8ce0 con 0x7fb33c15de30 2024-03-30T00:46:10.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.578+0000 7fb339600640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb318066690 0x7fb318068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:10.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.578+0000 7fb339600640 1 -- 172.21.15.66:0/3874834666 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fb33411f720 con 0x7fb33c15de30 2024-03-30T00:46:10.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.578+0000 7fb340a00640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb318066690 0x7fb318068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:10.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.578+0000 7fb340a00640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb318066690 0x7fb318068b50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00a9b0 tx=0x7fb32c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:10.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.582+0000 7fb339600640 1 -- 172.21.15.66:0/3874834666 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb3340ecdc0 con 0x7fb33c15de30 2024-03-30T00:46:10.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.778+0000 7fb343400640 1 -- 172.21.15.66:0/3874834666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb33c0b2be0 con 0x7fb33c15de30 2024-03-30T00:46:10.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.778+0000 7fb339600640 1 -- 172.21.15.66:0/3874834666 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fb3340bcde0 con 0x7fb33c15de30 2024-03-30T00:46:10.788 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:10.788 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/254323195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":254323195},{"type":"v1","addr":"172.21.15.156:6835","nonce":254323195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2084095154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2084095154},{"type":"v1","addr":"172.21.15.156:6834","nonce":2084095154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1221930804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1221930804},{"type":"v1","addr":"172.21.15.66:6836","nonce":1221930804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":7629,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1808235916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1808235916},{"type":"v1","addr":"172.21.15.66:6837","nonce":1808235916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[]} 2024-03-30T00:46:10.788 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 53 2024-03-30T00:46:10.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 -- 172.21.15.66:0/3874834666 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb318066690 msgr2=0x7fb318068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:10.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb318066690 0x7fb318068b50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00a9b0 tx=0x7fb32c008040 comp rx=0 tx=0).stop 2024-03-30T00:46:10.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 -- 172.21.15.66:0/3874834666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33c15de30 msgr2=0x7fb33c14b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:10.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33c15de30 0x7fb33c14b940 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb33401d1a0 tx=0x7fb334019700 comp rx=0 tx=0).stop 2024-03-30T00:46:10.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 -- 172.21.15.66:0/3874834666 shutdown_connections 2024-03-30T00:46:10.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb318066690 0x7fb318068b50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb33c15de30 0x7fb33c14b940 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0ce740 0x7fb33c15a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 --2- 172.21.15.66:0/3874834666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0af420 0x7fb33c0b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:10.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 -- 172.21.15.66:0/3874834666 >> 172.21.15.66:0/3874834666 conn(0x7fb33c0d0540 msgr2=0x7fb33c0aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:10.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 -- 172.21.15.66:0/3874834666 shutdown_connections 2024-03-30T00:46:10.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:10.782+0000 7fb343400640 1 -- 172.21.15.66:0/3874834666 wait complete. 2024-03-30T00:46:10.810 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T00:46:10.810 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-30T00:46:10.811 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T00:46:10.811 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:46:10.811 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T00:46:10.811 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-30T00:46:10.811 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T00:46:10.811 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:46:10.811 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T00:46:10.812 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-30T00:46:10.812 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T00:46:10.812 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:46:10.812 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T00:46:10.812 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-30T00:46:10.812 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T00:46:10.812 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:46:10.822 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T00:46:10.822 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T00:46:10.823 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T00:46:10.823 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T00:46:10.826 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T00:46:10.826 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T00:46:10.827 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T00:46:10.827 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T00:46:10.867 INFO:tasks.ceph.mds.a:Started 2024-03-30T00:46:10.867 DEBUG:teuthology.parallel:result is None 2024-03-30T00:46:10.868 INFO:tasks.ceph.mds.c:Started 2024-03-30T00:46:10.869 DEBUG:teuthology.parallel:result is None 2024-03-30T00:46:10.877 INFO:tasks.ceph.mds.b:Started 2024-03-30T00:46:10.877 DEBUG:teuthology.parallel:result is None 2024-03-30T00:46:10.878 INFO:tasks.ceph.mds.d:Started 2024-03-30T00:46:10.878 DEBUG:teuthology.parallel:result is None 2024-03-30T00:46:10.879 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-30T00:46:10.953 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:46:10.954 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:46:10.969 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2024-03-30T00:46:10.972 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2024-03-30T00:46:11.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f506fc00640 1 Processor -- start 2024-03-30T00:46:11.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f506fc00640 1 -- start start 2024-03-30T00:46:11.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f506fc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50680cc560 0x7f50680d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:11.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f506fc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f506806f8e0 0x7f506806e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:11.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f506fc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506806b0b0 con 0x7f506806e630 2024-03-30T00:46:11.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f506fc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5068160640 con 0x7f506806f8e0 2024-03-30T00:46:11.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f506fc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506806eaa0 con 0x7f50680cc560 2024-03-30T00:46:11.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f506dc00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f506806e630 0x7f50680ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48960/0 (socket says 172.21.15.66:48960) 2024-03-30T00:46:11.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f506dc00640 1 -- 172.21.15.66:0/609161549 learned_addr learned my addr 172.21.15.66:0/609161549 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:11.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f506d200640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50680cc560 0x7f50680d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:11.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f505fe00640 1 --2- 172.21.15.66:0/609161549 >> v2:172.21.15.156:3300/0 conn(0x7f506806f8e0 0x7f506806e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:11.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666765091 0 0) 0x7f506806b0b0 con 0x7f506806e630 2024-03-30T00:46:11.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f504c003530 con 0x7f506806e630 2024-03-30T00:46:11.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2140985983 0 0) 0x7f504c003530 con 0x7f506806e630 2024-03-30T00:46:11.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50680cc560 msgr2=0x7f50680d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:11.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f5067e00640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50680cc560 0x7f50680d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:11.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 >> v2:172.21.15.156:3300/0 conn(0x7f506806f8e0 msgr2=0x7f506806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:11.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f5067e00640 1 --2- 172.21.15.66:0/609161549 >> v2:172.21.15.156:3300/0 conn(0x7f506806f8e0 0x7f506806e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:11.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506806b390 con 0x7f506806e630 2024-03-30T00:46:11.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f5060001a00 con 0x7f506806e630 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f505fe00640 1 --2- 172.21.15.66:0/609161549 >> v2:172.21.15.156:3300/0 conn(0x7f506806f8e0 0x7f506806e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.022+0000 7f506d200640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50680cc560 0x7f50680d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 >> v1:172.21.15.66:6789/0 conn(0x7f506806e630 legacy=0x7f50680ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f5067e00640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c003980 0x7f504c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f5067e00640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 0x7f504c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f5067e00640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50680cc560 0x7f504c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5068160640 con 0x7f504c0047d0 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506806eaa0 con 0x7f504c003980 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c00b990 con 0x7f50680cc560 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f505fe00640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 0x7f504c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f506d200640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c003980 0x7f504c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f506dc00640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50680cc560 0x7f504c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:11.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f505fe00640 1 -- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50680cc560 msgr2=0x7f504c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f505fe00640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50680cc560 0x7f504c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f505fe00640 1 -- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c003980 msgr2=0x7f504c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f505fe00640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c003980 0x7f504c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f505fe00640 1 -- 172.21.15.66:0/609161549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5058009420 con 0x7f504c0047d0 2024-03-30T00:46:11.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f506dc00640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50680cc560 0x7f504c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f505fe00640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 0x7f504c004c50 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5050009cb0 tx=0x7f505001cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46084a747cb7941 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500a5870 con 0x7f504c0047d0 2024-03-30T00:46:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50500a59d0 con 0x7f504c0047d0 2024-03-30T00:46:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.026+0000 7f5067e00640 1 -- 172.21.15.66:0/609161549 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500a5d80 con 0x7f504c0047d0 2024-03-30T00:46:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 -- 172.21.15.66:0/609161549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 msgr2=0x7f504c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 --2- 172.21.15.66:0/609161549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 0x7f504c004c50 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5050009cb0 tx=0x7f505001cc90 comp rx=0 tx=0).stop 2024-03-30T00:46:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506dc00640 1 -- 172.21.15.66:0/609161549 reap_dead start 2024-03-30T00:46:11.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 -- 172.21.15.66:0/609161549 shutdown_connections 2024-03-30T00:46:11.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 -- 172.21.15.66:0/609161549 >> 172.21.15.66:0/609161549 conn(0x7f50680d0540 msgr2=0x7f50680d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:11.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 -- 172.21.15.66:0/609161549 shutdown_connections 2024-03-30T00:46:11.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 -- 172.21.15.66:0/609161549 wait complete. 2024-03-30T00:46:11.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 Processor -- start 2024-03-30T00:46:11.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 -- start start 2024-03-30T00:46:11.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 0x7f50680b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:11.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f506814e750 0x7f50681e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506814ebd0 0x7f50681e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506815ebd0 con 0x7f504c0047d0 2024-03-30T00:46:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50680c53e0 con 0x7f506814ebd0 2024-03-30T00:46:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506fc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50680c56c0 con 0x7f506814e750 2024-03-30T00:46:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 0x7f50680b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 0x7f50680b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44222/0 (socket says 172.21.15.66:44222) 2024-03-30T00:46:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506d200640 1 -- 172.21.15.66:0/1456211536 learned_addr learned my addr 172.21.15.66:0/1456211536 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f505fe00640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f506814e750 0x7f50681e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506dc00640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506814ebd0 0x7f50681e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:11.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506d200640 1 -- 172.21.15.66:0/1456211536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f506814e750 msgr2=0x7f50681e54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506d200640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f506814e750 0x7f50681e54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506d200640 1 -- 172.21.15.66:0/1456211536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506814ebd0 msgr2=0x7f50681e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.030+0000 7f506d200640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506814ebd0 0x7f50681e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f506d200640 1 -- 172.21.15.66:0/1456211536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50680c59a0 con 0x7f504c0047d0 2024-03-30T00:46:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f506d200640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 0x7f50680b21b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5058011cd0 tx=0x7f505800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f5065600640 1 -- 172.21.15.66:0/1456211536 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5058011d00 con 0x7f504c0047d0 2024-03-30T00:46:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f5065600640 1 -- 172.21.15.66:0/1456211536 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5058011e60 con 0x7f504c0047d0 2024-03-30T00:46:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f5065600640 1 -- 172.21.15.66:0/1456211536 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5058004240 con 0x7f504c0047d0 2024-03-30T00:46:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f506fc00640 1 -- 172.21.15.66:0/1456211536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50680c5c80 con 0x7f504c0047d0 2024-03-30T00:46:11.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f506fc00640 1 -- 172.21.15.66:0/1456211536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50680c4800 con 0x7f504c0047d0 2024-03-30T00:46:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f506fc00640 1 -- 172.21.15.66:0/1456211536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5068066c30 con 0x7f504c0047d0 2024-03-30T00:46:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f5065600640 1 -- 172.21.15.66:0/1456211536 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f50580043a0 con 0x7f504c0047d0 2024-03-30T00:46:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f5065600640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 0x7f5044068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f5065600640 1 -- 172.21.15.66:0/1456211536 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f5058084640 con 0x7f504c0047d0 2024-03-30T00:46:11.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.034+0000 7f505fe00640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 0x7f5044068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:11.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.038+0000 7f505fe00640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 0x7f5044068b50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f50500af000 tx=0x7f50500046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:11.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.042+0000 7f5065600640 1 -- 172.21.15.66:0/1456211536 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5058051b20 con 0x7f504c0047d0 2024-03-30T00:46:11.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.214+0000 7f506fc00640 1 -- 172.21.15.66:0/1456211536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f50680cfcb0 con 0x7f504c0047d0 2024-03-30T00:46:11.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.986+0000 7f5065600640 1 -- 172.21.15.66:0/1456211536 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v91) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5058022020 con 0x7f504c0047d0 2024-03-30T00:46:11.995 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-03-30T00:46:11.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 -- 172.21.15.66:0/1456211536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 msgr2=0x7f5044068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 0x7f5044068b50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f50500af000 tx=0x7f50500046d0 comp rx=0 tx=0).stop 2024-03-30T00:46:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 -- 172.21.15.66:0/1456211536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 msgr2=0x7f50680b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 0x7f50680b21b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5058011cd0 tx=0x7f505800f610 comp rx=0 tx=0).stop 2024-03-30T00:46:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 -- 172.21.15.66:0/1456211536 shutdown_connections 2024-03-30T00:46:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f506814ebd0 0x7f50681e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f506814e750 0x7f50681e54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5044066690 0x7f5044068b50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 --2- 172.21.15.66:0/1456211536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0047d0 0x7f50680b21b0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 -- 172.21.15.66:0/1456211536 >> 172.21.15.66:0/1456211536 conn(0x7f50680d0540 msgr2=0x7f50680ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 -- 172.21.15.66:0/1456211536 shutdown_connections 2024-03-30T00:46:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:11.990+0000 7f506fc00640 1 -- 172.21.15.66:0/1456211536 wait complete. 2024-03-30T00:46:12.022 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-30T00:46:12.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.158+0000 7f7e9d600640 1 Processor -- start 2024-03-30T00:46:12.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.158+0000 7f7e9d600640 1 -- start start 2024-03-30T00:46:12.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.158+0000 7f7e9d600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e980ce520 0x7f7e980ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.158+0000 7f7e9d600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7e980cabb0 0x7f7e980cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.158+0000 7f7e9d600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9806d070 con 0x7f7e9815a640 2024-03-30T00:46:12.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.158+0000 7f7e9d600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e980cbba0 con 0x7f7e980cabb0 2024-03-30T00:46:12.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.158+0000 7f7e9d600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980cee40 con 0x7f7e980ce520 2024-03-30T00:46:12.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.158+0000 7f7e97400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7e980cabb0 0x7f7e980cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.158+0000 7f7e8fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e980ce520 0x7f7e980ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.158+0000 7f7e96a00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7e9815a640 0x7f7e9806dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:49994/0 (socket says 172.21.15.66:49994) 2024-03-30T00:46:12.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.158+0000 7f7e96a00640 1 -- 172.21.15.66:0/1534683996 learned_addr learned my addr 172.21.15.66:0/1534683996 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:12.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e96000640 1 -- 172.21.15.66:0/1534683996 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816131580 0 0) 0x7f7e9806d070 con 0x7f7e9815a640 2024-03-30T00:46:12.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e8fe00640 1 -- 172.21.15.66:0/1534683996 >> v2:172.21.15.156:3300/0 conn(0x7f7e980cabb0 msgr2=0x7f7e980cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:12.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e8fe00640 1 --2- 172.21.15.66:0/1534683996 >> v2:172.21.15.156:3300/0 conn(0x7f7e980cabb0 0x7f7e980cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e8fe00640 1 -- 172.21.15.66:0/1534683996 >> v1:172.21.15.156:6789/0 conn(0x7f7e9815a640 legacy=0x7f7e9806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e8fe00640 1 -- 172.21.15.66:0/1534683996 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980cb8c0 con 0x7f7e980ce520 2024-03-30T00:46:12.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e97400640 1 --2- 172.21.15.66:0/1534683996 >> v2:172.21.15.156:3300/0 conn(0x7f7e980cabb0 0x7f7e980cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:46:12.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e8fe00640 1 --2- 172.21.15.66:0/1534683996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e980ce520 0x7f7e980ce900 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e800097c0 tx=0x7f7e8001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43f76ca1421d15da server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:12.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e96000640 1 -- 172.21.15.66:0/1534683996 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e800ab670 con 0x7f7e980ce520 2024-03-30T00:46:12.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e96000640 1 -- 172.21.15.66:0/1534683996 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e800ab7d0 con 0x7f7e980ce520 2024-03-30T00:46:12.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e96000640 1 -- 172.21.15.66:0/1534683996 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e800abb60 con 0x7f7e980ce520 2024-03-30T00:46:12.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e9d600640 1 -- 172.21.15.66:0/1534683996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e980ce520 msgr2=0x7f7e980ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e9d600640 1 --2- 172.21.15.66:0/1534683996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e980ce520 0x7f7e980ce900 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e800097c0 tx=0x7f7e8001c470 comp rx=0 tx=0).stop 2024-03-30T00:46:12.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e9d600640 1 -- 172.21.15.66:0/1534683996 shutdown_connections 2024-03-30T00:46:12.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e9d600640 1 --2- 172.21.15.66:0/1534683996 >> v2:172.21.15.156:3300/0 conn(0x7f7e980cabb0 0x7f7e980cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e9d600640 1 --2- 172.21.15.66:0/1534683996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e980ce520 0x7f7e980ce900 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.162+0000 7f7e9d600640 1 -- 172.21.15.66:0/1534683996 >> 172.21.15.66:0/1534683996 conn(0x7f7e98069f60 msgr2=0x7f7e9806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:12.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e9d600640 1 -- 172.21.15.66:0/1534683996 shutdown_connections 2024-03-30T00:46:12.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e9d600640 1 -- 172.21.15.66:0/1534683996 wait complete. 2024-03-30T00:46:12.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e9d600640 1 Processor -- start 2024-03-30T00:46:12.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e9d600640 1 -- start start 2024-03-30T00:46:12.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e9d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980cabb0 0x7f7e9814e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e9d600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e980ce520 0x7f7e9815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e9d600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9815a640 0x7f7e98148c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e9d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e981510b0 con 0x7f7e980cabb0 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e9d600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9814f1f0 con 0x7f7e9815a640 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e9d600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9814f4d0 con 0x7f7e980ce520 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e96a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980cabb0 0x7f7e9814e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e96a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980cabb0 0x7f7e9814e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44238/0 (socket says 172.21.15.66:44238) 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e96a00640 1 -- 172.21.15.66:0/1619053293 learned_addr learned my addr 172.21.15.66:0/1619053293 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e97400640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9815a640 0x7f7e98148c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e8fe00640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e980ce520 0x7f7e9815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e96a00640 1 -- 172.21.15.66:0/1619053293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e980ce520 msgr2=0x7f7e9815ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e96a00640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e980ce520 0x7f7e9815ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e96a00640 1 -- 172.21.15.66:0/1619053293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9815a640 msgr2=0x7f7e98148c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e96a00640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9815a640 0x7f7e98148c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e96a00640 1 -- 172.21.15.66:0/1619053293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e90009a70 con 0x7f7e980cabb0 2024-03-30T00:46:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e97400640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9815a640 0x7f7e98148c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e96a00640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980cabb0 0x7f7e9814e880 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800c480 tx=0x7f7e8800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e8ea00640 1 -- 172.21.15.66:0/1619053293 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e880074e0 con 0x7f7e980cabb0 2024-03-30T00:46:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e8ea00640 1 -- 172.21.15.66:0/1619053293 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e88007660 con 0x7f7e980cabb0 2024-03-30T00:46:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e8ea00640 1 -- 172.21.15.66:0/1619053293 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88004a30 con 0x7f7e980cabb0 2024-03-30T00:46:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.166+0000 7f7e9d600640 1 -- 172.21.15.66:0/1619053293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e80009420 con 0x7f7e980cabb0 2024-03-30T00:46:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.170+0000 7f7e9d600640 1 -- 172.21.15.66:0/1619053293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9814fa90 con 0x7f7e980cabb0 2024-03-30T00:46:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.170+0000 7f7e9d600640 1 -- 172.21.15.66:0/1619053293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e98066c30 con 0x7f7e980cabb0 2024-03-30T00:46:12.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.170+0000 7f7e8ea00640 1 -- 172.21.15.66:0/1619053293 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7e88004b90 con 0x7f7e980cabb0 2024-03-30T00:46:12.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.170+0000 7f7e8ea00640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e74066690 0x7f7e74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.170+0000 7f7e8ea00640 1 -- 172.21.15.66:0/1619053293 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7e880831a0 con 0x7f7e980cabb0 2024-03-30T00:46:12.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.170+0000 7f7e8fe00640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e74066690 0x7f7e74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.170+0000 7f7e8fe00640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e74066690 0x7f7e74068b50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7e800af000 tx=0x7f7e800a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:12.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.174+0000 7f7e8ea00640 1 -- 172.21.15.66:0/1619053293 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7e880541a0 con 0x7f7e980cabb0 2024-03-30T00:46:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 -- 172.21.15.66:0/1619053293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7e980cb4e0 con 0x7f7e980cabb0 2024-03-30T00:46:12.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e8ea00640 1 -- 172.21.15.66:0/1619053293 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7e88053d30 con 0x7f7e980cabb0 2024-03-30T00:46:12.398 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:12.398 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:46:12.398 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-03-30T00:46:12.398 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.398 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm857MDRAAu2pldF20deZfoSDDXuTF/A==", 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMmNFGhAAlHS6ZpIoJc7tlYazCISiLw==", 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.399 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmuiBsFRAA2kez6ylAylKTWsQ1S2cahA==", 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.400 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm/oV3IRAAWn/zG1cLK+Raa2pYPzHd7Q==", 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-03-30T00:46:12.401 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAbXwdmf3HtOhAAyRcyb0YUJcJca5fjmZ3Frw==", 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAfXwdml7cdBRAAs639bBHTPyfhfVZMPRMfYA==", 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:46:12.402 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAiXwdmeS6KChAAZjLAbKSL7Vswi3qQ4304bA==", 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-03-30T00:46:12.403 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAlXwdmeuigEBAAiPIPheowKBnn3UZWwpdr+Q==", 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQApXwdm+bQNHRAAAb/lufdb7Fe+EFevd23gtA==", 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:46:12.404 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAsXwdmReL1IhAAwCs1cS+P6J8VCOYdrnrPQg==", 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.405 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAvXwdmA15vKBAAmpnKrDM/Y0xuBGxdyjrkWA==", 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAyXwdmrl02LhAA6JfEnugoz6l4nsXl6/vq0Q==", 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:46:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmZjh7JRAA3EFOImM03iOljcwmA7/xWw==", 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.407 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmkZv0LhAAgwCqW25+J1VYO31mWwDtLQ==", 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmLG1/KxAAmNIZ9ZXdqPRQjCXvz7wC9A==", 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:46:12.408 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmyJ/nNBAAIgiUoUCbTeLEjL1iaPFw1A==", 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAXXwdmwWxQLRAAoFyZnFsIo1NvQlzRN+UHmw==", 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-03-30T00:46:12.410 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmEF+xCRAAqRVFguki881+4azP8lsf0g==", 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMg32BRAAUWBF+QbwRGsopcAoBay8wA==", 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:46:12.411 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.412 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:12.412 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-03-30T00:46:12.412 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:46:12.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 -- 172.21.15.66:0/1619053293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e74066690 msgr2=0x7f7e74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e74066690 0x7f7e74068b50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7e800af000 tx=0x7f7e800a7000 comp rx=0 tx=0).stop 2024-03-30T00:46:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 -- 172.21.15.66:0/1619053293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980cabb0 msgr2=0x7f7e9814e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980cabb0 0x7f7e9814e880 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800c480 tx=0x7f7e8800c940 comp rx=0 tx=0).stop 2024-03-30T00:46:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 -- 172.21.15.66:0/1619053293 shutdown_connections 2024-03-30T00:46:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9815a640 0x7f7e98148c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e980ce520 0x7f7e9815ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7e74066690 0x7f7e74068b50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 --2- 172.21.15.66:0/1619053293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980cabb0 0x7f7e9814e880 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 -- 172.21.15.66:0/1619053293 >> 172.21.15.66:0/1619053293 conn(0x7f7e98069f60 msgr2=0x7f7e98158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 -- 172.21.15.66:0/1619053293 shutdown_connections 2024-03-30T00:46:12.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.390+0000 7f7e9d600640 1 -- 172.21.15.66:0/1619053293 wait complete. 2024-03-30T00:46:12.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:46:12.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.550+0000 7f9431200640 1 Processor -- start 2024-03-30T00:46:12.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.550+0000 7f9431200640 1 -- start start 2024-03-30T00:46:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.550+0000 7f9431200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f942c0cd2d0 0x7f942c0cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.550+0000 7f9431200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f942c0cc8d0 0x7f942c06c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.550+0000 7f9431200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c0cb620 con 0x7f942c0cad00 2024-03-30T00:46:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.550+0000 7f9431200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c0cbec0 con 0x7f942c0cc8d0 2024-03-30T00:46:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.550+0000 7f9431200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c0ca660 con 0x7f942c0cd2d0 2024-03-30T00:46:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.550+0000 7f942a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f942c0cc8d0 0x7f942c06c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.554+0000 7f942a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f942c0cc8d0 0x7f942c06c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44242/0 (socket says 172.21.15.66:44242) 2024-03-30T00:46:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.554+0000 7f942a800640 1 -- 172.21.15.66:0/671094463 learned_addr learned my addr 172.21.15.66:0/671094463 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.554+0000 7f9421400640 1 --2- 172.21.15.66:0/671094463 >> v2:172.21.15.156:3300/0 conn(0x7f942c0cd2d0 0x7f942c0cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.554+0000 7f942a800640 1 -- 172.21.15.66:0/671094463 >> v2:172.21.15.156:3300/0 conn(0x7f942c0cd2d0 msgr2=0x7f942c0cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:12.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.554+0000 7f942a800640 1 --2- 172.21.15.66:0/671094463 >> v2:172.21.15.156:3300/0 conn(0x7f942c0cd2d0 0x7f942c0cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.554+0000 7f942a800640 1 -- 172.21.15.66:0/671094463 >> v1:172.21.15.156:6789/0 conn(0x7f942c0cad00 legacy=0x7f942c0c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.554+0000 7f942a800640 1 -- 172.21.15.66:0/671094463 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0ca940 con 0x7f942c0cc8d0 2024-03-30T00:46:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.554+0000 7f942a800640 1 --2- 172.21.15.66:0/671094463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c0cc8d0 0x7f942c06c030 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f942c0cba10 tx=0x7f942401e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babece629ec71e5d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.554+0000 7f9429400640 1 -- 172.21.15.66:0/671094463 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424004230 con 0x7f942c0cc8d0 2024-03-30T00:46:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.554+0000 7f9429400640 1 -- 172.21.15.66:0/671094463 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94240ab070 con 0x7f942c0cc8d0 2024-03-30T00:46:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.554+0000 7f9429400640 1 -- 172.21.15.66:0/671094463 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240a7870 con 0x7f942c0cc8d0 2024-03-30T00:46:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 -- 172.21.15.66:0/671094463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c0cc8d0 msgr2=0x7f942c06c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 --2- 172.21.15.66:0/671094463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c0cc8d0 0x7f942c06c030 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f942c0cba10 tx=0x7f942401e9a0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 -- 172.21.15.66:0/671094463 shutdown_connections 2024-03-30T00:46:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 --2- 172.21.15.66:0/671094463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c0cc8d0 0x7f942c06c030 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 --2- 172.21.15.66:0/671094463 >> v2:172.21.15.156:3300/0 conn(0x7f942c0cd2d0 0x7f942c0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 -- 172.21.15.66:0/671094463 >> 172.21.15.66:0/671094463 conn(0x7f942c069f60 msgr2=0x7f942c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 -- 172.21.15.66:0/671094463 shutdown_connections 2024-03-30T00:46:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 -- 172.21.15.66:0/671094463 wait complete. 2024-03-30T00:46:12.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 Processor -- start 2024-03-30T00:46:12.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 -- start start 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f942c0cad00 0x7f942c158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c0cc8d0 0x7f942c0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f942c0cd2d0 0x7f942c0ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c158840 con 0x7f942c0cc8d0 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0c6990 con 0x7f942c0cd2d0 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9431200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c15a540 con 0x7f942c0cad00 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9421400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c0cc8d0 0x7f942c0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9421400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c0cc8d0 0x7f942c0c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44256/0 (socket says 172.21.15.66:44256) 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9421400640 1 -- 172.21.15.66:0/3589737619 learned_addr learned my addr 172.21.15.66:0/3589737619 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f942a800640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f942c0cd2d0 0x7f942c0ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9429e00640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f942c0cad00 0x7f942c158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9421400640 1 -- 172.21.15.66:0/3589737619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f942c0cad00 msgr2=0x7f942c158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9421400640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f942c0cad00 0x7f942c158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9421400640 1 -- 172.21.15.66:0/3589737619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f942c0cd2d0 msgr2=0x7f942c0ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9421400640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f942c0cd2d0 0x7f942c0ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9421400640 1 -- 172.21.15.66:0/3589737619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942400b780 con 0x7f942c0cc8d0 2024-03-30T00:46:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f942a800640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f942c0cd2d0 0x7f942c0ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.558+0000 7f9421400640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c0cc8d0 0x7f942c0c5da0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f941400e870 tx=0x7f941400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.562+0000 7f9422a00640 1 -- 172.21.15.66:0/3589737619 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941400ee60 con 0x7f942c0cc8d0 2024-03-30T00:46:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.562+0000 7f9422a00640 1 -- 172.21.15.66:0/3589737619 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941400c940 con 0x7f942c0cc8d0 2024-03-30T00:46:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.562+0000 7f9422a00640 1 -- 172.21.15.66:0/3589737619 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941400ccb0 con 0x7f942c0cc8d0 2024-03-30T00:46:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.562+0000 7f9431200640 1 -- 172.21.15.66:0/3589737619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c15ada0 con 0x7f942c0cc8d0 2024-03-30T00:46:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.562+0000 7f9431200640 1 -- 172.21.15.66:0/3589737619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c15a750 con 0x7f942c0cc8d0 2024-03-30T00:46:12.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.562+0000 7f9431200640 1 -- 172.21.15.66:0/3589737619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c15af30 con 0x7f942c0cc8d0 2024-03-30T00:46:12.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.562+0000 7f9422a00640 1 -- 172.21.15.66:0/3589737619 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9414013470 con 0x7f942c0cc8d0 2024-03-30T00:46:12.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.566+0000 7f9422a00640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9408066690 0x7f9408068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.566+0000 7f9422a00640 1 -- 172.21.15.66:0/3589737619 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9414084830 con 0x7f942c0cc8d0 2024-03-30T00:46:12.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.566+0000 7f9429e00640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9408066690 0x7f9408068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.566+0000 7f9429e00640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9408066690 0x7f9408068b50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f941c0012f0 tx=0x7f941c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.566+0000 7f9422a00640 1 -- 172.21.15.66:0/3589737619 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9414051d90 con 0x7f942c0cc8d0 2024-03-30T00:46:12.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.730+0000 7f9431200640 1 -- 172.21.15.66:0/3589737619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f942c066c30 con 0x7f942c0cc8d0 2024-03-30T00:46:12.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.730+0000 7f9422a00640 1 -- 172.21.15.66:0/3589737619 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+12595 (secure 0 0 0) 0x7f9414017090 con 0x7f942c0cc8d0 2024-03-30T00:46:12.740 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:12.740 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:46:11.985477+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:46:12.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 -- 172.21.15.66:0/3589737619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9408066690 msgr2=0x7f9408068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9408066690 0x7f9408068b50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f941c0012f0 tx=0x7f941c009040 comp rx=0 tx=0).stop 2024-03-30T00:46:12.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 -- 172.21.15.66:0/3589737619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c0cc8d0 msgr2=0x7f942c0c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c0cc8d0 0x7f942c0c5da0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f941400e870 tx=0x7f941400ed30 comp rx=0 tx=0).stop 2024-03-30T00:46:12.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 -- 172.21.15.66:0/3589737619 shutdown_connections 2024-03-30T00:46:12.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f942c0cd2d0 0x7f942c0ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9408066690 0x7f9408068b50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f942c0cc8d0 0x7f942c0c5da0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 --2- 172.21.15.66:0/3589737619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f942c0cad00 0x7f942c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 -- 172.21.15.66:0/3589737619 >> 172.21.15.66:0/3589737619 conn(0x7f942c069f60 msgr2=0x7f942c06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:12.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 -- 172.21.15.66:0/3589737619 shutdown_connections 2024-03-30T00:46:12.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.734+0000 7f9431200640 1 -- 172.21.15.66:0/3589737619 wait complete. 2024-03-30T00:46:12.761 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:46:12.762 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:46:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.894+0000 7f8d7a600640 1 Processor -- start 2024-03-30T00:46:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d7a600640 1 -- start start 2024-03-30T00:46:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d7a600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8d7415a640 0x7f8d7406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d7a600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 0x7f8d740ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d7a600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7406d070 con 0x7f8d740ce520 2024-03-30T00:46:12.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d7a600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d740cbba0 con 0x7f8d7415a640 2024-03-30T00:46:12.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d7a600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740cee90 con 0x7f8d740cb2c0 2024-03-30T00:46:12.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d73400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8d7415a640 0x7f8d7406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d73e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 0x7f8d740ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d73400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8d7415a640 0x7f8d7406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:45076/0 (socket says 172.21.15.66:45076) 2024-03-30T00:46:12.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d73e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 0x7f8d740ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:41362/0 (socket says 172.21.15.66:41362) 2024-03-30T00:46:12.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d73400640 1 -- 172.21.15.66:0/4148986296 learned_addr learned my addr 172.21.15.66:0/4148986296 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:12.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d73400640 1 -- 172.21.15.66:0/4148986296 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 msgr2=0x7f8d740ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:12.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d73400640 1 --2- 172.21.15.66:0/4148986296 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 0x7f8d740ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d73400640 1 -- 172.21.15.66:0/4148986296 >> v1:172.21.15.156:6789/0 conn(0x7f8d740ce520 legacy=0x7f8d740cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.898+0000 7f8d73400640 1 -- 172.21.15.66:0/4148986296 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7406e2c0 con 0x7f8d7415a640 2024-03-30T00:46:12.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d73400640 1 --2- 172.21.15.66:0/4148986296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7415a640 0x7f8d7406dd80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009870 tx=0x7f8d6401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b92bbcfae862879 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:12.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d72000640 1 -- 172.21.15.66:0/4148986296 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640ab650 con 0x7f8d7415a640 2024-03-30T00:46:12.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d72000640 1 -- 172.21.15.66:0/4148986296 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d640ab7b0 con 0x7f8d7415a640 2024-03-30T00:46:12.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d72000640 1 -- 172.21.15.66:0/4148986296 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640abb40 con 0x7f8d7415a640 2024-03-30T00:46:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d7a600640 1 -- 172.21.15.66:0/4148986296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7415a640 msgr2=0x7f8d7406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d7a600640 1 --2- 172.21.15.66:0/4148986296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7415a640 0x7f8d7406dd80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009870 tx=0x7f8d6401c560 comp rx=0 tx=0).stop 2024-03-30T00:46:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d7a600640 1 -- 172.21.15.66:0/4148986296 shutdown_connections 2024-03-30T00:46:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d7a600640 1 --2- 172.21.15.66:0/4148986296 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 0x7f8d740ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d7a600640 1 --2- 172.21.15.66:0/4148986296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d7415a640 0x7f8d7406dd80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d7a600640 1 -- 172.21.15.66:0/4148986296 >> 172.21.15.66:0/4148986296 conn(0x7f8d74069f60 msgr2=0x7f8d7406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d7a600640 1 -- 172.21.15.66:0/4148986296 shutdown_connections 2024-03-30T00:46:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d7a600640 1 -- 172.21.15.66:0/4148986296 wait complete. 2024-03-30T00:46:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d7a600640 1 Processor -- start 2024-03-30T00:46:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.902+0000 7f8d7a600640 1 -- start start 2024-03-30T00:46:12.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d7a600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 0x7f8d740cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d7a600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d740ce520 0x7f8d74157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d7a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7415a640 0x7f8d74156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d7a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74159e40 con 0x7f8d7415a640 2024-03-30T00:46:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d7a600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d741598f0 con 0x7f8d740ce520 2024-03-30T00:46:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d7a600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d741582b0 con 0x7f8d740cb2c0 2024-03-30T00:46:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d73e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7415a640 0x7f8d74156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d73e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7415a640 0x7f8d74156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44264/0 (socket says 172.21.15.66:44264) 2024-03-30T00:46:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d73e00640 1 -- 172.21.15.66:0/3409477651 learned_addr learned my addr 172.21.15.66:0/3409477651 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d73400640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 0x7f8d740cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d72a00640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d740ce520 0x7f8d74157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d73e00640 1 -- 172.21.15.66:0/3409477651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 msgr2=0x7f8d740cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d73e00640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 0x7f8d740cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d73e00640 1 -- 172.21.15.66:0/3409477651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d740ce520 msgr2=0x7f8d74157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d73e00640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d740ce520 0x7f8d74157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d73e00640 1 -- 172.21.15.66:0/3409477651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d640094d0 con 0x7f8d7415a640 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d73400640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 0x7f8d740cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d72a00640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d740ce520 0x7f8d74157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d73e00640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7415a640 0x7f8d74156370 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009dd0 tx=0x7f8d6c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d6b400640 1 -- 172.21.15.66:0/3409477651 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c014480 con 0x7f8d7415a640 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d6b400640 1 -- 172.21.15.66:0/3409477651 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c00f040 con 0x7f8d7415a640 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d6b400640 1 -- 172.21.15.66:0/3409477651 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c014840 con 0x7f8d7415a640 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d7a600640 1 -- 172.21.15.66:0/3409477651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7415d370 con 0x7f8d7415a640 2024-03-30T00:46:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d7a600640 1 -- 172.21.15.66:0/3409477651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7415d650 con 0x7f8d7415a640 2024-03-30T00:46:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d7a600640 1 -- 172.21.15.66:0/3409477651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d74159750 con 0x7f8d7415a640 2024-03-30T00:46:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.906+0000 7f8d6b400640 1 -- 172.21.15.66:0/3409477651 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8d6c0149a0 con 0x7f8d7415a640 2024-03-30T00:46:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.910+0000 7f8d6b400640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d54066690 0x7f8d54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.910+0000 7f8d6b400640 1 -- 172.21.15.66:0/3409477651 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8d6c084af0 con 0x7f8d7415a640 2024-03-30T00:46:12.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.910+0000 7f8d73400640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d54066690 0x7f8d54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:12.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.910+0000 7f8d73400640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d54066690 0x7f8d54068b50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8d640af000 tx=0x7f8d64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:12.914+0000 7f8d6b400640 1 -- 172.21.15.66:0/3409477651 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8d74159750 con 0x7f8d7415a640 2024-03-30T00:46:13.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.074+0000 7f8d7a600640 1 -- 172.21.15.66:0/3409477651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d74159750 con 0x7f8d7415a640 2024-03-30T00:46:13.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.074+0000 7f8d6b400640 1 -- 172.21.15.66:0/3409477651 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d74159750 con 0x7f8d7415a640 2024-03-30T00:46:13.084 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:46:13.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.074+0000 7f8d7a600640 1 -- 172.21.15.66:0/3409477651 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d54066690 msgr2=0x7f8d54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.078+0000 7f8d7a600640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d54066690 0x7f8d54068b50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8d640af000 tx=0x7f8d64008040 comp rx=0 tx=0).stop 2024-03-30T00:46:13.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.078+0000 7f8d7a600640 1 -- 172.21.15.66:0/3409477651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7415a640 msgr2=0x7f8d74156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.078+0000 7f8d7a600640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7415a640 0x7f8d74156370 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009dd0 tx=0x7f8d6c00c490 comp rx=0 tx=0).stop 2024-03-30T00:46:13.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.078+0000 7f8d7a600640 1 -- 172.21.15.66:0/3409477651 shutdown_connections 2024-03-30T00:46:13.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.078+0000 7f8d7a600640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d54066690 0x7f8d54068b50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.078+0000 7f8d7a600640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d7415a640 0x7f8d74156370 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.078+0000 7f8d7a600640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d740ce520 0x7f8d74157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.078+0000 7f8d7a600640 1 --2- 172.21.15.66:0/3409477651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d740cb2c0 0x7f8d740cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.078+0000 7f8d7a600640 1 -- 172.21.15.66:0/3409477651 >> 172.21.15.66:0/3409477651 conn(0x7f8d74069f60 msgr2=0x7f8d740adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:13.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.078+0000 7f8d7a600640 1 -- 172.21.15.66:0/3409477651 shutdown_connections 2024-03-30T00:46:13.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.078+0000 7f8d7a600640 1 -- 172.21.15.66:0/3409477651 wait complete. 2024-03-30T00:46:13.104 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T00:46:13.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffabbe00640 1 Processor -- start 2024-03-30T00:46:13.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffabbe00640 1 -- start start 2024-03-30T00:46:13.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffabbe00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ffabc0ad9c0 0x7ffabc0ab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:13.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffabbe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffabc0b1520 con 0x7ffabc06fee0 2024-03-30T00:46:13.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffabbe00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffabc0b1800 con 0x7ffabc06ce30 2024-03-30T00:46:13.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffabbe00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffabc0b1ae0 con 0x7ffabc0ad9c0 2024-03-30T00:46:13.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffaba000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ffabc06fee0 0x7ffabc06c720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48964/0 (socket says 172.21.15.66:48964) 2024-03-30T00:46:13.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffaba000640 1 -- 172.21.15.66:0/4283053805 learned_addr learned my addr 172.21.15.66:0/4283053805 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:13.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffabaa00640 1 --2- 172.21.15.66:0/4283053805 >> v2:172.21.15.156:3300/0 conn(0x7ffabc0ad9c0 0x7ffabc0ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:13.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffab9600640 1 -- 172.21.15.66:0/4283053805 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732121627 0 0) 0x7ffabc0b1520 con 0x7ffabc06fee0 2024-03-30T00:46:13.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffab9600640 1 -- 172.21.15.66:0/4283053805 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaa0003530 con 0x7ffabc06fee0 2024-03-30T00:46:13.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffabaa00640 1 -- 172.21.15.66:0/4283053805 >> v1:172.21.15.156:6789/0 conn(0x7ffabc06ce30 legacy=0x7ffabc0b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffab9600640 1 -- 172.21.15.66:0/4283053805 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259268894 0 0) 0x7ffabc0b1800 con 0x7ffabc06ce30 2024-03-30T00:46:13.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffabaa00640 1 -- 172.21.15.66:0/4283053805 >> v1:172.21.15.66:6789/0 conn(0x7ffabc06fee0 legacy=0x7ffabc06c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.238+0000 7ffabaa00640 1 -- 172.21.15.66:0/4283053805 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc0b1dc0 con 0x7ffabc0ad9c0 2024-03-30T00:46:13.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffabaa00640 1 --2- 172.21.15.66:0/4283053805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc0ad9c0 0x7ffabc0ab630 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0097c0 tx=0x7ffaac01d6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e207a9fdc23795c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffab9600640 1 -- 172.21.15.66:0/4283053805 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0a7690 con 0x7ffabc0ad9c0 2024-03-30T00:46:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffab9600640 1 -- 172.21.15.66:0/4283053805 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac0a77f0 con 0x7ffabc0ad9c0 2024-03-30T00:46:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffab9600640 1 -- 172.21.15.66:0/4283053805 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0a7b80 con 0x7ffabc0ad9c0 2024-03-30T00:46:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffabbe00640 1 -- 172.21.15.66:0/4283053805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc0ad9c0 msgr2=0x7ffabc0ab630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffabbe00640 1 --2- 172.21.15.66:0/4283053805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc0ad9c0 0x7ffabc0ab630 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0097c0 tx=0x7ffaac01d6a0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffabbe00640 1 -- 172.21.15.66:0/4283053805 shutdown_connections 2024-03-30T00:46:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffabbe00640 1 --2- 172.21.15.66:0/4283053805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc0ad9c0 0x7ffabc0ab630 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffabbe00640 1 -- 172.21.15.66:0/4283053805 >> 172.21.15.66:0/4283053805 conn(0x7ffabc0afe40 msgr2=0x7ffabc0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffabbe00640 1 -- 172.21.15.66:0/4283053805 shutdown_connections 2024-03-30T00:46:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffabbe00640 1 -- 172.21.15.66:0/4283053805 wait complete. 2024-03-30T00:46:13.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffabbe00640 1 Processor -- start 2024-03-30T00:46:13.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.242+0000 7ffabbe00640 1 -- start start 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabbe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffabc06ce30 0x7ffabc0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabbe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc06fee0 0x7ffabc0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffabc156680 0x7ffabc150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabbe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0c8160 con 0x7ffabc156680 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabbe00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc150840 con 0x7ffabc06fee0 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabbe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc150b20 con 0x7ffabc06ce30 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffabc156680 0x7ffabc150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffabc156680 0x7ffabc150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44270/0 (socket says 172.21.15.66:44270) 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabb400640 1 -- 172.21.15.66:0/2659495612 learned_addr learned my addr 172.21.15.66:0/2659495612 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffaba000640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc06fee0 0x7ffabc0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabaa00640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffabc06ce30 0x7ffabc0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabb400640 1 -- 172.21.15.66:0/2659495612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffabc06ce30 msgr2=0x7ffabc0c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabb400640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffabc06ce30 0x7ffabc0c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabb400640 1 -- 172.21.15.66:0/2659495612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc06fee0 msgr2=0x7ffabc0c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabb400640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc06fee0 0x7ffabc0c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabb400640 1 -- 172.21.15.66:0/2659495612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac009420 con 0x7ffabc156680 2024-03-30T00:46:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabb400640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffabc156680 0x7ffabc150e00 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ffab4001570 tx=0x7ffab400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffab3400640 1 -- 172.21.15.66:0/2659495612 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4003db0 con 0x7ffabc156680 2024-03-30T00:46:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffab3400640 1 -- 172.21.15.66:0/2659495612 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab4005070 con 0x7ffabc156680 2024-03-30T00:46:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabbe00640 1 -- 172.21.15.66:0/2659495612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc0cae00 con 0x7ffabc156680 2024-03-30T00:46:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffab3400640 1 -- 172.21.15.66:0/2659495612 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4005420 con 0x7ffabc156680 2024-03-30T00:46:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabbe00640 1 -- 172.21.15.66:0/2659495612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc0ca040 con 0x7ffabc156680 2024-03-30T00:46:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.246+0000 7ffabbe00640 1 -- 172.21.15.66:0/2659495612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffabc0cb320 con 0x7ffabc156680 2024-03-30T00:46:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.250+0000 7ffab3400640 1 -- 172.21.15.66:0/2659495612 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ffab4005580 con 0x7ffabc156680 2024-03-30T00:46:13.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.250+0000 7ffab3400640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa9c066690 0x7ffa9c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:13.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.250+0000 7ffab3400640 1 -- 172.21.15.66:0/2659495612 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ffab40836f0 con 0x7ffabc156680 2024-03-30T00:46:13.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.250+0000 7ffabaa00640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa9c066690 0x7ffa9c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:13.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.250+0000 7ffabaa00640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa9c066690 0x7ffa9c068b50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0b0000 tx=0x7ffaac0b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.254+0000 7ffab3400640 1 -- 172.21.15.66:0/2659495612 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ffabc0cb320 con 0x7ffabc156680 2024-03-30T00:46:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 -- 172.21.15.66:0/2659495612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ffabc0cb320 con 0x7ffabc156680 2024-03-30T00:46:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffab3400640 1 -- 172.21.15.66:0/2659495612 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v91) v1 ==== 84+0+65 (secure 0 0 0) 0x7ffabc0cb320 con 0x7ffabc156680 2024-03-30T00:46:13.426 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:13.426 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:46:13.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:46:13.426 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-03-30T00:46:13.427 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-03-30T00:46:13.427 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:13.427 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:46:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 -- 172.21.15.66:0/2659495612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa9c066690 msgr2=0x7ffa9c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa9c066690 0x7ffa9c068b50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0b0000 tx=0x7ffaac0b3040 comp rx=0 tx=0).stop 2024-03-30T00:46:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 -- 172.21.15.66:0/2659495612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffabc156680 msgr2=0x7ffabc150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffabc156680 0x7ffabc150e00 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ffab4001570 tx=0x7ffab400cd20 comp rx=0 tx=0).stop 2024-03-30T00:46:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 -- 172.21.15.66:0/2659495612 shutdown_connections 2024-03-30T00:46:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffa9c066690 0x7ffa9c068b50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffabc156680 0x7ffabc150e00 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffabc06fee0 0x7ffabc0c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 --2- 172.21.15.66:0/2659495612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffabc06ce30 0x7ffabc0c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 -- 172.21.15.66:0/2659495612 >> 172.21.15.66:0/2659495612 conn(0x7ffabc0afe40 msgr2=0x7ffabc0aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 -- 172.21.15.66:0/2659495612 shutdown_connections 2024-03-30T00:46:13.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.418+0000 7ffabbe00640 1 -- 172.21.15.66:0/2659495612 wait complete. 2024-03-30T00:46:13.447 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T00:46:13.447 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T00:46:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.582+0000 7f0a9da00640 1 Processor -- start 2024-03-30T00:46:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.582+0000 7f0a9da00640 1 -- start start 2024-03-30T00:46:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.582+0000 7f0a9da00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0a9815ba00 0x7f0a980ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.582+0000 7f0a9da00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0a980cecf0 0x7f0a981579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.582+0000 7f0a9da00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a980cbf80 con 0x7f0a980acaf0 2024-03-30T00:46:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.582+0000 7f0a9da00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a980b2be0 con 0x7f0a9815ba00 2024-03-30T00:46:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.582+0000 7f0a9da00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a980cc820 con 0x7f0a980cecf0 2024-03-30T00:46:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.582+0000 7f0a8e000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0a9815ba00 0x7f0a980ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.582+0000 7f0a8e000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0a9815ba00 0x7f0a980ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44280/0 (socket says 172.21.15.66:44280) 2024-03-30T00:46:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.582+0000 7f0a8e000640 1 -- 172.21.15.66:0/387926863 learned_addr learned my addr 172.21.15.66:0/387926863 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a8e000640 1 -- 172.21.15.66:0/387926863 >> v2:172.21.15.156:3300/0 conn(0x7f0a980cecf0 msgr2=0x7f0a981579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a8e000640 1 --2- 172.21.15.66:0/387926863 >> v2:172.21.15.156:3300/0 conn(0x7f0a980cecf0 0x7f0a981579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a8e000640 1 -- 172.21.15.66:0/387926863 >> v1:172.21.15.156:6789/0 conn(0x7f0a980acaf0 legacy=0x7f0a980b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a8e000640 1 -- 172.21.15.66:0/387926863 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a980cf110 con 0x7f0a9815ba00 2024-03-30T00:46:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a8e000640 1 --2- 172.21.15.66:0/387926863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a9815ba00 0x7f0a980ce7b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0a980ad9c0 tx=0x7f0a8001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f505af5d3c6556ff server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:13.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a96000640 1 -- 172.21.15.66:0/387926863 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800ab620 con 0x7f0a9815ba00 2024-03-30T00:46:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a96000640 1 -- 172.21.15.66:0/387926863 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a800ab780 con 0x7f0a9815ba00 2024-03-30T00:46:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a96000640 1 -- 172.21.15.66:0/387926863 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800abb10 con 0x7f0a9815ba00 2024-03-30T00:46:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a9da00640 1 -- 172.21.15.66:0/387926863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a9815ba00 msgr2=0x7f0a980ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a9da00640 1 --2- 172.21.15.66:0/387926863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a9815ba00 0x7f0a980ce7b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0a980ad9c0 tx=0x7f0a8001c420 comp rx=0 tx=0).stop 2024-03-30T00:46:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a9da00640 1 -- 172.21.15.66:0/387926863 shutdown_connections 2024-03-30T00:46:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a9da00640 1 --2- 172.21.15.66:0/387926863 >> v2:172.21.15.156:3300/0 conn(0x7f0a980cecf0 0x7f0a981579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a9da00640 1 --2- 172.21.15.66:0/387926863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a9815ba00 0x7f0a980ce7b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a9da00640 1 -- 172.21.15.66:0/387926863 >> 172.21.15.66:0/387926863 conn(0x7f0a9806a190 msgr2=0x7f0a9806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a9da00640 1 -- 172.21.15.66:0/387926863 shutdown_connections 2024-03-30T00:46:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.586+0000 7f0a9da00640 1 -- 172.21.15.66:0/387926863 wait complete. 2024-03-30T00:46:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a9da00640 1 Processor -- start 2024-03-30T00:46:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a9da00640 1 -- start start 2024-03-30T00:46:13.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a9da00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a980acaf0 0x7f0a98096b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:13.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a9da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a980cecf0 0x7f0a98071540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:13.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a9da00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9815ba00 0x7f0a98071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:13.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a9da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a981576e0 con 0x7f0a980cecf0 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a9da00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98153ee0 con 0x7f0a9815ba00 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a9da00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980cb780 con 0x7f0a980acaf0 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a980cecf0 0x7f0a98071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a980cecf0 0x7f0a98071540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44286/0 (socket says 172.21.15.66:44286) 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8e000640 1 -- 172.21.15.66:0/3231481035 learned_addr learned my addr 172.21.15.66:0/3231481035 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a97400640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9815ba00 0x7f0a98071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a96a00640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a980acaf0 0x7f0a98096b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8e000640 1 -- 172.21.15.66:0/3231481035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a980acaf0 msgr2=0x7f0a98096b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8e000640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a980acaf0 0x7f0a98096b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8e000640 1 -- 172.21.15.66:0/3231481035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9815ba00 msgr2=0x7f0a98071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8e000640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9815ba00 0x7f0a98071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8e000640 1 -- 172.21.15.66:0/3231481035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a800094d0 con 0x7f0a980cecf0 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a97400640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9815ba00 0x7f0a98071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:13.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8e000640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a980cecf0 0x7f0a98071540 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0a800ad040 tx=0x7f0a800191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:13.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8f400640 1 -- 172.21.15.66:0/3231481035 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800bf410 con 0x7f0a980cecf0 2024-03-30T00:46:13.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8f400640 1 -- 172.21.15.66:0/3231481035 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a800bf590 con 0x7f0a980cecf0 2024-03-30T00:46:13.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a8f400640 1 -- 172.21.15.66:0/3231481035 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800bf940 con 0x7f0a980cecf0 2024-03-30T00:46:13.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a9da00640 1 -- 172.21.15.66:0/3231481035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a98152f10 con 0x7f0a980cecf0 2024-03-30T00:46:13.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.590+0000 7f0a9da00640 1 -- 172.21.15.66:0/3231481035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a980974d0 con 0x7f0a980cecf0 2024-03-30T00:46:13.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.594+0000 7f0a9da00640 1 -- 172.21.15.66:0/3231481035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a98157820 con 0x7f0a980cecf0 2024-03-30T00:46:13.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.594+0000 7f0a8f400640 1 -- 172.21.15.66:0/3231481035 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0a800b5400 con 0x7f0a980cecf0 2024-03-30T00:46:13.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.594+0000 7f0a8f400640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0a74066690 0x7f0a74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:13.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.594+0000 7f0a8f400640 1 -- 172.21.15.66:0/3231481035 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0a8011eac0 con 0x7f0a980cecf0 2024-03-30T00:46:13.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.594+0000 7f0a96a00640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0a74066690 0x7f0a74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:13.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.594+0000 7f0a96a00640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0a74066690 0x7f0a74068b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800b810 tx=0x7f0a88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:13.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.598+0000 7f0a8f400640 1 -- 172.21.15.66:0/3231481035 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0a98157820 con 0x7f0a980cecf0 2024-03-30T00:46:13.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.762+0000 7f0a9da00640 1 -- 172.21.15.66:0/3231481035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0a98157820 con 0x7f0a980cecf0 2024-03-30T00:46:13.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.862+0000 7f0a8f400640 1 -- 172.21.15.66:0/3231481035 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v92) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0a98157820 con 0x7f0a980cecf0 2024-03-30T00:46:13.871 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-03-30T00:46:13.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 -- 172.21.15.66:0/3231481035 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0a74066690 msgr2=0x7f0a74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0a74066690 0x7f0a74068b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800b810 tx=0x7f0a88008040 comp rx=0 tx=0).stop 2024-03-30T00:46:13.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 -- 172.21.15.66:0/3231481035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a980cecf0 msgr2=0x7f0a98071540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:13.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a980cecf0 0x7f0a98071540 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0a800ad040 tx=0x7f0a800191c0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 -- 172.21.15.66:0/3231481035 shutdown_connections 2024-03-30T00:46:13.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9815ba00 0x7f0a98071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0a74066690 0x7f0a74068b50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a980cecf0 0x7f0a98071540 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 --2- 172.21.15.66:0/3231481035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a980acaf0 0x7f0a98096b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:13.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 -- 172.21.15.66:0/3231481035 >> 172.21.15.66:0/3231481035 conn(0x7f0a9806a190 msgr2=0x7f0a980ad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:13.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 -- 172.21.15.66:0/3231481035 shutdown_connections 2024-03-30T00:46:13.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:13.866+0000 7f0a9da00640 1 -- 172.21.15.66:0/3231481035 wait complete. 2024-03-30T00:46:13.897 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T00:46:14.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.034+0000 7f7ccdc00640 1 Processor -- start 2024-03-30T00:46:14.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.034+0000 7f7ccdc00640 1 -- start start 2024-03-30T00:46:14.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7ccdc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7cc815cc00 0x7f7cc8099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:14.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7ccdc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc80ceef0 con 0x7f7cc8098a20 2024-03-30T00:46:14.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7ccdc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc8099ca0 con 0x7f7cc80b18b0 2024-03-30T00:46:14.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7ccdc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc806c360 con 0x7f7cc815cc00 2024-03-30T00:46:14.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7cc6a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7cc815cc00 0x7f7cc8099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:14.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7cc6a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7cc815cc00 0x7f7cc8099760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44292/0 (socket says 172.21.15.66:44292) 2024-03-30T00:46:14.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7cc6a00640 1 -- 172.21.15.66:0/755523084 learned_addr learned my addr 172.21.15.66:0/755523084 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:14.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7cc7400640 1 --1- 172.21.15.66:0/755523084 >> v1:172.21.15.66:6789/0 conn(0x7f7cc8098a20 0x7f7cc80cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48974/0 (socket says 172.21.15.66:48974) 2024-03-30T00:46:14.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7cc6a00640 1 -- 172.21.15.66:0/755523084 >> v1:172.21.15.156:6789/0 conn(0x7f7cc80b18b0 legacy=0x7f7cc80cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:14.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7cc6a00640 1 -- 172.21.15.66:0/755523084 >> v1:172.21.15.66:6789/0 conn(0x7f7cc8098a20 legacy=0x7f7cc80cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:14.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7cc6a00640 1 -- 172.21.15.66:0/755523084 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc806bac0 con 0x7f7cc815cc00 2024-03-30T00:46:14.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.038+0000 7f7cc6000640 1 -- 172.21.15.66:0/755523084 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681440913 0 0) 0x7f7cc80ceef0 con 0x7f7cc8098a20 2024-03-30T00:46:14.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7cc6a00640 1 --2- 172.21.15.66:0/755523084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cc815cc00 0x7f7cc8099760 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009870 tx=0x7f7cb801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b1d6d0705618e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:14.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7cc6000640 1 -- 172.21.15.66:0/755523084 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80a8a50 con 0x7f7cc815cc00 2024-03-30T00:46:14.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7cc6000640 1 -- 172.21.15.66:0/755523084 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb80a8bb0 con 0x7f7cc815cc00 2024-03-30T00:46:14.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7cc6000640 1 -- 172.21.15.66:0/755523084 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80b0440 con 0x7f7cc815cc00 2024-03-30T00:46:14.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7ccdc00640 1 -- 172.21.15.66:0/755523084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cc815cc00 msgr2=0x7f7cc8099760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:14.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7ccdc00640 1 --2- 172.21.15.66:0/755523084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cc815cc00 0x7f7cc8099760 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009870 tx=0x7f7cb801ed50 comp rx=0 tx=0).stop 2024-03-30T00:46:14.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7ccdc00640 1 -- 172.21.15.66:0/755523084 shutdown_connections 2024-03-30T00:46:14.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7ccdc00640 1 --2- 172.21.15.66:0/755523084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cc815cc00 0x7f7cc8099760 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:14.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7ccdc00640 1 -- 172.21.15.66:0/755523084 >> 172.21.15.66:0/755523084 conn(0x7f7cc80b20a0 msgr2=0x7f7cc80b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:14.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7ccdc00640 1 -- 172.21.15.66:0/755523084 shutdown_connections 2024-03-30T00:46:14.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7ccdc00640 1 -- 172.21.15.66:0/755523084 wait complete. 2024-03-30T00:46:14.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7ccdc00640 1 Processor -- start 2024-03-30T00:46:14.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.042+0000 7f7ccdc00640 1 -- start start 2024-03-30T00:46:14.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7ccdc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7cc8098a20 0x7f7cc809dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:14.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7ccdc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cc80b18b0 0x7f7cc8159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:14.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7ccdc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cc815cc00 0x7f7cc81f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:14.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7ccdc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc80c9850 con 0x7f7cc815cc00 2024-03-30T00:46:14.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7ccdc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc81f6640 con 0x7f7cc80b18b0 2024-03-30T00:46:14.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7ccdc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc80c9aa0 con 0x7f7cc8098a20 2024-03-30T00:46:14.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cc7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cc815cc00 0x7f7cc81f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:14.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cc7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cc815cc00 0x7f7cc81f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44306/0 (socket says 172.21.15.66:44306) 2024-03-30T00:46:14.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cc7400640 1 -- 172.21.15.66:0/2010323647 learned_addr learned my addr 172.21.15.66:0/2010323647 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:14.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cbfe00640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cc80b18b0 0x7f7cc8159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cc6a00640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7cc8098a20 0x7f7cc809dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cc7400640 1 -- 172.21.15.66:0/2010323647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7cc8098a20 msgr2=0x7f7cc809dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cc7400640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7cc8098a20 0x7f7cc809dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cc7400640 1 -- 172.21.15.66:0/2010323647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cc80b18b0 msgr2=0x7f7cc8159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cc7400640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cc80b18b0 0x7f7cc8159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cc7400640 1 -- 172.21.15.66:0/2010323647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb80094d0 con 0x7f7cc815cc00 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cc7400640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cc815cc00 0x7f7cc81f6100 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000c7a0 tx=0x7f7cc000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cbea00640 1 -- 172.21.15.66:0/2010323647 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc000a640 con 0x7f7cc815cc00 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cbea00640 1 -- 172.21.15.66:0/2010323647 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc000a7a0 con 0x7f7cc815cc00 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cbea00640 1 -- 172.21.15.66:0/2010323647 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc000abd0 con 0x7f7cc815cc00 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7ccdc00640 1 -- 172.21.15.66:0/2010323647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc80c9d80 con 0x7f7cc815cc00 2024-03-30T00:46:14.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7ccdc00640 1 -- 172.21.15.66:0/2010323647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc80b34b0 con 0x7f7cc815cc00 2024-03-30T00:46:14.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7ccdc00640 1 -- 172.21.15.66:0/2010323647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc80b36d0 con 0x7f7cc815cc00 2024-03-30T00:46:14.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.046+0000 7f7cbea00640 1 -- 172.21.15.66:0/2010323647 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7cc000ad30 con 0x7f7cc815cc00 2024-03-30T00:46:14.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.050+0000 7f7cbea00640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ca4066690 0x7f7ca4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:14.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.050+0000 7f7cbea00640 1 -- 172.21.15.66:0/2010323647 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f7cc0083aa0 con 0x7f7cc815cc00 2024-03-30T00:46:14.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.050+0000 7f7cc6a00640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ca4066690 0x7f7ca4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:14.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.050+0000 7f7cc6a00640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ca4066690 0x7f7ca4068b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80b2000 tx=0x7f7cb80aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:14.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.054+0000 7f7cbea00640 1 -- 172.21.15.66:0/2010323647 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7cc0050eb0 con 0x7f7cc815cc00 2024-03-30T00:46:14.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.214+0000 7f7ccdc00640 1 -- 172.21.15.66:0/2010323647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7cc809e280 con 0x7f7cc815cc00 2024-03-30T00:46:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.866+0000 7f7cbea00640 1 -- 172.21.15.66:0/2010323647 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v93) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7cc0018030 con 0x7f7cc815cc00 2024-03-30T00:46:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-03-30T00:46:14.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.874+0000 7f7ccdc00640 1 -- 172.21.15.66:0/2010323647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ca4066690 msgr2=0x7f7ca4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:14.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.874+0000 7f7ccdc00640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ca4066690 0x7f7ca4068b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80b2000 tx=0x7f7cb80aa070 comp rx=0 tx=0).stop 2024-03-30T00:46:14.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.874+0000 7f7ccdc00640 1 -- 172.21.15.66:0/2010323647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cc815cc00 msgr2=0x7f7cc81f6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:14.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.874+0000 7f7ccdc00640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cc815cc00 0x7f7cc81f6100 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000c7a0 tx=0x7f7cc000cc60 comp rx=0 tx=0).stop 2024-03-30T00:46:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.878+0000 7f7ccdc00640 1 -- 172.21.15.66:0/2010323647 shutdown_connections 2024-03-30T00:46:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.878+0000 7f7ccdc00640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ca4066690 0x7f7ca4068b50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.878+0000 7f7ccdc00640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cc815cc00 0x7f7cc81f6100 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:14.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.878+0000 7f7ccdc00640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cc80b18b0 0x7f7cc8159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:14.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.878+0000 7f7ccdc00640 1 --2- 172.21.15.66:0/2010323647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7cc8098a20 0x7f7cc809dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:14.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.878+0000 7f7ccdc00640 1 -- 172.21.15.66:0/2010323647 >> 172.21.15.66:0/2010323647 conn(0x7f7cc80b20a0 msgr2=0x7f7cc81605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:14.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.882+0000 7f7ccdc00640 1 -- 172.21.15.66:0/2010323647 shutdown_connections 2024-03-30T00:46:14.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:14.882+0000 7f7ccdc00640 1 -- 172.21.15.66:0/2010323647 wait complete. 2024-03-30T00:46:14.919 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T00:46:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.054+0000 7f9f3b400640 1 Processor -- start 2024-03-30T00:46:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.054+0000 7f9f3b400640 1 -- start start 2024-03-30T00:46:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.054+0000 7f9f3b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f340cf2f0 0x7f9f3406b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.054+0000 7f9f3b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9f340cb570 0x7f9f340cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.054+0000 7f9f3b400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f340cb0e0 con 0x7f9f3415ca80 2024-03-30T00:46:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.054+0000 7f9f3b400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f340cf7f0 con 0x7f9f340cb570 2024-03-30T00:46:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.054+0000 7f9f3b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f340cfad0 con 0x7f9f340cf2f0 2024-03-30T00:46:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.054+0000 7f9f39400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9f340cb570 0x7f9f340cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.054+0000 7f9f39400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9f340cb570 0x7f9f340cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44316/0 (socket says 172.21.15.66:44316) 2024-03-30T00:46:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.054+0000 7f9f39400640 1 -- 172.21.15.66:0/3275485392 learned_addr learned my addr 172.21.15.66:0/3275485392 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.054+0000 7f9f2be00640 1 --2- 172.21.15.66:0/3275485392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f340cf2f0 0x7f9f3406b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f39400640 1 -- 172.21.15.66:0/3275485392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f340cf2f0 msgr2=0x7f9f3406b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f39400640 1 --2- 172.21.15.66:0/3275485392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f340cf2f0 0x7f9f3406b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f39400640 1 -- 172.21.15.66:0/3275485392 >> v1:172.21.15.156:6789/0 conn(0x7f9f3415ca80 legacy=0x7f9f3406b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f39400640 1 -- 172.21.15.66:0/3275485392 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f340cad70 con 0x7f9f340cb570 2024-03-30T00:46:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f33e00640 1 -- 172.21.15.66:0/3275485392 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688188128 0 0) 0x7f9f340cb0e0 con 0x7f9f3415ca80 2024-03-30T00:46:15.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f39400640 1 --2- 172.21.15.66:0/3275485392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f340cb570 0x7f9f340cba20 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009e50 tx=0x7f9f2c01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7c9d5bde8cf923 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f33e00640 1 -- 172.21.15.66:0/3275485392 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0af400 con 0x7f9f340cb570 2024-03-30T00:46:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f33e00640 1 -- 172.21.15.66:0/3275485392 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c0ab050 con 0x7f9f340cb570 2024-03-30T00:46:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f33e00640 1 -- 172.21.15.66:0/3275485392 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0af780 con 0x7f9f340cb570 2024-03-30T00:46:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f3b400640 1 -- 172.21.15.66:0/3275485392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f340cb570 msgr2=0x7f9f340cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f3b400640 1 --2- 172.21.15.66:0/3275485392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f340cb570 0x7f9f340cba20 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009e50 tx=0x7f9f2c01f210 comp rx=0 tx=0).stop 2024-03-30T00:46:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f3b400640 1 -- 172.21.15.66:0/3275485392 shutdown_connections 2024-03-30T00:46:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f3b400640 1 --2- 172.21.15.66:0/3275485392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f340cb570 0x7f9f340cba20 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f3b400640 1 --2- 172.21.15.66:0/3275485392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f340cf2f0 0x7f9f3406b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.058+0000 7f9f3b400640 1 -- 172.21.15.66:0/3275485392 >> 172.21.15.66:0/3275485392 conn(0x7f9f340b2090 msgr2=0x7f9f340b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 -- 172.21.15.66:0/3275485392 shutdown_connections 2024-03-30T00:46:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 -- 172.21.15.66:0/3275485392 wait complete. 2024-03-30T00:46:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 Processor -- start 2024-03-30T00:46:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 -- start start 2024-03-30T00:46:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f340cb570 0x7f9f3409b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f340cf2f0 0x7f9f3409d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f3415ca80 0x7f9f340c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f340b4360 con 0x7f9f340cb570 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f340b4bc0 con 0x7f9f3415ca80 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3409e0c0 con 0x7f9f340cf2f0 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f38a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f340cb570 0x7f9f3409b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f38a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f340cb570 0x7f9f3409b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44320/0 (socket says 172.21.15.66:44320) 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f38a00640 1 -- 172.21.15.66:0/2755549538 learned_addr learned my addr 172.21.15.66:0/2755549538 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f2be00640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f340cf2f0 0x7f9f3409d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f39400640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f3415ca80 0x7f9f340c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f38a00640 1 -- 172.21.15.66:0/2755549538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f340cf2f0 msgr2=0x7f9f3409d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f38a00640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f340cf2f0 0x7f9f3409d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f38a00640 1 -- 172.21.15.66:0/2755549538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f3415ca80 msgr2=0x7f9f340c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f38a00640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f3415ca80 0x7f9f340c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f38a00640 1 -- 172.21.15.66:0/2755549538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c009b30 con 0x7f9f340cb570 2024-03-30T00:46:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f2be00640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f340cf2f0 0x7f9f3409d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f39400640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f3415ca80 0x7f9f340c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f38a00640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f340cb570 0x7f9f3409b4c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400c410 tx=0x7f9f2400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f31600640 1 -- 172.21.15.66:0/2755549538 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f240054e0 con 0x7f9f340cb570 2024-03-30T00:46:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f31600640 1 -- 172.21.15.66:0/2755549538 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f24003600 con 0x7f9f340cb570 2024-03-30T00:46:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f31600640 1 -- 172.21.15.66:0/2755549538 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f240039b0 con 0x7f9f340cb570 2024-03-30T00:46:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 -- 172.21.15.66:0/2755549538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f340b4570 con 0x7f9f340cb570 2024-03-30T00:46:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.062+0000 7f9f3b400640 1 -- 172.21.15.66:0/2755549538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f341f2010 con 0x7f9f340cb570 2024-03-30T00:46:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.066+0000 7f9f3b400640 1 -- 172.21.15.66:0/2755549538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f34066c30 con 0x7f9f340cb570 2024-03-30T00:46:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.066+0000 7f9f31600640 1 -- 172.21.15.66:0/2755549538 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9f24003b10 con 0x7f9f340cb570 2024-03-30T00:46:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.066+0000 7f9f31600640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f10066690 0x7f9f10068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.066+0000 7f9f31600640 1 -- 172.21.15.66:0/2755549538 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f9f24083910 con 0x7f9f340cb570 2024-03-30T00:46:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.066+0000 7f9f2be00640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f10066690 0x7f9f10068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.070+0000 7f9f2be00640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f10066690 0x7f9f10068b50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9f340b1d90 tx=0x7f9f1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:15.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.070+0000 7f9f31600640 1 -- 172.21.15.66:0/2755549538 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9f24054590 con 0x7f9f340cb570 2024-03-30T00:46:15.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.266+0000 7f9f3b400640 1 -- 172.21.15.66:0/2755549538 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9f3415c220 con 0x7f9f340cb570 2024-03-30T00:46:15.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.278+0000 7f9f31600640 1 -- 172.21.15.66:0/2755549538 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v55) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9f24017020 con 0x7f9f340cb570 2024-03-30T00:46:15.290 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 13 and data pool 14 2024-03-30T00:46:15.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 -- 172.21.15.66:0/2755549538 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f10066690 msgr2=0x7f9f10068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f10066690 0x7f9f10068b50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9f340b1d90 tx=0x7f9f1c008040 comp rx=0 tx=0).stop 2024-03-30T00:46:15.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 -- 172.21.15.66:0/2755549538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f340cb570 msgr2=0x7f9f3409b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f340cb570 0x7f9f3409b4c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400c410 tx=0x7f9f2400c8d0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 -- 172.21.15.66:0/2755549538 shutdown_connections 2024-03-30T00:46:15.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f3415ca80 0x7f9f340c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f340cf2f0 0x7f9f3409d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9f10066690 0x7f9f10068b50 secure :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9f340b1d90 tx=0x7f9f1c008040 comp rx=0 tx=0).stop 2024-03-30T00:46:15.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 --2- 172.21.15.66:0/2755549538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f340cb570 0x7f9f3409b4c0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 -- 172.21.15.66:0/2755549538 >> 172.21.15.66:0/2755549538 conn(0x7f9f340b2090 msgr2=0x7f9f3406e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:15.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 -- 172.21.15.66:0/2755549538 shutdown_connections 2024-03-30T00:46:15.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.286+0000 7f9f3b400640 1 -- 172.21.15.66:0/2755549538 wait complete. 2024-03-30T00:46:15.330 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:46:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867e200640 1 Processor -- start 2024-03-30T00:46:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867e200640 1 -- start start 2024-03-30T00:46:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867e200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f867815f1e0 0x7f86780d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867e200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f86780d0620 0x7f86780ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867e200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780d0a70 con 0x7f867815a960 2024-03-30T00:46:15.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867e200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867815ed50 con 0x7f86780d0620 2024-03-30T00:46:15.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867e200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86780b6280 con 0x7f867815f1e0 2024-03-30T00:46:15.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867d800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f86780d0620 0x7f86780ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867d800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f86780d0620 0x7f86780ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44326/0 (socket says 172.21.15.66:44326) 2024-03-30T00:46:15.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867d800640 1 -- 172.21.15.66:0/3170465969 learned_addr learned my addr 172.21.15.66:0/3170465969 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:15.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f866fe00640 1 --2- 172.21.15.66:0/3170465969 >> v2:172.21.15.156:3300/0 conn(0x7f867815f1e0 0x7f86780d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867d800640 1 -- 172.21.15.66:0/3170465969 >> v2:172.21.15.156:3300/0 conn(0x7f867815f1e0 msgr2=0x7f86780d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867d800640 1 --2- 172.21.15.66:0/3170465969 >> v2:172.21.15.156:3300/0 conn(0x7f867815f1e0 0x7f86780d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867d800640 1 -- 172.21.15.66:0/3170465969 >> v1:172.21.15.156:6789/0 conn(0x7f867815a960 legacy=0x7f86780caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f867d800640 1 -- 172.21.15.66:0/3170465969 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780cb560 con 0x7f86780d0620 2024-03-30T00:46:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.462+0000 7f866fe00640 1 --2- 172.21.15.66:0/3170465969 >> v2:172.21.15.156:3300/0 conn(0x7f867815f1e0 0x7f86780d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:15.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f867d800640 1 --2- 172.21.15.66:0/3170465969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86780d0620 0x7f86780ce460 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f86780b5da0 tx=0x7f867001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb72e0ae18315d79 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f8677e00640 1 -- 172.21.15.66:0/3170465969 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700a57f0 con 0x7f86780d0620 2024-03-30T00:46:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f8677e00640 1 -- 172.21.15.66:0/3170465969 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86700a5950 con 0x7f86780d0620 2024-03-30T00:46:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f8677e00640 1 -- 172.21.15.66:0/3170465969 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700a5ce0 con 0x7f86780d0620 2024-03-30T00:46:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f867e200640 1 -- 172.21.15.66:0/3170465969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86780d0620 msgr2=0x7f86780ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f867e200640 1 --2- 172.21.15.66:0/3170465969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86780d0620 0x7f86780ce460 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f86780b5da0 tx=0x7f867001c930 comp rx=0 tx=0).stop 2024-03-30T00:46:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f867e200640 1 -- 172.21.15.66:0/3170465969 shutdown_connections 2024-03-30T00:46:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f867e200640 1 --2- 172.21.15.66:0/3170465969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86780d0620 0x7f86780ce460 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f867e200640 1 --2- 172.21.15.66:0/3170465969 >> v2:172.21.15.156:3300/0 conn(0x7f867815f1e0 0x7f86780d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f867e200640 1 -- 172.21.15.66:0/3170465969 >> 172.21.15.66:0/3170465969 conn(0x7f867806e0d0 msgr2=0x7f867806e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f867e200640 1 -- 172.21.15.66:0/3170465969 shutdown_connections 2024-03-30T00:46:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f867e200640 1 -- 172.21.15.66:0/3170465969 wait complete. 2024-03-30T00:46:15.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.466+0000 7f867e200640 1 Processor -- start 2024-03-30T00:46:15.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867e200640 1 -- start start 2024-03-30T00:46:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867e200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86780d0620 0x7f86781514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f867815a960 0x7f86781541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867e200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f867815f1e0 0x7f867815d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c7290 con 0x7f867815a960 2024-03-30T00:46:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867e200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c6a80 con 0x7f86780d0620 2024-03-30T00:46:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867e200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c6cd0 con 0x7f867815f1e0 2024-03-30T00:46:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f866fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f867815a960 0x7f86781541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f866fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f867815a960 0x7f86781541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44336/0 (socket says 172.21.15.66:44336) 2024-03-30T00:46:15.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f866fe00640 1 -- 172.21.15.66:0/1461477300 learned_addr learned my addr 172.21.15.66:0/1461477300 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867d800640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f867815f1e0 0x7f867815d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867ce00640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86780d0620 0x7f86781514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f866fe00640 1 -- 172.21.15.66:0/1461477300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f867815f1e0 msgr2=0x7f867815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f866fe00640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f867815f1e0 0x7f867815d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f866fe00640 1 -- 172.21.15.66:0/1461477300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86780d0620 msgr2=0x7f86781514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f866fe00640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86780d0620 0x7f86781514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f866fe00640 1 -- 172.21.15.66:0/1461477300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670009b30 con 0x7f867815a960 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f866fe00640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f867815a960 0x7f86781541a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f866000d800 tx=0x7f866000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f8675600640 1 -- 172.21.15.66:0/1461477300 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86600096a0 con 0x7f867815a960 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f8675600640 1 -- 172.21.15.66:0/1461477300 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8660009800 con 0x7f867815a960 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867e200640 1 -- 172.21.15.66:0/1461477300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86780c6fb0 con 0x7f867815a960 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f8675600640 1 -- 172.21.15.66:0/1461477300 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660009bb0 con 0x7f867815a960 2024-03-30T00:46:15.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867e200640 1 -- 172.21.15.66:0/1461477300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86780c8530 con 0x7f867815a960 2024-03-30T00:46:15.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.470+0000 7f867e200640 1 -- 172.21.15.66:0/1461477300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86780c80b0 con 0x7f867815a960 2024-03-30T00:46:15.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.474+0000 7f8675600640 1 -- 172.21.15.66:0/1461477300 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8660009d10 con 0x7f867815a960 2024-03-30T00:46:15.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.474+0000 7f8675600640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86540666c0 0x7f8654068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.474+0000 7f8675600640 1 -- 172.21.15.66:0/1461477300 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f86600847c0 con 0x7f867815a960 2024-03-30T00:46:15.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.474+0000 7f867ce00640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86540666c0 0x7f8654068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.474+0000 7f867ce00640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86540666c0 0x7f8654068b80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f86780c73f0 tx=0x7f8668009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:15.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.478+0000 7f8675600640 1 -- 172.21.15.66:0/1461477300 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f86780c80b0 con 0x7f867815a960 2024-03-30T00:46:15.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.642+0000 7f867e200640 1 -- 172.21.15.66:0/1461477300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86780c80b0 con 0x7f867815a960 2024-03-30T00:46:15.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.642+0000 7f8675600640 1 -- 172.21.15.66:0/1461477300 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16262 (secure 0 0 0) 0x7f86780c80b0 con 0x7f867815a960 2024-03-30T00:46:15.651 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:15.651 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":94,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:46:15.271646+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:46:13.811742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-30T00:46:14.265142+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.642+0000 7f867e200640 1 -- 172.21.15.66:0/1461477300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86540666c0 msgr2=0x7f8654068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.642+0000 7f867e200640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86540666c0 0x7f8654068b80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f86780c73f0 tx=0x7f8668009040 comp rx=0 tx=0).stop 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.642+0000 7f867e200640 1 -- 172.21.15.66:0/1461477300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f867815a960 msgr2=0x7f86781541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.642+0000 7f867e200640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f867815a960 0x7f86781541a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f866000d800 tx=0x7f866000dcc0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.646+0000 7f867e200640 1 -- 172.21.15.66:0/1461477300 shutdown_connections 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.646+0000 7f867e200640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f867815f1e0 0x7f867815d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.646+0000 7f867e200640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f86540666c0 0x7f8654068b80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.646+0000 7f867e200640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f867815a960 0x7f86781541a0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.646+0000 7f867e200640 1 --2- 172.21.15.66:0/1461477300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86780d0620 0x7f86781514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.646+0000 7f867e200640 1 -- 172.21.15.66:0/1461477300 >> 172.21.15.66:0/1461477300 conn(0x7f867806e0d0 msgr2=0x7f86780caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.646+0000 7f867e200640 1 -- 172.21.15.66:0/1461477300 shutdown_connections 2024-03-30T00:46:15.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.646+0000 7f867e200640 1 -- 172.21.15.66:0/1461477300 wait complete. 2024-03-30T00:46:15.672 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:46:15.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.802+0000 7ff27a800640 1 Processor -- start 2024-03-30T00:46:15.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.802+0000 7ff27a800640 1 -- start start 2024-03-30T00:46:15.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.806+0000 7ff27a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff2740cda90 0x7ff2740ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.806+0000 7ff27a800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2740cd3e0 con 0x7ff2740cde70 2024-03-30T00:46:15.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.806+0000 7ff27a800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27415a990 con 0x7ff27415a370 2024-03-30T00:46:15.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.806+0000 7ff27a800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2740b1b80 con 0x7ff2740cda90 2024-03-30T00:46:15.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.806+0000 7ff273400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff2740cda90 0x7ff2740ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.806+0000 7ff26aa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff2740cde70 0x7ff274159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48980/0 (socket says 172.21.15.66:48980) 2024-03-30T00:46:15.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.806+0000 7ff26aa00640 1 -- 172.21.15.66:0/1720236215 learned_addr learned my addr 172.21.15.66:0/1720236215 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:15.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.806+0000 7ff273400640 1 -- 172.21.15.66:0/1720236215 >> v1:172.21.15.156:6789/0 conn(0x7ff27415a370 legacy=0x7ff274156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:15.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.806+0000 7ff273400640 1 -- 172.21.15.66:0/1720236215 >> v1:172.21.15.66:6789/0 conn(0x7ff2740cde70 legacy=0x7ff274159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.806+0000 7ff272a00640 1 -- 172.21.15.66:0/1720236215 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423780151 0 0) 0x7ff2740cd3e0 con 0x7ff2740cde70 2024-03-30T00:46:15.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.806+0000 7ff273400640 1 -- 172.21.15.66:0/1720236215 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2740b11e0 con 0x7ff2740cda90 2024-03-30T00:46:15.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff273400640 1 --2- 172.21.15.66:0/1720236215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2740cda90 0x7ff2740ce520 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff2640097c0 tx=0x7ff26401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6613966f8b74b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:15.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff272a00640 1 -- 172.21.15.66:0/1720236215 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2640ab600 con 0x7ff2740cda90 2024-03-30T00:46:15.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff272a00640 1 -- 172.21.15.66:0/1720236215 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2640a7050 con 0x7ff2740cda90 2024-03-30T00:46:15.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff272a00640 1 -- 172.21.15.66:0/1720236215 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2640ab980 con 0x7ff2740cda90 2024-03-30T00:46:15.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff27a800640 1 -- 172.21.15.66:0/1720236215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2740cda90 msgr2=0x7ff2740ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff27a800640 1 --2- 172.21.15.66:0/1720236215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2740cda90 0x7ff2740ce520 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff2640097c0 tx=0x7ff26401c3d0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff27a800640 1 -- 172.21.15.66:0/1720236215 shutdown_connections 2024-03-30T00:46:15.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff27a800640 1 --2- 172.21.15.66:0/1720236215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2740cda90 0x7ff2740ce520 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff27a800640 1 -- 172.21.15.66:0/1720236215 >> 172.21.15.66:0/1720236215 conn(0x7ff274069f60 msgr2=0x7ff27406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:15.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff27a800640 1 -- 172.21.15.66:0/1720236215 shutdown_connections 2024-03-30T00:46:15.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff27a800640 1 -- 172.21.15.66:0/1720236215 wait complete. 2024-03-30T00:46:15.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff27a800640 1 Processor -- start 2024-03-30T00:46:15.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.810+0000 7ff27a800640 1 -- start start 2024-03-30T00:46:15.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff27a800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2740cda90 0x7ff274164c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff27a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2740cde70 0x7ff274097830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff27a800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff27415a370 0x7ff27415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff27a800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2741647a0 con 0x7ff2740cde70 2024-03-30T00:46:15.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff27a800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff274071fd0 con 0x7ff27415a370 2024-03-30T00:46:15.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff27a800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff274164150 con 0x7ff2740cda90 2024-03-30T00:46:15.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2740cde70 0x7ff274097830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2740cde70 0x7ff274097830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44348/0 (socket says 172.21.15.66:44348) 2024-03-30T00:46:15.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26aa00640 1 -- 172.21.15.66:0/2506137557 learned_addr learned my addr 172.21.15.66:0/2506137557 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff273400640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2740cda90 0x7ff274164c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff273e00640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff27415a370 0x7ff27415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26aa00640 1 -- 172.21.15.66:0/2506137557 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2740cda90 msgr2=0x7ff274164c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26aa00640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2740cda90 0x7ff274164c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26aa00640 1 -- 172.21.15.66:0/2506137557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff27415a370 msgr2=0x7ff27415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26aa00640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff27415a370 0x7ff27415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26aa00640 1 -- 172.21.15.66:0/2506137557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff264009420 con 0x7ff2740cde70 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff273400640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2740cda90 0x7ff274164c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26aa00640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2740cde70 0x7ff274097830 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00c4b0 tx=0x7ff25c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26be00640 1 -- 172.21.15.66:0/2506137557 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0074e0 con 0x7ff2740cde70 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26be00640 1 -- 172.21.15.66:0/2506137557 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c007660 con 0x7ff2740cde70 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff27a800640 1 -- 172.21.15.66:0/2506137557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff274164430 con 0x7ff2740cde70 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26be00640 1 -- 172.21.15.66:0/2506137557 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c004a30 con 0x7ff2740cde70 2024-03-30T00:46:15.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff27a800640 1 -- 172.21.15.66:0/2506137557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff274097d70 con 0x7ff2740cde70 2024-03-30T00:46:15.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff27a800640 1 -- 172.21.15.66:0/2506137557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff274066c30 con 0x7ff2740cde70 2024-03-30T00:46:15.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.814+0000 7ff26be00640 1 -- 172.21.15.66:0/2506137557 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff25c004b90 con 0x7ff2740cde70 2024-03-30T00:46:15.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.818+0000 7ff26be00640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff250066690 0x7ff250068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:15.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.818+0000 7ff26be00640 1 -- 172.21.15.66:0/2506137557 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff25c083890 con 0x7ff2740cde70 2024-03-30T00:46:15.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.818+0000 7ff273400640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff250066690 0x7ff250068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:15.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.818+0000 7ff273400640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff250066690 0x7ff250068b50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff2640ad000 tx=0x7ff264008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:15.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.822+0000 7ff26be00640 1 -- 172.21.15.66:0/2506137557 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff25c050ae0 con 0x7ff2740cde70 2024-03-30T00:46:15.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.986+0000 7ff27a800640 1 -- 172.21.15.66:0/2506137557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff274165280 con 0x7ff2740cde70 2024-03-30T00:46:15.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.986+0000 7ff26be00640 1 -- 172.21.15.66:0/2506137557 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16262 (secure 0 0 0) 0x7ff25c018090 con 0x7ff2740cde70 2024-03-30T00:46:15.995 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:15.995 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":94,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:46:15.271646+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:46:13.811742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-30T00:46:14.265142+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:46:15.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 -- 172.21.15.66:0/2506137557 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff250066690 msgr2=0x7ff250068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff250066690 0x7ff250068b50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff2640ad000 tx=0x7ff264008040 comp rx=0 tx=0).stop 2024-03-30T00:46:15.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 -- 172.21.15.66:0/2506137557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2740cde70 msgr2=0x7ff274097830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:15.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2740cde70 0x7ff274097830 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00c4b0 tx=0x7ff25c00c970 comp rx=0 tx=0).stop 2024-03-30T00:46:15.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 -- 172.21.15.66:0/2506137557 shutdown_connections 2024-03-30T00:46:15.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff27415a370 0x7ff27415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff250066690 0x7ff250068b50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2740cde70 0x7ff274097830 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 --2- 172.21.15.66:0/2506137557 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2740cda90 0x7ff274164c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:15.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 -- 172.21.15.66:0/2506137557 >> 172.21.15.66:0/2506137557 conn(0x7ff274069f60 msgr2=0x7ff27406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:15.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 -- 172.21.15.66:0/2506137557 shutdown_connections 2024-03-30T00:46:15.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:15.990+0000 7ff27a800640 1 -- 172.21.15.66:0/2506137557 wait complete. 2024-03-30T00:46:16.016 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T00:46:16.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.150+0000 7efcd9000640 1 Processor -- start 2024-03-30T00:46:16.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.150+0000 7efcd9000640 1 -- start start 2024-03-30T00:46:16.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd9000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd40cda90 0x7efcd40ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:16.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd9000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd40cd3e0 con 0x7efcd415a370 2024-03-30T00:46:16.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd9000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd415a990 con 0x7efcd40cde70 2024-03-30T00:46:16.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd9000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40b1b80 con 0x7efcd40cda90 2024-03-30T00:46:16.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd2800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efcd415a370 0x7efcd4156910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48994/0 (socket says 172.21.15.66:48994) 2024-03-30T00:46:16.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd2800640 1 -- 172.21.15.66:0/124253561 learned_addr learned my addr 172.21.15.66:0/124253561 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:16.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1e00640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd40cda90 0x7efcd40ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120842082 0 0) 0x7efcd40cd3e0 con 0x7efcd415a370 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcb8003530 con 0x7efcd415a370 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552007175 0 0) 0x7efcd415a990 con 0x7efcd40cde70 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd40cd3e0 con 0x7efcd40cde70 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2082841331 0 0) 0x7efcb8003530 con 0x7efcd415a370 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd40cda90 msgr2=0x7efcd40ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd40cda90 0x7efcd40ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 >> v1:172.21.15.156:6789/0 conn(0x7efcd40cde70 legacy=0x7efcd4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd40b11e0 con 0x7efcd415a370 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7efccc001680 con 0x7efcd415a370 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 >> v1:172.21.15.66:6789/0 conn(0x7efcd415a370 legacy=0x7efcd4156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb8003b40 0x7efcb8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:16.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 0x7efcb8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd40cda90 0x7efcb80051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40b1b80 con 0x7efcb80047d0 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb800baa0 con 0x7efcb8003b40 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb800bc60 con 0x7efcd40cda90 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcc9400640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 0x7efcb8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd1e00640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb8003b40 0x7efcb8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcd2800640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd40cda90 0x7efcb80051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcc9400640 1 -- 172.21.15.66:0/124253561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd40cda90 msgr2=0x7efcb80051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcc9400640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd40cda90 0x7efcb80051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcc9400640 1 -- 172.21.15.66:0/124253561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb8003b40 msgr2=0x7efcb8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcc9400640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb8003b40 0x7efcb8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:16.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.154+0000 7efcc9400640 1 -- 172.21.15.66:0/124253561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb8003530 con 0x7efcb80047d0 2024-03-30T00:46:16.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.158+0000 7efcc9400640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 0x7efcb8004c50 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0018d0 tx=0x7efcbc01c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bbe06a0a28edea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:16.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.158+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0050d0 con 0x7efcb80047d0 2024-03-30T00:46:16.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.158+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc005250 con 0x7efcb80047d0 2024-03-30T00:46:16.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.158+0000 7efcd1400640 1 -- 172.21.15.66:0/124253561 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc003600 con 0x7efcb80047d0 2024-03-30T00:46:16.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.158+0000 7efcd9000640 1 -- 172.21.15.66:0/124253561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 msgr2=0x7efcb8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:16.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.158+0000 7efcd9000640 1 --2- 172.21.15.66:0/124253561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 0x7efcb8004c50 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0018d0 tx=0x7efcbc01c580 comp rx=0 tx=0).stop 2024-03-30T00:46:16.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.158+0000 7efcd2800640 1 -- 172.21.15.66:0/124253561 reap_dead start 2024-03-30T00:46:16.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.158+0000 7efcd9000640 1 -- 172.21.15.66:0/124253561 shutdown_connections 2024-03-30T00:46:16.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.158+0000 7efcd9000640 1 -- 172.21.15.66:0/124253561 >> 172.21.15.66:0/124253561 conn(0x7efcd4069f60 msgr2=0x7efcd406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:16.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.158+0000 7efcd9000640 1 -- 172.21.15.66:0/124253561 shutdown_connections 2024-03-30T00:46:16.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.158+0000 7efcd9000640 1 -- 172.21.15.66:0/124253561 wait complete. 2024-03-30T00:46:16.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd9000640 1 Processor -- start 2024-03-30T00:46:16.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd9000640 1 -- start start 2024-03-30T00:46:16.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 0x7efcd41f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd9000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcd4164640 0x7efcd40ac060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd9000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd4164b90 0x7efcd40ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd9000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40ae470 con 0x7efcb80047d0 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd9000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40ae6c0 con 0x7efcd4164640 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd9000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40ae9a0 con 0x7efcd4164b90 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 0x7efcd41f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 0x7efcd41f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44366/0 (socket says 172.21.15.66:44366) 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd1e00640 1 -- 172.21.15.66:0/365965631 learned_addr learned my addr 172.21.15.66:0/365965631 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd2800640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd4164b90 0x7efcd40ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcc9400640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcd4164640 0x7efcd40ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd1e00640 1 -- 172.21.15.66:0/365965631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd4164b90 msgr2=0x7efcd40ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd1e00640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd4164b90 0x7efcd40ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:16.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd1e00640 1 -- 172.21.15.66:0/365965631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcd4164640 msgr2=0x7efcd40ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:16.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd1e00640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcd4164640 0x7efcd40ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:16.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd1e00640 1 -- 172.21.15.66:0/365965631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd40aec80 con 0x7efcb80047d0 2024-03-30T00:46:16.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcc9400640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcd4164640 0x7efcd40ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:16.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd1e00640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 0x7efcd41f4c90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7efcc4011e30 tx=0x7efcc400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:16.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efccaa00640 1 -- 172.21.15.66:0/365965631 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc400e040 con 0x7efcb80047d0 2024-03-30T00:46:16.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efccaa00640 1 -- 172.21.15.66:0/365965631 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc400f040 con 0x7efcb80047d0 2024-03-30T00:46:16.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd9000640 1 -- 172.21.15.66:0/365965631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd40aef60 con 0x7efcb80047d0 2024-03-30T00:46:16.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efccaa00640 1 -- 172.21.15.66:0/365965631 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4009c00 con 0x7efcb80047d0 2024-03-30T00:46:16.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd9000640 1 -- 172.21.15.66:0/365965631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd41f51d0 con 0x7efcb80047d0 2024-03-30T00:46:16.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.162+0000 7efcd9000640 1 -- 172.21.15.66:0/365965631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd4066c30 con 0x7efcb80047d0 2024-03-30T00:46:16.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.166+0000 7efccaa00640 1 -- 172.21.15.66:0/365965631 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efcc4015850 con 0x7efcb80047d0 2024-03-30T00:46:16.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.166+0000 7efccaa00640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efcb0066690 0x7efcb0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:16.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.166+0000 7efccaa00640 1 -- 172.21.15.66:0/365965631 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efcc4085070 con 0x7efcb80047d0 2024-03-30T00:46:16.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.166+0000 7efcc9400640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efcb0066690 0x7efcb0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:16.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.166+0000 7efcc9400640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efcb0066690 0x7efcb0068b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0ad000 tx=0x7efcbc0a6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:16.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.170+0000 7efccaa00640 1 -- 172.21.15.66:0/365965631 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efcc40b6980 con 0x7efcb80047d0 2024-03-30T00:46:16.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:16.370+0000 7efcd9000640 1 -- 172.21.15.66:0/365965631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efcd40b3170 con 0x7efcb80047d0 2024-03-30T00:46:16.788 INFO:tasks.ceph.mds.a.smithi066.stderr:2024-03-30T00:46:16.782+0000 7f6422a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:46:16.866 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:46:16.870 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:46:17.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.274+0000 7efccaa00640 1 -- 172.21.15.66:0/365965631 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7efcc4052200 con 0x7efcb80047d0 2024-03-30T00:46:17.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 -- 172.21.15.66:0/365965631 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efcb0066690 msgr2=0x7efcb0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:17.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efcb0066690 0x7efcb0068b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0ad000 tx=0x7efcbc0a6040 comp rx=0 tx=0).stop 2024-03-30T00:46:17.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 -- 172.21.15.66:0/365965631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 msgr2=0x7efcd41f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:17.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 0x7efcd41f4c90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7efcc4011e30 tx=0x7efcc400f580 comp rx=0 tx=0).stop 2024-03-30T00:46:17.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 -- 172.21.15.66:0/365965631 shutdown_connections 2024-03-30T00:46:17.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcd4164b90 0x7efcd40ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:17.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcd4164640 0x7efcd40ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:17.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efcb0066690 0x7efcb0068b50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:17.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 --2- 172.21.15.66:0/365965631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb80047d0 0x7efcd41f4c90 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:17.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 -- 172.21.15.66:0/365965631 >> 172.21.15.66:0/365965631 conn(0x7efcd4069f60 msgr2=0x7efcd415e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:17.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 -- 172.21.15.66:0/365965631 shutdown_connections 2024-03-30T00:46:17.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.278+0000 7efcd9000640 1 -- 172.21.15.66:0/365965631 wait complete. 2024-03-30T00:46:17.309 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T00:46:17.310 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T00:46:17.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.446+0000 7fcbc7200640 1 Processor -- start 2024-03-30T00:46:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.446+0000 7fcbc7200640 1 -- start start 2024-03-30T00:46:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.446+0000 7fcbc7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcbc815a720 0x7fcbc80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.446+0000 7fcbc7200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbc80b3170 0x7fcbc80b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc7200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcbc80d07c0 0x7fcbc80d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc80cfb00 con 0x7fcbc815a720 2024-03-30T00:46:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc7200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc80d11b0 con 0x7fcbc80d07c0 2024-03-30T00:46:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc7200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80cb5b0 con 0x7fcbc80b3170 2024-03-30T00:46:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcbc815a720 0x7fcbc80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcbc815a720 0x7fcbc80ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44372/0 (socket says 172.21.15.66:44372) 2024-03-30T00:46:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5e00640 1 -- 172.21.15.66:0/778567553 learned_addr learned my addr 172.21.15.66:0/778567553 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc6800640 1 --2- 172.21.15.66:0/778567553 >> v2:172.21.15.156:3300/0 conn(0x7fcbc80d07c0 0x7fcbc80d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbbfe00640 1 --2- 172.21.15.66:0/778567553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbc80b3170 0x7fcbc80b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5e00640 1 -- 172.21.15.66:0/778567553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbc80b3170 msgr2=0x7fcbc80b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:17.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5e00640 1 --2- 172.21.15.66:0/778567553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbc80b3170 0x7fcbc80b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:17.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5e00640 1 -- 172.21.15.66:0/778567553 >> v2:172.21.15.156:3300/0 conn(0x7fcbc80d07c0 msgr2=0x7fcbc80d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:17.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5e00640 1 --2- 172.21.15.66:0/778567553 >> v2:172.21.15.156:3300/0 conn(0x7fcbc80d07c0 0x7fcbc80d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:17.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5e00640 1 -- 172.21.15.66:0/778567553 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc815ab00 con 0x7fcbc815a720 2024-03-30T00:46:17.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc6800640 1 --2- 172.21.15.66:0/778567553 >> v2:172.21.15.156:3300/0 conn(0x7fcbc80d07c0 0x7fcbc80d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:17.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5e00640 1 --2- 172.21.15.66:0/778567553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbc815a720 0x7fcbc80ca970 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80afd10 tx=0x7fcbb801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3da69e48155f359 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:17.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5400640 1 -- 172.21.15.66:0/778567553 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80aa640 con 0x7fcbc815a720 2024-03-30T00:46:17.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5400640 1 -- 172.21.15.66:0/778567553 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb80aa7a0 con 0x7fcbc815a720 2024-03-30T00:46:17.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.450+0000 7fcbc5400640 1 -- 172.21.15.66:0/778567553 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80aab30 con 0x7fcbc815a720 2024-03-30T00:46:17.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 -- 172.21.15.66:0/778567553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbc815a720 msgr2=0x7fcbc80ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:17.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 --2- 172.21.15.66:0/778567553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbc815a720 0x7fcbc80ca970 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80afd10 tx=0x7fcbb801c520 comp rx=0 tx=0).stop 2024-03-30T00:46:17.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 -- 172.21.15.66:0/778567553 shutdown_connections 2024-03-30T00:46:17.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 --2- 172.21.15.66:0/778567553 >> v2:172.21.15.156:3300/0 conn(0x7fcbc80d07c0 0x7fcbc80d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:17.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 --2- 172.21.15.66:0/778567553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbc80b3170 0x7fcbc80b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:17.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 --2- 172.21.15.66:0/778567553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbc815a720 0x7fcbc80ca970 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:17.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 -- 172.21.15.66:0/778567553 >> 172.21.15.66:0/778567553 conn(0x7fcbc8069d90 msgr2=0x7fcbc806a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:17.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 -- 172.21.15.66:0/778567553 shutdown_connections 2024-03-30T00:46:17.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 -- 172.21.15.66:0/778567553 wait complete. 2024-03-30T00:46:17.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 Processor -- start 2024-03-30T00:46:17.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 -- start start 2024-03-30T00:46:17.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbc80b3170 0x7fcbc81577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:17.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbc80d07c0 0x7fcbc8149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:17.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbc815a720 0x7fcbc814e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:17.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc814a0d0 con 0x7fcbc815a720 2024-03-30T00:46:17.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc806b040 con 0x7fcbc80d07c0 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc7200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc806b320 con 0x7fcbc80b3170 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbc815a720 0x7fcbc814e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbc815a720 0x7fcbc814e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44388/0 (socket says 172.21.15.66:44388) 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc6800640 1 -- 172.21.15.66:0/2103585407 learned_addr learned my addr 172.21.15.66:0/2103585407 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc5e00640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbc80b3170 0x7fcbc81577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbbfe00640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbc80d07c0 0x7fcbc8149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc6800640 1 -- 172.21.15.66:0/2103585407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbc80b3170 msgr2=0x7fcbc81577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc6800640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbc80b3170 0x7fcbc81577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc6800640 1 -- 172.21.15.66:0/2103585407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbc80d07c0 msgr2=0x7fcbc8149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc6800640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbc80d07c0 0x7fcbc8149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.454+0000 7fcbc6800640 1 -- 172.21.15.66:0/2103585407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb80094d0 con 0x7fcbc815a720 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.458+0000 7fcbc6800640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbc815a720 0x7fcbc814e2e0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009fa0 tx=0x7fcbc000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:17.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.458+0000 7fcbbe000640 1 -- 172.21.15.66:0/2103585407 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0003ba0 con 0x7fcbc815a720 2024-03-30T00:46:17.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.458+0000 7fcbbe000640 1 -- 172.21.15.66:0/2103585407 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc000f040 con 0x7fcbc815a720 2024-03-30T00:46:17.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.458+0000 7fcbbe000640 1 -- 172.21.15.66:0/2103585407 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0004030 con 0x7fcbc815a720 2024-03-30T00:46:17.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.458+0000 7fcbc7200640 1 -- 172.21.15.66:0/2103585407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc806b600 con 0x7fcbc815a720 2024-03-30T00:46:17.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.458+0000 7fcbc7200640 1 -- 172.21.15.66:0/2103585407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc806bbc0 con 0x7fcbc815a720 2024-03-30T00:46:17.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.458+0000 7fcbc7200640 1 -- 172.21.15.66:0/2103585407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc806bde0 con 0x7fcbc815a720 2024-03-30T00:46:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.458+0000 7fcbbe000640 1 -- 172.21.15.66:0/2103585407 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcbc0004190 con 0x7fcbc815a720 2024-03-30T00:46:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.458+0000 7fcbbe000640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcba4066690 0x7fcba4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.458+0000 7fcbbe000640 1 -- 172.21.15.66:0/2103585407 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcbc0084570 con 0x7fcbc815a720 2024-03-30T00:46:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.458+0000 7fcbc5e00640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcba4066690 0x7fcba4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:17.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.462+0000 7fcbc5e00640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcba4066690 0x7fcba4068b50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80ae000 tx=0x7fcbb80b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:17.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.462+0000 7fcbbe000640 1 -- 172.21.15.66:0/2103585407 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcbc00518c0 con 0x7fcbc815a720 2024-03-30T00:46:17.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:17.658+0000 7fcbc7200640 1 -- 172.21.15.66:0/2103585407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcbc8073e00 con 0x7fcbc815a720 2024-03-30T00:46:18.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.282+0000 7fcbbe000640 1 -- 172.21.15.66:0/2103585407 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcbc0054ee0 con 0x7fcbc815a720 2024-03-30T00:46:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 -- 172.21.15.66:0/2103585407 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcba4066690 msgr2=0x7fcba4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcba4066690 0x7fcba4068b50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80ae000 tx=0x7fcbb80b5040 comp rx=0 tx=0).stop 2024-03-30T00:46:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 -- 172.21.15.66:0/2103585407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbc815a720 msgr2=0x7fcbc814e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbc815a720 0x7fcbc814e2e0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009fa0 tx=0x7fcbc000d560 comp rx=0 tx=0).stop 2024-03-30T00:46:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 -- 172.21.15.66:0/2103585407 shutdown_connections 2024-03-30T00:46:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcba4066690 0x7fcba4068b50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbc815a720 0x7fcbc814e2e0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbc80d07c0 0x7fcbc8149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 --2- 172.21.15.66:0/2103585407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbc80b3170 0x7fcbc81577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 -- 172.21.15.66:0/2103585407 >> 172.21.15.66:0/2103585407 conn(0x7fcbc8069d90 msgr2=0x7fcbc815e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:18.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 -- 172.21.15.66:0/2103585407 shutdown_connections 2024-03-30T00:46:18.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.286+0000 7fcbc7200640 1 -- 172.21.15.66:0/2103585407 wait complete. 2024-03-30T00:46:18.313 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T00:46:18.314 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:46:18.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f881e600640 1 Processor -- start 2024-03-30T00:46:18.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f881e600640 1 -- start start 2024-03-30T00:46:18.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f881e600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f88180cc560 0x7f88180d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f881e600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 0x7f881806e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f881e600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f881806b0b0 con 0x7f881806e630 2024-03-30T00:46:18.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f881e600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f881806eac0 con 0x7f88180cc560 2024-03-30T00:46:18.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f881e600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8818160640 con 0x7f881806f8e0 2024-03-30T00:46:18.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f8817400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f88180cc560 0x7f88180d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f880ea00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 0x7f881806e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f8817400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f88180cc560 0x7f88180d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:45218/0 (socket says 172.21.15.66:45218) 2024-03-30T00:46:18.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f8817400640 1 -- 172.21.15.66:0/457913283 learned_addr learned my addr 172.21.15.66:0/457913283 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:18.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f880ea00640 1 --2- 172.21.15.66:0/457913283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 0x7f881806e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:41448/0 (socket says 172.21.15.66:41448) 2024-03-30T00:46:18.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f880ea00640 1 -- 172.21.15.66:0/457913283 >> v2:172.21.15.156:3300/0 conn(0x7f88180cc560 msgr2=0x7f88180d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:18.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f880ea00640 1 --2- 172.21.15.66:0/457913283 >> v2:172.21.15.156:3300/0 conn(0x7f88180cc560 0x7f88180d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f880ea00640 1 -- 172.21.15.66:0/457913283 >> v1:172.21.15.156:6789/0 conn(0x7f881806e630 legacy=0x7f88180ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f880ea00640 1 -- 172.21.15.66:0/457913283 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881806b390 con 0x7f881806f8e0 2024-03-30T00:46:18.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.446+0000 7f8817400640 1 --2- 172.21.15.66:0/457913283 >> v2:172.21.15.156:3300/0 conn(0x7f88180cc560 0x7f88180d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:46:18.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f880ea00640 1 --2- 172.21.15.66:0/457913283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 0x7f881806e0f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8800009d90 tx=0x7f880001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=984f14ddfe8c09f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:18.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f8816a00640 1 -- 172.21.15.66:0/457913283 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88000a5920 con 0x7f881806f8e0 2024-03-30T00:46:18.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f8816a00640 1 -- 172.21.15.66:0/457913283 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88000a5a80 con 0x7f881806f8e0 2024-03-30T00:46:18.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f8816a00640 1 -- 172.21.15.66:0/457913283 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88000a5e10 con 0x7f881806f8e0 2024-03-30T00:46:18.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f881e600640 1 -- 172.21.15.66:0/457913283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 msgr2=0x7f881806e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f881e600640 1 --2- 172.21.15.66:0/457913283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 0x7f881806e0f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8800009d90 tx=0x7f880001c940 comp rx=0 tx=0).stop 2024-03-30T00:46:18.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f881e600640 1 -- 172.21.15.66:0/457913283 shutdown_connections 2024-03-30T00:46:18.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f881e600640 1 --2- 172.21.15.66:0/457913283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 0x7f881806e0f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f881e600640 1 --2- 172.21.15.66:0/457913283 >> v2:172.21.15.156:3300/0 conn(0x7f88180cc560 0x7f88180d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f881e600640 1 -- 172.21.15.66:0/457913283 >> 172.21.15.66:0/457913283 conn(0x7f88180d0540 msgr2=0x7f88180d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:18.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f881e600640 1 -- 172.21.15.66:0/457913283 shutdown_connections 2024-03-30T00:46:18.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.450+0000 7f881e600640 1 -- 172.21.15.66:0/457913283 wait complete. 2024-03-30T00:46:18.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f881e600640 1 Processor -- start 2024-03-30T00:46:18.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f881e600640 1 -- start start 2024-03-30T00:46:18.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f881e600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881806e630 0x7f88180b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f881e600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 0x7f88180c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f881e600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88180cc560 0x7f88180c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f881e600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88180c8090 con 0x7f881806e630 2024-03-30T00:46:18.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f881e600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88180c8ea0 con 0x7f88180cc560 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f881e600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88180c8290 con 0x7f881806f8e0 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f8817400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881806e630 0x7f88180b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f8817400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881806e630 0x7f88180b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44402/0 (socket says 172.21.15.66:44402) 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f8817400640 1 -- 172.21.15.66:0/2539500796 learned_addr learned my addr 172.21.15.66:0/2539500796 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f880ea00640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 0x7f88180c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f8817e00640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88180cc560 0x7f88180c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f8817400640 1 -- 172.21.15.66:0/2539500796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 msgr2=0x7f88180c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f8817400640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 0x7f88180c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f8817400640 1 -- 172.21.15.66:0/2539500796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88180cc560 msgr2=0x7f88180c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f8817400640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88180cc560 0x7f88180c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f8817400640 1 -- 172.21.15.66:0/2539500796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88080094d0 con 0x7f881806e630 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f8817400640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881806e630 0x7f88180b1800 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f880800e9d0 tx=0x7f880800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f880fe00640 1 -- 172.21.15.66:0/2539500796 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880800ca60 con 0x7f881806e630 2024-03-30T00:46:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f880fe00640 1 -- 172.21.15.66:0/2539500796 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880800cbc0 con 0x7f881806e630 2024-03-30T00:46:18.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f881e600640 1 -- 172.21.15.66:0/2539500796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8800009a70 con 0x7f881806e630 2024-03-30T00:46:18.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f880fe00640 1 -- 172.21.15.66:0/2539500796 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808013470 con 0x7f881806e630 2024-03-30T00:46:18.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.454+0000 7f881e600640 1 -- 172.21.15.66:0/2539500796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88180c8850 con 0x7f881806e630 2024-03-30T00:46:18.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.458+0000 7f881e600640 1 -- 172.21.15.66:0/2539500796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8818066c30 con 0x7f881806e630 2024-03-30T00:46:18.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.458+0000 7f880fe00640 1 -- 172.21.15.66:0/2539500796 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f88080135d0 con 0x7f881806e630 2024-03-30T00:46:18.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.458+0000 7f880fe00640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f4066690 0x7f87f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.458+0000 7f880fe00640 1 -- 172.21.15.66:0/2539500796 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f88080846f0 con 0x7f881806e630 2024-03-30T00:46:18.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.458+0000 7f880ea00640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f4066690 0x7f87f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.458+0000 7f880ea00640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f4066690 0x7f87f4068b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f88000af000 tx=0x7f88000a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:18.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.462+0000 7f880fe00640 1 -- 172.21.15.66:0/2539500796 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8808051880 con 0x7f881806e630 2024-03-30T00:46:18.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.662+0000 7f881e600640 1 -- 172.21.15.66:0/2539500796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88180afa60 con 0x7f881806e630 2024-03-30T00:46:18.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.662+0000 7f880fe00640 1 -- 172.21.15.66:0/2539500796 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f8808082e60 con 0x7f881806e630 2024-03-30T00:46:18.672 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:46:18.278734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6836/1147664160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1147664160},{"type":"v1","addr":"172.21.15.66:6836","nonce":1147664160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:46:18.672 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 58 2024-03-30T00:46:18.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 -- 172.21.15.66:0/2539500796 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f4066690 msgr2=0x7f87f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f4066690 0x7f87f4068b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f88000af000 tx=0x7f88000a7070 comp rx=0 tx=0).stop 2024-03-30T00:46:18.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 -- 172.21.15.66:0/2539500796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881806e630 msgr2=0x7f88180b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881806e630 0x7f88180b1800 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f880800e9d0 tx=0x7f880800ee90 comp rx=0 tx=0).stop 2024-03-30T00:46:18.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 -- 172.21.15.66:0/2539500796 shutdown_connections 2024-03-30T00:46:18.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88180cc560 0x7f88180c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f881806f8e0 0x7f88180c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f87f4066690 0x7f87f4068b50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 --2- 172.21.15.66:0/2539500796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f881806e630 0x7f88180b1800 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 -- 172.21.15.66:0/2539500796 >> 172.21.15.66:0/2539500796 conn(0x7f88180d0540 msgr2=0x7f88180ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:18.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 -- 172.21.15.66:0/2539500796 shutdown_connections 2024-03-30T00:46:18.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.666+0000 7f881e600640 1 -- 172.21.15.66:0/2539500796 wait complete. 2024-03-30T00:46:18.693 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:46:18.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9ae400640 1 Processor -- start 2024-03-30T00:46:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9ae400640 1 -- start start 2024-03-30T00:46:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9ae400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a815e370 0x7fd9a80cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9ae400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd9a8069d90 0x7fd9a806a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9ae400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a80cf160 con 0x7fd9a815a960 2024-03-30T00:46:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9ae400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a806a760 con 0x7fd9a8069d90 2024-03-30T00:46:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9ae400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a815ad40 con 0x7fd9a815e370 2024-03-30T00:46:18.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9a7e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd9a8069d90 0x7fd9a806a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd99fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a815e370 0x7fd9a80cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9a7400640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd9a815a960 0x7fd9a80cb020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:47670/0 (socket says 172.21.15.66:47670) 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9a7e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd9a8069d90 0x7fd9a806a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:60764/0 (socket says 172.21.15.66:60764) 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9a7400640 1 -- 172.21.15.66:0/260735584 learned_addr learned my addr 172.21.15.66:0/260735584 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9a6a00640 1 -- 172.21.15.66:0/260735584 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252652176 0 0) 0x7fd998001120 con 0x7fd9a815a960 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9a6a00640 1 -- 172.21.15.66:0/260735584 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd98c003530 con 0x7fd9a815a960 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9a7e00640 1 -- 172.21.15.66:0/260735584 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a815e370 msgr2=0x7fd9a80cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/260735584 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a815e370 0x7fd9a80cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9a7e00640 1 -- 172.21.15.66:0/260735584 >> v1:172.21.15.156:6789/0 conn(0x7fd9a815a960 legacy=0x7fd9a80cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.826+0000 7fd9a7e00640 1 -- 172.21.15.66:0/260735584 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a815aed0 con 0x7fd9a8069d90 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/260735584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a8069d90 0x7fd9a806a220 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009d90 tx=0x7fd9a001f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1216f08477b5a802 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9a6a00640 1 -- 172.21.15.66:0/260735584 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00af600 con 0x7fd9a8069d90 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9a6a00640 1 -- 172.21.15.66:0/260735584 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a00ab050 con 0x7fd9a8069d90 2024-03-30T00:46:18.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9a6a00640 1 -- 172.21.15.66:0/260735584 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00af980 con 0x7fd9a8069d90 2024-03-30T00:46:18.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9ae400640 1 -- 172.21.15.66:0/260735584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a8069d90 msgr2=0x7fd9a806a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9ae400640 1 --2- 172.21.15.66:0/260735584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a8069d90 0x7fd9a806a220 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009d90 tx=0x7fd9a001f150 comp rx=0 tx=0).stop 2024-03-30T00:46:18.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9ae400640 1 -- 172.21.15.66:0/260735584 shutdown_connections 2024-03-30T00:46:18.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9ae400640 1 --2- 172.21.15.66:0/260735584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a8069d90 0x7fd9a806a220 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9ae400640 1 --2- 172.21.15.66:0/260735584 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a815e370 0x7fd9a80cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9ae400640 1 -- 172.21.15.66:0/260735584 >> 172.21.15.66:0/260735584 conn(0x7fd9a8098a20 msgr2=0x7fd9a80b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:18.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9ae400640 1 -- 172.21.15.66:0/260735584 shutdown_connections 2024-03-30T00:46:18.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.830+0000 7fd9ae400640 1 -- 172.21.15.66:0/260735584 wait complete. 2024-03-30T00:46:18.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9ae400640 1 Processor -- start 2024-03-30T00:46:18.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9ae400640 1 -- start start 2024-03-30T00:46:18.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9ae400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a8069d90 0x7fd9a814c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9ae400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a815a960 0x7fd9a80c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9ae400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a815e370 0x7fd9a80c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9ae400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80c7500 con 0x7fd9a815e370 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9ae400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80c3960 con 0x7fd9a8069d90 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9ae400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80c6eb0 con 0x7fd9a815a960 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a815e370 0x7fd9a80c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a815e370 0x7fd9a80c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54276/0 (socket says 172.21.15.66:54276) 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9a7e00640 1 -- 172.21.15.66:0/955242628 learned_addr learned my addr 172.21.15.66:0/955242628 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9a7400640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a8069d90 0x7fd9a814c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd99fe00640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a815a960 0x7fd9a80c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9a7e00640 1 -- 172.21.15.66:0/955242628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a815a960 msgr2=0x7fd9a80c2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a815a960 0x7fd9a80c2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9a7e00640 1 -- 172.21.15.66:0/955242628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a8069d90 msgr2=0x7fd9a814c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:18.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a8069d90 0x7fd9a814c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:18.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9a7e00640 1 -- 172.21.15.66:0/955242628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0009a70 con 0x7fd9a815e370 2024-03-30T00:46:18.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9a7e00640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a815e370 0x7fd9a80c5130 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd9a001d800 tx=0x7fd9a001bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:18.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd99f400640 1 -- 172.21.15.66:0/955242628 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00afbf0 con 0x7fd9a815e370 2024-03-30T00:46:18.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd99f400640 1 -- 172.21.15.66:0/955242628 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a00afd50 con 0x7fd9a815e370 2024-03-30T00:46:18.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9ae400640 1 -- 172.21.15.66:0/955242628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a80c7190 con 0x7fd9a815e370 2024-03-30T00:46:18.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd99f400640 1 -- 172.21.15.66:0/955242628 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00ba660 con 0x7fd9a815e370 2024-03-30T00:46:18.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9ae400640 1 -- 172.21.15.66:0/955242628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a80c3c40 con 0x7fd9a815e370 2024-03-30T00:46:18.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.834+0000 7fd9ae400640 1 -- 172.21.15.66:0/955242628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a80c7670 con 0x7fd9a815e370 2024-03-30T00:46:18.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.838+0000 7fd99f400640 1 -- 172.21.15.66:0/955242628 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd9a00ba7c0 con 0x7fd9a815e370 2024-03-30T00:46:18.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.838+0000 7fd99f400640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 0x7fd984068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:18.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.838+0000 7fd99f400640 1 -- 172.21.15.66:0/955242628 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd9a0122ca0 con 0x7fd9a815e370 2024-03-30T00:46:18.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.838+0000 7fd9a7400640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 0x7fd984068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:18.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.838+0000 7fd9a7400640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 0x7fd984068b50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd998005890 tx=0x7fd998009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:18.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:18.842+0000 7fd99f400640 1 -- 172.21.15.66:0/955242628 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd9a00efe50 con 0x7fd9a815e370 2024-03-30T00:46:19.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.006+0000 7fd9ae400640 1 -- 172.21.15.66:0/955242628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9a8066c30 con 0x7fd9a815e370 2024-03-30T00:46:19.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd99f400640 1 -- 172.21.15.66:0/955242628 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16249 (secure 0 0 0) 0x7fd9a00be020 con 0x7fd9a815e370 2024-03-30T00:46:19.017 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:19.017 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":95,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:46:16.274340+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:46:13.811742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-30T00:46:14.265142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:46:19.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 -- 172.21.15.66:0/955242628 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 msgr2=0x7fd984068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 0x7fd984068b50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd998005890 tx=0x7fd998009040 comp rx=0 tx=0).stop 2024-03-30T00:46:19.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 -- 172.21.15.66:0/955242628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a815e370 msgr2=0x7fd9a80c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a815e370 0x7fd9a80c5130 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd9a001d800 tx=0x7fd9a001bef0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 -- 172.21.15.66:0/955242628 shutdown_connections 2024-03-30T00:46:19.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd984066690 0x7fd984068b50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9a815e370 0x7fd9a80c5130 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9a815a960 0x7fd9a80c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 --2- 172.21.15.66:0/955242628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a8069d90 0x7fd9a814c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 -- 172.21.15.66:0/955242628 >> 172.21.15.66:0/955242628 conn(0x7fd9a8098a20 msgr2=0x7fd9a80ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:19.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 -- 172.21.15.66:0/955242628 shutdown_connections 2024-03-30T00:46:19.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.010+0000 7fd9ae400640 1 -- 172.21.15.66:0/955242628 wait complete. 2024-03-30T00:46:19.038 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T00:46:19.038 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:46:19.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d12600640 1 Processor -- start 2024-03-30T00:46:19.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d12600640 1 -- start start 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d12600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d0c15a640 0x7f2d0c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d12600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2d0c0cabb0 0x7f2d0c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d12600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c06d070 con 0x7f2d0c0ce520 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d12600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c0cbba0 con 0x7f2d0c0cabb0 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d12600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c06e2c0 con 0x7f2d0c15a640 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d02a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2d0c0ce520 0x7f2d0c0ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55560/0 (socket says 172.21.15.66:55560) 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d02a00640 1 -- 172.21.15.66:0/3848969174 learned_addr learned my addr 172.21.15.66:0/3848969174 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0b400640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d0c15a640 0x7f2d0c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0be00640 1 --2- 172.21.15.66:0/3848969174 >> v2:172.21.15.156:3300/0 conn(0x7f2d0c0cabb0 0x7f2d0c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608560157 0 0) 0x7f2d0c06d070 con 0x7f2d0c0ce520 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf0003530 con 0x7f2d0c0ce520 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2951442734 0 0) 0x7f2cf0003530 con 0x7f2d0c0ce520 2024-03-30T00:46:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d0c15a640 msgr2=0x7f2d0c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d0c15a640 0x7f2d0c06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 >> v2:172.21.15.156:3300/0 conn(0x7f2d0c0cabb0 msgr2=0x7f2d0c0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 --2- 172.21.15.66:0/3848969174 >> v2:172.21.15.156:3300/0 conn(0x7f2d0c0cabb0 0x7f2d0c0cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0cb8c0 con 0x7f2d0c0ce520 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f2cf4001680 con 0x7f2d0c0ce520 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 >> v1:172.21.15.66:6789/0 conn(0x7f2d0c0ce520 legacy=0x7f2d0c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d0c15a640 0x7f2cf0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cf0003f00 0x7f2cf00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cf0004d10 0x7f2cf0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0cbba0 con 0x7f2cf0003f00 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c06e2c0 con 0x7f2cf0004d10 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf000b990 con 0x7f2d0c15a640 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d02a00640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cf0003f00 0x7f2cf00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0b400640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d0c15a640 0x7f2cf0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.174+0000 7f2d0be00640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cf0004d10 0x7f2cf0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d02a00640 1 -- 172.21.15.66:0/3848969174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d0c15a640 msgr2=0x7f2cf0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:19.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d02a00640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d0c15a640 0x7f2cf0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d02a00640 1 -- 172.21.15.66:0/3848969174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cf0004d10 msgr2=0x7f2cf0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:19.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d02a00640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cf0004d10 0x7f2cf0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d02a00640 1 -- 172.21.15.66:0/3848969174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0003530 con 0x7f2cf0003f00 2024-03-30T00:46:19.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d02a00640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cf0003f00 0x7f2cf00047d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c210 tx=0x7f2cf401f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d56a86bb34370c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:19.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4005ae0 con 0x7f2cf0003f00 2024-03-30T00:46:19.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf40ab070 con 0x7f2cf0003f00 2024-03-30T00:46:19.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d0aa00640 1 -- 172.21.15.66:0/3848969174 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4007070 con 0x7f2cf0003f00 2024-03-30T00:46:19.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d12600640 1 -- 172.21.15.66:0/3848969174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cf0003f00 msgr2=0x7f2cf00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d12600640 1 --2- 172.21.15.66:0/3848969174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cf0003f00 0x7f2cf00047d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c210 tx=0x7f2cf401f370 comp rx=0 tx=0).stop 2024-03-30T00:46:19.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d0be00640 1 -- 172.21.15.66:0/3848969174 reap_dead start 2024-03-30T00:46:19.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d12600640 1 -- 172.21.15.66:0/3848969174 shutdown_connections 2024-03-30T00:46:19.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d12600640 1 -- 172.21.15.66:0/3848969174 >> 172.21.15.66:0/3848969174 conn(0x7f2d0c069f60 msgr2=0x7f2d0c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:19.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d12600640 1 -- 172.21.15.66:0/3848969174 shutdown_connections 2024-03-30T00:46:19.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.178+0000 7f2d12600640 1 -- 172.21.15.66:0/3848969174 wait complete. 2024-03-30T00:46:19.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d12600640 1 Processor -- start 2024-03-30T00:46:19.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d12600640 1 -- start start 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d12600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf0003f00 0x7f2d0c0c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d12600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d0c0ca2a0 0x7f2d0c1f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d12600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0abf00 0x7f2d0c0ac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d12600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0c6760 con 0x7f2d0c0ca2a0 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d12600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0ac8c0 con 0x7f2d0c0abf00 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d12600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0c8130 con 0x7f2cf0003f00 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d02a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d0c0ca2a0 0x7f2d0c1f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d02a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d0c0ca2a0 0x7f2d0c1f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54292/0 (socket says 172.21.15.66:54292) 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d02a00640 1 -- 172.21.15.66:0/2851450829 learned_addr learned my addr 172.21.15.66:0/2851450829 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d02a00640 1 -- 172.21.15.66:0/2851450829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf0003f00 msgr2=0x7f2d0c0c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d0be00640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0abf00 0x7f2d0c0ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d0b400640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf0003f00 0x7f2d0c0c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d02a00640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf0003f00 0x7f2d0c0c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d02a00640 1 -- 172.21.15.66:0/2851450829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0abf00 msgr2=0x7f2d0c0ac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d02a00640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0abf00 0x7f2d0c0ac380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d02a00640 1 -- 172.21.15.66:0/2851450829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0c6970 con 0x7f2d0c0ca2a0 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d0be00640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0abf00 0x7f2d0c0ac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d0b400640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf0003f00 0x7f2d0c0c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d02a00640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d0c0ca2a0 0x7f2d0c1f16a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c210 tx=0x7f2cf0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d03e00640 1 -- 172.21.15.66:0/2851450829 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf40b7070 con 0x7f2d0c0ca2a0 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d03e00640 1 -- 172.21.15.66:0/2851450829 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf4005a40 con 0x7f2d0c0ca2a0 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d12600640 1 -- 172.21.15.66:0/2851450829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c0c6c50 con 0x7f2d0c0ca2a0 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d03e00640 1 -- 172.21.15.66:0/2851450829 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4005df0 con 0x7f2d0c0ca2a0 2024-03-30T00:46:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.182+0000 7f2d12600640 1 -- 172.21.15.66:0/2851450829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c0c7210 con 0x7f2d0c0ca2a0 2024-03-30T00:46:19.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.186+0000 7f2d12600640 1 -- 172.21.15.66:0/2851450829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c066c30 con 0x7f2d0c0ca2a0 2024-03-30T00:46:19.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.186+0000 7f2d03e00640 1 -- 172.21.15.66:0/2851450829 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2cf40ad020 con 0x7f2d0c0ca2a0 2024-03-30T00:46:19.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.186+0000 7f2d03e00640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2ce80666c0 0x7f2ce8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.186+0000 7f2d03e00640 1 -- 172.21.15.66:0/2851450829 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2cf4121070 con 0x7f2d0c0ca2a0 2024-03-30T00:46:19.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.186+0000 7f2d0b400640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2ce80666c0 0x7f2ce8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.186+0000 7f2d0b400640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2ce80666c0 0x7f2ce8068b80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009a00 tx=0x7f2cfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:19.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.190+0000 7f2d03e00640 1 -- 172.21.15.66:0/2851450829 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2cf4152980 con 0x7f2d0c0ca2a0 2024-03-30T00:46:19.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.350+0000 7f2d12600640 1 -- 172.21.15.66:0/2851450829 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d0c06d860 con 0x7f2ce80666c0 2024-03-30T00:46:19.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.358+0000 7f2d03e00640 1 -- 172.21.15.66:0/2851450829 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289367 (secure 0 0 0) 0x7f2cfc013960 con 0x7f2ce80666c0 2024-03-30T00:46:19.369 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:19.371 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:46:19.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 -- 172.21.15.66:0/2851450829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2ce80666c0 msgr2=0x7f2ce8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2ce80666c0 0x7f2ce8068b80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009a00 tx=0x7f2cfc008040 comp rx=0 tx=0).stop 2024-03-30T00:46:19.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 -- 172.21.15.66:0/2851450829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d0c0ca2a0 msgr2=0x7f2d0c1f16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d0c0ca2a0 0x7f2d0c1f16a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c210 tx=0x7f2cf0005800 comp rx=0 tx=0).stop 2024-03-30T00:46:19.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 -- 172.21.15.66:0/2851450829 shutdown_connections 2024-03-30T00:46:19.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0abf00 0x7f2d0c0ac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2ce80666c0 0x7f2ce8068b80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d0c0ca2a0 0x7f2d0c1f16a0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 --2- 172.21.15.66:0/2851450829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf0003f00 0x7f2d0c0c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 -- 172.21.15.66:0/2851450829 >> 172.21.15.66:0/2851450829 conn(0x7f2d0c069f60 msgr2=0x7f2d0c0ab790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:19.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 -- 172.21.15.66:0/2851450829 shutdown_connections 2024-03-30T00:46:19.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.362+0000 7f2d12600640 1 -- 172.21.15.66:0/2851450829 wait complete. 2024-03-30T00:46:19.389 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-03-30T00:46:18.936556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259320,"kb_used_data":10376,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715400,"statfs":{"total":773094113280,"available":772828569600,"internally_reserved":0,"allocated":10625024,"data_stored":5459860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.071481"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290689+0000","last_change":"2024-03-30T00:46:15.308059+0000","last_active":"2024-03-30T00:46:16.290689+0000","last_peered":"2024-03-30T00:46:16.290689+0000","last_clean":"2024-03-30T00:46:16.290689+0000","last_became_active":"2024-03-30T00:46:15.307439+0000","last_became_peered":"2024-03-30T00:46:15.307439+0000","last_unstale":"2024-03-30T00:46:16.290689+0000","last_undegraded":"2024-03-30T00:46:16.290689+0000","last_fullsized":"2024-03-30T00:46:16.290689+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:00:36.076780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290389+0000","last_change":"2024-03-30T00:46:15.305235+0000","last_active":"2024-03-30T00:46:16.290389+0000","last_peered":"2024-03-30T00:46:16.290389+0000","last_clean":"2024-03-30T00:46:16.290389+0000","last_became_active":"2024-03-30T00:46:15.304112+0000","last_became_peered":"2024-03-30T00:46:15.304112+0000","last_unstale":"2024-03-30T00:46:16.290389+0000","last_undegraded":"2024-03-30T00:46:16.290389+0000","last_fullsized":"2024-03-30T00:46:16.290389+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:23:43.769848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290003+0000","last_change":"2024-03-30T00:46:15.302482+0000","last_active":"2024-03-30T00:46:16.290003+0000","last_peered":"2024-03-30T00:46:16.290003+0000","last_clean":"2024-03-30T00:46:16.290003+0000","last_became_active":"2024-03-30T00:46:15.301827+0000","last_became_peered":"2024-03-30T00:46:15.301827+0000","last_unstale":"2024-03-30T00:46:16.290003+0000","last_undegraded":"2024-03-30T00:46:16.290003+0000","last_fullsized":"2024-03-30T00:46:16.290003+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:54:38.237992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290003+0000","last_change":"2024-03-30T00:46:15.300660+0000","last_active":"2024-03-30T00:46:16.290003+0000","last_peered":"2024-03-30T00:46:16.290003+0000","last_clean":"2024-03-30T00:46:16.290003+0000","last_became_active":"2024-03-30T00:46:15.300096+0000","last_became_peered":"2024-03-30T00:46:15.300096+0000","last_unstale":"2024-03-30T00:46:16.290003+0000","last_undegraded":"2024-03-30T00:46:16.290003+0000","last_fullsized":"2024-03-30T00:46:16.290003+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:11.158261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281726+0000","last_change":"2024-03-30T00:46:15.306764+0000","last_active":"2024-03-30T00:46:16.281726+0000","last_peered":"2024-03-30T00:46:16.281726+0000","last_clean":"2024-03-30T00:46:16.281726+0000","last_became_active":"2024-03-30T00:46:15.302960+0000","last_became_peered":"2024-03-30T00:46:15.302960+0000","last_unstale":"2024-03-30T00:46:16.281726+0000","last_undegraded":"2024-03-30T00:46:16.281726+0000","last_fullsized":"2024-03-30T00:46:16.281726+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:07.099879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289309+0000","last_change":"2024-03-30T00:46:15.295303+0000","last_active":"2024-03-30T00:46:16.289309+0000","last_peered":"2024-03-30T00:46:16.289309+0000","last_clean":"2024-03-30T00:46:16.289309+0000","last_became_active":"2024-03-30T00:46:15.294925+0000","last_became_peered":"2024-03-30T00:46:15.294925+0000","last_unstale":"2024-03-30T00:46:16.289309+0000","last_undegraded":"2024-03-30T00:46:16.289309+0000","last_fullsized":"2024-03-30T00:46:16.289309+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:47:00.660679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281426+0000","last_change":"2024-03-30T00:46:15.298495+0000","last_active":"2024-03-30T00:46:16.281426+0000","last_peered":"2024-03-30T00:46:16.281426+0000","last_clean":"2024-03-30T00:46:16.281426+0000","last_became_active":"2024-03-30T00:46:15.298152+0000","last_became_peered":"2024-03-30T00:46:15.298152+0000","last_unstale":"2024-03-30T00:46:16.281426+0000","last_undegraded":"2024-03-30T00:46:16.281426+0000","last_fullsized":"2024-03-30T00:46:16.281426+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:46:25.620009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288461+0000","last_change":"2024-03-30T00:46:15.291458+0000","last_active":"2024-03-30T00:46:16.288461+0000","last_peered":"2024-03-30T00:46:16.288461+0000","last_clean":"2024-03-30T00:46:16.288461+0000","last_became_active":"2024-03-30T00:46:15.291196+0000","last_became_peered":"2024-03-30T00:46:15.291196+0000","last_unstale":"2024-03-30T00:46:16.288461+0000","last_undegraded":"2024-03-30T00:46:16.288461+0000","last_fullsized":"2024-03-30T00:46:16.288461+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:07.099879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281438+0000","last_change":"2024-03-30T00:46:15.307046+0000","last_active":"2024-03-30T00:46:16.281438+0000","last_peered":"2024-03-30T00:46:16.281438+0000","last_clean":"2024-03-30T00:46:16.281438+0000","last_became_active":"2024-03-30T00:46:15.306778+0000","last_became_peered":"2024-03-30T00:46:15.306778+0000","last_unstale":"2024-03-30T00:46:16.281438+0000","last_undegraded":"2024-03-30T00:46:16.281438+0000","last_fullsized":"2024-03-30T00:46:16.281438+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:43.055191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278739+0000","last_change":"2024-03-30T00:46:15.310550+0000","last_active":"2024-03-30T00:46:16.278739+0000","last_peered":"2024-03-30T00:46:16.278739+0000","last_clean":"2024-03-30T00:46:16.278739+0000","last_became_active":"2024-03-30T00:46:15.310273+0000","last_became_peered":"2024-03-30T00:46:15.310273+0000","last_unstale":"2024-03-30T00:46:16.278739+0000","last_undegraded":"2024-03-30T00:46:16.278739+0000","last_fullsized":"2024-03-30T00:46:16.278739+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:46:16.515611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.282074+0000","last_change":"2024-03-30T00:46:15.302362+0000","last_active":"2024-03-30T00:46:16.282074+0000","last_peered":"2024-03-30T00:46:16.282074+0000","last_clean":"2024-03-30T00:46:16.282074+0000","last_became_active":"2024-03-30T00:46:15.302103+0000","last_became_peered":"2024-03-30T00:46:15.302103+0000","last_unstale":"2024-03-30T00:46:16.282074+0000","last_undegraded":"2024-03-30T00:46:16.282074+0000","last_fullsized":"2024-03-30T00:46:16.282074+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:48.443688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278866+0000","last_change":"2024-03-30T00:46:15.299163+0000","last_active":"2024-03-30T00:46:16.278866+0000","last_peered":"2024-03-30T00:46:16.278866+0000","last_clean":"2024-03-30T00:46:16.278866+0000","last_became_active":"2024-03-30T00:46:15.298676+0000","last_became_peered":"2024-03-30T00:46:15.298676+0000","last_unstale":"2024-03-30T00:46:16.278866+0000","last_undegraded":"2024-03-30T00:46:16.278866+0000","last_fullsized":"2024-03-30T00:46:16.278866+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:06:17.671736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288610+0000","last_change":"2024-03-30T00:46:15.286247+0000","last_active":"2024-03-30T00:46:16.288610+0000","last_peered":"2024-03-30T00:46:16.288610+0000","last_clean":"2024-03-30T00:46:16.288610+0000","last_became_active":"2024-03-30T00:46:15.285983+0000","last_became_peered":"2024-03-30T00:46:15.285983+0000","last_unstale":"2024-03-30T00:46:16.288610+0000","last_undegraded":"2024-03-30T00:46:16.288610+0000","last_fullsized":"2024-03-30T00:46:16.288610+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:24:51.568922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281259+0000","last_change":"2024-03-30T00:46:15.307475+0000","last_active":"2024-03-30T00:46:16.281259+0000","last_peered":"2024-03-30T00:46:16.281259+0000","last_clean":"2024-03-30T00:46:16.281259+0000","last_became_active":"2024-03-30T00:46:15.307270+0000","last_became_peered":"2024-03-30T00:46:15.307270+0000","last_unstale":"2024-03-30T00:46:16.281259+0000","last_undegraded":"2024-03-30T00:46:16.281259+0000","last_fullsized":"2024-03-30T00:46:16.281259+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:24:36.463049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288558+0000","last_change":"2024-03-30T00:46:15.301244+0000","last_active":"2024-03-30T00:46:16.288558+0000","last_peered":"2024-03-30T00:46:16.288558+0000","last_clean":"2024-03-30T00:46:16.288558+0000","last_became_active":"2024-03-30T00:46:15.300770+0000","last_became_peered":"2024-03-30T00:46:15.300770+0000","last_unstale":"2024-03-30T00:46:16.288558+0000","last_undegraded":"2024-03-30T00:46:16.288558+0000","last_fullsized":"2024-03-30T00:46:16.288558+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:48.443688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289229+0000","last_change":"2024-03-30T00:46:15.304348+0000","last_active":"2024-03-30T00:46:16.289229+0000","last_peered":"2024-03-30T00:46:16.289229+0000","last_clean":"2024-03-30T00:46:16.289229+0000","last_became_active":"2024-03-30T00:46:15.303919+0000","last_became_peered":"2024-03-30T00:46:15.303919+0000","last_unstale":"2024-03-30T00:46:16.289229+0000","last_undegraded":"2024-03-30T00:46:16.289229+0000","last_fullsized":"2024-03-30T00:46:16.289229+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:39:45.212715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290587+0000","last_change":"2024-03-30T00:46:15.304020+0000","last_active":"2024-03-30T00:46:16.290587+0000","last_peered":"2024-03-30T00:46:16.290587+0000","last_clean":"2024-03-30T00:46:16.290587+0000","last_became_active":"2024-03-30T00:46:15.303772+0000","last_became_peered":"2024-03-30T00:46:15.303772+0000","last_unstale":"2024-03-30T00:46:16.290587+0000","last_undegraded":"2024-03-30T00:46:16.290587+0000","last_fullsized":"2024-03-30T00:46:16.290587+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:13:03.746083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290431+0000","last_change":"2024-03-30T00:46:15.302560+0000","last_active":"2024-03-30T00:46:16.290431+0000","last_peered":"2024-03-30T00:46:16.290431+0000","last_clean":"2024-03-30T00:46:16.290431+0000","last_became_active":"2024-03-30T00:46:15.302315+0000","last_became_peered":"2024-03-30T00:46:15.302315+0000","last_unstale":"2024-03-30T00:46:16.290431+0000","last_undegraded":"2024-03-30T00:46:16.290431+0000","last_fullsized":"2024-03-30T00:46:16.290431+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:12:12.474874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.282076+0000","last_change":"2024-03-30T00:46:15.309778+0000","last_active":"2024-03-30T00:46:16.282076+0000","last_peered":"2024-03-30T00:46:16.282076+0000","last_clean":"2024-03-30T00:46:16.282076+0000","last_became_active":"2024-03-30T00:46:15.309532+0000","last_became_peered":"2024-03-30T00:46:15.309532+0000","last_unstale":"2024-03-30T00:46:16.282076+0000","last_undegraded":"2024-03-30T00:46:16.282076+0000","last_fullsized":"2024-03-30T00:46:16.282076+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:36:45.859172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290181+0000","last_change":"2024-03-30T00:46:15.296665+0000","last_active":"2024-03-30T00:46:16.290181+0000","last_peered":"2024-03-30T00:46:16.290181+0000","last_clean":"2024-03-30T00:46:16.290181+0000","last_became_active":"2024-03-30T00:46:15.296286+0000","last_became_peered":"2024-03-30T00:46:15.296286+0000","last_unstale":"2024-03-30T00:46:16.290181+0000","last_undegraded":"2024-03-30T00:46:16.290181+0000","last_fullsized":"2024-03-30T00:46:16.290181+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:20:44.405287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281441+0000","last_change":"2024-03-30T00:46:15.296193+0000","last_active":"2024-03-30T00:46:16.281441+0000","last_peered":"2024-03-30T00:46:16.281441+0000","last_clean":"2024-03-30T00:46:16.281441+0000","last_became_active":"2024-03-30T00:46:15.295934+0000","last_became_peered":"2024-03-30T00:46:15.295934+0000","last_unstale":"2024-03-30T00:46:16.281441+0000","last_undegraded":"2024-03-30T00:46:16.281441+0000","last_fullsized":"2024-03-30T00:46:16.281441+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:11.158261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281302+0000","last_change":"2024-03-30T00:46:15.307418+0000","last_active":"2024-03-30T00:46:16.281302+0000","last_peered":"2024-03-30T00:46:16.281302+0000","last_clean":"2024-03-30T00:46:16.281302+0000","last_became_active":"2024-03-30T00:46:15.306973+0000","last_became_peered":"2024-03-30T00:46:15.306973+0000","last_unstale":"2024-03-30T00:46:16.281302+0000","last_undegraded":"2024-03-30T00:46:16.281302+0000","last_fullsized":"2024-03-30T00:46:16.281302+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:22:27.380550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290547+0000","last_change":"2024-03-30T00:46:15.300421+0000","last_active":"2024-03-30T00:46:16.290547+0000","last_peered":"2024-03-30T00:46:16.290547+0000","last_clean":"2024-03-30T00:46:16.290547+0000","last_became_active":"2024-03-30T00:46:15.300027+0000","last_became_peered":"2024-03-30T00:46:15.300027+0000","last_unstale":"2024-03-30T00:46:16.290547+0000","last_undegraded":"2024-03-30T00:46:16.290547+0000","last_fullsized":"2024-03-30T00:46:16.290547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:35.986067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289194+0000","last_change":"2024-03-30T00:46:15.293486+0000","last_active":"2024-03-30T00:46:16.289194+0000","last_peered":"2024-03-30T00:46:16.289194+0000","last_clean":"2024-03-30T00:46:16.289194+0000","last_became_active":"2024-03-30T00:46:15.293239+0000","last_became_peered":"2024-03-30T00:46:15.293239+0000","last_unstale":"2024-03-30T00:46:16.289194+0000","last_undegraded":"2024-03-30T00:46:16.289194+0000","last_fullsized":"2024-03-30T00:46:16.289194+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:56:20.676119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290527+0000","last_change":"2024-03-30T00:46:15.299088+0000","last_active":"2024-03-30T00:46:16.290527+0000","last_peered":"2024-03-30T00:46:16.290527+0000","last_clean":"2024-03-30T00:46:16.290527+0000","last_became_active":"2024-03-30T00:46:15.298746+0000","last_became_peered":"2024-03-30T00:46:15.298746+0000","last_unstale":"2024-03-30T00:46:16.290527+0000","last_undegraded":"2024-03-30T00:46:16.290527+0000","last_fullsized":"2024-03-30T00:46:16.290527+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:12:17.417262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281782+0000","last_change":"2024-03-30T00:46:15.310274+0000","last_active":"2024-03-30T00:46:16.281782+0000","last_peered":"2024-03-30T00:46:16.281782+0000","last_clean":"2024-03-30T00:46:16.281782+0000","last_became_active":"2024-03-30T00:46:15.309768+0000","last_became_peered":"2024-03-30T00:46:15.309768+0000","last_unstale":"2024-03-30T00:46:16.281782+0000","last_undegraded":"2024-03-30T00:46:16.281782+0000","last_fullsized":"2024-03-30T00:46:16.281782+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:01:25.221218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278944+0000","last_change":"2024-03-30T00:46:15.299498+0000","last_active":"2024-03-30T00:46:16.278944+0000","last_peered":"2024-03-30T00:46:16.278944+0000","last_clean":"2024-03-30T00:46:16.278944+0000","last_became_active":"2024-03-30T00:46:15.299192+0000","last_became_peered":"2024-03-30T00:46:15.299192+0000","last_unstale":"2024-03-30T00:46:16.278944+0000","last_undegraded":"2024-03-30T00:46:16.278944+0000","last_fullsized":"2024-03-30T00:46:16.278944+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:23:39.012673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290272+0000","last_change":"2024-03-30T00:46:15.294718+0000","last_active":"2024-03-30T00:46:16.290272+0000","last_peered":"2024-03-30T00:46:16.290272+0000","last_clean":"2024-03-30T00:46:16.290272+0000","last_became_active":"2024-03-30T00:46:15.294466+0000","last_became_peered":"2024-03-30T00:46:15.294466+0000","last_unstale":"2024-03-30T00:46:16.290272+0000","last_undegraded":"2024-03-30T00:46:16.290272+0000","last_fullsized":"2024-03-30T00:46:16.290272+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:08.355340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281869+0000","last_change":"2024-03-30T00:46:15.310622+0000","last_active":"2024-03-30T00:46:16.281869+0000","last_peered":"2024-03-30T00:46:16.281869+0000","last_clean":"2024-03-30T00:46:16.281869+0000","last_became_active":"2024-03-30T00:46:15.308324+0000","last_became_peered":"2024-03-30T00:46:15.308324+0000","last_unstale":"2024-03-30T00:46:16.281869+0000","last_undegraded":"2024-03-30T00:46:16.281869+0000","last_fullsized":"2024-03-30T00:46:16.281869+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:17:56.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281502+0000","last_change":"2024-03-30T00:46:15.307248+0000","last_active":"2024-03-30T00:46:16.281502+0000","last_peered":"2024-03-30T00:46:16.281502+0000","last_clean":"2024-03-30T00:46:16.281502+0000","last_became_active":"2024-03-30T00:46:15.306700+0000","last_became_peered":"2024-03-30T00:46:15.306700+0000","last_unstale":"2024-03-30T00:46:16.281502+0000","last_undegraded":"2024-03-30T00:46:16.281502+0000","last_fullsized":"2024-03-30T00:46:16.281502+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:08.355340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290359+0000","last_change":"2024-03-30T00:46:15.305732+0000","last_active":"2024-03-30T00:46:16.290359+0000","last_peered":"2024-03-30T00:46:16.290359+0000","last_clean":"2024-03-30T00:46:16.290359+0000","last_became_active":"2024-03-30T00:46:15.305395+0000","last_became_peered":"2024-03-30T00:46:15.305395+0000","last_unstale":"2024-03-30T00:46:16.290359+0000","last_undegraded":"2024-03-30T00:46:16.290359+0000","last_fullsized":"2024-03-30T00:46:16.290359+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:46:25.620009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289317+0000","last_change":"2024-03-30T00:46:15.297837+0000","last_active":"2024-03-30T00:46:16.289317+0000","last_peered":"2024-03-30T00:46:16.289317+0000","last_clean":"2024-03-30T00:46:16.289317+0000","last_became_active":"2024-03-30T00:46:15.297554+0000","last_became_peered":"2024-03-30T00:46:15.297554+0000","last_unstale":"2024-03-30T00:46:16.289317+0000","last_undegraded":"2024-03-30T00:46:16.289317+0000","last_fullsized":"2024-03-30T00:46:16.289317+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:14:44.676649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289608+0000","last_change":"2024-03-30T00:46:15.294294+0000","last_active":"2024-03-30T00:46:16.289608+0000","last_peered":"2024-03-30T00:46:16.289608+0000","last_clean":"2024-03-30T00:46:16.289608+0000","last_became_active":"2024-03-30T00:46:15.293879+0000","last_became_peered":"2024-03-30T00:46:15.293879+0000","last_unstale":"2024-03-30T00:46:16.289608+0000","last_undegraded":"2024-03-30T00:46:16.289608+0000","last_fullsized":"2024-03-30T00:46:16.289608+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:54:55.569737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"94'4","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.291123+0000","last_change":"2024-03-30T00:46:14.892711+0000","last_active":"2024-03-30T00:46:16.291123+0000","last_peered":"2024-03-30T00:46:16.291123+0000","last_clean":"2024-03-30T00:46:16.291123+0000","last_became_active":"2024-03-30T00:46:14.892462+0000","last_became_peered":"2024-03-30T00:46:14.892462+0000","last_unstale":"2024-03-30T00:46:16.291123+0000","last_undegraded":"2024-03-30T00:46:16.291123+0000","last_fullsized":"2024-03-30T00:46:16.291123+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:38:33.451309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290004+0000","last_change":"2024-03-30T00:46:15.305329+0000","last_active":"2024-03-30T00:46:16.290004+0000","last_peered":"2024-03-30T00:46:16.290004+0000","last_clean":"2024-03-30T00:46:16.290004+0000","last_became_active":"2024-03-30T00:46:15.305099+0000","last_became_peered":"2024-03-30T00:46:15.305099+0000","last_unstale":"2024-03-30T00:46:16.290004+0000","last_undegraded":"2024-03-30T00:46:16.290004+0000","last_fullsized":"2024-03-30T00:46:16.290004+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:29.566350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281853+0000","last_change":"2024-03-30T00:46:14.893880+0000","last_active":"2024-03-30T00:46:16.281853+0000","last_peered":"2024-03-30T00:46:16.281853+0000","last_clean":"2024-03-30T00:46:16.281853+0000","last_became_active":"2024-03-30T00:46:14.893628+0000","last_became_peered":"2024-03-30T00:46:14.893628+0000","last_unstale":"2024-03-30T00:46:16.281853+0000","last_undegraded":"2024-03-30T00:46:16.281853+0000","last_fullsized":"2024-03-30T00:46:16.281853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:39.993735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281801+0000","last_change":"2024-03-30T00:46:15.295961+0000","last_active":"2024-03-30T00:46:16.281801+0000","last_peered":"2024-03-30T00:46:16.281801+0000","last_clean":"2024-03-30T00:46:16.281801+0000","last_became_active":"2024-03-30T00:46:15.295659+0000","last_became_peered":"2024-03-30T00:46:15.295659+0000","last_unstale":"2024-03-30T00:46:16.281801+0000","last_undegraded":"2024-03-30T00:46:16.281801+0000","last_fullsized":"2024-03-30T00:46:16.281801+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:13.184530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281679+0000","last_change":"2024-03-30T00:46:14.888612+0000","last_active":"2024-03-30T00:46:16.281679+0000","last_peered":"2024-03-30T00:46:16.281679+0000","last_clean":"2024-03-30T00:46:16.281679+0000","last_became_active":"2024-03-30T00:46:14.888341+0000","last_became_peered":"2024-03-30T00:46:14.888341+0000","last_unstale":"2024-03-30T00:46:16.281679+0000","last_undegraded":"2024-03-30T00:46:16.281679+0000","last_fullsized":"2024-03-30T00:46:16.281679+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:48:36.319396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290712+0000","last_change":"2024-03-30T00:46:15.292459+0000","last_active":"2024-03-30T00:46:16.290712+0000","last_peered":"2024-03-30T00:46:16.290712+0000","last_clean":"2024-03-30T00:46:16.290712+0000","last_became_active":"2024-03-30T00:46:15.290565+0000","last_became_peered":"2024-03-30T00:46:15.290565+0000","last_unstale":"2024-03-30T00:46:16.290712+0000","last_undegraded":"2024-03-30T00:46:16.290712+0000","last_fullsized":"2024-03-30T00:46:16.290712+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:02:51.248469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290931+0000","last_change":"2024-03-30T00:46:14.889403+0000","last_active":"2024-03-30T00:46:16.290931+0000","last_peered":"2024-03-30T00:46:16.290931+0000","last_clean":"2024-03-30T00:46:16.290931+0000","last_became_active":"2024-03-30T00:46:14.888289+0000","last_became_peered":"2024-03-30T00:46:14.888289+0000","last_unstale":"2024-03-30T00:46:16.290931+0000","last_undegraded":"2024-03-30T00:46:16.290931+0000","last_fullsized":"2024-03-30T00:46:16.290931+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:01:16.519724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288463+0000","last_change":"2024-03-30T00:46:15.300282+0000","last_active":"2024-03-30T00:46:16.288463+0000","last_peered":"2024-03-30T00:46:16.288463+0000","last_clean":"2024-03-30T00:46:16.288463+0000","last_became_active":"2024-03-30T00:46:15.299988+0000","last_became_peered":"2024-03-30T00:46:15.299988+0000","last_unstale":"2024-03-30T00:46:16.288463+0000","last_undegraded":"2024-03-30T00:46:16.288463+0000","last_fullsized":"2024-03-30T00:46:16.288463+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:43.055191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281647+0000","last_change":"2024-03-30T00:46:14.892764+0000","last_active":"2024-03-30T00:46:16.281647+0000","last_peered":"2024-03-30T00:46:16.281647+0000","last_clean":"2024-03-30T00:46:16.281647+0000","last_became_active":"2024-03-30T00:46:14.892520+0000","last_became_peered":"2024-03-30T00:46:14.892520+0000","last_unstale":"2024-03-30T00:46:16.281647+0000","last_undegraded":"2024-03-30T00:46:16.281647+0000","last_fullsized":"2024-03-30T00:46:16.281647+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:16:38.775300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290625+0000","last_change":"2024-03-30T00:46:15.289253+0000","last_active":"2024-03-30T00:46:16.290625+0000","last_peered":"2024-03-30T00:46:16.290625+0000","last_clean":"2024-03-30T00:46:16.290625+0000","last_became_active":"2024-03-30T00:46:15.288992+0000","last_became_peered":"2024-03-30T00:46:15.288992+0000","last_unstale":"2024-03-30T00:46:16.290625+0000","last_undegraded":"2024-03-30T00:46:16.290625+0000","last_fullsized":"2024-03-30T00:46:16.290625+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:16:39.777547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288984+0000","last_change":"2024-03-30T00:46:14.887491+0000","last_active":"2024-03-30T00:46:16.288984+0000","last_peered":"2024-03-30T00:46:16.288984+0000","last_clean":"2024-03-30T00:46:16.288984+0000","last_became_active":"2024-03-30T00:46:14.887178+0000","last_became_peered":"2024-03-30T00:46:14.887178+0000","last_unstale":"2024-03-30T00:46:16.288984+0000","last_undegraded":"2024-03-30T00:46:16.288984+0000","last_fullsized":"2024-03-30T00:46:16.288984+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:30.462631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289244+0000","last_change":"2024-03-30T00:46:15.295155+0000","last_active":"2024-03-30T00:46:16.289244+0000","last_peered":"2024-03-30T00:46:16.289244+0000","last_clean":"2024-03-30T00:46:16.289244+0000","last_became_active":"2024-03-30T00:46:15.294564+0000","last_became_peered":"2024-03-30T00:46:15.294564+0000","last_unstale":"2024-03-30T00:46:16.289244+0000","last_undegraded":"2024-03-30T00:46:16.289244+0000","last_fullsized":"2024-03-30T00:46:16.289244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:32:21.356152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289528+0000","last_change":"2024-03-30T00:46:14.889581+0000","last_active":"2024-03-30T00:46:16.289528+0000","last_peered":"2024-03-30T00:46:16.289528+0000","last_clean":"2024-03-30T00:46:16.289528+0000","last_became_active":"2024-03-30T00:46:14.889336+0000","last_became_peered":"2024-03-30T00:46:14.889336+0000","last_unstale":"2024-03-30T00:46:16.289528+0000","last_undegraded":"2024-03-30T00:46:16.289528+0000","last_fullsized":"2024-03-30T00:46:16.289528+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:20:43.403040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289714+0000","last_change":"2024-03-30T00:46:15.300746+0000","last_active":"2024-03-30T00:46:16.289714+0000","last_peered":"2024-03-30T00:46:16.289714+0000","last_clean":"2024-03-30T00:46:16.289714+0000","last_became_active":"2024-03-30T00:46:15.300051+0000","last_became_peered":"2024-03-30T00:46:15.300051+0000","last_unstale":"2024-03-30T00:46:16.289714+0000","last_undegraded":"2024-03-30T00:46:16.289714+0000","last_fullsized":"2024-03-30T00:46:16.289714+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:23:43.769848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281541+0000","last_change":"2024-03-30T00:46:14.891615+0000","last_active":"2024-03-30T00:46:16.281541+0000","last_peered":"2024-03-30T00:46:16.281541+0000","last_clean":"2024-03-30T00:46:16.281541+0000","last_became_active":"2024-03-30T00:46:14.891310+0000","last_became_peered":"2024-03-30T00:46:14.891310+0000","last_unstale":"2024-03-30T00:46:16.281541+0000","last_undegraded":"2024-03-30T00:46:16.281541+0000","last_fullsized":"2024-03-30T00:46:16.281541+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:39.993735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281302+0000","last_change":"2024-03-30T00:46:15.305106+0000","last_active":"2024-03-30T00:46:16.281302+0000","last_peered":"2024-03-30T00:46:16.281302+0000","last_clean":"2024-03-30T00:46:16.281302+0000","last_became_active":"2024-03-30T00:46:15.304843+0000","last_became_peered":"2024-03-30T00:46:15.304843+0000","last_unstale":"2024-03-30T00:46:16.281302+0000","last_undegraded":"2024-03-30T00:46:16.281302+0000","last_fullsized":"2024-03-30T00:46:16.281302+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:20:44.405287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290690+0000","last_change":"2024-03-30T00:46:14.885281+0000","last_active":"2024-03-30T00:46:16.290690+0000","last_peered":"2024-03-30T00:46:16.290690+0000","last_clean":"2024-03-30T00:46:16.290690+0000","last_became_active":"2024-03-30T00:46:14.885024+0000","last_became_peered":"2024-03-30T00:46:14.885024+0000","last_unstale":"2024-03-30T00:46:16.290690+0000","last_undegraded":"2024-03-30T00:46:16.290690+0000","last_fullsized":"2024-03-30T00:46:16.290690+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:19:51.018234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281168+0000","last_change":"2024-03-30T00:46:15.309120+0000","last_active":"2024-03-30T00:46:16.281168+0000","last_peered":"2024-03-30T00:46:16.281168+0000","last_clean":"2024-03-30T00:46:16.281168+0000","last_became_active":"2024-03-30T00:46:15.308613+0000","last_became_peered":"2024-03-30T00:46:15.308613+0000","last_unstale":"2024-03-30T00:46:16.281168+0000","last_undegraded":"2024-03-30T00:46:16.281168+0000","last_fullsized":"2024-03-30T00:46:16.281168+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:27:04.442085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290411+0000","last_change":"2024-03-30T00:46:14.890990+0000","last_active":"2024-03-30T00:46:16.290411+0000","last_peered":"2024-03-30T00:46:16.290411+0000","last_clean":"2024-03-30T00:46:16.290411+0000","last_became_active":"2024-03-30T00:46:14.890723+0000","last_became_peered":"2024-03-30T00:46:14.890723+0000","last_unstale":"2024-03-30T00:46:16.290411+0000","last_undegraded":"2024-03-30T00:46:16.290411+0000","last_fullsized":"2024-03-30T00:46:16.290411+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:47.441441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290334+0000","last_change":"2024-03-30T00:46:15.295516+0000","last_active":"2024-03-30T00:46:16.290334+0000","last_peered":"2024-03-30T00:46:16.290334+0000","last_clean":"2024-03-30T00:46:16.290334+0000","last_became_active":"2024-03-30T00:46:15.295186+0000","last_became_peered":"2024-03-30T00:46:15.295186+0000","last_unstale":"2024-03-30T00:46:16.290334+0000","last_undegraded":"2024-03-30T00:46:16.290334+0000","last_fullsized":"2024-03-30T00:46:16.290334+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:49:08.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.282157+0000","last_change":"2024-03-30T00:46:14.897986+0000","last_active":"2024-03-30T00:46:16.282157+0000","last_peered":"2024-03-30T00:46:16.282157+0000","last_clean":"2024-03-30T00:46:16.282157+0000","last_became_active":"2024-03-30T00:46:14.897688+0000","last_became_peered":"2024-03-30T00:46:14.897688+0000","last_unstale":"2024-03-30T00:46:16.282157+0000","last_undegraded":"2024-03-30T00:46:16.282157+0000","last_fullsized":"2024-03-30T00:46:16.282157+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:19:51.018234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290562+0000","last_change":"2024-03-30T00:46:15.304158+0000","last_active":"2024-03-30T00:46:16.290562+0000","last_peered":"2024-03-30T00:46:16.290562+0000","last_clean":"2024-03-30T00:46:16.290562+0000","last_became_active":"2024-03-30T00:46:15.303898+0000","last_became_peered":"2024-03-30T00:46:15.303898+0000","last_unstale":"2024-03-30T00:46:16.290562+0000","last_undegraded":"2024-03-30T00:46:16.290562+0000","last_fullsized":"2024-03-30T00:46:16.290562+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:12:17.417262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289806+0000","last_change":"2024-03-30T00:46:14.887195+0000","last_active":"2024-03-30T00:46:16.289806+0000","last_peered":"2024-03-30T00:46:16.289806+0000","last_clean":"2024-03-30T00:46:16.289806+0000","last_became_active":"2024-03-30T00:46:14.886205+0000","last_became_peered":"2024-03-30T00:46:14.886205+0000","last_unstale":"2024-03-30T00:46:16.289806+0000","last_undegraded":"2024-03-30T00:46:16.289806+0000","last_fullsized":"2024-03-30T00:46:16.289806+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:05:45.340862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288579+0000","last_change":"2024-03-30T00:46:15.292179+0000","last_active":"2024-03-30T00:46:16.288579+0000","last_peered":"2024-03-30T00:46:16.288579+0000","last_clean":"2024-03-30T00:46:16.288579+0000","last_became_active":"2024-03-30T00:46:15.291360+0000","last_became_peered":"2024-03-30T00:46:15.291360+0000","last_unstale":"2024-03-30T00:46:16.288579+0000","last_undegraded":"2024-03-30T00:46:16.288579+0000","last_fullsized":"2024-03-30T00:46:16.288579+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:27:04.442085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281951+0000","last_change":"2024-03-30T00:46:14.891171+0000","last_active":"2024-03-30T00:46:16.281951+0000","last_peered":"2024-03-30T00:46:16.281951+0000","last_clean":"2024-03-30T00:46:16.281951+0000","last_became_active":"2024-03-30T00:46:14.890866+0000","last_became_peered":"2024-03-30T00:46:14.890866+0000","last_unstale":"2024-03-30T00:46:16.281951+0000","last_undegraded":"2024-03-30T00:46:16.281951+0000","last_fullsized":"2024-03-30T00:46:16.281951+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:19:51.018234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.280954+0000","last_change":"2024-03-30T00:46:15.303091+0000","last_active":"2024-03-30T00:46:16.280954+0000","last_peered":"2024-03-30T00:46:16.280954+0000","last_clean":"2024-03-30T00:46:16.280954+0000","last_became_active":"2024-03-30T00:46:15.302755+0000","last_became_peered":"2024-03-30T00:46:15.302755+0000","last_unstale":"2024-03-30T00:46:16.280954+0000","last_undegraded":"2024-03-30T00:46:16.280954+0000","last_fullsized":"2024-03-30T00:46:16.280954+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:17:56.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289346+0000","last_change":"2024-03-30T00:46:14.893558+0000","last_active":"2024-03-30T00:46:16.289346+0000","last_peered":"2024-03-30T00:46:16.289346+0000","last_clean":"2024-03-30T00:46:16.289346+0000","last_became_active":"2024-03-30T00:46:14.893205+0000","last_became_peered":"2024-03-30T00:46:14.893205+0000","last_unstale":"2024-03-30T00:46:16.289346+0000","last_undegraded":"2024-03-30T00:46:16.289346+0000","last_fullsized":"2024-03-30T00:46:16.289346+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:26:57.918249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278635+0000","last_change":"2024-03-30T00:46:15.304449+0000","last_active":"2024-03-30T00:46:16.278635+0000","last_peered":"2024-03-30T00:46:16.278635+0000","last_clean":"2024-03-30T00:46:16.278635+0000","last_became_active":"2024-03-30T00:46:15.304141+0000","last_became_peered":"2024-03-30T00:46:15.304141+0000","last_unstale":"2024-03-30T00:46:16.278635+0000","last_undegraded":"2024-03-30T00:46:16.278635+0000","last_fullsized":"2024-03-30T00:46:16.278635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:52.772358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288634+0000","last_change":"2024-03-30T00:46:14.884431+0000","last_active":"2024-03-30T00:46:16.288634+0000","last_peered":"2024-03-30T00:46:16.288634+0000","last_clean":"2024-03-30T00:46:16.288634+0000","last_became_active":"2024-03-30T00:46:14.882004+0000","last_became_peered":"2024-03-30T00:46:14.882004+0000","last_unstale":"2024-03-30T00:46:16.288634+0000","last_undegraded":"2024-03-30T00:46:16.288634+0000","last_fullsized":"2024-03-30T00:46:16.288634+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:39.993735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278895+0000","last_change":"2024-03-30T00:46:15.310128+0000","last_active":"2024-03-30T00:46:16.278895+0000","last_peered":"2024-03-30T00:46:16.278895+0000","last_clean":"2024-03-30T00:46:16.278895+0000","last_became_active":"2024-03-30T00:46:15.309871+0000","last_became_peered":"2024-03-30T00:46:15.309871+0000","last_unstale":"2024-03-30T00:46:16.278895+0000","last_undegraded":"2024-03-30T00:46:16.278895+0000","last_fullsized":"2024-03-30T00:46:16.278895+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:00.840705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":19,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288699+0000","last_change":"2024-03-30T00:46:14.882528+0000","last_active":"2024-03-30T00:46:16.288699+0000","last_peered":"2024-03-30T00:46:16.288699+0000","last_clean":"2024-03-30T00:46:16.288699+0000","last_became_active":"2024-03-30T00:46:14.882285+0000","last_became_peered":"2024-03-30T00:46:14.882285+0000","last_unstale":"2024-03-30T00:46:16.288699+0000","last_undegraded":"2024-03-30T00:46:16.288699+0000","last_fullsized":"2024-03-30T00:46:16.288699+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:34.983820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289698+0000","last_change":"2024-03-30T00:46:15.294166+0000","last_active":"2024-03-30T00:46:16.289698+0000","last_peered":"2024-03-30T00:46:16.289698+0000","last_clean":"2024-03-30T00:46:16.289698+0000","last_became_active":"2024-03-30T00:46:15.293471+0000","last_became_peered":"2024-03-30T00:46:15.293471+0000","last_unstale":"2024-03-30T00:46:16.289698+0000","last_undegraded":"2024-03-30T00:46:16.289698+0000","last_fullsized":"2024-03-30T00:46:16.289698+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:12.847342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.291180+0000","last_change":"2024-03-30T00:46:14.888408+0000","last_active":"2024-03-30T00:46:16.291180+0000","last_peered":"2024-03-30T00:46:16.291180+0000","last_clean":"2024-03-30T00:46:16.291180+0000","last_became_active":"2024-03-30T00:46:14.888146+0000","last_became_peered":"2024-03-30T00:46:14.888146+0000","last_unstale":"2024-03-30T00:46:16.291180+0000","last_undegraded":"2024-03-30T00:46:16.291180+0000","last_fullsized":"2024-03-30T00:46:16.291180+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:57:40.214555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.291061+0000","last_change":"2024-03-30T00:46:15.297608+0000","last_active":"2024-03-30T00:46:16.291061+0000","last_peered":"2024-03-30T00:46:16.291061+0000","last_clean":"2024-03-30T00:46:16.291061+0000","last_became_active":"2024-03-30T00:46:15.297314+0000","last_became_peered":"2024-03-30T00:46:15.297314+0000","last_unstale":"2024-03-30T00:46:16.291061+0000","last_undegraded":"2024-03-30T00:46:16.291061+0000","last_fullsized":"2024-03-30T00:46:16.291061+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:31.464878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281899+0000","last_change":"2024-03-30T00:46:14.883401+0000","last_active":"2024-03-30T00:46:16.281899+0000","last_peered":"2024-03-30T00:46:16.281899+0000","last_clean":"2024-03-30T00:46:16.281899+0000","last_became_active":"2024-03-30T00:46:14.883151+0000","last_became_peered":"2024-03-30T00:46:14.883151+0000","last_unstale":"2024-03-30T00:46:16.281899+0000","last_undegraded":"2024-03-30T00:46:16.281899+0000","last_fullsized":"2024-03-30T00:46:16.281899+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:34.983820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281770+0000","last_change":"2024-03-30T00:46:15.310099+0000","last_active":"2024-03-30T00:46:16.281770+0000","last_peered":"2024-03-30T00:46:16.281770+0000","last_clean":"2024-03-30T00:46:16.281770+0000","last_became_active":"2024-03-30T00:46:15.309420+0000","last_became_peered":"2024-03-30T00:46:15.309420+0000","last_unstale":"2024-03-30T00:46:16.281770+0000","last_undegraded":"2024-03-30T00:46:16.281770+0000","last_fullsized":"2024-03-30T00:46:16.281770+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:12:17.417262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.282218+0000","last_change":"2024-03-30T00:46:14.895153+0000","last_active":"2024-03-30T00:46:16.282218+0000","last_peered":"2024-03-30T00:46:16.282218+0000","last_clean":"2024-03-30T00:46:16.282218+0000","last_became_active":"2024-03-30T00:46:14.894650+0000","last_became_peered":"2024-03-30T00:46:14.894650+0000","last_unstale":"2024-03-30T00:46:16.282218+0000","last_undegraded":"2024-03-30T00:46:16.282218+0000","last_fullsized":"2024-03-30T00:46:16.282218+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:34.983820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289404+0000","last_change":"2024-03-30T00:46:15.295130+0000","last_active":"2024-03-30T00:46:16.289404+0000","last_peered":"2024-03-30T00:46:16.289404+0000","last_clean":"2024-03-30T00:46:16.289404+0000","last_became_active":"2024-03-30T00:46:15.294760+0000","last_became_peered":"2024-03-30T00:46:15.294760+0000","last_unstale":"2024-03-30T00:46:16.289404+0000","last_undegraded":"2024-03-30T00:46:16.289404+0000","last_fullsized":"2024-03-30T00:46:16.289404+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:49:14.217720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290180+0000","last_change":"2024-03-30T00:46:14.894486+0000","last_active":"2024-03-30T00:46:16.290180+0000","last_peered":"2024-03-30T00:46:16.290180+0000","last_clean":"2024-03-30T00:46:16.290180+0000","last_became_active":"2024-03-30T00:46:14.894164+0000","last_became_peered":"2024-03-30T00:46:14.894164+0000","last_unstale":"2024-03-30T00:46:16.290180+0000","last_undegraded":"2024-03-30T00:46:16.290180+0000","last_fullsized":"2024-03-30T00:46:16.290180+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:39.993735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290274+0000","last_change":"2024-03-30T00:46:15.294059+0000","last_active":"2024-03-30T00:46:16.290274+0000","last_peered":"2024-03-30T00:46:16.290274+0000","last_clean":"2024-03-30T00:46:16.290274+0000","last_became_active":"2024-03-30T00:46:15.293719+0000","last_became_peered":"2024-03-30T00:46:15.293719+0000","last_unstale":"2024-03-30T00:46:16.290274+0000","last_undegraded":"2024-03-30T00:46:16.290274+0000","last_fullsized":"2024-03-30T00:46:16.290274+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:24:51.568922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278797+0000","last_change":"2024-03-30T00:46:14.884891+0000","last_active":"2024-03-30T00:46:16.278797+0000","last_peered":"2024-03-30T00:46:16.278797+0000","last_clean":"2024-03-30T00:46:16.278797+0000","last_became_active":"2024-03-30T00:46:14.884373+0000","last_became_peered":"2024-03-30T00:46:14.884373+0000","last_unstale":"2024-03-30T00:46:16.278797+0000","last_undegraded":"2024-03-30T00:46:16.278797+0000","last_fullsized":"2024-03-30T00:46:16.278797+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:16.504340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290248+0000","last_change":"2024-03-30T00:46:15.305896+0000","last_active":"2024-03-30T00:46:16.290248+0000","last_peered":"2024-03-30T00:46:16.290248+0000","last_clean":"2024-03-30T00:46:16.290248+0000","last_became_active":"2024-03-30T00:46:15.305611+0000","last_became_peered":"2024-03-30T00:46:15.305611+0000","last_unstale":"2024-03-30T00:46:16.290248+0000","last_undegraded":"2024-03-30T00:46:16.290248+0000","last_fullsized":"2024-03-30T00:46:16.290248+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:13.184530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290375+0000","last_change":"2024-03-30T00:46:14.888244+0000","last_active":"2024-03-30T00:46:16.290375+0000","last_peered":"2024-03-30T00:46:16.290375+0000","last_clean":"2024-03-30T00:46:16.290375+0000","last_became_active":"2024-03-30T00:46:14.887973+0000","last_became_peered":"2024-03-30T00:46:14.887973+0000","last_unstale":"2024-03-30T00:46:16.290375+0000","last_undegraded":"2024-03-30T00:46:16.290375+0000","last_fullsized":"2024-03-30T00:46:16.290375+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:40:16.023181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289444+0000","last_change":"2024-03-30T00:46:15.304651+0000","last_active":"2024-03-30T00:46:16.289444+0000","last_peered":"2024-03-30T00:46:16.289444+0000","last_clean":"2024-03-30T00:46:16.289444+0000","last_became_active":"2024-03-30T00:46:15.304395+0000","last_became_peered":"2024-03-30T00:46:15.304395+0000","last_unstale":"2024-03-30T00:46:16.289444+0000","last_undegraded":"2024-03-30T00:46:16.289444+0000","last_fullsized":"2024-03-30T00:46:16.289444+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:17.650237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278627+0000","last_change":"2024-03-30T00:46:14.879896+0000","last_active":"2024-03-30T00:46:16.278627+0000","last_peered":"2024-03-30T00:46:16.278627+0000","last_clean":"2024-03-30T00:46:16.278627+0000","last_became_active":"2024-03-30T00:46:14.879637+0000","last_became_peered":"2024-03-30T00:46:14.879637+0000","last_unstale":"2024-03-30T00:46:16.278627+0000","last_undegraded":"2024-03-30T00:46:16.278627+0000","last_fullsized":"2024-03-30T00:46:16.278627+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:16:58.396814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290966+0000","last_change":"2024-03-30T00:46:15.291716+0000","last_active":"2024-03-30T00:46:16.290966+0000","last_peered":"2024-03-30T00:46:16.290966+0000","last_clean":"2024-03-30T00:46:16.290966+0000","last_became_active":"2024-03-30T00:46:15.291407+0000","last_became_peered":"2024-03-30T00:46:15.291407+0000","last_unstale":"2024-03-30T00:46:16.290966+0000","last_undegraded":"2024-03-30T00:46:16.290966+0000","last_fullsized":"2024-03-30T00:46:16.290966+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:25:42.971905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281554+0000","last_change":"2024-03-30T00:46:14.887651+0000","last_active":"2024-03-30T00:46:16.281554+0000","last_peered":"2024-03-30T00:46:16.281554+0000","last_clean":"2024-03-30T00:46:16.281554+0000","last_became_active":"2024-03-30T00:46:14.887323+0000","last_became_peered":"2024-03-30T00:46:14.887323+0000","last_unstale":"2024-03-30T00:46:16.281554+0000","last_undegraded":"2024-03-30T00:46:16.281554+0000","last_fullsized":"2024-03-30T00:46:16.281554+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:25.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290126+0000","last_change":"2024-03-30T00:46:15.304851+0000","last_active":"2024-03-30T00:46:16.290126+0000","last_peered":"2024-03-30T00:46:16.290126+0000","last_clean":"2024-03-30T00:46:16.290126+0000","last_became_active":"2024-03-30T00:46:15.301757+0000","last_became_peered":"2024-03-30T00:46:15.301757+0000","last_unstale":"2024-03-30T00:46:16.290126+0000","last_undegraded":"2024-03-30T00:46:16.290126+0000","last_fullsized":"2024-03-30T00:46:16.290126+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:19:41.175787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281883+0000","last_change":"2024-03-30T00:46:14.895879+0000","last_active":"2024-03-30T00:46:16.281883+0000","last_peered":"2024-03-30T00:46:16.281883+0000","last_clean":"2024-03-30T00:46:16.281883+0000","last_became_active":"2024-03-30T00:46:14.895595+0000","last_became_peered":"2024-03-30T00:46:14.895595+0000","last_unstale":"2024-03-30T00:46:16.281883+0000","last_undegraded":"2024-03-30T00:46:16.281883+0000","last_fullsized":"2024-03-30T00:46:16.281883+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:48:36.319396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290251+0000","last_change":"2024-03-30T00:46:15.304589+0000","last_active":"2024-03-30T00:46:16.290251+0000","last_peered":"2024-03-30T00:46:16.290251+0000","last_clean":"2024-03-30T00:46:16.290251+0000","last_became_active":"2024-03-30T00:46:15.304299+0000","last_became_peered":"2024-03-30T00:46:15.304299+0000","last_unstale":"2024-03-30T00:46:16.290251+0000","last_undegraded":"2024-03-30T00:46:16.290251+0000","last_fullsized":"2024-03-30T00:46:16.290251+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:36:51.229729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.282211+0000","last_change":"2024-03-30T00:46:14.897571+0000","last_active":"2024-03-30T00:46:16.282211+0000","last_peered":"2024-03-30T00:46:16.282211+0000","last_clean":"2024-03-30T00:46:16.282211+0000","last_became_active":"2024-03-30T00:46:14.897375+0000","last_became_peered":"2024-03-30T00:46:14.897375+0000","last_unstale":"2024-03-30T00:46:16.282211+0000","last_undegraded":"2024-03-30T00:46:16.282211+0000","last_fullsized":"2024-03-30T00:46:16.282211+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:30.462631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290615+0000","last_change":"2024-03-30T00:46:15.305570+0000","last_active":"2024-03-30T00:46:16.290615+0000","last_peered":"2024-03-30T00:46:16.290615+0000","last_clean":"2024-03-30T00:46:16.290615+0000","last_became_active":"2024-03-30T00:46:15.305249+0000","last_became_peered":"2024-03-30T00:46:15.305249+0000","last_unstale":"2024-03-30T00:46:16.290615+0000","last_undegraded":"2024-03-30T00:46:16.290615+0000","last_fullsized":"2024-03-30T00:46:16.290615+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:27:04.442085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.291025+0000","last_change":"2024-03-30T00:46:14.892456+0000","last_active":"2024-03-30T00:46:16.291025+0000","last_peered":"2024-03-30T00:46:16.291025+0000","last_clean":"2024-03-30T00:46:16.291025+0000","last_became_active":"2024-03-30T00:46:14.892131+0000","last_became_peered":"2024-03-30T00:46:14.892131+0000","last_unstale":"2024-03-30T00:46:16.291025+0000","last_undegraded":"2024-03-30T00:46:16.291025+0000","last_fullsized":"2024-03-30T00:46:16.291025+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:05.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278565+0000","last_change":"2024-03-30T00:46:15.296122+0000","last_active":"2024-03-30T00:46:16.278565+0000","last_peered":"2024-03-30T00:46:16.278565+0000","last_clean":"2024-03-30T00:46:16.278565+0000","last_became_active":"2024-03-30T00:46:15.295861+0000","last_became_peered":"2024-03-30T00:46:15.295861+0000","last_unstale":"2024-03-30T00:46:16.278565+0000","last_undegraded":"2024-03-30T00:46:16.278565+0000","last_fullsized":"2024-03-30T00:46:16.278565+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:56:32.560264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289592+0000","last_change":"2024-03-30T00:46:14.894642+0000","last_active":"2024-03-30T00:46:16.289592+0000","last_peered":"2024-03-30T00:46:16.289592+0000","last_clean":"2024-03-30T00:46:16.289592+0000","last_became_active":"2024-03-30T00:46:14.894391+0000","last_became_peered":"2024-03-30T00:46:14.894391+0000","last_unstale":"2024-03-30T00:46:16.289592+0000","last_undegraded":"2024-03-30T00:46:16.289592+0000","last_fullsized":"2024-03-30T00:46:16.289592+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:51.291654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289385+0000","last_change":"2024-03-30T00:46:15.306548+0000","last_active":"2024-03-30T00:46:16.289385+0000","last_peered":"2024-03-30T00:46:16.289385+0000","last_clean":"2024-03-30T00:46:16.289385+0000","last_became_active":"2024-03-30T00:46:15.306113+0000","last_became_peered":"2024-03-30T00:46:15.306113+0000","last_unstale":"2024-03-30T00:46:16.289385+0000","last_undegraded":"2024-03-30T00:46:16.289385+0000","last_fullsized":"2024-03-30T00:46:16.289385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:05:46.343109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290357+0000","last_change":"2024-03-30T00:46:14.893727+0000","last_active":"2024-03-30T00:46:16.290357+0000","last_peered":"2024-03-30T00:46:16.290357+0000","last_clean":"2024-03-30T00:46:16.290357+0000","last_became_active":"2024-03-30T00:46:14.893417+0000","last_became_peered":"2024-03-30T00:46:14.893417+0000","last_unstale":"2024-03-30T00:46:16.290357+0000","last_undegraded":"2024-03-30T00:46:16.290357+0000","last_fullsized":"2024-03-30T00:46:16.290357+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:25.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290683+0000","last_change":"2024-03-30T00:46:15.302710+0000","last_active":"2024-03-30T00:46:16.290683+0000","last_peered":"2024-03-30T00:46:16.290683+0000","last_clean":"2024-03-30T00:46:16.290683+0000","last_became_active":"2024-03-30T00:46:15.302449+0000","last_became_peered":"2024-03-30T00:46:15.302449+0000","last_unstale":"2024-03-30T00:46:16.290683+0000","last_undegraded":"2024-03-30T00:46:16.290683+0000","last_fullsized":"2024-03-30T00:46:16.290683+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:48:37.321643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290695+0000","last_change":"2024-03-30T00:46:14.890025+0000","last_active":"2024-03-30T00:46:16.290695+0000","last_peered":"2024-03-30T00:46:16.290695+0000","last_clean":"2024-03-30T00:46:16.290695+0000","last_became_active":"2024-03-30T00:46:14.889762+0000","last_became_peered":"2024-03-30T00:46:14.889762+0000","last_unstale":"2024-03-30T00:46:16.290695+0000","last_undegraded":"2024-03-30T00:46:16.290695+0000","last_fullsized":"2024-03-30T00:46:16.290695+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:39.810863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289583+0000","last_change":"2024-03-30T00:46:15.296766+0000","last_active":"2024-03-30T00:46:16.289583+0000","last_peered":"2024-03-30T00:46:16.289583+0000","last_clean":"2024-03-30T00:46:16.289583+0000","last_became_active":"2024-03-30T00:46:15.296497+0000","last_became_peered":"2024-03-30T00:46:15.296497+0000","last_unstale":"2024-03-30T00:46:16.289583+0000","last_undegraded":"2024-03-30T00:46:16.289583+0000","last_fullsized":"2024-03-30T00:46:16.289583+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:57.757325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281588+0000","last_change":"2024-03-30T00:46:14.886828+0000","last_active":"2024-03-30T00:46:16.281588+0000","last_peered":"2024-03-30T00:46:16.281588+0000","last_clean":"2024-03-30T00:46:16.281588+0000","last_became_active":"2024-03-30T00:46:14.886576+0000","last_became_peered":"2024-03-30T00:46:14.886576+0000","last_unstale":"2024-03-30T00:46:16.281588+0000","last_undegraded":"2024-03-30T00:46:16.281588+0000","last_fullsized":"2024-03-30T00:46:16.281588+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:30.462631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281448+0000","last_change":"2024-03-30T00:46:15.309687+0000","last_active":"2024-03-30T00:46:16.281448+0000","last_peered":"2024-03-30T00:46:16.281448+0000","last_clean":"2024-03-30T00:46:16.281448+0000","last_became_active":"2024-03-30T00:46:15.309436+0000","last_became_peered":"2024-03-30T00:46:15.309436+0000","last_unstale":"2024-03-30T00:46:16.281448+0000","last_undegraded":"2024-03-30T00:46:16.281448+0000","last_fullsized":"2024-03-30T00:46:16.281448+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:12:17.417262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290347+0000","last_change":"2024-03-30T00:46:14.876213+0000","last_active":"2024-03-30T00:46:16.290347+0000","last_peered":"2024-03-30T00:46:16.290347+0000","last_clean":"2024-03-30T00:46:16.290347+0000","last_became_active":"2024-03-30T00:46:14.875930+0000","last_became_peered":"2024-03-30T00:46:14.875930+0000","last_unstale":"2024-03-30T00:46:16.290347+0000","last_undegraded":"2024-03-30T00:46:16.290347+0000","last_fullsized":"2024-03-30T00:46:16.290347+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:19:51.018234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":241,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-30T00:46:15.281868+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:46:15.281868+0000","last_peered":"2024-03-30T00:46:15.281868+0000","last_clean":"2024-03-30T00:46:15.281868+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:46:15.281868+0000","last_undegraded":"2024-03-30T00:46:15.281868+0000","last_fullsized":"2024-03-30T00:46:15.281868+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1773568,"data_stored":1124724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77000000000000002,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65700000000000003,"15min":0.64100000000000001},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86899999999999999,"5min":1.149,"15min":1.149},"last":0.752}]},{"osd":1,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67200000000000004,"15min":0.63600000000000001},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.436},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95199999999999996,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.749,"15min":0.77800000000000002},"min":{"1min":0.52100000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92900000000000005,"5min":1.2450000000000001,"15min":1.6020000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73299999999999998,"15min":0.77700000000000002},"min":{"1min":0.59599999999999997,"5min":0.5,"15min":0.5},"max":{"1min":0.89400000000000002,"5min":1.026,"15min":1.6619999999999999},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.78100000000000003},"min":{"1min":0.54300000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88400000000000001,"5min":1.2270000000000001,"15min":2.0150000000000001},"last":0.62},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78300000000000003,"15min":0.81499999999999995},"min":{"1min":0.61399999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.98299999999999998,"5min":1.0429999999999999,"15min":1.9490000000000001},"last":0.73299999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.626},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.82899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39500000000000002},"max":{"1min":0.80600000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78100000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94199999999999995,"5min":1.121,"15min":1.121},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.75900000000000001},"min":{"1min":0.43099999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":1.0049999999999999,"15min":1.026},"last":0.68500000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71099999999999997,"15min":0.72799999999999998},"min":{"1min":0.50600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.94599999999999995,"5min":1.214,"15min":1.214},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95499999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.83999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1191936,"data_stored":536468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.68000000000000005,"15min":0.86699999999999999},"min":{"1min":0.52700000000000002,"5min":0.39100000000000001,"15min":0.35399999999999998},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":4.0490000000000004},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.52700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.47},"last":0.72499999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.66700000000000004,"15min":0.875},"min":{"1min":0.50600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93400000000000005,"5min":1.0249999999999999,"15min":4.3170000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.70299999999999996,"15min":0.94299999999999995},"min":{"1min":0.57699999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95499999999999996,"5min":1.018,"15min":4.4160000000000004},"last":0.69199999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.71399999999999997,"15min":0.93300000000000005},"min":{"1min":0.65500000000000003,"5min":0.436,"15min":0.436},"max":{"1min":1.24,"5min":1.24,"15min":4.375},"last":0.751},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70199999999999996,"15min":0.93400000000000005},"min":{"1min":0.504,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":4.2960000000000003},"last":0.67400000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72699999999999998,"15min":0.95099999999999996},"min":{"1min":0.56899999999999995,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":4.3399999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.94199999999999995},"min":{"1min":0.55500000000000005,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.258,"5min":1.258,"15min":4.6020000000000003},"last":0.70999999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64100000000000001,"15min":0.66300000000000003},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88100000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.752},"min":{"1min":0.55600000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":1.036,"15min":1.036},"last":0.495}]},{"osd":5,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.622,"15min":0.67800000000000005},"min":{"1min":0.42199999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82899999999999996,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67900000000000005,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95499999999999996,"5min":1.101,"15min":1.101},"last":0.53800000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68300000000000005,"15min":0.71099999999999997},"min":{"1min":0.497,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.89800000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86199999999999999,"5min":1.036,"15min":1.036},"last":0.65800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1171456,"data_stored":534300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.55400000000000005,"5min":0.376,"15min":0.376},"max":{"1min":0.97599999999999998,"5min":0.97899999999999998,"15min":1.095},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.747,"5min":0.67400000000000004,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.7569999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98199999999999998,"5min":0.999,"15min":1.7330000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.72399999999999998},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95499999999999996,"5min":0.96299999999999997,"15min":1.3260000000000001},"last":0.73899999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71099999999999997,"15min":0.72099999999999997},"min":{"1min":0.57299999999999995,"5min":0.45000000000000001,"15min":0.44600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.68100000000000005,"15min":0.71399999999999997},"min":{"1min":0.505,"5min":0.442,"15min":0.442},"max":{"1min":0.95999999999999996,"5min":0.97799999999999998,"15min":1.2789999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70799999999999996,"15min":0.78400000000000003},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.0169999999999999,"5min":1.0269999999999999,"15min":1.7},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.72299999999999998,"15min":0.75700000000000001},"min":{"1min":0.59799999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.181},"last":0.80400000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57299999999999995,"15min":0.623},"min":{"1min":0.438,"5min":0.309,"15min":0.309},"max":{"1min":0.82499999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69899999999999995,"15min":0.76300000000000001},"min":{"1min":0.51200000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97399999999999998,"5min":1.0509999999999999,"15min":1.615},"last":0.77100000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.629,"15min":0.66800000000000004},"min":{"1min":0.47299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.97399999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63600000000000001,"15min":0.70099999999999996},"min":{"1min":0.48899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":1.0700000000000001,"15min":1.569},"last":0.71599999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69599999999999995,"15min":0.74199999999999999},"min":{"1min":0.47099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.88600000000000001,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.82499999999999996},"min":{"1min":0.58799999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99199999999999999,"5min":1.032,"15min":1.383},"last":0.86399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1179648,"data_stored":534460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.46200000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.52700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.85699999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.57199999999999995,"5min":0.45500000000000002,"15min":0.45300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.2909999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.47599999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93400000000000005,"5min":1.0169999999999999,"15min":1.1060000000000001},"last":0.77200000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.51400000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.86399999999999999,"5min":0.93500000000000005,"15min":1.0600000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.75600000000000001},"min":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.46700000000000003},"max":{"1min":0.97099999999999997,"5min":1.2190000000000001,"15min":1.228},"last":0.86499999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85999999999999999,"5min":1.196,"15min":1.27},"last":0.746},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.77400000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1,"5min":1,"15min":1.1739999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61499999999999999,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89400000000000002,"5min":0.92100000000000004,"15min":1.329},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64600000000000002,"15min":0.73299999999999998},"min":{"1min":0.34799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92300000000000004,"5min":0.93500000000000005,"15min":1.3819999999999999},"last":0.80000000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.69099999999999995},"min":{"1min":0.38800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84099999999999997,"5min":0.98899999999999999,"15min":1.2130000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":0.74099999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82999999999999996,"5min":0.96199999999999997,"15min":1.3220000000000001},"last":0.81899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.72899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81699999999999995,"5min":0.999,"15min":1.6910000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72799999999999998,"15min":0.81200000000000006},"min":{"1min":0.49299999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91500000000000004,"5min":1.069,"15min":1.5009999999999999},"last":0.92800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1171456,"data_stored":534300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45000000000000001,"15min":0.46700000000000003},"min":{"1min":0.34300000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63700000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.53600000000000003},"min":{"1min":0.373,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70399999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38900000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.48899999999999999,"15min":0.51300000000000001},"min":{"1min":0.41799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.69299999999999995,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.496},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52200000000000002,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73399999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.436}]},{"osd":2,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.495,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.86499999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55700000000000005,"15min":0.58599999999999997},"min":{"1min":0.47299999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.81299999999999994},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.747,"15min":0.77100000000000002},"min":{"1min":0.60499999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.02,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76300000000000001,"15min":0.78300000000000003},"min":{"1min":0.50800000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0329999999999999,"5min":1.0760000000000001,"15min":1.0860000000000001},"last":0.64500000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.56699999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0369999999999999,"5min":1.1060000000000001,"15min":1.125},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.80700000000000005},"min":{"1min":0.68200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.999,"5min":1.0629999999999999,"15min":1.2390000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.77900000000000003,"15min":0.81599999999999995},"min":{"1min":0.65300000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.995,"5min":1.1699999999999999,"15min":1.226},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78000000000000003,"15min":0.79400000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.004,"5min":1.167,"15min":1.167},"last":0.78300000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80800000000000005,"15min":0.85399999999999998},"min":{"1min":0.63500000000000001,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0389999999999999,"5min":1.1799999999999999,"15min":1.202},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.82199999999999995,"15min":0.88800000000000001},"min":{"1min":0.68300000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.0449999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.1579999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1171456,"data_stored":534300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.443},"min":{"1min":0.34200000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67800000000000005,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49199999999999999,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84099999999999997,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.505}]},{"osd":1,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.38400000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81299999999999994,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.371},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.47099999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.752,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.57999999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49399999999999999,"15min":0.502},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.40000000000000002},"max":{"1min":0.88800000000000001,"5min":1.907,"15min":1.907},"last":0.38900000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80900000000000005,"15min":0.81699999999999995},"min":{"1min":0.65100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1919999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.80700000000000005,"15min":0.84899999999999998},"min":{"1min":0.60899999999999999,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2110000000000001,"5min":1.972,"15min":1.972},"last":0.70799999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.80500000000000005,"15min":0.83599999999999997},"min":{"1min":0.63100000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.2549999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81499999999999995,"15min":0.82599999999999996},"min":{"1min":0.63,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.286,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.82999999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.65400000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.177,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.82799999999999996,"15min":0.82599999999999996},"min":{"1min":0.67800000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.242,"5min":1.867,"15min":1.867},"last":0.74399999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81799999999999995,"15min":0.86799999999999999},"min":{"1min":0.68300000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.1140000000000001,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.83699999999999997,"15min":0.86099999999999999},"min":{"1min":0.64700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.365,"5min":2.851,"15min":2.851},"last":0.80100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1175552,"data_stored":534370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.29999999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.70199999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.36599999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47699999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.439,"15min":0.46200000000000002},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.52500000000000002},"min":{"1min":0.42899999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81799999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84099999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.42599999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53000000000000003,"15min":0.48399999999999999},"min":{"1min":0.45100000000000001,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.52800000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.745,"15min":0.72099999999999997},"min":{"1min":0.51400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.77500000000000002,"15min":0.751},"min":{"1min":0.66900000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.079,"5min":1.2,"15min":1.2},"last":0.80300000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.61699999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.109,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.627},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77700000000000002,"15min":0.76600000000000001},"min":{"1min":0.56599999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.82199999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.69599999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.874},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.78800000000000003,"15min":0.77600000000000002},"min":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0189999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.80000000000000004,"15min":0.79900000000000004},"min":{"1min":0.59399999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2050000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.80700000000000005,"15min":0.78400000000000003},"min":{"1min":0.64800000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1180000000000001,"5min":1.236,"15min":1.236},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1789952,"data_stored":1126938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42799999999999999,"15min":0.46800000000000003},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.625,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.435},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.35699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.502}]},{"osd":2,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.51900000000000002},"min":{"1min":0.38300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82599999999999996,"5min":1.02,"15min":1.02},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.34300000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77900000000000003,"5min":1.006,"15min":1.006},"last":0.46700000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55100000000000005,"15min":0.58999999999999997},"min":{"1min":0.45500000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88900000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55000000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.50600000000000001,"5min":0.504,"15min":0.47899999999999998},"max":{"1min":1.1140000000000001,"5min":1.234,"15min":1.5169999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82799999999999996,"15min":0.81000000000000005},"min":{"1min":0.56000000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0329999999999999,"5min":1.252,"15min":1.252},"last":0.82899999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77900000000000003,"15min":0.76600000000000001},"min":{"1min":0.52200000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96099999999999997,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78600000000000003,"15min":0.78000000000000003},"min":{"1min":0.625,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0780000000000001,"5min":1.2609999999999999,"15min":1.462},"last":0.70299999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.54500000000000004},"max":{"1min":0.98899999999999999,"5min":1.196,"15min":1.5920000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.57799999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0429999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.65800000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.095,"5min":1.351,"15min":1.3600000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.58399999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1759999999999999,"5min":1.2070000000000001,"15min":1.556},"last":0.77100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:46:19.396 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:46:19.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.562+0000 7f7c8de00640 1 Processor -- start 2024-03-30T00:46:19.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.562+0000 7f7c8de00640 1 -- start start 2024-03-30T00:46:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c8de00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7c880cb110 0x7f7c880afd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880ce880 0x7f7c880ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c8de00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c880aecf0 con 0x7f7c88069d90 2024-03-30T00:46:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c8de00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c880aefd0 con 0x7f7c880cb110 2024-03-30T00:46:19.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c8de00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c880af2b0 con 0x7f7c880ce880 2024-03-30T00:46:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c86a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7c880cb110 0x7f7c880afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c7fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880ce880 0x7f7c880ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c86a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7c880cb110 0x7f7c880afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:60814/0 (socket says 172.21.15.66:60814) 2024-03-30T00:46:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c87400640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7c88069d90 0x7f7c880c9b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:47680/0 (socket says 172.21.15.66:47680) 2024-03-30T00:46:19.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c7fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880ce880 0x7f7c880ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:47760/0 (socket says 172.21.15.66:47760) 2024-03-30T00:46:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c86a00640 1 -- 172.21.15.66:0/1403144137 learned_addr learned my addr 172.21.15.66:0/1403144137 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c86a00640 1 -- 172.21.15.66:0/1403144137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880ce880 msgr2=0x7f7c880ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c86a00640 1 --2- 172.21.15.66:0/1403144137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880ce880 0x7f7c880ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c86a00640 1 -- 172.21.15.66:0/1403144137 >> v1:172.21.15.156:6789/0 conn(0x7f7c88069d90 legacy=0x7f7c880c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c86a00640 1 -- 172.21.15.66:0/1403144137 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c880af590 con 0x7f7c880cb110 2024-03-30T00:46:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c86000640 1 -- 172.21.15.66:0/1403144137 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195927411 0 0) 0x7f7c880aecf0 con 0x7f7c88069d90 2024-03-30T00:46:19.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.566+0000 7f7c7fe00640 1 --2- 172.21.15.66:0/1403144137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880ce880 0x7f7c880ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:46:19.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c86a00640 1 --2- 172.21.15.66:0/1403144137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c880cb110 0x7f7c880afd90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009870 tx=0x7f7c7801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a945847d80f1c33 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:19.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c86000640 1 -- 172.21.15.66:0/1403144137 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780a7070 con 0x7f7c880cb110 2024-03-30T00:46:19.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c86000640 1 -- 172.21.15.66:0/1403144137 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c780ab430 con 0x7f7c880cb110 2024-03-30T00:46:19.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c86000640 1 -- 172.21.15.66:0/1403144137 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780ab7c0 con 0x7f7c880cb110 2024-03-30T00:46:19.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c8de00640 1 -- 172.21.15.66:0/1403144137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c880cb110 msgr2=0x7f7c880afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c8de00640 1 --2- 172.21.15.66:0/1403144137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c880cb110 0x7f7c880afd90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009870 tx=0x7f7c7801c560 comp rx=0 tx=0).stop 2024-03-30T00:46:19.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c8de00640 1 -- 172.21.15.66:0/1403144137 shutdown_connections 2024-03-30T00:46:19.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c8de00640 1 --2- 172.21.15.66:0/1403144137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880ce880 0x7f7c880ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c8de00640 1 --2- 172.21.15.66:0/1403144137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c880cb110 0x7f7c880afd90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c8de00640 1 -- 172.21.15.66:0/1403144137 >> 172.21.15.66:0/1403144137 conn(0x7f7c88098a20 msgr2=0x7f7c880cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:19.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c8de00640 1 -- 172.21.15.66:0/1403144137 shutdown_connections 2024-03-30T00:46:19.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c8de00640 1 -- 172.21.15.66:0/1403144137 wait complete. 2024-03-30T00:46:19.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c8de00640 1 Processor -- start 2024-03-30T00:46:19.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.570+0000 7f7c8de00640 1 -- start start 2024-03-30T00:46:19.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c88069d90 0x7f7c880c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880cb110 0x7f7c880c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c8de00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c880ce880 0x7f7c880c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c8de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c880b3c60 con 0x7f7c88069d90 2024-03-30T00:46:19.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c8de00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c88158b80 con 0x7f7c880ce880 2024-03-30T00:46:19.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c8de00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c880b2fe0 con 0x7f7c880cb110 2024-03-30T00:46:19.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c86a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c88069d90 0x7f7c880c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c86a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c88069d90 0x7f7c880c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54302/0 (socket says 172.21.15.66:54302) 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c86a00640 1 -- 172.21.15.66:0/360570287 learned_addr learned my addr 172.21.15.66:0/360570287 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c7fe00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880cb110 0x7f7c880c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c87400640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c880ce880 0x7f7c880c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c86a00640 1 -- 172.21.15.66:0/360570287 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880cb110 msgr2=0x7f7c880c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c86a00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880cb110 0x7f7c880c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c86a00640 1 -- 172.21.15.66:0/360570287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c880ce880 msgr2=0x7f7c880c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c86a00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c880ce880 0x7f7c880c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c86a00640 1 -- 172.21.15.66:0/360570287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c70009420 con 0x7f7c88069d90 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c87400640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c880ce880 0x7f7c880c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c86a00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c88069d90 0x7f7c880c84a0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7c780ad040 tx=0x7f7c78019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c7f400640 1 -- 172.21.15.66:0/360570287 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780bc070 con 0x7f7c88069d90 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c7f400640 1 -- 172.21.15.66:0/360570287 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c780b4420 con 0x7f7c88069d90 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c8de00640 1 -- 172.21.15.66:0/360570287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c780094d0 con 0x7f7c88069d90 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c7f400640 1 -- 172.21.15.66:0/360570287 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780b47d0 con 0x7f7c88069d90 2024-03-30T00:46:19.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c8de00640 1 -- 172.21.15.66:0/360570287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c880b35a0 con 0x7f7c88069d90 2024-03-30T00:46:19.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c8de00640 1 -- 172.21.15.66:0/360570287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c88066c30 con 0x7f7c88069d90 2024-03-30T00:46:19.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.574+0000 7f7c7f400640 1 -- 172.21.15.66:0/360570287 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7c780b4930 con 0x7f7c88069d90 2024-03-30T00:46:19.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.578+0000 7f7c7f400640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7c64066690 0x7f7c64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.578+0000 7f7c7f400640 1 -- 172.21.15.66:0/360570287 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7c7811db50 con 0x7f7c88069d90 2024-03-30T00:46:19.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.578+0000 7f7c7fe00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7c64066690 0x7f7c64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.578+0000 7f7c7fe00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7c64066690 0x7f7c64068b50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7c8815d7d0 tx=0x7f7c70009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:19.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.582+0000 7f7c7f400640 1 -- 172.21.15.66:0/360570287 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7c780eae10 con 0x7f7c88069d90 2024-03-30T00:46:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.738+0000 7f7c8de00640 1 -- 172.21.15.66:0/360570287 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c8806a5f0 con 0x7f7c64066690 2024-03-30T00:46:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.746+0000 7f7c7f400640 1 -- 172.21.15.66:0/360570287 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289367 (secure 0 0 0) 0x7f7c70009940 con 0x7f7c64066690 2024-03-30T00:46:19.757 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:46:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 -- 172.21.15.66:0/360570287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7c64066690 msgr2=0x7f7c64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7c64066690 0x7f7c64068b50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7c8815d7d0 tx=0x7f7c70009190 comp rx=0 tx=0).stop 2024-03-30T00:46:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 -- 172.21.15.66:0/360570287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c88069d90 msgr2=0x7f7c880c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c88069d90 0x7f7c880c84a0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7c780ad040 tx=0x7f7c78019300 comp rx=0 tx=0).stop 2024-03-30T00:46:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 -- 172.21.15.66:0/360570287 shutdown_connections 2024-03-30T00:46:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c880ce880 0x7f7c880c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c880cb110 0x7f7c880c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7c64066690 0x7f7c64068b50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 --2- 172.21.15.66:0/360570287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c88069d90 0x7f7c880c84a0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 -- 172.21.15.66:0/360570287 >> 172.21.15.66:0/360570287 conn(0x7f7c88098a20 msgr2=0x7f7c8815ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 -- 172.21.15.66:0/360570287 shutdown_connections 2024-03-30T00:46:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.750+0000 7f7c8de00640 1 -- 172.21.15.66:0/360570287 wait complete. 2024-03-30T00:46:19.777 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-03-30T00:46:18.936556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259320,"kb_used_data":10376,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715400,"statfs":{"total":773094113280,"available":772828569600,"internally_reserved":0,"allocated":10625024,"data_stored":5459860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.071481"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290689+0000","last_change":"2024-03-30T00:46:15.308059+0000","last_active":"2024-03-30T00:46:16.290689+0000","last_peered":"2024-03-30T00:46:16.290689+0000","last_clean":"2024-03-30T00:46:16.290689+0000","last_became_active":"2024-03-30T00:46:15.307439+0000","last_became_peered":"2024-03-30T00:46:15.307439+0000","last_unstale":"2024-03-30T00:46:16.290689+0000","last_undegraded":"2024-03-30T00:46:16.290689+0000","last_fullsized":"2024-03-30T00:46:16.290689+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:00:36.076780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290389+0000","last_change":"2024-03-30T00:46:15.305235+0000","last_active":"2024-03-30T00:46:16.290389+0000","last_peered":"2024-03-30T00:46:16.290389+0000","last_clean":"2024-03-30T00:46:16.290389+0000","last_became_active":"2024-03-30T00:46:15.304112+0000","last_became_peered":"2024-03-30T00:46:15.304112+0000","last_unstale":"2024-03-30T00:46:16.290389+0000","last_undegraded":"2024-03-30T00:46:16.290389+0000","last_fullsized":"2024-03-30T00:46:16.290389+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:23:43.769848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290003+0000","last_change":"2024-03-30T00:46:15.302482+0000","last_active":"2024-03-30T00:46:16.290003+0000","last_peered":"2024-03-30T00:46:16.290003+0000","last_clean":"2024-03-30T00:46:16.290003+0000","last_became_active":"2024-03-30T00:46:15.301827+0000","last_became_peered":"2024-03-30T00:46:15.301827+0000","last_unstale":"2024-03-30T00:46:16.290003+0000","last_undegraded":"2024-03-30T00:46:16.290003+0000","last_fullsized":"2024-03-30T00:46:16.290003+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:54:38.237992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290003+0000","last_change":"2024-03-30T00:46:15.300660+0000","last_active":"2024-03-30T00:46:16.290003+0000","last_peered":"2024-03-30T00:46:16.290003+0000","last_clean":"2024-03-30T00:46:16.290003+0000","last_became_active":"2024-03-30T00:46:15.300096+0000","last_became_peered":"2024-03-30T00:46:15.300096+0000","last_unstale":"2024-03-30T00:46:16.290003+0000","last_undegraded":"2024-03-30T00:46:16.290003+0000","last_fullsized":"2024-03-30T00:46:16.290003+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:11.158261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281726+0000","last_change":"2024-03-30T00:46:15.306764+0000","last_active":"2024-03-30T00:46:16.281726+0000","last_peered":"2024-03-30T00:46:16.281726+0000","last_clean":"2024-03-30T00:46:16.281726+0000","last_became_active":"2024-03-30T00:46:15.302960+0000","last_became_peered":"2024-03-30T00:46:15.302960+0000","last_unstale":"2024-03-30T00:46:16.281726+0000","last_undegraded":"2024-03-30T00:46:16.281726+0000","last_fullsized":"2024-03-30T00:46:16.281726+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:07.099879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289309+0000","last_change":"2024-03-30T00:46:15.295303+0000","last_active":"2024-03-30T00:46:16.289309+0000","last_peered":"2024-03-30T00:46:16.289309+0000","last_clean":"2024-03-30T00:46:16.289309+0000","last_became_active":"2024-03-30T00:46:15.294925+0000","last_became_peered":"2024-03-30T00:46:15.294925+0000","last_unstale":"2024-03-30T00:46:16.289309+0000","last_undegraded":"2024-03-30T00:46:16.289309+0000","last_fullsized":"2024-03-30T00:46:16.289309+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:47:00.660679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281426+0000","last_change":"2024-03-30T00:46:15.298495+0000","last_active":"2024-03-30T00:46:16.281426+0000","last_peered":"2024-03-30T00:46:16.281426+0000","last_clean":"2024-03-30T00:46:16.281426+0000","last_became_active":"2024-03-30T00:46:15.298152+0000","last_became_peered":"2024-03-30T00:46:15.298152+0000","last_unstale":"2024-03-30T00:46:16.281426+0000","last_undegraded":"2024-03-30T00:46:16.281426+0000","last_fullsized":"2024-03-30T00:46:16.281426+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:46:25.620009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288461+0000","last_change":"2024-03-30T00:46:15.291458+0000","last_active":"2024-03-30T00:46:16.288461+0000","last_peered":"2024-03-30T00:46:16.288461+0000","last_clean":"2024-03-30T00:46:16.288461+0000","last_became_active":"2024-03-30T00:46:15.291196+0000","last_became_peered":"2024-03-30T00:46:15.291196+0000","last_unstale":"2024-03-30T00:46:16.288461+0000","last_undegraded":"2024-03-30T00:46:16.288461+0000","last_fullsized":"2024-03-30T00:46:16.288461+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:07.099879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281438+0000","last_change":"2024-03-30T00:46:15.307046+0000","last_active":"2024-03-30T00:46:16.281438+0000","last_peered":"2024-03-30T00:46:16.281438+0000","last_clean":"2024-03-30T00:46:16.281438+0000","last_became_active":"2024-03-30T00:46:15.306778+0000","last_became_peered":"2024-03-30T00:46:15.306778+0000","last_unstale":"2024-03-30T00:46:16.281438+0000","last_undegraded":"2024-03-30T00:46:16.281438+0000","last_fullsized":"2024-03-30T00:46:16.281438+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:43.055191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278739+0000","last_change":"2024-03-30T00:46:15.310550+0000","last_active":"2024-03-30T00:46:16.278739+0000","last_peered":"2024-03-30T00:46:16.278739+0000","last_clean":"2024-03-30T00:46:16.278739+0000","last_became_active":"2024-03-30T00:46:15.310273+0000","last_became_peered":"2024-03-30T00:46:15.310273+0000","last_unstale":"2024-03-30T00:46:16.278739+0000","last_undegraded":"2024-03-30T00:46:16.278739+0000","last_fullsized":"2024-03-30T00:46:16.278739+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:46:16.515611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.282074+0000","last_change":"2024-03-30T00:46:15.302362+0000","last_active":"2024-03-30T00:46:16.282074+0000","last_peered":"2024-03-30T00:46:16.282074+0000","last_clean":"2024-03-30T00:46:16.282074+0000","last_became_active":"2024-03-30T00:46:15.302103+0000","last_became_peered":"2024-03-30T00:46:15.302103+0000","last_unstale":"2024-03-30T00:46:16.282074+0000","last_undegraded":"2024-03-30T00:46:16.282074+0000","last_fullsized":"2024-03-30T00:46:16.282074+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:48.443688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278866+0000","last_change":"2024-03-30T00:46:15.299163+0000","last_active":"2024-03-30T00:46:16.278866+0000","last_peered":"2024-03-30T00:46:16.278866+0000","last_clean":"2024-03-30T00:46:16.278866+0000","last_became_active":"2024-03-30T00:46:15.298676+0000","last_became_peered":"2024-03-30T00:46:15.298676+0000","last_unstale":"2024-03-30T00:46:16.278866+0000","last_undegraded":"2024-03-30T00:46:16.278866+0000","last_fullsized":"2024-03-30T00:46:16.278866+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:06:17.671736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288610+0000","last_change":"2024-03-30T00:46:15.286247+0000","last_active":"2024-03-30T00:46:16.288610+0000","last_peered":"2024-03-30T00:46:16.288610+0000","last_clean":"2024-03-30T00:46:16.288610+0000","last_became_active":"2024-03-30T00:46:15.285983+0000","last_became_peered":"2024-03-30T00:46:15.285983+0000","last_unstale":"2024-03-30T00:46:16.288610+0000","last_undegraded":"2024-03-30T00:46:16.288610+0000","last_fullsized":"2024-03-30T00:46:16.288610+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:24:51.568922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281259+0000","last_change":"2024-03-30T00:46:15.307475+0000","last_active":"2024-03-30T00:46:16.281259+0000","last_peered":"2024-03-30T00:46:16.281259+0000","last_clean":"2024-03-30T00:46:16.281259+0000","last_became_active":"2024-03-30T00:46:15.307270+0000","last_became_peered":"2024-03-30T00:46:15.307270+0000","last_unstale":"2024-03-30T00:46:16.281259+0000","last_undegraded":"2024-03-30T00:46:16.281259+0000","last_fullsized":"2024-03-30T00:46:16.281259+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:24:36.463049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288558+0000","last_change":"2024-03-30T00:46:15.301244+0000","last_active":"2024-03-30T00:46:16.288558+0000","last_peered":"2024-03-30T00:46:16.288558+0000","last_clean":"2024-03-30T00:46:16.288558+0000","last_became_active":"2024-03-30T00:46:15.300770+0000","last_became_peered":"2024-03-30T00:46:15.300770+0000","last_unstale":"2024-03-30T00:46:16.288558+0000","last_undegraded":"2024-03-30T00:46:16.288558+0000","last_fullsized":"2024-03-30T00:46:16.288558+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:48.443688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289229+0000","last_change":"2024-03-30T00:46:15.304348+0000","last_active":"2024-03-30T00:46:16.289229+0000","last_peered":"2024-03-30T00:46:16.289229+0000","last_clean":"2024-03-30T00:46:16.289229+0000","last_became_active":"2024-03-30T00:46:15.303919+0000","last_became_peered":"2024-03-30T00:46:15.303919+0000","last_unstale":"2024-03-30T00:46:16.289229+0000","last_undegraded":"2024-03-30T00:46:16.289229+0000","last_fullsized":"2024-03-30T00:46:16.289229+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:39:45.212715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290587+0000","last_change":"2024-03-30T00:46:15.304020+0000","last_active":"2024-03-30T00:46:16.290587+0000","last_peered":"2024-03-30T00:46:16.290587+0000","last_clean":"2024-03-30T00:46:16.290587+0000","last_became_active":"2024-03-30T00:46:15.303772+0000","last_became_peered":"2024-03-30T00:46:15.303772+0000","last_unstale":"2024-03-30T00:46:16.290587+0000","last_undegraded":"2024-03-30T00:46:16.290587+0000","last_fullsized":"2024-03-30T00:46:16.290587+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:13:03.746083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290431+0000","last_change":"2024-03-30T00:46:15.302560+0000","last_active":"2024-03-30T00:46:16.290431+0000","last_peered":"2024-03-30T00:46:16.290431+0000","last_clean":"2024-03-30T00:46:16.290431+0000","last_became_active":"2024-03-30T00:46:15.302315+0000","last_became_peered":"2024-03-30T00:46:15.302315+0000","last_unstale":"2024-03-30T00:46:16.290431+0000","last_undegraded":"2024-03-30T00:46:16.290431+0000","last_fullsized":"2024-03-30T00:46:16.290431+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:12:12.474874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.282076+0000","last_change":"2024-03-30T00:46:15.309778+0000","last_active":"2024-03-30T00:46:16.282076+0000","last_peered":"2024-03-30T00:46:16.282076+0000","last_clean":"2024-03-30T00:46:16.282076+0000","last_became_active":"2024-03-30T00:46:15.309532+0000","last_became_peered":"2024-03-30T00:46:15.309532+0000","last_unstale":"2024-03-30T00:46:16.282076+0000","last_undegraded":"2024-03-30T00:46:16.282076+0000","last_fullsized":"2024-03-30T00:46:16.282076+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:36:45.859172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290181+0000","last_change":"2024-03-30T00:46:15.296665+0000","last_active":"2024-03-30T00:46:16.290181+0000","last_peered":"2024-03-30T00:46:16.290181+0000","last_clean":"2024-03-30T00:46:16.290181+0000","last_became_active":"2024-03-30T00:46:15.296286+0000","last_became_peered":"2024-03-30T00:46:15.296286+0000","last_unstale":"2024-03-30T00:46:16.290181+0000","last_undegraded":"2024-03-30T00:46:16.290181+0000","last_fullsized":"2024-03-30T00:46:16.290181+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:20:44.405287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281441+0000","last_change":"2024-03-30T00:46:15.296193+0000","last_active":"2024-03-30T00:46:16.281441+0000","last_peered":"2024-03-30T00:46:16.281441+0000","last_clean":"2024-03-30T00:46:16.281441+0000","last_became_active":"2024-03-30T00:46:15.295934+0000","last_became_peered":"2024-03-30T00:46:15.295934+0000","last_unstale":"2024-03-30T00:46:16.281441+0000","last_undegraded":"2024-03-30T00:46:16.281441+0000","last_fullsized":"2024-03-30T00:46:16.281441+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:11.158261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281302+0000","last_change":"2024-03-30T00:46:15.307418+0000","last_active":"2024-03-30T00:46:16.281302+0000","last_peered":"2024-03-30T00:46:16.281302+0000","last_clean":"2024-03-30T00:46:16.281302+0000","last_became_active":"2024-03-30T00:46:15.306973+0000","last_became_peered":"2024-03-30T00:46:15.306973+0000","last_unstale":"2024-03-30T00:46:16.281302+0000","last_undegraded":"2024-03-30T00:46:16.281302+0000","last_fullsized":"2024-03-30T00:46:16.281302+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:22:27.380550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290547+0000","last_change":"2024-03-30T00:46:15.300421+0000","last_active":"2024-03-30T00:46:16.290547+0000","last_peered":"2024-03-30T00:46:16.290547+0000","last_clean":"2024-03-30T00:46:16.290547+0000","last_became_active":"2024-03-30T00:46:15.300027+0000","last_became_peered":"2024-03-30T00:46:15.300027+0000","last_unstale":"2024-03-30T00:46:16.290547+0000","last_undegraded":"2024-03-30T00:46:16.290547+0000","last_fullsized":"2024-03-30T00:46:16.290547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:35.986067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289194+0000","last_change":"2024-03-30T00:46:15.293486+0000","last_active":"2024-03-30T00:46:16.289194+0000","last_peered":"2024-03-30T00:46:16.289194+0000","last_clean":"2024-03-30T00:46:16.289194+0000","last_became_active":"2024-03-30T00:46:15.293239+0000","last_became_peered":"2024-03-30T00:46:15.293239+0000","last_unstale":"2024-03-30T00:46:16.289194+0000","last_undegraded":"2024-03-30T00:46:16.289194+0000","last_fullsized":"2024-03-30T00:46:16.289194+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:56:20.676119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290527+0000","last_change":"2024-03-30T00:46:15.299088+0000","last_active":"2024-03-30T00:46:16.290527+0000","last_peered":"2024-03-30T00:46:16.290527+0000","last_clean":"2024-03-30T00:46:16.290527+0000","last_became_active":"2024-03-30T00:46:15.298746+0000","last_became_peered":"2024-03-30T00:46:15.298746+0000","last_unstale":"2024-03-30T00:46:16.290527+0000","last_undegraded":"2024-03-30T00:46:16.290527+0000","last_fullsized":"2024-03-30T00:46:16.290527+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:12:17.417262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281782+0000","last_change":"2024-03-30T00:46:15.310274+0000","last_active":"2024-03-30T00:46:16.281782+0000","last_peered":"2024-03-30T00:46:16.281782+0000","last_clean":"2024-03-30T00:46:16.281782+0000","last_became_active":"2024-03-30T00:46:15.309768+0000","last_became_peered":"2024-03-30T00:46:15.309768+0000","last_unstale":"2024-03-30T00:46:16.281782+0000","last_undegraded":"2024-03-30T00:46:16.281782+0000","last_fullsized":"2024-03-30T00:46:16.281782+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:01:25.221218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278944+0000","last_change":"2024-03-30T00:46:15.299498+0000","last_active":"2024-03-30T00:46:16.278944+0000","last_peered":"2024-03-30T00:46:16.278944+0000","last_clean":"2024-03-30T00:46:16.278944+0000","last_became_active":"2024-03-30T00:46:15.299192+0000","last_became_peered":"2024-03-30T00:46:15.299192+0000","last_unstale":"2024-03-30T00:46:16.278944+0000","last_undegraded":"2024-03-30T00:46:16.278944+0000","last_fullsized":"2024-03-30T00:46:16.278944+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:23:39.012673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290272+0000","last_change":"2024-03-30T00:46:15.294718+0000","last_active":"2024-03-30T00:46:16.290272+0000","last_peered":"2024-03-30T00:46:16.290272+0000","last_clean":"2024-03-30T00:46:16.290272+0000","last_became_active":"2024-03-30T00:46:15.294466+0000","last_became_peered":"2024-03-30T00:46:15.294466+0000","last_unstale":"2024-03-30T00:46:16.290272+0000","last_undegraded":"2024-03-30T00:46:16.290272+0000","last_fullsized":"2024-03-30T00:46:16.290272+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:08.355340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281869+0000","last_change":"2024-03-30T00:46:15.310622+0000","last_active":"2024-03-30T00:46:16.281869+0000","last_peered":"2024-03-30T00:46:16.281869+0000","last_clean":"2024-03-30T00:46:16.281869+0000","last_became_active":"2024-03-30T00:46:15.308324+0000","last_became_peered":"2024-03-30T00:46:15.308324+0000","last_unstale":"2024-03-30T00:46:16.281869+0000","last_undegraded":"2024-03-30T00:46:16.281869+0000","last_fullsized":"2024-03-30T00:46:16.281869+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:17:56.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281502+0000","last_change":"2024-03-30T00:46:15.307248+0000","last_active":"2024-03-30T00:46:16.281502+0000","last_peered":"2024-03-30T00:46:16.281502+0000","last_clean":"2024-03-30T00:46:16.281502+0000","last_became_active":"2024-03-30T00:46:15.306700+0000","last_became_peered":"2024-03-30T00:46:15.306700+0000","last_unstale":"2024-03-30T00:46:16.281502+0000","last_undegraded":"2024-03-30T00:46:16.281502+0000","last_fullsized":"2024-03-30T00:46:16.281502+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:08.355340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290359+0000","last_change":"2024-03-30T00:46:15.305732+0000","last_active":"2024-03-30T00:46:16.290359+0000","last_peered":"2024-03-30T00:46:16.290359+0000","last_clean":"2024-03-30T00:46:16.290359+0000","last_became_active":"2024-03-30T00:46:15.305395+0000","last_became_peered":"2024-03-30T00:46:15.305395+0000","last_unstale":"2024-03-30T00:46:16.290359+0000","last_undegraded":"2024-03-30T00:46:16.290359+0000","last_fullsized":"2024-03-30T00:46:16.290359+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:46:25.620009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289317+0000","last_change":"2024-03-30T00:46:15.297837+0000","last_active":"2024-03-30T00:46:16.289317+0000","last_peered":"2024-03-30T00:46:16.289317+0000","last_clean":"2024-03-30T00:46:16.289317+0000","last_became_active":"2024-03-30T00:46:15.297554+0000","last_became_peered":"2024-03-30T00:46:15.297554+0000","last_unstale":"2024-03-30T00:46:16.289317+0000","last_undegraded":"2024-03-30T00:46:16.289317+0000","last_fullsized":"2024-03-30T00:46:16.289317+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:14:44.676649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289608+0000","last_change":"2024-03-30T00:46:15.294294+0000","last_active":"2024-03-30T00:46:16.289608+0000","last_peered":"2024-03-30T00:46:16.289608+0000","last_clean":"2024-03-30T00:46:16.289608+0000","last_became_active":"2024-03-30T00:46:15.293879+0000","last_became_peered":"2024-03-30T00:46:15.293879+0000","last_unstale":"2024-03-30T00:46:16.289608+0000","last_undegraded":"2024-03-30T00:46:16.289608+0000","last_fullsized":"2024-03-30T00:46:16.289608+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:54:55.569737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"94'4","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.291123+0000","last_change":"2024-03-30T00:46:14.892711+0000","last_active":"2024-03-30T00:46:16.291123+0000","last_peered":"2024-03-30T00:46:16.291123+0000","last_clean":"2024-03-30T00:46:16.291123+0000","last_became_active":"2024-03-30T00:46:14.892462+0000","last_became_peered":"2024-03-30T00:46:14.892462+0000","last_unstale":"2024-03-30T00:46:16.291123+0000","last_undegraded":"2024-03-30T00:46:16.291123+0000","last_fullsized":"2024-03-30T00:46:16.291123+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:38:33.451309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290004+0000","last_change":"2024-03-30T00:46:15.305329+0000","last_active":"2024-03-30T00:46:16.290004+0000","last_peered":"2024-03-30T00:46:16.290004+0000","last_clean":"2024-03-30T00:46:16.290004+0000","last_became_active":"2024-03-30T00:46:15.305099+0000","last_became_peered":"2024-03-30T00:46:15.305099+0000","last_unstale":"2024-03-30T00:46:16.290004+0000","last_undegraded":"2024-03-30T00:46:16.290004+0000","last_fullsized":"2024-03-30T00:46:16.290004+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:29.566350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281853+0000","last_change":"2024-03-30T00:46:14.893880+0000","last_active":"2024-03-30T00:46:16.281853+0000","last_peered":"2024-03-30T00:46:16.281853+0000","last_clean":"2024-03-30T00:46:16.281853+0000","last_became_active":"2024-03-30T00:46:14.893628+0000","last_became_peered":"2024-03-30T00:46:14.893628+0000","last_unstale":"2024-03-30T00:46:16.281853+0000","last_undegraded":"2024-03-30T00:46:16.281853+0000","last_fullsized":"2024-03-30T00:46:16.281853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:39.993735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281801+0000","last_change":"2024-03-30T00:46:15.295961+0000","last_active":"2024-03-30T00:46:16.281801+0000","last_peered":"2024-03-30T00:46:16.281801+0000","last_clean":"2024-03-30T00:46:16.281801+0000","last_became_active":"2024-03-30T00:46:15.295659+0000","last_became_peered":"2024-03-30T00:46:15.295659+0000","last_unstale":"2024-03-30T00:46:16.281801+0000","last_undegraded":"2024-03-30T00:46:16.281801+0000","last_fullsized":"2024-03-30T00:46:16.281801+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:13.184530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281679+0000","last_change":"2024-03-30T00:46:14.888612+0000","last_active":"2024-03-30T00:46:16.281679+0000","last_peered":"2024-03-30T00:46:16.281679+0000","last_clean":"2024-03-30T00:46:16.281679+0000","last_became_active":"2024-03-30T00:46:14.888341+0000","last_became_peered":"2024-03-30T00:46:14.888341+0000","last_unstale":"2024-03-30T00:46:16.281679+0000","last_undegraded":"2024-03-30T00:46:16.281679+0000","last_fullsized":"2024-03-30T00:46:16.281679+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:48:36.319396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290712+0000","last_change":"2024-03-30T00:46:15.292459+0000","last_active":"2024-03-30T00:46:16.290712+0000","last_peered":"2024-03-30T00:46:16.290712+0000","last_clean":"2024-03-30T00:46:16.290712+0000","last_became_active":"2024-03-30T00:46:15.290565+0000","last_became_peered":"2024-03-30T00:46:15.290565+0000","last_unstale":"2024-03-30T00:46:16.290712+0000","last_undegraded":"2024-03-30T00:46:16.290712+0000","last_fullsized":"2024-03-30T00:46:16.290712+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:02:51.248469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290931+0000","last_change":"2024-03-30T00:46:14.889403+0000","last_active":"2024-03-30T00:46:16.290931+0000","last_peered":"2024-03-30T00:46:16.290931+0000","last_clean":"2024-03-30T00:46:16.290931+0000","last_became_active":"2024-03-30T00:46:14.888289+0000","last_became_peered":"2024-03-30T00:46:14.888289+0000","last_unstale":"2024-03-30T00:46:16.290931+0000","last_undegraded":"2024-03-30T00:46:16.290931+0000","last_fullsized":"2024-03-30T00:46:16.290931+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:01:16.519724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288463+0000","last_change":"2024-03-30T00:46:15.300282+0000","last_active":"2024-03-30T00:46:16.288463+0000","last_peered":"2024-03-30T00:46:16.288463+0000","last_clean":"2024-03-30T00:46:16.288463+0000","last_became_active":"2024-03-30T00:46:15.299988+0000","last_became_peered":"2024-03-30T00:46:15.299988+0000","last_unstale":"2024-03-30T00:46:16.288463+0000","last_undegraded":"2024-03-30T00:46:16.288463+0000","last_fullsized":"2024-03-30T00:46:16.288463+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:43.055191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281647+0000","last_change":"2024-03-30T00:46:14.892764+0000","last_active":"2024-03-30T00:46:16.281647+0000","last_peered":"2024-03-30T00:46:16.281647+0000","last_clean":"2024-03-30T00:46:16.281647+0000","last_became_active":"2024-03-30T00:46:14.892520+0000","last_became_peered":"2024-03-30T00:46:14.892520+0000","last_unstale":"2024-03-30T00:46:16.281647+0000","last_undegraded":"2024-03-30T00:46:16.281647+0000","last_fullsized":"2024-03-30T00:46:16.281647+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:16:38.775300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290625+0000","last_change":"2024-03-30T00:46:15.289253+0000","last_active":"2024-03-30T00:46:16.290625+0000","last_peered":"2024-03-30T00:46:16.290625+0000","last_clean":"2024-03-30T00:46:16.290625+0000","last_became_active":"2024-03-30T00:46:15.288992+0000","last_became_peered":"2024-03-30T00:46:15.288992+0000","last_unstale":"2024-03-30T00:46:16.290625+0000","last_undegraded":"2024-03-30T00:46:16.290625+0000","last_fullsized":"2024-03-30T00:46:16.290625+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:16:39.777547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288984+0000","last_change":"2024-03-30T00:46:14.887491+0000","last_active":"2024-03-30T00:46:16.288984+0000","last_peered":"2024-03-30T00:46:16.288984+0000","last_clean":"2024-03-30T00:46:16.288984+0000","last_became_active":"2024-03-30T00:46:14.887178+0000","last_became_peered":"2024-03-30T00:46:14.887178+0000","last_unstale":"2024-03-30T00:46:16.288984+0000","last_undegraded":"2024-03-30T00:46:16.288984+0000","last_fullsized":"2024-03-30T00:46:16.288984+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:30.462631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289244+0000","last_change":"2024-03-30T00:46:15.295155+0000","last_active":"2024-03-30T00:46:16.289244+0000","last_peered":"2024-03-30T00:46:16.289244+0000","last_clean":"2024-03-30T00:46:16.289244+0000","last_became_active":"2024-03-30T00:46:15.294564+0000","last_became_peered":"2024-03-30T00:46:15.294564+0000","last_unstale":"2024-03-30T00:46:16.289244+0000","last_undegraded":"2024-03-30T00:46:16.289244+0000","last_fullsized":"2024-03-30T00:46:16.289244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:32:21.356152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289528+0000","last_change":"2024-03-30T00:46:14.889581+0000","last_active":"2024-03-30T00:46:16.289528+0000","last_peered":"2024-03-30T00:46:16.289528+0000","last_clean":"2024-03-30T00:46:16.289528+0000","last_became_active":"2024-03-30T00:46:14.889336+0000","last_became_peered":"2024-03-30T00:46:14.889336+0000","last_unstale":"2024-03-30T00:46:16.289528+0000","last_undegraded":"2024-03-30T00:46:16.289528+0000","last_fullsized":"2024-03-30T00:46:16.289528+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:20:43.403040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289714+0000","last_change":"2024-03-30T00:46:15.300746+0000","last_active":"2024-03-30T00:46:16.289714+0000","last_peered":"2024-03-30T00:46:16.289714+0000","last_clean":"2024-03-30T00:46:16.289714+0000","last_became_active":"2024-03-30T00:46:15.300051+0000","last_became_peered":"2024-03-30T00:46:15.300051+0000","last_unstale":"2024-03-30T00:46:16.289714+0000","last_undegraded":"2024-03-30T00:46:16.289714+0000","last_fullsized":"2024-03-30T00:46:16.289714+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:23:43.769848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281541+0000","last_change":"2024-03-30T00:46:14.891615+0000","last_active":"2024-03-30T00:46:16.281541+0000","last_peered":"2024-03-30T00:46:16.281541+0000","last_clean":"2024-03-30T00:46:16.281541+0000","last_became_active":"2024-03-30T00:46:14.891310+0000","last_became_peered":"2024-03-30T00:46:14.891310+0000","last_unstale":"2024-03-30T00:46:16.281541+0000","last_undegraded":"2024-03-30T00:46:16.281541+0000","last_fullsized":"2024-03-30T00:46:16.281541+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:39.993735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281302+0000","last_change":"2024-03-30T00:46:15.305106+0000","last_active":"2024-03-30T00:46:16.281302+0000","last_peered":"2024-03-30T00:46:16.281302+0000","last_clean":"2024-03-30T00:46:16.281302+0000","last_became_active":"2024-03-30T00:46:15.304843+0000","last_became_peered":"2024-03-30T00:46:15.304843+0000","last_unstale":"2024-03-30T00:46:16.281302+0000","last_undegraded":"2024-03-30T00:46:16.281302+0000","last_fullsized":"2024-03-30T00:46:16.281302+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:20:44.405287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290690+0000","last_change":"2024-03-30T00:46:14.885281+0000","last_active":"2024-03-30T00:46:16.290690+0000","last_peered":"2024-03-30T00:46:16.290690+0000","last_clean":"2024-03-30T00:46:16.290690+0000","last_became_active":"2024-03-30T00:46:14.885024+0000","last_became_peered":"2024-03-30T00:46:14.885024+0000","last_unstale":"2024-03-30T00:46:16.290690+0000","last_undegraded":"2024-03-30T00:46:16.290690+0000","last_fullsized":"2024-03-30T00:46:16.290690+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:19:51.018234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281168+0000","last_change":"2024-03-30T00:46:15.309120+0000","last_active":"2024-03-30T00:46:16.281168+0000","last_peered":"2024-03-30T00:46:16.281168+0000","last_clean":"2024-03-30T00:46:16.281168+0000","last_became_active":"2024-03-30T00:46:15.308613+0000","last_became_peered":"2024-03-30T00:46:15.308613+0000","last_unstale":"2024-03-30T00:46:16.281168+0000","last_undegraded":"2024-03-30T00:46:16.281168+0000","last_fullsized":"2024-03-30T00:46:16.281168+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:27:04.442085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290411+0000","last_change":"2024-03-30T00:46:14.890990+0000","last_active":"2024-03-30T00:46:16.290411+0000","last_peered":"2024-03-30T00:46:16.290411+0000","last_clean":"2024-03-30T00:46:16.290411+0000","last_became_active":"2024-03-30T00:46:14.890723+0000","last_became_peered":"2024-03-30T00:46:14.890723+0000","last_unstale":"2024-03-30T00:46:16.290411+0000","last_undegraded":"2024-03-30T00:46:16.290411+0000","last_fullsized":"2024-03-30T00:46:16.290411+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:47.441441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290334+0000","last_change":"2024-03-30T00:46:15.295516+0000","last_active":"2024-03-30T00:46:16.290334+0000","last_peered":"2024-03-30T00:46:16.290334+0000","last_clean":"2024-03-30T00:46:16.290334+0000","last_became_active":"2024-03-30T00:46:15.295186+0000","last_became_peered":"2024-03-30T00:46:15.295186+0000","last_unstale":"2024-03-30T00:46:16.290334+0000","last_undegraded":"2024-03-30T00:46:16.290334+0000","last_fullsized":"2024-03-30T00:46:16.290334+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:49:08.894562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.282157+0000","last_change":"2024-03-30T00:46:14.897986+0000","last_active":"2024-03-30T00:46:16.282157+0000","last_peered":"2024-03-30T00:46:16.282157+0000","last_clean":"2024-03-30T00:46:16.282157+0000","last_became_active":"2024-03-30T00:46:14.897688+0000","last_became_peered":"2024-03-30T00:46:14.897688+0000","last_unstale":"2024-03-30T00:46:16.282157+0000","last_undegraded":"2024-03-30T00:46:16.282157+0000","last_fullsized":"2024-03-30T00:46:16.282157+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:19:51.018234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290562+0000","last_change":"2024-03-30T00:46:15.304158+0000","last_active":"2024-03-30T00:46:16.290562+0000","last_peered":"2024-03-30T00:46:16.290562+0000","last_clean":"2024-03-30T00:46:16.290562+0000","last_became_active":"2024-03-30T00:46:15.303898+0000","last_became_peered":"2024-03-30T00:46:15.303898+0000","last_unstale":"2024-03-30T00:46:16.290562+0000","last_undegraded":"2024-03-30T00:46:16.290562+0000","last_fullsized":"2024-03-30T00:46:16.290562+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:12:17.417262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289806+0000","last_change":"2024-03-30T00:46:14.887195+0000","last_active":"2024-03-30T00:46:16.289806+0000","last_peered":"2024-03-30T00:46:16.289806+0000","last_clean":"2024-03-30T00:46:16.289806+0000","last_became_active":"2024-03-30T00:46:14.886205+0000","last_became_peered":"2024-03-30T00:46:14.886205+0000","last_unstale":"2024-03-30T00:46:16.289806+0000","last_undegraded":"2024-03-30T00:46:16.289806+0000","last_fullsized":"2024-03-30T00:46:16.289806+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:05:45.340862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288579+0000","last_change":"2024-03-30T00:46:15.292179+0000","last_active":"2024-03-30T00:46:16.288579+0000","last_peered":"2024-03-30T00:46:16.288579+0000","last_clean":"2024-03-30T00:46:16.288579+0000","last_became_active":"2024-03-30T00:46:15.291360+0000","last_became_peered":"2024-03-30T00:46:15.291360+0000","last_unstale":"2024-03-30T00:46:16.288579+0000","last_undegraded":"2024-03-30T00:46:16.288579+0000","last_fullsized":"2024-03-30T00:46:16.288579+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:27:04.442085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281951+0000","last_change":"2024-03-30T00:46:14.891171+0000","last_active":"2024-03-30T00:46:16.281951+0000","last_peered":"2024-03-30T00:46:16.281951+0000","last_clean":"2024-03-30T00:46:16.281951+0000","last_became_active":"2024-03-30T00:46:14.890866+0000","last_became_peered":"2024-03-30T00:46:14.890866+0000","last_unstale":"2024-03-30T00:46:16.281951+0000","last_undegraded":"2024-03-30T00:46:16.281951+0000","last_fullsized":"2024-03-30T00:46:16.281951+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:19:51.018234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.280954+0000","last_change":"2024-03-30T00:46:15.303091+0000","last_active":"2024-03-30T00:46:16.280954+0000","last_peered":"2024-03-30T00:46:16.280954+0000","last_clean":"2024-03-30T00:46:16.280954+0000","last_became_active":"2024-03-30T00:46:15.302755+0000","last_became_peered":"2024-03-30T00:46:15.302755+0000","last_unstale":"2024-03-30T00:46:16.280954+0000","last_undegraded":"2024-03-30T00:46:16.280954+0000","last_fullsized":"2024-03-30T00:46:16.280954+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:17:56.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289346+0000","last_change":"2024-03-30T00:46:14.893558+0000","last_active":"2024-03-30T00:46:16.289346+0000","last_peered":"2024-03-30T00:46:16.289346+0000","last_clean":"2024-03-30T00:46:16.289346+0000","last_became_active":"2024-03-30T00:46:14.893205+0000","last_became_peered":"2024-03-30T00:46:14.893205+0000","last_unstale":"2024-03-30T00:46:16.289346+0000","last_undegraded":"2024-03-30T00:46:16.289346+0000","last_fullsized":"2024-03-30T00:46:16.289346+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:26:57.918249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278635+0000","last_change":"2024-03-30T00:46:15.304449+0000","last_active":"2024-03-30T00:46:16.278635+0000","last_peered":"2024-03-30T00:46:16.278635+0000","last_clean":"2024-03-30T00:46:16.278635+0000","last_became_active":"2024-03-30T00:46:15.304141+0000","last_became_peered":"2024-03-30T00:46:15.304141+0000","last_unstale":"2024-03-30T00:46:16.278635+0000","last_undegraded":"2024-03-30T00:46:16.278635+0000","last_fullsized":"2024-03-30T00:46:16.278635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:52.772358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288634+0000","last_change":"2024-03-30T00:46:14.884431+0000","last_active":"2024-03-30T00:46:16.288634+0000","last_peered":"2024-03-30T00:46:16.288634+0000","last_clean":"2024-03-30T00:46:16.288634+0000","last_became_active":"2024-03-30T00:46:14.882004+0000","last_became_peered":"2024-03-30T00:46:14.882004+0000","last_unstale":"2024-03-30T00:46:16.288634+0000","last_undegraded":"2024-03-30T00:46:16.288634+0000","last_fullsized":"2024-03-30T00:46:16.288634+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:39.993735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278895+0000","last_change":"2024-03-30T00:46:15.310128+0000","last_active":"2024-03-30T00:46:16.278895+0000","last_peered":"2024-03-30T00:46:16.278895+0000","last_clean":"2024-03-30T00:46:16.278895+0000","last_became_active":"2024-03-30T00:46:15.309871+0000","last_became_peered":"2024-03-30T00:46:15.309871+0000","last_unstale":"2024-03-30T00:46:16.278895+0000","last_undegraded":"2024-03-30T00:46:16.278895+0000","last_fullsized":"2024-03-30T00:46:16.278895+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:00.840705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":19,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.288699+0000","last_change":"2024-03-30T00:46:14.882528+0000","last_active":"2024-03-30T00:46:16.288699+0000","last_peered":"2024-03-30T00:46:16.288699+0000","last_clean":"2024-03-30T00:46:16.288699+0000","last_became_active":"2024-03-30T00:46:14.882285+0000","last_became_peered":"2024-03-30T00:46:14.882285+0000","last_unstale":"2024-03-30T00:46:16.288699+0000","last_undegraded":"2024-03-30T00:46:16.288699+0000","last_fullsized":"2024-03-30T00:46:16.288699+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:34.983820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289698+0000","last_change":"2024-03-30T00:46:15.294166+0000","last_active":"2024-03-30T00:46:16.289698+0000","last_peered":"2024-03-30T00:46:16.289698+0000","last_clean":"2024-03-30T00:46:16.289698+0000","last_became_active":"2024-03-30T00:46:15.293471+0000","last_became_peered":"2024-03-30T00:46:15.293471+0000","last_unstale":"2024-03-30T00:46:16.289698+0000","last_undegraded":"2024-03-30T00:46:16.289698+0000","last_fullsized":"2024-03-30T00:46:16.289698+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:41:12.847342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.291180+0000","last_change":"2024-03-30T00:46:14.888408+0000","last_active":"2024-03-30T00:46:16.291180+0000","last_peered":"2024-03-30T00:46:16.291180+0000","last_clean":"2024-03-30T00:46:16.291180+0000","last_became_active":"2024-03-30T00:46:14.888146+0000","last_became_peered":"2024-03-30T00:46:14.888146+0000","last_unstale":"2024-03-30T00:46:16.291180+0000","last_undegraded":"2024-03-30T00:46:16.291180+0000","last_fullsized":"2024-03-30T00:46:16.291180+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:57:40.214555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.291061+0000","last_change":"2024-03-30T00:46:15.297608+0000","last_active":"2024-03-30T00:46:16.291061+0000","last_peered":"2024-03-30T00:46:16.291061+0000","last_clean":"2024-03-30T00:46:16.291061+0000","last_became_active":"2024-03-30T00:46:15.297314+0000","last_became_peered":"2024-03-30T00:46:15.297314+0000","last_unstale":"2024-03-30T00:46:16.291061+0000","last_undegraded":"2024-03-30T00:46:16.291061+0000","last_fullsized":"2024-03-30T00:46:16.291061+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:31.464878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281899+0000","last_change":"2024-03-30T00:46:14.883401+0000","last_active":"2024-03-30T00:46:16.281899+0000","last_peered":"2024-03-30T00:46:16.281899+0000","last_clean":"2024-03-30T00:46:16.281899+0000","last_became_active":"2024-03-30T00:46:14.883151+0000","last_became_peered":"2024-03-30T00:46:14.883151+0000","last_unstale":"2024-03-30T00:46:16.281899+0000","last_undegraded":"2024-03-30T00:46:16.281899+0000","last_fullsized":"2024-03-30T00:46:16.281899+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:34.983820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281770+0000","last_change":"2024-03-30T00:46:15.310099+0000","last_active":"2024-03-30T00:46:16.281770+0000","last_peered":"2024-03-30T00:46:16.281770+0000","last_clean":"2024-03-30T00:46:16.281770+0000","last_became_active":"2024-03-30T00:46:15.309420+0000","last_became_peered":"2024-03-30T00:46:15.309420+0000","last_unstale":"2024-03-30T00:46:16.281770+0000","last_undegraded":"2024-03-30T00:46:16.281770+0000","last_fullsized":"2024-03-30T00:46:16.281770+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:12:17.417262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.282218+0000","last_change":"2024-03-30T00:46:14.895153+0000","last_active":"2024-03-30T00:46:16.282218+0000","last_peered":"2024-03-30T00:46:16.282218+0000","last_clean":"2024-03-30T00:46:16.282218+0000","last_became_active":"2024-03-30T00:46:14.894650+0000","last_became_peered":"2024-03-30T00:46:14.894650+0000","last_unstale":"2024-03-30T00:46:16.282218+0000","last_undegraded":"2024-03-30T00:46:16.282218+0000","last_fullsized":"2024-03-30T00:46:16.282218+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:50:34.983820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289404+0000","last_change":"2024-03-30T00:46:15.295130+0000","last_active":"2024-03-30T00:46:16.289404+0000","last_peered":"2024-03-30T00:46:16.289404+0000","last_clean":"2024-03-30T00:46:16.289404+0000","last_became_active":"2024-03-30T00:46:15.294760+0000","last_became_peered":"2024-03-30T00:46:15.294760+0000","last_unstale":"2024-03-30T00:46:16.289404+0000","last_undegraded":"2024-03-30T00:46:16.289404+0000","last_fullsized":"2024-03-30T00:46:16.289404+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:49:14.217720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290180+0000","last_change":"2024-03-30T00:46:14.894486+0000","last_active":"2024-03-30T00:46:16.290180+0000","last_peered":"2024-03-30T00:46:16.290180+0000","last_clean":"2024-03-30T00:46:16.290180+0000","last_became_active":"2024-03-30T00:46:14.894164+0000","last_became_peered":"2024-03-30T00:46:14.894164+0000","last_unstale":"2024-03-30T00:46:16.290180+0000","last_undegraded":"2024-03-30T00:46:16.290180+0000","last_fullsized":"2024-03-30T00:46:16.290180+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:39.993735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290274+0000","last_change":"2024-03-30T00:46:15.294059+0000","last_active":"2024-03-30T00:46:16.290274+0000","last_peered":"2024-03-30T00:46:16.290274+0000","last_clean":"2024-03-30T00:46:16.290274+0000","last_became_active":"2024-03-30T00:46:15.293719+0000","last_became_peered":"2024-03-30T00:46:15.293719+0000","last_unstale":"2024-03-30T00:46:16.290274+0000","last_undegraded":"2024-03-30T00:46:16.290274+0000","last_fullsized":"2024-03-30T00:46:16.290274+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:24:51.568922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278797+0000","last_change":"2024-03-30T00:46:14.884891+0000","last_active":"2024-03-30T00:46:16.278797+0000","last_peered":"2024-03-30T00:46:16.278797+0000","last_clean":"2024-03-30T00:46:16.278797+0000","last_became_active":"2024-03-30T00:46:14.884373+0000","last_became_peered":"2024-03-30T00:46:14.884373+0000","last_unstale":"2024-03-30T00:46:16.278797+0000","last_undegraded":"2024-03-30T00:46:16.278797+0000","last_fullsized":"2024-03-30T00:46:16.278797+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:16.504340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290248+0000","last_change":"2024-03-30T00:46:15.305896+0000","last_active":"2024-03-30T00:46:16.290248+0000","last_peered":"2024-03-30T00:46:16.290248+0000","last_clean":"2024-03-30T00:46:16.290248+0000","last_became_active":"2024-03-30T00:46:15.305611+0000","last_became_peered":"2024-03-30T00:46:15.305611+0000","last_unstale":"2024-03-30T00:46:16.290248+0000","last_undegraded":"2024-03-30T00:46:16.290248+0000","last_fullsized":"2024-03-30T00:46:16.290248+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:52:13.184530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290375+0000","last_change":"2024-03-30T00:46:14.888244+0000","last_active":"2024-03-30T00:46:16.290375+0000","last_peered":"2024-03-30T00:46:16.290375+0000","last_clean":"2024-03-30T00:46:16.290375+0000","last_became_active":"2024-03-30T00:46:14.887973+0000","last_became_peered":"2024-03-30T00:46:14.887973+0000","last_unstale":"2024-03-30T00:46:16.290375+0000","last_undegraded":"2024-03-30T00:46:16.290375+0000","last_fullsized":"2024-03-30T00:46:16.290375+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:40:16.023181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289444+0000","last_change":"2024-03-30T00:46:15.304651+0000","last_active":"2024-03-30T00:46:16.289444+0000","last_peered":"2024-03-30T00:46:16.289444+0000","last_clean":"2024-03-30T00:46:16.289444+0000","last_became_active":"2024-03-30T00:46:15.304395+0000","last_became_peered":"2024-03-30T00:46:15.304395+0000","last_unstale":"2024-03-30T00:46:16.289444+0000","last_undegraded":"2024-03-30T00:46:16.289444+0000","last_fullsized":"2024-03-30T00:46:16.289444+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:17.650237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278627+0000","last_change":"2024-03-30T00:46:14.879896+0000","last_active":"2024-03-30T00:46:16.278627+0000","last_peered":"2024-03-30T00:46:16.278627+0000","last_clean":"2024-03-30T00:46:16.278627+0000","last_became_active":"2024-03-30T00:46:14.879637+0000","last_became_peered":"2024-03-30T00:46:14.879637+0000","last_unstale":"2024-03-30T00:46:16.278627+0000","last_undegraded":"2024-03-30T00:46:16.278627+0000","last_fullsized":"2024-03-30T00:46:16.278627+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:16:58.396814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290966+0000","last_change":"2024-03-30T00:46:15.291716+0000","last_active":"2024-03-30T00:46:16.290966+0000","last_peered":"2024-03-30T00:46:16.290966+0000","last_clean":"2024-03-30T00:46:16.290966+0000","last_became_active":"2024-03-30T00:46:15.291407+0000","last_became_peered":"2024-03-30T00:46:15.291407+0000","last_unstale":"2024-03-30T00:46:16.290966+0000","last_undegraded":"2024-03-30T00:46:16.290966+0000","last_fullsized":"2024-03-30T00:46:16.290966+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:25:42.971905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281554+0000","last_change":"2024-03-30T00:46:14.887651+0000","last_active":"2024-03-30T00:46:16.281554+0000","last_peered":"2024-03-30T00:46:16.281554+0000","last_clean":"2024-03-30T00:46:16.281554+0000","last_became_active":"2024-03-30T00:46:14.887323+0000","last_became_peered":"2024-03-30T00:46:14.887323+0000","last_unstale":"2024-03-30T00:46:16.281554+0000","last_undegraded":"2024-03-30T00:46:16.281554+0000","last_fullsized":"2024-03-30T00:46:16.281554+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:25.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290126+0000","last_change":"2024-03-30T00:46:15.304851+0000","last_active":"2024-03-30T00:46:16.290126+0000","last_peered":"2024-03-30T00:46:16.290126+0000","last_clean":"2024-03-30T00:46:16.290126+0000","last_became_active":"2024-03-30T00:46:15.301757+0000","last_became_peered":"2024-03-30T00:46:15.301757+0000","last_unstale":"2024-03-30T00:46:16.290126+0000","last_undegraded":"2024-03-30T00:46:16.290126+0000","last_fullsized":"2024-03-30T00:46:16.290126+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:19:41.175787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"94'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281883+0000","last_change":"2024-03-30T00:46:14.895879+0000","last_active":"2024-03-30T00:46:16.281883+0000","last_peered":"2024-03-30T00:46:16.281883+0000","last_clean":"2024-03-30T00:46:16.281883+0000","last_became_active":"2024-03-30T00:46:14.895595+0000","last_became_peered":"2024-03-30T00:46:14.895595+0000","last_unstale":"2024-03-30T00:46:16.281883+0000","last_undegraded":"2024-03-30T00:46:16.281883+0000","last_fullsized":"2024-03-30T00:46:16.281883+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:48:36.319396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290251+0000","last_change":"2024-03-30T00:46:15.304589+0000","last_active":"2024-03-30T00:46:16.290251+0000","last_peered":"2024-03-30T00:46:16.290251+0000","last_clean":"2024-03-30T00:46:16.290251+0000","last_became_active":"2024-03-30T00:46:15.304299+0000","last_became_peered":"2024-03-30T00:46:15.304299+0000","last_unstale":"2024-03-30T00:46:16.290251+0000","last_undegraded":"2024-03-30T00:46:16.290251+0000","last_fullsized":"2024-03-30T00:46:16.290251+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:36:51.229729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.282211+0000","last_change":"2024-03-30T00:46:14.897571+0000","last_active":"2024-03-30T00:46:16.282211+0000","last_peered":"2024-03-30T00:46:16.282211+0000","last_clean":"2024-03-30T00:46:16.282211+0000","last_became_active":"2024-03-30T00:46:14.897375+0000","last_became_peered":"2024-03-30T00:46:14.897375+0000","last_unstale":"2024-03-30T00:46:16.282211+0000","last_undegraded":"2024-03-30T00:46:16.282211+0000","last_fullsized":"2024-03-30T00:46:16.282211+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:30.462631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290615+0000","last_change":"2024-03-30T00:46:15.305570+0000","last_active":"2024-03-30T00:46:16.290615+0000","last_peered":"2024-03-30T00:46:16.290615+0000","last_clean":"2024-03-30T00:46:16.290615+0000","last_became_active":"2024-03-30T00:46:15.305249+0000","last_became_peered":"2024-03-30T00:46:15.305249+0000","last_unstale":"2024-03-30T00:46:16.290615+0000","last_undegraded":"2024-03-30T00:46:16.290615+0000","last_fullsized":"2024-03-30T00:46:16.290615+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:27:04.442085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.291025+0000","last_change":"2024-03-30T00:46:14.892456+0000","last_active":"2024-03-30T00:46:16.291025+0000","last_peered":"2024-03-30T00:46:16.291025+0000","last_clean":"2024-03-30T00:46:16.291025+0000","last_became_active":"2024-03-30T00:46:14.892131+0000","last_became_peered":"2024-03-30T00:46:14.892131+0000","last_unstale":"2024-03-30T00:46:16.291025+0000","last_undegraded":"2024-03-30T00:46:16.291025+0000","last_fullsized":"2024-03-30T00:46:16.291025+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:05.119504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.278565+0000","last_change":"2024-03-30T00:46:15.296122+0000","last_active":"2024-03-30T00:46:16.278565+0000","last_peered":"2024-03-30T00:46:16.278565+0000","last_clean":"2024-03-30T00:46:16.278565+0000","last_became_active":"2024-03-30T00:46:15.295861+0000","last_became_peered":"2024-03-30T00:46:15.295861+0000","last_unstale":"2024-03-30T00:46:16.278565+0000","last_undegraded":"2024-03-30T00:46:16.278565+0000","last_fullsized":"2024-03-30T00:46:16.278565+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:56:32.560264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289592+0000","last_change":"2024-03-30T00:46:14.894642+0000","last_active":"2024-03-30T00:46:16.289592+0000","last_peered":"2024-03-30T00:46:16.289592+0000","last_clean":"2024-03-30T00:46:16.289592+0000","last_became_active":"2024-03-30T00:46:14.894391+0000","last_became_peered":"2024-03-30T00:46:14.894391+0000","last_unstale":"2024-03-30T00:46:16.289592+0000","last_undegraded":"2024-03-30T00:46:16.289592+0000","last_fullsized":"2024-03-30T00:46:16.289592+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:51.291654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289385+0000","last_change":"2024-03-30T00:46:15.306548+0000","last_active":"2024-03-30T00:46:16.289385+0000","last_peered":"2024-03-30T00:46:16.289385+0000","last_clean":"2024-03-30T00:46:16.289385+0000","last_became_active":"2024-03-30T00:46:15.306113+0000","last_became_peered":"2024-03-30T00:46:15.306113+0000","last_unstale":"2024-03-30T00:46:16.289385+0000","last_undegraded":"2024-03-30T00:46:16.289385+0000","last_fullsized":"2024-03-30T00:46:16.289385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:05:46.343109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290357+0000","last_change":"2024-03-30T00:46:14.893727+0000","last_active":"2024-03-30T00:46:16.290357+0000","last_peered":"2024-03-30T00:46:16.290357+0000","last_clean":"2024-03-30T00:46:16.290357+0000","last_became_active":"2024-03-30T00:46:14.893417+0000","last_became_peered":"2024-03-30T00:46:14.893417+0000","last_unstale":"2024-03-30T00:46:16.290357+0000","last_undegraded":"2024-03-30T00:46:16.290357+0000","last_fullsized":"2024-03-30T00:46:16.290357+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:25.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290683+0000","last_change":"2024-03-30T00:46:15.302710+0000","last_active":"2024-03-30T00:46:16.290683+0000","last_peered":"2024-03-30T00:46:16.290683+0000","last_clean":"2024-03-30T00:46:16.290683+0000","last_became_active":"2024-03-30T00:46:15.302449+0000","last_became_peered":"2024-03-30T00:46:15.302449+0000","last_unstale":"2024-03-30T00:46:16.290683+0000","last_undegraded":"2024-03-30T00:46:16.290683+0000","last_fullsized":"2024-03-30T00:46:16.290683+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:48:37.321643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290695+0000","last_change":"2024-03-30T00:46:14.890025+0000","last_active":"2024-03-30T00:46:16.290695+0000","last_peered":"2024-03-30T00:46:16.290695+0000","last_clean":"2024-03-30T00:46:16.290695+0000","last_became_active":"2024-03-30T00:46:14.889762+0000","last_became_peered":"2024-03-30T00:46:14.889762+0000","last_unstale":"2024-03-30T00:46:16.290695+0000","last_undegraded":"2024-03-30T00:46:16.290695+0000","last_fullsized":"2024-03-30T00:46:16.290695+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:39.810863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.289583+0000","last_change":"2024-03-30T00:46:15.296766+0000","last_active":"2024-03-30T00:46:16.289583+0000","last_peered":"2024-03-30T00:46:16.289583+0000","last_clean":"2024-03-30T00:46:16.289583+0000","last_became_active":"2024-03-30T00:46:15.296497+0000","last_became_peered":"2024-03-30T00:46:15.296497+0000","last_unstale":"2024-03-30T00:46:16.289583+0000","last_undegraded":"2024-03-30T00:46:16.289583+0000","last_fullsized":"2024-03-30T00:46:16.289583+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:57.757325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281588+0000","last_change":"2024-03-30T00:46:14.886828+0000","last_active":"2024-03-30T00:46:16.281588+0000","last_peered":"2024-03-30T00:46:16.281588+0000","last_clean":"2024-03-30T00:46:16.281588+0000","last_became_active":"2024-03-30T00:46:14.886576+0000","last_became_peered":"2024-03-30T00:46:14.886576+0000","last_unstale":"2024-03-30T00:46:16.281588+0000","last_undegraded":"2024-03-30T00:46:16.281588+0000","last_fullsized":"2024-03-30T00:46:16.281588+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:30.462631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.281448+0000","last_change":"2024-03-30T00:46:15.309687+0000","last_active":"2024-03-30T00:46:16.281448+0000","last_peered":"2024-03-30T00:46:16.281448+0000","last_clean":"2024-03-30T00:46:16.281448+0000","last_became_active":"2024-03-30T00:46:15.309436+0000","last_became_peered":"2024-03-30T00:46:15.309436+0000","last_unstale":"2024-03-30T00:46:16.281448+0000","last_undegraded":"2024-03-30T00:46:16.281448+0000","last_fullsized":"2024-03-30T00:46:16.281448+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:14.864196+0000","last_clean_scrub_stamp":"2024-03-30T00:46:14.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:12:17.417262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-30T00:46:16.290347+0000","last_change":"2024-03-30T00:46:14.876213+0000","last_active":"2024-03-30T00:46:16.290347+0000","last_peered":"2024-03-30T00:46:16.290347+0000","last_clean":"2024-03-30T00:46:16.290347+0000","last_became_active":"2024-03-30T00:46:14.875930+0000","last_became_peered":"2024-03-30T00:46:14.875930+0000","last_unstale":"2024-03-30T00:46:16.290347+0000","last_undegraded":"2024-03-30T00:46:16.290347+0000","last_fullsized":"2024-03-30T00:46:16.290347+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:46:13.861949+0000","last_clean_scrub_stamp":"2024-03-30T00:46:13.861949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:19:51.018234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":241,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-30T00:46:15.281868+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:46:15.281868+0000","last_peered":"2024-03-30T00:46:15.281868+0000","last_clean":"2024-03-30T00:46:15.281868+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:46:15.281868+0000","last_undegraded":"2024-03-30T00:46:15.281868+0000","last_fullsized":"2024-03-30T00:46:15.281868+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1773568,"data_stored":1124724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77000000000000002,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65700000000000003,"15min":0.64100000000000001},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86899999999999999,"5min":1.149,"15min":1.149},"last":0.752}]},{"osd":1,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67200000000000004,"15min":0.63600000000000001},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.436},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95199999999999996,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.749,"15min":0.77800000000000002},"min":{"1min":0.52100000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92900000000000005,"5min":1.2450000000000001,"15min":1.6020000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73299999999999998,"15min":0.77700000000000002},"min":{"1min":0.59599999999999997,"5min":0.5,"15min":0.5},"max":{"1min":0.89400000000000002,"5min":1.026,"15min":1.6619999999999999},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.78100000000000003},"min":{"1min":0.54300000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88400000000000001,"5min":1.2270000000000001,"15min":2.0150000000000001},"last":0.62},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78300000000000003,"15min":0.81499999999999995},"min":{"1min":0.61399999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.98299999999999998,"5min":1.0429999999999999,"15min":1.9490000000000001},"last":0.73299999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.626},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.82899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39500000000000002},"max":{"1min":0.80600000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78100000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94199999999999995,"5min":1.121,"15min":1.121},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.75900000000000001},"min":{"1min":0.43099999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":1.0049999999999999,"15min":1.026},"last":0.68500000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71099999999999997,"15min":0.72799999999999998},"min":{"1min":0.50600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.94599999999999995,"5min":1.214,"15min":1.214},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95499999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.83999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1191936,"data_stored":536468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.68000000000000005,"15min":0.86699999999999999},"min":{"1min":0.52700000000000002,"5min":0.39100000000000001,"15min":0.35399999999999998},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":4.0490000000000004},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.52700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.47},"last":0.72499999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.66700000000000004,"15min":0.875},"min":{"1min":0.50600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93400000000000005,"5min":1.0249999999999999,"15min":4.3170000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.70299999999999996,"15min":0.94299999999999995},"min":{"1min":0.57699999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95499999999999996,"5min":1.018,"15min":4.4160000000000004},"last":0.69199999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.71399999999999997,"15min":0.93300000000000005},"min":{"1min":0.65500000000000003,"5min":0.436,"15min":0.436},"max":{"1min":1.24,"5min":1.24,"15min":4.375},"last":0.751},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70199999999999996,"15min":0.93400000000000005},"min":{"1min":0.504,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":4.2960000000000003},"last":0.67400000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72699999999999998,"15min":0.95099999999999996},"min":{"1min":0.56899999999999995,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":4.3399999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.94199999999999995},"min":{"1min":0.55500000000000005,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.258,"5min":1.258,"15min":4.6020000000000003},"last":0.70999999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64100000000000001,"15min":0.66300000000000003},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88100000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.752},"min":{"1min":0.55600000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":1.036,"15min":1.036},"last":0.495}]},{"osd":5,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.622,"15min":0.67800000000000005},"min":{"1min":0.42199999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82899999999999996,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67900000000000005,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95499999999999996,"5min":1.101,"15min":1.101},"last":0.53800000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68300000000000005,"15min":0.71099999999999997},"min":{"1min":0.497,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.89800000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86199999999999999,"5min":1.036,"15min":1.036},"last":0.65800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1171456,"data_stored":534300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.55400000000000005,"5min":0.376,"15min":0.376},"max":{"1min":0.97599999999999998,"5min":0.97899999999999998,"15min":1.095},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.747,"5min":0.67400000000000004,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.7569999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98199999999999998,"5min":0.999,"15min":1.7330000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.72399999999999998},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95499999999999996,"5min":0.96299999999999997,"15min":1.3260000000000001},"last":0.73899999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71099999999999997,"15min":0.72099999999999997},"min":{"1min":0.57299999999999995,"5min":0.45000000000000001,"15min":0.44600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.68100000000000005,"15min":0.71399999999999997},"min":{"1min":0.505,"5min":0.442,"15min":0.442},"max":{"1min":0.95999999999999996,"5min":0.97799999999999998,"15min":1.2789999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70799999999999996,"15min":0.78400000000000003},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.0169999999999999,"5min":1.0269999999999999,"15min":1.7},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.72299999999999998,"15min":0.75700000000000001},"min":{"1min":0.59799999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.181},"last":0.80400000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57299999999999995,"15min":0.623},"min":{"1min":0.438,"5min":0.309,"15min":0.309},"max":{"1min":0.82499999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69899999999999995,"15min":0.76300000000000001},"min":{"1min":0.51200000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97399999999999998,"5min":1.0509999999999999,"15min":1.615},"last":0.77100000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.629,"15min":0.66800000000000004},"min":{"1min":0.47299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.97399999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63600000000000001,"15min":0.70099999999999996},"min":{"1min":0.48899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":1.0700000000000001,"15min":1.569},"last":0.71599999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69599999999999995,"15min":0.74199999999999999},"min":{"1min":0.47099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.88600000000000001,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.82499999999999996},"min":{"1min":0.58799999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99199999999999999,"5min":1.032,"15min":1.383},"last":0.86399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1179648,"data_stored":534460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.46200000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.52700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.85699999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.57199999999999995,"5min":0.45500000000000002,"15min":0.45300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.2909999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.47599999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93400000000000005,"5min":1.0169999999999999,"15min":1.1060000000000001},"last":0.77200000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.51400000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.86399999999999999,"5min":0.93500000000000005,"15min":1.0600000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.75600000000000001},"min":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.46700000000000003},"max":{"1min":0.97099999999999997,"5min":1.2190000000000001,"15min":1.228},"last":0.86499999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70299999999999996,"15min":0.70699999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85999999999999999,"5min":1.196,"15min":1.27},"last":0.746},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.77400000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1,"5min":1,"15min":1.1739999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61499999999999999,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89400000000000002,"5min":0.92100000000000004,"15min":1.329},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64600000000000002,"15min":0.73299999999999998},"min":{"1min":0.34799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92300000000000004,"5min":0.93500000000000005,"15min":1.3819999999999999},"last":0.80000000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.69099999999999995},"min":{"1min":0.38800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84099999999999997,"5min":0.98899999999999999,"15min":1.2130000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":0.74099999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82999999999999996,"5min":0.96199999999999997,"15min":1.3220000000000001},"last":0.81899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.72899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81699999999999995,"5min":0.999,"15min":1.6910000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72799999999999998,"15min":0.81200000000000006},"min":{"1min":0.49299999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91500000000000004,"5min":1.069,"15min":1.5009999999999999},"last":0.92800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1171456,"data_stored":534300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45000000000000001,"15min":0.46700000000000003},"min":{"1min":0.34300000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63700000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.53600000000000003},"min":{"1min":0.373,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70399999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38900000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.48899999999999999,"15min":0.51300000000000001},"min":{"1min":0.41799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.69299999999999995,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.496},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52200000000000002,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73399999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.436}]},{"osd":2,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.495,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.86499999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55700000000000005,"15min":0.58599999999999997},"min":{"1min":0.47299999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.81299999999999994},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.747,"15min":0.77100000000000002},"min":{"1min":0.60499999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.02,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76300000000000001,"15min":0.78300000000000003},"min":{"1min":0.50800000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0329999999999999,"5min":1.0760000000000001,"15min":1.0860000000000001},"last":0.64500000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.56699999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0369999999999999,"5min":1.1060000000000001,"15min":1.125},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.80700000000000005},"min":{"1min":0.68200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.999,"5min":1.0629999999999999,"15min":1.2390000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.77900000000000003,"15min":0.81599999999999995},"min":{"1min":0.65300000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.995,"5min":1.1699999999999999,"15min":1.226},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78000000000000003,"15min":0.79400000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.004,"5min":1.167,"15min":1.167},"last":0.78300000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80800000000000005,"15min":0.85399999999999998},"min":{"1min":0.63500000000000001,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0389999999999999,"5min":1.1799999999999999,"15min":1.202},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.82199999999999995,"15min":0.88800000000000001},"min":{"1min":0.68300000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.0449999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.1579999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1171456,"data_stored":534300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.443},"min":{"1min":0.34200000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67800000000000005,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49199999999999999,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84099999999999997,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.505}]},{"osd":1,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.38400000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81299999999999994,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.371},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.47099999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.752,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.57999999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49399999999999999,"15min":0.502},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.40000000000000002},"max":{"1min":0.88800000000000001,"5min":1.907,"15min":1.907},"last":0.38900000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80900000000000005,"15min":0.81699999999999995},"min":{"1min":0.65100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1919999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.80700000000000005,"15min":0.84899999999999998},"min":{"1min":0.60899999999999999,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2110000000000001,"5min":1.972,"15min":1.972},"last":0.70799999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.80500000000000005,"15min":0.83599999999999997},"min":{"1min":0.63100000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.2549999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81499999999999995,"15min":0.82599999999999996},"min":{"1min":0.63,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.286,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.82999999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.65400000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.177,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.82799999999999996,"15min":0.82599999999999996},"min":{"1min":0.67800000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.242,"5min":1.867,"15min":1.867},"last":0.74399999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81799999999999995,"15min":0.86799999999999999},"min":{"1min":0.68300000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.1140000000000001,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.83699999999999997,"15min":0.86099999999999999},"min":{"1min":0.64700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.365,"5min":2.851,"15min":2.851},"last":0.80100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1175552,"data_stored":534370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.29999999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.70199999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.36599999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47699999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.439,"15min":0.46200000000000002},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.52500000000000002},"min":{"1min":0.42899999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81799999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84099999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.42599999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53000000000000003,"15min":0.48399999999999999},"min":{"1min":0.45100000000000001,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.52800000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.745,"15min":0.72099999999999997},"min":{"1min":0.51400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.77500000000000002,"15min":0.751},"min":{"1min":0.66900000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.079,"5min":1.2,"15min":1.2},"last":0.80300000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.61699999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.109,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.627},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77700000000000002,"15min":0.76600000000000001},"min":{"1min":0.56599999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.82199999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.69599999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.874},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.78800000000000003,"15min":0.77600000000000002},"min":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0189999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.80000000000000004,"15min":0.79900000000000004},"min":{"1min":0.59399999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2050000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.80700000000000005,"15min":0.78400000000000003},"min":{"1min":0.64800000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1180000000000001,"5min":1.236,"15min":1.236},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1789952,"data_stored":1126938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42799999999999999,"15min":0.46800000000000003},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.625,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.435},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.35699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.502}]},{"osd":2,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.51900000000000002},"min":{"1min":0.38300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82599999999999996,"5min":1.02,"15min":1.02},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.34300000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77900000000000003,"5min":1.006,"15min":1.006},"last":0.46700000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55100000000000005,"15min":0.58999999999999997},"min":{"1min":0.45500000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88900000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55000000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.50600000000000001,"5min":0.504,"15min":0.47899999999999998},"max":{"1min":1.1140000000000001,"5min":1.234,"15min":1.5169999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82799999999999996,"15min":0.81000000000000005},"min":{"1min":0.56000000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0329999999999999,"5min":1.252,"15min":1.252},"last":0.82899999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77900000000000003,"15min":0.76600000000000001},"min":{"1min":0.52200000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96099999999999997,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78600000000000003,"15min":0.78000000000000003},"min":{"1min":0.625,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0780000000000001,"5min":1.2609999999999999,"15min":1.462},"last":0.70299999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.54500000000000004},"max":{"1min":0.98899999999999999,"5min":1.196,"15min":1.5920000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.57799999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0429999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.65800000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.095,"5min":1.351,"15min":1.3600000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.58399999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1759999999999999,"5min":1.2070000000000001,"15min":1.556},"last":0.77100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:46:19.782 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T00:46:19.782 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-30T00:46:19.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.958+0000 7f9d88600640 1 Processor -- start 2024-03-30T00:46:19.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.958+0000 7f9d88600640 1 -- start start 2024-03-30T00:46:19.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.958+0000 7f9d88600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d8015a770 0x7f9d800c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d88600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9d800b1db0 0x7f9d800cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d88600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d800ca130 con 0x7f9d800ce780 2024-03-30T00:46:19.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d88600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d800cb120 con 0x7f9d800b1db0 2024-03-30T00:46:19.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d88600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8015ab50 con 0x7f9d8015a770 2024-03-30T00:46:19.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d85200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9d800b1db0 0x7f9d800cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d85c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d8015a770 0x7f9d800c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d85c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d8015a770 0x7f9d800c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:47792/0 (socket says 172.21.15.66:47792) 2024-03-30T00:46:19.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d85c00640 1 -- 172.21.15.66:0/3923368355 learned_addr learned my addr 172.21.15.66:0/3923368355 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:19.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d85200640 1 -- 172.21.15.66:0/3923368355 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d8015a770 msgr2=0x7f9d800c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:19.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d85200640 1 --2- 172.21.15.66:0/3923368355 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d8015a770 0x7f9d800c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d85200640 1 -- 172.21.15.66:0/3923368355 >> v1:172.21.15.156:6789/0 conn(0x7f9d800ce780 legacy=0x7f9d800710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.962+0000 7f9d85200640 1 -- 172.21.15.66:0/3923368355 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d800ca400 con 0x7f9d800b1db0 2024-03-30T00:46:19.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d85200640 1 --2- 172.21.15.66:0/3923368355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d800b1db0 0x7f9d800cb540 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0097c0 tx=0x7f9d6c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7a566b0b95f0bbb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:19.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d7fe00640 1 -- 172.21.15.66:0/3923368355 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0ad680 con 0x7f9d800b1db0 2024-03-30T00:46:19.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d7fe00640 1 -- 172.21.15.66:0/3923368355 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c0ad7e0 con 0x7f9d800b1db0 2024-03-30T00:46:19.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d7fe00640 1 -- 172.21.15.66:0/3923368355 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0adb70 con 0x7f9d800b1db0 2024-03-30T00:46:19.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 -- 172.21.15.66:0/3923368355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d800b1db0 msgr2=0x7f9d800cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 --2- 172.21.15.66:0/3923368355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d800b1db0 0x7f9d800cb540 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0097c0 tx=0x7f9d6c01c470 comp rx=0 tx=0).stop 2024-03-30T00:46:19.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 -- 172.21.15.66:0/3923368355 shutdown_connections 2024-03-30T00:46:19.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 --2- 172.21.15.66:0/3923368355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d800b1db0 0x7f9d800cb540 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 --2- 172.21.15.66:0/3923368355 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d8015a770 0x7f9d800c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 -- 172.21.15.66:0/3923368355 >> 172.21.15.66:0/3923368355 conn(0x7f9d8006a190 msgr2=0x7f9d8006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:19.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 -- 172.21.15.66:0/3923368355 shutdown_connections 2024-03-30T00:46:19.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 -- 172.21.15.66:0/3923368355 wait complete. 2024-03-30T00:46:19.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 Processor -- start 2024-03-30T00:46:19.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 -- start start 2024-03-30T00:46:19.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d800b1db0 0x7f9d801622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800ce780 0x7f9d800c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d8015a770 0x7f9d80153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800c2d60 con 0x7f9d800ce780 2024-03-30T00:46:19.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800b0980 con 0x7f9d800b1db0 2024-03-30T00:46:19.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d88600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800c1e90 con 0x7f9d8015a770 2024-03-30T00:46:19.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d85200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800ce780 0x7f9d800c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d85200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800ce780 0x7f9d800c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54306/0 (socket says 172.21.15.66:54306) 2024-03-30T00:46:19.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d85200640 1 -- 172.21.15.66:0/4181475396 learned_addr learned my addr 172.21.15.66:0/4181475396 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:19.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d85c00640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d800b1db0 0x7f9d801622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.966+0000 7f9d86600640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d8015a770 0x7f9d80153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d85200640 1 -- 172.21.15.66:0/4181475396 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d8015a770 msgr2=0x7f9d80153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d85200640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d8015a770 0x7f9d80153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d85200640 1 -- 172.21.15.66:0/4181475396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d800b1db0 msgr2=0x7f9d801622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:19.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d85200640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d800b1db0 0x7f9d801622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:19.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d85200640 1 -- 172.21.15.66:0/4181475396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c009420 con 0x7f9d800ce780 2024-03-30T00:46:19.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d85200640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800ce780 0x7f9d800c44d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c01a0a0 tx=0x7f9d6c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:19.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d7d600640 1 -- 172.21.15.66:0/4181475396 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0add60 con 0x7f9d800ce780 2024-03-30T00:46:19.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d7d600640 1 -- 172.21.15.66:0/4181475396 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c0b8450 con 0x7f9d800ce780 2024-03-30T00:46:19.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d7d600640 1 -- 172.21.15.66:0/4181475396 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0b8800 con 0x7f9d800ce780 2024-03-30T00:46:19.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d88600640 1 -- 172.21.15.66:0/4181475396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d800cf120 con 0x7f9d800ce780 2024-03-30T00:46:19.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d88600640 1 -- 172.21.15.66:0/4181475396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d800c23c0 con 0x7f9d800ce780 2024-03-30T00:46:19.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d88600640 1 -- 172.21.15.66:0/4181475396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d800c2ea0 con 0x7f9d800ce780 2024-03-30T00:46:19.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d7d600640 1 -- 172.21.15.66:0/4181475396 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9d6c0b4070 con 0x7f9d800ce780 2024-03-30T00:46:19.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d7d600640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d60066690 0x7f9d60068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:19.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.970+0000 7f9d7d600640 1 -- 172.21.15.66:0/4181475396 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9d6c11e860 con 0x7f9d800ce780 2024-03-30T00:46:19.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.974+0000 7f9d85c00640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d60066690 0x7f9d60068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:19.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.974+0000 7f9d85c00640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d60066690 0x7f9d60068b50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009ab0 tx=0x7f9d74009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:19.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:19.974+0000 7f9d7d600640 1 -- 172.21.15.66:0/4181475396 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9d800c2ea0 con 0x7f9d800ce780 2024-03-30T00:46:20.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.194+0000 7f9d88600640 1 -- 172.21.15.66:0/4181475396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9d800c2ea0 con 0x7f9d800ce780 2024-03-30T00:46:20.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.198+0000 7f9d7d600640 1 -- 172.21.15.66:0/4181475396 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9d800c2ea0 con 0x7f9d800ce780 2024-03-30T00:46:20.209 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-03-30T00:46:20.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 -- 172.21.15.66:0/4181475396 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d60066690 msgr2=0x7f9d60068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:20.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d60066690 0x7f9d60068b50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009ab0 tx=0x7f9d74009240 comp rx=0 tx=0).stop 2024-03-30T00:46:20.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 -- 172.21.15.66:0/4181475396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800ce780 msgr2=0x7f9d800c44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:20.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800ce780 0x7f9d800c44d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c01a0a0 tx=0x7f9d6c01acb0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 -- 172.21.15.66:0/4181475396 shutdown_connections 2024-03-30T00:46:20.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d8015a770 0x7f9d80153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d60066690 0x7f9d60068b50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d800ce780 0x7f9d800c44d0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 --2- 172.21.15.66:0/4181475396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d800b1db0 0x7f9d801622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 -- 172.21.15.66:0/4181475396 >> 172.21.15.66:0/4181475396 conn(0x7f9d8006a190 msgr2=0x7f9d800c8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:20.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 -- 172.21.15.66:0/4181475396 shutdown_connections 2024-03-30T00:46:20.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.202+0000 7f9d88600640 1 -- 172.21.15.66:0/4181475396 wait complete. 2024-03-30T00:46:20.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:46:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.362+0000 7f6492a00640 1 Processor -- start 2024-03-30T00:46:20.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.362+0000 7f6492a00640 1 -- start start 2024-03-30T00:46:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6492a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f648c0cec30 0x7f648c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:20.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6492a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f648c0cde70 0x7f648c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6492a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648c15a6e0 con 0x7f648c0cda90 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6492a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f648c0b1b80 con 0x7f648c0cec30 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6492a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f648c0cd3e0 con 0x7f648c0cde70 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6483e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f648c0cec30 0x7f648c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6483e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f648c0cec30 0x7f648c159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54316/0 (socket says 172.21.15.66:54316) 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6483e00640 1 -- 172.21.15.66:0/1044612805 learned_addr learned my addr 172.21.15.66:0/1044612805 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6490a00640 1 --2- 172.21.15.66:0/1044612805 >> v2:172.21.15.156:3300/0 conn(0x7f648c0cde70 0x7f648c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6483e00640 1 -- 172.21.15.66:0/1044612805 >> v2:172.21.15.156:3300/0 conn(0x7f648c0cde70 msgr2=0x7f648c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6483e00640 1 --2- 172.21.15.66:0/1044612805 >> v2:172.21.15.156:3300/0 conn(0x7f648c0cde70 0x7f648c15a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6483e00640 1 -- 172.21.15.66:0/1044612805 >> v1:172.21.15.156:6789/0 conn(0x7f648c0cda90 legacy=0x7f648c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6483e00640 1 -- 172.21.15.66:0/1044612805 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c15a990 con 0x7f648c0cec30 2024-03-30T00:46:20.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.366+0000 7f6490a00640 1 --2- 172.21.15.66:0/1044612805 >> v2:172.21.15.156:3300/0 conn(0x7f648c0cde70 0x7f648c15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:20.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6483e00640 1 --2- 172.21.15.66:0/1044612805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f648c0cec30 0x7f648c159c60 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f648c0adcf0 tx=0x7f647401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66ddc81139b1c51 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:20.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f648b400640 1 -- 172.21.15.66:0/1044612805 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64740aa5f0 con 0x7f648c0cec30 2024-03-30T00:46:20.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f648b400640 1 -- 172.21.15.66:0/1044612805 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64740aa750 con 0x7f648c0cec30 2024-03-30T00:46:20.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f648b400640 1 -- 172.21.15.66:0/1044612805 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64740aaae0 con 0x7f648c0cec30 2024-03-30T00:46:20.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 -- 172.21.15.66:0/1044612805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f648c0cec30 msgr2=0x7f648c159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:20.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 --2- 172.21.15.66:0/1044612805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f648c0cec30 0x7f648c159c60 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f648c0adcf0 tx=0x7f647401c490 comp rx=0 tx=0).stop 2024-03-30T00:46:20.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 -- 172.21.15.66:0/1044612805 shutdown_connections 2024-03-30T00:46:20.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 --2- 172.21.15.66:0/1044612805 >> v2:172.21.15.156:3300/0 conn(0x7f648c0cde70 0x7f648c15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 --2- 172.21.15.66:0/1044612805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f648c0cec30 0x7f648c159c60 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 -- 172.21.15.66:0/1044612805 >> 172.21.15.66:0/1044612805 conn(0x7f648c069f60 msgr2=0x7f648c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:20.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 -- 172.21.15.66:0/1044612805 shutdown_connections 2024-03-30T00:46:20.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 -- 172.21.15.66:0/1044612805 wait complete. 2024-03-30T00:46:20.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 Processor -- start 2024-03-30T00:46:20.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 -- start start 2024-03-30T00:46:20.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f648c0cda90 0x7f648c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:20.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f648c0cde70 0x7f648c158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:20.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f648c0cec30 0x7f648c15b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:20.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648c0c1e20 con 0x7f648c0cde70 2024-03-30T00:46:20.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648c164130 con 0x7f648c0cec30 2024-03-30T00:46:20.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6492a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648c0c20c0 con 0x7f648c0cda90 2024-03-30T00:46:20.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6483e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f648c0cde70 0x7f648c158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:20.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6483e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f648c0cde70 0x7f648c158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54322/0 (socket says 172.21.15.66:54322) 2024-03-30T00:46:20.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.370+0000 7f6483e00640 1 -- 172.21.15.66:0/3296450099 learned_addr learned my addr 172.21.15.66:0/3296450099 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:20.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6483e00640 1 -- 172.21.15.66:0/3296450099 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f648c0cda90 msgr2=0x7f648c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:20.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6483e00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f648c0cda90 0x7f648c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6483e00640 1 -- 172.21.15.66:0/3296450099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f648c0cec30 msgr2=0x7f648c15b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:20.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6483e00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f648c0cec30 0x7f648c15b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6483e00640 1 -- 172.21.15.66:0/3296450099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64740094d0 con 0x7f648c0cde70 2024-03-30T00:46:20.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6483e00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f648c0cde70 0x7f648c158e20 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f64740190a0 tx=0x7f647401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:20.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6488c00640 1 -- 172.21.15.66:0/3296450099 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64740b3800 con 0x7f648c0cde70 2024-03-30T00:46:20.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6488c00640 1 -- 172.21.15.66:0/3296450099 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64740b3980 con 0x7f648c0cde70 2024-03-30T00:46:20.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6492a00640 1 -- 172.21.15.66:0/3296450099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648c162fe0 con 0x7f648c0cde70 2024-03-30T00:46:20.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6488c00640 1 -- 172.21.15.66:0/3296450099 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64740b3d30 con 0x7f648c0cde70 2024-03-30T00:46:20.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6492a00640 1 -- 172.21.15.66:0/3296450099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648c163560 con 0x7f648c0cde70 2024-03-30T00:46:20.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6492a00640 1 -- 172.21.15.66:0/3296450099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648c066c30 con 0x7f648c0cde70 2024-03-30T00:46:20.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6488c00640 1 -- 172.21.15.66:0/3296450099 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f64740b2380 con 0x7f648c0cde70 2024-03-30T00:46:20.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6488c00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6468066690 0x7f6468068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:20.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f6488c00640 1 -- 172.21.15.66:0/3296450099 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f64740a7070 con 0x7f648c0cde70 2024-03-30T00:46:20.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.374+0000 7f648be00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6468066690 0x7f6468068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:20.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.378+0000 7f648be00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6468066690 0x7f6468068b50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f647c000f70 tx=0x7f647c0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:20.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.378+0000 7f6488c00640 1 -- 172.21.15.66:0/3296450099 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f64740ebb30 con 0x7f648c0cde70 2024-03-30T00:46:20.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.574+0000 7f6492a00640 1 -- 172.21.15.66:0/3296450099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f648c0b3170 con 0x7f648c0cde70 2024-03-30T00:46:20.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.574+0000 7f6488c00640 1 -- 172.21.15.66:0/3296450099 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f64740ef150 con 0x7f648c0cde70 2024-03-30T00:46:20.583 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:20.584 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:46:18.278734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6836/1147664160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1147664160},{"type":"v1","addr":"172.21.15.66:6836","nonce":1147664160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:46:20.584 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 58 2024-03-30T00:46:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 -- 172.21.15.66:0/3296450099 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6468066690 msgr2=0x7f6468068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6468066690 0x7f6468068b50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f647c000f70 tx=0x7f647c0079c0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 -- 172.21.15.66:0/3296450099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f648c0cde70 msgr2=0x7f648c158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f648c0cde70 0x7f648c158e20 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f64740190a0 tx=0x7f647401acd0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 -- 172.21.15.66:0/3296450099 shutdown_connections 2024-03-30T00:46:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f648c0cec30 0x7f648c15b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6468066690 0x7f6468068b50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f648c0cde70 0x7f648c158e20 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 --2- 172.21.15.66:0/3296450099 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f648c0cda90 0x7f648c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:20.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 -- 172.21.15.66:0/3296450099 >> 172.21.15.66:0/3296450099 conn(0x7f648c069f60 msgr2=0x7f648c15da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:20.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 -- 172.21.15.66:0/3296450099 shutdown_connections 2024-03-30T00:46:20.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:20.578+0000 7f6492a00640 1 -- 172.21.15.66:0/3296450099 wait complete. 2024-03-30T00:46:21.007 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:46:15.271965+0000', 'modified': '2024-03-30T00:46:18.278734+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7692}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7692': {'gid': 7692, 'name': 'a', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6836/1147664160', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 1147664160}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 1147664160}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:46:21.007 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:46:21.008 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-30T00:46:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe517e00640 1 Processor -- start 2024-03-30T00:46:21.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe517e00640 1 -- start start 2024-03-30T00:46:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe517e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe51815a720 0x7fe5180ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe517e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe5180b3170 0x7fe5180b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe517e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5180cb5b0 con 0x7fe5180d07c0 2024-03-30T00:46:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe517e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5180cfb00 con 0x7fe5180b3170 2024-03-30T00:46:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe517e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51815ab00 con 0x7fe51815a720 2024-03-30T00:46:21.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe50fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe5180b3170 0x7fe5180b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe50fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe5180b3170 0x7fe5180b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54338/0 (socket says 172.21.15.66:54338) 2024-03-30T00:46:21.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe50fe00640 1 -- 172.21.15.66:0/3655894074 learned_addr learned my addr 172.21.15.66:0/3655894074 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:21.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe516a00640 1 --2- 172.21.15.66:0/3655894074 >> v2:172.21.15.156:3300/0 conn(0x7fe51815a720 0x7fe5180ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe50fe00640 1 -- 172.21.15.66:0/3655894074 >> v2:172.21.15.156:3300/0 conn(0x7fe51815a720 msgr2=0x7fe5180ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe50fe00640 1 --2- 172.21.15.66:0/3655894074 >> v2:172.21.15.156:3300/0 conn(0x7fe51815a720 0x7fe5180ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe50fe00640 1 -- 172.21.15.66:0/3655894074 >> v1:172.21.15.156:6789/0 conn(0x7fe5180d07c0 legacy=0x7fe5180ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.142+0000 7fe50fe00640 1 -- 172.21.15.66:0/3655894074 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5180cb2f0 con 0x7fe5180b3170 2024-03-30T00:46:21.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe50fe00640 1 --2- 172.21.15.66:0/3655894074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5180b3170 0x7fe5180b3570 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe5180b27e0 tx=0x7fe50001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29383623ae236d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:21.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe516000640 1 -- 172.21.15.66:0/3655894074 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5000aa650 con 0x7fe5180b3170 2024-03-30T00:46:21.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe516000640 1 -- 172.21.15.66:0/3655894074 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5000aa7b0 con 0x7fe5180b3170 2024-03-30T00:46:21.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe516000640 1 -- 172.21.15.66:0/3655894074 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5000aab40 con 0x7fe5180b3170 2024-03-30T00:46:21.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe517e00640 1 -- 172.21.15.66:0/3655894074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5180b3170 msgr2=0x7fe5180b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe517e00640 1 --2- 172.21.15.66:0/3655894074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5180b3170 0x7fe5180b3570 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe5180b27e0 tx=0x7fe50001c4f0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe517e00640 1 -- 172.21.15.66:0/3655894074 shutdown_connections 2024-03-30T00:46:21.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe517e00640 1 --2- 172.21.15.66:0/3655894074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5180b3170 0x7fe5180b3570 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe517e00640 1 --2- 172.21.15.66:0/3655894074 >> v2:172.21.15.156:3300/0 conn(0x7fe51815a720 0x7fe5180ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe517e00640 1 -- 172.21.15.66:0/3655894074 >> 172.21.15.66:0/3655894074 conn(0x7fe518069d90 msgr2=0x7fe51806a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:21.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe517e00640 1 -- 172.21.15.66:0/3655894074 shutdown_connections 2024-03-30T00:46:21.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.146+0000 7fe517e00640 1 -- 172.21.15.66:0/3655894074 wait complete. 2024-03-30T00:46:21.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517e00640 1 Processor -- start 2024-03-30T00:46:21.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517e00640 1 -- start start 2024-03-30T00:46:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5180b3170 0x7fe51806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5180d07c0 0x7fe5180b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe51815a720 0x7fe518152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51806bf00 con 0x7fe5180b3170 2024-03-30T00:46:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51806c1e0 con 0x7fe5180d07c0 2024-03-30T00:46:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51806c4c0 con 0x7fe51815a720 2024-03-30T00:46:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe516a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5180b3170 0x7fe51806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe516a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5180b3170 0x7fe51806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54342/0 (socket says 172.21.15.66:54342) 2024-03-30T00:46:21.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe516a00640 1 -- 172.21.15.66:0/1054436587 learned_addr learned my addr 172.21.15.66:0/1054436587 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe50fe00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5180d07c0 0x7fe5180b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517400640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe51815a720 0x7fe518152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe516a00640 1 -- 172.21.15.66:0/1054436587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe51815a720 msgr2=0x7fe518152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe516a00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe51815a720 0x7fe518152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe516a00640 1 -- 172.21.15.66:0/1054436587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5180d07c0 msgr2=0x7fe5180b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe516a00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5180d07c0 0x7fe5180b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe516a00640 1 -- 172.21.15.66:0/1054436587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5000094d0 con 0x7fe5180b3170 2024-03-30T00:46:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517400640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe51815a720 0x7fe518152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:21.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe516a00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5180b3170 0x7fe51806b8b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe50800e870 tx=0x7fe50800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe50ea00640 1 -- 172.21.15.66:0/1054436587 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508009510 con 0x7fe5180b3170 2024-03-30T00:46:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe50ea00640 1 -- 172.21.15.66:0/1054436587 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50800eec0 con 0x7fe5180b3170 2024-03-30T00:46:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe50ea00640 1 -- 172.21.15.66:0/1054436587 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50800cb40 con 0x7fe5180b3170 2024-03-30T00:46:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517e00640 1 -- 172.21.15.66:0/1054436587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51806c7a0 con 0x7fe5180b3170 2024-03-30T00:46:21.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.150+0000 7fe517e00640 1 -- 172.21.15.66:0/1054436587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5180b1170 con 0x7fe5180b3170 2024-03-30T00:46:21.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe50ea00640 1 -- 172.21.15.66:0/1054436587 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe50800cca0 con 0x7fe5180b3170 2024-03-30T00:46:21.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe50ea00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4f4066690 0x7fe4f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe50ea00640 1 -- 172.21.15.66:0/1054436587 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe508084460 con 0x7fe5180b3170 2024-03-30T00:46:21.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe50fe00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4f4066690 0x7fe4f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe520ee4000 1 Processor -- start 2024-03-30T00:46:21.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe520ee4000 1 -- start start 2024-03-30T00:46:21.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe50fe00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4f4066690 0x7fe4f4068b50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe5000ae000 tx=0x7fe5000a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:21.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07ddb0 0x55e76c07e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07e6d0 0x55e76c033f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c0348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe520ee4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e76befc5a0 con 0x55e76c07ddb0 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe520ee4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e76be89af0 con 0x55e76c07e6d0 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.154+0000 7fe520ee4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e76be870c0 con 0x55e76c0344c0 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe516a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07ddb0 0x55e76c07e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe516a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07ddb0 0x55e76c07e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54358/0 (socket says 172.21.15.66:54358) 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe516a00640 1 -- 172.21.15.66:0/2482792188 learned_addr learned my addr 172.21.15.66:0/2482792188 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe516a00640 1 -- 172.21.15.66:0/2482792188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 msgr2=0x55e76c0348a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe516a00640 1 --2- 172.21.15.66:0/2482792188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c0348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe516a00640 1 -- 172.21.15.66:0/2482792188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07e6d0 msgr2=0x55e76c033f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe516a00640 1 --2- 172.21.15.66:0/2482792188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07e6d0 0x55e76c033f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe516a00640 1 -- 172.21.15.66:0/2482792188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e76bb7e890 con 0x55e76c07ddb0 2024-03-30T00:46:21.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe516a00640 1 --2- 172.21.15.66:0/2482792188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07ddb0 0x55e76c07e190 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe50805d180 tx=0x7fe508054d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e247db56302713 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe507400640 1 -- 172.21.15.66:0/2482792188 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508053660 con 0x55e76c07ddb0 2024-03-30T00:46:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe507400640 1 -- 172.21.15.66:0/2482792188 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5080519d0 con 0x55e76c07ddb0 2024-03-30T00:46:21.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe507400640 1 -- 172.21.15.66:0/2482792188 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508051b40 con 0x55e76c07ddb0 2024-03-30T00:46:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 -- 172.21.15.66:0/2482792188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07ddb0 msgr2=0x55e76c07e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2482792188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07ddb0 0x55e76c07e190 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe50805d180 tx=0x7fe508054d50 comp rx=0 tx=0).stop 2024-03-30T00:46:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 -- 172.21.15.66:0/2482792188 shutdown_connections 2024-03-30T00:46:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2482792188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c0348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2482792188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07e6d0 0x55e76c033f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2482792188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07ddb0 0x55e76c07e190 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 -- 172.21.15.66:0/2482792188 >> 172.21.15.66:0/2482792188 conn(0x55e76c03a830 msgr2=0x55e76c032860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 -- 172.21.15.66:0/2482792188 shutdown_connections 2024-03-30T00:46:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 -- 172.21.15.66:0/2482792188 wait complete. 2024-03-30T00:46:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 -- start start 2024-03-30T00:46:21.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 Processor -- start 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c07d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c039210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e76be829b0 con 0x55e76c07e6d0 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e76be21c00 con 0x55e76c07ddb0 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e76be40070 con 0x55e76c0344c0 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe520ee4000 10 client.0 _ll_register_callbacks cb 0x55e76c0200e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe517400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c039210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.158+0000 7fe516a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c07d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe50fe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe516a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c07d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:47830/0 (socket says 172.21.15.66:47830) 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe516a00640 1 -- 172.21.15.66:0/1290633996 learned_addr learned my addr 172.21.15.66:0/1290633996 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe517400640 1 -- 172.21.15.66:0/1290633996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 msgr2=0x55e76c07d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe517400640 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c07d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe517400640 1 -- 172.21.15.66:0/1290633996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 msgr2=0x55e76c038cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe517400640 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c038cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe517400640 1 -- 172.21.15.66:0/1290633996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5180b1170 con 0x55e76c07e6d0 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe50fe00640 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c038cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe517400640 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c039210 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe51000d400 tx=0x7fe51000d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7656ecb4b1cae6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe4fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe4fbe00640 1 -- 172.21.15.66:0/1290633996 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000b070 con 0x55e76c07e6d0 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe4fbe00640 1 -- 172.21.15.66:0/1290633996 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe510013bc0 con 0x55e76c07e6d0 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 -- 172.21.15.66:0/1290633996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e76be3f5f0 con 0x55e76c07e6d0 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe4fbe00640 1 -- 172.21.15.66:0/1290633996 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe510013de0 con 0x55e76c07e6d0 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 10 client.7809 fetch_fsmap learned FSMap version 58 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 -- 172.21.15.66:0/1290633996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55e76be47ab0 con 0x55e76c07e6d0 2024-03-30T00:46:21.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe4fbe00640 1 -- 172.21.15.66:0/1290633996 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e76be3f5f0 con 0x55e76c07e6d0 2024-03-30T00:46:21.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe4fbe00640 1 -- 172.21.15.66:0/1290633996 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 58) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe510003980 con 0x55e76c07e6d0 2024-03-30T00:46:21.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 10 client.7809 fetch_fsmap finished waiting for FSMap version 58 2024-03-30T00:46:21.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 10 client.7809 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:46:21.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 20 client.7809 populate_metadata read hostname 'smithi066' 2024-03-30T00:46:21.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.66:6835/1147664160,v1:172.21.15.66:6836/1147664160] conn(0x55e76c07f310 0x55e76c08f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 4 client.7809 mds_command: new command op to 7692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:46:21.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 -- 172.21.15.66:0/1290633996 --> [v2:172.21.15.66:6835/1147664160,v1:172.21.15.66:6836/1147664160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e76be7f790 con 0x55e76c07f310 2024-03-30T00:46:21.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe516a00640 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.66:6835/1147664160,v1:172.21.15.66:6836/1147664160] conn(0x55e76c07f310 0x55e76c08f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe516a00640 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.66:6835/1147664160,v1:172.21.15.66:6836/1147664160] conn(0x55e76c07f310 0x55e76c08f530 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba313264ff14f2a server_cookie=3bda4dae02db3efd in_seq=0 out_seq=0 2024-03-30T00:46:21.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe4fbe00640 10 client.7809 ms_handle_connect on v2:172.21.15.66:6835/1147664160 2024-03-30T00:46:21.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe4fbe00640 1 -- 172.21.15.66:0/1290633996 <== mds.0 v2:172.21.15.66:6835/1147664160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e76be7f790 con 0x55e76c07f310 2024-03-30T00:46:21.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe4fbe00640 10 client.7809 handle_command_reply: tid=0 2024-03-30T00:46:21.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 client.7809 shutdown 2024-03-30T00:46:21.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 -- 172.21.15.66:0/1290633996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 msgr2=0x55e76c039210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c039210 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe51000d400 tx=0x7fe51000d8c0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 -- 172.21.15.66:0/1290633996 shutdown_connections 2024-03-30T00:46:21.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c039210 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c038cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.66:6835/1147664160,v1:172.21.15.66:6836/1147664160] conn(0x55e76c07f310 0x55e76c08f530 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 --2- 172.21.15.66:0/1290633996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c07d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 -- 172.21.15.66:0/1290633996 >> 172.21.15.66:0/1290633996 conn(0x55e76c03a830 msgr2=0x55e76c032f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:21.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe4fbe00640 0 client.7809 ms_handle_reset on v2:172.21.15.66:6835/1147664160 2024-03-30T00:46:21.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 -- 172.21.15.66:0/1290633996 shutdown_connections 2024-03-30T00:46:21.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 1 -- 172.21.15.66:0/1290633996 wait complete. 2024-03-30T00:46:21.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.162+0000 7fe520ee4000 20 client.7809 trim_cache size 0 max 16384 2024-03-30T00:46:21.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.178+0000 7fe520ee4000 1 Processor -- start 2024-03-30T00:46:21.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.178+0000 7fe520ee4000 1 -- start start 2024-03-30T00:46:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.178+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c07c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.178+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c07cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.178+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c07f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.178+0000 7fe520ee4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e76bb3f940 con 0x55e76c07e6d0 2024-03-30T00:46:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.178+0000 7fe520ee4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e76c0374d0 con 0x55e76c07ddb0 2024-03-30T00:46:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.178+0000 7fe520ee4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e76be1fc10 con 0x55e76c0344c0 2024-03-30T00:46:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c07f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c07f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54376/0 (socket says 172.21.15.66:54376) 2024-03-30T00:46:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 -- 172.21.15.66:0/2397938781 learned_addr learned my addr 172.21.15.66:0/2397938781 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe516a00640 1 --2- 172.21.15.66:0/2397938781 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c07c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe50fe00640 1 --2- 172.21.15.66:0/2397938781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c07cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 -- 172.21.15.66:0/2397938781 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 msgr2=0x55e76c07c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 --2- 172.21.15.66:0/2397938781 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c07c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 -- 172.21.15.66:0/2397938781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 msgr2=0x55e76c07cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 --2- 172.21.15.66:0/2397938781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c07cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 -- 172.21.15.66:0/2397938781 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e76be2dc00 con 0x55e76c07e6d0 2024-03-30T00:46:21.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 --2- 172.21.15.66:0/2397938781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c07f6f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe510001d40 tx=0x7fe51000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e42be3c26c1b07 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:21.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe507e00640 1 -- 172.21.15.66:0/2397938781 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510001ff0 con 0x55e76c07e6d0 2024-03-30T00:46:21.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe507e00640 1 -- 172.21.15.66:0/2397938781 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51000ddc0 con 0x55e76c07e6d0 2024-03-30T00:46:21.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe507e00640 1 -- 172.21.15.66:0/2397938781 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510005350 con 0x55e76c07e6d0 2024-03-30T00:46:21.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 -- 172.21.15.66:0/2397938781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 msgr2=0x55e76c07f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2397938781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c07f6f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe510001d40 tx=0x7fe51000b000 comp rx=0 tx=0).stop 2024-03-30T00:46:21.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 -- 172.21.15.66:0/2397938781 shutdown_connections 2024-03-30T00:46:21.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2397938781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c07f6f0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2397938781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c07cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2397938781 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c07c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 -- 172.21.15.66:0/2397938781 >> 172.21.15.66:0/2397938781 conn(0x55e76c03a830 msgr2=0x55e76c0387a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:21.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 -- 172.21.15.66:0/2397938781 shutdown_connections 2024-03-30T00:46:21.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 -- 172.21.15.66:0/2397938781 wait complete. 2024-03-30T00:46:21.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 -- start start 2024-03-30T00:46:21.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 Processor -- start 2024-03-30T00:46:21.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c0dbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c0824b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c0829f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e76be7f790 con 0x55e76c07e6d0 2024-03-30T00:46:21.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e76be0ac80 con 0x55e76c07ddb0 2024-03-30T00:46:21.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e76be3f5f0 con 0x55e76c0344c0 2024-03-30T00:46:21.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe520ee4000 10 client.0 _ll_register_callbacks cb 0x55e76be47590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:46:21.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c0829f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c0829f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54392/0 (socket says 172.21.15.66:54392) 2024-03-30T00:46:21.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe517400640 1 -- 172.21.15.66:0/2050104414 learned_addr learned my addr 172.21.15.66:0/2050104414 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:21.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe516a00640 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c0dbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.182+0000 7fe50fe00640 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c0824b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517400640 1 -- 172.21.15.66:0/2050104414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 msgr2=0x55e76c0dbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517400640 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c0dbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517400640 1 -- 172.21.15.66:0/2050104414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 msgr2=0x55e76c0824b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517400640 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c0824b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517400640 1 -- 172.21.15.66:0/2050104414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e76be47ab0 con 0x55e76c07e6d0 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe516a00640 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c0dbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe50fe00640 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c0824b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517400640 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c0829f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe510003e10 tx=0x7fe5100014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cbd5f0b1683cc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe4fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe4fbe00640 1 -- 172.21.15.66:0/2050104414 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510001ff0 con 0x55e76c07e6d0 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe4fbe00640 1 -- 172.21.15.66:0/2050104414 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe510027650 con 0x55e76c07e6d0 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 -- 172.21.15.66:0/2050104414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e76be52460 con 0x55e76c07e6d0 2024-03-30T00:46:21.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe4fbe00640 1 -- 172.21.15.66:0/2050104414 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe510001650 con 0x55e76c07e6d0 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 10 client.7815 fetch_fsmap learned FSMap version 58 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 -- 172.21.15.66:0/2050104414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55e76bb7ecc0 con 0x55e76c07e6d0 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe4fbe00640 1 -- 172.21.15.66:0/2050104414 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e76be52460 con 0x55e76c07e6d0 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe4fbe00640 1 -- 172.21.15.66:0/2050104414 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 58) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe510027850 con 0x55e76c07e6d0 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 10 client.7815 fetch_fsmap finished waiting for FSMap version 58 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 10 client.7815 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 20 client.7815 populate_metadata read hostname 'smithi066' 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.66:6835/1147664160,v1:172.21.15.66:6836/1147664160] conn(0x55e76c082f30 0x55e76c083310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 4 client.7815 mds_command: new command op to 7692 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 -- 172.21.15.66:0/2050104414 --> [v2:172.21.15.66:6835/1147664160,v1:172.21.15.66:6836/1147664160] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e76be4b9c0 con 0x55e76c082f30 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe516a00640 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.66:6835/1147664160,v1:172.21.15.66:6836/1147664160] conn(0x55e76c082f30 0x55e76c083310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe516a00640 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.66:6835/1147664160,v1:172.21.15.66:6836/1147664160] conn(0x55e76c082f30 0x55e76c083310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9a523466adf6e0 server_cookie=f88472971cf035b3 in_seq=0 out_seq=0 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe4fbe00640 10 client.7815 ms_handle_connect on v2:172.21.15.66:6835/1147664160 2024-03-30T00:46:21.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe4fbe00640 1 -- 172.21.15.66:0/2050104414 <== mds.0 v2:172.21.15.66:6835/1147664160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e76be4b9c0 con 0x55e76c082f30 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe4fbe00640 10 client.7815 handle_command_reply: tid=0 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 client.7815 shutdown 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 -- 172.21.15.66:0/2050104414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 msgr2=0x55e76c0829f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c0829f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe510003e10 tx=0x7fe5100014b0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 -- 172.21.15.66:0/2050104414 shutdown_connections 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e76c07e6d0 0x55e76c0829f0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e76c07ddb0 0x55e76c0824b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.66:6835/1147664160,v1:172.21.15.66:6836/1147664160] conn(0x55e76c082f30 0x55e76c083310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 --2- 172.21.15.66:0/2050104414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55e76c0344c0 0x55e76c0dbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 -- 172.21.15.66:0/2050104414 >> 172.21.15.66:0/2050104414 conn(0x55e76c03a830 msgr2=0x55e76c07b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe4fbe00640 0 client.7815 ms_handle_reset on v2:172.21.15.66:6835/1147664160 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 -- 172.21.15.66:0/2050104414 shutdown_connections 2024-03-30T00:46:21.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 1 -- 172.21.15.66:0/2050104414 wait complete. 2024-03-30T00:46:21.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe520ee4000 20 client.7815 trim_cache size 0 max 16384 2024-03-30T00:46:21.200 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:46:21.200 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:46:21.200 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:46:21.200 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 7692, 2024-03-30T00:46:21.200 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:46:21.200 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:46:21.200 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:46:21.200 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 5.9155729089999998, 2024-03-30T00:46:21.200 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 58, 2024-03-30T00:46:21.200 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 95, 2024-03-30T00:46:21.201 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 95, 2024-03-30T00:46:21.201 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 10.236928491 2024-03-30T00:46:21.201 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:46:21.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 -- 172.21.15.66:0/1054436587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4f4066690 msgr2=0x7fe4f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4f4066690 0x7fe4f4068b50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe5000ae000 tx=0x7fe5000a7000 comp rx=0 tx=0).stop 2024-03-30T00:46:21.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 -- 172.21.15.66:0/1054436587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5180b3170 msgr2=0x7fe51806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5180b3170 0x7fe51806b8b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe50800e870 tx=0x7fe50800ec70 comp rx=0 tx=0).stop 2024-03-30T00:46:21.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 -- 172.21.15.66:0/1054436587 shutdown_connections 2024-03-30T00:46:21.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe51815a720 0x7fe518152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5180d07c0 0x7fe5180b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe4f4066690 0x7fe4f4068b50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 --2- 172.21.15.66:0/1054436587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5180b3170 0x7fe51806b8b0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 -- 172.21.15.66:0/1054436587 >> 172.21.15.66:0/1054436587 conn(0x7fe518069d90 msgr2=0x7fe51806e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:21.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 -- 172.21.15.66:0/1054436587 shutdown_connections 2024-03-30T00:46:21.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.186+0000 7fe517e00640 1 -- 172.21.15.66:0/1054436587 wait complete. 2024-03-30T00:46:21.215 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:46:21.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.346+0000 7ff766c00640 1 Processor -- start 2024-03-30T00:46:21.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.346+0000 7ff766c00640 1 -- start start 2024-03-30T00:46:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.346+0000 7ff766c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7600cabb0 0x7ff7600cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.346+0000 7ff766c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76006d070 con 0x7ff7600ce520 2024-03-30T00:46:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.346+0000 7ff766c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7600cbba0 con 0x7ff76015a640 2024-03-30T00:46:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.346+0000 7ff766c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600cb8c0 con 0x7ff7600cabb0 2024-03-30T00:46:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.346+0000 7ff757400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff7600ce520 0x7ff7600ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55576/0 (socket says 172.21.15.66:55576) 2024-03-30T00:46:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.346+0000 7ff757400640 1 -- 172.21.15.66:0/927696984 learned_addr learned my addr 172.21.15.66:0/927696984 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:21.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff764c00640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7600cabb0 0x7ff7600cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047974826 0 0) 0x7ff76006d070 con 0x7ff7600ce520 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff744003530 con 0x7ff7600ce520 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361954852 0 0) 0x7ff7600cbba0 con 0x7ff76015a640 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff76006d070 con 0x7ff76015a640 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2274501994 0 0) 0x7ff744003530 con 0x7ff7600ce520 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7600cabb0 msgr2=0x7ff7600cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7600cabb0 0x7ff7600cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 >> v1:172.21.15.156:6789/0 conn(0x7ff76015a640 legacy=0x7ff76006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76006d2a0 con 0x7ff7600ce520 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7ff748001600 con 0x7ff7600ce520 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 >> v1:172.21.15.66:6789/0 conn(0x7ff7600ce520 legacy=0x7ff7600ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff744003b40 0x7ff744003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 0x7ff744004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7600cabb0 0x7ff744005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600cb8c0 con 0x7ff7440047d0 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74400ba70 con 0x7ff744003b40 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74400bc30 con 0x7ff7600cabb0 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff757400640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 0x7ff744004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff764c00640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7600cabb0 0x7ff744005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75fe00640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff744003b40 0x7ff744003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff757400640 1 -- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7600cabb0 msgr2=0x7ff744005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff757400640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7600cabb0 0x7ff744005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff757400640 1 -- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff744003b40 msgr2=0x7ff744003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff757400640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff744003b40 0x7ff744003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff757400640 1 -- 172.21.15.66:0/927696984 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff744003530 con 0x7ff7440047d0 2024-03-30T00:46:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.350+0000 7ff75fe00640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff744003b40 0x7ff744003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:21.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff757400640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 0x7ff744004c50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff748007f50 tx=0x7ff74801caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1338ea1dae787e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748005ba0 con 0x7ff7440047d0 2024-03-30T00:46:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff748005d20 con 0x7ff7440047d0 2024-03-30T00:46:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff75f400640 1 -- 172.21.15.66:0/927696984 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7480a59d0 con 0x7ff7440047d0 2024-03-30T00:46:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 -- 172.21.15.66:0/927696984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 msgr2=0x7ff744004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 --2- 172.21.15.66:0/927696984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 0x7ff744004c50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff748007f50 tx=0x7ff74801caa0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff764c00640 1 -- 172.21.15.66:0/927696984 reap_dead start 2024-03-30T00:46:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 -- 172.21.15.66:0/927696984 shutdown_connections 2024-03-30T00:46:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 -- 172.21.15.66:0/927696984 >> 172.21.15.66:0/927696984 conn(0x7ff760069f60 msgr2=0x7ff76006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 -- 172.21.15.66:0/927696984 shutdown_connections 2024-03-30T00:46:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 -- 172.21.15.66:0/927696984 wait complete. 2024-03-30T00:46:21.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 Processor -- start 2024-03-30T00:46:21.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 -- start start 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 0x7ff7600ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7601f1860 0x7ff7601581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7601586e0 0x7ff7600c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff760158b60 con 0x7ff7440047d0 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600ca1a0 con 0x7ff7601586e0 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff766c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601f1ce0 con 0x7ff7601f1860 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.354+0000 7ff75fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 0x7ff7600ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 0x7ff7600ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54402/0 (socket says 172.21.15.66:54402) 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75fe00640 1 -- 172.21.15.66:0/4211319918 learned_addr learned my addr 172.21.15.66:0/4211319918 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff757400640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7601f1860 0x7ff7601581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff764c00640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7601586e0 0x7ff7600c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75fe00640 1 -- 172.21.15.66:0/4211319918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7601f1860 msgr2=0x7ff7601581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75fe00640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7601f1860 0x7ff7601581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75fe00640 1 -- 172.21.15.66:0/4211319918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7601586e0 msgr2=0x7ff7600c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75fe00640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7601586e0 0x7ff7600c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75fe00640 1 -- 172.21.15.66:0/4211319918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7600c9590 con 0x7ff7440047d0 2024-03-30T00:46:21.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75fe00640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 0x7ff7600ca630 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff750011d30 tx=0x7ff75000f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:21.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75cc00640 1 -- 172.21.15.66:0/4211319918 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750015610 con 0x7ff7440047d0 2024-03-30T00:46:21.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75cc00640 1 -- 172.21.15.66:0/4211319918 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff750015770 con 0x7ff7440047d0 2024-03-30T00:46:21.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75cc00640 1 -- 172.21.15.66:0/4211319918 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750015ae0 con 0x7ff7440047d0 2024-03-30T00:46:21.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff766c00640 1 -- 172.21.15.66:0/4211319918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7600c9870 con 0x7ff7440047d0 2024-03-30T00:46:21.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff766c00640 1 -- 172.21.15.66:0/4211319918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7600c9e30 con 0x7ff7440047d0 2024-03-30T00:46:21.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff766c00640 1 -- 172.21.15.66:0/4211319918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff760066c30 con 0x7ff7440047d0 2024-03-30T00:46:21.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75cc00640 1 -- 172.21.15.66:0/4211319918 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff750015c40 con 0x7ff7440047d0 2024-03-30T00:46:21.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.358+0000 7ff75cc00640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff73c0666c0 0x7ff73c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:21.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.362+0000 7ff75cc00640 1 -- 172.21.15.66:0/4211319918 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff750087b10 con 0x7ff7440047d0 2024-03-30T00:46:21.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.362+0000 7ff757400640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff73c0666c0 0x7ff73c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:21.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.362+0000 7ff757400640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff73c0666c0 0x7ff73c068b80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff748007f20 tx=0x7ff7480a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:21.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.366+0000 7ff75cc00640 1 -- 172.21.15.66:0/4211319918 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff750054c10 con 0x7ff7440047d0 2024-03-30T00:46:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 -- 172.21.15.66:0/4211319918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff7600c4e10 con 0x7ff7440047d0 2024-03-30T00:46:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff75cc00640 1 -- 172.21.15.66:0/4211319918 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff750007600 con 0x7ff7440047d0 2024-03-30T00:46:21.570 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:46:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 -- 172.21.15.66:0/4211319918 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff73c0666c0 msgr2=0x7ff73c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff73c0666c0 0x7ff73c068b80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff748007f20 tx=0x7ff7480a7070 comp rx=0 tx=0).stop 2024-03-30T00:46:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 -- 172.21.15.66:0/4211319918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 msgr2=0x7ff7600ca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 0x7ff7600ca630 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff750011d30 tx=0x7ff75000f540 comp rx=0 tx=0).stop 2024-03-30T00:46:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 -- 172.21.15.66:0/4211319918 shutdown_connections 2024-03-30T00:46:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7601586e0 0x7ff7600c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7601f1860 0x7ff7601581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff73c0666c0 0x7ff73c068b80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 --2- 172.21.15.66:0/4211319918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7440047d0 0x7ff7600ca630 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 -- 172.21.15.66:0/4211319918 >> 172.21.15.66:0/4211319918 conn(0x7ff760069f60 msgr2=0x7ff7600adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 -- 172.21.15.66:0/4211319918 shutdown_connections 2024-03-30T00:46:21.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:21.562+0000 7ff766c00640 1 -- 172.21.15.66:0/4211319918 wait complete. 2024-03-30T00:46:21.591 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:46:21.591 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:46:21.591 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:46:21.591 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:46:21.591 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:21.592 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:46:21.592 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:46:21.592 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:46:21.592 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:46:21.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:21.592 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:46:21.601 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:46:21.602 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:21.603 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:46:21.603 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:21.603 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:46:21.603 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42097sec preferred_lft 42097sec 2024-03-30T00:46:21.603 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:46:21.603 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:21.603 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:46:21.604 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:21.604 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:46:21.604 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:21.604 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:46:21.604 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:21.604 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:46:21.604 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:21.604 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:46:21.604 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:21.604 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:46:21.604 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:46:21.605 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:46:21.605 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:21.605 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:46:21.605 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:46:21.605 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:46:21.605 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:21.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:21.605 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:46:21.663 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:46:21.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:21.665 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:46:21.720 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:46:21.720 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:46:21.720 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:46:21.721 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:46:21.721 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:46:21.721 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:46:21.721 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:46:21.721 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:46:21.721 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:46:21.721 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:46:21.721 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:46:21.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:21.721 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:46:21.722 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:46:21.722 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:46:21.722 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:46:21.722 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:46:21.722 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:46:21.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:21.818 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:46:21.827 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:46:21.828 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:46:21.828 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:46:21.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:21.828 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:46:21.888 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:46:21.888 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:46:21.888 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:21.940 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:46:21.940 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:46:21.941 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:21.992 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:46:22.053 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:46:22.053 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:46:22.118 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:46:22.120 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:46:22.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:22.120 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:46:22.176 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:46:22.177 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:46:22.223 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:46:22.244 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:46:22.245 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:46:22.311 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:46:22.312 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:46:22.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:22.313 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:46:22.319 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:46:22.310+0000 7f6ba7247580 -1 init, newargv = 0x55ddb80143b0 newargc=15 2024-03-30T00:46:22.319 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[47490]: starting ceph client 2024-03-30T00:46:22.325 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[47490]: starting fuse 2024-03-30T00:46:22.327 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:46:22.328 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:46:22.328 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:46:22.328 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:46:22.328 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:46:22.328 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:46:22.328 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:46:22.328 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:46:22.328 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:46:22.328 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:46:22.328 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:46:22.328 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:46:22.328 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:46:22.329 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:46:22.330 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:46:22.449 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.47490.asok 2024-03-30T00:46:22.634 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47490.asok 2024-03-30T00:46:22.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:22.634 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47490.asok status 2024-03-30T00:46:22.808 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:46:22.808 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:46:22.808 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:46:22.808 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:46:22.808 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:46:22.808 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "47490", 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 7827, 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 7827 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:46:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 4272241300 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 4272241300 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.7827 192.168.0.1:0/4272241300", 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/4272241300", 2024-03-30T00:46:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:46:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 58, 2024-03-30T00:46:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 95, 2024-03-30T00:46:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:46:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:46:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:46:22.811 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:46:22.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:22.823 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:46:22.823 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:22.832 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:46:22.832 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:22.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:22.832 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:22.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:22.897 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-03-30T00:46:22.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:22.956 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-03-30T00:46:23.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:23.013 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-03-30T00:46:23.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:23.071 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-03-30T00:46:23.127 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:46:23.127 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:46:23.127 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:46:23.127 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:46:23.127 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-03-30T00:46:23.127 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:46:23.231 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627776 2024-03-30T00:46:23.422 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:46:23.422 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:46:23.473 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:46:23.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.646+0000 7f8e37200640 1 Processor -- start 2024-03-30T00:46:23.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.646+0000 7f8e37200640 1 -- start start 2024-03-30T00:46:23.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e37200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8e300cf2f0 0x7f8e3006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:23.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e37200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e300cb570 0x7f8e300cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:23.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e37200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e300cb0e0 con 0x7f8e3015ca80 2024-03-30T00:46:23.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e37200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e300cf7f0 con 0x7f8e300cf2f0 2024-03-30T00:46:23.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e37200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300cfad0 con 0x7f8e300cb570 2024-03-30T00:46:23.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e27400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8e300cf2f0 0x7f8e3006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:23.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e27400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8e300cf2f0 0x7f8e3006b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54412/0 (socket says 172.21.15.66:54412) 2024-03-30T00:46:23.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e27400640 1 -- 172.21.15.66:0/2507201950 learned_addr learned my addr 172.21.15.66:0/2507201950 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:23.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e27400640 1 -- 172.21.15.66:0/2507201950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e300cb570 msgr2=0x7f8e300cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:23.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e27400640 1 --2- 172.21.15.66:0/2507201950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e300cb570 0x7f8e300cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:23.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e27400640 1 -- 172.21.15.66:0/2507201950 >> v1:172.21.15.66:6789/0 conn(0x7f8e3015ca80 legacy=0x7f8e3006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:23.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.650+0000 7f8e27400640 1 -- 172.21.15.66:0/2507201950 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e300cad70 con 0x7f8e300cf2f0 2024-03-30T00:46:23.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e27400640 1 --2- 172.21.15.66:0/2507201950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e300cf2f0 0x7f8e3006b7c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009870 tx=0x7f8e1801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71c7c0341b510a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:23.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e2f400640 1 -- 172.21.15.66:0/2507201950 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180a8a50 con 0x7f8e300cf2f0 2024-03-30T00:46:23.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e2f400640 1 -- 172.21.15.66:0/2507201950 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e180a8bb0 con 0x7f8e300cf2f0 2024-03-30T00:46:23.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e2f400640 1 -- 172.21.15.66:0/2507201950 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180b0440 con 0x7f8e300cf2f0 2024-03-30T00:46:23.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 -- 172.21.15.66:0/2507201950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e300cf2f0 msgr2=0x7f8e3006b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:23.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 --2- 172.21.15.66:0/2507201950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e300cf2f0 0x7f8e3006b7c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009870 tx=0x7f8e1801ed50 comp rx=0 tx=0).stop 2024-03-30T00:46:23.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 -- 172.21.15.66:0/2507201950 shutdown_connections 2024-03-30T00:46:23.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 --2- 172.21.15.66:0/2507201950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e300cb570 0x7f8e300cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:23.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 --2- 172.21.15.66:0/2507201950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e300cf2f0 0x7f8e3006b7c0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:23.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 -- 172.21.15.66:0/2507201950 >> 172.21.15.66:0/2507201950 conn(0x7f8e300b2090 msgr2=0x7f8e300b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:23.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 -- 172.21.15.66:0/2507201950 shutdown_connections 2024-03-30T00:46:23.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 -- 172.21.15.66:0/2507201950 wait complete. 2024-03-30T00:46:23.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 Processor -- start 2024-03-30T00:46:23.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 -- start start 2024-03-30T00:46:23.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e300cb570 0x7f8e30155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:23.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e300cf2f0 0x7f8e300c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:23.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e3015ca80 0x7f8e300c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:23.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300b3c40 con 0x7f8e3015ca80 2024-03-30T00:46:23.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30156050 con 0x7f8e300cb570 2024-03-30T00:46:23.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e37200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3015dfd0 con 0x7f8e300cf2f0 2024-03-30T00:46:23.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e35200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e3015ca80 0x7f8e300c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:23.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e35200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e3015ca80 0x7f8e300c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54422/0 (socket says 172.21.15.66:54422) 2024-03-30T00:46:23.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e35200640 1 -- 172.21.15.66:0/728641644 learned_addr learned my addr 172.21.15.66:0/728641644 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:23.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e27400640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e300cf2f0 0x7f8e300c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:23.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.654+0000 7f8e2fe00640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e300cb570 0x7f8e30155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:23.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e35200640 1 -- 172.21.15.66:0/728641644 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e300cf2f0 msgr2=0x7f8e300c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:23.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e35200640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e300cf2f0 0x7f8e300c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:23.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e35200640 1 -- 172.21.15.66:0/728641644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e300cb570 msgr2=0x7f8e30155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:23.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e35200640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e300cb570 0x7f8e30155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:23.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e35200640 1 -- 172.21.15.66:0/728641644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e180094d0 con 0x7f8e3015ca80 2024-03-30T00:46:23.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e35200640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e3015ca80 0x7f8e300c9710 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800b970 tx=0x7f8e2800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:23.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e2cc00640 1 -- 172.21.15.66:0/728641644 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800cb40 con 0x7f8e3015ca80 2024-03-30T00:46:23.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e2cc00640 1 -- 172.21.15.66:0/728641644 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2800ccc0 con 0x7f8e3015ca80 2024-03-30T00:46:23.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e2cc00640 1 -- 172.21.15.66:0/728641644 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28014680 con 0x7f8e3015ca80 2024-03-30T00:46:23.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e37200640 1 -- 172.21.15.66:0/728641644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3015e2b0 con 0x7f8e3015ca80 2024-03-30T00:46:23.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e37200640 1 -- 172.21.15.66:0/728641644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3015e870 con 0x7f8e3015ca80 2024-03-30T00:46:23.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e37200640 1 -- 172.21.15.66:0/728641644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e300b3db0 con 0x7f8e3015ca80 2024-03-30T00:46:23.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e2cc00640 1 -- 172.21.15.66:0/728641644 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8e2801d3b0 con 0x7f8e3015ca80 2024-03-30T00:46:23.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e2cc00640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8e0c066690 0x7f8e0c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:23.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e2cc00640 1 -- 172.21.15.66:0/728641644 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8e280841b0 con 0x7f8e3015ca80 2024-03-30T00:46:23.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.658+0000 7f8e2fe00640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8e0c066690 0x7f8e0c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:23.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.662+0000 7f8e2fe00640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8e0c066690 0x7f8e0c068b50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8e20001770 tx=0x7f8e20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:23.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.662+0000 7f8e2cc00640 1 -- 172.21.15.66:0/728641644 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8e28051480 con 0x7f8e3015ca80 2024-03-30T00:46:23.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.826+0000 7f8e37200640 1 -- 172.21.15.66:0/728641644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e30066c30 con 0x7f8e3015ca80 2024-03-30T00:46:23.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e2cc00640 1 -- 172.21.15.66:0/728641644 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16249 (secure 0 0 0) 0x7f8e28054aa0 con 0x7f8e3015ca80 2024-03-30T00:46:23.837 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:23.838 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":95,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:46:16.274340+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:46:13.811742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-30T00:46:14.265142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 -- 172.21.15.66:0/728641644 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8e0c066690 msgr2=0x7f8e0c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8e0c066690 0x7f8e0c068b50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8e20001770 tx=0x7f8e20009040 comp rx=0 tx=0).stop 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 -- 172.21.15.66:0/728641644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e3015ca80 msgr2=0x7f8e300c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e3015ca80 0x7f8e300c9710 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800b970 tx=0x7f8e2800be30 comp rx=0 tx=0).stop 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 -- 172.21.15.66:0/728641644 shutdown_connections 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8e0c066690 0x7f8e0c068b50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e3015ca80 0x7f8e300c9710 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e300cf2f0 0x7f8e300c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 --2- 172.21.15.66:0/728641644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e300cb570 0x7f8e30155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 -- 172.21.15.66:0/728641644 >> 172.21.15.66:0/728641644 conn(0x7f8e300b2090 msgr2=0x7f8e300cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 -- 172.21.15.66:0/728641644 shutdown_connections 2024-03-30T00:46:23.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.830+0000 7f8e37200640 1 -- 172.21.15.66:0/728641644 wait complete. 2024-03-30T00:46:23.859 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:46:13.811742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:46:14.265142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:46:23.859 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:46:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.994+0000 7fb1fcc00640 1 Processor -- start 2024-03-30T00:46:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.994+0000 7fb1fcc00640 1 -- start start 2024-03-30T00:46:24.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.994+0000 7fb1fcc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb1f80cda90 0x7fb1f80ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.994+0000 7fb1fcc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb1f80cde70 0x7fb1f80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.994+0000 7fb1fcc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f815a990 con 0x7fb1f8159c60 2024-03-30T00:46:24.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.994+0000 7fb1fcc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f80b1b80 con 0x7fb1f80cde70 2024-03-30T00:46:24.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.994+0000 7fb1fcc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f80cd3e0 con 0x7fb1f80cda90 2024-03-30T00:46:24.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.994+0000 7fb1ed400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb1f80cde70 0x7fb1f80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.994+0000 7fb1ed400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb1f80cde70 0x7fb1f80cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54430/0 (socket says 172.21.15.66:54430) 2024-03-30T00:46:24.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.994+0000 7fb1ed400640 1 -- 172.21.15.66:0/524472588 learned_addr learned my addr 172.21.15.66:0/524472588 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:24.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.998+0000 7fb1f5e00640 1 --2- 172.21.15.66:0/524472588 >> v2:172.21.15.156:3300/0 conn(0x7fb1f80cda90 0x7fb1f80ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.998+0000 7fb1ed400640 1 -- 172.21.15.66:0/524472588 >> v2:172.21.15.156:3300/0 conn(0x7fb1f80cda90 msgr2=0x7fb1f80ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:24.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.998+0000 7fb1ed400640 1 --2- 172.21.15.66:0/524472588 >> v2:172.21.15.156:3300/0 conn(0x7fb1f80cda90 0x7fb1f80ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.998+0000 7fb1ed400640 1 -- 172.21.15.66:0/524472588 >> v1:172.21.15.156:6789/0 conn(0x7fb1f8159c60 legacy=0x7fb1f815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.998+0000 7fb1ed400640 1 -- 172.21.15.66:0/524472588 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f80b11e0 con 0x7fb1f80cde70 2024-03-30T00:46:24.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.998+0000 7fb1ed400640 1 --2- 172.21.15.66:0/524472588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f80cde70 0x7fb1f80cea60 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb1f80cf3f0 tx=0x7fb1e001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2ce7ff33b4358e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.998+0000 7fb1f5400640 1 -- 172.21.15.66:0/524472588 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00ac600 con 0x7fb1f80cde70 2024-03-30T00:46:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.998+0000 7fb1f5400640 1 -- 172.21.15.66:0/524472588 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e00a8050 con 0x7fb1f80cde70 2024-03-30T00:46:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.998+0000 7fb1f5400640 1 -- 172.21.15.66:0/524472588 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00ac980 con 0x7fb1f80cde70 2024-03-30T00:46:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:23.998+0000 7fb1fcc00640 1 -- 172.21.15.66:0/524472588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f80cde70 msgr2=0x7fb1f80cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 --2- 172.21.15.66:0/524472588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f80cde70 0x7fb1f80cea60 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb1f80cf3f0 tx=0x7fb1e001c490 comp rx=0 tx=0).stop 2024-03-30T00:46:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 -- 172.21.15.66:0/524472588 shutdown_connections 2024-03-30T00:46:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 --2- 172.21.15.66:0/524472588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f80cde70 0x7fb1f80cea60 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 --2- 172.21.15.66:0/524472588 >> v2:172.21.15.156:3300/0 conn(0x7fb1f80cda90 0x7fb1f80ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 -- 172.21.15.66:0/524472588 >> 172.21.15.66:0/524472588 conn(0x7fb1f8069f60 msgr2=0x7fb1f806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 -- 172.21.15.66:0/524472588 shutdown_connections 2024-03-30T00:46:24.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 -- 172.21.15.66:0/524472588 wait complete. 2024-03-30T00:46:24.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 Processor -- start 2024-03-30T00:46:24.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 -- start start 2024-03-30T00:46:24.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1f80cda90 0x7fb1f8154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f80cde70 0x7fb1f81538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f8159c60 0x7fb1f80ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80af360 con 0x7fb1f8159c60 2024-03-30T00:46:24.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f8155200 con 0x7fb1f80cde70 2024-03-30T00:46:24.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1fcc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80aeda0 con 0x7fb1f80cda90 2024-03-30T00:46:24.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1f6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f8159c60 0x7fb1f80ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1f5e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1f80cda90 0x7fb1f8154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1f6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f8159c60 0x7fb1f80ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54434/0 (socket says 172.21.15.66:54434) 2024-03-30T00:46:24.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1f6800640 1 -- 172.21.15.66:0/654061222 learned_addr learned my addr 172.21.15.66:0/654061222 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:24.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1ed400640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f80cde70 0x7fb1f81538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1f6800640 1 -- 172.21.15.66:0/654061222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1f80cda90 msgr2=0x7fb1f8154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1f6800640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1f80cda90 0x7fb1f8154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1f6800640 1 -- 172.21.15.66:0/654061222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f80cde70 msgr2=0x7fb1f81538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1f6800640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f80cde70 0x7fb1f81538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1f6800640 1 -- 172.21.15.66:0/654061222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e00094d0 con 0x7fb1f8159c60 2024-03-30T00:46:24.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.002+0000 7fb1f6800640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f8159c60 0x7fb1f80ac050 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c770 tx=0x7fb1f000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:24.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.006+0000 7fb1eea00640 1 -- 172.21.15.66:0/654061222 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0003b80 con 0x7fb1f8159c60 2024-03-30T00:46:24.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.006+0000 7fb1eea00640 1 -- 172.21.15.66:0/654061222 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f0003ce0 con 0x7fb1f8159c60 2024-03-30T00:46:24.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.006+0000 7fb1eea00640 1 -- 172.21.15.66:0/654061222 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f000aa90 con 0x7fb1f8159c60 2024-03-30T00:46:24.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.006+0000 7fb1fcc00640 1 -- 172.21.15.66:0/654061222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f80af080 con 0x7fb1f8159c60 2024-03-30T00:46:24.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.006+0000 7fb1fcc00640 1 -- 172.21.15.66:0/654061222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f8157ab0 con 0x7fb1f8159c60 2024-03-30T00:46:24.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.006+0000 7fb1fcc00640 1 -- 172.21.15.66:0/654061222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f8157630 con 0x7fb1f8159c60 2024-03-30T00:46:24.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.006+0000 7fb1eea00640 1 -- 172.21.15.66:0/654061222 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb1f000abf0 con 0x7fb1f8159c60 2024-03-30T00:46:24.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.006+0000 7fb1eea00640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1d4066690 0x7fb1d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.006+0000 7fb1eea00640 1 -- 172.21.15.66:0/654061222 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb1f0084060 con 0x7fb1f8159c60 2024-03-30T00:46:24.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.006+0000 7fb1f5e00640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1d4066690 0x7fb1d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.006+0000 7fb1f5e00640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1d4066690 0x7fb1d4068b50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009ac0 tx=0x7fb1e8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:24.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.010+0000 7fb1eea00640 1 -- 172.21.15.66:0/654061222 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb1f00b5980 con 0x7fb1f8159c60 2024-03-30T00:46:24.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.170+0000 7fb1fcc00640 1 -- 172.21.15.66:0/654061222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1f80c9080 con 0x7fb1f8159c60 2024-03-30T00:46:24.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.170+0000 7fb1eea00640 1 -- 172.21.15.66:0/654061222 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1f8157630 con 0x7fb1f8159c60 2024-03-30T00:46:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:46:24.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 -- 172.21.15.66:0/654061222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1d4066690 msgr2=0x7fb1d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1d4066690 0x7fb1d4068b50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009ac0 tx=0x7fb1e8009190 comp rx=0 tx=0).stop 2024-03-30T00:46:24.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 -- 172.21.15.66:0/654061222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f8159c60 msgr2=0x7fb1f80ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f8159c60 0x7fb1f80ac050 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c770 tx=0x7fb1f000cc30 comp rx=0 tx=0).stop 2024-03-30T00:46:24.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 -- 172.21.15.66:0/654061222 shutdown_connections 2024-03-30T00:46:24.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb1d4066690 0x7fb1d4068b50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f8159c60 0x7fb1f80ac050 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1f80cde70 0x7fb1f81538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 --2- 172.21.15.66:0/654061222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1f80cda90 0x7fb1f8154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 -- 172.21.15.66:0/654061222 >> 172.21.15.66:0/654061222 conn(0x7fb1f8069f60 msgr2=0x7fb1f815d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:24.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 -- 172.21.15.66:0/654061222 shutdown_connections 2024-03-30T00:46:24.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.174+0000 7fb1fcc00640 1 -- 172.21.15.66:0/654061222 wait complete. 2024-03-30T00:46:24.202 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:46:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.334+0000 7f37e1000640 1 Processor -- start 2024-03-30T00:46:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.334+0000 7f37e1000640 1 -- start start 2024-03-30T00:46:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37e1000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f37dc15a640 0x7f37dc06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37e1000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f37dc0ce520 0x7f37dc0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37e1000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37dc0cabb0 0x7f37dc0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37e1000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37dc06d070 con 0x7f37dc0ce520 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37e1000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37dc0cbba0 con 0x7f37dc15a640 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37e1000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc0cee90 con 0x7f37dc0cabb0 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37d9400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f37dc0ce520 0x7f37dc0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37d9400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f37dc0ce520 0x7f37dc0ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54448/0 (socket says 172.21.15.66:54448) 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37d9400640 1 -- 172.21.15.66:0/1508742531 learned_addr learned my addr 172.21.15.66:0/1508742531 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37da800640 1 --2- 172.21.15.66:0/1508742531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37dc0cabb0 0x7f37dc0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37d9400640 1 -- 172.21.15.66:0/1508742531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37dc0cabb0 msgr2=0x7f37dc0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37d9e00640 1 --2- 172.21.15.66:0/1508742531 >> v2:172.21.15.156:3300/0 conn(0x7f37dc15a640 0x7f37dc06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37d9400640 1 --2- 172.21.15.66:0/1508742531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37dc0cabb0 0x7f37dc0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37d9400640 1 -- 172.21.15.66:0/1508742531 >> v2:172.21.15.156:3300/0 conn(0x7f37dc15a640 msgr2=0x7f37dc06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:24.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37d9400640 1 --2- 172.21.15.66:0/1508742531 >> v2:172.21.15.156:3300/0 conn(0x7f37dc15a640 0x7f37dc06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37d9400640 1 -- 172.21.15.66:0/1508742531 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc06e2c0 con 0x7f37dc0ce520 2024-03-30T00:46:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37da800640 1 --2- 172.21.15.66:0/1508742531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37dc0cabb0 0x7f37dc0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:24.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.338+0000 7f37d9e00640 1 --2- 172.21.15.66:0/1508742531 >> v2:172.21.15.156:3300/0 conn(0x7f37dc15a640 0x7f37dc06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:24.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37d9400640 1 --2- 172.21.15.66:0/1508742531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37dc0ce520 0x7f37dc0ce950 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0cf160 tx=0x7f37c801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa2a5c0edd816f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:24.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37d8a00640 1 -- 172.21.15.66:0/1508742531 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c80aa610 con 0x7f37dc0ce520 2024-03-30T00:46:24.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37d8a00640 1 -- 172.21.15.66:0/1508742531 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c80a6040 con 0x7f37dc0ce520 2024-03-30T00:46:24.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37d8a00640 1 -- 172.21.15.66:0/1508742531 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c80aa990 con 0x7f37dc0ce520 2024-03-30T00:46:24.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 -- 172.21.15.66:0/1508742531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37dc0ce520 msgr2=0x7f37dc0ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 --2- 172.21.15.66:0/1508742531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37dc0ce520 0x7f37dc0ce950 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0cf160 tx=0x7f37c801c4f0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 -- 172.21.15.66:0/1508742531 shutdown_connections 2024-03-30T00:46:24.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 --2- 172.21.15.66:0/1508742531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37dc0cabb0 0x7f37dc0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 --2- 172.21.15.66:0/1508742531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37dc0ce520 0x7f37dc0ce950 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 --2- 172.21.15.66:0/1508742531 >> v2:172.21.15.156:3300/0 conn(0x7f37dc15a640 0x7f37dc06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 -- 172.21.15.66:0/1508742531 >> 172.21.15.66:0/1508742531 conn(0x7f37dc069f60 msgr2=0x7f37dc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:24.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 -- 172.21.15.66:0/1508742531 shutdown_connections 2024-03-30T00:46:24.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 -- 172.21.15.66:0/1508742531 wait complete. 2024-03-30T00:46:24.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 Processor -- start 2024-03-30T00:46:24.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 -- start start 2024-03-30T00:46:24.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37dc0cabb0 0x7f37dc0c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37dc0ce520 0x7f37dc1640b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37dc15a640 0x7f37dc0ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc0afc00 con 0x7f37dc0cabb0 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc0aeb60 con 0x7f37dc0ce520 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37e1000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc0aee40 con 0x7f37dc15a640 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.342+0000 7f37d9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37dc0cabb0 0x7f37dc0c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37dc0cabb0 0x7f37dc0c3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54464/0 (socket says 172.21.15.66:54464) 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d9e00640 1 -- 172.21.15.66:0/4249416893 learned_addr learned my addr 172.21.15.66:0/4249416893 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37da800640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37dc15a640 0x7f37dc0ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d9400640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37dc0ce520 0x7f37dc1640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d9e00640 1 -- 172.21.15.66:0/4249416893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37dc15a640 msgr2=0x7f37dc0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d9e00640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37dc15a640 0x7f37dc0ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d9e00640 1 -- 172.21.15.66:0/4249416893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37dc0ce520 msgr2=0x7f37dc1640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d9e00640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37dc0ce520 0x7f37dc1640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d9e00640 1 -- 172.21.15.66:0/4249416893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c80094d0 con 0x7f37dc0cabb0 2024-03-30T00:46:24.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d9400640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37dc0ce520 0x7f37dc1640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:24.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d9e00640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37dc0cabb0 0x7f37dc0c3d10 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00c7f0 tx=0x7f37cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:24.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d2000640 1 -- 172.21.15.66:0/4249416893 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc007820 con 0x7f37dc0cabb0 2024-03-30T00:46:24.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d2000640 1 -- 172.21.15.66:0/4249416893 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37cc007980 con 0x7f37dc0cabb0 2024-03-30T00:46:24.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d2000640 1 -- 172.21.15.66:0/4249416893 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc007d30 con 0x7f37dc0cabb0 2024-03-30T00:46:24.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37e1000640 1 -- 172.21.15.66:0/4249416893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37dc0af120 con 0x7f37dc0cabb0 2024-03-30T00:46:24.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37e1000640 1 -- 172.21.15.66:0/4249416893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37dc0af6e0 con 0x7f37dc0cabb0 2024-03-30T00:46:24.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37e1000640 1 -- 172.21.15.66:0/4249416893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37dc066c30 con 0x7f37dc0cabb0 2024-03-30T00:46:24.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d2000640 1 -- 172.21.15.66:0/4249416893 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f37cc024480 con 0x7f37dc0cabb0 2024-03-30T00:46:24.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.346+0000 7f37d2000640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37bc066690 0x7f37bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.350+0000 7f37d2000640 1 -- 172.21.15.66:0/4249416893 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f37cc084b40 con 0x7f37dc0cabb0 2024-03-30T00:46:24.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.350+0000 7f37d9400640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37bc066690 0x7f37bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.350+0000 7f37d9400640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37bc066690 0x7f37bc068b50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f37c80ae000 tx=0x7f37c80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:24.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.350+0000 7f37d2000640 1 -- 172.21.15.66:0/4249416893 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f37cc051d50 con 0x7f37dc0cabb0 2024-03-30T00:46:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 -- 172.21.15.66:0/4249416893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37dc0caf90 con 0x7f37dc0cabb0 2024-03-30T00:46:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37d2000640 1 -- 172.21.15.66:0/4249416893 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37cc01f020 con 0x7f37dc0cabb0 2024-03-30T00:46:24.519 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 -- 172.21.15.66:0/4249416893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37bc066690 msgr2=0x7f37bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37bc066690 0x7f37bc068b50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f37c80ae000 tx=0x7f37c80a7000 comp rx=0 tx=0).stop 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 -- 172.21.15.66:0/4249416893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37dc0cabb0 msgr2=0x7f37dc0c3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37dc0cabb0 0x7f37dc0c3d10 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00c7f0 tx=0x7f37cc00ccb0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 -- 172.21.15.66:0/4249416893 shutdown_connections 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37dc15a640 0x7f37dc0ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37dc0ce520 0x7f37dc1640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f37bc066690 0x7f37bc068b50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 --2- 172.21.15.66:0/4249416893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37dc0cabb0 0x7f37dc0c3d10 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 -- 172.21.15.66:0/4249416893 >> 172.21.15.66:0/4249416893 conn(0x7f37dc069f60 msgr2=0x7f37dc158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 -- 172.21.15.66:0/4249416893 shutdown_connections 2024-03-30T00:46:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.510+0000 7f37e1000640 1 -- 172.21.15.66:0/4249416893 wait complete. 2024-03-30T00:46:24.540 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:46:24.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.674+0000 7fa4b5a00640 1 Processor -- start 2024-03-30T00:46:24.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.674+0000 7fa4b5a00640 1 -- start start 2024-03-30T00:46:24.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4b5a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4b00cec30 0x7fa4b0159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4b5a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b00cd3e0 con 0x7fa4b00cda90 2024-03-30T00:46:24.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4b5a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b015a990 con 0x7fa4b00cde70 2024-03-30T00:46:24.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4b5a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b00b1b80 con 0x7fa4b00cec30 2024-03-30T00:46:24.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4aea00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa4b00cda90 0x7fa4b00ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55594/0 (socket says 172.21.15.66:55594) 2024-03-30T00:46:24.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4aea00640 1 -- 172.21.15.66:0/378015072 learned_addr learned my addr 172.21.15.66:0/378015072 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:24.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4ae000640 1 --2- 172.21.15.66:0/378015072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4b00cec30 0x7fa4b0159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4ad600640 1 -- 172.21.15.66:0/378015072 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599527682 0 0) 0x7fa4b00cd3e0 con 0x7fa4b00cda90 2024-03-30T00:46:24.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4ad600640 1 -- 172.21.15.66:0/378015072 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa494003530 con 0x7fa4b00cda90 2024-03-30T00:46:24.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4ad600640 1 -- 172.21.15.66:0/378015072 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219369630 0 0) 0x7fa4b015a990 con 0x7fa4b00cde70 2024-03-30T00:46:24.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4ad600640 1 -- 172.21.15.66:0/378015072 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b00cd3e0 con 0x7fa4b00cde70 2024-03-30T00:46:24.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4ae000640 1 -- 172.21.15.66:0/378015072 >> v1:172.21.15.156:6789/0 conn(0x7fa4b00cde70 legacy=0x7fa4b015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4ae000640 1 -- 172.21.15.66:0/378015072 >> v1:172.21.15.66:6789/0 conn(0x7fa4b00cda90 legacy=0x7fa4b00ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.678+0000 7fa4ae000640 1 -- 172.21.15.66:0/378015072 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b00b11e0 con 0x7fa4b00cec30 2024-03-30T00:46:24.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4ae000640 1 --2- 172.21.15.66:0/378015072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4b00cec30 0x7fa4b0159c60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0097f0 tx=0x7fa49c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b3850a5af8eacbf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:24.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4ad600640 1 -- 172.21.15.66:0/378015072 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c0ab650 con 0x7fa4b00cec30 2024-03-30T00:46:24.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4ad600640 1 -- 172.21.15.66:0/378015072 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49c0ab7b0 con 0x7fa4b00cec30 2024-03-30T00:46:24.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4ad600640 1 -- 172.21.15.66:0/378015072 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c0abb20 con 0x7fa4b00cec30 2024-03-30T00:46:24.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 -- 172.21.15.66:0/378015072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4b00cec30 msgr2=0x7fa4b0159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 --2- 172.21.15.66:0/378015072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4b00cec30 0x7fa4b0159c60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0097f0 tx=0x7fa49c01c450 comp rx=0 tx=0).stop 2024-03-30T00:46:24.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 -- 172.21.15.66:0/378015072 shutdown_connections 2024-03-30T00:46:24.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 --2- 172.21.15.66:0/378015072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4b00cec30 0x7fa4b0159c60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 -- 172.21.15.66:0/378015072 >> 172.21.15.66:0/378015072 conn(0x7fa4b0069f60 msgr2=0x7fa4b006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:24.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 -- 172.21.15.66:0/378015072 shutdown_connections 2024-03-30T00:46:24.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 -- 172.21.15.66:0/378015072 wait complete. 2024-03-30T00:46:24.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 Processor -- start 2024-03-30T00:46:24.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 -- start start 2024-03-30T00:46:24.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b00cda90 0x7fa4b01560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4b00cde70 0x7fa4b00ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4b00cec30 0x7fa4b00cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b0159730 con 0x7fa4b00cda90 2024-03-30T00:46:24.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b0158d30 con 0x7fa4b00cec30 2024-03-30T00:46:24.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.682+0000 7fa4b5a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b0158770 con 0x7fa4b00cde70 2024-03-30T00:46:24.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4aea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b00cda90 0x7fa4b01560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4aea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b00cda90 0x7fa4b01560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54478/0 (socket says 172.21.15.66:54478) 2024-03-30T00:46:24.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4aea00640 1 -- 172.21.15.66:0/966946383 learned_addr learned my addr 172.21.15.66:0/966946383 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:24.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4ae000640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4b00cde70 0x7fa4b00ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4af400640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4b00cec30 0x7fa4b00cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4aea00640 1 -- 172.21.15.66:0/966946383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4b00cde70 msgr2=0x7fa4b00ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4aea00640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4b00cde70 0x7fa4b00ab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4aea00640 1 -- 172.21.15.66:0/966946383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4b00cec30 msgr2=0x7fa4b00cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4aea00640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4b00cec30 0x7fa4b00cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4aea00640 1 -- 172.21.15.66:0/966946383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c009420 con 0x7fa4b00cda90 2024-03-30T00:46:24.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4af400640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4b00cec30 0x7fa4b00cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:24.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4aea00640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b00cda90 0x7fa4b01560b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000c4d0 tx=0x7fa4a000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:24.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4a6a00640 1 -- 172.21.15.66:0/966946383 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0005cf0 con 0x7fa4b00cda90 2024-03-30T00:46:24.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4a6a00640 1 -- 172.21.15.66:0/966946383 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a0005e50 con 0x7fa4b00cda90 2024-03-30T00:46:24.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4a6a00640 1 -- 172.21.15.66:0/966946383 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a00072b0 con 0x7fa4b00cda90 2024-03-30T00:46:24.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4b5a00640 1 -- 172.21.15.66:0/966946383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b0158a50 con 0x7fa4b00cda90 2024-03-30T00:46:24.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4b5a00640 1 -- 172.21.15.66:0/966946383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b0162aa0 con 0x7fa4b00cda90 2024-03-30T00:46:24.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4b5a00640 1 -- 172.21.15.66:0/966946383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b0162620 con 0x7fa4b00cda90 2024-03-30T00:46:24.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.686+0000 7fa4a6a00640 1 -- 172.21.15.66:0/966946383 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa4a0007410 con 0x7fa4b00cda90 2024-03-30T00:46:24.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.690+0000 7fa4a6a00640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa4900666c0 0x7fa490068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:24.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.690+0000 7fa4a6a00640 1 -- 172.21.15.66:0/966946383 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa4a0083890 con 0x7fa4b00cda90 2024-03-30T00:46:24.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.690+0000 7fa4ae000640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa4900666c0 0x7fa490068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:24.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.690+0000 7fa4ae000640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa4900666c0 0x7fa490068b80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0af000 tx=0x7fa49c0b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:24.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.694+0000 7fa4a6a00640 1 -- 172.21.15.66:0/966946383 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa4b0162620 con 0x7fa4b00cda90 2024-03-30T00:46:24.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 -- 172.21.15.66:0/966946383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4b0162620 con 0x7fa4b00cda90 2024-03-30T00:46:24.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4a6a00640 1 -- 172.21.15.66:0/966946383 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4b0162620 con 0x7fa4b00cda90 2024-03-30T00:46:24.863 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:46:24.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 -- 172.21.15.66:0/966946383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa4900666c0 msgr2=0x7fa490068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa4900666c0 0x7fa490068b80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0af000 tx=0x7fa49c0b6040 comp rx=0 tx=0).stop 2024-03-30T00:46:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 -- 172.21.15.66:0/966946383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b00cda90 msgr2=0x7fa4b01560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b00cda90 0x7fa4b01560b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000c4d0 tx=0x7fa4a000c990 comp rx=0 tx=0).stop 2024-03-30T00:46:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 -- 172.21.15.66:0/966946383 shutdown_connections 2024-03-30T00:46:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4b00cec30 0x7fa4b00cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4b00cde70 0x7fa4b00ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa4900666c0 0x7fa490068b80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 --2- 172.21.15.66:0/966946383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b00cda90 0x7fa4b01560b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:24.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 -- 172.21.15.66:0/966946383 >> 172.21.15.66:0/966946383 conn(0x7fa4b0069f60 msgr2=0x7fa4b00cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:24.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 -- 172.21.15.66:0/966946383 shutdown_connections 2024-03-30T00:46:24.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:24.854+0000 7fa4b5a00640 1 -- 172.21.15.66:0/966946383 wait complete. 2024-03-30T00:46:24.885 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:46:25.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd01be00640 1 Processor -- start 2024-03-30T00:46:25.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd01be00640 1 -- start start 2024-03-30T00:46:25.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd01be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd01415de30 0x7fd0140cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd01be00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd0140af420 0x7fd0140d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd01be00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0140cbb90 con 0x7fd0140ce740 2024-03-30T00:46:25.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd01be00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0140ce1b0 con 0x7fd01415de30 2024-03-30T00:46:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd01be00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0140afa10 con 0x7fd0140af420 2024-03-30T00:46:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd019400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd01415de30 0x7fd0140cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd019400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd01415de30 0x7fd0140cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54486/0 (socket says 172.21.15.66:54486) 2024-03-30T00:46:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd019400640 1 -- 172.21.15.66:0/875474101 learned_addr learned my addr 172.21.15.66:0/875474101 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd018a00640 1 --2- 172.21.15.66:0/875474101 >> v2:172.21.15.156:3300/0 conn(0x7fd0140af420 0x7fd0140d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd019400640 1 -- 172.21.15.66:0/875474101 >> v2:172.21.15.156:3300/0 conn(0x7fd0140af420 msgr2=0x7fd0140d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd019400640 1 --2- 172.21.15.66:0/875474101 >> v2:172.21.15.156:3300/0 conn(0x7fd0140af420 0x7fd0140d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd019400640 1 -- 172.21.15.66:0/875474101 >> v1:172.21.15.156:6789/0 conn(0x7fd0140ce740 legacy=0x7fd01406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:25.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.018+0000 7fd019400640 1 -- 172.21.15.66:0/875474101 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0140cf430 con 0x7fd01415de30 2024-03-30T00:46:25.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd019400640 1 --2- 172.21.15.66:0/875474101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd01415de30 0x7fd0140cc040 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd014070080 tx=0x7fd00801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743d9e58a1d8ce8f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:25.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd013e00640 1 -- 172.21.15.66:0/875474101 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0080a5740 con 0x7fd01415de30 2024-03-30T00:46:25.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd013e00640 1 -- 172.21.15.66:0/875474101 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0080a58a0 con 0x7fd01415de30 2024-03-30T00:46:25.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd013e00640 1 -- 172.21.15.66:0/875474101 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0080a5c30 con 0x7fd01415de30 2024-03-30T00:46:25.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd01be00640 1 -- 172.21.15.66:0/875474101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd01415de30 msgr2=0x7fd0140cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd01be00640 1 --2- 172.21.15.66:0/875474101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd01415de30 0x7fd0140cc040 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd014070080 tx=0x7fd00801c920 comp rx=0 tx=0).stop 2024-03-30T00:46:25.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd01be00640 1 -- 172.21.15.66:0/875474101 shutdown_connections 2024-03-30T00:46:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd01be00640 1 --2- 172.21.15.66:0/875474101 >> v2:172.21.15.156:3300/0 conn(0x7fd0140af420 0x7fd0140d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd01be00640 1 --2- 172.21.15.66:0/875474101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd01415de30 0x7fd0140cc040 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd01be00640 1 -- 172.21.15.66:0/875474101 >> 172.21.15.66:0/875474101 conn(0x7fd0140d0540 msgr2=0x7fd0140d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd01be00640 1 -- 172.21.15.66:0/875474101 shutdown_connections 2024-03-30T00:46:25.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.022+0000 7fd01be00640 1 -- 172.21.15.66:0/875474101 wait complete. 2024-03-30T00:46:25.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd01be00640 1 Processor -- start 2024-03-30T00:46:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd01be00640 1 -- start start 2024-03-30T00:46:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd01be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0140af420 0x7fd014156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd01be00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0140ce740 0x7fd0140c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd01be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd01415de30 0x7fd0140ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd01be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01415a0e0 con 0x7fd0140af420 2024-03-30T00:46:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd01be00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0141594d0 con 0x7fd0140ce740 2024-03-30T00:46:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd01be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0141597b0 con 0x7fd01415de30 2024-03-30T00:46:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd019400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0140af420 0x7fd014156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd019400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0140af420 0x7fd014156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54496/0 (socket says 172.21.15.66:54496) 2024-03-30T00:46:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd019400640 1 -- 172.21.15.66:0/722935078 learned_addr learned my addr 172.21.15.66:0/722935078 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:25.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd018a00640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0140ce740 0x7fd0140c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd019e00640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd01415de30 0x7fd0140ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd019400640 1 -- 172.21.15.66:0/722935078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd01415de30 msgr2=0x7fd0140ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd019400640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd01415de30 0x7fd0140ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd019400640 1 -- 172.21.15.66:0/722935078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0140ce740 msgr2=0x7fd0140c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd019400640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0140ce740 0x7fd0140c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd019400640 1 -- 172.21.15.66:0/722935078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd008009b30 con 0x7fd0140af420 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd018a00640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0140ce740 0x7fd0140c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd019400640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0140af420 0x7fd014156bc0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd0080af040 tx=0x7fd0080196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd011600640 1 -- 172.21.15.66:0/722935078 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0080b8670 con 0x7fd0140af420 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd011600640 1 -- 172.21.15.66:0/722935078 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0080b87d0 con 0x7fd0140af420 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd011600640 1 -- 172.21.15.66:0/722935078 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0080b8b80 con 0x7fd0140af420 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd01be00640 1 -- 172.21.15.66:0/722935078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd014159a90 con 0x7fd0140af420 2024-03-30T00:46:25.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd01be00640 1 -- 172.21.15.66:0/722935078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0140c5830 con 0x7fd0140af420 2024-03-30T00:46:25.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.026+0000 7fd01be00640 1 -- 172.21.15.66:0/722935078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0140cdf60 con 0x7fd0140af420 2024-03-30T00:46:25.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.030+0000 7fd011600640 1 -- 172.21.15.66:0/722935078 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd0080b8ce0 con 0x7fd0140af420 2024-03-30T00:46:25.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.030+0000 7fd011600640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcff4066690 0x7fcff4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.030+0000 7fd011600640 1 -- 172.21.15.66:0/722935078 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd00811fa00 con 0x7fd0140af420 2024-03-30T00:46:25.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.030+0000 7fd018a00640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcff4066690 0x7fcff4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.030+0000 7fd018a00640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcff4066690 0x7fcff4068b50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd0140b0050 tx=0x7fd000008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:25.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.034+0000 7fd011600640 1 -- 172.21.15.66:0/722935078 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd0080ecde0 con 0x7fd0140af420 2024-03-30T00:46:25.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.206+0000 7fd01be00640 1 -- 172.21.15.66:0/722935078 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd014066c30 con 0x7fd0140af420 2024-03-30T00:46:25.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.206+0000 7fd011600640 1 -- 172.21.15.66:0/722935078 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v95) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd0080bc080 con 0x7fd0140af420 2024-03-30T00:46:25.215 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:46:25.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.206+0000 7fd01be00640 1 -- 172.21.15.66:0/722935078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcff4066690 msgr2=0x7fcff4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.206+0000 7fd01be00640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcff4066690 0x7fcff4068b50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd0140b0050 tx=0x7fd000008040 comp rx=0 tx=0).stop 2024-03-30T00:46:25.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.206+0000 7fd01be00640 1 -- 172.21.15.66:0/722935078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0140af420 msgr2=0x7fd014156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.206+0000 7fd01be00640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0140af420 0x7fd014156bc0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd0080af040 tx=0x7fd0080196c0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.210+0000 7fd01be00640 1 -- 172.21.15.66:0/722935078 shutdown_connections 2024-03-30T00:46:25.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.210+0000 7fd01be00640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd01415de30 0x7fd0140ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.210+0000 7fd01be00640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0140ce740 0x7fd0140c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.210+0000 7fd01be00640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcff4066690 0x7fcff4068b50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.210+0000 7fd01be00640 1 --2- 172.21.15.66:0/722935078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0140af420 0x7fd014156bc0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.210+0000 7fd01be00640 1 -- 172.21.15.66:0/722935078 >> 172.21.15.66:0/722935078 conn(0x7fd0140d0540 msgr2=0x7fd01415b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:25.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.210+0000 7fd01be00640 1 -- 172.21.15.66:0/722935078 shutdown_connections 2024-03-30T00:46:25.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.210+0000 7fd01be00640 1 -- 172.21.15.66:0/722935078 wait complete. 2024-03-30T00:46:25.236 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:46:25.237 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:46:25.245 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:46:25.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.426+0000 7fd28a400640 1 Processor -- start 2024-03-30T00:46:25.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.426+0000 7fd28a400640 1 -- start start 2024-03-30T00:46:25.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.426+0000 7fd28a400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2840cda90 0x7fd2840ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd28a400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd2840cde70 0x7fd2840cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd28a400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28415a990 con 0x7fd284159c60 2024-03-30T00:46:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd28a400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2840b1b80 con 0x7fd2840cde70 2024-03-30T00:46:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd28a400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840cd3e0 con 0x7fd2840cda90 2024-03-30T00:46:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd282a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd2840cde70 0x7fd2840cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd283e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd284159c60 0x7fd28415a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55600/0 (socket says 172.21.15.66:55600) 2024-03-30T00:46:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd283e00640 1 -- 172.21.15.66:0/1204226419 learned_addr learned my addr 172.21.15.66:0/1204226419 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd283400640 1 --2- 172.21.15.66:0/1204226419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2840cda90 0x7fd2840ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd282a00640 1 -- 172.21.15.66:0/1204226419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2840cda90 msgr2=0x7fd2840ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd282a00640 1 --2- 172.21.15.66:0/1204226419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2840cda90 0x7fd2840ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd282a00640 1 -- 172.21.15.66:0/1204226419 >> v1:172.21.15.66:6789/0 conn(0x7fd284159c60 legacy=0x7fd28415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd282a00640 1 -- 172.21.15.66:0/1204226419 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2840b11e0 con 0x7fd2840cde70 2024-03-30T00:46:25.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd282a00640 1 --2- 172.21.15.66:0/1204226419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2840cde70 0x7fd2840cea60 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fd2700097c0 tx=0x7fd27001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40b45a4cfe95d29 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd282000640 1 -- 172.21.15.66:0/1204226419 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700ab600 con 0x7fd2840cde70 2024-03-30T00:46:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd282000640 1 -- 172.21.15.66:0/1204226419 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2700a7050 con 0x7fd2840cde70 2024-03-30T00:46:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.430+0000 7fd282000640 1 -- 172.21.15.66:0/1204226419 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700ab980 con 0x7fd2840cde70 2024-03-30T00:46:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 -- 172.21.15.66:0/1204226419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2840cde70 msgr2=0x7fd2840cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 --2- 172.21.15.66:0/1204226419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2840cde70 0x7fd2840cea60 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fd2700097c0 tx=0x7fd27001c410 comp rx=0 tx=0).stop 2024-03-30T00:46:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 -- 172.21.15.66:0/1204226419 shutdown_connections 2024-03-30T00:46:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 --2- 172.21.15.66:0/1204226419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2840cde70 0x7fd2840cea60 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 --2- 172.21.15.66:0/1204226419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2840cda90 0x7fd2840ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 -- 172.21.15.66:0/1204226419 >> 172.21.15.66:0/1204226419 conn(0x7fd284069f60 msgr2=0x7fd28406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:25.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 -- 172.21.15.66:0/1204226419 shutdown_connections 2024-03-30T00:46:25.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 -- 172.21.15.66:0/1204226419 wait complete. 2024-03-30T00:46:25.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 Processor -- start 2024-03-30T00:46:25.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 -- start start 2024-03-30T00:46:25.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2840cda90 0x7fd284097870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2840cde70 0x7fd284150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd284159c60 0x7fd2840c1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd284097db0 con 0x7fd2840cde70 2024-03-30T00:46:25.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd284150b60 con 0x7fd284159c60 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd28a400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd284150e30 con 0x7fd2840cda90 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd282a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2840cde70 0x7fd284150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd282a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2840cde70 0x7fd284150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54520/0 (socket says 172.21.15.66:54520) 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd282a00640 1 -- 172.21.15.66:0/123373186 learned_addr learned my addr 172.21.15.66:0/123373186 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd283e00640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd284159c60 0x7fd2840c1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd283400640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2840cda90 0x7fd284097870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd282a00640 1 -- 172.21.15.66:0/123373186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2840cda90 msgr2=0x7fd284097870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd282a00640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2840cda90 0x7fd284097870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd282a00640 1 -- 172.21.15.66:0/123373186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd284159c60 msgr2=0x7fd2840c1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd282a00640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd284159c60 0x7fd2840c1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd282a00640 1 -- 172.21.15.66:0/123373186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd270009420 con 0x7fd2840cde70 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.434+0000 7fd283e00640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd284159c60 0x7fd2840c1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:25.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.438+0000 7fd282a00640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2840cde70 0x7fd284150620 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd2700ae040 tx=0x7fd2700191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:25.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.438+0000 7fd27b400640 1 -- 172.21.15.66:0/123373186 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700b4610 con 0x7fd2840cde70 2024-03-30T00:46:25.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.438+0000 7fd27b400640 1 -- 172.21.15.66:0/123373186 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2700abe90 con 0x7fd2840cde70 2024-03-30T00:46:25.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.438+0000 7fd28a400640 1 -- 172.21.15.66:0/123373186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd284096ab0 con 0x7fd2840cde70 2024-03-30T00:46:25.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.438+0000 7fd27b400640 1 -- 172.21.15.66:0/123373186 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700b49d0 con 0x7fd2840cde70 2024-03-30T00:46:25.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.438+0000 7fd28a400640 1 -- 172.21.15.66:0/123373186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd284097070 con 0x7fd2840cde70 2024-03-30T00:46:25.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.438+0000 7fd28a400640 1 -- 172.21.15.66:0/123373186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28415bf10 con 0x7fd2840cde70 2024-03-30T00:46:25.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.438+0000 7fd27b400640 1 -- 172.21.15.66:0/123373186 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd2700b2040 con 0x7fd2840cde70 2024-03-30T00:46:25.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.438+0000 7fd27b400640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2640666c0 0x7fd264068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.438+0000 7fd27b400640 1 -- 172.21.15.66:0/123373186 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd27011e2c0 con 0x7fd2840cde70 2024-03-30T00:46:25.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.438+0000 7fd283400640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2640666c0 0x7fd264068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.442+0000 7fd283400640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2640666c0 0x7fd264068b80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd27400b930 tx=0x7fd274008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:25.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.442+0000 7fd27b400640 1 -- 172.21.15.66:0/123373186 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd2700eb450 con 0x7fd2840cde70 2024-03-30T00:46:25.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 -- 172.21.15.66:0/123373186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd284066c30 con 0x7fd2840cde70 2024-03-30T00:46:25.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd27b400640 1 -- 172.21.15.66:0/123373186 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16249 (secure 0 0 0) 0x7fd2700f3e60 con 0x7fd2840cde70 2024-03-30T00:46:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:25.618 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":95,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:46:16.274340+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:46:13.811742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-30T00:46:14.265142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:46:25.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 -- 172.21.15.66:0/123373186 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2640666c0 msgr2=0x7fd264068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2640666c0 0x7fd264068b80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd27400b930 tx=0x7fd274008040 comp rx=0 tx=0).stop 2024-03-30T00:46:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 -- 172.21.15.66:0/123373186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2840cde70 msgr2=0x7fd284150620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2840cde70 0x7fd284150620 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd2700ae040 tx=0x7fd2700191b0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 -- 172.21.15.66:0/123373186 shutdown_connections 2024-03-30T00:46:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd284159c60 0x7fd2840c1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd2640666c0 0x7fd264068b80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2840cde70 0x7fd284150620 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 --2- 172.21.15.66:0/123373186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd2840cda90 0x7fd284097870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 -- 172.21.15.66:0/123373186 >> 172.21.15.66:0/123373186 conn(0x7fd284069f60 msgr2=0x7fd2840cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 -- 172.21.15.66:0/123373186 shutdown_connections 2024-03-30T00:46:25.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.610+0000 7fd28a400640 1 -- 172.21.15.66:0/123373186 wait complete. 2024-03-30T00:46:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:46:13.811742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:46:14.265142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:46:25.640 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:46:25.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.774+0000 7f4073400640 1 Processor -- start 2024-03-30T00:46:25.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.774+0000 7f4073400640 1 -- start start 2024-03-30T00:46:25.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.774+0000 7f4073400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f406c15ca10 0x7f406c06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.774+0000 7f4073400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f406c0ae4e0 0x7f406c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.774+0000 7f4073400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c06e7d0 con 0x7f406c0ce820 2024-03-30T00:46:25.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.774+0000 7f4073400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c0b1f70 con 0x7f406c0ae4e0 2024-03-30T00:46:25.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.774+0000 7f4073400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c06f1a0 con 0x7f406c15ca10 2024-03-30T00:46:25.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.774+0000 7f406be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f406c0ae4e0 0x7f406c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.774+0000 7f406be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f406c0ae4e0 0x7f406c0cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54536/0 (socket says 172.21.15.66:54536) 2024-03-30T00:46:25.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.774+0000 7f406be00640 1 -- 172.21.15.66:0/1344941991 learned_addr learned my addr 172.21.15.66:0/1344941991 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:25.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.778+0000 7f4070a00640 1 --2- 172.21.15.66:0/1344941991 >> v2:172.21.15.156:3300/0 conn(0x7f406c15ca10 0x7f406c06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.778+0000 7f406be00640 1 -- 172.21.15.66:0/1344941991 >> v2:172.21.15.156:3300/0 conn(0x7f406c15ca10 msgr2=0x7f406c06ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:25.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.778+0000 7f406be00640 1 --2- 172.21.15.66:0/1344941991 >> v2:172.21.15.156:3300/0 conn(0x7f406c15ca10 0x7f406c06ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.778+0000 7f406be00640 1 -- 172.21.15.66:0/1344941991 >> v1:172.21.15.156:6789/0 conn(0x7f406c0ce820 legacy=0x7f406c160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.778+0000 7f406be00640 1 -- 172.21.15.66:0/1344941991 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c15cdf0 con 0x7f406c0ae4e0 2024-03-30T00:46:25.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.778+0000 7f406be00640 1 --2- 172.21.15.66:0/1344941991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f406c0ae4e0 0x7f406c0cb620 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f406c06f9f0 tx=0x7f405801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8ff5736237ce48 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:25.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.778+0000 7f406b400640 1 -- 172.21.15.66:0/1344941991 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40580aa650 con 0x7f406c0ae4e0 2024-03-30T00:46:25.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.778+0000 7f406b400640 1 -- 172.21.15.66:0/1344941991 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40580aa7b0 con 0x7f406c0ae4e0 2024-03-30T00:46:25.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.778+0000 7f406b400640 1 -- 172.21.15.66:0/1344941991 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40580aab40 con 0x7f406c0ae4e0 2024-03-30T00:46:25.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 -- 172.21.15.66:0/1344941991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f406c0ae4e0 msgr2=0x7f406c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 --2- 172.21.15.66:0/1344941991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f406c0ae4e0 0x7f406c0cb620 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f406c06f9f0 tx=0x7f405801c4f0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 -- 172.21.15.66:0/1344941991 shutdown_connections 2024-03-30T00:46:25.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 --2- 172.21.15.66:0/1344941991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f406c0ae4e0 0x7f406c0cb620 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 --2- 172.21.15.66:0/1344941991 >> v2:172.21.15.156:3300/0 conn(0x7f406c15ca10 0x7f406c06ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 -- 172.21.15.66:0/1344941991 >> 172.21.15.66:0/1344941991 conn(0x7f406c098a20 msgr2=0x7f406c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:25.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 -- 172.21.15.66:0/1344941991 shutdown_connections 2024-03-30T00:46:25.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 -- 172.21.15.66:0/1344941991 wait complete. 2024-03-30T00:46:25.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 Processor -- start 2024-03-30T00:46:25.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 -- start start 2024-03-30T00:46:25.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c0ae4e0 0x7f406c156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f406c0ce820 0x7f406c0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f406c15ca10 0x7f406c0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0b4070 con 0x7f406c0ce820 2024-03-30T00:46:25.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0b4350 con 0x7f406c0ae4e0 2024-03-30T00:46:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4073400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0b4630 con 0x7f406c15ca10 2024-03-30T00:46:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f406be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f406c0ce820 0x7f406c0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f406be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f406c0ce820 0x7f406c0ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54546/0 (socket says 172.21.15.66:54546) 2024-03-30T00:46:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f406be00640 1 -- 172.21.15.66:0/378748502 learned_addr learned my addr 172.21.15.66:0/378748502 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4070a00640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c0ae4e0 0x7f406c156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4071400640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f406c15ca10 0x7f406c0ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f406be00640 1 -- 172.21.15.66:0/378748502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f406c15ca10 msgr2=0x7f406c0ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f406be00640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f406c15ca10 0x7f406c0ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f406be00640 1 -- 172.21.15.66:0/378748502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c0ae4e0 msgr2=0x7f406c156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f406be00640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c0ae4e0 0x7f406c156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f406be00640 1 -- 172.21.15.66:0/378748502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40580094d0 con 0x7f406c0ce820 2024-03-30T00:46:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.782+0000 7f4070a00640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c0ae4e0 0x7f406c156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:46:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.786+0000 7f406be00640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f406c0ce820 0x7f406c0ad130 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f40580ac040 tx=0x7f405801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.786+0000 7f4068c00640 1 -- 172.21.15.66:0/378748502 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40580b3800 con 0x7f406c0ce820 2024-03-30T00:46:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.786+0000 7f4068c00640 1 -- 172.21.15.66:0/378748502 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40580b3960 con 0x7f406c0ce820 2024-03-30T00:46:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.786+0000 7f4068c00640 1 -- 172.21.15.66:0/378748502 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40580b3d10 con 0x7f406c0ce820 2024-03-30T00:46:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.786+0000 7f4073400640 1 -- 172.21.15.66:0/378748502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406c0b4910 con 0x7f406c0ce820 2024-03-30T00:46:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.786+0000 7f4073400640 1 -- 172.21.15.66:0/378748502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c156dc0 con 0x7f406c0ce820 2024-03-30T00:46:25.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.786+0000 7f4073400640 1 -- 172.21.15.66:0/378748502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406c0b4770 con 0x7f406c0ce820 2024-03-30T00:46:25.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.786+0000 7f4068c00640 1 -- 172.21.15.66:0/378748502 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f40580b2380 con 0x7f406c0ce820 2024-03-30T00:46:25.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.786+0000 7f4068c00640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f404c066690 0x7f404c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:25.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.786+0000 7f4068c00640 1 -- 172.21.15.66:0/378748502 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f405811ea10 con 0x7f406c0ce820 2024-03-30T00:46:25.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.786+0000 7f4070a00640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f404c066690 0x7f404c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:25.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.790+0000 7f4070a00640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f404c066690 0x7f404c068b50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4060009f50 tx=0x7f4060009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:25.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.790+0000 7f4068c00640 1 -- 172.21.15.66:0/378748502 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f406c0b4770 con 0x7f406c0ce820 2024-03-30T00:46:25.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.954+0000 7f4073400640 1 -- 172.21.15.66:0/378748502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f406c0b4770 con 0x7f406c0ce820 2024-03-30T00:46:25.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.954+0000 7f4068c00640 1 -- 172.21.15.66:0/378748502 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f406c0b4770 con 0x7f406c0ce820 2024-03-30T00:46:25.963 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:46:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 -- 172.21.15.66:0/378748502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f404c066690 msgr2=0x7f404c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f404c066690 0x7f404c068b50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4060009f50 tx=0x7f4060009190 comp rx=0 tx=0).stop 2024-03-30T00:46:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 -- 172.21.15.66:0/378748502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f406c0ce820 msgr2=0x7f406c0ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f406c0ce820 0x7f406c0ad130 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f40580ac040 tx=0x7f405801ad30 comp rx=0 tx=0).stop 2024-03-30T00:46:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 -- 172.21.15.66:0/378748502 shutdown_connections 2024-03-30T00:46:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f406c15ca10 0x7f406c0ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f404c066690 0x7f404c068b50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f406c0ce820 0x7f406c0ad130 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 --2- 172.21.15.66:0/378748502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f406c0ae4e0 0x7f406c156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 -- 172.21.15.66:0/378748502 >> 172.21.15.66:0/378748502 conn(0x7f406c098a20 msgr2=0x7f406c0ac2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 -- 172.21.15.66:0/378748502 shutdown_connections 2024-03-30T00:46:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:25.958+0000 7f4073400640 1 -- 172.21.15.66:0/378748502 wait complete. 2024-03-30T00:46:25.985 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:46:26.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.118+0000 7f1455e00640 1 Processor -- start 2024-03-30T00:46:26.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.118+0000 7f1455e00640 1 -- start start 2024-03-30T00:46:26.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.118+0000 7f1455e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f14500cca40 0x7f14500cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.118+0000 7f1455e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f145006b140 0x7f145006b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.118+0000 7f1455e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1450099250 0x7f14500996a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.118+0000 7f1455e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145006bac0 con 0x7f14500cca40 2024-03-30T00:46:26.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.118+0000 7f1455e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1450098ac0 con 0x7f1450099250 2024-03-30T00:46:26.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.118+0000 7f1455e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14500cf3f0 con 0x7f145006b140 2024-03-30T00:46:26.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f14500cca40 0x7f14500cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f14500cca40 0x7f14500cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54550/0 (socket says 172.21.15.66:54550) 2024-03-30T00:46:26.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144ea00640 1 -- 172.21.15.66:0/4027123274 learned_addr learned my addr 172.21.15.66:0/4027123274 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:26.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144f400640 1 --2- 172.21.15.66:0/4027123274 >> v2:172.21.15.156:3300/0 conn(0x7f1450099250 0x7f14500996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f1447e00640 1 --2- 172.21.15.66:0/4027123274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f145006b140 0x7f145006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144ea00640 1 -- 172.21.15.66:0/4027123274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f145006b140 msgr2=0x7f145006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:26.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144ea00640 1 --2- 172.21.15.66:0/4027123274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f145006b140 0x7f145006b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144ea00640 1 -- 172.21.15.66:0/4027123274 >> v2:172.21.15.156:3300/0 conn(0x7f1450099250 msgr2=0x7f14500996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:26.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144ea00640 1 --2- 172.21.15.66:0/4027123274 >> v2:172.21.15.156:3300/0 conn(0x7f1450099250 0x7f14500996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144ea00640 1 -- 172.21.15.66:0/4027123274 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14500cf670 con 0x7f14500cca40 2024-03-30T00:46:26.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144ea00640 1 --2- 172.21.15.66:0/4027123274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14500cca40 0x7f14500cac90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f14500b2ef0 tx=0x7f144001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31391cac742f4527 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:26.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144e000640 1 -- 172.21.15.66:0/4027123274 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14400aa610 con 0x7f14500cca40 2024-03-30T00:46:26.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144e000640 1 -- 172.21.15.66:0/4027123274 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14400a6040 con 0x7f14500cca40 2024-03-30T00:46:26.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.122+0000 7f144e000640 1 -- 172.21.15.66:0/4027123274 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14400aa990 con 0x7f14500cca40 2024-03-30T00:46:26.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 -- 172.21.15.66:0/4027123274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14500cca40 msgr2=0x7f14500cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 --2- 172.21.15.66:0/4027123274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14500cca40 0x7f14500cac90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f14500b2ef0 tx=0x7f144001c4f0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 -- 172.21.15.66:0/4027123274 shutdown_connections 2024-03-30T00:46:26.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 --2- 172.21.15.66:0/4027123274 >> v2:172.21.15.156:3300/0 conn(0x7f1450099250 0x7f14500996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 --2- 172.21.15.66:0/4027123274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f145006b140 0x7f145006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 --2- 172.21.15.66:0/4027123274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14500cca40 0x7f14500cac90 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 -- 172.21.15.66:0/4027123274 >> 172.21.15.66:0/4027123274 conn(0x7f14500b2130 msgr2=0x7f14500b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:26.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 -- 172.21.15.66:0/4027123274 shutdown_connections 2024-03-30T00:46:26.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 -- 172.21.15.66:0/4027123274 wait complete. 2024-03-30T00:46:26.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 Processor -- start 2024-03-30T00:46:26.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 -- start start 2024-03-30T00:46:26.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f145006b140 0x7f1450155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1450099250 0x7f1450159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14500cca40 0x7f145006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14500c7ed0 con 0x7f14500cca40 2024-03-30T00:46:26.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14500c7bf0 con 0x7f1450099250 2024-03-30T00:46:26.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1455e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14500b5a30 con 0x7f145006b140 2024-03-30T00:46:26.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f144f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14500cca40 0x7f145006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f144f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14500cca40 0x7f145006e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54564/0 (socket says 172.21.15.66:54564) 2024-03-30T00:46:26.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f144f400640 1 -- 172.21.15.66:0/1365385672 learned_addr learned my addr 172.21.15.66:0/1365385672 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:26.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f144ea00640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f145006b140 0x7f1450155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f1447e00640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1450099250 0x7f1450159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f144f400640 1 -- 172.21.15.66:0/1365385672 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f145006b140 msgr2=0x7f1450155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f144f400640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f145006b140 0x7f1450155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f144f400640 1 -- 172.21.15.66:0/1365385672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1450099250 msgr2=0x7f1450159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f144f400640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1450099250 0x7f1450159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.126+0000 7f144f400640 1 -- 172.21.15.66:0/1365385672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14400094d0 con 0x7f14500cca40 2024-03-30T00:46:26.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.130+0000 7f144f400640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14500cca40 0x7f145006e320 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f1448007e40 tx=0x7f144800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:26.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.130+0000 7f1447400640 1 -- 172.21.15.66:0/1365385672 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448014620 con 0x7f14500cca40 2024-03-30T00:46:26.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.130+0000 7f1447400640 1 -- 172.21.15.66:0/1365385672 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144800ee90 con 0x7f14500cca40 2024-03-30T00:46:26.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.130+0000 7f1447400640 1 -- 172.21.15.66:0/1365385672 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14480149e0 con 0x7f14500cca40 2024-03-30T00:46:26.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.130+0000 7f1455e00640 1 -- 172.21.15.66:0/1365385672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14500b5d10 con 0x7f14500cca40 2024-03-30T00:46:26.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.130+0000 7f1455e00640 1 -- 172.21.15.66:0/1365385672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14500b5ff0 con 0x7f14500cca40 2024-03-30T00:46:26.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.130+0000 7f1455e00640 1 -- 172.21.15.66:0/1365385672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14500677d0 con 0x7f14500cca40 2024-03-30T00:46:26.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.130+0000 7f1447400640 1 -- 172.21.15.66:0/1365385672 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1448014b40 con 0x7f14500cca40 2024-03-30T00:46:26.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.130+0000 7f1447400640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f142c066690 0x7f142c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.130+0000 7f1447400640 1 -- 172.21.15.66:0/1365385672 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1448084bd0 con 0x7f14500cca40 2024-03-30T00:46:26.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.130+0000 7f144ea00640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f142c066690 0x7f142c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.134+0000 7f144ea00640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f142c066690 0x7f142c068b50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f14400ae000 tx=0x7f14400b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:26.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.134+0000 7f1447400640 1 -- 172.21.15.66:0/1365385672 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1448051e10 con 0x7f14500cca40 2024-03-30T00:46:26.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.294+0000 7f1455e00640 1 -- 172.21.15.66:0/1365385672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14500b5890 con 0x7f14500cca40 2024-03-30T00:46:26.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1447400640 1 -- 172.21.15.66:0/1365385672 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14500b5890 con 0x7f14500cca40 2024-03-30T00:46:26.305 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:46:26.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 -- 172.21.15.66:0/1365385672 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f142c066690 msgr2=0x7f142c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f142c066690 0x7f142c068b50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f14400ae000 tx=0x7f14400b5040 comp rx=0 tx=0).stop 2024-03-30T00:46:26.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 -- 172.21.15.66:0/1365385672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14500cca40 msgr2=0x7f145006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14500cca40 0x7f145006e320 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f1448007e40 tx=0x7f144800e4b0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 -- 172.21.15.66:0/1365385672 shutdown_connections 2024-03-30T00:46:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f142c066690 0x7f142c068b50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14500cca40 0x7f145006e320 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1450099250 0x7f1450159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 --2- 172.21.15.66:0/1365385672 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f145006b140 0x7f1450155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 -- 172.21.15.66:0/1365385672 >> 172.21.15.66:0/1365385672 conn(0x7f14500b2130 msgr2=0x7f1450099660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 -- 172.21.15.66:0/1365385672 shutdown_connections 2024-03-30T00:46:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.298+0000 7f1455e00640 1 -- 172.21.15.66:0/1365385672 wait complete. 2024-03-30T00:46:26.326 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:46:26.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.458+0000 7fcd58400640 1 Processor -- start 2024-03-30T00:46:26.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.458+0000 7fcd58400640 1 -- start start 2024-03-30T00:46:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.462+0000 7fcd58400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcd500cda90 0x7fcd500ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.462+0000 7fcd58400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd500cd3e0 con 0x7fcd5015a370 2024-03-30T00:46:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.462+0000 7fcd58400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5015a990 con 0x7fcd500cde70 2024-03-30T00:46:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.462+0000 7fcd58400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd500b1b80 con 0x7fcd500cda90 2024-03-30T00:46:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.462+0000 7fcd55a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcd500cda90 0x7fcd500ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.462+0000 7fcd55a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcd500cda90 0x7fcd500ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54580/0 (socket says 172.21.15.66:54580) 2024-03-30T00:46:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.462+0000 7fcd55a00640 1 -- 172.21.15.66:0/2023486642 learned_addr learned my addr 172.21.15.66:0/2023486642 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.462+0000 7fcd55a00640 1 -- 172.21.15.66:0/2023486642 >> v1:172.21.15.156:6789/0 conn(0x7fcd500cde70 legacy=0x7fcd50159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.462+0000 7fcd55a00640 1 -- 172.21.15.66:0/2023486642 >> v1:172.21.15.66:6789/0 conn(0x7fcd5015a370 legacy=0x7fcd50156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.462+0000 7fcd55a00640 1 -- 172.21.15.66:0/2023486642 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd500b11e0 con 0x7fcd500cda90 2024-03-30T00:46:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd55a00640 1 --2- 172.21.15.66:0/2023486642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd500cda90 0x7fcd500ce520 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009870 tx=0x7fcd4401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8baf40710ea4ce7d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd4fe00640 1 -- 172.21.15.66:0/2023486642 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd440ac640 con 0x7fcd500cda90 2024-03-30T00:46:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd4fe00640 1 -- 172.21.15.66:0/2023486642 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd440ac7a0 con 0x7fcd500cda90 2024-03-30T00:46:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd4fe00640 1 -- 172.21.15.66:0/2023486642 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd440acb30 con 0x7fcd500cda90 2024-03-30T00:46:26.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 -- 172.21.15.66:0/2023486642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd500cda90 msgr2=0x7fcd500ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 --2- 172.21.15.66:0/2023486642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd500cda90 0x7fcd500ce520 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009870 tx=0x7fcd4401c4d0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 -- 172.21.15.66:0/2023486642 shutdown_connections 2024-03-30T00:46:26.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 --2- 172.21.15.66:0/2023486642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd500cda90 0x7fcd500ce520 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 -- 172.21.15.66:0/2023486642 >> 172.21.15.66:0/2023486642 conn(0x7fcd50069f60 msgr2=0x7fcd5006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:26.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 -- 172.21.15.66:0/2023486642 shutdown_connections 2024-03-30T00:46:26.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 -- 172.21.15.66:0/2023486642 wait complete. 2024-03-30T00:46:26.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 Processor -- start 2024-03-30T00:46:26.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 -- start start 2024-03-30T00:46:26.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd500cda90 0x7fcd50097a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd500cde70 0x7fcd50162620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd5015a370 0x7fcd501566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd501576a0 con 0x7fcd500cde70 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500ac890 con 0x7fcd500cda90 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd58400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500acb70 con 0x7fcd5015a370 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.466+0000 7fcd55000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd500cde70 0x7fcd50162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd55000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd500cde70 0x7fcd50162620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54588/0 (socket says 172.21.15.66:54588) 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd55000640 1 -- 172.21.15.66:0/1614211978 learned_addr learned my addr 172.21.15.66:0/1614211978 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd55a00640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd500cda90 0x7fcd50097a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd56400640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd5015a370 0x7fcd501566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd55000640 1 -- 172.21.15.66:0/1614211978 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd5015a370 msgr2=0x7fcd501566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd55000640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd5015a370 0x7fcd501566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd55000640 1 -- 172.21.15.66:0/1614211978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd500cda90 msgr2=0x7fcd50097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd55000640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd500cda90 0x7fcd50097a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd55000640 1 -- 172.21.15.66:0/1614211978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440094d0 con 0x7fcd500cde70 2024-03-30T00:46:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd56400640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd5015a370 0x7fcd501566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:26.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd55000640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd500cde70 0x7fcd50162620 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00c4d0 tx=0x7fcd3c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:26.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd4d600640 1 -- 172.21.15.66:0/1614211978 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c0052d0 con 0x7fcd500cde70 2024-03-30T00:46:26.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd4d600640 1 -- 172.21.15.66:0/1614211978 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c005430 con 0x7fcd500cde70 2024-03-30T00:46:26.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd58400640 1 -- 172.21.15.66:0/1614211978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd500ace50 con 0x7fcd500cde70 2024-03-30T00:46:26.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd4d600640 1 -- 172.21.15.66:0/1614211978 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c012830 con 0x7fcd500cde70 2024-03-30T00:46:26.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd58400640 1 -- 172.21.15.66:0/1614211978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd500ac580 con 0x7fcd500cde70 2024-03-30T00:46:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd58400640 1 -- 172.21.15.66:0/1614211978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd50066c30 con 0x7fcd500cde70 2024-03-30T00:46:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd4d600640 1 -- 172.21.15.66:0/1614211978 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcd3c012990 con 0x7fcd500cde70 2024-03-30T00:46:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd4d600640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd300666c0 0x7fcd30068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd4d600640 1 -- 172.21.15.66:0/1614211978 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcd3c083d00 con 0x7fcd500cde70 2024-03-30T00:46:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.470+0000 7fcd55a00640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd300666c0 0x7fcd30068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.474+0000 7fcd55a00640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd300666c0 0x7fcd30068b80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcd440ae000 tx=0x7fcd440a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:26.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.474+0000 7fcd4d600640 1 -- 172.21.15.66:0/1614211978 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcd3c050f20 con 0x7fcd500cde70 2024-03-30T00:46:26.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.634+0000 7fcd58400640 1 -- 172.21.15.66:0/1614211978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd500b3170 con 0x7fcd500cde70 2024-03-30T00:46:26.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.634+0000 7fcd4d600640 1 -- 172.21.15.66:0/1614211978 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd3c054540 con 0x7fcd500cde70 2024-03-30T00:46:26.645 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:46:26.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 -- 172.21.15.66:0/1614211978 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd300666c0 msgr2=0x7fcd30068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd300666c0 0x7fcd30068b80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcd440ae000 tx=0x7fcd440a7000 comp rx=0 tx=0).stop 2024-03-30T00:46:26.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 -- 172.21.15.66:0/1614211978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd500cde70 msgr2=0x7fcd50162620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd500cde70 0x7fcd50162620 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00c4d0 tx=0x7fcd3c00c990 comp rx=0 tx=0).stop 2024-03-30T00:46:26.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 -- 172.21.15.66:0/1614211978 shutdown_connections 2024-03-30T00:46:26.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd5015a370 0x7fcd501566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd300666c0 0x7fcd30068b80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd500cde70 0x7fcd50162620 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 --2- 172.21.15.66:0/1614211978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd500cda90 0x7fcd50097a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 -- 172.21.15.66:0/1614211978 >> 172.21.15.66:0/1614211978 conn(0x7fcd50069f60 msgr2=0x7fcd500afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:26.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 -- 172.21.15.66:0/1614211978 shutdown_connections 2024-03-30T00:46:26.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.638+0000 7fcd58400640 1 -- 172.21.15.66:0/1614211978 wait complete. 2024-03-30T00:46:26.666 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:46:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.798+0000 7efc35a00640 1 Processor -- start 2024-03-30T00:46:26.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.798+0000 7efc35a00640 1 -- start start 2024-03-30T00:46:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc35a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efc300cec30 0x7efc30159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc35a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efc300cde70 0x7efc3015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc35a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3015a6e0 con 0x7efc300cda90 2024-03-30T00:46:26.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc35a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc300b1b80 con 0x7efc300cec30 2024-03-30T00:46:26.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc35a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc300cd3e0 con 0x7efc300cde70 2024-03-30T00:46:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc27e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efc300cec30 0x7efc30159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc27e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efc300cec30 0x7efc30159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54598/0 (socket says 172.21.15.66:54598) 2024-03-30T00:46:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc27e00640 1 -- 172.21.15.66:0/1649618764 learned_addr learned my addr 172.21.15.66:0/1649618764 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc2f400640 1 --2- 172.21.15.66:0/1649618764 >> v2:172.21.15.156:3300/0 conn(0x7efc300cde70 0x7efc3015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc27e00640 1 -- 172.21.15.66:0/1649618764 >> v2:172.21.15.156:3300/0 conn(0x7efc300cde70 msgr2=0x7efc3015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc27e00640 1 --2- 172.21.15.66:0/1649618764 >> v2:172.21.15.156:3300/0 conn(0x7efc300cde70 0x7efc3015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc27e00640 1 -- 172.21.15.66:0/1649618764 >> v1:172.21.15.156:6789/0 conn(0x7efc300cda90 legacy=0x7efc300ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.802+0000 7efc27e00640 1 -- 172.21.15.66:0/1649618764 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3015a990 con 0x7efc300cec30 2024-03-30T00:46:26.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc27e00640 1 --2- 172.21.15.66:0/1649618764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc300cec30 0x7efc30159c60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efc300adcf0 tx=0x7efc1801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89c437f73b91f9f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:26.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc2e000640 1 -- 172.21.15.66:0/1649618764 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180aa5f0 con 0x7efc300cec30 2024-03-30T00:46:26.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc2e000640 1 -- 172.21.15.66:0/1649618764 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc180aa750 con 0x7efc300cec30 2024-03-30T00:46:26.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc2e000640 1 -- 172.21.15.66:0/1649618764 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180aaae0 con 0x7efc300cec30 2024-03-30T00:46:26.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 -- 172.21.15.66:0/1649618764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc300cec30 msgr2=0x7efc30159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 --2- 172.21.15.66:0/1649618764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc300cec30 0x7efc30159c60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efc300adcf0 tx=0x7efc1801c490 comp rx=0 tx=0).stop 2024-03-30T00:46:26.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 -- 172.21.15.66:0/1649618764 shutdown_connections 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 --2- 172.21.15.66:0/1649618764 >> v2:172.21.15.156:3300/0 conn(0x7efc300cde70 0x7efc3015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 --2- 172.21.15.66:0/1649618764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc300cec30 0x7efc30159c60 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 -- 172.21.15.66:0/1649618764 >> 172.21.15.66:0/1649618764 conn(0x7efc30069f60 msgr2=0x7efc3006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 -- 172.21.15.66:0/1649618764 shutdown_connections 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 -- 172.21.15.66:0/1649618764 wait complete. 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 Processor -- start 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 -- start start 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc300cda90 0x7efc30152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc300cde70 0x7efc30157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc300cec30 0x7efc3015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300adce0 con 0x7efc300cda90 2024-03-30T00:46:26.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc30164020 con 0x7efc300cde70 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.806+0000 7efc35a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc30157bb0 con 0x7efc300cec30 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc2ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc300cda90 0x7efc30152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc2ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc300cda90 0x7efc30152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54606/0 (socket says 172.21.15.66:54606) 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc2ea00640 1 -- 172.21.15.66:0/4180991159 learned_addr learned my addr 172.21.15.66:0/4180991159 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc2ea00640 1 -- 172.21.15.66:0/4180991159 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc300cec30 msgr2=0x7efc3015b1e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc2ea00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc300cec30 0x7efc3015b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc2ea00640 1 -- 172.21.15.66:0/4180991159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc300cde70 msgr2=0x7efc30157670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc2ea00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc300cde70 0x7efc30157670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc2ea00640 1 -- 172.21.15.66:0/4180991159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc180094d0 con 0x7efc300cda90 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc2ea00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc300cda90 0x7efc30152aa0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7efc2000c530 tx=0x7efc2000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc26a00640 1 -- 172.21.15.66:0/4180991159 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc200052d0 con 0x7efc300cda90 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc26a00640 1 -- 172.21.15.66:0/4180991159 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc20005450 con 0x7efc300cda90 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc26a00640 1 -- 172.21.15.66:0/4180991159 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20012870 con 0x7efc300cda90 2024-03-30T00:46:26.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc35a00640 1 -- 172.21.15.66:0/4180991159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc300ad940 con 0x7efc300cda90 2024-03-30T00:46:26.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc35a00640 1 -- 172.21.15.66:0/4180991159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc300af590 con 0x7efc300cda90 2024-03-30T00:46:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc35a00640 1 -- 172.21.15.66:0/4180991159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc300ad7a0 con 0x7efc300cda90 2024-03-30T00:46:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc26a00640 1 -- 172.21.15.66:0/4180991159 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efc200129d0 con 0x7efc300cda90 2024-03-30T00:46:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.810+0000 7efc26a00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc0c0666c0 0x7efc0c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.814+0000 7efc26a00640 1 -- 172.21.15.66:0/4180991159 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efc20083d00 con 0x7efc300cda90 2024-03-30T00:46:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.814+0000 7efc27e00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc0c0666c0 0x7efc0c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.814+0000 7efc27e00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc0c0666c0 0x7efc0c068b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efc180ae000 tx=0x7efc18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:26.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.814+0000 7efc26a00640 1 -- 172.21.15.66:0/4180991159 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efc300ad7a0 con 0x7efc300cda90 2024-03-30T00:46:26.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.986+0000 7efc35a00640 1 -- 172.21.15.66:0/4180991159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efc300ad7a0 con 0x7efc300cda90 2024-03-30T00:46:26.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc26a00640 1 -- 172.21.15.66:0/4180991159 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v95) v1 ==== 80+0+0 (secure 0 0 0) 0x7efc300ad7a0 con 0x7efc300cda90 2024-03-30T00:46:26.997 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:46:26.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 -- 172.21.15.66:0/4180991159 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc0c0666c0 msgr2=0x7efc0c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:26.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc0c0666c0 0x7efc0c068b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efc180ae000 tx=0x7efc18008040 comp rx=0 tx=0).stop 2024-03-30T00:46:26.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 -- 172.21.15.66:0/4180991159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc300cda90 msgr2=0x7efc30152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc300cda90 0x7efc30152aa0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7efc2000c530 tx=0x7efc2000c9f0 comp rx=0 tx=0).stop 2024-03-30T00:46:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 -- 172.21.15.66:0/4180991159 shutdown_connections 2024-03-30T00:46:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc300cec30 0x7efc3015b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc300cde70 0x7efc30157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efc0c0666c0 0x7efc0c068b80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 --2- 172.21.15.66:0/4180991159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc300cda90 0x7efc30152aa0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 -- 172.21.15.66:0/4180991159 >> 172.21.15.66:0/4180991159 conn(0x7efc30069f60 msgr2=0x7efc3015d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 -- 172.21.15.66:0/4180991159 shutdown_connections 2024-03-30T00:46:27.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:26.990+0000 7efc35a00640 1 -- 172.21.15.66:0/4180991159 wait complete. 2024-03-30T00:46:27.018 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:46:27.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:27.018 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:27.036 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[47490]: fuse finished with error 0 and tester_r 0 2024-03-30T00:46:27.039 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:46:33.141 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:46:33.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:33.141 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:46:33.153 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:46:33.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.338+0000 7fb5b3400640 1 Processor -- start 2024-03-30T00:46:33.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.338+0000 7fb5b3400640 1 -- start start 2024-03-30T00:46:33.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5b3400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb5ac1610c0 0x7fb5ac1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5b3400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ac06da20 con 0x7fb5ac1609f0 2024-03-30T00:46:33.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5b3400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ac06dc60 con 0x7fb5ac157060 2024-03-30T00:46:33.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5b3400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ac157440 con 0x7fb5ac1610c0 2024-03-30T00:46:33.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5b1400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb5ac1609f0 0x7fb5ac0abc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48858/0 (socket says 172.21.15.66:48858) 2024-03-30T00:46:33.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5b1400640 1 -- 172.21.15.66:0/863152518 learned_addr learned my addr 172.21.15.66:0/863152518 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:33.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5abe00640 1 --2- 172.21.15.66:0/863152518 >> v2:172.21.15.156:3300/0 conn(0x7fb5ac1610c0 0x7fb5ac1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838273430 0 0) 0x7fb5a40014a0 con 0x7fb5ac1609f0 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb594003530 con 0x7fb5ac1609f0 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3043267390 0 0) 0x7fb594003530 con 0x7fb5ac1609f0 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 >> v2:172.21.15.156:3300/0 conn(0x7fb5ac1610c0 msgr2=0x7fb5ac1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 --2- 172.21.15.66:0/863152518 >> v2:172.21.15.156:3300/0 conn(0x7fb5ac1610c0 0x7fb5ac1604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 >> v1:172.21.15.156:6789/0 conn(0x7fb5ac157060 legacy=0x7fb5ac06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac06df40 con 0x7fb5ac1609f0 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fb5a4001b90 con 0x7fb5ac1609f0 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 >> v1:172.21.15.66:6789/0 conn(0x7fb5ac1609f0 legacy=0x7fb5ac0abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 --2- 172.21.15.66:0/863152518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb594003980 0x7fb594003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 --2- 172.21.15.66:0/863152518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 0x7fb594004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 --2- 172.21.15.66:0/863152518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5940052b0 0x7fb59400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac157440 con 0x7fb5940047d0 2024-03-30T00:46:33.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59400c180 con 0x7fb594003980 2024-03-30T00:46:33.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59400c340 con 0x7fb5940052b0 2024-03-30T00:46:33.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5abe00640 1 --2- 172.21.15.66:0/863152518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 0x7fb594004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5b0a00640 1 --2- 172.21.15.66:0/863152518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb594003980 0x7fb594003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5b1400640 1 --2- 172.21.15.66:0/863152518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5940052b0 0x7fb59400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5abe00640 1 -- 172.21.15.66:0/863152518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5940052b0 msgr2=0x7fb59400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:33.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5abe00640 1 --2- 172.21.15.66:0/863152518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5940052b0 0x7fb59400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5abe00640 1 -- 172.21.15.66:0/863152518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb594003980 msgr2=0x7fb594003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:33.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5abe00640 1 --2- 172.21.15.66:0/863152518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb594003980 0x7fb594003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5abe00640 1 -- 172.21.15.66:0/863152518 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb594003530 con 0x7fb5940047d0 2024-03-30T00:46:33.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.342+0000 7fb5b1400640 1 -- 172.21.15.66:0/863152518 reap_dead start 2024-03-30T00:46:33.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5abe00640 1 --2- 172.21.15.66:0/863152518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 0x7fb594004c50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac1614c0 tx=0x7fb59801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd93339c9ff91208 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:33.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5980a8450 con 0x7fb5940047d0 2024-03-30T00:46:33.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5980a85b0 con 0x7fb5940047d0 2024-03-30T00:46:33.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5ab400640 1 -- 172.21.15.66:0/863152518 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5980a8960 con 0x7fb5940047d0 2024-03-30T00:46:33.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5b3400640 1 -- 172.21.15.66:0/863152518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 msgr2=0x7fb594004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:33.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5b3400640 1 --2- 172.21.15.66:0/863152518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 0x7fb594004c50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac1614c0 tx=0x7fb59801ca10 comp rx=0 tx=0).stop 2024-03-30T00:46:33.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5b3400640 1 -- 172.21.15.66:0/863152518 shutdown_connections 2024-03-30T00:46:33.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5b3400640 1 --2- 172.21.15.66:0/863152518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 0x7fb594004c50 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5b3400640 1 -- 172.21.15.66:0/863152518 >> 172.21.15.66:0/863152518 conn(0x7fb5ac071750 msgr2=0x7fb5ac069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:33.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5b3400640 1 -- 172.21.15.66:0/863152518 shutdown_connections 2024-03-30T00:46:33.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5b3400640 1 -- 172.21.15.66:0/863152518 wait complete. 2024-03-30T00:46:33.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5b3400640 1 Processor -- start 2024-03-30T00:46:33.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.346+0000 7fb5b3400640 1 -- start start 2024-03-30T00:46:33.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 0x7fb5ac137ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b3400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac150650 0x7fb5ac0ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b3400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5ac150ad0 0x7fb5ac0acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b3400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac14f8f0 con 0x7fb5940047d0 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b3400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac13b3c0 con 0x7fb5ac150650 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b3400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac14fb70 con 0x7fb5ac150ad0 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 0x7fb5ac137ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 0x7fb5ac137ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45792/0 (socket says 172.21.15.66:45792) 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b0a00640 1 -- 172.21.15.66:0/1955908771 learned_addr learned my addr 172.21.15.66:0/1955908771 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5abe00640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac150650 0x7fb5ac0ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b1400640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5ac150ad0 0x7fb5ac0acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b0a00640 1 -- 172.21.15.66:0/1955908771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5ac150ad0 msgr2=0x7fb5ac0acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b0a00640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5ac150ad0 0x7fb5ac0acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b0a00640 1 -- 172.21.15.66:0/1955908771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac150650 msgr2=0x7fb5ac0ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:33.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b0a00640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac150650 0x7fb5ac0ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b0a00640 1 -- 172.21.15.66:0/1955908771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac14fe50 con 0x7fb5940047d0 2024-03-30T00:46:33.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5abe00640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac150650 0x7fb5ac0ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:33.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b0a00640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 0x7fb5ac137ea0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000f7f0 tx=0x7fb5a000fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:33.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5a8c00640 1 -- 172.21.15.66:0/1955908771 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a00050d0 con 0x7fb5940047d0 2024-03-30T00:46:33.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5a8c00640 1 -- 172.21.15.66:0/1955908771 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a0005230 con 0x7fb5940047d0 2024-03-30T00:46:33.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b3400640 1 -- 172.21.15.66:0/1955908771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ac150130 con 0x7fb5940047d0 2024-03-30T00:46:33.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5a8c00640 1 -- 172.21.15.66:0/1955908771 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000d730 con 0x7fb5940047d0 2024-03-30T00:46:33.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b3400640 1 -- 172.21.15.66:0/1955908771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ac1387d0 con 0x7fb5940047d0 2024-03-30T00:46:33.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5b3400640 1 -- 172.21.15.66:0/1955908771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ac066c30 con 0x7fb5940047d0 2024-03-30T00:46:33.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.350+0000 7fb5a8c00640 1 -- 172.21.15.66:0/1955908771 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb5a000d890 con 0x7fb5940047d0 2024-03-30T00:46:33.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.354+0000 7fb5a8c00640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb58c0666e0 0x7fb58c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.354+0000 7fb5a8c00640 1 -- 172.21.15.66:0/1955908771 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb5a00841c0 con 0x7fb5940047d0 2024-03-30T00:46:33.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.354+0000 7fb5abe00640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb58c0666e0 0x7fb58c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.354+0000 7fb5abe00640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb58c0666e0 0x7fb58c068ba0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb5980ad000 tx=0x7fb5980b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:33.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.358+0000 7fb5a8c00640 1 -- 172.21.15.66:0/1955908771 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb5a00514e0 con 0x7fb5940047d0 2024-03-30T00:46:33.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 -- 172.21.15.66:0/1955908771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5ac071f30 con 0x7fb5940047d0 2024-03-30T00:46:33.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5a8c00640 1 -- 172.21.15.66:0/1955908771 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fb5a0054b00 con 0x7fb5940047d0 2024-03-30T00:46:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:33.562 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:46:18.278734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6836/1147664160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1147664160},{"type":"v1","addr":"172.21.15.66:6836","nonce":1147664160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:46:33.562 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 58 2024-03-30T00:46:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 -- 172.21.15.66:0/1955908771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb58c0666e0 msgr2=0x7fb58c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb58c0666e0 0x7fb58c068ba0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb5980ad000 tx=0x7fb5980b2040 comp rx=0 tx=0).stop 2024-03-30T00:46:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 -- 172.21.15.66:0/1955908771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 msgr2=0x7fb5ac137ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 0x7fb5ac137ea0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000f7f0 tx=0x7fb5a000fcb0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 -- 172.21.15.66:0/1955908771 shutdown_connections 2024-03-30T00:46:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5ac150ad0 0x7fb5ac0acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac150650 0x7fb5ac0ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb58c0666e0 0x7fb58c068ba0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 --2- 172.21.15.66:0/1955908771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5940047d0 0x7fb5ac137ea0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 -- 172.21.15.66:0/1955908771 >> 172.21.15.66:0/1955908771 conn(0x7fb5ac071750 msgr2=0x7fb5ac0af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 -- 172.21.15.66:0/1955908771 shutdown_connections 2024-03-30T00:46:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.554+0000 7fb5b3400640 1 -- 172.21.15.66:0/1955908771 wait complete. 2024-03-30T00:46:33.584 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-30T00:46:33.757 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-03-30T00:46:33.757 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-30T00:46:33.757 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:46:33.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f10e00640 1 Processor -- start 2024-03-30T00:46:33.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f10e00640 1 -- start start 2024-03-30T00:46:33.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f10e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f0c0cdb60 0x7f3f0c15be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f10e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3f0c15c380 0x7f3f0c0b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f10e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0ce0d0 con 0x7f3f0c0cc3f0 2024-03-30T00:46:33.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f10e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c1551d0 con 0x7f3f0c15c380 2024-03-30T00:46:33.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f10e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c065e30 con 0x7f3f0c0cdb60 2024-03-30T00:46:33.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f09e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f3f0c0cc3f0 0x7f3f0c0ae110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48866/0 (socket says 172.21.15.66:48866) 2024-03-30T00:46:33.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f09e00640 1 -- 172.21.15.66:0/27722910 learned_addr learned my addr 172.21.15.66:0/27722910 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:33.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f0a800640 1 --2- 172.21.15.66:0/27722910 >> v2:172.21.15.66:3300/0 conn(0x7f3f0c15c380 0x7f3f0c0b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f03e00640 1 --2- 172.21.15.66:0/27722910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f0c0cdb60 0x7f3f0c15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f09400640 1 -- 172.21.15.66:0/27722910 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715642340 0 0) 0x7f3f0c0ce0d0 con 0x7f3f0c0cc3f0 2024-03-30T00:46:33.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f0a800640 1 -- 172.21.15.66:0/27722910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f0c0cdb60 msgr2=0x7f3f0c15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:33.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f0a800640 1 --2- 172.21.15.66:0/27722910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f0c0cdb60 0x7f3f0c15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f0a800640 1 -- 172.21.15.66:0/27722910 >> v1:172.21.15.66:6789/0 conn(0x7f3f0c0cc3f0 legacy=0x7f3f0c0ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:33.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f0a800640 1 -- 172.21.15.66:0/27722910 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0ce3b0 con 0x7f3f0c15c380 2024-03-30T00:46:33.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.890+0000 7f3f03e00640 1 --2- 172.21.15.66:0/27722910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f0c0cdb60 0x7f3f0c15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:33.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f0a800640 1 --2- 172.21.15.66:0/27722910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0c15c380 0x7f3f0c0b3f70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009d90 tx=0x7f3f0401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969927eee7b73e1b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:33.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f09400640 1 -- 172.21.15.66:0/27722910 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040a5830 con 0x7f3f0c15c380 2024-03-30T00:46:33.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f09400640 1 -- 172.21.15.66:0/27722910 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f040a5990 con 0x7f3f0c15c380 2024-03-30T00:46:33.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f09400640 1 -- 172.21.15.66:0/27722910 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040a5d20 con 0x7f3f0c15c380 2024-03-30T00:46:33.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f10e00640 1 -- 172.21.15.66:0/27722910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0c15c380 msgr2=0x7f3f0c0b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:33.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f10e00640 1 --2- 172.21.15.66:0/27722910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0c15c380 0x7f3f0c0b3f70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009d90 tx=0x7f3f0401c970 comp rx=0 tx=0).stop 2024-03-30T00:46:33.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f10e00640 1 -- 172.21.15.66:0/27722910 shutdown_connections 2024-03-30T00:46:33.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f10e00640 1 --2- 172.21.15.66:0/27722910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0c15c380 0x7f3f0c0b3f70 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f10e00640 1 --2- 172.21.15.66:0/27722910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f0c0cdb60 0x7f3f0c15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f10e00640 1 -- 172.21.15.66:0/27722910 >> 172.21.15.66:0/27722910 conn(0x7f3f0c071750 msgr2=0x7f3f0c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:33.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f10e00640 1 -- 172.21.15.66:0/27722910 shutdown_connections 2024-03-30T00:46:33.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.894+0000 7f3f10e00640 1 -- 172.21.15.66:0/27722910 wait complete. 2024-03-30T00:46:33.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f10e00640 1 Processor -- start 2024-03-30T00:46:33.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f10e00640 1 -- start start 2024-03-30T00:46:33.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f10e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f0c0cc3f0 0x7f3f0c14d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f10e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f0c0cdb60 0x7f3f0c146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f10e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0c15c380 0x7f3f0c146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f10e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c146c80 con 0x7f3f0c15c380 2024-03-30T00:46:33.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f10e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c06cc00 con 0x7f3f0c0cdb60 2024-03-30T00:46:33.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f10e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c06cee0 con 0x7f3f0c0cc3f0 2024-03-30T00:46:33.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f0a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0c15c380 0x7f3f0c146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f0a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0c15c380 0x7f3f0c146740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45798/0 (socket says 172.21.15.66:45798) 2024-03-30T00:46:33.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f0a800640 1 -- 172.21.15.66:0/3788718213 learned_addr learned my addr 172.21.15.66:0/3788718213 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:33.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f09e00640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f0c0cc3f0 0x7f3f0c14d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f03e00640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f0c0cdb60 0x7f3f0c146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f0a800640 1 -- 172.21.15.66:0/3788718213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f0c0cc3f0 msgr2=0x7f3f0c14d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f0a800640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f0c0cc3f0 0x7f3f0c14d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f0a800640 1 -- 172.21.15.66:0/3788718213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f0c0cdb60 msgr2=0x7f3f0c146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f0a800640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f0c0cdb60 0x7f3f0c146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f0a800640 1 -- 172.21.15.66:0/3788718213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04009a70 con 0x7f3f0c15c380 2024-03-30T00:46:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f0a800640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0c15c380 0x7f3f0c146740 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3f040b0040 tx=0x7f3f04019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f02000640 1 -- 172.21.15.66:0/3788718213 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040ad850 con 0x7f3f0c15c380 2024-03-30T00:46:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f02000640 1 -- 172.21.15.66:0/3788718213 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f040ad9b0 con 0x7f3f0c15c380 2024-03-30T00:46:33.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f02000640 1 -- 172.21.15.66:0/3788718213 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040add80 con 0x7f3f0c15c380 2024-03-30T00:46:33.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f10e00640 1 -- 172.21.15.66:0/3788718213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c06d1c0 con 0x7f3f0c15c380 2024-03-30T00:46:33.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.898+0000 7f3f10e00640 1 -- 172.21.15.66:0/3788718213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c06d780 con 0x7f3f0c15c380 2024-03-30T00:46:33.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.902+0000 7f3f10e00640 1 -- 172.21.15.66:0/3788718213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c06d970 con 0x7f3f0c15c380 2024-03-30T00:46:33.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.902+0000 7f3f02000640 1 -- 172.21.15.66:0/3788718213 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3f040b4050 con 0x7f3f0c15c380 2024-03-30T00:46:33.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.902+0000 7f3f02000640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ee8066690 0x7f3ee8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:33.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.902+0000 7f3f02000640 1 -- 172.21.15.66:0/3788718213 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f3f0411f1c0 con 0x7f3f0c15c380 2024-03-30T00:46:33.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.902+0000 7f3f09e00640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ee8066690 0x7f3ee8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:33.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.902+0000 7f3f09e00640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ee8066690 0x7f3ee8068b50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3efc000f70 tx=0x7f3efc007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:33.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:33.906+0000 7f3f02000640 1 -- 172.21.15.66:0/3788718213 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3f040ec490 con 0x7f3f0c15c380 2024-03-30T00:46:34.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.098+0000 7f3f10e00640 1 -- 172.21.15.66:0/3788718213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3f0c066c30 con 0x7f3f0c15c380 2024-03-30T00:46:34.108 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:46:34.098+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:46:34.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.110+0000 7f3f02000640 1 -- 172.21.15.66:0/3788718213 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3f040efab0 con 0x7f3f0c15c380 2024-03-30T00:46:34.119 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:46:34.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 -- 172.21.15.66:0/3788718213 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ee8066690 msgr2=0x7f3ee8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ee8066690 0x7f3ee8068b50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3efc000f70 tx=0x7f3efc007b60 comp rx=0 tx=0).stop 2024-03-30T00:46:34.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 -- 172.21.15.66:0/3788718213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0c15c380 msgr2=0x7f3f0c146740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0c15c380 0x7f3f0c146740 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3f040b0040 tx=0x7f3f04019710 comp rx=0 tx=0).stop 2024-03-30T00:46:34.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 -- 172.21.15.66:0/3788718213 shutdown_connections 2024-03-30T00:46:34.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3ee8066690 0x7f3ee8068b50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0c15c380 0x7f3f0c146740 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f0c0cdb60 0x7f3f0c146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 --2- 172.21.15.66:0/3788718213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f0c0cc3f0 0x7f3f0c14d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 -- 172.21.15.66:0/3788718213 >> 172.21.15.66:0/3788718213 conn(0x7f3f0c071750 msgr2=0x7f3f0c153670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:34.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 -- 172.21.15.66:0/3788718213 shutdown_connections 2024-03-30T00:46:34.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.114+0000 7f3f10e00640 1 -- 172.21.15.66:0/3788718213 wait complete. 2024-03-30T00:46:34.128 INFO:tasks.ceph.mds.a.smithi066.stderr: -3924> 2024-03-30T00:46:16.782+0000 7f6422a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:46:34.142 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-03-30T00:46:34.177 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:46:34.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a59649080 1 Processor -- start 2024-03-30T00:46:34.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a59649080 1 -- start start 2024-03-30T00:46:34.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a59649080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564c32510dd0 0x564c325104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a59649080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c325b5780 0x564c32526790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a59649080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 0x564c3264c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a59649080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3248c1e0 con 0x564c325b5ed0 2024-03-30T00:46:34.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a59649080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c32427960 con 0x564c325b5780 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a59649080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c32427600 con 0x564c32510dd0 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a56600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 0x564c3264c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a56600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 0x564c3264c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45824/0 (socket says 172.21.15.66:45824) 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a56600640 1 -- 172.21.15.66:0/3728289734 learned_addr learned my addr 172.21.15.66:0/3728289734 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a55200640 1 --2- 172.21.15.66:0/3728289734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c325b5780 0x564c32526790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a55c00640 1 --2- 172.21.15.66:0/3728289734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564c32510dd0 0x564c325104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a56600640 1 -- 172.21.15.66:0/3728289734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564c32510dd0 msgr2=0x564c325104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a56600640 1 --2- 172.21.15.66:0/3728289734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564c32510dd0 0x564c325104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a56600640 1 -- 172.21.15.66:0/3728289734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c325b5780 msgr2=0x564c32526790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a56600640 1 --2- 172.21.15.66:0/3728289734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c325b5780 0x564c32526790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a56600640 1 -- 172.21.15.66:0/3728289734 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a44005d30 con 0x564c325b5ed0 2024-03-30T00:46:34.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.182+0000 7f7a56600640 1 --2- 172.21.15.66:0/3728289734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 0x564c3264c0a0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7a50006fc0 tx=0x7f7a50001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d8bd479b83049f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a4f400640 1 -- 172.21.15.66:0/3728289734 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000f510 con 0x564c325b5ed0 2024-03-30T00:46:34.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a4f400640 1 -- 172.21.15.66:0/3728289734 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5000f670 con 0x564c325b5ed0 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a4f400640 1 -- 172.21.15.66:0/3728289734 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000fa20 con 0x564c325b5ed0 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 -- 172.21.15.66:0/3728289734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 msgr2=0x564c3264c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 --2- 172.21.15.66:0/3728289734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 0x564c3264c0a0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7a50006fc0 tx=0x7f7a50001940 comp rx=0 tx=0).stop 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 -- 172.21.15.66:0/3728289734 shutdown_connections 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 --2- 172.21.15.66:0/3728289734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 0x564c3264c0a0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 --2- 172.21.15.66:0/3728289734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c325b5780 0x564c32526790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 --2- 172.21.15.66:0/3728289734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564c32510dd0 0x564c325104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 -- 172.21.15.66:0/3728289734 >> 172.21.15.66:0/3728289734 conn(0x564c3250b650 msgr2=0x564c325b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 -- 172.21.15.66:0/3728289734 shutdown_connections 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 -- 172.21.15.66:0/3728289734 wait complete. 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 Processor -- start 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 -- start start 2024-03-30T00:46:34.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c32510dd0 0x564c32526840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564c325b5780 0x564c325ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 0x564c325b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3241e690 con 0x564c325b5ed0 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c325b1020 con 0x564c32510dd0 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a59649080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c325b12c0 con 0x564c325b5780 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a56600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 0x564c325b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a56600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 0x564c325b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45840/0 (socket says 172.21.15.66:45840) 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a56600640 1 -- 172.21.15.66:0/251320489 learned_addr learned my addr 172.21.15.66:0/251320489 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a55c00640 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c32510dd0 0x564c32526840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a55200640 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564c325b5780 0x564c325ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a56600640 1 -- 172.21.15.66:0/251320489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564c325b5780 msgr2=0x564c325ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a56600640 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564c325b5780 0x564c325ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a56600640 1 -- 172.21.15.66:0/251320489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c32510dd0 msgr2=0x564c32526840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a56600640 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c32510dd0 0x564c32526840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a56600640 1 -- 172.21.15.66:0/251320489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c32450a60 con 0x564c325b5ed0 2024-03-30T00:46:34.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a55200640 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564c325b5780 0x564c325ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:34.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a56600640 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 0x564c325b4510 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f7a50014040 tx=0x7f7a50001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a4e000640 1 -- 172.21.15.66:0/251320489 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000f510 con 0x564c325b5ed0 2024-03-30T00:46:34.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a4e000640 1 -- 172.21.15.66:0/251320489 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5001e8b0 con 0x564c325b5ed0 2024-03-30T00:46:34.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.186+0000 7f7a4e000640 1 -- 172.21.15.66:0/251320489 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5001ece0 con 0x564c325b5ed0 2024-03-30T00:46:34.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.190+0000 7f7a59649080 1 -- 172.21.15.66:0/251320489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564c32429d00 con 0x564c325b5ed0 2024-03-30T00:46:34.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.190+0000 7f7a59649080 1 -- 172.21.15.66:0/251320489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c32429760 con 0x564c325b5ed0 2024-03-30T00:46:34.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.190+0000 7f7a4e000640 1 -- 172.21.15.66:0/251320489 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7a50011410 con 0x564c325b5ed0 2024-03-30T00:46:34.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.190+0000 7f7a4e000640 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7a380650d0 0x7f7a38067590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.190+0000 7f7a4e000640 1 -- 172.21.15.66:0/251320489 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f7a50084830 con 0x564c325b5ed0 2024-03-30T00:46:34.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.190+0000 7f7a55c00640 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7a380650d0 0x7f7a38067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.190+0000 7f7a59649080 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x564c3250a340 0x564c325adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.190+0000 7f7a59649080 1 -- 172.21.15.66:0/251320489 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x564c325ac610 con 0x564c3250a340 2024-03-30T00:46:34.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.190+0000 7f7a55200640 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x564c3250a340 0x564c325adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.190+0000 7f7a55c00640 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7a380650d0 0x7f7a38067590 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7a48000c10 tx=0x7f7a48010040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.190+0000 7f7a55200640 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x564c3250a340 0x564c325adf20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a55200640 1 -- 172.21.15.66:0/251320489 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v97'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7a440a5850 con 0x564c3250a340 2024-03-30T00:46:34.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a59649080 1 -- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x564c3250a340 msgr2=0x564c325adf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a59649080 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x564c3250a340 0x564c325adf20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a59649080 1 -- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7a380650d0 msgr2=0x7f7a38067590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a59649080 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7a380650d0 0x7f7a38067590 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7a48000c10 tx=0x7f7a48010040 comp rx=0 tx=0).stop 2024-03-30T00:46:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a59649080 1 -- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 msgr2=0x564c325b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a59649080 1 --2- 172.21.15.66:0/251320489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564c325b5ed0 0x564c325b4510 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f7a50014040 tx=0x7f7a50001260 comp rx=0 tx=0).stop 2024-03-30T00:46:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a56600640 1 -- 172.21.15.66:0/251320489 reap_dead start 2024-03-30T00:46:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a59649080 1 -- 172.21.15.66:0/251320489 shutdown_connections 2024-03-30T00:46:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a59649080 1 -- 172.21.15.66:0/251320489 >> 172.21.15.66:0/251320489 conn(0x564c3250b650 msgr2=0x564c325b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a59649080 1 -- 172.21.15.66:0/251320489 shutdown_connections 2024-03-30T00:46:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.194+0000 7f7a59649080 1 -- 172.21.15.66:0/251320489 wait complete. 2024-03-30T00:46:34.208 DEBUG:teuthology.orchestra.run.smithi066:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-03-30T00:46:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.286+0000 7f5ab47ab400 1 -- start start 2024-03-30T00:46:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.286+0000 7f5ab47ab400 1 Processor -- start 2024-03-30T00:46:34.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3a993800 0x559d3aa1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3a993400 0x559d3aa1eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3a993000 0x559d3aa1f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3aa60a00 con 0x559d3a993800 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3aa60780 con 0x559d3a993400 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d3aa60640 con 0x559d3a993000 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3a993800 0x559d3aa1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3a993800 0x559d3aa1e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45850/0 (socket says 172.21.15.66:45850) 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 -- 172.21.15.66:0/2861537345 learned_addr learned my addr 172.21.15.66:0/2861537345 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/2861537345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3a993400 0x559d3aa1eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab3200640 1 --2- 172.21.15.66:0/2861537345 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3a993000 0x559d3aa1f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 -- 172.21.15.66:0/2861537345 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3a993000 msgr2=0x559d3aa1f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 --2- 172.21.15.66:0/2861537345 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3a993000 0x559d3aa1f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 -- 172.21.15.66:0/2861537345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3a993400 msgr2=0x559d3aa1eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 --2- 172.21.15.66:0/2861537345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3a993400 0x559d3aa1eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 -- 172.21.15.66:0/2861537345 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3a9c89c0 con 0x559d3a993800 2024-03-30T00:46:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/2861537345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3a993400 0x559d3aa1eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:34.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 --2- 172.21.15.66:0/2861537345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3a993800 0x559d3aa1e580 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x559d3b802d50 tx=0x559d3b89cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e479485b1b1937cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5aaf600640 1 -- 172.21.15.66:0/2861537345 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559d3a9e7600 con 0x559d3a993800 2024-03-30T00:46:34.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5aaf600640 1 -- 172.21.15.66:0/2861537345 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559d3a9f0480 con 0x559d3a993800 2024-03-30T00:46:34.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5aaf600640 1 -- 172.21.15.66:0/2861537345 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559d3a9e71e0 con 0x559d3a993800 2024-03-30T00:46:34.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2861537345 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d3a9c8b60 con 0x559d3a993800 2024-03-30T00:46:34.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2861537345 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559d3a9c8340 con 0x559d3a993800 2024-03-30T00:46:34.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 4 waiting for MDS map... 2024-03-30T00:46:34.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5aaf600640 1 -- 172.21.15.66:0/2861537345 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x559d3a9d2c40 con 0x559d3a993800 2024-03-30T00:46:34.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5aaf600640 1 -- 172.21.15.66:0/2861537345 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 59) v1 ==== 2573+0+0 (secure 0 0 0) 0x559d3a9e0f00 con 0x559d3a993800 2024-03-30T00:46:34.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 4 Got MDS map 59 2024-03-30T00:46:34.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 10 main: JournalTool::main 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 4 main: JournalTool: connecting to RADOS... 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 Processor -- start 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 -- start start 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3b8aac00 0x559d3aa1f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab000 0x559d3aa1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3b8ab400 0x559d3aa20100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a9d9e00 con 0x559d3b8ab000 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3aa60780 con 0x559d3b8ab400 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d3aa60500 con 0x559d3b8aac00 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab000 0x559d3aa1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab000 0x559d3aa1fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45852/0 (socket says 172.21.15.66:45852) 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.290+0000 7f5ab2800640 1 -- 172.21.15.66:0/2593655859 learned_addr learned my addr 172.21.15.66:0/2593655859 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab3200640 1 --2- 172.21.15.66:0/2593655859 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3b8aac00 0x559d3aa1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/2593655859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3b8ab400 0x559d3aa20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab2800640 1 -- 172.21.15.66:0/2593655859 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3b8aac00 msgr2=0x559d3aa1f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab2800640 1 --2- 172.21.15.66:0/2593655859 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3b8aac00 0x559d3aa1f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab2800640 1 -- 172.21.15.66:0/2593655859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3b8ab400 msgr2=0x559d3aa20100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab2800640 1 --2- 172.21.15.66:0/2593655859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3b8ab400 0x559d3aa20100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab2800640 1 -- 172.21.15.66:0/2593655859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3a9c8000 con 0x559d3b8ab000 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab3200640 1 --2- 172.21.15.66:0/2593655859 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3b8aac00 0x559d3aa1f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/2593655859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3b8ab400 0x559d3aa20100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab2800640 1 --2- 172.21.15.66:0/2593655859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab000 0x559d3aa1fb80 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x559d3b9120c0 tx=0x559d3aa6c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384e03b0b2716138 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5aace00640 1 -- 172.21.15.66:0/2593655859 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559d3a9e6f20 con 0x559d3b8ab000 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5aace00640 1 -- 172.21.15.66:0/2593655859 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559d3b766d80 con 0x559d3b8ab000 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5aace00640 1 -- 172.21.15.66:0/2593655859 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559d3a9e7600 con 0x559d3b8ab000 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2593655859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab000 msgr2=0x559d3aa1fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/2593655859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab000 0x559d3aa1fb80 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x559d3b9120c0 tx=0x559d3aa6c120 comp rx=0 tx=0).stop 2024-03-30T00:46:34.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2593655859 shutdown_connections 2024-03-30T00:46:34.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/2593655859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3b8ab400 0x559d3aa20100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/2593655859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab000 0x559d3aa1fb80 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/2593655859 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3b8aac00 0x559d3aa1f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2593655859 >> 172.21.15.66:0/2593655859 conn(0x559d3a992c00 msgr2=0x559d3a9c1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:34.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2593655859 shutdown_connections 2024-03-30T00:46:34.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2593655859 wait complete. 2024-03-30T00:46:34.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 Processor -- start 2024-03-30T00:46:34.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 -- start start 2024-03-30T00:46:34.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3b8aac00 0x559d3aa21180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3b8ab000 0x559d3aa20c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab400 0x559d3aa20680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3aa603c0 con 0x559d3b8ab400 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3aa60280 con 0x559d3b8aac00 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d3aa60140 con 0x559d3b8ab000 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab400 0x559d3aa20680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab400 0x559d3aa20680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45858/0 (socket says 172.21.15.66:45858) 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab1e00640 1 -- 172.21.15.66:0/3058566827 learned_addr learned my addr 172.21.15.66:0/3058566827 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab2800640 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3b8ab000 0x559d3aa20c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab3200640 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3b8aac00 0x559d3aa21180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab1e00640 1 -- 172.21.15.66:0/3058566827 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3b8ab000 msgr2=0x559d3aa20c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3b8ab000 0x559d3aa20c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab1e00640 1 -- 172.21.15.66:0/3058566827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3b8aac00 msgr2=0x559d3aa21180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3b8aac00 0x559d3aa21180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab1e00640 1 -- 172.21.15.66:0/3058566827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3a9c89c0 con 0x559d3b8ab400 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab400 0x559d3aa20680 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x559d3b912d20 tx=0x559d3aa6d860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5aab000640 1 -- 172.21.15.66:0/3058566827 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559d3a9e7080 con 0x559d3b8ab400 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5aab000640 1 -- 172.21.15.66:0/3058566827 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559d3b766300 con 0x559d3b8ab400 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5aab000640 1 -- 172.21.15.66:0/3058566827 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559d3a9e6f20 con 0x559d3b8ab400 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.294+0000 7f5ab47ab400 1 -- 172.21.15.66:0/3058566827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559d3a9c8b60 con 0x559d3b8ab400 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.298+0000 7f5ab47ab400 1 -- 172.21.15.66:0/3058566827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d3a9c8ea0 con 0x559d3b8ab400 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.298+0000 7f5ab47ab400 4 main: JournalTool: resolving pool 13 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.298+0000 7f5aab000640 1 -- 172.21.15.66:0/3058566827 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x559d3aa5f080 con 0x559d3b8ab400 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.298+0000 7f5aab000640 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x559d3ad60000 0x559d3aa21700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.298+0000 7f5aab000640 1 -- 172.21.15.66:0/3058566827 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x559d3a9d2a80 con 0x559d3b8ab400 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.298+0000 7f5ab47ab400 4 main: JournalTool: creating IoCtx.. 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.298+0000 7f5ab47ab400 4 main: Executing for rank 0 2024-03-30T00:46:34.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.298+0000 7f5ab3200640 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x559d3ad60000 0x559d3aa21700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.298+0000 7f5ab3200640 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x559d3ad60000 0x559d3aa21700 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x559d3b802bd0 tx=0x559d3ad6d800 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.298+0000 7f5ab47ab400 1 -- start start 2024-03-30T00:46:34.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.298+0000 7f5ab47ab400 1 Processor -- start 2024-03-30T00:46:34.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8abc00 0x559d3aa20100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3ad61000 0x559d3aa1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3ad61400 0x559d3aa1f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3aa60280 con 0x559d3b8abc00 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3aa603c0 con 0x559d3ad61400 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d3aa60a00 con 0x559d3ad61000 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8abc00 0x559d3aa20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8abc00 0x559d3aa20100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45872/0 (socket says 172.21.15.66:45872) 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab3200640 1 -- 172.21.15.66:0/936937716 learned_addr learned my addr 172.21.15.66:0/936937716 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab2800640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3ad61000 0x559d3aa1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3ad61400 0x559d3aa1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab3200640 1 -- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3ad61000 msgr2=0x559d3aa1fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab3200640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3ad61000 0x559d3aa1fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab3200640 1 -- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3ad61400 msgr2=0x559d3aa1f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab3200640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3ad61400 0x559d3aa1f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab3200640 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3acb1520 con 0x559d3b8abc00 2024-03-30T00:46:34.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3ad61400 0x559d3aa1f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab3200640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8abc00 0x559d3aa20100 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x559d3ad39140 tx=0x559d3b89cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df38c9199555e4fa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5aa7400640 1 -- 172.21.15.66:0/936937716 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559d3a9e66e0 con 0x559d3b8abc00 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5aa7400640 1 -- 172.21.15.66:0/936937716 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559d3a9f0180 con 0x559d3b8abc00 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5aa7400640 1 -- 172.21.15.66:0/936937716 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559d3a9e69a0 con 0x559d3b8abc00 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d3acb1380 con 0x559d3b8abc00 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559d3acb11e0 con 0x559d3b8abc00 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 4 waiting for MDS map... 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5aa7400640 1 -- 172.21.15.66:0/936937716 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x559d3a9d28c0 con 0x559d3b8abc00 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5aa7400640 1 -- 172.21.15.66:0/936937716 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 59) v1 ==== 2573+0+0 (secure 0 0 0) 0x559d3add8000 con 0x559d3b8abc00 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 4 Got MDS map 59 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x559d3ad61800 0x559d3ade2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3ad61c00 con 0x559d3ad61800 2024-03-30T00:46:34.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab2800640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x559d3ad61800 0x559d3ade2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.302+0000 7f5ab2800640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x559d3ad61800 0x559d3ade2000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=160153e90a562a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.306+0000 7f5ab2800640 1 -- 172.21.15.66:0/936937716 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x559d3b898b40 con 0x559d3ad61800 2024-03-30T00:46:34.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.306+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x559d3add1000 0x559d3ade2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.306+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3add1400 con 0x559d3add1000 2024-03-30T00:46:34.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.306+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x559d3add1000 0x559d3ade2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.306+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x559d3add1000 0x559d3ade2580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=8bdf74fac7765aed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.306+0000 7f5ab1e00640 1 -- 172.21.15.66:0/936937716 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x559d3b898d80 con 0x559d3add1000 2024-03-30T00:46:34.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.306+0000 7f5aa5600640 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3adea000 con 0x559d3add1000 2024-03-30T00:46:34.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.306+0000 7f5aa5600640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x559d3adea400 0x559d3ade4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.306+0000 7f5aa5600640 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3adea800 con 0x559d3adea400 2024-03-30T00:46:34.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.306+0000 7f5ab1e00640 1 -- 172.21.15.66:0/936937716 <== osd.0 v2:172.21.15.66:6800/27456 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x559d3b898d80 con 0x559d3add1000 2024-03-30T00:46:34.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.306+0000 7f5ab3200640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x559d3adea400 0x559d3ade4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab3200640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x559d3adea400 0x559d3ade4680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=ec1736b69fb8f88f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab3200640 1 -- 172.21.15.66:0/936937716 <== osd.4 v2:172.21.15.156:6800/27641 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559d3b898fc0 con 0x559d3adea400 2024-03-30T00:46:34.318 INFO:teuthology.orchestra.run.smithi066.stdout:old journal was 4209788~892 2024-03-30T00:46:34.318 INFO:teuthology.orchestra.run.smithi066.stdout:new journal start will be 8388608 (4177928 bytes past old end) 2024-03-30T00:46:34.318 INFO:teuthology.orchestra.run.smithi066.stdout:writing journal head 2024-03-30T00:46:34.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3add1800 con 0x559d3add1000 2024-03-30T00:46:34.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab1e00640 1 -- 172.21.15.66:0/936937716 <== osd.0 v2:172.21.15.66:6800/27456 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v97'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x559d3b898d80 con 0x559d3add1000 2024-03-30T00:46:34.319 INFO:teuthology.orchestra.run.smithi066.stdout:writing EResetJournal entry 2024-03-30T00:46:34.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3add1c00 con 0x559d3adea400 2024-03-30T00:46:34.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x559d3adeac00 0x559d3ade2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3adeb000 con 0x559d3adeac00 2024-03-30T00:46:34.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3adeb400 con 0x559d3ad61800 2024-03-30T00:46:34.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x559d3adeb800 0x559d3ade3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3adebc00 con 0x559d3adeb800 2024-03-30T00:46:34.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3adf4000 con 0x559d3adea400 2024-03-30T00:46:34.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x559d3adf4400 0x559d3ade3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3adf4800 con 0x559d3adf4400 2024-03-30T00:46:34.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x559d3adeb800 0x559d3ade3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab2800640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x559d3adeac00 0x559d3ade2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab3200640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x559d3adf4400 0x559d3ade3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.310+0000 7f5ab1e00640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x559d3adeb800 0x559d3ade3080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=3e47216666a7de4f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.314+0000 7f5ab3200640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x559d3adf4400 0x559d3ade3600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=8b3be6656638e434 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.314+0000 7f5ab2800640 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x559d3adeac00 0x559d3ade2b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=9cb5f8f66b059ae server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.314+0000 7f5ab3200640 1 -- 172.21.15.66:0/936937716 <== osd.4 v2:172.21.15.156:6800/27641 2 ==== osd_op_reply(6 200.00000002 [delete] v97'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559d3b898fc0 con 0x559d3adea400 2024-03-30T00:46:34.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.314+0000 7f5ab3200640 1 -- 172.21.15.66:0/936937716 <== osd.4 v2:172.21.15.156:6800/27641 3 ==== osd_op_reply(10 200.00000006 [delete] v97'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559d3b898fc0 con 0x559d3adea400 2024-03-30T00:46:34.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.314+0000 7f5ab1e00640 1 -- 172.21.15.66:0/936937716 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(9 200.00000005 [delete] v97'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559d3b898d80 con 0x559d3adeb800 2024-03-30T00:46:34.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.314+0000 7f5ab2800640 1 -- 172.21.15.66:0/936937716 <== osd.6 v2:172.21.15.156:6816/27642 2 ==== osd_op_reply(8 200.00000004 [delete] v97'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559d3b898b40 con 0x559d3ad61800 2024-03-30T00:46:34.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.314+0000 7f5ab3200640 1 -- 172.21.15.66:0/936937716 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== osd_op_reply(11 200.00000007 [delete] v97'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559d3b898fc0 con 0x559d3adf4400 2024-03-30T00:46:34.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab2800640 1 -- 172.21.15.66:0/936937716 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== osd_op_reply(7 200.00000003 [delete] v97'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559d3b898b40 con 0x559d3adeac00 2024-03-30T00:46:34.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5aa5600640 1 -- 172.21.15.66:0/936937716 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x559d3ad61c00 con 0x559d3adea400 2024-03-30T00:46:34.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab3200640 1 -- 172.21.15.66:0/936937716 <== osd.4 v2:172.21.15.156:6800/27641 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v97'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x559d3b898fc0 con 0x559d3adea400 2024-03-30T00:46:34.327 INFO:teuthology.orchestra.run.smithi066.stdout:done 2024-03-30T00:46:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x559d3add1000 msgr2=0x559d3ade2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x559d3add1000 0x559d3ade2580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x559d3adeb800 msgr2=0x559d3ade3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x559d3adeb800 0x559d3ade3080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x559d3adea400 msgr2=0x559d3ade4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x559d3adea400 0x559d3ade4680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x559d3adeac00 msgr2=0x559d3ade2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x559d3adeac00 0x559d3ade2b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x559d3ad61800 msgr2=0x559d3ade2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x559d3ad61800 0x559d3ade2000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x559d3adf4400 msgr2=0x559d3ade3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x559d3adf4400 0x559d3ade3600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab2800640 1 -- 172.21.15.66:0/936937716 reap_dead start 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8abc00 msgr2=0x559d3aa20100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab2800640 1 -- 172.21.15.66:0/936937716 reap_dead start 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/936937716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8abc00 0x559d3aa20100 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x559d3ad39140 tx=0x559d3b89cc00 comp rx=0 tx=0).stop 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab2800640 1 -- 172.21.15.66:0/936937716 reap_dead start 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab2800640 1 -- 172.21.15.66:0/936937716 reap_dead start 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 shutdown_connections 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 >> 172.21.15.66:0/936937716 conn(0x559d3b8ab800 msgr2=0x559d3adb0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 shutdown_connections 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/936937716 wait complete. 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/3058566827 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x559d3ad60000 msgr2=0x559d3aa21700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x559d3ad60000 0x559d3aa21700 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x559d3b802bd0 tx=0x559d3ad6d800 comp rx=0 tx=0).stop 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/3058566827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab400 msgr2=0x559d3aa20680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab400 0x559d3aa20680 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x559d3b912d20 tx=0x559d3aa6d860 comp rx=0 tx=0).stop 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/3058566827 shutdown_connections 2024-03-30T00:46:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x559d3ad60000 0x559d3aa21700 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3b8ab400 0x559d3aa20680 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3b8ab000 0x559d3aa20c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/3058566827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3b8aac00 0x559d3aa21180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/3058566827 >> 172.21.15.66:0/3058566827 conn(0x559d3a992c00 msgr2=0x559d3a9c1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/3058566827 shutdown_connections 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.318+0000 7f5ab47ab400 1 -- 172.21.15.66:0/3058566827 wait complete. 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.322+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2861537345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3a993800 msgr2=0x559d3aa1e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.322+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/2861537345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3a993800 0x559d3aa1e580 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x559d3b802d50 tx=0x559d3b89cd20 comp rx=0 tx=0).stop 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.322+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2861537345 shutdown_connections 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.322+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/2861537345 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559d3a993000 0x559d3aa1f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.322+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/2861537345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3a993400 0x559d3aa1eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.322+0000 7f5ab47ab400 1 --2- 172.21.15.66:0/2861537345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d3a993800 0x559d3aa1e580 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.322+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2861537345 >> 172.21.15.66:0/2861537345 conn(0x559d3a993c00 msgr2=0x559d3a9c1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.322+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2861537345 shutdown_connections 2024-03-30T00:46:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.322+0000 7f5ab47ab400 1 -- 172.21.15.66:0/2861537345 wait complete. 2024-03-30T00:46:34.334 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.125789, result: old journal was 4209788~892 new journal start will be 8388608 (4177928 bytes past old end) writing journal head writing EResetJournal entry done 2024-03-30T00:46:34.334 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-30T00:46:34.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff79d400640 1 Processor -- start 2024-03-30T00:46:34.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff79d400640 1 -- start start 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff79d400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7980cad00 0x7ff7980c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff79d400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff7980cd2d0 0x7ff7980cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff79d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7980cc8d0 0x7ff79806c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff79d400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7980cbec0 con 0x7ff7980cc8d0 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff79d400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7980c8e80 con 0x7ff7980cd2d0 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff79d400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cb620 con 0x7ff7980cad00 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff797200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7980cc8d0 0x7ff79806c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff797200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7980cc8d0 0x7ff79806c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45882/0 (socket says 172.21.15.66:45882) 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff797200640 1 -- 172.21.15.66:0/1670255217 learned_addr learned my addr 172.21.15.66:0/1670255217 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff78de00640 1 --2- 172.21.15.66:0/1670255217 >> v2:172.21.15.156:3300/0 conn(0x7ff7980cd2d0 0x7ff7980cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.510+0000 7ff796800640 1 --2- 172.21.15.66:0/1670255217 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7980cad00 0x7ff7980c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff797200640 1 -- 172.21.15.66:0/1670255217 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7980cad00 msgr2=0x7ff7980c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:34.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff797200640 1 --2- 172.21.15.66:0/1670255217 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7980cad00 0x7ff7980c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff797200640 1 -- 172.21.15.66:0/1670255217 >> v2:172.21.15.156:3300/0 conn(0x7ff7980cd2d0 msgr2=0x7ff7980cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:34.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff797200640 1 --2- 172.21.15.66:0/1670255217 >> v2:172.21.15.156:3300/0 conn(0x7ff7980cd2d0 0x7ff7980cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff797200640 1 -- 172.21.15.66:0/1670255217 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980ca660 con 0x7ff7980cc8d0 2024-03-30T00:46:34.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff796800640 1 --2- 172.21.15.66:0/1670255217 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7980cad00 0x7ff7980c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:34.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff797200640 1 --2- 172.21.15.66:0/1670255217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7980cc8d0 0x7ff79806c030 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff7980cb920 tx=0x7ff79001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7a3ba191d982b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff795e00640 1 -- 172.21.15.66:0/1670255217 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900aa610 con 0x7ff7980cc8d0 2024-03-30T00:46:34.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff795e00640 1 -- 172.21.15.66:0/1670255217 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7900a6040 con 0x7ff7980cc8d0 2024-03-30T00:46:34.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff795e00640 1 -- 172.21.15.66:0/1670255217 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900aa990 con 0x7ff7980cc8d0 2024-03-30T00:46:34.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff79d400640 1 -- 172.21.15.66:0/1670255217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7980cc8d0 msgr2=0x7ff79806c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff79d400640 1 --2- 172.21.15.66:0/1670255217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7980cc8d0 0x7ff79806c030 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff7980cb920 tx=0x7ff79001c4f0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff79d400640 1 -- 172.21.15.66:0/1670255217 shutdown_connections 2024-03-30T00:46:34.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff79d400640 1 --2- 172.21.15.66:0/1670255217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7980cc8d0 0x7ff79806c030 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff79d400640 1 --2- 172.21.15.66:0/1670255217 >> v2:172.21.15.156:3300/0 conn(0x7ff7980cd2d0 0x7ff7980cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff79d400640 1 --2- 172.21.15.66:0/1670255217 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7980cad00 0x7ff7980c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.514+0000 7ff79d400640 1 -- 172.21.15.66:0/1670255217 >> 172.21.15.66:0/1670255217 conn(0x7ff798069f60 msgr2=0x7ff79806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:34.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff79d400640 1 -- 172.21.15.66:0/1670255217 shutdown_connections 2024-03-30T00:46:34.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff79d400640 1 -- 172.21.15.66:0/1670255217 wait complete. 2024-03-30T00:46:34.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff79d400640 1 Processor -- start 2024-03-30T00:46:34.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff79d400640 1 -- start start 2024-03-30T00:46:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff79d400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7980cad00 0x7ff7980c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff79d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7980cc8d0 0x7ff7981f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff79d400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980cd2d0 0x7ff7980c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff79d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980c81a0 con 0x7ff7980cc8d0 2024-03-30T00:46:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff79d400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7981629e0 con 0x7ff7980cd2d0 2024-03-30T00:46:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff79d400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7981f6ae0 con 0x7ff7980cad00 2024-03-30T00:46:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7980cc8d0 0x7ff7981f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7980cc8d0 0x7ff7981f65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45888/0 (socket says 172.21.15.66:45888) 2024-03-30T00:46:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78de00640 1 -- 172.21.15.66:0/1837361688 learned_addr learned my addr 172.21.15.66:0/1837361688 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff796800640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7980cad00 0x7ff7980c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff797200640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980cd2d0 0x7ff7980c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78de00640 1 -- 172.21.15.66:0/1837361688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7980cad00 msgr2=0x7ff7980c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78de00640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7980cad00 0x7ff7980c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78de00640 1 -- 172.21.15.66:0/1837361688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980cd2d0 msgr2=0x7ff7980c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:34.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78de00640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980cd2d0 0x7ff7980c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:34.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78de00640 1 -- 172.21.15.66:0/1837361688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900094d0 con 0x7ff7980cc8d0 2024-03-30T00:46:34.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78de00640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7980cc8d0 0x7ff7981f65a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff78000e930 tx=0x7ff78000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78f400640 1 -- 172.21.15.66:0/1837361688 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780009690 con 0x7ff7980cc8d0 2024-03-30T00:46:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78f400640 1 -- 172.21.15.66:0/1837361688 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78000ca00 con 0x7ff7980cc8d0 2024-03-30T00:46:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff78f400640 1 -- 172.21.15.66:0/1837361688 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000cdb0 con 0x7ff7980cc8d0 2024-03-30T00:46:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.518+0000 7ff79d400640 1 -- 172.21.15.66:0/1837361688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff798161dd0 con 0x7ff7980cc8d0 2024-03-30T00:46:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.522+0000 7ff79d400640 1 -- 172.21.15.66:0/1837361688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff798162390 con 0x7ff7980cc8d0 2024-03-30T00:46:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.522+0000 7ff79d400640 1 -- 172.21.15.66:0/1837361688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff798162b70 con 0x7ff7980cc8d0 2024-03-30T00:46:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.522+0000 7ff78f400640 1 -- 172.21.15.66:0/1837361688 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff780013470 con 0x7ff7980cc8d0 2024-03-30T00:46:34.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.522+0000 7ff78f400640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff774066690 0x7ff774068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:34.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.522+0000 7ff78f400640 1 -- 172.21.15.66:0/1837361688 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7ff780084430 con 0x7ff7980cc8d0 2024-03-30T00:46:34.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.522+0000 7ff796800640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff774066690 0x7ff774068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:34.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.526+0000 7ff796800640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff774066690 0x7ff774068b50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff78800afa0 tx=0x7ff78800c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:34.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.526+0000 7ff78f400640 1 -- 172.21.15.66:0/1837361688 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff780051790 con 0x7ff7980cc8d0 2024-03-30T00:46:34.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:34.726+0000 7ff79d400640 1 -- 172.21.15.66:0/1837361688 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7ff798066c30 con 0x7ff7980cc8d0 2024-03-30T00:46:35.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.110+0000 7ff78f400640 1 -- 172.21.15.66:0/1837361688 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v60) v1 ==== 162+0+0 (secure 0 0 0) 0x7ff780054db0 con 0x7ff7980cc8d0 2024-03-30T00:46:35.120 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-30T00:46:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 -- 172.21.15.66:0/1837361688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff774066690 msgr2=0x7ff774068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff774066690 0x7ff774068b50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff78800afa0 tx=0x7ff78800c040 comp rx=0 tx=0).stop 2024-03-30T00:46:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 -- 172.21.15.66:0/1837361688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7980cc8d0 msgr2=0x7ff7981f65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7980cc8d0 0x7ff7981f65a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff78000e930 tx=0x7ff78000edf0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 -- 172.21.15.66:0/1837361688 shutdown_connections 2024-03-30T00:46:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980cd2d0 0x7ff7980c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff774066690 0x7ff774068b50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7980cc8d0 0x7ff7981f65a0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 --2- 172.21.15.66:0/1837361688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7980cad00 0x7ff7980c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 -- 172.21.15.66:0/1837361688 >> 172.21.15.66:0/1837361688 conn(0x7ff798069f60 msgr2=0x7ff7980acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 -- 172.21.15.66:0/1837361688 shutdown_connections 2024-03-30T00:46:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.114+0000 7ff79d400640 1 -- 172.21.15.66:0/1837361688 wait complete. 2024-03-30T00:46:35.146 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:46:35.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4157c00640 1 Processor -- start 2024-03-30T00:46:35.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4157c00640 1 -- start start 2024-03-30T00:46:35.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4157c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4150098c80 0x7f4150099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:35.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4157c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f41500cf320 0x7f41500ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4157c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41500cbd90 con 0x7f4150099ca0 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4157c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41500cc070 con 0x7f41500cf320 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4157c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41500cc350 con 0x7f4150098c80 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4147e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f41500cf320 0x7f41500ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4147e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f41500cf320 0x7f41500ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45902/0 (socket says 172.21.15.66:45902) 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4147e00640 1 -- 172.21.15.66:0/1986103482 learned_addr learned my addr 172.21.15.66:0/1986103482 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4155c00640 1 --1- 172.21.15.66:0/1986103482 >> v1:172.21.15.66:6789/0 conn(0x7f4150099ca0 0x7f41500ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48892/0 (socket says 172.21.15.66:48892) 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4155200640 1 --2- 172.21.15.66:0/1986103482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4150098c80 0x7f4150099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4147e00640 1 -- 172.21.15.66:0/1986103482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4150098c80 msgr2=0x7f4150099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4147e00640 1 --2- 172.21.15.66:0/1986103482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4150098c80 0x7f4150099760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4147e00640 1 -- 172.21.15.66:0/1986103482 >> v1:172.21.15.66:6789/0 conn(0x7f4150099ca0 legacy=0x7f41500ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f4147e00640 1 -- 172.21.15.66:0/1986103482 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41500b3530 con 0x7f41500cf320 2024-03-30T00:46:35.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.278+0000 7f414fe00640 1 -- 172.21.15.66:0/1986103482 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100031173 0 0) 0x7f41500cbd90 con 0x7f4150099ca0 2024-03-30T00:46:35.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f4147e00640 1 --2- 172.21.15.66:0/1986103482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41500cf320 0x7f41500ce210 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f4138009870 tx=0x7f413801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cd068d35a0c618 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:35.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f414fe00640 1 -- 172.21.15.66:0/1986103482 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41380a8a50 con 0x7f41500cf320 2024-03-30T00:46:35.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f414fe00640 1 -- 172.21.15.66:0/1986103482 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41380a8bb0 con 0x7f41500cf320 2024-03-30T00:46:35.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f414fe00640 1 -- 172.21.15.66:0/1986103482 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41380b0440 con 0x7f41500cf320 2024-03-30T00:46:35.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f4157c00640 1 -- 172.21.15.66:0/1986103482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41500cf320 msgr2=0x7f41500ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:35.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f4157c00640 1 --2- 172.21.15.66:0/1986103482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41500cf320 0x7f41500ce210 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f4138009870 tx=0x7f413801ed50 comp rx=0 tx=0).stop 2024-03-30T00:46:35.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f4157c00640 1 -- 172.21.15.66:0/1986103482 shutdown_connections 2024-03-30T00:46:35.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f4157c00640 1 --2- 172.21.15.66:0/1986103482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41500cf320 0x7f41500ce210 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f4157c00640 1 --2- 172.21.15.66:0/1986103482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4150098c80 0x7f4150099760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f4157c00640 1 -- 172.21.15.66:0/1986103482 >> 172.21.15.66:0/1986103482 conn(0x7f41500b2130 msgr2=0x7f41500b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:35.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f4157c00640 1 -- 172.21.15.66:0/1986103482 shutdown_connections 2024-03-30T00:46:35.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.282+0000 7f4157c00640 1 -- 172.21.15.66:0/1986103482 wait complete. 2024-03-30T00:46:35.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4157c00640 1 Processor -- start 2024-03-30T00:46:35.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4157c00640 1 -- start start 2024-03-30T00:46:35.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4157c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4150098c80 0x7f4150153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:35.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4157c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4150099ca0 0x7f41500c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:35.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4157c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41500cf320 0x7f41500c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:35.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4157c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41500c97c0 con 0x7f4150099ca0 2024-03-30T00:46:35.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4157c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4150154130 con 0x7f41500cf320 2024-03-30T00:46:35.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4157c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41500ca120 con 0x7f4150098c80 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4147e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4150099ca0 0x7f41500c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4147e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4150099ca0 0x7f41500c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45906/0 (socket says 172.21.15.66:45906) 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4147e00640 1 -- 172.21.15.66:0/1971554778 learned_addr learned my addr 172.21.15.66:0/1971554778 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4155200640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4150098c80 0x7f4150153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4147e00640 1 -- 172.21.15.66:0/1971554778 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4150098c80 msgr2=0x7f4150153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4147e00640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4150098c80 0x7f4150153bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4155c00640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41500cf320 0x7f41500c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4147e00640 1 -- 172.21.15.66:0/1971554778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41500cf320 msgr2=0x7f41500c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4147e00640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41500cf320 0x7f41500c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4147e00640 1 -- 172.21.15.66:0/1971554778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41380094d0 con 0x7f4150099ca0 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4155c00640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41500cf320 0x7f41500c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4147e00640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4150099ca0 0x7f41500c9be0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f41380b3040 tx=0x7f413801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f414d600640 1 -- 172.21.15.66:0/1971554778 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41380be070 con 0x7f4150099ca0 2024-03-30T00:46:35.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f414d600640 1 -- 172.21.15.66:0/1971554778 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41380b9420 con 0x7f4150099ca0 2024-03-30T00:46:35.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4157c00640 1 -- 172.21.15.66:0/1971554778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41500c8b40 con 0x7f4150099ca0 2024-03-30T00:46:35.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f414d600640 1 -- 172.21.15.66:0/1971554778 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41380b97f0 con 0x7f4150099ca0 2024-03-30T00:46:35.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.286+0000 7f4157c00640 1 -- 172.21.15.66:0/1971554778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41500c9100 con 0x7f4150099ca0 2024-03-30T00:46:35.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.290+0000 7f4157c00640 1 -- 172.21.15.66:0/1971554778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41500677d0 con 0x7f4150099ca0 2024-03-30T00:46:35.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.290+0000 7f414d600640 1 -- 172.21.15.66:0/1971554778 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f41380b7070 con 0x7f4150099ca0 2024-03-30T00:46:35.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.290+0000 7f414d600640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f412c066690 0x7f412c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:35.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.290+0000 7f414d600640 1 -- 172.21.15.66:0/1971554778 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f4138121ba0 con 0x7f4150099ca0 2024-03-30T00:46:35.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.290+0000 7f4155200640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f412c066690 0x7f412c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:35.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.290+0000 7f4155200640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f412c066690 0x7f412c068b50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4140009790 tx=0x7f4140008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:35.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.294+0000 7f414d600640 1 -- 172.21.15.66:0/1971554778 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f41380eecf0 con 0x7f4150099ca0 2024-03-30T00:46:35.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 -- 172.21.15.66:0/1971554778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41500c9900 con 0x7f4150099ca0 2024-03-30T00:46:35.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f414d600640 1 -- 172.21.15.66:0/1971554778 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f41500c9900 con 0x7f4150099ca0 2024-03-30T00:46:35.502 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:35.502 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:46:35.115472+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"a","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.66:6836/1626022080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1626022080},{"type":"v1","addr":"172.21.15.66:6836","nonce":1626022080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:46:35.502 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 61 2024-03-30T00:46:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 -- 172.21.15.66:0/1971554778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f412c066690 msgr2=0x7f412c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f412c066690 0x7f412c068b50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4140009790 tx=0x7f4140008040 comp rx=0 tx=0).stop 2024-03-30T00:46:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 -- 172.21.15.66:0/1971554778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4150099ca0 msgr2=0x7f41500c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4150099ca0 0x7f41500c9be0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f41380b3040 tx=0x7f413801baf0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 -- 172.21.15.66:0/1971554778 shutdown_connections 2024-03-30T00:46:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41500cf320 0x7f41500c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f412c066690 0x7f412c068b50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4150099ca0 0x7f41500c9be0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 --2- 172.21.15.66:0/1971554778 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4150098c80 0x7f4150153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 -- 172.21.15.66:0/1971554778 >> 172.21.15.66:0/1971554778 conn(0x7f41500b2130 msgr2=0x7f41500b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 -- 172.21.15.66:0/1971554778 shutdown_connections 2024-03-30T00:46:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:35.494+0000 7f4157c00640 1 -- 172.21.15.66:0/1971554778 wait complete. 2024-03-30T00:46:35.926 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:46:15.271965+0000', 'modified': '2024-03-30T00:46:35.115472+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7920}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7920': {'gid': 7920, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.66:6836/1626022080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 1626022080}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 1626022080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:46:35.927 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7920:up:replay 2024-03-30T00:46:36.928 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:46:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ed5200640 1 Processor -- start 2024-03-30T00:46:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ed5200640 1 -- start start 2024-03-30T00:46:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ed5200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1ed00cabb0 0x7f1ed00cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ed5200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed006d070 con 0x7f1ed015a640 2024-03-30T00:46:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ed5200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed00cbba0 con 0x7f1ed00ce520 2024-03-30T00:46:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ed5200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed00cb8c0 con 0x7f1ed00cabb0 2024-03-30T00:46:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ece800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1ed00cabb0 0x7f1ed00cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ecde00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1ed015a640 0x7f1ed006dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48902/0 (socket says 172.21.15.66:48902) 2024-03-30T00:46:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ecde00640 1 -- 172.21.15.66:0/420009417 learned_addr learned my addr 172.21.15.66:0/420009417 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ece800640 1 -- 172.21.15.66:0/420009417 >> v1:172.21.15.156:6789/0 conn(0x7f1ed00ce520 legacy=0x7f1ed00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ece800640 1 -- 172.21.15.66:0/420009417 >> v1:172.21.15.66:6789/0 conn(0x7f1ed015a640 legacy=0x7f1ed006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ecca00640 1 -- 172.21.15.66:0/420009417 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719182671 0 0) 0x7f1ed006d070 con 0x7f1ed015a640 2024-03-30T00:46:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.074+0000 7f1ece800640 1 -- 172.21.15.66:0/420009417 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed006d2a0 con 0x7f1ed00cabb0 2024-03-30T00:46:37.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ece800640 1 --2- 172.21.15.66:0/420009417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed00cabb0 0x7f1ed00cafe0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009d90 tx=0x7f1ec801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1ad70479cb3f6c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:37.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ecca00640 1 -- 172.21.15.66:0/420009417 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec80a5a00 con 0x7f1ed00cabb0 2024-03-30T00:46:37.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ecca00640 1 -- 172.21.15.66:0/420009417 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec80a5b60 con 0x7f1ed00cabb0 2024-03-30T00:46:37.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ecca00640 1 -- 172.21.15.66:0/420009417 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec80ad440 con 0x7f1ed00cabb0 2024-03-30T00:46:37.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ed5200640 1 -- 172.21.15.66:0/420009417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed00cabb0 msgr2=0x7f1ed00cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:37.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ed5200640 1 --2- 172.21.15.66:0/420009417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed00cabb0 0x7f1ed00cafe0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009d90 tx=0x7f1ec801c940 comp rx=0 tx=0).stop 2024-03-30T00:46:37.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ed5200640 1 -- 172.21.15.66:0/420009417 shutdown_connections 2024-03-30T00:46:37.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ed5200640 1 --2- 172.21.15.66:0/420009417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed00cabb0 0x7f1ed00cafe0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:37.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ed5200640 1 -- 172.21.15.66:0/420009417 >> 172.21.15.66:0/420009417 conn(0x7f1ed0069f60 msgr2=0x7f1ed006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:37.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ed5200640 1 -- 172.21.15.66:0/420009417 shutdown_connections 2024-03-30T00:46:37.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ed5200640 1 -- 172.21.15.66:0/420009417 wait complete. 2024-03-30T00:46:37.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ed5200640 1 Processor -- start 2024-03-30T00:46:37.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.078+0000 7f1ed5200640 1 -- start start 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ed5200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed00cabb0 0x7f1ed0071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ed5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed00ce520 0x7f1ed0070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ed5200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ed015a640 0x7f1ed0071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ed5200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed00cbec0 con 0x7f1ed00ce520 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ed5200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed0165f10 con 0x7f1ed00cabb0 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ed5200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed00cc160 con 0x7f1ed015a640 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ecd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed00ce520 0x7f1ed0070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ecd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed00ce520 0x7f1ed0070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45932/0 (socket says 172.21.15.66:45932) 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ecd400640 1 -- 172.21.15.66:0/646778800 learned_addr learned my addr 172.21.15.66:0/646778800 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ecde00640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed00cabb0 0x7f1ed0071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ece800640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ed015a640 0x7f1ed0071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:37.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ecd400640 1 -- 172.21.15.66:0/646778800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ed015a640 msgr2=0x7f1ed0071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:37.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ecd400640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ed015a640 0x7f1ed0071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:37.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ecd400640 1 -- 172.21.15.66:0/646778800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed00cabb0 msgr2=0x7f1ed0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:37.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ecd400640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed00cabb0 0x7f1ed0071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:37.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ecd400640 1 -- 172.21.15.66:0/646778800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec8009a70 con 0x7f1ed00ce520 2024-03-30T00:46:37.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ecd400640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed00ce520 0x7f1ed0070f20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00c500 tx=0x7f1ebc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:37.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ec6000640 1 -- 172.21.15.66:0/646778800 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc005220 con 0x7f1ed00ce520 2024-03-30T00:46:37.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ec6000640 1 -- 172.21.15.66:0/646778800 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc005380 con 0x7f1ed00ce520 2024-03-30T00:46:37.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ed5200640 1 -- 172.21.15.66:0/646778800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed00cc440 con 0x7f1ed00ce520 2024-03-30T00:46:37.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ec6000640 1 -- 172.21.15.66:0/646778800 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc007270 con 0x7f1ed00ce520 2024-03-30T00:46:37.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ed5200640 1 -- 172.21.15.66:0/646778800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed0165ba0 con 0x7f1ed00ce520 2024-03-30T00:46:37.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ed5200640 1 -- 172.21.15.66:0/646778800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed0066c30 con 0x7f1ed00ce520 2024-03-30T00:46:37.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.082+0000 7f1ec6000640 1 -- 172.21.15.66:0/646778800 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1ebc0073d0 con 0x7f1ed00ce520 2024-03-30T00:46:37.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.086+0000 7f1ec6000640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1eb0066690 0x7f1eb0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:37.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.086+0000 7f1ec6000640 1 -- 172.21.15.66:0/646778800 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f1ebc083c30 con 0x7f1ed00ce520 2024-03-30T00:46:37.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.086+0000 7f1ecde00640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1eb0066690 0x7f1eb0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:37.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.086+0000 7f1ecde00640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1eb0066690 0x7f1eb0068b50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0007bc0 tx=0x7f1ec0007b10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:37.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.090+0000 7f1ec6000640 1 -- 172.21.15.66:0/646778800 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1ebc050db0 con 0x7f1ed00ce520 2024-03-30T00:46:37.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.282+0000 7f1ed5200640 1 -- 172.21.15.66:0/646778800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ed00c45e0 con 0x7f1ed00ce520 2024-03-30T00:46:37.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ec6000640 1 -- 172.21.15.66:0/646778800 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f1ebc0543d0 con 0x7f1ed00ce520 2024-03-30T00:46:37.293 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:37.293 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:46:37.120104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"a","rank":0,"incarnation":61,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.66:6836/1626022080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1626022080},{"type":"v1","addr":"172.21.15.66:6836","nonce":1626022080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:46:37.293 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 63 2024-03-30T00:46:37.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 -- 172.21.15.66:0/646778800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1eb0066690 msgr2=0x7f1eb0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:37.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1eb0066690 0x7f1eb0068b50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0007bc0 tx=0x7f1ec0007b10 comp rx=0 tx=0).stop 2024-03-30T00:46:37.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 -- 172.21.15.66:0/646778800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed00ce520 msgr2=0x7f1ed0070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:37.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed00ce520 0x7f1ed0070f20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00c500 tx=0x7f1ebc00c9c0 comp rx=0 tx=0).stop 2024-03-30T00:46:37.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 -- 172.21.15.66:0/646778800 shutdown_connections 2024-03-30T00:46:37.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ed015a640 0x7f1ed0071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:37.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1eb0066690 0x7f1eb0068b50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:37.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed00ce520 0x7f1ed0070f20 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:37.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 --2- 172.21.15.66:0/646778800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed00cabb0 0x7f1ed0071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:37.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 -- 172.21.15.66:0/646778800 >> 172.21.15.66:0/646778800 conn(0x7f1ed0069f60 msgr2=0x7f1ed00ac330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:37.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 -- 172.21.15.66:0/646778800 shutdown_connections 2024-03-30T00:46:37.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:37.286+0000 7f1ed5200640 1 -- 172.21.15.66:0/646778800 wait complete. 2024-03-30T00:46:37.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:46:15.271965+0000', 'modified': '2024-03-30T00:46:37.120104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7920}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7920': {'gid': 7920, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.66:6836/1626022080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 1626022080}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 1626022080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:46:37.717 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7920:up:rejoin 2024-03-30T00:46:38.718 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:46:38.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f875dc00640 1 Processor -- start 2024-03-30T00:46:38.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f875dc00640 1 -- start start 2024-03-30T00:46:38.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f875dc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875806e800 0x7f87580d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:38.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f875dc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875806b0b0 con 0x7f875806f8e0 2024-03-30T00:46:38.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f875dc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875806b390 con 0x7f87580cc560 2024-03-30T00:46:38.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f875dc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8758160640 con 0x7f875806e800 2024-03-30T00:46:38.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f8757400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f875806f8e0 0x7f87580ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:59514/0 (socket says 172.21.15.66:59514) 2024-03-30T00:46:38.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f8757400640 1 -- 172.21.15.66:0/3011578235 learned_addr learned my addr 172.21.15.66:0/3011578235 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:38.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f874e000640 1 --2- 172.21.15.66:0/3011578235 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875806e800 0x7f87580d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:38.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f8756000640 1 -- 172.21.15.66:0/3011578235 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694990488 0 0) 0x7f875806b0b0 con 0x7f875806f8e0 2024-03-30T00:46:38.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f8756000640 1 -- 172.21.15.66:0/3011578235 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f873c003530 con 0x7f875806f8e0 2024-03-30T00:46:38.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f8756000640 1 -- 172.21.15.66:0/3011578235 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304455442 0 0) 0x7f875806b390 con 0x7f87580cc560 2024-03-30T00:46:38.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f874e000640 1 -- 172.21.15.66:0/3011578235 >> v1:172.21.15.156:6789/0 conn(0x7f87580cc560 legacy=0x7f875806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f874e000640 1 -- 172.21.15.66:0/3011578235 >> v1:172.21.15.66:6789/0 conn(0x7f875806f8e0 legacy=0x7f87580ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.862+0000 7f874e000640 1 -- 172.21.15.66:0/3011578235 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875806b670 con 0x7f875806e800 2024-03-30T00:46:38.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.866+0000 7f874e000640 1 --2- 172.21.15.66:0/3011578235 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875806e800 0x7f87580d1060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f87400097f0 tx=0x7f874001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=404969b42650f292 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:38.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.866+0000 7f8756000640 1 -- 172.21.15.66:0/3011578235 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87400aa430 con 0x7f875806e800 2024-03-30T00:46:38.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.866+0000 7f8756000640 1 -- 172.21.15.66:0/3011578235 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87400aa590 con 0x7f875806e800 2024-03-30T00:46:38.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.866+0000 7f8756000640 1 -- 172.21.15.66:0/3011578235 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87400aa900 con 0x7f875806e800 2024-03-30T00:46:38.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.866+0000 7f875dc00640 1 -- 172.21.15.66:0/3011578235 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875806e800 msgr2=0x7f87580d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:38.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.866+0000 7f875dc00640 1 --2- 172.21.15.66:0/3011578235 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875806e800 0x7f87580d1060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f87400097f0 tx=0x7f874001c4b0 comp rx=0 tx=0).stop 2024-03-30T00:46:38.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.866+0000 7f875dc00640 1 -- 172.21.15.66:0/3011578235 shutdown_connections 2024-03-30T00:46:38.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.866+0000 7f875dc00640 1 --2- 172.21.15.66:0/3011578235 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875806e800 0x7f87580d1060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:38.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.866+0000 7f875dc00640 1 -- 172.21.15.66:0/3011578235 >> 172.21.15.66:0/3011578235 conn(0x7f87580d0540 msgr2=0x7f87580d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:38.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.866+0000 7f875dc00640 1 -- 172.21.15.66:0/3011578235 shutdown_connections 2024-03-30T00:46:38.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.866+0000 7f875dc00640 1 -- 172.21.15.66:0/3011578235 wait complete. 2024-03-30T00:46:38.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f875dc00640 1 Processor -- start 2024-03-30T00:46:38.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f875dc00640 1 -- start start 2024-03-30T00:46:38.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f875dc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875806e800 0x7f87580b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:38.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f875dc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875806f8e0 0x7f875815fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:38.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f875dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87580cc560 0x7f8758157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:38.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f875dc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8758156c00 con 0x7f87580cc560 2024-03-30T00:46:38.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f875dc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87580b0b40 con 0x7f875806f8e0 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f875dc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87580b0e20 con 0x7f875806e800 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f8757400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87580cc560 0x7f8758157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f8757400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87580cc560 0x7f8758157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50840/0 (socket says 172.21.15.66:50840) 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f8757400640 1 -- 172.21.15.66:0/2756267862 learned_addr learned my addr 172.21.15.66:0/2756267862 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f874e000640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875806f8e0 0x7f875815fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f8756a00640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875806e800 0x7f87580b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f8757400640 1 -- 172.21.15.66:0/2756267862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875806e800 msgr2=0x7f87580b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f8757400640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875806e800 0x7f87580b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f8757400640 1 -- 172.21.15.66:0/2756267862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875806f8e0 msgr2=0x7f875815fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f8757400640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875806f8e0 0x7f875815fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f8757400640 1 -- 172.21.15.66:0/2756267862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8740009420 con 0x7f87580cc560 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f874e000640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875806f8e0 0x7f875815fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:38.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f8757400640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87580cc560 0x7f8758157010 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f875000cb40 tx=0x7f875000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:38.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f874f400640 1 -- 172.21.15.66:0/2756267862 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875000aca0 con 0x7f87580cc560 2024-03-30T00:46:38.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f874f400640 1 -- 172.21.15.66:0/2756267862 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875000ae00 con 0x7f87580cc560 2024-03-30T00:46:38.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f875dc00640 1 -- 172.21.15.66:0/2756267862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87580b1100 con 0x7f87580cc560 2024-03-30T00:46:38.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f874f400640 1 -- 172.21.15.66:0/2756267862 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750012620 con 0x7f87580cc560 2024-03-30T00:46:38.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.870+0000 7f875dc00640 1 -- 172.21.15.66:0/2756267862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87580b16c0 con 0x7f87580cc560 2024-03-30T00:46:38.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.874+0000 7f875dc00640 1 -- 172.21.15.66:0/2756267862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8758066c30 con 0x7f87580cc560 2024-03-30T00:46:38.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.874+0000 7f874f400640 1 -- 172.21.15.66:0/2756267862 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8750007070 con 0x7f87580cc560 2024-03-30T00:46:38.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.874+0000 7f874f400640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8734066690 0x7f8734068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:38.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.874+0000 7f874f400640 1 -- 172.21.15.66:0/2756267862 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8750084530 con 0x7f87580cc560 2024-03-30T00:46:38.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.874+0000 7f8756a00640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8734066690 0x7f8734068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:38.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.874+0000 7f8756a00640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8734066690 0x7f8734068b50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8748000f70 tx=0x7f8748009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:38.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:38.878+0000 7f874f400640 1 -- 172.21.15.66:0/2756267862 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8750051570 con 0x7f87580cc560 2024-03-30T00:46:39.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 -- 172.21.15.66:0/2756267862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f875806f510 con 0x7f87580cc560 2024-03-30T00:46:39.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f874f400640 1 -- 172.21.15.66:0/2756267862 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f8750054b90 con 0x7f87580cc560 2024-03-30T00:46:39.085 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:46:39.086 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:46:38.121763+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/1626022080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1626022080},{"type":"v1","addr":"172.21.15.66:6836","nonce":1626022080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:46:39.086 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 64 2024-03-30T00:46:39.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 -- 172.21.15.66:0/2756267862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8734066690 msgr2=0x7f8734068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8734066690 0x7f8734068b50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8748000f70 tx=0x7f8748009040 comp rx=0 tx=0).stop 2024-03-30T00:46:39.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 -- 172.21.15.66:0/2756267862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87580cc560 msgr2=0x7f8758157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87580cc560 0x7f8758157010 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f875000cb40 tx=0x7f875000a060 comp rx=0 tx=0).stop 2024-03-30T00:46:39.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 -- 172.21.15.66:0/2756267862 shutdown_connections 2024-03-30T00:46:39.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8734066690 0x7f8734068b50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87580cc560 0x7f8758157010 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875806f8e0 0x7f875815fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 --2- 172.21.15.66:0/2756267862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875806e800 0x7f87580b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 -- 172.21.15.66:0/2756267862 >> 172.21.15.66:0/2756267862 conn(0x7f87580d0540 msgr2=0x7f87580cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:39.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 -- 172.21.15.66:0/2756267862 shutdown_connections 2024-03-30T00:46:39.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.078+0000 7f875dc00640 1 -- 172.21.15.66:0/2756267862 wait complete. 2024-03-30T00:46:39.510 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:46:15.271965+0000', 'modified': '2024-03-30T00:46:38.121763+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7920}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7920': {'gid': 7920, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.66:6836/1626022080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 1626022080}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 1626022080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:46:39.510 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:46:39.510 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-30T00:46:39.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.642+0000 7f793dc00640 1 Processor -- start 2024-03-30T00:46:39.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.642+0000 7f793dc00640 1 -- start start 2024-03-30T00:46:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f793dc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f793815a670 0x7f79380b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f793dc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f79380d01d0 0x7f79380d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f793dc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79380cb520 con 0x7f79380ce710 2024-03-30T00:46:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f793dc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79380d0b80 con 0x7f793815a670 2024-03-30T00:46:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f793dc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79380cd500 con 0x7f79380d01d0 2024-03-30T00:46:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f7936a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f793815a670 0x7f79380b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f7936a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f793815a670 0x7f79380b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50856/0 (socket says 172.21.15.66:50856) 2024-03-30T00:46:39.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f7936a00640 1 -- 172.21.15.66:0/4203528124 learned_addr learned my addr 172.21.15.66:0/4203528124 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:39.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f7936a00640 1 -- 172.21.15.66:0/4203528124 >> v2:172.21.15.156:3300/0 conn(0x7f79380d01d0 msgr2=0x7f79380d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f7936a00640 1 --2- 172.21.15.66:0/4203528124 >> v2:172.21.15.156:3300/0 conn(0x7f79380d01d0 0x7f79380d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f7936a00640 1 -- 172.21.15.66:0/4203528124 >> v1:172.21.15.66:6789/0 conn(0x7f79380ce710 legacy=0x7f79380ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f7936000640 1 -- 172.21.15.66:0/4203528124 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102937758 0 0) 0x7f79380cb520 con 0x7f79380ce710 2024-03-30T00:46:39.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.646+0000 7f7936a00640 1 -- 172.21.15.66:0/4203528124 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79380cd7d0 con 0x7f793815a670 2024-03-30T00:46:39.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f7936a00640 1 --2- 172.21.15.66:0/4203528124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f793815a670 0x7f79380b0200 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f79380cab90 tx=0x7f792801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90735818c76ea1f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:39.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f7936000640 1 -- 172.21.15.66:0/4203528124 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79280aa650 con 0x7f793815a670 2024-03-30T00:46:39.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f7936000640 1 -- 172.21.15.66:0/4203528124 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79280aa7b0 con 0x7f793815a670 2024-03-30T00:46:39.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f7936000640 1 -- 172.21.15.66:0/4203528124 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79280aab40 con 0x7f793815a670 2024-03-30T00:46:39.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f793dc00640 1 -- 172.21.15.66:0/4203528124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f793815a670 msgr2=0x7f79380b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f793dc00640 1 --2- 172.21.15.66:0/4203528124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f793815a670 0x7f79380b0200 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f79380cab90 tx=0x7f792801c4f0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f793dc00640 1 -- 172.21.15.66:0/4203528124 shutdown_connections 2024-03-30T00:46:39.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f793dc00640 1 --2- 172.21.15.66:0/4203528124 >> v2:172.21.15.156:3300/0 conn(0x7f79380d01d0 0x7f79380d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f793dc00640 1 --2- 172.21.15.66:0/4203528124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f793815a670 0x7f79380b0200 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f793dc00640 1 -- 172.21.15.66:0/4203528124 >> 172.21.15.66:0/4203528124 conn(0x7f7938099870 msgr2=0x7f7938089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:39.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f793dc00640 1 -- 172.21.15.66:0/4203528124 shutdown_connections 2024-03-30T00:46:39.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f793dc00640 1 -- 172.21.15.66:0/4203528124 wait complete. 2024-03-30T00:46:39.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f793dc00640 1 Processor -- start 2024-03-30T00:46:39.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.650+0000 7f793dc00640 1 -- start start 2024-03-30T00:46:39.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f793dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79380ce710 0x7f7938153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f793dc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79380d01d0 0x7f793806bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f793dc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f793815a670 0x7f793806c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f793dc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79380c5260 con 0x7f79380ce710 2024-03-30T00:46:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f793dc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79380c8b40 con 0x7f793815a670 2024-03-30T00:46:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f793dc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79380c54b0 con 0x7f79380d01d0 2024-03-30T00:46:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f7936a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79380ce710 0x7f7938153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f7936a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79380ce710 0x7f7938153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50860/0 (socket says 172.21.15.66:50860) 2024-03-30T00:46:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f7936a00640 1 -- 172.21.15.66:0/3057243023 learned_addr learned my addr 172.21.15.66:0/3057243023 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f7936a00640 1 -- 172.21.15.66:0/3057243023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79380d01d0 msgr2=0x7f793806bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f7936a00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79380d01d0 0x7f793806bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f7936a00640 1 -- 172.21.15.66:0/3057243023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f793815a670 msgr2=0x7f793806c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f7936a00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f793815a670 0x7f793806c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f7936a00640 1 -- 172.21.15.66:0/3057243023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79280094d0 con 0x7f79380ce710 2024-03-30T00:46:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f7936a00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79380ce710 0x7f7938153ae0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7928019100 tx=0x7f792801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:39.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f792ea00640 1 -- 172.21.15.66:0/3057243023 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79280b3800 con 0x7f79380ce710 2024-03-30T00:46:39.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f792ea00640 1 -- 172.21.15.66:0/3057243023 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79280b3980 con 0x7f79380ce710 2024-03-30T00:46:39.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f792ea00640 1 -- 172.21.15.66:0/3057243023 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79280b3d30 con 0x7f79380ce710 2024-03-30T00:46:39.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f793dc00640 1 -- 172.21.15.66:0/3057243023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793815abd0 con 0x7f79380ce710 2024-03-30T00:46:39.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f793dc00640 1 -- 172.21.15.66:0/3057243023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79380c5bd0 con 0x7f79380ce710 2024-03-30T00:46:39.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.654+0000 7f792ea00640 1 -- 172.21.15.66:0/3057243023 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f79280c73a0 con 0x7f79380ce710 2024-03-30T00:46:39.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792ea00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7914066690 0x7f7914068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792ea00640 1 -- 172.21.15.66:0/3057243023 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f792811ef90 con 0x7f79380ce710 2024-03-30T00:46:39.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792fe00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7914066690 0x7f7914068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 Processor -- start 2024-03-30T00:46:39.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792fe00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7914066690 0x7f7914068b50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7920007fd0 tx=0x7f7920007920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:39.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 -- start start 2024-03-30T00:46:39.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e082c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e07e1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e07e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e06aa5a0 con 0x5614e082c6d0 2024-03-30T00:46:39.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e0637af0 con 0x5614e082bdb0 2024-03-30T00:46:39.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e06350c0 con 0x5614e07e24c0 2024-03-30T00:46:39.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e07e1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e07e1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50872/0 (socket says 172.21.15.66:50872) 2024-03-30T00:46:39.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792fe00640 1 -- 172.21.15.66:0/2961639419 learned_addr learned my addr 172.21.15.66:0/2961639419 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:39.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7936a00640 1 --2- 172.21.15.66:0/2961639419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e082c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7937400640 1 --2- 172.21.15.66:0/2961639419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e07e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792fe00640 1 -- 172.21.15.66:0/2961639419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 msgr2=0x5614e07e28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792fe00640 1 --2- 172.21.15.66:0/2961639419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e07e28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792fe00640 1 -- 172.21.15.66:0/2961639419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 msgr2=0x5614e082c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792fe00640 1 --2- 172.21.15.66:0/2961639419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e082c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792fe00640 1 -- 172.21.15.66:0/2961639419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e032c890 con 0x5614e082c6d0 2024-03-30T00:46:39.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f792fe00640 1 --2- 172.21.15.66:0/2961639419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e07e1f80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f79200054d0 tx=0x7f792000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f825afdf04b08f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:39.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7927400640 1 -- 172.21.15.66:0/2961639419 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792001b070 con 0x5614e082c6d0 2024-03-30T00:46:39.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7927400640 1 -- 172.21.15.66:0/2961639419 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7920012650 con 0x5614e082c6d0 2024-03-30T00:46:39.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7927400640 1 -- 172.21.15.66:0/2961639419 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79200129e0 con 0x5614e082c6d0 2024-03-30T00:46:39.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 -- 172.21.15.66:0/2961639419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 msgr2=0x5614e07e1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 --2- 172.21.15.66:0/2961639419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e07e1f80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f79200054d0 tx=0x7f792000f000 comp rx=0 tx=0).stop 2024-03-30T00:46:39.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 -- 172.21.15.66:0/2961639419 shutdown_connections 2024-03-30T00:46:39.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 --2- 172.21.15.66:0/2961639419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e07e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 --2- 172.21.15.66:0/2961639419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e07e1f80 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 --2- 172.21.15.66:0/2961639419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e082c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 -- 172.21.15.66:0/2961639419 >> 172.21.15.66:0/2961639419 conn(0x5614e07e8830 msgr2=0x5614e07e0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:39.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 -- 172.21.15.66:0/2961639419 shutdown_connections 2024-03-30T00:46:39.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.658+0000 7f7941152000 1 -- 172.21.15.66:0/2961639419 wait complete. 2024-03-30T00:46:39.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 -- start start 2024-03-30T00:46:39.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 Processor -- start 2024-03-30T00:46:39.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e082b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082bdb0 0x5614e07e6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082c6d0 0x5614e07e6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e06309b0 con 0x5614e082bdb0 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e05cfc00 con 0x5614e082c6d0 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e05ee070 con 0x5614e07e24c0 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 10 client.0 _ll_register_callbacks cb 0x5614e07ce0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f792fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082bdb0 0x5614e07e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f792fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082bdb0 0x5614e07e6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50878/0 (socket says 172.21.15.66:50878) 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f792fe00640 1 -- 172.21.15.66:0/3741434601 learned_addr learned my addr 172.21.15.66:0/3741434601 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7936a00640 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e082b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7937400640 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082c6d0 0x5614e07e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f792fe00640 1 -- 172.21.15.66:0/3741434601 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 msgr2=0x5614e082b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f792fe00640 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e082b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f792fe00640 1 -- 172.21.15.66:0/3741434601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082c6d0 msgr2=0x5614e07e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f792fe00640 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082c6d0 0x5614e07e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f792fe00640 1 -- 172.21.15.66:0/3741434601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79200051b0 con 0x5614e082bdb0 2024-03-30T00:46:39.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f792fe00640 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082bdb0 0x5614e07e6230 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x5614e05ed470 tx=0x7f7920001100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c256618a61067e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:39.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f791be00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:46:39.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f791be00640 1 -- 172.21.15.66:0/3741434601 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920007250 con 0x5614e082bdb0 2024-03-30T00:46:39.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f791be00640 1 -- 172.21.15.66:0/3741434601 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f79200073b0 con 0x5614e082bdb0 2024-03-30T00:46:39.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 -- 172.21.15.66:0/3741434601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614e05ed5f0 con 0x5614e082bdb0 2024-03-30T00:46:39.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f791be00640 1 -- 172.21.15.66:0/3741434601 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7920007570 con 0x5614e082bdb0 2024-03-30T00:46:39.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 10 client.7992 fetch_fsmap learned FSMap version 64 2024-03-30T00:46:39.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 -- 172.21.15.66:0/3741434601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x5614e05f5ab0 con 0x5614e082bdb0 2024-03-30T00:46:39.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f791be00640 1 -- 172.21.15.66:0/3741434601 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614e05ed5f0 con 0x5614e082bdb0 2024-03-30T00:46:39.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f791be00640 1 -- 172.21.15.66:0/3741434601 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 64) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f79200067c0 con 0x5614e082bdb0 2024-03-30T00:46:39.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 10 client.7992 fetch_fsmap finished waiting for FSMap version 64 2024-03-30T00:46:39.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 10 client.7992 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 20 client.7992 populate_metadata read hostname 'smithi066' 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5614e07e7180 0x5614e07e7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 4 client.7992 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7941152000 1 -- 172.21.15.66:0/3741434601 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614e062d790 con 0x5614e07e7180 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.662+0000 7f7936a00640 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5614e07e7180 0x5614e07e7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7936a00640 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5614e07e7180 0x5614e07e7560 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95389edc1baae433 server_cookie=e07abdda32a52769 in_seq=0 out_seq=0 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f791be00640 10 client.7992 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f791be00640 1 -- 172.21.15.66:0/3741434601 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5614e062d790 con 0x5614e07e7180 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f791be00640 10 client.7992 handle_command_reply: tid=0 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 1 client.7992 shutdown 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 1 -- 172.21.15.66:0/3741434601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082bdb0 msgr2=0x5614e07e6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082bdb0 0x5614e07e6230 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x5614e05ed470 tx=0x7f7920001100 comp rx=0 tx=0).stop 2024-03-30T00:46:39.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 1 -- 172.21.15.66:0/3741434601 shutdown_connections 2024-03-30T00:46:39.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5614e07e7180 0x5614e07e7560 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082c6d0 0x5614e07e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082bdb0 0x5614e07e6230 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 1 --2- 172.21.15.66:0/3741434601 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e082b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f791be00640 0 client.7992 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:39.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 1 -- 172.21.15.66:0/3741434601 >> 172.21.15.66:0/3741434601 conn(0x5614e07e8830 msgr2=0x5614e07e0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:39.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 1 -- 172.21.15.66:0/3741434601 shutdown_connections 2024-03-30T00:46:39.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 1 -- 172.21.15.66:0/3741434601 wait complete. 2024-03-30T00:46:39.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.666+0000 7f7941152000 20 client.7992 trim_cache size 0 max 16384 2024-03-30T00:46:39.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 Processor -- start 2024-03-30T00:46:39.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 -- start start 2024-03-30T00:46:39.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e0885040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e0885580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e0885ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e02ed940 con 0x5614e082c6d0 2024-03-30T00:46:39.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e05cdc10 con 0x5614e082bdb0 2024-03-30T00:46:39.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e0633510 con 0x5614e07e24c0 2024-03-30T00:46:39.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7937400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e0885ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7937400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e0885ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50884/0 (socket says 172.21.15.66:50884) 2024-03-30T00:46:39.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7937400640 1 -- 172.21.15.66:0/1577906368 learned_addr learned my addr 172.21.15.66:0/1577906368 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:39.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f792fe00640 1 --2- 172.21.15.66:0/1577906368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e0885580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7936a00640 1 --2- 172.21.15.66:0/1577906368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e0885040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7937400640 1 -- 172.21.15.66:0/1577906368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 msgr2=0x5614e0885040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7937400640 1 --2- 172.21.15.66:0/1577906368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e0885040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7937400640 1 -- 172.21.15.66:0/1577906368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 msgr2=0x5614e0885580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7937400640 1 --2- 172.21.15.66:0/1577906368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e0885580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7937400640 1 -- 172.21.15.66:0/1577906368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e05dbc00 con 0x5614e082c6d0 2024-03-30T00:46:39.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7937400640 1 --2- 172.21.15.66:0/1577906368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e0885ac0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f7930009f60 tx=0x7f7930009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d667d54f1db760 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7927e00640 1 -- 172.21.15.66:0/1577906368 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930009ae0 con 0x5614e082c6d0 2024-03-30T00:46:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7927e00640 1 -- 172.21.15.66:0/1577906368 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7930012040 con 0x5614e082c6d0 2024-03-30T00:46:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7927e00640 1 -- 172.21.15.66:0/1577906368 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930015980 con 0x5614e082c6d0 2024-03-30T00:46:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 -- 172.21.15.66:0/1577906368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 msgr2=0x5614e0885ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 --2- 172.21.15.66:0/1577906368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e0885ac0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f7930009f60 tx=0x7f7930009670 comp rx=0 tx=0).stop 2024-03-30T00:46:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 -- 172.21.15.66:0/1577906368 shutdown_connections 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 --2- 172.21.15.66:0/1577906368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e082c6d0 0x5614e0885ac0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 --2- 172.21.15.66:0/1577906368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e0885580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 --2- 172.21.15.66:0/1577906368 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e07e24c0 0x5614e0885040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 -- 172.21.15.66:0/1577906368 >> 172.21.15.66:0/1577906368 conn(0x5614e07e8830 msgr2=0x5614e0829700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 -- 172.21.15.66:0/1577906368 shutdown_connections 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.682+0000 7f7941152000 1 -- 172.21.15.66:0/1577906368 wait complete. 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 -- start start 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 Processor -- start 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e07e24c0 0x5614e0886090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e083f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e082c6d0 0x5614e083faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e062ddb0 con 0x5614e07e24c0 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e0673ac0 con 0x5614e082bdb0 2024-03-30T00:46:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e062e1c0 con 0x5614e082c6d0 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 10 client.0 _ll_register_callbacks cb 0x5614e05f5590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7936a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e07e24c0 0x5614e0886090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7936a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e07e24c0 0x5614e0886090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50886/0 (socket says 172.21.15.66:50886) 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7936a00640 1 -- 172.21.15.66:0/112963513 learned_addr learned my addr 172.21.15.66:0/112963513 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f792fe00640 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e083f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7937400640 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e082c6d0 0x5614e083faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7936a00640 1 -- 172.21.15.66:0/112963513 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e082c6d0 msgr2=0x5614e083faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7936a00640 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e082c6d0 0x5614e083faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7936a00640 1 -- 172.21.15.66:0/112963513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 msgr2=0x5614e083f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7936a00640 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e083f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7936a00640 1 -- 172.21.15.66:0/112963513 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7930009c40 con 0x5614e07e24c0 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7936a00640 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e07e24c0 0x5614e0886090 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f792811d210 tx=0x7f79280f2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576fd20f7ed20744 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f791be00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f791be00640 1 -- 172.21.15.66:0/112963513 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79280ee220 con 0x5614e07e24c0 2024-03-30T00:46:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f791be00640 1 -- 172.21.15.66:0/112963513 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f79280ef2b0 con 0x5614e07e24c0 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 -- 172.21.15.66:0/112963513 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614e0600460 con 0x5614e07e24c0 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f791be00640 1 -- 172.21.15.66:0/112963513 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79280ef470 con 0x5614e07e24c0 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 10 client.7998 fetch_fsmap learned FSMap version 64 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 -- 172.21.15.66:0/112963513 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x5614e032ccc0 con 0x5614e07e24c0 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f791be00640 1 -- 172.21.15.66:0/112963513 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614e0600460 con 0x5614e07e24c0 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f791be00640 1 -- 172.21.15.66:0/112963513 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 64) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f79280f0d20 con 0x5614e07e24c0 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 10 client.7998 fetch_fsmap finished waiting for FSMap version 64 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 10 client.7998 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 20 client.7998 populate_metadata read hostname 'smithi066' 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5614e0840030 0x5614e0840410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 4 client.7998 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f7941152000 1 -- 172.21.15.66:0/112963513 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5614e05f99c0 con 0x5614e0840030 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f792fe00640 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5614e0840030 0x5614e0840410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f792fe00640 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5614e0840030 0x5614e0840410 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9834d7ab09dd692 server_cookie=f3ffc971ec095d5f in_seq=0 out_seq=0 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.686+0000 7f791be00640 10 client.7998 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f791be00640 1 -- 172.21.15.66:0/112963513 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5614e05f99c0 con 0x5614e0840030 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f791be00640 10 client.7998 handle_command_reply: tid=0 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 1 client.7998 shutdown 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 1 -- 172.21.15.66:0/112963513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e07e24c0 msgr2=0x5614e0886090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e07e24c0 0x5614e0886090 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f792811d210 tx=0x7f79280f2040 comp rx=0 tx=0).stop 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 1 -- 172.21.15.66:0/112963513 shutdown_connections 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5614e0840030 0x5614e0840410 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614e082c6d0 0x5614e083faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614e082bdb0 0x5614e083f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 1 --2- 172.21.15.66:0/112963513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614e07e24c0 0x5614e0886090 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f791be00640 0 client.7998 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 1 -- 172.21.15.66:0/112963513 >> 172.21.15.66:0/112963513 conn(0x5614e07e8830 msgr2=0x5614e0829e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 1 -- 172.21.15.66:0/112963513 shutdown_connections 2024-03-30T00:46:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 1 -- 172.21.15.66:0/112963513 wait complete. 2024-03-30T00:46:39.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f7941152000 20 client.7998 trim_cache size 0 max 16384 2024-03-30T00:46:39.703 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:46:39.703 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:46:39.703 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:46:39.703 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 7920, 2024-03-30T00:46:39.703 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:46:39.703 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:46:39.703 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:46:39.703 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 4.5713410330000004, 2024-03-30T00:46:39.703 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 64, 2024-03-30T00:46:39.704 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 97, 2024-03-30T00:46:39.704 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 97, 2024-03-30T00:46:39.704 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 5.5147903979999997 2024-03-30T00:46:39.704 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:46:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 -- 172.21.15.66:0/3057243023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7914066690 msgr2=0x7f7914068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7914066690 0x7f7914068b50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7920007fd0 tx=0x7f7920007920 comp rx=0 tx=0).stop 2024-03-30T00:46:39.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 -- 172.21.15.66:0/3057243023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79380ce710 msgr2=0x7f7938153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79380ce710 0x7f7938153ae0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7928019100 tx=0x7f792801ad30 comp rx=0 tx=0).stop 2024-03-30T00:46:39.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 -- 172.21.15.66:0/3057243023 shutdown_connections 2024-03-30T00:46:39.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f793815a670 0x7f793806c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79380d01d0 0x7f793806bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7914066690 0x7f7914068b50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 --2- 172.21.15.66:0/3057243023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79380ce710 0x7f7938153ae0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 -- 172.21.15.66:0/3057243023 >> 172.21.15.66:0/3057243023 conn(0x7f7938099870 msgr2=0x7f793806f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:39.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 -- 172.21.15.66:0/3057243023 shutdown_connections 2024-03-30T00:46:39.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.690+0000 7f793dc00640 1 -- 172.21.15.66:0/3057243023 wait complete. 2024-03-30T00:46:39.717 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:46:39.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd21a000640 1 Processor -- start 2024-03-30T00:46:39.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd21a000640 1 -- start start 2024-03-30T00:46:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd21a000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd214160bc0 0x7fd214167840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd21a000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21406da20 con 0x7fd2141610c0 2024-03-30T00:46:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd21a000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21406dc60 con 0x7fd214157060 2024-03-30T00:46:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd21a000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd214157440 con 0x7fd214160bc0 2024-03-30T00:46:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd20be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd2141610c0 0x7fd2141604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:59530/0 (socket says 172.21.15.66:59530) 2024-03-30T00:46:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd20be00640 1 -- 172.21.15.66:0/2873920200 learned_addr learned my addr 172.21.15.66:0/2873920200 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:46:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd213e00640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd214160bc0 0x7fd214167840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169292292 0 0) 0x7fd1fc001120 con 0x7fd2141610c0 2024-03-30T00:46:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1f8003530 con 0x7fd2141610c0 2024-03-30T00:46:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 396857186 0 0) 0x7fd1f8003530 con 0x7fd2141610c0 2024-03-30T00:46:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd214160bc0 msgr2=0x7fd214167840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd212a00640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd214160bc0 0x7fd214167840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 >> v1:172.21.15.156:6789/0 conn(0x7fd214157060 legacy=0x7fd21406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21406df40 con 0x7fd2141610c0 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fd1fc001810 con 0x7fd2141610c0 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.850+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 >> v1:172.21.15.66:6789/0 conn(0x7fd2141610c0 legacy=0x7fd2141604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd212a00640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 0x7fd1f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd212a00640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1f80047d0 0x7fd1f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd212a00640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd214160bc0 0x7fd1f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd214157440 con 0x7fd1f80039b0 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f800baf0 con 0x7fd1f80047d0 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f800bcb0 con 0x7fd214160bc0 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd213400640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 0x7fd1f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd213e00640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd214160bc0 0x7fd1f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd20be00640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1f80047d0 0x7fd1f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd213400640 1 -- 172.21.15.66:0/2873920200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd214160bc0 msgr2=0x7fd1f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd213400640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd214160bc0 0x7fd1f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd213400640 1 -- 172.21.15.66:0/2873920200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1f80047d0 msgr2=0x7fd1f8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:39.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd213400640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1f80047d0 0x7fd1f8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd213400640 1 -- 172.21.15.66:0/2873920200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f8003530 con 0x7fd1f80039b0 2024-03-30T00:46:39.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd213400640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 0x7fd1f8003d90 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fd204001860 tx=0x7fd20401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7074de0c4f9c109 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:39.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040ab600 con 0x7fd1f80039b0 2024-03-30T00:46:39.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2040a7050 con 0x7fd1f80039b0 2024-03-30T00:46:39.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.854+0000 7fd212a00640 1 -- 172.21.15.66:0/2873920200 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040ab9a0 con 0x7fd1f80039b0 2024-03-30T00:46:39.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 -- 172.21.15.66:0/2873920200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 msgr2=0x7fd1f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 --2- 172.21.15.66:0/2873920200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 0x7fd1f8003d90 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fd204001860 tx=0x7fd20401c520 comp rx=0 tx=0).stop 2024-03-30T00:46:39.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd213e00640 1 -- 172.21.15.66:0/2873920200 reap_dead start 2024-03-30T00:46:39.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 -- 172.21.15.66:0/2873920200 shutdown_connections 2024-03-30T00:46:39.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 -- 172.21.15.66:0/2873920200 >> 172.21.15.66:0/2873920200 conn(0x7fd214071750 msgr2=0x7fd214069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:39.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 -- 172.21.15.66:0/2873920200 shutdown_connections 2024-03-30T00:46:39.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 -- 172.21.15.66:0/2873920200 wait complete. 2024-03-30T00:46:39.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 Processor -- start 2024-03-30T00:46:39.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 -- start start 2024-03-30T00:46:39.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 0x7fd2140ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd21414f810 0x7fd21414eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd214157580 0x7fd214152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21414f630 con 0x7fd1f80039b0 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21414f0a0 con 0x7fd214157580 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd21a000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd214150850 con 0x7fd21414f810 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd213400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 0x7fd2140ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd213400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 0x7fd2140ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50894/0 (socket says 172.21.15.66:50894) 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd213400640 1 -- 172.21.15.66:0/322514247 learned_addr learned my addr 172.21.15.66:0/322514247 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd20be00640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd21414f810 0x7fd21414eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd213e00640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd214157580 0x7fd214152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd213400640 1 -- 172.21.15.66:0/322514247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd21414f810 msgr2=0x7fd21414eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd213400640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd21414f810 0x7fd21414eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd213400640 1 -- 172.21.15.66:0/322514247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd214157580 msgr2=0x7fd214152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd213400640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd214157580 0x7fd214152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:39.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.858+0000 7fd213400640 1 -- 172.21.15.66:0/322514247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd214150b30 con 0x7fd1f80039b0 2024-03-30T00:46:39.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.862+0000 7fd213400640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 0x7fd2140ce100 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd2040017b0 tx=0x7fd1f8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:39.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.862+0000 7fd20b400640 1 -- 172.21.15.66:0/322514247 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204005270 con 0x7fd1f80039b0 2024-03-30T00:46:39.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.862+0000 7fd20b400640 1 -- 172.21.15.66:0/322514247 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2040053d0 con 0x7fd1f80039b0 2024-03-30T00:46:39.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.862+0000 7fd20b400640 1 -- 172.21.15.66:0/322514247 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040abb80 con 0x7fd1f80039b0 2024-03-30T00:46:39.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.862+0000 7fd21a000640 1 -- 172.21.15.66:0/322514247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd214150e10 con 0x7fd1f80039b0 2024-03-30T00:46:39.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.862+0000 7fd21a000640 1 -- 172.21.15.66:0/322514247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21414f2c0 con 0x7fd1f80039b0 2024-03-30T00:46:39.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.862+0000 7fd21a000640 1 -- 172.21.15.66:0/322514247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd214066c30 con 0x7fd1f80039b0 2024-03-30T00:46:39.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.862+0000 7fd20b400640 1 -- 172.21.15.66:0/322514247 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd2040b2050 con 0x7fd1f80039b0 2024-03-30T00:46:39.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.862+0000 7fd20b400640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1f0066690 0x7fd1f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:39.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.862+0000 7fd20b400640 1 -- 172.21.15.66:0/322514247 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fd20411e340 con 0x7fd1f80039b0 2024-03-30T00:46:39.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.862+0000 7fd20be00640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1f0066690 0x7fd1f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:39.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.866+0000 7fd20be00640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1f0066690 0x7fd1f0068b50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd2140b39b0 tx=0x7fd1fc007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:39.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:39.866+0000 7fd20b400640 1 -- 172.21.15.66:0/322514247 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd2040eb540 con 0x7fd1f80039b0 2024-03-30T00:46:40.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.062+0000 7fd21a000640 1 -- 172.21.15.66:0/322514247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd2140b3f50 con 0x7fd1f80039b0 2024-03-30T00:46:40.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd20b400640 1 -- 172.21.15.66:0/322514247 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd2040eeb60 con 0x7fd1f80039b0 2024-03-30T00:46:40.073 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:46:40.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 -- 172.21.15.66:0/322514247 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1f0066690 msgr2=0x7fd1f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:40.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1f0066690 0x7fd1f0068b50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd2140b39b0 tx=0x7fd1fc007d50 comp rx=0 tx=0).stop 2024-03-30T00:46:40.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 -- 172.21.15.66:0/322514247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 msgr2=0x7fd2140ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:40.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 0x7fd2140ce100 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd2040017b0 tx=0x7fd1f8005800 comp rx=0 tx=0).stop 2024-03-30T00:46:40.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 -- 172.21.15.66:0/322514247 shutdown_connections 2024-03-30T00:46:40.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd214157580 0x7fd214152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:40.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd21414f810 0x7fd21414eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:40.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1f0066690 0x7fd1f0068b50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:40.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 --2- 172.21.15.66:0/322514247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80039b0 0x7fd2140ce100 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:40.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 -- 172.21.15.66:0/322514247 >> 172.21.15.66:0/322514247 conn(0x7fd214071750 msgr2=0x7fd2140730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:40.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 -- 172.21.15.66:0/322514247 shutdown_connections 2024-03-30T00:46:40.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:40.066+0000 7fd21a000640 1 -- 172.21.15.66:0/322514247 wait complete. 2024-03-30T00:46:40.094 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:46:40.095 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:46:40.095 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:46:40.095 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:46:40.095 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:40.095 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:46:40.095 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:46:40.095 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:46:40.095 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:46:40.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:40.095 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:46:40.105 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:46:40.105 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:46:40.105 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:46:40.106 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42078sec preferred_lft 42078sec 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:46:40.107 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:40.108 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:46:40.108 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:46:40.108 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:46:40.108 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:40.108 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:46:40.108 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:46:40.108 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:46:40.108 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:46:40.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:40.108 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:46:40.127 INFO:tasks.ceph.mds.a.smithi066.stderr:2024-03-30T00:46:40.118+0000 7f8a28e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:46:40.145 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:46:40.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:40.148 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:46:40.200 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:46:40.200 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:46:40.200 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:46:40.200 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:46:40.200 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:46:40.200 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:46:40.200 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:46:40.201 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:46:40.201 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:46:40.201 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:46:40.201 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:46:40.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:40.201 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:46:40.201 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:46:40.201 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:46:40.201 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:46:40.201 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:46:40.202 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:46:40.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:40.295 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:46:40.303 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:46:40.303 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:46:40.304 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:46:40.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:40.304 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:46:40.368 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:46:40.368 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:46:40.368 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:40.420 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:46:40.420 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:46:40.421 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:40.472 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:46:40.533 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:46:40.533 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:46:40.599 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:46:40.600 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:46:40.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:40.601 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:46:40.652 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:46:40.652 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:46:40.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:46:40.720 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:46:40.720 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:46:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:46:40.789 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:46:40.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:40.790 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:46:40.795 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:46:40.786+0000 7f82961fc580 -1 init, newargv = 0x563692555db0 newargc=15 2024-03-30T00:46:40.795 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[48168]: starting ceph client 2024-03-30T00:46:40.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[48168]: starting fuse 2024-03-30T00:46:40.804 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:46:40.804 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:46:40.806 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:46:40.807 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:46:40.807 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:46:40.807 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:46:40.807 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:46:40.933 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.48168.asok 2024-03-30T00:46:41.117 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48168.asok 2024-03-30T00:46:41.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:41.117 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48168.asok status 2024-03-30T00:46:41.291 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "48168", 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 8013, 2024-03-30T00:46:41.292 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 8013 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2595009659 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:46:41.293 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2595009659 2024-03-30T00:46:41.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:46:41.294 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.8013 192.168.0.1:0/2595009659", 2024-03-30T00:46:41.294 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/2595009659", 2024-03-30T00:46:41.294 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:46:41.294 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 64, 2024-03-30T00:46:41.294 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 97, 2024-03-30T00:46:41.294 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:46:41.294 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:46:41.294 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:46:41.294 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:46:41.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:41.311 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:46:41.311 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:41.321 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:46:41.321 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:41.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:41.321 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:46:41.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:46:41.383 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-03-30T00:46:41.440 INFO:tasks.ceph.mds.a.smithi066.stderr:2024-03-30T00:46:41.434+0000 7f8a24800640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-30T00:46:41.441 INFO:teuthology.orchestra.run.smithi066.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-03-30T00:46:41.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:46:41.443 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-03-30T00:46:41.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.618+0000 7f7af3000640 1 Processor -- start 2024-03-30T00:46:41.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.618+0000 7f7af3000640 1 -- start start 2024-03-30T00:46:41.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7af3000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec06e7f0 0x7f7aec06ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7af3000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7aec0cbd30 0x7f7aec06d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7af3000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aec0cbb50 con 0x7f7aec0cb650 2024-03-30T00:46:41.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7af3000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aec06a860 con 0x7f7aec0cbd30 2024-03-30T00:46:41.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7af3000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0cd340 con 0x7f7aec06e7f0 2024-03-30T00:46:41.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7ae3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7aec0cbd30 0x7f7aec06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7ae3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7aec0cbd30 0x7f7aec06d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50906/0 (socket says 172.21.15.66:50906) 2024-03-30T00:46:41.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7ae3400640 1 -- 172.21.15.66:0/4285458097 learned_addr learned my addr 172.21.15.66:0/4285458097 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7ae3400640 1 -- 172.21.15.66:0/4285458097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec06e7f0 msgr2=0x7f7aec06ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7aebe00640 1 --2- 172.21.15.66:0/4285458097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec06e7f0 0x7f7aec06ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7ae3400640 1 --2- 172.21.15.66:0/4285458097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec06e7f0 0x7f7aec06ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7ae3400640 1 -- 172.21.15.66:0/4285458097 >> v1:172.21.15.66:6789/0 conn(0x7f7aec0cb650 legacy=0x7f7aec06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7aeb400640 1 -- 172.21.15.66:0/4285458097 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549357209 0 0) 0x7f7aec0cbb50 con 0x7f7aec0cb650 2024-03-30T00:46:41.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7ae3400640 1 -- 172.21.15.66:0/4285458097 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec06f110 con 0x7f7aec0cbd30 2024-03-30T00:46:41.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.622+0000 7f7aebe00640 1 --2- 172.21.15.66:0/4285458097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec06e7f0 0x7f7aec06ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7ae3400640 1 --2- 172.21.15.66:0/4285458097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aec0cbd30 0x7f7aec06d220 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009870 tx=0x7f7ad401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85935da1058693b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7aeb400640 1 -- 172.21.15.66:0/4285458097 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40aa620 con 0x7f7aec0cbd30 2024-03-30T00:46:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7aeb400640 1 -- 172.21.15.66:0/4285458097 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad40aa780 con 0x7f7aec0cbd30 2024-03-30T00:46:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7aeb400640 1 -- 172.21.15.66:0/4285458097 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40aab10 con 0x7f7aec0cbd30 2024-03-30T00:46:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 -- 172.21.15.66:0/4285458097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aec0cbd30 msgr2=0x7f7aec06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 --2- 172.21.15.66:0/4285458097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aec0cbd30 0x7f7aec06d220 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009870 tx=0x7f7ad401c4a0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 -- 172.21.15.66:0/4285458097 shutdown_connections 2024-03-30T00:46:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 --2- 172.21.15.66:0/4285458097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aec0cbd30 0x7f7aec06d220 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 --2- 172.21.15.66:0/4285458097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec06e7f0 0x7f7aec06ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 -- 172.21.15.66:0/4285458097 >> 172.21.15.66:0/4285458097 conn(0x7f7aec0b1030 msgr2=0x7f7aec0b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 -- 172.21.15.66:0/4285458097 shutdown_connections 2024-03-30T00:46:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 -- 172.21.15.66:0/4285458097 wait complete. 2024-03-30T00:46:41.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 Processor -- start 2024-03-30T00:46:41.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 -- start start 2024-03-30T00:46:41.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aec06e7f0 0x7f7aec0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7aec0cb650 0x7f7aec1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec0cbd30 0x7f7aec0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0b2940 con 0x7f7aec06e7f0 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0b2c20 con 0x7f7aec0cb650 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af3000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0b2f00 con 0x7f7aec0cbd30 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7aebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aec06e7f0 0x7f7aec0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7aebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aec06e7f0 0x7f7aec0b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50914/0 (socket says 172.21.15.66:50914) 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7aebe00640 1 -- 172.21.15.66:0/4016187391 learned_addr learned my addr 172.21.15.66:0/4016187391 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7ae3400640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7aec0cb650 0x7f7aec1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.626+0000 7f7af1000640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec0cbd30 0x7f7aec0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7aebe00640 1 -- 172.21.15.66:0/4016187391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec0cbd30 msgr2=0x7f7aec0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7aebe00640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec0cbd30 0x7f7aec0b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7aebe00640 1 -- 172.21.15.66:0/4016187391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7aec0cb650 msgr2=0x7f7aec1587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7aebe00640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7aec0cb650 0x7f7aec1587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7aebe00640 1 -- 172.21.15.66:0/4016187391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad40094d0 con 0x7f7aec06e7f0 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7af1000640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec0cbd30 0x7f7aec0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7aebe00640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aec06e7f0 0x7f7aec0b39e0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00c7f0 tx=0x7f7adc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7ae8c00640 1 -- 172.21.15.66:0/4016187391 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc007820 con 0x7f7aec06e7f0 2024-03-30T00:46:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7ae8c00640 1 -- 172.21.15.66:0/4016187391 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc007980 con 0x7f7aec06e7f0 2024-03-30T00:46:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7af3000640 1 -- 172.21.15.66:0/4016187391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec0aedc0 con 0x7f7aec06e7f0 2024-03-30T00:46:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7ae8c00640 1 -- 172.21.15.66:0/4016187391 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc007d30 con 0x7f7aec06e7f0 2024-03-30T00:46:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7af3000640 1 -- 172.21.15.66:0/4016187391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec0b34c0 con 0x7f7aec06e7f0 2024-03-30T00:46:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7ae8c00640 1 -- 172.21.15.66:0/4016187391 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7adc0113f0 con 0x7f7aec06e7f0 2024-03-30T00:46:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7ae8c00640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ac80666c0 0x7f7ac8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7ae8c00640 1 -- 172.21.15.66:0/4016187391 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f7adc084650 con 0x7f7aec06e7f0 2024-03-30T00:46:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.630+0000 7f7ae3400640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ac80666c0 0x7f7ac8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 Processor -- start 2024-03-30T00:46:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 -- start start 2024-03-30T00:46:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7ae3400640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ac80666c0 0x7f7ac8068b80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40ae000 tx=0x7f7ad4004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec485db820 0x55ec485dbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec48592910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48593230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4845b100 con 0x55ec48592e50 2024-03-30T00:46:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec483e8d10 con 0x55ec485dc140 2024-03-30T00:46:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec483e62e0 con 0x55ec485db820 2024-03-30T00:46:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48593230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48593230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50918/0 (socket says 172.21.15.66:50918) 2024-03-30T00:46:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af1000640 1 -- 172.21.15.66:0/3980303247 learned_addr learned my addr 172.21.15.66:0/3980303247 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7ae3400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec48592910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7aebe00640 1 --2- 172.21.15.66:0/3980303247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec485db820 0x55ec485dbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af1000640 1 -- 172.21.15.66:0/3980303247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec485db820 msgr2=0x55ec485dbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af1000640 1 --2- 172.21.15.66:0/3980303247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec485db820 0x55ec485dbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af1000640 1 -- 172.21.15.66:0/3980303247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 msgr2=0x55ec48592910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af1000640 1 --2- 172.21.15.66:0/3980303247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec48592910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af1000640 1 -- 172.21.15.66:0/3980303247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec480ddfb0 con 0x55ec48592e50 2024-03-30T00:46:41.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af1000640 1 --2- 172.21.15.66:0/3980303247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48593230 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4007ba0 tx=0x7f7ae4002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23f1a5dd0bf83cb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7ae0c00640 1 -- 172.21.15.66:0/3980303247 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4003b90 con 0x55ec48592e50 2024-03-30T00:46:41.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7ae0c00640 1 -- 172.21.15.66:0/3980303247 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae4003cf0 con 0x55ec48592e50 2024-03-30T00:46:41.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7ae0c00640 1 -- 172.21.15.66:0/3980303247 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400ea60 con 0x55ec48592e50 2024-03-30T00:46:41.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 -- 172.21.15.66:0/3980303247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 msgr2=0x55ec48593230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 --2- 172.21.15.66:0/3980303247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48593230 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4007ba0 tx=0x7f7ae4002130 comp rx=0 tx=0).stop 2024-03-30T00:46:41.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 -- 172.21.15.66:0/3980303247 shutdown_connections 2024-03-30T00:46:41.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 --2- 172.21.15.66:0/3980303247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48593230 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 --2- 172.21.15.66:0/3980303247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec48592910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 --2- 172.21.15.66:0/3980303247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec485db820 0x55ec485dbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 -- 172.21.15.66:0/3980303247 >> 172.21.15.66:0/3980303247 conn(0x55ec48391280 msgr2=0x55ec485911f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 -- 172.21.15.66:0/3980303247 shutdown_connections 2024-03-30T00:46:41.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.634+0000 7f7af64a3000 1 -- 172.21.15.66:0/3980303247 wait complete. 2024-03-30T00:46:41.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 -- start start 2024-03-30T00:46:41.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 Processor -- start 2024-03-30T00:46:41.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec48592e50 0x55ec4859d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec485e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec485e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec483e1bd0 con 0x55ec485db820 2024-03-30T00:46:41.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec48380f60 con 0x55ec485dc140 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec4839f180 con 0x55ec48592e50 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 10 client.0 _ll_register_callbacks cb 0x55ec483db240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ae3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec485e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ae3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec485e3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50930/0 (socket says 172.21.15.66:50930) 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ae3400640 1 -- 172.21.15.66:0/2778171190 learned_addr learned my addr 172.21.15.66:0/2778171190 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7aebe00640 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec48592e50 0x55ec4859d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af1000640 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec485e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ae3400640 1 -- 172.21.15.66:0/2778171190 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec48592e50 msgr2=0x55ec4859d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ae3400640 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec48592e50 0x55ec4859d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ae3400640 1 -- 172.21.15.66:0/2778171190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 msgr2=0x55ec485e40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ae3400640 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec485e40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ae3400640 1 -- 172.21.15.66:0/2778171190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae400f040 con 0x55ec485db820 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af1000640 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec485e40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ae3400640 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec485e3b90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f7ad401c4e0 tx=0x7f7ad4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3475c2bd6de990 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ad9600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:46:41.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ad9600640 1 -- 172.21.15.66:0/2778171190 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40ce070 con 0x55ec485db820 2024-03-30T00:46:41.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ad9600640 1 -- 172.21.15.66:0/2778171190 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f7ad40043c0 con 0x55ec485db820 2024-03-30T00:46:41.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 -- 172.21.15.66:0/2778171190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec483b1850 con 0x55ec485db820 2024-03-30T00:46:41.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ad9600640 1 -- 172.21.15.66:0/2778171190 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ad40c4050 con 0x55ec485db820 2024-03-30T00:46:41.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 10 client.8028 fetch_fsmap learned FSMap version 64 2024-03-30T00:46:41.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 -- 172.21.15.66:0/2778171190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55ec483a6bc0 con 0x55ec485db820 2024-03-30T00:46:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ad9600640 1 -- 172.21.15.66:0/2778171190 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec483b1850 con 0x55ec485db820 2024-03-30T00:46:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7ad9600640 1 -- 172.21.15.66:0/2778171190 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 64) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7ad4003620 con 0x55ec485db820 2024-03-30T00:46:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 10 client.8028 fetch_fsmap finished waiting for FSMap version 64 2024-03-30T00:46:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 10 client.8028 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:46:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 20 client.8028 populate_metadata read hostname 'smithi066' 2024-03-30T00:46:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ec48598a80 0x55ec485edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 4 client.8028 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:46:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7af64a3000 1 -- 172.21.15.66:0/2778171190 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec483de9b0 con 0x55ec48598a80 2024-03-30T00:46:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.638+0000 7f7aebe00640 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ec48598a80 0x55ec485edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7aebe00640 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ec48598a80 0x55ec485edfe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb7b067c2620a66 server_cookie=337498fe09feeb80 in_seq=0 out_seq=0 2024-03-30T00:46:41.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7ad9600640 10 client.8028 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7ad9600640 1 -- 172.21.15.66:0/2778171190 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec483de9b0 con 0x55ec48598a80 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7ad9600640 10 client.8028 handle_command_reply: tid=0 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 1 client.8028 shutdown 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 1 -- 172.21.15.66:0/2778171190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 msgr2=0x55ec485e3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec485e3b90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f7ad401c4e0 tx=0x7f7ad4019240 comp rx=0 tx=0).stop 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 1 -- 172.21.15.66:0/2778171190 shutdown_connections 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ec48598a80 0x55ec485edfe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec485e40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec485e3b90 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 1 --2- 172.21.15.66:0/2778171190 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec48592e50 0x55ec4859d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7ad9600640 0 client.8028 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 1 -- 172.21.15.66:0/2778171190 >> 172.21.15.66:0/2778171190 conn(0x55ec48391280 msgr2=0x55ec485918f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 1 -- 172.21.15.66:0/2778171190 shutdown_connections 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 1 -- 172.21.15.66:0/2778171190 wait complete. 2024-03-30T00:46:41.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.642+0000 7f7af64a3000 20 client.8028 trim_cache size 0 max 16384 2024-03-30T00:46:41.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.654+0000 7f7af64a3000 1 Processor -- start 2024-03-30T00:46:41.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.654+0000 7f7af64a3000 1 -- start start 2024-03-30T00:46:41.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48635780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec485db820 0x55ec48635cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec48636200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec485974e0 con 0x55ec48592e50 2024-03-30T00:46:41.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4809ec30 con 0x55ec485dc140 2024-03-30T00:46:41.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec483b1fb0 con 0x55ec485db820 2024-03-30T00:46:41.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7aebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48635780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7aebe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48635780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50942/0 (socket says 172.21.15.66:50942) 2024-03-30T00:46:41.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7aebe00640 1 -- 172.21.15.66:0/3862955958 learned_addr learned my addr 172.21.15.66:0/3862955958 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7ae3400640 1 --2- 172.21.15.66:0/3862955958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec485db820 0x55ec48635cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af1000640 1 --2- 172.21.15.66:0/3862955958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec48636200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7aebe00640 1 -- 172.21.15.66:0/3862955958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec485db820 msgr2=0x55ec48635cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7aebe00640 1 --2- 172.21.15.66:0/3862955958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec485db820 0x55ec48635cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7aebe00640 1 -- 172.21.15.66:0/3862955958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 msgr2=0x55ec48636200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7aebe00640 1 --2- 172.21.15.66:0/3862955958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec48636200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7aebe00640 1 -- 172.21.15.66:0/3862955958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec4838cf10 con 0x55ec48592e50 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7aebe00640 1 --2- 172.21.15.66:0/3862955958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48635780 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0554a0 tx=0x7f7adc013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276dbdfd8df24f85 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7ae1600640 1 -- 172.21.15.66:0/3862955958 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc053650 con 0x55ec48592e50 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7ae1600640 1 -- 172.21.15.66:0/3862955958 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc0829e0 con 0x55ec48592e50 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7ae1600640 1 -- 172.21.15.66:0/3862955958 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc051c20 con 0x55ec48592e50 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 -- 172.21.15.66:0/3862955958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 msgr2=0x55ec48635780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 --2- 172.21.15.66:0/3862955958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48635780 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0554a0 tx=0x7f7adc013040 comp rx=0 tx=0).stop 2024-03-30T00:46:41.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 -- 172.21.15.66:0/3862955958 shutdown_connections 2024-03-30T00:46:41.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 --2- 172.21.15.66:0/3862955958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec48636200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 --2- 172.21.15.66:0/3862955958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec485db820 0x55ec48635cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 --2- 172.21.15.66:0/3862955958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec48592e50 0x55ec48635780 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 -- 172.21.15.66:0/3862955958 >> 172.21.15.66:0/3862955958 conn(0x55ec48391280 msgr2=0x55ec485d9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 -- 172.21.15.66:0/3862955958 shutdown_connections 2024-03-30T00:46:41.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 -- 172.21.15.66:0/3862955958 wait complete. 2024-03-30T00:46:41.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 -- start start 2024-03-30T00:46:41.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.658+0000 7f7af64a3000 1 Processor -- start 2024-03-30T00:46:41.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec48592e50 0x55ec485ea370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec4863c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec4863c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec485f28c0 con 0x55ec485db820 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec483df3e0 con 0x55ec485dc140 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec485f7d50 con 0x55ec48592e50 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 10 client.0 _ll_register_callbacks cb 0x55ec483927a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ae3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec4863c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ae3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec4863c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50954/0 (socket says 172.21.15.66:50954) 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ae3400640 1 -- 172.21.15.66:0/4205105205 learned_addr learned my addr 172.21.15.66:0/4205105205 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af1000640 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec4863c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7aebe00640 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec48592e50 0x55ec485ea370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ae3400640 1 -- 172.21.15.66:0/4205105205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec48592e50 msgr2=0x55ec485ea370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ae3400640 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec48592e50 0x55ec485ea370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ae3400640 1 -- 172.21.15.66:0/4205105205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 msgr2=0x55ec4863c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ae3400640 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec4863c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ae3400640 1 -- 172.21.15.66:0/4205105205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec0b34c0 con 0x55ec485db820 2024-03-30T00:46:41.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ae3400640 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec4863c0f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7ad401ca10 tx=0x7f7ad4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbc3a9d3e69554c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ad9600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ad9600640 1 -- 172.21.15.66:0/4205105205 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40ce070 con 0x55ec485db820 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ad9600640 1 -- 172.21.15.66:0/4205105205 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f7ad40040d0 con 0x55ec485db820 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 -- 172.21.15.66:0/4205105205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec48595af0 con 0x55ec485db820 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ad9600640 1 -- 172.21.15.66:0/4205105205 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ad40c4040 con 0x55ec485db820 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 10 client.8034 fetch_fsmap learned FSMap version 64 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 -- 172.21.15.66:0/4205105205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55ec48380a80 con 0x55ec485db820 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ad9600640 1 -- 172.21.15.66:0/4205105205 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec48595af0 con 0x55ec485db820 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ad9600640 1 -- 172.21.15.66:0/4205105205 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 64) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7ad40d1070 con 0x55ec485db820 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 10 client.8034 fetch_fsmap finished waiting for FSMap version 64 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 10 client.8034 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 20 client.8034 populate_metadata read hostname 'smithi066' 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ec48640fb0 0x55ec486433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 4 client.8034 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 -- 172.21.15.66:0/4205105205 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x55ec483b1570 con 0x55ec48640fb0 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7aebe00640 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ec48640fb0 0x55ec486433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7aebe00640 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ec48640fb0 0x55ec486433a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48fd3e11aff83706 server_cookie=dacce8261a0a3f8e in_seq=0 out_seq=0 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ad9600640 10 client.8034 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ad9600640 1 -- 172.21.15.66:0/4205105205 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ec483b1570 con 0x55ec48640fb0 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ad9600640 10 client.8034 handle_command_reply: tid=0 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 client.8034 shutdown 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 -- 172.21.15.66:0/4205105205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 msgr2=0x55ec4863c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec4863c0f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7ad401ca10 tx=0x7f7ad4008040 comp rx=0 tx=0).stop 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 -- 172.21.15.66:0/4205105205 shutdown_connections 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ec48640fb0 0x55ec486433a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec485dc140 0x55ec4863c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec485db820 0x55ec4863c0f0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 --2- 172.21.15.66:0/4205105205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ec48592e50 0x55ec485ea370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 -- 172.21.15.66:0/4205105205 >> 172.21.15.66:0/4205105205 conn(0x55ec48391280 msgr2=0x55ec485d9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7ad9600640 0 client.8034 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 -- 172.21.15.66:0/4205105205 shutdown_connections 2024-03-30T00:46:41.676 INFO:tasks.ceph.mds.a.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f8a24800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-03-30T00:46:23.006182+0000 1=1+0) n(v0 rc2024-03-30T00:46:23.120156+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x55d4e38c3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-30T00:46:23.006182+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-30T00:46:23.120156+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-03-30T00:46:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.662+0000 7f7af64a3000 1 -- 172.21.15.66:0/4205105205 wait complete. 2024-03-30T00:46:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af64a3000 20 client.8034 trim_cache size 0 max 16384 2024-03-30T00:46:41.677 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:46:41.677 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T00:46:41.677 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "1b732c21-895d-4f66-b05e-194c5810942c", 2024-03-30T00:46:41.677 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T00:46:41.677 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:46:41.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 -- 172.21.15.66:0/4016187391 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ac80666c0 msgr2=0x7f7ac8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ac80666c0 0x7f7ac8068b80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40ae000 tx=0x7f7ad4004740 comp rx=0 tx=0).stop 2024-03-30T00:46:41.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 -- 172.21.15.66:0/4016187391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aec06e7f0 msgr2=0x7f7aec0b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aec06e7f0 0x7f7aec0b39e0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00c7f0 tx=0x7f7adc00ccb0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 -- 172.21.15.66:0/4016187391 shutdown_connections 2024-03-30T00:46:41.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7aec0cbd30 0x7f7aec0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7aec0cb650 0x7f7aec1587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ac80666c0 0x7f7ac8068b80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 --2- 172.21.15.66:0/4016187391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aec06e7f0 0x7f7aec0b39e0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 -- 172.21.15.66:0/4016187391 >> 172.21.15.66:0/4016187391 conn(0x7f7aec0b1030 msgr2=0x7f7aec0ad0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 -- 172.21.15.66:0/4016187391 shutdown_connections 2024-03-30T00:46:41.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.666+0000 7f7af3000640 1 -- 172.21.15.66:0/4016187391 wait complete. 2024-03-30T00:46:41.692 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-30T00:46:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.826+0000 7f561b200640 1 Processor -- start 2024-03-30T00:46:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.826+0000 7f561b200640 1 -- start start 2024-03-30T00:46:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.826+0000 7f561b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f561c0b3170 0x7f561c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.826+0000 7f561b200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f561c0d07c0 0x7f561c0d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.826+0000 7f561b200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561c0cb5b0 con 0x7f561c15a720 2024-03-30T00:46:41.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.826+0000 7f561b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561c0cfb00 con 0x7f561c0b3170 2024-03-30T00:46:41.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.826+0000 7f561b200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561c0d1170 con 0x7f561c0d07c0 2024-03-30T00:46:41.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.826+0000 7f5611400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f561c0b3170 0x7f561c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.826+0000 7f5611400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f561c0b3170 0x7f561c0b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50958/0 (socket says 172.21.15.66:50958) 2024-03-30T00:46:41.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.826+0000 7f5611400640 1 -- 172.21.15.66:0/929557090 learned_addr learned my addr 172.21.15.66:0/929557090 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.826+0000 7f561a800640 1 --2- 172.21.15.66:0/929557090 >> v2:172.21.15.156:3300/0 conn(0x7f561c0d07c0 0x7f561c0d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f5611400640 1 -- 172.21.15.66:0/929557090 >> v2:172.21.15.156:3300/0 conn(0x7f561c0d07c0 msgr2=0x7f561c0d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f5611400640 1 --2- 172.21.15.66:0/929557090 >> v2:172.21.15.156:3300/0 conn(0x7f561c0d07c0 0x7f561c0d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f5611400640 1 -- 172.21.15.66:0/929557090 >> v1:172.21.15.156:6789/0 conn(0x7f561c15a720 legacy=0x7f561c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f5611400640 1 -- 172.21.15.66:0/929557090 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c0cb080 con 0x7f561c0b3170 2024-03-30T00:46:41.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f5611400640 1 --2- 172.21.15.66:0/929557090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561c0b3170 0x7f561c0b3570 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f561c0ca900 tx=0x7f560401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6096c021117a80 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f5619400640 1 -- 172.21.15.66:0/929557090 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56040ac630 con 0x7f561c0b3170 2024-03-30T00:46:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f5619400640 1 -- 172.21.15.66:0/929557090 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56040ac790 con 0x7f561c0b3170 2024-03-30T00:46:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f5619400640 1 -- 172.21.15.66:0/929557090 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56040acb20 con 0x7f561c0b3170 2024-03-30T00:46:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f561b200640 1 -- 172.21.15.66:0/929557090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561c0b3170 msgr2=0x7f561c0b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f561b200640 1 --2- 172.21.15.66:0/929557090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561c0b3170 0x7f561c0b3570 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f561c0ca900 tx=0x7f560401c4e0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f561b200640 1 -- 172.21.15.66:0/929557090 shutdown_connections 2024-03-30T00:46:41.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f561b200640 1 --2- 172.21.15.66:0/929557090 >> v2:172.21.15.156:3300/0 conn(0x7f561c0d07c0 0x7f561c0d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f561b200640 1 --2- 172.21.15.66:0/929557090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561c0b3170 0x7f561c0b3570 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f561b200640 1 -- 172.21.15.66:0/929557090 >> 172.21.15.66:0/929557090 conn(0x7f561c069d90 msgr2=0x7f561c06a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f561b200640 1 -- 172.21.15.66:0/929557090 shutdown_connections 2024-03-30T00:46:41.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.830+0000 7f561b200640 1 -- 172.21.15.66:0/929557090 wait complete. 2024-03-30T00:46:41.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f561b200640 1 Processor -- start 2024-03-30T00:46:41.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f561b200640 1 -- start start 2024-03-30T00:46:41.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f561b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f561c0b3170 0x7f561c15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f561b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561c0d07c0 0x7f561c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f561b200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f561c15a720 0x7f561c150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f561b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c15d2a0 con 0x7f561c0d07c0 2024-03-30T00:46:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f561b200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0c91e0 con 0x7f561c15a720 2024-03-30T00:46:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f561b200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0c4fb0 con 0x7f561c0b3170 2024-03-30T00:46:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5611400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561c0d07c0 0x7f561c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5611400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561c0d07c0 0x7f561c15c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50968/0 (socket says 172.21.15.66:50968) 2024-03-30T00:46:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5611400640 1 -- 172.21.15.66:0/2478298502 learned_addr learned my addr 172.21.15.66:0/2478298502 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5611400640 1 -- 172.21.15.66:0/2478298502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f561c0b3170 msgr2=0x7f561c15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5611400640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f561c0b3170 0x7f561c15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5611400640 1 -- 172.21.15.66:0/2478298502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f561c15a720 msgr2=0x7f561c150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5611400640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f561c15a720 0x7f561c150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5611400640 1 -- 172.21.15.66:0/2478298502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56040094d0 con 0x7f561c0d07c0 2024-03-30T00:46:41.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5611400640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561c0d07c0 0x7f561c15c360 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f560401ab90 tx=0x7f560401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5612a00640 1 -- 172.21.15.66:0/2478298502 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56040b7830 con 0x7f561c0d07c0 2024-03-30T00:46:41.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5612a00640 1 -- 172.21.15.66:0/2478298502 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56040b7990 con 0x7f561c0d07c0 2024-03-30T00:46:41.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f5612a00640 1 -- 172.21.15.66:0/2478298502 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56040b7d20 con 0x7f561c0d07c0 2024-03-30T00:46:41.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f561b200640 1 -- 172.21.15.66:0/2478298502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561c0c9430 con 0x7f561c0d07c0 2024-03-30T00:46:41.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.834+0000 7f561b200640 1 -- 172.21.15.66:0/2478298502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561c0c99f0 con 0x7f561c0d07c0 2024-03-30T00:46:41.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.838+0000 7f5612a00640 1 -- 172.21.15.66:0/2478298502 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f56040b3040 con 0x7f561c0d07c0 2024-03-30T00:46:41.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.838+0000 7f5612a00640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55f8066690 0x7f55f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.838+0000 7f5612a00640 1 -- 172.21.15.66:0/2478298502 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f560411e8e0 con 0x7f561c0d07c0 2024-03-30T00:46:41.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.838+0000 7f5619e00640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55f8066690 0x7f55f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.838+0000 7f5623ef5000 1 Processor -- start 2024-03-30T00:46:41.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.838+0000 7f5623ef5000 1 -- start start 2024-03-30T00:46:41.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.838+0000 7f5619e00640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55f8066690 0x7f55f8068b50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f560c00ab90 tx=0x7f560c009230 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea69091e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690454c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69045a00 0x55ea69045de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68f10630 con 0x55ea69092390 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68e9dbd0 con 0x55ea69091a70 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea68e9b1a0 con 0x55ea69045a00 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5611400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5611400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690454c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50974/0 (socket says 172.21.15.66:50974) 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5611400640 1 -- 172.21.15.66:0/2536673534 learned_addr learned my addr 172.21.15.66:0/2536673534 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5619e00640 1 --2- 172.21.15.66:0/2536673534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea69091e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f561a800640 1 --2- 172.21.15.66:0/2536673534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69045a00 0x55ea69045de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5611400640 1 -- 172.21.15.66:0/2536673534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69045a00 msgr2=0x55ea69045de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5611400640 1 --2- 172.21.15.66:0/2536673534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69045a00 0x55ea69045de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5611400640 1 -- 172.21.15.66:0/2536673534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 msgr2=0x55ea69091e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5611400640 1 --2- 172.21.15.66:0/2536673534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea69091e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5611400640 1 -- 172.21.15.66:0/2536673534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea68b92e20 con 0x55ea69092390 2024-03-30T00:46:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f561a800640 1 --2- 172.21.15.66:0/2536673534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69045a00 0x55ea69045de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:46:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5611400640 1 --2- 172.21.15.66:0/2536673534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690454c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f56040ef600 tx=0x7f5604019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858147e8bf2d1731 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f560aa00640 1 -- 172.21.15.66:0/2536673534 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56040f7a60 con 0x55ea69092390 2024-03-30T00:46:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f560aa00640 1 -- 172.21.15.66:0/2536673534 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56040eda10 con 0x55ea69092390 2024-03-30T00:46:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f560aa00640 1 -- 172.21.15.66:0/2536673534 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56040ebd80 con 0x55ea69092390 2024-03-30T00:46:41.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- 172.21.15.66:0/2536673534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 msgr2=0x55ea690454c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 --2- 172.21.15.66:0/2536673534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690454c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f56040ef600 tx=0x7f5604019280 comp rx=0 tx=0).stop 2024-03-30T00:46:41.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- 172.21.15.66:0/2536673534 shutdown_connections 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 --2- 172.21.15.66:0/2536673534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69045a00 0x55ea69045de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 --2- 172.21.15.66:0/2536673534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690454c0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 --2- 172.21.15.66:0/2536673534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea69091e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- 172.21.15.66:0/2536673534 >> 172.21.15.66:0/2536673534 conn(0x55ea68e46160 msgr2=0x55ea69074820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- 172.21.15.66:0/2536673534 shutdown_connections 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- 172.21.15.66:0/2536673534 wait complete. 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- start start 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 Processor -- start 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69045a00 0x55ea6904e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea6904e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea69079e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68e96a90 con 0x55ea69092390 2024-03-30T00:46:41.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68e35d40 con 0x55ea69091a70 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea68e53f30 con 0x55ea69045a00 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5623ef5000 10 client.0 _ll_register_callbacks cb 0x55ea68ef8810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f561a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea69079e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f561a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea69079e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50978/0 (socket says 172.21.15.66:50978) 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f561a800640 1 -- 172.21.15.66:0/3114866899 learned_addr learned my addr 172.21.15.66:0/3114866899 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5619e00640 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69045a00 0x55ea6904e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.842+0000 7f5611400640 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea6904e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f561a800640 1 -- 172.21.15.66:0/3114866899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69045a00 msgr2=0x55ea6904e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f561a800640 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69045a00 0x55ea6904e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f561a800640 1 -- 172.21.15.66:0/3114866899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 msgr2=0x55ea6904e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f561a800640 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea6904e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f561a800640 1 -- 172.21.15.66:0/3114866899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561c0c99f0 con 0x55ea69092390 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f561a800640 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea69079e60 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f561400bf10 tx=0x7f561400f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32819e6aed894309 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f55ff400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:46:41.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f55ff400640 1 -- 172.21.15.66:0/3114866899 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614014660 con 0x55ea69092390 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f55ff400640 1 -- 172.21.15.66:0/3114866899 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f56140147c0 con 0x55ea69092390 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 -- 172.21.15.66:0/3114866899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea68e953e0 con 0x55ea69092390 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f55ff400640 1 -- 172.21.15.66:0/3114866899 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5614014980 con 0x55ea69092390 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 10 client.8046 fetch_fsmap learned FSMap version 64 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 -- 172.21.15.66:0/3114866899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55ea68e5baa0 con 0x55ea69092390 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f55ff400640 1 -- 172.21.15.66:0/3114866899 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea68e953e0 con 0x55ea69092390 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f55ff400640 1 -- 172.21.15.66:0/3114866899 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 64) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5614014ae0 con 0x55ea69092390 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 10 client.8046 fetch_fsmap finished waiting for FSMap version 64 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 10 client.8046 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 20 client.8046 populate_metadata read hostname 'smithi066' 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ea6907a870 0x55ea690a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 4 client.8046 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 -- 172.21.15.66:0/3114866899 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea68e66d00 con 0x55ea6907a870 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5619e00640 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ea6907a870 0x55ea690a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5619e00640 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ea6907a870 0x55ea690a7b60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4363a7e3f9b97fa1 server_cookie=501a2d690ccf341d in_seq=0 out_seq=0 2024-03-30T00:46:41.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f55ff400640 10 client.8046 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f55ff400640 1 -- 172.21.15.66:0/3114866899 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ea68e66d00 con 0x55ea6907a870 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f55ff400640 10 client.8046 handle_command_reply: tid=0 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 client.8046 shutdown 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 -- 172.21.15.66:0/3114866899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 msgr2=0x55ea69079e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea69079e60 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f561400bf10 tx=0x7f561400f5b0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 -- 172.21.15.66:0/3114866899 shutdown_connections 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ea6907a870 0x55ea690a7b60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea69079e60 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea6904e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3114866899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69045a00 0x55ea6904e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f55ff400640 0 client.8046 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 -- 172.21.15.66:0/3114866899 >> 172.21.15.66:0/3114866899 conn(0x55ea68e46160 msgr2=0x55ea69044bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 -- 172.21.15.66:0/3114866899 shutdown_connections 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 1 -- 172.21.15.66:0/3114866899 wait complete. 2024-03-30T00:46:41.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.846+0000 7f5623ef5000 20 client.8046 trim_cache size 0 max 16384 2024-03-30T00:46:41.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5623ef5000 1 Processor -- start 2024-03-30T00:46:41.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5623ef5000 1 -- start start 2024-03-30T00:46:41.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69045a00 0x55ea69045050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69091a70 0x55ea690744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69092390 0x55ea690749f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5623ef5000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68b53aa0 con 0x55ea69045a00 2024-03-30T00:46:41.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5623ef5000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68e57c60 con 0x55ea69092390 2024-03-30T00:46:41.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5623ef5000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea6904b2e0 con 0x55ea69091a70 2024-03-30T00:46:41.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5619e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69045a00 0x55ea69045050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5619e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69045a00 0x55ea69045050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50980/0 (socket says 172.21.15.66:50980) 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5619e00640 1 -- 172.21.15.66:0/190135224 learned_addr learned my addr 172.21.15.66:0/190135224 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5611400640 1 --2- 172.21.15.66:0/190135224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69091a70 0x55ea690744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f561a800640 1 --2- 172.21.15.66:0/190135224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69092390 0x55ea690749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5619e00640 1 -- 172.21.15.66:0/190135224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69091a70 msgr2=0x55ea690744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5619e00640 1 --2- 172.21.15.66:0/190135224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69091a70 0x55ea690744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5619e00640 1 -- 172.21.15.66:0/190135224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69092390 msgr2=0x55ea690749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5619e00640 1 --2- 172.21.15.66:0/190135224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69092390 0x55ea690749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5619e00640 1 -- 172.21.15.66:0/190135224 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea68e41df0 con 0x55ea69045a00 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f561a800640 1 --2- 172.21.15.66:0/190135224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69092390 0x55ea690749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f5619e00640 1 --2- 172.21.15.66:0/190135224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69045a00 0x55ea69045050 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f560c0189f0 tx=0x7f560c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fa7041ffd7ed01 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f560b400640 1 -- 172.21.15.66:0/190135224 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c01f040 con 0x55ea69045a00 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f560b400640 1 -- 172.21.15.66:0/190135224 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560c0245b0 con 0x55ea69045a00 2024-03-30T00:46:41.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.862+0000 7f560b400640 1 -- 172.21.15.66:0/190135224 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c019660 con 0x55ea69045a00 2024-03-30T00:46:41.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- 172.21.15.66:0/190135224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69045a00 msgr2=0x55ea69045050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 --2- 172.21.15.66:0/190135224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69045a00 0x55ea69045050 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f560c0189f0 tx=0x7f560c00f000 comp rx=0 tx=0).stop 2024-03-30T00:46:41.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- 172.21.15.66:0/190135224 shutdown_connections 2024-03-30T00:46:41.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 --2- 172.21.15.66:0/190135224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69092390 0x55ea690749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 --2- 172.21.15.66:0/190135224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea69091a70 0x55ea690744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 --2- 172.21.15.66:0/190135224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69045a00 0x55ea69045050 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- 172.21.15.66:0/190135224 >> 172.21.15.66:0/190135224 conn(0x55ea68e46160 msgr2=0x55ea69073da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- 172.21.15.66:0/190135224 shutdown_connections 2024-03-30T00:46:41.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- 172.21.15.66:0/190135224 wait complete. 2024-03-30T00:46:41.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- start start 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 Processor -- start 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea690f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690f6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea690a30c0 0x55ea690fabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68f67470 con 0x55ea69092390 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68eda170 con 0x55ea69091a70 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea68e660b0 con 0x55ea690a30c0 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 10 client.0 _ll_register_callbacks cb 0x7f56040ef440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5611400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5611400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690f6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50996/0 (socket says 172.21.15.66:50996) 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5611400640 1 -- 172.21.15.66:0/3082797590 learned_addr learned my addr 172.21.15.66:0/3082797590 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5619e00640 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea690f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f561a800640 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea690a30c0 0x55ea690fabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5611400640 1 -- 172.21.15.66:0/3082797590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea690a30c0 msgr2=0x55ea690fabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5611400640 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea690a30c0 0x55ea690fabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5611400640 1 -- 172.21.15.66:0/3082797590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 msgr2=0x55ea690f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5611400640 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea690f5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5611400640 1 -- 172.21.15.66:0/3082797590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c012070 con 0x55ea69092390 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5611400640 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690f6250 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f560401c9c0 tx=0x7f560411d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f1e30255195124 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f55ff400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f55ff400640 1 -- 172.21.15.66:0/3082797590 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56040eb9b0 con 0x55ea69092390 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f55ff400640 1 -- 172.21.15.66:0/3082797590 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f56040f7a60 con 0x55ea69092390 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- 172.21.15.66:0/3082797590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea68ec5a20 con 0x55ea69092390 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f55ff400640 1 -- 172.21.15.66:0/3082797590 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56040eeb60 con 0x55ea69092390 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 10 client.8052 fetch_fsmap learned FSMap version 64 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- 172.21.15.66:0/3082797590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55ea68e35960 con 0x55ea69092390 2024-03-30T00:46:41.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f55ff400640 1 -- 172.21.15.66:0/3082797590 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea68ec5a20 con 0x55ea69092390 2024-03-30T00:46:41.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f55ff400640 1 -- 172.21.15.66:0/3082797590 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 64) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f56040f75d0 con 0x55ea69092390 2024-03-30T00:46:41.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 10 client.8052 fetch_fsmap finished waiting for FSMap version 64 2024-03-30T00:46:41.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 10 client.8052 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:46:41.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 20 client.8052 populate_metadata read hostname 'smithi066' 2024-03-30T00:46:41.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ea690fb110 0x55ea690fd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:46:41.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 4 client.8052 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:46:41.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.866+0000 7f5623ef5000 1 -- 172.21.15.66:0/3082797590 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ea68e8f550 con 0x55ea690fb110 2024-03-30T00:46:41.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5619e00640 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ea690fb110 0x55ea690fd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:46:41.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5619e00640 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ea690fb110 0x55ea690fd500 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df86820e718b596 server_cookie=10b7f75af8c1641b in_seq=0 out_seq=0 2024-03-30T00:46:41.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f55ff400640 10 client.8052 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:41.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f55ff400640 1 -- 172.21.15.66:0/3082797590 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x55ea68e8f550 con 0x55ea690fb110 2024-03-30T00:46:41.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f55ff400640 10 client.8052 handle_command_reply: tid=0 2024-03-30T00:46:41.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 1 client.8052 shutdown 2024-03-30T00:46:41.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 1 -- 172.21.15.66:0/3082797590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 msgr2=0x55ea690f6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690f6250 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f560401c9c0 tx=0x7f560411d000 comp rx=0 tx=0).stop 2024-03-30T00:46:41.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 1 -- 172.21.15.66:0/3082797590 shutdown_connections 2024-03-30T00:46:41.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55ea690fb110 0x55ea690fd500 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea690a30c0 0x55ea690fabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea69092390 0x55ea690f6250 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 1 --2- 172.21.15.66:0/3082797590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea69091a70 0x55ea690f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f55ff400640 0 client.8052 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:46:41.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 1 -- 172.21.15.66:0/3082797590 >> 172.21.15.66:0/3082797590 conn(0x55ea68e46160 msgr2=0x55ea690744a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 1 -- 172.21.15.66:0/3082797590 shutdown_connections 2024-03-30T00:46:41.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 1 -- 172.21.15.66:0/3082797590 wait complete. 2024-03-30T00:46:41.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f5623ef5000 20 client.8052 trim_cache size 0 max 16384 2024-03-30T00:46:41.883 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:46:41.883 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:46:41.883 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-03-30T00:46:41.883 INFO:teuthology.orchestra.run.smithi066.stdout: "1b732c21-895d-4f66-b05e-194c5810942c": { 2024-03-30T00:46:41.883 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/dir", 2024-03-30T00:46:41.883 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "1b732c21-895d-4f66-b05e-194c5810942c", 2024-03-30T00:46:41.883 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive" 2024-03-30T00:46:41.883 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:41.883 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:46:41.883 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:46:41.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 -- 172.21.15.66:0/2478298502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55f8066690 msgr2=0x7f55f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55f8066690 0x7f55f8068b50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f560c00ab90 tx=0x7f560c009230 comp rx=0 tx=0).stop 2024-03-30T00:46:41.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 -- 172.21.15.66:0/2478298502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561c0d07c0 msgr2=0x7f561c15c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:46:41.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561c0d07c0 0x7f561c15c360 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f560401ab90 tx=0x7f560401ad20 comp rx=0 tx=0).stop 2024-03-30T00:46:41.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 -- 172.21.15.66:0/2478298502 shutdown_connections 2024-03-30T00:46:41.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f561c15a720 0x7f561c150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55f8066690 0x7f55f8068b50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f561c0d07c0 0x7f561c15c360 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 --2- 172.21.15.66:0/2478298502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f561c0b3170 0x7f561c15cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:46:41.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 -- 172.21.15.66:0/2478298502 >> 172.21.15.66:0/2478298502 conn(0x7f561c069d90 msgr2=0x7f561c0ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:46:41.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 -- 172.21.15.66:0/2478298502 shutdown_connections 2024-03-30T00:46:41.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:46:41.870+0000 7f561b200640 1 -- 172.21.15.66:0/2478298502 wait complete. 2024-03-30T00:46:41.897 INFO:tasks.cephfs.filesystem:scrub status for tag:1b732c21-895d-4f66-b05e-194c5810942c - {'path': '/dir', 'tag': '1b732c21-895d-4f66-b05e-194c5810942c', 'options': 'recursive'} 2024-03-30T00:46:45.569 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:46:45.562+0000 7f1c7fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-30T00:46:46.921 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:46:46.925 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:47:11.899 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-30T00:47:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.042+0000 7f0b90a00640 1 Processor -- start 2024-03-30T00:47:12.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b90a00640 1 -- start start 2024-03-30T00:47:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b90a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b8c0acae0 0x7f0b8c0ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b90a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0b8c06d7e0 0x7f0b8c0ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b90a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b8c06beb0 0x7f0b8c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b90a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8c0acec0 con 0x7f0b8c06beb0 2024-03-30T00:47:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b90a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8c15de20 con 0x7f0b8c06d7e0 2024-03-30T00:47:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b90a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c15d060 con 0x7f0b8c0acae0 2024-03-30T00:47:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b8be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b8c06beb0 0x7f0b8c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b8be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b8c06beb0 0x7f0b8c06c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53884/0 (socket says 172.21.15.66:53884) 2024-03-30T00:47:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b8be00640 1 -- 172.21.15.66:0/1927910052 learned_addr learned my addr 172.21.15.66:0/1927910052 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b8b400640 1 --2- 172.21.15.66:0/1927910052 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b8c0acae0 0x7f0b8c0ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b82a00640 1 --2- 172.21.15.66:0/1927910052 >> v2:172.21.15.156:3300/0 conn(0x7f0b8c06d7e0 0x7f0b8c0ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b8be00640 1 -- 172.21.15.66:0/1927910052 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b8c0acae0 msgr2=0x7f0b8c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b8be00640 1 --2- 172.21.15.66:0/1927910052 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b8c0acae0 0x7f0b8c0ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b8be00640 1 -- 172.21.15.66:0/1927910052 >> v2:172.21.15.156:3300/0 conn(0x7f0b8c06d7e0 msgr2=0x7f0b8c0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b8be00640 1 --2- 172.21.15.66:0/1927910052 >> v2:172.21.15.156:3300/0 conn(0x7f0b8c06d7e0 0x7f0b8c0ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.046+0000 7f0b8be00640 1 -- 172.21.15.66:0/1927910052 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c15d340 con 0x7f0b8c06beb0 2024-03-30T00:47:12.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b8be00640 1 --2- 172.21.15.66:0/1927910052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c06beb0 0x7f0b8c06c340 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0b42e0 tx=0x7f0b8401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25eb028a383c0415 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b8aa00640 1 -- 172.21.15.66:0/1927910052 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840ac410 con 0x7f0b8c06beb0 2024-03-30T00:47:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b8aa00640 1 -- 172.21.15.66:0/1927910052 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b840ac570 con 0x7f0b8c06beb0 2024-03-30T00:47:12.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b8aa00640 1 -- 172.21.15.66:0/1927910052 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840ac900 con 0x7f0b8c06beb0 2024-03-30T00:47:12.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b90a00640 1 -- 172.21.15.66:0/1927910052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c06beb0 msgr2=0x7f0b8c06c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b90a00640 1 --2- 172.21.15.66:0/1927910052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c06beb0 0x7f0b8c06c340 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0b42e0 tx=0x7f0b8401c4f0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b90a00640 1 -- 172.21.15.66:0/1927910052 shutdown_connections 2024-03-30T00:47:12.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b90a00640 1 --2- 172.21.15.66:0/1927910052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c06beb0 0x7f0b8c06c340 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b90a00640 1 --2- 172.21.15.66:0/1927910052 >> v2:172.21.15.156:3300/0 conn(0x7f0b8c06d7e0 0x7f0b8c0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b90a00640 1 --2- 172.21.15.66:0/1927910052 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b8c0acae0 0x7f0b8c0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b90a00640 1 -- 172.21.15.66:0/1927910052 >> 172.21.15.66:0/1927910052 conn(0x7f0b8c06a190 msgr2=0x7f0b8c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b90a00640 1 -- 172.21.15.66:0/1927910052 shutdown_connections 2024-03-30T00:47:12.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b90a00640 1 -- 172.21.15.66:0/1927910052 wait complete. 2024-03-30T00:47:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b90a00640 1 Processor -- start 2024-03-30T00:47:12.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.050+0000 7f0b90a00640 1 -- start start 2024-03-30T00:47:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b90a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b8c06beb0 0x7f0b8c1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b90a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b8c06d7e0 0x7f0b8c158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b90a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0acae0 0x7f0b8c155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b90a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c144e90 con 0x7f0b8c0acae0 2024-03-30T00:47:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b90a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0c25b0 con 0x7f0b8c06beb0 2024-03-30T00:47:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b90a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c144860 con 0x7f0b8c06d7e0 2024-03-30T00:47:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b8be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0acae0 0x7f0b8c155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b8be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0acae0 0x7f0b8c155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53892/0 (socket says 172.21.15.66:53892) 2024-03-30T00:47:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b8be00640 1 -- 172.21.15.66:0/1487872707 learned_addr learned my addr 172.21.15.66:0/1487872707 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b8b400640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b8c06beb0 0x7f0b8c1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b82a00640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b8c06d7e0 0x7f0b8c158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b8be00640 1 -- 172.21.15.66:0/1487872707 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b8c06d7e0 msgr2=0x7f0b8c158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b8be00640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b8c06d7e0 0x7f0b8c158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b8be00640 1 -- 172.21.15.66:0/1487872707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b8c06beb0 msgr2=0x7f0b8c1589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b8be00640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b8c06beb0 0x7f0b8c1589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b8be00640 1 -- 172.21.15.66:0/1487872707 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b840094d0 con 0x7f0b8c0acae0 2024-03-30T00:47:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b8be00640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0acae0 0x7f0b8c155f30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0b8401cd90 tx=0x7f0b84019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b83e00640 1 -- 172.21.15.66:0/1487872707 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840ac410 con 0x7f0b8c0acae0 2024-03-30T00:47:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b83e00640 1 -- 172.21.15.66:0/1487872707 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b840b45c0 con 0x7f0b8c0acae0 2024-03-30T00:47:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b83e00640 1 -- 172.21.15.66:0/1487872707 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840b4970 con 0x7f0b8c0acae0 2024-03-30T00:47:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b90a00640 1 -- 172.21.15.66:0/1487872707 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8c1f8d90 con 0x7f0b8c0acae0 2024-03-30T00:47:12.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.054+0000 7f0b90a00640 1 -- 172.21.15.66:0/1487872707 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c156470 con 0x7f0b8c0acae0 2024-03-30T00:47:12.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b83e00640 1 -- 172.21.15.66:0/1487872707 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0b840b4ad0 con 0x7f0b8c0acae0 2024-03-30T00:47:12.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b83e00640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b68066690 0x7f0b68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b83e00640 1 -- 172.21.15.66:0/1487872707 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f0b8411e950 con 0x7f0b8c0acae0 2024-03-30T00:47:12.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b9555e000 1 Processor -- start 2024-03-30T00:47:12.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b8b400640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b68066690 0x7f0b68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b9555e000 1 -- start start 2024-03-30T00:47:12.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b8b400640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b68066690 0x7f0b68068b50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009ac0 tx=0x7f0b7c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a98fa70 0x55688a98fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9434c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a943a00 0x55688a943de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b9555e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688a80e630 con 0x55688a990390 2024-03-30T00:47:12.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b9555e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688a79bbd0 con 0x55688a943a00 2024-03-30T00:47:12.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b9555e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55688a7991a0 con 0x55688a98fa70 2024-03-30T00:47:12.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b82a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9434c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b82a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9434c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53898/0 (socket says 172.21.15.66:53898) 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b82a00640 1 -- 172.21.15.66:0/3257459607 learned_addr learned my addr 172.21.15.66:0/3257459607 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.058+0000 7f0b8b400640 1 --2- 172.21.15.66:0/3257459607 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a98fa70 0x55688a98fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b8be00640 1 --2- 172.21.15.66:0/3257459607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a943a00 0x55688a943de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b82a00640 1 -- 172.21.15.66:0/3257459607 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a98fa70 msgr2=0x55688a98fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b82a00640 1 --2- 172.21.15.66:0/3257459607 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a98fa70 0x55688a98fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b82a00640 1 -- 172.21.15.66:0/3257459607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a943a00 msgr2=0x55688a943de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b82a00640 1 --2- 172.21.15.66:0/3257459607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a943a00 0x55688a943de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b82a00640 1 -- 172.21.15.66:0/3257459607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55688a490e20 con 0x55688a990390 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b8be00640 1 --2- 172.21.15.66:0/3257459607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a943a00 0x55688a943de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b82a00640 1 --2- 172.21.15.66:0/3257459607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9434c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f0b740114d0 tx=0x7f0b7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416b5bca59023017 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b7be00640 1 -- 172.21.15.66:0/3257459607 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74010040 con 0x55688a990390 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b7be00640 1 -- 172.21.15.66:0/3257459607 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b74016920 con 0x55688a990390 2024-03-30T00:47:12.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b7be00640 1 -- 172.21.15.66:0/3257459607 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74016cb0 con 0x55688a990390 2024-03-30T00:47:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 -- 172.21.15.66:0/3257459607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 msgr2=0x55688a9434c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3257459607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9434c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f0b740114d0 tx=0x7f0b7400b040 comp rx=0 tx=0).stop 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 -- 172.21.15.66:0/3257459607 shutdown_connections 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3257459607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a943a00 0x55688a943de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3257459607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9434c0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3257459607 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a98fa70 0x55688a98fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 -- 172.21.15.66:0/3257459607 >> 172.21.15.66:0/3257459607 conn(0x55688a744160 msgr2=0x55688a972820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 -- 172.21.15.66:0/3257459607 shutdown_connections 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 -- 172.21.15.66:0/3257459607 wait complete. 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 -- start start 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 Processor -- start 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a94c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a98fa70 0x55688a973a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a990390 0x55688a973f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688a794a90 con 0x55688a943a00 2024-03-30T00:47:12.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688a733d40 con 0x55688a990390 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55688a751f30 con 0x55688a98fa70 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b9555e000 10 client.0 _ll_register_callbacks cb 0x55688a7f6810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b8b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a94c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b8b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a94c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53902/0 (socket says 172.21.15.66:53902) 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b8b400640 1 -- 172.21.15.66:0/2587328536 learned_addr learned my addr 172.21.15.66:0/2587328536 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b8be00640 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a990390 0x55688a973f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b82a00640 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a98fa70 0x55688a973a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b8b400640 1 -- 172.21.15.66:0/2587328536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a98fa70 msgr2=0x55688a973a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b8b400640 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a98fa70 0x55688a973a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b8b400640 1 -- 172.21.15.66:0/2587328536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a990390 msgr2=0x55688a973f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.062+0000 7f0b8b400640 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a990390 0x55688a973f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b8b400640 1 -- 172.21.15.66:0/2587328536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b740111b0 con 0x55688a943a00 2024-03-30T00:47:12.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b82a00640 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a98fa70 0x55688a973a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:12.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b8b400640 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a94c810 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0045f0 tx=0x7f0b7c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8879724e189b4c14 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b78c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:12.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b78c00640 1 -- 172.21.15.66:0/2587328536 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c01c040 con 0x55688a943a00 2024-03-30T00:47:12.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b78c00640 1 -- 172.21.15.66:0/2587328536 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f0b7c023450 con 0x55688a943a00 2024-03-30T00:47:12.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 -- 172.21.15.66:0/2587328536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55688a7933e0 con 0x55688a943a00 2024-03-30T00:47:12.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b78c00640 1 -- 172.21.15.66:0/2587328536 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b7c0040f0 con 0x55688a943a00 2024-03-30T00:47:12.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 10 client.8064 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:12.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 -- 172.21.15.66:0/2587328536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55688a759aa0 con 0x55688a943a00 2024-03-30T00:47:12.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b78c00640 1 -- 172.21.15.66:0/2587328536 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55688a7933e0 con 0x55688a943a00 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b78c00640 1 -- 172.21.15.66:0/2587328536 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0b7c02c3f0 con 0x55688a943a00 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 10 client.8064 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 10 client.8064 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 20 client.8064 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55688a9972d0 0x55688a9976b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 4 client.8064 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 -- 172.21.15.66:0/2587328536 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55688a764d00 con 0x55688a9972d0 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b82a00640 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55688a9972d0 0x55688a9976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b78c00640 10 client.8064 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b82a00640 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55688a9972d0 0x55688a9976b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f044926a8e87d8 server_cookie=f52c9da3b3944b08 in_seq=0 out_seq=0 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b78c00640 1 -- 172.21.15.66:0/2587328536 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55688a764d00 con 0x55688a9972d0 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b78c00640 10 client.8064 handle_command_reply: tid=0 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 client.8064 shutdown 2024-03-30T00:47:12.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 -- 172.21.15.66:0/2587328536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 msgr2=0x55688a94c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a94c810 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0045f0 tx=0x7f0b7c0046d0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 -- 172.21.15.66:0/2587328536 shutdown_connections 2024-03-30T00:47:12.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55688a9972d0 0x55688a9976b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a990390 0x55688a973f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a98fa70 0x55688a973a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 --2- 172.21.15.66:0/2587328536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a94c810 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b78c00640 0 client.8064 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:12.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 -- 172.21.15.66:0/2587328536 >> 172.21.15.66:0/2587328536 conn(0x55688a744160 msgr2=0x55688a942bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 -- 172.21.15.66:0/2587328536 shutdown_connections 2024-03-30T00:47:12.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 1 -- 172.21.15.66:0/2587328536 wait complete. 2024-03-30T00:47:12.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.066+0000 7f0b9555e000 20 client.8064 trim_cache size 0 max 16384 2024-03-30T00:47:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b9555e000 1 Processor -- start 2024-03-30T00:47:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b9555e000 1 -- start start 2024-03-30T00:47:12.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a942c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a98fa70 0x55688a943170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a990390 0x55688a943de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b9555e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688a451aa0 con 0x55688a943a00 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b9555e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688a93d930 con 0x55688a98fa70 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b9555e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55688a791e90 con 0x55688a990390 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b8b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a942c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b8b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a942c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53908/0 (socket says 172.21.15.66:53908) 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b8b400640 1 -- 172.21.15.66:0/3821331989 learned_addr learned my addr 172.21.15.66:0/3821331989 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b8be00640 1 --2- 172.21.15.66:0/3821331989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a990390 0x55688a943de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b82a00640 1 --2- 172.21.15.66:0/3821331989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a98fa70 0x55688a943170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b8b400640 1 -- 172.21.15.66:0/3821331989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a990390 msgr2=0x55688a943de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b8b400640 1 --2- 172.21.15.66:0/3821331989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a990390 0x55688a943de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b8b400640 1 -- 172.21.15.66:0/3821331989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a98fa70 msgr2=0x55688a943170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b8b400640 1 --2- 172.21.15.66:0/3821331989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a98fa70 0x55688a943170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b8b400640 1 -- 172.21.15.66:0/3821331989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c0b00e0 con 0x55688a943a00 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b8be00640 1 --2- 172.21.15.66:0/3821331989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a990390 0x55688a943de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:12.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b82a00640 1 --2- 172.21.15.66:0/3821331989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a98fa70 0x55688a943170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.082+0000 7f0b8b400640 1 --2- 172.21.15.66:0/3821331989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a942c30 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c011d60 tx=0x7f0b7c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee119712d1682ad4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b80c00640 1 -- 172.21.15.66:0/3821331989 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c003ab0 con 0x55688a943a00 2024-03-30T00:47:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b80c00640 1 -- 172.21.15.66:0/3821331989 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c01c040 con 0x55688a943a00 2024-03-30T00:47:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b80c00640 1 -- 172.21.15.66:0/3821331989 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c02c950 con 0x55688a943a00 2024-03-30T00:47:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 -- 172.21.15.66:0/3821331989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 msgr2=0x55688a942c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3821331989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a942c30 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c011d60 tx=0x7f0b7c01d000 comp rx=0 tx=0).stop 2024-03-30T00:47:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 -- 172.21.15.66:0/3821331989 shutdown_connections 2024-03-30T00:47:12.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3821331989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a990390 0x55688a943de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3821331989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a98fa70 0x55688a943170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3821331989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a943a00 0x55688a942c30 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 -- 172.21.15.66:0/3821331989 >> 172.21.15.66:0/3821331989 conn(0x55688a744160 msgr2=0x55688a94b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 -- 172.21.15.66:0/3821331989 shutdown_connections 2024-03-30T00:47:12.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 -- 172.21.15.66:0/3821331989 wait complete. 2024-03-30T00:47:12.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 -- start start 2024-03-30T00:47:12.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 Processor -- start 2024-03-30T00:47:12.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a943a00 0x55688a9473f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a98fa70 0x55688a9ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688a7d8170 con 0x55688a990390 2024-03-30T00:47:12.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688a793920 con 0x55688a98fa70 2024-03-30T00:47:12.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55688a7922a0 con 0x55688a943a00 2024-03-30T00:47:12.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 10 client.0 _ll_register_callbacks cb 0x7f0b74009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:12.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b8be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b8be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9ab720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53910/0 (socket says 172.21.15.66:53910) 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b8be00640 1 -- 172.21.15.66:0/3163876739 learned_addr learned my addr 172.21.15.66:0/3163876739 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b82a00640 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a98fa70 0x55688a9ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b8b400640 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a943a00 0x55688a9473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b8be00640 1 -- 172.21.15.66:0/3163876739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a943a00 msgr2=0x55688a9473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b8be00640 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a943a00 0x55688a9473f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b8be00640 1 -- 172.21.15.66:0/3163876739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a98fa70 msgr2=0x55688a9ab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b8be00640 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a98fa70 0x55688a9ab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b8be00640 1 -- 172.21.15.66:0/3163876739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55688a759aa0 con 0x55688a990390 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b8be00640 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9ab720 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0b840ebdb0 tx=0x7f0b840f3770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5fa393b89189 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b78c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b78c00640 1 -- 172.21.15.66:0/3163876739 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840f7760 con 0x55688a990390 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b78c00640 1 -- 172.21.15.66:0/3163876739 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f0b840f78c0 con 0x55688a990390 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.086+0000 7f0b9555e000 1 -- 172.21.15.66:0/3163876739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55688a751970 con 0x55688a990390 2024-03-30T00:47:12.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b78c00640 1 -- 172.21.15.66:0/3163876739 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b840f3150 con 0x55688a990390 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 10 client.8070 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 -- 172.21.15.66:0/3163876739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55688a73fdf0 con 0x55688a990390 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b78c00640 1 -- 172.21.15.66:0/3163876739 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55688a751970 con 0x55688a990390 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b78c00640 1 -- 172.21.15.66:0/3163876739 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0b840b3030 con 0x55688a990390 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 10 client.8070 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 10 client.8070 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 20 client.8070 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55688a9abc60 0x55688a9eff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 4 client.8070 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 -- 172.21.15.66:0/3163876739 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55688a7c3a20 con 0x55688a9abc60 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b8b400640 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55688a9abc60 0x55688a9eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b8b400640 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55688a9abc60 0x55688a9eff10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d3d9b74da6e118 server_cookie=ca2b4dc49953682a in_seq=0 out_seq=0 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b78c00640 10 client.8070 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:12.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b78c00640 1 -- 172.21.15.66:0/3163876739 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55688a7c3a20 con 0x55688a9abc60 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b78c00640 10 client.8070 handle_command_reply: tid=0 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 client.8070 shutdown 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 -- 172.21.15.66:0/3163876739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 msgr2=0x55688a9ab720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9ab720 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0b840ebdb0 tx=0x7f0b840f3770 comp rx=0 tx=0).stop 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 -- 172.21.15.66:0/3163876739 shutdown_connections 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55688a9abc60 0x55688a9eff10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688a990390 0x55688a9ab720 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55688a98fa70 0x55688a9ab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 --2- 172.21.15.66:0/3163876739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55688a943a00 0x55688a9473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b78c00640 0 client.8070 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 -- 172.21.15.66:0/3163876739 >> 172.21.15.66:0/3163876739 conn(0x55688a744160 msgr2=0x55688a941c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 -- 172.21.15.66:0/3163876739 shutdown_connections 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 1 -- 172.21.15.66:0/3163876739 wait complete. 2024-03-30T00:47:12.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b9555e000 20 client.8070 trim_cache size 0 max 16384 2024-03-30T00:47:12.105 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:47:12.105 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T00:47:12.105 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-03-30T00:47:12.105 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:47:12.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 -- 172.21.15.66:0/1487872707 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b68066690 msgr2=0x7f0b68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b68066690 0x7f0b68068b50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009ac0 tx=0x7f0b7c009190 comp rx=0 tx=0).stop 2024-03-30T00:47:12.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 -- 172.21.15.66:0/1487872707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0acae0 msgr2=0x7f0b8c155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0acae0 0x7f0b8c155f30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0b8401cd90 tx=0x7f0b84019290 comp rx=0 tx=0).stop 2024-03-30T00:47:12.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 -- 172.21.15.66:0/1487872707 shutdown_connections 2024-03-30T00:47:12.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0b68066690 0x7f0b68068b50 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0acae0 0x7f0b8c155f30 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b8c06d7e0 0x7f0b8c158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 --2- 172.21.15.66:0/1487872707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b8c06beb0 0x7f0b8c1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 -- 172.21.15.66:0/1487872707 >> 172.21.15.66:0/1487872707 conn(0x7f0b8c06a190 msgr2=0x7f0b8c15bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 -- 172.21.15.66:0/1487872707 shutdown_connections 2024-03-30T00:47:12.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.090+0000 7f0b90a00640 1 -- 172.21.15.66:0/1487872707 wait complete. 2024-03-30T00:47:12.118 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:47:12.118 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:47:12.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.250+0000 7fcd6e600640 1 Processor -- start 2024-03-30T00:47:12.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.250+0000 7fcd6e600640 1 -- start start 2024-03-30T00:47:12.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.250+0000 7fcd6e600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd680ce520 0x7fcd680ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.250+0000 7fcd6e600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcd680cabb0 0x7fcd680cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.250+0000 7fcd6e600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6806d070 con 0x7fcd6815a640 2024-03-30T00:47:12.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.250+0000 7fcd6e600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd680cbba0 con 0x7fcd680cabb0 2024-03-30T00:47:12.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.250+0000 7fcd6e600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680cee40 con 0x7fcd680ce520 2024-03-30T00:47:12.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.250+0000 7fcd67e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcd680cabb0 0x7fcd680cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.250+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd680ce520 0x7fcd680ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.254+0000 7fcd67400640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fcd6815a640 0x7fcd6806dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:44492/0 (socket says 172.21.15.66:44492) 2024-03-30T00:47:12.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.254+0000 7fcd67400640 1 -- 172.21.15.66:0/1675133513 learned_addr learned my addr 172.21.15.66:0/1675133513 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:12.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.254+0000 7fcd5ea00640 1 -- 172.21.15.66:0/1675133513 >> v2:172.21.15.156:3300/0 conn(0x7fcd680cabb0 msgr2=0x7fcd680cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.254+0000 7fcd5ea00640 1 --2- 172.21.15.66:0/1675133513 >> v2:172.21.15.156:3300/0 conn(0x7fcd680cabb0 0x7fcd680cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.254+0000 7fcd5ea00640 1 -- 172.21.15.66:0/1675133513 >> v1:172.21.15.156:6789/0 conn(0x7fcd6815a640 legacy=0x7fcd6806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.254+0000 7fcd5ea00640 1 -- 172.21.15.66:0/1675133513 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680cb8c0 con 0x7fcd680ce520 2024-03-30T00:47:12.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.254+0000 7fcd5ea00640 1 --2- 172.21.15.66:0/1675133513 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd680ce520 0x7fcd680ce900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097c0 tx=0x7fcd5001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6514686aa9565e36 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.254+0000 7fcd66a00640 1 -- 172.21.15.66:0/1675133513 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500ab670 con 0x7fcd680ce520 2024-03-30T00:47:12.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.254+0000 7fcd66a00640 1 -- 172.21.15.66:0/1675133513 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd500ab7d0 con 0x7fcd680ce520 2024-03-30T00:47:12.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.254+0000 7fcd66a00640 1 -- 172.21.15.66:0/1675133513 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500abb60 con 0x7fcd680ce520 2024-03-30T00:47:12.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 -- 172.21.15.66:0/1675133513 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd680ce520 msgr2=0x7fcd680ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 --2- 172.21.15.66:0/1675133513 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd680ce520 0x7fcd680ce900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097c0 tx=0x7fcd5001c470 comp rx=0 tx=0).stop 2024-03-30T00:47:12.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 -- 172.21.15.66:0/1675133513 shutdown_connections 2024-03-30T00:47:12.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 --2- 172.21.15.66:0/1675133513 >> v2:172.21.15.156:3300/0 conn(0x7fcd680cabb0 0x7fcd680cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 --2- 172.21.15.66:0/1675133513 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd680ce520 0x7fcd680ce900 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 -- 172.21.15.66:0/1675133513 >> 172.21.15.66:0/1675133513 conn(0x7fcd68069f60 msgr2=0x7fcd6806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 -- 172.21.15.66:0/1675133513 shutdown_connections 2024-03-30T00:47:12.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 -- 172.21.15.66:0/1675133513 wait complete. 2024-03-30T00:47:12.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 Processor -- start 2024-03-30T00:47:12.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 -- start start 2024-03-30T00:47:12.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd680cabb0 0x7fcd680af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd680ce520 0x7fcd680c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd6815a640 0x7fcd680c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680c4000 con 0x7fcd680ce520 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680c45c0 con 0x7fcd6815a640 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd6e600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680ca580 con 0x7fcd680cabb0 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd680ce520 0x7fcd680c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd680ce520 0x7fcd680c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53920/0 (socket says 172.21.15.66:53920) 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd5ea00640 1 -- 172.21.15.66:0/2644815893 learned_addr learned my addr 172.21.15.66:0/2644815893 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd67e00640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd6815a640 0x7fcd680c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd67400640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd680cabb0 0x7fcd680af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd5ea00640 1 -- 172.21.15.66:0/2644815893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd680cabb0 msgr2=0x7fcd680af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd5ea00640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd680cabb0 0x7fcd680af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd5ea00640 1 -- 172.21.15.66:0/2644815893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd6815a640 msgr2=0x7fcd680c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd5ea00640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd6815a640 0x7fcd680c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.258+0000 7fcd5ea00640 1 -- 172.21.15.66:0/2644815893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50009420 con 0x7fcd680ce520 2024-03-30T00:47:12.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.262+0000 7fcd5ea00640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd680ce520 0x7fcd680c9a60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fcd5001a0a0 tx=0x7fcd5001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.262+0000 7fcd5fe00640 1 -- 172.21.15.66:0/2644815893 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500b5810 con 0x7fcd680ce520 2024-03-30T00:47:12.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.262+0000 7fcd5fe00640 1 -- 172.21.15.66:0/2644815893 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd500b5970 con 0x7fcd680ce520 2024-03-30T00:47:12.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.262+0000 7fcd6e600640 1 -- 172.21.15.66:0/2644815893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd680ca860 con 0x7fcd680ce520 2024-03-30T00:47:12.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.262+0000 7fcd5fe00640 1 -- 172.21.15.66:0/2644815893 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500b5d20 con 0x7fcd680ce520 2024-03-30T00:47:12.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.262+0000 7fcd6e600640 1 -- 172.21.15.66:0/2644815893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd680c4a50 con 0x7fcd680ce520 2024-03-30T00:47:12.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.262+0000 7fcd6e600640 1 -- 172.21.15.66:0/2644815893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd68066c30 con 0x7fcd680ce520 2024-03-30T00:47:12.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.262+0000 7fcd5fe00640 1 -- 172.21.15.66:0/2644815893 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcd500bf410 con 0x7fcd680ce520 2024-03-30T00:47:12.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.262+0000 7fcd5fe00640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd44066690 0x7fcd44068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.262+0000 7fcd5fe00640 1 -- 172.21.15.66:0/2644815893 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fcd5011f0e0 con 0x7fcd680ce520 2024-03-30T00:47:12.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.262+0000 7fcd67400640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd44066690 0x7fcd44068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.266+0000 7fcd67400640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd44066690 0x7fcd44068b50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005890 tx=0x7fcd58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.266+0000 7fcd5fe00640 1 -- 172.21.15.66:0/2644815893 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcd500ec370 con 0x7fcd680ce520 2024-03-30T00:47:12.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.462+0000 7fcd6e600640 1 -- 172.21.15.66:0/2644815893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd680c4be0 con 0x7fcd680ce520 2024-03-30T00:47:12.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.462+0000 7fcd5fe00640 1 -- 172.21.15.66:0/2644815893 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fcd500ef990 con 0x7fcd680ce520 2024-03-30T00:47:12.471 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:47:12.471 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:46:45.565028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6836/1626022080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1626022080},{"type":"v1","addr":"172.21.15.66:6836","nonce":1626022080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:47:12.471 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 65 2024-03-30T00:47:12.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 -- 172.21.15.66:0/2644815893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd44066690 msgr2=0x7fcd44068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd44066690 0x7fcd44068b50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005890 tx=0x7fcd58009040 comp rx=0 tx=0).stop 2024-03-30T00:47:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 -- 172.21.15.66:0/2644815893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd680ce520 msgr2=0x7fcd680c9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd680ce520 0x7fcd680c9a60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fcd5001a0a0 tx=0x7fcd5001acb0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 -- 172.21.15.66:0/2644815893 shutdown_connections 2024-03-30T00:47:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd6815a640 0x7fcd680c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd44066690 0x7fcd44068b50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd680ce520 0x7fcd680c9a60 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 --2- 172.21.15.66:0/2644815893 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd680cabb0 0x7fcd680af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 -- 172.21.15.66:0/2644815893 >> 172.21.15.66:0/2644815893 conn(0x7fcd68069f60 msgr2=0x7fcd680b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 -- 172.21.15.66:0/2644815893 shutdown_connections 2024-03-30T00:47:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.466+0000 7fcd6e600640 1 -- 172.21.15.66:0/2644815893 wait complete. 2024-03-30T00:47:12.493 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-03-30T00:47:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.626+0000 7fdbd5600640 1 Processor -- start 2024-03-30T00:47:12.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.626+0000 7fdbd5600640 1 -- start start 2024-03-30T00:47:12.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.626+0000 7fdbd5600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd006cb90 0x7fdbd00d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.626+0000 7fdbd5600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd015d250 con 0x7fdbd00cdef0 2024-03-30T00:47:12.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.626+0000 7fdbd5600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd015d530 con 0x7fdbd015c6b0 2024-03-30T00:47:12.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.626+0000 7fdbd5600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd006b1f0 con 0x7fdbd006cb90 2024-03-30T00:47:12.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.626+0000 7fdbd4c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdbd00cdef0 0x7fdbd01604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60552/0 (socket says 172.21.15.66:60552) 2024-03-30T00:47:12.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.626+0000 7fdbd4c00640 1 -- 172.21.15.66:0/1826130682 learned_addr learned my addr 172.21.15.66:0/1826130682 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:12.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/1826130682 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd006cb90 0x7fdbd00d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbcf400640 1 -- 172.21.15.66:0/1826130682 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054600035 0 0) 0x7fdbd015d250 con 0x7fdbd00cdef0 2024-03-30T00:47:12.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbcf400640 1 -- 172.21.15.66:0/1826130682 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb4003530 con 0x7fdbd00cdef0 2024-03-30T00:47:12.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbcf400640 1 -- 172.21.15.66:0/1826130682 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043099139 0 0) 0x7fdbd015d530 con 0x7fdbd015c6b0 2024-03-30T00:47:12.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbcf400640 1 -- 172.21.15.66:0/1826130682 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd015d250 con 0x7fdbd015c6b0 2024-03-30T00:47:12.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbc7e00640 1 -- 172.21.15.66:0/1826130682 >> v1:172.21.15.156:6789/0 conn(0x7fdbd015c6b0 legacy=0x7fdbd015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbc7e00640 1 -- 172.21.15.66:0/1826130682 >> v1:172.21.15.66:6789/0 conn(0x7fdbd00cdef0 legacy=0x7fdbd01604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbc7e00640 1 -- 172.21.15.66:0/1826130682 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd015d810 con 0x7fdbd006cb90 2024-03-30T00:47:12.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/1826130682 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd006cb90 0x7fdbd00d0c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80097f0 tx=0x7fdbb801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fe941aef60be8bf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbcf400640 1 -- 172.21.15.66:0/1826130682 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80ac630 con 0x7fdbd006cb90 2024-03-30T00:47:12.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbcf400640 1 -- 172.21.15.66:0/1826130682 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb80ac790 con 0x7fdbd006cb90 2024-03-30T00:47:12.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.630+0000 7fdbcf400640 1 -- 172.21.15.66:0/1826130682 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80acb20 con 0x7fdbd006cb90 2024-03-30T00:47:12.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 -- 172.21.15.66:0/1826130682 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd006cb90 msgr2=0x7fdbd00d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 --2- 172.21.15.66:0/1826130682 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd006cb90 0x7fdbd00d0c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80097f0 tx=0x7fdbb801c4b0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 -- 172.21.15.66:0/1826130682 shutdown_connections 2024-03-30T00:47:12.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 --2- 172.21.15.66:0/1826130682 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd006cb90 0x7fdbd00d0c50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 -- 172.21.15.66:0/1826130682 >> 172.21.15.66:0/1826130682 conn(0x7fdbd0069d90 msgr2=0x7fdbd006a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 -- 172.21.15.66:0/1826130682 shutdown_connections 2024-03-30T00:47:12.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 -- 172.21.15.66:0/1826130682 wait complete. 2024-03-30T00:47:12.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 Processor -- start 2024-03-30T00:47:12.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 -- start start 2024-03-30T00:47:12.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd006cb90 0x7fdbd015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd00cdef0 0x7fdbd01587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbd015c6b0 0x7fdbd01538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd0158ce0 con 0x7fdbd00cdef0 2024-03-30T00:47:12.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00ac260 con 0x7fdbd015c6b0 2024-03-30T00:47:12.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd5600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd0158ef0 con 0x7fdbd006cb90 2024-03-30T00:47:12.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd00cdef0 0x7fdbd01587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd00cdef0 0x7fdbd01587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53936/0 (socket says 172.21.15.66:53936) 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbc7e00640 1 -- 172.21.15.66:0/1689624121 learned_addr learned my addr 172.21.15.66:0/1689624121 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbcfe00640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd006cb90 0x7fdbd015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbd4c00640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbd015c6b0 0x7fdbd01538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbc7e00640 1 -- 172.21.15.66:0/1689624121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd006cb90 msgr2=0x7fdbd015ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd006cb90 0x7fdbd015ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbc7e00640 1 -- 172.21.15.66:0/1689624121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbd015c6b0 msgr2=0x7fdbd01538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.634+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbd015c6b0 0x7fdbd01538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.638+0000 7fdbc7e00640 1 -- 172.21.15.66:0/1689624121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb8009420 con 0x7fdbd00cdef0 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.638+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd00cdef0 0x7fdbd01587a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80190c0 tx=0x7fdbb8019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.638+0000 7fdbccc00640 1 -- 172.21.15.66:0/1689624121 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80b5610 con 0x7fdbd00cdef0 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.638+0000 7fdbccc00640 1 -- 172.21.15.66:0/1689624121 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb80b5790 con 0x7fdbd00cdef0 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.638+0000 7fdbccc00640 1 -- 172.21.15.66:0/1689624121 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80b5b40 con 0x7fdbd00cdef0 2024-03-30T00:47:12.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.638+0000 7fdbd5600640 1 -- 172.21.15.66:0/1689624121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd01591d0 con 0x7fdbd00cdef0 2024-03-30T00:47:12.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.638+0000 7fdbd5600640 1 -- 172.21.15.66:0/1689624121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd0159790 con 0x7fdbd00cdef0 2024-03-30T00:47:12.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.638+0000 7fdbccc00640 1 -- 172.21.15.66:0/1689624121 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdbb80b3040 con 0x7fdbd00cdef0 2024-03-30T00:47:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbccc00640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbac066690 0x7fdbac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbccc00640 1 -- 172.21.15.66:0/1689624121 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fdbb811e4d0 con 0x7fdbd00cdef0 2024-03-30T00:47:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbcfe00640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbac066690 0x7fdbac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 Processor -- start 2024-03-30T00:47:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 -- start start 2024-03-30T00:47:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbcfe00640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbac066690 0x7fdbac068b50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00011f0 tx=0x7fdbc0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895290 0x55c60e895670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e84bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 0x55c60e84c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60e714340 con 0x55c60e895bb0 2024-03-30T00:47:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60e6a0cd0 con 0x55c60e84c1b0 2024-03-30T00:47:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c60e69e2a0 con 0x55c60e895290 2024-03-30T00:47:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e84bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e84bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53940/0 (socket says 172.21.15.66:53940) 2024-03-30T00:47:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc7e00640 1 -- 172.21.15.66:0/308073311 learned_addr learned my addr 172.21.15.66:0/308073311 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc7e00640 1 -- 172.21.15.66:0/308073311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895290 msgr2=0x55c60e895670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/308073311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895290 0x55c60e895670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc7e00640 1 -- 172.21.15.66:0/308073311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 msgr2=0x55c60e84c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/308073311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 0x55c60e84c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc7e00640 1 -- 172.21.15.66:0/308073311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c60e38c4b0 con 0x55c60e895bb0 2024-03-30T00:47:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/308073311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e84bc70 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80eefe0 tx=0x7fdbb80ef1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3888b0c849fa70bf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc4c00640 1 -- 172.21.15.66:0/308073311 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80ef2e0 con 0x55c60e895bb0 2024-03-30T00:47:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc4c00640 1 -- 172.21.15.66:0/308073311 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb80f3110 con 0x55c60e895bb0 2024-03-30T00:47:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbc4c00640 1 -- 172.21.15.66:0/308073311 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80f76d0 con 0x55c60e895bb0 2024-03-30T00:47:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 -- 172.21.15.66:0/308073311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 msgr2=0x55c60e84bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 --2- 172.21.15.66:0/308073311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e84bc70 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80eefe0 tx=0x7fdbb80ef1b0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 -- 172.21.15.66:0/308073311 shutdown_connections 2024-03-30T00:47:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 --2- 172.21.15.66:0/308073311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 0x55c60e84c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 --2- 172.21.15.66:0/308073311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e84bc70 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 --2- 172.21.15.66:0/308073311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895290 0x55c60e895670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 -- 172.21.15.66:0/308073311 >> 172.21.15.66:0/308073311 conn(0x55c60e848160 msgr2=0x55c60e84a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 -- 172.21.15.66:0/308073311 shutdown_connections 2024-03-30T00:47:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.642+0000 7fdbda050000 1 -- 172.21.15.66:0/308073311 wait complete. 2024-03-30T00:47:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 -- start start 2024-03-30T00:47:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 Processor -- start 2024-03-30T00:47:12.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 0x55c60e851650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895290 0x55c60e851b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895bb0 0x55c60e856e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60e699b90 con 0x55c60e895290 2024-03-30T00:47:12.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60e6570a0 con 0x55c60e84c1b0 2024-03-30T00:47:12.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c60e656d40 con 0x55c60e895bb0 2024-03-30T00:47:12.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 10 client.0 _ll_register_callbacks cb 0x55c60e74ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:12.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895290 0x55c60e851b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895290 0x55c60e851b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53956/0 (socket says 172.21.15.66:53956) 2024-03-30T00:47:12.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbc7e00640 1 -- 172.21.15.66:0/1277702275 learned_addr learned my addr 172.21.15.66:0/1277702275 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbd4c00640 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895bb0 0x55c60e856e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbcfe00640 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 0x55c60e851650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbc7e00640 1 -- 172.21.15.66:0/1277702275 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895bb0 msgr2=0x55c60e856e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895bb0 0x55c60e856e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbc7e00640 1 -- 172.21.15.66:0/1277702275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 msgr2=0x55c60e851650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 0x55c60e851650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbc7e00640 1 -- 172.21.15.66:0/1277702275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd0159790 con 0x55c60e895290 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895290 0x55c60e851b90 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fdbb801ca20 tx=0x7fdbb801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6fa21b383c0efe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbbd600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbbd600640 1 -- 172.21.15.66:0/1277702275 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80f3190 con 0x55c60e895290 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbbd600640 1 -- 172.21.15.66:0/1277702275 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fdbb80f76d0 con 0x55c60e895290 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 -- 172.21.15.66:0/1277702275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c60e669550 con 0x55c60e895290 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbbd600640 1 -- 172.21.15.66:0/1277702275 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbb810d0d0 con 0x55c60e895290 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 10 client.8085 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 -- 172.21.15.66:0/1277702275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55c60e65eba0 con 0x55c60e895290 2024-03-30T00:47:12.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbbd600640 1 -- 172.21.15.66:0/1277702275 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb8111080 con 0x55c60e895290 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbbd600640 1 -- 172.21.15.66:0/1277702275 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdbb811e0b0 con 0x55c60e895290 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 10 client.8085 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 10 client.8085 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 20 client.8085 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55c60e857850 0x55c60e8a6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 4 client.8085 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbda050000 1 -- 172.21.15.66:0/1277702275 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c60e696970 con 0x55c60e857850 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbcfe00640 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55c60e857850 0x55c60e8a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbcfe00640 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55c60e857850 0x55c60e8a6330 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=572af27fac82f903 server_cookie=bac5670a80ccf54e in_seq=0 out_seq=0 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.646+0000 7fdbbd600640 10 client.8085 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbbd600640 1 -- 172.21.15.66:0/1277702275 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c60e696970 con 0x55c60e857850 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbbd600640 10 client.8085 handle_command_reply: tid=0 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 1 client.8085 shutdown 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 1 -- 172.21.15.66:0/1277702275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895290 msgr2=0x55c60e851b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895290 0x55c60e851b90 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fdbb801ca20 tx=0x7fdbb801c4b0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 1 -- 172.21.15.66:0/1277702275 shutdown_connections 2024-03-30T00:47:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55c60e857850 0x55c60e8a6330 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895bb0 0x55c60e856e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895290 0x55c60e851b90 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbbd600640 0 client.8085 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 1 --2- 172.21.15.66:0/1277702275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 0x55c60e851650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 1 -- 172.21.15.66:0/1277702275 >> 172.21.15.66:0/1277702275 conn(0x55c60e848160 msgr2=0x55c60e84ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 1 -- 172.21.15.66:0/1277702275 shutdown_connections 2024-03-30T00:47:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 1 -- 172.21.15.66:0/1277702275 wait complete. 2024-03-30T00:47:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.650+0000 7fdbda050000 20 client.8085 trim_cache size 0 max 16384 2024-03-30T00:47:12.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 Processor -- start 2024-03-30T00:47:12.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 -- start start 2024-03-30T00:47:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 0x55c60e84b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895290 0x55c60e84baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60e356c40 con 0x55c60e895bb0 2024-03-30T00:47:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60e64af00 con 0x55c60e84c1b0 2024-03-30T00:47:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c60e636a40 con 0x55c60e895290 2024-03-30T00:47:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbd4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbd4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53960/0 (socket says 172.21.15.66:53960) 2024-03-30T00:47:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbd4c00640 1 -- 172.21.15.66:0/1416246796 learned_addr learned my addr 172.21.15.66:0/1416246796 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbcfe00640 1 --2- 172.21.15.66:0/1416246796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 0x55c60e84b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/1416246796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895290 0x55c60e84baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbd4c00640 1 -- 172.21.15.66:0/1416246796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895290 msgr2=0x55c60e84baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbd4c00640 1 --2- 172.21.15.66:0/1416246796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895290 0x55c60e84baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbd4c00640 1 -- 172.21.15.66:0/1416246796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 msgr2=0x55c60e84b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbd4c00640 1 --2- 172.21.15.66:0/1416246796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 0x55c60e84b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbd4c00640 1 -- 172.21.15.66:0/1416246796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c60e644ef0 con 0x55c60e895bb0 2024-03-30T00:47:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/1416246796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895290 0x55c60e84baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbd4c00640 1 --2- 172.21.15.66:0/1416246796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8a4320 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8001f50 tx=0x7fdbc8011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c843db409cd2c00e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbc5600640 1 -- 172.21.15.66:0/1416246796 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8011e50 con 0x55c60e895bb0 2024-03-30T00:47:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbc5600640 1 -- 172.21.15.66:0/1416246796 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc8005be0 con 0x55c60e895bb0 2024-03-30T00:47:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbc5600640 1 -- 172.21.15.66:0/1416246796 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80070d0 con 0x55c60e895bb0 2024-03-30T00:47:12.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 -- 172.21.15.66:0/1416246796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 msgr2=0x55c60e8a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 --2- 172.21.15.66:0/1416246796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8a4320 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8001f50 tx=0x7fdbc8011620 comp rx=0 tx=0).stop 2024-03-30T00:47:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 -- 172.21.15.66:0/1416246796 shutdown_connections 2024-03-30T00:47:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 --2- 172.21.15.66:0/1416246796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8a4320 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 --2- 172.21.15.66:0/1416246796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e895290 0x55c60e84baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 --2- 172.21.15.66:0/1416246796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e84c1b0 0x55c60e84b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 -- 172.21.15.66:0/1416246796 >> 172.21.15.66:0/1416246796 conn(0x55c60e848160 msgr2=0x55c60e851510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 -- 172.21.15.66:0/1416246796 shutdown_connections 2024-03-30T00:47:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 -- 172.21.15.66:0/1416246796 wait complete. 2024-03-30T00:47:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.662+0000 7fdbda050000 1 -- start start 2024-03-30T00:47:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 Processor -- start 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e84c1b0 0x55c60e8a7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e895290 0x55c60e8a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8ef980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60e653300 con 0x55c60e895bb0 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60e693960 con 0x55c60e895290 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c60e699b90 con 0x55c60e84c1b0 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 10 client.0 _ll_register_callbacks cb 0x55c60e737870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbd4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbd4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8ef980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53970/0 (socket says 172.21.15.66:53970) 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbd4c00640 1 -- 172.21.15.66:0/2584163121 learned_addr learned my addr 172.21.15.66:0/2584163121 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbc7e00640 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e895290 0x55c60e8a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbcfe00640 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e84c1b0 0x55c60e8a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbd4c00640 1 -- 172.21.15.66:0/2584163121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e84c1b0 msgr2=0x55c60e8a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbd4c00640 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e84c1b0 0x55c60e8a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbd4c00640 1 -- 172.21.15.66:0/2584163121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e895290 msgr2=0x55c60e8a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbd4c00640 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e895290 0x55c60e8a76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbd4c00640 1 -- 172.21.15.66:0/2584163121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc800f040 con 0x55c60e895bb0 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbd4c00640 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8ef980 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009010 tx=0x7fdbc8001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67370ebb0275a52c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbbd600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbbd600640 1 -- 172.21.15.66:0/2584163121 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8011e50 con 0x55c60e895bb0 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 -- 172.21.15.66:0/2584163121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c60e649620 con 0x55c60e895bb0 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbbd600640 1 -- 172.21.15.66:0/2584163121 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fdbc8007910 con 0x55c60e895bb0 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbbd600640 1 -- 172.21.15.66:0/2584163121 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbc8007ad0 con 0x55c60e895bb0 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 10 client.8091 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 -- 172.21.15.66:0/2584163121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55c60e395fc0 con 0x55c60e895bb0 2024-03-30T00:47:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbbd600640 1 -- 172.21.15.66:0/2584163121 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c60e649620 con 0x55c60e895bb0 2024-03-30T00:47:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbbd600640 1 -- 172.21.15.66:0/2584163121 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdbc8028470 con 0x55c60e895bb0 2024-03-30T00:47:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 10 client.8091 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 10 client.8091 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:47:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 20 client.8091 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55c60e8efec0 0x55c60e8f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 4 client.8091 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-30T00:47:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 -- 172.21.15.66:0/2584163121 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55c60e6984e0 con 0x55c60e8efec0 2024-03-30T00:47:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbcfe00640 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55c60e8efec0 0x55c60e8f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbbd600640 10 client.8091 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:12.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbcfe00640 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55c60e8efec0 0x55c60e8f02a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b5fccab4f6dfe57 server_cookie=ac319ceda9a7a2f8 in_seq=0 out_seq=0 2024-03-30T00:47:12.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbbd600640 1 -- 172.21.15.66:0/2584163121 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+419 (crc 0 0 0) 0x55c60e6984e0 con 0x55c60e8efec0 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbbd600640 10 client.8091 handle_command_reply: tid=0 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 client.8091 shutdown 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 -- 172.21.15.66:0/2584163121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 msgr2=0x55c60e8ef980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.666+0000 7fdbda050000 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8ef980 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009010 tx=0x7fdbc8001260 comp rx=0 tx=0).stop 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbda050000 1 -- 172.21.15.66:0/2584163121 shutdown_connections 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbda050000 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55c60e8efec0 0x55c60e8f02a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbda050000 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c60e895bb0 0x55c60e8ef980 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbda050000 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c60e895290 0x55c60e8a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbda050000 1 --2- 172.21.15.66:0/2584163121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c60e84c1b0 0x55c60e8a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbbd600640 0 client.8091 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbda050000 1 -- 172.21.15.66:0/2584163121 >> 172.21.15.66:0/2584163121 conn(0x55c60e848160 msgr2=0x55c60e84a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbda050000 1 -- 172.21.15.66:0/2584163121 shutdown_connections 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbda050000 1 -- 172.21.15.66:0/2584163121 wait complete. 2024-03-30T00:47:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbda050000 20 client.8091 trim_cache size 0 max 16384 2024-03-30T00:47:12.682 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:47:12.682 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:47:12.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:47:12.682 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2024-03-30T00:47:12.682 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 183660447, 2024-03-30T00:47:12.682 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627777, 2024-03-30T00:47:12.682 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/testdir/hardlink" 2024-03-30T00:47:12.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:47:12.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:47:12.682 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2024-03-30T00:47:12.682 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 2360735214, 2024-03-30T00:47:12.683 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627776, 2024-03-30T00:47:12.683 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/dir" 2024-03-30T00:47:12.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:47:12.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:47:12.683 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "dir_frag", 2024-03-30T00:47:12.683 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 2376692389, 2024-03-30T00:47:12.683 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627776, 2024-03-30T00:47:12.683 INFO:teuthology.orchestra.run.smithi066.stdout: "frag": "*", 2024-03-30T00:47:12.683 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/dir" 2024-03-30T00:47:12.683 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:47:12.683 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:47:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 -- 172.21.15.66:0/1689624121 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbac066690 msgr2=0x7fdbac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbac066690 0x7fdbac068b50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00011f0 tx=0x7fdbc0009040 comp rx=0 tx=0).stop 2024-03-30T00:47:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 -- 172.21.15.66:0/1689624121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd00cdef0 msgr2=0x7fdbd01587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd00cdef0 0x7fdbd01587a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80190c0 tx=0x7fdbb8019250 comp rx=0 tx=0).stop 2024-03-30T00:47:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 -- 172.21.15.66:0/1689624121 shutdown_connections 2024-03-30T00:47:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbd015c6b0 0x7fdbd01538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbac066690 0x7fdbac068b50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbd00cdef0 0x7fdbd01587a0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 --2- 172.21.15.66:0/1689624121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd006cb90 0x7fdbd015ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 -- 172.21.15.66:0/1689624121 >> 172.21.15.66:0/1689624121 conn(0x7fdbd0069d90 msgr2=0x7fdbd01540f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 -- 172.21.15.66:0/1689624121 shutdown_connections 2024-03-30T00:47:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.670+0000 7fdbd5600640 1 -- 172.21.15.66:0/1689624121 wait complete. 2024-03-30T00:47:12.697 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-30T00:47:12.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.826+0000 7f8276c00640 1 Processor -- start 2024-03-30T00:47:12.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.826+0000 7f8276c00640 1 -- start start 2024-03-30T00:47:12.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8276c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f827015c970 0x7f82700adf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8276c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82700ae480 0x7f827006aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8276c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827015cd50 con 0x7f827006b410 2024-03-30T00:47:12.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8276c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827006aab0 con 0x7f827015c970 2024-03-30T00:47:12.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8276c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8270069e30 con 0x7f82700ae480 2024-03-30T00:47:12.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f826fe00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f827015c970 0x7f82700adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8267400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82700ae480 0x7f827006aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8274c00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f827006b410 0x7f827006c930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:44502/0 (socket says 172.21.15.66:44502) 2024-03-30T00:47:12.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8274c00640 1 -- 172.21.15.66:0/1467659115 learned_addr learned my addr 172.21.15.66:0/1467659115 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:12.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f826f400640 1 -- 172.21.15.66:0/1467659115 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303385211 0 0) 0x7f827015cd50 con 0x7f827006b410 2024-03-30T00:47:12.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8267400640 1 -- 172.21.15.66:0/1467659115 >> v2:172.21.15.156:3300/0 conn(0x7f827015c970 msgr2=0x7f82700adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:12.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8267400640 1 --2- 172.21.15.66:0/1467659115 >> v2:172.21.15.156:3300/0 conn(0x7f827015c970 0x7f82700adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8267400640 1 -- 172.21.15.66:0/1467659115 >> v1:172.21.15.156:6789/0 conn(0x7f827006b410 legacy=0x7f827006c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.830+0000 7f8267400640 1 -- 172.21.15.66:0/1467659115 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827006a110 con 0x7f82700ae480 2024-03-30T00:47:12.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f8267400640 1 --2- 172.21.15.66:0/1467659115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82700ae480 0x7f827006aed0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f82580097c0 tx=0x7f825801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cd5d0275f5419cd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f826f400640 1 -- 172.21.15.66:0/1467659115 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82580ab670 con 0x7f82700ae480 2024-03-30T00:47:12.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f826f400640 1 -- 172.21.15.66:0/1467659115 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82580ab7d0 con 0x7f82700ae480 2024-03-30T00:47:12.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f826f400640 1 -- 172.21.15.66:0/1467659115 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82580abb60 con 0x7f82700ae480 2024-03-30T00:47:12.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f8276c00640 1 -- 172.21.15.66:0/1467659115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82700ae480 msgr2=0x7f827006aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f8276c00640 1 --2- 172.21.15.66:0/1467659115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82700ae480 0x7f827006aed0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f82580097c0 tx=0x7f825801c470 comp rx=0 tx=0).stop 2024-03-30T00:47:12.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f8276c00640 1 -- 172.21.15.66:0/1467659115 shutdown_connections 2024-03-30T00:47:12.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f8276c00640 1 --2- 172.21.15.66:0/1467659115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82700ae480 0x7f827006aed0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f8276c00640 1 --2- 172.21.15.66:0/1467659115 >> v2:172.21.15.156:3300/0 conn(0x7f827015c970 0x7f82700adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f8276c00640 1 -- 172.21.15.66:0/1467659115 >> 172.21.15.66:0/1467659115 conn(0x7f82700b08f0 msgr2=0x7f82700b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:12.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f8276c00640 1 -- 172.21.15.66:0/1467659115 shutdown_connections 2024-03-30T00:47:12.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f8276c00640 1 -- 172.21.15.66:0/1467659115 wait complete. 2024-03-30T00:47:12.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f8276c00640 1 Processor -- start 2024-03-30T00:47:12.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.834+0000 7f8276c00640 1 -- start start 2024-03-30T00:47:12.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f8276c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f827006b410 0x7f8270152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f8276c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82700ae480 0x7f82701542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f8276c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f827015c970 0x7f8270158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f8276c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82700b2420 con 0x7f827006b410 2024-03-30T00:47:12.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f8276c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82700b1a80 con 0x7f82700ae480 2024-03-30T00:47:12.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f8276c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82700b1d60 con 0x7f827015c970 2024-03-30T00:47:12.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f827006b410 0x7f8270152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f827006b410 0x7f8270152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53974/0 (socket says 172.21.15.66:53974) 2024-03-30T00:47:12.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826fe00640 1 -- 172.21.15.66:0/322968188 learned_addr learned my addr 172.21.15.66:0/322968188 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:12.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f8267400640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82700ae480 0x7f82701542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f8274c00640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f827015c970 0x7f8270158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826fe00640 1 -- 172.21.15.66:0/322968188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f827015c970 msgr2=0x7f8270158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826fe00640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f827015c970 0x7f8270158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826fe00640 1 -- 172.21.15.66:0/322968188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82700ae480 msgr2=0x7f82701542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:12.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826fe00640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82700ae480 0x7f82701542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:12.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826fe00640 1 -- 172.21.15.66:0/322968188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258009420 con 0x7f827006b410 2024-03-30T00:47:12.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826fe00640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f827006b410 0x7f8270152a10 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f826000e870 tx=0x7f826000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826cc00640 1 -- 172.21.15.66:0/322968188 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260009510 con 0x7f827006b410 2024-03-30T00:47:12.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826cc00640 1 -- 172.21.15.66:0/322968188 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826000eec0 con 0x7f827006b410 2024-03-30T00:47:12.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f826cc00640 1 -- 172.21.15.66:0/322968188 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826000cb40 con 0x7f827006b410 2024-03-30T00:47:12.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f8276c00640 1 -- 172.21.15.66:0/322968188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82700b2040 con 0x7f827006b410 2024-03-30T00:47:12.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f8276c00640 1 -- 172.21.15.66:0/322968188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8270152f50 con 0x7f827006b410 2024-03-30T00:47:12.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.838+0000 7f8276c00640 1 -- 172.21.15.66:0/322968188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8270161340 con 0x7f827006b410 2024-03-30T00:47:12.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.842+0000 7f826cc00640 1 -- 172.21.15.66:0/322968188 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f826000cca0 con 0x7f827006b410 2024-03-30T00:47:12.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.842+0000 7f826cc00640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f824c0666c0 0x7f824c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:12.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.842+0000 7f826cc00640 1 -- 172.21.15.66:0/322968188 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8260084470 con 0x7f827006b410 2024-03-30T00:47:12.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.842+0000 7f8267400640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f824c0666c0 0x7f824c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:12.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.842+0000 7f8267400640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f824c0666c0 0x7f824c068b80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f82580af000 tx=0x7f82580a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:12.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:12.846+0000 7f826cc00640 1 -- 172.21.15.66:0/322968188 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f82600515e0 con 0x7f827006b410 2024-03-30T00:47:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 -- 172.21.15.66:0/322968188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8270066c30 con 0x7f827006b410 2024-03-30T00:47:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f826cc00640 1 -- 172.21.15.66:0/322968188 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8260054c00 con 0x7f827006b410 2024-03-30T00:47:13.051 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:47:13.051 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-30T00:47:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 -- 172.21.15.66:0/322968188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f824c0666c0 msgr2=0x7f824c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f824c0666c0 0x7f824c068b80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f82580af000 tx=0x7f82580a7000 comp rx=0 tx=0).stop 2024-03-30T00:47:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 -- 172.21.15.66:0/322968188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f827006b410 msgr2=0x7f8270152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f827006b410 0x7f8270152a10 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f826000e870 tx=0x7f826000ec70 comp rx=0 tx=0).stop 2024-03-30T00:47:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 -- 172.21.15.66:0/322968188 shutdown_connections 2024-03-30T00:47:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f827015c970 0x7f8270158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82700ae480 0x7f82701542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f824c0666c0 0x7f824c068b80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 --2- 172.21.15.66:0/322968188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f827006b410 0x7f8270152a10 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 -- 172.21.15.66:0/322968188 >> 172.21.15.66:0/322968188 conn(0x7f82700b08f0 msgr2=0x7f827006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 -- 172.21.15.66:0/322968188 shutdown_connections 2024-03-30T00:47:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.042+0000 7f8276c00640 1 -- 172.21.15.66:0/322968188 wait complete. 2024-03-30T00:47:13.072 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-30T00:47:13.072 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-03-30T00:47:13.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.206+0000 7fbad3a00640 1 Processor -- start 2024-03-30T00:47:13.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.206+0000 7fbad3a00640 1 -- start start 2024-03-30T00:47:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad3a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbacc15e100 0x7fbacc099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad3a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbacc099ca0 0x7fbacc098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad3a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbacc0cc180 con 0x7fbacc098f60 2024-03-30T00:47:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad3a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbacc0993f0 con 0x7fbacc15e100 2024-03-30T00:47:13.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad3a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc15d550 con 0x7fbacc099ca0 2024-03-30T00:47:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad1000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbacc15e100 0x7fbacc099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad1a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fbacc098f60 0x7fbacc15e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60566/0 (socket says 172.21.15.66:60566) 2024-03-30T00:47:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad1a00640 1 -- 172.21.15.66:0/2481499399 learned_addr learned my addr 172.21.15.66:0/2481499399 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbac3e00640 1 --2- 172.21.15.66:0/2481499399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbacc099ca0 0x7fbacc098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad1000640 1 -- 172.21.15.66:0/2481499399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbacc099ca0 msgr2=0x7fbacc098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad1000640 1 --2- 172.21.15.66:0/2481499399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbacc099ca0 0x7fbacc098a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad1000640 1 -- 172.21.15.66:0/2481499399 >> v1:172.21.15.66:6789/0 conn(0x7fbacc098f60 legacy=0x7fbacc15e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbad1000640 1 -- 172.21.15.66:0/2481499399 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc15e4e0 con 0x7fbacc15e100 2024-03-30T00:47:13.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.210+0000 7fbacbe00640 1 -- 172.21.15.66:0/2481499399 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244857389 0 0) 0x7fbacc0cc180 con 0x7fbacc098f60 2024-03-30T00:47:13.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad1000640 1 --2- 172.21.15.66:0/2481499399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbacc15e100 0x7fbacc099760 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0097c0 tx=0x7fbabc01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab20825f1a382ac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbacbe00640 1 -- 172.21.15.66:0/2481499399 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0a8990 con 0x7fbacc15e100 2024-03-30T00:47:13.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbacbe00640 1 -- 172.21.15.66:0/2481499399 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc0a8af0 con 0x7fbacc15e100 2024-03-30T00:47:13.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbacbe00640 1 -- 172.21.15.66:0/2481499399 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0a8e80 con 0x7fbacc15e100 2024-03-30T00:47:13.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 -- 172.21.15.66:0/2481499399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbacc15e100 msgr2=0x7fbacc099760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 --2- 172.21.15.66:0/2481499399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbacc15e100 0x7fbacc099760 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0097c0 tx=0x7fbabc01ec90 comp rx=0 tx=0).stop 2024-03-30T00:47:13.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 -- 172.21.15.66:0/2481499399 shutdown_connections 2024-03-30T00:47:13.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 --2- 172.21.15.66:0/2481499399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbacc099ca0 0x7fbacc098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 --2- 172.21.15.66:0/2481499399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbacc15e100 0x7fbacc099760 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 -- 172.21.15.66:0/2481499399 >> 172.21.15.66:0/2481499399 conn(0x7fbacc0b2140 msgr2=0x7fbacc0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 -- 172.21.15.66:0/2481499399 shutdown_connections 2024-03-30T00:47:13.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 -- 172.21.15.66:0/2481499399 wait complete. 2024-03-30T00:47:13.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 Processor -- start 2024-03-30T00:47:13.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 -- start start 2024-03-30T00:47:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbacc098f60 0x7fbacc159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbacc099ca0 0x7fbacc0c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbacc15e100 0x7fbacc0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0c7550 con 0x7fbacc099ca0 2024-03-30T00:47:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0c70c0 con 0x7fbacc098f60 2024-03-30T00:47:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.214+0000 7fbad3a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc15a240 con 0x7fbacc15e100 2024-03-30T00:47:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbacc099ca0 0x7fbacc0c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbacc099ca0 0x7fbacc0c46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53990/0 (socket says 172.21.15.66:53990) 2024-03-30T00:47:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac3e00640 1 -- 172.21.15.66:0/756178595 learned_addr learned my addr 172.21.15.66:0/756178595 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:13.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac3e00640 1 -- 172.21.15.66:0/756178595 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbacc15e100 msgr2=0x7fbacc0c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac3e00640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbacc15e100 0x7fbacc0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac3e00640 1 -- 172.21.15.66:0/756178595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbacc098f60 msgr2=0x7fbacc159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbad1000640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbacc098f60 0x7fbacc159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac3e00640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbacc098f60 0x7fbacc159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac3e00640 1 -- 172.21.15.66:0/756178595 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc009420 con 0x7fbacc099ca0 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbad1000640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbacc098f60 0x7fbacc159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac3e00640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbacc099ca0 0x7fbacc0c46f0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fbab400d830 tx=0x7fbab400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac9600640 1 -- 172.21.15.66:0/756178595 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab400b6c0 con 0x7fbacc099ca0 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac9600640 1 -- 172.21.15.66:0/756178595 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab400b820 con 0x7fbacc099ca0 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbad3a00640 1 -- 172.21.15.66:0/756178595 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc0c6210 con 0x7fbacc099ca0 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac9600640 1 -- 172.21.15.66:0/756178595 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab400bb90 con 0x7fbacc099ca0 2024-03-30T00:47:13.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbad3a00640 1 -- 172.21.15.66:0/756178595 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc0c6790 con 0x7fbacc099ca0 2024-03-30T00:47:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac9600640 1 -- 172.21.15.66:0/756178595 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbab401e460 con 0x7fbacc099ca0 2024-03-30T00:47:13.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.218+0000 7fbac9600640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbaa8066690 0x7fbaa8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbac9600640 1 -- 172.21.15.66:0/756178595 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fbab4084bd0 con 0x7fbacc099ca0 2024-03-30T00:47:13.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1000640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbaa8066690 0x7fbaa8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 Processor -- start 2024-03-30T00:47:13.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 -- start start 2024-03-30T00:47:13.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1000640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbaa8066690 0x7fbaa8068b50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0b2000 tx=0x7fbabc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd3e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefd3e650 0x55eaefcf4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcf5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaefbbc170 con 0x55eaefcf4f60 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaefb49d80 con 0x55eaefd3dd30 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaefb47350 con 0x55eaefd3e650 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcf5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcf5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54000/0 (socket says 172.21.15.66:54000) 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1a00640 1 -- 172.21.15.66:0/3888278021 learned_addr learned my addr 172.21.15.66:0/3888278021 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbac3e00640 1 --2- 172.21.15.66:0/3888278021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefd3e650 0x55eaefcf4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1000640 1 --2- 172.21.15.66:0/3888278021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd3e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1a00640 1 -- 172.21.15.66:0/3888278021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefd3e650 msgr2=0x55eaefcf4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1a00640 1 --2- 172.21.15.66:0/3888278021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefd3e650 0x55eaefcf4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1a00640 1 -- 172.21.15.66:0/3888278021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 msgr2=0x55eaefd3e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1a00640 1 --2- 172.21.15.66:0/3888278021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd3e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1a00640 1 -- 172.21.15.66:0/3888278021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaef83f070 con 0x55eaefcf4f60 2024-03-30T00:47:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad1a00640 1 --2- 172.21.15.66:0/3888278021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcf5340 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fbac4001cb0 tx=0x7fbac4002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca75a5e4b9e87cd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbac1600640 1 -- 172.21.15.66:0/3888278021 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac400c7e0 con 0x55eaefcf4f60 2024-03-30T00:47:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbac1600640 1 -- 172.21.15.66:0/3888278021 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac400c960 con 0x55eaefcf4f60 2024-03-30T00:47:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbac1600640 1 -- 172.21.15.66:0/3888278021 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac400cd50 con 0x55eaefcf4f60 2024-03-30T00:47:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 -- 172.21.15.66:0/3888278021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 msgr2=0x55eaefcf5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 --2- 172.21.15.66:0/3888278021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcf5340 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fbac4001cb0 tx=0x7fbac4002170 comp rx=0 tx=0).stop 2024-03-30T00:47:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 -- 172.21.15.66:0/3888278021 shutdown_connections 2024-03-30T00:47:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 --2- 172.21.15.66:0/3888278021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcf5340 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 --2- 172.21.15.66:0/3888278021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefd3e650 0x55eaefcf4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 --2- 172.21.15.66:0/3888278021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd3e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 -- 172.21.15.66:0/3888278021 >> 172.21.15.66:0/3888278021 conn(0x55eaefcf0f10 msgr2=0x55eaefcf3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 -- 172.21.15.66:0/3888278021 shutdown_connections 2024-03-30T00:47:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.222+0000 7fbad6e17000 1 -- 172.21.15.66:0/3888278021 wait complete. 2024-03-30T00:47:13.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 -- start start 2024-03-30T00:47:13.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 Processor -- start 2024-03-30T00:47:13.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcfa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd3ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefd3e650 0x55eaefd3f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaefb42c40 con 0x55eaefcf4f60 2024-03-30T00:47:13.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaefae1fa0 con 0x55eaefd3dd30 2024-03-30T00:47:13.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaefb001c0 con 0x55eaefd3e650 2024-03-30T00:47:13.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 10 client.0 _ll_register_callbacks cb 0x55eaefce0060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:13.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcfa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcfa9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54004/0 (socket says 172.21.15.66:54004) 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad1000640 1 -- 172.21.15.66:0/1015957386 learned_addr learned my addr 172.21.15.66:0/1015957386 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbac3e00640 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd3ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad1a00640 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefd3e650 0x55eaefd3f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad1000640 1 -- 172.21.15.66:0/1015957386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefd3e650 msgr2=0x55eaefd3f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad1000640 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefd3e650 0x55eaefd3f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad1000640 1 -- 172.21.15.66:0/1015957386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 msgr2=0x55eaefd3ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad1000640 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd3ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad1000640 1 -- 172.21.15.66:0/1015957386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac40018e0 con 0x55eaefcf4f60 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad1000640 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcfa9a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fbabc01d340 tx=0x7fbabc01d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b72e0f3d85398fe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbaba000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbaba000640 1 -- 172.21.15.66:0/1015957386 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0bb620 con 0x55eaefcf4f60 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbaba000640 1 -- 172.21.15.66:0/1015957386 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fbabc0c1040 con 0x55eaefcf4f60 2024-03-30T00:47:13.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 -- 172.21.15.66:0/1015957386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eaefb3fa20 con 0x55eaefcf4f60 2024-03-30T00:47:13.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbaba000640 1 -- 172.21.15.66:0/1015957386 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbabc0bb780 con 0x55eaefcf4f60 2024-03-30T00:47:13.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 10 client.8109 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:13.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 -- 172.21.15.66:0/1015957386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55eaefb07ba0 con 0x55eaefcf4f60 2024-03-30T00:47:13.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbaba000640 1 -- 172.21.15.66:0/1015957386 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eaefb3fa20 con 0x55eaefcf4f60 2024-03-30T00:47:13.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbaba000640 1 -- 172.21.15.66:0/1015957386 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbabc0bb8e0 con 0x55eaefcf4f60 2024-03-30T00:47:13.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 10 client.8109 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:13.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 10 client.8109 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:13.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 20 client.8109 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:13.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55eaefd452c0 0x55eaefd456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 4 client.8109 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:47:13.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbad6e17000 1 -- 172.21.15.66:0/1015957386 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eaefb41590 con 0x55eaefd452c0 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.226+0000 7fbac3e00640 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55eaefd452c0 0x55eaefd456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbac3e00640 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55eaefd452c0 0x55eaefd456a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e11e4adb938a29 server_cookie=7459125c362663d4 in_seq=0 out_seq=0 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbaba000640 10 client.8109 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbaba000640 1 -- 172.21.15.66:0/1015957386 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eaefb41590 con 0x55eaefd452c0 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbaba000640 10 client.8109 handle_command_reply: tid=0 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 1 client.8109 shutdown 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 1 -- 172.21.15.66:0/1015957386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 msgr2=0x55eaefcfa9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcfa9a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fbabc01d340 tx=0x7fbabc01d4d0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 1 -- 172.21.15.66:0/1015957386 shutdown_connections 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55eaefd452c0 0x55eaefd456a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefd3e650 0x55eaefd3f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd3ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 1 --2- 172.21.15.66:0/1015957386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefcf4f60 0x55eaefcfa9a0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbaba000640 0 client.8109 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:13.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 1 -- 172.21.15.66:0/1015957386 >> 172.21.15.66:0/1015957386 conn(0x55eaefcf0f10 msgr2=0x55eaefcf3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 1 -- 172.21.15.66:0/1015957386 shutdown_connections 2024-03-30T00:47:13.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 1 -- 172.21.15.66:0/1015957386 wait complete. 2024-03-30T00:47:13.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.230+0000 7fbad6e17000 20 client.8109 trim_cache size 0 max 16384 2024-03-30T00:47:13.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.242+0000 7fbad6e17000 1 Processor -- start 2024-03-30T00:47:13.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 -- start start 2024-03-30T00:47:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefcf4f60 0x55eaefcf4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefcf4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd3fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaef7ffcf0 con 0x55eaefd3e650 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaefb09420 con 0x55eaefd3dd30 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaefb860c0 con 0x55eaefcf4f60 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd3fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd3fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54020/0 (socket says 172.21.15.66:54020) 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad1a00640 1 -- 172.21.15.66:0/3164839204 learned_addr learned my addr 172.21.15.66:0/3164839204 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbac3e00640 1 --2- 172.21.15.66:0/3164839204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefcf4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad1000640 1 --2- 172.21.15.66:0/3164839204 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefcf4f60 0x55eaefcf4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad1a00640 1 -- 172.21.15.66:0/3164839204 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefcf4f60 msgr2=0x55eaefcf4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad1a00640 1 --2- 172.21.15.66:0/3164839204 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefcf4f60 0x55eaefcf4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad1a00640 1 -- 172.21.15.66:0/3164839204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 msgr2=0x55eaefcf4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad1a00640 1 --2- 172.21.15.66:0/3164839204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefcf4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad1a00640 1 -- 172.21.15.66:0/3164839204 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaefaedef0 con 0x55eaefd3e650 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad1000640 1 --2- 172.21.15.66:0/3164839204 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefcf4f60 0x55eaefcf4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbac3e00640 1 --2- 172.21.15.66:0/3164839204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefcf4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad1a00640 1 --2- 172.21.15.66:0/3164839204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd3fa20 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fbac4000f90 tx=0x7fbac4006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb85aee21d3cda3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbac2000640 1 -- 172.21.15.66:0/3164839204 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4007cf0 con 0x55eaefd3e650 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbac2000640 1 -- 172.21.15.66:0/3164839204 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac400c7e0 con 0x55eaefd3e650 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbac2000640 1 -- 172.21.15.66:0/3164839204 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac400cb70 con 0x55eaefd3e650 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 -- 172.21.15.66:0/3164839204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 msgr2=0x55eaefd3fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 --2- 172.21.15.66:0/3164839204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd3fa20 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fbac4000f90 tx=0x7fbac4006000 comp rx=0 tx=0).stop 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 -- 172.21.15.66:0/3164839204 shutdown_connections 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 --2- 172.21.15.66:0/3164839204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd3fa20 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 --2- 172.21.15.66:0/3164839204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefcf4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 --2- 172.21.15.66:0/3164839204 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefcf4f60 0x55eaefcf4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 -- 172.21.15.66:0/3164839204 >> 172.21.15.66:0/3164839204 conn(0x55eaefcf0f10 msgr2=0x55eaefcf9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 -- 172.21.15.66:0/3164839204 shutdown_connections 2024-03-30T00:47:13.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 -- 172.21.15.66:0/3164839204 wait complete. 2024-03-30T00:47:13.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 -- start start 2024-03-30T00:47:13.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.246+0000 7fbad6e17000 1 Processor -- start 2024-03-30T00:47:13.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefcf4f60 0x55eaefd9d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd9d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd42240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaefb05680 con 0x55eaefd3e650 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaefb40040 con 0x55eaefd3dd30 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaefb41ad0 con 0x55eaefcf4f60 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 10 client.0 _ll_register_callbacks cb 0x55eaefbbc0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd42240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd42240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54032/0 (socket says 172.21.15.66:54032) 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1a00640 1 -- 172.21.15.66:0/310644759 learned_addr learned my addr 172.21.15.66:0/310644759 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbac3e00640 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd9d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1000640 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefcf4f60 0x55eaefd9d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1a00640 1 -- 172.21.15.66:0/310644759 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefcf4f60 msgr2=0x55eaefd9d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1a00640 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefcf4f60 0x55eaefd9d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1a00640 1 -- 172.21.15.66:0/310644759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 msgr2=0x55eaefd9d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1a00640 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd9d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1a00640 1 -- 172.21.15.66:0/310644759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaef83f070 con 0x55eaefd3e650 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1a00640 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd42240 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fbac40012f0 tx=0x7fbac4001410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d912861339f9e9b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbaba000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbaba000640 1 -- 172.21.15.66:0/310644759 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4021880 con 0x55eaefd3e650 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbaba000640 1 -- 172.21.15.66:0/310644759 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fbac40219e0 con 0x55eaefd3e650 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 -- 172.21.15.66:0/310644759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eaefaff5f0 con 0x55eaefd3e650 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbaba000640 1 -- 172.21.15.66:0/310644759 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbac400cea0 con 0x55eaefd3e650 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 10 client.8115 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:13.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 -- 172.21.15.66:0/310644759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55eaefae1a60 con 0x55eaefd3e650 2024-03-30T00:47:13.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbaba000640 1 -- 172.21.15.66:0/310644759 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eaefaff5f0 con 0x55eaefd3e650 2024-03-30T00:47:13.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbaba000640 1 -- 172.21.15.66:0/310644759 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbac4021ba0 con 0x55eaefd3e650 2024-03-30T00:47:13.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 10 client.8115 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:13.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 10 client.8115 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:13.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 20 client.8115 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:13.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55eaefd42780 0x55eaefd42b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 4 client.8115 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-03-30T00:47:13.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 -- 172.21.15.66:0/310644759 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55eaefb01410 con 0x55eaefd42780 2024-03-30T00:47:13.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1000640 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55eaefd42780 0x55eaefd42b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad1000640 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55eaefd42780 0x55eaefd42b60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d89f8bcd42ac0098 server_cookie=51ab28c1b15626cd in_seq=0 out_seq=0 2024-03-30T00:47:13.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbaba000640 10 client.8115 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:13.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbaba000640 1 -- 172.21.15.66:0/310644759 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55eaefb01410 con 0x55eaefd42780 2024-03-30T00:47:13.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbaba000640 10 client.8115 handle_command_reply: tid=0 2024-03-30T00:47:13.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 client.8115 shutdown 2024-03-30T00:47:13.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 -- 172.21.15.66:0/310644759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 msgr2=0x55eaefd42240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd42240 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fbac40012f0 tx=0x7fbac4001410 comp rx=0 tx=0).stop 2024-03-30T00:47:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 -- 172.21.15.66:0/310644759 shutdown_connections 2024-03-30T00:47:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55eaefd42780 0x55eaefd42b60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eaefd3e650 0x55eaefd42240 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaefd3dd30 0x55eaefd9d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 --2- 172.21.15.66:0/310644759 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55eaefcf4f60 0x55eaefd9d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbaba000640 0 client.8115 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 -- 172.21.15.66:0/310644759 >> 172.21.15.66:0/310644759 conn(0x55eaefcf0f10 msgr2=0x55eaefcf9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 -- 172.21.15.66:0/310644759 shutdown_connections 2024-03-30T00:47:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.250+0000 7fbad6e17000 1 -- 172.21.15.66:0/310644759 wait complete. 2024-03-30T00:47:13.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad6e17000 20 client.8115 trim_cache size 0 max 16384 2024-03-30T00:47:13.269 INFO:tasks.ceph.mds.a.smithi066.stderr:2024-03-30T00:47:13.254+0000 7f8a24800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-03-30T00:46:23.006182+0000 1=1+0)->f(v1 m2024-03-30T00:46:23.006182+0000) n(v0 rc2024-03-30T00:46:23.120156+0000 2=1+1)->n(v0 rc2024-03-30T00:46:23.120156+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55d4e38c3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-30T00:46:23.006182+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-30T00:46:23.120156+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-03-30T00:47:13.269 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:47:13.269 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T00:47:13.269 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "979d9d85-08ed-4117-b221-d31ad6e3987a", 2024-03-30T00:47:13.270 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T00:47:13.270 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:47:13.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 -- 172.21.15.66:0/756178595 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbaa8066690 msgr2=0x7fbaa8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbaa8066690 0x7fbaa8068b50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0b2000 tx=0x7fbabc008040 comp rx=0 tx=0).stop 2024-03-30T00:47:13.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 -- 172.21.15.66:0/756178595 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbacc099ca0 msgr2=0x7fbacc0c46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbacc099ca0 0x7fbacc0c46f0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fbab400d830 tx=0x7fbab400dcf0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 -- 172.21.15.66:0/756178595 shutdown_connections 2024-03-30T00:47:13.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbacc15e100 0x7fbacc0c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbaa8066690 0x7fbaa8068b50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbacc099ca0 0x7fbacc0c46f0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 --2- 172.21.15.66:0/756178595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbacc098f60 0x7fbacc159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 -- 172.21.15.66:0/756178595 >> 172.21.15.66:0/756178595 conn(0x7fbacc0b2140 msgr2=0x7fbacc0cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 -- 172.21.15.66:0/756178595 shutdown_connections 2024-03-30T00:47:13.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.254+0000 7fbad3a00640 1 -- 172.21.15.66:0/756178595 wait complete. 2024-03-30T00:47:13.282 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-30T00:47:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.414+0000 7f61c0600640 1 Processor -- start 2024-03-30T00:47:13.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.414+0000 7f61c0600640 1 -- start start 2024-03-30T00:47:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61c0600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f61b815a640 0x7f61b806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61c0600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f61b80ce520 0x7f61b80ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61c0600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b806d070 con 0x7f61b80cabb0 2024-03-30T00:47:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61c0600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b80cbba0 con 0x7f61b80ce520 2024-03-30T00:47:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61c0600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b80cee90 con 0x7f61b815a640 2024-03-30T00:47:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61bd200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f61b80ce520 0x7f61b80ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61bd200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f61b80ce520 0x7f61b80ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54048/0 (socket says 172.21.15.66:54048) 2024-03-30T00:47:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61bd200640 1 -- 172.21.15.66:0/2050861600 learned_addr learned my addr 172.21.15.66:0/2050861600 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61bdc00640 1 --2- 172.21.15.66:0/2050861600 >> v2:172.21.15.156:3300/0 conn(0x7f61b815a640 0x7f61b806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61bd200640 1 -- 172.21.15.66:0/2050861600 >> v2:172.21.15.156:3300/0 conn(0x7f61b815a640 msgr2=0x7f61b806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61bd200640 1 --2- 172.21.15.66:0/2050861600 >> v2:172.21.15.156:3300/0 conn(0x7f61b815a640 0x7f61b806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61bd200640 1 -- 172.21.15.66:0/2050861600 >> v1:172.21.15.156:6789/0 conn(0x7f61b80cabb0 legacy=0x7f61b80cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.418+0000 7f61bd200640 1 -- 172.21.15.66:0/2050861600 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b806e2c0 con 0x7f61b80ce520 2024-03-30T00:47:13.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61bd200640 1 --2- 172.21.15.66:0/2050861600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b80ce520 0x7f61b80ce950 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f61b80cf160 tx=0x7f61a401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19349763b2ac4a6a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61b7e00640 1 -- 172.21.15.66:0/2050861600 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a40ab610 con 0x7f61b80ce520 2024-03-30T00:47:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61b7e00640 1 -- 172.21.15.66:0/2050861600 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a40ab770 con 0x7f61b80ce520 2024-03-30T00:47:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61b7e00640 1 -- 172.21.15.66:0/2050861600 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a40abb00 con 0x7f61b80ce520 2024-03-30T00:47:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61c0600640 1 -- 172.21.15.66:0/2050861600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b80ce520 msgr2=0x7f61b80ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61c0600640 1 --2- 172.21.15.66:0/2050861600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b80ce520 0x7f61b80ce950 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f61b80cf160 tx=0x7f61a401c520 comp rx=0 tx=0).stop 2024-03-30T00:47:13.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61c0600640 1 -- 172.21.15.66:0/2050861600 shutdown_connections 2024-03-30T00:47:13.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61c0600640 1 --2- 172.21.15.66:0/2050861600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b80ce520 0x7f61b80ce950 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61c0600640 1 --2- 172.21.15.66:0/2050861600 >> v2:172.21.15.156:3300/0 conn(0x7f61b815a640 0x7f61b806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61c0600640 1 -- 172.21.15.66:0/2050861600 >> 172.21.15.66:0/2050861600 conn(0x7f61b8069f60 msgr2=0x7f61b806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61c0600640 1 -- 172.21.15.66:0/2050861600 shutdown_connections 2024-03-30T00:47:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61c0600640 1 -- 172.21.15.66:0/2050861600 wait complete. 2024-03-30T00:47:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61c0600640 1 Processor -- start 2024-03-30T00:47:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.422+0000 7f61c0600640 1 -- start start 2024-03-30T00:47:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61c0600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b80cabb0 0x7f61b8162510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61c0600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61b80ce520 0x7f61b8156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61c0600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b815a640 0x7f61b80cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61c0600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b8162a50 con 0x7f61b80cabb0 2024-03-30T00:47:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61c0600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b81592b0 con 0x7f61b815a640 2024-03-30T00:47:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61c0600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b8159590 con 0x7f61b80ce520 2024-03-30T00:47:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61bdc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b80cabb0 0x7f61b8162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61bdc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b80cabb0 0x7f61b8162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54060/0 (socket says 172.21.15.66:54060) 2024-03-30T00:47:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61bdc00640 1 -- 172.21.15.66:0/3195369570 learned_addr learned my addr 172.21.15.66:0/3195369570 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61bd200640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61b80ce520 0x7f61b8156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61be600640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b815a640 0x7f61b80cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61bdc00640 1 -- 172.21.15.66:0/3195369570 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61b80ce520 msgr2=0x7f61b8156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61bdc00640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61b80ce520 0x7f61b8156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61bdc00640 1 -- 172.21.15.66:0/3195369570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b815a640 msgr2=0x7f61b80cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61bdc00640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b815a640 0x7f61b80cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61bdc00640 1 -- 172.21.15.66:0/3195369570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a40094d0 con 0x7f61b80cabb0 2024-03-30T00:47:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61be600640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b815a640 0x7f61b80cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61bdc00640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b80cabb0 0x7f61b8162510 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e870 tx=0x7f61ac00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61b5600640 1 -- 172.21.15.66:0/3195369570 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac009510 con 0x7f61b80cabb0 2024-03-30T00:47:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61b5600640 1 -- 172.21.15.66:0/3195369570 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac00eec0 con 0x7f61b80cabb0 2024-03-30T00:47:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61b5600640 1 -- 172.21.15.66:0/3195369570 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00cb40 con 0x7f61b80cabb0 2024-03-30T00:47:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61c0600640 1 -- 172.21.15.66:0/3195369570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b8159870 con 0x7f61b80cabb0 2024-03-30T00:47:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61c0600640 1 -- 172.21.15.66:0/3195369570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b8156940 con 0x7f61b80cabb0 2024-03-30T00:47:13.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.426+0000 7f61b5600640 1 -- 172.21.15.66:0/3195369570 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f61ac00cca0 con 0x7f61b80cabb0 2024-03-30T00:47:13.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61b5600640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6198066690 0x7f6198068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61b5600640 1 -- 172.21.15.66:0/3195369570 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f61ac084490 con 0x7f61b80cabb0 2024-03-30T00:47:13.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bd200640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6198066690 0x7f6198068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61c3ae6000 1 Processor -- start 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61c3ae6000 1 -- start start 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bd200640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6198066690 0x7f6198068b50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f61a40af000 tx=0x7f61a40b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b5a70 0x564e9a0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 0x564e9a0694c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a069a00 0x564e9a069de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e99f34630 con 0x564e9a0b5a70 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e99ec1bd0 con 0x564e9a0b6390 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e99ebf1a0 con 0x564e9a069a00 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bdc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b5a70 0x564e9a0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bdc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b5a70 0x564e9a0b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54064/0 (socket says 172.21.15.66:54064) 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bdc00640 1 -- 172.21.15.66:0/1677255082 learned_addr learned my addr 172.21.15.66:0/1677255082 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bd200640 1 --2- 172.21.15.66:0/1677255082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 0x564e9a0694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61be600640 1 --2- 172.21.15.66:0/1677255082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a069a00 0x564e9a069de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bdc00640 1 -- 172.21.15.66:0/1677255082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a069a00 msgr2=0x564e9a069de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bdc00640 1 --2- 172.21.15.66:0/1677255082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a069a00 0x564e9a069de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bdc00640 1 -- 172.21.15.66:0/1677255082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 msgr2=0x564e9a0694c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bdc00640 1 --2- 172.21.15.66:0/1677255082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 0x564e9a0694c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bdc00640 1 -- 172.21.15.66:0/1677255082 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e99bb6e20 con 0x564e9a0b5a70 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bd200640 1 --2- 172.21.15.66:0/1677255082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 0x564e9a0694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61be600640 1 --2- 172.21.15.66:0/1677255082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a069a00 0x564e9a069de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61bdc00640 1 --2- 172.21.15.66:0/1677255082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b5a70 0x564e9a0b5e50 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f61ac05d0d0 tx=0x7f61ac054d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0147b8d415dab0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61aa000640 1 -- 172.21.15.66:0/1677255082 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac0519d0 con 0x564e9a0b5a70 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61aa000640 1 -- 172.21.15.66:0/1677255082 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac051b30 con 0x564e9a0b5a70 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61aa000640 1 -- 172.21.15.66:0/1677255082 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac0549d0 con 0x564e9a0b5a70 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1677255082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b5a70 msgr2=0x564e9a0b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.430+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/1677255082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b5a70 0x564e9a0b5e50 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f61ac05d0d0 tx=0x7f61ac054d50 comp rx=0 tx=0).stop 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1677255082 shutdown_connections 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/1677255082 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a069a00 0x564e9a069de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/1677255082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 0x564e9a0694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/1677255082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b5a70 0x564e9a0b5e50 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1677255082 >> 172.21.15.66:0/1677255082 conn(0x564e99e6a160 msgr2=0x564e9a098820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1677255082 shutdown_connections 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1677255082 wait complete. 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 -- start start 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 Processor -- start 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a069a00 0x564e9a0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b5a70 0x564e9a099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b6390 0x564e9a099f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e99ebaa90 con 0x564e9a0b6390 2024-03-30T00:47:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e99e59d40 con 0x564e9a069a00 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e99e77f30 con 0x564e9a0b5a70 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 10 client.0 _ll_register_callbacks cb 0x564e99f1c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61be600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b6390 0x564e9a099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61be600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b6390 0x564e9a099f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54076/0 (socket says 172.21.15.66:54076) 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61be600640 1 -- 172.21.15.66:0/223842839 learned_addr learned my addr 172.21.15.66:0/223842839 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61bdc00640 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a069a00 0x564e9a0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61bd200640 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b5a70 0x564e9a099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61be600640 1 -- 172.21.15.66:0/223842839 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b5a70 msgr2=0x564e9a099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61be600640 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b5a70 0x564e9a099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61be600640 1 -- 172.21.15.66:0/223842839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a069a00 msgr2=0x564e9a0727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61be600640 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a069a00 0x564e9a0727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61be600640 1 -- 172.21.15.66:0/223842839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b8156940 con 0x564e9a0b6390 2024-03-30T00:47:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61be600640 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b6390 0x564e9a099f90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f61b000f4f0 tx=0x7f61b000f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d3edf220074150 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f619ea00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f619ea00640 1 -- 172.21.15.66:0/223842839 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0003b90 con 0x564e9a0b6390 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f619ea00640 1 -- 172.21.15.66:0/223842839 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f61b0003cf0 con 0x564e9a0b6390 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 -- 172.21.15.66:0/223842839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e99eb93e0 con 0x564e9a0b6390 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f619ea00640 1 -- 172.21.15.66:0/223842839 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b00018e0 con 0x564e9a0b6390 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 10 client.8127 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 -- 172.21.15.66:0/223842839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x564e99e7faa0 con 0x564e9a0b6390 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f619ea00640 1 -- 172.21.15.66:0/223842839 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e99eb93e0 con 0x564e9a0b6390 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f619ea00640 1 -- 172.21.15.66:0/223842839 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f61b0012070 con 0x564e9a0b6390 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 10 client.8127 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 10 client.8127 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 20 client.8127 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x564e9a0bd2d0 0x564e9a0bd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 4 client.8127 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:47:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.434+0000 7f61c3ae6000 1 -- 172.21.15.66:0/223842839 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e99e8ad00 con 0x564e9a0bd2d0 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61bdc00640 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x564e9a0bd2d0 0x564e9a0bd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61bdc00640 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x564e9a0bd2d0 0x564e9a0bd6b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e876f8509378b95b server_cookie=f9fc9a0305d7e06d in_seq=0 out_seq=0 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f619ea00640 10 client.8127 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f619ea00640 1 -- 172.21.15.66:0/223842839 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564e99e8ad00 con 0x564e9a0bd2d0 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f619ea00640 10 client.8127 handle_command_reply: tid=0 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 1 client.8127 shutdown 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 1 -- 172.21.15.66:0/223842839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b6390 msgr2=0x564e9a099f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b6390 0x564e9a099f90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f61b000f4f0 tx=0x7f61b000f9b0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 1 -- 172.21.15.66:0/223842839 shutdown_connections 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x564e9a0bd2d0 0x564e9a0bd6b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a0b6390 0x564e9a099f90 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b5a70 0x564e9a099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/223842839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a069a00 0x564e9a0727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f619ea00640 0 client.8127 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 1 -- 172.21.15.66:0/223842839 >> 172.21.15.66:0/223842839 conn(0x564e99e6a160 msgr2=0x564e9a068bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 1 -- 172.21.15.66:0/223842839 shutdown_connections 2024-03-30T00:47:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 1 -- 172.21.15.66:0/223842839 wait complete. 2024-03-30T00:47:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.438+0000 7f61c3ae6000 20 client.8127 trim_cache size 0 max 16384 2024-03-30T00:47:13.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 Processor -- start 2024-03-30T00:47:13.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 -- start start 2024-03-30T00:47:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b5a70 0x564e9a071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b6390 0x564e9a069de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e99b77aa0 con 0x564e9a069a00 2024-03-30T00:47:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e99e7bc60 con 0x564e9a0b5a70 2024-03-30T00:47:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e9a099c80 con 0x564e9a0b6390 2024-03-30T00:47:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61bdc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61bdc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a071690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54088/0 (socket says 172.21.15.66:54088) 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61bdc00640 1 -- 172.21.15.66:0/3662789640 learned_addr learned my addr 172.21.15.66:0/3662789640 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61bd200640 1 --2- 172.21.15.66:0/3662789640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b5a70 0x564e9a071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61be600640 1 --2- 172.21.15.66:0/3662789640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b6390 0x564e9a069de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61bdc00640 1 -- 172.21.15.66:0/3662789640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b6390 msgr2=0x564e9a069de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61bdc00640 1 --2- 172.21.15.66:0/3662789640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b6390 0x564e9a069de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61bdc00640 1 -- 172.21.15.66:0/3662789640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b5a70 msgr2=0x564e9a071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61bdc00640 1 --2- 172.21.15.66:0/3662789640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b5a70 0x564e9a071bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61bdc00640 1 -- 172.21.15.66:0/3662789640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b80c9670 con 0x564e9a069a00 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61be600640 1 --2- 172.21.15.66:0/3662789640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b6390 0x564e9a069de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61bdc00640 1 --2- 172.21.15.66:0/3662789640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a071690 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f61ac053610 tx=0x7f61ac021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b83512b667f796b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61aaa00640 1 -- 172.21.15.66:0/3662789640 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac080070 con 0x564e9a069a00 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61aaa00640 1 -- 172.21.15.66:0/3662789640 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac05b0c0 con 0x564e9a069a00 2024-03-30T00:47:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61aaa00640 1 -- 172.21.15.66:0/3662789640 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac053780 con 0x564e9a069a00 2024-03-30T00:47:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 -- 172.21.15.66:0/3662789640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 msgr2=0x564e9a071690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/3662789640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a071690 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f61ac053610 tx=0x7f61ac021000 comp rx=0 tx=0).stop 2024-03-30T00:47:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 -- 172.21.15.66:0/3662789640 shutdown_connections 2024-03-30T00:47:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/3662789640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b6390 0x564e9a069de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/3662789640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b5a70 0x564e9a071bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/3662789640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a071690 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 -- 172.21.15.66:0/3662789640 >> 172.21.15.66:0/3662789640 conn(0x564e99e6a160 msgr2=0x564e9a070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 -- 172.21.15.66:0/3662789640 shutdown_connections 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 -- 172.21.15.66:0/3662789640 wait complete. 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.454+0000 7f61c3ae6000 1 -- start start 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 Processor -- start 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a0f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b5a70 0x564e9a06c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 0x564e9a06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9a09a220 con 0x564e9a069a00 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e99e84640 con 0x564e9a0b6390 2024-03-30T00:47:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e99e84ee0 con 0x564e9a0b5a70 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 10 client.0 _ll_register_callbacks cb 0x7f61ac055090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bdc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a0f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bdc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a0f8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54098/0 (socket says 172.21.15.66:54098) 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bdc00640 1 -- 172.21.15.66:0/1713721582 learned_addr learned my addr 172.21.15.66:0/1713721582 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bd200640 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b5a70 0x564e9a06c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61be600640 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 0x564e9a06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bdc00640 1 -- 172.21.15.66:0/1713721582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b5a70 msgr2=0x564e9a06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bdc00640 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b5a70 0x564e9a06c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bdc00640 1 -- 172.21.15.66:0/1713721582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 msgr2=0x564e9a06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bdc00640 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 0x564e9a06cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bdc00640 1 -- 172.21.15.66:0/1713721582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e99bb6e20 con 0x564e9a069a00 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61be600640 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 0x564e9a06cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bdc00640 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a0f8de0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00c520 tx=0x7f61ac0825e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ca396353d3ee38 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f619ea00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f619ea00640 1 -- 172.21.15.66:0/1713721582 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac080040 con 0x564e9a069a00 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f619ea00640 1 -- 172.21.15.66:0/1713721582 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f61ac0519d0 con 0x564e9a069a00 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1713721582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e9a09a360 con 0x564e9a069a00 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f619ea00640 1 -- 172.21.15.66:0/1713721582 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61ac05d170 con 0x564e9a069a00 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 10 client.8133 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1713721582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x564e99e65df0 con 0x564e9a069a00 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f619ea00640 1 -- 172.21.15.66:0/1713721582 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac07a020 con 0x564e9a069a00 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f619ea00640 1 -- 172.21.15.66:0/1713721582 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f61ac011070 con 0x564e9a069a00 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 10 client.8133 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 10 client.8133 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 20 client.8133 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x564e9a06d080 0x564e9a06d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 4 client.8133 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:47:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1713721582 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e99e77970 con 0x564e9a06d080 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bd200640 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x564e9a06d080 0x564e9a06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61bd200640 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x564e9a06d080 0x564e9a06d490 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a04c33a955189d server_cookie=873216ba361e15fc in_seq=0 out_seq=0 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f619ea00640 10 client.8133 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f619ea00640 1 -- 172.21.15.66:0/1713721582 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x564e99e77970 con 0x564e9a06d080 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f619ea00640 10 client.8133 handle_command_reply: tid=0 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 client.8133 shutdown 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1713721582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 msgr2=0x564e9a0f8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a0f8de0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00c520 tx=0x7f61ac0825e0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1713721582 shutdown_connections 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x564e9a06d080 0x564e9a06d490 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e9a0b6390 0x564e9a06cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564e9a0b5a70 0x564e9a06c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 --2- 172.21.15.66:0/1713721582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e9a069a00 0x564e9a0f8de0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f619ea00640 0 client.8133 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1713721582 >> 172.21.15.66:0/1713721582 conn(0x564e99e6a160 msgr2=0x564e9a071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1713721582 shutdown_connections 2024-03-30T00:47:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.458+0000 7f61c3ae6000 1 -- 172.21.15.66:0/1713721582 wait complete. 2024-03-30T00:47:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c3ae6000 20 client.8133 trim_cache size 0 max 16384 2024-03-30T00:47:13.473 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:47:13.473 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:47:13.474 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-03-30T00:47:13.474 INFO:teuthology.orchestra.run.smithi066.stdout: "979d9d85-08ed-4117-b221-d31ad6e3987a": { 2024-03-30T00:47:13.474 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/dir", 2024-03-30T00:47:13.474 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "979d9d85-08ed-4117-b221-d31ad6e3987a", 2024-03-30T00:47:13.474 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,repair" 2024-03-30T00:47:13.474 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:47:13.474 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:47:13.474 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 -- 172.21.15.66:0/3195369570 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6198066690 msgr2=0x7f6198068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6198066690 0x7f6198068b50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f61a40af000 tx=0x7f61a40b6040 comp rx=0 tx=0).stop 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 -- 172.21.15.66:0/3195369570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b80cabb0 msgr2=0x7f61b8162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b80cabb0 0x7f61b8162510 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e870 tx=0x7f61ac00ec70 comp rx=0 tx=0).stop 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 -- 172.21.15.66:0/3195369570 shutdown_connections 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b815a640 0x7f61b80cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61b80ce520 0x7f61b8156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6198066690 0x7f6198068b50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 --2- 172.21.15.66:0/3195369570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b80cabb0 0x7f61b8162510 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 -- 172.21.15.66:0/3195369570 >> 172.21.15.66:0/3195369570 conn(0x7f61b8069f60 msgr2=0x7f61b806a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 -- 172.21.15.66:0/3195369570 shutdown_connections 2024-03-30T00:47:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:13.462+0000 7f61c0600640 1 -- 172.21.15.66:0/3195369570 wait complete. 2024-03-30T00:47:13.488 INFO:tasks.cephfs.filesystem:scrub status for tag:979d9d85-08ed-4117-b221-d31ad6e3987a - {'path': '/dir', 'tag': '979d9d85-08ed-4117-b221-d31ad6e3987a', 'options': 'recursive,repair'} 2024-03-30T00:47:16.988 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:47:16.992 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:47:43.489 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-30T00:47:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe941c00640 1 Processor -- start 2024-03-30T00:47:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe941c00640 1 -- start start 2024-03-30T00:47:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe941c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe93c157060 0x7fe93c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe941c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe93c160bc0 0x7fe93c0abc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe941c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe93c06dc60 con 0x7fe93c1610c0 2024-03-30T00:47:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe941c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe93c157440 con 0x7fe93c157060 2024-03-30T00:47:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe941c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c06df40 con 0x7fe93c160bc0 2024-03-30T00:47:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe93aa00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe93c157060 0x7fe93c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe93b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe93c160bc0 0x7fe93c0abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe93aa00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe93c157060 0x7fe93c06d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:38012/0 (socket says 172.21.15.66:38012) 2024-03-30T00:47:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe93aa00640 1 -- 172.21.15.66:0/2522792408 learned_addr learned my addr 172.21.15.66:0/2522792408 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe93b400640 1 -- 172.21.15.66:0/2522792408 >> v2:172.21.15.156:3300/0 conn(0x7fe93c157060 msgr2=0x7fe93c06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe93b400640 1 --2- 172.21.15.66:0/2522792408 >> v2:172.21.15.156:3300/0 conn(0x7fe93c157060 0x7fe93c06d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe93b400640 1 -- 172.21.15.66:0/2522792408 >> v1:172.21.15.156:6789/0 conn(0x7fe93c1610c0 legacy=0x7fe93c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe93b400640 1 -- 172.21.15.66:0/2522792408 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c06e220 con 0x7fe93c160bc0 2024-03-30T00:47:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.637+0000 7fe93aa00640 1 --2- 172.21.15.66:0/2522792408 >> v2:172.21.15.156:3300/0 conn(0x7fe93c157060 0x7fe93c06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:47:43.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe93b400640 1 --2- 172.21.15.66:0/2522792408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe93c160bc0 0x7fe93c0abc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe934009d90 tx=0x7fe93401c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa45973ed4856f69 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:43.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe93a000640 1 -- 172.21.15.66:0/2522792408 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c06da60 con 0x7fe93c160bc0 2024-03-30T00:47:43.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe93a000640 1 -- 172.21.15.66:0/2522792408 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9340a5820 con 0x7fe93c160bc0 2024-03-30T00:47:43.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe93a000640 1 -- 172.21.15.66:0/2522792408 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340a5bb0 con 0x7fe93c160bc0 2024-03-30T00:47:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe941c00640 1 -- 172.21.15.66:0/2522792408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe93c160bc0 msgr2=0x7fe93c0abc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe941c00640 1 --2- 172.21.15.66:0/2522792408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe93c160bc0 0x7fe93c0abc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe934009d90 tx=0x7fe93401c850 comp rx=0 tx=0).stop 2024-03-30T00:47:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe941c00640 1 -- 172.21.15.66:0/2522792408 shutdown_connections 2024-03-30T00:47:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe941c00640 1 --2- 172.21.15.66:0/2522792408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe93c160bc0 0x7fe93c0abc00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe941c00640 1 --2- 172.21.15.66:0/2522792408 >> v2:172.21.15.156:3300/0 conn(0x7fe93c157060 0x7fe93c06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe941c00640 1 -- 172.21.15.66:0/2522792408 >> 172.21.15.66:0/2522792408 conn(0x7fe93c071750 msgr2=0x7fe93c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe941c00640 1 -- 172.21.15.66:0/2522792408 shutdown_connections 2024-03-30T00:47:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.641+0000 7fe941c00640 1 -- 172.21.15.66:0/2522792408 wait complete. 2024-03-30T00:47:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe941c00640 1 Processor -- start 2024-03-30T00:47:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe941c00640 1 -- start start 2024-03-30T00:47:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe941c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c157060 0x7fe93c14f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe941c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c160bc0 0x7fe93c14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe941c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe93c1610c0 0x7fe93c14dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe941c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c0b3910 con 0x7fe93c160bc0 2024-03-30T00:47:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe941c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c14de40 con 0x7fe93c157060 2024-03-30T00:47:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe941c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c0b3290 con 0x7fe93c1610c0 2024-03-30T00:47:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe933e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c160bc0 0x7fe93c14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe933e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c160bc0 0x7fe93c14d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55284/0 (socket says 172.21.15.66:55284) 2024-03-30T00:47:43.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe933e00640 1 -- 172.21.15.66:0/24068475 learned_addr learned my addr 172.21.15.66:0/24068475 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:43.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe93aa00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c157060 0x7fe93c14f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe93b400640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe93c1610c0 0x7fe93c14dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe933e00640 1 -- 172.21.15.66:0/24068475 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe93c1610c0 msgr2=0x7fe93c14dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe933e00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe93c1610c0 0x7fe93c14dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe933e00640 1 -- 172.21.15.66:0/24068475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c157060 msgr2=0x7fe93c14f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe933e00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c157060 0x7fe93c14f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe933e00640 1 -- 172.21.15.66:0/24068475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c0094d0 con 0x7fe93c160bc0 2024-03-30T00:47:43.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe933e00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c160bc0 0x7fe93c14d900 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe92400c4d0 tx=0x7fe92400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:43.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe932a00640 1 -- 172.21.15.66:0/24068475 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9240052d0 con 0x7fe93c160bc0 2024-03-30T00:47:43.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe932a00640 1 -- 172.21.15.66:0/24068475 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe924005430 con 0x7fe93c160bc0 2024-03-30T00:47:43.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe932a00640 1 -- 172.21.15.66:0/24068475 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924012830 con 0x7fe93c160bc0 2024-03-30T00:47:43.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe941c00640 1 -- 172.21.15.66:0/24068475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe934009a70 con 0x7fe93c160bc0 2024-03-30T00:47:43.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.645+0000 7fe941c00640 1 -- 172.21.15.66:0/24068475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93c0b3530 con 0x7fe93c160bc0 2024-03-30T00:47:43.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe932a00640 1 -- 172.21.15.66:0/24068475 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe924012990 con 0x7fe93c160bc0 2024-03-30T00:47:43.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe932a00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9180666c0 0x7fe918068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe932a00640 1 -- 172.21.15.66:0/24068475 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fe924083da0 con 0x7fe93c160bc0 2024-03-30T00:47:43.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe93aa00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9180666c0 0x7fe918068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe944fa9000 1 Processor -- start 2024-03-30T00:47:43.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe944fa9000 1 -- start start 2024-03-30T00:47:43.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe93aa00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9180666c0 0x7fe918068b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0b0730 tx=0x7fe92c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:43.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abfc5a70 0x55b7abfc5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abf79a00 0x55b7abf79de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe944fa9000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7abe44630 con 0x55b7abfc6390 2024-03-30T00:47:43.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe944fa9000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7abdd1bd0 con 0x55b7abfc5a70 2024-03-30T00:47:43.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe944fa9000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7abdcf1a0 con 0x55b7abf79a00 2024-03-30T00:47:43.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe933e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe933e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf794c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55296/0 (socket says 172.21.15.66:55296) 2024-03-30T00:47:43.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe933e00640 1 -- 172.21.15.66:0/2042155711 learned_addr learned my addr 172.21.15.66:0/2042155711 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:43.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe93aa00640 1 --2- 172.21.15.66:0/2042155711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abfc5a70 0x55b7abfc5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.649+0000 7fe93b400640 1 --2- 172.21.15.66:0/2042155711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abf79a00 0x55b7abf79de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe933e00640 1 -- 172.21.15.66:0/2042155711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abf79a00 msgr2=0x55b7abf79de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe933e00640 1 --2- 172.21.15.66:0/2042155711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abf79a00 0x55b7abf79de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe933e00640 1 -- 172.21.15.66:0/2042155711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abfc5a70 msgr2=0x55b7abfc5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe933e00640 1 --2- 172.21.15.66:0/2042155711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abfc5a70 0x55b7abfc5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe933e00640 1 -- 172.21.15.66:0/2042155711 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7abac6e20 con 0x55b7abfc6390 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe933e00640 1 --2- 172.21.15.66:0/2042155711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf794c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe924081f90 tx=0x7fe92400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c8186892553f18 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe92b400640 1 -- 172.21.15.66:0/2042155711 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924052f70 con 0x55b7abfc6390 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe92b400640 1 -- 172.21.15.66:0/2042155711 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9240512e0 con 0x55b7abfc6390 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe92b400640 1 -- 172.21.15.66:0/2042155711 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924012e80 con 0x55b7abfc6390 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 -- 172.21.15.66:0/2042155711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 msgr2=0x55b7abf794c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 --2- 172.21.15.66:0/2042155711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf794c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe924081f90 tx=0x7fe92400f000 comp rx=0 tx=0).stop 2024-03-30T00:47:43.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 -- 172.21.15.66:0/2042155711 shutdown_connections 2024-03-30T00:47:43.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 --2- 172.21.15.66:0/2042155711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abf79a00 0x55b7abf79de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 --2- 172.21.15.66:0/2042155711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf794c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 --2- 172.21.15.66:0/2042155711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abfc5a70 0x55b7abfc5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 -- 172.21.15.66:0/2042155711 >> 172.21.15.66:0/2042155711 conn(0x55b7abd7a160 msgr2=0x55b7abfa8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:43.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 -- 172.21.15.66:0/2042155711 shutdown_connections 2024-03-30T00:47:43.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 -- 172.21.15.66:0/2042155711 wait complete. 2024-03-30T00:47:43.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 -- start start 2024-03-30T00:47:43.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 Processor -- start 2024-03-30T00:47:43.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abf79a00 0x55b7abf827b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abfc5a70 0x55b7abfa9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc6390 0x55b7abfa9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7abdcaa90 con 0x55b7abf79a00 2024-03-30T00:47:43.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7abd69d40 con 0x55b7abfc5a70 2024-03-30T00:47:43.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7abd87f30 con 0x55b7abfc6390 2024-03-30T00:47:43.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe944fa9000 10 client.0 _ll_register_callbacks cb 0x55b7abe2c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe93aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abf79a00 0x55b7abf827b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe93aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abf79a00 0x55b7abf827b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55300/0 (socket says 172.21.15.66:55300) 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe93aa00640 1 -- 172.21.15.66:0/915963932 learned_addr learned my addr 172.21.15.66:0/915963932 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe933e00640 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abfc5a70 0x55b7abfa9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe93b400640 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc6390 0x55b7abfa9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe93aa00640 1 -- 172.21.15.66:0/915963932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc6390 msgr2=0x55b7abfa9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe93aa00640 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc6390 0x55b7abfa9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe93aa00640 1 -- 172.21.15.66:0/915963932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abfc5a70 msgr2=0x55b7abfa9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe93aa00640 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abfc5a70 0x55b7abfa9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.653+0000 7fe93aa00640 1 -- 172.21.15.66:0/915963932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93c0b3530 con 0x55b7abf79a00 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe93aa00640 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abf79a00 0x55b7abf827b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe92c014b10 tx=0x7fe92c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef6c5b55a83c7a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe91fe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe91fe00640 1 -- 172.21.15.66:0/915963932 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0138b0 con 0x55b7abf79a00 2024-03-30T00:47:43.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe91fe00640 1 -- 172.21.15.66:0/915963932 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fe92c01c040 con 0x55b7abf79a00 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 -- 172.21.15.66:0/915963932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7abdc93e0 con 0x55b7abf79a00 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe91fe00640 1 -- 172.21.15.66:0/915963932 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe92c013a10 con 0x55b7abf79a00 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 10 client.8142 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 -- 172.21.15.66:0/915963932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55b7abd8faa0 con 0x55b7abf79a00 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe91fe00640 1 -- 172.21.15.66:0/915963932 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b7abdc93e0 con 0x55b7abf79a00 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe91fe00640 1 -- 172.21.15.66:0/915963932 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe92c013b70 con 0x55b7abf79a00 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 10 client.8142 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 10 client.8142 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 20 client.8142 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55b7abfcd2d0 0x55b7abfcd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 4 client.8142 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 -- 172.21.15.66:0/915963932 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7abd9ad00 con 0x55b7abfcd2d0 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe933e00640 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55b7abfcd2d0 0x55b7abfcd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe933e00640 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55b7abfcd2d0 0x55b7abfcd6e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8b466eab4d2cb95 server_cookie=385aa9dbaf216ab3 in_seq=0 out_seq=0 2024-03-30T00:47:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe91fe00640 10 client.8142 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe91fe00640 1 -- 172.21.15.66:0/915963932 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b7abd9ad00 con 0x55b7abfcd2d0 2024-03-30T00:47:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe91fe00640 10 client.8142 handle_command_reply: tid=0 2024-03-30T00:47:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 client.8142 shutdown 2024-03-30T00:47:43.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 -- 172.21.15.66:0/915963932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abf79a00 msgr2=0x55b7abf827b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abf79a00 0x55b7abf827b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe92c014b10 tx=0x7fe92c0046d0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 -- 172.21.15.66:0/915963932 shutdown_connections 2024-03-30T00:47:43.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55b7abfcd2d0 0x55b7abfcd6e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc6390 0x55b7abfa9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abfc5a70 0x55b7abfa9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 --2- 172.21.15.66:0/915963932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abf79a00 0x55b7abf827b0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe91fe00640 0 client.8142 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:43.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 -- 172.21.15.66:0/915963932 >> 172.21.15.66:0/915963932 conn(0x55b7abd7a160 msgr2=0x55b7abf78bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:43.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 -- 172.21.15.66:0/915963932 shutdown_connections 2024-03-30T00:47:43.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 1 -- 172.21.15.66:0/915963932 wait complete. 2024-03-30T00:47:43.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.657+0000 7fe944fa9000 20 client.8142 trim_cache size 0 max 16384 2024-03-30T00:47:43.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe944fa9000 1 Processor -- start 2024-03-30T00:47:43.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe944fa9000 1 -- start start 2024-03-30T00:47:43.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abf79a00 0x55b7abf78c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc5a70 0x55b7abf79170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf79de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe944fa9000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7aba87aa0 con 0x55b7abfc6390 2024-03-30T00:47:43.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe944fa9000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7abdcea90 con 0x55b7abf79a00 2024-03-30T00:47:43.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe944fa9000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7abdc6280 con 0x55b7abfc5a70 2024-03-30T00:47:43.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe93b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf79de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe93b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf79de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55308/0 (socket says 172.21.15.66:55308) 2024-03-30T00:47:43.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe93b400640 1 -- 172.21.15.66:0/3535254072 learned_addr learned my addr 172.21.15.66:0/3535254072 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:43.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe93aa00640 1 --2- 172.21.15.66:0/3535254072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abf79a00 0x55b7abf78c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe933e00640 1 --2- 172.21.15.66:0/3535254072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc5a70 0x55b7abf79170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe93b400640 1 -- 172.21.15.66:0/3535254072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc5a70 msgr2=0x55b7abf79170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe93b400640 1 --2- 172.21.15.66:0/3535254072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc5a70 0x55b7abf79170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe93b400640 1 -- 172.21.15.66:0/3535254072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abf79a00 msgr2=0x55b7abf78c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe93b400640 1 --2- 172.21.15.66:0/3535254072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abf79a00 0x55b7abf78c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe93b400640 1 -- 172.21.15.66:0/3535254072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7abd75df0 con 0x55b7abfc6390 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe93aa00640 1 --2- 172.21.15.66:0/3535254072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abf79a00 0x55b7abf78c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.673+0000 7fe93b400640 1 --2- 172.21.15.66:0/3535254072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf79de0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe934009d90 tx=0x7fe9340a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9141e5643570db72 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe92be00640 1 -- 172.21.15.66:0/3535254072 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340bd040 con 0x55b7abfc6390 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe92be00640 1 -- 172.21.15.66:0/3535254072 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9340a5df0 con 0x55b7abfc6390 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe92be00640 1 -- 172.21.15.66:0/3535254072 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340ad640 con 0x55b7abfc6390 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 -- 172.21.15.66:0/3535254072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 msgr2=0x55b7abf79de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 --2- 172.21.15.66:0/3535254072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf79de0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe934009d90 tx=0x7fe9340a7070 comp rx=0 tx=0).stop 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 -- 172.21.15.66:0/3535254072 shutdown_connections 2024-03-30T00:47:43.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 --2- 172.21.15.66:0/3535254072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7abf79de0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 --2- 172.21.15.66:0/3535254072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc5a70 0x55b7abf79170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 --2- 172.21.15.66:0/3535254072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abf79a00 0x55b7abf78c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 -- 172.21.15.66:0/3535254072 >> 172.21.15.66:0/3535254072 conn(0x55b7abd7a160 msgr2=0x55b7abf81810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:43.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 -- 172.21.15.66:0/3535254072 shutdown_connections 2024-03-30T00:47:43.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 -- 172.21.15.66:0/3535254072 wait complete. 2024-03-30T00:47:43.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 -- start start 2024-03-30T00:47:43.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 Processor -- start 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abf79a00 0x55b7abf7cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc5a70 0x55b7ac0252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7ac025820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7abe0e170 con 0x55b7abfc6390 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7abd60a90 con 0x55b7abf79a00 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7abf73930 con 0x55b7abfc5a70 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 10 client.0 _ll_register_callbacks cb 0x7fe924050f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe93b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7ac025820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe93b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7ac025820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55320/0 (socket says 172.21.15.66:55320) 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe93b400640 1 -- 172.21.15.66:0/4061450499 learned_addr learned my addr 172.21.15.66:0/4061450499 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe93aa00640 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abf79a00 0x55b7abf7cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe933e00640 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc5a70 0x55b7ac0252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe93b400640 1 -- 172.21.15.66:0/4061450499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc5a70 msgr2=0x55b7ac0252e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe93b400640 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc5a70 0x55b7ac0252e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe93b400640 1 -- 172.21.15.66:0/4061450499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abf79a00 msgr2=0x55b7abf7cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe93b400640 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abf79a00 0x55b7abf7cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe93b400640 1 -- 172.21.15.66:0/4061450499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93c06e220 con 0x55b7abfc6390 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe933e00640 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc5a70 0x55b7ac0252e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe93b400640 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7ac025820 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe93401a480 tx=0x7fe93401b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3f18dd5ffccb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe91fe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe91fe00640 1 -- 172.21.15.66:0/4061450499 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934004140 con 0x55b7abfc6390 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe91fe00640 1 -- 172.21.15.66:0/4061450499 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fe9340cdcc0 con 0x55b7abfc6390 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.677+0000 7fe944fa9000 1 -- 172.21.15.66:0/4061450499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7abd87970 con 0x55b7abfc6390 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe91fe00640 1 -- 172.21.15.66:0/4061450499 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9340042a0 con 0x55b7abfc6390 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 10 client.8148 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 -- 172.21.15.66:0/4061450499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55b7abd69960 con 0x55b7abfc6390 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe91fe00640 1 -- 172.21.15.66:0/4061450499 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b7abd87970 con 0x55b7abfc6390 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe91fe00640 1 -- 172.21.15.66:0/4061450499 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe9340ca070 con 0x55b7abfc6390 2024-03-30T00:47:43.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 10 client.8148 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 10 client.8148 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 20 client.8148 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55b7abfe11e0 0x55b7abfe15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 4 client.8148 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 -- 172.21.15.66:0/4061450499 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b7abdf9a20 con 0x55b7abfe11e0 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe93aa00640 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55b7abfe11e0 0x55b7abfe15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe93aa00640 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55b7abfe11e0 0x55b7abfe15f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d4e19a32fc54cb server_cookie=e68a556bedb6b3d5 in_seq=0 out_seq=0 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe91fe00640 10 client.8148 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe91fe00640 1 -- 172.21.15.66:0/4061450499 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b7abdf9a20 con 0x55b7abfe11e0 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe91fe00640 10 client.8148 handle_command_reply: tid=0 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 client.8148 shutdown 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 -- 172.21.15.66:0/4061450499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 msgr2=0x55b7ac025820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7ac025820 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe93401a480 tx=0x7fe93401b090 comp rx=0 tx=0).stop 2024-03-30T00:47:43.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 -- 172.21.15.66:0/4061450499 shutdown_connections 2024-03-30T00:47:43.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x55b7abfe11e0 0x55b7abfe15f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7abfc6390 0x55b7ac025820 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b7abfc5a70 0x55b7ac0252e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 --2- 172.21.15.66:0/4061450499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7abf79a00 0x55b7abf7cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 -- 172.21.15.66:0/4061450499 >> 172.21.15.66:0/4061450499 conn(0x55b7abd7a160 msgr2=0x55b7abf77c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:43.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe91fe00640 0 client.8148 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:43.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 -- 172.21.15.66:0/4061450499 shutdown_connections 2024-03-30T00:47:43.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 1 -- 172.21.15.66:0/4061450499 wait complete. 2024-03-30T00:47:43.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe944fa9000 20 client.8148 trim_cache size 0 max 16384 2024-03-30T00:47:43.694 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:47:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T00:47:43.695 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-03-30T00:47:43.695 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:47:43.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 -- 172.21.15.66:0/24068475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9180666c0 msgr2=0x7fe918068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9180666c0 0x7fe918068b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0b0730 tx=0x7fe92c009240 comp rx=0 tx=0).stop 2024-03-30T00:47:43.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 -- 172.21.15.66:0/24068475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c160bc0 msgr2=0x7fe93c14d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c160bc0 0x7fe93c14d900 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe92400c4d0 tx=0x7fe92400c990 comp rx=0 tx=0).stop 2024-03-30T00:47:43.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 -- 172.21.15.66:0/24068475 shutdown_connections 2024-03-30T00:47:43.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe93c1610c0 0x7fe93c14dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9180666c0 0x7fe918068b80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe93c160bc0 0x7fe93c14d900 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 --2- 172.21.15.66:0/24068475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c157060 0x7fe93c14f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 -- 172.21.15.66:0/24068475 >> 172.21.15.66:0/24068475 conn(0x7fe93c071750 msgr2=0x7fe93c15b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:43.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 -- 172.21.15.66:0/24068475 shutdown_connections 2024-03-30T00:47:43.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.681+0000 7fe941c00640 1 -- 172.21.15.66:0/24068475 wait complete. 2024-03-30T00:47:43.712 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:47:43.712 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:47:43.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.841+0000 7fec3de00640 1 Processor -- start 2024-03-30T00:47:43.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.841+0000 7fec3de00640 1 -- start start 2024-03-30T00:47:43.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec3de00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fec380cabb0 0x7fec380cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec3de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3806d070 con 0x7fec3815a640 2024-03-30T00:47:43.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec3de00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec380cbba0 con 0x7fec380ce520 2024-03-30T00:47:43.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec3de00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec380cb8c0 con 0x7fec380cabb0 2024-03-30T00:47:43.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fec3815a640 0x7fec3806dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55320/0 (socket says 172.21.15.66:55320) 2024-03-30T00:47:43.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36a00640 1 -- 172.21.15.66:0/3731216306 learned_addr learned my addr 172.21.15.66:0/3731216306 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:43.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec37400640 1 --2- 172.21.15.66:0/3731216306 >> v2:172.21.15.156:3300/0 conn(0x7fec380cabb0 0x7fec380cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873010200 0 0) 0x7fec3806d070 con 0x7fec3815a640 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec1c003530 con 0x7fec3815a640 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 365740191 0 0) 0x7fec1c003530 con 0x7fec3815a640 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 >> v2:172.21.15.156:3300/0 conn(0x7fec380cabb0 msgr2=0x7fec380cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 --2- 172.21.15.66:0/3731216306 >> v2:172.21.15.156:3300/0 conn(0x7fec380cabb0 0x7fec380cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 >> v1:172.21.15.156:6789/0 conn(0x7fec380ce520 legacy=0x7fec380ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3806d2a0 con 0x7fec3815a640 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fec28001680 con 0x7fec3815a640 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 >> v1:172.21.15.66:6789/0 conn(0x7fec3815a640 legacy=0x7fec3806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 --2- 172.21.15.66:0/3731216306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec1c003980 0x7fec1c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 --2- 172.21.15.66:0/3731216306 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec1c0047d0 0x7fec1c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 --2- 172.21.15.66:0/3731216306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec1c005190 0x7fec1c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380cb8c0 con 0x7fec1c003980 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c00c030 con 0x7fec1c005190 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c00c1f0 con 0x7fec1c0047d0 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36a00640 1 --2- 172.21.15.66:0/3731216306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec1c003980 0x7fec1c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec2fe00640 1 --2- 172.21.15.66:0/3731216306 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec1c0047d0 0x7fec1c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec37400640 1 --2- 172.21.15.66:0/3731216306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec1c005190 0x7fec1c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36a00640 1 -- 172.21.15.66:0/3731216306 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec1c0047d0 msgr2=0x7fec1c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36a00640 1 --2- 172.21.15.66:0/3731216306 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec1c0047d0 0x7fec1c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36a00640 1 -- 172.21.15.66:0/3731216306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec1c005190 msgr2=0x7fec1c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36a00640 1 --2- 172.21.15.66:0/3731216306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec1c005190 0x7fec1c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec37400640 1 -- 172.21.15.66:0/3731216306 reap_dead start 2024-03-30T00:47:43.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.845+0000 7fec36a00640 1 -- 172.21.15.66:0/3731216306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c003530 con 0x7fec1c003980 2024-03-30T00:47:43.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec36a00640 1 --2- 172.21.15.66:0/3731216306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec1c003980 0x7fec1c003da0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fec2800c240 tx=0x7fec2801cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1bebe0d5c8bba9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:43.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28005ae0 con 0x7fec1c003980 2024-03-30T00:47:43.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec28005c40 con 0x7fec1c003980 2024-03-30T00:47:43.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec36000640 1 -- 172.21.15.66:0/3731216306 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec280a5ad0 con 0x7fec1c003980 2024-03-30T00:47:43.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec3de00640 1 -- 172.21.15.66:0/3731216306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec1c003980 msgr2=0x7fec1c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec3de00640 1 --2- 172.21.15.66:0/3731216306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec1c003980 0x7fec1c003da0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fec2800c240 tx=0x7fec2801cbb0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec3de00640 1 -- 172.21.15.66:0/3731216306 shutdown_connections 2024-03-30T00:47:43.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec3de00640 1 --2- 172.21.15.66:0/3731216306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec1c003980 0x7fec1c003da0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec3de00640 1 -- 172.21.15.66:0/3731216306 >> 172.21.15.66:0/3731216306 conn(0x7fec38069f60 msgr2=0x7fec3806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:43.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec3de00640 1 -- 172.21.15.66:0/3731216306 shutdown_connections 2024-03-30T00:47:43.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec3de00640 1 -- 172.21.15.66:0/3731216306 wait complete. 2024-03-30T00:47:43.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec3de00640 1 Processor -- start 2024-03-30T00:47:43.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.849+0000 7fec3de00640 1 -- start start 2024-03-30T00:47:43.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec3de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec1c003980 0x7fec380c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec3de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec380cc870 0x7fec3815d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec3de00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec380c5b20 0x7fec380c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec3de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380c8be0 con 0x7fec380cc870 2024-03-30T00:47:43.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec3de00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380c8460 con 0x7fec380c5b20 2024-03-30T00:47:43.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec3de00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec380c64e0 con 0x7fec1c003980 2024-03-30T00:47:43.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec380cc870 0x7fec3815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec380cc870 0x7fec3815d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55338/0 (socket says 172.21.15.66:55338) 2024-03-30T00:47:43.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2fe00640 1 -- 172.21.15.66:0/1638171715 learned_addr learned my addr 172.21.15.66:0/1638171715 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:43.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec36a00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec1c003980 0x7fec380c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec37400640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec380c5b20 0x7fec380c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2fe00640 1 -- 172.21.15.66:0/1638171715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec1c003980 msgr2=0x7fec380c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2fe00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec1c003980 0x7fec380c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2fe00640 1 -- 172.21.15.66:0/1638171715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec380c5b20 msgr2=0x7fec380c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:43.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2fe00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec380c5b20 0x7fec380c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:43.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2fe00640 1 -- 172.21.15.66:0/1638171715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec380cd000 con 0x7fec380cc870 2024-03-30T00:47:43.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2fe00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec380cc870 0x7fec3815d180 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fec2000f770 tx=0x7fec2000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:43.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2f400640 1 -- 172.21.15.66:0/1638171715 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20001350 con 0x7fec380cc870 2024-03-30T00:47:43.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2f400640 1 -- 172.21.15.66:0/1638171715 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec200050d0 con 0x7fec380cc870 2024-03-30T00:47:43.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec3de00640 1 -- 172.21.15.66:0/1638171715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec380cd2e0 con 0x7fec380cc870 2024-03-30T00:47:43.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2f400640 1 -- 172.21.15.66:0/1638171715 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20005480 con 0x7fec380cc870 2024-03-30T00:47:43.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec3de00640 1 -- 172.21.15.66:0/1638171715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec380cccd0 con 0x7fec380cc870 2024-03-30T00:47:43.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec3de00640 1 -- 172.21.15.66:0/1638171715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec380c8940 con 0x7fec380cc870 2024-03-30T00:47:43.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.853+0000 7fec2f400640 1 -- 172.21.15.66:0/1638171715 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fec2000d730 con 0x7fec380cc870 2024-03-30T00:47:43.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.857+0000 7fec2f400640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec14066690 0x7fec14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:43.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.857+0000 7fec2f400640 1 -- 172.21.15.66:0/1638171715 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fec20084240 con 0x7fec380cc870 2024-03-30T00:47:43.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.857+0000 7fec36a00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec14066690 0x7fec14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:43.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.857+0000 7fec36a00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec14066690 0x7fec14068b50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fec280ab000 tx=0x7fec280a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:43.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:43.861+0000 7fec2f400640 1 -- 172.21.15.66:0/1638171715 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fec380c8940 con 0x7fec380cc870 2024-03-30T00:47:44.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.053+0000 7fec3de00640 1 -- 172.21.15.66:0/1638171715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec380c8940 con 0x7fec380cc870 2024-03-30T00:47:44.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.053+0000 7fec2f400640 1 -- 172.21.15.66:0/1638171715 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fec380c8940 con 0x7fec380cc870 2024-03-30T00:47:44.063 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:47:44.063 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:46:45.565028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6836/1626022080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1626022080},{"type":"v1","addr":"172.21.15.66:6836","nonce":1626022080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:47:44.064 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 65 2024-03-30T00:47:44.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 -- 172.21.15.66:0/1638171715 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec14066690 msgr2=0x7fec14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec14066690 0x7fec14068b50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fec280ab000 tx=0x7fec280a7070 comp rx=0 tx=0).stop 2024-03-30T00:47:44.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 -- 172.21.15.66:0/1638171715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec380cc870 msgr2=0x7fec3815d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec380cc870 0x7fec3815d180 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fec2000f770 tx=0x7fec2000fc30 comp rx=0 tx=0).stop 2024-03-30T00:47:44.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 -- 172.21.15.66:0/1638171715 shutdown_connections 2024-03-30T00:47:44.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec380c5b20 0x7fec380c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fec14066690 0x7fec14068b50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec380cc870 0x7fec3815d180 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 --2- 172.21.15.66:0/1638171715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec1c003980 0x7fec380c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 -- 172.21.15.66:0/1638171715 >> 172.21.15.66:0/1638171715 conn(0x7fec38069f60 msgr2=0x7fec380abce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:44.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 -- 172.21.15.66:0/1638171715 shutdown_connections 2024-03-30T00:47:44.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.057+0000 7fec3de00640 1 -- 172.21.15.66:0/1638171715 wait complete. 2024-03-30T00:47:44.084 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-03-30T00:47:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f8671e00640 1 Processor -- start 2024-03-30T00:47:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f8671e00640 1 -- start start 2024-03-30T00:47:44.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f8671e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f866c15a820 0x7f866c0b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f8671e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f866c06a4a0 0x7f866c164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f8671e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c0cba40 con 0x7f866c0b0700 2024-03-30T00:47:44.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f8671e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866c0cc6c0 con 0x7f866c15a820 2024-03-30T00:47:44.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f8671e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866c0b0b90 con 0x7f866c06a4a0 2024-03-30T00:47:44.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f866aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f866c15a820 0x7f866c0b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f866aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f866c15a820 0x7f866c0b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55348/0 (socket says 172.21.15.66:55348) 2024-03-30T00:47:44.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f866aa00640 1 -- 172.21.15.66:0/3341220633 learned_addr learned my addr 172.21.15.66:0/3341220633 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:44.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f866b400640 1 --2- 172.21.15.66:0/3341220633 >> v2:172.21.15.156:3300/0 conn(0x7f866c06a4a0 0x7f866c164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f866aa00640 1 -- 172.21.15.66:0/3341220633 >> v2:172.21.15.156:3300/0 conn(0x7f866c06a4a0 msgr2=0x7f866c164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:44.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f866aa00640 1 --2- 172.21.15.66:0/3341220633 >> v2:172.21.15.156:3300/0 conn(0x7f866c06a4a0 0x7f866c164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f866aa00640 1 -- 172.21.15.66:0/3341220633 >> v1:172.21.15.156:6789/0 conn(0x7f866c0b0700 legacy=0x7f866c069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.217+0000 7f866aa00640 1 -- 172.21.15.66:0/3341220633 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0cbd20 con 0x7f866c15a820 2024-03-30T00:47:44.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f866aa00640 1 --2- 172.21.15.66:0/3341220633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c15a820 0x7f866c0b01c0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f866c0c2ef0 tx=0x7f865c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f37933396649b1e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f866a000640 1 -- 172.21.15.66:0/3341220633 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0ab600 con 0x7f866c15a820 2024-03-30T00:47:44.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f866a000640 1 -- 172.21.15.66:0/3341220633 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c0a7050 con 0x7f866c15a820 2024-03-30T00:47:44.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f866a000640 1 -- 172.21.15.66:0/3341220633 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0ab980 con 0x7f866c15a820 2024-03-30T00:47:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f8671e00640 1 -- 172.21.15.66:0/3341220633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c15a820 msgr2=0x7f866c0b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f8671e00640 1 --2- 172.21.15.66:0/3341220633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c15a820 0x7f866c0b01c0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f866c0c2ef0 tx=0x7f865c01c490 comp rx=0 tx=0).stop 2024-03-30T00:47:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f8671e00640 1 -- 172.21.15.66:0/3341220633 shutdown_connections 2024-03-30T00:47:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f8671e00640 1 --2- 172.21.15.66:0/3341220633 >> v2:172.21.15.156:3300/0 conn(0x7f866c06a4a0 0x7f866c164400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f8671e00640 1 --2- 172.21.15.66:0/3341220633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c15a820 0x7f866c0b01c0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f8671e00640 1 -- 172.21.15.66:0/3341220633 >> 172.21.15.66:0/3341220633 conn(0x7f866c0cf410 msgr2=0x7f866c0cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f8671e00640 1 -- 172.21.15.66:0/3341220633 shutdown_connections 2024-03-30T00:47:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.221+0000 7f8671e00640 1 -- 172.21.15.66:0/3341220633 wait complete. 2024-03-30T00:47:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8671e00640 1 Processor -- start 2024-03-30T00:47:44.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8671e00640 1 -- start start 2024-03-30T00:47:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8671e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f866c06a4a0 0x7f866c0992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8671e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c0b0700 0x7f866c0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8671e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f866c15a820 0x7f866c0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8671e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0997f0 con 0x7f866c0b0700 2024-03-30T00:47:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8671e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0ad3c0 con 0x7f866c06a4a0 2024-03-30T00:47:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8671e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0ad680 con 0x7f866c15a820 2024-03-30T00:47:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c0b0700 0x7f866c0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c0b0700 0x7f866c0c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55350/0 (socket says 172.21.15.66:55350) 2024-03-30T00:47:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f866b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f866c15a820 0x7f866c0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663e00640 1 -- 172.21.15.66:0/883576188 learned_addr learned my addr 172.21.15.66:0/883576188 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f866aa00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f866c06a4a0 0x7f866c0992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663e00640 1 -- 172.21.15.66:0/883576188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f866c15a820 msgr2=0x7f866c0c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663e00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f866c15a820 0x7f866c0c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663e00640 1 -- 172.21.15.66:0/883576188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f866c06a4a0 msgr2=0x7f866c0992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663e00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f866c06a4a0 0x7f866c0992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663e00640 1 -- 172.21.15.66:0/883576188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c0094d0 con 0x7f866c0b0700 2024-03-30T00:47:44.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663e00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c0b0700 0x7f866c0c8ea0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f865400c500 tx=0x7f865400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663400640 1 -- 172.21.15.66:0/883576188 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654005220 con 0x7f866c0b0700 2024-03-30T00:47:44.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663400640 1 -- 172.21.15.66:0/883576188 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8654005380 con 0x7f866c0b0700 2024-03-30T00:47:44.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8663400640 1 -- 172.21.15.66:0/883576188 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654007270 con 0x7f866c0b0700 2024-03-30T00:47:44.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8671e00640 1 -- 172.21.15.66:0/883576188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c0ccc60 con 0x7f866c0b0700 2024-03-30T00:47:44.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.225+0000 7f8671e00640 1 -- 172.21.15.66:0/883576188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c0adc40 con 0x7f866c0b0700 2024-03-30T00:47:44.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f8663400640 1 -- 172.21.15.66:0/883576188 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f86540073d0 con 0x7f866c0b0700 2024-03-30T00:47:44.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f8663400640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8648066690 0x7f8648068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f8663400640 1 -- 172.21.15.66:0/883576188 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8654083bf0 con 0x7f866c0b0700 2024-03-30T00:47:44.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f866aa00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8648066690 0x7f8648068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f867517d000 1 Processor -- start 2024-03-30T00:47:44.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f867517d000 1 -- start start 2024-03-30T00:47:44.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f866aa00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8648066690 0x7f8648068b50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f865c0ad000 tx=0x7f865c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f867517d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9f43670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f867517d000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9ef9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f867517d000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9efa1b0 0x5591e9efa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f867517d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e9dc2340 con 0x5591e9f43290 2024-03-30T00:47:44.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f867517d000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e9d4ecd0 con 0x5591e9efa1b0 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f867517d000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591e9d4c2a0 con 0x5591e9f43bb0 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f866aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9f43670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f866aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9f43670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55364/0 (socket says 172.21.15.66:55364) 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.229+0000 7f866aa00640 1 -- 172.21.15.66:0/540727524 learned_addr learned my addr 172.21.15.66:0/540727524 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f8663e00640 1 --2- 172.21.15.66:0/540727524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9ef9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 -- 172.21.15.66:0/540727524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 msgr2=0x5591e9ef9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 --2- 172.21.15.66:0/540727524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9ef9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 -- 172.21.15.66:0/540727524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9efa1b0 msgr2=0x5591e9efa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 --2- 172.21.15.66:0/540727524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9efa1b0 0x5591e9efa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 -- 172.21.15.66:0/540727524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591e9a3a4b0 con 0x5591e9f43290 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f8663e00640 1 --2- 172.21.15.66:0/540727524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9ef9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 --2- 172.21.15.66:0/540727524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9f43670 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f865c097d60 tx=0x7f865c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990d0cd16b575198 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f865be00640 1 -- 172.21.15.66:0/540727524 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0c1480 con 0x5591e9f43290 2024-03-30T00:47:44.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f865be00640 1 -- 172.21.15.66:0/540727524 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c0bc070 con 0x5591e9f43290 2024-03-30T00:47:44.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f865be00640 1 -- 172.21.15.66:0/540727524 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0c17e0 con 0x5591e9f43290 2024-03-30T00:47:44.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 -- 172.21.15.66:0/540727524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 msgr2=0x5591e9f43670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 --2- 172.21.15.66:0/540727524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9f43670 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f865c097d60 tx=0x7f865c004740 comp rx=0 tx=0).stop 2024-03-30T00:47:44.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 -- 172.21.15.66:0/540727524 shutdown_connections 2024-03-30T00:47:44.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 --2- 172.21.15.66:0/540727524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9efa1b0 0x5591e9efa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 --2- 172.21.15.66:0/540727524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9ef9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 --2- 172.21.15.66:0/540727524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9f43670 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 -- 172.21.15.66:0/540727524 >> 172.21.15.66:0/540727524 conn(0x5591e9ef6160 msgr2=0x5591e9ef8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:44.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 -- 172.21.15.66:0/540727524 shutdown_connections 2024-03-30T00:47:44.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 -- 172.21.15.66:0/540727524 wait complete. 2024-03-30T00:47:44.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 -- start start 2024-03-30T00:47:44.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 Processor -- start 2024-03-30T00:47:44.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9eff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9f43290 0x5591e9effbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9f04e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e9d47b90 con 0x5591e9efa1b0 2024-03-30T00:47:44.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e9d050a0 con 0x5591e9f43290 2024-03-30T00:47:44.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591e9d04d40 con 0x5591e9f43bb0 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f867517d000 10 client.0 _ll_register_callbacks cb 0x5591e9dfcc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9eff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9eff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55374/0 (socket says 172.21.15.66:55374) 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 -- 172.21.15.66:0/2606435171 learned_addr learned my addr 172.21.15.66:0/2606435171 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866b400640 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9f04e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f8663e00640 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9f43290 0x5591e9effbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 -- 172.21.15.66:0/2606435171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 msgr2=0x5591e9f04e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9f04e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 -- 172.21.15.66:0/2606435171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9f43290 msgr2=0x5591e9effbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.233+0000 7f866aa00640 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9f43290 0x5591e9effbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f866aa00640 1 -- 172.21.15.66:0/2606435171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c0cbd20 con 0x5591e9efa1b0 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f866aa00640 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9eff6b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f865c0190a0 tx=0x7f865c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a07565a478a19e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8658c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8658c00640 1 -- 172.21.15.66:0/2606435171 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0c19b0 con 0x5591e9efa1b0 2024-03-30T00:47:44.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8658c00640 1 -- 172.21.15.66:0/2606435171 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f865c0c1b10 con 0x5591e9efa1b0 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 -- 172.21.15.66:0/2606435171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591e9d17550 con 0x5591e9efa1b0 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8658c00640 1 -- 172.21.15.66:0/2606435171 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f865c0c1cd0 con 0x5591e9efa1b0 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 10 client.8166 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 -- 172.21.15.66:0/2606435171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5591e9d0cba0 con 0x5591e9efa1b0 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8658c00640 1 -- 172.21.15.66:0/2606435171 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5591e9d17550 con 0x5591e9efa1b0 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8658c00640 1 -- 172.21.15.66:0/2606435171 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f865c0b5410 con 0x5591e9efa1b0 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 10 client.8166 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 10 client.8166 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 20 client.8166 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5591e9f52320 0x5591e9f54710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 4 client.8166 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 -- 172.21.15.66:0/2606435171 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591e9d44970 con 0x5591e9f52320 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8663e00640 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5591e9f52320 0x5591e9f54710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8658c00640 10 client.8166 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8663e00640 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5591e9f52320 0x5591e9f54710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67d1ba1e54ded968 server_cookie=2effc06175603431 in_seq=0 out_seq=0 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8658c00640 1 -- 172.21.15.66:0/2606435171 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5591e9d44970 con 0x5591e9f52320 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8658c00640 10 client.8166 handle_command_reply: tid=0 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 client.8166 shutdown 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 -- 172.21.15.66:0/2606435171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 msgr2=0x5591e9eff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9eff6b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f865c0190a0 tx=0x7f865c0046d0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 -- 172.21.15.66:0/2606435171 shutdown_connections 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5591e9f52320 0x5591e9f54710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9f04e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f8658c00640 0 client.8166 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9f43290 0x5591e9effbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 --2- 172.21.15.66:0/2606435171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9eff6b0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 -- 172.21.15.66:0/2606435171 >> 172.21.15.66:0/2606435171 conn(0x5591e9ef6160 msgr2=0x5591e9ef8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 -- 172.21.15.66:0/2606435171 shutdown_connections 2024-03-30T00:47:44.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 1 -- 172.21.15.66:0/2606435171 wait complete. 2024-03-30T00:47:44.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.237+0000 7f867517d000 20 client.8166 trim_cache size 0 max 16384 2024-03-30T00:47:44.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.249+0000 7f867517d000 1 Processor -- start 2024-03-30T00:47:44.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.249+0000 7f867517d000 1 -- start start 2024-03-30T00:47:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.249+0000 7f867517d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9ef9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.249+0000 7f867517d000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43290 0x5591e9ef9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.249+0000 7f867517d000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9f43bb0 0x5591e9f5dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.249+0000 7f867517d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e9a04c40 con 0x5591e9efa1b0 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.249+0000 7f867517d000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e9d46a20 con 0x5591e9f43bb0 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.249+0000 7f867517d000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591e9d4bb90 con 0x5591e9f43290 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.249+0000 7f866aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9ef9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f866aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9ef9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55378/0 (socket says 172.21.15.66:55378) 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f866aa00640 1 -- 172.21.15.66:0/3404721547 learned_addr learned my addr 172.21.15.66:0/3404721547 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f866b400640 1 --2- 172.21.15.66:0/3404721547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9f43bb0 0x5591e9f5dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8663e00640 1 --2- 172.21.15.66:0/3404721547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43290 0x5591e9ef9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f866aa00640 1 -- 172.21.15.66:0/3404721547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43290 msgr2=0x5591e9ef9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f866aa00640 1 --2- 172.21.15.66:0/3404721547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43290 0x5591e9ef9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f866aa00640 1 -- 172.21.15.66:0/3404721547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9f43bb0 msgr2=0x5591e9f5dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f866aa00640 1 --2- 172.21.15.66:0/3404721547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9f43bb0 0x5591e9f5dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f866aa00640 1 -- 172.21.15.66:0/3404721547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591e9f42ce0 con 0x5591e9efa1b0 2024-03-30T00:47:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8663e00640 1 --2- 172.21.15.66:0/3404721547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43290 0x5591e9ef9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f866b400640 1 --2- 172.21.15.66:0/3404721547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9f43bb0 0x5591e9f5dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f866aa00640 1 --2- 172.21.15.66:0/3404721547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9ef9560 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f865c01ca00 tx=0x7f865c0bc000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970387c57581b609 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8660c00640 1 -- 172.21.15.66:0/3404721547 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0a4da0 con 0x5591e9efa1b0 2024-03-30T00:47:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8660c00640 1 -- 172.21.15.66:0/3404721547 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c0abb20 con 0x5591e9efa1b0 2024-03-30T00:47:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8660c00640 1 -- 172.21.15.66:0/3404721547 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0b5b00 con 0x5591e9efa1b0 2024-03-30T00:47:44.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 -- 172.21.15.66:0/3404721547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 msgr2=0x5591e9ef9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 --2- 172.21.15.66:0/3404721547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9ef9560 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f865c01ca00 tx=0x7f865c0bc000 comp rx=0 tx=0).stop 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 -- 172.21.15.66:0/3404721547 shutdown_connections 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 --2- 172.21.15.66:0/3404721547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9f43bb0 0x5591e9f5dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 --2- 172.21.15.66:0/3404721547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43290 0x5591e9ef9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 --2- 172.21.15.66:0/3404721547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9efa1b0 0x5591e9ef9560 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 -- 172.21.15.66:0/3404721547 >> 172.21.15.66:0/3404721547 conn(0x5591e9ef6160 msgr2=0x5591e9eff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 -- 172.21.15.66:0/3404721547 shutdown_connections 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 -- 172.21.15.66:0/3404721547 wait complete. 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 -- start start 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 Processor -- start 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9efa1b0 0x5591e9fa5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9fa6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9f60920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e9d4ff60 con 0x5591e9f43290 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e9cf0520 con 0x5591e9efa1b0 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591e9d453a0 con 0x5591e9f43bb0 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f867517d000 10 client.0 _ll_register_callbacks cb 0x5591e9de5870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8663e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9fa6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8663e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9fa6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55386/0 (socket says 172.21.15.66:55386) 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8663e00640 1 -- 172.21.15.66:0/2563340476 learned_addr learned my addr 172.21.15.66:0/2563340476 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8663e00640 1 -- 172.21.15.66:0/2563340476 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 msgr2=0x5591e9f60920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8663e00640 1 --2- 172.21.15.66:0/2563340476 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9f60920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8663e00640 1 -- 172.21.15.66:0/2563340476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9efa1b0 msgr2=0x5591e9fa5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8663e00640 1 --2- 172.21.15.66:0/2563340476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9efa1b0 0x5591e9fa5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.253+0000 7f8663e00640 1 -- 172.21.15.66:0/2563340476 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c0b5e70 con 0x5591e9f43290 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8663e00640 1 --2- 172.21.15.66:0/2563340476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9fa6090 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f8654050f30 tx=0x7f86540588e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d7865dda11b8e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8658c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8658c00640 1 -- 172.21.15.66:0/2563340476 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654082040 con 0x5591e9f43290 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8658c00640 1 -- 172.21.15.66:0/2563340476 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f8654071200 con 0x5591e9f43290 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 -- 172.21.15.66:0/2563340476 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591e9d464e0 con 0x5591e9f43290 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8658c00640 1 -- 172.21.15.66:0/2563340476 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f865406c220 con 0x5591e9f43290 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 10 client.8172 fetch_fsmap learned FSMap version 65 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 -- 172.21.15.66:0/2563340476 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5591e9cf2ef0 con 0x5591e9f43290 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8658c00640 1 -- 172.21.15.66:0/2563340476 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8654076020 con 0x5591e9f43290 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8658c00640 1 -- 172.21.15.66:0/2563340476 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8654019030 con 0x5591e9f43290 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 10 client.8172 fetch_fsmap finished waiting for FSMap version 65 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 10 client.8172 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 20 client.8172 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 --2- 172.21.15.66:0/2563340476 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5591e9faca20 0x5591e9faee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 4 client.8172 mds_command: new command op to 7920 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-30T00:47:44.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 -- 172.21.15.66:0/2563340476 --> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5591e9d40650 con 0x5591e9faca20 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f866aa00640 1 --2- 172.21.15.66:0/2563340476 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5591e9faca20 0x5591e9faee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f866aa00640 1 --2- 172.21.15.66:0/2563340476 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5591e9faca20 0x5591e9faee10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84037e2cf295d2f7 server_cookie=c5f902dea6760b60 in_seq=0 out_seq=0 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8658c00640 10 client.8172 ms_handle_connect on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8658c00640 1 -- 172.21.15.66:0/2563340476 <== mds.0 v2:172.21.15.66:6835/1626022080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+143 (crc 0 0 0) 0x5591e9d40650 con 0x5591e9faca20 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8658c00640 10 client.8172 handle_command_reply: tid=0 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 client.8172 shutdown 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 -- 172.21.15.66:0/2563340476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 msgr2=0x5591e9fa6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 --2- 172.21.15.66:0/2563340476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9fa6090 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f8654050f30 tx=0x7f86540588e0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 -- 172.21.15.66:0/2563340476 shutdown_connections 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 --2- 172.21.15.66:0/2563340476 >> [v2:172.21.15.66:6835/1626022080,v1:172.21.15.66:6836/1626022080] conn(0x5591e9faca20 0x5591e9faee10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 --2- 172.21.15.66:0/2563340476 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591e9f43bb0 0x5591e9f60920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 --2- 172.21.15.66:0/2563340476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591e9f43290 0x5591e9fa6090 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 --2- 172.21.15.66:0/2563340476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591e9efa1b0 0x5591e9fa5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8658c00640 0 client.8172 ms_handle_reset on v2:172.21.15.66:6835/1626022080 2024-03-30T00:47:44.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 -- 172.21.15.66:0/2563340476 >> 172.21.15.66:0/2563340476 conn(0x5591e9ef6160 msgr2=0x5591e9ef8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:44.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 -- 172.21.15.66:0/2563340476 shutdown_connections 2024-03-30T00:47:44.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 1 -- 172.21.15.66:0/2563340476 wait complete. 2024-03-30T00:47:44.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f867517d000 20 client.8172 trim_cache size 0 max 16384 2024-03-30T00:47:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:47:44.272 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:47:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:47:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2024-03-30T00:47:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 183660447, 2024-03-30T00:47:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627777, 2024-03-30T00:47:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/testdir/hardlink" 2024-03-30T00:47:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:47:44.273 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:47:44.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 -- 172.21.15.66:0/883576188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8648066690 msgr2=0x7f8648068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8648066690 0x7f8648068b50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f865c0ad000 tx=0x7f865c008040 comp rx=0 tx=0).stop 2024-03-30T00:47:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 -- 172.21.15.66:0/883576188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c0b0700 msgr2=0x7f866c0c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c0b0700 0x7f866c0c8ea0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f865400c500 tx=0x7f865400c9c0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 -- 172.21.15.66:0/883576188 shutdown_connections 2024-03-30T00:47:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f866c15a820 0x7f866c0c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8648066690 0x7f8648068b50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f866c0b0700 0x7f866c0c8ea0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 --2- 172.21.15.66:0/883576188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f866c06a4a0 0x7f866c0992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 -- 172.21.15.66:0/883576188 >> 172.21.15.66:0/883576188 conn(0x7f866c0cf410 msgr2=0x7f866c0b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 -- 172.21.15.66:0/883576188 shutdown_connections 2024-03-30T00:47:44.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.257+0000 7f8671e00640 1 -- 172.21.15.66:0/883576188 wait complete. 2024-03-30T00:47:44.286 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:47:44.286 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:47:44.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:47:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.465+0000 7fa83b400640 1 Processor -- start 2024-03-30T00:47:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.465+0000 7fa83b400640 1 -- start start 2024-03-30T00:47:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.465+0000 7fa83b400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa8340b0fb0 0x7fa83406b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.465+0000 7fa83b400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa834098ac0 con 0x7fa8340cca40 2024-03-30T00:47:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.465+0000 7fa83b400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8340cf3f0 con 0x7fa8340b1420 2024-03-30T00:47:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.465+0000 7fa83b400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8340cf670 con 0x7fa8340b0fb0 2024-03-30T00:47:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.465+0000 7fa838a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa8340cca40 0x7fa8340cac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55336/0 (socket says 172.21.15.66:55336) 2024-03-30T00:47:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.465+0000 7fa838a00640 1 -- 172.21.15.66:0/183172401 learned_addr learned my addr 172.21.15.66:0/183172401 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.469+0000 7fa833e00640 1 --2- 172.21.15.66:0/183172401 >> v2:172.21.15.156:3300/0 conn(0x7fa8340b0fb0 0x7fa83406b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.469+0000 7fa833400640 1 -- 172.21.15.66:0/183172401 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046777637 0 0) 0x7fa834098ac0 con 0x7fa8340cca40 2024-03-30T00:47:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.469+0000 7fa833400640 1 -- 172.21.15.66:0/183172401 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa81c003530 con 0x7fa8340cca40 2024-03-30T00:47:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.469+0000 7fa833400640 1 -- 172.21.15.66:0/183172401 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3093175390 0 0) 0x7fa81c003530 con 0x7fa8340cca40 2024-03-30T00:47:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.469+0000 7fa833e00640 1 -- 172.21.15.66:0/183172401 >> v1:172.21.15.156:6789/0 conn(0x7fa8340b1420 legacy=0x7fa834099250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.469+0000 7fa833e00640 1 -- 172.21.15.66:0/183172401 >> v1:172.21.15.66:6789/0 conn(0x7fa8340cca40 legacy=0x7fa8340cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.469+0000 7fa833e00640 1 -- 172.21.15.66:0/183172401 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8340cf950 con 0x7fa8340b0fb0 2024-03-30T00:47:44.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.469+0000 7fa833e00640 1 --2- 172.21.15.66:0/183172401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8340b0fb0 0x7fa83406b140 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa8200097c0 tx=0x7fa82001c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6543052e913cceb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.469+0000 7fa833400640 1 -- 172.21.15.66:0/183172401 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828001680 con 0x7fa8340b0fb0 2024-03-30T00:47:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.469+0000 7fa833400640 1 -- 172.21.15.66:0/183172401 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8200a7050 con 0x7fa8340b0fb0 2024-03-30T00:47:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.469+0000 7fa833400640 1 -- 172.21.15.66:0/183172401 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8200ab600 con 0x7fa8340b0fb0 2024-03-30T00:47:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 -- 172.21.15.66:0/183172401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8340b0fb0 msgr2=0x7fa83406b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 --2- 172.21.15.66:0/183172401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8340b0fb0 0x7fa83406b140 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa8200097c0 tx=0x7fa82001c390 comp rx=0 tx=0).stop 2024-03-30T00:47:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 -- 172.21.15.66:0/183172401 shutdown_connections 2024-03-30T00:47:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 --2- 172.21.15.66:0/183172401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8340b0fb0 0x7fa83406b140 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 -- 172.21.15.66:0/183172401 >> 172.21.15.66:0/183172401 conn(0x7fa8340b2130 msgr2=0x7fa8340b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 -- 172.21.15.66:0/183172401 shutdown_connections 2024-03-30T00:47:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 -- 172.21.15.66:0/183172401 wait complete. 2024-03-30T00:47:44.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 Processor -- start 2024-03-30T00:47:44.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 -- start start 2024-03-30T00:47:44.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8340b0fb0 0x7fa83415fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8340b1420 0x7fa8341f9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8340cca40 0x7fa8341f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340c8120 con 0x7fa8340b1420 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8341553d0 con 0x7fa8340b0fb0 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa83b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8341556b0 con 0x7fa8340cca40 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa833e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8340b1420 0x7fa8341f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa833e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8340b1420 0x7fa8341f9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55394/0 (socket says 172.21.15.66:55394) 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa833e00640 1 -- 172.21.15.66:0/2688592928 learned_addr learned my addr 172.21.15.66:0/2688592928 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa838a00640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8340b0fb0 0x7fa83415fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa839400640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8340cca40 0x7fa8341f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa833e00640 1 -- 172.21.15.66:0/2688592928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8340cca40 msgr2=0x7fa8341f9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa833e00640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8340cca40 0x7fa8341f9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa833e00640 1 -- 172.21.15.66:0/2688592928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8340b0fb0 msgr2=0x7fa83415fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa833e00640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8340b0fb0 0x7fa83415fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa833e00640 1 -- 172.21.15.66:0/2688592928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa820009420 con 0x7fa8340b1420 2024-03-30T00:47:44.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.473+0000 7fa839400640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8340cca40 0x7fa8341f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:44.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa833e00640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8340b1420 0x7fa8341f9500 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fa820018fa0 tx=0x7fa820019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa830c00640 1 -- 172.21.15.66:0/2688592928 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8200b4400 con 0x7fa8340b1420 2024-03-30T00:47:44.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa830c00640 1 -- 172.21.15.66:0/2688592928 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8200b4560 con 0x7fa8340b1420 2024-03-30T00:47:44.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa830c00640 1 -- 172.21.15.66:0/2688592928 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8200b48f0 con 0x7fa8340b1420 2024-03-30T00:47:44.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa83b400640 1 -- 172.21.15.66:0/2688592928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa834155990 con 0x7fa8340b1420 2024-03-30T00:47:44.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa83b400640 1 -- 172.21.15.66:0/2688592928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa834160140 con 0x7fa8340b1420 2024-03-30T00:47:44.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa83b400640 1 -- 172.21.15.66:0/2688592928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8340677d0 con 0x7fa8340b1420 2024-03-30T00:47:44.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa830c00640 1 -- 172.21.15.66:0/2688592928 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa8200b2040 con 0x7fa8340b1420 2024-03-30T00:47:44.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa830c00640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa814066690 0x7fa814068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa830c00640 1 -- 172.21.15.66:0/2688592928 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fa82011e2d0 con 0x7fa8340b1420 2024-03-30T00:47:44.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa838a00640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa814066690 0x7fa814068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.477+0000 7fa838a00640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa814066690 0x7fa814068b50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa828000fc0 tx=0x7fa8280019c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.481+0000 7fa830c00640 1 -- 172.21.15.66:0/2688592928 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa8200eb420 con 0x7fa8340b1420 2024-03-30T00:47:44.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.641+0000 7fa83b400640 1 -- 172.21.15.66:0/2688592928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa834066c30 con 0x7fa8340b1420 2024-03-30T00:47:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa830c00640 1 -- 172.21.15.66:0/2688592928 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16571 (secure 0 0 0) 0x7fa8200b4a50 con 0x7fa8340b1420 2024-03-30T00:47:44.653 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:47:44.654 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":97,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:46:34.103855+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:46:13.811742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-30T00:46:14.265142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":7}]},{"pgid":"14.1a","mappings":[{"from":1,"to":5}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6836/1147664160":"2024-03-31T00:46:34.103831+0000","172.21.15.66:6835/1147664160":"2024-03-31T00:46:34.103831+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:47:44.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 -- 172.21.15.66:0/2688592928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa814066690 msgr2=0x7fa814068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa814066690 0x7fa814068b50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa828000fc0 tx=0x7fa8280019c0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 -- 172.21.15.66:0/2688592928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8340b1420 msgr2=0x7fa8341f9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8340b1420 0x7fa8341f9500 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fa820018fa0 tx=0x7fa820019130 comp rx=0 tx=0).stop 2024-03-30T00:47:44.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 -- 172.21.15.66:0/2688592928 shutdown_connections 2024-03-30T00:47:44.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8340cca40 0x7fa8341f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa814066690 0x7fa814068b50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8340b1420 0x7fa8341f9500 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 --2- 172.21.15.66:0/2688592928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8340b0fb0 0x7fa83415fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 -- 172.21.15.66:0/2688592928 >> 172.21.15.66:0/2688592928 conn(0x7fa8340b2130 msgr2=0x7fa8340b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:44.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 -- 172.21.15.66:0/2688592928 shutdown_connections 2024-03-30T00:47:44.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.645+0000 7fa83b400640 1 -- 172.21.15.66:0/2688592928 wait complete. 2024-03-30T00:47:44.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:46:13.811742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:46:14.265142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:47:44.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:47:44.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e1be00640 1 Processor -- start 2024-03-30T00:47:44.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e1be00640 1 -- start start 2024-03-30T00:47:44.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e1be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4e1415de30 0x7f4e140cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e1be00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e140af420 0x7f4e140d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e1be00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e140cbb90 con 0x7f4e140ce740 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e1be00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e140ce1b0 con 0x7f4e1415de30 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e1be00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e140afa10 con 0x7f4e140af420 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e19400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4e1415de30 0x7f4e140cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e19400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4e1415de30 0x7f4e140cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55396/0 (socket says 172.21.15.66:55396) 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e19400640 1 -- 172.21.15.66:0/2837814050 learned_addr learned my addr 172.21.15.66:0/2837814050 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e18a00640 1 --2- 172.21.15.66:0/2837814050 >> v2:172.21.15.156:3300/0 conn(0x7f4e140af420 0x7f4e140d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e19400640 1 -- 172.21.15.66:0/2837814050 >> v2:172.21.15.156:3300/0 conn(0x7f4e140af420 msgr2=0x7f4e140d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e19400640 1 --2- 172.21.15.66:0/2837814050 >> v2:172.21.15.156:3300/0 conn(0x7f4e140af420 0x7f4e140d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e19400640 1 -- 172.21.15.66:0/2837814050 >> v1:172.21.15.156:6789/0 conn(0x7f4e140ce740 legacy=0x7f4e1406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:44.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.809+0000 7f4e19400640 1 -- 172.21.15.66:0/2837814050 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e140cf430 con 0x7f4e1415de30 2024-03-30T00:47:44.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e19400640 1 --2- 172.21.15.66:0/2837814050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e1415de30 0x7f4e140cc040 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f4e14070080 tx=0x7f4e0801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551ed2ade0b94e35 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e13e00640 1 -- 172.21.15.66:0/2837814050 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e080ac410 con 0x7f4e1415de30 2024-03-30T00:47:44.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e13e00640 1 -- 172.21.15.66:0/2837814050 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e080ac570 con 0x7f4e1415de30 2024-03-30T00:47:44.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e13e00640 1 -- 172.21.15.66:0/2837814050 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e080ac900 con 0x7f4e1415de30 2024-03-30T00:47:44.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e1be00640 1 -- 172.21.15.66:0/2837814050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e1415de30 msgr2=0x7f4e140cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e1be00640 1 --2- 172.21.15.66:0/2837814050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e1415de30 0x7f4e140cc040 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f4e14070080 tx=0x7f4e0801c450 comp rx=0 tx=0).stop 2024-03-30T00:47:44.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e1be00640 1 -- 172.21.15.66:0/2837814050 shutdown_connections 2024-03-30T00:47:44.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e1be00640 1 --2- 172.21.15.66:0/2837814050 >> v2:172.21.15.156:3300/0 conn(0x7f4e140af420 0x7f4e140d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e1be00640 1 --2- 172.21.15.66:0/2837814050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e1415de30 0x7f4e140cc040 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e1be00640 1 -- 172.21.15.66:0/2837814050 >> 172.21.15.66:0/2837814050 conn(0x7f4e140d0540 msgr2=0x7f4e140d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e1be00640 1 -- 172.21.15.66:0/2837814050 shutdown_connections 2024-03-30T00:47:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.813+0000 7f4e1be00640 1 -- 172.21.15.66:0/2837814050 wait complete. 2024-03-30T00:47:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e1be00640 1 Processor -- start 2024-03-30T00:47:44.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e1be00640 1 -- start start 2024-03-30T00:47:44.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e1be00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140af420 0x7f4e141499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e1be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e140ce740 0x7f4e1406aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e1be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e1415de30 0x7f4e140c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e1be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1406b960 con 0x7f4e1415de30 2024-03-30T00:47:44.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e1be00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1406bc40 con 0x7f4e140af420 2024-03-30T00:47:44.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e1be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1406bf20 con 0x7f4e140ce740 2024-03-30T00:47:44.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e19e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e1415de30 0x7f4e140c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e19e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e1415de30 0x7f4e140c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55410/0 (socket says 172.21.15.66:55410) 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e19e00640 1 -- 172.21.15.66:0/2819702221 learned_addr learned my addr 172.21.15.66:0/2819702221 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e19400640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140af420 0x7f4e141499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e18a00640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e140ce740 0x7f4e1406aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e19e00640 1 -- 172.21.15.66:0/2819702221 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e140ce740 msgr2=0x7f4e1406aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e19e00640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e140ce740 0x7f4e1406aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e19e00640 1 -- 172.21.15.66:0/2819702221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140af420 msgr2=0x7f4e141499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e19e00640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140af420 0x7f4e141499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e19e00640 1 -- 172.21.15.66:0/2819702221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e080094d0 con 0x7f4e1415de30 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e19e00640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e1415de30 0x7f4e140c50c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00c780 tx=0x7f4e0c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e11600640 1 -- 172.21.15.66:0/2819702221 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c004b70 con 0x7f4e1415de30 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e11600640 1 -- 172.21.15.66:0/2819702221 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c00ce90 con 0x7f4e1415de30 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e11600640 1 -- 172.21.15.66:0/2819702221 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c00a850 con 0x7f4e1415de30 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e1be00640 1 -- 172.21.15.66:0/2819702221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1406c200 con 0x7f4e1415de30 2024-03-30T00:47:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.817+0000 7f4e1be00640 1 -- 172.21.15.66:0/2819702221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e14163b50 con 0x7f4e1415de30 2024-03-30T00:47:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.821+0000 7f4e1be00640 1 -- 172.21.15.66:0/2819702221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e140cdf60 con 0x7f4e1415de30 2024-03-30T00:47:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.821+0000 7f4e11600640 1 -- 172.21.15.66:0/2819702221 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4e0c00a9b0 con 0x7f4e1415de30 2024-03-30T00:47:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.821+0000 7f4e11600640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 0x7f4df4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.821+0000 7f4e11600640 1 -- 172.21.15.66:0/2819702221 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f4e0c084530 con 0x7f4e1415de30 2024-03-30T00:47:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.821+0000 7f4e19400640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 0x7f4df4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:44.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.821+0000 7f4e19400640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 0x7f4df4068b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4e080ae000 tx=0x7f4e08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:44.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.825+0000 7f4e11600640 1 -- 172.21.15.66:0/2819702221 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4e0c0515f0 con 0x7f4e1415de30 2024-03-30T00:47:44.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.985+0000 7f4e1be00640 1 -- 172.21.15.66:0/2819702221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e14066c30 con 0x7f4e1415de30 2024-03-30T00:47:44.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e11600640 1 -- 172.21.15.66:0/2819702221 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e0c054c10 con 0x7f4e1415de30 2024-03-30T00:47:44.997 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:47:44.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 -- 172.21.15.66:0/2819702221 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 msgr2=0x7f4df4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 0x7f4df4068b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4e080ae000 tx=0x7f4e08008040 comp rx=0 tx=0).stop 2024-03-30T00:47:44.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 -- 172.21.15.66:0/2819702221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e1415de30 msgr2=0x7f4e140c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:44.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e1415de30 0x7f4e140c50c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00c780 tx=0x7f4e0c00cc40 comp rx=0 tx=0).stop 2024-03-30T00:47:44.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 -- 172.21.15.66:0/2819702221 shutdown_connections 2024-03-30T00:47:45.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4df4066690 0x7f4df4068b50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e1415de30 0x7f4e140c50c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e140ce740 0x7f4e1406aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 --2- 172.21.15.66:0/2819702221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e140af420 0x7f4e141499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 -- 172.21.15.66:0/2819702221 >> 172.21.15.66:0/2819702221 conn(0x7f4e140d0540 msgr2=0x7f4e1415bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:45.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 -- 172.21.15.66:0/2819702221 shutdown_connections 2024-03-30T00:47:45.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:44.989+0000 7f4e1be00640 1 -- 172.21.15.66:0/2819702221 wait complete. 2024-03-30T00:47:45.018 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:47:45.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde90400640 1 Processor -- start 2024-03-30T00:47:45.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde90400640 1 -- start start 2024-03-30T00:47:45.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde90400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fde880cde70 0x7fde8815a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde90400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde880cd3e0 con 0x7fde880cec30 2024-03-30T00:47:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde90400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8815a990 con 0x7fde880cda90 2024-03-30T00:47:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde90400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde880b1b80 con 0x7fde880cde70 2024-03-30T00:47:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde8e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fde880cde70 0x7fde8815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde8e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fde880cde70 0x7fde8815a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55420/0 (socket says 172.21.15.66:55420) 2024-03-30T00:47:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde8d000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fde880cec30 0x7fde88159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55340/0 (socket says 172.21.15.66:55340) 2024-03-30T00:47:45.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde8e400640 1 -- 172.21.15.66:0/3177807438 learned_addr learned my addr 172.21.15.66:0/3177807438 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:45.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde8e400640 1 -- 172.21.15.66:0/3177807438 >> v1:172.21.15.156:6789/0 conn(0x7fde880cda90 legacy=0x7fde880ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde8e400640 1 -- 172.21.15.66:0/3177807438 >> v1:172.21.15.66:6789/0 conn(0x7fde880cec30 legacy=0x7fde88159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde8e400640 1 -- 172.21.15.66:0/3177807438 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880b11e0 con 0x7fde880cde70 2024-03-30T00:47:45.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.153+0000 7fde87e00640 1 -- 172.21.15.66:0/3177807438 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660134068 0 0) 0x7fde880cd3e0 con 0x7fde880cec30 2024-03-30T00:47:45.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde8e400640 1 --2- 172.21.15.66:0/3177807438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde880cde70 0x7fde8815a370 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fde80009e50 tx=0x7fde8001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bf2adde437cd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:45.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde87e00640 1 -- 172.21.15.66:0/3177807438 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800a5860 con 0x7fde880cde70 2024-03-30T00:47:45.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde87e00640 1 -- 172.21.15.66:0/3177807438 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde800a59c0 con 0x7fde880cde70 2024-03-30T00:47:45.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde87e00640 1 -- 172.21.15.66:0/3177807438 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800a5d50 con 0x7fde880cde70 2024-03-30T00:47:45.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde90400640 1 -- 172.21.15.66:0/3177807438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde880cde70 msgr2=0x7fde8815a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde90400640 1 --2- 172.21.15.66:0/3177807438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde880cde70 0x7fde8815a370 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fde80009e50 tx=0x7fde8001c9a0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde90400640 1 -- 172.21.15.66:0/3177807438 shutdown_connections 2024-03-30T00:47:45.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde90400640 1 --2- 172.21.15.66:0/3177807438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde880cde70 0x7fde8815a370 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde90400640 1 -- 172.21.15.66:0/3177807438 >> 172.21.15.66:0/3177807438 conn(0x7fde88069f60 msgr2=0x7fde8806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:45.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde90400640 1 -- 172.21.15.66:0/3177807438 shutdown_connections 2024-03-30T00:47:45.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde90400640 1 -- 172.21.15.66:0/3177807438 wait complete. 2024-03-30T00:47:45.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde90400640 1 Processor -- start 2024-03-30T00:47:45.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.157+0000 7fde90400640 1 -- start start 2024-03-30T00:47:45.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde90400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde880cda90 0x7fde880715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde90400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde880cde70 0x7fde88163d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde90400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde880cec30 0x7fde881642d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde90400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde88155490 con 0x7fde880cda90 2024-03-30T00:47:45.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde90400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde88164810 con 0x7fde880cde70 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde90400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde88153580 con 0x7fde880cec30 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde880cda90 0x7fde880715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde880cda90 0x7fde880715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55424/0 (socket says 172.21.15.66:55424) 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8da00640 1 -- 172.21.15.66:0/1042149271 learned_addr learned my addr 172.21.15.66:0/1042149271 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8d000640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde880cde70 0x7fde88163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8e400640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde880cec30 0x7fde881642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8da00640 1 -- 172.21.15.66:0/1042149271 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde880cec30 msgr2=0x7fde881642d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8da00640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde880cec30 0x7fde881642d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8da00640 1 -- 172.21.15.66:0/1042149271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde880cde70 msgr2=0x7fde88163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8da00640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde880cde70 0x7fde88163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8da00640 1 -- 172.21.15.66:0/1042149271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde80009b30 con 0x7fde880cda90 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8d000640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde880cde70 0x7fde88163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde8da00640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde880cda90 0x7fde880715f0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00c3f0 tx=0x7fde7c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde85600640 1 -- 172.21.15.66:0/1042149271 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c0052d0 con 0x7fde880cda90 2024-03-30T00:47:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde85600640 1 -- 172.21.15.66:0/1042149271 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c005430 con 0x7fde880cda90 2024-03-30T00:47:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde90400640 1 -- 172.21.15.66:0/1042149271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde88152fc0 con 0x7fde880cda90 2024-03-30T00:47:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde85600640 1 -- 172.21.15.66:0/1042149271 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c003840 con 0x7fde880cda90 2024-03-30T00:47:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde90400640 1 -- 172.21.15.66:0/1042149271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde88155120 con 0x7fde880cda90 2024-03-30T00:47:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.161+0000 7fde90400640 1 -- 172.21.15.66:0/1042149271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde88066c30 con 0x7fde880cda90 2024-03-30T00:47:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.165+0000 7fde85600640 1 -- 172.21.15.66:0/1042149271 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fde7c0039a0 con 0x7fde880cda90 2024-03-30T00:47:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.165+0000 7fde85600640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde68066690 0x7fde68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.165+0000 7fde85600640 1 -- 172.21.15.66:0/1042149271 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fde7c0839a0 con 0x7fde880cda90 2024-03-30T00:47:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.165+0000 7fde8d000640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde68066690 0x7fde68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.165+0000 7fde8d000640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde68066690 0x7fde68068b50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fde880c1c40 tx=0x7fde74007c60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:45.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.169+0000 7fde85600640 1 -- 172.21.15.66:0/1042149271 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fde7c050a80 con 0x7fde880cda90 2024-03-30T00:47:45.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.325+0000 7fde90400640 1 -- 172.21.15.66:0/1042149271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde880c8460 con 0x7fde880cda90 2024-03-30T00:47:45.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.325+0000 7fde85600640 1 -- 172.21.15.66:0/1042149271 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde7c0170a0 con 0x7fde880cda90 2024-03-30T00:47:45.337 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:47:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 -- 172.21.15.66:0/1042149271 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde68066690 msgr2=0x7fde68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde68066690 0x7fde68068b50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fde880c1c40 tx=0x7fde74007c60 comp rx=0 tx=0).stop 2024-03-30T00:47:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 -- 172.21.15.66:0/1042149271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde880cda90 msgr2=0x7fde880715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde880cda90 0x7fde880715f0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00c3f0 tx=0x7fde7c00c8b0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 -- 172.21.15.66:0/1042149271 shutdown_connections 2024-03-30T00:47:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde880cec30 0x7fde881642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde880cde70 0x7fde88163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fde68066690 0x7fde68068b50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 --2- 172.21.15.66:0/1042149271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde880cda90 0x7fde880715f0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 -- 172.21.15.66:0/1042149271 >> 172.21.15.66:0/1042149271 conn(0x7fde88069f60 msgr2=0x7fde8815aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:45.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 -- 172.21.15.66:0/1042149271 shutdown_connections 2024-03-30T00:47:45.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.329+0000 7fde90400640 1 -- 172.21.15.66:0/1042149271 wait complete. 2024-03-30T00:47:45.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:47:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.489+0000 7f9a59a00640 1 Processor -- start 2024-03-30T00:47:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.489+0000 7f9a59a00640 1 -- start start 2024-03-30T00:47:45.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a59a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a54160490 0x7f9a54160870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a59a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a5415c200 0x7f9a5415c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a59a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a540afdf0 con 0x7f9a540b0290 2024-03-30T00:47:45.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a59a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a540b0e30 con 0x7f9a54160490 2024-03-30T00:47:45.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a59a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a540b15d0 con 0x7f9a5415c200 2024-03-30T00:47:45.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a52a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a54160490 0x7f9a54160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a52a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a54160490 0x7f9a54160870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55434/0 (socket says 172.21.15.66:55434) 2024-03-30T00:47:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a52a00640 1 -- 172.21.15.66:0/907411949 learned_addr learned my addr 172.21.15.66:0/907411949 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a52a00640 1 -- 172.21.15.66:0/907411949 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a5415c200 msgr2=0x7f9a5415c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a52a00640 1 --2- 172.21.15.66:0/907411949 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a5415c200 0x7f9a5415c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a52a00640 1 -- 172.21.15.66:0/907411949 >> v1:172.21.15.156:6789/0 conn(0x7f9a540b0290 legacy=0x7f9a540b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a52a00640 1 -- 172.21.15.66:0/907411949 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a54160db0 con 0x7f9a54160490 2024-03-30T00:47:45.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a52a00640 1 --2- 172.21.15.66:0/907411949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a54160490 0x7f9a54160870 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009870 tx=0x7f9a4401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc96c6dceb65410 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:45.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a52000640 1 -- 172.21.15.66:0/907411949 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a440aa610 con 0x7f9a54160490 2024-03-30T00:47:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a52000640 1 -- 172.21.15.66:0/907411949 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a440a6040 con 0x7f9a54160490 2024-03-30T00:47:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.493+0000 7f9a52000640 1 -- 172.21.15.66:0/907411949 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a440aa990 con 0x7f9a54160490 2024-03-30T00:47:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 -- 172.21.15.66:0/907411949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a54160490 msgr2=0x7f9a54160870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 --2- 172.21.15.66:0/907411949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a54160490 0x7f9a54160870 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009870 tx=0x7f9a4401c4f0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 -- 172.21.15.66:0/907411949 shutdown_connections 2024-03-30T00:47:45.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 --2- 172.21.15.66:0/907411949 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a5415c200 0x7f9a5415c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 --2- 172.21.15.66:0/907411949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a54160490 0x7f9a54160870 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 -- 172.21.15.66:0/907411949 >> 172.21.15.66:0/907411949 conn(0x7f9a54099870 msgr2=0x7f9a54089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:45.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 -- 172.21.15.66:0/907411949 shutdown_connections 2024-03-30T00:47:45.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 -- 172.21.15.66:0/907411949 wait complete. 2024-03-30T00:47:45.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 Processor -- start 2024-03-30T00:47:45.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 -- start start 2024-03-30T00:47:45.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a540b0290 0x7f9a54152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a5415c200 0x7f9a540cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a54160490 0x7f9a540cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a541526d0 con 0x7f9a540b0290 2024-03-30T00:47:45.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a540d0710 con 0x7f9a54160490 2024-03-30T00:47:45.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a59a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54152830 con 0x7f9a5415c200 2024-03-30T00:47:45.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a52a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a540b0290 0x7f9a54152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a52a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a540b0290 0x7f9a54152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55436/0 (socket says 172.21.15.66:55436) 2024-03-30T00:47:45.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a52a00640 1 -- 172.21.15.66:0/1762981208 learned_addr learned my addr 172.21.15.66:0/1762981208 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a53400640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a54160490 0x7f9a540cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a4be00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a5415c200 0x7f9a540cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a52a00640 1 -- 172.21.15.66:0/1762981208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a5415c200 msgr2=0x7f9a540cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a52a00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a5415c200 0x7f9a540cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a52a00640 1 -- 172.21.15.66:0/1762981208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a54160490 msgr2=0x7f9a540cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a52a00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a54160490 0x7f9a540cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a52a00640 1 -- 172.21.15.66:0/1762981208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a440094d0 con 0x7f9a540b0290 2024-03-30T00:47:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a4be00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a5415c200 0x7f9a540cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.497+0000 7f9a53400640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a54160490 0x7f9a540cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.501+0000 7f9a52a00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a540b0290 0x7f9a54152190 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f9a440ac040 tx=0x7f9a4401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.501+0000 7f9a4aa00640 1 -- 172.21.15.66:0/1762981208 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a440bd5f0 con 0x7f9a540b0290 2024-03-30T00:47:45.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.501+0000 7f9a4aa00640 1 -- 172.21.15.66:0/1762981208 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a440bd770 con 0x7f9a540b0290 2024-03-30T00:47:45.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.501+0000 7f9a4aa00640 1 -- 172.21.15.66:0/1762981208 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a440bdb20 con 0x7f9a540b0290 2024-03-30T00:47:45.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.501+0000 7f9a59a00640 1 -- 172.21.15.66:0/1762981208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a540d0960 con 0x7f9a540b0290 2024-03-30T00:47:45.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.501+0000 7f9a59a00640 1 -- 172.21.15.66:0/1762981208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a54152d60 con 0x7f9a540b0290 2024-03-30T00:47:45.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.501+0000 7f9a59a00640 1 -- 172.21.15.66:0/1762981208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a54066c30 con 0x7f9a540b0290 2024-03-30T00:47:45.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.501+0000 7f9a4aa00640 1 -- 172.21.15.66:0/1762981208 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9a440b33f0 con 0x7f9a540b0290 2024-03-30T00:47:45.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.501+0000 7f9a4aa00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a30066690 0x7f9a30068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.501+0000 7f9a4aa00640 1 -- 172.21.15.66:0/1762981208 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f9a4411f030 con 0x7f9a540b0290 2024-03-30T00:47:45.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.501+0000 7f9a4be00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a30066690 0x7f9a30068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.505+0000 7f9a4be00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a30066690 0x7f9a30068b50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9a5406c8c0 tx=0x7f9a3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.505+0000 7f9a4aa00640 1 -- 172.21.15.66:0/1762981208 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9a440ec2d0 con 0x7f9a540b0290 2024-03-30T00:47:45.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.665+0000 7f9a59a00640 1 -- 172.21.15.66:0/1762981208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a5406d160 con 0x7f9a540b0290 2024-03-30T00:47:45.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a4aa00640 1 -- 172.21.15.66:0/1762981208 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a440ef8f0 con 0x7f9a540b0290 2024-03-30T00:47:45.677 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:47:45.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 -- 172.21.15.66:0/1762981208 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a30066690 msgr2=0x7f9a30068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a30066690 0x7f9a30068b50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9a5406c8c0 tx=0x7f9a3c008040 comp rx=0 tx=0).stop 2024-03-30T00:47:45.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 -- 172.21.15.66:0/1762981208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a540b0290 msgr2=0x7f9a54152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a540b0290 0x7f9a54152190 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f9a440ac040 tx=0x7f9a4401ad30 comp rx=0 tx=0).stop 2024-03-30T00:47:45.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 -- 172.21.15.66:0/1762981208 shutdown_connections 2024-03-30T00:47:45.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a54160490 0x7f9a540cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a5415c200 0x7f9a540cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a30066690 0x7f9a30068b50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 --2- 172.21.15.66:0/1762981208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a540b0290 0x7f9a54152190 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 -- 172.21.15.66:0/1762981208 >> 172.21.15.66:0/1762981208 conn(0x7f9a54099870 msgr2=0x7f9a5406eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:45.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 -- 172.21.15.66:0/1762981208 shutdown_connections 2024-03-30T00:47:45.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.669+0000 7f9a59a00640 1 -- 172.21.15.66:0/1762981208 wait complete. 2024-03-30T00:47:45.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:47:45.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.829+0000 7f4f9b600640 1 Processor -- start 2024-03-30T00:47:45.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.829+0000 7f4f9b600640 1 -- start start 2024-03-30T00:47:45.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f9b600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f940ae4e0 0x7f4f940cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f9b600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ce820 0x7f4f940cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f9b600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9406e7d0 con 0x7f4f9415ca10 2024-03-30T00:47:45.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f9b600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f940b1f70 con 0x7f4f940ae4e0 2024-03-30T00:47:45.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f9b600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f940cf1f0 con 0x7f4f940ce820 2024-03-30T00:47:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f8be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f940ae4e0 0x7f4f940cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f8be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f940ae4e0 0x7f4f940cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55452/0 (socket says 172.21.15.66:55452) 2024-03-30T00:47:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f8be00640 1 -- 172.21.15.66:0/490741385 learned_addr learned my addr 172.21.15.66:0/490741385 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f99600640 1 --2- 172.21.15.66:0/490741385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ce820 0x7f4f940cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f8be00640 1 -- 172.21.15.66:0/490741385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ce820 msgr2=0x7f4f940cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f8be00640 1 --2- 172.21.15.66:0/490741385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ce820 0x7f4f940cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f8be00640 1 -- 172.21.15.66:0/490741385 >> v1:172.21.15.156:6789/0 conn(0x7f4f9415ca10 legacy=0x7f4f9406ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f8be00640 1 -- 172.21.15.66:0/490741385 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9415cdf0 con 0x7f4f940ae4e0 2024-03-30T00:47:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.833+0000 7f4f99600640 1 --2- 172.21.15.66:0/490741385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ce820 0x7f4f940cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:45.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f8be00640 1 --2- 172.21.15.66:0/490741385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f940ae4e0 0x7f4f940cb620 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0098a0 tx=0x7f4f7c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32711b3c75cfefd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:45.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f93e00640 1 -- 172.21.15.66:0/490741385 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c0aa670 con 0x7f4f940ae4e0 2024-03-30T00:47:45.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f93e00640 1 -- 172.21.15.66:0/490741385 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7c0aa7d0 con 0x7f4f940ae4e0 2024-03-30T00:47:45.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f93e00640 1 -- 172.21.15.66:0/490741385 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c0aab60 con 0x7f4f940ae4e0 2024-03-30T00:47:45.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 -- 172.21.15.66:0/490741385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f940ae4e0 msgr2=0x7f4f940cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 --2- 172.21.15.66:0/490741385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f940ae4e0 0x7f4f940cb620 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0098a0 tx=0x7f4f7c01c510 comp rx=0 tx=0).stop 2024-03-30T00:47:45.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 -- 172.21.15.66:0/490741385 shutdown_connections 2024-03-30T00:47:45.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 --2- 172.21.15.66:0/490741385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ce820 0x7f4f940cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 --2- 172.21.15.66:0/490741385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f940ae4e0 0x7f4f940cb620 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 -- 172.21.15.66:0/490741385 >> 172.21.15.66:0/490741385 conn(0x7f4f94098a20 msgr2=0x7f4f940b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:45.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 -- 172.21.15.66:0/490741385 shutdown_connections 2024-03-30T00:47:45.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 -- 172.21.15.66:0/490741385 wait complete. 2024-03-30T00:47:45.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 Processor -- start 2024-03-30T00:47:45.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 -- start start 2024-03-30T00:47:45.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ae4e0 0x7f4f94154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f940ce820 0x7f4f940b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9415ca10 0x7f4f9415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f940b3920 con 0x7f4f9415ca10 2024-03-30T00:47:45.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f940b3db0 con 0x7f4f940ce820 2024-03-30T00:47:45.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.837+0000 7f4f9b600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f940b2d10 con 0x7f4f940ae4e0 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f99600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9415ca10 0x7f4f9415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f99600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9415ca10 0x7f4f9415cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55464/0 (socket says 172.21.15.66:55464) 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f99600640 1 -- 172.21.15.66:0/1752671369 learned_addr learned my addr 172.21.15.66:0/1752671369 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f98c00640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ae4e0 0x7f4f94154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f8be00640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f940ce820 0x7f4f940b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f99600640 1 -- 172.21.15.66:0/1752671369 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ae4e0 msgr2=0x7f4f94154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f99600640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ae4e0 0x7f4f94154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f99600640 1 -- 172.21.15.66:0/1752671369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f940ce820 msgr2=0x7f4f940b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f99600640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f940ce820 0x7f4f940b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f99600640 1 -- 172.21.15.66:0/1752671369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c0094d0 con 0x7f4f9415ca10 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f98c00640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ae4e0 0x7f4f94154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f99600640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9415ca10 0x7f4f9415cfd0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009fa0 tx=0x7f4f8c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f91600640 1 -- 172.21.15.66:0/1752671369 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c003ba0 con 0x7f4f9415ca10 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f91600640 1 -- 172.21.15.66:0/1752671369 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c00f040 con 0x7f4f9415ca10 2024-03-30T00:47:45.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f9b600640 1 -- 172.21.15.66:0/1752671369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f940b2ff0 con 0x7f4f9415ca10 2024-03-30T00:47:45.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f91600640 1 -- 172.21.15.66:0/1752671369 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c004030 con 0x7f4f9415ca10 2024-03-30T00:47:45.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f9b600640 1 -- 172.21.15.66:0/1752671369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f940b35b0 con 0x7f4f9415ca10 2024-03-30T00:47:45.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f9b600640 1 -- 172.21.15.66:0/1752671369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f940b3a90 con 0x7f4f9415ca10 2024-03-30T00:47:45.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.841+0000 7f4f91600640 1 -- 172.21.15.66:0/1752671369 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4f8c004190 con 0x7f4f9415ca10 2024-03-30T00:47:45.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.845+0000 7f4f91600640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f70066690 0x7f4f70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:45.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.845+0000 7f4f91600640 1 -- 172.21.15.66:0/1752671369 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f4f8c084730 con 0x7f4f9415ca10 2024-03-30T00:47:45.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.845+0000 7f4f98c00640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f70066690 0x7f4f70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:45.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.845+0000 7f4f98c00640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f70066690 0x7f4f70068b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4f940c9860 tx=0x7f4f84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:45.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:45.849+0000 7f4f91600640 1 -- 172.21.15.66:0/1752671369 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4f8c0518a0 con 0x7f4f9415ca10 2024-03-30T00:47:46.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.021+0000 7f4f9b600640 1 -- 172.21.15.66:0/1752671369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4f94066c30 con 0x7f4f9415ca10 2024-03-30T00:47:46.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.021+0000 7f4f91600640 1 -- 172.21.15.66:0/1752671369 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v97) v1 ==== 80+0+122 (secure 0 0 0) 0x7f4f8c054ec0 con 0x7f4f9415ca10 2024-03-30T00:47:46.032 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6836/1147664160 2024-03-31T00:46:34.103831+0000 2024-03-30T00:47:46.033 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6835/1147664160 2024-03-31T00:46:34.103831+0000 2024-03-30T00:47:46.033 INFO:teuthology.orchestra.run.smithi066.stderr:listed 2 entries 2024-03-30T00:47:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 -- 172.21.15.66:0/1752671369 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f70066690 msgr2=0x7f4f70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f70066690 0x7f4f70068b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4f940c9860 tx=0x7f4f84009040 comp rx=0 tx=0).stop 2024-03-30T00:47:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 -- 172.21.15.66:0/1752671369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9415ca10 msgr2=0x7f4f9415cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9415ca10 0x7f4f9415cfd0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009fa0 tx=0x7f4f8c00d560 comp rx=0 tx=0).stop 2024-03-30T00:47:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 -- 172.21.15.66:0/1752671369 shutdown_connections 2024-03-30T00:47:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f70066690 0x7f4f70068b50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9415ca10 0x7f4f9415cfd0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f940ce820 0x7f4f940b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 --2- 172.21.15.66:0/1752671369 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f940ae4e0 0x7f4f94154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 -- 172.21.15.66:0/1752671369 >> 172.21.15.66:0/1752671369 conn(0x7f4f94098a20 msgr2=0x7f4f9415bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 -- 172.21.15.66:0/1752671369 shutdown_connections 2024-03-30T00:47:46.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.025+0000 7f4f9b600640 1 -- 172.21.15.66:0/1752671369 wait complete. 2024-03-30T00:47:46.054 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:47:46.054 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:47:46.063 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:47:46.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.233+0000 7f000ac00640 1 Processor -- start 2024-03-30T00:47:46.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.233+0000 7f000ac00640 1 -- start start 2024-03-30T00:47:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.233+0000 7f000ac00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f000415ca10 0x7f000406ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.233+0000 7f000ac00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00040ce820 0x7f00040cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.233+0000 7f000ac00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000406e7d0 con 0x7f00040ae4e0 2024-03-30T00:47:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.237+0000 7f000ac00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00040b1f70 con 0x7f000415ca10 2024-03-30T00:47:46.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.237+0000 7f000ac00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00040cf1f0 con 0x7f00040ce820 2024-03-30T00:47:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.237+0000 7f0008c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00040ce820 0x7f00040cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.237+0000 7f0003e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f000415ca10 0x7f000406ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.237+0000 7f0003e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f000415ca10 0x7f000406ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:38222/0 (socket says 172.21.15.66:38222) 2024-03-30T00:47:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.237+0000 7f0008c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00040ce820 0x7f00040cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:40330/0 (socket says 172.21.15.66:40330) 2024-03-30T00:47:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.237+0000 7f0003e00640 1 -- 172.21.15.66:0/2231982887 learned_addr learned my addr 172.21.15.66:0/2231982887 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.237+0000 7f0003e00640 1 -- 172.21.15.66:0/2231982887 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00040ce820 msgr2=0x7f00040cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.237+0000 7f0003e00640 1 --2- 172.21.15.66:0/2231982887 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00040ce820 0x7f00040cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.237+0000 7f0003e00640 1 -- 172.21.15.66:0/2231982887 >> v1:172.21.15.156:6789/0 conn(0x7f00040ae4e0 legacy=0x7f00040cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.237+0000 7f0003e00640 1 -- 172.21.15.66:0/2231982887 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000406f1a0 con 0x7f000415ca10 2024-03-30T00:47:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f0003e00640 1 --2- 172.21.15.66:0/2231982887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f000415ca10 0x7f000406ec60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efff40098a0 tx=0x7efff401c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=316fe1410a9c15cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f0003400640 1 -- 172.21.15.66:0/2231982887 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40a9620 con 0x7f000415ca10 2024-03-30T00:47:46.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f0003400640 1 -- 172.21.15.66:0/2231982887 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff40a9780 con 0x7f000415ca10 2024-03-30T00:47:46.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f0003400640 1 -- 172.21.15.66:0/2231982887 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40a9b10 con 0x7f000415ca10 2024-03-30T00:47:46.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 -- 172.21.15.66:0/2231982887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f000415ca10 msgr2=0x7f000406ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 --2- 172.21.15.66:0/2231982887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f000415ca10 0x7f000406ec60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efff40098a0 tx=0x7efff401c540 comp rx=0 tx=0).stop 2024-03-30T00:47:46.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 -- 172.21.15.66:0/2231982887 shutdown_connections 2024-03-30T00:47:46.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 --2- 172.21.15.66:0/2231982887 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00040ce820 0x7f00040cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 --2- 172.21.15.66:0/2231982887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f000415ca10 0x7f000406ec60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 -- 172.21.15.66:0/2231982887 >> 172.21.15.66:0/2231982887 conn(0x7f0004098a20 msgr2=0x7f00040b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:46.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 -- 172.21.15.66:0/2231982887 shutdown_connections 2024-03-30T00:47:46.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 -- 172.21.15.66:0/2231982887 wait complete. 2024-03-30T00:47:46.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 Processor -- start 2024-03-30T00:47:46.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 -- start start 2024-03-30T00:47:46.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00040ae4e0 0x7f00041555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.241+0000 7f000ac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00040ce820 0x7f0004157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f000ac00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f000415ca10 0x7f000415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f000ac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0004155b20 con 0x7f00040ce820 2024-03-30T00:47:46.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f000ac00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0004156700 con 0x7f00040ae4e0 2024-03-30T00:47:46.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f000ac00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00041569e0 con 0x7f000415ca10 2024-03-30T00:47:46.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7efffbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00040ce820 0x7f0004157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0003e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00040ae4e0 0x7f00041555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7efffbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00040ce820 0x7f0004157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55466/0 (socket says 172.21.15.66:55466) 2024-03-30T00:47:46.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7efffbe00640 1 -- 172.21.15.66:0/4142189314 learned_addr learned my addr 172.21.15.66:0/4142189314 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:46.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0008c00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f000415ca10 0x7f000415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0003e00640 1 -- 172.21.15.66:0/4142189314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f000415ca10 msgr2=0x7f000415a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0003e00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f000415ca10 0x7f000415a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0003e00640 1 -- 172.21.15.66:0/4142189314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00040ce820 msgr2=0x7f0004157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0003e00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00040ce820 0x7f0004157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0003e00640 1 -- 172.21.15.66:0/4142189314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff40094d0 con 0x7f00040ae4e0 2024-03-30T00:47:46.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7efffbe00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00040ce820 0x7f0004157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:47:46.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0003e00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00040ae4e0 0x7f00041555e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efff40aa000 tx=0x7efff40192e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:46.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0000c00640 1 -- 172.21.15.66:0/4142189314 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40b95f0 con 0x7f00040ae4e0 2024-03-30T00:47:46.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0000c00640 1 -- 172.21.15.66:0/4142189314 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff40b9750 con 0x7f00040ae4e0 2024-03-30T00:47:46.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0000c00640 1 -- 172.21.15.66:0/4142189314 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40b9b00 con 0x7f00040ae4e0 2024-03-30T00:47:46.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f000ac00640 1 -- 172.21.15.66:0/4142189314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0004156cc0 con 0x7f00040ae4e0 2024-03-30T00:47:46.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f000ac00640 1 -- 172.21.15.66:0/4142189314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00040c3e60 con 0x7f00040ae4e0 2024-03-30T00:47:46.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f000ac00640 1 -- 172.21.15.66:0/4142189314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0004157170 con 0x7f00040ae4e0 2024-03-30T00:47:46.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.245+0000 7f0000c00640 1 -- 172.21.15.66:0/4142189314 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efff40b9c60 con 0x7f00040ae4e0 2024-03-30T00:47:46.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.249+0000 7f0000c00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7effe0066690 0x7effe0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.249+0000 7f0000c00640 1 -- 172.21.15.66:0/4142189314 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7efff411edd0 con 0x7f00040ae4e0 2024-03-30T00:47:46.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.249+0000 7efffbe00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7effe0066690 0x7effe0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.249+0000 7efffbe00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7effe0066690 0x7effe0068b50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f00040c3c50 tx=0x7effec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:46.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.253+0000 7f0000c00640 1 -- 172.21.15.66:0/4142189314 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efff40ebfc0 con 0x7f00040ae4e0 2024-03-30T00:47:46.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.413+0000 7f000ac00640 1 -- 172.21.15.66:0/4142189314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0004156b20 con 0x7f00040ae4e0 2024-03-30T00:47:46.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f0000c00640 1 -- 172.21.15.66:0/4142189314 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16571 (secure 0 0 0) 0x7f0004156b20 con 0x7f00040ae4e0 2024-03-30T00:47:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:47:46.426 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":97,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:46:34.103855+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:46:13.811742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-30T00:46:14.265142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":7}]},{"pgid":"14.1a","mappings":[{"from":1,"to":5}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6836/1147664160":"2024-03-31T00:46:34.103831+0000","172.21.15.66:6835/1147664160":"2024-03-31T00:46:34.103831+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:47:46.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 -- 172.21.15.66:0/4142189314 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7effe0066690 msgr2=0x7effe0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7effe0066690 0x7effe0068b50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f00040c3c50 tx=0x7effec009040 comp rx=0 tx=0).stop 2024-03-30T00:47:46.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 -- 172.21.15.66:0/4142189314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00040ae4e0 msgr2=0x7f00041555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00040ae4e0 0x7f00041555e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efff40aa000 tx=0x7efff40192e0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 -- 172.21.15.66:0/4142189314 shutdown_connections 2024-03-30T00:47:46.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f000415ca10 0x7f000415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7effe0066690 0x7effe0068b50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00040ce820 0x7f0004157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 --2- 172.21.15.66:0/4142189314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00040ae4e0 0x7f00041555e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 -- 172.21.15.66:0/4142189314 >> 172.21.15.66:0/4142189314 conn(0x7f0004098a20 msgr2=0x7f00040cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:46.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 -- 172.21.15.66:0/4142189314 shutdown_connections 2024-03-30T00:47:46.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.417+0000 7f000ac00640 1 -- 172.21.15.66:0/4142189314 wait complete. 2024-03-30T00:47:46.447 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:46:13.811742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:46:14.265142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:47:46.448 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:47:46.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.581+0000 7f9121c00640 1 Processor -- start 2024-03-30T00:47:46.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.581+0000 7f9121c00640 1 -- start start 2024-03-30T00:47:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9121c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f911c0b3170 0x7f911c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9121c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911c0cb5b0 con 0x7f911c15a720 2024-03-30T00:47:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9121c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911c0cb080 con 0x7f911c0d07c0 2024-03-30T00:47:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9121c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0cfb00 con 0x7f911c0b3170 2024-03-30T00:47:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f911c15a720 0x7f911c0ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55346/0 (socket says 172.21.15.66:55346) 2024-03-30T00:47:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911be00640 1 -- 172.21.15.66:0/973584357 learned_addr learned my addr 172.21.15.66:0/973584357 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:46.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9113e00640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f911c0b3170 0x7f911c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642570978 0 0) 0x7f910c001120 con 0x7f911c15a720 2024-03-30T00:47:46.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9100003530 con 0x7f911c15a720 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3640040800 0 0) 0x7f9100003530 con 0x7f911c15a720 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f911c0b3170 msgr2=0x7f911c0b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f911c0b3170 0x7f911c0b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 >> v1:172.21.15.156:6789/0 conn(0x7f911c0d07c0 legacy=0x7f911c0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c15ab00 con 0x7f911c15a720 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f910c001810 con 0x7f911c15a720 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 >> v1:172.21.15.66:6789/0 conn(0x7f911c15a720 legacy=0x7f911c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91000039b0 0x7f9100003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91000047d0 0x7f9100004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f911c0b3170 0x7f9100005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0cfb00 con 0x7f91000047d0 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910000ba40 con 0x7f91000039b0 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910000bc00 con 0x7f911c0b3170 2024-03-30T00:47:46.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9113e00640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91000047d0 0x7f9100004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f911be00640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91000039b0 0x7f9100003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9121200640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f911c0b3170 0x7f9100005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9113e00640 1 -- 172.21.15.66:0/973584357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f911c0b3170 msgr2=0x7f9100005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9113e00640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f911c0b3170 0x7f9100005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9113e00640 1 -- 172.21.15.66:0/973584357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91000039b0 msgr2=0x7f9100003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9113e00640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91000039b0 0x7f9100003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.585+0000 7f9113e00640 1 -- 172.21.15.66:0/973584357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9100003530 con 0x7f91000047d0 2024-03-30T00:47:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f9113e00640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91000047d0 0x7f9100004c50 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f911c0b3ab0 tx=0x7f910401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db3364fea99554d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040a64e0 con 0x7f91000047d0 2024-03-30T00:47:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91040a6640 con 0x7f91000047d0 2024-03-30T00:47:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f911b400640 1 -- 172.21.15.66:0/973584357 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040a69b0 con 0x7f91000047d0 2024-03-30T00:47:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f9121c00640 1 -- 172.21.15.66:0/973584357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91000047d0 msgr2=0x7f9100004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f9121c00640 1 --2- 172.21.15.66:0/973584357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91000047d0 0x7f9100004c50 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f911c0b3ab0 tx=0x7f910401c9f0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f9121200640 1 -- 172.21.15.66:0/973584357 reap_dead start 2024-03-30T00:47:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f9121c00640 1 -- 172.21.15.66:0/973584357 shutdown_connections 2024-03-30T00:47:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f9121c00640 1 -- 172.21.15.66:0/973584357 >> 172.21.15.66:0/973584357 conn(0x7f911c069d90 msgr2=0x7f911c06a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f9121c00640 1 -- 172.21.15.66:0/973584357 shutdown_connections 2024-03-30T00:47:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f9121c00640 1 -- 172.21.15.66:0/973584357 wait complete. 2024-03-30T00:47:46.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f9121c00640 1 Processor -- start 2024-03-30T00:47:46.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.589+0000 7f9121c00640 1 -- start start 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91000047d0 0x7f911c1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f911c0ade80 0x7f911c0ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c15c310 0x7f911c15c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c15daf0 con 0x7f911c15c310 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c15cfe0 con 0x7f911c0ade80 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c15d250 con 0x7f91000047d0 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c15c310 0x7f911c15c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c15c310 0x7f911c15c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55488/0 (socket says 172.21.15.66:55488) 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121200640 1 -- 172.21.15.66:0/952131350 learned_addr learned my addr 172.21.15.66:0/952131350 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f911be00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91000047d0 0x7f911c1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9113e00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f911c0ade80 0x7f911c0ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121200640 1 -- 172.21.15.66:0/952131350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91000047d0 msgr2=0x7f911c1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121200640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91000047d0 0x7f911c1517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121200640 1 -- 172.21.15.66:0/952131350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f911c0ade80 msgr2=0x7f911c0ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121200640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f911c0ade80 0x7f911c0ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121200640 1 -- 172.21.15.66:0/952131350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c15d530 con 0x7f911c15c310 2024-03-30T00:47:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f911be00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91000047d0 0x7f911c1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9113e00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f911c0ade80 0x7f911c0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121200640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c15c310 0x7f911c15c790 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f911c0cd990 tx=0x7f911400fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9118c00640 1 -- 172.21.15.66:0/952131350 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114006d70 con 0x7f911c15c310 2024-03-30T00:47:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9118c00640 1 -- 172.21.15.66:0/952131350 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9114003980 con 0x7f911c15c310 2024-03-30T00:47:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9118c00640 1 -- 172.21.15.66:0/952131350 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114003d30 con 0x7f911c15c310 2024-03-30T00:47:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121c00640 1 -- 172.21.15.66:0/952131350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c15d810 con 0x7f911c15c310 2024-03-30T00:47:46.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121c00640 1 -- 172.21.15.66:0/952131350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c159cf0 con 0x7f911c15c310 2024-03-30T00:47:46.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.593+0000 7f9121c00640 1 -- 172.21.15.66:0/952131350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911c073e00 con 0x7f911c15c310 2024-03-30T00:47:46.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.597+0000 7f9118c00640 1 -- 172.21.15.66:0/952131350 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9114005110 con 0x7f911c15c310 2024-03-30T00:47:46.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.597+0000 7f9118c00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f90f80666c0 0x7f90f8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.597+0000 7f9118c00640 1 -- 172.21.15.66:0/952131350 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f9114083a30 con 0x7f911c15c310 2024-03-30T00:47:46.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.597+0000 7f911be00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f90f80666c0 0x7f90f8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.597+0000 7f911be00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f90f80666c0 0x7f90f8068b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f910c000c10 tx=0x7f910c001120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.597+0000 7f9118c00640 1 -- 172.21.15.66:0/952131350 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9114050c00 con 0x7f911c15c310 2024-03-30T00:47:46.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 -- 172.21.15.66:0/952131350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f911c15dc30 con 0x7f911c15c310 2024-03-30T00:47:46.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9118c00640 1 -- 172.21.15.66:0/952131350 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9114054220 con 0x7f911c15c310 2024-03-30T00:47:46.770 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:47:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 -- 172.21.15.66:0/952131350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f90f80666c0 msgr2=0x7f90f8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f90f80666c0 0x7f90f8068b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f910c000c10 tx=0x7f910c001120 comp rx=0 tx=0).stop 2024-03-30T00:47:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 -- 172.21.15.66:0/952131350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c15c310 msgr2=0x7f911c15c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c15c310 0x7f911c15c790 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f911c0cd990 tx=0x7f911400fcd0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 -- 172.21.15.66:0/952131350 shutdown_connections 2024-03-30T00:47:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f90f80666c0 0x7f90f8068b80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c15c310 0x7f911c15c790 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f911c0ade80 0x7f911c0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 --2- 172.21.15.66:0/952131350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91000047d0 0x7f911c1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 -- 172.21.15.66:0/952131350 >> 172.21.15.66:0/952131350 conn(0x7f911c069d90 msgr2=0x7f911c15e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 -- 172.21.15.66:0/952131350 shutdown_connections 2024-03-30T00:47:46.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.761+0000 7f9121c00640 1 -- 172.21.15.66:0/952131350 wait complete. 2024-03-30T00:47:46.791 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:47:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.925+0000 7f4517600640 1 Processor -- start 2024-03-30T00:47:46.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.925+0000 7f4517600640 1 -- start start 2024-03-30T00:47:46.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.925+0000 7f4517600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45100cc560 0x7f45100d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.925+0000 7f4517600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451006b0b0 con 0x7f451006e800 2024-03-30T00:47:46.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.925+0000 7f4517600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451006b390 con 0x7f451006f8e0 2024-03-30T00:47:46.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.925+0000 7f4517600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4510160640 con 0x7f45100cc560 2024-03-30T00:47:46.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.925+0000 7f4515600640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f451006e800 0x7f45100ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55360/0 (socket says 172.21.15.66:55360) 2024-03-30T00:47:46.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.925+0000 7f4515600640 1 -- 172.21.15.66:0/2977883185 learned_addr learned my addr 172.21.15.66:0/2977883185 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:46.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f4514c00640 1 --2- 172.21.15.66:0/2977883185 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45100cc560 0x7f45100d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f450fe00640 1 -- 172.21.15.66:0/2977883185 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107077241 0 0) 0x7f451006b0b0 con 0x7f451006e800 2024-03-30T00:47:46.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f450fe00640 1 -- 172.21.15.66:0/2977883185 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44f4003530 con 0x7f451006e800 2024-03-30T00:47:46.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f450fe00640 1 -- 172.21.15.66:0/2977883185 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492967978 0 0) 0x7f451006b390 con 0x7f451006f8e0 2024-03-30T00:47:46.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f450fe00640 1 -- 172.21.15.66:0/2977883185 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f451006b0b0 con 0x7f451006f8e0 2024-03-30T00:47:46.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f4514c00640 1 -- 172.21.15.66:0/2977883185 >> v1:172.21.15.156:6789/0 conn(0x7f451006f8e0 legacy=0x7f451006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f4514c00640 1 -- 172.21.15.66:0/2977883185 >> v1:172.21.15.66:6789/0 conn(0x7f451006e800 legacy=0x7f45100ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f4514c00640 1 -- 172.21.15.66:0/2977883185 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451006b670 con 0x7f45100cc560 2024-03-30T00:47:46.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f4514c00640 1 --2- 172.21.15.66:0/2977883185 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45100cc560 0x7f45100d1060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f45000097f0 tx=0x7f450001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e166012e5a2417a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:46.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f450fe00640 1 -- 172.21.15.66:0/2977883185 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45000ac620 con 0x7f45100cc560 2024-03-30T00:47:46.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f450fe00640 1 -- 172.21.15.66:0/2977883185 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45000ac780 con 0x7f45100cc560 2024-03-30T00:47:46.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.929+0000 7f450fe00640 1 -- 172.21.15.66:0/2977883185 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45000acaf0 con 0x7f45100cc560 2024-03-30T00:47:46.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 -- 172.21.15.66:0/2977883185 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45100cc560 msgr2=0x7f45100d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 --2- 172.21.15.66:0/2977883185 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45100cc560 0x7f45100d1060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f45000097f0 tx=0x7f450001c4b0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 -- 172.21.15.66:0/2977883185 shutdown_connections 2024-03-30T00:47:46.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 --2- 172.21.15.66:0/2977883185 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45100cc560 0x7f45100d1060 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 -- 172.21.15.66:0/2977883185 >> 172.21.15.66:0/2977883185 conn(0x7f45100d0540 msgr2=0x7f45100d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:46.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 -- 172.21.15.66:0/2977883185 shutdown_connections 2024-03-30T00:47:46.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 -- 172.21.15.66:0/2977883185 wait complete. 2024-03-30T00:47:46.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 Processor -- start 2024-03-30T00:47:46.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 -- start start 2024-03-30T00:47:46.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451006e800 0x7f4510154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f451006f8e0 0x7f45101571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45100cc560 0x7f451015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45100c87b0 con 0x7f451006e800 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45100c8a90 con 0x7f451006f8e0 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4517600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45100c8d70 con 0x7f45100cc560 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4514c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451006e800 0x7f4510154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4514c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451006e800 0x7f4510154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55498/0 (socket says 172.21.15.66:55498) 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4514c00640 1 -- 172.21.15.66:0/2599099310 learned_addr learned my addr 172.21.15.66:0/2599099310 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4515600640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45100cc560 0x7f451015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4507e00640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f451006f8e0 0x7f45101571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4514c00640 1 -- 172.21.15.66:0/2599099310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45100cc560 msgr2=0x7f451015c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4514c00640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45100cc560 0x7f451015c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4514c00640 1 -- 172.21.15.66:0/2599099310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f451006f8e0 msgr2=0x7f45101571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:46.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4514c00640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f451006f8e0 0x7f45101571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:46.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.933+0000 7f4514c00640 1 -- 172.21.15.66:0/2599099310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4500009420 con 0x7f451006e800 2024-03-30T00:47:46.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.937+0000 7f4514c00640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451006e800 0x7f4510154a10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f450001a0e0 tx=0x7f4500019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:46.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.937+0000 7f450d600640 1 -- 172.21.15.66:0/2599099310 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45000ac620 con 0x7f451006e800 2024-03-30T00:47:46.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.937+0000 7f450d600640 1 -- 172.21.15.66:0/2599099310 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45000ac8f0 con 0x7f451006e800 2024-03-30T00:47:46.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.937+0000 7f450d600640 1 -- 172.21.15.66:0/2599099310 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45000acc60 con 0x7f451006e800 2024-03-30T00:47:46.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.937+0000 7f4517600640 1 -- 172.21.15.66:0/2599099310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45100c9050 con 0x7f451006e800 2024-03-30T00:47:46.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.937+0000 7f4517600640 1 -- 172.21.15.66:0/2599099310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4510154f50 con 0x7f451006e800 2024-03-30T00:47:46.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.937+0000 7f4517600640 1 -- 172.21.15.66:0/2599099310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4510066c30 con 0x7f451006e800 2024-03-30T00:47:46.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.937+0000 7f450d600640 1 -- 172.21.15.66:0/2599099310 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f45000aa070 con 0x7f451006e800 2024-03-30T00:47:46.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.937+0000 7f450d600640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44ec066690 0x7f44ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:46.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.937+0000 7f450d600640 1 -- 172.21.15.66:0/2599099310 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f450011e8c0 con 0x7f451006e800 2024-03-30T00:47:46.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.937+0000 7f4507e00640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44ec066690 0x7f44ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:46.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.941+0000 7f4507e00640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44ec066690 0x7f44ec068b50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f451006e770 tx=0x7f44f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:46.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:46.941+0000 7f450d600640 1 -- 172.21.15.66:0/2599099310 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f45000ebc60 con 0x7f451006e800 2024-03-30T00:47:47.057 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:47:47.062 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:47:47.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.101+0000 7f4517600640 1 -- 172.21.15.66:0/2599099310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45100c9500 con 0x7f451006e800 2024-03-30T00:47:47.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.101+0000 7f450d600640 1 -- 172.21.15.66:0/2599099310 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45000ef280 con 0x7f451006e800 2024-03-30T00:47:47.113 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:47:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 -- 172.21.15.66:0/2599099310 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44ec066690 msgr2=0x7f44ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44ec066690 0x7f44ec068b50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f451006e770 tx=0x7f44f8009040 comp rx=0 tx=0).stop 2024-03-30T00:47:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 -- 172.21.15.66:0/2599099310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451006e800 msgr2=0x7f4510154a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451006e800 0x7f4510154a10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f450001a0e0 tx=0x7f4500019250 comp rx=0 tx=0).stop 2024-03-30T00:47:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 -- 172.21.15.66:0/2599099310 shutdown_connections 2024-03-30T00:47:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45100cc560 0x7f451015c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f451006f8e0 0x7f45101571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44ec066690 0x7f44ec068b50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 --2- 172.21.15.66:0/2599099310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451006e800 0x7f4510154a10 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 -- 172.21.15.66:0/2599099310 >> 172.21.15.66:0/2599099310 conn(0x7f45100d0540 msgr2=0x7f451015b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:47.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 -- 172.21.15.66:0/2599099310 shutdown_connections 2024-03-30T00:47:47.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.105+0000 7f4517600640 1 -- 172.21.15.66:0/2599099310 wait complete. 2024-03-30T00:47:47.134 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:47:47.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa469e00640 1 Processor -- start 2024-03-30T00:47:47.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa469e00640 1 -- start start 2024-03-30T00:47:47.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa469e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa4640cda90 0x7fa4640ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:47.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa469e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa46415a370 0x7fa4640cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa469e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46415a990 con 0x7fa4640cde70 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa469e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4640b1b80 con 0x7fa4640cda90 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa469e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4640cd3e0 con 0x7fa46415a370 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa462a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa4640cda90 0x7fa4640ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa45be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa4640cde70 0x7fa464159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55376/0 (socket says 172.21.15.66:55376) 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa45be00640 1 -- 172.21.15.66:0/3653306108 learned_addr learned my addr 172.21.15.66:0/3653306108 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa462a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa4640cda90 0x7fa4640ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55506/0 (socket says 172.21.15.66:55506) 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa463400640 1 --2- 172.21.15.66:0/3653306108 >> v2:172.21.15.156:3300/0 conn(0x7fa46415a370 0x7fa4640cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa462a00640 1 -- 172.21.15.66:0/3653306108 >> v2:172.21.15.156:3300/0 conn(0x7fa46415a370 msgr2=0x7fa4640cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa462a00640 1 --2- 172.21.15.66:0/3653306108 >> v2:172.21.15.156:3300/0 conn(0x7fa46415a370 0x7fa4640cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa462a00640 1 -- 172.21.15.66:0/3653306108 >> v1:172.21.15.66:6789/0 conn(0x7fa4640cde70 legacy=0x7fa464159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa462000640 1 -- 172.21.15.66:0/3653306108 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865954215 0 0) 0x7fa46415a990 con 0x7fa4640cde70 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa462a00640 1 -- 172.21.15.66:0/3653306108 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4640b11e0 con 0x7fa4640cda90 2024-03-30T00:47:47.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.317+0000 7fa463400640 1 --2- 172.21.15.66:0/3653306108 >> v2:172.21.15.156:3300/0 conn(0x7fa46415a370 0x7fa4640cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa462a00640 1 --2- 172.21.15.66:0/3653306108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4640cda90 0x7fa4640ce520 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa4640b3140 tx=0x7fa45401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f229ba5fda72ca49 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa462000640 1 -- 172.21.15.66:0/3653306108 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4540ab600 con 0x7fa4640cda90 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa462000640 1 -- 172.21.15.66:0/3653306108 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4540a7050 con 0x7fa4640cda90 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa462000640 1 -- 172.21.15.66:0/3653306108 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4540ab980 con 0x7fa4640cda90 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa469e00640 1 -- 172.21.15.66:0/3653306108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4640cda90 msgr2=0x7fa4640ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa469e00640 1 --2- 172.21.15.66:0/3653306108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4640cda90 0x7fa4640ce520 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa4640b3140 tx=0x7fa45401c3f0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa469e00640 1 -- 172.21.15.66:0/3653306108 shutdown_connections 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa469e00640 1 --2- 172.21.15.66:0/3653306108 >> v2:172.21.15.156:3300/0 conn(0x7fa46415a370 0x7fa4640cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa469e00640 1 --2- 172.21.15.66:0/3653306108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4640cda90 0x7fa4640ce520 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa469e00640 1 -- 172.21.15.66:0/3653306108 >> 172.21.15.66:0/3653306108 conn(0x7fa464069f60 msgr2=0x7fa46406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa469e00640 1 -- 172.21.15.66:0/3653306108 shutdown_connections 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.321+0000 7fa469e00640 1 -- 172.21.15.66:0/3653306108 wait complete. 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa469e00640 1 Processor -- start 2024-03-30T00:47:47.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa469e00640 1 -- start start 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa469e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4640cda90 0x7fa46406db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa469e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4640cde70 0x7fa46406ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa469e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa46415a370 0x7fa4641591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa469e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464159700 con 0x7fa4640cde70 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa469e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464071810 con 0x7fa4640cda90 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa469e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa464072000 con 0x7fa46415a370 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4640cde70 0x7fa46406ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4640cde70 0x7fa46406ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55522/0 (socket says 172.21.15.66:55522) 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45be00640 1 -- 172.21.15.66:0/3843999105 learned_addr learned my addr 172.21.15.66:0/3843999105 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa462a00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4640cda90 0x7fa46406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa463400640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa46415a370 0x7fa4641591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45be00640 1 -- 172.21.15.66:0/3843999105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa46415a370 msgr2=0x7fa4641591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:47.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45be00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa46415a370 0x7fa4641591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45be00640 1 -- 172.21.15.66:0/3843999105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4640cda90 msgr2=0x7fa46406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45be00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4640cda90 0x7fa46406db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45be00640 1 -- 172.21.15.66:0/3843999105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa454009420 con 0x7fa4640cde70 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa463400640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa46415a370 0x7fa4641591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45be00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4640cde70 0x7fa46406ff40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c4e0 tx=0x7fa44c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45b400640 1 -- 172.21.15.66:0/3843999105 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c0074e0 con 0x7fa4640cde70 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45b400640 1 -- 172.21.15.66:0/3843999105 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c007660 con 0x7fa4640cde70 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa45b400640 1 -- 172.21.15.66:0/3843999105 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c004a30 con 0x7fa4640cde70 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa469e00640 1 -- 172.21.15.66:0/3843999105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46406d680 con 0x7fa4640cde70 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa469e00640 1 -- 172.21.15.66:0/3843999105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa464071d20 con 0x7fa4640cde70 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.325+0000 7fa469e00640 1 -- 172.21.15.66:0/3843999105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46406d810 con 0x7fa4640cde70 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.329+0000 7fa45b400640 1 -- 172.21.15.66:0/3843999105 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa44c004b90 con 0x7fa4640cde70 2024-03-30T00:47:47.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.329+0000 7fa45b400640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa440066690 0x7fa440068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:47.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.329+0000 7fa45b400640 1 -- 172.21.15.66:0/3843999105 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fa44c083580 con 0x7fa4640cde70 2024-03-30T00:47:47.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.329+0000 7fa462a00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa440066690 0x7fa440068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:47.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.329+0000 7fa462a00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa440066690 0x7fa440068b50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa4540ad000 tx=0x7fa4540a4f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:47.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.333+0000 7fa45b400640 1 -- 172.21.15.66:0/3843999105 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa44c050830 con 0x7fa4640cde70 2024-03-30T00:47:47.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 -- 172.21.15.66:0/3843999105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa464066c30 con 0x7fa4640cde70 2024-03-30T00:47:47.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa45b400640 1 -- 172.21.15.66:0/3843999105 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa44c0180a0 con 0x7fa4640cde70 2024-03-30T00:47:47.863 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:47:47.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 -- 172.21.15.66:0/3843999105 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa440066690 msgr2=0x7fa440068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:47.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa440066690 0x7fa440068b50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa4540ad000 tx=0x7fa4540a4f90 comp rx=0 tx=0).stop 2024-03-30T00:47:47.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 -- 172.21.15.66:0/3843999105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4640cde70 msgr2=0x7fa46406ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:47.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4640cde70 0x7fa46406ff40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c4e0 tx=0x7fa44c00c9a0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 -- 172.21.15.66:0/3843999105 shutdown_connections 2024-03-30T00:47:47.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa46415a370 0x7fa4641591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa440066690 0x7fa440068b50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4640cde70 0x7fa46406ff40 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 --2- 172.21.15.66:0/3843999105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4640cda90 0x7fa46406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:47.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 -- 172.21.15.66:0/3843999105 >> 172.21.15.66:0/3843999105 conn(0x7fa464069f60 msgr2=0x7fa46415b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:47.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 -- 172.21.15.66:0/3843999105 shutdown_connections 2024-03-30T00:47:47.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:47.493+0000 7fa469e00640 1 -- 172.21.15.66:0/3843999105 wait complete. 2024-03-30T00:47:47.867 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:47:48.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.001+0000 7fcd46800640 1 Processor -- start 2024-03-30T00:47:48.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.001+0000 7fcd46800640 1 -- start start 2024-03-30T00:47:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.001+0000 7fcd46800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcd400cc8d0 0x7fcd4006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.001+0000 7fcd46800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd400ca660 con 0x7fcd400cad00 2024-03-30T00:47:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.001+0000 7fcd46800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd400ca940 con 0x7fcd400cd2d0 2024-03-30T00:47:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.001+0000 7fcd46800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd400cbec0 con 0x7fcd400cc8d0 2024-03-30T00:47:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fcd400cad00 0x7fcd400c8940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55382/0 (socket says 172.21.15.66:55382) 2024-03-30T00:47:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3f400640 1 -- 172.21.15.66:0/1364289148 learned_addr learned my addr 172.21.15.66:0/1364289148 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:48.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3fe00640 1 --2- 172.21.15.66:0/1364289148 >> v2:172.21.15.66:3300/0 conn(0x7fcd400cc8d0 0x7fcd4006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3ea00640 1 -- 172.21.15.66:0/1364289148 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326873514 0 0) 0x7fcd400ca660 con 0x7fcd400cad00 2024-03-30T00:47:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3ea00640 1 -- 172.21.15.66:0/1364289148 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd24003530 con 0x7fcd400cad00 2024-03-30T00:47:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3fe00640 1 -- 172.21.15.66:0/1364289148 >> v1:172.21.15.156:6789/0 conn(0x7fcd400cd2d0 legacy=0x7fcd400cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3fe00640 1 -- 172.21.15.66:0/1364289148 >> v1:172.21.15.66:6789/0 conn(0x7fcd400cad00 legacy=0x7fcd400c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3fe00640 1 -- 172.21.15.66:0/1364289148 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400cc0d0 con 0x7fcd400cc8d0 2024-03-30T00:47:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3fe00640 1 --2- 172.21.15.66:0/1364289148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd400cc8d0 0x7fcd4006c030 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800a960 tx=0x7fcd3801d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcaf5b15464f2373 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3ea00640 1 -- 172.21.15.66:0/1364289148 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380a6930 con 0x7fcd400cc8d0 2024-03-30T00:47:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3ea00640 1 -- 172.21.15.66:0/1364289148 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd380a6a90 con 0x7fcd400cc8d0 2024-03-30T00:47:48.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.005+0000 7fcd3ea00640 1 -- 172.21.15.66:0/1364289148 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380a6e20 con 0x7fcd400cc8d0 2024-03-30T00:47:48.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 -- 172.21.15.66:0/1364289148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd400cc8d0 msgr2=0x7fcd4006c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 --2- 172.21.15.66:0/1364289148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd400cc8d0 0x7fcd4006c030 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800a960 tx=0x7fcd3801d840 comp rx=0 tx=0).stop 2024-03-30T00:47:48.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 -- 172.21.15.66:0/1364289148 shutdown_connections 2024-03-30T00:47:48.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 --2- 172.21.15.66:0/1364289148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd400cc8d0 0x7fcd4006c030 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 -- 172.21.15.66:0/1364289148 >> 172.21.15.66:0/1364289148 conn(0x7fcd40069f60 msgr2=0x7fcd4006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 -- 172.21.15.66:0/1364289148 shutdown_connections 2024-03-30T00:47:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 -- 172.21.15.66:0/1364289148 wait complete. 2024-03-30T00:47:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 Processor -- start 2024-03-30T00:47:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 -- start start 2024-03-30T00:47:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd400cad00 0x7fcd40159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd400cc8d0 0x7fcd401574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd400cd2d0 0x7fcd4015adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400c7b00 con 0x7fcd400cad00 2024-03-30T00:47:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400c7de0 con 0x7fcd400cd2d0 2024-03-30T00:47:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd46800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400c80c0 con 0x7fcd400cc8d0 2024-03-30T00:47:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd3f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd400cad00 0x7fcd40159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd3f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd400cad00 0x7fcd40159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55538/0 (socket says 172.21.15.66:55538) 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd3f400640 1 -- 172.21.15.66:0/1520623380 learned_addr learned my addr 172.21.15.66:0/1520623380 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd3fe00640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd400cd2d0 0x7fcd4015adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd36a00640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd400cc8d0 0x7fcd401574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd3f400640 1 -- 172.21.15.66:0/1520623380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd400cc8d0 msgr2=0x7fcd401574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd3f400640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd400cc8d0 0x7fcd401574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd3f400640 1 -- 172.21.15.66:0/1520623380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd400cd2d0 msgr2=0x7fcd4015adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd3f400640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd400cd2d0 0x7fcd4015adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd3f400640 1 -- 172.21.15.66:0/1520623380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3800a5c0 con 0x7fcd400cad00 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.009+0000 7fcd36a00640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd400cc8d0 0x7fcd401574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.013+0000 7fcd3f400640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd400cad00 0x7fcd40159340 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c4d0 tx=0x7fcd3000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.013+0000 7fcd37e00640 1 -- 172.21.15.66:0/1520623380 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30005cf0 con 0x7fcd400cad00 2024-03-30T00:47:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.013+0000 7fcd37e00640 1 -- 172.21.15.66:0/1520623380 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd30005e50 con 0x7fcd400cad00 2024-03-30T00:47:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.013+0000 7fcd37e00640 1 -- 172.21.15.66:0/1520623380 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30007270 con 0x7fcd400cad00 2024-03-30T00:47:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.013+0000 7fcd46800640 1 -- 172.21.15.66:0/1520623380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4015bcd0 con 0x7fcd400cad00 2024-03-30T00:47:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.013+0000 7fcd46800640 1 -- 172.21.15.66:0/1520623380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd40159880 con 0x7fcd400cad00 2024-03-30T00:47:48.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.013+0000 7fcd46800640 1 -- 172.21.15.66:0/1520623380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd40066c30 con 0x7fcd400cad00 2024-03-30T00:47:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.013+0000 7fcd37e00640 1 -- 172.21.15.66:0/1520623380 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcd300073d0 con 0x7fcd400cad00 2024-03-30T00:47:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.013+0000 7fcd37e00640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd1c0666c0 0x7fcd1c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.013+0000 7fcd37e00640 1 -- 172.21.15.66:0/1520623380 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fcd30083850 con 0x7fcd400cad00 2024-03-30T00:47:48.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.013+0000 7fcd36a00640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd1c0666c0 0x7fcd1c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:48.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.017+0000 7fcd36a00640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd1c0666c0 0x7fcd1c068b80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcd28007fd0 tx=0x7fcd28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:48.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.017+0000 7fcd37e00640 1 -- 172.21.15.66:0/1520623380 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcd30050a50 con 0x7fcd400cad00 2024-03-30T00:47:48.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 -- 172.21.15.66:0/1520623380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd400715c0 con 0x7fcd400cad00 2024-03-30T00:47:48.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd37e00640 1 -- 172.21.15.66:0/1520623380 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v97) v1 ==== 80+0+122 (secure 0 0 0) 0x7fcd30017090 con 0x7fcd400cad00 2024-03-30T00:47:48.202 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6836/1147664160 2024-03-31T00:46:34.103831+0000 2024-03-30T00:47:48.202 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6835/1147664160 2024-03-31T00:46:34.103831+0000 2024-03-30T00:47:48.202 INFO:teuthology.orchestra.run.smithi066.stderr:listed 2 entries 2024-03-30T00:47:48.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 -- 172.21.15.66:0/1520623380 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd1c0666c0 msgr2=0x7fcd1c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd1c0666c0 0x7fcd1c068b80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcd28007fd0 tx=0x7fcd28009040 comp rx=0 tx=0).stop 2024-03-30T00:47:48.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 -- 172.21.15.66:0/1520623380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd400cad00 msgr2=0x7fcd40159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd400cad00 0x7fcd40159340 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c4d0 tx=0x7fcd3000c990 comp rx=0 tx=0).stop 2024-03-30T00:47:48.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 -- 172.21.15.66:0/1520623380 shutdown_connections 2024-03-30T00:47:48.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd400cd2d0 0x7fcd4015adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd400cc8d0 0x7fcd401574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcd1c0666c0 0x7fcd1c068b80 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 --2- 172.21.15.66:0/1520623380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd400cad00 0x7fcd40159340 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 -- 172.21.15.66:0/1520623380 >> 172.21.15.66:0/1520623380 conn(0x7fcd40069f60 msgr2=0x7fcd400c4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:48.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 -- 172.21.15.66:0/1520623380 shutdown_connections 2024-03-30T00:47:48.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.193+0000 7fcd46800640 1 -- 172.21.15.66:0/1520623380 wait complete. 2024-03-30T00:47:48.222 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:47:48.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:48.222 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:47:48.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[48168]: fuse finished with error 0 and tester_r 0 2024-03-30T00:47:48.243 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:47:48.274 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:47:48.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:48.274 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:47:48.330 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:47:48.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.501+0000 7f4f5cc00640 1 Processor -- start 2024-03-30T00:47:48.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.501+0000 7f4f5cc00640 1 -- start start 2024-03-30T00:47:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f5cc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f5815db90 0x7f4f5815e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f5cc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f58070b50 0x7f4f58070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f5cc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f5806c410 con 0x7f4f580ce6b0 2024-03-30T00:47:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f5cc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f58071520 con 0x7f4f5815db90 2024-03-30T00:47:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f5cc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5806c6b0 con 0x7f4f58070b50 2024-03-30T00:47:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f4fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f5815db90 0x7f4f5815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f4fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f5815db90 0x7f4f5815e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55550/0 (socket says 172.21.15.66:55550) 2024-03-30T00:47:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f4fe00640 1 -- 172.21.15.66:0/990913586 learned_addr learned my addr 172.21.15.66:0/990913586 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f56800640 1 --2- 172.21.15.66:0/990913586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f58070b50 0x7f4f58070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:48.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f4fe00640 1 -- 172.21.15.66:0/990913586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f58070b50 msgr2=0x7f4f58070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:48.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f4fe00640 1 --2- 172.21.15.66:0/990913586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f58070b50 0x7f4f58070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f4fe00640 1 -- 172.21.15.66:0/990913586 >> v1:172.21.15.156:6789/0 conn(0x7f4f580ce6b0 legacy=0x7f4f580cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f4fe00640 1 -- 172.21.15.66:0/990913586 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5806c990 con 0x7f4f5815db90 2024-03-30T00:47:48.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f56800640 1 --2- 172.21.15.66:0/990913586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f58070b50 0x7f4f58070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:48.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.505+0000 7f4f4fe00640 1 --2- 172.21.15.66:0/990913586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f5815db90 0x7f4f5815e070 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009870 tx=0x7f4f4001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe31cdcef6dde6e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:48.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f55400640 1 -- 172.21.15.66:0/990913586 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f400a6040 con 0x7f4f5815db90 2024-03-30T00:47:48.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f55400640 1 -- 172.21.15.66:0/990913586 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f400aa600 con 0x7f4f5815db90 2024-03-30T00:47:48.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f55400640 1 -- 172.21.15.66:0/990913586 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f400aa990 con 0x7f4f5815db90 2024-03-30T00:47:48.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 -- 172.21.15.66:0/990913586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f5815db90 msgr2=0x7f4f5815e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 --2- 172.21.15.66:0/990913586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f5815db90 0x7f4f5815e070 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009870 tx=0x7f4f4001c4a0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 -- 172.21.15.66:0/990913586 shutdown_connections 2024-03-30T00:47:48.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 --2- 172.21.15.66:0/990913586 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f58070b50 0x7f4f58070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 --2- 172.21.15.66:0/990913586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f5815db90 0x7f4f5815e070 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 -- 172.21.15.66:0/990913586 >> 172.21.15.66:0/990913586 conn(0x7f4f58066100 msgr2=0x7f4f58069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:48.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 -- 172.21.15.66:0/990913586 shutdown_connections 2024-03-30T00:47:48.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 -- 172.21.15.66:0/990913586 wait complete. 2024-03-30T00:47:48.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 Processor -- start 2024-03-30T00:47:48.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 -- start start 2024-03-30T00:47:48.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f58070b50 0x7f4f5814eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:48.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f580ce6b0 0x7f4f5815c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:48.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f5815db90 0x7f4f58156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:48.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5814e1c0 con 0x7f4f58070b50 2024-03-30T00:47:48.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f58157110 con 0x7f4f580ce6b0 2024-03-30T00:47:48.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f5cc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5814e410 con 0x7f4f5815db90 2024-03-30T00:47:48.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f55e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f58070b50 0x7f4f5814eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f55e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f58070b50 0x7f4f5814eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55558/0 (socket says 172.21.15.66:55558) 2024-03-30T00:47:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.509+0000 7f4f55e00640 1 -- 172.21.15.66:0/3567744401 learned_addr learned my addr 172.21.15.66:0/3567744401 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f56800640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f5815db90 0x7f4f58156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f4fe00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f580ce6b0 0x7f4f5815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f55e00640 1 -- 172.21.15.66:0/3567744401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f5815db90 msgr2=0x7f4f58156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f55e00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f5815db90 0x7f4f58156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f55e00640 1 -- 172.21.15.66:0/3567744401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f580ce6b0 msgr2=0x7f4f5815c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f55e00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f580ce6b0 0x7f4f5815c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f55e00640 1 -- 172.21.15.66:0/3567744401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f400094d0 con 0x7f4f58070b50 2024-03-30T00:47:48.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f4fe00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f580ce6b0 0x7f4f5815c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:48.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f56800640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f5815db90 0x7f4f58156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:48.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f55e00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f58070b50 0x7f4f5814eed0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800c500 tx=0x7f4f4800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:48.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f4e000640 1 -- 172.21.15.66:0/3567744401 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48005220 con 0x7f4f58070b50 2024-03-30T00:47:48.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f4e000640 1 -- 172.21.15.66:0/3567744401 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f48005380 con 0x7f4f58070b50 2024-03-30T00:47:48.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f4e000640 1 -- 172.21.15.66:0/3567744401 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48007270 con 0x7f4f58070b50 2024-03-30T00:47:48.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f5cc00640 1 -- 172.21.15.66:0/3567744401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5814e6f0 con 0x7f4f58070b50 2024-03-30T00:47:48.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f5cc00640 1 -- 172.21.15.66:0/3567744401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5814a2f0 con 0x7f4f58070b50 2024-03-30T00:47:48.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f5cc00640 1 -- 172.21.15.66:0/3567744401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5806e320 con 0x7f4f58070b50 2024-03-30T00:47:48.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f4e000640 1 -- 172.21.15.66:0/3567744401 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4f480073d0 con 0x7f4f58070b50 2024-03-30T00:47:48.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f4e000640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f34066690 0x7f4f34068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:48.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f4e000640 1 -- 172.21.15.66:0/3567744401 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f4f48083d80 con 0x7f4f58070b50 2024-03-30T00:47:48.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.513+0000 7f4f4fe00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f34066690 0x7f4f34068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:48.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.517+0000 7f4f4fe00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f34066690 0x7f4f34068b50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4f400ae000 tx=0x7f4f400a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:48.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.517+0000 7f4f4e000640 1 -- 172.21.15.66:0/3567744401 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4f48050e50 con 0x7f4f58070b50 2024-03-30T00:47:48.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.713+0000 7f4f5cc00640 1 -- 172.21.15.66:0/3567744401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f58066c30 con 0x7f4f58070b50 2024-03-30T00:47:48.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.713+0000 7f4f4e000640 1 -- 172.21.15.66:0/3567744401 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f4f48054470 con 0x7f4f58070b50 2024-03-30T00:47:48.725 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:47:48.725 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:46:45.565028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6836/1626022080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1626022080},{"type":"v1","addr":"172.21.15.66:6836","nonce":1626022080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:47:48.725 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 65 2024-03-30T00:47:48.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 -- 172.21.15.66:0/3567744401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f34066690 msgr2=0x7f4f34068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f34066690 0x7f4f34068b50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4f400ae000 tx=0x7f4f400a7000 comp rx=0 tx=0).stop 2024-03-30T00:47:48.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 -- 172.21.15.66:0/3567744401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f58070b50 msgr2=0x7f4f5814eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:48.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f58070b50 0x7f4f5814eed0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800c500 tx=0x7f4f4800c9c0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 -- 172.21.15.66:0/3567744401 shutdown_connections 2024-03-30T00:47:48.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f5815db90 0x7f4f58156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f580ce6b0 0x7f4f5815c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f34066690 0x7f4f34068b50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 --2- 172.21.15.66:0/3567744401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f58070b50 0x7f4f5814eed0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:48.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 -- 172.21.15.66:0/3567744401 >> 172.21.15.66:0/3567744401 conn(0x7f4f58066100 msgr2=0x7f4f58154420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:48.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 -- 172.21.15.66:0/3567744401 shutdown_connections 2024-03-30T00:47:48.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:48.717+0000 7f4f5cc00640 1 -- 172.21.15.66:0/3567744401 wait complete. 2024-03-30T00:47:48.745 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-30T00:47:48.914 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-03-30T00:47:48.915 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-30T00:47:48.915 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:47:49.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.045+0000 7fd80ec00640 1 Processor -- start 2024-03-30T00:47:49.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.045+0000 7fd80ec00640 1 -- start start 2024-03-30T00:47:49.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd80ec00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd8080ce6b0 0x7fd80815e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd80ec00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd80815db90 0x7fd8080cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd80ec00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80806c410 con 0x7fd808070b50 2024-03-30T00:47:49.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd80ec00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080cefd0 con 0x7fd80815db90 2024-03-30T00:47:49.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd80ec00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd80806c6b0 con 0x7fd8080ce6b0 2024-03-30T00:47:49.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd7ff400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd80815db90 0x7fd8080cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd7ff400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd80815db90 0x7fd8080cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54946/0 (socket says 172.21.15.66:54946) 2024-03-30T00:47:49.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd7ff400640 1 -- 172.21.15.66:0/228856872 learned_addr learned my addr 172.21.15.66:0/228856872 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd7ff400640 1 -- 172.21.15.66:0/228856872 >> v2:172.21.15.156:3300/0 conn(0x7fd8080ce6b0 msgr2=0x7fd80815e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd7ff400640 1 --2- 172.21.15.66:0/228856872 >> v2:172.21.15.156:3300/0 conn(0x7fd8080ce6b0 0x7fd80815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd7ff400640 1 -- 172.21.15.66:0/228856872 >> v1:172.21.15.156:6789/0 conn(0x7fd808070b50 legacy=0x7fd80806d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.049+0000 7fd7ff400640 1 -- 172.21.15.66:0/228856872 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80806c990 con 0x7fd80815db90 2024-03-30T00:47:49.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd7ff400640 1 --2- 172.21.15.66:0/228856872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd80815db90 0x7fd8080cea90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd8080cf5d0 tx=0x7fd7f001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ac64358f1db87e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd807400640 1 -- 172.21.15.66:0/228856872 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00ab620 con 0x7fd80815db90 2024-03-30T00:47:49.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd807400640 1 -- 172.21.15.66:0/228856872 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f00ab780 con 0x7fd80815db90 2024-03-30T00:47:49.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd807400640 1 -- 172.21.15.66:0/228856872 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00abb10 con 0x7fd80815db90 2024-03-30T00:47:49.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd80ec00640 1 -- 172.21.15.66:0/228856872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd80815db90 msgr2=0x7fd8080cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd80ec00640 1 --2- 172.21.15.66:0/228856872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd80815db90 0x7fd8080cea90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd8080cf5d0 tx=0x7fd7f001c420 comp rx=0 tx=0).stop 2024-03-30T00:47:49.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd80ec00640 1 -- 172.21.15.66:0/228856872 shutdown_connections 2024-03-30T00:47:49.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd80ec00640 1 --2- 172.21.15.66:0/228856872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd80815db90 0x7fd8080cea90 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd80ec00640 1 --2- 172.21.15.66:0/228856872 >> v2:172.21.15.156:3300/0 conn(0x7fd8080ce6b0 0x7fd80815e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd80ec00640 1 -- 172.21.15.66:0/228856872 >> 172.21.15.66:0/228856872 conn(0x7fd808066100 msgr2=0x7fd808069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd80ec00640 1 -- 172.21.15.66:0/228856872 shutdown_connections 2024-03-30T00:47:49.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd80ec00640 1 -- 172.21.15.66:0/228856872 wait complete. 2024-03-30T00:47:49.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd80ec00640 1 Processor -- start 2024-03-30T00:47:49.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.053+0000 7fd80ec00640 1 -- start start 2024-03-30T00:47:49.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd80ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd808070b50 0x7fd808149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd80ec00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8080ce6b0 0x7fd80814a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd80ec00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd80815db90 0x7fd80814ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd80ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80814a870 con 0x7fd808070b50 2024-03-30T00:47:49.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd80ec00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8081525e0 con 0x7fd80815db90 2024-03-30T00:47:49.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd80ec00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8081528c0 con 0x7fd8080ce6b0 2024-03-30T00:47:49.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd807e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd808070b50 0x7fd808149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd807e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd808070b50 0x7fd808149df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54960/0 (socket says 172.21.15.66:54960) 2024-03-30T00:47:49.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd807e00640 1 -- 172.21.15.66:0/2035931015 learned_addr learned my addr 172.21.15.66:0/2035931015 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd7ff400640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8080ce6b0 0x7fd80814a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd80cc00640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd80815db90 0x7fd80814ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd807e00640 1 -- 172.21.15.66:0/2035931015 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8080ce6b0 msgr2=0x7fd80814a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd807e00640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8080ce6b0 0x7fd80814a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd807e00640 1 -- 172.21.15.66:0/2035931015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd80815db90 msgr2=0x7fd80814ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd807e00640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd80815db90 0x7fd80814ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd807e00640 1 -- 172.21.15.66:0/2035931015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00094d0 con 0x7fd808070b50 2024-03-30T00:47:49.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd807e00640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd808070b50 0x7fd808149df0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800ba50 tx=0x7fd7f800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd804c00640 1 -- 172.21.15.66:0/2035931015 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f800cbe0 con 0x7fd808070b50 2024-03-30T00:47:49.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd804c00640 1 -- 172.21.15.66:0/2035931015 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f800cd60 con 0x7fd808070b50 2024-03-30T00:47:49.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd804c00640 1 -- 172.21.15.66:0/2035931015 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8014680 con 0x7fd808070b50 2024-03-30T00:47:49.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd80ec00640 1 -- 172.21.15.66:0/2035931015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd808152ba0 con 0x7fd808070b50 2024-03-30T00:47:49.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd80ec00640 1 -- 172.21.15.66:0/2035931015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8081538c0 con 0x7fd808070b50 2024-03-30T00:47:49.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd80ec00640 1 -- 172.21.15.66:0/2035931015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80806e320 con 0x7fd808070b50 2024-03-30T00:47:49.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.057+0000 7fd804c00640 1 -- 172.21.15.66:0/2035931015 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd7f80147e0 con 0x7fd808070b50 2024-03-30T00:47:49.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.061+0000 7fd804c00640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd7e40666c0 0x7fd7e4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.061+0000 7fd804c00640 1 -- 172.21.15.66:0/2035931015 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fd7f8084300 con 0x7fd808070b50 2024-03-30T00:47:49.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.061+0000 7fd7ff400640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd7e40666c0 0x7fd7e4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.061+0000 7fd7ff400640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd7e40666c0 0x7fd7e4068b80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00af000 tx=0x7fd7f00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.065+0000 7fd804c00640 1 -- 172.21.15.66:0/2035931015 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd7f8051450 con 0x7fd808070b50 2024-03-30T00:47:49.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.257+0000 7fd80ec00640 1 -- 172.21.15.66:0/2035931015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd808066c30 con 0x7fd808070b50 2024-03-30T00:47:49.269 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:47:49.261+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:47:49.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd804c00640 1 -- 172.21.15.66:0/2035931015 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd7f8059e60 con 0x7fd808070b50 2024-03-30T00:47:49.277 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:47:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 -- 172.21.15.66:0/2035931015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd7e40666c0 msgr2=0x7fd7e4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd7e40666c0 0x7fd7e4068b80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00af000 tx=0x7fd7f00a7000 comp rx=0 tx=0).stop 2024-03-30T00:47:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 -- 172.21.15.66:0/2035931015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd808070b50 msgr2=0x7fd808149df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd808070b50 0x7fd808149df0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800ba50 tx=0x7fd7f800bf10 comp rx=0 tx=0).stop 2024-03-30T00:47:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 -- 172.21.15.66:0/2035931015 shutdown_connections 2024-03-30T00:47:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd80815db90 0x7fd80814ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8080ce6b0 0x7fd80814a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd7e40666c0 0x7fd7e4068b80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 --2- 172.21.15.66:0/2035931015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd808070b50 0x7fd808149df0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 -- 172.21.15.66:0/2035931015 >> 172.21.15.66:0/2035931015 conn(0x7fd808066100 msgr2=0x7fd8080b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 -- 172.21.15.66:0/2035931015 shutdown_connections 2024-03-30T00:47:49.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.269+0000 7fd80ec00640 1 -- 172.21.15.66:0/2035931015 wait complete. 2024-03-30T00:47:49.287 INFO:tasks.ceph.mds.a.smithi066.stderr: -4267> 2024-03-30T00:46:40.118+0000 7f8a28e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:47:49.288 INFO:tasks.ceph.mds.a.smithi066.stderr: -3937> 2024-03-30T00:46:41.434+0000 7f8a24800640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-30T00:47:49.288 INFO:tasks.ceph.mds.a.smithi066.stderr: -3775> 2024-03-30T00:46:41.662+0000 7f8a24800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-03-30T00:46:23.006182+0000 1=1+0) n(v0 rc2024-03-30T00:46:23.120156+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x55d4e38c3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-30T00:46:23.006182+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-30T00:46:23.120156+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-03-30T00:47:49.292 INFO:tasks.ceph.mds.a.smithi066.stderr: -2172> 2024-03-30T00:47:13.254+0000 7f8a24800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-03-30T00:46:23.006182+0000 1=1+0)->f(v1 m2024-03-30T00:46:23.006182+0000) n(v0 rc2024-03-30T00:46:23.120156+0000 2=1+1)->n(v0 rc2024-03-30T00:46:23.120156+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55d4e38c3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-30T00:46:23.006182+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-30T00:46:23.120156+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-03-30T00:47:49.299 DEBUG:teuthology.orchestra.run.smithi066:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-03-30T00:47:49.335 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:47:49.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175d04a8c0 1 -- start start 2024-03-30T00:47:49.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175d04a8c0 1 Processor -- start 2024-03-30T00:47:49.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175d04a8c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda4fcb800 0x55dda5056580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175d04a8c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda4fcb400 0x55dda5056b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175d04a8c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda4fcb000 0x55dda5057080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175d04a8c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5098a00 con 0x55dda4fcb000 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175d04a8c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5098780 con 0x55dda4fcb400 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175d04a8c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dda5098640 con 0x55dda4fcb800 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda4fcb000 0x55dda5057080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda4fcb000 0x55dda5057080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54982/0 (socket says 172.21.15.66:54982) 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175b000640 1 -- 172.21.15.66:0/2120229253 learned_addr learned my addr 172.21.15.66:0/2120229253 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175a600640 1 --2- 172.21.15.66:0/2120229253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda4fcb800 0x55dda5056580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f1759c00640 1 --2- 172.21.15.66:0/2120229253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda4fcb400 0x55dda5056b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175b000640 1 -- 172.21.15.66:0/2120229253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda4fcb800 msgr2=0x55dda5056580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175b000640 1 --2- 172.21.15.66:0/2120229253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda4fcb800 0x55dda5056580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175b000640 1 -- 172.21.15.66:0/2120229253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda4fcb400 msgr2=0x55dda5056b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175b000640 1 --2- 172.21.15.66:0/2120229253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda4fcb400 0x55dda5056b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175b000640 1 -- 172.21.15.66:0/2120229253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dda4ffe9c0 con 0x55dda4fcb000 2024-03-30T00:47:49.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175a600640 1 --2- 172.21.15.66:0/2120229253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda4fcb800 0x55dda5056580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.333+0000 7f175b000640 1 --2- 172.21.15.66:0/2120229253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda4fcb000 0x55dda5057080 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x55dda5e3f380 tx=0x55dda5d28420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f55759649e062cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1757400640 1 -- 172.21.15.66:0/2120229253 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dda501f600 con 0x55dda4fcb000 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1757400640 1 -- 172.21.15.66:0/2120229253 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dda5028180 con 0x55dda4fcb000 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1757400640 1 -- 172.21.15.66:0/2120229253 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dda501f1e0 con 0x55dda4fcb000 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 -- 172.21.15.66:0/2120229253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dda4ffeb60 con 0x55dda4fcb000 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 -- 172.21.15.66:0/2120229253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55dda4ffe340 con 0x55dda4fcb000 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 4 waiting for MDS map... 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1757400640 1 -- 172.21.15.66:0/2120229253 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x55dda500ac40 con 0x55dda4fcb000 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1757400640 1 -- 172.21.15.66:0/2120229253 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 66) v1 ==== 2573+0+0 (secure 0 0 0) 0x55dda5018f00 con 0x55dda4fcb000 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 4 Got MDS map 66 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 4 datascan.main: Using metadata pool output 2024-03-30T00:47:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 4 datascan.main: connecting to RADOS... 2024-03-30T00:47:49.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 Processor -- start 2024-03-30T00:47:49.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 -- start start 2024-03-30T00:47:49.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee4c00 0x55dda5057600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ee5000 0x55dda5057b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda5ee5400 0x55dda5058100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5011e00 con 0x55dda5ee4c00 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5098780 con 0x55dda5ee5000 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dda5098500 con 0x55dda5ee5400 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1759c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee4c00 0x55dda5057600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1759c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee4c00 0x55dda5057600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54986/0 (socket says 172.21.15.66:54986) 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1759c00640 1 -- 172.21.15.66:0/3234797124 learned_addr learned my addr 172.21.15.66:0/3234797124 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1759c00640 1 -- 172.21.15.66:0/3234797124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda5ee5400 msgr2=0x55dda5058100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1759c00640 1 --2- 172.21.15.66:0/3234797124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda5ee5400 0x55dda5058100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1759c00640 1 -- 172.21.15.66:0/3234797124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ee5000 msgr2=0x55dda5057b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1759c00640 1 --2- 172.21.15.66:0/3234797124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ee5000 0x55dda5057b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1759c00640 1 -- 172.21.15.66:0/3234797124 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dda4ffe000 con 0x55dda5ee4c00 2024-03-30T00:47:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1759c00640 1 --2- 172.21.15.66:0/3234797124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee4c00 0x55dda5057600 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x55dda5e3e9f0 tx=0x55dda5d28240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf055425f19c961d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1754c00640 1 -- 172.21.15.66:0/3234797124 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dda501f080 con 0x55dda5ee4c00 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1754c00640 1 -- 172.21.15.66:0/3234797124 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dda5028900 con 0x55dda5ee4c00 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f1754c00640 1 -- 172.21.15.66:0/3234797124 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dda501f600 con 0x55dda5ee4c00 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3234797124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee4c00 msgr2=0x55dda5057600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3234797124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee4c00 0x55dda5057600 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x55dda5e3e9f0 tx=0x55dda5d28240 comp rx=0 tx=0).stop 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3234797124 shutdown_connections 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3234797124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda5ee5400 0x55dda5058100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3234797124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ee5000 0x55dda5057b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3234797124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee4c00 0x55dda5057600 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.337+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3234797124 >> 172.21.15.66:0/3234797124 conn(0x55dda4fcac00 msgr2=0x55dda4ff7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3234797124 shutdown_connections 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3234797124 wait complete. 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 Processor -- start 2024-03-30T00:47:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 -- start start 2024-03-30T00:47:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda5ee4c00 0x55dda5059180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee5000 0x55dda5058c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ee5400 0x55dda5058680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda50983c0 con 0x55dda5ee5000 2024-03-30T00:47:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5098280 con 0x55dda5ee5400 2024-03-30T00:47:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dda5098140 con 0x55dda5ee4c00 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee5000 0x55dda5058c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f1759c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda5ee4c00 0x55dda5059180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee5000 0x55dda5058c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55002/0 (socket says 172.21.15.66:55002) 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f1759c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda5ee4c00 0x55dda5059180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:35526/0 (socket says 172.21.15.66:35526) 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 learned_addr learned my addr 172.21.15.66:0/3825757189 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f1759c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ee5400 0x55dda5058680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda5ee4c00 msgr2=0x55dda5059180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175a600640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda5ee4c00 0x55dda5059180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ee5400 msgr2=0x55dda5058680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175a600640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ee5400 0x55dda5058680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dda4ffe1a0 con 0x55dda5ee5000 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175a600640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee5000 0x55dda5058c00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x55dda5ef73e0 tx=0x55dda5ef8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f1752e00640 1 -- 172.21.15.66:0/3825757189 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dda501f080 con 0x55dda5ee5000 2024-03-30T00:47:49.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f1752e00640 1 -- 172.21.15.66:0/3825757189 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dda5daf680 con 0x55dda5ee5000 2024-03-30T00:47:49.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dda4ffe000 con 0x55dda5ee5000 2024-03-30T00:47:49.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f1752e00640 1 -- 172.21.15.66:0/3825757189 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dda501f600 con 0x55dda5ee5000 2024-03-30T00:47:49.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dda4ffe9c0 con 0x55dda5ee5000 2024-03-30T00:47:49.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.341+0000 7f175d04a8c0 4 datascan.init: resolving metadata pool 13 2024-03-30T00:47:49.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f1752e00640 1 -- 172.21.15.66:0/3825757189 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x55dda5002840 con 0x55dda5ee5000 2024-03-30T00:47:49.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f1752e00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55dda52bcc00 0x55dda5059700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f1752e00640 1 -- 172.21.15.66:0/3825757189 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x55dda500aa80 con 0x55dda5ee5000 2024-03-30T00:47:49.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f175d04a8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-30T00:47:49.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f175d04a8c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-03-30T00:47:49.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f175d04a8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-30T00:47:49.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f1759c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55dda52bcc00 0x55dda5059700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x55dda5ee5800 0x55dda5058100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee5c00 con 0x55dda5ee5800 2024-03-30T00:47:49.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f1759c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x55dda5ee5800 0x55dda5058100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f1759c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55dda52bcc00 0x55dda5059700 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55dda5ef66f0 tx=0x55dda5d28360 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.345+0000 7f1759c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x55dda5ee5800 0x55dda5058100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.349+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.2 v2:172.21.15.66:6816/27464 1 ==== osd_op_reply(1 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda5ee5800 2024-03-30T00:47:49.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.349+0000 7f1754200640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55dda52bd000 0x55dda53b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.349+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda52bd400 con 0x55dda52bd000 2024-03-30T00:47:49.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.349+0000 7f175b000640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55dda52bd000 0x55dda53b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.349+0000 7f175b000640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55dda52bd000 0x55dda53b0000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.353+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== osd_op_reply(2 [pgnls start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.353+0000 7f1754c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55dda52bdc00 0x55dda53b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.353+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53ba000 con 0x55dda52bdc00 2024-03-30T00:47:49.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.353+0000 7f175a600640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55dda52bdc00 0x55dda53b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.353+0000 7f175a600640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55dda52bdc00 0x55dda53b0580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.353+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(3 [pgnls start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda52bdc00 2024-03-30T00:47:49.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.353+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda52bd800 con 0x55dda52bd000 2024-03-30T00:47:49.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 2 ==== osd_op_reply(4 [pgnls start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53ba400 con 0x55dda52bd000 2024-03-30T00:47:49.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 3 ==== osd_op_reply(5 [pgnls start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f1754200640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x55dda52bd400 0x55dda53b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee4800 con 0x55dda52bd400 2024-03-30T00:47:49.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f1759c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x55dda52bd400 0x55dda53b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f1759c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x55dda52bd400 0x55dda53b0b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.3 v2:172.21.15.66:6808/27471 1 ==== osd_op_reply(6 [pgnls start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda52bd400 2024-03-30T00:47:49.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53ba400 con 0x55dda52bdc00 2024-03-30T00:47:49.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.1 v2:172.21.15.66:6821/27458 2 ==== osd_op_reply(7 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda52bdc00 2024-03-30T00:47:49.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee4400 con 0x55dda52bdc00 2024-03-30T00:47:49.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.1 v2:172.21.15.66:6821/27458 3 ==== osd_op_reply(8 [pgnls start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda52bdc00 2024-03-30T00:47:49.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f1754c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55dda53ba800 0x55dda53b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda52bd800 con 0x55dda53ba800 2024-03-30T00:47:49.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f175b000640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55dda53ba800 0x55dda53b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.357+0000 7f175b000640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55dda53ba800 0x55dda53b1600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(9 [pgnls start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53ba800 2024-03-30T00:47:49.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bac00 con 0x55dda52bdc00 2024-03-30T00:47:49.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.1 v2:172.21.15.66:6821/27458 4 ==== osd_op_reply(10 [pgnls start_epoch 98 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda52bdc00 2024-03-30T00:47:49.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f1754c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x55dda53ba000 0x55dda53b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53ba400 con 0x55dda53ba000 2024-03-30T00:47:49.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f175a600640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x55dda53ba000 0x55dda53b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f175a600640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x55dda53ba000 0x55dda53b1b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 1 ==== osd_op_reply(11 [pgnls start_epoch 98 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bb000 con 0x55dda5ee5800 2024-03-30T00:47:49.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.2 v2:172.21.15.66:6816/27464 2 ==== osd_op_reply(12 [pgnls start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda5ee5800 2024-03-30T00:47:49.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bac00 con 0x55dda52bd000 2024-03-30T00:47:49.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 4 ==== osd_op_reply(13 [pgnls start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f1754200640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55dda53bb400 0x55dda53b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.361+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bb800 con 0x55dda53bb400 2024-03-30T00:47:49.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1759c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55dda53bb400 0x55dda53b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1759c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55dda53bb400 0x55dda53b1080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== osd_op_reply(14 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda53bb400 2024-03-30T00:47:49.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee4400 con 0x55dda52bd000 2024-03-30T00:47:49.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 5 ==== osd_op_reply(15 [pgnls start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bb000 con 0x55dda52bdc00 2024-03-30T00:47:49.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.1 v2:172.21.15.66:6821/27458 5 ==== osd_op_reply(16 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda52bdc00 2024-03-30T00:47:49.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bbc00 con 0x55dda53ba800 2024-03-30T00:47:49.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.6 v2:172.21.15.156:6816/27642 2 ==== osd_op_reply(17 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53ba800 2024-03-30T00:47:49.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee5c00 con 0x55dda53ba000 2024-03-30T00:47:49.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 2 ==== osd_op_reply(18 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cc000 con 0x55dda5ee5800 2024-03-30T00:47:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.2 v2:172.21.15.66:6816/27464 3 ==== osd_op_reply(19 [pgnls start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda5ee5800 2024-03-30T00:47:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee4800 con 0x55dda53ba000 2024-03-30T00:47:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 3 ==== osd_op_reply(20 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1754c00640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x55dda53cc400 0x55dda53b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cc800 con 0x55dda53cc400 2024-03-30T00:47:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.365+0000 7f175b000640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x55dda53cc400 0x55dda53b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.369+0000 7f175b000640 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x55dda53cc400 0x55dda53b2100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.369+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== osd_op_reply(21 [pgnls start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53cc400 2024-03-30T00:47:49.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.369+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5db1400 con 0x55dda53ba000 2024-03-30T00:47:49.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.369+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 4 ==== osd_op_reply(22 [pgnls start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.369+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53ccc00 con 0x55dda52bd000 2024-03-30T00:47:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.369+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 6 ==== osd_op_reply(23 [pgnls start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.369+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5db1000 con 0x55dda52bd000 2024-03-30T00:47:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.369+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 7 ==== osd_op_reply(24 [pgnls start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.369+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cd000 con 0x55dda52bd000 2024-03-30T00:47:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.369+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 8 ==== osd_op_reply(25 [pgnls start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.369+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee4400 con 0x55dda53ba000 2024-03-30T00:47:49.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 5 ==== osd_op_reply(26 [pgnls start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cd400 con 0x55dda52bd400 2024-03-30T00:47:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.3 v2:172.21.15.66:6808/27471 2 ==== osd_op_reply(27 [pgnls start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda52bd400 2024-03-30T00:47:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bac00 con 0x55dda5ee5800 2024-03-30T00:47:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.2 v2:172.21.15.66:6816/27464 4 ==== osd_op_reply(28 [pgnls start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda5ee5800 2024-03-30T00:47:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cc800 con 0x55dda53ba000 2024-03-30T00:47:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 6 ==== osd_op_reply(29 [pgnls start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bbc00 con 0x55dda52bd400 2024-03-30T00:47:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.3 v2:172.21.15.66:6808/27471 3 ==== osd_op_reply(30 [pgnls start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda52bd400 2024-03-30T00:47:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53ccc00 con 0x55dda53bb400 2024-03-30T00:47:49.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.7 v2:172.21.15.156:6824/27643 2 ==== osd_op_reply(31 [pgnls start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda53bb400 2024-03-30T00:47:49.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53ba400 con 0x55dda53cc400 2024-03-30T00:47:49.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.5 v2:172.21.15.156:6808/27644 2 ==== osd_op_reply(32 [pgnls start_epoch 98 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53cc400 2024-03-30T00:47:49.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cd800 con 0x55dda52bd400 2024-03-30T00:47:49.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.3 v2:172.21.15.66:6808/27471 4 ==== osd_op_reply(33 [pgnls start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda52bd400 2024-03-30T00:47:49.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bb000 con 0x55dda53cc400 2024-03-30T00:47:49.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.5 v2:172.21.15.156:6808/27644 3 ==== osd_op_reply(34 [pgnls start_epoch 98 out=49b] v95'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53cc400 2024-03-30T00:47:49.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cdc00 con 0x55dda53ba000 2024-03-30T00:47:49.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 7 ==== osd_op_reply(35 [pgnls start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.373+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda52bd800 con 0x55dda52bdc00 2024-03-30T00:47:49.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.1 v2:172.21.15.66:6821/27458 6 ==== osd_op_reply(36 [pgnls start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda52bdc00 2024-03-30T00:47:49.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5db1400 con 0x55dda53cc400 2024-03-30T00:47:49.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.5 v2:172.21.15.156:6808/27644 4 ==== osd_op_reply(37 [pgnls start_epoch 98 out=49b] v95'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53cc400 2024-03-30T00:47:49.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cc000 con 0x55dda53ba800 2024-03-30T00:47:49.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.6 v2:172.21.15.156:6816/27642 3 ==== osd_op_reply(38 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53ba800 2024-03-30T00:47:49.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee5c00 con 0x55dda53ba800 2024-03-30T00:47:49.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.6 v2:172.21.15.156:6816/27642 4 ==== osd_op_reply(39 [pgnls start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53ba800 2024-03-30T00:47:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cd400 con 0x55dda53bb400 2024-03-30T00:47:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.7 v2:172.21.15.156:6824/27643 3 ==== osd_op_reply(40 [pgnls start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda53bb400 2024-03-30T00:47:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee4800 con 0x55dda52bd000 2024-03-30T00:47:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 9 ==== osd_op_reply(41 [pgnls start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53ccc00 con 0x55dda5ee5800 2024-03-30T00:47:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.2 v2:172.21.15.66:6816/27464 5 ==== osd_op_reply(42 [pgnls start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda5ee5800 2024-03-30T00:47:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee5c00 con 0x55dda53bb400 2024-03-30T00:47:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.7 v2:172.21.15.156:6824/27643 4 ==== osd_op_reply(43 [pgnls start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda53bb400 2024-03-30T00:47:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bb800 con 0x55dda53ba000 2024-03-30T00:47:49.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 8 ==== osd_op_reply(44 [pgnls start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cd000 con 0x55dda53ba800 2024-03-30T00:47:49.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.6 v2:172.21.15.156:6816/27642 5 ==== osd_op_reply(45 [pgnls start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53ba800 2024-03-30T00:47:49.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bac00 con 0x55dda53ba000 2024-03-30T00:47:49.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 9 ==== osd_op_reply(46 [pgnls start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cc000 con 0x55dda52bd000 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 10 ==== osd_op_reply(47 [pgnls start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.377+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bbc00 con 0x55dda52bd400 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.3 v2:172.21.15.66:6808/27471 5 ==== osd_op_reply(48 [pgnls start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda52bd400 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5db1000 con 0x55dda52bd000 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 11 ==== osd_op_reply(49 [pgnls start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bb800 con 0x55dda53bb400 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.7 v2:172.21.15.156:6824/27643 5 ==== osd_op_reply(50 [pgnls start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda53bb400 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5db1400 con 0x55dda5ee5800 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.2 v2:172.21.15.66:6816/27464 6 ==== osd_op_reply(51 [pgnls start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda5ee5800 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bac00 con 0x55dda52bdc00 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.1 v2:172.21.15.66:6821/27458 7 ==== osd_op_reply(52 [pgnls start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda52bdc00 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53ba400 con 0x55dda52bd000 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 12 ==== osd_op_reply(53 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda52bd800 con 0x55dda53ba000 2024-03-30T00:47:49.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 10 ==== osd_op_reply(54 [pgnls start_epoch 98 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bb000 con 0x55dda52bdc00 2024-03-30T00:47:49.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.1 v2:172.21.15.66:6821/27458 8 ==== osd_op_reply(55 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda52bdc00 2024-03-30T00:47:49.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cc800 con 0x55dda53ba000 2024-03-30T00:47:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 11 ==== osd_op_reply(56 [pgnls start_epoch 98 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5db1400 con 0x55dda53cc400 2024-03-30T00:47:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.5 v2:172.21.15.156:6808/27644 5 ==== osd_op_reply(57 [pgnls start_epoch 98 out=81b] v96'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53cc400 2024-03-30T00:47:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.381+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53dc000 con 0x55dda53cc400 2024-03-30T00:47:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.5 v2:172.21.15.156:6808/27644 6 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53cc400 2024-03-30T00:47:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f175d04a8c0 10 datascan.operator(): handling object 1099511627777.0 2024-03-30T00:47:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53dc400 con 0x55dda53cc400 2024-03-30T00:47:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.5 v2:172.21.15.156:6808/27644 7 ==== osd_op_reply(59 10000000001.00000000 [call] v98'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53cc400 2024-03-30T00:47:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53dc800 con 0x55dda53cc400 2024-03-30T00:47:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.5 v2:172.21.15.156:6808/27644 8 ==== osd_op_reply(60 [pgnls start_epoch 98 out=49b] v95'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda53cc400 2024-03-30T00:47:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bbc00 con 0x55dda52bdc00 2024-03-30T00:47:49.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.1 v2:172.21.15.66:6821/27458 9 ==== osd_op_reply(61 [pgnls start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda52bdc00 2024-03-30T00:47:49.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53cdc00 con 0x55dda53ba000 2024-03-30T00:47:49.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 <== osd.4 v2:172.21.15.156:6800/27641 12 ==== osd_op_reply(62 [pgnls start_epoch 98 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22fc0 con 0x55dda53ba000 2024-03-30T00:47:49.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53bb800 con 0x55dda53bb400 2024-03-30T00:47:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.7 v2:172.21.15.156:6824/27643 6 ==== osd_op_reply(63 [pgnls start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda53bb400 2024-03-30T00:47:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.385+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee4400 con 0x55dda52bd000 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 13 ==== osd_op_reply(64 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f1754c00640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda5ee5c00 con 0x55dda52bd000 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175b000640 1 -- 172.21.15.66:0/3825757189 <== osd.0 v2:172.21.15.66:6800/27456 14 ==== osd_op_reply(65 [pgnls start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22d80 con 0x55dda52bd000 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f1754200640 1 -- 172.21.15.66:0/3825757189 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55dda53dc000 con 0x55dda52bd400 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f1759c00640 1 -- 172.21.15.66:0/3825757189 <== osd.3 v2:172.21.15.66:6808/27471 6 ==== osd_op_reply(66 [pgnls start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55dda5d22b40 con 0x55dda52bd400 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55dda52bd000 msgr2=0x55dda53b0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55dda52bd000 0x55dda53b0000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55dda52bdc00 msgr2=0x55dda53b0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55dda52bdc00 0x55dda53b0580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x55dda5ee5800 msgr2=0x55dda5058100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x55dda5ee5800 0x55dda5058100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x55dda52bd400 msgr2=0x55dda53b0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x55dda52bd400 0x55dda53b0b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x55dda53ba000 msgr2=0x55dda53b1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x55dda53ba000 0x55dda53b1b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x55dda53cc400 msgr2=0x55dda53b2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x55dda53cc400 0x55dda53b2100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55dda53ba800 msgr2=0x55dda53b1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55dda53ba800 0x55dda53b1600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55dda53bb400 msgr2=0x55dda53b1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55dda53bb400 0x55dda53b1080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 reap_dead start 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 reap_dead start 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 reap_dead start 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175a600640 1 -- 172.21.15.66:0/3825757189 reap_dead start 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55dda52bcc00 msgr2=0x55dda5059700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55dda52bcc00 0x55dda5059700 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55dda5ef66f0 tx=0x55dda5d28360 comp rx=0 tx=0).stop 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee5000 msgr2=0x55dda5058c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee5000 0x55dda5058c00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x55dda5ef73e0 tx=0x55dda5ef8c00 comp rx=0 tx=0).stop 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 shutdown_connections 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55dda52bcc00 0x55dda5059700 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/3825757189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda5ee5000 0x55dda5058c00 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 >> 172.21.15.66:0/3825757189 conn(0x55dda4fcac00 msgr2=0x55dda4ff7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 shutdown_connections 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/3825757189 wait complete. 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/2120229253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda4fcb000 msgr2=0x55dda5057080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/2120229253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda4fcb000 0x55dda5057080 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x55dda5e3f380 tx=0x55dda5d28420 comp rx=0 tx=0).stop 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/2120229253 shutdown_connections 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/2120229253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dda4fcb000 0x55dda5057080 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/2120229253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda4fcb400 0x55dda5056b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 --2- 172.21.15.66:0/2120229253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55dda4fcb800 0x55dda5056580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/2120229253 >> 172.21.15.66:0/2120229253 conn(0x55dda4fcbc00 msgr2=0x55dda4ff7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/2120229253 shutdown_connections 2024-03-30T00:47:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.389+0000 7f175d04a8c0 1 -- 172.21.15.66:0/2120229253 wait complete. 2024-03-30T00:47:49.404 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.104936, result: 2024-03-30T00:47:49.404 DEBUG:teuthology.orchestra.run.smithi066:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-03-30T00:47:49.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff7d3e8c0 1 -- start start 2024-03-30T00:47:49.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff7d3e8c0 1 Processor -- start 2024-03-30T00:47:49.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff7d3e8c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1659b000 0x557d16626580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff7d3e8c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1659b400 0x557d16626b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff7d3e8c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1659b800 0x557d16627080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff7d3e8c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d16668a00 con 0x557d1659b400 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff7d3e8c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d16668780 con 0x557d1659b000 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff7d3e8c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d16668640 con 0x557d1659b800 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1659b400 0x557d16626b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1659b400 0x557d16626b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55016/0 (socket says 172.21.15.66:55016) 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff5400640 1 -- 172.21.15.66:0/2460862562 learned_addr learned my addr 172.21.15.66:0/2460862562 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff5e00640 1 --2- 172.21.15.66:0/2460862562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1659b000 0x557d16626580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff6800640 1 --2- 172.21.15.66:0/2460862562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1659b800 0x557d16627080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff5400640 1 -- 172.21.15.66:0/2460862562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1659b800 msgr2=0x557d16627080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff5400640 1 --2- 172.21.15.66:0/2460862562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1659b800 0x557d16627080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff5400640 1 -- 172.21.15.66:0/2460862562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1659b000 msgr2=0x557d16626580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff5400640 1 --2- 172.21.15.66:0/2460862562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1659b000 0x557d16626580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff5400640 1 -- 172.21.15.66:0/2460862562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d165ce9c0 con 0x557d1659b400 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff6800640 1 --2- 172.21.15.66:0/2460862562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1659b800 0x557d16627080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.481+0000 7fcff5400640 1 --2- 172.21.15.66:0/2460862562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1659b400 0x557d16626b00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x557d17400a80 tx=0x557d1736e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f857ea84c682e60 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff2c00640 1 -- 172.21.15.66:0/2460862562 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d165ef080 con 0x557d1659b400 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff2c00640 1 -- 172.21.15.66:0/2460862562 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557d17378300 con 0x557d1659b400 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff2c00640 1 -- 172.21.15.66:0/2460862562 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d165ef600 con 0x557d1659b400 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/2460862562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d165ce1a0 con 0x557d1659b400 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/2460862562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557d165ce9c0 con 0x557d1659b400 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 4 waiting for MDS map... 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff2c00640 1 -- 172.21.15.66:0/2460862562 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x557d165dac40 con 0x557d1659b400 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff2c00640 1 -- 172.21.15.66:0/2460862562 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 66) v1 ==== 2573+0+0 (secure 0 0 0) 0x557d165e8f00 con 0x557d1659b400 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 4 Got MDS map 66 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 4 datascan.main: Using metadata pool output 2024-03-30T00:47:49.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 4 datascan.main: connecting to RADOS... 2024-03-30T00:47:49.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 Processor -- start 2024-03-30T00:47:49.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 -- start start 2024-03-30T00:47:49.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1737f800 0x557d16627600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1737fc00 0x557d16627b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751e000 0x557d16628100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d165e1e00 con 0x557d1751e000 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d16668780 con 0x557d1737f800 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d16668a00 con 0x557d1737fc00 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751e000 0x557d16628100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751e000 0x557d16628100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55020/0 (socket says 172.21.15.66:55020) 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff5400640 1 -- 172.21.15.66:0/3278121533 learned_addr learned my addr 172.21.15.66:0/3278121533 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff5e00640 1 --2- 172.21.15.66:0/3278121533 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1737fc00 0x557d16627b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff6800640 1 --2- 172.21.15.66:0/3278121533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1737f800 0x557d16627600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff5400640 1 -- 172.21.15.66:0/3278121533 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1737fc00 msgr2=0x557d16627b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff5400640 1 --2- 172.21.15.66:0/3278121533 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1737fc00 0x557d16627b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff5400640 1 -- 172.21.15.66:0/3278121533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1737f800 msgr2=0x557d16627600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff5400640 1 --2- 172.21.15.66:0/3278121533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1737f800 0x557d16627600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff5400640 1 -- 172.21.15.66:0/3278121533 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d165ceb60 con 0x557d1751e000 2024-03-30T00:47:49.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff5400640 1 --2- 172.21.15.66:0/3278121533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751e000 0x557d16628100 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x557d17401770 tx=0x557d1736e300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d909c154bace12a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff0400640 1 -- 172.21.15.66:0/3278121533 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d165ef1e0 con 0x557d1751e000 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff0400640 1 -- 172.21.15.66:0/3278121533 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557d17378480 con 0x557d1751e000 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff0400640 1 -- 172.21.15.66:0/3278121533 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d165eef20 con 0x557d1751e000 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3278121533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751e000 msgr2=0x557d16628100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.485+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3278121533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751e000 0x557d16628100 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x557d17401770 tx=0x557d1736e300 comp rx=0 tx=0).stop 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3278121533 shutdown_connections 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3278121533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751e000 0x557d16628100 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3278121533 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1737fc00 0x557d16627b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3278121533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1737f800 0x557d16627600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3278121533 >> 172.21.15.66:0/3278121533 conn(0x557d1737f400 msgr2=0x557d165c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3278121533 shutdown_connections 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3278121533 wait complete. 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 Processor -- start 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 -- start start 2024-03-30T00:47:49.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1737fc00 0x557d16629180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1737f800 0x557d16628c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751f000 0x557d16628680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d16668500 con 0x557d1751f000 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d166683c0 con 0x557d1737f800 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d16668280 con 0x557d1737fc00 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751f000 0x557d16628680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff6800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1737fc00 0x557d16629180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751f000 0x557d16628680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55034/0 (socket says 172.21.15.66:55034) 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 learned_addr learned my addr 172.21.15.66:0/3871615408 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff6800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1737fc00 0x557d16629180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:35566/0 (socket says 172.21.15.66:35566) 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff5e00640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1737f800 0x557d16628c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1737fc00 msgr2=0x557d16629180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff5400640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1737fc00 0x557d16629180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1737f800 msgr2=0x557d16628c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff5400640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1737f800 0x557d16628c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d165ce1a0 con 0x557d1751f000 2024-03-30T00:47:49.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff5400640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751f000 0x557d16628680 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x557d174c8510 tx=0x557d1736e2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcfee600640 1 -- 172.21.15.66:0/3871615408 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d165ef1e0 con 0x557d1751f000 2024-03-30T00:47:49.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcfee600640 1 -- 172.21.15.66:0/3871615408 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557d17378c00 con 0x557d1751f000 2024-03-30T00:47:49.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcfee600640 1 -- 172.21.15.66:0/3871615408 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d165eef20 con 0x557d1751f000 2024-03-30T00:47:49.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557d165ce9c0 con 0x557d1751f000 2024-03-30T00:47:49.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d165ceb60 con 0x557d1751f000 2024-03-30T00:47:49.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.489+0000 7fcff7d3e8c0 4 datascan.init: resolving metadata pool 13 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcfee600640 1 -- 172.21.15.66:0/3871615408 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x557d165d2840 con 0x557d1751f000 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcfee600640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x557d16968000 0x557d16629700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcfee600640 1 -- 172.21.15.66:0/3871615408 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x557d165daa80 con 0x557d1751f000 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff7d3e8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff7d3e8c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff7d3e8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x557d1751ec00 0x557d16628100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff6800640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x557d16968000 0x557d16629700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e800 con 0x557d1751ec00 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff5e00640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x557d1751ec00 0x557d16628100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff6800640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x557d16968000 0x557d16629700 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x557d166ae960 tx=0x557d174d3ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff5e00640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x557d1751ec00 0x557d16628100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv24 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x557d1751e400 0x557d16627b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e000 con 0x557d1751e400 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff6800640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x557d1751e400 0x557d16627b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff6800640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x557d1751e400 0x557d16627b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x557d16969000 0x557d16627600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16969400 con 0x557d16969000 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.493+0000 7fcff5e00640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x557d16969000 0x557d16627600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff5e00640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x557d16969000 0x557d16627600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e800 con 0x557d1751ec00 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff0400640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x557d16969400 0x557d1698a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e000 con 0x557d16969400 2024-03-30T00:47:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff5400640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x557d16969400 0x557d1698a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff5400640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x557d16969400 0x557d1698a000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e800 con 0x557d1751ec00 2024-03-30T00:47:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e400 con 0x557d1751ec00 2024-03-30T00:47:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcfefa00640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x557d1698ec00 0x557d1698a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e800 con 0x557d1698ec00 2024-03-30T00:47:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff6800640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x557d1698ec00 0x557d1698a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff6800640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x557d1698ec00 0x557d1698a580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.3 v2:172.21.15.66:6808/27471 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698ec00 2024-03-30T00:47:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.497+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698f000 con 0x557d16969400 2024-03-30T00:47:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e400 con 0x557d16969400 2024-03-30T00:47:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698f400 con 0x557d1751e400 2024-03-30T00:47:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e800 con 0x557d16969400 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 98 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff0400640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x557d1698f800 0x557d1698ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e800 con 0x557d1698f800 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff5e00640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x557d1698f800 0x557d1698ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff5e00640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x557d1698f800 0x557d1698ab00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 98 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698fc00 con 0x557d16969000 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698f400 con 0x557d1751ec00 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcfefa00640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x557d169a0000 0x557d1698b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0400 con 0x557d169a0000 2024-03-30T00:47:49.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.501+0000 7fcff5400640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x557d169a0000 0x557d1698b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff5400640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x557d169a0000 0x557d1698b600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d169a0000 2024-03-30T00:47:49.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e000 con 0x557d1751ec00 2024-03-30T00:47:49.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0800 con 0x557d16969400 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 5 ==== osd_op_reply(18 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16968c00 con 0x557d1751e400 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0c00 con 0x557d1698f800 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e800 con 0x557d16969000 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1000 con 0x557d1698f800 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff0400640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x557d1698fc00 0x557d1698b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698f400 con 0x557d1698fc00 2024-03-30T00:47:49.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff6800640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x557d1698fc00 0x557d1698b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff6800640 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x557d1698fc00 0x557d1698b080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.505+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0400 con 0x557d1698f800 2024-03-30T00:47:49.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e000 con 0x557d1751ec00 2024-03-30T00:47:49.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1400 con 0x557d1751ec00 2024-03-30T00:47:49.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698f000 con 0x557d1751ec00 2024-03-30T00:47:49.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e000 con 0x557d1698f800 2024-03-30T00:47:49.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0c00 con 0x557d1698ec00 2024-03-30T00:47:49.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.3 v2:172.21.15.66:6808/27471 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698ec00 2024-03-30T00:47:49.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751fc00 con 0x557d16969000 2024-03-30T00:47:49.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1000 con 0x557d1698f800 2024-03-30T00:47:49.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e800 con 0x557d1698ec00 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.3 v2:172.21.15.66:6808/27471 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698ec00 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0400 con 0x557d169a0000 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.7 v2:172.21.15.156:6824/27643 2 ==== osd_op_reply(33 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d169a0000 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e400 con 0x557d1698fc00 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 98 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.509+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1800 con 0x557d1698ec00 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.3 v2:172.21.15.66:6808/27471 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698ec00 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0c00 con 0x557d1698fc00 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 98 out=49b] v95'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e800 con 0x557d1698f800 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1c00 con 0x557d16969400 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 6 ==== osd_op_reply(38 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e000 con 0x557d1698fc00 2024-03-30T00:47:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 4 ==== osd_op_reply(39 [pgnls-filter start_epoch 98 out=49b] v95'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698f400 con 0x557d1751e400 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698f000 con 0x557d1751e400 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e400 con 0x557d169a0000 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.7 v2:172.21.15.156:6824/27643 3 ==== osd_op_reply(42 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d169a0000 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e800 con 0x557d1751ec00 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.513+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e800 con 0x557d16969000 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1400 con 0x557d169a0000 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.7 v2:172.21.15.156:6824/27643 4 ==== osd_op_reply(45 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d169a0000 2024-03-30T00:47:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16968c00 con 0x557d1698f800 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1000 con 0x557d1751e400 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1800 con 0x557d1698f800 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e000 con 0x557d1751ec00 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0c00 con 0x557d1698ec00 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.3 v2:172.21.15.66:6808/27471 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698ec00 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751fc00 con 0x557d1751ec00 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1000 con 0x557d169a0000 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.7 v2:172.21.15.156:6824/27643 5 ==== osd_op_reply(52 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d169a0000 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16994000 con 0x557d16969000 2024-03-30T00:47:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e000 con 0x557d16969400 2024-03-30T00:47:49.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16994400 con 0x557d1751ec00 2024-03-30T00:47:49.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.517+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698f400 con 0x557d1698f800 2024-03-30T00:47:49.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 98 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16994800 con 0x557d16969400 2024-03-30T00:47:49.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698f000 con 0x557d1698f800 2024-03-30T00:47:49.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 98 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 98 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16994c00 con 0x557d1698fc00 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 98 out=81b] v98'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16969800 con 0x557d16969000 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16995000 con 0x557d1751ec00 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0800 con 0x557d16969400 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16995400 con 0x557d1751ec00 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0400 con 0x557d1751ec00 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16995800 con 0x557d1698ec00 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.3 v2:172.21.15.66:6808/27471 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698ec00 2024-03-30T00:47:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1c00 con 0x557d16969400 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 10 ==== osd_op_reply(66 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16995c00 con 0x557d16969400 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 11 ==== osd_op_reply(67 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1400 con 0x557d1751e400 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.521+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16994000 con 0x557d16969400 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 98 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1000 con 0x557d1698f800 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 98 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e800 con 0x557d16969000 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16968c00 con 0x557d1751ec00 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e000 con 0x557d169a0000 2024-03-30T00:47:49.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.7 v2:172.21.15.156:6824/27643 6 ==== osd_op_reply(73 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d169a0000 2024-03-30T00:47:49.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e400 con 0x557d1751ec00 2024-03-30T00:47:49.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751fc00 con 0x557d16969400 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e000 con 0x557d1751e400 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1800 con 0x557d1698f800 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e800 con 0x557d16969000 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 9 ==== osd_op_reply(78 [pgnls-filter start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0400 con 0x557d1698f800 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.525+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169be000 con 0x557d1698fc00 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 6 ==== osd_op_reply(80 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1000 con 0x557d1698f800 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169be400 con 0x557d1751ec00 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16994400 con 0x557d1751ec00 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169be800 con 0x557d1751ec00 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16995000 con 0x557d1698f800 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169bec00 con 0x557d1698ec00 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.3 v2:172.21.15.66:6808/27471 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698ec00 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16995400 con 0x557d16969000 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 10 ==== osd_op_reply(87 [pgnls-filter start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169bf000 con 0x557d1698f800 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16969800 con 0x557d1698ec00 2024-03-30T00:47:49.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.3 v2:172.21.15.66:6808/27471 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698ec00 2024-03-30T00:47:49.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169bf400 con 0x557d169a0000 2024-03-30T00:47:49.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.7 v2:172.21.15.156:6824/27643 7 ==== osd_op_reply(90 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d169a0000 2024-03-30T00:47:49.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.529+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698f400 con 0x557d1698fc00 2024-03-30T00:47:49.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 98 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169bf800 con 0x557d1698ec00 2024-03-30T00:47:49.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.3 v2:172.21.15.66:6808/27471 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698ec00 2024-03-30T00:47:49.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698f000 con 0x557d1698fc00 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 8 ==== osd_op_reply(93 [pgnls-filter start_epoch 98 out=49b] v95'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169bfc00 con 0x557d1698f800 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0800 con 0x557d16969400 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 14 ==== osd_op_reply(95 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16994800 con 0x557d1698fc00 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 9 ==== osd_op_reply(96 [pgnls-filter start_epoch 98 out=49b] v95'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1c00 con 0x557d1751e400 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16995c00 con 0x557d1751e400 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0c00 con 0x557d169a0000 2024-03-30T00:47:49.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.7 v2:172.21.15.156:6824/27643 8 ==== osd_op_reply(99 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d169a0000 2024-03-30T00:47:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16994000 con 0x557d1751ec00 2024-03-30T00:47:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 98 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.533+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1400 con 0x557d16969000 2024-03-30T00:47:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 11 ==== osd_op_reply(101 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16994c00 con 0x557d169a0000 2024-03-30T00:47:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.7 v2:172.21.15.156:6824/27643 9 ==== osd_op_reply(102 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d169a0000 2024-03-30T00:47:49.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e000 con 0x557d1698f800 2024-03-30T00:47:49.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169bf400 con 0x557d1751e400 2024-03-30T00:47:49.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e400 con 0x557d1698f800 2024-03-30T00:47:49.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169be000 con 0x557d1751ec00 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e800 con 0x557d1698ec00 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.3 v2:172.21.15.66:6808/27471 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698ec00 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169bec00 con 0x557d1751ec00 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16968c00 con 0x557d169a0000 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.7 v2:172.21.15.156:6824/27643 10 ==== osd_op_reply(109 [pgnls-filter start_epoch 98 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d169a0000 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169be400 con 0x557d16969000 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.2 v2:172.21.15.66:6816/27464 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 98 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d16969000 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.537+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e000 con 0x557d16969400 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 15 ==== osd_op_reply(111 [pgnls-filter start_epoch 98 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1800 con 0x557d1751ec00 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751fc00 con 0x557d1698f800 2024-03-30T00:47:49.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 98 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a0400 con 0x557d16969400 2024-03-30T00:47:49.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 16 ==== osd_op_reply(114 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1751e800 con 0x557d1698f800 2024-03-30T00:47:49.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 98 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1000 con 0x557d1698fc00 2024-03-30T00:47:49.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 10 ==== osd_op_reply(116 [pgnls-filter start_epoch 98 out=81b] v98'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 10 datascan.operator(): handling object 10000000001.0 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d16969c00 con 0x557d1698fc00 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 11 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169be800 con 0x557d16969400 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 17 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169bf000 con 0x557d1751e400 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.6 v2:172.21.15.156:6816/27642 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+520 (crc 0 0 0) 0x557d174a4d80 con 0x557d1751e400 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-03-30T00:47:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-03-30T00:47:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 20 datascan.get_frag_of: frag is * 2024-03-30T00:47:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1400 con 0x557d16969400 2024-03-30T00:47:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 18 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-03-30T00:47:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-03-30T00:47:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.541+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169a1c00 con 0x557d16969400 2024-03-30T00:47:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 19 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff7d3e8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-30T00:47:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff7d3e8c0 20 datascan.read_dentry: file_head not found in result 2024-03-30T00:47:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff7d3e8c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-03-30T00:47:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e000 con 0x557d16969400 2024-03-30T00:47:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 20 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v98'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff7d3e8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-30T00:47:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d1698e400 con 0x557d1698fc00 2024-03-30T00:47:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.5 v2:172.21.15.156:6808/27644 12 ==== osd_op_reply(123 [pgnls-filter start_epoch 98 out=49b] v95'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698fc00 2024-03-30T00:47:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169be800 con 0x557d16969400 2024-03-30T00:47:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.1 v2:172.21.15.66:6821/27458 21 ==== osd_op_reply(124 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d16969400 2024-03-30T00:47:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169cc000 con 0x557d1698f800 2024-03-30T00:47:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.4 v2:172.21.15.156:6800/27641 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 98 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1698f800 2024-03-30T00:47:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169bfc00 con 0x557d169a0000 2024-03-30T00:47:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff5400640 1 -- 172.21.15.66:0/3871615408 <== osd.7 v2:172.21.15.156:6824/27643 11 ==== osd_op_reply(126 [pgnls-filter start_epoch 98 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4fc0 con 0x557d169a0000 2024-03-30T00:47:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169cc400 con 0x557d1751ec00 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.545+0000 7fcfefa00640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169be000 con 0x557d1751ec00 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 <== osd.0 v2:172.21.15.66:6800/27456 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 98 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4b40 con 0x557d1751ec00 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff0400640 1 -- 172.21.15.66:0/3871615408 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 98 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x557d169cc800 con 0x557d1698ec00 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff6800640 1 -- 172.21.15.66:0/3871615408 <== osd.3 v2:172.21.15.66:6808/27471 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 98 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d174a4d80 con 0x557d1698ec00 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x557d1751ec00 msgr2=0x557d16628100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x557d1751ec00 0x557d16628100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x557d16969400 msgr2=0x557d1698a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x557d16969400 0x557d1698a000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x557d16969000 msgr2=0x557d16627600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x557d16969000 0x557d16627600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x557d1698ec00 msgr2=0x557d1698a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x557d1698ec00 0x557d1698a580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x557d1698f800 msgr2=0x557d1698ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x557d1698f800 0x557d1698ab00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x557d1698fc00 msgr2=0x557d1698b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x557d1698fc00 0x557d1698b080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x557d1751e400 msgr2=0x557d16627b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x557d1751e400 0x557d16627b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 reap_dead start 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x557d169a0000 msgr2=0x557d1698b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x557d169a0000 0x557d1698b600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 reap_dead start 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 reap_dead start 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 reap_dead start 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff5e00640 1 -- 172.21.15.66:0/3871615408 reap_dead start 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x557d16968000 msgr2=0x557d16629700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x557d16968000 0x557d16629700 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x557d166ae960 tx=0x557d174d3ce0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751f000 msgr2=0x557d16628680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751f000 0x557d16628680 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x557d174c8510 tx=0x557d1736e2a0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 shutdown_connections 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x557d16968000 0x557d16629700 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/3871615408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1751f000 0x557d16628680 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 >> 172.21.15.66:0/3871615408 conn(0x557d1737f400 msgr2=0x557d165c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 shutdown_connections 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/3871615408 wait complete. 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/2460862562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1659b400 msgr2=0x557d16626b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/2460862562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1659b400 0x557d16626b00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x557d17400a80 tx=0x557d1736e000 comp rx=0 tx=0).stop 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/2460862562 shutdown_connections 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/2460862562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557d1659b800 0x557d16627080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/2460862562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d1659b400 0x557d16626b00 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 --2- 172.21.15.66:0/2460862562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d1659b000 0x557d16626580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/2460862562 >> 172.21.15.66:0/2460862562 conn(0x557d1659ac00 msgr2=0x557d165c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/2460862562 shutdown_connections 2024-03-30T00:47:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.549+0000 7fcff7d3e8c0 1 -- 172.21.15.66:0/2460862562 wait complete. 2024-03-30T00:47:49.564 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.160056, result: 2024-03-30T00:47:49.564 DEBUG:teuthology.orchestra.run.smithi066:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-03-30T00:47:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e04f668c0 1 -- start start 2024-03-30T00:47:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e04f668c0 1 Processor -- start 2024-03-30T00:47:49.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e04f668c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35673d800 0x55b3567c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e04f668c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b35673d400 0x55b3567c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e04f668c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35673d000 0x55b3567c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e04f668c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35680a780 con 0x55b35673d000 2024-03-30T00:47:49.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e04f668c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35680aa00 con 0x55b35673d800 2024-03-30T00:47:49.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e04f668c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b35680a640 con 0x55b35673d400 2024-03-30T00:47:49.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e03a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35673d000 0x55b3567c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e03000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35673d800 0x55b3567c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e03a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35673d000 0x55b3567c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55048/0 (socket says 172.21.15.66:55048) 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e03a00640 1 -- 172.21.15.66:0/923234378 learned_addr learned my addr 172.21.15.66:0/923234378 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e02600640 1 --2- 172.21.15.66:0/923234378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b35673d400 0x55b3567c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e03a00640 1 -- 172.21.15.66:0/923234378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b35673d400 msgr2=0x55b3567c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e03a00640 1 --2- 172.21.15.66:0/923234378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b35673d400 0x55b3567c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e03a00640 1 -- 172.21.15.66:0/923234378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35673d800 msgr2=0x55b3567c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e03a00640 1 --2- 172.21.15.66:0/923234378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35673d800 0x55b3567c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e03a00640 1 -- 172.21.15.66:0/923234378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3567709c0 con 0x55b35673d000 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e03a00640 1 --2- 172.21.15.66:0/923234378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35673d000 0x55b3567c9080 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x55b3575a89c0 tx=0x55b357652120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2f4aa136dc98d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0dffe00640 1 -- 172.21.15.66:0/923234378 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b356791080 con 0x55b35673d000 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0dffe00640 1 -- 172.21.15.66:0/923234378 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b357518480 con 0x55b35673d000 2024-03-30T00:47:49.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e04f668c0 1 -- 172.21.15.66:0/923234378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3567701a0 con 0x55b35673d000 2024-03-30T00:47:49.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e04f668c0 1 -- 172.21.15.66:0/923234378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b3567709c0 con 0x55b35673d000 2024-03-30T00:47:49.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0e04f668c0 4 waiting for MDS map... 2024-03-30T00:47:49.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.641+0000 7f0dffe00640 1 -- 172.21.15.66:0/923234378 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b356791600 con 0x55b35673d000 2024-03-30T00:47:49.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0dffe00640 1 -- 172.21.15.66:0/923234378 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x55b35677cc40 con 0x55b35673d000 2024-03-30T00:47:49.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0dffe00640 1 -- 172.21.15.66:0/923234378 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 66) v1 ==== 2573+0+0 (secure 0 0 0) 0x55b35678b800 con 0x55b35673d000 2024-03-30T00:47:49.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 4 Got MDS map 66 2024-03-30T00:47:49.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 4 datascan.main: Using metadata pool output 2024-03-30T00:47:49.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 4 datascan.main: connecting to RADOS... 2024-03-30T00:47:49.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 Processor -- start 2024-03-30T00:47:49.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 -- start start 2024-03-30T00:47:49.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751b800 0x55b3567c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35751bc00 0x55b3567c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b3576be000 0x55b3567ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b356783e00 con 0x55b35751b800 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35680aa00 con 0x55b35751bc00 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b35680a780 con 0x55b3576be000 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e02600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751b800 0x55b3567c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e02600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751b800 0x55b3567c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55056/0 (socket says 172.21.15.66:55056) 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e03000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35751bc00 0x55b3567c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e02600640 1 -- 172.21.15.66:0/2301558268 learned_addr learned my addr 172.21.15.66:0/2301558268 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e02600640 1 --2- 172.21.15.66:0/2301558268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b3576be000 0x55b3567ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e02600640 1 -- 172.21.15.66:0/2301558268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b3576be000 msgr2=0x55b3567ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e02600640 1 --2- 172.21.15.66:0/2301558268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b3576be000 0x55b3567ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e02600640 1 -- 172.21.15.66:0/2301558268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35751bc00 msgr2=0x55b3567c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e02600640 1 --2- 172.21.15.66:0/2301558268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35751bc00 0x55b3567c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e02600640 1 -- 172.21.15.66:0/2301558268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b356770b60 con 0x55b35751b800 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e02600640 1 --2- 172.21.15.66:0/2301558268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751b800 0x55b3567c9600 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x55b3576c0120 tx=0x55b35766cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344450e111233cf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0dfd600640 1 -- 172.21.15.66:0/2301558268 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b356791080 con 0x55b35751b800 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0dfd600640 1 -- 172.21.15.66:0/2301558268 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b35679a900 con 0x55b35751b800 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0dfd600640 1 -- 172.21.15.66:0/2301558268 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b356791600 con 0x55b35751b800 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 -- 172.21.15.66:0/2301558268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751b800 msgr2=0x55b3567c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/2301558268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751b800 0x55b3567c9600 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x55b3576c0120 tx=0x55b35766cc60 comp rx=0 tx=0).stop 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 -- 172.21.15.66:0/2301558268 shutdown_connections 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/2301558268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b3576be000 0x55b3567ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/2301558268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35751bc00 0x55b3567c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/2301558268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751b800 0x55b3567c9600 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 -- 172.21.15.66:0/2301558268 >> 172.21.15.66:0/2301558268 conn(0x55b35673cc00 msgr2=0x55b356769800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 -- 172.21.15.66:0/2301558268 shutdown_connections 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.645+0000 7f0e04f668c0 1 -- 172.21.15.66:0/2301558268 wait complete. 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e04f668c0 1 Processor -- start 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e04f668c0 1 -- start start 2024-03-30T00:47:49.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e04f668c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751bc00 0x55b3567cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e04f668c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b35751b800 0x55b3567cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e04f668c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3576bf800 0x55b3567ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e04f668c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35680a500 con 0x55b35751bc00 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e04f668c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35680a3c0 con 0x55b3576bf800 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e04f668c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b35680a280 con 0x55b35751b800 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e02600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751bc00 0x55b3567cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e02600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751bc00 0x55b3567cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55062/0 (socket says 172.21.15.66:55062) 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 learned_addr learned my addr 172.21.15.66:0/282548703 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e03000640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b35751b800 0x55b3567cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3576bf800 0x55b3567ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b35751b800 msgr2=0x55b3567cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b35751b800 0x55b3567cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3576bf800 msgr2=0x55b3567ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b3576bf800 0x55b3567ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b356770340 con 0x55b35751bc00 2024-03-30T00:47:49.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751bc00 0x55b3567cb180 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x55b3576c13e0 tx=0x55b35766cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0dfb800640 1 -- 172.21.15.66:0/282548703 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b356790dc0 con 0x55b35751bc00 2024-03-30T00:47:49.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0dfb800640 1 -- 172.21.15.66:0/282548703 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b357519500 con 0x55b35751bc00 2024-03-30T00:47:49.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0dfb800640 1 -- 172.21.15.66:0/282548703 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b356791080 con 0x55b35751bc00 2024-03-30T00:47:49.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b356770000 con 0x55b35751bc00 2024-03-30T00:47:49.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b356770ea0 con 0x55b35751bc00 2024-03-30T00:47:49.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0e04f668c0 4 datascan.init: resolving metadata pool 13 2024-03-30T00:47:49.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.649+0000 7f0dfb800640 1 -- 172.21.15.66:0/282548703 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x55b356774840 con 0x55b35751bc00 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0dfb800640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55b3576c6800 0x55b3567cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0dfb800640 1 -- 172.21.15.66:0/282548703 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x55b35677ca80 con 0x55b35751bc00 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e04f668c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e04f668c0 4 datascan.main: resolving metadata pool 13 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e03000640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55b3576c6800 0x55b3567cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55b3576bf400 0x55b3567ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576bf000 con 0x55b3576bf400 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55b3576bf400 0x55b3567ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e03000640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55b3576c6800 0x55b3567cb700 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55b356850690 tx=0x55b35766dec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55b3576bf400 0x55b3567ca100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== osd_op_reply(1 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55b3576bf000 0x55b356b1e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576c7800 con 0x55b3576bf000 2024-03-30T00:47:49.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e03a00640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55b3576bf000 0x55b356b1e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e03a00640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55b3576bf000 0x55b356b1e000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(2 [pgnls start_epoch 98 out=177b] v98'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e03a00640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x55b3576c7800 0x55b356b1e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b35751b400 con 0x55b3576c7800 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e03000640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x55b3576c7800 0x55b356b1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.653+0000 7f0e03000640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x55b3576c7800 0x55b356b1e580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.2 v2:172.21.15.66:6816/27464 1 ==== osd_op_reply(3 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cfc0 con 0x55b3576c7800 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b35751b400 con 0x55b3576bf400 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 2 ==== osd_op_reply(4 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55b35751b400 0x55b356b1eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576c7c00 con 0x55b35751b400 2024-03-30T00:47:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55b35751b400 0x55b356b1eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55b35751b400 0x55b356b1eb00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(5 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55b3576c7c00 0x55b356b1f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e000 con 0x55b3576c7c00 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e03a00640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55b3576c7c00 0x55b356b1f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e03a00640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55b3576c7c00 0x55b356b1f080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== osd_op_reply(6 [pgnls start_epoch 98 out=149b] v97'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576c7c00 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e000 con 0x55b3576bf000 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 2 ==== osd_op_reply(7 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e000 con 0x55b35751b400 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 2 ==== osd_op_reply(8 [pgnls start_epoch 98 out=73b] v97'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x55b356b2e000 0x55b356b1f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e400 con 0x55b356b2e000 2024-03-30T00:47:49.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.657+0000 7f0e03000640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x55b356b2e000 0x55b356b1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03000640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x55b356b2e000 0x55b356b1f600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== osd_op_reply(9 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cfc0 con 0x55b356b2e000 2024-03-30T00:47:49.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e400 con 0x55b3576c7c00 2024-03-30T00:47:49.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.0 v2:172.21.15.66:6800/27456 2 ==== osd_op_reply(10 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576c7c00 2024-03-30T00:47:49.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e400 con 0x55b3576c7800 2024-03-30T00:47:49.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.2 v2:172.21.15.66:6816/27464 2 ==== osd_op_reply(11 [pgnls start_epoch 98 out=81b] v97'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b35764cfc0 con 0x55b3576c7800 2024-03-30T00:47:49.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e400 con 0x55b3576bf000 2024-03-30T00:47:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 3 ==== osd_op_reply(12 [pgnls start_epoch 98 out=49b] v97'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03a00640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x55b356b2e400 0x55b356b20680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2f000 con 0x55b356b2e400 2024-03-30T00:47:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x55b356b2e400 0x55b356b20680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x55b356b2e400 0x55b356b20680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 1 ==== osd_op_reply(13 [pgnls start_epoch 98 out=77b] v97'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e02600640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x55b356b2f000 0x55b356b1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b356b2f000 2024-03-30T00:47:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03a00640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x55b356b2f000 0x55b356b1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03a00640 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x55b356b2f000 0x55b356b1fb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.3 v2:172.21.15.66:6808/27471 1 ==== osd_op_reply(14 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b356b2f000 2024-03-30T00:47:49.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.661+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b3576bf400 2024-03-30T00:47:49.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 3 ==== osd_op_reply(15 [pgnls start_epoch 98 out=73b] v97'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b356b2e400 2024-03-30T00:47:49.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 2 ==== osd_op_reply(16 [pgnls start_epoch 98 out=49b] v97'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b35751b400 2024-03-30T00:47:49.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 3 ==== osd_op_reply(17 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b3576bf000 2024-03-30T00:47:49.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 4 ==== osd_op_reply(18 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b356b2e000 2024-03-30T00:47:49.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.5 v2:172.21.15.156:6808/27644 2 ==== osd_op_reply(19 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cfc0 con 0x55b356b2e000 2024-03-30T00:47:49.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b356b2f000 2024-03-30T00:47:49.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.3 v2:172.21.15.66:6808/27471 2 ==== osd_op_reply(20 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b356b2f000 2024-03-30T00:47:49.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b356b2e400 2024-03-30T00:47:49.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 3 ==== osd_op_reply(21 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b356b2e400 2024-03-30T00:47:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 4 ==== osd_op_reply(22 [pgnls start_epoch 98 out=97b] v97'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b356b2e400 2024-03-30T00:47:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 5 ==== osd_op_reply(23 [pgnls start_epoch 98 out=74b] v94'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b3576bf400 2024-03-30T00:47:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 4 ==== osd_op_reply(24 [pgnls start_epoch 98 out=73b] v97'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.665+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b3576c7c00 2024-03-30T00:47:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.0 v2:172.21.15.66:6800/27456 3 ==== osd_op_reply(25 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576c7c00 2024-03-30T00:47:49.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b3576bf400 2024-03-30T00:47:49.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 5 ==== osd_op_reply(26 [pgnls start_epoch 98 out=98b] v97'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b356b2e000 2024-03-30T00:47:49.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.5 v2:172.21.15.156:6808/27644 3 ==== osd_op_reply(27 [pgnls start_epoch 98 out=76b] v97'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55b35764cfc0 con 0x55b356b2e000 2024-03-30T00:47:49.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b356b2e400 2024-03-30T00:47:49.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 6 ==== osd_op_reply(28 [pgnls start_epoch 98 out=73b] v97'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b3576bf400 2024-03-30T00:47:49.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 6 ==== osd_op_reply(29 [pgnls start_epoch 98 out=73b] v97'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b356b2e000 2024-03-30T00:47:49.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.5 v2:172.21.15.156:6808/27644 4 ==== osd_op_reply(30 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cfc0 con 0x55b356b2e000 2024-03-30T00:47:49.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b35751b400 2024-03-30T00:47:49.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 4 ==== osd_op_reply(31 [pgnls start_epoch 98 out=73b] v94'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b35751b400 2024-03-30T00:47:49.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 5 ==== osd_op_reply(32 [pgnls start_epoch 98 out=101b] v97'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-03-30T00:47:49.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576bec00 con 0x55b3576bf000 2024-03-30T00:47:49.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-03-30T00:47:49.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576be800 con 0x55b3576bf000 2024-03-30T00:47:49.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-03-30T00:47:49.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.669+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576be400 con 0x55b3576bf000 2024-03-30T00:47:49.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 20 datascan.scan_links: injected ino detected 2024-03-30T00:47:49.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-03-30T00:47:49.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576be000 con 0x55b3576bf000 2024-03-30T00:47:49.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-03-30T00:47:49.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576bec00 con 0x55b3576bf000 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576be800 con 0x55b3576c7c00 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.0 v2:172.21.15.66:6800/27456 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576c7c00 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576be400 con 0x55b35751b400 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-03-30T00:47:49.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2fc00 con 0x55b3576c7800 2024-03-30T00:47:49.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.2 v2:172.21.15.66:6816/27464 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55b35764cfc0 con 0x55b3576c7800 2024-03-30T00:47:49.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-03-30T00:47:49.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-30T00:47:49.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-03-30T00:47:49.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2f800 con 0x55b3576bf400 2024-03-30T00:47:49.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-03-30T00:47:49.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2f400 con 0x55b356b2e400 2024-03-30T00:47:49.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-03-30T00:47:49.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-30T00:47:49.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-03-30T00:47:49.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-30T00:47:49.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-03-30T00:47:49.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b46000 con 0x55b3576bf400 2024-03-30T00:47:49.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b46400 con 0x55b3576bf400 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.673+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b46800 con 0x55b3576bf400 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-03-30T00:47:49.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e04f668c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b46c00 con 0x55b35751b400 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576bf400 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 11 ==== osd_op_reply(47 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576bf000 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 10 ==== osd_op_reply(48 [pgnls start_epoch 98 out=177b] v98'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576c7800 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.2 v2:172.21.15.66:6816/27464 4 ==== osd_op_reply(49 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cfc0 con 0x55b3576c7800 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576bf400 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 12 ==== osd_op_reply(50 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b35751b400 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 8 ==== osd_op_reply(51 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576c7c00 2024-03-30T00:47:49.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.0 v2:172.21.15.66:6800/27456 5 ==== osd_op_reply(52 [pgnls start_epoch 98 out=149b] v97'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576c7c00 2024-03-30T00:47:49.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576bf000 2024-03-30T00:47:49.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 11 ==== osd_op_reply(53 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b35751b400 2024-03-30T00:47:49.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 9 ==== osd_op_reply(54 [pgnls start_epoch 98 out=73b] v97'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2e000 2024-03-30T00:47:49.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.5 v2:172.21.15.156:6808/27644 5 ==== osd_op_reply(55 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cfc0 con 0x55b356b2e000 2024-03-30T00:47:49.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.677+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576c7c00 2024-03-30T00:47:49.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.0 v2:172.21.15.66:6800/27456 6 ==== osd_op_reply(56 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576c7c00 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576c7800 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.2 v2:172.21.15.66:6816/27464 5 ==== osd_op_reply(57 [pgnls start_epoch 98 out=81b] v97'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b35764cfc0 con 0x55b3576c7800 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576bf000 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 12 ==== osd_op_reply(58 [pgnls start_epoch 98 out=49b] v97'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2e400 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 8 ==== osd_op_reply(59 [pgnls start_epoch 98 out=77b] v97'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2f000 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.3 v2:172.21.15.66:6808/27471 3 ==== osd_op_reply(60 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b356b2f000 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576bf400 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 13 ==== osd_op_reply(61 [pgnls start_epoch 98 out=73b] v97'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2e400 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 9 ==== osd_op_reply(62 [pgnls start_epoch 98 out=49b] v97'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b35751b400 2024-03-30T00:47:49.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 10 ==== osd_op_reply(63 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576bf000 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 13 ==== osd_op_reply(64 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2e000 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.5 v2:172.21.15.156:6808/27644 6 ==== osd_op_reply(65 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cfc0 con 0x55b356b2e000 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2f000 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.3 v2:172.21.15.66:6808/27471 4 ==== osd_op_reply(66 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b356b2f000 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.681+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2e400 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 10 ==== osd_op_reply(67 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2e400 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 11 ==== osd_op_reply(68 [pgnls start_epoch 98 out=97b] v97'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2e400 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 12 ==== osd_op_reply(69 [pgnls start_epoch 98 out=74b] v94'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576bf400 2024-03-30T00:47:49.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 14 ==== osd_op_reply(70 [pgnls start_epoch 98 out=73b] v97'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576c7c00 2024-03-30T00:47:49.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.0 v2:172.21.15.66:6800/27456 7 ==== osd_op_reply(71 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576c7c00 2024-03-30T00:47:49.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576bf400 2024-03-30T00:47:49.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 15 ==== osd_op_reply(72 [pgnls start_epoch 98 out=98b] v97'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2e000 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.5 v2:172.21.15.156:6808/27644 7 ==== osd_op_reply(73 [pgnls start_epoch 98 out=76b] v97'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55b35764cfc0 con 0x55b356b2e000 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2e400 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 13 ==== osd_op_reply(74 [pgnls start_epoch 98 out=73b] v97'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576bf400 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 16 ==== osd_op_reply(75 [pgnls start_epoch 98 out=73b] v97'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b356b2e000 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.5 v2:172.21.15.156:6808/27644 8 ==== osd_op_reply(76 [pgnls start_epoch 98 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b35764cfc0 con 0x55b356b2e000 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b35751b400 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 11 ==== osd_op_reply(77 [pgnls start_epoch 98 out=73b] v94'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.685+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 98 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b35751b400 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 12 ==== osd_op_reply(78 [pgnls start_epoch 98 out=101b] v97'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-03-30T00:47:49.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47400 con 0x55b3576bf000 2024-03-30T00:47:49.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-03-30T00:47:49.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47800 con 0x55b3576bf000 2024-03-30T00:47:49.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-03-30T00:47:49.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47c00 con 0x55b3576bf000 2024-03-30T00:47:49.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 20 datascan.scan_links: injected ino detected 2024-03-30T00:47:49.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-03-30T00:47:49.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-03-30T00:47:49.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-03-30T00:47:49.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b46000 con 0x55b3576bf000 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2ec00 con 0x55b3576bf000 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2e800 con 0x55b3576c7c00 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.0 v2:172.21.15.66:6800/27456 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576c7c00 2024-03-30T00:47:49.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2f800 con 0x55b35751b400 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b2f400 con 0x55b3576c7800 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 <== osd.2 v2:172.21.15.66:6816/27464 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55b35764cfc0 con 0x55b3576c7800 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576be400 con 0x55b3576bf400 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47400 con 0x55b356b2e400 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-03-30T00:47:49.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47800 con 0x55b3576bf400 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.689+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47c00 con 0x55b3576bf400 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-03-30T00:47:49.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b46000 con 0x55b3576bf400 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576be000 con 0x55b35751b400 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.6 v2:172.21.15.156:6816/27642 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x55b35764cb40 con 0x55b35751b400 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-03-30T00:47:49.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-03-30T00:47:49.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576bec00 con 0x55b3576bf000 2024-03-30T00:47:49.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-30T00:47:49.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b3576be800 con 0x55b3576bf000 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v98'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: processing 1 injected_inos 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b46400 con 0x55b3576bf000 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 20 datascan.scan_links: first is now 2 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.693+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b46800 con 0x55b3576bf000 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e03a00640 1 -- 172.21.15.66:0/282548703 <== osd.1 v2:172.21.15.66:6821/27458 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v98'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b35764cd80 con 0x55b3576bf000 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-30T00:47:49.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 10 datascan.scan_links: updating inotable 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b46c00 con 0x55b3576bf400 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47000 con 0x55b3576bf400 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.7 v2:172.21.15.156:6824/27643 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v98'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55b35764cb40 con 0x55b3576bf400 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 10 datascan.scan_links: updating snaptable 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v8 -- 0x55b356b47400 con 0x55b356b2e400 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e02600640 1 -- 172.21.15.66:0/282548703 <== osd.4 v2:172.21.15.156:6800/27641 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55b35764cb40 con 0x55b356b2e400 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55b3576c7c00 msgr2=0x55b356b1f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55b3576c7c00 0x55b356b1f080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55b3576bf000 msgr2=0x55b356b1e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55b3576bf000 0x55b356b1e000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x55b3576c7800 msgr2=0x55b356b1e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x55b3576c7800 0x55b356b1e580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x55b356b2f000 msgr2=0x55b356b1fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x55b356b2f000 0x55b356b1fb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x55b356b2e400 msgr2=0x55b356b20680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x55b356b2e400 0x55b356b20680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 reap_dead start 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x55b356b2e000 msgr2=0x55b356b1f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x55b356b2e000 0x55b356b1f600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55b35751b400 msgr2=0x55b356b1eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55b35751b400 0x55b356b1eb00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55b3576bf400 msgr2=0x55b3567ca100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55b3576bf400 0x55b3567ca100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e03000640 1 -- 172.21.15.66:0/282548703 reap_dead start 2024-03-30T00:47:49.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55b3576c6800 msgr2=0x55b3567cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55b3576c6800 0x55b3567cb700 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55b356850690 tx=0x55b35766dec0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751bc00 msgr2=0x55b3567cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751bc00 0x55b3567cb180 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x55b3576c13e0 tx=0x55b35766cc00 comp rx=0 tx=0).stop 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 shutdown_connections 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55b3576c6800 0x55b3567cb700 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/282548703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35751bc00 0x55b3567cb180 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 >> 172.21.15.66:0/282548703 conn(0x55b35673cc00 msgr2=0x55b356769800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 shutdown_connections 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.697+0000 7f0e04f668c0 1 -- 172.21.15.66:0/282548703 wait complete. 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.701+0000 7f0e04f668c0 1 -- 172.21.15.66:0/923234378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35673d000 msgr2=0x55b3567c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.701+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/923234378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35673d000 0x55b3567c9080 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x55b3575a89c0 tx=0x55b357652120 comp rx=0 tx=0).stop 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.701+0000 7f0e04f668c0 1 -- 172.21.15.66:0/923234378 shutdown_connections 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.701+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/923234378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b35673d000 0x55b3567c9080 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.701+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/923234378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55b35673d400 0x55b3567c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.701+0000 7f0e04f668c0 1 --2- 172.21.15.66:0/923234378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35673d800 0x55b3567c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.701+0000 7f0e04f668c0 1 -- 172.21.15.66:0/923234378 >> 172.21.15.66:0/923234378 conn(0x55b35673dc00 msgr2=0x55b356769000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.701+0000 7f0e04f668c0 1 -- 172.21.15.66:0/923234378 shutdown_connections 2024-03-30T00:47:49.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.701+0000 7f0e04f668c0 1 -- 172.21.15.66:0/923234378 wait complete. 2024-03-30T00:47:49.715 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.150466, result: 2024-03-30T00:47:49.715 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-30T00:47:49.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.889+0000 7f5d4e800640 1 Processor -- start 2024-03-30T00:47:49.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.889+0000 7f5d4e800640 1 -- start start 2024-03-30T00:47:49.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d4e800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d4815ca80 0x7f5d4806b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d4e800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d480cb0e0 con 0x7f5d480cb570 2024-03-30T00:47:49.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d4e800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d480cf7f0 con 0x7f5d480cf2f0 2024-03-30T00:47:49.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d4e800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480cfad0 con 0x7f5d4815ca80 2024-03-30T00:47:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d47e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5d480cb570 0x7f5d480cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40500/0 (socket says 172.21.15.66:40500) 2024-03-30T00:47:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d47e00640 1 -- 172.21.15.66:0/4026532055 learned_addr learned my addr 172.21.15.66:0/4026532055 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d47400640 1 --2- 172.21.15.66:0/4026532055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d4815ca80 0x7f5d4806b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d46a00640 1 -- 172.21.15.66:0/4026532055 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795384414 0 0) 0x7f5d480cb0e0 con 0x7f5d480cb570 2024-03-30T00:47:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d46a00640 1 -- 172.21.15.66:0/4026532055 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d2c003530 con 0x7f5d480cb570 2024-03-30T00:47:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d46a00640 1 -- 172.21.15.66:0/4026532055 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 549599003 0 0) 0x7f5d480cf7f0 con 0x7f5d480cf2f0 2024-03-30T00:47:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d46a00640 1 -- 172.21.15.66:0/4026532055 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d480cb0e0 con 0x7f5d480cf2f0 2024-03-30T00:47:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d47400640 1 -- 172.21.15.66:0/4026532055 >> v1:172.21.15.156:6789/0 conn(0x7f5d480cf2f0 legacy=0x7f5d4806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d47400640 1 -- 172.21.15.66:0/4026532055 >> v1:172.21.15.66:6789/0 conn(0x7f5d480cb570 legacy=0x7f5d480cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.893+0000 7f5d47400640 1 -- 172.21.15.66:0/4026532055 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480cad70 con 0x7f5d4815ca80 2024-03-30T00:47:49.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d47400640 1 --2- 172.21.15.66:0/4026532055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d4815ca80 0x7f5d4806b0b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5d380097f0 tx=0x7f5d3801ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec0a9949a7ac2de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d46a00640 1 -- 172.21.15.66:0/4026532055 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d380a89d0 con 0x7f5d4815ca80 2024-03-30T00:47:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d46a00640 1 -- 172.21.15.66:0/4026532055 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d380a8b30 con 0x7f5d4815ca80 2024-03-30T00:47:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d46a00640 1 -- 172.21.15.66:0/4026532055 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d380a8ea0 con 0x7f5d4815ca80 2024-03-30T00:47:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d4e800640 1 -- 172.21.15.66:0/4026532055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d4815ca80 msgr2=0x7f5d4806b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d4e800640 1 --2- 172.21.15.66:0/4026532055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d4815ca80 0x7f5d4806b0b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5d380097f0 tx=0x7f5d3801ecd0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d4e800640 1 -- 172.21.15.66:0/4026532055 shutdown_connections 2024-03-30T00:47:49.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d4e800640 1 --2- 172.21.15.66:0/4026532055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d4815ca80 0x7f5d4806b0b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d4e800640 1 -- 172.21.15.66:0/4026532055 >> 172.21.15.66:0/4026532055 conn(0x7f5d480b2090 msgr2=0x7f5d480b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:49.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d4e800640 1 -- 172.21.15.66:0/4026532055 shutdown_connections 2024-03-30T00:47:49.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d4e800640 1 -- 172.21.15.66:0/4026532055 wait complete. 2024-03-30T00:47:49.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d4e800640 1 Processor -- start 2024-03-30T00:47:49.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.897+0000 7f5d4e800640 1 -- start start 2024-03-30T00:47:49.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d4e800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480cb570 0x7f5d48146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d4e800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d480cf2f0 0x7f5d480c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d4e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4815ca80 0x7f5d481565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d4e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48144f70 con 0x7f5d4815ca80 2024-03-30T00:47:49.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d4e800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48144360 con 0x7f5d480cb570 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d4e800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48144640 con 0x7f5d480cf2f0 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d47400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480cb570 0x7f5d48146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d47e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4815ca80 0x7f5d481565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d47400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480cb570 0x7f5d48146500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:56472/0 (socket says 172.21.15.66:56472) 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d47400640 1 -- 172.21.15.66:0/1555815325 learned_addr learned my addr 172.21.15.66:0/1555815325 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d3ea00640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d480cf2f0 0x7f5d480c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d47400640 1 -- 172.21.15.66:0/1555815325 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d480cf2f0 msgr2=0x7f5d480c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d47400640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d480cf2f0 0x7f5d480c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d47400640 1 -- 172.21.15.66:0/1555815325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4815ca80 msgr2=0x7f5d481565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d47400640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4815ca80 0x7f5d481565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d47400640 1 -- 172.21.15.66:0/1555815325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d38009420 con 0x7f5d480cb570 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d47400640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480cb570 0x7f5d48146500 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5d3801c900 tx=0x7f5d3801ba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d3fe00640 1 -- 172.21.15.66:0/1555815325 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d380bf070 con 0x7f5d480cb570 2024-03-30T00:47:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d3fe00640 1 -- 172.21.15.66:0/1555815325 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38006e20 con 0x7f5d480cb570 2024-03-30T00:47:49.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d3fe00640 1 -- 172.21.15.66:0/1555815325 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d380b0d20 con 0x7f5d480cb570 2024-03-30T00:47:49.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d4e800640 1 -- 172.21.15.66:0/1555815325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d48144920 con 0x7f5d480cb570 2024-03-30T00:47:49.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d4e800640 1 -- 172.21.15.66:0/1555815325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d480c7bb0 con 0x7f5d480cb570 2024-03-30T00:47:49.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d4e800640 1 -- 172.21.15.66:0/1555815325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d48066c30 con 0x7f5d480cb570 2024-03-30T00:47:49.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.901+0000 7f5d3fe00640 1 -- 172.21.15.66:0/1555815325 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5d380ba440 con 0x7f5d480cb570 2024-03-30T00:47:49.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.905+0000 7f5d3fe00640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d24066690 0x7f5d24068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:49.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.905+0000 7f5d3fe00640 1 -- 172.21.15.66:0/1555815325 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f5d38121ba0 con 0x7f5d480cb570 2024-03-30T00:47:49.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.905+0000 7f5d3ea00640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d24066690 0x7f5d24068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:49.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.905+0000 7f5d3ea00640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d24066690 0x7f5d24068b50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5d48146a40 tx=0x7f5d30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:49.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:49.909+0000 7f5d3fe00640 1 -- 172.21.15.66:0/1555815325 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5d380eecc0 con 0x7f5d480cb570 2024-03-30T00:47:50.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.105+0000 7f5d4e800640 1 -- 172.21.15.66:0/1555815325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f5d480b2bb0 con 0x7f5d480cb570 2024-03-30T00:47:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.269+0000 7f5d3fe00640 1 -- 172.21.15.66:0/1555815325 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v67) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5d380f22e0 con 0x7f5d480cb570 2024-03-30T00:47:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-30T00:47:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 -- 172.21.15.66:0/1555815325 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d24066690 msgr2=0x7f5d24068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d24066690 0x7f5d24068b50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5d48146a40 tx=0x7f5d30009040 comp rx=0 tx=0).stop 2024-03-30T00:47:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 -- 172.21.15.66:0/1555815325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480cb570 msgr2=0x7f5d48146500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480cb570 0x7f5d48146500 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5d3801c900 tx=0x7f5d3801ba70 comp rx=0 tx=0).stop 2024-03-30T00:47:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 -- 172.21.15.66:0/1555815325 shutdown_connections 2024-03-30T00:47:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d24066690 0x7f5d24068b50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d4815ca80 0x7f5d481565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d480cf2f0 0x7f5d480c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 --2- 172.21.15.66:0/1555815325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480cb570 0x7f5d48146500 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 -- 172.21.15.66:0/1555815325 >> 172.21.15.66:0/1555815325 conn(0x7f5d480b2090 msgr2=0x7f5d4806e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:50.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 -- 172.21.15.66:0/1555815325 shutdown_connections 2024-03-30T00:47:50.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.273+0000 7f5d4e800640 1 -- 172.21.15.66:0/1555815325 wait complete. 2024-03-30T00:47:50.305 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:47:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.433+0000 7f8ec7200640 1 Processor -- start 2024-03-30T00:47:50.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec7200640 1 -- start start 2024-03-30T00:47:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8ec815c730 0x7f8ec80cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec7200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ec80cfb40 0x7f8ec80c5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec7200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec80cffa0 con 0x7f8ec80c5d70 2024-03-30T00:47:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec80c4340 con 0x7f8ec815c730 2024-03-30T00:47:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec7200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec80c6200 con 0x7f8ec80cfb40 2024-03-30T00:47:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec5e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8ec815c730 0x7f8ec80cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec5e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8ec815c730 0x7f8ec80cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55072/0 (socket says 172.21.15.66:55072) 2024-03-30T00:47:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec5e00640 1 -- 172.21.15.66:0/273416163 learned_addr learned my addr 172.21.15.66:0/273416163 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ebfe00640 1 --2- 172.21.15.66:0/273416163 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ec80cfb40 0x7f8ec80c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec5e00640 1 -- 172.21.15.66:0/273416163 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ec80cfb40 msgr2=0x7f8ec80c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:50.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec5e00640 1 --2- 172.21.15.66:0/273416163 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ec80cfb40 0x7f8ec80c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec5e00640 1 -- 172.21.15.66:0/273416163 >> v1:172.21.15.156:6789/0 conn(0x7f8ec80c5d70 legacy=0x7f8ec8069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:50.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.437+0000 7f8ec5e00640 1 -- 172.21.15.66:0/273416163 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec80c53a0 con 0x7f8ec815c730 2024-03-30T00:47:50.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec5e00640 1 --2- 172.21.15.66:0/273416163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ec815c730 0x7f8ec80cf600 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009870 tx=0x7f8eb801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb34c7621f4cddc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:50.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec5400640 1 -- 172.21.15.66:0/273416163 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb80ac620 con 0x7f8ec815c730 2024-03-30T00:47:50.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec5400640 1 -- 172.21.15.66:0/273416163 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb80ac780 con 0x7f8ec815c730 2024-03-30T00:47:50.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec5400640 1 -- 172.21.15.66:0/273416163 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb80acb10 con 0x7f8ec815c730 2024-03-30T00:47:50.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec7200640 1 -- 172.21.15.66:0/273416163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ec815c730 msgr2=0x7f8ec80cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:50.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec7200640 1 --2- 172.21.15.66:0/273416163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ec815c730 0x7f8ec80cf600 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009870 tx=0x7f8eb801c4a0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec7200640 1 -- 172.21.15.66:0/273416163 shutdown_connections 2024-03-30T00:47:50.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec7200640 1 --2- 172.21.15.66:0/273416163 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ec80cfb40 0x7f8ec80c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec7200640 1 --2- 172.21.15.66:0/273416163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ec815c730 0x7f8ec80cf600 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec7200640 1 -- 172.21.15.66:0/273416163 >> 172.21.15.66:0/273416163 conn(0x7f8ec806b070 msgr2=0x7f8ec806b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:50.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec7200640 1 -- 172.21.15.66:0/273416163 shutdown_connections 2024-03-30T00:47:50.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec7200640 1 -- 172.21.15.66:0/273416163 wait complete. 2024-03-30T00:47:50.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec7200640 1 Processor -- start 2024-03-30T00:47:50.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.441+0000 7f8ec7200640 1 -- start start 2024-03-30T00:47:50.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec7200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec80c5d70 0x7f8ec81430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec7200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ec80cfb40 0x7f8ec815eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ec815c730 0x7f8ec80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec8152b10 con 0x7f8ec815c730 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec7200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec81521e0 con 0x7f8ec80c5d70 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec7200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec81524c0 con 0x7f8ec80cfb40 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ec815c730 0x7f8ec80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec5e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec80c5d70 0x7f8ec81430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec5e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec80c5d70 0x7f8ec81430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:56480/0 (socket says 172.21.15.66:56480) 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ec815c730 0x7f8ec80b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55086/0 (socket says 172.21.15.66:55086) 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec5e00640 1 -- 172.21.15.66:0/866107623 learned_addr learned my addr 172.21.15.66:0/866107623 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ebfe00640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ec80cfb40 0x7f8ec815eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec6800640 1 -- 172.21.15.66:0/866107623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ec80cfb40 msgr2=0x7f8ec815eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec6800640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ec80cfb40 0x7f8ec815eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec6800640 1 -- 172.21.15.66:0/866107623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec80c5d70 msgr2=0x7f8ec81430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:50.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec6800640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec80c5d70 0x7f8ec81430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec6800640 1 -- 172.21.15.66:0/866107623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb80094d0 con 0x7f8ec815c730 2024-03-30T00:47:50.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec6800640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ec815c730 0x7f8ec80b2090 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000c700 tx=0x7f8ec000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:50.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ebe000640 1 -- 172.21.15.66:0/866107623 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0003b80 con 0x7f8ec815c730 2024-03-30T00:47:50.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ebe000640 1 -- 172.21.15.66:0/866107623 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec0003ce0 con 0x7f8ec815c730 2024-03-30T00:47:50.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ebe000640 1 -- 172.21.15.66:0/866107623 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0005310 con 0x7f8ec815c730 2024-03-30T00:47:50.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec7200640 1 -- 172.21.15.66:0/866107623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec81527a0 con 0x7f8ec815c730 2024-03-30T00:47:50.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec7200640 1 -- 172.21.15.66:0/866107623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec8143610 con 0x7f8ec815c730 2024-03-30T00:47:50.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.445+0000 7f8ec7200640 1 -- 172.21.15.66:0/866107623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec8152cc0 con 0x7f8ec815c730 2024-03-30T00:47:50.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.449+0000 7f8ebe000640 1 -- 172.21.15.66:0/866107623 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8ec0005470 con 0x7f8ec815c730 2024-03-30T00:47:50.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.449+0000 7f8ebe000640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ea40666c0 0x7f8ea4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:50.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.449+0000 7f8ebe000640 1 -- 172.21.15.66:0/866107623 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f8ec0083ff0 con 0x7f8ec815c730 2024-03-30T00:47:50.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.449+0000 7f8ec5e00640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ea40666c0 0x7f8ea4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:50.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.449+0000 7f8ec5e00640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ea40666c0 0x7f8ea4068b80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009870 tx=0x7f8eb80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:50.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.453+0000 7f8ebe000640 1 -- 172.21.15.66:0/866107623 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8ec00b5980 con 0x7f8ec815c730 2024-03-30T00:47:50.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.645+0000 7f8ec7200640 1 -- 172.21.15.66:0/866107623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ec8073cf0 con 0x7f8ec815c730 2024-03-30T00:47:50.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ebe000640 1 -- 172.21.15.66:0/866107623 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f8ec8152cc0 con 0x7f8ec815c730 2024-03-30T00:47:50.657 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:47:50.657 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:47:50.275354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"a","rank":0,"incarnation":68,"state":"up:replay","state_seq":1,"addr":"172.21.15.66:6836/2419209461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2419209461},{"type":"v1","addr":"172.21.15.66:6836","nonce":2419209461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:47:50.657 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 68 2024-03-30T00:47:50.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 -- 172.21.15.66:0/866107623 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ea40666c0 msgr2=0x7f8ea4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:50.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ea40666c0 0x7f8ea4068b80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009870 tx=0x7f8eb80a7000 comp rx=0 tx=0).stop 2024-03-30T00:47:50.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 -- 172.21.15.66:0/866107623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ec815c730 msgr2=0x7f8ec80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:50.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ec815c730 0x7f8ec80b2090 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000c700 tx=0x7f8ec000cbc0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 -- 172.21.15.66:0/866107623 shutdown_connections 2024-03-30T00:47:50.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8ea40666c0 0x7f8ea4068b80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ec815c730 0x7f8ec80b2090 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ec80cfb40 0x7f8ec815eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 --2- 172.21.15.66:0/866107623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec80c5d70 0x7f8ec81430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:50.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 -- 172.21.15.66:0/866107623 >> 172.21.15.66:0/866107623 conn(0x7f8ec806b070 msgr2=0x7f8ec8153fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:50.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 -- 172.21.15.66:0/866107623 shutdown_connections 2024-03-30T00:47:50.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:50.649+0000 7f8ec7200640 1 -- 172.21.15.66:0/866107623 wait complete. 2024-03-30T00:47:51.080 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:46:15.271965+0000', 'modified': '2024-03-30T00:47:50.275354+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8259}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8259': {'gid': 8259, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.66:6836/2419209461', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 2419209461}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 2419209461}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:47:51.081 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8259:up:replay 2024-03-30T00:47:52.081 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:47:52.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f662ce00640 1 Processor -- start 2024-03-30T00:47:52.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f662ce00640 1 -- start start 2024-03-30T00:47:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f662ce00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f662806e800 0x7f66280d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f662ce00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f662806f8e0 0x7f66280ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f662ce00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662806b0b0 con 0x7f66280cc560 2024-03-30T00:47:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f662ce00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6628160640 con 0x7f662806f8e0 2024-03-30T00:47:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f662ce00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662806b390 con 0x7f662806e800 2024-03-30T00:47:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f6626800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f662806f8e0 0x7f66280ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f6626800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f662806f8e0 0x7f66280ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55100/0 (socket says 172.21.15.66:55100) 2024-03-30T00:47:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f6626800640 1 -- 172.21.15.66:0/1323377136 learned_addr learned my addr 172.21.15.66:0/1323377136 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f6625400640 1 --2- 172.21.15.66:0/1323377136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f662806e800 0x7f66280d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:52.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f6626800640 1 -- 172.21.15.66:0/1323377136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f662806e800 msgr2=0x7f66280d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:52.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f6626800640 1 --2- 172.21.15.66:0/1323377136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f662806e800 0x7f66280d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:52.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f6626800640 1 -- 172.21.15.66:0/1323377136 >> v1:172.21.15.156:6789/0 conn(0x7f66280cc560 legacy=0x7f662806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:52.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.225+0000 7f6626800640 1 -- 172.21.15.66:0/1323377136 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662806b670 con 0x7f662806f8e0 2024-03-30T00:47:52.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f6626800640 1 --2- 172.21.15.66:0/1323377136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f662806f8e0 0x7f66280ce8d0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6620009e50 tx=0x7f662001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650526b1564b7207 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:52.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f6624a00640 1 -- 172.21.15.66:0/1323377136 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66200a58d0 con 0x7f662806f8e0 2024-03-30T00:47:52.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f6624a00640 1 -- 172.21.15.66:0/1323377136 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66200a5a30 con 0x7f662806f8e0 2024-03-30T00:47:52.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f6624a00640 1 -- 172.21.15.66:0/1323377136 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66200a5dc0 con 0x7f662806f8e0 2024-03-30T00:47:52.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f662ce00640 1 -- 172.21.15.66:0/1323377136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f662806f8e0 msgr2=0x7f66280ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:52.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f662ce00640 1 --2- 172.21.15.66:0/1323377136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f662806f8e0 0x7f66280ce8d0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6620009e50 tx=0x7f662001ca00 comp rx=0 tx=0).stop 2024-03-30T00:47:52.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f662ce00640 1 -- 172.21.15.66:0/1323377136 shutdown_connections 2024-03-30T00:47:52.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f662ce00640 1 --2- 172.21.15.66:0/1323377136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f662806f8e0 0x7f66280ce8d0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:52.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f662ce00640 1 --2- 172.21.15.66:0/1323377136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f662806e800 0x7f66280d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:52.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f662ce00640 1 -- 172.21.15.66:0/1323377136 >> 172.21.15.66:0/1323377136 conn(0x7f66280d0540 msgr2=0x7f66280d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:52.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f662ce00640 1 -- 172.21.15.66:0/1323377136 shutdown_connections 2024-03-30T00:47:52.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f662ce00640 1 -- 172.21.15.66:0/1323377136 wait complete. 2024-03-30T00:47:52.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f662ce00640 1 Processor -- start 2024-03-30T00:47:52.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.229+0000 7f662ce00640 1 -- start start 2024-03-30T00:47:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f662ce00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f662806e800 0x7f66281500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f662ce00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f662806f8e0 0x7f66280b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f662ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66280cc560 0x7f6628158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f662ce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280c8500 con 0x7f66280cc560 2024-03-30T00:47:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f662ce00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628150630 con 0x7f662806f8e0 2024-03-30T00:47:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f662ce00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66280c7610 con 0x7f662806e800 2024-03-30T00:47:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6626800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66280cc560 0x7f6628158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6626800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66280cc560 0x7f6628158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55104/0 (socket says 172.21.15.66:55104) 2024-03-30T00:47:52.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6626800640 1 -- 172.21.15.66:0/3600829907 learned_addr learned my addr 172.21.15.66:0/3600829907 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6625e00640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f662806e800 0x7f66281500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6625400640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f662806f8e0 0x7f66280b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6626800640 1 -- 172.21.15.66:0/3600829907 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f662806e800 msgr2=0x7f66281500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6626800640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f662806e800 0x7f66281500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6626800640 1 -- 172.21.15.66:0/3600829907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f662806f8e0 msgr2=0x7f66280b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6626800640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f662806f8e0 0x7f66280b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6626800640 1 -- 172.21.15.66:0/3600829907 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620009b30 con 0x7f66280cc560 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6625400640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f662806f8e0 0x7f66280b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f6626800640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66280cc560 0x7f6628158230 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f66200af040 tx=0x7f66200197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f661e000640 1 -- 172.21.15.66:0/3600829907 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66200b8670 con 0x7f66280cc560 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f661e000640 1 -- 172.21.15.66:0/3600829907 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66200b87d0 con 0x7f66280cc560 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f661e000640 1 -- 172.21.15.66:0/3600829907 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66200b8b80 con 0x7f66280cc560 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f662ce00640 1 -- 172.21.15.66:0/3600829907 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66280c78f0 con 0x7f66280cc560 2024-03-30T00:47:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f662ce00640 1 -- 172.21.15.66:0/3600829907 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66280c7eb0 con 0x7f66280cc560 2024-03-30T00:47:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f662ce00640 1 -- 172.21.15.66:0/3600829907 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66280c8670 con 0x7f66280cc560 2024-03-30T00:47:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.233+0000 7f661e000640 1 -- 172.21.15.66:0/3600829907 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f66200b8ce0 con 0x7f66280cc560 2024-03-30T00:47:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.237+0000 7f661e000640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f66080666e0 0x7f6608068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.237+0000 7f661e000640 1 -- 172.21.15.66:0/3600829907 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f662011fae0 con 0x7f66280cc560 2024-03-30T00:47:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.237+0000 7f6625e00640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f66080666e0 0x7f6608068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:52.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.237+0000 7f6625e00640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f66080666e0 0x7f6608068ba0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6618000f70 tx=0x7f6618009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:52.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.241+0000 7f661e000640 1 -- 172.21.15.66:0/3600829907 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f66200ecc60 con 0x7f66280cc560 2024-03-30T00:47:52.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.437+0000 7f662ce00640 1 -- 172.21.15.66:0/3600829907 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6628066c30 con 0x7f66280cc560 2024-03-30T00:47:52.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.437+0000 7f661e000640 1 -- 172.21.15.66:0/3600829907 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f66200f0280 con 0x7f66280cc560 2024-03-30T00:47:52.449 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:47:52.449 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":70,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:47:52.278693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"a","rank":0,"incarnation":68,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.66:6836/2419209461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2419209461},{"type":"v1","addr":"172.21.15.66:6836","nonce":2419209461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:47:52.449 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 70 2024-03-30T00:47:52.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 -- 172.21.15.66:0/3600829907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f66080666e0 msgr2=0x7f6608068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:52.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f66080666e0 0x7f6608068ba0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6618000f70 tx=0x7f6618009040 comp rx=0 tx=0).stop 2024-03-30T00:47:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 -- 172.21.15.66:0/3600829907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66280cc560 msgr2=0x7f6628158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66280cc560 0x7f6628158230 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f66200af040 tx=0x7f66200197a0 comp rx=0 tx=0).stop 2024-03-30T00:47:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 -- 172.21.15.66:0/3600829907 shutdown_connections 2024-03-30T00:47:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f66080666e0 0x7f6608068ba0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66280cc560 0x7f6628158230 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f662806f8e0 0x7f66280b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 --2- 172.21.15.66:0/3600829907 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f662806e800 0x7f66281500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 -- 172.21.15.66:0/3600829907 >> 172.21.15.66:0/3600829907 conn(0x7f66280d0540 msgr2=0x7f662806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 -- 172.21.15.66:0/3600829907 shutdown_connections 2024-03-30T00:47:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:52.441+0000 7f662ce00640 1 -- 172.21.15.66:0/3600829907 wait complete. 2024-03-30T00:47:52.874 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:46:15.271965+0000', 'modified': '2024-03-30T00:47:52.278693+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8259}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8259': {'gid': 8259, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.66:6836/2419209461', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 2419209461}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 2419209461}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:47:52.874 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8259:up:rejoin 2024-03-30T00:47:53.875 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:47:54.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.017+0000 7f93f1600640 1 Processor -- start 2024-03-30T00:47:54.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.017+0000 7f93f1600640 1 -- start start 2024-03-30T00:47:54.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.017+0000 7f93f1600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f93ec15a640 0x7f93ec06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.017+0000 7f93f1600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93ec0ce520 0x7f93ec0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.017+0000 7f93f1600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f93ec0cabb0 0x7f93ec0cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.017+0000 7f93f1600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ec06d070 con 0x7f93ec0cabb0 2024-03-30T00:47:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.017+0000 7f93f1600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ec0cbba0 con 0x7f93ec15a640 2024-03-30T00:47:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.017+0000 7f93f1600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec06e2c0 con 0x7f93ec0ce520 2024-03-30T00:47:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93eb400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f93ec0cabb0 0x7f93ec0cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93eb400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f93ec0cabb0 0x7f93ec0cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55112/0 (socket says 172.21.15.66:55112) 2024-03-30T00:47:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93eb400640 1 -- 172.21.15.66:0/2122302946 learned_addr learned my addr 172.21.15.66:0/2122302946 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93eaa00640 1 --2- 172.21.15.66:0/2122302946 >> v2:172.21.15.156:3300/0 conn(0x7f93ec15a640 0x7f93ec06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93e3e00640 1 --2- 172.21.15.66:0/2122302946 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93ec0ce520 0x7f93ec0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93eb400640 1 -- 172.21.15.66:0/2122302946 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93ec0ce520 msgr2=0x7f93ec0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93eb400640 1 --2- 172.21.15.66:0/2122302946 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93ec0ce520 0x7f93ec0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93eb400640 1 -- 172.21.15.66:0/2122302946 >> v2:172.21.15.156:3300/0 conn(0x7f93ec15a640 msgr2=0x7f93ec06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93eb400640 1 --2- 172.21.15.66:0/2122302946 >> v2:172.21.15.156:3300/0 conn(0x7f93ec15a640 0x7f93ec06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93eb400640 1 -- 172.21.15.66:0/2122302946 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec0cee60 con 0x7f93ec0cabb0 2024-03-30T00:47:54.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93eb400640 1 --2- 172.21.15.66:0/2122302946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ec0cabb0 0x7f93ec0cb000 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0cf160 tx=0x7f93e401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c8bcee1ab6ba8b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:54.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93ea000640 1 -- 172.21.15.66:0/2122302946 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e40a5840 con 0x7f93ec0cabb0 2024-03-30T00:47:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93ea000640 1 -- 172.21.15.66:0/2122302946 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e40a59a0 con 0x7f93ec0cabb0 2024-03-30T00:47:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.021+0000 7f93ea000640 1 -- 172.21.15.66:0/2122302946 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e40a5d30 con 0x7f93ec0cabb0 2024-03-30T00:47:54.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 -- 172.21.15.66:0/2122302946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ec0cabb0 msgr2=0x7f93ec0cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 --2- 172.21.15.66:0/2122302946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ec0cabb0 0x7f93ec0cb000 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0cf160 tx=0x7f93e401c9c0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 -- 172.21.15.66:0/2122302946 shutdown_connections 2024-03-30T00:47:54.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 --2- 172.21.15.66:0/2122302946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ec0cabb0 0x7f93ec0cb000 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 --2- 172.21.15.66:0/2122302946 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93ec0ce520 0x7f93ec0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 --2- 172.21.15.66:0/2122302946 >> v2:172.21.15.156:3300/0 conn(0x7f93ec15a640 0x7f93ec06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 -- 172.21.15.66:0/2122302946 >> 172.21.15.66:0/2122302946 conn(0x7f93ec069f60 msgr2=0x7f93ec06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:54.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 -- 172.21.15.66:0/2122302946 shutdown_connections 2024-03-30T00:47:54.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 -- 172.21.15.66:0/2122302946 wait complete. 2024-03-30T00:47:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 Processor -- start 2024-03-30T00:47:54.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 -- start start 2024-03-30T00:47:54.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93ec0cabb0 0x7f93ec0971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93ec0ce520 0x7f93ec152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ec15a640 0x7f93ec153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec0cdb40 con 0x7f93ec15a640 2024-03-30T00:47:54.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec153a80 con 0x7f93ec0ce520 2024-03-30T00:47:54.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93f1600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec153d60 con 0x7f93ec0cabb0 2024-03-30T00:47:54.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93eb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ec15a640 0x7f93ec153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93eb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ec15a640 0x7f93ec153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55128/0 (socket says 172.21.15.66:55128) 2024-03-30T00:47:54.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93eb400640 1 -- 172.21.15.66:0/2235492339 learned_addr learned my addr 172.21.15.66:0/2235492339 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:54.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93eaa00640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93ec0cabb0 0x7f93ec0971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93e3e00640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93ec0ce520 0x7f93ec152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93eb400640 1 -- 172.21.15.66:0/2235492339 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93ec0cabb0 msgr2=0x7f93ec0971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93eb400640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93ec0cabb0 0x7f93ec0971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93eb400640 1 -- 172.21.15.66:0/2235492339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93ec0ce520 msgr2=0x7f93ec152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93eb400640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93ec0ce520 0x7f93ec152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93eb400640 1 -- 172.21.15.66:0/2235492339 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e4009b30 con 0x7f93ec15a640 2024-03-30T00:47:54.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.025+0000 7f93eb400640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ec15a640 0x7f93ec153120 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f93e40af040 tx=0x7f93e4019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:54.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.029+0000 7f93e2a00640 1 -- 172.21.15.66:0/2235492339 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e40b8670 con 0x7f93ec15a640 2024-03-30T00:47:54.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.029+0000 7f93e2a00640 1 -- 172.21.15.66:0/2235492339 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e40b8810 con 0x7f93ec15a640 2024-03-30T00:47:54.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.029+0000 7f93e2a00640 1 -- 172.21.15.66:0/2235492339 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e40b8bc0 con 0x7f93ec15a640 2024-03-30T00:47:54.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.029+0000 7f93f1600640 1 -- 172.21.15.66:0/2235492339 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec155200 con 0x7f93ec15a640 2024-03-30T00:47:54.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.029+0000 7f93f1600640 1 -- 172.21.15.66:0/2235492339 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec1557c0 con 0x7f93ec15a640 2024-03-30T00:47:54.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.029+0000 7f93f1600640 1 -- 172.21.15.66:0/2235492339 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ec1559b0 con 0x7f93ec15a640 2024-03-30T00:47:54.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.029+0000 7f93e2a00640 1 -- 172.21.15.66:0/2235492339 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f93e40b8d20 con 0x7f93ec15a640 2024-03-30T00:47:54.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.029+0000 7f93e2a00640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f93c8066690 0x7f93c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.029+0000 7f93e2a00640 1 -- 172.21.15.66:0/2235492339 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f93e411fb30 con 0x7f93ec15a640 2024-03-30T00:47:54.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.029+0000 7f93eaa00640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f93c8066690 0x7f93c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.033+0000 7f93eaa00640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f93c8066690 0x7f93c8068b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0af300 tx=0x7f93dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:54.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.033+0000 7f93e2a00640 1 -- 172.21.15.66:0/2235492339 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f93e40ecd80 con 0x7f93ec15a640 2024-03-30T00:47:54.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 -- 172.21.15.66:0/2235492339 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93ec066c30 con 0x7f93ec15a640 2024-03-30T00:47:54.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93e2a00640 1 -- 172.21.15.66:0/2235492339 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f93e40f03a0 con 0x7f93ec15a640 2024-03-30T00:47:54.239 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:47:54.239 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:47:53.284661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/2419209461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2419209461},{"type":"v1","addr":"172.21.15.66:6836","nonce":2419209461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:47:54.239 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 71 2024-03-30T00:47:54.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 -- 172.21.15.66:0/2235492339 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f93c8066690 msgr2=0x7f93c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f93c8066690 0x7f93c8068b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0af300 tx=0x7f93dc008040 comp rx=0 tx=0).stop 2024-03-30T00:47:54.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 -- 172.21.15.66:0/2235492339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ec15a640 msgr2=0x7f93ec153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ec15a640 0x7f93ec153120 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f93e40af040 tx=0x7f93e4019760 comp rx=0 tx=0).stop 2024-03-30T00:47:54.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 -- 172.21.15.66:0/2235492339 shutdown_connections 2024-03-30T00:47:54.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f93c8066690 0x7f93c8068b50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ec15a640 0x7f93ec153120 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93ec0ce520 0x7f93ec152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 --2- 172.21.15.66:0/2235492339 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93ec0cabb0 0x7f93ec0971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 -- 172.21.15.66:0/2235492339 >> 172.21.15.66:0/2235492339 conn(0x7f93ec069f60 msgr2=0x7f93ec0af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:54.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 -- 172.21.15.66:0/2235492339 shutdown_connections 2024-03-30T00:47:54.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.229+0000 7f93f1600640 1 -- 172.21.15.66:0/2235492339 wait complete. 2024-03-30T00:47:54.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:46:15.271965+0000', 'modified': '2024-03-30T00:47:53.284661+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8259}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8259': {'gid': 8259, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.66:6836/2419209461', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 2419209461}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 2419209461}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:47:54.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:47:54.663 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-30T00:47:54.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38ec00640 1 Processor -- start 2024-03-30T00:47:54.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38ec00640 1 -- start start 2024-03-30T00:47:54.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38ec00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe3880ce520 0x7fe3880ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38ec00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38806d070 con 0x7fe3880cabb0 2024-03-30T00:47:54.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38ec00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3880cbba0 con 0x7fe38815a640 2024-03-30T00:47:54.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38ec00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3880cee70 con 0x7fe3880ce520 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38cc00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe3880cabb0 0x7fe3880cafe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40504/0 (socket says 172.21.15.66:40504) 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38cc00640 1 -- 172.21.15.66:0/1499191338 learned_addr learned my addr 172.21.15.66:0/1499191338 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe387400640 1 --2- 172.21.15.66:0/1499191338 >> v2:172.21.15.156:3300/0 conn(0x7fe3880ce520 0x7fe3880ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414978573 0 0) 0x7fe38806d070 con 0x7fe3880cabb0 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe36c003530 con 0x7fe3880cabb0 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 195402941 0 0) 0x7fe36c003530 con 0x7fe3880cabb0 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 >> v2:172.21.15.156:3300/0 conn(0x7fe3880ce520 msgr2=0x7fe3880ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 --2- 172.21.15.66:0/1499191338 >> v2:172.21.15.156:3300/0 conn(0x7fe3880ce520 0x7fe3880ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 >> v1:172.21.15.156:6789/0 conn(0x7fe38815a640 legacy=0x7fe38806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3880cb8c0 con 0x7fe3880cabb0 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fe380001a00 con 0x7fe3880cabb0 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe387400640 1 --2- 172.21.15.66:0/1499191338 >> v2:172.21.15.156:3300/0 conn(0x7fe3880ce520 0x7fe3880ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:47:54.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 >> v1:172.21.15.66:6789/0 conn(0x7fe3880cabb0 legacy=0x7fe3880cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 --2- 172.21.15.66:0/1499191338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe36c003980 0x7fe36c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 --2- 172.21.15.66:0/1499191338 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0047d0 0x7fe36c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 --2- 172.21.15.66:0/1499191338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c005190 0x7fe36c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3880cee70 con 0x7fe36c005190 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c00c030 con 0x7fe36c003980 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c00c1f0 con 0x7fe36c0047d0 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38cc00640 1 --2- 172.21.15.66:0/1499191338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c005190 0x7fe36c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe387400640 1 --2- 172.21.15.66:0/1499191338 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0047d0 0x7fe36c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe387e00640 1 --2- 172.21.15.66:0/1499191338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe36c003980 0x7fe36c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38cc00640 1 -- 172.21.15.66:0/1499191338 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0047d0 msgr2=0x7fe36c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38cc00640 1 --2- 172.21.15.66:0/1499191338 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0047d0 0x7fe36c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38cc00640 1 -- 172.21.15.66:0/1499191338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe36c003980 msgr2=0x7fe36c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38cc00640 1 --2- 172.21.15.66:0/1499191338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe36c003980 0x7fe36c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.797+0000 7fe38cc00640 1 -- 172.21.15.66:0/1499191338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c003530 con 0x7fe36c005190 2024-03-30T00:47:54.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe38cc00640 1 -- 172.21.15.66:0/1499191338 reap_dead start 2024-03-30T00:47:54.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe38cc00640 1 --2- 172.21.15.66:0/1499191338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c005190 0x7fe36c00b990 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe38000a830 tx=0x7fe38001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2be034f26a1ca7e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:54.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3800072d0 con 0x7fe36c005190 2024-03-30T00:47:54.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3800a8070 con 0x7fe36c005190 2024-03-30T00:47:54.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe386a00640 1 -- 172.21.15.66:0/1499191338 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3800076b0 con 0x7fe36c005190 2024-03-30T00:47:54.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe38ec00640 1 -- 172.21.15.66:0/1499191338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c005190 msgr2=0x7fe36c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe38ec00640 1 --2- 172.21.15.66:0/1499191338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c005190 0x7fe36c00b990 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe38000a830 tx=0x7fe38001d360 comp rx=0 tx=0).stop 2024-03-30T00:47:54.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe38ec00640 1 -- 172.21.15.66:0/1499191338 shutdown_connections 2024-03-30T00:47:54.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe38ec00640 1 --2- 172.21.15.66:0/1499191338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe36c005190 0x7fe36c00b990 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe38ec00640 1 -- 172.21.15.66:0/1499191338 >> 172.21.15.66:0/1499191338 conn(0x7fe388069f60 msgr2=0x7fe38806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:54.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe38ec00640 1 -- 172.21.15.66:0/1499191338 shutdown_connections 2024-03-30T00:47:54.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.801+0000 7fe38ec00640 1 -- 172.21.15.66:0/1499191338 wait complete. 2024-03-30T00:47:54.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe38ec00640 1 Processor -- start 2024-03-30T00:47:54.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe38ec00640 1 -- start start 2024-03-30T00:47:54.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe38ec00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c005190 0x7fe38815aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe38ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3880adde0 0x7fe388156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe38ec00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3880c7a30 0x7fe3880c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe38ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38815d730 con 0x7fe3880adde0 2024-03-30T00:47:54.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe38ec00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3880c83f0 con 0x7fe3880c7a30 2024-03-30T00:47:54.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe38ec00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3880ae260 con 0x7fe36c005190 2024-03-30T00:47:54.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe387400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3880adde0 0x7fe388156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe387400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3880adde0 0x7fe388156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55146/0 (socket says 172.21.15.66:55146) 2024-03-30T00:47:54.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe387400640 1 -- 172.21.15.66:0/436940660 learned_addr learned my addr 172.21.15.66:0/436940660 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:54.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe38cc00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3880c7a30 0x7fe3880c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe387e00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c005190 0x7fe38815aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe387400640 1 -- 172.21.15.66:0/436940660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c005190 msgr2=0x7fe38815aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe387400640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c005190 0x7fe38815aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe387400640 1 -- 172.21.15.66:0/436940660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3880c7a30 msgr2=0x7fe3880c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe387400640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3880c7a30 0x7fe3880c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe387400640 1 -- 172.21.15.66:0/436940660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3881572b0 con 0x7fe3880adde0 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe387e00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c005190 0x7fe38815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe387400640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3880adde0 0x7fe388156930 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fe374011cd0 tx=0x7fe37400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe37fe00640 1 -- 172.21.15.66:0/436940660 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400fd20 con 0x7fe3880adde0 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe37fe00640 1 -- 172.21.15.66:0/436940660 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37400fe80 con 0x7fe3880adde0 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe37fe00640 1 -- 172.21.15.66:0/436940660 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374004240 con 0x7fe3880adde0 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe38ec00640 1 -- 172.21.15.66:0/436940660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3881579b0 con 0x7fe3880adde0 2024-03-30T00:47:54.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.805+0000 7fe38ec00640 1 -- 172.21.15.66:0/436940660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3880713a0 con 0x7fe3880adde0 2024-03-30T00:47:54.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.809+0000 7fe37fe00640 1 -- 172.21.15.66:0/436940660 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe3740043a0 con 0x7fe3880adde0 2024-03-30T00:47:54.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.809+0000 7fe37fe00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe368066690 0x7fe368068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.809+0000 7fe37fe00640 1 -- 172.21.15.66:0/436940660 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fe374084b00 con 0x7fe3880adde0 2024-03-30T00:47:54.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.809+0000 7fe387e00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe368066690 0x7fe368068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.809+0000 7fe392039000 1 Processor -- start 2024-03-30T00:47:54.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.809+0000 7fe392039000 1 -- start start 2024-03-30T00:47:54.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.809+0000 7fe387e00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe368066690 0x7fe368068b50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe3881f4c80 tx=0x7fe378009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:54.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a1105190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a11056d0 0x55d9a10baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a10bb4c0 0x55d9a10bb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a0f835a0 con 0x55d9a1104db0 2024-03-30T00:47:54.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a0f10af0 con 0x55d9a11056d0 2024-03-30T00:47:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9a0f0e0c0 con 0x55d9a10bb4c0 2024-03-30T00:47:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe387e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a1105190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe387e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a1105190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55152/0 (socket says 172.21.15.66:55152) 2024-03-30T00:47:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe387e00640 1 -- 172.21.15.66:0/467511354 learned_addr learned my addr 172.21.15.66:0/467511354 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe387400640 1 --2- 172.21.15.66:0/467511354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a11056d0 0x55d9a10baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe38cc00640 1 --2- 172.21.15.66:0/467511354 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a10bb4c0 0x55d9a10bb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe387e00640 1 -- 172.21.15.66:0/467511354 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a10bb4c0 msgr2=0x55d9a10bb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe387e00640 1 --2- 172.21.15.66:0/467511354 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a10bb4c0 0x55d9a10bb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe387e00640 1 -- 172.21.15.66:0/467511354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a11056d0 msgr2=0x55d9a10baf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe387e00640 1 --2- 172.21.15.66:0/467511354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a11056d0 0x55d9a10baf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe387e00640 1 -- 172.21.15.66:0/467511354 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9a0c05890 con 0x55d9a1104db0 2024-03-30T00:47:54.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe387400640 1 --2- 172.21.15.66:0/467511354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a11056d0 0x55d9a10baf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:54.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe387e00640 1 --2- 172.21.15.66:0/467511354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a1105190 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fe378001680 tx=0x7fe378013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b36ee25f9eb03c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:54.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe37cc00640 1 -- 172.21.15.66:0/467511354 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378014900 con 0x55d9a1104db0 2024-03-30T00:47:54.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe37cc00640 1 -- 172.21.15.66:0/467511354 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe378014a60 con 0x55d9a1104db0 2024-03-30T00:47:54.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe37cc00640 1 -- 172.21.15.66:0/467511354 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378014dd0 con 0x55d9a1104db0 2024-03-30T00:47:54.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 -- 172.21.15.66:0/467511354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 msgr2=0x55d9a1105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 --2- 172.21.15.66:0/467511354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a1105190 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fe378001680 tx=0x7fe378013000 comp rx=0 tx=0).stop 2024-03-30T00:47:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 -- 172.21.15.66:0/467511354 shutdown_connections 2024-03-30T00:47:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 --2- 172.21.15.66:0/467511354 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a10bb4c0 0x55d9a10bb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 --2- 172.21.15.66:0/467511354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a11056d0 0x55d9a10baf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 --2- 172.21.15.66:0/467511354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a1105190 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 -- 172.21.15.66:0/467511354 >> 172.21.15.66:0/467511354 conn(0x55d9a10c1830 msgr2=0x55d9a10b9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 -- 172.21.15.66:0/467511354 shutdown_connections 2024-03-30T00:47:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 -- 172.21.15.66:0/467511354 wait complete. 2024-03-30T00:47:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 -- start start 2024-03-30T00:47:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.813+0000 7fe392039000 1 Processor -- start 2024-03-30T00:47:54.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a10bb4c0 0x55d9a1104740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a10bf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a11056d0 0x55d9a10bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a0f099b0 con 0x55d9a1104db0 2024-03-30T00:47:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a0ea8c00 con 0x55d9a10bb4c0 2024-03-30T00:47:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9a0ec7070 con 0x55d9a11056d0 2024-03-30T00:47:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 10 client.0 _ll_register_callbacks cb 0x55d9a10a70e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a10bf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a10bb4c0 0x55d9a1104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a10bf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55160/0 (socket says 172.21.15.66:55160) 2024-03-30T00:47:54.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387400640 1 -- 172.21.15.66:0/3105555675 learned_addr learned my addr 172.21.15.66:0/3105555675 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe38cc00640 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a11056d0 0x55d9a10bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387400640 1 -- 172.21.15.66:0/3105555675 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a11056d0 msgr2=0x55d9a10bf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387400640 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a11056d0 0x55d9a10bf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387400640 1 -- 172.21.15.66:0/3105555675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a10bb4c0 msgr2=0x55d9a1104740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387400640 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a10bb4c0 0x55d9a1104740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387400640 1 -- 172.21.15.66:0/3105555675 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37800f040 con 0x55d9a1104db0 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387400640 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a10bf020 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fe3740555d0 tx=0x7fe374056110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc19549d55a1dc29 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe371600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe371600640 1 -- 172.21.15.66:0/3105555675 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37405dcc0 con 0x55d9a1104db0 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe371600640 1 -- 172.21.15.66:0/3105555675 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fe374054dc0 con 0x55d9a1104db0 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 -- 172.21.15.66:0/3105555675 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9a0ec65f0 con 0x55d9a1104db0 2024-03-30T00:47:54.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe371600640 1 -- 172.21.15.66:0/3105555675 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe374054f80 con 0x55d9a1104db0 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 10 client.8334 fetch_fsmap learned FSMap version 71 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 -- 172.21.15.66:0/3105555675 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55d9a0eceab0 con 0x55d9a1104db0 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe371600640 1 -- 172.21.15.66:0/3105555675 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d9a0ec65f0 con 0x55d9a1104db0 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe371600640 1 -- 172.21.15.66:0/3105555675 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe37401b030 con 0x55d9a1104db0 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 10 client.8334 fetch_fsmap finished waiting for FSMap version 71 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 10 client.8334 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 20 client.8334 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d9a10c02d0 0x55d9a11162e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 4 client.8334 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 -- 172.21.15.66:0/3105555675 --> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9a0f06790 con 0x55d9a10c02d0 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387e00640 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d9a10c02d0 0x55d9a11162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe387e00640 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d9a10c02d0 0x55d9a11162e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4552d9f7ef7e3765 server_cookie=842589f4e8745144 in_seq=0 out_seq=0 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe371600640 10 client.8334 ms_handle_connect on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe371600640 1 -- 172.21.15.66:0/3105555675 <== mds.0 v2:172.21.15.66:6835/2419209461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d9a0f06790 con 0x55d9a10c02d0 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe371600640 10 client.8334 handle_command_reply: tid=0 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 client.8334 shutdown 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 -- 172.21.15.66:0/3105555675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 msgr2=0x55d9a10bf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a10bf020 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fe3740555d0 tx=0x7fe374056110 comp rx=0 tx=0).stop 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 -- 172.21.15.66:0/3105555675 shutdown_connections 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d9a10c02d0 0x55d9a11162e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a11056d0 0x55d9a10bf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a1104db0 0x55d9a10bf020 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 --2- 172.21.15.66:0/3105555675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a10bb4c0 0x55d9a1104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 -- 172.21.15.66:0/3105555675 >> 172.21.15.66:0/3105555675 conn(0x55d9a10c1830 msgr2=0x55d9a10b9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe371600640 0 client.8334 ms_handle_reset on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.817+0000 7fe392039000 1 -- 172.21.15.66:0/3105555675 shutdown_connections 2024-03-30T00:47:54.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.821+0000 7fe392039000 1 -- 172.21.15.66:0/3105555675 wait complete. 2024-03-30T00:47:54.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.821+0000 7fe392039000 20 client.8334 trim_cache size 0 max 16384 2024-03-30T00:47:54.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.833+0000 7fe392039000 1 Processor -- start 2024-03-30T00:47:54.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.833+0000 7fe392039000 1 -- start start 2024-03-30T00:47:54.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a10bb4c0 0x55d9a1103720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a1104db0 0x55d9a1103c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a11056d0 0x55d9a11152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a0bc6940 con 0x55d9a11056d0 2024-03-30T00:47:54.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a10febd0 con 0x55d9a10bb4c0 2024-03-30T00:47:54.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9a0ea6c10 con 0x55d9a1104db0 2024-03-30T00:47:54.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe38cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a11056d0 0x55d9a11152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe38cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a11056d0 0x55d9a11152e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55164/0 (socket says 172.21.15.66:55164) 2024-03-30T00:47:54.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe38cc00640 1 -- 172.21.15.66:0/3918114821 learned_addr learned my addr 172.21.15.66:0/3918114821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:54.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe387e00640 1 --2- 172.21.15.66:0/3918114821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a10bb4c0 0x55d9a1103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe387400640 1 --2- 172.21.15.66:0/3918114821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a1104db0 0x55d9a1103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe38cc00640 1 -- 172.21.15.66:0/3918114821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a1104db0 msgr2=0x55d9a1103c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe38cc00640 1 --2- 172.21.15.66:0/3918114821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a1104db0 0x55d9a1103c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe38cc00640 1 -- 172.21.15.66:0/3918114821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a10bb4c0 msgr2=0x55d9a1103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe38cc00640 1 --2- 172.21.15.66:0/3918114821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a10bb4c0 0x55d9a1103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe38cc00640 1 -- 172.21.15.66:0/3918114821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9a0eb4c00 con 0x55d9a11056d0 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe387e00640 1 --2- 172.21.15.66:0/3918114821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a10bb4c0 0x55d9a1103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe38cc00640 1 --2- 172.21.15.66:0/3918114821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a11056d0 0x55d9a11152e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe380000f90 tx=0x7fe380001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d5cb0864b373e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe37d600640 1 -- 172.21.15.66:0/3918114821 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380001be0 con 0x55d9a11056d0 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe37d600640 1 -- 172.21.15.66:0/3918114821 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3800017c0 con 0x55d9a11056d0 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe37d600640 1 -- 172.21.15.66:0/3918114821 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380004b70 con 0x55d9a11056d0 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- 172.21.15.66:0/3918114821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a11056d0 msgr2=0x55d9a11152e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 --2- 172.21.15.66:0/3918114821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a11056d0 0x55d9a11152e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe380000f90 tx=0x7fe380001d90 comp rx=0 tx=0).stop 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- 172.21.15.66:0/3918114821 shutdown_connections 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 --2- 172.21.15.66:0/3918114821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a11056d0 0x55d9a11152e0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 --2- 172.21.15.66:0/3918114821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a1104db0 0x55d9a1103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 --2- 172.21.15.66:0/3918114821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a10bb4c0 0x55d9a1103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- 172.21.15.66:0/3918114821 >> 172.21.15.66:0/3918114821 conn(0x55d9a10c1830 msgr2=0x55d9a1103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- 172.21.15.66:0/3918114821 shutdown_connections 2024-03-30T00:47:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- 172.21.15.66:0/3918114821 wait complete. 2024-03-30T00:47:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- start start 2024-03-30T00:47:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 Processor -- start 2024-03-30T00:47:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a10bb4c0 0x55d9a1162450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a1104db0 0x55d9a1118590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a11056d0 0x55d9a1118ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a0f071c0 con 0x55d9a10bb4c0 2024-03-30T00:47:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a10b42e0 con 0x55d9a11056d0 2024-03-30T00:47:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.837+0000 7fe392039000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9a10ff900 con 0x55d9a1104db0 2024-03-30T00:47:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 10 client.0 _ll_register_callbacks cb 0x55d9a0ece590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a10bb4c0 0x55d9a1162450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a10bb4c0 0x55d9a1162450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55174/0 (socket says 172.21.15.66:55174) 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387e00640 1 -- 172.21.15.66:0/1351099621 learned_addr learned my addr 172.21.15.66:0/1351099621 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387400640 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a1104db0 0x55d9a1118590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe38cc00640 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a11056d0 0x55d9a1118ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387e00640 1 -- 172.21.15.66:0/1351099621 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a1104db0 msgr2=0x55d9a1118590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387e00640 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a1104db0 0x55d9a1118590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387e00640 1 -- 172.21.15.66:0/1351099621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a11056d0 msgr2=0x55d9a1118ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387e00640 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a11056d0 0x55d9a1118ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387e00640 1 -- 172.21.15.66:0/1351099621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3800b4040 con 0x55d9a10bb4c0 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387e00640 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a10bb4c0 0x55d9a1162450 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe378025040 tx=0x7fe378001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598fc620f5f38b01 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe371600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe371600640 1 -- 172.21.15.66:0/1351099621 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378014640 con 0x55d9a10bb4c0 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe371600640 1 -- 172.21.15.66:0/1351099621 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fe37801c050 con 0x55d9a10bb4c0 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 -- 172.21.15.66:0/1351099621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9a0ed9460 con 0x55d9a10bb4c0 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe371600640 1 -- 172.21.15.66:0/1351099621 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe378014980 con 0x55d9a10bb4c0 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 10 client.8340 fetch_fsmap learned FSMap version 71 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 -- 172.21.15.66:0/1351099621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55d9a0c05cc0 con 0x55d9a10bb4c0 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe371600640 1 -- 172.21.15.66:0/1351099621 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d9a0ed9460 con 0x55d9a10bb4c0 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe371600640 1 -- 172.21.15.66:0/1351099621 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe37803b020 con 0x55d9a10bb4c0 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 10 client.8340 fetch_fsmap finished waiting for FSMap version 71 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 10 client.8340 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 20 client.8340 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d9a1119010 0x55d9a11193f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 4 client.8340 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:47:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 -- 172.21.15.66:0/1351099621 --> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d9a0ed29c0 con 0x55d9a1119010 2024-03-30T00:47:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387400640 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d9a1119010 0x55d9a11193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe387400640 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d9a1119010 0x55d9a11193f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d4425c2352a6cc server_cookie=40a65ace508ef275 in_seq=0 out_seq=0 2024-03-30T00:47:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe371600640 10 client.8340 ms_handle_connect on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe371600640 1 -- 172.21.15.66:0/1351099621 <== mds.0 v2:172.21.15.66:6835/2419209461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d9a0ed29c0 con 0x55d9a1119010 2024-03-30T00:47:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe371600640 10 client.8340 handle_command_reply: tid=0 2024-03-30T00:47:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 client.8340 shutdown 2024-03-30T00:47:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 -- 172.21.15.66:0/1351099621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a10bb4c0 msgr2=0x55d9a1162450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a10bb4c0 0x55d9a1162450 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe378025040 tx=0x7fe378001130 comp rx=0 tx=0).stop 2024-03-30T00:47:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 -- 172.21.15.66:0/1351099621 shutdown_connections 2024-03-30T00:47:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9a11056d0 0x55d9a1118ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d9a1119010 0x55d9a11193f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d9a1104db0 0x55d9a1118590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 --2- 172.21.15.66:0/1351099621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d9a10bb4c0 0x55d9a1162450 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe371600640 0 client.8340 ms_handle_reset on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:54.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 -- 172.21.15.66:0/1351099621 >> 172.21.15.66:0/1351099621 conn(0x55d9a10c1830 msgr2=0x55d9a1103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:54.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 -- 172.21.15.66:0/1351099621 shutdown_connections 2024-03-30T00:47:54.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 1 -- 172.21.15.66:0/1351099621 wait complete. 2024-03-30T00:47:54.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe392039000 20 client.8340 trim_cache size 0 max 16384 2024-03-30T00:47:54.857 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:47:54.858 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:47:54.858 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:47:54.858 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 8259, 2024-03-30T00:47:54.858 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:47:54.858 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:47:54.858 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:47:54.858 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 4.5657921369999999, 2024-03-30T00:47:54.858 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 71, 2024-03-30T00:47:54.858 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 98, 2024-03-30T00:47:54.858 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 98, 2024-03-30T00:47:54.858 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 5.5120758030000001 2024-03-30T00:47:54.859 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:47:54.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe38ec00640 1 -- 172.21.15.66:0/436940660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe368066690 msgr2=0x7fe368068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe38ec00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe368066690 0x7fe368068b50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe3881f4c80 tx=0x7fe378009040 comp rx=0 tx=0).stop 2024-03-30T00:47:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe38ec00640 1 -- 172.21.15.66:0/436940660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3880adde0 msgr2=0x7fe388156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe38ec00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3880adde0 0x7fe388156930 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fe374011cd0 tx=0x7fe37400f610 comp rx=0 tx=0).stop 2024-03-30T00:47:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe38ec00640 1 -- 172.21.15.66:0/436940660 shutdown_connections 2024-03-30T00:47:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe38ec00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3880c7a30 0x7fe3880c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe38ec00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe368066690 0x7fe368068b50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe38ec00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3880adde0 0x7fe388156930 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe38ec00640 1 --2- 172.21.15.66:0/436940660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c005190 0x7fe38815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.841+0000 7fe38ec00640 1 -- 172.21.15.66:0/436940660 >> 172.21.15.66:0/436940660 conn(0x7fe388069f60 msgr2=0x7fe3880af230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.845+0000 7fe38ec00640 1 -- 172.21.15.66:0/436940660 shutdown_connections 2024-03-30T00:47:54.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:54.845+0000 7fe38ec00640 1 -- 172.21.15.66:0/436940660 wait complete. 2024-03-30T00:47:54.872 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:47:55.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.001+0000 7fe7b3000640 1 Processor -- start 2024-03-30T00:47:55.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.001+0000 7fe7b3000640 1 -- start start 2024-03-30T00:47:55.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7b3000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe7ac0cec30 0x7fe7ac159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:55.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7b3000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7ac0cde70 0x7fe7ac15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:55.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7b3000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ac15a6e0 con 0x7fe7ac0cda90 2024-03-30T00:47:55.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7b3000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ac0b1b80 con 0x7fe7ac0cec30 2024-03-30T00:47:55.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7b3000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0cd3e0 con 0x7fe7ac0cde70 2024-03-30T00:47:55.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7a3400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe7ac0cec30 0x7fe7ac159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:55.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7b1000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7ac0cde70 0x7fe7ac15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:55.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7a3400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe7ac0cec30 0x7fe7ac159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:56564/0 (socket says 172.21.15.66:56564) 2024-03-30T00:47:55.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7a3400640 1 -- 172.21.15.66:0/3112928539 learned_addr learned my addr 172.21.15.66:0/3112928539 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:55.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7a3400640 1 -- 172.21.15.66:0/3112928539 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7ac0cde70 msgr2=0x7fe7ac15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:55.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7a3400640 1 --2- 172.21.15.66:0/3112928539 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7ac0cde70 0x7fe7ac15a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:55.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7a3400640 1 -- 172.21.15.66:0/3112928539 >> v1:172.21.15.156:6789/0 conn(0x7fe7ac0cda90 legacy=0x7fe7ac0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:55.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.005+0000 7fe7a3400640 1 -- 172.21.15.66:0/3112928539 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac15a990 con 0x7fe7ac0cec30 2024-03-30T00:47:55.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7a3400640 1 --2- 172.21.15.66:0/3112928539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac0cec30 0x7fe7ac159c60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe794009870 tx=0x7fe79401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3078ccfd53d1aecc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:55.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7ab400640 1 -- 172.21.15.66:0/3112928539 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940aa430 con 0x7fe7ac0cec30 2024-03-30T00:47:55.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7ab400640 1 -- 172.21.15.66:0/3112928539 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7940aa590 con 0x7fe7ac0cec30 2024-03-30T00:47:55.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7ab400640 1 -- 172.21.15.66:0/3112928539 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940aa920 con 0x7fe7ac0cec30 2024-03-30T00:47:55.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 -- 172.21.15.66:0/3112928539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac0cec30 msgr2=0x7fe7ac159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:55.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 --2- 172.21.15.66:0/3112928539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac0cec30 0x7fe7ac159c60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe794009870 tx=0x7fe79401c4d0 comp rx=0 tx=0).stop 2024-03-30T00:47:55.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 -- 172.21.15.66:0/3112928539 shutdown_connections 2024-03-30T00:47:55.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 --2- 172.21.15.66:0/3112928539 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7ac0cde70 0x7fe7ac15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:55.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 --2- 172.21.15.66:0/3112928539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac0cec30 0x7fe7ac159c60 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:55.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 -- 172.21.15.66:0/3112928539 >> 172.21.15.66:0/3112928539 conn(0x7fe7ac069f60 msgr2=0x7fe7ac06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:55.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 -- 172.21.15.66:0/3112928539 shutdown_connections 2024-03-30T00:47:55.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 -- 172.21.15.66:0/3112928539 wait complete. 2024-03-30T00:47:55.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 Processor -- start 2024-03-30T00:47:55.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 -- start start 2024-03-30T00:47:55.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7ac0cda90 0x7fe7ac096d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:55.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7ac0cde70 0x7fe7ac0ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:55.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.009+0000 7fe7b3000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac0cec30 0x7fe7ac0aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:55.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7b3000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0ab480 con 0x7fe7ac0cde70 2024-03-30T00:47:55.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7b3000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0ab760 con 0x7fe7ac0cec30 2024-03-30T00:47:55.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7b3000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0aba40 con 0x7fe7ac0cda90 2024-03-30T00:47:55.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7ac0cde70 0x7fe7ac0ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:55.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7ac0cde70 0x7fe7ac0ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55182/0 (socket says 172.21.15.66:55182) 2024-03-30T00:47:55.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a3400640 1 -- 172.21.15.66:0/2512552464 learned_addr learned my addr 172.21.15.66:0/2512552464 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:55.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7b1000640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac0cec30 0x7fe7ac0aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:55.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7abe00640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7ac0cda90 0x7fe7ac096d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:55.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a3400640 1 -- 172.21.15.66:0/2512552464 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7ac0cda90 msgr2=0x7fe7ac096d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:55.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a3400640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7ac0cda90 0x7fe7ac096d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:55.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a3400640 1 -- 172.21.15.66:0/2512552464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac0cec30 msgr2=0x7fe7ac0aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:55.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a3400640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac0cec30 0x7fe7ac0aca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:55.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a3400640 1 -- 172.21.15.66:0/2512552464 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7940094d0 con 0x7fe7ac0cde70 2024-03-30T00:47:55.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a3400640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7ac0cde70 0x7fe7ac0ac520 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe79401a100 tx=0x7fe79401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:55.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a8c00640 1 -- 172.21.15.66:0/2512552464 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940aa430 con 0x7fe7ac0cde70 2024-03-30T00:47:55.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a8c00640 1 -- 172.21.15.66:0/2512552464 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7940b3800 con 0x7fe7ac0cde70 2024-03-30T00:47:55.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a8c00640 1 -- 172.21.15.66:0/2512552464 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940b3bb0 con 0x7fe7ac0cde70 2024-03-30T00:47:55.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7b3000640 1 -- 172.21.15.66:0/2512552464 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ac0abd20 con 0x7fe7ac0cde70 2024-03-30T00:47:55.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7b3000640 1 -- 172.21.15.66:0/2512552464 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac0975a0 con 0x7fe7ac0cde70 2024-03-30T00:47:55.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7b3000640 1 -- 172.21.15.66:0/2512552464 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ac066c30 con 0x7fe7ac0cde70 2024-03-30T00:47:55.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.013+0000 7fe7a8c00640 1 -- 172.21.15.66:0/2512552464 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe7940b3d10 con 0x7fe7ac0cde70 2024-03-30T00:47:55.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.017+0000 7fe7a8c00640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe788066690 0x7fe788068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:55.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.017+0000 7fe7a8c00640 1 -- 172.21.15.66:0/2512552464 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fe79411ebf0 con 0x7fe7ac0cde70 2024-03-30T00:47:55.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.017+0000 7fe7abe00640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe788066690 0x7fe788068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:55.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.017+0000 7fe7abe00640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe788066690 0x7fe788068b50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac15b740 tx=0x7fe79c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:55.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.017+0000 7fe7a8c00640 1 -- 172.21.15.66:0/2512552464 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe7940ebec0 con 0x7fe7ac0cde70 2024-03-30T00:47:55.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.213+0000 7fe7b3000640 1 -- 172.21.15.66:0/2512552464 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe7ac0ac1d0 con 0x7fe7ac0cde70 2024-03-30T00:47:55.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.213+0000 7fe7a8c00640 1 -- 172.21.15.66:0/2512552464 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe7940ef4e0 con 0x7fe7ac0cde70 2024-03-30T00:47:55.224 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:47:55.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 -- 172.21.15.66:0/2512552464 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe788066690 msgr2=0x7fe788068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:55.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe788066690 0x7fe788068b50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac15b740 tx=0x7fe79c009040 comp rx=0 tx=0).stop 2024-03-30T00:47:55.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 -- 172.21.15.66:0/2512552464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7ac0cde70 msgr2=0x7fe7ac0ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:55.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7ac0cde70 0x7fe7ac0ac520 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe79401a100 tx=0x7fe79401ad10 comp rx=0 tx=0).stop 2024-03-30T00:47:55.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 -- 172.21.15.66:0/2512552464 shutdown_connections 2024-03-30T00:47:55.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac0cec30 0x7fe7ac0aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:55.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe788066690 0x7fe788068b50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:55.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7ac0cde70 0x7fe7ac0ac520 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:55.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 --2- 172.21.15.66:0/2512552464 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7ac0cda90 0x7fe7ac096d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:55.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 -- 172.21.15.66:0/2512552464 >> 172.21.15.66:0/2512552464 conn(0x7fe7ac069f60 msgr2=0x7fe7ac06c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:55.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 -- 172.21.15.66:0/2512552464 shutdown_connections 2024-03-30T00:47:55.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:55.217+0000 7fe7b3000640 1 -- 172.21.15.66:0/2512552464 wait complete. 2024-03-30T00:47:55.245 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:47:55.245 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:47:55.245 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:47:55.245 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:47:55.245 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:47:55.246 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:47:55.246 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:47:55.246 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:47:55.246 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:47:55.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:55.246 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:47:55.256 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:47:55.256 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:47:55.256 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:47:55.256 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:47:55.256 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:47:55.256 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:47:55.256 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:47:55.256 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:47:55.256 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42003sec preferred_lft 42003sec 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:47:55.257 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:47:55.258 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:47:55.259 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:47:55.259 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:47:55.259 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:47:55.259 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:47:55.259 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:47:55.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:55.259 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:47:55.288 INFO:tasks.ceph.mds.a.smithi066.stderr:2024-03-30T00:47:55.277+0000 7f9400400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:47:55.306 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:47:55.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:55.309 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:47:55.366 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:47:55.366 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:47:55.366 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:47:55.366 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:47:55.366 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:47:55.366 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:47:55.367 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:47:55.367 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:47:55.367 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:47:55.367 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:47:55.367 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:47:55.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:55.367 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:47:55.367 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:47:55.367 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:47:55.368 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:47:55.368 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:47:55.368 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:47:55.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:55.460 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:47:55.469 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:47:55.469 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:47:55.469 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:47:55.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:55.469 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:47:55.532 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:47:55.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:47:55.532 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:47:55.580 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:47:55.581 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:47:55.581 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:47:55.633 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:47:55.697 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:47:55.697 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:47:55.759 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:47:55.761 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:47:55.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:55.761 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:47:55.816 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:47:55.817 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:47:55.863 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:47:55.884 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:47:55.884 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:47:55.943 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:47:55.945 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:47:55.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:55.946 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:47:55.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:47:55.949+0000 7f5f28614580 -1 init, newargv = 0x55586815bc80 newargc=15 2024-03-30T00:47:55.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[49348]: starting ceph client 2024-03-30T00:47:55.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[49348]: starting fuse 2024-03-30T00:47:55.967 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:47:55.968 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:47:55.968 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:47:55.968 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:47:55.968 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:47:55.968 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:47:55.968 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:47:55.968 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:47:55.968 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:47:55.968 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:47:55.969 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:47:55.970 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:47:55.970 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:47:55.970 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:47:55.970 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:47:56.093 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.49348.asok 2024-03-30T00:47:56.277 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49348.asok 2024-03-30T00:47:56.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:56.277 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49348.asok status 2024-03-30T00:47:56.457 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:47:56.457 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:47:56.457 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:47:56.457 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:47:56.457 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:47:56.457 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:47:56.457 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:47:56.457 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "49348", 2024-03-30T00:47:56.457 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 8349, 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 8349 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:47:56.458 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2503755311 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2503755311 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.8349 192.168.0.1:0/2503755311", 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/2503755311", 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:47:56.459 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 71, 2024-03-30T00:47:56.460 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 98, 2024-03-30T00:47:56.460 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:47:56.460 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:47:56.460 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:47:56.460 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:47:56.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:56.472 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:47:56.472 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:47:56.481 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:47:56.481 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:47:56.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:47:56.481 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:47:56.544 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-03-30T00:47:56.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.717+0000 7f36c3800640 1 Processor -- start 2024-03-30T00:47:56.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.717+0000 7f36c3800640 1 -- start start 2024-03-30T00:47:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c3800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36bc15cab0 0x7f36bc06ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c3800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f36bc06b2e0 0x7f36bc06ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c3800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f36bc06e300 0x7f36bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c3800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36bc06b740 con 0x7f36bc06e300 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c3800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36bc06d930 con 0x7f36bc06b2e0 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c3800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc15ce90 con 0x7f36bc15cab0 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c1800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f36bc06e300 0x7f36bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c1800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f36bc06e300 0x7f36bc06e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55194/0 (socket says 172.21.15.66:55194) 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c1800640 1 -- 172.21.15.66:0/1460581398 learned_addr learned my addr 172.21.15.66:0/1460581398 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c0e00640 1 --2- 172.21.15.66:0/1460581398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36bc15cab0 0x7f36bc06ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36b3e00640 1 --2- 172.21.15.66:0/1460581398 >> v2:172.21.15.156:3300/0 conn(0x7f36bc06b2e0 0x7f36bc06ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c1800640 1 -- 172.21.15.66:0/1460581398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36bc15cab0 msgr2=0x7f36bc06ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c1800640 1 --2- 172.21.15.66:0/1460581398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36bc15cab0 0x7f36bc06ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c1800640 1 -- 172.21.15.66:0/1460581398 >> v2:172.21.15.156:3300/0 conn(0x7f36bc06b2e0 msgr2=0x7f36bc06ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c1800640 1 --2- 172.21.15.66:0/1460581398 >> v2:172.21.15.156:3300/0 conn(0x7f36bc06b2e0 0x7f36bc06ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c1800640 1 -- 172.21.15.66:0/1460581398 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc0caf10 con 0x7f36bc06e300 2024-03-30T00:47:56.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.721+0000 7f36c0e00640 1 --2- 172.21.15.66:0/1460581398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36bc15cab0 0x7f36bc06ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:56.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c1800640 1 --2- 172.21.15.66:0/1460581398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc06e300 0x7f36bc06e770 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0705c0 tx=0x7f36b401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ea5277f1f4470 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36bbe00640 1 -- 172.21.15.66:0/1460581398 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b40a5750 con 0x7f36bc06e300 2024-03-30T00:47:56.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36bbe00640 1 -- 172.21.15.66:0/1460581398 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b40a58b0 con 0x7f36bc06e300 2024-03-30T00:47:56.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36bbe00640 1 -- 172.21.15.66:0/1460581398 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b40a5c40 con 0x7f36bc06e300 2024-03-30T00:47:56.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c3800640 1 -- 172.21.15.66:0/1460581398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc06e300 msgr2=0x7f36bc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c3800640 1 --2- 172.21.15.66:0/1460581398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc06e300 0x7f36bc06e770 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0705c0 tx=0x7f36b401c9c0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c3800640 1 -- 172.21.15.66:0/1460581398 shutdown_connections 2024-03-30T00:47:56.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c3800640 1 --2- 172.21.15.66:0/1460581398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc06e300 0x7f36bc06e770 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c3800640 1 --2- 172.21.15.66:0/1460581398 >> v2:172.21.15.156:3300/0 conn(0x7f36bc06b2e0 0x7f36bc06ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c3800640 1 --2- 172.21.15.66:0/1460581398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36bc15cab0 0x7f36bc06ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c3800640 1 -- 172.21.15.66:0/1460581398 >> 172.21.15.66:0/1460581398 conn(0x7f36bc0708d0 msgr2=0x7f36bc070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c3800640 1 -- 172.21.15.66:0/1460581398 shutdown_connections 2024-03-30T00:47:56.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c3800640 1 -- 172.21.15.66:0/1460581398 wait complete. 2024-03-30T00:47:56.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c3800640 1 Processor -- start 2024-03-30T00:47:56.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.725+0000 7f36c3800640 1 -- start start 2024-03-30T00:47:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36c3800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc06b2e0 0x7f36bc15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36c3800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc06e300 0x7f36bc15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36c3800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36bc15cab0 0x7f36bc0c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36c3800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc14c360 con 0x7f36bc06e300 2024-03-30T00:47:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36c3800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc15bbc0 con 0x7f36bc06b2e0 2024-03-30T00:47:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36c3800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc15b600 con 0x7f36bc15cab0 2024-03-30T00:47:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc06e300 0x7f36bc15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc06e300 0x7f36bc15afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55202/0 (socket says 172.21.15.66:55202) 2024-03-30T00:47:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b3e00640 1 -- 172.21.15.66:0/3352320960 learned_addr learned my addr 172.21.15.66:0/3352320960 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36c0e00640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc06b2e0 0x7f36bc15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36c1800640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36bc15cab0 0x7f36bc0c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b3e00640 1 -- 172.21.15.66:0/3352320960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36bc15cab0 msgr2=0x7f36bc0c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b3e00640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36bc15cab0 0x7f36bc0c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b3e00640 1 -- 172.21.15.66:0/3352320960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc06b2e0 msgr2=0x7f36bc15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b3e00640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc06b2e0 0x7f36bc15aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b3e00640 1 -- 172.21.15.66:0/3352320960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b4009b30 con 0x7f36bc06e300 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36c0e00640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc06b2e0 0x7f36bc15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b3e00640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc06e300 0x7f36bc15afa0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f36a400d800 tx=0x7f36a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b9600640 1 -- 172.21.15.66:0/3352320960 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a40096a0 con 0x7f36bc06e300 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b9600640 1 -- 172.21.15.66:0/3352320960 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a4009800 con 0x7f36bc06e300 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36c3800640 1 -- 172.21.15.66:0/3352320960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36bc15b8e0 con 0x7f36bc06e300 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36b9600640 1 -- 172.21.15.66:0/3352320960 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4009b70 con 0x7f36bc06e300 2024-03-30T00:47:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.729+0000 7f36c3800640 1 -- 172.21.15.66:0/3352320960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc1573c0 con 0x7f36bc06e300 2024-03-30T00:47:56.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36b9600640 1 -- 172.21.15.66:0/3352320960 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f36a4009cd0 con 0x7f36bc06e300 2024-03-30T00:47:56.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36b9600640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3698066690 0x7f3698068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36b9600640 1 -- 172.21.15.66:0/3352320960 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f36a4084b00 con 0x7f36bc06e300 2024-03-30T00:47:56.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c0e00640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3698066690 0x7f3698068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c6c09000 1 Processor -- start 2024-03-30T00:47:56.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c6c09000 1 -- start start 2024-03-30T00:47:56.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c0e00640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3698066690 0x7f3698068b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0079d0 tx=0x7f36ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0bfd30 0x55ddaf0c0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0c0650 0x55ddaf076a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf077340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c6c09000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddaef3e170 con 0x55ddaf0bfd30 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c6c09000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddaeecbd80 con 0x55ddaf076f60 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c6c09000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddaeec9350 con 0x55ddaf0c0650 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0bfd30 0x55ddaf0c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0bfd30 0x55ddaf0c0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55214/0 (socket says 172.21.15.66:55214) 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c0e00640 1 -- 172.21.15.66:0/3157565646 learned_addr learned my addr 172.21.15.66:0/3157565646 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c1800640 1 --2- 172.21.15.66:0/3157565646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf077340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36b3e00640 1 --2- 172.21.15.66:0/3157565646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0c0650 0x55ddaf076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c0e00640 1 -- 172.21.15.66:0/3157565646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0c0650 msgr2=0x55ddaf076a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c0e00640 1 --2- 172.21.15.66:0/3157565646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0c0650 0x55ddaf076a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c0e00640 1 -- 172.21.15.66:0/3157565646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 msgr2=0x55ddaf077340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c0e00640 1 --2- 172.21.15.66:0/3157565646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf077340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c0e00640 1 -- 172.21.15.66:0/3157565646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddaebc1070 con 0x55ddaf0bfd30 2024-03-30T00:47:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36b3e00640 1 --2- 172.21.15.66:0/3157565646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0c0650 0x55ddaf076a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36c0e00640 1 --2- 172.21.15.66:0/3157565646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0bfd30 0x55ddaf0c0110 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00fa90 tx=0x7f36ac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aed795cedc161cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36b1600640 1 -- 172.21.15.66:0/3157565646 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac014be0 con 0x55ddaf0bfd30 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36b1600640 1 -- 172.21.15.66:0/3157565646 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac019070 con 0x55ddaf0bfd30 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.733+0000 7f36b1600640 1 -- 172.21.15.66:0/3157565646 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac020440 con 0x55ddaf0bfd30 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 -- 172.21.15.66:0/3157565646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0bfd30 msgr2=0x55ddaf0c0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 --2- 172.21.15.66:0/3157565646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0bfd30 0x55ddaf0c0110 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00fa90 tx=0x7f36ac012000 comp rx=0 tx=0).stop 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 -- 172.21.15.66:0/3157565646 shutdown_connections 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 --2- 172.21.15.66:0/3157565646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf077340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 --2- 172.21.15.66:0/3157565646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0c0650 0x55ddaf076a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 --2- 172.21.15.66:0/3157565646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0bfd30 0x55ddaf0c0110 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 -- 172.21.15.66:0/3157565646 >> 172.21.15.66:0/3157565646 conn(0x55ddaf072f10 msgr2=0x55ddaf075300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 -- 172.21.15.66:0/3157565646 shutdown_connections 2024-03-30T00:47:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 -- 172.21.15.66:0/3157565646 wait complete. 2024-03-30T00:47:56.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 -- start start 2024-03-30T00:47:56.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 Processor -- start 2024-03-30T00:47:56.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf07c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0acc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddaeec4c40 con 0x55ddaf0c0650 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddaee63fa0 con 0x55ddaf076f60 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddaee821c0 con 0x55ddaf0bfd30 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 10 client.0 _ll_register_callbacks cb 0x55ddaf062060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55216/0 (socket says 172.21.15.66:55216) 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c1800640 1 -- 172.21.15.66:0/543218137 learned_addr learned my addr 172.21.15.66:0/543218137 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c0e00640 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf07c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36b3e00640 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c1800640 1 -- 172.21.15.66:0/543218137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 msgr2=0x55ddaf0acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c1800640 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c1800640 1 -- 172.21.15.66:0/543218137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 msgr2=0x55ddaf07c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c1800640 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf07c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c1800640 1 -- 172.21.15.66:0/543218137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac00e040 con 0x55ddaf0c0650 2024-03-30T00:47:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c1800640 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0ad1a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f36b401d260 tx=0x7f36b401b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a483235ea9071cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36aa000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36aa000640 1 -- 172.21.15.66:0/543218137 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b40a5e10 con 0x55ddaf0c0650 2024-03-30T00:47:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36aa000640 1 -- 172.21.15.66:0/543218137 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f36b40c1d00 con 0x55ddaf0c0650 2024-03-30T00:47:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.737+0000 7f36c6c09000 1 -- 172.21.15.66:0/543218137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddaeec1a20 con 0x55ddaf0c0650 2024-03-30T00:47:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36aa000640 1 -- 172.21.15.66:0/543218137 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36b40c1ec0 con 0x55ddaf0c0650 2024-03-30T00:47:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 10 client.8361 fetch_fsmap learned FSMap version 71 2024-03-30T00:47:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 -- 172.21.15.66:0/543218137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55ddaee89ba0 con 0x55ddaf0c0650 2024-03-30T00:47:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36aa000640 1 -- 172.21.15.66:0/543218137 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddaeec1a20 con 0x55ddaf0c0650 2024-03-30T00:47:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36aa000640 1 -- 172.21.15.66:0/543218137 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f36b40b63f0 con 0x55ddaf0c0650 2024-03-30T00:47:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 10 client.8361 fetch_fsmap finished waiting for FSMap version 71 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 10 client.8361 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 20 client.8361 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55ddaf0c1470 0x55ddaf0d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 4 client.8361 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 -- 172.21.15.66:0/543218137 --> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddaeec3590 con 0x55ddaf0c1470 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c0e00640 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55ddaf0c1470 0x55ddaf0d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c0e00640 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55ddaf0c1470 0x55ddaf0d1ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a488d209130a0790 server_cookie=f556603a9e398e33 in_seq=0 out_seq=0 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36aa000640 10 client.8361 ms_handle_connect on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36aa000640 1 -- 172.21.15.66:0/543218137 <== mds.0 v2:172.21.15.66:6835/2419209461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ddaeec3590 con 0x55ddaf0c1470 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36aa000640 10 client.8361 handle_command_reply: tid=0 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 client.8361 shutdown 2024-03-30T00:47:56.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 -- 172.21.15.66:0/543218137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 msgr2=0x55ddaf0ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0ad1a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f36b401d260 tx=0x7f36b401b200 comp rx=0 tx=0).stop 2024-03-30T00:47:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 -- 172.21.15.66:0/543218137 shutdown_connections 2024-03-30T00:47:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0ad1a0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55ddaf0c1470 0x55ddaf0d1ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 --2- 172.21.15.66:0/543218137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf07c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36aa000640 0 client.8361 ms_handle_reset on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 -- 172.21.15.66:0/543218137 >> 172.21.15.66:0/543218137 conn(0x55ddaf072f10 msgr2=0x55ddaf075b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 -- 172.21.15.66:0/543218137 shutdown_connections 2024-03-30T00:47:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 1 -- 172.21.15.66:0/543218137 wait complete. 2024-03-30T00:47:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.741+0000 7f36c6c09000 20 client.8361 trim_cache size 0 max 16384 2024-03-30T00:47:56.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 Processor -- start 2024-03-30T00:47:56.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 -- start start 2024-03-30T00:47:56.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf076f60 0x55ddaf0be720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0bec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf0c0650 0x55ddaf0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddaeb81cf0 con 0x55ddaf076f60 2024-03-30T00:47:56.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddaeec0430 con 0x55ddaf0c0650 2024-03-30T00:47:56.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddaeec77a0 con 0x55ddaf0bfd30 2024-03-30T00:47:56.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf076f60 0x55ddaf0be720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf076f60 0x55ddaf0be720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55224/0 (socket says 172.21.15.66:55224) 2024-03-30T00:47:56.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c0e00640 1 -- 172.21.15.66:0/1791007576 learned_addr learned my addr 172.21.15.66:0/1791007576 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:56.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c0e00640 1 -- 172.21.15.66:0/1791007576 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 msgr2=0x55ddaf0bec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c0e00640 1 --2- 172.21.15.66:0/1791007576 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0bec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c0e00640 1 -- 172.21.15.66:0/1791007576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf0c0650 msgr2=0x55ddaf0cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c1800640 1 --2- 172.21.15.66:0/1791007576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf0c0650 0x55ddaf0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c0e00640 1 --2- 172.21.15.66:0/1791007576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf0c0650 0x55ddaf0cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c0e00640 1 -- 172.21.15.66:0/1791007576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddaee6fef0 con 0x55ddaf076f60 2024-03-30T00:47:56.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c1800640 1 --2- 172.21.15.66:0/1791007576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf0c0650 0x55ddaf0cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:47:56.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c0e00640 1 --2- 172.21.15.66:0/1791007576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf076f60 0x55ddaf0be720 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00fa60 tx=0x7f36ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a17e9baa786589a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36b2000640 1 -- 172.21.15.66:0/1791007576 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac02f030 con 0x55ddaf076f60 2024-03-30T00:47:56.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36b2000640 1 -- 172.21.15.66:0/1791007576 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac030040 con 0x55ddaf076f60 2024-03-30T00:47:56.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36b2000640 1 -- 172.21.15.66:0/1791007576 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac033730 con 0x55ddaf076f60 2024-03-30T00:47:56.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 -- 172.21.15.66:0/1791007576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf076f60 msgr2=0x55ddaf0be720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 --2- 172.21.15.66:0/1791007576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf076f60 0x55ddaf0be720 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00fa60 tx=0x7f36ac014000 comp rx=0 tx=0).stop 2024-03-30T00:47:56.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 -- 172.21.15.66:0/1791007576 shutdown_connections 2024-03-30T00:47:56.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 --2- 172.21.15.66:0/1791007576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf0c0650 0x55ddaf0cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 --2- 172.21.15.66:0/1791007576 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0bec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 --2- 172.21.15.66:0/1791007576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf076f60 0x55ddaf0be720 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 -- 172.21.15.66:0/1791007576 >> 172.21.15.66:0/1791007576 conn(0x55ddaf072f10 msgr2=0x55ddaf076210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 -- 172.21.15.66:0/1791007576 shutdown_connections 2024-03-30T00:47:56.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.757+0000 7f36c6c09000 1 -- 172.21.15.66:0/1791007576 wait complete. 2024-03-30T00:47:56.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 -- start start 2024-03-30T00:47:56.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 Processor -- start 2024-03-30T00:47:56.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf0d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0db3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0db8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddaee4c150 con 0x55ddaf0c0650 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddaee68c10 con 0x55ddaf076f60 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddaf0d6980 con 0x55ddaf0bfd30 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 10 client.0 _ll_register_callbacks cb 0x55ddaef3e0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0db8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0db8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55232/0 (socket says 172.21.15.66:55232) 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c1800640 1 -- 172.21.15.66:0/4259734668 learned_addr learned my addr 172.21.15.66:0/4259734668 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c0e00640 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf0d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36b3e00640 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0db3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c1800640 1 -- 172.21.15.66:0/4259734668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 msgr2=0x55ddaf0db3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c1800640 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0db3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c1800640 1 -- 172.21.15.66:0/4259734668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 msgr2=0x55ddaf0d3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c1800640 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf0d3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c1800640 1 -- 172.21.15.66:0/4259734668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddaebc1070 con 0x55ddaf0c0650 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c0e00640 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf0d3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36b3e00640 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0db3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c1800640 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0db8f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f36b401b240 tx=0x7f36b4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b0fb7e53dded3d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36aa000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36aa000640 1 -- 172.21.15.66:0/4259734668 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b40b6950 con 0x55ddaf0c0650 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36aa000640 1 -- 172.21.15.66:0/4259734668 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f36b40a5c30 con 0x55ddaf0c0650 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 -- 172.21.15.66:0/4259734668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddaee815f0 con 0x55ddaf0c0650 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36aa000640 1 -- 172.21.15.66:0/4259734668 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36b40a5df0 con 0x55ddaf0c0650 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 10 client.8367 fetch_fsmap learned FSMap version 71 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 -- 172.21.15.66:0/4259734668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55ddaee63a60 con 0x55ddaf0c0650 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36aa000640 1 -- 172.21.15.66:0/4259734668 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddaee815f0 con 0x55ddaf0c0650 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36aa000640 1 -- 172.21.15.66:0/4259734668 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f36b40d0020 con 0x55ddaf0c0650 2024-03-30T00:47:56.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 10 client.8367 fetch_fsmap finished waiting for FSMap version 71 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 10 client.8367 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 20 client.8367 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55ddaf0dbe30 0x55ddaf11f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 4 client.8367 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c6c09000 1 -- 172.21.15.66:0/4259734668 --> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55ddaee83410 con 0x55ddaf0dbe30 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.761+0000 7f36c0e00640 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55ddaf0dbe30 0x55ddaf11f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36aa000640 10 client.8367 ms_handle_connect on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c0e00640 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55ddaf0dbe30 0x55ddaf11f3d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e7ae206b12f4bf1 server_cookie=86036b367cdc3643 in_seq=0 out_seq=0 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36aa000640 1 -- 172.21.15.66:0/4259734668 <== mds.0 v2:172.21.15.66:6835/2419209461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ddaee83410 con 0x55ddaf0dbe30 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36aa000640 10 client.8367 handle_command_reply: tid=0 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 1 client.8367 shutdown 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 1 -- 172.21.15.66:0/4259734668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 msgr2=0x55ddaf0db8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0db8f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f36b401b240 tx=0x7f36b4009810 comp rx=0 tx=0).stop 2024-03-30T00:47:56.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 1 -- 172.21.15.66:0/4259734668 shutdown_connections 2024-03-30T00:47:56.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddaf0c0650 0x55ddaf0db8f0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55ddaf0dbe30 0x55ddaf11f3d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ddaf0bfd30 0x55ddaf0db3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 1 --2- 172.21.15.66:0/4259734668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ddaf076f60 0x55ddaf0d3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36aa000640 0 client.8367 ms_handle_reset on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:56.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 1 -- 172.21.15.66:0/4259734668 >> 172.21.15.66:0/4259734668 conn(0x55ddaf072f10 msgr2=0x55ddaf0bd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 1 -- 172.21.15.66:0/4259734668 shutdown_connections 2024-03-30T00:47:56.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 1 -- 172.21.15.66:0/4259734668 wait complete. 2024-03-30T00:47:56.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c6c09000 20 client.8367 trim_cache size 0 max 16384 2024-03-30T00:47:56.780 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:47:56.781 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T00:47:56.781 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "08b1ec23-9bce-4806-982e-95e55fbea3ef", 2024-03-30T00:47:56.781 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T00:47:56.781 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:47:56.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 -- 172.21.15.66:0/3352320960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3698066690 msgr2=0x7f3698068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3698066690 0x7f3698068b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0079d0 tx=0x7f36ac008040 comp rx=0 tx=0).stop 2024-03-30T00:47:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 -- 172.21.15.66:0/3352320960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc06e300 msgr2=0x7f36bc15afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc06e300 0x7f36bc15afa0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f36a400d800 tx=0x7f36a400dcc0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 -- 172.21.15.66:0/3352320960 shutdown_connections 2024-03-30T00:47:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36bc15cab0 0x7f36bc0c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3698066690 0x7f3698068b50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc06e300 0x7f36bc15afa0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 --2- 172.21.15.66:0/3352320960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc06b2e0 0x7f36bc15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 -- 172.21.15.66:0/3352320960 >> 172.21.15.66:0/3352320960 conn(0x7f36bc0708d0 msgr2=0x7f36bc0cd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 -- 172.21.15.66:0/3352320960 shutdown_connections 2024-03-30T00:47:56.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f36c3800640 1 -- 172.21.15.66:0/3352320960 wait complete. 2024-03-30T00:47:56.783 INFO:tasks.ceph.mds.a.smithi066.stderr:2024-03-30T00:47:56.765+0000 7f93fbe00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56258bef6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:47:56.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-30T00:47:56.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.925+0000 7fcbe7e00640 1 Processor -- start 2024-03-30T00:47:56.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.925+0000 7fcbe7e00640 1 -- start start 2024-03-30T00:47:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbe80b0500 0x7fcbe80b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcbe815b780 0x7fcbe806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe7e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe80cd000 con 0x7fcbe815a7d0 2024-03-30T00:47:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe7e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe80b0e20 con 0x7fcbe815b780 2024-03-30T00:47:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe7e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe80ccd20 con 0x7fcbe80b0500 2024-03-30T00:47:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcbe815b780 0x7fcbe806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbdfe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fcbe815a7d0 0x7fcbe80cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40520/0 (socket says 172.21.15.66:40520) 2024-03-30T00:47:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbdfe00640 1 -- 172.21.15.66:0/3310638240 learned_addr learned my addr 172.21.15.66:0/3310638240 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:47:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcbe815b780 0x7fcbe806c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55236/0 (socket says 172.21.15.66:55236) 2024-03-30T00:47:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/3310638240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbe80b0500 0x7fcbe80b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe7400640 1 -- 172.21.15.66:0/3310638240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbe80b0500 msgr2=0x7fcbe80b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe7400640 1 --2- 172.21.15.66:0/3310638240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbe80b0500 0x7fcbe80b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe7400640 1 -- 172.21.15.66:0/3310638240 >> v1:172.21.15.66:6789/0 conn(0x7fcbe815a7d0 legacy=0x7fcbe80cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe7400640 1 -- 172.21.15.66:0/3310638240 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe815aca0 con 0x7fcbe815b780 2024-03-30T00:47:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.929+0000 7fcbe6000640 1 -- 172.21.15.66:0/3310638240 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670732628 0 0) 0x7fcbe80cd000 con 0x7fcbe815a7d0 2024-03-30T00:47:56.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7400640 1 --2- 172.21.15.66:0/3310638240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe815b780 0x7fcbe806c3b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009d90 tx=0x7fcbe001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f11adbb070492d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe6000640 1 -- 172.21.15.66:0/3310638240 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00a58e0 con 0x7fcbe815b780 2024-03-30T00:47:56.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe6000640 1 -- 172.21.15.66:0/3310638240 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe00a5a40 con 0x7fcbe815b780 2024-03-30T00:47:56.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe6000640 1 -- 172.21.15.66:0/3310638240 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00a5dd0 con 0x7fcbe815b780 2024-03-30T00:47:56.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7e00640 1 -- 172.21.15.66:0/3310638240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe815b780 msgr2=0x7fcbe806c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7e00640 1 --2- 172.21.15.66:0/3310638240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe815b780 0x7fcbe806c3b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009d90 tx=0x7fcbe001c820 comp rx=0 tx=0).stop 2024-03-30T00:47:56.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7e00640 1 -- 172.21.15.66:0/3310638240 shutdown_connections 2024-03-30T00:47:56.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7e00640 1 --2- 172.21.15.66:0/3310638240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe815b780 0x7fcbe806c3b0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7e00640 1 --2- 172.21.15.66:0/3310638240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbe80b0500 0x7fcbe80b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7e00640 1 -- 172.21.15.66:0/3310638240 >> 172.21.15.66:0/3310638240 conn(0x7fcbe8065cd0 msgr2=0x7fcbe8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7e00640 1 -- 172.21.15.66:0/3310638240 shutdown_connections 2024-03-30T00:47:56.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7e00640 1 -- 172.21.15.66:0/3310638240 wait complete. 2024-03-30T00:47:56.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7e00640 1 Processor -- start 2024-03-30T00:47:56.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7e00640 1 -- start start 2024-03-30T00:47:56.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.933+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbe80b0500 0x7fcbe813ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe815a7d0 0x7fcbe8070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbe815b780 0x7fcbe80707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbe7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe8152b70 con 0x7fcbe815a7d0 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbe7e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe8152e50 con 0x7fcbe815b780 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbe7e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe8153130 con 0x7fcbe80b0500 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe815a7d0 0x7fcbe8070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe815a7d0 0x7fcbe8070280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55248/0 (socket says 172.21.15.66:55248) 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2051610642 learned_addr learned my addr 172.21.15.66:0/2051610642 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbe80b0500 0x7fcbe813ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbe7400640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbe815b780 0x7fcbe80707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2051610642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbe80b0500 msgr2=0x7fcbe813ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbe80b0500 0x7fcbe813ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2051610642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbe815b780 msgr2=0x7fcbe80707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbe815b780 0x7fcbe80707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2051610642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe0009a70 con 0x7fcbe815a7d0 2024-03-30T00:47:56.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe815a7d0 0x7fcbe8070280 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000c4b0 tx=0x7fcbd000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdea00640 1 -- 172.21.15.66:0/2051610642 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd00074e0 con 0x7fcbe815a7d0 2024-03-30T00:47:56.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdea00640 1 -- 172.21.15.66:0/2051610642 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd0007660 con 0x7fcbe815a7d0 2024-03-30T00:47:56.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdea00640 1 -- 172.21.15.66:0/2051610642 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0004a30 con 0x7fcbe815a7d0 2024-03-30T00:47:56.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbe7e00640 1 -- 172.21.15.66:0/2051610642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe8153410 con 0x7fcbe815a7d0 2024-03-30T00:47:56.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbe7e00640 1 -- 172.21.15.66:0/2051610642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe8158cb0 con 0x7fcbe815a7d0 2024-03-30T00:47:56.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.937+0000 7fcbdea00640 1 -- 172.21.15.66:0/2051610642 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcbd0004b90 con 0x7fcbe815a7d0 2024-03-30T00:47:56.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbdea00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbc4066690 0x7fcbc4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbdea00640 1 -- 172.21.15.66:0/2051610642 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fcbd0083650 con 0x7fcbe815a7d0 2024-03-30T00:47:56.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbc4066690 0x7fcbc4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 Processor -- start 2024-03-30T00:47:56.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 -- start start 2024-03-30T00:47:56.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbc4066690 0x7fcbc4068b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8165400 tx=0x7fcbd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f55e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f56390 0x55d879f094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f09a00 0x55d879f09de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d879dd4630 con 0x55d879f55a70 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d879d61bd0 con 0x55d879f56390 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d879d5f1a0 con 0x55d879f09a00 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f55e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f55e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55256/0 (socket says 172.21.15.66:55256) 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe6a00640 1 -- 172.21.15.66:0/4208209187 learned_addr learned my addr 172.21.15.66:0/4208209187 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/4208209187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f56390 0x55d879f094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe7400640 1 --2- 172.21.15.66:0/4208209187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f09a00 0x55d879f09de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe6a00640 1 -- 172.21.15.66:0/4208209187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f09a00 msgr2=0x55d879f09de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/4208209187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f09a00 0x55d879f09de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe6a00640 1 -- 172.21.15.66:0/4208209187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f56390 msgr2=0x55d879f094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/4208209187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f56390 0x55d879f094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe6a00640 1 -- 172.21.15.66:0/4208209187 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d879a56e20 con 0x55d879f55a70 2024-03-30T00:47:56.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/4208209187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f56390 0x55d879f094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:56.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/4208209187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f55e50 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8007fd0 tx=0x7fcbd8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3100feb32c9fee4e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbd7400640 1 -- 172.21.15.66:0/4208209187 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8016cd0 con 0x55d879f55a70 2024-03-30T00:47:56.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbd7400640 1 -- 172.21.15.66:0/4208209187 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd801a070 con 0x55d879f55a70 2024-03-30T00:47:56.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbd7400640 1 -- 172.21.15.66:0/4208209187 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8021640 con 0x55d879f55a70 2024-03-30T00:47:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 -- 172.21.15.66:0/4208209187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 msgr2=0x55d879f55e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/4208209187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f55e50 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8007fd0 tx=0x7fcbd8013000 comp rx=0 tx=0).stop 2024-03-30T00:47:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 -- 172.21.15.66:0/4208209187 shutdown_connections 2024-03-30T00:47:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/4208209187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f09a00 0x55d879f09de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/4208209187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f56390 0x55d879f094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/4208209187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f55e50 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 -- 172.21.15.66:0/4208209187 >> 172.21.15.66:0/4208209187 conn(0x55d879d0a160 msgr2=0x55d879f38820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 -- 172.21.15.66:0/4208209187 shutdown_connections 2024-03-30T00:47:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.941+0000 7fcbf0c95000 1 -- 172.21.15.66:0/4208209187 wait complete. 2024-03-30T00:47:56.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 -- start start 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 Processor -- start 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f09a00 0x55d879f127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f39a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f56390 0x55d879f39f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d879d5aa90 con 0x55d879f55a70 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d879cf9d40 con 0x55d879f09a00 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d879d17f30 con 0x55d879f56390 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 10 client.0 _ll_register_callbacks cb 0x55d879dbc810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f39a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f39a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55260/0 (socket says 172.21.15.66:55260) 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2830376112 learned_addr learned my addr 172.21.15.66:0/2830376112 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbe7400640 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f56390 0x55d879f39f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f09a00 0x55d879f127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2830376112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f56390 msgr2=0x55d879f39f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f56390 0x55d879f39f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2830376112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f09a00 msgr2=0x55d879f127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f09a00 0x55d879f127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2830376112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd800e040 con 0x55d879f55a70 2024-03-30T00:47:56.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f09a00 0x55d879f127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:47:56.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f39a50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0000fc0 tx=0x7fcbd00544f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a03de77601bf6a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbcbe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:56.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2830376112 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd005c5a0 con 0x55d879f55a70 2024-03-30T00:47:56.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2830376112 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fcbd005a070 con 0x55d879f55a70 2024-03-30T00:47:56.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2830376112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d879d593e0 con 0x55d879f55a70 2024-03-30T00:47:56.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2830376112 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbd005a230 con 0x55d879f55a70 2024-03-30T00:47:56.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 10 client.8382 fetch_fsmap learned FSMap version 71 2024-03-30T00:47:56.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2830376112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55d879d1faa0 con 0x55d879f55a70 2024-03-30T00:47:56.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2830376112 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd006f080 con 0x55d879f55a70 2024-03-30T00:47:56.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2830376112 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcbd005a390 con 0x55d879f55a70 2024-03-30T00:47:56.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 10 client.8382 fetch_fsmap finished waiting for FSMap version 71 2024-03-30T00:47:56.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 10 client.8382 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:56.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 20 client.8382 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:56.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d879f5ce00 0x55d879f5d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 4 client.8382 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:47:56.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2830376112 --> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d879d2ad00 con 0x55d879f5ce00 2024-03-30T00:47:56.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.945+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d879f5ce00 0x55d879f5d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d879f5ce00 0x55d879f5d1e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cabe73261005d38 server_cookie=30a9ba6feb77e997 in_seq=0 out_seq=0 2024-03-30T00:47:56.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbcbe00640 10 client.8382 ms_handle_connect on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:56.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2830376112 <== mds.0 v2:172.21.15.66:6835/2419209461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d879d2ad00 con 0x55d879f5ce00 2024-03-30T00:47:56.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbcbe00640 10 client.8382 handle_command_reply: tid=0 2024-03-30T00:47:56.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 1 client.8382 shutdown 2024-03-30T00:47:56.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2830376112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 msgr2=0x55d879f39a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f39a50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0000fc0 tx=0x7fcbd00544f0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2830376112 shutdown_connections 2024-03-30T00:47:56.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d879f5ce00 0x55d879f5d1e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f56390 0x55d879f39f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f39a50 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbcbe00640 0 client.8382 ms_handle_reset on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:56.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2830376112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f09a00 0x55d879f127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2830376112 >> 172.21.15.66:0/2830376112 conn(0x55d879d0a160 msgr2=0x55d879f08bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2830376112 shutdown_connections 2024-03-30T00:47:56.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2830376112 wait complete. 2024-03-30T00:47:56.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.949+0000 7fcbf0c95000 20 client.8382 trim_cache size 0 max 16384 2024-03-30T00:47:56.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 Processor -- start 2024-03-30T00:47:56.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 -- start start 2024-03-30T00:47:56.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f09a00 0x55d879f08c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f09170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f56390 0x55d879f09de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d879a17aa0 con 0x55d879f55a70 2024-03-30T00:47:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d879d1bc60 con 0x55d879f56390 2024-03-30T00:47:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d879f0f0b0 con 0x55d879f09a00 2024-03-30T00:47:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f09170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f09170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55268/0 (socket says 172.21.15.66:55268) 2024-03-30T00:47:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbdfe00640 1 -- 172.21.15.66:0/3228442448 learned_addr learned my addr 172.21.15.66:0/3228442448 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/3228442448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f09a00 0x55d879f08c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbe7400640 1 --2- 172.21.15.66:0/3228442448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f56390 0x55d879f09de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbdfe00640 1 -- 172.21.15.66:0/3228442448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f09a00 msgr2=0x55d879f08c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/3228442448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f09a00 0x55d879f08c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbdfe00640 1 -- 172.21.15.66:0/3228442448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f56390 msgr2=0x55d879f09de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/3228442448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f56390 0x55d879f09de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbdfe00640 1 -- 172.21.15.66:0/3228442448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d879d05df0 con 0x55d879f55a70 2024-03-30T00:47:56.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/3228442448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f09170 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0058430 tx=0x7fcbd0057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13474e0d0f8d8c2f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbd7e00640 1 -- 172.21.15.66:0/3228442448 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000e040 con 0x55d879f55a70 2024-03-30T00:47:56.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbd7e00640 1 -- 172.21.15.66:0/3228442448 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d879d593e0 con 0x55d879f55a70 2024-03-30T00:47:56.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbd7e00640 1 -- 172.21.15.66:0/3228442448 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd005c5e0 con 0x55d879f55a70 2024-03-30T00:47:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 -- 172.21.15.66:0/3228442448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 msgr2=0x55d879f09170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/3228442448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f09170 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0058430 tx=0x7fcbd0057000 comp rx=0 tx=0).stop 2024-03-30T00:47:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 -- 172.21.15.66:0/3228442448 shutdown_connections 2024-03-30T00:47:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/3228442448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f56390 0x55d879f09de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/3228442448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879f09170 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/3228442448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f09a00 0x55d879f08c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 -- 172.21.15.66:0/3228442448 >> 172.21.15.66:0/3228442448 conn(0x55d879d0a160 msgr2=0x55d879f11840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 -- 172.21.15.66:0/3228442448 shutdown_connections 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 -- 172.21.15.66:0/3228442448 wait complete. 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.965+0000 7fcbf0c95000 1 -- start start 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 Processor -- start 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f09a00 0x55d879fb6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879fb6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f56390 0x55d879f0cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d879d5d5f0 con 0x55d879f55a70 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d879d59920 con 0x55d879f09a00 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d879d5ea90 con 0x55d879f56390 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 10 client.0 _ll_register_callbacks cb 0x7fcbd8007790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879fb6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879fb6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55278/0 (socket says 172.21.15.66:55278) 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2237458793 learned_addr learned my addr 172.21.15.66:0/2237458793 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:47:56.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbe7400640 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f56390 0x55d879f0cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f09a00 0x55d879fb6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2237458793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f56390 msgr2=0x55d879f0cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f56390 0x55d879f0cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2237458793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f09a00 msgr2=0x55d879fb6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f09a00 0x55d879fb6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbdfe00640 1 -- 172.21.15.66:0/2237458793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d879d1faa0 con 0x55d879f55a70 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbdfe00640 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879fb6740 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0050f10 tx=0x7fcbd0081750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab61ae76869a110 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbcbe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2237458793 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000e040 con 0x55d879f55a70 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2237458793 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fcbd001dd00 con 0x55d879f55a70 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2237458793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d879d17970 con 0x55d879f55a70 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2237458793 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbd005c650 con 0x55d879f55a70 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 10 client.8388 fetch_fsmap learned FSMap version 71 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2237458793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55d879cf9960 con 0x55d879f55a70 2024-03-30T00:47:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2237458793 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d879d17970 con 0x55d879f55a70 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2237458793 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcbd007a070 con 0x55d879f55a70 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 10 client.8388 fetch_fsmap finished waiting for FSMap version 71 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 10 client.8388 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 20 client.8388 populate_metadata read hostname 'smithi066' 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d879f0d0e0 0x55d879f0d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 4 client.8388 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2237458793 --> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d879d89a20 con 0x55d879f0d0e0 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d879f0d0e0 0x55d879f0d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbe6a00640 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d879f0d0e0 0x55d879f0d4f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=602b7a358c1c5edb server_cookie=d8058624c54123c5 in_seq=0 out_seq=0 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbcbe00640 10 client.8388 ms_handle_connect on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbcbe00640 1 -- 172.21.15.66:0/2237458793 <== mds.0 v2:172.21.15.66:6835/2419209461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x55d879d89a20 con 0x55d879f0d0e0 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbcbe00640 10 client.8388 handle_command_reply: tid=0 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 client.8388 shutdown 2024-03-30T00:47:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2237458793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 msgr2=0x55d879fb6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879fb6740 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0050f10 tx=0x7fcbd0081750 comp rx=0 tx=0).stop 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2237458793 shutdown_connections 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x55d879f0d0e0 0x55d879f0d4f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55d879f56390 0x55d879f0cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d879f55a70 0x55d879fb6740 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 --2- 172.21.15.66:0/2237458793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d879f09a00 0x55d879fb6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2237458793 >> 172.21.15.66:0/2237458793 conn(0x55d879d0a160 msgr2=0x55d879f07c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.969+0000 7fcbcbe00640 0 client.8388 ms_handle_reset on v2:172.21.15.66:6835/2419209461 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2237458793 shutdown_connections 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbf0c95000 1 -- 172.21.15.66:0/2237458793 wait complete. 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbf0c95000 20 client.8388 trim_cache size 0 max 16384 2024-03-30T00:47:56.985 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:47:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:47:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-03-30T00:47:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "08b1ec23-9bce-4806-982e-95e55fbea3ef": { 2024-03-30T00:47:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/dir", 2024-03-30T00:47:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "08b1ec23-9bce-4806-982e-95e55fbea3ef", 2024-03-30T00:47:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,repair" 2024-03-30T00:47:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:47:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:47:56.986 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:47:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 -- 172.21.15.66:0/2051610642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbc4066690 msgr2=0x7fcbc4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbc4066690 0x7fcbc4068b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8165400 tx=0x7fcbd8008040 comp rx=0 tx=0).stop 2024-03-30T00:47:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 -- 172.21.15.66:0/2051610642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe815a7d0 msgr2=0x7fcbe8070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:47:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe815a7d0 0x7fcbe8070280 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000c4b0 tx=0x7fcbd000c970 comp rx=0 tx=0).stop 2024-03-30T00:47:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 -- 172.21.15.66:0/2051610642 shutdown_connections 2024-03-30T00:47:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbe815b780 0x7fcbe80707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcbc4066690 0x7fcbc4068b50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe815a7d0 0x7fcbe8070280 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 --2- 172.21.15.66:0/2051610642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbe80b0500 0x7fcbe813ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:47:56.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 -- 172.21.15.66:0/2051610642 >> 172.21.15.66:0/2051610642 conn(0x7fcbe8065cd0 msgr2=0x7fcbe80cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:47:56.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 -- 172.21.15.66:0/2051610642 shutdown_connections 2024-03-30T00:47:56.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:47:56.973+0000 7fcbe7e00640 1 -- 172.21.15.66:0/2051610642 wait complete. 2024-03-30T00:47:57.001 INFO:tasks.cephfs.filesystem:scrub status for tag:08b1ec23-9bce-4806-982e-95e55fbea3ef - {'path': '/dir', 'tag': '08b1ec23-9bce-4806-982e-95e55fbea3ef', 'options': 'recursive,repair'} 2024-03-30T00:48:17.309 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:48:17.314 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:48:27.003 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-30T00:48:27.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4ca0c00640 1 Processor -- start 2024-03-30T00:48:27.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4ca0c00640 1 -- start start 2024-03-30T00:48:27.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4ca0c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4c9c0cda90 0x7f4c9c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4ca0c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9c0cd3e0 con 0x7f4c9c0cde70 2024-03-30T00:48:27.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4ca0c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9c15a990 con 0x7f4c9c15a370 2024-03-30T00:48:27.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4ca0c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c0b1b80 con 0x7f4c9c0cda90 2024-03-30T00:48:27.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4c99e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4c9c0cda90 0x7f4c9c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4c99e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4c9c0cda90 0x7f4c9c0ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44600/0 (socket says 172.21.15.66:44600) 2024-03-30T00:48:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4c99e00640 1 -- 172.21.15.66:0/669807871 learned_addr learned my addr 172.21.15.66:0/669807871 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4c91400640 1 --1- 172.21.15.66:0/669807871 >> v1:172.21.15.66:6789/0 conn(0x7f4c9c0cde70 0x7f4c9c159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45292/0 (socket says 172.21.15.66:45292) 2024-03-30T00:48:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4c99e00640 1 -- 172.21.15.66:0/669807871 >> v1:172.21.15.156:6789/0 conn(0x7f4c9c15a370 legacy=0x7f4c9c156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4c99e00640 1 -- 172.21.15.66:0/669807871 >> v1:172.21.15.66:6789/0 conn(0x7f4c9c0cde70 legacy=0x7f4c9c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4c99e00640 1 -- 172.21.15.66:0/669807871 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c0b11e0 con 0x7f4c9c0cda90 2024-03-30T00:48:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.149+0000 7f4c99400640 1 -- 172.21.15.66:0/669807871 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266782913 0 0) 0x7f4c9c0cd3e0 con 0x7f4c9c0cde70 2024-03-30T00:48:27.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.153+0000 7f4c99e00640 1 --2- 172.21.15.66:0/669807871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c9c0cda90 0x7f4c9c0ce520 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009870 tx=0x7f4c8c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb39a2b19ceceab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.153+0000 7f4c99400640 1 -- 172.21.15.66:0/669807871 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0ab600 con 0x7f4c9c0cda90 2024-03-30T00:48:27.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.153+0000 7f4c99400640 1 -- 172.21.15.66:0/669807871 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c0a7050 con 0x7f4c9c0cda90 2024-03-30T00:48:27.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.153+0000 7f4c99400640 1 -- 172.21.15.66:0/669807871 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0ab980 con 0x7f4c9c0cda90 2024-03-30T00:48:27.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.153+0000 7f4ca0c00640 1 -- 172.21.15.66:0/669807871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c9c0cda90 msgr2=0x7f4c9c0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.153+0000 7f4ca0c00640 1 --2- 172.21.15.66:0/669807871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c9c0cda90 0x7f4c9c0ce520 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009870 tx=0x7f4c8c01c4d0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.153+0000 7f4ca0c00640 1 -- 172.21.15.66:0/669807871 shutdown_connections 2024-03-30T00:48:27.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.153+0000 7f4ca0c00640 1 --2- 172.21.15.66:0/669807871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c9c0cda90 0x7f4c9c0ce520 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.153+0000 7f4ca0c00640 1 -- 172.21.15.66:0/669807871 >> 172.21.15.66:0/669807871 conn(0x7f4c9c069f60 msgr2=0x7f4c9c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.153+0000 7f4ca0c00640 1 -- 172.21.15.66:0/669807871 shutdown_connections 2024-03-30T00:48:27.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.153+0000 7f4ca0c00640 1 -- 172.21.15.66:0/669807871 wait complete. 2024-03-30T00:48:27.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4ca0c00640 1 Processor -- start 2024-03-30T00:48:27.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4ca0c00640 1 -- start start 2024-03-30T00:48:27.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4ca0c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c9c0cda90 0x7f4c9c15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4ca0c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c9c0cde70 0x7f4c9c0c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4ca0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c9c15a370 0x7f4c9c097a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4ca0c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c158bb0 con 0x7f4c9c15a370 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4ca0c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c158e90 con 0x7f4c9c0cda90 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4ca0c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c159170 con 0x7f4c9c0cde70 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c9c15a370 0x7f4c9c097a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c9c15a370 0x7f4c9c097a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44614/0 (socket says 172.21.15.66:44614) 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c9a800640 1 -- 172.21.15.66:0/3049682192 learned_addr learned my addr 172.21.15.66:0/3049682192 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c99e00640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c9c0cda90 0x7f4c9c15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c91400640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c9c0cde70 0x7f4c9c0c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c9a800640 1 -- 172.21.15.66:0/3049682192 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c9c0cde70 msgr2=0x7f4c9c0c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c9a800640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c9c0cde70 0x7f4c9c0c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c9a800640 1 -- 172.21.15.66:0/3049682192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c9c0cda90 msgr2=0x7f4c9c15dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c9a800640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c9c0cda90 0x7f4c9c15dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c9a800640 1 -- 172.21.15.66:0/3049682192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0094d0 con 0x7f4c9c15a370 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c9a800640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c9c15a370 0x7f4c9c097a60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400c340 tx=0x7f4c9400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c92a00640 1 -- 172.21.15.66:0/3049682192 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94005220 con 0x7f4c9c15a370 2024-03-30T00:48:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c92a00640 1 -- 172.21.15.66:0/3049682192 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c94005380 con 0x7f4c9c15a370 2024-03-30T00:48:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4ca0c00640 1 -- 172.21.15.66:0/3049682192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c159450 con 0x7f4c9c15a370 2024-03-30T00:48:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4c92a00640 1 -- 172.21.15.66:0/3049682192 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94007270 con 0x7f4c9c15a370 2024-03-30T00:48:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.157+0000 7f4ca0c00640 1 -- 172.21.15.66:0/3049682192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c0c84a0 con 0x7f4c9c15a370 2024-03-30T00:48:27.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c92a00640 1 -- 172.21.15.66:0/3049682192 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4c940073d0 con 0x7f4c9c15a370 2024-03-30T00:48:27.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c92a00640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c78066690 0x7f4c78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c92a00640 1 -- 172.21.15.66:0/3049682192 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f4c94083800 con 0x7f4c9c15a370 2024-03-30T00:48:27.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c99e00640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c78066690 0x7f4c78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4ca3ff2000 1 Processor -- start 2024-03-30T00:48:27.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4ca3ff2000 1 -- start start 2024-03-30T00:48:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c99e00640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c78066690 0x7f4c78068b50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0ad000 tx=0x7f4c8c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c6ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c6f390 0x561b62c224c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62c22a00 0x561b62c22de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b62aed630 con 0x561b62c6ea70 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b62a7abd0 con 0x561b62c22a00 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b62a781a0 con 0x561b62c6f390 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c99e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c6ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c99e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c6ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44628/0 (socket says 172.21.15.66:44628) 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c99e00640 1 -- 172.21.15.66:0/2330644884 learned_addr learned my addr 172.21.15.66:0/2330644884 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c91400640 1 --2- 172.21.15.66:0/2330644884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c6f390 0x561b62c224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c9a800640 1 --2- 172.21.15.66:0/2330644884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62c22a00 0x561b62c22de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c99e00640 1 -- 172.21.15.66:0/2330644884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c6f390 msgr2=0x561b62c224c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c99e00640 1 --2- 172.21.15.66:0/2330644884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c6f390 0x561b62c224c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c99e00640 1 -- 172.21.15.66:0/2330644884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62c22a00 msgr2=0x561b62c22de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c99e00640 1 --2- 172.21.15.66:0/2330644884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62c22a00 0x561b62c22de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c99e00640 1 -- 172.21.15.66:0/2330644884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b6276fe20 con 0x561b62c6ea70 2024-03-30T00:48:27.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c9a800640 1 --2- 172.21.15.66:0/2330644884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62c22a00 0x561b62c22de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:27.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c99e00640 1 --2- 172.21.15.66:0/2330644884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c6ee50 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c01ca40 tx=0x7f4c8c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7a548a20a26e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c8aa00640 1 -- 172.21.15.66:0/2330644884 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c003da0 con 0x561b62c6ea70 2024-03-30T00:48:27.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.161+0000 7f4c8aa00640 1 -- 172.21.15.66:0/2330644884 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c0bd040 con 0x561b62c6ea70 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c8aa00640 1 -- 172.21.15.66:0/2330644884 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0c3680 con 0x561b62c6ea70 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/2330644884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 msgr2=0x561b62c6ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/2330644884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c6ee50 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c01ca40 tx=0x7f4c8c004740 comp rx=0 tx=0).stop 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/2330644884 shutdown_connections 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/2330644884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62c22a00 0x561b62c22de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/2330644884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c6f390 0x561b62c224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/2330644884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c6ee50 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/2330644884 >> 172.21.15.66:0/2330644884 conn(0x561b62a23160 msgr2=0x561b62c51820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/2330644884 shutdown_connections 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/2330644884 wait complete. 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 -- start start 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 Processor -- start 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62c22a00 0x561b62c2aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c6ea70 0x561b62c52a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6f390 0x561b62c52f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b62a73a90 con 0x561b62c6f390 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b62a12d40 con 0x561b62c22a00 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b62a30f30 con 0x561b62c6ea70 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 10 client.0 _ll_register_callbacks cb 0x561b62ad5810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6f390 0x561b62c52f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6f390 0x561b62c52f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44632/0 (socket says 172.21.15.66:44632) 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c9a800640 1 -- 172.21.15.66:0/874757473 learned_addr learned my addr 172.21.15.66:0/874757473 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c91400640 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c6ea70 0x561b62c52a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c9a800640 1 -- 172.21.15.66:0/874757473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c6ea70 msgr2=0x561b62c52a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c9a800640 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c6ea70 0x561b62c52a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c9a800640 1 -- 172.21.15.66:0/874757473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62c22a00 msgr2=0x561b62c2aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c9a800640 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62c22a00 0x561b62c2aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c9a800640 1 -- 172.21.15.66:0/874757473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c0b11e0 con 0x561b62c6f390 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c9a800640 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6f390 0x561b62c52f90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400a040 tx=0x7f4c94007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a1e3c4045a7f0a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c7f400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c7f400640 1 -- 172.21.15.66:0/874757473 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94058160 con 0x561b62c6f390 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4c7f400640 1 -- 172.21.15.66:0/874757473 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f4c94058360 con 0x561b62c6f390 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.165+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/874757473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b62a723e0 con 0x561b62c6f390 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4c7f400640 1 -- 172.21.15.66:0/874757473 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c9405c920 con 0x561b62c6f390 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 10 client.8403 fetch_fsmap learned FSMap version 71 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/874757473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x561b62a38aa0 con 0x561b62c6f390 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4c7f400640 1 -- 172.21.15.66:0/874757473 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561b62a723e0 con 0x561b62c6f390 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4c7f400640 1 -- 172.21.15.66:0/874757473 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4c94070080 con 0x561b62c6f390 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 10 client.8403 fetch_fsmap finished waiting for FSMap version 71 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 10 client.8403 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 20 client.8403 populate_metadata read hostname 'smithi066' 2024-03-30T00:48:27.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x561b62c75eb0 0x561b62c76290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 4 client.8403 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:48:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/874757473 --> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b62a43d00 con 0x561b62c75eb0 2024-03-30T00:48:27.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4c99e00640 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x561b62c75eb0 0x561b62c76290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4c99e00640 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x561b62c75eb0 0x561b62c76290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fdb90cc1091e98a server_cookie=1e1311d573a3c8b7 in_seq=0 out_seq=0 2024-03-30T00:48:27.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4c7f400640 10 client.8403 ms_handle_connect on v2:172.21.15.66:6835/2419209461 2024-03-30T00:48:27.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4c7f400640 1 -- 172.21.15.66:0/874757473 <== mds.0 v2:172.21.15.66:6835/2419209461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561b62a43d00 con 0x561b62c75eb0 2024-03-30T00:48:27.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4c7f400640 10 client.8403 handle_command_reply: tid=0 2024-03-30T00:48:27.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 client.8403 shutdown 2024-03-30T00:48:27.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/874757473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6f390 msgr2=0x561b62c52f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6f390 0x561b62c52f90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400a040 tx=0x7f4c94007680 comp rx=0 tx=0).stop 2024-03-30T00:48:27.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/874757473 shutdown_connections 2024-03-30T00:48:27.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6f390 0x561b62c52f90 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x561b62c75eb0 0x561b62c76290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c6ea70 0x561b62c52a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/874757473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62c22a00 0x561b62c2aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4c7f400640 0 client.8403 ms_handle_reset on v2:172.21.15.66:6835/2419209461 2024-03-30T00:48:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/874757473 >> 172.21.15.66:0/874757473 conn(0x561b62a23160 msgr2=0x561b62c21bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/874757473 shutdown_connections 2024-03-30T00:48:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/874757473 wait complete. 2024-03-30T00:48:27.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.169+0000 7f4ca3ff2000 20 client.8403 trim_cache size 0 max 16384 2024-03-30T00:48:27.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 Processor -- start 2024-03-30T00:48:27.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 -- start start 2024-03-30T00:48:27.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62a23160 0x561b62c299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c22a00 0x561b62c29f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c22de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b62730aa0 con 0x561b62c6ea70 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b62a6f280 con 0x561b62a23160 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b62b44470 con 0x561b62c22a00 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c22de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c22de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44648/0 (socket says 172.21.15.66:44648) 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c9a800640 1 -- 172.21.15.66:0/4116768114 learned_addr learned my addr 172.21.15.66:0/4116768114 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c91400640 1 --2- 172.21.15.66:0/4116768114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c22a00 0x561b62c29f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c99e00640 1 --2- 172.21.15.66:0/4116768114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62a23160 0x561b62c299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c9a800640 1 -- 172.21.15.66:0/4116768114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c22a00 msgr2=0x561b62c29f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c9a800640 1 --2- 172.21.15.66:0/4116768114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c22a00 0x561b62c29f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c9a800640 1 -- 172.21.15.66:0/4116768114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62a23160 msgr2=0x561b62c299e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c9a800640 1 --2- 172.21.15.66:0/4116768114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62a23160 0x561b62c299e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c9a800640 1 -- 172.21.15.66:0/4116768114 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b62a1edf0 con 0x561b62c6ea70 2024-03-30T00:48:27.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c9a800640 1 --2- 172.21.15.66:0/4116768114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c22de0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4c94050ae0 tx=0x7f4c94083070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7244468877d5fa69 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c8b400640 1 -- 172.21.15.66:0/4116768114 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9405c920 con 0x561b62c6ea70 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c8b400640 1 -- 172.21.15.66:0/4116768114 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9406d050 con 0x561b62c6ea70 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4c8b400640 1 -- 172.21.15.66:0/4116768114 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94050920 con 0x561b62c6ea70 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/4116768114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 msgr2=0x561b62c22de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/4116768114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c22de0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4c94050ae0 tx=0x7f4c94083070 comp rx=0 tx=0).stop 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/4116768114 shutdown_connections 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/4116768114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62c22de0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/4116768114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c22a00 0x561b62c29f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/4116768114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62a23160 0x561b62c299e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/4116768114 >> 172.21.15.66:0/4116768114 conn(0x561b62c6def0 msgr2=0x561b62c21b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/4116768114 shutdown_connections 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.185+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/4116768114 wait complete. 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 -- start start 2024-03-30T00:48:27.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 Processor -- start 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62a23160 0x561b62c25a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c22a00 0x561b62c25fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62ccb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b62a3f390 con 0x561b62c6ea70 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b62ab7170 con 0x561b62a23160 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b62a712a0 con 0x561b62c22a00 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 10 client.0 _ll_register_callbacks cb 0x7f4c8c0ba070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62ccb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62ccb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44654/0 (socket says 172.21.15.66:44654) 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c9a800640 1 -- 172.21.15.66:0/1651585175 learned_addr learned my addr 172.21.15.66:0/1651585175 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c91400640 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c22a00 0x561b62c25fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c99e00640 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62a23160 0x561b62c25a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c9a800640 1 -- 172.21.15.66:0/1651585175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c22a00 msgr2=0x561b62c25fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c9a800640 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c22a00 0x561b62c25fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c9a800640 1 -- 172.21.15.66:0/1651585175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62a23160 msgr2=0x561b62c25a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c9a800640 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62a23160 0x561b62c25a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c9a800640 1 -- 172.21.15.66:0/1651585175 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b62a38aa0 con 0x561b62c6ea70 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c91400640 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c22a00 0x561b62c25fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c99e00640 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62a23160 0x561b62c25a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c9a800640 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62ccb060 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f4c940547a0 tx=0x7f4c94081900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e022375acc3b07b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c7f400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c7f400640 1 -- 172.21.15.66:0/1651585175 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9405c920 con 0x561b62c6ea70 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c7f400640 1 -- 172.21.15.66:0/1651585175 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f4c94058360 con 0x561b62c6ea70 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/1651585175 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b62a30970 con 0x561b62c6ea70 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c7f400640 1 -- 172.21.15.66:0/1651585175 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c94050920 con 0x561b62c6ea70 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 10 client.8409 fetch_fsmap learned FSMap version 71 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/1651585175 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x561b62a12960 con 0x561b62c6ea70 2024-03-30T00:48:27.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c7f400640 1 -- 172.21.15.66:0/1651585175 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561b62a30970 con 0x561b62c6ea70 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c7f400640 1 -- 172.21.15.66:0/1651585175 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4c94070070 con 0x561b62c6ea70 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 10 client.8409 fetch_fsmap finished waiting for FSMap version 71 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 10 client.8409 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 20 client.8409 populate_metadata read hostname 'smithi066' 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x561b62c6f390 0x561b62ccb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 4 client.8409 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/1651585175 --> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561b62aa2a20 con 0x561b62c6f390 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c99e00640 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x561b62c6f390 0x561b62ccb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c7f400640 10 client.8409 ms_handle_connect on v2:172.21.15.66:6835/2419209461 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.189+0000 7f4c99e00640 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x561b62c6f390 0x561b62ccb5a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a69b86c0beec2d3b server_cookie=e4d95b9fe0bad77b in_seq=0 out_seq=0 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4c7f400640 1 -- 172.21.15.66:0/1651585175 <== mds.0 v2:172.21.15.66:6835/2419209461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561b62aa2a20 con 0x561b62c6f390 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4c7f400640 10 client.8409 handle_command_reply: tid=0 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 1 client.8409 shutdown 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/1651585175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 msgr2=0x561b62ccb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62ccb060 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f4c940547a0 tx=0x7f4c94081900 comp rx=0 tx=0).stop 2024-03-30T00:48:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/1651585175 shutdown_connections 2024-03-30T00:48:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b62c6ea70 0x561b62ccb060 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x561b62c6f390 0x561b62ccb5a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561b62c22a00 0x561b62c25fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 1 --2- 172.21.15.66:0/1651585175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561b62a23160 0x561b62c25a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4c7f400640 0 client.8409 ms_handle_reset on v2:172.21.15.66:6835/2419209461 2024-03-30T00:48:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/1651585175 >> 172.21.15.66:0/1651585175 conn(0x561b62c6def0 msgr2=0x561b62c289d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/1651585175 shutdown_connections 2024-03-30T00:48:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 1 -- 172.21.15.66:0/1651585175 wait complete. 2024-03-30T00:48:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca3ff2000 20 client.8409 trim_cache size 0 max 16384 2024-03-30T00:48:27.206 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:48:27.207 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T00:48:27.207 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-03-30T00:48:27.207 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:48:27.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 -- 172.21.15.66:0/3049682192 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c78066690 msgr2=0x7f4c78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c78066690 0x7f4c78068b50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0ad000 tx=0x7f4c8c0a6000 comp rx=0 tx=0).stop 2024-03-30T00:48:27.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 -- 172.21.15.66:0/3049682192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c9c15a370 msgr2=0x7f4c9c097a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c9c15a370 0x7f4c9c097a60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400c340 tx=0x7f4c9400c800 comp rx=0 tx=0).stop 2024-03-30T00:48:27.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 -- 172.21.15.66:0/3049682192 shutdown_connections 2024-03-30T00:48:27.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c78066690 0x7f4c78068b50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c9c15a370 0x7f4c9c097a60 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c9c0cde70 0x7f4c9c0c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 --2- 172.21.15.66:0/3049682192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c9c0cda90 0x7f4c9c15dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 -- 172.21.15.66:0/3049682192 >> 172.21.15.66:0/3049682192 conn(0x7f4c9c069f60 msgr2=0x7f4c9c0c9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 -- 172.21.15.66:0/3049682192 shutdown_connections 2024-03-30T00:48:27.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.193+0000 7f4ca0c00640 1 -- 172.21.15.66:0/3049682192 wait complete. 2024-03-30T00:48:27.222 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:48:27.222 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:48:27.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.353+0000 7fd06b000640 1 Processor -- start 2024-03-30T00:48:27.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.353+0000 7fd06b000640 1 -- start start 2024-03-30T00:48:27.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.353+0000 7fd06b000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd0641610c0 0x7fd0641604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.353+0000 7fd06b000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd0641609f0 0x7fd0640abc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.353+0000 7fd06b000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06406da20 con 0x7fd064157060 2024-03-30T00:48:27.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.353+0000 7fd06b000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd064157440 con 0x7fd0641609f0 2024-03-30T00:48:27.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.353+0000 7fd06b000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd064160e80 con 0x7fd0641610c0 2024-03-30T00:48:27.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.353+0000 7fd069000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd0641609f0 0x7fd0640abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.353+0000 7fd069000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd0641609f0 0x7fd0640abc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44656/0 (socket says 172.21.15.66:44656) 2024-03-30T00:48:27.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.353+0000 7fd069000640 1 -- 172.21.15.66:0/1967779990 learned_addr learned my addr 172.21.15.66:0/1967779990 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.357+0000 7fd05be00640 1 --2- 172.21.15.66:0/1967779990 >> v2:172.21.15.156:3300/0 conn(0x7fd0641610c0 0x7fd0641604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.357+0000 7fd069000640 1 -- 172.21.15.66:0/1967779990 >> v2:172.21.15.156:3300/0 conn(0x7fd0641610c0 msgr2=0x7fd0641604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.357+0000 7fd069000640 1 --2- 172.21.15.66:0/1967779990 >> v2:172.21.15.156:3300/0 conn(0x7fd0641610c0 0x7fd0641604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.357+0000 7fd069000640 1 -- 172.21.15.66:0/1967779990 >> v1:172.21.15.156:6789/0 conn(0x7fd064157060 legacy=0x7fd06406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.357+0000 7fd069000640 1 -- 172.21.15.66:0/1967779990 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06406dc60 con 0x7fd0641609f0 2024-03-30T00:48:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.357+0000 7fd069000640 1 --2- 172.21.15.66:0/1967779990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0641609f0 0x7fd0640abc00 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fd06406f3c0 tx=0x7fd05c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4829aae42e7f6286 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.357+0000 7fd063400640 1 -- 172.21.15.66:0/1967779990 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0a5640 con 0x7fd0641609f0 2024-03-30T00:48:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.357+0000 7fd063400640 1 -- 172.21.15.66:0/1967779990 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c0a57a0 con 0x7fd0641609f0 2024-03-30T00:48:27.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.357+0000 7fd063400640 1 -- 172.21.15.66:0/1967779990 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0a5b30 con 0x7fd0641609f0 2024-03-30T00:48:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.357+0000 7fd06b000640 1 -- 172.21.15.66:0/1967779990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0641609f0 msgr2=0x7fd0640abc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.357+0000 7fd06b000640 1 --2- 172.21.15.66:0/1967779990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0641609f0 0x7fd0640abc00 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fd06406f3c0 tx=0x7fd05c01c890 comp rx=0 tx=0).stop 2024-03-30T00:48:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 -- 172.21.15.66:0/1967779990 shutdown_connections 2024-03-30T00:48:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 --2- 172.21.15.66:0/1967779990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0641609f0 0x7fd0640abc00 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 --2- 172.21.15.66:0/1967779990 >> v2:172.21.15.156:3300/0 conn(0x7fd0641610c0 0x7fd0641604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 -- 172.21.15.66:0/1967779990 >> 172.21.15.66:0/1967779990 conn(0x7fd064071750 msgr2=0x7fd064069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 -- 172.21.15.66:0/1967779990 shutdown_connections 2024-03-30T00:48:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 -- 172.21.15.66:0/1967779990 wait complete. 2024-03-30T00:48:27.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 Processor -- start 2024-03-30T00:48:27.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 -- start start 2024-03-30T00:48:27.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd064157060 0x7fd0640b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd0641609f0 0x7fd0640b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0641610c0 0x7fd06414ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06414f210 con 0x7fd0641610c0 2024-03-30T00:48:27.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640b28e0 con 0x7fd064157060 2024-03-30T00:48:27.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd06b000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640b2bc0 con 0x7fd0641609f0 2024-03-30T00:48:27.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd069000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0641610c0 0x7fd06414ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd069000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0641610c0 0x7fd06414ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44660/0 (socket says 172.21.15.66:44660) 2024-03-30T00:48:27.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd069000640 1 -- 172.21.15.66:0/2187876603 learned_addr learned my addr 172.21.15.66:0/2187876603 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd05be00640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd0641609f0 0x7fd0640b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd063e00640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd064157060 0x7fd0640b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd069000640 1 -- 172.21.15.66:0/2187876603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd0641609f0 msgr2=0x7fd0640b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd069000640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd0641609f0 0x7fd0640b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd069000640 1 -- 172.21.15.66:0/2187876603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd064157060 msgr2=0x7fd0640b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd069000640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd064157060 0x7fd0640b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd069000640 1 -- 172.21.15.66:0/2187876603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c009b30 con 0x7fd0641610c0 2024-03-30T00:48:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd069000640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0641610c0 0x7fd06414ecd0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd05c01a4c0 tx=0x7fd05c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.361+0000 7fd060c00640 1 -- 172.21.15.66:0/2187876603 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0b8670 con 0x7fd0641610c0 2024-03-30T00:48:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.365+0000 7fd060c00640 1 -- 172.21.15.66:0/2187876603 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c0b87f0 con 0x7fd0641610c0 2024-03-30T00:48:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.365+0000 7fd06b000640 1 -- 172.21.15.66:0/2187876603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0640b2ea0 con 0x7fd0641610c0 2024-03-30T00:48:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.365+0000 7fd060c00640 1 -- 172.21.15.66:0/2187876603 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0b8ba0 con 0x7fd0641610c0 2024-03-30T00:48:27.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.365+0000 7fd06b000640 1 -- 172.21.15.66:0/2187876603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0640b3460 con 0x7fd0641610c0 2024-03-30T00:48:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.365+0000 7fd06b000640 1 -- 172.21.15.66:0/2187876603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0640b3680 con 0x7fd0641610c0 2024-03-30T00:48:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.365+0000 7fd060c00640 1 -- 172.21.15.66:0/2187876603 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd05c0b8d00 con 0x7fd0641610c0 2024-03-30T00:48:27.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.365+0000 7fd060c00640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd040066690 0x7fd040068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.365+0000 7fd060c00640 1 -- 172.21.15.66:0/2187876603 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fd05c11f730 con 0x7fd0641610c0 2024-03-30T00:48:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.365+0000 7fd063e00640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd040066690 0x7fd040068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.365+0000 7fd063e00640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd040066690 0x7fd040068b50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd05400b810 tx=0x7fd054008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.369+0000 7fd060c00640 1 -- 172.21.15.66:0/2187876603 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd05c0ec950 con 0x7fd0641610c0 2024-03-30T00:48:27.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 -- 172.21.15.66:0/2187876603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd06406da20 con 0x7fd0641610c0 2024-03-30T00:48:27.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd060c00640 1 -- 172.21.15.66:0/2187876603 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fd06406da20 con 0x7fd0641610c0 2024-03-30T00:48:27.574 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:27.575 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:47:53.284661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/2419209461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2419209461},{"type":"v1","addr":"172.21.15.66:6836","nonce":2419209461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:48:27.575 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 71 2024-03-30T00:48:27.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 -- 172.21.15.66:0/2187876603 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd040066690 msgr2=0x7fd040068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd040066690 0x7fd040068b50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd05400b810 tx=0x7fd054008040 comp rx=0 tx=0).stop 2024-03-30T00:48:27.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 -- 172.21.15.66:0/2187876603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0641610c0 msgr2=0x7fd06414ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0641610c0 0x7fd06414ecd0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd05c01a4c0 tx=0x7fd05c019630 comp rx=0 tx=0).stop 2024-03-30T00:48:27.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 -- 172.21.15.66:0/2187876603 shutdown_connections 2024-03-30T00:48:27.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd040066690 0x7fd040068b50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0641610c0 0x7fd06414ecd0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd0641609f0 0x7fd0640b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 --2- 172.21.15.66:0/2187876603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd064157060 0x7fd0640b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 -- 172.21.15.66:0/2187876603 >> 172.21.15.66:0/2187876603 conn(0x7fd064071750 msgr2=0x7fd064156810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 -- 172.21.15.66:0/2187876603 shutdown_connections 2024-03-30T00:48:27.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.565+0000 7fd06b000640 1 -- 172.21.15.66:0/2187876603 wait complete. 2024-03-30T00:48:27.595 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-03-30T00:48:27.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f723bc00640 1 Processor -- start 2024-03-30T00:48:27.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f723bc00640 1 -- start start 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f723bc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f723415a640 0x7f723406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f723bc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f72340ce520 0x7f72340ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f723bc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f72340cabb0 0x7f72340cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f723bc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723406d070 con 0x7f72340cabb0 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f723bc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72340cbba0 con 0x7f72340ce520 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f723bc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723406e2c0 con 0x7f723415a640 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f7239c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f72340cabb0 0x7f72340cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f7239c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f72340cabb0 0x7f72340cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44662/0 (socket says 172.21.15.66:44662) 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f7239c00640 1 -- 172.21.15.66:0/2210806473 learned_addr learned my addr 172.21.15.66:0/2210806473 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f7239200640 1 --2- 172.21.15.66:0/2210806473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f723415a640 0x7f723406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f7233e00640 1 --2- 172.21.15.66:0/2210806473 >> v2:172.21.15.156:3300/0 conn(0x7f72340ce520 0x7f72340ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f7239c00640 1 -- 172.21.15.66:0/2210806473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f723415a640 msgr2=0x7f723406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f7239c00640 1 --2- 172.21.15.66:0/2210806473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f723415a640 0x7f723406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f7239c00640 1 -- 172.21.15.66:0/2210806473 >> v2:172.21.15.156:3300/0 conn(0x7f72340ce520 msgr2=0x7f72340ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f7239c00640 1 --2- 172.21.15.66:0/2210806473 >> v2:172.21.15.156:3300/0 conn(0x7f72340ce520 0x7f72340ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f7239c00640 1 -- 172.21.15.66:0/2210806473 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72340cee60 con 0x7f72340cabb0 2024-03-30T00:48:27.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.729+0000 7f7233e00640 1 --2- 172.21.15.66:0/2210806473 >> v2:172.21.15.156:3300/0 conn(0x7f72340ce520 0x7f72340ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:27.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f7239c00640 1 --2- 172.21.15.66:0/2210806473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72340cabb0 0x7f72340cb000 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f72340cbe00 tx=0x7f722c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82cd242cc902c8a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f7233400640 1 -- 172.21.15.66:0/2210806473 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c0a5840 con 0x7f72340cabb0 2024-03-30T00:48:27.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f7233400640 1 -- 172.21.15.66:0/2210806473 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722c0a59a0 con 0x7f72340cabb0 2024-03-30T00:48:27.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f7233400640 1 -- 172.21.15.66:0/2210806473 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c0a5d30 con 0x7f72340cabb0 2024-03-30T00:48:27.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f723bc00640 1 -- 172.21.15.66:0/2210806473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72340cabb0 msgr2=0x7f72340cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f723bc00640 1 --2- 172.21.15.66:0/2210806473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72340cabb0 0x7f72340cb000 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f72340cbe00 tx=0x7f722c01c9c0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f723bc00640 1 -- 172.21.15.66:0/2210806473 shutdown_connections 2024-03-30T00:48:27.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f723bc00640 1 --2- 172.21.15.66:0/2210806473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72340cabb0 0x7f72340cb000 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f723bc00640 1 --2- 172.21.15.66:0/2210806473 >> v2:172.21.15.156:3300/0 conn(0x7f72340ce520 0x7f72340ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f723bc00640 1 --2- 172.21.15.66:0/2210806473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f723415a640 0x7f723406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f723bc00640 1 -- 172.21.15.66:0/2210806473 >> 172.21.15.66:0/2210806473 conn(0x7f7234069f60 msgr2=0x7f723406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f723bc00640 1 -- 172.21.15.66:0/2210806473 shutdown_connections 2024-03-30T00:48:27.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.733+0000 7f723bc00640 1 -- 172.21.15.66:0/2210806473 wait complete. 2024-03-30T00:48:27.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f723bc00640 1 Processor -- start 2024-03-30T00:48:27.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f723bc00640 1 -- start start 2024-03-30T00:48:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f723bc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72340cabb0 0x7f72340cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f723bc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72340ce520 0x7f7234097030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f723bc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f723415a640 0x7f7234071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f723bc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7234097570 con 0x7f72340cabb0 2024-03-30T00:48:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f723bc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7234159b40 con 0x7f723415a640 2024-03-30T00:48:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f723bc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72340cd0b0 con 0x7f72340ce520 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7239200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72340cabb0 0x7f72340cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7239200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72340cabb0 0x7f72340cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44672/0 (socket says 172.21.15.66:44672) 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7239200640 1 -- 172.21.15.66:0/1237681850 learned_addr learned my addr 172.21.15.66:0/1237681850 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7233e00640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72340ce520 0x7f7234097030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7239c00640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f723415a640 0x7f7234071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7239200640 1 -- 172.21.15.66:0/1237681850 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72340ce520 msgr2=0x7f7234097030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7239200640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72340ce520 0x7f7234097030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7239200640 1 -- 172.21.15.66:0/1237681850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f723415a640 msgr2=0x7f7234071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7239200640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f723415a640 0x7f7234071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7239200640 1 -- 172.21.15.66:0/1237681850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c009b30 con 0x7f72340cabb0 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7239200640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72340cabb0 0x7f72340cd8b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f722800d5f0 tx=0x7f722800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7230c00640 1 -- 172.21.15.66:0/1237681850 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72280125f0 con 0x7f72340cabb0 2024-03-30T00:48:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7230c00640 1 -- 172.21.15.66:0/1237681850 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7228012750 con 0x7f72340cabb0 2024-03-30T00:48:27.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f7230c00640 1 -- 172.21.15.66:0/1237681850 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228012ae0 con 0x7f72340cabb0 2024-03-30T00:48:27.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f723bc00640 1 -- 172.21.15.66:0/1237681850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72340cd390 con 0x7f72340cabb0 2024-03-30T00:48:27.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.737+0000 7f723bc00640 1 -- 172.21.15.66:0/1237681850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723415cfe0 con 0x7f72340cabb0 2024-03-30T00:48:27.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f7230c00640 1 -- 172.21.15.66:0/1237681850 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7228012c40 con 0x7f72340cabb0 2024-03-30T00:48:27.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f7230c00640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7214066690 0x7f7214068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f7230c00640 1 -- 172.21.15.66:0/1237681850 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f7228084950 con 0x7f72340cabb0 2024-03-30T00:48:27.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f7233e00640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7214066690 0x7f7214068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f723f0d8000 1 Processor -- start 2024-03-30T00:48:27.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f7233e00640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7214066690 0x7f7214068b50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f72341f8430 tx=0x7f7220008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f723f0d8000 1 -- start start 2024-03-30T00:48:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357a290 0x5616e357a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357abb0 0x5616e3530c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e3531590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f723f0d8000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e33f9340 con 0x5616e357abb0 2024-03-30T00:48:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f723f0d8000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e3385cd0 con 0x5616e35311b0 2024-03-30T00:48:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f723f0d8000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616e33832a0 con 0x5616e357a290 2024-03-30T00:48:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f7233e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357abb0 0x5616e3530c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f7233e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357abb0 0x5616e3530c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44678/0 (socket says 172.21.15.66:44678) 2024-03-30T00:48:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f7233e00640 1 -- 172.21.15.66:0/167520176 learned_addr learned my addr 172.21.15.66:0/167520176 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.741+0000 7f7239200640 1 --2- 172.21.15.66:0/167520176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357a290 0x5616e357a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f7239c00640 1 --2- 172.21.15.66:0/167520176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e3531590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f7233e00640 1 -- 172.21.15.66:0/167520176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357a290 msgr2=0x5616e357a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f7233e00640 1 --2- 172.21.15.66:0/167520176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357a290 0x5616e357a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f7233e00640 1 -- 172.21.15.66:0/167520176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 msgr2=0x5616e3531590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f7233e00640 1 --2- 172.21.15.66:0/167520176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e3531590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f7233e00640 1 -- 172.21.15.66:0/167520176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e30714b0 con 0x5616e357abb0 2024-03-30T00:48:27.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f7233e00640 1 --2- 172.21.15.66:0/167520176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357abb0 0x5616e3530c70 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f722000a870 tx=0x7f7220011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756b0bcbb7fc8da8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f7225600640 1 -- 172.21.15.66:0/167520176 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220013ca0 con 0x5616e357abb0 2024-03-30T00:48:27.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f7225600640 1 -- 172.21.15.66:0/167520176 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7220018070 con 0x5616e357abb0 2024-03-30T00:48:27.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f7225600640 1 -- 172.21.15.66:0/167520176 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722001f440 con 0x5616e357abb0 2024-03-30T00:48:27.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 -- 172.21.15.66:0/167520176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357abb0 msgr2=0x5616e3530c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 --2- 172.21.15.66:0/167520176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357abb0 0x5616e3530c70 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f722000a870 tx=0x7f7220011000 comp rx=0 tx=0).stop 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 -- 172.21.15.66:0/167520176 shutdown_connections 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 --2- 172.21.15.66:0/167520176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e3531590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 --2- 172.21.15.66:0/167520176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357abb0 0x5616e3530c70 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 --2- 172.21.15.66:0/167520176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357a290 0x5616e357a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 -- 172.21.15.66:0/167520176 >> 172.21.15.66:0/167520176 conn(0x5616e352d160 msgr2=0x5616e352f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 -- 172.21.15.66:0/167520176 shutdown_connections 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 -- 172.21.15.66:0/167520176 wait complete. 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 -- start start 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 Processor -- start 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e35366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 0x5616e3536bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 0x5616e353be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e337eb90 con 0x5616e357a290 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e333c0a0 con 0x5616e35311b0 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616e333bd40 con 0x5616e357abb0 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.745+0000 7f723f0d8000 10 client.0 _ll_register_callbacks cb 0x5616e3433c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7239200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e35366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7233e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 0x5616e3536bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7239200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e35366b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:51718/0 (socket says 172.21.15.66:51718) 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7239200640 1 -- 172.21.15.66:0/671702606 learned_addr learned my addr 172.21.15.66:0/671702606 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7239c00640 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 0x5616e353be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7233e00640 1 -- 172.21.15.66:0/671702606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 msgr2=0x5616e353be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7233e00640 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 0x5616e353be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7233e00640 1 -- 172.21.15.66:0/671702606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 msgr2=0x5616e35366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7233e00640 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e35366b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7233e00640 1 -- 172.21.15.66:0/671702606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722000e820 con 0x5616e357a290 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7233e00640 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 0x5616e3536bf0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f722000a260 tx=0x7f7220004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3bc69eabeb2386 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f721a000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f721a000640 1 -- 172.21.15.66:0/671702606 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220006450 con 0x5616e357a290 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f721a000640 1 -- 172.21.15.66:0/671702606 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f72200065b0 con 0x5616e357a290 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 -- 172.21.15.66:0/671702606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616e334e550 con 0x5616e357a290 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f721a000640 1 -- 172.21.15.66:0/671702606 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7220006790 con 0x5616e357a290 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 10 client.8427 fetch_fsmap learned FSMap version 71 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 -- 172.21.15.66:0/671702606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x5616e3343ba0 con 0x5616e357a290 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f721a000640 1 -- 172.21.15.66:0/671702606 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5616e334e550 con 0x5616e357a290 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f721a000640 1 -- 172.21.15.66:0/671702606 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7220030030 con 0x5616e357a290 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 10 client.8427 fetch_fsmap finished waiting for FSMap version 71 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 10 client.8427 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 20 client.8427 populate_metadata read hostname 'smithi066' 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x5616e3589320 0x5616e358b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 4 client.8427 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:48:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 -- 172.21.15.66:0/671702606 --> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616e337b970 con 0x5616e3589320 2024-03-30T00:48:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7239200640 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x5616e3589320 0x5616e358b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f7239200640 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x5616e3589320 0x5616e358b710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e940e825a3ac34e server_cookie=6f64d8c34e146d85 in_seq=0 out_seq=0 2024-03-30T00:48:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f721a000640 10 client.8427 ms_handle_connect on v2:172.21.15.66:6835/2419209461 2024-03-30T00:48:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f721a000640 1 -- 172.21.15.66:0/671702606 <== mds.0 v2:172.21.15.66:6835/2419209461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5616e337b970 con 0x5616e3589320 2024-03-30T00:48:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f721a000640 10 client.8427 handle_command_reply: tid=0 2024-03-30T00:48:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 client.8427 shutdown 2024-03-30T00:48:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 -- 172.21.15.66:0/671702606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 msgr2=0x5616e3536bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 0x5616e3536bf0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f722000a260 tx=0x7f7220004740 comp rx=0 tx=0).stop 2024-03-30T00:48:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 -- 172.21.15.66:0/671702606 shutdown_connections 2024-03-30T00:48:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x5616e3589320 0x5616e358b710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 0x5616e353be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 0x5616e3536bf0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 --2- 172.21.15.66:0/671702606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e35366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 -- 172.21.15.66:0/671702606 >> 172.21.15.66:0/671702606 conn(0x5616e352d160 msgr2=0x5616e352fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f721a000640 0 client.8427 ms_handle_reset on v2:172.21.15.66:6835/2419209461 2024-03-30T00:48:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 -- 172.21.15.66:0/671702606 shutdown_connections 2024-03-30T00:48:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 1 -- 172.21.15.66:0/671702606 wait complete. 2024-03-30T00:48:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.749+0000 7f723f0d8000 20 client.8427 trim_cache size 0 max 16384 2024-03-30T00:48:27.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.761+0000 7f723f0d8000 1 Processor -- start 2024-03-30T00:48:27.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.761+0000 7f723f0d8000 1 -- start start 2024-03-30T00:48:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e358d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 0x5616e358d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 0x5616e3589700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e303bc40 con 0x5616e357a290 2024-03-30T00:48:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e337a380 con 0x5616e35311b0 2024-03-30T00:48:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616e3382b90 con 0x5616e357abb0 2024-03-30T00:48:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7233e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 0x5616e358d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7233e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 0x5616e358d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44688/0 (socket says 172.21.15.66:44688) 2024-03-30T00:48:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7233e00640 1 -- 172.21.15.66:0/3204564289 learned_addr learned my addr 172.21.15.66:0/3204564289 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7239200640 1 --2- 172.21.15.66:0/3204564289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e358d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7233e00640 1 -- 172.21.15.66:0/3204564289 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 msgr2=0x5616e3589700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7233e00640 1 --2- 172.21.15.66:0/3204564289 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 0x5616e3589700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7233e00640 1 -- 172.21.15.66:0/3204564289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 msgr2=0x5616e358d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7233e00640 1 --2- 172.21.15.66:0/3204564289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e358d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7233e00640 1 -- 172.21.15.66:0/3204564289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e3579ce0 con 0x5616e357a290 2024-03-30T00:48:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7239200640 1 --2- 172.21.15.66:0/3204564289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e358d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7233e00640 1 --2- 172.21.15.66:0/3204564289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 0x5616e358d7a0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f722000a530 tx=0x7f7220029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c7482a3b8c6901 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7226000640 1 -- 172.21.15.66:0/3204564289 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722001f7b0 con 0x5616e357a290 2024-03-30T00:48:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7226000640 1 -- 172.21.15.66:0/3204564289 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7220018040 con 0x5616e357a290 2024-03-30T00:48:27.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f7226000640 1 -- 172.21.15.66:0/3204564289 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722001fa60 con 0x5616e357a290 2024-03-30T00:48:27.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 -- 172.21.15.66:0/3204564289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 msgr2=0x5616e358d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 --2- 172.21.15.66:0/3204564289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 0x5616e358d7a0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f722000a530 tx=0x7f7220029000 comp rx=0 tx=0).stop 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 -- 172.21.15.66:0/3204564289 shutdown_connections 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 --2- 172.21.15.66:0/3204564289 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 0x5616e3589700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 --2- 172.21.15.66:0/3204564289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e357a290 0x5616e358d7a0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 --2- 172.21.15.66:0/3204564289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e35311b0 0x5616e358d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 -- 172.21.15.66:0/3204564289 >> 172.21.15.66:0/3204564289 conn(0x5616e352d160 msgr2=0x5616e3530460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 -- 172.21.15.66:0/3204564289 shutdown_connections 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 -- 172.21.15.66:0/3204564289 wait complete. 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 -- start start 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.765+0000 7f723f0d8000 1 Processor -- start 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e35311b0 0x5616e358cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e357a290 0x5616e35d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 0x5616e35d4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e35285e0 con 0x5616e35311b0 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e337c3a0 con 0x5616e357a290 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616e33816f0 con 0x5616e357abb0 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 10 client.0 _ll_register_callbacks cb 0x5616e341c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f7239200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e35311b0 0x5616e358cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f7239200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e35311b0 0x5616e358cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44692/0 (socket says 172.21.15.66:44692) 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f7239200640 1 -- 172.21.15.66:0/3759090120 learned_addr learned my addr 172.21.15.66:0/3759090120 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f7239200640 1 -- 172.21.15.66:0/3759090120 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 msgr2=0x5616e35d4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f7239200640 1 --2- 172.21.15.66:0/3759090120 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 0x5616e35d4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f7239200640 1 -- 172.21.15.66:0/3759090120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e357a290 msgr2=0x5616e35d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f7239200640 1 --2- 172.21.15.66:0/3759090120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e357a290 0x5616e35d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f7239200640 1 -- 172.21.15.66:0/3759090120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616e3343ba0 con 0x5616e35311b0 2024-03-30T00:48:27.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f7239200640 1 --2- 172.21.15.66:0/3759090120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e35311b0 0x5616e358cd10 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7228051b80 tx=0x7f7228051af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c500d79060a24da server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f721a000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f721a000640 1 -- 172.21.15.66:0/3759090120 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228053880 con 0x5616e35311b0 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f721a000640 1 -- 172.21.15.66:0/3759090120 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f7228015d40 con 0x5616e35311b0 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 -- 172.21.15.66:0/3759090120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616e332e620 con 0x5616e35311b0 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f721a000640 1 -- 172.21.15.66:0/3759090120 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7228051d90 con 0x5616e35311b0 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 10 client.8433 fetch_fsmap learned FSMap version 71 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 -- 172.21.15.66:0/3759090120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x5616e3329ef0 con 0x5616e35311b0 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f721a000640 1 -- 172.21.15.66:0/3759090120 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5616e332e620 con 0x5616e35311b0 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f721a000640 1 -- 172.21.15.66:0/3759090120 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f72280590c0 con 0x5616e35311b0 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 10 client.8433 fetch_fsmap finished waiting for FSMap version 71 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 10 client.8433 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 20 client.8433 populate_metadata read hostname 'smithi066' 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 --2- 172.21.15.66:0/3759090120 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x5616e35d5380 0x5616e35d5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 4 client.8433 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-30T00:48:27.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 -- 172.21.15.66:0/3759090120 --> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5616e337d4e0 con 0x5616e35d5380 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f7233e00640 1 --2- 172.21.15.66:0/3759090120 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x5616e35d5380 0x5616e35d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f7233e00640 1 --2- 172.21.15.66:0/3759090120 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x5616e35d5380 0x5616e35d5760 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74b12645e1cca22d server_cookie=b7d642ffc83b27c3 in_seq=0 out_seq=0 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f721a000640 10 client.8433 ms_handle_connect on v2:172.21.15.66:6835/2419209461 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f721a000640 1 -- 172.21.15.66:0/3759090120 <== mds.0 v2:172.21.15.66:6835/2419209461 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5616e337d4e0 con 0x5616e35d5380 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f721a000640 10 client.8433 handle_command_reply: tid=0 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 client.8433 shutdown 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 -- 172.21.15.66:0/3759090120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e35311b0 msgr2=0x5616e358cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 --2- 172.21.15.66:0/3759090120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e35311b0 0x5616e358cd10 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7228051b80 tx=0x7f7228051af0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 -- 172.21.15.66:0/3759090120 shutdown_connections 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 --2- 172.21.15.66:0/3759090120 >> [v2:172.21.15.66:6835/2419209461,v1:172.21.15.66:6836/2419209461] conn(0x5616e35d5380 0x5616e35d5760 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 --2- 172.21.15.66:0/3759090120 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5616e357abb0 0x5616e35d4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 --2- 172.21.15.66:0/3759090120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616e357a290 0x5616e35d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f721a000640 0 client.8433 ms_handle_reset on v2:172.21.15.66:6835/2419209461 2024-03-30T00:48:27.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 --2- 172.21.15.66:0/3759090120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5616e35311b0 0x5616e358cd10 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 -- 172.21.15.66:0/3759090120 >> 172.21.15.66:0/3759090120 conn(0x5616e352d160 msgr2=0x5616e3535660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 -- 172.21.15.66:0/3759090120 shutdown_connections 2024-03-30T00:48:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 1 -- 172.21.15.66:0/3759090120 wait complete. 2024-03-30T00:48:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723f0d8000 20 client.8433 trim_cache size 0 max 16384 2024-03-30T00:48:27.785 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:27.785 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2024-03-30T00:48:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723bc00640 1 -- 172.21.15.66:0/1237681850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7214066690 msgr2=0x7f7214068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.769+0000 7f723bc00640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7214066690 0x7f7214068b50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f72341f8430 tx=0x7f7220008040 comp rx=0 tx=0).stop 2024-03-30T00:48:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.773+0000 7f723bc00640 1 -- 172.21.15.66:0/1237681850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72340cabb0 msgr2=0x7f72340cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.773+0000 7f723bc00640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72340cabb0 0x7f72340cd8b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f722800d5f0 tx=0x7f722800dab0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.773+0000 7f723bc00640 1 -- 172.21.15.66:0/1237681850 shutdown_connections 2024-03-30T00:48:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.773+0000 7f723bc00640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f723415a640 0x7f7234071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.773+0000 7f723bc00640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72340ce520 0x7f7234097030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.773+0000 7f723bc00640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7214066690 0x7f7214068b50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.773+0000 7f723bc00640 1 --2- 172.21.15.66:0/1237681850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72340cabb0 0x7f72340cd8b0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.773+0000 7f723bc00640 1 -- 172.21.15.66:0/1237681850 >> 172.21.15.66:0/1237681850 conn(0x7f7234069f60 msgr2=0x7f723415b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.773+0000 7f723bc00640 1 -- 172.21.15.66:0/1237681850 shutdown_connections 2024-03-30T00:48:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.773+0000 7f723bc00640 1 -- 172.21.15.66:0/1237681850 wait complete. 2024-03-30T00:48:27.800 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-30T00:48:27.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.929+0000 7f4816200640 1 Processor -- start 2024-03-30T00:48:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.929+0000 7f4816200640 1 -- start start 2024-03-30T00:48:27.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.929+0000 7f4816200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f48100ce520 0x7f48100ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.929+0000 7f4816200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48100cabb0 0x7f48100cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.929+0000 7f4816200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481006d070 con 0x7f481015a640 2024-03-30T00:48:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.929+0000 7f4816200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48100cbba0 con 0x7f48100ce520 2024-03-30T00:48:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.929+0000 7f4816200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48100cee70 con 0x7f48100cabb0 2024-03-30T00:48:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.933+0000 7f4806a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f48100ce520 0x7f48100ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.933+0000 7f4806a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f48100ce520 0x7f48100ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44702/0 (socket says 172.21.15.66:44702) 2024-03-30T00:48:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.933+0000 7f4806a00640 1 -- 172.21.15.66:0/4001108885 learned_addr learned my addr 172.21.15.66:0/4001108885 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.933+0000 7f480fe00640 1 --2- 172.21.15.66:0/4001108885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48100cabb0 0x7f48100cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.933+0000 7f4806a00640 1 -- 172.21.15.66:0/4001108885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48100cabb0 msgr2=0x7f48100cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.933+0000 7f4806a00640 1 --2- 172.21.15.66:0/4001108885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48100cabb0 0x7f48100cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.933+0000 7f4806a00640 1 -- 172.21.15.66:0/4001108885 >> v1:172.21.15.156:6789/0 conn(0x7f481015a640 legacy=0x7f481006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.933+0000 7f4806a00640 1 -- 172.21.15.66:0/4001108885 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100cb8c0 con 0x7f48100ce520 2024-03-30T00:48:27.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.933+0000 7f4806a00640 1 --2- 172.21.15.66:0/4001108885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48100ce520 0x7f48100ce930 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009e50 tx=0x7f47f801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a67d22d42927394 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.933+0000 7f480ea00640 1 -- 172.21.15.66:0/4001108885 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80a58d0 con 0x7f48100ce520 2024-03-30T00:48:27.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f480ea00640 1 -- 172.21.15.66:0/4001108885 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f80a5a30 con 0x7f48100ce520 2024-03-30T00:48:27.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f480ea00640 1 -- 172.21.15.66:0/4001108885 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80a5dc0 con 0x7f48100ce520 2024-03-30T00:48:27.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 -- 172.21.15.66:0/4001108885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48100ce520 msgr2=0x7f48100ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 --2- 172.21.15.66:0/4001108885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48100ce520 0x7f48100ce930 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009e50 tx=0x7f47f801ca00 comp rx=0 tx=0).stop 2024-03-30T00:48:27.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 -- 172.21.15.66:0/4001108885 shutdown_connections 2024-03-30T00:48:27.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 --2- 172.21.15.66:0/4001108885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48100cabb0 0x7f48100cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 --2- 172.21.15.66:0/4001108885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48100ce520 0x7f48100ce930 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 -- 172.21.15.66:0/4001108885 >> 172.21.15.66:0/4001108885 conn(0x7f4810069f60 msgr2=0x7f481006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:27.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 -- 172.21.15.66:0/4001108885 shutdown_connections 2024-03-30T00:48:27.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 -- 172.21.15.66:0/4001108885 wait complete. 2024-03-30T00:48:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 Processor -- start 2024-03-30T00:48:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 -- start start 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48100cabb0 0x7f48100cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48100ce520 0x7f48100c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f481015a640 0x7f48100c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100c4c90 con 0x7f48100ce520 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4810162410 con 0x7f481015a640 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4816200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48100cd6f0 con 0x7f48100cabb0 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4806a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48100ce520 0x7f48100c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4806a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48100ce520 0x7f48100c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44704/0 (socket says 172.21.15.66:44704) 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f4806a00640 1 -- 172.21.15.66:0/1833425589 learned_addr learned my addr 172.21.15.66:0/1833425589 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f480f400640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48100cabb0 0x7f48100cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.937+0000 7f480fe00640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f481015a640 0x7f48100c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4806a00640 1 -- 172.21.15.66:0/1833425589 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48100cabb0 msgr2=0x7f48100cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4806a00640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48100cabb0 0x7f48100cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4806a00640 1 -- 172.21.15.66:0/1833425589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f481015a640 msgr2=0x7f48100c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:27.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4806a00640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f481015a640 0x7f48100c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:27.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4806a00640 1 -- 172.21.15.66:0/1833425589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f8009b30 con 0x7f48100ce520 2024-03-30T00:48:27.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4806a00640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48100ce520 0x7f48100c4750 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f47f80af040 tx=0x7f47f80197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4807e00640 1 -- 172.21.15.66:0/1833425589 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80b8880 con 0x7f48100ce520 2024-03-30T00:48:27.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4807e00640 1 -- 172.21.15.66:0/1833425589 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f80b8a20 con 0x7f48100ce520 2024-03-30T00:48:27.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4807e00640 1 -- 172.21.15.66:0/1833425589 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80b8dd0 con 0x7f48100ce520 2024-03-30T00:48:27.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4816200640 1 -- 172.21.15.66:0/1833425589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48100cd9d0 con 0x7f48100ce520 2024-03-30T00:48:27.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4816200640 1 -- 172.21.15.66:0/1833425589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4810162660 con 0x7f48100ce520 2024-03-30T00:48:27.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4816200640 1 -- 172.21.15.66:0/1833425589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4810066c30 con 0x7f48100ce520 2024-03-30T00:48:27.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4807e00640 1 -- 172.21.15.66:0/1833425589 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f47f80b6440 con 0x7f48100ce520 2024-03-30T00:48:27.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4807e00640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47ec066690 0x7f47ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:27.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f4807e00640 1 -- 172.21.15.66:0/1833425589 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f47f811faa0 con 0x7f48100ce520 2024-03-30T00:48:27.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.941+0000 7f480f400640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47ec066690 0x7f47ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:27.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.945+0000 7f480f400640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47ec066690 0x7f47ec068b50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4800007fd0 tx=0x7f4800009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:27.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:27.945+0000 7f4807e00640 1 -- 172.21.15.66:0/1833425589 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f47f80eceb0 con 0x7f48100ce520 2024-03-30T00:48:28.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 -- 172.21.15.66:0/1833425589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f481006a720 con 0x7f48100ce520 2024-03-30T00:48:28.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4807e00640 1 -- 172.21.15.66:0/1833425589 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f47f811f040 con 0x7f48100ce520 2024-03-30T00:48:28.150 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:28.150 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-30T00:48:28.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 -- 172.21.15.66:0/1833425589 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47ec066690 msgr2=0x7f47ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47ec066690 0x7f47ec068b50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4800007fd0 tx=0x7f4800009040 comp rx=0 tx=0).stop 2024-03-30T00:48:28.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 -- 172.21.15.66:0/1833425589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48100ce520 msgr2=0x7f48100c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48100ce520 0x7f48100c4750 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f47f80af040 tx=0x7f47f80197a0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 -- 172.21.15.66:0/1833425589 shutdown_connections 2024-03-30T00:48:28.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f481015a640 0x7f48100c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f47ec066690 0x7f47ec068b50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48100ce520 0x7f48100c4750 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 --2- 172.21.15.66:0/1833425589 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48100cabb0 0x7f48100cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 -- 172.21.15.66:0/1833425589 >> 172.21.15.66:0/1833425589 conn(0x7f4810069f60 msgr2=0x7f4810072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:28.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 -- 172.21.15.66:0/1833425589 shutdown_connections 2024-03-30T00:48:28.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.141+0000 7f4816200640 1 -- 172.21.15.66:0/1833425589 wait complete. 2024-03-30T00:48:28.171 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-30T00:48:28.172 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:48:28.172 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:48:28.268 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:48:28.356 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:48:28.357 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:48:28.357 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:48:28.366 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:48:28.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.537+0000 7f495fe00640 1 Processor -- start 2024-03-30T00:48:28.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.537+0000 7f495fe00640 1 -- start start 2024-03-30T00:48:28.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f495fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f49580cde70 0x7f495815a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f495fe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49580cd3e0 con 0x7f49580cec30 2024-03-30T00:48:28.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f495fe00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495815a990 con 0x7f49580cda90 2024-03-30T00:48:28.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f495fe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49580b1b80 con 0x7f49580cde70 2024-03-30T00:48:28.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f495de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f49580cde70 0x7f495815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:28.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f495de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f49580cde70 0x7f495815a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44706/0 (socket says 172.21.15.66:44706) 2024-03-30T00:48:28.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f495de00640 1 -- 172.21.15.66:0/1195959871 learned_addr learned my addr 172.21.15.66:0/1195959871 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:28.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f495ca00640 1 --1- 172.21.15.66:0/1195959871 >> v1:172.21.15.66:6789/0 conn(0x7f49580cec30 0x7f4958159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45296/0 (socket says 172.21.15.66:45296) 2024-03-30T00:48:28.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f495de00640 1 -- 172.21.15.66:0/1195959871 >> v1:172.21.15.156:6789/0 conn(0x7f49580cda90 legacy=0x7f49580ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f495de00640 1 -- 172.21.15.66:0/1195959871 >> v1:172.21.15.66:6789/0 conn(0x7f49580cec30 legacy=0x7f4958159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f495de00640 1 -- 172.21.15.66:0/1195959871 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49580b11e0 con 0x7f49580cde70 2024-03-30T00:48:28.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.541+0000 7f4957e00640 1 -- 172.21.15.66:0/1195959871 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679133543 0 0) 0x7f49580cd3e0 con 0x7f49580cec30 2024-03-30T00:48:28.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f495de00640 1 --2- 172.21.15.66:0/1195959871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49580cde70 0x7f495815a370 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f4950009e50 tx=0x7f495001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192edaca41813985 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:28.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f4957e00640 1 -- 172.21.15.66:0/1195959871 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500a5860 con 0x7f49580cde70 2024-03-30T00:48:28.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f4957e00640 1 -- 172.21.15.66:0/1195959871 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49500a59c0 con 0x7f49580cde70 2024-03-30T00:48:28.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f4957e00640 1 -- 172.21.15.66:0/1195959871 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500a5d50 con 0x7f49580cde70 2024-03-30T00:48:28.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f495fe00640 1 -- 172.21.15.66:0/1195959871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49580cde70 msgr2=0x7f495815a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f495fe00640 1 --2- 172.21.15.66:0/1195959871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49580cde70 0x7f495815a370 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f4950009e50 tx=0x7f495001c9a0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f495fe00640 1 -- 172.21.15.66:0/1195959871 shutdown_connections 2024-03-30T00:48:28.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f495fe00640 1 --2- 172.21.15.66:0/1195959871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49580cde70 0x7f495815a370 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f495fe00640 1 -- 172.21.15.66:0/1195959871 >> 172.21.15.66:0/1195959871 conn(0x7f4958069f60 msgr2=0x7f495806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:28.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f495fe00640 1 -- 172.21.15.66:0/1195959871 shutdown_connections 2024-03-30T00:48:28.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f495fe00640 1 -- 172.21.15.66:0/1195959871 wait complete. 2024-03-30T00:48:28.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f495fe00640 1 Processor -- start 2024-03-30T00:48:28.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.545+0000 7f495fe00640 1 -- start start 2024-03-30T00:48:28.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49580cda90 0x7f4958162620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495fe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49580cde70 0x7f4958163f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49580cec30 0x7f4958095c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49581562e0 con 0x7f49580cda90 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495fe00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4958164450 con 0x7f49580cde70 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495fe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49580c3b70 con 0x7f49580cec30 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49580cda90 0x7f4958162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49580cda90 0x7f4958162620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44722/0 (socket says 172.21.15.66:44722) 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495d400640 1 -- 172.21.15.66:0/2990305287 learned_addr learned my addr 172.21.15.66:0/2990305287 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495d400640 1 -- 172.21.15.66:0/2990305287 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49580cec30 msgr2=0x7f4958095c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495d400640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49580cec30 0x7f4958095c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495d400640 1 -- 172.21.15.66:0/2990305287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49580cde70 msgr2=0x7f4958163f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495d400640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49580cde70 0x7f4958163f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495d400640 1 -- 172.21.15.66:0/2990305287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4950009b30 con 0x7f49580cda90 2024-03-30T00:48:28.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495d400640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49580cda90 0x7f4958162620 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f494c00c3c0 tx=0x7f494c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:28.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f4955600640 1 -- 172.21.15.66:0/2990305287 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c0052d0 con 0x7f49580cda90 2024-03-30T00:48:28.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f4955600640 1 -- 172.21.15.66:0/2990305287 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c005430 con 0x7f49580cda90 2024-03-30T00:48:28.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f4955600640 1 -- 172.21.15.66:0/2990305287 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c003840 con 0x7f49580cda90 2024-03-30T00:48:28.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495fe00640 1 -- 172.21.15.66:0/2990305287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4958162cb0 con 0x7f49580cda90 2024-03-30T00:48:28.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495fe00640 1 -- 172.21.15.66:0/2990305287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4958163270 con 0x7f49580cda90 2024-03-30T00:48:28.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f495fe00640 1 -- 172.21.15.66:0/2990305287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4958163490 con 0x7f49580cda90 2024-03-30T00:48:28.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.549+0000 7f4955600640 1 -- 172.21.15.66:0/2990305287 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f494c0039a0 con 0x7f49580cda90 2024-03-30T00:48:28.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.553+0000 7f4955600640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4938066690 0x7f4938068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.553+0000 7f4955600640 1 -- 172.21.15.66:0/2990305287 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f494c0839c0 con 0x7f49580cda90 2024-03-30T00:48:28.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.553+0000 7f495ca00640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4938066690 0x7f4938068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:28.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.553+0000 7f495ca00640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4938066690 0x7f4938068b50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4944000c10 tx=0x7f4944007910 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:28.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.557+0000 7f4955600640 1 -- 172.21.15.66:0/2990305287 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f494c050a30 con 0x7f49580cda90 2024-03-30T00:48:28.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.717+0000 7f495fe00640 1 -- 172.21.15.66:0/2990305287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4958066c30 con 0x7f49580cda90 2024-03-30T00:48:28.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f4955600640 1 -- 172.21.15.66:0/2990305287 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f494c017090 con 0x7f49580cda90 2024-03-30T00:48:28.730 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:28.730 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":98,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:47:49.264161+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:46:13.811742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-30T00:46:14.265142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":7}]},{"pgid":"14.1a","mappings":[{"from":1,"to":5}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/1626022080":"2024-03-31T00:47:49.264137+0000","172.21.15.66:6836/1626022080":"2024-03-31T00:47:49.264137+0000","172.21.15.66:6836/1147664160":"2024-03-31T00:46:34.103831+0000","172.21.15.66:6835/1147664160":"2024-03-31T00:46:34.103831+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:48:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 -- 172.21.15.66:0/2990305287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4938066690 msgr2=0x7f4938068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4938066690 0x7f4938068b50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4944000c10 tx=0x7f4944007910 comp rx=0 tx=0).stop 2024-03-30T00:48:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 -- 172.21.15.66:0/2990305287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49580cda90 msgr2=0x7f4958162620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49580cda90 0x7f4958162620 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f494c00c3c0 tx=0x7f494c00c880 comp rx=0 tx=0).stop 2024-03-30T00:48:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 -- 172.21.15.66:0/2990305287 shutdown_connections 2024-03-30T00:48:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49580cec30 0x7f4958095c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49580cde70 0x7f4958163f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4938066690 0x7f4938068b50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 --2- 172.21.15.66:0/2990305287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49580cda90 0x7f4958162620 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 -- 172.21.15.66:0/2990305287 >> 172.21.15.66:0/2990305287 conn(0x7f4958069f60 msgr2=0x7f495815b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 -- 172.21.15.66:0/2990305287 shutdown_connections 2024-03-30T00:48:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.721+0000 7f495fe00640 1 -- 172.21.15.66:0/2990305287 wait complete. 2024-03-30T00:48:28.750 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:46:13.811742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:46:14.265142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:48:28.751 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:48:28.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bda000640 1 Processor -- start 2024-03-30T00:48:28.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bda000640 1 -- start start 2024-03-30T00:48:28.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bda000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6bd415de30 0x7f6bd40cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bda000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd40cbb90 con 0x7f6bd40af420 2024-03-30T00:48:28.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bda000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd40ce1b0 con 0x7f6bd40cee50 2024-03-30T00:48:28.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bda000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd40afa10 con 0x7f6bd415de30 2024-03-30T00:48:28.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bcaa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6bd40af420 0x7f6bd40ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45808/0 (socket says 172.21.15.66:45808) 2024-03-30T00:48:28.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bcaa00640 1 -- 172.21.15.66:0/2922842267 learned_addr learned my addr 172.21.15.66:0/2922842267 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:28.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bd3400640 1 --2- 172.21.15.66:0/2922842267 >> v2:172.21.15.156:3300/0 conn(0x7f6bd415de30 0x7f6bd40cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851529995 0 0) 0x7f6bd40cbb90 con 0x7f6bd40af420 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb8003530 con 0x7f6bd40af420 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 214552238 0 0) 0x7f6bb8003530 con 0x7f6bd40af420 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 >> v2:172.21.15.156:3300/0 conn(0x7f6bd415de30 msgr2=0x7f6bd40cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bd2a00640 1 --2- 172.21.15.66:0/2922842267 >> v2:172.21.15.156:3300/0 conn(0x7f6bd415de30 0x7f6bd40cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 >> v1:172.21.15.156:6789/0 conn(0x7f6bd40cee50 legacy=0x7f6bd406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd40cf430 con 0x7f6bd40af420 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.885+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f6bbc0016a0 con 0x7f6bd40af420 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 >> v1:172.21.15.66:6789/0 conn(0x7f6bd40af420 legacy=0x7f6bd40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd2a00640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 0x7f6bb8003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd2a00640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb8004380 0x7f6bb8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd2a00640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bb80050f0 0x7f6bb800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd40afa10 con 0x7f6bb80050f0 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb800c030 con 0x7f6bb8004380 2024-03-30T00:48:28.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb800c1f0 con 0x7f6bb8003980 2024-03-30T00:48:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd3e00640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bb80050f0 0x7f6bb800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bcaa00640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb8004380 0x7f6bb8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd3400640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 0x7f6bb8003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd3400640 1 -- 172.21.15.66:0/2922842267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb8004380 msgr2=0x7f6bb8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd3400640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb8004380 0x7f6bb8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd3400640 1 -- 172.21.15.66:0/2922842267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bb80050f0 msgr2=0x7f6bb800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd3400640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bb80050f0 0x7f6bb800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd3400640 1 -- 172.21.15.66:0/2922842267 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb8003530 con 0x7f6bb8003980 2024-03-30T00:48:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd3e00640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bb80050f0 0x7f6bb800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:28.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.889+0000 7f6bd3e00640 1 -- 172.21.15.66:0/2922842267 reap_dead start 2024-03-30T00:48:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bd3400640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 0x7f6bb8003d60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009ae0 tx=0x7f6bc401ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c904332b41692a1c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40a5550 con 0x7f6bb8003980 2024-03-30T00:48:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc40a56b0 con 0x7f6bb8003980 2024-03-30T00:48:28.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bd2a00640 1 -- 172.21.15.66:0/2922842267 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40a5a60 con 0x7f6bb8003980 2024-03-30T00:48:28.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 -- 172.21.15.66:0/2922842267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 msgr2=0x7f6bb8003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 0x7f6bb8003d60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009ae0 tx=0x7f6bc401ca70 comp rx=0 tx=0).stop 2024-03-30T00:48:28.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 -- 172.21.15.66:0/2922842267 shutdown_connections 2024-03-30T00:48:28.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 --2- 172.21.15.66:0/2922842267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 0x7f6bb8003d60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 -- 172.21.15.66:0/2922842267 >> 172.21.15.66:0/2922842267 conn(0x7f6bd40d0540 msgr2=0x7f6bd40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:28.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 -- 172.21.15.66:0/2922842267 shutdown_connections 2024-03-30T00:48:28.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 -- 172.21.15.66:0/2922842267 wait complete. 2024-03-30T00:48:28.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 Processor -- start 2024-03-30T00:48:28.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 -- start start 2024-03-30T00:48:28.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 0x7f6bd4159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd40b3160 0x7f6bd40b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd40b24f0 0x7f6bd406bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd40b2970 con 0x7f6bd40b24f0 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd40c46b0 con 0x7f6bd40b3160 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bda000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd40b2e30 con 0x7f6bb8003980 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.893+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd40b24f0 0x7f6bd406bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd40b24f0 0x7f6bd406bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48370/0 (socket says 172.21.15.66:48370) 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bd3e00640 1 -- 172.21.15.66:0/1224361587 learned_addr learned my addr 172.21.15.66:0/1224361587 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bcaa00640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd40b3160 0x7f6bd40b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bd3400640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 0x7f6bd4159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bd3e00640 1 -- 172.21.15.66:0/1224361587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 msgr2=0x7f6bd4159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bd3e00640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 0x7f6bd4159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bd3e00640 1 -- 172.21.15.66:0/1224361587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd40b3160 msgr2=0x7f6bd40b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bd3e00640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd40b3160 0x7f6bd40b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bd3e00640 1 -- 172.21.15.66:0/1224361587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd40b2b50 con 0x7f6bd40b24f0 2024-03-30T00:48:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bcaa00640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd40b3160 0x7f6bd40b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bd3400640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 0x7f6bd4159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bd3e00640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd40b24f0 0x7f6bd406bef0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f6bd406ad40 tx=0x7f6bcc012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bcbe00640 1 -- 172.21.15.66:0/1224361587 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc007390 con 0x7f6bd40b24f0 2024-03-30T00:48:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bcbe00640 1 -- 172.21.15.66:0/1224361587 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc010b20 con 0x7f6bd40b24f0 2024-03-30T00:48:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bcbe00640 1 -- 172.21.15.66:0/1224361587 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc010ed0 con 0x7f6bd40b24f0 2024-03-30T00:48:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bda000640 1 -- 172.21.15.66:0/1224361587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd40c4060 con 0x7f6bd40b24f0 2024-03-30T00:48:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bda000640 1 -- 172.21.15.66:0/1224361587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd415aac0 con 0x7f6bd40b24f0 2024-03-30T00:48:28.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bda000640 1 -- 172.21.15.66:0/1224361587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd40cdf60 con 0x7f6bd40b24f0 2024-03-30T00:48:28.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.897+0000 7f6bcbe00640 1 -- 172.21.15.66:0/1224361587 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6bcc016480 con 0x7f6bd40b24f0 2024-03-30T00:48:28.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.901+0000 7f6bcbe00640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bb00666c0 0x7f6bb0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:28.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.901+0000 7f6bcbe00640 1 -- 172.21.15.66:0/1224361587 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f6bcc087960 con 0x7f6bd40b24f0 2024-03-30T00:48:28.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.901+0000 7f6bd3400640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bb00666c0 0x7f6bb0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:28.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.901+0000 7f6bd3400640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bb00666c0 0x7f6bb0068b80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40af000 tx=0x7f6bc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:28.905+0000 7f6bcbe00640 1 -- 172.21.15.66:0/1224361587 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6bcc054a60 con 0x7f6bd40b24f0 2024-03-30T00:48:29.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.061+0000 7f6bda000640 1 -- 172.21.15.66:0/1224361587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6bd40c4840 con 0x7f6bd40b24f0 2024-03-30T00:48:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.061+0000 7f6bcbe00640 1 -- 172.21.15.66:0/1224361587 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6bcc058080 con 0x7f6bd40b24f0 2024-03-30T00:48:29.073 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:48:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 -- 172.21.15.66:0/1224361587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bb00666c0 msgr2=0x7f6bb0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bb00666c0 0x7f6bb0068b80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40af000 tx=0x7f6bc4008040 comp rx=0 tx=0).stop 2024-03-30T00:48:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 -- 172.21.15.66:0/1224361587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd40b24f0 msgr2=0x7f6bd406bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd40b24f0 0x7f6bd406bef0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f6bd406ad40 tx=0x7f6bcc012d00 comp rx=0 tx=0).stop 2024-03-30T00:48:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 -- 172.21.15.66:0/1224361587 shutdown_connections 2024-03-30T00:48:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bb00666c0 0x7f6bb0068b80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bd40b24f0 0x7f6bd406bef0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd40b3160 0x7f6bd40b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 --2- 172.21.15.66:0/1224361587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bb8003980 0x7f6bd4159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 -- 172.21.15.66:0/1224361587 >> 172.21.15.66:0/1224361587 conn(0x7f6bd40d0540 msgr2=0x7f6bd40adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 -- 172.21.15.66:0/1224361587 shutdown_connections 2024-03-30T00:48:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.065+0000 7f6bda000640 1 -- 172.21.15.66:0/1224361587 wait complete. 2024-03-30T00:48:29.095 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:48:29.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46fd200640 1 Processor -- start 2024-03-30T00:48:29.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46fd200640 1 -- start start 2024-03-30T00:48:29.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46fd200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f46f8099060 0x7f46f80aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46fd200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46f80b0470 0x7f46f80cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46fd200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f8099760 con 0x7f46f8098c80 2024-03-30T00:48:29.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46fd200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f80cb8f0 con 0x7f46f8099060 2024-03-30T00:48:29.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46fd200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f80999b0 con 0x7f46f80b0470 2024-03-30T00:48:29.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46ed400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f46f8099060 0x7f46f80aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46ed400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f46f8099060 0x7f46f80aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48372/0 (socket says 172.21.15.66:48372) 2024-03-30T00:48:29.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46ed400640 1 -- 172.21.15.66:0/577140777 learned_addr learned my addr 172.21.15.66:0/577140777 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:29.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46f6800640 1 --2- 172.21.15.66:0/577140777 >> v2:172.21.15.156:3300/0 conn(0x7f46f80b0470 0x7f46f80cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46ed400640 1 -- 172.21.15.66:0/577140777 >> v2:172.21.15.156:3300/0 conn(0x7f46f80b0470 msgr2=0x7f46f80cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:29.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.229+0000 7f46ed400640 1 --2- 172.21.15.66:0/577140777 >> v2:172.21.15.156:3300/0 conn(0x7f46f80b0470 0x7f46f80cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46ed400640 1 -- 172.21.15.66:0/577140777 >> v1:172.21.15.156:6789/0 conn(0x7f46f8098c80 legacy=0x7f46f80ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46ed400640 1 -- 172.21.15.66:0/577140777 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f8099c90 con 0x7f46f8099060 2024-03-30T00:48:29.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46f6800640 1 --2- 172.21.15.66:0/577140777 >> v2:172.21.15.156:3300/0 conn(0x7f46f80b0470 0x7f46f80cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:29.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46ed400640 1 --2- 172.21.15.66:0/577140777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46f8099060 0x7f46f80aff30 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f46f80721a0 tx=0x7f46e001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bb43f56fec15b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:29.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46f5400640 1 -- 172.21.15.66:0/577140777 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e00ab600 con 0x7f46f8099060 2024-03-30T00:48:29.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46f5400640 1 -- 172.21.15.66:0/577140777 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e00ab760 con 0x7f46f8099060 2024-03-30T00:48:29.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46f5400640 1 -- 172.21.15.66:0/577140777 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e00abaf0 con 0x7f46f8099060 2024-03-30T00:48:29.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46fd200640 1 -- 172.21.15.66:0/577140777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46f8099060 msgr2=0x7f46f80aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46fd200640 1 --2- 172.21.15.66:0/577140777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46f8099060 0x7f46f80aff30 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f46f80721a0 tx=0x7f46e001c4f0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46fd200640 1 -- 172.21.15.66:0/577140777 shutdown_connections 2024-03-30T00:48:29.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46fd200640 1 --2- 172.21.15.66:0/577140777 >> v2:172.21.15.156:3300/0 conn(0x7f46f80b0470 0x7f46f80cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46fd200640 1 --2- 172.21.15.66:0/577140777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46f8099060 0x7f46f80aff30 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46fd200640 1 -- 172.21.15.66:0/577140777 >> 172.21.15.66:0/577140777 conn(0x7f46f80b20c0 msgr2=0x7f46f80b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46fd200640 1 -- 172.21.15.66:0/577140777 shutdown_connections 2024-03-30T00:48:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.233+0000 7f46fd200640 1 -- 172.21.15.66:0/577140777 wait complete. 2024-03-30T00:48:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46fd200640 1 Processor -- start 2024-03-30T00:48:29.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46fd200640 1 -- start start 2024-03-30T00:48:29.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46fd200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46f8098c80 0x7f46f80c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46fd200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46f8099060 0x7f46f80ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46fd200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46f80b0470 0x7f46f80aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46fd200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80c6cf0 con 0x7f46f8098c80 2024-03-30T00:48:29.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46fd200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80c60e0 con 0x7f46f80b0470 2024-03-30T00:48:29.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46fd200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80c63c0 con 0x7f46f8099060 2024-03-30T00:48:29.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46f8098c80 0x7f46f80c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46f8098c80 0x7f46f80c59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48386/0 (socket says 172.21.15.66:48386) 2024-03-30T00:48:29.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46f5e00640 1 -- 172.21.15.66:0/956313726 learned_addr learned my addr 172.21.15.66:0/956313726 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:29.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46ed400640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46f8099060 0x7f46f80ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46f6800640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46f80b0470 0x7f46f80aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46f5e00640 1 -- 172.21.15.66:0/956313726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46f8099060 msgr2=0x7f46f80ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46f5e00640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46f8099060 0x7f46f80ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46f5e00640 1 -- 172.21.15.66:0/956313726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46f80b0470 msgr2=0x7f46f80aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46f5e00640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46f80b0470 0x7f46f80aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46f5e00640 1 -- 172.21.15.66:0/956313726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00094d0 con 0x7f46f8098c80 2024-03-30T00:48:29.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46f6800640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46f80b0470 0x7f46f80aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:29.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46f5e00640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46f8098c80 0x7f46f80c59e0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c3c0 tx=0x7f46e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:29.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46eea00640 1 -- 172.21.15.66:0/956313726 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e80052d0 con 0x7f46f8098c80 2024-03-30T00:48:29.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46eea00640 1 -- 172.21.15.66:0/956313726 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e8005430 con 0x7f46f8098c80 2024-03-30T00:48:29.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46eea00640 1 -- 172.21.15.66:0/956313726 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8003840 con 0x7f46f8098c80 2024-03-30T00:48:29.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46fd200640 1 -- 172.21.15.66:0/956313726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f80c66a0 con 0x7f46f8098c80 2024-03-30T00:48:29.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.237+0000 7f46fd200640 1 -- 172.21.15.66:0/956313726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f80c4a50 con 0x7f46f8098c80 2024-03-30T00:48:29.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.241+0000 7f46fd200640 1 -- 172.21.15.66:0/956313726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f8066c30 con 0x7f46f8098c80 2024-03-30T00:48:29.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.241+0000 7f46eea00640 1 -- 172.21.15.66:0/956313726 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f46e80039a0 con 0x7f46f8098c80 2024-03-30T00:48:29.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.241+0000 7f46eea00640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46d4066690 0x7f46d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.241+0000 7f46eea00640 1 -- 172.21.15.66:0/956313726 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f46e80838f0 con 0x7f46f8098c80 2024-03-30T00:48:29.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.241+0000 7f46ed400640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46d4066690 0x7f46d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.241+0000 7f46ed400640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46d4066690 0x7f46d4068b50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f46e00af000 tx=0x7f46e00b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:29.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.245+0000 7f46eea00640 1 -- 172.21.15.66:0/956313726 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f46e8050a10 con 0x7f46f8098c80 2024-03-30T00:48:29.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.409+0000 7f46fd200640 1 -- 172.21.15.66:0/956313726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f46f80b2d20 con 0x7f46f8098c80 2024-03-30T00:48:29.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.409+0000 7f46eea00640 1 -- 172.21.15.66:0/956313726 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46e8054030 con 0x7f46f8098c80 2024-03-30T00:48:29.421 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:48:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 -- 172.21.15.66:0/956313726 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46d4066690 msgr2=0x7f46d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46d4066690 0x7f46d4068b50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f46e00af000 tx=0x7f46e00b6040 comp rx=0 tx=0).stop 2024-03-30T00:48:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 -- 172.21.15.66:0/956313726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46f8098c80 msgr2=0x7f46f80c59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46f8098c80 0x7f46f80c59e0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c3c0 tx=0x7f46e800c880 comp rx=0 tx=0).stop 2024-03-30T00:48:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 -- 172.21.15.66:0/956313726 shutdown_connections 2024-03-30T00:48:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46f80b0470 0x7f46f80aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46f8099060 0x7f46f80ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46d4066690 0x7f46d4068b50 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 --2- 172.21.15.66:0/956313726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46f8098c80 0x7f46f80c59e0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 -- 172.21.15.66:0/956313726 >> 172.21.15.66:0/956313726 conn(0x7f46f80b20c0 msgr2=0x7f46f80ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 -- 172.21.15.66:0/956313726 shutdown_connections 2024-03-30T00:48:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.413+0000 7f46fd200640 1 -- 172.21.15.66:0/956313726 wait complete. 2024-03-30T00:48:29.442 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:48:29.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.573+0000 7fb53cc00640 1 Processor -- start 2024-03-30T00:48:29.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.573+0000 7fb53cc00640 1 -- start start 2024-03-30T00:48:29.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb53cc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5380ce520 0x7fb5380ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb53cc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb53806d070 con 0x7fb5380cabb0 2024-03-30T00:48:29.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb53cc00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5380cbba0 con 0x7fb53815a640 2024-03-30T00:48:29.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb53cc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5380cee40 con 0x7fb5380ce520 2024-03-30T00:48:29.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb536800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb5380cabb0 0x7fb5380cb000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45820/0 (socket says 172.21.15.66:45820) 2024-03-30T00:48:29.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb536800640 1 -- 172.21.15.66:0/2461289043 learned_addr learned my addr 172.21.15.66:0/2461289043 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb52fe00640 1 --2- 172.21.15.66:0/2461289043 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5380ce520 0x7fb5380ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb535400640 1 -- 172.21.15.66:0/2461289043 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069976473 0 0) 0x7fb53806d070 con 0x7fb5380cabb0 2024-03-30T00:48:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb535400640 1 -- 172.21.15.66:0/2461289043 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb51c003530 con 0x7fb5380cabb0 2024-03-30T00:48:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb52fe00640 1 -- 172.21.15.66:0/2461289043 >> v1:172.21.15.156:6789/0 conn(0x7fb53815a640 legacy=0x7fb53806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb535400640 1 -- 172.21.15.66:0/2461289043 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405531837 0 0) 0x7fb5380cbba0 con 0x7fb53815a640 2024-03-30T00:48:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb52fe00640 1 -- 172.21.15.66:0/2461289043 >> v1:172.21.15.66:6789/0 conn(0x7fb5380cabb0 legacy=0x7fb5380cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.577+0000 7fb52fe00640 1 -- 172.21.15.66:0/2461289043 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5380cb8c0 con 0x7fb5380ce520 2024-03-30T00:48:29.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb52fe00640 1 --2- 172.21.15.66:0/2461289043 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5380ce520 0x7fb5380ce900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb5200097f0 tx=0x7fb52001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95608241108f4e8e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:29.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb535400640 1 -- 172.21.15.66:0/2461289043 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5200ac630 con 0x7fb5380ce520 2024-03-30T00:48:29.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb535400640 1 -- 172.21.15.66:0/2461289043 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5200ac790 con 0x7fb5380ce520 2024-03-30T00:48:29.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb535400640 1 -- 172.21.15.66:0/2461289043 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5200acb20 con 0x7fb5380ce520 2024-03-30T00:48:29.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb53cc00640 1 -- 172.21.15.66:0/2461289043 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5380ce520 msgr2=0x7fb5380ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb53cc00640 1 --2- 172.21.15.66:0/2461289043 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5380ce520 0x7fb5380ce900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb5200097f0 tx=0x7fb52001c4e0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb53cc00640 1 -- 172.21.15.66:0/2461289043 shutdown_connections 2024-03-30T00:48:29.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb53cc00640 1 --2- 172.21.15.66:0/2461289043 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5380ce520 0x7fb5380ce900 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb53cc00640 1 -- 172.21.15.66:0/2461289043 >> 172.21.15.66:0/2461289043 conn(0x7fb538069f60 msgr2=0x7fb53806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:29.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb53cc00640 1 -- 172.21.15.66:0/2461289043 shutdown_connections 2024-03-30T00:48:29.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb53cc00640 1 -- 172.21.15.66:0/2461289043 wait complete. 2024-03-30T00:48:29.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb53cc00640 1 Processor -- start 2024-03-30T00:48:29.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.581+0000 7fb53cc00640 1 -- start start 2024-03-30T00:48:29.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb53cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5380cabb0 0x7fb5380ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb53cc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5380ce520 0x7fb5380b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb53cc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb53815a640 0x7fb538155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb53cc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb538154e40 con 0x7fb5380cabb0 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb53cc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5381569b0 con 0x7fb5380ce520 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb53cc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb538156c90 con 0x7fb53815a640 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb535e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5380cabb0 0x7fb5380ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb535e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5380cabb0 0x7fb5380ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48398/0 (socket says 172.21.15.66:48398) 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb535e00640 1 -- 172.21.15.66:0/132892176 learned_addr learned my addr 172.21.15.66:0/132892176 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb52fe00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5380ce520 0x7fb5380b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb536800640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb53815a640 0x7fb538155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb535e00640 1 -- 172.21.15.66:0/132892176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb53815a640 msgr2=0x7fb538155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb535e00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb53815a640 0x7fb538155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb535e00640 1 -- 172.21.15.66:0/132892176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5380ce520 msgr2=0x7fb5380b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb535e00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5380ce520 0x7fb5380b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb535e00640 1 -- 172.21.15.66:0/132892176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520009420 con 0x7fb5380cabb0 2024-03-30T00:48:29.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb535e00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5380cabb0 0x7fb5380ca2f0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb528001ec0 tx=0x7fb52800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:29.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb52e000640 1 -- 172.21.15.66:0/132892176 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5280054e0 con 0x7fb5380cabb0 2024-03-30T00:48:29.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb52e000640 1 -- 172.21.15.66:0/132892176 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb528003600 con 0x7fb5380cabb0 2024-03-30T00:48:29.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb53cc00640 1 -- 172.21.15.66:0/132892176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb538156f70 con 0x7fb5380cabb0 2024-03-30T00:48:29.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb52e000640 1 -- 172.21.15.66:0/132892176 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5280039b0 con 0x7fb5380cabb0 2024-03-30T00:48:29.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb53cc00640 1 -- 172.21.15.66:0/132892176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5380ca830 con 0x7fb5380cabb0 2024-03-30T00:48:29.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.585+0000 7fb53cc00640 1 -- 172.21.15.66:0/132892176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb538066c30 con 0x7fb5380cabb0 2024-03-30T00:48:29.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.589+0000 7fb52e000640 1 -- 172.21.15.66:0/132892176 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb528003b10 con 0x7fb5380cabb0 2024-03-30T00:48:29.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.589+0000 7fb52e000640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb514066690 0x7fb514068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.589+0000 7fb52e000640 1 -- 172.21.15.66:0/132892176 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fb528083a70 con 0x7fb5380cabb0 2024-03-30T00:48:29.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.589+0000 7fb52fe00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb514066690 0x7fb514068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.589+0000 7fb52fe00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb514066690 0x7fb514068b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb5200ae000 tx=0x7fb5200a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:29.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.593+0000 7fb52e000640 1 -- 172.21.15.66:0/132892176 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb528050b00 con 0x7fb5380cabb0 2024-03-30T00:48:29.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.753+0000 7fb53cc00640 1 -- 172.21.15.66:0/132892176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb53806dd80 con 0x7fb5380cabb0 2024-03-30T00:48:29.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.753+0000 7fb52e000640 1 -- 172.21.15.66:0/132892176 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb528017030 con 0x7fb5380cabb0 2024-03-30T00:48:29.764 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:48:29.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 -- 172.21.15.66:0/132892176 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb514066690 msgr2=0x7fb514068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb514066690 0x7fb514068b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb5200ae000 tx=0x7fb5200a7000 comp rx=0 tx=0).stop 2024-03-30T00:48:29.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 -- 172.21.15.66:0/132892176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5380cabb0 msgr2=0x7fb5380ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5380cabb0 0x7fb5380ca2f0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb528001ec0 tx=0x7fb52800c8e0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 -- 172.21.15.66:0/132892176 shutdown_connections 2024-03-30T00:48:29.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb53815a640 0x7fb538155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5380ce520 0x7fb5380b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb514066690 0x7fb514068b50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 --2- 172.21.15.66:0/132892176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5380cabb0 0x7fb5380ca2f0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 -- 172.21.15.66:0/132892176 >> 172.21.15.66:0/132892176 conn(0x7fb538069f60 msgr2=0x7fb53806a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:29.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 -- 172.21.15.66:0/132892176 shutdown_connections 2024-03-30T00:48:29.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.757+0000 7fb53cc00640 1 -- 172.21.15.66:0/132892176 wait complete. 2024-03-30T00:48:29.785 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:48:29.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9e9000640 1 Processor -- start 2024-03-30T00:48:29.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9e9000640 1 -- start start 2024-03-30T00:48:29.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9e9000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb9e415c750 0x7fb9e40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9e9000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb9e415cb30 0x7fb9e406a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9e9000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e40b1800 con 0x7fb9e40ce5d0 2024-03-30T00:48:29.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9e9000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e40afcf0 con 0x7fb9e415cb30 2024-03-30T00:48:29.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9e9000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e40b06a0 con 0x7fb9e415c750 2024-03-30T00:48:29.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9daa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb9e415cb30 0x7fb9e406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9daa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb9e415cb30 0x7fb9e406a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48414/0 (socket says 172.21.15.66:48414) 2024-03-30T00:48:29.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9daa00640 1 -- 172.21.15.66:0/3940926324 learned_addr learned my addr 172.21.15.66:0/3940926324 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:29.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9e3400640 1 --2- 172.21.15.66:0/3940926324 >> v2:172.21.15.156:3300/0 conn(0x7fb9e415c750 0x7fb9e40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9daa00640 1 -- 172.21.15.66:0/3940926324 >> v2:172.21.15.156:3300/0 conn(0x7fb9e415c750 msgr2=0x7fb9e40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:29.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9daa00640 1 --2- 172.21.15.66:0/3940926324 >> v2:172.21.15.156:3300/0 conn(0x7fb9e415c750 0x7fb9e40b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9daa00640 1 -- 172.21.15.66:0/3940926324 >> v1:172.21.15.66:6789/0 conn(0x7fb9e40ce5d0 legacy=0x7fb9e40cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9daa00640 1 -- 172.21.15.66:0/3940926324 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e40b0980 con 0x7fb9e415cb30 2024-03-30T00:48:29.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.917+0000 7fb9e3400640 1 --2- 172.21.15.66:0/3940926324 >> v2:172.21.15.156:3300/0 conn(0x7fb9e415c750 0x7fb9e40b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:29.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9daa00640 1 --2- 172.21.15.66:0/3940926324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9e415cb30 0x7fb9e406a750 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40ccf40 tx=0x7fb9cc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6237df19fb527882 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:29.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9e2a00640 1 -- 172.21.15.66:0/3940926324 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc0ad680 con 0x7fb9e415cb30 2024-03-30T00:48:29.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9e2a00640 1 -- 172.21.15.66:0/3940926324 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9cc0ad7e0 con 0x7fb9e415cb30 2024-03-30T00:48:29.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9e2a00640 1 -- 172.21.15.66:0/3940926324 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc0adb70 con 0x7fb9e415cb30 2024-03-30T00:48:29.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9e9000640 1 -- 172.21.15.66:0/3940926324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9e415cb30 msgr2=0x7fb9e406a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9e9000640 1 --2- 172.21.15.66:0/3940926324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9e415cb30 0x7fb9e406a750 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40ccf40 tx=0x7fb9cc01c490 comp rx=0 tx=0).stop 2024-03-30T00:48:29.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9e9000640 1 -- 172.21.15.66:0/3940926324 shutdown_connections 2024-03-30T00:48:29.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9e9000640 1 --2- 172.21.15.66:0/3940926324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9e415cb30 0x7fb9e406a750 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9e9000640 1 --2- 172.21.15.66:0/3940926324 >> v2:172.21.15.156:3300/0 conn(0x7fb9e415c750 0x7fb9e40b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9e9000640 1 -- 172.21.15.66:0/3940926324 >> 172.21.15.66:0/3940926324 conn(0x7fb9e4098a20 msgr2=0x7fb9e40b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:29.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9e9000640 1 -- 172.21.15.66:0/3940926324 shutdown_connections 2024-03-30T00:48:29.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.921+0000 7fb9e9000640 1 -- 172.21.15.66:0/3940926324 wait complete. 2024-03-30T00:48:29.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e9000640 1 Processor -- start 2024-03-30T00:48:29.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e9000640 1 -- start start 2024-03-30T00:48:29.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9e40ce5d0 0x7fb9e40b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e9000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb9e415c750 0x7fb9e40cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e9000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9e415cb30 0x7fb9e40c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e9000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40c3cb0 con 0x7fb9e40ce5d0 2024-03-30T00:48:29.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e9000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40c83e0 con 0x7fb9e415cb30 2024-03-30T00:48:29.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e9000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40c86c0 con 0x7fb9e415c750 2024-03-30T00:48:29.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9e40ce5d0 0x7fb9e40b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9e40ce5d0 0x7fb9e40b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48418/0 (socket says 172.21.15.66:48418) 2024-03-30T00:48:29.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e3400640 1 -- 172.21.15.66:0/985325787 learned_addr learned my addr 172.21.15.66:0/985325787 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:29.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e3400640 1 -- 172.21.15.66:0/985325787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb9e415c750 msgr2=0x7fb9e40cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e3400640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb9e415c750 0x7fb9e40cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e3e00640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9e415cb30 0x7fb9e40c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e3400640 1 -- 172.21.15.66:0/985325787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9e415cb30 msgr2=0x7fb9e40c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e3400640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9e415cb30 0x7fb9e40c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e3400640 1 -- 172.21.15.66:0/985325787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc0094d0 con 0x7fb9e40ce5d0 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e3e00640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9e415cb30 0x7fb9e40c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e3400640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9e40ce5d0 0x7fb9e40b4840 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400c7c0 tx=0x7fb9d400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9dbe00640 1 -- 172.21.15.66:0/985325787 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4007a30 con 0x7fb9e40ce5d0 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9dbe00640 1 -- 172.21.15.66:0/985325787 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4007b90 con 0x7fb9e40ce5d0 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e9000640 1 -- 172.21.15.66:0/985325787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e40c89a0 con 0x7fb9e40ce5d0 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9dbe00640 1 -- 172.21.15.66:0/985325787 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400ce70 con 0x7fb9e40ce5d0 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e9000640 1 -- 172.21.15.66:0/985325787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e40b4d80 con 0x7fb9e40ce5d0 2024-03-30T00:48:29.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.925+0000 7fb9e9000640 1 -- 172.21.15.66:0/985325787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e4073e00 con 0x7fb9e40ce5d0 2024-03-30T00:48:29.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.929+0000 7fb9dbe00640 1 -- 172.21.15.66:0/985325787 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb9d40113f0 con 0x7fb9e40ce5d0 2024-03-30T00:48:29.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.929+0000 7fb9dbe00640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9c0066690 0x7fb9c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:29.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.929+0000 7fb9dbe00640 1 -- 172.21.15.66:0/985325787 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fb9d40848f0 con 0x7fb9e40ce5d0 2024-03-30T00:48:29.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.929+0000 7fb9daa00640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9c0066690 0x7fb9c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:29.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.929+0000 7fb9daa00640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9c0066690 0x7fb9c0068b50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0af000 tx=0x7fb9cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:29.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:29.933+0000 7fb9dbe00640 1 -- 172.21.15.66:0/985325787 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb9d40519f0 con 0x7fb9e40ce5d0 2024-03-30T00:48:30.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.105+0000 7fb9e9000640 1 -- 172.21.15.66:0/985325787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb9e40b53a0 con 0x7fb9e40ce5d0 2024-03-30T00:48:30.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.105+0000 7fb9dbe00640 1 -- 172.21.15.66:0/985325787 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v98) v1 ==== 80+0+244 (secure 0 0 0) 0x7fb9d401e110 con 0x7fb9e40ce5d0 2024-03-30T00:48:30.117 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6835/1626022080 2024-03-31T00:47:49.264137+0000 2024-03-30T00:48:30.117 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6836/1626022080 2024-03-31T00:47:49.264137+0000 2024-03-30T00:48:30.117 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6836/1147664160 2024-03-31T00:46:34.103831+0000 2024-03-30T00:48:30.117 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6835/1147664160 2024-03-31T00:46:34.103831+0000 2024-03-30T00:48:30.117 INFO:teuthology.orchestra.run.smithi066.stderr:listed 4 entries 2024-03-30T00:48:30.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 -- 172.21.15.66:0/985325787 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9c0066690 msgr2=0x7fb9c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:30.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9c0066690 0x7fb9c0068b50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0af000 tx=0x7fb9cc008040 comp rx=0 tx=0).stop 2024-03-30T00:48:30.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 -- 172.21.15.66:0/985325787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9e40ce5d0 msgr2=0x7fb9e40b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:30.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9e40ce5d0 0x7fb9e40b4840 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400c7c0 tx=0x7fb9d400cc80 comp rx=0 tx=0).stop 2024-03-30T00:48:30.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 -- 172.21.15.66:0/985325787 shutdown_connections 2024-03-30T00:48:30.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9e415cb30 0x7fb9e40c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb9e415c750 0x7fb9e40cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb9c0066690 0x7fb9c0068b50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 --2- 172.21.15.66:0/985325787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9e40ce5d0 0x7fb9e40b4840 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 -- 172.21.15.66:0/985325787 >> 172.21.15.66:0/985325787 conn(0x7fb9e4098a20 msgr2=0x7fb9e406e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:30.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 -- 172.21.15.66:0/985325787 shutdown_connections 2024-03-30T00:48:30.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.109+0000 7fb9e9000640 1 -- 172.21.15.66:0/985325787 wait complete. 2024-03-30T00:48:30.137 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:48:30.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:30.138 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:48:30.156 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[49348]: fuse finished with error 0 and tester_r 0 2024-03-30T00:48:30.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:48:30.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:30.159 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:48:30.245 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:48:30.245 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:48:30.255 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:48:30.255 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:48:30.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:30.255 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T00:48:30.305 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-30T00:48:30.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:48:30.306 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:48:30.306 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:48:30.315 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:48:30.315 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:48:30.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:30.315 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-30T00:48:30.364 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-30T00:48:30.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:48:30.366 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:48:30.366 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:48:30.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:48:30.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:48:30.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:30.375 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-30T00:48:30.426 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-30T00:48:30.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:48:30.427 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:48:30.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.557+0000 7ff89f400640 1 Processor -- start 2024-03-30T00:48:30.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.557+0000 7ff89f400640 1 -- start start 2024-03-30T00:48:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.557+0000 7ff89f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8980cda90 0x7ff8980ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.557+0000 7ff89f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8980cde70 0x7ff8980cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.557+0000 7ff89f400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89815a990 con 0x7ff898159c60 2024-03-30T00:48:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.557+0000 7ff89f400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8980b1b80 con 0x7ff8980cde70 2024-03-30T00:48:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.557+0000 7ff89f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980cd3e0 con 0x7ff8980cda90 2024-03-30T00:48:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.557+0000 7ff897e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8980cde70 0x7ff8980cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.557+0000 7ff897e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8980cde70 0x7ff8980cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48430/0 (socket says 172.21.15.66:48430) 2024-03-30T00:48:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.557+0000 7ff897e00640 1 -- 172.21.15.66:0/3578715150 learned_addr learned my addr 172.21.15.66:0/3578715150 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff89ca00640 1 --2- 172.21.15.66:0/3578715150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8980cda90 0x7ff8980ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff897e00640 1 -- 172.21.15.66:0/3578715150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8980cda90 msgr2=0x7ff8980ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff897e00640 1 --2- 172.21.15.66:0/3578715150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8980cda90 0x7ff8980ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff897e00640 1 -- 172.21.15.66:0/3578715150 >> v1:172.21.15.156:6789/0 conn(0x7ff898159c60 legacy=0x7ff89815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff897e00640 1 -- 172.21.15.66:0/3578715150 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8980b11e0 con 0x7ff8980cde70 2024-03-30T00:48:30.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff89ca00640 1 --2- 172.21.15.66:0/3578715150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8980cda90 0x7ff8980ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff897e00640 1 --2- 172.21.15.66:0/3578715150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8980cde70 0x7ff8980cea60 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ff884009870 tx=0x7ff88401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a46ef4c8533cd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff897400640 1 -- 172.21.15.66:0/3578715150 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840ad660 con 0x7ff8980cde70 2024-03-30T00:48:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff897400640 1 -- 172.21.15.66:0/3578715150 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8840ad7c0 con 0x7ff8980cde70 2024-03-30T00:48:30.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff897400640 1 -- 172.21.15.66:0/3578715150 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840adb50 con 0x7ff8980cde70 2024-03-30T00:48:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff89f400640 1 -- 172.21.15.66:0/3578715150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8980cde70 msgr2=0x7ff8980cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.561+0000 7ff89f400640 1 --2- 172.21.15.66:0/3578715150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8980cde70 0x7ff8980cea60 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ff884009870 tx=0x7ff88401c490 comp rx=0 tx=0).stop 2024-03-30T00:48:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 -- 172.21.15.66:0/3578715150 shutdown_connections 2024-03-30T00:48:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 --2- 172.21.15.66:0/3578715150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8980cde70 0x7ff8980cea60 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 --2- 172.21.15.66:0/3578715150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8980cda90 0x7ff8980ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 -- 172.21.15.66:0/3578715150 >> 172.21.15.66:0/3578715150 conn(0x7ff898069f60 msgr2=0x7ff89806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 -- 172.21.15.66:0/3578715150 shutdown_connections 2024-03-30T00:48:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 -- 172.21.15.66:0/3578715150 wait complete. 2024-03-30T00:48:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 Processor -- start 2024-03-30T00:48:30.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 -- start start 2024-03-30T00:48:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8980cda90 0x7ff898152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8980cde70 0x7ff8980aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff898159c60 0x7ff8981645c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff898164130 con 0x7ff8980cda90 2024-03-30T00:48:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980ac2e0 con 0x7ff8980cde70 2024-03-30T00:48:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff898163800 con 0x7ff898159c60 2024-03-30T00:48:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8980cda90 0x7ff898152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8980cda90 0x7ff898152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48438/0 (socket says 172.21.15.66:48438) 2024-03-30T00:48:30.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89ca00640 1 -- 172.21.15.66:0/4160911442 learned_addr learned my addr 172.21.15.66:0/4160911442 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff897e00640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8980cde70 0x7ff8980aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89d400640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff898159c60 0x7ff8981645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89ca00640 1 -- 172.21.15.66:0/4160911442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff898159c60 msgr2=0x7ff8981645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89ca00640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff898159c60 0x7ff8981645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89ca00640 1 -- 172.21.15.66:0/4160911442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8980cde70 msgr2=0x7ff8980aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89ca00640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8980cde70 0x7ff8980aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89ca00640 1 -- 172.21.15.66:0/4160911442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8840094d0 con 0x7ff8980cda90 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff897e00640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8980cde70 0x7ff8980aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89ca00640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8980cda90 0x7ff898152df0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00c7c0 tx=0x7ff88c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff894c00640 1 -- 172.21.15.66:0/4160911442 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c007a30 con 0x7ff8980cda90 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff894c00640 1 -- 172.21.15.66:0/4160911442 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c007b90 con 0x7ff8980cda90 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff894c00640 1 -- 172.21.15.66:0/4160911442 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00ce70 con 0x7ff8980cda90 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.565+0000 7ff89f400640 1 -- 172.21.15.66:0/4160911442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff898163ae0 con 0x7ff8980cda90 2024-03-30T00:48:30.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.569+0000 7ff89f400640 1 -- 172.21.15.66:0/4160911442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8980abf60 con 0x7ff8980cda90 2024-03-30T00:48:30.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.569+0000 7ff89f400640 1 -- 172.21.15.66:0/4160911442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff898066c30 con 0x7ff8980cda90 2024-03-30T00:48:30.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.569+0000 7ff894c00640 1 -- 172.21.15.66:0/4160911442 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff88c0113f0 con 0x7ff8980cda90 2024-03-30T00:48:30.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.569+0000 7ff894c00640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff878066690 0x7ff878068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.569+0000 7ff894c00640 1 -- 172.21.15.66:0/4160911442 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7ff88c084990 con 0x7ff8980cda90 2024-03-30T00:48:30.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.569+0000 7ff897e00640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff878066690 0x7ff878068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.573+0000 7ff897e00640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff878066690 0x7ff878068b50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff8840af000 tx=0x7ff884008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:30.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.573+0000 7ff894c00640 1 -- 172.21.15.66:0/4160911442 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff88c051a00 con 0x7ff8980cda90 2024-03-30T00:48:30.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 -- 172.21.15.66:0/4160911442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff898165280 con 0x7ff8980cda90 2024-03-30T00:48:30.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff894c00640 1 -- 172.21.15.66:0/4160911442 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4740 (secure 0 0 0) 0x7ff88c055020 con 0x7ff8980cda90 2024-03-30T00:48:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:30.782 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:47:53.284661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/2419209461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2419209461},{"type":"v1","addr":"172.21.15.66:6836","nonce":2419209461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:48:30.783 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 71 2024-03-30T00:48:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 -- 172.21.15.66:0/4160911442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff878066690 msgr2=0x7ff878068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff878066690 0x7ff878068b50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff8840af000 tx=0x7ff884008040 comp rx=0 tx=0).stop 2024-03-30T00:48:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 -- 172.21.15.66:0/4160911442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8980cda90 msgr2=0x7ff898152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8980cda90 0x7ff898152df0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00c7c0 tx=0x7ff88c00cc80 comp rx=0 tx=0).stop 2024-03-30T00:48:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 -- 172.21.15.66:0/4160911442 shutdown_connections 2024-03-30T00:48:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff898159c60 0x7ff8981645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8980cde70 0x7ff8980aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff878066690 0x7ff878068b50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 --2- 172.21.15.66:0/4160911442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8980cda90 0x7ff898152df0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 -- 172.21.15.66:0/4160911442 >> 172.21.15.66:0/4160911442 conn(0x7ff898069f60 msgr2=0x7ff89806d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 -- 172.21.15.66:0/4160911442 shutdown_connections 2024-03-30T00:48:30.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.773+0000 7ff89f400640 1 -- 172.21.15.66:0/4160911442 wait complete. 2024-03-30T00:48:30.804 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:48:30.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd392c00640 1 Processor -- start 2024-03-30T00:48:30.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd392c00640 1 -- start start 2024-03-30T00:48:30.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd392c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c0ce520 0x7fd38c0ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd392c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd38c0cabb0 0x7fd38c0cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd392c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c06d070 con 0x7fd38c15a640 2024-03-30T00:48:30.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd392c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0cbba0 con 0x7fd38c0cabb0 2024-03-30T00:48:30.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd392c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0cee40 con 0x7fd38c0ce520 2024-03-30T00:48:30.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd38be00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd38c15a640 0x7fd38c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:42874/0 (socket says 172.21.15.66:42874) 2024-03-30T00:48:30.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd38be00640 1 -- 172.21.15.66:0/1451551037 learned_addr learned my addr 172.21.15.66:0/1451551037 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:30.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd383400640 1 --2- 172.21.15.66:0/1451551037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c0ce520 0x7fd38c0ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd390c00640 1 --2- 172.21.15.66:0/1451551037 >> v2:172.21.15.156:3300/0 conn(0x7fd38c0cabb0 0x7fd38c0cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd390c00640 1 -- 172.21.15.66:0/1451551037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c0ce520 msgr2=0x7fd38c0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:30.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd390c00640 1 --2- 172.21.15.66:0/1451551037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c0ce520 0x7fd38c0ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd390c00640 1 -- 172.21.15.66:0/1451551037 >> v1:172.21.15.156:6789/0 conn(0x7fd38c15a640 legacy=0x7fd38c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:30.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd390c00640 1 -- 172.21.15.66:0/1451551037 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0cb8c0 con 0x7fd38c0cabb0 2024-03-30T00:48:30.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.937+0000 7fd383400640 1 --2- 172.21.15.66:0/1451551037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c0ce520 0x7fd38c0ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:30.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd390c00640 1 --2- 172.21.15.66:0/1451551037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd38c0cabb0 0x7fd38c0cb000 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd3840097c0 tx=0x7fd38401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4de908a0c4f1c32 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:30.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd38b400640 1 -- 172.21.15.66:0/1451551037 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840ab670 con 0x7fd38c0cabb0 2024-03-30T00:48:30.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd38b400640 1 -- 172.21.15.66:0/1451551037 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3840ab7d0 con 0x7fd38c0cabb0 2024-03-30T00:48:30.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd38b400640 1 -- 172.21.15.66:0/1451551037 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840abb60 con 0x7fd38c0cabb0 2024-03-30T00:48:30.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd392c00640 1 -- 172.21.15.66:0/1451551037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd38c0cabb0 msgr2=0x7fd38c0cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:30.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd392c00640 1 --2- 172.21.15.66:0/1451551037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd38c0cabb0 0x7fd38c0cb000 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd3840097c0 tx=0x7fd38401c470 comp rx=0 tx=0).stop 2024-03-30T00:48:30.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd392c00640 1 -- 172.21.15.66:0/1451551037 shutdown_connections 2024-03-30T00:48:30.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd392c00640 1 --2- 172.21.15.66:0/1451551037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd38c0cabb0 0x7fd38c0cb000 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd392c00640 1 --2- 172.21.15.66:0/1451551037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c0ce520 0x7fd38c0ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd392c00640 1 -- 172.21.15.66:0/1451551037 >> 172.21.15.66:0/1451551037 conn(0x7fd38c069f60 msgr2=0x7fd38c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:30.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd392c00640 1 -- 172.21.15.66:0/1451551037 shutdown_connections 2024-03-30T00:48:30.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.941+0000 7fd392c00640 1 -- 172.21.15.66:0/1451551037 wait complete. 2024-03-30T00:48:30.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd392c00640 1 Processor -- start 2024-03-30T00:48:30.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd392c00640 1 -- start start 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd392c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd38c0cabb0 0x7fd38c0ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd392c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd38c0ce520 0x7fd38c096a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd392c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c15a640 0x7fd38c0cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd392c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0ae040 con 0x7fd38c0cabb0 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd392c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0af4b0 con 0x7fd38c0ce520 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd392c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0af750 con 0x7fd38c15a640 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd38be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd38c0cabb0 0x7fd38c0ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd38be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd38c0cabb0 0x7fd38c0ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48454/0 (socket says 172.21.15.66:48454) 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd38be00640 1 -- 172.21.15.66:0/3041792662 learned_addr learned my addr 172.21.15.66:0/3041792662 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd383400640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd38c0ce520 0x7fd38c096a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd390c00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c15a640 0x7fd38c0cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd38be00640 1 -- 172.21.15.66:0/3041792662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c15a640 msgr2=0x7fd38c0cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:30.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd38be00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c15a640 0x7fd38c0cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd38be00640 1 -- 172.21.15.66:0/3041792662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd38c0ce520 msgr2=0x7fd38c096a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:30.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd38be00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd38c0ce520 0x7fd38c096a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:30.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd38be00640 1 -- 172.21.15.66:0/3041792662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd374009420 con 0x7fd38c0cabb0 2024-03-30T00:48:30.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd390c00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c15a640 0x7fd38c0cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:30.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd383400640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd38c0ce520 0x7fd38c096a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:30.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd38be00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd38c0cabb0 0x7fd38c0ca0e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00cbc0 tx=0x7fd37c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:30.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd388c00640 1 -- 172.21.15.66:0/3041792662 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c00aca0 con 0x7fd38c0cabb0 2024-03-30T00:48:30.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd388c00640 1 -- 172.21.15.66:0/3041792662 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c00ae00 con 0x7fd38c0cabb0 2024-03-30T00:48:30.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd388c00640 1 -- 172.21.15.66:0/3041792662 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c012600 con 0x7fd38c0cabb0 2024-03-30T00:48:30.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd392c00640 1 -- 172.21.15.66:0/3041792662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd384009420 con 0x7fd38c0cabb0 2024-03-30T00:48:30.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.945+0000 7fd392c00640 1 -- 172.21.15.66:0/3041792662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c0c9be0 con 0x7fd38c0cabb0 2024-03-30T00:48:30.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.949+0000 7fd392c00640 1 -- 172.21.15.66:0/3041792662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c066c30 con 0x7fd38c0cabb0 2024-03-30T00:48:30.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.949+0000 7fd388c00640 1 -- 172.21.15.66:0/3041792662 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd37c007070 con 0x7fd38c0cabb0 2024-03-30T00:48:30.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.949+0000 7fd388c00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd368066690 0x7fd368068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:30.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.949+0000 7fd388c00640 1 -- 172.21.15.66:0/3041792662 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fd37c0842c0 con 0x7fd38c0cabb0 2024-03-30T00:48:30.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.949+0000 7fd383400640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd368066690 0x7fd368068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:30.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.949+0000 7fd383400640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd368066690 0x7fd368068b50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd38c06a810 tx=0x7fd374008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:30.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:30.953+0000 7fd388c00640 1 -- 172.21.15.66:0/3041792662 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd37c0513e0 con 0x7fd38c0cabb0 2024-03-30T00:48:31.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.113+0000 7fd392c00640 1 -- 172.21.15.66:0/3041792662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd38c0b0a20 con 0x7fd38c0cabb0 2024-03-30T00:48:31.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd388c00640 1 -- 172.21.15.66:0/3041792662 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16701 (secure 0 0 0) 0x7fd37c054a00 con 0x7fd38c0cabb0 2024-03-30T00:48:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:31.126 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":98,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:47:49.264161+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:46:13.811742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-30T00:46:14.265142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":7}]},{"pgid":"14.1a","mappings":[{"from":1,"to":5}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/1626022080":"2024-03-31T00:47:49.264137+0000","172.21.15.66:6836/1626022080":"2024-03-31T00:47:49.264137+0000","172.21.15.66:6836/1147664160":"2024-03-31T00:46:34.103831+0000","172.21.15.66:6835/1147664160":"2024-03-31T00:46:34.103831+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 -- 172.21.15.66:0/3041792662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd368066690 msgr2=0x7fd368068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd368066690 0x7fd368068b50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd38c06a810 tx=0x7fd374008040 comp rx=0 tx=0).stop 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 -- 172.21.15.66:0/3041792662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd38c0cabb0 msgr2=0x7fd38c0ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd38c0cabb0 0x7fd38c0ca0e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00cbc0 tx=0x7fd37c00a060 comp rx=0 tx=0).stop 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 -- 172.21.15.66:0/3041792662 shutdown_connections 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd38c15a640 0x7fd38c0cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd38c0ce520 0x7fd38c096a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd368066690 0x7fd368068b50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 --2- 172.21.15.66:0/3041792662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd38c0cabb0 0x7fd38c0ca0e0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 -- 172.21.15.66:0/3041792662 >> 172.21.15.66:0/3041792662 conn(0x7fd38c069f60 msgr2=0x7fd38c15c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 -- 172.21.15.66:0/3041792662 shutdown_connections 2024-03-30T00:48:31.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.117+0000 7fd392c00640 1 -- 172.21.15.66:0/3041792662 wait complete. 2024-03-30T00:48:31.148 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:46:13.811742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:46:14.265142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:48:31.149 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:48:31.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.281+0000 7efecc600640 1 Processor -- start 2024-03-30T00:48:31.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.281+0000 7efecc600640 1 -- start start 2024-03-30T00:48:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.281+0000 7efecc600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efec40cda90 0x7efec40ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.281+0000 7efecc600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efec40cde70 0x7efec40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.281+0000 7efecc600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec415a990 con 0x7efec4159c60 2024-03-30T00:48:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.281+0000 7efecc600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec40b1b80 con 0x7efec40cde70 2024-03-30T00:48:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.281+0000 7efecc600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec40cd3e0 con 0x7efec40cda90 2024-03-30T00:48:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efec9200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efec40cde70 0x7efec40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efeca600640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efec4159c60 0x7efec415a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45838/0 (socket says 172.21.15.66:45838) 2024-03-30T00:48:31.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efeca600640 1 -- 172.21.15.66:0/541845079 learned_addr learned my addr 172.21.15.66:0/541845079 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:31.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efec9c00640 1 --2- 172.21.15.66:0/541845079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efec40cda90 0x7efec40ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efec9200640 1 -- 172.21.15.66:0/541845079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efec40cda90 msgr2=0x7efec40ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:31.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efec9200640 1 --2- 172.21.15.66:0/541845079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efec40cda90 0x7efec40ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efec9200640 1 -- 172.21.15.66:0/541845079 >> v1:172.21.15.66:6789/0 conn(0x7efec4159c60 legacy=0x7efec415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efec9200640 1 -- 172.21.15.66:0/541845079 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec40b11e0 con 0x7efec40cde70 2024-03-30T00:48:31.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efec3e00640 1 -- 172.21.15.66:0/541845079 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797820061 0 0) 0x7efec415a990 con 0x7efec4159c60 2024-03-30T00:48:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efec9200640 1 --2- 172.21.15.66:0/541845079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efec40cde70 0x7efec40cea60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7efeb00097c0 tx=0x7efeb001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd929318cdd9e65 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:31.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efec3e00640 1 -- 172.21.15.66:0/541845079 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb00ab600 con 0x7efec40cde70 2024-03-30T00:48:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efec3e00640 1 -- 172.21.15.66:0/541845079 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb00a7050 con 0x7efec40cde70 2024-03-30T00:48:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.285+0000 7efec3e00640 1 -- 172.21.15.66:0/541845079 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb00ab980 con 0x7efec40cde70 2024-03-30T00:48:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 -- 172.21.15.66:0/541845079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efec40cde70 msgr2=0x7efec40cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 --2- 172.21.15.66:0/541845079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efec40cde70 0x7efec40cea60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7efeb00097c0 tx=0x7efeb001c410 comp rx=0 tx=0).stop 2024-03-30T00:48:31.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 -- 172.21.15.66:0/541845079 shutdown_connections 2024-03-30T00:48:31.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 --2- 172.21.15.66:0/541845079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efec40cde70 0x7efec40cea60 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 --2- 172.21.15.66:0/541845079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efec40cda90 0x7efec40ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 -- 172.21.15.66:0/541845079 >> 172.21.15.66:0/541845079 conn(0x7efec4069f60 msgr2=0x7efec406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:31.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 -- 172.21.15.66:0/541845079 shutdown_connections 2024-03-30T00:48:31.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 -- 172.21.15.66:0/541845079 wait complete. 2024-03-30T00:48:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 Processor -- start 2024-03-30T00:48:31.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 -- start start 2024-03-30T00:48:31.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efec40cda90 0x7efec4097720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efec40cde70 0x7efec4159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efec4159c60 0x7efec40c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4158df0 con 0x7efec4159c60 2024-03-30T00:48:31.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec41581e0 con 0x7efec40cde70 2024-03-30T00:48:31.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efecc600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec41584c0 con 0x7efec40cda90 2024-03-30T00:48:31.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efeca600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efec4159c60 0x7efec40c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efeca600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efec4159c60 0x7efec40c20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48462/0 (socket says 172.21.15.66:48462) 2024-03-30T00:48:31.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efeca600640 1 -- 172.21.15.66:0/1973781702 learned_addr learned my addr 172.21.15.66:0/1973781702 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:31.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efec9200640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efec40cde70 0x7efec4159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efec9c00640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efec40cda90 0x7efec4097720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efeca600640 1 -- 172.21.15.66:0/1973781702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efec40cda90 msgr2=0x7efec4097720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efeca600640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efec40cda90 0x7efec4097720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efeca600640 1 -- 172.21.15.66:0/1973781702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efec40cde70 msgr2=0x7efec4159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efeca600640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efec40cde70 0x7efec4159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efeca600640 1 -- 172.21.15.66:0/1973781702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0009420 con 0x7efec4159c60 2024-03-30T00:48:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.289+0000 7efec9c00640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efec40cda90 0x7efec4097720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:31.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.293+0000 7efeca600640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efec4159c60 0x7efec40c20e0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7efebc00c700 tx=0x7efebc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.293+0000 7efec1600640 1 -- 172.21.15.66:0/1973781702 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00ce10 con 0x7efec4159c60 2024-03-30T00:48:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.293+0000 7efec1600640 1 -- 172.21.15.66:0/1973781702 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc00a640 con 0x7efec4159c60 2024-03-30T00:48:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.293+0000 7efec1600640 1 -- 172.21.15.66:0/1973781702 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00aa70 con 0x7efec4159c60 2024-03-30T00:48:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.293+0000 7efecc600640 1 -- 172.21.15.66:0/1973781702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec41587a0 con 0x7efec4159c60 2024-03-30T00:48:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.293+0000 7efecc600640 1 -- 172.21.15.66:0/1973781702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec4097c60 con 0x7efec4159c60 2024-03-30T00:48:31.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.293+0000 7efecc600640 1 -- 172.21.15.66:0/1973781702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec4158f60 con 0x7efec4159c60 2024-03-30T00:48:31.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.293+0000 7efec1600640 1 -- 172.21.15.66:0/1973781702 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efebc00abd0 con 0x7efec4159c60 2024-03-30T00:48:31.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.293+0000 7efec1600640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efea40666e0 0x7efea4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.293+0000 7efec1600640 1 -- 172.21.15.66:0/1973781702 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7efebc083c50 con 0x7efec4159c60 2024-03-30T00:48:31.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.297+0000 7efec9c00640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efea40666e0 0x7efea4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.297+0000 7efec9c00640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efea40666e0 0x7efea4068ba0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efeb8007a70 tx=0x7efeb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:31.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.301+0000 7efec1600640 1 -- 172.21.15.66:0/1973781702 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efebc050df0 con 0x7efec4159c60 2024-03-30T00:48:31.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 -- 172.21.15.66:0/1973781702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efec4066c30 con 0x7efec4159c60 2024-03-30T00:48:31.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efec1600640 1 -- 172.21.15.66:0/1973781702 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7efebc054410 con 0x7efec4159c60 2024-03-30T00:48:31.471 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:48:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 -- 172.21.15.66:0/1973781702 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efea40666e0 msgr2=0x7efea4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efea40666e0 0x7efea4068ba0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efeb8007a70 tx=0x7efeb8008040 comp rx=0 tx=0).stop 2024-03-30T00:48:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 -- 172.21.15.66:0/1973781702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efec4159c60 msgr2=0x7efec40c20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efec4159c60 0x7efec40c20e0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7efebc00c700 tx=0x7efebc00cbc0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 -- 172.21.15.66:0/1973781702 shutdown_connections 2024-03-30T00:48:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efea40666e0 0x7efea4068ba0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efec4159c60 0x7efec40c20e0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efec40cde70 0x7efec4159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 --2- 172.21.15.66:0/1973781702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efec40cda90 0x7efec4097720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 -- 172.21.15.66:0/1973781702 >> 172.21.15.66:0/1973781702 conn(0x7efec4069f60 msgr2=0x7efec40afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:31.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 -- 172.21.15.66:0/1973781702 shutdown_connections 2024-03-30T00:48:31.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.461+0000 7efecc600640 1 -- 172.21.15.66:0/1973781702 wait complete. 2024-03-30T00:48:31.492 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:48:31.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.625+0000 7f639a200640 1 Processor -- start 2024-03-30T00:48:31.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.625+0000 7f639a200640 1 -- start start 2024-03-30T00:48:31.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.625+0000 7f639a200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f639406f8e0 0x7f63940d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f639a200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639406b0b0 con 0x7f63940cc560 2024-03-30T00:48:31.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f639a200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639406b390 con 0x7f639406e800 2024-03-30T00:48:31.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f639a200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6394160640 con 0x7f639406f8e0 2024-03-30T00:48:31.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6393400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f63940cc560 0x7f639406e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45840/0 (socket says 172.21.15.66:45840) 2024-03-30T00:48:31.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6393400640 1 -- 172.21.15.66:0/1653087217 learned_addr learned my addr 172.21.15.66:0/1653087217 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6393e00640 1 --2- 172.21.15.66:0/1653087217 >> v2:172.21.15.156:3300/0 conn(0x7f639406f8e0 0x7f63940d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510908266 0 0) 0x7f639406b0b0 con 0x7f63940cc560 2024-03-30T00:48:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6378003530 con 0x7f63940cc560 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3669241051 0 0) 0x7f6378003530 con 0x7f63940cc560 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 >> v2:172.21.15.156:3300/0 conn(0x7f639406f8e0 msgr2=0x7f63940d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 --2- 172.21.15.66:0/1653087217 >> v2:172.21.15.156:3300/0 conn(0x7f639406f8e0 0x7f63940d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 >> v1:172.21.15.156:6789/0 conn(0x7f639406e800 legacy=0x7f63940ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639406b670 con 0x7f63940cc560 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f6384001a00 con 0x7f63940cc560 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 >> v1:172.21.15.66:6789/0 conn(0x7f63940cc560 legacy=0x7f639406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 --2- 172.21.15.66:0/1653087217 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6378003980 0x7f6378003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 --2- 172.21.15.66:0/1653087217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63780047d0 0x7f6378004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 --2- 172.21.15.66:0/1653087217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6378005190 0x7f637800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6394160640 con 0x7f63780047d0 2024-03-30T00:48:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637800c030 con 0x7f6378005190 2024-03-30T00:48:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637800c1f0 con 0x7f6378003980 2024-03-30T00:48:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f638aa00640 1 --2- 172.21.15.66:0/1653087217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63780047d0 0x7f6378004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6393400640 1 --2- 172.21.15.66:0/1653087217 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6378003980 0x7f6378003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6393e00640 1 --2- 172.21.15.66:0/1653087217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6378005190 0x7f637800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f638aa00640 1 -- 172.21.15.66:0/1653087217 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6378003980 msgr2=0x7f6378003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f638aa00640 1 --2- 172.21.15.66:0/1653087217 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6378003980 0x7f6378003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f638aa00640 1 -- 172.21.15.66:0/1653087217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6378005190 msgr2=0x7f637800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f638aa00640 1 --2- 172.21.15.66:0/1653087217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6378005190 0x7f637800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f638aa00640 1 -- 172.21.15.66:0/1653087217 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6378003530 con 0x7f63780047d0 2024-03-30T00:48:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.629+0000 7f6393e00640 1 -- 172.21.15.66:0/1653087217 reap_dead start 2024-03-30T00:48:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f638aa00640 1 --2- 172.21.15.66:0/1653087217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63780047d0 0x7f6378004c50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f637c001780 tx=0x7f637c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d03f230681853f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c0052d0 con 0x7f63780047d0 2024-03-30T00:48:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c0a7050 con 0x7f63780047d0 2024-03-30T00:48:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f6392a00640 1 -- 172.21.15.66:0/1653087217 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c003600 con 0x7f63780047d0 2024-03-30T00:48:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f639a200640 1 -- 172.21.15.66:0/1653087217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63780047d0 msgr2=0x7f6378004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f639a200640 1 --2- 172.21.15.66:0/1653087217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63780047d0 0x7f6378004c50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f637c001780 tx=0x7f637c01c400 comp rx=0 tx=0).stop 2024-03-30T00:48:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f639a200640 1 -- 172.21.15.66:0/1653087217 shutdown_connections 2024-03-30T00:48:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f639a200640 1 --2- 172.21.15.66:0/1653087217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63780047d0 0x7f6378004c50 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f639a200640 1 -- 172.21.15.66:0/1653087217 >> 172.21.15.66:0/1653087217 conn(0x7f63940d0540 msgr2=0x7f63940d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f639a200640 1 -- 172.21.15.66:0/1653087217 shutdown_connections 2024-03-30T00:48:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f639a200640 1 -- 172.21.15.66:0/1653087217 wait complete. 2024-03-30T00:48:31.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f639a200640 1 Processor -- start 2024-03-30T00:48:31.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.633+0000 7f639a200640 1 -- start start 2024-03-30T00:48:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f639a200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63780047d0 0x7f63940c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f639a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63940b3160 0x7f63940b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f639a200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f639414a5e0 0x7f639414aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f639a200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940b3b20 con 0x7f63940b3160 2024-03-30T00:48:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f639a200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940ca020 con 0x7f63780047d0 2024-03-30T00:48:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f639a200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639414afa0 con 0x7f639414a5e0 2024-03-30T00:48:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63940b3160 0x7f63940b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63940b3160 0x7f63940b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48484/0 (socket says 172.21.15.66:48484) 2024-03-30T00:48:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638aa00640 1 -- 172.21.15.66:0/2585912602 learned_addr learned my addr 172.21.15.66:0/2585912602 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f6393400640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63780047d0 0x7f63940c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f6393e00640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f639414a5e0 0x7f639414aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638aa00640 1 -- 172.21.15.66:0/2585912602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f639414a5e0 msgr2=0x7f639414aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638aa00640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f639414a5e0 0x7f639414aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638aa00640 1 -- 172.21.15.66:0/2585912602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63780047d0 msgr2=0x7f63940c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638aa00640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63780047d0 0x7f63940c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638aa00640 1 -- 172.21.15.66:0/2585912602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940c8a40 con 0x7f63940b3160 2024-03-30T00:48:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638aa00640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63940b3160 0x7f63940b35e0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f637c005040 tx=0x7f6378005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638be00640 1 -- 172.21.15.66:0/2585912602 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c003e40 con 0x7f63940b3160 2024-03-30T00:48:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638be00640 1 -- 172.21.15.66:0/2585912602 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c007a40 con 0x7f63940b3160 2024-03-30T00:48:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638be00640 1 -- 172.21.15.66:0/2585912602 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c007270 con 0x7f63940b3160 2024-03-30T00:48:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f639a200640 1 -- 172.21.15.66:0/2585912602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63940c8d20 con 0x7f63940b3160 2024-03-30T00:48:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f639a200640 1 -- 172.21.15.66:0/2585912602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63940c92e0 con 0x7f63940b3160 2024-03-30T00:48:31.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f639a200640 1 -- 172.21.15.66:0/2585912602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6394066c30 con 0x7f63940b3160 2024-03-30T00:48:31.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.637+0000 7f638be00640 1 -- 172.21.15.66:0/2585912602 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f637c0073d0 con 0x7f63940b3160 2024-03-30T00:48:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.641+0000 7f638be00640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63700666c0 0x7f6370068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.641+0000 7f638be00640 1 -- 172.21.15.66:0/2585912602 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f637c11e950 con 0x7f63940b3160 2024-03-30T00:48:31.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.641+0000 7f6393400640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63700666c0 0x7f6370068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.641+0000 7f6393400640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63700666c0 0x7f6370068b80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6384001b30 tx=0x7f63840017c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:31.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.645+0000 7f638be00640 1 -- 172.21.15.66:0/2585912602 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f637c0eb9c0 con 0x7f63940b3160 2024-03-30T00:48:31.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.805+0000 7f639a200640 1 -- 172.21.15.66:0/2585912602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63940d0af0 con 0x7f63940b3160 2024-03-30T00:48:31.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.805+0000 7f638be00640 1 -- 172.21.15.66:0/2585912602 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f637c0eefe0 con 0x7f63940b3160 2024-03-30T00:48:31.817 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:48:31.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 -- 172.21.15.66:0/2585912602 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63700666c0 msgr2=0x7f6370068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63700666c0 0x7f6370068b80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6384001b30 tx=0x7f63840017c0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 -- 172.21.15.66:0/2585912602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63940b3160 msgr2=0x7f63940b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63940b3160 0x7f63940b35e0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f637c005040 tx=0x7f6378005800 comp rx=0 tx=0).stop 2024-03-30T00:48:31.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 -- 172.21.15.66:0/2585912602 shutdown_connections 2024-03-30T00:48:31.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f639414a5e0 0x7f639414aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f63700666c0 0x7f6370068b80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63940b3160 0x7f63940b35e0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 --2- 172.21.15.66:0/2585912602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63780047d0 0x7f63940c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 -- 172.21.15.66:0/2585912602 >> 172.21.15.66:0/2585912602 conn(0x7f63940d0540 msgr2=0x7f63940d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:31.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 -- 172.21.15.66:0/2585912602 shutdown_connections 2024-03-30T00:48:31.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.809+0000 7f639a200640 1 -- 172.21.15.66:0/2585912602 wait complete. 2024-03-30T00:48:31.838 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:48:31.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd473800640 1 Processor -- start 2024-03-30T00:48:31.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd473800640 1 -- start start 2024-03-30T00:48:31.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd473800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd46c15ca80 0x7fd46c06b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd473800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd46c0cf2f0 0x7fd46c06b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd473800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46c0cb0e0 con 0x7fd46c06bb30 2024-03-30T00:48:31.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd473800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46c0cf7f0 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd473800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46c0cfad0 con 0x7fd46c15ca80 2024-03-30T00:48:31.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd471800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd46c06bb30 0x7fd46c098a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45844/0 (socket says 172.21.15.66:45844) 2024-03-30T00:48:31.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd46be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd46c0cf2f0 0x7fd46c06b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd471800640 1 -- 172.21.15.66:0/1118081975 learned_addr learned my addr 172.21.15.66:0/1118081975 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:31.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd470e00640 1 --2- 172.21.15.66:0/1118081975 >> v2:172.21.15.156:3300/0 conn(0x7fd46c15ca80 0x7fd46c06b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd46be00640 1 -- 172.21.15.66:0/1118081975 >> v2:172.21.15.156:3300/0 conn(0x7fd46c15ca80 msgr2=0x7fd46c06b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:31.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd46be00640 1 --2- 172.21.15.66:0/1118081975 >> v2:172.21.15.156:3300/0 conn(0x7fd46c15ca80 0x7fd46c06b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd46be00640 1 -- 172.21.15.66:0/1118081975 >> v1:172.21.15.66:6789/0 conn(0x7fd46c06bb30 legacy=0x7fd46c098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.973+0000 7fd46be00640 1 -- 172.21.15.66:0/1118081975 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c0cad70 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd46be00640 1 --2- 172.21.15.66:0/1118081975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0cf2f0 0x7fd46c06b5f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fd46c070ed0 tx=0x7fd45801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198581eab2132ac3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:31.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd46b400640 1 -- 172.21.15.66:0/1118081975 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4580a8970 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd46b400640 1 -- 172.21.15.66:0/1118081975 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4580a8ad0 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd46b400640 1 -- 172.21.15.66:0/1118081975 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4580a8e60 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd473800640 1 -- 172.21.15.66:0/1118081975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0cf2f0 msgr2=0x7fd46c06b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd473800640 1 --2- 172.21.15.66:0/1118081975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0cf2f0 0x7fd46c06b5f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fd46c070ed0 tx=0x7fd45801ec70 comp rx=0 tx=0).stop 2024-03-30T00:48:31.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd473800640 1 -- 172.21.15.66:0/1118081975 shutdown_connections 2024-03-30T00:48:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd473800640 1 --2- 172.21.15.66:0/1118081975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0cf2f0 0x7fd46c06b5f0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd473800640 1 --2- 172.21.15.66:0/1118081975 >> v2:172.21.15.156:3300/0 conn(0x7fd46c15ca80 0x7fd46c06b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd473800640 1 -- 172.21.15.66:0/1118081975 >> 172.21.15.66:0/1118081975 conn(0x7fd46c0b2090 msgr2=0x7fd46c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd473800640 1 -- 172.21.15.66:0/1118081975 shutdown_connections 2024-03-30T00:48:31.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd473800640 1 -- 172.21.15.66:0/1118081975 wait complete. 2024-03-30T00:48:31.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.977+0000 7fd473800640 1 Processor -- start 2024-03-30T00:48:31.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd473800640 1 -- start start 2024-03-30T00:48:31.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd473800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd46c06bb30 0x7fd46c0c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd473800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0cf2f0 0x7fd46c0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd473800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd46c15ca80 0x7fd46c0c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd473800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0c9730 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd473800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c15d450 con 0x7fd46c06bb30 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd473800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0c9980 con 0x7fd46c15ca80 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd46be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0cf2f0 0x7fd46c0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd46be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0cf2f0 0x7fd46c0c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48496/0 (socket says 172.21.15.66:48496) 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd46be00640 1 -- 172.21.15.66:0/1441169483 learned_addr learned my addr 172.21.15.66:0/1441169483 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd470e00640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd46c06bb30 0x7fd46c0c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd471800640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd46c15ca80 0x7fd46c0c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd46be00640 1 -- 172.21.15.66:0/1441169483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd46c15ca80 msgr2=0x7fd46c0c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd46be00640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd46c15ca80 0x7fd46c0c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd46be00640 1 -- 172.21.15.66:0/1441169483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd46c06bb30 msgr2=0x7fd46c0c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:31.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd46be00640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd46c06bb30 0x7fd46c0c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:31.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd46be00640 1 -- 172.21.15.66:0/1441169483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd458009420 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd46be00640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0cf2f0 0x7fd46c0c7670 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fd45801b880 tx=0x7fd45801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:31.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd468c00640 1 -- 172.21.15.66:0/1441169483 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4580b9630 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd468c00640 1 -- 172.21.15.66:0/1441169483 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4580b97b0 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd473800640 1 -- 172.21.15.66:0/1441169483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46c0c9c60 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd468c00640 1 -- 172.21.15.66:0/1441169483 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4580b9b60 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd473800640 1 -- 172.21.15.66:0/1441169483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c0ca220 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.981+0000 7fd473800640 1 -- 172.21.15.66:0/1441169483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c0ca410 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.985+0000 7fd468c00640 1 -- 172.21.15.66:0/1441169483 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd4580b7040 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.985+0000 7fd468c00640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 0x7fd44c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.985+0000 7fd468c00640 1 -- 172.21.15.66:0/1441169483 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7fd4581224d0 con 0x7fd46c0cf2f0 2024-03-30T00:48:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.985+0000 7fd470e00640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 0x7fd44c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:31.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.985+0000 7fd470e00640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 0x7fd44c068b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0c9480 tx=0x7fd460008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:31.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:31.989+0000 7fd468c00640 1 -- 172.21.15.66:0/1441169483 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd4580ef710 con 0x7fd46c0cf2f0 2024-03-30T00:48:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.149+0000 7fd473800640 1 -- 172.21.15.66:0/1441169483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd46c066c30 con 0x7fd46c0cf2f0 2024-03-30T00:48:32.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.149+0000 7fd468c00640 1 -- 172.21.15.66:0/1441169483 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4580b50a0 con 0x7fd46c0cf2f0 2024-03-30T00:48:32.159 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:48:32.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.149+0000 7fd473800640 1 -- 172.21.15.66:0/1441169483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 msgr2=0x7fd44c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.149+0000 7fd473800640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 0x7fd44c068b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0c9480 tx=0x7fd460008040 comp rx=0 tx=0).stop 2024-03-30T00:48:32.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.149+0000 7fd473800640 1 -- 172.21.15.66:0/1441169483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0cf2f0 msgr2=0x7fd46c0c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.149+0000 7fd473800640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0cf2f0 0x7fd46c0c7670 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fd45801b880 tx=0x7fd45801ba10 comp rx=0 tx=0).stop 2024-03-30T00:48:32.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.153+0000 7fd473800640 1 -- 172.21.15.66:0/1441169483 shutdown_connections 2024-03-30T00:48:32.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.153+0000 7fd473800640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd46c15ca80 0x7fd46c0c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.153+0000 7fd473800640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd44c066690 0x7fd44c068b50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.153+0000 7fd473800640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd46c0cf2f0 0x7fd46c0c7670 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.153+0000 7fd473800640 1 --2- 172.21.15.66:0/1441169483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd46c06bb30 0x7fd46c0c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.153+0000 7fd473800640 1 -- 172.21.15.66:0/1441169483 >> 172.21.15.66:0/1441169483 conn(0x7fd46c0b2090 msgr2=0x7fd46c0cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:32.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.153+0000 7fd473800640 1 -- 172.21.15.66:0/1441169483 shutdown_connections 2024-03-30T00:48:32.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.153+0000 7fd473800640 1 -- 172.21.15.66:0/1441169483 wait complete. 2024-03-30T00:48:32.181 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:48:32.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.313+0000 7f0986600640 1 Processor -- start 2024-03-30T00:48:32.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.313+0000 7f0986600640 1 -- start start 2024-03-30T00:48:32.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.313+0000 7f0986600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f09800cde70 0x7f098015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:32.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.313+0000 7f0986600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09800cd3e0 con 0x7f09800cda90 2024-03-30T00:48:32.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.313+0000 7f0986600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098015a990 con 0x7f09800cec30 2024-03-30T00:48:32.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.313+0000 7f0986600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09800b1b80 con 0x7f09800cde70 2024-03-30T00:48:32.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.313+0000 7f097f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f09800cda90 0x7f09800ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45856/0 (socket says 172.21.15.66:45856) 2024-03-30T00:48:32.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.313+0000 7f097f400640 1 -- 172.21.15.66:0/2907009639 learned_addr learned my addr 172.21.15.66:0/2907009639 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:32.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.313+0000 7f097fe00640 1 --2- 172.21.15.66:0/2907009639 >> v2:172.21.15.66:3300/0 conn(0x7f09800cde70 0x7f098015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:32.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.313+0000 7f097ea00640 1 -- 172.21.15.66:0/2907009639 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653829925 0 0) 0x7f09800cd3e0 con 0x7f09800cda90 2024-03-30T00:48:32.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.313+0000 7f097ea00640 1 -- 172.21.15.66:0/2907009639 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0964005740 con 0x7f09800cda90 2024-03-30T00:48:32.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f097fe00640 1 -- 172.21.15.66:0/2907009639 >> v1:172.21.15.156:6789/0 conn(0x7f09800cec30 legacy=0x7f0980159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f097fe00640 1 -- 172.21.15.66:0/2907009639 >> v1:172.21.15.66:6789/0 conn(0x7f09800cda90 legacy=0x7f09800ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f097fe00640 1 -- 172.21.15.66:0/2907009639 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09800b11e0 con 0x7f09800cde70 2024-03-30T00:48:32.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f097fe00640 1 --2- 172.21.15.66:0/2907009639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09800cde70 0x7f098015a370 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f09780097c0 tx=0x7f097801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be642d564c3f714 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:32.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f097ea00640 1 -- 172.21.15.66:0/2907009639 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09780ab610 con 0x7f09800cde70 2024-03-30T00:48:32.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f097ea00640 1 -- 172.21.15.66:0/2907009639 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09780ab770 con 0x7f09800cde70 2024-03-30T00:48:32.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f097ea00640 1 -- 172.21.15.66:0/2907009639 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09780abb00 con 0x7f09800cde70 2024-03-30T00:48:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f0986600640 1 -- 172.21.15.66:0/2907009639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09800cde70 msgr2=0x7f098015a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f0986600640 1 --2- 172.21.15.66:0/2907009639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09800cde70 0x7f098015a370 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f09780097c0 tx=0x7f097801c410 comp rx=0 tx=0).stop 2024-03-30T00:48:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f0986600640 1 -- 172.21.15.66:0/2907009639 shutdown_connections 2024-03-30T00:48:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f0986600640 1 --2- 172.21.15.66:0/2907009639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09800cde70 0x7f098015a370 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f0986600640 1 -- 172.21.15.66:0/2907009639 >> 172.21.15.66:0/2907009639 conn(0x7f0980069f60 msgr2=0x7f098006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f0986600640 1 -- 172.21.15.66:0/2907009639 shutdown_connections 2024-03-30T00:48:32.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.317+0000 7f0986600640 1 -- 172.21.15.66:0/2907009639 wait complete. 2024-03-30T00:48:32.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0986600640 1 Processor -- start 2024-03-30T00:48:32.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0986600640 1 -- start start 2024-03-30T00:48:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0986600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09800cda90 0x7f09800cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0986600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09800cde70 0x7f09800aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0986600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09800cec30 0x7f098015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0986600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09801f8ee0 con 0x7f09800cda90 2024-03-30T00:48:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0986600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09801f82d0 con 0x7f09800cec30 2024-03-30T00:48:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0986600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09801f85b0 con 0x7f09800cde70 2024-03-30T00:48:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f097f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09800cda90 0x7f09800cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:32.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f097f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09800cda90 0x7f09800cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48502/0 (socket says 172.21.15.66:48502) 2024-03-30T00:48:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f097f400640 1 -- 172.21.15.66:0/788191565 learned_addr learned my addr 172.21.15.66:0/788191565 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f097fe00640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09800cec30 0x7f098015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0976a00640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09800cde70 0x7f09800aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f097f400640 1 -- 172.21.15.66:0/788191565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09800cde70 msgr2=0x7f09800aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f097f400640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09800cde70 0x7f09800aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f097f400640 1 -- 172.21.15.66:0/788191565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09800cec30 msgr2=0x7f098015b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f097f400640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09800cec30 0x7f098015b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f097f400640 1 -- 172.21.15.66:0/788191565 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0978009420 con 0x7f09800cda90 2024-03-30T00:48:32.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f097f400640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09800cda90 0x7f09800cc760 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f097000cbc0 tx=0x7f097000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:32.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0977e00640 1 -- 172.21.15.66:0/788191565 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097000ad40 con 0x7f09800cda90 2024-03-30T00:48:32.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0977e00640 1 -- 172.21.15.66:0/788191565 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0970012400 con 0x7f09800cda90 2024-03-30T00:48:32.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0977e00640 1 -- 172.21.15.66:0/788191565 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097000aea0 con 0x7f09800cda90 2024-03-30T00:48:32.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0986600640 1 -- 172.21.15.66:0/788191565 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09801f8890 con 0x7f09800cda90 2024-03-30T00:48:32.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.321+0000 7f0986600640 1 -- 172.21.15.66:0/788191565 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09800ac350 con 0x7f09800cda90 2024-03-30T00:48:32.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.325+0000 7f0986600640 1 -- 172.21.15.66:0/788191565 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0980066c30 con 0x7f09800cda90 2024-03-30T00:48:32.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.325+0000 7f0977e00640 1 -- 172.21.15.66:0/788191565 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0970007070 con 0x7f09800cda90 2024-03-30T00:48:32.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.325+0000 7f0977e00640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f095c066690 0x7f095c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:32.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.325+0000 7f0977e00640 1 -- 172.21.15.66:0/788191565 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f09700843c0 con 0x7f09800cda90 2024-03-30T00:48:32.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.325+0000 7f0976a00640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f095c066690 0x7f095c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:32.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.325+0000 7f0976a00640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f095c066690 0x7f095c068b50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f09800ac930 tx=0x7f0968009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:32.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.329+0000 7f0977e00640 1 -- 172.21.15.66:0/788191565 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f09700514e0 con 0x7f09800cda90 2024-03-30T00:48:32.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.521+0000 7f0986600640 1 -- 172.21.15.66:0/788191565 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09800b3170 con 0x7f09800cda90 2024-03-30T00:48:32.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0977e00640 1 -- 172.21.15.66:0/788191565 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f0970054b00 con 0x7f09800cda90 2024-03-30T00:48:32.534 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:32.534 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:47:53.284661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/2419209461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2419209461},{"type":"v1","addr":"172.21.15.66:6836","nonce":2419209461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:48:32.534 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 71 2024-03-30T00:48:32.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 -- 172.21.15.66:0/788191565 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f095c066690 msgr2=0x7f095c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f095c066690 0x7f095c068b50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f09800ac930 tx=0x7f0968009040 comp rx=0 tx=0).stop 2024-03-30T00:48:32.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 -- 172.21.15.66:0/788191565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09800cda90 msgr2=0x7f09800cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09800cda90 0x7f09800cc760 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f097000cbc0 tx=0x7f097000a430 comp rx=0 tx=0).stop 2024-03-30T00:48:32.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 -- 172.21.15.66:0/788191565 shutdown_connections 2024-03-30T00:48:32.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09800cec30 0x7f098015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09800cde70 0x7f09800aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f095c066690 0x7f095c068b50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 --2- 172.21.15.66:0/788191565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09800cda90 0x7f09800cc760 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 -- 172.21.15.66:0/788191565 >> 172.21.15.66:0/788191565 conn(0x7f0980069f60 msgr2=0x7f09800ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:32.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 -- 172.21.15.66:0/788191565 shutdown_connections 2024-03-30T00:48:32.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.525+0000 7f0986600640 1 -- 172.21.15.66:0/788191565 wait complete. 2024-03-30T00:48:32.554 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:48:32.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ec000640 1 Processor -- start 2024-03-30T00:48:32.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ec000640 1 -- start start 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ec000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f53e40cda90 0x7f53e40ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ec000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53e415a370 0x7f53e40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ec000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e415a990 con 0x7f53e40cde70 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ec000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40b1b80 con 0x7f53e415a370 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ec000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40cd3e0 con 0x7f53e40cda90 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ea000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53e415a370 0x7f53e40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ea000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53e415a370 0x7f53e40cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48506/0 (socket says 172.21.15.66:48506) 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ea000640 1 -- 172.21.15.66:0/2909138317 learned_addr learned my addr 172.21.15.66:0/2909138317 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53e9600640 1 --2- 172.21.15.66:0/2909138317 >> v2:172.21.15.156:3300/0 conn(0x7f53e40cda90 0x7f53e40ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ea000640 1 -- 172.21.15.66:0/2909138317 >> v2:172.21.15.156:3300/0 conn(0x7f53e40cda90 msgr2=0x7f53e40ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ea000640 1 --2- 172.21.15.66:0/2909138317 >> v2:172.21.15.156:3300/0 conn(0x7f53e40cda90 0x7f53e40ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ea000640 1 -- 172.21.15.66:0/2909138317 >> v1:172.21.15.156:6789/0 conn(0x7f53e40cde70 legacy=0x7f53e4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:32.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.685+0000 7f53ea000640 1 -- 172.21.15.66:0/2909138317 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e40b11e0 con 0x7f53e415a370 2024-03-30T00:48:32.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53ea000640 1 --2- 172.21.15.66:0/2909138317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e415a370 0x7f53e40cea60 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f53e40cf3f0 tx=0x7f53dc01eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb81b04ed0e1dd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:32.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53e3e00640 1 -- 172.21.15.66:0/2909138317 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00b2d0 con 0x7f53e415a370 2024-03-30T00:48:32.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53e3e00640 1 -- 172.21.15.66:0/2909138317 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc00b430 con 0x7f53e415a370 2024-03-30T00:48:32.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53e3e00640 1 -- 172.21.15.66:0/2909138317 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0a7b20 con 0x7f53e415a370 2024-03-30T00:48:32.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53ec000640 1 -- 172.21.15.66:0/2909138317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e415a370 msgr2=0x7f53e40cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53ec000640 1 --2- 172.21.15.66:0/2909138317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e415a370 0x7f53e40cea60 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f53e40cf3f0 tx=0x7f53dc01eb20 comp rx=0 tx=0).stop 2024-03-30T00:48:32.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53ec000640 1 -- 172.21.15.66:0/2909138317 shutdown_connections 2024-03-30T00:48:32.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53ec000640 1 --2- 172.21.15.66:0/2909138317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e415a370 0x7f53e40cea60 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53ec000640 1 --2- 172.21.15.66:0/2909138317 >> v2:172.21.15.156:3300/0 conn(0x7f53e40cda90 0x7f53e40ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53ec000640 1 -- 172.21.15.66:0/2909138317 >> 172.21.15.66:0/2909138317 conn(0x7f53e4069f60 msgr2=0x7f53e406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:32.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53ec000640 1 -- 172.21.15.66:0/2909138317 shutdown_connections 2024-03-30T00:48:32.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.689+0000 7f53ec000640 1 -- 172.21.15.66:0/2909138317 wait complete. 2024-03-30T00:48:32.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53ec000640 1 Processor -- start 2024-03-30T00:48:32.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53ec000640 1 -- start start 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53ec000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e40cda90 0x7f53e4155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53ec000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e40cde70 0x7f53e4158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53ec000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e415a370 0x7f53e41f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53ec000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4070a20 con 0x7f53e40cde70 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53ec000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40c7a30 con 0x7f53e40cda90 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53ec000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40c7750 con 0x7f53e415a370 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e40cde70 0x7f53e4158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e40cde70 0x7f53e4158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48518/0 (socket says 172.21.15.66:48518) 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e8c00640 1 -- 172.21.15.66:0/2579930985 learned_addr learned my addr 172.21.15.66:0/2579930985 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53ea000640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e415a370 0x7f53e41f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e9600640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e40cda90 0x7f53e4155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:32.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e8c00640 1 -- 172.21.15.66:0/2579930985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e415a370 msgr2=0x7f53e41f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e8c00640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e415a370 0x7f53e41f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e8c00640 1 -- 172.21.15.66:0/2579930985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e40cda90 msgr2=0x7f53e4155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e8c00640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e40cda90 0x7f53e4155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e8c00640 1 -- 172.21.15.66:0/2579930985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc00bd40 con 0x7f53e40cde70 2024-03-30T00:48:32.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e8c00640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e40cde70 0x7f53e4158560 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f53d000c410 tx=0x7f53d000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:32.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e1600640 1 -- 172.21.15.66:0/2579930985 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0011790 con 0x7f53e40cde70 2024-03-30T00:48:32.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e1600640 1 -- 172.21.15.66:0/2579930985 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d00118f0 con 0x7f53e40cde70 2024-03-30T00:48:32.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53ec000640 1 -- 172.21.15.66:0/2579930985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e40703d0 con 0x7f53e40cde70 2024-03-30T00:48:32.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53e1600640 1 -- 172.21.15.66:0/2579930985 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0011ca0 con 0x7f53e40cde70 2024-03-30T00:48:32.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.693+0000 7f53ec000640 1 -- 172.21.15.66:0/2579930985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e4158aa0 con 0x7f53e40cde70 2024-03-30T00:48:32.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.697+0000 7f53ec000640 1 -- 172.21.15.66:0/2579930985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e4158c60 con 0x7f53e40cde70 2024-03-30T00:48:32.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.697+0000 7f53e1600640 1 -- 172.21.15.66:0/2579930985 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f53d001c460 con 0x7f53e40cde70 2024-03-30T00:48:32.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.697+0000 7f53e1600640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53c4066690 0x7f53c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:32.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.697+0000 7f53e1600640 1 -- 172.21.15.66:0/2579930985 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f53d0083fe0 con 0x7f53e40cde70 2024-03-30T00:48:32.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.697+0000 7f53e9600640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53c4066690 0x7f53c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:32.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.697+0000 7f53e9600640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53c4066690 0x7f53c4068b50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f53e40c1a10 tx=0x7f53d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:32.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.701+0000 7f53e1600640 1 -- 172.21.15.66:0/2579930985 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f53d0051140 con 0x7f53e40cde70 2024-03-30T00:48:32.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.865+0000 7f53ec000640 1 -- 172.21.15.66:0/2579930985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53e4066c30 con 0x7f53e40cde70 2024-03-30T00:48:32.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.865+0000 7f53e1600640 1 -- 172.21.15.66:0/2579930985 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f53d0054760 con 0x7f53e40cde70 2024-03-30T00:48:32.875 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:32.875 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":98,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:47:49.264161+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:46:13.811742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-30T00:46:14.265142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":7}]},{"pgid":"14.1a","mappings":[{"from":1,"to":5}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/1626022080":"2024-03-31T00:47:49.264137+0000","172.21.15.66:6836/1626022080":"2024-03-31T00:47:49.264137+0000","172.21.15.66:6836/1147664160":"2024-03-31T00:46:34.103831+0000","172.21.15.66:6835/1147664160":"2024-03-31T00:46:34.103831+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:48:32.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.865+0000 7f53ec000640 1 -- 172.21.15.66:0/2579930985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53c4066690 msgr2=0x7f53c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.865+0000 7f53ec000640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53c4066690 0x7f53c4068b50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f53e40c1a10 tx=0x7f53d8008040 comp rx=0 tx=0).stop 2024-03-30T00:48:32.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.865+0000 7f53ec000640 1 -- 172.21.15.66:0/2579930985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e40cde70 msgr2=0x7f53e4158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:32.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.865+0000 7f53ec000640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e40cde70 0x7f53e4158560 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f53d000c410 tx=0x7f53d000c8d0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.869+0000 7f53ec000640 1 -- 172.21.15.66:0/2579930985 shutdown_connections 2024-03-30T00:48:32.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.869+0000 7f53ec000640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e415a370 0x7f53e41f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.869+0000 7f53ec000640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53c4066690 0x7f53c4068b50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.869+0000 7f53ec000640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e40cde70 0x7f53e4158560 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.869+0000 7f53ec000640 1 --2- 172.21.15.66:0/2579930985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e40cda90 0x7f53e4155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:32.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.869+0000 7f53ec000640 1 -- 172.21.15.66:0/2579930985 >> 172.21.15.66:0/2579930985 conn(0x7f53e4069f60 msgr2=0x7f53e415c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:32.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.869+0000 7f53ec000640 1 -- 172.21.15.66:0/2579930985 shutdown_connections 2024-03-30T00:48:32.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:32.869+0000 7f53ec000640 1 -- 172.21.15.66:0/2579930985 wait complete. 2024-03-30T00:48:32.897 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T00:48:32.898 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:48:33.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.029+0000 7f5190200640 1 Processor -- start 2024-03-30T00:48:33.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.029+0000 7f5190200640 1 -- start start 2024-03-30T00:48:33.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f5190200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f518815ca80 0x7f518806b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f5190200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51880cb570 0x7f51880cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f5190200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51880cb0e0 con 0x7f51880cf2f0 2024-03-30T00:48:33.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f5190200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51880cf7f0 con 0x7f518815ca80 2024-03-30T00:48:33.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f5190200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51880cfad0 con 0x7f51880cb570 2024-03-30T00:48:33.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f518ce00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f51880cf2f0 0x7f518806b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45870/0 (socket says 172.21.15.66:45870) 2024-03-30T00:48:33.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f518ce00640 1 -- 172.21.15.66:0/1336595286 learned_addr learned my addr 172.21.15.66:0/1336595286 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:33.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f518e200640 1 --2- 172.21.15.66:0/1336595286 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51880cb570 0x7f51880cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f5187e00640 1 -- 172.21.15.66:0/1336595286 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579359381 0 0) 0x7f51880cb0e0 con 0x7f51880cf2f0 2024-03-30T00:48:33.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f518d800640 1 --2- 172.21.15.66:0/1336595286 >> v2:172.21.15.156:3300/0 conn(0x7f518815ca80 0x7f518806b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f5187e00640 1 -- 172.21.15.66:0/1336595286 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5170003530 con 0x7f51880cf2f0 2024-03-30T00:48:33.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f518e200640 1 -- 172.21.15.66:0/1336595286 >> v2:172.21.15.156:3300/0 conn(0x7f518815ca80 msgr2=0x7f518806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:33.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f518e200640 1 --2- 172.21.15.66:0/1336595286 >> v2:172.21.15.156:3300/0 conn(0x7f518815ca80 0x7f518806b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f518e200640 1 -- 172.21.15.66:0/1336595286 >> v1:172.21.15.66:6789/0 conn(0x7f51880cf2f0 legacy=0x7f518806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f518e200640 1 -- 172.21.15.66:0/1336595286 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51880cad70 con 0x7f51880cb570 2024-03-30T00:48:33.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.033+0000 7f518d800640 1 --2- 172.21.15.66:0/1336595286 >> v2:172.21.15.156:3300/0 conn(0x7f518815ca80 0x7f518806b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:33.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f518e200640 1 --2- 172.21.15.66:0/1336595286 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51880cb570 0x7f51880cba20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5180009d90 tx=0x7f518001f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d14df172e3190b6d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:33.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5187e00640 1 -- 172.21.15.66:0/1336595286 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800af600 con 0x7f51880cb570 2024-03-30T00:48:33.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5187e00640 1 -- 172.21.15.66:0/1336595286 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51800ab050 con 0x7f51880cb570 2024-03-30T00:48:33.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5187e00640 1 -- 172.21.15.66:0/1336595286 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800af980 con 0x7f51880cb570 2024-03-30T00:48:33.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 -- 172.21.15.66:0/1336595286 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51880cb570 msgr2=0x7f51880cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 --2- 172.21.15.66:0/1336595286 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51880cb570 0x7f51880cba20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5180009d90 tx=0x7f518001f150 comp rx=0 tx=0).stop 2024-03-30T00:48:33.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 -- 172.21.15.66:0/1336595286 shutdown_connections 2024-03-30T00:48:33.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 --2- 172.21.15.66:0/1336595286 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51880cb570 0x7f51880cba20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 --2- 172.21.15.66:0/1336595286 >> v2:172.21.15.156:3300/0 conn(0x7f518815ca80 0x7f518806b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 -- 172.21.15.66:0/1336595286 >> 172.21.15.66:0/1336595286 conn(0x7f51880b2090 msgr2=0x7f51880b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:33.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 -- 172.21.15.66:0/1336595286 shutdown_connections 2024-03-30T00:48:33.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 -- 172.21.15.66:0/1336595286 wait complete. 2024-03-30T00:48:33.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 Processor -- start 2024-03-30T00:48:33.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 -- start start 2024-03-30T00:48:33.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51880cb570 0x7f51880b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880cf2f0 0x7f51881f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518815ca80 0x7f5188154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51881f2f30 con 0x7f51880cb570 2024-03-30T00:48:33.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518809c880 con 0x7f51880cf2f0 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.037+0000 7f5190200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5188154fb0 con 0x7f518815ca80 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f518d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51880cb570 0x7f51880b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f518d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51880cb570 0x7f51880b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48524/0 (socket says 172.21.15.66:48524) 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f518d800640 1 -- 172.21.15.66:0/2456545362 learned_addr learned my addr 172.21.15.66:0/2456545362 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f518ce00640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880cf2f0 0x7f51881f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f518e200640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518815ca80 0x7f5188154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f518d800640 1 -- 172.21.15.66:0/2456545362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518815ca80 msgr2=0x7f5188154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f518d800640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518815ca80 0x7f5188154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f518d800640 1 -- 172.21.15.66:0/2456545362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880cf2f0 msgr2=0x7f51881f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f518d800640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880cf2f0 0x7f51881f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f518d800640 1 -- 172.21.15.66:0/2456545362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5180009a70 con 0x7f51880cb570 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f518d800640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51880cb570 0x7f51880b5930 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f517c00c7f0 tx=0x7f517c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:33.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f5185600640 1 -- 172.21.15.66:0/2456545362 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c007820 con 0x7f51880cb570 2024-03-30T00:48:33.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f5185600640 1 -- 172.21.15.66:0/2456545362 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c007980 con 0x7f51880cb570 2024-03-30T00:48:33.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f5185600640 1 -- 172.21.15.66:0/2456545362 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c007d30 con 0x7f51880cb570 2024-03-30T00:48:33.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f5190200640 1 -- 172.21.15.66:0/2456545362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51881f2970 con 0x7f51880cb570 2024-03-30T00:48:33.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f5190200640 1 -- 172.21.15.66:0/2456545362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518809cb00 con 0x7f51880cb570 2024-03-30T00:48:33.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f5190200640 1 -- 172.21.15.66:0/2456545362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5188066c30 con 0x7f51880cb570 2024-03-30T00:48:33.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.041+0000 7f5185600640 1 -- 172.21.15.66:0/2456545362 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f517c0113f0 con 0x7f51880cb570 2024-03-30T00:48:33.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.045+0000 7f5185600640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 0x7f5168068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.045+0000 7f5185600640 1 -- 172.21.15.66:0/2456545362 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f517c084860 con 0x7f51880cb570 2024-03-30T00:48:33.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.045+0000 7f518ce00640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 0x7f5168068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.045+0000 7f518ce00640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 0x7f5168068b50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f518809c670 tx=0x7f5174009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:33.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.045+0000 7f5185600640 1 -- 172.21.15.66:0/2456545362 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f517c0518d0 con 0x7f51880cb570 2024-03-30T00:48:33.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.241+0000 7f5190200640 1 -- 172.21.15.66:0/2456545362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51880b2bb0 con 0x7f51880cb570 2024-03-30T00:48:33.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.241+0000 7f5185600640 1 -- 172.21.15.66:0/2456545362 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f517c054ef0 con 0x7f51880cb570 2024-03-30T00:48:33.253 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:33.253 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:47:53.284661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/2419209461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2419209461},{"type":"v1","addr":"172.21.15.66:6836","nonce":2419209461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:48:33.253 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 71 2024-03-30T00:48:33.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 -- 172.21.15.66:0/2456545362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 msgr2=0x7f5168068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 0x7f5168068b50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f518809c670 tx=0x7f5174009040 comp rx=0 tx=0).stop 2024-03-30T00:48:33.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 -- 172.21.15.66:0/2456545362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51880cb570 msgr2=0x7f51880b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51880cb570 0x7f51880b5930 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f517c00c7f0 tx=0x7f517c00ccb0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 -- 172.21.15.66:0/2456545362 shutdown_connections 2024-03-30T00:48:33.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f518815ca80 0x7f5188154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880cf2f0 0x7f51881f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5168066690 0x7f5168068b50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 --2- 172.21.15.66:0/2456545362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51880cb570 0x7f51880b5930 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 -- 172.21.15.66:0/2456545362 >> 172.21.15.66:0/2456545362 conn(0x7f51880b2090 msgr2=0x7f51880cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:33.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 -- 172.21.15.66:0/2456545362 shutdown_connections 2024-03-30T00:48:33.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.245+0000 7f5190200640 1 -- 172.21.15.66:0/2456545362 wait complete. 2024-03-30T00:48:33.273 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:48:33.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cd1a00640 1 Processor -- start 2024-03-30T00:48:33.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cd1a00640 1 -- start start 2024-03-30T00:48:33.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cd1a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ccc0ce520 0x7f1ccc0ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cd1a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ccc06d070 con 0x7f1ccc15a640 2024-03-30T00:48:33.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cd1a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ccc0cbba0 con 0x7f1ccc0cabb0 2024-03-30T00:48:33.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cd1a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0cee40 con 0x7f1ccc0ce520 2024-03-30T00:48:33.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1ccaa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1ccc15a640 0x7f1ccc06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45876/0 (socket says 172.21.15.66:45876) 2024-03-30T00:48:33.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1ccaa00640 1 -- 172.21.15.66:0/3618007933 learned_addr learned my addr 172.21.15.66:0/3618007933 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:33.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cc2000640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ccc0ce520 0x7f1ccc0ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585025340 0 0) 0x7f1ccc06d070 con 0x7f1ccc15a640 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb0003530 con 0x7f1ccc15a640 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3591669247 0 0) 0x7f1cb0003530 con 0x7f1ccc15a640 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ccc0ce520 msgr2=0x7f1ccc0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ccc0ce520 0x7f1ccc0ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 >> v1:172.21.15.156:6789/0 conn(0x7f1ccc0cabb0 legacy=0x7f1ccc0cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc0cb8c0 con 0x7f1ccc15a640 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f1cbc001680 con 0x7f1ccc15a640 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 >> v1:172.21.15.66:6789/0 conn(0x7f1ccc15a640 legacy=0x7f1ccc06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cb00039b0 0x7f1cb0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb0004360 0x7f1cb0004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ccc0ce520 0x7f1cb0005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0cee40 con 0x7f1cb00039b0 2024-03-30T00:48:33.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb000b9c0 con 0x7f1cb0004360 2024-03-30T00:48:33.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb000bb80 con 0x7f1ccc0ce520 2024-03-30T00:48:33.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1ccaa00640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cb00039b0 0x7f1cb0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1cc2000640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb0004360 0x7f1cb0004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1ccb400640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ccc0ce520 0x7f1cb0005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1ccaa00640 1 -- 172.21.15.66:0/3618007933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ccc0ce520 msgr2=0x7f1cb0005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:33.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1ccaa00640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1ccc0ce520 0x7f1cb0005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1ccaa00640 1 -- 172.21.15.66:0/3618007933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb0004360 msgr2=0x7f1cb0004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:33.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1ccaa00640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cb0004360 0x7f1cb0004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.405+0000 7f1ccaa00640 1 -- 172.21.15.66:0/3618007933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb0003530 con 0x7f1cb00039b0 2024-03-30T00:48:33.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.409+0000 7f1ccaa00640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cb00039b0 0x7f1cb0003d90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00c190 tx=0x7f1cbc01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995afc06e346d7df server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:33.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.409+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc005ae0 con 0x7f1cb00039b0 2024-03-30T00:48:33.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.409+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc005c60 con 0x7f1cb00039b0 2024-03-30T00:48:33.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.409+0000 7f1cca000640 1 -- 172.21.15.66:0/3618007933 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0072b0 con 0x7f1cb00039b0 2024-03-30T00:48:33.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.409+0000 7f1cd1a00640 1 -- 172.21.15.66:0/3618007933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cb00039b0 msgr2=0x7f1cb0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.409+0000 7f1cd1a00640 1 --2- 172.21.15.66:0/3618007933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cb00039b0 0x7f1cb0003d90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00c190 tx=0x7f1cbc01cc00 comp rx=0 tx=0).stop 2024-03-30T00:48:33.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.409+0000 7f1ccb400640 1 -- 172.21.15.66:0/3618007933 reap_dead start 2024-03-30T00:48:33.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.409+0000 7f1cd1a00640 1 -- 172.21.15.66:0/3618007933 shutdown_connections 2024-03-30T00:48:33.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.409+0000 7f1cd1a00640 1 -- 172.21.15.66:0/3618007933 >> 172.21.15.66:0/3618007933 conn(0x7f1ccc069f60 msgr2=0x7f1ccc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:33.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.409+0000 7f1cd1a00640 1 -- 172.21.15.66:0/3618007933 shutdown_connections 2024-03-30T00:48:33.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.409+0000 7f1cd1a00640 1 -- 172.21.15.66:0/3618007933 wait complete. 2024-03-30T00:48:33.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cd1a00640 1 Processor -- start 2024-03-30T00:48:33.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cd1a00640 1 -- start start 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cd1a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1cb00039b0 0x7f1ccc0c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cd1a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ccc0c6fa0 0x7f1ccc1f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cd1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ccc0cced0 0x7f1ccc0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cd1a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc157ab0 con 0x7f1ccc0cced0 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cd1a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0cd890 con 0x7f1ccc0c6fa0 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cd1a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0c9ec0 con 0x7f1cb00039b0 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1ccb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ccc0cced0 0x7f1ccc0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1ccb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ccc0cced0 0x7f1ccc0cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48546/0 (socket says 172.21.15.66:48546) 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1ccb400640 1 -- 172.21.15.66:0/876633040 learned_addr learned my addr 172.21.15.66:0/876633040 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1ccb400640 1 -- 172.21.15.66:0/876633040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1cb00039b0 msgr2=0x7f1ccc0c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1ccb400640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1cb00039b0 0x7f1ccc0c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1ccb400640 1 -- 172.21.15.66:0/876633040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ccc0c6fa0 msgr2=0x7f1ccc1f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1ccb400640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ccc0c6fa0 0x7f1ccc1f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1ccb400640 1 -- 172.21.15.66:0/876633040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc0c7560 con 0x7f1ccc0cced0 2024-03-30T00:48:33.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1ccb400640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ccc0cced0 0x7f1ccc0cd350 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0b1810 tx=0x7f1cc400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:33.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cc3400640 1 -- 172.21.15.66:0/876633040 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4006d70 con 0x7f1ccc0cced0 2024-03-30T00:48:33.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cc3400640 1 -- 172.21.15.66:0/876633040 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc4003980 con 0x7f1ccc0cced0 2024-03-30T00:48:33.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cd1a00640 1 -- 172.21.15.66:0/876633040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ccc0c7840 con 0x7f1ccc0cced0 2024-03-30T00:48:33.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cc3400640 1 -- 172.21.15.66:0/876633040 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4003d30 con 0x7f1ccc0cced0 2024-03-30T00:48:33.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.413+0000 7f1cd1a00640 1 -- 172.21.15.66:0/876633040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ccc0c7e00 con 0x7f1ccc0cced0 2024-03-30T00:48:33.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.417+0000 7f1cd1a00640 1 -- 172.21.15.66:0/876633040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ccc066c30 con 0x7f1ccc0cced0 2024-03-30T00:48:33.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.417+0000 7f1cc3400640 1 -- 172.21.15.66:0/876633040 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1cc400d800 con 0x7f1ccc0cced0 2024-03-30T00:48:33.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.417+0000 7f1cc3400640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ca80666c0 0x7f1ca8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.417+0000 7f1cc3400640 1 -- 172.21.15.66:0/876633040 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f1cc4083a20 con 0x7f1ccc0cced0 2024-03-30T00:48:33.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.417+0000 7f1ccaa00640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ca80666c0 0x7f1ca8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.417+0000 7f1ccaa00640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ca80666c0 0x7f1ca8068b80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0ab000 tx=0x7f1cbc001990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:33.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.421+0000 7f1cc3400640 1 -- 172.21.15.66:0/876633040 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1cc4050b20 con 0x7f1ccc0cced0 2024-03-30T00:48:33.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.617+0000 7f1cd1a00640 1 -- 172.21.15.66:0/876633040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ccc0adc60 con 0x7f1ccc0cced0 2024-03-30T00:48:33.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.617+0000 7f1cc3400640 1 -- 172.21.15.66:0/876633040 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f1cc4019090 con 0x7f1ccc0cced0 2024-03-30T00:48:33.628 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:33.628 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:46:15.271965+0000","modified":"2024-03-30T00:47:53.284661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/2419209461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2419209461},{"type":"v1","addr":"172.21.15.66:6836","nonce":2419209461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-30T00:48:33.628 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 71 2024-03-30T00:48:33.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 -- 172.21.15.66:0/876633040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ca80666c0 msgr2=0x7f1ca8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ca80666c0 0x7f1ca8068b80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0ab000 tx=0x7f1cbc001990 comp rx=0 tx=0).stop 2024-03-30T00:48:33.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 -- 172.21.15.66:0/876633040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ccc0cced0 msgr2=0x7f1ccc0cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ccc0cced0 0x7f1ccc0cd350 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0b1810 tx=0x7f1cc400ff70 comp rx=0 tx=0).stop 2024-03-30T00:48:33.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 -- 172.21.15.66:0/876633040 shutdown_connections 2024-03-30T00:48:33.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1ca80666c0 0x7f1ca8068b80 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ccc0cced0 0x7f1ccc0cd350 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ccc0c6fa0 0x7f1ccc1f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 --2- 172.21.15.66:0/876633040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1cb00039b0 0x7f1ccc0c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 -- 172.21.15.66:0/876633040 >> 172.21.15.66:0/876633040 conn(0x7f1ccc069f60 msgr2=0x7f1ccc0aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:33.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 -- 172.21.15.66:0/876633040 shutdown_connections 2024-03-30T00:48:33.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.621+0000 7f1cd1a00640 1 -- 172.21.15.66:0/876633040 wait complete. 2024-03-30T00:48:33.650 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:48:33.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4087e00640 1 Processor -- start 2024-03-30T00:48:33.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4087e00640 1 -- start start 2024-03-30T00:48:33.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4087e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f408015de30 0x7f40800cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4087e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40800af420 0x7f40800d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4087e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40800cbb90 con 0x7f40800ce740 2024-03-30T00:48:33.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4087e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40800ce1b0 con 0x7f408015de30 2024-03-30T00:48:33.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4087e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40800afa10 con 0x7f40800af420 2024-03-30T00:48:33.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4085e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f40800ce740 0x7f408006e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45882/0 (socket says 172.21.15.66:45882) 2024-03-30T00:48:33.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4085e00640 1 -- 172.21.15.66:0/232878186 learned_addr learned my addr 172.21.15.66:0/232878186 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:33.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4084a00640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40800af420 0x7f40800d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4085400640 1 --2- 172.21.15.66:0/232878186 >> v2:172.21.15.156:3300/0 conn(0x7f408015de30 0x7f40800cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38371591 0 0) 0x7f40800cbb90 con 0x7f40800ce740 2024-03-30T00:48:33.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4068003530 con 0x7f40800ce740 2024-03-30T00:48:33.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1092195189 0 0) 0x7f4068003530 con 0x7f40800ce740 2024-03-30T00:48:33.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40800af420 msgr2=0x7f40800d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:33.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40800af420 0x7f40800d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 >> v2:172.21.15.156:3300/0 conn(0x7f408015de30 msgr2=0x7f40800cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:33.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 --2- 172.21.15.66:0/232878186 >> v2:172.21.15.156:3300/0 conn(0x7f408015de30 0x7f40800cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40800cf430 con 0x7f40800ce740 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f4078001a00 con 0x7f40800ce740 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4085400640 1 --2- 172.21.15.66:0/232878186 >> v2:172.21.15.156:3300/0 conn(0x7f408015de30 0x7f40800cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 >> v1:172.21.15.66:6789/0 conn(0x7f40800ce740 legacy=0x7f408006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 0x7f4068003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40680047d0 0x7f4068004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40800af420 0x7f4068005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40800ce1b0 con 0x7f4068003980 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40800afa10 con 0x7f40680047d0 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406800b9c0 con 0x7f40800af420 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.781+0000 7f4085400640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 0x7f4068003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f4084a00640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40680047d0 0x7f4068004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f4085e00640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40800af420 0x7f4068005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f4085400640 1 -- 172.21.15.66:0/232878186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40800af420 msgr2=0x7f4068005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:33.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f4085400640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40800af420 0x7f4068005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f4085400640 1 -- 172.21.15.66:0/232878186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40680047d0 msgr2=0x7f4068004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:33.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f4085400640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40680047d0 0x7f4068004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f4085400640 1 -- 172.21.15.66:0/232878186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4068003530 con 0x7f4068003980 2024-03-30T00:48:33.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f4085400640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 0x7f4068003da0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f4074009cb0 tx=0x7f407401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5213bea754462db5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:33.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40740a5760 con 0x7f4068003980 2024-03-30T00:48:33.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40740a58c0 con 0x7f4068003980 2024-03-30T00:48:33.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f407fe00640 1 -- 172.21.15.66:0/232878186 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40740a5c30 con 0x7f4068003980 2024-03-30T00:48:33.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f4087e00640 1 -- 172.21.15.66:0/232878186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 msgr2=0x7f4068003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f4087e00640 1 --2- 172.21.15.66:0/232878186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 0x7f4068003da0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f4074009cb0 tx=0x7f407401cc30 comp rx=0 tx=0).stop 2024-03-30T00:48:33.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.785+0000 7f4085e00640 1 -- 172.21.15.66:0/232878186 reap_dead start 2024-03-30T00:48:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 -- 172.21.15.66:0/232878186 shutdown_connections 2024-03-30T00:48:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 -- 172.21.15.66:0/232878186 >> 172.21.15.66:0/232878186 conn(0x7f40800d0540 msgr2=0x7f40800d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 -- 172.21.15.66:0/232878186 shutdown_connections 2024-03-30T00:48:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 -- 172.21.15.66:0/232878186 wait complete. 2024-03-30T00:48:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 Processor -- start 2024-03-30T00:48:33.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 -- start start 2024-03-30T00:48:33.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 0x7f4080164a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f408015acd0 0x7f408015b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f408014e590 0x7f408014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408015c560 con 0x7f4068003980 2024-03-30T00:48:33.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40800ca3e0 con 0x7f408015acd0 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4087e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408015c980 con 0x7f408014e590 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4085400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 0x7f4080164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4085400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 0x7f4080164a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48572/0 (socket says 172.21.15.66:48572) 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4085400640 1 -- 172.21.15.66:0/2547453267 learned_addr learned my addr 172.21.15.66:0/2547453267 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4084a00640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f408015acd0 0x7f408015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4085e00640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f408014e590 0x7f408014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4085400640 1 -- 172.21.15.66:0/2547453267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f408014e590 msgr2=0x7f408014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4085400640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f408014e590 0x7f408014ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4085400640 1 -- 172.21.15.66:0/2547453267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f408015acd0 msgr2=0x7f408015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4085400640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f408015acd0 0x7f408015b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4085400640 1 -- 172.21.15.66:0/2547453267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408015b690 con 0x7f4068003980 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.789+0000 7f4085400640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 0x7f4080164a40 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f407401d4d0 tx=0x7f4068005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.793+0000 7f407d600640 1 -- 172.21.15.66:0/2547453267 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074006e40 con 0x7f4068003980 2024-03-30T00:48:33.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.793+0000 7f407d600640 1 -- 172.21.15.66:0/2547453267 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4074004020 con 0x7f4068003980 2024-03-30T00:48:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.793+0000 7f407d600640 1 -- 172.21.15.66:0/2547453267 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40740043d0 con 0x7f4068003980 2024-03-30T00:48:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.793+0000 7f4087e00640 1 -- 172.21.15.66:0/2547453267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408014ef50 con 0x7f4068003980 2024-03-30T00:48:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.793+0000 7f4087e00640 1 -- 172.21.15.66:0/2547453267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408015bbc0 con 0x7f4068003980 2024-03-30T00:48:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.793+0000 7f4087e00640 1 -- 172.21.15.66:0/2547453267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40800cdf60 con 0x7f4068003980 2024-03-30T00:48:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.793+0000 7f407d600640 1 -- 172.21.15.66:0/2547453267 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f40740b4040 con 0x7f4068003980 2024-03-30T00:48:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.793+0000 7f407d600640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f40600666c0 0x7f4060068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.793+0000 7f407d600640 1 -- 172.21.15.66:0/2547453267 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f407411dae0 con 0x7f4068003980 2024-03-30T00:48:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.793+0000 7f4084a00640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f40600666c0 0x7f4060068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.793+0000 7f4084a00640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f40600666c0 0x7f4060068b80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f40800c7a60 tx=0x7f406c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.797+0000 7f407d600640 1 -- 172.21.15.66:0/2547453267 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f40740eaca0 con 0x7f4068003980 2024-03-30T00:48:33.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.961+0000 7f4087e00640 1 -- 172.21.15.66:0/2547453267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f408015c6a0 con 0x7f4068003980 2024-03-30T00:48:33.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.961+0000 7f407d600640 1 -- 172.21.15.66:0/2547453267 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f408015c6a0 con 0x7f4068003980 2024-03-30T00:48:33.972 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:33.972 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":98,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:47:49.264161+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:46:13.811742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-30T00:46:14.265142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":7}]},{"pgid":"14.1a","mappings":[{"from":1,"to":5}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/1626022080":"2024-03-31T00:47:49.264137+0000","172.21.15.66:6836/1626022080":"2024-03-31T00:47:49.264137+0000","172.21.15.66:6836/1147664160":"2024-03-31T00:46:34.103831+0000","172.21.15.66:6835/1147664160":"2024-03-31T00:46:34.103831+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:48:33.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 -- 172.21.15.66:0/2547453267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f40600666c0 msgr2=0x7f4060068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f40600666c0 0x7f4060068b80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f40800c7a60 tx=0x7f406c009190 comp rx=0 tx=0).stop 2024-03-30T00:48:33.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 -- 172.21.15.66:0/2547453267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 msgr2=0x7f4080164a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 0x7f4080164a40 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f407401d4d0 tx=0x7f4068005800 comp rx=0 tx=0).stop 2024-03-30T00:48:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 -- 172.21.15.66:0/2547453267 shutdown_connections 2024-03-30T00:48:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f408014e590 0x7f408014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f408015acd0 0x7f408015b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f40600666c0 0x7f4060068b80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 --2- 172.21.15.66:0/2547453267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4068003980 0x7f4080164a40 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 -- 172.21.15.66:0/2547453267 >> 172.21.15.66:0/2547453267 conn(0x7f40800d0540 msgr2=0x7f40800caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 -- 172.21.15.66:0/2547453267 shutdown_connections 2024-03-30T00:48:33.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:33.965+0000 7f4087e00640 1 -- 172.21.15.66:0/2547453267 wait complete. 2024-03-30T00:48:33.994 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:48:34.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.125+0000 7f5089600640 1 Processor -- start 2024-03-30T00:48:34.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.125+0000 7f5089600640 1 -- start start 2024-03-30T00:48:34.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.125+0000 7f5089600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f508415ca10 0x7f508406ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.125+0000 7f5089600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f50840ae4e0 0x7f50840cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.125+0000 7f5089600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508406e7d0 con 0x7f50840ce820 2024-03-30T00:48:34.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.125+0000 7f5089600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50840b1f70 con 0x7f50840ae4e0 2024-03-30T00:48:34.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.125+0000 7f5089600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508406f1a0 con 0x7f508415ca10 2024-03-30T00:48:34.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.125+0000 7f507be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f50840ae4e0 0x7f50840cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.125+0000 7f507be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f50840ae4e0 0x7f50840cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48574/0 (socket says 172.21.15.66:48574) 2024-03-30T00:48:34.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.125+0000 7f507be00640 1 -- 172.21.15.66:0/438313102 learned_addr learned my addr 172.21.15.66:0/438313102 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:34.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.125+0000 7f5082a00640 1 --2- 172.21.15.66:0/438313102 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f508415ca10 0x7f508406ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f507be00640 1 -- 172.21.15.66:0/438313102 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f508415ca10 msgr2=0x7f508406ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:34.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f507be00640 1 --2- 172.21.15.66:0/438313102 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f508415ca10 0x7f508406ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f507be00640 1 -- 172.21.15.66:0/438313102 >> v1:172.21.15.156:6789/0 conn(0x7f50840ce820 legacy=0x7f5084160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:34.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f507be00640 1 -- 172.21.15.66:0/438313102 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508415cdf0 con 0x7f50840ae4e0 2024-03-30T00:48:34.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f507be00640 1 --2- 172.21.15.66:0/438313102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50840ae4e0 0x7f50840cb620 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f506c009870 tx=0x7f506c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ed307c2e601df1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:34.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f5082000640 1 -- 172.21.15.66:0/438313102 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c0a95f0 con 0x7f50840ae4e0 2024-03-30T00:48:34.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f5082000640 1 -- 172.21.15.66:0/438313102 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c0a9750 con 0x7f50840ae4e0 2024-03-30T00:48:34.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f5082000640 1 -- 172.21.15.66:0/438313102 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c0a9ae0 con 0x7f50840ae4e0 2024-03-30T00:48:34.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f5089600640 1 -- 172.21.15.66:0/438313102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50840ae4e0 msgr2=0x7f50840cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:34.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f5089600640 1 --2- 172.21.15.66:0/438313102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50840ae4e0 0x7f50840cb620 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f506c009870 tx=0x7f506c01c530 comp rx=0 tx=0).stop 2024-03-30T00:48:34.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f5089600640 1 -- 172.21.15.66:0/438313102 shutdown_connections 2024-03-30T00:48:34.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f5089600640 1 --2- 172.21.15.66:0/438313102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50840ae4e0 0x7f50840cb620 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f5089600640 1 --2- 172.21.15.66:0/438313102 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f508415ca10 0x7f508406ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f5089600640 1 -- 172.21.15.66:0/438313102 >> 172.21.15.66:0/438313102 conn(0x7f5084098a20 msgr2=0x7f50840b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:34.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f5089600640 1 -- 172.21.15.66:0/438313102 shutdown_connections 2024-03-30T00:48:34.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.129+0000 7f5089600640 1 -- 172.21.15.66:0/438313102 wait complete. 2024-03-30T00:48:34.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5089600640 1 Processor -- start 2024-03-30T00:48:34.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5089600640 1 -- start start 2024-03-30T00:48:34.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5089600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50840ae4e0 0x7f508415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5089600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50840ce820 0x7f50841547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5089600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508415ca10 0x7f5084154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5089600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50840c4ac0 con 0x7f508415ca10 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5089600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50840c4da0 con 0x7f50840ce820 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5089600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50840c5080 con 0x7f50840ae4e0 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5083400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508415ca10 0x7f5084154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5083400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508415ca10 0x7f5084154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48582/0 (socket says 172.21.15.66:48582) 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5083400640 1 -- 172.21.15.66:0/2320441879 learned_addr learned my addr 172.21.15.66:0/2320441879 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5082a00640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50840ae4e0 0x7f508415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f507be00640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50840ce820 0x7f50841547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5083400640 1 -- 172.21.15.66:0/2320441879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50840ae4e0 msgr2=0x7f508415ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5083400640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50840ae4e0 0x7f508415ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5083400640 1 -- 172.21.15.66:0/2320441879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50840ce820 msgr2=0x7f50841547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:34.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5083400640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50840ce820 0x7f50841547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5083400640 1 -- 172.21.15.66:0/2320441879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c0094d0 con 0x7f508415ca10 2024-03-30T00:48:34.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5083400640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508415ca10 0x7f5084154ce0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c770 tx=0x7f507c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:34.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f507aa00640 1 -- 172.21.15.66:0/2320441879 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c003b80 con 0x7f508415ca10 2024-03-30T00:48:34.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f507aa00640 1 -- 172.21.15.66:0/2320441879 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c003ce0 con 0x7f508415ca10 2024-03-30T00:48:34.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f507aa00640 1 -- 172.21.15.66:0/2320441879 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c00aa90 con 0x7f508415ca10 2024-03-30T00:48:34.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5089600640 1 -- 172.21.15.66:0/2320441879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50840c5360 con 0x7f508415ca10 2024-03-30T00:48:34.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.133+0000 7f5089600640 1 -- 172.21.15.66:0/2320441879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508415b220 con 0x7f508415ca10 2024-03-30T00:48:34.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.137+0000 7f5089600640 1 -- 172.21.15.66:0/2320441879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5084073650 con 0x7f508415ca10 2024-03-30T00:48:34.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.137+0000 7f507aa00640 1 -- 172.21.15.66:0/2320441879 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f507c00abf0 con 0x7f508415ca10 2024-03-30T00:48:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.137+0000 7f507aa00640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f50600666c0 0x7f5060068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.137+0000 7f507aa00640 1 -- 172.21.15.66:0/2320441879 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f507c0841a0 con 0x7f508415ca10 2024-03-30T00:48:34.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.137+0000 7f5082a00640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f50600666c0 0x7f5060068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.137+0000 7f5082a00640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f50600666c0 0x7f5060068b80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5074009af0 tx=0x7f5074008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:34.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.141+0000 7f507aa00640 1 -- 172.21.15.66:0/2320441879 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f507c051320 con 0x7f508415ca10 2024-03-30T00:48:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.337+0000 7f5089600640 1 -- 172.21.15.66:0/2320441879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5084066c30 con 0x7f508415ca10 2024-03-30T00:48:34.346 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:48:34.337+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:48:34.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.345+0000 7f507aa00640 1 -- 172.21.15.66:0/2320441879 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7f507c054940 con 0x7f508415ca10 2024-03-30T00:48:34.354 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:48:34.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.345+0000 7f5089600640 1 -- 172.21.15.66:0/2320441879 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f50600666c0 msgr2=0x7f5060068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:34.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.345+0000 7f5089600640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f50600666c0 0x7f5060068b80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5074009af0 tx=0x7f5074008040 comp rx=0 tx=0).stop 2024-03-30T00:48:34.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.345+0000 7f5089600640 1 -- 172.21.15.66:0/2320441879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508415ca10 msgr2=0x7f5084154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:34.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.345+0000 7f5089600640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508415ca10 0x7f5084154ce0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c770 tx=0x7f507c00cc30 comp rx=0 tx=0).stop 2024-03-30T00:48:34.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.345+0000 7f5089600640 1 -- 172.21.15.66:0/2320441879 shutdown_connections 2024-03-30T00:48:34.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.345+0000 7f5089600640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f50600666c0 0x7f5060068b80 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.345+0000 7f5089600640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f508415ca10 0x7f5084154ce0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.345+0000 7f5089600640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50840ce820 0x7f50841547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.345+0000 7f5089600640 1 --2- 172.21.15.66:0/2320441879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50840ae4e0 0x7f508415ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.345+0000 7f5089600640 1 -- 172.21.15.66:0/2320441879 >> 172.21.15.66:0/2320441879 conn(0x7f5084098a20 msgr2=0x7f50840ca790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:34.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.349+0000 7f5089600640 1 -- 172.21.15.66:0/2320441879 shutdown_connections 2024-03-30T00:48:34.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.349+0000 7f5089600640 1 -- 172.21.15.66:0/2320441879 wait complete. 2024-03-30T00:48:34.365 INFO:tasks.ceph.mds.a.smithi066.stderr: -2157> 2024-03-30T00:47:55.277+0000 7f9400400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:48:34.367 INFO:tasks.ceph.mds.a.smithi066.stderr: -1792> 2024-03-30T00:47:56.765+0000 7f93fbe00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56258bef6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:48:34.383 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T00:48:34.417 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:48:34.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80bd800640 1 Processor -- start 2024-03-30T00:48:34.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80bd800640 1 -- start start 2024-03-30T00:48:34.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80bd800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b80cec30 0x7f80b8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80bd800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80b80cde70 0x7f80b815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80bd800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b815a6e0 con 0x7f80b80cda90 2024-03-30T00:48:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80bd800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b80b1b80 con 0x7f80b80cde70 2024-03-30T00:48:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80bd800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80cd3e0 con 0x7f80b80cec30 2024-03-30T00:48:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f80b80cda90 0x7f80b80ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45888/0 (socket says 172.21.15.66:45888) 2024-03-30T00:48:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6a00640 1 -- 172.21.15.66:0/3556515112 learned_addr learned my addr 172.21.15.66:0/3556515112 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b7400640 1 --2- 172.21.15.66:0/3556515112 >> v2:172.21.15.156:3300/0 conn(0x7f80b80cde70 0x7f80b815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80ae000640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b80cec30 0x7f80b8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263590762 0 0) 0x7f80b815a6e0 con 0x7f80b80cda90 2024-03-30T00:48:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f809c003530 con 0x7f80b80cda90 2024-03-30T00:48:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1946136327 0 0) 0x7f809c003530 con 0x7f80b80cda90 2024-03-30T00:48:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b80cec30 msgr2=0x7f80b8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b80cec30 0x7f80b8159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 >> v2:172.21.15.156:3300/0 conn(0x7f80b80cde70 msgr2=0x7f80b815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 --2- 172.21.15.66:0/3556515112 >> v2:172.21.15.156:3300/0 conn(0x7f80b80cde70 0x7f80b815a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b815a990 con 0x7f80b80cda90 2024-03-30T00:48:34.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f80a8001680 con 0x7f80b80cda90 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b7400640 1 --2- 172.21.15.66:0/3556515112 >> v2:172.21.15.156:3300/0 conn(0x7f80b80cde70 0x7f80b815a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80ae000640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b80cec30 0x7f80b8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 >> v1:172.21.15.66:6789/0 conn(0x7f80b80cda90 legacy=0x7f80b80ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b80cec30 0x7f809c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c003ec0 0x7f809c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c004d10 0x7f809c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80b1b80 con 0x7f809c004d10 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80cd3e0 con 0x7f809c003ec0 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c00b990 con 0x7f80b80cec30 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b6a00640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b80cec30 0x7f809c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80b7400640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c004d10 0x7f809c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.517+0000 7f80ae000640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c003ec0 0x7f809c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80b7400640 1 -- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b80cec30 msgr2=0x7f809c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:34.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80b7400640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b80cec30 0x7f809c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80b7400640 1 -- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c003ec0 msgr2=0x7f809c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:34.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80b7400640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c003ec0 0x7f809c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80b7400640 1 -- 172.21.15.66:0/3556515112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a0009420 con 0x7f809c004d10 2024-03-30T00:48:34.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80b7400640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c004d10 0x7f809c005190 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f80b000e350 tx=0x7f80b001ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891e3ad99adca790 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:34.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b00a5e90 con 0x7f809c004d10 2024-03-30T00:48:34.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b000f040 con 0x7f809c004d10 2024-03-30T00:48:34.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80b6000640 1 -- 172.21.15.66:0/3556515112 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b00ad640 con 0x7f809c004d10 2024-03-30T00:48:34.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80bd800640 1 -- 172.21.15.66:0/3556515112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c004d10 msgr2=0x7f809c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:34.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80bd800640 1 --2- 172.21.15.66:0/3556515112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c004d10 0x7f809c005190 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f80b000e350 tx=0x7f80b001ce80 comp rx=0 tx=0).stop 2024-03-30T00:48:34.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80b7400640 1 -- 172.21.15.66:0/3556515112 reap_dead start 2024-03-30T00:48:34.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80bd800640 1 -- 172.21.15.66:0/3556515112 shutdown_connections 2024-03-30T00:48:34.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80bd800640 1 -- 172.21.15.66:0/3556515112 >> 172.21.15.66:0/3556515112 conn(0x7f80b8069f60 msgr2=0x7f80b806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:34.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.521+0000 7f80bd800640 1 -- 172.21.15.66:0/3556515112 shutdown_connections 2024-03-30T00:48:34.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80bd800640 1 -- 172.21.15.66:0/3556515112 wait complete. 2024-03-30T00:48:34.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80bd800640 1 Processor -- start 2024-03-30T00:48:34.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80bd800640 1 -- start start 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80bd800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b8097960 0x7f80b8151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80bd800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80af390 0x7f80b8070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80bd800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b8071460 0x7f80b81502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80bd800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80718e0 con 0x7f80b80af390 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80bd800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80b07e0 con 0x7f80b8071460 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80bd800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b8072130 con 0x7f80b8097960 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80ae000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80af390 0x7f80b8070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80ae000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80af390 0x7f80b8070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48622/0 (socket says 172.21.15.66:48622) 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80ae000640 1 -- 172.21.15.66:0/154282517 learned_addr learned my addr 172.21.15.66:0/154282517 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80b7400640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b8071460 0x7f80b81502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80b6a00640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b8097960 0x7f80b8151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80ae000640 1 -- 172.21.15.66:0/154282517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b8097960 msgr2=0x7f80b8151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80ae000640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b8097960 0x7f80b8151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80ae000640 1 -- 172.21.15.66:0/154282517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b8071460 msgr2=0x7f80b81502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80ae000640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b8071460 0x7f80b81502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80ae000640 1 -- 172.21.15.66:0/154282517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b8071e00 con 0x7f80b80af390 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80b7400640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b8071460 0x7f80b81502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80b6a00640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b8097960 0x7f80b8151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80ae000640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80af390 0x7f80b8070f20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f80a0011d00 tx=0x7f80a000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80af400640 1 -- 172.21.15.66:0/154282517 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000fcc0 con 0x7f80b80af390 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80af400640 1 -- 172.21.15.66:0/154282517 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a000fe20 con 0x7f80b80af390 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80af400640 1 -- 172.21.15.66:0/154282517 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0004240 con 0x7f80b80af390 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80bd800640 1 -- 172.21.15.66:0/154282517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b80b0500 con 0x7f80b80af390 2024-03-30T00:48:34.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.525+0000 7f80bd800640 1 -- 172.21.15.66:0/154282517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b8150810 con 0x7f80b80af390 2024-03-30T00:48:34.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.529+0000 7f80bd800640 1 -- 172.21.15.66:0/154282517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b8071c60 con 0x7f80b80af390 2024-03-30T00:48:34.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.529+0000 7f80af400640 1 -- 172.21.15.66:0/154282517 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f80a00043a0 con 0x7f80b80af390 2024-03-30T00:48:34.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.529+0000 7f80af400640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 0x7f8094068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:34.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.529+0000 7f80af400640 1 -- 172.21.15.66:0/154282517 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6403+0+0 (secure 0 0 0) 0x7f80a0084b90 con 0x7f80b80af390 2024-03-30T00:48:34.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.529+0000 7f80b6a00640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 0x7f8094068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:34.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.529+0000 7f80b6a00640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 0x7f8094068b80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f80b815c060 tx=0x7f80a8007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:34.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.533+0000 7f80af400640 1 -- 172.21.15.66:0/154282517 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f80b8071c60 con 0x7f80b80af390 2024-03-30T00:48:34.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:34.729+0000 7f80bd800640 1 -- 172.21.15.66:0/154282517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f80b8071c60 con 0x7f80b80af390 2024-03-30T00:48:35.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.345+0000 7f80af400640 1 -- 172.21.15.66:0/154282517 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f80b8071c60 con 0x7f80b80af390 2024-03-30T00:48:35.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 -- 172.21.15.66:0/154282517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 msgr2=0x7f8094068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:35.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 0x7f8094068b80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f80b815c060 tx=0x7f80a8007900 comp rx=0 tx=0).stop 2024-03-30T00:48:35.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 -- 172.21.15.66:0/154282517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80af390 msgr2=0x7f80b8070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:35.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80af390 0x7f80b8070f20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f80a0011d00 tx=0x7f80a000f610 comp rx=0 tx=0).stop 2024-03-30T00:48:35.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 -- 172.21.15.66:0/154282517 shutdown_connections 2024-03-30T00:48:35.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b8071460 0x7f80b81502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:35.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f80940666c0 0x7f8094068b80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:35.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80b80af390 0x7f80b8070f20 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:35.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 --2- 172.21.15.66:0/154282517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b8097960 0x7f80b8151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:35.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 -- 172.21.15.66:0/154282517 >> 172.21.15.66:0/154282517 conn(0x7f80b8069f60 msgr2=0x7f80b80cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:35.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 -- 172.21.15.66:0/154282517 shutdown_connections 2024-03-30T00:48:35.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.349+0000 7f80bd800640 1 -- 172.21.15.66:0/154282517 wait complete. 2024-03-30T00:48:35.381 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T00:48:35.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0fe00640 1 Processor -- start 2024-03-30T00:48:35.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0fe00640 1 -- start start 2024-03-30T00:48:35.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0d080acaf0 0x7f0d080b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:35.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0fe00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0d0815ba00 0x7f0d080ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:35.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0fe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d080cbf80 con 0x7f0d080cecf0 2024-03-30T00:48:35.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0fe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d080cc820 con 0x7f0d080acaf0 2024-03-30T00:48:35.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0fe00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d080cf110 con 0x7f0d0815ba00 2024-03-30T00:48:35.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0d080acaf0 0x7f0d080b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:35.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0d080acaf0 0x7f0d080b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48628/0 (socket says 172.21.15.66:48628) 2024-03-30T00:48:35.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0de00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0d080cecf0 0x7f0d081579e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45902/0 (socket says 172.21.15.66:45902) 2024-03-30T00:48:35.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0d400640 1 -- 172.21.15.66:0/2044518943 learned_addr learned my addr 172.21.15.66:0/2044518943 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:35.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0d400640 1 -- 172.21.15.66:0/2044518943 >> v2:172.21.15.156:3300/0 conn(0x7f0d0815ba00 msgr2=0x7f0d080ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:35.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0d400640 1 --2- 172.21.15.66:0/2044518943 >> v2:172.21.15.156:3300/0 conn(0x7f0d0815ba00 0x7f0d080ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:35.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0d400640 1 -- 172.21.15.66:0/2044518943 >> v1:172.21.15.66:6789/0 conn(0x7f0d080cecf0 legacy=0x7f0d081579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:35.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d0d400640 1 -- 172.21.15.66:0/2044518943 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d080b2dd0 con 0x7f0d080acaf0 2024-03-30T00:48:35.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.565+0000 7f0d07e00640 1 -- 172.21.15.66:0/2044518943 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676549986 0 0) 0x7f0d080cbf80 con 0x7f0d080cecf0 2024-03-30T00:48:35.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d0d400640 1 --2- 172.21.15.66:0/2044518943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080acaf0 0x7f0d080b24d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f0d080c1e10 tx=0x7f0cfc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb39b83da7facfc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:35.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d07e00640 1 -- 172.21.15.66:0/2044518943 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0ab660 con 0x7f0d080acaf0 2024-03-30T00:48:35.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d07e00640 1 -- 172.21.15.66:0/2044518943 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc0ab7c0 con 0x7f0d080acaf0 2024-03-30T00:48:35.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d07e00640 1 -- 172.21.15.66:0/2044518943 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0abb50 con 0x7f0d080acaf0 2024-03-30T00:48:35.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d0fe00640 1 -- 172.21.15.66:0/2044518943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080acaf0 msgr2=0x7f0d080b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:35.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d0fe00640 1 --2- 172.21.15.66:0/2044518943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080acaf0 0x7f0d080b24d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f0d080c1e10 tx=0x7f0cfc01c460 comp rx=0 tx=0).stop 2024-03-30T00:48:35.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d0fe00640 1 -- 172.21.15.66:0/2044518943 shutdown_connections 2024-03-30T00:48:35.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d0fe00640 1 --2- 172.21.15.66:0/2044518943 >> v2:172.21.15.156:3300/0 conn(0x7f0d0815ba00 0x7f0d080ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:35.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d0fe00640 1 --2- 172.21.15.66:0/2044518943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080acaf0 0x7f0d080b24d0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:35.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d0fe00640 1 -- 172.21.15.66:0/2044518943 >> 172.21.15.66:0/2044518943 conn(0x7f0d0806a190 msgr2=0x7f0d0806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:35.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d0fe00640 1 -- 172.21.15.66:0/2044518943 shutdown_connections 2024-03-30T00:48:35.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.569+0000 7f0d0fe00640 1 -- 172.21.15.66:0/2044518943 wait complete. 2024-03-30T00:48:35.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0fe00640 1 Processor -- start 2024-03-30T00:48:35.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0fe00640 1 -- start start 2024-03-30T00:48:35.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0fe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d080acaf0 0x7f0d08157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:35.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080cecf0 0x7f0d08071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:35.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0fe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d0815ba00 0x7f0d081541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:35.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0806d990 con 0x7f0d080cecf0 2024-03-30T00:48:35.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0fe00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0806dbe0 con 0x7f0d0815ba00 2024-03-30T00:48:35.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0fe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080c5600 con 0x7f0d080acaf0 2024-03-30T00:48:35.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080cecf0 0x7f0d08071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:35.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080cecf0 0x7f0d08071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48642/0 (socket says 172.21.15.66:48642) 2024-03-30T00:48:35.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0ca00640 1 -- 172.21.15.66:0/793683358 learned_addr learned my addr 172.21.15.66:0/793683358 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:35.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0d400640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d080acaf0 0x7f0d08157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:35.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0de00640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d0815ba00 0x7f0d081541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:35.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0ca00640 1 -- 172.21.15.66:0/793683358 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d080acaf0 msgr2=0x7f0d08157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:35.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0ca00640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d080acaf0 0x7f0d08157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:35.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0ca00640 1 -- 172.21.15.66:0/793683358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d0815ba00 msgr2=0x7f0d081541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:35.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0ca00640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d0815ba00 0x7f0d081541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:35.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0ca00640 1 -- 172.21.15.66:0/793683358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc0094d0 con 0x7f0d080cecf0 2024-03-30T00:48:35.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0ca00640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080cecf0 0x7f0d08071190 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400b480 tx=0x7f0cf400b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:35.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d05600640 1 -- 172.21.15.66:0/793683358 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4010870 con 0x7f0d080cecf0 2024-03-30T00:48:35.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d05600640 1 -- 172.21.15.66:0/793683358 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf40109d0 con 0x7f0d080cecf0 2024-03-30T00:48:35.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d05600640 1 -- 172.21.15.66:0/793683358 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4010d80 con 0x7f0d080cecf0 2024-03-30T00:48:35.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0fe00640 1 -- 172.21.15.66:0/793683358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d080c58e0 con 0x7f0d080cecf0 2024-03-30T00:48:35.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0fe00640 1 -- 172.21.15.66:0/793683358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d08154730 con 0x7f0d080cecf0 2024-03-30T00:48:35.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.573+0000 7f0d0fe00640 1 -- 172.21.15.66:0/793683358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d080c5460 con 0x7f0d080cecf0 2024-03-30T00:48:35.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.577+0000 7f0d05600640 1 -- 172.21.15.66:0/793683358 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0cf400f3c0 con 0x7f0d080cecf0 2024-03-30T00:48:35.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.577+0000 7f0d05600640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ce8066690 0x7f0ce8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:35.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.577+0000 7f0d05600640 1 -- 172.21.15.66:0/793683358 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6403+0+0 (secure 0 0 0) 0x7f0cf40844a0 con 0x7f0d080cecf0 2024-03-30T00:48:35.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.577+0000 7f0d0d400640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ce8066690 0x7f0ce8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:35.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.577+0000 7f0d0d400640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ce8066690 0x7f0ce8068b50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0af000 tx=0x7f0cfc0b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:35.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.581+0000 7f0d05600640 1 -- 172.21.15.66:0/793683358 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0d080c5460 con 0x7f0d080cecf0 2024-03-30T00:48:35.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:35.745+0000 7f0d0fe00640 1 -- 172.21.15.66:0/793683358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0d080c5460 con 0x7f0d080cecf0 2024-03-30T00:48:36.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.681+0000 7f0d05600640 1 -- 172.21.15.66:0/793683358 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0d080c5460 con 0x7f0d080cecf0 2024-03-30T00:48:36.693 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-03-30T00:48:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.685+0000 7f0d0fe00640 1 -- 172.21.15.66:0/793683358 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ce8066690 msgr2=0x7f0ce8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.685+0000 7f0d0fe00640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ce8066690 0x7f0ce8068b50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0af000 tx=0x7f0cfc0b6040 comp rx=0 tx=0).stop 2024-03-30T00:48:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.685+0000 7f0d0fe00640 1 -- 172.21.15.66:0/793683358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080cecf0 msgr2=0x7f0d08071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.685+0000 7f0d0fe00640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080cecf0 0x7f0d08071190 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400b480 tx=0x7f0cf400b940 comp rx=0 tx=0).stop 2024-03-30T00:48:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.685+0000 7f0d0fe00640 1 -- 172.21.15.66:0/793683358 shutdown_connections 2024-03-30T00:48:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.685+0000 7f0d0fe00640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d0815ba00 0x7f0d081541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.685+0000 7f0d0fe00640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0ce8066690 0x7f0ce8068b50 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:36.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.685+0000 7f0d0fe00640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080cecf0 0x7f0d08071190 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:36.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.685+0000 7f0d0fe00640 1 --2- 172.21.15.66:0/793683358 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0d080acaf0 0x7f0d08157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:36.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.685+0000 7f0d0fe00640 1 -- 172.21.15.66:0/793683358 >> 172.21.15.66:0/793683358 conn(0x7f0d0806a190 msgr2=0x7f0d080ab590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:36.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.689+0000 7f0d0fe00640 1 -- 172.21.15.66:0/793683358 shutdown_connections 2024-03-30T00:48:36.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.689+0000 7f0d0fe00640 1 -- 172.21.15.66:0/793683358 wait complete. 2024-03-30T00:48:36.721 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T00:48:36.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7036c00640 1 Processor -- start 2024-03-30T00:48:36.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7036c00640 1 -- start start 2024-03-30T00:48:36.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7036c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f70301594f0 0x7f70300b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:36.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7036c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f70300ce7b0 0x7f70300cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7036c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70301598d0 0x7f7030159d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7036c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70300b1a60 con 0x7f70300ce7b0 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7036c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70300cf1a0 con 0x7f70301594f0 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7036c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70300cda90 con 0x7f70301598d0 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7027400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f70300ce7b0 0x7f70300cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7027400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f70300ce7b0 0x7f70300cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48656/0 (socket says 172.21.15.66:48656) 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7027400640 1 -- 172.21.15.66:0/2677817726 learned_addr learned my addr 172.21.15.66:0/2677817726 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f702fe00640 1 --2- 172.21.15.66:0/2677817726 >> v2:172.21.15.156:3300/0 conn(0x7f70301594f0 0x7f70300b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7034c00640 1 --2- 172.21.15.66:0/2677817726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70301598d0 0x7f7030159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7027400640 1 -- 172.21.15.66:0/2677817726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70301598d0 msgr2=0x7f7030159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7027400640 1 --2- 172.21.15.66:0/2677817726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70301598d0 0x7f7030159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7027400640 1 -- 172.21.15.66:0/2677817726 >> v2:172.21.15.156:3300/0 conn(0x7f70301594f0 msgr2=0x7f70300b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:36.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7027400640 1 --2- 172.21.15.66:0/2677817726 >> v2:172.21.15.156:3300/0 conn(0x7f70301594f0 0x7f70300b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:36.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.861+0000 7f7027400640 1 -- 172.21.15.66:0/2677817726 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70300b13c0 con 0x7f70300ce7b0 2024-03-30T00:48:36.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f7027400640 1 --2- 172.21.15.66:0/2677817726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70300ce7b0 0x7f70300cec60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f70300cf3a0 tx=0x7f701801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0c67c20ab2283 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:36.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f702f400640 1 -- 172.21.15.66:0/2677817726 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70180aa660 con 0x7f70300ce7b0 2024-03-30T00:48:36.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f702f400640 1 -- 172.21.15.66:0/2677817726 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70180aa7c0 con 0x7f70300ce7b0 2024-03-30T00:48:36.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f702f400640 1 -- 172.21.15.66:0/2677817726 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70180aab50 con 0x7f70300ce7b0 2024-03-30T00:48:36.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f7036c00640 1 -- 172.21.15.66:0/2677817726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70300ce7b0 msgr2=0x7f70300cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:36.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f7036c00640 1 --2- 172.21.15.66:0/2677817726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70300ce7b0 0x7f70300cec60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f70300cf3a0 tx=0x7f701801c520 comp rx=0 tx=0).stop 2024-03-30T00:48:36.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f7036c00640 1 -- 172.21.15.66:0/2677817726 shutdown_connections 2024-03-30T00:48:36.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f7036c00640 1 --2- 172.21.15.66:0/2677817726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70301598d0 0x7f7030159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:36.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f7036c00640 1 --2- 172.21.15.66:0/2677817726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70300ce7b0 0x7f70300cec60 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:36.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f7036c00640 1 --2- 172.21.15.66:0/2677817726 >> v2:172.21.15.156:3300/0 conn(0x7f70301594f0 0x7f70300b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:36.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f7036c00640 1 -- 172.21.15.66:0/2677817726 >> 172.21.15.66:0/2677817726 conn(0x7f703006a190 msgr2=0x7f703006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:36.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f7036c00640 1 -- 172.21.15.66:0/2677817726 shutdown_connections 2024-03-30T00:48:36.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.865+0000 7f7036c00640 1 -- 172.21.15.66:0/2677817726 wait complete. 2024-03-30T00:48:36.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7036c00640 1 Processor -- start 2024-03-30T00:48:36.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7036c00640 1 -- start start 2024-03-30T00:48:36.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7036c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70300ce7b0 0x7f7030158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7036c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70301594f0 0x7f703006ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7036c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70301598d0 0x7f70300c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7036c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7030070580 con 0x7f70301598d0 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7036c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70300c79f0 con 0x7f70301594f0 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7036c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703006d200 con 0x7f70300ce7b0 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7034c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70301598d0 0x7f70300c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7034c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70301598d0 0x7f70300c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48662/0 (socket says 172.21.15.66:48662) 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7034c00640 1 -- 172.21.15.66:0/691796098 learned_addr learned my addr 172.21.15.66:0/691796098 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7027400640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70301594f0 0x7f703006ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f702fe00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70300ce7b0 0x7f7030158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7034c00640 1 -- 172.21.15.66:0/691796098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70300ce7b0 msgr2=0x7f7030158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7034c00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70300ce7b0 0x7f7030158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:36.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7034c00640 1 -- 172.21.15.66:0/691796098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70301594f0 msgr2=0x7f703006ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:36.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7034c00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70301594f0 0x7f703006ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:36.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7034c00640 1 -- 172.21.15.66:0/691796098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70180094d0 con 0x7f70301598d0 2024-03-30T00:48:36.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f702fe00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70300ce7b0 0x7f7030158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:36.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7034c00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70301598d0 0x7f70300c89e0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f7028009d30 tx=0x7f702800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:36.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f702cc00640 1 -- 172.21.15.66:0/691796098 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028009ba0 con 0x7f70301598d0 2024-03-30T00:48:36.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f702cc00640 1 -- 172.21.15.66:0/691796098 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702800f040 con 0x7f70301598d0 2024-03-30T00:48:36.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7036c00640 1 -- 172.21.15.66:0/691796098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703006ff50 con 0x7f70301598d0 2024-03-30T00:48:36.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f702cc00640 1 -- 172.21.15.66:0/691796098 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70280146c0 con 0x7f70301598d0 2024-03-30T00:48:36.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.869+0000 7f7036c00640 1 -- 172.21.15.66:0/691796098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703006d700 con 0x7f70301598d0 2024-03-30T00:48:36.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.873+0000 7f7036c00640 1 -- 172.21.15.66:0/691796098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70300706c0 con 0x7f70301598d0 2024-03-30T00:48:36.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.873+0000 7f702cc00640 1 -- 172.21.15.66:0/691796098 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7028014820 con 0x7f70301598d0 2024-03-30T00:48:36.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.873+0000 7f702cc00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f700c066690 0x7f700c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:36.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.873+0000 7f702cc00640 1 -- 172.21.15.66:0/691796098 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5965+0+0 (secure 0 0 0) 0x7f7028084d50 con 0x7f70301598d0 2024-03-30T00:48:36.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.873+0000 7f702fe00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f700c066690 0x7f700c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:36.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.873+0000 7f702fe00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f700c066690 0x7f700c068b50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7020009b30 tx=0x7f7020008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:36.877+0000 7f702cc00640 1 -- 172.21.15.66:0/691796098 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f70300706c0 con 0x7f70301598d0 2024-03-30T00:48:37.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.037+0000 7f7036c00640 1 -- 172.21.15.66:0/691796098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f70300706c0 con 0x7f70301598d0 2024-03-30T00:48:37.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.693+0000 7f702cc00640 1 -- 172.21.15.66:0/691796098 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) v1 ==== 169+0+0 (secure 0 0 0) 0x7f70300706c0 con 0x7f70301598d0 2024-03-30T00:48:37.704 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-03-30T00:48:37.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 -- 172.21.15.66:0/691796098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f700c066690 msgr2=0x7f700c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:37.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f700c066690 0x7f700c068b50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7020009b30 tx=0x7f7020008040 comp rx=0 tx=0).stop 2024-03-30T00:48:37.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 -- 172.21.15.66:0/691796098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70301598d0 msgr2=0x7f70300c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:37.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70301598d0 0x7f70300c89e0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f7028009d30 tx=0x7f702800c490 comp rx=0 tx=0).stop 2024-03-30T00:48:37.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 -- 172.21.15.66:0/691796098 shutdown_connections 2024-03-30T00:48:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f700c066690 0x7f700c068b50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70301598d0 0x7f70300c89e0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70301594f0 0x7f703006ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 --2- 172.21.15.66:0/691796098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70300ce7b0 0x7f7030158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 -- 172.21.15.66:0/691796098 >> 172.21.15.66:0/691796098 conn(0x7f703006a190 msgr2=0x7f70300aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 -- 172.21.15.66:0/691796098 shutdown_connections 2024-03-30T00:48:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.697+0000 7f7036c00640 1 -- 172.21.15.66:0/691796098 wait complete. 2024-03-30T00:48:37.733 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-30T00:48:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.869+0000 7f8d53800640 1 Processor -- start 2024-03-30T00:48:37.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.869+0000 7f8d53800640 1 -- start start 2024-03-30T00:48:37.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d53800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d4c0cda90 0x7f8d4c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d53800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8d4c15a370 0x7f8d4c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d53800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4c15a990 con 0x7f8d4c0cde70 2024-03-30T00:48:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d53800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d4c0b1b80 con 0x7f8d4c15a370 2024-03-30T00:48:37.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d53800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c0cd3e0 con 0x7f8d4c0cda90 2024-03-30T00:48:37.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d51800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8d4c15a370 0x7f8d4c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:37.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d50e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d4c0cda90 0x7f8d4c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:37.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d51800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8d4c15a370 0x7f8d4c0cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:40666/0 (socket says 172.21.15.66:40666) 2024-03-30T00:48:37.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d51800640 1 -- 172.21.15.66:0/2793302674 learned_addr learned my addr 172.21.15.66:0/2793302674 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:37.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d51800640 1 -- 172.21.15.66:0/2793302674 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d4c0cda90 msgr2=0x7f8d4c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:37.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d51800640 1 --2- 172.21.15.66:0/2793302674 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d4c0cda90 0x7f8d4c0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:37.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d51800640 1 -- 172.21.15.66:0/2793302674 >> v1:172.21.15.156:6789/0 conn(0x7f8d4c0cde70 legacy=0x7f8d4c159c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:48:37.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d51800640 1 -- 172.21.15.66:0/2793302674 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c0b11e0 con 0x7f8d4c15a370 2024-03-30T00:48:37.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.873+0000 7f8d50e00640 1 --2- 172.21.15.66:0/2793302674 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d4c0cda90 0x7f8d4c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:37.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d51800640 1 --2- 172.21.15.66:0/2793302674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d4c15a370 0x7f8d4c0cea60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009e50 tx=0x7f8d4401c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e4e1d62bba0d74 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:37.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d4be00640 1 -- 172.21.15.66:0/2793302674 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d440a56f0 con 0x7f8d4c15a370 2024-03-30T00:48:37.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d4be00640 1 -- 172.21.15.66:0/2793302674 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d440a5850 con 0x7f8d4c15a370 2024-03-30T00:48:37.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d4be00640 1 -- 172.21.15.66:0/2793302674 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d440a5be0 con 0x7f8d4c15a370 2024-03-30T00:48:37.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 -- 172.21.15.66:0/2793302674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d4c15a370 msgr2=0x7f8d4c0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:37.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 --2- 172.21.15.66:0/2793302674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d4c15a370 0x7f8d4c0cea60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009e50 tx=0x7f8d4401c960 comp rx=0 tx=0).stop 2024-03-30T00:48:37.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 -- 172.21.15.66:0/2793302674 shutdown_connections 2024-03-30T00:48:37.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 --2- 172.21.15.66:0/2793302674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d4c15a370 0x7f8d4c0cea60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:37.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 --2- 172.21.15.66:0/2793302674 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d4c0cda90 0x7f8d4c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:37.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 -- 172.21.15.66:0/2793302674 >> 172.21.15.66:0/2793302674 conn(0x7f8d4c069f60 msgr2=0x7f8d4c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:37.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 -- 172.21.15.66:0/2793302674 shutdown_connections 2024-03-30T00:48:37.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 -- 172.21.15.66:0/2793302674 wait complete. 2024-03-30T00:48:37.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 Processor -- start 2024-03-30T00:48:37.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 -- start start 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d4c0cda90 0x7f8d4c159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d4c0cde70 0x7f8d4c155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d4c15a370 0x7f8d4c1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.877+0000 7f8d53800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c1528d0 con 0x7f8d4c0cda90 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d53800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c0ad860 con 0x7f8d4c0cde70 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d53800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c151b10 con 0x7f8d4c15a370 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d50e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d4c0cda90 0x7f8d4c159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d50e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d4c0cda90 0x7f8d4c159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48666/0 (socket says 172.21.15.66:48666) 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d50e00640 1 -- 172.21.15.66:0/48459070 learned_addr learned my addr 172.21.15.66:0/48459070 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d51800640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d4c15a370 0x7f8d4c1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d43e00640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d4c0cde70 0x7f8d4c155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:37.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d50e00640 1 -- 172.21.15.66:0/48459070 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d4c15a370 msgr2=0x7f8d4c1556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:37.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d50e00640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d4c15a370 0x7f8d4c1556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:37.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d50e00640 1 -- 172.21.15.66:0/48459070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d4c0cde70 msgr2=0x7f8d4c155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:37.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d50e00640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d4c0cde70 0x7f8d4c155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:37.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d50e00640 1 -- 172.21.15.66:0/48459070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c009420 con 0x7f8d4c0cda90 2024-03-30T00:48:37.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d50e00640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d4c0cda90 0x7f8d4c159080 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00ebe0 tx=0x7f8d3c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:37.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d49600640 1 -- 172.21.15.66:0/48459070 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00eee0 con 0x7f8d4c0cda90 2024-03-30T00:48:37.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d49600640 1 -- 172.21.15.66:0/48459070 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c00cc30 con 0x7f8d4c0cda90 2024-03-30T00:48:37.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d53800640 1 -- 172.21.15.66:0/48459070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d44009b30 con 0x7f8d4c0cda90 2024-03-30T00:48:37.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d49600640 1 -- 172.21.15.66:0/48459070 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c013470 con 0x7f8d4c0cda90 2024-03-30T00:48:37.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d53800640 1 -- 172.21.15.66:0/48459070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c1520d0 con 0x7f8d4c0cda90 2024-03-30T00:48:37.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d53800640 1 -- 172.21.15.66:0/48459070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4c066c30 con 0x7f8d4c0cda90 2024-03-30T00:48:37.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.881+0000 7f8d49600640 1 -- 172.21.15.66:0/48459070 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8d3c0135d0 con 0x7f8d4c0cda90 2024-03-30T00:48:37.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.885+0000 7f8d49600640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d28066690 0x7f8d28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:37.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.885+0000 7f8d49600640 1 -- 172.21.15.66:0/48459070 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f8d3c0845e0 con 0x7f8d4c0cda90 2024-03-30T00:48:37.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.885+0000 7f8d43e00640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d28066690 0x7f8d28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:37.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.885+0000 7f8d43e00640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d28066690 0x7f8d28068b50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0c95f0 tx=0x7f8d340090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:37.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:37.889+0000 7f8d49600640 1 -- 172.21.15.66:0/48459070 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8d3c051ba0 con 0x7f8d4c0cda90 2024-03-30T00:48:38.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.049+0000 7f8d53800640 1 -- 172.21.15.66:0/48459070 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8d4c0c1990 con 0x7f8d4c0cda90 2024-03-30T00:48:38.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.053+0000 7f8d49600640 1 -- 172.21.15.66:0/48459070 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8d3c0551c0 con 0x7f8d4c0cda90 2024-03-30T00:48:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 -- 172.21.15.66:0/48459070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d28066690 msgr2=0x7f8d28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d28066690 0x7f8d28068b50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0c95f0 tx=0x7f8d340090e0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 -- 172.21.15.66:0/48459070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d4c0cda90 msgr2=0x7f8d4c159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d4c0cda90 0x7f8d4c159080 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00ebe0 tx=0x7f8d3c00c520 comp rx=0 tx=0).stop 2024-03-30T00:48:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 -- 172.21.15.66:0/48459070 shutdown_connections 2024-03-30T00:48:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d4c15a370 0x7f8d4c1556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d4c0cde70 0x7f8d4c155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8d28066690 0x7f8d28068b50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 --2- 172.21.15.66:0/48459070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d4c0cda90 0x7f8d4c159080 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 -- 172.21.15.66:0/48459070 >> 172.21.15.66:0/48459070 conn(0x7f8d4c069f60 msgr2=0x7f8d4c06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 -- 172.21.15.66:0/48459070 shutdown_connections 2024-03-30T00:48:38.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.057+0000 7f8d53800640 1 -- 172.21.15.66:0/48459070 wait complete. 2024-03-30T00:48:38.087 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-03-30T00:48:38.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.217+0000 7f6e5e800640 1 Processor -- start 2024-03-30T00:48:38.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.217+0000 7f6e5e800640 1 -- start start 2024-03-30T00:48:38.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.217+0000 7f6e5e800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e58160c10 0x7f6e580c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.217+0000 7f6e5e800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6e580c19d0 0x7f6e580cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.217+0000 7f6e5e800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e580c9a30 con 0x7f6e580b19a0 2024-03-30T00:48:38.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.217+0000 7f6e5e800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e580c9d10 con 0x7f6e580c19d0 2024-03-30T00:48:38.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.217+0000 7f6e5e800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580c9ff0 con 0x7f6e58160c10 2024-03-30T00:48:38.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.217+0000 7f6e4ea00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6e580b19a0 0x7f6e580c12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45908/0 (socket says 172.21.15.66:45908) 2024-03-30T00:48:38.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.217+0000 7f6e4ea00640 1 -- 172.21.15.66:0/907167153 learned_addr learned my addr 172.21.15.66:0/907167153 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:38.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e57e00640 1 --2- 172.21.15.66:0/907167153 >> v2:172.21.15.156:3300/0 conn(0x7f6e580c19d0 0x7f6e580cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e57400640 1 --2- 172.21.15.66:0/907167153 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e58160c10 0x7f6e580c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e56a00640 1 -- 172.21.15.66:0/907167153 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930357863 0 0) 0x7f6e580c9a30 con 0x7f6e580b19a0 2024-03-30T00:48:38.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e56a00640 1 -- 172.21.15.66:0/907167153 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e3c003530 con 0x7f6e580b19a0 2024-03-30T00:48:38.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e56a00640 1 -- 172.21.15.66:0/907167153 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2549810619 0 0) 0x7f6e3c003530 con 0x7f6e580b19a0 2024-03-30T00:48:38.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e57e00640 1 -- 172.21.15.66:0/907167153 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e58160c10 msgr2=0x7f6e580c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:38.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e57e00640 1 --2- 172.21.15.66:0/907167153 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e58160c10 0x7f6e580c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e57e00640 1 -- 172.21.15.66:0/907167153 >> v1:172.21.15.66:6789/0 conn(0x7f6e580b19a0 legacy=0x7f6e580c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e57e00640 1 -- 172.21.15.66:0/907167153 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580ca2d0 con 0x7f6e580c19d0 2024-03-30T00:48:38.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e57e00640 1 --2- 172.21.15.66:0/907167153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580c19d0 0x7f6e580cc280 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6e500097c0 tx=0x7f6e5001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=944117d1aa037aac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:38.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e56a00640 1 -- 172.21.15.66:0/907167153 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e580c9830 con 0x7f6e580c19d0 2024-03-30T00:48:38.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e56a00640 1 -- 172.21.15.66:0/907167153 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e500a7050 con 0x7f6e580c19d0 2024-03-30T00:48:38.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.221+0000 7f6e56a00640 1 -- 172.21.15.66:0/907167153 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40001680 con 0x7f6e580c19d0 2024-03-30T00:48:38.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 -- 172.21.15.66:0/907167153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580c19d0 msgr2=0x7f6e580cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 --2- 172.21.15.66:0/907167153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580c19d0 0x7f6e580cc280 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6e500097c0 tx=0x7f6e5001c3c0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 -- 172.21.15.66:0/907167153 shutdown_connections 2024-03-30T00:48:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 --2- 172.21.15.66:0/907167153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e580c19d0 0x7f6e580cc280 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 --2- 172.21.15.66:0/907167153 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e58160c10 0x7f6e580c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 -- 172.21.15.66:0/907167153 >> 172.21.15.66:0/907167153 conn(0x7f6e58069f60 msgr2=0x7f6e5806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 -- 172.21.15.66:0/907167153 shutdown_connections 2024-03-30T00:48:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 -- 172.21.15.66:0/907167153 wait complete. 2024-03-30T00:48:38.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 Processor -- start 2024-03-30T00:48:38.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 -- start start 2024-03-30T00:48:38.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e580b19a0 0x7f6e5806c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e580c19d0 0x7f6e5806c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58160c10 0x7f6e58157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580ab360 con 0x7f6e580c19d0 2024-03-30T00:48:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580ab010 con 0x7f6e58160c10 2024-03-30T00:48:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e5e800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815daa0 con 0x7f6e580b19a0 2024-03-30T00:48:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e4ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e580c19d0 0x7f6e5806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e4ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e580c19d0 0x7f6e5806c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48668/0 (socket says 172.21.15.66:48668) 2024-03-30T00:48:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e4ea00640 1 -- 172.21.15.66:0/368745885 learned_addr learned my addr 172.21.15.66:0/368745885 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e57e00640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58160c10 0x7f6e58157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e57400640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e580b19a0 0x7f6e5806c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e4ea00640 1 -- 172.21.15.66:0/368745885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e580b19a0 msgr2=0x7f6e5806c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e4ea00640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e580b19a0 0x7f6e5806c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e4ea00640 1 -- 172.21.15.66:0/368745885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58160c10 msgr2=0x7f6e58157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e4ea00640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58160c10 0x7f6e58157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e4ea00640 1 -- 172.21.15.66:0/368745885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50009420 con 0x7f6e580c19d0 2024-03-30T00:48:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.225+0000 7f6e57400640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e580b19a0 0x7f6e5806c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.229+0000 7f6e4ea00640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e580c19d0 0x7f6e5806c570 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c790 tx=0x7f6e4000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.229+0000 7f6e4fe00640 1 -- 172.21.15.66:0/368745885 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40005ae0 con 0x7f6e580c19d0 2024-03-30T00:48:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.229+0000 7f6e4fe00640 1 -- 172.21.15.66:0/368745885 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e40005c40 con 0x7f6e580c19d0 2024-03-30T00:48:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.229+0000 7f6e4fe00640 1 -- 172.21.15.66:0/368745885 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40007070 con 0x7f6e580c19d0 2024-03-30T00:48:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.229+0000 7f6e5e800640 1 -- 172.21.15.66:0/368745885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5815dd80 con 0x7f6e580c19d0 2024-03-30T00:48:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.229+0000 7f6e5e800640 1 -- 172.21.15.66:0/368745885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e580700f0 con 0x7f6e580c19d0 2024-03-30T00:48:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.229+0000 7f6e5e800640 1 -- 172.21.15.66:0/368745885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e580ab4a0 con 0x7f6e580c19d0 2024-03-30T00:48:38.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.229+0000 7f6e4fe00640 1 -- 172.21.15.66:0/368745885 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6e400071d0 con 0x7f6e580c19d0 2024-03-30T00:48:38.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.233+0000 7f6e4fe00640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6e34066690 0x7f6e34068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.233+0000 7f6e4fe00640 1 -- 172.21.15.66:0/368745885 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f6e40084720 con 0x7f6e580c19d0 2024-03-30T00:48:38.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.233+0000 7f6e57400640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6e34066690 0x7f6e34068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.233+0000 7f6e57400640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6e34066690 0x7f6e34068b50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6e58158120 tx=0x7f6e4800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.233+0000 7f6e4fe00640 1 -- 172.21.15.66:0/368745885 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6e40051d10 con 0x7f6e580c19d0 2024-03-30T00:48:38.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.453+0000 7f6e5e800640 1 -- 172.21.15.66:0/368745885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f6e580ab4a0 con 0x7f6e580c19d0 2024-03-30T00:48:38.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.689+0000 7f6e4fe00640 1 -- 172.21.15.66:0/368745885 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v426) v1 ==== 167+0+0 (secure 0 0 0) 0x7f6e580ab4a0 con 0x7f6e580c19d0 2024-03-30T00:48:38.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 -- 172.21.15.66:0/368745885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6e34066690 msgr2=0x7f6e34068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6e34066690 0x7f6e34068b50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6e58158120 tx=0x7f6e4800b040 comp rx=0 tx=0).stop 2024-03-30T00:48:38.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 -- 172.21.15.66:0/368745885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e580c19d0 msgr2=0x7f6e5806c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e580c19d0 0x7f6e5806c570 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c790 tx=0x7f6e4000cc50 comp rx=0 tx=0).stop 2024-03-30T00:48:38.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 -- 172.21.15.66:0/368745885 shutdown_connections 2024-03-30T00:48:38.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e58160c10 0x7f6e58157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6e34066690 0x7f6e34068b50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e580c19d0 0x7f6e5806c570 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 --2- 172.21.15.66:0/368745885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e580b19a0 0x7f6e5806c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 -- 172.21.15.66:0/368745885 >> 172.21.15.66:0/368745885 conn(0x7f6e58069f60 msgr2=0x7f6e580b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:38.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 -- 172.21.15.66:0/368745885 shutdown_connections 2024-03-30T00:48:38.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.693+0000 7f6e5e800640 1 -- 172.21.15.66:0/368745885 wait complete. 2024-03-30T00:48:38.721 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-30T00:48:38.721 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-30T00:48:38.721 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:48:38.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f3591600640 1 Processor -- start 2024-03-30T00:48:38.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f3591600640 1 -- start start 2024-03-30T00:48:38.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f3591600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f358c0cda90 0x7f358c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f3591600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c0cd3e0 con 0x7f358c15a370 2024-03-30T00:48:38.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f3591600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c15a990 con 0x7f358c0cde70 2024-03-30T00:48:38.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f3591600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c0b1b80 con 0x7f358c0cda90 2024-03-30T00:48:38.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358b400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f358c15a370 0x7f358c156910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34590/0 (socket says 172.21.15.66:34590) 2024-03-30T00:48:38.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358b400640 1 -- 172.21.15.66:0/1791360433 learned_addr learned my addr 172.21.15.66:0/1791360433 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:38.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358aa00640 1 --2- 172.21.15.66:0/1791360433 >> v2:172.21.15.156:3300/0 conn(0x7f358c0cda90 0x7f358c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660403671 0 0) 0x7f358c0cd3e0 con 0x7f358c15a370 2024-03-30T00:48:38.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3570003530 con 0x7f358c15a370 2024-03-30T00:48:38.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217556527 0 0) 0x7f358c15a990 con 0x7f358c0cde70 2024-03-30T00:48:38.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f358c0cd3e0 con 0x7f358c0cde70 2024-03-30T00:48:38.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 771048304 0 0) 0x7f3570003530 con 0x7f358c15a370 2024-03-30T00:48:38.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 >> v2:172.21.15.156:3300/0 conn(0x7f358c0cda90 msgr2=0x7f358c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:38.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 --2- 172.21.15.66:0/1791360433 >> v2:172.21.15.156:3300/0 conn(0x7f358c0cda90 0x7f358c0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 >> v1:172.21.15.156:6789/0 conn(0x7f358c0cde70 legacy=0x7f358c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0b11e0 con 0x7f358c15a370 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f3584001a00 con 0x7f358c15a370 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 >> v1:172.21.15.66:6789/0 conn(0x7f358c15a370 legacy=0x7f358c156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3570003b10 0x7f3570003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35700047d0 0x7f3570004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 0x7f357000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0b1b80 con 0x7f3570005190 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357000c0c0 con 0x7f3570003b10 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357000c280 con 0x7f35700047d0 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358b400640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 0x7f357000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f3583e00640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35700047d0 0x7f3570004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358aa00640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3570003b10 0x7f3570003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358b400640 1 -- 172.21.15.66:0/1791360433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35700047d0 msgr2=0x7f3570004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:38.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358b400640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35700047d0 0x7f3570004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358b400640 1 -- 172.21.15.66:0/1791360433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3570003b10 msgr2=0x7f3570003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:38.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358b400640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3570003b10 0x7f3570003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358b400640 1 -- 172.21.15.66:0/1791360433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3570003530 con 0x7f3570005190 2024-03-30T00:48:38.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.853+0000 7f358aa00640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3570003b10 0x7f3570003f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:38.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f358b400640 1 -- 172.21.15.66:0/1791360433 reap_dead start 2024-03-30T00:48:38.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f358b400640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 0x7f357000b990 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f358400a7c0 tx=0x7f358401f4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb031d854704f82c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:38.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840ae5f0 con 0x7f3570005190 2024-03-30T00:48:38.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35840ae770 con 0x7f3570005190 2024-03-30T00:48:38.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f358a000640 1 -- 172.21.15.66:0/1791360433 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840aeb20 con 0x7f3570005190 2024-03-30T00:48:38.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f3591600640 1 -- 172.21.15.66:0/1791360433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 msgr2=0x7f357000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f3591600640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 0x7f357000b990 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f358400a7c0 tx=0x7f358401f4d0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f3591600640 1 -- 172.21.15.66:0/1791360433 shutdown_connections 2024-03-30T00:48:38.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f3591600640 1 --2- 172.21.15.66:0/1791360433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 0x7f357000b990 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f3591600640 1 -- 172.21.15.66:0/1791360433 >> 172.21.15.66:0/1791360433 conn(0x7f358c069f60 msgr2=0x7f358c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:38.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f3591600640 1 -- 172.21.15.66:0/1791360433 shutdown_connections 2024-03-30T00:48:38.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.857+0000 7f3591600640 1 -- 172.21.15.66:0/1791360433 wait complete. 2024-03-30T00:48:38.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3591600640 1 Processor -- start 2024-03-30T00:48:38.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3591600640 1 -- start start 2024-03-30T00:48:38.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3591600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 0x7f358c1f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3591600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f358c0ab010 0x7f358c09ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3591600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f358c0c5550 0x7f358c0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3591600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0705a0 con 0x7f3570005190 2024-03-30T00:48:38.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3591600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c09bf70 con 0x7f358c0ab010 2024-03-30T00:48:38.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3591600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1f26a0 con 0x7f358c0c5550 2024-03-30T00:48:38.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f358aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 0x7f358c1f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f358aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 0x7f358c1f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35692/0 (socket says 172.21.15.66:35692) 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f358aa00640 1 -- 172.21.15.66:0/2934262871 learned_addr learned my addr 172.21.15.66:0/2934262871 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f358aa00640 1 -- 172.21.15.66:0/2934262871 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f358c0c5550 msgr2=0x7f358c0c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f358aa00640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f358c0c5550 0x7f358c0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f358aa00640 1 -- 172.21.15.66:0/2934262871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f358c0ab010 msgr2=0x7f358c09ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f358aa00640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f358c0ab010 0x7f358c09ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f358aa00640 1 -- 172.21.15.66:0/2934262871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c1f2980 con 0x7f3570005190 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f358aa00640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 0x7f358c1f20b0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f357c009160 tx=0x7f357c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3582a00640 1 -- 172.21.15.66:0/2934262871 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c004220 con 0x7f3570005190 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3582a00640 1 -- 172.21.15.66:0/2934262871 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c004380 con 0x7f3570005190 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3582a00640 1 -- 172.21.15.66:0/2934262871 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c015890 con 0x7f3570005190 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3591600640 1 -- 172.21.15.66:0/2934262871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c1f2c60 con 0x7f3570005190 2024-03-30T00:48:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.861+0000 7f3591600640 1 -- 172.21.15.66:0/2934262871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c0c59e0 con 0x7f3570005190 2024-03-30T00:48:38.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.865+0000 7f3591600640 1 -- 172.21.15.66:0/2934262871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c1f3110 con 0x7f3570005190 2024-03-30T00:48:38.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.865+0000 7f3582a00640 1 -- 172.21.15.66:0/2934262871 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f357c0159f0 con 0x7f3570005190 2024-03-30T00:48:38.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.865+0000 7f3582a00640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3568066690 0x7f3568068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:38.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.865+0000 7f3582a00640 1 -- 172.21.15.66:0/2934262871 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f357c0845e0 con 0x7f3570005190 2024-03-30T00:48:38.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.865+0000 7f3583e00640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3568066690 0x7f3568068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:38.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.865+0000 7f3583e00640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3568066690 0x7f3568068b50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3574007b70 tx=0x7f3574007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:38.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:38.869+0000 7f3582a00640 1 -- 172.21.15.66:0/2934262871 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f357c0519c0 con 0x7f3570005190 2024-03-30T00:48:39.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.033+0000 7f3591600640 1 -- 172.21.15.66:0/2934262871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f358c0c8c50 con 0x7f3570005190 2024-03-30T00:48:39.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.033+0000 7f3582a00640 1 -- 172.21.15.66:0/2934262871 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+12985 (secure 0 0 0) 0x7f357c054fe0 con 0x7f3570005190 2024-03-30T00:48:39.043 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:39.044 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":101,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:48:37.685400+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6836/2419209461":"2024-03-31T00:48:34.340940+0000","172.21.15.66:6835/1626022080":"2024-03-31T00:47:49.264137+0000","172.21.15.66:6835/2419209461":"2024-03-31T00:48:34.340940+0000","172.21.15.66:6836/1626022080":"2024-03-31T00:47:49.264137+0000","172.21.15.66:6836/1147664160":"2024-03-31T00:46:34.103831+0000","172.21.15.66:6835/1147664160":"2024-03-31T00:46:34.103831+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:48:39.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 -- 172.21.15.66:0/2934262871 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3568066690 msgr2=0x7f3568068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3568066690 0x7f3568068b50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3574007b70 tx=0x7f3574007a80 comp rx=0 tx=0).stop 2024-03-30T00:48:39.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 -- 172.21.15.66:0/2934262871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 msgr2=0x7f358c1f20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 0x7f358c1f20b0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f357c009160 tx=0x7f357c011a60 comp rx=0 tx=0).stop 2024-03-30T00:48:39.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 -- 172.21.15.66:0/2934262871 shutdown_connections 2024-03-30T00:48:39.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f358c0c5550 0x7f358c0c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f358c0ab010 0x7f358c09ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3568066690 0x7f3568068b50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 --2- 172.21.15.66:0/2934262871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3570005190 0x7f358c1f20b0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 -- 172.21.15.66:0/2934262871 >> 172.21.15.66:0/2934262871 conn(0x7f358c069f60 msgr2=0x7f358c0cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:39.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 -- 172.21.15.66:0/2934262871 shutdown_connections 2024-03-30T00:48:39.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.037+0000 7f3591600640 1 -- 172.21.15.66:0/2934262871 wait complete. 2024-03-30T00:48:39.065 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:48:39.065 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:48:39.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.197+0000 7f7424e00640 1 Processor -- start 2024-03-30T00:48:39.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.197+0000 7f7424e00640 1 -- start start 2024-03-30T00:48:39.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f7424e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f742015ca80 0x7f742006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f7424e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74200cb0e0 con 0x7f74200cf2f0 2024-03-30T00:48:39.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f7424e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74200cf7f0 con 0x7f74200cb570 2024-03-30T00:48:39.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f7424e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74200cfad0 con 0x7f742015ca80 2024-03-30T00:48:39.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f7415400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f74200cf2f0 0x7f742006b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34596/0 (socket says 172.21.15.66:34596) 2024-03-30T00:48:39.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f7415400640 1 -- 172.21.15.66:0/1835412012 learned_addr learned my addr 172.21.15.66:0/1835412012 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:39.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741de00640 1 --2- 172.21.15.66:0/1835412012 >> v2:172.21.15.156:3300/0 conn(0x7f742015ca80 0x7f742006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107210168 0 0) 0x7f74200cb0e0 con 0x7f74200cf2f0 2024-03-30T00:48:39.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7404003530 con 0x7f74200cf2f0 2024-03-30T00:48:39.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 92205784 0 0) 0x7f7404003530 con 0x7f74200cf2f0 2024-03-30T00:48:39.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 >> v2:172.21.15.156:3300/0 conn(0x7f742015ca80 msgr2=0x7f742006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:39.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 --2- 172.21.15.66:0/1835412012 >> v2:172.21.15.156:3300/0 conn(0x7f742015ca80 0x7f742006b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 >> v1:172.21.15.156:6789/0 conn(0x7f74200cb570 legacy=0x7f74200cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74200cad70 con 0x7f74200cf2f0 2024-03-30T00:48:39.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f7408001680 con 0x7f74200cf2f0 2024-03-30T00:48:39.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 >> v1:172.21.15.66:6789/0 conn(0x7f74200cf2f0 legacy=0x7f742006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 --2- 172.21.15.66:0/1835412012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74040039d0 0x7f7404003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 --2- 172.21.15.66:0/1835412012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74040047d0 0x7f7404004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 --2- 172.21.15.66:0/1835412012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7404005190 0x7f740400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74200cfad0 con 0x7f7404005190 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740400c070 con 0x7f74040047d0 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740400c230 con 0x7f74040039d0 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741e800640 1 --2- 172.21.15.66:0/1835412012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7404005190 0x7f740400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741de00640 1 --2- 172.21.15.66:0/1835412012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74040039d0 0x7f7404003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f7415400640 1 --2- 172.21.15.66:0/1835412012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74040047d0 0x7f7404004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741e800640 1 -- 172.21.15.66:0/1835412012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74040039d0 msgr2=0x7f7404003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741e800640 1 --2- 172.21.15.66:0/1835412012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74040039d0 0x7f7404003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741e800640 1 -- 172.21.15.66:0/1835412012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74040047d0 msgr2=0x7f7404004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741e800640 1 --2- 172.21.15.66:0/1835412012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74040047d0 0x7f7404004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.201+0000 7f741e800640 1 -- 172.21.15.66:0/1835412012 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7404003530 con 0x7f7404005190 2024-03-30T00:48:39.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f741e800640 1 -- 172.21.15.66:0/1835412012 reap_dead start 2024-03-30T00:48:39.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f741e800640 1 --2- 172.21.15.66:0/1835412012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7404005190 0x7f740400b990 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7418001e70 tx=0x7f741801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756d17dd44c0ff39 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:39.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418003b80 con 0x7f7404005190 2024-03-30T00:48:39.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7418003d20 con 0x7f7404005190 2024-03-30T00:48:39.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f741d400640 1 -- 172.21.15.66:0/1835412012 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418005260 con 0x7f7404005190 2024-03-30T00:48:39.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f7424e00640 1 -- 172.21.15.66:0/1835412012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7404005190 msgr2=0x7f740400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f7424e00640 1 --2- 172.21.15.66:0/1835412012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7404005190 0x7f740400b990 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7418001e70 tx=0x7f741801c780 comp rx=0 tx=0).stop 2024-03-30T00:48:39.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f7424e00640 1 -- 172.21.15.66:0/1835412012 shutdown_connections 2024-03-30T00:48:39.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f7424e00640 1 --2- 172.21.15.66:0/1835412012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7404005190 0x7f740400b990 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f7424e00640 1 -- 172.21.15.66:0/1835412012 >> 172.21.15.66:0/1835412012 conn(0x7f74200b2090 msgr2=0x7f74200b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:39.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f7424e00640 1 -- 172.21.15.66:0/1835412012 shutdown_connections 2024-03-30T00:48:39.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f7424e00640 1 -- 172.21.15.66:0/1835412012 wait complete. 2024-03-30T00:48:39.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.205+0000 7f7424e00640 1 Processor -- start 2024-03-30T00:48:39.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7424e00640 1 -- start start 2024-03-30T00:48:39.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7424e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7404005190 0x7f7420159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7424e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f742006f2e0 0x7f742006f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7424e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74200c76e0 0x7f74200c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7424e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742006fca0 con 0x7f742006f2e0 2024-03-30T00:48:39.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7424e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742015a1e0 con 0x7f74200c76e0 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7424e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74200c8410 con 0x7f7404005190 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7415400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f742006f2e0 0x7f742006f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7415400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f742006f2e0 0x7f742006f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35702/0 (socket says 172.21.15.66:35702) 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7415400640 1 -- 172.21.15.66:0/283126341 learned_addr learned my addr 172.21.15.66:0/283126341 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f741de00640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7404005190 0x7f7420159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f741e800640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74200c76e0 0x7f74200c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7415400640 1 -- 172.21.15.66:0/283126341 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7404005190 msgr2=0x7f7420159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7415400640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7404005190 0x7f7420159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7415400640 1 -- 172.21.15.66:0/283126341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74200c76e0 msgr2=0x7f74200c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7415400640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74200c76e0 0x7f74200c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7415400640 1 -- 172.21.15.66:0/283126341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74200c86f0 con 0x7f742006f2e0 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f741e800640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74200c76e0 0x7f74200c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7415400640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f742006f2e0 0x7f742006f760 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7408004ed0 tx=0x7f7408007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:39.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7416a00640 1 -- 172.21.15.66:0/283126341 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740800f7b0 con 0x7f742006f2e0 2024-03-30T00:48:39.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7416a00640 1 -- 172.21.15.66:0/283126341 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740800f930 con 0x7f742006f2e0 2024-03-30T00:48:39.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7416a00640 1 -- 172.21.15.66:0/283126341 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740800fce0 con 0x7f742006f2e0 2024-03-30T00:48:39.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7424e00640 1 -- 172.21.15.66:0/283126341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74200c89d0 con 0x7f742006f2e0 2024-03-30T00:48:39.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7424e00640 1 -- 172.21.15.66:0/283126341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74200c80b0 con 0x7f742006f2e0 2024-03-30T00:48:39.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.209+0000 7f7424e00640 1 -- 172.21.15.66:0/283126341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742009b4c0 con 0x7f742006f2e0 2024-03-30T00:48:39.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.213+0000 7f7416a00640 1 -- 172.21.15.66:0/283126341 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f740801d460 con 0x7f742006f2e0 2024-03-30T00:48:39.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.213+0000 7f7416a00640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73fc066690 0x7f73fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.213+0000 7f7416a00640 1 -- 172.21.15.66:0/283126341 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f74080840e0 con 0x7f742006f2e0 2024-03-30T00:48:39.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.213+0000 7f741de00640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73fc066690 0x7f73fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.213+0000 7f741de00640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73fc066690 0x7f73fc068b50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f74200cacd0 tx=0x7f7410009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:39.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.217+0000 7f7416a00640 1 -- 172.21.15.66:0/283126341 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7408051550 con 0x7f742006f2e0 2024-03-30T00:48:39.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 -- 172.21.15.66:0/283126341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74200c8bc0 con 0x7f742006f2e0 2024-03-30T00:48:39.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7416a00640 1 -- 172.21.15.66:0/283126341 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7408054b70 con 0x7f742006f2e0 2024-03-30T00:48:39.399 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 -- 172.21.15.66:0/283126341 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73fc066690 msgr2=0x7f73fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73fc066690 0x7f73fc068b50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f74200cacd0 tx=0x7f7410009190 comp rx=0 tx=0).stop 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 -- 172.21.15.66:0/283126341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f742006f2e0 msgr2=0x7f742006f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f742006f2e0 0x7f742006f760 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7408004ed0 tx=0x7f7408007430 comp rx=0 tx=0).stop 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 -- 172.21.15.66:0/283126341 shutdown_connections 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74200c76e0 0x7f74200c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f73fc066690 0x7f73fc068b50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f742006f2e0 0x7f742006f760 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 --2- 172.21.15.66:0/283126341 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7404005190 0x7f7420159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 -- 172.21.15.66:0/283126341 >> 172.21.15.66:0/283126341 conn(0x7f74200b2090 msgr2=0x7f74200cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 -- 172.21.15.66:0/283126341 shutdown_connections 2024-03-30T00:48:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.389+0000 7f7424e00640 1 -- 172.21.15.66:0/283126341 wait complete. 2024-03-30T00:48:39.420 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-03-30T00:48:39.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a2aa00640 1 Processor -- start 2024-03-30T00:48:39.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a2aa00640 1 -- start start 2024-03-30T00:48:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a2aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3a240cda90 0x7f3a240ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a240cde70 0x7f3a240cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a2aa00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2415a990 con 0x7f3a24159c60 2024-03-30T00:48:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a2aa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a240b1b80 con 0x7f3a240cda90 2024-03-30T00:48:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a2aa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a240cd3e0 con 0x7f3a240cde70 2024-03-30T00:48:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a23e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3a240cda90 0x7f3a240ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a23e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3a240cda90 0x7f3a240ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35708/0 (socket says 172.21.15.66:35708) 2024-03-30T00:48:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a23e00640 1 -- 172.21.15.66:0/3478825945 learned_addr learned my addr 172.21.15.66:0/3478825945 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a1b400640 1 --2- 172.21.15.66:0/3478825945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a240cde70 0x7f3a240cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a23e00640 1 -- 172.21.15.66:0/3478825945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a240cde70 msgr2=0x7f3a240cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a23e00640 1 --2- 172.21.15.66:0/3478825945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a240cde70 0x7f3a240cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a23e00640 1 -- 172.21.15.66:0/3478825945 >> v1:172.21.15.156:6789/0 conn(0x7f3a24159c60 legacy=0x7f3a2415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.553+0000 7f3a23e00640 1 -- 172.21.15.66:0/3478825945 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a240b11e0 con 0x7f3a240cda90 2024-03-30T00:48:39.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a23e00640 1 --2- 172.21.15.66:0/3478825945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a240cda90 0x7f3a240ce520 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009e50 tx=0x7f3a1401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4090f75dd87da25a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:39.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a23400640 1 -- 172.21.15.66:0/3478825945 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140a5770 con 0x7f3a240cda90 2024-03-30T00:48:39.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a23400640 1 -- 172.21.15.66:0/3478825945 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a140a58d0 con 0x7f3a240cda90 2024-03-30T00:48:39.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a23400640 1 -- 172.21.15.66:0/3478825945 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140a5c60 con 0x7f3a240cda90 2024-03-30T00:48:39.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3478825945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a240cda90 msgr2=0x7f3a240ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a2aa00640 1 --2- 172.21.15.66:0/3478825945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a240cda90 0x7f3a240ce520 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009e50 tx=0x7f3a1401c9a0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3478825945 shutdown_connections 2024-03-30T00:48:39.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a2aa00640 1 --2- 172.21.15.66:0/3478825945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a240cde70 0x7f3a240cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a2aa00640 1 --2- 172.21.15.66:0/3478825945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a240cda90 0x7f3a240ce520 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3478825945 >> 172.21.15.66:0/3478825945 conn(0x7f3a24069f60 msgr2=0x7f3a2406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3478825945 shutdown_connections 2024-03-30T00:48:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3478825945 wait complete. 2024-03-30T00:48:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a2aa00640 1 Processor -- start 2024-03-30T00:48:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.557+0000 7f3a2aa00640 1 -- start start 2024-03-30T00:48:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a240cda90 0x7f3a24154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a240cde70 0x7f3a240c3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a24159c60 0x7f3a240ac060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a2aa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a24070000 con 0x7f3a240cde70 2024-03-30T00:48:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a2aa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a240702e0 con 0x7f3a240cda90 2024-03-30T00:48:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a2aa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a240705c0 con 0x7f3a24159c60 2024-03-30T00:48:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a1b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a240cde70 0x7f3a240c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a1b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a240cde70 0x7f3a240c3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35718/0 (socket says 172.21.15.66:35718) 2024-03-30T00:48:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a1b400640 1 -- 172.21.15.66:0/3742910853 learned_addr learned my addr 172.21.15.66:0/3742910853 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a23e00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a240cda90 0x7f3a24154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a28a00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a24159c60 0x7f3a240ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a1b400640 1 -- 172.21.15.66:0/3742910853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a24159c60 msgr2=0x7f3a240ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a1b400640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a24159c60 0x7f3a240ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a1b400640 1 -- 172.21.15.66:0/3742910853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a240cda90 msgr2=0x7f3a24154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a1b400640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a240cda90 0x7f3a24154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a1b400640 1 -- 172.21.15.66:0/3742910853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a14009b30 con 0x7f3a240cde70 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a28a00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a24159c60 0x7f3a240ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a1b400640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a240cde70 0x7f3a240c3300 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d800 tx=0x7f3a0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a20c00640 1 -- 172.21.15.66:0/3742910853 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00b6c0 con 0x7f3a240cde70 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a20c00640 1 -- 172.21.15.66:0/3742910853 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c00b820 con 0x7f3a240cde70 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a20c00640 1 -- 172.21.15.66:0/3742910853 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00bb90 con 0x7f3a240cde70 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3742910853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a240708a0 con 0x7f3a240cde70 2024-03-30T00:48:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3742910853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a240acdd0 con 0x7f3a240cde70 2024-03-30T00:48:39.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3742910853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a24066c30 con 0x7f3a240cde70 2024-03-30T00:48:39.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.561+0000 7f3a20c00640 1 -- 172.21.15.66:0/3742910853 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3a0c00bcf0 con 0x7f3a240cde70 2024-03-30T00:48:39.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.565+0000 7f3a20c00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a000666c0 0x7f3a00068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.565+0000 7f3a20c00640 1 -- 172.21.15.66:0/3742910853 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f3a0c084560 con 0x7f3a240cde70 2024-03-30T00:48:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.565+0000 7f3a23e00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a000666c0 0x7f3a00068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.565+0000 7f3a23e00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a000666c0 0x7f3a00068b80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3a140ae000 tx=0x7f3a140a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:39.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.569+0000 7f3a20c00640 1 -- 172.21.15.66:0/3742910853 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3a0c0519d0 con 0x7f3a240cde70 2024-03-30T00:48:39.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:39.785+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3742910853 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f3a24070140 con 0x7f3a240cde70 2024-03-30T00:48:40.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.701+0000 7f3a20c00640 1 -- 172.21.15.66:0/3742910853 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v428) v1 ==== 149+0+0 (secure 0 0 0) 0x7f3a24070140 con 0x7f3a240cde70 2024-03-30T00:48:40.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3742910853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a000666c0 msgr2=0x7f3a00068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:40.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a000666c0 0x7f3a00068b80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3a140ae000 tx=0x7f3a140a7070 comp rx=0 tx=0).stop 2024-03-30T00:48:40.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3742910853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a240cde70 msgr2=0x7f3a240c3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:40.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a240cde70 0x7f3a240c3300 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d800 tx=0x7f3a0c00dcc0 comp rx=0 tx=0).stop 2024-03-30T00:48:40.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3742910853 shutdown_connections 2024-03-30T00:48:40.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a24159c60 0x7f3a240ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:40.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a000666c0 0x7f3a00068b80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:40.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a240cde70 0x7f3a240c3300 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:40.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 --2- 172.21.15.66:0/3742910853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a240cda90 0x7f3a24154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:40.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3742910853 >> 172.21.15.66:0/3742910853 conn(0x7f3a24069f60 msgr2=0x7f3a2406d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:40.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3742910853 shutdown_connections 2024-03-30T00:48:40.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.705+0000 7f3a2aa00640 1 -- 172.21.15.66:0/3742910853 wait complete. 2024-03-30T00:48:40.737 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-30T00:48:40.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff902000640 1 Processor -- start 2024-03-30T00:48:40.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff902000640 1 -- start start 2024-03-30T00:48:40.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff902000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff8fc15a640 0x7ff8fc06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:40.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff902000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8fc0ce520 0x7ff8fc0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:40.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff902000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc06d070 con 0x7ff8fc0cabb0 2024-03-30T00:48:40.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff902000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc0cbba0 con 0x7ff8fc0ce520 2024-03-30T00:48:40.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff902000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc0cee90 con 0x7ff8fc15a640 2024-03-30T00:48:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff8f3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8fc0ce520 0x7ff8fc0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff8f3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8fc0ce520 0x7ff8fc0ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35722/0 (socket says 172.21.15.66:35722) 2024-03-30T00:48:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff8f3e00640 1 -- 172.21.15.66:0/1929286617 learned_addr learned my addr 172.21.15.66:0/1929286617 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff8fbe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff8fc0cabb0 0x7ff8fc0cafe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34606/0 (socket says 172.21.15.66:34606) 2024-03-30T00:48:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff8fb400640 1 --2- 172.21.15.66:0/1929286617 >> v2:172.21.15.156:3300/0 conn(0x7ff8fc15a640 0x7ff8fc06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff8f3e00640 1 -- 172.21.15.66:0/1929286617 >> v2:172.21.15.156:3300/0 conn(0x7ff8fc15a640 msgr2=0x7ff8fc06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff8f3e00640 1 --2- 172.21.15.66:0/1929286617 >> v2:172.21.15.156:3300/0 conn(0x7ff8fc15a640 0x7ff8fc06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff8f3e00640 1 -- 172.21.15.66:0/1929286617 >> v1:172.21.15.66:6789/0 conn(0x7ff8fc0cabb0 legacy=0x7ff8fc0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff8f3e00640 1 -- 172.21.15.66:0/1929286617 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc06e2c0 con 0x7ff8fc0ce520 2024-03-30T00:48:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.877+0000 7ff8fb400640 1 --2- 172.21.15.66:0/1929286617 >> v2:172.21.15.156:3300/0 conn(0x7ff8fc15a640 0x7ff8fc06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:40.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff8f3e00640 1 --2- 172.21.15.66:0/1929286617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8fc0ce520 0x7ff8fc0ce950 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0cf160 tx=0x7ff8e401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9443e33b9920662 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:40.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff8faa00640 1 -- 172.21.15.66:0/1929286617 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40ab610 con 0x7ff8fc0ce520 2024-03-30T00:48:40.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff8faa00640 1 -- 172.21.15.66:0/1929286617 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e40ab770 con 0x7ff8fc0ce520 2024-03-30T00:48:40.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff8faa00640 1 -- 172.21.15.66:0/1929286617 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40abb00 con 0x7ff8fc0ce520 2024-03-30T00:48:40.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff902000640 1 -- 172.21.15.66:0/1929286617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8fc0ce520 msgr2=0x7ff8fc0ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:40.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff902000640 1 --2- 172.21.15.66:0/1929286617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8fc0ce520 0x7ff8fc0ce950 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0cf160 tx=0x7ff8e401c520 comp rx=0 tx=0).stop 2024-03-30T00:48:40.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff902000640 1 -- 172.21.15.66:0/1929286617 shutdown_connections 2024-03-30T00:48:40.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff902000640 1 --2- 172.21.15.66:0/1929286617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8fc0ce520 0x7ff8fc0ce950 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:40.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff902000640 1 --2- 172.21.15.66:0/1929286617 >> v2:172.21.15.156:3300/0 conn(0x7ff8fc15a640 0x7ff8fc06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:40.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff902000640 1 -- 172.21.15.66:0/1929286617 >> 172.21.15.66:0/1929286617 conn(0x7ff8fc069f60 msgr2=0x7ff8fc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:40.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff902000640 1 -- 172.21.15.66:0/1929286617 shutdown_connections 2024-03-30T00:48:40.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.881+0000 7ff902000640 1 -- 172.21.15.66:0/1929286617 wait complete. 2024-03-30T00:48:40.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff902000640 1 Processor -- start 2024-03-30T00:48:40.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff902000640 1 -- start start 2024-03-30T00:48:40.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff902000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8fc0cabb0 0x7ff8fc097830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:40.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff902000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8fc0ce520 0x7ff8fc0abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:40.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff902000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8fc15a640 0x7ff8fc0ac130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:40.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff902000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc1650a0 con 0x7ff8fc15a640 2024-03-30T00:48:40.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff902000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0aecf0 con 0x7ff8fc0ce520 2024-03-30T00:48:40.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff902000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0aefb0 con 0x7ff8fc0cabb0 2024-03-30T00:48:40.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8fbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8fc15a640 0x7ff8fc0ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:40.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8fbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8fc15a640 0x7ff8fc0ac130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35732/0 (socket says 172.21.15.66:35732) 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8fbe00640 1 -- 172.21.15.66:0/3388803730 learned_addr learned my addr 172.21.15.66:0/3388803730 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8f3e00640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8fc0ce520 0x7ff8fc0abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8fb400640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8fc0cabb0 0x7ff8fc097830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8fbe00640 1 -- 172.21.15.66:0/3388803730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8fc0cabb0 msgr2=0x7ff8fc097830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8fbe00640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8fc0cabb0 0x7ff8fc097830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8fbe00640 1 -- 172.21.15.66:0/3388803730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8fc0ce520 msgr2=0x7ff8fc0abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8fbe00640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8fc0ce520 0x7ff8fc0abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8fbe00640 1 -- 172.21.15.66:0/3388803730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e40094d0 con 0x7ff8fc15a640 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8f3e00640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8fc0ce520 0x7ff8fc0abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8fbe00640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8fc15a640 0x7ff8fc0ac130 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400c7a0 tx=0x7ff8f400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8f3400640 1 -- 172.21.15.66:0/3388803730 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400a640 con 0x7ff8fc15a640 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8f3400640 1 -- 172.21.15.66:0/3388803730 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f400a7a0 con 0x7ff8fc15a640 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff902000640 1 -- 172.21.15.66:0/3388803730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc0af290 con 0x7ff8fc15a640 2024-03-30T00:48:40.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff8f3400640 1 -- 172.21.15.66:0/3388803730 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400ab10 con 0x7ff8fc15a640 2024-03-30T00:48:40.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff902000640 1 -- 172.21.15.66:0/3388803730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc0af850 con 0x7ff8fc15a640 2024-03-30T00:48:40.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.885+0000 7ff902000640 1 -- 172.21.15.66:0/3388803730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc06a660 con 0x7ff8fc15a640 2024-03-30T00:48:40.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.889+0000 7ff8f3400640 1 -- 172.21.15.66:0/3388803730 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff8f400ac70 con 0x7ff8fc15a640 2024-03-30T00:48:40.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.889+0000 7ff8f3400640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8d8066690 0x7ff8d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:40.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.889+0000 7ff8f3400640 1 -- 172.21.15.66:0/3388803730 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5435+0+0 (secure 0 0 0) 0x7ff8f40837e0 con 0x7ff8fc15a640 2024-03-30T00:48:40.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.889+0000 7ff8fb400640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8d8066690 0x7ff8d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:40.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.889+0000 7ff8fb400640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8d8066690 0x7ff8d8068b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec006fd0 tx=0x7ff8ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:40.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:40.893+0000 7ff8f3400640 1 -- 172.21.15.66:0/3388803730 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff8f4050d00 con 0x7ff8fc15a640 2024-03-30T00:48:41.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.057+0000 7ff902000640 1 -- 172.21.15.66:0/3388803730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff8fc0b0800 con 0x7ff8fc15a640 2024-03-30T00:48:41.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.061+0000 7ff8f3400640 1 -- 172.21.15.66:0/3388803730 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff8f40170c0 con 0x7ff8fc15a640 2024-03-30T00:48:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 -- 172.21.15.66:0/3388803730 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8d8066690 msgr2=0x7ff8d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8d8066690 0x7ff8d8068b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec006fd0 tx=0x7ff8ec008040 comp rx=0 tx=0).stop 2024-03-30T00:48:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 -- 172.21.15.66:0/3388803730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8fc15a640 msgr2=0x7ff8fc0ac130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8fc15a640 0x7ff8fc0ac130 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400c7a0 tx=0x7ff8f400cc60 comp rx=0 tx=0).stop 2024-03-30T00:48:41.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 -- 172.21.15.66:0/3388803730 shutdown_connections 2024-03-30T00:48:41.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff8d8066690 0x7ff8d8068b50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8fc15a640 0x7ff8fc0ac130 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8fc0ce520 0x7ff8fc0abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 --2- 172.21.15.66:0/3388803730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8fc0cabb0 0x7ff8fc097830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 -- 172.21.15.66:0/3388803730 >> 172.21.15.66:0/3388803730 conn(0x7ff8fc069f60 msgr2=0x7ff8fc0ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:41.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 -- 172.21.15.66:0/3388803730 shutdown_connections 2024-03-30T00:48:41.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.065+0000 7ff902000640 1 -- 172.21.15.66:0/3388803730 wait complete. 2024-03-30T00:48:41.093 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:48:41.093 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:48:41.171 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:48:41.252 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:48:41.252 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:48:41.252 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:48:41.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:48:41.261 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:48:41.261 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:48:41.271 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:48:41.271 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:48:41.271 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:48:41.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:48:41.313 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:48:41.313 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:48:41.321 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:48:41.321 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-30T00:48:41.322 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:48:41.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.485+0000 7fe4b4600640 1 Processor -- start 2024-03-30T00:48:41.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.485+0000 7fe4b4600640 1 -- start start 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4b4600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe4ac0b0fb0 0x7fe4ac06b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4b4600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ac098ac0 con 0x7fe4ac0cca40 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4b4600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ac0cf3f0 con 0x7fe4ac0b1420 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4b4600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4ac0cf670 con 0x7fe4ac0b0fb0 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4b1200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe4ac0b0fb0 0x7fe4ac06b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4b1c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe4ac0cca40 0x7fe4ac0cac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34622/0 (socket says 172.21.15.66:34622) 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4b1200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe4ac0b0fb0 0x7fe4ac06b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35738/0 (socket says 172.21.15.66:35738) 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4b1c00640 1 -- 172.21.15.66:0/2431914202 learned_addr learned my addr 172.21.15.66:0/2431914202 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4b1200640 1 -- 172.21.15.66:0/2431914202 >> v1:172.21.15.156:6789/0 conn(0x7fe4ac0b1420 legacy=0x7fe4ac099250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4b1200640 1 -- 172.21.15.66:0/2431914202 >> v1:172.21.15.66:6789/0 conn(0x7fe4ac0cca40 legacy=0x7fe4ac0cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4b1200640 1 -- 172.21.15.66:0/2431914202 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac0cf950 con 0x7fe4ac0b0fb0 2024-03-30T00:48:41.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.489+0000 7fe4abe00640 1 -- 172.21.15.66:0/2431914202 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180531866 0 0) 0x7fe4ac098ac0 con 0x7fe4ac0cca40 2024-03-30T00:48:41.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b1200640 1 --2- 172.21.15.66:0/2431914202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ac0b0fb0 0x7fe4ac06b140 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fe4980097c0 tx=0x7fe49801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8b17b8795b00f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:41.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4abe00640 1 -- 172.21.15.66:0/2431914202 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4980ab600 con 0x7fe4ac0b0fb0 2024-03-30T00:48:41.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4abe00640 1 -- 172.21.15.66:0/2431914202 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4980a7050 con 0x7fe4ac0b0fb0 2024-03-30T00:48:41.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4abe00640 1 -- 172.21.15.66:0/2431914202 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4980ab980 con 0x7fe4ac0b0fb0 2024-03-30T00:48:41.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 -- 172.21.15.66:0/2431914202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ac0b0fb0 msgr2=0x7fe4ac06b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 --2- 172.21.15.66:0/2431914202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ac0b0fb0 0x7fe4ac06b140 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fe4980097c0 tx=0x7fe49801c430 comp rx=0 tx=0).stop 2024-03-30T00:48:41.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 -- 172.21.15.66:0/2431914202 shutdown_connections 2024-03-30T00:48:41.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 --2- 172.21.15.66:0/2431914202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ac0b0fb0 0x7fe4ac06b140 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 -- 172.21.15.66:0/2431914202 >> 172.21.15.66:0/2431914202 conn(0x7fe4ac0b2130 msgr2=0x7fe4ac0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:41.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 -- 172.21.15.66:0/2431914202 shutdown_connections 2024-03-30T00:48:41.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 -- 172.21.15.66:0/2431914202 wait complete. 2024-03-30T00:48:41.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 Processor -- start 2024-03-30T00:48:41.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 -- start start 2024-03-30T00:48:41.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4ac0b0fb0 0x7fe4ac15cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:41.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0b1420 0x7fe4ac15ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ac0cca40 0x7fe4ac155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac15bf20 con 0x7fe4ac0cca40 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac15c1c0 con 0x7fe4ac0b1420 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.493+0000 7fe4b4600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac15c4a0 con 0x7fe4ac0b0fb0 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b2600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ac0cca40 0x7fe4ac155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b2600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ac0cca40 0x7fe4ac155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35740/0 (socket says 172.21.15.66:35740) 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b2600640 1 -- 172.21.15.66:0/1854673973 learned_addr learned my addr 172.21.15.66:0/1854673973 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b1c00640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4ac0b0fb0 0x7fe4ac15cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b1200640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0b1420 0x7fe4ac15ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b2600640 1 -- 172.21.15.66:0/1854673973 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4ac0b0fb0 msgr2=0x7fe4ac15cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b2600640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4ac0b0fb0 0x7fe4ac15cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b2600640 1 -- 172.21.15.66:0/1854673973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0b1420 msgr2=0x7fe4ac15ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b2600640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0b1420 0x7fe4ac15ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b2600640 1 -- 172.21.15.66:0/1854673973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe498009420 con 0x7fe4ac0cca40 2024-03-30T00:48:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b1c00640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4ac0b0fb0 0x7fe4ac15cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b1200640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0b1420 0x7fe4ac15ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b2600640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ac0cca40 0x7fe4ac155b40 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c780 tx=0x7fe4a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4a9600640 1 -- 172.21.15.66:0/1854673973 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4004b70 con 0x7fe4ac0cca40 2024-03-30T00:48:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4a9600640 1 -- 172.21.15.66:0/1854673973 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a400ce90 con 0x7fe4ac0cca40 2024-03-30T00:48:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b4600640 1 -- 172.21.15.66:0/1854673973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac15c780 con 0x7fe4ac0cca40 2024-03-30T00:48:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4a9600640 1 -- 172.21.15.66:0/1854673973 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a400a850 con 0x7fe4ac0cca40 2024-03-30T00:48:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b4600640 1 -- 172.21.15.66:0/1854673973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac158bb0 con 0x7fe4ac0cca40 2024-03-30T00:48:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4b4600640 1 -- 172.21.15.66:0/1854673973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac0677d0 con 0x7fe4ac0cca40 2024-03-30T00:48:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.497+0000 7fe4a9600640 1 -- 172.21.15.66:0/1854673973 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe4a400a9b0 con 0x7fe4ac0cca40 2024-03-30T00:48:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.501+0000 7fe4a9600640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe48c066690 0x7fe48c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.501+0000 7fe4a9600640 1 -- 172.21.15.66:0/1854673973 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fe4a40842f0 con 0x7fe4ac0cca40 2024-03-30T00:48:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.501+0000 7fe4b1c00640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe48c066690 0x7fe48c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:41.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.501+0000 7fe4b1c00640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe48c066690 0x7fe48c068b50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0000f70 tx=0x7fe4a0007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:41.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.505+0000 7fe4a9600640 1 -- 172.21.15.66:0/1854673973 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe4a4051760 con 0x7fe4ac0cca40 2024-03-30T00:48:41.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.697+0000 7fe4b4600640 1 -- 172.21.15.66:0/1854673973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4ac15c9c0 con 0x7fe4ac0cca40 2024-03-30T00:48:41.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.701+0000 7fe4a9600640 1 -- 172.21.15.66:0/1854673973 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fe4a405b030 con 0x7fe4ac0cca40 2024-03-30T00:48:41.712 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:41.712 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3965268788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3965268788},{"type":"v1","addr":"172.21.15.66:6837","nonce":3965268788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/211473838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":211473838},{"type":"v1","addr":"172.21.15.156:6833","nonce":211473838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":6557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1778248869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1778248869},{"type":"v1","addr":"172.21.15.156:6835","nonce":1778248869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":8556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1083812760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1083812760},{"type":"v1","addr":"172.21.15.66:6836","nonce":1083812760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[]} 2024-03-30T00:48:41.712 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 73 2024-03-30T00:48:41.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 -- 172.21.15.66:0/1854673973 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe48c066690 msgr2=0x7fe48c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe48c066690 0x7fe48c068b50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0000f70 tx=0x7fe4a0007cd0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 -- 172.21.15.66:0/1854673973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ac0cca40 msgr2=0x7fe4ac155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ac0cca40 0x7fe4ac155b40 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c780 tx=0x7fe4a400cc40 comp rx=0 tx=0).stop 2024-03-30T00:48:41.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 -- 172.21.15.66:0/1854673973 shutdown_connections 2024-03-30T00:48:41.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe48c066690 0x7fe48c068b50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4ac0cca40 0x7fe4ac155b40 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0b1420 0x7fe4ac15ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 --2- 172.21.15.66:0/1854673973 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4ac0b0fb0 0x7fe4ac15cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 -- 172.21.15.66:0/1854673973 >> 172.21.15.66:0/1854673973 conn(0x7fe4ac0b2130 msgr2=0x7fe4ac099250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:41.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 -- 172.21.15.66:0/1854673973 shutdown_connections 2024-03-30T00:48:41.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.705+0000 7fe4b4600640 1 -- 172.21.15.66:0/1854673973 wait complete. 2024-03-30T00:48:41.736 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T00:48:41.736 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-30T00:48:41.737 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T00:48:41.737 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:48:41.737 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T00:48:41.737 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-30T00:48:41.737 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T00:48:41.737 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:48:41.738 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T00:48:41.738 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-30T00:48:41.738 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T00:48:41.738 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:48:41.738 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T00:48:41.738 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-30T00:48:41.738 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T00:48:41.739 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:48:41.748 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T00:48:41.748 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T00:48:41.749 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T00:48:41.750 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T00:48:41.751 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T00:48:41.752 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T00:48:41.752 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T00:48:41.752 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T00:48:41.799 INFO:tasks.ceph.mds.a:Started 2024-03-30T00:48:41.799 DEBUG:teuthology.parallel:result is None 2024-03-30T00:48:41.800 INFO:tasks.ceph.mds.b:Started 2024-03-30T00:48:41.800 DEBUG:teuthology.parallel:result is None 2024-03-30T00:48:41.801 INFO:tasks.ceph.mds.c:Started 2024-03-30T00:48:41.801 DEBUG:teuthology.parallel:result is None 2024-03-30T00:48:41.802 INFO:tasks.ceph.mds.d:Started 2024-03-30T00:48:41.802 DEBUG:teuthology.parallel:result is None 2024-03-30T00:48:41.802 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-30T00:48:41.886 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:48:41.886 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:48:41.888 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2024-03-30T00:48:41.888 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2024-03-30T00:48:41.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.941+0000 7f3ae5e00640 1 Processor -- start 2024-03-30T00:48:41.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.941+0000 7f3ae5e00640 1 -- start start 2024-03-30T00:48:41.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3ae5e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ae015de30 0x7f3ae00cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:41.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3ae5e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae00cbb90 con 0x7f3ae00cee50 2024-03-30T00:48:41.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3ae5e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae00ce1b0 con 0x7f3ae00af420 2024-03-30T00:48:41.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3ae5e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae00afa10 con 0x7f3ae015de30 2024-03-30T00:48:41.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3adea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ae015de30 0x7f3ae00cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:41.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3adea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3ae015de30 0x7f3ae00cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35764/0 (socket says 172.21.15.66:35764) 2024-03-30T00:48:41.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3adea00640 1 -- 172.21.15.66:0/3583100915 learned_addr learned my addr 172.21.15.66:0/3583100915 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:41.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3adea00640 1 -- 172.21.15.66:0/3583100915 >> v1:172.21.15.156:6789/0 conn(0x7f3ae00af420 legacy=0x7f3ae00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3adea00640 1 -- 172.21.15.66:0/3583100915 >> v1:172.21.15.66:6789/0 conn(0x7f3ae00cee50 legacy=0x7f3ae006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3adea00640 1 -- 172.21.15.66:0/3583100915 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae00cf430 con 0x7f3ae015de30 2024-03-30T00:48:41.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3ade000640 1 -- 172.21.15.66:0/3583100915 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287134533 0 0) 0x7f3ae00ce1b0 con 0x7f3ae00af420 2024-03-30T00:48:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.945+0000 7f3adea00640 1 --2- 172.21.15.66:0/3583100915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae015de30 0x7f3ae00cc040 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009e50 tx=0x7f3ad001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cb8a71beafba95 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ade000640 1 -- 172.21.15.66:0/3583100915 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00a5770 con 0x7f3ae015de30 2024-03-30T00:48:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ade000640 1 -- 172.21.15.66:0/3583100915 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad00a58d0 con 0x7f3ae015de30 2024-03-30T00:48:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ade000640 1 -- 172.21.15.66:0/3583100915 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00a5c60 con 0x7f3ae015de30 2024-03-30T00:48:41.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 -- 172.21.15.66:0/3583100915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae015de30 msgr2=0x7f3ae00cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 --2- 172.21.15.66:0/3583100915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae015de30 0x7f3ae00cc040 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009e50 tx=0x7f3ad001c9a0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 -- 172.21.15.66:0/3583100915 shutdown_connections 2024-03-30T00:48:41.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 --2- 172.21.15.66:0/3583100915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae015de30 0x7f3ae00cc040 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 -- 172.21.15.66:0/3583100915 >> 172.21.15.66:0/3583100915 conn(0x7f3ae00d0540 msgr2=0x7f3ae00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:41.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 -- 172.21.15.66:0/3583100915 shutdown_connections 2024-03-30T00:48:41.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 -- 172.21.15.66:0/3583100915 wait complete. 2024-03-30T00:48:41.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 Processor -- start 2024-03-30T00:48:41.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 -- start start 2024-03-30T00:48:41.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ae00af420 0x7f3ae01f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:41.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ae00cee50 0x7f3ae00c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:41.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae015de30 0x7f3ae006afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:41.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae01f66d0 con 0x7f3ae015de30 2024-03-30T00:48:41.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae01574e0 con 0x7f3ae00af420 2024-03-30T00:48:41.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ae5e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae01577c0 con 0x7f3ae00cee50 2024-03-30T00:48:41.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3adf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae015de30 0x7f3ae006afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:41.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3adf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae015de30 0x7f3ae006afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35770/0 (socket says 172.21.15.66:35770) 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3adf400640 1 -- 172.21.15.66:0/625026820 learned_addr learned my addr 172.21.15.66:0/625026820 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3adea00640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ae00af420 0x7f3ae01f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.949+0000 7f3ad6000640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ae00cee50 0x7f3ae00c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3adea00640 1 -- 172.21.15.66:0/625026820 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ae00cee50 msgr2=0x7f3ae00c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3adea00640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ae00cee50 0x7f3ae00c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3adea00640 1 -- 172.21.15.66:0/625026820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae015de30 msgr2=0x7f3ae006afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3adea00640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae015de30 0x7f3ae006afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3adea00640 1 -- 172.21.15.66:0/625026820 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0009b30 con 0x7f3ae00af420 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3adf400640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae015de30 0x7f3ae006afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3adea00640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ae00af420 0x7f3ae01f6190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00ab000 tx=0x7f3ad0019740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3ad7e00640 1 -- 172.21.15.66:0/625026820 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00b8670 con 0x7f3ae00af420 2024-03-30T00:48:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3ad7e00640 1 -- 172.21.15.66:0/625026820 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad00b87d0 con 0x7f3ae00af420 2024-03-30T00:48:41.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3ae5e00640 1 -- 172.21.15.66:0/625026820 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae0157aa0 con 0x7f3ae00af420 2024-03-30T00:48:41.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3ad7e00640 1 -- 172.21.15.66:0/625026820 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00b8b40 con 0x7f3ae00af420 2024-03-30T00:48:41.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3ae5e00640 1 -- 172.21.15.66:0/625026820 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae0158060 con 0x7f3ae00af420 2024-03-30T00:48:41.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3ae5e00640 1 -- 172.21.15.66:0/625026820 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae00cdf60 con 0x7f3ae00af420 2024-03-30T00:48:41.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3ad7e00640 1 -- 172.21.15.66:0/625026820 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3ad00b8ca0 con 0x7f3ae00af420 2024-03-30T00:48:41.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.953+0000 7f3ad7e00640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3abc066690 0x7f3abc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:41.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.957+0000 7f3ad7e00640 1 -- 172.21.15.66:0/625026820 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f3ad011f550 con 0x7f3ae00af420 2024-03-30T00:48:41.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.957+0000 7f3ad6000640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3abc066690 0x7f3abc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:41.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.957+0000 7f3ad6000640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3abc066690 0x7f3abc068b50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00c6ce0 tx=0x7f3ac8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:41.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:41.957+0000 7f3ad7e00640 1 -- 172.21.15.66:0/625026820 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3ad00ecb90 con 0x7f3ae00af420 2024-03-30T00:48:42.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.133+0000 7f3ae5e00640 1 -- 172.21.15.66:0/625026820 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3ae0066c30 con 0x7f3ae00af420 2024-03-30T00:48:42.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.917+0000 7f3ad7e00640 1 -- 172.21.15.66:0/625026820 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v106) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3ad00f01b0 con 0x7f3ae00af420 2024-03-30T00:48:42.929 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-03-30T00:48:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 -- 172.21.15.66:0/625026820 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3abc066690 msgr2=0x7f3abc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3abc066690 0x7f3abc068b50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00c6ce0 tx=0x7f3ac8009040 comp rx=0 tx=0).stop 2024-03-30T00:48:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 -- 172.21.15.66:0/625026820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ae00af420 msgr2=0x7f3ae01f6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ae00af420 0x7f3ae01f6190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00ab000 tx=0x7f3ad0019740 comp rx=0 tx=0).stop 2024-03-30T00:48:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 -- 172.21.15.66:0/625026820 shutdown_connections 2024-03-30T00:48:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3abc066690 0x7f3abc068b50 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae015de30 0x7f3ae006afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ae00cee50 0x7f3ae00c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 --2- 172.21.15.66:0/625026820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ae00af420 0x7f3ae01f6190 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 -- 172.21.15.66:0/625026820 >> 172.21.15.66:0/625026820 conn(0x7f3ae00d0540 msgr2=0x7f3ae00ae360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 -- 172.21.15.66:0/625026820 shutdown_connections 2024-03-30T00:48:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:42.921+0000 7f3ae5e00640 1 -- 172.21.15.66:0/625026820 wait complete. 2024-03-30T00:48:42.953 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-30T00:48:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.089+0000 7f6c47e00640 1 Processor -- start 2024-03-30T00:48:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.089+0000 7f6c47e00640 1 -- start start 2024-03-30T00:48:43.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c47e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6c400cec30 0x7f6c40159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c47e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c400cde70 0x7f6c4015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c47e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4015a6e0 con 0x7f6c400cda90 2024-03-30T00:48:43.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c47e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c400b1b80 con 0x7f6c400cec30 2024-03-30T00:48:43.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c47e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400cd3e0 con 0x7f6c400cde70 2024-03-30T00:48:43.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c45400640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f6c400cda90 0x7f6c400ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:56712/0 (socket says 172.21.15.66:56712) 2024-03-30T00:48:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c45400640 1 -- 172.21.15.66:0/2393930538 learned_addr learned my addr 172.21.15.66:0/2393930538 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c44a00640 1 --2- 172.21.15.66:0/2393930538 >> v2:172.21.15.156:3300/0 conn(0x7f6c400cec30 0x7f6c40159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c45e00640 1 --2- 172.21.15.66:0/2393930538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c400cde70 0x7f6c4015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c3fe00640 1 -- 172.21.15.66:0/2393930538 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783171240 0 0) 0x7f6c4015a6e0 con 0x7f6c400cda90 2024-03-30T00:48:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c3fe00640 1 -- 172.21.15.66:0/2393930538 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c28003530 con 0x7f6c400cda90 2024-03-30T00:48:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c45e00640 1 -- 172.21.15.66:0/2393930538 >> v2:172.21.15.156:3300/0 conn(0x7f6c400cec30 msgr2=0x7f6c40159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c45e00640 1 --2- 172.21.15.66:0/2393930538 >> v2:172.21.15.156:3300/0 conn(0x7f6c400cec30 0x7f6c40159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c45e00640 1 -- 172.21.15.66:0/2393930538 >> v1:172.21.15.156:6789/0 conn(0x7f6c400cda90 legacy=0x7f6c400ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.093+0000 7f6c45e00640 1 -- 172.21.15.66:0/2393930538 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4015a990 con 0x7f6c400cde70 2024-03-30T00:48:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c45e00640 1 --2- 172.21.15.66:0/2393930538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c400cde70 0x7f6c4015a1a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009d90 tx=0x7f6c3801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb5e8b200a896b3a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c3fe00640 1 -- 172.21.15.66:0/2393930538 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c380a59a0 con 0x7f6c400cde70 2024-03-30T00:48:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c3fe00640 1 -- 172.21.15.66:0/2393930538 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c380a5b00 con 0x7f6c400cde70 2024-03-30T00:48:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c3fe00640 1 -- 172.21.15.66:0/2393930538 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c380a5e90 con 0x7f6c400cde70 2024-03-30T00:48:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 -- 172.21.15.66:0/2393930538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c400cde70 msgr2=0x7f6c4015a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 --2- 172.21.15.66:0/2393930538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c400cde70 0x7f6c4015a1a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009d90 tx=0x7f6c3801c8e0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 -- 172.21.15.66:0/2393930538 shutdown_connections 2024-03-30T00:48:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 --2- 172.21.15.66:0/2393930538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c400cde70 0x7f6c4015a1a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 --2- 172.21.15.66:0/2393930538 >> v2:172.21.15.156:3300/0 conn(0x7f6c400cec30 0x7f6c40159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 -- 172.21.15.66:0/2393930538 >> 172.21.15.66:0/2393930538 conn(0x7f6c40069f60 msgr2=0x7f6c4006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 -- 172.21.15.66:0/2393930538 shutdown_connections 2024-03-30T00:48:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 -- 172.21.15.66:0/2393930538 wait complete. 2024-03-30T00:48:43.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 Processor -- start 2024-03-30T00:48:43.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 -- start start 2024-03-30T00:48:43.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c400cda90 0x7f6c400717e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c400cde70 0x7f6c40163c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c400cec30 0x7f6c4015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c40071d20 con 0x7f6c400cda90 2024-03-30T00:48:43.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c40148860 con 0x7f6c400cec30 2024-03-30T00:48:43.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.097+0000 7f6c47e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c40162ab0 con 0x7f6c400cde70 2024-03-30T00:48:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c45400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c400cda90 0x7f6c400717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c45400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c400cda90 0x7f6c400717e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35780/0 (socket says 172.21.15.66:35780) 2024-03-30T00:48:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c45400640 1 -- 172.21.15.66:0/2552889862 learned_addr learned my addr 172.21.15.66:0/2552889862 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c45e00640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c400cec30 0x7f6c4015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c44a00640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c400cde70 0x7f6c40163c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c45400640 1 -- 172.21.15.66:0/2552889862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c400cde70 msgr2=0x7f6c40163c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c45400640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c400cde70 0x7f6c40163c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c45400640 1 -- 172.21.15.66:0/2552889862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c400cec30 msgr2=0x7f6c4015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c45400640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c400cec30 0x7f6c4015d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c45400640 1 -- 172.21.15.66:0/2552889862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c38009a70 con 0x7f6c400cda90 2024-03-30T00:48:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c45400640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c400cda90 0x7f6c400717e0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c4d0 tx=0x7f6c3400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c3d600640 1 -- 172.21.15.66:0/2552889862 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c34005cf0 con 0x7f6c400cda90 2024-03-30T00:48:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c3d600640 1 -- 172.21.15.66:0/2552889862 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c34005e50 con 0x7f6c400cda90 2024-03-30T00:48:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c3d600640 1 -- 172.21.15.66:0/2552889862 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c34007270 con 0x7f6c400cda90 2024-03-30T00:48:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c47e00640 1 -- 172.21.15.66:0/2552889862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c40162d90 con 0x7f6c400cda90 2024-03-30T00:48:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c47e00640 1 -- 172.21.15.66:0/2552889862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c40163350 con 0x7f6c400cda90 2024-03-30T00:48:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c47e00640 1 -- 172.21.15.66:0/2552889862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c40066c30 con 0x7f6c400cda90 2024-03-30T00:48:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c3d600640 1 -- 172.21.15.66:0/2552889862 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6c340073d0 con 0x7f6c400cda90 2024-03-30T00:48:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.101+0000 7f6c3d600640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c20066690 0x7f6c20068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.105+0000 7f6c3d600640 1 -- 172.21.15.66:0/2552889862 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6c340834a0 con 0x7f6c400cda90 2024-03-30T00:48:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.105+0000 7f6c44a00640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c20066690 0x7f6c20068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.105+0000 7f6c44a00640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c20066690 0x7f6c20068b50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6c40163710 tx=0x7f6c2c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.105+0000 7f6c3d600640 1 -- 172.21.15.66:0/2552889862 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6c34050ab0 con 0x7f6c400cda90 2024-03-30T00:48:43.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 -- 172.21.15.66:0/2552889862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6c400b3170 con 0x7f6c400cda90 2024-03-30T00:48:43.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c3d600640 1 -- 172.21.15.66:0/2552889862 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6c340540d0 con 0x7f6c400cda90 2024-03-30T00:48:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm857MDRAAu2pldF20deZfoSDDXuTF/A==", 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMmNFGhAAlHS6ZpIoJc7tlYazCISiLw==", 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmuiBsFRAA2kez6ylAylKTWsQ1S2cahA==", 2024-03-30T00:48:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm/oV3IRAAWn/zG1cLK+Raa2pYPzHd7Q==", 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:48:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAbXwdmf3HtOhAAyRcyb0YUJcJca5fjmZ3Frw==", 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-03-30T00:48:43.339 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAfXwdml7cdBRAAs639bBHTPyfhfVZMPRMfYA==", 2024-03-30T00:48:43.340 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.340 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:48:43.340 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:48:43.340 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.340 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAiXwdmeS6KChAAZjLAbKSL7Vswi3qQ4304bA==", 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAlXwdmeuigEBAAiPIPheowKBnn3UZWwpdr+Q==", 2024-03-30T00:48:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQApXwdm+bQNHRAAAb/lufdb7Fe+EFevd23gtA==", 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAsXwdmReL1IhAAwCs1cS+P6J8VCOYdrnrPQg==", 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAvXwdmA15vKBAAmpnKrDM/Y0xuBGxdyjrkWA==", 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAyXwdmrl02LhAA6JfEnugoz6l4nsXl6/vq0Q==", 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-03-30T00:48:43.345 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmZjh7JRAA3EFOImM03iOljcwmA7/xWw==", 2024-03-30T00:48:43.345 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.345 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:48:43.345 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-03-30T00:48:43.345 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-30T00:48:43.345 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.345 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmkZv0LhAAgwCqW25+J1VYO31mWwDtLQ==", 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmLG1/KxAAmNIZ9ZXdqPRQjCXvz7wC9A==", 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:48:43.346 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmyJ/nNBAAIgiUoUCbTeLEjL1iaPFw1A==", 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.347 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAXXwdmwWxQLRAAoFyZnFsIo1NvQlzRN+UHmw==", 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmEF+xCRAAqRVFguki881+4azP8lsf0g==", 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:48:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMg32BRAAUWBF+QbwRGsopcAoBay8wA==", 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-03-30T00:48:43.349 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:48:43.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 -- 172.21.15.66:0/2552889862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c20066690 msgr2=0x7f6c20068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c20066690 0x7f6c20068b50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6c40163710 tx=0x7f6c2c009190 comp rx=0 tx=0).stop 2024-03-30T00:48:43.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 -- 172.21.15.66:0/2552889862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c400cda90 msgr2=0x7f6c400717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c400cda90 0x7f6c400717e0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c4d0 tx=0x7f6c3400c990 comp rx=0 tx=0).stop 2024-03-30T00:48:43.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 -- 172.21.15.66:0/2552889862 shutdown_connections 2024-03-30T00:48:43.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c400cec30 0x7f6c4015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c400cde70 0x7f6c40163c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c20066690 0x7f6c20068b50 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 --2- 172.21.15.66:0/2552889862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c400cda90 0x7f6c400717e0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 -- 172.21.15.66:0/2552889862 >> 172.21.15.66:0/2552889862 conn(0x7f6c40069f60 msgr2=0x7f6c40164a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:43.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 -- 172.21.15.66:0/2552889862 shutdown_connections 2024-03-30T00:48:43.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.325+0000 7f6c47e00640 1 -- 172.21.15.66:0/2552889862 wait complete. 2024-03-30T00:48:43.356 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:48:43.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.485+0000 7fe61fa00640 1 Processor -- start 2024-03-30T00:48:43.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.485+0000 7fe61fa00640 1 -- start start 2024-03-30T00:48:43.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61fa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe61815ca80 0x7fe61806b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61fa00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe6180cb570 0x7fe6180cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61fa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6180cb0e0 con 0x7fe6180cf2f0 2024-03-30T00:48:43.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61fa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6180cf7f0 con 0x7fe61815ca80 2024-03-30T00:48:43.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61fa00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6180cfad0 con 0x7fe6180cb570 2024-03-30T00:48:43.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61d000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe61815ca80 0x7fe61806b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61d000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe61815ca80 0x7fe61806b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35788/0 (socket says 172.21.15.66:35788) 2024-03-30T00:48:43.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61d000640 1 -- 172.21.15.66:0/1792783333 learned_addr learned my addr 172.21.15.66:0/1792783333 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:43.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61da00640 1 --2- 172.21.15.66:0/1792783333 >> v2:172.21.15.156:3300/0 conn(0x7fe6180cb570 0x7fe6180cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61d000640 1 -- 172.21.15.66:0/1792783333 >> v2:172.21.15.156:3300/0 conn(0x7fe6180cb570 msgr2=0x7fe6180cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:43.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61d000640 1 --2- 172.21.15.66:0/1792783333 >> v2:172.21.15.156:3300/0 conn(0x7fe6180cb570 0x7fe6180cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61d000640 1 -- 172.21.15.66:0/1792783333 >> v1:172.21.15.66:6789/0 conn(0x7fe6180cf2f0 legacy=0x7fe61806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.489+0000 7fe61d000640 1 -- 172.21.15.66:0/1792783333 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6180cad70 con 0x7fe61815ca80 2024-03-30T00:48:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61d000640 1 --2- 172.21.15.66:0/1792783333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe61815ca80 0x7fe61806b0b0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe618071d80 tx=0x7fe60801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f3c8cfb03af104 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe617e00640 1 -- 172.21.15.66:0/1792783333 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080a8a10 con 0x7fe61815ca80 2024-03-30T00:48:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe617e00640 1 -- 172.21.15.66:0/1792783333 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6080a8b70 con 0x7fe61815ca80 2024-03-30T00:48:43.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe617e00640 1 -- 172.21.15.66:0/1792783333 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080b0440 con 0x7fe61815ca80 2024-03-30T00:48:43.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 -- 172.21.15.66:0/1792783333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe61815ca80 msgr2=0x7fe61806b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 --2- 172.21.15.66:0/1792783333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe61815ca80 0x7fe61806b0b0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe618071d80 tx=0x7fe60801ed10 comp rx=0 tx=0).stop 2024-03-30T00:48:43.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 -- 172.21.15.66:0/1792783333 shutdown_connections 2024-03-30T00:48:43.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 --2- 172.21.15.66:0/1792783333 >> v2:172.21.15.156:3300/0 conn(0x7fe6180cb570 0x7fe6180cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 --2- 172.21.15.66:0/1792783333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe61815ca80 0x7fe61806b0b0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 -- 172.21.15.66:0/1792783333 >> 172.21.15.66:0/1792783333 conn(0x7fe6180b2090 msgr2=0x7fe6180b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:43.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 -- 172.21.15.66:0/1792783333 shutdown_connections 2024-03-30T00:48:43.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 -- 172.21.15.66:0/1792783333 wait complete. 2024-03-30T00:48:43.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 Processor -- start 2024-03-30T00:48:43.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 -- start start 2024-03-30T00:48:43.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6180cb570 0x7fe6180c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6180cf2f0 0x7fe6181528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe61815ca80 0x7fe6180ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180c7710 con 0x7fe61815ca80 2024-03-30T00:48:43.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180c79f0 con 0x7fe6180cf2f0 2024-03-30T00:48:43.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.493+0000 7fe61fa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180c7cd0 con 0x7fe6180cb570 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe61815ca80 0x7fe6180ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe61815ca80 0x7fe6180ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35802/0 (socket says 172.21.15.66:35802) 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61da00640 1 -- 172.21.15.66:0/958078142 learned_addr learned my addr 172.21.15.66:0/958078142 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe60fe00640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6180cf2f0 0x7fe6181528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61d000640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6180cb570 0x7fe6180c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61da00640 1 -- 172.21.15.66:0/958078142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6180cb570 msgr2=0x7fe6180c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61da00640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6180cb570 0x7fe6180c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61da00640 1 -- 172.21.15.66:0/958078142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6180cf2f0 msgr2=0x7fe6181528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61da00640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6180cf2f0 0x7fe6181528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61da00640 1 -- 172.21.15.66:0/958078142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6080094d0 con 0x7fe61815ca80 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61da00640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe61815ca80 0x7fe6180ccc10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fe61000cd70 tx=0x7fe61000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe615600640 1 -- 172.21.15.66:0/958078142 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610003ba0 con 0x7fe61815ca80 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe615600640 1 -- 172.21.15.66:0/958078142 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe610003d00 con 0x7fe61815ca80 2024-03-30T00:48:43.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe615600640 1 -- 172.21.15.66:0/958078142 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610004240 con 0x7fe61815ca80 2024-03-30T00:48:43.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61fa00640 1 -- 172.21.15.66:0/958078142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6180c7fb0 con 0x7fe61815ca80 2024-03-30T00:48:43.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61fa00640 1 -- 172.21.15.66:0/958078142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe618152df0 con 0x7fe61815ca80 2024-03-30T00:48:43.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe61fa00640 1 -- 172.21.15.66:0/958078142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe618066c30 con 0x7fe61815ca80 2024-03-30T00:48:43.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.497+0000 7fe615600640 1 -- 172.21.15.66:0/958078142 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe6100043a0 con 0x7fe61815ca80 2024-03-30T00:48:43.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.501+0000 7fe615600640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 0x7fe5f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.501+0000 7fe615600640 1 -- 172.21.15.66:0/958078142 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe610012070 con 0x7fe61815ca80 2024-03-30T00:48:43.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.501+0000 7fe61d000640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 0x7fe5f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.501+0000 7fe61d000640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 0x7fe5f4068b50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe6080b2000 tx=0x7fe6080aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:43.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.501+0000 7fe615600640 1 -- 172.21.15.66:0/958078142 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe610051e90 con 0x7fe61815ca80 2024-03-30T00:48:43.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.661+0000 7fe61fa00640 1 -- 172.21.15.66:0/958078142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6180c8460 con 0x7fe61815ca80 2024-03-30T00:48:43.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe615600640 1 -- 172.21.15.66:0/958078142 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+12596 (secure 0 0 0) 0x7fe6100554b0 con 0x7fe61815ca80 2024-03-30T00:48:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:43.674 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":106,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:48:42.916316+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:48:43.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 -- 172.21.15.66:0/958078142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 msgr2=0x7fe5f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 0x7fe5f4068b50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe6080b2000 tx=0x7fe6080aa070 comp rx=0 tx=0).stop 2024-03-30T00:48:43.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 -- 172.21.15.66:0/958078142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe61815ca80 msgr2=0x7fe6180ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe61815ca80 0x7fe6180ccc10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fe61000cd70 tx=0x7fe61000a660 comp rx=0 tx=0).stop 2024-03-30T00:48:43.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 -- 172.21.15.66:0/958078142 shutdown_connections 2024-03-30T00:48:43.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 0x7fe5f4068b50 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe61815ca80 0x7fe6180ccc10 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6180cf2f0 0x7fe6181528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 --2- 172.21.15.66:0/958078142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6180cb570 0x7fe6180c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 -- 172.21.15.66:0/958078142 >> 172.21.15.66:0/958078142 conn(0x7fe6180b2090 msgr2=0x7fe61806ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:43.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 -- 172.21.15.66:0/958078142 shutdown_connections 2024-03-30T00:48:43.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.665+0000 7fe61fa00640 1 -- 172.21.15.66:0/958078142 wait complete. 2024-03-30T00:48:43.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:48:43.696 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:48:43.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf09600640 1 Processor -- start 2024-03-30T00:48:43.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf09600640 1 -- start start 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf09600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf040cda90 0x7fdf040ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf09600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdf0415a370 0x7fdf040cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf09600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0415a990 con 0x7fdf040cde70 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf09600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf040b1b80 con 0x7fdf0415a370 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf09600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040cd3e0 con 0x7fdf040cda90 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf03400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdf0415a370 0x7fdf040cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf03400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdf0415a370 0x7fdf040cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35816/0 (socket says 172.21.15.66:35816) 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf03400640 1 -- 172.21.15.66:0/1999662617 learned_addr learned my addr 172.21.15.66:0/1999662617 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf02a00640 1 --2- 172.21.15.66:0/1999662617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf040cda90 0x7fdf040ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf03400640 1 -- 172.21.15.66:0/1999662617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf040cda90 msgr2=0x7fdf040ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf03400640 1 --2- 172.21.15.66:0/1999662617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf040cda90 0x7fdf040ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf03400640 1 -- 172.21.15.66:0/1999662617 >> v1:172.21.15.156:6789/0 conn(0x7fdf040cde70 legacy=0x7fdf04159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.829+0000 7fdf03400640 1 -- 172.21.15.66:0/1999662617 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040b11e0 con 0x7fdf0415a370 2024-03-30T00:48:43.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf03400640 1 --2- 172.21.15.66:0/1999662617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0415a370 0x7fdf040cea60 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009e50 tx=0x7fdefc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0557651544f4b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:43.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf02000640 1 -- 172.21.15.66:0/1999662617 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0a5770 con 0x7fdf0415a370 2024-03-30T00:48:43.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf02000640 1 -- 172.21.15.66:0/1999662617 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc0a58d0 con 0x7fdf0415a370 2024-03-30T00:48:43.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf02000640 1 -- 172.21.15.66:0/1999662617 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0a5c60 con 0x7fdf0415a370 2024-03-30T00:48:43.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf09600640 1 -- 172.21.15.66:0/1999662617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0415a370 msgr2=0x7fdf040cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf09600640 1 --2- 172.21.15.66:0/1999662617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0415a370 0x7fdf040cea60 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009e50 tx=0x7fdefc01c9a0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf09600640 1 -- 172.21.15.66:0/1999662617 shutdown_connections 2024-03-30T00:48:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf09600640 1 --2- 172.21.15.66:0/1999662617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0415a370 0x7fdf040cea60 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf09600640 1 --2- 172.21.15.66:0/1999662617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf040cda90 0x7fdf040ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf09600640 1 -- 172.21.15.66:0/1999662617 >> 172.21.15.66:0/1999662617 conn(0x7fdf04069f60 msgr2=0x7fdf0406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf09600640 1 -- 172.21.15.66:0/1999662617 shutdown_connections 2024-03-30T00:48:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.833+0000 7fdf09600640 1 -- 172.21.15.66:0/1999662617 wait complete. 2024-03-30T00:48:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf09600640 1 Processor -- start 2024-03-30T00:48:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf09600640 1 -- start start 2024-03-30T00:48:43.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf09600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf040cda90 0x7fdf040add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf09600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf040cde70 0x7fdf04153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf09600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0415a370 0x7fdf040ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf09600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04154150 con 0x7fdf0415a370 2024-03-30T00:48:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf09600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040ae290 con 0x7fdf040cda90 2024-03-30T00:48:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf09600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04153cb0 con 0x7fdf040cde70 2024-03-30T00:48:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf03400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0415a370 0x7fdf040ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf03400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0415a370 0x7fdf040ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35830/0 (socket says 172.21.15.66:35830) 2024-03-30T00:48:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf03400640 1 -- 172.21.15.66:0/904463885 learned_addr learned my addr 172.21.15.66:0/904463885 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdefa000640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf040cde70 0x7fdf04153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf02a00640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf040cda90 0x7fdf040add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf03400640 1 -- 172.21.15.66:0/904463885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf040cde70 msgr2=0x7fdf04153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf03400640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf040cde70 0x7fdf04153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf03400640 1 -- 172.21.15.66:0/904463885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf040cda90 msgr2=0x7fdf040add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:43.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf03400640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf040cda90 0x7fdf040add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:43.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf03400640 1 -- 172.21.15.66:0/904463885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc009b30 con 0x7fdf0415a370 2024-03-30T00:48:43.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf03400640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0415a370 0x7fdf040ab010 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0af040 tx=0x7fdefc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:43.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdefb400640 1 -- 172.21.15.66:0/904463885 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0b8640 con 0x7fdf0415a370 2024-03-30T00:48:43.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdefb400640 1 -- 172.21.15.66:0/904463885 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc0b87c0 con 0x7fdf0415a370 2024-03-30T00:48:43.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf09600640 1 -- 172.21.15.66:0/904463885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf040af090 con 0x7fdf0415a370 2024-03-30T00:48:43.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdefb400640 1 -- 172.21.15.66:0/904463885 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0b8b70 con 0x7fdf0415a370 2024-03-30T00:48:43.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.837+0000 7fdf09600640 1 -- 172.21.15.66:0/904463885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf040ae760 con 0x7fdf0415a370 2024-03-30T00:48:43.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.841+0000 7fdf09600640 1 -- 172.21.15.66:0/904463885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf040af220 con 0x7fdf0415a370 2024-03-30T00:48:43.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.841+0000 7fdefb400640 1 -- 172.21.15.66:0/904463885 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdefc0b8cd0 con 0x7fdf0415a370 2024-03-30T00:48:43.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.841+0000 7fdefb400640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdee0066690 0x7fdee0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:43.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.841+0000 7fdefb400640 1 -- 172.21.15.66:0/904463885 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fdefc11ece0 con 0x7fdf0415a370 2024-03-30T00:48:43.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.841+0000 7fdf02a00640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdee0066690 0x7fdee0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:43.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.841+0000 7fdf02a00640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdee0066690 0x7fdee0068b50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0079c0 tx=0x7fdeec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:43.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:43.845+0000 7fdefb400640 1 -- 172.21.15.66:0/904463885 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdefc0ec300 con 0x7fdf0415a370 2024-03-30T00:48:44.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.009+0000 7fdf09600640 1 -- 172.21.15.66:0/904463885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf04066c30 con 0x7fdf0415a370 2024-03-30T00:48:44.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.009+0000 7fdefb400640 1 -- 172.21.15.66:0/904463885 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdefc0ef920 con 0x7fdf0415a370 2024-03-30T00:48:44.021 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:48:44.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 -- 172.21.15.66:0/904463885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdee0066690 msgr2=0x7fdee0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdee0066690 0x7fdee0068b50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0079c0 tx=0x7fdeec008040 comp rx=0 tx=0).stop 2024-03-30T00:48:44.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 -- 172.21.15.66:0/904463885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0415a370 msgr2=0x7fdf040ab010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0415a370 0x7fdf040ab010 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0af040 tx=0x7fdefc019740 comp rx=0 tx=0).stop 2024-03-30T00:48:44.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 -- 172.21.15.66:0/904463885 shutdown_connections 2024-03-30T00:48:44.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdee0066690 0x7fdee0068b50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0415a370 0x7fdf040ab010 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf040cde70 0x7fdf04153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 --2- 172.21.15.66:0/904463885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf040cda90 0x7fdf040add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 -- 172.21.15.66:0/904463885 >> 172.21.15.66:0/904463885 conn(0x7fdf04069f60 msgr2=0x7fdf0406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:44.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 -- 172.21.15.66:0/904463885 shutdown_connections 2024-03-30T00:48:44.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.013+0000 7fdf09600640 1 -- 172.21.15.66:0/904463885 wait complete. 2024-03-30T00:48:44.043 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T00:48:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe357800640 1 Processor -- start 2024-03-30T00:48:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe357800640 1 -- start start 2024-03-30T00:48:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe357800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe3500cc560 0x7fe3500d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe357800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe35006e800 0x7fe3500ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe357800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35006b0b0 con 0x7fe35006f8e0 2024-03-30T00:48:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe357800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe350160640 con 0x7fe35006e800 2024-03-30T00:48:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe357800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35006b390 con 0x7fe3500cc560 2024-03-30T00:48:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe355800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe35006e800 0x7fe3500ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe355800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe35006e800 0x7fe3500ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35838/0 (socket says 172.21.15.66:35838) 2024-03-30T00:48:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe355800640 1 -- 172.21.15.66:0/2839985862 learned_addr learned my addr 172.21.15.66:0/2839985862 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe354e00640 1 --2- 172.21.15.66:0/2839985862 >> v2:172.21.15.156:3300/0 conn(0x7fe3500cc560 0x7fe3500d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe355800640 1 -- 172.21.15.66:0/2839985862 >> v2:172.21.15.156:3300/0 conn(0x7fe3500cc560 msgr2=0x7fe3500d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:44.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe355800640 1 --2- 172.21.15.66:0/2839985862 >> v2:172.21.15.156:3300/0 conn(0x7fe3500cc560 0x7fe3500d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe355800640 1 -- 172.21.15.66:0/2839985862 >> v1:172.21.15.156:6789/0 conn(0x7fe35006f8e0 legacy=0x7fe35006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe355800640 1 -- 172.21.15.66:0/2839985862 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35006b670 con 0x7fe35006e800 2024-03-30T00:48:44.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.173+0000 7fe354e00640 1 --2- 172.21.15.66:0/2839985862 >> v2:172.21.15.156:3300/0 conn(0x7fe3500cc560 0x7fe3500d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:44.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe355800640 1 --2- 172.21.15.66:0/2839985862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35006e800 0x7fe3500ce8d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe350072c50 tx=0x7fe34801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35124e71b5a4905 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe34f400640 1 -- 172.21.15.66:0/2839985862 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3480a58a0 con 0x7fe35006e800 2024-03-30T00:48:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe34f400640 1 -- 172.21.15.66:0/2839985862 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3480a5a00 con 0x7fe35006e800 2024-03-30T00:48:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe34f400640 1 -- 172.21.15.66:0/2839985862 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3480a5d90 con 0x7fe35006e800 2024-03-30T00:48:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe357800640 1 -- 172.21.15.66:0/2839985862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35006e800 msgr2=0x7fe3500ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe357800640 1 --2- 172.21.15.66:0/2839985862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35006e800 0x7fe3500ce8d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe350072c50 tx=0x7fe34801c9c0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe357800640 1 -- 172.21.15.66:0/2839985862 shutdown_connections 2024-03-30T00:48:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe357800640 1 --2- 172.21.15.66:0/2839985862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35006e800 0x7fe3500ce8d0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe357800640 1 --2- 172.21.15.66:0/2839985862 >> v2:172.21.15.156:3300/0 conn(0x7fe3500cc560 0x7fe3500d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe357800640 1 -- 172.21.15.66:0/2839985862 >> 172.21.15.66:0/2839985862 conn(0x7fe3500d0540 msgr2=0x7fe3500d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe357800640 1 -- 172.21.15.66:0/2839985862 shutdown_connections 2024-03-30T00:48:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.177+0000 7fe357800640 1 -- 172.21.15.66:0/2839985862 wait complete. 2024-03-30T00:48:44.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe357800640 1 Processor -- start 2024-03-30T00:48:44.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe357800640 1 -- start start 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe357800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe35006e800 0x7fe35014a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe357800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35006f8e0 0x7fe35014c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe357800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500cc560 0x7fe35015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe357800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500c7720 con 0x7fe35006f8e0 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe357800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe350152450 con 0x7fe3500cc560 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe357800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe350151e40 con 0x7fe35006e800 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35006f8e0 0x7fe35014c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35006f8e0 0x7fe35014c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35844/0 (socket says 172.21.15.66:35844) 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34fe00640 1 -- 172.21.15.66:0/2616892023 learned_addr learned my addr 172.21.15.66:0/2616892023 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe354e00640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe35006e800 0x7fe35014a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe355800640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500cc560 0x7fe35015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34fe00640 1 -- 172.21.15.66:0/2616892023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe35006e800 msgr2=0x7fe35014a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34fe00640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe35006e800 0x7fe35014a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34fe00640 1 -- 172.21.15.66:0/2616892023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500cc560 msgr2=0x7fe35015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34fe00640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500cc560 0x7fe35015dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34fe00640 1 -- 172.21.15.66:0/2616892023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe348009b30 con 0x7fe35006f8e0 2024-03-30T00:48:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe355800640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500cc560 0x7fe35015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34fe00640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35006f8e0 0x7fe35014c120 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00c480 tx=0x7fe33c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34cc00640 1 -- 172.21.15.66:0/2616892023 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0052d0 con 0x7fe35006f8e0 2024-03-30T00:48:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34cc00640 1 -- 172.21.15.66:0/2616892023 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c005450 con 0x7fe35006f8e0 2024-03-30T00:48:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe357800640 1 -- 172.21.15.66:0/2616892023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35015eff0 con 0x7fe35006f8e0 2024-03-30T00:48:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe34cc00640 1 -- 172.21.15.66:0/2616892023 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c012870 con 0x7fe35006f8e0 2024-03-30T00:48:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe357800640 1 -- 172.21.15.66:0/2616892023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3501520e0 con 0x7fe35006f8e0 2024-03-30T00:48:44.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.181+0000 7fe357800640 1 -- 172.21.15.66:0/2616892023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3501525e0 con 0x7fe35006f8e0 2024-03-30T00:48:44.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.185+0000 7fe34cc00640 1 -- 172.21.15.66:0/2616892023 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe33c0129d0 con 0x7fe35006f8e0 2024-03-30T00:48:44.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.185+0000 7fe34cc00640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe330066690 0x7fe330068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.185+0000 7fe34cc00640 1 -- 172.21.15.66:0/2616892023 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe33c083b50 con 0x7fe35006f8e0 2024-03-30T00:48:44.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.185+0000 7fe354e00640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe330066690 0x7fe330068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.185+0000 7fe354e00640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe330066690 0x7fe330068b50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe3440045f0 tx=0x7fe344008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:44.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.189+0000 7fe34cc00640 1 -- 172.21.15.66:0/2616892023 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe33c051030 con 0x7fe35006f8e0 2024-03-30T00:48:44.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.349+0000 7fe357800640 1 -- 172.21.15.66:0/2616892023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe350066c30 con 0x7fe35006f8e0 2024-03-30T00:48:44.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.349+0000 7fe34cc00640 1 -- 172.21.15.66:0/2616892023 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v106) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe33c054650 con 0x7fe35006f8e0 2024-03-30T00:48:44.360 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:44.360 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:48:44.360 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:48:44.360 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-03-30T00:48:44.360 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-03-30T00:48:44.360 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:44.360 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:48:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.349+0000 7fe357800640 1 -- 172.21.15.66:0/2616892023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe330066690 msgr2=0x7fe330068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.349+0000 7fe357800640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe330066690 0x7fe330068b50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe3440045f0 tx=0x7fe344008040 comp rx=0 tx=0).stop 2024-03-30T00:48:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.349+0000 7fe357800640 1 -- 172.21.15.66:0/2616892023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35006f8e0 msgr2=0x7fe35014c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.349+0000 7fe357800640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35006f8e0 0x7fe35014c120 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00c480 tx=0x7fe33c00c940 comp rx=0 tx=0).stop 2024-03-30T00:48:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.353+0000 7fe357800640 1 -- 172.21.15.66:0/2616892023 shutdown_connections 2024-03-30T00:48:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.353+0000 7fe357800640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500cc560 0x7fe35015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.353+0000 7fe357800640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe330066690 0x7fe330068b50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.353+0000 7fe357800640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35006f8e0 0x7fe35014c120 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.353+0000 7fe357800640 1 --2- 172.21.15.66:0/2616892023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe35006e800 0x7fe35014a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.353+0000 7fe357800640 1 -- 172.21.15.66:0/2616892023 >> 172.21.15.66:0/2616892023 conn(0x7fe3500d0540 msgr2=0x7fe35006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:44.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.353+0000 7fe357800640 1 -- 172.21.15.66:0/2616892023 shutdown_connections 2024-03-30T00:48:44.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.353+0000 7fe357800640 1 -- 172.21.15.66:0/2616892023 wait complete. 2024-03-30T00:48:44.383 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T00:48:44.383 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T00:48:44.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.517+0000 7f6b15c00640 1 Processor -- start 2024-03-30T00:48:44.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.517+0000 7f6b15c00640 1 -- start start 2024-03-30T00:48:44.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b15c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015ba00 0x7f6b100ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b15c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b100cecf0 0x7f6b101579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b15c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b100cbf80 con 0x7f6b100acaf0 2024-03-30T00:48:44.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b15c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b100b2be0 con 0x7f6b100cecf0 2024-03-30T00:48:44.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b15c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100cf180 con 0x7f6b1015ba00 2024-03-30T00:48:44.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0ea00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6b100acaf0 0x7f6b100b24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34670/0 (socket says 172.21.15.66:34670) 2024-03-30T00:48:44.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0ea00640 1 -- 172.21.15.66:0/2805882400 learned_addr learned my addr 172.21.15.66:0/2805882400 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:44.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b06000640 1 --2- 172.21.15.66:0/2805882400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015ba00 0x7f6b100ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932662712 0 0) 0x7f6b100cbf80 con 0x7f6b100acaf0 2024-03-30T00:48:44.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6af4003530 con 0x7f6b100acaf0 2024-03-30T00:48:44.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2848528624 0 0) 0x7f6af4003530 con 0x7f6b100acaf0 2024-03-30T00:48:44.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015ba00 msgr2=0x7f6b100ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 --2- 172.21.15.66:0/2805882400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015ba00 0x7f6b100ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 >> v2:172.21.15.156:3300/0 conn(0x7f6b100cecf0 msgr2=0x7f6b101579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 --2- 172.21.15.66:0/2805882400 >> v2:172.21.15.156:3300/0 conn(0x7f6b100cecf0 0x7f6b101579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100cc820 con 0x7f6b100acaf0 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f6b00001680 con 0x7f6b100acaf0 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 >> v1:172.21.15.66:6789/0 conn(0x7f6b100acaf0 legacy=0x7f6b100b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 --2- 172.21.15.66:0/2805882400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015ba00 0x7f6af4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 --2- 172.21.15.66:0/2805882400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af4003f00 0x7f6af40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 --2- 172.21.15.66:0/2805882400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6af4004d10 0x7f6af4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100cf180 con 0x7f6af4003f00 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af400bb20 con 0x7f6af4004d10 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af400bcb0 con 0x7f6b1015ba00 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b06000640 1 --2- 172.21.15.66:0/2805882400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af4003f00 0x7f6af40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b06000640 1 -- 172.21.15.66:0/2805882400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015ba00 msgr2=0x7f6af4003980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:48:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b06000640 1 --2- 172.21.15.66:0/2805882400 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015ba00 0x7f6af4003980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b06000640 1 -- 172.21.15.66:0/2805882400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6af4004d10 msgr2=0x7f6af4005190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:48:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b06000640 1 --2- 172.21.15.66:0/2805882400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6af4004d10 0x7f6af4005190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.521+0000 7f6b06000640 1 -- 172.21.15.66:0/2805882400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af4003530 con 0x7f6af4003f00 2024-03-30T00:48:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b06000640 1 --2- 172.21.15.66:0/2805882400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af4003f00 0x7f6af40047d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009ae0 tx=0x7f6af801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c0ba2960f970c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af80a6510 con 0x7f6af4003f00 2024-03-30T00:48:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af80a6670 con 0x7f6af4003f00 2024-03-30T00:48:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b0e000640 1 -- 172.21.15.66:0/2805882400 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af80a6a20 con 0x7f6af4003f00 2024-03-30T00:48:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b15c00640 1 -- 172.21.15.66:0/2805882400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af4003f00 msgr2=0x7f6af40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b15c00640 1 --2- 172.21.15.66:0/2805882400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6af4003f00 0x7f6af40047d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009ae0 tx=0x7f6af801ca20 comp rx=0 tx=0).stop 2024-03-30T00:48:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b0f400640 1 -- 172.21.15.66:0/2805882400 reap_dead start 2024-03-30T00:48:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b15c00640 1 -- 172.21.15.66:0/2805882400 shutdown_connections 2024-03-30T00:48:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b15c00640 1 -- 172.21.15.66:0/2805882400 >> 172.21.15.66:0/2805882400 conn(0x7f6b1006a190 msgr2=0x7f6b1006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b15c00640 1 -- 172.21.15.66:0/2805882400 shutdown_connections 2024-03-30T00:48:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b15c00640 1 -- 172.21.15.66:0/2805882400 wait complete. 2024-03-30T00:48:44.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.525+0000 7f6b15c00640 1 Processor -- start 2024-03-30T00:48:44.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b15c00640 1 -- start start 2024-03-30T00:48:44.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b15c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6af4003f00 0x7f6b10157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b15c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015fea0 0x7f6b10160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b15c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b100adec0 0x7f6b100ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b15c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10160860 con 0x7f6b100adec0 2024-03-30T00:48:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b15c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100afc20 con 0x7f6af4003f00 2024-03-30T00:48:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b15c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100aeb80 con 0x7f6b1015fea0 2024-03-30T00:48:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b0f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b100adec0 0x7f6b100ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b0f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b100adec0 0x7f6b100ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35862/0 (socket says 172.21.15.66:35862) 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b0f400640 1 -- 172.21.15.66:0/820687625 learned_addr learned my addr 172.21.15.66:0/820687625 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b0ea00640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6af4003f00 0x7f6b10157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b06000640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015fea0 0x7f6b10160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b0f400640 1 -- 172.21.15.66:0/820687625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015fea0 msgr2=0x7f6b10160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b0f400640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015fea0 0x7f6b10160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b0f400640 1 -- 172.21.15.66:0/820687625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6af4003f00 msgr2=0x7f6b10157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b0f400640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6af4003f00 0x7f6b10157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b0f400640 1 -- 172.21.15.66:0/820687625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100aee60 con 0x7f6b100adec0 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b0ea00640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6af4003f00 0x7f6b10157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b06000640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015fea0 0x7f6b10160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b0f400640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b100adec0 0x7f6b100ae340 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6b100b1600 tx=0x7f6b0800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b07400640 1 -- 172.21.15.66:0/820687625 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800aa20 con 0x7f6b100adec0 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b07400640 1 -- 172.21.15.66:0/820687625 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0800aba0 con 0x7f6b100adec0 2024-03-30T00:48:44.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b15c00640 1 -- 172.21.15.66:0/820687625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b100af140 con 0x7f6b100adec0 2024-03-30T00:48:44.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b07400640 1 -- 172.21.15.66:0/820687625 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08012400 con 0x7f6b100adec0 2024-03-30T00:48:44.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b15c00640 1 -- 172.21.15.66:0/820687625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b100af700 con 0x7f6b100adec0 2024-03-30T00:48:44.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.529+0000 7f6b15c00640 1 -- 172.21.15.66:0/820687625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b10066c30 con 0x7f6b100adec0 2024-03-30T00:48:44.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.533+0000 7f6b07400640 1 -- 172.21.15.66:0/820687625 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6b0801b3b0 con 0x7f6b100adec0 2024-03-30T00:48:44.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.533+0000 7f6b07400640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6aec066690 0x7f6aec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:44.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.533+0000 7f6b07400640 1 -- 172.21.15.66:0/820687625 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6b08083ad0 con 0x7f6b100adec0 2024-03-30T00:48:44.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.533+0000 7f6b0ea00640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6aec066690 0x7f6aec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:44.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.533+0000 7f6b0ea00640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6aec066690 0x7f6aec068b50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6b00001ec0 tx=0x7f6b00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:44.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.537+0000 7f6b07400640 1 -- 172.21.15.66:0/820687625 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6b08051040 con 0x7f6b100adec0 2024-03-30T00:48:44.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.701+0000 7f6b15c00640 1 -- 172.21.15.66:0/820687625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6b100c8cf0 con 0x7f6b100adec0 2024-03-30T00:48:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.921+0000 7f6b07400640 1 -- 172.21.15.66:0/820687625 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v107) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6b080170b0 con 0x7f6b100adec0 2024-03-30T00:48:44.932 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-03-30T00:48:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 -- 172.21.15.66:0/820687625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6aec066690 msgr2=0x7f6aec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6aec066690 0x7f6aec068b50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6b00001ec0 tx=0x7f6b00009040 comp rx=0 tx=0).stop 2024-03-30T00:48:44.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 -- 172.21.15.66:0/820687625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b100adec0 msgr2=0x7f6b100ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:44.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b100adec0 0x7f6b100ae340 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6b100b1600 tx=0x7f6b0800cf00 comp rx=0 tx=0).stop 2024-03-30T00:48:44.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 -- 172.21.15.66:0/820687625 shutdown_connections 2024-03-30T00:48:44.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6aec066690 0x7f6aec068b50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b100adec0 0x7f6b100ae340 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1015fea0 0x7f6b10160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 --2- 172.21.15.66:0/820687625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6af4003f00 0x7f6b10157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:44.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 -- 172.21.15.66:0/820687625 >> 172.21.15.66:0/820687625 conn(0x7f6b1006a190 msgr2=0x7f6b100abdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:44.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 -- 172.21.15.66:0/820687625 shutdown_connections 2024-03-30T00:48:44.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:44.925+0000 7f6b15c00640 1 -- 172.21.15.66:0/820687625 wait complete. 2024-03-30T00:48:44.959 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T00:48:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3ab5000640 1 Processor -- start 2024-03-30T00:48:45.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3ab5000640 1 -- start start 2024-03-30T00:48:45.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3ab5000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3ab015a770 0x7f3ab00c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:45.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3ab5000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ab00b1db0 0x7f3ab00cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:45.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3ab5000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab00ca130 con 0x7f3ab00ce780 2024-03-30T00:48:45.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3ab5000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab00cb120 con 0x7f3ab015a770 2024-03-30T00:48:45.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3ab5000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab015ab50 con 0x7f3ab00b1db0 2024-03-30T00:48:45.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3aae800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f3ab00ce780 0x7f3ab00710e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34678/0 (socket says 172.21.15.66:34678) 2024-03-30T00:48:45.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3aae800640 1 -- 172.21.15.66:0/1705682796 learned_addr learned my addr 172.21.15.66:0/1705682796 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3aad400640 1 --2- 172.21.15.66:0/1705682796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ab00b1db0 0x7f3ab00cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3aade00640 1 --2- 172.21.15.66:0/1705682796 >> v2:172.21.15.156:3300/0 conn(0x7f3ab015a770 0x7f3ab00c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3aaca00640 1 -- 172.21.15.66:0/1705682796 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836316744 0 0) 0x7f3ab00ca130 con 0x7f3ab00ce780 2024-03-30T00:48:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3aaca00640 1 -- 172.21.15.66:0/1705682796 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a94003530 con 0x7f3ab00ce780 2024-03-30T00:48:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3aaca00640 1 -- 172.21.15.66:0/1705682796 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3023444672 0 0) 0x7f3a94003530 con 0x7f3ab00ce780 2024-03-30T00:48:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3aad400640 1 -- 172.21.15.66:0/1705682796 >> v2:172.21.15.156:3300/0 conn(0x7f3ab015a770 msgr2=0x7f3ab00c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:45.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3aad400640 1 --2- 172.21.15.66:0/1705682796 >> v2:172.21.15.156:3300/0 conn(0x7f3ab015a770 0x7f3ab00c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:45.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3aad400640 1 -- 172.21.15.66:0/1705682796 >> v1:172.21.15.66:6789/0 conn(0x7f3ab00ce780 legacy=0x7f3ab00710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:45.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.097+0000 7f3aad400640 1 -- 172.21.15.66:0/1705682796 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab00ca400 con 0x7f3ab00b1db0 2024-03-30T00:48:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3aad400640 1 --2- 172.21.15.66:0/1705682796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ab00b1db0 0x7f3ab00cb540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0097c0 tx=0x7f3a9c01c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b904fe338988a05c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3aaca00640 1 -- 172.21.15.66:0/1705682796 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8001a00 con 0x7f3ab00b1db0 2024-03-30T00:48:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3aaca00640 1 -- 172.21.15.66:0/1705682796 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c0a7050 con 0x7f3ab00b1db0 2024-03-30T00:48:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3aaca00640 1 -- 172.21.15.66:0/1705682796 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0ab600 con 0x7f3ab00b1db0 2024-03-30T00:48:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3ab5000640 1 -- 172.21.15.66:0/1705682796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ab00b1db0 msgr2=0x7f3ab00cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3ab5000640 1 --2- 172.21.15.66:0/1705682796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ab00b1db0 0x7f3ab00cb540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0097c0 tx=0x7f3a9c01c390 comp rx=0 tx=0).stop 2024-03-30T00:48:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3ab5000640 1 -- 172.21.15.66:0/1705682796 shutdown_connections 2024-03-30T00:48:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3ab5000640 1 --2- 172.21.15.66:0/1705682796 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ab00b1db0 0x7f3ab00cb540 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3ab5000640 1 --2- 172.21.15.66:0/1705682796 >> v2:172.21.15.156:3300/0 conn(0x7f3ab015a770 0x7f3ab00c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3ab5000640 1 -- 172.21.15.66:0/1705682796 >> 172.21.15.66:0/1705682796 conn(0x7f3ab006a190 msgr2=0x7f3ab006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:45.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3ab5000640 1 -- 172.21.15.66:0/1705682796 shutdown_connections 2024-03-30T00:48:45.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.101+0000 7f3ab5000640 1 -- 172.21.15.66:0/1705682796 wait complete. 2024-03-30T00:48:45.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3ab5000640 1 Processor -- start 2024-03-30T00:48:45.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3ab5000640 1 -- start start 2024-03-30T00:48:45.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3ab5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ab00b1db0 0x7f3ab00ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3ab5000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ab00ce780 0x7f3ab00ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3ab5000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab015a770 0x7f3ab00cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3ab5000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0158700 con 0x7f3ab00b1db0 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3ab5000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0150d80 con 0x7f3ab015a770 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3ab5000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0151000 con 0x7f3ab00ce780 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3aade00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ab00b1db0 0x7f3ab00ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3aade00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ab00b1db0 0x7f3ab00ab190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35874/0 (socket says 172.21.15.66:35874) 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3aade00640 1 -- 172.21.15.66:0/2363248206 learned_addr learned my addr 172.21.15.66:0/2363248206 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3aad400640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ab00ce780 0x7f3ab00ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3aae800640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab015a770 0x7f3ab00cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3aade00640 1 -- 172.21.15.66:0/2363248206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ab00ce780 msgr2=0x7f3ab00ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3aade00640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ab00ce780 0x7f3ab00ab6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3aade00640 1 -- 172.21.15.66:0/2363248206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab015a770 msgr2=0x7f3ab00cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:45.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3aade00640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab015a770 0x7f3ab00cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:45.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3aade00640 1 -- 172.21.15.66:0/2363248206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c009420 con 0x7f3ab00b1db0 2024-03-30T00:48:45.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.105+0000 7f3aade00640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ab00b1db0 0x7f3ab00ab190 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000e870 tx=0x7f3aa000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:45.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.109+0000 7f3aa6000640 1 -- 172.21.15.66:0/2363248206 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0009510 con 0x7f3ab00b1db0 2024-03-30T00:48:45.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.109+0000 7f3aa6000640 1 -- 172.21.15.66:0/2363248206 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa000eec0 con 0x7f3ab00b1db0 2024-03-30T00:48:45.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.109+0000 7f3aa6000640 1 -- 172.21.15.66:0/2363248206 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000cb40 con 0x7f3ab00b1db0 2024-03-30T00:48:45.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.109+0000 7f3ab5000640 1 -- 172.21.15.66:0/2363248206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab01512e0 con 0x7f3ab00b1db0 2024-03-30T00:48:45.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.109+0000 7f3ab5000640 1 -- 172.21.15.66:0/2363248206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab01518a0 con 0x7f3ab00b1db0 2024-03-30T00:48:45.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.109+0000 7f3ab5000640 1 -- 172.21.15.66:0/2363248206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab0066c30 con 0x7f3ab00b1db0 2024-03-30T00:48:45.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.109+0000 7f3aa6000640 1 -- 172.21.15.66:0/2363248206 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3aa000cca0 con 0x7f3ab00b1db0 2024-03-30T00:48:45.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.109+0000 7f3aa6000640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a90066690 0x7f3a90068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:45.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.109+0000 7f3aa6000640 1 -- 172.21.15.66:0/2363248206 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f3aa0084120 con 0x7f3ab00b1db0 2024-03-30T00:48:45.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.109+0000 7f3aad400640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a90066690 0x7f3a90068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:45.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.113+0000 7f3aad400640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a90066690 0x7f3a90068b50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0ad000 tx=0x7f3a9c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:45.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.117+0000 7f3aa6000640 1 -- 172.21.15.66:0/2363248206 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3aa00515e0 con 0x7f3ab00b1db0 2024-03-30T00:48:45.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.277+0000 7f3ab5000640 1 -- 172.21.15.66:0/2363248206 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3ab00c8730 con 0x7f3ab00b1db0 2024-03-30T00:48:45.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.929+0000 7f3aa6000640 1 -- 172.21.15.66:0/2363248206 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v108) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3aa0054c00 con 0x7f3ab00b1db0 2024-03-30T00:48:45.940 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-03-30T00:48:45.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 -- 172.21.15.66:0/2363248206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a90066690 msgr2=0x7f3a90068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:45.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a90066690 0x7f3a90068b50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0ad000 tx=0x7f3a9c0a6000 comp rx=0 tx=0).stop 2024-03-30T00:48:45.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 -- 172.21.15.66:0/2363248206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ab00b1db0 msgr2=0x7f3ab00ab190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:45.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ab00b1db0 0x7f3ab00ab190 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000e870 tx=0x7f3aa000ec70 comp rx=0 tx=0).stop 2024-03-30T00:48:45.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 -- 172.21.15.66:0/2363248206 shutdown_connections 2024-03-30T00:48:45.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab015a770 0x7f3ab00cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:45.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ab00ce780 0x7f3ab00ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:45.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a90066690 0x7f3a90068b50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:45.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 --2- 172.21.15.66:0/2363248206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ab00b1db0 0x7f3ab00ab190 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:45.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 -- 172.21.15.66:0/2363248206 >> 172.21.15.66:0/2363248206 conn(0x7f3ab006a190 msgr2=0x7f3ab0152590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:45.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 -- 172.21.15.66:0/2363248206 shutdown_connections 2024-03-30T00:48:45.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:45.933+0000 7f3ab5000640 1 -- 172.21.15.66:0/2363248206 wait complete. 2024-03-30T00:48:45.980 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T00:48:46.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.113+0000 7f2cade00640 1 Processor -- start 2024-03-30T00:48:46.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.113+0000 7f2cade00640 1 -- start start 2024-03-30T00:48:46.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.113+0000 7f2cade00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2ca80ce520 0x7f2ca80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.113+0000 7f2cade00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ca80cabb0 0x7f2ca80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.113+0000 7f2cade00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca806d070 con 0x7f2ca815a640 2024-03-30T00:48:46.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.113+0000 7f2cade00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca80cbba0 con 0x7f2ca80ce520 2024-03-30T00:48:46.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.113+0000 7f2cade00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80cee70 con 0x7f2ca80cabb0 2024-03-30T00:48:46.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.117+0000 7f2c9e000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2ca80ce520 0x7f2ca80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.117+0000 7f2c9e000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2ca80ce520 0x7f2ca80ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:44424/0 (socket says 172.21.15.66:44424) 2024-03-30T00:48:46.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.117+0000 7f2c9e000640 1 -- 172.21.15.66:0/2608707919 learned_addr learned my addr 172.21.15.66:0/2608707919 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:46.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.117+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ca80cabb0 0x7f2ca80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.117+0000 7f2c9e000640 1 -- 172.21.15.66:0/2608707919 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ca80cabb0 msgr2=0x7f2ca80cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:46.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.117+0000 7f2c9e000640 1 --2- 172.21.15.66:0/2608707919 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ca80cabb0 0x7f2ca80cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.117+0000 7f2c9e000640 1 -- 172.21.15.66:0/2608707919 >> v1:172.21.15.156:6789/0 conn(0x7f2ca815a640 legacy=0x7f2ca806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.117+0000 7f2c9e000640 1 -- 172.21.15.66:0/2608707919 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80cb8c0 con 0x7f2ca80ce520 2024-03-30T00:48:46.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.117+0000 7f2ca7400640 1 --2- 172.21.15.66:0/2608707919 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ca80cabb0 0x7f2ca80cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:46.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.117+0000 7f2c9e000640 1 --2- 172.21.15.66:0/2608707919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ca80ce520 0x7f2ca80ce930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009870 tx=0x7f2c9001c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dcfc88a71af37c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:46.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2ca6000640 1 -- 172.21.15.66:0/2608707919 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c900a95f0 con 0x7f2ca80ce520 2024-03-30T00:48:46.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2ca6000640 1 -- 172.21.15.66:0/2608707919 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c900a9750 con 0x7f2ca80ce520 2024-03-30T00:48:46.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2ca6000640 1 -- 172.21.15.66:0/2608707919 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c900a9ae0 con 0x7f2ca80ce520 2024-03-30T00:48:46.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 -- 172.21.15.66:0/2608707919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ca80ce520 msgr2=0x7f2ca80ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 --2- 172.21.15.66:0/2608707919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ca80ce520 0x7f2ca80ce930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009870 tx=0x7f2c9001c530 comp rx=0 tx=0).stop 2024-03-30T00:48:46.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 -- 172.21.15.66:0/2608707919 shutdown_connections 2024-03-30T00:48:46.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 --2- 172.21.15.66:0/2608707919 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ca80cabb0 0x7f2ca80cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 --2- 172.21.15.66:0/2608707919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ca80ce520 0x7f2ca80ce930 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 -- 172.21.15.66:0/2608707919 >> 172.21.15.66:0/2608707919 conn(0x7f2ca8069f60 msgr2=0x7f2ca806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:46.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 -- 172.21.15.66:0/2608707919 shutdown_connections 2024-03-30T00:48:46.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 -- 172.21.15.66:0/2608707919 wait complete. 2024-03-30T00:48:46.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 Processor -- start 2024-03-30T00:48:46.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 -- start start 2024-03-30T00:48:46.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ca80cabb0 0x7f2ca80978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ca80ce520 0x7f2ca8151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ca815a640 0x7f2ca8162410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80cbff0 con 0x7f2ca80ce520 2024-03-30T00:48:46.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80cc290 con 0x7f2ca815a640 2024-03-30T00:48:46.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2cade00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80cc570 con 0x7f2ca80cabb0 2024-03-30T00:48:46.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ca80ce520 0x7f2ca8151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ca80ce520 0x7f2ca8151b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35880/0 (socket says 172.21.15.66:35880) 2024-03-30T00:48:46.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2c9e000640 1 -- 172.21.15.66:0/2346504754 learned_addr learned my addr 172.21.15.66:0/2346504754 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:46.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2ca7400640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ca815a640 0x7f2ca8162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.121+0000 7f2ca6a00640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ca80cabb0 0x7f2ca80978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9e000640 1 -- 172.21.15.66:0/2346504754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ca80cabb0 msgr2=0x7f2ca80978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9e000640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ca80cabb0 0x7f2ca80978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9e000640 1 -- 172.21.15.66:0/2346504754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ca815a640 msgr2=0x7f2ca8162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9e000640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ca815a640 0x7f2ca8162410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9e000640 1 -- 172.21.15.66:0/2346504754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca0009420 con 0x7f2ca80ce520 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9e000640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ca80ce520 0x7f2ca8151b00 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2c900ab040 tx=0x7f2c9001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9fe00640 1 -- 172.21.15.66:0/2346504754 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c900bc650 con 0x7f2ca80ce520 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9fe00640 1 -- 172.21.15.66:0/2346504754 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c900bc7d0 con 0x7f2ca80ce520 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9fe00640 1 -- 172.21.15.66:0/2346504754 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c900bcb80 con 0x7f2ca80ce520 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2cade00640 1 -- 172.21.15.66:0/2346504754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c900094d0 con 0x7f2ca80ce520 2024-03-30T00:48:46.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2cade00640 1 -- 172.21.15.66:0/2346504754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca8097e10 con 0x7f2ca80ce520 2024-03-30T00:48:46.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2cade00640 1 -- 172.21.15.66:0/2346504754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca8066c30 con 0x7f2ca80ce520 2024-03-30T00:48:46.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9fe00640 1 -- 172.21.15.66:0/2346504754 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2c900bcce0 con 0x7f2ca80ce520 2024-03-30T00:48:46.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9fe00640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c84066690 0x7f2c84068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.125+0000 7f2c9fe00640 1 -- 172.21.15.66:0/2346504754 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f2c9011eaf0 con 0x7f2ca80ce520 2024-03-30T00:48:46.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.129+0000 7f2ca6a00640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c84066690 0x7f2c84068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.129+0000 7f2ca6a00640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c84066690 0x7f2c84068b50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2c980012f0 tx=0x7f2c98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:46.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.133+0000 7f2c9fe00640 1 -- 172.21.15.66:0/2346504754 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2c900ebce0 con 0x7f2ca80ce520 2024-03-30T00:48:46.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.325+0000 7f2cade00640 1 -- 172.21.15.66:0/2346504754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2ca80b28b0 con 0x7f2ca80ce520 2024-03-30T00:48:46.346 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 15 and data pool 16 2024-03-30T00:48:46.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.333+0000 7f2c9fe00640 1 -- 172.21.15.66:0/2346504754 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v75) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2c900ef300 con 0x7f2ca80ce520 2024-03-30T00:48:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.341+0000 7f2cade00640 1 -- 172.21.15.66:0/2346504754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c84066690 msgr2=0x7f2c84068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.341+0000 7f2cade00640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c84066690 0x7f2c84068b50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2c980012f0 tx=0x7f2c98009040 comp rx=0 tx=0).stop 2024-03-30T00:48:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.341+0000 7f2cade00640 1 -- 172.21.15.66:0/2346504754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ca80ce520 msgr2=0x7f2ca8151b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.341+0000 7f2cade00640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ca80ce520 0x7f2ca8151b00 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2c900ab040 tx=0x7f2c9001ad70 comp rx=0 tx=0).stop 2024-03-30T00:48:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.341+0000 7f2cade00640 1 -- 172.21.15.66:0/2346504754 shutdown_connections 2024-03-30T00:48:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.341+0000 7f2cade00640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ca815a640 0x7f2ca8162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.341+0000 7f2cade00640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2c84066690 0x7f2c84068b50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.341+0000 7f2cade00640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ca80ce520 0x7f2ca8151b00 secure :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2c900ab040 tx=0x7f2c9001ad70 comp rx=0 tx=0).stop 2024-03-30T00:48:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.341+0000 7f2cade00640 1 --2- 172.21.15.66:0/2346504754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ca80cabb0 0x7f2ca80978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.341+0000 7f2cade00640 1 -- 172.21.15.66:0/2346504754 >> 172.21.15.66:0/2346504754 conn(0x7f2ca8069f60 msgr2=0x7f2ca80c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:46.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.349+0000 7f2cade00640 1 -- 172.21.15.66:0/2346504754 shutdown_connections 2024-03-30T00:48:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.349+0000 7f2cade00640 1 -- 172.21.15.66:0/2346504754 wait complete. 2024-03-30T00:48:46.394 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:48:46.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc050e00640 1 Processor -- start 2024-03-30T00:48:46.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc050e00640 1 -- start start 2024-03-30T00:48:46.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc050e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc04c0ad950 0x7fc04c0d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc050e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04c0b1870 con 0x7fc04c0d0a00 2024-03-30T00:48:46.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc050e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04c0ae9f0 con 0x7fc04c15db90 2024-03-30T00:48:46.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc050e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04c0d0eb0 con 0x7fc04c0ad950 2024-03-30T00:48:46.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc043e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc04c0ad950 0x7fc04c0d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc043e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc04c0ad950 0x7fc04c0d04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35890/0 (socket says 172.21.15.66:35890) 2024-03-30T00:48:46.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc043e00640 1 -- 172.21.15.66:0/688653886 learned_addr learned my addr 172.21.15.66:0/688653886 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:46.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc043e00640 1 -- 172.21.15.66:0/688653886 >> v1:172.21.15.156:6789/0 conn(0x7fc04c15db90 legacy=0x7fc04c0ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc043e00640 1 -- 172.21.15.66:0/688653886 >> v1:172.21.15.66:6789/0 conn(0x7fc04c0d0a00 legacy=0x7fc04c0aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.525+0000 7fc043e00640 1 -- 172.21.15.66:0/688653886 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c0adde0 con 0x7fc04c0ad950 2024-03-30T00:48:46.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.529+0000 7fc043e00640 1 --2- 172.21.15.66:0/688653886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc04c0ad950 0x7fc04c0d04c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fc0340098d0 tx=0x7fc03401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c51cc2d6b3833a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:46.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.529+0000 7fc049400640 1 -- 172.21.15.66:0/688653886 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340aa670 con 0x7fc04c0ad950 2024-03-30T00:48:46.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.529+0000 7fc049400640 1 -- 172.21.15.66:0/688653886 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0340aa7d0 con 0x7fc04c0ad950 2024-03-30T00:48:46.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.529+0000 7fc049400640 1 -- 172.21.15.66:0/688653886 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340aab60 con 0x7fc04c0ad950 2024-03-30T00:48:46.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.529+0000 7fc050e00640 1 -- 172.21.15.66:0/688653886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc04c0ad950 msgr2=0x7fc04c0d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.529+0000 7fc050e00640 1 --2- 172.21.15.66:0/688653886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc04c0ad950 0x7fc04c0d04c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fc0340098d0 tx=0x7fc03401c510 comp rx=0 tx=0).stop 2024-03-30T00:48:46.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.529+0000 7fc050e00640 1 -- 172.21.15.66:0/688653886 shutdown_connections 2024-03-30T00:48:46.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.529+0000 7fc050e00640 1 --2- 172.21.15.66:0/688653886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc04c0ad950 0x7fc04c0d04c0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.529+0000 7fc050e00640 1 -- 172.21.15.66:0/688653886 >> 172.21.15.66:0/688653886 conn(0x7fc04c089fb0 msgr2=0x7fc04c08a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:46.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.529+0000 7fc050e00640 1 -- 172.21.15.66:0/688653886 shutdown_connections 2024-03-30T00:48:46.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.529+0000 7fc050e00640 1 -- 172.21.15.66:0/688653886 wait complete. 2024-03-30T00:48:46.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc050e00640 1 Processor -- start 2024-03-30T00:48:46.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc050e00640 1 -- start start 2024-03-30T00:48:46.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc050e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc04c0d0a00 0x7fc04c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc050e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc04c15db90 0x7fc04c15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc050e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc04c0b0df0 0x7fc04c14dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc050e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c0c7a50 con 0x7fc04c0b0df0 2024-03-30T00:48:46.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc050e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c14ec50 con 0x7fc04c0d0a00 2024-03-30T00:48:46.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc050e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c15c210 con 0x7fc04c15db90 2024-03-30T00:48:46.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc049e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc04c0d0a00 0x7fc04c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc04a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc04c0b0df0 0x7fc04c14dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc049e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc04c0d0a00 0x7fc04c15bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:44432/0 (socket says 172.21.15.66:44432) 2024-03-30T00:48:46.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc049e00640 1 -- 172.21.15.66:0/1813781191 learned_addr learned my addr 172.21.15.66:0/1813781191 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:46.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc043e00640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc04c15db90 0x7fc04c15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc04a800640 1 -- 172.21.15.66:0/1813781191 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc04c15db90 msgr2=0x7fc04c15a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc04a800640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc04c15db90 0x7fc04c15a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc04a800640 1 -- 172.21.15.66:0/1813781191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc04c0d0a00 msgr2=0x7fc04c15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc04a800640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc04c0d0a00 0x7fc04c15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc04a800640 1 -- 172.21.15.66:0/1813781191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0340094d0 con 0x7fc04c0b0df0 2024-03-30T00:48:46.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc049e00640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc04c0d0a00 0x7fc04c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:46.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc043e00640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc04c15db90 0x7fc04c15a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:46.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc04a800640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc04c0b0df0 0x7fc04c14dd40 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fc04400c700 tx=0x7fc04400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:46.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc042000640 1 -- 172.21.15.66:0/1813781191 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc044003b80 con 0x7fc04c0b0df0 2024-03-30T00:48:46.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc042000640 1 -- 172.21.15.66:0/1813781191 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc044003ce0 con 0x7fc04c0b0df0 2024-03-30T00:48:46.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc042000640 1 -- 172.21.15.66:0/1813781191 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04400aa30 con 0x7fc04c0b0df0 2024-03-30T00:48:46.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc050e00640 1 -- 172.21.15.66:0/1813781191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04c15aea0 con 0x7fc04c0b0df0 2024-03-30T00:48:46.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.533+0000 7fc050e00640 1 -- 172.21.15.66:0/1813781191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04c15b460 con 0x7fc04c0b0df0 2024-03-30T00:48:46.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.537+0000 7fc050e00640 1 -- 172.21.15.66:0/1813781191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04c066c30 con 0x7fc04c0b0df0 2024-03-30T00:48:46.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.537+0000 7fc042000640 1 -- 172.21.15.66:0/1813781191 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc04400ab90 con 0x7fc04c0b0df0 2024-03-30T00:48:46.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.537+0000 7fc042000640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0280666c0 0x7fc028068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.537+0000 7fc042000640 1 -- 172.21.15.66:0/1813781191 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc044083f20 con 0x7fc04c0b0df0 2024-03-30T00:48:46.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.537+0000 7fc049e00640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0280666c0 0x7fc028068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.537+0000 7fc049e00640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0280666c0 0x7fc028068b80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc03c001c00 tx=0x7fc03c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:46.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.541+0000 7fc042000640 1 -- 172.21.15.66:0/1813781191 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc0440511f0 con 0x7fc04c0b0df0 2024-03-30T00:48:46.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.701+0000 7fc050e00640 1 -- 172.21.15.66:0/1813781191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc04c06ef00 con 0x7fc04c0b0df0 2024-03-30T00:48:46.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc042000640 1 -- 172.21.15.66:0/1813781191 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16261 (secure 0 0 0) 0x7fc044054810 con 0x7fc04c0b0df0 2024-03-30T00:48:46.715 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:46.715 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":109,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:48:46.332007+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:48:44.750350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-30T00:48:45.327928+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:48:46.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 -- 172.21.15.66:0/1813781191 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0280666c0 msgr2=0x7fc028068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0280666c0 0x7fc028068b80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc03c001c00 tx=0x7fc03c009040 comp rx=0 tx=0).stop 2024-03-30T00:48:46.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 -- 172.21.15.66:0/1813781191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc04c0b0df0 msgr2=0x7fc04c14dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc04c0b0df0 0x7fc04c14dd40 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fc04400c700 tx=0x7fc04400cbc0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 -- 172.21.15.66:0/1813781191 shutdown_connections 2024-03-30T00:48:46.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc0280666c0 0x7fc028068b80 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc04c0b0df0 0x7fc04c14dd40 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc04c15db90 0x7fc04c15a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 --2- 172.21.15.66:0/1813781191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc04c0d0a00 0x7fc04c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 -- 172.21.15.66:0/1813781191 >> 172.21.15.66:0/1813781191 conn(0x7fc04c089fb0 msgr2=0x7fc04c0cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:46.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 -- 172.21.15.66:0/1813781191 shutdown_connections 2024-03-30T00:48:46.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.705+0000 7fc050e00640 1 -- 172.21.15.66:0/1813781191 wait complete. 2024-03-30T00:48:46.736 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:48:46.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.865+0000 7f5c94a00640 1 Processor -- start 2024-03-30T00:48:46.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.865+0000 7f5c94a00640 1 -- start start 2024-03-30T00:48:46.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c94a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5c900cda90 0x7f5c900ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c94a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c9015a370 0x7f5c900cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c94a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9015a990 con 0x7f5c900cde70 2024-03-30T00:48:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c94a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c900b1b80 con 0x7f5c900cda90 2024-03-30T00:48:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c94a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c900cd3e0 con 0x7f5c9015a370 2024-03-30T00:48:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c8de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5c900cda90 0x7f5c900ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c8de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5c900cda90 0x7f5c900ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35912/0 (socket says 172.21.15.66:35912) 2024-03-30T00:48:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c8de00640 1 -- 172.21.15.66:0/2469246651 learned_addr learned my addr 172.21.15.66:0/2469246651 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c85400640 1 --1- 172.21.15.66:0/2469246651 >> v1:172.21.15.66:6789/0 conn(0x7f5c900cde70 0x7f5c90159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34690/0 (socket says 172.21.15.66:34690) 2024-03-30T00:48:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c8e800640 1 --2- 172.21.15.66:0/2469246651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c9015a370 0x7f5c900cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c8de00640 1 -- 172.21.15.66:0/2469246651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c9015a370 msgr2=0x7f5c900cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c8de00640 1 --2- 172.21.15.66:0/2469246651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c9015a370 0x7f5c900cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c8de00640 1 -- 172.21.15.66:0/2469246651 >> v1:172.21.15.66:6789/0 conn(0x7f5c900cde70 legacy=0x7f5c90159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.869+0000 7f5c8de00640 1 -- 172.21.15.66:0/2469246651 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900b11e0 con 0x7f5c900cda90 2024-03-30T00:48:46.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c8de00640 1 --2- 172.21.15.66:0/2469246651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c900cda90 0x7f5c900ce520 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009870 tx=0x7f5c8001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe7cfdb18613365 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:46.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c8d400640 1 -- 172.21.15.66:0/2469246651 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800ac640 con 0x7f5c900cda90 2024-03-30T00:48:46.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c8d400640 1 -- 172.21.15.66:0/2469246651 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c800ac7a0 con 0x7f5c900cda90 2024-03-30T00:48:46.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c8d400640 1 -- 172.21.15.66:0/2469246651 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800acb30 con 0x7f5c900cda90 2024-03-30T00:48:46.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 -- 172.21.15.66:0/2469246651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c900cda90 msgr2=0x7f5c900ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 --2- 172.21.15.66:0/2469246651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c900cda90 0x7f5c900ce520 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009870 tx=0x7f5c8001c4d0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 -- 172.21.15.66:0/2469246651 shutdown_connections 2024-03-30T00:48:46.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 --2- 172.21.15.66:0/2469246651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c9015a370 0x7f5c900cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 --2- 172.21.15.66:0/2469246651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c900cda90 0x7f5c900ce520 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 -- 172.21.15.66:0/2469246651 >> 172.21.15.66:0/2469246651 conn(0x7f5c90069f60 msgr2=0x7f5c9006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:46.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 -- 172.21.15.66:0/2469246651 shutdown_connections 2024-03-30T00:48:46.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 -- 172.21.15.66:0/2469246651 wait complete. 2024-03-30T00:48:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 Processor -- start 2024-03-30T00:48:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 -- start start 2024-03-30T00:48:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c900cda90 0x7f5c900acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.873+0000 7f5c94a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c900cde70 0x7f5c90162ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c94a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c9015a370 0x7f5c90159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c94a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c90164860 con 0x7f5c900cda90 2024-03-30T00:48:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c94a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c90162850 con 0x7f5c900cde70 2024-03-30T00:48:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c94a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c90163fc0 con 0x7f5c9015a370 2024-03-30T00:48:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c8de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c900cda90 0x7f5c900acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c8de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c900cda90 0x7f5c900acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35914/0 (socket says 172.21.15.66:35914) 2024-03-30T00:48:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c8de00640 1 -- 172.21.15.66:0/1662645917 learned_addr learned my addr 172.21.15.66:0/1662645917 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c85400640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c900cde70 0x7f5c90162ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c8e800640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c9015a370 0x7f5c90159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c8de00640 1 -- 172.21.15.66:0/1662645917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c9015a370 msgr2=0x7f5c90159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c8de00640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c9015a370 0x7f5c90159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c8de00640 1 -- 172.21.15.66:0/1662645917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c900cde70 msgr2=0x7f5c90162ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c8de00640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c900cde70 0x7f5c90162ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c8de00640 1 -- 172.21.15.66:0/1662645917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c800094d0 con 0x7f5c900cda90 2024-03-30T00:48:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c8e800640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c9015a370 0x7f5c90159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c85400640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c900cde70 0x7f5c90162ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c8de00640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c900cda90 0x7f5c900acb50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5c8001ab80 tx=0x7f5c8001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:46.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c86a00640 1 -- 172.21.15.66:0/1662645917 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800ac640 con 0x7f5c900cda90 2024-03-30T00:48:46.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c86a00640 1 -- 172.21.15.66:0/1662645917 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c800b7860 con 0x7f5c900cda90 2024-03-30T00:48:46.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c86a00640 1 -- 172.21.15.66:0/1662645917 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800b7c10 con 0x7f5c900cda90 2024-03-30T00:48:46.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c94a00640 1 -- 172.21.15.66:0/1662645917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c901642a0 con 0x7f5c900cda90 2024-03-30T00:48:46.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c94a00640 1 -- 172.21.15.66:0/1662645917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c90163c00 con 0x7f5c900cda90 2024-03-30T00:48:46.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c94a00640 1 -- 172.21.15.66:0/1662645917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c90162620 con 0x7f5c900cda90 2024-03-30T00:48:46.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c86a00640 1 -- 172.21.15.66:0/1662645917 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5c800b3070 con 0x7f5c900cda90 2024-03-30T00:48:46.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.877+0000 7f5c86a00640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c6c0666c0 0x7f5c6c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:46.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.881+0000 7f5c86a00640 1 -- 172.21.15.66:0/1662645917 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5c8011ebe0 con 0x7f5c900cda90 2024-03-30T00:48:46.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.881+0000 7f5c85400640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c6c0666c0 0x7f5c6c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:46.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.881+0000 7f5c85400640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c6c0666c0 0x7f5c6c068b80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5c900c9610 tx=0x7f5c78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:46.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:46.881+0000 7f5c86a00640 1 -- 172.21.15.66:0/1662645917 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5c90162620 con 0x7f5c900cda90 2024-03-30T00:48:47.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.045+0000 7f5c94a00640 1 -- 172.21.15.66:0/1662645917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c90162620 con 0x7f5c900cda90 2024-03-30T00:48:47.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.045+0000 7f5c86a00640 1 -- 172.21.15.66:0/1662645917 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16261 (secure 0 0 0) 0x7f5c90162620 con 0x7f5c900cda90 2024-03-30T00:48:47.056 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:47.056 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":109,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:48:46.332007+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:48:44.750350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-30T00:48:45.327928+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:48:47.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 -- 172.21.15.66:0/1662645917 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c6c0666c0 msgr2=0x7f5c6c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:47.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c6c0666c0 0x7f5c6c068b80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5c900c9610 tx=0x7f5c78009040 comp rx=0 tx=0).stop 2024-03-30T00:48:47.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 -- 172.21.15.66:0/1662645917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c900cda90 msgr2=0x7f5c900acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:47.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c900cda90 0x7f5c900acb50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5c8001ab80 tx=0x7f5c8001ad10 comp rx=0 tx=0).stop 2024-03-30T00:48:47.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 -- 172.21.15.66:0/1662645917 shutdown_connections 2024-03-30T00:48:47.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c9015a370 0x7f5c90159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:47.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c900cde70 0x7f5c90162ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:47.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c6c0666c0 0x7f5c6c068b80 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:47.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 --2- 172.21.15.66:0/1662645917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c900cda90 0x7f5c900acb50 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:47.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 -- 172.21.15.66:0/1662645917 >> 172.21.15.66:0/1662645917 conn(0x7f5c90069f60 msgr2=0x7f5c900cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:47.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 -- 172.21.15.66:0/1662645917 shutdown_connections 2024-03-30T00:48:47.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.049+0000 7f5c94a00640 1 -- 172.21.15.66:0/1662645917 wait complete. 2024-03-30T00:48:47.077 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T00:48:47.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc26400640 1 Processor -- start 2024-03-30T00:48:47.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc26400640 1 -- start start 2024-03-30T00:48:47.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc26400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdc200cf2f0 0x7fdc2006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:47.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc26400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdc200cb570 0x7fdc200cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:47.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc26400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc200cb0e0 con 0x7fdc2015ca80 2024-03-30T00:48:47.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc26400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc200cf7f0 con 0x7fdc200cb570 2024-03-30T00:48:47.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc26400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc200cfad0 con 0x7fdc200cf2f0 2024-03-30T00:48:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc1f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdc2015ca80 0x7fdc2006b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34692/0 (socket says 172.21.15.66:34692) 2024-03-30T00:48:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc1f400640 1 -- 172.21.15.66:0/3847291836 learned_addr learned my addr 172.21.15.66:0/3847291836 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc1fe00640 1 --2- 172.21.15.66:0/3847291836 >> v2:172.21.15.66:3300/0 conn(0x7fdc200cb570 0x7fdc200cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc1ea00640 1 --2- 172.21.15.66:0/3847291836 >> v2:172.21.15.156:3300/0 conn(0x7fdc200cf2f0 0x7fdc2006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc1fe00640 1 -- 172.21.15.66:0/3847291836 >> v2:172.21.15.156:3300/0 conn(0x7fdc200cf2f0 msgr2=0x7fdc2006b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc1fe00640 1 --2- 172.21.15.66:0/3847291836 >> v2:172.21.15.156:3300/0 conn(0x7fdc200cf2f0 0x7fdc2006b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc1fe00640 1 -- 172.21.15.66:0/3847291836 >> v1:172.21.15.66:6789/0 conn(0x7fdc2015ca80 legacy=0x7fdc2006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc1fe00640 1 -- 172.21.15.66:0/3847291836 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc200cad70 con 0x7fdc200cb570 2024-03-30T00:48:47.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.213+0000 7fdc1ea00640 1 --2- 172.21.15.66:0/3847291836 >> v2:172.21.15.156:3300/0 conn(0x7fdc200cf2f0 0x7fdc2006b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:47.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc1fe00640 1 --2- 172.21.15.66:0/3847291836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc200cb570 0x7fdc200cba20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fdc2006ced0 tx=0x7fdc1801f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6048c58d1259928 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:47.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc1e000640 1 -- 172.21.15.66:0/3847291836 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180af600 con 0x7fdc200cb570 2024-03-30T00:48:47.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc1e000640 1 -- 172.21.15.66:0/3847291836 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc180ab050 con 0x7fdc200cb570 2024-03-30T00:48:47.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc1e000640 1 -- 172.21.15.66:0/3847291836 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180af980 con 0x7fdc200cb570 2024-03-30T00:48:47.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc26400640 1 -- 172.21.15.66:0/3847291836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc200cb570 msgr2=0x7fdc200cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:47.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc26400640 1 --2- 172.21.15.66:0/3847291836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc200cb570 0x7fdc200cba20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fdc2006ced0 tx=0x7fdc1801f110 comp rx=0 tx=0).stop 2024-03-30T00:48:47.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc26400640 1 -- 172.21.15.66:0/3847291836 shutdown_connections 2024-03-30T00:48:47.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc26400640 1 --2- 172.21.15.66:0/3847291836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc200cb570 0x7fdc200cba20 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:47.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc26400640 1 --2- 172.21.15.66:0/3847291836 >> v2:172.21.15.156:3300/0 conn(0x7fdc200cf2f0 0x7fdc2006b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:47.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc26400640 1 -- 172.21.15.66:0/3847291836 >> 172.21.15.66:0/3847291836 conn(0x7fdc200b2090 msgr2=0x7fdc200b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:47.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc26400640 1 -- 172.21.15.66:0/3847291836 shutdown_connections 2024-03-30T00:48:47.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc26400640 1 -- 172.21.15.66:0/3847291836 wait complete. 2024-03-30T00:48:47.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc26400640 1 Processor -- start 2024-03-30T00:48:47.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.217+0000 7fdc26400640 1 -- start start 2024-03-30T00:48:47.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc26400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc200cb570 0x7fdc2006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:47.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc26400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc200cf2f0 0x7fdc200b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:47.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc26400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2015ca80 0x7fdc200b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:47.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc26400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2009df30 con 0x7fdc200cf2f0 2024-03-30T00:48:47.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc26400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200b5fd0 con 0x7fdc2015ca80 2024-03-30T00:48:47.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc26400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200b42b0 con 0x7fdc200cb570 2024-03-30T00:48:47.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc1ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc200cf2f0 0x7fdc200b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:47.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc1ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc200cf2f0 0x7fdc200b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35926/0 (socket says 172.21.15.66:35926) 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc1ea00640 1 -- 172.21.15.66:0/637114697 learned_addr learned my addr 172.21.15.66:0/637114697 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc1f400640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc200cb570 0x7fdc2006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc1fe00640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2015ca80 0x7fdc200b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc1ea00640 1 -- 172.21.15.66:0/637114697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc200cb570 msgr2=0x7fdc2006e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc1ea00640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc200cb570 0x7fdc2006e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc1ea00640 1 -- 172.21.15.66:0/637114697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2015ca80 msgr2=0x7fdc200b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc1ea00640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2015ca80 0x7fdc200b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc1ea00640 1 -- 172.21.15.66:0/637114697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc18009a70 con 0x7fdc200cf2f0 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc1ea00640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc200cf2f0 0x7fdc200b5460 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c7c0 tx=0x7fdc0c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc17400640 1 -- 172.21.15.66:0/637114697 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c007a30 con 0x7fdc200cf2f0 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc17400640 1 -- 172.21.15.66:0/637114697 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c007b90 con 0x7fdc200cf2f0 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc17400640 1 -- 172.21.15.66:0/637114697 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00ce70 con 0x7fdc200cf2f0 2024-03-30T00:48:47.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc26400640 1 -- 172.21.15.66:0/637114697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc200b4590 con 0x7fdc200cf2f0 2024-03-30T00:48:47.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc26400640 1 -- 172.21.15.66:0/637114697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc200b4b50 con 0x7fdc200cf2f0 2024-03-30T00:48:47.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.221+0000 7fdc26400640 1 -- 172.21.15.66:0/637114697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc200b4d40 con 0x7fdc200cf2f0 2024-03-30T00:48:47.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.225+0000 7fdc17400640 1 -- 172.21.15.66:0/637114697 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdc0c0113f0 con 0x7fdc200cf2f0 2024-03-30T00:48:47.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.225+0000 7fdc17400640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc00066690 0x7fdc00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:47.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.225+0000 7fdc17400640 1 -- 172.21.15.66:0/637114697 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdc0c084760 con 0x7fdc200cf2f0 2024-03-30T00:48:47.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.225+0000 7fdc1f400640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc00066690 0x7fdc00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:47.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.225+0000 7fdc1f400640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc00066690 0x7fdc00068b50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdc10005890 tx=0x7fdc10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:47.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.229+0000 7fdc17400640 1 -- 172.21.15.66:0/637114697 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdc0c0519a0 con 0x7fdc200cf2f0 2024-03-30T00:48:47.368 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:48:47.371 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:48:47.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:47.425+0000 7fdc26400640 1 -- 172.21.15.66:0/637114697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdc20066c30 con 0x7fdc200cf2f0 2024-03-30T00:48:47.729 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-03-30T00:48:47.721+0000 7f4912e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:48:48.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.337+0000 7fdc17400640 1 -- 172.21.15.66:0/637114697 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v78) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdc0c054fc0 con 0x7fdc200cf2f0 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 -- 172.21.15.66:0/637114697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc00066690 msgr2=0x7fdc00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc00066690 0x7fdc00068b50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdc10005890 tx=0x7fdc10009040 comp rx=0 tx=0).stop 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 -- 172.21.15.66:0/637114697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc200cf2f0 msgr2=0x7fdc200b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc200cf2f0 0x7fdc200b5460 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c7c0 tx=0x7fdc0c00cc80 comp rx=0 tx=0).stop 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 -- 172.21.15.66:0/637114697 shutdown_connections 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2015ca80 0x7fdc200b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdc00066690 0x7fdc00068b50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc200cf2f0 0x7fdc200b5460 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 --2- 172.21.15.66:0/637114697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc200cb570 0x7fdc2006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 -- 172.21.15.66:0/637114697 >> 172.21.15.66:0/637114697 conn(0x7fdc200b2090 msgr2=0x7fdc2015acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 -- 172.21.15.66:0/637114697 shutdown_connections 2024-03-30T00:48:48.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.341+0000 7fdc26400640 1 -- 172.21.15.66:0/637114697 wait complete. 2024-03-30T00:48:48.374 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T00:48:48.374 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T00:48:48.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8760c00640 1 Processor -- start 2024-03-30T00:48:48.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8760c00640 1 -- start start 2024-03-30T00:48:48.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8760c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875c0cc560 0x7f875c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:48.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8760c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f875c06f8e0 0x7f875c06e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8760c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f875c06e630 0x7f875c0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8760c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875c160640 con 0x7f875c06f8e0 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8760c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875c06eaa0 con 0x7f875c06e630 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8760c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c06b0b0 con 0x7f875c0cc560 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8753e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f875c06f8e0 0x7f875c06e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8753e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f875c06f8e0 0x7f875c06e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35942/0 (socket says 172.21.15.66:35942) 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8753e00640 1 -- 172.21.15.66:0/1020179664 learned_addr learned my addr 172.21.15.66:0/1020179664 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8759e00640 1 --2- 172.21.15.66:0/1020179664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875c0cc560 0x7f875c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f875a800640 1 --2- 172.21.15.66:0/1020179664 >> v2:172.21.15.156:3300/0 conn(0x7f875c06e630 0x7f875c0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8753e00640 1 -- 172.21.15.66:0/1020179664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875c0cc560 msgr2=0x7f875c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8753e00640 1 --2- 172.21.15.66:0/1020179664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875c0cc560 0x7f875c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8753e00640 1 -- 172.21.15.66:0/1020179664 >> v2:172.21.15.156:3300/0 conn(0x7f875c06e630 msgr2=0x7f875c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8753e00640 1 --2- 172.21.15.66:0/1020179664 >> v2:172.21.15.156:3300/0 conn(0x7f875c06e630 0x7f875c0ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:48.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f8753e00640 1 -- 172.21.15.66:0/1020179664 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c06b390 con 0x7f875c06f8e0 2024-03-30T00:48:48.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.557+0000 7f875a800640 1 --2- 172.21.15.66:0/1020179664 >> v2:172.21.15.156:3300/0 conn(0x7f875c06e630 0x7f875c0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:48.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8753e00640 1 --2- 172.21.15.66:0/1020179664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f875c06f8e0 0x7f875c06e0f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f875c06dac0 tx=0x7f874401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4562775b718ba0b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:48.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8759400640 1 -- 172.21.15.66:0/1020179664 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87440ab600 con 0x7f875c06f8e0 2024-03-30T00:48:48.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8759400640 1 -- 172.21.15.66:0/1020179664 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87440a7050 con 0x7f875c06f8e0 2024-03-30T00:48:48.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8759400640 1 -- 172.21.15.66:0/1020179664 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87440ab980 con 0x7f875c06f8e0 2024-03-30T00:48:48.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8760c00640 1 -- 172.21.15.66:0/1020179664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f875c06f8e0 msgr2=0x7f875c06e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:48.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8760c00640 1 --2- 172.21.15.66:0/1020179664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f875c06f8e0 0x7f875c06e0f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f875c06dac0 tx=0x7f874401c230 comp rx=0 tx=0).stop 2024-03-30T00:48:48.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8760c00640 1 -- 172.21.15.66:0/1020179664 shutdown_connections 2024-03-30T00:48:48.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8760c00640 1 --2- 172.21.15.66:0/1020179664 >> v2:172.21.15.156:3300/0 conn(0x7f875c06e630 0x7f875c0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:48.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8760c00640 1 --2- 172.21.15.66:0/1020179664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f875c06f8e0 0x7f875c06e0f0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:48.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8760c00640 1 --2- 172.21.15.66:0/1020179664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875c0cc560 0x7f875c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:48.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8760c00640 1 -- 172.21.15.66:0/1020179664 >> 172.21.15.66:0/1020179664 conn(0x7f875c0d0540 msgr2=0x7f875c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:48.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8760c00640 1 -- 172.21.15.66:0/1020179664 shutdown_connections 2024-03-30T00:48:48.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.561+0000 7f8760c00640 1 -- 172.21.15.66:0/1020179664 wait complete. 2024-03-30T00:48:48.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8760c00640 1 Processor -- start 2024-03-30T00:48:48.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8760c00640 1 -- start start 2024-03-30T00:48:48.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8760c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875c06e630 0x7f875c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:48.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8760c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875c06f8e0 0x7f875c0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:48.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8760c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f875c0cc560 0x7f875c1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:48.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8760c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c0b0470 con 0x7f875c0cc560 2024-03-30T00:48:48.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8760c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c0b06a0 con 0x7f875c06e630 2024-03-30T00:48:48.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8760c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c153e00 con 0x7f875c06f8e0 2024-03-30T00:48:48.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f875a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f875c0cc560 0x7f875c1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8759e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875c06e630 0x7f875c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8753e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875c06f8e0 0x7f875c0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8753e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875c06f8e0 0x7f875c0aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:57586/0 (socket says 172.21.15.66:57586) 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8753e00640 1 -- 172.21.15.66:0/2057150652 learned_addr learned my addr 172.21.15.66:0/2057150652 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8759e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875c06e630 0x7f875c0b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:44486/0 (socket says 172.21.15.66:44486) 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f875a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f875c0cc560 0x7f875c1538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35944/0 (socket says 172.21.15.66:35944) 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f875a800640 1 -- 172.21.15.66:0/2057150652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875c06f8e0 msgr2=0x7f875c0aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f875a800640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875c06f8e0 0x7f875c0aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f875a800640 1 -- 172.21.15.66:0/2057150652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875c06e630 msgr2=0x7f875c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f875a800640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875c06e630 0x7f875c0b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f875a800640 1 -- 172.21.15.66:0/2057150652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87440094d0 con 0x7f875c0cc560 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f875a800640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f875c0cc560 0x7f875c1538c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f8754007fd0 tx=0x7f875400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8752000640 1 -- 172.21.15.66:0/2057150652 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400d840 con 0x7f875c0cc560 2024-03-30T00:48:48.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8752000640 1 -- 172.21.15.66:0/2057150652 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875400d9a0 con 0x7f875c0cc560 2024-03-30T00:48:48.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8752000640 1 -- 172.21.15.66:0/2057150652 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400dd50 con 0x7f875c0cc560 2024-03-30T00:48:48.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8760c00640 1 -- 172.21.15.66:0/2057150652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875c0c7970 con 0x7f875c0cc560 2024-03-30T00:48:48.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.565+0000 7f8760c00640 1 -- 172.21.15.66:0/2057150652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875c0c7f30 con 0x7f875c0cc560 2024-03-30T00:48:48.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.569+0000 7f8760c00640 1 -- 172.21.15.66:0/2057150652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875c066c30 con 0x7f875c0cc560 2024-03-30T00:48:48.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.569+0000 7f8752000640 1 -- 172.21.15.66:0/2057150652 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f875401e3b0 con 0x7f875c0cc560 2024-03-30T00:48:48.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.569+0000 7f8752000640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8738066690 0x7f8738068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:48.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.569+0000 7f8752000640 1 -- 172.21.15.66:0/2057150652 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8754084b20 con 0x7f875c0cc560 2024-03-30T00:48:48.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.569+0000 7f8759e00640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8738066690 0x7f8738068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:48.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.569+0000 7f8759e00640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8738066690 0x7f8738068b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f875c0b1990 tx=0x7f874c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:48.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.573+0000 7f8752000640 1 -- 172.21.15.66:0/2057150652 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8754051e90 con 0x7f875c0cc560 2024-03-30T00:48:48.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:48.769+0000 7f8760c00640 1 -- 172.21.15.66:0/2057150652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f875c0aee60 con 0x7f875c0cc560 2024-03-30T00:48:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8752000640 1 -- 172.21.15.66:0/2057150652 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v79) v1 ==== 122+0+0 (secure 0 0 0) 0x7f875401a020 con 0x7f875c0cc560 2024-03-30T00:48:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 -- 172.21.15.66:0/2057150652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8738066690 msgr2=0x7f8738068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8738066690 0x7f8738068b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f875c0b1990 tx=0x7f874c008040 comp rx=0 tx=0).stop 2024-03-30T00:48:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 -- 172.21.15.66:0/2057150652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f875c0cc560 msgr2=0x7f875c1538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f875c0cc560 0x7f875c1538c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f8754007fd0 tx=0x7f875400a710 comp rx=0 tx=0).stop 2024-03-30T00:48:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 -- 172.21.15.66:0/2057150652 shutdown_connections 2024-03-30T00:48:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8738066690 0x7f8738068b50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f875c0cc560 0x7f875c1538c0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f875c06f8e0 0x7f875c0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 --2- 172.21.15.66:0/2057150652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f875c06e630 0x7f875c0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 -- 172.21.15.66:0/2057150652 >> 172.21.15.66:0/2057150652 conn(0x7f875c0d0540 msgr2=0x7f875c0adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:49.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 -- 172.21.15.66:0/2057150652 shutdown_connections 2024-03-30T00:48:49.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.341+0000 7f8760c00640 1 -- 172.21.15.66:0/2057150652 wait complete. 2024-03-30T00:48:49.376 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T00:48:49.376 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:48:49.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.509+0000 7fedcca00640 1 Processor -- start 2024-03-30T00:48:49.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.509+0000 7fedcca00640 1 -- start start 2024-03-30T00:48:49.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedcca00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fedc80af420 0x7fedc80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedcca00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc80cbb90 con 0x7fedc80cee50 2024-03-30T00:48:49.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedcca00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc80ce1b0 con 0x7fedc815de30 2024-03-30T00:48:49.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedcca00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc80afa10 con 0x7fedc80af420 2024-03-30T00:48:49.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedbd400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fedc80cee50 0x7fedc806e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53914/0 (socket says 172.21.15.66:53914) 2024-03-30T00:48:49.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedbd400640 1 -- 172.21.15.66:0/1286113358 learned_addr learned my addr 172.21.15.66:0/1286113358 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc6800640 1 --2- 172.21.15.66:0/1286113358 >> v2:172.21.15.156:3300/0 conn(0x7fedc80af420 0x7fedc80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503052007 0 0) 0x7fedc80cbb90 con 0x7fedc80cee50 2024-03-30T00:48:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedac003530 con 0x7fedc80cee50 2024-03-30T00:48:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 99708722 0 0) 0x7fedac003530 con 0x7fedc80cee50 2024-03-30T00:48:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 >> v2:172.21.15.156:3300/0 conn(0x7fedc80af420 msgr2=0x7fedc80cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 --2- 172.21.15.66:0/1286113358 >> v2:172.21.15.156:3300/0 conn(0x7fedc80af420 0x7fedc80cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 >> v1:172.21.15.156:6789/0 conn(0x7fedc815de30 legacy=0x7fedc80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80cf430 con 0x7fedc80cee50 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fedb0001680 con 0x7fedc80cee50 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 >> v1:172.21.15.66:6789/0 conn(0x7fedc80cee50 legacy=0x7fedc806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fedac003980 0x7fedac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedac0047d0 0x7fedac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedac005190 0x7fedac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80afa10 con 0x7fedac0047d0 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac00c030 con 0x7fedac005190 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac00c1f0 con 0x7fedac003980 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedbd400640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedac0047d0 0x7fedac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc6800640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedac005190 0x7fedac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc5e00640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fedac003980 0x7fedac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedbd400640 1 -- 172.21.15.66:0/1286113358 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fedac003980 msgr2=0x7fedac003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedbd400640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fedac003980 0x7fedac003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedbd400640 1 -- 172.21.15.66:0/1286113358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedac005190 msgr2=0x7fedac00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:49.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedbd400640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedac005190 0x7fedac00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedbd400640 1 -- 172.21.15.66:0/1286113358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac003530 con 0x7fedac0047d0 2024-03-30T00:48:49.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc6800640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedac005190 0x7fedac00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:49.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.513+0000 7fedc6800640 1 -- 172.21.15.66:0/1286113358 reap_dead start 2024-03-30T00:48:49.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedbd400640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedac0047d0 0x7fedac004c50 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fedb000c240 tx=0x7fedb001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b40e1c53472047 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:49.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0005ae0 con 0x7fedac0047d0 2024-03-30T00:48:49.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb00ab070 con 0x7fedac0047d0 2024-03-30T00:48:49.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedc5400640 1 -- 172.21.15.66:0/1286113358 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0007070 con 0x7fedac0047d0 2024-03-30T00:48:49.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedcca00640 1 -- 172.21.15.66:0/1286113358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedac0047d0 msgr2=0x7fedac004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedcca00640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedac0047d0 0x7fedac004c50 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fedb000c240 tx=0x7fedb001f390 comp rx=0 tx=0).stop 2024-03-30T00:48:49.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedcca00640 1 -- 172.21.15.66:0/1286113358 shutdown_connections 2024-03-30T00:48:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedcca00640 1 --2- 172.21.15.66:0/1286113358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedac0047d0 0x7fedac004c50 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedcca00640 1 -- 172.21.15.66:0/1286113358 >> 172.21.15.66:0/1286113358 conn(0x7fedc80d0540 msgr2=0x7fedc80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedcca00640 1 -- 172.21.15.66:0/1286113358 shutdown_connections 2024-03-30T00:48:49.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedcca00640 1 -- 172.21.15.66:0/1286113358 wait complete. 2024-03-30T00:48:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.517+0000 7fedcca00640 1 Processor -- start 2024-03-30T00:48:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedcca00640 1 -- start start 2024-03-30T00:48:49.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedcca00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedac0047d0 0x7fedc80cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedcca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedc80c8660 0x7fedc80c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedcca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fedc8150b20 0x7fedc8150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedcca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80b4d60 con 0x7fedc80c8660 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedcca00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80c91b0 con 0x7fedac0047d0 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedcca00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80c9490 con 0x7fedc8150b20 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedc80c8660 0x7fedc80c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedc80c8660 0x7fedc80c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41652/0 (socket says 172.21.15.66:41652) 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbd400640 1 -- 172.21.15.66:0/1444078828 learned_addr learned my addr 172.21.15.66:0/1444078828 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedc5e00640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedac0047d0 0x7fedc80cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedc6800640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fedc8150b20 0x7fedc8150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbd400640 1 -- 172.21.15.66:0/1444078828 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fedc8150b20 msgr2=0x7fedc8150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbd400640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fedc8150b20 0x7fedc8150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbd400640 1 -- 172.21.15.66:0/1444078828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedac0047d0 msgr2=0x7fedc80cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbd400640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedac0047d0 0x7fedc80cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbd400640 1 -- 172.21.15.66:0/1444078828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80b46a0 con 0x7fedc80c8660 2024-03-30T00:48:49.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbd400640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedc80c8660 0x7fedc80c8ac0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fedb001f1c0 tx=0x7fedac005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:49.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbea00640 1 -- 172.21.15.66:0/1444078828 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0001680 con 0x7fedc80c8660 2024-03-30T00:48:49.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbea00640 1 -- 172.21.15.66:0/1444078828 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb0007070 con 0x7fedc80c8660 2024-03-30T00:48:49.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedbea00640 1 -- 172.21.15.66:0/1444078828 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0007400 con 0x7fedc80c8660 2024-03-30T00:48:49.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedcca00640 1 -- 172.21.15.66:0/1444078828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc80b4980 con 0x7fedc80c8660 2024-03-30T00:48:49.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedcca00640 1 -- 172.21.15.66:0/1444078828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc80cda60 con 0x7fedc80c8660 2024-03-30T00:48:49.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.521+0000 7fedcca00640 1 -- 172.21.15.66:0/1444078828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc80cdf60 con 0x7fedc80c8660 2024-03-30T00:48:49.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.525+0000 7fedbea00640 1 -- 172.21.15.66:0/1444078828 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fedb00b9030 con 0x7fedc80c8660 2024-03-30T00:48:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.525+0000 7fedbea00640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feda4066690 0x7feda4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.525+0000 7fedbea00640 1 -- 172.21.15.66:0/1444078828 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fedb00b7070 con 0x7fedc80c8660 2024-03-30T00:48:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.525+0000 7fedc5e00640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feda4066690 0x7feda4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.525+0000 7fedc5e00640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feda4066690 0x7feda4068b50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fedb8007e30 tx=0x7fedb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:49.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.529+0000 7fedbea00640 1 -- 172.21.15.66:0/1444078828 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fedb00ee450 con 0x7fedc80c8660 2024-03-30T00:48:49.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.721+0000 7fedcca00640 1 -- 172.21.15.66:0/1444078828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedc8066c30 con 0x7fedc80c8660 2024-03-30T00:48:49.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.721+0000 7fedbea00640 1 -- 172.21.15.66:0/1444078828 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fedb00c1090 con 0x7fedc80c8660 2024-03-30T00:48:49.733 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:49.733 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":79,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:48:49.341688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8646":{"gid":8646,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/2988412199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2988412199},{"type":"v1","addr":"172.21.15.66:6837","nonce":2988412199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:48:49.734 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 79 2024-03-30T00:48:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 -- 172.21.15.66:0/1444078828 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feda4066690 msgr2=0x7feda4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feda4066690 0x7feda4068b50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fedb8007e30 tx=0x7fedb8009040 comp rx=0 tx=0).stop 2024-03-30T00:48:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 -- 172.21.15.66:0/1444078828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedc80c8660 msgr2=0x7fedc80c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedc80c8660 0x7fedc80c8ac0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fedb001f1c0 tx=0x7fedac005800 comp rx=0 tx=0).stop 2024-03-30T00:48:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 -- 172.21.15.66:0/1444078828 shutdown_connections 2024-03-30T00:48:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fedc8150b20 0x7fedc8150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feda4066690 0x7feda4068b50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedc80c8660 0x7fedc80c8ac0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 --2- 172.21.15.66:0/1444078828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedac0047d0 0x7fedc80cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 -- 172.21.15.66:0/1444078828 >> 172.21.15.66:0/1444078828 conn(0x7fedc80d0540 msgr2=0x7fedc80ada30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 -- 172.21.15.66:0/1444078828 shutdown_connections 2024-03-30T00:48:49.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.725+0000 7fedcca00640 1 -- 172.21.15.66:0/1444078828 wait complete. 2024-03-30T00:48:49.754 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:48:49.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.885+0000 7f15dd600640 1 Processor -- start 2024-03-30T00:48:49.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.885+0000 7f15dd600640 1 -- start start 2024-03-30T00:48:49.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.885+0000 7f15dd600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d80cf2f0 0x7f15d806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.885+0000 7f15dd600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d80cb0e0 con 0x7f15d815ca80 2024-03-30T00:48:49.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.885+0000 7f15dd600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d80cf7f0 con 0x7f15d80cb570 2024-03-30T00:48:49.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.885+0000 7f15dd600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d80cfad0 con 0x7f15d80cf2f0 2024-03-30T00:48:49.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.885+0000 7f15d6a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f15d815ca80 0x7f15d806b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53928/0 (socket says 172.21.15.66:53928) 2024-03-30T00:48:49.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.885+0000 7f15d6a00640 1 -- 172.21.15.66:0/3940143176 learned_addr learned my addr 172.21.15.66:0/3940143176 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:49.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.889+0000 7f15d6000640 1 --2- 172.21.15.66:0/3940143176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d80cf2f0 0x7f15d806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.889+0000 7f15d5600640 1 -- 172.21.15.66:0/3940143176 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374646113 0 0) 0x7f15d80cb0e0 con 0x7f15d815ca80 2024-03-30T00:48:49.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.889+0000 7f15d5600640 1 -- 172.21.15.66:0/3940143176 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15bc003530 con 0x7f15d815ca80 2024-03-30T00:48:49.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.889+0000 7f15d6000640 1 -- 172.21.15.66:0/3940143176 >> v1:172.21.15.156:6789/0 conn(0x7f15d80cb570 legacy=0x7f15d80cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.889+0000 7f15d6000640 1 -- 172.21.15.66:0/3940143176 >> v1:172.21.15.66:6789/0 conn(0x7f15d815ca80 legacy=0x7f15d806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.889+0000 7f15d6000640 1 -- 172.21.15.66:0/3940143176 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d80cad70 con 0x7f15d80cf2f0 2024-03-30T00:48:49.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.889+0000 7f15d6000640 1 --2- 172.21.15.66:0/3940143176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d80cf2f0 0x7f15d806b7c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15c40097f0 tx=0x7f15c401ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd176bc244eccef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:49.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.889+0000 7f15d5600640 1 -- 172.21.15.66:0/3940143176 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c40a89d0 con 0x7f15d80cf2f0 2024-03-30T00:48:49.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.889+0000 7f15d5600640 1 -- 172.21.15.66:0/3940143176 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c40a8b30 con 0x7f15d80cf2f0 2024-03-30T00:48:49.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.889+0000 7f15d5600640 1 -- 172.21.15.66:0/3940143176 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c40a8ea0 con 0x7f15d80cf2f0 2024-03-30T00:48:49.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 -- 172.21.15.66:0/3940143176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d80cf2f0 msgr2=0x7f15d806b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 --2- 172.21.15.66:0/3940143176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d80cf2f0 0x7f15d806b7c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15c40097f0 tx=0x7f15c401ecd0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 -- 172.21.15.66:0/3940143176 shutdown_connections 2024-03-30T00:48:49.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 --2- 172.21.15.66:0/3940143176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d80cf2f0 0x7f15d806b7c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 -- 172.21.15.66:0/3940143176 >> 172.21.15.66:0/3940143176 conn(0x7f15d80b2090 msgr2=0x7f15d80b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:49.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 -- 172.21.15.66:0/3940143176 shutdown_connections 2024-03-30T00:48:49.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 -- 172.21.15.66:0/3940143176 wait complete. 2024-03-30T00:48:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 Processor -- start 2024-03-30T00:48:49.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 -- start start 2024-03-30T00:48:49.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15d80cb570 0x7f15d80b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15d80cf2f0 0x7f15d80c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d815ca80 0x7f15d814b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d80c9b10 con 0x7f15d80cb570 2024-03-30T00:48:49.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d814bf20 con 0x7f15d80cf2f0 2024-03-30T00:48:49.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15dd600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d80b5300 con 0x7f15d815ca80 2024-03-30T00:48:49.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15d80cb570 0x7f15d80b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15d80cb570 0x7f15d80b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41660/0 (socket says 172.21.15.66:41660) 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d6a00640 1 -- 172.21.15.66:0/4093869108 learned_addr learned my addr 172.21.15.66:0/4093869108 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d6000640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15d80cf2f0 0x7f15d80c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d7400640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d815ca80 0x7f15d814b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d6a00640 1 -- 172.21.15.66:0/4093869108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d815ca80 msgr2=0x7f15d814b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d6a00640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d815ca80 0x7f15d814b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d6a00640 1 -- 172.21.15.66:0/4093869108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15d80cf2f0 msgr2=0x7f15d80c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d6a00640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15d80cf2f0 0x7f15d80c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d6a00640 1 -- 172.21.15.66:0/4093869108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c4009420 con 0x7f15d80cb570 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d7400640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d815ca80 0x7f15d814b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.893+0000 7f15d6a00640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15d80cb570 0x7f15d80b3ba0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c420 tx=0x7f15c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.897+0000 7f15cea00640 1 -- 172.21.15.66:0/4093869108 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8005cf0 con 0x7f15d80cb570 2024-03-30T00:48:49.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.897+0000 7f15cea00640 1 -- 172.21.15.66:0/4093869108 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8005e50 con 0x7f15d80cb570 2024-03-30T00:48:49.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.897+0000 7f15cea00640 1 -- 172.21.15.66:0/4093869108 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c80072d0 con 0x7f15d80cb570 2024-03-30T00:48:49.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.897+0000 7f15dd600640 1 -- 172.21.15.66:0/4093869108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d80b4260 con 0x7f15d80cb570 2024-03-30T00:48:49.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.897+0000 7f15dd600640 1 -- 172.21.15.66:0/4093869108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d80b4820 con 0x7f15d80cb570 2024-03-30T00:48:49.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.897+0000 7f15dd600640 1 -- 172.21.15.66:0/4093869108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d8066c30 con 0x7f15d80cb570 2024-03-30T00:48:49.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.897+0000 7f15cea00640 1 -- 172.21.15.66:0/4093869108 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f15c8007430 con 0x7f15d80cb570 2024-03-30T00:48:49.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.897+0000 7f15cea00640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15b8066690 0x7f15b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:49.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.897+0000 7f15cea00640 1 -- 172.21.15.66:0/4093869108 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f15c8083880 con 0x7f15d80cb570 2024-03-30T00:48:49.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.897+0000 7f15d6000640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15b8066690 0x7f15b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:49.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.897+0000 7f15d6000640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15b8066690 0x7f15b8068b50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f15c40b2000 tx=0x7f15c40aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:49.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:49.901+0000 7f15cea00640 1 -- 172.21.15.66:0/4093869108 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f15c8050ac0 con 0x7f15d80cb570 2024-03-30T00:48:50.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.065+0000 7f15dd600640 1 -- 172.21.15.66:0/4093869108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15d809e270 con 0x7f15d80cb570 2024-03-30T00:48:50.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.065+0000 7f15cea00640 1 -- 172.21.15.66:0/4093869108 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16248 (secure 0 0 0) 0x7f15c8017090 con 0x7f15d80cb570 2024-03-30T00:48:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:50.075 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":110,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:48:47.334669+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:48:44.750350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-30T00:48:45.327928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:48:50.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 -- 172.21.15.66:0/4093869108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15b8066690 msgr2=0x7f15b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15b8066690 0x7f15b8068b50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f15c40b2000 tx=0x7f15c40aa070 comp rx=0 tx=0).stop 2024-03-30T00:48:50.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 -- 172.21.15.66:0/4093869108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15d80cb570 msgr2=0x7f15d80b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15d80cb570 0x7f15d80b3ba0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c420 tx=0x7f15c800c8e0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 -- 172.21.15.66:0/4093869108 shutdown_connections 2024-03-30T00:48:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15d815ca80 0x7f15d814b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15d80cf2f0 0x7f15d80c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f15b8066690 0x7f15b8068b50 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 --2- 172.21.15.66:0/4093869108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15d80cb570 0x7f15d80b3ba0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 -- 172.21.15.66:0/4093869108 >> 172.21.15.66:0/4093869108 conn(0x7f15d80b2090 msgr2=0x7f15d80ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 -- 172.21.15.66:0/4093869108 shutdown_connections 2024-03-30T00:48:50.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.069+0000 7f15dd600640 1 -- 172.21.15.66:0/4093869108 wait complete. 2024-03-30T00:48:50.100 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T00:48:50.100 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:48:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.229+0000 7fdf42800640 1 Processor -- start 2024-03-30T00:48:50.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.229+0000 7fdf42800640 1 -- start start 2024-03-30T00:48:50.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf42800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdf3c0cabb0 0x7fdf3c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:50.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf42800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c06d070 con 0x7fdf3c0ce520 2024-03-30T00:48:50.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf42800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c0cbba0 con 0x7fdf3c15a640 2024-03-30T00:48:50.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf42800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c0cb8c0 con 0x7fdf3c0cabb0 2024-03-30T00:48:50.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf33e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdf3c0ce520 0x7fdf3c0ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53930/0 (socket says 172.21.15.66:53930) 2024-03-30T00:48:50.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf33e00640 1 -- 172.21.15.66:0/271543536 learned_addr learned my addr 172.21.15.66:0/271543536 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:50.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf3be00640 1 --2- 172.21.15.66:0/271543536 >> v2:172.21.15.156:3300/0 conn(0x7fdf3c0cabb0 0x7fdf3c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:50.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf3aa00640 1 -- 172.21.15.66:0/271543536 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093831074 0 0) 0x7fdf3c06d070 con 0x7fdf3c0ce520 2024-03-30T00:48:50.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf3aa00640 1 -- 172.21.15.66:0/271543536 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf20003530 con 0x7fdf3c0ce520 2024-03-30T00:48:50.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf3aa00640 1 -- 172.21.15.66:0/271543536 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368148564 0 0) 0x7fdf3c0cbba0 con 0x7fdf3c15a640 2024-03-30T00:48:50.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf3aa00640 1 -- 172.21.15.66:0/271543536 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf3c06d070 con 0x7fdf3c15a640 2024-03-30T00:48:50.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf3be00640 1 -- 172.21.15.66:0/271543536 >> v1:172.21.15.156:6789/0 conn(0x7fdf3c15a640 legacy=0x7fdf3c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf3aa00640 1 -- 172.21.15.66:0/271543536 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1960847370 0 0) 0x7fdf20003530 con 0x7fdf3c0ce520 2024-03-30T00:48:50.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf3be00640 1 -- 172.21.15.66:0/271543536 >> v1:172.21.15.66:6789/0 conn(0x7fdf3c0ce520 legacy=0x7fdf3c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.233+0000 7fdf3be00640 1 -- 172.21.15.66:0/271543536 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c06d2a0 con 0x7fdf3c0cabb0 2024-03-30T00:48:50.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf3be00640 1 --2- 172.21.15.66:0/271543536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c0cabb0 0x7fdf3c0cafe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009d90 tx=0x7fdf3401c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=909e0d582e864352 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:50.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf3aa00640 1 -- 172.21.15.66:0/271543536 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24001680 con 0x7fdf3c0cabb0 2024-03-30T00:48:50.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf3aa00640 1 -- 172.21.15.66:0/271543536 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf340a56a0 con 0x7fdf3c0cabb0 2024-03-30T00:48:50.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf3aa00640 1 -- 172.21.15.66:0/271543536 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf340a5a30 con 0x7fdf3c0cabb0 2024-03-30T00:48:50.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf42800640 1 -- 172.21.15.66:0/271543536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c0cabb0 msgr2=0x7fdf3c0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf42800640 1 --2- 172.21.15.66:0/271543536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c0cabb0 0x7fdf3c0cafe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009d90 tx=0x7fdf3401c860 comp rx=0 tx=0).stop 2024-03-30T00:48:50.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf42800640 1 -- 172.21.15.66:0/271543536 shutdown_connections 2024-03-30T00:48:50.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf42800640 1 --2- 172.21.15.66:0/271543536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c0cabb0 0x7fdf3c0cafe0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf42800640 1 -- 172.21.15.66:0/271543536 >> 172.21.15.66:0/271543536 conn(0x7fdf3c069f60 msgr2=0x7fdf3c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:50.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf42800640 1 -- 172.21.15.66:0/271543536 shutdown_connections 2024-03-30T00:48:50.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf42800640 1 -- 172.21.15.66:0/271543536 wait complete. 2024-03-30T00:48:50.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf42800640 1 Processor -- start 2024-03-30T00:48:50.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.237+0000 7fdf42800640 1 -- start start 2024-03-30T00:48:50.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf42800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c0cabb0 0x7fdf3c09bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:50.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf42800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf3c0ce520 0x7fdf3c0bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:50.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf42800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3c15a640 0x7fdf3c15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:50.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf42800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0bf6e0 con 0x7fdf3c0ce520 2024-03-30T00:48:50.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf42800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0bedb0 con 0x7fdf3c0cabb0 2024-03-30T00:48:50.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf42800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0bf090 con 0x7fdf3c15a640 2024-03-30T00:48:50.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf3c0ce520 0x7fdf3c0bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:50.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf3c0ce520 0x7fdf3c0bfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41666/0 (socket says 172.21.15.66:41666) 2024-03-30T00:48:50.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33e00640 1 -- 172.21.15.66:0/496566933 learned_addr learned my addr 172.21.15.66:0/496566933 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf3b400640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c0cabb0 0x7fdf3c09bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf3be00640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3c15a640 0x7fdf3c15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33e00640 1 -- 172.21.15.66:0/496566933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3c15a640 msgr2=0x7fdf3c15cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33e00640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3c15a640 0x7fdf3c15cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33e00640 1 -- 172.21.15.66:0/496566933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c0cabb0 msgr2=0x7fdf3c09bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33e00640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c0cabb0 0x7fdf3c09bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33e00640 1 -- 172.21.15.66:0/496566933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34009a70 con 0x7fdf3c0ce520 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33e00640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf3c0ce520 0x7fdf3c0bfb70 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c780 tx=0x7fdf2400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33400640 1 -- 172.21.15.66:0/496566933 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24005cf0 con 0x7fdf3c0ce520 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33400640 1 -- 172.21.15.66:0/496566933 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf24005e50 con 0x7fdf3c0ce520 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf42800640 1 -- 172.21.15.66:0/496566933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c0bf370 con 0x7fdf3c0ce520 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33400640 1 -- 172.21.15.66:0/496566933 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf240072b0 con 0x7fdf3c0ce520 2024-03-30T00:48:50.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf42800640 1 -- 172.21.15.66:0/496566933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c0cca20 con 0x7fdf3c0ce520 2024-03-30T00:48:50.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf42800640 1 -- 172.21.15.66:0/496566933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c0ccca0 con 0x7fdf3c0ce520 2024-03-30T00:48:50.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.241+0000 7fdf33400640 1 -- 172.21.15.66:0/496566933 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdf24007410 con 0x7fdf3c0ce520 2024-03-30T00:48:50.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.245+0000 7fdf33400640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf180666c0 0x7fdf18068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:50.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.245+0000 7fdf33400640 1 -- 172.21.15.66:0/496566933 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdf24083a50 con 0x7fdf3c0ce520 2024-03-30T00:48:50.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.245+0000 7fdf3b400640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf180666c0 0x7fdf18068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:50.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.245+0000 7fdf3b400640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf180666c0 0x7fdf18068b80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0011f0 tx=0x7fdf2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:50.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.249+0000 7fdf33400640 1 -- 172.21.15.66:0/496566933 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdf24054700 con 0x7fdf3c0ce520 2024-03-30T00:48:50.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.405+0000 7fdf42800640 1 -- 172.21.15.66:0/496566933 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf3c0ab820 con 0x7fdf180666c0 2024-03-30T00:48:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.413+0000 7fdf33400640 1 -- 172.21.15.66:0/496566933 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289924 (secure 0 0 0) 0x7fdf2c0050d0 con 0x7fdf180666c0 2024-03-30T00:48:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:50.431 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:48:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 -- 172.21.15.66:0/496566933 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf180666c0 msgr2=0x7fdf18068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf180666c0 0x7fdf18068b80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0011f0 tx=0x7fdf2c009040 comp rx=0 tx=0).stop 2024-03-30T00:48:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 -- 172.21.15.66:0/496566933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf3c0ce520 msgr2=0x7fdf3c0bfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf3c0ce520 0x7fdf3c0bfb70 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c780 tx=0x7fdf2400cc40 comp rx=0 tx=0).stop 2024-03-30T00:48:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 -- 172.21.15.66:0/496566933 shutdown_connections 2024-03-30T00:48:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3c15a640 0x7fdf3c15cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf180666c0 0x7fdf18068b80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf3c0ce520 0x7fdf3c0bfb70 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 --2- 172.21.15.66:0/496566933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3c0cabb0 0x7fdf3c09bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 -- 172.21.15.66:0/496566933 >> 172.21.15.66:0/496566933 conn(0x7fdf3c069f60 msgr2=0x7fdf3c0730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:50.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 -- 172.21.15.66:0/496566933 shutdown_connections 2024-03-30T00:48:50.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.417+0000 7fdf42800640 1 -- 172.21.15.66:0/496566933 wait complete. 2024-03-30T00:48:50.447 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-03-30T00:48:48.971981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260756,"kb_used_data":11796,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754713964,"statfs":{"total":773094113280,"available":772827099136,"internally_reserved":0,"allocated":12079104,"data_stored":6162352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.046466"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342740+0000","last_change":"2024-03-30T00:48:46.358839+0000","last_active":"2024-03-30T00:48:47.342740+0000","last_peered":"2024-03-30T00:48:47.342740+0000","last_clean":"2024-03-30T00:48:47.342740+0000","last_became_active":"2024-03-30T00:48:46.358440+0000","last_became_peered":"2024-03-30T00:48:46.358440+0000","last_unstale":"2024-03-30T00:48:47.342740+0000","last_undegraded":"2024-03-30T00:48:47.342740+0000","last_fullsized":"2024-03-30T00:48:47.342740+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:56:27.359011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.364515+0000","last_change":"2024-03-30T00:48:46.360649+0000","last_active":"2024-03-30T00:48:46.364515+0000","last_peered":"2024-03-30T00:48:46.364515+0000","last_clean":"2024-03-30T00:48:46.364515+0000","last_became_active":"2024-03-30T00:48:46.360379+0000","last_became_peered":"2024-03-30T00:48:46.360379+0000","last_unstale":"2024-03-30T00:48:46.364515+0000","last_undegraded":"2024-03-30T00:48:46.364515+0000","last_fullsized":"2024-03-30T00:48:46.364515+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:02:13.552134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.343082+0000","last_change":"2024-03-30T00:48:46.361524+0000","last_active":"2024-03-30T00:48:47.343082+0000","last_peered":"2024-03-30T00:48:47.343082+0000","last_clean":"2024-03-30T00:48:47.343082+0000","last_became_active":"2024-03-30T00:48:46.360112+0000","last_became_peered":"2024-03-30T00:48:46.360112+0000","last_unstale":"2024-03-30T00:48:47.343082+0000","last_undegraded":"2024-03-30T00:48:47.343082+0000","last_fullsized":"2024-03-30T00:48:47.343082+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:36:34.824322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.361282+0000","last_change":"2024-03-30T00:48:46.354072+0000","last_active":"2024-03-30T00:48:46.361282+0000","last_peered":"2024-03-30T00:48:46.361282+0000","last_clean":"2024-03-30T00:48:46.361282+0000","last_became_active":"2024-03-30T00:48:46.353785+0000","last_became_peered":"2024-03-30T00:48:46.353785+0000","last_unstale":"2024-03-30T00:48:46.361282+0000","last_undegraded":"2024-03-30T00:48:46.361282+0000","last_fullsized":"2024-03-30T00:48:46.361282+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:52:19.120171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.374217+0000","last_change":"2024-03-30T00:48:46.372058+0000","last_active":"2024-03-30T00:48:46.374217+0000","last_peered":"2024-03-30T00:48:46.374217+0000","last_clean":"2024-03-30T00:48:46.374217+0000","last_became_active":"2024-03-30T00:48:46.371692+0000","last_became_peered":"2024-03-30T00:48:46.371692+0000","last_unstale":"2024-03-30T00:48:46.374217+0000","last_undegraded":"2024-03-30T00:48:46.374217+0000","last_fullsized":"2024-03-30T00:48:46.374217+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:22:56.906794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.372179+0000","last_change":"2024-03-30T00:48:46.370835+0000","last_active":"2024-03-30T00:48:46.372179+0000","last_peered":"2024-03-30T00:48:46.372179+0000","last_clean":"2024-03-30T00:48:46.372179+0000","last_became_active":"2024-03-30T00:48:46.370405+0000","last_became_peered":"2024-03-30T00:48:46.370405+0000","last_unstale":"2024-03-30T00:48:46.372179+0000","last_undegraded":"2024-03-30T00:48:46.372179+0000","last_fullsized":"2024-03-30T00:48:46.372179+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:56:24.032619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371083+0000","last_change":"2024-03-30T00:48:46.365333+0000","last_active":"2024-03-30T00:48:46.371083+0000","last_peered":"2024-03-30T00:48:46.371083+0000","last_clean":"2024-03-30T00:48:46.371083+0000","last_became_active":"2024-03-30T00:48:46.365032+0000","last_became_peered":"2024-03-30T00:48:46.365032+0000","last_unstale":"2024-03-30T00:48:46.371083+0000","last_undegraded":"2024-03-30T00:48:46.371083+0000","last_fullsized":"2024-03-30T00:48:46.371083+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:33:41.569588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.341943+0000","last_change":"2024-03-30T00:48:46.366483+0000","last_active":"2024-03-30T00:48:47.341943+0000","last_peered":"2024-03-30T00:48:47.341943+0000","last_clean":"2024-03-30T00:48:47.341943+0000","last_became_active":"2024-03-30T00:48:46.366059+0000","last_became_peered":"2024-03-30T00:48:46.366059+0000","last_unstale":"2024-03-30T00:48:47.341943+0000","last_undegraded":"2024-03-30T00:48:47.341943+0000","last_fullsized":"2024-03-30T00:48:47.341943+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:56:14.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371420+0000","last_change":"2024-03-30T00:48:46.366380+0000","last_active":"2024-03-30T00:48:46.371420+0000","last_peered":"2024-03-30T00:48:46.371420+0000","last_clean":"2024-03-30T00:48:46.371420+0000","last_became_active":"2024-03-30T00:48:46.365280+0000","last_became_peered":"2024-03-30T00:48:46.365280+0000","last_unstale":"2024-03-30T00:48:46.371420+0000","last_undegraded":"2024-03-30T00:48:46.371420+0000","last_fullsized":"2024-03-30T00:48:46.371420+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:00.738587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.370142+0000","last_change":"2024-03-30T00:48:46.363288+0000","last_active":"2024-03-30T00:48:46.370142+0000","last_peered":"2024-03-30T00:48:46.370142+0000","last_clean":"2024-03-30T00:48:46.370142+0000","last_became_active":"2024-03-30T00:48:46.363052+0000","last_became_peered":"2024-03-30T00:48:46.363052+0000","last_unstale":"2024-03-30T00:48:46.370142+0000","last_undegraded":"2024-03-30T00:48:46.370142+0000","last_fullsized":"2024-03-30T00:48:46.370142+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:42.396870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368575+0000","last_change":"2024-03-30T00:48:46.363747+0000","last_active":"2024-03-30T00:48:46.368575+0000","last_peered":"2024-03-30T00:48:46.368575+0000","last_clean":"2024-03-30T00:48:46.368575+0000","last_became_active":"2024-03-30T00:48:46.363492+0000","last_became_peered":"2024-03-30T00:48:46.363492+0000","last_unstale":"2024-03-30T00:48:46.368575+0000","last_undegraded":"2024-03-30T00:48:46.368575+0000","last_fullsized":"2024-03-30T00:48:46.368575+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:42.926851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.365745+0000","last_change":"2024-03-30T00:48:46.361393+0000","last_active":"2024-03-30T00:48:46.365745+0000","last_peered":"2024-03-30T00:48:46.365745+0000","last_clean":"2024-03-30T00:48:46.365745+0000","last_became_active":"2024-03-30T00:48:46.361124+0000","last_became_peered":"2024-03-30T00:48:46.361124+0000","last_unstale":"2024-03-30T00:48:46.365745+0000","last_undegraded":"2024-03-30T00:48:46.365745+0000","last_fullsized":"2024-03-30T00:48:46.365745+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:21:09.576366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.359720+0000","last_change":"2024-03-30T00:48:46.353587+0000","last_active":"2024-03-30T00:48:46.359720+0000","last_peered":"2024-03-30T00:48:46.359720+0000","last_clean":"2024-03-30T00:48:46.359720+0000","last_became_active":"2024-03-30T00:48:46.353285+0000","last_became_peered":"2024-03-30T00:48:46.353285+0000","last_unstale":"2024-03-30T00:48:46.359720+0000","last_undegraded":"2024-03-30T00:48:46.359720+0000","last_fullsized":"2024-03-30T00:48:46.359720+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:59:46.888843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345533+0000","last_change":"2024-03-30T00:48:46.356726+0000","last_active":"2024-03-30T00:48:47.345533+0000","last_peered":"2024-03-30T00:48:47.345533+0000","last_clean":"2024-03-30T00:48:47.345533+0000","last_became_active":"2024-03-30T00:48:46.356457+0000","last_became_peered":"2024-03-30T00:48:46.356457+0000","last_unstale":"2024-03-30T00:48:47.345533+0000","last_undegraded":"2024-03-30T00:48:47.345533+0000","last_fullsized":"2024-03-30T00:48:47.345533+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:31:43.779593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345432+0000","last_change":"2024-03-30T00:48:46.353250+0000","last_active":"2024-03-30T00:48:47.345432+0000","last_peered":"2024-03-30T00:48:47.345432+0000","last_clean":"2024-03-30T00:48:47.345432+0000","last_became_active":"2024-03-30T00:48:46.352913+0000","last_became_peered":"2024-03-30T00:48:46.352913+0000","last_unstale":"2024-03-30T00:48:47.345432+0000","last_undegraded":"2024-03-30T00:48:47.345432+0000","last_fullsized":"2024-03-30T00:48:47.345432+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:21:19.264827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345480+0000","last_change":"2024-03-30T00:48:46.358038+0000","last_active":"2024-03-30T00:48:47.345480+0000","last_peered":"2024-03-30T00:48:47.345480+0000","last_clean":"2024-03-30T00:48:47.345480+0000","last_became_active":"2024-03-30T00:48:46.355461+0000","last_became_peered":"2024-03-30T00:48:46.355461+0000","last_unstale":"2024-03-30T00:48:47.345480+0000","last_undegraded":"2024-03-30T00:48:47.345480+0000","last_fullsized":"2024-03-30T00:48:47.345480+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:51.246897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342775+0000","last_change":"2024-03-30T00:48:46.360594+0000","last_active":"2024-03-30T00:48:47.342775+0000","last_peered":"2024-03-30T00:48:47.342775+0000","last_clean":"2024-03-30T00:48:47.342775+0000","last_became_active":"2024-03-30T00:48:46.360281+0000","last_became_peered":"2024-03-30T00:48:46.360281+0000","last_unstale":"2024-03-30T00:48:47.342775+0000","last_undegraded":"2024-03-30T00:48:47.342775+0000","last_fullsized":"2024-03-30T00:48:47.342775+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:53.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371253+0000","last_change":"2024-03-30T00:48:46.365562+0000","last_active":"2024-03-30T00:48:46.371253+0000","last_peered":"2024-03-30T00:48:46.371253+0000","last_clean":"2024-03-30T00:48:46.371253+0000","last_became_active":"2024-03-30T00:48:46.365242+0000","last_became_peered":"2024-03-30T00:48:46.365242+0000","last_unstale":"2024-03-30T00:48:46.371253+0000","last_undegraded":"2024-03-30T00:48:46.371253+0000","last_fullsized":"2024-03-30T00:48:46.371253+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:27:58.176108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.343170+0000","last_change":"2024-03-30T00:48:46.363232+0000","last_active":"2024-03-30T00:48:47.343170+0000","last_peered":"2024-03-30T00:48:47.343170+0000","last_clean":"2024-03-30T00:48:47.343170+0000","last_became_active":"2024-03-30T00:48:46.362978+0000","last_became_peered":"2024-03-30T00:48:46.362978+0000","last_unstale":"2024-03-30T00:48:47.343170+0000","last_undegraded":"2024-03-30T00:48:47.343170+0000","last_fullsized":"2024-03-30T00:48:47.343170+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:51:39.826307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.356379+0000","last_change":"2024-03-30T00:48:46.352196+0000","last_active":"2024-03-30T00:48:46.356379+0000","last_peered":"2024-03-30T00:48:46.356379+0000","last_clean":"2024-03-30T00:48:46.356379+0000","last_became_active":"2024-03-30T00:48:46.351902+0000","last_became_peered":"2024-03-30T00:48:46.351902+0000","last_unstale":"2024-03-30T00:48:46.356379+0000","last_undegraded":"2024-03-30T00:48:46.356379+0000","last_fullsized":"2024-03-30T00:48:46.356379+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:47:32.075977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342494+0000","last_change":"2024-03-30T00:48:46.350063+0000","last_active":"2024-03-30T00:48:47.342494+0000","last_peered":"2024-03-30T00:48:47.342494+0000","last_clean":"2024-03-30T00:48:47.342494+0000","last_became_active":"2024-03-30T00:48:46.349723+0000","last_became_peered":"2024-03-30T00:48:46.349723+0000","last_unstale":"2024-03-30T00:48:47.342494+0000","last_undegraded":"2024-03-30T00:48:47.342494+0000","last_fullsized":"2024-03-30T00:48:47.342494+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:40:02.486030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.374150+0000","last_change":"2024-03-30T00:48:46.371465+0000","last_active":"2024-03-30T00:48:46.374150+0000","last_peered":"2024-03-30T00:48:46.374150+0000","last_clean":"2024-03-30T00:48:46.374150+0000","last_became_active":"2024-03-30T00:48:46.371211+0000","last_became_peered":"2024-03-30T00:48:46.371211+0000","last_unstale":"2024-03-30T00:48:46.374150+0000","last_undegraded":"2024-03-30T00:48:46.374150+0000","last_fullsized":"2024-03-30T00:48:46.374150+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:07.947403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342283+0000","last_change":"2024-03-30T00:48:46.360790+0000","last_active":"2024-03-30T00:48:47.342283+0000","last_peered":"2024-03-30T00:48:47.342283+0000","last_clean":"2024-03-30T00:48:47.342283+0000","last_became_active":"2024-03-30T00:48:46.360319+0000","last_became_peered":"2024-03-30T00:48:46.360319+0000","last_unstale":"2024-03-30T00:48:47.342283+0000","last_undegraded":"2024-03-30T00:48:47.342283+0000","last_fullsized":"2024-03-30T00:48:47.342283+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:07:50.101198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.365321+0000","last_change":"2024-03-30T00:48:46.357688+0000","last_active":"2024-03-30T00:48:46.365321+0000","last_peered":"2024-03-30T00:48:46.365321+0000","last_clean":"2024-03-30T00:48:46.365321+0000","last_became_active":"2024-03-30T00:48:46.357354+0000","last_became_peered":"2024-03-30T00:48:46.357354+0000","last_unstale":"2024-03-30T00:48:46.365321+0000","last_undegraded":"2024-03-30T00:48:46.365321+0000","last_fullsized":"2024-03-30T00:48:46.365321+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:27.485835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368002+0000","last_change":"2024-03-30T00:48:46.360387+0000","last_active":"2024-03-30T00:48:46.368002+0000","last_peered":"2024-03-30T00:48:46.368002+0000","last_clean":"2024-03-30T00:48:46.368002+0000","last_became_active":"2024-03-30T00:48:46.358825+0000","last_became_peered":"2024-03-30T00:48:46.358825+0000","last_unstale":"2024-03-30T00:48:46.368002+0000","last_undegraded":"2024-03-30T00:48:46.368002+0000","last_fullsized":"2024-03-30T00:48:46.368002+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:30:00.322906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.356689+0000","last_change":"2024-03-30T00:48:46.352467+0000","last_active":"2024-03-30T00:48:46.356689+0000","last_peered":"2024-03-30T00:48:46.356689+0000","last_clean":"2024-03-30T00:48:46.356689+0000","last_became_active":"2024-03-30T00:48:46.352111+0000","last_became_peered":"2024-03-30T00:48:46.352111+0000","last_unstale":"2024-03-30T00:48:46.356689+0000","last_undegraded":"2024-03-30T00:48:46.356689+0000","last_fullsized":"2024-03-30T00:48:46.356689+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:59:46.888843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368538+0000","last_change":"2024-03-30T00:48:46.363230+0000","last_active":"2024-03-30T00:48:46.368538+0000","last_peered":"2024-03-30T00:48:46.368538+0000","last_clean":"2024-03-30T00:48:46.368538+0000","last_became_active":"2024-03-30T00:48:46.361934+0000","last_became_peered":"2024-03-30T00:48:46.361934+0000","last_unstale":"2024-03-30T00:48:46.368538+0000","last_undegraded":"2024-03-30T00:48:46.368538+0000","last_fullsized":"2024-03-30T00:48:46.368538+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:21:19.264827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371469+0000","last_change":"2024-03-30T00:48:46.365628+0000","last_active":"2024-03-30T00:48:46.371469+0000","last_peered":"2024-03-30T00:48:46.371469+0000","last_clean":"2024-03-30T00:48:46.371469+0000","last_became_active":"2024-03-30T00:48:46.363911+0000","last_became_peered":"2024-03-30T00:48:46.363911+0000","last_unstale":"2024-03-30T00:48:46.371469+0000","last_undegraded":"2024-03-30T00:48:46.371469+0000","last_fullsized":"2024-03-30T00:48:46.371469+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:35:26.863153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342530+0000","last_change":"2024-03-30T00:48:46.344665+0000","last_active":"2024-03-30T00:48:47.342530+0000","last_peered":"2024-03-30T00:48:47.342530+0000","last_clean":"2024-03-30T00:48:47.342530+0000","last_became_active":"2024-03-30T00:48:46.344351+0000","last_became_peered":"2024-03-30T00:48:46.344351+0000","last_unstale":"2024-03-30T00:48:47.342530+0000","last_undegraded":"2024-03-30T00:48:47.342530+0000","last_fullsized":"2024-03-30T00:48:47.342530+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:57:00.091791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.353817+0000","last_change":"2024-03-30T00:48:46.350224+0000","last_active":"2024-03-30T00:48:46.353817+0000","last_peered":"2024-03-30T00:48:46.353817+0000","last_clean":"2024-03-30T00:48:46.353817+0000","last_became_active":"2024-03-30T00:48:46.349951+0000","last_became_peered":"2024-03-30T00:48:46.349951+0000","last_unstale":"2024-03-30T00:48:46.353817+0000","last_undegraded":"2024-03-30T00:48:46.353817+0000","last_fullsized":"2024-03-30T00:48:46.353817+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:00:21.826773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.363976+0000","last_change":"2024-03-30T00:48:46.358190+0000","last_active":"2024-03-30T00:48:46.363976+0000","last_peered":"2024-03-30T00:48:46.363976+0000","last_clean":"2024-03-30T00:48:46.363976+0000","last_became_active":"2024-03-30T00:48:46.357848+0000","last_became_peered":"2024-03-30T00:48:46.357848+0000","last_unstale":"2024-03-30T00:48:46.363976+0000","last_undegraded":"2024-03-30T00:48:46.363976+0000","last_fullsized":"2024-03-30T00:48:46.363976+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:41:31.484548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342008+0000","last_change":"2024-03-30T00:48:46.350662+0000","last_active":"2024-03-30T00:48:47.342008+0000","last_peered":"2024-03-30T00:48:47.342008+0000","last_clean":"2024-03-30T00:48:47.342008+0000","last_became_active":"2024-03-30T00:48:46.346445+0000","last_became_peered":"2024-03-30T00:48:46.346445+0000","last_unstale":"2024-03-30T00:48:47.342008+0000","last_undegraded":"2024-03-30T00:48:47.342008+0000","last_fullsized":"2024-03-30T00:48:47.342008+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:05:38.458418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342299+0000","last_change":"2024-03-30T00:48:46.355746+0000","last_active":"2024-03-30T00:48:47.342299+0000","last_peered":"2024-03-30T00:48:47.342299+0000","last_clean":"2024-03-30T00:48:47.342299+0000","last_became_active":"2024-03-30T00:48:46.354935+0000","last_became_peered":"2024-03-30T00:48:46.354935+0000","last_unstale":"2024-03-30T00:48:47.342299+0000","last_undegraded":"2024-03-30T00:48:47.342299+0000","last_fullsized":"2024-03-30T00:48:47.342299+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:53:23.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"109'4","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.369164+0000","last_change":"2024-03-30T00:48:45.955252+0000","last_active":"2024-03-30T00:48:46.369164+0000","last_peered":"2024-03-30T00:48:46.369164+0000","last_clean":"2024-03-30T00:48:46.369164+0000","last_became_active":"2024-03-30T00:48:45.953941+0000","last_became_peered":"2024-03-30T00:48:45.953941+0000","last_unstale":"2024-03-30T00:48:46.369164+0000","last_undegraded":"2024-03-30T00:48:46.369164+0000","last_fullsized":"2024-03-30T00:48:46.369164+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:48.275866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342131+0000","last_change":"2024-03-30T00:48:46.361656+0000","last_active":"2024-03-30T00:48:47.342131+0000","last_peered":"2024-03-30T00:48:47.342131+0000","last_clean":"2024-03-30T00:48:47.342131+0000","last_became_active":"2024-03-30T00:48:46.361392+0000","last_became_peered":"2024-03-30T00:48:46.361392+0000","last_unstale":"2024-03-30T00:48:47.342131+0000","last_undegraded":"2024-03-30T00:48:47.342131+0000","last_fullsized":"2024-03-30T00:48:47.342131+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:39:10.093491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"109'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342895+0000","last_change":"2024-03-30T00:48:45.940875+0000","last_active":"2024-03-30T00:48:47.342895+0000","last_peered":"2024-03-30T00:48:47.342895+0000","last_clean":"2024-03-30T00:48:47.342895+0000","last_became_active":"2024-03-30T00:48:45.940525+0000","last_became_peered":"2024-03-30T00:48:45.940525+0000","last_unstale":"2024-03-30T00:48:47.342895+0000","last_undegraded":"2024-03-30T00:48:47.342895+0000","last_fullsized":"2024-03-30T00:48:47.342895+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:25:11.691579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.375517+0000","last_change":"2024-03-30T00:48:46.374550+0000","last_active":"2024-03-30T00:48:46.375517+0000","last_peered":"2024-03-30T00:48:46.375517+0000","last_clean":"2024-03-30T00:48:46.375517+0000","last_became_active":"2024-03-30T00:48:46.374271+0000","last_became_peered":"2024-03-30T00:48:46.374271+0000","last_unstale":"2024-03-30T00:48:46.375517+0000","last_undegraded":"2024-03-30T00:48:46.375517+0000","last_fullsized":"2024-03-30T00:48:46.375517+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:20:15.867799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.369295+0000","last_change":"2024-03-30T00:48:45.958340+0000","last_active":"2024-03-30T00:48:46.369295+0000","last_peered":"2024-03-30T00:48:46.369295+0000","last_clean":"2024-03-30T00:48:46.369295+0000","last_became_active":"2024-03-30T00:48:45.958069+0000","last_became_peered":"2024-03-30T00:48:45.958069+0000","last_unstale":"2024-03-30T00:48:46.369295+0000","last_undegraded":"2024-03-30T00:48:46.369295+0000","last_fullsized":"2024-03-30T00:48:46.369295+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:41.392529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.363196+0000","last_change":"2024-03-30T00:48:46.353746+0000","last_active":"2024-03-30T00:48:46.363196+0000","last_peered":"2024-03-30T00:48:46.363196+0000","last_clean":"2024-03-30T00:48:46.363196+0000","last_became_active":"2024-03-30T00:48:46.353515+0000","last_became_peered":"2024-03-30T00:48:46.353515+0000","last_unstale":"2024-03-30T00:48:46.363196+0000","last_undegraded":"2024-03-30T00:48:46.363196+0000","last_fullsized":"2024-03-30T00:48:46.363196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:41:06.772377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.372229+0000","last_change":"2024-03-30T00:48:45.952866+0000","last_active":"2024-03-30T00:48:46.372229+0000","last_peered":"2024-03-30T00:48:46.372229+0000","last_clean":"2024-03-30T00:48:46.372229+0000","last_became_active":"2024-03-30T00:48:45.952622+0000","last_became_peered":"2024-03-30T00:48:45.952622+0000","last_unstale":"2024-03-30T00:48:46.372229+0000","last_undegraded":"2024-03-30T00:48:46.372229+0000","last_fullsized":"2024-03-30T00:48:46.372229+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:00:59.734245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371947+0000","last_change":"2024-03-30T00:48:46.370357+0000","last_active":"2024-03-30T00:48:46.371947+0000","last_peered":"2024-03-30T00:48:46.371947+0000","last_clean":"2024-03-30T00:48:46.371947+0000","last_became_active":"2024-03-30T00:48:46.370070+0000","last_became_peered":"2024-03-30T00:48:46.370070+0000","last_unstale":"2024-03-30T00:48:46.371947+0000","last_undegraded":"2024-03-30T00:48:46.371947+0000","last_fullsized":"2024-03-30T00:48:46.371947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:53:11.764633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"109'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345561+0000","last_change":"2024-03-30T00:48:45.949823+0000","last_active":"2024-03-30T00:48:47.345561+0000","last_peered":"2024-03-30T00:48:47.345561+0000","last_clean":"2024-03-30T00:48:47.345561+0000","last_became_active":"2024-03-30T00:48:45.949208+0000","last_became_peered":"2024-03-30T00:48:45.949208+0000","last_unstale":"2024-03-30T00:48:47.345561+0000","last_undegraded":"2024-03-30T00:48:47.345561+0000","last_fullsized":"2024-03-30T00:48:47.345561+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:41:30.480206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.343332+0000","last_change":"2024-03-30T00:48:46.355259+0000","last_active":"2024-03-30T00:48:47.343332+0000","last_peered":"2024-03-30T00:48:47.343332+0000","last_clean":"2024-03-30T00:48:47.343332+0000","last_became_active":"2024-03-30T00:48:46.354653+0000","last_became_peered":"2024-03-30T00:48:46.354653+0000","last_unstale":"2024-03-30T00:48:47.343332+0000","last_undegraded":"2024-03-30T00:48:47.343332+0000","last_fullsized":"2024-03-30T00:48:47.343332+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:39:33.512770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.369393+0000","last_change":"2024-03-30T00:48:45.955849+0000","last_active":"2024-03-30T00:48:46.369393+0000","last_peered":"2024-03-30T00:48:46.369393+0000","last_clean":"2024-03-30T00:48:46.369393+0000","last_became_active":"2024-03-30T00:48:45.955597+0000","last_became_peered":"2024-03-30T00:48:45.955597+0000","last_unstale":"2024-03-30T00:48:46.369393+0000","last_undegraded":"2024-03-30T00:48:46.369393+0000","last_fullsized":"2024-03-30T00:48:46.369393+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:41:18.345346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368415+0000","last_change":"2024-03-30T00:48:46.364040+0000","last_active":"2024-03-30T00:48:46.368415+0000","last_peered":"2024-03-30T00:48:46.368415+0000","last_clean":"2024-03-30T00:48:46.368415+0000","last_became_active":"2024-03-30T00:48:46.363451+0000","last_became_peered":"2024-03-30T00:48:46.363451+0000","last_unstale":"2024-03-30T00:48:46.368415+0000","last_undegraded":"2024-03-30T00:48:46.368415+0000","last_fullsized":"2024-03-30T00:48:46.368415+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:41:19.349688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.343220+0000","last_change":"2024-03-30T00:48:45.960360+0000","last_active":"2024-03-30T00:48:46.343220+0000","last_peered":"2024-03-30T00:48:46.343220+0000","last_clean":"2024-03-30T00:48:46.343220+0000","last_became_active":"2024-03-30T00:48:45.960023+0000","last_became_peered":"2024-03-30T00:48:45.960023+0000","last_unstale":"2024-03-30T00:48:46.343220+0000","last_undegraded":"2024-03-30T00:48:46.343220+0000","last_fullsized":"2024-03-30T00:48:46.343220+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:26.481493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.361451+0000","last_change":"2024-03-30T00:48:46.356891+0000","last_active":"2024-03-30T00:48:46.361451+0000","last_peered":"2024-03-30T00:48:46.361451+0000","last_clean":"2024-03-30T00:48:46.361451+0000","last_became_active":"2024-03-30T00:48:46.356641+0000","last_became_peered":"2024-03-30T00:48:46.356641+0000","last_unstale":"2024-03-30T00:48:46.361451+0000","last_undegraded":"2024-03-30T00:48:46.361451+0000","last_fullsized":"2024-03-30T00:48:46.361451+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:21:09.576366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.345824+0000","last_change":"2024-03-30T00:48:45.950404+0000","last_active":"2024-03-30T00:48:46.345824+0000","last_peered":"2024-03-30T00:48:46.345824+0000","last_clean":"2024-03-30T00:48:46.345824+0000","last_became_active":"2024-03-30T00:48:45.950148+0000","last_became_peered":"2024-03-30T00:48:45.950148+0000","last_unstale":"2024-03-30T00:48:46.345824+0000","last_undegraded":"2024-03-30T00:48:46.345824+0000","last_fullsized":"2024-03-30T00:48:46.345824+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:26.481493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342215+0000","last_change":"2024-03-30T00:48:46.364449+0000","last_active":"2024-03-30T00:48:47.342215+0000","last_peered":"2024-03-30T00:48:47.342215+0000","last_clean":"2024-03-30T00:48:47.342215+0000","last_became_active":"2024-03-30T00:48:46.363266+0000","last_became_peered":"2024-03-30T00:48:46.363266+0000","last_unstale":"2024-03-30T00:48:47.342215+0000","last_undegraded":"2024-03-30T00:48:47.342215+0000","last_fullsized":"2024-03-30T00:48:47.342215+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:44:31.304932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.343889+0000","last_change":"2024-03-30T00:48:45.958138+0000","last_active":"2024-03-30T00:48:46.343889+0000","last_peered":"2024-03-30T00:48:46.343889+0000","last_clean":"2024-03-30T00:48:46.343889+0000","last_became_active":"2024-03-30T00:48:45.957832+0000","last_became_peered":"2024-03-30T00:48:45.957832+0000","last_unstale":"2024-03-30T00:48:46.343889+0000","last_undegraded":"2024-03-30T00:48:46.343889+0000","last_fullsized":"2024-03-30T00:48:46.343889+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:23:44.224176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.370227+0000","last_change":"2024-03-30T00:48:46.358789+0000","last_active":"2024-03-30T00:48:46.370227+0000","last_peered":"2024-03-30T00:48:46.370227+0000","last_clean":"2024-03-30T00:48:46.370227+0000","last_became_active":"2024-03-30T00:48:46.358535+0000","last_became_peered":"2024-03-30T00:48:46.358535+0000","last_unstale":"2024-03-30T00:48:46.370227+0000","last_undegraded":"2024-03-30T00:48:46.370227+0000","last_fullsized":"2024-03-30T00:48:46.370227+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:49.280208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342681+0000","last_change":"2024-03-30T00:48:45.954446+0000","last_active":"2024-03-30T00:48:47.342681+0000","last_peered":"2024-03-30T00:48:47.342681+0000","last_clean":"2024-03-30T00:48:47.342681+0000","last_became_active":"2024-03-30T00:48:45.954156+0000","last_became_peered":"2024-03-30T00:48:45.954156+0000","last_unstale":"2024-03-30T00:48:47.342681+0000","last_undegraded":"2024-03-30T00:48:47.342681+0000","last_fullsized":"2024-03-30T00:48:47.342681+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:54:39.885925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342311+0000","last_change":"2024-03-30T00:48:46.370061+0000","last_active":"2024-03-30T00:48:47.342311+0000","last_peered":"2024-03-30T00:48:47.342311+0000","last_clean":"2024-03-30T00:48:47.342311+0000","last_became_active":"2024-03-30T00:48:46.369658+0000","last_became_peered":"2024-03-30T00:48:46.369658+0000","last_unstale":"2024-03-30T00:48:47.342311+0000","last_undegraded":"2024-03-30T00:48:47.342311+0000","last_fullsized":"2024-03-30T00:48:47.342311+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:50.412709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"109'3","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345658+0000","last_change":"2024-03-30T00:48:45.957152+0000","last_active":"2024-03-30T00:48:47.345658+0000","last_peered":"2024-03-30T00:48:47.345658+0000","last_clean":"2024-03-30T00:48:47.345658+0000","last_became_active":"2024-03-30T00:48:45.956742+0000","last_became_peered":"2024-03-30T00:48:45.956742+0000","last_unstale":"2024-03-30T00:48:47.345658+0000","last_undegraded":"2024-03-30T00:48:47.345658+0000","last_fullsized":"2024-03-30T00:48:47.345658+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:00:59.734245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.370808+0000","last_change":"2024-03-30T00:48:46.361007+0000","last_active":"2024-03-30T00:48:46.370808+0000","last_peered":"2024-03-30T00:48:46.370808+0000","last_clean":"2024-03-30T00:48:46.370808+0000","last_became_active":"2024-03-30T00:48:46.360730+0000","last_became_peered":"2024-03-30T00:48:46.360730+0000","last_unstale":"2024-03-30T00:48:46.370808+0000","last_undegraded":"2024-03-30T00:48:46.370808+0000","last_fullsized":"2024-03-30T00:48:46.370808+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:08:46.368598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"109'6","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.375813+0000","last_change":"2024-03-30T00:48:45.951301+0000","last_active":"2024-03-30T00:48:46.375813+0000","last_peered":"2024-03-30T00:48:46.375813+0000","last_clean":"2024-03-30T00:48:46.375813+0000","last_became_active":"2024-03-30T00:48:45.951020+0000","last_became_peered":"2024-03-30T00:48:45.951020+0000","last_unstale":"2024-03-30T00:48:46.375813+0000","last_undegraded":"2024-03-30T00:48:46.375813+0000","last_fullsized":"2024-03-30T00:48:46.375813+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:16:40.272043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368414+0000","last_change":"2024-03-30T00:48:46.359956+0000","last_active":"2024-03-30T00:48:46.368414+0000","last_peered":"2024-03-30T00:48:46.368414+0000","last_clean":"2024-03-30T00:48:46.368414+0000","last_became_active":"2024-03-30T00:48:46.359701+0000","last_became_peered":"2024-03-30T00:48:46.359701+0000","last_unstale":"2024-03-30T00:48:46.368414+0000","last_undegraded":"2024-03-30T00:48:46.368414+0000","last_fullsized":"2024-03-30T00:48:46.368414+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:14.615490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.374492+0000","last_change":"2024-03-30T00:48:45.952608+0000","last_active":"2024-03-30T00:48:46.374492+0000","last_peered":"2024-03-30T00:48:46.374492+0000","last_clean":"2024-03-30T00:48:46.374492+0000","last_became_active":"2024-03-30T00:48:45.952171+0000","last_became_peered":"2024-03-30T00:48:45.952171+0000","last_unstale":"2024-03-30T00:48:46.374492+0000","last_undegraded":"2024-03-30T00:48:46.374492+0000","last_fullsized":"2024-03-30T00:48:46.374492+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:27:57.171766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371056+0000","last_change":"2024-03-30T00:48:46.364718+0000","last_active":"2024-03-30T00:48:46.371056+0000","last_peered":"2024-03-30T00:48:46.371056+0000","last_clean":"2024-03-30T00:48:46.371056+0000","last_became_active":"2024-03-30T00:48:46.364407+0000","last_became_peered":"2024-03-30T00:48:46.364407+0000","last_unstale":"2024-03-30T00:48:46.371056+0000","last_undegraded":"2024-03-30T00:48:46.371056+0000","last_fullsized":"2024-03-30T00:48:46.371056+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:43.737434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.346753+0000","last_change":"2024-03-30T00:48:45.954635+0000","last_active":"2024-03-30T00:48:46.346753+0000","last_peered":"2024-03-30T00:48:46.346753+0000","last_clean":"2024-03-30T00:48:46.346753+0000","last_became_active":"2024-03-30T00:48:45.954388+0000","last_became_peered":"2024-03-30T00:48:45.954388+0000","last_unstale":"2024-03-30T00:48:46.346753+0000","last_undegraded":"2024-03-30T00:48:46.346753+0000","last_fullsized":"2024-03-30T00:48:46.346753+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:22:55.902452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342876+0000","last_change":"2024-03-30T00:48:46.351573+0000","last_active":"2024-03-30T00:48:47.342876+0000","last_peered":"2024-03-30T00:48:47.342876+0000","last_clean":"2024-03-30T00:48:47.342876+0000","last_became_active":"2024-03-30T00:48:46.351284+0000","last_became_peered":"2024-03-30T00:48:46.351284+0000","last_unstale":"2024-03-30T00:48:47.342876+0000","last_undegraded":"2024-03-30T00:48:47.342876+0000","last_fullsized":"2024-03-30T00:48:47.342876+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:16:41.276384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342211+0000","last_change":"2024-03-30T00:48:45.949283+0000","last_active":"2024-03-30T00:48:47.342211+0000","last_peered":"2024-03-30T00:48:47.342211+0000","last_clean":"2024-03-30T00:48:47.342211+0000","last_became_active":"2024-03-30T00:48:45.949000+0000","last_became_peered":"2024-03-30T00:48:45.949000+0000","last_unstale":"2024-03-30T00:48:47.342211+0000","last_undegraded":"2024-03-30T00:48:47.342211+0000","last_fullsized":"2024-03-30T00:48:47.342211+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:43:23.706446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342807+0000","last_change":"2024-03-30T00:48:46.360602+0000","last_active":"2024-03-30T00:48:47.342807+0000","last_peered":"2024-03-30T00:48:47.342807+0000","last_clean":"2024-03-30T00:48:47.342807+0000","last_became_active":"2024-03-30T00:48:46.360326+0000","last_became_peered":"2024-03-30T00:48:46.360326+0000","last_unstale":"2024-03-30T00:48:47.342807+0000","last_undegraded":"2024-03-30T00:48:47.342807+0000","last_fullsized":"2024-03-30T00:48:47.342807+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:42:06.568226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"109'8","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368977+0000","last_change":"2024-03-30T00:48:45.960549+0000","last_active":"2024-03-30T00:48:46.368977+0000","last_peered":"2024-03-30T00:48:46.368977+0000","last_clean":"2024-03-30T00:48:46.368977+0000","last_became_active":"2024-03-30T00:48:45.960277+0000","last_became_peered":"2024-03-30T00:48:45.960277+0000","last_unstale":"2024-03-30T00:48:46.368977+0000","last_undegraded":"2024-03-30T00:48:46.368977+0000","last_fullsized":"2024-03-30T00:48:46.368977+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:22:55.902452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342036+0000","last_change":"2024-03-30T00:48:46.351031+0000","last_active":"2024-03-30T00:48:47.342036+0000","last_peered":"2024-03-30T00:48:47.342036+0000","last_clean":"2024-03-30T00:48:47.342036+0000","last_became_active":"2024-03-30T00:48:46.350744+0000","last_became_peered":"2024-03-30T00:48:46.350744+0000","last_unstale":"2024-03-30T00:48:47.342036+0000","last_undegraded":"2024-03-30T00:48:47.342036+0000","last_fullsized":"2024-03-30T00:48:47.342036+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:09.771227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371722+0000","last_change":"2024-03-30T00:48:45.958654+0000","last_active":"2024-03-30T00:48:46.371722+0000","last_peered":"2024-03-30T00:48:46.371722+0000","last_clean":"2024-03-30T00:48:46.371722+0000","last_became_active":"2024-03-30T00:48:45.958343+0000","last_became_peered":"2024-03-30T00:48:45.958343+0000","last_unstale":"2024-03-30T00:48:46.371722+0000","last_undegraded":"2024-03-30T00:48:46.371722+0000","last_fullsized":"2024-03-30T00:48:46.371722+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:40:01.481688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.365665+0000","last_change":"2024-03-30T00:48:46.358559+0000","last_active":"2024-03-30T00:48:46.365665+0000","last_peered":"2024-03-30T00:48:46.365665+0000","last_clean":"2024-03-30T00:48:46.365665+0000","last_became_active":"2024-03-30T00:48:46.358259+0000","last_became_peered":"2024-03-30T00:48:46.358259+0000","last_unstale":"2024-03-30T00:48:46.365665+0000","last_undegraded":"2024-03-30T00:48:46.365665+0000","last_fullsized":"2024-03-30T00:48:46.365665+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:32:36.057968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"109'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345695+0000","last_change":"2024-03-30T00:48:45.955648+0000","last_active":"2024-03-30T00:48:47.345695+0000","last_peered":"2024-03-30T00:48:47.345695+0000","last_clean":"2024-03-30T00:48:47.345695+0000","last_became_active":"2024-03-30T00:48:45.955355+0000","last_became_peered":"2024-03-30T00:48:45.955355+0000","last_unstale":"2024-03-30T00:48:47.345695+0000","last_undegraded":"2024-03-30T00:48:47.345695+0000","last_fullsized":"2024-03-30T00:48:47.345695+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:42.733092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.374331+0000","last_change":"2024-03-30T00:48:46.373105+0000","last_active":"2024-03-30T00:48:46.374331+0000","last_peered":"2024-03-30T00:48:46.374331+0000","last_clean":"2024-03-30T00:48:46.374331+0000","last_became_active":"2024-03-30T00:48:46.371847+0000","last_became_peered":"2024-03-30T00:48:46.371847+0000","last_unstale":"2024-03-30T00:48:46.374331+0000","last_undegraded":"2024-03-30T00:48:46.374331+0000","last_fullsized":"2024-03-30T00:48:46.374331+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:31:57.388123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.354065+0000","last_change":"2024-03-30T00:48:45.953601+0000","last_active":"2024-03-30T00:48:46.354065+0000","last_peered":"2024-03-30T00:48:46.354065+0000","last_clean":"2024-03-30T00:48:46.354065+0000","last_became_active":"2024-03-30T00:48:45.953320+0000","last_became_peered":"2024-03-30T00:48:45.953320+0000","last_unstale":"2024-03-30T00:48:46.354065+0000","last_undegraded":"2024-03-30T00:48:46.354065+0000","last_fullsized":"2024-03-30T00:48:46.354065+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:35:25.858812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342657+0000","last_change":"2024-03-30T00:48:46.358372+0000","last_active":"2024-03-30T00:48:47.342657+0000","last_peered":"2024-03-30T00:48:47.342657+0000","last_clean":"2024-03-30T00:48:47.342657+0000","last_became_active":"2024-03-30T00:48:46.358092+0000","last_became_peered":"2024-03-30T00:48:46.358092+0000","last_unstale":"2024-03-30T00:48:47.342657+0000","last_undegraded":"2024-03-30T00:48:47.342657+0000","last_fullsized":"2024-03-30T00:48:47.342657+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:39.685433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"109'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342578+0000","last_change":"2024-03-30T00:48:45.954514+0000","last_active":"2024-03-30T00:48:47.342578+0000","last_peered":"2024-03-30T00:48:47.342578+0000","last_clean":"2024-03-30T00:48:47.342578+0000","last_became_active":"2024-03-30T00:48:45.954272+0000","last_became_peered":"2024-03-30T00:48:45.954272+0000","last_unstale":"2024-03-30T00:48:47.342578+0000","last_undegraded":"2024-03-30T00:48:47.342578+0000","last_fullsized":"2024-03-30T00:48:47.342578+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:13:56.705237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.343251+0000","last_change":"2024-03-30T00:48:46.362004+0000","last_active":"2024-03-30T00:48:47.343251+0000","last_peered":"2024-03-30T00:48:47.343251+0000","last_clean":"2024-03-30T00:48:47.343251+0000","last_became_active":"2024-03-30T00:48:46.361778+0000","last_became_peered":"2024-03-30T00:48:46.361778+0000","last_unstale":"2024-03-30T00:48:47.343251+0000","last_undegraded":"2024-03-30T00:48:47.343251+0000","last_fullsized":"2024-03-30T00:48:47.343251+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:25:21.100530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342210+0000","last_change":"2024-03-30T00:48:45.954058+0000","last_active":"2024-03-30T00:48:47.342210+0000","last_peered":"2024-03-30T00:48:47.342210+0000","last_clean":"2024-03-30T00:48:47.342210+0000","last_became_active":"2024-03-30T00:48:45.953762+0000","last_became_peered":"2024-03-30T00:48:45.953762+0000","last_unstale":"2024-03-30T00:48:47.342210+0000","last_undegraded":"2024-03-30T00:48:47.342210+0000","last_fullsized":"2024-03-30T00:48:47.342210+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:03:20.122059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.361374+0000","last_change":"2024-03-30T00:48:46.355935+0000","last_active":"2024-03-30T00:48:46.361374+0000","last_peered":"2024-03-30T00:48:46.361374+0000","last_clean":"2024-03-30T00:48:46.361374+0000","last_became_active":"2024-03-30T00:48:46.355691+0000","last_became_peered":"2024-03-30T00:48:46.355691+0000","last_unstale":"2024-03-30T00:48:46.361374+0000","last_undegraded":"2024-03-30T00:48:46.361374+0000","last_fullsized":"2024-03-30T00:48:46.361374+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:47:32.075977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.346358+0000","last_change":"2024-03-30T00:48:45.955237+0000","last_active":"2024-03-30T00:48:46.346358+0000","last_peered":"2024-03-30T00:48:46.346358+0000","last_clean":"2024-03-30T00:48:46.346358+0000","last_became_active":"2024-03-30T00:48:45.954978+0000","last_became_peered":"2024-03-30T00:48:45.954978+0000","last_unstale":"2024-03-30T00:48:46.346358+0000","last_undegraded":"2024-03-30T00:48:46.346358+0000","last_fullsized":"2024-03-30T00:48:46.346358+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:05:37.454076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342909+0000","last_change":"2024-03-30T00:48:46.368396+0000","last_active":"2024-03-30T00:48:47.342909+0000","last_peered":"2024-03-30T00:48:47.342909+0000","last_clean":"2024-03-30T00:48:47.342909+0000","last_became_active":"2024-03-30T00:48:46.368118+0000","last_became_peered":"2024-03-30T00:48:46.368118+0000","last_unstale":"2024-03-30T00:48:47.342909+0000","last_undegraded":"2024-03-30T00:48:47.342909+0000","last_fullsized":"2024-03-30T00:48:47.342909+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:17.678809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342095+0000","last_change":"2024-03-30T00:48:45.953604+0000","last_active":"2024-03-30T00:48:47.342095+0000","last_peered":"2024-03-30T00:48:47.342095+0000","last_clean":"2024-03-30T00:48:47.342095+0000","last_became_active":"2024-03-30T00:48:45.953301+0000","last_became_peered":"2024-03-30T00:48:45.953301+0000","last_unstale":"2024-03-30T00:48:47.342095+0000","last_undegraded":"2024-03-30T00:48:47.342095+0000","last_fullsized":"2024-03-30T00:48:47.342095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:24:20.981909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342097+0000","last_change":"2024-03-30T00:48:46.358539+0000","last_active":"2024-03-30T00:48:47.342097+0000","last_peered":"2024-03-30T00:48:47.342097+0000","last_clean":"2024-03-30T00:48:47.342097+0000","last_became_active":"2024-03-30T00:48:46.358126+0000","last_became_peered":"2024-03-30T00:48:46.358126+0000","last_unstale":"2024-03-30T00:48:47.342097+0000","last_undegraded":"2024-03-30T00:48:47.342097+0000","last_fullsized":"2024-03-30T00:48:47.342097+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:36:35.379415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.373699+0000","last_change":"2024-03-30T00:48:45.959117+0000","last_active":"2024-03-30T00:48:46.373699+0000","last_peered":"2024-03-30T00:48:46.373699+0000","last_clean":"2024-03-30T00:48:46.373699+0000","last_became_active":"2024-03-30T00:48:45.958810+0000","last_became_peered":"2024-03-30T00:48:45.958810+0000","last_unstale":"2024-03-30T00:48:46.373699+0000","last_undegraded":"2024-03-30T00:48:46.373699+0000","last_fullsized":"2024-03-30T00:48:46.373699+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:33:40.565247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.351404+0000","last_change":"2024-03-30T00:48:46.349402+0000","last_active":"2024-03-30T00:48:46.351404+0000","last_peered":"2024-03-30T00:48:46.351404+0000","last_clean":"2024-03-30T00:48:46.351404+0000","last_became_active":"2024-03-30T00:48:46.349078+0000","last_became_peered":"2024-03-30T00:48:46.349078+0000","last_unstale":"2024-03-30T00:48:46.351404+0000","last_undegraded":"2024-03-30T00:48:46.351404+0000","last_fullsized":"2024-03-30T00:48:46.351404+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:04.678998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342654+0000","last_change":"2024-03-30T00:48:45.948993+0000","last_active":"2024-03-30T00:48:47.342654+0000","last_peered":"2024-03-30T00:48:47.342654+0000","last_clean":"2024-03-30T00:48:47.342654+0000","last_became_active":"2024-03-30T00:48:45.948576+0000","last_became_peered":"2024-03-30T00:48:45.948576+0000","last_unstale":"2024-03-30T00:48:47.342654+0000","last_undegraded":"2024-03-30T00:48:47.342654+0000","last_fullsized":"2024-03-30T00:48:47.342654+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:57:12.133595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.361217+0000","last_change":"2024-03-30T00:48:46.355243+0000","last_active":"2024-03-30T00:48:46.361217+0000","last_peered":"2024-03-30T00:48:46.361217+0000","last_clean":"2024-03-30T00:48:46.361217+0000","last_became_active":"2024-03-30T00:48:46.354924+0000","last_became_peered":"2024-03-30T00:48:46.354924+0000","last_unstale":"2024-03-30T00:48:46.361217+0000","last_undegraded":"2024-03-30T00:48:46.361217+0000","last_fullsized":"2024-03-30T00:48:46.361217+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:04.678998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.345825+0000","last_change":"2024-03-30T00:48:45.953819+0000","last_active":"2024-03-30T00:48:46.345825+0000","last_peered":"2024-03-30T00:48:46.345825+0000","last_clean":"2024-03-30T00:48:46.345825+0000","last_became_active":"2024-03-30T00:48:45.953546+0000","last_became_peered":"2024-03-30T00:48:45.953546+0000","last_unstale":"2024-03-30T00:48:46.345825+0000","last_undegraded":"2024-03-30T00:48:46.345825+0000","last_fullsized":"2024-03-30T00:48:46.345825+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:39:09.089149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368859+0000","last_change":"2024-03-30T00:48:46.366335+0000","last_active":"2024-03-30T00:48:46.368859+0000","last_peered":"2024-03-30T00:48:46.368859+0000","last_clean":"2024-03-30T00:48:46.368859+0000","last_became_active":"2024-03-30T00:48:46.363018+0000","last_became_peered":"2024-03-30T00:48:46.363018+0000","last_unstale":"2024-03-30T00:48:46.368859+0000","last_undegraded":"2024-03-30T00:48:46.368859+0000","last_fullsized":"2024-03-30T00:48:46.368859+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:07.947403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.349332+0000","last_change":"2024-03-30T00:48:45.948987+0000","last_active":"2024-03-30T00:48:46.349332+0000","last_peered":"2024-03-30T00:48:46.349332+0000","last_clean":"2024-03-30T00:48:46.349332+0000","last_became_active":"2024-03-30T00:48:45.948116+0000","last_became_peered":"2024-03-30T00:48:45.948116+0000","last_unstale":"2024-03-30T00:48:46.349332+0000","last_undegraded":"2024-03-30T00:48:46.349332+0000","last_fullsized":"2024-03-30T00:48:46.349332+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:09:13.900315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342589+0000","last_change":"2024-03-30T00:48:46.368207+0000","last_active":"2024-03-30T00:48:47.342589+0000","last_peered":"2024-03-30T00:48:47.342589+0000","last_clean":"2024-03-30T00:48:47.342589+0000","last_became_active":"2024-03-30T00:48:46.367960+0000","last_became_peered":"2024-03-30T00:48:46.367960+0000","last_unstale":"2024-03-30T00:48:47.342589+0000","last_undegraded":"2024-03-30T00:48:47.342589+0000","last_fullsized":"2024-03-30T00:48:47.342589+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:18:19.133951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.342727+0000","last_change":"2024-03-30T00:48:45.958035+0000","last_active":"2024-03-30T00:48:46.342727+0000","last_peered":"2024-03-30T00:48:46.342727+0000","last_clean":"2024-03-30T00:48:46.342727+0000","last_became_active":"2024-03-30T00:48:45.957642+0000","last_became_peered":"2024-03-30T00:48:45.957642+0000","last_unstale":"2024-03-30T00:48:46.342727+0000","last_undegraded":"2024-03-30T00:48:46.342727+0000","last_fullsized":"2024-03-30T00:48:46.342727+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:48.275866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342736+0000","last_change":"2024-03-30T00:48:46.361607+0000","last_active":"2024-03-30T00:48:47.342736+0000","last_peered":"2024-03-30T00:48:47.342736+0000","last_clean":"2024-03-30T00:48:47.342736+0000","last_became_active":"2024-03-30T00:48:46.361317+0000","last_became_peered":"2024-03-30T00:48:46.361317+0000","last_unstale":"2024-03-30T00:48:47.342736+0000","last_undegraded":"2024-03-30T00:48:47.342736+0000","last_fullsized":"2024-03-30T00:48:47.342736+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:54:22.267941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342313+0000","last_change":"2024-03-30T00:48:45.951105+0000","last_active":"2024-03-30T00:48:47.342313+0000","last_peered":"2024-03-30T00:48:47.342313+0000","last_clean":"2024-03-30T00:48:47.342313+0000","last_became_active":"2024-03-30T00:48:45.950747+0000","last_became_peered":"2024-03-30T00:48:45.950747+0000","last_unstale":"2024-03-30T00:48:47.342313+0000","last_undegraded":"2024-03-30T00:48:47.342313+0000","last_fullsized":"2024-03-30T00:48:47.342313+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:05:02.086394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345728+0000","last_change":"2024-03-30T00:48:46.353140+0000","last_active":"2024-03-30T00:48:47.345728+0000","last_peered":"2024-03-30T00:48:47.345728+0000","last_clean":"2024-03-30T00:48:47.345728+0000","last_became_active":"2024-03-30T00:48:46.352835+0000","last_became_peered":"2024-03-30T00:48:46.352835+0000","last_unstale":"2024-03-30T00:48:47.345728+0000","last_undegraded":"2024-03-30T00:48:47.345728+0000","last_fullsized":"2024-03-30T00:48:47.345728+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:42.926851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.346467+0000","last_change":"2024-03-30T00:48:45.960700+0000","last_active":"2024-03-30T00:48:46.346467+0000","last_peered":"2024-03-30T00:48:46.346467+0000","last_clean":"2024-03-30T00:48:46.346467+0000","last_became_active":"2024-03-30T00:48:45.959348+0000","last_became_peered":"2024-03-30T00:48:45.959348+0000","last_unstale":"2024-03-30T00:48:46.346467+0000","last_undegraded":"2024-03-30T00:48:46.346467+0000","last_fullsized":"2024-03-30T00:48:46.346467+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:08.766885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371960+0000","last_change":"2024-03-30T00:48:46.369972+0000","last_active":"2024-03-30T00:48:46.371960+0000","last_peered":"2024-03-30T00:48:46.371960+0000","last_clean":"2024-03-30T00:48:46.371960+0000","last_became_active":"2024-03-30T00:48:46.369716+0000","last_became_peered":"2024-03-30T00:48:46.369716+0000","last_unstale":"2024-03-30T00:48:46.371960+0000","last_undegraded":"2024-03-30T00:48:46.371960+0000","last_fullsized":"2024-03-30T00:48:46.371960+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:51.246897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342111+0000","last_change":"2024-03-30T00:48:45.948969+0000","last_active":"2024-03-30T00:48:47.342111+0000","last_peered":"2024-03-30T00:48:47.342111+0000","last_clean":"2024-03-30T00:48:47.342111+0000","last_became_active":"2024-03-30T00:48:45.948649+0000","last_became_peered":"2024-03-30T00:48:45.948649+0000","last_unstale":"2024-03-30T00:48:47.342111+0000","last_undegraded":"2024-03-30T00:48:47.342111+0000","last_fullsized":"2024-03-30T00:48:47.342111+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:24.403326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.362159+0000","last_change":"2024-03-30T00:48:46.359078+0000","last_active":"2024-03-30T00:48:46.362159+0000","last_peered":"2024-03-30T00:48:46.362159+0000","last_clean":"2024-03-30T00:48:46.362159+0000","last_became_active":"2024-03-30T00:48:46.358830+0000","last_became_peered":"2024-03-30T00:48:46.358830+0000","last_unstale":"2024-03-30T00:48:46.362159+0000","last_undegraded":"2024-03-30T00:48:46.362159+0000","last_fullsized":"2024-03-30T00:48:46.362159+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:31:43.779593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"109'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342933+0000","last_change":"2024-03-30T00:48:45.956923+0000","last_active":"2024-03-30T00:48:47.342933+0000","last_peered":"2024-03-30T00:48:47.342933+0000","last_clean":"2024-03-30T00:48:47.342933+0000","last_became_active":"2024-03-30T00:48:45.956632+0000","last_became_peered":"2024-03-30T00:48:45.956632+0000","last_unstale":"2024-03-30T00:48:47.342933+0000","last_undegraded":"2024-03-30T00:48:47.342933+0000","last_fullsized":"2024-03-30T00:48:47.342933+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:22.704908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":257,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.340753+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:48:46.340753+0000","last_peered":"2024-03-30T00:48:46.340753+0000","last_clean":"2024-03-30T00:48:46.340753+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:48:46.340753+0000","last_undegraded":"2024-03-30T00:48:46.340753+0000","last_fullsized":"2024-03-30T00:48:46.340753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574960,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1953792,"data_stored":1211812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.38200000000000001},"max":{"1min":0.93899999999999995,"5min":0.98499999999999999,"15min":1.0860000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.64300000000000002},"min":{"1min":0.438,"5min":0.40899999999999997,"15min":0.40500000000000003},"max":{"1min":0.88,"5min":0.95599999999999996,"15min":1.149},"last":0.57999999999999996}]},{"osd":1,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.66700000000000004,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.436},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67800000000000005,"15min":0.67900000000000005},"min":{"1min":0.442,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.3029999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.77000000000000002},"min":{"1min":0.503,"5min":0.46200000000000002,"15min":0.45000000000000001},"max":{"1min":0.95699999999999996,"5min":1.085,"15min":1.6020000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73699999999999999,"15min":0.76700000000000002},"min":{"1min":0.59399999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97899999999999998,"5min":1.034,"15min":1.6619999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.76800000000000002},"min":{"1min":0.52600000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":2.0150000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.96299999999999997,"5min":0.98399999999999999,"15min":1.9490000000000001},"last":0.72199999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.42499999999999999,"5min":0.38800000000000001,"15min":0.37},"max":{"1min":0.80500000000000005,"5min":1.2,"15min":1.2},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.62,"15min":0.627},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.89000000000000001,"15min":0.97899999999999998},"last":0.79500000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.69399999999999995},"min":{"1min":0.47499999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89300000000000002,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71999999999999997,"15min":0.752},"min":{"1min":0.47299999999999998,"5min":0.43099999999999999,"15min":0.41699999999999998},"max":{"1min":0.86499999999999999,"5min":0.97499999999999998,"15min":1.026},"last":0.89200000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":0.88300000000000001,"5min":0.95099999999999996,"15min":1.214},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69499999999999995,"15min":0.68799999999999994},"min":{"1min":0.45800000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90300000000000002,"5min":1.0149999999999999,"15min":1.0309999999999999},"last":0.878}]}]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1372160,"data_stored":623452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70999999999999996,"15min":0.83499999999999996},"min":{"1min":0.51000000000000001,"5min":0.41599999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":1.2969999999999999,"15min":4.0490000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70399999999999996,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.92300000000000004,"5min":1.278,"15min":1.47},"last":0.49199999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70499999999999996,"15min":0.84099999999999997},"min":{"1min":0.56200000000000006,"5min":0.44500000000000001,"15min":0.36899999999999999},"max":{"1min":1.0469999999999999,"5min":1.204,"15min":4.3170000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72799999999999998,"15min":0.89900000000000002},"min":{"1min":0.47199999999999998,"5min":0.46400000000000002,"15min":0.42999999999999999},"max":{"1min":0.88800000000000001,"5min":1.018,"15min":4.4160000000000004},"last":0.76100000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73299999999999998,"15min":0.89200000000000002},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.876,"5min":1.24,"15min":4.375},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71099999999999997,"15min":0.88700000000000001},"min":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.38600000000000001},"max":{"1min":1.0229999999999999,"5min":1.3129999999999999,"15min":4.2960000000000003},"last":0.68500000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75700000000000001,"15min":0.91300000000000003},"min":{"1min":0.56599999999999995,"5min":0.53000000000000003,"15min":0.47199999999999998},"max":{"1min":0.97199999999999998,"5min":1.2390000000000001,"15min":4.3399999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.89500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.41999999999999998},"max":{"1min":1.081,"5min":1.258,"15min":4.6020000000000003},"last":0.59899999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63500000000000001,"15min":0.66200000000000003},"min":{"1min":0.39900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89000000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70899999999999996,"15min":0.74299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41799999999999998,"15min":0.39700000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.036},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.66700000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0720000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.69799999999999995},"min":{"1min":0.46300000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86599999999999999,"5min":1.101,"15min":1.101},"last":0.83999999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.65300000000000002,"15min":0.69899999999999995},"min":{"1min":0.48699999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.89700000000000002,"5min":0.99299999999999999,"15min":1.0089999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.68700000000000006,"15min":0.71999999999999997},"min":{"1min":0.50700000000000001,"5min":0.46700000000000003,"15min":0.45700000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.72399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574960,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1351680,"data_stored":621284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66900000000000004,"15min":0.66400000000000003},"min":{"1min":0.51800000000000002,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":0.85999999999999999,"5min":0.97899999999999998,"15min":1.095},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69399999999999995,"15min":0.753},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.38800000000000001},"max":{"1min":0.878,"5min":1.0489999999999999,"15min":1.7569999999999999},"last":0.56499999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69699999999999995,"15min":0.73399999999999999},"min":{"1min":0.59399999999999997,"5min":0.46300000000000002,"15min":0.43099999999999999},"max":{"1min":0.95999999999999996,"5min":0.999,"15min":1.7330000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70499999999999996,"15min":0.71899999999999997},"min":{"1min":0.57299999999999995,"5min":0.45700000000000002,"15min":0.42399999999999999},"max":{"1min":0.92300000000000004,"5min":0.96299999999999997,"15min":1.3260000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.52000000000000002,"5min":0.45900000000000002,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69699999999999995,"15min":0.70699999999999996},"min":{"1min":0.47899999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.93500000000000005,"5min":0.97799999999999998,"15min":1.2789999999999999},"last":0.58099999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.72299999999999998,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.7},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.63100000000000001,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.91300000000000003,"5min":0.94499999999999995,"15min":1.181},"last":0.59199999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57199999999999995,"15min":0.61199999999999999},"min":{"1min":0.39100000000000001,"5min":0.34200000000000003,"15min":0.309},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.69699999999999995,"15min":0.748},"min":{"1min":0.46700000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88700000000000001,"5min":1.0509999999999999,"15min":1.615},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.61799999999999999,"15min":0.65600000000000003},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76000000000000001,"5min":0.91100000000000003,"15min":0.97399999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.38400000000000001,"15min":0.35199999999999998},"max":{"1min":0.92000000000000004,"5min":0.92800000000000005,"15min":1.569},"last":0.45400000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68200000000000005,"15min":0.72999999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.99099999999999999,"15min":1.0149999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.75800000000000001,"15min":0.81200000000000006},"min":{"1min":0.53500000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.86099999999999999,"5min":0.99199999999999999,"15min":1.383},"last":0.66600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1359872,"data_stored":621376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.53600000000000003,"5min":0.44400000000000001,"15min":0.34899999999999998},"max":{"1min":0.97099999999999997,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.66400000000000003},"min":{"1min":0.53800000000000003,"5min":0.40200000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.59799999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.70899999999999996,"15min":0.72099999999999997},"min":{"1min":0.51100000000000001,"5min":0.45500000000000002,"15min":0.45300000000000001},"max":{"1min":0.95399999999999996,"5min":0.95899999999999996,"15min":1.2909999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.53100000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92700000000000005,"5min":1.113,"15min":1.113},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.58199999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.745,"15min":0.755},"min":{"1min":0.61899999999999999,"5min":0.502,"15min":0.46700000000000003},"max":{"1min":0.93300000000000005,"5min":1.2010000000000001,"15min":1.228},"last":0.85399999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.71499999999999997,"15min":0.70999999999999996},"min":{"1min":0.60099999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89600000000000002,"5min":1.0660000000000001,"15min":1.27},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73299999999999998,"15min":0.76800000000000002},"min":{"1min":0.53200000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.99399999999999999,"5min":1,"15min":1.1739999999999999},"last":0.80900000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61799999999999999,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86299999999999999,"5min":0.92100000000000004,"15min":1.329},"last":0.876},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66000000000000003,"15min":0.71899999999999997},"min":{"1min":0.41599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86299999999999999,"5min":0.95199999999999996,"15min":1.3819999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.67500000000000004},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84099999999999997,"5min":0.88,"15min":1.2130000000000001},"last":0.745},{"interface":"front","average":{"1min":0.63,"5min":0.63400000000000001,"15min":0.71899999999999997},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95799999999999996,"5min":0.96199999999999997,"15min":1.3220000000000001},"last":0.82099999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63800000000000001,"15min":0.70999999999999996},"min":{"1min":0.45000000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91500000000000004,"5min":0.96199999999999997,"15min":1.6910000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72099999999999997,"15min":0.79500000000000004},"min":{"1min":0.55900000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93600000000000005,"5min":0.98899999999999999,"15min":1.5009999999999999},"last":0.77700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574961,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1355776,"data_stored":621866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45300000000000001,"15min":0.46400000000000002},"min":{"1min":0.28799999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64800000000000002,"5min":0.748,"15min":1.0529999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50800000000000001,"15min":0.52900000000000003},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.24199999999999999},"max":{"1min":0.73599999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45500000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.64400000000000002,"5min":0.79100000000000004,"15min":0.80900000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.32100000000000001},"max":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":1.2150000000000001},"last":0.48299999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.376,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.46200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71699999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44}]},{"osd":4,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.46300000000000002},"max":{"1min":0.98399999999999999,"5min":1.101,"15min":1.2470000000000001},"last":0.626},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.58399999999999996,"5min":0.495,"15min":0.48499999999999999},"max":{"1min":1.0609999999999999,"5min":1.0760000000000001,"15min":1.0860000000000001},"last":0.51800000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.51100000000000001},"max":{"1min":0.94499999999999995,"5min":1.0620000000000001,"15min":1.125},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78400000000000003,"15min":0.80100000000000005},"min":{"1min":0.64100000000000001,"5min":0.53100000000000003,"15min":0.48599999999999999},"max":{"1min":1.0549999999999999,"5min":1.0680000000000001,"15min":1.2390000000000001},"last":0.755}]},{"osd":6,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.51400000000000001,"15min":0.499},"max":{"1min":0.95799999999999996,"5min":1.157,"15min":1.226},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79800000000000004,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1120000000000001,"5min":1.167,"15min":1.167},"last":0.54100000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83199999999999996,"15min":0.84799999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.202},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.878},"min":{"1min":0.58599999999999997,"5min":0.54900000000000004,"15min":0.497},"max":{"1min":1.1639999999999999,"5min":1.2010000000000001,"15min":1.3160000000000001},"last":0.77600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1368064,"data_stored":627938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.442,"15min":0.44600000000000001},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.627,"5min":0.67800000000000005,"15min":4.0999999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.51200000000000001},"min":{"1min":0.307,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78500000000000003,"5min":0.84099999999999997,"15min":1.9239999999999999},"last":0.45200000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50700000000000001,"15min":0.52700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71299999999999997,"5min":0.81299999999999994,"15min":3.1139999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69499999999999995,"5min":0.77200000000000002,"15min":2.4159999999999999},"last":0.39700000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.498,"15min":0.498},"min":{"1min":0.36299999999999999,"5min":0.33200000000000002,"15min":0.32700000000000001},"max":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56999999999999995,"15min":0.58099999999999996},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.76300000000000001,"5min":0.88800000000000001,"15min":1.907},"last":0.59599999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.80800000000000005},"min":{"1min":0.57899999999999996,"5min":0.504,"15min":0.504},"max":{"1min":1.0229999999999999,"5min":1.55,"15min":1.9470000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79200000000000004,"15min":0.83199999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0860000000000001,"5min":1.292,"15min":1.972},"last":0.54100000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78200000000000003,"15min":0.82199999999999995},"min":{"1min":0.54900000000000004,"5min":0.504,"15min":0.504},"max":{"1min":0.96799999999999997,"5min":1.2549999999999999,"15min":2.4569999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.81499999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97699999999999998,"5min":1.5820000000000001,"15min":2.8039999999999998},"last":0.65800000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79200000000000004,"15min":0.81299999999999994},"min":{"1min":0.61299999999999999,"5min":0.56000000000000005,"15min":0.54700000000000004},"max":{"1min":1.014,"5min":1.645,"15min":2.8849999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81699999999999995,"15min":0.81799999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.02,"5min":1.6140000000000001,"15min":1.867},"last":0.752}]},{"osd":7,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79900000000000004,"15min":0.84999999999999998},"min":{"1min":0.625,"5min":0.495,"15min":0.495},"max":{"1min":1.0429999999999999,"5min":1.2649999999999999,"15min":2.3690000000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81799999999999995,"15min":0.84699999999999998},"min":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.52700000000000002},"max":{"1min":1.0369999999999999,"5min":1.4990000000000001,"15min":2.851},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1355776,"data_stored":621318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.60799999999999998,"5min":0.72199999999999998,"15min":0.80300000000000005},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.55500000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.45600000000000002,"15min":0.46200000000000002},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69099999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69099999999999995,"5min":0.84099999999999997,"15min":0.89400000000000002},"last":0.47899999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.46899999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76000000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53600000000000003,"15min":0.495},"min":{"1min":0.38800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77400000000000002,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60799999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73699999999999999,"15min":0.72499999999999998},"min":{"1min":0.52200000000000002,"5min":0.51400000000000001,"15min":0.38800000000000001},"max":{"1min":1.0349999999999999,"5min":1.175,"15min":1.175},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.999,"5min":1.079,"15min":1.2},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.052,"5min":1.109,"15min":1.3180000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77100000000000002,"15min":0.76800000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.996,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78300000000000003,"15min":0.78400000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.503},"max":{"1min":0.97499999999999998,"5min":1.179,"15min":1.179},"last":0.627},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76800000000000002,"15min":0.77300000000000002},"min":{"1min":0.59399999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.2150000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.66000000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.016,"5min":1.2050000000000001,"15min":1.2669999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.56200000000000006,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98899999999999999,"5min":1.1180000000000001,"15min":1.236},"last":0.58699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574961,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1961984,"data_stored":1213306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.433,"15min":0.46300000000000002},"min":{"1min":0.33600000000000002,"5min":0.27400000000000002,"15min":0.245},"max":{"1min":0.71699999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.36699999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66400000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.33500000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.50600000000000001,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60299999999999998,"5min":1.02,"15min":1.02},"last":0.503},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39000000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67000000000000004,"5min":0.83099999999999996,"15min":1.006},"last":0.41299999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.39000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68400000000000005,"5min":0.71599999999999997,"15min":0.84799999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.40999999999999998,"15min":0.39400000000000002},"max":{"1min":0.81499999999999995,"5min":0.88900000000000001,"15min":0.96899999999999997},"last":0.435}]},{"osd":4,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81799999999999995,"15min":0.80400000000000005},"min":{"1min":0.57699999999999996,"5min":0.50600000000000001,"15min":0.47899999999999998},"max":{"1min":1.0780000000000001,"5min":1.137,"15min":1.5169999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.81299999999999994},"min":{"1min":0.70399999999999996,"5min":0.56000000000000005,"15min":0.46400000000000002},"max":{"1min":1.079,"5min":1.145,"15min":1.252},"last":0.749}]},{"osd":5,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.61399999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0189999999999999,"5min":1.1919999999999999,"15min":1.3080000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78700000000000003,"15min":0.78000000000000003},"min":{"1min":0.65900000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.022,"5min":1.1439999999999999,"15min":1.462},"last":0.877}]},{"osd":6,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.66300000000000003,"5min":0.57799999999999996,"15min":0.54500000000000004},"max":{"1min":1.131,"5min":1.131,"15min":1.5920000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.67200000000000004,"5min":0.51500000000000001,"15min":0.46999999999999997},"max":{"1min":1.0549999999999999,"5min":1.1659999999999999,"15min":1.2230000000000001},"last":0.81999999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81399999999999995,"15min":0.81299999999999994},"min":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.3600000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.65900000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94899999999999995,"5min":1.2070000000000001,"15min":1.556},"last":0.76100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:48:50.453 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:48:50.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.617+0000 7fa668600640 1 Processor -- start 2024-03-30T00:48:50.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.617+0000 7fa668600640 1 -- start start 2024-03-30T00:48:50.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa668600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6600cad00 0x7fa6600c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:50.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa668600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa6600cd2d0 0x7fa6600cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:50.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa668600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6600c8e80 con 0x7fa6600cc8d0 2024-03-30T00:48:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa668600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6600cbec0 con 0x7fa6600cad00 2024-03-30T00:48:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa668600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6600cb620 con 0x7fa6600cd2d0 2024-03-30T00:48:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa665c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6600cad00 0x7fa6600c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa665c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6600cad00 0x7fa6600c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41680/0 (socket says 172.21.15.66:41680) 2024-03-30T00:48:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa665c00640 1 -- 172.21.15.66:0/663713538 learned_addr learned my addr 172.21.15.66:0/663713538 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa665200640 1 --2- 172.21.15.66:0/663713538 >> v2:172.21.15.156:3300/0 conn(0x7fa6600cd2d0 0x7fa6600cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa665c00640 1 -- 172.21.15.66:0/663713538 >> v2:172.21.15.156:3300/0 conn(0x7fa6600cd2d0 msgr2=0x7fa6600cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa665c00640 1 --2- 172.21.15.66:0/663713538 >> v2:172.21.15.156:3300/0 conn(0x7fa6600cd2d0 0x7fa6600cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa665c00640 1 -- 172.21.15.66:0/663713538 >> v1:172.21.15.156:6789/0 conn(0x7fa6600cc8d0 legacy=0x7fa66006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.621+0000 7fa665c00640 1 -- 172.21.15.66:0/663713538 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6600ca660 con 0x7fa6600cad00 2024-03-30T00:48:50.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa665c00640 1 --2- 172.21.15.66:0/663713538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600cad00 0x7fa6600c8940 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa6600cba10 tx=0x7fa65401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f453764ed5ae6c97 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa65fe00640 1 -- 172.21.15.66:0/663713538 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6540ac600 con 0x7fa6600cad00 2024-03-30T00:48:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa65fe00640 1 -- 172.21.15.66:0/663713538 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6540a8050 con 0x7fa6600cad00 2024-03-30T00:48:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa65fe00640 1 -- 172.21.15.66:0/663713538 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6540ac980 con 0x7fa6600cad00 2024-03-30T00:48:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa668600640 1 -- 172.21.15.66:0/663713538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600cad00 msgr2=0x7fa6600c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa668600640 1 --2- 172.21.15.66:0/663713538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600cad00 0x7fa6600c8940 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa6600cba10 tx=0x7fa65401c4b0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa668600640 1 -- 172.21.15.66:0/663713538 shutdown_connections 2024-03-30T00:48:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa668600640 1 --2- 172.21.15.66:0/663713538 >> v2:172.21.15.156:3300/0 conn(0x7fa6600cd2d0 0x7fa6600cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa668600640 1 --2- 172.21.15.66:0/663713538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600cad00 0x7fa6600c8940 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa668600640 1 -- 172.21.15.66:0/663713538 >> 172.21.15.66:0/663713538 conn(0x7fa660069f60 msgr2=0x7fa66006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:50.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa668600640 1 -- 172.21.15.66:0/663713538 shutdown_connections 2024-03-30T00:48:50.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa668600640 1 -- 172.21.15.66:0/663713538 wait complete. 2024-03-30T00:48:50.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa668600640 1 Processor -- start 2024-03-30T00:48:50.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.625+0000 7fa668600640 1 -- start start 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa668600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6600cad00 0x7fa66015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa668600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600cc8d0 0x7fa660159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa668600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6600cd2d0 0x7fa660159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa668600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6600c5d90 con 0x7fa6600cc8d0 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa668600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66015c230 con 0x7fa6600cad00 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa668600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa660167870 con 0x7fa6600cd2d0 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa665200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600cc8d0 0x7fa660159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa665200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600cc8d0 0x7fa660159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41686/0 (socket says 172.21.15.66:41686) 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa665200640 1 -- 172.21.15.66:0/4086288243 learned_addr learned my addr 172.21.15.66:0/4086288243 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa665c00640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6600cad00 0x7fa66015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa666600640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6600cd2d0 0x7fa660159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa665200640 1 -- 172.21.15.66:0/4086288243 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6600cd2d0 msgr2=0x7fa660159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa665200640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6600cd2d0 0x7fa660159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa665200640 1 -- 172.21.15.66:0/4086288243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6600cad00 msgr2=0x7fa66015bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa665200640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6600cad00 0x7fa66015bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa665200640 1 -- 172.21.15.66:0/4086288243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6540094d0 con 0x7fa6600cc8d0 2024-03-30T00:48:50.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa666600640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6600cd2d0 0x7fa660159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:50.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa665200640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600cc8d0 0x7fa660159340 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00d5c0 tx=0x7fa64c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:50.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa65d600640 1 -- 172.21.15.66:0/4086288243 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c004240 con 0x7fa6600cc8d0 2024-03-30T00:48:50.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa65d600640 1 -- 172.21.15.66:0/4086288243 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c0043a0 con 0x7fa6600cc8d0 2024-03-30T00:48:50.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa668600640 1 -- 172.21.15.66:0/4086288243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6600c5460 con 0x7fa6600cc8d0 2024-03-30T00:48:50.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa65d600640 1 -- 172.21.15.66:0/4086288243 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c003840 con 0x7fa6600cc8d0 2024-03-30T00:48:50.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa668600640 1 -- 172.21.15.66:0/4086288243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6600c5a20 con 0x7fa6600cc8d0 2024-03-30T00:48:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa668600640 1 -- 172.21.15.66:0/4086288243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6600c5f00 con 0x7fa6600cc8d0 2024-03-30T00:48:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.629+0000 7fa65d600640 1 -- 172.21.15.66:0/4086288243 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa64c0039a0 con 0x7fa6600cc8d0 2024-03-30T00:48:50.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.633+0000 7fa65d600640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa640066690 0x7fa640068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:50.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.633+0000 7fa65d600640 1 -- 172.21.15.66:0/4086288243 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa64c0844f0 con 0x7fa6600cc8d0 2024-03-30T00:48:50.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.633+0000 7fa665c00640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa640066690 0x7fa640068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:50.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.633+0000 7fa665c00640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa640066690 0x7fa640068b50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa6540ae000 tx=0x7fa654008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:50.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.637+0000 7fa65d600640 1 -- 172.21.15.66:0/4086288243 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa64c051680 con 0x7fa6600cc8d0 2024-03-30T00:48:50.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.793+0000 7fa668600640 1 -- 172.21.15.66:0/4086288243 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6600ca310 con 0x7fa640066690 2024-03-30T00:48:50.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.801+0000 7fa65d600640 1 -- 172.21.15.66:0/4086288243 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289924 (secure 0 0 0) 0x7fa6540ab3a0 con 0x7fa640066690 2024-03-30T00:48:50.814 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:48:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 -- 172.21.15.66:0/4086288243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa640066690 msgr2=0x7fa640068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa640066690 0x7fa640068b50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa6540ae000 tx=0x7fa654008040 comp rx=0 tx=0).stop 2024-03-30T00:48:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 -- 172.21.15.66:0/4086288243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600cc8d0 msgr2=0x7fa660159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600cc8d0 0x7fa660159340 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00d5c0 tx=0x7fa64c00da80 comp rx=0 tx=0).stop 2024-03-30T00:48:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 -- 172.21.15.66:0/4086288243 shutdown_connections 2024-03-30T00:48:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa6600cd2d0 0x7fa660159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa640066690 0x7fa640068b50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600cc8d0 0x7fa660159340 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 --2- 172.21.15.66:0/4086288243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6600cad00 0x7fa66015bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 -- 172.21.15.66:0/4086288243 >> 172.21.15.66:0/4086288243 conn(0x7fa660069f60 msgr2=0x7fa6600af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 -- 172.21.15.66:0/4086288243 shutdown_connections 2024-03-30T00:48:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:50.805+0000 7fa668600640 1 -- 172.21.15.66:0/4086288243 wait complete. 2024-03-30T00:48:50.834 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-03-30T00:48:48.971981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260756,"kb_used_data":11796,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754713964,"statfs":{"total":773094113280,"available":772827099136,"internally_reserved":0,"allocated":12079104,"data_stored":6162352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.046466"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342740+0000","last_change":"2024-03-30T00:48:46.358839+0000","last_active":"2024-03-30T00:48:47.342740+0000","last_peered":"2024-03-30T00:48:47.342740+0000","last_clean":"2024-03-30T00:48:47.342740+0000","last_became_active":"2024-03-30T00:48:46.358440+0000","last_became_peered":"2024-03-30T00:48:46.358440+0000","last_unstale":"2024-03-30T00:48:47.342740+0000","last_undegraded":"2024-03-30T00:48:47.342740+0000","last_fullsized":"2024-03-30T00:48:47.342740+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:56:27.359011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.364515+0000","last_change":"2024-03-30T00:48:46.360649+0000","last_active":"2024-03-30T00:48:46.364515+0000","last_peered":"2024-03-30T00:48:46.364515+0000","last_clean":"2024-03-30T00:48:46.364515+0000","last_became_active":"2024-03-30T00:48:46.360379+0000","last_became_peered":"2024-03-30T00:48:46.360379+0000","last_unstale":"2024-03-30T00:48:46.364515+0000","last_undegraded":"2024-03-30T00:48:46.364515+0000","last_fullsized":"2024-03-30T00:48:46.364515+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:02:13.552134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.343082+0000","last_change":"2024-03-30T00:48:46.361524+0000","last_active":"2024-03-30T00:48:47.343082+0000","last_peered":"2024-03-30T00:48:47.343082+0000","last_clean":"2024-03-30T00:48:47.343082+0000","last_became_active":"2024-03-30T00:48:46.360112+0000","last_became_peered":"2024-03-30T00:48:46.360112+0000","last_unstale":"2024-03-30T00:48:47.343082+0000","last_undegraded":"2024-03-30T00:48:47.343082+0000","last_fullsized":"2024-03-30T00:48:47.343082+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:36:34.824322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.361282+0000","last_change":"2024-03-30T00:48:46.354072+0000","last_active":"2024-03-30T00:48:46.361282+0000","last_peered":"2024-03-30T00:48:46.361282+0000","last_clean":"2024-03-30T00:48:46.361282+0000","last_became_active":"2024-03-30T00:48:46.353785+0000","last_became_peered":"2024-03-30T00:48:46.353785+0000","last_unstale":"2024-03-30T00:48:46.361282+0000","last_undegraded":"2024-03-30T00:48:46.361282+0000","last_fullsized":"2024-03-30T00:48:46.361282+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:52:19.120171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.374217+0000","last_change":"2024-03-30T00:48:46.372058+0000","last_active":"2024-03-30T00:48:46.374217+0000","last_peered":"2024-03-30T00:48:46.374217+0000","last_clean":"2024-03-30T00:48:46.374217+0000","last_became_active":"2024-03-30T00:48:46.371692+0000","last_became_peered":"2024-03-30T00:48:46.371692+0000","last_unstale":"2024-03-30T00:48:46.374217+0000","last_undegraded":"2024-03-30T00:48:46.374217+0000","last_fullsized":"2024-03-30T00:48:46.374217+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:22:56.906794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.372179+0000","last_change":"2024-03-30T00:48:46.370835+0000","last_active":"2024-03-30T00:48:46.372179+0000","last_peered":"2024-03-30T00:48:46.372179+0000","last_clean":"2024-03-30T00:48:46.372179+0000","last_became_active":"2024-03-30T00:48:46.370405+0000","last_became_peered":"2024-03-30T00:48:46.370405+0000","last_unstale":"2024-03-30T00:48:46.372179+0000","last_undegraded":"2024-03-30T00:48:46.372179+0000","last_fullsized":"2024-03-30T00:48:46.372179+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:56:24.032619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371083+0000","last_change":"2024-03-30T00:48:46.365333+0000","last_active":"2024-03-30T00:48:46.371083+0000","last_peered":"2024-03-30T00:48:46.371083+0000","last_clean":"2024-03-30T00:48:46.371083+0000","last_became_active":"2024-03-30T00:48:46.365032+0000","last_became_peered":"2024-03-30T00:48:46.365032+0000","last_unstale":"2024-03-30T00:48:46.371083+0000","last_undegraded":"2024-03-30T00:48:46.371083+0000","last_fullsized":"2024-03-30T00:48:46.371083+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:33:41.569588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.341943+0000","last_change":"2024-03-30T00:48:46.366483+0000","last_active":"2024-03-30T00:48:47.341943+0000","last_peered":"2024-03-30T00:48:47.341943+0000","last_clean":"2024-03-30T00:48:47.341943+0000","last_became_active":"2024-03-30T00:48:46.366059+0000","last_became_peered":"2024-03-30T00:48:46.366059+0000","last_unstale":"2024-03-30T00:48:47.341943+0000","last_undegraded":"2024-03-30T00:48:47.341943+0000","last_fullsized":"2024-03-30T00:48:47.341943+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:56:14.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371420+0000","last_change":"2024-03-30T00:48:46.366380+0000","last_active":"2024-03-30T00:48:46.371420+0000","last_peered":"2024-03-30T00:48:46.371420+0000","last_clean":"2024-03-30T00:48:46.371420+0000","last_became_active":"2024-03-30T00:48:46.365280+0000","last_became_peered":"2024-03-30T00:48:46.365280+0000","last_unstale":"2024-03-30T00:48:46.371420+0000","last_undegraded":"2024-03-30T00:48:46.371420+0000","last_fullsized":"2024-03-30T00:48:46.371420+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:00.738587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.370142+0000","last_change":"2024-03-30T00:48:46.363288+0000","last_active":"2024-03-30T00:48:46.370142+0000","last_peered":"2024-03-30T00:48:46.370142+0000","last_clean":"2024-03-30T00:48:46.370142+0000","last_became_active":"2024-03-30T00:48:46.363052+0000","last_became_peered":"2024-03-30T00:48:46.363052+0000","last_unstale":"2024-03-30T00:48:46.370142+0000","last_undegraded":"2024-03-30T00:48:46.370142+0000","last_fullsized":"2024-03-30T00:48:46.370142+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:42.396870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368575+0000","last_change":"2024-03-30T00:48:46.363747+0000","last_active":"2024-03-30T00:48:46.368575+0000","last_peered":"2024-03-30T00:48:46.368575+0000","last_clean":"2024-03-30T00:48:46.368575+0000","last_became_active":"2024-03-30T00:48:46.363492+0000","last_became_peered":"2024-03-30T00:48:46.363492+0000","last_unstale":"2024-03-30T00:48:46.368575+0000","last_undegraded":"2024-03-30T00:48:46.368575+0000","last_fullsized":"2024-03-30T00:48:46.368575+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:42.926851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.365745+0000","last_change":"2024-03-30T00:48:46.361393+0000","last_active":"2024-03-30T00:48:46.365745+0000","last_peered":"2024-03-30T00:48:46.365745+0000","last_clean":"2024-03-30T00:48:46.365745+0000","last_became_active":"2024-03-30T00:48:46.361124+0000","last_became_peered":"2024-03-30T00:48:46.361124+0000","last_unstale":"2024-03-30T00:48:46.365745+0000","last_undegraded":"2024-03-30T00:48:46.365745+0000","last_fullsized":"2024-03-30T00:48:46.365745+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:21:09.576366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.359720+0000","last_change":"2024-03-30T00:48:46.353587+0000","last_active":"2024-03-30T00:48:46.359720+0000","last_peered":"2024-03-30T00:48:46.359720+0000","last_clean":"2024-03-30T00:48:46.359720+0000","last_became_active":"2024-03-30T00:48:46.353285+0000","last_became_peered":"2024-03-30T00:48:46.353285+0000","last_unstale":"2024-03-30T00:48:46.359720+0000","last_undegraded":"2024-03-30T00:48:46.359720+0000","last_fullsized":"2024-03-30T00:48:46.359720+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:59:46.888843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345533+0000","last_change":"2024-03-30T00:48:46.356726+0000","last_active":"2024-03-30T00:48:47.345533+0000","last_peered":"2024-03-30T00:48:47.345533+0000","last_clean":"2024-03-30T00:48:47.345533+0000","last_became_active":"2024-03-30T00:48:46.356457+0000","last_became_peered":"2024-03-30T00:48:46.356457+0000","last_unstale":"2024-03-30T00:48:47.345533+0000","last_undegraded":"2024-03-30T00:48:47.345533+0000","last_fullsized":"2024-03-30T00:48:47.345533+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:31:43.779593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345432+0000","last_change":"2024-03-30T00:48:46.353250+0000","last_active":"2024-03-30T00:48:47.345432+0000","last_peered":"2024-03-30T00:48:47.345432+0000","last_clean":"2024-03-30T00:48:47.345432+0000","last_became_active":"2024-03-30T00:48:46.352913+0000","last_became_peered":"2024-03-30T00:48:46.352913+0000","last_unstale":"2024-03-30T00:48:47.345432+0000","last_undegraded":"2024-03-30T00:48:47.345432+0000","last_fullsized":"2024-03-30T00:48:47.345432+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:21:19.264827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345480+0000","last_change":"2024-03-30T00:48:46.358038+0000","last_active":"2024-03-30T00:48:47.345480+0000","last_peered":"2024-03-30T00:48:47.345480+0000","last_clean":"2024-03-30T00:48:47.345480+0000","last_became_active":"2024-03-30T00:48:46.355461+0000","last_became_peered":"2024-03-30T00:48:46.355461+0000","last_unstale":"2024-03-30T00:48:47.345480+0000","last_undegraded":"2024-03-30T00:48:47.345480+0000","last_fullsized":"2024-03-30T00:48:47.345480+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:51.246897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342775+0000","last_change":"2024-03-30T00:48:46.360594+0000","last_active":"2024-03-30T00:48:47.342775+0000","last_peered":"2024-03-30T00:48:47.342775+0000","last_clean":"2024-03-30T00:48:47.342775+0000","last_became_active":"2024-03-30T00:48:46.360281+0000","last_became_peered":"2024-03-30T00:48:46.360281+0000","last_unstale":"2024-03-30T00:48:47.342775+0000","last_undegraded":"2024-03-30T00:48:47.342775+0000","last_fullsized":"2024-03-30T00:48:47.342775+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:53.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371253+0000","last_change":"2024-03-30T00:48:46.365562+0000","last_active":"2024-03-30T00:48:46.371253+0000","last_peered":"2024-03-30T00:48:46.371253+0000","last_clean":"2024-03-30T00:48:46.371253+0000","last_became_active":"2024-03-30T00:48:46.365242+0000","last_became_peered":"2024-03-30T00:48:46.365242+0000","last_unstale":"2024-03-30T00:48:46.371253+0000","last_undegraded":"2024-03-30T00:48:46.371253+0000","last_fullsized":"2024-03-30T00:48:46.371253+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:27:58.176108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.343170+0000","last_change":"2024-03-30T00:48:46.363232+0000","last_active":"2024-03-30T00:48:47.343170+0000","last_peered":"2024-03-30T00:48:47.343170+0000","last_clean":"2024-03-30T00:48:47.343170+0000","last_became_active":"2024-03-30T00:48:46.362978+0000","last_became_peered":"2024-03-30T00:48:46.362978+0000","last_unstale":"2024-03-30T00:48:47.343170+0000","last_undegraded":"2024-03-30T00:48:47.343170+0000","last_fullsized":"2024-03-30T00:48:47.343170+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:51:39.826307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.356379+0000","last_change":"2024-03-30T00:48:46.352196+0000","last_active":"2024-03-30T00:48:46.356379+0000","last_peered":"2024-03-30T00:48:46.356379+0000","last_clean":"2024-03-30T00:48:46.356379+0000","last_became_active":"2024-03-30T00:48:46.351902+0000","last_became_peered":"2024-03-30T00:48:46.351902+0000","last_unstale":"2024-03-30T00:48:46.356379+0000","last_undegraded":"2024-03-30T00:48:46.356379+0000","last_fullsized":"2024-03-30T00:48:46.356379+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:47:32.075977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342494+0000","last_change":"2024-03-30T00:48:46.350063+0000","last_active":"2024-03-30T00:48:47.342494+0000","last_peered":"2024-03-30T00:48:47.342494+0000","last_clean":"2024-03-30T00:48:47.342494+0000","last_became_active":"2024-03-30T00:48:46.349723+0000","last_became_peered":"2024-03-30T00:48:46.349723+0000","last_unstale":"2024-03-30T00:48:47.342494+0000","last_undegraded":"2024-03-30T00:48:47.342494+0000","last_fullsized":"2024-03-30T00:48:47.342494+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:40:02.486030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.374150+0000","last_change":"2024-03-30T00:48:46.371465+0000","last_active":"2024-03-30T00:48:46.374150+0000","last_peered":"2024-03-30T00:48:46.374150+0000","last_clean":"2024-03-30T00:48:46.374150+0000","last_became_active":"2024-03-30T00:48:46.371211+0000","last_became_peered":"2024-03-30T00:48:46.371211+0000","last_unstale":"2024-03-30T00:48:46.374150+0000","last_undegraded":"2024-03-30T00:48:46.374150+0000","last_fullsized":"2024-03-30T00:48:46.374150+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:07.947403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342283+0000","last_change":"2024-03-30T00:48:46.360790+0000","last_active":"2024-03-30T00:48:47.342283+0000","last_peered":"2024-03-30T00:48:47.342283+0000","last_clean":"2024-03-30T00:48:47.342283+0000","last_became_active":"2024-03-30T00:48:46.360319+0000","last_became_peered":"2024-03-30T00:48:46.360319+0000","last_unstale":"2024-03-30T00:48:47.342283+0000","last_undegraded":"2024-03-30T00:48:47.342283+0000","last_fullsized":"2024-03-30T00:48:47.342283+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:07:50.101198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.365321+0000","last_change":"2024-03-30T00:48:46.357688+0000","last_active":"2024-03-30T00:48:46.365321+0000","last_peered":"2024-03-30T00:48:46.365321+0000","last_clean":"2024-03-30T00:48:46.365321+0000","last_became_active":"2024-03-30T00:48:46.357354+0000","last_became_peered":"2024-03-30T00:48:46.357354+0000","last_unstale":"2024-03-30T00:48:46.365321+0000","last_undegraded":"2024-03-30T00:48:46.365321+0000","last_fullsized":"2024-03-30T00:48:46.365321+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:27.485835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368002+0000","last_change":"2024-03-30T00:48:46.360387+0000","last_active":"2024-03-30T00:48:46.368002+0000","last_peered":"2024-03-30T00:48:46.368002+0000","last_clean":"2024-03-30T00:48:46.368002+0000","last_became_active":"2024-03-30T00:48:46.358825+0000","last_became_peered":"2024-03-30T00:48:46.358825+0000","last_unstale":"2024-03-30T00:48:46.368002+0000","last_undegraded":"2024-03-30T00:48:46.368002+0000","last_fullsized":"2024-03-30T00:48:46.368002+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:30:00.322906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.356689+0000","last_change":"2024-03-30T00:48:46.352467+0000","last_active":"2024-03-30T00:48:46.356689+0000","last_peered":"2024-03-30T00:48:46.356689+0000","last_clean":"2024-03-30T00:48:46.356689+0000","last_became_active":"2024-03-30T00:48:46.352111+0000","last_became_peered":"2024-03-30T00:48:46.352111+0000","last_unstale":"2024-03-30T00:48:46.356689+0000","last_undegraded":"2024-03-30T00:48:46.356689+0000","last_fullsized":"2024-03-30T00:48:46.356689+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:59:46.888843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368538+0000","last_change":"2024-03-30T00:48:46.363230+0000","last_active":"2024-03-30T00:48:46.368538+0000","last_peered":"2024-03-30T00:48:46.368538+0000","last_clean":"2024-03-30T00:48:46.368538+0000","last_became_active":"2024-03-30T00:48:46.361934+0000","last_became_peered":"2024-03-30T00:48:46.361934+0000","last_unstale":"2024-03-30T00:48:46.368538+0000","last_undegraded":"2024-03-30T00:48:46.368538+0000","last_fullsized":"2024-03-30T00:48:46.368538+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:21:19.264827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371469+0000","last_change":"2024-03-30T00:48:46.365628+0000","last_active":"2024-03-30T00:48:46.371469+0000","last_peered":"2024-03-30T00:48:46.371469+0000","last_clean":"2024-03-30T00:48:46.371469+0000","last_became_active":"2024-03-30T00:48:46.363911+0000","last_became_peered":"2024-03-30T00:48:46.363911+0000","last_unstale":"2024-03-30T00:48:46.371469+0000","last_undegraded":"2024-03-30T00:48:46.371469+0000","last_fullsized":"2024-03-30T00:48:46.371469+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:35:26.863153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342530+0000","last_change":"2024-03-30T00:48:46.344665+0000","last_active":"2024-03-30T00:48:47.342530+0000","last_peered":"2024-03-30T00:48:47.342530+0000","last_clean":"2024-03-30T00:48:47.342530+0000","last_became_active":"2024-03-30T00:48:46.344351+0000","last_became_peered":"2024-03-30T00:48:46.344351+0000","last_unstale":"2024-03-30T00:48:47.342530+0000","last_undegraded":"2024-03-30T00:48:47.342530+0000","last_fullsized":"2024-03-30T00:48:47.342530+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:57:00.091791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.353817+0000","last_change":"2024-03-30T00:48:46.350224+0000","last_active":"2024-03-30T00:48:46.353817+0000","last_peered":"2024-03-30T00:48:46.353817+0000","last_clean":"2024-03-30T00:48:46.353817+0000","last_became_active":"2024-03-30T00:48:46.349951+0000","last_became_peered":"2024-03-30T00:48:46.349951+0000","last_unstale":"2024-03-30T00:48:46.353817+0000","last_undegraded":"2024-03-30T00:48:46.353817+0000","last_fullsized":"2024-03-30T00:48:46.353817+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:00:21.826773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.363976+0000","last_change":"2024-03-30T00:48:46.358190+0000","last_active":"2024-03-30T00:48:46.363976+0000","last_peered":"2024-03-30T00:48:46.363976+0000","last_clean":"2024-03-30T00:48:46.363976+0000","last_became_active":"2024-03-30T00:48:46.357848+0000","last_became_peered":"2024-03-30T00:48:46.357848+0000","last_unstale":"2024-03-30T00:48:46.363976+0000","last_undegraded":"2024-03-30T00:48:46.363976+0000","last_fullsized":"2024-03-30T00:48:46.363976+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:41:31.484548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342008+0000","last_change":"2024-03-30T00:48:46.350662+0000","last_active":"2024-03-30T00:48:47.342008+0000","last_peered":"2024-03-30T00:48:47.342008+0000","last_clean":"2024-03-30T00:48:47.342008+0000","last_became_active":"2024-03-30T00:48:46.346445+0000","last_became_peered":"2024-03-30T00:48:46.346445+0000","last_unstale":"2024-03-30T00:48:47.342008+0000","last_undegraded":"2024-03-30T00:48:47.342008+0000","last_fullsized":"2024-03-30T00:48:47.342008+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:05:38.458418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342299+0000","last_change":"2024-03-30T00:48:46.355746+0000","last_active":"2024-03-30T00:48:47.342299+0000","last_peered":"2024-03-30T00:48:47.342299+0000","last_clean":"2024-03-30T00:48:47.342299+0000","last_became_active":"2024-03-30T00:48:46.354935+0000","last_became_peered":"2024-03-30T00:48:46.354935+0000","last_unstale":"2024-03-30T00:48:47.342299+0000","last_undegraded":"2024-03-30T00:48:47.342299+0000","last_fullsized":"2024-03-30T00:48:47.342299+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:53:23.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"109'4","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.369164+0000","last_change":"2024-03-30T00:48:45.955252+0000","last_active":"2024-03-30T00:48:46.369164+0000","last_peered":"2024-03-30T00:48:46.369164+0000","last_clean":"2024-03-30T00:48:46.369164+0000","last_became_active":"2024-03-30T00:48:45.953941+0000","last_became_peered":"2024-03-30T00:48:45.953941+0000","last_unstale":"2024-03-30T00:48:46.369164+0000","last_undegraded":"2024-03-30T00:48:46.369164+0000","last_fullsized":"2024-03-30T00:48:46.369164+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:48.275866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342131+0000","last_change":"2024-03-30T00:48:46.361656+0000","last_active":"2024-03-30T00:48:47.342131+0000","last_peered":"2024-03-30T00:48:47.342131+0000","last_clean":"2024-03-30T00:48:47.342131+0000","last_became_active":"2024-03-30T00:48:46.361392+0000","last_became_peered":"2024-03-30T00:48:46.361392+0000","last_unstale":"2024-03-30T00:48:47.342131+0000","last_undegraded":"2024-03-30T00:48:47.342131+0000","last_fullsized":"2024-03-30T00:48:47.342131+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:39:10.093491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"109'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342895+0000","last_change":"2024-03-30T00:48:45.940875+0000","last_active":"2024-03-30T00:48:47.342895+0000","last_peered":"2024-03-30T00:48:47.342895+0000","last_clean":"2024-03-30T00:48:47.342895+0000","last_became_active":"2024-03-30T00:48:45.940525+0000","last_became_peered":"2024-03-30T00:48:45.940525+0000","last_unstale":"2024-03-30T00:48:47.342895+0000","last_undegraded":"2024-03-30T00:48:47.342895+0000","last_fullsized":"2024-03-30T00:48:47.342895+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:25:11.691579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.375517+0000","last_change":"2024-03-30T00:48:46.374550+0000","last_active":"2024-03-30T00:48:46.375517+0000","last_peered":"2024-03-30T00:48:46.375517+0000","last_clean":"2024-03-30T00:48:46.375517+0000","last_became_active":"2024-03-30T00:48:46.374271+0000","last_became_peered":"2024-03-30T00:48:46.374271+0000","last_unstale":"2024-03-30T00:48:46.375517+0000","last_undegraded":"2024-03-30T00:48:46.375517+0000","last_fullsized":"2024-03-30T00:48:46.375517+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:20:15.867799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.369295+0000","last_change":"2024-03-30T00:48:45.958340+0000","last_active":"2024-03-30T00:48:46.369295+0000","last_peered":"2024-03-30T00:48:46.369295+0000","last_clean":"2024-03-30T00:48:46.369295+0000","last_became_active":"2024-03-30T00:48:45.958069+0000","last_became_peered":"2024-03-30T00:48:45.958069+0000","last_unstale":"2024-03-30T00:48:46.369295+0000","last_undegraded":"2024-03-30T00:48:46.369295+0000","last_fullsized":"2024-03-30T00:48:46.369295+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:00:41.392529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.363196+0000","last_change":"2024-03-30T00:48:46.353746+0000","last_active":"2024-03-30T00:48:46.363196+0000","last_peered":"2024-03-30T00:48:46.363196+0000","last_clean":"2024-03-30T00:48:46.363196+0000","last_became_active":"2024-03-30T00:48:46.353515+0000","last_became_peered":"2024-03-30T00:48:46.353515+0000","last_unstale":"2024-03-30T00:48:46.363196+0000","last_undegraded":"2024-03-30T00:48:46.363196+0000","last_fullsized":"2024-03-30T00:48:46.363196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:41:06.772377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.372229+0000","last_change":"2024-03-30T00:48:45.952866+0000","last_active":"2024-03-30T00:48:46.372229+0000","last_peered":"2024-03-30T00:48:46.372229+0000","last_clean":"2024-03-30T00:48:46.372229+0000","last_became_active":"2024-03-30T00:48:45.952622+0000","last_became_peered":"2024-03-30T00:48:45.952622+0000","last_unstale":"2024-03-30T00:48:46.372229+0000","last_undegraded":"2024-03-30T00:48:46.372229+0000","last_fullsized":"2024-03-30T00:48:46.372229+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:00:59.734245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371947+0000","last_change":"2024-03-30T00:48:46.370357+0000","last_active":"2024-03-30T00:48:46.371947+0000","last_peered":"2024-03-30T00:48:46.371947+0000","last_clean":"2024-03-30T00:48:46.371947+0000","last_became_active":"2024-03-30T00:48:46.370070+0000","last_became_peered":"2024-03-30T00:48:46.370070+0000","last_unstale":"2024-03-30T00:48:46.371947+0000","last_undegraded":"2024-03-30T00:48:46.371947+0000","last_fullsized":"2024-03-30T00:48:46.371947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:53:11.764633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"109'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345561+0000","last_change":"2024-03-30T00:48:45.949823+0000","last_active":"2024-03-30T00:48:47.345561+0000","last_peered":"2024-03-30T00:48:47.345561+0000","last_clean":"2024-03-30T00:48:47.345561+0000","last_became_active":"2024-03-30T00:48:45.949208+0000","last_became_peered":"2024-03-30T00:48:45.949208+0000","last_unstale":"2024-03-30T00:48:47.345561+0000","last_undegraded":"2024-03-30T00:48:47.345561+0000","last_fullsized":"2024-03-30T00:48:47.345561+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:41:30.480206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.343332+0000","last_change":"2024-03-30T00:48:46.355259+0000","last_active":"2024-03-30T00:48:47.343332+0000","last_peered":"2024-03-30T00:48:47.343332+0000","last_clean":"2024-03-30T00:48:47.343332+0000","last_became_active":"2024-03-30T00:48:46.354653+0000","last_became_peered":"2024-03-30T00:48:46.354653+0000","last_unstale":"2024-03-30T00:48:47.343332+0000","last_undegraded":"2024-03-30T00:48:47.343332+0000","last_fullsized":"2024-03-30T00:48:47.343332+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:39:33.512770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.369393+0000","last_change":"2024-03-30T00:48:45.955849+0000","last_active":"2024-03-30T00:48:46.369393+0000","last_peered":"2024-03-30T00:48:46.369393+0000","last_clean":"2024-03-30T00:48:46.369393+0000","last_became_active":"2024-03-30T00:48:45.955597+0000","last_became_peered":"2024-03-30T00:48:45.955597+0000","last_unstale":"2024-03-30T00:48:46.369393+0000","last_undegraded":"2024-03-30T00:48:46.369393+0000","last_fullsized":"2024-03-30T00:48:46.369393+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:41:18.345346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368415+0000","last_change":"2024-03-30T00:48:46.364040+0000","last_active":"2024-03-30T00:48:46.368415+0000","last_peered":"2024-03-30T00:48:46.368415+0000","last_clean":"2024-03-30T00:48:46.368415+0000","last_became_active":"2024-03-30T00:48:46.363451+0000","last_became_peered":"2024-03-30T00:48:46.363451+0000","last_unstale":"2024-03-30T00:48:46.368415+0000","last_undegraded":"2024-03-30T00:48:46.368415+0000","last_fullsized":"2024-03-30T00:48:46.368415+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:41:19.349688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.343220+0000","last_change":"2024-03-30T00:48:45.960360+0000","last_active":"2024-03-30T00:48:46.343220+0000","last_peered":"2024-03-30T00:48:46.343220+0000","last_clean":"2024-03-30T00:48:46.343220+0000","last_became_active":"2024-03-30T00:48:45.960023+0000","last_became_peered":"2024-03-30T00:48:45.960023+0000","last_unstale":"2024-03-30T00:48:46.343220+0000","last_undegraded":"2024-03-30T00:48:46.343220+0000","last_fullsized":"2024-03-30T00:48:46.343220+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:26.481493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.361451+0000","last_change":"2024-03-30T00:48:46.356891+0000","last_active":"2024-03-30T00:48:46.361451+0000","last_peered":"2024-03-30T00:48:46.361451+0000","last_clean":"2024-03-30T00:48:46.361451+0000","last_became_active":"2024-03-30T00:48:46.356641+0000","last_became_peered":"2024-03-30T00:48:46.356641+0000","last_unstale":"2024-03-30T00:48:46.361451+0000","last_undegraded":"2024-03-30T00:48:46.361451+0000","last_fullsized":"2024-03-30T00:48:46.361451+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:21:09.576366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.345824+0000","last_change":"2024-03-30T00:48:45.950404+0000","last_active":"2024-03-30T00:48:46.345824+0000","last_peered":"2024-03-30T00:48:46.345824+0000","last_clean":"2024-03-30T00:48:46.345824+0000","last_became_active":"2024-03-30T00:48:45.950148+0000","last_became_peered":"2024-03-30T00:48:45.950148+0000","last_unstale":"2024-03-30T00:48:46.345824+0000","last_undegraded":"2024-03-30T00:48:46.345824+0000","last_fullsized":"2024-03-30T00:48:46.345824+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:26.481493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342215+0000","last_change":"2024-03-30T00:48:46.364449+0000","last_active":"2024-03-30T00:48:47.342215+0000","last_peered":"2024-03-30T00:48:47.342215+0000","last_clean":"2024-03-30T00:48:47.342215+0000","last_became_active":"2024-03-30T00:48:46.363266+0000","last_became_peered":"2024-03-30T00:48:46.363266+0000","last_unstale":"2024-03-30T00:48:47.342215+0000","last_undegraded":"2024-03-30T00:48:47.342215+0000","last_fullsized":"2024-03-30T00:48:47.342215+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:44:31.304932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.343889+0000","last_change":"2024-03-30T00:48:45.958138+0000","last_active":"2024-03-30T00:48:46.343889+0000","last_peered":"2024-03-30T00:48:46.343889+0000","last_clean":"2024-03-30T00:48:46.343889+0000","last_became_active":"2024-03-30T00:48:45.957832+0000","last_became_peered":"2024-03-30T00:48:45.957832+0000","last_unstale":"2024-03-30T00:48:46.343889+0000","last_undegraded":"2024-03-30T00:48:46.343889+0000","last_fullsized":"2024-03-30T00:48:46.343889+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:23:44.224176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.370227+0000","last_change":"2024-03-30T00:48:46.358789+0000","last_active":"2024-03-30T00:48:46.370227+0000","last_peered":"2024-03-30T00:48:46.370227+0000","last_clean":"2024-03-30T00:48:46.370227+0000","last_became_active":"2024-03-30T00:48:46.358535+0000","last_became_peered":"2024-03-30T00:48:46.358535+0000","last_unstale":"2024-03-30T00:48:46.370227+0000","last_undegraded":"2024-03-30T00:48:46.370227+0000","last_fullsized":"2024-03-30T00:48:46.370227+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:49.280208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342681+0000","last_change":"2024-03-30T00:48:45.954446+0000","last_active":"2024-03-30T00:48:47.342681+0000","last_peered":"2024-03-30T00:48:47.342681+0000","last_clean":"2024-03-30T00:48:47.342681+0000","last_became_active":"2024-03-30T00:48:45.954156+0000","last_became_peered":"2024-03-30T00:48:45.954156+0000","last_unstale":"2024-03-30T00:48:47.342681+0000","last_undegraded":"2024-03-30T00:48:47.342681+0000","last_fullsized":"2024-03-30T00:48:47.342681+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:54:39.885925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342311+0000","last_change":"2024-03-30T00:48:46.370061+0000","last_active":"2024-03-30T00:48:47.342311+0000","last_peered":"2024-03-30T00:48:47.342311+0000","last_clean":"2024-03-30T00:48:47.342311+0000","last_became_active":"2024-03-30T00:48:46.369658+0000","last_became_peered":"2024-03-30T00:48:46.369658+0000","last_unstale":"2024-03-30T00:48:47.342311+0000","last_undegraded":"2024-03-30T00:48:47.342311+0000","last_fullsized":"2024-03-30T00:48:47.342311+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:50.412709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"109'3","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345658+0000","last_change":"2024-03-30T00:48:45.957152+0000","last_active":"2024-03-30T00:48:47.345658+0000","last_peered":"2024-03-30T00:48:47.345658+0000","last_clean":"2024-03-30T00:48:47.345658+0000","last_became_active":"2024-03-30T00:48:45.956742+0000","last_became_peered":"2024-03-30T00:48:45.956742+0000","last_unstale":"2024-03-30T00:48:47.345658+0000","last_undegraded":"2024-03-30T00:48:47.345658+0000","last_fullsized":"2024-03-30T00:48:47.345658+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:00:59.734245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.370808+0000","last_change":"2024-03-30T00:48:46.361007+0000","last_active":"2024-03-30T00:48:46.370808+0000","last_peered":"2024-03-30T00:48:46.370808+0000","last_clean":"2024-03-30T00:48:46.370808+0000","last_became_active":"2024-03-30T00:48:46.360730+0000","last_became_peered":"2024-03-30T00:48:46.360730+0000","last_unstale":"2024-03-30T00:48:46.370808+0000","last_undegraded":"2024-03-30T00:48:46.370808+0000","last_fullsized":"2024-03-30T00:48:46.370808+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:08:46.368598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"109'6","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.375813+0000","last_change":"2024-03-30T00:48:45.951301+0000","last_active":"2024-03-30T00:48:46.375813+0000","last_peered":"2024-03-30T00:48:46.375813+0000","last_clean":"2024-03-30T00:48:46.375813+0000","last_became_active":"2024-03-30T00:48:45.951020+0000","last_became_peered":"2024-03-30T00:48:45.951020+0000","last_unstale":"2024-03-30T00:48:46.375813+0000","last_undegraded":"2024-03-30T00:48:46.375813+0000","last_fullsized":"2024-03-30T00:48:46.375813+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:16:40.272043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368414+0000","last_change":"2024-03-30T00:48:46.359956+0000","last_active":"2024-03-30T00:48:46.368414+0000","last_peered":"2024-03-30T00:48:46.368414+0000","last_clean":"2024-03-30T00:48:46.368414+0000","last_became_active":"2024-03-30T00:48:46.359701+0000","last_became_peered":"2024-03-30T00:48:46.359701+0000","last_unstale":"2024-03-30T00:48:46.368414+0000","last_undegraded":"2024-03-30T00:48:46.368414+0000","last_fullsized":"2024-03-30T00:48:46.368414+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:14.615490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.374492+0000","last_change":"2024-03-30T00:48:45.952608+0000","last_active":"2024-03-30T00:48:46.374492+0000","last_peered":"2024-03-30T00:48:46.374492+0000","last_clean":"2024-03-30T00:48:46.374492+0000","last_became_active":"2024-03-30T00:48:45.952171+0000","last_became_peered":"2024-03-30T00:48:45.952171+0000","last_unstale":"2024-03-30T00:48:46.374492+0000","last_undegraded":"2024-03-30T00:48:46.374492+0000","last_fullsized":"2024-03-30T00:48:46.374492+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:27:57.171766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371056+0000","last_change":"2024-03-30T00:48:46.364718+0000","last_active":"2024-03-30T00:48:46.371056+0000","last_peered":"2024-03-30T00:48:46.371056+0000","last_clean":"2024-03-30T00:48:46.371056+0000","last_became_active":"2024-03-30T00:48:46.364407+0000","last_became_peered":"2024-03-30T00:48:46.364407+0000","last_unstale":"2024-03-30T00:48:46.371056+0000","last_undegraded":"2024-03-30T00:48:46.371056+0000","last_fullsized":"2024-03-30T00:48:46.371056+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:43.737434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.346753+0000","last_change":"2024-03-30T00:48:45.954635+0000","last_active":"2024-03-30T00:48:46.346753+0000","last_peered":"2024-03-30T00:48:46.346753+0000","last_clean":"2024-03-30T00:48:46.346753+0000","last_became_active":"2024-03-30T00:48:45.954388+0000","last_became_peered":"2024-03-30T00:48:45.954388+0000","last_unstale":"2024-03-30T00:48:46.346753+0000","last_undegraded":"2024-03-30T00:48:46.346753+0000","last_fullsized":"2024-03-30T00:48:46.346753+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:22:55.902452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342876+0000","last_change":"2024-03-30T00:48:46.351573+0000","last_active":"2024-03-30T00:48:47.342876+0000","last_peered":"2024-03-30T00:48:47.342876+0000","last_clean":"2024-03-30T00:48:47.342876+0000","last_became_active":"2024-03-30T00:48:46.351284+0000","last_became_peered":"2024-03-30T00:48:46.351284+0000","last_unstale":"2024-03-30T00:48:47.342876+0000","last_undegraded":"2024-03-30T00:48:47.342876+0000","last_fullsized":"2024-03-30T00:48:47.342876+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:16:41.276384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342211+0000","last_change":"2024-03-30T00:48:45.949283+0000","last_active":"2024-03-30T00:48:47.342211+0000","last_peered":"2024-03-30T00:48:47.342211+0000","last_clean":"2024-03-30T00:48:47.342211+0000","last_became_active":"2024-03-30T00:48:45.949000+0000","last_became_peered":"2024-03-30T00:48:45.949000+0000","last_unstale":"2024-03-30T00:48:47.342211+0000","last_undegraded":"2024-03-30T00:48:47.342211+0000","last_fullsized":"2024-03-30T00:48:47.342211+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:43:23.706446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342807+0000","last_change":"2024-03-30T00:48:46.360602+0000","last_active":"2024-03-30T00:48:47.342807+0000","last_peered":"2024-03-30T00:48:47.342807+0000","last_clean":"2024-03-30T00:48:47.342807+0000","last_became_active":"2024-03-30T00:48:46.360326+0000","last_became_peered":"2024-03-30T00:48:46.360326+0000","last_unstale":"2024-03-30T00:48:47.342807+0000","last_undegraded":"2024-03-30T00:48:47.342807+0000","last_fullsized":"2024-03-30T00:48:47.342807+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:42:06.568226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"109'8","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368977+0000","last_change":"2024-03-30T00:48:45.960549+0000","last_active":"2024-03-30T00:48:46.368977+0000","last_peered":"2024-03-30T00:48:46.368977+0000","last_clean":"2024-03-30T00:48:46.368977+0000","last_became_active":"2024-03-30T00:48:45.960277+0000","last_became_peered":"2024-03-30T00:48:45.960277+0000","last_unstale":"2024-03-30T00:48:46.368977+0000","last_undegraded":"2024-03-30T00:48:46.368977+0000","last_fullsized":"2024-03-30T00:48:46.368977+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:22:55.902452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342036+0000","last_change":"2024-03-30T00:48:46.351031+0000","last_active":"2024-03-30T00:48:47.342036+0000","last_peered":"2024-03-30T00:48:47.342036+0000","last_clean":"2024-03-30T00:48:47.342036+0000","last_became_active":"2024-03-30T00:48:46.350744+0000","last_became_peered":"2024-03-30T00:48:46.350744+0000","last_unstale":"2024-03-30T00:48:47.342036+0000","last_undegraded":"2024-03-30T00:48:47.342036+0000","last_fullsized":"2024-03-30T00:48:47.342036+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:09.771227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371722+0000","last_change":"2024-03-30T00:48:45.958654+0000","last_active":"2024-03-30T00:48:46.371722+0000","last_peered":"2024-03-30T00:48:46.371722+0000","last_clean":"2024-03-30T00:48:46.371722+0000","last_became_active":"2024-03-30T00:48:45.958343+0000","last_became_peered":"2024-03-30T00:48:45.958343+0000","last_unstale":"2024-03-30T00:48:46.371722+0000","last_undegraded":"2024-03-30T00:48:46.371722+0000","last_fullsized":"2024-03-30T00:48:46.371722+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:40:01.481688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.365665+0000","last_change":"2024-03-30T00:48:46.358559+0000","last_active":"2024-03-30T00:48:46.365665+0000","last_peered":"2024-03-30T00:48:46.365665+0000","last_clean":"2024-03-30T00:48:46.365665+0000","last_became_active":"2024-03-30T00:48:46.358259+0000","last_became_peered":"2024-03-30T00:48:46.358259+0000","last_unstale":"2024-03-30T00:48:46.365665+0000","last_undegraded":"2024-03-30T00:48:46.365665+0000","last_fullsized":"2024-03-30T00:48:46.365665+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:32:36.057968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"109'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345695+0000","last_change":"2024-03-30T00:48:45.955648+0000","last_active":"2024-03-30T00:48:47.345695+0000","last_peered":"2024-03-30T00:48:47.345695+0000","last_clean":"2024-03-30T00:48:47.345695+0000","last_became_active":"2024-03-30T00:48:45.955355+0000","last_became_peered":"2024-03-30T00:48:45.955355+0000","last_unstale":"2024-03-30T00:48:47.345695+0000","last_undegraded":"2024-03-30T00:48:47.345695+0000","last_fullsized":"2024-03-30T00:48:47.345695+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:42.733092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.374331+0000","last_change":"2024-03-30T00:48:46.373105+0000","last_active":"2024-03-30T00:48:46.374331+0000","last_peered":"2024-03-30T00:48:46.374331+0000","last_clean":"2024-03-30T00:48:46.374331+0000","last_became_active":"2024-03-30T00:48:46.371847+0000","last_became_peered":"2024-03-30T00:48:46.371847+0000","last_unstale":"2024-03-30T00:48:46.374331+0000","last_undegraded":"2024-03-30T00:48:46.374331+0000","last_fullsized":"2024-03-30T00:48:46.374331+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:31:57.388123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.354065+0000","last_change":"2024-03-30T00:48:45.953601+0000","last_active":"2024-03-30T00:48:46.354065+0000","last_peered":"2024-03-30T00:48:46.354065+0000","last_clean":"2024-03-30T00:48:46.354065+0000","last_became_active":"2024-03-30T00:48:45.953320+0000","last_became_peered":"2024-03-30T00:48:45.953320+0000","last_unstale":"2024-03-30T00:48:46.354065+0000","last_undegraded":"2024-03-30T00:48:46.354065+0000","last_fullsized":"2024-03-30T00:48:46.354065+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:35:25.858812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342657+0000","last_change":"2024-03-30T00:48:46.358372+0000","last_active":"2024-03-30T00:48:47.342657+0000","last_peered":"2024-03-30T00:48:47.342657+0000","last_clean":"2024-03-30T00:48:47.342657+0000","last_became_active":"2024-03-30T00:48:46.358092+0000","last_became_peered":"2024-03-30T00:48:46.358092+0000","last_unstale":"2024-03-30T00:48:47.342657+0000","last_undegraded":"2024-03-30T00:48:47.342657+0000","last_fullsized":"2024-03-30T00:48:47.342657+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:39.685433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"109'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342578+0000","last_change":"2024-03-30T00:48:45.954514+0000","last_active":"2024-03-30T00:48:47.342578+0000","last_peered":"2024-03-30T00:48:47.342578+0000","last_clean":"2024-03-30T00:48:47.342578+0000","last_became_active":"2024-03-30T00:48:45.954272+0000","last_became_peered":"2024-03-30T00:48:45.954272+0000","last_unstale":"2024-03-30T00:48:47.342578+0000","last_undegraded":"2024-03-30T00:48:47.342578+0000","last_fullsized":"2024-03-30T00:48:47.342578+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:13:56.705237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.343251+0000","last_change":"2024-03-30T00:48:46.362004+0000","last_active":"2024-03-30T00:48:47.343251+0000","last_peered":"2024-03-30T00:48:47.343251+0000","last_clean":"2024-03-30T00:48:47.343251+0000","last_became_active":"2024-03-30T00:48:46.361778+0000","last_became_peered":"2024-03-30T00:48:46.361778+0000","last_unstale":"2024-03-30T00:48:47.343251+0000","last_undegraded":"2024-03-30T00:48:47.343251+0000","last_fullsized":"2024-03-30T00:48:47.343251+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:25:21.100530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342210+0000","last_change":"2024-03-30T00:48:45.954058+0000","last_active":"2024-03-30T00:48:47.342210+0000","last_peered":"2024-03-30T00:48:47.342210+0000","last_clean":"2024-03-30T00:48:47.342210+0000","last_became_active":"2024-03-30T00:48:45.953762+0000","last_became_peered":"2024-03-30T00:48:45.953762+0000","last_unstale":"2024-03-30T00:48:47.342210+0000","last_undegraded":"2024-03-30T00:48:47.342210+0000","last_fullsized":"2024-03-30T00:48:47.342210+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:03:20.122059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.361374+0000","last_change":"2024-03-30T00:48:46.355935+0000","last_active":"2024-03-30T00:48:46.361374+0000","last_peered":"2024-03-30T00:48:46.361374+0000","last_clean":"2024-03-30T00:48:46.361374+0000","last_became_active":"2024-03-30T00:48:46.355691+0000","last_became_peered":"2024-03-30T00:48:46.355691+0000","last_unstale":"2024-03-30T00:48:46.361374+0000","last_undegraded":"2024-03-30T00:48:46.361374+0000","last_fullsized":"2024-03-30T00:48:46.361374+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:47:32.075977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.346358+0000","last_change":"2024-03-30T00:48:45.955237+0000","last_active":"2024-03-30T00:48:46.346358+0000","last_peered":"2024-03-30T00:48:46.346358+0000","last_clean":"2024-03-30T00:48:46.346358+0000","last_became_active":"2024-03-30T00:48:45.954978+0000","last_became_peered":"2024-03-30T00:48:45.954978+0000","last_unstale":"2024-03-30T00:48:46.346358+0000","last_undegraded":"2024-03-30T00:48:46.346358+0000","last_fullsized":"2024-03-30T00:48:46.346358+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:05:37.454076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342909+0000","last_change":"2024-03-30T00:48:46.368396+0000","last_active":"2024-03-30T00:48:47.342909+0000","last_peered":"2024-03-30T00:48:47.342909+0000","last_clean":"2024-03-30T00:48:47.342909+0000","last_became_active":"2024-03-30T00:48:46.368118+0000","last_became_peered":"2024-03-30T00:48:46.368118+0000","last_unstale":"2024-03-30T00:48:47.342909+0000","last_undegraded":"2024-03-30T00:48:47.342909+0000","last_fullsized":"2024-03-30T00:48:47.342909+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:17.678809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342095+0000","last_change":"2024-03-30T00:48:45.953604+0000","last_active":"2024-03-30T00:48:47.342095+0000","last_peered":"2024-03-30T00:48:47.342095+0000","last_clean":"2024-03-30T00:48:47.342095+0000","last_became_active":"2024-03-30T00:48:45.953301+0000","last_became_peered":"2024-03-30T00:48:45.953301+0000","last_unstale":"2024-03-30T00:48:47.342095+0000","last_undegraded":"2024-03-30T00:48:47.342095+0000","last_fullsized":"2024-03-30T00:48:47.342095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:24:20.981909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342097+0000","last_change":"2024-03-30T00:48:46.358539+0000","last_active":"2024-03-30T00:48:47.342097+0000","last_peered":"2024-03-30T00:48:47.342097+0000","last_clean":"2024-03-30T00:48:47.342097+0000","last_became_active":"2024-03-30T00:48:46.358126+0000","last_became_peered":"2024-03-30T00:48:46.358126+0000","last_unstale":"2024-03-30T00:48:47.342097+0000","last_undegraded":"2024-03-30T00:48:47.342097+0000","last_fullsized":"2024-03-30T00:48:47.342097+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:36:35.379415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.373699+0000","last_change":"2024-03-30T00:48:45.959117+0000","last_active":"2024-03-30T00:48:46.373699+0000","last_peered":"2024-03-30T00:48:46.373699+0000","last_clean":"2024-03-30T00:48:46.373699+0000","last_became_active":"2024-03-30T00:48:45.958810+0000","last_became_peered":"2024-03-30T00:48:45.958810+0000","last_unstale":"2024-03-30T00:48:46.373699+0000","last_undegraded":"2024-03-30T00:48:46.373699+0000","last_fullsized":"2024-03-30T00:48:46.373699+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:33:40.565247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.351404+0000","last_change":"2024-03-30T00:48:46.349402+0000","last_active":"2024-03-30T00:48:46.351404+0000","last_peered":"2024-03-30T00:48:46.351404+0000","last_clean":"2024-03-30T00:48:46.351404+0000","last_became_active":"2024-03-30T00:48:46.349078+0000","last_became_peered":"2024-03-30T00:48:46.349078+0000","last_unstale":"2024-03-30T00:48:46.351404+0000","last_undegraded":"2024-03-30T00:48:46.351404+0000","last_fullsized":"2024-03-30T00:48:46.351404+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:04.678998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342654+0000","last_change":"2024-03-30T00:48:45.948993+0000","last_active":"2024-03-30T00:48:47.342654+0000","last_peered":"2024-03-30T00:48:47.342654+0000","last_clean":"2024-03-30T00:48:47.342654+0000","last_became_active":"2024-03-30T00:48:45.948576+0000","last_became_peered":"2024-03-30T00:48:45.948576+0000","last_unstale":"2024-03-30T00:48:47.342654+0000","last_undegraded":"2024-03-30T00:48:47.342654+0000","last_fullsized":"2024-03-30T00:48:47.342654+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:57:12.133595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.361217+0000","last_change":"2024-03-30T00:48:46.355243+0000","last_active":"2024-03-30T00:48:46.361217+0000","last_peered":"2024-03-30T00:48:46.361217+0000","last_clean":"2024-03-30T00:48:46.361217+0000","last_became_active":"2024-03-30T00:48:46.354924+0000","last_became_peered":"2024-03-30T00:48:46.354924+0000","last_unstale":"2024-03-30T00:48:46.361217+0000","last_undegraded":"2024-03-30T00:48:46.361217+0000","last_fullsized":"2024-03-30T00:48:46.361217+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:04.678998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.345825+0000","last_change":"2024-03-30T00:48:45.953819+0000","last_active":"2024-03-30T00:48:46.345825+0000","last_peered":"2024-03-30T00:48:46.345825+0000","last_clean":"2024-03-30T00:48:46.345825+0000","last_became_active":"2024-03-30T00:48:45.953546+0000","last_became_peered":"2024-03-30T00:48:45.953546+0000","last_unstale":"2024-03-30T00:48:46.345825+0000","last_undegraded":"2024-03-30T00:48:46.345825+0000","last_fullsized":"2024-03-30T00:48:46.345825+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:39:09.089149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.368859+0000","last_change":"2024-03-30T00:48:46.366335+0000","last_active":"2024-03-30T00:48:46.368859+0000","last_peered":"2024-03-30T00:48:46.368859+0000","last_clean":"2024-03-30T00:48:46.368859+0000","last_became_active":"2024-03-30T00:48:46.363018+0000","last_became_peered":"2024-03-30T00:48:46.363018+0000","last_unstale":"2024-03-30T00:48:46.368859+0000","last_undegraded":"2024-03-30T00:48:46.368859+0000","last_fullsized":"2024-03-30T00:48:46.368859+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:07.947403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.349332+0000","last_change":"2024-03-30T00:48:45.948987+0000","last_active":"2024-03-30T00:48:46.349332+0000","last_peered":"2024-03-30T00:48:46.349332+0000","last_clean":"2024-03-30T00:48:46.349332+0000","last_became_active":"2024-03-30T00:48:45.948116+0000","last_became_peered":"2024-03-30T00:48:45.948116+0000","last_unstale":"2024-03-30T00:48:46.349332+0000","last_undegraded":"2024-03-30T00:48:46.349332+0000","last_fullsized":"2024-03-30T00:48:46.349332+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:09:13.900315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342589+0000","last_change":"2024-03-30T00:48:46.368207+0000","last_active":"2024-03-30T00:48:47.342589+0000","last_peered":"2024-03-30T00:48:47.342589+0000","last_clean":"2024-03-30T00:48:47.342589+0000","last_became_active":"2024-03-30T00:48:46.367960+0000","last_became_peered":"2024-03-30T00:48:46.367960+0000","last_unstale":"2024-03-30T00:48:47.342589+0000","last_undegraded":"2024-03-30T00:48:47.342589+0000","last_fullsized":"2024-03-30T00:48:47.342589+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:18:19.133951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.342727+0000","last_change":"2024-03-30T00:48:45.958035+0000","last_active":"2024-03-30T00:48:46.342727+0000","last_peered":"2024-03-30T00:48:46.342727+0000","last_clean":"2024-03-30T00:48:46.342727+0000","last_became_active":"2024-03-30T00:48:45.957642+0000","last_became_peered":"2024-03-30T00:48:45.957642+0000","last_unstale":"2024-03-30T00:48:46.342727+0000","last_undegraded":"2024-03-30T00:48:46.342727+0000","last_fullsized":"2024-03-30T00:48:46.342727+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:48.275866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342736+0000","last_change":"2024-03-30T00:48:46.361607+0000","last_active":"2024-03-30T00:48:47.342736+0000","last_peered":"2024-03-30T00:48:47.342736+0000","last_clean":"2024-03-30T00:48:47.342736+0000","last_became_active":"2024-03-30T00:48:46.361317+0000","last_became_peered":"2024-03-30T00:48:46.361317+0000","last_unstale":"2024-03-30T00:48:47.342736+0000","last_undegraded":"2024-03-30T00:48:47.342736+0000","last_fullsized":"2024-03-30T00:48:47.342736+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:54:22.267941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342313+0000","last_change":"2024-03-30T00:48:45.951105+0000","last_active":"2024-03-30T00:48:47.342313+0000","last_peered":"2024-03-30T00:48:47.342313+0000","last_clean":"2024-03-30T00:48:47.342313+0000","last_became_active":"2024-03-30T00:48:45.950747+0000","last_became_peered":"2024-03-30T00:48:45.950747+0000","last_unstale":"2024-03-30T00:48:47.342313+0000","last_undegraded":"2024-03-30T00:48:47.342313+0000","last_fullsized":"2024-03-30T00:48:47.342313+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:05:02.086394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.345728+0000","last_change":"2024-03-30T00:48:46.353140+0000","last_active":"2024-03-30T00:48:47.345728+0000","last_peered":"2024-03-30T00:48:47.345728+0000","last_clean":"2024-03-30T00:48:47.345728+0000","last_became_active":"2024-03-30T00:48:46.352835+0000","last_became_peered":"2024-03-30T00:48:46.352835+0000","last_unstale":"2024-03-30T00:48:47.345728+0000","last_undegraded":"2024-03-30T00:48:47.345728+0000","last_fullsized":"2024-03-30T00:48:47.345728+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:42.926851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.346467+0000","last_change":"2024-03-30T00:48:45.960700+0000","last_active":"2024-03-30T00:48:46.346467+0000","last_peered":"2024-03-30T00:48:46.346467+0000","last_clean":"2024-03-30T00:48:46.346467+0000","last_became_active":"2024-03-30T00:48:45.959348+0000","last_became_peered":"2024-03-30T00:48:45.959348+0000","last_unstale":"2024-03-30T00:48:46.346467+0000","last_undegraded":"2024-03-30T00:48:46.346467+0000","last_fullsized":"2024-03-30T00:48:46.346467+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:08.766885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.371960+0000","last_change":"2024-03-30T00:48:46.369972+0000","last_active":"2024-03-30T00:48:46.371960+0000","last_peered":"2024-03-30T00:48:46.371960+0000","last_clean":"2024-03-30T00:48:46.371960+0000","last_became_active":"2024-03-30T00:48:46.369716+0000","last_became_peered":"2024-03-30T00:48:46.369716+0000","last_unstale":"2024-03-30T00:48:46.371960+0000","last_undegraded":"2024-03-30T00:48:46.371960+0000","last_fullsized":"2024-03-30T00:48:46.371960+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:51.246897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342111+0000","last_change":"2024-03-30T00:48:45.948969+0000","last_active":"2024-03-30T00:48:47.342111+0000","last_peered":"2024-03-30T00:48:47.342111+0000","last_clean":"2024-03-30T00:48:47.342111+0000","last_became_active":"2024-03-30T00:48:45.948649+0000","last_became_peered":"2024-03-30T00:48:45.948649+0000","last_unstale":"2024-03-30T00:48:47.342111+0000","last_undegraded":"2024-03-30T00:48:47.342111+0000","last_fullsized":"2024-03-30T00:48:47.342111+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:24.403326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.362159+0000","last_change":"2024-03-30T00:48:46.359078+0000","last_active":"2024-03-30T00:48:46.362159+0000","last_peered":"2024-03-30T00:48:46.362159+0000","last_clean":"2024-03-30T00:48:46.362159+0000","last_became_active":"2024-03-30T00:48:46.358830+0000","last_became_peered":"2024-03-30T00:48:46.358830+0000","last_unstale":"2024-03-30T00:48:46.362159+0000","last_undegraded":"2024-03-30T00:48:46.362159+0000","last_fullsized":"2024-03-30T00:48:46.362159+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:45.925701+0000","last_clean_scrub_stamp":"2024-03-30T00:48:45.925701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:31:43.779593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"109'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-03-30T00:48:47.342933+0000","last_change":"2024-03-30T00:48:45.956923+0000","last_active":"2024-03-30T00:48:47.342933+0000","last_peered":"2024-03-30T00:48:47.342933+0000","last_clean":"2024-03-30T00:48:47.342933+0000","last_became_active":"2024-03-30T00:48:45.956632+0000","last_became_peered":"2024-03-30T00:48:45.956632+0000","last_unstale":"2024-03-30T00:48:47.342933+0000","last_undegraded":"2024-03-30T00:48:47.342933+0000","last_fullsized":"2024-03-30T00:48:47.342933+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:48:44.921359+0000","last_clean_scrub_stamp":"2024-03-30T00:48:44.921359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:22.704908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":257,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-30T00:48:46.340753+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:48:46.340753+0000","last_peered":"2024-03-30T00:48:46.340753+0000","last_clean":"2024-03-30T00:48:46.340753+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:48:46.340753+0000","last_undegraded":"2024-03-30T00:48:46.340753+0000","last_fullsized":"2024-03-30T00:48:46.340753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574960,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1953792,"data_stored":1211812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.38200000000000001},"max":{"1min":0.93899999999999995,"5min":0.98499999999999999,"15min":1.0860000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.64300000000000002},"min":{"1min":0.438,"5min":0.40899999999999997,"15min":0.40500000000000003},"max":{"1min":0.88,"5min":0.95599999999999996,"15min":1.149},"last":0.57999999999999996}]},{"osd":1,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.66700000000000004,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.436},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67800000000000005,"15min":0.67900000000000005},"min":{"1min":0.442,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.3029999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.77000000000000002},"min":{"1min":0.503,"5min":0.46200000000000002,"15min":0.45000000000000001},"max":{"1min":0.95699999999999996,"5min":1.085,"15min":1.6020000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73699999999999999,"15min":0.76700000000000002},"min":{"1min":0.59399999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97899999999999998,"5min":1.034,"15min":1.6619999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.76800000000000002},"min":{"1min":0.52600000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":2.0150000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.96299999999999997,"5min":0.98399999999999999,"15min":1.9490000000000001},"last":0.72199999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.42499999999999999,"5min":0.38800000000000001,"15min":0.37},"max":{"1min":0.80500000000000005,"5min":1.2,"15min":1.2},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.62,"15min":0.627},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.89000000000000001,"15min":0.97899999999999998},"last":0.79500000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.69399999999999995},"min":{"1min":0.47499999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89300000000000002,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71999999999999997,"15min":0.752},"min":{"1min":0.47299999999999998,"5min":0.43099999999999999,"15min":0.41699999999999998},"max":{"1min":0.86499999999999999,"5min":0.97499999999999998,"15min":1.026},"last":0.89200000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":0.88300000000000001,"5min":0.95099999999999996,"15min":1.214},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69499999999999995,"15min":0.68799999999999994},"min":{"1min":0.45800000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90300000000000002,"5min":1.0149999999999999,"15min":1.0309999999999999},"last":0.878}]}]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1372160,"data_stored":623452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70999999999999996,"15min":0.83499999999999996},"min":{"1min":0.51000000000000001,"5min":0.41599999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":1.2969999999999999,"15min":4.0490000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70399999999999996,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.92300000000000004,"5min":1.278,"15min":1.47},"last":0.49199999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70499999999999996,"15min":0.84099999999999997},"min":{"1min":0.56200000000000006,"5min":0.44500000000000001,"15min":0.36899999999999999},"max":{"1min":1.0469999999999999,"5min":1.204,"15min":4.3170000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72799999999999998,"15min":0.89900000000000002},"min":{"1min":0.47199999999999998,"5min":0.46400000000000002,"15min":0.42999999999999999},"max":{"1min":0.88800000000000001,"5min":1.018,"15min":4.4160000000000004},"last":0.76100000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73299999999999998,"15min":0.89200000000000002},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.876,"5min":1.24,"15min":4.375},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71099999999999997,"15min":0.88700000000000001},"min":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.38600000000000001},"max":{"1min":1.0229999999999999,"5min":1.3129999999999999,"15min":4.2960000000000003},"last":0.68500000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75700000000000001,"15min":0.91300000000000003},"min":{"1min":0.56599999999999995,"5min":0.53000000000000003,"15min":0.47199999999999998},"max":{"1min":0.97199999999999998,"5min":1.2390000000000001,"15min":4.3399999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.89500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.41999999999999998},"max":{"1min":1.081,"5min":1.258,"15min":4.6020000000000003},"last":0.59899999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63500000000000001,"15min":0.66200000000000003},"min":{"1min":0.39900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89000000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70899999999999996,"15min":0.74299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41799999999999998,"15min":0.39700000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.036},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.66700000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0720000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.69799999999999995},"min":{"1min":0.46300000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86599999999999999,"5min":1.101,"15min":1.101},"last":0.83999999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.65300000000000002,"15min":0.69899999999999995},"min":{"1min":0.48699999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.89700000000000002,"5min":0.99299999999999999,"15min":1.0089999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.68700000000000006,"15min":0.71999999999999997},"min":{"1min":0.50700000000000001,"5min":0.46700000000000003,"15min":0.45700000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.72399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574960,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1351680,"data_stored":621284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66900000000000004,"15min":0.66400000000000003},"min":{"1min":0.51800000000000002,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":0.85999999999999999,"5min":0.97899999999999998,"15min":1.095},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69399999999999995,"15min":0.753},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.38800000000000001},"max":{"1min":0.878,"5min":1.0489999999999999,"15min":1.7569999999999999},"last":0.56499999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69699999999999995,"15min":0.73399999999999999},"min":{"1min":0.59399999999999997,"5min":0.46300000000000002,"15min":0.43099999999999999},"max":{"1min":0.95999999999999996,"5min":0.999,"15min":1.7330000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70499999999999996,"15min":0.71899999999999997},"min":{"1min":0.57299999999999995,"5min":0.45700000000000002,"15min":0.42399999999999999},"max":{"1min":0.92300000000000004,"5min":0.96299999999999997,"15min":1.3260000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.52000000000000002,"5min":0.45900000000000002,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69699999999999995,"15min":0.70699999999999996},"min":{"1min":0.47899999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.93500000000000005,"5min":0.97799999999999998,"15min":1.2789999999999999},"last":0.58099999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.72299999999999998,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.7},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.63100000000000001,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.91300000000000003,"5min":0.94499999999999995,"15min":1.181},"last":0.59199999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57199999999999995,"15min":0.61199999999999999},"min":{"1min":0.39100000000000001,"5min":0.34200000000000003,"15min":0.309},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.69699999999999995,"15min":0.748},"min":{"1min":0.46700000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88700000000000001,"5min":1.0509999999999999,"15min":1.615},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.61799999999999999,"15min":0.65600000000000003},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76000000000000001,"5min":0.91100000000000003,"15min":0.97399999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.38400000000000001,"15min":0.35199999999999998},"max":{"1min":0.92000000000000004,"5min":0.92800000000000005,"15min":1.569},"last":0.45400000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68200000000000005,"15min":0.72999999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.99099999999999999,"15min":1.0149999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.75800000000000001,"15min":0.81200000000000006},"min":{"1min":0.53500000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.86099999999999999,"5min":0.99199999999999999,"15min":1.383},"last":0.66600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1359872,"data_stored":621376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.53600000000000003,"5min":0.44400000000000001,"15min":0.34899999999999998},"max":{"1min":0.97099999999999997,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.66400000000000003},"min":{"1min":0.53800000000000003,"5min":0.40200000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.59799999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.70899999999999996,"15min":0.72099999999999997},"min":{"1min":0.51100000000000001,"5min":0.45500000000000002,"15min":0.45300000000000001},"max":{"1min":0.95399999999999996,"5min":0.95899999999999996,"15min":1.2909999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.53100000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92700000000000005,"5min":1.113,"15min":1.113},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.58199999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.745,"15min":0.755},"min":{"1min":0.61899999999999999,"5min":0.502,"15min":0.46700000000000003},"max":{"1min":0.93300000000000005,"5min":1.2010000000000001,"15min":1.228},"last":0.85399999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.71499999999999997,"15min":0.70999999999999996},"min":{"1min":0.60099999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89600000000000002,"5min":1.0660000000000001,"15min":1.27},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73299999999999998,"15min":0.76800000000000002},"min":{"1min":0.53200000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.99399999999999999,"5min":1,"15min":1.1739999999999999},"last":0.80900000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61799999999999999,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86299999999999999,"5min":0.92100000000000004,"15min":1.329},"last":0.876},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66000000000000003,"15min":0.71899999999999997},"min":{"1min":0.41599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86299999999999999,"5min":0.95199999999999996,"15min":1.3819999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.67500000000000004},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84099999999999997,"5min":0.88,"15min":1.2130000000000001},"last":0.745},{"interface":"front","average":{"1min":0.63,"5min":0.63400000000000001,"15min":0.71899999999999997},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95799999999999996,"5min":0.96199999999999997,"15min":1.3220000000000001},"last":0.82099999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63800000000000001,"15min":0.70999999999999996},"min":{"1min":0.45000000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91500000000000004,"5min":0.96199999999999997,"15min":1.6910000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72099999999999997,"15min":0.79500000000000004},"min":{"1min":0.55900000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93600000000000005,"5min":0.98899999999999999,"15min":1.5009999999999999},"last":0.77700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574961,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1355776,"data_stored":621866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45300000000000001,"15min":0.46400000000000002},"min":{"1min":0.28799999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64800000000000002,"5min":0.748,"15min":1.0529999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50800000000000001,"15min":0.52900000000000003},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.24199999999999999},"max":{"1min":0.73599999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45500000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.64400000000000002,"5min":0.79100000000000004,"15min":0.80900000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.32100000000000001},"max":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":1.2150000000000001},"last":0.48299999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.376,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.46200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71699999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44}]},{"osd":4,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.46300000000000002},"max":{"1min":0.98399999999999999,"5min":1.101,"15min":1.2470000000000001},"last":0.626},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.58399999999999996,"5min":0.495,"15min":0.48499999999999999},"max":{"1min":1.0609999999999999,"5min":1.0760000000000001,"15min":1.0860000000000001},"last":0.51800000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.51100000000000001},"max":{"1min":0.94499999999999995,"5min":1.0620000000000001,"15min":1.125},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78400000000000003,"15min":0.80100000000000005},"min":{"1min":0.64100000000000001,"5min":0.53100000000000003,"15min":0.48599999999999999},"max":{"1min":1.0549999999999999,"5min":1.0680000000000001,"15min":1.2390000000000001},"last":0.755}]},{"osd":6,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.51400000000000001,"15min":0.499},"max":{"1min":0.95799999999999996,"5min":1.157,"15min":1.226},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79800000000000004,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1120000000000001,"5min":1.167,"15min":1.167},"last":0.54100000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83199999999999996,"15min":0.84799999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.202},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.878},"min":{"1min":0.58599999999999997,"5min":0.54900000000000004,"15min":0.497},"max":{"1min":1.1639999999999999,"5min":1.2010000000000001,"15min":1.3160000000000001},"last":0.77600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1368064,"data_stored":627938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.442,"15min":0.44600000000000001},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.627,"5min":0.67800000000000005,"15min":4.0999999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.51200000000000001},"min":{"1min":0.307,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78500000000000003,"5min":0.84099999999999997,"15min":1.9239999999999999},"last":0.45200000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50700000000000001,"15min":0.52700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71299999999999997,"5min":0.81299999999999994,"15min":3.1139999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69499999999999995,"5min":0.77200000000000002,"15min":2.4159999999999999},"last":0.39700000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.498,"15min":0.498},"min":{"1min":0.36299999999999999,"5min":0.33200000000000002,"15min":0.32700000000000001},"max":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56999999999999995,"15min":0.58099999999999996},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.76300000000000001,"5min":0.88800000000000001,"15min":1.907},"last":0.59599999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.80800000000000005},"min":{"1min":0.57899999999999996,"5min":0.504,"15min":0.504},"max":{"1min":1.0229999999999999,"5min":1.55,"15min":1.9470000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79200000000000004,"15min":0.83199999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0860000000000001,"5min":1.292,"15min":1.972},"last":0.54100000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78200000000000003,"15min":0.82199999999999995},"min":{"1min":0.54900000000000004,"5min":0.504,"15min":0.504},"max":{"1min":0.96799999999999997,"5min":1.2549999999999999,"15min":2.4569999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.81499999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97699999999999998,"5min":1.5820000000000001,"15min":2.8039999999999998},"last":0.65800000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79200000000000004,"15min":0.81299999999999994},"min":{"1min":0.61299999999999999,"5min":0.56000000000000005,"15min":0.54700000000000004},"max":{"1min":1.014,"5min":1.645,"15min":2.8849999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81699999999999995,"15min":0.81799999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.02,"5min":1.6140000000000001,"15min":1.867},"last":0.752}]},{"osd":7,"last update":"Sat Mar 30 00:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79900000000000004,"15min":0.84999999999999998},"min":{"1min":0.625,"5min":0.495,"15min":0.495},"max":{"1min":1.0429999999999999,"5min":1.2649999999999999,"15min":2.3690000000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81799999999999995,"15min":0.84699999999999998},"min":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.52700000000000002},"max":{"1min":1.0369999999999999,"5min":1.4990000000000001,"15min":2.851},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1355776,"data_stored":621318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.60799999999999998,"5min":0.72199999999999998,"15min":0.80300000000000005},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.55500000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.45600000000000002,"15min":0.46200000000000002},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69099999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69099999999999995,"5min":0.84099999999999997,"15min":0.89400000000000002},"last":0.47899999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.46899999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76000000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53600000000000003,"15min":0.495},"min":{"1min":0.38800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77400000000000002,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60799999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73699999999999999,"15min":0.72499999999999998},"min":{"1min":0.52200000000000002,"5min":0.51400000000000001,"15min":0.38800000000000001},"max":{"1min":1.0349999999999999,"5min":1.175,"15min":1.175},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.999,"5min":1.079,"15min":1.2},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.052,"5min":1.109,"15min":1.3180000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77100000000000002,"15min":0.76800000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.996,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78300000000000003,"15min":0.78400000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.503},"max":{"1min":0.97499999999999998,"5min":1.179,"15min":1.179},"last":0.627},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76800000000000002,"15min":0.77300000000000002},"min":{"1min":0.59399999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.2150000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.66000000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.016,"5min":1.2050000000000001,"15min":1.2669999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.56200000000000006,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98899999999999999,"5min":1.1180000000000001,"15min":1.236},"last":0.58699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574961,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1961984,"data_stored":1213306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.433,"15min":0.46300000000000002},"min":{"1min":0.33600000000000002,"5min":0.27400000000000002,"15min":0.245},"max":{"1min":0.71699999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.36699999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66400000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.33500000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.50600000000000001,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60299999999999998,"5min":1.02,"15min":1.02},"last":0.503},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39000000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67000000000000004,"5min":0.83099999999999996,"15min":1.006},"last":0.41299999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.39000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68400000000000005,"5min":0.71599999999999997,"15min":0.84799999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.40999999999999998,"15min":0.39400000000000002},"max":{"1min":0.81499999999999995,"5min":0.88900000000000001,"15min":0.96899999999999997},"last":0.435}]},{"osd":4,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81799999999999995,"15min":0.80400000000000005},"min":{"1min":0.57699999999999996,"5min":0.50600000000000001,"15min":0.47899999999999998},"max":{"1min":1.0780000000000001,"5min":1.137,"15min":1.5169999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.81299999999999994},"min":{"1min":0.70399999999999996,"5min":0.56000000000000005,"15min":0.46400000000000002},"max":{"1min":1.079,"5min":1.145,"15min":1.252},"last":0.749}]},{"osd":5,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.61399999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0189999999999999,"5min":1.1919999999999999,"15min":1.3080000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78700000000000003,"15min":0.78000000000000003},"min":{"1min":0.65900000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.022,"5min":1.1439999999999999,"15min":1.462},"last":0.877}]},{"osd":6,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.66300000000000003,"5min":0.57799999999999996,"15min":0.54500000000000004},"max":{"1min":1.131,"5min":1.131,"15min":1.5920000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.67200000000000004,"5min":0.51500000000000001,"15min":0.46999999999999997},"max":{"1min":1.0549999999999999,"5min":1.1659999999999999,"15min":1.2230000000000001},"last":0.81999999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81399999999999995,"15min":0.81299999999999994},"min":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.3600000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.65900000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94899999999999995,"5min":1.2070000000000001,"15min":1.556},"last":0.76100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:48:50.839 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T00:48:50.839 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-30T00:48:51.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.013+0000 7f0602000640 1 Processor -- start 2024-03-30T00:48:51.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.013+0000 7f0602000640 1 -- start start 2024-03-30T00:48:51.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.013+0000 7f0602000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc0ce780 0x7f05fc0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.013+0000 7f0602000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc0cbc50 con 0x7f05fc0b1db0 2024-03-30T00:48:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.013+0000 7f0602000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc0ca400 con 0x7f05fc15a770 2024-03-30T00:48:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.013+0000 7f0602000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0cb120 con 0x7f05fc0ce780 2024-03-30T00:48:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.013+0000 7f05f3e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f05fc0b1db0 0x7f05fc0cb540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53940/0 (socket says 172.21.15.66:53940) 2024-03-30T00:48:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.013+0000 7f05f3e00640 1 -- 172.21.15.66:0/1222388023 learned_addr learned my addr 172.21.15.66:0/1222388023 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05fbe00640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc0ce780 0x7f05fc0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050557593 0 0) 0x7f05fc0cbc50 con 0x7f05fc0b1db0 2024-03-30T00:48:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e0003530 con 0x7f05fc0b1db0 2024-03-30T00:48:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065819466 0 0) 0x7f05fc0ca400 con 0x7f05fc15a770 2024-03-30T00:48:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05fc0cbc50 con 0x7f05fc15a770 2024-03-30T00:48:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3367746014 0 0) 0x7f05e0003530 con 0x7f05fc0b1db0 2024-03-30T00:48:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc0ce780 msgr2=0x7f05fc0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc0ce780 0x7f05fc0cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 >> v1:172.21.15.156:6789/0 conn(0x7f05fc15a770 legacy=0x7f05fc0c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0ca6e0 con 0x7f05fc0b1db0 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f05e40016a0 con 0x7f05fc0b1db0 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 >> v1:172.21.15.66:6789/0 conn(0x7f05fc0b1db0 legacy=0x7f05fc0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc0ce780 0x7f05e0003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 0x7f05e00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05e0004d10 0x7f05e0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0cb120 con 0x7f05e0004080 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e000ba40 con 0x7f05e0004d10 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e000bc00 con 0x7f05fc0ce780 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05f3e00640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 0x7f05e00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05fb400640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc0ce780 0x7f05e0003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05fbe00640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05e0004d10 0x7f05e0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05f3e00640 1 -- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc0ce780 msgr2=0x7f05e0003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05f3e00640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc0ce780 0x7f05e0003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05f3e00640 1 -- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05e0004d10 msgr2=0x7f05e0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:51.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05f3e00640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05e0004d10 0x7f05e0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05f3e00640 1 -- 172.21.15.66:0/1222388023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e0003530 con 0x7f05e0004080 2024-03-30T00:48:51.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.017+0000 7f05fbe00640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05e0004d10 0x7f05e0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:51.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f05f3e00640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 0x7f05e00047d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0c8980 tx=0x7f05e401d0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc340486455fe3a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:51.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4005ae0 con 0x7f05e0004080 2024-03-30T00:48:51.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e40a8070 con 0x7f05e0004080 2024-03-30T00:48:51.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f05faa00640 1 -- 172.21.15.66:0/1222388023 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4005ea0 con 0x7f05e0004080 2024-03-30T00:48:51.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 -- 172.21.15.66:0/1222388023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 msgr2=0x7f05e00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 --2- 172.21.15.66:0/1222388023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 0x7f05e00047d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0c8980 tx=0x7f05e401d0c0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f05fbe00640 1 -- 172.21.15.66:0/1222388023 reap_dead start 2024-03-30T00:48:51.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 -- 172.21.15.66:0/1222388023 shutdown_connections 2024-03-30T00:48:51.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 -- 172.21.15.66:0/1222388023 >> 172.21.15.66:0/1222388023 conn(0x7f05fc06a190 msgr2=0x7f05fc06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:51.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 -- 172.21.15.66:0/1222388023 shutdown_connections 2024-03-30T00:48:51.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 -- 172.21.15.66:0/1222388023 wait complete. 2024-03-30T00:48:51.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 Processor -- start 2024-03-30T00:48:51.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 -- start start 2024-03-30T00:48:51.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 0x7f05fc154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc1560e0 0x7f05fc0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05fc0cd770 0x7f05fc1f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0cdbf0 con 0x7f05e0004080 2024-03-30T00:48:51.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0c5aa0 con 0x7f05fc0cd770 2024-03-30T00:48:51.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.021+0000 7f0602000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0ad790 con 0x7f05fc1560e0 2024-03-30T00:48:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 0x7f05fc154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 0x7f05fc154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41706/0 (socket says 172.21.15.66:41706) 2024-03-30T00:48:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05fb400640 1 -- 172.21.15.66:0/977221125 learned_addr learned my addr 172.21.15.66:0/977221125 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05f3e00640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc1560e0 0x7f05fc0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05fbe00640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05fc0cd770 0x7f05fc1f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05fb400640 1 -- 172.21.15.66:0/977221125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc1560e0 msgr2=0x7f05fc0cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05fb400640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc1560e0 0x7f05fc0cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05fb400640 1 -- 172.21.15.66:0/977221125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05fc0cd770 msgr2=0x7f05fc1f1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05fb400640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05fc0cd770 0x7f05fc1f1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05fb400640 1 -- 172.21.15.66:0/977221125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc1f2ad0 con 0x7f05e0004080 2024-03-30T00:48:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05fbe00640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05fc0cd770 0x7f05fc1f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05f3e00640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc1560e0 0x7f05fc0cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:51.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05fb400640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 0x7f05fc154370 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f05ec012970 tx=0x7f05ec012e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05f3400640 1 -- 172.21.15.66:0/977221125 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec0070a0 con 0x7f05e0004080 2024-03-30T00:48:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05f3400640 1 -- 172.21.15.66:0/977221125 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec010c30 con 0x7f05e0004080 2024-03-30T00:48:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f0602000640 1 -- 172.21.15.66:0/977221125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc1f2db0 con 0x7f05e0004080 2024-03-30T00:48:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05f3400640 1 -- 172.21.15.66:0/977221125 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec016480 con 0x7f05e0004080 2024-03-30T00:48:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f0602000640 1 -- 172.21.15.66:0/977221125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc0c5730 con 0x7f05e0004080 2024-03-30T00:48:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f0602000640 1 -- 172.21.15.66:0/977221125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc066c30 con 0x7f05e0004080 2024-03-30T00:48:51.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05f3400640 1 -- 172.21.15.66:0/977221125 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f05ec0165e0 con 0x7f05e0004080 2024-03-30T00:48:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05f3400640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05d8066690 0x7f05d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05f3400640 1 -- 172.21.15.66:0/977221125 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f05ec087c70 con 0x7f05e0004080 2024-03-30T00:48:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.025+0000 7f05f3e00640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05d8066690 0x7f05d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.029+0000 7f05f3e00640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05d8066690 0x7f05d8068b50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0c8980 tx=0x7f05e4001e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:51.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.029+0000 7f05f3400640 1 -- 172.21.15.66:0/977221125 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f05ec054e00 con 0x7f05e0004080 2024-03-30T00:48:51.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.245+0000 7f0602000640 1 -- 172.21.15.66:0/977221125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f05fc0ae1e0 con 0x7f05e0004080 2024-03-30T00:48:51.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.249+0000 7f05f3400640 1 -- 172.21.15.66:0/977221125 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f05ec058420 con 0x7f05e0004080 2024-03-30T00:48:51.259 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-03-30T00:48:51.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 -- 172.21.15.66:0/977221125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05d8066690 msgr2=0x7f05d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05d8066690 0x7f05d8068b50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0c8980 tx=0x7f05e4001e80 comp rx=0 tx=0).stop 2024-03-30T00:48:51.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 -- 172.21.15.66:0/977221125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 msgr2=0x7f05fc154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 0x7f05fc154370 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f05ec012970 tx=0x7f05ec012e30 comp rx=0 tx=0).stop 2024-03-30T00:48:51.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 -- 172.21.15.66:0/977221125 shutdown_connections 2024-03-30T00:48:51.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05fc0cd770 0x7f05fc1f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05fc1560e0 0x7f05fc0cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05d8066690 0x7f05d8068b50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 --2- 172.21.15.66:0/977221125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05e0004080 0x7f05fc154370 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 -- 172.21.15.66:0/977221125 >> 172.21.15.66:0/977221125 conn(0x7f05fc06a190 msgr2=0x7f05fc0ae400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:51.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 -- 172.21.15.66:0/977221125 shutdown_connections 2024-03-30T00:48:51.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.253+0000 7f0602000640 1 -- 172.21.15.66:0/977221125 wait complete. 2024-03-30T00:48:51.281 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:48:51.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52e000640 1 Processor -- start 2024-03-30T00:48:51.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52e000640 1 -- start start 2024-03-30T00:48:51.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52e000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa52815ca30 0x7fa52806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52e000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa52806d710 0x7fa528069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52e000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa5280cda40 0x7fa52806a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52e000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa528160590 con 0x7fa5280cda40 2024-03-30T00:48:51.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52e000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52806db90 con 0x7fa52815ca30 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52e000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa528160850 con 0x7fa52806d710 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52d600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa5280cda40 0x7fa52806a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52d600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa5280cda40 0x7fa52806a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41716/0 (socket says 172.21.15.66:41716) 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52d600640 1 -- 172.21.15.66:0/4250293336 learned_addr learned my addr 172.21.15.66:0/4250293336 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52cc00640 1 --2- 172.21.15.66:0/4250293336 >> v2:172.21.15.156:3300/0 conn(0x7fa52815ca30 0x7fa52806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa527e00640 1 --2- 172.21.15.66:0/4250293336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa52806d710 0x7fa528069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52d600640 1 -- 172.21.15.66:0/4250293336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa52806d710 msgr2=0x7fa528069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52d600640 1 --2- 172.21.15.66:0/4250293336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa52806d710 0x7fa528069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52d600640 1 -- 172.21.15.66:0/4250293336 >> v2:172.21.15.156:3300/0 conn(0x7fa52815ca30 msgr2=0x7fa52806d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52d600640 1 --2- 172.21.15.66:0/4250293336 >> v2:172.21.15.156:3300/0 conn(0x7fa52815ca30 0x7fa52806d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa52d600640 1 -- 172.21.15.66:0/4250293336 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa528160b30 con 0x7fa5280cda40 2024-03-30T00:48:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.413+0000 7fa527e00640 1 --2- 172.21.15.66:0/4250293336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa52806d710 0x7fa528069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:51.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52d600640 1 --2- 172.21.15.66:0/4250293336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5280cda40 0x7fa52806a2d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa528072770 tx=0x7fa52001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bf84764025a729 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa527400640 1 -- 172.21.15.66:0/4250293336 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5200a5860 con 0x7fa5280cda40 2024-03-30T00:48:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa527400640 1 -- 172.21.15.66:0/4250293336 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5200a59c0 con 0x7fa5280cda40 2024-03-30T00:48:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa527400640 1 -- 172.21.15.66:0/4250293336 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5200a5d50 con 0x7fa5280cda40 2024-03-30T00:48:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52e000640 1 -- 172.21.15.66:0/4250293336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5280cda40 msgr2=0x7fa52806a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52e000640 1 --2- 172.21.15.66:0/4250293336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5280cda40 0x7fa52806a2d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa528072770 tx=0x7fa52001c9c0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52e000640 1 -- 172.21.15.66:0/4250293336 shutdown_connections 2024-03-30T00:48:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52e000640 1 --2- 172.21.15.66:0/4250293336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5280cda40 0x7fa52806a2d0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52e000640 1 --2- 172.21.15.66:0/4250293336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa52806d710 0x7fa528069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52e000640 1 --2- 172.21.15.66:0/4250293336 >> v2:172.21.15.156:3300/0 conn(0x7fa52815ca30 0x7fa52806d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52e000640 1 -- 172.21.15.66:0/4250293336 >> 172.21.15.66:0/4250293336 conn(0x7fa5280cbdc0 msgr2=0x7fa5280cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52e000640 1 -- 172.21.15.66:0/4250293336 shutdown_connections 2024-03-30T00:48:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52e000640 1 -- 172.21.15.66:0/4250293336 wait complete. 2024-03-30T00:48:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52e000640 1 Processor -- start 2024-03-30T00:48:51.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.417+0000 7fa52e000640 1 -- start start 2024-03-30T00:48:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa52806d710 0x7fa5280c4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52e000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5280cda40 0x7fa5281585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52e000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa52815ca30 0x7fa52814f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52e000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa528158e40 con 0x7fa52806d710 2024-03-30T00:48:51.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52e000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa528159120 con 0x7fa5280cda40 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52e000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa528159400 con 0x7fa52815ca30 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa52806d710 0x7fa5280c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa52806d710 0x7fa5280c4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41724/0 (socket says 172.21.15.66:41724) 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52cc00640 1 -- 172.21.15.66:0/2657731745 learned_addr learned my addr 172.21.15.66:0/2657731745 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa527e00640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5280cda40 0x7fa5281585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52d600640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa52815ca30 0x7fa52814f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52cc00640 1 -- 172.21.15.66:0/2657731745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa52815ca30 msgr2=0x7fa52814f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52cc00640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa52815ca30 0x7fa52814f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52cc00640 1 -- 172.21.15.66:0/2657731745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5280cda40 msgr2=0x7fa5281585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52cc00640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5280cda40 0x7fa5281585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52cc00640 1 -- 172.21.15.66:0/2657731745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa520009b30 con 0x7fa52806d710 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52cc00640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa52806d710 0x7fa5280c4590 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00d5f0 tx=0x7fa51c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa524c00640 1 -- 172.21.15.66:0/2657731745 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0125f0 con 0x7fa52806d710 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa524c00640 1 -- 172.21.15.66:0/2657731745 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c012750 con 0x7fa52806d710 2024-03-30T00:48:51.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa524c00640 1 -- 172.21.15.66:0/2657731745 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c012ae0 con 0x7fa52806d710 2024-03-30T00:48:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52e000640 1 -- 172.21.15.66:0/2657731745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5281596e0 con 0x7fa52806d710 2024-03-30T00:48:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52e000640 1 -- 172.21.15.66:0/2657731745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa528158b30 con 0x7fa52806d710 2024-03-30T00:48:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa52e000640 1 -- 172.21.15.66:0/2657731745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa528067110 con 0x7fa52806d710 2024-03-30T00:48:51.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.421+0000 7fa524c00640 1 -- 172.21.15.66:0/2657731745 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa51c012c40 con 0x7fa52806d710 2024-03-30T00:48:51.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.425+0000 7fa524c00640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa508066690 0x7fa508068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:51.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.425+0000 7fa524c00640 1 -- 172.21.15.66:0/2657731745 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa51c0846c0 con 0x7fa52806d710 2024-03-30T00:48:51.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.425+0000 7fa527e00640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa508066690 0x7fa508068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:51.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.425+0000 7fa527e00640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa508066690 0x7fa508068b50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa528156750 tx=0x7fa514008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:51.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.429+0000 7fa524c00640 1 -- 172.21.15.66:0/2657731745 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa51c051990 con 0x7fa52806d710 2024-03-30T00:48:51.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 -- 172.21.15.66:0/2657731745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa528073cf0 con 0x7fa52806d710 2024-03-30T00:48:51.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa524c00640 1 -- 172.21.15.66:0/2657731745 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4739 (secure 0 0 0) 0x7fa51c054fb0 con 0x7fa52806d710 2024-03-30T00:48:51.638 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:51.638 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":79,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:48:49.341688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8646":{"gid":8646,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/2988412199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2988412199},{"type":"v1","addr":"172.21.15.66:6837","nonce":2988412199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:48:51.638 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 79 2024-03-30T00:48:51.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 -- 172.21.15.66:0/2657731745 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa508066690 msgr2=0x7fa508068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa508066690 0x7fa508068b50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa528156750 tx=0x7fa514008040 comp rx=0 tx=0).stop 2024-03-30T00:48:51.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 -- 172.21.15.66:0/2657731745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa52806d710 msgr2=0x7fa5280c4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:51.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa52806d710 0x7fa5280c4590 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00d5f0 tx=0x7fa51c00dab0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 -- 172.21.15.66:0/2657731745 shutdown_connections 2024-03-30T00:48:51.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa52815ca30 0x7fa52814f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5280cda40 0x7fa5281585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa508066690 0x7fa508068b50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 --2- 172.21.15.66:0/2657731745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa52806d710 0x7fa5280c4590 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:51.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 -- 172.21.15.66:0/2657731745 >> 172.21.15.66:0/2657731745 conn(0x7fa5280cbdc0 msgr2=0x7fa5280cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:51.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 -- 172.21.15.66:0/2657731745 shutdown_connections 2024-03-30T00:48:51.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:51.629+0000 7fa52e000640 1 -- 172.21.15.66:0/2657731745 wait complete. 2024-03-30T00:48:52.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:48:46.332321+0000', 'modified': '2024-03-30T00:48:49.341688+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8646}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8646': {'gid': 8646, 'name': 'c', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6837/2988412199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 2988412199}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 2988412199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:48:52.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:48:52.063 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T00:48:52.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.193+0000 7efc00c00640 1 Processor -- start 2024-03-30T00:48:52.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.193+0000 7efc00c00640 1 -- start start 2024-03-30T00:48:52.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efc00c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efbfc0cde70 0x7efbfc15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efc00c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbfc0cd3e0 con 0x7efbfc0cda90 2024-03-30T00:48:52.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efc00c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbfc15a990 con 0x7efbfc0cec30 2024-03-30T00:48:52.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efc00c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbfc0b1b80 con 0x7efbfc0cde70 2024-03-30T00:48:52.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efbf9e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efbfc0cda90 0x7efbfc0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53952/0 (socket says 172.21.15.66:53952) 2024-03-30T00:48:52.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efbf9e00640 1 -- 172.21.15.66:0/1436594707 learned_addr learned my addr 172.21.15.66:0/1436594707 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:52.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efbfa800640 1 --2- 172.21.15.66:0/1436594707 >> v2:172.21.15.156:3300/0 conn(0x7efbfc0cde70 0x7efbfc15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efbf9400640 1 -- 172.21.15.66:0/1436594707 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925963147 0 0) 0x7efbfc0cd3e0 con 0x7efbfc0cda90 2024-03-30T00:48:52.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efbf9400640 1 -- 172.21.15.66:0/1436594707 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbe0005740 con 0x7efbfc0cda90 2024-03-30T00:48:52.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efbfa800640 1 -- 172.21.15.66:0/1436594707 >> v1:172.21.15.156:6789/0 conn(0x7efbfc0cec30 legacy=0x7efbfc159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efbf9400640 1 -- 172.21.15.66:0/1436594707 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592158216 0 0) 0x7efbfc15a990 con 0x7efbfc0cec30 2024-03-30T00:48:52.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efbfa800640 1 -- 172.21.15.66:0/1436594707 >> v1:172.21.15.66:6789/0 conn(0x7efbfc0cda90 legacy=0x7efbfc0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.197+0000 7efbfa800640 1 -- 172.21.15.66:0/1436594707 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc0b11e0 con 0x7efbfc0cde70 2024-03-30T00:48:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efbfa800640 1 --2- 172.21.15.66:0/1436594707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0cde70 0x7efbfc15a370 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efbf40097c0 tx=0x7efbf401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90182e8b06d0f487 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efbf9400640 1 -- 172.21.15.66:0/1436594707 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40ab610 con 0x7efbfc0cde70 2024-03-30T00:48:52.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efbf9400640 1 -- 172.21.15.66:0/1436594707 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf40ab770 con 0x7efbfc0cde70 2024-03-30T00:48:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efbf9400640 1 -- 172.21.15.66:0/1436594707 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40abae0 con 0x7efbfc0cde70 2024-03-30T00:48:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 -- 172.21.15.66:0/1436594707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0cde70 msgr2=0x7efbfc15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 --2- 172.21.15.66:0/1436594707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0cde70 0x7efbfc15a370 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efbf40097c0 tx=0x7efbf401c410 comp rx=0 tx=0).stop 2024-03-30T00:48:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 -- 172.21.15.66:0/1436594707 shutdown_connections 2024-03-30T00:48:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 --2- 172.21.15.66:0/1436594707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0cde70 0x7efbfc15a370 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 -- 172.21.15.66:0/1436594707 >> 172.21.15.66:0/1436594707 conn(0x7efbfc069f60 msgr2=0x7efbfc06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 -- 172.21.15.66:0/1436594707 shutdown_connections 2024-03-30T00:48:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 -- 172.21.15.66:0/1436594707 wait complete. 2024-03-30T00:48:52.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 Processor -- start 2024-03-30T00:48:52.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 -- start start 2024-03-30T00:48:52.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efbfc0cda90 0x7efbfc164df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0cde70 0x7efbfc06fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efbfc0cec30 0x7efbfc070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc165330 con 0x7efbfc0cec30 2024-03-30T00:48:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc14f980 con 0x7efbfc0cde70 2024-03-30T00:48:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.201+0000 7efc00c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc150950 con 0x7efbfc0cda90 2024-03-30T00:48:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbfa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efbfc0cec30 0x7efbfc070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbfa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efbfc0cec30 0x7efbfc070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41728/0 (socket says 172.21.15.66:41728) 2024-03-30T00:48:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbfa800640 1 -- 172.21.15.66:0/4155818331 learned_addr learned my addr 172.21.15.66:0/4155818331 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbf1400640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0cde70 0x7efbfc06fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbf9e00640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efbfc0cda90 0x7efbfc164df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbfa800640 1 -- 172.21.15.66:0/4155818331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efbfc0cda90 msgr2=0x7efbfc164df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbfa800640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efbfc0cda90 0x7efbfc164df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbfa800640 1 -- 172.21.15.66:0/4155818331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0cde70 msgr2=0x7efbfc06fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbfa800640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0cde70 0x7efbfc06fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbfa800640 1 -- 172.21.15.66:0/4155818331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf4009420 con 0x7efbfc0cec30 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbfa800640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efbfc0cec30 0x7efbfc070410 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7efbf40ad040 tx=0x7efbf401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbf2a00640 1 -- 172.21.15.66:0/4155818331 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40bf610 con 0x7efbfc0cec30 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbf2a00640 1 -- 172.21.15.66:0/4155818331 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf40bf770 con 0x7efbfc0cec30 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbf2a00640 1 -- 172.21.15.66:0/4155818331 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40bfae0 con 0x7efbfc0cec30 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efc00c00640 1 -- 172.21.15.66:0/4155818331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbfc150020 con 0x7efbfc0cec30 2024-03-30T00:48:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efc00c00640 1 -- 172.21.15.66:0/4155818331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc1505e0 con 0x7efbfc0cec30 2024-03-30T00:48:52.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbf2a00640 1 -- 172.21.15.66:0/4155818331 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efbf40bfc40 con 0x7efbfc0cec30 2024-03-30T00:48:52.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.205+0000 7efbf2a00640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efbd80666c0 0x7efbd8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbf2a00640 1 -- 172.21.15.66:0/4155818331 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efbf411eb60 con 0x7efbfc0cec30 2024-03-30T00:48:52.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbf9e00640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efbd80666c0 0x7efbd8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 Processor -- start 2024-03-30T00:48:52.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 -- start start 2024-03-30T00:48:52.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbf9e00640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efbd80666c0 0x7efbd8068b80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efbec0012f0 tx=0x7efbec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:52.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18a1db0 0x5611c18a2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18a26d0 0x5611c1857f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18584c0 0x5611c18588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c17205a0 con 0x5611c18584c0 2024-03-30T00:48:52.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c16adaf0 con 0x5611c18a1db0 2024-03-30T00:48:52.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611c16ab0c0 con 0x5611c18a26d0 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbfa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18584c0 0x5611c18588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbf1400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18a26d0 0x5611c1857f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbfa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18584c0 0x5611c18588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41744/0 (socket says 172.21.15.66:41744) 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbfa800640 1 -- 172.21.15.66:0/563977928 learned_addr learned my addr 172.21.15.66:0/563977928 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbf9e00640 1 --2- 172.21.15.66:0/563977928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18a1db0 0x5611c18a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbfa800640 1 -- 172.21.15.66:0/563977928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18a26d0 msgr2=0x5611c1857f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbfa800640 1 --2- 172.21.15.66:0/563977928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18a26d0 0x5611c1857f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbfa800640 1 -- 172.21.15.66:0/563977928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18a1db0 msgr2=0x5611c18a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbfa800640 1 --2- 172.21.15.66:0/563977928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18a1db0 0x5611c18a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbfa800640 1 -- 172.21.15.66:0/563977928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611c13a2890 con 0x5611c18584c0 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbfa800640 1 --2- 172.21.15.66:0/563977928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18584c0 0x5611c18588a0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7efbf40f7870 tx=0x7efbf40ef310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6801f25b8f9693e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbeaa00640 1 -- 172.21.15.66:0/563977928 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40f37e0 con 0x5611c18584c0 2024-03-30T00:48:52.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbeaa00640 1 -- 172.21.15.66:0/563977928 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf40f7da0 con 0x5611c18584c0 2024-03-30T00:48:52.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efbeaa00640 1 -- 172.21.15.66:0/563977928 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40ec0c0 con 0x5611c18584c0 2024-03-30T00:48:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 -- 172.21.15.66:0/563977928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18584c0 msgr2=0x5611c18588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 --2- 172.21.15.66:0/563977928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18584c0 0x5611c18588a0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7efbf40f7870 tx=0x7efbf40ef310 comp rx=0 tx=0).stop 2024-03-30T00:48:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 -- 172.21.15.66:0/563977928 shutdown_connections 2024-03-30T00:48:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 --2- 172.21.15.66:0/563977928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18584c0 0x5611c18588a0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 --2- 172.21.15.66:0/563977928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18a26d0 0x5611c1857f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 --2- 172.21.15.66:0/563977928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18a1db0 0x5611c18a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 -- 172.21.15.66:0/563977928 >> 172.21.15.66:0/563977928 conn(0x5611c185e830 msgr2=0x5611c1856860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 -- 172.21.15.66:0/563977928 shutdown_connections 2024-03-30T00:48:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.209+0000 7efc03f71000 1 -- 172.21.15.66:0/563977928 wait complete. 2024-03-30T00:48:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 -- start start 2024-03-30T00:48:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 Processor -- start 2024-03-30T00:48:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18584c0 0x5611c185cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18a1db0 0x5611c185d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a26d0 0x5611c18a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c16a69b0 con 0x5611c18a26d0 2024-03-30T00:48:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c1645c00 con 0x5611c18a1db0 2024-03-30T00:48:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611c1664070 con 0x5611c18584c0 2024-03-30T00:48:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 10 client.0 _ll_register_callbacks cb 0x5611c18440e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:48:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbfa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a26d0 0x5611c18a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbfa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a26d0 0x5611c18a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41748/0 (socket says 172.21.15.66:41748) 2024-03-30T00:48:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbf9e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18584c0 0x5611c185cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbfa800640 1 -- 172.21.15.66:0/2323817522 learned_addr learned my addr 172.21.15.66:0/2323817522 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:52.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbf1400640 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18a1db0 0x5611c185d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbfa800640 1 -- 172.21.15.66:0/2323817522 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18584c0 msgr2=0x5611c185cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbfa800640 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18584c0 0x5611c185cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbfa800640 1 -- 172.21.15.66:0/2323817522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18a1db0 msgr2=0x5611c185d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbfa800640 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18a1db0 0x5611c185d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbfa800640 1 -- 172.21.15.66:0/2323817522 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc1505e0 con 0x5611c18a26d0 2024-03-30T00:48:52.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbf9e00640 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18584c0 0x5611c185cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:52.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbfa800640 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a26d0 0x5611c18a2ab0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7efbf4097ee0 tx=0x7efbf401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e364503a10ec5f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:52.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbdf400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:48:52.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbdf400640 1 -- 172.21.15.66:0/2323817522 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf41073d0 con 0x5611c18a26d0 2024-03-30T00:48:52.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbdf400640 1 -- 172.21.15.66:0/2323817522 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efbf40f7da0 con 0x5611c18a26d0 2024-03-30T00:48:52.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 -- 172.21.15.66:0/2323817522 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611c16635f0 con 0x5611c18a26d0 2024-03-30T00:48:52.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbdf400640 1 -- 172.21.15.66:0/2323817522 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbf40f37e0 con 0x5611c18a26d0 2024-03-30T00:48:52.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 10 client.8775 fetch_fsmap learned FSMap version 79 2024-03-30T00:48:52.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 -- 172.21.15.66:0/2323817522 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5611c166bab0 con 0x5611c18a26d0 2024-03-30T00:48:52.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbdf400640 1 -- 172.21.15.66:0/2323817522 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5611c16635f0 con 0x5611c18a26d0 2024-03-30T00:48:52.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbdf400640 1 -- 172.21.15.66:0/2323817522 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 79) v1 ==== 3114+0+0 (secure 0 0 0) 0x7efbf40c8020 con 0x5611c18a26d0 2024-03-30T00:48:52.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 10 client.8775 fetch_fsmap finished waiting for FSMap version 79 2024-03-30T00:48:52.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 10 client.8775 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:48:52.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 20 client.8775 populate_metadata read hostname 'smithi066' 2024-03-30T00:48:52.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5611c18a34c0 0x5611c18b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 4 client.8775 mds_command: new command op to 8646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:48:52.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efc03f71000 1 -- 172.21.15.66:0/2323817522 --> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611c16a3790 con 0x5611c18a34c0 2024-03-30T00:48:52.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.213+0000 7efbf9e00640 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5611c18a34c0 0x5611c18b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efbf9e00640 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5611c18a34c0 0x5611c18b2e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b58910a19d35ec6 server_cookie=a5b41dce9154005a in_seq=0 out_seq=0 2024-03-30T00:48:52.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efbdf400640 10 client.8775 ms_handle_connect on v2:172.21.15.66:6835/2988412199 2024-03-30T00:48:52.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efbdf400640 1 -- 172.21.15.66:0/2323817522 <== mds.0 v2:172.21.15.66:6835/2988412199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5611c16a3790 con 0x5611c18a34c0 2024-03-30T00:48:52.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efbdf400640 10 client.8775 handle_command_reply: tid=0 2024-03-30T00:48:52.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 1 client.8775 shutdown 2024-03-30T00:48:52.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 1 -- 172.21.15.66:0/2323817522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a26d0 msgr2=0x5611c18a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a26d0 0x5611c18a2ab0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7efbf4097ee0 tx=0x7efbf401c410 comp rx=0 tx=0).stop 2024-03-30T00:48:52.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 1 -- 172.21.15.66:0/2323817522 shutdown_connections 2024-03-30T00:48:52.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a26d0 0x5611c18a2ab0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5611c18a34c0 0x5611c18b2e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18a1db0 0x5611c185d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 1 --2- 172.21.15.66:0/2323817522 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18584c0 0x5611c185cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efbdf400640 0 client.8775 ms_handle_reset on v2:172.21.15.66:6835/2988412199 2024-03-30T00:48:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 1 -- 172.21.15.66:0/2323817522 >> 172.21.15.66:0/2323817522 conn(0x5611c185e830 msgr2=0x5611c1856f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 1 -- 172.21.15.66:0/2323817522 shutdown_connections 2024-03-30T00:48:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 1 -- 172.21.15.66:0/2323817522 wait complete. 2024-03-30T00:48:52.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.217+0000 7efc03f71000 20 client.8775 trim_cache size 0 max 16384 2024-03-30T00:48:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 Processor -- start 2024-03-30T00:48:52.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 -- start start 2024-03-30T00:48:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c185e830 0x5611c18a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18584c0 0x5611c18a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c1363940 con 0x5611c18a1db0 2024-03-30T00:48:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c1643c10 con 0x5611c185e830 2024-03-30T00:48:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611c16e9ac0 con 0x5611c18584c0 2024-03-30T00:48:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbfa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbfa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41752/0 (socket says 172.21.15.66:41752) 2024-03-30T00:48:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbfa800640 1 -- 172.21.15.66:0/4079449669 learned_addr learned my addr 172.21.15.66:0/4079449669 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:52.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbf9e00640 1 --2- 172.21.15.66:0/4079449669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c185e830 0x5611c18a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbf1400640 1 --2- 172.21.15.66:0/4079449669 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18584c0 0x5611c18a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbfa800640 1 -- 172.21.15.66:0/4079449669 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18584c0 msgr2=0x5611c18a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbfa800640 1 --2- 172.21.15.66:0/4079449669 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18584c0 0x5611c18a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbfa800640 1 -- 172.21.15.66:0/4079449669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c185e830 msgr2=0x5611c18a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbfa800640 1 --2- 172.21.15.66:0/4079449669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c185e830 0x5611c18a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbfa800640 1 -- 172.21.15.66:0/4079449669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611c1651c00 con 0x5611c18a1db0 2024-03-30T00:48:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbfa800640 1 --2- 172.21.15.66:0/4079449669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b0e40 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7efbf40ef940 tx=0x7efbf40c1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95017512d156f4e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbeb400640 1 -- 172.21.15.66:0/4079449669 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40f7da0 con 0x5611c18a1db0 2024-03-30T00:48:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbeb400640 1 -- 172.21.15.66:0/4079449669 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf40f37e0 con 0x5611c18a1db0 2024-03-30T00:48:52.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efbeb400640 1 -- 172.21.15.66:0/4079449669 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf41073d0 con 0x5611c18a1db0 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 -- 172.21.15.66:0/4079449669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 msgr2=0x5611c18b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 --2- 172.21.15.66:0/4079449669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b0e40 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7efbf40ef940 tx=0x7efbf40c1000 comp rx=0 tx=0).stop 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 -- 172.21.15.66:0/4079449669 shutdown_connections 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 --2- 172.21.15.66:0/4079449669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b0e40 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 --2- 172.21.15.66:0/4079449669 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c18584c0 0x5611c18a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 --2- 172.21.15.66:0/4079449669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c185e830 0x5611c18a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 -- 172.21.15.66:0/4079449669 >> 172.21.15.66:0/4079449669 conn(0x5611c18ae760 msgr2=0x5611c185ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 -- 172.21.15.66:0/4079449669 shutdown_connections 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.233+0000 7efc03f71000 1 -- 172.21.15.66:0/4079449669 wait complete. 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 -- start start 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 Processor -- start 2024-03-30T00:48:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18584c0 0x5611c18ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c185e830 0x5611c1900300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c16a9510 con 0x5611c18a1db0 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c18ae080 con 0x5611c18584c0 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611c16a41c0 con 0x5611c185e830 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 10 client.0 _ll_register_callbacks cb 0x5611c166b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbfa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbfa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41768/0 (socket says 172.21.15.66:41768) 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbfa800640 1 -- 172.21.15.66:0/1027193130 learned_addr learned my addr 172.21.15.66:0/1027193130 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbf9e00640 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18584c0 0x5611c18ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbf1400640 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c185e830 0x5611c1900300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbfa800640 1 -- 172.21.15.66:0/1027193130 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c185e830 msgr2=0x5611c1900300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbfa800640 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c185e830 0x5611c1900300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbfa800640 1 -- 172.21.15.66:0/1027193130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18584c0 msgr2=0x5611c18ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbfa800640 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18584c0 0x5611c18ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbfa800640 1 -- 172.21.15.66:0/1027193130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611c166bab0 con 0x5611c18a1db0 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbfa800640 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b3660 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7efbf4097ee0 tx=0x7efbf40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d811b761886aca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbdf400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbdf400640 1 -- 172.21.15.66:0/1027193130 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40f7da0 con 0x5611c18a1db0 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbdf400640 1 -- 172.21.15.66:0/1027193130 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efbf40ee910 con 0x5611c18a1db0 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 -- 172.21.15.66:0/1027193130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611c1676460 con 0x5611c18a1db0 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbdf400640 1 -- 172.21.15.66:0/1027193130 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbf40eeb10 con 0x5611c18a1db0 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 10 client.8781 fetch_fsmap learned FSMap version 79 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 -- 172.21.15.66:0/1027193130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5611c13a2cc0 con 0x5611c18a1db0 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbdf400640 1 -- 172.21.15.66:0/1027193130 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5611c1676460 con 0x5611c18a1db0 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbdf400640 1 -- 172.21.15.66:0/1027193130 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 79) v1 ==== 3114+0+0 (secure 0 0 0) 0x7efbf40c8070 con 0x5611c18a1db0 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 10 client.8781 fetch_fsmap finished waiting for FSMap version 79 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 10 client.8781 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:48:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 20 client.8781 populate_metadata read hostname 'smithi066' 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5611c18a26d0 0x5611c18b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 4 client.8781 mds_command: new command op to 8646 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 -- 172.21.15.66:0/1027193130 --> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5611c166f9c0 con 0x5611c18a26d0 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbf9e00640 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5611c18a26d0 0x5611c18b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbf9e00640 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5611c18a26d0 0x5611c18b3f30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7cf20369f924d5 server_cookie=65fcc8be55ccf1c7 in_seq=0 out_seq=0 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbdf400640 10 client.8781 ms_handle_connect on v2:172.21.15.66:6835/2988412199 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbdf400640 1 -- 172.21.15.66:0/1027193130 <== mds.0 v2:172.21.15.66:6835/2988412199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5611c166f9c0 con 0x5611c18a26d0 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efbdf400640 10 client.8781 handle_command_reply: tid=0 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.237+0000 7efc03f71000 1 client.8781 shutdown 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc03f71000 1 -- 172.21.15.66:0/1027193130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 msgr2=0x5611c18b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc03f71000 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b3660 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7efbf4097ee0 tx=0x7efbf40191b0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc03f71000 1 -- 172.21.15.66:0/1027193130 shutdown_connections 2024-03-30T00:48:52.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc03f71000 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5611c18a1db0 0x5611c18b3660 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc03f71000 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5611c185e830 0x5611c1900300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc03f71000 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5611c18a26d0 0x5611c18b3f30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc03f71000 1 --2- 172.21.15.66:0/1027193130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611c18584c0 0x5611c18ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc03f71000 1 -- 172.21.15.66:0/1027193130 >> 172.21.15.66:0/1027193130 conn(0x5611c18ae760 msgr2=0x5611c185d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:52.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efbdf400640 0 client.8781 ms_handle_reset on v2:172.21.15.66:6835/2988412199 2024-03-30T00:48:52.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc03f71000 1 -- 172.21.15.66:0/1027193130 shutdown_connections 2024-03-30T00:48:52.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc03f71000 1 -- 172.21.15.66:0/1027193130 wait complete. 2024-03-30T00:48:52.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc03f71000 20 client.8781 trim_cache size 0 max 16384 2024-03-30T00:48:52.255 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:48:52.255 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:48:52.255 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:48:52.256 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 8646, 2024-03-30T00:48:52.256 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:48:52.256 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:48:52.256 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:48:52.256 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 5.9094494600000003, 2024-03-30T00:48:52.256 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 79, 2024-03-30T00:48:52.256 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 110, 2024-03-30T00:48:52.256 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 110, 2024-03-30T00:48:52.256 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 10.357749453 2024-03-30T00:48:52.256 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:48:52.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 -- 172.21.15.66:0/4155818331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efbd80666c0 msgr2=0x7efbd8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efbd80666c0 0x7efbd8068b80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efbec0012f0 tx=0x7efbec009040 comp rx=0 tx=0).stop 2024-03-30T00:48:52.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 -- 172.21.15.66:0/4155818331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efbfc0cec30 msgr2=0x7efbfc070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efbfc0cec30 0x7efbfc070410 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7efbf40ad040 tx=0x7efbf401ac50 comp rx=0 tx=0).stop 2024-03-30T00:48:52.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 -- 172.21.15.66:0/4155818331 shutdown_connections 2024-03-30T00:48:52.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7efbd80666c0 0x7efbd8068b80 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efbfc0cec30 0x7efbfc070410 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0cde70 0x7efbfc06fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 --2- 172.21.15.66:0/4155818331 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efbfc0cda90 0x7efbfc164df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 -- 172.21.15.66:0/4155818331 >> 172.21.15.66:0/4155818331 conn(0x7efbfc069f60 msgr2=0x7efbfc15af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:52.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 -- 172.21.15.66:0/4155818331 shutdown_connections 2024-03-30T00:48:52.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.241+0000 7efc00c00640 1 -- 172.21.15.66:0/4155818331 wait complete. 2024-03-30T00:48:52.269 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:48:52.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.397+0000 7fb2f3a00640 1 Processor -- start 2024-03-30T00:48:52.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.397+0000 7fb2f3a00640 1 -- start start 2024-03-30T00:48:52.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f3a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb2ec06cdc0 0x7fb2ec06d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f3a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0cc350 0x7fb2ec071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f3a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ec160540 con 0x7fb2ec0cbec0 2024-03-30T00:48:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f3a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ec0cb8c0 con 0x7fb2ec06cdc0 2024-03-30T00:48:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f3a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec15deb0 con 0x7fb2ec0cc350 2024-03-30T00:48:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2e3e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb2ec0cbec0 0x7fb2ec0713c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53960/0 (socket says 172.21.15.66:53960) 2024-03-30T00:48:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2e3e00640 1 -- 172.21.15.66:0/2550350997 learned_addr learned my addr 172.21.15.66:0/2550350997 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f1a00640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0cc350 0x7fb2ec071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f1000640 1 --2- 172.21.15.66:0/2550350997 >> v2:172.21.15.156:3300/0 conn(0x7fb2ec06cdc0 0x7fb2ec06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514245399 0 0) 0x7fb2ec160540 con 0x7fb2ec0cbec0 2024-03-30T00:48:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d0003530 con 0x7fb2ec0cbec0 2024-03-30T00:48:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3467968017 0 0) 0x7fb2d0003530 con 0x7fb2ec0cbec0 2024-03-30T00:48:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0cc350 msgr2=0x7fb2ec071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0cc350 0x7fb2ec071ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 >> v2:172.21.15.156:3300/0 conn(0x7fb2ec06cdc0 msgr2=0x7fb2ec06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 --2- 172.21.15.66:0/2550350997 >> v2:172.21.15.156:3300/0 conn(0x7fb2ec06cdc0 0x7fb2ec06d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec160790 con 0x7fb2ec0cbec0 2024-03-30T00:48:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fb2d4001680 con 0x7fb2ec0cbec0 2024-03-30T00:48:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 >> v1:172.21.15.66:6789/0 conn(0x7fb2ec0cbec0 legacy=0x7fb2ec0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d0003980 0x7fb2d0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d00047d0 0x7fb2d0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0cc350 0x7fb2d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0cb8c0 con 0x7fb2d0003980 2024-03-30T00:48:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec15deb0 con 0x7fb2d00047d0 2024-03-30T00:48:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d000b990 con 0x7fb2ec0cc350 2024-03-30T00:48:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f1000640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d0003980 0x7fb2d0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2e3e00640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d00047d0 0x7fb2d0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f1a00640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0cc350 0x7fb2d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f1000640 1 -- 172.21.15.66:0/2550350997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0cc350 msgr2=0x7fb2d0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f1000640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0cc350 0x7fb2d0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f1000640 1 -- 172.21.15.66:0/2550350997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d00047d0 msgr2=0x7fb2d0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f1000640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d00047d0 0x7fb2d0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.401+0000 7fb2f1000640 1 -- 172.21.15.66:0/2550350997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0003530 con 0x7fb2d0003980 2024-03-30T00:48:52.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2f1000640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d0003980 0x7fb2d0003da0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009ae0 tx=0x7fb2dc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05c3087a77e1fc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:52.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0a64b0 con 0x7fb2d0003980 2024-03-30T00:48:52.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc0a6610 con 0x7fb2d0003980 2024-03-30T00:48:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2ebe00640 1 -- 172.21.15.66:0/2550350997 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0a69c0 con 0x7fb2d0003980 2024-03-30T00:48:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2550350997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d0003980 msgr2=0x7fb2d0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2f3a00640 1 --2- 172.21.15.66:0/2550350997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d0003980 0x7fb2d0003da0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009ae0 tx=0x7fb2dc01cab0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2f1a00640 1 -- 172.21.15.66:0/2550350997 reap_dead start 2024-03-30T00:48:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2550350997 shutdown_connections 2024-03-30T00:48:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2550350997 >> 172.21.15.66:0/2550350997 conn(0x7fb2ec0ce2f0 msgr2=0x7fb2ec0ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2550350997 shutdown_connections 2024-03-30T00:48:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2550350997 wait complete. 2024-03-30T00:48:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2f3a00640 1 Processor -- start 2024-03-30T00:48:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.405+0000 7fb2f3a00640 1 -- start start 2024-03-30T00:48:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2f3a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d0003980 0x7fb2ec0c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2f3a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2ec15a350 0x7fb2ec0c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2f3a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0b3710 0x7fb2ec15f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2f3a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0c4560 con 0x7fb2ec15a350 2024-03-30T00:48:52.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2f3a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0c4790 con 0x7fb2d0003980 2024-03-30T00:48:52.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2f3a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0c2390 con 0x7fb2ec0b3710 2024-03-30T00:48:52.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2ec15a350 0x7fb2ec0c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2ec15a350 0x7fb2ec0c2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41784/0 (socket says 172.21.15.66:41784) 2024-03-30T00:48:52.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e3e00640 1 -- 172.21.15.66:0/2668368696 learned_addr learned my addr 172.21.15.66:0/2668368696 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:52.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e3e00640 1 -- 172.21.15.66:0/2668368696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0b3710 msgr2=0x7fb2ec15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e3e00640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0b3710 0x7fb2ec15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e3e00640 1 -- 172.21.15.66:0/2668368696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d0003980 msgr2=0x7fb2ec0c4020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T00:48:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e3e00640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d0003980 0x7fb2ec0c4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e3e00640 1 -- 172.21.15.66:0/2668368696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0c2670 con 0x7fb2ec15a350 2024-03-30T00:48:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e3e00640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2ec15a350 0x7fb2ec0c2e70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400e760 tx=0x7fb2d400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e9600640 1 -- 172.21.15.66:0/2668368696 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d400ee70 con 0x7fb2ec15a350 2024-03-30T00:48:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e9600640 1 -- 172.21.15.66:0/2668368696 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d400c850 con 0x7fb2ec15a350 2024-03-30T00:48:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e9600640 1 -- 172.21.15.66:0/2668368696 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d400cc00 con 0x7fb2ec15a350 2024-03-30T00:48:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2668368696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec0c2950 con 0x7fb2ec15a350 2024-03-30T00:48:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2668368696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec0c4a70 con 0x7fb2ec15a350 2024-03-30T00:48:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2668368696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec0adac0 con 0x7fb2ec15a350 2024-03-30T00:48:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.409+0000 7fb2e9600640 1 -- 172.21.15.66:0/2668368696 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb2d401d460 con 0x7fb2ec15a350 2024-03-30T00:48:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.413+0000 7fb2e9600640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2c8066690 0x7fb2c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:52.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.413+0000 7fb2e9600640 1 -- 172.21.15.66:0/2668368696 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb2d4084260 con 0x7fb2ec15a350 2024-03-30T00:48:52.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.413+0000 7fb2f1000640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2c8066690 0x7fb2c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:52.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.413+0000 7fb2f1000640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2c8066690 0x7fb2c8068b50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0af000 tx=0x7fb2dc0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:52.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.417+0000 7fb2e9600640 1 -- 172.21.15.66:0/2668368696 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb2d40513f0 con 0x7fb2ec15a350 2024-03-30T00:48:52.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.609+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2668368696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb2ec0c2b90 con 0x7fb2ec15a350 2024-03-30T00:48:52.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2e9600640 1 -- 172.21.15.66:0/2668368696 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb2ec0c2b90 con 0x7fb2ec15a350 2024-03-30T00:48:52.621 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:48:52.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2668368696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2c8066690 msgr2=0x7fb2c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2c8066690 0x7fb2c8068b50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0af000 tx=0x7fb2dc0046d0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2668368696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2ec15a350 msgr2=0x7fb2ec0c2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2ec15a350 0x7fb2ec0c2e70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400e760 tx=0x7fb2d400ec20 comp rx=0 tx=0).stop 2024-03-30T00:48:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2668368696 shutdown_connections 2024-03-30T00:48:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2ec0b3710 0x7fb2ec15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb2c8066690 0x7fb2c8068b50 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2ec15a350 0x7fb2ec0c2e70 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 --2- 172.21.15.66:0/2668368696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2d0003980 0x7fb2ec0c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2668368696 >> 172.21.15.66:0/2668368696 conn(0x7fb2ec0ce2f0 msgr2=0x7fb2ec0713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2668368696 shutdown_connections 2024-03-30T00:48:52.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:52.613+0000 7fb2f3a00640 1 -- 172.21.15.66:0/2668368696 wait complete. 2024-03-30T00:48:52.642 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:48:52.642 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:48:52.643 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:48:52.643 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:48:52.643 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:48:52.643 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:48:52.643 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:48:52.643 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:48:52.643 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:48:52.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:52.643 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:48:52.653 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:48:52.653 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:48:52.653 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:48:52.653 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:52.653 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:48:52.653 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:52.653 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:48:52.653 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:52.653 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41946sec preferred_lft 41946sec 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:52.654 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:48:52.655 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:48:52.656 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:52.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:52.656 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:48:52.714 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:48:52.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:52.717 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:48:52.773 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:48:52.773 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:48:52.773 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:48:52.773 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:48:52.773 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:48:52.773 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:48:52.774 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:48:52.774 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:48:52.774 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:48:52.774 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:48:52.774 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:48:52.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:52.774 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:48:52.774 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:48:52.774 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:48:52.775 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:48:52.775 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:48:52.775 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:48:52.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:52.867 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:48:52.876 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:48:52.876 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:48:52.876 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:48:52.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:52.876 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:48:52.941 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:48:52.941 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:48:52.941 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:48:52.998 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:48:52.998 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:48:52.998 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:48:53.052 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:48:53.113 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:48:53.113 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:48:53.175 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:48:53.176 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:48:53.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:53.177 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:48:53.232 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:48:53.233 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:48:53.275 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:48:53.296 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:48:53.297 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:48:53.356 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:48:53.357 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:48:53.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:53.358 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:48:53.370 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:48:53.361+0000 7fb937203580 -1 init, newargv = 0x55eb122bafb0 newargc=15 2024-03-30T00:48:53.370 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[51003]: starting ceph client 2024-03-30T00:48:53.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[51003]: starting fuse 2024-03-30T00:48:53.379 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:48:53.379 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:48:53.379 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:48:53.379 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:48:53.380 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:48:53.381 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:48:53.381 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:48:53.381 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:48:53.381 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:48:53.381 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:48:53.381 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:48:53.381 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:48:53.381 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:48:53.381 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:48:53.381 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:48:53.501 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.51003.asok 2024-03-30T00:48:53.686 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51003.asok 2024-03-30T00:48:53.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:53.686 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51003.asok status 2024-03-30T00:48:53.866 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:48:53.866 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:48:53.866 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:48:53.866 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:48:53.866 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:48:53.866 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:48:53.866 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:48:53.866 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "51003", 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 8796, 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 8796 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2526263862 2024-03-30T00:48:53.867 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2526263862 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.8796 192.168.0.1:0/2526263862", 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/2526263862", 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 79, 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 110, 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:48:53.868 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:48:53.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:53.880 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:48:53.880 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:48:53.889 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:48:53.889 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:48:53.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:53.890 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:48:53.953 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:48:53.953 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:48:54.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:54.005 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-30T00:48:54.144 INFO:teuthology.orchestra.run.smithi066.stderr:6+0 records in 2024-03-30T00:48:54.144 INFO:teuthology.orchestra.run.smithi066.stderr:6+0 records out 2024-03-30T00:48:54.144 INFO:teuthology.orchestra.run.smithi066.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0806315 s, 78.0 MB/s 2024-03-30T00:48:54.145 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:48:54.145 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:48:54.154 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:48:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.325+0000 7f5cb5400640 1 Processor -- start 2024-03-30T00:48:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.325+0000 7f5cb5400640 1 -- start start 2024-03-30T00:48:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.325+0000 7f5cb5400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5cb00ce520 0x7f5cb00ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.325+0000 7f5cb5400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5cb00cabb0 0x7f5cb00cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.325+0000 7f5cb5400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb006d070 con 0x7f5cb015a640 2024-03-30T00:48:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.325+0000 7f5cb5400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb00cbba0 con 0x7f5cb00cabb0 2024-03-30T00:48:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.325+0000 7f5cb5400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb00cee70 con 0x7f5cb00ce520 2024-03-30T00:48:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5caf200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5cb00cabb0 0x7f5cb00cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5cae800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5cb015a640 0x7f5cb006dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53966/0 (socket says 172.21.15.66:53966) 2024-03-30T00:48:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5cae800640 1 -- 172.21.15.66:0/2214436723 learned_addr learned my addr 172.21.15.66:0/2214436723 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5ca5e00640 1 --2- 172.21.15.66:0/2214436723 >> v2:172.21.15.156:3300/0 conn(0x7f5cb00ce520 0x7f5cb00ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5caf200640 1 -- 172.21.15.66:0/2214436723 >> v2:172.21.15.156:3300/0 conn(0x7f5cb00ce520 msgr2=0x7f5cb00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5caf200640 1 --2- 172.21.15.66:0/2214436723 >> v2:172.21.15.156:3300/0 conn(0x7f5cb00ce520 0x7f5cb00ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5caf200640 1 -- 172.21.15.66:0/2214436723 >> v1:172.21.15.66:6789/0 conn(0x7f5cb015a640 legacy=0x7f5cb006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5caf200640 1 -- 172.21.15.66:0/2214436723 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb00cb8c0 con 0x7f5cb00cabb0 2024-03-30T00:48:54.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5caf200640 1 --2- 172.21.15.66:0/2214436723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb00cabb0 0x7f5cb00cafe0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00c88d0 tx=0x7f5ca801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458f1b12d63998cb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5cade00640 1 -- 172.21.15.66:0/2214436723 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80a59c0 con 0x7f5cb00cabb0 2024-03-30T00:48:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5cade00640 1 -- 172.21.15.66:0/2214436723 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca80a5b20 con 0x7f5cb00cabb0 2024-03-30T00:48:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.329+0000 7f5cade00640 1 -- 172.21.15.66:0/2214436723 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80ad440 con 0x7f5cb00cabb0 2024-03-30T00:48:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 -- 172.21.15.66:0/2214436723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb00cabb0 msgr2=0x7f5cb00cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 --2- 172.21.15.66:0/2214436723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb00cabb0 0x7f5cb00cafe0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00c88d0 tx=0x7f5ca801c900 comp rx=0 tx=0).stop 2024-03-30T00:48:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 -- 172.21.15.66:0/2214436723 shutdown_connections 2024-03-30T00:48:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 --2- 172.21.15.66:0/2214436723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb00cabb0 0x7f5cb00cafe0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 --2- 172.21.15.66:0/2214436723 >> v2:172.21.15.156:3300/0 conn(0x7f5cb00ce520 0x7f5cb00ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 -- 172.21.15.66:0/2214436723 >> 172.21.15.66:0/2214436723 conn(0x7f5cb0069f60 msgr2=0x7f5cb006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 -- 172.21.15.66:0/2214436723 shutdown_connections 2024-03-30T00:48:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 -- 172.21.15.66:0/2214436723 wait complete. 2024-03-30T00:48:54.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 Processor -- start 2024-03-30T00:48:54.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 -- start start 2024-03-30T00:48:54.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb00cabb0 0x7f5cb0152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb00ce520 0x7f5cb01524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb015a640 0x7f5cb00af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb01529e0 con 0x7f5cb00ce520 2024-03-30T00:48:54.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb0163820 con 0x7f5cb015a640 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cb5400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb0163b00 con 0x7f5cb00cabb0 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb00ce520 0x7f5cb01524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb00ce520 0x7f5cb01524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41796/0 (socket says 172.21.15.66:41796) 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5ca5e00640 1 -- 172.21.15.66:0/635896929 learned_addr learned my addr 172.21.15.66:0/635896929 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5cae800640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb00cabb0 0x7f5cb0152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5caf200640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb015a640 0x7f5cb00af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5ca5e00640 1 -- 172.21.15.66:0/635896929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb00cabb0 msgr2=0x7f5cb0152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5ca5e00640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb00cabb0 0x7f5cb0152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5ca5e00640 1 -- 172.21.15.66:0/635896929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb015a640 msgr2=0x7f5cb00af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5ca5e00640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb015a640 0x7f5cb00af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5ca5e00640 1 -- 172.21.15.66:0/635896929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca8009a70 con 0x7f5cb00ce520 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5caf200640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb015a640 0x7f5cb00af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.333+0000 7f5ca5e00640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb00ce520 0x7f5cb01524a0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800d800 tx=0x7f5c9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.337+0000 7f5ca7400640 1 -- 172.21.15.66:0/635896929 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800b6c0 con 0x7f5cb00ce520 2024-03-30T00:48:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.337+0000 7f5ca7400640 1 -- 172.21.15.66:0/635896929 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9800b820 con 0x7f5cb00ce520 2024-03-30T00:48:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.337+0000 7f5ca7400640 1 -- 172.21.15.66:0/635896929 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800bb90 con 0x7f5cb00ce520 2024-03-30T00:48:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.337+0000 7f5cb5400640 1 -- 172.21.15.66:0/635896929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb0163de0 con 0x7f5cb00ce520 2024-03-30T00:48:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.337+0000 7f5cb5400640 1 -- 172.21.15.66:0/635896929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb01643a0 con 0x7f5cb00ce520 2024-03-30T00:48:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.337+0000 7f5cb5400640 1 -- 172.21.15.66:0/635896929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb0164560 con 0x7f5cb00ce520 2024-03-30T00:48:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.337+0000 7f5ca7400640 1 -- 172.21.15.66:0/635896929 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5c9800bcf0 con 0x7f5cb00ce520 2024-03-30T00:48:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.337+0000 7f5ca7400640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c8c066690 0x7f5c8c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.337+0000 7f5ca7400640 1 -- 172.21.15.66:0/635896929 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5c98084910 con 0x7f5cb00ce520 2024-03-30T00:48:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.337+0000 7f5cae800640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c8c066690 0x7f5c8c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.337+0000 7f5cae800640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c8c066690 0x7f5c8c068b50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0005890 tx=0x7f5ca0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:54.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.341+0000 7f5ca7400640 1 -- 172.21.15.66:0/635896929 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5c98051aa0 con 0x7f5cb00ce520 2024-03-30T00:48:54.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.509+0000 7f5cb5400640 1 -- 172.21.15.66:0/635896929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cb00ad490 con 0x7f5cb00ce520 2024-03-30T00:48:54.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.509+0000 7f5ca7400640 1 -- 172.21.15.66:0/635896929 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16248 (secure 0 0 0) 0x7f5c980550c0 con 0x7f5cb00ce520 2024-03-30T00:48:54.520 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:54.520 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":110,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:48:47.334669+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:48:44.750350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-30T00:48:45.327928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:48:54.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 -- 172.21.15.66:0/635896929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c8c066690 msgr2=0x7f5c8c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c8c066690 0x7f5c8c068b50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0005890 tx=0x7f5ca0009040 comp rx=0 tx=0).stop 2024-03-30T00:48:54.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 -- 172.21.15.66:0/635896929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb00ce520 msgr2=0x7f5cb01524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb00ce520 0x7f5cb01524a0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800d800 tx=0x7f5c9800dcc0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 -- 172.21.15.66:0/635896929 shutdown_connections 2024-03-30T00:48:54.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb015a640 0x7f5cb00af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5c8c066690 0x7f5c8c068b50 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb00ce520 0x7f5cb01524a0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 --2- 172.21.15.66:0/635896929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb00cabb0 0x7f5cb0152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 -- 172.21.15.66:0/635896929 >> 172.21.15.66:0/635896929 conn(0x7f5cb0069f60 msgr2=0x7f5cb015be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:54.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 -- 172.21.15.66:0/635896929 shutdown_connections 2024-03-30T00:48:54.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.513+0000 7f5cb5400640 1 -- 172.21.15.66:0/635896929 wait complete. 2024-03-30T00:48:54.540 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:48:44.750350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:48:45.327928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:48:54.541 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:48:54.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.673+0000 7fe37c400640 1 Processor -- start 2024-03-30T00:48:54.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.673+0000 7fe37c400640 1 -- start start 2024-03-30T00:48:54.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.673+0000 7fe37c400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415a640 0x7fe37406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.673+0000 7fe37c400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe3740ce520 0x7fe3740ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.673+0000 7fe37c400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37406d070 con 0x7fe3740cabb0 2024-03-30T00:48:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.673+0000 7fe37c400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3740cbba0 con 0x7fe3740ce520 2024-03-30T00:48:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.673+0000 7fe37c400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37406e2c0 con 0x7fe37415a640 2024-03-30T00:48:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.673+0000 7fe37a400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe3740cabb0 0x7fe3740cb000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53976/0 (socket says 172.21.15.66:53976) 2024-03-30T00:48:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.673+0000 7fe37a400640 1 -- 172.21.15.66:0/169270499 learned_addr learned my addr 172.21.15.66:0/169270499 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe379a00640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415a640 0x7fe37406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe379000640 1 --2- 172.21.15.66:0/169270499 >> v2:172.21.15.156:3300/0 conn(0x7fe3740ce520 0x7fe3740ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103527895 0 0) 0x7fe37406d070 con 0x7fe3740cabb0 2024-03-30T00:48:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe35c003530 con 0x7fe3740cabb0 2024-03-30T00:48:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1676113246 0 0) 0x7fe35c003530 con 0x7fe3740cabb0 2024-03-30T00:48:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415a640 msgr2=0x7fe37406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415a640 0x7fe37406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 >> v2:172.21.15.156:3300/0 conn(0x7fe3740ce520 msgr2=0x7fe3740ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 --2- 172.21.15.66:0/169270499 >> v2:172.21.15.156:3300/0 conn(0x7fe3740ce520 0x7fe3740ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740cee60 con 0x7fe3740cabb0 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fe36c001a00 con 0x7fe3740cabb0 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 >> v1:172.21.15.66:6789/0 conn(0x7fe3740cabb0 legacy=0x7fe3740cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe35c003980 0x7fe35c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35c0047d0 0x7fe35c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415a640 0x7fe35c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740cbba0 con 0x7fe35c0047d0 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37406e2c0 con 0x7fe35c003980 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c00b990 con 0x7fe37415a640 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe379000640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35c0047d0 0x7fe35c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe379a00640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe35c003980 0x7fe35c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe37a400640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415a640 0x7fe35c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe379000640 1 -- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415a640 msgr2=0x7fe35c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:54.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe379000640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415a640 0x7fe35c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe379000640 1 -- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe35c003980 msgr2=0x7fe35c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:54.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe379000640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe35c003980 0x7fe35c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe379000640 1 -- 172.21.15.66:0/169270499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c003530 con 0x7fe35c0047d0 2024-03-30T00:48:54.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.677+0000 7fe37a400640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415a640 0x7fe35c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:54.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe379000640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35c0047d0 0x7fe35c004c50 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fe360009ae0 tx=0x7fe36001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e244cd4a9d15045 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:54.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600a55c0 con 0x7fe35c0047d0 2024-03-30T00:48:54.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3600a5720 con 0x7fe35c0047d0 2024-03-30T00:48:54.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe373e00640 1 -- 172.21.15.66:0/169270499 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600a5ad0 con 0x7fe35c0047d0 2024-03-30T00:48:54.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 -- 172.21.15.66:0/169270499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35c0047d0 msgr2=0x7fe35c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 --2- 172.21.15.66:0/169270499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe35c0047d0 0x7fe35c004c50 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fe360009ae0 tx=0x7fe36001cab0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37a400640 1 -- 172.21.15.66:0/169270499 reap_dead start 2024-03-30T00:48:54.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 -- 172.21.15.66:0/169270499 shutdown_connections 2024-03-30T00:48:54.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 -- 172.21.15.66:0/169270499 >> 172.21.15.66:0/169270499 conn(0x7fe374069f60 msgr2=0x7fe37406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:54.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 -- 172.21.15.66:0/169270499 shutdown_connections 2024-03-30T00:48:54.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 -- 172.21.15.66:0/169270499 wait complete. 2024-03-30T00:48:54.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 Processor -- start 2024-03-30T00:48:54.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 -- start start 2024-03-30T00:48:54.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe35c0047d0 0x7fe3740c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3740c7b70 0x7fe3741f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415d330 0x7fe3741f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740c5cb0 con 0x7fe3740c7b70 2024-03-30T00:48:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3741f51b0 con 0x7fe35c0047d0 2024-03-30T00:48:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.681+0000 7fe37c400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740708a0 con 0x7fe37415d330 2024-03-30T00:48:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe379000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3740c7b70 0x7fe3741f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe379000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3740c7b70 0x7fe3741f2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41814/0 (socket says 172.21.15.66:41814) 2024-03-30T00:48:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe379000640 1 -- 172.21.15.66:0/2484600882 learned_addr learned my addr 172.21.15.66:0/2484600882 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe379a00640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe35c0047d0 0x7fe3740c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe37a400640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415d330 0x7fe3741f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe379000640 1 -- 172.21.15.66:0/2484600882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415d330 msgr2=0x7fe3741f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe379000640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415d330 0x7fe3741f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe379000640 1 -- 172.21.15.66:0/2484600882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe35c0047d0 msgr2=0x7fe3740c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe379000640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe35c0047d0 0x7fe3740c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe379000640 1 -- 172.21.15.66:0/2484600882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740c8330 con 0x7fe3740c7b70 2024-03-30T00:48:54.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe379000640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3740c7b70 0x7fe3741f2fa0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fe36001c8e0 tx=0x7fe35c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:54.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe371600640 1 -- 172.21.15.66:0/2484600882 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600be070 con 0x7fe3740c7b70 2024-03-30T00:48:54.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe371600640 1 -- 172.21.15.66:0/2484600882 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe360004020 con 0x7fe3740c7b70 2024-03-30T00:48:54.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe37c400640 1 -- 172.21.15.66:0/2484600882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37406ff70 con 0x7fe3740c7b70 2024-03-30T00:48:54.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe371600640 1 -- 172.21.15.66:0/2484600882 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360004390 con 0x7fe3740c7b70 2024-03-30T00:48:54.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe37c400640 1 -- 172.21.15.66:0/2484600882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe374070530 con 0x7fe3740c7b70 2024-03-30T00:48:54.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe37c400640 1 -- 172.21.15.66:0/2484600882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37406d840 con 0x7fe3740c7b70 2024-03-30T00:48:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.685+0000 7fe371600640 1 -- 172.21.15.66:0/2484600882 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe3600b4040 con 0x7fe3740c7b70 2024-03-30T00:48:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.689+0000 7fe371600640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe354066690 0x7fe354068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.689+0000 7fe371600640 1 -- 172.21.15.66:0/2484600882 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe36011e060 con 0x7fe3740c7b70 2024-03-30T00:48:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.689+0000 7fe379a00640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe354066690 0x7fe354068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.689+0000 7fe379a00640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe354066690 0x7fe354068b50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe3740b12c0 tx=0x7fe368008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:54.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.693+0000 7fe371600640 1 -- 172.21.15.66:0/2484600882 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe37406d840 con 0x7fe3740c7b70 2024-03-30T00:48:54.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 -- 172.21.15.66:0/2484600882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe37406d840 con 0x7fe3740c7b70 2024-03-30T00:48:54.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe371600640 1 -- 172.21.15.66:0/2484600882 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe37406d840 con 0x7fe3740c7b70 2024-03-30T00:48:54.863 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:48:54.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 -- 172.21.15.66:0/2484600882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe354066690 msgr2=0x7fe354068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe354066690 0x7fe354068b50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe3740b12c0 tx=0x7fe368008040 comp rx=0 tx=0).stop 2024-03-30T00:48:54.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 -- 172.21.15.66:0/2484600882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3740c7b70 msgr2=0x7fe3741f2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:54.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3740c7b70 0x7fe3741f2fa0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fe36001c8e0 tx=0x7fe35c005800 comp rx=0 tx=0).stop 2024-03-30T00:48:54.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 -- 172.21.15.66:0/2484600882 shutdown_connections 2024-03-30T00:48:54.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe37415d330 0x7fe3741f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe354066690 0x7fe354068b50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3740c7b70 0x7fe3741f2fa0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 --2- 172.21.15.66:0/2484600882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe35c0047d0 0x7fe3740c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:54.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 -- 172.21.15.66:0/2484600882 >> 172.21.15.66:0/2484600882 conn(0x7fe374069f60 msgr2=0x7fe3740b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:54.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 -- 172.21.15.66:0/2484600882 shutdown_connections 2024-03-30T00:48:54.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:54.853+0000 7fe37c400640 1 -- 172.21.15.66:0/2484600882 wait complete. 2024-03-30T00:48:54.886 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:48:55.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1459400640 1 Processor -- start 2024-03-30T00:48:55.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1459400640 1 -- start start 2024-03-30T00:48:55.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1459400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f145415a640 0x7f145406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1459400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145406d070 con 0x7f14540cb2c0 2024-03-30T00:48:55.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1459400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14540cbba0 con 0x7f14540ce520 2024-03-30T00:48:55.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1459400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145406e2c0 con 0x7f145415a640 2024-03-30T00:48:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1453200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f14540cb2c0 0x7f14540ab010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53988/0 (socket says 172.21.15.66:53988) 2024-03-30T00:48:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1453200640 1 -- 172.21.15.66:0/1566068721 learned_addr learned my addr 172.21.15.66:0/1566068721 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1452800640 1 --2- 172.21.15.66:0/1566068721 >> v2:172.21.15.66:3300/0 conn(0x7f145415a640 0x7f145406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1452800640 1 -- 172.21.15.66:0/1566068721 >> v1:172.21.15.156:6789/0 conn(0x7f14540ce520 legacy=0x7f14540cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1451e00640 1 -- 172.21.15.66:0/1566068721 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920157202 0 0) 0x7f145406d070 con 0x7f14540cb2c0 2024-03-30T00:48:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1452800640 1 -- 172.21.15.66:0/1566068721 >> v1:172.21.15.66:6789/0 conn(0x7f14540cb2c0 legacy=0x7f14540ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.021+0000 7f1452800640 1 -- 172.21.15.66:0/1566068721 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14540cb8c0 con 0x7f145415a640 2024-03-30T00:48:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.025+0000 7f1452800640 1 --2- 172.21.15.66:0/1566068721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f145415a640 0x7f145406dd80 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f14440097c0 tx=0x7f144401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad467f6da9989d14 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.025+0000 7f1451e00640 1 -- 172.21.15.66:0/1566068721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444004230 con 0x7f145415a640 2024-03-30T00:48:55.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.025+0000 7f1451e00640 1 -- 172.21.15.66:0/1566068721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14440a7050 con 0x7f145415a640 2024-03-30T00:48:55.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.025+0000 7f1451e00640 1 -- 172.21.15.66:0/1566068721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444003620 con 0x7f145415a640 2024-03-30T00:48:55.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.025+0000 7f1459400640 1 -- 172.21.15.66:0/1566068721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f145415a640 msgr2=0x7f145406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.025+0000 7f1459400640 1 --2- 172.21.15.66:0/1566068721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f145415a640 0x7f145406dd80 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f14440097c0 tx=0x7f144401c460 comp rx=0 tx=0).stop 2024-03-30T00:48:55.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.025+0000 7f1459400640 1 -- 172.21.15.66:0/1566068721 shutdown_connections 2024-03-30T00:48:55.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.025+0000 7f1459400640 1 --2- 172.21.15.66:0/1566068721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f145415a640 0x7f145406dd80 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.025+0000 7f1459400640 1 -- 172.21.15.66:0/1566068721 >> 172.21.15.66:0/1566068721 conn(0x7f1454069f60 msgr2=0x7f145406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.025+0000 7f1459400640 1 -- 172.21.15.66:0/1566068721 shutdown_connections 2024-03-30T00:48:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.025+0000 7f1459400640 1 -- 172.21.15.66:0/1566068721 wait complete. 2024-03-30T00:48:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1459400640 1 Processor -- start 2024-03-30T00:48:55.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1459400640 1 -- start start 2024-03-30T00:48:55.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1459400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14540cb2c0 0x7f14541560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1459400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14540ce520 0x7f14540c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1459400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f145415a640 0x7f1454151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1459400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1454154df0 con 0x7f145415a640 2024-03-30T00:48:55.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1459400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1454155030 con 0x7f14540cb2c0 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1459400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1454155310 con 0x7f14540ce520 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1453200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f145415a640 0x7f1454151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1453200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f145415a640 0x7f1454151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41838/0 (socket says 172.21.15.66:41838) 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1453200640 1 -- 172.21.15.66:0/2526919885 learned_addr learned my addr 172.21.15.66:0/2526919885 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1452800640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14540cb2c0 0x7f14541560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1449e00640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14540ce520 0x7f14540c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1453200640 1 -- 172.21.15.66:0/2526919885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14540ce520 msgr2=0x7f14540c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1453200640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14540ce520 0x7f14540c3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1453200640 1 -- 172.21.15.66:0/2526919885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14540cb2c0 msgr2=0x7f14541560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1453200640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14540cb2c0 0x7f14541560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1453200640 1 -- 172.21.15.66:0/2526919885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444009420 con 0x7f145415a640 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1453200640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f145415a640 0x7f1454151710 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f144c00c7a0 tx=0x7f144c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f144b400640 1 -- 172.21.15.66:0/2526919885 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c00a640 con 0x7f145415a640 2024-03-30T00:48:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f144b400640 1 -- 172.21.15.66:0/2526919885 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144c00a7a0 con 0x7f145415a640 2024-03-30T00:48:55.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f144b400640 1 -- 172.21.15.66:0/2526919885 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c00abd0 con 0x7f145415a640 2024-03-30T00:48:55.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1459400640 1 -- 172.21.15.66:0/2526919885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14541555f0 con 0x7f145415a640 2024-03-30T00:48:55.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.029+0000 7f1459400640 1 -- 172.21.15.66:0/2526919885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1454155bb0 con 0x7f145415a640 2024-03-30T00:48:55.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.033+0000 7f1459400640 1 -- 172.21.15.66:0/2526919885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1454155dd0 con 0x7f145415a640 2024-03-30T00:48:55.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.033+0000 7f144b400640 1 -- 172.21.15.66:0/2526919885 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f144c00ad30 con 0x7f145415a640 2024-03-30T00:48:55.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.033+0000 7f144b400640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1430066690 0x7f1430068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.033+0000 7f144b400640 1 -- 172.21.15.66:0/2526919885 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f144c0838e0 con 0x7f145415a640 2024-03-30T00:48:55.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.033+0000 7f1452800640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1430066690 0x7f1430068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.033+0000 7f1452800640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1430066690 0x7f1430068b50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f14440ad000 tx=0x7f14440a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:55.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.037+0000 7f144b400640 1 -- 172.21.15.66:0/2526919885 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f144c050a90 con 0x7f145415a640 2024-03-30T00:48:55.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 -- 172.21.15.66:0/2526919885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1454066c30 con 0x7f145415a640 2024-03-30T00:48:55.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f144b400640 1 -- 172.21.15.66:0/2526919885 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f144c018070 con 0x7f145415a640 2024-03-30T00:48:55.210 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:48:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 -- 172.21.15.66:0/2526919885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1430066690 msgr2=0x7f1430068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1430066690 0x7f1430068b50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f14440ad000 tx=0x7f14440a6000 comp rx=0 tx=0).stop 2024-03-30T00:48:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 -- 172.21.15.66:0/2526919885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f145415a640 msgr2=0x7f1454151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f145415a640 0x7f1454151710 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f144c00c7a0 tx=0x7f144c00cc60 comp rx=0 tx=0).stop 2024-03-30T00:48:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 -- 172.21.15.66:0/2526919885 shutdown_connections 2024-03-30T00:48:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1430066690 0x7f1430068b50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f145415a640 0x7f1454151710 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14540ce520 0x7f14540c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 --2- 172.21.15.66:0/2526919885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14540cb2c0 0x7f14541560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 -- 172.21.15.66:0/2526919885 >> 172.21.15.66:0/2526919885 conn(0x7f1454069f60 msgr2=0x7f145415be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:55.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 -- 172.21.15.66:0/2526919885 shutdown_connections 2024-03-30T00:48:55.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.201+0000 7f1459400640 1 -- 172.21.15.66:0/2526919885 wait complete. 2024-03-30T00:48:55.232 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:48:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb91800640 1 Processor -- start 2024-03-30T00:48:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb91800640 1 -- start start 2024-03-30T00:48:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb91800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcb8c0cc560 0x7fcb8c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb91800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb8c06f8e0 0x7fcb8c06e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb91800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c06b0b0 con 0x7fcb8c06e630 2024-03-30T00:48:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb91800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8c06eac0 con 0x7fcb8c0cc560 2024-03-30T00:48:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb91800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c160640 con 0x7fcb8c06f8e0 2024-03-30T00:48:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb8b400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fcb8c06e630 0x7fcb8c0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:54004/0 (socket says 172.21.15.66:54004) 2024-03-30T00:48:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb8b400640 1 -- 172.21.15.66:0/480344472 learned_addr learned my addr 172.21.15.66:0/480344472 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb8aa00640 1 --2- 172.21.15.66:0/480344472 >> v2:172.21.15.156:3300/0 conn(0x7fcb8c0cc560 0x7fcb8c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb82000640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb8c06f8e0 0x7fcb8c06e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162402765 0 0) 0x7fcb8c06b0b0 con 0x7fcb8c06e630 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.365+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb70003530 con 0x7fcb8c06e630 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2755396294 0 0) 0x7fcb70003530 con 0x7fcb8c06e630 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb8c06f8e0 msgr2=0x7fcb8c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb8c06f8e0 0x7fcb8c06e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 >> v2:172.21.15.156:3300/0 conn(0x7fcb8c0cc560 msgr2=0x7fcb8c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 --2- 172.21.15.66:0/480344472 >> v2:172.21.15.156:3300/0 conn(0x7fcb8c0cc560 0x7fcb8c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c06b390 con 0x7fcb8c06e630 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fcb84001680 con 0x7fcb8c06e630 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8aa00640 1 --2- 172.21.15.66:0/480344472 >> v2:172.21.15.156:3300/0 conn(0x7fcb8c0cc560 0x7fcb8c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:55.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 >> v1:172.21.15.66:6789/0 conn(0x7fcb8c06e630 legacy=0x7fcb8c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb70003980 0x7fcb70003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb8c06f8e0 0x7fcb700047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb70004d10 0x7fcb70005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c06eac0 con 0x7fcb70003980 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c160640 con 0x7fcb70004d10 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7000b990 con 0x7fcb8c06f8e0 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8aa00640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb70003980 0x7fcb70003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb82000640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb8c06f8e0 0x7fcb700047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8b400640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb70004d10 0x7fcb70005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8aa00640 1 -- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb8c06f8e0 msgr2=0x7fcb700047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8aa00640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb8c06f8e0 0x7fcb700047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8aa00640 1 -- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb70004d10 msgr2=0x7fcb70005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8aa00640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb70004d10 0x7fcb70005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8aa00640 1 -- 172.21.15.66:0/480344472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb70003530 con 0x7fcb70003980 2024-03-30T00:48:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8b400640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb70004d10 0x7fcb70005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8aa00640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb70003980 0x7fcb70003da0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00e380 tx=0x7fcb7c01ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b8fc393a8dc1dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.369+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0a5d00 con 0x7fcb70003980 2024-03-30T00:48:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c00f040 con 0x7fcb70003980 2024-03-30T00:48:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb8a000640 1 -- 172.21.15.66:0/480344472 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0ad640 con 0x7fcb70003980 2024-03-30T00:48:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 -- 172.21.15.66:0/480344472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb70003980 msgr2=0x7fcb70003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 --2- 172.21.15.66:0/480344472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb70003980 0x7fcb70003da0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00e380 tx=0x7fcb7c01ce30 comp rx=0 tx=0).stop 2024-03-30T00:48:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb8b400640 1 -- 172.21.15.66:0/480344472 reap_dead start 2024-03-30T00:48:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 -- 172.21.15.66:0/480344472 shutdown_connections 2024-03-30T00:48:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 -- 172.21.15.66:0/480344472 >> 172.21.15.66:0/480344472 conn(0x7fcb8c0d0540 msgr2=0x7fcb8c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 -- 172.21.15.66:0/480344472 shutdown_connections 2024-03-30T00:48:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 -- 172.21.15.66:0/480344472 wait complete. 2024-03-30T00:48:55.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 Processor -- start 2024-03-30T00:48:55.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 -- start start 2024-03-30T00:48:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb70003980 0x7fcb8c158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb8c0c4900 0x7fcb8c0c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c1f5360 0x7fcb8c0c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1f6420 con 0x7fcb8c1f5360 2024-03-30T00:48:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c159120 con 0x7fcb8c0c4900 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb91800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1f58a0 con 0x7fcb70003980 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb8b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c1f5360 0x7fcb8c0c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb8aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb70003980 0x7fcb8c158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb8b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c1f5360 0x7fcb8c0c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41858/0 (socket says 172.21.15.66:41858) 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb8b400640 1 -- 172.21.15.66:0/2664868000 learned_addr learned my addr 172.21.15.66:0/2664868000 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.373+0000 7fcb82000640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb8c0c4900 0x7fcb8c0c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb8b400640 1 -- 172.21.15.66:0/2664868000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb70003980 msgr2=0x7fcb8c158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb8b400640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb70003980 0x7fcb8c158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb8b400640 1 -- 172.21.15.66:0/2664868000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb8c0c4900 msgr2=0x7fcb8c0c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb8b400640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb8c0c4900 0x7fcb8c0c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb8b400640 1 -- 172.21.15.66:0/2664868000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c1f5b80 con 0x7fcb8c1f5360 2024-03-30T00:48:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb8b400640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c1f5360 0x7fcb8c0c5290 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c072bd0 tx=0x7fcb8400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:55.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb83400640 1 -- 172.21.15.66:0/2664868000 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84012590 con 0x7fcb8c1f5360 2024-03-30T00:48:55.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb83400640 1 -- 172.21.15.66:0/2664868000 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb840126f0 con 0x7fcb8c1f5360 2024-03-30T00:48:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb83400640 1 -- 172.21.15.66:0/2664868000 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84012ae0 con 0x7fcb8c1f5360 2024-03-30T00:48:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb91800640 1 -- 172.21.15.66:0/2664868000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c1f5e60 con 0x7fcb8c1f5360 2024-03-30T00:48:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb91800640 1 -- 172.21.15.66:0/2664868000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c0ae4a0 con 0x7fcb8c1f5360 2024-03-30T00:48:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb91800640 1 -- 172.21.15.66:0/2664868000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c066c30 con 0x7fcb8c1f5360 2024-03-30T00:48:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb83400640 1 -- 172.21.15.66:0/2664868000 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcb84012c40 con 0x7fcb8c1f5360 2024-03-30T00:48:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb83400640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcb68066690 0x7fcb68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb83400640 1 -- 172.21.15.66:0/2664868000 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcb84083f10 con 0x7fcb8c1f5360 2024-03-30T00:48:55.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.377+0000 7fcb8aa00640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcb68066690 0x7fcb68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.381+0000 7fcb8aa00640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcb68066690 0x7fcb68068b50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0af000 tx=0x7fcb7c004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:55.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.381+0000 7fcb83400640 1 -- 172.21.15.66:0/2664868000 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcb840510a0 con 0x7fcb8c1f5360 2024-03-30T00:48:55.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.541+0000 7fcb91800640 1 -- 172.21.15.66:0/2664868000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb8c066c30 con 0x7fcb8c1f5360 2024-03-30T00:48:55.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.541+0000 7fcb83400640 1 -- 172.21.15.66:0/2664868000 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb840546c0 con 0x7fcb8c1f5360 2024-03-30T00:48:55.552 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:48:55.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 -- 172.21.15.66:0/2664868000 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcb68066690 msgr2=0x7fcb68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcb68066690 0x7fcb68068b50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0af000 tx=0x7fcb7c004ac0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 -- 172.21.15.66:0/2664868000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c1f5360 msgr2=0x7fcb8c0c5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c1f5360 0x7fcb8c0c5290 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c072bd0 tx=0x7fcb8400ead0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 -- 172.21.15.66:0/2664868000 shutdown_connections 2024-03-30T00:48:55.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcb68066690 0x7fcb68068b50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c1f5360 0x7fcb8c0c5290 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb8c0c4900 0x7fcb8c0c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 --2- 172.21.15.66:0/2664868000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb70003980 0x7fcb8c158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 -- 172.21.15.66:0/2664868000 >> 172.21.15.66:0/2664868000 conn(0x7fcb8c0d0540 msgr2=0x7fcb8c15e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:55.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 -- 172.21.15.66:0/2664868000 shutdown_connections 2024-03-30T00:48:55.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.545+0000 7fcb91800640 1 -- 172.21.15.66:0/2664868000 wait complete. 2024-03-30T00:48:55.573 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:48:55.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.709+0000 7f7793a00640 1 Processor -- start 2024-03-30T00:48:55.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.709+0000 7f7793a00640 1 -- start start 2024-03-30T00:48:55.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.709+0000 7f7793a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f778c0ae4e0 0x7f778c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.709+0000 7f7793a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f778c0ce820 0x7f778c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.709+0000 7f7793a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778c06e7d0 con 0x7f778c15ca10 2024-03-30T00:48:55.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.709+0000 7f7793a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778c0b1f70 con 0x7f778c0ae4e0 2024-03-30T00:48:55.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.709+0000 7f7793a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778c0cf1f0 con 0x7f778c0ce820 2024-03-30T00:48:55.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.709+0000 7f7791000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f778c15ca10 0x7f778c06ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:54016/0 (socket says 172.21.15.66:54016) 2024-03-30T00:48:55.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.709+0000 7f7791000640 1 -- 172.21.15.66:0/3846664332 learned_addr learned my addr 172.21.15.66:0/3846664332 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:55.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.709+0000 7f7783e00640 1 --2- 172.21.15.66:0/3846664332 >> v2:172.21.15.66:3300/0 conn(0x7f778c0ae4e0 0x7f778c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.709+0000 7f7791a00640 1 --2- 172.21.15.66:0/3846664332 >> v2:172.21.15.156:3300/0 conn(0x7f778c0ce820 0x7f778c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f778be00640 1 -- 172.21.15.66:0/3846664332 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997713540 0 0) 0x7f778c06e7d0 con 0x7f778c15ca10 2024-03-30T00:48:55.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7783e00640 1 -- 172.21.15.66:0/3846664332 >> v2:172.21.15.156:3300/0 conn(0x7f778c0ce820 msgr2=0x7f778c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:55.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7783e00640 1 --2- 172.21.15.66:0/3846664332 >> v2:172.21.15.156:3300/0 conn(0x7f778c0ce820 0x7f778c0cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7783e00640 1 -- 172.21.15.66:0/3846664332 >> v1:172.21.15.66:6789/0 conn(0x7f778c15ca10 legacy=0x7f778c06ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7783e00640 1 -- 172.21.15.66:0/3846664332 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c15cdf0 con 0x7f778c0ae4e0 2024-03-30T00:48:55.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7783e00640 1 --2- 172.21.15.66:0/3846664332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778c0ae4e0 0x7f778c0cb620 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f778c06ab10 tx=0x7f777401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567242bb752e9bf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:55.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f778be00640 1 -- 172.21.15.66:0/3846664332 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740ab650 con 0x7f778c0ae4e0 2024-03-30T00:48:55.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f778be00640 1 -- 172.21.15.66:0/3846664332 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77740ab7b0 con 0x7f778c0ae4e0 2024-03-30T00:48:55.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f778be00640 1 -- 172.21.15.66:0/3846664332 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740abb40 con 0x7f778c0ae4e0 2024-03-30T00:48:55.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7793a00640 1 -- 172.21.15.66:0/3846664332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778c0ae4e0 msgr2=0x7f778c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7793a00640 1 --2- 172.21.15.66:0/3846664332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778c0ae4e0 0x7f778c0cb620 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f778c06ab10 tx=0x7f777401c450 comp rx=0 tx=0).stop 2024-03-30T00:48:55.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7793a00640 1 -- 172.21.15.66:0/3846664332 shutdown_connections 2024-03-30T00:48:55.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7793a00640 1 --2- 172.21.15.66:0/3846664332 >> v2:172.21.15.156:3300/0 conn(0x7f778c0ce820 0x7f778c0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7793a00640 1 --2- 172.21.15.66:0/3846664332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778c0ae4e0 0x7f778c0cb620 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7793a00640 1 -- 172.21.15.66:0/3846664332 >> 172.21.15.66:0/3846664332 conn(0x7f778c098a20 msgr2=0x7f778c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:55.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7793a00640 1 -- 172.21.15.66:0/3846664332 shutdown_connections 2024-03-30T00:48:55.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.713+0000 7f7793a00640 1 -- 172.21.15.66:0/3846664332 wait complete. 2024-03-30T00:48:55.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7793a00640 1 Processor -- start 2024-03-30T00:48:55.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7793a00640 1 -- start start 2024-03-30T00:48:55.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7793a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f778c0ae4e0 0x7f778c15b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7793a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f778c0ce820 0x7f778c0c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7793a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778c15ca10 0x7f778c0b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7793a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c141ec0 con 0x7f778c15ca10 2024-03-30T00:48:55.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7793a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0b1470 con 0x7f778c0ce820 2024-03-30T00:48:55.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7793a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778c15a540 con 0x7f778c0ae4e0 2024-03-30T00:48:55.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7791a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778c15ca10 0x7f778c0b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7791a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778c15ca10 0x7f778c0b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41876/0 (socket says 172.21.15.66:41876) 2024-03-30T00:48:55.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7791a00640 1 -- 172.21.15.66:0/1337715738 learned_addr learned my addr 172.21.15.66:0/1337715738 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:55.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7783e00640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f778c0ce820 0x7f778c0c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7791000640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f778c0ae4e0 0x7f778c15b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7791a00640 1 -- 172.21.15.66:0/1337715738 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f778c0ae4e0 msgr2=0x7f778c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7791a00640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f778c0ae4e0 0x7f778c15b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7791a00640 1 -- 172.21.15.66:0/1337715738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f778c0ce820 msgr2=0x7f778c0c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7791a00640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f778c0ce820 0x7f778c0c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7791a00640 1 -- 172.21.15.66:0/1337715738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7774009420 con 0x7f778c15ca10 2024-03-30T00:48:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7791000640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f778c0ae4e0 0x7f778c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7791a00640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778c15ca10 0x7f778c0b0f30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7784007e40 tx=0x7f778400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7789600640 1 -- 172.21.15.66:0/1337715738 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784014620 con 0x7f778c15ca10 2024-03-30T00:48:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7789600640 1 -- 172.21.15.66:0/1337715738 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778400ee90 con 0x7f778c15ca10 2024-03-30T00:48:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7793a00640 1 -- 172.21.15.66:0/1337715738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778c15a820 con 0x7f778c15ca10 2024-03-30T00:48:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7789600640 1 -- 172.21.15.66:0/1337715738 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77840149e0 con 0x7f778c15ca10 2024-03-30T00:48:55.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.717+0000 7f7793a00640 1 -- 172.21.15.66:0/1337715738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c15ade0 con 0x7f778c15ca10 2024-03-30T00:48:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.721+0000 7f7793a00640 1 -- 172.21.15.66:0/1337715738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778c066c30 con 0x7f778c15ca10 2024-03-30T00:48:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.721+0000 7f7789600640 1 -- 172.21.15.66:0/1337715738 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7784014b40 con 0x7f778c15ca10 2024-03-30T00:48:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.721+0000 7f7789600640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7768066690 0x7f7768068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:55.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.721+0000 7f7789600640 1 -- 172.21.15.66:0/1337715738 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7784084cd0 con 0x7f778c15ca10 2024-03-30T00:48:55.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.721+0000 7f7791000640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7768066690 0x7f7768068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:55.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.721+0000 7f7791000640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7768066690 0x7f7768068b50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f777c007fd0 tx=0x7f777c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:55.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.725+0000 7f7789600640 1 -- 172.21.15.66:0/1337715738 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7784051ef0 con 0x7f778c15ca10 2024-03-30T00:48:55.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.901+0000 7f7793a00640 1 -- 172.21.15.66:0/1337715738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f778c0c6340 con 0x7f778c15ca10 2024-03-30T00:48:55.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7789600640 1 -- 172.21.15.66:0/1337715738 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7784055510 con 0x7f778c15ca10 2024-03-30T00:48:55.913 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:48:55.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 -- 172.21.15.66:0/1337715738 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7768066690 msgr2=0x7f7768068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7768066690 0x7f7768068b50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f777c007fd0 tx=0x7f777c009040 comp rx=0 tx=0).stop 2024-03-30T00:48:55.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 -- 172.21.15.66:0/1337715738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778c15ca10 msgr2=0x7f778c0b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:55.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778c15ca10 0x7f778c0b0f30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7784007e40 tx=0x7f778400e4b0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 -- 172.21.15.66:0/1337715738 shutdown_connections 2024-03-30T00:48:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7768066690 0x7f7768068b50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f778c15ca10 0x7f778c0b0f30 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f778c0ce820 0x7f778c0c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 --2- 172.21.15.66:0/1337715738 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f778c0ae4e0 0x7f778c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 -- 172.21.15.66:0/1337715738 >> 172.21.15.66:0/1337715738 conn(0x7f778c098a20 msgr2=0x7f778c158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 -- 172.21.15.66:0/1337715738 shutdown_connections 2024-03-30T00:48:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:55.905+0000 7f7793a00640 1 -- 172.21.15.66:0/1337715738 wait complete. 2024-03-30T00:48:55.935 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:48:55.935 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:48:55.946 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:48:56.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.121+0000 7f8ad1e00640 1 Processor -- start 2024-03-30T00:48:56.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.121+0000 7f8ad1e00640 1 -- start start 2024-03-30T00:48:56.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad1e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8acc0abf40 0x7f8acc0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad1e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8acc161040 0x7f8acc163430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad1e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8acc06be90 con 0x7f8acc06fee0 2024-03-30T00:48:56.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad1e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8acc06c170 con 0x7f8acc161040 2024-03-30T00:48:56.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad1e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8acc0b1bf0 con 0x7f8acc0abf40 2024-03-30T00:48:56.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad1400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8acc161040 0x7f8acc163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ac3e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f8acc06fee0 0x7f8acc070340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:54032/0 (socket says 172.21.15.66:54032) 2024-03-30T00:48:56.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ac3e00640 1 -- 172.21.15.66:0/1917169668 learned_addr learned my addr 172.21.15.66:0/1917169668 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:56.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad0a00640 1 --2- 172.21.15.66:0/1917169668 >> v2:172.21.15.156:3300/0 conn(0x7f8acc0abf40 0x7f8acc0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad1400640 1 -- 172.21.15.66:0/1917169668 >> v2:172.21.15.156:3300/0 conn(0x7f8acc0abf40 msgr2=0x7f8acc0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:56.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad1400640 1 --2- 172.21.15.66:0/1917169668 >> v2:172.21.15.156:3300/0 conn(0x7f8acc0abf40 0x7f8acc0b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad1400640 1 -- 172.21.15.66:0/1917169668 >> v1:172.21.15.66:6789/0 conn(0x7f8acc06fee0 legacy=0x7f8acc070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8acbe00640 1 -- 172.21.15.66:0/1917169668 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523879658 0 0) 0x7f8acc06be90 con 0x7f8acc06fee0 2024-03-30T00:48:56.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad1400640 1 -- 172.21.15.66:0/1917169668 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc0b1ed0 con 0x7f8acc161040 2024-03-30T00:48:56.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad0a00640 1 --2- 172.21.15.66:0/1917169668 >> v2:172.21.15.156:3300/0 conn(0x7f8acc0abf40 0x7f8acc0b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:56.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8ad1400640 1 --2- 172.21.15.66:0/1917169668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8acc161040 0x7f8acc163430 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0cabc0 tx=0x7f8ac401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb48cddba4921f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:56.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.125+0000 7f8acbe00640 1 -- 172.21.15.66:0/1917169668 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40a5920 con 0x7f8acc161040 2024-03-30T00:48:56.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8acbe00640 1 -- 172.21.15.66:0/1917169668 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac40a5a80 con 0x7f8acc161040 2024-03-30T00:48:56.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8acbe00640 1 -- 172.21.15.66:0/1917169668 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40a5e10 con 0x7f8acc161040 2024-03-30T00:48:56.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 -- 172.21.15.66:0/1917169668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8acc161040 msgr2=0x7f8acc163430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 --2- 172.21.15.66:0/1917169668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8acc161040 0x7f8acc163430 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0cabc0 tx=0x7f8ac401c900 comp rx=0 tx=0).stop 2024-03-30T00:48:56.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 -- 172.21.15.66:0/1917169668 shutdown_connections 2024-03-30T00:48:56.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 --2- 172.21.15.66:0/1917169668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8acc161040 0x7f8acc163430 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 --2- 172.21.15.66:0/1917169668 >> v2:172.21.15.156:3300/0 conn(0x7f8acc0abf40 0x7f8acc0b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 -- 172.21.15.66:0/1917169668 >> 172.21.15.66:0/1917169668 conn(0x7f8acc0afe40 msgr2=0x7f8acc0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:56.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 -- 172.21.15.66:0/1917169668 shutdown_connections 2024-03-30T00:48:56.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 -- 172.21.15.66:0/1917169668 wait complete. 2024-03-30T00:48:56.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 Processor -- start 2024-03-30T00:48:56.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 -- start start 2024-03-30T00:48:56.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8acc06fee0 0x7f8acc162120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8acc0abf40 0x7f8acc161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8acc161040 0x7f8acc1576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc157290 con 0x7f8acc161040 2024-03-30T00:48:56.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc0c8470 con 0x7f8acc06fee0 2024-03-30T00:48:56.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc0c8750 con 0x7f8acc0abf40 2024-03-30T00:48:56.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8acc161040 0x7f8acc1576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8acc161040 0x7f8acc1576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41896/0 (socket says 172.21.15.66:41896) 2024-03-30T00:48:56.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1400640 1 -- 172.21.15.66:0/2535049131 learned_addr learned my addr 172.21.15.66:0/2535049131 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:56.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ac3e00640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8acc0abf40 0x7f8acc161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad0a00640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8acc06fee0 0x7f8acc162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1400640 1 -- 172.21.15.66:0/2535049131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8acc0abf40 msgr2=0x7f8acc161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1400640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8acc0abf40 0x7f8acc161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1400640 1 -- 172.21.15.66:0/2535049131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8acc06fee0 msgr2=0x7f8acc162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1400640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8acc06fee0 0x7f8acc162120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.129+0000 7f8ad1400640 1 -- 172.21.15.66:0/2535049131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4009a70 con 0x7f8acc161040 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ad0a00640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8acc06fee0 0x7f8acc162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ad1400640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8acc161040 0x7f8acc1576b0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4019510 tx=0x7f8ac40196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ac9600640 1 -- 172.21.15.66:0/2535049131 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40ad850 con 0x7f8acc161040 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ac9600640 1 -- 172.21.15.66:0/2535049131 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac40ad9b0 con 0x7f8acc161040 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ac9600640 1 -- 172.21.15.66:0/2535049131 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40add60 con 0x7f8acc161040 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ad1e00640 1 -- 172.21.15.66:0/2535049131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8acc0c8a30 con 0x7f8acc161040 2024-03-30T00:48:56.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ad1e00640 1 -- 172.21.15.66:0/2535049131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8acc162660 con 0x7f8acc161040 2024-03-30T00:48:56.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ad1e00640 1 -- 172.21.15.66:0/2535049131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8acc1573d0 con 0x7f8acc161040 2024-03-30T00:48:56.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ac9600640 1 -- 172.21.15.66:0/2535049131 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8ac40b4050 con 0x7f8acc161040 2024-03-30T00:48:56.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ac9600640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8aa8066690 0x7f8aa8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ac9600640 1 -- 172.21.15.66:0/2535049131 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8ac411f260 con 0x7f8acc161040 2024-03-30T00:48:56.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.133+0000 7f8ad0a00640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8aa8066690 0x7f8aa8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.137+0000 7f8ad0a00640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8aa8066690 0x7f8aa8068b50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009980 tx=0x7f8abc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:56.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.137+0000 7f8ac9600640 1 -- 172.21.15.66:0/2535049131 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8acc1573d0 con 0x7f8acc161040 2024-03-30T00:48:56.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 -- 172.21.15.66:0/2535049131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8acc0c8890 con 0x7f8acc161040 2024-03-30T00:48:56.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ac9600640 1 -- 172.21.15.66:0/2535049131 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16248 (secure 0 0 0) 0x7f8acc0c8890 con 0x7f8acc161040 2024-03-30T00:48:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:56.311 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":110,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:48:47.334669+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:48:44.750350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-30T00:48:45.327928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:48:56.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 -- 172.21.15.66:0/2535049131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8aa8066690 msgr2=0x7f8aa8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8aa8066690 0x7f8aa8068b50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009980 tx=0x7f8abc008040 comp rx=0 tx=0).stop 2024-03-30T00:48:56.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 -- 172.21.15.66:0/2535049131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8acc161040 msgr2=0x7f8acc1576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8acc161040 0x7f8acc1576b0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4019510 tx=0x7f8ac40196a0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 -- 172.21.15.66:0/2535049131 shutdown_connections 2024-03-30T00:48:56.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8aa8066690 0x7f8aa8068b50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8acc161040 0x7f8acc1576b0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8acc0abf40 0x7f8acc161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 --2- 172.21.15.66:0/2535049131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8acc06fee0 0x7f8acc162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 -- 172.21.15.66:0/2535049131 >> 172.21.15.66:0/2535049131 conn(0x7f8acc0afe40 msgr2=0x7f8acc14f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:56.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 -- 172.21.15.66:0/2535049131 shutdown_connections 2024-03-30T00:48:56.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.301+0000 7f8ad1e00640 1 -- 172.21.15.66:0/2535049131 wait complete. 2024-03-30T00:48:56.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:48:44.750350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:48:45.327928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:48:56.333 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:48:56.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.464+0000 7f8c8d200640 1 Processor -- start 2024-03-30T00:48:56.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.464+0000 7f8c8d200640 1 -- start start 2024-03-30T00:48:56.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.464+0000 7f8c8d200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c880ce520 0x7f8c880ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.464+0000 7f8c8d200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8806d070 con 0x7f8c880cabb0 2024-03-30T00:48:56.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.464+0000 7f8c8d200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c880cbba0 con 0x7f8c8815a640 2024-03-30T00:48:56.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.464+0000 7f8c8d200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880cee40 con 0x7f8c880ce520 2024-03-30T00:48:56.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c86800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f8c880cabb0 0x7f8c880cb000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:54040/0 (socket says 172.21.15.66:54040) 2024-03-30T00:48:56.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c86800640 1 -- 172.21.15.66:0/1724797663 learned_addr learned my addr 172.21.15.66:0/1724797663 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:56.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c7d400640 1 --2- 172.21.15.66:0/1724797663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c880ce520 0x7f8c880ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c85400640 1 -- 172.21.15.66:0/1724797663 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547500680 0 0) 0x7f8c8806d070 con 0x7f8c880cabb0 2024-03-30T00:48:56.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c85400640 1 -- 172.21.15.66:0/1724797663 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c6c003530 con 0x7f8c880cabb0 2024-03-30T00:48:56.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c85400640 1 -- 172.21.15.66:0/1724797663 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094964321 0 0) 0x7f8c880cbba0 con 0x7f8c8815a640 2024-03-30T00:48:56.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c85400640 1 -- 172.21.15.66:0/1724797663 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c8806d070 con 0x7f8c8815a640 2024-03-30T00:48:56.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c7d400640 1 -- 172.21.15.66:0/1724797663 >> v1:172.21.15.156:6789/0 conn(0x7f8c8815a640 legacy=0x7f8c8806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c7d400640 1 -- 172.21.15.66:0/1724797663 >> v1:172.21.15.66:6789/0 conn(0x7f8c880cabb0 legacy=0x7f8c880cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c7d400640 1 -- 172.21.15.66:0/1724797663 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c880cb8c0 con 0x7f8c880ce520 2024-03-30T00:48:56.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c7d400640 1 --2- 172.21.15.66:0/1724797663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c880ce520 0x7f8c880ce900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c700097f0 tx=0x7f8c7001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9cfc591e82b8d94 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:56.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.468+0000 7f8c85400640 1 -- 172.21.15.66:0/1724797663 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c700ac630 con 0x7f8c880ce520 2024-03-30T00:48:56.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c85400640 1 -- 172.21.15.66:0/1724797663 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c700ac790 con 0x7f8c880ce520 2024-03-30T00:48:56.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c85400640 1 -- 172.21.15.66:0/1724797663 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c700acb20 con 0x7f8c880ce520 2024-03-30T00:48:56.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 -- 172.21.15.66:0/1724797663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c880ce520 msgr2=0x7f8c880ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 --2- 172.21.15.66:0/1724797663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c880ce520 0x7f8c880ce900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c700097f0 tx=0x7f8c7001c4e0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 -- 172.21.15.66:0/1724797663 shutdown_connections 2024-03-30T00:48:56.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 --2- 172.21.15.66:0/1724797663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c880ce520 0x7f8c880ce900 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 -- 172.21.15.66:0/1724797663 >> 172.21.15.66:0/1724797663 conn(0x7f8c88069f60 msgr2=0x7f8c8806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:56.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 -- 172.21.15.66:0/1724797663 shutdown_connections 2024-03-30T00:48:56.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 -- 172.21.15.66:0/1724797663 wait complete. 2024-03-30T00:48:56.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 Processor -- start 2024-03-30T00:48:56.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 -- start start 2024-03-30T00:48:56.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c880cabb0 0x7f8c880c0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c880ce520 0x7f8c880aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8815a640 0x7f8c8815cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880cc570 con 0x7f8c8815a640 2024-03-30T00:48:56.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880ca630 con 0x7f8c880ce520 2024-03-30T00:48:56.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c8d200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880cbf20 con 0x7f8c880cabb0 2024-03-30T00:48:56.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c86800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8815a640 0x7f8c8815cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c86800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8815a640 0x7f8c8815cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41904/0 (socket says 172.21.15.66:41904) 2024-03-30T00:48:56.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c86800640 1 -- 172.21.15.66:0/679253641 learned_addr learned my addr 172.21.15.66:0/679253641 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:56.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c7d400640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c880ce520 0x7f8c880aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.472+0000 7f8c85e00640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c880cabb0 0x7f8c880c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c86800640 1 -- 172.21.15.66:0/679253641 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c880cabb0 msgr2=0x7f8c880c0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c86800640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c880cabb0 0x7f8c880c0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c86800640 1 -- 172.21.15.66:0/679253641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c880ce520 msgr2=0x7f8c880aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c86800640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c880ce520 0x7f8c880aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c86800640 1 -- 172.21.15.66:0/679253641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c70009420 con 0x7f8c8815a640 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c86800640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8815a640 0x7f8c8815cc00 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c450 tx=0x7f8c8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c7ea00640 1 -- 172.21.15.66:0/679253641 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80005db0 con 0x7f8c8815a640 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c7ea00640 1 -- 172.21.15.66:0/679253641 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80007070 con 0x7f8c8815a640 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c7ea00640 1 -- 172.21.15.66:0/679253641 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80007400 con 0x7f8c8815a640 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c8d200640 1 -- 172.21.15.66:0/679253641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c880cc200 con 0x7f8c8815a640 2024-03-30T00:48:56.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c8d200640 1 -- 172.21.15.66:0/679253641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c880af500 con 0x7f8c8815a640 2024-03-30T00:48:56.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c8d200640 1 -- 172.21.15.66:0/679253641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c880cc720 con 0x7f8c8815a640 2024-03-30T00:48:56.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c7ea00640 1 -- 172.21.15.66:0/679253641 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8c80007560 con 0x7f8c8815a640 2024-03-30T00:48:56.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c7ea00640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c64066690 0x7f8c64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c7ea00640 1 -- 172.21.15.66:0/679253641 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8c800837e0 con 0x7f8c8815a640 2024-03-30T00:48:56.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.476+0000 7f8c85e00640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c64066690 0x7f8c64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.480+0000 7f8c85e00640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c64066690 0x7f8c64068b50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8c78001570 tx=0x7f8c78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:56.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.480+0000 7f8c7ea00640 1 -- 172.21.15.66:0/679253641 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8c880cc720 con 0x7f8c8815a640 2024-03-30T00:48:56.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.640+0000 7f8c8d200640 1 -- 172.21.15.66:0/679253641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c880cc720 con 0x7f8c8815a640 2024-03-30T00:48:56.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.640+0000 7f8c7ea00640 1 -- 172.21.15.66:0/679253641 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c880cc720 con 0x7f8c8815a640 2024-03-30T00:48:56.653 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:48:56.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 -- 172.21.15.66:0/679253641 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c64066690 msgr2=0x7f8c64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c64066690 0x7f8c64068b50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8c78001570 tx=0x7f8c78009040 comp rx=0 tx=0).stop 2024-03-30T00:48:56.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 -- 172.21.15.66:0/679253641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8815a640 msgr2=0x7f8c8815cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8815a640 0x7f8c8815cc00 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c450 tx=0x7f8c8000c910 comp rx=0 tx=0).stop 2024-03-30T00:48:56.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 -- 172.21.15.66:0/679253641 shutdown_connections 2024-03-30T00:48:56.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8c64066690 0x7f8c64068b50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c8815a640 0x7f8c8815cc00 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c880ce520 0x7f8c880aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 --2- 172.21.15.66:0/679253641 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c880cabb0 0x7f8c880c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 -- 172.21.15.66:0/679253641 >> 172.21.15.66:0/679253641 conn(0x7f8c88069f60 msgr2=0x7f8c880afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:56.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 -- 172.21.15.66:0/679253641 shutdown_connections 2024-03-30T00:48:56.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.644+0000 7f8c8d200640 1 -- 172.21.15.66:0/679253641 wait complete. 2024-03-30T00:48:56.673 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:48:56.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.804+0000 7f3c74800640 1 Processor -- start 2024-03-30T00:48:56.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c74800640 1 -- start start 2024-03-30T00:48:56.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c74800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3c6c0cda90 0x7f3c6c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c74800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3c6c0cde70 0x7f3c6c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c74800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3c6c159c60 0x7f3c6c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c74800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c6c0b1b80 con 0x7f3c6c0cde70 2024-03-30T00:48:56.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c74800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c6c15a990 con 0x7f3c6c159c60 2024-03-30T00:48:56.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c74800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c0b11e0 con 0x7f3c6c0cda90 2024-03-30T00:48:56.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c71400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3c6c0cde70 0x7f3c6c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c71400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3c6c0cde70 0x7f3c6c0cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41918/0 (socket says 172.21.15.66:41918) 2024-03-30T00:48:56.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c71400640 1 -- 172.21.15.66:0/2780456655 learned_addr learned my addr 172.21.15.66:0/2780456655 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:56.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c71e00640 1 --2- 172.21.15.66:0/2780456655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3c6c0cda90 0x7f3c6c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c72800640 1 --2- 172.21.15.66:0/2780456655 >> v2:172.21.15.156:3300/0 conn(0x7f3c6c159c60 0x7f3c6c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c71400640 1 -- 172.21.15.66:0/2780456655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3c6c0cda90 msgr2=0x7f3c6c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:56.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c71400640 1 --2- 172.21.15.66:0/2780456655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3c6c0cda90 0x7f3c6c0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c71400640 1 -- 172.21.15.66:0/2780456655 >> v2:172.21.15.156:3300/0 conn(0x7f3c6c159c60 msgr2=0x7f3c6c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:56.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c71400640 1 --2- 172.21.15.66:0/2780456655 >> v2:172.21.15.156:3300/0 conn(0x7f3c6c159c60 0x7f3c6c15a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.808+0000 7f3c71400640 1 -- 172.21.15.66:0/2780456655 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c0b14c0 con 0x7f3c6c0cde70 2024-03-30T00:48:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c71400640 1 --2- 172.21.15.66:0/2780456655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c0cde70 0x7f3c6c0cea60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0724d0 tx=0x7f3c5c01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550b172dc2129b21 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c70a00640 1 -- 172.21.15.66:0/2780456655 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0a7c10 con 0x7f3c6c0cde70 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c70a00640 1 -- 172.21.15.66:0/2780456655 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c0a7d70 con 0x7f3c6c0cde70 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c70a00640 1 -- 172.21.15.66:0/2780456655 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0af660 con 0x7f3c6c0cde70 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c74800640 1 -- 172.21.15.66:0/2780456655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c0cde70 msgr2=0x7f3c6c0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c74800640 1 --2- 172.21.15.66:0/2780456655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c0cde70 0x7f3c6c0cea60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0724d0 tx=0x7f3c5c01eb60 comp rx=0 tx=0).stop 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c74800640 1 -- 172.21.15.66:0/2780456655 shutdown_connections 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c74800640 1 --2- 172.21.15.66:0/2780456655 >> v2:172.21.15.156:3300/0 conn(0x7f3c6c159c60 0x7f3c6c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c74800640 1 --2- 172.21.15.66:0/2780456655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c0cde70 0x7f3c6c0cea60 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c74800640 1 --2- 172.21.15.66:0/2780456655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3c6c0cda90 0x7f3c6c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c74800640 1 -- 172.21.15.66:0/2780456655 >> 172.21.15.66:0/2780456655 conn(0x7f3c6c069f60 msgr2=0x7f3c6c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c74800640 1 -- 172.21.15.66:0/2780456655 shutdown_connections 2024-03-30T00:48:56.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c74800640 1 -- 172.21.15.66:0/2780456655 wait complete. 2024-03-30T00:48:56.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c74800640 1 Processor -- start 2024-03-30T00:48:56.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.812+0000 7f3c74800640 1 -- start start 2024-03-30T00:48:56.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c74800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3c6c0cda90 0x7f3c6c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c74800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c6c0cde70 0x7f3c6c157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c74800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c159c60 0x7f3c6c1581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c74800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c156720 con 0x7f3c6c159c60 2024-03-30T00:48:56.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c74800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c1587b0 con 0x7f3c6c0cde70 2024-03-30T00:48:56.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c74800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c158a90 con 0x7f3c6c0cda90 2024-03-30T00:48:56.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c72800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c159c60 0x7f3c6c1581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c72800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c159c60 0x7f3c6c1581a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41922/0 (socket says 172.21.15.66:41922) 2024-03-30T00:48:56.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c72800640 1 -- 172.21.15.66:0/1252703406 learned_addr learned my addr 172.21.15.66:0/1252703406 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:56.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c71400640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c6c0cde70 0x7f3c6c157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c71e00640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3c6c0cda90 0x7f3c6c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c72800640 1 -- 172.21.15.66:0/1252703406 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3c6c0cda90 msgr2=0x7f3c6c1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c72800640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3c6c0cda90 0x7f3c6c1561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c72800640 1 -- 172.21.15.66:0/1252703406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c6c0cde70 msgr2=0x7f3c6c157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c72800640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c6c0cde70 0x7f3c6c157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c72800640 1 -- 172.21.15.66:0/1252703406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c00b770 con 0x7f3c6c159c60 2024-03-30T00:48:56.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c72800640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c159c60 0x7f3c6c1581a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3c68007e40 tx=0x7f3c6800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:56.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c66000640 1 -- 172.21.15.66:0/1252703406 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68014620 con 0x7f3c6c159c60 2024-03-30T00:48:56.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c66000640 1 -- 172.21.15.66:0/1252703406 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6800ee90 con 0x7f3c6c159c60 2024-03-30T00:48:56.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c66000640 1 -- 172.21.15.66:0/1252703406 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680149e0 con 0x7f3c6c159c60 2024-03-30T00:48:56.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c74800640 1 -- 172.21.15.66:0/1252703406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c158d70 con 0x7f3c6c159c60 2024-03-30T00:48:56.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c74800640 1 -- 172.21.15.66:0/1252703406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c159330 con 0x7f3c6c159c60 2024-03-30T00:48:56.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c74800640 1 -- 172.21.15.66:0/1252703406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6c066c30 con 0x7f3c6c159c60 2024-03-30T00:48:56.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.816+0000 7f3c66000640 1 -- 172.21.15.66:0/1252703406 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3c68014b40 con 0x7f3c6c159c60 2024-03-30T00:48:56.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.820+0000 7f3c66000640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3c500666c0 0x7f3c50068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:56.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.820+0000 7f3c66000640 1 -- 172.21.15.66:0/1252703406 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3c68084bd0 con 0x7f3c6c159c60 2024-03-30T00:48:56.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.820+0000 7f3c71e00640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3c500666c0 0x7f3c50068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:56.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.820+0000 7f3c71e00640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3c500666c0 0x7f3c50068b80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3c600045f0 tx=0x7f3c60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:56.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.824+0000 7f3c66000640 1 -- 172.21.15.66:0/1252703406 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3c68051e10 con 0x7f3c6c159c60 2024-03-30T00:48:56.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 -- 172.21.15.66:0/1252703406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3c6c0b3170 con 0x7f3c6c159c60 2024-03-30T00:48:56.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c66000640 1 -- 172.21.15.66:0/1252703406 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3c68055430 con 0x7f3c6c159c60 2024-03-30T00:48:56.995 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:48:56.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 -- 172.21.15.66:0/1252703406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3c500666c0 msgr2=0x7f3c50068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3c500666c0 0x7f3c50068b80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3c600045f0 tx=0x7f3c60008040 comp rx=0 tx=0).stop 2024-03-30T00:48:56.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 -- 172.21.15.66:0/1252703406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c159c60 msgr2=0x7f3c6c1581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:56.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c159c60 0x7f3c6c1581a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3c68007e40 tx=0x7f3c6800e4b0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 -- 172.21.15.66:0/1252703406 shutdown_connections 2024-03-30T00:48:56.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3c500666c0 0x7f3c50068b80 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c6c159c60 0x7f3c6c1581a0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c6c0cde70 0x7f3c6c157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 --2- 172.21.15.66:0/1252703406 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3c6c0cda90 0x7f3c6c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:56.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 -- 172.21.15.66:0/1252703406 >> 172.21.15.66:0/1252703406 conn(0x7f3c6c069f60 msgr2=0x7f3c6c06c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:56.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 -- 172.21.15.66:0/1252703406 shutdown_connections 2024-03-30T00:48:56.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:56.984+0000 7f3c74800640 1 -- 172.21.15.66:0/1252703406 wait complete. 2024-03-30T00:48:57.014 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:48:57.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f1325e00640 1 Processor -- start 2024-03-30T00:48:57.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f1325e00640 1 -- start start 2024-03-30T00:48:57.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f1325e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f13200ce780 0x7f13200cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f1325e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13200cd5e0 0x7f13200cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f1325e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13200cbec0 con 0x7f13200ceb60 2024-03-30T00:48:57.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f1325e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13200cdfa0 con 0x7f13200ce780 2024-03-30T00:48:57.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f1325e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13200cce50 con 0x7f13200cd5e0 2024-03-30T00:48:57.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f131ea00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f13200ce780 0x7f13200cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f131f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13200cd5e0 0x7f13200cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f131ea00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f13200ce780 0x7f13200cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:44822/0 (socket says 172.21.15.66:44822) 2024-03-30T00:48:57.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f131ea00640 1 -- 172.21.15.66:0/3736707379 learned_addr learned my addr 172.21.15.66:0/3736707379 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:57.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f1316000640 1 --1- 172.21.15.66:0/3736707379 >> v1:172.21.15.156:6789/0 conn(0x7f13200ceb60 0x7f132015ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:52224/0 (socket says 172.21.15.66:52224) 2024-03-30T00:48:57.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f131ea00640 1 -- 172.21.15.66:0/3736707379 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13200cd5e0 msgr2=0x7f13200cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:57.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f131ea00640 1 --2- 172.21.15.66:0/3736707379 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13200cd5e0 0x7f13200cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f131ea00640 1 -- 172.21.15.66:0/3736707379 >> v1:172.21.15.156:6789/0 conn(0x7f13200ceb60 legacy=0x7f132015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f131ea00640 1 -- 172.21.15.66:0/3736707379 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132015d970 con 0x7f13200ce780 2024-03-30T00:48:57.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.148+0000 7f131f400640 1 --2- 172.21.15.66:0/3736707379 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13200cd5e0 0x7f13200cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:48:57.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f131ea00640 1 --2- 172.21.15.66:0/3736707379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13200ce780 0x7f13200cc910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1310009870 tx=0x7f131001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46f24e3280118d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:57.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f131e000640 1 -- 172.21.15.66:0/3736707379 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100a6040 con 0x7f13200ce780 2024-03-30T00:48:57.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f131e000640 1 -- 172.21.15.66:0/3736707379 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13100aa600 con 0x7f13200ce780 2024-03-30T00:48:57.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f131e000640 1 -- 172.21.15.66:0/3736707379 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100aa990 con 0x7f13200ce780 2024-03-30T00:48:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f1325e00640 1 -- 172.21.15.66:0/3736707379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13200ce780 msgr2=0x7f13200cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f1325e00640 1 --2- 172.21.15.66:0/3736707379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13200ce780 0x7f13200cc910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1310009870 tx=0x7f131001c4a0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f1325e00640 1 -- 172.21.15.66:0/3736707379 shutdown_connections 2024-03-30T00:48:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f1325e00640 1 --2- 172.21.15.66:0/3736707379 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13200cd5e0 0x7f13200cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f1325e00640 1 --2- 172.21.15.66:0/3736707379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13200ce780 0x7f13200cc910 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f1325e00640 1 -- 172.21.15.66:0/3736707379 >> 172.21.15.66:0/3736707379 conn(0x7f1320069f60 msgr2=0x7f132006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f1325e00640 1 -- 172.21.15.66:0/3736707379 shutdown_connections 2024-03-30T00:48:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.152+0000 7f1325e00640 1 -- 172.21.15.66:0/3736707379 wait complete. 2024-03-30T00:48:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1325e00640 1 Processor -- start 2024-03-30T00:48:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1325e00640 1 -- start start 2024-03-30T00:48:57.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1325e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13200cd5e0 0x7f13200979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1325e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13200ce780 0x7f13200ac750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1325e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13200ceb60 0x7f132014e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1325e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13200ac330 con 0x7f13200cd5e0 2024-03-30T00:48:57.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1325e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1320097f10 con 0x7f13200ce780 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1325e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132006ce40 con 0x7f13200ceb60 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f131ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13200cd5e0 0x7f13200979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f131ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13200cd5e0 0x7f13200979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41934/0 (socket says 172.21.15.66:41934) 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f131ea00640 1 -- 172.21.15.66:0/1077607126 learned_addr learned my addr 172.21.15.66:0/1077607126 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f131f400640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13200ceb60 0x7f132014e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1316000640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13200ce780 0x7f13200ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f131ea00640 1 -- 172.21.15.66:0/1077607126 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13200ceb60 msgr2=0x7f132014e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f131ea00640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13200ceb60 0x7f132014e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f131ea00640 1 -- 172.21.15.66:0/1077607126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13200ce780 msgr2=0x7f13200ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f131ea00640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13200ce780 0x7f13200ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f131ea00640 1 -- 172.21.15.66:0/1077607126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318009a70 con 0x7f13200cd5e0 2024-03-30T00:48:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f131ea00640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13200cd5e0 0x7f13200979d0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f131001ab50 tx=0x7f131001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1317e00640 1 -- 172.21.15.66:0/1077607126 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100b3600 con 0x7f13200cd5e0 2024-03-30T00:48:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1317e00640 1 -- 172.21.15.66:0/1077607126 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13100b3760 con 0x7f13200cd5e0 2024-03-30T00:48:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1317e00640 1 -- 172.21.15.66:0/1077607126 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100b3af0 con 0x7f13200cd5e0 2024-03-30T00:48:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1325e00640 1 -- 172.21.15.66:0/1077607126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13100094d0 con 0x7f13200cd5e0 2024-03-30T00:48:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.156+0000 7f1325e00640 1 -- 172.21.15.66:0/1077607126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13201558f0 con 0x7f13200cd5e0 2024-03-30T00:48:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.160+0000 7f1325e00640 1 -- 172.21.15.66:0/1077607126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13200ac5a0 con 0x7f13200cd5e0 2024-03-30T00:48:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.160+0000 7f1317e00640 1 -- 172.21.15.66:0/1077607126 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f13100b3c50 con 0x7f13200cd5e0 2024-03-30T00:48:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.160+0000 7f1317e00640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12fc0666c0 0x7f12fc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.160+0000 7f1317e00640 1 -- 172.21.15.66:0/1077607126 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f131011ec20 con 0x7f13200cd5e0 2024-03-30T00:48:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.160+0000 7f1316000640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12fc0666c0 0x7f12fc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.160+0000 7f1316000640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12fc0666c0 0x7f12fc068b80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f13201f88b0 tx=0x7f1308007e50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:57.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.164+0000 7f1317e00640 1 -- 172.21.15.66:0/1077607126 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f13100ebef0 con 0x7f13200cd5e0 2024-03-30T00:48:57.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 -- 172.21.15.66:0/1077607126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1320066c30 con 0x7f13200cd5e0 2024-03-30T00:48:57.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1317e00640 1 -- 172.21.15.66:0/1077607126 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13100ef510 con 0x7f13200cd5e0 2024-03-30T00:48:57.334 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:48:57.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 -- 172.21.15.66:0/1077607126 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12fc0666c0 msgr2=0x7f12fc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12fc0666c0 0x7f12fc068b80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f13201f88b0 tx=0x7f1308007e50 comp rx=0 tx=0).stop 2024-03-30T00:48:57.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 -- 172.21.15.66:0/1077607126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13200cd5e0 msgr2=0x7f13200979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13200cd5e0 0x7f13200979d0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f131001ab50 tx=0x7f131001ace0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 -- 172.21.15.66:0/1077607126 shutdown_connections 2024-03-30T00:48:57.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f13200ceb60 0x7f132014e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13200ce780 0x7f13200ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12fc0666c0 0x7f12fc068b80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 --2- 172.21.15.66:0/1077607126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13200cd5e0 0x7f13200979d0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 -- 172.21.15.66:0/1077607126 >> 172.21.15.66:0/1077607126 conn(0x7f1320069f60 msgr2=0x7f13200cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:57.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 -- 172.21.15.66:0/1077607126 shutdown_connections 2024-03-30T00:48:57.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.324+0000 7f1325e00640 1 -- 172.21.15.66:0/1077607126 wait complete. 2024-03-30T00:48:57.355 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:48:57.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.484+0000 7fd661200640 1 Processor -- start 2024-03-30T00:48:57.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.484+0000 7fd661200640 1 -- start start 2024-03-30T00:48:57.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.484+0000 7fd661200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd65c071260 0x7fd65c15e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.484+0000 7fd661200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c06c410 con 0x7fd65c15db90 2024-03-30T00:48:57.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.484+0000 7fd661200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c06c6b0 con 0x7fd65c0ce6b0 2024-03-30T00:48:57.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.484+0000 7fd661200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65c06c990 con 0x7fd65c071260 2024-03-30T00:48:57.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.488+0000 7fd65a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd65c071260 0x7fd65c15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.488+0000 7fd651400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd65c15db90 0x7fd65c070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:54046/0 (socket says 172.21.15.66:54046) 2024-03-30T00:48:57.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.488+0000 7fd651400640 1 -- 172.21.15.66:0/4082310823 learned_addr learned my addr 172.21.15.66:0/4082310823 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:57.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.488+0000 7fd65a800640 1 -- 172.21.15.66:0/4082310823 >> v1:172.21.15.156:6789/0 conn(0x7fd65c0ce6b0 legacy=0x7fd65c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.488+0000 7fd65a800640 1 -- 172.21.15.66:0/4082310823 >> v1:172.21.15.66:6789/0 conn(0x7fd65c15db90 legacy=0x7fd65c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.488+0000 7fd65a800640 1 -- 172.21.15.66:0/4082310823 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c06cc70 con 0x7fd65c071260 2024-03-30T00:48:57.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.488+0000 7fd659400640 1 -- 172.21.15.66:0/4082310823 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553872791 0 0) 0x7fd65c06c410 con 0x7fd65c15db90 2024-03-30T00:48:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.488+0000 7fd65a800640 1 --2- 172.21.15.66:0/4082310823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65c071260 0x7fd65c15e070 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fd6540097c0 tx=0x7fd65401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5093b748e50f19 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.488+0000 7fd659400640 1 -- 172.21.15.66:0/4082310823 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6540ab600 con 0x7fd65c071260 2024-03-30T00:48:57.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.488+0000 7fd659400640 1 -- 172.21.15.66:0/4082310823 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6540ab760 con 0x7fd65c071260 2024-03-30T00:48:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.488+0000 7fd659400640 1 -- 172.21.15.66:0/4082310823 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6540abaf0 con 0x7fd65c071260 2024-03-30T00:48:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 -- 172.21.15.66:0/4082310823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65c071260 msgr2=0x7fd65c15e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 --2- 172.21.15.66:0/4082310823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65c071260 0x7fd65c15e070 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fd6540097c0 tx=0x7fd65401c3e0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 -- 172.21.15.66:0/4082310823 shutdown_connections 2024-03-30T00:48:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 --2- 172.21.15.66:0/4082310823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65c071260 0x7fd65c15e070 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 -- 172.21.15.66:0/4082310823 >> 172.21.15.66:0/4082310823 conn(0x7fd65c066100 msgr2=0x7fd65c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:57.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 -- 172.21.15.66:0/4082310823 shutdown_connections 2024-03-30T00:48:57.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 -- 172.21.15.66:0/4082310823 wait complete. 2024-03-30T00:48:57.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 Processor -- start 2024-03-30T00:48:57.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 -- start start 2024-03-30T00:48:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65c071260 0x7fd65c14e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd65c0ce6b0 0x7fd65c14c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd65c15db90 0x7fd65c151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c14cca0 con 0x7fd65c071260 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c158480 con 0x7fd65c15db90 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd661200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c158760 con 0x7fd65c0ce6b0 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd659e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65c071260 0x7fd65c14e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd659e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65c071260 0x7fd65c14e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41958/0 (socket says 172.21.15.66:41958) 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd659e00640 1 -- 172.21.15.66:0/814124862 learned_addr learned my addr 172.21.15.66:0/814124862 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd651400640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd65c0ce6b0 0x7fd65c14c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd65a800640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd65c15db90 0x7fd65c151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd659e00640 1 -- 172.21.15.66:0/814124862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd65c0ce6b0 msgr2=0x7fd65c14c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd659e00640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd65c0ce6b0 0x7fd65c14c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd659e00640 1 -- 172.21.15.66:0/814124862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd65c15db90 msgr2=0x7fd65c151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd659e00640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd65c15db90 0x7fd65c151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd659e00640 1 -- 172.21.15.66:0/814124862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd654009420 con 0x7fd65c071260 2024-03-30T00:48:57.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd65a800640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd65c15db90 0x7fd65c151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:57.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd659e00640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65c071260 0x7fd65c14e9a0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00c500 tx=0x7fd64c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:57.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.492+0000 7fd652a00640 1 -- 172.21.15.66:0/814124862 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c005220 con 0x7fd65c071260 2024-03-30T00:48:57.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.496+0000 7fd652a00640 1 -- 172.21.15.66:0/814124862 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c005380 con 0x7fd65c071260 2024-03-30T00:48:57.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.496+0000 7fd661200640 1 -- 172.21.15.66:0/814124862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c06e190 con 0x7fd65c071260 2024-03-30T00:48:57.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.496+0000 7fd652a00640 1 -- 172.21.15.66:0/814124862 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c007270 con 0x7fd65c071260 2024-03-30T00:48:57.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.496+0000 7fd661200640 1 -- 172.21.15.66:0/814124862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c158d20 con 0x7fd65c071260 2024-03-30T00:48:57.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.496+0000 7fd661200640 1 -- 172.21.15.66:0/814124862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c06e320 con 0x7fd65c071260 2024-03-30T00:48:57.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.496+0000 7fd652a00640 1 -- 172.21.15.66:0/814124862 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd64c0073d0 con 0x7fd65c071260 2024-03-30T00:48:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.496+0000 7fd652a00640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd638066690 0x7fd638068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.496+0000 7fd652a00640 1 -- 172.21.15.66:0/814124862 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd64c083cc0 con 0x7fd65c071260 2024-03-30T00:48:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.496+0000 7fd651400640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd638066690 0x7fd638068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.496+0000 7fd651400640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd638066690 0x7fd638068b50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd644005fd0 tx=0x7fd644005c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.500+0000 7fd652a00640 1 -- 172.21.15.66:0/814124862 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd64c08a080 con 0x7fd65c071260 2024-03-30T00:48:57.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.676+0000 7fd661200640 1 -- 172.21.15.66:0/814124862 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd65c066c30 con 0x7fd65c071260 2024-03-30T00:48:57.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.676+0000 7fd652a00640 1 -- 172.21.15.66:0/814124862 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd64c050e60 con 0x7fd65c071260 2024-03-30T00:48:57.687 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:48:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 -- 172.21.15.66:0/814124862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd638066690 msgr2=0x7fd638068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd638066690 0x7fd638068b50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd644005fd0 tx=0x7fd644005c20 comp rx=0 tx=0).stop 2024-03-30T00:48:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 -- 172.21.15.66:0/814124862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65c071260 msgr2=0x7fd65c14e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65c071260 0x7fd65c14e9a0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00c500 tx=0x7fd64c00c9c0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 -- 172.21.15.66:0/814124862 shutdown_connections 2024-03-30T00:48:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd65c15db90 0x7fd65c151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd65c0ce6b0 0x7fd65c14c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd638066690 0x7fd638068b50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 --2- 172.21.15.66:0/814124862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd65c071260 0x7fd65c14e9a0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 -- 172.21.15.66:0/814124862 >> 172.21.15.66:0/814124862 conn(0x7fd65c066100 msgr2=0x7fd65c070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 -- 172.21.15.66:0/814124862 shutdown_connections 2024-03-30T00:48:57.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.680+0000 7fd661200640 1 -- 172.21.15.66:0/814124862 wait complete. 2024-03-30T00:48:57.708 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:48:57.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:57.708 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:48:57.726 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[51003]: fuse finished with error 0 and tester_r 0 2024-03-30T00:48:57.731 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:48:57.756 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:48:57.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:57.757 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:48:57.806 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:48:57.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.976+0000 7faae9c00640 1 Processor -- start 2024-03-30T00:48:57.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.976+0000 7faae9c00640 1 -- start start 2024-03-30T00:48:57.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae9c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faae40cd2d0 0x7faae40cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae9c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faae40cc8d0 0x7faae406c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae9c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae40cb620 con 0x7faae40cad00 2024-03-30T00:48:57.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae9c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae40cbec0 con 0x7faae40cc8d0 2024-03-30T00:48:57.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae9c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae40ca660 con 0x7faae40cd2d0 2024-03-30T00:48:57.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faae40cc8d0 0x7faae406c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faae40cc8d0 0x7faae406c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41960/0 (socket says 172.21.15.66:41960) 2024-03-30T00:48:57.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae3400640 1 -- 172.21.15.66:0/1497086503 learned_addr learned my addr 172.21.15.66:0/1497086503 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:57.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faadbe00640 1 --2- 172.21.15.66:0/1497086503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faae40cd2d0 0x7faae40cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae3400640 1 -- 172.21.15.66:0/1497086503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faae40cd2d0 msgr2=0x7faae40cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:57.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae3400640 1 --2- 172.21.15.66:0/1497086503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faae40cd2d0 0x7faae40cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae3400640 1 -- 172.21.15.66:0/1497086503 >> v1:172.21.15.156:6789/0 conn(0x7faae40cad00 legacy=0x7faae40c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.980+0000 7faae3400640 1 -- 172.21.15.66:0/1497086503 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae40ca940 con 0x7faae40cc8d0 2024-03-30T00:48:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae3400640 1 --2- 172.21.15.66:0/1497086503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faae40cc8d0 0x7faae406c030 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7faadc009e50 tx=0x7faadc01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b226e44d699a51 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae2000640 1 -- 172.21.15.66:0/1497086503 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc0a57c0 con 0x7faae40cc8d0 2024-03-30T00:48:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae2000640 1 -- 172.21.15.66:0/1497086503 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faadc0a5920 con 0x7faae40cc8d0 2024-03-30T00:48:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae2000640 1 -- 172.21.15.66:0/1497086503 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc0a5cb0 con 0x7faae40cc8d0 2024-03-30T00:48:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 -- 172.21.15.66:0/1497086503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faae40cc8d0 msgr2=0x7faae406c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 --2- 172.21.15.66:0/1497086503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faae40cc8d0 0x7faae406c030 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7faadc009e50 tx=0x7faadc01ca00 comp rx=0 tx=0).stop 2024-03-30T00:48:57.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 -- 172.21.15.66:0/1497086503 shutdown_connections 2024-03-30T00:48:57.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 --2- 172.21.15.66:0/1497086503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faae40cc8d0 0x7faae406c030 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 --2- 172.21.15.66:0/1497086503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faae40cd2d0 0x7faae40cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 -- 172.21.15.66:0/1497086503 >> 172.21.15.66:0/1497086503 conn(0x7faae4069f60 msgr2=0x7faae406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:57.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 -- 172.21.15.66:0/1497086503 shutdown_connections 2024-03-30T00:48:57.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 -- 172.21.15.66:0/1497086503 wait complete. 2024-03-30T00:48:57.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 Processor -- start 2024-03-30T00:48:57.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 -- start start 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faae40cad00 0x7faae41541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faae40cc8d0 0x7faae409bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faae40cd2d0 0x7faae4151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae4156410 con 0x7faae40cc8d0 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae4155e50 con 0x7faae40cad00 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.984+0000 7faae9c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae4156130 con 0x7faae40cd2d0 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faae40cc8d0 0x7faae409bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faae40cc8d0 0x7faae409bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41970/0 (socket says 172.21.15.66:41970) 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadbe00640 1 -- 172.21.15.66:0/2802081147 learned_addr learned my addr 172.21.15.66:0/2802081147 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faae3400640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faae40cd2d0 0x7faae4151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faae2a00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faae40cad00 0x7faae41541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadbe00640 1 -- 172.21.15.66:0/2802081147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faae40cd2d0 msgr2=0x7faae4151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadbe00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faae40cd2d0 0x7faae4151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadbe00640 1 -- 172.21.15.66:0/2802081147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faae40cad00 msgr2=0x7faae41541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadbe00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faae40cad00 0x7faae41541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadbe00640 1 -- 172.21.15.66:0/2802081147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc009b30 con 0x7faae40cc8d0 2024-03-30T00:48:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadbe00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faae40cc8d0 0x7faae409bbc0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7faacc00e870 tx=0x7faacc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:57.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadaa00640 1 -- 172.21.15.66:0/2802081147 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc00ee60 con 0x7faae40cc8d0 2024-03-30T00:48:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadaa00640 1 -- 172.21.15.66:0/2802081147 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faacc00c940 con 0x7faae40cc8d0 2024-03-30T00:48:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faae9c00640 1 -- 172.21.15.66:0/2802081147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae40c5560 con 0x7faae40cc8d0 2024-03-30T00:48:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadaa00640 1 -- 172.21.15.66:0/2802081147 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc00ccb0 con 0x7faae40cc8d0 2024-03-30T00:48:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faae9c00640 1 -- 172.21.15.66:0/2802081147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae409c100 con 0x7faae40cc8d0 2024-03-30T00:48:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faae9c00640 1 -- 172.21.15.66:0/2802081147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae40c53a0 con 0x7faae40cc8d0 2024-03-30T00:48:58.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadaa00640 1 -- 172.21.15.66:0/2802081147 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faacc013470 con 0x7faae40cc8d0 2024-03-30T00:48:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.988+0000 7faadaa00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faac0066690 0x7faac0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.992+0000 7faadaa00640 1 -- 172.21.15.66:0/2802081147 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faacc0842e0 con 0x7faae40cc8d0 2024-03-30T00:48:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.992+0000 7faae2a00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faac0066690 0x7faac0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.992+0000 7faae2a00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faac0066690 0x7faac0068b50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7faad4007fd0 tx=0x7faad4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:58.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:57.992+0000 7faadaa00640 1 -- 172.21.15.66:0/2802081147 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faacc0515c0 con 0x7faae40cc8d0 2024-03-30T00:48:58.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.188+0000 7faae9c00640 1 -- 172.21.15.66:0/2802081147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faae4066c30 con 0x7faae40cc8d0 2024-03-30T00:48:58.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.188+0000 7faadaa00640 1 -- 172.21.15.66:0/2802081147 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4739 (secure 0 0 0) 0x7faacc054be0 con 0x7faae40cc8d0 2024-03-30T00:48:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:58.200 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":79,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:48:49.341688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8646":{"gid":8646,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/2988412199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2988412199},{"type":"v1","addr":"172.21.15.66:6837","nonce":2988412199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:48:58.200 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 79 2024-03-30T00:48:58.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 -- 172.21.15.66:0/2802081147 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faac0066690 msgr2=0x7faac0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faac0066690 0x7faac0068b50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7faad4007fd0 tx=0x7faad4009040 comp rx=0 tx=0).stop 2024-03-30T00:48:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 -- 172.21.15.66:0/2802081147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faae40cc8d0 msgr2=0x7faae409bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faae40cc8d0 0x7faae409bbc0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7faacc00e870 tx=0x7faacc00ed30 comp rx=0 tx=0).stop 2024-03-30T00:48:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 -- 172.21.15.66:0/2802081147 shutdown_connections 2024-03-30T00:48:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faae40cd2d0 0x7faae4151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7faac0066690 0x7faac0068b50 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faae40cc8d0 0x7faae409bbc0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 --2- 172.21.15.66:0/2802081147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faae40cad00 0x7faae41541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 -- 172.21.15.66:0/2802081147 >> 172.21.15.66:0/2802081147 conn(0x7faae4069f60 msgr2=0x7faae40af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 -- 172.21.15.66:0/2802081147 shutdown_connections 2024-03-30T00:48:58.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.192+0000 7faae9c00640 1 -- 172.21.15.66:0/2802081147 wait complete. 2024-03-30T00:48:58.223 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-30T00:48:58.391 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-03-30T00:48:58.391 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-30T00:48:58.391 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:48:58.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.520+0000 7f2d52800640 1 Processor -- start 2024-03-30T00:48:58.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.520+0000 7f2d52800640 1 -- start start 2024-03-30T00:48:58.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d52800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2d4c0cf320 0x7f2d4c15eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d52800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 0x7f2d4c15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d52800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4c099e70 con 0x7f2d4c098c80 2024-03-30T00:48:58.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d52800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4c0cbd90 con 0x7f2d4c0cf320 2024-03-30T00:48:58.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d52800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0cc070 con 0x7f2d4c099060 2024-03-30T00:48:58.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d4b400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2d4c098c80 0x7f2d4c099760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:54050/0 (socket says 172.21.15.66:54050) 2024-03-30T00:48:58.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d4b400640 1 -- 172.21.15.66:0/2450182412 learned_addr learned my addr 172.21.15.66:0/2450182412 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:48:58.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d4be00640 1 --2- 172.21.15.66:0/2450182412 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 0x7f2d4c15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d43e00640 1 --2- 172.21.15.66:0/2450182412 >> v2:172.21.15.156:3300/0 conn(0x7f2d4c0cf320 0x7f2d4c15eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d4aa00640 1 -- 172.21.15.66:0/2450182412 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547663238 0 0) 0x7f2d4c099e70 con 0x7f2d4c098c80 2024-03-30T00:48:58.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d4aa00640 1 -- 172.21.15.66:0/2450182412 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d30003530 con 0x7f2d4c098c80 2024-03-30T00:48:58.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d4be00640 1 -- 172.21.15.66:0/2450182412 >> v2:172.21.15.156:3300/0 conn(0x7f2d4c0cf320 msgr2=0x7f2d4c15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:58.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d4be00640 1 --2- 172.21.15.66:0/2450182412 >> v2:172.21.15.156:3300/0 conn(0x7f2d4c0cf320 0x7f2d4c15eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d4be00640 1 -- 172.21.15.66:0/2450182412 >> v1:172.21.15.66:6789/0 conn(0x7f2d4c098c80 legacy=0x7f2d4c099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d4be00640 1 -- 172.21.15.66:0/2450182412 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c0cc350 con 0x7f2d4c099060 2024-03-30T00:48:58.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.524+0000 7f2d4be00640 1 --2- 172.21.15.66:0/2450182412 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 0x7f2d4c15f0d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009d90 tx=0x7f2d4401f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48e8006bcb49bb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d4aa00640 1 -- 172.21.15.66:0/2450182412 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d440af600 con 0x7f2d4c099060 2024-03-30T00:48:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d4aa00640 1 -- 172.21.15.66:0/2450182412 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d440ab050 con 0x7f2d4c099060 2024-03-30T00:48:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d4aa00640 1 -- 172.21.15.66:0/2450182412 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d440af980 con 0x7f2d4c099060 2024-03-30T00:48:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 -- 172.21.15.66:0/2450182412 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 msgr2=0x7f2d4c15f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 --2- 172.21.15.66:0/2450182412 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 0x7f2d4c15f0d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009d90 tx=0x7f2d4401f150 comp rx=0 tx=0).stop 2024-03-30T00:48:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 -- 172.21.15.66:0/2450182412 shutdown_connections 2024-03-30T00:48:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 --2- 172.21.15.66:0/2450182412 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 0x7f2d4c15f0d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 --2- 172.21.15.66:0/2450182412 >> v2:172.21.15.156:3300/0 conn(0x7f2d4c0cf320 0x7f2d4c15eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 -- 172.21.15.66:0/2450182412 >> 172.21.15.66:0/2450182412 conn(0x7f2d4c0b2130 msgr2=0x7f2d4c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 -- 172.21.15.66:0/2450182412 shutdown_connections 2024-03-30T00:48:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 -- 172.21.15.66:0/2450182412 wait complete. 2024-03-30T00:48:58.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 Processor -- start 2024-03-30T00:48:58.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 -- start start 2024-03-30T00:48:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d4c098c80 0x7f2d4c14daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 0x7f2d4c09ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d4c0cf320 0x7f2d4c0c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0c5850 con 0x7f2d4c0cf320 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c14dfe0 con 0x7f2d4c098c80 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d52800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c09d820 con 0x7f2d4c099060 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d4c0cf320 0x7f2d4c0c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d4c0cf320 0x7f2d4c0c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41974/0 (socket says 172.21.15.66:41974) 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.528+0000 7f2d4be00640 1 -- 172.21.15.66:0/1168671564 learned_addr learned my addr 172.21.15.66:0/1168671564 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d4b400640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d4c098c80 0x7f2d4c14daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d43e00640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 0x7f2d4c09ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d4be00640 1 -- 172.21.15.66:0/1168671564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 msgr2=0x7f2d4c09ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d4be00640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 0x7f2d4c09ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d4be00640 1 -- 172.21.15.66:0/1168671564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d4c098c80 msgr2=0x7f2d4c14daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d4be00640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d4c098c80 0x7f2d4c14daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d4be00640 1 -- 172.21.15.66:0/1168671564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d44009a70 con 0x7f2d4c0cf320 2024-03-30T00:48:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d4b400640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d4c098c80 0x7f2d4c14daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d43e00640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 0x7f2d4c09ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d4be00640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d4c0cf320 0x7f2d4c0c8f30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f2d440b2040 tx=0x7f2d4401bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d43400640 1 -- 172.21.15.66:0/1168671564 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d44009450 con 0x7f2d4c0cf320 2024-03-30T00:48:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d43400640 1 -- 172.21.15.66:0/1168671564 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d440b9410 con 0x7f2d4c0cf320 2024-03-30T00:48:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d43400640 1 -- 172.21.15.66:0/1168671564 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d440b97c0 con 0x7f2d4c0cf320 2024-03-30T00:48:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d52800640 1 -- 172.21.15.66:0/1168671564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c09db00 con 0x7f2d4c0cf320 2024-03-30T00:48:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d52800640 1 -- 172.21.15.66:0/1168671564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c09e0c0 con 0x7f2d4c0cf320 2024-03-30T00:48:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d52800640 1 -- 172.21.15.66:0/1168671564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4c0677d0 con 0x7f2d4c0cf320 2024-03-30T00:48:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.532+0000 7f2d43400640 1 -- 172.21.15.66:0/1168671564 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2d440b6050 con 0x7f2d4c0cf320 2024-03-30T00:48:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.536+0000 7f2d43400640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2d28066690 0x7f2d28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.536+0000 7f2d43400640 1 -- 172.21.15.66:0/1168671564 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2d44122880 con 0x7f2d4c0cf320 2024-03-30T00:48:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.536+0000 7f2d4b400640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2d28066690 0x7f2d28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.536+0000 7f2d4b400640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2d28066690 0x7f2d28068b50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c005890 tx=0x7f2d3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:58.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.540+0000 7f2d43400640 1 -- 172.21.15.66:0/1168671564 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2d440ef980 con 0x7f2d4c0cf320 2024-03-30T00:48:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 -- 172.21.15.66:0/1168671564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d4c0c5990 con 0x7f2d4c0cf320 2024-03-30T00:48:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d43400640 1 -- 172.21.15.66:0/1168671564 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f2d4c0c5990 con 0x7f2d4c0cf320 2024-03-30T00:48:58.750 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:48:58.750 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":79,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:48:49.341688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8646":{"gid":8646,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/2988412199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2988412199},{"type":"v1","addr":"172.21.15.66:6837","nonce":2988412199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:48:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 79 2024-03-30T00:48:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 -- 172.21.15.66:0/1168671564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2d28066690 msgr2=0x7f2d28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2d28066690 0x7f2d28068b50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c005890 tx=0x7f2d3c009040 comp rx=0 tx=0).stop 2024-03-30T00:48:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 -- 172.21.15.66:0/1168671564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d4c0cf320 msgr2=0x7f2d4c0c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d4c0cf320 0x7f2d4c0c8f30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f2d440b2040 tx=0x7f2d4401bef0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 -- 172.21.15.66:0/1168671564 shutdown_connections 2024-03-30T00:48:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2d28066690 0x7f2d28068b50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d4c0cf320 0x7f2d4c0c8f30 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2d4c099060 0x7f2d4c09ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 --2- 172.21.15.66:0/1168671564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d4c098c80 0x7f2d4c14daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 -- 172.21.15.66:0/1168671564 >> 172.21.15.66:0/1168671564 conn(0x7f2d4c0b2130 msgr2=0x7f2d4c099760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 -- 172.21.15.66:0/1168671564 shutdown_connections 2024-03-30T00:48:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.740+0000 7f2d52800640 1 -- 172.21.15.66:0/1168671564 wait complete. 2024-03-30T00:48:58.772 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-03-30T00:48:58.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 Processor -- start 2024-03-30T00:48:58.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 -- start start 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8050fee30 0x55c8050cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c805118910 0x55c8050cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c805118cf0 0x55c805188a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c805014960 con 0x55c8050fee30 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c805014600 con 0x55c805118910 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c80500b690 con 0x55c805118cf0 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8940e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8050fee30 0x55c8050cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8940e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8050fee30 0x55c8050cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53070/0 (socket says 172.21.15.66:53070) 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8940e00640 1 -- 172.21.15.66:0/649945423 learned_addr learned my addr 172.21.15.66:0/649945423 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8941800640 1 --2- 172.21.15.66:0/649945423 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c805118cf0 0x55c805188a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f893be00640 1 --2- 172.21.15.66:0/649945423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c805118910 0x55c8050cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8940e00640 1 -- 172.21.15.66:0/649945423 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c805118cf0 msgr2=0x55c805188a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:58.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8940e00640 1 --2- 172.21.15.66:0/649945423 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c805118cf0 0x55c805188a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8940e00640 1 -- 172.21.15.66:0/649945423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c805118910 msgr2=0x55c8050cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:58.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8940e00640 1 --2- 172.21.15.66:0/649945423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c805118910 0x55c8050cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8940e00640 1 -- 172.21.15.66:0/649945423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8934005df0 con 0x55c8050fee30 2024-03-30T00:48:58.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8940e00640 1 --2- 172.21.15.66:0/649945423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8050fee30 0x55c8050cd820 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f89340af040 tx=0x7f8934003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3df9ef96d016ce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:58.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f893aa00640 1 -- 172.21.15.66:0/649945423 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89340a5bb0 con 0x55c8050fee30 2024-03-30T00:48:58.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f893aa00640 1 -- 172.21.15.66:0/649945423 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89340a5d10 con 0x55c8050fee30 2024-03-30T00:48:58.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f893aa00640 1 -- 172.21.15.66:0/649945423 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89340b7670 con 0x55c8050fee30 2024-03-30T00:48:58.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 -- 172.21.15.66:0/649945423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8050fee30 msgr2=0x55c8050cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 --2- 172.21.15.66:0/649945423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8050fee30 0x55c8050cd820 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f89340af040 tx=0x7f8934003030 comp rx=0 tx=0).stop 2024-03-30T00:48:58.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 -- 172.21.15.66:0/649945423 shutdown_connections 2024-03-30T00:48:58.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 --2- 172.21.15.66:0/649945423 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c805118cf0 0x55c805188a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 --2- 172.21.15.66:0/649945423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c805118910 0x55c8050cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 --2- 172.21.15.66:0/649945423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8050fee30 0x55c8050cd820 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 -- 172.21.15.66:0/649945423 >> 172.21.15.66:0/649945423 conn(0x55c8050fd2f0 msgr2=0x55c805189ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:58.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 -- 172.21.15.66:0/649945423 shutdown_connections 2024-03-30T00:48:58.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 -- 172.21.15.66:0/649945423 wait complete. 2024-03-30T00:48:58.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 Processor -- start 2024-03-30T00:48:58.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.808+0000 7f8943b38080 1 -- start start 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8943b38080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8050fee30 0x55c8051a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8943b38080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c805118910 0x55c80523faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8943b38080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c805118cf0 0x55c80523cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8943b38080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c80519b120 con 0x55c805118910 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8943b38080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8051a6460 con 0x55c8050fee30 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8943b38080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8051a6700 con 0x55c805118cf0 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f893be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c805118910 0x55c80523faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f893be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c805118910 0x55c80523faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53082/0 (socket says 172.21.15.66:53082) 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f893be00640 1 -- 172.21.15.66:0/890903522 learned_addr learned my addr 172.21.15.66:0/890903522 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8941800640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c805118cf0 0x55c80523cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8940e00640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8050fee30 0x55c8051a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f893be00640 1 -- 172.21.15.66:0/890903522 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c805118cf0 msgr2=0x55c80523cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f893be00640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c805118cf0 0x55c80523cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f893be00640 1 -- 172.21.15.66:0/890903522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8050fee30 msgr2=0x55c8051a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f893be00640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8050fee30 0x55c8051a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f893be00640 1 -- 172.21.15.66:0/890903522 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c80503da60 con 0x55c805118910 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8941800640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c805118cf0 0x55c80523cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8940e00640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8050fee30 0x55c8051a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f893be00640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c805118910 0x55c80523faf0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f892c00a6a0 tx=0x7f892c00ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8939600640 1 -- 172.21.15.66:0/890903522 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c00aea0 con 0x55c805118910 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8939600640 1 -- 172.21.15.66:0/890903522 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c008950 con 0x55c805118910 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8939600640 1 -- 172.21.15.66:0/890903522 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c008d00 con 0x55c805118910 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8943b38080 1 -- 172.21.15.66:0/890903522 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c8050238c0 con 0x55c805118910 2024-03-30T00:48:58.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8943b38080 1 -- 172.21.15.66:0/890903522 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c805016d00 con 0x55c805118910 2024-03-30T00:48:58.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.812+0000 7f8939600640 1 -- 172.21.15.66:0/890903522 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f892c006040 con 0x55c805118910 2024-03-30T00:48:58.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8939600640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f89200650a0 0x7f8920067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8939600640 1 -- 172.21.15.66:0/890903522 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f892c082980 con 0x55c805118910 2024-03-30T00:48:58.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8940e00640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f89200650a0 0x7f8920067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55c80523d280 0x55c80519d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:58.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 -- 172.21.15.66:0/890903522 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55c8051a5ee0 con 0x55c80523d280 2024-03-30T00:48:58.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8941800640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55c80523d280 0x55c80519d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:58.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8940e00640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f89200650a0 0x7f8920067560 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f89340af040 tx=0x7f89340a8000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:58.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8941800640 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55c80523d280 0x55c80519d980 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:58.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8941800640 1 -- 172.21.15.66:0/890903522 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7f893c00e8f0 con 0x55c80523d280 2024-03-30T00:48:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 -- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55c80523d280 msgr2=0x55c80519d980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55c80523d280 0x55c80519d980 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 -- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f89200650a0 msgr2=0x7f8920067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f89200650a0 0x7f8920067560 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f89340af040 tx=0x7f89340a8000 comp rx=0 tx=0).stop 2024-03-30T00:48:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 -- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c805118910 msgr2=0x55c80523faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 --2- 172.21.15.66:0/890903522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c805118910 0x55c80523faf0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f892c00a6a0 tx=0x7f892c00ab60 comp rx=0 tx=0).stop 2024-03-30T00:48:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8941800640 1 -- 172.21.15.66:0/890903522 reap_dead start 2024-03-30T00:48:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 -- 172.21.15.66:0/890903522 shutdown_connections 2024-03-30T00:48:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 -- 172.21.15.66:0/890903522 >> 172.21.15.66:0/890903522 conn(0x55c8050fd2f0 msgr2=0x55c8051a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 -- 172.21.15.66:0/890903522 shutdown_connections 2024-03-30T00:48:58.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:58.816+0000 7f8943b38080 1 -- 172.21.15.66:0/890903522 wait complete. 2024-03-30T00:48:58.832 INFO:teuthology.orchestra.run.smithi066.stdout:����� 2024-03-30T00:48:58.832 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:48:59.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.000+0000 7fa9fca00640 1 Processor -- start 2024-03-30T00:48:59.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.000+0000 7fa9fca00640 1 -- start start 2024-03-30T00:48:59.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.000+0000 7fa9fca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa9f815a810 0x7fa9f80ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.000+0000 7fa9fca00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa9f80cf290 0x7fa9f80c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.000+0000 7fa9fca00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f80c4600 con 0x7fa9f8099430 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9fca00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f80cc510 con 0x7fa9f815a810 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9fca00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f80998c0 con 0x7fa9f80cf290 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9f5e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa9f815a810 0x7fa9f80ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9f5e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa9f815a810 0x7fa9f80ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53088/0 (socket says 172.21.15.66:53088) 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9f5e00640 1 -- 172.21.15.66:0/1123016503 learned_addr learned my addr 172.21.15.66:0/1123016503 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9f6800640 1 --2- 172.21.15.66:0/1123016503 >> v2:172.21.15.156:3300/0 conn(0x7fa9f80cf290 0x7fa9f80c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9f5e00640 1 -- 172.21.15.66:0/1123016503 >> v2:172.21.15.156:3300/0 conn(0x7fa9f80cf290 msgr2=0x7fa9f80c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9f5e00640 1 --2- 172.21.15.66:0/1123016503 >> v2:172.21.15.156:3300/0 conn(0x7fa9f80cf290 0x7fa9f80c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9f5e00640 1 -- 172.21.15.66:0/1123016503 >> v1:172.21.15.156:6789/0 conn(0x7fa9f8099430 legacy=0x7fa9f80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:48:59.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9f5e00640 1 -- 172.21.15.66:0/1123016503 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f8099ba0 con 0x7fa9f815a810 2024-03-30T00:48:59.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9f5e00640 1 --2- 172.21.15.66:0/1123016503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f815a810 0x7fa9f80ce380 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80c75d0 tx=0x7fa9e801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9651c976cdbe8722 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:59.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.004+0000 7fa9f5400640 1 -- 172.21.15.66:0/1123016503 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80a8a10 con 0x7fa9f815a810 2024-03-30T00:48:59.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9f5400640 1 -- 172.21.15.66:0/1123016503 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e80a8b70 con 0x7fa9f815a810 2024-03-30T00:48:59.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9f5400640 1 -- 172.21.15.66:0/1123016503 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80b0440 con 0x7fa9f815a810 2024-03-30T00:48:59.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 -- 172.21.15.66:0/1123016503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f815a810 msgr2=0x7fa9f80ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:59.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 --2- 172.21.15.66:0/1123016503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f815a810 0x7fa9f80ce380 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80c75d0 tx=0x7fa9e801ed10 comp rx=0 tx=0).stop 2024-03-30T00:48:59.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 -- 172.21.15.66:0/1123016503 shutdown_connections 2024-03-30T00:48:59.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 --2- 172.21.15.66:0/1123016503 >> v2:172.21.15.156:3300/0 conn(0x7fa9f80cf290 0x7fa9f80c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:59.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 --2- 172.21.15.66:0/1123016503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f815a810 0x7fa9f80ce380 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:59.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 -- 172.21.15.66:0/1123016503 >> 172.21.15.66:0/1123016503 conn(0x7fa9f80b2090 msgr2=0x7fa9f80b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:59.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 -- 172.21.15.66:0/1123016503 shutdown_connections 2024-03-30T00:48:59.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 -- 172.21.15.66:0/1123016503 wait complete. 2024-03-30T00:48:59.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 Processor -- start 2024-03-30T00:48:59.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 -- start start 2024-03-30T00:48:59.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f80cf290 0x7fa9f8155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9f815a810 0x7fa9f8153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9f806aea0 0x7fa9f806b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f8155b40 con 0x7fa9f80cf290 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f8154030 con 0x7fa9f815a810 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9fca00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f806bd40 con 0x7fa9f806aea0 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f80cf290 0x7fa9f8155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9f5e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f80cf290 0x7fa9f8155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53100/0 (socket says 172.21.15.66:53100) 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9f5e00640 1 -- 172.21.15.66:0/2150017667 learned_addr learned my addr 172.21.15.66:0/2150017667 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9f6800640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9f806aea0 0x7fa9f806b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9efe00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9f815a810 0x7fa9f8153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9f5e00640 1 -- 172.21.15.66:0/2150017667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9f806aea0 msgr2=0x7fa9f806b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9f5e00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9f806aea0 0x7fa9f806b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9f5e00640 1 -- 172.21.15.66:0/2150017667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9f815a810 msgr2=0x7fa9f8153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:59.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.008+0000 7fa9f5e00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9f815a810 0x7fa9f8153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:59.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9f5e00640 1 -- 172.21.15.66:0/2150017667 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e80094d0 con 0x7fa9f80cf290 2024-03-30T00:48:59.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9f5e00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f80cf290 0x7fa9f8155030 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa9e801b920 tx=0x7fa9e801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:59.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9ee000640 1 -- 172.21.15.66:0/2150017667 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80ba600 con 0x7fa9f80cf290 2024-03-30T00:48:59.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9ee000640 1 -- 172.21.15.66:0/2150017667 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e80ba760 con 0x7fa9f80cf290 2024-03-30T00:48:59.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9ee000640 1 -- 172.21.15.66:0/2150017667 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80bab10 con 0x7fa9f80cf290 2024-03-30T00:48:59.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9fca00640 1 -- 172.21.15.66:0/2150017667 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f806ba60 con 0x7fa9f80cf290 2024-03-30T00:48:59.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9fca00640 1 -- 172.21.15.66:0/2150017667 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f8154830 con 0x7fa9f80cf290 2024-03-30T00:48:59.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9fca00640 1 -- 172.21.15.66:0/2150017667 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f8066c30 con 0x7fa9f80cf290 2024-03-30T00:48:59.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9ee000640 1 -- 172.21.15.66:0/2150017667 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa9e80bac70 con 0x7fa9f80cf290 2024-03-30T00:48:59.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9ee000640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa9d4066690 0x7fa9d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:48:59.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9ee000640 1 -- 172.21.15.66:0/2150017667 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa9e81226a0 con 0x7fa9f80cf290 2024-03-30T00:48:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.012+0000 7fa9efe00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa9d4066690 0x7fa9d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:48:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.016+0000 7fa9efe00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa9d4066690 0x7fa9d4068b50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80b2b40 tx=0x7fa9e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:48:59.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.016+0000 7fa9ee000640 1 -- 172.21.15.66:0/2150017667 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa9e80ef970 con 0x7fa9f80cf290 2024-03-30T00:48:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.216+0000 7fa9fca00640 1 -- 172.21.15.66:0/2150017667 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa9f80b2bb0 con 0x7fa9f80cf290 2024-03-30T00:48:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.216+0000 7fa9ee000640 1 -- 172.21.15.66:0/2150017667 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa9e80f2f90 con 0x7fa9f80cf290 2024-03-30T00:48:59.228 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:48:59.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 -- 172.21.15.66:0/2150017667 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa9d4066690 msgr2=0x7fa9d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:59.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa9d4066690 0x7fa9d4068b50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80b2b40 tx=0x7fa9e0009040 comp rx=0 tx=0).stop 2024-03-30T00:48:59.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 -- 172.21.15.66:0/2150017667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f80cf290 msgr2=0x7fa9f8155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:48:59.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f80cf290 0x7fa9f8155030 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa9e801b920 tx=0x7fa9e801bab0 comp rx=0 tx=0).stop 2024-03-30T00:48:59.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 -- 172.21.15.66:0/2150017667 shutdown_connections 2024-03-30T00:48:59.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9f806aea0 0x7fa9f806b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:59.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9f815a810 0x7fa9f8153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:59.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa9d4066690 0x7fa9d4068b50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:59.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 --2- 172.21.15.66:0/2150017667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f80cf290 0x7fa9f8155030 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:48:59.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 -- 172.21.15.66:0/2150017667 >> 172.21.15.66:0/2150017667 conn(0x7fa9f80b2090 msgr2=0x7fa9f80c4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:48:59.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 -- 172.21.15.66:0/2150017667 shutdown_connections 2024-03-30T00:48:59.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:48:59.220+0000 7fa9fca00640 1 -- 172.21.15.66:0/2150017667 wait complete. 2024-03-30T00:48:59.249 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:48:59.249 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:48:59.249 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:48:59.249 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:48:59.249 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:48:59.249 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:48:59.249 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:48:59.249 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:48:59.249 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:48:59.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:59.249 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:48:59.259 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:48:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41939sec preferred_lft 41939sec 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:48:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:48:59.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:59.263 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:48:59.319 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:48:59.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:59.321 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:48:59.377 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:48:59.377 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:48:59.377 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:48:59.377 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:48:59.377 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:48:59.377 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:48:59.378 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:48:59.378 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:48:59.378 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:48:59.378 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:48:59.378 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:48:59.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:59.378 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:48:59.378 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:48:59.378 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:48:59.378 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:48:59.379 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:48:59.379 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:48:59.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:59.471 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:48:59.479 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:48:59.479 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:48:59.480 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:48:59.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:59.481 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:48:59.540 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:48:59.541 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:48:59.541 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:48:59.589 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:48:59.589 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:48:59.589 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:48:59.640 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:48:59.702 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:48:59.751 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:48:59.770 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:48:59.772 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:48:59.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:59.772 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:48:59.821 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:48:59.821 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:48:59.867 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:48:59.887 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:48:59.888 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:48:59.947 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:48:59.949 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:48:59.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:48:59.949 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:48:59.962 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:48:59.952+0000 7f135004b580 -1 init, newargv = 0x55c5875fcf60 newargc=15 2024-03-30T00:48:59.962 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[51453]: starting ceph client 2024-03-30T00:48:59.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[51453]: starting fuse 2024-03-30T00:48:59.971 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:48:59.971 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:48:59.971 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:48:59.971 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:48:59.971 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:48:59.971 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:48:59.971 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:48:59.971 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:48:59.971 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:48:59.972 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:48:59.973 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:48:59.973 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:49:00.093 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.51453.asok 2024-03-30T00:49:00.277 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51453.asok 2024-03-30T00:49:00.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:49:00.277 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51453.asok status 2024-03-30T00:49:00.453 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:49:00.453 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:49:00.453 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:49:00.453 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:49:00.453 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:49:00.453 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:49:00.453 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:49:00.453 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "51453", 2024-03-30T00:49:00.453 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:49:00.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:49:00.453 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 8901, 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 8901 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 4035773102 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:49:00.454 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:49:00.455 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 4035773102 2024-03-30T00:49:00.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:49:00.455 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.8901 192.168.0.1:0/4035773102", 2024-03-30T00:49:00.455 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/4035773102", 2024-03-30T00:49:00.455 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:49:00.455 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 79, 2024-03-30T00:49:00.455 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 110, 2024-03-30T00:49:00.455 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:49:00.455 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:49:00.455 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:49:00.455 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:49:00.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:49:00.468 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:49:00.468 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:49:00.478 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:49:00.478 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:49:00.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:49:00.478 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:49:00.540 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:49:00.540 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:49:00.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:49:00.589 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-30T00:49:00.721 INFO:teuthology.orchestra.run.smithi066.stderr:6+0 records in 2024-03-30T00:49:00.721 INFO:teuthology.orchestra.run.smithi066.stderr:6+0 records out 2024-03-30T00:49:00.721 INFO:teuthology.orchestra.run.smithi066.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0747078 s, 84.2 MB/s 2024-03-30T00:49:00.722 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:49:00.722 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:49:00.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:49:00.732 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-30T00:49:00.852 INFO:teuthology.orchestra.run.smithi066.stderr:6+0 records in 2024-03-30T00:49:00.853 INFO:teuthology.orchestra.run.smithi066.stderr:6+0 records out 2024-03-30T00:49:00.853 INFO:teuthology.orchestra.run.smithi066.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0690711 s, 91.1 MB/s 2024-03-30T00:49:00.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:49:00.854 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-30T00:49:00.867 INFO:teuthology.orchestra.run.smithi066.stdout:./ 2024-03-30T00:49:00.867 INFO:teuthology.orchestra.run.smithi066.stdout:./file2_sixmegs 2024-03-30T00:49:00.867 INFO:teuthology.orchestra.run.smithi066.stdout:./file3_sixmegs 2024-03-30T00:49:00.867 INFO:teuthology.orchestra.run.smithi066.stdout:./file1_sixmegs 2024-03-30T00:49:00.868 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:49:00.868 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:49:00.868 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:49:00.869 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:49:00.869 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-30T00:49:00.869 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:49:00.976 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:49:01.166 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:49:01.166 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:49:01.166 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:49:01.166 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:49:01.166 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-03-30T00:49:01.166 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:49:01.277 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627777 2024-03-30T00:49:01.467 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:49:01.467 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:49:01.467 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:49:01.467 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:49:01.467 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-03-30T00:49:01.468 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:49:01.577 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627778 2024-03-30T00:49:01.766 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:49:01.767 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:49:01.767 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:49:01.767 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:49:01.767 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-03-30T00:49:01.767 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:49:01.873 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627776 2024-03-30T00:49:02.062 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:49:02.062 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:49:02.118 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:49:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.288+0000 7f6a7b400640 1 Processor -- start 2024-03-30T00:49:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.288+0000 7f6a7b400640 1 -- start start 2024-03-30T00:49:02.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a7b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a7415a640 0x7f6a7406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a7b400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6a740cabb0 0x7f6a740cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a7b400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7406d070 con 0x7f6a740ce520 2024-03-30T00:49:02.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a7b400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a740cbba0 con 0x7f6a740cabb0 2024-03-30T00:49:02.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a7b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7406e2c0 con 0x7f6a7415a640 2024-03-30T00:49:02.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a6be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6a740ce520 0x7f6a740ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56274/0 (socket says 172.21.15.66:56274) 2024-03-30T00:49:02.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a6be00640 1 -- 172.21.15.66:0/538602214 learned_addr learned my addr 172.21.15.66:0/538602214 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a79400640 1 --2- 172.21.15.66:0/538602214 >> v2:172.21.15.156:3300/0 conn(0x7f6a740cabb0 0x7f6a740cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a78a00640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a7415a640 0x7f6a7406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285235875 0 0) 0x7f6a7406d070 con 0x7f6a740ce520 2024-03-30T00:49:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a58003530 con 0x7f6a740ce520 2024-03-30T00:49:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2438129862 0 0) 0x7f6a58003530 con 0x7f6a740ce520 2024-03-30T00:49:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a7415a640 msgr2=0x7f6a7406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a7415a640 0x7f6a7406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 >> v2:172.21.15.156:3300/0 conn(0x7f6a740cabb0 msgr2=0x7f6a740cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 --2- 172.21.15.66:0/538602214 >> v2:172.21.15.156:3300/0 conn(0x7f6a740cabb0 0x7f6a740cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a740cb8c0 con 0x7f6a740ce520 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f6a5c001680 con 0x7f6a740ce520 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a79400640 1 --2- 172.21.15.66:0/538602214 >> v2:172.21.15.156:3300/0 conn(0x7f6a740cabb0 0x7f6a740cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 >> v1:172.21.15.66:6789/0 conn(0x7f6a740ce520 legacy=0x7f6a740ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a58003980 0x7f6a58003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a7415a640 0x7f6a580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a58004d10 0x7f6a58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a740cbba0 con 0x7f6a58003980 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7406e2c0 con 0x7f6a58004d10 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5800b990 con 0x7f6a7415a640 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a78a00640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a58003980 0x7f6a58003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a79400640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a58004d10 0x7f6a58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a6be00640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a7415a640 0x7f6a580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a78a00640 1 -- 172.21.15.66:0/538602214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a7415a640 msgr2=0x7f6a580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:02.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a78a00640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a7415a640 0x7f6a580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a78a00640 1 -- 172.21.15.66:0/538602214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a58004d10 msgr2=0x7f6a58005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:02.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a78a00640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a58004d10 0x7f6a58005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.292+0000 7f6a78a00640 1 -- 172.21.15.66:0/538602214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a58003530 con 0x7f6a58003980 2024-03-30T00:49:02.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.296+0000 7f6a78a00640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a58003980 0x7f6a58003da0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009ae0 tx=0x7f6a6401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ee0dd447e632ac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:02.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.296+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a640a5520 con 0x7f6a58003980 2024-03-30T00:49:02.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.296+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a640a5680 con 0x7f6a58003980 2024-03-30T00:49:02.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.296+0000 7f6a73e00640 1 -- 172.21.15.66:0/538602214 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a640a5a30 con 0x7f6a58003980 2024-03-30T00:49:02.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.296+0000 7f6a7b400640 1 -- 172.21.15.66:0/538602214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a58003980 msgr2=0x7f6a58003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:02.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.296+0000 7f6a7b400640 1 --2- 172.21.15.66:0/538602214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a58003980 0x7f6a58003da0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009ae0 tx=0x7f6a6401ca40 comp rx=0 tx=0).stop 2024-03-30T00:49:02.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.296+0000 7f6a79400640 1 -- 172.21.15.66:0/538602214 reap_dead start 2024-03-30T00:49:02.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.296+0000 7f6a7b400640 1 -- 172.21.15.66:0/538602214 shutdown_connections 2024-03-30T00:49:02.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.296+0000 7f6a7b400640 1 -- 172.21.15.66:0/538602214 >> 172.21.15.66:0/538602214 conn(0x7f6a74069f60 msgr2=0x7f6a7406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:02.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.296+0000 7f6a7b400640 1 -- 172.21.15.66:0/538602214 shutdown_connections 2024-03-30T00:49:02.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.296+0000 7f6a7b400640 1 -- 172.21.15.66:0/538602214 wait complete. 2024-03-30T00:49:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a7b400640 1 Processor -- start 2024-03-30T00:49:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a7b400640 1 -- start start 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a7b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a58003980 0x7f6a740c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a7b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a74159030 0x7f6a741f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a7b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740abd60 0x7f6a740ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a7b400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a740ac720 con 0x7f6a740abd60 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a7b400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a741598b0 con 0x7f6a58003980 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a7b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a740c5da0 con 0x7f6a74159030 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a79400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740abd60 0x7f6a740ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a79400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740abd60 0x7f6a740ac1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53116/0 (socket says 172.21.15.66:53116) 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a79400640 1 -- 172.21.15.66:0/1479235954 learned_addr learned my addr 172.21.15.66:0/1479235954 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a78a00640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a58003980 0x7f6a740c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a6be00640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a74159030 0x7f6a741f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a79400640 1 -- 172.21.15.66:0/1479235954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a74159030 msgr2=0x7f6a741f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a79400640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a74159030 0x7f6a741f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a79400640 1 -- 172.21.15.66:0/1479235954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a58003980 msgr2=0x7f6a740c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a79400640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a58003980 0x7f6a740c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a79400640 1 -- 172.21.15.66:0/1479235954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a740cc530 con 0x7f6a740abd60 2024-03-30T00:49:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a79400640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740abd60 0x7f6a740ac1e0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009f50 tx=0x7f6a6c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a71600640 1 -- 172.21.15.66:0/1479235954 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c01c070 con 0x7f6a740abd60 2024-03-30T00:49:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a71600640 1 -- 172.21.15.66:0/1479235954 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c00f040 con 0x7f6a740abd60 2024-03-30T00:49:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a71600640 1 -- 172.21.15.66:0/1479235954 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00e040 con 0x7f6a740abd60 2024-03-30T00:49:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a7b400640 1 -- 172.21.15.66:0/1479235954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a740cbf20 con 0x7f6a740abd60 2024-03-30T00:49:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a7b400640 1 -- 172.21.15.66:0/1479235954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a740c5fd0 con 0x7f6a740abd60 2024-03-30T00:49:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.300+0000 7f6a7b400640 1 -- 172.21.15.66:0/1479235954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a74066c30 con 0x7f6a740abd60 2024-03-30T00:49:02.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.304+0000 7f6a71600640 1 -- 172.21.15.66:0/1479235954 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6a6c015430 con 0x7f6a740abd60 2024-03-30T00:49:02.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.304+0000 7f6a71600640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a50066690 0x7f6a50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.304+0000 7f6a71600640 1 -- 172.21.15.66:0/1479235954 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6a6c0843d0 con 0x7f6a740abd60 2024-03-30T00:49:02.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.304+0000 7f6a78a00640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a50066690 0x7f6a50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.304+0000 7f6a78a00640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a50066690 0x7f6a50068b50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6a640af000 tx=0x7f6a640046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:02.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.308+0000 7f6a71600640 1 -- 172.21.15.66:0/1479235954 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6a6c051610 con 0x7f6a740abd60 2024-03-30T00:49:02.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.472+0000 7f6a7b400640 1 -- 172.21.15.66:0/1479235954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a7415da10 con 0x7f6a740abd60 2024-03-30T00:49:02.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.472+0000 7f6a71600640 1 -- 172.21.15.66:0/1479235954 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16248 (secure 0 0 0) 0x7f6a6c054c30 con 0x7f6a740abd60 2024-03-30T00:49:02.485 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:02.485 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":110,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:48:47.334669+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:48:44.750350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-30T00:48:45.327928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:49:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 -- 172.21.15.66:0/1479235954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a50066690 msgr2=0x7f6a50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a50066690 0x7f6a50068b50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6a640af000 tx=0x7f6a640046d0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 -- 172.21.15.66:0/1479235954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740abd60 msgr2=0x7f6a740ac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740abd60 0x7f6a740ac1e0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009f50 tx=0x7f6a6c00f580 comp rx=0 tx=0).stop 2024-03-30T00:49:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 -- 172.21.15.66:0/1479235954 shutdown_connections 2024-03-30T00:49:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a50066690 0x7f6a50068b50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a740abd60 0x7f6a740ac1e0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a74159030 0x7f6a741f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 --2- 172.21.15.66:0/1479235954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a58003980 0x7f6a740c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 -- 172.21.15.66:0/1479235954 >> 172.21.15.66:0/1479235954 conn(0x7f6a74069f60 msgr2=0x7f6a7415e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 -- 172.21.15.66:0/1479235954 shutdown_connections 2024-03-30T00:49:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.476+0000 7f6a7b400640 1 -- 172.21.15.66:0/1479235954 wait complete. 2024-03-30T00:49:02.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:48:44.750350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:48:45.327928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:49:02.507 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:49:02.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.640+0000 7f65ea000640 1 Processor -- start 2024-03-30T00:49:02.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.640+0000 7f65ea000640 1 -- start start 2024-03-30T00:49:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.640+0000 7f65ea000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f65e415de30 0x7f65e40cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.640+0000 7f65ea000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65e40af420 0x7f65e40d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.640+0000 7f65ea000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f65e40ce740 0x7f65e40cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.640+0000 7f65ea000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e40cf110 con 0x7f65e415de30 2024-03-30T00:49:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.640+0000 7f65ea000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e40cbb90 con 0x7f65e40ce740 2024-03-30T00:49:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.640+0000 7f65ea000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40ce1b0 con 0x7f65e40af420 2024-03-30T00:49:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.640+0000 7f65e3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f65e415de30 0x7f65e40cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.640+0000 7f65e3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f65e415de30 0x7f65e40cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53126/0 (socket says 172.21.15.66:53126) 2024-03-30T00:49:02.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.640+0000 7f65e3400640 1 -- 172.21.15.66:0/2141414365 learned_addr learned my addr 172.21.15.66:0/2141414365 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65e3e00640 1 --2- 172.21.15.66:0/2141414365 >> v2:172.21.15.156:3300/0 conn(0x7f65e40ce740 0x7f65e40cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65daa00640 1 --2- 172.21.15.66:0/2141414365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65e40af420 0x7f65e40d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65e3400640 1 -- 172.21.15.66:0/2141414365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65e40af420 msgr2=0x7f65e40d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65e3400640 1 --2- 172.21.15.66:0/2141414365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65e40af420 0x7f65e40d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65e3400640 1 -- 172.21.15.66:0/2141414365 >> v2:172.21.15.156:3300/0 conn(0x7f65e40ce740 msgr2=0x7f65e40cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65e3400640 1 --2- 172.21.15.66:0/2141414365 >> v2:172.21.15.156:3300/0 conn(0x7f65e40ce740 0x7f65e40cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65e3400640 1 -- 172.21.15.66:0/2141414365 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40afa10 con 0x7f65e415de30 2024-03-30T00:49:02.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65daa00640 1 --2- 172.21.15.66:0/2141414365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65e40af420 0x7f65e40d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65e3400640 1 --2- 172.21.15.66:0/2141414365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65e415de30 0x7f65e40cc040 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f65e4070b00 tx=0x7f65d401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68dfa942444566e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65e2a00640 1 -- 172.21.15.66:0/2141414365 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d40a57a0 con 0x7f65e415de30 2024-03-30T00:49:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65e2a00640 1 -- 172.21.15.66:0/2141414365 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d40a5900 con 0x7f65e415de30 2024-03-30T00:49:02.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65e2a00640 1 -- 172.21.15.66:0/2141414365 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d40a5c90 con 0x7f65e415de30 2024-03-30T00:49:02.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65ea000640 1 -- 172.21.15.66:0/2141414365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65e415de30 msgr2=0x7f65e40cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:02.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.644+0000 7f65ea000640 1 --2- 172.21.15.66:0/2141414365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65e415de30 0x7f65e40cc040 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f65e4070b00 tx=0x7f65d401c900 comp rx=0 tx=0).stop 2024-03-30T00:49:02.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 -- 172.21.15.66:0/2141414365 shutdown_connections 2024-03-30T00:49:02.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 --2- 172.21.15.66:0/2141414365 >> v2:172.21.15.156:3300/0 conn(0x7f65e40ce740 0x7f65e40cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 --2- 172.21.15.66:0/2141414365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65e40af420 0x7f65e40d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 --2- 172.21.15.66:0/2141414365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65e415de30 0x7f65e40cc040 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 -- 172.21.15.66:0/2141414365 >> 172.21.15.66:0/2141414365 conn(0x7f65e40d0540 msgr2=0x7f65e40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:02.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 -- 172.21.15.66:0/2141414365 shutdown_connections 2024-03-30T00:49:02.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 -- 172.21.15.66:0/2141414365 wait complete. 2024-03-30T00:49:02.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 Processor -- start 2024-03-30T00:49:02.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 -- start start 2024-03-30T00:49:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65e40af420 0x7f65e415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65e40ce740 0x7f65e4163b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65e415de30 0x7f65e40b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40b4870 con 0x7f65e40ce740 2024-03-30T00:49:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40b4b50 con 0x7f65e40af420 2024-03-30T00:49:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4163330 con 0x7f65e415de30 2024-03-30T00:49:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65daa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65e40ce740 0x7f65e4163b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65daa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65e40ce740 0x7f65e4163b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53142/0 (socket says 172.21.15.66:53142) 2024-03-30T00:49:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65daa00640 1 -- 172.21.15.66:0/1712581452 learned_addr learned my addr 172.21.15.66:0/1712581452 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:02.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65e3e00640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65e415de30 0x7f65e40b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65e3400640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65e40af420 0x7f65e415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65daa00640 1 -- 172.21.15.66:0/1712581452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65e415de30 msgr2=0x7f65e40b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:02.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65daa00640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65e415de30 0x7f65e40b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65daa00640 1 -- 172.21.15.66:0/1712581452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65e40af420 msgr2=0x7f65e415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:02.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65daa00640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65e40af420 0x7f65e415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65daa00640 1 -- 172.21.15.66:0/1712581452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4009b30 con 0x7f65e40ce740 2024-03-30T00:49:02.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65daa00640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65e40ce740 0x7f65e4163b30 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c7c0 tx=0x7f65cc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:02.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65dbe00640 1 -- 172.21.15.66:0/1712581452 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc007a30 con 0x7f65e40ce740 2024-03-30T00:49:02.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65dbe00640 1 -- 172.21.15.66:0/1712581452 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc007b90 con 0x7f65e40ce740 2024-03-30T00:49:02.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.648+0000 7f65ea000640 1 -- 172.21.15.66:0/1712581452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e4163610 con 0x7f65e40ce740 2024-03-30T00:49:02.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.652+0000 7f65dbe00640 1 -- 172.21.15.66:0/1712581452 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc00ce70 con 0x7f65e40ce740 2024-03-30T00:49:02.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.652+0000 7f65ea000640 1 -- 172.21.15.66:0/1712581452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e4164240 con 0x7f65e40ce740 2024-03-30T00:49:02.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.652+0000 7f65ea000640 1 -- 172.21.15.66:0/1712581452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e40cdf60 con 0x7f65e40ce740 2024-03-30T00:49:02.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.652+0000 7f65dbe00640 1 -- 172.21.15.66:0/1712581452 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f65cc0113f0 con 0x7f65e40ce740 2024-03-30T00:49:02.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.652+0000 7f65dbe00640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65c00666c0 0x7f65c0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.652+0000 7f65dbe00640 1 -- 172.21.15.66:0/1712581452 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f65cc084330 con 0x7f65e40ce740 2024-03-30T00:49:02.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.652+0000 7f65e3400640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65c00666c0 0x7f65c0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.652+0000 7f65e3400640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65c00666c0 0x7f65c0068b80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f65d40ae000 tx=0x7f65d40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:02.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.656+0000 7f65dbe00640 1 -- 172.21.15.66:0/1712581452 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f65cc051600 con 0x7f65e40ce740 2024-03-30T00:49:02.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 -- 172.21.15.66:0/1712581452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65e40b49b0 con 0x7f65e40ce740 2024-03-30T00:49:02.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65dbe00640 1 -- 172.21.15.66:0/1712581452 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65e40b49b0 con 0x7f65e40ce740 2024-03-30T00:49:02.830 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:49:02.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 -- 172.21.15.66:0/1712581452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65c00666c0 msgr2=0x7f65c0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:02.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65c00666c0 0x7f65c0068b80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f65d40ae000 tx=0x7f65d40a7070 comp rx=0 tx=0).stop 2024-03-30T00:49:02.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 -- 172.21.15.66:0/1712581452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65e40ce740 msgr2=0x7f65e4163b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:02.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65e40ce740 0x7f65e4163b30 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c7c0 tx=0x7f65cc00cc80 comp rx=0 tx=0).stop 2024-03-30T00:49:02.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 -- 172.21.15.66:0/1712581452 shutdown_connections 2024-03-30T00:49:02.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65e415de30 0x7f65e40b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65c00666c0 0x7f65c0068b80 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65e40ce740 0x7f65e4163b30 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 --2- 172.21.15.66:0/1712581452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65e40af420 0x7f65e415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 -- 172.21.15.66:0/1712581452 >> 172.21.15.66:0/1712581452 conn(0x7f65e40d0540 msgr2=0x7f65e40ad910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 -- 172.21.15.66:0/1712581452 shutdown_connections 2024-03-30T00:49:02.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.820+0000 7f65ea000640 1 -- 172.21.15.66:0/1712581452 wait complete. 2024-03-30T00:49:02.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:49:02.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.984+0000 7f96ee400640 1 Processor -- start 2024-03-30T00:49:02.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96ee400640 1 -- start start 2024-03-30T00:49:02.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96ee400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f96e80cd2d0 0x7f96e80cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96ee400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96e80cc8d0 0x7f96e806c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:02.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96ee400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e80cb620 con 0x7f96e80cad00 2024-03-30T00:49:02.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96ee400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80cbec0 con 0x7f96e80cd2d0 2024-03-30T00:49:02.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96ee400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80ca660 con 0x7f96e80cc8d0 2024-03-30T00:49:02.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96e7400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f96e80cad00 0x7f96e80c8940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56286/0 (socket says 172.21.15.66:56286) 2024-03-30T00:49:02.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96e7400640 1 -- 172.21.15.66:0/3190727692 learned_addr learned my addr 172.21.15.66:0/3190727692 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:02.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96dea00640 1 --2- 172.21.15.66:0/3190727692 >> v2:172.21.15.66:3300/0 conn(0x7f96e80cd2d0 0x7f96e80cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96e7e00640 1 --2- 172.21.15.66:0/3190727692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96e80cc8d0 0x7f96e806c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:02.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96e6a00640 1 -- 172.21.15.66:0/3190727692 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45086687 0 0) 0x7f96e80cb620 con 0x7f96e80cad00 2024-03-30T00:49:02.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96dea00640 1 -- 172.21.15.66:0/3190727692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96e80cc8d0 msgr2=0x7f96e806c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:03.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96dea00640 1 --2- 172.21.15.66:0/3190727692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96e80cc8d0 0x7f96e806c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96dea00640 1 -- 172.21.15.66:0/3190727692 >> v1:172.21.15.66:6789/0 conn(0x7f96e80cad00 legacy=0x7f96e80c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.988+0000 7f96dea00640 1 -- 172.21.15.66:0/3190727692 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80ca940 con 0x7f96e80cd2d0 2024-03-30T00:49:03.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96dea00640 1 --2- 172.21.15.66:0/3190727692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96e80cd2d0 0x7f96e80cb0e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f96d00097c0 tx=0x7f96d001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f33aee1d77091c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:03.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96e6a00640 1 -- 172.21.15.66:0/3190727692 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00ab670 con 0x7f96e80cd2d0 2024-03-30T00:49:03.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96e6a00640 1 -- 172.21.15.66:0/3190727692 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d00ab7d0 con 0x7f96e80cd2d0 2024-03-30T00:49:03.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96e6a00640 1 -- 172.21.15.66:0/3190727692 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00abb60 con 0x7f96e80cd2d0 2024-03-30T00:49:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96ee400640 1 -- 172.21.15.66:0/3190727692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96e80cd2d0 msgr2=0x7f96e80cb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96ee400640 1 --2- 172.21.15.66:0/3190727692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96e80cd2d0 0x7f96e80cb0e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f96d00097c0 tx=0x7f96d001c470 comp rx=0 tx=0).stop 2024-03-30T00:49:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96ee400640 1 -- 172.21.15.66:0/3190727692 shutdown_connections 2024-03-30T00:49:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96ee400640 1 --2- 172.21.15.66:0/3190727692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96e80cc8d0 0x7f96e806c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96ee400640 1 --2- 172.21.15.66:0/3190727692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96e80cd2d0 0x7f96e80cb0e0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96ee400640 1 -- 172.21.15.66:0/3190727692 >> 172.21.15.66:0/3190727692 conn(0x7f96e8069f60 msgr2=0x7f96e806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96ee400640 1 -- 172.21.15.66:0/3190727692 shutdown_connections 2024-03-30T00:49:03.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96ee400640 1 -- 172.21.15.66:0/3190727692 wait complete. 2024-03-30T00:49:03.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96ee400640 1 Processor -- start 2024-03-30T00:49:03.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.992+0000 7f96ee400640 1 -- start start 2024-03-30T00:49:03.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96ee400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96e80cad00 0x7f96e80c1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96ee400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96e80cc8d0 0x7f96e80abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96ee400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96e80cd2d0 0x7f96e80ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96ee400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8150b40 con 0x7f96e80cd2d0 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96ee400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80c1790 con 0x7f96e80cad00 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96ee400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8097010 con 0x7f96e80cc8d0 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96e80cd2d0 0x7f96e80ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96e80cd2d0 0x7f96e80ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53156/0 (socket says 172.21.15.66:53156) 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96e7e00640 1 -- 172.21.15.66:0/513386243 learned_addr learned my addr 172.21.15.66:0/513386243 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96dea00640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96e80cc8d0 0x7f96e80abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96e7400640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96e80cad00 0x7f96e80c1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96e7e00640 1 -- 172.21.15.66:0/513386243 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96e80cc8d0 msgr2=0x7f96e80abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96e7e00640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96e80cc8d0 0x7f96e80abc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96e7e00640 1 -- 172.21.15.66:0/513386243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96e80cad00 msgr2=0x7f96e80c1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96e7e00640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96e80cad00 0x7f96e80c1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96e7e00640 1 -- 172.21.15.66:0/513386243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d0009420 con 0x7f96e80cd2d0 2024-03-30T00:49:03.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96e7e00640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96e80cd2d0 0x7f96e80ab010 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f96e0007d80 tx=0x7f96e000d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:03.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96dfe00640 1 -- 172.21.15.66:0/513386243 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e000f040 con 0x7f96e80cd2d0 2024-03-30T00:49:03.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96dfe00640 1 -- 172.21.15.66:0/513386243 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e000dda0 con 0x7f96e80cd2d0 2024-03-30T00:49:03.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96ee400640 1 -- 172.21.15.66:0/513386243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e80972f0 con 0x7f96e80cd2d0 2024-03-30T00:49:03.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96dfe00640 1 -- 172.21.15.66:0/513386243 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0003be0 con 0x7f96e80cd2d0 2024-03-30T00:49:03.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96ee400640 1 -- 172.21.15.66:0/513386243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e8151070 con 0x7f96e80cd2d0 2024-03-30T00:49:03.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96ee400640 1 -- 172.21.15.66:0/513386243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e8097530 con 0x7f96e80cd2d0 2024-03-30T00:49:03.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:02.996+0000 7f96dfe00640 1 -- 172.21.15.66:0/513386243 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f96e0003d40 con 0x7f96e80cd2d0 2024-03-30T00:49:03.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.000+0000 7f96dfe00640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96c4066690 0x7f96c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.000+0000 7f96dfe00640 1 -- 172.21.15.66:0/513386243 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f96e00842b0 con 0x7f96e80cd2d0 2024-03-30T00:49:03.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.000+0000 7f96e7400640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96c4066690 0x7f96c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.000+0000 7f96e7400640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96c4066690 0x7f96c4068b50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f96d8007fd0 tx=0x7f96d8007d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:03.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.004+0000 7f96dfe00640 1 -- 172.21.15.66:0/513386243 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f96e8097530 con 0x7f96e80cd2d0 2024-03-30T00:49:03.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.164+0000 7f96ee400640 1 -- 172.21.15.66:0/513386243 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96e8097530 con 0x7f96e80cd2d0 2024-03-30T00:49:03.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.164+0000 7f96dfe00640 1 -- 172.21.15.66:0/513386243 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96e8097530 con 0x7f96e80cd2d0 2024-03-30T00:49:03.175 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:49:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 -- 172.21.15.66:0/513386243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96c4066690 msgr2=0x7f96c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96c4066690 0x7f96c4068b50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f96d8007fd0 tx=0x7f96d8007d60 comp rx=0 tx=0).stop 2024-03-30T00:49:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 -- 172.21.15.66:0/513386243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96e80cd2d0 msgr2=0x7f96e80ab010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96e80cd2d0 0x7f96e80ab010 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f96e0007d80 tx=0x7f96e000d490 comp rx=0 tx=0).stop 2024-03-30T00:49:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 -- 172.21.15.66:0/513386243 shutdown_connections 2024-03-30T00:49:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96c4066690 0x7f96c4068b50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96e80cd2d0 0x7f96e80ab010 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96e80cc8d0 0x7f96e80abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 --2- 172.21.15.66:0/513386243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96e80cad00 0x7f96e80c1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 -- 172.21.15.66:0/513386243 >> 172.21.15.66:0/513386243 conn(0x7f96e8069f60 msgr2=0x7f96e80adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:03.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 -- 172.21.15.66:0/513386243 shutdown_connections 2024-03-30T00:49:03.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.168+0000 7f96ee400640 1 -- 172.21.15.66:0/513386243 wait complete. 2024-03-30T00:49:03.196 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:49:03.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.328+0000 7f5047400640 1 Processor -- start 2024-03-30T00:49:03.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.328+0000 7f5047400640 1 -- start start 2024-03-30T00:49:03.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.328+0000 7f5047400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504015a640 0x7f504006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.328+0000 7f5047400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504006d070 con 0x7f50400cabb0 2024-03-30T00:49:03.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.328+0000 7f5047400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50400cbba0 con 0x7f50400ce520 2024-03-30T00:49:03.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.328+0000 7f5047400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504006e2c0 con 0x7f504015a640 2024-03-30T00:49:03.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.328+0000 7f5045400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f50400cabb0 0x7f50400cb040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56290/0 (socket says 172.21.15.66:56290) 2024-03-30T00:49:03.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.328+0000 7f5045400640 1 -- 172.21.15.66:0/474763968 learned_addr learned my addr 172.21.15.66:0/474763968 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:03.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.328+0000 7f5044a00640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504015a640 0x7f504006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861370660 0 0) 0x7f504006d070 con 0x7f50400cabb0 2024-03-30T00:49:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5024003530 con 0x7f50400cabb0 2024-03-30T00:49:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1300650275 0 0) 0x7f5024003530 con 0x7f50400cabb0 2024-03-30T00:49:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504015a640 msgr2=0x7f504006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504015a640 0x7f504006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 >> v1:172.21.15.156:6789/0 conn(0x7f50400ce520 legacy=0x7f50400ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400cb8c0 con 0x7f50400cabb0 2024-03-30T00:49:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f5038001a00 con 0x7f50400cabb0 2024-03-30T00:49:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 >> v1:172.21.15.66:6789/0 conn(0x7f50400cabb0 legacy=0x7f50400cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50240039b0 0x7f5024003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5024004310 0x7f50240047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 0x7f502400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504006e2c0 con 0x7f5024004ff0 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502400bf00 con 0x7f5024004310 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502400c0c0 con 0x7f50240039b0 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f5044a00640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504015a640 0x7f504006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f5045400640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 0x7f502400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f5037e00640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5024004310 0x7f50240047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f5044a00640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50240039b0 0x7f5024003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f5045400640 1 -- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50240039b0 msgr2=0x7f5024003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f5045400640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50240039b0 0x7f5024003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f5045400640 1 -- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5024004310 msgr2=0x7f50240047f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f5045400640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5024004310 0x7f50240047f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.332+0000 7f5045400640 1 -- 172.21.15.66:0/474763968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5024003530 con 0x7f5024004ff0 2024-03-30T00:49:03.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5045400640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 0x7f502400b990 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f503800a830 tx=0x7f503801fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155fc6d5306f11cb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:03.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50380aa6f0 con 0x7f5024004ff0 2024-03-30T00:49:03.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50380aa870 con 0x7f5024004ff0 2024-03-30T00:49:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f503fe00640 1 -- 172.21.15.66:0/474763968 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50380aac60 con 0x7f5024004ff0 2024-03-30T00:49:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 -- 172.21.15.66:0/474763968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 msgr2=0x7f502400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 --2- 172.21.15.66:0/474763968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 0x7f502400b990 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f503800a830 tx=0x7f503801fac0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5045400640 1 -- 172.21.15.66:0/474763968 reap_dead start 2024-03-30T00:49:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 -- 172.21.15.66:0/474763968 shutdown_connections 2024-03-30T00:49:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 -- 172.21.15.66:0/474763968 >> 172.21.15.66:0/474763968 conn(0x7f5040069f60 msgr2=0x7f504006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 -- 172.21.15.66:0/474763968 shutdown_connections 2024-03-30T00:49:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 -- 172.21.15.66:0/474763968 wait complete. 2024-03-30T00:49:03.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 Processor -- start 2024-03-30T00:49:03.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 -- start start 2024-03-30T00:49:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 0x7f5040156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50400c53c0 0x7f50400b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50400cda50 0x7f50400b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400c1f20 con 0x7f5024004ff0 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400c5850 con 0x7f50400c53c0 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.336+0000 7f5047400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50400c5b30 con 0x7f50400cda50 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5044a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 0x7f5040156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5044a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 0x7f5040156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53180/0 (socket says 172.21.15.66:53180) 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5044a00640 1 -- 172.21.15.66:0/1065198714 learned_addr learned my addr 172.21.15.66:0/1065198714 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5044a00640 1 -- 172.21.15.66:0/1065198714 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50400cda50 msgr2=0x7f50400b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5044a00640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50400cda50 0x7f50400b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5044a00640 1 -- 172.21.15.66:0/1065198714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50400c53c0 msgr2=0x7f50400b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5044a00640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50400c53c0 0x7f50400b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5044a00640 1 -- 172.21.15.66:0/1065198714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400c5e10 con 0x7f5024004ff0 2024-03-30T00:49:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5044a00640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 0x7f5040156500 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f50300119a0 tx=0x7f5030011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f503d600640 1 -- 172.21.15.66:0/1065198714 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503000fb30 con 0x7f5024004ff0 2024-03-30T00:49:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f503d600640 1 -- 172.21.15.66:0/1065198714 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503000fc90 con 0x7f5024004ff0 2024-03-30T00:49:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f503d600640 1 -- 172.21.15.66:0/1065198714 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030014440 con 0x7f5024004ff0 2024-03-30T00:49:03.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5047400640 1 -- 172.21.15.66:0/1065198714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50400c60f0 con 0x7f5024004ff0 2024-03-30T00:49:03.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5047400640 1 -- 172.21.15.66:0/1065198714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50400cc8f0 con 0x7f5024004ff0 2024-03-30T00:49:03.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f5047400640 1 -- 172.21.15.66:0/1065198714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5040066c30 con 0x7f5024004ff0 2024-03-30T00:49:03.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f503d600640 1 -- 172.21.15.66:0/1065198714 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f50300145a0 con 0x7f5024004ff0 2024-03-30T00:49:03.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f503d600640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f501c0666c0 0x7f501c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.340+0000 7f503d600640 1 -- 172.21.15.66:0/1065198714 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5030084c80 con 0x7f5024004ff0 2024-03-30T00:49:03.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.344+0000 7f5037e00640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f501c0666c0 0x7f501c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.344+0000 7f5037e00640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f501c0666c0 0x7f501c068b80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f50400cc650 tx=0x7f5028009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:03.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.344+0000 7f503d600640 1 -- 172.21.15.66:0/1065198714 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5030051e10 con 0x7f5024004ff0 2024-03-30T00:49:03.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.504+0000 7f5047400640 1 -- 172.21.15.66:0/1065198714 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f50400ac3e0 con 0x7f5024004ff0 2024-03-30T00:49:03.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.504+0000 7f503d600640 1 -- 172.21.15.66:0/1065198714 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5030022020 con 0x7f5024004ff0 2024-03-30T00:49:03.516 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:49:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 -- 172.21.15.66:0/1065198714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f501c0666c0 msgr2=0x7f501c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f501c0666c0 0x7f501c068b80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f50400cc650 tx=0x7f5028009040 comp rx=0 tx=0).stop 2024-03-30T00:49:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 -- 172.21.15.66:0/1065198714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 msgr2=0x7f5040156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 0x7f5040156500 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f50300119a0 tx=0x7f5030011e60 comp rx=0 tx=0).stop 2024-03-30T00:49:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 -- 172.21.15.66:0/1065198714 shutdown_connections 2024-03-30T00:49:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50400cda50 0x7f50400b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50400c53c0 0x7f50400b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f501c0666c0 0x7f501c068b80 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 --2- 172.21.15.66:0/1065198714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5024004ff0 0x7f5040156500 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 -- 172.21.15.66:0/1065198714 >> 172.21.15.66:0/1065198714 conn(0x7f5040069f60 msgr2=0x7f504015cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:03.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 -- 172.21.15.66:0/1065198714 shutdown_connections 2024-03-30T00:49:03.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.508+0000 7f5047400640 1 -- 172.21.15.66:0/1065198714 wait complete. 2024-03-30T00:49:03.537 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:49:03.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.668+0000 7f6644200640 1 Processor -- start 2024-03-30T00:49:03.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.668+0000 7f6644200640 1 -- start start 2024-03-30T00:49:03.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f6644200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f663c0cf2f0 0x7f663c06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f6644200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663c0cb0e0 con 0x7f663c15ca80 2024-03-30T00:49:03.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f6644200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663c0cf7f0 con 0x7f663c0cb570 2024-03-30T00:49:03.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f6644200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c0cfad0 con 0x7f663c0cf2f0 2024-03-30T00:49:03.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f6641800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f663c15ca80 0x7f663c06b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56302/0 (socket says 172.21.15.66:56302) 2024-03-30T00:49:03.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f6641800640 1 -- 172.21.15.66:0/933093512 learned_addr learned my addr 172.21.15.66:0/933093512 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:03.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f6640e00640 1 --2- 172.21.15.66:0/933093512 >> v2:172.21.15.66:3300/0 conn(0x7f663c0cf2f0 0x7f663c06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f663be00640 1 -- 172.21.15.66:0/933093512 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168959460 0 0) 0x7f663c0cb0e0 con 0x7f663c15ca80 2024-03-30T00:49:03.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f6640e00640 1 -- 172.21.15.66:0/933093512 >> v1:172.21.15.156:6789/0 conn(0x7f663c0cb570 legacy=0x7f663c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f6640e00640 1 -- 172.21.15.66:0/933093512 >> v1:172.21.15.66:6789/0 conn(0x7f663c15ca80 legacy=0x7f663c06b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f6640e00640 1 -- 172.21.15.66:0/933093512 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0cad70 con 0x7f663c0cf2f0 2024-03-30T00:49:03.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.672+0000 7f6640e00640 1 --2- 172.21.15.66:0/933093512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f663c0cf2f0 0x7f663c06b7c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f66280097c0 tx=0x7f662801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf214099e3af62d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:03.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f663be00640 1 -- 172.21.15.66:0/933093512 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66280a8990 con 0x7f663c0cf2f0 2024-03-30T00:49:03.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f663be00640 1 -- 172.21.15.66:0/933093512 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66280a8af0 con 0x7f663c0cf2f0 2024-03-30T00:49:03.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f663be00640 1 -- 172.21.15.66:0/933093512 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66280a8e80 con 0x7f663c0cf2f0 2024-03-30T00:49:03.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 -- 172.21.15.66:0/933093512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f663c0cf2f0 msgr2=0x7f663c06b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 --2- 172.21.15.66:0/933093512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f663c0cf2f0 0x7f663c06b7c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f66280097c0 tx=0x7f662801ec90 comp rx=0 tx=0).stop 2024-03-30T00:49:03.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 -- 172.21.15.66:0/933093512 shutdown_connections 2024-03-30T00:49:03.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 --2- 172.21.15.66:0/933093512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f663c0cf2f0 0x7f663c06b7c0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 -- 172.21.15.66:0/933093512 >> 172.21.15.66:0/933093512 conn(0x7f663c0b2090 msgr2=0x7f663c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:03.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 -- 172.21.15.66:0/933093512 shutdown_connections 2024-03-30T00:49:03.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 -- 172.21.15.66:0/933093512 wait complete. 2024-03-30T00:49:03.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 Processor -- start 2024-03-30T00:49:03.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 -- start start 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f663c0cb570 0x7f663c158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f663c0cf2f0 0x7f663c0c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f663c15ca80 0x7f663c0c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0c97a0 con 0x7f663c0cb570 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0c8f00 con 0x7f663c0cf2f0 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6644200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0c91e0 con 0x7f663c15ca80 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6641800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f663c0cb570 0x7f663c158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6641800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f663c0cb570 0x7f663c158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53198/0 (socket says 172.21.15.66:53198) 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6641800640 1 -- 172.21.15.66:0/2184527275 learned_addr learned my addr 172.21.15.66:0/2184527275 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6640e00640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f663c0cf2f0 0x7f663c0c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6642200640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f663c15ca80 0x7f663c0c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6641800640 1 -- 172.21.15.66:0/2184527275 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f663c15ca80 msgr2=0x7f663c0c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6641800640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f663c15ca80 0x7f663c0c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6641800640 1 -- 172.21.15.66:0/2184527275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f663c0cf2f0 msgr2=0x7f663c0c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6641800640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f663c0cf2f0 0x7f663c0c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.676+0000 7f6641800640 1 -- 172.21.15.66:0/2184527275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628009420 con 0x7f663c0cb570 2024-03-30T00:49:03.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.680+0000 7f6641800640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f663c0cb570 0x7f663c158a20 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f663000c410 tx=0x7f663000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:03.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.680+0000 7f6639600640 1 -- 172.21.15.66:0/2184527275 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66300074e0 con 0x7f663c0cb570 2024-03-30T00:49:03.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.680+0000 7f6639600640 1 -- 172.21.15.66:0/2184527275 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6630007640 con 0x7f663c0cb570 2024-03-30T00:49:03.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.680+0000 7f6639600640 1 -- 172.21.15.66:0/2184527275 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630004a30 con 0x7f663c0cb570 2024-03-30T00:49:03.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.680+0000 7f6644200640 1 -- 172.21.15.66:0/2184527275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c0c94c0 con 0x7f663c0cb570 2024-03-30T00:49:03.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.680+0000 7f6644200640 1 -- 172.21.15.66:0/2184527275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c158f60 con 0x7f663c0cb570 2024-03-30T00:49:03.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.680+0000 7f6644200640 1 -- 172.21.15.66:0/2184527275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c066c30 con 0x7f663c0cb570 2024-03-30T00:49:03.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.680+0000 7f6639600640 1 -- 172.21.15.66:0/2184527275 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6630004b90 con 0x7f663c0cb570 2024-03-30T00:49:03.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.680+0000 7f6639600640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f661c066690 0x7f661c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:03.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.680+0000 7f6639600640 1 -- 172.21.15.66:0/2184527275 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6630084090 con 0x7f663c0cb570 2024-03-30T00:49:03.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.684+0000 7f6640e00640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f661c066690 0x7f661c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:03.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.684+0000 7f6640e00640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f661c066690 0x7f661c068b50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f66280b2000 tx=0x7f6628008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:03.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.684+0000 7f6639600640 1 -- 172.21.15.66:0/2184527275 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f66300512c0 con 0x7f663c0cb570 2024-03-30T00:49:03.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6644200640 1 -- 172.21.15.66:0/2184527275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f663c1f21d0 con 0x7f663c0cb570 2024-03-30T00:49:03.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6639600640 1 -- 172.21.15.66:0/2184527275 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7f66300548e0 con 0x7f663c0cb570 2024-03-30T00:49:03.871 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:49:03.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6644200640 1 -- 172.21.15.66:0/2184527275 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f661c066690 msgr2=0x7f661c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6644200640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f661c066690 0x7f661c068b50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f66280b2000 tx=0x7f6628008040 comp rx=0 tx=0).stop 2024-03-30T00:49:03.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6644200640 1 -- 172.21.15.66:0/2184527275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f663c0cb570 msgr2=0x7f663c158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:03.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6644200640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f663c0cb570 0x7f663c158a20 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f663000c410 tx=0x7f663000c8d0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6644200640 1 -- 172.21.15.66:0/2184527275 shutdown_connections 2024-03-30T00:49:03.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6644200640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f663c15ca80 0x7f663c0c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6644200640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f663c0cf2f0 0x7f663c0c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6644200640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f661c066690 0x7f661c068b50 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6644200640 1 --2- 172.21.15.66:0/2184527275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f663c0cb570 0x7f663c158a20 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:03.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.860+0000 7f6644200640 1 -- 172.21.15.66:0/2184527275 >> 172.21.15.66:0/2184527275 conn(0x7f663c0b2090 msgr2=0x7f663c0cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:03.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.864+0000 7f6644200640 1 -- 172.21.15.66:0/2184527275 shutdown_connections 2024-03-30T00:49:03.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:03.864+0000 7f6644200640 1 -- 172.21.15.66:0/2184527275 wait complete. 2024-03-30T00:49:03.892 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:49:03.893 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:49:03.902 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:49:04.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.072+0000 7fea1b800640 1 Processor -- start 2024-03-30T00:49:04.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea1b800640 1 -- start start 2024-03-30T00:49:04.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea1b800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea140cda90 0x7fea140ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea1b800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fea1415a370 0x7fea140cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea1b800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1415a990 con 0x7fea140cde70 2024-03-30T00:49:04.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea1b800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea140b1b80 con 0x7fea1415a370 2024-03-30T00:49:04.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea1b800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea140cd3e0 con 0x7fea140cda90 2024-03-30T00:49:04.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea0be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fea140cde70 0x7fea14159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56304/0 (socket says 172.21.15.66:56304) 2024-03-30T00:49:04.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea0be00640 1 -- 172.21.15.66:0/877128024 learned_addr learned my addr 172.21.15.66:0/877128024 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:04.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea18e00640 1 --2- 172.21.15.66:0/877128024 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea140cda90 0x7fea140ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea19800640 1 --2- 172.21.15.66:0/877128024 >> v2:172.21.15.156:3300/0 conn(0x7fea1415a370 0x7fea140cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea13e00640 1 -- 172.21.15.66:0/877128024 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431251766 0 0) 0x7fea1415a990 con 0x7fea140cde70 2024-03-30T00:49:04.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea13e00640 1 -- 172.21.15.66:0/877128024 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9f8003530 con 0x7fea140cde70 2024-03-30T00:49:04.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea18e00640 1 -- 172.21.15.66:0/877128024 >> v2:172.21.15.156:3300/0 conn(0x7fea1415a370 msgr2=0x7fea140cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:04.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea18e00640 1 --2- 172.21.15.66:0/877128024 >> v2:172.21.15.156:3300/0 conn(0x7fea1415a370 0x7fea140cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea18e00640 1 -- 172.21.15.66:0/877128024 >> v1:172.21.15.66:6789/0 conn(0x7fea140cde70 legacy=0x7fea14159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.076+0000 7fea18e00640 1 -- 172.21.15.66:0/877128024 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea140b11e0 con 0x7fea140cda90 2024-03-30T00:49:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea18e00640 1 --2- 172.21.15.66:0/877128024 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea140cda90 0x7fea140ce520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fea040097c0 tx=0x7fea0401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57be5f9f47f7a335 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea13e00640 1 -- 172.21.15.66:0/877128024 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea040ab600 con 0x7fea140cda90 2024-03-30T00:49:04.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea13e00640 1 -- 172.21.15.66:0/877128024 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea040a7050 con 0x7fea140cda90 2024-03-30T00:49:04.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea13e00640 1 -- 172.21.15.66:0/877128024 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea040ab980 con 0x7fea140cda90 2024-03-30T00:49:04.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea1b800640 1 -- 172.21.15.66:0/877128024 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea140cda90 msgr2=0x7fea140ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea1b800640 1 --2- 172.21.15.66:0/877128024 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea140cda90 0x7fea140ce520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fea040097c0 tx=0x7fea0401c410 comp rx=0 tx=0).stop 2024-03-30T00:49:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea1b800640 1 -- 172.21.15.66:0/877128024 shutdown_connections 2024-03-30T00:49:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea1b800640 1 --2- 172.21.15.66:0/877128024 >> v2:172.21.15.156:3300/0 conn(0x7fea1415a370 0x7fea140cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea1b800640 1 --2- 172.21.15.66:0/877128024 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea140cda90 0x7fea140ce520 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea1b800640 1 -- 172.21.15.66:0/877128024 >> 172.21.15.66:0/877128024 conn(0x7fea14069f60 msgr2=0x7fea1406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea1b800640 1 -- 172.21.15.66:0/877128024 shutdown_connections 2024-03-30T00:49:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea1b800640 1 -- 172.21.15.66:0/877128024 wait complete. 2024-03-30T00:49:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.080+0000 7fea1b800640 1 Processor -- start 2024-03-30T00:49:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea1b800640 1 -- start start 2024-03-30T00:49:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea1b800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea140cda90 0x7fea141523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea1b800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea140cde70 0x7fea14154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea1b800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea1415a370 0x7fea141f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea1b800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1415bbd0 con 0x7fea140cde70 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea1b800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea14071da0 con 0x7fea140cda90 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea1b800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea14071b00 con 0x7fea1415a370 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea0be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea140cde70 0x7fea14154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea0be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea140cde70 0x7fea14154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53200/0 (socket says 172.21.15.66:53200) 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea0be00640 1 -- 172.21.15.66:0/3060609842 learned_addr learned my addr 172.21.15.66:0/3060609842 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea18e00640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea140cda90 0x7fea141523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea19800640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea1415a370 0x7fea141f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea0be00640 1 -- 172.21.15.66:0/3060609842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea1415a370 msgr2=0x7fea141f8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea0be00640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea1415a370 0x7fea141f8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea0be00640 1 -- 172.21.15.66:0/3060609842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea140cda90 msgr2=0x7fea141523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea0be00640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea140cda90 0x7fea141523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea0be00640 1 -- 172.21.15.66:0/3060609842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04009420 con 0x7fea140cde70 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea0be00640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea140cde70 0x7fea14154ad0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00c4d0 tx=0x7fe9fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:04.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea11600640 1 -- 172.21.15.66:0/3060609842 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc005cf0 con 0x7fea140cde70 2024-03-30T00:49:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea11600640 1 -- 172.21.15.66:0/3060609842 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc005e50 con 0x7fea140cde70 2024-03-30T00:49:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea11600640 1 -- 172.21.15.66:0/3060609842 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc007270 con 0x7fea140cde70 2024-03-30T00:49:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea1b800640 1 -- 172.21.15.66:0/3060609842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1415b930 con 0x7fea140cde70 2024-03-30T00:49:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea1b800640 1 -- 172.21.15.66:0/3060609842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea14162fe0 con 0x7fea140cde70 2024-03-30T00:49:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.084+0000 7fea1b800640 1 -- 172.21.15.66:0/3060609842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea14164d00 con 0x7fea140cde70 2024-03-30T00:49:04.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.088+0000 7fea11600640 1 -- 172.21.15.66:0/3060609842 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe9fc0073d0 con 0x7fea140cde70 2024-03-30T00:49:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.088+0000 7fea11600640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9f00666c0 0x7fe9f0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.088+0000 7fea11600640 1 -- 172.21.15.66:0/3060609842 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe9fc083870 con 0x7fea140cde70 2024-03-30T00:49:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.088+0000 7fea18e00640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9f00666c0 0x7fe9f0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.088+0000 7fea18e00640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9f00666c0 0x7fe9f0068b80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fea040ad000 tx=0x7fea04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:04.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.092+0000 7fea11600640 1 -- 172.21.15.66:0/3060609842 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fea14164d00 con 0x7fea140cde70 2024-03-30T00:49:04.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.252+0000 7fea1b800640 1 -- 172.21.15.66:0/3060609842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea14164d00 con 0x7fea140cde70 2024-03-30T00:49:04.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea11600640 1 -- 172.21.15.66:0/3060609842 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16248 (secure 0 0 0) 0x7fea14164d00 con 0x7fea140cde70 2024-03-30T00:49:04.266 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:04.266 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":110,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:48:47.334669+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:48:44.750350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-30T00:48:45.327928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:49:04.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 -- 172.21.15.66:0/3060609842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9f00666c0 msgr2=0x7fe9f0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9f00666c0 0x7fe9f0068b80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fea040ad000 tx=0x7fea04008040 comp rx=0 tx=0).stop 2024-03-30T00:49:04.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 -- 172.21.15.66:0/3060609842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea140cde70 msgr2=0x7fea14154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea140cde70 0x7fea14154ad0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00c4d0 tx=0x7fe9fc00c990 comp rx=0 tx=0).stop 2024-03-30T00:49:04.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 -- 172.21.15.66:0/3060609842 shutdown_connections 2024-03-30T00:49:04.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea1415a370 0x7fea141f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe9f00666c0 0x7fe9f0068b80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea140cde70 0x7fea14154ad0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 --2- 172.21.15.66:0/3060609842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea140cda90 0x7fea141523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 -- 172.21.15.66:0/3060609842 >> 172.21.15.66:0/3060609842 conn(0x7fea14069f60 msgr2=0x7fea1415c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:04.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 -- 172.21.15.66:0/3060609842 shutdown_connections 2024-03-30T00:49:04.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.256+0000 7fea1b800640 1 -- 172.21.15.66:0/3060609842 wait complete. 2024-03-30T00:49:04.287 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:48:44.750350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:48:45.327928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:49:04.289 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:49:04.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.420+0000 7f3a63e00640 1 Processor -- start 2024-03-30T00:49:04.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.420+0000 7f3a63e00640 1 -- start start 2024-03-30T00:49:04.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a63e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3a6415a720 0x7f3a640ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a63e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3a640b3170 0x7f3a640b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a63e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a640cb5b0 con 0x7f3a640d07c0 2024-03-30T00:49:04.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a63e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a640cfb00 con 0x7f3a6415a720 2024-03-30T00:49:04.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a63e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6415ab00 con 0x7f3a640b3170 2024-03-30T00:49:04.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a62a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3a6415a720 0x7f3a640ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a63400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f3a640d07c0 0x7f3a640ccce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56320/0 (socket says 172.21.15.66:56320) 2024-03-30T00:49:04.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a63400640 1 -- 172.21.15.66:0/867955795 learned_addr learned my addr 172.21.15.66:0/867955795 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:04.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a5a000640 1 --2- 172.21.15.66:0/867955795 >> v2:172.21.15.156:3300/0 conn(0x7f3a640b3170 0x7f3a640b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a62a00640 1 -- 172.21.15.66:0/867955795 >> v2:172.21.15.156:3300/0 conn(0x7f3a640b3170 msgr2=0x7f3a640b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:04.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a62a00640 1 --2- 172.21.15.66:0/867955795 >> v2:172.21.15.156:3300/0 conn(0x7f3a640b3170 0x7f3a640b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a62a00640 1 -- 172.21.15.66:0/867955795 >> v1:172.21.15.66:6789/0 conn(0x7f3a640d07c0 legacy=0x7f3a640ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a62a00640 1 -- 172.21.15.66:0/867955795 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a640cb2f0 con 0x7f3a6415a720 2024-03-30T00:49:04.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a62000640 1 -- 172.21.15.66:0/867955795 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266032448 0 0) 0x7f3a5c001120 con 0x7f3a640d07c0 2024-03-30T00:49:04.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.424+0000 7f3a62a00640 1 --2- 172.21.15.66:0/867955795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a6415a720 0x7f3a640ca970 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f3a640cc740 tx=0x7f3a5401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4931c615a0c19464 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:04.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a62000640 1 -- 172.21.15.66:0/867955795 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a540a59c0 con 0x7f3a6415a720 2024-03-30T00:49:04.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a62000640 1 -- 172.21.15.66:0/867955795 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a540a5b20 con 0x7f3a6415a720 2024-03-30T00:49:04.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a62000640 1 -- 172.21.15.66:0/867955795 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a540ad440 con 0x7f3a6415a720 2024-03-30T00:49:04.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 -- 172.21.15.66:0/867955795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a6415a720 msgr2=0x7f3a640ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 --2- 172.21.15.66:0/867955795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a6415a720 0x7f3a640ca970 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f3a640cc740 tx=0x7f3a5401c900 comp rx=0 tx=0).stop 2024-03-30T00:49:04.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 -- 172.21.15.66:0/867955795 shutdown_connections 2024-03-30T00:49:04.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 --2- 172.21.15.66:0/867955795 >> v2:172.21.15.156:3300/0 conn(0x7f3a640b3170 0x7f3a640b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 --2- 172.21.15.66:0/867955795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a6415a720 0x7f3a640ca970 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 -- 172.21.15.66:0/867955795 >> 172.21.15.66:0/867955795 conn(0x7f3a64069d90 msgr2=0x7f3a6406a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:04.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 -- 172.21.15.66:0/867955795 shutdown_connections 2024-03-30T00:49:04.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 -- 172.21.15.66:0/867955795 wait complete. 2024-03-30T00:49:04.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 Processor -- start 2024-03-30T00:49:04.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 -- start start 2024-03-30T00:49:04.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a640b3170 0x7f3a640b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a640d07c0 0x7f3a64163740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6415a720 0x7f3a64163c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640b3e50 con 0x7f3a640d07c0 2024-03-30T00:49:04.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640b4130 con 0x7f3a6415a720 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640b4410 con 0x7f3a640b3170 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a5a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a640d07c0 0x7f3a64163740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a5a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a640d07c0 0x7f3a64163740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53222/0 (socket says 172.21.15.66:53222) 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a5a000640 1 -- 172.21.15.66:0/4005497765 learned_addr learned my addr 172.21.15.66:0/4005497765 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a62a00640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a640b3170 0x7f3a640b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.428+0000 7f3a63400640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6415a720 0x7f3a64163c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5a000640 1 -- 172.21.15.66:0/4005497765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a640b3170 msgr2=0x7f3a640b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5a000640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a640b3170 0x7f3a640b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5a000640 1 -- 172.21.15.66:0/4005497765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6415a720 msgr2=0x7f3a64163c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5a000640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6415a720 0x7f3a64163c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5a000640 1 -- 172.21.15.66:0/4005497765 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a54009a70 con 0x7f3a640d07c0 2024-03-30T00:49:04.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5a000640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a640d07c0 0x7f3a64163740 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00d5d0 tx=0x7f3a4c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:04.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5b400640 1 -- 172.21.15.66:0/4005497765 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c004240 con 0x7f3a640d07c0 2024-03-30T00:49:04.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5b400640 1 -- 172.21.15.66:0/4005497765 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4c0043a0 con 0x7f3a640d07c0 2024-03-30T00:49:04.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a63e00640 1 -- 172.21.15.66:0/4005497765 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a640b46f0 con 0x7f3a640d07c0 2024-03-30T00:49:04.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5b400640 1 -- 172.21.15.66:0/4005497765 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c003840 con 0x7f3a640d07c0 2024-03-30T00:49:04.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a63e00640 1 -- 172.21.15.66:0/4005497765 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a640b3ae0 con 0x7f3a640d07c0 2024-03-30T00:49:04.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a63e00640 1 -- 172.21.15.66:0/4005497765 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a64073e00 con 0x7f3a640d07c0 2024-03-30T00:49:04.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5b400640 1 -- 172.21.15.66:0/4005497765 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3a4c0039a0 con 0x7f3a640d07c0 2024-03-30T00:49:04.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5b400640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a40066690 0x7f3a40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a5b400640 1 -- 172.21.15.66:0/4005497765 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3a4c084390 con 0x7f3a640d07c0 2024-03-30T00:49:04.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.432+0000 7f3a62a00640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a40066690 0x7f3a40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.436+0000 7f3a62a00640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a40066690 0x7f3a40068b50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3a540af000 tx=0x7f3a540a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:04.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.436+0000 7f3a5b400640 1 -- 172.21.15.66:0/4005497765 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3a4c0515d0 con 0x7f3a640d07c0 2024-03-30T00:49:04.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.596+0000 7f3a63e00640 1 -- 172.21.15.66:0/4005497765 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a640af750 con 0x7f3a640d07c0 2024-03-30T00:49:04.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.596+0000 7f3a5b400640 1 -- 172.21.15.66:0/4005497765 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a4c054bf0 con 0x7f3a640d07c0 2024-03-30T00:49:04.609 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:49:04.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 -- 172.21.15.66:0/4005497765 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a40066690 msgr2=0x7f3a40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a40066690 0x7f3a40068b50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3a540af000 tx=0x7f3a540a7070 comp rx=0 tx=0).stop 2024-03-30T00:49:04.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 -- 172.21.15.66:0/4005497765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a640d07c0 msgr2=0x7f3a64163740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a640d07c0 0x7f3a64163740 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00d5d0 tx=0x7f3a4c00da90 comp rx=0 tx=0).stop 2024-03-30T00:49:04.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 -- 172.21.15.66:0/4005497765 shutdown_connections 2024-03-30T00:49:04.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6415a720 0x7f3a64163c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3a40066690 0x7f3a40068b50 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a640d07c0 0x7f3a64163740 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 --2- 172.21.15.66:0/4005497765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a640b3170 0x7f3a640b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 -- 172.21.15.66:0/4005497765 >> 172.21.15.66:0/4005497765 conn(0x7f3a64069d90 msgr2=0x7f3a6406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:04.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 -- 172.21.15.66:0/4005497765 shutdown_connections 2024-03-30T00:49:04.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.600+0000 7f3a63e00640 1 -- 172.21.15.66:0/4005497765 wait complete. 2024-03-30T00:49:04.631 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:49:04.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b1d400640 1 Processor -- start 2024-03-30T00:49:04.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b1d400640 1 -- start start 2024-03-30T00:49:04.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b1d400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2b180ae4e0 0x7f2b180cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b1d400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b180ce820 0x7f2b180cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b1d400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1806e7d0 con 0x7f2b1815ca10 2024-03-30T00:49:04.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b1d400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b180b1f70 con 0x7f2b180ae4e0 2024-03-30T00:49:04.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b1d400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b180cf1f0 con 0x7f2b180ce820 2024-03-30T00:49:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b15e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2b180ae4e0 0x7f2b180cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b17200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b180ce820 0x7f2b180cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b15e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2b180ae4e0 0x7f2b180cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:56474/0 (socket says 172.21.15.66:56474) 2024-03-30T00:49:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b16800640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2b1815ca10 0x7f2b1806ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:50128/0 (socket says 172.21.15.66:50128) 2024-03-30T00:49:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b15e00640 1 -- 172.21.15.66:0/1414052285 learned_addr learned my addr 172.21.15.66:0/1414052285 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b15e00640 1 -- 172.21.15.66:0/1414052285 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b180ce820 msgr2=0x7f2b180cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b15e00640 1 --2- 172.21.15.66:0/1414052285 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b180ce820 0x7f2b180cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b15e00640 1 -- 172.21.15.66:0/1414052285 >> v1:172.21.15.156:6789/0 conn(0x7f2b1815ca10 legacy=0x7f2b1806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.764+0000 7f2b15e00640 1 -- 172.21.15.66:0/1414052285 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1815cdf0 con 0x7f2b180ae4e0 2024-03-30T00:49:04.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b15e00640 1 --2- 172.21.15.66:0/1414052285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b180ae4e0 0x7f2b180cb620 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2b040098a0 tx=0x7f2b0401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c5bac1c1b94c1ab server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:04.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b15400640 1 -- 172.21.15.66:0/1414052285 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b040aa670 con 0x7f2b180ae4e0 2024-03-30T00:49:04.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b15400640 1 -- 172.21.15.66:0/1414052285 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b040aa7d0 con 0x7f2b180ae4e0 2024-03-30T00:49:04.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b15400640 1 -- 172.21.15.66:0/1414052285 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b040aab60 con 0x7f2b180ae4e0 2024-03-30T00:49:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b1d400640 1 -- 172.21.15.66:0/1414052285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b180ae4e0 msgr2=0x7f2b180cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b1d400640 1 --2- 172.21.15.66:0/1414052285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b180ae4e0 0x7f2b180cb620 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2b040098a0 tx=0x7f2b0401c510 comp rx=0 tx=0).stop 2024-03-30T00:49:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b1d400640 1 -- 172.21.15.66:0/1414052285 shutdown_connections 2024-03-30T00:49:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b1d400640 1 --2- 172.21.15.66:0/1414052285 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b180ce820 0x7f2b180cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b1d400640 1 --2- 172.21.15.66:0/1414052285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b180ae4e0 0x7f2b180cb620 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b1d400640 1 -- 172.21.15.66:0/1414052285 >> 172.21.15.66:0/1414052285 conn(0x7f2b18098a20 msgr2=0x7f2b180b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b1d400640 1 -- 172.21.15.66:0/1414052285 shutdown_connections 2024-03-30T00:49:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.768+0000 7f2b1d400640 1 -- 172.21.15.66:0/1414052285 wait complete. 2024-03-30T00:49:04.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b1d400640 1 Processor -- start 2024-03-30T00:49:04.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b1d400640 1 -- start start 2024-03-30T00:49:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b1d400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b180ae4e0 0x7f2b18152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b1d400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b180ce820 0x7f2b180b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b1d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b1815ca10 0x7f2b180af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b1d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1815d190 con 0x7f2b1815ca10 2024-03-30T00:49:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b1d400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b18154260 con 0x7f2b180ae4e0 2024-03-30T00:49:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b1d400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b181539c0 con 0x7f2b180ce820 2024-03-30T00:49:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b17200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b1815ca10 0x7f2b180af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b17200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b1815ca10 0x7f2b180af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53228/0 (socket says 172.21.15.66:53228) 2024-03-30T00:49:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b17200640 1 -- 172.21.15.66:0/3928979046 learned_addr learned my addr 172.21.15.66:0/3928979046 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b15e00640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b180ce820 0x7f2b180b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b16800640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b180ae4e0 0x7f2b18152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b17200640 1 -- 172.21.15.66:0/3928979046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b180ce820 msgr2=0x7f2b180b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b17200640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b180ce820 0x7f2b180b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b17200640 1 -- 172.21.15.66:0/3928979046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b180ae4e0 msgr2=0x7f2b18152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b17200640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b180ae4e0 0x7f2b18152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b17200640 1 -- 172.21.15.66:0/3928979046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b040094d0 con 0x7f2b1815ca10 2024-03-30T00:49:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b17200640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b1815ca10 0x7f2b180af910 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000efc0 tx=0x7f2b1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b0ea00640 1 -- 172.21.15.66:0/3928979046 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10014480 con 0x7f2b1815ca10 2024-03-30T00:49:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b0ea00640 1 -- 172.21.15.66:0/3928979046 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1000f040 con 0x7f2b1815ca10 2024-03-30T00:49:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b0ea00640 1 -- 172.21.15.66:0/3928979046 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10014820 con 0x7f2b1815ca10 2024-03-30T00:49:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b1d400640 1 -- 172.21.15.66:0/3928979046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b18153ca0 con 0x7f2b1815ca10 2024-03-30T00:49:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b1d400640 1 -- 172.21.15.66:0/3928979046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b18152c80 con 0x7f2b1815ca10 2024-03-30T00:49:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.772+0000 7f2b1d400640 1 -- 172.21.15.66:0/3928979046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1815d300 con 0x7f2b1815ca10 2024-03-30T00:49:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.776+0000 7f2b0ea00640 1 -- 172.21.15.66:0/3928979046 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2b10014980 con 0x7f2b1815ca10 2024-03-30T00:49:04.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.776+0000 7f2b0ea00640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2af8066690 0x7f2af8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:04.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.776+0000 7f2b0ea00640 1 -- 172.21.15.66:0/3928979046 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2b10084e10 con 0x7f2b1815ca10 2024-03-30T00:49:04.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.776+0000 7f2b16800640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2af8066690 0x7f2af8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:04.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.776+0000 7f2b16800640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2af8066690 0x7f2af8068b50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2b08000f70 tx=0x7f2b08007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:04.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.780+0000 7f2b0ea00640 1 -- 172.21.15.66:0/3928979046 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2b1815d300 con 0x7f2b1815ca10 2024-03-30T00:49:04.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.940+0000 7f2b1d400640 1 -- 172.21.15.66:0/3928979046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b1815d2d0 con 0x7f2b1815ca10 2024-03-30T00:49:04.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.940+0000 7f2b0ea00640 1 -- 172.21.15.66:0/3928979046 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b100180a0 con 0x7f2b1815ca10 2024-03-30T00:49:04.952 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:49:04.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 -- 172.21.15.66:0/3928979046 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2af8066690 msgr2=0x7f2af8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2af8066690 0x7f2af8068b50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2b08000f70 tx=0x7f2b08007cb0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 -- 172.21.15.66:0/3928979046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b1815ca10 msgr2=0x7f2b180af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:04.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b1815ca10 0x7f2b180af910 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000efc0 tx=0x7f2b1000c490 comp rx=0 tx=0).stop 2024-03-30T00:49:04.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 -- 172.21.15.66:0/3928979046 shutdown_connections 2024-03-30T00:49:04.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2af8066690 0x7f2af8068b50 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b1815ca10 0x7f2b180af910 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b180ce820 0x7f2b180b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 --2- 172.21.15.66:0/3928979046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b180ae4e0 0x7f2b18152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:04.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 -- 172.21.15.66:0/3928979046 >> 172.21.15.66:0/3928979046 conn(0x7f2b18098a20 msgr2=0x7f2b180b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:04.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 -- 172.21.15.66:0/3928979046 shutdown_connections 2024-03-30T00:49:04.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:04.944+0000 7f2b1d400640 1 -- 172.21.15.66:0/3928979046 wait complete. 2024-03-30T00:49:04.974 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:49:05.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.104+0000 7fcd22e00640 1 Processor -- start 2024-03-30T00:49:05.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.104+0000 7fcd22e00640 1 -- start start 2024-03-30T00:49:05.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd22e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd1c0cda90 0x7fcd1c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd22e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcd1c0cde70 0x7fcd1c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd22e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c15a990 con 0x7fcd1c159c60 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd22e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c0b1b80 con 0x7fcd1c0cde70 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd22e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0cd3e0 con 0x7fcd1c0cda90 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd20e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fcd1c159c60 0x7fcd1c15a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56326/0 (socket says 172.21.15.66:56326) 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd20e00640 1 -- 172.21.15.66:0/712765926 learned_addr learned my addr 172.21.15.66:0/712765926 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd13400640 1 --2- 172.21.15.66:0/712765926 >> v2:172.21.15.66:3300/0 conn(0x7fcd1c0cde70 0x7fcd1c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd1be00640 1 --2- 172.21.15.66:0/712765926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd1c0cda90 0x7fcd1c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd13400640 1 -- 172.21.15.66:0/712765926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd1c0cda90 msgr2=0x7fcd1c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd13400640 1 --2- 172.21.15.66:0/712765926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd1c0cda90 0x7fcd1c0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd1b400640 1 -- 172.21.15.66:0/712765926 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837111113 0 0) 0x7fcd1c15a990 con 0x7fcd1c159c60 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd13400640 1 -- 172.21.15.66:0/712765926 >> v1:172.21.15.66:6789/0 conn(0x7fcd1c159c60 legacy=0x7fcd1c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.108+0000 7fcd13400640 1 -- 172.21.15.66:0/712765926 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c0b11e0 con 0x7fcd1c0cde70 2024-03-30T00:49:05.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd13400640 1 --2- 172.21.15.66:0/712765926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1c0cde70 0x7fcd1c0cea60 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097c0 tx=0x7fcd0401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aec40b7ad6b47a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:05.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd1b400640 1 -- 172.21.15.66:0/712765926 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd040ab600 con 0x7fcd1c0cde70 2024-03-30T00:49:05.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd1b400640 1 -- 172.21.15.66:0/712765926 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd040a7050 con 0x7fcd1c0cde70 2024-03-30T00:49:05.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd1b400640 1 -- 172.21.15.66:0/712765926 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd040ab980 con 0x7fcd1c0cde70 2024-03-30T00:49:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 -- 172.21.15.66:0/712765926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1c0cde70 msgr2=0x7fcd1c0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 --2- 172.21.15.66:0/712765926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1c0cde70 0x7fcd1c0cea60 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097c0 tx=0x7fcd0401c410 comp rx=0 tx=0).stop 2024-03-30T00:49:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 -- 172.21.15.66:0/712765926 shutdown_connections 2024-03-30T00:49:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 --2- 172.21.15.66:0/712765926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1c0cde70 0x7fcd1c0cea60 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 --2- 172.21.15.66:0/712765926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd1c0cda90 0x7fcd1c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 -- 172.21.15.66:0/712765926 >> 172.21.15.66:0/712765926 conn(0x7fcd1c069f60 msgr2=0x7fcd1c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 -- 172.21.15.66:0/712765926 shutdown_connections 2024-03-30T00:49:05.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 -- 172.21.15.66:0/712765926 wait complete. 2024-03-30T00:49:05.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 Processor -- start 2024-03-30T00:49:05.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 -- start start 2024-03-30T00:49:05.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd1c0cda90 0x7fcd1c156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1c0cde70 0x7fcd1c0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.112+0000 7fcd22e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1c159c60 0x7fcd1c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd22e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c1567b0 con 0x7fcd1c159c60 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd22e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c163e60 con 0x7fcd1c0cde70 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd22e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c162ae0 con 0x7fcd1c0cda90 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd20e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1c159c60 0x7fcd1c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd20e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1c159c60 0x7fcd1c158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53238/0 (socket says 172.21.15.66:53238) 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd20e00640 1 -- 172.21.15.66:0/2405059740 learned_addr learned my addr 172.21.15.66:0/2405059740 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd1be00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd1c0cda90 0x7fcd1c156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd13400640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1c0cde70 0x7fcd1c0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd20e00640 1 -- 172.21.15.66:0/2405059740 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd1c0cda90 msgr2=0x7fcd1c156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd20e00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd1c0cda90 0x7fcd1c156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd20e00640 1 -- 172.21.15.66:0/2405059740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1c0cde70 msgr2=0x7fcd1c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd20e00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1c0cde70 0x7fcd1c0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd20e00640 1 -- 172.21.15.66:0/2405059740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd04009420 con 0x7fcd1c159c60 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd1be00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd1c0cda90 0x7fcd1c156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:05.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd13400640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1c0cde70 0x7fcd1c0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd20e00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1c159c60 0x7fcd1c158070 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400c3e0 tx=0x7fcd1400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd18c00640 1 -- 172.21.15.66:0/2405059740 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd140074e0 con 0x7fcd1c159c60 2024-03-30T00:49:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd18c00640 1 -- 172.21.15.66:0/2405059740 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd14007660 con 0x7fcd1c159c60 2024-03-30T00:49:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd22e00640 1 -- 172.21.15.66:0/2405059740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c162dc0 con 0x7fcd1c159c60 2024-03-30T00:49:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd18c00640 1 -- 172.21.15.66:0/2405059740 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14004a30 con 0x7fcd1c159c60 2024-03-30T00:49:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd22e00640 1 -- 172.21.15.66:0/2405059740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c164640 con 0x7fcd1c159c60 2024-03-30T00:49:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd22e00640 1 -- 172.21.15.66:0/2405059740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c066c30 con 0x7fcd1c159c60 2024-03-30T00:49:05.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.116+0000 7fcd18c00640 1 -- 172.21.15.66:0/2405059740 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcd14004b90 con 0x7fcd1c159c60 2024-03-30T00:49:05.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.120+0000 7fcd18c00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccf8066690 0x7fccf8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.120+0000 7fcd18c00640 1 -- 172.21.15.66:0/2405059740 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcd14084020 con 0x7fcd1c159c60 2024-03-30T00:49:05.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.120+0000 7fcd1be00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccf8066690 0x7fccf8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.120+0000 7fcd1be00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccf8066690 0x7fccf8068b50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c007330 tx=0x7fcd0c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:05.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.124+0000 7fcd18c00640 1 -- 172.21.15.66:0/2405059740 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcd14054ce0 con 0x7fcd1c159c60 2024-03-30T00:49:05.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 -- 172.21.15.66:0/2405059740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd1c1642f0 con 0x7fcd1c159c60 2024-03-30T00:49:05.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd18c00640 1 -- 172.21.15.66:0/2405059740 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd14001090 con 0x7fcd1c159c60 2024-03-30T00:49:05.295 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 -- 172.21.15.66:0/2405059740 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccf8066690 msgr2=0x7fccf8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccf8066690 0x7fccf8068b50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c007330 tx=0x7fcd0c00b040 comp rx=0 tx=0).stop 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 -- 172.21.15.66:0/2405059740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1c159c60 msgr2=0x7fcd1c158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1c159c60 0x7fcd1c158070 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400c3e0 tx=0x7fcd1400c8a0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 -- 172.21.15.66:0/2405059740 shutdown_connections 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccf8066690 0x7fccf8068b50 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd1c159c60 0x7fcd1c158070 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1c0cde70 0x7fcd1c0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 --2- 172.21.15.66:0/2405059740 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd1c0cda90 0x7fcd1c156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 -- 172.21.15.66:0/2405059740 >> 172.21.15.66:0/2405059740 conn(0x7fcd1c069f60 msgr2=0x7fcd1c0afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 -- 172.21.15.66:0/2405059740 shutdown_connections 2024-03-30T00:49:05.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.284+0000 7fcd22e00640 1 -- 172.21.15.66:0/2405059740 wait complete. 2024-03-30T00:49:05.316 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:49:05.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.444+0000 7feee7e00640 1 Processor -- start 2024-03-30T00:49:05.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee7e00640 1 -- start start 2024-03-30T00:49:05.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feee00b0fb0 0x7feee006b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee7e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7feee00b13f0 0x7feee006b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee7e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee0098ac0 con 0x7feee00cca40 2024-03-30T00:49:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee7e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee00cf3f0 con 0x7feee00b13f0 2024-03-30T00:49:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee7e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee00cf670 con 0x7feee00b0fb0 2024-03-30T00:49:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee5400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7feee00cca40 0x7feee00cac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56338/0 (socket says 172.21.15.66:56338) 2024-03-30T00:49:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee5400640 1 -- 172.21.15.66:0/261425518 learned_addr learned my addr 172.21.15.66:0/261425518 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee4a00640 1 --2- 172.21.15.66:0/261425518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feee00b0fb0 0x7feee006b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee5e00640 1 --2- 172.21.15.66:0/261425518 >> v2:172.21.15.156:3300/0 conn(0x7feee00b13f0 0x7feee006b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feedfe00640 1 -- 172.21.15.66:0/261425518 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300514046 0 0) 0x7feee0098ac0 con 0x7feee00cca40 2024-03-30T00:49:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feedfe00640 1 -- 172.21.15.66:0/261425518 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec8003530 con 0x7feee00cca40 2024-03-30T00:49:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee4a00640 1 -- 172.21.15.66:0/261425518 >> v2:172.21.15.156:3300/0 conn(0x7feee00b13f0 msgr2=0x7feee006b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee4a00640 1 --2- 172.21.15.66:0/261425518 >> v2:172.21.15.156:3300/0 conn(0x7feee00b13f0 0x7feee006b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee4a00640 1 -- 172.21.15.66:0/261425518 >> v1:172.21.15.66:6789/0 conn(0x7feee00cca40 legacy=0x7feee00cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.448+0000 7feee4a00640 1 -- 172.21.15.66:0/261425518 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee00cf950 con 0x7feee00b0fb0 2024-03-30T00:49:05.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feee4a00640 1 --2- 172.21.15.66:0/261425518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feee00b0fb0 0x7feee006b140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feecc0097c0 tx=0x7feecc01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699f90b82f2fb1c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:05.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feedfe00640 1 -- 172.21.15.66:0/261425518 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0ab600 con 0x7feee00b0fb0 2024-03-30T00:49:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feedfe00640 1 -- 172.21.15.66:0/261425518 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc0ab760 con 0x7feee00b0fb0 2024-03-30T00:49:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feedfe00640 1 -- 172.21.15.66:0/261425518 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0abad0 con 0x7feee00b0fb0 2024-03-30T00:49:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feee7e00640 1 -- 172.21.15.66:0/261425518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feee00b0fb0 msgr2=0x7feee006b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feee7e00640 1 --2- 172.21.15.66:0/261425518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feee00b0fb0 0x7feee006b140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feecc0097c0 tx=0x7feecc01c470 comp rx=0 tx=0).stop 2024-03-30T00:49:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feee7e00640 1 -- 172.21.15.66:0/261425518 shutdown_connections 2024-03-30T00:49:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feee7e00640 1 --2- 172.21.15.66:0/261425518 >> v2:172.21.15.156:3300/0 conn(0x7feee00b13f0 0x7feee006b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feee7e00640 1 --2- 172.21.15.66:0/261425518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feee00b0fb0 0x7feee006b140 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feee7e00640 1 -- 172.21.15.66:0/261425518 >> 172.21.15.66:0/261425518 conn(0x7feee00b2130 msgr2=0x7feee00b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feee7e00640 1 -- 172.21.15.66:0/261425518 shutdown_connections 2024-03-30T00:49:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feee7e00640 1 -- 172.21.15.66:0/261425518 wait complete. 2024-03-30T00:49:05.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feee7e00640 1 Processor -- start 2024-03-30T00:49:05.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.452+0000 7feee7e00640 1 -- start start 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee7e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feee00b0fb0 0x7feee00b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee00b13f0 0x7feee00b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feee00cca40 0x7feee00c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee00b58c0 con 0x7feee00b13f0 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee7e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee00b4ff0 con 0x7feee00b0fb0 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee7e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee00b6220 con 0x7feee00cca40 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee00b13f0 0x7feee00b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee00b13f0 0x7feee00b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53248/0 (socket says 172.21.15.66:53248) 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee4a00640 1 -- 172.21.15.66:0/3180719809 learned_addr learned my addr 172.21.15.66:0/3180719809 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee5e00640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feee00cca40 0x7feee00c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee5400640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feee00b0fb0 0x7feee00b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee4a00640 1 -- 172.21.15.66:0/3180719809 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feee00cca40 msgr2=0x7feee00c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee4a00640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feee00cca40 0x7feee00c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee4a00640 1 -- 172.21.15.66:0/3180719809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feee00b0fb0 msgr2=0x7feee00b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee4a00640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feee00b0fb0 0x7feee00b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee4a00640 1 -- 172.21.15.66:0/3180719809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc009420 con 0x7feee00b13f0 2024-03-30T00:49:05.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee4a00640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee00b13f0 0x7feee00b5ce0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7feecc01cd10 tx=0x7feecc01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:05.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feedd600640 1 -- 172.21.15.66:0/3180719809 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0bf610 con 0x7feee00b13f0 2024-03-30T00:49:05.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feedd600640 1 -- 172.21.15.66:0/3180719809 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc0bf770 con 0x7feee00b13f0 2024-03-30T00:49:05.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feedd600640 1 -- 172.21.15.66:0/3180719809 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0bfae0 con 0x7feee00b13f0 2024-03-30T00:49:05.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee7e00640 1 -- 172.21.15.66:0/3180719809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee00b4c40 con 0x7feee00b13f0 2024-03-30T00:49:05.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee7e00640 1 -- 172.21.15.66:0/3180719809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee00b54e0 con 0x7feee00b13f0 2024-03-30T00:49:05.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.456+0000 7feee7e00640 1 -- 172.21.15.66:0/3180719809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee00677d0 con 0x7feee00b13f0 2024-03-30T00:49:05.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.460+0000 7feedd600640 1 -- 172.21.15.66:0/3180719809 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7feecc0bfc40 con 0x7feee00b13f0 2024-03-30T00:49:05.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.460+0000 7feedd600640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feec0066690 0x7feec0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:05.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.460+0000 7feedd600640 1 -- 172.21.15.66:0/3180719809 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7feecc11ea50 con 0x7feee00b13f0 2024-03-30T00:49:05.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.460+0000 7feee5400640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feec0066690 0x7feec0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:05.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.460+0000 7feee5400640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feec0066690 0x7feec0068b50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feed4007fd0 tx=0x7feed4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:05.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.464+0000 7feedd600640 1 -- 172.21.15.66:0/3180719809 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7feecc0ebee0 con 0x7feee00b13f0 2024-03-30T00:49:05.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.636+0000 7feee7e00640 1 -- 172.21.15.66:0/3180719809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feee00b5a00 con 0x7feee00b13f0 2024-03-30T00:49:05.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.636+0000 7feedd600640 1 -- 172.21.15.66:0/3180719809 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7feee00b5a00 con 0x7feee00b13f0 2024-03-30T00:49:05.648 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:49:05.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 -- 172.21.15.66:0/3180719809 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feec0066690 msgr2=0x7feec0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feec0066690 0x7feec0068b50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feed4007fd0 tx=0x7feed4009040 comp rx=0 tx=0).stop 2024-03-30T00:49:05.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 -- 172.21.15.66:0/3180719809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee00b13f0 msgr2=0x7feee00b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:05.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee00b13f0 0x7feee00b5ce0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7feecc01cd10 tx=0x7feecc01acb0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 -- 172.21.15.66:0/3180719809 shutdown_connections 2024-03-30T00:49:05.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feee00cca40 0x7feee00c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7feec0066690 0x7feec0068b50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee00b13f0 0x7feee00b5ce0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 --2- 172.21.15.66:0/3180719809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feee00b0fb0 0x7feee00b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:05.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 -- 172.21.15.66:0/3180719809 >> 172.21.15.66:0/3180719809 conn(0x7feee00b2130 msgr2=0x7feee00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:05.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 -- 172.21.15.66:0/3180719809 shutdown_connections 2024-03-30T00:49:05.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:05.640+0000 7feee7e00640 1 -- 172.21.15.66:0/3180719809 wait complete. 2024-03-30T00:49:05.670 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:49:05.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:49:05.671 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:49:05.689 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[51453]: fuse finished with error 0 and tester_r 0 2024-03-30T00:49:05.694 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:49:11.796 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:49:11.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:49:11.797 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:49:11.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:49:12.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.988+0000 7f379aa00640 1 Processor -- start 2024-03-30T00:49:12.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.988+0000 7f379aa00640 1 -- start start 2024-03-30T00:49:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.992+0000 7f379aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f379415aa40 0x7f37940cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.992+0000 7f379aa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379415eab0 con 0x7f379415e5d0 2024-03-30T00:49:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.992+0000 7f379aa00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379415ed90 con 0x7f379415ae20 2024-03-30T00:49:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.992+0000 7f379aa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37940cec50 con 0x7f379415aa40 2024-03-30T00:49:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.992+0000 7f3799600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f379415aa40 0x7f37940cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.992+0000 7f3799600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f379415aa40 0x7f37940cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39690/0 (socket says 172.21.15.66:39690) 2024-03-30T00:49:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.992+0000 7f3799600640 1 -- 172.21.15.66:0/3618353791 learned_addr learned my addr 172.21.15.66:0/3618353791 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.992+0000 7f3799600640 1 -- 172.21.15.66:0/3618353791 >> v1:172.21.15.156:6789/0 conn(0x7f379415ae20 legacy=0x7f37940ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.992+0000 7f3799600640 1 -- 172.21.15.66:0/3618353791 >> v1:172.21.15.66:6789/0 conn(0x7f379415e5d0 legacy=0x7f37940ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.992+0000 7f3799600640 1 -- 172.21.15.66:0/3618353791 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379415f070 con 0x7f379415aa40 2024-03-30T00:49:12.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.992+0000 7f3799600640 1 --2- 172.21.15.66:0/3618353791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f379415aa40 0x7f37940cb500 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f3788009870 tx=0x7f378801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb47ed31912e45d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f3793e00640 1 -- 172.21.15.66:0/3618353791 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37880ab600 con 0x7f379415aa40 2024-03-30T00:49:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f3793e00640 1 -- 172.21.15.66:0/3618353791 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37880ab760 con 0x7f379415aa40 2024-03-30T00:49:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f3793e00640 1 -- 172.21.15.66:0/3618353791 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37880abaf0 con 0x7f379415aa40 2024-03-30T00:49:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 -- 172.21.15.66:0/3618353791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f379415aa40 msgr2=0x7f37940cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 --2- 172.21.15.66:0/3618353791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f379415aa40 0x7f37940cb500 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f3788009870 tx=0x7f378801c4d0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 -- 172.21.15.66:0/3618353791 shutdown_connections 2024-03-30T00:49:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 --2- 172.21.15.66:0/3618353791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f379415aa40 0x7f37940cb500 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 -- 172.21.15.66:0/3618353791 >> 172.21.15.66:0/3618353791 conn(0x7f379406e0d0 msgr2=0x7f379406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 -- 172.21.15.66:0/3618353791 shutdown_connections 2024-03-30T00:49:12.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 -- 172.21.15.66:0/3618353791 wait complete. 2024-03-30T00:49:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 Processor -- start 2024-03-30T00:49:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 -- start start 2024-03-30T00:49:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f379415aa40 0x7f3794157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f379415ae20 0x7f3794152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f379415e5d0 0x7f3794152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3794159b80 con 0x7f379415e5d0 2024-03-30T00:49:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3794159e20 con 0x7f379415aa40 2024-03-30T00:49:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379aa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379415a100 con 0x7f379415ae20 2024-03-30T00:49:12.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f379415e5d0 0x7f3794152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f379415e5d0 0x7f3794152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39706/0 (socket says 172.21.15.66:39706) 2024-03-30T00:49:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f379a000640 1 -- 172.21.15.66:0/1474556184 learned_addr learned my addr 172.21.15.66:0/1474556184 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f3799600640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f379415aa40 0x7f3794157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:11.996+0000 7f3798c00640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f379415ae20 0x7f3794152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f379a000640 1 -- 172.21.15.66:0/1474556184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f379415ae20 msgr2=0x7f3794152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f379a000640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f379415ae20 0x7f3794152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f379a000640 1 -- 172.21.15.66:0/1474556184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f379415aa40 msgr2=0x7f3794157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f379a000640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f379415aa40 0x7f3794157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f379a000640 1 -- 172.21.15.66:0/1474556184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880094d0 con 0x7f379415e5d0 2024-03-30T00:49:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f379a000640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f379415e5d0 0x7f3794152a40 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f378c00c7a0 tx=0x7f378c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f3791600640 1 -- 172.21.15.66:0/1474556184 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c003b80 con 0x7f379415e5d0 2024-03-30T00:49:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f3791600640 1 -- 172.21.15.66:0/1474556184 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c003ce0 con 0x7f379415e5d0 2024-03-30T00:49:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f3791600640 1 -- 172.21.15.66:0/1474556184 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c00aa30 con 0x7f379415e5d0 2024-03-30T00:49:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f379aa00640 1 -- 172.21.15.66:0/1474556184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37940ca540 con 0x7f379415e5d0 2024-03-30T00:49:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f379aa00640 1 -- 172.21.15.66:0/1474556184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37940affd0 con 0x7f379415e5d0 2024-03-30T00:49:12.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f379aa00640 1 -- 172.21.15.66:0/1474556184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37940b1770 con 0x7f379415e5d0 2024-03-30T00:49:12.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f3791600640 1 -- 172.21.15.66:0/1474556184 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f378c00ab90 con 0x7f379415e5d0 2024-03-30T00:49:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.000+0000 7f3791600640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3774066690 0x7f3774068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.004+0000 7f3791600640 1 -- 172.21.15.66:0/1474556184 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f378c083fe0 con 0x7f379415e5d0 2024-03-30T00:49:12.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.004+0000 7f3799600640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3774066690 0x7f3774068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.004+0000 7f3799600640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3774066690 0x7f3774068b50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f37880af000 tx=0x7f37880b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.008+0000 7f3791600640 1 -- 172.21.15.66:0/1474556184 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f37940b1770 con 0x7f379415e5d0 2024-03-30T00:49:12.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.200+0000 7f379aa00640 1 -- 172.21.15.66:0/1474556184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37940b1770 con 0x7f379415e5d0 2024-03-30T00:49:12.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f3791600640 1 -- 172.21.15.66:0/1474556184 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f37940b1770 con 0x7f379415e5d0 2024-03-30T00:49:12.214 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:12.214 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":79,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:48:49.341688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8646":{"gid":8646,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/2988412199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2988412199},{"type":"v1","addr":"172.21.15.66:6837","nonce":2988412199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:49:12.214 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 79 2024-03-30T00:49:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 -- 172.21.15.66:0/1474556184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3774066690 msgr2=0x7f3774068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3774066690 0x7f3774068b50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f37880af000 tx=0x7f37880b6040 comp rx=0 tx=0).stop 2024-03-30T00:49:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 -- 172.21.15.66:0/1474556184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f379415e5d0 msgr2=0x7f3794152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f379415e5d0 0x7f3794152a40 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f378c00c7a0 tx=0x7f378c00cc60 comp rx=0 tx=0).stop 2024-03-30T00:49:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 -- 172.21.15.66:0/1474556184 shutdown_connections 2024-03-30T00:49:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3774066690 0x7f3774068b50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f379415e5d0 0x7f3794152a40 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f379415ae20 0x7f3794152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 --2- 172.21.15.66:0/1474556184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f379415aa40 0x7f3794157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 -- 172.21.15.66:0/1474556184 >> 172.21.15.66:0/1474556184 conn(0x7f379406e0d0 msgr2=0x7f37940b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 -- 172.21.15.66:0/1474556184 shutdown_connections 2024-03-30T00:49:12.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.204+0000 7f379aa00640 1 -- 172.21.15.66:0/1474556184 wait complete. 2024-03-30T00:49:12.235 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-30T00:49:12.409 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-03-30T00:49:12.409 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-30T00:49:12.409 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:49:12.409 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:49:12.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:49:12.419 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi066.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-03-30T00:49:12.419 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:49:12.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:49:12.419 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:49:12.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:49:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-30T00:49:12.472 DEBUG:teuthology.orchestra.run.smithi066:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-30T00:49:12.515 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-03-30T00:49:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.648+0000 7ff9af200640 1 Processor -- start 2024-03-30T00:49:12.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.648+0000 7ff9af200640 1 -- start start 2024-03-30T00:49:12.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff9af200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff9a815de70 0x7ff9a80b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff9af200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff9a8099760 0x7ff9a8098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff9af200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a80ce230 con 0x7ff9a8099bf0 2024-03-30T00:49:12.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff9af200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a80cbe90 con 0x7ff9a8099760 2024-03-30T00:49:12.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff9af200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a80cc170 con 0x7ff9a815de70 2024-03-30T00:49:12.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff99f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff9a8099760 0x7ff9a8098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff99f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff9a8099760 0x7ff9a8098a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39708/0 (socket says 172.21.15.66:39708) 2024-03-30T00:49:12.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff99f400640 1 -- 172.21.15.66:0/2517954708 learned_addr learned my addr 172.21.15.66:0/2517954708 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/2517954708 >> v2:172.21.15.156:3300/0 conn(0x7ff9a815de70 0x7ff9a80b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff99f400640 1 -- 172.21.15.66:0/2517954708 >> v2:172.21.15.156:3300/0 conn(0x7ff9a815de70 msgr2=0x7ff9a80b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff99f400640 1 --2- 172.21.15.66:0/2517954708 >> v2:172.21.15.156:3300/0 conn(0x7ff9a815de70 0x7ff9a80b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff99f400640 1 -- 172.21.15.66:0/2517954708 >> v1:172.21.15.156:6789/0 conn(0x7ff9a8099bf0 legacy=0x7ff9a80ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff99f400640 1 -- 172.21.15.66:0/2517954708 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a80cc450 con 0x7ff9a8099760 2024-03-30T00:49:12.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.652+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/2517954708 >> v2:172.21.15.156:3300/0 conn(0x7ff9a815de70 0x7ff9a80b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:12.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff99f400640 1 --2- 172.21.15.66:0/2517954708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9a8099760 0x7ff9a8098a20 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80714a0 tx=0x7ff99001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46468a23e915a917 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff9a7400640 1 -- 172.21.15.66:0/2517954708 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9900ab5f0 con 0x7ff9a8099760 2024-03-30T00:49:12.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff9a7400640 1 -- 172.21.15.66:0/2517954708 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9900ab750 con 0x7ff9a8099760 2024-03-30T00:49:12.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff9a7400640 1 -- 172.21.15.66:0/2517954708 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9900abae0 con 0x7ff9a8099760 2024-03-30T00:49:12.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff9af200640 1 -- 172.21.15.66:0/2517954708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9a8099760 msgr2=0x7ff9a8098a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff9af200640 1 --2- 172.21.15.66:0/2517954708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9a8099760 0x7ff9a8098a20 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80714a0 tx=0x7ff99001c4e0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff9af200640 1 -- 172.21.15.66:0/2517954708 shutdown_connections 2024-03-30T00:49:12.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff9af200640 1 --2- 172.21.15.66:0/2517954708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9a8099760 0x7ff9a8098a20 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff9af200640 1 --2- 172.21.15.66:0/2517954708 >> v2:172.21.15.156:3300/0 conn(0x7ff9a815de70 0x7ff9a80b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff9af200640 1 -- 172.21.15.66:0/2517954708 >> 172.21.15.66:0/2517954708 conn(0x7ff9a80b2150 msgr2=0x7ff9a80b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff9af200640 1 -- 172.21.15.66:0/2517954708 shutdown_connections 2024-03-30T00:49:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.656+0000 7ff9af200640 1 -- 172.21.15.66:0/2517954708 wait complete. 2024-03-30T00:49:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9af200640 1 Processor -- start 2024-03-30T00:49:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9af200640 1 -- start start 2024-03-30T00:49:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9af200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9a8099760 0x7ff9a80b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9af200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9a8099bf0 0x7ff9a806aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9af200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9a815de70 0x7ff9a81642e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9af200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a806b3e0 con 0x7ff9a815de70 2024-03-30T00:49:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9af200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a806b680 con 0x7ff9a8099bf0 2024-03-30T00:49:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9af200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a806b960 con 0x7ff9a8099760 2024-03-30T00:49:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9ad200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9a815de70 0x7ff9a81642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff99f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9a8099bf0 0x7ff9a806aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9ad200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9a815de70 0x7ff9a81642e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39712/0 (socket says 172.21.15.66:39712) 2024-03-30T00:49:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9ad200640 1 -- 172.21.15.66:0/212992296 learned_addr learned my addr 172.21.15.66:0/212992296 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9a8099760 0x7ff9a80b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9ad200640 1 -- 172.21.15.66:0/212992296 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9a8099760 msgr2=0x7ff9a80b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9ad200640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9a8099760 0x7ff9a80b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9ad200640 1 -- 172.21.15.66:0/212992296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9a8099bf0 msgr2=0x7ff9a806aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9ad200640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9a8099bf0 0x7ff9a806aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9ad200640 1 -- 172.21.15.66:0/212992296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9900094d0 con 0x7ff9a815de70 2024-03-30T00:49:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff99f400640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9a8099bf0 0x7ff9a806aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:49:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.660+0000 7ff9ad200640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9a815de70 0x7ff9a81642e0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c780 tx=0x7ff9a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.664+0000 7ff9a4c00640 1 -- 172.21.15.66:0/212992296 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0004b70 con 0x7ff9a815de70 2024-03-30T00:49:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.664+0000 7ff9a4c00640 1 -- 172.21.15.66:0/212992296 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a000ce90 con 0x7ff9a815de70 2024-03-30T00:49:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.664+0000 7ff9a4c00640 1 -- 172.21.15.66:0/212992296 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000a850 con 0x7ff9a815de70 2024-03-30T00:49:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.664+0000 7ff9af200640 1 -- 172.21.15.66:0/212992296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a806bc40 con 0x7ff9a815de70 2024-03-30T00:49:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.664+0000 7ff9af200640 1 -- 172.21.15.66:0/212992296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a806cb80 con 0x7ff9a815de70 2024-03-30T00:49:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.664+0000 7ff9a4c00640 1 -- 172.21.15.66:0/212992296 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff9a000a9b0 con 0x7ff9a815de70 2024-03-30T00:49:12.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.664+0000 7ff9a4c00640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff984066690 0x7ff984068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.664+0000 7ff9a4c00640 1 -- 172.21.15.66:0/212992296 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff9a0084270 con 0x7ff9a815de70 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.664+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff984066690 0x7ff984068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.664+0000 7ff9b258a000 1 Processor -- start 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 -- start start 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff984066690 0x7ff984068b50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff998006fd0 tx=0x7ff998008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094dd30 0x556f7094e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 0x556f70904a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f70904f60 0x556f70905340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f707cc170 con 0x556f7094dd30 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f70759d80 con 0x556f70904f60 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f70757350 con 0x556f7094e650 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094dd30 0x556f7094e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094dd30 0x556f7094e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39720/0 (socket says 172.21.15.66:39720) 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9a7e00640 1 -- 172.21.15.66:0/2705075547 learned_addr learned my addr 172.21.15.66:0/2705075547 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff99f400640 1 --2- 172.21.15.66:0/2705075547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 0x556f70904a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9ad200640 1 --2- 172.21.15.66:0/2705075547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f70904f60 0x556f70905340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9a7e00640 1 -- 172.21.15.66:0/2705075547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 msgr2=0x556f70904a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/2705075547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 0x556f70904a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9a7e00640 1 -- 172.21.15.66:0/2705075547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f70904f60 msgr2=0x556f70905340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/2705075547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f70904f60 0x556f70905340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9a7e00640 1 -- 172.21.15.66:0/2705075547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f7044f070 con 0x556f7094dd30 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff99f400640 1 --2- 172.21.15.66:0/2705075547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 0x556f70904a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/2705075547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094dd30 0x556f7094e110 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7ff99800fd60 tx=0x7ff99800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6616c06da70d591d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff99cc00640 1 -- 172.21.15.66:0/2705075547 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998013620 con 0x556f7094dd30 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff99cc00640 1 -- 172.21.15.66:0/2705075547 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99800c040 con 0x556f7094dd30 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff99cc00640 1 -- 172.21.15.66:0/2705075547 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998013980 con 0x556f7094dd30 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 -- 172.21.15.66:0/2705075547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094dd30 msgr2=0x556f7094e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 --2- 172.21.15.66:0/2705075547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094dd30 0x556f7094e110 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7ff99800fd60 tx=0x7ff99800af90 comp rx=0 tx=0).stop 2024-03-30T00:49:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 -- 172.21.15.66:0/2705075547 shutdown_connections 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 --2- 172.21.15.66:0/2705075547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f70904f60 0x556f70905340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 --2- 172.21.15.66:0/2705075547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 0x556f70904a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 --2- 172.21.15.66:0/2705075547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094dd30 0x556f7094e110 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 -- 172.21.15.66:0/2705075547 >> 172.21.15.66:0/2705075547 conn(0x556f70900f10 msgr2=0x556f70903300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 -- 172.21.15.66:0/2705075547 shutdown_connections 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.668+0000 7ff9b258a000 1 -- 172.21.15.66:0/2705075547 wait complete. 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 -- start start 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 Processor -- start 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f70904f60 0x556f7090a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f7094dd30 0x556f7094ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 0x556f7094f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f70752c40 con 0x556f70904f60 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f706f1fa0 con 0x556f7094dd30 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f707101c0 con 0x556f7094e650 2024-03-30T00:49:12.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 10 client.0 _ll_register_callbacks cb 0x556f708f0060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f70904f60 0x556f7090a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f70904f60 0x556f7090a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39734/0 (socket says 172.21.15.66:39734) 2024-03-30T00:49:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9a7e00640 1 -- 172.21.15.66:0/4120507698 learned_addr learned my addr 172.21.15.66:0/4120507698 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff99f400640 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f7094dd30 0x556f7094ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9ad200640 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 0x556f7094f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9a7e00640 1 -- 172.21.15.66:0/4120507698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 msgr2=0x556f7094f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 0x556f7094f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9a7e00640 1 -- 172.21.15.66:0/4120507698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f7094dd30 msgr2=0x556f7094ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f7094dd30 0x556f7094ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9a7e00640 1 -- 172.21.15.66:0/4120507698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99800fa40 con 0x556f70904f60 2024-03-30T00:49:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9ad200640 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 0x556f7094f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f70904f60 0x556f7090a8e0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7ff998008010 tx=0x7ff998004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa02920675b581c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff995600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff995600640 1 -- 172.21.15.66:0/4120507698 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980275f0 con 0x556f70904f60 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff995600640 1 -- 172.21.15.66:0/4120507698 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff99801a040 con 0x556f70904f60 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 -- 172.21.15.66:0/4120507698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f7074fa20 con 0x556f70904f60 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff995600640 1 -- 172.21.15.66:0/4120507698 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff99800a750 con 0x556f70904f60 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 10 client.8991 fetch_fsmap learned FSMap version 79 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 -- 172.21.15.66:0/4120507698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x556f70717ba0 con 0x556f70904f60 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff995600640 1 -- 172.21.15.66:0/4120507698 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f7074fa20 con 0x556f70904f60 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff995600640 1 -- 172.21.15.66:0/4120507698 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 79) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff998011040 con 0x556f70904f60 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 10 client.8991 fetch_fsmap finished waiting for FSMap version 79 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 10 client.8991 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 20 client.8991 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x556f709552c0 0x556f709556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 4 client.8991 mds_command: new command op to 8646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:49:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff9b258a000 1 -- 172.21.15.66:0/4120507698 --> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f70751590 con 0x556f709552c0 2024-03-30T00:49:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.672+0000 7ff99f400640 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x556f709552c0 0x556f709556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff99f400640 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x556f709552c0 0x556f709556d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d077b5358461220 server_cookie=5904fc4dc88e3a75 in_seq=0 out_seq=0 2024-03-30T00:49:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff995600640 10 client.8991 ms_handle_connect on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98e3a00640 1 Processor -- start 2024-03-30T00:49:12.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98e3a00640 1 -- start start 2024-03-30T00:49:12.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff995600640 1 -- 172.21.15.66:0/4120507698 <== mds.0 v2:172.21.15.66:6835/2988412199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f70751590 con 0x556f709552c0 2024-03-30T00:49:12.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff995600640 10 client.8991 handle_command_reply: tid=0 2024-03-30T00:49:12.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 1 client.8991 shutdown 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 1 -- 172.21.15.66:0/4120507698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f70904f60 msgr2=0x556f7090a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f70904f60 0x556f7090a8e0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7ff998008010 tx=0x7ff998004740 comp rx=0 tx=0).stop 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 1 -- 172.21.15.66:0/4120507698 shutdown_connections 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094e650 0x556f7094f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x556f709552c0 0x556f709556d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f7094dd30 0x556f7094ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 1 --2- 172.21.15.66:0/4120507698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f70904f60 0x556f7090a8e0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff995600640 0 client.8991 ms_handle_reset on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 1 -- 172.21.15.66:0/4120507698 >> 172.21.15.66:0/4120507698 conn(0x556f70900f10 msgr2=0x556f70903b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 1 -- 172.21.15.66:0/4120507698 shutdown_connections 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 1 -- 172.21.15.66:0/4120507698 wait complete. 2024-03-30T00:49:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7ff9b258a000 20 client.8991 trim_cache size 0 max 16384 2024-03-30T00:49:12.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98e3a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f98dc15a630 0x7f98dc065b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98e3a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f98dc15aa10 0x7f98dc06ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98e3a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98dc06b2e0 0x7f98dc0abd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98e3a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc06b750 con 0x7f98dc15aa10 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98e3a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc15e420 con 0x7f98dc15a630 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98e3a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc15e700 con 0x7f98dc06b2e0 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98dbe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f98dc15aa10 0x7f98dc06ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98dbe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f98dc15aa10 0x7f98dc06ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39746/0 (socket says 172.21.15.66:39746) 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98dbe00640 1 -- 172.21.15.66:0/201142570 learned_addr learned my addr 172.21.15.66:0/201142570 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98e1000640 1 --2- 172.21.15.66:0/201142570 >> v2:172.21.15.156:3300/0 conn(0x7f98dc15a630 0x7f98dc065b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98e1a00640 1 --2- 172.21.15.66:0/201142570 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98dc06b2e0 0x7f98dc0abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98dbe00640 1 -- 172.21.15.66:0/201142570 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98dc06b2e0 msgr2=0x7f98dc0abd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98dbe00640 1 --2- 172.21.15.66:0/201142570 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98dc06b2e0 0x7f98dc0abd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98dbe00640 1 -- 172.21.15.66:0/201142570 >> v2:172.21.15.156:3300/0 conn(0x7f98dc15a630 msgr2=0x7f98dc065b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98dbe00640 1 --2- 172.21.15.66:0/201142570 >> v2:172.21.15.156:3300/0 conn(0x7f98dc15a630 0x7f98dc065b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98dbe00640 1 -- 172.21.15.66:0/201142570 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc15e9e0 con 0x7f98dc15aa10 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.676+0000 7f98e1a00640 1 --2- 172.21.15.66:0/201142570 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98dc06b2e0 0x7f98dc0abd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.680+0000 7f98dbe00640 1 --2- 172.21.15.66:0/201142570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98dc15aa10 0x7f98dc06ada0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f98dc06bd70 tx=0x7f98c801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f12b6ef4f0633ba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.680+0000 7f98db400640 1 -- 172.21.15.66:0/201142570 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c80ab600 con 0x7f98dc15aa10 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.680+0000 7f98db400640 1 -- 172.21.15.66:0/201142570 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c80a7050 con 0x7f98dc15aa10 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98db400640 1 -- 172.21.15.66:0/201142570 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c80ab980 con 0x7f98dc15aa10 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 -- 172.21.15.66:0/201142570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98dc15aa10 msgr2=0x7f98dc06ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 --2- 172.21.15.66:0/201142570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98dc15aa10 0x7f98dc06ada0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f98dc06bd70 tx=0x7f98c801c230 comp rx=0 tx=0).stop 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 -- 172.21.15.66:0/201142570 shutdown_connections 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 --2- 172.21.15.66:0/201142570 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98dc06b2e0 0x7f98dc0abd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 --2- 172.21.15.66:0/201142570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98dc15aa10 0x7f98dc06ada0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 --2- 172.21.15.66:0/201142570 >> v2:172.21.15.156:3300/0 conn(0x7f98dc15a630 0x7f98dc065b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 -- 172.21.15.66:0/201142570 >> 172.21.15.66:0/201142570 conn(0x7f98dc070950 msgr2=0x7f98dc070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 -- 172.21.15.66:0/201142570 shutdown_connections 2024-03-30T00:49:12.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 -- 172.21.15.66:0/201142570 wait complete. 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 Processor -- start 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 -- start start 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98dc06b2e0 0x7f98dc14e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98dc15a630 0x7f98dc1f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98dc15aa10 0x7f98dc154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc14ece0 con 0x7f98dc06b2e0 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc14fb00 con 0x7f98dc15aa10 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e3a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc14eef0 con 0x7f98dc15a630 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98dc06b2e0 0x7f98dc14e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e1000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98dc06b2e0 0x7f98dc14e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39756/0 (socket says 172.21.15.66:39756) 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.684+0000 7f98e1000640 1 -- 172.21.15.66:0/3937438150 learned_addr learned my addr 172.21.15.66:0/3937438150 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98e1a00640 1 --2- 172.21.15.66:0/3937438150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98dc15aa10 0x7f98dc154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98e1000640 1 -- 172.21.15.66:0/3937438150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98dc15a630 msgr2=0x7f98dc1f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98dbe00640 1 --2- 172.21.15.66:0/3937438150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98dc15a630 0x7f98dc1f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98e1000640 1 --2- 172.21.15.66:0/3937438150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98dc15a630 0x7f98dc1f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98e1000640 1 -- 172.21.15.66:0/3937438150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98dc15aa10 msgr2=0x7f98dc154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98e1000640 1 --2- 172.21.15.66:0/3937438150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98dc15aa10 0x7f98dc154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98e1000640 1 -- 172.21.15.66:0/3937438150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c80094d0 con 0x7f98dc06b2e0 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98e1a00640 1 --2- 172.21.15.66:0/3937438150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98dc15aa10 0x7f98dc154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98dbe00640 1 --2- 172.21.15.66:0/3937438150 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98dc15a630 0x7f98dc1f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98e1000640 1 --2- 172.21.15.66:0/3937438150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98dc06b2e0 0x7f98dc14e7a0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f98d000e930 tx=0x7f98d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98d8c00640 1 -- 172.21.15.66:0/3937438150 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0009690 con 0x7f98dc06b2e0 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98d8c00640 1 -- 172.21.15.66:0/3937438150 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d000ca00 con 0x7f98dc06b2e0 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98e3a00640 1 -- 172.21.15.66:0/3937438150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc14f1d0 con 0x7f98dc06b2e0 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98d8c00640 1 -- 172.21.15.66:0/3937438150 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d000cd70 con 0x7f98dc06b2e0 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98e3a00640 1 -- 172.21.15.66:0/3937438150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc14f790 con 0x7f98dc06b2e0 2024-03-30T00:49:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98e3a00640 1 -- 172.21.15.66:0/3937438150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f98dc06c0b0 con 0x7f98dc06b2e0 2024-03-30T00:49:12.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98d8c00640 1 -- 172.21.15.66:0/3937438150 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f98d0013470 con 0x7f98dc06b2e0 2024-03-30T00:49:12.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.688+0000 7f98d8c00640 1 --2- 172.21.15.66:0/3937438150 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f98bc066690 0x7f98bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.692+0000 7f98d8c00640 1 -- 172.21.15.66:0/3937438150 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f98d0084990 con 0x7f98dc06b2e0 2024-03-30T00:49:12.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.692+0000 7f98d8c00640 1 -- 172.21.15.66:0/3937438150 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+511 (secure 0 0 0) 0x7f98d00852b0 con 0x7f98dc06b2e0 2024-03-30T00:49:12.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.692+0000 7f98dbe00640 1 --2- 172.21.15.66:0/3937438150 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f98bc066690 0x7f98bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.692+0000 7f98e3a00640 1 -- 172.21.15.66:0/3937438150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f98dc067340 con 0x7f98dc06b2e0 2024-03-30T00:49:12.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.692+0000 7f98dbe00640 1 --2- 172.21.15.66:0/3937438150 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f98bc066690 0x7f98bc068b50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f98c80ad000 tx=0x7f98c80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9b258a000 1 Processor -- start 2024-03-30T00:49:12.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9b258a000 1 -- start start 2024-03-30T00:49:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f70904f60 0x556f709542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094dd30 0x556f70904310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f70904850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9b258a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7040fcf0 con 0x556f7094e650 2024-03-30T00:49:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9b258a000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7071a230 con 0x556f70904f60 2024-03-30T00:49:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9b258a000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f70713090 con 0x556f7094dd30 2024-03-30T00:49:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9ad200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f70904850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9ad200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f70904850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39760/0 (socket says 172.21.15.66:39760) 2024-03-30T00:49:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9ad200640 1 -- 172.21.15.66:0/3098273720 learned_addr learned my addr 172.21.15.66:0/3098273720 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/3098273720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f70904f60 0x556f709542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff99f400640 1 --2- 172.21.15.66:0/3098273720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094dd30 0x556f70904310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9ad200640 1 -- 172.21.15.66:0/3098273720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094dd30 msgr2=0x556f70904310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9ad200640 1 --2- 172.21.15.66:0/3098273720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094dd30 0x556f70904310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9ad200640 1 -- 172.21.15.66:0/3098273720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f70904f60 msgr2=0x556f709542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9ad200640 1 --2- 172.21.15.66:0/3098273720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f70904f60 0x556f709542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9ad200640 1 -- 172.21.15.66:0/3098273720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f706fdef0 con 0x556f7094e650 2024-03-30T00:49:12.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/3098273720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f70904f60 0x556f709542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:12.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff9ad200640 1 --2- 172.21.15.66:0/3098273720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f70904850 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00517a0 tx=0x7ff9a0084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc83a8c0fd6ced0f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff99d600640 1 -- 172.21.15.66:0/3098273720 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a005d5e0 con 0x556f7094e650 2024-03-30T00:49:12.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff99d600640 1 -- 172.21.15.66:0/3098273720 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a006c050 con 0x556f7094e650 2024-03-30T00:49:12.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.696+0000 7ff99d600640 1 -- 172.21.15.66:0/3098273720 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0053640 con 0x556f7094e650 2024-03-30T00:49:12.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 -- 172.21.15.66:0/3098273720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 msgr2=0x556f70904850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 --2- 172.21.15.66:0/3098273720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f70904850 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00517a0 tx=0x7ff9a0084040 comp rx=0 tx=0).stop 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 -- 172.21.15.66:0/3098273720 shutdown_connections 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 --2- 172.21.15.66:0/3098273720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f70904850 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 --2- 172.21.15.66:0/3098273720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f7094dd30 0x556f70904310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 --2- 172.21.15.66:0/3098273720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f70904f60 0x556f709542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 -- 172.21.15.66:0/3098273720 >> 172.21.15.66:0/3098273720 conn(0x556f70900f10 msgr2=0x556f70909940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 -- 172.21.15.66:0/3098273720 shutdown_connections 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 -- 172.21.15.66:0/3098273720 wait complete. 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 -- start start 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 Processor -- start 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f70904f60 0x556f709ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f7094dd30 0x556f70952d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f709532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f706efd30 con 0x556f7094e650 2024-03-30T00:49:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f707ee3c0 con 0x556f7094dd30 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f70964610 con 0x556f70904f60 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 10 client.0 _ll_register_callbacks cb 0x556f707cc0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9ad200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f709532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9ad200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f709532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39766/0 (socket says 172.21.15.66:39766) 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9ad200640 1 -- 172.21.15.66:0/867930408 learned_addr learned my addr 172.21.15.66:0/867930408 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f70904f60 0x556f709ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff99f400640 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f7094dd30 0x556f70952d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9ad200640 1 -- 172.21.15.66:0/867930408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f70904f60 msgr2=0x556f709ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9ad200640 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f70904f60 0x556f709ac420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9ad200640 1 -- 172.21.15.66:0/867930408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f7094dd30 msgr2=0x556f70952d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9ad200640 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f7094dd30 0x556f70952d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9ad200640 1 -- 172.21.15.66:0/867930408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a806cb80 con 0x556f7094e650 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9ad200640 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f709532c0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00591e0 tx=0x7ff9a0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e003755e861b358 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff995600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff995600640 1 -- 172.21.15.66:0/867930408 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00536c0 con 0x556f7094e650 2024-03-30T00:49:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff995600640 1 -- 172.21.15.66:0/867930408 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff9a00519b0 con 0x556f7094e650 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff9b258a000 1 -- 172.21.15.66:0/867930408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f7070f5f0 con 0x556f7094e650 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.700+0000 7ff995600640 1 -- 172.21.15.66:0/867930408 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9a007c3a0 con 0x556f7094e650 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 10 client.9003 fetch_fsmap learned FSMap version 79 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 -- 172.21.15.66:0/867930408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x556f706f1a60 con 0x556f7094e650 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff995600640 1 -- 172.21.15.66:0/867930408 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f7070f5f0 con 0x556f7094e650 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff995600640 1 -- 172.21.15.66:0/867930408 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 79) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff9a0082020 con 0x556f7094e650 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 10 client.9003 fetch_fsmap finished waiting for FSMap version 79 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 10 client.9003 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 20 client.9003 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x556f70953800 0x556f70953be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 4 client.9003 mds_command: new command op to 8646 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 -- 172.21.15.66:0/867930408 --> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x556f70711410 con 0x556f70953800 2024-03-30T00:49:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x556f70953800 0x556f70953be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9a7e00640 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x556f70953800 0x556f70953be0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eef6927f408596 server_cookie=af331a5d4b46386e in_seq=0 out_seq=0 2024-03-30T00:49:12.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff995600640 10 client.9003 ms_handle_connect on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:12.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff995600640 1 -- 172.21.15.66:0/867930408 <== mds.0 v2:172.21.15.66:6835/2988412199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556f70711410 con 0x556f70953800 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff995600640 10 client.9003 handle_command_reply: tid=0 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 client.9003 shutdown 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 -- 172.21.15.66:0/867930408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 msgr2=0x556f709532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f709532c0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00591e0 tx=0x7ff9a0007f60 comp rx=0 tx=0).stop 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 -- 172.21.15.66:0/867930408 shutdown_connections 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f7094e650 0x556f709532c0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x556f70953800 0x556f70953be0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f7094dd30 0x556f70952d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 --2- 172.21.15.66:0/867930408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x556f70904f60 0x556f709ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 -- 172.21.15.66:0/867930408 >> 172.21.15.66:0/867930408 conn(0x556f70900f10 msgr2=0x556f70903300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff995600640 0 client.9003 ms_handle_reset on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 -- 172.21.15.66:0/867930408 shutdown_connections 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 1 -- 172.21.15.66:0/867930408 wait complete. 2024-03-30T00:49:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9b258a000 20 client.9003 trim_cache size 0 max 16384 2024-03-30T00:49:12.719 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:49:12.719 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T00:49:12.719 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "c8ba4f39-9dd1-498f-b8ba-4ecd0c759dfd", 2024-03-30T00:49:12.719 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T00:49:12.719 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 -- 172.21.15.66:0/212992296 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff984066690 msgr2=0x7ff984068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff984066690 0x7ff984068b50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff998006fd0 tx=0x7ff998008040 comp rx=0 tx=0).stop 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 -- 172.21.15.66:0/212992296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9a815de70 msgr2=0x7ff9a81642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9a815de70 0x7ff9a81642e0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c780 tx=0x7ff9a000cc40 comp rx=0 tx=0).stop 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 -- 172.21.15.66:0/212992296 shutdown_connections 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff984066690 0x7ff984068b50 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9a815de70 0x7ff9a81642e0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9a8099bf0 0x7ff9a806aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 --2- 172.21.15.66:0/212992296 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9a8099760 0x7ff9a80b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 -- 172.21.15.66:0/212992296 >> 172.21.15.66:0/212992296 conn(0x7ff9a80b2150 msgr2=0x7ff9a8159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 -- 172.21.15.66:0/212992296 shutdown_connections 2024-03-30T00:49:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.704+0000 7ff9af200640 1 -- 172.21.15.66:0/212992296 wait complete. 2024-03-30T00:49:12.735 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-30T00:49:12.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd759c00640 1 Processor -- start 2024-03-30T00:49:12.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd759c00640 1 -- start start 2024-03-30T00:49:12.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd759c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd7540cca40 0x7fd7540cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd759c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd754098ac0 con 0x7fd754099960 2024-03-30T00:49:12.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd759c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7540cf3f0 con 0x7fd75406b140 2024-03-30T00:49:12.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd759c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7540b16c0 con 0x7fd7540cca40 2024-03-30T00:49:12.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd753400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd754099960 0x7fd7540b0fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:48574/0 (socket says 172.21.15.66:48574) 2024-03-30T00:49:12.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd753400640 1 -- 172.21.15.66:0/2298133991 learned_addr learned my addr 172.21.15.66:0/2298133991 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:12.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd752a00640 1 --2- 172.21.15.66:0/2298133991 >> v2:172.21.15.156:3300/0 conn(0x7fd7540cca40 0x7fd7540cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd752000640 1 -- 172.21.15.66:0/2298133991 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526849182 0 0) 0x7fd754098ac0 con 0x7fd754099960 2024-03-30T00:49:12.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd752000640 1 -- 172.21.15.66:0/2298133991 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd738003530 con 0x7fd754099960 2024-03-30T00:49:12.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd752a00640 1 -- 172.21.15.66:0/2298133991 >> v1:172.21.15.156:6789/0 conn(0x7fd75406b140 legacy=0x7fd754099250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd752a00640 1 -- 172.21.15.66:0/2298133991 >> v1:172.21.15.66:6789/0 conn(0x7fd754099960 legacy=0x7fd7540b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.868+0000 7fd752a00640 1 -- 172.21.15.66:0/2298133991 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd754099d70 con 0x7fd7540cca40 2024-03-30T00:49:12.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.872+0000 7fd752a00640 1 --2- 172.21.15.66:0/2298133991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7540cca40 0x7fd7540cac90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd7440097c0 tx=0x7fd74401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8731c7f48db84065 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.872+0000 7fd752000640 1 -- 172.21.15.66:0/2298133991 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440ab670 con 0x7fd7540cca40 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.872+0000 7fd752000640 1 -- 172.21.15.66:0/2298133991 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7440ab7d0 con 0x7fd7540cca40 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.872+0000 7fd752000640 1 -- 172.21.15.66:0/2298133991 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440abb60 con 0x7fd7540cca40 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.872+0000 7fd759c00640 1 -- 172.21.15.66:0/2298133991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7540cca40 msgr2=0x7fd7540cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.872+0000 7fd759c00640 1 --2- 172.21.15.66:0/2298133991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7540cca40 0x7fd7540cac90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd7440097c0 tx=0x7fd74401c470 comp rx=0 tx=0).stop 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.872+0000 7fd759c00640 1 -- 172.21.15.66:0/2298133991 shutdown_connections 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.872+0000 7fd759c00640 1 --2- 172.21.15.66:0/2298133991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7540cca40 0x7fd7540cac90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.872+0000 7fd759c00640 1 -- 172.21.15.66:0/2298133991 >> 172.21.15.66:0/2298133991 conn(0x7fd7540b2130 msgr2=0x7fd7540b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.872+0000 7fd759c00640 1 -- 172.21.15.66:0/2298133991 shutdown_connections 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.872+0000 7fd759c00640 1 -- 172.21.15.66:0/2298133991 wait complete. 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd759c00640 1 Processor -- start 2024-03-30T00:49:12.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd759c00640 1 -- start start 2024-03-30T00:49:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd759c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd75406b140 0x7fd7540c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd759c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd754099960 0x7fd7540c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd759c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540cca40 0x7fd7540cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd759c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd754152660 con 0x7fd7540cca40 2024-03-30T00:49:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd759c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7540b6040 con 0x7fd754099960 2024-03-30T00:49:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd759c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75415d430 con 0x7fd75406b140 2024-03-30T00:49:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd753400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540cca40 0x7fd7540cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd753400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540cca40 0x7fd7540cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39774/0 (socket says 172.21.15.66:39774) 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd753400640 1 -- 172.21.15.66:0/918425790 learned_addr learned my addr 172.21.15.66:0/918425790 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd74a000640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd754099960 0x7fd7540c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd752a00640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd75406b140 0x7fd7540c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd753400640 1 -- 172.21.15.66:0/918425790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd75406b140 msgr2=0x7fd7540c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd753400640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd75406b140 0x7fd7540c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd753400640 1 -- 172.21.15.66:0/918425790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd754099960 msgr2=0x7fd7540c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd753400640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd754099960 0x7fd7540c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd753400640 1 -- 172.21.15.66:0/918425790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd744009420 con 0x7fd7540cca40 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd753400640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540cca40 0x7fd7540cd4b0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00c4d0 tx=0x7fd74c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd74b400640 1 -- 172.21.15.66:0/918425790 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c005cf0 con 0x7fd7540cca40 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd74b400640 1 -- 172.21.15.66:0/918425790 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74c005e50 con 0x7fd7540cca40 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd74b400640 1 -- 172.21.15.66:0/918425790 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c007270 con 0x7fd7540cca40 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd759c00640 1 -- 172.21.15.66:0/918425790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75415d710 con 0x7fd7540cca40 2024-03-30T00:49:12.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.876+0000 7fd759c00640 1 -- 172.21.15.66:0/918425790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75415dcd0 con 0x7fd7540cca40 2024-03-30T00:49:12.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd74b400640 1 -- 172.21.15.66:0/918425790 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd74c0073d0 con 0x7fd7540cca40 2024-03-30T00:49:12.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd74b400640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd730066690 0x7fd730068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd74b400640 1 -- 172.21.15.66:0/918425790 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd74c083490 con 0x7fd7540cca40 2024-03-30T00:49:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd752a00640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd730066690 0x7fd730068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd75d066000 1 Processor -- start 2024-03-30T00:49:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd75d066000 1 -- start start 2024-03-30T00:49:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd752a00640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd730066690 0x7fd730068b50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd7440af000 tx=0x7fd7440a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd75d066000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 0x558929c92e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd75d066000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c93390 0x558929c464c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd75d066000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 0x558929c46de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd75d066000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929b11630 con 0x558929c93390 2024-03-30T00:49:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd75d066000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929a9ebd0 con 0x558929c92a70 2024-03-30T00:49:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd75d066000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558929a9c1a0 con 0x558929c46a00 2024-03-30T00:49:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd74a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c93390 0x558929c464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd74a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c93390 0x558929c464c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39782/0 (socket says 172.21.15.66:39782) 2024-03-30T00:49:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd74a000640 1 -- 172.21.15.66:0/703649993 learned_addr learned my addr 172.21.15.66:0/703649993 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd753400640 1 --2- 172.21.15.66:0/703649993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 0x558929c46de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.880+0000 7fd752a00640 1 --2- 172.21.15.66:0/703649993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 0x558929c92e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd74a000640 1 -- 172.21.15.66:0/703649993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 msgr2=0x558929c46de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd74a000640 1 --2- 172.21.15.66:0/703649993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 0x558929c46de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd74a000640 1 -- 172.21.15.66:0/703649993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 msgr2=0x558929c92e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd74a000640 1 --2- 172.21.15.66:0/703649993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 0x558929c92e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd74a000640 1 -- 172.21.15.66:0/703649993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558929793e20 con 0x558929c93390 2024-03-30T00:49:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd74a000640 1 --2- 172.21.15.66:0/703649993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c93390 0x558929c464c0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd73c001c20 tx=0x7fd73c0020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ca00898aacd906 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd743400640 1 -- 172.21.15.66:0/703649993 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c003b90 con 0x558929c93390 2024-03-30T00:49:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd743400640 1 -- 172.21.15.66:0/703649993 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c003cf0 con 0x558929c93390 2024-03-30T00:49:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd743400640 1 -- 172.21.15.66:0/703649993 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c00ea00 con 0x558929c93390 2024-03-30T00:49:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 -- 172.21.15.66:0/703649993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c93390 msgr2=0x558929c464c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 --2- 172.21.15.66:0/703649993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c93390 0x558929c464c0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd73c001c20 tx=0x7fd73c0020e0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 -- 172.21.15.66:0/703649993 shutdown_connections 2024-03-30T00:49:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 --2- 172.21.15.66:0/703649993 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 0x558929c46de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 --2- 172.21.15.66:0/703649993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c93390 0x558929c464c0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 --2- 172.21.15.66:0/703649993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 0x558929c92e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 -- 172.21.15.66:0/703649993 >> 172.21.15.66:0/703649993 conn(0x558929a47160 msgr2=0x558929c75820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 -- 172.21.15.66:0/703649993 shutdown_connections 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 -- 172.21.15.66:0/703649993 wait complete. 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 -- start start 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 Processor -- start 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c46a00 0x558929c4f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c92a70 0x558929c76a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c93390 0x558929c76f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929a97a90 con 0x558929c92a70 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929a36d40 con 0x558929c46a00 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558929a54f30 con 0x558929c93390 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd75d066000 10 client.0 _ll_register_callbacks cb 0x558929af9810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd74a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c92a70 0x558929c76a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd74a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c92a70 0x558929c76a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39786/0 (socket says 172.21.15.66:39786) 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd74a000640 1 -- 172.21.15.66:0/2495563335 learned_addr learned my addr 172.21.15.66:0/2495563335 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd752a00640 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c46a00 0x558929c4f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.884+0000 7fd753400640 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c93390 0x558929c76f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd74a000640 1 -- 172.21.15.66:0/2495563335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c93390 msgr2=0x558929c76f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd74a000640 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c93390 0x558929c76f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd74a000640 1 -- 172.21.15.66:0/2495563335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c46a00 msgr2=0x558929c4f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd74a000640 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c46a00 0x558929c4f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd74a000640 1 -- 172.21.15.66:0/2495563335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c00f040 con 0x558929c92a70 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd752a00640 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c46a00 0x558929c4f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd74a000640 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c92a70 0x558929c76a50 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x558929a9ff40 tx=0x7fd73c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e1da65300e7ebf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd737e00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd737e00640 1 -- 172.21.15.66:0/2495563335 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0014d0 con 0x558929c92a70 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd737e00640 1 -- 172.21.15.66:0/2495563335 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd73c00ea10 con 0x558929c92a70 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 -- 172.21.15.66:0/2495563335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558929a963e0 con 0x558929c92a70 2024-03-30T00:49:12.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd737e00640 1 -- 172.21.15.66:0/2495563335 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd73c00ebd0 con 0x558929c92a70 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 10 client.9015 fetch_fsmap learned FSMap version 79 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 -- 172.21.15.66:0/2495563335 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x558929a5caa0 con 0x558929c92a70 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd737e00640 1 -- 172.21.15.66:0/2495563335 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558929a963e0 con 0x558929c92a70 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd737e00640 1 -- 172.21.15.66:0/2495563335 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 79) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd73c00ed30 con 0x558929c92a70 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 10 client.9015 fetch_fsmap finished waiting for FSMap version 79 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 10 client.9015 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 20 client.9015 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x558929c9a2d0 0x558929c9a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 4 client.9015 mds_command: new command op to 8646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 -- 172.21.15.66:0/2495563335 --> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558929a67d00 con 0x558929c9a2d0 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd752a00640 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x558929c9a2d0 0x558929c9a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd752a00640 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x558929c9a2d0 0x558929c9a6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1fc2aa99cbc68f7 server_cookie=cfcfcabde6d33a93 in_seq=0 out_seq=0 2024-03-30T00:49:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd737e00640 10 client.9015 ms_handle_connect on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd737e00640 1 -- 172.21.15.66:0/2495563335 <== mds.0 v2:172.21.15.66:6835/2988412199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558929a67d00 con 0x558929c9a2d0 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd737e00640 10 client.9015 handle_command_reply: tid=0 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 client.9015 shutdown 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 -- 172.21.15.66:0/2495563335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c92a70 msgr2=0x558929c76a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c92a70 0x558929c76a50 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x558929a9ff40 tx=0x7fd73c001260 comp rx=0 tx=0).stop 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 -- 172.21.15.66:0/2495563335 shutdown_connections 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c93390 0x558929c76f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929c92a70 0x558929c76a50 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x558929c9a2d0 0x558929c9a6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 --2- 172.21.15.66:0/2495563335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c46a00 0x558929c4f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 -- 172.21.15.66:0/2495563335 >> 172.21.15.66:0/2495563335 conn(0x558929a47160 msgr2=0x558929c45bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd737e00640 0 client.9015 ms_handle_reset on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 -- 172.21.15.66:0/2495563335 shutdown_connections 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 1 -- 172.21.15.66:0/2495563335 wait complete. 2024-03-30T00:49:12.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.888+0000 7fd75d066000 20 client.9015 trim_cache size 0 max 16384 2024-03-30T00:49:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd75d066000 1 Processor -- start 2024-03-30T00:49:12.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd75d066000 1 -- start start 2024-03-30T00:49:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd75d066000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929c45c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd75d066000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 0x558929c46170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd75d066000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 0x558929c46de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd75d066000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929754aa0 con 0x558929a47160 2024-03-30T00:49:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd75d066000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929aef920 con 0x558929c92a70 2024-03-30T00:49:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd75d066000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558929a93280 con 0x558929c46a00 2024-03-30T00:49:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd752a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929c45c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd752a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929c45c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39798/0 (socket says 172.21.15.66:39798) 2024-03-30T00:49:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd752a00640 1 -- 172.21.15.66:0/3197878248 learned_addr learned my addr 172.21.15.66:0/3197878248 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd752a00640 1 -- 172.21.15.66:0/3197878248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 msgr2=0x558929c46170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd752a00640 1 --2- 172.21.15.66:0/3197878248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 0x558929c46170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd752a00640 1 -- 172.21.15.66:0/3197878248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 msgr2=0x558929c46de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd752a00640 1 --2- 172.21.15.66:0/3197878248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 0x558929c46de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.904+0000 7fd752a00640 1 -- 172.21.15.66:0/3197878248 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558929a42df0 con 0x558929a47160 2024-03-30T00:49:12.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd752a00640 1 --2- 172.21.15.66:0/3197878248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929c45c30 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fd7440ad040 tx=0x7fd744004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ea7e065c4920fc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd743e00640 1 -- 172.21.15.66:0/3197878248 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440c6040 con 0x558929a47160 2024-03-30T00:49:12.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd743e00640 1 -- 172.21.15.66:0/3197878248 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7440c7040 con 0x558929a47160 2024-03-30T00:49:12.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd743e00640 1 -- 172.21.15.66:0/3197878248 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440cf4d0 con 0x558929a47160 2024-03-30T00:49:12.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 -- 172.21.15.66:0/3197878248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 msgr2=0x558929c45c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 --2- 172.21.15.66:0/3197878248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929c45c30 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fd7440ad040 tx=0x7fd744004740 comp rx=0 tx=0).stop 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 -- 172.21.15.66:0/3197878248 shutdown_connections 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 --2- 172.21.15.66:0/3197878248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 0x558929c46de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 --2- 172.21.15.66:0/3197878248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 0x558929c46170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 --2- 172.21.15.66:0/3197878248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929c45c30 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 -- 172.21.15.66:0/3197878248 >> 172.21.15.66:0/3197878248 conn(0x558929c76dc0 msgr2=0x558929c4e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 -- 172.21.15.66:0/3197878248 shutdown_connections 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 -- 172.21.15.66:0/3197878248 wait complete. 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 -- start start 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 Processor -- start 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929cf2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 0x558929c49ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 0x558929c4a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929c4f860 con 0x558929a47160 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929a94e90 con 0x558929c92a70 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558929c76c80 con 0x558929c46a00 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 10 client.0 _ll_register_callbacks cb 0x7fd73c001690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd752a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929cf2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd752a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929cf2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39804/0 (socket says 172.21.15.66:39804) 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd752a00640 1 -- 172.21.15.66:0/3979855232 learned_addr learned my addr 172.21.15.66:0/3979855232 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd752a00640 1 -- 172.21.15.66:0/3979855232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 msgr2=0x558929c49ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd752a00640 1 --2- 172.21.15.66:0/3979855232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 0x558929c49ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd752a00640 1 -- 172.21.15.66:0/3979855232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 msgr2=0x558929c4a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd752a00640 1 --2- 172.21.15.66:0/3979855232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 0x558929c4a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd752a00640 1 -- 172.21.15.66:0/3979855232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7440bb070 con 0x558929a47160 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd752a00640 1 --2- 172.21.15.66:0/3979855232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929cf2710 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fd74401c950 tx=0x7fd74401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11035fab24ee7b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:12.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd737e00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd737e00640 1 -- 172.21.15.66:0/3979855232 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440c6070 con 0x558929a47160 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd737e00640 1 -- 172.21.15.66:0/3979855232 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd744005e20 con 0x558929a47160 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.908+0000 7fd75d066000 1 -- 172.21.15.66:0/3979855232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558929a54970 con 0x558929a47160 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd737e00640 1 -- 172.21.15.66:0/3979855232 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7440cf2d0 con 0x558929a47160 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 10 client.9021 fetch_fsmap learned FSMap version 79 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 -- 172.21.15.66:0/3979855232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x558929a36960 con 0x558929a47160 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd737e00640 1 -- 172.21.15.66:0/3979855232 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558929a54970 con 0x558929a47160 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd737e00640 1 -- 172.21.15.66:0/3979855232 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 79) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd7440cf430 con 0x558929a47160 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 10 client.9021 fetch_fsmap finished waiting for FSMap version 79 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 10 client.9021 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 20 client.9021 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 --2- 172.21.15.66:0/3979855232 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x558929c93390 0x558929cae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:12.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 4 client.9021 mds_command: new command op to 8646 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 -- 172.21.15.66:0/3979855232 --> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558929ac6a20 con 0x558929c93390 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd74a000640 1 --2- 172.21.15.66:0/3979855232 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x558929c93390 0x558929cae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd74a000640 1 --2- 172.21.15.66:0/3979855232 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x558929c93390 0x558929cae1e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b0ef26ad8777d9 server_cookie=31c8ef10b66b42a2 in_seq=0 out_seq=0 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd737e00640 10 client.9021 ms_handle_connect on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd737e00640 1 -- 172.21.15.66:0/3979855232 <== mds.0 v2:172.21.15.66:6835/2988412199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x558929ac6a20 con 0x558929c93390 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd737e00640 10 client.9021 handle_command_reply: tid=0 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 client.9021 shutdown 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 -- 172.21.15.66:0/3979855232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 msgr2=0x558929cf2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 --2- 172.21.15.66:0/3979855232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929cf2710 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fd74401c950 tx=0x7fd74401acb0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 -- 172.21.15.66:0/3979855232 shutdown_connections 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 --2- 172.21.15.66:0/3979855232 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x558929c93390 0x558929cae1e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 --2- 172.21.15.66:0/3979855232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558929c92a70 0x558929c4a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 --2- 172.21.15.66:0/3979855232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558929c46a00 0x558929c49ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 --2- 172.21.15.66:0/3979855232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558929a47160 0x558929cf2710 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd737e00640 0 client.9021 ms_handle_reset on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 -- 172.21.15.66:0/3979855232 >> 172.21.15.66:0/3979855232 conn(0x558929c76dc0 msgr2=0x558929c44c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 -- 172.21.15.66:0/3979855232 shutdown_connections 2024-03-30T00:49:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 1 -- 172.21.15.66:0/3979855232 wait complete. 2024-03-30T00:49:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd75d066000 20 client.9021 trim_cache size 0 max 16384 2024-03-30T00:49:12.927 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:49:12.927 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:49:12.927 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-03-30T00:49:12.927 INFO:teuthology.orchestra.run.smithi066.stdout: "c8ba4f39-9dd1-498f-b8ba-4ecd0c759dfd": { 2024-03-30T00:49:12.927 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T00:49:12.928 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "c8ba4f39-9dd1-498f-b8ba-4ecd0c759dfd", 2024-03-30T00:49:12.928 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,repair" 2024-03-30T00:49:12.928 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:49:12.928 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:49:12.928 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 -- 172.21.15.66:0/918425790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd730066690 msgr2=0x7fd730068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd730066690 0x7fd730068b50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd7440af000 tx=0x7fd7440a7000 comp rx=0 tx=0).stop 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 -- 172.21.15.66:0/918425790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540cca40 msgr2=0x7fd7540cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540cca40 0x7fd7540cd4b0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00c4d0 tx=0x7fd74c00c990 comp rx=0 tx=0).stop 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 -- 172.21.15.66:0/918425790 shutdown_connections 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd730066690 0x7fd730068b50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540cca40 0x7fd7540cd4b0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd754099960 0x7fd7540c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 --2- 172.21.15.66:0/918425790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd75406b140 0x7fd7540c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 -- 172.21.15.66:0/918425790 >> 172.21.15.66:0/918425790 conn(0x7fd7540b2130 msgr2=0x7fd75415c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 -- 172.21.15.66:0/918425790 shutdown_connections 2024-03-30T00:49:12.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:12.912+0000 7fd759c00640 1 -- 172.21.15.66:0/918425790 wait complete. 2024-03-30T00:49:12.943 INFO:tasks.cephfs.filesystem:scrub status for tag:c8ba4f39-9dd1-498f-b8ba-4ecd0c759dfd - {'path': '/', 'tag': 'c8ba4f39-9dd1-498f-b8ba-4ecd0c759dfd', 'options': 'recursive,repair'} 2024-03-30T00:49:13.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:13.660+0000 7f98d8c00640 1 -- 172.21.15.66:0/3937438150 <== mon.0 v2:172.21.15.66:3300/0 7 ==== log(4 entries from seq 1 at 2024-03-30T00:49:12.709820+0000) v1 ==== 695+0+0 (secure 0 0 0) 0x7f98d0055fa0 con 0x7f98dc06b2e0 2024-03-30T00:49:13.672 INFO:teuthology.orchestra.run.smithi066.stdout: cluster: 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: id: 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: health: HEALTH_OK 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: services: 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: mgr: y(active, since 9m), standbys: x 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: mds: 1/1 daemons up, 3 standby 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: data: 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: volumes: 1/1 healthy 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: pools: 3 pools, 97 pgs 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: objects: 30 objects, 19 MiB 2024-03-30T00:49:13.673 INFO:teuthology.orchestra.run.smithi066.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-03-30T00:49:13.674 INFO:teuthology.orchestra.run.smithi066.stdout: pgs: 97 active+clean 2024-03-30T00:49:13.674 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:13.674 INFO:teuthology.orchestra.run.smithi066.stdout: io: 2024-03-30T00:49:13.674 INFO:teuthology.orchestra.run.smithi066.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 2 op/s wr 2024-03-30T00:49:13.674 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:13.674 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:13.674 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:12.709820+0000 mds.c [INF] scrub queued for path: / 2024-03-30T00:49:13.674 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:12.709823+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-30T00:49:13.674 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:12.709827+0000 mds.c [INF] scrub summary: active paths [/] 2024-03-30T00:49:13.674 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:12.711162+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-30T00:49:17.423 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:49:17.427 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:49:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:17.672+0000 7f98d8c00640 1 -- 172.21.15.66:0/3937438150 <== mon.0 v2:172.21.15.66:3300/0 8 ==== log(1 entries from seq 5 at 2024-03-30T00:49:16.890588+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f98d00596d0 con 0x7f98dc06b2e0 2024-03-30T00:49:17.683 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:16.890588+0000 mds.c [INF] scrub summary: idle 2024-03-30T00:49:33.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:33.704+0000 7f98d8c00640 1 -- 172.21.15.66:0/3937438150 <== mon.0 v2:172.21.15.66:3300/0 9 ==== log(1 entries from seq 1228 at 2024-03-30T00:49:33.605321+0000) v1 ==== 317+0+0 (secure 0 0 0) 0x7f98d005dc90 con 0x7f98dc06b2e0 2024-03-30T00:49:34.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:34.712+0000 7f98d8c00640 1 -- 172.21.15.66:0/3937438150 <== mon.0 v2:172.21.15.66:3300/0 10 ==== log(1 entries from seq 1229 at 2024-03-30T00:49:33.714716+0000) v1 ==== 319+0+0 (secure 0 0 0) 0x7f98d0085000 con 0x7f98dc06b2e0 2024-03-30T00:49:42.944 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-30T00:49:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.092+0000 7f394aa00640 1 Processor -- start 2024-03-30T00:49:43.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.092+0000 7f394aa00640 1 -- start start 2024-03-30T00:49:43.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.092+0000 7f394aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f39440ce520 0x7f39440cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.092+0000 7f394aa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39440ca660 con 0x7f39440ce980 2024-03-30T00:49:43.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.092+0000 7f394aa00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39440ca940 con 0x7f39440cd2d0 2024-03-30T00:49:43.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.092+0000 7f394aa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39440cbec0 con 0x7f39440ce520 2024-03-30T00:49:43.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.096+0000 7f393b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f39440ce520 0x7f39440cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.096+0000 7f3948a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f39440ce980 0x7f39440cad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56798/0 (socket says 172.21.15.66:56798) 2024-03-30T00:49:43.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.096+0000 7f3948a00640 1 -- 172.21.15.66:0/3975684533 learned_addr learned my addr 172.21.15.66:0/3975684533 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:43.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.096+0000 7f393b400640 1 -- 172.21.15.66:0/3975684533 >> v1:172.21.15.156:6789/0 conn(0x7f39440cd2d0 legacy=0x7f39440c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.096+0000 7f393b400640 1 -- 172.21.15.66:0/3975684533 >> v1:172.21.15.66:6789/0 conn(0x7f39440ce980 legacy=0x7f39440cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.096+0000 7f393b400640 1 -- 172.21.15.66:0/3975684533 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39440cc0d0 con 0x7f39440ce520 2024-03-30T00:49:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.096+0000 7f3943400640 1 -- 172.21.15.66:0/3975684533 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236805884 0 0) 0x7f39440ca660 con 0x7f39440ce980 2024-03-30T00:49:43.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.096+0000 7f393b400640 1 --2- 172.21.15.66:0/3975684533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39440ce520 0x7f39440cc8d0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f392c009d90 tx=0x7f392c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dab065b37b2200a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.096+0000 7f3943400640 1 -- 172.21.15.66:0/3975684533 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c0a5a20 con 0x7f39440ce520 2024-03-30T00:49:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.096+0000 7f3943400640 1 -- 172.21.15.66:0/3975684533 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392c0a5b80 con 0x7f39440ce520 2024-03-30T00:49:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.096+0000 7f3943400640 1 -- 172.21.15.66:0/3975684533 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c0ad440 con 0x7f39440ce520 2024-03-30T00:49:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 -- 172.21.15.66:0/3975684533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39440ce520 msgr2=0x7f39440cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 --2- 172.21.15.66:0/3975684533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39440ce520 0x7f39440cc8d0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f392c009d90 tx=0x7f392c01c940 comp rx=0 tx=0).stop 2024-03-30T00:49:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 -- 172.21.15.66:0/3975684533 shutdown_connections 2024-03-30T00:49:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 --2- 172.21.15.66:0/3975684533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39440ce520 0x7f39440cc8d0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 -- 172.21.15.66:0/3975684533 >> 172.21.15.66:0/3975684533 conn(0x7f3944069f60 msgr2=0x7f394406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 -- 172.21.15.66:0/3975684533 shutdown_connections 2024-03-30T00:49:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 -- 172.21.15.66:0/3975684533 wait complete. 2024-03-30T00:49:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 Processor -- start 2024-03-30T00:49:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 -- start start 2024-03-30T00:49:43.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39440cd2d0 0x7f3944097960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440ce520 0x7f3944155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39440ce980 0x7f3944157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944071f30 con 0x7f39440ce980 2024-03-30T00:49:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944097ea0 con 0x7f39440ce520 2024-03-30T00:49:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f394aa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39441542b0 con 0x7f39440cd2d0 2024-03-30T00:49:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f3948a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39440ce980 0x7f3944157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f3948a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39440ce980 0x7f3944157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39762/0 (socket says 172.21.15.66:39762) 2024-03-30T00:49:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f3948a00640 1 -- 172.21.15.66:0/3083810889 learned_addr learned my addr 172.21.15.66:0/3083810889 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f3943e00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39440cd2d0 0x7f3944097960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f393b400640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440ce520 0x7f3944155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f3948a00640 1 -- 172.21.15.66:0/3083810889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39440cd2d0 msgr2=0x7f3944097960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f3948a00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39440cd2d0 0x7f3944097960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f3948a00640 1 -- 172.21.15.66:0/3083810889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440ce520 msgr2=0x7f3944155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f3948a00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440ce520 0x7f3944155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.100+0000 7f3948a00640 1 -- 172.21.15.66:0/3083810889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c009a70 con 0x7f39440ce980 2024-03-30T00:49:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f3948a00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39440ce980 0x7f3944157a60 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f393c00c410 tx=0x7f393c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f3940c00640 1 -- 172.21.15.66:0/3083810889 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c0074e0 con 0x7f39440ce980 2024-03-30T00:49:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f3940c00640 1 -- 172.21.15.66:0/3083810889 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c007660 con 0x7f39440ce980 2024-03-30T00:49:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f394aa00640 1 -- 172.21.15.66:0/3083810889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3944154590 con 0x7f39440ce980 2024-03-30T00:49:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f3940c00640 1 -- 172.21.15.66:0/3083810889 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c004a30 con 0x7f39440ce980 2024-03-30T00:49:43.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f394aa00640 1 -- 172.21.15.66:0/3083810889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3944154b50 con 0x7f39440ce980 2024-03-30T00:49:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f3940c00640 1 -- 172.21.15.66:0/3083810889 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f393c004c50 con 0x7f39440ce980 2024-03-30T00:49:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f3940c00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3920066690 0x7f3920068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f3940c00640 1 -- 172.21.15.66:0/3083810889 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f393c084110 con 0x7f39440ce980 2024-03-30T00:49:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f3943e00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3920066690 0x7f3920068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f394de71000 1 Processor -- start 2024-03-30T00:49:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.104+0000 7f394de71000 1 -- start start 2024-03-30T00:49:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3943e00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3920066690 0x7f3920068b50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3934000f70 tx=0x7f3934009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623f2390 0x5647623a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623a5a00 0x5647623a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762270630 con 0x5647623f1a70 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647621fdbd0 con 0x5647623a5a00 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647621fb1a0 con 0x5647623f2390 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3943e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3943e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623f1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39768/0 (socket says 172.21.15.66:39768) 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3943e00640 1 -- 172.21.15.66:0/3189459801 learned_addr learned my addr 172.21.15.66:0/3189459801 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f393b400640 1 --2- 172.21.15.66:0/3189459801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623f2390 0x5647623a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3948a00640 1 --2- 172.21.15.66:0/3189459801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623a5a00 0x5647623a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3943e00640 1 -- 172.21.15.66:0/3189459801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623f2390 msgr2=0x5647623a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3943e00640 1 --2- 172.21.15.66:0/3189459801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623f2390 0x5647623a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3943e00640 1 -- 172.21.15.66:0/3189459801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623a5a00 msgr2=0x5647623a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3943e00640 1 --2- 172.21.15.66:0/3189459801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623a5a00 0x5647623a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3943e00640 1 -- 172.21.15.66:0/3189459801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564761ef2e20 con 0x5647623f1a70 2024-03-30T00:49:43.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3943e00640 1 --2- 172.21.15.66:0/3189459801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623f1e50 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f3934001560 tx=0x7f3934011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bac8401e79569c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3938c00640 1 -- 172.21.15.66:0/3189459801 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934003600 con 0x5647623f1a70 2024-03-30T00:49:43.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3938c00640 1 -- 172.21.15.66:0/3189459801 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3934003760 con 0x5647623f1a70 2024-03-30T00:49:43.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f3938c00640 1 -- 172.21.15.66:0/3189459801 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934003ad0 con 0x5647623f1a70 2024-03-30T00:49:43.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 -- 172.21.15.66:0/3189459801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 msgr2=0x5647623f1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 --2- 172.21.15.66:0/3189459801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623f1e50 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f3934001560 tx=0x7f3934011000 comp rx=0 tx=0).stop 2024-03-30T00:49:43.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 -- 172.21.15.66:0/3189459801 shutdown_connections 2024-03-30T00:49:43.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 --2- 172.21.15.66:0/3189459801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623a5a00 0x5647623a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 --2- 172.21.15.66:0/3189459801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623f2390 0x5647623a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 --2- 172.21.15.66:0/3189459801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623f1e50 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 -- 172.21.15.66:0/3189459801 >> 172.21.15.66:0/3189459801 conn(0x5647621a6160 msgr2=0x5647623d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 -- 172.21.15.66:0/3189459801 shutdown_connections 2024-03-30T00:49:43.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 -- 172.21.15.66:0/3189459801 wait complete. 2024-03-30T00:49:43.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 -- start start 2024-03-30T00:49:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.108+0000 7f394de71000 1 Processor -- start 2024-03-30T00:49:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623a5a00 0x5647623adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f2390 0x5647623d5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647621f6a90 con 0x5647623f1a70 2024-03-30T00:49:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564762195d40 con 0x5647623f2390 2024-03-30T00:49:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647621b3f30 con 0x5647623a5a00 2024-03-30T00:49:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 10 client.0 _ll_register_callbacks cb 0x564762258810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f393b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f393b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623d5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39770/0 (socket says 172.21.15.66:39770) 2024-03-30T00:49:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f393b400640 1 -- 172.21.15.66:0/2113632979 learned_addr learned my addr 172.21.15.66:0/2113632979 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3943e00640 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623a5a00 0x5647623adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3948a00640 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f2390 0x5647623d5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f393b400640 1 -- 172.21.15.66:0/2113632979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623a5a00 msgr2=0x5647623adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f393b400640 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623a5a00 0x5647623adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f393b400640 1 -- 172.21.15.66:0/2113632979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f2390 msgr2=0x5647623d5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f393b400640 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f2390 0x5647623d5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f393b400640 1 -- 172.21.15.66:0/2113632979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393400f040 con 0x5647623f1a70 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f393b400640 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623d5a50 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f392c01c980 tx=0x7f392c01b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e212ed878908b9b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3931600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3931600640 1 -- 172.21.15.66:0/2113632979 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c003e10 con 0x5647623f1a70 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3931600640 1 -- 172.21.15.66:0/2113632979 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f392c004040 con 0x5647623f1a70 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 1 -- 172.21.15.66:0/2113632979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647621f53e0 con 0x5647623f1a70 2024-03-30T00:49:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3931600640 1 -- 172.21.15.66:0/2113632979 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f392c004200 con 0x5647623f1a70 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 10 client.9036 fetch_fsmap learned FSMap version 79 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 1 -- 172.21.15.66:0/2113632979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5647621bbaa0 con 0x5647623f1a70 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3931600640 1 -- 172.21.15.66:0/2113632979 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5647621f53e0 con 0x5647623f1a70 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3931600640 1 -- 172.21.15.66:0/2113632979 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 79) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f392c004360 con 0x5647623f1a70 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 10 client.9036 fetch_fsmap finished waiting for FSMap version 79 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 10 client.9036 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 20 client.9036 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5647623f8eb0 0x5647623f9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 4 client.9036 mds_command: new command op to 8646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 1 -- 172.21.15.66:0/2113632979 --> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647621c6d00 con 0x5647623f8eb0 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3943e00640 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5647623f8eb0 0x5647623f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3943e00640 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5647623f8eb0 0x5647623f9290 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5afefcf8c954809 server_cookie=8f130b9e9a75d59e in_seq=0 out_seq=0 2024-03-30T00:49:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3931600640 10 client.9036 ms_handle_connect on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3931600640 1 -- 172.21.15.66:0/2113632979 <== mds.0 v2:172.21.15.66:6835/2988412199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647621c6d00 con 0x5647623f8eb0 2024-03-30T00:49:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f3931600640 10 client.9036 handle_command_reply: tid=0 2024-03-30T00:49:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.112+0000 7f394de71000 1 client.9036 shutdown 2024-03-30T00:49:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f394de71000 1 -- 172.21.15.66:0/2113632979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 msgr2=0x5647623d5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f394de71000 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623d5a50 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f392c01c980 tx=0x7f392c01b180 comp rx=0 tx=0).stop 2024-03-30T00:49:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f394de71000 1 -- 172.21.15.66:0/2113632979 shutdown_connections 2024-03-30T00:49:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f394de71000 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5647623f8eb0 0x5647623f9290 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f394de71000 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f2390 0x5647623d5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f394de71000 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f1a70 0x5647623d5a50 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f394de71000 1 --2- 172.21.15.66:0/2113632979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623a5a00 0x5647623adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f3931600640 0 client.9036 ms_handle_reset on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:43.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f394de71000 1 -- 172.21.15.66:0/2113632979 >> 172.21.15.66:0/2113632979 conn(0x5647621a6160 msgr2=0x5647623a4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f394de71000 1 -- 172.21.15.66:0/2113632979 shutdown_connections 2024-03-30T00:49:43.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f394de71000 1 -- 172.21.15.66:0/2113632979 wait complete. 2024-03-30T00:49:43.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.116+0000 7f394de71000 20 client.9036 trim_cache size 0 max 16384 2024-03-30T00:49:43.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.128+0000 7f394de71000 1 Processor -- start 2024-03-30T00:49:43.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.128+0000 7f394de71000 1 -- start start 2024-03-30T00:49:43.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.128+0000 7f394de71000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623a5a00 0x5647623ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.128+0000 7f394de71000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f1a70 0x5647623acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.128+0000 7f394de71000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f2390 0x5647623a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.128+0000 7f394de71000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564761eb3aa0 con 0x5647623f2390 2024-03-30T00:49:43.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.128+0000 7f394de71000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647621b7c60 con 0x5647623f1a70 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.128+0000 7f394de71000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647623d5c80 con 0x5647623a5a00 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3948a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f2390 0x5647623a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3948a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f2390 0x5647623a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39780/0 (socket says 172.21.15.66:39780) 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3948a00640 1 -- 172.21.15.66:0/3839022921 learned_addr learned my addr 172.21.15.66:0/3839022921 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f393b400640 1 --2- 172.21.15.66:0/3839022921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f1a70 0x5647623acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3943e00640 1 --2- 172.21.15.66:0/3839022921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623a5a00 0x5647623ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3948a00640 1 -- 172.21.15.66:0/3839022921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623a5a00 msgr2=0x5647623ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3948a00640 1 --2- 172.21.15.66:0/3839022921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623a5a00 0x5647623ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3948a00640 1 -- 172.21.15.66:0/3839022921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f1a70 msgr2=0x5647623acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3948a00640 1 --2- 172.21.15.66:0/3839022921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f1a70 0x5647623acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3948a00640 1 -- 172.21.15.66:0/3839022921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647621a1df0 con 0x5647623f2390 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3948a00640 1 --2- 172.21.15.66:0/3839022921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f2390 0x5647623a5de0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f393c054ed0 tx=0x7f393c084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e129e829880f67f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3939600640 1 -- 172.21.15.66:0/3839022921 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c054f40 con 0x5647623f2390 2024-03-30T00:49:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3939600640 1 -- 172.21.15.66:0/3839022921 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c058d70 con 0x5647623f2390 2024-03-30T00:49:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3939600640 1 -- 172.21.15.66:0/3839022921 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c05d330 con 0x5647623f2390 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 -- 172.21.15.66:0/3839022921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f2390 msgr2=0x5647623a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 --2- 172.21.15.66:0/3839022921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f2390 0x5647623a5de0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f393c054ed0 tx=0x7f393c084040 comp rx=0 tx=0).stop 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 -- 172.21.15.66:0/3839022921 shutdown_connections 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 --2- 172.21.15.66:0/3839022921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623f2390 0x5647623a5de0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 --2- 172.21.15.66:0/3839022921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f1a70 0x5647623acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 --2- 172.21.15.66:0/3839022921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623a5a00 0x5647623ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 -- 172.21.15.66:0/3839022921 >> 172.21.15.66:0/3839022921 conn(0x5647621a6160 msgr2=0x5647623a44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 -- 172.21.15.66:0/3839022921 shutdown_connections 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 -- 172.21.15.66:0/3839022921 wait complete. 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 -- start start 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 Processor -- start 2024-03-30T00:49:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623a5a00 0x5647623a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623f1a70 0x56476244d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f2390 0x56476244dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647621c0640 con 0x5647623a5a00 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56476223a170 con 0x5647623f2390 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647623adb20 con 0x5647623f1a70 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f394de71000 10 client.0 _ll_register_callbacks cb 0x7f3934001d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3943e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623a5a00 0x5647623a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3943e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623a5a00 0x5647623a8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39794/0 (socket says 172.21.15.66:39794) 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3943e00640 1 -- 172.21.15.66:0/1867251272 learned_addr learned my addr 172.21.15.66:0/1867251272 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f393b400640 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623f1a70 0x56476244d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.132+0000 7f3948a00640 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f2390 0x56476244dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3943e00640 1 -- 172.21.15.66:0/1867251272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623f1a70 msgr2=0x56476244d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3943e00640 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623f1a70 0x56476244d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3943e00640 1 -- 172.21.15.66:0/1867251272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f2390 msgr2=0x56476244dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3943e00640 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f2390 0x56476244dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3943e00640 1 -- 172.21.15.66:0/1867251272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3944154b50 con 0x5647623a5a00 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3943e00640 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623a5a00 0x5647623a8e50 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x5647621fef40 tx=0x7f3934000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4487336b7e662c77 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3931600640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3931600640 1 -- 172.21.15.66:0/1867251272 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934003600 con 0x5647623a5a00 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3931600640 1 -- 172.21.15.66:0/1867251272 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f393402e910 con 0x5647623a5a00 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 -- 172.21.15.66:0/1867251272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647621b3970 con 0x5647623a5a00 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3931600640 1 -- 172.21.15.66:0/1867251272 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f393402ead0 con 0x5647623a5a00 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 10 client.9042 fetch_fsmap learned FSMap version 79 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 -- 172.21.15.66:0/1867251272 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x564762195960 con 0x5647623a5a00 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3931600640 1 -- 172.21.15.66:0/1867251272 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5647621b3970 con 0x5647623a5a00 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3931600640 1 -- 172.21.15.66:0/1867251272 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 79) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f393402ec30 con 0x5647623a5a00 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 10 client.9042 fetch_fsmap finished waiting for FSMap version 79 2024-03-30T00:49:43.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 10 client.9042 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:49:43.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 20 client.9042 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:43.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5647623a9390 0x56476244e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 4 client.9042 mds_command: new command op to 8646 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:49:43.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 -- 172.21.15.66:0/1867251272 --> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564762225a20 con 0x5647623a9390 2024-03-30T00:49:43.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f393b400640 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5647623a9390 0x56476244e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f393b400640 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5647623a9390 0x56476244e1f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=580af01a0c08166c server_cookie=cfdf4b745a2bd2e4 in_seq=0 out_seq=0 2024-03-30T00:49:43.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3931600640 10 client.9042 ms_handle_connect on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:43.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3931600640 1 -- 172.21.15.66:0/1867251272 <== mds.0 v2:172.21.15.66:6835/2988412199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564762225a20 con 0x5647623a9390 2024-03-30T00:49:43.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3931600640 10 client.9042 handle_command_reply: tid=0 2024-03-30T00:49:43.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 client.9042 shutdown 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 -- 172.21.15.66:0/1867251272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623a5a00 msgr2=0x5647623a8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623a5a00 0x5647623a8e50 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x5647621fef40 tx=0x7f3934000ee0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 -- 172.21.15.66:0/1867251272 shutdown_connections 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.66:6835/2988412199,v1:172.21.15.66:6837/2988412199] conn(0x5647623a9390 0x56476244e1f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647623f2390 0x56476244dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5647623f1a70 0x56476244d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 --2- 172.21.15.66:0/1867251272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647623a5a00 0x5647623a8e50 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f3931600640 0 client.9042 ms_handle_reset on v2:172.21.15.66:6835/2988412199 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 -- 172.21.15.66:0/1867251272 >> 172.21.15.66:0/1867251272 conn(0x5647621a6160 msgr2=0x5647623a4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 -- 172.21.15.66:0/1867251272 shutdown_connections 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 1 -- 172.21.15.66:0/1867251272 wait complete. 2024-03-30T00:49:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394de71000 20 client.9042 trim_cache size 0 max 16384 2024-03-30T00:49:43.149 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:49:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T00:49:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-03-30T00:49:43.149 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 -- 172.21.15.66:0/3083810889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3920066690 msgr2=0x7f3920068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3920066690 0x7f3920068b50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3934000f70 tx=0x7f3934009040 comp rx=0 tx=0).stop 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 -- 172.21.15.66:0/3083810889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39440ce980 msgr2=0x7f3944157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39440ce980 0x7f3944157a60 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f393c00c410 tx=0x7f393c00c810 comp rx=0 tx=0).stop 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 -- 172.21.15.66:0/3083810889 shutdown_connections 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3920066690 0x7f3920068b50 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39440ce980 0x7f3944157a60 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440ce520 0x7f3944155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 --2- 172.21.15.66:0/3083810889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39440cd2d0 0x7f3944097960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 -- 172.21.15.66:0/3083810889 >> 172.21.15.66:0/3083810889 conn(0x7f3944069f60 msgr2=0x7f39440af130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 -- 172.21.15.66:0/3083810889 shutdown_connections 2024-03-30T00:49:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.136+0000 7f394aa00640 1 -- 172.21.15.66:0/3083810889 wait complete. 2024-03-30T00:49:43.163 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:49:43.264 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:49:43.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe61ee00640 1 Processor -- start 2024-03-30T00:49:43.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe61ee00640 1 -- start start 2024-03-30T00:49:43.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe61ee00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6180cda90 0x7fe6180ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe61ee00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe6180cde70 0x7fe6180cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe61ee00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61815a990 con 0x7fe618159c60 2024-03-30T00:49:43.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe61ee00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6180b1b80 con 0x7fe6180cda90 2024-03-30T00:49:43.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe61ee00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6180cd3e0 con 0x7fe6180cde70 2024-03-30T00:49:43.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe617e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6180cda90 0x7fe6180ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe617e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6180cda90 0x7fe6180ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39796/0 (socket says 172.21.15.66:39796) 2024-03-30T00:49:43.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe617e00640 1 -- 172.21.15.66:0/1337037653 learned_addr learned my addr 172.21.15.66:0/1337037653 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe60f400640 1 --2- 172.21.15.66:0/1337037653 >> v2:172.21.15.156:3300/0 conn(0x7fe6180cde70 0x7fe6180cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe617e00640 1 -- 172.21.15.66:0/1337037653 >> v2:172.21.15.156:3300/0 conn(0x7fe6180cde70 msgr2=0x7fe6180cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe617e00640 1 --2- 172.21.15.66:0/1337037653 >> v2:172.21.15.156:3300/0 conn(0x7fe6180cde70 0x7fe6180cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe617e00640 1 -- 172.21.15.66:0/1337037653 >> v1:172.21.15.156:6789/0 conn(0x7fe618159c60 legacy=0x7fe61815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.400+0000 7fe617e00640 1 -- 172.21.15.66:0/1337037653 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6180b11e0 con 0x7fe6180cda90 2024-03-30T00:49:43.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe617e00640 1 --2- 172.21.15.66:0/1337037653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6180cda90 0x7fe6180ce520 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fe6180cc320 tx=0x7fe60801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3503873a21387e71 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe617400640 1 -- 172.21.15.66:0/1337037653 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080ac600 con 0x7fe6180cda90 2024-03-30T00:49:43.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe617400640 1 -- 172.21.15.66:0/1337037653 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6080a8050 con 0x7fe6180cda90 2024-03-30T00:49:43.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe617400640 1 -- 172.21.15.66:0/1337037653 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080ac980 con 0x7fe6180cda90 2024-03-30T00:49:43.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe61ee00640 1 -- 172.21.15.66:0/1337037653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6180cda90 msgr2=0x7fe6180ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe61ee00640 1 --2- 172.21.15.66:0/1337037653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6180cda90 0x7fe6180ce520 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fe6180cc320 tx=0x7fe60801c490 comp rx=0 tx=0).stop 2024-03-30T00:49:43.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe61ee00640 1 -- 172.21.15.66:0/1337037653 shutdown_connections 2024-03-30T00:49:43.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe61ee00640 1 --2- 172.21.15.66:0/1337037653 >> v2:172.21.15.156:3300/0 conn(0x7fe6180cde70 0x7fe6180cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe61ee00640 1 --2- 172.21.15.66:0/1337037653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6180cda90 0x7fe6180ce520 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe61ee00640 1 -- 172.21.15.66:0/1337037653 >> 172.21.15.66:0/1337037653 conn(0x7fe618069f60 msgr2=0x7fe61806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe61ee00640 1 -- 172.21.15.66:0/1337037653 shutdown_connections 2024-03-30T00:49:43.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.404+0000 7fe61ee00640 1 -- 172.21.15.66:0/1337037653 wait complete. 2024-03-30T00:49:43.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ee00640 1 Processor -- start 2024-03-30T00:49:43.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ee00640 1 -- start start 2024-03-30T00:49:43.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6180cda90 0x7fe61815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6180cde70 0x7fe6180c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ee00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe618159c60 0x7fe618158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618096cf0 con 0x7fe6180cde70 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ee00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180c5dd0 con 0x7fe618159c60 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe618095f30 con 0x7fe6180cda90 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe60f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6180cde70 0x7fe6180c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe60f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6180cde70 0x7fe6180c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39800/0 (socket says 172.21.15.66:39800) 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe60f400640 1 -- 172.21.15.66:0/2679793742 learned_addr learned my addr 172.21.15.66:0/2679793742 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe617e00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6180cda90 0x7fe61815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ce00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe618159c60 0x7fe618158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe60f400640 1 -- 172.21.15.66:0/2679793742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6180cda90 msgr2=0x7fe61815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe60f400640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6180cda90 0x7fe61815aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe60f400640 1 -- 172.21.15.66:0/2679793742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe618159c60 msgr2=0x7fe618158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe60f400640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe618159c60 0x7fe618158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe60f400640 1 -- 172.21.15.66:0/2679793742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6080094d0 con 0x7fe6180cde70 2024-03-30T00:49:43.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe617e00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6180cda90 0x7fe61815aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:43.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe60f400640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6180cde70 0x7fe6180c53c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fe60000e900 tx=0x7fe60000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe614c00640 1 -- 172.21.15.66:0/2679793742 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600009690 con 0x7fe6180cde70 2024-03-30T00:49:43.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe614c00640 1 -- 172.21.15.66:0/2679793742 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60000ca00 con 0x7fe6180cde70 2024-03-30T00:49:43.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe614c00640 1 -- 172.21.15.66:0/2679793742 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60000cdb0 con 0x7fe6180cde70 2024-03-30T00:49:43.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ee00640 1 -- 172.21.15.66:0/2679793742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe618096210 con 0x7fe6180cde70 2024-03-30T00:49:43.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ee00640 1 -- 172.21.15.66:0/2679793742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6180967d0 con 0x7fe6180cde70 2024-03-30T00:49:43.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.408+0000 7fe61ee00640 1 -- 172.21.15.66:0/2679793742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe618066c30 con 0x7fe6180cde70 2024-03-30T00:49:43.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.412+0000 7fe614c00640 1 -- 172.21.15.66:0/2679793742 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe600013470 con 0x7fe6180cde70 2024-03-30T00:49:43.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.412+0000 7fe614c00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 0x7fe5f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.412+0000 7fe614c00640 1 -- 172.21.15.66:0/2679793742 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe600084610 con 0x7fe6180cde70 2024-03-30T00:49:43.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.412+0000 7fe617e00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 0x7fe5f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.412+0000 7fe617e00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 0x7fe5f4068b50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe6080ae000 tx=0x7fe608008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.416+0000 7fe614c00640 1 -- 172.21.15.66:0/2679793742 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe600051780 con 0x7fe6180cde70 2024-03-30T00:49:43.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.608+0000 7fe61ee00640 1 -- 172.21.15.66:0/2679793742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe6180b3170 con 0x7fe6180cde70 2024-03-30T00:49:43.618 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:49:43.612+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:49:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe614c00640 1 -- 172.21.15.66:0/2679793742 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe600054da0 con 0x7fe6180cde70 2024-03-30T00:49:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:49:43.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 -- 172.21.15.66:0/2679793742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 msgr2=0x7fe5f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 0x7fe5f4068b50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe6080ae000 tx=0x7fe608008040 comp rx=0 tx=0).stop 2024-03-30T00:49:43.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 -- 172.21.15.66:0/2679793742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6180cde70 msgr2=0x7fe6180c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6180cde70 0x7fe6180c53c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fe60000e900 tx=0x7fe60000edc0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 -- 172.21.15.66:0/2679793742 shutdown_connections 2024-03-30T00:49:43.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe618159c60 0x7fe618158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe5f4066690 0x7fe5f4068b50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6180cde70 0x7fe6180c53c0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 --2- 172.21.15.66:0/2679793742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6180cda90 0x7fe61815aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 -- 172.21.15.66:0/2679793742 >> 172.21.15.66:0/2679793742 conn(0x7fe618069f60 msgr2=0x7fe6180af580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 -- 172.21.15.66:0/2679793742 shutdown_connections 2024-03-30T00:49:43.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.620+0000 7fe61ee00640 1 -- 172.21.15.66:0/2679793742 wait complete. 2024-03-30T00:49:43.642 INFO:tasks.ceph.mds.c.smithi066.stderr: -6010> 2024-03-30T00:48:47.721+0000 7f4912e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:49:43.656 DEBUG:teuthology.orchestra.run.smithi066:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-03-30T00:49:43.698 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:49:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.696+0000 7fe286b07400 1 -- start start 2024-03-30T00:49:43.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 1 Processor -- start 2024-03-30T00:49:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c84fd8f800 0x55c84fe1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c84fd8f400 0x55c84fe1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c84fd8f000 0x55c84fe1b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84fe5c780 con 0x55c84fd8f800 2024-03-30T00:49:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84fe5ca00 con 0x55c84fd8f400 2024-03-30T00:49:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c84fe5c640 con 0x55c84fd8f000 2024-03-30T00:49:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe284c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c84fd8f800 0x55c84fe1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe284c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c84fd8f800 0x55c84fe1a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39834/0 (socket says 172.21.15.66:39834) 2024-03-30T00:49:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe284c00640 1 -- 172.21.15.66:0/680765458 learned_addr learned my addr 172.21.15.66:0/680765458 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe285600640 1 --2- 172.21.15.66:0/680765458 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c84fd8f000 0x55c84fe1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe284200640 1 --2- 172.21.15.66:0/680765458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c84fd8f400 0x55c84fe1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe284c00640 1 -- 172.21.15.66:0/680765458 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c84fd8f000 msgr2=0x55c84fe1b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe284c00640 1 --2- 172.21.15.66:0/680765458 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c84fd8f000 0x55c84fe1b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe284c00640 1 -- 172.21.15.66:0/680765458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c84fd8f400 msgr2=0x55c84fe1ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe284c00640 1 --2- 172.21.15.66:0/680765458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c84fd8f400 0x55c84fe1ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe284c00640 1 -- 172.21.15.66:0/680765458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c84fdc49c0 con 0x55c84fd8f800 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe285600640 1 --2- 172.21.15.66:0/680765458 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c84fd8f000 0x55c84fe1b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe284c00640 1 --2- 172.21.15.66:0/680765458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c84fd8f800 0x55c84fe1a580 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x55c850c0acc0 tx=0x55c850c9c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad704d9d9a30c71f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe281a00640 1 -- 172.21.15.66:0/680765458 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c84fde3600 con 0x55c84fd8f800 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe281a00640 1 -- 172.21.15.66:0/680765458 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c84fdec480 con 0x55c84fd8f800 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe281a00640 1 -- 172.21.15.66:0/680765458 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c84fde31e0 con 0x55c84fd8f800 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 1 -- 172.21.15.66:0/680765458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c84fdc4b60 con 0x55c84fd8f800 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 1 -- 172.21.15.66:0/680765458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c84fdc4340 con 0x55c84fd8f800 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 4 waiting for MDS map... 2024-03-30T00:49:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe281a00640 1 -- 172.21.15.66:0/680765458 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x55c84fdcec40 con 0x55c84fd8f800 2024-03-30T00:49:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe281a00640 1 -- 172.21.15.66:0/680765458 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 80) v1 ==== 2573+0+0 (secure 0 0 0) 0x55c84fddcf00 con 0x55c84fd8f800 2024-03-30T00:49:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 4 Got MDS map 80 2024-03-30T00:49:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 10 main: JournalTool::main 2024-03-30T00:49:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 4 main: JournalTool: connecting to RADOS... 2024-03-30T00:49:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 1 Processor -- start 2024-03-30T00:49:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.700+0000 7fe286b07400 1 -- start start 2024-03-30T00:49:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c850caac00 0x55c84fe1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c850cab000 0x55c84fe1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 0x55c84fe1c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84fdd5e00 con 0x55c850cab400 2024-03-30T00:49:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84fe5ca00 con 0x55c850caac00 2024-03-30T00:49:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c84fe5c500 con 0x55c850cab000 2024-03-30T00:49:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 0x55c84fe1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 0x55c84fe1c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39850/0 (socket says 172.21.15.66:39850) 2024-03-30T00:49:43.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe285600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c850caac00 0x55c84fe1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 -- 172.21.15.66:0/1546464646 learned_addr learned my addr 172.21.15.66:0/1546464646 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284c00640 1 --2- 172.21.15.66:0/1546464646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c850cab000 0x55c84fe1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 -- 172.21.15.66:0/1546464646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c850cab000 msgr2=0x55c84fe1bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 --2- 172.21.15.66:0/1546464646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c850cab000 0x55c84fe1bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 -- 172.21.15.66:0/1546464646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c850caac00 msgr2=0x55c84fe1b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 --2- 172.21.15.66:0/1546464646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c850caac00 0x55c84fe1b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 -- 172.21.15.66:0/1546464646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c84fdc4000 con 0x55c850cab400 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 --2- 172.21.15.66:0/1546464646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 0x55c84fe1c100 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x55c850072030 tx=0x55c850c9c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e826f5e50b1b0376 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe27f200640 1 -- 172.21.15.66:0/1546464646 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c84fde3080 con 0x55c850cab400 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe27f200640 1 -- 172.21.15.66:0/1546464646 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c850b66600 con 0x55c850cab400 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe27f200640 1 -- 172.21.15.66:0/1546464646 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c84fde3600 con 0x55c850cab400 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- 172.21.15.66:0/1546464646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 msgr2=0x55c84fe1c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 --2- 172.21.15.66:0/1546464646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 0x55c84fe1c100 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x55c850072030 tx=0x55c850c9c240 comp rx=0 tx=0).stop 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- 172.21.15.66:0/1546464646 shutdown_connections 2024-03-30T00:49:43.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 --2- 172.21.15.66:0/1546464646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 0x55c84fe1c100 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 --2- 172.21.15.66:0/1546464646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c850cab000 0x55c84fe1bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 --2- 172.21.15.66:0/1546464646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c850caac00 0x55c84fe1b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- 172.21.15.66:0/1546464646 >> 172.21.15.66:0/1546464646 conn(0x55c84fd8ec00 msgr2=0x55c84fdbd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- 172.21.15.66:0/1546464646 shutdown_connections 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- 172.21.15.66:0/1546464646 wait complete. 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 Processor -- start 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- start start 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c850caac00 0x55c84fe1d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c850cab000 0x55c84fe1cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 0x55c84fe1c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84fe5c3c0 con 0x55c850cab400 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84fe5c280 con 0x55c850caac00 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe286b07400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c84fe5c140 con 0x55c850cab000 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 0x55c84fe1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 0x55c84fe1c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39864/0 (socket says 172.21.15.66:39864) 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284200640 1 -- 172.21.15.66:0/2847630745 learned_addr learned my addr 172.21.15.66:0/2847630745 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe285600640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c850caac00 0x55c84fe1d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.704+0000 7fe284c00640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c850cab000 0x55c84fe1cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe284200640 1 -- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c850cab000 msgr2=0x55c84fe1cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe284200640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c850cab000 0x55c84fe1cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe284200640 1 -- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c850caac00 msgr2=0x55c84fe1d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe284200640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c850caac00 0x55c84fe1d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe284200640 1 -- 172.21.15.66:0/2847630745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c84fdc41a0 con 0x55c850cab400 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe284200640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 0x55c84fe1c680 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x55c850072360 tx=0x55c850c9c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe27d400640 1 -- 172.21.15.66:0/2847630745 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c84fde31e0 con 0x55c850cab400 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe27d400640 1 -- 172.21.15.66:0/2847630745 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c850b66900 con 0x55c850cab400 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe27d400640 1 -- 172.21.15.66:0/2847630745 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c84fde2dc0 con 0x55c850cab400 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c84fdc4000 con 0x55c850cab400 2024-03-30T00:49:43.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c84fdc49c0 con 0x55c850cab400 2024-03-30T00:49:43.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe286b07400 4 main: JournalTool: resolving pool 15 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe27d400640 1 -- 172.21.15.66:0/2847630745 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x55c84fe5b080 con 0x55c850cab400 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe27d400640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55c85015a000 0x55c84fe1d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe27d400640 1 -- 172.21.15.66:0/2847630745 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x55c84fdcea80 con 0x55c850cab400 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe286b07400 4 main: JournalTool: creating IoCtx.. 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe286b07400 4 main: Executing for rank 0 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe286b07400 4 Filtering by inode '1099511627777' 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe286b07400 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55c850cab800 0x55c84fe1c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v8 -- 0x55c850cabc00 con 0x55c850cab800 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe285600640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55c85015a000 0x55c84fe1d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe284c00640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55c850cab800 0x55c84fe1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.708+0000 7fe285600640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55c85015a000 0x55c84fe1d700 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x55c850b4a180 tx=0x55c850ca1ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.712+0000 7fe284c00640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55c850cab800 0x55c84fe1c100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.712+0000 7fe284c00640 1 -- 172.21.15.66:0/2847630745 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55c850c92b40 con 0x55c850cab800 2024-03-30T00:49:43.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.712+0000 7fe286b07400 4 Pointer 400.00000000 is readable 2024-03-30T00:49:43.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.712+0000 7fe286b07400 4 JournalScanner::scan: reading header object '200.00000000' 2024-03-30T00:49:43.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.712+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v8 -- 0x55c85015b000 con 0x55c850cab800 2024-03-30T00:49:43.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.712+0000 7fe284c00640 1 -- 172.21.15.66:0/2847630745 <== osd.6 v2:172.21.15.156:6816/27642 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv29 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55c850c92b40 con 0x55c850cab800 2024-03-30T00:49:43.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.712+0000 7fe286b07400 10 Header 0x400000 0x405ee2 0x40625e 2024-03-30T00:49:43.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.712+0000 7fe286b07400 10 Starting journal scan from offset 0x405ee2 2024-03-30T00:49:43.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.712+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24290~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v8 -- 0x55c85015b400 con 0x55c850cab800 2024-03-30T00:49:43.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe284c00640 1 -- 172.21.15.66:0/2847630745 <== osd.6 v2:172.21.15.156:6816/27642 3 ==== osd_op_reply(3 200.00000001 [read 24290~892 out=892b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x55c850c92b40 con 0x55c850cab800 2024-03-30T00:49:43.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-03-30T00:49:43.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 10 Parsing data, 0x37c bytes available 2024-03-30T00:49:43.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 10 Attempting decode at 0x405ee2 2024-03-30T00:49:43.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 10 Consumed 0x37c bytes 2024-03-30T00:49:43.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 10 Valid entry at 0x405ee2 2024-03-30T00:49:43.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55c85015b800 0x55c84fe1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v8 -- 0x55c85015bc00 con 0x55c85015b800 2024-03-30T00:49:43.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe285600640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55c85015b800 0x55c84fe1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe285600640 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55c85015b800 0x55c84fe1bb80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe285600640 1 -- 172.21.15.66:0/2847630745 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55c850c92d80 con 0x55c85015b800 2024-03-30T00:49:43.724 INFO:teuthology.orchestra.run.smithi066.stdout:Events by type: 2024-03-30T00:49:43.724 INFO:teuthology.orchestra.run.smithi066.stdout:Errors: 0 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 4 Reached end of journal objects 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 4 Scanned objects, 0 missing, 1 valid 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 4 Events scanned, 0 gaps 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 4 Found 1 valid events 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 4 Selected 0 events events for processing 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 4 0 events 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55c850cab800 msgr2=0x55c84fe1c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x55c850cab800 0x55c84fe1c100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55c85015b800 msgr2=0x55c84fe1bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x55c85015b800 0x55c84fe1bb80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55c85015a000 msgr2=0x55c84fe1d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55c85015a000 0x55c84fe1d700 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x55c850b4a180 tx=0x55c850ca1ce0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 msgr2=0x55c84fe1c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 --2- 172.21.15.66:0/2847630745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c850cab400 0x55c84fe1c680 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x55c850072360 tx=0x55c850c9c180 comp rx=0 tx=0).stop 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe284200640 1 -- 172.21.15.66:0/2847630745 reap_dead start 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe284200640 1 -- 172.21.15.66:0/2847630745 reap_dead start 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 shutdown_connections 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 >> 172.21.15.66:0/2847630745 conn(0x55c84fd8ec00 msgr2=0x55c84fdbd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 shutdown_connections 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/2847630745 wait complete. 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/680765458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c84fd8f800 msgr2=0x55c84fe1a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 --2- 172.21.15.66:0/680765458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c84fd8f800 0x55c84fe1a580 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x55c850c0acc0 tx=0x55c850c9c420 comp rx=0 tx=0).stop 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/680765458 shutdown_connections 2024-03-30T00:49:43.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 --2- 172.21.15.66:0/680765458 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c84fd8f000 0x55c84fe1b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 --2- 172.21.15.66:0/680765458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c84fd8f400 0x55c84fe1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 --2- 172.21.15.66:0/680765458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c84fd8f800 0x55c84fe1a580 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/680765458 >> 172.21.15.66:0/680765458 conn(0x55c84fd8fc00 msgr2=0x55c84fdbd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/680765458 shutdown_connections 2024-03-30T00:49:43.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.716+0000 7fe286b07400 1 -- 172.21.15.66:0/680765458 wait complete. 2024-03-30T00:49:43.729 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.072965, result: Events by type: Errors: 0 2024-03-30T00:49:43.729 DEBUG:teuthology.orchestra.run.smithi066:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-03-30T00:49:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.816+0000 7f4a17a4a400 1 -- start start 2024-03-30T00:49:43.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.816+0000 7f4a17a4a400 1 Processor -- start 2024-03-30T00:49:43.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.816+0000 7f4a17a4a400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b567b800 0x5653b5706580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.816+0000 7f4a17a4a400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b567b400 0x5653b5706b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.816+0000 7f4a17a4a400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b567b000 0x5653b5707080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.816+0000 7f4a17a4a400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b56c1e00 con 0x5653b567b000 2024-03-30T00:49:43.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.816+0000 7f4a17a4a400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b5748a00 con 0x5653b567b400 2024-03-30T00:49:43.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.816+0000 7f4a17a4a400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653b5748640 con 0x5653b567b800 2024-03-30T00:49:43.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a15a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b567b000 0x5653b5707080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a15a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b567b000 0x5653b5707080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39880/0 (socket says 172.21.15.66:39880) 2024-03-30T00:49:43.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a15a00640 1 -- 172.21.15.66:0/3478578910 learned_addr learned my addr 172.21.15.66:0/3478578910 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a14600640 1 --2- 172.21.15.66:0/3478578910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b567b400 0x5653b5706b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a15000640 1 --2- 172.21.15.66:0/3478578910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b567b800 0x5653b5706580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a15a00640 1 -- 172.21.15.66:0/3478578910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b567b800 msgr2=0x5653b5706580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a15a00640 1 --2- 172.21.15.66:0/3478578910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b567b800 0x5653b5706580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a15a00640 1 -- 172.21.15.66:0/3478578910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b567b400 msgr2=0x5653b5706b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a15a00640 1 --2- 172.21.15.66:0/3478578910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b567b400 0x5653b5706b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a15a00640 1 -- 172.21.15.66:0/3478578910 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653b56b0b60 con 0x5653b567b000 2024-03-30T00:49:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a15a00640 1 --2- 172.21.15.66:0/3478578910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b567b000 0x5653b5707080 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x5653b64e8d80 tx=0x5653b6588240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbbbb8db5416357 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a11e00640 1 -- 172.21.15.66:0/3478578910 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653b56cf600 con 0x5653b567b000 2024-03-30T00:49:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a11e00640 1 -- 172.21.15.66:0/3478578910 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5653b56d8900 con 0x5653b567b000 2024-03-30T00:49:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a11e00640 1 -- 172.21.15.66:0/3478578910 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653b56cf1e0 con 0x5653b567b000 2024-03-30T00:49:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 1 -- 172.21.15.66:0/3478578910 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653b56b0340 con 0x5653b567b000 2024-03-30T00:49:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 1 -- 172.21.15.66:0/3478578910 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5653b56b0000 con 0x5653b567b000 2024-03-30T00:49:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 4 waiting for MDS map... 2024-03-30T00:49:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a11e00640 1 -- 172.21.15.66:0/3478578910 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x5653b56bac40 con 0x5653b567b000 2024-03-30T00:49:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a11e00640 1 -- 172.21.15.66:0/3478578910 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 80) v1 ==== 2573+0+0 (secure 0 0 0) 0x5653b56c8f00 con 0x5653b567b000 2024-03-30T00:49:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 4 Got MDS map 80 2024-03-30T00:49:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 10 main: JournalTool::main 2024-03-30T00:49:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 4 main: JournalTool: connecting to RADOS... 2024-03-30T00:49:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 1 Processor -- start 2024-03-30T00:49:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 1 -- start start 2024-03-30T00:49:43.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b6448c00 0x5653b5707600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b6449000 0x5653b5707b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6449400 0x5653b5708100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b5748500 con 0x5653b6449400 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b57483c0 con 0x5653b6448c00 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a17a4a400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653b5748a00 con 0x5653b6449000 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a14600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6449400 0x5653b5708100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a14600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6449400 0x5653b5708100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39888/0 (socket says 172.21.15.66:39888) 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a14600640 1 -- 172.21.15.66:0/1631526971 learned_addr learned my addr 172.21.15.66:0/1631526971 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a14600640 1 --2- 172.21.15.66:0/1631526971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b6448c00 0x5653b5707600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.820+0000 7f4a15000640 1 --2- 172.21.15.66:0/1631526971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b6449000 0x5653b5707b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 -- 172.21.15.66:0/1631526971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b6449000 msgr2=0x5653b5707b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 --2- 172.21.15.66:0/1631526971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b6449000 0x5653b5707b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 -- 172.21.15.66:0/1631526971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b6448c00 msgr2=0x5653b5707600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 --2- 172.21.15.66:0/1631526971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b6448c00 0x5653b5707600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 -- 172.21.15.66:0/1631526971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653b56b0ea0 con 0x5653b6449400 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 --2- 172.21.15.66:0/1631526971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6449400 0x5653b5708100 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x5653b6604030 tx=0x5653b65accc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67009c3716494ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a0f600640 1 -- 172.21.15.66:0/1631526971 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653b56cf600 con 0x5653b6449400 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a0f600640 1 -- 172.21.15.66:0/1631526971 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5653b56d8900 con 0x5653b6449400 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a0f600640 1 -- 172.21.15.66:0/1631526971 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653b56cf1e0 con 0x5653b6449400 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 -- 172.21.15.66:0/1631526971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6449400 msgr2=0x5653b5708100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/1631526971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6449400 0x5653b5708100 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x5653b6604030 tx=0x5653b65accc0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 -- 172.21.15.66:0/1631526971 shutdown_connections 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/1631526971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6449400 0x5653b5708100 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/1631526971 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b6449000 0x5653b5707b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/1631526971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b6448c00 0x5653b5707600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 -- 172.21.15.66:0/1631526971 >> 172.21.15.66:0/1631526971 conn(0x5653b567ac00 msgr2=0x5653b56a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 -- 172.21.15.66:0/1631526971 shutdown_connections 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 -- 172.21.15.66:0/1631526971 wait complete. 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 Processor -- start 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 -- start start 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6448c00 0x5653b5709180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b6449000 0x5653b5708c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b6449400 0x5653b5708680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b5748280 con 0x5653b6448c00 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b5748140 con 0x5653b6449000 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653b5748000 con 0x5653b6449400 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6448c00 0x5653b5709180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6448c00 0x5653b5709180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39898/0 (socket says 172.21.15.66:39898) 2024-03-30T00:49:43.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 -- 172.21.15.66:0/2798395342 learned_addr learned my addr 172.21.15.66:0/2798395342 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a15000640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b6449000 0x5653b5708c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b6449400 0x5653b5708680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 -- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b6449400 msgr2=0x5653b5708680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b6449400 0x5653b5708680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 -- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b6449000 msgr2=0x5653b5708c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b6449000 0x5653b5708c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 -- 172.21.15.66:0/2798395342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653b56b0b60 con 0x5653b6448c00 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a14600640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6448c00 0x5653b5709180 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x5653b6604cf0 tx=0x5653b65acc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a0d800640 1 -- 172.21.15.66:0/2798395342 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653b56cf600 con 0x5653b6448c00 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a0d800640 1 -- 172.21.15.66:0/2798395342 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5653b644b500 con 0x5653b6448c00 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a0d800640 1 -- 172.21.15.66:0/2798395342 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5653b56cf1e0 con 0x5653b6448c00 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5653b56b0340 con 0x5653b6448c00 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653b56b0000 con 0x5653b6448c00 2024-03-30T00:49:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.824+0000 7f4a17a4a400 4 main: JournalTool: resolving pool 15 2024-03-30T00:49:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a0d800640 1 -- 172.21.15.66:0/2798395342 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x5653b5746840 con 0x5653b6448c00 2024-03-30T00:49:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a0d800640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5653b5966c00 0x5653b5709700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a0d800640 1 -- 172.21.15.66:0/2798395342 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x5653b56baa80 con 0x5653b6448c00 2024-03-30T00:49:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a17a4a400 4 main: JournalTool: creating IoCtx.. 2024-03-30T00:49:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a17a4a400 4 main: Executing for rank 0 2024-03-30T00:49:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a17a4a400 4 Filtering by inode '1099511627778' 2024-03-30T00:49:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5653b6449800 0x5653b5708100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a15000640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5653b5966c00 0x5653b5709700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v8 -- 0x5653b6449c00 con 0x5653b6449800 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a14600640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5653b6449800 0x5653b5708100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a15000640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5653b5966c00 0x5653b5709700 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x5653b56b3a10 tx=0x5653b63fbf20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a14600640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5653b6449800 0x5653b5708100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a14600640 1 -- 172.21.15.66:0/2798395342 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x5653b657ab40 con 0x5653b6449800 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a17a4a400 4 Pointer 400.00000000 is readable 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a17a4a400 4 JournalScanner::scan: reading header object '200.00000000' 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v8 -- 0x5653b5967000 con 0x5653b6449800 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a14600640 1 -- 172.21.15.66:0/2798395342 <== osd.6 v2:172.21.15.156:6816/27642 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv29 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x5653b657ab40 con 0x5653b6449800 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a17a4a400 10 Header 0x400000 0x405ee2 0x40625e 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a17a4a400 10 Starting journal scan from offset 0x405ee2 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.828+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24290~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v8 -- 0x5653b5967400 con 0x5653b6449800 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a14600640 1 -- 172.21.15.66:0/2798395342 <== osd.6 v2:172.21.15.156:6816/27642 3 ==== osd_op_reply(3 200.00000001 [read 24290~892 out=892b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x5653b657ab40 con 0x5653b6449800 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-03-30T00:49:43.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 10 Parsing data, 0x37c bytes available 2024-03-30T00:49:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 10 Attempting decode at 0x405ee2 2024-03-30T00:49:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 10 Consumed 0x37c bytes 2024-03-30T00:49:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 10 Valid entry at 0x405ee2 2024-03-30T00:49:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x5653b5967800 0x5653b5707b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v8 -- 0x5653b5967c00 con 0x5653b5967800 2024-03-30T00:49:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a15a00640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x5653b5967800 0x5653b5707b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a15a00640 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x5653b5967800 0x5653b5707b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a15a00640 1 -- 172.21.15.66:0/2798395342 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5653b657ad80 con 0x5653b5967800 2024-03-30T00:49:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 4 Reached end of journal objects 2024-03-30T00:49:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 4 Scanned objects, 0 missing, 1 valid 2024-03-30T00:49:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 4 Events scanned, 0 gaps 2024-03-30T00:49:43.843 INFO:teuthology.orchestra.run.smithi066.stdout:Events by type: 2024-03-30T00:49:43.843 INFO:teuthology.orchestra.run.smithi066.stdout:Errors: 0 2024-03-30T00:49:43.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 4 Found 1 valid events 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 4 Selected 0 events events for processing 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 4 0 events 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5653b6449800 msgr2=0x5653b5708100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5653b6449800 0x5653b5708100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x5653b5967800 msgr2=0x5653b5707b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x5653b5967800 0x5653b5707b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5653b5966c00 msgr2=0x5653b5709700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5653b5966c00 0x5653b5709700 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x5653b56b3a10 tx=0x5653b63fbf20 comp rx=0 tx=0).stop 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6448c00 msgr2=0x5653b5709180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/2798395342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b6448c00 0x5653b5709180 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x5653b6604cf0 tx=0x5653b65acc00 comp rx=0 tx=0).stop 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a15000640 1 -- 172.21.15.66:0/2798395342 reap_dead start 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a15000640 1 -- 172.21.15.66:0/2798395342 reap_dead start 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 shutdown_connections 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 >> 172.21.15.66:0/2798395342 conn(0x5653b567ac00 msgr2=0x5653b56a6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 shutdown_connections 2024-03-30T00:49:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/2798395342 wait complete. 2024-03-30T00:49:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/3478578910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b567b000 msgr2=0x5653b5707080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/3478578910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b567b000 0x5653b5707080 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x5653b64e8d80 tx=0x5653b6588240 comp rx=0 tx=0).stop 2024-03-30T00:49:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/3478578910 shutdown_connections 2024-03-30T00:49:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/3478578910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653b567b000 0x5653b5707080 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/3478578910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653b567b400 0x5653b5706b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 --2- 172.21.15.66:0/3478578910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5653b567b800 0x5653b5706580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/3478578910 >> 172.21.15.66:0/3478578910 conn(0x5653b567bc00 msgr2=0x5653b56a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/3478578910 shutdown_connections 2024-03-30T00:49:43.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.832+0000 7f4a17a4a400 1 -- 172.21.15.66:0/3478578910 wait complete. 2024-03-30T00:49:43.847 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.117334, result: Events by type: Errors: 0 2024-03-30T00:49:43.847 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-03-30T00:49:43.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcfe36080 1 Processor -- start 2024-03-30T00:49:43.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcfe36080 1 -- start start 2024-03-30T00:49:43.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcfe36080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 0x56069edc2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcfe36080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56069ed36cf0 0x56069edbf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcfe36080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56069eda6290 0x56069edbf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcfe36080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069ec32960 con 0x56069ed36910 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcfe36080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069ec32600 con 0x56069eda6290 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcfe36080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x56069ec29690 con 0x56069ed36cf0 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 0x56069edc2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 0x56069edc2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39924/0 (socket says 172.21.15.66:39924) 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcd000640 1 -- 172.21.15.66:0/1812280885 learned_addr learned my addr 172.21.15.66:0/1812280885 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dc7e00640 1 --2- 172.21.15.66:0/1812280885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56069ed36cf0 0x56069edbf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcda00640 1 --2- 172.21.15.66:0/1812280885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56069eda6290 0x56069edbf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcd000640 1 -- 172.21.15.66:0/1812280885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56069ed36cf0 msgr2=0x56069edbf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcd000640 1 --2- 172.21.15.66:0/1812280885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56069ed36cf0 0x56069edbf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcd000640 1 -- 172.21.15.66:0/1812280885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56069eda6290 msgr2=0x56069edbf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcd000640 1 --2- 172.21.15.66:0/1812280885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56069eda6290 0x56069edbf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcd000640 1 -- 172.21.15.66:0/1812280885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc800a1c0 con 0x56069ed36910 2024-03-30T00:49:43.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcd000640 1 --2- 172.21.15.66:0/1812280885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 0x56069edc2fb0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0001d90 tx=0x7f7dc0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa46cdc393c657de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dc6a00640 1 -- 172.21.15.66:0/1812280885 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0014070 con 0x56069ed36910 2024-03-30T00:49:43.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dc6a00640 1 -- 172.21.15.66:0/1812280885 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc000f410 con 0x56069ed36910 2024-03-30T00:49:43.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dc6a00640 1 -- 172.21.15.66:0/1812280885 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000f7a0 con 0x56069ed36910 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcfe36080 1 -- 172.21.15.66:0/1812280885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 msgr2=0x56069edc2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.924+0000 7f7dcfe36080 1 --2- 172.21.15.66:0/1812280885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 0x56069edc2fb0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0001d90 tx=0x7f7dc0001e80 comp rx=0 tx=0).stop 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 -- 172.21.15.66:0/1812280885 shutdown_connections 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 --2- 172.21.15.66:0/1812280885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56069eda6290 0x56069edbf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 --2- 172.21.15.66:0/1812280885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56069ed36cf0 0x56069edbf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 --2- 172.21.15.66:0/1812280885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 0x56069edc2fb0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 -- 172.21.15.66:0/1812280885 >> 172.21.15.66:0/1812280885 conn(0x56069ed1b2f0 msgr2=0x56069ed19da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 -- 172.21.15.66:0/1812280885 shutdown_connections 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 -- 172.21.15.66:0/1812280885 wait complete. 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 Processor -- start 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 -- start start 2024-03-30T00:49:43.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 0x56069edc3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56069ed36cf0 0x56069edc4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56069eda6290 0x56069edc81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069edc5190 con 0x56069ed36910 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069edc5430 con 0x56069eda6290 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x56069edc56d0 con 0x56069ed36cf0 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 0x56069edc3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 0x56069edc3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39938/0 (socket says 172.21.15.66:39938) 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcd000640 1 -- 172.21.15.66:0/3092055308 learned_addr learned my addr 172.21.15.66:0/3092055308 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dc7e00640 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56069ed36cf0 0x56069edc4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcda00640 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56069eda6290 0x56069edc81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcd000640 1 -- 172.21.15.66:0/3092055308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56069ed36cf0 msgr2=0x56069edc4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcd000640 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56069ed36cf0 0x56069edc4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcd000640 1 -- 172.21.15.66:0/3092055308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56069eda6290 msgr2=0x56069edc81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcd000640 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56069eda6290 0x56069edc81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcd000640 1 -- 172.21.15.66:0/3092055308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56069ec34970 con 0x56069ed36910 2024-03-30T00:49:43.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcda00640 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56069eda6290 0x56069edc81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:43.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcd000640 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 0x56069edc3190 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0000fe0 tx=0x7f7dc0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dc5600640 1 -- 172.21.15.66:0/3092055308 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0014040 con 0x56069ed36910 2024-03-30T00:49:43.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dc5600640 1 -- 172.21.15.66:0/3092055308 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc001b860 con 0x56069ed36910 2024-03-30T00:49:43.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 -- 172.21.15.66:0/3092055308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56069ec418c0 con 0x56069ed36910 2024-03-30T00:49:43.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dc5600640 1 -- 172.21.15.66:0/3092055308 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc001bc90 con 0x56069ed36910 2024-03-30T00:49:43.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.928+0000 7f7dcfe36080 1 -- 172.21.15.66:0/3092055308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56069ec34d00 con 0x56069ed36910 2024-03-30T00:49:43.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.932+0000 7f7dc5600640 1 -- 172.21.15.66:0/3092055308 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7dc00193c0 con 0x56069ed36910 2024-03-30T00:49:43.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.932+0000 7f7dc5600640 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7dac0650a0 0x7f7dac067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.932+0000 7f7dc5600640 1 -- 172.21.15.66:0/3092055308 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7dc00840d0 con 0x56069ed36910 2024-03-30T00:49:43.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.932+0000 7f7dc7e00640 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7dac0650a0 0x7f7dac067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x56069edc36d0 0x56069ed95080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:43.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 -- 172.21.15.66:0/3092055308 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x56069ed955c0 con 0x56069edc36d0 2024-03-30T00:49:43.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcda00640 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x56069edc36d0 0x56069ed95080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:43.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dc7e00640 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7dac0650a0 0x7f7dac067560 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7db8005410 tx=0x7f7db800c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcda00640 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x56069edc36d0 0x56069ed95080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:43.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcda00640 1 -- 172.21.15.66:0/3092055308 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v113'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7dc80ab900 con 0x56069edc36d0 2024-03-30T00:49:43.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 -- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x56069edc36d0 msgr2=0x56069ed95080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x56069edc36d0 0x56069ed95080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 -- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7dac0650a0 msgr2=0x7f7dac067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7dac0650a0 0x7f7dac067560 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7db8005410 tx=0x7f7db800c040 comp rx=0 tx=0).stop 2024-03-30T00:49:43.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 -- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 msgr2=0x56069edc3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:43.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 --2- 172.21.15.66:0/3092055308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56069ed36910 0x56069edc3190 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0000fe0 tx=0x7f7dc0000bd0 comp rx=0 tx=0).stop 2024-03-30T00:49:43.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcda00640 1 -- 172.21.15.66:0/3092055308 reap_dead start 2024-03-30T00:49:43.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 -- 172.21.15.66:0/3092055308 shutdown_connections 2024-03-30T00:49:43.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 -- 172.21.15.66:0/3092055308 >> 172.21.15.66:0/3092055308 conn(0x56069ed1b2f0 msgr2=0x56069eda5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:43.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 -- 172.21.15.66:0/3092055308 shutdown_connections 2024-03-30T00:49:43.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:43.936+0000 7f7dcfe36080 1 -- 172.21.15.66:0/3092055308 wait complete. 2024-03-30T00:49:43.948 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-30T00:49:44.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.124+0000 7fda18e00640 1 Processor -- start 2024-03-30T00:49:44.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.124+0000 7fda18e00640 1 -- start start 2024-03-30T00:49:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.124+0000 7fda18e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fda1415a640 0x7fda1406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.124+0000 7fda18e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140ce520 0x7fda140ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.124+0000 7fda18e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1406d070 con 0x7fda140cabb0 2024-03-30T00:49:44.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.124+0000 7fda18e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda140cbba0 con 0x7fda1415a640 2024-03-30T00:49:44.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.124+0000 7fda18e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1406e2c0 con 0x7fda140ce520 2024-03-30T00:49:44.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.124+0000 7fda11e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fda1415a640 0x7fda1406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.124+0000 7fda11e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fda1415a640 0x7fda1406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39948/0 (socket says 172.21.15.66:39948) 2024-03-30T00:49:44.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.124+0000 7fda11e00640 1 -- 172.21.15.66:0/3452742631 learned_addr learned my addr 172.21.15.66:0/3452742631 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.124+0000 7fda0be00640 1 --2- 172.21.15.66:0/3452742631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140ce520 0x7fda140ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda11e00640 1 -- 172.21.15.66:0/3452742631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140ce520 msgr2=0x7fda140ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda11e00640 1 --2- 172.21.15.66:0/3452742631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140ce520 0x7fda140ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda11e00640 1 -- 172.21.15.66:0/3452742631 >> v1:172.21.15.156:6789/0 conn(0x7fda140cabb0 legacy=0x7fda140cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda11e00640 1 -- 172.21.15.66:0/3452742631 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda140cee60 con 0x7fda1415a640 2024-03-30T00:49:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda0be00640 1 --2- 172.21.15.66:0/3452742631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140ce520 0x7fda140ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda11e00640 1 --2- 172.21.15.66:0/3452742631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda1415a640 0x7fda1406dd80 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fda04009870 tx=0x7fda0401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09264e593f5f0dc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda11400640 1 -- 172.21.15.66:0/3452742631 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda040ab650 con 0x7fda1415a640 2024-03-30T00:49:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda11400640 1 -- 172.21.15.66:0/3452742631 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda040ab7b0 con 0x7fda1415a640 2024-03-30T00:49:44.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda11400640 1 -- 172.21.15.66:0/3452742631 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda040abb40 con 0x7fda1415a640 2024-03-30T00:49:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda18e00640 1 -- 172.21.15.66:0/3452742631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda1415a640 msgr2=0x7fda1406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda18e00640 1 --2- 172.21.15.66:0/3452742631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda1415a640 0x7fda1406dd80 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fda04009870 tx=0x7fda0401c560 comp rx=0 tx=0).stop 2024-03-30T00:49:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda18e00640 1 -- 172.21.15.66:0/3452742631 shutdown_connections 2024-03-30T00:49:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda18e00640 1 --2- 172.21.15.66:0/3452742631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140ce520 0x7fda140ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda18e00640 1 --2- 172.21.15.66:0/3452742631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda1415a640 0x7fda1406dd80 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda18e00640 1 -- 172.21.15.66:0/3452742631 >> 172.21.15.66:0/3452742631 conn(0x7fda14069f60 msgr2=0x7fda1406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda18e00640 1 -- 172.21.15.66:0/3452742631 shutdown_connections 2024-03-30T00:49:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.128+0000 7fda18e00640 1 -- 172.21.15.66:0/3452742631 wait complete. 2024-03-30T00:49:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda18e00640 1 Processor -- start 2024-03-30T00:49:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda18e00640 1 -- start start 2024-03-30T00:49:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda18e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140cabb0 0x7fda14156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda18e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda140ce520 0x7fda14162f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda18e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda1415a640 0x7fda14156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda18e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141592b0 con 0x7fda140ce520 2024-03-30T00:49:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda18e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1415c010 con 0x7fda1415a640 2024-03-30T00:49:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda18e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda14156cd0 con 0x7fda140cabb0 2024-03-30T00:49:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda140ce520 0x7fda14162f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda140ce520 0x7fda14162f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39958/0 (socket says 172.21.15.66:39958) 2024-03-30T00:49:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0be00640 1 -- 172.21.15.66:0/4003005017 learned_addr learned my addr 172.21.15.66:0/4003005017 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:44.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda11e00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140cabb0 0x7fda14156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda12800640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda1415a640 0x7fda14156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0be00640 1 -- 172.21.15.66:0/4003005017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140cabb0 msgr2=0x7fda14156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0be00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140cabb0 0x7fda14156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0be00640 1 -- 172.21.15.66:0/4003005017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda1415a640 msgr2=0x7fda14156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0be00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda1415a640 0x7fda14156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0be00640 1 -- 172.21.15.66:0/4003005017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040094d0 con 0x7fda140ce520 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda11e00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140cabb0 0x7fda14156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0be00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda140ce520 0x7fda14162f20 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e810 tx=0x7fd9fc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0a000640 1 -- 172.21.15.66:0/4003005017 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00c8d0 con 0x7fda140ce520 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0a000640 1 -- 172.21.15.66:0/4003005017 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc00ca30 con 0x7fda140ce520 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda18e00640 1 -- 172.21.15.66:0/4003005017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda14163460 con 0x7fda140ce520 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda0a000640 1 -- 172.21.15.66:0/4003005017 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00cdc0 con 0x7fda140ce520 2024-03-30T00:49:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.132+0000 7fda18e00640 1 -- 172.21.15.66:0/4003005017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda14159740 con 0x7fda140ce520 2024-03-30T00:49:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.136+0000 7fda18e00640 1 -- 172.21.15.66:0/4003005017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1415c230 con 0x7fda140ce520 2024-03-30T00:49:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.136+0000 7fda0a000640 1 -- 172.21.15.66:0/4003005017 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd9fc013410 con 0x7fda140ce520 2024-03-30T00:49:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.136+0000 7fda0a000640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9f00666c0 0x7fd9f0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.136+0000 7fda0a000640 1 -- 172.21.15.66:0/4003005017 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fd9fc084110 con 0x7fda140ce520 2024-03-30T00:49:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.136+0000 7fda11e00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9f00666c0 0x7fd9f0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.136+0000 7fda11e00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9f00666c0 0x7fd9f0068b80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fda040af000 tx=0x7fda040a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.140+0000 7fda0a000640 1 -- 172.21.15.66:0/4003005017 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd9fc0512e0 con 0x7fda140ce520 2024-03-30T00:49:44.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.340+0000 7fda18e00640 1 -- 172.21.15.66:0/4003005017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fda14066c30 con 0x7fda140ce520 2024-03-30T00:49:44.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.620+0000 7fda0a000640 1 -- 172.21.15.66:0/4003005017 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v81) v1 ==== 162+0+0 (secure 0 0 0) 0x7fd9fc054900 con 0x7fda140ce520 2024-03-30T00:49:44.629 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-30T00:49:44.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 -- 172.21.15.66:0/4003005017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9f00666c0 msgr2=0x7fd9f0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:44.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9f00666c0 0x7fd9f0068b80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fda040af000 tx=0x7fda040a7000 comp rx=0 tx=0).stop 2024-03-30T00:49:44.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 -- 172.21.15.66:0/4003005017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda140ce520 msgr2=0x7fda14162f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:44.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda140ce520 0x7fda14162f20 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e810 tx=0x7fd9fc00ecd0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 -- 172.21.15.66:0/4003005017 shutdown_connections 2024-03-30T00:49:44.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda1415a640 0x7fda14156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd9f00666c0 0x7fd9f0068b80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda140ce520 0x7fda14162f20 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 --2- 172.21.15.66:0/4003005017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda140cabb0 0x7fda14156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 -- 172.21.15.66:0/4003005017 >> 172.21.15.66:0/4003005017 conn(0x7fda14069f60 msgr2=0x7fda140afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:44.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 -- 172.21.15.66:0/4003005017 shutdown_connections 2024-03-30T00:49:44.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.624+0000 7fda18e00640 1 -- 172.21.15.66:0/4003005017 wait complete. 2024-03-30T00:49:44.656 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:49:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0530a00640 1 Processor -- start 2024-03-30T00:49:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0530a00640 1 -- start start 2024-03-30T00:49:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0530a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f052c06e800 0x7f052c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0530a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052c06b0b0 con 0x7f052c0cc560 2024-03-30T00:49:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0530a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052c06b390 con 0x7f052c06f8e0 2024-03-30T00:49:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0530a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c160640 con 0x7f052c06e800 2024-03-30T00:49:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f052c0cc560 0x7f052c06e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:56814/0 (socket says 172.21.15.66:56814) 2024-03-30T00:49:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529e00640 1 -- 172.21.15.66:0/3469706989 learned_addr learned my addr 172.21.15.66:0/3469706989 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:44.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0523e00640 1 --2- 172.21.15.66:0/3469706989 >> v2:172.21.15.156:3300/0 conn(0x7f052c06e800 0x7f052c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679368496 0 0) 0x7f052c06b0b0 con 0x7f052c0cc560 2024-03-30T00:49:44.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0510003530 con 0x7f052c0cc560 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3717395575 0 0) 0x7f0510003530 con 0x7f052c0cc560 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 >> v2:172.21.15.156:3300/0 conn(0x7f052c06e800 msgr2=0x7f052c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 --2- 172.21.15.66:0/3469706989 >> v2:172.21.15.156:3300/0 conn(0x7f052c06e800 0x7f052c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 >> v1:172.21.15.156:6789/0 conn(0x7f052c06f8e0 legacy=0x7f052c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c06b670 con 0x7f052c0cc560 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f051c001680 con 0x7f052c0cc560 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 >> v1:172.21.15.66:6789/0 conn(0x7f052c0cc560 legacy=0x7f052c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 --2- 172.21.15.66:0/3469706989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0510003980 0x7f0510003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 --2- 172.21.15.66:0/3469706989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05100047d0 0x7f0510004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 --2- 172.21.15.66:0/3469706989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0510005190 0x7f051000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c160640 con 0x7f05100047d0 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051000c030 con 0x7f0510003980 2024-03-30T00:49:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051000c1f0 con 0x7f0510005190 2024-03-30T00:49:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0523e00640 1 --2- 172.21.15.66:0/3469706989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05100047d0 0x7f0510004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0529e00640 1 --2- 172.21.15.66:0/3469706989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0510003980 0x7f0510003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f052a800640 1 --2- 172.21.15.66:0/3469706989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0510005190 0x7f051000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0523e00640 1 -- 172.21.15.66:0/3469706989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0510005190 msgr2=0x7f051000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0523e00640 1 --2- 172.21.15.66:0/3469706989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0510005190 0x7f051000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0523e00640 1 -- 172.21.15.66:0/3469706989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0510003980 msgr2=0x7f0510003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0523e00640 1 --2- 172.21.15.66:0/3469706989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0510003980 0x7f0510003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f0523e00640 1 -- 172.21.15.66:0/3469706989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0510003530 con 0x7f05100047d0 2024-03-30T00:49:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.792+0000 7f052a800640 1 -- 172.21.15.66:0/3469706989 reap_dead start 2024-03-30T00:49:44.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.796+0000 7f0523e00640 1 --2- 172.21.15.66:0/3469706989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05100047d0 0x7f0510004c50 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f0514009ab0 tx=0x7f051401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748c7c4e9439d9fc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.796+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140a6440 con 0x7f05100047d0 2024-03-30T00:49:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.796+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05140a65a0 con 0x7f05100047d0 2024-03-30T00:49:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.796+0000 7f0529400640 1 -- 172.21.15.66:0/3469706989 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140a6950 con 0x7f05100047d0 2024-03-30T00:49:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.796+0000 7f0530a00640 1 -- 172.21.15.66:0/3469706989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05100047d0 msgr2=0x7f0510004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.796+0000 7f0530a00640 1 --2- 172.21.15.66:0/3469706989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05100047d0 0x7f0510004c50 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f0514009ab0 tx=0x7f051401ca50 comp rx=0 tx=0).stop 2024-03-30T00:49:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.796+0000 7f0530a00640 1 -- 172.21.15.66:0/3469706989 shutdown_connections 2024-03-30T00:49:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.796+0000 7f0530a00640 1 --2- 172.21.15.66:0/3469706989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05100047d0 0x7f0510004c50 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.796+0000 7f0530a00640 1 -- 172.21.15.66:0/3469706989 >> 172.21.15.66:0/3469706989 conn(0x7f052c0d0540 msgr2=0x7f052c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.796+0000 7f0530a00640 1 -- 172.21.15.66:0/3469706989 shutdown_connections 2024-03-30T00:49:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.796+0000 7f0530a00640 1 -- 172.21.15.66:0/3469706989 wait complete. 2024-03-30T00:49:44.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0530a00640 1 Processor -- start 2024-03-30T00:49:44.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0530a00640 1 -- start start 2024-03-30T00:49:44.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0530a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05100047d0 0x7f052c0c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0530a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f052c0b2790 0x7f052c0b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0530a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f052c0b16d0 0x7f052c1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0530a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0c8ec0 con 0x7f052c0b16d0 2024-03-30T00:49:44.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0530a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0b2330 con 0x7f05100047d0 2024-03-30T00:49:44.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0530a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0b2050 con 0x7f052c0b2790 2024-03-30T00:49:44.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f052a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f052c0b16d0 0x7f052c1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f052a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f052c0b16d0 0x7f052c1559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39988/0 (socket says 172.21.15.66:39988) 2024-03-30T00:49:44.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f052a800640 1 -- 172.21.15.66:0/201200642 learned_addr learned my addr 172.21.15.66:0/201200642 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:44.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0529e00640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05100047d0 0x7f052c0c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0523e00640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f052c0b2790 0x7f052c0b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f052a800640 1 -- 172.21.15.66:0/201200642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f052c0b2790 msgr2=0x7f052c0b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:44.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f052a800640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f052c0b2790 0x7f052c0b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f052a800640 1 -- 172.21.15.66:0/201200642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05100047d0 msgr2=0x7f052c0c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:44.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f052a800640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05100047d0 0x7f052c0c3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:44.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f052a800640 1 -- 172.21.15.66:0/201200642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c0c8800 con 0x7f052c0b16d0 2024-03-30T00:49:44.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f052a800640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f052c0b16d0 0x7f052c1559e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f052c072bd0 tx=0x7f052400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:44.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0522000640 1 -- 172.21.15.66:0/201200642 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524003980 con 0x7f052c0b16d0 2024-03-30T00:49:44.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0522000640 1 -- 172.21.15.66:0/201200642 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0524003ae0 con 0x7f052c0b16d0 2024-03-30T00:49:44.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0522000640 1 -- 172.21.15.66:0/201200642 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05240050d0 con 0x7f052c0b16d0 2024-03-30T00:49:44.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0530a00640 1 -- 172.21.15.66:0/201200642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c0c8ae0 con 0x7f052c0b16d0 2024-03-30T00:49:44.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.800+0000 7f0530a00640 1 -- 172.21.15.66:0/201200642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c0b5fb0 con 0x7f052c0b16d0 2024-03-30T00:49:44.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.804+0000 7f0530a00640 1 -- 172.21.15.66:0/201200642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c0c8660 con 0x7f052c0b16d0 2024-03-30T00:49:44.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.804+0000 7f0522000640 1 -- 172.21.15.66:0/201200642 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0524005230 con 0x7f052c0b16d0 2024-03-30T00:49:44.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.804+0000 7f0522000640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0508066690 0x7f0508068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:44.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.804+0000 7f0522000640 1 -- 172.21.15.66:0/201200642 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f0524083e80 con 0x7f052c0b16d0 2024-03-30T00:49:44.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.804+0000 7f0529e00640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0508066690 0x7f0508068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:44.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.804+0000 7f0529e00640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0508066690 0x7f0508068b50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f051c001060 tx=0x7f051c001c60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:44.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:44.808+0000 7f0522000640 1 -- 172.21.15.66:0/201200642 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f052c0c8660 con 0x7f052c0b16d0 2024-03-30T00:49:45.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 -- 172.21.15.66:0/201200642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f052c0c9000 con 0x7f052c0b16d0 2024-03-30T00:49:45.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0522000640 1 -- 172.21.15.66:0/201200642 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f052c0c9000 con 0x7f052c0b16d0 2024-03-30T00:49:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:45.011 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:49:44.623725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9057},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9057":{"gid":9057,"name":"c","rank":0,"incarnation":82,"state":"up:replay","state_seq":1,"addr":"172.21.15.66:6837/282507324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":282507324},{"type":"v1","addr":"172.21.15.66:6837","nonce":282507324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:49:45.011 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 82 2024-03-30T00:49:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 -- 172.21.15.66:0/201200642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0508066690 msgr2=0x7f0508068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0508066690 0x7f0508068b50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f051c001060 tx=0x7f051c001c60 comp rx=0 tx=0).stop 2024-03-30T00:49:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 -- 172.21.15.66:0/201200642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f052c0b16d0 msgr2=0x7f052c1559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f052c0b16d0 0x7f052c1559e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f052c072bd0 tx=0x7f052400ec10 comp rx=0 tx=0).stop 2024-03-30T00:49:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 -- 172.21.15.66:0/201200642 shutdown_connections 2024-03-30T00:49:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0508066690 0x7f0508068b50 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:45.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f052c0b16d0 0x7f052c1559e0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:45.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f052c0b2790 0x7f052c0b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:45.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 --2- 172.21.15.66:0/201200642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05100047d0 0x7f052c0c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:45.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 -- 172.21.15.66:0/201200642 >> 172.21.15.66:0/201200642 conn(0x7f052c0d0540 msgr2=0x7f052c15faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:45.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 -- 172.21.15.66:0/201200642 shutdown_connections 2024-03-30T00:49:45.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:45.004+0000 7f0530a00640 1 -- 172.21.15.66:0/201200642 wait complete. 2024-03-30T00:49:45.434 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:48:46.332321+0000', 'modified': '2024-03-30T00:49:44.623725+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9057}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9057': {'gid': 9057, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.66:6837/282507324', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 282507324}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 282507324}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:49:45.434 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9057:up:replay 2024-03-30T00:49:46.435 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:49:46.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.580+0000 7f62e4e00640 1 Processor -- start 2024-03-30T00:49:46.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.580+0000 7f62e4e00640 1 -- start start 2024-03-30T00:49:46.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.580+0000 7f62e4e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f62e015ca80 0x7f62e006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:46.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.580+0000 7f62e4e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62e00cb570 0x7f62e00cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:46.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62e4e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e00cb0e0 con 0x7f62e00cf2f0 2024-03-30T00:49:46.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62e4e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e00cf7f0 con 0x7f62e015ca80 2024-03-30T00:49:46.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62e4e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00cfad0 con 0x7f62e00cb570 2024-03-30T00:49:46.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62dde00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f62e015ca80 0x7f62e006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:46.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62dde00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f62e015ca80 0x7f62e006b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39990/0 (socket says 172.21.15.66:39990) 2024-03-30T00:49:46.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62dde00640 1 -- 172.21.15.66:0/4196566087 learned_addr learned my addr 172.21.15.66:0/4196566087 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:46.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62de800640 1 --2- 172.21.15.66:0/4196566087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62e00cb570 0x7f62e00cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:46.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62dde00640 1 -- 172.21.15.66:0/4196566087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62e00cb570 msgr2=0x7f62e00cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:46.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62dde00640 1 --2- 172.21.15.66:0/4196566087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62e00cb570 0x7f62e00cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:46.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62dde00640 1 -- 172.21.15.66:0/4196566087 >> v1:172.21.15.156:6789/0 conn(0x7f62e00cf2f0 legacy=0x7f62e006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:46.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62dde00640 1 -- 172.21.15.66:0/4196566087 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e00cad70 con 0x7f62e015ca80 2024-03-30T00:49:46.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62de800640 1 --2- 172.21.15.66:0/4196566087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62e00cb570 0x7f62e00cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62dde00640 1 --2- 172.21.15.66:0/4196566087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e015ca80 0x7f62e006b0b0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009870 tx=0x7f62d001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1f6f48eb0938b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62dd400640 1 -- 172.21.15.66:0/4196566087 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00a8a50 con 0x7f62e015ca80 2024-03-30T00:49:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62dd400640 1 -- 172.21.15.66:0/4196566087 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d00a8bb0 con 0x7f62e015ca80 2024-03-30T00:49:46.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.584+0000 7f62dd400640 1 -- 172.21.15.66:0/4196566087 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00b0440 con 0x7f62e015ca80 2024-03-30T00:49:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 -- 172.21.15.66:0/4196566087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e015ca80 msgr2=0x7f62e006b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 --2- 172.21.15.66:0/4196566087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e015ca80 0x7f62e006b0b0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009870 tx=0x7f62d001ed50 comp rx=0 tx=0).stop 2024-03-30T00:49:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 -- 172.21.15.66:0/4196566087 shutdown_connections 2024-03-30T00:49:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 --2- 172.21.15.66:0/4196566087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62e00cb570 0x7f62e00cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:46.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 --2- 172.21.15.66:0/4196566087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e015ca80 0x7f62e006b0b0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:46.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 -- 172.21.15.66:0/4196566087 >> 172.21.15.66:0/4196566087 conn(0x7f62e00b2090 msgr2=0x7f62e00b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:46.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 -- 172.21.15.66:0/4196566087 shutdown_connections 2024-03-30T00:49:46.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 -- 172.21.15.66:0/4196566087 wait complete. 2024-03-30T00:49:46.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 Processor -- start 2024-03-30T00:49:46.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 -- start start 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62e00cb570 0x7f62e00b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e00cf2f0 0x7f62e00c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e015ca80 0x7f62e00c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00c8980 con 0x7f62e00cf2f0 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e0158e40 con 0x7f62e015ca80 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62e4e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00c7d00 con 0x7f62e00cb570 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62d5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e00cf2f0 0x7f62e00c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62d5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e00cf2f0 0x7f62e00c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40000/0 (socket says 172.21.15.66:40000) 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62d5400640 1 -- 172.21.15.66:0/505823170 learned_addr learned my addr 172.21.15.66:0/505823170 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62dde00640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62e00cb570 0x7f62e00b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62de800640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e015ca80 0x7f62e00c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:46.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62d5400640 1 -- 172.21.15.66:0/505823170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62e00cb570 msgr2=0x7f62e00b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62d5400640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62e00cb570 0x7f62e00b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62d5400640 1 -- 172.21.15.66:0/505823170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e015ca80 msgr2=0x7f62e00c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62d5400640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e015ca80 0x7f62e00c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.588+0000 7f62d5400640 1 -- 172.21.15.66:0/505823170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00094d0 con 0x7f62e00cf2f0 2024-03-30T00:49:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.592+0000 7f62d5400640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e00cf2f0 0x7f62e00c7670 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c500 tx=0x7f62c800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.592+0000 7f62d6a00640 1 -- 172.21.15.66:0/505823170 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8005220 con 0x7f62e00cf2f0 2024-03-30T00:49:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.592+0000 7f62d6a00640 1 -- 172.21.15.66:0/505823170 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c8005380 con 0x7f62e00cf2f0 2024-03-30T00:49:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.592+0000 7f62e4e00640 1 -- 172.21.15.66:0/505823170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e00c7fe0 con 0x7f62e00cf2f0 2024-03-30T00:49:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.592+0000 7f62d6a00640 1 -- 172.21.15.66:0/505823170 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8007270 con 0x7f62e00cf2f0 2024-03-30T00:49:46.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.592+0000 7f62e4e00640 1 -- 172.21.15.66:0/505823170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e00c85a0 con 0x7f62e00cf2f0 2024-03-30T00:49:46.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.592+0000 7f62e4e00640 1 -- 172.21.15.66:0/505823170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e00c8ac0 con 0x7f62e00cf2f0 2024-03-30T00:49:46.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.592+0000 7f62d6a00640 1 -- 172.21.15.66:0/505823170 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f62c80073d0 con 0x7f62e00cf2f0 2024-03-30T00:49:46.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.592+0000 7f62d6a00640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62bc066690 0x7f62bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:46.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.592+0000 7f62d6a00640 1 -- 172.21.15.66:0/505823170 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f62c8083810 con 0x7f62e00cf2f0 2024-03-30T00:49:46.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.592+0000 7f62dde00640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62bc066690 0x7f62bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:46.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.596+0000 7f62dde00640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62bc066690 0x7f62bc068b50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f62d00b2000 tx=0x7f62d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:46.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.596+0000 7f62d6a00640 1 -- 172.21.15.66:0/505823170 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f62e00c8ac0 con 0x7f62e00cf2f0 2024-03-30T00:49:46.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.796+0000 7f62e4e00640 1 -- 172.21.15.66:0/505823170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62e00c8ac0 con 0x7f62e00cf2f0 2024-03-30T00:49:46.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62d6a00640 1 -- 172.21.15.66:0/505823170 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f62e00c8ac0 con 0x7f62e00cf2f0 2024-03-30T00:49:46.806 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:46.806 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:49:46.629088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9057},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9057":{"gid":9057,"name":"c","rank":0,"incarnation":82,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.66:6837/282507324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":282507324},{"type":"v1","addr":"172.21.15.66:6837","nonce":282507324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:49:46.806 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 84 2024-03-30T00:49:46.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 -- 172.21.15.66:0/505823170 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62bc066690 msgr2=0x7f62bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:46.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62bc066690 0x7f62bc068b50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f62d00b2000 tx=0x7f62d0009040 comp rx=0 tx=0).stop 2024-03-30T00:49:46.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 -- 172.21.15.66:0/505823170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e00cf2f0 msgr2=0x7f62e00c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:46.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e00cf2f0 0x7f62e00c7670 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c500 tx=0x7f62c800c9c0 comp rx=0 tx=0).stop 2024-03-30T00:49:46.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 -- 172.21.15.66:0/505823170 shutdown_connections 2024-03-30T00:49:46.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62e015ca80 0x7f62e00c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:46.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62bc066690 0x7f62bc068b50 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:46.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62e00cf2f0 0x7f62e00c7670 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:46.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 --2- 172.21.15.66:0/505823170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62e00cb570 0x7f62e00b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:46.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 -- 172.21.15.66:0/505823170 >> 172.21.15.66:0/505823170 conn(0x7f62e00b2090 msgr2=0x7f62e00cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:46.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 -- 172.21.15.66:0/505823170 shutdown_connections 2024-03-30T00:49:46.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:46.800+0000 7f62e4e00640 1 -- 172.21.15.66:0/505823170 wait complete. 2024-03-30T00:49:47.230 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:48:46.332321+0000', 'modified': '2024-03-30T00:49:46.629088+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9057}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9057': {'gid': 9057, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.66:6837/282507324', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 282507324}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 282507324}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:49:47.230 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9057:up:rejoin 2024-03-30T00:49:47.582 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:49:47.586 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:49:48.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:49:48.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bfaa00640 1 Processor -- start 2024-03-30T00:49:48.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bfaa00640 1 -- start start 2024-03-30T00:49:48.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bfaa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3bf415a640 0x7f3bf406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:48.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bfaa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf40ce520 0x7f3bf40ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:48.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bfaa00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf406d070 con 0x7f3bf40cabb0 2024-03-30T00:49:48.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bfaa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf40cbba0 con 0x7f3bf415a640 2024-03-30T00:49:48.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bfaa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf406e2c0 con 0x7f3bf40ce520 2024-03-30T00:49:48.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bf3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3bf415a640 0x7f3bf406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:48.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bf3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3bf415a640 0x7f3bf406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40014/0 (socket says 172.21.15.66:40014) 2024-03-30T00:49:48.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bf3e00640 1 -- 172.21.15.66:0/2530350087 learned_addr learned my addr 172.21.15.66:0/2530350087 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:48.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3beb400640 1 --2- 172.21.15.66:0/2530350087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf40ce520 0x7f3bf40ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:48.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bf3e00640 1 -- 172.21.15.66:0/2530350087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf40ce520 msgr2=0x7f3bf40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:48.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bf3e00640 1 --2- 172.21.15.66:0/2530350087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf40ce520 0x7f3bf40ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:48.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bf3e00640 1 -- 172.21.15.66:0/2530350087 >> v1:172.21.15.156:6789/0 conn(0x7f3bf40cabb0 legacy=0x7f3bf40cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:48.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.376+0000 7f3bf3e00640 1 -- 172.21.15.66:0/2530350087 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf40cee60 con 0x7f3bf415a640 2024-03-30T00:49:48.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bf3e00640 1 --2- 172.21.15.66:0/2530350087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bf415a640 0x7f3bf406dd80 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009870 tx=0x7f3be401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1101e48ce2b9373 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:48.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bf3400640 1 -- 172.21.15.66:0/2530350087 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be40ab650 con 0x7f3bf415a640 2024-03-30T00:49:48.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bf3400640 1 -- 172.21.15.66:0/2530350087 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be40ab7b0 con 0x7f3bf415a640 2024-03-30T00:49:48.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bf3400640 1 -- 172.21.15.66:0/2530350087 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be40abb40 con 0x7f3bf415a640 2024-03-30T00:49:48.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bfaa00640 1 -- 172.21.15.66:0/2530350087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bf415a640 msgr2=0x7f3bf406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:48.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bfaa00640 1 --2- 172.21.15.66:0/2530350087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bf415a640 0x7f3bf406dd80 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009870 tx=0x7f3be401c560 comp rx=0 tx=0).stop 2024-03-30T00:49:48.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bfaa00640 1 -- 172.21.15.66:0/2530350087 shutdown_connections 2024-03-30T00:49:48.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bfaa00640 1 --2- 172.21.15.66:0/2530350087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf40ce520 0x7f3bf40ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:48.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bfaa00640 1 --2- 172.21.15.66:0/2530350087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bf415a640 0x7f3bf406dd80 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:48.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bfaa00640 1 -- 172.21.15.66:0/2530350087 >> 172.21.15.66:0/2530350087 conn(0x7f3bf4069f60 msgr2=0x7f3bf406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:48.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bfaa00640 1 -- 172.21.15.66:0/2530350087 shutdown_connections 2024-03-30T00:49:48.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.380+0000 7f3bfaa00640 1 -- 172.21.15.66:0/2530350087 wait complete. 2024-03-30T00:49:48.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bfaa00640 1 Processor -- start 2024-03-30T00:49:48.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bfaa00640 1 -- start start 2024-03-30T00:49:48.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bfaa00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf40cabb0 0x7f3bf40abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:48.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bfaa00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf40ce520 0x7f3bf40ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:48.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bfaa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bf415a640 0x7f3bf41556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bfaa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf40cc790 con 0x7f3bf415a640 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bfaa00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4156290 con 0x7f3bf40ce520 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bfaa00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4156570 con 0x7f3bf40cabb0 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bf415a640 0x7f3bf41556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bf415a640 0x7f3bf41556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40024/0 (socket says 172.21.15.66:40024) 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf8a00640 1 -- 172.21.15.66:0/4162532698 learned_addr learned my addr 172.21.15.66:0/4162532698 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf3e00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf40cabb0 0x7f3bf40abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3beb400640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf40ce520 0x7f3bf40ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf8a00640 1 -- 172.21.15.66:0/4162532698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf40cabb0 msgr2=0x7f3bf40abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf8a00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf40cabb0 0x7f3bf40abc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf8a00640 1 -- 172.21.15.66:0/4162532698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf40ce520 msgr2=0x7f3bf40ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf8a00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf40ce520 0x7f3bf40ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf8a00640 1 -- 172.21.15.66:0/4162532698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be40094d0 con 0x7f3bf415a640 2024-03-30T00:49:48.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf8a00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bf415a640 0x7f3bf41556b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f3bec00c770 tx=0x7f3bec00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:48.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf0c00640 1 -- 172.21.15.66:0/4162532698 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec003b80 con 0x7f3bf415a640 2024-03-30T00:49:48.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf0c00640 1 -- 172.21.15.66:0/4162532698 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec003ce0 con 0x7f3bf415a640 2024-03-30T00:49:48.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bf0c00640 1 -- 172.21.15.66:0/4162532698 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec00aa90 con 0x7f3bf415a640 2024-03-30T00:49:48.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bfaa00640 1 -- 172.21.15.66:0/4162532698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf4156850 con 0x7f3bf415a640 2024-03-30T00:49:48.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.384+0000 7f3bfaa00640 1 -- 172.21.15.66:0/4162532698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf40cd390 con 0x7f3bf415a640 2024-03-30T00:49:48.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.388+0000 7f3bfaa00640 1 -- 172.21.15.66:0/4162532698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf4152ad0 con 0x7f3bf415a640 2024-03-30T00:49:48.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.388+0000 7f3bf0c00640 1 -- 172.21.15.66:0/4162532698 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3bec00abf0 con 0x7f3bf415a640 2024-03-30T00:49:48.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.388+0000 7f3bf0c00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3bd0066690 0x7f3bd0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:48.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.388+0000 7f3bf0c00640 1 -- 172.21.15.66:0/4162532698 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f3bec083ff0 con 0x7f3bf415a640 2024-03-30T00:49:48.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.388+0000 7f3bf3e00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3bd0066690 0x7f3bd0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:48.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.388+0000 7f3bf3e00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3bd0066690 0x7f3bd0068b50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3be40af000 tx=0x7f3be40b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:48.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.392+0000 7f3bf0c00640 1 -- 172.21.15.66:0/4162532698 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3bec0b5980 con 0x7f3bf415a640 2024-03-30T00:49:48.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.584+0000 7f3bfaa00640 1 -- 172.21.15.66:0/4162532698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bf4066c30 con 0x7f3bf415a640 2024-03-30T00:49:48.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.584+0000 7f3bf0c00640 1 -- 172.21.15.66:0/4162532698 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f3bf4152ad0 con 0x7f3bf415a640 2024-03-30T00:49:48.592 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:48.592 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:49:47.630758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9057},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9057":{"gid":9057,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6837/282507324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":282507324},{"type":"v1","addr":"172.21.15.66:6837","nonce":282507324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:49:48.592 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 85 2024-03-30T00:49:48.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 -- 172.21.15.66:0/4162532698 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3bd0066690 msgr2=0x7f3bd0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:48.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3bd0066690 0x7f3bd0068b50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3be40af000 tx=0x7f3be40b6040 comp rx=0 tx=0).stop 2024-03-30T00:49:48.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 -- 172.21.15.66:0/4162532698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bf415a640 msgr2=0x7f3bf41556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:48.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bf415a640 0x7f3bf41556b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f3bec00c770 tx=0x7f3bec00cc30 comp rx=0 tx=0).stop 2024-03-30T00:49:48.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 -- 172.21.15.66:0/4162532698 shutdown_connections 2024-03-30T00:49:48.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3bd0066690 0x7f3bd0068b50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:48.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bf415a640 0x7f3bf41556b0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:48.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf40ce520 0x7f3bf40ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:48.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 --2- 172.21.15.66:0/4162532698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf40cabb0 0x7f3bf40abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:48.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 -- 172.21.15.66:0/4162532698 >> 172.21.15.66:0/4162532698 conn(0x7f3bf4069f60 msgr2=0x7f3bf40ad7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:48.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 -- 172.21.15.66:0/4162532698 shutdown_connections 2024-03-30T00:49:48.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:48.588+0000 7f3bfaa00640 1 -- 172.21.15.66:0/4162532698 wait complete. 2024-03-30T00:49:49.017 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:48:46.332321+0000', 'modified': '2024-03-30T00:49:47.630758+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9057}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9057': {'gid': 9057, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.66:6837/282507324', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 282507324}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 282507324}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:49:49.017 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:49:49.017 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T00:49:49.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.152+0000 7f0f78a00640 1 Processor -- start 2024-03-30T00:49:49.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.152+0000 7f0f78a00640 1 -- start start 2024-03-30T00:49:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.152+0000 7f0f78a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f7415a640 0x7f0f7406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f78a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0f740cb2c0 0x7f0f740ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f78a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7406d070 con 0x7f0f740ce520 2024-03-30T00:49:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f78a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f740cbba0 con 0x7f0f740cb2c0 2024-03-30T00:49:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f78a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f740cee90 con 0x7f0f7415a640 2024-03-30T00:49:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f72800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0f740cb2c0 0x7f0f740ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f72800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0f740cb2c0 0x7f0f740ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49400/0 (socket says 172.21.15.66:49400) 2024-03-30T00:49:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f72800640 1 -- 172.21.15.66:0/784942561 learned_addr learned my addr 172.21.15.66:0/784942561 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f72800640 1 -- 172.21.15.66:0/784942561 >> v2:172.21.15.156:3300/0 conn(0x7f0f7415a640 msgr2=0x7f0f7406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f71e00640 1 --2- 172.21.15.66:0/784942561 >> v2:172.21.15.156:3300/0 conn(0x7f0f7415a640 0x7f0f7406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f72800640 1 --2- 172.21.15.66:0/784942561 >> v2:172.21.15.156:3300/0 conn(0x7f0f7415a640 0x7f0f7406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f72800640 1 -- 172.21.15.66:0/784942561 >> v1:172.21.15.156:6789/0 conn(0x7f0f740ce520 legacy=0x7f0f740cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f72800640 1 -- 172.21.15.66:0/784942561 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7406e2c0 con 0x7f0f740cb2c0 2024-03-30T00:49:49.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f71e00640 1 --2- 172.21.15.66:0/784942561 >> v2:172.21.15.156:3300/0 conn(0x7f0f7415a640 0x7f0f7406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:49.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f72800640 1 --2- 172.21.15.66:0/784942561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f740cb2c0 0x7f0f740ce950 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f0f740cf160 tx=0x7f0f6c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d505263fb613397f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f71400640 1 -- 172.21.15.66:0/784942561 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0a5760 con 0x7f0f740cb2c0 2024-03-30T00:49:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f71400640 1 -- 172.21.15.66:0/784942561 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c0a58c0 con 0x7f0f740cb2c0 2024-03-30T00:49:49.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.156+0000 7f0f71400640 1 -- 172.21.15.66:0/784942561 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0a5c50 con 0x7f0f740cb2c0 2024-03-30T00:49:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 -- 172.21.15.66:0/784942561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f740cb2c0 msgr2=0x7f0f740ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 --2- 172.21.15.66:0/784942561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f740cb2c0 0x7f0f740ce950 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f0f740cf160 tx=0x7f0f6c01c9b0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 -- 172.21.15.66:0/784942561 shutdown_connections 2024-03-30T00:49:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 --2- 172.21.15.66:0/784942561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f740cb2c0 0x7f0f740ce950 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 --2- 172.21.15.66:0/784942561 >> v2:172.21.15.156:3300/0 conn(0x7f0f7415a640 0x7f0f7406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 -- 172.21.15.66:0/784942561 >> 172.21.15.66:0/784942561 conn(0x7f0f74069f60 msgr2=0x7f0f7406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 -- 172.21.15.66:0/784942561 shutdown_connections 2024-03-30T00:49:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 -- 172.21.15.66:0/784942561 wait complete. 2024-03-30T00:49:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 Processor -- start 2024-03-30T00:49:49.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 -- start start 2024-03-30T00:49:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f740cb2c0 0x7f0f741637c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f740ce520 0x7f0f740ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7415a640 0x7f0f740caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f740cb0d0 con 0x7f0f7415a640 2024-03-30T00:49:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f740ace10 con 0x7f0f740ce520 2024-03-30T00:49:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f78a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f74144990 con 0x7f0f740cb2c0 2024-03-30T00:49:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f72800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7415a640 0x7f0f740caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f72800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7415a640 0x7f0f740caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49416/0 (socket says 172.21.15.66:49416) 2024-03-30T00:49:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f72800640 1 -- 172.21.15.66:0/901432619 learned_addr learned my addr 172.21.15.66:0/901432619 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f72800640 1 -- 172.21.15.66:0/901432619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f740cb2c0 msgr2=0x7f0f741637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f72800640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f740cb2c0 0x7f0f741637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f72800640 1 -- 172.21.15.66:0/901432619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f740ce520 msgr2=0x7f0f740ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f69400640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f740ce520 0x7f0f740ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f72800640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f740ce520 0x7f0f740ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f72800640 1 -- 172.21.15.66:0/901432619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c009b30 con 0x7f0f7415a640 2024-03-30T00:49:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.160+0000 7f0f69400640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f740ce520 0x7f0f740ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.164+0000 7f0f72800640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7415a640 0x7f0f740caa70 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c01b060 tx=0x7f0f6c019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.164+0000 7f0f6aa00640 1 -- 172.21.15.66:0/901432619 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0b8670 con 0x7f0f7415a640 2024-03-30T00:49:49.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.164+0000 7f0f6aa00640 1 -- 172.21.15.66:0/901432619 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c0b87d0 con 0x7f0f7415a640 2024-03-30T00:49:49.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.164+0000 7f0f6aa00640 1 -- 172.21.15.66:0/901432619 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0b8b80 con 0x7f0f7415a640 2024-03-30T00:49:49.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.164+0000 7f0f78a00640 1 -- 172.21.15.66:0/901432619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f74144c70 con 0x7f0f7415a640 2024-03-30T00:49:49.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.164+0000 7f0f78a00640 1 -- 172.21.15.66:0/901432619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f74145230 con 0x7f0f7415a640 2024-03-30T00:49:49.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.164+0000 7f0f6aa00640 1 -- 172.21.15.66:0/901432619 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0f6c0b8ce0 con 0x7f0f7415a640 2024-03-30T00:49:49.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.164+0000 7f0f6aa00640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f50066690 0x7f0f50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.164+0000 7f0f6aa00640 1 -- 172.21.15.66:0/901432619 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f0f6c11f9f0 con 0x7f0f7415a640 2024-03-30T00:49:49.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.164+0000 7f0f71e00640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f50066690 0x7f0f50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 Processor -- start 2024-03-30T00:49:49.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 -- start start 2024-03-30T00:49:49.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f71e00640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f50066690 0x7f0f50068b50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400a890 tx=0x7f0f64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561475308db0 0x561475309190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x5614752bef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614752bf4c0 0x5614752bf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614751875a0 con 0x5614753096d0 2024-03-30T00:49:49.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561475114af0 con 0x561475308db0 2024-03-30T00:49:49.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614751120c0 con 0x5614752bf4c0 2024-03-30T00:49:49.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f69400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x5614752bef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f69400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x5614752bef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49426/0 (socket says 172.21.15.66:49426) 2024-03-30T00:49:49.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f69400640 1 -- 172.21.15.66:0/485684957 learned_addr learned my addr 172.21.15.66:0/485684957 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f69400640 1 -- 172.21.15.66:0/485684957 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614752bf4c0 msgr2=0x5614752bf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f69400640 1 --2- 172.21.15.66:0/485684957 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614752bf4c0 0x5614752bf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f69400640 1 -- 172.21.15.66:0/485684957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561475308db0 msgr2=0x561475309190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f69400640 1 --2- 172.21.15.66:0/485684957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561475308db0 0x561475309190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f69400640 1 -- 172.21.15.66:0/485684957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561474e09890 con 0x5614753096d0 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f69400640 1 --2- 172.21.15.66:0/485684957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x5614752bef80 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f0f740aca50 tx=0x7f0f5c001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29a54b299f79e15 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f62a00640 1 -- 172.21.15.66:0/485684957 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c00b070 con 0x5614753096d0 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f62a00640 1 -- 172.21.15.66:0/485684957 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5c00d6d0 con 0x5614753096d0 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f62a00640 1 -- 172.21.15.66:0/485684957 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c00da80 con 0x5614753096d0 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 -- 172.21.15.66:0/485684957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 msgr2=0x5614752bef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/485684957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x5614752bef80 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f0f740aca50 tx=0x7f0f5c001bb0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 -- 172.21.15.66:0/485684957 shutdown_connections 2024-03-30T00:49:49.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/485684957 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614752bf4c0 0x5614752bf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/485684957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x5614752bef80 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/485684957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561475308db0 0x561475309190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 -- 172.21.15.66:0/485684957 >> 172.21.15.66:0/485684957 conn(0x5614752c5830 msgr2=0x5614752bd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 -- 172.21.15.66:0/485684957 shutdown_connections 2024-03-30T00:49:49.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 -- 172.21.15.66:0/485684957 wait complete. 2024-03-30T00:49:49.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 -- start start 2024-03-30T00:49:49.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.168+0000 7f0f7be5e000 1 Processor -- start 2024-03-30T00:49:49.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614752bf4c0 0x5614752c4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561475308db0 0x561475308220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614753096d0 0x561475308760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147510d9b0 con 0x561475308db0 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614750acc00 con 0x5614752bf4c0 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614750cb070 con 0x5614753096d0 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 10 client.0 _ll_register_callbacks cb 0x5614752ab0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f69400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561475308db0 0x561475308220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f69400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561475308db0 0x561475308220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49428/0 (socket says 172.21.15.66:49428) 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f69400640 1 -- 172.21.15.66:0/4097517385 learned_addr learned my addr 172.21.15.66:0/4097517385 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f72800640 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614753096d0 0x561475308760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f71e00640 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614752bf4c0 0x5614752c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f69400640 1 -- 172.21.15.66:0/4097517385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614753096d0 msgr2=0x561475308760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f69400640 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614753096d0 0x561475308760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f69400640 1 -- 172.21.15.66:0/4097517385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614752bf4c0 msgr2=0x5614752c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f69400640 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614752bf4c0 0x5614752c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f69400640 1 -- 172.21.15.66:0/4097517385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c001380 con 0x561475308db0 2024-03-30T00:49:49.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f69400640 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561475308db0 0x561475308220 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009010 tx=0x7f0f5c000fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8625a02a07d81f49 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f57400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f57400640 1 -- 172.21.15.66:0/4097517385 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c0067a0 con 0x561475308db0 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 1 -- 172.21.15.66:0/4097517385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614750ca5f0 con 0x561475308db0 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f57400640 1 -- 172.21.15.66:0/4097517385 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f0f5c006900 con 0x561475308db0 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f57400640 1 -- 172.21.15.66:0/4097517385 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f5c006ac0 con 0x561475308db0 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 10 client.9123 fetch_fsmap learned FSMap version 85 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 1 -- 172.21.15.66:0/4097517385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x5614750d2ab0 con 0x561475308db0 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f57400640 1 -- 172.21.15.66:0/4097517385 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614750ca5f0 con 0x561475308db0 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f57400640 1 -- 172.21.15.66:0/4097517385 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0f5c006c20 con 0x561475308db0 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 10 client.9123 fetch_fsmap finished waiting for FSMap version 85 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 10 client.9123 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 20 client.9123 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:49.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x56147530a2e0 0x56147531a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 4 client.9123 mds_command: new command op to 9057 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f7be5e000 1 -- 172.21.15.66:0/4097517385 --> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56147510a790 con 0x56147530a2e0 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f71e00640 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x56147530a2e0 0x56147531a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f71e00640 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x56147530a2e0 0x56147531a530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2233ed9075bf9a4 server_cookie=753d426a3cb372ca in_seq=0 out_seq=0 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.172+0000 7f0f57400640 10 client.9123 ms_handle_connect on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f57400640 1 -- 172.21.15.66:0/4097517385 <== mds.0 v2:172.21.15.66:6835/282507324 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56147510a790 con 0x56147530a2e0 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f57400640 10 client.9123 handle_command_reply: tid=0 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 1 client.9123 shutdown 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 1 -- 172.21.15.66:0/4097517385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561475308db0 msgr2=0x561475308220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561475308db0 0x561475308220 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009010 tx=0x7f0f5c000fc0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 1 -- 172.21.15.66:0/4097517385 shutdown_connections 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x56147530a2e0 0x56147531a530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614753096d0 0x561475308760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561475308db0 0x561475308220 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/4097517385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614752bf4c0 0x5614752c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f57400640 0 client.9123 ms_handle_reset on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 1 -- 172.21.15.66:0/4097517385 >> 172.21.15.66:0/4097517385 conn(0x5614752c5830 msgr2=0x5614752bdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 1 -- 172.21.15.66:0/4097517385 shutdown_connections 2024-03-30T00:49:49.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 1 -- 172.21.15.66:0/4097517385 wait complete. 2024-03-30T00:49:49.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.176+0000 7f0f7be5e000 20 client.9123 trim_cache size 0 max 16384 2024-03-30T00:49:49.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 Processor -- start 2024-03-30T00:49:49.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 -- start start 2024-03-30T00:49:49.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614752bf4c0 0x561475307eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561475308db0 0x5614753083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614753096d0 0x56147530a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561474dca940 con 0x5614752bf4c0 2024-03-30T00:49:49.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614751aa400 con 0x561475308db0 2024-03-30T00:49:49.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614750aac10 con 0x5614753096d0 2024-03-30T00:49:49.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f71e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614752bf4c0 0x561475307eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f71e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614752bf4c0 0x561475307eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49432/0 (socket says 172.21.15.66:49432) 2024-03-30T00:49:49.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f71e00640 1 -- 172.21.15.66:0/3347057668 learned_addr learned my addr 172.21.15.66:0/3347057668 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f71e00640 1 -- 172.21.15.66:0/3347057668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614753096d0 msgr2=0x56147530a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f71e00640 1 --2- 172.21.15.66:0/3347057668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614753096d0 0x56147530a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f71e00640 1 -- 172.21.15.66:0/3347057668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561475308db0 msgr2=0x5614753083f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f71e00640 1 --2- 172.21.15.66:0/3347057668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561475308db0 0x5614753083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f71e00640 1 -- 172.21.15.66:0/3347057668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614752c3b90 con 0x5614752bf4c0 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f71e00640 1 --2- 172.21.15.66:0/3347057668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614752bf4c0 0x561475307eb0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400cae0 tx=0x7f0f64006be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b408e882999cd09 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f63400640 1 -- 172.21.15.66:0/3347057668 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f640240b0 con 0x5614752bf4c0 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f63400640 1 -- 172.21.15.66:0/3347057668 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f64024210 con 0x5614752bf4c0 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f63400640 1 -- 172.21.15.66:0/3347057668 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f640245c0 con 0x5614752bf4c0 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 -- 172.21.15.66:0/3347057668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614752bf4c0 msgr2=0x561475307eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/3347057668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614752bf4c0 0x561475307eb0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400cae0 tx=0x7f0f64006be0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 -- 172.21.15.66:0/3347057668 shutdown_connections 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/3347057668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5614753096d0 0x56147530a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/3347057668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561475308db0 0x5614753083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/3347057668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614752bf4c0 0x561475307eb0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 -- 172.21.15.66:0/3347057668 >> 172.21.15.66:0/3347057668 conn(0x5614752c5830 msgr2=0x561475306790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 -- 172.21.15.66:0/3347057668 shutdown_connections 2024-03-30T00:49:49.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 -- 172.21.15.66:0/3347057668 wait complete. 2024-03-30T00:49:49.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 -- start start 2024-03-30T00:49:49.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.192+0000 7f0f7be5e000 1 Processor -- start 2024-03-30T00:49:49.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614752bf4c0 0x561475367bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561475308db0 0x561475368110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x56147530d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561475150ac0 con 0x5614753096d0 2024-03-30T00:49:49.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614752c3a50 con 0x5614752bf4c0 2024-03-30T00:49:49.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x56147531ece0 con 0x561475308db0 2024-03-30T00:49:49.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 10 client.0 _ll_register_callbacks cb 0x5614750d2590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f72800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x56147530d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f72800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x56147530d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49448/0 (socket says 172.21.15.66:49448) 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f72800640 1 -- 172.21.15.66:0/645140007 learned_addr learned my addr 172.21.15.66:0/645140007 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f71e00640 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614752bf4c0 0x561475367bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f69400640 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561475308db0 0x561475368110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f72800640 1 -- 172.21.15.66:0/645140007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561475308db0 msgr2=0x561475368110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f72800640 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561475308db0 0x561475368110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f72800640 1 -- 172.21.15.66:0/645140007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614752bf4c0 msgr2=0x561475367bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f72800640 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614752bf4c0 0x561475367bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f72800640 1 -- 172.21.15.66:0/645140007 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6400cb90 con 0x5614753096d0 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f69400640 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561475308db0 0x561475368110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:49.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f72800640 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x56147530d340 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0094c0 tx=0x7f0f6c0f0a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bda8690c00c047 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f57400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f57400640 1 -- 172.21.15.66:0/645140007 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c11b070 con 0x5614753096d0 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f57400640 1 -- 172.21.15.66:0/645140007 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f0f6c0ac3a0 con 0x5614753096d0 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 -- 172.21.15.66:0/645140007 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614750dd460 con 0x5614753096d0 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f57400640 1 -- 172.21.15.66:0/645140007 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f6c0ac560 con 0x5614753096d0 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 10 client.9129 fetch_fsmap learned FSMap version 85 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 -- 172.21.15.66:0/645140007 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x5614750b8c00 con 0x5614753096d0 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f57400640 1 -- 172.21.15.66:0/645140007 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614750dd460 con 0x5614753096d0 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f57400640 1 -- 172.21.15.66:0/645140007 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0f6c005d20 con 0x5614753096d0 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 10 client.9129 fetch_fsmap finished waiting for FSMap version 85 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 10 client.9129 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 20 client.9129 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x56147530d880 0x56147530dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 4 client.9129 mds_command: new command op to 9057 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 -- 172.21.15.66:0/645140007 --> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5614750d69c0 con 0x56147530d880 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f71e00640 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x56147530d880 0x56147530dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f57400640 10 client.9129 ms_handle_connect on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f71e00640 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x56147530d880 0x56147530dc60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f202bddfdbf84d21 server_cookie=be0fb817fbc24f15 in_seq=0 out_seq=0 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f57400640 1 -- 172.21.15.66:0/645140007 <== mds.0 v2:172.21.15.66:6835/282507324 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5614750d69c0 con 0x56147530d880 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f57400640 10 client.9129 handle_command_reply: tid=0 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 client.9129 shutdown 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 -- 172.21.15.66:0/645140007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 msgr2=0x56147530d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x56147530d340 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0094c0 tx=0x7f0f6c0f0a90 comp rx=0 tx=0).stop 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 -- 172.21.15.66:0/645140007 shutdown_connections 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x56147530d880 0x56147530dc60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614753096d0 0x56147530d340 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561475308db0 0x561475368110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f57400640 0 client.9129 ms_handle_reset on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 --2- 172.21.15.66:0/645140007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614752bf4c0 0x561475367bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 -- 172.21.15.66:0/645140007 >> 172.21.15.66:0/645140007 conn(0x5614752c5830 msgr2=0x5614752c4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 -- 172.21.15.66:0/645140007 shutdown_connections 2024-03-30T00:49:49.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.196+0000 7f0f7be5e000 1 -- 172.21.15.66:0/645140007 wait complete. 2024-03-30T00:49:49.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f7be5e000 20 client.9129 trim_cache size 0 max 16384 2024-03-30T00:49:49.210 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:49:49.210 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:49:49.210 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:49:49.211 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 9057, 2024-03-30T00:49:49.211 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:49:49.211 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:49:49.211 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:49:49.211 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 4.5700448519999997, 2024-03-30T00:49:49.211 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 85, 2024-03-30T00:49:49.211 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 113, 2024-03-30T00:49:49.211 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 113, 2024-03-30T00:49:49.211 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 5.5026362549999996 2024-03-30T00:49:49.211 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:49:49.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 -- 172.21.15.66:0/901432619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f50066690 msgr2=0x7f0f50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f50066690 0x7f0f50068b50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400a890 tx=0x7f0f64008040 comp rx=0 tx=0).stop 2024-03-30T00:49:49.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 -- 172.21.15.66:0/901432619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7415a640 msgr2=0x7f0f740caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7415a640 0x7f0f740caa70 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c01b060 tx=0x7f0f6c019750 comp rx=0 tx=0).stop 2024-03-30T00:49:49.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 -- 172.21.15.66:0/901432619 shutdown_connections 2024-03-30T00:49:49.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f50066690 0x7f0f50068b50 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7415a640 0x7f0f740caa70 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f740ce520 0x7f0f740ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 --2- 172.21.15.66:0/901432619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f740cb2c0 0x7f0f741637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 -- 172.21.15.66:0/901432619 >> 172.21.15.66:0/901432619 conn(0x7f0f74069f60 msgr2=0x7f0f741649f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 -- 172.21.15.66:0/901432619 shutdown_connections 2024-03-30T00:49:49.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.200+0000 7f0f78a00640 1 -- 172.21.15.66:0/901432619 wait complete. 2024-03-30T00:49:49.224 DEBUG:teuthology.orchestra.run.smithi066:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-30T00:49:49.227 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-03-30T00:49:49.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.364+0000 7f7b25200640 1 Processor -- start 2024-03-30T00:49:49.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.364+0000 7f7b25200640 1 -- start start 2024-03-30T00:49:49.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.364+0000 7f7b25200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b200b1280 0x7f7b200cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.364+0000 7f7b25200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b200ce930 con 0x7f7b200b0b20 2024-03-30T00:49:49.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.364+0000 7f7b25200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b20099800 con 0x7f7b20098a20 2024-03-30T00:49:49.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.364+0000 7f7b25200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b20099d50 con 0x7f7b200b1280 2024-03-30T00:49:49.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.364+0000 7f7b1de00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7b200b0b20 0x7f7b2006bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:47420/0 (socket says 172.21.15.66:47420) 2024-03-30T00:49:49.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.364+0000 7f7b1de00640 1 -- 172.21.15.66:0/1671934374 learned_addr learned my addr 172.21.15.66:0/1671934374 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:49:49.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.364+0000 7f7b17e00640 1 --2- 172.21.15.66:0/1671934374 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b200b1280 0x7f7b200cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.364+0000 7f7b1d400640 1 -- 172.21.15.66:0/1671934374 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235223677 0 0) 0x7f7b200ce930 con 0x7f7b200b0b20 2024-03-30T00:49:49.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.368+0000 7f7b1d400640 1 -- 172.21.15.66:0/1671934374 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b04003530 con 0x7f7b200b0b20 2024-03-30T00:49:49.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.368+0000 7f7b1d400640 1 -- 172.21.15.66:0/1671934374 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682389505 0 0) 0x7f7b20099800 con 0x7f7b20098a20 2024-03-30T00:49:49.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.368+0000 7f7b1d400640 1 -- 172.21.15.66:0/1671934374 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b200ce930 con 0x7f7b20098a20 2024-03-30T00:49:49.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.368+0000 7f7b17e00640 1 -- 172.21.15.66:0/1671934374 >> v1:172.21.15.156:6789/0 conn(0x7f7b20098a20 legacy=0x7f7b200cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.368+0000 7f7b17e00640 1 -- 172.21.15.66:0/1671934374 >> v1:172.21.15.66:6789/0 conn(0x7f7b200b0b20 legacy=0x7f7b2006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.368+0000 7f7b1d400640 1 -- 172.21.15.66:0/1671934374 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3951104078 0 0) 0x7f7b04003530 con 0x7f7b200b0b20 2024-03-30T00:49:49.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.368+0000 7f7b17e00640 1 -- 172.21.15.66:0/1671934374 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b200ceb60 con 0x7f7b200b1280 2024-03-30T00:49:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.368+0000 7f7b17e00640 1 --2- 172.21.15.66:0/1671934374 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b200b1280 0x7f7b200cacb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b080097f0 tx=0x7f7b0801c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f82bb3e819b7d8a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.368+0000 7f7b1d400640 1 -- 172.21.15.66:0/1671934374 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10001680 con 0x7f7b200b1280 2024-03-30T00:49:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.368+0000 7f7b1d400640 1 -- 172.21.15.66:0/1671934374 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b080a7050 con 0x7f7b200b1280 2024-03-30T00:49:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.368+0000 7f7b1d400640 1 -- 172.21.15.66:0/1671934374 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08004250 con 0x7f7b200b1280 2024-03-30T00:49:49.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 -- 172.21.15.66:0/1671934374 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b200b1280 msgr2=0x7f7b200cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 --2- 172.21.15.66:0/1671934374 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b200b1280 0x7f7b200cacb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b080097f0 tx=0x7f7b0801c3d0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 -- 172.21.15.66:0/1671934374 shutdown_connections 2024-03-30T00:49:49.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 --2- 172.21.15.66:0/1671934374 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b200b1280 0x7f7b200cacb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 -- 172.21.15.66:0/1671934374 >> 172.21.15.66:0/1671934374 conn(0x7f7b200b2150 msgr2=0x7f7b200b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 -- 172.21.15.66:0/1671934374 shutdown_connections 2024-03-30T00:49:49.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 -- 172.21.15.66:0/1671934374 wait complete. 2024-03-30T00:49:49.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 Processor -- start 2024-03-30T00:49:49.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 -- start start 2024-03-30T00:49:49.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b20098a20 0x7f7b2014d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b200b0b20 0x7f7b200c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b200b1280 0x7f7b200af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b200c5080 con 0x7f7b200b1280 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b200c70d0 con 0x7f7b200b0b20 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b25200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b200af470 con 0x7f7b20098a20 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b1e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b200b1280 0x7f7b200af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b1e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b200b1280 0x7f7b200af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49450/0 (socket says 172.21.15.66:49450) 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b1e800640 1 -- 172.21.15.66:0/1668050783 learned_addr learned my addr 172.21.15.66:0/1668050783 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b1de00640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b20098a20 0x7f7b2014d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b17e00640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b200b0b20 0x7f7b200c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b1e800640 1 -- 172.21.15.66:0/1668050783 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b20098a20 msgr2=0x7f7b2014d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b1e800640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b20098a20 0x7f7b2014d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b1e800640 1 -- 172.21.15.66:0/1668050783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b200b0b20 msgr2=0x7f7b200c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b1e800640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b200b0b20 0x7f7b200c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b1e800640 1 -- 172.21.15.66:0/1668050783 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b08009420 con 0x7f7b200b1280 2024-03-30T00:49:49.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.372+0000 7f7b1e800640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b200b1280 0x7f7b200af900 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800c790 tx=0x7f7b1800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.376+0000 7f7b16000640 1 -- 172.21.15.66:0/1668050783 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1800cea0 con 0x7f7b200b1280 2024-03-30T00:49:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.376+0000 7f7b16000640 1 -- 172.21.15.66:0/1668050783 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b18003b80 con 0x7f7b200b1280 2024-03-30T00:49:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.376+0000 7f7b16000640 1 -- 172.21.15.66:0/1668050783 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1800a850 con 0x7f7b200b1280 2024-03-30T00:49:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.376+0000 7f7b25200640 1 -- 172.21.15.66:0/1668050783 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b200ae860 con 0x7f7b200b1280 2024-03-30T00:49:49.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.376+0000 7f7b25200640 1 -- 172.21.15.66:0/1668050783 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b200aee20 con 0x7f7b200b1280 2024-03-30T00:49:49.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.376+0000 7f7b16000640 1 -- 172.21.15.66:0/1668050783 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7b1800a9b0 con 0x7f7b200b1280 2024-03-30T00:49:49.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.376+0000 7f7b16000640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7afc066690 0x7f7afc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b16000640 1 -- 172.21.15.66:0/1668050783 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7b18084190 con 0x7f7b200b1280 2024-03-30T00:49:49.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 Processor -- start 2024-03-30T00:49:49.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1de00640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7afc066690 0x7f7afc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 -- start start 2024-03-30T00:49:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1de00640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7afc066690 0x7f7afc068b50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7b10000f70 tx=0x7f7b10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46147d30 0x561a46148110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46148650 0x561a460fea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460ff340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a45fc6170 con 0x561a460fef60 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a45f53d80 con 0x561a46148650 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a45f51350 con 0x561a46147d30 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46147d30 0x561a46148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460ff340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49466/0 (socket says 172.21.15.66:49466) 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1e800640 1 -- 172.21.15.66:0/413162061 learned_addr learned my addr 172.21.15.66:0/413162061 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b17e00640 1 --2- 172.21.15.66:0/413162061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46148650 0x561a460fea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1e800640 1 -- 172.21.15.66:0/413162061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46147d30 msgr2=0x561a46148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1e800640 1 --2- 172.21.15.66:0/413162061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46147d30 0x561a46148110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1e800640 1 -- 172.21.15.66:0/413162061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46148650 msgr2=0x561a460fea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1e800640 1 --2- 172.21.15.66:0/413162061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46148650 0x561a460fea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1e800640 1 -- 172.21.15.66:0/413162061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a45c49070 con 0x561a460fef60 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b17e00640 1 --2- 172.21.15.66:0/413162061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46148650 0x561a460fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b1e800640 1 --2- 172.21.15.66:0/413162061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460ff340 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f7b18055030 tx=0x7f7b180014e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0440b2f45674c3a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b0ea00640 1 -- 172.21.15.66:0/413162061 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18058ed0 con 0x561a460fef60 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b0ea00640 1 -- 172.21.15.66:0/413162061 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1805d490 con 0x561a460fef60 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b0ea00640 1 -- 172.21.15.66:0/413162061 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18053440 con 0x561a460fef60 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 -- 172.21.15.66:0/413162061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 msgr2=0x561a460ff340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 --2- 172.21.15.66:0/413162061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460ff340 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f7b18055030 tx=0x7f7b180014e0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 -- 172.21.15.66:0/413162061 shutdown_connections 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 --2- 172.21.15.66:0/413162061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460ff340 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 --2- 172.21.15.66:0/413162061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46148650 0x561a460fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 --2- 172.21.15.66:0/413162061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46147d30 0x561a46148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 -- 172.21.15.66:0/413162061 >> 172.21.15.66:0/413162061 conn(0x561a460faf10 msgr2=0x561a460fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 -- 172.21.15.66:0/413162061 shutdown_connections 2024-03-30T00:49:49.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 -- 172.21.15.66:0/413162061 wait complete. 2024-03-30T00:49:49.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.380+0000 7f7b286bd000 1 -- start start 2024-03-30T00:49:49.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 1 Processor -- start 2024-03-30T00:49:49.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a46104970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46147d30 0x561a46134bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a46135130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a45f4cc40 con 0x561a460fef60 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a45eebfa0 con 0x561a46147d30 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a45f0a1c0 con 0x561a46148650 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 10 client.0 _ll_register_callbacks cb 0x561a460ea060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b1de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a46104970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b1de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a46104970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49474/0 (socket says 172.21.15.66:49474) 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b1de00640 1 -- 172.21.15.66:0/3098050886 learned_addr learned my addr 172.21.15.66:0/3098050886 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b17e00640 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46147d30 0x561a46134bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b1e800640 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a46135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b1de00640 1 -- 172.21.15.66:0/3098050886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 msgr2=0x561a46135130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b1de00640 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a46135130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b1de00640 1 -- 172.21.15.66:0/3098050886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46147d30 msgr2=0x561a46134bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b1de00640 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46147d30 0x561a46134bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b1de00640 1 -- 172.21.15.66:0/3098050886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b200aee20 con 0x561a460fef60 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b1de00640 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a46104970 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f7b100073b0 tx=0x7f7b10001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8c6a4910fce1ab server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b03400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b03400640 1 -- 172.21.15.66:0/3098050886 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10013420 con 0x561a460fef60 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b03400640 1 -- 172.21.15.66:0/3098050886 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7b10022c50 con 0x561a460fef60 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 1 -- 172.21.15.66:0/3098050886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a45f49a20 con 0x561a460fef60 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b03400640 1 -- 172.21.15.66:0/3098050886 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b10022e10 con 0x561a460fef60 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 10 client.9141 fetch_fsmap learned FSMap version 85 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 1 -- 172.21.15.66:0/3098050886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x561a45f11ba0 con 0x561a460fef60 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b03400640 1 -- 172.21.15.66:0/3098050886 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561a45f49a20 con 0x561a460fef60 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b03400640 1 -- 172.21.15.66:0/3098050886 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7b1002a070 con 0x561a460fef60 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 10 client.9141 fetch_fsmap finished waiting for FSMap version 85 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 10 client.9141 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:49:49.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 20 client.9141 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:49.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x561a46149110 0x561a461494f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 4 client.9141 mds_command: new command op to 9057 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:49:49.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b286bd000 1 -- 172.21.15.66:0/3098050886 --> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a45f4b590 con 0x561a46149110 2024-03-30T00:49:49.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.384+0000 7f7b17e00640 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x561a46149110 0x561a461494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b17e00640 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x561a46149110 0x561a461494f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d77504a1293b4ea server_cookie=1335ae8705d3dd61 in_seq=0 out_seq=0 2024-03-30T00:49:49.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b03400640 10 client.9141 ms_handle_connect on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b03400640 1 -- 172.21.15.66:0/3098050886 <== mds.0 v2:172.21.15.66:6835/282507324 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561a45f4b590 con 0x561a46149110 2024-03-30T00:49:49.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b03400640 10 client.9141 handle_command_reply: tid=0 2024-03-30T00:49:49.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 1 client.9141 shutdown 2024-03-30T00:49:49.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 1 -- 172.21.15.66:0/3098050886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 msgr2=0x561a46104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a46104970 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f7b100073b0 tx=0x7f7b10001160 comp rx=0 tx=0).stop 2024-03-30T00:49:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 1 -- 172.21.15.66:0/3098050886 shutdown_connections 2024-03-30T00:49:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x561a46149110 0x561a461494f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a46135130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46147d30 0x561a46134bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 1 --2- 172.21.15.66:0/3098050886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a46104970 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b03400640 0 client.9141 ms_handle_reset on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 1 -- 172.21.15.66:0/3098050886 >> 172.21.15.66:0/3098050886 conn(0x561a460faf10 msgr2=0x561a460fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 1 -- 172.21.15.66:0/3098050886 shutdown_connections 2024-03-30T00:49:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 1 -- 172.21.15.66:0/3098050886 wait complete. 2024-03-30T00:49:49.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.388+0000 7f7b286bd000 20 client.9141 trim_cache size 0 max 16384 2024-03-30T00:49:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42be000640 1 Processor -- start 2024-03-30T00:49:49.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42be000640 1 -- start start 2024-03-30T00:49:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42be000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f42b80ce680 0x7f42b80b0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42be000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f42b80ccf80 0x7f42b80cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42be000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b80cb2f0 con 0x7f42b815aa50 2024-03-30T00:49:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42be000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b80cd8f0 con 0x7f42b80ce680 2024-03-30T00:49:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42be000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b815b540 con 0x7f42b80ccf80 2024-03-30T00:49:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42afe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f42b80ce680 0x7f42b80b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42afe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f42b80ce680 0x7f42b80b0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49488/0 (socket says 172.21.15.66:49488) 2024-03-30T00:49:49.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42afe00640 1 -- 172.21.15.66:0/2272543119 learned_addr learned my addr 172.21.15.66:0/2272543119 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42b7e00640 1 --2- 172.21.15.66:0/2272543119 >> v2:172.21.15.156:3300/0 conn(0x7f42b80ccf80 0x7f42b80cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42afe00640 1 -- 172.21.15.66:0/2272543119 >> v2:172.21.15.156:3300/0 conn(0x7f42b80ccf80 msgr2=0x7f42b80cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42afe00640 1 --2- 172.21.15.66:0/2272543119 >> v2:172.21.15.156:3300/0 conn(0x7f42b80ccf80 0x7f42b80cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42afe00640 1 -- 172.21.15.66:0/2272543119 >> v1:172.21.15.156:6789/0 conn(0x7f42b815aa50 legacy=0x7f42b815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42afe00640 1 -- 172.21.15.66:0/2272543119 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b815c060 con 0x7f42b80ce680 2024-03-30T00:49:49.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.392+0000 7f42b7e00640 1 --2- 172.21.15.66:0/2272543119 >> v2:172.21.15.156:3300/0 conn(0x7f42b80ccf80 0x7f42b80cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:49.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42afe00640 1 --2- 172.21.15.66:0/2272543119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42b80ce680 0x7f42b80b0340 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f42b80d05f0 tx=0x7f42a001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c7d39ca30f9b13 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42b6a00640 1 -- 172.21.15.66:0/2272543119 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a00ad640 con 0x7f42b80ce680 2024-03-30T00:49:49.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42b6a00640 1 -- 172.21.15.66:0/2272543119 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a00ad7a0 con 0x7f42b80ce680 2024-03-30T00:49:49.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42b6a00640 1 -- 172.21.15.66:0/2272543119 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a00adb30 con 0x7f42b80ce680 2024-03-30T00:49:49.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42be000640 1 -- 172.21.15.66:0/2272543119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42b80ce680 msgr2=0x7f42b80b0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42be000640 1 --2- 172.21.15.66:0/2272543119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42b80ce680 0x7f42b80b0340 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f42b80d05f0 tx=0x7f42a001c390 comp rx=0 tx=0).stop 2024-03-30T00:49:49.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42be000640 1 -- 172.21.15.66:0/2272543119 shutdown_connections 2024-03-30T00:49:49.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42be000640 1 --2- 172.21.15.66:0/2272543119 >> v2:172.21.15.156:3300/0 conn(0x7f42b80ccf80 0x7f42b80cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42be000640 1 --2- 172.21.15.66:0/2272543119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42b80ce680 0x7f42b80b0340 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42be000640 1 -- 172.21.15.66:0/2272543119 >> 172.21.15.66:0/2272543119 conn(0x7f42b80d0620 msgr2=0x7f42b80d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42be000640 1 -- 172.21.15.66:0/2272543119 shutdown_connections 2024-03-30T00:49:49.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.396+0000 7f42be000640 1 -- 172.21.15.66:0/2272543119 wait complete. 2024-03-30T00:49:49.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42be000640 1 Processor -- start 2024-03-30T00:49:49.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42be000640 1 -- start start 2024-03-30T00:49:49.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42be000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42b80ccf80 0x7f42b80b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42be000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42b80ce680 0x7f42b80b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42be000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42b815aa50 0x7f42b81f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42be000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80afba0 con 0x7f42b815aa50 2024-03-30T00:49:49.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42be000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80c4150 con 0x7f42b80ce680 2024-03-30T00:49:49.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42be000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80c4430 con 0x7f42b80ccf80 2024-03-30T00:49:49.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42b7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42b815aa50 0x7f42b81f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42b7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42b815aa50 0x7f42b81f8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49494/0 (socket says 172.21.15.66:49494) 2024-03-30T00:49:49.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42b7e00640 1 -- 172.21.15.66:0/263904469 learned_addr learned my addr 172.21.15.66:0/263904469 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42afe00640 1 --2- 172.21.15.66:0/263904469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42b80ce680 0x7f42b80b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42b7400640 1 --2- 172.21.15.66:0/263904469 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42b80ccf80 0x7f42b80b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42b7e00640 1 -- 172.21.15.66:0/263904469 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42b80ccf80 msgr2=0x7f42b80b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42b7e00640 1 --2- 172.21.15.66:0/263904469 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42b80ccf80 0x7f42b80b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42b7e00640 1 -- 172.21.15.66:0/263904469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42b80ce680 msgr2=0x7f42b80b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42b7e00640 1 --2- 172.21.15.66:0/263904469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42b80ce680 0x7f42b80b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42b7e00640 1 -- 172.21.15.66:0/263904469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a00094d0 con 0x7f42b815aa50 2024-03-30T00:49:49.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42b7e00640 1 --2- 172.21.15.66:0/263904469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42b815aa50 0x7f42b81f8280 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f42b000cef0 tx=0x7f42b000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42af400640 1 -- 172.21.15.66:0/263904469 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b000d840 con 0x7f42b815aa50 2024-03-30T00:49:49.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42af400640 1 -- 172.21.15.66:0/263904469 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b000d9c0 con 0x7f42b815aa50 2024-03-30T00:49:49.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42be000640 1 -- 172.21.15.66:0/263904469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b80c4710 con 0x7f42b815aa50 2024-03-30T00:49:49.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42af400640 1 -- 172.21.15.66:0/263904469 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b000dd70 con 0x7f42b815aa50 2024-03-30T00:49:49.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.400+0000 7f42be000640 1 -- 172.21.15.66:0/263904469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b80c4fb0 con 0x7f42b815aa50 2024-03-30T00:49:49.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f42be000640 1 -- 172.21.15.66:0/263904469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f42b80b1f10 con 0x7f42b815aa50 2024-03-30T00:49:49.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f42af400640 1 -- 172.21.15.66:0/263904469 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f42b001e3b0 con 0x7f42b815aa50 2024-03-30T00:49:49.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f42af400640 1 --2- 172.21.15.66:0/263904469 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4294066690 0x7f4294068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f42af400640 1 -- 172.21.15.66:0/263904469 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f42b0084d00 con 0x7f42b815aa50 2024-03-30T00:49:49.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b286bd000 1 Processor -- start 2024-03-30T00:49:49.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f42b7400640 1 --2- 172.21.15.66:0/263904469 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4294066690 0x7f4294068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f42af400640 1 -- 172.21.15.66:0/263904469 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+447 (secure 0 0 0) 0x7f42b0087260 con 0x7f42b815aa50 2024-03-30T00:49:49.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b286bd000 1 -- start start 2024-03-30T00:49:49.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f42be000640 1 -- 172.21.15.66:0/263904469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f42b8066c30 con 0x7f42b815aa50 2024-03-30T00:49:49.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f42b7400640 1 --2- 172.21.15.66:0/263904469 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4294066690 0x7f4294068b50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f42a80098e0 tx=0x7f42a8009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46147d30 0x561a460fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a461590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b286bd000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a45c09cf0 con 0x561a460fef60 2024-03-30T00:49:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b286bd000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a45f48430 con 0x561a46147d30 2024-03-30T00:49:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b286bd000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a45f4bad0 con 0x561a46148650 2024-03-30T00:49:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b1de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b1de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49506/0 (socket says 172.21.15.66:49506) 2024-03-30T00:49:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b1de00640 1 -- 172.21.15.66:0/3443277226 learned_addr learned my addr 172.21.15.66:0/3443277226 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b1e800640 1 --2- 172.21.15.66:0/3443277226 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a461590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.404+0000 7f7b17e00640 1 --2- 172.21.15.66:0/3443277226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46147d30 0x561a460fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b1de00640 1 -- 172.21.15.66:0/3443277226 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 msgr2=0x561a461590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b1de00640 1 --2- 172.21.15.66:0/3443277226 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a461590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b1de00640 1 -- 172.21.15.66:0/3443277226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46147d30 msgr2=0x561a460fe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b1de00640 1 --2- 172.21.15.66:0/3443277226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46147d30 0x561a460fe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b1de00640 1 -- 172.21.15.66:0/3443277226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a45ef7ef0 con 0x561a460fef60 2024-03-30T00:49:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b1de00640 1 --2- 172.21.15.66:0/3443277226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460fe2a0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f7b10005fd0 tx=0x7f7b1001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b17608aeb2abf46 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b0f400640 1 -- 172.21.15.66:0/3443277226 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1000eb80 con 0x561a460fef60 2024-03-30T00:49:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b0f400640 1 -- 172.21.15.66:0/3443277226 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b10013420 con 0x561a460fef60 2024-03-30T00:49:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b0f400640 1 -- 172.21.15.66:0/3443277226 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10013670 con 0x561a460fef60 2024-03-30T00:49:49.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 -- 172.21.15.66:0/3443277226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 msgr2=0x561a460fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 --2- 172.21.15.66:0/3443277226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460fe2a0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f7b10005fd0 tx=0x7f7b1001a000 comp rx=0 tx=0).stop 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 -- 172.21.15.66:0/3443277226 shutdown_connections 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 --2- 172.21.15.66:0/3443277226 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a461590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 --2- 172.21.15.66:0/3443277226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a46147d30 0x561a460fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 --2- 172.21.15.66:0/3443277226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a460fef60 0x561a460fe2a0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 -- 172.21.15.66:0/3443277226 >> 172.21.15.66:0/3443277226 conn(0x561a460faf10 msgr2=0x561a460fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 -- 172.21.15.66:0/3443277226 shutdown_connections 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 -- 172.21.15.66:0/3443277226 wait complete. 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 -- start start 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 Processor -- start 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a460fef60 0x561a4618da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a46147d30 0x561a4615c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a4615c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a45f13420 con 0x561a46147d30 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a45f900c0 con 0x561a460fef60 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a45ee9d30 con 0x561a46148650 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b286bd000 10 client.0 _ll_register_callbacks cb 0x561a45fc60c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b17e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a46147d30 0x561a4615c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b17e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a46147d30 0x561a4615c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49516/0 (socket says 172.21.15.66:49516) 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b17e00640 1 -- 172.21.15.66:0/1436376844 learned_addr learned my addr 172.21.15.66:0/1436376844 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b1de00640 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a460fef60 0x561a4618da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b1e800640 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a4615c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b17e00640 1 -- 172.21.15.66:0/1436376844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 msgr2=0x561a4615c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b17e00640 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a4615c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b17e00640 1 -- 172.21.15.66:0/1436376844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a460fef60 msgr2=0x561a4618da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b17e00640 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a460fef60 0x561a4618da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.408+0000 7f7b17e00640 1 -- 172.21.15.66:0/1436376844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a45f11ba0 con 0x561a46147d30 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b17e00640 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a46147d30 0x561a4615c2f0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f7b080097f0 tx=0x7f7b0801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2cedf658c24a48 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b03400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b03400640 1 -- 172.21.15.66:0/1436376844 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b080c7070 con 0x561a46147d30 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b03400640 1 -- 172.21.15.66:0/1436376844 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7b08004350 con 0x561a46147d30 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 -- 172.21.15.66:0/1436376844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a45f095f0 con 0x561a46147d30 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b03400640 1 -- 172.21.15.66:0/1436376844 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b080ba040 con 0x561a46147d30 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 10 client.9153 fetch_fsmap learned FSMap version 85 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 -- 172.21.15.66:0/1436376844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x561a45eeba60 con 0x561a46147d30 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b03400640 1 -- 172.21.15.66:0/1436376844 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561a45f095f0 con 0x561a46147d30 2024-03-30T00:49:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b03400640 1 -- 172.21.15.66:0/1436376844 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7b080c2980 con 0x561a46147d30 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 10 client.9153 fetch_fsmap finished waiting for FSMap version 85 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 10 client.9153 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 20 client.9153 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x561a4615cd70 0x561a4615d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 4 client.9153 mds_command: new command op to 9057 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 -- 172.21.15.66:0/1436376844 --> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x561a45f0b410 con 0x561a4615cd70 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b1de00640 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x561a4615cd70 0x561a4615d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b1de00640 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x561a4615cd70 0x561a4615d180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd9af3045968dc server_cookie=bb244e6baca2bc5f in_seq=0 out_seq=0 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b03400640 10 client.9153 ms_handle_connect on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b03400640 1 -- 172.21.15.66:0/1436376844 <== mds.0 v2:172.21.15.66:6835/282507324 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561a45f0b410 con 0x561a4615cd70 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b03400640 10 client.9153 handle_command_reply: tid=0 2024-03-30T00:49:49.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 client.9153 shutdown 2024-03-30T00:49:49.426 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f76aa600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-03-30T00:49:49.426 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f76aa600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-03-30T00:49:49.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 -- 172.21.15.66:0/1436376844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a46147d30 msgr2=0x561a4615c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a46147d30 0x561a4615c2f0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f7b080097f0 tx=0x7f7b0801ac10 comp rx=0 tx=0).stop 2024-03-30T00:49:49.426 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f76aa600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-03-30T00:49:49.427 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f76aa600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-03-30T00:49:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 -- 172.21.15.66:0/1436376844 shutdown_connections 2024-03-30T00:49:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x561a4615cd70 0x561a4615d180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x561a46148650 0x561a4615c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561a46147d30 0x561a4615c2f0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 --2- 172.21.15.66:0/1436376844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a460fef60 0x561a4618da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 -- 172.21.15.66:0/1436376844 >> 172.21.15.66:0/1436376844 conn(0x561a460faf10 msgr2=0x561a460fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b03400640 0 client.9153 ms_handle_reset on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 -- 172.21.15.66:0/1436376844 shutdown_connections 2024-03-30T00:49:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 1 -- 172.21.15.66:0/1436376844 wait complete. 2024-03-30T00:49:49.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.412+0000 7f7b286bd000 20 client.9153 trim_cache size 0 max 16384 2024-03-30T00:49:49.428 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:49:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T00:49:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "aa831b09-e5ae-406e-9cc0-0742aaad7496", 2024-03-30T00:49:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T00:49:49.428 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:49:49.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 -- 172.21.15.66:0/1668050783 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7afc066690 msgr2=0x7f7afc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7afc066690 0x7f7afc068b50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7b10000f70 tx=0x7f7b10009040 comp rx=0 tx=0).stop 2024-03-30T00:49:49.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 -- 172.21.15.66:0/1668050783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b200b1280 msgr2=0x7f7b200af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b200b1280 0x7f7b200af900 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800c790 tx=0x7f7b1800cc50 comp rx=0 tx=0).stop 2024-03-30T00:49:49.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 -- 172.21.15.66:0/1668050783 shutdown_connections 2024-03-30T00:49:49.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7afc066690 0x7f7afc068b50 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b200b1280 0x7f7b200af900 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b200b0b20 0x7f7b200c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 --2- 172.21.15.66:0/1668050783 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b20098a20 0x7f7b2014d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 -- 172.21.15.66:0/1668050783 >> 172.21.15.66:0/1668050783 conn(0x7f7b200b2150 msgr2=0x7f7b2006f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 -- 172.21.15.66:0/1668050783 shutdown_connections 2024-03-30T00:49:49.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.416+0000 7f7b25200640 1 -- 172.21.15.66:0/1668050783 wait complete. 2024-03-30T00:49:49.441 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-30T00:49:49.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.576+0000 7fae0da00640 1 Processor -- start 2024-03-30T00:49:49.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.576+0000 7fae0da00640 1 -- start start 2024-03-30T00:49:49.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.576+0000 7fae0da00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fae0815a640 0x7fae0806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fae0da00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae080ce520 0x7fae080ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fae0da00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae080cabb0 0x7fae080cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fae0da00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0806d070 con 0x7fae080ce520 2024-03-30T00:49:49.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fae0da00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae080cbba0 con 0x7fae0815a640 2024-03-30T00:49:49.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fae0da00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae080cee90 con 0x7fae080cabb0 2024-03-30T00:49:49.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fadfe000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae080ce520 0x7fae080ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fadfe000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae080ce520 0x7fae080ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49524/0 (socket says 172.21.15.66:49524) 2024-03-30T00:49:49.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fadfe000640 1 -- 172.21.15.66:0/3967497298 learned_addr learned my addr 172.21.15.66:0/3967497298 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fae07400640 1 --2- 172.21.15.66:0/3967497298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae080cabb0 0x7fae080cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fae06a00640 1 --2- 172.21.15.66:0/3967497298 >> v2:172.21.15.156:3300/0 conn(0x7fae0815a640 0x7fae0806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fadfe000640 1 -- 172.21.15.66:0/3967497298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae080cabb0 msgr2=0x7fae080cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fadfe000640 1 --2- 172.21.15.66:0/3967497298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae080cabb0 0x7fae080cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fadfe000640 1 -- 172.21.15.66:0/3967497298 >> v2:172.21.15.156:3300/0 conn(0x7fae0815a640 msgr2=0x7fae0806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fadfe000640 1 --2- 172.21.15.66:0/3967497298 >> v2:172.21.15.156:3300/0 conn(0x7fae0815a640 0x7fae0806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fadfe000640 1 -- 172.21.15.66:0/3967497298 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0806e2c0 con 0x7fae080ce520 2024-03-30T00:49:49.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fadfe000640 1 --2- 172.21.15.66:0/3967497298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae080ce520 0x7fae080ce950 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fae080cf160 tx=0x7fadf001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b71e88e591a6cd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fae06000640 1 -- 172.21.15.66:0/3967497298 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00aa610 con 0x7fae080ce520 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fae06000640 1 -- 172.21.15.66:0/3967497298 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf00a6040 con 0x7fae080ce520 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.580+0000 7fae06000640 1 -- 172.21.15.66:0/3967497298 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00aa990 con 0x7fae080ce520 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 -- 172.21.15.66:0/3967497298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae080ce520 msgr2=0x7fae080ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 --2- 172.21.15.66:0/3967497298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae080ce520 0x7fae080ce950 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fae080cf160 tx=0x7fadf001c4f0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 -- 172.21.15.66:0/3967497298 shutdown_connections 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 --2- 172.21.15.66:0/3967497298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae080cabb0 0x7fae080cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 --2- 172.21.15.66:0/3967497298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae080ce520 0x7fae080ce950 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 --2- 172.21.15.66:0/3967497298 >> v2:172.21.15.156:3300/0 conn(0x7fae0815a640 0x7fae0806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 -- 172.21.15.66:0/3967497298 >> 172.21.15.66:0/3967497298 conn(0x7fae08069f60 msgr2=0x7fae0806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 -- 172.21.15.66:0/3967497298 shutdown_connections 2024-03-30T00:49:49.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 -- 172.21.15.66:0/3967497298 wait complete. 2024-03-30T00:49:49.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 Processor -- start 2024-03-30T00:49:49.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 -- start start 2024-03-30T00:49:49.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae080cabb0 0x7fae080ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae080ce520 0x7fae080c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae0815a640 0x7fae080c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae080c3110 con 0x7fae0815a640 2024-03-30T00:49:49.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae080c7ea0 con 0x7fae080ce520 2024-03-30T00:49:49.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae0da00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae080c8180 con 0x7fae080cabb0 2024-03-30T00:49:49.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae07400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae0815a640 0x7fae080c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae07400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae0815a640 0x7fae080c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49532/0 (socket says 172.21.15.66:49532) 2024-03-30T00:49:49.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae07400640 1 -- 172.21.15.66:0/2353534473 learned_addr learned my addr 172.21.15.66:0/2353534473 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae06a00640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae080cabb0 0x7fae080ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fadfe000640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae080ce520 0x7fae080c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae07400640 1 -- 172.21.15.66:0/2353534473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae080cabb0 msgr2=0x7fae080ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae07400640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae080cabb0 0x7fae080ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae07400640 1 -- 172.21.15.66:0/2353534473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae080ce520 msgr2=0x7fae080c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae07400640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae080ce520 0x7fae080c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fae07400640 1 -- 172.21.15.66:0/2353534473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf00094d0 con 0x7fae0815a640 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.584+0000 7fadfe000640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae080ce520 0x7fae080c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fae07400640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae0815a640 0x7fae080c74a0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fae0000d800 tx=0x7fae0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fadff400640 1 -- 172.21.15.66:0/2353534473 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae000096c0 con 0x7fae0815a640 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fadff400640 1 -- 172.21.15.66:0/2353534473 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae00009820 con 0x7fae0815a640 2024-03-30T00:49:49.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fae0da00640 1 -- 172.21.15.66:0/2353534473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae080c3320 con 0x7fae0815a640 2024-03-30T00:49:49.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fadff400640 1 -- 172.21.15.66:0/2353534473 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00009bd0 con 0x7fae0815a640 2024-03-30T00:49:49.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fae0da00640 1 -- 172.21.15.66:0/2353534473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae080c6660 con 0x7fae0815a640 2024-03-30T00:49:49.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fadff400640 1 -- 172.21.15.66:0/2353534473 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fae00009d30 con 0x7fae0815a640 2024-03-30T00:49:49.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fadff400640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fade4066690 0x7fade4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fadff400640 1 -- 172.21.15.66:0/2353534473 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fae00084980 con 0x7fae0815a640 2024-03-30T00:49:49.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fae06a00640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fade4066690 0x7fade4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fae10dad000 1 Processor -- start 2024-03-30T00:49:49.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.588+0000 7fae10dad000 1 -- start start 2024-03-30T00:49:49.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae06a00640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fade4066690 0x7fade4068b50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fadf8003ab0 tx=0x7fadf8009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180866e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f180867390 0x55f18081a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 0x55f18081ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1806e5630 con 0x55f180866a70 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f180672bd0 con 0x55f18081aa00 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1806701a0 con 0x55f180867390 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae06a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180866e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae06a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180866e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49544/0 (socket says 172.21.15.66:49544) 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae06a00640 1 -- 172.21.15.66:0/2918818795 learned_addr learned my addr 172.21.15.66:0/2918818795 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae07400640 1 --2- 172.21.15.66:0/2918818795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 0x55f18081ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fadfe000640 1 --2- 172.21.15.66:0/2918818795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f180867390 0x55f18081a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae06a00640 1 -- 172.21.15.66:0/2918818795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f180867390 msgr2=0x55f18081a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae06a00640 1 --2- 172.21.15.66:0/2918818795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f180867390 0x55f18081a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae06a00640 1 -- 172.21.15.66:0/2918818795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 msgr2=0x55f18081ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae06a00640 1 --2- 172.21.15.66:0/2918818795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 0x55f18081ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae06a00640 1 -- 172.21.15.66:0/2918818795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f180367e20 con 0x55f180866a70 2024-03-30T00:49:49.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae07400640 1 --2- 172.21.15.66:0/2918818795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 0x55f18081ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:49.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fadfe000640 1 --2- 172.21.15.66:0/2918818795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f180867390 0x55f18081a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:49:49.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae06a00640 1 --2- 172.21.15.66:0/2918818795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180866e50 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fadf8004840 tx=0x7fadf8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3674f6d1983ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fadf7400640 1 -- 172.21.15.66:0/2918818795 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8017e40 con 0x55f180866a70 2024-03-30T00:49:49.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fadf7400640 1 -- 172.21.15.66:0/2918818795 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf80155f0 con 0x55f180866a70 2024-03-30T00:49:49.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fadf7400640 1 -- 172.21.15.66:0/2918818795 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8015960 con 0x55f180866a70 2024-03-30T00:49:49.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 -- 172.21.15.66:0/2918818795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 msgr2=0x55f180866e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 --2- 172.21.15.66:0/2918818795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180866e50 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fadf8004840 tx=0x7fadf8013000 comp rx=0 tx=0).stop 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 -- 172.21.15.66:0/2918818795 shutdown_connections 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 --2- 172.21.15.66:0/2918818795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 0x55f18081ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 --2- 172.21.15.66:0/2918818795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f180867390 0x55f18081a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 --2- 172.21.15.66:0/2918818795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180866e50 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 -- 172.21.15.66:0/2918818795 >> 172.21.15.66:0/2918818795 conn(0x55f18061b160 msgr2=0x55f180849820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 -- 172.21.15.66:0/2918818795 shutdown_connections 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 -- 172.21.15.66:0/2918818795 wait complete. 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 -- start start 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.592+0000 7fae10dad000 1 Processor -- start 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f18081aa00 0x55f180822ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f180866a70 0x55f18084aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f180867390 0x55f18084af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18066ba90 con 0x55f18081aa00 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18060ad40 con 0x55f180867390 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f180628f30 con 0x55f180866a70 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 10 client.0 _ll_register_callbacks cb 0x55f1806cd810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae06a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f18081aa00 0x55f180822ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae06a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f18081aa00 0x55f180822ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49546/0 (socket says 172.21.15.66:49546) 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae06a00640 1 -- 172.21.15.66:0/688408038 learned_addr learned my addr 172.21.15.66:0/688408038 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadfe000640 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f180866a70 0x55f18084aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae07400640 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f180867390 0x55f18084af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae06a00640 1 -- 172.21.15.66:0/688408038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f180866a70 msgr2=0x55f18084aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae06a00640 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f180866a70 0x55f18084aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae06a00640 1 -- 172.21.15.66:0/688408038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f180867390 msgr2=0x55f18084af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae06a00640 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f180867390 0x55f18084af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae06a00640 1 -- 172.21.15.66:0/688408038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf8011040 con 0x55f18081aa00 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae07400640 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f180867390 0x55f18084af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:49.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae06a00640 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f18081aa00 0x55f180822ad0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fadf8004410 tx=0x7fadf8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2c2c8b5392b09d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadebe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadebe00640 1 -- 172.21.15.66:0/688408038 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8015e30 con 0x55f18081aa00 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadebe00640 1 -- 172.21.15.66:0/688408038 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fadf8017e40 con 0x55f18081aa00 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 -- 172.21.15.66:0/688408038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f18066a3e0 con 0x55f18081aa00 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadebe00640 1 -- 172.21.15.66:0/688408038 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadf802b410 con 0x55f18081aa00 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 10 client.9165 fetch_fsmap learned FSMap version 85 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 -- 172.21.15.66:0/688408038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55f180630aa0 con 0x55f18081aa00 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadebe00640 1 -- 172.21.15.66:0/688408038 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f18066a3e0 con 0x55f18081aa00 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadebe00640 1 -- 172.21.15.66:0/688408038 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fadf8031020 con 0x55f18081aa00 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 10 client.9165 fetch_fsmap finished waiting for FSMap version 85 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 10 client.9165 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:49:49.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 20 client.9165 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55f1808761e0 0x55f1808785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 4 client.9165 mds_command: new command op to 9057 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 -- 172.21.15.66:0/688408038 --> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f18063bd00 con 0x55f1808761e0 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadfe000640 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55f1808761e0 0x55f1808785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadebe00640 10 client.9165 ms_handle_connect on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadfe000640 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55f1808761e0 0x55f1808785d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd119e062af8d71a server_cookie=65f182600d888090 in_seq=0 out_seq=0 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadebe00640 1 -- 172.21.15.66:0/688408038 <== mds.0 v2:172.21.15.66:6835/282507324 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f18063bd00 con 0x55f1808761e0 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fadebe00640 10 client.9165 handle_command_reply: tid=0 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 client.9165 shutdown 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 -- 172.21.15.66:0/688408038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f18081aa00 msgr2=0x55f180822ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.596+0000 7fae10dad000 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f18081aa00 0x55f180822ad0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fadf8004410 tx=0x7fadf8004ac0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.600+0000 7fae10dad000 1 -- 172.21.15.66:0/688408038 shutdown_connections 2024-03-30T00:49:49.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.600+0000 7fae10dad000 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55f1808761e0 0x55f1808785d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.600+0000 7fae10dad000 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f180867390 0x55f18084af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.600+0000 7fae10dad000 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f180866a70 0x55f18084aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.600+0000 7fae10dad000 1 --2- 172.21.15.66:0/688408038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f18081aa00 0x55f180822ad0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.600+0000 7fadebe00640 0 client.9165 ms_handle_reset on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.600+0000 7fae10dad000 1 -- 172.21.15.66:0/688408038 >> 172.21.15.66:0/688408038 conn(0x55f18061b160 msgr2=0x55f180819bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.600+0000 7fae10dad000 1 -- 172.21.15.66:0/688408038 shutdown_connections 2024-03-30T00:49:49.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.600+0000 7fae10dad000 1 -- 172.21.15.66:0/688408038 wait complete. 2024-03-30T00:49:49.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.600+0000 7fae10dad000 20 client.9165 trim_cache size 0 max 16384 2024-03-30T00:49:49.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.612+0000 7fae10dad000 1 Processor -- start 2024-03-30T00:49:49.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.612+0000 7fae10dad000 1 -- start start 2024-03-30T00:49:49.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.612+0000 7fae10dad000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f18061b160 0x55f1808219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 0x55f180821f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180881990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f180328aa0 con 0x55f180866a70 2024-03-30T00:49:49.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1806692a0 con 0x55f18081aa00 2024-03-30T00:49:49.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f18066e5f0 con 0x55f18061b160 2024-03-30T00:49:49.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180881990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180881990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49550/0 (socket says 172.21.15.66:49550) 2024-03-30T00:49:49.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 -- 172.21.15.66:0/1763102854 learned_addr learned my addr 172.21.15.66:0/1763102854 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae06a00640 1 --2- 172.21.15.66:0/1763102854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f18061b160 0x55f1808219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fadfe000640 1 --2- 172.21.15.66:0/1763102854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 0x55f180821f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 -- 172.21.15.66:0/1763102854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f18061b160 msgr2=0x55f1808219e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 --2- 172.21.15.66:0/1763102854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f18061b160 0x55f1808219e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 -- 172.21.15.66:0/1763102854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 msgr2=0x55f180821f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 --2- 172.21.15.66:0/1763102854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 0x55f180821f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 -- 172.21.15.66:0/1763102854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae081f7040 con 0x55f180866a70 2024-03-30T00:49:49.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fadfe000640 1 --2- 172.21.15.66:0/1763102854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 0x55f180821f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:49:49.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 --2- 172.21.15.66:0/1763102854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180881990 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fae00051d10 tx=0x7fae000558d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fa557b4c1c42e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fadf7e00640 1 -- 172.21.15.66:0/1763102854 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0006d070 con 0x55f180866a70 2024-03-30T00:49:49.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fadf7e00640 1 -- 172.21.15.66:0/1763102854 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0006d1d0 con 0x55f180866a70 2024-03-30T00:49:49.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fadf7e00640 1 -- 172.21.15.66:0/1763102854 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00053bb0 con 0x55f180866a70 2024-03-30T00:49:49.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- 172.21.15.66:0/1763102854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 msgr2=0x55f180881990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 --2- 172.21.15.66:0/1763102854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180881990 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fae00051d10 tx=0x7fae000558d0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- 172.21.15.66:0/1763102854 shutdown_connections 2024-03-30T00:49:49.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 --2- 172.21.15.66:0/1763102854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f180881990 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 --2- 172.21.15.66:0/1763102854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18081aa00 0x55f180821f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 --2- 172.21.15.66:0/1763102854 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f18061b160 0x55f1808219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- 172.21.15.66:0/1763102854 >> 172.21.15.66:0/1763102854 conn(0x55f180823540 msgr2=0x55f180819b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- 172.21.15.66:0/1763102854 shutdown_connections 2024-03-30T00:49:49.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- 172.21.15.66:0/1763102854 wait complete. 2024-03-30T00:49:49.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- start start 2024-03-30T00:49:49.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 Processor -- start 2024-03-30T00:49:49.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18061b160 0x55f1808aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f18081aa00 0x55f1808af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f1808afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f180724ec0 con 0x55f180866a70 2024-03-30T00:49:49.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f180601a20 con 0x55f18061b160 2024-03-30T00:49:49.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1806af170 con 0x55f18081aa00 2024-03-30T00:49:49.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae10dad000 10 client.0 _ll_register_callbacks cb 0x7fadf8009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:49:49.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f1808afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f1808afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49554/0 (socket says 172.21.15.66:49554) 2024-03-30T00:49:49.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae06a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18061b160 0x55f1808aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fae07400640 1 -- 172.21.15.66:0/3325253840 learned_addr learned my addr 172.21.15.66:0/3325253840 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.616+0000 7fadfe000640 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f18081aa00 0x55f1808af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae07400640 1 -- 172.21.15.66:0/3325253840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f18081aa00 msgr2=0x55f1808af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae07400640 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f18081aa00 0x55f1808af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae07400640 1 -- 172.21.15.66:0/3325253840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18061b160 msgr2=0x55f1808aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae07400640 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18061b160 0x55f1808aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae07400640 1 -- 172.21.15.66:0/3325253840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae080c6660 con 0x55f180866a70 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae07400640 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f1808afa10 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fae0000d7d0 tx=0x7fae0000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92773b1b59f5929 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fadebe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fadebe00640 1 -- 172.21.15.66:0/3325253840 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0006d040 con 0x55f180866a70 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fadebe00640 1 -- 172.21.15.66:0/3325253840 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fae00051f20 con 0x55f180866a70 2024-03-30T00:49:49.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 -- 172.21.15.66:0/3325253840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1808662c0 con 0x55f180866a70 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fadebe00640 1 -- 172.21.15.66:0/3325253840 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae000520e0 con 0x55f180866a70 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 10 client.9171 fetch_fsmap learned FSMap version 85 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 -- 172.21.15.66:0/3325253840 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55f180616df0 con 0x55f180866a70 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fadebe00640 1 -- 172.21.15.66:0/3325253840 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae00077020 con 0x55f180866a70 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fadebe00640 1 -- 172.21.15.66:0/3325253840 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fae0005d770 con 0x55f180866a70 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 10 client.9171 fetch_fsmap finished waiting for FSMap version 85 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 10 client.9171 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 20 client.9171 populate_metadata read hostname 'smithi066' 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55f180867390 0x55f1808aff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 4 client.9171 mds_command: new command op to 9057 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 -- 172.21.15.66:0/3325253840 --> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f180628970 con 0x55f180867390 2024-03-30T00:49:49.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae06a00640 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55f180867390 0x55f1808aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fadebe00640 10 client.9171 ms_handle_connect on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae06a00640 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55f180867390 0x55f1808aff50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ad041ae31f12cb server_cookie=9ec29e35fb39cc00 in_seq=0 out_seq=0 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fadebe00640 1 -- 172.21.15.66:0/3325253840 <== mds.0 v2:172.21.15.66:6835/282507324 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55f180628970 con 0x55f180867390 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fadebe00640 10 client.9171 handle_command_reply: tid=0 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 client.9171 shutdown 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 -- 172.21.15.66:0/3325253840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 msgr2=0x55f1808afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f1808afa10 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fae0000d7d0 tx=0x7fae0000bf50 comp rx=0 tx=0).stop 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 -- 172.21.15.66:0/3325253840 shutdown_connections 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55f180867390 0x55f1808aff50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f180866a70 0x55f1808afa10 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55f18081aa00 0x55f1808af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 --2- 172.21.15.66:0/3325253840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18061b160 0x55f1808aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 -- 172.21.15.66:0/3325253840 >> 172.21.15.66:0/3325253840 conn(0x55f180823540 msgr2=0x55f1808209d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fadebe00640 0 client.9171 ms_handle_reset on v2:172.21.15.66:6835/282507324 2024-03-30T00:49:49.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 -- 172.21.15.66:0/3325253840 shutdown_connections 2024-03-30T00:49:49.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 1 -- 172.21.15.66:0/3325253840 wait complete. 2024-03-30T00:49:49.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae10dad000 20 client.9171 trim_cache size 0 max 16384 2024-03-30T00:49:49.633 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:49:49.633 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:49:49.633 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-03-30T00:49:49.634 INFO:teuthology.orchestra.run.smithi066.stdout: "aa831b09-e5ae-406e-9cc0-0742aaad7496": { 2024-03-30T00:49:49.634 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T00:49:49.634 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "aa831b09-e5ae-406e-9cc0-0742aaad7496", 2024-03-30T00:49:49.634 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,repair" 2024-03-30T00:49:49.634 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:49:49.634 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:49:49.634 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:49:49.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 -- 172.21.15.66:0/2353534473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fade4066690 msgr2=0x7fade4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fade4066690 0x7fade4068b50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fadf8003ab0 tx=0x7fadf8009830 comp rx=0 tx=0).stop 2024-03-30T00:49:49.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 -- 172.21.15.66:0/2353534473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae0815a640 msgr2=0x7fae080c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:49:49.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae0815a640 0x7fae080c74a0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fae0000d800 tx=0x7fae0000dcc0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 -- 172.21.15.66:0/2353534473 shutdown_connections 2024-03-30T00:49:49.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fade4066690 0x7fade4068b50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae0815a640 0x7fae080c74a0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae080ce520 0x7fae080c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 --2- 172.21.15.66:0/2353534473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae080cabb0 0x7fae080ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:49:49.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 -- 172.21.15.66:0/2353534473 >> 172.21.15.66:0/2353534473 conn(0x7fae08069f60 msgr2=0x7fae08159740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:49:49.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 -- 172.21.15.66:0/2353534473 shutdown_connections 2024-03-30T00:49:49.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.620+0000 7fae0da00640 1 -- 172.21.15.66:0/2353534473 wait complete. 2024-03-30T00:49:49.637 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-03-30T00:49:49.628+0000 7f76aec00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:49:49.647 INFO:tasks.cephfs.filesystem:scrub status for tag:aa831b09-e5ae-406e-9cc0-0742aaad7496 - {'path': '/', 'tag': 'aa831b09-e5ae-406e-9cc0-0742aaad7496', 'options': 'recursive,repair'} 2024-03-30T00:49:49.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:49.768+0000 7f42af400640 1 -- 172.21.15.66:0/263904469 <== mon.0 v2:172.21.15.66:3300/0 7 ==== log(8 entries from seq 1 at 2024-03-30T00:49:49.414626+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7f42b0055d00 con 0x7f42b815aa50 2024-03-30T00:49:49.775 INFO:teuthology.orchestra.run.smithi066.stdout: cluster: 2024-03-30T00:49:49.775 INFO:teuthology.orchestra.run.smithi066.stdout: id: 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:49:49.775 INFO:teuthology.orchestra.run.smithi066.stdout: health: HEALTH_OK 2024-03-30T00:49:49.775 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:49.775 INFO:teuthology.orchestra.run.smithi066.stdout: services: 2024-03-30T00:49:49.775 INFO:teuthology.orchestra.run.smithi066.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-03-30T00:49:49.775 INFO:teuthology.orchestra.run.smithi066.stdout: mgr: y(active, since 10m), standbys: x 2024-03-30T00:49:49.775 INFO:teuthology.orchestra.run.smithi066.stdout: mds: 1/1 daemons up, 3 standby 2024-03-30T00:49:49.775 INFO:teuthology.orchestra.run.smithi066.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout: data: 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout: volumes: 1/1 healthy 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout: pools: 3 pools, 97 pgs 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout: objects: 30 objects, 19 MiB 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout: pgs: 97 active+clean 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:49.414626+0000 mds.c [INF] scrub queued for path: / 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:49.414629+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:49.414633+0000 mds.c [INF] scrub summary: active paths [/] 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:49.415247+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:49.415252+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-03-30T00:49:49.776 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:49.415421+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-03-30T00:49:49.777 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:49.415428+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-03-30T00:49:49.777 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:49.415954+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-30T00:49:54.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:49:54.776+0000 7f42af400640 1 -- 172.21.15.66:0/263904469 <== mon.0 v2:172.21.15.66:3300/0 8 ==== log(1 entries from seq 9 at 2024-03-30T00:49:53.700217+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f42b0059a80 con 0x7f42b815aa50 2024-03-30T00:49:54.784 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:49:53.700217+0000 mds.c [INF] scrub summary: idle 2024-03-30T00:50:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:00.788+0000 7f42af400640 1 -- 172.21.15.66:0/263904469 <== mon.0 v2:172.21.15.66:3300/0 9 ==== log(1 entries from seq 1259 at 2024-03-30T00:50:00.000120+0000) v1 ==== 187+0+0 (secure 0 0 0) 0x7f42b005e040 con 0x7f42b815aa50 2024-03-30T00:50:00.796 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:50:00.000120+0000 mon.a [INF] overall HEALTH_OK 2024-03-30T00:50:17.634 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:50:17.638 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:50:19.649 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-30T00:50:19.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.796+0000 7f1e73200640 1 Processor -- start 2024-03-30T00:50:19.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.796+0000 7f1e73200640 1 -- start start 2024-03-30T00:50:19.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.796+0000 7f1e73200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e6c15a720 0x7f1e6c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.796+0000 7f1e73200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1e6c0d0ed0 0x7f1e6c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.796+0000 7f1e73200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6c0cb5b0 con 0x7f1e6c0b3170 2024-03-30T00:50:19.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.796+0000 7f1e73200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6c0cfb00 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.796+0000 7f1e73200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c15ab00 con 0x7f1e6c15a720 2024-03-30T00:50:19.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.796+0000 7f1e72800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1e6c0d0ed0 0x7f1e6c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.796+0000 7f1e71400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1e6c0b3170 0x7f1e6c0d07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55262/0 (socket says 172.21.15.66:55262) 2024-03-30T00:50:19.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.796+0000 7f1e71400640 1 -- 172.21.15.66:0/721910291 learned_addr learned my addr 172.21.15.66:0/721910291 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:19.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.796+0000 7f1e71e00640 1 --2- 172.21.15.66:0/721910291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e6c15a720 0x7f1e6c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.800+0000 7f1e72800640 1 -- 172.21.15.66:0/721910291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e6c15a720 msgr2=0x7f1e6c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:19.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.800+0000 7f1e72800640 1 --2- 172.21.15.66:0/721910291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e6c15a720 0x7f1e6c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.800+0000 7f1e72800640 1 -- 172.21.15.66:0/721910291 >> v1:172.21.15.66:6789/0 conn(0x7f1e6c0b3170 legacy=0x7f1e6c0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:19.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.800+0000 7f1e70a00640 1 -- 172.21.15.66:0/721910291 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719813825 0 0) 0x7f1e5c001120 con 0x7f1e6c0b3170 2024-03-30T00:50:19.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.800+0000 7f1e72800640 1 -- 172.21.15.66:0/721910291 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c0cb2f0 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.800+0000 7f1e72800640 1 --2- 172.21.15.66:0/721910291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e6c0d0ed0 0x7f1e6c0b3570 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009dc0 tx=0x7f1e6801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2615fb56c1f894de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:19.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.800+0000 7f1e70a00640 1 -- 172.21.15.66:0/721910291 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680a59e0 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.800+0000 7f1e70a00640 1 -- 172.21.15.66:0/721910291 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e680a5b40 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.800+0000 7f1e70a00640 1 -- 172.21.15.66:0/721910291 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680ad440 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.800+0000 7f1e73200640 1 -- 172.21.15.66:0/721910291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e6c0d0ed0 msgr2=0x7f1e6c0b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.800+0000 7f1e73200640 1 --2- 172.21.15.66:0/721910291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e6c0d0ed0 0x7f1e6c0b3570 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009dc0 tx=0x7f1e6801c920 comp rx=0 tx=0).stop 2024-03-30T00:50:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 -- 172.21.15.66:0/721910291 shutdown_connections 2024-03-30T00:50:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 --2- 172.21.15.66:0/721910291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e6c0d0ed0 0x7f1e6c0b3570 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 --2- 172.21.15.66:0/721910291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e6c15a720 0x7f1e6c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 -- 172.21.15.66:0/721910291 >> 172.21.15.66:0/721910291 conn(0x7f1e6c069d90 msgr2=0x7f1e6c06a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 -- 172.21.15.66:0/721910291 shutdown_connections 2024-03-30T00:50:19.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 -- 172.21.15.66:0/721910291 wait complete. 2024-03-30T00:50:19.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 Processor -- start 2024-03-30T00:50:19.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 -- start start 2024-03-30T00:50:19.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e6c0b3170 0x7f1e6c1526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e6c0d0ed0 0x7f1e6c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e6c15a720 0x7f1e6c1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c0c55b0 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c0c5890 con 0x7f1e6c0b3170 2024-03-30T00:50:19.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e73200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c0c5b70 con 0x7f1e6c15a720 2024-03-30T00:50:19.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e71400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e6c0d0ed0 0x7f1e6c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e71400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e6c0d0ed0 0x7f1e6c153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35312/0 (socket says 172.21.15.66:35312) 2024-03-30T00:50:19.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e71400640 1 -- 172.21.15.66:0/3255058128 learned_addr learned my addr 172.21.15.66:0/3255058128 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:19.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e72800640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e6c15a720 0x7f1e6c1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e71e00640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e6c0b3170 0x7f1e6c1526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e71400640 1 -- 172.21.15.66:0/3255058128 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e6c15a720 msgr2=0x7f1e6c1589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:19.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e71400640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e6c15a720 0x7f1e6c1589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e71400640 1 -- 172.21.15.66:0/3255058128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e6c0b3170 msgr2=0x7f1e6c1526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:19.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e71400640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e6c0b3170 0x7f1e6c1526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e71400640 1 -- 172.21.15.66:0/3255058128 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68009a70 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.804+0000 7f1e71400640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e6c0d0ed0 0x7f1e6c153840 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00c780 tx=0x7f1e5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.808+0000 7f1e66000640 1 -- 172.21.15.66:0/3255058128 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0072d0 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.808+0000 7f1e66000640 1 -- 172.21.15.66:0/3255058128 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5c007450 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.808+0000 7f1e66000640 1 -- 172.21.15.66:0/3255058128 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0047f0 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.808+0000 7f1e73200640 1 -- 172.21.15.66:0/3255058128 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6c0c5e50 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.808+0000 7f1e73200640 1 -- 172.21.15.66:0/3255058128 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6c152ee0 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.808+0000 7f1e66000640 1 -- 172.21.15.66:0/3255058128 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1e5c004950 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.808+0000 7f1e66000640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e50066690 0x7f1e50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.808+0000 7f1e66000640 1 -- 172.21.15.66:0/3255058128 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1e5c0833b0 con 0x7f1e6c0d0ed0 2024-03-30T00:50:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e71e00640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e50066690 0x7f1e50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 Processor -- start 2024-03-30T00:50:19.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 -- start start 2024-03-30T00:50:19.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e71e00640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e50066690 0x7f1e50068b50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1e600079c0 tx=0x7f1e60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:19.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673252a70 0x55c673252e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c6732064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673206a00 0x55c673206de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6730d1630 con 0x55c673206a00 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67305ebd0 con 0x55c673253390 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c67305c1a0 con 0x55c673252a70 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e72800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673206a00 0x55c673206de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e72800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673206a00 0x55c673206de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35320/0 (socket says 172.21.15.66:35320) 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e72800640 1 -- 172.21.15.66:0/1603055821 learned_addr learned my addr 172.21.15.66:0/1603055821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e72800640 1 -- 172.21.15.66:0/1603055821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673252a70 msgr2=0x55c673252e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e72800640 1 --2- 172.21.15.66:0/1603055821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673252a70 0x55c673252e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e72800640 1 -- 172.21.15.66:0/1603055821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 msgr2=0x55c6732064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e72800640 1 --2- 172.21.15.66:0/1603055821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c6732064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e72800640 1 -- 172.21.15.66:0/1603055821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c672d53e20 con 0x55c673206a00 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e72800640 1 --2- 172.21.15.66:0/1603055821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673206a00 0x55c673206de0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1e6801afd0 tx=0x7f1e68004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e953559319386d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e5aa00640 1 -- 172.21.15.66:0/1603055821 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68003ba0 con 0x55c673206a00 2024-03-30T00:50:19.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e5aa00640 1 -- 172.21.15.66:0/1603055821 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68003d00 con 0x55c673206a00 2024-03-30T00:50:19.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e5aa00640 1 -- 172.21.15.66:0/1603055821 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68004240 con 0x55c673206a00 2024-03-30T00:50:19.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1603055821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673206a00 msgr2=0x55c673206de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:19.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/1603055821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673206a00 0x55c673206de0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1e6801afd0 tx=0x7f1e68004ac0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1603055821 shutdown_connections 2024-03-30T00:50:19.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/1603055821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673206a00 0x55c673206de0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/1603055821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c6732064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/1603055821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673252a70 0x55c673252e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1603055821 >> 172.21.15.66:0/1603055821 conn(0x55c673007160 msgr2=0x55c673235820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:19.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1603055821 shutdown_connections 2024-03-30T00:50:19.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1603055821 wait complete. 2024-03-30T00:50:19.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.812+0000 7f1e77b6c000 1 -- start start 2024-03-30T00:50:19.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 Processor -- start 2024-03-30T00:50:19.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673206a00 0x55c67320f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673236a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c673236f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c673057a90 con 0x55c673252a70 2024-03-30T00:50:19.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c672ff6d40 con 0x55c673253390 2024-03-30T00:50:19.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c673014f30 con 0x55c673206a00 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 10 client.0 _ll_register_callbacks cb 0x55c6730b9810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673236a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673236a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35324/0 (socket says 172.21.15.66:35324) 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71400640 1 -- 172.21.15.66:0/2937163188 learned_addr learned my addr 172.21.15.66:0/2937163188 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e72800640 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c673236f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71400640 1 -- 172.21.15.66:0/2937163188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673206a00 msgr2=0x55c67320f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71e00640 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673206a00 0x55c67320f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71400640 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673206a00 0x55c67320f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71400640 1 -- 172.21.15.66:0/2937163188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 msgr2=0x55c673236f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71400640 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c673236f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71400640 1 -- 172.21.15.66:0/2937163188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6c0cb2f0 con 0x55c673252a70 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e72800640 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c673236f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71e00640 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673206a00 0x55c67320f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:19.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71400640 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673236a50 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0507e0 tx=0x7f1e5c0542e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912b5ea4a7f9e980 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e4f400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e4f400640 1 -- 172.21.15.66:0/2937163188 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c05c620 con 0x55c673252a70 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e4f400640 1 -- 172.21.15.66:0/2937163188 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1e5c05a0d0 con 0x55c673252a70 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2937163188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6730563e0 con 0x55c673252a70 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e4f400640 1 -- 172.21.15.66:0/2937163188 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e5c05a290 con 0x55c673252a70 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 10 client.9186 fetch_fsmap learned FSMap version 85 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2937163188 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55c67301caa0 con 0x55c673252a70 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e4f400640 1 -- 172.21.15.66:0/2937163188 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c6730563e0 con 0x55c673252a70 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e4f400640 1 -- 172.21.15.66:0/2937163188 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1e5c0812f0 con 0x55c673252a70 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 10 client.9186 fetch_fsmap finished waiting for FSMap version 85 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 10 client.9186 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 20 client.9186 populate_metadata read hostname 'smithi066' 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55c67325a2d0 0x55c67325a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 4 client.9186 mds_command: new command op to 9057 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:50:19.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2937163188 --> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c673027d00 con 0x55c67325a2d0 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71e00640 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55c67325a2d0 0x55c67325a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e71e00640 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55c67325a2d0 0x55c67325a6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2354065b358b0ed7 server_cookie=a18601d943b14699 in_seq=0 out_seq=0 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e4f400640 10 client.9186 ms_handle_connect on v2:172.21.15.66:6835/282507324 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e4f400640 1 -- 172.21.15.66:0/2937163188 <== mds.0 v2:172.21.15.66:6835/282507324 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c673027d00 con 0x55c67325a2d0 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e4f400640 10 client.9186 handle_command_reply: tid=0 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.816+0000 7f1e77b6c000 1 client.9186 shutdown 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2937163188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 msgr2=0x55c673236a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673236a50 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0507e0 tx=0x7f1e5c0542e0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2937163188 shutdown_connections 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55c67325a2d0 0x55c67325a6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c673236f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673236a50 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/2937163188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673206a00 0x55c67320f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e4f400640 0 client.9186 ms_handle_reset on v2:172.21.15.66:6835/282507324 2024-03-30T00:50:19.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2937163188 >> 172.21.15.66:0/2937163188 conn(0x55c673007160 msgr2=0x55c673205bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:19.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2937163188 shutdown_connections 2024-03-30T00:50:19.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2937163188 wait complete. 2024-03-30T00:50:19.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.820+0000 7f1e77b6c000 20 client.9186 trim_cache size 0 max 16384 2024-03-30T00:50:19.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.832+0000 7f1e77b6c000 1 Processor -- start 2024-03-30T00:50:19.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.832+0000 7f1e77b6c000 1 -- start start 2024-03-30T00:50:19.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673206a00 0x55c673205c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673206170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673253390 0x55c673206de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c672d14aa0 con 0x55c673252a70 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c673023390 con 0x55c673206a00 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c673237220 con 0x55c673253390 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e71400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673206170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e71400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673206170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35334/0 (socket says 172.21.15.66:35334) 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e71400640 1 -- 172.21.15.66:0/2116795392 learned_addr learned my addr 172.21.15.66:0/2116795392 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e71e00640 1 --2- 172.21.15.66:0/2116795392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673206a00 0x55c673205c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e72800640 1 --2- 172.21.15.66:0/2116795392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673253390 0x55c673206de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e71400640 1 -- 172.21.15.66:0/2116795392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673253390 msgr2=0x55c673206de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e71400640 1 --2- 172.21.15.66:0/2116795392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673253390 0x55c673206de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e71400640 1 -- 172.21.15.66:0/2116795392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673206a00 msgr2=0x55c673205c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:19.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e71400640 1 --2- 172.21.15.66:0/2116795392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673206a00 0x55c673205c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e71400640 1 -- 172.21.15.66:0/2116795392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c673002df0 con 0x55c673252a70 2024-03-30T00:50:19.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e72800640 1 --2- 172.21.15.66:0/2116795392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673253390 0x55c673206de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:19.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e71400640 1 --2- 172.21.15.66:0/2116795392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673206170 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c000f70 tx=0x7f1e5c059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1036461d7591af4f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:19.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e5b400640 1 -- 172.21.15.66:0/2116795392 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c053e20 con 0x55c673252a70 2024-03-30T00:50:19.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e5b400640 1 -- 172.21.15.66:0/2116795392 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5c05c620 con 0x55c673252a70 2024-03-30T00:50:19.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e5b400640 1 -- 172.21.15.66:0/2116795392 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c05c280 con 0x55c673252a70 2024-03-30T00:50:19.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2116795392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 msgr2=0x55c673206170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:19.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/2116795392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673206170 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c000f70 tx=0x7f1e5c059000 comp rx=0 tx=0).stop 2024-03-30T00:50:19.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2116795392 shutdown_connections 2024-03-30T00:50:19.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/2116795392 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673253390 0x55c673206de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/2116795392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673206170 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/2116795392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673206a00 0x55c673205c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2116795392 >> 172.21.15.66:0/2116795392 conn(0x55c673007160 msgr2=0x55c67320dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:19.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2116795392 shutdown_connections 2024-03-30T00:50:19.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- 172.21.15.66:0/2116795392 wait complete. 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- start start 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 Processor -- start 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673206a00 0x55c673209a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673209fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c6732aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c673110ec0 con 0x55c673252a70 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6730270b0 con 0x55c673253390 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c673128470 con 0x55c673206a00 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e77b6c000 10 client.0 _ll_register_callbacks cb 0x7f1e68009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.836+0000 7f1e71400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673209fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e71400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673209fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35346/0 (socket says 172.21.15.66:35346) 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e71400640 1 -- 172.21.15.66:0/1892927631 learned_addr learned my addr 172.21.15.66:0/1892927631 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e71e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673206a00 0x55c673209a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e72800640 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c6732aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e71400640 1 -- 172.21.15.66:0/1892927631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673206a00 msgr2=0x55c673209a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e71400640 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673206a00 0x55c673209a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e71400640 1 -- 172.21.15.66:0/1892927631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 msgr2=0x55c6732aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e71400640 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c6732aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e71400640 1 -- 172.21.15.66:0/1892927631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c67301caa0 con 0x55c673252a70 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e71400640 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673209fa0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0507e0 tx=0x7f1e5c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96281938ec81db0b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e4f400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e4f400640 1 -- 172.21.15.66:0/1892927631 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c05c230 con 0x55c673252a70 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e4f400640 1 -- 172.21.15.66:0/1892927631 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1e5c06be40 con 0x55c673252a70 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1892927631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c673237360 con 0x55c673252a70 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e4f400640 1 -- 172.21.15.66:0/1892927631 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e5c05c390 con 0x55c673252a70 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 10 client.9192 fetch_fsmap learned FSMap version 85 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1892927631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55c672ff6960 con 0x55c673252a70 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e4f400640 1 -- 172.21.15.66:0/1892927631 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c673237360 con 0x55c673252a70 2024-03-30T00:50:19.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e4f400640 1 -- 172.21.15.66:0/1892927631 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1e5c0537d0 con 0x55c673252a70 2024-03-30T00:50:19.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 10 client.9192 fetch_fsmap finished waiting for FSMap version 85 2024-03-30T00:50:19.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 10 client.9192 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-30T00:50:19.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 20 client.9192 populate_metadata read hostname 'smithi066' 2024-03-30T00:50:19.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55c6732af060 0x55c6732af470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:19.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 4 client.9192 mds_command: new command op to 9057 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:50:19.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1892927631 --> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c673014970 con 0x55c6732af060 2024-03-30T00:50:19.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e71e00640 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55c6732af060 0x55c6732af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:19.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e71e00640 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55c6732af060 0x55c6732af470 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b4d49de51254802 server_cookie=a2df596b5f544e2f in_seq=0 out_seq=0 2024-03-30T00:50:19.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e4f400640 10 client.9192 ms_handle_connect on v2:172.21.15.66:6835/282507324 2024-03-30T00:50:19.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e4f400640 1 -- 172.21.15.66:0/1892927631 <== mds.0 v2:172.21.15.66:6835/282507324 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c673014970 con 0x55c6732af060 2024-03-30T00:50:19.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e4f400640 10 client.9192 handle_command_reply: tid=0 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 client.9192 shutdown 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1892927631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 msgr2=0x55c673209fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673209fa0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0507e0 tx=0x7f1e5c01e040 comp rx=0 tx=0).stop 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1892927631 shutdown_connections 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.66:6835/282507324,v1:172.21.15.66:6837/282507324] conn(0x55c6732af060 0x55c6732af470 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c673253390 0x55c6732aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c673252a70 0x55c673209fa0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e4f400640 0 client.9192 ms_handle_reset on v2:172.21.15.66:6835/282507324 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 --2- 172.21.15.66:0/1892927631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c673206a00 0x55c673209a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1892927631 >> 172.21.15.66:0/1892927631 conn(0x55c673007160 msgr2=0x55c67320e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1892927631 shutdown_connections 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 1 -- 172.21.15.66:0/1892927631 wait complete. 2024-03-30T00:50:19.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e77b6c000 20 client.9192 trim_cache size 0 max 16384 2024-03-30T00:50:19.853 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:50:19.853 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T00:50:19.853 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2024-03-30T00:50:19.853 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:50:19.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 -- 172.21.15.66:0/3255058128 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e50066690 msgr2=0x7f1e50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:19.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e50066690 0x7f1e50068b50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1e600079c0 tx=0x7f1e60008040 comp rx=0 tx=0).stop 2024-03-30T00:50:19.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 -- 172.21.15.66:0/3255058128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e6c0d0ed0 msgr2=0x7f1e6c153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:19.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e6c0d0ed0 0x7f1e6c153840 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00c780 tx=0x7f1e5c00cc40 comp rx=0 tx=0).stop 2024-03-30T00:50:19.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 -- 172.21.15.66:0/3255058128 shutdown_connections 2024-03-30T00:50:19.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e6c15a720 0x7f1e6c1589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1e50066690 0x7f1e50068b50 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e6c0d0ed0 0x7f1e6c153840 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 --2- 172.21.15.66:0/3255058128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e6c0b3170 0x7f1e6c1526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:19.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 -- 172.21.15.66:0/3255058128 >> 172.21.15.66:0/3255058128 conn(0x7f1e6c069d90 msgr2=0x7f1e6c1570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:19.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 -- 172.21.15.66:0/3255058128 shutdown_connections 2024-03-30T00:50:19.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:19.840+0000 7f1e73200640 1 -- 172.21.15.66:0/3255058128 wait complete. 2024-03-30T00:50:19.868 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:50:19.969 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:50:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb9da00640 1 Processor -- start 2024-03-30T00:50:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb9da00640 1 -- start start 2024-03-30T00:50:20.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb9da00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbb980ce520 0x7fbb980ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb9da00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 0x7fbb980cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb9da00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9806d070 con 0x7fbb9815a640 2024-03-30T00:50:20.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb9da00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb980cbba0 con 0x7fbb980ce520 2024-03-30T00:50:20.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb9da00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980cee70 con 0x7fbb980cabb0 2024-03-30T00:50:20.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb8e000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbb980ce520 0x7fbb980ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb97400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 0x7fbb980cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb97400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 0x7fbb980cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:48180/0 (socket says 172.21.15.66:48180) 2024-03-30T00:50:20.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb8e000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbb980ce520 0x7fbb980ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:52490/0 (socket says 172.21.15.66:52490) 2024-03-30T00:50:20.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb97400640 1 -- 172.21.15.66:0/4289724101 learned_addr learned my addr 172.21.15.66:0/4289724101 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:20.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb96a00640 1 --1- 172.21.15.66:0/4289724101 >> v1:172.21.15.156:6789/0 conn(0x7fbb9815a640 0x7fbb9806dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:60116/0 (socket says 172.21.15.66:60116) 2024-03-30T00:50:20.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb8e000640 1 -- 172.21.15.66:0/4289724101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 msgr2=0x7fbb980cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:20.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb8e000640 1 --2- 172.21.15.66:0/4289724101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 0x7fbb980cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb8e000640 1 -- 172.21.15.66:0/4289724101 >> v1:172.21.15.156:6789/0 conn(0x7fbb9815a640 legacy=0x7fbb9806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb96000640 1 -- 172.21.15.66:0/4289724101 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291403646 0 0) 0x7fbb9806d070 con 0x7fbb9815a640 2024-03-30T00:50:20.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb8e000640 1 -- 172.21.15.66:0/4289724101 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb980cb8c0 con 0x7fbb980ce520 2024-03-30T00:50:20.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.104+0000 7fbb97400640 1 --2- 172.21.15.66:0/4289724101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 0x7fbb980cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:20.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb8e000640 1 --2- 172.21.15.66:0/4289724101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb980ce520 0x7fbb980ce930 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009d90 tx=0x7fbb8001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cd45c7a80b7d752 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:20.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb96000640 1 -- 172.21.15.66:0/4289724101 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb800a5a00 con 0x7fbb980ce520 2024-03-30T00:50:20.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb96000640 1 -- 172.21.15.66:0/4289724101 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb800a5b60 con 0x7fbb980ce520 2024-03-30T00:50:20.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb96000640 1 -- 172.21.15.66:0/4289724101 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb800ad440 con 0x7fbb980ce520 2024-03-30T00:50:20.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb9da00640 1 -- 172.21.15.66:0/4289724101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb980ce520 msgr2=0x7fbb980ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb9da00640 1 --2- 172.21.15.66:0/4289724101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb980ce520 0x7fbb980ce930 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009d90 tx=0x7fbb8001c940 comp rx=0 tx=0).stop 2024-03-30T00:50:20.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb9da00640 1 -- 172.21.15.66:0/4289724101 shutdown_connections 2024-03-30T00:50:20.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb9da00640 1 --2- 172.21.15.66:0/4289724101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 0x7fbb980cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb9da00640 1 --2- 172.21.15.66:0/4289724101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb980ce520 0x7fbb980ce930 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb9da00640 1 -- 172.21.15.66:0/4289724101 >> 172.21.15.66:0/4289724101 conn(0x7fbb98069f60 msgr2=0x7fbb9806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:20.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb9da00640 1 -- 172.21.15.66:0/4289724101 shutdown_connections 2024-03-30T00:50:20.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.108+0000 7fbb9da00640 1 -- 172.21.15.66:0/4289724101 wait complete. 2024-03-30T00:50:20.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb9da00640 1 Processor -- start 2024-03-30T00:50:20.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb9da00640 1 -- start start 2024-03-30T00:50:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb9da00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 0x7fbb980c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb9da00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb980ce520 0x7fbb98153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb9da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb9815a640 0x7fbb980c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb9da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98071ce0 con 0x7fbb9815a640 2024-03-30T00:50:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb9da00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98158fd0 con 0x7fbb980ce520 2024-03-30T00:50:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb9da00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980c81b0 con 0x7fbb980cabb0 2024-03-30T00:50:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb97400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb9815a640 0x7fbb980c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb97400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb9815a640 0x7fbb980c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35350/0 (socket says 172.21.15.66:35350) 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb97400640 1 -- 172.21.15.66:0/278795834 learned_addr learned my addr 172.21.15.66:0/278795834 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb96a00640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 0x7fbb980c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb8e000640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb980ce520 0x7fbb98153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb97400640 1 -- 172.21.15.66:0/278795834 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 msgr2=0x7fbb980c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb97400640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 0x7fbb980c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb97400640 1 -- 172.21.15.66:0/278795834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb980ce520 msgr2=0x7fbb98153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb97400640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb980ce520 0x7fbb98153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb97400640 1 -- 172.21.15.66:0/278795834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb900094d0 con 0x7fbb9815a640 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb97400640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb9815a640 0x7fbb980c74d0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000e950 tx=0x7fbb9000ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb8f400640 1 -- 172.21.15.66:0/278795834 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000ca00 con 0x7fbb9815a640 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb8f400640 1 -- 172.21.15.66:0/278795834 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9000cb60 con 0x7fbb9815a640 2024-03-30T00:50:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb8f400640 1 -- 172.21.15.66:0/278795834 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90004020 con 0x7fbb9815a640 2024-03-30T00:50:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb9da00640 1 -- 172.21.15.66:0/278795834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb80009a70 con 0x7fbb9815a640 2024-03-30T00:50:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.112+0000 7fbb9da00640 1 -- 172.21.15.66:0/278795834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb98071200 con 0x7fbb9815a640 2024-03-30T00:50:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.116+0000 7fbb9da00640 1 -- 172.21.15.66:0/278795834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb98066c30 con 0x7fbb9815a640 2024-03-30T00:50:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.116+0000 7fbb8f400640 1 -- 172.21.15.66:0/278795834 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbb90004180 con 0x7fbb9815a640 2024-03-30T00:50:20.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.116+0000 7fbb8f400640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb74066690 0x7fbb74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.116+0000 7fbb8f400640 1 -- 172.21.15.66:0/278795834 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fbb90084280 con 0x7fbb9815a640 2024-03-30T00:50:20.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.116+0000 7fbb96a00640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb74066690 0x7fbb74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.116+0000 7fbb96a00640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb74066690 0x7fbb74068b50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbb88000f70 tx=0x7fbb88007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:20.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.120+0000 7fbb8f400640 1 -- 172.21.15.66:0/278795834 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbb90051470 con 0x7fbb9815a640 2024-03-30T00:50:20.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.316+0000 7fbb9da00640 1 -- 172.21.15.66:0/278795834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbb98073460 con 0x7fbb9815a640 2024-03-30T00:50:20.323 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:50:20.316+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:50:20.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.328+0000 7fbb8f400640 1 -- 172.21.15.66:0/278795834 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbb90054a90 con 0x7fbb9815a640 2024-03-30T00:50:20.335 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:50:20.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 -- 172.21.15.66:0/278795834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb74066690 msgr2=0x7fbb74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb74066690 0x7fbb74068b50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbb88000f70 tx=0x7fbb88007df0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 -- 172.21.15.66:0/278795834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb9815a640 msgr2=0x7fbb980c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb9815a640 0x7fbb980c74d0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000e950 tx=0x7fbb9000ee10 comp rx=0 tx=0).stop 2024-03-30T00:50:20.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 -- 172.21.15.66:0/278795834 shutdown_connections 2024-03-30T00:50:20.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fbb74066690 0x7fbb74068b50 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb9815a640 0x7fbb980c74d0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb980ce520 0x7fbb98153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 --2- 172.21.15.66:0/278795834 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbb980cabb0 0x7fbb980c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 -- 172.21.15.66:0/278795834 >> 172.21.15.66:0/278795834 conn(0x7fbb98069f60 msgr2=0x7fbb980ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:20.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 -- 172.21.15.66:0/278795834 shutdown_connections 2024-03-30T00:50:20.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.332+0000 7fbb9da00640 1 -- 172.21.15.66:0/278795834 wait complete. 2024-03-30T00:50:20.342 INFO:tasks.ceph.mds.c.smithi066.stderr: -1014> 2024-03-30T00:49:49.412+0000 7f76aa600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-03-30T00:50:20.343 INFO:tasks.ceph.mds.c.smithi066.stderr: -1011> 2024-03-30T00:49:49.412+0000 7f76aa600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-03-30T00:50:20.343 INFO:tasks.ceph.mds.c.smithi066.stderr: -994> 2024-03-30T00:49:49.412+0000 7f76aa600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-03-30T00:50:20.343 INFO:tasks.ceph.mds.c.smithi066.stderr: -991> 2024-03-30T00:49:49.412+0000 7f76aa600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-03-30T00:50:20.343 INFO:tasks.ceph.mds.c.smithi066.stderr: -865> 2024-03-30T00:49:49.628+0000 7f76aec00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:50:20.358 DEBUG:teuthology.orchestra.run.smithi066:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-03-30T00:50:20.385 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:50:20.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.396+0000 7faa27692400 1 -- start start 2024-03-30T00:50:20.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.396+0000 7faa27692400 1 Processor -- start 2024-03-30T00:50:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea60aa7800 0x55ea60b32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea60aa7400 0x55ea60b32b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea60aa7000 0x55ea60b33080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea60b74a00 con 0x55ea60aa7400 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea60b74780 con 0x55ea60aa7800 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea60b74640 con 0x55ea60aa7000 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa24e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea60aa7400 0x55ea60b32b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa24e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea60aa7400 0x55ea60b32b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35386/0 (socket says 172.21.15.66:35386) 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa24e00640 1 -- 172.21.15.66:0/3082129468 learned_addr learned my addr 172.21.15.66:0/3082129468 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa26200640 1 --2- 172.21.15.66:0/3082129468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea60aa7000 0x55ea60b33080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa25800640 1 --2- 172.21.15.66:0/3082129468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea60aa7800 0x55ea60b32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa24e00640 1 -- 172.21.15.66:0/3082129468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea60aa7000 msgr2=0x55ea60b33080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa24e00640 1 --2- 172.21.15.66:0/3082129468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea60aa7000 0x55ea60b33080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa24e00640 1 -- 172.21.15.66:0/3082129468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea60aa7800 msgr2=0x55ea60b32580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:20.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa24e00640 1 --2- 172.21.15.66:0/3082129468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea60aa7800 0x55ea60b32580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa24e00640 1 -- 172.21.15.66:0/3082129468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea60adc9c0 con 0x55ea60aa7400 2024-03-30T00:50:20.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa24e00640 1 --2- 172.21.15.66:0/3082129468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea60aa7400 0x55ea60b32b00 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x55ea61878c90 tx=0x55ea619aed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182841761522a88a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa22600640 1 -- 172.21.15.66:0/3082129468 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ea60afb600 con 0x55ea60aa7400 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa22600640 1 -- 172.21.15.66:0/3082129468 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea60b04480 con 0x55ea60aa7400 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa22600640 1 -- 172.21.15.66:0/3082129468 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ea60afb1e0 con 0x55ea60aa7400 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 -- 172.21.15.66:0/3082129468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea60adcb60 con 0x55ea60aa7400 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 -- 172.21.15.66:0/3082129468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ea60adc340 con 0x55ea60aa7400 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 4 waiting for MDS map... 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa22600640 1 -- 172.21.15.66:0/3082129468 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x55ea60ae6c40 con 0x55ea60aa7400 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa22600640 1 -- 172.21.15.66:0/3082129468 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 86) v1 ==== 2573+0+0 (secure 0 0 0) 0x55ea60af4f00 con 0x55ea60aa7400 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 4 Got MDS map 86 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 10 main: main 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 4 main: connecting to RADOS... 2024-03-30T00:50:20.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 Processor -- start 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 -- start start 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b6c00 0x55ea60b33600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea619b7000 0x55ea60b33b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea619b7400 0x55ea60b34100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea60aede00 con 0x55ea619b6c00 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea60b74780 con 0x55ea619b7400 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.400+0000 7faa27692400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea60b74500 con 0x55ea619b7000 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b6c00 0x55ea60b33600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b6c00 0x55ea60b33600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35398/0 (socket says 172.21.15.66:35398) 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 -- 172.21.15.66:0/609086830 learned_addr learned my addr 172.21.15.66:0/609086830 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa25800640 1 --2- 172.21.15.66:0/609086830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea619b7000 0x55ea60b33b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa24e00640 1 --2- 172.21.15.66:0/609086830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea619b7400 0x55ea60b34100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 -- 172.21.15.66:0/609086830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea619b7000 msgr2=0x55ea60b33b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 --2- 172.21.15.66:0/609086830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea619b7000 0x55ea60b33b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 -- 172.21.15.66:0/609086830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea619b7400 msgr2=0x55ea60b34100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 --2- 172.21.15.66:0/609086830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea619b7400 0x55ea60b34100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 -- 172.21.15.66:0/609086830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea60adc000 con 0x55ea619b6c00 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa24e00640 1 --2- 172.21.15.66:0/609086830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea619b7400 0x55ea60b34100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 --2- 172.21.15.66:0/609086830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b6c00 0x55ea60b33600 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x55ea61879b00 tx=0x55ea619dcc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e55f30b405fc77 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa1fe00640 1 -- 172.21.15.66:0/609086830 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ea60afb600 con 0x55ea619b6c00 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa1fe00640 1 -- 172.21.15.66:0/609086830 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea6187ca80 con 0x55ea619b6c00 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa1fe00640 1 -- 172.21.15.66:0/609086830 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ea60afb1e0 con 0x55ea619b6c00 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 -- 172.21.15.66:0/609086830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b6c00 msgr2=0x55ea60b33600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 --2- 172.21.15.66:0/609086830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b6c00 0x55ea60b33600 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x55ea61879b00 tx=0x55ea619dcc00 comp rx=0 tx=0).stop 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 -- 172.21.15.66:0/609086830 shutdown_connections 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 --2- 172.21.15.66:0/609086830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea619b7400 0x55ea60b34100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 --2- 172.21.15.66:0/609086830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea619b7000 0x55ea60b33b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 --2- 172.21.15.66:0/609086830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b6c00 0x55ea60b33600 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 -- 172.21.15.66:0/609086830 >> 172.21.15.66:0/609086830 conn(0x55ea60aa6c00 msgr2=0x55ea60ad5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:20.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 -- 172.21.15.66:0/609086830 shutdown_connections 2024-03-30T00:50:20.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 -- 172.21.15.66:0/609086830 wait complete. 2024-03-30T00:50:20.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 Processor -- start 2024-03-30T00:50:20.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 -- start start 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea619b6c00 0x55ea60b35180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea619b7000 0x55ea60b34c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b7400 0x55ea60b34680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea60b743c0 con 0x55ea619b7400 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea60b74280 con 0x55ea619b7000 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa27692400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea60b74140 con 0x55ea619b6c00 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa24e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b7400 0x55ea60b34680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea619b6c00 0x55ea60b35180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa25800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea619b7000 0x55ea60b34c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea619b6c00 0x55ea60b35180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:48244/0 (socket says 172.21.15.66:48244) 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 -- 172.21.15.66:0/1918281640 learned_addr learned my addr 172.21.15.66:0/1918281640 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa24e00640 1 -- 172.21.15.66:0/1918281640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea619b6c00 msgr2=0x55ea60b35180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa24e00640 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea619b6c00 0x55ea60b35180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa24e00640 1 -- 172.21.15.66:0/1918281640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea619b7000 msgr2=0x55ea60b34c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa24e00640 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea619b7000 0x55ea60b34c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa24e00640 1 -- 172.21.15.66:0/1918281640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea60adc9c0 con 0x55ea619b7400 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa26200640 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea619b6c00 0x55ea60b35180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.404+0000 7faa25800640 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea619b7000 0x55ea60b34c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa24e00640 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b7400 0x55ea60b34680 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x55ea60d92d80 tx=0x55ea60b800c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa1e000640 1 -- 172.21.15.66:0/1918281640 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ea60afaf20 con 0x55ea619b7400 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa1e000640 1 -- 172.21.15.66:0/1918281640 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea6187cc00 con 0x55ea619b7400 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa27692400 1 -- 172.21.15.66:0/1918281640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ea60adcb60 con 0x55ea619b7400 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa1e000640 1 -- 172.21.15.66:0/1918281640 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ea60afb600 con 0x55ea619b7400 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa27692400 1 -- 172.21.15.66:0/1918281640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea60adcea0 con 0x55ea619b7400 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa27692400 4 main: resolving pool 15 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa1e000640 1 -- 172.21.15.66:0/1918281640 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x55ea60b72840 con 0x55ea619b7400 2024-03-30T00:50:20.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa1e000640 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55ea60d8bc00 0x55ea60b35700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa1e000640 1 -- 172.21.15.66:0/1918281640 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x55ea60ae6a80 con 0x55ea619b7400 2024-03-30T00:50:20.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa27692400 4 main: creating IoCtx.. 2024-03-30T00:50:20.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa27692400 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55ea619b7800 0x55ea60b34100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa26200640 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55ea60d8bc00 0x55ea60b35700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa27692400 1 -- 172.21.15.66:0/1918281640 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) v8 -- 0x55ea619b7c00 con 0x55ea619b7800 2024-03-30T00:50:20.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa25800640 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55ea619b7800 0x55ea60b34100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa26200640 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55ea60d8bc00 0x55ea60b35700 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x55ea60d92c30 tx=0x55ea619aec60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:20.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.408+0000 7faa25800640 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55ea619b7800 0x55ea60b34100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:20.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa25800640 1 -- 172.21.15.66:0/1918281640 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55ea6186ab40 con 0x55ea619b7800 2024-03-30T00:50:20.422 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:50:20.422 INFO:teuthology.orchestra.run.smithi066.stdout: "0": { 2024-03-30T00:50:20.422 INFO:teuthology.orchestra.run.smithi066.stdout: "data": { 2024-03-30T00:50:20.422 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 5, 2024-03-30T00:50:20.422 INFO:teuthology.orchestra.run.smithi066.stdout: "inotable": { 2024-03-30T00:50:20.422 INFO:teuthology.orchestra.run.smithi066.stdout: "projected_free": [ 2024-03-30T00:50:20.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:20.422 INFO:teuthology.orchestra.run.smithi066.stdout: "start": 1099511627779, 2024-03-30T00:50:20.422 INFO:teuthology.orchestra.run.smithi066.stdout: "len": 1099511627773 2024-03-30T00:50:20.422 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: "free": [ 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: "start": 1099511627779, 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: "len": 1099511627773 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: "result": 0 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:50:20.423 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55ea619b7800 msgr2=0x55ea60b34100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x55ea619b7800 0x55ea60b34100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55ea60d8bc00 msgr2=0x55ea60b35700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x55ea60d8bc00 0x55ea60b35700 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x55ea60d92c30 tx=0x55ea619aec60 comp rx=0 tx=0).stop 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b7400 msgr2=0x55ea60b34680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 --2- 172.21.15.66:0/1918281640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea619b7400 0x55ea60b34680 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x55ea60d92d80 tx=0x55ea60b800c0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa25800640 1 -- 172.21.15.66:0/1918281640 reap_dead start 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/1918281640 shutdown_connections 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/1918281640 >> 172.21.15.66:0/1918281640 conn(0x55ea60aa6c00 msgr2=0x55ea60ad5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/1918281640 shutdown_connections 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/1918281640 wait complete. 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/3082129468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea60aa7400 msgr2=0x55ea60b32b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 --2- 172.21.15.66:0/3082129468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea60aa7400 0x55ea60b32b00 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x55ea61878c90 tx=0x55ea619aed20 comp rx=0 tx=0).stop 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/3082129468 shutdown_connections 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 --2- 172.21.15.66:0/3082129468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ea60aa7000 0x55ea60b33080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 --2- 172.21.15.66:0/3082129468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ea60aa7400 0x55ea60b32b00 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 --2- 172.21.15.66:0/3082129468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea60aa7800 0x55ea60b32580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/3082129468 >> 172.21.15.66:0/3082129468 conn(0x55ea60aa7c00 msgr2=0x55ea60ad5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:20.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/3082129468 shutdown_connections 2024-03-30T00:50:20.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.412+0000 7faa27692400 1 -- 172.21.15.66:0/3082129468 wait complete. 2024-03-30T00:50:20.427 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.069349, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-03-30T00:50:20.427 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:50:20.427 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:50:20.516 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:50:20.592 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:50:20.592 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:50:20.592 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:50:20.601 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:50:20.601 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:50:20.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:20.601 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:50:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-30T00:50:20.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:50:20.654 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:50:20.654 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:50:20.664 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:50:20.664 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:50:20.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:20.664 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T00:50:20.712 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-30T00:50:20.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:50:20.714 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:50:20.714 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:50:20.722 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:50:20.723 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:50:20.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:20.723 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-30T00:50:20.772 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-30T00:50:20.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:50:20.773 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:50:20.773 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:50:20.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:50:20.783 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:50:20.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:20.783 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-30T00:50:20.833 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-30T00:50:20.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:50:20.834 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:50:20.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8abba00640 1 Processor -- start 2024-03-30T00:50:20.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8abba00640 1 -- start start 2024-03-30T00:50:20.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8abba00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8ab415de30 0x7f8ab40cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8abba00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8ab40af420 0x7f8ab40d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8abba00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab40cbb90 con 0x7f8ab40ce740 2024-03-30T00:50:20.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8abba00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab40ce1b0 con 0x7f8ab415de30 2024-03-30T00:50:20.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8abba00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab40afa10 con 0x7f8ab40af420 2024-03-30T00:50:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8ab9000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8ab415de30 0x7f8ab40cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8ab9000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8ab415de30 0x7f8ab40cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35422/0 (socket says 172.21.15.66:35422) 2024-03-30T00:50:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8ab9000640 1 -- 172.21.15.66:0/3909901328 learned_addr learned my addr 172.21.15.66:0/3909901328 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8aabe00640 1 --2- 172.21.15.66:0/3909901328 >> v2:172.21.15.156:3300/0 conn(0x7f8ab40af420 0x7f8ab40d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8ab9000640 1 -- 172.21.15.66:0/3909901328 >> v2:172.21.15.156:3300/0 conn(0x7f8ab40af420 msgr2=0x7f8ab40d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8ab9000640 1 --2- 172.21.15.66:0/3909901328 >> v2:172.21.15.156:3300/0 conn(0x7f8ab40af420 0x7f8ab40d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8ab9000640 1 -- 172.21.15.66:0/3909901328 >> v1:172.21.15.66:6789/0 conn(0x7f8ab40ce740 legacy=0x7f8ab406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.968+0000 7f8ab9000640 1 -- 172.21.15.66:0/3909901328 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab40cf430 con 0x7f8ab415de30 2024-03-30T00:50:20.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8ab9000640 1 --2- 172.21.15.66:0/3909901328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab415de30 0x7f8ab40cc040 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4070080 tx=0x7f8aa401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1312f2a8b5a6f512 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:20.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8ab3e00640 1 -- 172.21.15.66:0/3909901328 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40ab5f0 con 0x7f8ab415de30 2024-03-30T00:50:20.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8ab3e00640 1 -- 172.21.15.66:0/3909901328 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa40ab750 con 0x7f8ab415de30 2024-03-30T00:50:20.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8ab3e00640 1 -- 172.21.15.66:0/3909901328 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40abae0 con 0x7f8ab415de30 2024-03-30T00:50:20.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8abba00640 1 -- 172.21.15.66:0/3909901328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab415de30 msgr2=0x7f8ab40cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8abba00640 1 --2- 172.21.15.66:0/3909901328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab415de30 0x7f8ab40cc040 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4070080 tx=0x7f8aa401c490 comp rx=0 tx=0).stop 2024-03-30T00:50:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8abba00640 1 -- 172.21.15.66:0/3909901328 shutdown_connections 2024-03-30T00:50:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8abba00640 1 --2- 172.21.15.66:0/3909901328 >> v2:172.21.15.156:3300/0 conn(0x7f8ab40af420 0x7f8ab40d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8abba00640 1 --2- 172.21.15.66:0/3909901328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab415de30 0x7f8ab40cc040 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8abba00640 1 -- 172.21.15.66:0/3909901328 >> 172.21.15.66:0/3909901328 conn(0x7f8ab40d0540 msgr2=0x7f8ab40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8abba00640 1 -- 172.21.15.66:0/3909901328 shutdown_connections 2024-03-30T00:50:20.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.972+0000 7f8abba00640 1 -- 172.21.15.66:0/3909901328 wait complete. 2024-03-30T00:50:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8abba00640 1 Processor -- start 2024-03-30T00:50:20.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8abba00640 1 -- start start 2024-03-30T00:50:20.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8abba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab40af420 0x7f8ab41574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8abba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ab40ce740 0x7f8ab40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8abba00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ab415de30 0x7f8ab415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8abba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40b4350 con 0x7f8ab40af420 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8abba00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40b11b0 con 0x7f8ab415de30 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8abba00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40b1490 con 0x7f8ab40ce740 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab40af420 0x7f8ab41574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab40af420 0x7f8ab41574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35438/0 (socket says 172.21.15.66:35438) 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab9000640 1 -- 172.21.15.66:0/248579349 learned_addr learned my addr 172.21.15.66:0/248579349 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8aabe00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ab40ce740 0x7f8ab40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab9a00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ab415de30 0x7f8ab415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab9000640 1 -- 172.21.15.66:0/248579349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ab40ce740 msgr2=0x7f8ab40b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab9000640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ab40ce740 0x7f8ab40b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab9000640 1 -- 172.21.15.66:0/248579349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ab415de30 msgr2=0x7f8ab415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab9000640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ab415de30 0x7f8ab415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab9000640 1 -- 172.21.15.66:0/248579349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa40094d0 con 0x7f8ab40af420 2024-03-30T00:50:20.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8aabe00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ab40ce740 0x7f8ab40b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:20.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab9a00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ab415de30 0x7f8ab415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:20.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab9000640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab40af420 0x7f8ab41574d0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40190a0 tx=0x7f8aa401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:20.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab1600640 1 -- 172.21.15.66:0/248579349 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40b4800 con 0x7f8ab40af420 2024-03-30T00:50:20.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab1600640 1 -- 172.21.15.66:0/248579349 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa40b4960 con 0x7f8ab40af420 2024-03-30T00:50:20.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8ab1600640 1 -- 172.21.15.66:0/248579349 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40b4d10 con 0x7f8ab40af420 2024-03-30T00:50:20.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8abba00640 1 -- 172.21.15.66:0/248579349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab40b1770 con 0x7f8ab40af420 2024-03-30T00:50:20.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.976+0000 7f8abba00640 1 -- 172.21.15.66:0/248579349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab40b4000 con 0x7f8ab40af420 2024-03-30T00:50:20.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.980+0000 7f8abba00640 1 -- 172.21.15.66:0/248579349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab40cdf60 con 0x7f8ab40af420 2024-03-30T00:50:20.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.980+0000 7f8ab1600640 1 -- 172.21.15.66:0/248579349 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8aa40c7390 con 0x7f8ab40af420 2024-03-30T00:50:20.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.980+0000 7f8ab1600640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a900666e0 0x7f8a90068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:20.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.980+0000 7f8ab1600640 1 -- 172.21.15.66:0/248579349 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f8aa40a8050 con 0x7f8ab40af420 2024-03-30T00:50:20.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.980+0000 7f8aabe00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a900666e0 0x7f8a90068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:20.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.980+0000 7f8aabe00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a900666e0 0x7f8a90068ba0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40c9e90 tx=0x7f8a9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:20.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:20.984+0000 7f8ab1600640 1 -- 172.21.15.66:0/248579349 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8aa40ebe00 con 0x7f8ab40af420 2024-03-30T00:50:21.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.180+0000 7f8abba00640 1 -- 172.21.15.66:0/248579349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ab4065fd0 con 0x7f8ab40af420 2024-03-30T00:50:21.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.180+0000 7f8ab1600640 1 -- 172.21.15.66:0/248579349 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+3976 (secure 0 0 0) 0x7f8aa40ef420 con 0x7f8ab40af420 2024-03-30T00:50:21.188 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:21.188 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":86,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":86,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:50:20.318282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:50:21.188 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 86 2024-03-30T00:50:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 -- 172.21.15.66:0/248579349 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a900666e0 msgr2=0x7f8a90068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a900666e0 0x7f8a90068ba0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40c9e90 tx=0x7f8a9c008040 comp rx=0 tx=0).stop 2024-03-30T00:50:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 -- 172.21.15.66:0/248579349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab40af420 msgr2=0x7f8ab41574d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab40af420 0x7f8ab41574d0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40190a0 tx=0x7f8aa401acd0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 -- 172.21.15.66:0/248579349 shutdown_connections 2024-03-30T00:50:21.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ab415de30 0x7f8ab415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ab40ce740 0x7f8ab40b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8a900666e0 0x7f8a90068ba0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 --2- 172.21.15.66:0/248579349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab40af420 0x7f8ab41574d0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 -- 172.21.15.66:0/248579349 >> 172.21.15.66:0/248579349 conn(0x7f8ab40d0540 msgr2=0x7f8ab40caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:21.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 -- 172.21.15.66:0/248579349 shutdown_connections 2024-03-30T00:50:21.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.184+0000 7f8abba00640 1 -- 172.21.15.66:0/248579349 wait complete. 2024-03-30T00:50:21.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:50:21.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed16e00640 1 Processor -- start 2024-03-30T00:50:21.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed16e00640 1 -- start start 2024-03-30T00:50:21.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed16e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fed100cde70 0x7fed1015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed16e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed100cd3e0 con 0x7fed100cec30 2024-03-30T00:50:21.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed16e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1015a990 con 0x7fed100cda90 2024-03-30T00:50:21.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed16e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed100b1b80 con 0x7fed100cde70 2024-03-30T00:50:21.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed14e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fed100cde70 0x7fed1015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed07400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fed100cec30 0x7fed10159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55278/0 (socket says 172.21.15.66:55278) 2024-03-30T00:50:21.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed07400640 1 -- 172.21.15.66:0/2528705689 learned_addr learned my addr 172.21.15.66:0/2528705689 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:21.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed0f400640 1 -- 172.21.15.66:0/2528705689 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870788818 0 0) 0x7fed100cd3e0 con 0x7fed100cec30 2024-03-30T00:50:21.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed14e00640 1 -- 172.21.15.66:0/2528705689 >> v1:172.21.15.156:6789/0 conn(0x7fed100cda90 legacy=0x7fed100ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed14e00640 1 -- 172.21.15.66:0/2528705689 >> v1:172.21.15.66:6789/0 conn(0x7fed100cec30 legacy=0x7fed10159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.344+0000 7fed14e00640 1 -- 172.21.15.66:0/2528705689 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed100b11e0 con 0x7fed100cde70 2024-03-30T00:50:21.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed14e00640 1 --2- 172.21.15.66:0/2528705689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed100cde70 0x7fed1015a370 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fed08009d90 tx=0x7fed0801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da036bc1c524dc32 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:21.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed0f400640 1 -- 172.21.15.66:0/2528705689 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed080a59a0 con 0x7fed100cde70 2024-03-30T00:50:21.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed0f400640 1 -- 172.21.15.66:0/2528705689 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed080a5b00 con 0x7fed100cde70 2024-03-30T00:50:21.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed0f400640 1 -- 172.21.15.66:0/2528705689 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed080a5e90 con 0x7fed100cde70 2024-03-30T00:50:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed16e00640 1 -- 172.21.15.66:0/2528705689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed100cde70 msgr2=0x7fed1015a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed16e00640 1 --2- 172.21.15.66:0/2528705689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed100cde70 0x7fed1015a370 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fed08009d90 tx=0x7fed0801c8e0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed16e00640 1 -- 172.21.15.66:0/2528705689 shutdown_connections 2024-03-30T00:50:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed16e00640 1 --2- 172.21.15.66:0/2528705689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed100cde70 0x7fed1015a370 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed16e00640 1 -- 172.21.15.66:0/2528705689 >> 172.21.15.66:0/2528705689 conn(0x7fed10069f60 msgr2=0x7fed1006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed16e00640 1 -- 172.21.15.66:0/2528705689 shutdown_connections 2024-03-30T00:50:21.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed16e00640 1 -- 172.21.15.66:0/2528705689 wait complete. 2024-03-30T00:50:21.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed16e00640 1 Processor -- start 2024-03-30T00:50:21.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.348+0000 7fed16e00640 1 -- start start 2024-03-30T00:50:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed16e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed100cda90 0x7fed100c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed16e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed100cde70 0x7fed1006c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed16e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed100cec30 0x7fed1006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed16e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed100c2320 con 0x7fed100cda90 2024-03-30T00:50:21.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed16e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed10155480 con 0x7fed100cde70 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed16e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed10094660 con 0x7fed100cec30 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed100cda90 0x7fed100c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed100cda90 0x7fed100c1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35460/0 (socket says 172.21.15.66:35460) 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0fe00640 1 -- 172.21.15.66:0/3449809061 learned_addr learned my addr 172.21.15.66:0/3449809061 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed14e00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed100cec30 0x7fed1006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed07400640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed100cde70 0x7fed1006c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0fe00640 1 -- 172.21.15.66:0/3449809061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed100cec30 msgr2=0x7fed1006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0fe00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed100cec30 0x7fed1006cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0fe00640 1 -- 172.21.15.66:0/3449809061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed100cde70 msgr2=0x7fed1006c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0fe00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed100cde70 0x7fed1006c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0fe00640 1 -- 172.21.15.66:0/3449809061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed08009a70 con 0x7fed100cda90 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed14e00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed100cec30 0x7fed1006cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0fe00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed100cda90 0x7fed100c1de0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fed0000c3c0 tx=0x7fed0000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:21.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0cc00640 1 -- 172.21.15.66:0/3449809061 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000052d0 con 0x7fed100cda90 2024-03-30T00:50:21.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0cc00640 1 -- 172.21.15.66:0/3449809061 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed00005430 con 0x7fed100cda90 2024-03-30T00:50:21.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0cc00640 1 -- 172.21.15.66:0/3449809061 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00003840 con 0x7fed100cda90 2024-03-30T00:50:21.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed16e00640 1 -- 172.21.15.66:0/3449809061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed100938a0 con 0x7fed100cda90 2024-03-30T00:50:21.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed16e00640 1 -- 172.21.15.66:0/3449809061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed10093e60 con 0x7fed100cda90 2024-03-30T00:50:21.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed16e00640 1 -- 172.21.15.66:0/3449809061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed10066c30 con 0x7fed100cda90 2024-03-30T00:50:21.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.352+0000 7fed0cc00640 1 -- 172.21.15.66:0/3449809061 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fed000039a0 con 0x7fed100cda90 2024-03-30T00:50:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.356+0000 7fed0cc00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fecec066690 0x7fecec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.356+0000 7fed0cc00640 1 -- 172.21.15.66:0/3449809061 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fed000838e0 con 0x7fed100cda90 2024-03-30T00:50:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.356+0000 7fed07400640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fecec066690 0x7fecec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.356+0000 7fed07400640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fecec066690 0x7fecec068b50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fecf8007fd0 tx=0x7fecf8007ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.360+0000 7fed0cc00640 1 -- 172.21.15.66:0/3449809061 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fed00050a30 con 0x7fed100cda90 2024-03-30T00:50:21.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.524+0000 7fed16e00640 1 -- 172.21.15.66:0/3449809061 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed100b3170 con 0x7fed100cda90 2024-03-30T00:50:21.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.524+0000 7fed0cc00640 1 -- 172.21.15.66:0/3449809061 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16555 (secure 0 0 0) 0x7fed00054050 con 0x7fed100cda90 2024-03-30T00:50:21.532 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:21.532 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":114,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:20.317752+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:48:44.750350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-30T00:48:45.327928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.35","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6837/282507324":"2024-03-31T00:50:20.317728+0000","172.21.15.66:6835/282507324":"2024-03-31T00:50:20.317728+0000","172.21.15.66:6837/2988412199":"2024-03-31T00:49:43.612805+0000","172.21.15.66:6835/2988412199":"2024-03-31T00:49:43.612805+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:50:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 -- 172.21.15.66:0/3449809061 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fecec066690 msgr2=0x7fecec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fecec066690 0x7fecec068b50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fecf8007fd0 tx=0x7fecf8007ba0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 -- 172.21.15.66:0/3449809061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed100cda90 msgr2=0x7fed100c1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed100cda90 0x7fed100c1de0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fed0000c3c0 tx=0x7fed0000c880 comp rx=0 tx=0).stop 2024-03-30T00:50:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 -- 172.21.15.66:0/3449809061 shutdown_connections 2024-03-30T00:50:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed100cec30 0x7fed1006cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed100cde70 0x7fed1006c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fecec066690 0x7fecec068b50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 --2- 172.21.15.66:0/3449809061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed100cda90 0x7fed100c1de0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 -- 172.21.15.66:0/3449809061 >> 172.21.15.66:0/3449809061 conn(0x7fed10069f60 msgr2=0x7fed1006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 -- 172.21.15.66:0/3449809061 shutdown_connections 2024-03-30T00:50:21.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.528+0000 7fed16e00640 1 -- 172.21.15.66:0/3449809061 wait complete. 2024-03-30T00:50:21.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:48:44.750350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:48:45.327928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:50:21.554 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:50:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd598400640 1 Processor -- start 2024-03-30T00:50:21.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd598400640 1 -- start start 2024-03-30T00:50:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd598400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd5900cf2f0 0x7fd59006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd598400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5900cb0e0 con 0x7fd59015ca80 2024-03-30T00:50:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd598400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5900cf7f0 con 0x7fd5900cb570 2024-03-30T00:50:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd598400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5900cfad0 con 0x7fd5900cf2f0 2024-03-30T00:50:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd595a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd59015ca80 0x7fd59006b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55288/0 (socket says 172.21.15.66:55288) 2024-03-30T00:50:21.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd595a00640 1 -- 172.21.15.66:0/941810278 learned_addr learned my addr 172.21.15.66:0/941810278 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd595000640 1 --2- 172.21.15.66:0/941810278 >> v2:172.21.15.156:3300/0 conn(0x7fd5900cf2f0 0x7fd59006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125944888 0 0) 0x7fd5900cb0e0 con 0x7fd59015ca80 2024-03-30T00:50:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd578003530 con 0x7fd59015ca80 2024-03-30T00:50:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389932748 0 0) 0x7fd5900cf7f0 con 0x7fd5900cb570 2024-03-30T00:50:21.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5900cb0e0 con 0x7fd5900cb570 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 967649036 0 0) 0x7fd578003530 con 0x7fd59015ca80 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 >> v2:172.21.15.156:3300/0 conn(0x7fd5900cf2f0 msgr2=0x7fd59006b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 --2- 172.21.15.66:0/941810278 >> v2:172.21.15.156:3300/0 conn(0x7fd5900cf2f0 0x7fd59006b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 >> v1:172.21.15.156:6789/0 conn(0x7fd5900cb570 legacy=0x7fd5900cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5900cad70 con 0x7fd59015ca80 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fd584001680 con 0x7fd59015ca80 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 >> v1:172.21.15.66:6789/0 conn(0x7fd59015ca80 legacy=0x7fd59006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 --2- 172.21.15.66:0/941810278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd578003b10 0x7fd578003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 --2- 172.21.15.66:0/941810278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5780047d0 0x7fd578004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 --2- 172.21.15.66:0/941810278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd578005190 0x7fd57800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900cfad0 con 0x7fd578003b10 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57800bf80 con 0x7fd5780047d0 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57800c140 con 0x7fd578005190 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd595a00640 1 --2- 172.21.15.66:0/941810278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd578003b10 0x7fd578003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd595000640 1 --2- 172.21.15.66:0/941810278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5780047d0 0x7fd578004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd596400640 1 --2- 172.21.15.66:0/941810278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd578005190 0x7fd57800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd595a00640 1 -- 172.21.15.66:0/941810278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd578005190 msgr2=0x7fd57800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:21.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd595a00640 1 --2- 172.21.15.66:0/941810278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd578005190 0x7fd57800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd595a00640 1 -- 172.21.15.66:0/941810278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5780047d0 msgr2=0x7fd578004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:21.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd595a00640 1 --2- 172.21.15.66:0/941810278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5780047d0 0x7fd578004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd595a00640 1 -- 172.21.15.66:0/941810278 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578003530 con 0x7fd578003b10 2024-03-30T00:50:21.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.692+0000 7fd596400640 1 -- 172.21.15.66:0/941810278 reap_dead start 2024-03-30T00:50:21.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.696+0000 7fd595a00640 1 --2- 172.21.15.66:0/941810278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd578003b10 0x7fd578003f10 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fd584001ec0 tx=0x7fd58401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4d72c58420fd92 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:21.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.696+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5840058e0 con 0x7fd578003b10 2024-03-30T00:50:21.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.696+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd584005a60 con 0x7fd578003b10 2024-03-30T00:50:21.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.696+0000 7fd58fe00640 1 -- 172.21.15.66:0/941810278 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584005e50 con 0x7fd578003b10 2024-03-30T00:50:21.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.696+0000 7fd598400640 1 -- 172.21.15.66:0/941810278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd578003b10 msgr2=0x7fd578003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.696+0000 7fd598400640 1 --2- 172.21.15.66:0/941810278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd578003b10 0x7fd578003f10 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fd584001ec0 tx=0x7fd58401cba0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.696+0000 7fd598400640 1 -- 172.21.15.66:0/941810278 shutdown_connections 2024-03-30T00:50:21.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.696+0000 7fd598400640 1 --2- 172.21.15.66:0/941810278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd578003b10 0x7fd578003f10 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.696+0000 7fd598400640 1 -- 172.21.15.66:0/941810278 >> 172.21.15.66:0/941810278 conn(0x7fd5900b2090 msgr2=0x7fd5900b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:21.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.696+0000 7fd598400640 1 -- 172.21.15.66:0/941810278 shutdown_connections 2024-03-30T00:50:21.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.696+0000 7fd598400640 1 -- 172.21.15.66:0/941810278 wait complete. 2024-03-30T00:50:21.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd598400640 1 Processor -- start 2024-03-30T00:50:21.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd598400640 1 -- start start 2024-03-30T00:50:21.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd598400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd578003b10 0x7fd5900c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd598400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd590158b00 0x7fd5900b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd598400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5900b59f0 0x7fd59015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd598400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900b3c00 con 0x7fd590158b00 2024-03-30T00:50:21.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd598400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900b5ea0 con 0x7fd578003b10 2024-03-30T00:50:21.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd598400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900b3260 con 0x7fd5900b59f0 2024-03-30T00:50:21.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd595000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd590158b00 0x7fd5900b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd595000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd590158b00 0x7fd5900b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35486/0 (socket says 172.21.15.66:35486) 2024-03-30T00:50:21.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd595000640 1 -- 172.21.15.66:0/1716222748 learned_addr learned my addr 172.21.15.66:0/1716222748 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:21.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd595a00640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd578003b10 0x7fd5900c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd596400640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5900b59f0 0x7fd59015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd595000640 1 -- 172.21.15.66:0/1716222748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5900b59f0 msgr2=0x7fd59015eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd595000640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5900b59f0 0x7fd59015eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd595000640 1 -- 172.21.15.66:0/1716222748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd578003b10 msgr2=0x7fd5900c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd595000640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd578003b10 0x7fd5900c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd595000640 1 -- 172.21.15.66:0/1716222748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5900b3540 con 0x7fd590158b00 2024-03-30T00:50:21.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd595000640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd590158b00 0x7fd5900b54b0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fd57c011ad0 tx=0x7fd57c011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:21.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd58d600640 1 -- 172.21.15.66:0/1716222748 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00fbb0 con 0x7fd590158b00 2024-03-30T00:50:21.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd58d600640 1 -- 172.21.15.66:0/1716222748 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c00fd10 con 0x7fd590158b00 2024-03-30T00:50:21.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd598400640 1 -- 172.21.15.66:0/1716222748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5900b3820 con 0x7fd590158b00 2024-03-30T00:50:21.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd58d600640 1 -- 172.21.15.66:0/1716222748 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c014680 con 0x7fd590158b00 2024-03-30T00:50:21.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd598400640 1 -- 172.21.15.66:0/1716222748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5900c83e0 con 0x7fd590158b00 2024-03-30T00:50:21.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.700+0000 7fd598400640 1 -- 172.21.15.66:0/1716222748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd590066c30 con 0x7fd590158b00 2024-03-30T00:50:21.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.704+0000 7fd58d600640 1 -- 172.21.15.66:0/1716222748 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd57c01d400 con 0x7fd590158b00 2024-03-30T00:50:21.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.704+0000 7fd58d600640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd570066690 0x7fd570068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:21.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.704+0000 7fd58d600640 1 -- 172.21.15.66:0/1716222748 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fd57c084b90 con 0x7fd590158b00 2024-03-30T00:50:21.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.704+0000 7fd595a00640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd570066690 0x7fd570068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:21.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.704+0000 7fd595a00640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd570066690 0x7fd570068b50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd584001ec0 tx=0x7fd584001a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:21.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.708+0000 7fd58d600640 1 -- 172.21.15.66:0/1716222748 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd57c051d90 con 0x7fd590158b00 2024-03-30T00:50:21.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 -- 172.21.15.66:0/1716222748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5900b1b70 con 0x7fd590158b00 2024-03-30T00:50:21.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd58d600640 1 -- 172.21.15.66:0/1716222748 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd57c0553b0 con 0x7fd590158b00 2024-03-30T00:50:21.878 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:50:21.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 -- 172.21.15.66:0/1716222748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd570066690 msgr2=0x7fd570068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd570066690 0x7fd570068b50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd584001ec0 tx=0x7fd584001a00 comp rx=0 tx=0).stop 2024-03-30T00:50:21.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 -- 172.21.15.66:0/1716222748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd590158b00 msgr2=0x7fd5900b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:21.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd590158b00 0x7fd5900b54b0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fd57c011ad0 tx=0x7fd57c011f90 comp rx=0 tx=0).stop 2024-03-30T00:50:21.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 -- 172.21.15.66:0/1716222748 shutdown_connections 2024-03-30T00:50:21.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5900b59f0 0x7fd59015eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd570066690 0x7fd570068b50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd590158b00 0x7fd5900b54b0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 --2- 172.21.15.66:0/1716222748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd578003b10 0x7fd5900c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:21.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 -- 172.21.15.66:0/1716222748 >> 172.21.15.66:0/1716222748 conn(0x7fd5900b2090 msgr2=0x7fd5900cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:21.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 -- 172.21.15.66:0/1716222748 shutdown_connections 2024-03-30T00:50:21.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:21.872+0000 7fd598400640 1 -- 172.21.15.66:0/1716222748 wait complete. 2024-03-30T00:50:21.899 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:50:22.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943fc00640 1 Processor -- start 2024-03-30T00:50:22.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943fc00640 1 -- start start 2024-03-30T00:50:22.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943fc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f943815a640 0x7f943806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943fc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94380ce520 0x7f94380ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943fc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f94380cabb0 0x7f94380cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943fc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943806d070 con 0x7f943815a640 2024-03-30T00:50:22.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943fc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94380cbba0 con 0x7f94380cabb0 2024-03-30T00:50:22.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943fc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943806e2c0 con 0x7f94380ce520 2024-03-30T00:50:22.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943d200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f943815a640 0x7f943806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943d200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f943815a640 0x7f943806dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35496/0 (socket says 172.21.15.66:35496) 2024-03-30T00:50:22.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943d200640 1 -- 172.21.15.66:0/3307648016 learned_addr learned my addr 172.21.15.66:0/3307648016 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:22.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943dc00640 1 --2- 172.21.15.66:0/3307648016 >> v2:172.21.15.156:3300/0 conn(0x7f94380cabb0 0x7f94380cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f942fe00640 1 --2- 172.21.15.66:0/3307648016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94380ce520 0x7f94380ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943d200640 1 -- 172.21.15.66:0/3307648016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94380ce520 msgr2=0x7f94380ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:22.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943d200640 1 --2- 172.21.15.66:0/3307648016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94380ce520 0x7f94380ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943d200640 1 -- 172.21.15.66:0/3307648016 >> v2:172.21.15.156:3300/0 conn(0x7f94380cabb0 msgr2=0x7f94380cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:22.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.036+0000 7f943d200640 1 --2- 172.21.15.66:0/3307648016 >> v2:172.21.15.156:3300/0 conn(0x7f94380cabb0 0x7f94380cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f943d200640 1 -- 172.21.15.66:0/3307648016 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380cee60 con 0x7f943815a640 2024-03-30T00:50:22.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f943d200640 1 --2- 172.21.15.66:0/3307648016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f943815a640 0x7f943806dd80 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f94380acfe0 tx=0x7f942801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c6c1cb76a0e75a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:22.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f9437e00640 1 -- 172.21.15.66:0/3307648016 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94280aa610 con 0x7f943815a640 2024-03-30T00:50:22.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f9437e00640 1 -- 172.21.15.66:0/3307648016 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94280a6040 con 0x7f943815a640 2024-03-30T00:50:22.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f9437e00640 1 -- 172.21.15.66:0/3307648016 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94280aa990 con 0x7f943815a640 2024-03-30T00:50:22.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f943fc00640 1 -- 172.21.15.66:0/3307648016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f943815a640 msgr2=0x7f943806dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f943fc00640 1 --2- 172.21.15.66:0/3307648016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f943815a640 0x7f943806dd80 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f94380acfe0 tx=0x7f942801c4f0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f943fc00640 1 -- 172.21.15.66:0/3307648016 shutdown_connections 2024-03-30T00:50:22.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f943fc00640 1 --2- 172.21.15.66:0/3307648016 >> v2:172.21.15.156:3300/0 conn(0x7f94380cabb0 0x7f94380cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f943fc00640 1 --2- 172.21.15.66:0/3307648016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94380ce520 0x7f94380ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f943fc00640 1 --2- 172.21.15.66:0/3307648016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f943815a640 0x7f943806dd80 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f943fc00640 1 -- 172.21.15.66:0/3307648016 >> 172.21.15.66:0/3307648016 conn(0x7f9438069f60 msgr2=0x7f943806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:22.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f943fc00640 1 -- 172.21.15.66:0/3307648016 shutdown_connections 2024-03-30T00:50:22.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.040+0000 7f943fc00640 1 -- 172.21.15.66:0/3307648016 wait complete. 2024-03-30T00:50:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f943fc00640 1 Processor -- start 2024-03-30T00:50:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f943fc00640 1 -- start start 2024-03-30T00:50:22.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f943fc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94380cabb0 0x7f94381537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f943fc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94380ce520 0x7f94380978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f943fc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f943815a640 0x7f94380aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f943fc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380c1da0 con 0x7f94380ce520 2024-03-30T00:50:22.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f943fc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9438152f10 con 0x7f943815a640 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f943fc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94381525e0 con 0x7f94380cabb0 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f942fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94380ce520 0x7f94380978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f942fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94380ce520 0x7f94380978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35512/0 (socket says 172.21.15.66:35512) 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f942fe00640 1 -- 172.21.15.66:0/2774305561 learned_addr learned my addr 172.21.15.66:0/2774305561 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f943d200640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94380cabb0 0x7f94381537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f943dc00640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f943815a640 0x7f94380aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f942fe00640 1 -- 172.21.15.66:0/2774305561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94380cabb0 msgr2=0x7f94381537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f942fe00640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94380cabb0 0x7f94381537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f942fe00640 1 -- 172.21.15.66:0/2774305561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f943815a640 msgr2=0x7f94380aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f942fe00640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f943815a640 0x7f94380aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f942fe00640 1 -- 172.21.15.66:0/2774305561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94280094d0 con 0x7f94380ce520 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f942fe00640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94380ce520 0x7f94380978b0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f942000e930 tx=0x7f942000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f9435600640 1 -- 172.21.15.66:0/2774305561 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420009690 con 0x7f94380ce520 2024-03-30T00:50:22.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f9435600640 1 -- 172.21.15.66:0/2774305561 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942000ca00 con 0x7f94380ce520 2024-03-30T00:50:22.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f943fc00640 1 -- 172.21.15.66:0/2774305561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94381528c0 con 0x7f94380ce520 2024-03-30T00:50:22.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.044+0000 7f9435600640 1 -- 172.21.15.66:0/2774305561 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942000cd70 con 0x7f94380ce520 2024-03-30T00:50:22.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.048+0000 7f943fc00640 1 -- 172.21.15.66:0/2774305561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9438153d30 con 0x7f94380ce520 2024-03-30T00:50:22.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.048+0000 7f943fc00640 1 -- 172.21.15.66:0/2774305561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9438066c30 con 0x7f94380ce520 2024-03-30T00:50:22.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.048+0000 7f9435600640 1 -- 172.21.15.66:0/2774305561 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9420013470 con 0x7f94380ce520 2024-03-30T00:50:22.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.048+0000 7f9435600640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9414066690 0x7f9414068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.048+0000 7f9435600640 1 -- 172.21.15.66:0/2774305561 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f94200845b0 con 0x7f94380ce520 2024-03-30T00:50:22.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.048+0000 7f943d200640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9414066690 0x7f9414068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.048+0000 7f943d200640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9414066690 0x7f9414068b50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f94280ae000 tx=0x7f94280b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.052+0000 7f9435600640 1 -- 172.21.15.66:0/2774305561 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f94200517e0 con 0x7f94380ce520 2024-03-30T00:50:22.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.212+0000 7f943fc00640 1 -- 172.21.15.66:0/2774305561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f943815bfe0 con 0x7f94380ce520 2024-03-30T00:50:22.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.212+0000 7f9435600640 1 -- 172.21.15.66:0/2774305561 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9420054e00 con 0x7f94380ce520 2024-03-30T00:50:22.220 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:50:22.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 -- 172.21.15.66:0/2774305561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9414066690 msgr2=0x7f9414068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9414066690 0x7f9414068b50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f94280ae000 tx=0x7f94280b5040 comp rx=0 tx=0).stop 2024-03-30T00:50:22.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 -- 172.21.15.66:0/2774305561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94380ce520 msgr2=0x7f94380978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94380ce520 0x7f94380978b0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f942000e930 tx=0x7f942000edf0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 -- 172.21.15.66:0/2774305561 shutdown_connections 2024-03-30T00:50:22.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f943815a640 0x7f94380aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9414066690 0x7f9414068b50 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94380ce520 0x7f94380978b0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 --2- 172.21.15.66:0/2774305561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94380cabb0 0x7f94381537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 -- 172.21.15.66:0/2774305561 >> 172.21.15.66:0/2774305561 conn(0x7f9438069f60 msgr2=0x7f943815c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:22.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 -- 172.21.15.66:0/2774305561 shutdown_connections 2024-03-30T00:50:22.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.216+0000 7f943fc00640 1 -- 172.21.15.66:0/2774305561 wait complete. 2024-03-30T00:50:22.240 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:50:22.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986be00640 1 Processor -- start 2024-03-30T00:50:22.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986be00640 1 -- start start 2024-03-30T00:50:22.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f986c06cb90 0x7f986c0d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986be00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c15d250 con 0x7f986c0cdef0 2024-03-30T00:50:22.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986be00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c15d530 con 0x7f986c15c6b0 2024-03-30T00:50:22.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c06b1f0 con 0x7f986c06cb90 2024-03-30T00:50:22.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986b400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f986c0cdef0 0x7f986c1604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55302/0 (socket says 172.21.15.66:55302) 2024-03-30T00:50:22.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986b400640 1 -- 172.21.15.66:0/2494461244 learned_addr learned my addr 172.21.15.66:0/2494461244 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:22.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f9863e00640 1 --2- 172.21.15.66:0/2494461244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f986c06cb90 0x7f986c0d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986a000640 1 -- 172.21.15.66:0/2494461244 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251142302 0 0) 0x7f986c15d250 con 0x7f986c0cdef0 2024-03-30T00:50:22.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986a000640 1 -- 172.21.15.66:0/2494461244 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9850003530 con 0x7f986c0cdef0 2024-03-30T00:50:22.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986a000640 1 -- 172.21.15.66:0/2494461244 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103182633 0 0) 0x7f986c15d530 con 0x7f986c15c6b0 2024-03-30T00:50:22.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f986a000640 1 -- 172.21.15.66:0/2494461244 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f986c15d250 con 0x7f986c15c6b0 2024-03-30T00:50:22.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f9863e00640 1 -- 172.21.15.66:0/2494461244 >> v1:172.21.15.156:6789/0 conn(0x7f986c15c6b0 legacy=0x7f986c15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f9863e00640 1 -- 172.21.15.66:0/2494461244 >> v1:172.21.15.66:6789/0 conn(0x7f986c0cdef0 legacy=0x7f986c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.380+0000 7f9863e00640 1 -- 172.21.15.66:0/2494461244 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c15d810 con 0x7f986c06cb90 2024-03-30T00:50:22.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f9863e00640 1 --2- 172.21.15.66:0/2494461244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f986c06cb90 0x7f986c0d0c50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f98540097f0 tx=0x7f985401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7222b96142e83801 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:22.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986a000640 1 -- 172.21.15.66:0/2494461244 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540ac630 con 0x7f986c06cb90 2024-03-30T00:50:22.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986a000640 1 -- 172.21.15.66:0/2494461244 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98540ac790 con 0x7f986c06cb90 2024-03-30T00:50:22.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986a000640 1 -- 172.21.15.66:0/2494461244 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540acb20 con 0x7f986c06cb90 2024-03-30T00:50:22.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986be00640 1 -- 172.21.15.66:0/2494461244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f986c06cb90 msgr2=0x7f986c0d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986be00640 1 --2- 172.21.15.66:0/2494461244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f986c06cb90 0x7f986c0d0c50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f98540097f0 tx=0x7f985401c4b0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986be00640 1 -- 172.21.15.66:0/2494461244 shutdown_connections 2024-03-30T00:50:22.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986be00640 1 --2- 172.21.15.66:0/2494461244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f986c06cb90 0x7f986c0d0c50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986be00640 1 -- 172.21.15.66:0/2494461244 >> 172.21.15.66:0/2494461244 conn(0x7f986c069d90 msgr2=0x7f986c06a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:22.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986be00640 1 -- 172.21.15.66:0/2494461244 shutdown_connections 2024-03-30T00:50:22.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986be00640 1 -- 172.21.15.66:0/2494461244 wait complete. 2024-03-30T00:50:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986be00640 1 Processor -- start 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.384+0000 7f986be00640 1 -- start start 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f986be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f986c06cb90 0x7f986c15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f986be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f986c0cdef0 0x7f986c1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f986be00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f986c15c6b0 0x7f986c1538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f986be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c158ce0 con 0x7f986c0cdef0 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f986be00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c0ac260 con 0x7f986c15c6b0 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f986be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c158ef0 con 0x7f986c06cb90 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9863e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f986c0cdef0 0x7f986c1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9863e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f986c0cdef0 0x7f986c1587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35526/0 (socket says 172.21.15.66:35526) 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9863e00640 1 -- 172.21.15.66:0/1377747267 learned_addr learned my addr 172.21.15.66:0/1377747267 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f986aa00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f986c06cb90 0x7f986c15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f986b400640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f986c15c6b0 0x7f986c1538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9863e00640 1 -- 172.21.15.66:0/1377747267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f986c06cb90 msgr2=0x7f986c15ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9863e00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f986c06cb90 0x7f986c15ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9863e00640 1 -- 172.21.15.66:0/1377747267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f986c15c6b0 msgr2=0x7f986c1538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9863e00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f986c15c6b0 0x7f986c1538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9863e00640 1 -- 172.21.15.66:0/1377747267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9854009420 con 0x7f986c0cdef0 2024-03-30T00:50:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9863e00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f986c0cdef0 0x7f986c1587a0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f98540190c0 tx=0x7f9854019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9862a00640 1 -- 172.21.15.66:0/1377747267 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540b5610 con 0x7f986c0cdef0 2024-03-30T00:50:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9862a00640 1 -- 172.21.15.66:0/1377747267 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98540b5790 con 0x7f986c0cdef0 2024-03-30T00:50:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f9862a00640 1 -- 172.21.15.66:0/1377747267 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540b5b40 con 0x7f986c0cdef0 2024-03-30T00:50:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f986be00640 1 -- 172.21.15.66:0/1377747267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c1591d0 con 0x7f986c0cdef0 2024-03-30T00:50:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f986be00640 1 -- 172.21.15.66:0/1377747267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c159790 con 0x7f986c0cdef0 2024-03-30T00:50:22.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.388+0000 7f986be00640 1 -- 172.21.15.66:0/1377747267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c0ac3d0 con 0x7f986c0cdef0 2024-03-30T00:50:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.392+0000 7f9862a00640 1 -- 172.21.15.66:0/1377747267 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f98540b3040 con 0x7f986c0cdef0 2024-03-30T00:50:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.392+0000 7f9862a00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9848066690 0x7f9848068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.392+0000 7f9862a00640 1 -- 172.21.15.66:0/1377747267 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f985411e420 con 0x7f986c0cdef0 2024-03-30T00:50:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.392+0000 7f986aa00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9848066690 0x7f9848068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.392+0000 7f986aa00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9848066690 0x7f9848068b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f985c0011f0 tx=0x7f985c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:22.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.396+0000 7f9862a00640 1 -- 172.21.15.66:0/1377747267 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f98540eb620 con 0x7f986c0cdef0 2024-03-30T00:50:22.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.552+0000 7f986be00640 1 -- 172.21.15.66:0/1377747267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f986c073e00 con 0x7f986c0cdef0 2024-03-30T00:50:22.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.552+0000 7f9862a00640 1 -- 172.21.15.66:0/1377747267 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98540eec40 con 0x7f986c0cdef0 2024-03-30T00:50:22.562 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:50:22.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 -- 172.21.15.66:0/1377747267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9848066690 msgr2=0x7f9848068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9848066690 0x7f9848068b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f985c0011f0 tx=0x7f985c009040 comp rx=0 tx=0).stop 2024-03-30T00:50:22.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 -- 172.21.15.66:0/1377747267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f986c0cdef0 msgr2=0x7f986c1587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f986c0cdef0 0x7f986c1587a0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f98540190c0 tx=0x7f9854019250 comp rx=0 tx=0).stop 2024-03-30T00:50:22.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 -- 172.21.15.66:0/1377747267 shutdown_connections 2024-03-30T00:50:22.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f986c15c6b0 0x7f986c1538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9848066690 0x7f9848068b50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f986c0cdef0 0x7f986c1587a0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 --2- 172.21.15.66:0/1377747267 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f986c06cb90 0x7f986c15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 -- 172.21.15.66:0/1377747267 >> 172.21.15.66:0/1377747267 conn(0x7f986c069d90 msgr2=0x7f986c1540f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:22.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 -- 172.21.15.66:0/1377747267 shutdown_connections 2024-03-30T00:50:22.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.556+0000 7f986be00640 1 -- 172.21.15.66:0/1377747267 wait complete. 2024-03-30T00:50:22.582 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:50:22.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a3e00640 1 Processor -- start 2024-03-30T00:50:22.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a3e00640 1 -- start start 2024-03-30T00:50:22.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff49c15c980 0x7ff49c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a3e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff49c0b1660 0x7ff49c06edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a3e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff49c0ae630 0x7ff49c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a3e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49c0cbc90 con 0x7ff49c15c980 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a3e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49c0cf220 con 0x7ff49c0b1660 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a3e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0cb650 con 0x7ff49c0ae630 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a1400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff49c15c980 0x7ff49c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a1400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff49c15c980 0x7ff49c06e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35530/0 (socket says 172.21.15.66:35530) 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a1400640 1 -- 172.21.15.66:0/2757458979 learned_addr learned my addr 172.21.15.66:0/2757458979 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a0a00640 1 --2- 172.21.15.66:0/2757458979 >> v2:172.21.15.156:3300/0 conn(0x7ff49c0b1660 0x7ff49c06edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a1e00640 1 --2- 172.21.15.66:0/2757458979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff49c0ae630 0x7ff49c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a1400640 1 -- 172.21.15.66:0/2757458979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff49c0ae630 msgr2=0x7ff49c06d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a1400640 1 --2- 172.21.15.66:0/2757458979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff49c0ae630 0x7ff49c06d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a1400640 1 -- 172.21.15.66:0/2757458979 >> v2:172.21.15.156:3300/0 conn(0x7ff49c0b1660 msgr2=0x7ff49c06edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:22.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a1400640 1 --2- 172.21.15.66:0/2757458979 >> v2:172.21.15.156:3300/0 conn(0x7ff49c0b1660 0x7ff49c06edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a1400640 1 -- 172.21.15.66:0/2757458979 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c0cb930 con 0x7ff49c15c980 2024-03-30T00:50:22.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a0a00640 1 --2- 172.21.15.66:0/2757458979 >> v2:172.21.15.156:3300/0 conn(0x7ff49c0b1660 0x7ff49c06edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:22.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.716+0000 7ff4a1e00640 1 --2- 172.21.15.66:0/2757458979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff49c0ae630 0x7ff49c06d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:22.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff4a1400640 1 --2- 172.21.15.66:0/2757458979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff49c15c980 0x7ff49c06e890 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff49c06f970 tx=0x7ff49001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0c00a3cbcd8ec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:22.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff49be00640 1 -- 172.21.15.66:0/2757458979 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4900ab600 con 0x7ff49c15c980 2024-03-30T00:50:22.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff49be00640 1 -- 172.21.15.66:0/2757458979 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4900a7050 con 0x7ff49c15c980 2024-03-30T00:50:22.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff49be00640 1 -- 172.21.15.66:0/2757458979 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4900ab980 con 0x7ff49c15c980 2024-03-30T00:50:22.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff4a3e00640 1 -- 172.21.15.66:0/2757458979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff49c15c980 msgr2=0x7ff49c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff4a3e00640 1 --2- 172.21.15.66:0/2757458979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff49c15c980 0x7ff49c06e890 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff49c06f970 tx=0x7ff49001c260 comp rx=0 tx=0).stop 2024-03-30T00:50:22.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff4a3e00640 1 -- 172.21.15.66:0/2757458979 shutdown_connections 2024-03-30T00:50:22.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff4a3e00640 1 --2- 172.21.15.66:0/2757458979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff49c0ae630 0x7ff49c06d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff4a3e00640 1 --2- 172.21.15.66:0/2757458979 >> v2:172.21.15.156:3300/0 conn(0x7ff49c0b1660 0x7ff49c06edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff4a3e00640 1 --2- 172.21.15.66:0/2757458979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff49c15c980 0x7ff49c06e890 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff4a3e00640 1 -- 172.21.15.66:0/2757458979 >> 172.21.15.66:0/2757458979 conn(0x7ff49c098a20 msgr2=0x7ff49c0b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:22.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff4a3e00640 1 -- 172.21.15.66:0/2757458979 shutdown_connections 2024-03-30T00:50:22.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.720+0000 7ff4a3e00640 1 -- 172.21.15.66:0/2757458979 wait complete. 2024-03-30T00:50:22.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a3e00640 1 Processor -- start 2024-03-30T00:50:22.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a3e00640 1 -- start start 2024-03-30T00:50:22.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff49c0ae630 0x7ff49c0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a3e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff49c0b1660 0x7ff49c1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a3e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff49c15c980 0x7ff49c149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0c6f00 con 0x7ff49c0ae630 2024-03-30T00:50:22.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a3e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0c5c40 con 0x7ff49c0b1660 2024-03-30T00:50:22.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a3e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0c7150 con 0x7ff49c15c980 2024-03-30T00:50:22.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff49c0ae630 0x7ff49c0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff49c0ae630 0x7ff49c0c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35534/0 (socket says 172.21.15.66:35534) 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a1400640 1 -- 172.21.15.66:0/3770276318 learned_addr learned my addr 172.21.15.66:0/3770276318 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a1e00640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff49c15c980 0x7ff49c149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a0a00640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff49c0b1660 0x7ff49c1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a1400640 1 -- 172.21.15.66:0/3770276318 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff49c15c980 msgr2=0x7ff49c149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a1400640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff49c15c980 0x7ff49c149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a1400640 1 -- 172.21.15.66:0/3770276318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff49c0b1660 msgr2=0x7ff49c1494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a1400640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff49c0b1660 0x7ff49c1494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a1400640 1 -- 172.21.15.66:0/3770276318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900094d0 con 0x7ff49c0ae630 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a0a00640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff49c0b1660 0x7ff49c1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a1400640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff49c0ae630 0x7ff49c0c81b0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff4900ae040 tx=0x7ff49001aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff499600640 1 -- 172.21.15.66:0/3770276318 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4900b6620 con 0x7ff49c0ae630 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff499600640 1 -- 172.21.15.66:0/3770276318 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4900b67a0 con 0x7ff49c0ae630 2024-03-30T00:50:22.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff499600640 1 -- 172.21.15.66:0/3770276318 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4900b6b50 con 0x7ff49c0ae630 2024-03-30T00:50:22.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a3e00640 1 -- 172.21.15.66:0/3770276318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c0c7430 con 0x7ff49c0ae630 2024-03-30T00:50:22.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a3e00640 1 -- 172.21.15.66:0/3770276318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c0c58d0 con 0x7ff49c0ae630 2024-03-30T00:50:22.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.724+0000 7ff4a3e00640 1 -- 172.21.15.66:0/3770276318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c066c30 con 0x7ff49c0ae630 2024-03-30T00:50:22.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.728+0000 7ff499600640 1 -- 172.21.15.66:0/3770276318 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff4900b2040 con 0x7ff49c0ae630 2024-03-30T00:50:22.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.728+0000 7ff499600640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff47c066690 0x7ff47c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:22.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.728+0000 7ff499600640 1 -- 172.21.15.66:0/3770276318 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7ff49011e830 con 0x7ff49c0ae630 2024-03-30T00:50:22.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.728+0000 7ff4a0a00640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff47c066690 0x7ff47c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:22.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.728+0000 7ff4a0a00640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff47c066690 0x7ff47c068b50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff488006fd0 tx=0x7ff488008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:22.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.732+0000 7ff499600640 1 -- 172.21.15.66:0/3770276318 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff4900ebaa0 con 0x7ff49c0ae630 2024-03-30T00:50:22.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 -- 172.21.15.66:0/3770276318 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff49c0cb390 con 0x7ff49c0ae630 2024-03-30T00:50:22.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff499600640 1 -- 172.21.15.66:0/3770276318 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4733 (secure 0 0 0) 0x7ff4900ef0c0 con 0x7ff49c0ae630 2024-03-30T00:50:22.935 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:22.935 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3688845855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":3688845855},{"type":"v1","addr":"172.21.15.66:6837","nonce":3688845855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":86,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:50:20.318282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:50:22.936 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 87 2024-03-30T00:50:22.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 -- 172.21.15.66:0/3770276318 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff47c066690 msgr2=0x7ff47c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff47c066690 0x7ff47c068b50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff488006fd0 tx=0x7ff488008040 comp rx=0 tx=0).stop 2024-03-30T00:50:22.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 -- 172.21.15.66:0/3770276318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff49c0ae630 msgr2=0x7ff49c0c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:22.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff49c0ae630 0x7ff49c0c81b0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff4900ae040 tx=0x7ff49001aaa0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 -- 172.21.15.66:0/3770276318 shutdown_connections 2024-03-30T00:50:22.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff49c15c980 0x7ff49c149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff49c0b1660 0x7ff49c1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff47c066690 0x7ff47c068b50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 --2- 172.21.15.66:0/3770276318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff49c0ae630 0x7ff49c0c81b0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:22.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 -- 172.21.15.66:0/3770276318 >> 172.21.15.66:0/3770276318 conn(0x7ff49c098a20 msgr2=0x7ff49c0ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:22.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 -- 172.21.15.66:0/3770276318 shutdown_connections 2024-03-30T00:50:22.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:22.928+0000 7ff4a3e00640 1 -- 172.21.15.66:0/3770276318 wait complete. 2024-03-30T00:50:22.957 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:50:23.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf3e00640 1 Processor -- start 2024-03-30T00:50:23.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf3e00640 1 -- start start 2024-03-30T00:50:23.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbf415c9f0 0x7fdbf406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf3e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf40ce070 con 0x7fdbf40b2b60 2024-03-30T00:50:23.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf3e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf40ce350 con 0x7fdbf406ba80 2024-03-30T00:50:23.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf3e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf406cf60 con 0x7fdbf415c9f0 2024-03-30T00:50:23.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf2000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdbf40b2b60 0x7fdbf406b370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55312/0 (socket says 172.21.15.66:55312) 2024-03-30T00:50:23.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf2000640 1 -- 172.21.15.66:0/3506212459 learned_addr learned my addr 172.21.15.66:0/3506212459 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:23.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf2a00640 1 --2- 172.21.15.66:0/3506212459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbf415c9f0 0x7fdbf406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf1600640 1 -- 172.21.15.66:0/3506212459 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533032053 0 0) 0x7fdbf40ce070 con 0x7fdbf40b2b60 2024-03-30T00:50:23.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf1600640 1 -- 172.21.15.66:0/3506212459 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd8003530 con 0x7fdbf40b2b60 2024-03-30T00:50:23.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf1600640 1 -- 172.21.15.66:0/3506212459 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524978620 0 0) 0x7fdbf40ce350 con 0x7fdbf406ba80 2024-03-30T00:50:23.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf2a00640 1 -- 172.21.15.66:0/3506212459 >> v1:172.21.15.156:6789/0 conn(0x7fdbf406ba80 legacy=0x7fdbf40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf2a00640 1 -- 172.21.15.66:0/3506212459 >> v1:172.21.15.66:6789/0 conn(0x7fdbf40b2b60 legacy=0x7fdbf406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.092+0000 7fdbf2a00640 1 -- 172.21.15.66:0/3506212459 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf40ce630 con 0x7fdbf415c9f0 2024-03-30T00:50:23.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf2a00640 1 --2- 172.21.15.66:0/3506212459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbf415c9f0 0x7fdbf406ca20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009820 tx=0x7fdbe401c310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f87a0a9846b6ea98 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:23.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf1600640 1 -- 172.21.15.66:0/3506212459 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0001680 con 0x7fdbf415c9f0 2024-03-30T00:50:23.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf1600640 1 -- 172.21.15.66:0/3506212459 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe40a7050 con 0x7fdbf415c9f0 2024-03-30T00:50:23.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf1600640 1 -- 172.21.15.66:0/3506212459 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40ab620 con 0x7fdbf415c9f0 2024-03-30T00:50:23.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf3e00640 1 -- 172.21.15.66:0/3506212459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbf415c9f0 msgr2=0x7fdbf406ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf3e00640 1 --2- 172.21.15.66:0/3506212459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbf415c9f0 0x7fdbf406ca20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009820 tx=0x7fdbe401c310 comp rx=0 tx=0).stop 2024-03-30T00:50:23.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf3e00640 1 -- 172.21.15.66:0/3506212459 shutdown_connections 2024-03-30T00:50:23.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf3e00640 1 --2- 172.21.15.66:0/3506212459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbf415c9f0 0x7fdbf406ca20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf3e00640 1 -- 172.21.15.66:0/3506212459 >> 172.21.15.66:0/3506212459 conn(0x7fdbf406e0d0 msgr2=0x7fdbf406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:23.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf3e00640 1 -- 172.21.15.66:0/3506212459 shutdown_connections 2024-03-30T00:50:23.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf3e00640 1 -- 172.21.15.66:0/3506212459 wait complete. 2024-03-30T00:50:23.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf3e00640 1 Processor -- start 2024-03-30T00:50:23.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.096+0000 7fdbf3e00640 1 -- start start 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf406ba80 0x7fdbf415dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf40b2b60 0x7fdbf40b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbf415c9f0 0x7fdbf40c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40c7db0 con 0x7fdbf40b2b60 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf3e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40ca1a0 con 0x7fdbf406ba80 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf3e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40c76f0 con 0x7fdbf415c9f0 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf40b2b60 0x7fdbf40b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf40b2b60 0x7fdbf40b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35542/0 (socket says 172.21.15.66:35542) 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf2a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf406ba80 0x7fdbf415dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf2000640 1 -- 172.21.15.66:0/80064305 learned_addr learned my addr 172.21.15.66:0/80064305 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf3400640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbf415c9f0 0x7fdbf40c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf2000640 1 -- 172.21.15.66:0/80064305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbf415c9f0 msgr2=0x7fdbf40c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf2000640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbf415c9f0 0x7fdbf40c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf2000640 1 -- 172.21.15.66:0/80064305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf406ba80 msgr2=0x7fdbf415dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf2000640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf406ba80 0x7fdbf415dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf2000640 1 -- 172.21.15.66:0/80064305 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4009420 con 0x7fdbf40b2b60 2024-03-30T00:50:23.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf2000640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf40b2b60 0x7fdbf40b47f0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00011f0 tx=0x7fdbe000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:23.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbeaa00640 1 -- 172.21.15.66:0/80064305 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00074e0 con 0x7fdbf40b2b60 2024-03-30T00:50:23.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbeaa00640 1 -- 172.21.15.66:0/80064305 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe0007640 con 0x7fdbf40b2b60 2024-03-30T00:50:23.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf3e00640 1 -- 172.21.15.66:0/80064305 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf40c79d0 con 0x7fdbf40b2b60 2024-03-30T00:50:23.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbeaa00640 1 -- 172.21.15.66:0/80064305 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0004a30 con 0x7fdbf40b2b60 2024-03-30T00:50:23.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf3e00640 1 -- 172.21.15.66:0/80064305 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf415e470 con 0x7fdbf40b2b60 2024-03-30T00:50:23.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.100+0000 7fdbf3e00640 1 -- 172.21.15.66:0/80064305 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf40c7ef0 con 0x7fdbf40b2b60 2024-03-30T00:50:23.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.104+0000 7fdbeaa00640 1 -- 172.21.15.66:0/80064305 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdbe0004b90 con 0x7fdbf40b2b60 2024-03-30T00:50:23.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.104+0000 7fdbeaa00640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbd4066690 0x7fdbd4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.104+0000 7fdbeaa00640 1 -- 172.21.15.66:0/80064305 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fdbe0083980 con 0x7fdbf40b2b60 2024-03-30T00:50:23.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.104+0000 7fdbf2a00640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbd4066690 0x7fdbd4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.104+0000 7fdbf2a00640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbd4066690 0x7fdbd4068b50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40ad000 tx=0x7fdbe40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:23.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.108+0000 7fdbeaa00640 1 -- 172.21.15.66:0/80064305 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdbf40c7ef0 con 0x7fdbf40b2b60 2024-03-30T00:50:23.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 -- 172.21.15.66:0/80064305 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbf40c7ef0 con 0x7fdbf40b2b60 2024-03-30T00:50:23.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbeaa00640 1 -- 172.21.15.66:0/80064305 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16555 (secure 0 0 0) 0x7fdbf40c7ef0 con 0x7fdbf40b2b60 2024-03-30T00:50:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:23.279 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":114,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:20.317752+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:48:44.750350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-30T00:48:45.327928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.35","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6837/282507324":"2024-03-31T00:50:20.317728+0000","172.21.15.66:6835/282507324":"2024-03-31T00:50:20.317728+0000","172.21.15.66:6837/2988412199":"2024-03-31T00:49:43.612805+0000","172.21.15.66:6835/2988412199":"2024-03-31T00:49:43.612805+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:50:23.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 -- 172.21.15.66:0/80064305 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbd4066690 msgr2=0x7fdbd4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbd4066690 0x7fdbd4068b50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40ad000 tx=0x7fdbe40a6000 comp rx=0 tx=0).stop 2024-03-30T00:50:23.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 -- 172.21.15.66:0/80064305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf40b2b60 msgr2=0x7fdbf40b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf40b2b60 0x7fdbf40b47f0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00011f0 tx=0x7fdbe000cc70 comp rx=0 tx=0).stop 2024-03-30T00:50:23.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 -- 172.21.15.66:0/80064305 shutdown_connections 2024-03-30T00:50:23.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbf415c9f0 0x7fdbf40c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdbd4066690 0x7fdbd4068b50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf40b2b60 0x7fdbf40b47f0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 --2- 172.21.15.66:0/80064305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbf406ba80 0x7fdbf415dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 -- 172.21.15.66:0/80064305 >> 172.21.15.66:0/80064305 conn(0x7fdbf406e0d0 msgr2=0x7fdbf4154aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:23.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 -- 172.21.15.66:0/80064305 shutdown_connections 2024-03-30T00:50:23.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.272+0000 7fdbf3e00640 1 -- 172.21.15.66:0/80064305 wait complete. 2024-03-30T00:50:23.300 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T00:50:23.300 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:50:23.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.432+0000 7f8f8b400640 1 Processor -- start 2024-03-30T00:50:23.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.432+0000 7f8f8b400640 1 -- start start 2024-03-30T00:50:23.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.432+0000 7f8f8b400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8f8415ca80 0x7f8f8406b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.432+0000 7f8f8b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f840cf2f0 0x7f8f8406b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.432+0000 7f8f8b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8f8406bb30 0x7f8f84098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.432+0000 7f8f8b400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f840cb0e0 con 0x7f8f8406bb30 2024-03-30T00:50:23.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.432+0000 7f8f8b400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f840cf7f0 con 0x7f8f8415ca80 2024-03-30T00:50:23.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.432+0000 7f8f8b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840cfad0 con 0x7f8f840cf2f0 2024-03-30T00:50:23.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f89400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8f8406bb30 0x7f8f84098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f89400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8f8406bb30 0x7f8f84098a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35556/0 (socket says 172.21.15.66:35556) 2024-03-30T00:50:23.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f89400640 1 -- 172.21.15.66:0/2240834667 learned_addr learned my addr 172.21.15.66:0/2240834667 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:23.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f83e00640 1 --2- 172.21.15.66:0/2240834667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f840cf2f0 0x7f8f8406b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f88a00640 1 --2- 172.21.15.66:0/2240834667 >> v2:172.21.15.156:3300/0 conn(0x7f8f8415ca80 0x7f8f8406b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f89400640 1 -- 172.21.15.66:0/2240834667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f840cf2f0 msgr2=0x7f8f8406b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:23.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f89400640 1 --2- 172.21.15.66:0/2240834667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f840cf2f0 0x7f8f8406b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f89400640 1 -- 172.21.15.66:0/2240834667 >> v2:172.21.15.156:3300/0 conn(0x7f8f8415ca80 msgr2=0x7f8f8406b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:23.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f89400640 1 --2- 172.21.15.66:0/2240834667 >> v2:172.21.15.156:3300/0 conn(0x7f8f8415ca80 0x7f8f8406b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f89400640 1 -- 172.21.15.66:0/2240834667 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f840cad70 con 0x7f8f8406bb30 2024-03-30T00:50:23.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f89400640 1 --2- 172.21.15.66:0/2240834667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f8406bb30 0x7f8f84098a20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f8f84070ed0 tx=0x7f8f7c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672261b8e7f39c83 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:23.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f83400640 1 -- 172.21.15.66:0/2240834667 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0af400 con 0x7f8f8406bb30 2024-03-30T00:50:23.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f83400640 1 -- 172.21.15.66:0/2240834667 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c0ab050 con 0x7f8f8406bb30 2024-03-30T00:50:23.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.436+0000 7f8f83400640 1 -- 172.21.15.66:0/2240834667 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0af780 con 0x7f8f8406bb30 2024-03-30T00:50:23.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 -- 172.21.15.66:0/2240834667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f8406bb30 msgr2=0x7f8f84098a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 --2- 172.21.15.66:0/2240834667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f8406bb30 0x7f8f84098a20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f8f84070ed0 tx=0x7f8f7c01f1e0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 -- 172.21.15.66:0/2240834667 shutdown_connections 2024-03-30T00:50:23.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 --2- 172.21.15.66:0/2240834667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f8406bb30 0x7f8f84098a20 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 --2- 172.21.15.66:0/2240834667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f840cf2f0 0x7f8f8406b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 --2- 172.21.15.66:0/2240834667 >> v2:172.21.15.156:3300/0 conn(0x7f8f8415ca80 0x7f8f8406b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 -- 172.21.15.66:0/2240834667 >> 172.21.15.66:0/2240834667 conn(0x7f8f840b2090 msgr2=0x7f8f840b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:23.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 -- 172.21.15.66:0/2240834667 shutdown_connections 2024-03-30T00:50:23.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 -- 172.21.15.66:0/2240834667 wait complete. 2024-03-30T00:50:23.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 Processor -- start 2024-03-30T00:50:23.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 -- start start 2024-03-30T00:50:23.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f8406bb30 0x7f8f8415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f840cf2f0 0x7f8f841f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8415ca80 0x7f8f8415b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840b3fe0 con 0x7f8f840cf2f0 2024-03-30T00:50:23.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840b5df0 con 0x7f8f8415ca80 2024-03-30T00:50:23.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f8b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840b5830 con 0x7f8f8406bb30 2024-03-30T00:50:23.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f840cf2f0 0x7f8f841f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f840cf2f0 0x7f8f841f22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35572/0 (socket says 172.21.15.66:35572) 2024-03-30T00:50:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f83e00640 1 -- 172.21.15.66:0/3220135452 learned_addr learned my addr 172.21.15.66:0/3220135452 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f88a00640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f8406bb30 0x7f8f8415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f89400640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8415ca80 0x7f8f8415b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f83e00640 1 -- 172.21.15.66:0/3220135452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f8406bb30 msgr2=0x7f8f8415ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f83e00640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f8406bb30 0x7f8f8415ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f83e00640 1 -- 172.21.15.66:0/3220135452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8415ca80 msgr2=0x7f8f8415b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f83e00640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8415ca80 0x7f8f8415b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.440+0000 7f8f83e00640 1 -- 172.21.15.66:0/3220135452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c009b30 con 0x7f8f840cf2f0 2024-03-30T00:50:23.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.444+0000 7f8f83e00640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f840cf2f0 0x7f8f841f22b0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000e8b0 tx=0x7f8f7000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:23.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.444+0000 7f8f80c00640 1 -- 172.21.15.66:0/3220135452 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7000c940 con 0x7f8f840cf2f0 2024-03-30T00:50:23.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.444+0000 7f8f80c00640 1 -- 172.21.15.66:0/3220135452 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7000caa0 con 0x7f8f840cf2f0 2024-03-30T00:50:23.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.444+0000 7f8f80c00640 1 -- 172.21.15.66:0/3220135452 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7000ce30 con 0x7f8f840cf2f0 2024-03-30T00:50:23.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.444+0000 7f8f8b400640 1 -- 172.21.15.66:0/3220135452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f840b5b10 con 0x7f8f840cf2f0 2024-03-30T00:50:23.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.444+0000 7f8f8b400640 1 -- 172.21.15.66:0/3220135452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8415b2c0 con 0x7f8f840cf2f0 2024-03-30T00:50:23.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.444+0000 7f8f8b400640 1 -- 172.21.15.66:0/3220135452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f840b5f60 con 0x7f8f840cf2f0 2024-03-30T00:50:23.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.444+0000 7f8f80c00640 1 -- 172.21.15.66:0/3220135452 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8f70013470 con 0x7f8f840cf2f0 2024-03-30T00:50:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.444+0000 7f8f80c00640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f640666c0 0x7f8f64068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.444+0000 7f8f80c00640 1 -- 172.21.15.66:0/3220135452 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f8f70084b40 con 0x7f8f840cf2f0 2024-03-30T00:50:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.444+0000 7f8f88a00640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f640666c0 0x7f8f64068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.448+0000 7f8f88a00640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f640666c0 0x7f8f64068b80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8f8406c650 tx=0x7f8f78009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:23.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.448+0000 7f8f80c00640 1 -- 172.21.15.66:0/3220135452 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8f70051ec0 con 0x7f8f840cf2f0 2024-03-30T00:50:23.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.640+0000 7f8f8b400640 1 -- 172.21.15.66:0/3220135452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f84098ca0 con 0x7f8f840cf2f0 2024-03-30T00:50:23.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f80c00640 1 -- 172.21.15.66:0/3220135452 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4733 (secure 0 0 0) 0x7f8f70021020 con 0x7f8f840cf2f0 2024-03-30T00:50:23.650 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:23.650 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3688845855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":3688845855},{"type":"v1","addr":"172.21.15.66:6837","nonce":3688845855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":86,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:50:20.318282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:50:23.651 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 87 2024-03-30T00:50:23.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 -- 172.21.15.66:0/3220135452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f640666c0 msgr2=0x7f8f64068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f640666c0 0x7f8f64068b80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8f8406c650 tx=0x7f8f78009190 comp rx=0 tx=0).stop 2024-03-30T00:50:23.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 -- 172.21.15.66:0/3220135452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f840cf2f0 msgr2=0x7f8f841f22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f840cf2f0 0x7f8f841f22b0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000e8b0 tx=0x7f8f7000ed70 comp rx=0 tx=0).stop 2024-03-30T00:50:23.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 -- 172.21.15.66:0/3220135452 shutdown_connections 2024-03-30T00:50:23.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8415ca80 0x7f8f8415b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8f640666c0 0x7f8f64068b80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f840cf2f0 0x7f8f841f22b0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 --2- 172.21.15.66:0/3220135452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f8406bb30 0x7f8f8415ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 -- 172.21.15.66:0/3220135452 >> 172.21.15.66:0/3220135452 conn(0x7f8f840b2090 msgr2=0x7f8f840c40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:23.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 -- 172.21.15.66:0/3220135452 shutdown_connections 2024-03-30T00:50:23.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.644+0000 7f8f8b400640 1 -- 172.21.15.66:0/3220135452 wait complete. 2024-03-30T00:50:23.671 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:50:23.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe837c00640 1 Processor -- start 2024-03-30T00:50:23.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe837c00640 1 -- start start 2024-03-30T00:50:23.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe837c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe8300cda90 0x7fe8300ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe837c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300cde70 0x7fe8300cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe837c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83015a990 con 0x7fe830159c60 2024-03-30T00:50:23.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe837c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8300b1b80 con 0x7fe8300cda90 2024-03-30T00:50:23.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe837c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300cd3e0 con 0x7fe8300cde70 2024-03-30T00:50:23.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe835200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe8300cda90 0x7fe8300ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe835200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe8300cda90 0x7fe8300ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35588/0 (socket says 172.21.15.66:35588) 2024-03-30T00:50:23.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe835200640 1 -- 172.21.15.66:0/3681929407 learned_addr learned my addr 172.21.15.66:0/3681929407 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:23.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe82fe00640 1 --2- 172.21.15.66:0/3681929407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300cde70 0x7fe8300cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe835200640 1 -- 172.21.15.66:0/3681929407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300cde70 msgr2=0x7fe8300cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:23.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe835200640 1 --2- 172.21.15.66:0/3681929407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300cde70 0x7fe8300cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe835200640 1 -- 172.21.15.66:0/3681929407 >> v1:172.21.15.156:6789/0 conn(0x7fe830159c60 legacy=0x7fe83015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe835200640 1 -- 172.21.15.66:0/3681929407 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8300b11e0 con 0x7fe8300cda90 2024-03-30T00:50:23.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.804+0000 7fe82fe00640 1 --2- 172.21.15.66:0/3681929407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300cde70 0x7fe8300cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:23.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe835200640 1 --2- 172.21.15.66:0/3681929407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8300cda90 0x7fe8300ce520 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe824009870 tx=0x7fe82401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8454afab969ee5b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:23.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe82f400640 1 -- 172.21.15.66:0/3681929407 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8240ac640 con 0x7fe8300cda90 2024-03-30T00:50:23.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe82f400640 1 -- 172.21.15.66:0/3681929407 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8240ac7a0 con 0x7fe8300cda90 2024-03-30T00:50:23.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe82f400640 1 -- 172.21.15.66:0/3681929407 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8240acb30 con 0x7fe8300cda90 2024-03-30T00:50:23.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe837c00640 1 -- 172.21.15.66:0/3681929407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8300cda90 msgr2=0x7fe8300ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe837c00640 1 --2- 172.21.15.66:0/3681929407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8300cda90 0x7fe8300ce520 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe824009870 tx=0x7fe82401c4d0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe837c00640 1 -- 172.21.15.66:0/3681929407 shutdown_connections 2024-03-30T00:50:23.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe837c00640 1 --2- 172.21.15.66:0/3681929407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300cde70 0x7fe8300cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe837c00640 1 --2- 172.21.15.66:0/3681929407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8300cda90 0x7fe8300ce520 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe837c00640 1 -- 172.21.15.66:0/3681929407 >> 172.21.15.66:0/3681929407 conn(0x7fe830069f60 msgr2=0x7fe83006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:23.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe837c00640 1 -- 172.21.15.66:0/3681929407 shutdown_connections 2024-03-30T00:50:23.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.808+0000 7fe837c00640 1 -- 172.21.15.66:0/3681929407 wait complete. 2024-03-30T00:50:23.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe837c00640 1 Processor -- start 2024-03-30T00:50:23.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe837c00640 1 -- start start 2024-03-30T00:50:23.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe837c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8300cda90 0x7fe830071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe837c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300cde70 0x7fe8300ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe837c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe830159c60 0x7fe8300ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe837c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300acfd0 con 0x7fe8300cda90 2024-03-30T00:50:23.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe837c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300ad2b0 con 0x7fe830159c60 2024-03-30T00:50:23.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe837c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe830157660 con 0x7fe8300cde70 2024-03-30T00:50:23.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe835200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8300cda90 0x7fe830071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe835200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8300cda90 0x7fe830071180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35596/0 (socket says 172.21.15.66:35596) 2024-03-30T00:50:23.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe835200640 1 -- 172.21.15.66:0/1685013972 learned_addr learned my addr 172.21.15.66:0/1685013972 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:23.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe835c00640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe830159c60 0x7fe8300ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe82fe00640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300cde70 0x7fe8300ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe835200640 1 -- 172.21.15.66:0/1685013972 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300cde70 msgr2=0x7fe8300ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe835200640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300cde70 0x7fe8300ac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe835200640 1 -- 172.21.15.66:0/1685013972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe830159c60 msgr2=0x7fe8300ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:23.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe835200640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe830159c60 0x7fe8300ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:23.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe835200640 1 -- 172.21.15.66:0/1685013972 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8240094d0 con 0x7fe8300cda90 2024-03-30T00:50:23.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe835200640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8300cda90 0x7fe830071180 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fe8240af040 tx=0x7fe824019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:23.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe82cc00640 1 -- 172.21.15.66:0/1685013972 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8240ba070 con 0x7fe8300cda90 2024-03-30T00:50:23.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe82cc00640 1 -- 172.21.15.66:0/1685013972 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8240b5440 con 0x7fe8300cda90 2024-03-30T00:50:23.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe82cc00640 1 -- 172.21.15.66:0/1685013972 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8240b57f0 con 0x7fe8300cda90 2024-03-30T00:50:23.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe837c00640 1 -- 172.21.15.66:0/1685013972 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe830157940 con 0x7fe8300cda90 2024-03-30T00:50:23.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.812+0000 7fe837c00640 1 -- 172.21.15.66:0/1685013972 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8300c4ad0 con 0x7fe8300cda90 2024-03-30T00:50:23.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.816+0000 7fe837c00640 1 -- 172.21.15.66:0/1685013972 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8300ad110 con 0x7fe8300cda90 2024-03-30T00:50:23.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.816+0000 7fe82cc00640 1 -- 172.21.15.66:0/1685013972 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe8240b3070 con 0x7fe8300cda90 2024-03-30T00:50:23.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.816+0000 7fe82cc00640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe810066690 0x7fe810068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:23.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.816+0000 7fe82cc00640 1 -- 172.21.15.66:0/1685013972 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fe82411de70 con 0x7fe8300cda90 2024-03-30T00:50:23.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.816+0000 7fe82fe00640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe810066690 0x7fe810068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:23.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.816+0000 7fe82fe00640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe810066690 0x7fe810068b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe8300c9670 tx=0x7fe81c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:23.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:23.820+0000 7fe82cc00640 1 -- 172.21.15.66:0/1685013972 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe8300ad110 con 0x7fe8300cda90 2024-03-30T00:50:24.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.012+0000 7fe837c00640 1 -- 172.21.15.66:0/1685013972 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8300ad110 con 0x7fe8300cda90 2024-03-30T00:50:24.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe82cc00640 1 -- 172.21.15.66:0/1685013972 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4733 (secure 0 0 0) 0x7fe8300ad110 con 0x7fe8300cda90 2024-03-30T00:50:24.022 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:24.022 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3688845855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":3688845855},{"type":"v1","addr":"172.21.15.66:6837","nonce":3688845855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":86,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:48:46.332321+0000","modified":"2024-03-30T00:50:20.318282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-30T00:50:24.022 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 87 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 -- 172.21.15.66:0/1685013972 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe810066690 msgr2=0x7fe810068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe810066690 0x7fe810068b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe8300c9670 tx=0x7fe81c008040 comp rx=0 tx=0).stop 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 -- 172.21.15.66:0/1685013972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8300cda90 msgr2=0x7fe830071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8300cda90 0x7fe830071180 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fe8240af040 tx=0x7fe824019270 comp rx=0 tx=0).stop 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 -- 172.21.15.66:0/1685013972 shutdown_connections 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe830159c60 0x7fe8300ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300cde70 0x7fe8300ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe810066690 0x7fe810068b50 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 --2- 172.21.15.66:0/1685013972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8300cda90 0x7fe830071180 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 -- 172.21.15.66:0/1685013972 >> 172.21.15.66:0/1685013972 conn(0x7fe830069f60 msgr2=0x7fe83015a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 -- 172.21.15.66:0/1685013972 shutdown_connections 2024-03-30T00:50:24.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.016+0000 7fe837c00640 1 -- 172.21.15.66:0/1685013972 wait complete. 2024-03-30T00:50:24.043 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:50:24.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dfe00640 1 Processor -- start 2024-03-30T00:50:24.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dfe00640 1 -- start start 2024-03-30T00:50:24.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dfe00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff3d80cee50 0x7ff3d80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dfe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3d80af420 0x7ff3d80d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dfe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d80cbb90 con 0x7ff3d815de30 2024-03-30T00:50:24.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dfe00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d80ce1b0 con 0x7ff3d80cee50 2024-03-30T00:50:24.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dfe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80afa10 con 0x7ff3d80af420 2024-03-30T00:50:24.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dd400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff3d815de30 0x7ff3d80ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55318/0 (socket says 172.21.15.66:55318) 2024-03-30T00:50:24.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dd400640 1 -- 172.21.15.66:0/625457468 learned_addr learned my addr 172.21.15.66:0/625457468 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:24.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dca00640 1 --2- 172.21.15.66:0/625457468 >> v2:172.21.15.156:3300/0 conn(0x7ff3d80cee50 0x7ff3d80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dde00640 1 --2- 172.21.15.66:0/625457468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3d80af420 0x7ff3d80d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3d7e00640 1 -- 172.21.15.66:0/625457468 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200009624 0 0) 0x7ff3d80cbb90 con 0x7ff3d815de30 2024-03-30T00:50:24.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3d7e00640 1 -- 172.21.15.66:0/625457468 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c0003530 con 0x7ff3d815de30 2024-03-30T00:50:24.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dca00640 1 -- 172.21.15.66:0/625457468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3d80af420 msgr2=0x7ff3d80d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:24.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dca00640 1 --2- 172.21.15.66:0/625457468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3d80af420 0x7ff3d80d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dca00640 1 -- 172.21.15.66:0/625457468 >> v1:172.21.15.66:6789/0 conn(0x7ff3d815de30 legacy=0x7ff3d80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:24.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.176+0000 7ff3dca00640 1 -- 172.21.15.66:0/625457468 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d80cf430 con 0x7ff3d80cee50 2024-03-30T00:50:24.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3dca00640 1 --2- 172.21.15.66:0/625457468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3d80cee50 0x7ff3d80cc040 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40097c0 tx=0x7ff3c401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc2c3525a3ffc17 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:24.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3d7e00640 1 -- 172.21.15.66:0/625457468 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40ab600 con 0x7ff3d80cee50 2024-03-30T00:50:24.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3d7e00640 1 -- 172.21.15.66:0/625457468 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c40a7050 con 0x7ff3d80cee50 2024-03-30T00:50:24.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3d7e00640 1 -- 172.21.15.66:0/625457468 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40ab960 con 0x7ff3d80cee50 2024-03-30T00:50:24.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3dfe00640 1 -- 172.21.15.66:0/625457468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3d80cee50 msgr2=0x7ff3d80cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:24.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3dfe00640 1 --2- 172.21.15.66:0/625457468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3d80cee50 0x7ff3d80cc040 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40097c0 tx=0x7ff3c401c410 comp rx=0 tx=0).stop 2024-03-30T00:50:24.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3dfe00640 1 -- 172.21.15.66:0/625457468 shutdown_connections 2024-03-30T00:50:24.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3dfe00640 1 --2- 172.21.15.66:0/625457468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3d80af420 0x7ff3d80d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3dfe00640 1 --2- 172.21.15.66:0/625457468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3d80cee50 0x7ff3d80cc040 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3dfe00640 1 -- 172.21.15.66:0/625457468 >> 172.21.15.66:0/625457468 conn(0x7ff3d80d0540 msgr2=0x7ff3d80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:24.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3dfe00640 1 -- 172.21.15.66:0/625457468 shutdown_connections 2024-03-30T00:50:24.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.180+0000 7ff3dfe00640 1 -- 172.21.15.66:0/625457468 wait complete. 2024-03-30T00:50:24.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dfe00640 1 Processor -- start 2024-03-30T00:50:24.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dfe00640 1 -- start start 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dfe00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3d80af420 0x7ff3d8150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dfe00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3d80cee50 0x7ff3d8157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d815de30 0x7ff3d80cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dfe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d81f8be0 con 0x7ff3d815de30 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dfe00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d81f82b0 con 0x7ff3d80cee50 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dfe00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d81f8590 con 0x7ff3d80af420 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d815de30 0x7ff3d80cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d815de30 0x7ff3d80cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35598/0 (socket says 172.21.15.66:35598) 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dde00640 1 -- 172.21.15.66:0/3923413521 learned_addr learned my addr 172.21.15.66:0/3923413521 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dca00640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3d80cee50 0x7ff3d8157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dd400640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3d80af420 0x7ff3d8150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dde00640 1 -- 172.21.15.66:0/3923413521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3d80af420 msgr2=0x7ff3d8150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:24.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dde00640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3d80af420 0x7ff3d8150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dde00640 1 -- 172.21.15.66:0/3923413521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3d80cee50 msgr2=0x7ff3d8157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:24.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dde00640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3d80cee50 0x7ff3d8157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dde00640 1 -- 172.21.15.66:0/3923413521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c4009420 con 0x7ff3d815de30 2024-03-30T00:50:24.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dde00640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d815de30 0x7ff3d80cd310 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000efc0 tx=0x7ff3d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:24.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3d5600640 1 -- 172.21.15.66:0/3923413521 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0014480 con 0x7ff3d815de30 2024-03-30T00:50:24.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3d5600640 1 -- 172.21.15.66:0/3923413521 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d000f040 con 0x7ff3d815de30 2024-03-30T00:50:24.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3d5600640 1 -- 172.21.15.66:0/3923413521 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0014820 con 0x7ff3d815de30 2024-03-30T00:50:24.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dfe00640 1 -- 172.21.15.66:0/3923413521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d81f8870 con 0x7ff3d815de30 2024-03-30T00:50:24.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.184+0000 7ff3dfe00640 1 -- 172.21.15.66:0/3923413521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d8158140 con 0x7ff3d815de30 2024-03-30T00:50:24.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.188+0000 7ff3dfe00640 1 -- 172.21.15.66:0/3923413521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d80cdf60 con 0x7ff3d815de30 2024-03-30T00:50:24.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.188+0000 7ff3d5600640 1 -- 172.21.15.66:0/3923413521 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff3d0014980 con 0x7ff3d815de30 2024-03-30T00:50:24.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.188+0000 7ff3d5600640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff3b8066690 0x7ff3b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.188+0000 7ff3d5600640 1 -- 172.21.15.66:0/3923413521 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7ff3d0084fb0 con 0x7ff3d815de30 2024-03-30T00:50:24.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.188+0000 7ff3dd400640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff3b8066690 0x7ff3b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.188+0000 7ff3dd400640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff3b8066690 0x7ff3b8068b50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc005890 tx=0x7ff3cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:24.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.192+0000 7ff3d5600640 1 -- 172.21.15.66:0/3923413521 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff3d0052080 con 0x7ff3d815de30 2024-03-30T00:50:24.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.356+0000 7ff3dfe00640 1 -- 172.21.15.66:0/3923413521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3d8066c30 con 0x7ff3d815de30 2024-03-30T00:50:24.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.356+0000 7ff3d5600640 1 -- 172.21.15.66:0/3923413521 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16555 (secure 0 0 0) 0x7ff3d0018090 con 0x7ff3d815de30 2024-03-30T00:50:24.364 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:24.365 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":114,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:20.317752+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:48:44.750350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-30T00:48:45.327928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.35","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6837/282507324":"2024-03-31T00:50:20.317728+0000","172.21.15.66:6835/282507324":"2024-03-31T00:50:20.317728+0000","172.21.15.66:6837/2988412199":"2024-03-31T00:49:43.612805+0000","172.21.15.66:6835/2988412199":"2024-03-31T00:49:43.612805+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:50:24.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 -- 172.21.15.66:0/3923413521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff3b8066690 msgr2=0x7ff3b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:24.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff3b8066690 0x7ff3b8068b50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc005890 tx=0x7ff3cc009040 comp rx=0 tx=0).stop 2024-03-30T00:50:24.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 -- 172.21.15.66:0/3923413521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d815de30 msgr2=0x7ff3d80cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:24.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d815de30 0x7ff3d80cd310 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000efc0 tx=0x7ff3d000c490 comp rx=0 tx=0).stop 2024-03-30T00:50:24.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 -- 172.21.15.66:0/3923413521 shutdown_connections 2024-03-30T00:50:24.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff3b8066690 0x7ff3b8068b50 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d815de30 0x7ff3d80cd310 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3d80cee50 0x7ff3d8157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 --2- 172.21.15.66:0/3923413521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff3d80af420 0x7ff3d8150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 -- 172.21.15.66:0/3923413521 >> 172.21.15.66:0/3923413521 conn(0x7ff3d80d0540 msgr2=0x7ff3d80ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:24.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 -- 172.21.15.66:0/3923413521 shutdown_connections 2024-03-30T00:50:24.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.360+0000 7ff3dfe00640 1 -- 172.21.15.66:0/3923413521 wait complete. 2024-03-30T00:50:24.385 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:50:24.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48be00640 1 Processor -- start 2024-03-30T00:50:24.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48be00640 1 -- start start 2024-03-30T00:50:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb48c06cb90 0x7fb48c0d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb48c0cdef0 0x7fb48c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48be00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c0ce880 con 0x7fb48c15c6b0 2024-03-30T00:50:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48be00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c06b1f0 con 0x7fb48c0cdef0 2024-03-30T00:50:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c15d250 con 0x7fb48c06cb90 2024-03-30T00:50:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb48c0cdef0 0x7fb48c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb48c0cdef0 0x7fb48c0ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35602/0 (socket says 172.21.15.66:35602) 2024-03-30T00:50:24.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48b400640 1 -- 172.21.15.66:0/3624910314 learned_addr learned my addr 172.21.15.66:0/3624910314 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb483e00640 1 --2- 172.21.15.66:0/3624910314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb48c06cb90 0x7fb48c0d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48b400640 1 -- 172.21.15.66:0/3624910314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb48c06cb90 msgr2=0x7fb48c0d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48b400640 1 --2- 172.21.15.66:0/3624910314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb48c06cb90 0x7fb48c0d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48b400640 1 -- 172.21.15.66:0/3624910314 >> v1:172.21.15.156:6789/0 conn(0x7fb48c15c6b0 legacy=0x7fb48c15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb48b400640 1 -- 172.21.15.66:0/3624910314 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c15d530 con 0x7fb48c0cdef0 2024-03-30T00:50:24.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.520+0000 7fb483e00640 1 --2- 172.21.15.66:0/3624910314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb48c06cb90 0x7fb48c0d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48b400640 1 --2- 172.21.15.66:0/3624910314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb48c0cdef0 0x7fb48c0ce340 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb484009e50 tx=0x7fb48401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5606e1ece39af459 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48a000640 1 -- 172.21.15.66:0/3624910314 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840a5750 con 0x7fb48c0cdef0 2024-03-30T00:50:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48a000640 1 -- 172.21.15.66:0/3624910314 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4840a58b0 con 0x7fb48c0cdef0 2024-03-30T00:50:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48a000640 1 -- 172.21.15.66:0/3624910314 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840a5c40 con 0x7fb48c0cdef0 2024-03-30T00:50:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48be00640 1 -- 172.21.15.66:0/3624910314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb48c0cdef0 msgr2=0x7fb48c0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48be00640 1 --2- 172.21.15.66:0/3624910314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb48c0cdef0 0x7fb48c0ce340 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb484009e50 tx=0x7fb48401c9c0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48be00640 1 -- 172.21.15.66:0/3624910314 shutdown_connections 2024-03-30T00:50:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48be00640 1 --2- 172.21.15.66:0/3624910314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb48c0cdef0 0x7fb48c0ce340 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48be00640 1 --2- 172.21.15.66:0/3624910314 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb48c06cb90 0x7fb48c0d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48be00640 1 -- 172.21.15.66:0/3624910314 >> 172.21.15.66:0/3624910314 conn(0x7fb48c069d90 msgr2=0x7fb48c06a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48be00640 1 -- 172.21.15.66:0/3624910314 shutdown_connections 2024-03-30T00:50:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.524+0000 7fb48be00640 1 -- 172.21.15.66:0/3624910314 wait complete. 2024-03-30T00:50:24.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48be00640 1 Processor -- start 2024-03-30T00:50:24.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48be00640 1 -- start start 2024-03-30T00:50:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48be00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb48c06cb90 0x7fb48c15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb48c0cdef0 0x7fb48c0ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb48c15c6b0 0x7fb48c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c15a970 con 0x7fb48c0cdef0 2024-03-30T00:50:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48be00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0b5c20 con 0x7fb48c06cb90 2024-03-30T00:50:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0b5f00 con 0x7fb48c15c6b0 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb48c0cdef0 0x7fb48c0ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb48c0cdef0 0x7fb48c0ab910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35604/0 (socket says 172.21.15.66:35604) 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483e00640 1 -- 172.21.15.66:0/2702301448 learned_addr learned my addr 172.21.15.66:0/2702301448 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48aa00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb48c06cb90 0x7fb48c15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48b400640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb48c15c6b0 0x7fb48c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483e00640 1 -- 172.21.15.66:0/2702301448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb48c15c6b0 msgr2=0x7fb48c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483e00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb48c15c6b0 0x7fb48c159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483e00640 1 -- 172.21.15.66:0/2702301448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb48c06cb90 msgr2=0x7fb48c15ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483e00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb48c06cb90 0x7fb48c15ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483e00640 1 -- 172.21.15.66:0/2702301448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb484009b30 con 0x7fb48c0cdef0 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483e00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb48c0cdef0 0x7fb48c0ab910 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c7c0 tx=0x7fb47400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483400640 1 -- 172.21.15.66:0/2702301448 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474007a30 con 0x7fb48c0cdef0 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483400640 1 -- 172.21.15.66:0/2702301448 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb474007b90 con 0x7fb48c0cdef0 2024-03-30T00:50:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb483400640 1 -- 172.21.15.66:0/2702301448 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47400ce70 con 0x7fb48c0cdef0 2024-03-30T00:50:24.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48be00640 1 -- 172.21.15.66:0/2702301448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c15a320 con 0x7fb48c0cdef0 2024-03-30T00:50:24.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48be00640 1 -- 172.21.15.66:0/2702301448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c15a010 con 0x7fb48c0cdef0 2024-03-30T00:50:24.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.528+0000 7fb48be00640 1 -- 172.21.15.66:0/2702301448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c073e00 con 0x7fb48c0cdef0 2024-03-30T00:50:24.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.532+0000 7fb483400640 1 -- 172.21.15.66:0/2702301448 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb4740113f0 con 0x7fb48c0cdef0 2024-03-30T00:50:24.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.532+0000 7fb483400640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4680666c0 0x7fb468068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:24.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.532+0000 7fb483400640 1 -- 172.21.15.66:0/2702301448 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fb474084860 con 0x7fb48c0cdef0 2024-03-30T00:50:24.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.532+0000 7fb48aa00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4680666c0 0x7fb468068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:24.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.532+0000 7fb48aa00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4680666c0 0x7fb468068b80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb48c15f7a0 tx=0x7fb47c009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:24.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.536+0000 7fb483400640 1 -- 172.21.15.66:0/2702301448 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb4740519d0 con 0x7fb48c0cdef0 2024-03-30T00:50:24.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:24.732+0000 7fb48be00640 1 -- 172.21.15.66:0/2702301448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb48c0b2ab0 con 0x7fb48c0cdef0 2024-03-30T00:50:25.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.652+0000 7fb483400640 1 -- 172.21.15.66:0/2702301448 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb474054ff0 con 0x7fb48c0cdef0 2024-03-30T00:50:25.660 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:50:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 -- 172.21.15.66:0/2702301448 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4680666c0 msgr2=0x7fb468068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4680666c0 0x7fb468068b80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb48c15f7a0 tx=0x7fb47c009060 comp rx=0 tx=0).stop 2024-03-30T00:50:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 -- 172.21.15.66:0/2702301448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb48c0cdef0 msgr2=0x7fb48c0ab910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb48c0cdef0 0x7fb48c0ab910 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c7c0 tx=0x7fb47400cc80 comp rx=0 tx=0).stop 2024-03-30T00:50:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 -- 172.21.15.66:0/2702301448 shutdown_connections 2024-03-30T00:50:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb48c15c6b0 0x7fb48c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4680666c0 0x7fb468068b80 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb48c0cdef0 0x7fb48c0ab910 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 --2- 172.21.15.66:0/2702301448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb48c06cb90 0x7fb48c15ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 -- 172.21.15.66:0/2702301448 >> 172.21.15.66:0/2702301448 conn(0x7fb48c069d90 msgr2=0x7fb48c0b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:25.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 -- 172.21.15.66:0/2702301448 shutdown_connections 2024-03-30T00:50:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.656+0000 7fb48be00640 1 -- 172.21.15.66:0/2702301448 wait complete. 2024-03-30T00:50:25.685 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T00:50:25.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.824+0000 7fb60ee00640 1 Processor -- start 2024-03-30T00:50:25.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.824+0000 7fb60ee00640 1 -- start start 2024-03-30T00:50:25.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb60ee00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb608157060 0x7fb60806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:25.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb60ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6081610c0 0x7fb6081604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:25.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb60ee00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60806dc60 con 0x7fb6081609f0 2024-03-30T00:50:25.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb60ee00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb608157440 con 0x7fb608157060 2024-03-30T00:50:25.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb60ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb608160ea0 con 0x7fb6081610c0 2024-03-30T00:50:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb607e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb608157060 0x7fb60806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb607e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb608157060 0x7fb60806d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35606/0 (socket says 172.21.15.66:35606) 2024-03-30T00:50:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb607e00640 1 -- 172.21.15.66:0/3441408132 learned_addr learned my addr 172.21.15.66:0/3441408132 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb5ff400640 1 --2- 172.21.15.66:0/3441408132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6081610c0 0x7fb6081604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb607e00640 1 -- 172.21.15.66:0/3441408132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6081610c0 msgr2=0x7fb6081604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb607e00640 1 --2- 172.21.15.66:0/3441408132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6081610c0 0x7fb6081604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb607e00640 1 -- 172.21.15.66:0/3441408132 >> v1:172.21.15.156:6789/0 conn(0x7fb6081609f0 legacy=0x7fb60815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb607e00640 1 -- 172.21.15.66:0/3441408132 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60806df40 con 0x7fb608157060 2024-03-30T00:50:25.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb607e00640 1 --2- 172.21.15.66:0/3441408132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb608157060 0x7fb60806d310 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009e50 tx=0x7fb5f801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8daa13c77b4b2b06 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:25.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.828+0000 7fb607400640 1 -- 172.21.15.66:0/3441408132 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60815dde0 con 0x7fb608157060 2024-03-30T00:50:25.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb607400640 1 -- 172.21.15.66:0/3441408132 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f80a56c0 con 0x7fb608157060 2024-03-30T00:50:25.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb607400640 1 -- 172.21.15.66:0/3441408132 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80a5a50 con 0x7fb608157060 2024-03-30T00:50:25.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 -- 172.21.15.66:0/3441408132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb608157060 msgr2=0x7fb60806d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:25.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 --2- 172.21.15.66:0/3441408132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb608157060 0x7fb60806d310 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009e50 tx=0x7fb5f801c910 comp rx=0 tx=0).stop 2024-03-30T00:50:25.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 -- 172.21.15.66:0/3441408132 shutdown_connections 2024-03-30T00:50:25.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 --2- 172.21.15.66:0/3441408132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6081610c0 0x7fb6081604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:25.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 --2- 172.21.15.66:0/3441408132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb608157060 0x7fb60806d310 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:25.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 -- 172.21.15.66:0/3441408132 >> 172.21.15.66:0/3441408132 conn(0x7fb608071750 msgr2=0x7fb608069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:25.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 -- 172.21.15.66:0/3441408132 shutdown_connections 2024-03-30T00:50:25.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 -- 172.21.15.66:0/3441408132 wait complete. 2024-03-30T00:50:25.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 Processor -- start 2024-03-30T00:50:25.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 -- start start 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb608157060 0x7fb60814f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6081609f0 0x7fb60814a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6081610c0 0x7fb60814a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60814aeb0 con 0x7fb6081609f0 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6080b43f0 con 0x7fb6081610c0 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60814c160 con 0x7fb608157060 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb5ff400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6081609f0 0x7fb60814a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb5ff400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6081609f0 0x7fb60814a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35618/0 (socket says 172.21.15.66:35618) 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb5ff400640 1 -- 172.21.15.66:0/1875304694 learned_addr learned my addr 172.21.15.66:0/1875304694 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb607e00640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb608157060 0x7fb60814f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb60ce00640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6081610c0 0x7fb60814a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb5ff400640 1 -- 172.21.15.66:0/1875304694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb608157060 msgr2=0x7fb60814f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:25.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb5ff400640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb608157060 0x7fb60814f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:25.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb5ff400640 1 -- 172.21.15.66:0/1875304694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6081610c0 msgr2=0x7fb60814a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:25.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb5ff400640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6081610c0 0x7fb60814a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:25.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.832+0000 7fb5ff400640 1 -- 172.21.15.66:0/1875304694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f8009b30 con 0x7fb6081609f0 2024-03-30T00:50:25.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.836+0000 7fb5ff400640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6081609f0 0x7fb60814a430 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000d7d0 tx=0x7fb5f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:25.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.836+0000 7fb604c00640 1 -- 172.21.15.66:0/1875304694 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000b6c0 con 0x7fb6081609f0 2024-03-30T00:50:25.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.836+0000 7fb604c00640 1 -- 172.21.15.66:0/1875304694 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f000b820 con 0x7fb6081609f0 2024-03-30T00:50:25.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.836+0000 7fb604c00640 1 -- 172.21.15.66:0/1875304694 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000bbd0 con 0x7fb6081609f0 2024-03-30T00:50:25.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.836+0000 7fb60ee00640 1 -- 172.21.15.66:0/1875304694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60814c440 con 0x7fb6081609f0 2024-03-30T00:50:25.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.836+0000 7fb60ee00640 1 -- 172.21.15.66:0/1875304694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60814ca00 con 0x7fb6081609f0 2024-03-30T00:50:25.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.836+0000 7fb60ee00640 1 -- 172.21.15.66:0/1875304694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb608066c30 con 0x7fb6081609f0 2024-03-30T00:50:25.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.836+0000 7fb604c00640 1 -- 172.21.15.66:0/1875304694 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb5f000bd30 con 0x7fb6081609f0 2024-03-30T00:50:25.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.836+0000 7fb604c00640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb5e4066690 0x7fb5e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:25.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.836+0000 7fb604c00640 1 -- 172.21.15.66:0/1875304694 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fb5f00848e0 con 0x7fb6081609f0 2024-03-30T00:50:25.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.836+0000 7fb607e00640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb5e4066690 0x7fb5e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:25.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.840+0000 7fb607e00640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb5e4066690 0x7fb5e4068b50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80ae000 tx=0x7fb5f80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:25.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:25.840+0000 7fb604c00640 1 -- 172.21.15.66:0/1875304694 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb5f00519a0 con 0x7fb6081609f0 2024-03-30T00:50:26.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.040+0000 7fb60ee00640 1 -- 172.21.15.66:0/1875304694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb608071f30 con 0x7fb6081609f0 2024-03-30T00:50:26.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb604c00640 1 -- 172.21.15.66:0/1875304694 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb5f0054fc0 con 0x7fb6081609f0 2024-03-30T00:50:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 -- 172.21.15.66:0/1875304694 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb5e4066690 msgr2=0x7fb5e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb5e4066690 0x7fb5e4068b50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80ae000 tx=0x7fb5f80a7070 comp rx=0 tx=0).stop 2024-03-30T00:50:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 -- 172.21.15.66:0/1875304694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6081609f0 msgr2=0x7fb60814a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6081609f0 0x7fb60814a430 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000d7d0 tx=0x7fb5f000dc90 comp rx=0 tx=0).stop 2024-03-30T00:50:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 -- 172.21.15.66:0/1875304694 shutdown_connections 2024-03-30T00:50:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6081610c0 0x7fb60814a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb5e4066690 0x7fb5e4068b50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6081609f0 0x7fb60814a430 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 --2- 172.21.15.66:0/1875304694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb608157060 0x7fb60814f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 -- 172.21.15.66:0/1875304694 >> 172.21.15.66:0/1875304694 conn(0x7fb608071750 msgr2=0x7fb60815aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:26.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 -- 172.21.15.66:0/1875304694 shutdown_connections 2024-03-30T00:50:26.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.656+0000 7fb60ee00640 1 -- 172.21.15.66:0/1875304694 wait complete. 2024-03-30T00:50:26.689 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T00:50:26.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff15c000640 1 Processor -- start 2024-03-30T00:50:26.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff15c000640 1 -- start start 2024-03-30T00:50:26.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff15c000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff15415cca0 0x7ff15415df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff15c000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff154099850 0x7ff154098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff15c000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff154098f60 0x7ff154160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff15c000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1540993f0 con 0x7ff15415cca0 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff15c000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15406bdf0 con 0x7ff154099850 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff15c000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15406b1e0 con 0x7ff154098f60 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff159600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff15415cca0 0x7ff15415df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff159600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff15415cca0 0x7ff15415df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35628/0 (socket says 172.21.15.66:35628) 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff159600640 1 -- 172.21.15.66:0/1507221280 learned_addr learned my addr 172.21.15.66:0/1507221280 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff158c00640 1 --2- 172.21.15.66:0/1507221280 >> v2:172.21.15.156:3300/0 conn(0x7ff154099850 0x7ff154098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff159600640 1 -- 172.21.15.66:0/1507221280 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff154098f60 msgr2=0x7ff154160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff15a000640 1 --2- 172.21.15.66:0/1507221280 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff154098f60 0x7ff154160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff159600640 1 --2- 172.21.15.66:0/1507221280 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff154098f60 0x7ff154160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff159600640 1 -- 172.21.15.66:0/1507221280 >> v2:172.21.15.156:3300/0 conn(0x7ff154099850 msgr2=0x7ff154098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff159600640 1 --2- 172.21.15.66:0/1507221280 >> v2:172.21.15.156:3300/0 conn(0x7ff154099850 0x7ff154098a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:26.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff159600640 1 -- 172.21.15.66:0/1507221280 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15406b4c0 con 0x7ff15415cca0 2024-03-30T00:50:26.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff158c00640 1 --2- 172.21.15.66:0/1507221280 >> v2:172.21.15.156:3300/0 conn(0x7ff154099850 0x7ff154098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:26.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.832+0000 7ff15a000640 1 --2- 172.21.15.66:0/1507221280 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff154098f60 0x7ff154160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:26.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff159600640 1 --2- 172.21.15.66:0/1507221280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff15415cca0 0x7ff15415df00 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ff1540ce6f0 tx=0x7ff14801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d423d94bacb76a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:26.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff153e00640 1 -- 172.21.15.66:0/1507221280 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1480ab600 con 0x7ff15415cca0 2024-03-30T00:50:26.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff153e00640 1 -- 172.21.15.66:0/1507221280 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1480a7050 con 0x7ff15415cca0 2024-03-30T00:50:26.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff153e00640 1 -- 172.21.15.66:0/1507221280 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1480ab980 con 0x7ff15415cca0 2024-03-30T00:50:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff15c000640 1 -- 172.21.15.66:0/1507221280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff15415cca0 msgr2=0x7ff15415df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff15c000640 1 --2- 172.21.15.66:0/1507221280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff15415cca0 0x7ff15415df00 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ff1540ce6f0 tx=0x7ff14801c230 comp rx=0 tx=0).stop 2024-03-30T00:50:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff15c000640 1 -- 172.21.15.66:0/1507221280 shutdown_connections 2024-03-30T00:50:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff15c000640 1 --2- 172.21.15.66:0/1507221280 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff154098f60 0x7ff154160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff15c000640 1 --2- 172.21.15.66:0/1507221280 >> v2:172.21.15.156:3300/0 conn(0x7ff154099850 0x7ff154098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff15c000640 1 --2- 172.21.15.66:0/1507221280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff15415cca0 0x7ff15415df00 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff15c000640 1 -- 172.21.15.66:0/1507221280 >> 172.21.15.66:0/1507221280 conn(0x7ff1540b2140 msgr2=0x7ff1540b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff15c000640 1 -- 172.21.15.66:0/1507221280 shutdown_connections 2024-03-30T00:50:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.836+0000 7ff15c000640 1 -- 172.21.15.66:0/1507221280 wait complete. 2024-03-30T00:50:26.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15c000640 1 Processor -- start 2024-03-30T00:50:26.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15c000640 1 -- start start 2024-03-30T00:50:26.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15c000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff154098f60 0x7ff1540c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:26.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15c000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff154099850 0x7ff1540c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:26.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15c000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff15415cca0 0x7ff1540c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:26.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15c000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff154157e40 con 0x7ff15415cca0 2024-03-30T00:50:26.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15c000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff154157230 con 0x7ff154099850 2024-03-30T00:50:26.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15c000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff154157510 con 0x7ff154098f60 2024-03-30T00:50:26.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff15415cca0 0x7ff1540c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:26.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff15415cca0 0x7ff1540c88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35634/0 (socket says 172.21.15.66:35634) 2024-03-30T00:50:26.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15a000640 1 -- 172.21.15.66:0/2299399232 learned_addr learned my addr 172.21.15.66:0/2299399232 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:26.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff158c00640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff154099850 0x7ff1540c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:26.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff159600640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff154098f60 0x7ff1540c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:26.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15a000640 1 -- 172.21.15.66:0/2299399232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff154098f60 msgr2=0x7ff1540c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:26.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15a000640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff154098f60 0x7ff1540c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:26.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15a000640 1 -- 172.21.15.66:0/2299399232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff154099850 msgr2=0x7ff1540c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:26.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15a000640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff154099850 0x7ff1540c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:26.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15a000640 1 -- 172.21.15.66:0/2299399232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1480094d0 con 0x7ff15415cca0 2024-03-30T00:50:26.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff158c00640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff154099850 0x7ff1540c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:26.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15a000640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff15415cca0 0x7ff1540c88a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00cef0 tx=0x7ff14c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:26.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff151600640 1 -- 172.21.15.66:0/2299399232 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c00d840 con 0x7ff15415cca0 2024-03-30T00:50:26.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff151600640 1 -- 172.21.15.66:0/2299399232 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c00d9c0 con 0x7ff15415cca0 2024-03-30T00:50:26.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff151600640 1 -- 172.21.15.66:0/2299399232 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c00dd70 con 0x7ff15415cca0 2024-03-30T00:50:26.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15c000640 1 -- 172.21.15.66:0/2299399232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15406c400 con 0x7ff15415cca0 2024-03-30T00:50:26.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.840+0000 7ff15c000640 1 -- 172.21.15.66:0/2299399232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff154157ad0 con 0x7ff15415cca0 2024-03-30T00:50:26.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.844+0000 7ff15c000640 1 -- 172.21.15.66:0/2299399232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15406c260 con 0x7ff15415cca0 2024-03-30T00:50:26.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.844+0000 7ff151600640 1 -- 172.21.15.66:0/2299399232 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff14c01e3b0 con 0x7ff15415cca0 2024-03-30T00:50:26.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.844+0000 7ff151600640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff134066690 0x7ff134068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:26.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.844+0000 7ff151600640 1 -- 172.21.15.66:0/2299399232 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6230+0+0 (secure 0 0 0) 0x7ff14c084d80 con 0x7ff15415cca0 2024-03-30T00:50:26.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.844+0000 7ff159600640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff134066690 0x7ff134068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:26.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.844+0000 7ff159600640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff134066690 0x7ff134068b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff1480ad000 tx=0x7ff1480a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:26.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:26.848+0000 7ff151600640 1 -- 172.21.15.66:0/2299399232 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff15406c260 con 0x7ff15415cca0 2024-03-30T00:50:27.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.012+0000 7ff15c000640 1 -- 172.21.15.66:0/2299399232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff15406c260 con 0x7ff15415cca0 2024-03-30T00:50:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.659+0000 7ff151600640 1 -- 172.21.15.66:0/2299399232 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff15406c260 con 0x7ff15415cca0 2024-03-30T00:50:27.669 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-03-30T00:50:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 -- 172.21.15.66:0/2299399232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff134066690 msgr2=0x7ff134068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff134066690 0x7ff134068b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff1480ad000 tx=0x7ff1480a6000 comp rx=0 tx=0).stop 2024-03-30T00:50:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 -- 172.21.15.66:0/2299399232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff15415cca0 msgr2=0x7ff1540c88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:27.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff15415cca0 0x7ff1540c88a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00cef0 tx=0x7ff14c00a710 comp rx=0 tx=0).stop 2024-03-30T00:50:27.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 -- 172.21.15.66:0/2299399232 shutdown_connections 2024-03-30T00:50:27.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff134066690 0x7ff134068b50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:27.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff15415cca0 0x7ff1540c88a0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:27.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff154099850 0x7ff1540c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:27.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 --2- 172.21.15.66:0/2299399232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff154098f60 0x7ff1540c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:27.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 -- 172.21.15.66:0/2299399232 >> 172.21.15.66:0/2299399232 conn(0x7ff1540b2140 msgr2=0x7ff1540cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:27.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 -- 172.21.15.66:0/2299399232 shutdown_connections 2024-03-30T00:50:27.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.663+0000 7ff15c000640 1 -- 172.21.15.66:0/2299399232 wait complete. 2024-03-30T00:50:27.696 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T00:50:27.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eee00640 1 Processor -- start 2024-03-30T00:50:27.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eee00640 1 -- start start 2024-03-30T00:50:27.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e8071930 0x7f29e806bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:27.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e80723c0 con 0x7f29e8158060 2024-03-30T00:50:27.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eee00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e8158440 con 0x7f29e80acd60 2024-03-30T00:50:27.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80ab080 con 0x7f29e8071930 2024-03-30T00:50:27.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eda00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f29e8158060 0x7f29e815b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55320/0 (socket says 172.21.15.66:55320) 2024-03-30T00:50:27.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eda00640 1 -- 172.21.15.66:0/2013528900 learned_addr learned my addr 172.21.15.66:0/2013528900 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:27.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29ed000640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e8071930 0x7f29e806bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:27.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869006819 0 0) 0x7f29e80723c0 con 0x7f29e8158060 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29d0003530 con 0x7f29e8158060 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1384568956 0 0) 0x7f29d0003530 con 0x7f29e8158060 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e8071930 msgr2=0x7f29e806bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e8071930 0x7f29e806bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 >> v1:172.21.15.156:6789/0 conn(0x7f29e80acd60 legacy=0x7f29e80ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e8071df0 con 0x7f29e8158060 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f29dc001680 con 0x7f29e8158060 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 >> v1:172.21.15.66:6789/0 conn(0x7f29e8158060 legacy=0x7f29e815b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 0x7f29d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e8071930 0x7f29d00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29d0004d10 0x7f29d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80ab080 con 0x7f29d00039b0 2024-03-30T00:50:27.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d000ba40 con 0x7f29d0004d10 2024-03-30T00:50:27.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d000bc00 con 0x7f29e8071930 2024-03-30T00:50:27.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eda00640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 0x7f29d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:27.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29ee400640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29d0004d10 0x7f29d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:27.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29ed000640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e8071930 0x7f29d00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:27.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eda00640 1 -- 172.21.15.66:0/2013528900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e8071930 msgr2=0x7f29d00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:27.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eda00640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e8071930 0x7f29d00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:27.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eda00640 1 -- 172.21.15.66:0/2013528900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29d0004d10 msgr2=0x7f29d0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:27.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eda00640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29d0004d10 0x7f29d0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:27.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.835+0000 7f29eda00640 1 -- 172.21.15.66:0/2013528900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d0003530 con 0x7f29d00039b0 2024-03-30T00:50:27.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.839+0000 7f29eda00640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 0x7f29d0003dd0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c210 tx=0x7f29dc01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797968603d83523e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:27.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.839+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc005ae0 con 0x7f29d00039b0 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.839+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc005c60 con 0x7f29d00039b0 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.839+0000 7f29e7e00640 1 -- 172.21.15.66:0/2013528900 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0a5800 con 0x7f29d00039b0 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.839+0000 7f29eee00640 1 -- 172.21.15.66:0/2013528900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 msgr2=0x7f29d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.839+0000 7f29eee00640 1 --2- 172.21.15.66:0/2013528900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 0x7f29d0003dd0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c210 tx=0x7f29dc01cc10 comp rx=0 tx=0).stop 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.839+0000 7f29ee400640 1 -- 172.21.15.66:0/2013528900 reap_dead start 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.839+0000 7f29eee00640 1 -- 172.21.15.66:0/2013528900 shutdown_connections 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.839+0000 7f29eee00640 1 -- 172.21.15.66:0/2013528900 >> 172.21.15.66:0/2013528900 conn(0x7f29e80afda0 msgr2=0x7f29e80b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.839+0000 7f29eee00640 1 -- 172.21.15.66:0/2013528900 shutdown_connections 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.839+0000 7f29eee00640 1 -- 172.21.15.66:0/2013528900 wait complete. 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eee00640 1 Processor -- start 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eee00640 1 -- start start 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 0x7f29e813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e813a190 0x7f29e80978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eee00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e80ae800 0x7f29e8156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e814d0c0 con 0x7f29d00039b0 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eee00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e814d330 con 0x7f29e80ae800 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e814d610 con 0x7f29e813a190 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eda00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 0x7f29e813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eda00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 0x7f29e813cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35664/0 (socket says 172.21.15.66:35664) 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eda00640 1 -- 172.21.15.66:0/4175295742 learned_addr learned my addr 172.21.15.66:0/4175295742 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29ee400640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e80ae800 0x7f29e8156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29ed000640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e813a190 0x7f29e80978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eda00640 1 -- 172.21.15.66:0/4175295742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e813a190 msgr2=0x7f29e80978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eda00640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e813a190 0x7f29e80978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eda00640 1 -- 172.21.15.66:0/4175295742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e80ae800 msgr2=0x7f29e8156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eda00640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e80ae800 0x7f29e8156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:27.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eda00640 1 -- 172.21.15.66:0/4175295742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e814d8f0 con 0x7f29d00039b0 2024-03-30T00:50:27.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eda00640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 0x7f29e813cd80 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c210 tx=0x7f29d0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:27.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29e5600640 1 -- 172.21.15.66:0/4175295742 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0b4070 con 0x7f29d00039b0 2024-03-30T00:50:27.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29e5600640 1 -- 172.21.15.66:0/4175295742 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc005dd0 con 0x7f29d00039b0 2024-03-30T00:50:27.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eee00640 1 -- 172.21.15.66:0/4175295742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e8097e30 con 0x7f29d00039b0 2024-03-30T00:50:27.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29e5600640 1 -- 172.21.15.66:0/4175295742 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc001660 con 0x7f29d00039b0 2024-03-30T00:50:27.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.843+0000 7f29eee00640 1 -- 172.21.15.66:0/4175295742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e80aec80 con 0x7f29d00039b0 2024-03-30T00:50:27.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.847+0000 7f29eee00640 1 -- 172.21.15.66:0/4175295742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e814d470 con 0x7f29d00039b0 2024-03-30T00:50:27.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.847+0000 7f29e5600640 1 -- 172.21.15.66:0/4175295742 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f29dc0a5800 con 0x7f29d00039b0 2024-03-30T00:50:27.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.847+0000 7f29e5600640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f29c8066690 0x7f29c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:27.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.847+0000 7f29e5600640 1 -- 172.21.15.66:0/4175295742 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f29dc11dac0 con 0x7f29d00039b0 2024-03-30T00:50:27.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.847+0000 7f29ed000640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f29c8066690 0x7f29c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:27.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.847+0000 7f29ed000640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f29c8066690 0x7f29c8068b50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f29e809c3a0 tx=0x7f29d4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:27.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:27.851+0000 7f29e5600640 1 -- 172.21.15.66:0/4175295742 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f29e814d470 con 0x7f29d00039b0 2024-03-30T00:50:28.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.015+0000 7f29eee00640 1 -- 172.21.15.66:0/4175295742 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f29e814d470 con 0x7f29d00039b0 2024-03-30T00:50:28.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.675+0000 7f29e5600640 1 -- 172.21.15.66:0/4175295742 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) v1 ==== 169+0+0 (secure 0 0 0) 0x7f29e814d470 con 0x7f29d00039b0 2024-03-30T00:50:28.682 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-03-30T00:50:28.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 -- 172.21.15.66:0/4175295742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f29c8066690 msgr2=0x7f29c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:28.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f29c8066690 0x7f29c8068b50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f29e809c3a0 tx=0x7f29d4009190 comp rx=0 tx=0).stop 2024-03-30T00:50:28.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 -- 172.21.15.66:0/4175295742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 msgr2=0x7f29e813cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:28.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 0x7f29e813cd80 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c210 tx=0x7f29d0005800 comp rx=0 tx=0).stop 2024-03-30T00:50:28.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 -- 172.21.15.66:0/4175295742 shutdown_connections 2024-03-30T00:50:28.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e80ae800 0x7f29e8156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:28.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29e813a190 0x7f29e80978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:28.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f29c8066690 0x7f29c8068b50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:28.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 --2- 172.21.15.66:0/4175295742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d00039b0 0x7f29e813cd80 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:28.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 -- 172.21.15.66:0/4175295742 >> 172.21.15.66:0/4175295742 conn(0x7f29e80afda0 msgr2=0x7f29e81606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:28.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 -- 172.21.15.66:0/4175295742 shutdown_connections 2024-03-30T00:50:28.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.679+0000 7f29eee00640 1 -- 172.21.15.66:0/4175295742 wait complete. 2024-03-30T00:50:28.709 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-30T00:50:28.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a0d400640 1 Processor -- start 2024-03-30T00:50:28.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a0d400640 1 -- start start 2024-03-30T00:50:28.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a0d400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0815a640 0x7f0a0806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:28.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a0d400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0a080cabb0 0x7f0a080cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:28.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a0d400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0806d070 con 0x7f0a080ce520 2024-03-30T00:50:28.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a0d400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a080cbba0 con 0x7f0a080cabb0 2024-03-30T00:50:28.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a0d400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0806e2c0 con 0x7f0a0815a640 2024-03-30T00:50:28.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a07200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0a080cabb0 0x7f0a080cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:28.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a06800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0815a640 0x7f0a0806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:28.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f09fde00640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f0a080ce520 0x7f0a080ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:60480/0 (socket says 172.21.15.66:60480) 2024-03-30T00:50:28.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a07200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0a080cabb0 0x7f0a080cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:51480/0 (socket says 172.21.15.66:51480) 2024-03-30T00:50:28.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a06800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0815a640 0x7f0a0806dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:37406/0 (socket says 172.21.15.66:37406) 2024-03-30T00:50:28.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f09fde00640 1 -- 172.21.15.66:0/3007684423 learned_addr learned my addr 172.21.15.66:0/3007684423 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:28.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a07200640 1 -- 172.21.15.66:0/3007684423 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0815a640 msgr2=0x7f0a0806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:28.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a07200640 1 --2- 172.21.15.66:0/3007684423 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0815a640 0x7f0a0806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:28.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a07200640 1 -- 172.21.15.66:0/3007684423 >> v1:172.21.15.156:6789/0 conn(0x7f0a080ce520 legacy=0x7f0a080ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:28.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.847+0000 7f0a07200640 1 -- 172.21.15.66:0/3007684423 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a080cb8c0 con 0x7f0a080cabb0 2024-03-30T00:50:28.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a07200640 1 --2- 172.21.15.66:0/3007684423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a080cabb0 0x7f0a080cb040 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0a000097c0 tx=0x7f0a0001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fde076633f7a769 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:28.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a05e00640 1 -- 172.21.15.66:0/3007684423 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000ab670 con 0x7f0a080cabb0 2024-03-30T00:50:28.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a05e00640 1 -- 172.21.15.66:0/3007684423 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a000ab7d0 con 0x7f0a080cabb0 2024-03-30T00:50:28.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a05e00640 1 -- 172.21.15.66:0/3007684423 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000abb60 con 0x7f0a080cabb0 2024-03-30T00:50:28.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a0d400640 1 -- 172.21.15.66:0/3007684423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a080cabb0 msgr2=0x7f0a080cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:28.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a0d400640 1 --2- 172.21.15.66:0/3007684423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a080cabb0 0x7f0a080cb040 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0a000097c0 tx=0x7f0a0001c470 comp rx=0 tx=0).stop 2024-03-30T00:50:28.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a0d400640 1 -- 172.21.15.66:0/3007684423 shutdown_connections 2024-03-30T00:50:28.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a0d400640 1 --2- 172.21.15.66:0/3007684423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a080cabb0 0x7f0a080cb040 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:28.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a0d400640 1 --2- 172.21.15.66:0/3007684423 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0815a640 0x7f0a0806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:28.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a0d400640 1 -- 172.21.15.66:0/3007684423 >> 172.21.15.66:0/3007684423 conn(0x7f0a08069f60 msgr2=0x7f0a0806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:28.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a0d400640 1 -- 172.21.15.66:0/3007684423 shutdown_connections 2024-03-30T00:50:28.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.851+0000 7f0a0d400640 1 -- 172.21.15.66:0/3007684423 wait complete. 2024-03-30T00:50:28.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a0d400640 1 Processor -- start 2024-03-30T00:50:28.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a0d400640 1 -- start start 2024-03-30T00:50:28.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a0d400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a080cabb0 0x7f0a080ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:28.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a0d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a080ce520 0x7f0a08162670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a0d400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0815a640 0x7f0a081f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a0d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a08158db0 con 0x7f0a080ce520 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a0d400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0815d620 con 0x7f0a080cabb0 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a0d400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080c46f0 con 0x7f0a0815a640 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09fde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a080ce520 0x7f0a08162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09fde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a080ce520 0x7f0a08162670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48916/0 (socket says 172.21.15.66:48916) 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a06800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a080cabb0 0x7f0a080ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09fde00640 1 -- 172.21.15.66:0/3177151207 learned_addr learned my addr 172.21.15.66:0/3177151207 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a07200640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0815a640 0x7f0a081f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09fde00640 1 -- 172.21.15.66:0/3177151207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0815a640 msgr2=0x7f0a081f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09fde00640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0815a640 0x7f0a081f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09fde00640 1 -- 172.21.15.66:0/3177151207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a080cabb0 msgr2=0x7f0a080ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09fde00640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a080cabb0 0x7f0a080ac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09fde00640 1 -- 172.21.15.66:0/3177151207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00009420 con 0x7f0a080ce520 2024-03-30T00:50:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a06800640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a080cabb0 0x7f0a080ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09fde00640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a080ce520 0x7f0a08162670 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f09f000c500 tx=0x7f09f000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09ff400640 1 -- 172.21.15.66:0/3177151207 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0005cf0 con 0x7f0a080ce520 2024-03-30T00:50:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09ff400640 1 -- 172.21.15.66:0/3177151207 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f0005e50 con 0x7f0a080ce520 2024-03-30T00:50:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f09ff400640 1 -- 172.21.15.66:0/3177151207 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f00072f0 con 0x7f0a080ce520 2024-03-30T00:50:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a0d400640 1 -- 172.21.15.66:0/3177151207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a08158fc0 con 0x7f0a080ce520 2024-03-30T00:50:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.855+0000 7f0a0d400640 1 -- 172.21.15.66:0/3177151207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a08162bb0 con 0x7f0a080ce520 2024-03-30T00:50:28.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.859+0000 7f0a0d400640 1 -- 172.21.15.66:0/3177151207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a08072c00 con 0x7f0a080ce520 2024-03-30T00:50:28.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.859+0000 7f09ff400640 1 -- 172.21.15.66:0/3177151207 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f09f0007450 con 0x7f0a080ce520 2024-03-30T00:50:28.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.859+0000 7f09ff400640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e4066690 0x7f09e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:28.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.859+0000 7f09ff400640 1 -- 172.21.15.66:0/3177151207 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f09f0083530 con 0x7f0a080ce520 2024-03-30T00:50:28.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.859+0000 7f0a06800640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e4066690 0x7f09e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:28.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.859+0000 7f0a06800640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e4066690 0x7f09e4068b50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f09f8009fd0 tx=0x7f09f800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:28.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:28.863+0000 7f09ff400640 1 -- 172.21.15.66:0/3177151207 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f09f0050aa0 con 0x7f0a080ce520 2024-03-30T00:50:29.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.023+0000 7f0a0d400640 1 -- 172.21.15.66:0/3177151207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0a08066c30 con 0x7f0a080ce520 2024-03-30T00:50:29.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.027+0000 7f09ff400640 1 -- 172.21.15.66:0/3177151207 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f09f00540c0 con 0x7f0a080ce520 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 -- 172.21.15.66:0/3177151207 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e4066690 msgr2=0x7f09e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e4066690 0x7f09e4068b50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f09f8009fd0 tx=0x7f09f800b040 comp rx=0 tx=0).stop 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 -- 172.21.15.66:0/3177151207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a080ce520 msgr2=0x7f0a08162670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a080ce520 0x7f0a08162670 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f09f000c500 tx=0x7f09f000c9c0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 -- 172.21.15.66:0/3177151207 shutdown_connections 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a0815a640 0x7f0a081f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09e4066690 0x7f09e4068b50 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a080ce520 0x7f0a08162670 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 --2- 172.21.15.66:0/3177151207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a080cabb0 0x7f0a080ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 -- 172.21.15.66:0/3177151207 >> 172.21.15.66:0/3177151207 conn(0x7f0a08069f60 msgr2=0x7f0a080afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 -- 172.21.15.66:0/3177151207 shutdown_connections 2024-03-30T00:50:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.031+0000 7f0a0d400640 1 -- 172.21.15.66:0/3177151207 wait complete. 2024-03-30T00:50:29.058 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-03-30T00:50:29.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.199+0000 7f4c94800640 1 Processor -- start 2024-03-30T00:50:29.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.199+0000 7f4c94800640 1 -- start start 2024-03-30T00:50:29.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c94800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c8c15de30 0x7f4c8c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c94800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4c8c0cee50 0x7f4c8c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c94800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c8c0cbb90 con 0x7f4c8c0af420 2024-03-30T00:50:29.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c94800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c8c0ce1b0 con 0x7f4c8c0cee50 2024-03-30T00:50:29.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c94800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0afa10 con 0x7f4c8c15de30 2024-03-30T00:50:29.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c91400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f4c8c0af420 0x7f4c8c0ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40860/0 (socket says 172.21.15.66:40860) 2024-03-30T00:50:29.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c91400640 1 -- 172.21.15.66:0/3095730879 learned_addr learned my addr 172.21.15.66:0/3095730879 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:29.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c91e00640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c8c15de30 0x7f4c8c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c92800640 1 --2- 172.21.15.66:0/3095730879 >> v2:172.21.15.156:3300/0 conn(0x7f4c8c0cee50 0x7f4c8c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301635443 0 0) 0x7f4c8c0cbb90 con 0x7f4c8c0af420 2024-03-30T00:50:29.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c74003530 con 0x7f4c8c0af420 2024-03-30T00:50:29.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2151790474 0 0) 0x7f4c74003530 con 0x7f4c8c0af420 2024-03-30T00:50:29.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c8c15de30 msgr2=0x7f4c8c0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:29.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c8c15de30 0x7f4c8c0cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 >> v2:172.21.15.156:3300/0 conn(0x7f4c8c0cee50 msgr2=0x7f4c8c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:29.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 --2- 172.21.15.66:0/3095730879 >> v2:172.21.15.156:3300/0 conn(0x7f4c8c0cee50 0x7f4c8c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0cf430 con 0x7f4c8c0af420 2024-03-30T00:50:29.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f4c7c0016a0 con 0x7f4c8c0af420 2024-03-30T00:50:29.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c92800640 1 --2- 172.21.15.66:0/3095730879 >> v2:172.21.15.156:3300/0 conn(0x7f4c8c0cee50 0x7f4c8c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 >> v1:172.21.15.66:6789/0 conn(0x7f4c8c0af420 legacy=0x7f4c8c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 0x7f4c74003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c740047d0 0x7f4c74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c74005190 0x7f4c7400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0ce1b0 con 0x7f4c74003980 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0afa10 con 0x7f4c74005190 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7400bf20 con 0x7f4c740047d0 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c91e00640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c8c15de30 0x7f4c8c0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c91e00640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 0x7f4c74003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c91400640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c740047d0 0x7f4c74004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c92800640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c74005190 0x7f4c7400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c91e00640 1 -- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c740047d0 msgr2=0x7f4c74004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c91e00640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c740047d0 0x7f4c74004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c91e00640 1 -- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c74005190 msgr2=0x7f4c7400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:29.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c91e00640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c74005190 0x7f4c7400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.203+0000 7f4c91e00640 1 -- 172.21.15.66:0/3095730879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c74003530 con 0x7f4c74003980 2024-03-30T00:50:29.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c91e00640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 0x7f4c74003da0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009b20 tx=0x7f4c8001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee7a3ee77dadcfb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:29.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80004030 con 0x7f4c74003980 2024-03-30T00:50:29.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80004190 con 0x7f4c74003980 2024-03-30T00:50:29.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c90a00640 1 -- 172.21.15.66:0/3095730879 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80004520 con 0x7f4c74003980 2024-03-30T00:50:29.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c94800640 1 -- 172.21.15.66:0/3095730879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 msgr2=0x7f4c74003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c94800640 1 --2- 172.21.15.66:0/3095730879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 0x7f4c74003da0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009b20 tx=0x7f4c8001cab0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c92800640 1 -- 172.21.15.66:0/3095730879 reap_dead start 2024-03-30T00:50:29.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c94800640 1 -- 172.21.15.66:0/3095730879 shutdown_connections 2024-03-30T00:50:29.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c94800640 1 -- 172.21.15.66:0/3095730879 >> 172.21.15.66:0/3095730879 conn(0x7f4c8c0d0540 msgr2=0x7f4c8c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:29.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c94800640 1 -- 172.21.15.66:0/3095730879 shutdown_connections 2024-03-30T00:50:29.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c94800640 1 -- 172.21.15.66:0/3095730879 wait complete. 2024-03-30T00:50:29.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c94800640 1 Processor -- start 2024-03-30T00:50:29.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.207+0000 7f4c94800640 1 -- start start 2024-03-30T00:50:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c94800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 0x7f4c8c0ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c94800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c8c141100 0x7f4c8c141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c94800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c8c0b5280 0x7f4c8c0b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c94800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0b4ae0 con 0x7f4c74003980 2024-03-30T00:50:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c94800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0c7790 con 0x7f4c8c141100 2024-03-30T00:50:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c94800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0c7a70 con 0x7f4c8c0b5280 2024-03-30T00:50:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c91e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 0x7f4c8c0ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c91e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 0x7f4c8c0ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48924/0 (socket says 172.21.15.66:48924) 2024-03-30T00:50:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c91e00640 1 -- 172.21.15.66:0/3388889389 learned_addr learned my addr 172.21.15.66:0/3388889389 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c91400640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c8c141100 0x7f4c8c141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c92800640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c8c0b5280 0x7f4c8c0b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c91e00640 1 -- 172.21.15.66:0/3388889389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c8c0b5280 msgr2=0x7f4c8c0b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c91e00640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c8c0b5280 0x7f4c8c0b5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c91e00640 1 -- 172.21.15.66:0/3388889389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c8c141100 msgr2=0x7f4c8c141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c91e00640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c8c141100 0x7f4c8c141580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c91e00640 1 -- 172.21.15.66:0/3388889389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0c7d50 con 0x7f4c74003980 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c91e00640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 0x7f4c8c0ae4c0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009b20 tx=0x7f4c74005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c86000640 1 -- 172.21.15.66:0/3388889389 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000f000 con 0x7f4c74003980 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c86000640 1 -- 172.21.15.66:0/3388889389 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80003ca0 con 0x7f4c74003980 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c86000640 1 -- 172.21.15.66:0/3388889389 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80004500 con 0x7f4c74003980 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c94800640 1 -- 172.21.15.66:0/3388889389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8c0b4d60 con 0x7f4c74003980 2024-03-30T00:50:29.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c94800640 1 -- 172.21.15.66:0/3388889389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8c147360 con 0x7f4c74003980 2024-03-30T00:50:29.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c94800640 1 -- 172.21.15.66:0/3388889389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8c0cdf60 con 0x7f4c74003980 2024-03-30T00:50:29.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.211+0000 7f4c86000640 1 -- 172.21.15.66:0/3388889389 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4c800a5590 con 0x7f4c74003980 2024-03-30T00:50:29.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.215+0000 7f4c86000640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c700666c0 0x7f4c70068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.215+0000 7f4c86000640 1 -- 172.21.15.66:0/3388889389 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f4c8011d4a0 con 0x7f4c74003980 2024-03-30T00:50:29.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.215+0000 7f4c91400640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c700666c0 0x7f4c70068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.215+0000 7f4c91400640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c700666c0 0x7f4c70068b80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0735d0 tx=0x7f4c7c007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:29.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.219+0000 7f4c86000640 1 -- 172.21.15.66:0/3388889389 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4c800eaa00 con 0x7f4c74003980 2024-03-30T00:50:29.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.439+0000 7f4c94800640 1 -- 172.21.15.66:0/3388889389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f4c8c0cdf60 con 0x7f4c74003980 2024-03-30T00:50:29.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.679+0000 7f4c86000640 1 -- 172.21.15.66:0/3388889389 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v506) v1 ==== 146+0+0 (secure 0 0 0) 0x7f4c800ee020 con 0x7f4c74003980 2024-03-30T00:50:29.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 -- 172.21.15.66:0/3388889389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c700666c0 msgr2=0x7f4c70068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c700666c0 0x7f4c70068b80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0735d0 tx=0x7f4c7c007900 comp rx=0 tx=0).stop 2024-03-30T00:50:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 -- 172.21.15.66:0/3388889389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 msgr2=0x7f4c8c0ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 0x7f4c8c0ae4c0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009b20 tx=0x7f4c74005800 comp rx=0 tx=0).stop 2024-03-30T00:50:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 -- 172.21.15.66:0/3388889389 shutdown_connections 2024-03-30T00:50:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c8c0b5280 0x7f4c8c0b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c8c141100 0x7f4c8c141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4c700666c0 0x7f4c70068b80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 --2- 172.21.15.66:0/3388889389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c74003980 0x7f4c8c0ae4c0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 -- 172.21.15.66:0/3388889389 >> 172.21.15.66:0/3388889389 conn(0x7f4c8c0d0540 msgr2=0x7f4c8c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 -- 172.21.15.66:0/3388889389 shutdown_connections 2024-03-30T00:50:29.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.683+0000 7f4c94800640 1 -- 172.21.15.66:0/3388889389 wait complete. 2024-03-30T00:50:29.709 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-30T00:50:29.710 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-30T00:50:29.710 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:50:29.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a9400640 1 Processor -- start 2024-03-30T00:50:29.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a9400640 1 -- start start 2024-03-30T00:50:29.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a9400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a406e800 0x7f05a40d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a9400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a406b0b0 con 0x7f05a40cc560 2024-03-30T00:50:29.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a9400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a406b390 con 0x7f05a406f8e0 2024-03-30T00:50:29.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a9400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a4160640 con 0x7f05a406e800 2024-03-30T00:50:29.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a2800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f05a40cc560 0x7f05a406e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40872/0 (socket says 172.21.15.66:40872) 2024-03-30T00:50:29.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a2800640 1 -- 172.21.15.66:0/3717515677 learned_addr learned my addr 172.21.15.66:0/3717515677 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f0599e00640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a406e800 0x7f05a40d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394853182 0 0) 0x7f05a406b0b0 con 0x7f05a40cc560 2024-03-30T00:50:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0588003530 con 0x7f05a40cc560 2024-03-30T00:50:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4286633540 0 0) 0x7f0588003530 con 0x7f05a40cc560 2024-03-30T00:50:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a406e800 msgr2=0x7f05a40d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a406e800 0x7f05a40d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 >> v1:172.21.15.156:6789/0 conn(0x7f05a406f8e0 legacy=0x7f05a40ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a406b670 con 0x7f05a40cc560 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f0594001680 con 0x7f05a40cc560 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 >> v1:172.21.15.66:6789/0 conn(0x7f05a40cc560 legacy=0x7f05a406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05880039b0 0x7f0588003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880047d0 0x7f0588004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a406e800 0x7f0588005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a4160640 con 0x7f05880047d0 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058800ba40 con 0x7f05880039b0 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058800bbd0 con 0x7f05a406e800 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f0599e00640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880047d0 0x7f0588004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.843+0000 7f05a2800640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05880039b0 0x7f0588003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f05a3200640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a406e800 0x7f0588005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f0599e00640 1 -- 172.21.15.66:0/3717515677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a406e800 msgr2=0x7f0588005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f0599e00640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a406e800 0x7f0588005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f0599e00640 1 -- 172.21.15.66:0/3717515677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05880039b0 msgr2=0x7f0588003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f0599e00640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05880039b0 0x7f0588003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f0599e00640 1 -- 172.21.15.66:0/3717515677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0588003530 con 0x7f05880047d0 2024-03-30T00:50:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f0599e00640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880047d0 0x7f0588004c50 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f05a40d15a0 tx=0x7f058c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ced11a82ca4f5d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c0a55e0 con 0x7f05880047d0 2024-03-30T00:50:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c0a5740 con 0x7f05880047d0 2024-03-30T00:50:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f05a1e00640 1 -- 172.21.15.66:0/3717515677 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c0a5ab0 con 0x7f05880047d0 2024-03-30T00:50:29.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f05a9400640 1 -- 172.21.15.66:0/3717515677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880047d0 msgr2=0x7f0588004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f05a9400640 1 --2- 172.21.15.66:0/3717515677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880047d0 0x7f0588004c50 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f05a40d15a0 tx=0x7f058c01c9f0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.847+0000 7f05a3200640 1 -- 172.21.15.66:0/3717515677 reap_dead start 2024-03-30T00:50:29.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 -- 172.21.15.66:0/3717515677 shutdown_connections 2024-03-30T00:50:29.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 -- 172.21.15.66:0/3717515677 >> 172.21.15.66:0/3717515677 conn(0x7f05a40d0540 msgr2=0x7f05a40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:29.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 -- 172.21.15.66:0/3717515677 shutdown_connections 2024-03-30T00:50:29.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 -- 172.21.15.66:0/3717515677 wait complete. 2024-03-30T00:50:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 Processor -- start 2024-03-30T00:50:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 -- start start 2024-03-30T00:50:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05880047d0 0x7f05a4156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05a40c9f90 0x7f05a40b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a40c4560 0x7f05a40c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a40b1a10 con 0x7f05a40c9f90 2024-03-30T00:50:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a4157000 con 0x7f05880047d0 2024-03-30T00:50:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a9400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a40b0b60 con 0x7f05a40c4560 2024-03-30T00:50:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f0599e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05a40c9f90 0x7f05a40b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f0599e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05a40c9f90 0x7f05a40b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48938/0 (socket says 172.21.15.66:48938) 2024-03-30T00:50:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f0599e00640 1 -- 172.21.15.66:0/3329871496 learned_addr learned my addr 172.21.15.66:0/3329871496 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a2800640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05880047d0 0x7f05a4156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f05a3200640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a40c4560 0x7f05a40c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f0599e00640 1 -- 172.21.15.66:0/3329871496 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a40c4560 msgr2=0x7f05a40c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f0599e00640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a40c4560 0x7f05a40c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f0599e00640 1 -- 172.21.15.66:0/3329871496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05880047d0 msgr2=0x7f05a4156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:29.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f0599e00640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05880047d0 0x7f05a4156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:29.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f0599e00640 1 -- 172.21.15.66:0/3329871496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a40b0e00 con 0x7f05a40c9f90 2024-03-30T00:50:29.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f0599e00640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05a40c9f90 0x7f05a40b0620 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f058c01c820 tx=0x7f0588005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:29.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.851+0000 7f059b400640 1 -- 172.21.15.66:0/3329871496 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c0ad850 con 0x7f05a40c9f90 2024-03-30T00:50:29.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.855+0000 7f059b400640 1 -- 172.21.15.66:0/3329871496 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c0ad9d0 con 0x7f05a40c9f90 2024-03-30T00:50:29.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.855+0000 7f059b400640 1 -- 172.21.15.66:0/3329871496 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c0add80 con 0x7f05a40c9f90 2024-03-30T00:50:29.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.855+0000 7f05a9400640 1 -- 172.21.15.66:0/3329871496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a40b10e0 con 0x7f05a40c9f90 2024-03-30T00:50:29.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.855+0000 7f05a9400640 1 -- 172.21.15.66:0/3329871496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a40b16a0 con 0x7f05a40c9f90 2024-03-30T00:50:29.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.855+0000 7f05a9400640 1 -- 172.21.15.66:0/3329871496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a4066c30 con 0x7f05a40c9f90 2024-03-30T00:50:29.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.855+0000 7f059b400640 1 -- 172.21.15.66:0/3329871496 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f058c0b4040 con 0x7f05a40c9f90 2024-03-30T00:50:29.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.855+0000 7f059b400640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05800666c0 0x7f0580068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:29.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.855+0000 7f059b400640 1 -- 172.21.15.66:0/3329871496 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f058c11e370 con 0x7f05a40c9f90 2024-03-30T00:50:29.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.855+0000 7f05a2800640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05800666c0 0x7f0580068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:29.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.855+0000 7f05a2800640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05800666c0 0x7f0580068b80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f05940017b0 tx=0x7f0594001620 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:29.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:29.859+0000 7f059b400640 1 -- 172.21.15.66:0/3329871496 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f058c0eb8e0 con 0x7f05a40c9f90 2024-03-30T00:50:30.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 -- 172.21.15.66:0/3329871496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05a40cb680 con 0x7f05a40c9f90 2024-03-30T00:50:30.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f059b400640 1 -- 172.21.15.66:0/3329871496 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+12861 (secure 0 0 0) 0x7f058c0eef00 con 0x7f05a40c9f90 2024-03-30T00:50:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:30.031 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":116,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:28.668049+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6837/282507324":"2024-03-31T00:50:20.317728+0000","172.21.15.66:6835/282507324":"2024-03-31T00:50:20.317728+0000","172.21.15.66:6837/2988412199":"2024-03-31T00:49:43.612805+0000","172.21.15.66:6835/2988412199":"2024-03-31T00:49:43.612805+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:50:30.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 -- 172.21.15.66:0/3329871496 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05800666c0 msgr2=0x7f0580068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05800666c0 0x7f0580068b80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f05940017b0 tx=0x7f0594001620 comp rx=0 tx=0).stop 2024-03-30T00:50:30.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 -- 172.21.15.66:0/3329871496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05a40c9f90 msgr2=0x7f05a40b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05a40c9f90 0x7f05a40b0620 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f058c01c820 tx=0x7f0588005800 comp rx=0 tx=0).stop 2024-03-30T00:50:30.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 -- 172.21.15.66:0/3329871496 shutdown_connections 2024-03-30T00:50:30.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05a40c4560 0x7f05a40c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f05800666c0 0x7f0580068b80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05a40c9f90 0x7f05a40b0620 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 --2- 172.21.15.66:0/3329871496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05880047d0 0x7f05a4156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 -- 172.21.15.66:0/3329871496 >> 172.21.15.66:0/3329871496 conn(0x7f05a40d0540 msgr2=0x7f05a40ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:30.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 -- 172.21.15.66:0/3329871496 shutdown_connections 2024-03-30T00:50:30.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.023+0000 7f05a9400640 1 -- 172.21.15.66:0/3329871496 wait complete. 2024-03-30T00:50:30.052 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:50:30.052 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:50:30.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.187+0000 7fb05ae00640 1 Processor -- start 2024-03-30T00:50:30.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.187+0000 7fb05ae00640 1 -- start start 2024-03-30T00:50:30.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.187+0000 7fb05ae00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0540cda90 0x7fb0540ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:30.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.187+0000 7fb05ae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0540cde70 0x7fb0540cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:30.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.187+0000 7fb05ae00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb05415a990 con 0x7fb054159c60 2024-03-30T00:50:30.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.187+0000 7fb05ae00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0540b1b80 con 0x7fb0540cda90 2024-03-30T00:50:30.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.187+0000 7fb05ae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540cd3e0 con 0x7fb0540cde70 2024-03-30T00:50:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb053e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0540cda90 0x7fb0540ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb04b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0540cde70 0x7fb0540cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb053e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0540cda90 0x7fb0540ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:51562/0 (socket says 172.21.15.66:51562) 2024-03-30T00:50:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb053e00640 1 -- 172.21.15.66:0/1596367352 learned_addr learned my addr 172.21.15.66:0/1596367352 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb04b400640 1 -- 172.21.15.66:0/1596367352 >> v2:172.21.15.156:3300/0 conn(0x7fb0540cda90 msgr2=0x7fb0540ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb04b400640 1 --2- 172.21.15.66:0/1596367352 >> v2:172.21.15.156:3300/0 conn(0x7fb0540cda90 0x7fb0540ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb04b400640 1 -- 172.21.15.66:0/1596367352 >> v1:172.21.15.156:6789/0 conn(0x7fb054159c60 legacy=0x7fb05415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb053400640 1 -- 172.21.15.66:0/1596367352 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835900211 0 0) 0x7fb05415a990 con 0x7fb054159c60 2024-03-30T00:50:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb04b400640 1 -- 172.21.15.66:0/1596367352 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540b11e0 con 0x7fb0540cde70 2024-03-30T00:50:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb053e00640 1 --2- 172.21.15.66:0/1596367352 >> v2:172.21.15.156:3300/0 conn(0x7fb0540cda90 0x7fb0540ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb04b400640 1 --2- 172.21.15.66:0/1596367352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0540cde70 0x7fb0540cea60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0097c0 tx=0x7fb03c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=453c3d6956d7c8f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:30.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb053400640 1 -- 172.21.15.66:0/1596367352 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0ab600 con 0x7fb0540cde70 2024-03-30T00:50:30.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb053400640 1 -- 172.21.15.66:0/1596367352 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c0a7050 con 0x7fb0540cde70 2024-03-30T00:50:30.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.191+0000 7fb053400640 1 -- 172.21.15.66:0/1596367352 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0ab980 con 0x7fb0540cde70 2024-03-30T00:50:30.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 -- 172.21.15.66:0/1596367352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0540cde70 msgr2=0x7fb0540cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 --2- 172.21.15.66:0/1596367352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0540cde70 0x7fb0540cea60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0097c0 tx=0x7fb03c01c410 comp rx=0 tx=0).stop 2024-03-30T00:50:30.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 -- 172.21.15.66:0/1596367352 shutdown_connections 2024-03-30T00:50:30.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 --2- 172.21.15.66:0/1596367352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0540cde70 0x7fb0540cea60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 --2- 172.21.15.66:0/1596367352 >> v2:172.21.15.156:3300/0 conn(0x7fb0540cda90 0x7fb0540ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 -- 172.21.15.66:0/1596367352 >> 172.21.15.66:0/1596367352 conn(0x7fb054069f60 msgr2=0x7fb05406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:30.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 -- 172.21.15.66:0/1596367352 shutdown_connections 2024-03-30T00:50:30.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 -- 172.21.15.66:0/1596367352 wait complete. 2024-03-30T00:50:30.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 Processor -- start 2024-03-30T00:50:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 -- start start 2024-03-30T00:50:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0540cda90 0x7fb0540c1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0540cde70 0x7fb0540c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb054159c60 0x7fb0540ab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540c1de0 con 0x7fb054159c60 2024-03-30T00:50:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540c1c10 con 0x7fb0540cda90 2024-03-30T00:50:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb05ae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540719d0 con 0x7fb0540cde70 2024-03-30T00:50:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb053e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0540cda90 0x7fb0540c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb058e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb054159c60 0x7fb0540ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb04b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0540cde70 0x7fb0540c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb053e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0540cda90 0x7fb0540c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:51568/0 (socket says 172.21.15.66:51568) 2024-03-30T00:50:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.195+0000 7fb053e00640 1 -- 172.21.15.66:0/905632937 learned_addr learned my addr 172.21.15.66:0/905632937 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:30.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb053e00640 1 -- 172.21.15.66:0/905632937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0540cde70 msgr2=0x7fb0540c16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb053e00640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0540cde70 0x7fb0540c16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb053e00640 1 -- 172.21.15.66:0/905632937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb054159c60 msgr2=0x7fb0540ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb053e00640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb054159c60 0x7fb0540ab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb053e00640 1 -- 172.21.15.66:0/905632937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04400bd40 con 0x7fb0540cda90 2024-03-30T00:50:30.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb058e00640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb054159c60 0x7fb0540ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:30.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb053e00640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0540cda90 0x7fb0540c1190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb04400e0f0 tx=0x7fb04400e5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb050c00640 1 -- 172.21.15.66:0/905632937 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400b310 con 0x7fb0540cda90 2024-03-30T00:50:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb050c00640 1 -- 172.21.15.66:0/905632937 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb044010030 con 0x7fb0540cda90 2024-03-30T00:50:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb050c00640 1 -- 172.21.15.66:0/905632937 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044009800 con 0x7fb0540cda90 2024-03-30T00:50:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb05ae00640 1 -- 172.21.15.66:0/905632937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c009420 con 0x7fb0540cda90 2024-03-30T00:50:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb05ae00640 1 -- 172.21.15.66:0/905632937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb054156420 con 0x7fb0540cda90 2024-03-30T00:50:30.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb05ae00640 1 -- 172.21.15.66:0/905632937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb054066c30 con 0x7fb0540cda90 2024-03-30T00:50:30.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb050c00640 1 -- 172.21.15.66:0/905632937 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb044009b70 con 0x7fb0540cda90 2024-03-30T00:50:30.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.199+0000 7fb050c00640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb030066690 0x7fb030068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:30.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.203+0000 7fb050c00640 1 -- 172.21.15.66:0/905632937 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fb044086bc0 con 0x7fb0540cda90 2024-03-30T00:50:30.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.203+0000 7fb04b400640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb030066690 0x7fb030068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:30.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.203+0000 7fb04b400640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb030066690 0x7fb030068b50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0ad000 tx=0x7fb03c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:30.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.207+0000 7fb050c00640 1 -- 172.21.15.66:0/905632937 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb044054130 con 0x7fb0540cda90 2024-03-30T00:50:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.363+0000 7fb05ae00640 1 -- 172.21.15.66:0/905632937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb05406cf30 con 0x7fb0540cda90 2024-03-30T00:50:30.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb050c00640 1 -- 172.21.15.66:0/905632937 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb044057750 con 0x7fb0540cda90 2024-03-30T00:50:30.374 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:50:30.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 -- 172.21.15.66:0/905632937 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb030066690 msgr2=0x7fb030068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb030066690 0x7fb030068b50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0ad000 tx=0x7fb03c008040 comp rx=0 tx=0).stop 2024-03-30T00:50:30.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 -- 172.21.15.66:0/905632937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0540cda90 msgr2=0x7fb0540c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0540cda90 0x7fb0540c1190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb04400e0f0 tx=0x7fb04400e5b0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 -- 172.21.15.66:0/905632937 shutdown_connections 2024-03-30T00:50:30.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb030066690 0x7fb030068b50 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb054159c60 0x7fb0540ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0540cde70 0x7fb0540c16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 --2- 172.21.15.66:0/905632937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0540cda90 0x7fb0540c1190 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 -- 172.21.15.66:0/905632937 >> 172.21.15.66:0/905632937 conn(0x7fb054069f60 msgr2=0x7fb05415ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:30.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 -- 172.21.15.66:0/905632937 shutdown_connections 2024-03-30T00:50:30.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.367+0000 7fb05ae00640 1 -- 172.21.15.66:0/905632937 wait complete. 2024-03-30T00:50:30.396 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-03-30T00:50:30.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.527+0000 7f523f400640 1 Processor -- start 2024-03-30T00:50:30.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.527+0000 7f523f400640 1 -- start start 2024-03-30T00:50:30.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f523f400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f52380cee50 0x7f52380cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:30.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f523f400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52380cbb90 con 0x7f52380af420 2024-03-30T00:50:30.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f523f400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52380ce1b0 con 0x7f523815de30 2024-03-30T00:50:30.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f523f400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52380afa10 con 0x7f52380cee50 2024-03-30T00:50:30.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f523d400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f52380af420 0x7f523806e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40888/0 (socket says 172.21.15.66:40888) 2024-03-30T00:50:30.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f523d400640 1 -- 172.21.15.66:0/2937650825 learned_addr learned my addr 172.21.15.66:0/2937650825 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:30.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f5237e00640 1 --2- 172.21.15.66:0/2937650825 >> v2:172.21.15.156:3300/0 conn(0x7f52380cee50 0x7f52380cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:30.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f5237400640 1 -- 172.21.15.66:0/2937650825 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019207180 0 0) 0x7f52380cbb90 con 0x7f52380af420 2024-03-30T00:50:30.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f5237400640 1 -- 172.21.15.66:0/2937650825 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5220003530 con 0x7f52380af420 2024-03-30T00:50:30.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f5237400640 1 -- 172.21.15.66:0/2937650825 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481924711 0 0) 0x7f52380ce1b0 con 0x7f523815de30 2024-03-30T00:50:30.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f5237400640 1 -- 172.21.15.66:0/2937650825 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52380cbb90 con 0x7f523815de30 2024-03-30T00:50:30.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f5237400640 1 -- 172.21.15.66:0/2937650825 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2663939200 0 0) 0x7f5220003530 con 0x7f52380af420 2024-03-30T00:50:30.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f5237e00640 1 -- 172.21.15.66:0/2937650825 >> v1:172.21.15.156:6789/0 conn(0x7f523815de30 legacy=0x7f52380ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f5237e00640 1 -- 172.21.15.66:0/2937650825 >> v1:172.21.15.66:6789/0 conn(0x7f52380af420 legacy=0x7f523806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.531+0000 7f5237e00640 1 -- 172.21.15.66:0/2937650825 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380cf430 con 0x7f52380cee50 2024-03-30T00:50:30.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f5237e00640 1 --2- 172.21.15.66:0/2937650825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52380cee50 0x7f52380cc040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f52240097c0 tx=0x7f522401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=988a5cd965848b62 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:30.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f5237400640 1 -- 172.21.15.66:0/2937650825 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230001a00 con 0x7f52380cee50 2024-03-30T00:50:30.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f5237400640 1 -- 172.21.15.66:0/2937650825 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52240a7050 con 0x7f52380cee50 2024-03-30T00:50:30.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f5237400640 1 -- 172.21.15.66:0/2937650825 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240ab620 con 0x7f52380cee50 2024-03-30T00:50:30.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f523f400640 1 -- 172.21.15.66:0/2937650825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52380cee50 msgr2=0x7f52380cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f523f400640 1 --2- 172.21.15.66:0/2937650825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52380cee50 0x7f52380cc040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f52240097c0 tx=0x7f522401c3c0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f523f400640 1 -- 172.21.15.66:0/2937650825 shutdown_connections 2024-03-30T00:50:30.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f523f400640 1 --2- 172.21.15.66:0/2937650825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52380cee50 0x7f52380cc040 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f523f400640 1 -- 172.21.15.66:0/2937650825 >> 172.21.15.66:0/2937650825 conn(0x7f52380d0540 msgr2=0x7f52380d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:30.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f523f400640 1 -- 172.21.15.66:0/2937650825 shutdown_connections 2024-03-30T00:50:30.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f523f400640 1 -- 172.21.15.66:0/2937650825 wait complete. 2024-03-30T00:50:30.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f523f400640 1 Processor -- start 2024-03-30T00:50:30.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.535+0000 7f523f400640 1 -- start start 2024-03-30T00:50:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52380af420 0x7f52380ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52380cee50 0x7f5238165c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f523815de30 0x7f5238157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523806c830 con 0x7f52380af420 2024-03-30T00:50:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523f400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523806bc20 con 0x7f523815de30 2024-03-30T00:50:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523806bf00 con 0x7f52380cee50 2024-03-30T00:50:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52380af420 0x7f52380ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52380af420 0x7f52380ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48952/0 (socket says 172.21.15.66:48952) 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523ca00640 1 -- 172.21.15.66:0/3359903415 learned_addr learned my addr 172.21.15.66:0/3359903415 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f5237e00640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52380cee50 0x7f5238165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523d400640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f523815de30 0x7f5238157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523ca00640 1 -- 172.21.15.66:0/3359903415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52380cee50 msgr2=0x7f5238165c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523ca00640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52380cee50 0x7f5238165c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523ca00640 1 -- 172.21.15.66:0/3359903415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f523815de30 msgr2=0x7f5238157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523ca00640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f523815de30 0x7f5238157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523ca00640 1 -- 172.21.15.66:0/3359903415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224009420 con 0x7f52380af420 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523d400640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f523815de30 0x7f5238157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523ca00640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52380af420 0x7f52380ae5b0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c430 tx=0x7f522c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f5234c00640 1 -- 172.21.15.66:0/3359903415 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c0052d0 con 0x7f52380af420 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f5234c00640 1 -- 172.21.15.66:0/3359903415 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c005450 con 0x7f52380af420 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523f400640 1 -- 172.21.15.66:0/3359903415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523806c1e0 con 0x7f52380af420 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f5234c00640 1 -- 172.21.15.66:0/3359903415 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c003840 con 0x7f52380af420 2024-03-30T00:50:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523f400640 1 -- 172.21.15.66:0/3359903415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523806b8b0 con 0x7f52380af420 2024-03-30T00:50:30.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f523f400640 1 -- 172.21.15.66:0/3359903415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52380cdf60 con 0x7f52380af420 2024-03-30T00:50:30.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.539+0000 7f5234c00640 1 -- 172.21.15.66:0/3359903415 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f522c0039a0 con 0x7f52380af420 2024-03-30T00:50:30.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.543+0000 7f5234c00640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5218066690 0x7f5218068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:30.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.543+0000 7f5234c00640 1 -- 172.21.15.66:0/3359903415 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f522c083aa0 con 0x7f52380af420 2024-03-30T00:50:30.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.543+0000 7f5237e00640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5218066690 0x7f5218068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:30.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.543+0000 7f5237e00640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5218066690 0x7f5218068b50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f52240ad000 tx=0x7f52240a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:30.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.543+0000 7f5234c00640 1 -- 172.21.15.66:0/3359903415 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f522c051000 con 0x7f52380af420 2024-03-30T00:50:30.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:30.763+0000 7f523f400640 1 -- 172.21.15.66:0/3359903415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f5238066c30 con 0x7f52380af420 2024-03-30T00:50:31.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.687+0000 7f5234c00640 1 -- 172.21.15.66:0/3359903415 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v508) v1 ==== 147+0+0 (secure 0 0 0) 0x7f522c0170d0 con 0x7f52380af420 2024-03-30T00:50:31.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 -- 172.21.15.66:0/3359903415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5218066690 msgr2=0x7f5218068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:31.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5218066690 0x7f5218068b50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f52240ad000 tx=0x7f52240a6000 comp rx=0 tx=0).stop 2024-03-30T00:50:31.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 -- 172.21.15.66:0/3359903415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52380af420 msgr2=0x7f52380ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:31.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52380af420 0x7f52380ae5b0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c430 tx=0x7f522c00c8f0 comp rx=0 tx=0).stop 2024-03-30T00:50:31.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 -- 172.21.15.66:0/3359903415 shutdown_connections 2024-03-30T00:50:31.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f523815de30 0x7f5238157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:31.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52380cee50 0x7f5238165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:31.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5218066690 0x7f5218068b50 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:31.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 --2- 172.21.15.66:0/3359903415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52380af420 0x7f52380ae5b0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:31.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 -- 172.21.15.66:0/3359903415 >> 172.21.15.66:0/3359903415 conn(0x7f52380d0540 msgr2=0x7f52380ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:31.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 -- 172.21.15.66:0/3359903415 shutdown_connections 2024-03-30T00:50:31.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.691+0000 7f523f400640 1 -- 172.21.15.66:0/3359903415 wait complete. 2024-03-30T00:50:31.723 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-30T00:50:31.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.863+0000 7f03c5200640 1 Processor -- start 2024-03-30T00:50:31.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.863+0000 7f03c5200640 1 -- start start 2024-03-30T00:50:31.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.863+0000 7f03c5200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03c0099130 0x7f03c00cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:31.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.863+0000 7f03c5200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c00cac60 con 0x7f03c0160150 2024-03-30T00:50:31.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.863+0000 7f03c5200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c00af780 con 0x7f03c00d00f0 2024-03-30T00:50:31.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.863+0000 7f03c5200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00afbc0 con 0x7f03c0099130 2024-03-30T00:50:31.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03bde00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f03c0160150 0x7f03c0098a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40892/0 (socket says 172.21.15.66:40892) 2024-03-30T00:50:31.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03bde00640 1 -- 172.21.15.66:0/4034580630 learned_addr learned my addr 172.21.15.66:0/4034580630 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:31.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03be800640 1 --2- 172.21.15.66:0/4034580630 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03c0099130 0x7f03c00cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:31.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03bd400640 1 -- 172.21.15.66:0/4034580630 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556274896 0 0) 0x7f03c00cac60 con 0x7f03c0160150 2024-03-30T00:50:31.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03bd400640 1 -- 172.21.15.66:0/4034580630 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a4003530 con 0x7f03c0160150 2024-03-30T00:50:31.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03bd400640 1 -- 172.21.15.66:0/4034580630 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023340088 0 0) 0x7f03c00af780 con 0x7f03c00d00f0 2024-03-30T00:50:31.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03bd400640 1 -- 172.21.15.66:0/4034580630 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03c00cac60 con 0x7f03c00d00f0 2024-03-30T00:50:31.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03be800640 1 -- 172.21.15.66:0/4034580630 >> v1:172.21.15.156:6789/0 conn(0x7f03c00d00f0 legacy=0x7f03c00cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:31.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03be800640 1 -- 172.21.15.66:0/4034580630 >> v1:172.21.15.66:6789/0 conn(0x7f03c0160150 legacy=0x7f03c0098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:31.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03be800640 1 -- 172.21.15.66:0/4034580630 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c00cf2c0 con 0x7f03c0099130 2024-03-30T00:50:31.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03be800640 1 --2- 172.21.15.66:0/4034580630 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03c0099130 0x7f03c00cb800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009dc0 tx=0x7f03c00cc550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e74d1f2f53459b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:31.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.867+0000 7f03bd400640 1 -- 172.21.15.66:0/4034580630 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b80a5a20 con 0x7f03c0099130 2024-03-30T00:50:31.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03bd400640 1 -- 172.21.15.66:0/4034580630 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b80a5b80 con 0x7f03c0099130 2024-03-30T00:50:31.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03bd400640 1 -- 172.21.15.66:0/4034580630 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b80ad440 con 0x7f03c0099130 2024-03-30T00:50:31.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 -- 172.21.15.66:0/4034580630 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03c0099130 msgr2=0x7f03c00cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:31.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 --2- 172.21.15.66:0/4034580630 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03c0099130 0x7f03c00cb800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009dc0 tx=0x7f03c00cc550 comp rx=0 tx=0).stop 2024-03-30T00:50:31.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 -- 172.21.15.66:0/4034580630 shutdown_connections 2024-03-30T00:50:31.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 --2- 172.21.15.66:0/4034580630 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03c0099130 0x7f03c00cb800 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:31.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 -- 172.21.15.66:0/4034580630 >> 172.21.15.66:0/4034580630 conn(0x7f03c00b2100 msgr2=0x7f03c00b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:31.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 -- 172.21.15.66:0/4034580630 shutdown_connections 2024-03-30T00:50:31.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 -- 172.21.15.66:0/4034580630 wait complete. 2024-03-30T00:50:31.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 Processor -- start 2024-03-30T00:50:31.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 -- start start 2024-03-30T00:50:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03c0099130 0x7f03c014ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03c00d00f0 0x7f03c014cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03c0160150 0x7f03c015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c014b080 con 0x7f03c0099130 2024-03-30T00:50:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c014cb00 con 0x7f03c0160150 2024-03-30T00:50:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03c5200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c014da20 con 0x7f03c00d00f0 2024-03-30T00:50:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03bde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03c0099130 0x7f03c014ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03bde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03c0099130 0x7f03c014ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48966/0 (socket says 172.21.15.66:48966) 2024-03-30T00:50:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03bde00640 1 -- 172.21.15.66:0/1162500754 learned_addr learned my addr 172.21.15.66:0/1162500754 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03b7e00640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03c00d00f0 0x7f03c014cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03be800640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03c0160150 0x7f03c015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03bde00640 1 -- 172.21.15.66:0/1162500754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03c00d00f0 msgr2=0x7f03c014cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03bde00640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03c00d00f0 0x7f03c014cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03bde00640 1 -- 172.21.15.66:0/1162500754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03c0160150 msgr2=0x7f03c015efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03bde00640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03c0160150 0x7f03c015efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.871+0000 7f03bde00640 1 -- 172.21.15.66:0/1162500754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b8009a70 con 0x7f03c0099130 2024-03-30T00:50:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.875+0000 7f03bde00640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03c0099130 0x7f03c014ab40 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c4d0 tx=0x7f03b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.875+0000 7f03b6000640 1 -- 172.21.15.66:0/1162500754 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0005cf0 con 0x7f03c0099130 2024-03-30T00:50:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.875+0000 7f03b6000640 1 -- 172.21.15.66:0/1162500754 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b0005e50 con 0x7f03c0099130 2024-03-30T00:50:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.875+0000 7f03b6000640 1 -- 172.21.15.66:0/1162500754 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b00072b0 con 0x7f03c0099130 2024-03-30T00:50:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.875+0000 7f03c5200640 1 -- 172.21.15.66:0/1162500754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c014bc10 con 0x7f03c0099130 2024-03-30T00:50:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.875+0000 7f03c5200640 1 -- 172.21.15.66:0/1162500754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c014c1d0 con 0x7f03c0099130 2024-03-30T00:50:31.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.875+0000 7f03c5200640 1 -- 172.21.15.66:0/1162500754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c0066c30 con 0x7f03c0099130 2024-03-30T00:50:31.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.875+0000 7f03b6000640 1 -- 172.21.15.66:0/1162500754 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f03b0007410 con 0x7f03c0099130 2024-03-30T00:50:31.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.875+0000 7f03b6000640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f039c0666e0 0x7f039c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.875+0000 7f03b6000640 1 -- 172.21.15.66:0/1162500754 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f03b00835c0 con 0x7f03c0099130 2024-03-30T00:50:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.875+0000 7f03b7e00640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f039c0666e0 0x7f039c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.879+0000 7f03b7e00640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f039c0666e0 0x7f039c068ba0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f03c00cbca0 tx=0x7f03a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:31.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:31.879+0000 7f03b6000640 1 -- 172.21.15.66:0/1162500754 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f03b0050b70 con 0x7f03c0099130 2024-03-30T00:50:32.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.047+0000 7f03c5200640 1 -- 172.21.15.66:0/1162500754 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f03c01518c0 con 0x7f03c0099130 2024-03-30T00:50:32.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03b6000640 1 -- 172.21.15.66:0/1162500754 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f03b0054190 con 0x7f03c0099130 2024-03-30T00:50:32.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 -- 172.21.15.66:0/1162500754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f039c0666e0 msgr2=0x7f039c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f039c0666e0 0x7f039c068ba0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f03c00cbca0 tx=0x7f03a8009040 comp rx=0 tx=0).stop 2024-03-30T00:50:32.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 -- 172.21.15.66:0/1162500754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03c0099130 msgr2=0x7f03c014ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03c0099130 0x7f03c014ab40 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c4d0 tx=0x7f03b000c990 comp rx=0 tx=0).stop 2024-03-30T00:50:32.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 -- 172.21.15.66:0/1162500754 shutdown_connections 2024-03-30T00:50:32.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03c0160150 0x7f03c015efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03c00d00f0 0x7f03c014cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f039c0666e0 0x7f039c068ba0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 --2- 172.21.15.66:0/1162500754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03c0099130 0x7f03c014ab40 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 -- 172.21.15.66:0/1162500754 >> 172.21.15.66:0/1162500754 conn(0x7f03c00b2100 msgr2=0x7f03c00cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:32.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 -- 172.21.15.66:0/1162500754 shutdown_connections 2024-03-30T00:50:32.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.055+0000 7f03c5200640 1 -- 172.21.15.66:0/1162500754 wait complete. 2024-03-30T00:50:32.083 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:50:32.084 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:50:32.180 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:50:32.224 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:50:32.224 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:50:32.224 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:50:32.233 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:50:32.233 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:50:32.233 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:50:32.273 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:50:32.273 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:50:32.273 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:50:32.285 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:50:32.285 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:50:32.285 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:50:32.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:50:32.325 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-30T00:50:32.325 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:50:32.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.459+0000 7f9668c00640 1 Processor -- start 2024-03-30T00:50:32.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.459+0000 7f9668c00640 1 -- start start 2024-03-30T00:50:32.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.459+0000 7f9668c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9664157060 0x7f966406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:32.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.459+0000 7f9668c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966406dc60 con 0x7f96641610c0 2024-03-30T00:50:32.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.459+0000 7f9668c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966406df40 con 0x7f9664160bc0 2024-03-30T00:50:32.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.459+0000 7f9668c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9664157440 con 0x7f9664157060 2024-03-30T00:50:32.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.463+0000 7f9661e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9664157060 0x7f966406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:32.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.463+0000 7f9661e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9664157060 0x7f966406d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48972/0 (socket says 172.21.15.66:48972) 2024-03-30T00:50:32.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.463+0000 7f9661e00640 1 -- 172.21.15.66:0/490184121 learned_addr learned my addr 172.21.15.66:0/490184121 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:32.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.463+0000 7f965be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f96641610c0 0x7f96641604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40906/0 (socket says 172.21.15.66:40906) 2024-03-30T00:50:32.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.463+0000 7f9661e00640 1 -- 172.21.15.66:0/490184121 >> v1:172.21.15.156:6789/0 conn(0x7f9664160bc0 legacy=0x7f96640abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:32.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.463+0000 7f9661e00640 1 -- 172.21.15.66:0/490184121 >> v1:172.21.15.66:6789/0 conn(0x7f96641610c0 legacy=0x7f96641604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.463+0000 7f9661e00640 1 -- 172.21.15.66:0/490184121 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966406e220 con 0x7f9664157060 2024-03-30T00:50:32.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.463+0000 7f9661e00640 1 --2- 172.21.15.66:0/490184121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664157060 0x7f966406d310 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f9654009870 tx=0x7f965401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ab5ac24afb1e18 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:32.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.463+0000 7f9661400640 1 -- 172.21.15.66:0/490184121 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966406da60 con 0x7f9664157060 2024-03-30T00:50:32.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.463+0000 7f9661400640 1 -- 172.21.15.66:0/490184121 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96540a7050 con 0x7f9664157060 2024-03-30T00:50:32.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.463+0000 7f9661400640 1 -- 172.21.15.66:0/490184121 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654004490 con 0x7f9664157060 2024-03-30T00:50:32.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 -- 172.21.15.66:0/490184121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664157060 msgr2=0x7f966406d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 --2- 172.21.15.66:0/490184121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664157060 0x7f966406d310 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f9654009870 tx=0x7f965401c440 comp rx=0 tx=0).stop 2024-03-30T00:50:32.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 -- 172.21.15.66:0/490184121 shutdown_connections 2024-03-30T00:50:32.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 --2- 172.21.15.66:0/490184121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664157060 0x7f966406d310 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 -- 172.21.15.66:0/490184121 >> 172.21.15.66:0/490184121 conn(0x7f9664071750 msgr2=0x7f9664069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:32.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 -- 172.21.15.66:0/490184121 shutdown_connections 2024-03-30T00:50:32.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 -- 172.21.15.66:0/490184121 wait complete. 2024-03-30T00:50:32.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 Processor -- start 2024-03-30T00:50:32.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 -- start start 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664157060 0x7f966414e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9664160bc0 0x7f9664137480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96641610c0 0x7f9664137d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966415ded0 con 0x7f9664157060 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966415e120 con 0x7f9664160bc0 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9668c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966415e400 con 0x7f96641610c0 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9661e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664157060 0x7f966414e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9661e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664157060 0x7f966414e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48980/0 (socket says 172.21.15.66:48980) 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9661e00640 1 -- 172.21.15.66:0/2372152620 learned_addr learned my addr 172.21.15.66:0/2372152620 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f965be00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9664160bc0 0x7f9664137480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9662800640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96641610c0 0x7f9664137d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9661e00640 1 -- 172.21.15.66:0/2372152620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96641610c0 msgr2=0x7f9664137d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9661e00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96641610c0 0x7f9664137d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9661e00640 1 -- 172.21.15.66:0/2372152620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9664160bc0 msgr2=0x7f9664137480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9661e00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9664160bc0 0x7f9664137480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9661e00640 1 -- 172.21.15.66:0/2372152620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540094d0 con 0x7f9664157060 2024-03-30T00:50:32.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f9661e00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664157060 0x7f966414e3c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f9654019050 tx=0x7f96540191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:32.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f965a000640 1 -- 172.21.15.66:0/2372152620 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654003bf0 con 0x7f9664157060 2024-03-30T00:50:32.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.467+0000 7f965a000640 1 -- 172.21.15.66:0/2372152620 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9654003d50 con 0x7f9664157060 2024-03-30T00:50:32.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.471+0000 7f965a000640 1 -- 172.21.15.66:0/2372152620 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540ab660 con 0x7f9664157060 2024-03-30T00:50:32.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.471+0000 7f9668c00640 1 -- 172.21.15.66:0/2372152620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966415e6e0 con 0x7f9664157060 2024-03-30T00:50:32.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.471+0000 7f9668c00640 1 -- 172.21.15.66:0/2372152620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966414e900 con 0x7f9664157060 2024-03-30T00:50:32.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.471+0000 7f9668c00640 1 -- 172.21.15.66:0/2372152620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9664066c30 con 0x7f9664157060 2024-03-30T00:50:32.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.471+0000 7f965a000640 1 -- 172.21.15.66:0/2372152620 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f96540b2050 con 0x7f9664157060 2024-03-30T00:50:32.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.471+0000 7f965a000640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96400666c0 0x7f9640068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:32.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.471+0000 7f965a000640 1 -- 172.21.15.66:0/2372152620 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f965411e590 con 0x7f9664157060 2024-03-30T00:50:32.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.471+0000 7f965be00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96400666c0 0x7f9640068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:32.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.471+0000 7f965be00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96400666c0 0x7f9640068b80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f966415ff50 tx=0x7f964c007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:32.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.475+0000 7f965a000640 1 -- 172.21.15.66:0/2372152620 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f96540eba50 con 0x7f9664157060 2024-03-30T00:50:32.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 -- 172.21.15.66:0/2372152620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f966415ff80 con 0x7f9664157060 2024-03-30T00:50:32.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f965a000640 1 -- 172.21.15.66:0/2372152620 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f96540b0de0 con 0x7f9664157060 2024-03-30T00:50:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:32.679 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":89,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/942755245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":942755245},{"type":"v1","addr":"172.21.15.156:6835","nonce":942755245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1902721840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1902721840},{"type":"v1","addr":"172.21.15.156:6834","nonce":1902721840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/1754707678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1754707678},{"type":"v1","addr":"172.21.15.66:6836","nonce":1754707678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3688845855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":3688845855},{"type":"v1","addr":"172.21.15.66:6837","nonce":3688845855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[]} 2024-03-30T00:50:32.679 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 89 2024-03-30T00:50:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 -- 172.21.15.66:0/2372152620 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96400666c0 msgr2=0x7f9640068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96400666c0 0x7f9640068b80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f966415ff50 tx=0x7f964c007cb0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 -- 172.21.15.66:0/2372152620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664157060 msgr2=0x7f966414e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664157060 0x7f966414e3c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f9654019050 tx=0x7f96540191e0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 -- 172.21.15.66:0/2372152620 shutdown_connections 2024-03-30T00:50:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96641610c0 0x7f9664137d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9664160bc0 0x7f9664137480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f96400666c0 0x7f9640068b80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 --2- 172.21.15.66:0/2372152620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9664157060 0x7f966414e3c0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 -- 172.21.15.66:0/2372152620 >> 172.21.15.66:0/2372152620 conn(0x7f9664071750 msgr2=0x7f9664156840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:32.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 -- 172.21.15.66:0/2372152620 shutdown_connections 2024-03-30T00:50:32.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.671+0000 7f9668c00640 1 -- 172.21.15.66:0/2372152620 wait complete. 2024-03-30T00:50:32.701 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T00:50:32.701 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-30T00:50:32.701 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T00:50:32.701 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:50:32.702 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T00:50:32.702 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-30T00:50:32.702 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T00:50:32.702 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:50:32.702 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T00:50:32.702 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-30T00:50:32.703 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T00:50:32.703 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:50:32.703 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T00:50:32.703 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-30T00:50:32.703 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T00:50:32.703 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:50:32.713 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T00:50:32.713 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T00:50:32.715 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T00:50:32.715 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T00:50:32.716 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T00:50:32.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T00:50:32.717 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T00:50:32.717 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T00:50:32.762 INFO:tasks.ceph.mds.b:Started 2024-03-30T00:50:32.763 DEBUG:teuthology.parallel:result is None 2024-03-30T00:50:32.763 INFO:tasks.ceph.mds.d:Started 2024-03-30T00:50:32.764 DEBUG:teuthology.parallel:result is None 2024-03-30T00:50:32.764 INFO:tasks.ceph.mds.a:Started 2024-03-30T00:50:32.765 DEBUG:teuthology.parallel:result is None 2024-03-30T00:50:32.767 INFO:tasks.ceph.mds.c:Started 2024-03-30T00:50:32.767 DEBUG:teuthology.parallel:result is None 2024-03-30T00:50:32.767 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-30T00:50:32.849 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2024-03-30T00:50:32.851 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:50:32.851 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2024-03-30T00:50:32.852 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T00:50:32.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.907+0000 7fcc61400640 1 Processor -- start 2024-03-30T00:50:32.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.907+0000 7fcc61400640 1 -- start start 2024-03-30T00:50:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc61400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcc5c0cec30 0x7fcc5c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc61400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc5c0cde70 0x7fcc5c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc61400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5c15a6e0 con 0x7fcc5c0cda90 2024-03-30T00:50:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc61400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5c0b1b80 con 0x7fcc5c0cec30 2024-03-30T00:50:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc61400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0cd3e0 con 0x7fcc5c0cde70 2024-03-30T00:50:32.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc51e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcc5c0cec30 0x7fcc5c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc5b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc5c0cde70 0x7fcc5c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc51e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcc5c0cec30 0x7fcc5c159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:51650/0 (socket says 172.21.15.66:51650) 2024-03-30T00:50:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc51e00640 1 -- 172.21.15.66:0/3132461504 learned_addr learned my addr 172.21.15.66:0/3132461504 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc5a800640 1 --1- 172.21.15.66:0/3132461504 >> v1:172.21.15.156:6789/0 conn(0x7fcc5c0cda90 0x7fcc5c0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:60544/0 (socket says 172.21.15.66:60544) 2024-03-30T00:50:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc51e00640 1 -- 172.21.15.66:0/3132461504 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc5c0cde70 msgr2=0x7fcc5c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc51e00640 1 --2- 172.21.15.66:0/3132461504 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc5c0cde70 0x7fcc5c15a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc51e00640 1 -- 172.21.15.66:0/3132461504 >> v1:172.21.15.156:6789/0 conn(0x7fcc5c0cda90 legacy=0x7fcc5c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc51e00640 1 -- 172.21.15.66:0/3132461504 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c15a990 con 0x7fcc5c0cec30 2024-03-30T00:50:32.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc51e00640 1 --2- 172.21.15.66:0/3132461504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc5c0cec30 0x7fcc5c159c60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400bb10 tx=0x7fcc4401eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c0731fb08a8a46d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:32.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc59e00640 1 -- 172.21.15.66:0/3132461504 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440a7c70 con 0x7fcc5c0cec30 2024-03-30T00:50:32.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.911+0000 7fcc59e00640 1 -- 172.21.15.66:0/3132461504 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc440a7dd0 con 0x7fcc5c0cec30 2024-03-30T00:50:32.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc59e00640 1 -- 172.21.15.66:0/3132461504 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440af660 con 0x7fcc5c0cec30 2024-03-30T00:50:32.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 -- 172.21.15.66:0/3132461504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc5c0cec30 msgr2=0x7fcc5c159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 --2- 172.21.15.66:0/3132461504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc5c0cec30 0x7fcc5c159c60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400bb10 tx=0x7fcc4401eba0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 -- 172.21.15.66:0/3132461504 shutdown_connections 2024-03-30T00:50:32.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 --2- 172.21.15.66:0/3132461504 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc5c0cde70 0x7fcc5c15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 --2- 172.21.15.66:0/3132461504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc5c0cec30 0x7fcc5c159c60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 -- 172.21.15.66:0/3132461504 >> 172.21.15.66:0/3132461504 conn(0x7fcc5c069f60 msgr2=0x7fcc5c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:32.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 -- 172.21.15.66:0/3132461504 shutdown_connections 2024-03-30T00:50:32.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 -- 172.21.15.66:0/3132461504 wait complete. 2024-03-30T00:50:32.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 Processor -- start 2024-03-30T00:50:32.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 -- start start 2024-03-30T00:50:32.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc5c0cda90 0x7fcc5c0c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:32.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc5c0cde70 0x7fcc5c158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:32.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc5c0cec30 0x7fcc5c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:32.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c157670 con 0x7fcc5c0cde70 2024-03-30T00:50:32.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c158b80 con 0x7fcc5c0cec30 2024-03-30T00:50:32.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc61400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c156260 con 0x7fcc5c0cda90 2024-03-30T00:50:32.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc51e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc5c0cde70 0x7fcc5c158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:32.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc51e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc5c0cde70 0x7fcc5c158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49020/0 (socket says 172.21.15.66:49020) 2024-03-30T00:50:32.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc51e00640 1 -- 172.21.15.66:0/1265824028 learned_addr learned my addr 172.21.15.66:0/1265824028 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:32.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc5a800640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc5c0cda90 0x7fcc5c0c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:32.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc5b200640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc5c0cec30 0x7fcc5c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:32.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc51e00640 1 -- 172.21.15.66:0/1265824028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc5c0cda90 msgr2=0x7fcc5c0c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc51e00640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc5c0cda90 0x7fcc5c0c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc51e00640 1 -- 172.21.15.66:0/1265824028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc5c0cec30 msgr2=0x7fcc5c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:32.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.915+0000 7fcc51e00640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc5c0cec30 0x7fcc5c159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:32.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc51e00640 1 -- 172.21.15.66:0/1265824028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4400b770 con 0x7fcc5c0cde70 2024-03-30T00:50:32.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc5b200640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc5c0cec30 0x7fcc5c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:32.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc51e00640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc5c0cde70 0x7fcc5c158640 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400b2c0 tx=0x7fcc4401b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:32.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc53400640 1 -- 172.21.15.66:0/1265824028 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440be070 con 0x7fcc5c0cde70 2024-03-30T00:50:32.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc53400640 1 -- 172.21.15.66:0/1265824028 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc440afb70 con 0x7fcc5c0cde70 2024-03-30T00:50:32.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc61400640 1 -- 172.21.15.66:0/1265824028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c157880 con 0x7fcc5c0cde70 2024-03-30T00:50:32.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc53400640 1 -- 172.21.15.66:0/1265824028 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44006ee0 con 0x7fcc5c0cde70 2024-03-30T00:50:32.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc61400640 1 -- 172.21.15.66:0/1265824028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5c157e40 con 0x7fcc5c0cde70 2024-03-30T00:50:32.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc61400640 1 -- 172.21.15.66:0/1265824028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5c066c30 con 0x7fcc5c0cde70 2024-03-30T00:50:32.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc53400640 1 -- 172.21.15.66:0/1265824028 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcc440b6040 con 0x7fcc5c0cde70 2024-03-30T00:50:32.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc53400640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcc380666c0 0x7fcc38068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:32.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.919+0000 7fcc53400640 1 -- 172.21.15.66:0/1265824028 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fcc44120680 con 0x7fcc5c0cde70 2024-03-30T00:50:32.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.923+0000 7fcc5a800640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcc380666c0 0x7fcc38068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:32.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.923+0000 7fcc5a800640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcc380666c0 0x7fcc38068b80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c000f70 tx=0x7fcc4c007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:32.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:32.927+0000 7fcc53400640 1 -- 172.21.15.66:0/1265824028 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcc440edab0 con 0x7fcc5c0cde70 2024-03-30T00:50:33.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.099+0000 7fcc61400640 1 -- 172.21.15.66:0/1265824028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcc5c0b3170 con 0x7fcc5c0cde70 2024-03-30T00:50:33.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc53400640 1 -- 172.21.15.66:0/1265824028 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcc440f10d0 con 0x7fcc5c0cde70 2024-03-30T00:50:33.883 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-03-30T00:50:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 -- 172.21.15.66:0/1265824028 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcc380666c0 msgr2=0x7fcc38068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcc380666c0 0x7fcc38068b80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c000f70 tx=0x7fcc4c007cb0 comp rx=0 tx=0).stop 2024-03-30T00:50:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 -- 172.21.15.66:0/1265824028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc5c0cde70 msgr2=0x7fcc5c158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc5c0cde70 0x7fcc5c158640 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400b2c0 tx=0x7fcc4401b940 comp rx=0 tx=0).stop 2024-03-30T00:50:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 -- 172.21.15.66:0/1265824028 shutdown_connections 2024-03-30T00:50:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc5c0cec30 0x7fcc5c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcc380666c0 0x7fcc38068b80 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc5c0cde70 0x7fcc5c158640 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 --2- 172.21.15.66:0/1265824028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc5c0cda90 0x7fcc5c0c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 -- 172.21.15.66:0/1265824028 >> 172.21.15.66:0/1265824028 conn(0x7fcc5c069f60 msgr2=0x7fcc5c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:33.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 -- 172.21.15.66:0/1265824028 shutdown_connections 2024-03-30T00:50:33.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:33.875+0000 7fcc61400640 1 -- 172.21.15.66:0/1265824028 wait complete. 2024-03-30T00:50:33.905 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-30T00:50:34.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08eb600640 1 Processor -- start 2024-03-30T00:50:34.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08eb600640 1 -- start start 2024-03-30T00:50:34.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08eb600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e415de30 0x7f08e40cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08eb600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f08e40af420 0x7f08e40d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08eb600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e40cbb90 con 0x7f08e40ce740 2024-03-30T00:50:34.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08eb600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e40ce1b0 con 0x7f08e40af420 2024-03-30T00:50:34.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08eb600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40afa10 con 0x7f08e415de30 2024-03-30T00:50:34.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e9600640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f08e40ce740 0x7f08e406e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40928/0 (socket says 172.21.15.66:40928) 2024-03-30T00:50:34.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e9600640 1 -- 172.21.15.66:0/2603337136 learned_addr learned my addr 172.21.15.66:0/2603337136 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:34.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3e00640 1 --2- 172.21.15.66:0/2603337136 >> v2:172.21.15.156:3300/0 conn(0x7f08e40af420 0x7f08e40d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e8c00640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e415de30 0x7f08e40cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136024715 0 0) 0x7f08e40cbb90 con 0x7f08e40ce740 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08cc003530 con 0x7f08e40ce740 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2697366759 0 0) 0x7f08cc003530 con 0x7f08e40ce740 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e415de30 msgr2=0x7f08e40cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e415de30 0x7f08e40cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 >> v2:172.21.15.156:3300/0 conn(0x7f08e40af420 msgr2=0x7f08e40d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 --2- 172.21.15.66:0/2603337136 >> v2:172.21.15.156:3300/0 conn(0x7f08e40af420 0x7f08e40d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e40cf430 con 0x7f08e40ce740 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f08dc001a00 con 0x7f08e40ce740 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 >> v1:172.21.15.66:6789/0 conn(0x7f08e40ce740 legacy=0x7f08e406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e415de30 0x7f08cc003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08cc003f00 0x7f08cc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc004d10 0x7f08cc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40ce1b0 con 0x7f08cc004d10 2024-03-30T00:50:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40afa10 con 0x7f08cc003f00 2024-03-30T00:50:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc00b9e0 con 0x7f08e415de30 2024-03-30T00:50:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e9600640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc004d10 0x7f08cc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e8c00640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e415de30 0x7f08cc003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e3e00640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08cc003f00 0x7f08cc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e9600640 1 -- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e415de30 msgr2=0x7f08cc003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e9600640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e415de30 0x7f08cc003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e9600640 1 -- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08cc003f00 msgr2=0x7f08cc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e9600640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08cc003f00 0x7f08cc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e9600640 1 -- 172.21.15.66:0/2603337136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc003530 con 0x7f08cc004d10 2024-03-30T00:50:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.039+0000 7f08e8c00640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e415de30 0x7f08cc003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:34.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.043+0000 7f08e9600640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc004d10 0x7f08cc005190 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00a890 tx=0x7f08dc01fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ba5213f3ddb8d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:34.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.043+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0aa710 con 0x7f08cc004d10 2024-03-30T00:50:34.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.043+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc0aa870 con 0x7f08cc004d10 2024-03-30T00:50:34.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.043+0000 7f08e3400640 1 -- 172.21.15.66:0/2603337136 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0aac80 con 0x7f08cc004d10 2024-03-30T00:50:34.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.043+0000 7f08eb600640 1 -- 172.21.15.66:0/2603337136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc004d10 msgr2=0x7f08cc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.043+0000 7f08eb600640 1 --2- 172.21.15.66:0/2603337136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc004d10 0x7f08cc005190 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00a890 tx=0x7f08dc01fae0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.043+0000 7f08e9600640 1 -- 172.21.15.66:0/2603337136 reap_dead start 2024-03-30T00:50:34.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.043+0000 7f08eb600640 1 -- 172.21.15.66:0/2603337136 shutdown_connections 2024-03-30T00:50:34.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.043+0000 7f08eb600640 1 -- 172.21.15.66:0/2603337136 >> 172.21.15.66:0/2603337136 conn(0x7f08e40d0540 msgr2=0x7f08e40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:34.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.043+0000 7f08eb600640 1 -- 172.21.15.66:0/2603337136 shutdown_connections 2024-03-30T00:50:34.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.043+0000 7f08eb600640 1 -- 172.21.15.66:0/2603337136 wait complete. 2024-03-30T00:50:34.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08eb600640 1 Processor -- start 2024-03-30T00:50:34.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08eb600640 1 -- start start 2024-03-30T00:50:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08eb600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e41522e0 0x7f08e41582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08eb600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e4153a10 0x7f08e4158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08eb600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e4156bb0 0x7f08e4157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08eb600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e4157bc0 con 0x7f08e4156bb0 2024-03-30T00:50:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08eb600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e41578e0 con 0x7f08e4153a10 2024-03-30T00:50:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08eb600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40c4210 con 0x7f08e41522e0 2024-03-30T00:50:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e4156bb0 0x7f08e4157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e9600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e4156bb0 0x7f08e4157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49044/0 (socket says 172.21.15.66:49044) 2024-03-30T00:50:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e9600640 1 -- 172.21.15.66:0/2400707398 learned_addr learned my addr 172.21.15.66:0/2400707398 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e8c00640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e41522e0 0x7f08e41582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e3e00640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e4153a10 0x7f08e4158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e9600640 1 -- 172.21.15.66:0/2400707398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e41522e0 msgr2=0x7f08e41582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e9600640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e41522e0 0x7f08e41582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e9600640 1 -- 172.21.15.66:0/2400707398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e4153a10 msgr2=0x7f08e4158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e9600640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e4153a10 0x7f08e4158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e9600640 1 -- 172.21.15.66:0/2400707398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e40c44f0 con 0x7f08e4156bb0 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e9600640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e4156bb0 0x7f08e4157030 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00a890 tx=0x7f08cc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e0c00640 1 -- 172.21.15.66:0/2400707398 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0aa710 con 0x7f08e4156bb0 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e0c00640 1 -- 172.21.15.66:0/2400707398 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc001fe0 con 0x7f08e4156bb0 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08e0c00640 1 -- 172.21.15.66:0/2400707398 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0019c0 con 0x7f08e4156bb0 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08eb600640 1 -- 172.21.15.66:0/2400707398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e40c47d0 con 0x7f08e4156bb0 2024-03-30T00:50:34.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.047+0000 7f08eb600640 1 -- 172.21.15.66:0/2400707398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e41526c0 con 0x7f08e4156bb0 2024-03-30T00:50:34.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.051+0000 7f08eb600640 1 -- 172.21.15.66:0/2400707398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e40cdf60 con 0x7f08e4156bb0 2024-03-30T00:50:34.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.051+0000 7f08e0c00640 1 -- 172.21.15.66:0/2400707398 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f08dc0bc460 con 0x7f08e4156bb0 2024-03-30T00:50:34.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.051+0000 7f08e0c00640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f08c4066690 0x7f08c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.051+0000 7f08e0c00640 1 -- 172.21.15.66:0/2400707398 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f08dc1214b0 con 0x7f08e4156bb0 2024-03-30T00:50:34.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.051+0000 7f08e8c00640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f08c4066690 0x7f08c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.051+0000 7f08e8c00640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f08c4066690 0x7f08c4068b50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009be0 tx=0x7f08d8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.055+0000 7f08e0c00640 1 -- 172.21.15.66:0/2400707398 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f08dc0eea50 con 0x7f08e4156bb0 2024-03-30T00:50:34.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.267+0000 7f08eb600640 1 -- 172.21.15.66:0/2400707398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f08e40c4070 con 0x7f08e4156bb0 2024-03-30T00:50:34.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08e0c00640 1 -- 172.21.15.66:0/2400707398 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f08e40c4070 con 0x7f08e4156bb0 2024-03-30T00:50:34.278 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:34.278 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:50:34.278 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-03-30T00:50:34.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.278 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm857MDRAAu2pldF20deZfoSDDXuTF/A==", 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMmNFGhAAlHS6ZpIoJc7tlYazCISiLw==", 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:50:34.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmuiBsFRAA2kez6ylAylKTWsQ1S2cahA==", 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdm/oV3IRAAWn/zG1cLK+Raa2pYPzHd7Q==", 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAbXwdmf3HtOhAAyRcyb0YUJcJca5fjmZ3Frw==", 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.281 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAfXwdml7cdBRAAs639bBHTPyfhfVZMPRMfYA==", 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.282 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAiXwdmeS6KChAAZjLAbKSL7Vswi3qQ4304bA==", 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAlXwdmeuigEBAAiPIPheowKBnn3UZWwpdr+Q==", 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.283 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQApXwdm+bQNHRAAAb/lufdb7Fe+EFevd23gtA==", 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAsXwdmReL1IhAAwCs1cS+P6J8VCOYdrnrPQg==", 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.284 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAvXwdmA15vKBAAmpnKrDM/Y0xuBGxdyjrkWA==", 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.285 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAyXwdmrl02LhAA6JfEnugoz6l4nsXl6/vq0Q==", 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmZjh7JRAA3EFOImM03iOljcwmA7/xWw==", 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-03-30T00:50:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmkZv0LhAAgwCqW25+J1VYO31mWwDtLQ==", 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.287 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmLG1/KxAAmNIZ9ZXdqPRQjCXvz7wC9A==", 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmyJ/nNBAAIgiUoUCbTeLEjL1iaPFw1A==", 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T00:50:34.288 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAXXwdmwWxQLRAAoFyZnFsIo1NvQlzRN+UHmw==", 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.289 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmEF+xCRAAqRVFguki881+4azP8lsf0g==", 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQAYXwdmMg32BRAAUWBF+QbwRGsopcAoBay8wA==", 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T00:50:34.290 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T00:50:34.291 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.291 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:34.291 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-03-30T00:50:34.291 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:50:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 -- 172.21.15.66:0/2400707398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f08c4066690 msgr2=0x7f08c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f08c4066690 0x7f08c4068b50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009be0 tx=0x7f08d8009190 comp rx=0 tx=0).stop 2024-03-30T00:50:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 -- 172.21.15.66:0/2400707398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e4156bb0 msgr2=0x7f08e4157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e4156bb0 0x7f08e4157030 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00a890 tx=0x7f08cc005800 comp rx=0 tx=0).stop 2024-03-30T00:50:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 -- 172.21.15.66:0/2400707398 shutdown_connections 2024-03-30T00:50:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f08c4066690 0x7f08c4068b50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08e4156bb0 0x7f08e4157030 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08e4153a10 0x7f08e4158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 --2- 172.21.15.66:0/2400707398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08e41522e0 0x7f08e41582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 -- 172.21.15.66:0/2400707398 >> 172.21.15.66:0/2400707398 conn(0x7f08e40d0540 msgr2=0x7f08e40aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 -- 172.21.15.66:0/2400707398 shutdown_connections 2024-03-30T00:50:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.271+0000 7f08eb600640 1 -- 172.21.15.66:0/2400707398 wait complete. 2024-03-30T00:50:34.299 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:50:34.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f2e000640 1 Processor -- start 2024-03-30T00:50:34.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f2e000640 1 -- start start 2024-03-30T00:50:34.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f2e000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f280cc560 0x7f6f280d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f2e000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6f2806f8e0 0x7f6f2806e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f2e000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2806b0b0 con 0x7f6f2806e630 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f2e000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f28160640 con 0x7f6f2806f8e0 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f2e000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2806eaa0 con 0x7f6f280cc560 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f1fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6f2806f8e0 0x7f6f2806e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f27e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6f2806e630 0x7f6f280ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40936/0 (socket says 172.21.15.66:40936) 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f1fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6f2806f8e0 0x7f6f2806e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49046/0 (socket says 172.21.15.66:49046) 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f27e00640 1 -- 172.21.15.66:0/2358894294 learned_addr learned my addr 172.21.15.66:0/2358894294 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f27400640 1 --2- 172.21.15.66:0/2358894294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f280cc560 0x7f6f280d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f1fe00640 1 -- 172.21.15.66:0/2358894294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f280cc560 msgr2=0x7f6f280d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f1fe00640 1 --2- 172.21.15.66:0/2358894294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f280cc560 0x7f6f280d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f1fe00640 1 -- 172.21.15.66:0/2358894294 >> v1:172.21.15.66:6789/0 conn(0x7f6f2806e630 legacy=0x7f6f280ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f1fe00640 1 -- 172.21.15.66:0/2358894294 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2806b390 con 0x7f6f2806f8e0 2024-03-30T00:50:34.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.431+0000 7f6f26a00640 1 -- 172.21.15.66:0/2358894294 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052170092 0 0) 0x7f6f2806b0b0 con 0x7f6f2806e630 2024-03-30T00:50:34.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f1fe00640 1 --2- 172.21.15.66:0/2358894294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f2806f8e0 0x7f6f2806e0f0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f6f100097c0 tx=0x7f6f1001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9cb964e42eb055 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:34.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f26a00640 1 -- 172.21.15.66:0/2358894294 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f100ad640 con 0x7f6f2806f8e0 2024-03-30T00:50:34.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f26a00640 1 -- 172.21.15.66:0/2358894294 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f100ad7a0 con 0x7f6f2806f8e0 2024-03-30T00:50:34.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f26a00640 1 -- 172.21.15.66:0/2358894294 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f100adb30 con 0x7f6f2806f8e0 2024-03-30T00:50:34.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f2e000640 1 -- 172.21.15.66:0/2358894294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f2806f8e0 msgr2=0x7f6f2806e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f2e000640 1 --2- 172.21.15.66:0/2358894294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f2806f8e0 0x7f6f2806e0f0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f6f100097c0 tx=0x7f6f1001c470 comp rx=0 tx=0).stop 2024-03-30T00:50:34.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f2e000640 1 -- 172.21.15.66:0/2358894294 shutdown_connections 2024-03-30T00:50:34.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f2e000640 1 --2- 172.21.15.66:0/2358894294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f2806f8e0 0x7f6f2806e0f0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f2e000640 1 --2- 172.21.15.66:0/2358894294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f280cc560 0x7f6f280d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f2e000640 1 -- 172.21.15.66:0/2358894294 >> 172.21.15.66:0/2358894294 conn(0x7f6f280d0540 msgr2=0x7f6f280d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:34.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f2e000640 1 -- 172.21.15.66:0/2358894294 shutdown_connections 2024-03-30T00:50:34.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.435+0000 7f6f2e000640 1 -- 172.21.15.66:0/2358894294 wait complete. 2024-03-30T00:50:34.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f2e000640 1 Processor -- start 2024-03-30T00:50:34.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f2e000640 1 -- start start 2024-03-30T00:50:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f2e000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f2806e630 0x7f6f281500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f2e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f2806f8e0 0x7f6f280ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f2e000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f280cc560 0x7f6f280ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f2e000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f281f9030 con 0x7f6f2806f8e0 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f2e000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2815d4f0 con 0x7f6f2806e630 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f2e000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2815d7d0 con 0x7f6f280cc560 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f2806f8e0 0x7f6f280ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f2806f8e0 0x7f6f280ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49054/0 (socket says 172.21.15.66:49054) 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1fe00640 1 -- 172.21.15.66:0/3142672505 learned_addr learned my addr 172.21.15.66:0/3142672505 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f27400640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f2806e630 0x7f6f281500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f27e00640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f280cc560 0x7f6f280ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1fe00640 1 -- 172.21.15.66:0/3142672505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f280cc560 msgr2=0x7f6f280ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1fe00640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f280cc560 0x7f6f280ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1fe00640 1 -- 172.21.15.66:0/3142672505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f2806e630 msgr2=0x7f6f281500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1fe00640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f2806e630 0x7f6f281500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1fe00640 1 -- 172.21.15.66:0/3142672505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f10009420 con 0x7f6f2806f8e0 2024-03-30T00:50:34.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1fe00640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f2806f8e0 0x7f6f280ae060 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6f100b0040 tx=0x7f6f10019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1f400640 1 -- 172.21.15.66:0/3142672505 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f100b6610 con 0x7f6f2806f8e0 2024-03-30T00:50:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1f400640 1 -- 172.21.15.66:0/3142672505 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f100a6040 con 0x7f6f2806f8e0 2024-03-30T00:50:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f2e000640 1 -- 172.21.15.66:0/3142672505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2815dab0 con 0x7f6f2806f8e0 2024-03-30T00:50:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f1f400640 1 -- 172.21.15.66:0/3142672505 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f100b69b0 con 0x7f6f2806f8e0 2024-03-30T00:50:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.439+0000 7f6f2e000640 1 -- 172.21.15.66:0/3142672505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2815e2b0 con 0x7f6f2806f8e0 2024-03-30T00:50:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.443+0000 7f6f2e000640 1 -- 172.21.15.66:0/3142672505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f28066c30 con 0x7f6f2806f8e0 2024-03-30T00:50:34.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.443+0000 7f6f1f400640 1 -- 172.21.15.66:0/3142672505 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6f100b4040 con 0x7f6f2806f8e0 2024-03-30T00:50:34.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.443+0000 7f6f1f400640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6f04066690 0x7f6f04068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.443+0000 7f6f1f400640 1 -- 172.21.15.66:0/3142672505 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6f1011e340 con 0x7f6f2806f8e0 2024-03-30T00:50:34.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.443+0000 7f6f27400640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6f04066690 0x7f6f04068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.443+0000 7f6f27400640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6f04066690 0x7f6f04068b50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009a50 tx=0x7f6f18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:34.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.447+0000 7f6f1f400640 1 -- 172.21.15.66:0/3142672505 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6f100eb8b0 con 0x7f6f2806f8e0 2024-03-30T00:50:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.611+0000 7f6f2e000640 1 -- 172.21.15.66:0/3142672505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f280adb80 con 0x7f6f2806f8e0 2024-03-30T00:50:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.611+0000 7f6f1f400640 1 -- 172.21.15.66:0/3142672505 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+12604 (secure 0 0 0) 0x7f6f100eeed0 con 0x7f6f2806f8e0 2024-03-30T00:50:34.619 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:34.620 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":121,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:33.871021+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.611+0000 7f6f2e000640 1 -- 172.21.15.66:0/3142672505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6f04066690 msgr2=0x7f6f04068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.611+0000 7f6f2e000640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6f04066690 0x7f6f04068b50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009a50 tx=0x7f6f18008040 comp rx=0 tx=0).stop 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.615+0000 7f6f2e000640 1 -- 172.21.15.66:0/3142672505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f2806f8e0 msgr2=0x7f6f280ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.615+0000 7f6f2e000640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f2806f8e0 0x7f6f280ae060 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6f100b0040 tx=0x7f6f10019210 comp rx=0 tx=0).stop 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.615+0000 7f6f2e000640 1 -- 172.21.15.66:0/3142672505 shutdown_connections 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.615+0000 7f6f2e000640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f280cc560 0x7f6f280ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.615+0000 7f6f2e000640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6f04066690 0x7f6f04068b50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.615+0000 7f6f2e000640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f2806f8e0 0x7f6f280ae060 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.615+0000 7f6f2e000640 1 --2- 172.21.15.66:0/3142672505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f2806e630 0x7f6f281500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.615+0000 7f6f2e000640 1 -- 172.21.15.66:0/3142672505 >> 172.21.15.66:0/3142672505 conn(0x7f6f280d0540 msgr2=0x7f6f280c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.615+0000 7f6f2e000640 1 -- 172.21.15.66:0/3142672505 shutdown_connections 2024-03-30T00:50:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.615+0000 7f6f2e000640 1 -- 172.21.15.66:0/3142672505 wait complete. 2024-03-30T00:50:34.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:50:34.641 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:50:34.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.775+0000 7f6a57000640 1 Processor -- start 2024-03-30T00:50:34.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.775+0000 7f6a57000640 1 -- start start 2024-03-30T00:50:34.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a57000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6a5015c780 0x7f6a500cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a57000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6a5015cb60 0x7f6a501604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a57000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a500b1d00 con 0x7f6a500cd260 2024-03-30T00:50:34.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a57000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a500cff70 con 0x7f6a5015cb60 2024-03-30T00:50:34.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a57000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a500cd820 con 0x7f6a5015c780 2024-03-30T00:50:34.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a47400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6a5015cb60 0x7f6a501604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a55000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6a500cd260 0x7f6a500d23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40940/0 (socket says 172.21.15.66:40940) 2024-03-30T00:50:34.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a55000640 1 -- 172.21.15.66:0/2846285340 learned_addr learned my addr 172.21.15.66:0/2846285340 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:34.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a4fe00640 1 --2- 172.21.15.66:0/2846285340 >> v2:172.21.15.156:3300/0 conn(0x7f6a5015c780 0x7f6a500cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a47400640 1 -- 172.21.15.66:0/2846285340 >> v2:172.21.15.156:3300/0 conn(0x7f6a5015c780 msgr2=0x7f6a500cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:34.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a47400640 1 --2- 172.21.15.66:0/2846285340 >> v2:172.21.15.156:3300/0 conn(0x7f6a5015c780 0x7f6a500cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a47400640 1 -- 172.21.15.66:0/2846285340 >> v1:172.21.15.66:6789/0 conn(0x7f6a500cd260 legacy=0x7f6a500d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a4f400640 1 -- 172.21.15.66:0/2846285340 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695594936 0 0) 0x7f6a500b1d00 con 0x7f6a500cd260 2024-03-30T00:50:34.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.779+0000 7f6a47400640 1 -- 172.21.15.66:0/2846285340 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a500cdb00 con 0x7f6a5015cb60 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a47400640 1 --2- 172.21.15.66:0/2846285340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a5015cb60 0x7f6a501604a0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f6a5009c120 tx=0x7f6a3801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c12c60e18dc844 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a4f400640 1 -- 172.21.15.66:0/2846285340 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a380ab650 con 0x7f6a5015cb60 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a4f400640 1 -- 172.21.15.66:0/2846285340 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a380ab7b0 con 0x7f6a5015cb60 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a4f400640 1 -- 172.21.15.66:0/2846285340 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a380abb40 con 0x7f6a5015cb60 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a57000640 1 -- 172.21.15.66:0/2846285340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a5015cb60 msgr2=0x7f6a501604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a57000640 1 --2- 172.21.15.66:0/2846285340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a5015cb60 0x7f6a501604a0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f6a5009c120 tx=0x7f6a3801c450 comp rx=0 tx=0).stop 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a57000640 1 -- 172.21.15.66:0/2846285340 shutdown_connections 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a57000640 1 --2- 172.21.15.66:0/2846285340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a5015cb60 0x7f6a501604a0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a57000640 1 --2- 172.21.15.66:0/2846285340 >> v2:172.21.15.156:3300/0 conn(0x7f6a5015c780 0x7f6a500cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a57000640 1 -- 172.21.15.66:0/2846285340 >> 172.21.15.66:0/2846285340 conn(0x7f6a5009ba30 msgr2=0x7f6a50089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a57000640 1 -- 172.21.15.66:0/2846285340 shutdown_connections 2024-03-30T00:50:34.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a57000640 1 -- 172.21.15.66:0/2846285340 wait complete. 2024-03-30T00:50:34.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a57000640 1 Processor -- start 2024-03-30T00:50:34.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.783+0000 7f6a57000640 1 -- start start 2024-03-30T00:50:34.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a57000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a500cd260 0x7f6a50152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a57000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a5015c780 0x7f6a500b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a57000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a5015cb60 0x7f6a5006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a57000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500b6530 con 0x7f6a5015cb60 2024-03-30T00:50:34.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a57000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a50156a00 con 0x7f6a500cd260 2024-03-30T00:50:34.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a57000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500b5920 con 0x7f6a5015c780 2024-03-30T00:50:34.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a55000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a5015cb60 0x7f6a5006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a55000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a5015cb60 0x7f6a5006b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49074/0 (socket says 172.21.15.66:49074) 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a55000640 1 -- 172.21.15.66:0/2266926399 learned_addr learned my addr 172.21.15.66:0/2266926399 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a47400640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a5015c780 0x7f6a500b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a4fe00640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a500cd260 0x7f6a50152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a55000640 1 -- 172.21.15.66:0/2266926399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a5015c780 msgr2=0x7f6a500b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a55000640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a5015c780 0x7f6a500b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a55000640 1 -- 172.21.15.66:0/2266926399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a500cd260 msgr2=0x7f6a50152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a55000640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a500cd260 0x7f6a50152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a55000640 1 -- 172.21.15.66:0/2266926399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38009420 con 0x7f6a5015cb60 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a4fe00640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a500cd260 0x7f6a50152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a55000640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a5015cb60 0x7f6a5006b2f0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f6a48001570 tx=0x7f6a4800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a4cc00640 1 -- 172.21.15.66:0/2266926399 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800cea0 con 0x7f6a5015cb60 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a4cc00640 1 -- 172.21.15.66:0/2266926399 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4800a5e0 con 0x7f6a5015cb60 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a4cc00640 1 -- 172.21.15.66:0/2266926399 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800a990 con 0x7f6a5015cb60 2024-03-30T00:50:34.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a57000640 1 -- 172.21.15.66:0/2266926399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a500d32f0 con 0x7f6a5015cb60 2024-03-30T00:50:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a57000640 1 -- 172.21.15.66:0/2266926399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a500b5ee0 con 0x7f6a5015cb60 2024-03-30T00:50:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.787+0000 7f6a57000640 1 -- 172.21.15.66:0/2266926399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a50066c30 con 0x7f6a5015cb60 2024-03-30T00:50:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.791+0000 7f6a4cc00640 1 -- 172.21.15.66:0/2266926399 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6a4800aaf0 con 0x7f6a5015cb60 2024-03-30T00:50:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.791+0000 7f6a4cc00640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a2c066690 0x7f6a2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.791+0000 7f6a4cc00640 1 -- 172.21.15.66:0/2266926399 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6a480837e0 con 0x7f6a5015cb60 2024-03-30T00:50:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.791+0000 7f6a4fe00640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a2c066690 0x7f6a2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:34.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.791+0000 7f6a4fe00640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a2c066690 0x7f6a2c068b50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6a501f9140 tx=0x7f6a40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:34.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.791+0000 7f6a4cc00640 1 -- 172.21.15.66:0/2266926399 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6a48050e80 con 0x7f6a5015cb60 2024-03-30T00:50:34.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.955+0000 7f6a57000640 1 -- 172.21.15.66:0/2266926399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a500b6670 con 0x7f6a5015cb60 2024-03-30T00:50:34.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.955+0000 7f6a4cc00640 1 -- 172.21.15.66:0/2266926399 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a500b6670 con 0x7f6a5015cb60 2024-03-30T00:50:34.965 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:50:34.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 -- 172.21.15.66:0/2266926399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a2c066690 msgr2=0x7f6a2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a2c066690 0x7f6a2c068b50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6a501f9140 tx=0x7f6a40008040 comp rx=0 tx=0).stop 2024-03-30T00:50:34.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 -- 172.21.15.66:0/2266926399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a5015cb60 msgr2=0x7f6a5006b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:34.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a5015cb60 0x7f6a5006b2f0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f6a48001570 tx=0x7f6a4800cc50 comp rx=0 tx=0).stop 2024-03-30T00:50:34.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 -- 172.21.15.66:0/2266926399 shutdown_connections 2024-03-30T00:50:34.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6a2c066690 0x7f6a2c068b50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a5015cb60 0x7f6a5006b2f0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6a5015c780 0x7f6a500b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 --2- 172.21.15.66:0/2266926399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a500cd260 0x7f6a50152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:34.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 -- 172.21.15.66:0/2266926399 >> 172.21.15.66:0/2266926399 conn(0x7f6a5009ba30 msgr2=0x7f6a500b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:34.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 -- 172.21.15.66:0/2266926399 shutdown_connections 2024-03-30T00:50:34.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:34.959+0000 7f6a57000640 1 -- 172.21.15.66:0/2266926399 wait complete. 2024-03-30T00:50:34.986 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T00:50:35.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7efa600640 1 Processor -- start 2024-03-30T00:50:35.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7efa600640 1 -- start start 2024-03-30T00:50:35.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7efa600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7ef415de30 0x7f7ef40cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7efa600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ef40cee50 0x7f7ef40d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7efa600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef40cbb90 con 0x7f7ef40af420 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7efa600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef40ce1b0 con 0x7f7ef415de30 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7efa600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef40afa10 con 0x7f7ef40cee50 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7eeaa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7ef40af420 0x7f7ef40ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40946/0 (socket says 172.21.15.66:40946) 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7eeaa00640 1 -- 172.21.15.66:0/2395920937 learned_addr learned my addr 172.21.15.66:0/2395920937 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7ef3e00640 1 --2- 172.21.15.66:0/2395920937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ef40cee50 0x7f7ef40d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7ef3400640 1 --2- 172.21.15.66:0/2395920937 >> v2:172.21.15.156:3300/0 conn(0x7f7ef415de30 0x7f7ef40cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7ef2a00640 1 -- 172.21.15.66:0/2395920937 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420294302 0 0) 0x7f7ef40cbb90 con 0x7f7ef40af420 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7ef2a00640 1 -- 172.21.15.66:0/2395920937 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ed8003530 con 0x7f7ef40af420 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7ef3e00640 1 -- 172.21.15.66:0/2395920937 >> v2:172.21.15.156:3300/0 conn(0x7f7ef415de30 msgr2=0x7f7ef40cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7ef3e00640 1 --2- 172.21.15.66:0/2395920937 >> v2:172.21.15.156:3300/0 conn(0x7f7ef415de30 0x7f7ef40cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7ef3e00640 1 -- 172.21.15.66:0/2395920937 >> v1:172.21.15.66:6789/0 conn(0x7f7ef40af420 legacy=0x7f7ef40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:35.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.119+0000 7f7ef3e00640 1 -- 172.21.15.66:0/2395920937 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef40cf430 con 0x7f7ef40cee50 2024-03-30T00:50:35.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7ef3e00640 1 --2- 172.21.15.66:0/2395920937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ef40cee50 0x7f7ef40d1060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009d90 tx=0x7f7eec01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=302abe3f4bb63190 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:35.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7ef2a00640 1 -- 172.21.15.66:0/2395920937 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec0a5900 con 0x7f7ef40cee50 2024-03-30T00:50:35.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7ef2a00640 1 -- 172.21.15.66:0/2395920937 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec0a5a60 con 0x7f7ef40cee50 2024-03-30T00:50:35.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7ef2a00640 1 -- 172.21.15.66:0/2395920937 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec0a5dd0 con 0x7f7ef40cee50 2024-03-30T00:50:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7efa600640 1 -- 172.21.15.66:0/2395920937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ef40cee50 msgr2=0x7f7ef40d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7efa600640 1 --2- 172.21.15.66:0/2395920937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ef40cee50 0x7f7ef40d1060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009d90 tx=0x7f7eec01c8e0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7efa600640 1 -- 172.21.15.66:0/2395920937 shutdown_connections 2024-03-30T00:50:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7efa600640 1 --2- 172.21.15.66:0/2395920937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ef40cee50 0x7f7ef40d1060 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7efa600640 1 --2- 172.21.15.66:0/2395920937 >> v2:172.21.15.156:3300/0 conn(0x7f7ef415de30 0x7f7ef40cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7efa600640 1 -- 172.21.15.66:0/2395920937 >> 172.21.15.66:0/2395920937 conn(0x7f7ef40d0540 msgr2=0x7f7ef40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7efa600640 1 -- 172.21.15.66:0/2395920937 shutdown_connections 2024-03-30T00:50:35.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.123+0000 7f7efa600640 1 -- 172.21.15.66:0/2395920937 wait complete. 2024-03-30T00:50:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7efa600640 1 Processor -- start 2024-03-30T00:50:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7efa600640 1 -- start start 2024-03-30T00:50:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7efa600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ef40af420 0x7f7ef4157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7efa600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ef40cee50 0x7f7ef4153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7efa600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ef415de30 0x7f7ef40b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7efa600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4155920 con 0x7f7ef40cee50 2024-03-30T00:50:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7efa600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4157c40 con 0x7f7ef415de30 2024-03-30T00:50:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7efa600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4155b70 con 0x7f7ef40af420 2024-03-30T00:50:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eeaa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ef40cee50 0x7f7ef4153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eeaa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ef40cee50 0x7f7ef4153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49082/0 (socket says 172.21.15.66:49082) 2024-03-30T00:50:35.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eeaa00640 1 -- 172.21.15.66:0/2146748545 learned_addr learned my addr 172.21.15.66:0/2146748545 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:35.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7ef3e00640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ef415de30 0x7f7ef40b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7ef3400640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ef40af420 0x7f7ef4157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eeaa00640 1 -- 172.21.15.66:0/2146748545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ef40af420 msgr2=0x7f7ef4157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:35.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eeaa00640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ef40af420 0x7f7ef4157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eeaa00640 1 -- 172.21.15.66:0/2146748545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ef415de30 msgr2=0x7f7ef40b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:35.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eeaa00640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ef415de30 0x7f7ef40b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eeaa00640 1 -- 172.21.15.66:0/2146748545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec009a70 con 0x7f7ef40cee50 2024-03-30T00:50:35.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eeaa00640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ef40cee50 0x7f7ef4153dc0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00c3c0 tx=0x7f7edc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:35.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eebe00640 1 -- 172.21.15.66:0/2146748545 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc005cf0 con 0x7f7ef40cee50 2024-03-30T00:50:35.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eebe00640 1 -- 172.21.15.66:0/2146748545 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc005e50 con 0x7f7ef40cee50 2024-03-30T00:50:35.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7eebe00640 1 -- 172.21.15.66:0/2146748545 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc0072f0 con 0x7f7ef40cee50 2024-03-30T00:50:35.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7efa600640 1 -- 172.21.15.66:0/2146748545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef4155e50 con 0x7f7ef40cee50 2024-03-30T00:50:35.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7efa600640 1 -- 172.21.15.66:0/2146748545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef4156410 con 0x7f7ef40cee50 2024-03-30T00:50:35.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.127+0000 7f7efa600640 1 -- 172.21.15.66:0/2146748545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef40cdf60 con 0x7f7ef40cee50 2024-03-30T00:50:35.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.131+0000 7f7eebe00640 1 -- 172.21.15.66:0/2146748545 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7edc007450 con 0x7f7ef40cee50 2024-03-30T00:50:35.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.131+0000 7f7eebe00640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ed0066690 0x7f7ed0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.131+0000 7f7eebe00640 1 -- 172.21.15.66:0/2146748545 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7edc083970 con 0x7f7ef40cee50 2024-03-30T00:50:35.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.131+0000 7f7ef3400640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ed0066690 0x7f7ed0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.131+0000 7f7ef3400640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ed0066690 0x7f7ed0068b50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009b60 tx=0x7f7ee4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:35.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.135+0000 7f7eebe00640 1 -- 172.21.15.66:0/2146748545 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7edc050f80 con 0x7f7ef40cee50 2024-03-30T00:50:35.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 -- 172.21.15.66:0/2146748545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7ef4066c30 con 0x7f7ef40cee50 2024-03-30T00:50:35.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7eebe00640 1 -- 172.21.15.66:0/2146748545 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v121) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7edc017070 con 0x7f7ef40cee50 2024-03-30T00:50:35.311 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:35.311 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T00:50:35.311 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T00:50:35.311 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-03-30T00:50:35.311 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-03-30T00:50:35.311 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:35.312 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T00:50:35.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 -- 172.21.15.66:0/2146748545 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ed0066690 msgr2=0x7f7ed0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:35.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ed0066690 0x7f7ed0068b50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009b60 tx=0x7f7ee4009190 comp rx=0 tx=0).stop 2024-03-30T00:50:35.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 -- 172.21.15.66:0/2146748545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ef40cee50 msgr2=0x7f7ef4153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:35.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ef40cee50 0x7f7ef4153dc0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00c3c0 tx=0x7f7edc00c880 comp rx=0 tx=0).stop 2024-03-30T00:50:35.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 -- 172.21.15.66:0/2146748545 shutdown_connections 2024-03-30T00:50:35.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ef415de30 0x7f7ef40b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7ed0066690 0x7f7ed0068b50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ef40cee50 0x7f7ef4153dc0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 --2- 172.21.15.66:0/2146748545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7ef40af420 0x7f7ef4157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 -- 172.21.15.66:0/2146748545 >> 172.21.15.66:0/2146748545 conn(0x7f7ef40d0540 msgr2=0x7f7ef40ad360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:35.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 -- 172.21.15.66:0/2146748545 shutdown_connections 2024-03-30T00:50:35.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.303+0000 7f7efa600640 1 -- 172.21.15.66:0/2146748545 wait complete. 2024-03-30T00:50:35.331 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T00:50:35.331 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T00:50:35.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.467+0000 7f7396600640 1 Processor -- start 2024-03-30T00:50:35.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.467+0000 7f7396600640 1 -- start start 2024-03-30T00:50:35.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.467+0000 7f7396600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f73900cbd30 0x7f739015c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.467+0000 7f7396600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f73900cb650 0x7f739015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.467+0000 7f7396600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73900cbb50 con 0x7f739006e7f0 2024-03-30T00:50:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.467+0000 7f7396600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739006a860 con 0x7f73900cbd30 2024-03-30T00:50:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.467+0000 7f7396600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73900cd340 con 0x7f73900cb650 2024-03-30T00:50:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.467+0000 7f7386a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f73900cbd30 0x7f739015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.467+0000 7f7386a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f73900cbd30 0x7f739015c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49094/0 (socket says 172.21.15.66:49094) 2024-03-30T00:50:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.467+0000 7f7386a00640 1 -- 172.21.15.66:0/1638591721 learned_addr learned my addr 172.21.15.66:0/1638591721 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.471+0000 7f738fe00640 1 --2- 172.21.15.66:0/1638591721 >> v2:172.21.15.156:3300/0 conn(0x7f73900cb650 0x7f739015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.471+0000 7f7386a00640 1 -- 172.21.15.66:0/1638591721 >> v2:172.21.15.156:3300/0 conn(0x7f73900cb650 msgr2=0x7f739015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.471+0000 7f7386a00640 1 --2- 172.21.15.66:0/1638591721 >> v2:172.21.15.156:3300/0 conn(0x7f73900cb650 0x7f739015c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.471+0000 7f7386a00640 1 -- 172.21.15.66:0/1638591721 >> v1:172.21.15.156:6789/0 conn(0x7f739006e7f0 legacy=0x7f739006ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.471+0000 7f7386a00640 1 -- 172.21.15.66:0/1638591721 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7390069ec0 con 0x7f73900cbd30 2024-03-30T00:50:35.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.471+0000 7f7386a00640 1 --2- 172.21.15.66:0/1638591721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73900cbd30 0x7f739015c160 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f7390072820 tx=0x7f737801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1a5c70bcf4ad server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:35.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.471+0000 7f738ea00640 1 -- 172.21.15.66:0/1638591721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73780ab680 con 0x7f73900cbd30 2024-03-30T00:50:35.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.471+0000 7f738ea00640 1 -- 172.21.15.66:0/1638591721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73780ab7e0 con 0x7f73900cbd30 2024-03-30T00:50:35.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.471+0000 7f738ea00640 1 -- 172.21.15.66:0/1638591721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73780abb70 con 0x7f73900cbd30 2024-03-30T00:50:35.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.471+0000 7f7396600640 1 -- 172.21.15.66:0/1638591721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73900cbd30 msgr2=0x7f739015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:35.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.471+0000 7f7396600640 1 --2- 172.21.15.66:0/1638591721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73900cbd30 0x7f739015c160 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f7390072820 tx=0x7f737801c460 comp rx=0 tx=0).stop 2024-03-30T00:50:35.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 -- 172.21.15.66:0/1638591721 shutdown_connections 2024-03-30T00:50:35.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 --2- 172.21.15.66:0/1638591721 >> v2:172.21.15.156:3300/0 conn(0x7f73900cb650 0x7f739015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 --2- 172.21.15.66:0/1638591721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73900cbd30 0x7f739015c160 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 -- 172.21.15.66:0/1638591721 >> 172.21.15.66:0/1638591721 conn(0x7f73900b1030 msgr2=0x7f73900b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:35.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 -- 172.21.15.66:0/1638591721 shutdown_connections 2024-03-30T00:50:35.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 -- 172.21.15.66:0/1638591721 wait complete. 2024-03-30T00:50:35.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 Processor -- start 2024-03-30T00:50:35.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 -- start start 2024-03-30T00:50:35.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f739006e7f0 0x7f73901520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73900cb650 0x7f739015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73900cbd30 0x7f739015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739006e3b0 con 0x7f73900cbd30 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739015e940 con 0x7f73900cb650 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739015ec20 con 0x7f739006e7f0 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f738fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73900cbd30 0x7f739015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f738fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73900cbd30 0x7f739015db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49100/0 (socket says 172.21.15.66:49100) 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f738fe00640 1 -- 172.21.15.66:0/1532969359 learned_addr learned my addr 172.21.15.66:0/1532969359 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f738f400640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f739006e7f0 0x7f73901520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7386a00640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73900cb650 0x7f739015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f738fe00640 1 -- 172.21.15.66:0/1532969359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f739006e7f0 msgr2=0x7f73901520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f738fe00640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f739006e7f0 0x7f73901520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f738fe00640 1 -- 172.21.15.66:0/1532969359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73900cb650 msgr2=0x7f739015f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f738fe00640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73900cb650 0x7f739015f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f738fe00640 1 -- 172.21.15.66:0/1532969359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73780094d0 con 0x7f73900cbd30 2024-03-30T00:50:35.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f738f400640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f739006e7f0 0x7f73901520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:35.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7386a00640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73900cb650 0x7f739015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:35.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f738fe00640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73900cbd30 0x7f739015db80 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f738800ef90 tx=0x7f738800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:35.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7387e00640 1 -- 172.21.15.66:0/1532969359 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388009ca0 con 0x7f73900cbd30 2024-03-30T00:50:35.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7387e00640 1 -- 172.21.15.66:0/1532969359 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738800f040 con 0x7f73900cbd30 2024-03-30T00:50:35.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7396600640 1 -- 172.21.15.66:0/1532969359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739015ef00 con 0x7f73900cbd30 2024-03-30T00:50:35.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.475+0000 7f7387e00640 1 -- 172.21.15.66:0/1532969359 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880146c0 con 0x7f73900cbd30 2024-03-30T00:50:35.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.479+0000 7f7396600640 1 -- 172.21.15.66:0/1532969359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739015f4c0 con 0x7f73900cbd30 2024-03-30T00:50:35.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.479+0000 7f7396600640 1 -- 172.21.15.66:0/1532969359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739015f6b0 con 0x7f73900cbd30 2024-03-30T00:50:35.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.479+0000 7f7387e00640 1 -- 172.21.15.66:0/1532969359 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7388014820 con 0x7f73900cbd30 2024-03-30T00:50:35.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.479+0000 7f7387e00640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f736c066690 0x7f736c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:35.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.479+0000 7f7387e00640 1 -- 172.21.15.66:0/1532969359 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7388084e50 con 0x7f73900cbd30 2024-03-30T00:50:35.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.479+0000 7f738f400640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f736c066690 0x7f736c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:35.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.479+0000 7f738f400640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f736c066690 0x7f736c068b50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7380000f70 tx=0x7f7380009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:35.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.483+0000 7f7387e00640 1 -- 172.21.15.66:0/1532969359 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f73880523c0 con 0x7f73900cbd30 2024-03-30T00:50:35.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:35.647+0000 7f7396600640 1 -- 172.21.15.66:0/1532969359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7390066c30 con 0x7f73900cbd30 2024-03-30T00:50:36.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.627+0000 7f7387e00640 1 -- 172.21.15.66:0/1532969359 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v122) v1 ==== 138+0+0 (secure 0 0 0) 0x7f73880180b0 con 0x7f73900cbd30 2024-03-30T00:50:36.634 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-03-30T00:50:36.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 -- 172.21.15.66:0/1532969359 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f736c066690 msgr2=0x7f736c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:36.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f736c066690 0x7f736c068b50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7380000f70 tx=0x7f7380009040 comp rx=0 tx=0).stop 2024-03-30T00:50:36.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 -- 172.21.15.66:0/1532969359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73900cbd30 msgr2=0x7f739015db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:36.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73900cbd30 0x7f739015db80 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f738800ef90 tx=0x7f738800c490 comp rx=0 tx=0).stop 2024-03-30T00:50:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 -- 172.21.15.66:0/1532969359 shutdown_connections 2024-03-30T00:50:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f736c066690 0x7f736c068b50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73900cbd30 0x7f739015db80 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73900cb650 0x7f739015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 --2- 172.21.15.66:0/1532969359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f739006e7f0 0x7f73901520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:36.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 -- 172.21.15.66:0/1532969359 >> 172.21.15.66:0/1532969359 conn(0x7f73900b1030 msgr2=0x7f73900aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:36.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 -- 172.21.15.66:0/1532969359 shutdown_connections 2024-03-30T00:50:36.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.631+0000 7f7396600640 1 -- 172.21.15.66:0/1532969359 wait complete. 2024-03-30T00:50:36.665 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T00:50:36.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.807+0000 7f5785800640 1 Processor -- start 2024-03-30T00:50:36.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.807+0000 7f5785800640 1 -- start start 2024-03-30T00:50:36.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f5785800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f578015a770 0x7f57800c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:36.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f5785800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57800b1db0 0x7f57800cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:36.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f5785800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57800ca130 con 0x7f57800ce780 2024-03-30T00:50:36.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f5785800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57800cb120 con 0x7f578015a770 2024-03-30T00:50:36.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f5785800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578015ab50 con 0x7f57800b1db0 2024-03-30T00:50:36.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f577ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f578015a770 0x7f57800c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:36.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f577ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f578015a770 0x7f57800c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49106/0 (socket says 172.21.15.66:49106) 2024-03-30T00:50:36.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f577f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f57800ce780 0x7f57800710e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40952/0 (socket says 172.21.15.66:40952) 2024-03-30T00:50:36.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f577ea00640 1 -- 172.21.15.66:0/2918226028 learned_addr learned my addr 172.21.15.66:0/2918226028 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:36.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f5776000640 1 --2- 172.21.15.66:0/2918226028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57800b1db0 0x7f57800cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:36.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f577ea00640 1 -- 172.21.15.66:0/2918226028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57800b1db0 msgr2=0x7f57800cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:36.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f577ea00640 1 --2- 172.21.15.66:0/2918226028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57800b1db0 0x7f57800cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:36.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f577ea00640 1 -- 172.21.15.66:0/2918226028 >> v1:172.21.15.66:6789/0 conn(0x7f57800ce780 legacy=0x7f57800710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:36.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f577ea00640 1 -- 172.21.15.66:0/2918226028 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57800ca400 con 0x7f578015a770 2024-03-30T00:50:36.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f5776000640 1 --2- 172.21.15.66:0/2918226028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57800b1db0 0x7f57800cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:36.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.811+0000 7f577e000640 1 -- 172.21.15.66:0/2918226028 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301653679 0 0) 0x7f57800ca130 con 0x7f57800ce780 2024-03-30T00:50:36.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f577ea00640 1 --2- 172.21.15.66:0/2918226028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f578015a770 0x7f57800c9bf0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f5770009870 tx=0x7f577001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5263feb37c3ed57f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:36.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f577e000640 1 -- 172.21.15.66:0/2918226028 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57700ab640 con 0x7f578015a770 2024-03-30T00:50:36.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f577e000640 1 -- 172.21.15.66:0/2918226028 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57700a7070 con 0x7f578015a770 2024-03-30T00:50:36.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f577e000640 1 -- 172.21.15.66:0/2918226028 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57700ab9c0 con 0x7f578015a770 2024-03-30T00:50:36.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f5785800640 1 -- 172.21.15.66:0/2918226028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f578015a770 msgr2=0x7f57800c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:36.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f5785800640 1 --2- 172.21.15.66:0/2918226028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f578015a770 0x7f57800c9bf0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f5770009870 tx=0x7f577001c530 comp rx=0 tx=0).stop 2024-03-30T00:50:36.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f5785800640 1 -- 172.21.15.66:0/2918226028 shutdown_connections 2024-03-30T00:50:36.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f5785800640 1 --2- 172.21.15.66:0/2918226028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57800b1db0 0x7f57800cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:36.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f5785800640 1 --2- 172.21.15.66:0/2918226028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f578015a770 0x7f57800c9bf0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:36.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f5785800640 1 -- 172.21.15.66:0/2918226028 >> 172.21.15.66:0/2918226028 conn(0x7f578006a190 msgr2=0x7f578006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:36.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f5785800640 1 -- 172.21.15.66:0/2918226028 shutdown_connections 2024-03-30T00:50:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f5785800640 1 -- 172.21.15.66:0/2918226028 wait complete. 2024-03-30T00:50:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f5785800640 1 Processor -- start 2024-03-30T00:50:36.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.815+0000 7f5785800640 1 -- start start 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5785800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57800b1db0 0x7f5780162540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5785800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800ce780 0x7f5780162a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5785800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f578015a770 0x7f5780156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5785800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57800cc3d0 con 0x7f57800ce780 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5785800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57800c9ff0 con 0x7f57800b1db0 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5785800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5780157600 con 0x7f578015a770 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5776000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800ce780 0x7f5780162a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5776000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800ce780 0x7f5780162a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49110/0 (socket says 172.21.15.66:49110) 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5776000640 1 -- 172.21.15.66:0/1338110505 learned_addr learned my addr 172.21.15.66:0/1338110505 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f577f400640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f578015a770 0x7f5780156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f577ea00640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57800b1db0 0x7f5780162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5776000640 1 -- 172.21.15.66:0/1338110505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f578015a770 msgr2=0x7f5780156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:36.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5776000640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f578015a770 0x7f5780156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:36.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5776000640 1 -- 172.21.15.66:0/1338110505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57800b1db0 msgr2=0x7f5780162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:36.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5776000640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57800b1db0 0x7f5780162540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:36.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5776000640 1 -- 172.21.15.66:0/1338110505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700094d0 con 0x7f57800ce780 2024-03-30T00:50:36.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f577f400640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f578015a770 0x7f5780156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:36.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5776000640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800ce780 0x7f5780162a80 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f576800e810 tx=0x7f576800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:36.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5777400640 1 -- 172.21.15.66:0/1338110505 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576800c8d0 con 0x7f57800ce780 2024-03-30T00:50:36.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5777400640 1 -- 172.21.15.66:0/1338110505 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576800ca30 con 0x7f57800ce780 2024-03-30T00:50:36.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5777400640 1 -- 172.21.15.66:0/1338110505 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576800cdc0 con 0x7f57800ce780 2024-03-30T00:50:36.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5785800640 1 -- 172.21.15.66:0/1338110505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57801578e0 con 0x7f57800ce780 2024-03-30T00:50:36.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5785800640 1 -- 172.21.15.66:0/1338110505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5780157bc0 con 0x7f57800ce780 2024-03-30T00:50:36.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5785800640 1 -- 172.21.15.66:0/1338110505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57800cc640 con 0x7f57800ce780 2024-03-30T00:50:36.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.819+0000 7f5777400640 1 -- 172.21.15.66:0/1338110505 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5768013410 con 0x7f57800ce780 2024-03-30T00:50:36.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.823+0000 7f5777400640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f575c066690 0x7f575c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:36.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.823+0000 7f5777400640 1 -- 172.21.15.66:0/1338110505 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f5768083e70 con 0x7f57800ce780 2024-03-30T00:50:36.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.823+0000 7f577ea00640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f575c066690 0x7f575c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:36.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.823+0000 7f577ea00640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f575c066690 0x7f575c068b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f57700af000 tx=0x7f57700b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:36.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.827+0000 7f5777400640 1 -- 172.21.15.66:0/1338110505 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5768051320 con 0x7f57800ce780 2024-03-30T00:50:36.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:36.987+0000 7f5785800640 1 -- 172.21.15.66:0/1338110505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5780066c30 con 0x7f57800ce780 2024-03-30T00:50:37.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.631+0000 7f5777400640 1 -- 172.21.15.66:0/1338110505 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v123) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5768054940 con 0x7f57800ce780 2024-03-30T00:50:37.640 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-03-30T00:50:37.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.639+0000 7f5785800640 1 -- 172.21.15.66:0/1338110505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f575c066690 msgr2=0x7f575c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:37.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.639+0000 7f5785800640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f575c066690 0x7f575c068b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f57700af000 tx=0x7f57700b6040 comp rx=0 tx=0).stop 2024-03-30T00:50:37.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.639+0000 7f5785800640 1 -- 172.21.15.66:0/1338110505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800ce780 msgr2=0x7f5780162a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:37.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.639+0000 7f5785800640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800ce780 0x7f5780162a80 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f576800e810 tx=0x7f576800ecd0 comp rx=0 tx=0).stop 2024-03-30T00:50:37.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.639+0000 7f5785800640 1 -- 172.21.15.66:0/1338110505 shutdown_connections 2024-03-30T00:50:37.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.639+0000 7f5785800640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f578015a770 0x7f5780156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:37.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.639+0000 7f5785800640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f575c066690 0x7f575c068b50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:37.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.639+0000 7f5785800640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57800ce780 0x7f5780162a80 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:37.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.639+0000 7f5785800640 1 --2- 172.21.15.66:0/1338110505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57800b1db0 0x7f5780162540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:37.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.639+0000 7f5785800640 1 -- 172.21.15.66:0/1338110505 >> 172.21.15.66:0/1338110505 conn(0x7f578006a190 msgr2=0x7f578006db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:37.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.639+0000 7f5785800640 1 -- 172.21.15.66:0/1338110505 shutdown_connections 2024-03-30T00:50:37.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.643+0000 7f5785800640 1 -- 172.21.15.66:0/1338110505 wait complete. 2024-03-30T00:50:37.682 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T00:50:37.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.819+0000 7ff8b1c00640 1 Processor -- start 2024-03-30T00:50:37.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.819+0000 7ff8b1c00640 1 -- start start 2024-03-30T00:50:37.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8b1c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8ac0cecf0 0x7ff8ac0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:37.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8b1c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff8ac06d7e0 0x7ff8ac0b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:37.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8b1c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ac15de20 con 0x7ff8ac0acae0 2024-03-30T00:50:37.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8b1c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ac0acec0 con 0x7ff8ac0cecf0 2024-03-30T00:50:37.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8b1c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ac15d060 con 0x7ff8ac06d7e0 2024-03-30T00:50:37.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8a3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8ac0cecf0 0x7ff8ac0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:37.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8a3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8ac0cecf0 0x7ff8ac0b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49114/0 (socket says 172.21.15.66:49114) 2024-03-30T00:50:37.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8a3400640 1 -- 172.21.15.66:0/1036476321 learned_addr learned my addr 172.21.15.66:0/1036476321 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:37.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8b1200640 1 --2- 172.21.15.66:0/1036476321 >> v2:172.21.15.156:3300/0 conn(0x7ff8ac06d7e0 0x7ff8ac0b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:37.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8a3400640 1 -- 172.21.15.66:0/1036476321 >> v2:172.21.15.156:3300/0 conn(0x7ff8ac06d7e0 msgr2=0x7ff8ac0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:37.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8a3400640 1 --2- 172.21.15.66:0/1036476321 >> v2:172.21.15.156:3300/0 conn(0x7ff8ac06d7e0 0x7ff8ac0b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:37.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8a3400640 1 -- 172.21.15.66:0/1036476321 >> v1:172.21.15.156:6789/0 conn(0x7ff8ac0acae0 legacy=0x7ff8ac0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:37.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8a3400640 1 -- 172.21.15.66:0/1036476321 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac15d340 con 0x7ff8ac0cecf0 2024-03-30T00:50:37.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.823+0000 7ff8b1200640 1 --2- 172.21.15.66:0/1036476321 >> v2:172.21.15.156:3300/0 conn(0x7ff8ac06d7e0 0x7ff8ac0b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:37.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8a3400640 1 --2- 172.21.15.66:0/1036476321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ac0cecf0 0x7ff8ac0b1640 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0cd1c0 tx=0x7ff89401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5ec8afa0186f94 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:37.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8ab400640 1 -- 172.21.15.66:0/1036476321 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8940a7c90 con 0x7ff8ac0cecf0 2024-03-30T00:50:37.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8ab400640 1 -- 172.21.15.66:0/1036476321 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8940a7df0 con 0x7ff8ac0cecf0 2024-03-30T00:50:37.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8ab400640 1 -- 172.21.15.66:0/1036476321 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8940af660 con 0x7ff8ac0cecf0 2024-03-30T00:50:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8b1c00640 1 -- 172.21.15.66:0/1036476321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ac0cecf0 msgr2=0x7ff8ac0b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8b1c00640 1 --2- 172.21.15.66:0/1036476321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ac0cecf0 0x7ff8ac0b1640 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0cd1c0 tx=0x7ff89401ebc0 comp rx=0 tx=0).stop 2024-03-30T00:50:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8b1c00640 1 -- 172.21.15.66:0/1036476321 shutdown_connections 2024-03-30T00:50:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8b1c00640 1 --2- 172.21.15.66:0/1036476321 >> v2:172.21.15.156:3300/0 conn(0x7ff8ac06d7e0 0x7ff8ac0b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8b1c00640 1 --2- 172.21.15.66:0/1036476321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ac0cecf0 0x7ff8ac0b1640 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8b1c00640 1 -- 172.21.15.66:0/1036476321 >> 172.21.15.66:0/1036476321 conn(0x7ff8ac06a190 msgr2=0x7ff8ac06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8b1c00640 1 -- 172.21.15.66:0/1036476321 shutdown_connections 2024-03-30T00:50:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8b1c00640 1 -- 172.21.15.66:0/1036476321 wait complete. 2024-03-30T00:50:37.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8b1c00640 1 Processor -- start 2024-03-30T00:50:37.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.827+0000 7ff8b1c00640 1 -- start start 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8b1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ac06d7e0 0x7ff8ac151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8b1c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8ac0acae0 0x7ff8ac0c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8b1c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8ac0cecf0 0x7ff8ac15af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8b1c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac152780 con 0x7ff8ac06d7e0 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8b1c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac163850 con 0x7ff8ac0acae0 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8b1c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac163570 con 0x7ff8ac0cecf0 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ac06d7e0 0x7ff8ac151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ac06d7e0 0x7ff8ac151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49124/0 (socket says 172.21.15.66:49124) 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8abe00640 1 -- 172.21.15.66:0/3120139245 learned_addr learned my addr 172.21.15.66:0/3120139245 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8b1200640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8ac0cecf0 0x7ff8ac15af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8a3400640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8ac0acae0 0x7ff8ac0c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:37.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8abe00640 1 -- 172.21.15.66:0/3120139245 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8ac0cecf0 msgr2=0x7ff8ac15af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:37.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8abe00640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8ac0cecf0 0x7ff8ac15af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:37.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8abe00640 1 -- 172.21.15.66:0/3120139245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8ac0acae0 msgr2=0x7ff8ac0c2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:37.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8abe00640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8ac0acae0 0x7ff8ac0c2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:37.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8abe00640 1 -- 172.21.15.66:0/3120139245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89400b770 con 0x7ff8ac06d7e0 2024-03-30T00:50:37.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8abe00640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ac06d7e0 0x7ff8ac151c70 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c500 tx=0x7ff89c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:37.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8a8c00640 1 -- 172.21.15.66:0/3120139245 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c005220 con 0x7ff8ac06d7e0 2024-03-30T00:50:37.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8a8c00640 1 -- 172.21.15.66:0/3120139245 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c005380 con 0x7ff8ac06d7e0 2024-03-30T00:50:37.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8a8c00640 1 -- 172.21.15.66:0/3120139245 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c007270 con 0x7ff8ac06d7e0 2024-03-30T00:50:37.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8b1c00640 1 -- 172.21.15.66:0/3120139245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac158f70 con 0x7ff8ac06d7e0 2024-03-30T00:50:37.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8b1c00640 1 -- 172.21.15.66:0/3120139245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac159530 con 0x7ff8ac06d7e0 2024-03-30T00:50:37.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8b1c00640 1 -- 172.21.15.66:0/3120139245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ac073e00 con 0x7ff8ac06d7e0 2024-03-30T00:50:37.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.831+0000 7ff8a8c00640 1 -- 172.21.15.66:0/3120139245 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff89c0073d0 con 0x7ff8ac06d7e0 2024-03-30T00:50:37.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.835+0000 7ff8a8c00640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff888066690 0x7ff888068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:37.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.835+0000 7ff8a8c00640 1 -- 172.21.15.66:0/3120139245 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5929+0+0 (secure 0 0 0) 0x7ff89c083c00 con 0x7ff8ac06d7e0 2024-03-30T00:50:37.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.835+0000 7ff8a3400640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff888066690 0x7ff888068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:37.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.835+0000 7ff8a3400640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff888066690 0x7ff888068b50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff8940b1000 tx=0x7ff8940a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:37.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:37.839+0000 7ff8a8c00640 1 -- 172.21.15.66:0/3120139245 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff89c050e00 con 0x7ff8ac06d7e0 2024-03-30T00:50:38.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.035+0000 7ff8b1c00640 1 -- 172.21.15.66:0/3120139245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff8ac0cb7a0 con 0x7ff8ac06d7e0 2024-03-30T00:50:38.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.047+0000 7ff8a8c00640 1 -- 172.21.15.66:0/3120139245 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v91) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff89c054420 con 0x7ff8ac06d7e0 2024-03-30T00:50:38.057 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 17 and data pool 18 2024-03-30T00:50:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.055+0000 7ff8b1c00640 1 -- 172.21.15.66:0/3120139245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff888066690 msgr2=0x7ff888068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.055+0000 7ff8b1c00640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff888066690 0x7ff888068b50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff8940b1000 tx=0x7ff8940a9070 comp rx=0 tx=0).stop 2024-03-30T00:50:38.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.055+0000 7ff8b1c00640 1 -- 172.21.15.66:0/3120139245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ac06d7e0 msgr2=0x7ff8ac151c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.055+0000 7ff8b1c00640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ac06d7e0 0x7ff8ac151c70 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c500 tx=0x7ff89c00c9c0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.059+0000 7ff8b1c00640 1 -- 172.21.15.66:0/3120139245 shutdown_connections 2024-03-30T00:50:38.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.059+0000 7ff8b1c00640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8ac0cecf0 0x7ff8ac15af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.059+0000 7ff8b1c00640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8ac0acae0 0x7ff8ac0c2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.059+0000 7ff8b1c00640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff888066690 0x7ff888068b50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.059+0000 7ff8b1c00640 1 --2- 172.21.15.66:0/3120139245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ac06d7e0 0x7ff8ac151c70 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.059+0000 7ff8b1c00640 1 -- 172.21.15.66:0/3120139245 >> 172.21.15.66:0/3120139245 conn(0x7ff8ac06a190 msgr2=0x7ff8ac0cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.059+0000 7ff8b1c00640 1 -- 172.21.15.66:0/3120139245 shutdown_connections 2024-03-30T00:50:38.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.059+0000 7ff8b1c00640 1 -- 172.21.15.66:0/3120139245 wait complete. 2024-03-30T00:50:38.102 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:50:38.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2ad6200640 1 Processor -- start 2024-03-30T00:50:38.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2ad6200640 1 -- start start 2024-03-30T00:50:38.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2ad6200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2ad015de30 0x7f2ad00cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2ad6200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ad00af420 0x7f2ad00d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2ad6200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad00cbb90 con 0x7f2ad00ce740 2024-03-30T00:50:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2ad6200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad00ce1b0 con 0x7f2ad015de30 2024-03-30T00:50:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2ad6200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00afa10 con 0x7f2ad00af420 2024-03-30T00:50:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2acf400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2ad015de30 0x7f2ad00cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2ac6a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ad00af420 0x7f2ad00d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2acf400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2ad015de30 0x7f2ad00cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:51796/0 (socket says 172.21.15.66:51796) 2024-03-30T00:50:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2ac6a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ad00af420 0x7f2ad00d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:37714/0 (socket says 172.21.15.66:37714) 2024-03-30T00:50:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2acf400640 1 -- 172.21.15.66:0/1944084515 learned_addr learned my addr 172.21.15.66:0/1944084515 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2acfe00640 1 --1- 172.21.15.66:0/1944084515 >> v1:172.21.15.156:6789/0 conn(0x7f2ad00ce740 0x7f2ad006e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:60570/0 (socket says 172.21.15.66:60570) 2024-03-30T00:50:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2acf400640 1 -- 172.21.15.66:0/1944084515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ad00af420 msgr2=0x7f2ad00d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2acf400640 1 --2- 172.21.15.66:0/1944084515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ad00af420 0x7f2ad00d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2acf400640 1 -- 172.21.15.66:0/1944084515 >> v1:172.21.15.156:6789/0 conn(0x7f2ad00ce740 legacy=0x7f2ad006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.235+0000 7f2acf400640 1 -- 172.21.15.66:0/1944084515 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad00cf430 con 0x7f2ad015de30 2024-03-30T00:50:38.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2acf400640 1 --2- 172.21.15.66:0/1944084515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad015de30 0x7f2ad00cc040 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009e50 tx=0x7f2ac001c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f408de4114659db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:38.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2acea00640 1 -- 172.21.15.66:0/1944084515 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00a57c0 con 0x7f2ad015de30 2024-03-30T00:50:38.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2acea00640 1 -- 172.21.15.66:0/1944084515 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac00a5920 con 0x7f2ad015de30 2024-03-30T00:50:38.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2acea00640 1 -- 172.21.15.66:0/1944084515 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00a5cb0 con 0x7f2ad015de30 2024-03-30T00:50:38.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2ad6200640 1 -- 172.21.15.66:0/1944084515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad015de30 msgr2=0x7f2ad00cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2ad6200640 1 --2- 172.21.15.66:0/1944084515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad015de30 0x7f2ad00cc040 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009e50 tx=0x7f2ac001c9a0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2ad6200640 1 -- 172.21.15.66:0/1944084515 shutdown_connections 2024-03-30T00:50:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2ad6200640 1 --2- 172.21.15.66:0/1944084515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ad00af420 0x7f2ad00d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2ad6200640 1 --2- 172.21.15.66:0/1944084515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad015de30 0x7f2ad00cc040 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2ad6200640 1 -- 172.21.15.66:0/1944084515 >> 172.21.15.66:0/1944084515 conn(0x7f2ad00d0540 msgr2=0x7f2ad00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2ad6200640 1 -- 172.21.15.66:0/1944084515 shutdown_connections 2024-03-30T00:50:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.239+0000 7f2ad6200640 1 -- 172.21.15.66:0/1944084515 wait complete. 2024-03-30T00:50:38.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ad6200640 1 Processor -- start 2024-03-30T00:50:38.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ad6200640 1 -- start start 2024-03-30T00:50:38.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ad6200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad00af420 0x7f2ad0144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ad6200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ad00ce740 0x7f2ad0146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ad6200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad015de30 0x7f2ad00c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ad6200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00c36a0 con 0x7f2ad015de30 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ad6200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00b2830 con 0x7f2ad00af420 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ad6200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00b2af0 con 0x7f2ad00ce740 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad015de30 0x7f2ad00c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2acf400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad00af420 0x7f2ad0144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad015de30 0x7f2ad00c3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49128/0 (socket says 172.21.15.66:49128) 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2acfe00640 1 -- 172.21.15.66:0/89577246 learned_addr learned my addr 172.21.15.66:0/89577246 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ac6a00640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ad00ce740 0x7f2ad0146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2acf400640 1 -- 172.21.15.66:0/89577246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ad00ce740 msgr2=0x7f2ad0146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2acf400640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ad00ce740 0x7f2ad0146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2acf400640 1 -- 172.21.15.66:0/89577246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad015de30 msgr2=0x7f2ad00c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2acf400640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad015de30 0x7f2ad00c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2acf400640 1 -- 172.21.15.66:0/89577246 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac0009b30 con 0x7f2ad00af420 2024-03-30T00:50:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2acf400640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad00af420 0x7f2ad0144530 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00ab000 tx=0x7f2ac0019740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ac7e00640 1 -- 172.21.15.66:0/89577246 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00b8670 con 0x7f2ad00af420 2024-03-30T00:50:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ac7e00640 1 -- 172.21.15.66:0/89577246 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac00b87d0 con 0x7f2ad00af420 2024-03-30T00:50:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ad6200640 1 -- 172.21.15.66:0/89577246 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad00b2dd0 con 0x7f2ad00af420 2024-03-30T00:50:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ac7e00640 1 -- 172.21.15.66:0/89577246 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00b8b60 con 0x7f2ad00af420 2024-03-30T00:50:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.243+0000 7f2ad6200640 1 -- 172.21.15.66:0/89577246 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad00b4e30 con 0x7f2ad00af420 2024-03-30T00:50:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.247+0000 7f2ad6200640 1 -- 172.21.15.66:0/89577246 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad00b49b0 con 0x7f2ad00af420 2024-03-30T00:50:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.247+0000 7f2ac7e00640 1 -- 172.21.15.66:0/89577246 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2ac00b8cc0 con 0x7f2ad00af420 2024-03-30T00:50:38.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.247+0000 7f2ac7e00640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2aac0666c0 0x7f2aac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.247+0000 7f2ac7e00640 1 -- 172.21.15.66:0/89577246 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2ac011f470 con 0x7f2ad00af420 2024-03-30T00:50:38.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.247+0000 7f2ac6a00640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2aac0666c0 0x7f2aac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.247+0000 7f2ac6a00640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2aac0666c0 0x7f2aac068b80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0165290 tx=0x7f2ab8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:38.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.251+0000 7f2ac7e00640 1 -- 172.21.15.66:0/89577246 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2ad00b49b0 con 0x7f2ad00af420 2024-03-30T00:50:38.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.411+0000 7f2ad6200640 1 -- 172.21.15.66:0/89577246 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ad00b49b0 con 0x7f2ad00af420 2024-03-30T00:50:38.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ac7e00640 1 -- 172.21.15.66:0/89577246 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16329 (secure 0 0 0) 0x7f2ad00b49b0 con 0x7f2ad00af420 2024-03-30T00:50:38.423 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:38.423 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":124,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:38.040176+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:50:38.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 -- 172.21.15.66:0/89577246 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2aac0666c0 msgr2=0x7f2aac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2aac0666c0 0x7f2aac068b80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0165290 tx=0x7f2ab8009190 comp rx=0 tx=0).stop 2024-03-30T00:50:38.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 -- 172.21.15.66:0/89577246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad00af420 msgr2=0x7f2ad0144530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad00af420 0x7f2ad0144530 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00ab000 tx=0x7f2ac0019740 comp rx=0 tx=0).stop 2024-03-30T00:50:38.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 -- 172.21.15.66:0/89577246 shutdown_connections 2024-03-30T00:50:38.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2aac0666c0 0x7f2aac068b80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad015de30 0x7f2ad00c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ad00ce740 0x7f2ad0146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 --2- 172.21.15.66:0/89577246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad00af420 0x7f2ad0144530 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 -- 172.21.15.66:0/89577246 >> 172.21.15.66:0/89577246 conn(0x7f2ad00d0540 msgr2=0x7f2ad006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:38.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 -- 172.21.15.66:0/89577246 shutdown_connections 2024-03-30T00:50:38.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.415+0000 7f2ad6200640 1 -- 172.21.15.66:0/89577246 wait complete. 2024-03-30T00:50:38.445 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:50:38.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.575+0000 7f9728800640 1 Processor -- start 2024-03-30T00:50:38.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.575+0000 7f9728800640 1 -- start start 2024-03-30T00:50:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9728800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f97200cec30 0x7f9720159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9728800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97200cd3e0 con 0x7f97200cda90 2024-03-30T00:50:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9728800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f972015a990 con 0x7f97200cde70 2024-03-30T00:50:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9728800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97200b1b80 con 0x7f97200cec30 2024-03-30T00:50:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9725e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f97200cda90 0x7f97200ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40960/0 (socket says 172.21.15.66:40960) 2024-03-30T00:50:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9725e00640 1 -- 172.21.15.66:0/431916220 learned_addr learned my addr 172.21.15.66:0/431916220 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9725400640 1 --2- 172.21.15.66:0/431916220 >> v2:172.21.15.156:3300/0 conn(0x7f97200cec30 0x7f9720159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9724a00640 1 -- 172.21.15.66:0/431916220 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196015234 0 0) 0x7f97200cd3e0 con 0x7f97200cda90 2024-03-30T00:50:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9724a00640 1 -- 172.21.15.66:0/431916220 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9708005740 con 0x7f97200cda90 2024-03-30T00:50:38.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9725400640 1 -- 172.21.15.66:0/431916220 >> v1:172.21.15.156:6789/0 conn(0x7f97200cde70 legacy=0x7f972015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9724a00640 1 -- 172.21.15.66:0/431916220 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617502898 0 0) 0x7f972015a990 con 0x7f97200cde70 2024-03-30T00:50:38.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9725400640 1 -- 172.21.15.66:0/431916220 >> v1:172.21.15.66:6789/0 conn(0x7f97200cda90 legacy=0x7f97200ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.579+0000 7f9725400640 1 -- 172.21.15.66:0/431916220 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97200b11e0 con 0x7f97200cec30 2024-03-30T00:50:38.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9725400640 1 --2- 172.21.15.66:0/431916220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97200cec30 0x7f9720159c60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f97100097c0 tx=0x7f971001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec988da4f4982585 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:38.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9724a00640 1 -- 172.21.15.66:0/431916220 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97100ab610 con 0x7f97200cec30 2024-03-30T00:50:38.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9724a00640 1 -- 172.21.15.66:0/431916220 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97100ab770 con 0x7f97200cec30 2024-03-30T00:50:38.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9724a00640 1 -- 172.21.15.66:0/431916220 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97100abb00 con 0x7f97200cec30 2024-03-30T00:50:38.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9728800640 1 -- 172.21.15.66:0/431916220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97200cec30 msgr2=0x7f9720159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9728800640 1 --2- 172.21.15.66:0/431916220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97200cec30 0x7f9720159c60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f97100097c0 tx=0x7f971001c410 comp rx=0 tx=0).stop 2024-03-30T00:50:38.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9728800640 1 -- 172.21.15.66:0/431916220 shutdown_connections 2024-03-30T00:50:38.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9728800640 1 --2- 172.21.15.66:0/431916220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97200cec30 0x7f9720159c60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9728800640 1 -- 172.21.15.66:0/431916220 >> 172.21.15.66:0/431916220 conn(0x7f9720069f60 msgr2=0x7f972006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:38.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9728800640 1 -- 172.21.15.66:0/431916220 shutdown_connections 2024-03-30T00:50:38.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9728800640 1 -- 172.21.15.66:0/431916220 wait complete. 2024-03-30T00:50:38.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9728800640 1 Processor -- start 2024-03-30T00:50:38.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.583+0000 7f9728800640 1 -- start start 2024-03-30T00:50:38.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9728800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97200cda90 0x7f97200c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9728800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97200cde70 0x7f97200c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9728800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97200cec30 0x7f97200c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9728800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97200c3750 con 0x7f97200cec30 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9728800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9720156060 con 0x7f97200cda90 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9728800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9720156300 con 0x7f97200cde70 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9726800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97200cec30 0x7f97200c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9726800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97200cec30 0x7f97200c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49132/0 (socket says 172.21.15.66:49132) 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9726800640 1 -- 172.21.15.66:0/3905747766 learned_addr learned my addr 172.21.15.66:0/3905747766 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9725400640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97200cde70 0x7f97200c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9725e00640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97200cda90 0x7f97200c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9726800640 1 -- 172.21.15.66:0/3905747766 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97200cde70 msgr2=0x7f97200c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9726800640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97200cde70 0x7f97200c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9726800640 1 -- 172.21.15.66:0/3905747766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97200cda90 msgr2=0x7f97200c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9726800640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97200cda90 0x7f97200c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9726800640 1 -- 172.21.15.66:0/3905747766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9710009420 con 0x7f97200cec30 2024-03-30T00:50:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9726800640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97200cec30 0x7f97200c4110 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f971c001570 tx=0x7f971c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:38.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f971a000640 1 -- 172.21.15.66:0/3905747766 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c003db0 con 0x7f97200cec30 2024-03-30T00:50:38.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f971a000640 1 -- 172.21.15.66:0/3905747766 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c005070 con 0x7f97200cec30 2024-03-30T00:50:38.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f971a000640 1 -- 172.21.15.66:0/3905747766 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c005420 con 0x7f97200cec30 2024-03-30T00:50:38.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9728800640 1 -- 172.21.15.66:0/3905747766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97201565e0 con 0x7f97200cec30 2024-03-30T00:50:38.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9728800640 1 -- 172.21.15.66:0/3905747766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97200c7280 con 0x7f97200cec30 2024-03-30T00:50:38.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f9728800640 1 -- 172.21.15.66:0/3905747766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972006dec0 con 0x7f97200cec30 2024-03-30T00:50:38.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.587+0000 7f971a000640 1 -- 172.21.15.66:0/3905747766 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f971c005580 con 0x7f97200cec30 2024-03-30T00:50:38.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.591+0000 7f971a000640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9704066690 0x7f9704068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.591+0000 7f971a000640 1 -- 172.21.15.66:0/3905747766 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f971c083b80 con 0x7f97200cec30 2024-03-30T00:50:38.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.591+0000 7f9725e00640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9704066690 0x7f9704068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.591+0000 7f9725e00640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9704066690 0x7f9704068b50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9714007fd0 tx=0x7f9714009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:38.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.595+0000 7f971a000640 1 -- 172.21.15.66:0/3905747766 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f971c050dc0 con 0x7f97200cec30 2024-03-30T00:50:38.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.759+0000 7f9728800640 1 -- 172.21.15.66:0/3905747766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f972015dfc0 con 0x7f97200cec30 2024-03-30T00:50:38.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.759+0000 7f971a000640 1 -- 172.21.15.66:0/3905747766 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16329 (secure 0 0 0) 0x7f971c083e60 con 0x7f97200cec30 2024-03-30T00:50:38.768 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:38.768 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":124,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:38.040176+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:50:38.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 -- 172.21.15.66:0/3905747766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9704066690 msgr2=0x7f9704068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9704066690 0x7f9704068b50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9714007fd0 tx=0x7f9714009040 comp rx=0 tx=0).stop 2024-03-30T00:50:38.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 -- 172.21.15.66:0/3905747766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97200cec30 msgr2=0x7f97200c4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97200cec30 0x7f97200c4110 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f971c001570 tx=0x7f971c00cd20 comp rx=0 tx=0).stop 2024-03-30T00:50:38.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 -- 172.21.15.66:0/3905747766 shutdown_connections 2024-03-30T00:50:38.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9704066690 0x7f9704068b50 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97200cec30 0x7f97200c4110 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97200cde70 0x7f97200c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 --2- 172.21.15.66:0/3905747766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97200cda90 0x7f97200c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 -- 172.21.15.66:0/3905747766 >> 172.21.15.66:0/3905747766 conn(0x7f9720069f60 msgr2=0x7f972015b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:38.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 -- 172.21.15.66:0/3905747766 shutdown_connections 2024-03-30T00:50:38.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.763+0000 7f9728800640 1 -- 172.21.15.66:0/3905747766 wait complete. 2024-03-30T00:50:38.790 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T00:50:38.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.923+0000 7fa529000640 1 Processor -- start 2024-03-30T00:50:38.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa529000640 1 -- start start 2024-03-30T00:50:38.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa529000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa5240b1420 0x7fa524099250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa529000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa524098ac0 con 0x7fa5240cca40 2024-03-30T00:50:38.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa529000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5240cf3f0 con 0x7fa5240b0fb0 2024-03-30T00:50:38.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa529000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5240cf670 con 0x7fa5240b1420 2024-03-30T00:50:38.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa522800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa5240b1420 0x7fa524099250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa522800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa5240b1420 0x7fa524099250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58776/0 (socket says 172.21.15.66:58776) 2024-03-30T00:50:38.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa522800640 1 -- 172.21.15.66:0/3184459943 learned_addr learned my addr 172.21.15.66:0/3184459943 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:38.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa522800640 1 -- 172.21.15.66:0/3184459943 >> v1:172.21.15.156:6789/0 conn(0x7fa5240b0fb0 legacy=0x7fa52406b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:38.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa522800640 1 -- 172.21.15.66:0/3184459943 >> v1:172.21.15.66:6789/0 conn(0x7fa5240cca40 legacy=0x7fa5240cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa522800640 1 -- 172.21.15.66:0/3184459943 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5240cf950 con 0x7fa5240b1420 2024-03-30T00:50:38.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.927+0000 7fa521400640 1 -- 172.21.15.66:0/3184459943 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468539854 0 0) 0x7fa524098ac0 con 0x7fa5240cca40 2024-03-30T00:50:38.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa522800640 1 --2- 172.21.15.66:0/3184459943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5240b1420 0x7fa524099250 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009870 tx=0x7fa51c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c1f197d6b36070 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:38.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa521400640 1 -- 172.21.15.66:0/3184459943 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0ab680 con 0x7fa5240b1420 2024-03-30T00:50:38.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa521400640 1 -- 172.21.15.66:0/3184459943 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c0ab7e0 con 0x7fa5240b1420 2024-03-30T00:50:38.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa521400640 1 -- 172.21.15.66:0/3184459943 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0abb70 con 0x7fa5240b1420 2024-03-30T00:50:38.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa529000640 1 -- 172.21.15.66:0/3184459943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5240b1420 msgr2=0x7fa524099250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa529000640 1 --2- 172.21.15.66:0/3184459943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5240b1420 0x7fa524099250 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009870 tx=0x7fa51c01c4f0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa529000640 1 -- 172.21.15.66:0/3184459943 shutdown_connections 2024-03-30T00:50:38.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa529000640 1 --2- 172.21.15.66:0/3184459943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5240b1420 0x7fa524099250 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa529000640 1 -- 172.21.15.66:0/3184459943 >> 172.21.15.66:0/3184459943 conn(0x7fa5240b2130 msgr2=0x7fa5240b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:38.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa529000640 1 -- 172.21.15.66:0/3184459943 shutdown_connections 2024-03-30T00:50:38.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa529000640 1 -- 172.21.15.66:0/3184459943 wait complete. 2024-03-30T00:50:38.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa529000640 1 Processor -- start 2024-03-30T00:50:38.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.931+0000 7fa529000640 1 -- start start 2024-03-30T00:50:38.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa529000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa5240b0fb0 0x7fa5241502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa529000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5240b1420 0x7fa5240cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa529000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5240cca40 0x7fa5241511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa529000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa524154100 con 0x7fa5240cca40 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa529000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa524153340 con 0x7fa5240b1420 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa529000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa524153620 con 0x7fa5240b0fb0 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa522800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5240cca40 0x7fa5241511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa522800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5240cca40 0x7fa5241511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58780/0 (socket says 172.21.15.66:58780) 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa522800640 1 -- 172.21.15.66:0/2058266218 learned_addr learned my addr 172.21.15.66:0/2058266218 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa522800640 1 -- 172.21.15.66:0/2058266218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa5240b0fb0 msgr2=0x7fa5241502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa519400640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5240b1420 0x7fa5240cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa521e00640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa5240b0fb0 0x7fa5241502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa522800640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa5240b0fb0 0x7fa5241502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa522800640 1 -- 172.21.15.66:0/2058266218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5240b1420 msgr2=0x7fa5240cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa522800640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5240b1420 0x7fa5240cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa522800640 1 -- 172.21.15.66:0/2058266218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c0094d0 con 0x7fa5240cca40 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa521e00640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa5240b0fb0 0x7fa5241502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa519400640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5240b1420 0x7fa5240cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:38.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa522800640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5240cca40 0x7fa5241511e0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0ad040 tx=0x7fa51c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:38.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa51aa00640 1 -- 172.21.15.66:0/2058266218 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0b48c0 con 0x7fa5240cca40 2024-03-30T00:50:38.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa51aa00640 1 -- 172.21.15.66:0/2058266218 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c0b4a40 con 0x7fa5240cca40 2024-03-30T00:50:38.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa51aa00640 1 -- 172.21.15.66:0/2058266218 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0b4df0 con 0x7fa5240cca40 2024-03-30T00:50:38.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa529000640 1 -- 172.21.15.66:0/2058266218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa524153900 con 0x7fa5240cca40 2024-03-30T00:50:38.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa529000640 1 -- 172.21.15.66:0/2058266218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5240b6110 con 0x7fa5240cca40 2024-03-30T00:50:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa529000640 1 -- 172.21.15.66:0/2058266218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5240677d0 con 0x7fa5240cca40 2024-03-30T00:50:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.935+0000 7fa51aa00640 1 -- 172.21.15.66:0/2058266218 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa51c0c7440 con 0x7fa5240cca40 2024-03-30T00:50:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.939+0000 7fa51aa00640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa500066690 0x7fa500068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.939+0000 7fa51aa00640 1 -- 172.21.15.66:0/2058266218 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa51c11f2b0 con 0x7fa5240cca40 2024-03-30T00:50:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.939+0000 7fa521e00640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa500066690 0x7fa500068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.939+0000 7fa521e00640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa500066690 0x7fa500068b50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa514007fd0 tx=0x7fa514009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:38.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:38.943+0000 7fa51aa00640 1 -- 172.21.15.66:0/2058266218 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa51c0ec4d0 con 0x7fa5240cca40 2024-03-30T00:50:39.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:39.139+0000 7fa529000640 1 -- 172.21.15.66:0/2058266218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa524153760 con 0x7fa5240cca40 2024-03-30T00:50:40.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.051+0000 7fa51aa00640 1 -- 172.21.15.66:0/2058266218 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v94) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa524153760 con 0x7fa5240cca40 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 -- 172.21.15.66:0/2058266218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa500066690 msgr2=0x7fa500068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa500066690 0x7fa500068b50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa514007fd0 tx=0x7fa514009040 comp rx=0 tx=0).stop 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 -- 172.21.15.66:0/2058266218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5240cca40 msgr2=0x7fa5241511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5240cca40 0x7fa5241511e0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0ad040 tx=0x7fa51c01ad30 comp rx=0 tx=0).stop 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 -- 172.21.15.66:0/2058266218 shutdown_connections 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa500066690 0x7fa500068b50 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5240cca40 0x7fa5241511e0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5240b1420 0x7fa5240cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 --2- 172.21.15.66:0/2058266218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa5240b0fb0 0x7fa5241502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 -- 172.21.15.66:0/2058266218 >> 172.21.15.66:0/2058266218 conn(0x7fa5240b2130 msgr2=0x7fa524158c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 -- 172.21.15.66:0/2058266218 shutdown_connections 2024-03-30T00:50:40.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.055+0000 7fa529000640 1 -- 172.21.15.66:0/2058266218 wait complete. 2024-03-30T00:50:40.085 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T00:50:40.085 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T00:50:40.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.223+0000 7f2abd400640 1 Processor -- start 2024-03-30T00:50:40.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.223+0000 7f2abd400640 1 -- start start 2024-03-30T00:50:40.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2abd400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2ab815fd70 0x7f2ab8069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:40.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2abd400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ab806d600 0x7f2ab806a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:40.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2abd400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab80cbaf0 con 0x7f2ab80cedb0 2024-03-30T00:50:40.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2abd400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab806d1e0 con 0x7f2ab815fd70 2024-03-30T00:50:40.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2abd400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80cbd40 con 0x7f2ab806d600 2024-03-30T00:50:40.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2ab7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2ab815fd70 0x7f2ab8069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:40.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2aaf400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2ab80cedb0 0x7f2ab8070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53464/0 (socket says 172.21.15.66:53464) 2024-03-30T00:50:40.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2ab7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2ab815fd70 0x7f2ab8069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58794/0 (socket says 172.21.15.66:58794) 2024-03-30T00:50:40.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2aaf400640 1 -- 172.21.15.66:0/1134568951 learned_addr learned my addr 172.21.15.66:0/1134568951 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:40.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2abca00640 1 --2- 172.21.15.66:0/1134568951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ab806d600 0x7f2ab806a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:40.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2ab7e00640 1 -- 172.21.15.66:0/1134568951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ab806d600 msgr2=0x7f2ab806a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:40.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2ab7e00640 1 --2- 172.21.15.66:0/1134568951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ab806d600 0x7f2ab806a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:40.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2ab7e00640 1 -- 172.21.15.66:0/1134568951 >> v1:172.21.15.66:6789/0 conn(0x7f2ab80cedb0 legacy=0x7f2ab8070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:40.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2ab7400640 1 -- 172.21.15.66:0/1134568951 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121287603 0 0) 0x7f2ab80cbaf0 con 0x7f2ab80cedb0 2024-03-30T00:50:40.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2ab7e00640 1 -- 172.21.15.66:0/1134568951 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab80cc020 con 0x7f2ab815fd70 2024-03-30T00:50:40.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.227+0000 7f2ab7e00640 1 --2- 172.21.15.66:0/1134568951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab815fd70 0x7f2ab8069d90 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80097c0 tx=0x7f2aa801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c4c6aaf08e4bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:40.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2ab7400640 1 -- 172.21.15.66:0/1134568951 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80ad680 con 0x7f2ab815fd70 2024-03-30T00:50:40.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2ab7400640 1 -- 172.21.15.66:0/1134568951 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa80ad7e0 con 0x7f2ab815fd70 2024-03-30T00:50:40.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2ab7400640 1 -- 172.21.15.66:0/1134568951 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80adb70 con 0x7f2ab815fd70 2024-03-30T00:50:40.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 -- 172.21.15.66:0/1134568951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab815fd70 msgr2=0x7f2ab8069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:40.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 --2- 172.21.15.66:0/1134568951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab815fd70 0x7f2ab8069d90 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80097c0 tx=0x7f2aa801c3e0 comp rx=0 tx=0).stop 2024-03-30T00:50:40.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 -- 172.21.15.66:0/1134568951 shutdown_connections 2024-03-30T00:50:40.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 --2- 172.21.15.66:0/1134568951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ab806d600 0x7f2ab806a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:40.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 --2- 172.21.15.66:0/1134568951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab815fd70 0x7f2ab8069d90 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:40.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 -- 172.21.15.66:0/1134568951 >> 172.21.15.66:0/1134568951 conn(0x7f2ab80b0ea0 msgr2=0x7f2ab80b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:40.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 -- 172.21.15.66:0/1134568951 shutdown_connections 2024-03-30T00:50:40.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 -- 172.21.15.66:0/1134568951 wait complete. 2024-03-30T00:50:40.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 Processor -- start 2024-03-30T00:50:40.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 -- start start 2024-03-30T00:50:40.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ab806d600 0x7f2ab814bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab80cedb0 0x7f2ab80ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ab815fd70 0x7f2ab80af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab8154f00 con 0x7f2ab80cedb0 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab814c320 con 0x7f2ab815fd70 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abd400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab81542f0 con 0x7f2ab806d600 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2aaf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab80cedb0 0x7f2ab80ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2aaf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab80cedb0 0x7f2ab80ab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58804/0 (socket says 172.21.15.66:58804) 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2aaf400640 1 -- 172.21.15.66:0/2184485960 learned_addr learned my addr 172.21.15.66:0/2184485960 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2abca00640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ab815fd70 0x7f2ab80af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.231+0000 7f2ab7e00640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ab806d600 0x7f2ab814bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2aaf400640 1 -- 172.21.15.66:0/2184485960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ab806d600 msgr2=0x7f2ab814bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:40.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2aaf400640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ab806d600 0x7f2ab814bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:40.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2aaf400640 1 -- 172.21.15.66:0/2184485960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ab815fd70 msgr2=0x7f2ab80af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:40.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2aaf400640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ab815fd70 0x7f2ab80af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:40.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2aaf400640 1 -- 172.21.15.66:0/2184485960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa8009420 con 0x7f2ab80cedb0 2024-03-30T00:50:40.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2ab7e00640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ab806d600 0x7f2ab814bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:40.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2aaf400640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab80cedb0 0x7f2ab80ab890 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000c450 tx=0x7f2aa000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:40.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2ab4c00640 1 -- 172.21.15.66:0/2184485960 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00074e0 con 0x7f2ab80cedb0 2024-03-30T00:50:40.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2ab4c00640 1 -- 172.21.15.66:0/2184485960 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa0007640 con 0x7f2ab80cedb0 2024-03-30T00:50:40.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2ab4c00640 1 -- 172.21.15.66:0/2184485960 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00049f0 con 0x7f2ab80cedb0 2024-03-30T00:50:40.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2abd400640 1 -- 172.21.15.66:0/2184485960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab81545d0 con 0x7f2ab80cedb0 2024-03-30T00:50:40.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2abd400640 1 -- 172.21.15.66:0/2184485960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab8154b90 con 0x7f2ab80cedb0 2024-03-30T00:50:40.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2abd400640 1 -- 172.21.15.66:0/2184485960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab81550b0 con 0x7f2ab80cedb0 2024-03-30T00:50:40.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2ab4c00640 1 -- 172.21.15.66:0/2184485960 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2aa0004b50 con 0x7f2ab80cedb0 2024-03-30T00:50:40.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2ab4c00640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a940666c0 0x7f2a94068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:40.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2ab4c00640 1 -- 172.21.15.66:0/2184485960 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2aa0083480 con 0x7f2ab80cedb0 2024-03-30T00:50:40.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.235+0000 7f2ab7e00640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a940666c0 0x7f2a94068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:40.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.239+0000 7f2ab7e00640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a940666c0 0x7f2a94068b80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80af000 tx=0x7f2aa80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:40.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.239+0000 7f2ab4c00640 1 -- 172.21.15.66:0/2184485960 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2ab81550b0 con 0x7f2ab80cedb0 2024-03-30T00:50:40.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:40.439+0000 7f2abd400640 1 -- 172.21.15.66:0/2184485960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2ab8073cf0 con 0x7f2ab80cedb0 2024-03-30T00:50:41.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.055+0000 7f2ab4c00640 1 -- 172.21.15.66:0/2184485960 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v95) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2aa00180c0 con 0x7f2ab80cedb0 2024-03-30T00:50:41.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 -- 172.21.15.66:0/2184485960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a940666c0 msgr2=0x7f2a94068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a940666c0 0x7f2a94068b80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80af000 tx=0x7f2aa80a7000 comp rx=0 tx=0).stop 2024-03-30T00:50:41.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 -- 172.21.15.66:0/2184485960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab80cedb0 msgr2=0x7f2ab80ab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab80cedb0 0x7f2ab80ab890 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000c450 tx=0x7f2aa000c910 comp rx=0 tx=0).stop 2024-03-30T00:50:41.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 -- 172.21.15.66:0/2184485960 shutdown_connections 2024-03-30T00:50:41.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ab815fd70 0x7f2ab80af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2a940666c0 0x7f2a94068b80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab80cedb0 0x7f2ab80ab890 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 --2- 172.21.15.66:0/2184485960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2ab806d600 0x7f2ab814bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 -- 172.21.15.66:0/2184485960 >> 172.21.15.66:0/2184485960 conn(0x7f2ab80b0ea0 msgr2=0x7f2ab80cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:41.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 -- 172.21.15.66:0/2184485960 shutdown_connections 2024-03-30T00:50:41.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.059+0000 7f2abd400640 1 -- 172.21.15.66:0/2184485960 wait complete. 2024-03-30T00:50:41.089 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T00:50:41.089 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:50:41.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f67800640 1 Processor -- start 2024-03-30T00:50:41.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f67800640 1 -- start start 2024-03-30T00:50:41.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f67800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f600cda90 0x7f4f600ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f67800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f600cde70 0x7f4f600cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f67800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6015a990 con 0x7f4f60159c60 2024-03-30T00:50:41.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f67800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f600b1b80 con 0x7f4f600cde70 2024-03-30T00:50:41.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f67800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f600cd3e0 con 0x7f4f600cda90 2024-03-30T00:50:41.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f5fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f600cde70 0x7f4f600cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f5fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f600cde70 0x7f4f600cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58816/0 (socket says 172.21.15.66:58816) 2024-03-30T00:50:41.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f5fe00640 1 -- 172.21.15.66:0/3414435853 learned_addr learned my addr 172.21.15.66:0/3414435853 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:41.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f65800640 1 --1- 172.21.15.66:0/3414435853 >> v1:172.21.15.66:6789/0 conn(0x7f4f60159c60 0x7f4f6015a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53470/0 (socket says 172.21.15.66:53470) 2024-03-30T00:50:41.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f64e00640 1 --2- 172.21.15.66:0/3414435853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f600cda90 0x7f4f600ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f5fe00640 1 -- 172.21.15.66:0/3414435853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f600cda90 msgr2=0x7f4f600ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:41.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f5fe00640 1 --2- 172.21.15.66:0/3414435853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f600cda90 0x7f4f600ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f5fe00640 1 -- 172.21.15.66:0/3414435853 >> v1:172.21.15.66:6789/0 conn(0x7f4f60159c60 legacy=0x7f4f6015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f5fe00640 1 -- 172.21.15.66:0/3414435853 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f600b11e0 con 0x7f4f600cde70 2024-03-30T00:50:41.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.223+0000 7f4f64e00640 1 --2- 172.21.15.66:0/3414435853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f600cda90 0x7f4f600ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f5fe00640 1 --2- 172.21.15.66:0/3414435853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f600cde70 0x7f4f600cea60 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009870 tx=0x7f4f4c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef7cbfa21058ac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f5f400640 1 -- 172.21.15.66:0/3414435853 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0ac640 con 0x7f4f600cde70 2024-03-30T00:50:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f5f400640 1 -- 172.21.15.66:0/3414435853 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c0ac7a0 con 0x7f4f600cde70 2024-03-30T00:50:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f5f400640 1 -- 172.21.15.66:0/3414435853 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0acb30 con 0x7f4f600cde70 2024-03-30T00:50:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f67800640 1 -- 172.21.15.66:0/3414435853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f600cde70 msgr2=0x7f4f600cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f67800640 1 --2- 172.21.15.66:0/3414435853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f600cde70 0x7f4f600cea60 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009870 tx=0x7f4f4c01c4d0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f67800640 1 -- 172.21.15.66:0/3414435853 shutdown_connections 2024-03-30T00:50:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f67800640 1 --2- 172.21.15.66:0/3414435853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f600cde70 0x7f4f600cea60 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f67800640 1 --2- 172.21.15.66:0/3414435853 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f600cda90 0x7f4f600ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f67800640 1 -- 172.21.15.66:0/3414435853 >> 172.21.15.66:0/3414435853 conn(0x7f4f60069f60 msgr2=0x7f4f6006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f67800640 1 -- 172.21.15.66:0/3414435853 shutdown_connections 2024-03-30T00:50:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.227+0000 7f4f67800640 1 -- 172.21.15.66:0/3414435853 wait complete. 2024-03-30T00:50:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f67800640 1 Processor -- start 2024-03-30T00:50:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f67800640 1 -- start start 2024-03-30T00:50:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f67800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f600cda90 0x7f4f60155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f67800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f600cde70 0x7f4f60071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f67800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f60159c60 0x7f4f6006c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f67800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f601579f0 con 0x7f4f600cde70 2024-03-30T00:50:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f67800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f601576a0 con 0x7f4f600cda90 2024-03-30T00:50:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f67800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f600abba0 con 0x7f4f60159c60 2024-03-30T00:50:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f600cde70 0x7f4f60071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f600cde70 0x7f4f60071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58820/0 (socket says 172.21.15.66:58820) 2024-03-30T00:50:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5fe00640 1 -- 172.21.15.66:0/1044231703 learned_addr learned my addr 172.21.15.66:0/1044231703 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f64e00640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f600cda90 0x7f4f60155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f65800640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f60159c60 0x7f4f6006c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5fe00640 1 -- 172.21.15.66:0/1044231703 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f60159c60 msgr2=0x7f4f6006c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5fe00640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f60159c60 0x7f4f6006c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5fe00640 1 -- 172.21.15.66:0/1044231703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f600cda90 msgr2=0x7f4f60155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5fe00640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f600cda90 0x7f4f60155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5fe00640 1 -- 172.21.15.66:0/1044231703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c0094d0 con 0x7f4f600cde70 2024-03-30T00:50:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5fe00640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f600cde70 0x7f4f60071660 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c01a100 tx=0x7f4f4c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:41.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5cc00640 1 -- 172.21.15.66:0/1044231703 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0b7860 con 0x7f4f600cde70 2024-03-30T00:50:41.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5cc00640 1 -- 172.21.15.66:0/1044231703 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c0b79c0 con 0x7f4f600cde70 2024-03-30T00:50:41.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f67800640 1 -- 172.21.15.66:0/1044231703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f600abe80 con 0x7f4f600cde70 2024-03-30T00:50:41.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f5cc00640 1 -- 172.21.15.66:0/1044231703 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0b7d30 con 0x7f4f600cde70 2024-03-30T00:50:41.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.231+0000 7f4f67800640 1 -- 172.21.15.66:0/1044231703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f60071ba0 con 0x7f4f600cde70 2024-03-30T00:50:41.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.235+0000 7f4f67800640 1 -- 172.21.15.66:0/1044231703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f60157b30 con 0x7f4f600cde70 2024-03-30T00:50:41.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.235+0000 7f4f5cc00640 1 -- 172.21.15.66:0/1044231703 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4f4c0b3070 con 0x7f4f600cde70 2024-03-30T00:50:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.235+0000 7f4f5cc00640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f40066690 0x7f4f40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.235+0000 7f4f5cc00640 1 -- 172.21.15.66:0/1044231703 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4f4c11ea90 con 0x7f4f600cde70 2024-03-30T00:50:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.235+0000 7f4f64e00640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f40066690 0x7f4f40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.235+0000 7f4f64e00640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f40066690 0x7f4f40068b50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4f540045f0 tx=0x7f4f54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:41.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.239+0000 7f4f5cc00640 1 -- 172.21.15.66:0/1044231703 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4f60157b30 con 0x7f4f600cde70 2024-03-30T00:50:41.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 -- 172.21.15.66:0/1044231703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f60157b30 con 0x7f4f600cde70 2024-03-30T00:50:41.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f5cc00640 1 -- 172.21.15.66:0/1044231703 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f4f60157b30 con 0x7f4f600cde70 2024-03-30T00:50:41.443 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:41.443 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":95,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:50:41.054441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9366":{"gid":9366,"name":"a","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6836/2683418851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2683418851},{"type":"v1","addr":"172.21.15.66:6836","nonce":2683418851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:50:41.444 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 95 2024-03-30T00:50:41.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 -- 172.21.15.66:0/1044231703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f40066690 msgr2=0x7f4f40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f40066690 0x7f4f40068b50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4f540045f0 tx=0x7f4f54008040 comp rx=0 tx=0).stop 2024-03-30T00:50:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 -- 172.21.15.66:0/1044231703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f600cde70 msgr2=0x7f4f60071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f600cde70 0x7f4f60071660 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c01a100 tx=0x7f4f4c01ad10 comp rx=0 tx=0).stop 2024-03-30T00:50:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 -- 172.21.15.66:0/1044231703 shutdown_connections 2024-03-30T00:50:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f60159c60 0x7f4f6006c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4f40066690 0x7f4f40068b50 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f600cde70 0x7f4f60071660 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 --2- 172.21.15.66:0/1044231703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f600cda90 0x7f4f60155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 -- 172.21.15.66:0/1044231703 >> 172.21.15.66:0/1044231703 conn(0x7f4f60069f60 msgr2=0x7f4f6006a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 -- 172.21.15.66:0/1044231703 shutdown_connections 2024-03-30T00:50:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.435+0000 7f4f67800640 1 -- 172.21.15.66:0/1044231703 wait complete. 2024-03-30T00:50:41.463 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:50:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1b4c00640 1 Processor -- start 2024-03-30T00:50:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1b4c00640 1 -- start start 2024-03-30T00:50:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1b4c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1b0154b70 0x7fb1b006d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1b4c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb1b015b5d0 0x7fb1b00ab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1b4c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b006d8a0 con 0x7fb1b00ab660 2024-03-30T00:50:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1b4c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b00abad0 con 0x7fb1b015b5d0 2024-03-30T00:50:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1b4c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b00ad420 con 0x7fb1b0154b70 2024-03-30T00:50:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1afe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb1b00ab660 0x7fb1b00b1140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53480/0 (socket says 172.21.15.66:53480) 2024-03-30T00:50:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1afe00640 1 -- 172.21.15.66:0/895372621 learned_addr learned my addr 172.21.15.66:0/895372621 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1a6a00640 1 --2- 172.21.15.66:0/895372621 >> v2:172.21.15.156:3300/0 conn(0x7fb1b015b5d0 0x7fb1b00ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1af400640 1 --2- 172.21.15.66:0/895372621 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1b0154b70 0x7fb1b006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1aea00640 1 -- 172.21.15.66:0/895372621 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890407012 0 0) 0x7fb1a8001120 con 0x7fb1b00ab660 2024-03-30T00:50:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1aea00640 1 -- 172.21.15.66:0/895372621 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb194003530 con 0x7fb1b00ab660 2024-03-30T00:50:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1a6a00640 1 -- 172.21.15.66:0/895372621 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1b0154b70 msgr2=0x7fb1b006d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1a6a00640 1 --2- 172.21.15.66:0/895372621 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1b0154b70 0x7fb1b006d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1a6a00640 1 -- 172.21.15.66:0/895372621 >> v1:172.21.15.66:6789/0 conn(0x7fb1b00ab660 legacy=0x7fb1b00b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.595+0000 7fb1a6a00640 1 -- 172.21.15.66:0/895372621 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b006c2c0 con 0x7fb1b015b5d0 2024-03-30T00:50:41.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1a6a00640 1 --2- 172.21.15.66:0/895372621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b015b5d0 0x7fb1b00ab120 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb1980097c0 tx=0x7fb19801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9767301e9175eb29 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:41.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1aea00640 1 -- 172.21.15.66:0/895372621 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980ab670 con 0x7fb1b015b5d0 2024-03-30T00:50:41.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1aea00640 1 -- 172.21.15.66:0/895372621 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1980ab7d0 con 0x7fb1b015b5d0 2024-03-30T00:50:41.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1aea00640 1 -- 172.21.15.66:0/895372621 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980abb40 con 0x7fb1b015b5d0 2024-03-30T00:50:41.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1b4c00640 1 -- 172.21.15.66:0/895372621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b015b5d0 msgr2=0x7fb1b00ab120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1b4c00640 1 --2- 172.21.15.66:0/895372621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b015b5d0 0x7fb1b00ab120 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb1980097c0 tx=0x7fb19801c470 comp rx=0 tx=0).stop 2024-03-30T00:50:41.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1b4c00640 1 -- 172.21.15.66:0/895372621 shutdown_connections 2024-03-30T00:50:41.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1b4c00640 1 --2- 172.21.15.66:0/895372621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b015b5d0 0x7fb1b00ab120 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1b4c00640 1 --2- 172.21.15.66:0/895372621 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1b0154b70 0x7fb1b006d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1b4c00640 1 -- 172.21.15.66:0/895372621 >> 172.21.15.66:0/895372621 conn(0x7fb1b00afe40 msgr2=0x7fb1b00b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:41.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1b4c00640 1 -- 172.21.15.66:0/895372621 shutdown_connections 2024-03-30T00:50:41.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.599+0000 7fb1b4c00640 1 -- 172.21.15.66:0/895372621 wait complete. 2024-03-30T00:50:41.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1b4c00640 1 Processor -- start 2024-03-30T00:50:41.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1b4c00640 1 -- start start 2024-03-30T00:50:41.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1b4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1b00ab660 0x7fb1b014ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1b4c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b0154b70 0x7fb1b014d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1b4c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1b015b5d0 0x7fb1b015c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1b4c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0151a30 con 0x7fb1b00ab660 2024-03-30T00:50:41.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1b4c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b01509b0 con 0x7fb1b0154b70 2024-03-30T00:50:41.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1b4c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0150c90 con 0x7fb1b015b5d0 2024-03-30T00:50:41.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1af400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1b00ab660 0x7fb1b014ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1af400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1b00ab660 0x7fb1b014ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58826/0 (socket says 172.21.15.66:58826) 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1af400640 1 -- 172.21.15.66:0/4289147120 learned_addr learned my addr 172.21.15.66:0/4289147120 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1afe00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1b015b5d0 0x7fb1b015c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1a6a00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b0154b70 0x7fb1b014d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1af400640 1 -- 172.21.15.66:0/4289147120 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1b015b5d0 msgr2=0x7fb1b015c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1af400640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1b015b5d0 0x7fb1b015c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1af400640 1 -- 172.21.15.66:0/4289147120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b0154b70 msgr2=0x7fb1b014d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1af400640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b0154b70 0x7fb1b014d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1af400640 1 -- 172.21.15.66:0/4289147120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb198009420 con 0x7fb1b00ab660 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1a6a00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b0154b70 0x7fb1b014d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1af400640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1b00ab660 0x7fb1b014ff80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000efc0 tx=0x7fb1a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1a7e00640 1 -- 172.21.15.66:0/4289147120 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0009b80 con 0x7fb1b00ab660 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1a7e00640 1 -- 172.21.15.66:0/4289147120 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a000f040 con 0x7fb1b00ab660 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1a7e00640 1 -- 172.21.15.66:0/4289147120 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00146a0 con 0x7fb1b00ab660 2024-03-30T00:50:41.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1b4c00640 1 -- 172.21.15.66:0/4289147120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b0150f70 con 0x7fb1b00ab660 2024-03-30T00:50:41.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.603+0000 7fb1b4c00640 1 -- 172.21.15.66:0/4289147120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b01504c0 con 0x7fb1b00ab660 2024-03-30T00:50:41.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.607+0000 7fb1b4c00640 1 -- 172.21.15.66:0/4289147120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b0150740 con 0x7fb1b00ab660 2024-03-30T00:50:41.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.607+0000 7fb1a7e00640 1 -- 172.21.15.66:0/4289147120 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb1a0014800 con 0x7fb1b00ab660 2024-03-30T00:50:41.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.607+0000 7fb1a7e00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb18c066690 0x7fb18c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.607+0000 7fb1a7e00640 1 -- 172.21.15.66:0/4289147120 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb1a0084e70 con 0x7fb1b00ab660 2024-03-30T00:50:41.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.607+0000 7fb1a6a00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb18c066690 0x7fb18c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.607+0000 7fb1a6a00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb18c066690 0x7fb18c068b50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb1980af000 tx=0x7fb1980a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:41.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.611+0000 7fb1a7e00640 1 -- 172.21.15.66:0/4289147120 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb1a0052010 con 0x7fb1b00ab660 2024-03-30T00:50:41.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.771+0000 7fb1b4c00640 1 -- 172.21.15.66:0/4289147120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1b0073cf0 con 0x7fb1b00ab660 2024-03-30T00:50:41.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1a7e00640 1 -- 172.21.15.66:0/4289147120 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16316 (secure 0 0 0) 0x7fb1a0055630 con 0x7fb1b00ab660 2024-03-30T00:50:41.783 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:41.783 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":125,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:39.043056+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:50:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 -- 172.21.15.66:0/4289147120 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb18c066690 msgr2=0x7fb18c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb18c066690 0x7fb18c068b50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb1980af000 tx=0x7fb1980a7000 comp rx=0 tx=0).stop 2024-03-30T00:50:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 -- 172.21.15.66:0/4289147120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1b00ab660 msgr2=0x7fb1b014ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1b00ab660 0x7fb1b014ff80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000efc0 tx=0x7fb1a000c490 comp rx=0 tx=0).stop 2024-03-30T00:50:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 -- 172.21.15.66:0/4289147120 shutdown_connections 2024-03-30T00:50:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1b015b5d0 0x7fb1b015c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b0154b70 0x7fb1b014d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb18c066690 0x7fb18c068b50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 --2- 172.21.15.66:0/4289147120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1b00ab660 0x7fb1b014ff80 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 -- 172.21.15.66:0/4289147120 >> 172.21.15.66:0/4289147120 conn(0x7fb1b00afe40 msgr2=0x7fb1b00b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:41.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 -- 172.21.15.66:0/4289147120 shutdown_connections 2024-03-30T00:50:41.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.775+0000 7fb1b4c00640 1 -- 172.21.15.66:0/4289147120 wait complete. 2024-03-30T00:50:41.804 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T00:50:41.805 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:50:41.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a22000640 1 Processor -- start 2024-03-30T00:50:41.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a22000640 1 -- start start 2024-03-30T00:50:41.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a22000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0a1c0cc590 0x7f0a1c15c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a22000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1c0afdf0 con 0x7f0a1c160490 2024-03-30T00:50:41.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a22000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1c0b1010 con 0x7f0a1c0b0290 2024-03-30T00:50:41.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a22000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1c0b0e30 con 0x7f0a1c0cc590 2024-03-30T00:50:41.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a1b400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0a1c160490 0x7f0a1c160870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53482/0 (socket says 172.21.15.66:53482) 2024-03-30T00:50:41.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a1b400640 1 -- 172.21.15.66:0/1003827157 learned_addr learned my addr 172.21.15.66:0/1003827157 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:41.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a1be00640 1 --2- 172.21.15.66:0/1003827157 >> v2:172.21.15.66:3300/0 conn(0x7f0a1c0cc590 0x7f0a1c15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a1be00640 1 -- 172.21.15.66:0/1003827157 >> v1:172.21.15.156:6789/0 conn(0x7f0a1c0b0290 legacy=0x7f0a1c0b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a1be00640 1 -- 172.21.15.66:0/1003827157 >> v1:172.21.15.66:6789/0 conn(0x7f0a1c160490 legacy=0x7f0a1c160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.939+0000 7f0a1be00640 1 -- 172.21.15.66:0/1003827157 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c0b12f0 con 0x7f0a1c0cc590 2024-03-30T00:50:41.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a1be00640 1 --2- 172.21.15.66:0/1003827157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a1c0cc590 0x7f0a1c15c200 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f0a14009d90 tx=0x7f0a1401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdd139e18f33d95 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:41.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a1a000640 1 -- 172.21.15.66:0/1003827157 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140a5850 con 0x7f0a1c0cc590 2024-03-30T00:50:41.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a1a000640 1 -- 172.21.15.66:0/1003827157 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a140a59b0 con 0x7f0a1c0cc590 2024-03-30T00:50:41.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a1a000640 1 -- 172.21.15.66:0/1003827157 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140a5d40 con 0x7f0a1c0cc590 2024-03-30T00:50:41.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a22000640 1 -- 172.21.15.66:0/1003827157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a1c0cc590 msgr2=0x7f0a1c15c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a22000640 1 --2- 172.21.15.66:0/1003827157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a1c0cc590 0x7f0a1c15c200 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f0a14009d90 tx=0x7f0a1401c8a0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a22000640 1 -- 172.21.15.66:0/1003827157 shutdown_connections 2024-03-30T00:50:41.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a22000640 1 --2- 172.21.15.66:0/1003827157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a1c0cc590 0x7f0a1c15c200 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a22000640 1 -- 172.21.15.66:0/1003827157 >> 172.21.15.66:0/1003827157 conn(0x7f0a1c099870 msgr2=0x7f0a1c089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:41.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a22000640 1 -- 172.21.15.66:0/1003827157 shutdown_connections 2024-03-30T00:50:41.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a22000640 1 -- 172.21.15.66:0/1003827157 wait complete. 2024-03-30T00:50:41.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a22000640 1 Processor -- start 2024-03-30T00:50:41.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.943+0000 7f0a22000640 1 -- start start 2024-03-30T00:50:41.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a22000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a1c0b0290 0x7f0a1c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a22000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a1c0cc590 0x7f0a1c152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a22000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a1c160490 0x7f0a1c0cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a22000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c158160 con 0x7f0a1c0b0290 2024-03-30T00:50:41.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a22000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c15d4d0 con 0x7f0a1c160490 2024-03-30T00:50:41.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a22000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c155480 con 0x7f0a1c0cc590 2024-03-30T00:50:41.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a1c0b0290 0x7f0a1c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a1c0b0290 0x7f0a1c15cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58852/0 (socket says 172.21.15.66:58852) 2024-03-30T00:50:41.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1b400640 1 -- 172.21.15.66:0/1845713390 learned_addr learned my addr 172.21.15.66:0/1845713390 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1be00640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a1c160490 0x7f0a1c0cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1aa00640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a1c0cc590 0x7f0a1c152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1b400640 1 -- 172.21.15.66:0/1845713390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a1c0cc590 msgr2=0x7f0a1c152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1b400640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a1c0cc590 0x7f0a1c152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1b400640 1 -- 172.21.15.66:0/1845713390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a1c160490 msgr2=0x7f0a1c0cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1b400640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a1c160490 0x7f0a1c0cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1b400640 1 -- 172.21.15.66:0/1845713390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a14009a70 con 0x7f0a1c0b0290 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1be00640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a1c160490 0x7f0a1c0cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a1b400640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a1c0b0290 0x7f0a1c15cf90 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00c530 tx=0x7f0a0c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a13400640 1 -- 172.21.15.66:0/1845713390 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c005cf0 con 0x7f0a1c0b0290 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a13400640 1 -- 172.21.15.66:0/1845713390 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c005e50 con 0x7f0a1c0b0290 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a13400640 1 -- 172.21.15.66:0/1845713390 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c007270 con 0x7f0a1c0b0290 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a22000640 1 -- 172.21.15.66:0/1845713390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1c1583b0 con 0x7f0a1c0b0290 2024-03-30T00:50:41.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a22000640 1 -- 172.21.15.66:0/1845713390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1c158970 con 0x7f0a1c0b0290 2024-03-30T00:50:41.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a22000640 1 -- 172.21.15.66:0/1845713390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1c066c30 con 0x7f0a1c0b0290 2024-03-30T00:50:41.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.947+0000 7f0a13400640 1 -- 172.21.15.66:0/1845713390 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0a0c0073d0 con 0x7f0a1c0b0290 2024-03-30T00:50:41.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.951+0000 7f0a13400640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09fc066690 0x7f09fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:41.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.951+0000 7f0a13400640 1 -- 172.21.15.66:0/1845713390 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0a0c0837a0 con 0x7f0a1c0b0290 2024-03-30T00:50:41.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.951+0000 7f0a1aa00640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09fc066690 0x7f09fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:41.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.951+0000 7f0a1aa00640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09fc066690 0x7f09fc068b50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0b65a0 tx=0x7f0a08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:41.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:41.955+0000 7f0a13400640 1 -- 172.21.15.66:0/1845713390 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0a0c050a70 con 0x7f0a1c0b0290 2024-03-30T00:50:42.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.111+0000 7f0a22000640 1 -- 172.21.15.66:0/1845713390 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a1c06d160 con 0x7f09fc066690 2024-03-30T00:50:42.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.119+0000 7f0a13400640 1 -- 172.21.15.66:0/1845713390 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286510 (secure 0 0 0) 0x7f0a1c06d160 con 0x7f09fc066690 2024-03-30T00:50:42.131 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:42.133 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:50:42.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 -- 172.21.15.66:0/1845713390 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09fc066690 msgr2=0x7f09fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09fc066690 0x7f09fc068b50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0b65a0 tx=0x7f0a08009040 comp rx=0 tx=0).stop 2024-03-30T00:50:42.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 -- 172.21.15.66:0/1845713390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a1c0b0290 msgr2=0x7f0a1c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a1c0b0290 0x7f0a1c15cf90 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00c530 tx=0x7f0a0c00c9f0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 -- 172.21.15.66:0/1845713390 shutdown_connections 2024-03-30T00:50:42.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a1c160490 0x7f0a1c0cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a1c0cc590 0x7f0a1c152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f09fc066690 0x7f09fc068b50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 --2- 172.21.15.66:0/1845713390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a1c0b0290 0x7f0a1c15cf90 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 -- 172.21.15.66:0/1845713390 >> 172.21.15.66:0/1845713390 conn(0x7f0a1c099870 msgr2=0x7f0a1c0aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:42.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 -- 172.21.15.66:0/1845713390 shutdown_connections 2024-03-30T00:50:42.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.123+0000 7f0a22000640 1 -- 172.21.15.66:0/1845713390 wait complete. 2024-03-30T00:50:42.152 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-03-30T00:50:41.000126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261988,"kb_used_data":13044,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712732,"statfs":{"total":773094113280,"available":772825837568,"internally_reserved":0,"allocated":13357056,"data_stored":6784763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.373741"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.639401+0000","last_change":"2024-03-30T00:50:37.639401+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.639401+0000","last_undegraded":"2024-03-30T00:50:37.639401+0000","last_fullsized":"2024-03-30T00:50:37.639401+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646197+0000","last_change":"2024-03-30T00:50:37.646197+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646197+0000","last_undegraded":"2024-03-30T00:50:37.646197+0000","last_fullsized":"2024-03-30T00:50:37.646197+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643997+0000","last_change":"2024-03-30T00:50:37.643997+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643997+0000","last_undegraded":"2024-03-30T00:50:37.643997+0000","last_fullsized":"2024-03-30T00:50:37.643997+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:53:58.506502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637414+0000","last_change":"2024-03-30T00:50:37.637414+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637414+0000","last_undegraded":"2024-03-30T00:50:37.637414+0000","last_fullsized":"2024-03-30T00:50:37.637414+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638917+0000","last_change":"2024-03-30T00:50:37.638917+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638917+0000","last_undegraded":"2024-03-30T00:50:37.638917+0000","last_fullsized":"2024-03-30T00:50:37.638917+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637248+0000","last_change":"2024-03-30T00:50:37.637248+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637248+0000","last_undegraded":"2024-03-30T00:50:37.637248+0000","last_fullsized":"2024-03-30T00:50:37.637248+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:16.922513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636753+0000","last_change":"2024-03-30T00:50:37.636753+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636753+0000","last_undegraded":"2024-03-30T00:50:37.636753+0000","last_fullsized":"2024-03-30T00:50:37.636753+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:10:54.265531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642083+0000","last_change":"2024-03-30T00:50:37.642083+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642083+0000","last_undegraded":"2024-03-30T00:50:37.642083+0000","last_fullsized":"2024-03-30T00:50:37.642083+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638063+0000","last_change":"2024-03-30T00:50:37.638063+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638063+0000","last_undegraded":"2024-03-30T00:50:37.638063+0000","last_fullsized":"2024-03-30T00:50:37.638063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.639455+0000","last_change":"2024-03-30T00:50:37.639455+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.639455+0000","last_undegraded":"2024-03-30T00:50:37.639455+0000","last_fullsized":"2024-03-30T00:50:37.639455+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:21:22.322038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640852+0000","last_change":"2024-03-30T00:50:37.640852+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640852+0000","last_undegraded":"2024-03-30T00:50:37.640852+0000","last_fullsized":"2024-03-30T00:50:37.640852+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638906+0000","last_change":"2024-03-30T00:50:37.638906+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638906+0000","last_undegraded":"2024-03-30T00:50:37.638906+0000","last_fullsized":"2024-03-30T00:50:37.638906+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:09:58.609620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641531+0000","last_change":"2024-03-30T00:50:37.641531+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641531+0000","last_undegraded":"2024-03-30T00:50:37.641531+0000","last_fullsized":"2024-03-30T00:50:37.641531+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.647544+0000","last_change":"2024-03-30T00:50:37.647544+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.647544+0000","last_undegraded":"2024-03-30T00:50:37.647544+0000","last_fullsized":"2024-03-30T00:50:37.647544+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638432+0000","last_change":"2024-03-30T00:50:37.638432+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638432+0000","last_undegraded":"2024-03-30T00:50:37.638432+0000","last_fullsized":"2024-03-30T00:50:37.638432+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637122+0000","last_change":"2024-03-30T00:50:37.637122+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637122+0000","last_undegraded":"2024-03-30T00:50:37.637122+0000","last_fullsized":"2024-03-30T00:50:37.637122+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:46.573525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643847+0000","last_change":"2024-03-30T00:50:37.643847+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643847+0000","last_undegraded":"2024-03-30T00:50:37.643847+0000","last_fullsized":"2024-03-30T00:50:37.643847+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645745+0000","last_change":"2024-03-30T00:50:37.645745+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645745+0000","last_undegraded":"2024-03-30T00:50:37.645745+0000","last_fullsized":"2024-03-30T00:50:37.645745+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:04:48.536583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646345+0000","last_change":"2024-03-30T00:50:37.646345+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646345+0000","last_undegraded":"2024-03-30T00:50:37.646345+0000","last_fullsized":"2024-03-30T00:50:37.646345+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:38.180300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640110+0000","last_change":"2024-03-30T00:50:37.640110+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640110+0000","last_undegraded":"2024-03-30T00:50:37.640110+0000","last_fullsized":"2024-03-30T00:50:37.640110+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:38:56.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638089+0000","last_change":"2024-03-30T00:50:37.638089+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638089+0000","last_undegraded":"2024-03-30T00:50:37.638089+0000","last_fullsized":"2024-03-30T00:50:37.638089+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.639932+0000","last_change":"2024-03-30T00:50:37.639932+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.639932+0000","last_undegraded":"2024-03-30T00:50:37.639932+0000","last_fullsized":"2024-03-30T00:50:37.639932+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636945+0000","last_change":"2024-03-30T00:50:37.636945+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636945+0000","last_undegraded":"2024-03-30T00:50:37.636945+0000","last_fullsized":"2024-03-30T00:50:37.636945+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641895+0000","last_change":"2024-03-30T00:50:37.641895+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641895+0000","last_undegraded":"2024-03-30T00:50:37.641895+0000","last_fullsized":"2024-03-30T00:50:37.641895+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645034+0000","last_change":"2024-03-30T00:50:37.645034+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645034+0000","last_undegraded":"2024-03-30T00:50:37.645034+0000","last_fullsized":"2024-03-30T00:50:37.645034+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642363+0000","last_change":"2024-03-30T00:50:37.642363+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642363+0000","last_undegraded":"2024-03-30T00:50:37.642363+0000","last_fullsized":"2024-03-30T00:50:37.642363+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:56.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645743+0000","last_change":"2024-03-30T00:50:37.645743+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645743+0000","last_undegraded":"2024-03-30T00:50:37.645743+0000","last_fullsized":"2024-03-30T00:50:37.645743+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:13.551635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638476+0000","last_change":"2024-03-30T00:50:37.638476+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638476+0000","last_undegraded":"2024-03-30T00:50:37.638476+0000","last_fullsized":"2024-03-30T00:50:37.638476+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:34:39.399906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640963+0000","last_change":"2024-03-30T00:50:37.640963+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640963+0000","last_undegraded":"2024-03-30T00:50:37.640963+0000","last_fullsized":"2024-03-30T00:50:37.640963+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:12.688382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.647144+0000","last_change":"2024-03-30T00:50:37.647144+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.647144+0000","last_undegraded":"2024-03-30T00:50:37.647144+0000","last_fullsized":"2024-03-30T00:50:37.647144+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645120+0000","last_change":"2024-03-30T00:50:37.645120+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645120+0000","last_undegraded":"2024-03-30T00:50:37.645120+0000","last_fullsized":"2024-03-30T00:50:37.645120+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:14.841469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641328+0000","last_change":"2024-03-30T00:50:37.641328+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641328+0000","last_undegraded":"2024-03-30T00:50:37.641328+0000","last_fullsized":"2024-03-30T00:50:37.641328+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641925+0000","last_change":"2024-03-30T00:50:37.641925+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641925+0000","last_undegraded":"2024-03-30T00:50:37.641925+0000","last_fullsized":"2024-03-30T00:50:37.641925+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659225+0000","last_change":"2024-03-30T00:50:37.658198+0000","last_active":"2024-03-30T00:50:37.659225+0000","last_peered":"2024-03-30T00:50:37.659225+0000","last_clean":"2024-03-30T00:50:37.659225+0000","last_became_active":"2024-03-30T00:50:37.657799+0000","last_became_peered":"2024-03-30T00:50:37.657799+0000","last_unstale":"2024-03-30T00:50:37.659225+0000","last_undegraded":"2024-03-30T00:50:37.659225+0000","last_fullsized":"2024-03-30T00:50:37.659225+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:55:09.373511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637154+0000","last_change":"2024-03-30T00:50:37.637154+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637154+0000","last_undegraded":"2024-03-30T00:50:37.637154+0000","last_fullsized":"2024-03-30T00:50:37.637154+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:58.975555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653380+0000","last_change":"2024-03-30T00:50:37.647732+0000","last_active":"2024-03-30T00:50:37.653380+0000","last_peered":"2024-03-30T00:50:37.653380+0000","last_clean":"2024-03-30T00:50:37.653380+0000","last_became_active":"2024-03-30T00:50:37.647438+0000","last_became_peered":"2024-03-30T00:50:37.647438+0000","last_unstale":"2024-03-30T00:50:37.653380+0000","last_undegraded":"2024-03-30T00:50:37.653380+0000","last_fullsized":"2024-03-30T00:50:37.653380+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:04.443031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642651+0000","last_change":"2024-03-30T00:50:37.642651+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642651+0000","last_undegraded":"2024-03-30T00:50:37.642651+0000","last_fullsized":"2024-03-30T00:50:37.642651+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656737+0000","last_change":"2024-03-30T00:50:37.651416+0000","last_active":"2024-03-30T00:50:37.656737+0000","last_peered":"2024-03-30T00:50:37.656737+0000","last_clean":"2024-03-30T00:50:37.656737+0000","last_became_active":"2024-03-30T00:50:37.651110+0000","last_became_peered":"2024-03-30T00:50:37.651110+0000","last_unstale":"2024-03-30T00:50:37.656737+0000","last_undegraded":"2024-03-30T00:50:37.656737+0000","last_fullsized":"2024-03-30T00:50:37.656737+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:04.443031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637468+0000","last_change":"2024-03-30T00:50:37.637468+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637468+0000","last_undegraded":"2024-03-30T00:50:37.637468+0000","last_fullsized":"2024-03-30T00:50:37.637468+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659094+0000","last_change":"2024-03-30T00:50:37.652287+0000","last_active":"2024-03-30T00:50:37.659094+0000","last_peered":"2024-03-30T00:50:37.659094+0000","last_clean":"2024-03-30T00:50:37.659094+0000","last_became_active":"2024-03-30T00:50:37.652025+0000","last_became_peered":"2024-03-30T00:50:37.652025+0000","last_unstale":"2024-03-30T00:50:37.659094+0000","last_undegraded":"2024-03-30T00:50:37.659094+0000","last_fullsized":"2024-03-30T00:50:37.659094+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:37.177730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640607+0000","last_change":"2024-03-30T00:50:37.640607+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640607+0000","last_undegraded":"2024-03-30T00:50:37.640607+0000","last_fullsized":"2024-03-30T00:50:37.640607+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.657930+0000","last_change":"2024-03-30T00:50:37.655896+0000","last_active":"2024-03-30T00:50:37.657930+0000","last_peered":"2024-03-30T00:50:37.657930+0000","last_clean":"2024-03-30T00:50:37.657930+0000","last_became_active":"2024-03-30T00:50:37.655623+0000","last_became_peered":"2024-03-30T00:50:37.655623+0000","last_unstale":"2024-03-30T00:50:37.657930+0000","last_undegraded":"2024-03-30T00:50:37.657930+0000","last_fullsized":"2024-03-30T00:50:37.657930+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:29.599324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637936+0000","last_change":"2024-03-30T00:50:37.637936+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637936+0000","last_undegraded":"2024-03-30T00:50:37.637936+0000","last_fullsized":"2024-03-30T00:50:37.637936+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659029+0000","last_change":"2024-03-30T00:50:37.657178+0000","last_active":"2024-03-30T00:50:37.659029+0000","last_peered":"2024-03-30T00:50:37.659029+0000","last_clean":"2024-03-30T00:50:37.659029+0000","last_became_active":"2024-03-30T00:50:37.656814+0000","last_became_peered":"2024-03-30T00:50:37.656814+0000","last_unstale":"2024-03-30T00:50:37.659029+0000","last_undegraded":"2024-03-30T00:50:37.659029+0000","last_fullsized":"2024-03-30T00:50:37.659029+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:11.406073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646745+0000","last_change":"2024-03-30T00:50:37.646745+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646745+0000","last_undegraded":"2024-03-30T00:50:37.646745+0000","last_fullsized":"2024-03-30T00:50:37.646745+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:51:44.264107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.657059+0000","last_change":"2024-03-30T00:50:37.654789+0000","last_active":"2024-03-30T00:50:37.657059+0000","last_peered":"2024-03-30T00:50:37.657059+0000","last_clean":"2024-03-30T00:50:37.657059+0000","last_became_active":"2024-03-30T00:50:37.654488+0000","last_became_peered":"2024-03-30T00:50:37.654488+0000","last_unstale":"2024-03-30T00:50:37.657059+0000","last_undegraded":"2024-03-30T00:50:37.657059+0000","last_fullsized":"2024-03-30T00:50:37.657059+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:41:51.590793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641119+0000","last_change":"2024-03-30T00:50:37.641119+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641119+0000","last_undegraded":"2024-03-30T00:50:37.641119+0000","last_fullsized":"2024-03-30T00:50:37.641119+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659480+0000","last_change":"2024-03-30T00:50:37.658253+0000","last_active":"2024-03-30T00:50:37.659480+0000","last_peered":"2024-03-30T00:50:37.659480+0000","last_clean":"2024-03-30T00:50:37.659480+0000","last_became_active":"2024-03-30T00:50:37.657996+0000","last_became_peered":"2024-03-30T00:50:37.657996+0000","last_unstale":"2024-03-30T00:50:37.659480+0000","last_undegraded":"2024-03-30T00:50:37.659480+0000","last_fullsized":"2024-03-30T00:50:37.659480+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:51:25.787216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.647075+0000","last_change":"2024-03-30T00:50:37.647075+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.647075+0000","last_undegraded":"2024-03-30T00:50:37.647075+0000","last_fullsized":"2024-03-30T00:50:37.647075+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:01.855290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.660222+0000","last_change":"2024-03-30T00:50:37.659011+0000","last_active":"2024-03-30T00:50:37.660222+0000","last_peered":"2024-03-30T00:50:37.660222+0000","last_clean":"2024-03-30T00:50:37.660222+0000","last_became_active":"2024-03-30T00:50:37.658720+0000","last_became_peered":"2024-03-30T00:50:37.658720+0000","last_unstale":"2024-03-30T00:50:37.660222+0000","last_undegraded":"2024-03-30T00:50:37.660222+0000","last_fullsized":"2024-03-30T00:50:37.660222+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:57.692452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642802+0000","last_change":"2024-03-30T00:50:37.642802+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642802+0000","last_undegraded":"2024-03-30T00:50:37.642802+0000","last_fullsized":"2024-03-30T00:50:37.642802+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653217+0000","last_change":"2024-03-30T00:50:37.651177+0000","last_active":"2024-03-30T00:50:37.653217+0000","last_peered":"2024-03-30T00:50:37.653217+0000","last_clean":"2024-03-30T00:50:37.653217+0000","last_became_active":"2024-03-30T00:50:37.650902+0000","last_became_peered":"2024-03-30T00:50:37.650902+0000","last_unstale":"2024-03-30T00:50:37.653217+0000","last_undegraded":"2024-03-30T00:50:37.653217+0000","last_fullsized":"2024-03-30T00:50:37.653217+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:28.311412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642586+0000","last_change":"2024-03-30T00:50:37.642586+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642586+0000","last_undegraded":"2024-03-30T00:50:37.642586+0000","last_fullsized":"2024-03-30T00:50:37.642586+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656565+0000","last_change":"2024-03-30T00:50:37.653495+0000","last_active":"2024-03-30T00:50:37.656565+0000","last_peered":"2024-03-30T00:50:37.656565+0000","last_clean":"2024-03-30T00:50:37.656565+0000","last_became_active":"2024-03-30T00:50:37.652706+0000","last_became_peered":"2024-03-30T00:50:37.652706+0000","last_unstale":"2024-03-30T00:50:37.656565+0000","last_undegraded":"2024-03-30T00:50:37.656565+0000","last_fullsized":"2024-03-30T00:50:37.656565+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:07:01.780302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.644570+0000","last_change":"2024-03-30T00:50:37.644570+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.644570+0000","last_undegraded":"2024-03-30T00:50:37.644570+0000","last_fullsized":"2024-03-30T00:50:37.644570+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:46:27.974997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653040+0000","last_change":"2024-03-30T00:50:37.648270+0000","last_active":"2024-03-30T00:50:37.653040+0000","last_peered":"2024-03-30T00:50:37.653040+0000","last_clean":"2024-03-30T00:50:37.653040+0000","last_became_active":"2024-03-30T00:50:37.647989+0000","last_became_peered":"2024-03-30T00:50:37.647989+0000","last_unstale":"2024-03-30T00:50:37.653040+0000","last_undegraded":"2024-03-30T00:50:37.653040+0000","last_fullsized":"2024-03-30T00:50:37.653040+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:53.076893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646353+0000","last_change":"2024-03-30T00:50:37.646353+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646353+0000","last_undegraded":"2024-03-30T00:50:37.646353+0000","last_fullsized":"2024-03-30T00:50:37.646353+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:05.445602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659297+0000","last_change":"2024-03-30T00:50:37.655775+0000","last_active":"2024-03-30T00:50:37.659297+0000","last_peered":"2024-03-30T00:50:37.659297+0000","last_clean":"2024-03-30T00:50:37.659297+0000","last_became_active":"2024-03-30T00:50:37.655033+0000","last_became_peered":"2024-03-30T00:50:37.655033+0000","last_unstale":"2024-03-30T00:50:37.659297+0000","last_undegraded":"2024-03-30T00:50:37.659297+0000","last_fullsized":"2024-03-30T00:50:37.659297+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:46:26.972427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638573+0000","last_change":"2024-03-30T00:50:37.638573+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638573+0000","last_undegraded":"2024-03-30T00:50:37.638573+0000","last_fullsized":"2024-03-30T00:50:37.638573+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.660319+0000","last_change":"2024-03-30T00:50:37.657373+0000","last_active":"2024-03-30T00:50:37.660319+0000","last_peered":"2024-03-30T00:50:37.660319+0000","last_clean":"2024-03-30T00:50:37.660319+0000","last_became_active":"2024-03-30T00:50:37.657068+0000","last_became_peered":"2024-03-30T00:50:37.657068+0000","last_unstale":"2024-03-30T00:50:37.660319+0000","last_undegraded":"2024-03-30T00:50:37.660319+0000","last_fullsized":"2024-03-30T00:50:37.660319+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:36.963155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640244+0000","last_change":"2024-03-30T00:50:37.640244+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640244+0000","last_undegraded":"2024-03-30T00:50:37.640244+0000","last_fullsized":"2024-03-30T00:50:37.640244+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:02:07.606929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.655268+0000","last_change":"2024-03-30T00:50:37.653596+0000","last_active":"2024-03-30T00:50:37.655268+0000","last_peered":"2024-03-30T00:50:37.655268+0000","last_clean":"2024-03-30T00:50:37.655268+0000","last_became_active":"2024-03-30T00:50:37.653299+0000","last_became_peered":"2024-03-30T00:50:37.653299+0000","last_unstale":"2024-03-30T00:50:37.655268+0000","last_undegraded":"2024-03-30T00:50:37.655268+0000","last_fullsized":"2024-03-30T00:50:37.655268+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:06.142142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641714+0000","last_change":"2024-03-30T00:50:37.641714+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641714+0000","last_undegraded":"2024-03-30T00:50:37.641714+0000","last_fullsized":"2024-03-30T00:50:37.641714+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.654489+0000","last_change":"2024-03-30T00:50:37.649310+0000","last_active":"2024-03-30T00:50:37.654489+0000","last_peered":"2024-03-30T00:50:37.654489+0000","last_clean":"2024-03-30T00:50:37.654489+0000","last_became_active":"2024-03-30T00:50:37.649015+0000","last_became_peered":"2024-03-30T00:50:37.649015+0000","last_unstale":"2024-03-30T00:50:37.654489+0000","last_undegraded":"2024-03-30T00:50:37.654489+0000","last_fullsized":"2024-03-30T00:50:37.654489+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:00.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637343+0000","last_change":"2024-03-30T00:50:37.637343+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637343+0000","last_undegraded":"2024-03-30T00:50:37.637343+0000","last_fullsized":"2024-03-30T00:50:37.637343+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658749+0000","last_change":"2024-03-30T00:50:37.655900+0000","last_active":"2024-03-30T00:50:37.658749+0000","last_peered":"2024-03-30T00:50:37.658749+0000","last_clean":"2024-03-30T00:50:37.658749+0000","last_became_active":"2024-03-30T00:50:37.655601+0000","last_became_peered":"2024-03-30T00:50:37.655601+0000","last_unstale":"2024-03-30T00:50:37.658749+0000","last_undegraded":"2024-03-30T00:50:37.658749+0000","last_fullsized":"2024-03-30T00:50:37.658749+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:33.394770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641239+0000","last_change":"2024-03-30T00:50:37.641239+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641239+0000","last_undegraded":"2024-03-30T00:50:37.641239+0000","last_fullsized":"2024-03-30T00:50:37.641239+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:09:05.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656296+0000","last_change":"2024-03-30T00:50:37.649475+0000","last_active":"2024-03-30T00:50:37.656296+0000","last_peered":"2024-03-30T00:50:37.656296+0000","last_clean":"2024-03-30T00:50:37.656296+0000","last_became_active":"2024-03-30T00:50:37.649055+0000","last_became_peered":"2024-03-30T00:50:37.649055+0000","last_unstale":"2024-03-30T00:50:37.656296+0000","last_undegraded":"2024-03-30T00:50:37.656296+0000","last_fullsized":"2024-03-30T00:50:37.656296+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:12.549064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643208+0000","last_change":"2024-03-30T00:50:37.643208+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643208+0000","last_undegraded":"2024-03-30T00:50:37.643208+0000","last_fullsized":"2024-03-30T00:50:37.643208+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:47.211883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658519+0000","last_change":"2024-03-30T00:50:37.656047+0000","last_active":"2024-03-30T00:50:37.658519+0000","last_peered":"2024-03-30T00:50:37.658519+0000","last_clean":"2024-03-30T00:50:37.658519+0000","last_became_active":"2024-03-30T00:50:37.655751+0000","last_became_peered":"2024-03-30T00:50:37.655751+0000","last_unstale":"2024-03-30T00:50:37.658519+0000","last_undegraded":"2024-03-30T00:50:37.658519+0000","last_fullsized":"2024-03-30T00:50:37.658519+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:00.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636644+0000","last_change":"2024-03-30T00:50:37.636644+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636644+0000","last_undegraded":"2024-03-30T00:50:37.636644+0000","last_fullsized":"2024-03-30T00:50:37.636644+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:52:46.481754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.651675+0000","last_change":"2024-03-30T00:50:37.646336+0000","last_active":"2024-03-30T00:50:37.651675+0000","last_peered":"2024-03-30T00:50:37.651675+0000","last_clean":"2024-03-30T00:50:37.651675+0000","last_became_active":"2024-03-30T00:50:37.646081+0000","last_became_peered":"2024-03-30T00:50:37.646081+0000","last_unstale":"2024-03-30T00:50:37.651675+0000","last_undegraded":"2024-03-30T00:50:37.651675+0000","last_fullsized":"2024-03-30T00:50:37.651675+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:40.726869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645084+0000","last_change":"2024-03-30T00:50:37.645084+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645084+0000","last_undegraded":"2024-03-30T00:50:37.645084+0000","last_fullsized":"2024-03-30T00:50:37.645084+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:04:19.985679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.652956+0000","last_change":"2024-03-30T00:50:37.648309+0000","last_active":"2024-03-30T00:50:37.652956+0000","last_peered":"2024-03-30T00:50:37.652956+0000","last_clean":"2024-03-30T00:50:37.652956+0000","last_became_active":"2024-03-30T00:50:37.646903+0000","last_became_peered":"2024-03-30T00:50:37.646903+0000","last_unstale":"2024-03-30T00:50:37.652956+0000","last_undegraded":"2024-03-30T00:50:37.652956+0000","last_fullsized":"2024-03-30T00:50:37.652956+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:20.839135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642635+0000","last_change":"2024-03-30T00:50:37.642635+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642635+0000","last_undegraded":"2024-03-30T00:50:37.642635+0000","last_fullsized":"2024-03-30T00:50:37.642635+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659418+0000","last_change":"2024-03-30T00:50:37.656069+0000","last_active":"2024-03-30T00:50:37.659418+0000","last_peered":"2024-03-30T00:50:37.659418+0000","last_clean":"2024-03-30T00:50:37.659418+0000","last_became_active":"2024-03-30T00:50:37.655828+0000","last_became_peered":"2024-03-30T00:50:37.655828+0000","last_unstale":"2024-03-30T00:50:37.659418+0000","last_undegraded":"2024-03-30T00:50:37.659418+0000","last_fullsized":"2024-03-30T00:50:37.659418+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:46.209312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643298+0000","last_change":"2024-03-30T00:50:37.643298+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643298+0000","last_undegraded":"2024-03-30T00:50:37.643298+0000","last_fullsized":"2024-03-30T00:50:37.643298+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:37.965725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656366+0000","last_change":"2024-03-30T00:50:37.650090+0000","last_active":"2024-03-30T00:50:37.656366+0000","last_peered":"2024-03-30T00:50:37.656366+0000","last_clean":"2024-03-30T00:50:37.656366+0000","last_became_active":"2024-03-30T00:50:37.649527+0000","last_became_peered":"2024-03-30T00:50:37.649527+0000","last_unstale":"2024-03-30T00:50:37.656366+0000","last_undegraded":"2024-03-30T00:50:37.656366+0000","last_fullsized":"2024-03-30T00:50:37.656366+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:37.177730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638050+0000","last_change":"2024-03-30T00:50:37.638050+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638050+0000","last_undegraded":"2024-03-30T00:50:37.638050+0000","last_fullsized":"2024-03-30T00:50:37.638050+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:19.197836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.654023+0000","last_change":"2024-03-30T00:50:37.648973+0000","last_active":"2024-03-30T00:50:37.654023+0000","last_peered":"2024-03-30T00:50:37.654023+0000","last_clean":"2024-03-30T00:50:37.654023+0000","last_became_active":"2024-03-30T00:50:37.648703+0000","last_became_peered":"2024-03-30T00:50:37.648703+0000","last_unstale":"2024-03-30T00:50:37.654023+0000","last_undegraded":"2024-03-30T00:50:37.654023+0000","last_fullsized":"2024-03-30T00:50:37.654023+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:53:36.133173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637305+0000","last_change":"2024-03-30T00:50:37.637305+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637305+0000","last_undegraded":"2024-03-30T00:50:37.637305+0000","last_fullsized":"2024-03-30T00:50:37.637305+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659783+0000","last_change":"2024-03-30T00:50:37.658787+0000","last_active":"2024-03-30T00:50:37.659783+0000","last_peered":"2024-03-30T00:50:37.659783+0000","last_clean":"2024-03-30T00:50:37.659783+0000","last_became_active":"2024-03-30T00:50:37.658512+0000","last_became_peered":"2024-03-30T00:50:37.658512+0000","last_unstale":"2024-03-30T00:50:37.659783+0000","last_undegraded":"2024-03-30T00:50:37.659783+0000","last_fullsized":"2024-03-30T00:50:37.659783+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:55:31.799226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637015+0000","last_change":"2024-03-30T00:50:37.637015+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637015+0000","last_undegraded":"2024-03-30T00:50:37.637015+0000","last_fullsized":"2024-03-30T00:50:37.637015+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658420+0000","last_change":"2024-03-30T00:50:37.656511+0000","last_active":"2024-03-30T00:50:37.658420+0000","last_peered":"2024-03-30T00:50:37.658420+0000","last_clean":"2024-03-30T00:50:37.658420+0000","last_became_active":"2024-03-30T00:50:37.655106+0000","last_became_peered":"2024-03-30T00:50:37.655106+0000","last_unstale":"2024-03-30T00:50:37.658420+0000","last_undegraded":"2024-03-30T00:50:37.658420+0000","last_fullsized":"2024-03-30T00:50:37.658420+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:04:28.343406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641788+0000","last_change":"2024-03-30T00:50:37.641788+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641788+0000","last_undegraded":"2024-03-30T00:50:37.641788+0000","last_fullsized":"2024-03-30T00:50:37.641788+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:34:31.524194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.657263+0000","last_change":"2024-03-30T00:50:37.655230+0000","last_active":"2024-03-30T00:50:37.657263+0000","last_peered":"2024-03-30T00:50:37.657263+0000","last_clean":"2024-03-30T00:50:37.657263+0000","last_became_active":"2024-03-30T00:50:37.654933+0000","last_became_peered":"2024-03-30T00:50:37.654933+0000","last_unstale":"2024-03-30T00:50:37.657263+0000","last_undegraded":"2024-03-30T00:50:37.657263+0000","last_fullsized":"2024-03-30T00:50:37.657263+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:59.607374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.635996+0000","last_change":"2024-03-30T00:50:37.635996+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.635996+0000","last_undegraded":"2024-03-30T00:50:37.635996+0000","last_fullsized":"2024-03-30T00:50:37.635996+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:52:14.331871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.650896+0000","last_change":"2024-03-30T00:50:37.647611+0000","last_active":"2024-03-30T00:50:37.650896+0000","last_peered":"2024-03-30T00:50:37.650896+0000","last_clean":"2024-03-30T00:50:37.650896+0000","last_became_active":"2024-03-30T00:50:37.647311+0000","last_became_peered":"2024-03-30T00:50:37.647311+0000","last_unstale":"2024-03-30T00:50:37.650896+0000","last_undegraded":"2024-03-30T00:50:37.650896+0000","last_fullsized":"2024-03-30T00:50:37.650896+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:21.467231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642870+0000","last_change":"2024-03-30T00:50:37.642870+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642870+0000","last_undegraded":"2024-03-30T00:50:37.642870+0000","last_fullsized":"2024-03-30T00:50:37.642870+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656761+0000","last_change":"2024-03-30T00:50:37.651832+0000","last_active":"2024-03-30T00:50:37.656761+0000","last_peered":"2024-03-30T00:50:37.656761+0000","last_clean":"2024-03-30T00:50:37.656761+0000","last_became_active":"2024-03-30T00:50:37.651529+0000","last_became_peered":"2024-03-30T00:50:37.651529+0000","last_unstale":"2024-03-30T00:50:37.656761+0000","last_undegraded":"2024-03-30T00:50:37.656761+0000","last_fullsized":"2024-03-30T00:50:37.656761+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:19.643626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636935+0000","last_change":"2024-03-30T00:50:37.636935+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636935+0000","last_undegraded":"2024-03-30T00:50:37.636935+0000","last_fullsized":"2024-03-30T00:50:37.636935+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656496+0000","last_change":"2024-03-30T00:50:37.654139+0000","last_active":"2024-03-30T00:50:37.656496+0000","last_peered":"2024-03-30T00:50:37.656496+0000","last_clean":"2024-03-30T00:50:37.656496+0000","last_became_active":"2024-03-30T00:50:37.653825+0000","last_became_peered":"2024-03-30T00:50:37.653825+0000","last_unstale":"2024-03-30T00:50:37.656496+0000","last_undegraded":"2024-03-30T00:50:37.656496+0000","last_fullsized":"2024-03-30T00:50:37.656496+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:47.043924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638023+0000","last_change":"2024-03-30T00:50:37.638023+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638023+0000","last_undegraded":"2024-03-30T00:50:37.638023+0000","last_fullsized":"2024-03-30T00:50:37.638023+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658174+0000","last_change":"2024-03-30T00:50:37.654797+0000","last_active":"2024-03-30T00:50:37.658174+0000","last_peered":"2024-03-30T00:50:37.658174+0000","last_clean":"2024-03-30T00:50:37.658174+0000","last_became_active":"2024-03-30T00:50:37.654506+0000","last_became_peered":"2024-03-30T00:50:37.654506+0000","last_unstale":"2024-03-30T00:50:37.658174+0000","last_undegraded":"2024-03-30T00:50:37.658174+0000","last_fullsized":"2024-03-30T00:50:37.658174+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:26.653343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637355+0000","last_change":"2024-03-30T00:50:37.637355+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637355+0000","last_undegraded":"2024-03-30T00:50:37.637355+0000","last_fullsized":"2024-03-30T00:50:37.637355+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653101+0000","last_change":"2024-03-30T00:50:37.648683+0000","last_active":"2024-03-30T00:50:37.653101+0000","last_peered":"2024-03-30T00:50:37.653101+0000","last_clean":"2024-03-30T00:50:37.653101+0000","last_became_active":"2024-03-30T00:50:37.648391+0000","last_became_peered":"2024-03-30T00:50:37.648391+0000","last_unstale":"2024-03-30T00:50:37.653101+0000","last_undegraded":"2024-03-30T00:50:37.653101+0000","last_fullsized":"2024-03-30T00:50:37.653101+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:13.838899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":271,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-03-30T00:50:36.635729+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:50:36.635729+0000","last_peered":"2024-03-30T00:50:36.635729+0000","last_clean":"2024-03-30T00:50:36.635729+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:50:36.635729+0000","last_undegraded":"2024-03-30T00:50:36.635729+0000","last_fullsized":"2024-03-30T00:50:36.635729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574982,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2113536,"data_stored":1290047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":0.97599999999999998,"15min":1.0860000000000001},"last":0.621},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.47399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84999999999999998,"5min":0.98299999999999998,"15min":1.149},"last":0.55500000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.503,"5min":0.45800000000000002,"15min":0.436},"max":{"1min":0.90800000000000003,"5min":1.03,"15min":1.03},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93000000000000005,"5min":0.95199999999999996,"15min":1.3029999999999999},"last":0.70399999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.76600000000000001},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.45000000000000001},"max":{"1min":1.0129999999999999,"5min":1.085,"15min":1.6020000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.73099999999999998,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95399999999999996,"5min":1.034,"15min":1.6619999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.75900000000000001},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99099999999999999,"5min":0.99299999999999999,"15min":2.0150000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.80000000000000004},"min":{"1min":0.63800000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.9490000000000001},"last":0.76400000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.37},"max":{"1min":0.97299999999999998,"5min":1.2,"15min":1.2},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.42999999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.97899999999999998},"last":0.52700000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96299999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":0.99199999999999999,"15min":1.026},"last":0.60699999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.92700000000000005,"5min":0.94599999999999995,"15min":1.214},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89200000000000002,"5min":1.0189999999999999,"15min":1.0309999999999999},"last":0.64900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1531904,"data_stored":706274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71599999999999997,"15min":0.79800000000000004},"min":{"1min":0.437,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.85799999999999998,"5min":1.2969999999999999,"15min":4.0490000000000004},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.68799999999999994},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":1.278,"15min":1.47},"last":0.96199999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.438,"15min":0.36899999999999999},"max":{"1min":0.95799999999999996,"5min":1.204,"15min":4.3170000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.85299999999999998},"min":{"1min":0.58399999999999996,"5min":0.47199999999999998,"15min":0.42999999999999999},"max":{"1min":0.92900000000000005,"5min":1.1120000000000001,"15min":4.4160000000000004},"last":0.91700000000000004}]},{"osd":2,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.745,"15min":0.84999999999999998},"min":{"1min":0.63400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98299999999999998,"5min":1.24,"15min":4.375},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.84299999999999997},"min":{"1min":0.51700000000000002,"5min":0.42999999999999999,"15min":0.38600000000000001},"max":{"1min":0.92500000000000004,"5min":1.3129999999999999,"15min":4.2960000000000003},"last":0.89900000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.86799999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0249999999999999,"5min":1.2390000000000001,"15min":4.3399999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73199999999999998,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.49099999999999999,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.258,"15min":4.6020000000000003},"last":0.85399999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.39200000000000002,"5min":0.39100000000000001,"15min":0.35999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.1870000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.73199999999999998},"min":{"1min":0.441,"5min":0.372,"15min":0.372},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.036},"last":0.82599999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.65700000000000003},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0720000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0109999999999999,"15min":1.101},"last":0.67900000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97499999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.45700000000000002},"max":{"1min":1.0860000000000001,"5min":1.103,"15min":1.103},"last":0.70499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574982,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":0.83299999999999996,"5min":0.97599999999999998,"15min":1.095},"last":0.63},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68700000000000006,"15min":0.73199999999999998},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":1.0489999999999999,"15min":1.7569999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.71999999999999997},"min":{"1min":0.47899999999999998,"5min":0.46600000000000003,"15min":0.43099999999999999},"max":{"1min":0.86599999999999999,"5min":0.98199999999999998,"15min":1.7330000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70999999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42399999999999999},"max":{"1min":0.92400000000000004,"5min":0.98799999999999999,"15min":1.3260000000000001},"last":0.73299999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.44600000000000001},"max":{"1min":0.89900000000000002,"5min":1.131,"15min":1.131},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.45900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.877,"5min":1.0609999999999999,"15min":1.2789999999999999},"last":0.51400000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.75},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.89100000000000001,"5min":1.0740000000000001,"15min":1.7},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.96099999999999997,"5min":1.1160000000000001,"15min":1.181},"last":0.56399999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.60299999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.309},"max":{"1min":0.876,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.433},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69199999999999995,"15min":0.73399999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91200000000000003,"5min":0.97799999999999998,"15min":1.615},"last":0.67000000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.84099999999999997,"5min":0.91100000000000003,"15min":0.97399999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.626,"15min":0.67300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.35199999999999998},"max":{"1min":0.84999999999999998,"5min":0.93700000000000006,"15min":1.569},"last":0.59199999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.71499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.377},"max":{"1min":0.85399999999999998,"5min":0.92200000000000004,"15min":1.0149999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88400000000000001,"5min":1.167,"15min":1.383},"last":0.64800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.34899999999999998},"max":{"1min":0.92400000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.67400000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98599999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.502}]},{"osd":1,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":0.92600000000000005,"5min":0.95899999999999996,"15min":1.2909999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.40699999999999997},"max":{"1min":1.006,"5min":1.113,"15min":1.113},"last":0.60799999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.59199999999999997,"5min":0.48999999999999999,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.73899999999999999,"15min":0.75},"min":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.46700000000000003},"max":{"1min":0.94899999999999995,"5min":1.163,"15min":1.228},"last":0.68300000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73599999999999999,"15min":0.71699999999999997},"min":{"1min":0.501,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.27},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.76000000000000001},"min":{"1min":0.59099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1,"15min":1.1739999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.621,"15min":0.65800000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.89400000000000002,"15min":1.329},"last":0.754},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66100000000000003,"15min":0.70399999999999996},"min":{"1min":0.57399999999999995,"5min":0.34799999999999998,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":0.99099999999999999,"15min":1.3819999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.65700000000000003},"min":{"1min":0.45700000000000002,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":1.2130000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.69699999999999995},"min":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.95799999999999996,"15min":1.3220000000000001},"last":0.85599999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.628,"15min":0.69099999999999995},"min":{"1min":0.46700000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":0.91500000000000004,"15min":1.6910000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.73399999999999999,"15min":0.78000000000000003},"min":{"1min":0.59699999999999998,"5min":0.49299999999999999,"15min":0.46600000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.5009999999999999},"last":0.873}]}]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.435,"15min":0.45500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.58699999999999997,"5min":0.748,"15min":1.0529999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.49099999999999999,"15min":0.51400000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.24199999999999999},"max":{"1min":0.63400000000000001,"5min":0.75,"15min":0.79100000000000004},"last":0.39500000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49399999999999999,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.80900000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63800000000000001,"5min":0.73399999999999999,"15min":1.2150000000000001},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.78300000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.46300000000000002},"max":{"1min":0.95699999999999996,"5min":1.1259999999999999,"15min":1.2470000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77100000000000002,"15min":0.77600000000000002},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.079,"15min":1.0860000000000001},"last":0.97099999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77100000000000002,"15min":0.77900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.51100000000000001},"max":{"1min":1.024,"5min":1.0369999999999999,"15min":1.125},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.77900000000000003,"15min":0.79100000000000004},"min":{"1min":0.628,"5min":0.52200000000000002,"15min":0.48599999999999999},"max":{"1min":0.86299999999999999,"5min":1.133,"15min":1.2390000000000001},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77200000000000002,"15min":0.79700000000000004},"min":{"1min":0.59999999999999998,"5min":0.51400000000000001,"15min":0.499},"max":{"1min":0.90100000000000002,"5min":1.091,"15min":1.226},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.038,"5min":1.1120000000000001,"15min":1.167},"last":0.94899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82999999999999996,"15min":0.84099999999999997},"min":{"1min":0.71399999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.123,"5min":1.1910000000000001,"15min":1.202},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83499999999999996,"15min":0.86299999999999999},"min":{"1min":0.57299999999999995,"5min":0.54900000000000004,"15min":0.497},"max":{"1min":1.095,"5min":1.1639999999999999,"15min":1.3160000000000001},"last":1.0189999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56899999999999995,"5min":0.70299999999999996,"15min":4.0999999999999996},"last":1.339},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.48799999999999999,"15min":0.50600000000000001},"min":{"1min":0.40699999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83999999999999997,"5min":0.84099999999999997,"15min":1.9239999999999999},"last":1.0900000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32600000000000001},"max":{"1min":0.61799999999999999,"5min":0.81299999999999994,"15min":3.1139999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.52400000000000002,"15min":0.53400000000000003},"min":{"1min":0.40600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.748,"5min":0.752,"15min":2.4159999999999999},"last":1.1830000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.499,"15min":0.497},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70099999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.749,"5min":0.88800000000000001,"15min":1.907},"last":0.95599999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77700000000000002,"15min":0.80100000000000005},"min":{"1min":0.59699999999999998,"5min":0.504,"15min":0.504},"max":{"1min":1.0900000000000001,"5min":1.1919999999999999,"15min":1.9470000000000001},"last":1.129},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78700000000000003,"15min":0.82099999999999995},"min":{"1min":0.628,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.163,"5min":1.2110000000000001,"15min":1.972},"last":1.2649999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.1220000000000001,"5min":1.2549999999999999,"15min":2.4569999999999999},"last":1.3},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.78900000000000003,"15min":0.81000000000000005},"min":{"1min":0.58399999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0389999999999999,"5min":1.286,"15min":2.8039999999999998},"last":0.70299999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0980000000000001,"5min":1.177,"15min":2.8849999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.0780000000000001,"5min":1.242,"15min":1.867},"last":0.996}]},{"osd":7,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79500000000000004,"15min":0.83699999999999997},"min":{"1min":0.60199999999999998,"5min":0.54700000000000004,"15min":0.495},"max":{"1min":1.0429999999999999,"5min":1.1140000000000001,"15min":2.3690000000000002},"last":1.2270000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.83799999999999997},"min":{"1min":0.70499999999999996,"5min":0.55800000000000005,"15min":0.52700000000000002},"max":{"1min":0.98999999999999999,"5min":1.365,"15min":2.851},"last":1.0089999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44,"15min":0.42799999999999999},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.253},"max":{"1min":0.63600000000000001,"5min":0.72199999999999998,"15min":0.80300000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.48399999999999999,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.69999999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33300000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.45900000000000002},"min":{"1min":0.39200000000000002,"5min":0.29999999999999999,"15min":0.29199999999999998},"max":{"1min":0.59199999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.372},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.64600000000000002,"5min":0.84099999999999997,"15min":0.89400000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.45300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.69199999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.499},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.38600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72999999999999998,"5min":0.83199999999999996,"15min":0.83999999999999997},"last":0.40600000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.75600000000000001,"15min":0.72899999999999998},"min":{"1min":0.58099999999999996,"5min":0.49199999999999999,"15min":0.38800000000000001},"max":{"1min":1.034,"5min":1.175,"15min":1.175},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76900000000000002,"15min":0.754},"min":{"1min":0.66300000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98599999999999999,"5min":1.079,"15min":1.2},"last":0.81399999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.76000000000000001},"min":{"1min":0.621,"5min":0.56799999999999995,"15min":0.51600000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.3180000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79400000000000004,"15min":0.77100000000000002},"min":{"1min":0.60299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.129,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.87}]},{"osd":6,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79700000000000004,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.0389999999999999,"5min":1.179,"15min":1.179},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0509999999999999,"15min":1.2150000000000001},"last":0.89300000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80300000000000005,"15min":0.79700000000000004},"min":{"1min":0.64500000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.024,"5min":1.2050000000000001,"15min":1.2669999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.81799999999999995,"15min":0.79200000000000004},"min":{"1min":0.69299999999999995,"5min":0.56200000000000006,"15min":0.48499999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.236},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2113536,"data_stored":1290047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.45700000000000002},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.245},"max":{"1min":0.69799999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.505},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68400000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.51000000000000001},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.67100000000000004,"5min":0.82599999999999996,"15min":1.02},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.432,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":0.77900000000000003,"15min":1.006},"last":0.54400000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.68000000000000005,"5min":0.754,"15min":0.84799999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.46200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":0.88900000000000001,"15min":0.96899999999999997},"last":0.60999999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.47899999999999998},"max":{"1min":1.1040000000000001,"5min":1.1140000000000001,"15min":1.5169999999999999},"last":0.878},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.46400000000000002},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.252},"last":0.81000000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.55200000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.3080000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0780000000000001,"15min":1.462},"last":0.746}]},{"osd":6,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79200000000000004,"15min":0.79900000000000004},"min":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.54500000000000004},"max":{"1min":1.0429999999999999,"5min":1.131,"15min":1.5920000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80000000000000004,"15min":0.79000000000000004},"min":{"1min":0.625,"5min":0.54600000000000004,"15min":0.46999999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.2230000000000001},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.81299999999999994},"min":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.3600000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.60399999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.556},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:50:42.158 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:50:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.327+0000 7f6207200640 1 Processor -- start 2024-03-30T00:50:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.327+0000 7f6207200640 1 -- start start 2024-03-30T00:50:42.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6207200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f620015ca10 0x7f620006ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6207200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62000ae4e0 0x7f62000cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6207200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f62000ce820 0x7f62000cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6207200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62000b1f70 con 0x7f62000ce820 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6207200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62000cf1f0 con 0x7f620015ca10 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6207200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620006e7d0 con 0x7f62000ae4e0 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6205200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f62000ce820 0x7f62000cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6205200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f62000ce820 0x7f62000cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58866/0 (socket says 172.21.15.66:58866) 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6205200640 1 -- 172.21.15.66:0/1084778798 learned_addr learned my addr 172.21.15.66:0/1084778798 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f61f7e00640 1 --2- 172.21.15.66:0/1084778798 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62000ae4e0 0x7f62000cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f61ffe00640 1 --2- 172.21.15.66:0/1084778798 >> v2:172.21.15.156:3300/0 conn(0x7f620015ca10 0x7f620006ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6205200640 1 -- 172.21.15.66:0/1084778798 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62000ae4e0 msgr2=0x7f62000cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6205200640 1 --2- 172.21.15.66:0/1084778798 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62000ae4e0 0x7f62000cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6205200640 1 -- 172.21.15.66:0/1084778798 >> v2:172.21.15.156:3300/0 conn(0x7f620015ca10 msgr2=0x7f620006ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6205200640 1 --2- 172.21.15.66:0/1084778798 >> v2:172.21.15.156:3300/0 conn(0x7f620015ca10 0x7f620006ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.331+0000 7f6205200640 1 -- 172.21.15.66:0/1084778798 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620006f1a0 con 0x7f62000ce820 2024-03-30T00:50:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6205200640 1 --2- 172.21.15.66:0/1084778798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000ce820 0x7f62000cecb0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f620006f9f0 tx=0x7f61f801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693ac5be7e2f58c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f61ff400640 1 -- 172.21.15.66:0/1084778798 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f80a5780 con 0x7f62000ce820 2024-03-30T00:50:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f61ff400640 1 -- 172.21.15.66:0/1084778798 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f80a58e0 con 0x7f62000ce820 2024-03-30T00:50:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f61ff400640 1 -- 172.21.15.66:0/1084778798 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f80a5c70 con 0x7f62000ce820 2024-03-30T00:50:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 -- 172.21.15.66:0/1084778798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000ce820 msgr2=0x7f62000cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 --2- 172.21.15.66:0/1084778798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000ce820 0x7f62000cecb0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f620006f9f0 tx=0x7f61f801c9f0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 -- 172.21.15.66:0/1084778798 shutdown_connections 2024-03-30T00:50:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 --2- 172.21.15.66:0/1084778798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000ce820 0x7f62000cecb0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 --2- 172.21.15.66:0/1084778798 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f62000ae4e0 0x7f62000cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 --2- 172.21.15.66:0/1084778798 >> v2:172.21.15.156:3300/0 conn(0x7f620015ca10 0x7f620006ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 -- 172.21.15.66:0/1084778798 >> 172.21.15.66:0/1084778798 conn(0x7f6200098a20 msgr2=0x7f62000b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:42.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 -- 172.21.15.66:0/1084778798 shutdown_connections 2024-03-30T00:50:42.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 -- 172.21.15.66:0/1084778798 wait complete. 2024-03-30T00:50:42.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 Processor -- start 2024-03-30T00:50:42.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 -- start start 2024-03-30T00:50:42.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000ae4e0 0x7f62000c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62000ce820 0x7f6200158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f620015ca10 0x7f62000c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6200158fb0 con 0x7f62000ae4e0 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62000c4250 con 0x7f62000ce820 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f6207200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6200158670 con 0x7f620015ca10 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f61ffe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000ae4e0 0x7f62000c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f61ffe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000ae4e0 0x7f62000c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58868/0 (socket says 172.21.15.66:58868) 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.335+0000 7f61ffe00640 1 -- 172.21.15.66:0/3373475498 learned_addr learned my addr 172.21.15.66:0/3373475498 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61f7e00640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62000ce820 0x7f6200158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f6205200640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f620015ca10 0x7f62000c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61ffe00640 1 -- 172.21.15.66:0/3373475498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f620015ca10 msgr2=0x7f62000c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61ffe00640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f620015ca10 0x7f62000c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61ffe00640 1 -- 172.21.15.66:0/3373475498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62000ce820 msgr2=0x7f6200158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61ffe00640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62000ce820 0x7f6200158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61ffe00640 1 -- 172.21.15.66:0/3373475498 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f8009b30 con 0x7f62000ae4e0 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f6205200640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f620015ca10 0x7f62000c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61f7e00640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62000ce820 0x7f6200158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61ffe00640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000ae4e0 0x7f62000c4670 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f61f000d5f0 tx=0x7f61f000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:42.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61fcc00640 1 -- 172.21.15.66:0/3373475498 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00125f0 con 0x7f62000ae4e0 2024-03-30T00:50:42.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61fcc00640 1 -- 172.21.15.66:0/3373475498 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f0012750 con 0x7f62000ae4e0 2024-03-30T00:50:42.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61fcc00640 1 -- 172.21.15.66:0/3373475498 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0012ae0 con 0x7f62000ae4e0 2024-03-30T00:50:42.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f6207200640 1 -- 172.21.15.66:0/3373475498 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62000c5ea0 con 0x7f62000ae4e0 2024-03-30T00:50:42.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f6207200640 1 -- 172.21.15.66:0/3373475498 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6200158cd0 con 0x7f62000ae4e0 2024-03-30T00:50:42.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f6207200640 1 -- 172.21.15.66:0/3373475498 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6200066c30 con 0x7f62000ae4e0 2024-03-30T00:50:42.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61fcc00640 1 -- 172.21.15.66:0/3373475498 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f61f0012c40 con 0x7f62000ae4e0 2024-03-30T00:50:42.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61fcc00640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f61dc066690 0x7f61dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61fcc00640 1 -- 172.21.15.66:0/3373475498 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f61f0084720 con 0x7f62000ae4e0 2024-03-30T00:50:42.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.339+0000 7f61f7e00640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f61dc066690 0x7f61dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.343+0000 7f61f7e00640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f61dc066690 0x7f61dc068b50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f62000c61a0 tx=0x7f61e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.343+0000 7f61fcc00640 1 -- 172.21.15.66:0/3373475498 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f61f0051940 con 0x7f62000ae4e0 2024-03-30T00:50:42.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.503+0000 7f6207200640 1 -- 172.21.15.66:0/3373475498 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62000cbb60 con 0x7f61dc066690 2024-03-30T00:50:42.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.507+0000 7f61fcc00640 1 -- 172.21.15.66:0/3373475498 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286510 (secure 0 0 0) 0x7f61e8007710 con 0x7f61dc066690 2024-03-30T00:50:42.520 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:42.522 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:50:42.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 -- 172.21.15.66:0/3373475498 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f61dc066690 msgr2=0x7f61dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f61dc066690 0x7f61dc068b50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f62000c61a0 tx=0x7f61e8008040 comp rx=0 tx=0).stop 2024-03-30T00:50:42.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 -- 172.21.15.66:0/3373475498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000ae4e0 msgr2=0x7f62000c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000ae4e0 0x7f62000c4670 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f61f000d5f0 tx=0x7f61f000dab0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 -- 172.21.15.66:0/3373475498 shutdown_connections 2024-03-30T00:50:42.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f620015ca10 0x7f62000c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62000ce820 0x7f6200158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f61dc066690 0x7f61dc068b50 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 --2- 172.21.15.66:0/3373475498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000ae4e0 0x7f62000c4670 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 -- 172.21.15.66:0/3373475498 >> 172.21.15.66:0/3373475498 conn(0x7f6200098a20 msgr2=0x7f62000ad2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:42.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 -- 172.21.15.66:0/3373475498 shutdown_connections 2024-03-30T00:50:42.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.511+0000 7f6207200640 1 -- 172.21.15.66:0/3373475498 wait complete. 2024-03-30T00:50:42.540 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-03-30T00:50:41.000126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261988,"kb_used_data":13044,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712732,"statfs":{"total":773094113280,"available":772825837568,"internally_reserved":0,"allocated":13357056,"data_stored":6784763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.373741"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.639401+0000","last_change":"2024-03-30T00:50:37.639401+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.639401+0000","last_undegraded":"2024-03-30T00:50:37.639401+0000","last_fullsized":"2024-03-30T00:50:37.639401+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646197+0000","last_change":"2024-03-30T00:50:37.646197+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646197+0000","last_undegraded":"2024-03-30T00:50:37.646197+0000","last_fullsized":"2024-03-30T00:50:37.646197+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643997+0000","last_change":"2024-03-30T00:50:37.643997+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643997+0000","last_undegraded":"2024-03-30T00:50:37.643997+0000","last_fullsized":"2024-03-30T00:50:37.643997+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:53:58.506502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637414+0000","last_change":"2024-03-30T00:50:37.637414+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637414+0000","last_undegraded":"2024-03-30T00:50:37.637414+0000","last_fullsized":"2024-03-30T00:50:37.637414+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638917+0000","last_change":"2024-03-30T00:50:37.638917+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638917+0000","last_undegraded":"2024-03-30T00:50:37.638917+0000","last_fullsized":"2024-03-30T00:50:37.638917+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637248+0000","last_change":"2024-03-30T00:50:37.637248+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637248+0000","last_undegraded":"2024-03-30T00:50:37.637248+0000","last_fullsized":"2024-03-30T00:50:37.637248+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:16.922513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636753+0000","last_change":"2024-03-30T00:50:37.636753+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636753+0000","last_undegraded":"2024-03-30T00:50:37.636753+0000","last_fullsized":"2024-03-30T00:50:37.636753+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:10:54.265531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642083+0000","last_change":"2024-03-30T00:50:37.642083+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642083+0000","last_undegraded":"2024-03-30T00:50:37.642083+0000","last_fullsized":"2024-03-30T00:50:37.642083+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638063+0000","last_change":"2024-03-30T00:50:37.638063+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638063+0000","last_undegraded":"2024-03-30T00:50:37.638063+0000","last_fullsized":"2024-03-30T00:50:37.638063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.639455+0000","last_change":"2024-03-30T00:50:37.639455+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.639455+0000","last_undegraded":"2024-03-30T00:50:37.639455+0000","last_fullsized":"2024-03-30T00:50:37.639455+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:21:22.322038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640852+0000","last_change":"2024-03-30T00:50:37.640852+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640852+0000","last_undegraded":"2024-03-30T00:50:37.640852+0000","last_fullsized":"2024-03-30T00:50:37.640852+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638906+0000","last_change":"2024-03-30T00:50:37.638906+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638906+0000","last_undegraded":"2024-03-30T00:50:37.638906+0000","last_fullsized":"2024-03-30T00:50:37.638906+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:09:58.609620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641531+0000","last_change":"2024-03-30T00:50:37.641531+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641531+0000","last_undegraded":"2024-03-30T00:50:37.641531+0000","last_fullsized":"2024-03-30T00:50:37.641531+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.647544+0000","last_change":"2024-03-30T00:50:37.647544+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.647544+0000","last_undegraded":"2024-03-30T00:50:37.647544+0000","last_fullsized":"2024-03-30T00:50:37.647544+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638432+0000","last_change":"2024-03-30T00:50:37.638432+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638432+0000","last_undegraded":"2024-03-30T00:50:37.638432+0000","last_fullsized":"2024-03-30T00:50:37.638432+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637122+0000","last_change":"2024-03-30T00:50:37.637122+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637122+0000","last_undegraded":"2024-03-30T00:50:37.637122+0000","last_fullsized":"2024-03-30T00:50:37.637122+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:46.573525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643847+0000","last_change":"2024-03-30T00:50:37.643847+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643847+0000","last_undegraded":"2024-03-30T00:50:37.643847+0000","last_fullsized":"2024-03-30T00:50:37.643847+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645745+0000","last_change":"2024-03-30T00:50:37.645745+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645745+0000","last_undegraded":"2024-03-30T00:50:37.645745+0000","last_fullsized":"2024-03-30T00:50:37.645745+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:04:48.536583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646345+0000","last_change":"2024-03-30T00:50:37.646345+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646345+0000","last_undegraded":"2024-03-30T00:50:37.646345+0000","last_fullsized":"2024-03-30T00:50:37.646345+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:38.180300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640110+0000","last_change":"2024-03-30T00:50:37.640110+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640110+0000","last_undegraded":"2024-03-30T00:50:37.640110+0000","last_fullsized":"2024-03-30T00:50:37.640110+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:38:56.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638089+0000","last_change":"2024-03-30T00:50:37.638089+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638089+0000","last_undegraded":"2024-03-30T00:50:37.638089+0000","last_fullsized":"2024-03-30T00:50:37.638089+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.639932+0000","last_change":"2024-03-30T00:50:37.639932+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.639932+0000","last_undegraded":"2024-03-30T00:50:37.639932+0000","last_fullsized":"2024-03-30T00:50:37.639932+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636945+0000","last_change":"2024-03-30T00:50:37.636945+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636945+0000","last_undegraded":"2024-03-30T00:50:37.636945+0000","last_fullsized":"2024-03-30T00:50:37.636945+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641895+0000","last_change":"2024-03-30T00:50:37.641895+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641895+0000","last_undegraded":"2024-03-30T00:50:37.641895+0000","last_fullsized":"2024-03-30T00:50:37.641895+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645034+0000","last_change":"2024-03-30T00:50:37.645034+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645034+0000","last_undegraded":"2024-03-30T00:50:37.645034+0000","last_fullsized":"2024-03-30T00:50:37.645034+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642363+0000","last_change":"2024-03-30T00:50:37.642363+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642363+0000","last_undegraded":"2024-03-30T00:50:37.642363+0000","last_fullsized":"2024-03-30T00:50:37.642363+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:56.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645743+0000","last_change":"2024-03-30T00:50:37.645743+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645743+0000","last_undegraded":"2024-03-30T00:50:37.645743+0000","last_fullsized":"2024-03-30T00:50:37.645743+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:13.551635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638476+0000","last_change":"2024-03-30T00:50:37.638476+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638476+0000","last_undegraded":"2024-03-30T00:50:37.638476+0000","last_fullsized":"2024-03-30T00:50:37.638476+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:34:39.399906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640963+0000","last_change":"2024-03-30T00:50:37.640963+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640963+0000","last_undegraded":"2024-03-30T00:50:37.640963+0000","last_fullsized":"2024-03-30T00:50:37.640963+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:12.688382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.647144+0000","last_change":"2024-03-30T00:50:37.647144+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.647144+0000","last_undegraded":"2024-03-30T00:50:37.647144+0000","last_fullsized":"2024-03-30T00:50:37.647144+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645120+0000","last_change":"2024-03-30T00:50:37.645120+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645120+0000","last_undegraded":"2024-03-30T00:50:37.645120+0000","last_fullsized":"2024-03-30T00:50:37.645120+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:14.841469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641328+0000","last_change":"2024-03-30T00:50:37.641328+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641328+0000","last_undegraded":"2024-03-30T00:50:37.641328+0000","last_fullsized":"2024-03-30T00:50:37.641328+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641925+0000","last_change":"2024-03-30T00:50:37.641925+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641925+0000","last_undegraded":"2024-03-30T00:50:37.641925+0000","last_fullsized":"2024-03-30T00:50:37.641925+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659225+0000","last_change":"2024-03-30T00:50:37.658198+0000","last_active":"2024-03-30T00:50:37.659225+0000","last_peered":"2024-03-30T00:50:37.659225+0000","last_clean":"2024-03-30T00:50:37.659225+0000","last_became_active":"2024-03-30T00:50:37.657799+0000","last_became_peered":"2024-03-30T00:50:37.657799+0000","last_unstale":"2024-03-30T00:50:37.659225+0000","last_undegraded":"2024-03-30T00:50:37.659225+0000","last_fullsized":"2024-03-30T00:50:37.659225+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:55:09.373511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637154+0000","last_change":"2024-03-30T00:50:37.637154+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637154+0000","last_undegraded":"2024-03-30T00:50:37.637154+0000","last_fullsized":"2024-03-30T00:50:37.637154+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:58.975555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653380+0000","last_change":"2024-03-30T00:50:37.647732+0000","last_active":"2024-03-30T00:50:37.653380+0000","last_peered":"2024-03-30T00:50:37.653380+0000","last_clean":"2024-03-30T00:50:37.653380+0000","last_became_active":"2024-03-30T00:50:37.647438+0000","last_became_peered":"2024-03-30T00:50:37.647438+0000","last_unstale":"2024-03-30T00:50:37.653380+0000","last_undegraded":"2024-03-30T00:50:37.653380+0000","last_fullsized":"2024-03-30T00:50:37.653380+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:04.443031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642651+0000","last_change":"2024-03-30T00:50:37.642651+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642651+0000","last_undegraded":"2024-03-30T00:50:37.642651+0000","last_fullsized":"2024-03-30T00:50:37.642651+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656737+0000","last_change":"2024-03-30T00:50:37.651416+0000","last_active":"2024-03-30T00:50:37.656737+0000","last_peered":"2024-03-30T00:50:37.656737+0000","last_clean":"2024-03-30T00:50:37.656737+0000","last_became_active":"2024-03-30T00:50:37.651110+0000","last_became_peered":"2024-03-30T00:50:37.651110+0000","last_unstale":"2024-03-30T00:50:37.656737+0000","last_undegraded":"2024-03-30T00:50:37.656737+0000","last_fullsized":"2024-03-30T00:50:37.656737+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:04.443031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637468+0000","last_change":"2024-03-30T00:50:37.637468+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637468+0000","last_undegraded":"2024-03-30T00:50:37.637468+0000","last_fullsized":"2024-03-30T00:50:37.637468+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659094+0000","last_change":"2024-03-30T00:50:37.652287+0000","last_active":"2024-03-30T00:50:37.659094+0000","last_peered":"2024-03-30T00:50:37.659094+0000","last_clean":"2024-03-30T00:50:37.659094+0000","last_became_active":"2024-03-30T00:50:37.652025+0000","last_became_peered":"2024-03-30T00:50:37.652025+0000","last_unstale":"2024-03-30T00:50:37.659094+0000","last_undegraded":"2024-03-30T00:50:37.659094+0000","last_fullsized":"2024-03-30T00:50:37.659094+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:37.177730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640607+0000","last_change":"2024-03-30T00:50:37.640607+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640607+0000","last_undegraded":"2024-03-30T00:50:37.640607+0000","last_fullsized":"2024-03-30T00:50:37.640607+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.657930+0000","last_change":"2024-03-30T00:50:37.655896+0000","last_active":"2024-03-30T00:50:37.657930+0000","last_peered":"2024-03-30T00:50:37.657930+0000","last_clean":"2024-03-30T00:50:37.657930+0000","last_became_active":"2024-03-30T00:50:37.655623+0000","last_became_peered":"2024-03-30T00:50:37.655623+0000","last_unstale":"2024-03-30T00:50:37.657930+0000","last_undegraded":"2024-03-30T00:50:37.657930+0000","last_fullsized":"2024-03-30T00:50:37.657930+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:29.599324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637936+0000","last_change":"2024-03-30T00:50:37.637936+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637936+0000","last_undegraded":"2024-03-30T00:50:37.637936+0000","last_fullsized":"2024-03-30T00:50:37.637936+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659029+0000","last_change":"2024-03-30T00:50:37.657178+0000","last_active":"2024-03-30T00:50:37.659029+0000","last_peered":"2024-03-30T00:50:37.659029+0000","last_clean":"2024-03-30T00:50:37.659029+0000","last_became_active":"2024-03-30T00:50:37.656814+0000","last_became_peered":"2024-03-30T00:50:37.656814+0000","last_unstale":"2024-03-30T00:50:37.659029+0000","last_undegraded":"2024-03-30T00:50:37.659029+0000","last_fullsized":"2024-03-30T00:50:37.659029+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:11.406073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646745+0000","last_change":"2024-03-30T00:50:37.646745+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646745+0000","last_undegraded":"2024-03-30T00:50:37.646745+0000","last_fullsized":"2024-03-30T00:50:37.646745+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:51:44.264107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.657059+0000","last_change":"2024-03-30T00:50:37.654789+0000","last_active":"2024-03-30T00:50:37.657059+0000","last_peered":"2024-03-30T00:50:37.657059+0000","last_clean":"2024-03-30T00:50:37.657059+0000","last_became_active":"2024-03-30T00:50:37.654488+0000","last_became_peered":"2024-03-30T00:50:37.654488+0000","last_unstale":"2024-03-30T00:50:37.657059+0000","last_undegraded":"2024-03-30T00:50:37.657059+0000","last_fullsized":"2024-03-30T00:50:37.657059+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:41:51.590793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641119+0000","last_change":"2024-03-30T00:50:37.641119+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641119+0000","last_undegraded":"2024-03-30T00:50:37.641119+0000","last_fullsized":"2024-03-30T00:50:37.641119+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659480+0000","last_change":"2024-03-30T00:50:37.658253+0000","last_active":"2024-03-30T00:50:37.659480+0000","last_peered":"2024-03-30T00:50:37.659480+0000","last_clean":"2024-03-30T00:50:37.659480+0000","last_became_active":"2024-03-30T00:50:37.657996+0000","last_became_peered":"2024-03-30T00:50:37.657996+0000","last_unstale":"2024-03-30T00:50:37.659480+0000","last_undegraded":"2024-03-30T00:50:37.659480+0000","last_fullsized":"2024-03-30T00:50:37.659480+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:51:25.787216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.647075+0000","last_change":"2024-03-30T00:50:37.647075+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.647075+0000","last_undegraded":"2024-03-30T00:50:37.647075+0000","last_fullsized":"2024-03-30T00:50:37.647075+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:01.855290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.660222+0000","last_change":"2024-03-30T00:50:37.659011+0000","last_active":"2024-03-30T00:50:37.660222+0000","last_peered":"2024-03-30T00:50:37.660222+0000","last_clean":"2024-03-30T00:50:37.660222+0000","last_became_active":"2024-03-30T00:50:37.658720+0000","last_became_peered":"2024-03-30T00:50:37.658720+0000","last_unstale":"2024-03-30T00:50:37.660222+0000","last_undegraded":"2024-03-30T00:50:37.660222+0000","last_fullsized":"2024-03-30T00:50:37.660222+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:57.692452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642802+0000","last_change":"2024-03-30T00:50:37.642802+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642802+0000","last_undegraded":"2024-03-30T00:50:37.642802+0000","last_fullsized":"2024-03-30T00:50:37.642802+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653217+0000","last_change":"2024-03-30T00:50:37.651177+0000","last_active":"2024-03-30T00:50:37.653217+0000","last_peered":"2024-03-30T00:50:37.653217+0000","last_clean":"2024-03-30T00:50:37.653217+0000","last_became_active":"2024-03-30T00:50:37.650902+0000","last_became_peered":"2024-03-30T00:50:37.650902+0000","last_unstale":"2024-03-30T00:50:37.653217+0000","last_undegraded":"2024-03-30T00:50:37.653217+0000","last_fullsized":"2024-03-30T00:50:37.653217+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:28.311412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642586+0000","last_change":"2024-03-30T00:50:37.642586+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642586+0000","last_undegraded":"2024-03-30T00:50:37.642586+0000","last_fullsized":"2024-03-30T00:50:37.642586+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656565+0000","last_change":"2024-03-30T00:50:37.653495+0000","last_active":"2024-03-30T00:50:37.656565+0000","last_peered":"2024-03-30T00:50:37.656565+0000","last_clean":"2024-03-30T00:50:37.656565+0000","last_became_active":"2024-03-30T00:50:37.652706+0000","last_became_peered":"2024-03-30T00:50:37.652706+0000","last_unstale":"2024-03-30T00:50:37.656565+0000","last_undegraded":"2024-03-30T00:50:37.656565+0000","last_fullsized":"2024-03-30T00:50:37.656565+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:07:01.780302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.644570+0000","last_change":"2024-03-30T00:50:37.644570+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.644570+0000","last_undegraded":"2024-03-30T00:50:37.644570+0000","last_fullsized":"2024-03-30T00:50:37.644570+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:46:27.974997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653040+0000","last_change":"2024-03-30T00:50:37.648270+0000","last_active":"2024-03-30T00:50:37.653040+0000","last_peered":"2024-03-30T00:50:37.653040+0000","last_clean":"2024-03-30T00:50:37.653040+0000","last_became_active":"2024-03-30T00:50:37.647989+0000","last_became_peered":"2024-03-30T00:50:37.647989+0000","last_unstale":"2024-03-30T00:50:37.653040+0000","last_undegraded":"2024-03-30T00:50:37.653040+0000","last_fullsized":"2024-03-30T00:50:37.653040+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:53.076893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646353+0000","last_change":"2024-03-30T00:50:37.646353+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646353+0000","last_undegraded":"2024-03-30T00:50:37.646353+0000","last_fullsized":"2024-03-30T00:50:37.646353+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:05.445602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659297+0000","last_change":"2024-03-30T00:50:37.655775+0000","last_active":"2024-03-30T00:50:37.659297+0000","last_peered":"2024-03-30T00:50:37.659297+0000","last_clean":"2024-03-30T00:50:37.659297+0000","last_became_active":"2024-03-30T00:50:37.655033+0000","last_became_peered":"2024-03-30T00:50:37.655033+0000","last_unstale":"2024-03-30T00:50:37.659297+0000","last_undegraded":"2024-03-30T00:50:37.659297+0000","last_fullsized":"2024-03-30T00:50:37.659297+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:46:26.972427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638573+0000","last_change":"2024-03-30T00:50:37.638573+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638573+0000","last_undegraded":"2024-03-30T00:50:37.638573+0000","last_fullsized":"2024-03-30T00:50:37.638573+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.660319+0000","last_change":"2024-03-30T00:50:37.657373+0000","last_active":"2024-03-30T00:50:37.660319+0000","last_peered":"2024-03-30T00:50:37.660319+0000","last_clean":"2024-03-30T00:50:37.660319+0000","last_became_active":"2024-03-30T00:50:37.657068+0000","last_became_peered":"2024-03-30T00:50:37.657068+0000","last_unstale":"2024-03-30T00:50:37.660319+0000","last_undegraded":"2024-03-30T00:50:37.660319+0000","last_fullsized":"2024-03-30T00:50:37.660319+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:36.963155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640244+0000","last_change":"2024-03-30T00:50:37.640244+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640244+0000","last_undegraded":"2024-03-30T00:50:37.640244+0000","last_fullsized":"2024-03-30T00:50:37.640244+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:02:07.606929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.655268+0000","last_change":"2024-03-30T00:50:37.653596+0000","last_active":"2024-03-30T00:50:37.655268+0000","last_peered":"2024-03-30T00:50:37.655268+0000","last_clean":"2024-03-30T00:50:37.655268+0000","last_became_active":"2024-03-30T00:50:37.653299+0000","last_became_peered":"2024-03-30T00:50:37.653299+0000","last_unstale":"2024-03-30T00:50:37.655268+0000","last_undegraded":"2024-03-30T00:50:37.655268+0000","last_fullsized":"2024-03-30T00:50:37.655268+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:06.142142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641714+0000","last_change":"2024-03-30T00:50:37.641714+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641714+0000","last_undegraded":"2024-03-30T00:50:37.641714+0000","last_fullsized":"2024-03-30T00:50:37.641714+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.654489+0000","last_change":"2024-03-30T00:50:37.649310+0000","last_active":"2024-03-30T00:50:37.654489+0000","last_peered":"2024-03-30T00:50:37.654489+0000","last_clean":"2024-03-30T00:50:37.654489+0000","last_became_active":"2024-03-30T00:50:37.649015+0000","last_became_peered":"2024-03-30T00:50:37.649015+0000","last_unstale":"2024-03-30T00:50:37.654489+0000","last_undegraded":"2024-03-30T00:50:37.654489+0000","last_fullsized":"2024-03-30T00:50:37.654489+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:00.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637343+0000","last_change":"2024-03-30T00:50:37.637343+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637343+0000","last_undegraded":"2024-03-30T00:50:37.637343+0000","last_fullsized":"2024-03-30T00:50:37.637343+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658749+0000","last_change":"2024-03-30T00:50:37.655900+0000","last_active":"2024-03-30T00:50:37.658749+0000","last_peered":"2024-03-30T00:50:37.658749+0000","last_clean":"2024-03-30T00:50:37.658749+0000","last_became_active":"2024-03-30T00:50:37.655601+0000","last_became_peered":"2024-03-30T00:50:37.655601+0000","last_unstale":"2024-03-30T00:50:37.658749+0000","last_undegraded":"2024-03-30T00:50:37.658749+0000","last_fullsized":"2024-03-30T00:50:37.658749+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:33.394770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641239+0000","last_change":"2024-03-30T00:50:37.641239+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641239+0000","last_undegraded":"2024-03-30T00:50:37.641239+0000","last_fullsized":"2024-03-30T00:50:37.641239+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:09:05.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656296+0000","last_change":"2024-03-30T00:50:37.649475+0000","last_active":"2024-03-30T00:50:37.656296+0000","last_peered":"2024-03-30T00:50:37.656296+0000","last_clean":"2024-03-30T00:50:37.656296+0000","last_became_active":"2024-03-30T00:50:37.649055+0000","last_became_peered":"2024-03-30T00:50:37.649055+0000","last_unstale":"2024-03-30T00:50:37.656296+0000","last_undegraded":"2024-03-30T00:50:37.656296+0000","last_fullsized":"2024-03-30T00:50:37.656296+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:12.549064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643208+0000","last_change":"2024-03-30T00:50:37.643208+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643208+0000","last_undegraded":"2024-03-30T00:50:37.643208+0000","last_fullsized":"2024-03-30T00:50:37.643208+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:47.211883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658519+0000","last_change":"2024-03-30T00:50:37.656047+0000","last_active":"2024-03-30T00:50:37.658519+0000","last_peered":"2024-03-30T00:50:37.658519+0000","last_clean":"2024-03-30T00:50:37.658519+0000","last_became_active":"2024-03-30T00:50:37.655751+0000","last_became_peered":"2024-03-30T00:50:37.655751+0000","last_unstale":"2024-03-30T00:50:37.658519+0000","last_undegraded":"2024-03-30T00:50:37.658519+0000","last_fullsized":"2024-03-30T00:50:37.658519+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:00.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636644+0000","last_change":"2024-03-30T00:50:37.636644+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636644+0000","last_undegraded":"2024-03-30T00:50:37.636644+0000","last_fullsized":"2024-03-30T00:50:37.636644+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:52:46.481754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.651675+0000","last_change":"2024-03-30T00:50:37.646336+0000","last_active":"2024-03-30T00:50:37.651675+0000","last_peered":"2024-03-30T00:50:37.651675+0000","last_clean":"2024-03-30T00:50:37.651675+0000","last_became_active":"2024-03-30T00:50:37.646081+0000","last_became_peered":"2024-03-30T00:50:37.646081+0000","last_unstale":"2024-03-30T00:50:37.651675+0000","last_undegraded":"2024-03-30T00:50:37.651675+0000","last_fullsized":"2024-03-30T00:50:37.651675+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:40.726869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645084+0000","last_change":"2024-03-30T00:50:37.645084+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645084+0000","last_undegraded":"2024-03-30T00:50:37.645084+0000","last_fullsized":"2024-03-30T00:50:37.645084+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:04:19.985679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.652956+0000","last_change":"2024-03-30T00:50:37.648309+0000","last_active":"2024-03-30T00:50:37.652956+0000","last_peered":"2024-03-30T00:50:37.652956+0000","last_clean":"2024-03-30T00:50:37.652956+0000","last_became_active":"2024-03-30T00:50:37.646903+0000","last_became_peered":"2024-03-30T00:50:37.646903+0000","last_unstale":"2024-03-30T00:50:37.652956+0000","last_undegraded":"2024-03-30T00:50:37.652956+0000","last_fullsized":"2024-03-30T00:50:37.652956+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:20.839135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642635+0000","last_change":"2024-03-30T00:50:37.642635+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642635+0000","last_undegraded":"2024-03-30T00:50:37.642635+0000","last_fullsized":"2024-03-30T00:50:37.642635+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659418+0000","last_change":"2024-03-30T00:50:37.656069+0000","last_active":"2024-03-30T00:50:37.659418+0000","last_peered":"2024-03-30T00:50:37.659418+0000","last_clean":"2024-03-30T00:50:37.659418+0000","last_became_active":"2024-03-30T00:50:37.655828+0000","last_became_peered":"2024-03-30T00:50:37.655828+0000","last_unstale":"2024-03-30T00:50:37.659418+0000","last_undegraded":"2024-03-30T00:50:37.659418+0000","last_fullsized":"2024-03-30T00:50:37.659418+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:46.209312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643298+0000","last_change":"2024-03-30T00:50:37.643298+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643298+0000","last_undegraded":"2024-03-30T00:50:37.643298+0000","last_fullsized":"2024-03-30T00:50:37.643298+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:37.965725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656366+0000","last_change":"2024-03-30T00:50:37.650090+0000","last_active":"2024-03-30T00:50:37.656366+0000","last_peered":"2024-03-30T00:50:37.656366+0000","last_clean":"2024-03-30T00:50:37.656366+0000","last_became_active":"2024-03-30T00:50:37.649527+0000","last_became_peered":"2024-03-30T00:50:37.649527+0000","last_unstale":"2024-03-30T00:50:37.656366+0000","last_undegraded":"2024-03-30T00:50:37.656366+0000","last_fullsized":"2024-03-30T00:50:37.656366+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:37.177730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638050+0000","last_change":"2024-03-30T00:50:37.638050+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638050+0000","last_undegraded":"2024-03-30T00:50:37.638050+0000","last_fullsized":"2024-03-30T00:50:37.638050+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:19.197836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.654023+0000","last_change":"2024-03-30T00:50:37.648973+0000","last_active":"2024-03-30T00:50:37.654023+0000","last_peered":"2024-03-30T00:50:37.654023+0000","last_clean":"2024-03-30T00:50:37.654023+0000","last_became_active":"2024-03-30T00:50:37.648703+0000","last_became_peered":"2024-03-30T00:50:37.648703+0000","last_unstale":"2024-03-30T00:50:37.654023+0000","last_undegraded":"2024-03-30T00:50:37.654023+0000","last_fullsized":"2024-03-30T00:50:37.654023+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:53:36.133173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637305+0000","last_change":"2024-03-30T00:50:37.637305+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637305+0000","last_undegraded":"2024-03-30T00:50:37.637305+0000","last_fullsized":"2024-03-30T00:50:37.637305+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659783+0000","last_change":"2024-03-30T00:50:37.658787+0000","last_active":"2024-03-30T00:50:37.659783+0000","last_peered":"2024-03-30T00:50:37.659783+0000","last_clean":"2024-03-30T00:50:37.659783+0000","last_became_active":"2024-03-30T00:50:37.658512+0000","last_became_peered":"2024-03-30T00:50:37.658512+0000","last_unstale":"2024-03-30T00:50:37.659783+0000","last_undegraded":"2024-03-30T00:50:37.659783+0000","last_fullsized":"2024-03-30T00:50:37.659783+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:55:31.799226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637015+0000","last_change":"2024-03-30T00:50:37.637015+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637015+0000","last_undegraded":"2024-03-30T00:50:37.637015+0000","last_fullsized":"2024-03-30T00:50:37.637015+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658420+0000","last_change":"2024-03-30T00:50:37.656511+0000","last_active":"2024-03-30T00:50:37.658420+0000","last_peered":"2024-03-30T00:50:37.658420+0000","last_clean":"2024-03-30T00:50:37.658420+0000","last_became_active":"2024-03-30T00:50:37.655106+0000","last_became_peered":"2024-03-30T00:50:37.655106+0000","last_unstale":"2024-03-30T00:50:37.658420+0000","last_undegraded":"2024-03-30T00:50:37.658420+0000","last_fullsized":"2024-03-30T00:50:37.658420+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:04:28.343406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641788+0000","last_change":"2024-03-30T00:50:37.641788+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641788+0000","last_undegraded":"2024-03-30T00:50:37.641788+0000","last_fullsized":"2024-03-30T00:50:37.641788+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:34:31.524194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.657263+0000","last_change":"2024-03-30T00:50:37.655230+0000","last_active":"2024-03-30T00:50:37.657263+0000","last_peered":"2024-03-30T00:50:37.657263+0000","last_clean":"2024-03-30T00:50:37.657263+0000","last_became_active":"2024-03-30T00:50:37.654933+0000","last_became_peered":"2024-03-30T00:50:37.654933+0000","last_unstale":"2024-03-30T00:50:37.657263+0000","last_undegraded":"2024-03-30T00:50:37.657263+0000","last_fullsized":"2024-03-30T00:50:37.657263+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:59.607374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.635996+0000","last_change":"2024-03-30T00:50:37.635996+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.635996+0000","last_undegraded":"2024-03-30T00:50:37.635996+0000","last_fullsized":"2024-03-30T00:50:37.635996+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:52:14.331871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.650896+0000","last_change":"2024-03-30T00:50:37.647611+0000","last_active":"2024-03-30T00:50:37.650896+0000","last_peered":"2024-03-30T00:50:37.650896+0000","last_clean":"2024-03-30T00:50:37.650896+0000","last_became_active":"2024-03-30T00:50:37.647311+0000","last_became_peered":"2024-03-30T00:50:37.647311+0000","last_unstale":"2024-03-30T00:50:37.650896+0000","last_undegraded":"2024-03-30T00:50:37.650896+0000","last_fullsized":"2024-03-30T00:50:37.650896+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:21.467231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642870+0000","last_change":"2024-03-30T00:50:37.642870+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642870+0000","last_undegraded":"2024-03-30T00:50:37.642870+0000","last_fullsized":"2024-03-30T00:50:37.642870+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656761+0000","last_change":"2024-03-30T00:50:37.651832+0000","last_active":"2024-03-30T00:50:37.656761+0000","last_peered":"2024-03-30T00:50:37.656761+0000","last_clean":"2024-03-30T00:50:37.656761+0000","last_became_active":"2024-03-30T00:50:37.651529+0000","last_became_peered":"2024-03-30T00:50:37.651529+0000","last_unstale":"2024-03-30T00:50:37.656761+0000","last_undegraded":"2024-03-30T00:50:37.656761+0000","last_fullsized":"2024-03-30T00:50:37.656761+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:19.643626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636935+0000","last_change":"2024-03-30T00:50:37.636935+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636935+0000","last_undegraded":"2024-03-30T00:50:37.636935+0000","last_fullsized":"2024-03-30T00:50:37.636935+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656496+0000","last_change":"2024-03-30T00:50:37.654139+0000","last_active":"2024-03-30T00:50:37.656496+0000","last_peered":"2024-03-30T00:50:37.656496+0000","last_clean":"2024-03-30T00:50:37.656496+0000","last_became_active":"2024-03-30T00:50:37.653825+0000","last_became_peered":"2024-03-30T00:50:37.653825+0000","last_unstale":"2024-03-30T00:50:37.656496+0000","last_undegraded":"2024-03-30T00:50:37.656496+0000","last_fullsized":"2024-03-30T00:50:37.656496+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:47.043924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638023+0000","last_change":"2024-03-30T00:50:37.638023+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638023+0000","last_undegraded":"2024-03-30T00:50:37.638023+0000","last_fullsized":"2024-03-30T00:50:37.638023+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658174+0000","last_change":"2024-03-30T00:50:37.654797+0000","last_active":"2024-03-30T00:50:37.658174+0000","last_peered":"2024-03-30T00:50:37.658174+0000","last_clean":"2024-03-30T00:50:37.658174+0000","last_became_active":"2024-03-30T00:50:37.654506+0000","last_became_peered":"2024-03-30T00:50:37.654506+0000","last_unstale":"2024-03-30T00:50:37.658174+0000","last_undegraded":"2024-03-30T00:50:37.658174+0000","last_fullsized":"2024-03-30T00:50:37.658174+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:26.653343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637355+0000","last_change":"2024-03-30T00:50:37.637355+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637355+0000","last_undegraded":"2024-03-30T00:50:37.637355+0000","last_fullsized":"2024-03-30T00:50:37.637355+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653101+0000","last_change":"2024-03-30T00:50:37.648683+0000","last_active":"2024-03-30T00:50:37.653101+0000","last_peered":"2024-03-30T00:50:37.653101+0000","last_clean":"2024-03-30T00:50:37.653101+0000","last_became_active":"2024-03-30T00:50:37.648391+0000","last_became_peered":"2024-03-30T00:50:37.648391+0000","last_unstale":"2024-03-30T00:50:37.653101+0000","last_undegraded":"2024-03-30T00:50:37.653101+0000","last_fullsized":"2024-03-30T00:50:37.653101+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:13.838899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":271,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-03-30T00:50:36.635729+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:50:36.635729+0000","last_peered":"2024-03-30T00:50:36.635729+0000","last_clean":"2024-03-30T00:50:36.635729+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:50:36.635729+0000","last_undegraded":"2024-03-30T00:50:36.635729+0000","last_fullsized":"2024-03-30T00:50:36.635729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574982,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2113536,"data_stored":1290047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":0.97599999999999998,"15min":1.0860000000000001},"last":0.621},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.47399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84999999999999998,"5min":0.98299999999999998,"15min":1.149},"last":0.55500000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.503,"5min":0.45800000000000002,"15min":0.436},"max":{"1min":0.90800000000000003,"5min":1.03,"15min":1.03},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93000000000000005,"5min":0.95199999999999996,"15min":1.3029999999999999},"last":0.70399999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.76600000000000001},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.45000000000000001},"max":{"1min":1.0129999999999999,"5min":1.085,"15min":1.6020000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.73099999999999998,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95399999999999996,"5min":1.034,"15min":1.6619999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.75900000000000001},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99099999999999999,"5min":0.99299999999999999,"15min":2.0150000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.80000000000000004},"min":{"1min":0.63800000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.9490000000000001},"last":0.76400000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.37},"max":{"1min":0.97299999999999998,"5min":1.2,"15min":1.2},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.42999999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.97899999999999998},"last":0.52700000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96299999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":0.99199999999999999,"15min":1.026},"last":0.60699999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.92700000000000005,"5min":0.94599999999999995,"15min":1.214},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89200000000000002,"5min":1.0189999999999999,"15min":1.0309999999999999},"last":0.64900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1531904,"data_stored":706274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71599999999999997,"15min":0.79800000000000004},"min":{"1min":0.437,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.85799999999999998,"5min":1.2969999999999999,"15min":4.0490000000000004},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.68799999999999994},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":1.278,"15min":1.47},"last":0.96199999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.438,"15min":0.36899999999999999},"max":{"1min":0.95799999999999996,"5min":1.204,"15min":4.3170000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.85299999999999998},"min":{"1min":0.58399999999999996,"5min":0.47199999999999998,"15min":0.42999999999999999},"max":{"1min":0.92900000000000005,"5min":1.1120000000000001,"15min":4.4160000000000004},"last":0.91700000000000004}]},{"osd":2,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.745,"15min":0.84999999999999998},"min":{"1min":0.63400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98299999999999998,"5min":1.24,"15min":4.375},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.84299999999999997},"min":{"1min":0.51700000000000002,"5min":0.42999999999999999,"15min":0.38600000000000001},"max":{"1min":0.92500000000000004,"5min":1.3129999999999999,"15min":4.2960000000000003},"last":0.89900000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.86799999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0249999999999999,"5min":1.2390000000000001,"15min":4.3399999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73199999999999998,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.49099999999999999,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.258,"15min":4.6020000000000003},"last":0.85399999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.39200000000000002,"5min":0.39100000000000001,"15min":0.35999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.1870000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.73199999999999998},"min":{"1min":0.441,"5min":0.372,"15min":0.372},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.036},"last":0.82599999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.65700000000000003},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0720000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0109999999999999,"15min":1.101},"last":0.67900000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97499999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.45700000000000002},"max":{"1min":1.0860000000000001,"5min":1.103,"15min":1.103},"last":0.70499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574982,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":0.83299999999999996,"5min":0.97599999999999998,"15min":1.095},"last":0.63},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68700000000000006,"15min":0.73199999999999998},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":1.0489999999999999,"15min":1.7569999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.71999999999999997},"min":{"1min":0.47899999999999998,"5min":0.46600000000000003,"15min":0.43099999999999999},"max":{"1min":0.86599999999999999,"5min":0.98199999999999998,"15min":1.7330000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70999999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42399999999999999},"max":{"1min":0.92400000000000004,"5min":0.98799999999999999,"15min":1.3260000000000001},"last":0.73299999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.44600000000000001},"max":{"1min":0.89900000000000002,"5min":1.131,"15min":1.131},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.45900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.877,"5min":1.0609999999999999,"15min":1.2789999999999999},"last":0.51400000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.75},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.89100000000000001,"5min":1.0740000000000001,"15min":1.7},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.96099999999999997,"5min":1.1160000000000001,"15min":1.181},"last":0.56399999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.60299999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.309},"max":{"1min":0.876,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.433},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69199999999999995,"15min":0.73399999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91200000000000003,"5min":0.97799999999999998,"15min":1.615},"last":0.67000000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.84099999999999997,"5min":0.91100000000000003,"15min":0.97399999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.626,"15min":0.67300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.35199999999999998},"max":{"1min":0.84999999999999998,"5min":0.93700000000000006,"15min":1.569},"last":0.59199999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.71499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.377},"max":{"1min":0.85399999999999998,"5min":0.92200000000000004,"15min":1.0149999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88400000000000001,"5min":1.167,"15min":1.383},"last":0.64800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.34899999999999998},"max":{"1min":0.92400000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.67400000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98599999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.502}]},{"osd":1,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":0.92600000000000005,"5min":0.95899999999999996,"15min":1.2909999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.40699999999999997},"max":{"1min":1.006,"5min":1.113,"15min":1.113},"last":0.60799999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.59199999999999997,"5min":0.48999999999999999,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.73899999999999999,"15min":0.75},"min":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.46700000000000003},"max":{"1min":0.94899999999999995,"5min":1.163,"15min":1.228},"last":0.68300000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73599999999999999,"15min":0.71699999999999997},"min":{"1min":0.501,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.27},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.76000000000000001},"min":{"1min":0.59099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1,"15min":1.1739999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.621,"15min":0.65800000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.89400000000000002,"15min":1.329},"last":0.754},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66100000000000003,"15min":0.70399999999999996},"min":{"1min":0.57399999999999995,"5min":0.34799999999999998,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":0.99099999999999999,"15min":1.3819999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.65700000000000003},"min":{"1min":0.45700000000000002,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":1.2130000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.69699999999999995},"min":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.95799999999999996,"15min":1.3220000000000001},"last":0.85599999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.628,"15min":0.69099999999999995},"min":{"1min":0.46700000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":0.91500000000000004,"15min":1.6910000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.73399999999999999,"15min":0.78000000000000003},"min":{"1min":0.59699999999999998,"5min":0.49299999999999999,"15min":0.46600000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.5009999999999999},"last":0.873}]}]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.435,"15min":0.45500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.58699999999999997,"5min":0.748,"15min":1.0529999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.49099999999999999,"15min":0.51400000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.24199999999999999},"max":{"1min":0.63400000000000001,"5min":0.75,"15min":0.79100000000000004},"last":0.39500000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49399999999999999,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.80900000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63800000000000001,"5min":0.73399999999999999,"15min":1.2150000000000001},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.78300000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.46300000000000002},"max":{"1min":0.95699999999999996,"5min":1.1259999999999999,"15min":1.2470000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77100000000000002,"15min":0.77600000000000002},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.079,"15min":1.0860000000000001},"last":0.97099999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77100000000000002,"15min":0.77900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.51100000000000001},"max":{"1min":1.024,"5min":1.0369999999999999,"15min":1.125},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.77900000000000003,"15min":0.79100000000000004},"min":{"1min":0.628,"5min":0.52200000000000002,"15min":0.48599999999999999},"max":{"1min":0.86299999999999999,"5min":1.133,"15min":1.2390000000000001},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77200000000000002,"15min":0.79700000000000004},"min":{"1min":0.59999999999999998,"5min":0.51400000000000001,"15min":0.499},"max":{"1min":0.90100000000000002,"5min":1.091,"15min":1.226},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.038,"5min":1.1120000000000001,"15min":1.167},"last":0.94899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82999999999999996,"15min":0.84099999999999997},"min":{"1min":0.71399999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.123,"5min":1.1910000000000001,"15min":1.202},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83499999999999996,"15min":0.86299999999999999},"min":{"1min":0.57299999999999995,"5min":0.54900000000000004,"15min":0.497},"max":{"1min":1.095,"5min":1.1639999999999999,"15min":1.3160000000000001},"last":1.0189999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56899999999999995,"5min":0.70299999999999996,"15min":4.0999999999999996},"last":1.339},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.48799999999999999,"15min":0.50600000000000001},"min":{"1min":0.40699999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83999999999999997,"5min":0.84099999999999997,"15min":1.9239999999999999},"last":1.0900000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32600000000000001},"max":{"1min":0.61799999999999999,"5min":0.81299999999999994,"15min":3.1139999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.52400000000000002,"15min":0.53400000000000003},"min":{"1min":0.40600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.748,"5min":0.752,"15min":2.4159999999999999},"last":1.1830000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.499,"15min":0.497},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70099999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.749,"5min":0.88800000000000001,"15min":1.907},"last":0.95599999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77700000000000002,"15min":0.80100000000000005},"min":{"1min":0.59699999999999998,"5min":0.504,"15min":0.504},"max":{"1min":1.0900000000000001,"5min":1.1919999999999999,"15min":1.9470000000000001},"last":1.129},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78700000000000003,"15min":0.82099999999999995},"min":{"1min":0.628,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.163,"5min":1.2110000000000001,"15min":1.972},"last":1.2649999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.1220000000000001,"5min":1.2549999999999999,"15min":2.4569999999999999},"last":1.3},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.78900000000000003,"15min":0.81000000000000005},"min":{"1min":0.58399999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0389999999999999,"5min":1.286,"15min":2.8039999999999998},"last":0.70299999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0980000000000001,"5min":1.177,"15min":2.8849999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.0780000000000001,"5min":1.242,"15min":1.867},"last":0.996}]},{"osd":7,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79500000000000004,"15min":0.83699999999999997},"min":{"1min":0.60199999999999998,"5min":0.54700000000000004,"15min":0.495},"max":{"1min":1.0429999999999999,"5min":1.1140000000000001,"15min":2.3690000000000002},"last":1.2270000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.83799999999999997},"min":{"1min":0.70499999999999996,"5min":0.55800000000000005,"15min":0.52700000000000002},"max":{"1min":0.98999999999999999,"5min":1.365,"15min":2.851},"last":1.0089999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44,"15min":0.42799999999999999},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.253},"max":{"1min":0.63600000000000001,"5min":0.72199999999999998,"15min":0.80300000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.48399999999999999,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.69999999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33300000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.45900000000000002},"min":{"1min":0.39200000000000002,"5min":0.29999999999999999,"15min":0.29199999999999998},"max":{"1min":0.59199999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.372},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.64600000000000002,"5min":0.84099999999999997,"15min":0.89400000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.45300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.69199999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.499},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.38600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72999999999999998,"5min":0.83199999999999996,"15min":0.83999999999999997},"last":0.40600000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.75600000000000001,"15min":0.72899999999999998},"min":{"1min":0.58099999999999996,"5min":0.49199999999999999,"15min":0.38800000000000001},"max":{"1min":1.034,"5min":1.175,"15min":1.175},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76900000000000002,"15min":0.754},"min":{"1min":0.66300000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98599999999999999,"5min":1.079,"15min":1.2},"last":0.81399999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.76000000000000001},"min":{"1min":0.621,"5min":0.56799999999999995,"15min":0.51600000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.3180000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79400000000000004,"15min":0.77100000000000002},"min":{"1min":0.60299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.129,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.87}]},{"osd":6,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79700000000000004,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.0389999999999999,"5min":1.179,"15min":1.179},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0509999999999999,"15min":1.2150000000000001},"last":0.89300000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80300000000000005,"15min":0.79700000000000004},"min":{"1min":0.64500000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.024,"5min":1.2050000000000001,"15min":1.2669999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.81799999999999995,"15min":0.79200000000000004},"min":{"1min":0.69299999999999995,"5min":0.56200000000000006,"15min":0.48499999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.236},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2113536,"data_stored":1290047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.45700000000000002},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.245},"max":{"1min":0.69799999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.505},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68400000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.51000000000000001},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.67100000000000004,"5min":0.82599999999999996,"15min":1.02},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.432,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":0.77900000000000003,"15min":1.006},"last":0.54400000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.68000000000000005,"5min":0.754,"15min":0.84799999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.46200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":0.88900000000000001,"15min":0.96899999999999997},"last":0.60999999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.47899999999999998},"max":{"1min":1.1040000000000001,"5min":1.1140000000000001,"15min":1.5169999999999999},"last":0.878},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.46400000000000002},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.252},"last":0.81000000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.55200000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.3080000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0780000000000001,"15min":1.462},"last":0.746}]},{"osd":6,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79200000000000004,"15min":0.79900000000000004},"min":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.54500000000000004},"max":{"1min":1.0429999999999999,"5min":1.131,"15min":1.5920000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80000000000000004,"15min":0.79000000000000004},"min":{"1min":0.625,"5min":0.54600000000000004,"15min":0.46999999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.2230000000000001},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.81299999999999994},"min":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.3600000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.60399999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.556},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:50:42.546 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:50:42.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.711+0000 7f4ab9400640 1 Processor -- start 2024-03-30T00:50:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.711+0000 7f4ab9400640 1 -- start start 2024-03-30T00:50:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab9400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4ab40cc3f0 0x7f4ab40ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab9400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ab40cdb60 0x7f4ab415be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab9400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4ab415c380 0x7f4ab40b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab9400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab41551d0 con 0x7f4ab415c380 2024-03-30T00:50:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab9400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab4065e30 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab9400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab40ce0d0 con 0x7f4ab40cdb60 2024-03-30T00:50:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab3200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4ab415c380 0x7f4ab40b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab3200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4ab415c380 0x7f4ab40b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58874/0 (socket says 172.21.15.66:58874) 2024-03-30T00:50:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab3200640 1 -- 172.21.15.66:0/1834044270 learned_addr learned my addr 172.21.15.66:0/1834044270 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab2800640 1 --2- 172.21.15.66:0/1834044270 >> v2:172.21.15.156:3300/0 conn(0x7f4ab40cc3f0 0x7f4ab40ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4aabe00640 1 --2- 172.21.15.66:0/1834044270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ab40cdb60 0x7f4ab415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab3200640 1 -- 172.21.15.66:0/1834044270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ab40cdb60 msgr2=0x7f4ab415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab3200640 1 --2- 172.21.15.66:0/1834044270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ab40cdb60 0x7f4ab415be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab3200640 1 -- 172.21.15.66:0/1834044270 >> v2:172.21.15.156:3300/0 conn(0x7f4ab40cc3f0 msgr2=0x7f4ab40ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab3200640 1 --2- 172.21.15.66:0/1834044270 >> v2:172.21.15.156:3300/0 conn(0x7f4ab40cc3f0 0x7f4ab40ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab3200640 1 -- 172.21.15.66:0/1834044270 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab40ce3b0 con 0x7f4ab415c380 2024-03-30T00:50:42.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.715+0000 7f4ab3200640 1 --2- 172.21.15.66:0/1834044270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab415c380 0x7f4ab40b3f70 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4070360 tx=0x7f4aac01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747407a58975a609 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:42.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab1e00640 1 -- 172.21.15.66:0/1834044270 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0a5670 con 0x7f4ab415c380 2024-03-30T00:50:42.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab1e00640 1 -- 172.21.15.66:0/1834044270 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac0a57d0 con 0x7f4ab415c380 2024-03-30T00:50:42.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab1e00640 1 -- 172.21.15.66:0/1834044270 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0a5b60 con 0x7f4ab415c380 2024-03-30T00:50:42.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 -- 172.21.15.66:0/1834044270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab415c380 msgr2=0x7f4ab40b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 --2- 172.21.15.66:0/1834044270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab415c380 0x7f4ab40b3f70 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4070360 tx=0x7f4aac01c9f0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 -- 172.21.15.66:0/1834044270 shutdown_connections 2024-03-30T00:50:42.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 --2- 172.21.15.66:0/1834044270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab415c380 0x7f4ab40b3f70 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 --2- 172.21.15.66:0/1834044270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ab40cdb60 0x7f4ab415be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 --2- 172.21.15.66:0/1834044270 >> v2:172.21.15.156:3300/0 conn(0x7f4ab40cc3f0 0x7f4ab40ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 -- 172.21.15.66:0/1834044270 >> 172.21.15.66:0/1834044270 conn(0x7f4ab4071750 msgr2=0x7f4ab4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:42.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 -- 172.21.15.66:0/1834044270 shutdown_connections 2024-03-30T00:50:42.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 -- 172.21.15.66:0/1834044270 wait complete. 2024-03-30T00:50:42.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 Processor -- start 2024-03-30T00:50:42.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 -- start start 2024-03-30T00:50:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab40cc3f0 0x7f4ab41643c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ab40cdb60 0x7f4ab414b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ab415c380 0x7f4ab41f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab414b740 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab41f9000 con 0x7f4ab415c380 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab9400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab414bc10 con 0x7f4ab40cdb60 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab40cc3f0 0x7f4ab41643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab40cc3f0 0x7f4ab41643c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58890/0 (socket says 172.21.15.66:58890) 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab2800640 1 -- 172.21.15.66:0/2714563778 learned_addr learned my addr 172.21.15.66:0/2714563778 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4aabe00640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ab40cdb60 0x7f4ab414b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.719+0000 7f4ab3200640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ab415c380 0x7f4ab41f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4ab2800640 1 -- 172.21.15.66:0/2714563778 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ab40cdb60 msgr2=0x7f4ab414b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4ab2800640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ab40cdb60 0x7f4ab414b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4ab2800640 1 -- 172.21.15.66:0/2714563778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ab415c380 msgr2=0x7f4ab41f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4ab2800640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ab415c380 0x7f4ab41f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4ab2800640 1 -- 172.21.15.66:0/2714563778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac009b30 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4ab2800640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab40cc3f0 0x7f4ab41643c0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400d800 tx=0x7f4aa400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:42.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4aaaa00640 1 -- 172.21.15.66:0/2714563778 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40096a0 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4aaaa00640 1 -- 172.21.15.66:0/2714563778 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa4009800 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4aaaa00640 1 -- 172.21.15.66:0/2714563778 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4009b70 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4ab9400640 1 -- 172.21.15.66:0/2714563778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab414fb20 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4ab9400640 1 -- 172.21.15.66:0/2714563778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab41f92a0 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4ab9400640 1 -- 172.21.15.66:0/2714563778 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab4066c30 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.723+0000 7f4aaaa00640 1 -- 172.21.15.66:0/2714563778 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4aa4009cd0 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.727+0000 7f4aaaa00640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4a90066690 0x7f4a90068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.727+0000 7f4aaaa00640 1 -- 172.21.15.66:0/2714563778 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4aa4084d30 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.727+0000 7f4aabe00640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4a90066690 0x7f4a90068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.727+0000 7f4aabe00640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4a90066690 0x7f4a90068b50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4ab41f8ec0 tx=0x7f4a9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:42.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.727+0000 7f4aaaa00640 1 -- 172.21.15.66:0/2714563778 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4aa40520f0 con 0x7f4ab40cc3f0 2024-03-30T00:50:42.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.891+0000 7f4ab9400640 1 -- 172.21.15.66:0/2714563778 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ab40ae650 con 0x7f4a90066690 2024-03-30T00:50:42.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.895+0000 7f4aaaa00640 1 -- 172.21.15.66:0/2714563778 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286510 (secure 0 0 0) 0x7f4a9c010960 con 0x7f4a90066690 2024-03-30T00:50:42.908 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:42.909 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.899+0000 7f4ab9400640 1 -- 172.21.15.66:0/2714563778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4a90066690 msgr2=0x7f4a90068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.899+0000 7f4ab9400640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4a90066690 0x7f4a90068b50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4ab41f8ec0 tx=0x7f4a9c008040 comp rx=0 tx=0).stop 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.899+0000 7f4ab9400640 1 -- 172.21.15.66:0/2714563778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab40cc3f0 msgr2=0x7f4ab41643c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.899+0000 7f4ab9400640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab40cc3f0 0x7f4ab41643c0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400d800 tx=0x7f4aa400dcc0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.899+0000 7f4ab9400640 1 -- 172.21.15.66:0/2714563778 shutdown_connections 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.899+0000 7f4ab9400640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ab415c380 0x7f4ab41f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.903+0000 7f4ab9400640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ab40cdb60 0x7f4ab414b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.903+0000 7f4ab9400640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4a90066690 0x7f4a90068b50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.903+0000 7f4ab9400640 1 --2- 172.21.15.66:0/2714563778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab40cc3f0 0x7f4ab41643c0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.903+0000 7f4ab9400640 1 -- 172.21.15.66:0/2714563778 >> 172.21.15.66:0/2714563778 conn(0x7f4ab4071750 msgr2=0x7f4ab40aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.903+0000 7f4ab9400640 1 -- 172.21.15.66:0/2714563778 shutdown_connections 2024-03-30T00:50:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:42.903+0000 7f4ab9400640 1 -- 172.21.15.66:0/2714563778 wait complete. 2024-03-30T00:50:42.929 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-03-30T00:50:41.000126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261988,"kb_used_data":13044,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712732,"statfs":{"total":773094113280,"available":772825837568,"internally_reserved":0,"allocated":13357056,"data_stored":6784763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.373741"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.639401+0000","last_change":"2024-03-30T00:50:37.639401+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.639401+0000","last_undegraded":"2024-03-30T00:50:37.639401+0000","last_fullsized":"2024-03-30T00:50:37.639401+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646197+0000","last_change":"2024-03-30T00:50:37.646197+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646197+0000","last_undegraded":"2024-03-30T00:50:37.646197+0000","last_fullsized":"2024-03-30T00:50:37.646197+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643997+0000","last_change":"2024-03-30T00:50:37.643997+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643997+0000","last_undegraded":"2024-03-30T00:50:37.643997+0000","last_fullsized":"2024-03-30T00:50:37.643997+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:53:58.506502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637414+0000","last_change":"2024-03-30T00:50:37.637414+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637414+0000","last_undegraded":"2024-03-30T00:50:37.637414+0000","last_fullsized":"2024-03-30T00:50:37.637414+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638917+0000","last_change":"2024-03-30T00:50:37.638917+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638917+0000","last_undegraded":"2024-03-30T00:50:37.638917+0000","last_fullsized":"2024-03-30T00:50:37.638917+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637248+0000","last_change":"2024-03-30T00:50:37.637248+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637248+0000","last_undegraded":"2024-03-30T00:50:37.637248+0000","last_fullsized":"2024-03-30T00:50:37.637248+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:16.922513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636753+0000","last_change":"2024-03-30T00:50:37.636753+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636753+0000","last_undegraded":"2024-03-30T00:50:37.636753+0000","last_fullsized":"2024-03-30T00:50:37.636753+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:10:54.265531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642083+0000","last_change":"2024-03-30T00:50:37.642083+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642083+0000","last_undegraded":"2024-03-30T00:50:37.642083+0000","last_fullsized":"2024-03-30T00:50:37.642083+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638063+0000","last_change":"2024-03-30T00:50:37.638063+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638063+0000","last_undegraded":"2024-03-30T00:50:37.638063+0000","last_fullsized":"2024-03-30T00:50:37.638063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.639455+0000","last_change":"2024-03-30T00:50:37.639455+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.639455+0000","last_undegraded":"2024-03-30T00:50:37.639455+0000","last_fullsized":"2024-03-30T00:50:37.639455+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:21:22.322038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640852+0000","last_change":"2024-03-30T00:50:37.640852+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640852+0000","last_undegraded":"2024-03-30T00:50:37.640852+0000","last_fullsized":"2024-03-30T00:50:37.640852+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638906+0000","last_change":"2024-03-30T00:50:37.638906+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638906+0000","last_undegraded":"2024-03-30T00:50:37.638906+0000","last_fullsized":"2024-03-30T00:50:37.638906+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:09:58.609620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641531+0000","last_change":"2024-03-30T00:50:37.641531+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641531+0000","last_undegraded":"2024-03-30T00:50:37.641531+0000","last_fullsized":"2024-03-30T00:50:37.641531+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.647544+0000","last_change":"2024-03-30T00:50:37.647544+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.647544+0000","last_undegraded":"2024-03-30T00:50:37.647544+0000","last_fullsized":"2024-03-30T00:50:37.647544+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638432+0000","last_change":"2024-03-30T00:50:37.638432+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638432+0000","last_undegraded":"2024-03-30T00:50:37.638432+0000","last_fullsized":"2024-03-30T00:50:37.638432+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637122+0000","last_change":"2024-03-30T00:50:37.637122+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637122+0000","last_undegraded":"2024-03-30T00:50:37.637122+0000","last_fullsized":"2024-03-30T00:50:37.637122+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:46.573525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643847+0000","last_change":"2024-03-30T00:50:37.643847+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643847+0000","last_undegraded":"2024-03-30T00:50:37.643847+0000","last_fullsized":"2024-03-30T00:50:37.643847+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645745+0000","last_change":"2024-03-30T00:50:37.645745+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645745+0000","last_undegraded":"2024-03-30T00:50:37.645745+0000","last_fullsized":"2024-03-30T00:50:37.645745+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:04:48.536583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646345+0000","last_change":"2024-03-30T00:50:37.646345+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646345+0000","last_undegraded":"2024-03-30T00:50:37.646345+0000","last_fullsized":"2024-03-30T00:50:37.646345+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:38.180300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640110+0000","last_change":"2024-03-30T00:50:37.640110+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640110+0000","last_undegraded":"2024-03-30T00:50:37.640110+0000","last_fullsized":"2024-03-30T00:50:37.640110+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:38:56.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638089+0000","last_change":"2024-03-30T00:50:37.638089+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638089+0000","last_undegraded":"2024-03-30T00:50:37.638089+0000","last_fullsized":"2024-03-30T00:50:37.638089+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.639932+0000","last_change":"2024-03-30T00:50:37.639932+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.639932+0000","last_undegraded":"2024-03-30T00:50:37.639932+0000","last_fullsized":"2024-03-30T00:50:37.639932+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636945+0000","last_change":"2024-03-30T00:50:37.636945+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636945+0000","last_undegraded":"2024-03-30T00:50:37.636945+0000","last_fullsized":"2024-03-30T00:50:37.636945+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641895+0000","last_change":"2024-03-30T00:50:37.641895+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641895+0000","last_undegraded":"2024-03-30T00:50:37.641895+0000","last_fullsized":"2024-03-30T00:50:37.641895+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645034+0000","last_change":"2024-03-30T00:50:37.645034+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645034+0000","last_undegraded":"2024-03-30T00:50:37.645034+0000","last_fullsized":"2024-03-30T00:50:37.645034+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642363+0000","last_change":"2024-03-30T00:50:37.642363+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642363+0000","last_undegraded":"2024-03-30T00:50:37.642363+0000","last_fullsized":"2024-03-30T00:50:37.642363+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:56.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645743+0000","last_change":"2024-03-30T00:50:37.645743+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645743+0000","last_undegraded":"2024-03-30T00:50:37.645743+0000","last_fullsized":"2024-03-30T00:50:37.645743+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:13.551635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638476+0000","last_change":"2024-03-30T00:50:37.638476+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638476+0000","last_undegraded":"2024-03-30T00:50:37.638476+0000","last_fullsized":"2024-03-30T00:50:37.638476+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:34:39.399906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640963+0000","last_change":"2024-03-30T00:50:37.640963+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640963+0000","last_undegraded":"2024-03-30T00:50:37.640963+0000","last_fullsized":"2024-03-30T00:50:37.640963+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:12.688382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.647144+0000","last_change":"2024-03-30T00:50:37.647144+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.647144+0000","last_undegraded":"2024-03-30T00:50:37.647144+0000","last_fullsized":"2024-03-30T00:50:37.647144+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645120+0000","last_change":"2024-03-30T00:50:37.645120+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645120+0000","last_undegraded":"2024-03-30T00:50:37.645120+0000","last_fullsized":"2024-03-30T00:50:37.645120+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:14.841469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641328+0000","last_change":"2024-03-30T00:50:37.641328+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641328+0000","last_undegraded":"2024-03-30T00:50:37.641328+0000","last_fullsized":"2024-03-30T00:50:37.641328+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641925+0000","last_change":"2024-03-30T00:50:37.641925+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641925+0000","last_undegraded":"2024-03-30T00:50:37.641925+0000","last_fullsized":"2024-03-30T00:50:37.641925+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659225+0000","last_change":"2024-03-30T00:50:37.658198+0000","last_active":"2024-03-30T00:50:37.659225+0000","last_peered":"2024-03-30T00:50:37.659225+0000","last_clean":"2024-03-30T00:50:37.659225+0000","last_became_active":"2024-03-30T00:50:37.657799+0000","last_became_peered":"2024-03-30T00:50:37.657799+0000","last_unstale":"2024-03-30T00:50:37.659225+0000","last_undegraded":"2024-03-30T00:50:37.659225+0000","last_fullsized":"2024-03-30T00:50:37.659225+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:55:09.373511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637154+0000","last_change":"2024-03-30T00:50:37.637154+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637154+0000","last_undegraded":"2024-03-30T00:50:37.637154+0000","last_fullsized":"2024-03-30T00:50:37.637154+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:58.975555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653380+0000","last_change":"2024-03-30T00:50:37.647732+0000","last_active":"2024-03-30T00:50:37.653380+0000","last_peered":"2024-03-30T00:50:37.653380+0000","last_clean":"2024-03-30T00:50:37.653380+0000","last_became_active":"2024-03-30T00:50:37.647438+0000","last_became_peered":"2024-03-30T00:50:37.647438+0000","last_unstale":"2024-03-30T00:50:37.653380+0000","last_undegraded":"2024-03-30T00:50:37.653380+0000","last_fullsized":"2024-03-30T00:50:37.653380+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:04.443031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642651+0000","last_change":"2024-03-30T00:50:37.642651+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642651+0000","last_undegraded":"2024-03-30T00:50:37.642651+0000","last_fullsized":"2024-03-30T00:50:37.642651+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656737+0000","last_change":"2024-03-30T00:50:37.651416+0000","last_active":"2024-03-30T00:50:37.656737+0000","last_peered":"2024-03-30T00:50:37.656737+0000","last_clean":"2024-03-30T00:50:37.656737+0000","last_became_active":"2024-03-30T00:50:37.651110+0000","last_became_peered":"2024-03-30T00:50:37.651110+0000","last_unstale":"2024-03-30T00:50:37.656737+0000","last_undegraded":"2024-03-30T00:50:37.656737+0000","last_fullsized":"2024-03-30T00:50:37.656737+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:04.443031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637468+0000","last_change":"2024-03-30T00:50:37.637468+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637468+0000","last_undegraded":"2024-03-30T00:50:37.637468+0000","last_fullsized":"2024-03-30T00:50:37.637468+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659094+0000","last_change":"2024-03-30T00:50:37.652287+0000","last_active":"2024-03-30T00:50:37.659094+0000","last_peered":"2024-03-30T00:50:37.659094+0000","last_clean":"2024-03-30T00:50:37.659094+0000","last_became_active":"2024-03-30T00:50:37.652025+0000","last_became_peered":"2024-03-30T00:50:37.652025+0000","last_unstale":"2024-03-30T00:50:37.659094+0000","last_undegraded":"2024-03-30T00:50:37.659094+0000","last_fullsized":"2024-03-30T00:50:37.659094+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:37.177730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640607+0000","last_change":"2024-03-30T00:50:37.640607+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640607+0000","last_undegraded":"2024-03-30T00:50:37.640607+0000","last_fullsized":"2024-03-30T00:50:37.640607+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.657930+0000","last_change":"2024-03-30T00:50:37.655896+0000","last_active":"2024-03-30T00:50:37.657930+0000","last_peered":"2024-03-30T00:50:37.657930+0000","last_clean":"2024-03-30T00:50:37.657930+0000","last_became_active":"2024-03-30T00:50:37.655623+0000","last_became_peered":"2024-03-30T00:50:37.655623+0000","last_unstale":"2024-03-30T00:50:37.657930+0000","last_undegraded":"2024-03-30T00:50:37.657930+0000","last_fullsized":"2024-03-30T00:50:37.657930+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:29.599324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637936+0000","last_change":"2024-03-30T00:50:37.637936+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637936+0000","last_undegraded":"2024-03-30T00:50:37.637936+0000","last_fullsized":"2024-03-30T00:50:37.637936+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659029+0000","last_change":"2024-03-30T00:50:37.657178+0000","last_active":"2024-03-30T00:50:37.659029+0000","last_peered":"2024-03-30T00:50:37.659029+0000","last_clean":"2024-03-30T00:50:37.659029+0000","last_became_active":"2024-03-30T00:50:37.656814+0000","last_became_peered":"2024-03-30T00:50:37.656814+0000","last_unstale":"2024-03-30T00:50:37.659029+0000","last_undegraded":"2024-03-30T00:50:37.659029+0000","last_fullsized":"2024-03-30T00:50:37.659029+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:11.406073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646745+0000","last_change":"2024-03-30T00:50:37.646745+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646745+0000","last_undegraded":"2024-03-30T00:50:37.646745+0000","last_fullsized":"2024-03-30T00:50:37.646745+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:51:44.264107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.657059+0000","last_change":"2024-03-30T00:50:37.654789+0000","last_active":"2024-03-30T00:50:37.657059+0000","last_peered":"2024-03-30T00:50:37.657059+0000","last_clean":"2024-03-30T00:50:37.657059+0000","last_became_active":"2024-03-30T00:50:37.654488+0000","last_became_peered":"2024-03-30T00:50:37.654488+0000","last_unstale":"2024-03-30T00:50:37.657059+0000","last_undegraded":"2024-03-30T00:50:37.657059+0000","last_fullsized":"2024-03-30T00:50:37.657059+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:41:51.590793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641119+0000","last_change":"2024-03-30T00:50:37.641119+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641119+0000","last_undegraded":"2024-03-30T00:50:37.641119+0000","last_fullsized":"2024-03-30T00:50:37.641119+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659480+0000","last_change":"2024-03-30T00:50:37.658253+0000","last_active":"2024-03-30T00:50:37.659480+0000","last_peered":"2024-03-30T00:50:37.659480+0000","last_clean":"2024-03-30T00:50:37.659480+0000","last_became_active":"2024-03-30T00:50:37.657996+0000","last_became_peered":"2024-03-30T00:50:37.657996+0000","last_unstale":"2024-03-30T00:50:37.659480+0000","last_undegraded":"2024-03-30T00:50:37.659480+0000","last_fullsized":"2024-03-30T00:50:37.659480+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:51:25.787216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.647075+0000","last_change":"2024-03-30T00:50:37.647075+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.647075+0000","last_undegraded":"2024-03-30T00:50:37.647075+0000","last_fullsized":"2024-03-30T00:50:37.647075+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:01.855290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.660222+0000","last_change":"2024-03-30T00:50:37.659011+0000","last_active":"2024-03-30T00:50:37.660222+0000","last_peered":"2024-03-30T00:50:37.660222+0000","last_clean":"2024-03-30T00:50:37.660222+0000","last_became_active":"2024-03-30T00:50:37.658720+0000","last_became_peered":"2024-03-30T00:50:37.658720+0000","last_unstale":"2024-03-30T00:50:37.660222+0000","last_undegraded":"2024-03-30T00:50:37.660222+0000","last_fullsized":"2024-03-30T00:50:37.660222+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:57.692452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642802+0000","last_change":"2024-03-30T00:50:37.642802+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642802+0000","last_undegraded":"2024-03-30T00:50:37.642802+0000","last_fullsized":"2024-03-30T00:50:37.642802+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653217+0000","last_change":"2024-03-30T00:50:37.651177+0000","last_active":"2024-03-30T00:50:37.653217+0000","last_peered":"2024-03-30T00:50:37.653217+0000","last_clean":"2024-03-30T00:50:37.653217+0000","last_became_active":"2024-03-30T00:50:37.650902+0000","last_became_peered":"2024-03-30T00:50:37.650902+0000","last_unstale":"2024-03-30T00:50:37.653217+0000","last_undegraded":"2024-03-30T00:50:37.653217+0000","last_fullsized":"2024-03-30T00:50:37.653217+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:28.311412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642586+0000","last_change":"2024-03-30T00:50:37.642586+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642586+0000","last_undegraded":"2024-03-30T00:50:37.642586+0000","last_fullsized":"2024-03-30T00:50:37.642586+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656565+0000","last_change":"2024-03-30T00:50:37.653495+0000","last_active":"2024-03-30T00:50:37.656565+0000","last_peered":"2024-03-30T00:50:37.656565+0000","last_clean":"2024-03-30T00:50:37.656565+0000","last_became_active":"2024-03-30T00:50:37.652706+0000","last_became_peered":"2024-03-30T00:50:37.652706+0000","last_unstale":"2024-03-30T00:50:37.656565+0000","last_undegraded":"2024-03-30T00:50:37.656565+0000","last_fullsized":"2024-03-30T00:50:37.656565+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:07:01.780302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.644570+0000","last_change":"2024-03-30T00:50:37.644570+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.644570+0000","last_undegraded":"2024-03-30T00:50:37.644570+0000","last_fullsized":"2024-03-30T00:50:37.644570+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:46:27.974997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653040+0000","last_change":"2024-03-30T00:50:37.648270+0000","last_active":"2024-03-30T00:50:37.653040+0000","last_peered":"2024-03-30T00:50:37.653040+0000","last_clean":"2024-03-30T00:50:37.653040+0000","last_became_active":"2024-03-30T00:50:37.647989+0000","last_became_peered":"2024-03-30T00:50:37.647989+0000","last_unstale":"2024-03-30T00:50:37.653040+0000","last_undegraded":"2024-03-30T00:50:37.653040+0000","last_fullsized":"2024-03-30T00:50:37.653040+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:53.076893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.646353+0000","last_change":"2024-03-30T00:50:37.646353+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.646353+0000","last_undegraded":"2024-03-30T00:50:37.646353+0000","last_fullsized":"2024-03-30T00:50:37.646353+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:05.445602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659297+0000","last_change":"2024-03-30T00:50:37.655775+0000","last_active":"2024-03-30T00:50:37.659297+0000","last_peered":"2024-03-30T00:50:37.659297+0000","last_clean":"2024-03-30T00:50:37.659297+0000","last_became_active":"2024-03-30T00:50:37.655033+0000","last_became_peered":"2024-03-30T00:50:37.655033+0000","last_unstale":"2024-03-30T00:50:37.659297+0000","last_undegraded":"2024-03-30T00:50:37.659297+0000","last_fullsized":"2024-03-30T00:50:37.659297+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:46:26.972427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638573+0000","last_change":"2024-03-30T00:50:37.638573+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638573+0000","last_undegraded":"2024-03-30T00:50:37.638573+0000","last_fullsized":"2024-03-30T00:50:37.638573+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.660319+0000","last_change":"2024-03-30T00:50:37.657373+0000","last_active":"2024-03-30T00:50:37.660319+0000","last_peered":"2024-03-30T00:50:37.660319+0000","last_clean":"2024-03-30T00:50:37.660319+0000","last_became_active":"2024-03-30T00:50:37.657068+0000","last_became_peered":"2024-03-30T00:50:37.657068+0000","last_unstale":"2024-03-30T00:50:37.660319+0000","last_undegraded":"2024-03-30T00:50:37.660319+0000","last_fullsized":"2024-03-30T00:50:37.660319+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:36.963155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.640244+0000","last_change":"2024-03-30T00:50:37.640244+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.640244+0000","last_undegraded":"2024-03-30T00:50:37.640244+0000","last_fullsized":"2024-03-30T00:50:37.640244+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:02:07.606929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.655268+0000","last_change":"2024-03-30T00:50:37.653596+0000","last_active":"2024-03-30T00:50:37.655268+0000","last_peered":"2024-03-30T00:50:37.655268+0000","last_clean":"2024-03-30T00:50:37.655268+0000","last_became_active":"2024-03-30T00:50:37.653299+0000","last_became_peered":"2024-03-30T00:50:37.653299+0000","last_unstale":"2024-03-30T00:50:37.655268+0000","last_undegraded":"2024-03-30T00:50:37.655268+0000","last_fullsized":"2024-03-30T00:50:37.655268+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:06.142142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641714+0000","last_change":"2024-03-30T00:50:37.641714+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641714+0000","last_undegraded":"2024-03-30T00:50:37.641714+0000","last_fullsized":"2024-03-30T00:50:37.641714+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.654489+0000","last_change":"2024-03-30T00:50:37.649310+0000","last_active":"2024-03-30T00:50:37.654489+0000","last_peered":"2024-03-30T00:50:37.654489+0000","last_clean":"2024-03-30T00:50:37.654489+0000","last_became_active":"2024-03-30T00:50:37.649015+0000","last_became_peered":"2024-03-30T00:50:37.649015+0000","last_unstale":"2024-03-30T00:50:37.654489+0000","last_undegraded":"2024-03-30T00:50:37.654489+0000","last_fullsized":"2024-03-30T00:50:37.654489+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:00.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637343+0000","last_change":"2024-03-30T00:50:37.637343+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637343+0000","last_undegraded":"2024-03-30T00:50:37.637343+0000","last_fullsized":"2024-03-30T00:50:37.637343+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658749+0000","last_change":"2024-03-30T00:50:37.655900+0000","last_active":"2024-03-30T00:50:37.658749+0000","last_peered":"2024-03-30T00:50:37.658749+0000","last_clean":"2024-03-30T00:50:37.658749+0000","last_became_active":"2024-03-30T00:50:37.655601+0000","last_became_peered":"2024-03-30T00:50:37.655601+0000","last_unstale":"2024-03-30T00:50:37.658749+0000","last_undegraded":"2024-03-30T00:50:37.658749+0000","last_fullsized":"2024-03-30T00:50:37.658749+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:33.394770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641239+0000","last_change":"2024-03-30T00:50:37.641239+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641239+0000","last_undegraded":"2024-03-30T00:50:37.641239+0000","last_fullsized":"2024-03-30T00:50:37.641239+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:09:05.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656296+0000","last_change":"2024-03-30T00:50:37.649475+0000","last_active":"2024-03-30T00:50:37.656296+0000","last_peered":"2024-03-30T00:50:37.656296+0000","last_clean":"2024-03-30T00:50:37.656296+0000","last_became_active":"2024-03-30T00:50:37.649055+0000","last_became_peered":"2024-03-30T00:50:37.649055+0000","last_unstale":"2024-03-30T00:50:37.656296+0000","last_undegraded":"2024-03-30T00:50:37.656296+0000","last_fullsized":"2024-03-30T00:50:37.656296+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:12.549064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643208+0000","last_change":"2024-03-30T00:50:37.643208+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643208+0000","last_undegraded":"2024-03-30T00:50:37.643208+0000","last_fullsized":"2024-03-30T00:50:37.643208+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:47.211883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658519+0000","last_change":"2024-03-30T00:50:37.656047+0000","last_active":"2024-03-30T00:50:37.658519+0000","last_peered":"2024-03-30T00:50:37.658519+0000","last_clean":"2024-03-30T00:50:37.658519+0000","last_became_active":"2024-03-30T00:50:37.655751+0000","last_became_peered":"2024-03-30T00:50:37.655751+0000","last_unstale":"2024-03-30T00:50:37.658519+0000","last_undegraded":"2024-03-30T00:50:37.658519+0000","last_fullsized":"2024-03-30T00:50:37.658519+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:00.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636644+0000","last_change":"2024-03-30T00:50:37.636644+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636644+0000","last_undegraded":"2024-03-30T00:50:37.636644+0000","last_fullsized":"2024-03-30T00:50:37.636644+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:52:46.481754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.651675+0000","last_change":"2024-03-30T00:50:37.646336+0000","last_active":"2024-03-30T00:50:37.651675+0000","last_peered":"2024-03-30T00:50:37.651675+0000","last_clean":"2024-03-30T00:50:37.651675+0000","last_became_active":"2024-03-30T00:50:37.646081+0000","last_became_peered":"2024-03-30T00:50:37.646081+0000","last_unstale":"2024-03-30T00:50:37.651675+0000","last_undegraded":"2024-03-30T00:50:37.651675+0000","last_fullsized":"2024-03-30T00:50:37.651675+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:40.726869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.645084+0000","last_change":"2024-03-30T00:50:37.645084+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.645084+0000","last_undegraded":"2024-03-30T00:50:37.645084+0000","last_fullsized":"2024-03-30T00:50:37.645084+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:04:19.985679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.652956+0000","last_change":"2024-03-30T00:50:37.648309+0000","last_active":"2024-03-30T00:50:37.652956+0000","last_peered":"2024-03-30T00:50:37.652956+0000","last_clean":"2024-03-30T00:50:37.652956+0000","last_became_active":"2024-03-30T00:50:37.646903+0000","last_became_peered":"2024-03-30T00:50:37.646903+0000","last_unstale":"2024-03-30T00:50:37.652956+0000","last_undegraded":"2024-03-30T00:50:37.652956+0000","last_fullsized":"2024-03-30T00:50:37.652956+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:20.839135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642635+0000","last_change":"2024-03-30T00:50:37.642635+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642635+0000","last_undegraded":"2024-03-30T00:50:37.642635+0000","last_fullsized":"2024-03-30T00:50:37.642635+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659418+0000","last_change":"2024-03-30T00:50:37.656069+0000","last_active":"2024-03-30T00:50:37.659418+0000","last_peered":"2024-03-30T00:50:37.659418+0000","last_clean":"2024-03-30T00:50:37.659418+0000","last_became_active":"2024-03-30T00:50:37.655828+0000","last_became_peered":"2024-03-30T00:50:37.655828+0000","last_unstale":"2024-03-30T00:50:37.659418+0000","last_undegraded":"2024-03-30T00:50:37.659418+0000","last_fullsized":"2024-03-30T00:50:37.659418+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:46.209312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.643298+0000","last_change":"2024-03-30T00:50:37.643298+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.643298+0000","last_undegraded":"2024-03-30T00:50:37.643298+0000","last_fullsized":"2024-03-30T00:50:37.643298+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:37.965725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656366+0000","last_change":"2024-03-30T00:50:37.650090+0000","last_active":"2024-03-30T00:50:37.656366+0000","last_peered":"2024-03-30T00:50:37.656366+0000","last_clean":"2024-03-30T00:50:37.656366+0000","last_became_active":"2024-03-30T00:50:37.649527+0000","last_became_peered":"2024-03-30T00:50:37.649527+0000","last_unstale":"2024-03-30T00:50:37.656366+0000","last_undegraded":"2024-03-30T00:50:37.656366+0000","last_fullsized":"2024-03-30T00:50:37.656366+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:37.177730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638050+0000","last_change":"2024-03-30T00:50:37.638050+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638050+0000","last_undegraded":"2024-03-30T00:50:37.638050+0000","last_fullsized":"2024-03-30T00:50:37.638050+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:19.197836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.654023+0000","last_change":"2024-03-30T00:50:37.648973+0000","last_active":"2024-03-30T00:50:37.654023+0000","last_peered":"2024-03-30T00:50:37.654023+0000","last_clean":"2024-03-30T00:50:37.654023+0000","last_became_active":"2024-03-30T00:50:37.648703+0000","last_became_peered":"2024-03-30T00:50:37.648703+0000","last_unstale":"2024-03-30T00:50:37.654023+0000","last_undegraded":"2024-03-30T00:50:37.654023+0000","last_fullsized":"2024-03-30T00:50:37.654023+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:53:36.133173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637305+0000","last_change":"2024-03-30T00:50:37.637305+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637305+0000","last_undegraded":"2024-03-30T00:50:37.637305+0000","last_fullsized":"2024-03-30T00:50:37.637305+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.659783+0000","last_change":"2024-03-30T00:50:37.658787+0000","last_active":"2024-03-30T00:50:37.659783+0000","last_peered":"2024-03-30T00:50:37.659783+0000","last_clean":"2024-03-30T00:50:37.659783+0000","last_became_active":"2024-03-30T00:50:37.658512+0000","last_became_peered":"2024-03-30T00:50:37.658512+0000","last_unstale":"2024-03-30T00:50:37.659783+0000","last_undegraded":"2024-03-30T00:50:37.659783+0000","last_fullsized":"2024-03-30T00:50:37.659783+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:55:31.799226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637015+0000","last_change":"2024-03-30T00:50:37.637015+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637015+0000","last_undegraded":"2024-03-30T00:50:37.637015+0000","last_fullsized":"2024-03-30T00:50:37.637015+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658420+0000","last_change":"2024-03-30T00:50:37.656511+0000","last_active":"2024-03-30T00:50:37.658420+0000","last_peered":"2024-03-30T00:50:37.658420+0000","last_clean":"2024-03-30T00:50:37.658420+0000","last_became_active":"2024-03-30T00:50:37.655106+0000","last_became_peered":"2024-03-30T00:50:37.655106+0000","last_unstale":"2024-03-30T00:50:37.658420+0000","last_undegraded":"2024-03-30T00:50:37.658420+0000","last_fullsized":"2024-03-30T00:50:37.658420+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:04:28.343406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.641788+0000","last_change":"2024-03-30T00:50:37.641788+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.641788+0000","last_undegraded":"2024-03-30T00:50:37.641788+0000","last_fullsized":"2024-03-30T00:50:37.641788+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:34:31.524194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.657263+0000","last_change":"2024-03-30T00:50:37.655230+0000","last_active":"2024-03-30T00:50:37.657263+0000","last_peered":"2024-03-30T00:50:37.657263+0000","last_clean":"2024-03-30T00:50:37.657263+0000","last_became_active":"2024-03-30T00:50:37.654933+0000","last_became_peered":"2024-03-30T00:50:37.654933+0000","last_unstale":"2024-03-30T00:50:37.657263+0000","last_undegraded":"2024-03-30T00:50:37.657263+0000","last_fullsized":"2024-03-30T00:50:37.657263+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:59.607374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.635996+0000","last_change":"2024-03-30T00:50:37.635996+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.635996+0000","last_undegraded":"2024-03-30T00:50:37.635996+0000","last_fullsized":"2024-03-30T00:50:37.635996+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:52:14.331871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.650896+0000","last_change":"2024-03-30T00:50:37.647611+0000","last_active":"2024-03-30T00:50:37.650896+0000","last_peered":"2024-03-30T00:50:37.650896+0000","last_clean":"2024-03-30T00:50:37.650896+0000","last_became_active":"2024-03-30T00:50:37.647311+0000","last_became_peered":"2024-03-30T00:50:37.647311+0000","last_unstale":"2024-03-30T00:50:37.650896+0000","last_undegraded":"2024-03-30T00:50:37.650896+0000","last_fullsized":"2024-03-30T00:50:37.650896+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:21.467231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.642870+0000","last_change":"2024-03-30T00:50:37.642870+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.642870+0000","last_undegraded":"2024-03-30T00:50:37.642870+0000","last_fullsized":"2024-03-30T00:50:37.642870+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656761+0000","last_change":"2024-03-30T00:50:37.651832+0000","last_active":"2024-03-30T00:50:37.656761+0000","last_peered":"2024-03-30T00:50:37.656761+0000","last_clean":"2024-03-30T00:50:37.656761+0000","last_became_active":"2024-03-30T00:50:37.651529+0000","last_became_peered":"2024-03-30T00:50:37.651529+0000","last_unstale":"2024-03-30T00:50:37.656761+0000","last_undegraded":"2024-03-30T00:50:37.656761+0000","last_fullsized":"2024-03-30T00:50:37.656761+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:19.643626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.636935+0000","last_change":"2024-03-30T00:50:37.636935+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.636935+0000","last_undegraded":"2024-03-30T00:50:37.636935+0000","last_fullsized":"2024-03-30T00:50:37.636935+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.656496+0000","last_change":"2024-03-30T00:50:37.654139+0000","last_active":"2024-03-30T00:50:37.656496+0000","last_peered":"2024-03-30T00:50:37.656496+0000","last_clean":"2024-03-30T00:50:37.656496+0000","last_became_active":"2024-03-30T00:50:37.653825+0000","last_became_peered":"2024-03-30T00:50:37.653825+0000","last_unstale":"2024-03-30T00:50:37.656496+0000","last_undegraded":"2024-03-30T00:50:37.656496+0000","last_fullsized":"2024-03-30T00:50:37.656496+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:47.043924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.638023+0000","last_change":"2024-03-30T00:50:37.638023+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.638023+0000","last_undegraded":"2024-03-30T00:50:37.638023+0000","last_fullsized":"2024-03-30T00:50:37.638023+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.658174+0000","last_change":"2024-03-30T00:50:37.654797+0000","last_active":"2024-03-30T00:50:37.658174+0000","last_peered":"2024-03-30T00:50:37.658174+0000","last_clean":"2024-03-30T00:50:37.658174+0000","last_became_active":"2024-03-30T00:50:37.654506+0000","last_became_peered":"2024-03-30T00:50:37.654506+0000","last_unstale":"2024-03-30T00:50:37.658174+0000","last_undegraded":"2024-03-30T00:50:37.658174+0000","last_fullsized":"2024-03-30T00:50:37.658174+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:26.653343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":123,"state":"creating+peering","last_fresh":"2024-03-30T00:50:37.637355+0000","last_change":"2024-03-30T00:50:37.637355+0000","last_active":"2024-03-30T00:50:37.625832+0000","last_peered":"2024-03-30T00:50:37.625832+0000","last_clean":"2024-03-30T00:50:37.625832+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T00:50:37.637355+0000","last_undegraded":"2024-03-30T00:50:37.637355+0000","last_fullsized":"2024-03-30T00:50:37.637355+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-30T00:50:37.653101+0000","last_change":"2024-03-30T00:50:37.648683+0000","last_active":"2024-03-30T00:50:37.653101+0000","last_peered":"2024-03-30T00:50:37.653101+0000","last_clean":"2024-03-30T00:50:37.653101+0000","last_became_active":"2024-03-30T00:50:37.648391+0000","last_became_peered":"2024-03-30T00:50:37.648391+0000","last_unstale":"2024-03-30T00:50:37.653101+0000","last_undegraded":"2024-03-30T00:50:37.653101+0000","last_fullsized":"2024-03-30T00:50:37.653101+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:13.838899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":271,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-03-30T00:50:36.635729+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:50:36.635729+0000","last_peered":"2024-03-30T00:50:36.635729+0000","last_clean":"2024-03-30T00:50:36.635729+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:50:36.635729+0000","last_undegraded":"2024-03-30T00:50:36.635729+0000","last_fullsized":"2024-03-30T00:50:36.635729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574982,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2113536,"data_stored":1290047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":0.97599999999999998,"15min":1.0860000000000001},"last":0.621},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.47399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84999999999999998,"5min":0.98299999999999998,"15min":1.149},"last":0.55500000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.503,"5min":0.45800000000000002,"15min":0.436},"max":{"1min":0.90800000000000003,"5min":1.03,"15min":1.03},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93000000000000005,"5min":0.95199999999999996,"15min":1.3029999999999999},"last":0.70399999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.76600000000000001},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.45000000000000001},"max":{"1min":1.0129999999999999,"5min":1.085,"15min":1.6020000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.73099999999999998,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95399999999999996,"5min":1.034,"15min":1.6619999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.75900000000000001},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99099999999999999,"5min":0.99299999999999999,"15min":2.0150000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.80000000000000004},"min":{"1min":0.63800000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.9490000000000001},"last":0.76400000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.37},"max":{"1min":0.97299999999999998,"5min":1.2,"15min":1.2},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.42999999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.97899999999999998},"last":0.52700000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96299999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":0.99199999999999999,"15min":1.026},"last":0.60699999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.92700000000000005,"5min":0.94599999999999995,"15min":1.214},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89200000000000002,"5min":1.0189999999999999,"15min":1.0309999999999999},"last":0.64900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1531904,"data_stored":706274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71599999999999997,"15min":0.79800000000000004},"min":{"1min":0.437,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.85799999999999998,"5min":1.2969999999999999,"15min":4.0490000000000004},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.68799999999999994},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":1.278,"15min":1.47},"last":0.96199999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.438,"15min":0.36899999999999999},"max":{"1min":0.95799999999999996,"5min":1.204,"15min":4.3170000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.85299999999999998},"min":{"1min":0.58399999999999996,"5min":0.47199999999999998,"15min":0.42999999999999999},"max":{"1min":0.92900000000000005,"5min":1.1120000000000001,"15min":4.4160000000000004},"last":0.91700000000000004}]},{"osd":2,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.745,"15min":0.84999999999999998},"min":{"1min":0.63400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98299999999999998,"5min":1.24,"15min":4.375},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.84299999999999997},"min":{"1min":0.51700000000000002,"5min":0.42999999999999999,"15min":0.38600000000000001},"max":{"1min":0.92500000000000004,"5min":1.3129999999999999,"15min":4.2960000000000003},"last":0.89900000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.86799999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0249999999999999,"5min":1.2390000000000001,"15min":4.3399999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73199999999999998,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.49099999999999999,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.258,"15min":4.6020000000000003},"last":0.85399999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.39200000000000002,"5min":0.39100000000000001,"15min":0.35999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.1870000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.73199999999999998},"min":{"1min":0.441,"5min":0.372,"15min":0.372},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.036},"last":0.82599999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.65700000000000003},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0720000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0109999999999999,"15min":1.101},"last":0.67900000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97499999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.45700000000000002},"max":{"1min":1.0860000000000001,"5min":1.103,"15min":1.103},"last":0.70499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574982,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":0.83299999999999996,"5min":0.97599999999999998,"15min":1.095},"last":0.63},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68700000000000006,"15min":0.73199999999999998},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":1.0489999999999999,"15min":1.7569999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.71999999999999997},"min":{"1min":0.47899999999999998,"5min":0.46600000000000003,"15min":0.43099999999999999},"max":{"1min":0.86599999999999999,"5min":0.98199999999999998,"15min":1.7330000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70999999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42399999999999999},"max":{"1min":0.92400000000000004,"5min":0.98799999999999999,"15min":1.3260000000000001},"last":0.73299999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.44600000000000001},"max":{"1min":0.89900000000000002,"5min":1.131,"15min":1.131},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.45900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.877,"5min":1.0609999999999999,"15min":1.2789999999999999},"last":0.51400000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.75},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.89100000000000001,"5min":1.0740000000000001,"15min":1.7},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.96099999999999997,"5min":1.1160000000000001,"15min":1.181},"last":0.56399999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.60299999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.309},"max":{"1min":0.876,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.433},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69199999999999995,"15min":0.73399999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91200000000000003,"5min":0.97799999999999998,"15min":1.615},"last":0.67000000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.84099999999999997,"5min":0.91100000000000003,"15min":0.97399999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.626,"15min":0.67300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.35199999999999998},"max":{"1min":0.84999999999999998,"5min":0.93700000000000006,"15min":1.569},"last":0.59199999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.71499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.377},"max":{"1min":0.85399999999999998,"5min":0.92200000000000004,"15min":1.0149999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88400000000000001,"5min":1.167,"15min":1.383},"last":0.64800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.34899999999999998},"max":{"1min":0.92400000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.67400000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98599999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.502}]},{"osd":1,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":0.92600000000000005,"5min":0.95899999999999996,"15min":1.2909999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.40699999999999997},"max":{"1min":1.006,"5min":1.113,"15min":1.113},"last":0.60799999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.59199999999999997,"5min":0.48999999999999999,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.73899999999999999,"15min":0.75},"min":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.46700000000000003},"max":{"1min":0.94899999999999995,"5min":1.163,"15min":1.228},"last":0.68300000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73599999999999999,"15min":0.71699999999999997},"min":{"1min":0.501,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.27},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.76000000000000001},"min":{"1min":0.59099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1,"15min":1.1739999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.621,"15min":0.65800000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.89400000000000002,"15min":1.329},"last":0.754},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66100000000000003,"15min":0.70399999999999996},"min":{"1min":0.57399999999999995,"5min":0.34799999999999998,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":0.99099999999999999,"15min":1.3819999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.65700000000000003},"min":{"1min":0.45700000000000002,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":1.2130000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.69699999999999995},"min":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.95799999999999996,"15min":1.3220000000000001},"last":0.85599999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.628,"15min":0.69099999999999995},"min":{"1min":0.46700000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":0.91500000000000004,"15min":1.6910000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.73399999999999999,"15min":0.78000000000000003},"min":{"1min":0.59699999999999998,"5min":0.49299999999999999,"15min":0.46600000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.5009999999999999},"last":0.873}]}]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.435,"15min":0.45500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.58699999999999997,"5min":0.748,"15min":1.0529999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.49099999999999999,"15min":0.51400000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.24199999999999999},"max":{"1min":0.63400000000000001,"5min":0.75,"15min":0.79100000000000004},"last":0.39500000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49399999999999999,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.80900000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63800000000000001,"5min":0.73399999999999999,"15min":1.2150000000000001},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.78300000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.46300000000000002},"max":{"1min":0.95699999999999996,"5min":1.1259999999999999,"15min":1.2470000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77100000000000002,"15min":0.77600000000000002},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.079,"15min":1.0860000000000001},"last":0.97099999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77100000000000002,"15min":0.77900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.51100000000000001},"max":{"1min":1.024,"5min":1.0369999999999999,"15min":1.125},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.77900000000000003,"15min":0.79100000000000004},"min":{"1min":0.628,"5min":0.52200000000000002,"15min":0.48599999999999999},"max":{"1min":0.86299999999999999,"5min":1.133,"15min":1.2390000000000001},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77200000000000002,"15min":0.79700000000000004},"min":{"1min":0.59999999999999998,"5min":0.51400000000000001,"15min":0.499},"max":{"1min":0.90100000000000002,"5min":1.091,"15min":1.226},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.038,"5min":1.1120000000000001,"15min":1.167},"last":0.94899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82999999999999996,"15min":0.84099999999999997},"min":{"1min":0.71399999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.123,"5min":1.1910000000000001,"15min":1.202},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83499999999999996,"15min":0.86299999999999999},"min":{"1min":0.57299999999999995,"5min":0.54900000000000004,"15min":0.497},"max":{"1min":1.095,"5min":1.1639999999999999,"15min":1.3160000000000001},"last":1.0189999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56899999999999995,"5min":0.70299999999999996,"15min":4.0999999999999996},"last":1.339},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.48799999999999999,"15min":0.50600000000000001},"min":{"1min":0.40699999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83999999999999997,"5min":0.84099999999999997,"15min":1.9239999999999999},"last":1.0900000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32600000000000001},"max":{"1min":0.61799999999999999,"5min":0.81299999999999994,"15min":3.1139999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.52400000000000002,"15min":0.53400000000000003},"min":{"1min":0.40600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.748,"5min":0.752,"15min":2.4159999999999999},"last":1.1830000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.499,"15min":0.497},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70099999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.749,"5min":0.88800000000000001,"15min":1.907},"last":0.95599999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77700000000000002,"15min":0.80100000000000005},"min":{"1min":0.59699999999999998,"5min":0.504,"15min":0.504},"max":{"1min":1.0900000000000001,"5min":1.1919999999999999,"15min":1.9470000000000001},"last":1.129},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78700000000000003,"15min":0.82099999999999995},"min":{"1min":0.628,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.163,"5min":1.2110000000000001,"15min":1.972},"last":1.2649999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.1220000000000001,"5min":1.2549999999999999,"15min":2.4569999999999999},"last":1.3},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.78900000000000003,"15min":0.81000000000000005},"min":{"1min":0.58399999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0389999999999999,"5min":1.286,"15min":2.8039999999999998},"last":0.70299999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0980000000000001,"5min":1.177,"15min":2.8849999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.0780000000000001,"5min":1.242,"15min":1.867},"last":0.996}]},{"osd":7,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79500000000000004,"15min":0.83699999999999997},"min":{"1min":0.60199999999999998,"5min":0.54700000000000004,"15min":0.495},"max":{"1min":1.0429999999999999,"5min":1.1140000000000001,"15min":2.3690000000000002},"last":1.2270000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.83799999999999997},"min":{"1min":0.70499999999999996,"5min":0.55800000000000005,"15min":0.52700000000000002},"max":{"1min":0.98999999999999999,"5min":1.365,"15min":2.851},"last":1.0089999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1519616,"data_stored":699679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44,"15min":0.42799999999999999},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.253},"max":{"1min":0.63600000000000001,"5min":0.72199999999999998,"15min":0.80300000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.48399999999999999,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.69999999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33300000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.45900000000000002},"min":{"1min":0.39200000000000002,"5min":0.29999999999999999,"15min":0.29199999999999998},"max":{"1min":0.59199999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.372},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.64600000000000002,"5min":0.84099999999999997,"15min":0.89400000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.45300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.69199999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.499},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.38600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72999999999999998,"5min":0.83199999999999996,"15min":0.83999999999999997},"last":0.40600000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.75600000000000001,"15min":0.72899999999999998},"min":{"1min":0.58099999999999996,"5min":0.49199999999999999,"15min":0.38800000000000001},"max":{"1min":1.034,"5min":1.175,"15min":1.175},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76900000000000002,"15min":0.754},"min":{"1min":0.66300000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98599999999999999,"5min":1.079,"15min":1.2},"last":0.81399999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.76000000000000001},"min":{"1min":0.621,"5min":0.56799999999999995,"15min":0.51600000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.3180000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79400000000000004,"15min":0.77100000000000002},"min":{"1min":0.60299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.129,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.87}]},{"osd":6,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79700000000000004,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.0389999999999999,"5min":1.179,"15min":1.179},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0509999999999999,"15min":1.2150000000000001},"last":0.89300000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80300000000000005,"15min":0.79700000000000004},"min":{"1min":0.64500000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.024,"5min":1.2050000000000001,"15min":1.2669999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.81799999999999995,"15min":0.79200000000000004},"min":{"1min":0.69299999999999995,"5min":0.56200000000000006,"15min":0.48499999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.236},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2113536,"data_stored":1290047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.45700000000000002},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.245},"max":{"1min":0.69799999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.505},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68400000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.51000000000000001},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.67100000000000004,"5min":0.82599999999999996,"15min":1.02},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.432,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":0.77900000000000003,"15min":1.006},"last":0.54400000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.68000000000000005,"5min":0.754,"15min":0.84799999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.46200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":0.88900000000000001,"15min":0.96899999999999997},"last":0.60999999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.47899999999999998},"max":{"1min":1.1040000000000001,"5min":1.1140000000000001,"15min":1.5169999999999999},"last":0.878},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.46400000000000002},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.252},"last":0.81000000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.55200000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.3080000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0780000000000001,"15min":1.462},"last":0.746}]},{"osd":6,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79200000000000004,"15min":0.79900000000000004},"min":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.54500000000000004},"max":{"1min":1.0429999999999999,"5min":1.131,"15min":1.5920000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80000000000000004,"15min":0.79000000000000004},"min":{"1min":0.625,"5min":0.54600000000000004,"15min":0.46999999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.2230000000000001},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.81299999999999994},"min":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.3600000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.60399999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.556},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:50:42.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3f is not active+clean 2024-03-30T00:50:42.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.639401+0000', 'last_change': '2024-03-30T00:50:37.639401+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.639401+0000', 'last_undegraded': '2024-03-30T00:50:37.639401+0000', 'last_fullsized': '2024-03-30T00:50:37.639401+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:50:42.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3e is not active+clean 2024-03-30T00:50:42.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.646197+0000', 'last_change': '2024-03-30T00:50:37.646197+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.646197+0000', 'last_undegraded': '2024-03-30T00:50:37.646197+0000', 'last_fullsized': '2024-03-30T00:50:37.646197+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:50:42.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3d is not active+clean 2024-03-30T00:50:42.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.643997+0000', 'last_change': '2024-03-30T00:50:37.643997+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.643997+0000', 'last_undegraded': '2024-03-30T00:50:37.643997+0000', 'last_fullsized': '2024-03-30T00:50:37.643997+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:53:58.506502+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:50:42.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3c is not active+clean 2024-03-30T00:50:42.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.637414+0000', 'last_change': '2024-03-30T00:50:37.637414+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.637414+0000', 'last_undegraded': '2024-03-30T00:50:37.637414+0000', 'last_fullsized': '2024-03-30T00:50:37.637414+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:50:42.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3b is not active+clean 2024-03-30T00:50:42.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.638917+0000', 'last_change': '2024-03-30T00:50:37.638917+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.638917+0000', 'last_undegraded': '2024-03-30T00:50:37.638917+0000', 'last_fullsized': '2024-03-30T00:50:37.638917+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:50:42.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3a is not active+clean 2024-03-30T00:50:42.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.637248+0000', 'last_change': '2024-03-30T00:50:37.637248+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.637248+0000', 'last_undegraded': '2024-03-30T00:50:37.637248+0000', 'last_fullsized': '2024-03-30T00:50:37.637248+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:18:16.922513+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:50:42.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.39 is not active+clean 2024-03-30T00:50:42.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.636753+0000', 'last_change': '2024-03-30T00:50:37.636753+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.636753+0000', 'last_undegraded': '2024-03-30T00:50:37.636753+0000', 'last_fullsized': '2024-03-30T00:50:37.636753+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T02:10:54.265531+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:50:42.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.38 is not active+clean 2024-03-30T00:50:42.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.642083+0000', 'last_change': '2024-03-30T00:50:37.642083+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.642083+0000', 'last_undegraded': '2024-03-30T00:50:37.642083+0000', 'last_fullsized': '2024-03-30T00:50:37.642083+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:50:42.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.37 is not active+clean 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.638063+0000', 'last_change': '2024-03-30T00:50:37.638063+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.638063+0000', 'last_undegraded': '2024-03-30T00:50:37.638063+0000', 'last_fullsized': '2024-03-30T00:50:37.638063+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.36 is not active+clean 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.639455+0000', 'last_change': '2024-03-30T00:50:37.639455+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.639455+0000', 'last_undegraded': '2024-03-30T00:50:37.639455+0000', 'last_fullsized': '2024-03-30T00:50:37.639455+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:21:22.322038+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.35 is not active+clean 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.640852+0000', 'last_change': '2024-03-30T00:50:37.640852+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.640852+0000', 'last_undegraded': '2024-03-30T00:50:37.640852+0000', 'last_fullsized': '2024-03-30T00:50:37.640852+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.34 is not active+clean 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.638906+0000', 'last_change': '2024-03-30T00:50:37.638906+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.638906+0000', 'last_undegraded': '2024-03-30T00:50:37.638906+0000', 'last_fullsized': '2024-03-30T00:50:37.638906+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:09:58.609620+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.33 is not active+clean 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.641531+0000', 'last_change': '2024-03-30T00:50:37.641531+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.641531+0000', 'last_undegraded': '2024-03-30T00:50:37.641531+0000', 'last_fullsized': '2024-03-30T00:50:37.641531+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.32 is not active+clean 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.647544+0000', 'last_change': '2024-03-30T00:50:37.647544+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.647544+0000', 'last_undegraded': '2024-03-30T00:50:37.647544+0000', 'last_fullsized': '2024-03-30T00:50:37.647544+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:50:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.31 is not active+clean 2024-03-30T00:50:42.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.638432+0000', 'last_change': '2024-03-30T00:50:37.638432+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.638432+0000', 'last_undegraded': '2024-03-30T00:50:37.638432+0000', 'last_fullsized': '2024-03-30T00:50:37.638432+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:50:42.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.30 is not active+clean 2024-03-30T00:50:42.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.637122+0000', 'last_change': '2024-03-30T00:50:37.637122+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.637122+0000', 'last_undegraded': '2024-03-30T00:50:37.637122+0000', 'last_fullsized': '2024-03-30T00:50:37.637122+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T05:21:46.573525+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:50:42.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2f is not active+clean 2024-03-30T00:50:42.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.643847+0000', 'last_change': '2024-03-30T00:50:37.643847+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.643847+0000', 'last_undegraded': '2024-03-30T00:50:37.643847+0000', 'last_fullsized': '2024-03-30T00:50:37.643847+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:50:42.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2e is not active+clean 2024-03-30T00:50:42.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.645745+0000', 'last_change': '2024-03-30T00:50:37.645745+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.645745+0000', 'last_undegraded': '2024-03-30T00:50:37.645745+0000', 'last_fullsized': '2024-03-30T00:50:37.645745+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:04:48.536583+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:50:42.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2d is not active+clean 2024-03-30T00:50:42.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.646345+0000', 'last_change': '2024-03-30T00:50:37.646345+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.646345+0000', 'last_undegraded': '2024-03-30T00:50:37.646345+0000', 'last_fullsized': '2024-03-30T00:50:37.646345+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:24:38.180300+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:50:42.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2c is not active+clean 2024-03-30T00:50:42.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.640110+0000', 'last_change': '2024-03-30T00:50:37.640110+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.640110+0000', 'last_undegraded': '2024-03-30T00:50:37.640110+0000', 'last_fullsized': '2024-03-30T00:50:37.640110+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:38:56.684685+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T00:50:42.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2b is not active+clean 2024-03-30T00:50:42.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.638089+0000', 'last_change': '2024-03-30T00:50:37.638089+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.638089+0000', 'last_undegraded': '2024-03-30T00:50:37.638089+0000', 'last_fullsized': '2024-03-30T00:50:37.638089+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:50:42.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2a is not active+clean 2024-03-30T00:50:42.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.639932+0000', 'last_change': '2024-03-30T00:50:37.639932+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.639932+0000', 'last_undegraded': '2024-03-30T00:50:37.639932+0000', 'last_fullsized': '2024-03-30T00:50:37.639932+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:50:42.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.29 is not active+clean 2024-03-30T00:50:42.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.636945+0000', 'last_change': '2024-03-30T00:50:37.636945+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.636945+0000', 'last_undegraded': '2024-03-30T00:50:37.636945+0000', 'last_fullsized': '2024-03-30T00:50:37.636945+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:50:42.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.28 is not active+clean 2024-03-30T00:50:42.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.641895+0000', 'last_change': '2024-03-30T00:50:37.641895+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.641895+0000', 'last_undegraded': '2024-03-30T00:50:37.641895+0000', 'last_fullsized': '2024-03-30T00:50:37.641895+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:50:42.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.27 is not active+clean 2024-03-30T00:50:42.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.645034+0000', 'last_change': '2024-03-30T00:50:37.645034+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.645034+0000', 'last_undegraded': '2024-03-30T00:50:37.645034+0000', 'last_fullsized': '2024-03-30T00:50:37.645034+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.26 is not active+clean 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.642363+0000', 'last_change': '2024-03-30T00:50:37.642363+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.642363+0000', 'last_undegraded': '2024-03-30T00:50:37.642363+0000', 'last_fullsized': '2024-03-30T00:50:37.642363+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:43:56.563733+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.25 is not active+clean 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.645743+0000', 'last_change': '2024-03-30T00:50:37.645743+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.645743+0000', 'last_undegraded': '2024-03-30T00:50:37.645743+0000', 'last_fullsized': '2024-03-30T00:50:37.645743+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:26:13.551635+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.24 is not active+clean 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.638476+0000', 'last_change': '2024-03-30T00:50:37.638476+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.638476+0000', 'last_undegraded': '2024-03-30T00:50:37.638476+0000', 'last_fullsized': '2024-03-30T00:50:37.638476+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T02:34:39.399906+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.23 is not active+clean 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.640963+0000', 'last_change': '2024-03-30T00:50:37.640963+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.640963+0000', 'last_undegraded': '2024-03-30T00:50:37.640963+0000', 'last_fullsized': '2024-03-30T00:50:37.640963+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:10:12.688382+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.22 is not active+clean 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.647144+0000', 'last_change': '2024-03-30T00:50:37.647144+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.647144+0000', 'last_undegraded': '2024-03-30T00:50:37.647144+0000', 'last_fullsized': '2024-03-30T00:50:37.647144+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.21 is not active+clean 2024-03-30T00:50:42.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.645120+0000', 'last_change': '2024-03-30T00:50:37.645120+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.645120+0000', 'last_undegraded': '2024-03-30T00:50:37.645120+0000', 'last_fullsized': '2024-03-30T00:50:37.645120+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:24:14.841469+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:50:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.20 is not active+clean 2024-03-30T00:50:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.641328+0000', 'last_change': '2024-03-30T00:50:37.641328+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.641328+0000', 'last_undegraded': '2024-03-30T00:50:37.641328+0000', 'last_fullsized': '2024-03-30T00:50:37.641328+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:50:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1c is not active+clean 2024-03-30T00:50:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.641925+0000', 'last_change': '2024-03-30T00:50:37.641925+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.641925+0000', 'last_undegraded': '2024-03-30T00:50:37.641925+0000', 'last_fullsized': '2024-03-30T00:50:37.641925+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:50:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1d is not active+clean 2024-03-30T00:50:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.637154+0000', 'last_change': '2024-03-30T00:50:37.637154+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.637154+0000', 'last_undegraded': '2024-03-30T00:50:37.637154+0000', 'last_fullsized': '2024-03-30T00:50:37.637154+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:44:58.975555+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:50:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1e is not active+clean 2024-03-30T00:50:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.642651+0000', 'last_change': '2024-03-30T00:50:37.642651+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.642651+0000', 'last_undegraded': '2024-03-30T00:50:37.642651+0000', 'last_fullsized': '2024-03-30T00:50:37.642651+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:50:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1f is not active+clean 2024-03-30T00:50:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.637468+0000', 'last_change': '2024-03-30T00:50:37.637468+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.637468+0000', 'last_undegraded': '2024-03-30T00:50:37.637468+0000', 'last_fullsized': '2024-03-30T00:50:37.637468+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:50:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.18 is not active+clean 2024-03-30T00:50:42.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.640607+0000', 'last_change': '2024-03-30T00:50:37.640607+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.640607+0000', 'last_undegraded': '2024-03-30T00:50:37.640607+0000', 'last_fullsized': '2024-03-30T00:50:37.640607+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:50:42.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.19 is not active+clean 2024-03-30T00:50:42.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.637936+0000', 'last_change': '2024-03-30T00:50:37.637936+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.637936+0000', 'last_undegraded': '2024-03-30T00:50:37.637936+0000', 'last_fullsized': '2024-03-30T00:50:37.637936+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:50:42.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1a is not active+clean 2024-03-30T00:50:42.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.646745+0000', 'last_change': '2024-03-30T00:50:37.646745+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.646745+0000', 'last_undegraded': '2024-03-30T00:50:37.646745+0000', 'last_fullsized': '2024-03-30T00:50:37.646745+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:51:44.264107+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:50:42.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1b is not active+clean 2024-03-30T00:50:42.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.641119+0000', 'last_change': '2024-03-30T00:50:37.641119+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.641119+0000', 'last_undegraded': '2024-03-30T00:50:37.641119+0000', 'last_fullsized': '2024-03-30T00:50:37.641119+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:50:42.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.14 is not active+clean 2024-03-30T00:50:42.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.647075+0000', 'last_change': '2024-03-30T00:50:37.647075+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.647075+0000', 'last_undegraded': '2024-03-30T00:50:37.647075+0000', 'last_fullsized': '2024-03-30T00:50:37.647075+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:20:01.855290+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:50:42.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.15 is not active+clean 2024-03-30T00:50:42.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.642802+0000', 'last_change': '2024-03-30T00:50:37.642802+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.642802+0000', 'last_undegraded': '2024-03-30T00:50:37.642802+0000', 'last_fullsized': '2024-03-30T00:50:37.642802+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:50:42.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.16 is not active+clean 2024-03-30T00:50:42.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.642586+0000', 'last_change': '2024-03-30T00:50:37.642586+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.642586+0000', 'last_undegraded': '2024-03-30T00:50:37.642586+0000', 'last_fullsized': '2024-03-30T00:50:37.642586+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:50:42.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.17 is not active+clean 2024-03-30T00:50:42.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.644570+0000', 'last_change': '2024-03-30T00:50:37.644570+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.644570+0000', 'last_undegraded': '2024-03-30T00:50:37.644570+0000', 'last_fullsized': '2024-03-30T00:50:37.644570+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:46:27.974997+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:50:42.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.10 is not active+clean 2024-03-30T00:50:42.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.646353+0000', 'last_change': '2024-03-30T00:50:37.646353+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.646353+0000', 'last_undegraded': '2024-03-30T00:50:37.646353+0000', 'last_fullsized': '2024-03-30T00:50:37.646353+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:36:05.445602+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T00:50:42.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.11 is not active+clean 2024-03-30T00:50:42.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.638573+0000', 'last_change': '2024-03-30T00:50:37.638573+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.638573+0000', 'last_undegraded': '2024-03-30T00:50:37.638573+0000', 'last_fullsized': '2024-03-30T00:50:37.638573+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:50:42.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.12 is not active+clean 2024-03-30T00:50:42.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.640244+0000', 'last_change': '2024-03-30T00:50:37.640244+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.640244+0000', 'last_undegraded': '2024-03-30T00:50:37.640244+0000', 'last_fullsized': '2024-03-30T00:50:37.640244+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:02:07.606929+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T00:50:42.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.c is not active+clean 2024-03-30T00:50:42.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.641714+0000', 'last_change': '2024-03-30T00:50:37.641714+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.641714+0000', 'last_undegraded': '2024-03-30T00:50:37.641714+0000', 'last_fullsized': '2024-03-30T00:50:37.641714+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:50:42.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.d is not active+clean 2024-03-30T00:50:42.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.637343+0000', 'last_change': '2024-03-30T00:50:37.637343+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.637343+0000', 'last_undegraded': '2024-03-30T00:50:37.637343+0000', 'last_fullsized': '2024-03-30T00:50:37.637343+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T00:50:42.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.e is not active+clean 2024-03-30T00:50:42.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.641239+0000', 'last_change': '2024-03-30T00:50:37.641239+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.641239+0000', 'last_undegraded': '2024-03-30T00:50:37.641239+0000', 'last_fullsized': '2024-03-30T00:50:37.641239+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:09:05.110381+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:50:42.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.f is not active+clean 2024-03-30T00:50:42.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.643208+0000', 'last_change': '2024-03-30T00:50:37.643208+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.643208+0000', 'last_undegraded': '2024-03-30T00:50:37.643208+0000', 'last_fullsized': '2024-03-30T00:50:37.643208+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T05:10:47.211883+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:50:42.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.8 is not active+clean 2024-03-30T00:50:42.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.636644+0000', 'last_change': '2024-03-30T00:50:37.636644+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.636644+0000', 'last_undegraded': '2024-03-30T00:50:37.636644+0000', 'last_fullsized': '2024-03-30T00:50:37.636644+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:52:46.481754+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:50:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.9 is not active+clean 2024-03-30T00:50:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.645084+0000', 'last_change': '2024-03-30T00:50:37.645084+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.645084+0000', 'last_undegraded': '2024-03-30T00:50:37.645084+0000', 'last_fullsized': '2024-03-30T00:50:37.645084+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T05:04:19.985679+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:50:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.a is not active+clean 2024-03-30T00:50:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.642635+0000', 'last_change': '2024-03-30T00:50:37.642635+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.642635+0000', 'last_undegraded': '2024-03-30T00:50:37.642635+0000', 'last_fullsized': '2024-03-30T00:50:37.642635+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:50:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.b is not active+clean 2024-03-30T00:50:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.643298+0000', 'last_change': '2024-03-30T00:50:37.643298+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.643298+0000', 'last_undegraded': '2024-03-30T00:50:37.643298+0000', 'last_fullsized': '2024-03-30T00:50:37.643298+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:50:37.965725+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T00:50:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.4 is not active+clean 2024-03-30T00:50:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.638050+0000', 'last_change': '2024-03-30T00:50:37.638050+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.638050+0000', 'last_undegraded': '2024-03-30T00:50:37.638050+0000', 'last_fullsized': '2024-03-30T00:50:37.638050+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:40:19.197836+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:50:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.5 is not active+clean 2024-03-30T00:50:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.637305+0000', 'last_change': '2024-03-30T00:50:37.637305+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.637305+0000', 'last_undegraded': '2024-03-30T00:50:37.637305+0000', 'last_fullsized': '2024-03-30T00:50:37.637305+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:50:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.6 is not active+clean 2024-03-30T00:50:42.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.637015+0000', 'last_change': '2024-03-30T00:50:37.637015+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.637015+0000', 'last_undegraded': '2024-03-30T00:50:37.637015+0000', 'last_fullsized': '2024-03-30T00:50:37.637015+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:50:42.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.7 is not active+clean 2024-03-30T00:50:42.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.641788+0000', 'last_change': '2024-03-30T00:50:37.641788+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.641788+0000', 'last_undegraded': '2024-03-30T00:50:37.641788+0000', 'last_fullsized': '2024-03-30T00:50:37.641788+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T05:34:31.524194+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T00:50:42.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.0 is not active+clean 2024-03-30T00:50:42.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.635996+0000', 'last_change': '2024-03-30T00:50:37.635996+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.635996+0000', 'last_undegraded': '2024-03-30T00:50:37.635996+0000', 'last_fullsized': '2024-03-30T00:50:37.635996+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:52:14.331871+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T00:50:42.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1 is not active+clean 2024-03-30T00:50:42.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.642870+0000', 'last_change': '2024-03-30T00:50:37.642870+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.642870+0000', 'last_undegraded': '2024-03-30T00:50:37.642870+0000', 'last_fullsized': '2024-03-30T00:50:37.642870+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T00:50:42.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2 is not active+clean 2024-03-30T00:50:42.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.636935+0000', 'last_change': '2024-03-30T00:50:37.636935+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.636935+0000', 'last_undegraded': '2024-03-30T00:50:37.636935+0000', 'last_fullsized': '2024-03-30T00:50:37.636935+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:50:42.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3 is not active+clean 2024-03-30T00:50:42.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.638023+0000', 'last_change': '2024-03-30T00:50:37.638023+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.638023+0000', 'last_undegraded': '2024-03-30T00:50:37.638023+0000', 'last_fullsized': '2024-03-30T00:50:37.638023+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T00:50:42.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.13 is not active+clean 2024-03-30T00:50:42.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 123, 'state': 'creating+peering', 'last_fresh': '2024-03-30T00:50:37.637355+0000', 'last_change': '2024-03-30T00:50:37.637355+0000', 'last_active': '2024-03-30T00:50:37.625832+0000', 'last_peered': '2024-03-30T00:50:37.625832+0000', 'last_clean': '2024-03-30T00:50:37.625832+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T00:50:37.637355+0000', 'last_undegraded': '2024-03-30T00:50:37.637355+0000', 'last_fullsized': '2024-03-30T00:50:37.637355+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'last_clean_scrub_stamp': '2024-03-30T00:50:37.625832+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T00:50:42.980 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-30T00:50:43.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.111+0000 7f531ca00640 1 Processor -- start 2024-03-30T00:50:43.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.111+0000 7f531ca00640 1 -- start start 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.111+0000 7f531ca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53180cda90 0x7f53180ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.111+0000 7f531ca00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f531815a370 0x7f53180cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.111+0000 7f531ca00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531815a990 con 0x7f53180cde70 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.111+0000 7f531ca00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53180b1b80 con 0x7f53180cda90 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.111+0000 7f531ca00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53180cd3e0 con 0x7f531815a370 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.111+0000 7f5315e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53180cda90 0x7f53180ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.111+0000 7f5315e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53180cda90 0x7f53180ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58906/0 (socket says 172.21.15.66:58906) 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.115+0000 7f5315e00640 1 -- 172.21.15.66:0/1120189244 learned_addr learned my addr 172.21.15.66:0/1120189244 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.115+0000 7f5316800640 1 --2- 172.21.15.66:0/1120189244 >> v2:172.21.15.156:3300/0 conn(0x7f531815a370 0x7f53180cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.115+0000 7f5315e00640 1 -- 172.21.15.66:0/1120189244 >> v2:172.21.15.156:3300/0 conn(0x7f531815a370 msgr2=0x7f53180cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.115+0000 7f5315e00640 1 --2- 172.21.15.66:0/1120189244 >> v2:172.21.15.156:3300/0 conn(0x7f531815a370 0x7f53180cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.115+0000 7f5315e00640 1 -- 172.21.15.66:0/1120189244 >> v1:172.21.15.156:6789/0 conn(0x7f53180cde70 legacy=0x7f5318159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.115+0000 7f5315e00640 1 -- 172.21.15.66:0/1120189244 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53180b11e0 con 0x7f53180cda90 2024-03-30T00:50:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.115+0000 7f5315e00640 1 --2- 172.21.15.66:0/1120189244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53180cda90 0x7f53180ce520 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f53180b3140 tx=0x7f530801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f450bb47f0b8e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.115+0000 7f5315400640 1 -- 172.21.15.66:0/1120189244 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53080ac600 con 0x7f53180cda90 2024-03-30T00:50:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.115+0000 7f5315400640 1 -- 172.21.15.66:0/1120189244 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53080a8050 con 0x7f53180cda90 2024-03-30T00:50:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.115+0000 7f5315400640 1 -- 172.21.15.66:0/1120189244 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53080ac980 con 0x7f53180cda90 2024-03-30T00:50:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 -- 172.21.15.66:0/1120189244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53180cda90 msgr2=0x7f53180ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 --2- 172.21.15.66:0/1120189244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53180cda90 0x7f53180ce520 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f53180b3140 tx=0x7f530801c490 comp rx=0 tx=0).stop 2024-03-30T00:50:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 -- 172.21.15.66:0/1120189244 shutdown_connections 2024-03-30T00:50:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 --2- 172.21.15.66:0/1120189244 >> v2:172.21.15.156:3300/0 conn(0x7f531815a370 0x7f53180cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 --2- 172.21.15.66:0/1120189244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53180cda90 0x7f53180ce520 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 -- 172.21.15.66:0/1120189244 >> 172.21.15.66:0/1120189244 conn(0x7f5318069f60 msgr2=0x7f531806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 -- 172.21.15.66:0/1120189244 shutdown_connections 2024-03-30T00:50:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 -- 172.21.15.66:0/1120189244 wait complete. 2024-03-30T00:50:43.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 Processor -- start 2024-03-30T00:50:43.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 -- start start 2024-03-30T00:50:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53180cda90 0x7f5318156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53180cde70 0x7f531815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f531815a370 0x7f531815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5318156790 con 0x7f53180cde70 2024-03-30T00:50:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53180c4390 con 0x7f53180cda90 2024-03-30T00:50:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f531ca00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53180c45c0 con 0x7f531815a370 2024-03-30T00:50:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f530d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53180cde70 0x7f531815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f530d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53180cde70 0x7f531815baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58918/0 (socket says 172.21.15.66:58918) 2024-03-30T00:50:43.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f530d400640 1 -- 172.21.15.66:0/2406942732 learned_addr learned my addr 172.21.15.66:0/2406942732 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:43.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f5315e00640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53180cda90 0x7f5318156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:43.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f5316800640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f531815a370 0x7f531815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:43.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f530d400640 1 -- 172.21.15.66:0/2406942732 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f531815a370 msgr2=0x7f531815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f530d400640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f531815a370 0x7f531815c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f530d400640 1 -- 172.21.15.66:0/2406942732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53180cda90 msgr2=0x7f5318156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f530d400640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53180cda90 0x7f5318156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f530d400640 1 -- 172.21.15.66:0/2406942732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53080094d0 con 0x7f53180cde70 2024-03-30T00:50:43.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f5316800640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f531815a370 0x7f531815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:43.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.119+0000 7f530d400640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53180cde70 0x7f531815baf0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f530000c7c0 tx=0x7f530000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.123+0000 7f530ea00640 1 -- 172.21.15.66:0/2406942732 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300003b80 con 0x7f53180cde70 2024-03-30T00:50:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.123+0000 7f530ea00640 1 -- 172.21.15.66:0/2406942732 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5300003ce0 con 0x7f53180cde70 2024-03-30T00:50:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.123+0000 7f530ea00640 1 -- 172.21.15.66:0/2406942732 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530000aa90 con 0x7f53180cde70 2024-03-30T00:50:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.123+0000 7f531ca00640 1 -- 172.21.15.66:0/2406942732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53180c48a0 con 0x7f53180cde70 2024-03-30T00:50:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.123+0000 7f531ca00640 1 -- 172.21.15.66:0/2406942732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53181f6ee0 con 0x7f53180cde70 2024-03-30T00:50:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.123+0000 7f531ca00640 1 -- 172.21.15.66:0/2406942732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5318066c30 con 0x7f53180cde70 2024-03-30T00:50:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.123+0000 7f530ea00640 1 -- 172.21.15.66:0/2406942732 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f530000abf0 con 0x7f53180cde70 2024-03-30T00:50:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.123+0000 7f530ea00640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52f4066690 0x7f52f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.123+0000 7f530ea00640 1 -- 172.21.15.66:0/2406942732 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5300084160 con 0x7f53180cde70 2024-03-30T00:50:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.123+0000 7f5315e00640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52f4066690 0x7f52f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.123+0000 7f5315e00640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52f4066690 0x7f52f4068b50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f53080ae000 tx=0x7f53080a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:43.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.127+0000 7f530ea00640 1 -- 172.21.15.66:0/2406942732 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f53000512f0 con 0x7f53180cde70 2024-03-30T00:50:43.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 -- 172.21.15.66:0/2406942732 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f53180b3170 con 0x7f53180cde70 2024-03-30T00:50:43.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f530ea00640 1 -- 172.21.15.66:0/2406942732 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1015 (secure 0 0 0) 0x7f5300054910 con 0x7f53180cde70 2024-03-30T00:50:43.354 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:43.354 INFO:teuthology.orchestra.run.smithi066.stdout:{"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":677,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1711759177,"num_in_osds":8,"osd_in_since":1711759169,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592730,"bytes_used":268652544,"bytes_avail":772825460736,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":95,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":9366}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-03-30T00:50:20.996467+0000","services":{}},"progress_events":{}} 2024-03-30T00:50:43.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 -- 172.21.15.66:0/2406942732 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52f4066690 msgr2=0x7f52f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52f4066690 0x7f52f4068b50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f53080ae000 tx=0x7f53080a7000 comp rx=0 tx=0).stop 2024-03-30T00:50:43.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 -- 172.21.15.66:0/2406942732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53180cde70 msgr2=0x7f531815baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53180cde70 0x7f531815baf0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f530000c7c0 tx=0x7f530000cc80 comp rx=0 tx=0).stop 2024-03-30T00:50:43.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 -- 172.21.15.66:0/2406942732 shutdown_connections 2024-03-30T00:50:43.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f531815a370 0x7f531815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f52f4066690 0x7f52f4068b50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53180cde70 0x7f531815baf0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 --2- 172.21.15.66:0/2406942732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53180cda90 0x7f5318156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 -- 172.21.15.66:0/2406942732 >> 172.21.15.66:0/2406942732 conn(0x7f5318069f60 msgr2=0x7f531815d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:43.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 -- 172.21.15.66:0/2406942732 shutdown_connections 2024-03-30T00:50:43.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.347+0000 7f531ca00640 1 -- 172.21.15.66:0/2406942732 wait complete. 2024-03-30T00:50:43.375 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-30T00:50:43.375 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:50:43.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55e7800640 1 Processor -- start 2024-03-30T00:50:43.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55e7800640 1 -- start start 2024-03-30T00:50:43.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55e7800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e006e800 0x7f55e00d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:43.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55e7800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e006b0b0 con 0x7f55e00cc560 2024-03-30T00:50:43.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55e7800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e006b390 con 0x7f55e006f8e0 2024-03-30T00:50:43.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55e7800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e0160640 con 0x7f55e006e800 2024-03-30T00:50:43.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55e4e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f55e00cc560 0x7f55e006e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53492/0 (socket says 172.21.15.66:53492) 2024-03-30T00:50:43.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55e4e00640 1 -- 172.21.15.66:0/2648454964 learned_addr learned my addr 172.21.15.66:0/2648454964 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:43.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55dfe00640 1 --2- 172.21.15.66:0/2648454964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e006e800 0x7f55e00d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:43.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55df400640 1 -- 172.21.15.66:0/2648454964 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704775537 0 0) 0x7f55e006b0b0 con 0x7f55e00cc560 2024-03-30T00:50:43.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55df400640 1 -- 172.21.15.66:0/2648454964 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55c8003530 con 0x7f55e00cc560 2024-03-30T00:50:43.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55df400640 1 -- 172.21.15.66:0/2648454964 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514274432 0 0) 0x7f55e006b390 con 0x7f55e006f8e0 2024-03-30T00:50:43.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55df400640 1 -- 172.21.15.66:0/2648454964 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55e006b0b0 con 0x7f55e006f8e0 2024-03-30T00:50:43.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55dfe00640 1 -- 172.21.15.66:0/2648454964 >> v1:172.21.15.156:6789/0 conn(0x7f55e006f8e0 legacy=0x7f55e00ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55dfe00640 1 -- 172.21.15.66:0/2648454964 >> v1:172.21.15.66:6789/0 conn(0x7f55e00cc560 legacy=0x7f55e006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.507+0000 7f55dfe00640 1 -- 172.21.15.66:0/2648454964 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e006b670 con 0x7f55e006e800 2024-03-30T00:50:43.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.511+0000 7f55dfe00640 1 --2- 172.21.15.66:0/2648454964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e006e800 0x7f55e00d1060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0097f0 tx=0x7f55cc01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5389d550ae01370e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:43.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.511+0000 7f55df400640 1 -- 172.21.15.66:0/2648454964 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0aa430 con 0x7f55e006e800 2024-03-30T00:50:43.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.511+0000 7f55df400640 1 -- 172.21.15.66:0/2648454964 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc0aa590 con 0x7f55e006e800 2024-03-30T00:50:43.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.511+0000 7f55df400640 1 -- 172.21.15.66:0/2648454964 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0aa900 con 0x7f55e006e800 2024-03-30T00:50:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.511+0000 7f55e7800640 1 -- 172.21.15.66:0/2648454964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e006e800 msgr2=0x7f55e00d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.511+0000 7f55e7800640 1 --2- 172.21.15.66:0/2648454964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e006e800 0x7f55e00d1060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0097f0 tx=0x7f55cc01c4b0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.511+0000 7f55e7800640 1 -- 172.21.15.66:0/2648454964 shutdown_connections 2024-03-30T00:50:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.511+0000 7f55e7800640 1 --2- 172.21.15.66:0/2648454964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e006e800 0x7f55e00d1060 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.511+0000 7f55e7800640 1 -- 172.21.15.66:0/2648454964 >> 172.21.15.66:0/2648454964 conn(0x7f55e00d0540 msgr2=0x7f55e00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.511+0000 7f55e7800640 1 -- 172.21.15.66:0/2648454964 shutdown_connections 2024-03-30T00:50:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.511+0000 7f55e7800640 1 -- 172.21.15.66:0/2648454964 wait complete. 2024-03-30T00:50:43.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e7800640 1 Processor -- start 2024-03-30T00:50:43.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e7800640 1 -- start start 2024-03-30T00:50:43.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e7800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e006e800 0x7f55e015d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e7800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55e006f8e0 0x7f55e015ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e7800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e00cc560 0x7f55e00add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e7800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e015d620 con 0x7f55e006e800 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e7800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00c7720 con 0x7f55e006f8e0 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e7800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00c7a00 con 0x7f55e00cc560 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e006e800 0x7f55e015d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e006e800 0x7f55e015d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58928/0 (socket says 172.21.15.66:58928) 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e4e00640 1 -- 172.21.15.66:0/1106580322 learned_addr learned my addr 172.21.15.66:0/1106580322 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55dfe00640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55e006f8e0 0x7f55e015ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e5800640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e00cc560 0x7f55e00add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e4e00640 1 -- 172.21.15.66:0/1106580322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e00cc560 msgr2=0x7f55e00add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e4e00640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e00cc560 0x7f55e00add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e4e00640 1 -- 172.21.15.66:0/1106580322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55e006f8e0 msgr2=0x7f55e015ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e4e00640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55e006f8e0 0x7f55e015ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e4e00640 1 -- 172.21.15.66:0/1106580322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc009420 con 0x7f55e006e800 2024-03-30T00:50:43.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e5800640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e00cc560 0x7f55e00add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:43.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e4e00640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e006e800 0x7f55e015d010 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c4f0 tx=0x7f55d400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:43.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55dcc00640 1 -- 172.21.15.66:0/1106580322 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4005cf0 con 0x7f55e006e800 2024-03-30T00:50:43.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55dcc00640 1 -- 172.21.15.66:0/1106580322 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d4005e50 con 0x7f55e006e800 2024-03-30T00:50:43.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55dcc00640 1 -- 172.21.15.66:0/1106580322 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4007270 con 0x7f55e006e800 2024-03-30T00:50:43.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e7800640 1 -- 172.21.15.66:0/1106580322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e00c7ce0 con 0x7f55e006e800 2024-03-30T00:50:43.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.515+0000 7f55e7800640 1 -- 172.21.15.66:0/1106580322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e015f4c0 con 0x7f55e006e800 2024-03-30T00:50:43.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.519+0000 7f55e7800640 1 -- 172.21.15.66:0/1106580322 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e0066c30 con 0x7f55e006e800 2024-03-30T00:50:43.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.519+0000 7f55dcc00640 1 -- 172.21.15.66:0/1106580322 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f55d40073d0 con 0x7f55e006e800 2024-03-30T00:50:43.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.519+0000 7f55dcc00640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55c0066690 0x7f55c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:43.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.519+0000 7f55dcc00640 1 -- 172.21.15.66:0/1106580322 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f55d4083810 con 0x7f55e006e800 2024-03-30T00:50:43.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.519+0000 7f55dfe00640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55c0066690 0x7f55c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:43.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.519+0000 7f55dfe00640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55c0066690 0x7f55c0068b50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0ae000 tx=0x7f55cc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:43.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.523+0000 7f55dcc00640 1 -- 172.21.15.66:0/1106580322 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f55d4050ae0 con 0x7f55e006e800 2024-03-30T00:50:43.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.679+0000 7f55e7800640 1 -- 172.21.15.66:0/1106580322 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55e00cc940 con 0x7f55c0066690 2024-03-30T00:50:43.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.687+0000 7f55dcc00640 1 -- 172.21.15.66:0/1106580322 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289664 (secure 0 0 0) 0x7f55cc0aa700 con 0x7f55c0066690 2024-03-30T00:50:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:43.701 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:50:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 -- 172.21.15.66:0/1106580322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55c0066690 msgr2=0x7f55c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55c0066690 0x7f55c0068b50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0ae000 tx=0x7f55cc0a7000 comp rx=0 tx=0).stop 2024-03-30T00:50:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 -- 172.21.15.66:0/1106580322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e006e800 msgr2=0x7f55e015d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e006e800 0x7f55e015d010 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c4f0 tx=0x7f55d400c9b0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 -- 172.21.15.66:0/1106580322 shutdown_connections 2024-03-30T00:50:43.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55e00cc560 0x7f55e00add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55e006f8e0 0x7f55e015ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f55c0066690 0x7f55c0068b50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 --2- 172.21.15.66:0/1106580322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e006e800 0x7f55e015d010 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 -- 172.21.15.66:0/1106580322 >> 172.21.15.66:0/1106580322 conn(0x7f55e00d0540 msgr2=0x7f55e00aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 -- 172.21.15.66:0/1106580322 shutdown_connections 2024-03-30T00:50:43.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:43.691+0000 7f55e7800640 1 -- 172.21.15.66:0/1106580322 wait complete. 2024-03-30T00:50:43.708 INFO:tasks.ceph.mds.a.smithi066.stderr:2024-03-30T00:50:43.699+0000 7f5857000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:50:43.719 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-03-30T00:50:43.000470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262356,"kb_used_data":13384,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712364,"statfs":{"total":773094113280,"available":772825460736,"internally_reserved":0,"allocated":13705216,"data_stored":6945372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001015"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.065924+0000","last_change":"2024-03-30T00:50:38.058216+0000","last_active":"2024-03-30T00:50:38.065924+0000","last_peered":"2024-03-30T00:50:38.065924+0000","last_clean":"2024-03-30T00:50:38.065924+0000","last_became_active":"2024-03-30T00:50:38.057791+0000","last_became_peered":"2024-03-30T00:50:38.057791+0000","last_unstale":"2024-03-30T00:50:38.065924+0000","last_undegraded":"2024-03-30T00:50:38.065924+0000","last_fullsized":"2024-03-30T00:50:38.065924+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.079054+0000","last_change":"2024-03-30T00:50:38.068156+0000","last_active":"2024-03-30T00:50:38.079054+0000","last_peered":"2024-03-30T00:50:38.079054+0000","last_clean":"2024-03-30T00:50:38.079054+0000","last_became_active":"2024-03-30T00:50:38.062931+0000","last_became_peered":"2024-03-30T00:50:38.062931+0000","last_unstale":"2024-03-30T00:50:38.079054+0000","last_undegraded":"2024-03-30T00:50:38.079054+0000","last_fullsized":"2024-03-30T00:50:38.079054+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.079360+0000","last_change":"2024-03-30T00:50:38.069777+0000","last_active":"2024-03-30T00:50:38.079360+0000","last_peered":"2024-03-30T00:50:38.079360+0000","last_clean":"2024-03-30T00:50:38.079360+0000","last_became_active":"2024-03-30T00:50:38.069461+0000","last_became_peered":"2024-03-30T00:50:38.069461+0000","last_unstale":"2024-03-30T00:50:38.079360+0000","last_undegraded":"2024-03-30T00:50:38.079360+0000","last_fullsized":"2024-03-30T00:50:38.079360+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:53:58.506502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.057834+0000","last_change":"2024-03-30T00:50:38.056037+0000","last_active":"2024-03-30T00:50:38.057834+0000","last_peered":"2024-03-30T00:50:38.057834+0000","last_clean":"2024-03-30T00:50:38.057834+0000","last_became_active":"2024-03-30T00:50:38.055796+0000","last_became_peered":"2024-03-30T00:50:38.055796+0000","last_unstale":"2024-03-30T00:50:38.057834+0000","last_undegraded":"2024-03-30T00:50:38.057834+0000","last_fullsized":"2024-03-30T00:50:38.057834+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.068304+0000","last_change":"2024-03-30T00:50:38.064340+0000","last_active":"2024-03-30T00:50:38.068304+0000","last_peered":"2024-03-30T00:50:38.068304+0000","last_clean":"2024-03-30T00:50:38.068304+0000","last_became_active":"2024-03-30T00:50:38.064079+0000","last_became_peered":"2024-03-30T00:50:38.064079+0000","last_unstale":"2024-03-30T00:50:38.068304+0000","last_undegraded":"2024-03-30T00:50:38.068304+0000","last_fullsized":"2024-03-30T00:50:38.068304+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.081030+0000","last_change":"2024-03-30T00:50:38.078616+0000","last_active":"2024-03-30T00:50:38.081030+0000","last_peered":"2024-03-30T00:50:38.081030+0000","last_clean":"2024-03-30T00:50:38.081030+0000","last_became_active":"2024-03-30T00:50:38.077950+0000","last_became_peered":"2024-03-30T00:50:38.077950+0000","last_unstale":"2024-03-30T00:50:38.081030+0000","last_undegraded":"2024-03-30T00:50:38.081030+0000","last_fullsized":"2024-03-30T00:50:38.081030+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:16.922513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.072189+0000","last_change":"2024-03-30T00:50:38.067925+0000","last_active":"2024-03-30T00:50:38.072189+0000","last_peered":"2024-03-30T00:50:38.072189+0000","last_clean":"2024-03-30T00:50:38.072189+0000","last_became_active":"2024-03-30T00:50:38.067684+0000","last_became_peered":"2024-03-30T00:50:38.067684+0000","last_unstale":"2024-03-30T00:50:38.072189+0000","last_undegraded":"2024-03-30T00:50:38.072189+0000","last_fullsized":"2024-03-30T00:50:38.072189+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:10:54.265531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.085039+0000","last_change":"2024-03-30T00:50:38.080042+0000","last_active":"2024-03-30T00:50:38.085039+0000","last_peered":"2024-03-30T00:50:38.085039+0000","last_clean":"2024-03-30T00:50:38.085039+0000","last_became_active":"2024-03-30T00:50:38.079814+0000","last_became_peered":"2024-03-30T00:50:38.079814+0000","last_unstale":"2024-03-30T00:50:38.085039+0000","last_undegraded":"2024-03-30T00:50:38.085039+0000","last_fullsized":"2024-03-30T00:50:38.085039+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077355+0000","last_change":"2024-03-30T00:50:38.074264+0000","last_active":"2024-03-30T00:50:38.077355+0000","last_peered":"2024-03-30T00:50:38.077355+0000","last_clean":"2024-03-30T00:50:38.077355+0000","last_became_active":"2024-03-30T00:50:38.073974+0000","last_became_peered":"2024-03-30T00:50:38.073974+0000","last_unstale":"2024-03-30T00:50:38.077355+0000","last_undegraded":"2024-03-30T00:50:38.077355+0000","last_fullsized":"2024-03-30T00:50:38.077355+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.076619+0000","last_change":"2024-03-30T00:50:38.075046+0000","last_active":"2024-03-30T00:50:38.076619+0000","last_peered":"2024-03-30T00:50:38.076619+0000","last_clean":"2024-03-30T00:50:38.076619+0000","last_became_active":"2024-03-30T00:50:38.074491+0000","last_became_peered":"2024-03-30T00:50:38.074491+0000","last_unstale":"2024-03-30T00:50:38.076619+0000","last_undegraded":"2024-03-30T00:50:38.076619+0000","last_fullsized":"2024-03-30T00:50:38.076619+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:21:22.322038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.069215+0000","last_change":"2024-03-30T00:50:38.065503+0000","last_active":"2024-03-30T00:50:38.069215+0000","last_peered":"2024-03-30T00:50:38.069215+0000","last_clean":"2024-03-30T00:50:38.069215+0000","last_became_active":"2024-03-30T00:50:38.065214+0000","last_became_peered":"2024-03-30T00:50:38.065214+0000","last_unstale":"2024-03-30T00:50:38.069215+0000","last_undegraded":"2024-03-30T00:50:38.069215+0000","last_fullsized":"2024-03-30T00:50:38.069215+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.072621+0000","last_change":"2024-03-30T00:50:38.063810+0000","last_active":"2024-03-30T00:50:38.072621+0000","last_peered":"2024-03-30T00:50:38.072621+0000","last_clean":"2024-03-30T00:50:38.072621+0000","last_became_active":"2024-03-30T00:50:38.063556+0000","last_became_peered":"2024-03-30T00:50:38.063556+0000","last_unstale":"2024-03-30T00:50:38.072621+0000","last_undegraded":"2024-03-30T00:50:38.072621+0000","last_fullsized":"2024-03-30T00:50:38.072621+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:09:58.609620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.069764+0000","last_change":"2024-03-30T00:50:38.060007+0000","last_active":"2024-03-30T00:50:38.069764+0000","last_peered":"2024-03-30T00:50:38.069764+0000","last_clean":"2024-03-30T00:50:38.069764+0000","last_became_active":"2024-03-30T00:50:38.059642+0000","last_became_peered":"2024-03-30T00:50:38.059642+0000","last_unstale":"2024-03-30T00:50:38.069764+0000","last_undegraded":"2024-03-30T00:50:38.069764+0000","last_fullsized":"2024-03-30T00:50:38.069764+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.082658+0000","last_change":"2024-03-30T00:50:38.077900+0000","last_active":"2024-03-30T00:50:38.082658+0000","last_peered":"2024-03-30T00:50:38.082658+0000","last_clean":"2024-03-30T00:50:38.082658+0000","last_became_active":"2024-03-30T00:50:38.076312+0000","last_became_peered":"2024-03-30T00:50:38.076312+0000","last_unstale":"2024-03-30T00:50:38.082658+0000","last_undegraded":"2024-03-30T00:50:38.082658+0000","last_fullsized":"2024-03-30T00:50:38.082658+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.058370+0000","last_change":"2024-03-30T00:50:38.056313+0000","last_active":"2024-03-30T00:50:38.058370+0000","last_peered":"2024-03-30T00:50:38.058370+0000","last_clean":"2024-03-30T00:50:38.058370+0000","last_became_active":"2024-03-30T00:50:38.056078+0000","last_became_peered":"2024-03-30T00:50:38.056078+0000","last_unstale":"2024-03-30T00:50:38.058370+0000","last_undegraded":"2024-03-30T00:50:38.058370+0000","last_fullsized":"2024-03-30T00:50:38.058370+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.071819+0000","last_change":"2024-03-30T00:50:38.066213+0000","last_active":"2024-03-30T00:50:38.071819+0000","last_peered":"2024-03-30T00:50:38.071819+0000","last_clean":"2024-03-30T00:50:38.071819+0000","last_became_active":"2024-03-30T00:50:38.065918+0000","last_became_peered":"2024-03-30T00:50:38.065918+0000","last_unstale":"2024-03-30T00:50:38.071819+0000","last_undegraded":"2024-03-30T00:50:38.071819+0000","last_fullsized":"2024-03-30T00:50:38.071819+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:46.573525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.069230+0000","last_change":"2024-03-30T00:50:38.058385+0000","last_active":"2024-03-30T00:50:38.069230+0000","last_peered":"2024-03-30T00:50:38.069230+0000","last_clean":"2024-03-30T00:50:38.069230+0000","last_became_active":"2024-03-30T00:50:38.058129+0000","last_became_peered":"2024-03-30T00:50:38.058129+0000","last_unstale":"2024-03-30T00:50:38.069230+0000","last_undegraded":"2024-03-30T00:50:38.069230+0000","last_fullsized":"2024-03-30T00:50:38.069230+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.079073+0000","last_change":"2024-03-30T00:50:38.068666+0000","last_active":"2024-03-30T00:50:38.079073+0000","last_peered":"2024-03-30T00:50:38.079073+0000","last_clean":"2024-03-30T00:50:38.079073+0000","last_became_active":"2024-03-30T00:50:38.062588+0000","last_became_peered":"2024-03-30T00:50:38.062588+0000","last_unstale":"2024-03-30T00:50:38.079073+0000","last_undegraded":"2024-03-30T00:50:38.079073+0000","last_fullsized":"2024-03-30T00:50:38.079073+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:04:48.536583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.076512+0000","last_change":"2024-03-30T00:50:38.071642+0000","last_active":"2024-03-30T00:50:38.076512+0000","last_peered":"2024-03-30T00:50:38.076512+0000","last_clean":"2024-03-30T00:50:38.076512+0000","last_became_active":"2024-03-30T00:50:38.071366+0000","last_became_peered":"2024-03-30T00:50:38.071366+0000","last_unstale":"2024-03-30T00:50:38.076512+0000","last_undegraded":"2024-03-30T00:50:38.076512+0000","last_fullsized":"2024-03-30T00:50:38.076512+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:38.180300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.075810+0000","last_change":"2024-03-30T00:50:38.071639+0000","last_active":"2024-03-30T00:50:38.075810+0000","last_peered":"2024-03-30T00:50:38.075810+0000","last_clean":"2024-03-30T00:50:38.075810+0000","last_became_active":"2024-03-30T00:50:38.071402+0000","last_became_peered":"2024-03-30T00:50:38.071402+0000","last_unstale":"2024-03-30T00:50:38.075810+0000","last_undegraded":"2024-03-30T00:50:38.075810+0000","last_fullsized":"2024-03-30T00:50:38.075810+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:38:56.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.070446+0000","last_change":"2024-03-30T00:50:38.064697+0000","last_active":"2024-03-30T00:50:38.070446+0000","last_peered":"2024-03-30T00:50:38.070446+0000","last_clean":"2024-03-30T00:50:38.070446+0000","last_became_active":"2024-03-30T00:50:38.064454+0000","last_became_peered":"2024-03-30T00:50:38.064454+0000","last_unstale":"2024-03-30T00:50:38.070446+0000","last_undegraded":"2024-03-30T00:50:38.070446+0000","last_fullsized":"2024-03-30T00:50:38.070446+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.075075+0000","last_change":"2024-03-30T00:50:38.074133+0000","last_active":"2024-03-30T00:50:38.075075+0000","last_peered":"2024-03-30T00:50:38.075075+0000","last_clean":"2024-03-30T00:50:38.075075+0000","last_became_active":"2024-03-30T00:50:38.073562+0000","last_became_peered":"2024-03-30T00:50:38.073562+0000","last_unstale":"2024-03-30T00:50:38.075075+0000","last_undegraded":"2024-03-30T00:50:38.075075+0000","last_fullsized":"2024-03-30T00:50:38.075075+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077548+0000","last_change":"2024-03-30T00:50:38.075689+0000","last_active":"2024-03-30T00:50:38.077548+0000","last_peered":"2024-03-30T00:50:38.077548+0000","last_clean":"2024-03-30T00:50:38.077548+0000","last_became_active":"2024-03-30T00:50:38.074623+0000","last_became_peered":"2024-03-30T00:50:38.074623+0000","last_unstale":"2024-03-30T00:50:38.077548+0000","last_undegraded":"2024-03-30T00:50:38.077548+0000","last_fullsized":"2024-03-30T00:50:38.077548+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077203+0000","last_change":"2024-03-30T00:50:38.065650+0000","last_active":"2024-03-30T00:50:38.077203+0000","last_peered":"2024-03-30T00:50:38.077203+0000","last_clean":"2024-03-30T00:50:38.077203+0000","last_became_active":"2024-03-30T00:50:38.063669+0000","last_became_peered":"2024-03-30T00:50:38.063669+0000","last_unstale":"2024-03-30T00:50:38.077203+0000","last_undegraded":"2024-03-30T00:50:38.077203+0000","last_fullsized":"2024-03-30T00:50:38.077203+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.066049+0000","last_change":"2024-03-30T00:50:38.062347+0000","last_active":"2024-03-30T00:50:38.066049+0000","last_peered":"2024-03-30T00:50:38.066049+0000","last_clean":"2024-03-30T00:50:38.066049+0000","last_became_active":"2024-03-30T00:50:38.059824+0000","last_became_peered":"2024-03-30T00:50:38.059824+0000","last_unstale":"2024-03-30T00:50:38.066049+0000","last_undegraded":"2024-03-30T00:50:38.066049+0000","last_fullsized":"2024-03-30T00:50:38.066049+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.073858+0000","last_change":"2024-03-30T00:50:38.070365+0000","last_active":"2024-03-30T00:50:38.073858+0000","last_peered":"2024-03-30T00:50:38.073858+0000","last_clean":"2024-03-30T00:50:38.073858+0000","last_became_active":"2024-03-30T00:50:38.070088+0000","last_became_peered":"2024-03-30T00:50:38.070088+0000","last_unstale":"2024-03-30T00:50:38.073858+0000","last_undegraded":"2024-03-30T00:50:38.073858+0000","last_fullsized":"2024-03-30T00:50:38.073858+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:56.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.076536+0000","last_change":"2024-03-30T00:50:38.071702+0000","last_active":"2024-03-30T00:50:38.076536+0000","last_peered":"2024-03-30T00:50:38.076536+0000","last_clean":"2024-03-30T00:50:38.076536+0000","last_became_active":"2024-03-30T00:50:38.070983+0000","last_became_peered":"2024-03-30T00:50:38.070983+0000","last_unstale":"2024-03-30T00:50:38.076536+0000","last_undegraded":"2024-03-30T00:50:38.076536+0000","last_fullsized":"2024-03-30T00:50:38.076536+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:13.551635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.070738+0000","last_change":"2024-03-30T00:50:38.063288+0000","last_active":"2024-03-30T00:50:38.070738+0000","last_peered":"2024-03-30T00:50:38.070738+0000","last_clean":"2024-03-30T00:50:38.070738+0000","last_became_active":"2024-03-30T00:50:38.063009+0000","last_became_peered":"2024-03-30T00:50:38.063009+0000","last_unstale":"2024-03-30T00:50:38.070738+0000","last_undegraded":"2024-03-30T00:50:38.070738+0000","last_fullsized":"2024-03-30T00:50:38.070738+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:34:39.399906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.084922+0000","last_change":"2024-03-30T00:50:38.079992+0000","last_active":"2024-03-30T00:50:38.084922+0000","last_peered":"2024-03-30T00:50:38.084922+0000","last_clean":"2024-03-30T00:50:38.084922+0000","last_became_active":"2024-03-30T00:50:38.079708+0000","last_became_peered":"2024-03-30T00:50:38.079708+0000","last_unstale":"2024-03-30T00:50:38.084922+0000","last_undegraded":"2024-03-30T00:50:38.084922+0000","last_fullsized":"2024-03-30T00:50:38.084922+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:12.688382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.084984+0000","last_change":"2024-03-30T00:50:38.078067+0000","last_active":"2024-03-30T00:50:38.084984+0000","last_peered":"2024-03-30T00:50:38.084984+0000","last_clean":"2024-03-30T00:50:38.084984+0000","last_became_active":"2024-03-30T00:50:38.077173+0000","last_became_peered":"2024-03-30T00:50:38.077173+0000","last_unstale":"2024-03-30T00:50:38.084984+0000","last_undegraded":"2024-03-30T00:50:38.084984+0000","last_fullsized":"2024-03-30T00:50:38.084984+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.076768+0000","last_change":"2024-03-30T00:50:38.064028+0000","last_active":"2024-03-30T00:50:38.076768+0000","last_peered":"2024-03-30T00:50:38.076768+0000","last_clean":"2024-03-30T00:50:38.076768+0000","last_became_active":"2024-03-30T00:50:38.063697+0000","last_became_peered":"2024-03-30T00:50:38.063697+0000","last_unstale":"2024-03-30T00:50:38.076768+0000","last_undegraded":"2024-03-30T00:50:38.076768+0000","last_fullsized":"2024-03-30T00:50:38.076768+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:14.841469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077890+0000","last_change":"2024-03-30T00:50:38.069222+0000","last_active":"2024-03-30T00:50:38.077890+0000","last_peered":"2024-03-30T00:50:38.077890+0000","last_clean":"2024-03-30T00:50:38.077890+0000","last_became_active":"2024-03-30T00:50:38.067422+0000","last_became_peered":"2024-03-30T00:50:38.067422+0000","last_unstale":"2024-03-30T00:50:38.077890+0000","last_undegraded":"2024-03-30T00:50:38.077890+0000","last_fullsized":"2024-03-30T00:50:38.077890+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.071379+0000","last_change":"2024-03-30T00:50:38.066103+0000","last_active":"2024-03-30T00:50:38.071379+0000","last_peered":"2024-03-30T00:50:38.071379+0000","last_clean":"2024-03-30T00:50:38.071379+0000","last_became_active":"2024-03-30T00:50:38.065844+0000","last_became_peered":"2024-03-30T00:50:38.065844+0000","last_unstale":"2024-03-30T00:50:38.071379+0000","last_undegraded":"2024-03-30T00:50:38.071379+0000","last_fullsized":"2024-03-30T00:50:38.071379+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"124'4","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.074584+0000","last_change":"2024-03-30T00:50:37.658198+0000","last_active":"2024-03-30T00:50:38.074584+0000","last_peered":"2024-03-30T00:50:38.074584+0000","last_clean":"2024-03-30T00:50:38.074584+0000","last_became_active":"2024-03-30T00:50:37.657799+0000","last_became_peered":"2024-03-30T00:50:37.657799+0000","last_unstale":"2024-03-30T00:50:38.074584+0000","last_undegraded":"2024-03-30T00:50:38.074584+0000","last_fullsized":"2024-03-30T00:50:38.074584+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:55:09.373511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077076+0000","last_change":"2024-03-30T00:50:38.072044+0000","last_active":"2024-03-30T00:50:38.077076+0000","last_peered":"2024-03-30T00:50:38.077076+0000","last_clean":"2024-03-30T00:50:38.077076+0000","last_became_active":"2024-03-30T00:50:38.071778+0000","last_became_peered":"2024-03-30T00:50:38.071778+0000","last_unstale":"2024-03-30T00:50:38.077076+0000","last_undegraded":"2024-03-30T00:50:38.077076+0000","last_fullsized":"2024-03-30T00:50:38.077076+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:58.975555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.068189+0000","last_change":"2024-03-30T00:50:37.647732+0000","last_active":"2024-03-30T00:50:38.068189+0000","last_peered":"2024-03-30T00:50:38.068189+0000","last_clean":"2024-03-30T00:50:38.068189+0000","last_became_active":"2024-03-30T00:50:37.647438+0000","last_became_peered":"2024-03-30T00:50:37.647438+0000","last_unstale":"2024-03-30T00:50:38.068189+0000","last_undegraded":"2024-03-30T00:50:38.068189+0000","last_fullsized":"2024-03-30T00:50:38.068189+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:04.443031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078771+0000","last_change":"2024-03-30T00:50:38.078450+0000","last_active":"2024-03-30T00:50:38.078771+0000","last_peered":"2024-03-30T00:50:38.078771+0000","last_clean":"2024-03-30T00:50:38.078771+0000","last_became_active":"2024-03-30T00:50:38.078156+0000","last_became_peered":"2024-03-30T00:50:38.078156+0000","last_unstale":"2024-03-30T00:50:38.078771+0000","last_undegraded":"2024-03-30T00:50:38.078771+0000","last_fullsized":"2024-03-30T00:50:38.078771+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.080007+0000","last_change":"2024-03-30T00:50:37.651416+0000","last_active":"2024-03-30T00:50:38.080007+0000","last_peered":"2024-03-30T00:50:38.080007+0000","last_clean":"2024-03-30T00:50:38.080007+0000","last_became_active":"2024-03-30T00:50:37.651110+0000","last_became_peered":"2024-03-30T00:50:37.651110+0000","last_unstale":"2024-03-30T00:50:38.080007+0000","last_undegraded":"2024-03-30T00:50:38.080007+0000","last_fullsized":"2024-03-30T00:50:38.080007+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:04.443031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.073970+0000","last_change":"2024-03-30T00:50:38.071349+0000","last_active":"2024-03-30T00:50:38.073970+0000","last_peered":"2024-03-30T00:50:38.073970+0000","last_clean":"2024-03-30T00:50:38.073970+0000","last_became_active":"2024-03-30T00:50:38.071109+0000","last_became_peered":"2024-03-30T00:50:38.071109+0000","last_unstale":"2024-03-30T00:50:38.073970+0000","last_undegraded":"2024-03-30T00:50:38.073970+0000","last_fullsized":"2024-03-30T00:50:38.073970+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.069162+0000","last_change":"2024-03-30T00:50:37.652287+0000","last_active":"2024-03-30T00:50:38.069162+0000","last_peered":"2024-03-30T00:50:38.069162+0000","last_clean":"2024-03-30T00:50:38.069162+0000","last_became_active":"2024-03-30T00:50:37.652025+0000","last_became_peered":"2024-03-30T00:50:37.652025+0000","last_unstale":"2024-03-30T00:50:38.069162+0000","last_undegraded":"2024-03-30T00:50:38.069162+0000","last_fullsized":"2024-03-30T00:50:38.069162+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:37.177730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.070066+0000","last_change":"2024-03-30T00:50:38.061496+0000","last_active":"2024-03-30T00:50:38.070066+0000","last_peered":"2024-03-30T00:50:38.070066+0000","last_clean":"2024-03-30T00:50:38.070066+0000","last_became_active":"2024-03-30T00:50:38.061193+0000","last_became_peered":"2024-03-30T00:50:38.061193+0000","last_unstale":"2024-03-30T00:50:38.070066+0000","last_undegraded":"2024-03-30T00:50:38.070066+0000","last_fullsized":"2024-03-30T00:50:38.070066+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.075441+0000","last_change":"2024-03-30T00:50:37.655896+0000","last_active":"2024-03-30T00:50:38.075441+0000","last_peered":"2024-03-30T00:50:38.075441+0000","last_clean":"2024-03-30T00:50:38.075441+0000","last_became_active":"2024-03-30T00:50:37.655623+0000","last_became_peered":"2024-03-30T00:50:37.655623+0000","last_unstale":"2024-03-30T00:50:38.075441+0000","last_undegraded":"2024-03-30T00:50:38.075441+0000","last_fullsized":"2024-03-30T00:50:38.075441+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:29.599324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.060610+0000","last_change":"2024-03-30T00:50:38.057290+0000","last_active":"2024-03-30T00:50:38.060610+0000","last_peered":"2024-03-30T00:50:38.060610+0000","last_clean":"2024-03-30T00:50:38.060610+0000","last_became_active":"2024-03-30T00:50:38.056944+0000","last_became_peered":"2024-03-30T00:50:38.056944+0000","last_unstale":"2024-03-30T00:50:38.060610+0000","last_undegraded":"2024-03-30T00:50:38.060610+0000","last_fullsized":"2024-03-30T00:50:38.060610+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.071682+0000","last_change":"2024-03-30T00:50:37.657178+0000","last_active":"2024-03-30T00:50:38.071682+0000","last_peered":"2024-03-30T00:50:38.071682+0000","last_clean":"2024-03-30T00:50:38.071682+0000","last_became_active":"2024-03-30T00:50:37.656814+0000","last_became_peered":"2024-03-30T00:50:37.656814+0000","last_unstale":"2024-03-30T00:50:38.071682+0000","last_undegraded":"2024-03-30T00:50:38.071682+0000","last_fullsized":"2024-03-30T00:50:38.071682+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:11.406073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.082677+0000","last_change":"2024-03-30T00:50:38.077938+0000","last_active":"2024-03-30T00:50:38.082677+0000","last_peered":"2024-03-30T00:50:38.082677+0000","last_clean":"2024-03-30T00:50:38.082677+0000","last_became_active":"2024-03-30T00:50:38.077043+0000","last_became_peered":"2024-03-30T00:50:38.077043+0000","last_unstale":"2024-03-30T00:50:38.082677+0000","last_undegraded":"2024-03-30T00:50:38.082677+0000","last_fullsized":"2024-03-30T00:50:38.082677+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:51:44.264107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.056952+0000","last_change":"2024-03-30T00:50:37.654789+0000","last_active":"2024-03-30T00:50:38.056952+0000","last_peered":"2024-03-30T00:50:38.056952+0000","last_clean":"2024-03-30T00:50:38.056952+0000","last_became_active":"2024-03-30T00:50:37.654488+0000","last_became_peered":"2024-03-30T00:50:37.654488+0000","last_unstale":"2024-03-30T00:50:38.056952+0000","last_undegraded":"2024-03-30T00:50:38.056952+0000","last_fullsized":"2024-03-30T00:50:38.056952+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:41:51.590793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.084820+0000","last_change":"2024-03-30T00:50:38.079799+0000","last_active":"2024-03-30T00:50:38.084820+0000","last_peered":"2024-03-30T00:50:38.084820+0000","last_clean":"2024-03-30T00:50:38.084820+0000","last_became_active":"2024-03-30T00:50:38.079537+0000","last_became_peered":"2024-03-30T00:50:38.079537+0000","last_unstale":"2024-03-30T00:50:38.084820+0000","last_undegraded":"2024-03-30T00:50:38.084820+0000","last_fullsized":"2024-03-30T00:50:38.084820+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.059013+0000","last_change":"2024-03-30T00:50:37.658253+0000","last_active":"2024-03-30T00:50:38.059013+0000","last_peered":"2024-03-30T00:50:38.059013+0000","last_clean":"2024-03-30T00:50:38.059013+0000","last_became_active":"2024-03-30T00:50:37.657996+0000","last_became_peered":"2024-03-30T00:50:37.657996+0000","last_unstale":"2024-03-30T00:50:38.059013+0000","last_undegraded":"2024-03-30T00:50:38.059013+0000","last_fullsized":"2024-03-30T00:50:38.059013+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:51:25.787216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.075723+0000","last_change":"2024-03-30T00:50:38.071292+0000","last_active":"2024-03-30T00:50:38.075723+0000","last_peered":"2024-03-30T00:50:38.075723+0000","last_clean":"2024-03-30T00:50:38.075723+0000","last_became_active":"2024-03-30T00:50:38.070916+0000","last_became_peered":"2024-03-30T00:50:38.070916+0000","last_unstale":"2024-03-30T00:50:38.075723+0000","last_undegraded":"2024-03-30T00:50:38.075723+0000","last_fullsized":"2024-03-30T00:50:38.075723+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:01.855290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.055152+0000","last_change":"2024-03-30T00:50:37.659011+0000","last_active":"2024-03-30T00:50:38.055152+0000","last_peered":"2024-03-30T00:50:38.055152+0000","last_clean":"2024-03-30T00:50:38.055152+0000","last_became_active":"2024-03-30T00:50:37.658720+0000","last_became_peered":"2024-03-30T00:50:37.658720+0000","last_unstale":"2024-03-30T00:50:38.055152+0000","last_undegraded":"2024-03-30T00:50:38.055152+0000","last_fullsized":"2024-03-30T00:50:38.055152+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:57.692452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078459+0000","last_change":"2024-03-30T00:50:38.075750+0000","last_active":"2024-03-30T00:50:38.078459+0000","last_peered":"2024-03-30T00:50:38.078459+0000","last_clean":"2024-03-30T00:50:38.078459+0000","last_became_active":"2024-03-30T00:50:38.075443+0000","last_became_peered":"2024-03-30T00:50:38.075443+0000","last_unstale":"2024-03-30T00:50:38.078459+0000","last_undegraded":"2024-03-30T00:50:38.078459+0000","last_fullsized":"2024-03-30T00:50:38.078459+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.059165+0000","last_change":"2024-03-30T00:50:37.651177+0000","last_active":"2024-03-30T00:50:38.059165+0000","last_peered":"2024-03-30T00:50:38.059165+0000","last_clean":"2024-03-30T00:50:38.059165+0000","last_became_active":"2024-03-30T00:50:37.650902+0000","last_became_peered":"2024-03-30T00:50:37.650902+0000","last_unstale":"2024-03-30T00:50:38.059165+0000","last_undegraded":"2024-03-30T00:50:38.059165+0000","last_fullsized":"2024-03-30T00:50:38.059165+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:28.311412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.079090+0000","last_change":"2024-03-30T00:50:38.070389+0000","last_active":"2024-03-30T00:50:38.079090+0000","last_peered":"2024-03-30T00:50:38.079090+0000","last_clean":"2024-03-30T00:50:38.079090+0000","last_became_active":"2024-03-30T00:50:38.066504+0000","last_became_peered":"2024-03-30T00:50:38.066504+0000","last_unstale":"2024-03-30T00:50:38.079090+0000","last_undegraded":"2024-03-30T00:50:38.079090+0000","last_fullsized":"2024-03-30T00:50:38.079090+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"124'3","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078389+0000","last_change":"2024-03-30T00:50:37.653495+0000","last_active":"2024-03-30T00:50:38.078389+0000","last_peered":"2024-03-30T00:50:38.078389+0000","last_clean":"2024-03-30T00:50:38.078389+0000","last_became_active":"2024-03-30T00:50:37.652706+0000","last_became_peered":"2024-03-30T00:50:37.652706+0000","last_unstale":"2024-03-30T00:50:38.078389+0000","last_undegraded":"2024-03-30T00:50:38.078389+0000","last_fullsized":"2024-03-30T00:50:38.078389+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:07:01.780302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078191+0000","last_change":"2024-03-30T00:50:38.075296+0000","last_active":"2024-03-30T00:50:38.078191+0000","last_peered":"2024-03-30T00:50:38.078191+0000","last_clean":"2024-03-30T00:50:38.078191+0000","last_became_active":"2024-03-30T00:50:38.075035+0000","last_became_peered":"2024-03-30T00:50:38.075035+0000","last_unstale":"2024-03-30T00:50:38.078191+0000","last_undegraded":"2024-03-30T00:50:38.078191+0000","last_fullsized":"2024-03-30T00:50:38.078191+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:46:27.974997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"124'6","reported_seq":17,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.085335+0000","last_change":"2024-03-30T00:50:37.648270+0000","last_active":"2024-03-30T00:50:38.085335+0000","last_peered":"2024-03-30T00:50:38.085335+0000","last_clean":"2024-03-30T00:50:38.085335+0000","last_became_active":"2024-03-30T00:50:37.647989+0000","last_became_peered":"2024-03-30T00:50:37.647989+0000","last_unstale":"2024-03-30T00:50:38.085335+0000","last_undegraded":"2024-03-30T00:50:38.085335+0000","last_fullsized":"2024-03-30T00:50:38.085335+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:53.076893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078212+0000","last_change":"2024-03-30T00:50:38.076639+0000","last_active":"2024-03-30T00:50:38.078212+0000","last_peered":"2024-03-30T00:50:38.078212+0000","last_clean":"2024-03-30T00:50:38.078212+0000","last_became_active":"2024-03-30T00:50:38.075236+0000","last_became_peered":"2024-03-30T00:50:38.075236+0000","last_unstale":"2024-03-30T00:50:38.078212+0000","last_undegraded":"2024-03-30T00:50:38.078212+0000","last_fullsized":"2024-03-30T00:50:38.078212+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:05.445602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.084971+0000","last_change":"2024-03-30T00:50:37.655775+0000","last_active":"2024-03-30T00:50:38.084971+0000","last_peered":"2024-03-30T00:50:38.084971+0000","last_clean":"2024-03-30T00:50:38.084971+0000","last_became_active":"2024-03-30T00:50:37.655033+0000","last_became_peered":"2024-03-30T00:50:37.655033+0000","last_unstale":"2024-03-30T00:50:38.084971+0000","last_undegraded":"2024-03-30T00:50:38.084971+0000","last_fullsized":"2024-03-30T00:50:38.084971+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:46:26.972427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.068048+0000","last_change":"2024-03-30T00:50:38.052149+0000","last_active":"2024-03-30T00:50:38.068048+0000","last_peered":"2024-03-30T00:50:38.068048+0000","last_clean":"2024-03-30T00:50:38.068048+0000","last_became_active":"2024-03-30T00:50:38.051636+0000","last_became_peered":"2024-03-30T00:50:38.051636+0000","last_unstale":"2024-03-30T00:50:38.068048+0000","last_undegraded":"2024-03-30T00:50:38.068048+0000","last_fullsized":"2024-03-30T00:50:38.068048+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.056299+0000","last_change":"2024-03-30T00:50:37.657373+0000","last_active":"2024-03-30T00:50:38.056299+0000","last_peered":"2024-03-30T00:50:38.056299+0000","last_clean":"2024-03-30T00:50:38.056299+0000","last_became_active":"2024-03-30T00:50:37.657068+0000","last_became_peered":"2024-03-30T00:50:37.657068+0000","last_unstale":"2024-03-30T00:50:38.056299+0000","last_undegraded":"2024-03-30T00:50:38.056299+0000","last_fullsized":"2024-03-30T00:50:38.056299+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:36.963155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.067184+0000","last_change":"2024-03-30T00:50:38.063510+0000","last_active":"2024-03-30T00:50:38.067184+0000","last_peered":"2024-03-30T00:50:38.067184+0000","last_clean":"2024-03-30T00:50:38.067184+0000","last_became_active":"2024-03-30T00:50:38.063256+0000","last_became_peered":"2024-03-30T00:50:38.063256+0000","last_unstale":"2024-03-30T00:50:38.067184+0000","last_undegraded":"2024-03-30T00:50:38.067184+0000","last_fullsized":"2024-03-30T00:50:38.067184+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:02:07.606929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.060004+0000","last_change":"2024-03-30T00:50:37.653596+0000","last_active":"2024-03-30T00:50:38.060004+0000","last_peered":"2024-03-30T00:50:38.060004+0000","last_clean":"2024-03-30T00:50:38.060004+0000","last_became_active":"2024-03-30T00:50:37.653299+0000","last_became_peered":"2024-03-30T00:50:37.653299+0000","last_unstale":"2024-03-30T00:50:38.060004+0000","last_undegraded":"2024-03-30T00:50:38.060004+0000","last_fullsized":"2024-03-30T00:50:38.060004+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:06.142142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078221+0000","last_change":"2024-03-30T00:50:38.073132+0000","last_active":"2024-03-30T00:50:38.078221+0000","last_peered":"2024-03-30T00:50:38.078221+0000","last_clean":"2024-03-30T00:50:38.078221+0000","last_became_active":"2024-03-30T00:50:38.072860+0000","last_became_peered":"2024-03-30T00:50:38.072860+0000","last_unstale":"2024-03-30T00:50:38.078221+0000","last_undegraded":"2024-03-30T00:50:38.078221+0000","last_fullsized":"2024-03-30T00:50:38.078221+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.073471+0000","last_change":"2024-03-30T00:50:37.649310+0000","last_active":"2024-03-30T00:50:38.073471+0000","last_peered":"2024-03-30T00:50:38.073471+0000","last_clean":"2024-03-30T00:50:38.073471+0000","last_became_active":"2024-03-30T00:50:37.649015+0000","last_became_peered":"2024-03-30T00:50:37.649015+0000","last_unstale":"2024-03-30T00:50:38.073471+0000","last_undegraded":"2024-03-30T00:50:38.073471+0000","last_fullsized":"2024-03-30T00:50:38.073471+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:00.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.075632+0000","last_change":"2024-03-30T00:50:38.074524+0000","last_active":"2024-03-30T00:50:38.075632+0000","last_peered":"2024-03-30T00:50:38.075632+0000","last_clean":"2024-03-30T00:50:38.075632+0000","last_became_active":"2024-03-30T00:50:38.074266+0000","last_became_peered":"2024-03-30T00:50:38.074266+0000","last_unstale":"2024-03-30T00:50:38.075632+0000","last_undegraded":"2024-03-30T00:50:38.075632+0000","last_fullsized":"2024-03-30T00:50:38.075632+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.074473+0000","last_change":"2024-03-30T00:50:37.655900+0000","last_active":"2024-03-30T00:50:38.074473+0000","last_peered":"2024-03-30T00:50:38.074473+0000","last_clean":"2024-03-30T00:50:38.074473+0000","last_became_active":"2024-03-30T00:50:37.655601+0000","last_became_peered":"2024-03-30T00:50:37.655601+0000","last_unstale":"2024-03-30T00:50:38.074473+0000","last_undegraded":"2024-03-30T00:50:38.074473+0000","last_fullsized":"2024-03-30T00:50:38.074473+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:33.394770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.073937+0000","last_change":"2024-03-30T00:50:38.069971+0000","last_active":"2024-03-30T00:50:38.073937+0000","last_peered":"2024-03-30T00:50:38.073937+0000","last_clean":"2024-03-30T00:50:38.073937+0000","last_became_active":"2024-03-30T00:50:38.069640+0000","last_became_peered":"2024-03-30T00:50:38.069640+0000","last_unstale":"2024-03-30T00:50:38.073937+0000","last_undegraded":"2024-03-30T00:50:38.073937+0000","last_fullsized":"2024-03-30T00:50:38.073937+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:09:05.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078669+0000","last_change":"2024-03-30T00:50:37.649475+0000","last_active":"2024-03-30T00:50:38.078669+0000","last_peered":"2024-03-30T00:50:38.078669+0000","last_clean":"2024-03-30T00:50:38.078669+0000","last_became_active":"2024-03-30T00:50:37.649055+0000","last_became_peered":"2024-03-30T00:50:37.649055+0000","last_unstale":"2024-03-30T00:50:38.078669+0000","last_undegraded":"2024-03-30T00:50:38.078669+0000","last_fullsized":"2024-03-30T00:50:38.078669+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:12.549064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.066027+0000","last_change":"2024-03-30T00:50:38.060240+0000","last_active":"2024-03-30T00:50:38.066027+0000","last_peered":"2024-03-30T00:50:38.066027+0000","last_clean":"2024-03-30T00:50:38.066027+0000","last_became_active":"2024-03-30T00:50:38.059835+0000","last_became_peered":"2024-03-30T00:50:38.059835+0000","last_unstale":"2024-03-30T00:50:38.066027+0000","last_undegraded":"2024-03-30T00:50:38.066027+0000","last_fullsized":"2024-03-30T00:50:38.066027+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:47.211883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.050373+0000","last_change":"2024-03-30T00:50:37.656047+0000","last_active":"2024-03-30T00:50:38.050373+0000","last_peered":"2024-03-30T00:50:38.050373+0000","last_clean":"2024-03-30T00:50:38.050373+0000","last_became_active":"2024-03-30T00:50:37.655751+0000","last_became_peered":"2024-03-30T00:50:37.655751+0000","last_unstale":"2024-03-30T00:50:38.050373+0000","last_undegraded":"2024-03-30T00:50:38.050373+0000","last_fullsized":"2024-03-30T00:50:38.050373+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:00.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.071836+0000","last_change":"2024-03-30T00:50:38.067072+0000","last_active":"2024-03-30T00:50:38.071836+0000","last_peered":"2024-03-30T00:50:38.071836+0000","last_clean":"2024-03-30T00:50:38.071836+0000","last_became_active":"2024-03-30T00:50:38.066790+0000","last_became_peered":"2024-03-30T00:50:38.066790+0000","last_unstale":"2024-03-30T00:50:38.071836+0000","last_undegraded":"2024-03-30T00:50:38.071836+0000","last_fullsized":"2024-03-30T00:50:38.071836+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:52:46.481754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.068037+0000","last_change":"2024-03-30T00:50:37.646336+0000","last_active":"2024-03-30T00:50:38.068037+0000","last_peered":"2024-03-30T00:50:38.068037+0000","last_clean":"2024-03-30T00:50:38.068037+0000","last_became_active":"2024-03-30T00:50:37.646081+0000","last_became_peered":"2024-03-30T00:50:37.646081+0000","last_unstale":"2024-03-30T00:50:38.068037+0000","last_undegraded":"2024-03-30T00:50:38.068037+0000","last_fullsized":"2024-03-30T00:50:38.068037+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:40.726869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.082724+0000","last_change":"2024-03-30T00:50:38.078166+0000","last_active":"2024-03-30T00:50:38.082724+0000","last_peered":"2024-03-30T00:50:38.082724+0000","last_clean":"2024-03-30T00:50:38.082724+0000","last_became_active":"2024-03-30T00:50:38.077911+0000","last_became_peered":"2024-03-30T00:50:38.077911+0000","last_unstale":"2024-03-30T00:50:38.082724+0000","last_undegraded":"2024-03-30T00:50:38.082724+0000","last_fullsized":"2024-03-30T00:50:38.082724+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:04:19.985679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.051995+0000","last_change":"2024-03-30T00:50:37.648309+0000","last_active":"2024-03-30T00:50:38.051995+0000","last_peered":"2024-03-30T00:50:38.051995+0000","last_clean":"2024-03-30T00:50:38.051995+0000","last_became_active":"2024-03-30T00:50:37.646903+0000","last_became_peered":"2024-03-30T00:50:37.646903+0000","last_unstale":"2024-03-30T00:50:38.051995+0000","last_undegraded":"2024-03-30T00:50:38.051995+0000","last_fullsized":"2024-03-30T00:50:38.051995+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:20.839135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.082464+0000","last_change":"2024-03-30T00:50:38.076759+0000","last_active":"2024-03-30T00:50:38.082464+0000","last_peered":"2024-03-30T00:50:38.082464+0000","last_clean":"2024-03-30T00:50:38.082464+0000","last_became_active":"2024-03-30T00:50:38.068983+0000","last_became_peered":"2024-03-30T00:50:38.068983+0000","last_unstale":"2024-03-30T00:50:38.082464+0000","last_undegraded":"2024-03-30T00:50:38.082464+0000","last_fullsized":"2024-03-30T00:50:38.082464+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.060100+0000","last_change":"2024-03-30T00:50:37.656069+0000","last_active":"2024-03-30T00:50:38.060100+0000","last_peered":"2024-03-30T00:50:38.060100+0000","last_clean":"2024-03-30T00:50:38.060100+0000","last_became_active":"2024-03-30T00:50:37.655828+0000","last_became_peered":"2024-03-30T00:50:37.655828+0000","last_unstale":"2024-03-30T00:50:38.060100+0000","last_undegraded":"2024-03-30T00:50:38.060100+0000","last_fullsized":"2024-03-30T00:50:38.060100+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:46.209312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078150+0000","last_change":"2024-03-30T00:50:38.072920+0000","last_active":"2024-03-30T00:50:38.078150+0000","last_peered":"2024-03-30T00:50:38.078150+0000","last_clean":"2024-03-30T00:50:38.078150+0000","last_became_active":"2024-03-30T00:50:38.072673+0000","last_became_peered":"2024-03-30T00:50:38.072673+0000","last_unstale":"2024-03-30T00:50:38.078150+0000","last_undegraded":"2024-03-30T00:50:38.078150+0000","last_fullsized":"2024-03-30T00:50:38.078150+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:37.965725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.050094+0000","last_change":"2024-03-30T00:50:37.650090+0000","last_active":"2024-03-30T00:50:38.050094+0000","last_peered":"2024-03-30T00:50:38.050094+0000","last_clean":"2024-03-30T00:50:38.050094+0000","last_became_active":"2024-03-30T00:50:37.649527+0000","last_became_peered":"2024-03-30T00:50:37.649527+0000","last_unstale":"2024-03-30T00:50:38.050094+0000","last_undegraded":"2024-03-30T00:50:38.050094+0000","last_fullsized":"2024-03-30T00:50:38.050094+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:37.177730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077103+0000","last_change":"2024-03-30T00:50:38.072192+0000","last_active":"2024-03-30T00:50:38.077103+0000","last_peered":"2024-03-30T00:50:38.077103+0000","last_clean":"2024-03-30T00:50:38.077103+0000","last_became_active":"2024-03-30T00:50:38.071946+0000","last_became_peered":"2024-03-30T00:50:38.071946+0000","last_unstale":"2024-03-30T00:50:38.077103+0000","last_undegraded":"2024-03-30T00:50:38.077103+0000","last_fullsized":"2024-03-30T00:50:38.077103+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:19.197836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.072836+0000","last_change":"2024-03-30T00:50:37.648973+0000","last_active":"2024-03-30T00:50:38.072836+0000","last_peered":"2024-03-30T00:50:38.072836+0000","last_clean":"2024-03-30T00:50:38.072836+0000","last_became_active":"2024-03-30T00:50:37.648703+0000","last_became_peered":"2024-03-30T00:50:37.648703+0000","last_unstale":"2024-03-30T00:50:38.072836+0000","last_undegraded":"2024-03-30T00:50:38.072836+0000","last_fullsized":"2024-03-30T00:50:38.072836+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:53:36.133173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.070622+0000","last_change":"2024-03-30T00:50:38.061500+0000","last_active":"2024-03-30T00:50:38.070622+0000","last_peered":"2024-03-30T00:50:38.070622+0000","last_clean":"2024-03-30T00:50:38.070622+0000","last_became_active":"2024-03-30T00:50:38.061083+0000","last_became_peered":"2024-03-30T00:50:38.061083+0000","last_unstale":"2024-03-30T00:50:38.070622+0000","last_undegraded":"2024-03-30T00:50:38.070622+0000","last_fullsized":"2024-03-30T00:50:38.070622+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"125'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-03-30T00:50:39.058059+0000","last_change":"2024-03-30T00:50:37.658787+0000","last_active":"2024-03-30T00:50:39.058059+0000","last_peered":"2024-03-30T00:50:39.058059+0000","last_clean":"2024-03-30T00:50:39.058059+0000","last_became_active":"2024-03-30T00:50:37.658512+0000","last_became_peered":"2024-03-30T00:50:37.658512+0000","last_unstale":"2024-03-30T00:50:39.058059+0000","last_undegraded":"2024-03-30T00:50:39.058059+0000","last_fullsized":"2024-03-30T00:50:39.058059+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:55:31.799226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.059764+0000","last_change":"2024-03-30T00:50:38.056667+0000","last_active":"2024-03-30T00:50:38.059764+0000","last_peered":"2024-03-30T00:50:38.059764+0000","last_clean":"2024-03-30T00:50:38.059764+0000","last_became_active":"2024-03-30T00:50:38.056040+0000","last_became_peered":"2024-03-30T00:50:38.056040+0000","last_unstale":"2024-03-30T00:50:38.059764+0000","last_undegraded":"2024-03-30T00:50:38.059764+0000","last_fullsized":"2024-03-30T00:50:38.059764+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.057131+0000","last_change":"2024-03-30T00:50:37.656511+0000","last_active":"2024-03-30T00:50:38.057131+0000","last_peered":"2024-03-30T00:50:38.057131+0000","last_clean":"2024-03-30T00:50:38.057131+0000","last_became_active":"2024-03-30T00:50:37.655106+0000","last_became_peered":"2024-03-30T00:50:37.655106+0000","last_unstale":"2024-03-30T00:50:38.057131+0000","last_undegraded":"2024-03-30T00:50:38.057131+0000","last_fullsized":"2024-03-30T00:50:38.057131+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:04:28.343406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.079154+0000","last_change":"2024-03-30T00:50:38.076758+0000","last_active":"2024-03-30T00:50:38.079154+0000","last_peered":"2024-03-30T00:50:38.079154+0000","last_clean":"2024-03-30T00:50:38.079154+0000","last_became_active":"2024-03-30T00:50:38.076404+0000","last_became_peered":"2024-03-30T00:50:38.076404+0000","last_unstale":"2024-03-30T00:50:38.079154+0000","last_undegraded":"2024-03-30T00:50:38.079154+0000","last_fullsized":"2024-03-30T00:50:38.079154+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:34:31.524194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.059782+0000","last_change":"2024-03-30T00:50:37.655230+0000","last_active":"2024-03-30T00:50:38.059782+0000","last_peered":"2024-03-30T00:50:38.059782+0000","last_clean":"2024-03-30T00:50:38.059782+0000","last_became_active":"2024-03-30T00:50:37.654933+0000","last_became_peered":"2024-03-30T00:50:37.654933+0000","last_unstale":"2024-03-30T00:50:38.059782+0000","last_undegraded":"2024-03-30T00:50:38.059782+0000","last_fullsized":"2024-03-30T00:50:38.059782+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:59.607374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.076131+0000","last_change":"2024-03-30T00:50:38.071227+0000","last_active":"2024-03-30T00:50:38.076131+0000","last_peered":"2024-03-30T00:50:38.076131+0000","last_clean":"2024-03-30T00:50:38.076131+0000","last_became_active":"2024-03-30T00:50:38.068099+0000","last_became_peered":"2024-03-30T00:50:38.068099+0000","last_unstale":"2024-03-30T00:50:38.076131+0000","last_undegraded":"2024-03-30T00:50:38.076131+0000","last_fullsized":"2024-03-30T00:50:38.076131+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:52:14.331871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.052644+0000","last_change":"2024-03-30T00:50:37.647611+0000","last_active":"2024-03-30T00:50:38.052644+0000","last_peered":"2024-03-30T00:50:38.052644+0000","last_clean":"2024-03-30T00:50:38.052644+0000","last_became_active":"2024-03-30T00:50:37.647311+0000","last_became_peered":"2024-03-30T00:50:37.647311+0000","last_unstale":"2024-03-30T00:50:38.052644+0000","last_undegraded":"2024-03-30T00:50:38.052644+0000","last_fullsized":"2024-03-30T00:50:38.052644+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:21.467231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078739+0000","last_change":"2024-03-30T00:50:38.069404+0000","last_active":"2024-03-30T00:50:38.078739+0000","last_peered":"2024-03-30T00:50:38.078739+0000","last_clean":"2024-03-30T00:50:38.078739+0000","last_became_active":"2024-03-30T00:50:38.069128+0000","last_became_peered":"2024-03-30T00:50:38.069128+0000","last_unstale":"2024-03-30T00:50:38.078739+0000","last_undegraded":"2024-03-30T00:50:38.078739+0000","last_fullsized":"2024-03-30T00:50:38.078739+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.052427+0000","last_change":"2024-03-30T00:50:37.651832+0000","last_active":"2024-03-30T00:50:38.052427+0000","last_peered":"2024-03-30T00:50:38.052427+0000","last_clean":"2024-03-30T00:50:38.052427+0000","last_became_active":"2024-03-30T00:50:37.651529+0000","last_became_peered":"2024-03-30T00:50:37.651529+0000","last_unstale":"2024-03-30T00:50:38.052427+0000","last_undegraded":"2024-03-30T00:50:38.052427+0000","last_fullsized":"2024-03-30T00:50:38.052427+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:19.643626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.073053+0000","last_change":"2024-03-30T00:50:38.068952+0000","last_active":"2024-03-30T00:50:38.073053+0000","last_peered":"2024-03-30T00:50:38.073053+0000","last_clean":"2024-03-30T00:50:38.073053+0000","last_became_active":"2024-03-30T00:50:38.068683+0000","last_became_peered":"2024-03-30T00:50:38.068683+0000","last_unstale":"2024-03-30T00:50:38.073053+0000","last_undegraded":"2024-03-30T00:50:38.073053+0000","last_fullsized":"2024-03-30T00:50:38.073053+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.062791+0000","last_change":"2024-03-30T00:50:37.654139+0000","last_active":"2024-03-30T00:50:38.062791+0000","last_peered":"2024-03-30T00:50:38.062791+0000","last_clean":"2024-03-30T00:50:38.062791+0000","last_became_active":"2024-03-30T00:50:37.653825+0000","last_became_peered":"2024-03-30T00:50:37.653825+0000","last_unstale":"2024-03-30T00:50:38.062791+0000","last_undegraded":"2024-03-30T00:50:38.062791+0000","last_fullsized":"2024-03-30T00:50:38.062791+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:47.043924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.068100+0000","last_change":"2024-03-30T00:50:38.052861+0000","last_active":"2024-03-30T00:50:38.068100+0000","last_peered":"2024-03-30T00:50:38.068100+0000","last_clean":"2024-03-30T00:50:38.068100+0000","last_became_active":"2024-03-30T00:50:38.052613+0000","last_became_peered":"2024-03-30T00:50:38.052613+0000","last_unstale":"2024-03-30T00:50:38.068100+0000","last_undegraded":"2024-03-30T00:50:38.068100+0000","last_fullsized":"2024-03-30T00:50:38.068100+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.050185+0000","last_change":"2024-03-30T00:50:37.654797+0000","last_active":"2024-03-30T00:50:38.050185+0000","last_peered":"2024-03-30T00:50:38.050185+0000","last_clean":"2024-03-30T00:50:38.050185+0000","last_became_active":"2024-03-30T00:50:37.654506+0000","last_became_peered":"2024-03-30T00:50:37.654506+0000","last_unstale":"2024-03-30T00:50:38.050185+0000","last_undegraded":"2024-03-30T00:50:38.050185+0000","last_fullsized":"2024-03-30T00:50:38.050185+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:26.653343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.070465+0000","last_change":"2024-03-30T00:50:38.066537+0000","last_active":"2024-03-30T00:50:38.070465+0000","last_peered":"2024-03-30T00:50:38.070465+0000","last_clean":"2024-03-30T00:50:38.070465+0000","last_became_active":"2024-03-30T00:50:38.066299+0000","last_became_peered":"2024-03-30T00:50:38.066299+0000","last_unstale":"2024-03-30T00:50:38.070465+0000","last_undegraded":"2024-03-30T00:50:38.070465+0000","last_fullsized":"2024-03-30T00:50:38.070465+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"124'8","reported_seq":19,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.084998+0000","last_change":"2024-03-30T00:50:37.648683+0000","last_active":"2024-03-30T00:50:38.084998+0000","last_peered":"2024-03-30T00:50:38.084998+0000","last_clean":"2024-03-30T00:50:38.084998+0000","last_became_active":"2024-03-30T00:50:37.648391+0000","last_became_peered":"2024-03-30T00:50:37.648391+0000","last_unstale":"2024-03-30T00:50:38.084998+0000","last_undegraded":"2024-03-30T00:50:38.084998+0000","last_fullsized":"2024-03-30T00:50:38.084998+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:13.838899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":273,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.050328+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:50:38.050328+0000","last_peered":"2024-03-30T00:50:38.050328+0000","last_clean":"2024-03-30T00:50:38.050328+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:50:38.050328+0000","last_undegraded":"2024-03-30T00:50:38.050328+0000","last_fullsized":"2024-03-30T00:50:38.050328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574983,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33240,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338600,"statfs":{"total":96636764160,"available":96602726400,"internally_reserved":0,"allocated":2166784,"data_stored":1311133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":0.97599999999999998,"15min":1.0860000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.47399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84999999999999998,"5min":0.98299999999999998,"15min":1.149},"last":0.80400000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.503,"5min":0.45800000000000002,"15min":0.436},"max":{"1min":0.90800000000000003,"5min":1.03,"15min":1.03},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93000000000000005,"5min":0.95199999999999996,"15min":1.3029999999999999},"last":0.754}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.76600000000000001},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.45000000000000001},"max":{"1min":1.0129999999999999,"5min":1.085,"15min":1.6020000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.73099999999999998,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95399999999999996,"5min":1.034,"15min":1.6619999999999999},"last":0.89600000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.75900000000000001},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99099999999999999,"5min":0.99299999999999999,"15min":2.0150000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.80000000000000004},"min":{"1min":0.63800000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.9490000000000001},"last":0.86799999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.37},"max":{"1min":0.97299999999999998,"5min":1.2,"15min":1.2},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.42999999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.97899999999999998},"last":0.56999999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96299999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":0.99199999999999999,"15min":1.026},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.92700000000000005,"5min":0.94599999999999995,"15min":1.214},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89200000000000002,"5min":1.0189999999999999,"15min":1.0309999999999999},"last":0.63300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1560576,"data_stored":720571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71599999999999997,"15min":0.79800000000000004},"min":{"1min":0.437,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.85799999999999998,"5min":1.2969999999999999,"15min":4.0490000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.68799999999999994},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":1.278,"15min":1.47},"last":0.64300000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.438,"15min":0.36899999999999999},"max":{"1min":0.95799999999999996,"5min":1.204,"15min":4.3170000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.85299999999999998},"min":{"1min":0.58399999999999996,"5min":0.47199999999999998,"15min":0.42999999999999999},"max":{"1min":0.92900000000000005,"5min":1.1120000000000001,"15min":4.4160000000000004},"last":0.72799999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.745,"15min":0.84999999999999998},"min":{"1min":0.63400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98299999999999998,"5min":1.24,"15min":4.375},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.84299999999999997},"min":{"1min":0.51700000000000002,"5min":0.42999999999999999,"15min":0.38600000000000001},"max":{"1min":0.92500000000000004,"5min":1.3129999999999999,"15min":4.2960000000000003},"last":0.61899999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.86799999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0249999999999999,"5min":1.2390000000000001,"15min":4.3399999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.73199999999999998,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.49099999999999999,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.258,"15min":4.6020000000000003},"last":0.63}]},{"osd":4,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.39200000000000002,"5min":0.39100000000000001,"15min":0.35999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.1870000000000001},"last":0.499},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.73199999999999998},"min":{"1min":0.441,"5min":0.372,"15min":0.372},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.036},"last":0.67400000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.65700000000000003},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0720000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0109999999999999,"15min":1.101},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97499999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.45700000000000002},"max":{"1min":1.0860000000000001,"5min":1.103,"15min":1.103},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1564672,"data_stored":720149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":0.83299999999999996,"5min":0.97599999999999998,"15min":1.095},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68700000000000006,"15min":0.73199999999999998},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":1.0489999999999999,"15min":1.7569999999999999},"last":0.69699999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.71999999999999997},"min":{"1min":0.47899999999999998,"5min":0.46600000000000003,"15min":0.43099999999999999},"max":{"1min":0.86599999999999999,"5min":0.98199999999999998,"15min":1.7330000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70999999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42399999999999999},"max":{"1min":0.92400000000000004,"5min":0.98799999999999999,"15min":1.3260000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.44600000000000001},"max":{"1min":0.89900000000000002,"5min":1.131,"15min":1.131},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.45900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.877,"5min":1.0609999999999999,"15min":1.2789999999999999},"last":0.624}]},{"osd":3,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.75},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.89100000000000001,"5min":1.0740000000000001,"15min":1.7},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.96099999999999997,"5min":1.1160000000000001,"15min":1.181},"last":0.67500000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.60299999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.309},"max":{"1min":0.876,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69199999999999995,"15min":0.73399999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91200000000000003,"5min":0.97799999999999998,"15min":1.615},"last":0.748}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.84099999999999997,"5min":0.91100000000000003,"15min":0.97399999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.626,"15min":0.67300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.35199999999999998},"max":{"1min":0.84999999999999998,"5min":0.93700000000000006,"15min":1.569},"last":0.58399999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.71499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.377},"max":{"1min":0.85399999999999998,"5min":0.92200000000000004,"15min":1.0149999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88400000000000001,"5min":1.167,"15min":1.383},"last":0.76500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574983,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1568768,"data_stored":721553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.34899999999999998},"max":{"1min":0.92400000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.67400000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98599999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.72699999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":0.92600000000000005,"5min":0.95899999999999996,"15min":1.2909999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.40699999999999997},"max":{"1min":1.006,"5min":1.113,"15min":1.113},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.59199999999999997,"5min":0.48999999999999999,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.73899999999999999,"15min":0.75},"min":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.46700000000000003},"max":{"1min":0.94899999999999995,"5min":1.163,"15min":1.228},"last":0.749}]},{"osd":3,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73599999999999999,"15min":0.71699999999999997},"min":{"1min":0.501,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.27},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.76000000000000001},"min":{"1min":0.59099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1,"15min":1.1739999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.621,"15min":0.65800000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.89400000000000002,"15min":1.329},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66100000000000003,"15min":0.70399999999999996},"min":{"1min":0.57399999999999995,"5min":0.34799999999999998,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":0.99099999999999999,"15min":1.3819999999999999},"last":0.58099999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.65700000000000003},"min":{"1min":0.45700000000000002,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":1.2130000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.69699999999999995},"min":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.95799999999999996,"15min":1.3220000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.628,"15min":0.69099999999999995},"min":{"1min":0.46700000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":0.91500000000000004,"15min":1.6910000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.73399999999999999,"15min":0.78000000000000003},"min":{"1min":0.59699999999999998,"5min":0.49299999999999999,"15min":0.46600000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.5009999999999999},"last":0.85799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574984,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1572864,"data_stored":721587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.435,"15min":0.45500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.58699999999999997,"5min":0.748,"15min":1.0529999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.49099999999999999,"15min":0.51400000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.24199999999999999},"max":{"1min":0.63400000000000001,"5min":0.75,"15min":0.79100000000000004},"last":0.36599999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49399999999999999,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.80900000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63800000000000001,"5min":0.73399999999999999,"15min":1.2150000000000001},"last":0.40799999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.78300000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47299999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.46300000000000002},"max":{"1min":0.95699999999999996,"5min":1.1259999999999999,"15min":1.2470000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77100000000000002,"15min":0.77600000000000002},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.079,"15min":1.0860000000000001},"last":0.59099999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77100000000000002,"15min":0.77900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.51100000000000001},"max":{"1min":1.024,"5min":1.0369999999999999,"15min":1.125},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.77900000000000003,"15min":0.79100000000000004},"min":{"1min":0.628,"5min":0.52200000000000002,"15min":0.48599999999999999},"max":{"1min":0.86299999999999999,"5min":1.133,"15min":1.2390000000000001},"last":0.748}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77200000000000002,"15min":0.79700000000000004},"min":{"1min":0.59999999999999998,"5min":0.51400000000000001,"15min":0.499},"max":{"1min":0.90100000000000002,"5min":1.091,"15min":1.226},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.038,"5min":1.1120000000000001,"15min":1.167},"last":0.71399999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82999999999999996,"15min":0.84099999999999997},"min":{"1min":0.71399999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.123,"5min":1.1910000000000001,"15min":1.202},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83499999999999996,"15min":0.86299999999999999},"min":{"1min":0.57299999999999995,"5min":0.54900000000000004,"15min":0.497},"max":{"1min":1.095,"5min":1.1639999999999999,"15min":1.3160000000000001},"last":0.73099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574983,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1556480,"data_stored":719989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56899999999999995,"5min":0.70299999999999996,"15min":4.0999999999999996},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.48799999999999999,"15min":0.50600000000000001},"min":{"1min":0.40699999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83999999999999997,"5min":0.84099999999999997,"15min":1.9239999999999999},"last":0.42699999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32600000000000001},"max":{"1min":0.61799999999999999,"5min":0.81299999999999994,"15min":3.1139999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.52400000000000002,"15min":0.53400000000000003},"min":{"1min":0.40600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.748,"5min":0.752,"15min":2.4159999999999999},"last":0.48699999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.499,"15min":0.497},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70099999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.749,"5min":0.88800000000000001,"15min":1.907},"last":0.45300000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77700000000000002,"15min":0.80100000000000005},"min":{"1min":0.59699999999999998,"5min":0.504,"15min":0.504},"max":{"1min":1.0900000000000001,"5min":1.1919999999999999,"15min":1.9470000000000001},"last":0.628},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78700000000000003,"15min":0.82099999999999995},"min":{"1min":0.628,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.163,"5min":1.2110000000000001,"15min":1.972},"last":0.73899999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.1220000000000001,"5min":1.2549999999999999,"15min":2.4569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.78900000000000003,"15min":0.81000000000000005},"min":{"1min":0.58399999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0389999999999999,"5min":1.286,"15min":2.8039999999999998},"last":0.60899999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0980000000000001,"5min":1.177,"15min":2.8849999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.0780000000000001,"5min":1.242,"15min":1.867},"last":0.67500000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79500000000000004,"15min":0.83699999999999997},"min":{"1min":0.60199999999999998,"5min":0.54700000000000004,"15min":0.495},"max":{"1min":1.0429999999999999,"5min":1.1140000000000001,"15min":2.3690000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.83799999999999997},"min":{"1min":0.70499999999999996,"5min":0.55800000000000005,"15min":0.52700000000000002},"max":{"1min":0.98999999999999999,"5min":1.365,"15min":2.851},"last":0.65500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1560576,"data_stored":720011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44,"15min":0.42799999999999999},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.253},"max":{"1min":0.63600000000000001,"5min":0.72199999999999998,"15min":0.80300000000000005},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.48399999999999999,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.69999999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49399999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.45900000000000002},"min":{"1min":0.39200000000000002,"5min":0.29999999999999999,"15min":0.29199999999999998},"max":{"1min":0.59199999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.64600000000000002,"5min":0.84099999999999997,"15min":0.89400000000000002},"last":0.52100000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.45300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.69199999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.38600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72999999999999998,"5min":0.83199999999999996,"15min":0.83999999999999997},"last":0.63300000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.75600000000000001,"15min":0.72899999999999998},"min":{"1min":0.58099999999999996,"5min":0.49199999999999999,"15min":0.38800000000000001},"max":{"1min":1.034,"5min":1.175,"15min":1.175},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76900000000000002,"15min":0.754},"min":{"1min":0.66300000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98599999999999999,"5min":1.079,"15min":1.2},"last":0.82999999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.76000000000000001},"min":{"1min":0.621,"5min":0.56799999999999995,"15min":0.51600000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.3180000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79400000000000004,"15min":0.77100000000000002},"min":{"1min":0.60299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.129,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.78600000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79700000000000004,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.0389999999999999,"5min":1.179,"15min":1.179},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0509999999999999,"15min":1.2150000000000001},"last":0.86699999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80300000000000005,"15min":0.79700000000000004},"min":{"1min":0.64500000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.024,"5min":1.2050000000000001,"15min":1.2669999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.81799999999999995,"15min":0.79200000000000004},"min":{"1min":0.69299999999999995,"5min":0.56200000000000006,"15min":0.48499999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.236},"last":0.85199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574984,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33220,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338620,"statfs":{"total":96636764160,"available":96602746880,"internally_reserved":0,"allocated":2154496,"data_stored":1310379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.45700000000000002},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.245},"max":{"1min":0.69799999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68400000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.51000000000000001},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.67100000000000004,"5min":0.82599999999999996,"15min":1.02},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.432,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":0.77900000000000003,"15min":1.006},"last":0.42399999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.68000000000000005,"5min":0.754,"15min":0.84799999999999998},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.46200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":0.88900000000000001,"15min":0.96899999999999997},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.47899999999999998},"max":{"1min":1.1040000000000001,"5min":1.1140000000000001,"15min":1.5169999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.46400000000000002},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.252},"last":0.69799999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.55200000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.3080000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0780000000000001,"15min":1.462},"last":0.57899999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79200000000000004,"15min":0.79900000000000004},"min":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.54500000000000004},"max":{"1min":1.0429999999999999,"5min":1.131,"15min":1.5920000000000001},"last":0.626},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80000000000000004,"15min":0.79000000000000004},"min":{"1min":0.625,"5min":0.54600000000000004,"15min":0.46999999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.2230000000000001},"last":0.59699999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.81299999999999994},"min":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.3600000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.60399999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.556},"last":0.64300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:50:46.725 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:50:46.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6beae00640 1 Processor -- start 2024-03-30T00:50:46.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6beae00640 1 -- start start 2024-03-30T00:50:46.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6beae00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6be415c9f0 0x7f6be406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:46.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6beae00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6be40b2b60 0x7f6be406b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:46.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6beae00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be40ce070 con 0x7f6be406b8b0 2024-03-30T00:50:46.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6beae00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be406bd60 con 0x7f6be40b2b60 2024-03-30T00:50:46.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6beae00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be406cf60 con 0x7f6be415c9f0 2024-03-30T00:50:46.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6be9000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6be40b2b60 0x7f6be406b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:46.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6be9000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6be40b2b60 0x7f6be406b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58940/0 (socket says 172.21.15.66:58940) 2024-03-30T00:50:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6be9000640 1 -- 172.21.15.66:0/3562731402 learned_addr learned my addr 172.21.15.66:0/3562731402 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6be9a00640 1 --2- 172.21.15.66:0/3562731402 >> v2:172.21.15.156:3300/0 conn(0x7f6be415c9f0 0x7f6be406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6be9000640 1 -- 172.21.15.66:0/3562731402 >> v2:172.21.15.156:3300/0 conn(0x7f6be415c9f0 msgr2=0x7f6be406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6be9000640 1 --2- 172.21.15.66:0/3562731402 >> v2:172.21.15.156:3300/0 conn(0x7f6be415c9f0 0x7f6be406ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6be9000640 1 -- 172.21.15.66:0/3562731402 >> v1:172.21.15.156:6789/0 conn(0x7f6be406b8b0 legacy=0x7f6be4160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:46.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.871+0000 7f6be9000640 1 -- 172.21.15.66:0/3562731402 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be40ce350 con 0x7f6be40b2b60 2024-03-30T00:50:46.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6be9000640 1 --2- 172.21.15.66:0/3562731402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be40b2b60 0x7f6be406b370 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f6be406d9d0 tx=0x7f6bd001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9d94c534b13044 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:46.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6be3e00640 1 -- 172.21.15.66:0/3562731402 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd00aa610 con 0x7f6be40b2b60 2024-03-30T00:50:46.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6be3e00640 1 -- 172.21.15.66:0/3562731402 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd00a6040 con 0x7f6be40b2b60 2024-03-30T00:50:46.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6be3e00640 1 -- 172.21.15.66:0/3562731402 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd00aa990 con 0x7f6be40b2b60 2024-03-30T00:50:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6beae00640 1 -- 172.21.15.66:0/3562731402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be40b2b60 msgr2=0x7f6be406b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6beae00640 1 --2- 172.21.15.66:0/3562731402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be40b2b60 0x7f6be406b370 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f6be406d9d0 tx=0x7f6bd001c4b0 comp rx=0 tx=0).stop 2024-03-30T00:50:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6beae00640 1 -- 172.21.15.66:0/3562731402 shutdown_connections 2024-03-30T00:50:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6beae00640 1 --2- 172.21.15.66:0/3562731402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be40b2b60 0x7f6be406b370 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6beae00640 1 --2- 172.21.15.66:0/3562731402 >> v2:172.21.15.156:3300/0 conn(0x7f6be415c9f0 0x7f6be406ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6beae00640 1 -- 172.21.15.66:0/3562731402 >> 172.21.15.66:0/3562731402 conn(0x7f6be406e0d0 msgr2=0x7f6be406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6beae00640 1 -- 172.21.15.66:0/3562731402 shutdown_connections 2024-03-30T00:50:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6beae00640 1 -- 172.21.15.66:0/3562731402 wait complete. 2024-03-30T00:50:46.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6beae00640 1 Processor -- start 2024-03-30T00:50:46.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.875+0000 7f6beae00640 1 -- start start 2024-03-30T00:50:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6beae00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6be406b8b0 0x7f6be40bee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6beae00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6be40b2b60 0x7f6be40b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6beae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be415c9f0 0x7f6be40b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6beae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be40bf7e0 con 0x7f6be415c9f0 2024-03-30T00:50:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6beae00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be40bf390 con 0x7f6be406b8b0 2024-03-30T00:50:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6beae00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be40bfa80 con 0x7f6be40b2b60 2024-03-30T00:50:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6bea400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be415c9f0 0x7f6be40b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6bea400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be415c9f0 0x7f6be40b57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58948/0 (socket says 172.21.15.66:58948) 2024-03-30T00:50:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6bea400640 1 -- 172.21.15.66:0/3368312255 learned_addr learned my addr 172.21.15.66:0/3368312255 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6be9a00640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6be406b8b0 0x7f6be40bee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6be9000640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6be40b2b60 0x7f6be40b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6bea400640 1 -- 172.21.15.66:0/3368312255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6be40b2b60 msgr2=0x7f6be40b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6bea400640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6be40b2b60 0x7f6be40b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6bea400640 1 -- 172.21.15.66:0/3368312255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6be406b8b0 msgr2=0x7f6be40bee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6bea400640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6be406b8b0 0x7f6be40bee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6bea400640 1 -- 172.21.15.66:0/3368312255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd00094d0 con 0x7f6be415c9f0 2024-03-30T00:50:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6bea400640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be415c9f0 0x7f6be40b57d0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00c780 tx=0x7f6bdc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6be1600640 1 -- 172.21.15.66:0/3368312255 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc004b70 con 0x7f6be415c9f0 2024-03-30T00:50:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6be1600640 1 -- 172.21.15.66:0/3368312255 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc00ce90 con 0x7f6be415c9f0 2024-03-30T00:50:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6be1600640 1 -- 172.21.15.66:0/3368312255 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00a850 con 0x7f6be415c9f0 2024-03-30T00:50:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6beae00640 1 -- 172.21.15.66:0/3368312255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be40bfd60 con 0x7f6be415c9f0 2024-03-30T00:50:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6beae00640 1 -- 172.21.15.66:0/3368312255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be40c1ea0 con 0x7f6be415c9f0 2024-03-30T00:50:46.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.879+0000 7f6beae00640 1 -- 172.21.15.66:0/3368312255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be40c23f0 con 0x7f6be415c9f0 2024-03-30T00:50:46.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.883+0000 7f6be1600640 1 -- 172.21.15.66:0/3368312255 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6bdc00a9b0 con 0x7f6be415c9f0 2024-03-30T00:50:46.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.883+0000 7f6be1600640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bc4066690 0x7f6bc4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:46.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.883+0000 7f6be1600640 1 -- 172.21.15.66:0/3368312255 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6bdc084420 con 0x7f6be415c9f0 2024-03-30T00:50:46.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.883+0000 7f6be9a00640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bc4066690 0x7f6bc4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:46.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.883+0000 7f6be9a00640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bc4066690 0x7f6bc4068b50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6be40ca510 tx=0x7f6bd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:46.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:46.887+0000 7f6be1600640 1 -- 172.21.15.66:0/3368312255 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6bdc051660 con 0x7f6be415c9f0 2024-03-30T00:50:47.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.047+0000 7f6beae00640 1 -- 172.21.15.66:0/3368312255 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6be40ceed0 con 0x7f6bc4066690 2024-03-30T00:50:47.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.055+0000 7f6be1600640 1 -- 172.21.15.66:0/3368312255 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289664 (secure 0 0 0) 0x7f6bd80108d0 con 0x7f6bc4066690 2024-03-30T00:50:47.065 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:47.066 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 -- 172.21.15.66:0/3368312255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bc4066690 msgr2=0x7f6bc4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bc4066690 0x7f6bc4068b50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6be40ca510 tx=0x7f6bd8008040 comp rx=0 tx=0).stop 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 -- 172.21.15.66:0/3368312255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be415c9f0 msgr2=0x7f6be40b57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be415c9f0 0x7f6be40b57d0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00c780 tx=0x7f6bdc00cc40 comp rx=0 tx=0).stop 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 -- 172.21.15.66:0/3368312255 shutdown_connections 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6bc4066690 0x7f6bc4068b50 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be415c9f0 0x7f6be40b57d0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6be40b2b60 0x7f6be40b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 --2- 172.21.15.66:0/3368312255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6be406b8b0 0x7f6be40bee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 -- 172.21.15.66:0/3368312255 >> 172.21.15.66:0/3368312255 conn(0x7f6be406e0d0 msgr2=0x7f6be406e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 -- 172.21.15.66:0/3368312255 shutdown_connections 2024-03-30T00:50:47.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.059+0000 7f6beae00640 1 -- 172.21.15.66:0/3368312255 wait complete. 2024-03-30T00:50:47.085 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-03-30T00:50:47.001414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262356,"kb_used_data":13384,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712364,"statfs":{"total":773094113280,"available":772825460736,"internally_reserved":0,"allocated":13705216,"data_stored":6945372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.958245"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.065924+0000","last_change":"2024-03-30T00:50:38.058216+0000","last_active":"2024-03-30T00:50:38.065924+0000","last_peered":"2024-03-30T00:50:38.065924+0000","last_clean":"2024-03-30T00:50:38.065924+0000","last_became_active":"2024-03-30T00:50:38.057791+0000","last_became_peered":"2024-03-30T00:50:38.057791+0000","last_unstale":"2024-03-30T00:50:38.065924+0000","last_undegraded":"2024-03-30T00:50:38.065924+0000","last_fullsized":"2024-03-30T00:50:38.065924+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.079054+0000","last_change":"2024-03-30T00:50:38.068156+0000","last_active":"2024-03-30T00:50:38.079054+0000","last_peered":"2024-03-30T00:50:38.079054+0000","last_clean":"2024-03-30T00:50:38.079054+0000","last_became_active":"2024-03-30T00:50:38.062931+0000","last_became_peered":"2024-03-30T00:50:38.062931+0000","last_unstale":"2024-03-30T00:50:38.079054+0000","last_undegraded":"2024-03-30T00:50:38.079054+0000","last_fullsized":"2024-03-30T00:50:38.079054+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.079360+0000","last_change":"2024-03-30T00:50:38.069777+0000","last_active":"2024-03-30T00:50:38.079360+0000","last_peered":"2024-03-30T00:50:38.079360+0000","last_clean":"2024-03-30T00:50:38.079360+0000","last_became_active":"2024-03-30T00:50:38.069461+0000","last_became_peered":"2024-03-30T00:50:38.069461+0000","last_unstale":"2024-03-30T00:50:38.079360+0000","last_undegraded":"2024-03-30T00:50:38.079360+0000","last_fullsized":"2024-03-30T00:50:38.079360+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:53:58.506502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.057834+0000","last_change":"2024-03-30T00:50:38.056037+0000","last_active":"2024-03-30T00:50:38.057834+0000","last_peered":"2024-03-30T00:50:38.057834+0000","last_clean":"2024-03-30T00:50:38.057834+0000","last_became_active":"2024-03-30T00:50:38.055796+0000","last_became_peered":"2024-03-30T00:50:38.055796+0000","last_unstale":"2024-03-30T00:50:38.057834+0000","last_undegraded":"2024-03-30T00:50:38.057834+0000","last_fullsized":"2024-03-30T00:50:38.057834+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.068304+0000","last_change":"2024-03-30T00:50:38.064340+0000","last_active":"2024-03-30T00:50:38.068304+0000","last_peered":"2024-03-30T00:50:38.068304+0000","last_clean":"2024-03-30T00:50:38.068304+0000","last_became_active":"2024-03-30T00:50:38.064079+0000","last_became_peered":"2024-03-30T00:50:38.064079+0000","last_unstale":"2024-03-30T00:50:38.068304+0000","last_undegraded":"2024-03-30T00:50:38.068304+0000","last_fullsized":"2024-03-30T00:50:38.068304+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.081030+0000","last_change":"2024-03-30T00:50:38.078616+0000","last_active":"2024-03-30T00:50:38.081030+0000","last_peered":"2024-03-30T00:50:38.081030+0000","last_clean":"2024-03-30T00:50:38.081030+0000","last_became_active":"2024-03-30T00:50:38.077950+0000","last_became_peered":"2024-03-30T00:50:38.077950+0000","last_unstale":"2024-03-30T00:50:38.081030+0000","last_undegraded":"2024-03-30T00:50:38.081030+0000","last_fullsized":"2024-03-30T00:50:38.081030+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:16.922513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.072189+0000","last_change":"2024-03-30T00:50:38.067925+0000","last_active":"2024-03-30T00:50:38.072189+0000","last_peered":"2024-03-30T00:50:38.072189+0000","last_clean":"2024-03-30T00:50:38.072189+0000","last_became_active":"2024-03-30T00:50:38.067684+0000","last_became_peered":"2024-03-30T00:50:38.067684+0000","last_unstale":"2024-03-30T00:50:38.072189+0000","last_undegraded":"2024-03-30T00:50:38.072189+0000","last_fullsized":"2024-03-30T00:50:38.072189+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:10:54.265531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.085039+0000","last_change":"2024-03-30T00:50:38.080042+0000","last_active":"2024-03-30T00:50:38.085039+0000","last_peered":"2024-03-30T00:50:38.085039+0000","last_clean":"2024-03-30T00:50:38.085039+0000","last_became_active":"2024-03-30T00:50:38.079814+0000","last_became_peered":"2024-03-30T00:50:38.079814+0000","last_unstale":"2024-03-30T00:50:38.085039+0000","last_undegraded":"2024-03-30T00:50:38.085039+0000","last_fullsized":"2024-03-30T00:50:38.085039+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077355+0000","last_change":"2024-03-30T00:50:38.074264+0000","last_active":"2024-03-30T00:50:38.077355+0000","last_peered":"2024-03-30T00:50:38.077355+0000","last_clean":"2024-03-30T00:50:38.077355+0000","last_became_active":"2024-03-30T00:50:38.073974+0000","last_became_peered":"2024-03-30T00:50:38.073974+0000","last_unstale":"2024-03-30T00:50:38.077355+0000","last_undegraded":"2024-03-30T00:50:38.077355+0000","last_fullsized":"2024-03-30T00:50:38.077355+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.076619+0000","last_change":"2024-03-30T00:50:38.075046+0000","last_active":"2024-03-30T00:50:38.076619+0000","last_peered":"2024-03-30T00:50:38.076619+0000","last_clean":"2024-03-30T00:50:38.076619+0000","last_became_active":"2024-03-30T00:50:38.074491+0000","last_became_peered":"2024-03-30T00:50:38.074491+0000","last_unstale":"2024-03-30T00:50:38.076619+0000","last_undegraded":"2024-03-30T00:50:38.076619+0000","last_fullsized":"2024-03-30T00:50:38.076619+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:21:22.322038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.069215+0000","last_change":"2024-03-30T00:50:38.065503+0000","last_active":"2024-03-30T00:50:38.069215+0000","last_peered":"2024-03-30T00:50:38.069215+0000","last_clean":"2024-03-30T00:50:38.069215+0000","last_became_active":"2024-03-30T00:50:38.065214+0000","last_became_peered":"2024-03-30T00:50:38.065214+0000","last_unstale":"2024-03-30T00:50:38.069215+0000","last_undegraded":"2024-03-30T00:50:38.069215+0000","last_fullsized":"2024-03-30T00:50:38.069215+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.072621+0000","last_change":"2024-03-30T00:50:38.063810+0000","last_active":"2024-03-30T00:50:38.072621+0000","last_peered":"2024-03-30T00:50:38.072621+0000","last_clean":"2024-03-30T00:50:38.072621+0000","last_became_active":"2024-03-30T00:50:38.063556+0000","last_became_peered":"2024-03-30T00:50:38.063556+0000","last_unstale":"2024-03-30T00:50:38.072621+0000","last_undegraded":"2024-03-30T00:50:38.072621+0000","last_fullsized":"2024-03-30T00:50:38.072621+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:09:58.609620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.069764+0000","last_change":"2024-03-30T00:50:38.060007+0000","last_active":"2024-03-30T00:50:38.069764+0000","last_peered":"2024-03-30T00:50:38.069764+0000","last_clean":"2024-03-30T00:50:38.069764+0000","last_became_active":"2024-03-30T00:50:38.059642+0000","last_became_peered":"2024-03-30T00:50:38.059642+0000","last_unstale":"2024-03-30T00:50:38.069764+0000","last_undegraded":"2024-03-30T00:50:38.069764+0000","last_fullsized":"2024-03-30T00:50:38.069764+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.082658+0000","last_change":"2024-03-30T00:50:38.077900+0000","last_active":"2024-03-30T00:50:38.082658+0000","last_peered":"2024-03-30T00:50:38.082658+0000","last_clean":"2024-03-30T00:50:38.082658+0000","last_became_active":"2024-03-30T00:50:38.076312+0000","last_became_peered":"2024-03-30T00:50:38.076312+0000","last_unstale":"2024-03-30T00:50:38.082658+0000","last_undegraded":"2024-03-30T00:50:38.082658+0000","last_fullsized":"2024-03-30T00:50:38.082658+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:44.859983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.058370+0000","last_change":"2024-03-30T00:50:38.056313+0000","last_active":"2024-03-30T00:50:38.058370+0000","last_peered":"2024-03-30T00:50:38.058370+0000","last_clean":"2024-03-30T00:50:38.058370+0000","last_became_active":"2024-03-30T00:50:38.056078+0000","last_became_peered":"2024-03-30T00:50:38.056078+0000","last_unstale":"2024-03-30T00:50:38.058370+0000","last_undegraded":"2024-03-30T00:50:38.058370+0000","last_fullsized":"2024-03-30T00:50:38.058370+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.071819+0000","last_change":"2024-03-30T00:50:38.066213+0000","last_active":"2024-03-30T00:50:38.071819+0000","last_peered":"2024-03-30T00:50:38.071819+0000","last_clean":"2024-03-30T00:50:38.071819+0000","last_became_active":"2024-03-30T00:50:38.065918+0000","last_became_peered":"2024-03-30T00:50:38.065918+0000","last_unstale":"2024-03-30T00:50:38.071819+0000","last_undegraded":"2024-03-30T00:50:38.071819+0000","last_fullsized":"2024-03-30T00:50:38.071819+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:46.573525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.069230+0000","last_change":"2024-03-30T00:50:38.058385+0000","last_active":"2024-03-30T00:50:38.069230+0000","last_peered":"2024-03-30T00:50:38.069230+0000","last_clean":"2024-03-30T00:50:38.069230+0000","last_became_active":"2024-03-30T00:50:38.058129+0000","last_became_peered":"2024-03-30T00:50:38.058129+0000","last_unstale":"2024-03-30T00:50:38.069230+0000","last_undegraded":"2024-03-30T00:50:38.069230+0000","last_fullsized":"2024-03-30T00:50:38.069230+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.079073+0000","last_change":"2024-03-30T00:50:38.068666+0000","last_active":"2024-03-30T00:50:38.079073+0000","last_peered":"2024-03-30T00:50:38.079073+0000","last_clean":"2024-03-30T00:50:38.079073+0000","last_became_active":"2024-03-30T00:50:38.062588+0000","last_became_peered":"2024-03-30T00:50:38.062588+0000","last_unstale":"2024-03-30T00:50:38.079073+0000","last_undegraded":"2024-03-30T00:50:38.079073+0000","last_fullsized":"2024-03-30T00:50:38.079073+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:04:48.536583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.076512+0000","last_change":"2024-03-30T00:50:38.071642+0000","last_active":"2024-03-30T00:50:38.076512+0000","last_peered":"2024-03-30T00:50:38.076512+0000","last_clean":"2024-03-30T00:50:38.076512+0000","last_became_active":"2024-03-30T00:50:38.071366+0000","last_became_peered":"2024-03-30T00:50:38.071366+0000","last_unstale":"2024-03-30T00:50:38.076512+0000","last_undegraded":"2024-03-30T00:50:38.076512+0000","last_fullsized":"2024-03-30T00:50:38.076512+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:38.180300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.075810+0000","last_change":"2024-03-30T00:50:38.071639+0000","last_active":"2024-03-30T00:50:38.075810+0000","last_peered":"2024-03-30T00:50:38.075810+0000","last_clean":"2024-03-30T00:50:38.075810+0000","last_became_active":"2024-03-30T00:50:38.071402+0000","last_became_peered":"2024-03-30T00:50:38.071402+0000","last_unstale":"2024-03-30T00:50:38.075810+0000","last_undegraded":"2024-03-30T00:50:38.075810+0000","last_fullsized":"2024-03-30T00:50:38.075810+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:38:56.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.070446+0000","last_change":"2024-03-30T00:50:38.064697+0000","last_active":"2024-03-30T00:50:38.070446+0000","last_peered":"2024-03-30T00:50:38.070446+0000","last_clean":"2024-03-30T00:50:38.070446+0000","last_became_active":"2024-03-30T00:50:38.064454+0000","last_became_peered":"2024-03-30T00:50:38.064454+0000","last_unstale":"2024-03-30T00:50:38.070446+0000","last_undegraded":"2024-03-30T00:50:38.070446+0000","last_fullsized":"2024-03-30T00:50:38.070446+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.075075+0000","last_change":"2024-03-30T00:50:38.074133+0000","last_active":"2024-03-30T00:50:38.075075+0000","last_peered":"2024-03-30T00:50:38.075075+0000","last_clean":"2024-03-30T00:50:38.075075+0000","last_became_active":"2024-03-30T00:50:38.073562+0000","last_became_peered":"2024-03-30T00:50:38.073562+0000","last_unstale":"2024-03-30T00:50:38.075075+0000","last_undegraded":"2024-03-30T00:50:38.075075+0000","last_fullsized":"2024-03-30T00:50:38.075075+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077548+0000","last_change":"2024-03-30T00:50:38.075689+0000","last_active":"2024-03-30T00:50:38.077548+0000","last_peered":"2024-03-30T00:50:38.077548+0000","last_clean":"2024-03-30T00:50:38.077548+0000","last_became_active":"2024-03-30T00:50:38.074623+0000","last_became_peered":"2024-03-30T00:50:38.074623+0000","last_unstale":"2024-03-30T00:50:38.077548+0000","last_undegraded":"2024-03-30T00:50:38.077548+0000","last_fullsized":"2024-03-30T00:50:38.077548+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077203+0000","last_change":"2024-03-30T00:50:38.065650+0000","last_active":"2024-03-30T00:50:38.077203+0000","last_peered":"2024-03-30T00:50:38.077203+0000","last_clean":"2024-03-30T00:50:38.077203+0000","last_became_active":"2024-03-30T00:50:38.063669+0000","last_became_peered":"2024-03-30T00:50:38.063669+0000","last_unstale":"2024-03-30T00:50:38.077203+0000","last_undegraded":"2024-03-30T00:50:38.077203+0000","last_fullsized":"2024-03-30T00:50:38.077203+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.066049+0000","last_change":"2024-03-30T00:50:38.062347+0000","last_active":"2024-03-30T00:50:38.066049+0000","last_peered":"2024-03-30T00:50:38.066049+0000","last_clean":"2024-03-30T00:50:38.066049+0000","last_became_active":"2024-03-30T00:50:38.059824+0000","last_became_peered":"2024-03-30T00:50:38.059824+0000","last_unstale":"2024-03-30T00:50:38.066049+0000","last_undegraded":"2024-03-30T00:50:38.066049+0000","last_fullsized":"2024-03-30T00:50:38.066049+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.073858+0000","last_change":"2024-03-30T00:50:38.070365+0000","last_active":"2024-03-30T00:50:38.073858+0000","last_peered":"2024-03-30T00:50:38.073858+0000","last_clean":"2024-03-30T00:50:38.073858+0000","last_became_active":"2024-03-30T00:50:38.070088+0000","last_became_peered":"2024-03-30T00:50:38.070088+0000","last_unstale":"2024-03-30T00:50:38.073858+0000","last_undegraded":"2024-03-30T00:50:38.073858+0000","last_fullsized":"2024-03-30T00:50:38.073858+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:56.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.076536+0000","last_change":"2024-03-30T00:50:38.071702+0000","last_active":"2024-03-30T00:50:38.076536+0000","last_peered":"2024-03-30T00:50:38.076536+0000","last_clean":"2024-03-30T00:50:38.076536+0000","last_became_active":"2024-03-30T00:50:38.070983+0000","last_became_peered":"2024-03-30T00:50:38.070983+0000","last_unstale":"2024-03-30T00:50:38.076536+0000","last_undegraded":"2024-03-30T00:50:38.076536+0000","last_fullsized":"2024-03-30T00:50:38.076536+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:13.551635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.070738+0000","last_change":"2024-03-30T00:50:38.063288+0000","last_active":"2024-03-30T00:50:38.070738+0000","last_peered":"2024-03-30T00:50:38.070738+0000","last_clean":"2024-03-30T00:50:38.070738+0000","last_became_active":"2024-03-30T00:50:38.063009+0000","last_became_peered":"2024-03-30T00:50:38.063009+0000","last_unstale":"2024-03-30T00:50:38.070738+0000","last_undegraded":"2024-03-30T00:50:38.070738+0000","last_fullsized":"2024-03-30T00:50:38.070738+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:34:39.399906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.084922+0000","last_change":"2024-03-30T00:50:38.079992+0000","last_active":"2024-03-30T00:50:38.084922+0000","last_peered":"2024-03-30T00:50:38.084922+0000","last_clean":"2024-03-30T00:50:38.084922+0000","last_became_active":"2024-03-30T00:50:38.079708+0000","last_became_peered":"2024-03-30T00:50:38.079708+0000","last_unstale":"2024-03-30T00:50:38.084922+0000","last_undegraded":"2024-03-30T00:50:38.084922+0000","last_fullsized":"2024-03-30T00:50:38.084922+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:12.688382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.084984+0000","last_change":"2024-03-30T00:50:38.078067+0000","last_active":"2024-03-30T00:50:38.084984+0000","last_peered":"2024-03-30T00:50:38.084984+0000","last_clean":"2024-03-30T00:50:38.084984+0000","last_became_active":"2024-03-30T00:50:38.077173+0000","last_became_peered":"2024-03-30T00:50:38.077173+0000","last_unstale":"2024-03-30T00:50:38.084984+0000","last_undegraded":"2024-03-30T00:50:38.084984+0000","last_fullsized":"2024-03-30T00:50:38.084984+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:49.424531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.076768+0000","last_change":"2024-03-30T00:50:38.064028+0000","last_active":"2024-03-30T00:50:38.076768+0000","last_peered":"2024-03-30T00:50:38.076768+0000","last_clean":"2024-03-30T00:50:38.076768+0000","last_became_active":"2024-03-30T00:50:38.063697+0000","last_became_peered":"2024-03-30T00:50:38.063697+0000","last_unstale":"2024-03-30T00:50:38.076768+0000","last_undegraded":"2024-03-30T00:50:38.076768+0000","last_fullsized":"2024-03-30T00:50:38.076768+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:14.841469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077890+0000","last_change":"2024-03-30T00:50:38.069222+0000","last_active":"2024-03-30T00:50:38.077890+0000","last_peered":"2024-03-30T00:50:38.077890+0000","last_clean":"2024-03-30T00:50:38.077890+0000","last_became_active":"2024-03-30T00:50:38.067422+0000","last_became_peered":"2024-03-30T00:50:38.067422+0000","last_unstale":"2024-03-30T00:50:38.077890+0000","last_undegraded":"2024-03-30T00:50:38.077890+0000","last_fullsized":"2024-03-30T00:50:38.077890+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.071379+0000","last_change":"2024-03-30T00:50:38.066103+0000","last_active":"2024-03-30T00:50:38.071379+0000","last_peered":"2024-03-30T00:50:38.071379+0000","last_clean":"2024-03-30T00:50:38.071379+0000","last_became_active":"2024-03-30T00:50:38.065844+0000","last_became_peered":"2024-03-30T00:50:38.065844+0000","last_unstale":"2024-03-30T00:50:38.071379+0000","last_undegraded":"2024-03-30T00:50:38.071379+0000","last_fullsized":"2024-03-30T00:50:38.071379+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"124'4","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.074584+0000","last_change":"2024-03-30T00:50:37.658198+0000","last_active":"2024-03-30T00:50:38.074584+0000","last_peered":"2024-03-30T00:50:38.074584+0000","last_clean":"2024-03-30T00:50:38.074584+0000","last_became_active":"2024-03-30T00:50:37.657799+0000","last_became_peered":"2024-03-30T00:50:37.657799+0000","last_unstale":"2024-03-30T00:50:38.074584+0000","last_undegraded":"2024-03-30T00:50:38.074584+0000","last_fullsized":"2024-03-30T00:50:38.074584+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:55:09.373511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077076+0000","last_change":"2024-03-30T00:50:38.072044+0000","last_active":"2024-03-30T00:50:38.077076+0000","last_peered":"2024-03-30T00:50:38.077076+0000","last_clean":"2024-03-30T00:50:38.077076+0000","last_became_active":"2024-03-30T00:50:38.071778+0000","last_became_peered":"2024-03-30T00:50:38.071778+0000","last_unstale":"2024-03-30T00:50:38.077076+0000","last_undegraded":"2024-03-30T00:50:38.077076+0000","last_fullsized":"2024-03-30T00:50:38.077076+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:58.975555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.068189+0000","last_change":"2024-03-30T00:50:37.647732+0000","last_active":"2024-03-30T00:50:38.068189+0000","last_peered":"2024-03-30T00:50:38.068189+0000","last_clean":"2024-03-30T00:50:38.068189+0000","last_became_active":"2024-03-30T00:50:37.647438+0000","last_became_peered":"2024-03-30T00:50:37.647438+0000","last_unstale":"2024-03-30T00:50:38.068189+0000","last_undegraded":"2024-03-30T00:50:38.068189+0000","last_fullsized":"2024-03-30T00:50:38.068189+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:04.443031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078771+0000","last_change":"2024-03-30T00:50:38.078450+0000","last_active":"2024-03-30T00:50:38.078771+0000","last_peered":"2024-03-30T00:50:38.078771+0000","last_clean":"2024-03-30T00:50:38.078771+0000","last_became_active":"2024-03-30T00:50:38.078156+0000","last_became_peered":"2024-03-30T00:50:38.078156+0000","last_unstale":"2024-03-30T00:50:38.078771+0000","last_undegraded":"2024-03-30T00:50:38.078771+0000","last_fullsized":"2024-03-30T00:50:38.078771+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.080007+0000","last_change":"2024-03-30T00:50:37.651416+0000","last_active":"2024-03-30T00:50:38.080007+0000","last_peered":"2024-03-30T00:50:38.080007+0000","last_clean":"2024-03-30T00:50:38.080007+0000","last_became_active":"2024-03-30T00:50:37.651110+0000","last_became_peered":"2024-03-30T00:50:37.651110+0000","last_unstale":"2024-03-30T00:50:38.080007+0000","last_undegraded":"2024-03-30T00:50:38.080007+0000","last_fullsized":"2024-03-30T00:50:38.080007+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:04.443031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.073970+0000","last_change":"2024-03-30T00:50:38.071349+0000","last_active":"2024-03-30T00:50:38.073970+0000","last_peered":"2024-03-30T00:50:38.073970+0000","last_clean":"2024-03-30T00:50:38.073970+0000","last_became_active":"2024-03-30T00:50:38.071109+0000","last_became_peered":"2024-03-30T00:50:38.071109+0000","last_unstale":"2024-03-30T00:50:38.073970+0000","last_undegraded":"2024-03-30T00:50:38.073970+0000","last_fullsized":"2024-03-30T00:50:38.073970+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:22.469801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.069162+0000","last_change":"2024-03-30T00:50:37.652287+0000","last_active":"2024-03-30T00:50:38.069162+0000","last_peered":"2024-03-30T00:50:38.069162+0000","last_clean":"2024-03-30T00:50:38.069162+0000","last_became_active":"2024-03-30T00:50:37.652025+0000","last_became_peered":"2024-03-30T00:50:37.652025+0000","last_unstale":"2024-03-30T00:50:38.069162+0000","last_undegraded":"2024-03-30T00:50:38.069162+0000","last_fullsized":"2024-03-30T00:50:38.069162+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:37.177730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.070066+0000","last_change":"2024-03-30T00:50:38.061496+0000","last_active":"2024-03-30T00:50:38.070066+0000","last_peered":"2024-03-30T00:50:38.070066+0000","last_clean":"2024-03-30T00:50:38.070066+0000","last_became_active":"2024-03-30T00:50:38.061193+0000","last_became_peered":"2024-03-30T00:50:38.061193+0000","last_unstale":"2024-03-30T00:50:38.070066+0000","last_undegraded":"2024-03-30T00:50:38.070066+0000","last_fullsized":"2024-03-30T00:50:38.070066+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:07.144712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.075441+0000","last_change":"2024-03-30T00:50:37.655896+0000","last_active":"2024-03-30T00:50:38.075441+0000","last_peered":"2024-03-30T00:50:38.075441+0000","last_clean":"2024-03-30T00:50:38.075441+0000","last_became_active":"2024-03-30T00:50:37.655623+0000","last_became_peered":"2024-03-30T00:50:37.655623+0000","last_unstale":"2024-03-30T00:50:38.075441+0000","last_undegraded":"2024-03-30T00:50:38.075441+0000","last_fullsized":"2024-03-30T00:50:38.075441+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:29.599324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.060610+0000","last_change":"2024-03-30T00:50:38.057290+0000","last_active":"2024-03-30T00:50:38.060610+0000","last_peered":"2024-03-30T00:50:38.060610+0000","last_clean":"2024-03-30T00:50:38.060610+0000","last_became_active":"2024-03-30T00:50:38.056944+0000","last_became_peered":"2024-03-30T00:50:38.056944+0000","last_unstale":"2024-03-30T00:50:38.060610+0000","last_undegraded":"2024-03-30T00:50:38.060610+0000","last_fullsized":"2024-03-30T00:50:38.060610+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:54:25.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.071682+0000","last_change":"2024-03-30T00:50:37.657178+0000","last_active":"2024-03-30T00:50:38.071682+0000","last_peered":"2024-03-30T00:50:38.071682+0000","last_clean":"2024-03-30T00:50:38.071682+0000","last_became_active":"2024-03-30T00:50:37.656814+0000","last_became_peered":"2024-03-30T00:50:37.656814+0000","last_unstale":"2024-03-30T00:50:38.071682+0000","last_undegraded":"2024-03-30T00:50:38.071682+0000","last_fullsized":"2024-03-30T00:50:38.071682+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:11.406073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.082677+0000","last_change":"2024-03-30T00:50:38.077938+0000","last_active":"2024-03-30T00:50:38.082677+0000","last_peered":"2024-03-30T00:50:38.082677+0000","last_clean":"2024-03-30T00:50:38.082677+0000","last_became_active":"2024-03-30T00:50:38.077043+0000","last_became_peered":"2024-03-30T00:50:38.077043+0000","last_unstale":"2024-03-30T00:50:38.082677+0000","last_undegraded":"2024-03-30T00:50:38.082677+0000","last_fullsized":"2024-03-30T00:50:38.082677+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:51:44.264107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.056952+0000","last_change":"2024-03-30T00:50:37.654789+0000","last_active":"2024-03-30T00:50:38.056952+0000","last_peered":"2024-03-30T00:50:38.056952+0000","last_clean":"2024-03-30T00:50:38.056952+0000","last_became_active":"2024-03-30T00:50:37.654488+0000","last_became_peered":"2024-03-30T00:50:37.654488+0000","last_unstale":"2024-03-30T00:50:38.056952+0000","last_undegraded":"2024-03-30T00:50:38.056952+0000","last_fullsized":"2024-03-30T00:50:38.056952+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:41:51.590793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.084820+0000","last_change":"2024-03-30T00:50:38.079799+0000","last_active":"2024-03-30T00:50:38.084820+0000","last_peered":"2024-03-30T00:50:38.084820+0000","last_clean":"2024-03-30T00:50:38.084820+0000","last_became_active":"2024-03-30T00:50:38.079537+0000","last_became_peered":"2024-03-30T00:50:38.079537+0000","last_unstale":"2024-03-30T00:50:38.084820+0000","last_undegraded":"2024-03-30T00:50:38.084820+0000","last_fullsized":"2024-03-30T00:50:38.084820+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:07:30.601895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.059013+0000","last_change":"2024-03-30T00:50:37.658253+0000","last_active":"2024-03-30T00:50:38.059013+0000","last_peered":"2024-03-30T00:50:38.059013+0000","last_clean":"2024-03-30T00:50:38.059013+0000","last_became_active":"2024-03-30T00:50:37.657996+0000","last_became_peered":"2024-03-30T00:50:37.657996+0000","last_unstale":"2024-03-30T00:50:38.059013+0000","last_undegraded":"2024-03-30T00:50:38.059013+0000","last_fullsized":"2024-03-30T00:50:38.059013+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:51:25.787216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.075723+0000","last_change":"2024-03-30T00:50:38.071292+0000","last_active":"2024-03-30T00:50:38.075723+0000","last_peered":"2024-03-30T00:50:38.075723+0000","last_clean":"2024-03-30T00:50:38.075723+0000","last_became_active":"2024-03-30T00:50:38.070916+0000","last_became_peered":"2024-03-30T00:50:38.070916+0000","last_unstale":"2024-03-30T00:50:38.075723+0000","last_undegraded":"2024-03-30T00:50:38.075723+0000","last_fullsized":"2024-03-30T00:50:38.075723+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:01.855290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.055152+0000","last_change":"2024-03-30T00:50:37.659011+0000","last_active":"2024-03-30T00:50:38.055152+0000","last_peered":"2024-03-30T00:50:38.055152+0000","last_clean":"2024-03-30T00:50:38.055152+0000","last_became_active":"2024-03-30T00:50:37.658720+0000","last_became_peered":"2024-03-30T00:50:37.658720+0000","last_unstale":"2024-03-30T00:50:38.055152+0000","last_undegraded":"2024-03-30T00:50:38.055152+0000","last_fullsized":"2024-03-30T00:50:38.055152+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:57:57.692452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078459+0000","last_change":"2024-03-30T00:50:38.075750+0000","last_active":"2024-03-30T00:50:38.078459+0000","last_peered":"2024-03-30T00:50:38.078459+0000","last_clean":"2024-03-30T00:50:38.078459+0000","last_became_active":"2024-03-30T00:50:38.075443+0000","last_became_peered":"2024-03-30T00:50:38.075443+0000","last_unstale":"2024-03-30T00:50:38.078459+0000","last_undegraded":"2024-03-30T00:50:38.078459+0000","last_fullsized":"2024-03-30T00:50:38.078459+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.059165+0000","last_change":"2024-03-30T00:50:37.651177+0000","last_active":"2024-03-30T00:50:38.059165+0000","last_peered":"2024-03-30T00:50:38.059165+0000","last_clean":"2024-03-30T00:50:38.059165+0000","last_became_active":"2024-03-30T00:50:37.650902+0000","last_became_peered":"2024-03-30T00:50:37.650902+0000","last_unstale":"2024-03-30T00:50:38.059165+0000","last_undegraded":"2024-03-30T00:50:38.059165+0000","last_fullsized":"2024-03-30T00:50:38.059165+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:28.311412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.079090+0000","last_change":"2024-03-30T00:50:38.070389+0000","last_active":"2024-03-30T00:50:38.079090+0000","last_peered":"2024-03-30T00:50:38.079090+0000","last_clean":"2024-03-30T00:50:38.079090+0000","last_became_active":"2024-03-30T00:50:38.066504+0000","last_became_peered":"2024-03-30T00:50:38.066504+0000","last_unstale":"2024-03-30T00:50:38.079090+0000","last_undegraded":"2024-03-30T00:50:38.079090+0000","last_fullsized":"2024-03-30T00:50:38.079090+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:51:42.504328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"124'3","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078389+0000","last_change":"2024-03-30T00:50:37.653495+0000","last_active":"2024-03-30T00:50:38.078389+0000","last_peered":"2024-03-30T00:50:38.078389+0000","last_clean":"2024-03-30T00:50:38.078389+0000","last_became_active":"2024-03-30T00:50:37.652706+0000","last_became_peered":"2024-03-30T00:50:37.652706+0000","last_unstale":"2024-03-30T00:50:38.078389+0000","last_undegraded":"2024-03-30T00:50:38.078389+0000","last_fullsized":"2024-03-30T00:50:38.078389+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:07:01.780302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078191+0000","last_change":"2024-03-30T00:50:38.075296+0000","last_active":"2024-03-30T00:50:38.078191+0000","last_peered":"2024-03-30T00:50:38.078191+0000","last_clean":"2024-03-30T00:50:38.078191+0000","last_became_active":"2024-03-30T00:50:38.075035+0000","last_became_peered":"2024-03-30T00:50:38.075035+0000","last_unstale":"2024-03-30T00:50:38.078191+0000","last_undegraded":"2024-03-30T00:50:38.078191+0000","last_fullsized":"2024-03-30T00:50:38.078191+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:46:27.974997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"124'6","reported_seq":17,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.085335+0000","last_change":"2024-03-30T00:50:37.648270+0000","last_active":"2024-03-30T00:50:38.085335+0000","last_peered":"2024-03-30T00:50:38.085335+0000","last_clean":"2024-03-30T00:50:38.085335+0000","last_became_active":"2024-03-30T00:50:37.647989+0000","last_became_peered":"2024-03-30T00:50:37.647989+0000","last_unstale":"2024-03-30T00:50:38.085335+0000","last_undegraded":"2024-03-30T00:50:38.085335+0000","last_fullsized":"2024-03-30T00:50:38.085335+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:53.076893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078212+0000","last_change":"2024-03-30T00:50:38.076639+0000","last_active":"2024-03-30T00:50:38.078212+0000","last_peered":"2024-03-30T00:50:38.078212+0000","last_clean":"2024-03-30T00:50:38.078212+0000","last_became_active":"2024-03-30T00:50:38.075236+0000","last_became_peered":"2024-03-30T00:50:38.075236+0000","last_unstale":"2024-03-30T00:50:38.078212+0000","last_undegraded":"2024-03-30T00:50:38.078212+0000","last_fullsized":"2024-03-30T00:50:38.078212+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:05.445602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.084971+0000","last_change":"2024-03-30T00:50:37.655775+0000","last_active":"2024-03-30T00:50:38.084971+0000","last_peered":"2024-03-30T00:50:38.084971+0000","last_clean":"2024-03-30T00:50:38.084971+0000","last_became_active":"2024-03-30T00:50:37.655033+0000","last_became_peered":"2024-03-30T00:50:37.655033+0000","last_unstale":"2024-03-30T00:50:38.084971+0000","last_undegraded":"2024-03-30T00:50:38.084971+0000","last_fullsized":"2024-03-30T00:50:38.084971+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:46:26.972427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.068048+0000","last_change":"2024-03-30T00:50:38.052149+0000","last_active":"2024-03-30T00:50:38.068048+0000","last_peered":"2024-03-30T00:50:38.068048+0000","last_clean":"2024-03-30T00:50:38.068048+0000","last_became_active":"2024-03-30T00:50:38.051636+0000","last_became_peered":"2024-03-30T00:50:38.051636+0000","last_unstale":"2024-03-30T00:50:38.068048+0000","last_undegraded":"2024-03-30T00:50:38.068048+0000","last_fullsized":"2024-03-30T00:50:38.068048+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.056299+0000","last_change":"2024-03-30T00:50:37.657373+0000","last_active":"2024-03-30T00:50:38.056299+0000","last_peered":"2024-03-30T00:50:38.056299+0000","last_clean":"2024-03-30T00:50:38.056299+0000","last_became_active":"2024-03-30T00:50:37.657068+0000","last_became_peered":"2024-03-30T00:50:37.657068+0000","last_unstale":"2024-03-30T00:50:38.056299+0000","last_undegraded":"2024-03-30T00:50:38.056299+0000","last_fullsized":"2024-03-30T00:50:38.056299+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:36.963155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.067184+0000","last_change":"2024-03-30T00:50:38.063510+0000","last_active":"2024-03-30T00:50:38.067184+0000","last_peered":"2024-03-30T00:50:38.067184+0000","last_clean":"2024-03-30T00:50:38.067184+0000","last_became_active":"2024-03-30T00:50:38.063256+0000","last_became_peered":"2024-03-30T00:50:38.063256+0000","last_unstale":"2024-03-30T00:50:38.067184+0000","last_undegraded":"2024-03-30T00:50:38.067184+0000","last_fullsized":"2024-03-30T00:50:38.067184+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:02:07.606929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.060004+0000","last_change":"2024-03-30T00:50:37.653596+0000","last_active":"2024-03-30T00:50:38.060004+0000","last_peered":"2024-03-30T00:50:38.060004+0000","last_clean":"2024-03-30T00:50:38.060004+0000","last_became_active":"2024-03-30T00:50:37.653299+0000","last_became_peered":"2024-03-30T00:50:37.653299+0000","last_unstale":"2024-03-30T00:50:38.060004+0000","last_undegraded":"2024-03-30T00:50:38.060004+0000","last_fullsized":"2024-03-30T00:50:38.060004+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:55:06.142142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078221+0000","last_change":"2024-03-30T00:50:38.073132+0000","last_active":"2024-03-30T00:50:38.078221+0000","last_peered":"2024-03-30T00:50:38.078221+0000","last_clean":"2024-03-30T00:50:38.078221+0000","last_became_active":"2024-03-30T00:50:38.072860+0000","last_became_peered":"2024-03-30T00:50:38.072860+0000","last_unstale":"2024-03-30T00:50:38.078221+0000","last_undegraded":"2024-03-30T00:50:38.078221+0000","last_fullsized":"2024-03-30T00:50:38.078221+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:41.548382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.073471+0000","last_change":"2024-03-30T00:50:37.649310+0000","last_active":"2024-03-30T00:50:38.073471+0000","last_peered":"2024-03-30T00:50:38.073471+0000","last_clean":"2024-03-30T00:50:38.073471+0000","last_became_active":"2024-03-30T00:50:37.649015+0000","last_became_peered":"2024-03-30T00:50:37.649015+0000","last_unstale":"2024-03-30T00:50:38.073471+0000","last_undegraded":"2024-03-30T00:50:38.073471+0000","last_fullsized":"2024-03-30T00:50:38.073471+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:00.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.075632+0000","last_change":"2024-03-30T00:50:38.074524+0000","last_active":"2024-03-30T00:50:38.075632+0000","last_peered":"2024-03-30T00:50:38.075632+0000","last_clean":"2024-03-30T00:50:38.075632+0000","last_became_active":"2024-03-30T00:50:38.074266+0000","last_became_peered":"2024-03-30T00:50:38.074266+0000","last_unstale":"2024-03-30T00:50:38.075632+0000","last_undegraded":"2024-03-30T00:50:38.075632+0000","last_fullsized":"2024-03-30T00:50:38.075632+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:59:22.686200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.074473+0000","last_change":"2024-03-30T00:50:37.655900+0000","last_active":"2024-03-30T00:50:38.074473+0000","last_peered":"2024-03-30T00:50:38.074473+0000","last_clean":"2024-03-30T00:50:38.074473+0000","last_became_active":"2024-03-30T00:50:37.655601+0000","last_became_peered":"2024-03-30T00:50:37.655601+0000","last_unstale":"2024-03-30T00:50:38.074473+0000","last_undegraded":"2024-03-30T00:50:38.074473+0000","last_fullsized":"2024-03-30T00:50:38.074473+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:33.394770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.073937+0000","last_change":"2024-03-30T00:50:38.069971+0000","last_active":"2024-03-30T00:50:38.073937+0000","last_peered":"2024-03-30T00:50:38.073937+0000","last_clean":"2024-03-30T00:50:38.073937+0000","last_became_active":"2024-03-30T00:50:38.069640+0000","last_became_peered":"2024-03-30T00:50:38.069640+0000","last_unstale":"2024-03-30T00:50:38.073937+0000","last_undegraded":"2024-03-30T00:50:38.073937+0000","last_fullsized":"2024-03-30T00:50:38.073937+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:09:05.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078669+0000","last_change":"2024-03-30T00:50:37.649475+0000","last_active":"2024-03-30T00:50:38.078669+0000","last_peered":"2024-03-30T00:50:38.078669+0000","last_clean":"2024-03-30T00:50:38.078669+0000","last_became_active":"2024-03-30T00:50:37.649055+0000","last_became_peered":"2024-03-30T00:50:37.649055+0000","last_unstale":"2024-03-30T00:50:38.078669+0000","last_undegraded":"2024-03-30T00:50:38.078669+0000","last_fullsized":"2024-03-30T00:50:38.078669+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:12.549064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.066027+0000","last_change":"2024-03-30T00:50:38.060240+0000","last_active":"2024-03-30T00:50:38.066027+0000","last_peered":"2024-03-30T00:50:38.066027+0000","last_clean":"2024-03-30T00:50:38.066027+0000","last_became_active":"2024-03-30T00:50:38.059835+0000","last_became_peered":"2024-03-30T00:50:38.059835+0000","last_unstale":"2024-03-30T00:50:38.066027+0000","last_undegraded":"2024-03-30T00:50:38.066027+0000","last_fullsized":"2024-03-30T00:50:38.066027+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:47.211883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.050373+0000","last_change":"2024-03-30T00:50:37.656047+0000","last_active":"2024-03-30T00:50:38.050373+0000","last_peered":"2024-03-30T00:50:38.050373+0000","last_clean":"2024-03-30T00:50:38.050373+0000","last_became_active":"2024-03-30T00:50:37.655751+0000","last_became_peered":"2024-03-30T00:50:37.655751+0000","last_unstale":"2024-03-30T00:50:38.050373+0000","last_undegraded":"2024-03-30T00:50:38.050373+0000","last_fullsized":"2024-03-30T00:50:38.050373+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:20:00.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.071836+0000","last_change":"2024-03-30T00:50:38.067072+0000","last_active":"2024-03-30T00:50:38.071836+0000","last_peered":"2024-03-30T00:50:38.071836+0000","last_clean":"2024-03-30T00:50:38.071836+0000","last_became_active":"2024-03-30T00:50:38.066790+0000","last_became_peered":"2024-03-30T00:50:38.066790+0000","last_unstale":"2024-03-30T00:50:38.071836+0000","last_undegraded":"2024-03-30T00:50:38.071836+0000","last_fullsized":"2024-03-30T00:50:38.071836+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:52:46.481754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.068037+0000","last_change":"2024-03-30T00:50:37.646336+0000","last_active":"2024-03-30T00:50:38.068037+0000","last_peered":"2024-03-30T00:50:38.068037+0000","last_clean":"2024-03-30T00:50:38.068037+0000","last_became_active":"2024-03-30T00:50:37.646081+0000","last_became_peered":"2024-03-30T00:50:37.646081+0000","last_unstale":"2024-03-30T00:50:38.068037+0000","last_undegraded":"2024-03-30T00:50:38.068037+0000","last_fullsized":"2024-03-30T00:50:38.068037+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:40.726869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.082724+0000","last_change":"2024-03-30T00:50:38.078166+0000","last_active":"2024-03-30T00:50:38.082724+0000","last_peered":"2024-03-30T00:50:38.082724+0000","last_clean":"2024-03-30T00:50:38.082724+0000","last_became_active":"2024-03-30T00:50:38.077911+0000","last_became_peered":"2024-03-30T00:50:38.077911+0000","last_unstale":"2024-03-30T00:50:38.082724+0000","last_undegraded":"2024-03-30T00:50:38.082724+0000","last_fullsized":"2024-03-30T00:50:38.082724+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:04:19.985679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.051995+0000","last_change":"2024-03-30T00:50:37.648309+0000","last_active":"2024-03-30T00:50:38.051995+0000","last_peered":"2024-03-30T00:50:38.051995+0000","last_clean":"2024-03-30T00:50:38.051995+0000","last_became_active":"2024-03-30T00:50:37.646903+0000","last_became_peered":"2024-03-30T00:50:37.646903+0000","last_unstale":"2024-03-30T00:50:38.051995+0000","last_undegraded":"2024-03-30T00:50:38.051995+0000","last_fullsized":"2024-03-30T00:50:38.051995+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:20.839135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.082464+0000","last_change":"2024-03-30T00:50:38.076759+0000","last_active":"2024-03-30T00:50:38.082464+0000","last_peered":"2024-03-30T00:50:38.082464+0000","last_clean":"2024-03-30T00:50:38.082464+0000","last_became_active":"2024-03-30T00:50:38.068983+0000","last_became_peered":"2024-03-30T00:50:38.068983+0000","last_unstale":"2024-03-30T00:50:38.082464+0000","last_undegraded":"2024-03-30T00:50:38.082464+0000","last_fullsized":"2024-03-30T00:50:38.082464+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:38:49.091051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.060100+0000","last_change":"2024-03-30T00:50:37.656069+0000","last_active":"2024-03-30T00:50:38.060100+0000","last_peered":"2024-03-30T00:50:38.060100+0000","last_clean":"2024-03-30T00:50:38.060100+0000","last_became_active":"2024-03-30T00:50:37.655828+0000","last_became_peered":"2024-03-30T00:50:37.655828+0000","last_unstale":"2024-03-30T00:50:38.060100+0000","last_undegraded":"2024-03-30T00:50:38.060100+0000","last_fullsized":"2024-03-30T00:50:38.060100+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:46.209312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078150+0000","last_change":"2024-03-30T00:50:38.072920+0000","last_active":"2024-03-30T00:50:38.078150+0000","last_peered":"2024-03-30T00:50:38.078150+0000","last_clean":"2024-03-30T00:50:38.078150+0000","last_became_active":"2024-03-30T00:50:38.072673+0000","last_became_peered":"2024-03-30T00:50:38.072673+0000","last_unstale":"2024-03-30T00:50:38.078150+0000","last_undegraded":"2024-03-30T00:50:38.078150+0000","last_fullsized":"2024-03-30T00:50:38.078150+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:50:37.965725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.050094+0000","last_change":"2024-03-30T00:50:37.650090+0000","last_active":"2024-03-30T00:50:38.050094+0000","last_peered":"2024-03-30T00:50:38.050094+0000","last_clean":"2024-03-30T00:50:38.050094+0000","last_became_active":"2024-03-30T00:50:37.649527+0000","last_became_peered":"2024-03-30T00:50:37.649527+0000","last_unstale":"2024-03-30T00:50:38.050094+0000","last_undegraded":"2024-03-30T00:50:38.050094+0000","last_fullsized":"2024-03-30T00:50:38.050094+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:24:37.177730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.077103+0000","last_change":"2024-03-30T00:50:38.072192+0000","last_active":"2024-03-30T00:50:38.077103+0000","last_peered":"2024-03-30T00:50:38.077103+0000","last_clean":"2024-03-30T00:50:38.077103+0000","last_became_active":"2024-03-30T00:50:38.071946+0000","last_became_peered":"2024-03-30T00:50:38.071946+0000","last_unstale":"2024-03-30T00:50:38.077103+0000","last_undegraded":"2024-03-30T00:50:38.077103+0000","last_fullsized":"2024-03-30T00:50:38.077103+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:19.197836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.072836+0000","last_change":"2024-03-30T00:50:37.648973+0000","last_active":"2024-03-30T00:50:38.072836+0000","last_peered":"2024-03-30T00:50:38.072836+0000","last_clean":"2024-03-30T00:50:38.072836+0000","last_became_active":"2024-03-30T00:50:37.648703+0000","last_became_peered":"2024-03-30T00:50:37.648703+0000","last_unstale":"2024-03-30T00:50:38.072836+0000","last_undegraded":"2024-03-30T00:50:38.072836+0000","last_fullsized":"2024-03-30T00:50:38.072836+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:53:36.133173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.070622+0000","last_change":"2024-03-30T00:50:38.061500+0000","last_active":"2024-03-30T00:50:38.070622+0000","last_peered":"2024-03-30T00:50:38.070622+0000","last_clean":"2024-03-30T00:50:38.070622+0000","last_became_active":"2024-03-30T00:50:38.061083+0000","last_became_peered":"2024-03-30T00:50:38.061083+0000","last_unstale":"2024-03-30T00:50:38.070622+0000","last_undegraded":"2024-03-30T00:50:38.070622+0000","last_fullsized":"2024-03-30T00:50:38.070622+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"125'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-03-30T00:50:39.058059+0000","last_change":"2024-03-30T00:50:37.658787+0000","last_active":"2024-03-30T00:50:39.058059+0000","last_peered":"2024-03-30T00:50:39.058059+0000","last_clean":"2024-03-30T00:50:39.058059+0000","last_became_active":"2024-03-30T00:50:37.658512+0000","last_became_peered":"2024-03-30T00:50:37.658512+0000","last_unstale":"2024-03-30T00:50:39.058059+0000","last_undegraded":"2024-03-30T00:50:39.058059+0000","last_fullsized":"2024-03-30T00:50:39.058059+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:55:31.799226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.059764+0000","last_change":"2024-03-30T00:50:38.056667+0000","last_active":"2024-03-30T00:50:38.059764+0000","last_peered":"2024-03-30T00:50:38.059764+0000","last_clean":"2024-03-30T00:50:38.059764+0000","last_became_active":"2024-03-30T00:50:38.056040+0000","last_became_peered":"2024-03-30T00:50:38.056040+0000","last_unstale":"2024-03-30T00:50:38.059764+0000","last_undegraded":"2024-03-30T00:50:38.059764+0000","last_fullsized":"2024-03-30T00:50:38.059764+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:54.079463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.057131+0000","last_change":"2024-03-30T00:50:37.656511+0000","last_active":"2024-03-30T00:50:38.057131+0000","last_peered":"2024-03-30T00:50:38.057131+0000","last_clean":"2024-03-30T00:50:38.057131+0000","last_became_active":"2024-03-30T00:50:37.655106+0000","last_became_peered":"2024-03-30T00:50:37.655106+0000","last_unstale":"2024-03-30T00:50:38.057131+0000","last_undegraded":"2024-03-30T00:50:38.057131+0000","last_fullsized":"2024-03-30T00:50:38.057131+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:04:28.343406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.079154+0000","last_change":"2024-03-30T00:50:38.076758+0000","last_active":"2024-03-30T00:50:38.079154+0000","last_peered":"2024-03-30T00:50:38.079154+0000","last_clean":"2024-03-30T00:50:38.079154+0000","last_became_active":"2024-03-30T00:50:38.076404+0000","last_became_peered":"2024-03-30T00:50:38.076404+0000","last_unstale":"2024-03-30T00:50:38.079154+0000","last_undegraded":"2024-03-30T00:50:38.079154+0000","last_fullsized":"2024-03-30T00:50:38.079154+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:34:31.524194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.059782+0000","last_change":"2024-03-30T00:50:37.655230+0000","last_active":"2024-03-30T00:50:38.059782+0000","last_peered":"2024-03-30T00:50:38.059782+0000","last_clean":"2024-03-30T00:50:38.059782+0000","last_became_active":"2024-03-30T00:50:37.654933+0000","last_became_peered":"2024-03-30T00:50:37.654933+0000","last_unstale":"2024-03-30T00:50:38.059782+0000","last_undegraded":"2024-03-30T00:50:38.059782+0000","last_fullsized":"2024-03-30T00:50:38.059782+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:59.607374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.076131+0000","last_change":"2024-03-30T00:50:38.071227+0000","last_active":"2024-03-30T00:50:38.076131+0000","last_peered":"2024-03-30T00:50:38.076131+0000","last_clean":"2024-03-30T00:50:38.076131+0000","last_became_active":"2024-03-30T00:50:38.068099+0000","last_became_peered":"2024-03-30T00:50:38.068099+0000","last_unstale":"2024-03-30T00:50:38.076131+0000","last_undegraded":"2024-03-30T00:50:38.076131+0000","last_fullsized":"2024-03-30T00:50:38.076131+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:52:14.331871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.052644+0000","last_change":"2024-03-30T00:50:37.647611+0000","last_active":"2024-03-30T00:50:38.052644+0000","last_peered":"2024-03-30T00:50:38.052644+0000","last_clean":"2024-03-30T00:50:38.052644+0000","last_became_active":"2024-03-30T00:50:37.647311+0000","last_became_peered":"2024-03-30T00:50:37.647311+0000","last_unstale":"2024-03-30T00:50:38.052644+0000","last_undegraded":"2024-03-30T00:50:38.052644+0000","last_fullsized":"2024-03-30T00:50:38.052644+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:55:21.467231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.078739+0000","last_change":"2024-03-30T00:50:38.069404+0000","last_active":"2024-03-30T00:50:38.078739+0000","last_peered":"2024-03-30T00:50:38.078739+0000","last_clean":"2024-03-30T00:50:38.078739+0000","last_became_active":"2024-03-30T00:50:38.069128+0000","last_became_peered":"2024-03-30T00:50:38.069128+0000","last_unstale":"2024-03-30T00:50:38.078739+0000","last_undegraded":"2024-03-30T00:50:38.078739+0000","last_fullsized":"2024-03-30T00:50:38.078739+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:44:50.505969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.052427+0000","last_change":"2024-03-30T00:50:37.651832+0000","last_active":"2024-03-30T00:50:38.052427+0000","last_peered":"2024-03-30T00:50:38.052427+0000","last_clean":"2024-03-30T00:50:38.052427+0000","last_became_active":"2024-03-30T00:50:37.651529+0000","last_became_peered":"2024-03-30T00:50:37.651529+0000","last_unstale":"2024-03-30T00:50:38.052427+0000","last_undegraded":"2024-03-30T00:50:38.052427+0000","last_fullsized":"2024-03-30T00:50:38.052427+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:19.643626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.073053+0000","last_change":"2024-03-30T00:50:38.068952+0000","last_active":"2024-03-30T00:50:38.073053+0000","last_peered":"2024-03-30T00:50:38.073053+0000","last_clean":"2024-03-30T00:50:38.073053+0000","last_became_active":"2024-03-30T00:50:38.068683+0000","last_became_peered":"2024-03-30T00:50:38.068683+0000","last_unstale":"2024-03-30T00:50:38.073053+0000","last_undegraded":"2024-03-30T00:50:38.073053+0000","last_fullsized":"2024-03-30T00:50:38.073053+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:41.220197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.062791+0000","last_change":"2024-03-30T00:50:37.654139+0000","last_active":"2024-03-30T00:50:38.062791+0000","last_peered":"2024-03-30T00:50:38.062791+0000","last_clean":"2024-03-30T00:50:38.062791+0000","last_became_active":"2024-03-30T00:50:37.653825+0000","last_became_peered":"2024-03-30T00:50:37.653825+0000","last_unstale":"2024-03-30T00:50:38.062791+0000","last_undegraded":"2024-03-30T00:50:38.062791+0000","last_fullsized":"2024-03-30T00:50:38.062791+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:47.043924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.068100+0000","last_change":"2024-03-30T00:50:38.052861+0000","last_active":"2024-03-30T00:50:38.068100+0000","last_peered":"2024-03-30T00:50:38.068100+0000","last_clean":"2024-03-30T00:50:38.068100+0000","last_became_active":"2024-03-30T00:50:38.052613+0000","last_became_peered":"2024-03-30T00:50:38.052613+0000","last_unstale":"2024-03-30T00:50:38.068100+0000","last_undegraded":"2024-03-30T00:50:38.068100+0000","last_fullsized":"2024-03-30T00:50:38.068100+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:29:30.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.050185+0000","last_change":"2024-03-30T00:50:37.654797+0000","last_active":"2024-03-30T00:50:38.050185+0000","last_peered":"2024-03-30T00:50:38.050185+0000","last_clean":"2024-03-30T00:50:38.050185+0000","last_became_active":"2024-03-30T00:50:37.654506+0000","last_became_peered":"2024-03-30T00:50:37.654506+0000","last_unstale":"2024-03-30T00:50:38.050185+0000","last_undegraded":"2024-03-30T00:50:38.050185+0000","last_fullsized":"2024-03-30T00:50:38.050185+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:26.653343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.070465+0000","last_change":"2024-03-30T00:50:38.066537+0000","last_active":"2024-03-30T00:50:38.070465+0000","last_peered":"2024-03-30T00:50:38.070465+0000","last_clean":"2024-03-30T00:50:38.070465+0000","last_became_active":"2024-03-30T00:50:38.066299+0000","last_became_peered":"2024-03-30T00:50:38.066299+0000","last_unstale":"2024-03-30T00:50:38.070465+0000","last_undegraded":"2024-03-30T00:50:38.070465+0000","last_fullsized":"2024-03-30T00:50:38.070465+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:37.625832+0000","last_clean_scrub_stamp":"2024-03-30T00:50:37.625832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:20.646196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"124'8","reported_seq":19,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.084998+0000","last_change":"2024-03-30T00:50:37.648683+0000","last_active":"2024-03-30T00:50:38.084998+0000","last_peered":"2024-03-30T00:50:38.084998+0000","last_clean":"2024-03-30T00:50:38.084998+0000","last_became_active":"2024-03-30T00:50:37.648391+0000","last_became_peered":"2024-03-30T00:50:37.648391+0000","last_unstale":"2024-03-30T00:50:38.084998+0000","last_undegraded":"2024-03-30T00:50:38.084998+0000","last_fullsized":"2024-03-30T00:50:38.084998+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:50:36.623262+0000","last_clean_scrub_stamp":"2024-03-30T00:50:36.623262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:24:13.838899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":273,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-30T00:50:38.050328+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:50:38.050328+0000","last_peered":"2024-03-30T00:50:38.050328+0000","last_clean":"2024-03-30T00:50:38.050328+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:50:38.050328+0000","last_undegraded":"2024-03-30T00:50:38.050328+0000","last_fullsized":"2024-03-30T00:50:38.050328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574983,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33240,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338600,"statfs":{"total":96636764160,"available":96602726400,"internally_reserved":0,"allocated":2166784,"data_stored":1311133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":0.97599999999999998,"15min":1.0860000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.47399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84999999999999998,"5min":0.98299999999999998,"15min":1.149},"last":0.80400000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.503,"5min":0.45800000000000002,"15min":0.436},"max":{"1min":0.90800000000000003,"5min":1.03,"15min":1.03},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93000000000000005,"5min":0.95199999999999996,"15min":1.3029999999999999},"last":0.754}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.76600000000000001},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.45000000000000001},"max":{"1min":1.0129999999999999,"5min":1.085,"15min":1.6020000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.73099999999999998,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95399999999999996,"5min":1.034,"15min":1.6619999999999999},"last":0.89600000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.75900000000000001},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99099999999999999,"5min":0.99299999999999999,"15min":2.0150000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.80000000000000004},"min":{"1min":0.63800000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.9490000000000001},"last":0.86799999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.37},"max":{"1min":0.97299999999999998,"5min":1.2,"15min":1.2},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.42999999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.97899999999999998},"last":0.56999999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96299999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":0.99199999999999999,"15min":1.026},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.92700000000000005,"5min":0.94599999999999995,"15min":1.214},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89200000000000002,"5min":1.0189999999999999,"15min":1.0309999999999999},"last":0.63300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1560576,"data_stored":720571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71599999999999997,"15min":0.79800000000000004},"min":{"1min":0.437,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.85799999999999998,"5min":1.2969999999999999,"15min":4.0490000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.68799999999999994},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":1.278,"15min":1.47},"last":0.64300000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.438,"15min":0.36899999999999999},"max":{"1min":0.95799999999999996,"5min":1.204,"15min":4.3170000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.85299999999999998},"min":{"1min":0.58399999999999996,"5min":0.47199999999999998,"15min":0.42999999999999999},"max":{"1min":0.92900000000000005,"5min":1.1120000000000001,"15min":4.4160000000000004},"last":0.72799999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.745,"15min":0.84999999999999998},"min":{"1min":0.63400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98299999999999998,"5min":1.24,"15min":4.375},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.84299999999999997},"min":{"1min":0.51700000000000002,"5min":0.42999999999999999,"15min":0.38600000000000001},"max":{"1min":0.92500000000000004,"5min":1.3129999999999999,"15min":4.2960000000000003},"last":0.61899999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.86799999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0249999999999999,"5min":1.2390000000000001,"15min":4.3399999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.73199999999999998,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.49099999999999999,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.258,"15min":4.6020000000000003},"last":0.63}]},{"osd":4,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.39200000000000002,"5min":0.39100000000000001,"15min":0.35999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.1870000000000001},"last":0.499},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.73199999999999998},"min":{"1min":0.441,"5min":0.372,"15min":0.372},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.036},"last":0.67400000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.65700000000000003},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0720000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0109999999999999,"15min":1.101},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:49:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97499999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.45700000000000002},"max":{"1min":1.0860000000000001,"5min":1.103,"15min":1.103},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1564672,"data_stored":720149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":0.83299999999999996,"5min":0.97599999999999998,"15min":1.095},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68700000000000006,"15min":0.73199999999999998},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":1.0489999999999999,"15min":1.7569999999999999},"last":0.69699999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.71999999999999997},"min":{"1min":0.47899999999999998,"5min":0.46600000000000003,"15min":0.43099999999999999},"max":{"1min":0.86599999999999999,"5min":0.98199999999999998,"15min":1.7330000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70999999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42399999999999999},"max":{"1min":0.92400000000000004,"5min":0.98799999999999999,"15min":1.3260000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.44600000000000001},"max":{"1min":0.89900000000000002,"5min":1.131,"15min":1.131},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.45900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.877,"5min":1.0609999999999999,"15min":1.2789999999999999},"last":0.624}]},{"osd":3,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.75},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.89100000000000001,"5min":1.0740000000000001,"15min":1.7},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.96099999999999997,"5min":1.1160000000000001,"15min":1.181},"last":0.67500000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.60299999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.309},"max":{"1min":0.876,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69199999999999995,"15min":0.73399999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91200000000000003,"5min":0.97799999999999998,"15min":1.615},"last":0.748}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.84099999999999997,"5min":0.91100000000000003,"15min":0.97399999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.626,"15min":0.67300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.35199999999999998},"max":{"1min":0.84999999999999998,"5min":0.93700000000000006,"15min":1.569},"last":0.58399999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.71499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.377},"max":{"1min":0.85399999999999998,"5min":0.92200000000000004,"15min":1.0149999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88400000000000001,"5min":1.167,"15min":1.383},"last":0.76500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574983,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1568768,"data_stored":721553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.34899999999999998},"max":{"1min":0.92400000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.67400000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98599999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.72699999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":0.92600000000000005,"5min":0.95899999999999996,"15min":1.2909999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.40699999999999997},"max":{"1min":1.006,"5min":1.113,"15min":1.113},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.59199999999999997,"5min":0.48999999999999999,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.73899999999999999,"15min":0.75},"min":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.46700000000000003},"max":{"1min":0.94899999999999995,"5min":1.163,"15min":1.228},"last":0.749}]},{"osd":3,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73599999999999999,"15min":0.71699999999999997},"min":{"1min":0.501,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.27},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.76000000000000001},"min":{"1min":0.59099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1,"15min":1.1739999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.621,"15min":0.65800000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.89400000000000002,"15min":1.329},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66100000000000003,"15min":0.70399999999999996},"min":{"1min":0.57399999999999995,"5min":0.34799999999999998,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":0.99099999999999999,"15min":1.3819999999999999},"last":0.58099999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.65700000000000003},"min":{"1min":0.45700000000000002,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":1.2130000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61599999999999999,"15min":0.69699999999999995},"min":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.95799999999999996,"15min":1.3220000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.628,"15min":0.69099999999999995},"min":{"1min":0.46700000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":0.91500000000000004,"15min":1.6910000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.73399999999999999,"15min":0.78000000000000003},"min":{"1min":0.59699999999999998,"5min":0.49299999999999999,"15min":0.46600000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.5009999999999999},"last":0.85799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574984,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1572864,"data_stored":721587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.435,"15min":0.45500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.58699999999999997,"5min":0.748,"15min":1.0529999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.49099999999999999,"15min":0.51400000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.24199999999999999},"max":{"1min":0.63400000000000001,"5min":0.75,"15min":0.79100000000000004},"last":0.36599999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49399999999999999,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.80900000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63800000000000001,"5min":0.73399999999999999,"15min":1.2150000000000001},"last":0.40799999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.78300000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47299999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.46300000000000002},"max":{"1min":0.95699999999999996,"5min":1.1259999999999999,"15min":1.2470000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77100000000000002,"15min":0.77600000000000002},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.079,"15min":1.0860000000000001},"last":0.59099999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77100000000000002,"15min":0.77900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.51100000000000001},"max":{"1min":1.024,"5min":1.0369999999999999,"15min":1.125},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.77900000000000003,"15min":0.79100000000000004},"min":{"1min":0.628,"5min":0.52200000000000002,"15min":0.48599999999999999},"max":{"1min":0.86299999999999999,"5min":1.133,"15min":1.2390000000000001},"last":0.748}]},{"osd":6,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77200000000000002,"15min":0.79700000000000004},"min":{"1min":0.59999999999999998,"5min":0.51400000000000001,"15min":0.499},"max":{"1min":0.90100000000000002,"5min":1.091,"15min":1.226},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.038,"5min":1.1120000000000001,"15min":1.167},"last":0.71399999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82999999999999996,"15min":0.84099999999999997},"min":{"1min":0.71399999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.123,"5min":1.1910000000000001,"15min":1.202},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83499999999999996,"15min":0.86299999999999999},"min":{"1min":0.57299999999999995,"5min":0.54900000000000004,"15min":0.497},"max":{"1min":1.095,"5min":1.1639999999999999,"15min":1.3160000000000001},"last":0.73099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574983,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1556480,"data_stored":719989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56899999999999995,"5min":0.70299999999999996,"15min":4.0999999999999996},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.48799999999999999,"15min":0.50600000000000001},"min":{"1min":0.40699999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83999999999999997,"5min":0.84099999999999997,"15min":1.9239999999999999},"last":0.42699999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32600000000000001},"max":{"1min":0.61799999999999999,"5min":0.81299999999999994,"15min":3.1139999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.52400000000000002,"15min":0.53400000000000003},"min":{"1min":0.40600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.748,"5min":0.752,"15min":2.4159999999999999},"last":0.48699999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.499,"15min":0.497},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70099999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.749,"5min":0.88800000000000001,"15min":1.907},"last":0.45300000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77700000000000002,"15min":0.80100000000000005},"min":{"1min":0.59699999999999998,"5min":0.504,"15min":0.504},"max":{"1min":1.0900000000000001,"5min":1.1919999999999999,"15min":1.9470000000000001},"last":0.628},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78700000000000003,"15min":0.82099999999999995},"min":{"1min":0.628,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.163,"5min":1.2110000000000001,"15min":1.972},"last":0.73899999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.1220000000000001,"5min":1.2549999999999999,"15min":2.4569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.78900000000000003,"15min":0.81000000000000005},"min":{"1min":0.58399999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0389999999999999,"5min":1.286,"15min":2.8039999999999998},"last":0.60899999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0980000000000001,"5min":1.177,"15min":2.8849999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.0780000000000001,"5min":1.242,"15min":1.867},"last":0.67500000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79500000000000004,"15min":0.83699999999999997},"min":{"1min":0.60199999999999998,"5min":0.54700000000000004,"15min":0.495},"max":{"1min":1.0429999999999999,"5min":1.1140000000000001,"15min":2.3690000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.83799999999999997},"min":{"1min":0.70499999999999996,"5min":0.55800000000000005,"15min":0.52700000000000002},"max":{"1min":0.98999999999999999,"5min":1.365,"15min":2.851},"last":0.65500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1560576,"data_stored":720011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44,"15min":0.42799999999999999},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.253},"max":{"1min":0.63600000000000001,"5min":0.72199999999999998,"15min":0.80300000000000005},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.48399999999999999,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.69999999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49399999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.45900000000000002},"min":{"1min":0.39200000000000002,"5min":0.29999999999999999,"15min":0.29199999999999998},"max":{"1min":0.59199999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.64600000000000002,"5min":0.84099999999999997,"15min":0.89400000000000002},"last":0.52100000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.45300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.69199999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.38600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72999999999999998,"5min":0.83199999999999996,"15min":0.83999999999999997},"last":0.63300000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.75600000000000001,"15min":0.72899999999999998},"min":{"1min":0.58099999999999996,"5min":0.49199999999999999,"15min":0.38800000000000001},"max":{"1min":1.034,"5min":1.175,"15min":1.175},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76900000000000002,"15min":0.754},"min":{"1min":0.66300000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98599999999999999,"5min":1.079,"15min":1.2},"last":0.82999999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.76000000000000001},"min":{"1min":0.621,"5min":0.56799999999999995,"15min":0.51600000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.3180000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79400000000000004,"15min":0.77100000000000002},"min":{"1min":0.60299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.129,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.78600000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79700000000000004,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.0389999999999999,"5min":1.179,"15min":1.179},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0509999999999999,"15min":1.2150000000000001},"last":0.86699999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80300000000000005,"15min":0.79700000000000004},"min":{"1min":0.64500000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.024,"5min":1.2050000000000001,"15min":1.2669999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.81799999999999995,"15min":0.79200000000000004},"min":{"1min":0.69299999999999995,"5min":0.56200000000000006,"15min":0.48499999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.236},"last":0.85199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574984,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33220,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338620,"statfs":{"total":96636764160,"available":96602746880,"internally_reserved":0,"allocated":2154496,"data_stored":1310379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.45700000000000002},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.245},"max":{"1min":0.69799999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68400000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.51000000000000001},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.67100000000000004,"5min":0.82599999999999996,"15min":1.02},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.432,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":0.77900000000000003,"15min":1.006},"last":0.42399999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.68000000000000005,"5min":0.754,"15min":0.84799999999999998},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.46200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":0.88900000000000001,"15min":0.96899999999999997},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.47899999999999998},"max":{"1min":1.1040000000000001,"5min":1.1140000000000001,"15min":1.5169999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.46400000000000002},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.252},"last":0.69799999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.55200000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.3080000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0780000000000001,"15min":1.462},"last":0.57899999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79200000000000004,"15min":0.79900000000000004},"min":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.54500000000000004},"max":{"1min":1.0429999999999999,"5min":1.131,"15min":1.5920000000000001},"last":0.626},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80000000000000004,"15min":0.79000000000000004},"min":{"1min":0.625,"5min":0.54600000000000004,"15min":0.46999999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.2230000000000001},"last":0.59699999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.81299999999999994},"min":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.3600000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.60399999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.556},"last":0.64300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:50:47.090 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T00:50:47.090 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-30T00:50:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.267+0000 7f5428200640 1 Processor -- start 2024-03-30T00:50:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.267+0000 7f5428200640 1 -- start start 2024-03-30T00:50:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.267+0000 7f5428200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f542015ca00 0x7f54200ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.267+0000 7f5428200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54200b16e0 con 0x7f5420097640 2024-03-30T00:50:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.267+0000 7f5428200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54200b1980 con 0x7f54200b11d0 2024-03-30T00:50:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.267+0000 7f5428200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54200b1c20 con 0x7f542015ca00 2024-03-30T00:50:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.267+0000 7f5426200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5420097640 0x7f54200abc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53508/0 (socket says 172.21.15.66:53508) 2024-03-30T00:50:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.267+0000 7f5426200640 1 -- 172.21.15.66:0/2947090293 learned_addr learned my addr 172.21.15.66:0/2947090293 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:47.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.271+0000 7f5425800640 1 --2- 172.21.15.66:0/2947090293 >> v2:172.21.15.156:3300/0 conn(0x7f542015ca00 0x7f54200ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:47.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.271+0000 7f541fe00640 1 -- 172.21.15.66:0/2947090293 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207657135 0 0) 0x7f54200b16e0 con 0x7f5420097640 2024-03-30T00:50:47.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.271+0000 7f541fe00640 1 -- 172.21.15.66:0/2947090293 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5408003530 con 0x7f5420097640 2024-03-30T00:50:47.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.271+0000 7f5425800640 1 -- 172.21.15.66:0/2947090293 >> v1:172.21.15.156:6789/0 conn(0x7f54200b11d0 legacy=0x7f54200ab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.271+0000 7f5425800640 1 -- 172.21.15.66:0/2947090293 >> v1:172.21.15.66:6789/0 conn(0x7f5420097640 legacy=0x7f54200abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.271+0000 7f5425800640 1 -- 172.21.15.66:0/2947090293 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200b1ee0 con 0x7f542015ca00 2024-03-30T00:50:47.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.271+0000 7f5425800640 1 --2- 172.21.15.66:0/2947090293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f542015ca00 0x7f54200ce780 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f54140097c0 tx=0x7f541401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d16395b67df76 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:47.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.271+0000 7f541fe00640 1 -- 172.21.15.66:0/2947090293 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54140aa620 con 0x7f542015ca00 2024-03-30T00:50:47.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.271+0000 7f541fe00640 1 -- 172.21.15.66:0/2947090293 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54140aa780 con 0x7f542015ca00 2024-03-30T00:50:47.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.271+0000 7f541fe00640 1 -- 172.21.15.66:0/2947090293 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54140aaaf0 con 0x7f542015ca00 2024-03-30T00:50:47.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 -- 172.21.15.66:0/2947090293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f542015ca00 msgr2=0x7f54200ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 --2- 172.21.15.66:0/2947090293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f542015ca00 0x7f54200ce780 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f54140097c0 tx=0x7f541401c4a0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 -- 172.21.15.66:0/2947090293 shutdown_connections 2024-03-30T00:50:47.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 --2- 172.21.15.66:0/2947090293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f542015ca00 0x7f54200ce780 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 -- 172.21.15.66:0/2947090293 >> 172.21.15.66:0/2947090293 conn(0x7f542006a190 msgr2=0x7f542006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:47.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 -- 172.21.15.66:0/2947090293 shutdown_connections 2024-03-30T00:50:47.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 -- 172.21.15.66:0/2947090293 wait complete. 2024-03-30T00:50:47.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 Processor -- start 2024-03-30T00:50:47.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 -- start start 2024-03-30T00:50:47.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5420097640 0x7f5420164ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54200b11d0 0x7f54201f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f542015ca00 0x7f54200ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5420165020 con 0x7f5420097640 2024-03-30T00:50:47.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54201f7bd0 con 0x7f54200b11d0 2024-03-30T00:50:47.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5428200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54201f7eb0 con 0x7f542015ca00 2024-03-30T00:50:47.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5425800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5420097640 0x7f5420164ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:47.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5425800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5420097640 0x7f5420164ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58950/0 (socket says 172.21.15.66:58950) 2024-03-30T00:50:47.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5425800640 1 -- 172.21.15.66:0/1817819030 learned_addr learned my addr 172.21.15.66:0/1817819030 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:47.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5425800640 1 -- 172.21.15.66:0/1817819030 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f542015ca00 msgr2=0x7f54200ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5425800640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f542015ca00 0x7f54200ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5425800640 1 -- 172.21.15.66:0/1817819030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54200b11d0 msgr2=0x7f54201f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5424e00640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54200b11d0 0x7f54201f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:47.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5425800640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54200b11d0 0x7f54201f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5425800640 1 -- 172.21.15.66:0/1817819030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414009420 con 0x7f5420097640 2024-03-30T00:50:47.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5424e00640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54200b11d0 0x7f54201f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:47.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.275+0000 7f5425800640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5420097640 0x7f5420164ae0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f54140097c0 tx=0x7f5414019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:47.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.279+0000 7f541d600640 1 -- 172.21.15.66:0/1817819030 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414004450 con 0x7f5420097640 2024-03-30T00:50:47.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.279+0000 7f541d600640 1 -- 172.21.15.66:0/1817819030 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54140aa8f0 con 0x7f5420097640 2024-03-30T00:50:47.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.279+0000 7f5428200640 1 -- 172.21.15.66:0/1817819030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54201f8190 con 0x7f5420097640 2024-03-30T00:50:47.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.279+0000 7f541d600640 1 -- 172.21.15.66:0/1817819030 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54140aac60 con 0x7f5420097640 2024-03-30T00:50:47.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.279+0000 7f5428200640 1 -- 172.21.15.66:0/1817819030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54201f8750 con 0x7f5420097640 2024-03-30T00:50:47.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.279+0000 7f5428200640 1 -- 172.21.15.66:0/1817819030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54201f8940 con 0x7f5420097640 2024-03-30T00:50:47.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.279+0000 7f541d600640 1 -- 172.21.15.66:0/1817819030 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5414003620 con 0x7f5420097640 2024-03-30T00:50:47.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.279+0000 7f541d600640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54000666c0 0x7f5400068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.279+0000 7f541d600640 1 -- 172.21.15.66:0/1817819030 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f541411e990 con 0x7f5420097640 2024-03-30T00:50:47.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.279+0000 7f5424e00640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54000666c0 0x7f5400068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:47.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.283+0000 7f5424e00640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54000666c0 0x7f5400068b80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f542006eb20 tx=0x7f540c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:47.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.283+0000 7f541d600640 1 -- 172.21.15.66:0/1817819030 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f54140ebc40 con 0x7f5420097640 2024-03-30T00:50:47.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.503+0000 7f5428200640 1 -- 172.21.15.66:0/1817819030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5420066c30 con 0x7f5420097640 2024-03-30T00:50:47.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f541d600640 1 -- 172.21.15.66:0/1817819030 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f54140c00e0 con 0x7f5420097640 2024-03-30T00:50:47.514 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-03-30T00:50:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 -- 172.21.15.66:0/1817819030 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54000666c0 msgr2=0x7f5400068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54000666c0 0x7f5400068b80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f542006eb20 tx=0x7f540c009040 comp rx=0 tx=0).stop 2024-03-30T00:50:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 -- 172.21.15.66:0/1817819030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5420097640 msgr2=0x7f5420164ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5420097640 0x7f5420164ae0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f54140097c0 tx=0x7f5414019240 comp rx=0 tx=0).stop 2024-03-30T00:50:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 -- 172.21.15.66:0/1817819030 shutdown_connections 2024-03-30T00:50:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f542015ca00 0x7f54200ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54200b11d0 0x7f54201f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f54000666c0 0x7f5400068b80 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 --2- 172.21.15.66:0/1817819030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5420097640 0x7f5420164ae0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 -- 172.21.15.66:0/1817819030 >> 172.21.15.66:0/1817819030 conn(0x7f542006a190 msgr2=0x7f5420070170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:47.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 -- 172.21.15.66:0/1817819030 shutdown_connections 2024-03-30T00:50:47.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.507+0000 7f5428200640 1 -- 172.21.15.66:0/1817819030 wait complete. 2024-03-30T00:50:47.535 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:50:47.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70ba000640 1 Processor -- start 2024-03-30T00:50:47.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70ba000640 1 -- start start 2024-03-30T00:50:47.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70ba000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f70b40cec30 0x7f70b4159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70ba000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70b40cde70 0x7f70b415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70ba000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b415a6e0 con 0x7f70b40cda90 2024-03-30T00:50:47.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70ba000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b40b1b80 con 0x7f70b40cec30 2024-03-30T00:50:47.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70ba000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40cd3e0 con 0x7f70b40cde70 2024-03-30T00:50:47.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70b3400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f70b40cda90 0x7f70b40ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53512/0 (socket says 172.21.15.66:53512) 2024-03-30T00:50:47.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70b3400640 1 -- 172.21.15.66:0/3123291034 learned_addr learned my addr 172.21.15.66:0/3123291034 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:47.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70aaa00640 1 --2- 172.21.15.66:0/3123291034 >> v2:172.21.15.156:3300/0 conn(0x7f70b40cec30 0x7f70b4159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:47.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70b3e00640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70b40cde70 0x7f70b415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797696268 0 0) 0x7f70b415a6e0 con 0x7f70b40cda90 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7098003530 con 0x7f70b40cda90 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3083513275 0 0) 0x7f7098003530 con 0x7f70b40cda90 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70b40cde70 msgr2=0x7f70b415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70b2a00640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70b40cde70 0x7f70b415a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.667+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 >> v2:172.21.15.156:3300/0 conn(0x7f70b40cec30 msgr2=0x7f70b4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 --2- 172.21.15.66:0/3123291034 >> v2:172.21.15.156:3300/0 conn(0x7f70b40cec30 0x7f70b4159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b415a990 con 0x7f70b40cda90 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f70a4001680 con 0x7f70b40cda90 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 >> v1:172.21.15.66:6789/0 conn(0x7f70b40cda90 legacy=0x7f70b40ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70b40cde70 0x7f7098003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7098003f50 0x7f70980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7098004d10 0x7f7098005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40b1b80 con 0x7f7098003f50 2024-03-30T00:50:47.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40cd3e0 con 0x7f7098004d10 2024-03-30T00:50:47.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709800ba40 con 0x7f70b40cde70 2024-03-30T00:50:47.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70aaa00640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7098003f50 0x7f70980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:47.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b3400640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70b40cde70 0x7f7098003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:47.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b3e00640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7098004d10 0x7f7098005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:47.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70aaa00640 1 -- 172.21.15.66:0/3123291034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70b40cde70 msgr2=0x7f7098003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:47.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70aaa00640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70b40cde70 0x7f7098003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70aaa00640 1 -- 172.21.15.66:0/3123291034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7098004d10 msgr2=0x7f7098005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:47.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70aaa00640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7098004d10 0x7f7098005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70aaa00640 1 -- 172.21.15.66:0/3123291034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7098003530 con 0x7f7098003f50 2024-03-30T00:50:47.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70aaa00640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7098003f50 0x7f70980047d0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f709c009ab0 tx=0x7f709c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12427d6dcd67e02e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:47.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c0a5570 con 0x7f7098003f50 2024-03-30T00:50:47.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c0a56d0 con 0x7f7098003f50 2024-03-30T00:50:47.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.671+0000 7f70b2a00640 1 -- 172.21.15.66:0/3123291034 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c0a5a80 con 0x7f7098003f50 2024-03-30T00:50:47.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 -- 172.21.15.66:0/3123291034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7098003f50 msgr2=0x7f70980047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 --2- 172.21.15.66:0/3123291034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7098003f50 0x7f70980047d0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f709c009ab0 tx=0x7f709c01ca90 comp rx=0 tx=0).stop 2024-03-30T00:50:47.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70b3e00640 1 -- 172.21.15.66:0/3123291034 reap_dead start 2024-03-30T00:50:47.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 -- 172.21.15.66:0/3123291034 shutdown_connections 2024-03-30T00:50:47.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 -- 172.21.15.66:0/3123291034 >> 172.21.15.66:0/3123291034 conn(0x7f70b4069f60 msgr2=0x7f70b406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:47.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 -- 172.21.15.66:0/3123291034 shutdown_connections 2024-03-30T00:50:47.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 -- 172.21.15.66:0/3123291034 wait complete. 2024-03-30T00:50:47.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 Processor -- start 2024-03-30T00:50:47.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 -- start start 2024-03-30T00:50:47.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7098003f50 0x7f70b4097940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70b40ad2b0 0x7f70b40ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70b406d390 0x7f70b4162620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b4071f60 con 0x7f70b406d390 2024-03-30T00:50:47.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b4097e80 con 0x7f70b40ad2b0 2024-03-30T00:50:47.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70ba000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b4070290 con 0x7f7098003f50 2024-03-30T00:50:47.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70b406d390 0x7f70b4162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:47.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70b406d390 0x7f70b4162620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58964/0 (socket says 172.21.15.66:58964) 2024-03-30T00:50:47.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70b3e00640 1 -- 172.21.15.66:0/3742569948 learned_addr learned my addr 172.21.15.66:0/3742569948 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:47.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70b3e00640 1 -- 172.21.15.66:0/3742569948 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7098003f50 msgr2=0x7f70b4097940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70b3e00640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7098003f50 0x7f70b4097940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70b3e00640 1 -- 172.21.15.66:0/3742569948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70b40ad2b0 msgr2=0x7f70b40ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70b3e00640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70b40ad2b0 0x7f70b40ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70b3e00640 1 -- 172.21.15.66:0/3742569948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b4071c80 con 0x7f70b406d390 2024-03-30T00:50:47.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.675+0000 7f70b3e00640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70b406d390 0x7f70b4162620 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f70b40cc320 tx=0x7f70ac00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:47.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.679+0000 7f70abe00640 1 -- 172.21.15.66:0/3742569948 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac00e040 con 0x7f70b406d390 2024-03-30T00:50:47.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.679+0000 7f70abe00640 1 -- 172.21.15.66:0/3742569948 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ac00f040 con 0x7f70b406d390 2024-03-30T00:50:47.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.679+0000 7f70abe00640 1 -- 172.21.15.66:0/3742569948 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac015430 con 0x7f70b406d390 2024-03-30T00:50:47.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.679+0000 7f70ba000640 1 -- 172.21.15.66:0/3742569948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b4151bd0 con 0x7f70b406d390 2024-03-30T00:50:47.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.679+0000 7f70ba000640 1 -- 172.21.15.66:0/3742569948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b406fed0 con 0x7f70b406d390 2024-03-30T00:50:47.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.679+0000 7f70ba000640 1 -- 172.21.15.66:0/3742569948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b4151a30 con 0x7f70b406d390 2024-03-30T00:50:47.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.679+0000 7f70abe00640 1 -- 172.21.15.66:0/3742569948 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f70ac015590 con 0x7f70b406d390 2024-03-30T00:50:47.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.679+0000 7f70abe00640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7090066690 0x7f7090068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:47.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.679+0000 7f70abe00640 1 -- 172.21.15.66:0/3742569948 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f70ac084930 con 0x7f70b406d390 2024-03-30T00:50:47.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.679+0000 7f70b3400640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7090066690 0x7f7090068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:47.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.683+0000 7f70b3400640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7090066690 0x7f7090068b50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f70a4007bd0 tx=0x7f70a4000f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:47.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.683+0000 7f70abe00640 1 -- 172.21.15.66:0/3742569948 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f70b4151a30 con 0x7f70b406d390 2024-03-30T00:50:47.698 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:50:47.701 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:50:47.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 -- 172.21.15.66:0/3742569948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70b4151a30 con 0x7f70b406d390 2024-03-30T00:50:47.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70abe00640 1 -- 172.21.15.66:0/3742569948 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4739 (secure 0 0 0) 0x7f70b4151a30 con 0x7f70b406d390 2024-03-30T00:50:47.890 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:47.890 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":95,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:50:41.054441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9366":{"gid":9366,"name":"a","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6836/2683418851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2683418851},{"type":"v1","addr":"172.21.15.66:6836","nonce":2683418851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:50:47.891 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 95 2024-03-30T00:50:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 -- 172.21.15.66:0/3742569948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7090066690 msgr2=0x7f7090068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7090066690 0x7f7090068b50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f70a4007bd0 tx=0x7f70a4000f60 comp rx=0 tx=0).stop 2024-03-30T00:50:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 -- 172.21.15.66:0/3742569948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70b406d390 msgr2=0x7f70b4162620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70b406d390 0x7f70b4162620 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f70b40cc320 tx=0x7f70ac00f580 comp rx=0 tx=0).stop 2024-03-30T00:50:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 -- 172.21.15.66:0/3742569948 shutdown_connections 2024-03-30T00:50:47.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7090066690 0x7f7090068b50 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70b406d390 0x7f70b4162620 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70b40ad2b0 0x7f70b40ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 --2- 172.21.15.66:0/3742569948 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7098003f50 0x7f70b4097940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:47.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 -- 172.21.15.66:0/3742569948 >> 172.21.15.66:0/3742569948 conn(0x7f70b4069f60 msgr2=0x7f70b40afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:47.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 -- 172.21.15.66:0/3742569948 shutdown_connections 2024-03-30T00:50:47.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:47.883+0000 7f70ba000640 1 -- 172.21.15.66:0/3742569948 wait complete. 2024-03-30T00:50:48.315 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:50:38.040525+0000', 'modified': '2024-03-30T00:50:41.054441+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9366}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9366': {'gid': 9366, 'name': 'a', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6836/2683418851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 2683418851}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 2683418851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:50:48.315 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:50:48.315 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-30T00:50:48.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.451+0000 7f6bae000640 1 Processor -- start 2024-03-30T00:50:48.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.451+0000 7f6bae000640 1 -- start start 2024-03-30T00:50:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.451+0000 7f6bae000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6ba815ca80 0x7f6ba806b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.451+0000 7f6bae000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6ba80cf2f0 0x7f6ba806b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.451+0000 7f6bae000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba80cb0e0 con 0x7f6ba806bb30 2024-03-30T00:50:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.451+0000 7f6bae000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba80cf7f0 con 0x7f6ba815ca80 2024-03-30T00:50:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.451+0000 7f6bae000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba80cfad0 con 0x7f6ba80cf2f0 2024-03-30T00:50:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.451+0000 7f6ba7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6ba815ca80 0x7f6ba806b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.451+0000 7f6ba7e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6ba806bb30 0x7f6ba8098a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53520/0 (socket says 172.21.15.66:53520) 2024-03-30T00:50:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.451+0000 7f6ba7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6ba815ca80 0x7f6ba806b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58978/0 (socket says 172.21.15.66:58978) 2024-03-30T00:50:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.451+0000 7f6ba7e00640 1 -- 172.21.15.66:0/2884743280 learned_addr learned my addr 172.21.15.66:0/2884743280 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.455+0000 7f6b9ea00640 1 --2- 172.21.15.66:0/2884743280 >> v2:172.21.15.156:3300/0 conn(0x7f6ba80cf2f0 0x7f6ba806b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.455+0000 7f6ba7400640 1 -- 172.21.15.66:0/2884743280 >> v2:172.21.15.156:3300/0 conn(0x7f6ba80cf2f0 msgr2=0x7f6ba806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.455+0000 7f6ba7400640 1 --2- 172.21.15.66:0/2884743280 >> v2:172.21.15.156:3300/0 conn(0x7f6ba80cf2f0 0x7f6ba806b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.455+0000 7f6ba7400640 1 -- 172.21.15.66:0/2884743280 >> v1:172.21.15.66:6789/0 conn(0x7f6ba806bb30 legacy=0x7f6ba8098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.455+0000 7f6ba7400640 1 -- 172.21.15.66:0/2884743280 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba80cad70 con 0x7f6ba815ca80 2024-03-30T00:50:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.455+0000 7f6ba6a00640 1 -- 172.21.15.66:0/2884743280 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642667231 0 0) 0x7f6ba80cb0e0 con 0x7f6ba806bb30 2024-03-30T00:50:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.455+0000 7f6ba7400640 1 --2- 172.21.15.66:0/2884743280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba815ca80 0x7f6ba806b0b0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f6ba806ced0 tx=0x7f6b9801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766a536c236e22ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.455+0000 7f6ba6a00640 1 -- 172.21.15.66:0/2884743280 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980a8970 con 0x7f6ba815ca80 2024-03-30T00:50:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.455+0000 7f6ba6a00640 1 -- 172.21.15.66:0/2884743280 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b980a8ad0 con 0x7f6ba815ca80 2024-03-30T00:50:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.455+0000 7f6ba6a00640 1 -- 172.21.15.66:0/2884743280 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980a8e60 con 0x7f6ba815ca80 2024-03-30T00:50:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 -- 172.21.15.66:0/2884743280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba815ca80 msgr2=0x7f6ba806b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 --2- 172.21.15.66:0/2884743280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba815ca80 0x7f6ba806b0b0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f6ba806ced0 tx=0x7f6b9801ec70 comp rx=0 tx=0).stop 2024-03-30T00:50:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 -- 172.21.15.66:0/2884743280 shutdown_connections 2024-03-30T00:50:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 --2- 172.21.15.66:0/2884743280 >> v2:172.21.15.156:3300/0 conn(0x7f6ba80cf2f0 0x7f6ba806b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 --2- 172.21.15.66:0/2884743280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba815ca80 0x7f6ba806b0b0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 -- 172.21.15.66:0/2884743280 >> 172.21.15.66:0/2884743280 conn(0x7f6ba80b2090 msgr2=0x7f6ba80b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 -- 172.21.15.66:0/2884743280 shutdown_connections 2024-03-30T00:50:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 -- 172.21.15.66:0/2884743280 wait complete. 2024-03-30T00:50:48.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 Processor -- start 2024-03-30T00:50:48.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 -- start start 2024-03-30T00:50:48.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ba806bb30 0x7f6ba815a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ba80cf2f0 0x7f6ba806ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba815ca80 0x7f6ba806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba809dd80 con 0x7f6ba815ca80 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba809e060 con 0x7f6ba806bb30 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6bae000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80c8980 con 0x7f6ba80cf2f0 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba815ca80 0x7f6ba806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba815ca80 0x7f6ba806f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58986/0 (socket says 172.21.15.66:58986) 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6ba7e00640 1 -- 172.21.15.66:0/478241201 learned_addr learned my addr 172.21.15.66:0/478241201 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6b9ea00640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ba80cf2f0 0x7f6ba806ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6ba7e00640 1 -- 172.21.15.66:0/478241201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ba80cf2f0 msgr2=0x7f6ba806ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ba80cf2f0 0x7f6ba806ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6ba7400640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ba806bb30 0x7f6ba815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6ba7e00640 1 -- 172.21.15.66:0/478241201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ba806bb30 msgr2=0x7f6ba815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ba806bb30 0x7f6ba815a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6ba7e00640 1 -- 172.21.15.66:0/478241201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b98009420 con 0x7f6ba815ca80 2024-03-30T00:50:48.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6ba7400640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ba806bb30 0x7f6ba815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:48.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.459+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba815ca80 0x7f6ba806f140 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c7a0 tx=0x7f6ba000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:48.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6b9fe00640 1 -- 172.21.15.66:0/478241201 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000a640 con 0x7f6ba815ca80 2024-03-30T00:50:48.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6b9fe00640 1 -- 172.21.15.66:0/478241201 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba000a7a0 con 0x7f6ba815ca80 2024-03-30T00:50:48.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6b9fe00640 1 -- 172.21.15.66:0/478241201 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000abd0 con 0x7f6ba815ca80 2024-03-30T00:50:48.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6bae000640 1 -- 172.21.15.66:0/478241201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba80c8c60 con 0x7f6ba815ca80 2024-03-30T00:50:48.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6bae000640 1 -- 172.21.15.66:0/478241201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba809d010 con 0x7f6ba815ca80 2024-03-30T00:50:48.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6b9fe00640 1 -- 172.21.15.66:0/478241201 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6ba000ad30 con 0x7f6ba815ca80 2024-03-30T00:50:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6b9fe00640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6b84066690 0x7f6b84068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6b9fe00640 1 -- 172.21.15.66:0/478241201 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6ba0083880 con 0x7f6ba815ca80 2024-03-30T00:50:48.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6ba7400640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6b84066690 0x7f6b84068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6bb13ea000 1 Processor -- start 2024-03-30T00:50:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6bb13ea000 1 -- start start 2024-03-30T00:50:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.463+0000 7f6ba7400640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6b84066690 0x7f6b84068b50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6b980b2000 tx=0x7f6b980aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:48.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b6f4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b6aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801b5725a0 con 0x55801b6f46d0 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801b4ffaf0 con 0x55801b6f3db0 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55801b4fd0c0 con 0x55801b6aa4c0 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b9ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b9ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6a9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58994/0 (socket says 172.21.15.66:58994) 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b9ea00640 1 -- 172.21.15.66:0/580078132 learned_addr learned my addr 172.21.15.66:0/580078132 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/580078132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b6aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6ba7400640 1 --2- 172.21.15.66:0/580078132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b6f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b9ea00640 1 -- 172.21.15.66:0/580078132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 msgr2=0x55801b6aa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b9ea00640 1 --2- 172.21.15.66:0/580078132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b6aa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b9ea00640 1 -- 172.21.15.66:0/580078132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 msgr2=0x55801b6f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b9ea00640 1 --2- 172.21.15.66:0/580078132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b6f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b9ea00640 1 -- 172.21.15.66:0/580078132 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55801b1f4890 con 0x55801b6f46d0 2024-03-30T00:50:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b9ea00640 1 --2- 172.21.15.66:0/580078132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6a9f80 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000b910 tx=0x7f6b9000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c699ecb1aca09c03 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b97e00640 1 -- 172.21.15.66:0/580078132 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9000fa20 con 0x55801b6f46d0 2024-03-30T00:50:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b97e00640 1 -- 172.21.15.66:0/580078132 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9000fb80 con 0x55801b6f46d0 2024-03-30T00:50:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6b97e00640 1 -- 172.21.15.66:0/580078132 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90014460 con 0x55801b6f46d0 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 -- 172.21.15.66:0/580078132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 msgr2=0x55801b6a9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/580078132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6a9f80 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000b910 tx=0x7f6b9000bdd0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 -- 172.21.15.66:0/580078132 shutdown_connections 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/580078132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b6aa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/580078132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6a9f80 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/580078132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b6f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 -- 172.21.15.66:0/580078132 >> 172.21.15.66:0/580078132 conn(0x55801b6b0830 msgr2=0x55801b6a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 -- 172.21.15.66:0/580078132 shutdown_connections 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 -- 172.21.15.66:0/580078132 wait complete. 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 -- start start 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 Processor -- start 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b6ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b6ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.467+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801b4f89b0 con 0x55801b6f46d0 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801b497c00 con 0x55801b6f3db0 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55801b4b6070 con 0x55801b6aa4c0 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 10 client.0 _ll_register_callbacks cb 0x55801b6960e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59008/0 (socket says 172.21.15.66:59008) 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7e00640 1 -- 172.21.15.66:0/536180508 learned_addr learned my addr 172.21.15.66:0/536180508 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7400640 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b6ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b9ea00640 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b6ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7e00640 1 -- 172.21.15.66:0/536180508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 msgr2=0x55801b6ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b6ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7e00640 1 -- 172.21.15.66:0/536180508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 msgr2=0x55801b6ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b6ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7e00640 1 -- 172.21.15.66:0/536180508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9000e040 con 0x55801b6f46d0 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7400640 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b6ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b9ea00640 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b6ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6aecb0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0081c60 tx=0x7f6ba0054f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba790befd835b4b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b94c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b94c00640 1 -- 172.21.15.66:0/536180508 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba005c8c0 con 0x55801b6f46d0 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b94c00640 1 -- 172.21.15.66:0/536180508 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6ba005cb50 con 0x55801b6f46d0 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 -- 172.21.15.66:0/536180508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55801b4b55f0 con 0x55801b6f46d0 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b94c00640 1 -- 172.21.15.66:0/536180508 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ba006c170 con 0x55801b6f46d0 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 10 client.9534 fetch_fsmap learned FSMap version 95 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 -- 172.21.15.66:0/536180508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x55801b4bdab0 con 0x55801b6f46d0 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b94c00640 1 -- 172.21.15.66:0/536180508 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55801b4b55f0 con 0x55801b6f46d0 2024-03-30T00:50:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b94c00640 1 -- 172.21.15.66:0/536180508 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 95) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6ba00580b0 con 0x55801b6f46d0 2024-03-30T00:50:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 10 client.9534 fetch_fsmap finished waiting for FSMap version 95 2024-03-30T00:50:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 10 client.9534 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:50:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 20 client.9534 populate_metadata read hostname 'smithi066' 2024-03-30T00:50:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x55801b6af6c0 0x55801b704e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 4 client.9534 mds_command: new command op to 9366 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:50:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 -- 172.21.15.66:0/536180508 --> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55801b4f5790 con 0x55801b6af6c0 2024-03-30T00:50:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7400640 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x55801b6af6c0 0x55801b704e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6ba7400640 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x55801b6af6c0 0x55801b704e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da6212754fd8e74 server_cookie=ae22010fbc61fcc9 in_seq=0 out_seq=0 2024-03-30T00:50:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b94c00640 10 client.9534 ms_handle_connect on v2:172.21.15.66:6834/2683418851 2024-03-30T00:50:48.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b94c00640 1 -- 172.21.15.66:0/536180508 <== mds.0 v2:172.21.15.66:6834/2683418851 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55801b4f5790 con 0x55801b6af6c0 2024-03-30T00:50:48.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b94c00640 10 client.9534 handle_command_reply: tid=0 2024-03-30T00:50:48.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 client.9534 shutdown 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 -- 172.21.15.66:0/536180508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 msgr2=0x55801b6aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6aecb0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0081c60 tx=0x7f6ba0054f20 comp rx=0 tx=0).stop 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 -- 172.21.15.66:0/536180508 shutdown_connections 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x55801b6af6c0 0x55801b704e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6aecb0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6b94c00640 0 client.9534 ms_handle_reset on v2:172.21.15.66:6834/2683418851 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b6ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/536180508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b6ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 -- 172.21.15.66:0/536180508 >> 172.21.15.66:0/536180508 conn(0x55801b6b0830 msgr2=0x55801b6a8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 -- 172.21.15.66:0/536180508 shutdown_connections 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 1 -- 172.21.15.66:0/536180508 wait complete. 2024-03-30T00:50:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.471+0000 7f6bb13ea000 20 client.9534 trim_cache size 0 max 16384 2024-03-30T00:50:48.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.487+0000 7f6bb13ea000 1 Processor -- start 2024-03-30T00:50:48.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.487+0000 7f6bb13ea000 1 -- start start 2024-03-30T00:50:48.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.487+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b74d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.487+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b74d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.487+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b74dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.487+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801b1b5940 con 0x55801b6f46d0 2024-03-30T00:50:48.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.487+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801b6a32e0 con 0x55801b6f3db0 2024-03-30T00:50:48.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.487+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55801b4fb510 con 0x55801b6aa4c0 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b74dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b74dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59024/0 (socket says 172.21.15.66:59024) 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 -- 172.21.15.66:0/1554799594 learned_addr learned my addr 172.21.15.66:0/1554799594 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7400640 1 --2- 172.21.15.66:0/1554799594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b74d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6b9ea00640 1 --2- 172.21.15.66:0/1554799594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b74d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 -- 172.21.15.66:0/1554799594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 msgr2=0x55801b74d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/1554799594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b74d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 -- 172.21.15.66:0/1554799594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 msgr2=0x55801b74d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/1554799594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b74d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 -- 172.21.15.66:0/1554799594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55801b4a3c00 con 0x55801b6f46d0 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6b9ea00640 1 --2- 172.21.15.66:0/1554799594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b74d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/1554799594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b74dac0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0050c10 tx=0x7f6ba0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ca25047d63cf0e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6b9cc00640 1 -- 172.21.15.66:0/1554799594 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0001840 con 0x55801b6f46d0 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6b9cc00640 1 -- 172.21.15.66:0/1554799594 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba006c170 con 0x55801b6f46d0 2024-03-30T00:50:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6b9cc00640 1 -- 172.21.15.66:0/1554799594 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba005cb50 con 0x55801b6f46d0 2024-03-30T00:50:48.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 -- 172.21.15.66:0/1554799594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 msgr2=0x55801b74dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/1554799594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b74dac0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0050c10 tx=0x7f6ba0013000 comp rx=0 tx=0).stop 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 -- 172.21.15.66:0/1554799594 shutdown_connections 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/1554799594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b74dac0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/1554799594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6f3db0 0x55801b74d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/1554799594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6aa4c0 0x55801b74d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 -- 172.21.15.66:0/1554799594 >> 172.21.15.66:0/1554799594 conn(0x55801b6b0830 msgr2=0x55801b6ada10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 -- 172.21.15.66:0/1554799594 shutdown_connections 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 -- 172.21.15.66:0/1554799594 wait complete. 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 -- start start 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 Processor -- start 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6aa4c0 0x55801b752010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6f3db0 0x55801b752550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801b599290 con 0x55801b6f46d0 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801b4c1ef0 con 0x55801b6aa4c0 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55801b4f5790 con 0x55801b6f3db0 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6bb13ea000 10 client.0 _ll_register_callbacks cb 0x55801b4bd590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6f19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59040/0 (socket says 172.21.15.66:59040) 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 -- 172.21.15.66:0/3568830517 learned_addr learned my addr 172.21.15.66:0/3568830517 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7400640 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6aa4c0 0x55801b752010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6b9ea00640 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6f3db0 0x55801b752550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 -- 172.21.15.66:0/3568830517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6f3db0 msgr2=0x55801b752550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6f3db0 0x55801b752550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 -- 172.21.15.66:0/3568830517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6aa4c0 msgr2=0x55801b752010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:48.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6aa4c0 0x55801b752010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.491+0000 7f6ba7e00640 1 -- 172.21.15.66:0/3568830517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55801b4bdab0 con 0x55801b6f46d0 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6ba7e00640 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6f19a0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0054460 tx=0x7f6ba0081b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5584d57db1a68ebe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6b94c00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6b94c00640 1 -- 172.21.15.66:0/3568830517 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0001840 con 0x55801b6f46d0 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6b94c00640 1 -- 172.21.15.66:0/3568830517 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6ba005cb50 con 0x55801b6f46d0 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 -- 172.21.15.66:0/3568830517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55801b4c8460 con 0x55801b6f46d0 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6b94c00640 1 -- 172.21.15.66:0/3568830517 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ba005c8d0 con 0x55801b6f46d0 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 10 client.9540 fetch_fsmap learned FSMap version 95 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 -- 172.21.15.66:0/3568830517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x55801b1f4cc0 con 0x55801b6f46d0 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6b94c00640 1 -- 172.21.15.66:0/3568830517 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba0066080 con 0x55801b6f46d0 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6b94c00640 1 -- 172.21.15.66:0/3568830517 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 95) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6ba0079030 con 0x55801b6f46d0 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 10 client.9540 fetch_fsmap finished waiting for FSMap version 95 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 10 client.9540 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 20 client.9540 populate_metadata read hostname 'smithi066' 2024-03-30T00:50:48.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x55801b6f1ee0 0x55801b6f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 4 client.9540 mds_command: new command op to 9366 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:50:48.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 -- 172.21.15.66:0/3568830517 --> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55801b4c19c0 con 0x55801b6f1ee0 2024-03-30T00:50:48.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6ba7400640 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x55801b6f1ee0 0x55801b6f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6ba7400640 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x55801b6f1ee0 0x55801b6f22c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3013c0fc8177b31a server_cookie=4c0fe4171e510844 in_seq=0 out_seq=0 2024-03-30T00:50:48.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6b94c00640 10 client.9540 ms_handle_connect on v2:172.21.15.66:6834/2683418851 2024-03-30T00:50:48.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6b94c00640 1 -- 172.21.15.66:0/3568830517 <== mds.0 v2:172.21.15.66:6834/2683418851 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55801b4c19c0 con 0x55801b6f1ee0 2024-03-30T00:50:48.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6b94c00640 10 client.9540 handle_command_reply: tid=0 2024-03-30T00:50:48.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 client.9540 shutdown 2024-03-30T00:50:48.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 -- 172.21.15.66:0/3568830517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 msgr2=0x55801b6f19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6f19a0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0054460 tx=0x7f6ba0081b30 comp rx=0 tx=0).stop 2024-03-30T00:50:48.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 -- 172.21.15.66:0/3568830517 shutdown_connections 2024-03-30T00:50:48.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x55801b6f1ee0 0x55801b6f22c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55801b6f46d0 0x55801b6f19a0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55801b6f3db0 0x55801b752550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 --2- 172.21.15.66:0/3568830517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55801b6aa4c0 0x55801b752010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6b94c00640 0 client.9540 ms_handle_reset on v2:172.21.15.66:6834/2683418851 2024-03-30T00:50:48.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 -- 172.21.15.66:0/3568830517 >> 172.21.15.66:0/3568830517 conn(0x55801b6b0830 msgr2=0x55801b6ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:48.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 -- 172.21.15.66:0/3568830517 shutdown_connections 2024-03-30T00:50:48.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 1 -- 172.21.15.66:0/3568830517 wait complete. 2024-03-30T00:50:48.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bb13ea000 20 client.9540 trim_cache size 0 max 16384 2024-03-30T00:50:48.508 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:50:48.508 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:50:48.508 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:50:48.509 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 9366, 2024-03-30T00:50:48.509 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:50:48.509 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:50:48.509 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:50:48.509 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 10.455341551, 2024-03-30T00:50:48.509 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 95, 2024-03-30T00:50:48.509 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 125, 2024-03-30T00:50:48.509 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 125, 2024-03-30T00:50:48.509 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 15.648195936 2024-03-30T00:50:48.509 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 -- 172.21.15.66:0/478241201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6b84066690 msgr2=0x7f6b84068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6b84066690 0x7f6b84068b50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6b980b2000 tx=0x7f6b980aa070 comp rx=0 tx=0).stop 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 -- 172.21.15.66:0/478241201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba815ca80 msgr2=0x7f6ba806f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba815ca80 0x7f6ba806f140 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c7a0 tx=0x7f6ba000cc60 comp rx=0 tx=0).stop 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 -- 172.21.15.66:0/478241201 shutdown_connections 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6b84066690 0x7f6b84068b50 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba815ca80 0x7f6ba806f140 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ba80cf2f0 0x7f6ba806ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 --2- 172.21.15.66:0/478241201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ba806bb30 0x7f6ba815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 -- 172.21.15.66:0/478241201 >> 172.21.15.66:0/478241201 conn(0x7f6ba80b2090 msgr2=0x7f6ba80b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 -- 172.21.15.66:0/478241201 shutdown_connections 2024-03-30T00:50:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.495+0000 7f6bae000640 1 -- 172.21.15.66:0/478241201 wait complete. 2024-03-30T00:50:48.523 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:50:48.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.655+0000 7fba2b200640 1 Processor -- start 2024-03-30T00:50:48.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.655+0000 7fba2b200640 1 -- start start 2024-03-30T00:50:48.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba2b200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fba2c0cb130 0x7fba2c165050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba2b200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2c0ccb50 con 0x7fba2c15a5d0 2024-03-30T00:50:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba2b200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2c0cce30 con 0x7fba2c0d0d00 2024-03-30T00:50:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba2b200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2c0cc960 con 0x7fba2c0cb130 2024-03-30T00:50:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba29e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fba2c15a5d0 0x7fba2c0cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53530/0 (socket says 172.21.15.66:53530) 2024-03-30T00:50:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba29e00640 1 -- 172.21.15.66:0/4262472987 learned_addr learned my addr 172.21.15.66:0/4262472987 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba2a800640 1 --2- 172.21.15.66:0/4262472987 >> v2:172.21.15.156:3300/0 conn(0x7fba2c0cb130 0x7fba2c165050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba29400640 1 -- 172.21.15.66:0/4262472987 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947288723 0 0) 0x7fba2c0ccb50 con 0x7fba2c15a5d0 2024-03-30T00:50:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba29400640 1 -- 172.21.15.66:0/4262472987 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba10003530 con 0x7fba2c15a5d0 2024-03-30T00:50:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba2a800640 1 -- 172.21.15.66:0/4262472987 >> v1:172.21.15.156:6789/0 conn(0x7fba2c0d0d00 legacy=0x7fba2c0caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba2a800640 1 -- 172.21.15.66:0/4262472987 >> v1:172.21.15.66:6789/0 conn(0x7fba2c15a5d0 legacy=0x7fba2c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.659+0000 7fba2a800640 1 -- 172.21.15.66:0/4262472987 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c0cd110 con 0x7fba2c0cb130 2024-03-30T00:50:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2a800640 1 --2- 172.21.15.66:0/4262472987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba2c0cb130 0x7fba2c165050 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fba24009d90 tx=0x7fba2401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38b5030704e0327d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba29400640 1 -- 172.21.15.66:0/4262472987 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240a5920 con 0x7fba2c0cb130 2024-03-30T00:50:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba29400640 1 -- 172.21.15.66:0/4262472987 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba240a5a80 con 0x7fba2c0cb130 2024-03-30T00:50:48.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba29400640 1 -- 172.21.15.66:0/4262472987 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240a5e10 con 0x7fba2c0cb130 2024-03-30T00:50:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2b200640 1 -- 172.21.15.66:0/4262472987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba2c0cb130 msgr2=0x7fba2c165050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2b200640 1 --2- 172.21.15.66:0/4262472987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba2c0cb130 0x7fba2c165050 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fba24009d90 tx=0x7fba2401c970 comp rx=0 tx=0).stop 2024-03-30T00:50:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2b200640 1 -- 172.21.15.66:0/4262472987 shutdown_connections 2024-03-30T00:50:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2b200640 1 --2- 172.21.15.66:0/4262472987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba2c0cb130 0x7fba2c165050 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2b200640 1 -- 172.21.15.66:0/4262472987 >> 172.21.15.66:0/4262472987 conn(0x7fba2c0b1f00 msgr2=0x7fba2c0b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2b200640 1 -- 172.21.15.66:0/4262472987 shutdown_connections 2024-03-30T00:50:48.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2b200640 1 -- 172.21.15.66:0/4262472987 wait complete. 2024-03-30T00:50:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2b200640 1 Processor -- start 2024-03-30T00:50:48.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2b200640 1 -- start start 2024-03-30T00:50:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2b200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba2c0cb130 0x7fba2c164c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.663+0000 7fba2b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba2c15a5d0 0x7fba2c149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2c14a770 0x7fba2c0c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c06aea0 con 0x7fba2c14a770 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2b200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c1556a0 con 0x7fba2c0cb130 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2b200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c155980 con 0x7fba2c15a5d0 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2c14a770 0x7fba2c0c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2c14a770 0x7fba2c0c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59056/0 (socket says 172.21.15.66:59056) 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2a800640 1 -- 172.21.15.66:0/2959451803 learned_addr learned my addr 172.21.15.66:0/2959451803 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba29e00640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba2c0cb130 0x7fba2c164c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba23e00640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba2c15a5d0 0x7fba2c149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2a800640 1 -- 172.21.15.66:0/2959451803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba2c15a5d0 msgr2=0x7fba2c149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2a800640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba2c15a5d0 0x7fba2c149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2a800640 1 -- 172.21.15.66:0/2959451803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba2c0cb130 msgr2=0x7fba2c164c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2a800640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba2c0cb130 0x7fba2c164c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2a800640 1 -- 172.21.15.66:0/2959451803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24009a70 con 0x7fba2c14a770 2024-03-30T00:50:48.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2a800640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2c14a770 0x7fba2c0c5140 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fba240b0040 tx=0x7fba24019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba22000640 1 -- 172.21.15.66:0/2959451803 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240ad850 con 0x7fba2c14a770 2024-03-30T00:50:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba22000640 1 -- 172.21.15.66:0/2959451803 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba240ad9b0 con 0x7fba2c14a770 2024-03-30T00:50:48.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba22000640 1 -- 172.21.15.66:0/2959451803 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240add60 con 0x7fba2c14a770 2024-03-30T00:50:48.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2b200640 1 -- 172.21.15.66:0/2959451803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c155c60 con 0x7fba2c14a770 2024-03-30T00:50:48.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2b200640 1 -- 172.21.15.66:0/2959451803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c1552c0 con 0x7fba2c14a770 2024-03-30T00:50:48.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba2b200640 1 -- 172.21.15.66:0/2959451803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c073cf0 con 0x7fba2c14a770 2024-03-30T00:50:48.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.667+0000 7fba22000640 1 -- 172.21.15.66:0/2959451803 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fba240b4050 con 0x7fba2c14a770 2024-03-30T00:50:48.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.671+0000 7fba22000640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fba08066690 0x7fba08068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:48.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.671+0000 7fba22000640 1 -- 172.21.15.66:0/2959451803 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fba2411f160 con 0x7fba2c14a770 2024-03-30T00:50:48.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.671+0000 7fba29e00640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fba08066690 0x7fba08068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:48.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.671+0000 7fba29e00640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fba08066690 0x7fba08068b50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0011f0 tx=0x7fba1c007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:48.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.675+0000 7fba22000640 1 -- 172.21.15.66:0/2959451803 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fba240ec3c0 con 0x7fba2c14a770 2024-03-30T00:50:48.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.871+0000 7fba2b200640 1 -- 172.21.15.66:0/2959451803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fba2c0c32b0 con 0x7fba2c14a770 2024-03-30T00:50:48.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.871+0000 7fba22000640 1 -- 172.21.15.66:0/2959451803 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v95) v1 ==== 53+0+73 (secure 0 0 0) 0x7fba240a58e0 con 0x7fba2c14a770 2024-03-30T00:50:48.880 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:50:48.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 -- 172.21.15.66:0/2959451803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fba08066690 msgr2=0x7fba08068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fba08066690 0x7fba08068b50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0011f0 tx=0x7fba1c007d40 comp rx=0 tx=0).stop 2024-03-30T00:50:48.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 -- 172.21.15.66:0/2959451803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2c14a770 msgr2=0x7fba2c0c5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2c14a770 0x7fba2c0c5140 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fba240b0040 tx=0x7fba24019710 comp rx=0 tx=0).stop 2024-03-30T00:50:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 -- 172.21.15.66:0/2959451803 shutdown_connections 2024-03-30T00:50:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fba08066690 0x7fba08068b50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba2c14a770 0x7fba2c0c5140 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba2c15a5d0 0x7fba2c149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 --2- 172.21.15.66:0/2959451803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba2c0cb130 0x7fba2c164c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 -- 172.21.15.66:0/2959451803 >> 172.21.15.66:0/2959451803 conn(0x7fba2c0b1f00 msgr2=0x7fba2c0caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 -- 172.21.15.66:0/2959451803 shutdown_connections 2024-03-30T00:50:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:48.875+0000 7fba2b200640 1 -- 172.21.15.66:0/2959451803 wait complete. 2024-03-30T00:50:48.901 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:50:48.901 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:50:48.901 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:50:48.901 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:50:48.901 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:50:48.901 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:50:48.901 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:50:48.902 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:50:48.902 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:50:48.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:48.902 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:50:48.912 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41829sec preferred_lft 41829sec 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:50:48.913 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:50:48.914 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:50:48.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:48.915 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:50:48.971 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:50:48.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:48.973 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:50:49.029 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:50:49.030 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:50:49.030 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:50:49.030 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:50:49.030 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:50:49.030 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:50:49.030 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:50:49.030 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:50:49.030 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:50:49.030 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:50:49.030 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:50:49.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:49.031 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:50:49.031 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:50:49.031 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:50:49.031 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:50:49.031 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:50:49.031 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:50:49.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:49.123 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:50:49.132 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:50:49.132 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:50:49.132 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:50:49.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:49.132 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:50:49.192 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:50:49.193 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:50:49.193 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:50:49.245 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:50:49.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:50:49.246 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:50:49.297 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:50:49.358 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:50:49.358 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:50:49.419 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:50:49.420 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:50:49.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:49.421 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:50:49.473 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:50:49.474 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:50:49.520 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:50:49.541 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:50:49.542 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:50:49.603 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:50:49.605 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:50:49.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:49.606 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:50:49.616 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:50:49.607+0000 7f326544a580 -1 init, newargv = 0x55af3c85a140 newargc=15 2024-03-30T00:50:49.616 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[53839]: starting ceph client 2024-03-30T00:50:49.622 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[53839]: starting fuse 2024-03-30T00:50:49.624 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:50:49.625 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:50:49.625 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:50:49.626 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:50:49.746 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.53839.asok 2024-03-30T00:50:49.930 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53839.asok 2024-03-30T00:50:49.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:49.931 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53839.asok status 2024-03-30T00:50:50.106 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:50:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:50:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:50:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:50:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:50:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:50:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:50:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "53839", 2024-03-30T00:50:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 9552, 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 9552 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 4002022247 2024-03-30T00:50:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 4002022247 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.9552 192.168.0.1:0/4002022247", 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/4002022247", 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 95, 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 125, 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:50:50.108 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:50:50.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:50.121 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:50:50.121 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:50:50.130 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:50:50.130 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:50:50.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:50.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:50:50.193 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:50:50.193 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:50:50.193 DEBUG:teuthology.orchestra.run.smithi066:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:50:50.193 DEBUG:teuthology.orchestra.run.smithi066:> parent = os.path.join(mount_point, "parent") 2024-03-30T00:50:50.194 DEBUG:teuthology.orchestra.run.smithi066:> os.mkdir(parent) 2024-03-30T00:50:50.194 DEBUG:teuthology.orchestra.run.smithi066:> flushed = os.path.join(parent, "flushed") 2024-03-30T00:50:50.194 DEBUG:teuthology.orchestra.run.smithi066:> os.mkdir(flushed) 2024-03-30T00:50:50.194 DEBUG:teuthology.orchestra.run.smithi066:> for f in ["alpha", "bravo", "charlie"]: 2024-03-30T00:50:50.194 DEBUG:teuthology.orchestra.run.smithi066:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-03-30T00:50:50.194 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:50:50.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:50.491 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-30T00:50:50.559 INFO:teuthology.orchestra.run.smithi066.stdout:./ 2024-03-30T00:50:50.559 INFO:teuthology.orchestra.run.smithi066.stdout:./parent 2024-03-30T00:50:50.559 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/flushed 2024-03-30T00:50:50.559 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/flushed/charlie 2024-03-30T00:50:50.559 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/flushed/alpha 2024-03-30T00:50:50.559 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/flushed/bravo 2024-03-30T00:50:50.560 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:50:50.560 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:50:50.560 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:50:50.560 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:50:50.560 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-30T00:50:50.561 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:50:50.667 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:50:50.856 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:50:50.857 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:50:50.857 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:50:50.857 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:50:50.857 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-30T00:50:50.857 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:50:50.965 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627776 2024-03-30T00:50:51.155 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:50:51.155 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:50:51.155 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:50:51.155 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:50:51.155 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-30T00:50:51.155 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:50:51.262 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627777 2024-03-30T00:50:51.451 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:50:51.451 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:50:51.451 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:50:51.451 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:50:51.451 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-30T00:50:51.451 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:50:51.557 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628280 2024-03-30T00:50:51.747 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:50:51.747 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:50:51.747 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:50:51.747 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:50:51.747 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-30T00:50:51.747 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:50:51.854 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627778 2024-03-30T00:50:52.044 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:50:52.044 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:50:52.044 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:50:52.044 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:50:52.044 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-03-30T00:50:52.044 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:50:52.150 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628279 2024-03-30T00:50:52.339 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:50:52.360 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:50:52.389 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:50:52.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.567+0000 7fa799c00640 1 Processor -- start 2024-03-30T00:50:52.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.567+0000 7fa799c00640 1 -- start start 2024-03-30T00:50:52.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.567+0000 7fa799c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa7940cda90 0x7fa7940ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.567+0000 7fa799c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa79415a370 0x7fa7940cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.567+0000 7fa799c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79415a990 con 0x7fa7940cde70 2024-03-30T00:50:52.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.567+0000 7fa799c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7940b1b80 con 0x7fa7940cda90 2024-03-30T00:50:52.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.567+0000 7fa799c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7940cd3e0 con 0x7fa79415a370 2024-03-30T00:50:52.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.567+0000 7fa792a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa7940cda90 0x7fa7940ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.567+0000 7fa792a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa7940cda90 0x7fa7940ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48340/0 (socket says 172.21.15.66:48340) 2024-03-30T00:50:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.567+0000 7fa792a00640 1 -- 172.21.15.66:0/2794762424 learned_addr learned my addr 172.21.15.66:0/2794762424 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa793400640 1 --2- 172.21.15.66:0/2794762424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa79415a370 0x7fa7940cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa792a00640 1 -- 172.21.15.66:0/2794762424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa79415a370 msgr2=0x7fa7940cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:52.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa792a00640 1 --2- 172.21.15.66:0/2794762424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa79415a370 0x7fa7940cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa792a00640 1 -- 172.21.15.66:0/2794762424 >> v1:172.21.15.66:6789/0 conn(0x7fa7940cde70 legacy=0x7fa794159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:52.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa792a00640 1 -- 172.21.15.66:0/2794762424 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7940b11e0 con 0x7fa7940cda90 2024-03-30T00:50:52.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa793400640 1 --2- 172.21.15.66:0/2794762424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa79415a370 0x7fa7940cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:52.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa792a00640 1 --2- 172.21.15.66:0/2794762424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7940cda90 0x7fa7940ce520 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fa784009870 tx=0x7fa78401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c322750d09ded568 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:52.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa792000640 1 -- 172.21.15.66:0/2794762424 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7840ac640 con 0x7fa7940cda90 2024-03-30T00:50:52.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa792000640 1 -- 172.21.15.66:0/2794762424 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7840ac7a0 con 0x7fa7940cda90 2024-03-30T00:50:52.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa792000640 1 -- 172.21.15.66:0/2794762424 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7840acb30 con 0x7fa7940cda90 2024-03-30T00:50:52.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa799c00640 1 -- 172.21.15.66:0/2794762424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7940cda90 msgr2=0x7fa7940ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:52.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.571+0000 7fa799c00640 1 --2- 172.21.15.66:0/2794762424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7940cda90 0x7fa7940ce520 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fa784009870 tx=0x7fa78401c4d0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 -- 172.21.15.66:0/2794762424 shutdown_connections 2024-03-30T00:50:52.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 --2- 172.21.15.66:0/2794762424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa79415a370 0x7fa7940cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 --2- 172.21.15.66:0/2794762424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7940cda90 0x7fa7940ce520 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 -- 172.21.15.66:0/2794762424 >> 172.21.15.66:0/2794762424 conn(0x7fa794069f60 msgr2=0x7fa79406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:52.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 -- 172.21.15.66:0/2794762424 shutdown_connections 2024-03-30T00:50:52.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 -- 172.21.15.66:0/2794762424 wait complete. 2024-03-30T00:50:52.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 Processor -- start 2024-03-30T00:50:52.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 -- start start 2024-03-30T00:50:52.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7940cda90 0x7fa79406d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7940cde70 0x7fa79406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa79415a370 0x7fa7940aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7940acdc0 con 0x7fa7940cda90 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79414fb40 con 0x7fa79415a370 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7940ac800 con 0x7fa7940cde70 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa792a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7940cda90 0x7fa79406d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa792a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7940cda90 0x7fa79406d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48352/0 (socket says 172.21.15.66:48352) 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa792a00640 1 -- 172.21.15.66:0/1241838462 learned_addr learned my addr 172.21.15.66:0/1241838462 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa792a00640 1 -- 172.21.15.66:0/1241838462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7940cde70 msgr2=0x7fa79406d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa792a00640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7940cde70 0x7fa79406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa792a00640 1 -- 172.21.15.66:0/1241838462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa79415a370 msgr2=0x7fa7940aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa792a00640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa79415a370 0x7fa7940aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa792a00640 1 -- 172.21.15.66:0/1241838462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7840094d0 con 0x7fa7940cda90 2024-03-30T00:50:52.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa792a00640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7940cda90 0x7fa79406d080 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fa78401ab80 tx=0x7fa78401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:52.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa78b400640 1 -- 172.21.15.66:0/1241838462 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7840b7860 con 0x7fa7940cda90 2024-03-30T00:50:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa78b400640 1 -- 172.21.15.66:0/1241838462 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7840b79c0 con 0x7fa7940cda90 2024-03-30T00:50:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 -- 172.21.15.66:0/1241838462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7940acae0 con 0x7fa7940cda90 2024-03-30T00:50:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa78b400640 1 -- 172.21.15.66:0/1241838462 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7840b7d70 con 0x7fa7940cda90 2024-03-30T00:50:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.575+0000 7fa799c00640 1 -- 172.21.15.66:0/1241838462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa794097c80 con 0x7fa7940cda90 2024-03-30T00:50:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.579+0000 7fa799c00640 1 -- 172.21.15.66:0/1241838462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa794066c30 con 0x7fa7940cda90 2024-03-30T00:50:52.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.579+0000 7fa78b400640 1 -- 172.21.15.66:0/1241838462 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa7840b3070 con 0x7fa7940cda90 2024-03-30T00:50:52.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.579+0000 7fa78b400640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa770066690 0x7fa770068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.579+0000 7fa78b400640 1 -- 172.21.15.66:0/1241838462 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa78411ece0 con 0x7fa7940cda90 2024-03-30T00:50:52.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.579+0000 7fa78a000640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa770066690 0x7fa770068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:52.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.579+0000 7fa78a000640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa770066690 0x7fa770068b50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa794165220 tx=0x7fa77c007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:52.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.583+0000 7fa78b400640 1 -- 172.21.15.66:0/1241838462 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa7840ec020 con 0x7fa7940cda90 2024-03-30T00:50:52.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 -- 172.21.15.66:0/1241838462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa79414ffd0 con 0x7fa7940cda90 2024-03-30T00:50:52.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa78b400640 1 -- 172.21.15.66:0/1241838462 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16316 (secure 0 0 0) 0x7fa7840bb090 con 0x7fa7940cda90 2024-03-30T00:50:52.759 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:52.759 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":125,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:39.043056+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:50:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 -- 172.21.15.66:0/1241838462 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa770066690 msgr2=0x7fa770068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa770066690 0x7fa770068b50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa794165220 tx=0x7fa77c007960 comp rx=0 tx=0).stop 2024-03-30T00:50:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 -- 172.21.15.66:0/1241838462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7940cda90 msgr2=0x7fa79406d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7940cda90 0x7fa79406d080 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fa78401ab80 tx=0x7fa78401ad10 comp rx=0 tx=0).stop 2024-03-30T00:50:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 -- 172.21.15.66:0/1241838462 shutdown_connections 2024-03-30T00:50:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa79415a370 0x7fa7940aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7940cde70 0x7fa79406d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa770066690 0x7fa770068b50 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 --2- 172.21.15.66:0/1241838462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7940cda90 0x7fa79406d080 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 -- 172.21.15.66:0/1241838462 >> 172.21.15.66:0/1241838462 conn(0x7fa794069f60 msgr2=0x7fa79415aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:52.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 -- 172.21.15.66:0/1241838462 shutdown_connections 2024-03-30T00:50:52.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.751+0000 7fa799c00640 1 -- 172.21.15.66:0/1241838462 wait complete. 2024-03-30T00:50:52.781 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:50:35.696965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:50:37.037165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:50:52.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:50:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.915+0000 7f5975600640 1 Processor -- start 2024-03-30T00:50:52.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f5975600640 1 -- start start 2024-03-30T00:50:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f5975600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59700cec30 0x7f5970159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f5975600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f59700cde70 0x7f597015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f5975600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597015a6e0 con 0x7f59700cda90 2024-03-30T00:50:52.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f5975600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59700b1b80 con 0x7f59700cde70 2024-03-30T00:50:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f5975600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59700cd3e0 con 0x7f59700cec30 2024-03-30T00:50:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f596f400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f59700cde70 0x7f597015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f596e000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59700cec30 0x7f5970159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f596f400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f59700cde70 0x7f597015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:41072/0 (socket says 172.21.15.66:41072) 2024-03-30T00:50:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f596f400640 1 -- 172.21.15.66:0/3059244260 learned_addr learned my addr 172.21.15.66:0/3059244260 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f596ea00640 1 --1- 172.21.15.66:0/3059244260 >> v1:172.21.15.156:6789/0 conn(0x7f59700cda90 0x7f59700ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:52906/0 (socket says 172.21.15.66:52906) 2024-03-30T00:50:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f596f400640 1 -- 172.21.15.66:0/3059244260 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59700cec30 msgr2=0x7f5970159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f596f400640 1 --2- 172.21.15.66:0/3059244260 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59700cec30 0x7f5970159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f596f400640 1 -- 172.21.15.66:0/3059244260 >> v1:172.21.15.156:6789/0 conn(0x7f59700cda90 legacy=0x7f59700ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:52.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.919+0000 7f596f400640 1 -- 172.21.15.66:0/3059244260 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597015a990 con 0x7f59700cde70 2024-03-30T00:50:52.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f596f400640 1 --2- 172.21.15.66:0/3059244260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59700cde70 0x7f597015a1a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5968009e50 tx=0x7f596801c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5517d8ef4ce8a19d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f596d600640 1 -- 172.21.15.66:0/3059244260 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59680a5860 con 0x7f59700cde70 2024-03-30T00:50:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f596d600640 1 -- 172.21.15.66:0/3059244260 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59680a59c0 con 0x7f59700cde70 2024-03-30T00:50:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f596d600640 1 -- 172.21.15.66:0/3059244260 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59680a5d50 con 0x7f59700cde70 2024-03-30T00:50:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f5975600640 1 -- 172.21.15.66:0/3059244260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59700cde70 msgr2=0x7f597015a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f5975600640 1 --2- 172.21.15.66:0/3059244260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59700cde70 0x7f597015a1a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5968009e50 tx=0x7f596801c9a0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f5975600640 1 -- 172.21.15.66:0/3059244260 shutdown_connections 2024-03-30T00:50:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f5975600640 1 --2- 172.21.15.66:0/3059244260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59700cde70 0x7f597015a1a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f5975600640 1 --2- 172.21.15.66:0/3059244260 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59700cec30 0x7f5970159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f5975600640 1 -- 172.21.15.66:0/3059244260 >> 172.21.15.66:0/3059244260 conn(0x7f5970069f60 msgr2=0x7f597006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f5975600640 1 -- 172.21.15.66:0/3059244260 shutdown_connections 2024-03-30T00:50:52.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f5975600640 1 -- 172.21.15.66:0/3059244260 wait complete. 2024-03-30T00:50:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f5975600640 1 Processor -- start 2024-03-30T00:50:52.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.923+0000 7f5975600640 1 -- start start 2024-03-30T00:50:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5975600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59700cda90 0x7f5970163100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5975600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59700cde70 0x7f597006fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5975600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59700cec30 0x7f59701f8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5975600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700719c0 con 0x7f59700cda90 2024-03-30T00:50:52.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5975600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5970070920 con 0x7f59700cec30 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5975600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5970070c00 con 0x7f59700cde70 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59700cda90 0x7f5970163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59700cda90 0x7f5970163100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48368/0 (socket says 172.21.15.66:48368) 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596ea00640 1 -- 172.21.15.66:0/1515615415 learned_addr learned my addr 172.21.15.66:0/1515615415 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596e000640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59700cde70 0x7f597006fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596f400640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59700cec30 0x7f59701f8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596ea00640 1 -- 172.21.15.66:0/1515615415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59700cde70 msgr2=0x7f597006fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596ea00640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59700cde70 0x7f597006fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596ea00640 1 -- 172.21.15.66:0/1515615415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59700cec30 msgr2=0x7f59701f8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596ea00640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59700cec30 0x7f59701f8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596ea00640 1 -- 172.21.15.66:0/1515615415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5968009b30 con 0x7f59700cda90 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596f400640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59700cec30 0x7f59701f8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f596ea00640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59700cda90 0x7f5970163100 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f596000c490 tx=0x7f596000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:52.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5966a00640 1 -- 172.21.15.66:0/1515615415 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960007480 con 0x7f59700cda90 2024-03-30T00:50:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5966a00640 1 -- 172.21.15.66:0/1515615415 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59600075e0 con 0x7f59700cda90 2024-03-30T00:50:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5975600640 1 -- 172.21.15.66:0/1515615415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5970070ee0 con 0x7f59700cda90 2024-03-30T00:50:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5966a00640 1 -- 172.21.15.66:0/1515615415 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960004a10 con 0x7f59700cda90 2024-03-30T00:50:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5975600640 1 -- 172.21.15.66:0/1515615415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59700714a0 con 0x7f59700cda90 2024-03-30T00:50:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5975600640 1 -- 172.21.15.66:0/1515615415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5970066c30 con 0x7f59700cda90 2024-03-30T00:50:52.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.927+0000 7f5966a00640 1 -- 172.21.15.66:0/1515615415 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5960004b70 con 0x7f59700cda90 2024-03-30T00:50:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.931+0000 7f5966a00640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5950066690 0x7f5950068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.931+0000 7f5966a00640 1 -- 172.21.15.66:0/1515615415 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5960083720 con 0x7f59700cda90 2024-03-30T00:50:52.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.931+0000 7f596e000640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5950066690 0x7f5950068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:52.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.931+0000 7f596e000640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5950066690 0x7f5950068b50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f59700b26b0 tx=0x7f595c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:52.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:52.935+0000 7f5966a00640 1 -- 172.21.15.66:0/1515615415 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5960050940 con 0x7f59700cda90 2024-03-30T00:50:53.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.099+0000 7f5975600640 1 -- 172.21.15.66:0/1515615415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59700cea10 con 0x7f59700cda90 2024-03-30T00:50:53.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.099+0000 7f5966a00640 1 -- 172.21.15.66:0/1515615415 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5960053f60 con 0x7f59700cda90 2024-03-30T00:50:53.109 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:50:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 -- 172.21.15.66:0/1515615415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5950066690 msgr2=0x7f5950068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5950066690 0x7f5950068b50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f59700b26b0 tx=0x7f595c008040 comp rx=0 tx=0).stop 2024-03-30T00:50:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 -- 172.21.15.66:0/1515615415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59700cda90 msgr2=0x7f5970163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59700cda90 0x7f5970163100 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f596000c490 tx=0x7f596000c950 comp rx=0 tx=0).stop 2024-03-30T00:50:53.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 -- 172.21.15.66:0/1515615415 shutdown_connections 2024-03-30T00:50:53.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59700cec30 0x7f59701f8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59700cde70 0x7f597006fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5950066690 0x7f5950068b50 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 --2- 172.21.15.66:0/1515615415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59700cda90 0x7f5970163100 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 -- 172.21.15.66:0/1515615415 >> 172.21.15.66:0/1515615415 conn(0x7f5970069f60 msgr2=0x7f59700cb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:53.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 -- 172.21.15.66:0/1515615415 shutdown_connections 2024-03-30T00:50:53.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.103+0000 7f5975600640 1 -- 172.21.15.66:0/1515615415 wait complete. 2024-03-30T00:50:53.130 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:50:53.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c7a00640 1 Processor -- start 2024-03-30T00:50:53.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c7a00640 1 -- start start 2024-03-30T00:50:53.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c7a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f11c00ce520 0x7f11c00ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c7a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c006d070 con 0x7f11c00cabb0 2024-03-30T00:50:53.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c7a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c00cbba0 con 0x7f11c015a640 2024-03-30T00:50:53.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c7a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c00cee70 con 0x7f11c00ce520 2024-03-30T00:50:53.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c5a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f11c00cabb0 0x7f11c00cafe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60236/0 (socket says 172.21.15.66:60236) 2024-03-30T00:50:53.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c5a00640 1 -- 172.21.15.66:0/45558122 learned_addr learned my addr 172.21.15.66:0/45558122 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bfe00640 1 --2- 172.21.15.66:0/45558122 >> v2:172.21.15.156:3300/0 conn(0x7f11c00ce520 0x7f11c00ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994369711 0 0) 0x7f11c006d070 con 0x7f11c00cabb0 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11a8003530 con 0x7f11c00cabb0 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1568789509 0 0) 0x7f11a8003530 con 0x7f11c00cabb0 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 >> v2:172.21.15.156:3300/0 conn(0x7f11c00ce520 msgr2=0x7f11c00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 --2- 172.21.15.66:0/45558122 >> v2:172.21.15.156:3300/0 conn(0x7f11c00ce520 0x7f11c00ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 >> v1:172.21.15.156:6789/0 conn(0x7f11c015a640 legacy=0x7f11c006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c00cb8c0 con 0x7f11c00cabb0 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f11b8001a00 con 0x7f11c00cabb0 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 >> v1:172.21.15.66:6789/0 conn(0x7f11c00cabb0 legacy=0x7f11c00cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 --2- 172.21.15.66:0/45558122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a80039d0 0x7f11a8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 --2- 172.21.15.66:0/45558122 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11a80047d0 0x7f11a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 --2- 172.21.15.66:0/45558122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 0x7f11a800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c00cee70 con 0x7f11a8005190 2024-03-30T00:50:53.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a800c090 con 0x7f11a80039d0 2024-03-30T00:50:53.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a800c250 con 0x7f11a80047d0 2024-03-30T00:50:53.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c5a00640 1 --2- 172.21.15.66:0/45558122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 0x7f11a800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11bfe00640 1 --2- 172.21.15.66:0/45558122 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11a80047d0 0x7f11a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c5000640 1 --2- 172.21.15.66:0/45558122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a80039d0 0x7f11a8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c5a00640 1 -- 172.21.15.66:0/45558122 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11a80047d0 msgr2=0x7f11a8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:53.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c5a00640 1 --2- 172.21.15.66:0/45558122 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11a80047d0 0x7f11a8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c5a00640 1 -- 172.21.15.66:0/45558122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a80039d0 msgr2=0x7f11a8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:53.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c5a00640 1 --2- 172.21.15.66:0/45558122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a80039d0 0x7f11a8003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.267+0000 7f11c5a00640 1 -- 172.21.15.66:0/45558122 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a8003530 con 0x7f11a8005190 2024-03-30T00:50:53.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11c5a00640 1 -- 172.21.15.66:0/45558122 reap_dead start 2024-03-30T00:50:53.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11c5a00640 1 --2- 172.21.15.66:0/45558122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 0x7f11a800b990 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f11b800a760 tx=0x7f11b801d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a15051f6be89f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:53.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80ad630 con 0x7f11a8005190 2024-03-30T00:50:53.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b80a8070 con 0x7f11a8005190 2024-03-30T00:50:53.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11bf400640 1 -- 172.21.15.66:0/45558122 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80ada10 con 0x7f11a8005190 2024-03-30T00:50:53.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11c7a00640 1 -- 172.21.15.66:0/45558122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 msgr2=0x7f11a800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11c7a00640 1 --2- 172.21.15.66:0/45558122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 0x7f11a800b990 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f11b800a760 tx=0x7f11b801d280 comp rx=0 tx=0).stop 2024-03-30T00:50:53.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11c7a00640 1 -- 172.21.15.66:0/45558122 shutdown_connections 2024-03-30T00:50:53.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11c7a00640 1 --2- 172.21.15.66:0/45558122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 0x7f11a800b990 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11c7a00640 1 -- 172.21.15.66:0/45558122 >> 172.21.15.66:0/45558122 conn(0x7f11c0069f60 msgr2=0x7f11c006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:53.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11c7a00640 1 -- 172.21.15.66:0/45558122 shutdown_connections 2024-03-30T00:50:53.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.271+0000 7f11c7a00640 1 -- 172.21.15.66:0/45558122 wait complete. 2024-03-30T00:50:53.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c7a00640 1 Processor -- start 2024-03-30T00:50:53.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c7a00640 1 -- start start 2024-03-30T00:50:53.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c7a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 0x7f11c01f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c7a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11c01572c0 0x7f11c00ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c7a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11c00ae9c0 0x7f11c015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c7a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c00aee50 con 0x7f11a8005190 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c7a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c015c020 con 0x7f11c01572c0 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c7a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c015c300 con 0x7f11c00ae9c0 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 0x7f11c01f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 0x7f11c01f2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48378/0 (socket says 172.21.15.66:48378) 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c5000640 1 -- 172.21.15.66:0/1973062236 learned_addr learned my addr 172.21.15.66:0/1973062236 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11bfe00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11c01572c0 0x7f11c00ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c5a00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11c00ae9c0 0x7f11c015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c5000640 1 -- 172.21.15.66:0/1973062236 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11c00ae9c0 msgr2=0x7f11c015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c5000640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11c00ae9c0 0x7f11c015b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c5000640 1 -- 172.21.15.66:0/1973062236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11c01572c0 msgr2=0x7f11c00ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c5000640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11c01572c0 0x7f11c00ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c5000640 1 -- 172.21.15.66:0/1973062236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c00c9140 con 0x7f11a8005190 2024-03-30T00:50:53.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11bfe00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11c01572c0 0x7f11c00ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:53.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c5000640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 0x7f11c01f2c00 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f11b400f7f0 tx=0x7f11b400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:53.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11bcc00640 1 -- 172.21.15.66:0/1973062236 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b40050d0 con 0x7f11a8005190 2024-03-30T00:50:53.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11bcc00640 1 -- 172.21.15.66:0/1973062236 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b4005230 con 0x7f11a8005190 2024-03-30T00:50:53.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11bcc00640 1 -- 172.21.15.66:0/1973062236 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b400d730 con 0x7f11a8005190 2024-03-30T00:50:53.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c7a00640 1 -- 172.21.15.66:0/1973062236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c00c9420 con 0x7f11a8005190 2024-03-30T00:50:53.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.275+0000 7f11c7a00640 1 -- 172.21.15.66:0/1973062236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c015b7f0 con 0x7f11a8005190 2024-03-30T00:50:53.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.279+0000 7f11c7a00640 1 -- 172.21.15.66:0/1973062236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c0066c30 con 0x7f11a8005190 2024-03-30T00:50:53.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.279+0000 7f11bcc00640 1 -- 172.21.15.66:0/1973062236 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f11b400d890 con 0x7f11a8005190 2024-03-30T00:50:53.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.279+0000 7f11bcc00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f11a00666c0 0x7f11a0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.279+0000 7f11bcc00640 1 -- 172.21.15.66:0/1973062236 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f11b40842e0 con 0x7f11a8005190 2024-03-30T00:50:53.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.279+0000 7f11bfe00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f11a00666c0 0x7f11a0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.279+0000 7f11bfe00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f11a00666c0 0x7f11a0068b80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009b60 tx=0x7f11ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:53.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.283+0000 7f11bcc00640 1 -- 172.21.15.66:0/1973062236 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f11b4051470 con 0x7f11a8005190 2024-03-30T00:50:53.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 -- 172.21.15.66:0/1973062236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11c00ad6a0 con 0x7f11a8005190 2024-03-30T00:50:53.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11bcc00640 1 -- 172.21.15.66:0/1973062236 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11b4054a90 con 0x7f11a8005190 2024-03-30T00:50:53.451 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:50:53.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 -- 172.21.15.66:0/1973062236 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f11a00666c0 msgr2=0x7f11a0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f11a00666c0 0x7f11a0068b80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009b60 tx=0x7f11ac008040 comp rx=0 tx=0).stop 2024-03-30T00:50:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 -- 172.21.15.66:0/1973062236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 msgr2=0x7f11c01f2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 0x7f11c01f2c00 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f11b400f7f0 tx=0x7f11b400fcb0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 -- 172.21.15.66:0/1973062236 shutdown_connections 2024-03-30T00:50:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11c00ae9c0 0x7f11c015b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11c01572c0 0x7f11c00ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f11a00666c0 0x7f11a0068b80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 --2- 172.21.15.66:0/1973062236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a8005190 0x7f11c01f2c00 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 -- 172.21.15.66:0/1973062236 >> 172.21.15.66:0/1973062236 conn(0x7f11c0069f60 msgr2=0x7f11c00af160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 -- 172.21.15.66:0/1973062236 shutdown_connections 2024-03-30T00:50:53.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.443+0000 7f11c7a00640 1 -- 172.21.15.66:0/1973062236 wait complete. 2024-03-30T00:50:53.472 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:50:53.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ecee00640 1 Processor -- start 2024-03-30T00:50:53.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ecee00640 1 -- start start 2024-03-30T00:50:53.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ecee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ec80cda90 0x7f9ec80ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ecee00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9ec80cde70 0x7f9ec80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ecee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec815a990 con 0x7f9ec8159c60 2024-03-30T00:50:53.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ecee00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec80b1b80 con 0x7f9ec80cde70 2024-03-30T00:50:53.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ecee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80cd3e0 con 0x7f9ec80cda90 2024-03-30T00:50:53.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ecce00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9ec8159c60 0x7f9ec815a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60248/0 (socket says 172.21.15.66:60248) 2024-03-30T00:50:53.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ecce00640 1 -- 172.21.15.66:0/267015980 learned_addr learned my addr 172.21.15.66:0/267015980 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:53.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ebf400640 1 --2- 172.21.15.66:0/267015980 >> v2:172.21.15.156:3300/0 conn(0x7f9ec80cde70 0x7f9ec80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7e00640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ec80cda90 0x7f9ec80ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247498947 0 0) 0x7f9ec815a990 con 0x7f9ec8159c60 2024-03-30T00:50:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eac005740 con 0x7f9ec8159c60 2024-03-30T00:50:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3102524539 0 0) 0x7f9eac005740 con 0x7f9ec8159c60 2024-03-30T00:50:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ec80cda90 msgr2=0x7f9ec80ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ec80cda90 0x7f9ec80ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 >> v2:172.21.15.156:3300/0 conn(0x7f9ec80cde70 msgr2=0x7f9ec80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 --2- 172.21.15.66:0/267015980 >> v2:172.21.15.156:3300/0 conn(0x7f9ec80cde70 0x7f9ec80cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80b11e0 con 0x7f9ec8159c60 2024-03-30T00:50:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f9ec0001680 con 0x7f9ec8159c60 2024-03-30T00:50:53.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 >> v1:172.21.15.66:6789/0 conn(0x7f9ec8159c60 legacy=0x7f9ec815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eac005b90 0x7f9eac005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac0069e0 0x7f9eac006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9eac0073a0 0x7f9eac00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80b1b80 con 0x7f9eac005b90 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80cd3e0 con 0x7f9eac0069e0 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac00e110 con 0x7f9eac0073a0 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7e00640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ec80cda90 0x7f9ec80ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ebf400640 1 --2- 172.21.15.66:0/267015980 >> v2:172.21.15.156:3300/0 conn(0x7f9ec80cde70 0x7f9ec80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ec7e00640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eac005b90 0x7f9eac005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ecce00640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9eac0073a0 0x7f9eac00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.607+0000 7f9ebf400640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac0069e0 0x7f9eac006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ec7e00640 1 -- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9eac0073a0 msgr2=0x7f9eac00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ec7e00640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9eac0073a0 0x7f9eac00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ec7e00640 1 -- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac0069e0 msgr2=0x7f9eac006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:53.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ec7e00640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac0069e0 0x7f9eac006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ec7e00640 1 -- 172.21.15.66:0/267015980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac005740 con 0x7f9eac005b90 2024-03-30T00:50:53.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ec7e00640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eac005b90 0x7f9eac005fb0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80cf050 tx=0x7f9eb801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5566c30e45bc47c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:53.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8004030 con 0x7f9eac005b90 2024-03-30T00:50:53.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb8004190 con 0x7f9eac005b90 2024-03-30T00:50:53.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ec7400640 1 -- 172.21.15.66:0/267015980 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8004520 con 0x7f9eac005b90 2024-03-30T00:50:53.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ecee00640 1 -- 172.21.15.66:0/267015980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eac005b90 msgr2=0x7f9eac005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ecee00640 1 --2- 172.21.15.66:0/267015980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eac005b90 0x7f9eac005fb0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80cf050 tx=0x7f9eb801ca50 comp rx=0 tx=0).stop 2024-03-30T00:50:53.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ecce00640 1 -- 172.21.15.66:0/267015980 reap_dead start 2024-03-30T00:50:53.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ecee00640 1 -- 172.21.15.66:0/267015980 shutdown_connections 2024-03-30T00:50:53.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ecee00640 1 -- 172.21.15.66:0/267015980 >> 172.21.15.66:0/267015980 conn(0x7f9ec8069f60 msgr2=0x7f9ec806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:53.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.611+0000 7f9ecee00640 1 -- 172.21.15.66:0/267015980 shutdown_connections 2024-03-30T00:50:53.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecee00640 1 -- 172.21.15.66:0/267015980 wait complete. 2024-03-30T00:50:53.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecee00640 1 Processor -- start 2024-03-30T00:50:53.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecee00640 1 -- start start 2024-03-30T00:50:53.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecee00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac005b90 0x7f9ec8156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecee00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ec8071a50 0x7f9ec815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ec815c140 0x7f9ec806fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec815c5c0 con 0x7f9ec815c140 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecee00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec8156790 con 0x7f9eac005b90 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecee00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec8071180 con 0x7f9ec8071a50 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ec815c140 0x7f9ec806fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ec815c140 0x7f9ec806fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48386/0 (socket says 172.21.15.66:48386) 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecce00640 1 -- 172.21.15.66:0/1763290104 learned_addr learned my addr 172.21.15.66:0/1763290104 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ebf400640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ec8071a50 0x7f9ec815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ec7e00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac005b90 0x7f9ec8156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecce00640 1 -- 172.21.15.66:0/1763290104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ec8071a50 msgr2=0x7f9ec815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecce00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ec8071a50 0x7f9ec815bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecce00640 1 -- 172.21.15.66:0/1763290104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac005b90 msgr2=0x7f9ec8156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecce00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac005b90 0x7f9ec8156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecce00640 1 -- 172.21.15.66:0/1763290104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80c3b10 con 0x7f9ec815c140 2024-03-30T00:50:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ec7e00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac005b90 0x7f9ec8156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecce00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ec815c140 0x7f9ec806fed0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80b3140 tx=0x7f9ec0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ec4c00640 1 -- 172.21.15.66:0/1763290104 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000f690 con 0x7f9ec815c140 2024-03-30T00:50:53.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ec4c00640 1 -- 172.21.15.66:0/1763290104 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec000f7f0 con 0x7f9ec815c140 2024-03-30T00:50:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ec4c00640 1 -- 172.21.15.66:0/1763290104 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000fba0 con 0x7f9ec815c140 2024-03-30T00:50:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecee00640 1 -- 172.21.15.66:0/1763290104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec80c3df0 con 0x7f9ec815c140 2024-03-30T00:50:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.615+0000 7f9ecee00640 1 -- 172.21.15.66:0/1763290104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec80c43b0 con 0x7f9ec815c140 2024-03-30T00:50:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.619+0000 7f9ecee00640 1 -- 172.21.15.66:0/1763290104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec8066c30 con 0x7f9ec815c140 2024-03-30T00:50:53.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.619+0000 7f9ec4c00640 1 -- 172.21.15.66:0/1763290104 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9ec000fd00 con 0x7f9ec815c140 2024-03-30T00:50:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.619+0000 7f9ec4c00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ea40666c0 0x7f9ea4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.619+0000 7f9ec4c00640 1 -- 172.21.15.66:0/1763290104 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9ec00836c0 con 0x7f9ec815c140 2024-03-30T00:50:53.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.619+0000 7f9ec7e00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ea40666c0 0x7f9ea4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.619+0000 7f9ec7e00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ea40666c0 0x7f9ea4068b80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80ad000 tx=0x7f9eb80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:53.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.623+0000 7f9ec4c00640 1 -- 172.21.15.66:0/1763290104 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9ec0050850 con 0x7f9ec815c140 2024-03-30T00:50:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 -- 172.21.15.66:0/1763290104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ec80adad0 con 0x7f9ec815c140 2024-03-30T00:50:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ec4c00640 1 -- 172.21.15.66:0/1763290104 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ec0053e70 con 0x7f9ec815c140 2024-03-30T00:50:53.799 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:50:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 -- 172.21.15.66:0/1763290104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ea40666c0 msgr2=0x7f9ea4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ea40666c0 0x7f9ea4068b80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80ad000 tx=0x7f9eb80046d0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 -- 172.21.15.66:0/1763290104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ec815c140 msgr2=0x7f9ec806fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ec815c140 0x7f9ec806fed0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80b3140 tx=0x7f9ec0004f90 comp rx=0 tx=0).stop 2024-03-30T00:50:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 -- 172.21.15.66:0/1763290104 shutdown_connections 2024-03-30T00:50:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9ea40666c0 0x7f9ea4068b80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ec815c140 0x7f9ec806fed0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ec8071a50 0x7f9ec815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 --2- 172.21.15.66:0/1763290104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac005b90 0x7f9ec8156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 -- 172.21.15.66:0/1763290104 >> 172.21.15.66:0/1763290104 conn(0x7f9ec8069f60 msgr2=0x7f9ec80afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 -- 172.21.15.66:0/1763290104 shutdown_connections 2024-03-30T00:50:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.791+0000 7f9ecee00640 1 -- 172.21.15.66:0/1763290104 wait complete. 2024-03-30T00:50:53.821 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:50:53.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb36400640 1 Processor -- start 2024-03-30T00:50:53.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb36400640 1 -- start start 2024-03-30T00:50:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb36400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ffb301535a0 0x7ffb3006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb36400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3015e520 con 0x7ffb30153980 2024-03-30T00:50:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb36400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3015d2a0 con 0x7ffb3015cd00 2024-03-30T00:50:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb36400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb30070900 con 0x7ffb301535a0 2024-03-30T00:50:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb35000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ffb301535a0 0x7ffb3006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb27e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ffb30153980 0x7ffb3015de10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60250/0 (socket says 172.21.15.66:60250) 2024-03-30T00:50:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb35000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ffb301535a0 0x7ffb3006c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48396/0 (socket says 172.21.15.66:48396) 2024-03-30T00:50:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb27e00640 1 -- 172.21.15.66:0/636503275 learned_addr learned my addr 172.21.15.66:0/636503275 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb35000640 1 -- 172.21.15.66:0/636503275 >> v1:172.21.15.156:6789/0 conn(0x7ffb3015cd00 legacy=0x7ffb300cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb35000640 1 -- 172.21.15.66:0/636503275 >> v1:172.21.15.66:6789/0 conn(0x7ffb30153980 legacy=0x7ffb3015de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb2fe00640 1 -- 172.21.15.66:0/636503275 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241062681 0 0) 0x7ffb18001120 con 0x7ffb30153980 2024-03-30T00:50:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.955+0000 7ffb35000640 1 -- 172.21.15.66:0/636503275 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3015d580 con 0x7ffb301535a0 2024-03-30T00:50:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.959+0000 7ffb35000640 1 --2- 172.21.15.66:0/636503275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb301535a0 0x7ffb3006c980 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7ffb200097f0 tx=0x7ffb2001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960b8ac0c759443e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.959+0000 7ffb2fe00640 1 -- 172.21.15.66:0/636503275 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200a8050 con 0x7ffb301535a0 2024-03-30T00:50:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.959+0000 7ffb2fe00640 1 -- 172.21.15.66:0/636503275 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb200ac5f0 con 0x7ffb301535a0 2024-03-30T00:50:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.959+0000 7ffb2fe00640 1 -- 172.21.15.66:0/636503275 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200ac980 con 0x7ffb301535a0 2024-03-30T00:50:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.959+0000 7ffb36400640 1 -- 172.21.15.66:0/636503275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb301535a0 msgr2=0x7ffb3006c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.959+0000 7ffb36400640 1 --2- 172.21.15.66:0/636503275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb301535a0 0x7ffb3006c980 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7ffb200097f0 tx=0x7ffb2001c4a0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.959+0000 7ffb36400640 1 -- 172.21.15.66:0/636503275 shutdown_connections 2024-03-30T00:50:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.959+0000 7ffb36400640 1 --2- 172.21.15.66:0/636503275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb301535a0 0x7ffb3006c980 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.959+0000 7ffb36400640 1 -- 172.21.15.66:0/636503275 >> 172.21.15.66:0/636503275 conn(0x7ffb3006fab0 msgr2=0x7ffb3006fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.959+0000 7ffb36400640 1 -- 172.21.15.66:0/636503275 shutdown_connections 2024-03-30T00:50:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.959+0000 7ffb36400640 1 -- 172.21.15.66:0/636503275 wait complete. 2024-03-30T00:50:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb36400640 1 Processor -- start 2024-03-30T00:50:53.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb36400640 1 -- start start 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb36400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb301535a0 0x7ffb300b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb36400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffb30153980 0x7ffb30149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb36400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb3015cd00 0x7ffb300b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb36400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300b57b0 con 0x7ffb3015cd00 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb36400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300b41d0 con 0x7ffb301535a0 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb36400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300b44b0 con 0x7ffb30153980 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb35a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb3015cd00 0x7ffb300b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb35000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb301535a0 0x7ffb300b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb27e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffb30153980 0x7ffb30149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb35a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb3015cd00 0x7ffb300b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48412/0 (socket says 172.21.15.66:48412) 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb35a00640 1 -- 172.21.15.66:0/1826411231 learned_addr learned my addr 172.21.15.66:0/1826411231 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb35a00640 1 -- 172.21.15.66:0/1826411231 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffb30153980 msgr2=0x7ffb30149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb35a00640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffb30153980 0x7ffb30149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb35a00640 1 -- 172.21.15.66:0/1826411231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb301535a0 msgr2=0x7ffb300b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:53.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb35a00640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb301535a0 0x7ffb300b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:53.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb35a00640 1 -- 172.21.15.66:0/1826411231 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb20009420 con 0x7ffb3015cd00 2024-03-30T00:50:53.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb35a00640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb3015cd00 0x7ffb300b38d0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800c730 tx=0x7ffb2800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:53.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb2d600640 1 -- 172.21.15.66:0/1826411231 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28003b80 con 0x7ffb3015cd00 2024-03-30T00:50:53.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb2d600640 1 -- 172.21.15.66:0/1826411231 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb28003ce0 con 0x7ffb3015cd00 2024-03-30T00:50:53.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb2d600640 1 -- 172.21.15.66:0/1826411231 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28005310 con 0x7ffb3015cd00 2024-03-30T00:50:53.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb36400640 1 -- 172.21.15.66:0/1826411231 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb300b4790 con 0x7ffb3015cd00 2024-03-30T00:50:53.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.963+0000 7ffb36400640 1 -- 172.21.15.66:0/1826411231 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb300b4d50 con 0x7ffb3015cd00 2024-03-30T00:50:53.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.967+0000 7ffb36400640 1 -- 172.21.15.66:0/1826411231 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb30073e00 con 0x7ffb3015cd00 2024-03-30T00:50:53.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.967+0000 7ffb2d600640 1 -- 172.21.15.66:0/1826411231 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ffb28005680 con 0x7ffb3015cd00 2024-03-30T00:50:53.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.967+0000 7ffb2d600640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffb0c066690 0x7ffb0c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:53.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.967+0000 7ffb2d600640 1 -- 172.21.15.66:0/1826411231 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffb28084130 con 0x7ffb3015cd00 2024-03-30T00:50:53.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.967+0000 7ffb35000640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffb0c066690 0x7ffb0c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:53.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.967+0000 7ffb35000640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffb0c066690 0x7ffb0c068b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ffb200ae000 tx=0x7ffb200a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:53.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:53.971+0000 7ffb2d600640 1 -- 172.21.15.66:0/1826411231 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ffb28051370 con 0x7ffb3015cd00 2024-03-30T00:50:54.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.147+0000 7ffb36400640 1 -- 172.21.15.66:0/1826411231 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffb300b6300 con 0x7ffb3015cd00 2024-03-30T00:50:54.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb2d600640 1 -- 172.21.15.66:0/1826411231 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffb28054990 con 0x7ffb3015cd00 2024-03-30T00:50:54.158 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:50:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 -- 172.21.15.66:0/1826411231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffb0c066690 msgr2=0x7ffb0c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffb0c066690 0x7ffb0c068b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ffb200ae000 tx=0x7ffb200a7000 comp rx=0 tx=0).stop 2024-03-30T00:50:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 -- 172.21.15.66:0/1826411231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb3015cd00 msgr2=0x7ffb300b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb3015cd00 0x7ffb300b38d0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800c730 tx=0x7ffb2800cbf0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 -- 172.21.15.66:0/1826411231 shutdown_connections 2024-03-30T00:50:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ffb0c066690 0x7ffb0c068b50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb3015cd00 0x7ffb300b38d0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffb30153980 0x7ffb30149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 --2- 172.21.15.66:0/1826411231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb301535a0 0x7ffb300b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 -- 172.21.15.66:0/1826411231 >> 172.21.15.66:0/1826411231 conn(0x7ffb3006fab0 msgr2=0x7ffb300b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 -- 172.21.15.66:0/1826411231 shutdown_connections 2024-03-30T00:50:54.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.151+0000 7ffb36400640 1 -- 172.21.15.66:0/1826411231 wait complete. 2024-03-30T00:50:54.178 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:50:54.178 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:50:54.188 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:50:54.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.359+0000 7fb6dd000640 1 Processor -- start 2024-03-30T00:50:54.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.359+0000 7fb6dd000640 1 -- start start 2024-03-30T00:50:54.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.359+0000 7fb6dd000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb6d80cabb0 0x7fb6d80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:54.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.359+0000 7fb6dd000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d806d070 con 0x7fb6d815a640 2024-03-30T00:50:54.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6dd000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d80cbba0 con 0x7fb6d80ce520 2024-03-30T00:50:54.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6dd000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d80cb8c0 con 0x7fb6d80cabb0 2024-03-30T00:50:54.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6d6800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb6d80cabb0 0x7fb6d80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:54.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6d6800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb6d80cabb0 0x7fb6d80cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48416/0 (socket says 172.21.15.66:48416) 2024-03-30T00:50:54.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6d6800640 1 -- 172.21.15.66:0/1634387701 learned_addr learned my addr 172.21.15.66:0/1634387701 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:54.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6d5e00640 1 --1- 172.21.15.66:0/1634387701 >> v1:172.21.15.66:6789/0 conn(0x7fb6d815a640 0x7fb6d806dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60262/0 (socket says 172.21.15.66:60262) 2024-03-30T00:50:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6d6800640 1 -- 172.21.15.66:0/1634387701 >> v1:172.21.15.156:6789/0 conn(0x7fb6d80ce520 legacy=0x7fb6d80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6d6800640 1 -- 172.21.15.66:0/1634387701 >> v1:172.21.15.66:6789/0 conn(0x7fb6d815a640 legacy=0x7fb6d806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6d6800640 1 -- 172.21.15.66:0/1634387701 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d806d2a0 con 0x7fb6d80cabb0 2024-03-30T00:50:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6d6800640 1 --2- 172.21.15.66:0/1634387701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d80cabb0 0x7fb6d80cafe0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009e50 tx=0x7fb6d001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfbf90ddb49f185 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6d5400640 1 -- 172.21.15.66:0/1634387701 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d00a58d0 con 0x7fb6d80cabb0 2024-03-30T00:50:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6d5400640 1 -- 172.21.15.66:0/1634387701 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d00a5a30 con 0x7fb6d80cabb0 2024-03-30T00:50:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.363+0000 7fb6d5400640 1 -- 172.21.15.66:0/1634387701 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d00a5dc0 con 0x7fb6d80cabb0 2024-03-30T00:50:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.367+0000 7fb6dd000640 1 -- 172.21.15.66:0/1634387701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d80cabb0 msgr2=0x7fb6d80cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.367+0000 7fb6dd000640 1 --2- 172.21.15.66:0/1634387701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d80cabb0 0x7fb6d80cafe0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009e50 tx=0x7fb6d001ca00 comp rx=0 tx=0).stop 2024-03-30T00:50:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.367+0000 7fb6dd000640 1 -- 172.21.15.66:0/1634387701 shutdown_connections 2024-03-30T00:50:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.367+0000 7fb6dd000640 1 --2- 172.21.15.66:0/1634387701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d80cabb0 0x7fb6d80cafe0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.367+0000 7fb6dd000640 1 -- 172.21.15.66:0/1634387701 >> 172.21.15.66:0/1634387701 conn(0x7fb6d8069f60 msgr2=0x7fb6d806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.367+0000 7fb6dd000640 1 -- 172.21.15.66:0/1634387701 shutdown_connections 2024-03-30T00:50:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.367+0000 7fb6dd000640 1 -- 172.21.15.66:0/1634387701 wait complete. 2024-03-30T00:50:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.367+0000 7fb6dd000640 1 Processor -- start 2024-03-30T00:50:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.367+0000 7fb6dd000640 1 -- start start 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6dd000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6d80cabb0 0x7fb6d80cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6dd000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d80ce520 0x7fb6d8155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6dd000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d815a640 0x7fb6d8155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6dd000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8071250 con 0x7fb6d815a640 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6dd000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80c1c10 con 0x7fb6d80ce520 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6dd000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80ae790 con 0x7fb6d80cabb0 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6d6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d815a640 0x7fb6d8155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6d6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d815a640 0x7fb6d8155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48432/0 (socket says 172.21.15.66:48432) 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6d6800640 1 -- 172.21.15.66:0/3399113229 learned_addr learned my addr 172.21.15.66:0/3399113229 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6cfe00640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d80ce520 0x7fb6d8155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6d5e00640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6d80cabb0 0x7fb6d80cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6d6800640 1 -- 172.21.15.66:0/3399113229 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6d80cabb0 msgr2=0x7fb6d80cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6d6800640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6d80cabb0 0x7fb6d80cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6d6800640 1 -- 172.21.15.66:0/3399113229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d80ce520 msgr2=0x7fb6d8155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6d6800640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d80ce520 0x7fb6d8155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6d6800640 1 -- 172.21.15.66:0/3399113229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d0009b30 con 0x7fb6d815a640 2024-03-30T00:50:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6d6800640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d815a640 0x7fb6d8155bc0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0019610 tx=0x7fb6d00197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:54.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6ce000640 1 -- 172.21.15.66:0/3399113229 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d00b8880 con 0x7fb6d815a640 2024-03-30T00:50:54.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6ce000640 1 -- 172.21.15.66:0/3399113229 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d00b8a00 con 0x7fb6d815a640 2024-03-30T00:50:54.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6ce000640 1 -- 172.21.15.66:0/3399113229 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d00b8db0 con 0x7fb6d815a640 2024-03-30T00:50:54.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6dd000640 1 -- 172.21.15.66:0/3399113229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d80aea70 con 0x7fb6d815a640 2024-03-30T00:50:54.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6dd000640 1 -- 172.21.15.66:0/3399113229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d80af030 con 0x7fb6d815a640 2024-03-30T00:50:54.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.371+0000 7fb6dd000640 1 -- 172.21.15.66:0/3399113229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d8066c30 con 0x7fb6d815a640 2024-03-30T00:50:54.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.375+0000 7fb6ce000640 1 -- 172.21.15.66:0/3399113229 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb6d00c3460 con 0x7fb6d815a640 2024-03-30T00:50:54.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.375+0000 7fb6ce000640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb6b4066690 0x7fb6b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:54.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.375+0000 7fb6ce000640 1 -- 172.21.15.66:0/3399113229 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb6d011fa90 con 0x7fb6d815a640 2024-03-30T00:50:54.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.375+0000 7fb6d5e00640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb6b4066690 0x7fb6b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:54.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.375+0000 7fb6d5e00640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb6b4066690 0x7fb6b4068b50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000f70 tx=0x7fb6c8007e50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:54.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.379+0000 7fb6ce000640 1 -- 172.21.15.66:0/3399113229 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb6d00ecda0 con 0x7fb6d815a640 2024-03-30T00:50:54.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.539+0000 7fb6dd000640 1 -- 172.21.15.66:0/3399113229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6d80af220 con 0x7fb6d815a640 2024-03-30T00:50:54.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6ce000640 1 -- 172.21.15.66:0/3399113229 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16316 (secure 0 0 0) 0x7fb6d00f03c0 con 0x7fb6d815a640 2024-03-30T00:50:54.550 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:50:54.550 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":125,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:39.043056+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:50:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 -- 172.21.15.66:0/3399113229 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb6b4066690 msgr2=0x7fb6b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb6b4066690 0x7fb6b4068b50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000f70 tx=0x7fb6c8007e50 comp rx=0 tx=0).stop 2024-03-30T00:50:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 -- 172.21.15.66:0/3399113229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d815a640 msgr2=0x7fb6d8155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d815a640 0x7fb6d8155bc0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0019610 tx=0x7fb6d00197a0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 -- 172.21.15.66:0/3399113229 shutdown_connections 2024-03-30T00:50:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb6b4066690 0x7fb6b4068b50 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d815a640 0x7fb6d8155bc0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6d80ce520 0x7fb6d8155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 --2- 172.21.15.66:0/3399113229 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6d80cabb0 0x7fb6d80cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 -- 172.21.15.66:0/3399113229 >> 172.21.15.66:0/3399113229 conn(0x7fb6d8069f60 msgr2=0x7fb6d80afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 -- 172.21.15.66:0/3399113229 shutdown_connections 2024-03-30T00:50:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.543+0000 7fb6dd000640 1 -- 172.21.15.66:0/3399113229 wait complete. 2024-03-30T00:50:54.572 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:50:35.696965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:50:37.037165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:50:54.572 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:50:54.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.707+0000 7f319e000640 1 Processor -- start 2024-03-30T00:50:54.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.707+0000 7f319e000640 1 -- start start 2024-03-30T00:50:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f319e000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f31980ccaf0 0x7f3198071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f319e000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ce120 0x7f3198071680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f319e000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319815acf0 con 0x7f31980cfcd0 2024-03-30T00:50:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f319e000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319815afb0 con 0x7f31980ccaf0 2024-03-30T00:50:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f319e000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319815b290 con 0x7f31980ce120 2024-03-30T00:50:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f318fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f31980ccaf0 0x7f3198071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f318fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f31980ccaf0 0x7f3198071140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48442/0 (socket says 172.21.15.66:48442) 2024-03-30T00:50:54.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f318fe00640 1 -- 172.21.15.66:0/2386459659 learned_addr learned my addr 172.21.15.66:0/2386459659 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f319cc00640 1 --1- 172.21.15.66:0/2386459659 >> v1:172.21.15.66:6789/0 conn(0x7f31980cfcd0 0x7f31980cc3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60266/0 (socket says 172.21.15.66:60266) 2024-03-30T00:50:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f319d600640 1 --2- 172.21.15.66:0/2386459659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ce120 0x7f3198071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f318fe00640 1 -- 172.21.15.66:0/2386459659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ce120 msgr2=0x7f3198071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f318fe00640 1 --2- 172.21.15.66:0/2386459659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ce120 0x7f3198071680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f318fe00640 1 -- 172.21.15.66:0/2386459659 >> v1:172.21.15.66:6789/0 conn(0x7f31980cfcd0 legacy=0x7f31980cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f318fe00640 1 -- 172.21.15.66:0/2386459659 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319815b570 con 0x7f31980ccaf0 2024-03-30T00:50:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f319d600640 1 --2- 172.21.15.66:0/2386459659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ce120 0x7f3198071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:54.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f3197e00640 1 -- 172.21.15.66:0/2386459659 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382593464 0 0) 0x7f319815acf0 con 0x7f31980cfcd0 2024-03-30T00:50:54.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.711+0000 7f318fe00640 1 --2- 172.21.15.66:0/2386459659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31980ccaf0 0x7f3198071140 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f3180009870 tx=0x7f318001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf084c2d176470dc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:54.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f3197e00640 1 -- 172.21.15.66:0/2386459659 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31800ab680 con 0x7f31980ccaf0 2024-03-30T00:50:54.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f3197e00640 1 -- 172.21.15.66:0/2386459659 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31800ab7e0 con 0x7f31980ccaf0 2024-03-30T00:50:54.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f3197e00640 1 -- 172.21.15.66:0/2386459659 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31800abb70 con 0x7f31980ccaf0 2024-03-30T00:50:54.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 -- 172.21.15.66:0/2386459659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31980ccaf0 msgr2=0x7f3198071140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 --2- 172.21.15.66:0/2386459659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31980ccaf0 0x7f3198071140 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f3180009870 tx=0x7f318001c4a0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 -- 172.21.15.66:0/2386459659 shutdown_connections 2024-03-30T00:50:54.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 --2- 172.21.15.66:0/2386459659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ce120 0x7f3198071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 --2- 172.21.15.66:0/2386459659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31980ccaf0 0x7f3198071140 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 -- 172.21.15.66:0/2386459659 >> 172.21.15.66:0/2386459659 conn(0x7f319806cf90 msgr2=0x7f319806d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:54.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 -- 172.21.15.66:0/2386459659 shutdown_connections 2024-03-30T00:50:54.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 -- 172.21.15.66:0/2386459659 wait complete. 2024-03-30T00:50:54.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 Processor -- start 2024-03-30T00:50:54.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 -- start start 2024-03-30T00:50:54.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ccaf0 0x7f31980afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31980ce120 0x7f31980b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31980cfcd0 0x7f31980b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980b0190 con 0x7f31980cfcd0 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980b2cd0 con 0x7f31980ce120 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319e000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31980b2f20 con 0x7f31980ccaf0 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31980cfcd0 0x7f31980b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31980cfcd0 0x7f31980b0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48448/0 (socket says 172.21.15.66:48448) 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319d600640 1 -- 172.21.15.66:0/2218207768 learned_addr learned my addr 172.21.15.66:0/2218207768 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f319cc00640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ccaf0 0x7f31980afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.715+0000 7f318fe00640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31980ce120 0x7f31980b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f319d600640 1 -- 172.21.15.66:0/2218207768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ccaf0 msgr2=0x7f31980afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f319d600640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ccaf0 0x7f31980afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f319d600640 1 -- 172.21.15.66:0/2218207768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31980ce120 msgr2=0x7f31980b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f319d600640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31980ce120 0x7f31980b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f319d600640 1 -- 172.21.15.66:0/2218207768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31800094d0 con 0x7f31980cfcd0 2024-03-30T00:50:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f318fe00640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31980ce120 0x7f31980b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f319cc00640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ccaf0 0x7f31980afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f319d600640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31980cfcd0 0x7f31980b0a00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f319000ce80 tx=0x7f319000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f3195600640 1 -- 172.21.15.66:0/2218207768 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000d840 con 0x7f31980cfcd0 2024-03-30T00:50:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f3195600640 1 -- 172.21.15.66:0/2218207768 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319000d9e0 con 0x7f31980cfcd0 2024-03-30T00:50:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f3195600640 1 -- 172.21.15.66:0/2218207768 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000dd90 con 0x7f31980cfcd0 2024-03-30T00:50:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f319e000640 1 -- 172.21.15.66:0/2218207768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31980b3200 con 0x7f31980cfcd0 2024-03-30T00:50:54.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f319e000640 1 -- 172.21.15.66:0/2218207768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31980b37c0 con 0x7f31980cfcd0 2024-03-30T00:50:54.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f319e000640 1 -- 172.21.15.66:0/2218207768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3198069eb0 con 0x7f31980cfcd0 2024-03-30T00:50:54.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.719+0000 7f3195600640 1 -- 172.21.15.66:0/2218207768 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3190015440 con 0x7f31980cfcd0 2024-03-30T00:50:54.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.723+0000 7f3195600640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3174066690 0x7f3174068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:54.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.723+0000 7f3195600640 1 -- 172.21.15.66:0/2218207768 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3190084cb0 con 0x7f31980cfcd0 2024-03-30T00:50:54.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.723+0000 7f319cc00640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3174066690 0x7f3174068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:54.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.723+0000 7f319cc00640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3174066690 0x7f3174068b50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3188000f70 tx=0x7f3188007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.727+0000 7f3195600640 1 -- 172.21.15.66:0/2218207768 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3190051ef0 con 0x7f31980cfcd0 2024-03-30T00:50:54.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 -- 172.21.15.66:0/2218207768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3198073e00 con 0x7f31980cfcd0 2024-03-30T00:50:54.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f3195600640 1 -- 172.21.15.66:0/2218207768 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f319001a020 con 0x7f31980cfcd0 2024-03-30T00:50:54.898 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:50:54.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 -- 172.21.15.66:0/2218207768 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3174066690 msgr2=0x7f3174068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3174066690 0x7f3174068b50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3188000f70 tx=0x7f3188007cf0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 -- 172.21.15.66:0/2218207768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31980cfcd0 msgr2=0x7f31980b0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:54.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31980cfcd0 0x7f31980b0a00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f319000ce80 tx=0x7f319000a710 comp rx=0 tx=0).stop 2024-03-30T00:50:54.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 -- 172.21.15.66:0/2218207768 shutdown_connections 2024-03-30T00:50:54.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f3174066690 0x7f3174068b50 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31980cfcd0 0x7f31980b0a00 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31980ce120 0x7f31980b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 --2- 172.21.15.66:0/2218207768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31980ccaf0 0x7f31980afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:54.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 -- 172.21.15.66:0/2218207768 >> 172.21.15.66:0/2218207768 conn(0x7f319806cf90 msgr2=0x7f319815a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:54.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 -- 172.21.15.66:0/2218207768 shutdown_connections 2024-03-30T00:50:54.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:54.891+0000 7f319e000640 1 -- 172.21.15.66:0/2218207768 wait complete. 2024-03-30T00:50:54.920 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:50:55.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f1253200640 1 Processor -- start 2024-03-30T00:50:55.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f1253200640 1 -- start start 2024-03-30T00:50:55.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f1253200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c0cf2f0 0x7f124c06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f1253200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c0cb0e0 con 0x7f124c15ca80 2024-03-30T00:50:55.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f1253200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c0cf7f0 con 0x7f124c0cb570 2024-03-30T00:50:55.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f1253200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0cfad0 con 0x7f124c0cf2f0 2024-03-30T00:50:55.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f124be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f124c15ca80 0x7f124c06b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60278/0 (socket says 172.21.15.66:60278) 2024-03-30T00:50:55.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f124be00640 1 -- 172.21.15.66:0/4059924213 learned_addr learned my addr 172.21.15.66:0/4059924213 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:55.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f1243400640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c0cf2f0 0x7f124c06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680735112 0 0) 0x7f124c0cb0e0 con 0x7f124c15ca80 2024-03-30T00:50:55.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1230003530 con 0x7f124c15ca80 2024-03-30T00:50:55.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3594008182 0 0) 0x7f1230003530 con 0x7f124c15ca80 2024-03-30T00:50:55.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c0cf2f0 msgr2=0x7f124c06b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f124b400640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c0cf2f0 0x7f124c06b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.055+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 >> v1:172.21.15.156:6789/0 conn(0x7f124c0cb570 legacy=0x7f124c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0cad70 con 0x7f124c15ca80 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f123c001a00 con 0x7f124c15ca80 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 >> v1:172.21.15.66:6789/0 conn(0x7f124c15ca80 legacy=0x7f124c06b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124b400640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12300039b0 0x7f1230003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124b400640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1230004360 0x7f1230004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124b400640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c0cf2f0 0x7f1230005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0cfad0 con 0x7f1230004360 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123000b990 con 0x7f12300039b0 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123000bb50 con 0x7f124c0cf2f0 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f1243400640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1230004360 0x7f1230004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124be00640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12300039b0 0x7f1230003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f1251200640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c0cf2f0 0x7f1230005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124be00640 1 -- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c0cf2f0 msgr2=0x7f1230005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:55.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124be00640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c0cf2f0 0x7f1230005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124be00640 1 -- 172.21.15.66:0/4059924213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1230004360 msgr2=0x7f1230004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:55.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124be00640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1230004360 0x7f1230004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124be00640 1 -- 172.21.15.66:0/4059924213 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1230003530 con 0x7f12300039b0 2024-03-30T00:50:55.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f1243400640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1230004360 0x7f1230004830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:50:55.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.059+0000 7f124be00640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12300039b0 0x7f1230003d90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f123c00a860 tx=0x7f123c01fac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b292b58a3150dc40 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:55.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c0aa6f0 con 0x7f12300039b0 2024-03-30T00:50:55.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c0aa870 con 0x7f12300039b0 2024-03-30T00:50:55.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f124b400640 1 -- 172.21.15.66:0/4059924213 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c0aac20 con 0x7f12300039b0 2024-03-30T00:50:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 -- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12300039b0 msgr2=0x7f1230003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 --2- 172.21.15.66:0/4059924213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12300039b0 0x7f1230003d90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f123c00a860 tx=0x7f123c01fac0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1251200640 1 -- 172.21.15.66:0/4059924213 reap_dead start 2024-03-30T00:50:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 -- 172.21.15.66:0/4059924213 shutdown_connections 2024-03-30T00:50:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 -- 172.21.15.66:0/4059924213 >> 172.21.15.66:0/4059924213 conn(0x7f124c0b2090 msgr2=0x7f124c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 -- 172.21.15.66:0/4059924213 shutdown_connections 2024-03-30T00:50:55.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 -- 172.21.15.66:0/4059924213 wait complete. 2024-03-30T00:50:55.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 Processor -- start 2024-03-30T00:50:55.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 -- start start 2024-03-30T00:50:55.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12300039b0 0x7f124c15a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c15ae30 0x7f124c09cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f124c159ee0 0x7f124c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c09e300 con 0x7f124c159ee0 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c09d260 con 0x7f12300039b0 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1253200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c09d540 con 0x7f124c15ae30 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1251200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f124c159ee0 0x7f124c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1251200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f124c159ee0 0x7f124c0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48470/0 (socket says 172.21.15.66:48470) 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1251200640 1 -- 172.21.15.66:0/1636478001 learned_addr learned my addr 172.21.15.66:0/1636478001 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f124be00640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12300039b0 0x7f124c15a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.063+0000 7f1243400640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c15ae30 0x7f124c09cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1251200640 1 -- 172.21.15.66:0/1636478001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c15ae30 msgr2=0x7f124c09cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1251200640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c15ae30 0x7f124c09cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1251200640 1 -- 172.21.15.66:0/1636478001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12300039b0 msgr2=0x7f124c15a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1251200640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12300039b0 0x7f124c15a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1251200640 1 -- 172.21.15.66:0/1636478001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c09d820 con 0x7f124c159ee0 2024-03-30T00:50:55.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f124be00640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12300039b0 0x7f124c15a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:55.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1251200640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f124c159ee0 0x7f124c0b5390 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f124c071450 tx=0x7f124400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:55.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1248c00640 1 -- 172.21.15.66:0/1636478001 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244005030 con 0x7f124c159ee0 2024-03-30T00:50:55.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1248c00640 1 -- 172.21.15.66:0/1636478001 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1244005190 con 0x7f124c159ee0 2024-03-30T00:50:55.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1248c00640 1 -- 172.21.15.66:0/1636478001 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440067e0 con 0x7f124c159ee0 2024-03-30T00:50:55.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1253200640 1 -- 172.21.15.66:0/1636478001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c09db00 con 0x7f124c159ee0 2024-03-30T00:50:55.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1253200640 1 -- 172.21.15.66:0/1636478001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c15a360 con 0x7f124c159ee0 2024-03-30T00:50:55.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1253200640 1 -- 172.21.15.66:0/1636478001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124c066c30 con 0x7f124c159ee0 2024-03-30T00:50:55.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1248c00640 1 -- 172.21.15.66:0/1636478001 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1244006940 con 0x7f124c159ee0 2024-03-30T00:50:55.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1248c00640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12280666c0 0x7f1228068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f1248c00640 1 -- 172.21.15.66:0/1636478001 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1244082fe0 con 0x7f124c159ee0 2024-03-30T00:50:55.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.067+0000 7f124be00640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12280666c0 0x7f1228068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.071+0000 7f124be00640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12280666c0 0x7f1228068b80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f123c0b1000 tx=0x7f123c0ab000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:55.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.071+0000 7f1248c00640 1 -- 172.21.15.66:0/1636478001 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1244050210 con 0x7f124c159ee0 2024-03-30T00:50:55.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.235+0000 7f1253200640 1 -- 172.21.15.66:0/1636478001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f124c09dfb0 con 0x7f124c159ee0 2024-03-30T00:50:55.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.235+0000 7f1248c00640 1 -- 172.21.15.66:0/1636478001 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1244053830 con 0x7f124c159ee0 2024-03-30T00:50:55.245 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:50:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 -- 172.21.15.66:0/1636478001 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12280666c0 msgr2=0x7f1228068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12280666c0 0x7f1228068b80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f123c0b1000 tx=0x7f123c0ab000 comp rx=0 tx=0).stop 2024-03-30T00:50:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 -- 172.21.15.66:0/1636478001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f124c159ee0 msgr2=0x7f124c0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f124c159ee0 0x7f124c0b5390 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f124c071450 tx=0x7f124400fc00 comp rx=0 tx=0).stop 2024-03-30T00:50:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 -- 172.21.15.66:0/1636478001 shutdown_connections 2024-03-30T00:50:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f12280666c0 0x7f1228068b80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f124c159ee0 0x7f124c0b5390 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f124c15ae30 0x7f124c09cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 --2- 172.21.15.66:0/1636478001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12300039b0 0x7f124c15a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 -- 172.21.15.66:0/1636478001 >> 172.21.15.66:0/1636478001 conn(0x7f124c0b2090 msgr2=0x7f124c0c60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 -- 172.21.15.66:0/1636478001 shutdown_connections 2024-03-30T00:50:55.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.239+0000 7f1253200640 1 -- 172.21.15.66:0/1636478001 wait complete. 2024-03-30T00:50:55.265 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:50:55.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.399+0000 7f20dd800640 1 Processor -- start 2024-03-30T00:50:55.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.399+0000 7f20dd800640 1 -- start start 2024-03-30T00:50:55.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.399+0000 7f20dd800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f20d815de30 0x7f20d80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.399+0000 7f20dd800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f20d80af420 0x7f20d80d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.399+0000 7f20dd800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d80cbb90 con 0x7f20d80ce740 2024-03-30T00:50:55.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.399+0000 7f20dd800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d80ce1b0 con 0x7f20d815de30 2024-03-30T00:50:55.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.399+0000 7f20dd800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d80afa10 con 0x7f20d80af420 2024-03-30T00:50:55.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.399+0000 7f20d6a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f20d815de30 0x7f20d80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.399+0000 7f20d7400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f20d80ce740 0x7f20d806e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60290/0 (socket says 172.21.15.66:60290) 2024-03-30T00:50:55.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.399+0000 7f20d7400640 1 -- 172.21.15.66:0/4278380763 learned_addr learned my addr 172.21.15.66:0/4278380763 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:55.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.399+0000 7f20cfe00640 1 --2- 172.21.15.66:0/4278380763 >> v2:172.21.15.156:3300/0 conn(0x7f20d80af420 0x7f20d80d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20d6a00640 1 -- 172.21.15.66:0/4278380763 >> v2:172.21.15.156:3300/0 conn(0x7f20d80af420 msgr2=0x7f20d80d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:55.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20d6a00640 1 --2- 172.21.15.66:0/4278380763 >> v2:172.21.15.156:3300/0 conn(0x7f20d80af420 0x7f20d80d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20d6a00640 1 -- 172.21.15.66:0/4278380763 >> v1:172.21.15.66:6789/0 conn(0x7f20d80ce740 legacy=0x7f20d806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20d6a00640 1 -- 172.21.15.66:0/4278380763 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d80cf430 con 0x7f20d815de30 2024-03-30T00:50:55.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20d6000640 1 -- 172.21.15.66:0/4278380763 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651575089 0 0) 0x7f20d80cbb90 con 0x7f20d80ce740 2024-03-30T00:50:55.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20d6a00640 1 --2- 172.21.15.66:0/4278380763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20d815de30 0x7f20d80cc040 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f20d8070080 tx=0x7f20c801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1a847ffd791f9c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:55.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20d6000640 1 -- 172.21.15.66:0/4278380763 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8004230 con 0x7f20d815de30 2024-03-30T00:50:55.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20d6000640 1 -- 172.21.15.66:0/4278380763 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c80a7050 con 0x7f20d815de30 2024-03-30T00:50:55.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20d6000640 1 -- 172.21.15.66:0/4278380763 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8003620 con 0x7f20d815de30 2024-03-30T00:50:55.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20dd800640 1 -- 172.21.15.66:0/4278380763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20d815de30 msgr2=0x7f20d80cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20dd800640 1 --2- 172.21.15.66:0/4278380763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20d815de30 0x7f20d80cc040 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f20d8070080 tx=0x7f20c801c3f0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20dd800640 1 -- 172.21.15.66:0/4278380763 shutdown_connections 2024-03-30T00:50:55.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20dd800640 1 --2- 172.21.15.66:0/4278380763 >> v2:172.21.15.156:3300/0 conn(0x7f20d80af420 0x7f20d80d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20dd800640 1 --2- 172.21.15.66:0/4278380763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20d815de30 0x7f20d80cc040 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20dd800640 1 -- 172.21.15.66:0/4278380763 >> 172.21.15.66:0/4278380763 conn(0x7f20d80d0540 msgr2=0x7f20d80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:55.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20dd800640 1 -- 172.21.15.66:0/4278380763 shutdown_connections 2024-03-30T00:50:55.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.403+0000 7f20dd800640 1 -- 172.21.15.66:0/4278380763 wait complete. 2024-03-30T00:50:55.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20dd800640 1 Processor -- start 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20dd800640 1 -- start start 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20dd800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20d80af420 0x7f20d8164240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20dd800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20d80ce740 0x7f20d81633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20dd800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d815de30 0x7f20d80b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20dd800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8162920 con 0x7f20d80ce740 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20dd800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8164a90 con 0x7f20d815de30 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20dd800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8164d70 con 0x7f20d80af420 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20d80ce740 0x7f20d81633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20d80ce740 0x7f20d81633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48490/0 (socket says 172.21.15.66:48490) 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cfe00640 1 -- 172.21.15.66:0/3983162370 learned_addr learned my addr 172.21.15.66:0/3983162370 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20d6a00640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20d80af420 0x7f20d8164240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cfe00640 1 -- 172.21.15.66:0/3983162370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20d80af420 msgr2=0x7f20d8164240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20d7400640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d815de30 0x7f20d80b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cfe00640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20d80af420 0x7f20d8164240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cfe00640 1 -- 172.21.15.66:0/3983162370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d815de30 msgr2=0x7f20d80b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cfe00640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d815de30 0x7f20d80b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cfe00640 1 -- 172.21.15.66:0/3983162370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c8009420 con 0x7f20d80ce740 2024-03-30T00:50:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20d6a00640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20d80af420 0x7f20d8164240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20d7400640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d815de30 0x7f20d80b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:50:55.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cfe00640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20d80ce740 0x7f20d81633e0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f20c000e870 tx=0x7f20c000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:55.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cea00640 1 -- 172.21.15.66:0/3983162370 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0009510 con 0x7f20d80ce740 2024-03-30T00:50:55.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cea00640 1 -- 172.21.15.66:0/3983162370 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c000eec0 con 0x7f20d80ce740 2024-03-30T00:50:55.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20cea00640 1 -- 172.21.15.66:0/3983162370 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c000cb40 con 0x7f20d80ce740 2024-03-30T00:50:55.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20dd800640 1 -- 172.21.15.66:0/3983162370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d8162be0 con 0x7f20d80ce740 2024-03-30T00:50:55.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.407+0000 7f20dd800640 1 -- 172.21.15.66:0/3983162370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d8164780 con 0x7f20d80ce740 2024-03-30T00:50:55.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.411+0000 7f20dd800640 1 -- 172.21.15.66:0/3983162370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d80cdf60 con 0x7f20d80ce740 2024-03-30T00:50:55.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.411+0000 7f20cea00640 1 -- 172.21.15.66:0/3983162370 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f20c000cca0 con 0x7f20d80ce740 2024-03-30T00:50:55.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.411+0000 7f20cea00640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f20b40666e0 0x7f20b4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.411+0000 7f20cea00640 1 -- 172.21.15.66:0/3983162370 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f20c0084460 con 0x7f20d80ce740 2024-03-30T00:50:55.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.411+0000 7f20d6a00640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f20b40666e0 0x7f20b4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.411+0000 7f20d6a00640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f20b40666e0 0x7f20b4068ba0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f20c80ad000 tx=0x7f20c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:55.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.415+0000 7f20cea00640 1 -- 172.21.15.66:0/3983162370 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f20c0051670 con 0x7f20d80ce740 2024-03-30T00:50:55.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.575+0000 7f20dd800640 1 -- 172.21.15.66:0/3983162370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20d8066c30 con 0x7f20d80ce740 2024-03-30T00:50:55.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.575+0000 7f20cea00640 1 -- 172.21.15.66:0/3983162370 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20c0054c90 con 0x7f20d80ce740 2024-03-30T00:50:55.585 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:50:55.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 -- 172.21.15.66:0/3983162370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f20b40666e0 msgr2=0x7f20b4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f20b40666e0 0x7f20b4068ba0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f20c80ad000 tx=0x7f20c8008040 comp rx=0 tx=0).stop 2024-03-30T00:50:55.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 -- 172.21.15.66:0/3983162370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20d80ce740 msgr2=0x7f20d81633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20d80ce740 0x7f20d81633e0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f20c000e870 tx=0x7f20c000ec70 comp rx=0 tx=0).stop 2024-03-30T00:50:55.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 -- 172.21.15.66:0/3983162370 shutdown_connections 2024-03-30T00:50:55.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d815de30 0x7f20d80b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f20b40666e0 0x7f20b4068ba0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20d80ce740 0x7f20d81633e0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 --2- 172.21.15.66:0/3983162370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20d80af420 0x7f20d8164240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 -- 172.21.15.66:0/3983162370 >> 172.21.15.66:0/3983162370 conn(0x7f20d80d0540 msgr2=0x7f20d80cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:55.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 -- 172.21.15.66:0/3983162370 shutdown_connections 2024-03-30T00:50:55.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.579+0000 7f20dd800640 1 -- 172.21.15.66:0/3983162370 wait complete. 2024-03-30T00:50:55.605 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:50:55.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b8200640 1 Processor -- start 2024-03-30T00:50:55.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b8200640 1 -- start start 2024-03-30T00:50:55.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b8200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd6b00cde70 0x7fd6b015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b8200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b00cd3e0 con 0x7fd6b00cec30 2024-03-30T00:50:55.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b8200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b015a990 con 0x7fd6b00cda90 2024-03-30T00:50:55.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b8200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b00b1b80 con 0x7fd6b00cde70 2024-03-30T00:50:55.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b4e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd6b00cec30 0x7fd6b0159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60304/0 (socket says 172.21.15.66:60304) 2024-03-30T00:50:55.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b4e00640 1 -- 172.21.15.66:0/860215035 learned_addr learned my addr 172.21.15.66:0/860215035 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:50:55.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214996560 0 0) 0x7fd6b00cd3e0 con 0x7fd6b00cec30 2024-03-30T00:50:55.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd698005740 con 0x7fd6b00cec30 2024-03-30T00:50:55.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b6200640 1 --2- 172.21.15.66:0/860215035 >> v2:172.21.15.156:3300/0 conn(0x7fd6b00cde70 0x7fd6b015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1643656184 0 0) 0x7fd698005740 con 0x7fd6b00cec30 2024-03-30T00:50:55.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 >> v2:172.21.15.156:3300/0 conn(0x7fd6b00cde70 msgr2=0x7fd6b015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:55.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 --2- 172.21.15.66:0/860215035 >> v2:172.21.15.156:3300/0 conn(0x7fd6b00cde70 0x7fd6b015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 >> v1:172.21.15.156:6789/0 conn(0x7fd6b00cda90 legacy=0x7fd6b00ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b00b11e0 con 0x7fd6b00cec30 2024-03-30T00:50:55.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fd69c001680 con 0x7fd6b00cec30 2024-03-30T00:50:55.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 >> v1:172.21.15.66:6789/0 conn(0x7fd6b00cec30 legacy=0x7fd6b0159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 --2- 172.21.15.66:0/860215035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd698005be0 0x7fd698006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 --2- 172.21.15.66:0/860215035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6980069e0 0x7fd698006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 --2- 172.21.15.66:0/860215035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6980073f0 0x7fd69800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00b1b80 con 0x7fd6980069e0 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69800e340 con 0x7fd6980073f0 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69800e500 con 0x7fd698005be0 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b4e00640 1 --2- 172.21.15.66:0/860215035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6980069e0 0x7fd698006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b5800640 1 --2- 172.21.15.66:0/860215035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd698005be0 0x7fd698006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b6200640 1 --2- 172.21.15.66:0/860215035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6980073f0 0x7fd69800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b4e00640 1 -- 172.21.15.66:0/860215035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd698005be0 msgr2=0x7fd698006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b4e00640 1 --2- 172.21.15.66:0/860215035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd698005be0 0x7fd698006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b4e00640 1 -- 172.21.15.66:0/860215035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6980073f0 msgr2=0x7fd69800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b4e00640 1 --2- 172.21.15.66:0/860215035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6980073f0 0x7fd69800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b4e00640 1 -- 172.21.15.66:0/860215035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698005740 con 0x7fd6980069e0 2024-03-30T00:50:55.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.747+0000 7fd6b6200640 1 -- 172.21.15.66:0/860215035 reap_dead start 2024-03-30T00:50:55.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.751+0000 7fd6b4e00640 1 --2- 172.21.15.66:0/860215035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6980069e0 0x7fd698006e60 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fd69c001ec0 tx=0x7fd69c01caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a81d641fa353ce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:55.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.751+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c005ae0 con 0x7fd6980069e0 2024-03-30T00:50:55.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.751+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c005c40 con 0x7fd6980069e0 2024-03-30T00:50:55.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.751+0000 7fd6afe00640 1 -- 172.21.15.66:0/860215035 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0072b0 con 0x7fd6980069e0 2024-03-30T00:50:55.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.751+0000 7fd6b8200640 1 -- 172.21.15.66:0/860215035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6980069e0 msgr2=0x7fd698006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.751+0000 7fd6b8200640 1 --2- 172.21.15.66:0/860215035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6980069e0 0x7fd698006e60 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fd69c001ec0 tx=0x7fd69c01caa0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.751+0000 7fd6b8200640 1 -- 172.21.15.66:0/860215035 shutdown_connections 2024-03-30T00:50:55.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.751+0000 7fd6b8200640 1 --2- 172.21.15.66:0/860215035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6980069e0 0x7fd698006e60 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.751+0000 7fd6b8200640 1 -- 172.21.15.66:0/860215035 >> 172.21.15.66:0/860215035 conn(0x7fd6b0069f60 msgr2=0x7fd6b006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:55.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.751+0000 7fd6b8200640 1 -- 172.21.15.66:0/860215035 shutdown_connections 2024-03-30T00:50:55.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.751+0000 7fd6b8200640 1 -- 172.21.15.66:0/860215035 wait complete. 2024-03-30T00:50:55.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b8200640 1 Processor -- start 2024-03-30T00:50:55.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b8200640 1 -- start start 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b8200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6980069e0 0x7fd6b00ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b8200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6b00ad050 0x7fd6b015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b8200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b006fed0 0x7fd6b0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b8200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00ab660 con 0x7fd6b00ad050 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b8200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00ab940 con 0x7fd6b006fed0 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b8200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00abc20 con 0x7fd6980069e0 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6b00ad050 0x7fd6b015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6b00ad050 0x7fd6b015bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48508/0 (socket says 172.21.15.66:48508) 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b4e00640 1 -- 172.21.15.66:0/2457912829 learned_addr learned my addr 172.21.15.66:0/2457912829 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b6200640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b006fed0 0x7fd6b0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b5800640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6980069e0 0x7fd6b00ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b4e00640 1 -- 172.21.15.66:0/2457912829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6980069e0 msgr2=0x7fd6b00ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b4e00640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6980069e0 0x7fd6b00ab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b4e00640 1 -- 172.21.15.66:0/2457912829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b006fed0 msgr2=0x7fd6b0070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b4e00640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b006fed0 0x7fd6b0070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b4e00640 1 -- 172.21.15.66:0/2457912829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b00abf00 con 0x7fd6b00ad050 2024-03-30T00:50:55.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b6200640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b006fed0 0x7fd6b0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:50:55.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b4e00640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6b00ad050 0x7fd6b015bef0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fd69c01cae0 tx=0x7fd698007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:55.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6ad600640 1 -- 172.21.15.66:0/2457912829 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0b4070 con 0x7fd6b00ad050 2024-03-30T00:50:55.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6ad600640 1 -- 172.21.15.66:0/2457912829 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c0015c0 con 0x7fd6b00ad050 2024-03-30T00:50:55.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6ad600640 1 -- 172.21.15.66:0/2457912829 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0072b0 con 0x7fd6b00ad050 2024-03-30T00:50:55.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b8200640 1 -- 172.21.15.66:0/2457912829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b00c69a0 con 0x7fd6b00ad050 2024-03-30T00:50:55.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.755+0000 7fd6b8200640 1 -- 172.21.15.66:0/2457912829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b015c430 con 0x7fd6b00ad050 2024-03-30T00:50:55.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.759+0000 7fd6b8200640 1 -- 172.21.15.66:0/2457912829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b00c6c50 con 0x7fd6b00ad050 2024-03-30T00:50:55.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.759+0000 7fd6ad600640 1 -- 172.21.15.66:0/2457912829 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd69c007410 con 0x7fd6b00ad050 2024-03-30T00:50:55.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.759+0000 7fd6ad600640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd6900666c0 0x7fd690068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:50:55.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.759+0000 7fd6ad600640 1 -- 172.21.15.66:0/2457912829 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd69c11da10 con 0x7fd6b00ad050 2024-03-30T00:50:55.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.759+0000 7fd6b5800640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd6900666c0 0x7fd690068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:50:55.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.759+0000 7fd6b5800640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd6900666c0 0x7fd690068b80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4000c10 tx=0x7fd6a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:50:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.763+0000 7fd6ad600640 1 -- 172.21.15.66:0/2457912829 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd6b00c6c50 con 0x7fd6b00ad050 2024-03-30T00:50:55.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.935+0000 7fd6b8200640 1 -- 172.21.15.66:0/2457912829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd6b00c6c50 con 0x7fd6b00ad050 2024-03-30T00:50:55.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.935+0000 7fd6ad600640 1 -- 172.21.15.66:0/2457912829 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd6b00c6c50 con 0x7fd6b00ad050 2024-03-30T00:50:55.944 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:50:55.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 -- 172.21.15.66:0/2457912829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd6900666c0 msgr2=0x7fd690068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd6900666c0 0x7fd690068b80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4000c10 tx=0x7fd6a4009040 comp rx=0 tx=0).stop 2024-03-30T00:50:55.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 -- 172.21.15.66:0/2457912829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6b00ad050 msgr2=0x7fd6b015bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:50:55.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6b00ad050 0x7fd6b015bef0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fd69c01cae0 tx=0x7fd698007a10 comp rx=0 tx=0).stop 2024-03-30T00:50:55.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 -- 172.21.15.66:0/2457912829 shutdown_connections 2024-03-30T00:50:55.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b006fed0 0x7fd6b0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd6900666c0 0x7fd690068b80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6b00ad050 0x7fd6b015bef0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 --2- 172.21.15.66:0/2457912829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6980069e0 0x7fd6b00ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:50:55.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 -- 172.21.15.66:0/2457912829 >> 172.21.15.66:0/2457912829 conn(0x7fd6b0069f60 msgr2=0x7fd6b006c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:50:55.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 -- 172.21.15.66:0/2457912829 shutdown_connections 2024-03-30T00:50:55.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:50:55.939+0000 7fd6b8200640 1 -- 172.21.15.66:0/2457912829 wait complete. 2024-03-30T00:50:55.965 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:50:55.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:50:55.965 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:50:55.984 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[53839]: fuse finished with error 0 and tester_r 0 2024-03-30T00:50:55.986 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:51:02.088 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:51:02.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:02.088 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:51:02.098 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-03-30T00:51:02.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facab200640 1 Processor -- start 2024-03-30T00:51:02.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facab200640 1 -- start start 2024-03-30T00:51:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facab200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faca415e0a0 0x7faca415e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facab200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faca40ca9d0 0x7faca40cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facab200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca415ba30 con 0x7faca4158210 2024-03-30T00:51:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facab200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca40ce7d0 con 0x7faca40ca9d0 2024-03-30T00:51:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facab200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca40cb320 con 0x7faca415e0a0 2024-03-30T00:51:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facaa800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faca40ca9d0 0x7faca40cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7faca9e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7faca4158210 0x7faca40cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36414/0 (socket says 172.21.15.66:36414) 2024-03-30T00:51:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7faca9e00640 1 -- 172.21.15.66:0/3688202822 learned_addr learned my addr 172.21.15.66:0/3688202822 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7faca9400640 1 --2- 172.21.15.66:0/3688202822 >> v2:172.21.15.156:3300/0 conn(0x7faca415e0a0 0x7faca415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facaa800640 1 -- 172.21.15.66:0/3688202822 >> v2:172.21.15.156:3300/0 conn(0x7faca415e0a0 msgr2=0x7faca415e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:02.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facaa800640 1 --2- 172.21.15.66:0/3688202822 >> v2:172.21.15.156:3300/0 conn(0x7faca415e0a0 0x7faca415e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facaa800640 1 -- 172.21.15.66:0/3688202822 >> v1:172.21.15.66:6789/0 conn(0x7faca4158210 legacy=0x7faca40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.287+0000 7facaa800640 1 -- 172.21.15.66:0/3688202822 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca415add0 con 0x7faca40ca9d0 2024-03-30T00:51:02.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7facaa800640 1 --2- 172.21.15.66:0/3688202822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca40ca9d0 0x7faca40cade0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7faca415f1a0 tx=0x7faca001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86dffa9219f97a18 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7faca8a00640 1 -- 172.21.15.66:0/3688202822 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00a59c0 con 0x7faca40ca9d0 2024-03-30T00:51:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7faca8a00640 1 -- 172.21.15.66:0/3688202822 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca00a5b20 con 0x7faca40ca9d0 2024-03-30T00:51:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7faca8a00640 1 -- 172.21.15.66:0/3688202822 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00ad440 con 0x7faca40ca9d0 2024-03-30T00:51:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7facab200640 1 -- 172.21.15.66:0/3688202822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca40ca9d0 msgr2=0x7faca40cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7facab200640 1 --2- 172.21.15.66:0/3688202822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca40ca9d0 0x7faca40cade0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7faca415f1a0 tx=0x7faca001c900 comp rx=0 tx=0).stop 2024-03-30T00:51:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7facab200640 1 -- 172.21.15.66:0/3688202822 shutdown_connections 2024-03-30T00:51:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7facab200640 1 --2- 172.21.15.66:0/3688202822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca40ca9d0 0x7faca40cade0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7facab200640 1 --2- 172.21.15.66:0/3688202822 >> v2:172.21.15.156:3300/0 conn(0x7faca415e0a0 0x7faca415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7facab200640 1 -- 172.21.15.66:0/3688202822 >> 172.21.15.66:0/3688202822 conn(0x7faca406fab0 msgr2=0x7faca406fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:02.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7facab200640 1 -- 172.21.15.66:0/3688202822 shutdown_connections 2024-03-30T00:51:02.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.291+0000 7facab200640 1 -- 172.21.15.66:0/3688202822 wait complete. 2024-03-30T00:51:02.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facab200640 1 Processor -- start 2024-03-30T00:51:02.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facab200640 1 -- start start 2024-03-30T00:51:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facab200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faca40ca9d0 0x7faca40ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facab200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faca4158210 0x7faca4152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facab200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca415e0a0 0x7faca41533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facab200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca40c9670 con 0x7faca415e0a0 2024-03-30T00:51:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facab200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca40c9950 con 0x7faca40ca9d0 2024-03-30T00:51:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facab200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca40c9c30 con 0x7faca4158210 2024-03-30T00:51:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facaa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca415e0a0 0x7faca41533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facaa800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca415e0a0 0x7faca41533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48666/0 (socket says 172.21.15.66:48666) 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facaa800640 1 -- 172.21.15.66:0/224001551 learned_addr learned my addr 172.21.15.66:0/224001551 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7faca9400640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faca4158210 0x7faca4152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7faca9e00640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faca40ca9d0 0x7faca40ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facaa800640 1 -- 172.21.15.66:0/224001551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faca4158210 msgr2=0x7faca4152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facaa800640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faca4158210 0x7faca4152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facaa800640 1 -- 172.21.15.66:0/224001551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faca40ca9d0 msgr2=0x7faca40ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facaa800640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faca40ca9d0 0x7faca40ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facaa800640 1 -- 172.21.15.66:0/224001551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca0009a70 con 0x7faca415e0a0 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7faca9400640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faca4158210 0x7faca4152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facaa800640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca415e0a0 0x7faca41533f0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7faca0019510 tx=0x7faca00196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7fac9e000640 1 -- 172.21.15.66:0/224001551 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00ade20 con 0x7faca415e0a0 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7fac9e000640 1 -- 172.21.15.66:0/224001551 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca0009450 con 0x7faca415e0a0 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7fac9e000640 1 -- 172.21.15.66:0/224001551 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00b9690 con 0x7faca415e0a0 2024-03-30T00:51:02.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facab200640 1 -- 172.21.15.66:0/224001551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca40c9f10 con 0x7faca415e0a0 2024-03-30T00:51:02.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.295+0000 7facab200640 1 -- 172.21.15.66:0/224001551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca40cd480 con 0x7faca415e0a0 2024-03-30T00:51:02.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7fac9e000640 1 -- 172.21.15.66:0/224001551 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faca00b97f0 con 0x7faca415e0a0 2024-03-30T00:51:02.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7fac9e000640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fac88066690 0x7fac88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7fac9e000640 1 -- 172.21.15.66:0/224001551 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faca011fca0 con 0x7faca415e0a0 2024-03-30T00:51:02.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9e00640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fac88066690 0x7fac88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7facafd3c000 1 Processor -- start 2024-03-30T00:51:02.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7facafd3c000 1 -- start start 2024-03-30T00:51:02.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9e00640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fac88066690 0x7fac88068b50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fac98005890 tx=0x7fac98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:02.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7facafd3c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9d47e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7facafd3c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d48390 0x5594c9cfb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7facafd3c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9cfba00 0x5594c9cfbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7facafd3c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c9bc6630 con 0x5594c9d47a70 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7facafd3c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c9b53bd0 con 0x5594c9d48390 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7facafd3c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594c9b511a0 con 0x5594c9cfba00 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9d47e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9d47e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48668/0 (socket says 172.21.15.66:48668) 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9e00640 1 -- 172.21.15.66:0/2952137793 learned_addr learned my addr 172.21.15.66:0/2952137793 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9400640 1 --2- 172.21.15.66:0/2952137793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d48390 0x5594c9cfb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7facaa800640 1 --2- 172.21.15.66:0/2952137793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9cfba00 0x5594c9cfbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9e00640 1 -- 172.21.15.66:0/2952137793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9cfba00 msgr2=0x5594c9cfbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9e00640 1 --2- 172.21.15.66:0/2952137793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9cfba00 0x5594c9cfbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9e00640 1 -- 172.21.15.66:0/2952137793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d48390 msgr2=0x5594c9cfb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:02.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9e00640 1 --2- 172.21.15.66:0/2952137793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d48390 0x5594c9cfb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9e00640 1 -- 172.21.15.66:0/2952137793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594c9848e20 con 0x5594c9d47a70 2024-03-30T00:51:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7faca9e00640 1 --2- 172.21.15.66:0/2952137793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9d47e50 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fac98001eb0 tx=0x7fac98011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03c13f52f97ec60 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.299+0000 7fac92a00640 1 -- 172.21.15.66:0/2952137793 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98005e10 con 0x5594c9d47a70 2024-03-30T00:51:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7fac92a00640 1 -- 172.21.15.66:0/2952137793 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9800e7b0 con 0x5594c9d47a70 2024-03-30T00:51:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7fac92a00640 1 -- 172.21.15.66:0/2952137793 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9800eb20 con 0x5594c9d47a70 2024-03-30T00:51:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 -- 172.21.15.66:0/2952137793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 msgr2=0x5594c9d47e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 --2- 172.21.15.66:0/2952137793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9d47e50 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fac98001eb0 tx=0x7fac98011000 comp rx=0 tx=0).stop 2024-03-30T00:51:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 -- 172.21.15.66:0/2952137793 shutdown_connections 2024-03-30T00:51:02.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 --2- 172.21.15.66:0/2952137793 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9cfba00 0x5594c9cfbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 --2- 172.21.15.66:0/2952137793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d48390 0x5594c9cfb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 --2- 172.21.15.66:0/2952137793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9d47e50 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 -- 172.21.15.66:0/2952137793 >> 172.21.15.66:0/2952137793 conn(0x5594c9afc160 msgr2=0x5594c9d2a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 -- 172.21.15.66:0/2952137793 shutdown_connections 2024-03-30T00:51:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 -- 172.21.15.66:0/2952137793 wait complete. 2024-03-30T00:51:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 -- start start 2024-03-30T00:51:02.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 Processor -- start 2024-03-30T00:51:02.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9d03ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d47a70 0x5594c9d2ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d48390 0x5594c9d2bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c9b4ca90 con 0x5594c9cfba00 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c9aebd40 con 0x5594c9d48390 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594c9b09f30 con 0x5594c9d47a70 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 10 client.0 _ll_register_callbacks cb 0x5594c9bae810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7faca9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9d03ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7faca9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9d03ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48672/0 (socket says 172.21.15.66:48672) 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7faca9e00640 1 -- 172.21.15.66:0/4197604795 learned_addr learned my addr 172.21.15.66:0/4197604795 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facaa800640 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d48390 0x5594c9d2bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7faca9400640 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d47a70 0x5594c9d2ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7faca9e00640 1 -- 172.21.15.66:0/4197604795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d47a70 msgr2=0x5594c9d2ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:02.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7faca9e00640 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d47a70 0x5594c9d2ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7faca9e00640 1 -- 172.21.15.66:0/4197604795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d48390 msgr2=0x5594c9d2bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7faca9e00640 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d48390 0x5594c9d2bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7faca9e00640 1 -- 172.21.15.66:0/4197604795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9800f040 con 0x5594c9cfba00 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7faca9e00640 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9d03ad0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fac9800e9e0 tx=0x7fac98000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6bee7e7e3f6179 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7fac87400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7fac87400640 1 -- 172.21.15.66:0/4197604795 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98027070 con 0x5594c9cfba00 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7fac87400640 1 -- 172.21.15.66:0/4197604795 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fac98005e10 con 0x5594c9cfba00 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.303+0000 7facafd3c000 1 -- 172.21.15.66:0/4197604795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594c9b4b3e0 con 0x5594c9cfba00 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7fac87400640 1 -- 172.21.15.66:0/4197604795 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac9800ed50 con 0x5594c9cfba00 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 10 client.9639 fetch_fsmap learned FSMap version 95 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 -- 172.21.15.66:0/4197604795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x5594c9b11aa0 con 0x5594c9cfba00 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7fac87400640 1 -- 172.21.15.66:0/4197604795 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5594c9b4b3e0 con 0x5594c9cfba00 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7fac87400640 1 -- 172.21.15.66:0/4197604795 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 95) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fac98021450 con 0x5594c9cfba00 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 10 client.9639 fetch_fsmap finished waiting for FSMap version 95 2024-03-30T00:51:02.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 10 client.9639 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-30T00:51:02.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 20 client.9639 populate_metadata read hostname 'smithi066' 2024-03-30T00:51:02.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x5594c9d4f090 0x5594c9d4f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 4 client.9639 mds_command: new command op to 9366 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:51:02.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 -- 172.21.15.66:0/4197604795 --> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594c9b1cd00 con 0x5594c9d4f090 2024-03-30T00:51:02.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7faca9400640 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x5594c9d4f090 0x5594c9d4f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7faca9400640 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x5594c9d4f090 0x5594c9d4f4a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1040a9d1c9d2154 server_cookie=2ac8f96c35bb0c67 in_seq=0 out_seq=0 2024-03-30T00:51:02.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7fac87400640 10 client.9639 ms_handle_connect on v2:172.21.15.66:6834/2683418851 2024-03-30T00:51:02.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7fac87400640 1 -- 172.21.15.66:0/4197604795 <== mds.0 v2:172.21.15.66:6834/2683418851 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5594c9b1cd00 con 0x5594c9d4f090 2024-03-30T00:51:02.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7fac87400640 10 client.9639 handle_command_reply: tid=0 2024-03-30T00:51:02.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 client.9639 shutdown 2024-03-30T00:51:02.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 -- 172.21.15.66:0/4197604795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 msgr2=0x5594c9d03ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9d03ad0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fac9800e9e0 tx=0x7fac98000ee0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 -- 172.21.15.66:0/4197604795 shutdown_connections 2024-03-30T00:51:02.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x5594c9d4f090 0x5594c9d4f4a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d48390 0x5594c9d2bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d47a70 0x5594c9d2ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 --2- 172.21.15.66:0/4197604795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9d03ad0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7fac87400640 0 client.9639 ms_handle_reset on v2:172.21.15.66:6834/2683418851 2024-03-30T00:51:02.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 -- 172.21.15.66:0/4197604795 >> 172.21.15.66:0/4197604795 conn(0x5594c9afc160 msgr2=0x5594c9cfabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:02.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 -- 172.21.15.66:0/4197604795 shutdown_connections 2024-03-30T00:51:02.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 1 -- 172.21.15.66:0/4197604795 wait complete. 2024-03-30T00:51:02.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.307+0000 7facafd3c000 20 client.9639 trim_cache size 0 max 16384 2024-03-30T00:51:02.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.319+0000 7facafd3c000 1 Processor -- start 2024-03-30T00:51:02.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.319+0000 7facafd3c000 1 -- start start 2024-03-30T00:51:02.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9cf9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d47a70 0x5594c9cfa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d48390 0x5594c9cfa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c9809aa0 con 0x5594c9cfba00 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c9b90170 con 0x5594c9d47a70 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594c9c1d470 con 0x5594c9d48390 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7faca9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9cf9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7faca9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9cf9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48676/0 (socket says 172.21.15.66:48676) 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7faca9e00640 1 -- 172.21.15.66:0/2682542008 learned_addr learned my addr 172.21.15.66:0/2682542008 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7faca9400640 1 --2- 172.21.15.66:0/2682542008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d47a70 0x5594c9cfa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facaa800640 1 --2- 172.21.15.66:0/2682542008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d48390 0x5594c9cfa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7faca9e00640 1 -- 172.21.15.66:0/2682542008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d48390 msgr2=0x5594c9cfa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7faca9e00640 1 --2- 172.21.15.66:0/2682542008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d48390 0x5594c9cfa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7faca9e00640 1 -- 172.21.15.66:0/2682542008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d47a70 msgr2=0x5594c9cfa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:02.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7faca9e00640 1 --2- 172.21.15.66:0/2682542008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d47a70 0x5594c9cfa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7faca9e00640 1 -- 172.21.15.66:0/2682542008 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594c9af7df0 con 0x5594c9cfba00 2024-03-30T00:51:02.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7faca9e00640 1 --2- 172.21.15.66:0/2682542008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9cf9c20 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fac98007600 tx=0x7fac98027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bf7de6dac8fa61 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:02.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7fac93400640 1 -- 172.21.15.66:0/2682542008 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98030020 con 0x5594c9cfba00 2024-03-30T00:51:02.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7fac93400640 1 -- 172.21.15.66:0/2682542008 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac98021b70 con 0x5594c9cfba00 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7fac93400640 1 -- 172.21.15.66:0/2682542008 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98001d40 con 0x5594c9cfba00 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 -- 172.21.15.66:0/2682542008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 msgr2=0x5594c9cf9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 --2- 172.21.15.66:0/2682542008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9cf9c20 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fac98007600 tx=0x7fac98027000 comp rx=0 tx=0).stop 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 -- 172.21.15.66:0/2682542008 shutdown_connections 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 --2- 172.21.15.66:0/2682542008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d48390 0x5594c9cfa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 --2- 172.21.15.66:0/2682542008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9d47a70 0x5594c9cfa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 --2- 172.21.15.66:0/2682542008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9cfba00 0x5594c9cf9c20 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 -- 172.21.15.66:0/2682542008 >> 172.21.15.66:0/2682542008 conn(0x5594c9afc160 msgr2=0x5594c9d019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 -- 172.21.15.66:0/2682542008 shutdown_connections 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 -- 172.21.15.66:0/2682542008 wait complete. 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 -- start start 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.323+0000 7facafd3c000 1 Processor -- start 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9cfba00 0x5594c9da69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9da6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d48390 0x5594c9cfe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c9d2beb0 con 0x5594c9d47a70 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c9b48280 con 0x5594c9cfba00 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594c9b54e60 con 0x5594c9d48390 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 10 client.0 _ll_register_callbacks cb 0x7fac98001470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9da6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9da6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48684/0 (socket says 172.21.15.66:48684) 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9400640 1 -- 172.21.15.66:0/4224774967 learned_addr learned my addr 172.21.15.66:0/4224774967 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facaa800640 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d48390 0x5594c9cfe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9e00640 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9cfba00 0x5594c9da69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9400640 1 -- 172.21.15.66:0/4224774967 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d48390 msgr2=0x5594c9cfe0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9400640 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d48390 0x5594c9cfe0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9400640 1 -- 172.21.15.66:0/4224774967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9cfba00 msgr2=0x5594c9da69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9400640 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9cfba00 0x5594c9da69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9400640 1 -- 172.21.15.66:0/4224774967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594c9b11aa0 con 0x5594c9d47a70 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9400640 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9da6f30 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fac9400b9f0 tx=0x7fac9401e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734a9f0f82f8a77c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7fac87400640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7fac87400640 1 -- 172.21.15.66:0/4224774967 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9401e880 con 0x5594c9d47a70 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7fac87400640 1 -- 172.21.15.66:0/4224774967 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fac9401b030 con 0x5594c9d47a70 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 1 -- 172.21.15.66:0/4224774967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594c9b09970 con 0x5594c9d47a70 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7fac87400640 1 -- 172.21.15.66:0/4224774967 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac9401e9e0 con 0x5594c9d47a70 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 10 client.9645 fetch_fsmap learned FSMap version 95 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 1 -- 172.21.15.66:0/4224774967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x5594c9aeb960 con 0x5594c9d47a70 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7fac87400640 1 -- 172.21.15.66:0/4224774967 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5594c9b09970 con 0x5594c9d47a70 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7fac87400640 1 -- 172.21.15.66:0/4224774967 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 95) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fac9401eb40 con 0x5594c9d47a70 2024-03-30T00:51:02.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 10 client.9645 fetch_fsmap finished waiting for FSMap version 95 2024-03-30T00:51:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 10 client.9645 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-30T00:51:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 20 client.9645 populate_metadata read hostname 'smithi066' 2024-03-30T00:51:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x5594c9cfe630 0x5594c9cfea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 4 client.9645 mds_command: new command op to 9366 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:51:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7facafd3c000 1 -- 172.21.15.66:0/4224774967 --> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5594c9b7ba20 con 0x5594c9cfe630 2024-03-30T00:51:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9e00640 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x5594c9cfe630 0x5594c9cfea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7faca9e00640 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x5594c9cfe630 0x5594c9cfea40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17f5e835c310db6 server_cookie=d92fc98b3bf61418 in_seq=0 out_seq=0 2024-03-30T00:51:02.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.327+0000 7fac87400640 10 client.9645 ms_handle_connect on v2:172.21.15.66:6834/2683418851 2024-03-30T00:51:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7fac87400640 1 -- 172.21.15.66:0/4224774967 <== mds.0 v2:172.21.15.66:6834/2683418851 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fac98019040 con 0x5594c9cfe630 2024-03-30T00:51:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7fac87400640 10 client.9645 handle_command_reply: tid=0 2024-03-30T00:51:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7facafd3c000 1 client.9645 shutdown 2024-03-30T00:51:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7facafd3c000 1 -- 172.21.15.66:0/4224774967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 msgr2=0x5594c9da6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7facafd3c000 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9da6f30 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fac9400b9f0 tx=0x7fac9401e660 comp rx=0 tx=0).stop 2024-03-30T00:51:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7facafd3c000 1 -- 172.21.15.66:0/4224774967 shutdown_connections 2024-03-30T00:51:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7facafd3c000 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.66:6834/2683418851,v1:172.21.15.66:6836/2683418851] conn(0x5594c9cfe630 0x5594c9cfea40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7facafd3c000 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5594c9d48390 0x5594c9cfe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7facafd3c000 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5594c9d47a70 0x5594c9da6f30 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7facafd3c000 1 --2- 172.21.15.66:0/4224774967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594c9cfba00 0x5594c9da69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7fac87400640 0 client.9645 ms_handle_reset on v2:172.21.15.66:6834/2683418851 2024-03-30T00:51:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7facafd3c000 1 -- 172.21.15.66:0/4224774967 >> 172.21.15.66:0/4224774967 conn(0x5594c9afc160 msgr2=0x5594c9d02260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7facafd3c000 1 -- 172.21.15.66:0/4224774967 shutdown_connections 2024-03-30T00:51:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.339+0000 7facafd3c000 1 -- 172.21.15.66:0/4224774967 wait complete. 2024-03-30T00:51:02.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facafd3c000 20 client.9645 trim_cache size 0 max 16384 2024-03-30T00:51:02.350 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:02.351 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "", 2024-03-30T00:51:02.351 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0 2024-03-30T00:51:02.351 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:02.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 -- 172.21.15.66:0/224001551 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fac88066690 msgr2=0x7fac88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fac88066690 0x7fac88068b50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fac98005890 tx=0x7fac98009040 comp rx=0 tx=0).stop 2024-03-30T00:51:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 -- 172.21.15.66:0/224001551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca415e0a0 msgr2=0x7faca41533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca415e0a0 0x7faca41533f0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7faca0019510 tx=0x7faca00196a0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 -- 172.21.15.66:0/224001551 shutdown_connections 2024-03-30T00:51:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fac88066690 0x7fac88068b50 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca415e0a0 0x7faca41533f0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faca4158210 0x7faca4152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 --2- 172.21.15.66:0/224001551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faca40ca9d0 0x7faca40ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 -- 172.21.15.66:0/224001551 >> 172.21.15.66:0/224001551 conn(0x7faca406fab0 msgr2=0x7faca40cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 -- 172.21.15.66:0/224001551 shutdown_connections 2024-03-30T00:51:02.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.343+0000 7facab200640 1 -- 172.21.15.66:0/224001551 wait complete. 2024-03-30T00:51:02.369 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:51:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.499+0000 7f6c9d400640 1 Processor -- start 2024-03-30T00:51:02.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.499+0000 7f6c9d400640 1 -- start start 2024-03-30T00:51:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c9d400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c980cdb60 0x7f6c9815be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c9d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c9815c380 0x7f6c980b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c9d400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c980ce0d0 con 0x7f6c980cc3f0 2024-03-30T00:51:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c9d400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c981551d0 con 0x7f6c9815c380 2024-03-30T00:51:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c9d400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c98065e30 con 0x7f6c980cdb60 2024-03-30T00:51:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c97200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c9815c380 0x7f6c980b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c97200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c9815c380 0x7f6c980b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48690/0 (socket says 172.21.15.66:48690) 2024-03-30T00:51:02.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c97200640 1 -- 172.21.15.66:0/2246259694 learned_addr learned my addr 172.21.15.66:0/2246259694 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:02.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c8de00640 1 --2- 172.21.15.66:0/2246259694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c980cdb60 0x7f6c9815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c97200640 1 -- 172.21.15.66:0/2246259694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c980cdb60 msgr2=0x7f6c9815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:02.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c97200640 1 --2- 172.21.15.66:0/2246259694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c980cdb60 0x7f6c9815be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c97200640 1 -- 172.21.15.66:0/2246259694 >> v1:172.21.15.156:6789/0 conn(0x7f6c980cc3f0 legacy=0x7f6c980ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:02.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.503+0000 7f6c97200640 1 -- 172.21.15.66:0/2246259694 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c980ce3b0 con 0x7f6c9815c380 2024-03-30T00:51:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c97200640 1 --2- 172.21.15.66:0/2246259694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c9815c380 0x7f6c980b3f70 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009870 tx=0x7f6c9001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c568efe7c31efc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c95e00640 1 -- 172.21.15.66:0/2246259694 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c900ad660 con 0x7f6c9815c380 2024-03-30T00:51:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c95e00640 1 -- 172.21.15.66:0/2246259694 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c900ad7c0 con 0x7f6c9815c380 2024-03-30T00:51:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c95e00640 1 -- 172.21.15.66:0/2246259694 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c900adb50 con 0x7f6c9815c380 2024-03-30T00:51:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c9d400640 1 -- 172.21.15.66:0/2246259694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c9815c380 msgr2=0x7f6c980b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c9d400640 1 --2- 172.21.15.66:0/2246259694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c9815c380 0x7f6c980b3f70 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009870 tx=0x7f6c9001c560 comp rx=0 tx=0).stop 2024-03-30T00:51:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c9d400640 1 -- 172.21.15.66:0/2246259694 shutdown_connections 2024-03-30T00:51:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c9d400640 1 --2- 172.21.15.66:0/2246259694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c9815c380 0x7f6c980b3f70 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c9d400640 1 --2- 172.21.15.66:0/2246259694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c980cdb60 0x7f6c9815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c9d400640 1 -- 172.21.15.66:0/2246259694 >> 172.21.15.66:0/2246259694 conn(0x7f6c98071750 msgr2=0x7f6c98069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c9d400640 1 -- 172.21.15.66:0/2246259694 shutdown_connections 2024-03-30T00:51:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c9d400640 1 -- 172.21.15.66:0/2246259694 wait complete. 2024-03-30T00:51:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c9d400640 1 Processor -- start 2024-03-30T00:51:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.507+0000 7f6c9d400640 1 -- start start 2024-03-30T00:51:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c9d400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c980cc3f0 0x7f6c9815e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c9d400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c980cdb60 0x7f6c981401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c9d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c9815c380 0x7f6c98155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c9d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c98141a50 con 0x7f6c9815c380 2024-03-30T00:51:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c9d400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9815e740 con 0x7f6c980cc3f0 2024-03-30T00:51:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c9d400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c98140e40 con 0x7f6c980cdb60 2024-03-30T00:51:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c97200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c9815c380 0x7f6c98155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c97200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c9815c380 0x7f6c98155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48706/0 (socket says 172.21.15.66:48706) 2024-03-30T00:51:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c97200640 1 -- 172.21.15.66:0/3187954332 learned_addr learned my addr 172.21.15.66:0/3187954332 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c8de00640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c980cdb60 0x7f6c981401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c96800640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c980cc3f0 0x7f6c9815e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c97200640 1 -- 172.21.15.66:0/3187954332 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c980cdb60 msgr2=0x7f6c981401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c97200640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c980cdb60 0x7f6c981401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c97200640 1 -- 172.21.15.66:0/3187954332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c980cc3f0 msgr2=0x7f6c9815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c97200640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c980cc3f0 0x7f6c9815e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c97200640 1 -- 172.21.15.66:0/3187954332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c900094d0 con 0x7f6c9815c380 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c96800640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c980cc3f0 0x7f6c9815e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c97200640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c9815c380 0x7f6c98155310 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f6c900b0040 tx=0x7f6c90019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c8f400640 1 -- 172.21.15.66:0/3187954332 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c900b6640 con 0x7f6c9815c380 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c8f400640 1 -- 172.21.15.66:0/3187954332 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c900b67a0 con 0x7f6c9815c380 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c8f400640 1 -- 172.21.15.66:0/3187954332 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c900b6b10 con 0x7f6c9815c380 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c9d400640 1 -- 172.21.15.66:0/3187954332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c98141120 con 0x7f6c9815c380 2024-03-30T00:51:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c9d400640 1 -- 172.21.15.66:0/3187954332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c981416e0 con 0x7f6c9815c380 2024-03-30T00:51:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c9d400640 1 -- 172.21.15.66:0/3187954332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c98141bc0 con 0x7f6c9815c380 2024-03-30T00:51:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.511+0000 7f6c8f400640 1 -- 172.21.15.66:0/3187954332 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6c900b4070 con 0x7f6c9815c380 2024-03-30T00:51:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.515+0000 7f6c8f400640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c74066690 0x7f6c74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.515+0000 7f6c8f400640 1 -- 172.21.15.66:0/3187954332 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6c9011e480 con 0x7f6c9815c380 2024-03-30T00:51:02.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.515+0000 7f6c96800640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c74066690 0x7f6c74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:02.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.515+0000 7f6c96800640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c74066690 0x7f6c74068b50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6c980b2e50 tx=0x7f6c88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:02.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.519+0000 7f6c8f400640 1 -- 172.21.15.66:0/3187954332 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6c900eb610 con 0x7f6c9815c380 2024-03-30T00:51:02.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.711+0000 7f6c9d400640 1 -- 172.21.15.66:0/3187954332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6c98066c30 con 0x7f6c9815c380 2024-03-30T00:51:02.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.711+0000 7f6c8f400640 1 -- 172.21.15.66:0/3187954332 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v95) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6c900eec30 con 0x7f6c9815c380 2024-03-30T00:51:02.721 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:51:02.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 -- 172.21.15.66:0/3187954332 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c74066690 msgr2=0x7f6c74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c74066690 0x7f6c74068b50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6c980b2e50 tx=0x7f6c88009040 comp rx=0 tx=0).stop 2024-03-30T00:51:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 -- 172.21.15.66:0/3187954332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c9815c380 msgr2=0x7f6c98155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c9815c380 0x7f6c98155310 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f6c900b0040 tx=0x7f6c90019300 comp rx=0 tx=0).stop 2024-03-30T00:51:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 -- 172.21.15.66:0/3187954332 shutdown_connections 2024-03-30T00:51:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c74066690 0x7f6c74068b50 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c9815c380 0x7f6c98155310 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c980cdb60 0x7f6c981401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 --2- 172.21.15.66:0/3187954332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c980cc3f0 0x7f6c9815e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 -- 172.21.15.66:0/3187954332 >> 172.21.15.66:0/3187954332 conn(0x7f6c98071750 msgr2=0x7f6c980ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 -- 172.21.15.66:0/3187954332 shutdown_connections 2024-03-30T00:51:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:02.715+0000 7f6c9d400640 1 -- 172.21.15.66:0/3187954332 wait complete. 2024-03-30T00:51:02.743 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:51:02.743 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:51:02.743 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:51:02.743 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:51:02.743 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:02.743 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:51:02.743 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:51:02.743 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:51:02.743 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:51:02.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:02.744 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:02.753 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41815sec preferred_lft 41815sec 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:02.754 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:51:02.755 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:02.755 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:51:02.755 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:51:02.755 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:51:02.755 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:02.755 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:51:02.755 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:51:02.755 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:51:02.755 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:02.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:02.756 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:51:02.815 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:51:02.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:02.817 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:51:02.873 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:51:02.873 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:51:02.874 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:51:02.874 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:51:02.874 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:51:02.874 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:51:02.874 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:51:02.874 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:51:02.874 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:51:02.874 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:51:02.874 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:51:02.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:02.875 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:51:02.875 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:51:02.875 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:51:02.875 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:51:02.875 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:51:02.875 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:51:02.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:02.968 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:51:02.977 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:51:02.977 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:51:02.978 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:51:02.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:02.978 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:51:03.041 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:51:03.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:51:03.041 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:03.098 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:51:03.098 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:51:03.098 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:03.149 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:51:03.209 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:51:03.210 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:51:03.271 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:51:03.273 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:51:03.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:03.274 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:51:03.329 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:51:03.329 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:51:03.376 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:51:03.396 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:51:03.396 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:51:03.455 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:51:03.457 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:51:03.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:03.458 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:51:03.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:51:03.463+0000 7fe5dbd59580 -1 init, newargv = 0x55f9a9486640 newargc=15 2024-03-30T00:51:03.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[54282]: starting ceph client 2024-03-30T00:51:03.478 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[54282]: starting fuse 2024-03-30T00:51:03.480 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:51:03.480 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:51:03.480 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:03.480 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:51:03.480 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:51:03.480 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:03.480 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:51:03.480 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:03.480 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:51:03.481 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:51:03.482 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:51:03.482 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:03.482 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:51:03.482 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:03.605 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.54282.asok 2024-03-30T00:51:03.790 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54282.asok 2024-03-30T00:51:03.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:03.790 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54282.asok status 2024-03-30T00:51:03.968 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:03.968 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:51:03.968 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:51:03.968 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:51:03.968 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:51:03.968 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "54282", 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 9657, 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 9657 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:51:03.969 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 423018045 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 423018045 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.9657 192.168.0.1:0/423018045", 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/423018045", 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 95, 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 125, 2024-03-30T00:51:03.970 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:51:03.971 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:51:03.971 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:51:03.971 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:03.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:03.983 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:51:03.983 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:03.993 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:51:03.993 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:03.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:03.993 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:04.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:04.056 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-03-30T00:51:04.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:04.116 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-03-30T00:51:04.230 INFO:teuthology.orchestra.run.smithi066.stderr:8+0 records in 2024-03-30T00:51:04.231 INFO:teuthology.orchestra.run.smithi066.stderr:8+0 records out 2024-03-30T00:51:04.231 INFO:teuthology.orchestra.run.smithi066.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.063271 s, 133 MB/s 2024-03-30T00:51:04.232 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:04.232 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:04.232 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:04.232 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:04.232 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-30T00:51:04.232 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:04.300 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627779 2024-03-30T00:51:04.489 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:04.489 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:04.489 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:04.489 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:04.490 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-30T00:51:04.490 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:04.598 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627780 2024-03-30T00:51:04.787 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:51:04.787 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:04.842 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:51:05.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.015+0000 7f3421000640 1 Processor -- start 2024-03-30T00:51:05.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.015+0000 7f3421000640 1 -- start start 2024-03-30T00:51:05.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3421000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f341c0ce520 0x7f341c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3421000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c0cabb0 0x7f341c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3421000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c06d070 con 0x7f341c15a640 2024-03-30T00:51:05.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3421000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c0cbba0 con 0x7f341c0ce520 2024-03-30T00:51:05.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3421000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0cee70 con 0x7f341c0cabb0 2024-03-30T00:51:05.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f341c15a640 0x7f341c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36424/0 (socket says 172.21.15.66:36424) 2024-03-30T00:51:05.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419e00640 1 -- 172.21.15.66:0/316211490 learned_addr learned my addr 172.21.15.66:0/316211490 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:05.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3411400640 1 --2- 172.21.15.66:0/316211490 >> v2:172.21.15.156:3300/0 conn(0x7f341c0ce520 0x7f341c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f341a800640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c0cabb0 0x7f341c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553343890 0 0) 0x7f341c06d070 con 0x7f341c15a640 2024-03-30T00:51:05.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3400003530 con 0x7f341c15a640 2024-03-30T00:51:05.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1563973397 0 0) 0x7f3400003530 con 0x7f341c15a640 2024-03-30T00:51:05.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c0cabb0 msgr2=0x7f341c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c0cabb0 0x7f341c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 >> v2:172.21.15.156:3300/0 conn(0x7f341c0ce520 msgr2=0x7f341c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 --2- 172.21.15.66:0/316211490 >> v2:172.21.15.156:3300/0 conn(0x7f341c0ce520 0x7f341c0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c0cb8c0 con 0x7f341c15a640 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f340c001680 con 0x7f341c15a640 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 >> v1:172.21.15.66:6789/0 conn(0x7f341c15a640 legacy=0x7f341c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 0x7f3400003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34000047d0 0x7f3400004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c0cabb0 0x7f3400005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0cbba0 con 0x7f3400003980 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0cee70 con 0x7f34000047d0 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340000b990 con 0x7f341c0cabb0 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419e00640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 0x7f3400003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3411400640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34000047d0 0x7f3400004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f341a800640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c0cabb0 0x7f3400005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419e00640 1 -- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c0cabb0 msgr2=0x7f3400005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:05.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419e00640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c0cabb0 0x7f3400005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419e00640 1 -- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34000047d0 msgr2=0x7f3400004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:05.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419e00640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34000047d0 0x7f3400004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3419e00640 1 -- 172.21.15.66:0/316211490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3400003530 con 0x7f3400003980 2024-03-30T00:51:05.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f341a800640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c0cabb0 0x7f3400005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:05.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.019+0000 7f3411400640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34000047d0 0x7f3400004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:05.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3419e00640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 0x7f3400003da0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c210 tx=0x7f340c01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aca09afe15c9c7a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:05.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c0058e0 con 0x7f3400003980 2024-03-30T00:51:05.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c005a40 con 0x7f3400003980 2024-03-30T00:51:05.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3419400640 1 -- 172.21.15.66:0/316211490 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c005dd0 con 0x7f3400003980 2024-03-30T00:51:05.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3421000640 1 -- 172.21.15.66:0/316211490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 msgr2=0x7f3400003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3421000640 1 --2- 172.21.15.66:0/316211490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 0x7f3400003da0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c210 tx=0x7f340c01cc00 comp rx=0 tx=0).stop 2024-03-30T00:51:05.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f341a800640 1 -- 172.21.15.66:0/316211490 reap_dead start 2024-03-30T00:51:05.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3421000640 1 -- 172.21.15.66:0/316211490 shutdown_connections 2024-03-30T00:51:05.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3421000640 1 -- 172.21.15.66:0/316211490 >> 172.21.15.66:0/316211490 conn(0x7f341c069f60 msgr2=0x7f341c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:05.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3421000640 1 -- 172.21.15.66:0/316211490 shutdown_connections 2024-03-30T00:51:05.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3421000640 1 -- 172.21.15.66:0/316211490 wait complete. 2024-03-30T00:51:05.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3421000640 1 Processor -- start 2024-03-30T00:51:05.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.023+0000 7f3421000640 1 -- start start 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3421000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 0x7f341c0c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3421000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f341c0ca2a0 0x7f341c1f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3421000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c1f49a0 0x7f341c162410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3421000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c1f4e20 con 0x7f3400003980 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3421000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c15db20 con 0x7f341c0ca2a0 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3421000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0ca830 con 0x7f341c1f49a0 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3419e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 0x7f341c0c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3419e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 0x7f341c0c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48728/0 (socket says 172.21.15.66:48728) 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3419e00640 1 -- 172.21.15.66:0/2654107502 learned_addr learned my addr 172.21.15.66:0/2654107502 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3411400640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f341c0ca2a0 0x7f341c1f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f341a800640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c1f49a0 0x7f341c162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3419e00640 1 -- 172.21.15.66:0/2654107502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c1f49a0 msgr2=0x7f341c162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3419e00640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c1f49a0 0x7f341c162410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3419e00640 1 -- 172.21.15.66:0/2654107502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f341c0ca2a0 msgr2=0x7f341c1f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3419e00640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f341c0ca2a0 0x7f341c1f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3419e00640 1 -- 172.21.15.66:0/2654107502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c1f5020 con 0x7f3400003980 2024-03-30T00:51:05.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3411400640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f341c0ca2a0 0x7f341c1f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:05.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3419e00640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 0x7f341c0c87b0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f340c007040 tx=0x7f3400005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:05.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3412a00640 1 -- 172.21.15.66:0/2654107502 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c0a5860 con 0x7f3400003980 2024-03-30T00:51:05.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3412a00640 1 -- 172.21.15.66:0/2654107502 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c0a59c0 con 0x7f3400003980 2024-03-30T00:51:05.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3412a00640 1 -- 172.21.15.66:0/2654107502 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c0a5d30 con 0x7f3400003980 2024-03-30T00:51:05.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3421000640 1 -- 172.21.15.66:0/2654107502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c1f5300 con 0x7f3400003980 2024-03-30T00:51:05.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3421000640 1 -- 172.21.15.66:0/2654107502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c0c8cf0 con 0x7f3400003980 2024-03-30T00:51:05.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.027+0000 7f3421000640 1 -- 172.21.15.66:0/2654107502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c066c30 con 0x7f3400003980 2024-03-30T00:51:05.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.031+0000 7f3412a00640 1 -- 172.21.15.66:0/2654107502 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f340c0047f0 con 0x7f3400003980 2024-03-30T00:51:05.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.031+0000 7f3412a00640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f33f80666c0 0x7f33f8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.031+0000 7f3412a00640 1 -- 172.21.15.66:0/2654107502 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f340c11d8a0 con 0x7f3400003980 2024-03-30T00:51:05.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.031+0000 7f3411400640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f33f80666c0 0x7f33f8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.031+0000 7f3411400640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f33f80666c0 0x7f33f8068b80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f341c0c3dd0 tx=0x7f3404008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:05.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.035+0000 7f3412a00640 1 -- 172.21.15.66:0/2654107502 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f340c0eaa30 con 0x7f3400003980 2024-03-30T00:51:05.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.195+0000 7f3421000640 1 -- 172.21.15.66:0/2654107502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f341c0abd20 con 0x7f3400003980 2024-03-30T00:51:05.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.195+0000 7f3412a00640 1 -- 172.21.15.66:0/2654107502 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16316 (secure 0 0 0) 0x7f340c0ee050 con 0x7f3400003980 2024-03-30T00:51:05.205 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:05.206 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":125,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:39.043056+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:51:05.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 -- 172.21.15.66:0/2654107502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f33f80666c0 msgr2=0x7f33f8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f33f80666c0 0x7f33f8068b80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f341c0c3dd0 tx=0x7f3404008040 comp rx=0 tx=0).stop 2024-03-30T00:51:05.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 -- 172.21.15.66:0/2654107502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 msgr2=0x7f341c0c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 0x7f341c0c87b0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f340c007040 tx=0x7f3400005800 comp rx=0 tx=0).stop 2024-03-30T00:51:05.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 -- 172.21.15.66:0/2654107502 shutdown_connections 2024-03-30T00:51:05.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f341c1f49a0 0x7f341c162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f341c0ca2a0 0x7f341c1f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f33f80666c0 0x7f33f8068b80 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 --2- 172.21.15.66:0/2654107502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3400003980 0x7f341c0c87b0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 -- 172.21.15.66:0/2654107502 >> 172.21.15.66:0/2654107502 conn(0x7f341c069f60 msgr2=0x7f341c0c3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 -- 172.21.15.66:0/2654107502 shutdown_connections 2024-03-30T00:51:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.199+0000 7f3421000640 1 -- 172.21.15.66:0/2654107502 wait complete. 2024-03-30T00:51:05.227 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:50:35.696965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:50:37.037165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:51:05.227 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:51:05.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a39400640 1 Processor -- start 2024-03-30T00:51:05.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a39400640 1 -- start start 2024-03-30T00:51:05.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a39400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a340cda90 0x7f9a340ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a39400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9a340cde70 0x7f9a340cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a39400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3415a990 con 0x7f9a34159c60 2024-03-30T00:51:05.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a39400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a340b1b80 con 0x7f9a340cda90 2024-03-30T00:51:05.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a39400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a340cd3e0 con 0x7f9a340cde70 2024-03-30T00:51:05.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a32800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a340cda90 0x7f9a340ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a32800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a340cda90 0x7f9a340ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48742/0 (socket says 172.21.15.66:48742) 2024-03-30T00:51:05.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a32800640 1 -- 172.21.15.66:0/2470319190 learned_addr learned my addr 172.21.15.66:0/2470319190 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:05.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a31e00640 1 --2- 172.21.15.66:0/2470319190 >> v2:172.21.15.156:3300/0 conn(0x7f9a340cde70 0x7f9a340cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a32800640 1 -- 172.21.15.66:0/2470319190 >> v2:172.21.15.156:3300/0 conn(0x7f9a340cde70 msgr2=0x7f9a340cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:05.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a32800640 1 --2- 172.21.15.66:0/2470319190 >> v2:172.21.15.156:3300/0 conn(0x7f9a340cde70 0x7f9a340cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a32800640 1 -- 172.21.15.66:0/2470319190 >> v1:172.21.15.156:6789/0 conn(0x7f9a34159c60 legacy=0x7f9a3415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.363+0000 7f9a32800640 1 -- 172.21.15.66:0/2470319190 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340b11e0 con 0x7f9a340cda90 2024-03-30T00:51:05.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a32800640 1 --2- 172.21.15.66:0/2470319190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a340cda90 0x7f9a340ce520 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f9a340cc320 tx=0x7f9a2401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4d7f978cb2cc8b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:05.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a31400640 1 -- 172.21.15.66:0/2470319190 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a240ac600 con 0x7f9a340cda90 2024-03-30T00:51:05.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a31400640 1 -- 172.21.15.66:0/2470319190 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a240a8050 con 0x7f9a340cda90 2024-03-30T00:51:05.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a31400640 1 -- 172.21.15.66:0/2470319190 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a240ac980 con 0x7f9a340cda90 2024-03-30T00:51:05.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a39400640 1 -- 172.21.15.66:0/2470319190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a340cda90 msgr2=0x7f9a340ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a39400640 1 --2- 172.21.15.66:0/2470319190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a340cda90 0x7f9a340ce520 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f9a340cc320 tx=0x7f9a2401c490 comp rx=0 tx=0).stop 2024-03-30T00:51:05.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a39400640 1 -- 172.21.15.66:0/2470319190 shutdown_connections 2024-03-30T00:51:05.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a39400640 1 --2- 172.21.15.66:0/2470319190 >> v2:172.21.15.156:3300/0 conn(0x7f9a340cde70 0x7f9a340cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a39400640 1 --2- 172.21.15.66:0/2470319190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a340cda90 0x7f9a340ce520 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a39400640 1 -- 172.21.15.66:0/2470319190 >> 172.21.15.66:0/2470319190 conn(0x7f9a34069f60 msgr2=0x7f9a3406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:05.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a39400640 1 -- 172.21.15.66:0/2470319190 shutdown_connections 2024-03-30T00:51:05.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a39400640 1 -- 172.21.15.66:0/2470319190 wait complete. 2024-03-30T00:51:05.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a39400640 1 Processor -- start 2024-03-30T00:51:05.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.367+0000 7f9a39400640 1 -- start start 2024-03-30T00:51:05.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a39400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a340cda90 0x7f9a34155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a39400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a340cde70 0x7f9a340c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a39400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a34159c60 0x7f9a340c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a39400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340c5460 con 0x7f9a34159c60 2024-03-30T00:51:05.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a39400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340c7e00 con 0x7f9a340cde70 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a39400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340c5700 con 0x7f9a340cda90 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a33200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a34159c60 0x7f9a340c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a33200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a34159c60 0x7f9a340c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48746/0 (socket says 172.21.15.66:48746) 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a33200640 1 -- 172.21.15.66:0/3831681748 learned_addr learned my addr 172.21.15.66:0/3831681748 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a32800640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a340cda90 0x7f9a34155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a31e00640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a340cde70 0x7f9a340c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a33200640 1 -- 172.21.15.66:0/3831681748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a340cda90 msgr2=0x7f9a34155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a33200640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a340cda90 0x7f9a34155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a33200640 1 -- 172.21.15.66:0/3831681748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a340cde70 msgr2=0x7f9a340c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a33200640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a340cde70 0x7f9a340c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a33200640 1 -- 172.21.15.66:0/3831681748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a240094d0 con 0x7f9a34159c60 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a33200640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a34159c60 0x7f9a340c7570 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c770 tx=0x7f9a2c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a2aa00640 1 -- 172.21.15.66:0/3831681748 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c003b80 con 0x7f9a34159c60 2024-03-30T00:51:05.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a2aa00640 1 -- 172.21.15.66:0/3831681748 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c003ce0 con 0x7f9a34159c60 2024-03-30T00:51:05.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a2aa00640 1 -- 172.21.15.66:0/3831681748 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00aa90 con 0x7f9a34159c60 2024-03-30T00:51:05.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a39400640 1 -- 172.21.15.66:0/3831681748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a340c80e0 con 0x7f9a34159c60 2024-03-30T00:51:05.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a39400640 1 -- 172.21.15.66:0/3831681748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a34156210 con 0x7f9a34159c60 2024-03-30T00:51:05.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a39400640 1 -- 172.21.15.66:0/3831681748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a34066c30 con 0x7f9a34159c60 2024-03-30T00:51:05.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.371+0000 7f9a2aa00640 1 -- 172.21.15.66:0/3831681748 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9a2c00abf0 con 0x7f9a34159c60 2024-03-30T00:51:05.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.375+0000 7f9a2aa00640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a14066690 0x7f9a14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.375+0000 7f9a2aa00640 1 -- 172.21.15.66:0/3831681748 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9a2c0841e0 con 0x7f9a34159c60 2024-03-30T00:51:05.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.375+0000 7f9a32800640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a14066690 0x7f9a14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.375+0000 7f9a32800640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a14066690 0x7f9a14068b50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9a240ae000 tx=0x7f9a24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:05.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.379+0000 7f9a2aa00640 1 -- 172.21.15.66:0/3831681748 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9a2c051370 con 0x7f9a34159c60 2024-03-30T00:51:05.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 -- 172.21.15.66:0/3831681748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a340b3170 con 0x7f9a34159c60 2024-03-30T00:51:05.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a2aa00640 1 -- 172.21.15.66:0/3831681748 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a2c054990 con 0x7f9a34159c60 2024-03-30T00:51:05.548 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:51:05.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 -- 172.21.15.66:0/3831681748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a14066690 msgr2=0x7f9a14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a14066690 0x7f9a14068b50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9a240ae000 tx=0x7f9a24008040 comp rx=0 tx=0).stop 2024-03-30T00:51:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 -- 172.21.15.66:0/3831681748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a34159c60 msgr2=0x7f9a340c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a34159c60 0x7f9a340c7570 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c770 tx=0x7f9a2c00cc30 comp rx=0 tx=0).stop 2024-03-30T00:51:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 -- 172.21.15.66:0/3831681748 shutdown_connections 2024-03-30T00:51:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9a14066690 0x7f9a14068b50 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a34159c60 0x7f9a340c7570 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a340cde70 0x7f9a340c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 --2- 172.21.15.66:0/3831681748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a340cda90 0x7f9a34155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 -- 172.21.15.66:0/3831681748 >> 172.21.15.66:0/3831681748 conn(0x7f9a34069f60 msgr2=0x7f9a3415acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 -- 172.21.15.66:0/3831681748 shutdown_connections 2024-03-30T00:51:05.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.539+0000 7f9a39400640 1 -- 172.21.15.66:0/3831681748 wait complete. 2024-03-30T00:51:05.569 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:51:05.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.707+0000 7fafd2400640 1 Processor -- start 2024-03-30T00:51:05.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.707+0000 7fafd2400640 1 -- start start 2024-03-30T00:51:05.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafd2400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fafcc098c80 0x7fafcc099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafd2400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafcc0cf320 0x7fafcc0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafd2400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc0cbd90 con 0x7fafcc099ca0 2024-03-30T00:51:05.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafd2400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafcc0cc070 con 0x7fafcc098c80 2024-03-30T00:51:05.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafd2400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0cc350 con 0x7fafcc0cf320 2024-03-30T00:51:05.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafcb400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fafcc098c80 0x7fafcc099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafc2a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafcc0cf320 0x7fafcc0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafcb400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fafcc098c80 0x7fafcc099760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:41818/0 (socket says 172.21.15.66:41818) 2024-03-30T00:51:05.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafcb400640 1 -- 172.21.15.66:0/1752953926 learned_addr learned my addr 172.21.15.66:0/1752953926 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:05.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafcb400640 1 -- 172.21.15.66:0/1752953926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafcc0cf320 msgr2=0x7fafcc0ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:05.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafcb400640 1 --2- 172.21.15.66:0/1752953926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafcc0cf320 0x7fafcc0ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafcb400640 1 -- 172.21.15.66:0/1752953926 >> v1:172.21.15.156:6789/0 conn(0x7fafcc099ca0 legacy=0x7fafcc0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafcb400640 1 -- 172.21.15.66:0/1752953926 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc0b3530 con 0x7fafcc098c80 2024-03-30T00:51:05.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.711+0000 7fafc2a00640 1 --2- 172.21.15.66:0/1752953926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafcc0cf320 0x7fafcc0ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:51:05.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafcb400640 1 --2- 172.21.15.66:0/1752953926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafcc098c80 0x7fafcc099760 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009e50 tx=0x7fafbc01f210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68c96b575c283fe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:05.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafcaa00640 1 -- 172.21.15.66:0/1752953926 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0af400 con 0x7fafcc098c80 2024-03-30T00:51:05.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafcaa00640 1 -- 172.21.15.66:0/1752953926 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc0ab050 con 0x7fafcc098c80 2024-03-30T00:51:05.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafcaa00640 1 -- 172.21.15.66:0/1752953926 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0af780 con 0x7fafcc098c80 2024-03-30T00:51:05.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafd2400640 1 -- 172.21.15.66:0/1752953926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafcc098c80 msgr2=0x7fafcc099760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafd2400640 1 --2- 172.21.15.66:0/1752953926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafcc098c80 0x7fafcc099760 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009e50 tx=0x7fafbc01f210 comp rx=0 tx=0).stop 2024-03-30T00:51:05.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafd2400640 1 -- 172.21.15.66:0/1752953926 shutdown_connections 2024-03-30T00:51:05.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafd2400640 1 --2- 172.21.15.66:0/1752953926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafcc0cf320 0x7fafcc0ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafd2400640 1 --2- 172.21.15.66:0/1752953926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafcc098c80 0x7fafcc099760 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafd2400640 1 -- 172.21.15.66:0/1752953926 >> 172.21.15.66:0/1752953926 conn(0x7fafcc0b2130 msgr2=0x7fafcc0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:05.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafd2400640 1 -- 172.21.15.66:0/1752953926 shutdown_connections 2024-03-30T00:51:05.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafd2400640 1 -- 172.21.15.66:0/1752953926 wait complete. 2024-03-30T00:51:05.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafd2400640 1 Processor -- start 2024-03-30T00:51:05.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.715+0000 7fafd2400640 1 -- start start 2024-03-30T00:51:05.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafd2400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafcc098c80 0x7fafcc0c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafd2400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc099ca0 0x7fafcc0c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafd2400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafcc0cf320 0x7fafcc159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafd2400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0c6760 con 0x7fafcc099ca0 2024-03-30T00:51:05.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafd2400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0c5960 con 0x7fafcc098c80 2024-03-30T00:51:05.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafd2400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0c9750 con 0x7fafcc0cf320 2024-03-30T00:51:05.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc099ca0 0x7fafcc0c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc099ca0 0x7fafcc0c5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48750/0 (socket says 172.21.15.66:48750) 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc2a00640 1 -- 172.21.15.66:0/3204529253 learned_addr learned my addr 172.21.15.66:0/3204529253 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafcb400640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafcc098c80 0x7fafcc0c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafcbe00640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafcc0cf320 0x7fafcc159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc2a00640 1 -- 172.21.15.66:0/3204529253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafcc0cf320 msgr2=0x7fafcc159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc2a00640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafcc0cf320 0x7fafcc159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc2a00640 1 -- 172.21.15.66:0/3204529253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafcc098c80 msgr2=0x7fafcc0c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc2a00640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafcc098c80 0x7fafcc0c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc2a00640 1 -- 172.21.15.66:0/3204529253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb4009420 con 0x7fafcc099ca0 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc2a00640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc099ca0 0x7fafcc0c5420 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9b0 tx=0x7fafb400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc3e00640 1 -- 172.21.15.66:0/3204529253 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb400c9a0 con 0x7fafcc099ca0 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc3e00640 1 -- 172.21.15.66:0/3204529253 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb400cb00 con 0x7fafcc099ca0 2024-03-30T00:51:05.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafd2400640 1 -- 172.21.15.66:0/3204529253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc009b30 con 0x7fafcc099ca0 2024-03-30T00:51:05.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc3e00640 1 -- 172.21.15.66:0/3204529253 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb400ce90 con 0x7fafcc099ca0 2024-03-30T00:51:05.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafd2400640 1 -- 172.21.15.66:0/3204529253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc0c8e20 con 0x7fafcc099ca0 2024-03-30T00:51:05.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafd2400640 1 -- 172.21.15.66:0/3204529253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafcc0677d0 con 0x7fafcc099ca0 2024-03-30T00:51:05.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.719+0000 7fafc3e00640 1 -- 172.21.15.66:0/3204529253 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fafb4013470 con 0x7fafcc099ca0 2024-03-30T00:51:05.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.723+0000 7fafc3e00640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafa8066690 0x7fafa8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:05.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.723+0000 7fafc3e00640 1 -- 172.21.15.66:0/3204529253 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fafb4084450 con 0x7fafcc099ca0 2024-03-30T00:51:05.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.723+0000 7fafcb400640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafa8066690 0x7fafa8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:05.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.723+0000 7fafcb400640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafa8066690 0x7fafa8068b50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0b1000 tx=0x7fafbc0aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:05.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.727+0000 7fafc3e00640 1 -- 172.21.15.66:0/3204529253 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fafb4051730 con 0x7fafcc099ca0 2024-03-30T00:51:05.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 -- 172.21.15.66:0/3204529253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fafcc066c30 con 0x7fafcc099ca0 2024-03-30T00:51:05.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafc3e00640 1 -- 172.21.15.66:0/3204529253 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7fafb4054d50 con 0x7fafcc099ca0 2024-03-30T00:51:05.899 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:51:05.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 -- 172.21.15.66:0/3204529253 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafa8066690 msgr2=0x7fafa8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafa8066690 0x7fafa8068b50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0b1000 tx=0x7fafbc0aa000 comp rx=0 tx=0).stop 2024-03-30T00:51:05.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 -- 172.21.15.66:0/3204529253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc099ca0 msgr2=0x7fafcc0c5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:05.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc099ca0 0x7fafcc0c5420 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9b0 tx=0x7fafb400ee70 comp rx=0 tx=0).stop 2024-03-30T00:51:05.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 -- 172.21.15.66:0/3204529253 shutdown_connections 2024-03-30T00:51:05.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafcc0cf320 0x7fafcc159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fafa8066690 0x7fafa8068b50 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc099ca0 0x7fafcc0c5420 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 --2- 172.21.15.66:0/3204529253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafcc098c80 0x7fafcc0c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:05.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 -- 172.21.15.66:0/3204529253 >> 172.21.15.66:0/3204529253 conn(0x7fafcc0b2130 msgr2=0x7fafcc0cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:05.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 -- 172.21.15.66:0/3204529253 shutdown_connections 2024-03-30T00:51:05.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:05.891+0000 7fafd2400640 1 -- 172.21.15.66:0/3204529253 wait complete. 2024-03-30T00:51:05.919 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:51:06.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907ec00640 1 Processor -- start 2024-03-30T00:51:06.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907ec00640 1 -- start start 2024-03-30T00:51:06.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907ec00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907815ca80 0x7f907806b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907ec00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f90780cb570 0x7f90780cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907ec00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90780cb0e0 con 0x7f90780cf2f0 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907ec00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90780cf7f0 con 0x7f90780cb570 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907ec00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90780cfad0 con 0x7f907815ca80 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907cc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f90780cb570 0x7f90780cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f906f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f90780cf2f0 0x7f907806b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36428/0 (socket says 172.21.15.66:36428) 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907cc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f90780cb570 0x7f90780cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48752/0 (socket says 172.21.15.66:48752) 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f906f400640 1 -- 172.21.15.66:0/3344838024 learned_addr learned my addr 172.21.15.66:0/3344838024 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f9077e00640 1 --2- 172.21.15.66:0/3344838024 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907815ca80 0x7f907806b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907cc00640 1 -- 172.21.15.66:0/3344838024 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907815ca80 msgr2=0x7f907806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907cc00640 1 --2- 172.21.15.66:0/3344838024 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907815ca80 0x7f907806b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907cc00640 1 -- 172.21.15.66:0/3344838024 >> v1:172.21.15.66:6789/0 conn(0x7f90780cf2f0 legacy=0x7f907806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f907cc00640 1 -- 172.21.15.66:0/3344838024 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90780cad70 con 0x7f90780cb570 2024-03-30T00:51:06.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.055+0000 7f9077e00640 1 --2- 172.21.15.66:0/3344838024 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907815ca80 0x7f907806b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:06.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f907cc00640 1 --2- 172.21.15.66:0/3344838024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90780cb570 0x7f90780cba20 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f9070009d90 tx=0x7f907001f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471aa9773271c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:06.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f9077400640 1 -- 172.21.15.66:0/3344838024 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90700af600 con 0x7f90780cb570 2024-03-30T00:51:06.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f9077400640 1 -- 172.21.15.66:0/3344838024 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90700ab050 con 0x7f90780cb570 2024-03-30T00:51:06.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f9077400640 1 -- 172.21.15.66:0/3344838024 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90700af980 con 0x7f90780cb570 2024-03-30T00:51:06.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f907ec00640 1 -- 172.21.15.66:0/3344838024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90780cb570 msgr2=0x7f90780cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f907ec00640 1 --2- 172.21.15.66:0/3344838024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90780cb570 0x7f90780cba20 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f9070009d90 tx=0x7f907001f150 comp rx=0 tx=0).stop 2024-03-30T00:51:06.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f907ec00640 1 -- 172.21.15.66:0/3344838024 shutdown_connections 2024-03-30T00:51:06.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f907ec00640 1 --2- 172.21.15.66:0/3344838024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90780cb570 0x7f90780cba20 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f907ec00640 1 --2- 172.21.15.66:0/3344838024 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907815ca80 0x7f907806b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f907ec00640 1 -- 172.21.15.66:0/3344838024 >> 172.21.15.66:0/3344838024 conn(0x7f90780b2090 msgr2=0x7f90780b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:06.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f907ec00640 1 -- 172.21.15.66:0/3344838024 shutdown_connections 2024-03-30T00:51:06.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.059+0000 7f907ec00640 1 -- 172.21.15.66:0/3344838024 wait complete. 2024-03-30T00:51:06.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907ec00640 1 Processor -- start 2024-03-30T00:51:06.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907ec00640 1 -- start start 2024-03-30T00:51:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907ec00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90780cb570 0x7f90781f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90780cf2f0 0x7f907809a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907ec00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907815ca80 0x7f907815b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907809ac10 con 0x7f90780cf2f0 2024-03-30T00:51:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907ec00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907809aef0 con 0x7f90780cb570 2024-03-30T00:51:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907ec00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907809b1d0 con 0x7f907815ca80 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f906f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90780cf2f0 0x7f907809a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f906f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90780cf2f0 0x7f907809a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48758/0 (socket says 172.21.15.66:48758) 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f906f400640 1 -- 172.21.15.66:0/966889749 learned_addr learned my addr 172.21.15.66:0/966889749 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907cc00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907815ca80 0x7f907815b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f9077e00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90780cb570 0x7f90781f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f906f400640 1 -- 172.21.15.66:0/966889749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907815ca80 msgr2=0x7f907815b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f906f400640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907815ca80 0x7f907815b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f906f400640 1 -- 172.21.15.66:0/966889749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90780cb570 msgr2=0x7f90781f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f906f400640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90780cb570 0x7f90781f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f906f400640 1 -- 172.21.15.66:0/966889749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9070009a70 con 0x7f90780cf2f0 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f9077e00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90780cb570 0x7f90781f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f906f400640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90780cf2f0 0x7f907809a310 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f906000c500 tx=0x7f906000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f9074c00640 1 -- 172.21.15.66:0/966889749 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060005cf0 con 0x7f90780cf2f0 2024-03-30T00:51:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f9074c00640 1 -- 172.21.15.66:0/966889749 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9060005e50 con 0x7f90780cf2f0 2024-03-30T00:51:06.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f9074c00640 1 -- 172.21.15.66:0/966889749 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90600072f0 con 0x7f90780cf2f0 2024-03-30T00:51:06.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907ec00640 1 -- 172.21.15.66:0/966889749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907809b4b0 con 0x7f90780cf2f0 2024-03-30T00:51:06.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907ec00640 1 -- 172.21.15.66:0/966889749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90781f2ab0 con 0x7f90780cf2f0 2024-03-30T00:51:06.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.063+0000 7f907ec00640 1 -- 172.21.15.66:0/966889749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9078066c30 con 0x7f90780cf2f0 2024-03-30T00:51:06.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.067+0000 7f9074c00640 1 -- 172.21.15.66:0/966889749 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9060007450 con 0x7f90780cf2f0 2024-03-30T00:51:06.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.067+0000 7f9074c00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9054066690 0x7f9054068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.067+0000 7f9074c00640 1 -- 172.21.15.66:0/966889749 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9060083820 con 0x7f90780cf2f0 2024-03-30T00:51:06.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.067+0000 7f9077e00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9054066690 0x7f9054068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.067+0000 7f9077e00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9054066690 0x7f9054068b50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f906800a9b0 tx=0x7f9068008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:06.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.071+0000 7f9074c00640 1 -- 172.21.15.66:0/966889749 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9060050af0 con 0x7f90780cf2f0 2024-03-30T00:51:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 -- 172.21.15.66:0/966889749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f907809b960 con 0x7f90780cf2f0 2024-03-30T00:51:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f9074c00640 1 -- 172.21.15.66:0/966889749 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9060017020 con 0x7f90780cf2f0 2024-03-30T00:51:06.239 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:51:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 -- 172.21.15.66:0/966889749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9054066690 msgr2=0x7f9054068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9054066690 0x7f9054068b50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f906800a9b0 tx=0x7f9068008040 comp rx=0 tx=0).stop 2024-03-30T00:51:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 -- 172.21.15.66:0/966889749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90780cf2f0 msgr2=0x7f907809a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90780cf2f0 0x7f907809a310 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f906000c500 tx=0x7f906000c9c0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 -- 172.21.15.66:0/966889749 shutdown_connections 2024-03-30T00:51:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907815ca80 0x7f907815b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9054066690 0x7f9054068b50 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90780cf2f0 0x7f907809a310 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 --2- 172.21.15.66:0/966889749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90780cb570 0x7f90781f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 -- 172.21.15.66:0/966889749 >> 172.21.15.66:0/966889749 conn(0x7f90780b2090 msgr2=0x7f90780ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 -- 172.21.15.66:0/966889749 shutdown_connections 2024-03-30T00:51:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.231+0000 7f907ec00640 1 -- 172.21.15.66:0/966889749 wait complete. 2024-03-30T00:51:06.260 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:51:06.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.391+0000 7f0222c00640 1 Processor -- start 2024-03-30T00:51:06.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.391+0000 7f0222c00640 1 -- start start 2024-03-30T00:51:06.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f0222c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f021c0ce6f0 0x7f021c06ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f0222c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f021c15b880 0x7f021c0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f0222c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021c06da40 con 0x7f021c0c9ba0 2024-03-30T00:51:06.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f0222c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021c1551f0 con 0x7f021c0ce6f0 2024-03-30T00:51:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f0222c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021c0c99f0 con 0x7f021c15b880 2024-03-30T00:51:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f021be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f021c0ce6f0 0x7f021c06ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f0220c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f021c0c9ba0 0x7f021c069d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36444/0 (socket says 172.21.15.66:36444) 2024-03-30T00:51:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f0220c00640 1 -- 172.21.15.66:0/876335104 learned_addr learned my addr 172.21.15.66:0/876335104 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:06.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f0213400640 1 --2- 172.21.15.66:0/876335104 >> v2:172.21.15.156:3300/0 conn(0x7f021c15b880 0x7f021c0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f021be00640 1 -- 172.21.15.66:0/876335104 >> v2:172.21.15.156:3300/0 conn(0x7f021c15b880 msgr2=0x7f021c0c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f021be00640 1 --2- 172.21.15.66:0/876335104 >> v2:172.21.15.156:3300/0 conn(0x7f021c15b880 0x7f021c0c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f021be00640 1 -- 172.21.15.66:0/876335104 >> v1:172.21.15.66:6789/0 conn(0x7f021c0c9ba0 legacy=0x7f021c069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f021b400640 1 -- 172.21.15.66:0/876335104 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273668552 0 0) 0x7f021c06da40 con 0x7f021c0c9ba0 2024-03-30T00:51:06.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.395+0000 7f021be00640 1 -- 172.21.15.66:0/876335104 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c06ce70 con 0x7f021c0ce6f0 2024-03-30T00:51:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f021be00640 1 --2- 172.21.15.66:0/876335104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f021c0ce6f0 0x7f021c06ded0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f021c071a40 tx=0x7f021c06acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d55807b3b99963 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f021b400640 1 -- 172.21.15.66:0/876335104 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c0ab600 con 0x7f021c0ce6f0 2024-03-30T00:51:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f021b400640 1 -- 172.21.15.66:0/876335104 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c0a7050 con 0x7f021c0ce6f0 2024-03-30T00:51:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f021b400640 1 -- 172.21.15.66:0/876335104 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c0ab980 con 0x7f021c0ce6f0 2024-03-30T00:51:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 -- 172.21.15.66:0/876335104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f021c0ce6f0 msgr2=0x7f021c06ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 --2- 172.21.15.66:0/876335104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f021c0ce6f0 0x7f021c06ded0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f021c071a40 tx=0x7f021c06acb0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 -- 172.21.15.66:0/876335104 shutdown_connections 2024-03-30T00:51:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 --2- 172.21.15.66:0/876335104 >> v2:172.21.15.156:3300/0 conn(0x7f021c15b880 0x7f021c0c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 --2- 172.21.15.66:0/876335104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f021c0ce6f0 0x7f021c06ded0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 -- 172.21.15.66:0/876335104 >> 172.21.15.66:0/876335104 conn(0x7f021c0cb610 msgr2=0x7f021c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:06.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 -- 172.21.15.66:0/876335104 shutdown_connections 2024-03-30T00:51:06.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 -- 172.21.15.66:0/876335104 wait complete. 2024-03-30T00:51:06.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 Processor -- start 2024-03-30T00:51:06.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 -- start start 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f021c0c9ba0 0x7f021c1503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f021c0ce6f0 0x7f021c152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f021c15b880 0x7f021c153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c14dd10 con 0x7f021c15b880 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c14df40 con 0x7f021c0ce6f0 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.399+0000 7f0222c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021c14e220 con 0x7f021c0c9ba0 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0220c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f021c15b880 0x7f021c153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0220c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f021c15b880 0x7f021c153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48772/0 (socket says 172.21.15.66:48772) 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0220c00640 1 -- 172.21.15.66:0/4180387626 learned_addr learned my addr 172.21.15.66:0/4180387626 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0213400640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f021c0ce6f0 0x7f021c152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0220c00640 1 -- 172.21.15.66:0/4180387626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f021c0c9ba0 msgr2=0x7f021c1503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f021be00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f021c0c9ba0 0x7f021c1503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0220c00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f021c0c9ba0 0x7f021c1503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0220c00640 1 -- 172.21.15.66:0/4180387626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f021c0ce6f0 msgr2=0x7f021c152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0220c00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f021c0ce6f0 0x7f021c152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0220c00640 1 -- 172.21.15.66:0/4180387626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c009420 con 0x7f021c15b880 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0213400640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f021c0ce6f0 0x7f021c152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f021be00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f021c0c9ba0 0x7f021c1503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0220c00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f021c15b880 0x7f021c153230 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f0214001570 tx=0x7f021400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0218c00640 1 -- 172.21.15.66:0/4180387626 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021400a640 con 0x7f021c15b880 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0218c00640 1 -- 172.21.15.66:0/4180387626 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021400a7c0 con 0x7f021c15b880 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0218c00640 1 -- 172.21.15.66:0/4180387626 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021400ab70 con 0x7f021c15b880 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0222c00640 1 -- 172.21.15.66:0/4180387626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021c0ca540 con 0x7f021c15b880 2024-03-30T00:51:06.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0222c00640 1 -- 172.21.15.66:0/4180387626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021c14e7e0 con 0x7f021c15b880 2024-03-30T00:51:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0222c00640 1 -- 172.21.15.66:0/4180387626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021c066c30 con 0x7f021c15b880 2024-03-30T00:51:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.403+0000 7f0218c00640 1 -- 172.21.15.66:0/4180387626 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f021400acd0 con 0x7f021c15b880 2024-03-30T00:51:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.407+0000 7f0218c00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01f8066690 0x7f01f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.407+0000 7f0218c00640 1 -- 172.21.15.66:0/4180387626 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0214083940 con 0x7f021c15b880 2024-03-30T00:51:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.407+0000 7f021be00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01f8066690 0x7f01f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.407+0000 7f021be00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01f8066690 0x7f01f8068b50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f020c0ad000 tx=0x7f020c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:06.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.411+0000 7f0218c00640 1 -- 172.21.15.66:0/4180387626 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0214050ad0 con 0x7f021c15b880 2024-03-30T00:51:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 -- 172.21.15.66:0/4180387626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f021c0ccae0 con 0x7f021c15b880 2024-03-30T00:51:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0218c00640 1 -- 172.21.15.66:0/4180387626 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0214018070 con 0x7f021c15b880 2024-03-30T00:51:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:51:06.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 -- 172.21.15.66:0/4180387626 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01f8066690 msgr2=0x7f01f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01f8066690 0x7f01f8068b50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f020c0ad000 tx=0x7f020c008040 comp rx=0 tx=0).stop 2024-03-30T00:51:06.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 -- 172.21.15.66:0/4180387626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f021c15b880 msgr2=0x7f021c153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f021c15b880 0x7f021c153230 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f0214001570 tx=0x7f021400cca0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 -- 172.21.15.66:0/4180387626 shutdown_connections 2024-03-30T00:51:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01f8066690 0x7f01f8068b50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f021c15b880 0x7f021c153230 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f021c0ce6f0 0x7f021c152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 --2- 172.21.15.66:0/4180387626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f021c0c9ba0 0x7f021c1503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 -- 172.21.15.66:0/4180387626 >> 172.21.15.66:0/4180387626 conn(0x7f021c0cb610 msgr2=0x7f021c0cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 -- 172.21.15.66:0/4180387626 shutdown_connections 2024-03-30T00:51:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.583+0000 7f0222c00640 1 -- 172.21.15.66:0/4180387626 wait complete. 2024-03-30T00:51:06.611 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:51:06.611 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:06.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:51:06.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.791+0000 7f48ef800640 1 Processor -- start 2024-03-30T00:51:06.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.791+0000 7f48ef800640 1 -- start start 2024-03-30T00:51:06.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.791+0000 7f48ef800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e80cda90 0x7f48e80ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.791+0000 7f48ef800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f48e815a370 0x7f48e80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.791+0000 7f48ef800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e815a990 con 0x7f48e80cde70 2024-03-30T00:51:06.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.791+0000 7f48ef800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e80b1b80 con 0x7f48e815a370 2024-03-30T00:51:06.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.791+0000 7f48ef800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e80cd3e0 con 0x7f48e80cda90 2024-03-30T00:51:06.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.791+0000 7f48dfe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f48e80cde70 0x7f48e8159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36452/0 (socket says 172.21.15.66:36452) 2024-03-30T00:51:06.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.791+0000 7f48dfe00640 1 -- 172.21.15.66:0/3290168367 learned_addr learned my addr 172.21.15.66:0/3290168367 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:06.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48ece00640 1 --2- 172.21.15.66:0/3290168367 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e80cda90 0x7f48e80ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48ed800640 1 --2- 172.21.15.66:0/3290168367 >> v2:172.21.15.156:3300/0 conn(0x7f48e815a370 0x7f48e80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48e7e00640 1 -- 172.21.15.66:0/3290168367 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715836437 0 0) 0x7f48e815a990 con 0x7f48e80cde70 2024-03-30T00:51:06.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48e7e00640 1 -- 172.21.15.66:0/3290168367 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48cc003530 con 0x7f48e80cde70 2024-03-30T00:51:06.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48ece00640 1 -- 172.21.15.66:0/3290168367 >> v2:172.21.15.156:3300/0 conn(0x7f48e815a370 msgr2=0x7f48e80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:06.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48ece00640 1 --2- 172.21.15.66:0/3290168367 >> v2:172.21.15.156:3300/0 conn(0x7f48e815a370 0x7f48e80cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48ece00640 1 -- 172.21.15.66:0/3290168367 >> v1:172.21.15.66:6789/0 conn(0x7f48e80cde70 legacy=0x7f48e8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48ece00640 1 -- 172.21.15.66:0/3290168367 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e80b11e0 con 0x7f48e80cda90 2024-03-30T00:51:06.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48ece00640 1 --2- 172.21.15.66:0/3290168367 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e80cda90 0x7f48e80ce520 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48d80097c0 tx=0x7f48d801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c27372a2490b60c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:06.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48e7e00640 1 -- 172.21.15.66:0/3290168367 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80ab600 con 0x7f48e80cda90 2024-03-30T00:51:06.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48e7e00640 1 -- 172.21.15.66:0/3290168367 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d80a7050 con 0x7f48e80cda90 2024-03-30T00:51:06.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.795+0000 7f48e7e00640 1 -- 172.21.15.66:0/3290168367 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80ab960 con 0x7f48e80cda90 2024-03-30T00:51:06.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 -- 172.21.15.66:0/3290168367 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e80cda90 msgr2=0x7f48e80ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 --2- 172.21.15.66:0/3290168367 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e80cda90 0x7f48e80ce520 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48d80097c0 tx=0x7f48d801c410 comp rx=0 tx=0).stop 2024-03-30T00:51:06.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 -- 172.21.15.66:0/3290168367 shutdown_connections 2024-03-30T00:51:06.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 --2- 172.21.15.66:0/3290168367 >> v2:172.21.15.156:3300/0 conn(0x7f48e815a370 0x7f48e80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 --2- 172.21.15.66:0/3290168367 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e80cda90 0x7f48e80ce520 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 -- 172.21.15.66:0/3290168367 >> 172.21.15.66:0/3290168367 conn(0x7f48e8069f60 msgr2=0x7f48e806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:06.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 -- 172.21.15.66:0/3290168367 shutdown_connections 2024-03-30T00:51:06.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 -- 172.21.15.66:0/3290168367 wait complete. 2024-03-30T00:51:06.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 Processor -- start 2024-03-30T00:51:06.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 -- start start 2024-03-30T00:51:06.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e80cda90 0x7f48e8155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48e80cde70 0x7f48e81632c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e815a370 0x7f48e8155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e8159690 con 0x7f48e80cde70 2024-03-30T00:51:06.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e81593b0 con 0x7f48e80cda90 2024-03-30T00:51:06.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ef800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e80ab0b0 con 0x7f48e815a370 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48dfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48e80cde70 0x7f48e81632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48dfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48e80cde70 0x7f48e81632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48780/0 (socket says 172.21.15.66:48780) 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48dfe00640 1 -- 172.21.15.66:0/1479318800 learned_addr learned my addr 172.21.15.66:0/1479318800 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ece00640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e80cda90 0x7f48e8155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48dfe00640 1 -- 172.21.15.66:0/1479318800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e815a370 msgr2=0x7f48e8155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ed800640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e815a370 0x7f48e8155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48dfe00640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e815a370 0x7f48e8155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48dfe00640 1 -- 172.21.15.66:0/1479318800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e80cda90 msgr2=0x7f48e8155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48dfe00640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e80cda90 0x7f48e8155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48dfe00640 1 -- 172.21.15.66:0/1479318800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d8009420 con 0x7f48e80cde70 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.799+0000 7f48ed800640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e815a370 0x7f48e8155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:06.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.803+0000 7f48dfe00640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48e80cde70 0x7f48e81632c0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c450 tx=0x7f48d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:06.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.803+0000 7f48e5600640 1 -- 172.21.15.66:0/1479318800 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0005cf0 con 0x7f48e80cde70 2024-03-30T00:51:06.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.803+0000 7f48e5600640 1 -- 172.21.15.66:0/1479318800 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d0005e50 con 0x7f48e80cde70 2024-03-30T00:51:06.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.803+0000 7f48e5600640 1 -- 172.21.15.66:0/1479318800 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0007290 con 0x7f48e80cde70 2024-03-30T00:51:06.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.803+0000 7f48ef800640 1 -- 172.21.15.66:0/1479318800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e80ab390 con 0x7f48e80cde70 2024-03-30T00:51:06.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.803+0000 7f48ef800640 1 -- 172.21.15.66:0/1479318800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e8155780 con 0x7f48e80cde70 2024-03-30T00:51:06.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.803+0000 7f48ef800640 1 -- 172.21.15.66:0/1479318800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e80ce4f0 con 0x7f48e80cde70 2024-03-30T00:51:06.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.803+0000 7f48e5600640 1 -- 172.21.15.66:0/1479318800 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f48d00073f0 con 0x7f48e80cde70 2024-03-30T00:51:06.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.803+0000 7f48e5600640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48c40666c0 0x7f48c4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:06.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.803+0000 7f48e5600640 1 -- 172.21.15.66:0/1479318800 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f48d0083920 con 0x7f48e80cde70 2024-03-30T00:51:06.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.807+0000 7f48ece00640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48c40666c0 0x7f48c4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:06.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.807+0000 7f48ece00640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48c40666c0 0x7f48c4068b80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f48d80ad000 tx=0x7f48d80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:06.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.811+0000 7f48e5600640 1 -- 172.21.15.66:0/1479318800 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f48d0050ab0 con 0x7f48e80cde70 2024-03-30T00:51:06.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.971+0000 7f48ef800640 1 -- 172.21.15.66:0/1479318800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48e8066c30 con 0x7f48e80cde70 2024-03-30T00:51:06.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.971+0000 7f48e5600640 1 -- 172.21.15.66:0/1479318800 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16316 (secure 0 0 0) 0x7f48d0017070 con 0x7f48e80cde70 2024-03-30T00:51:06.980 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:06.981 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":125,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:50:39.043056+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 -- 172.21.15.66:0/1479318800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48c40666c0 msgr2=0x7f48c4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48c40666c0 0x7f48c4068b80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f48d80ad000 tx=0x7f48d80a6000 comp rx=0 tx=0).stop 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 -- 172.21.15.66:0/1479318800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48e80cde70 msgr2=0x7f48e81632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48e80cde70 0x7f48e81632c0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c450 tx=0x7f48d000c910 comp rx=0 tx=0).stop 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 -- 172.21.15.66:0/1479318800 shutdown_connections 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e815a370 0x7f48e8155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48c40666c0 0x7f48c4068b80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48e80cde70 0x7f48e81632c0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 --2- 172.21.15.66:0/1479318800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e80cda90 0x7f48e8155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 -- 172.21.15.66:0/1479318800 >> 172.21.15.66:0/1479318800 conn(0x7f48e8069f60 msgr2=0x7f48e80affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 -- 172.21.15.66:0/1479318800 shutdown_connections 2024-03-30T00:51:06.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:06.975+0000 7f48ef800640 1 -- 172.21.15.66:0/1479318800 wait complete. 2024-03-30T00:51:07.002 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:50:35.696965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:50:37.037165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:51:07.003 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:51:07.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48cb200640 1 Processor -- start 2024-03-30T00:51:07.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48cb200640 1 -- start start 2024-03-30T00:51:07.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48cb200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f48c415a640 0x7f48c406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48cb200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48c40cb2c0 0x7f48c40ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48cb200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c406d070 con 0x7f48c40ce520 2024-03-30T00:51:07.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48cb200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c40cbba0 con 0x7f48c415a640 2024-03-30T00:51:07.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48cb200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c40cee90 con 0x7f48c40cb2c0 2024-03-30T00:51:07.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48c3e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f48c415a640 0x7f48c406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48c9200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48c40cb2c0 0x7f48c40ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48c3e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f48c415a640 0x7f48c406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:41874/0 (socket says 172.21.15.66:41874) 2024-03-30T00:51:07.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48c3e00640 1 -- 172.21.15.66:0/1003926079 learned_addr learned my addr 172.21.15.66:0/1003926079 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:07.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48c3e00640 1 -- 172.21.15.66:0/1003926079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48c40cb2c0 msgr2=0x7f48c40ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:07.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48c3e00640 1 --2- 172.21.15.66:0/1003926079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48c40cb2c0 0x7f48c40ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48c3e00640 1 -- 172.21.15.66:0/1003926079 >> v1:172.21.15.156:6789/0 conn(0x7f48c40ce520 legacy=0x7f48c40cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.139+0000 7f48c3e00640 1 -- 172.21.15.66:0/1003926079 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c406e2c0 con 0x7f48c415a640 2024-03-30T00:51:07.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48c3e00640 1 --2- 172.21.15.66:0/1003926079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c415a640 0x7f48c406dd80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009870 tx=0x7f48b401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55449a3740c0589b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:07.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48c3400640 1 -- 172.21.15.66:0/1003926079 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b40ab650 con 0x7f48c415a640 2024-03-30T00:51:07.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48c3400640 1 -- 172.21.15.66:0/1003926079 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b40ab7b0 con 0x7f48c415a640 2024-03-30T00:51:07.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48c3400640 1 -- 172.21.15.66:0/1003926079 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b40abb40 con 0x7f48c415a640 2024-03-30T00:51:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48cb200640 1 -- 172.21.15.66:0/1003926079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c415a640 msgr2=0x7f48c406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48cb200640 1 --2- 172.21.15.66:0/1003926079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c415a640 0x7f48c406dd80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009870 tx=0x7f48b401c560 comp rx=0 tx=0).stop 2024-03-30T00:51:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48cb200640 1 -- 172.21.15.66:0/1003926079 shutdown_connections 2024-03-30T00:51:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48cb200640 1 --2- 172.21.15.66:0/1003926079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48c40cb2c0 0x7f48c40ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48cb200640 1 --2- 172.21.15.66:0/1003926079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c415a640 0x7f48c406dd80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48cb200640 1 -- 172.21.15.66:0/1003926079 >> 172.21.15.66:0/1003926079 conn(0x7f48c4069f60 msgr2=0x7f48c406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:07.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48cb200640 1 -- 172.21.15.66:0/1003926079 shutdown_connections 2024-03-30T00:51:07.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.143+0000 7f48cb200640 1 -- 172.21.15.66:0/1003926079 wait complete. 2024-03-30T00:51:07.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48cb200640 1 Processor -- start 2024-03-30T00:51:07.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48cb200640 1 -- start start 2024-03-30T00:51:07.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48cb200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48c40cb2c0 0x7f48c40cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48cb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c40ce520 0x7f48c4157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48cb200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c415a640 0x7f48c4156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48cb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c4159e40 con 0x7f48c40ce520 2024-03-30T00:51:07.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48cb200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c41598f0 con 0x7f48c415a640 2024-03-30T00:51:07.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48cb200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c41582b0 con 0x7f48c40cb2c0 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c40ce520 0x7f48c4157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c40ce520 0x7f48c4157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48786/0 (socket says 172.21.15.66:48786) 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48bbe00640 1 -- 172.21.15.66:0/2348389613 learned_addr learned my addr 172.21.15.66:0/2348389613 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48c3e00640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48c40cb2c0 0x7f48c40cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48c9200640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c415a640 0x7f48c4156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48bbe00640 1 -- 172.21.15.66:0/2348389613 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48c40cb2c0 msgr2=0x7f48c40cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48bbe00640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48c40cb2c0 0x7f48c40cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48bbe00640 1 -- 172.21.15.66:0/2348389613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c415a640 msgr2=0x7f48c4156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48bbe00640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c415a640 0x7f48c4156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48bbe00640 1 -- 172.21.15.66:0/2348389613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b40094d0 con 0x7f48c40ce520 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48bbe00640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c40ce520 0x7f48c4157d70 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00c530 tx=0x7f48ac00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48c0c00640 1 -- 172.21.15.66:0/2348389613 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac005220 con 0x7f48c40ce520 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48c0c00640 1 -- 172.21.15.66:0/2348389613 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac005380 con 0x7f48c40ce520 2024-03-30T00:51:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48cb200640 1 -- 172.21.15.66:0/2348389613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c415d370 con 0x7f48c40ce520 2024-03-30T00:51:07.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48c0c00640 1 -- 172.21.15.66:0/2348389613 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac007270 con 0x7f48c40ce520 2024-03-30T00:51:07.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48cb200640 1 -- 172.21.15.66:0/2348389613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c415d650 con 0x7f48c40ce520 2024-03-30T00:51:07.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.147+0000 7f48cb200640 1 -- 172.21.15.66:0/2348389613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c4066c30 con 0x7f48c40ce520 2024-03-30T00:51:07.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.151+0000 7f48c0c00640 1 -- 172.21.15.66:0/2348389613 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f48ac0073d0 con 0x7f48c40ce520 2024-03-30T00:51:07.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.151+0000 7f48c0c00640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48a0066690 0x7f48a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.151+0000 7f48c0c00640 1 -- 172.21.15.66:0/2348389613 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f48ac083be0 con 0x7f48c40ce520 2024-03-30T00:51:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.151+0000 7f48c3e00640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48a0066690 0x7f48a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.151+0000 7f48c3e00640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48a0066690 0x7f48a0068b50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f48b40af000 tx=0x7f48b40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.155+0000 7f48c0c00640 1 -- 172.21.15.66:0/2348389613 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f48ac050e00 con 0x7f48c40ce520 2024-03-30T00:51:07.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.315+0000 7f48cb200640 1 -- 172.21.15.66:0/2348389613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48c4157ab0 con 0x7f48c40ce520 2024-03-30T00:51:07.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48c0c00640 1 -- 172.21.15.66:0/2348389613 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48ac054420 con 0x7f48c40ce520 2024-03-30T00:51:07.326 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:51:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 -- 172.21.15.66:0/2348389613 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48a0066690 msgr2=0x7f48a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48a0066690 0x7f48a0068b50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f48b40af000 tx=0x7f48b40a7000 comp rx=0 tx=0).stop 2024-03-30T00:51:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 -- 172.21.15.66:0/2348389613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c40ce520 msgr2=0x7f48c4157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c40ce520 0x7f48c4157d70 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00c530 tx=0x7f48ac00c9f0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 -- 172.21.15.66:0/2348389613 shutdown_connections 2024-03-30T00:51:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c415a640 0x7f48c4156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48a0066690 0x7f48a0068b50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c40ce520 0x7f48c4157d70 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 --2- 172.21.15.66:0/2348389613 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48c40cb2c0 0x7f48c40cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 -- 172.21.15.66:0/2348389613 >> 172.21.15.66:0/2348389613 conn(0x7f48c4069f60 msgr2=0x7f48c40adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:07.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 -- 172.21.15.66:0/2348389613 shutdown_connections 2024-03-30T00:51:07.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.319+0000 7f48cb200640 1 -- 172.21.15.66:0/2348389613 wait complete. 2024-03-30T00:51:07.347 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:51:07.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46dc600640 1 Processor -- start 2024-03-30T00:51:07.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46dc600640 1 -- start start 2024-03-30T00:51:07.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46dc600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d415a640 0x7f46d406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46dc600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d406d070 con 0x7f46d40ce520 2024-03-30T00:51:07.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46dc600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d40cbba0 con 0x7f46d40cabb0 2024-03-30T00:51:07.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46dc600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d406e2c0 con 0x7f46d415a640 2024-03-30T00:51:07.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46d9200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f46d40ce520 0x7f46d40ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36456/0 (socket says 172.21.15.66:36456) 2024-03-30T00:51:07.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46d9200640 1 -- 172.21.15.66:0/3800025256 learned_addr learned my addr 172.21.15.66:0/3800025256 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:07.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46d9c00640 1 --2- 172.21.15.66:0/3800025256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d415a640 0x7f46d406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46d3e00640 1 -- 172.21.15.66:0/3800025256 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448978641 0 0) 0x7f46d406d070 con 0x7f46d40ce520 2024-03-30T00:51:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46d3e00640 1 -- 172.21.15.66:0/3800025256 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46bc003530 con 0x7f46d40ce520 2024-03-30T00:51:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46d3e00640 1 -- 172.21.15.66:0/3800025256 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024238415 0 0) 0x7f46d40cbba0 con 0x7f46d40cabb0 2024-03-30T00:51:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46d3e00640 1 -- 172.21.15.66:0/3800025256 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46d406d070 con 0x7f46d40cabb0 2024-03-30T00:51:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46d9c00640 1 -- 172.21.15.66:0/3800025256 >> v1:172.21.15.156:6789/0 conn(0x7f46d40cabb0 legacy=0x7f46d40cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46d9c00640 1 -- 172.21.15.66:0/3800025256 >> v1:172.21.15.66:6789/0 conn(0x7f46d40ce520 legacy=0x7f46d40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.483+0000 7f46d9c00640 1 -- 172.21.15.66:0/3800025256 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d40cb8c0 con 0x7f46d415a640 2024-03-30T00:51:07.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.487+0000 7f46d9c00640 1 --2- 172.21.15.66:0/3800025256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d415a640 0x7f46d406dd80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46c80097f0 tx=0x7f46c801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b69f5ff958313323 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:07.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.487+0000 7f46d3e00640 1 -- 172.21.15.66:0/3800025256 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c80aa610 con 0x7f46d415a640 2024-03-30T00:51:07.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.487+0000 7f46d3e00640 1 -- 172.21.15.66:0/3800025256 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c80a6040 con 0x7f46d415a640 2024-03-30T00:51:07.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.487+0000 7f46d3e00640 1 -- 172.21.15.66:0/3800025256 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c80aa970 con 0x7f46d415a640 2024-03-30T00:51:07.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.487+0000 7f46dc600640 1 -- 172.21.15.66:0/3800025256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d415a640 msgr2=0x7f46d406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.487+0000 7f46dc600640 1 --2- 172.21.15.66:0/3800025256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d415a640 0x7f46d406dd80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46c80097f0 tx=0x7f46c801c4b0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.487+0000 7f46dc600640 1 -- 172.21.15.66:0/3800025256 shutdown_connections 2024-03-30T00:51:07.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.487+0000 7f46dc600640 1 --2- 172.21.15.66:0/3800025256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d415a640 0x7f46d406dd80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.487+0000 7f46dc600640 1 -- 172.21.15.66:0/3800025256 >> 172.21.15.66:0/3800025256 conn(0x7f46d4069f60 msgr2=0x7f46d406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:07.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.487+0000 7f46dc600640 1 -- 172.21.15.66:0/3800025256 shutdown_connections 2024-03-30T00:51:07.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.487+0000 7f46dc600640 1 -- 172.21.15.66:0/3800025256 wait complete. 2024-03-30T00:51:07.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46dc600640 1 Processor -- start 2024-03-30T00:51:07.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46dc600640 1 -- start start 2024-03-30T00:51:07.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46dc600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d40cabb0 0x7f46d4152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46dc600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d40ce520 0x7f46d4097960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46dc600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46d415a640 0x7f46d40c1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46dc600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d40cc4b0 con 0x7f46d40cabb0 2024-03-30T00:51:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46dc600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d40cbec0 con 0x7f46d415a640 2024-03-30T00:51:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46dc600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4097ea0 con 0x7f46d40ce520 2024-03-30T00:51:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d40cabb0 0x7f46d4152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d40cabb0 0x7f46d4152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48800/0 (socket says 172.21.15.66:48800) 2024-03-30T00:51:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d9c00640 1 -- 172.21.15.66:0/4003556019 learned_addr learned my addr 172.21.15.66:0/4003556019 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:07.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46da600640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46d415a640 0x7f46d40c1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d9c00640 1 -- 172.21.15.66:0/4003556019 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d40ce520 msgr2=0x7f46d4097960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d9200640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d40ce520 0x7f46d4097960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d9c00640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d40ce520 0x7f46d4097960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d9c00640 1 -- 172.21.15.66:0/4003556019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46d415a640 msgr2=0x7f46d40c1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d9c00640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46d415a640 0x7f46d40c1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d9c00640 1 -- 172.21.15.66:0/4003556019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c8009420 con 0x7f46d40cabb0 2024-03-30T00:51:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46da600640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46d415a640 0x7f46d40c1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d9200640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d40ce520 0x7f46d4097960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d9c00640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d40cabb0 0x7f46d4152030 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f46c801ab60 tx=0x7f46c8019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d1600640 1 -- 172.21.15.66:0/4003556019 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8003a30 con 0x7f46d40cabb0 2024-03-30T00:51:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d1600640 1 -- 172.21.15.66:0/4003556019 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c80b8070 con 0x7f46d40cabb0 2024-03-30T00:51:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46d1600640 1 -- 172.21.15.66:0/4003556019 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8003d90 con 0x7f46d40cabb0 2024-03-30T00:51:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46dc600640 1 -- 172.21.15.66:0/4003556019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d4158160 con 0x7f46d40cabb0 2024-03-30T00:51:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.491+0000 7f46dc600640 1 -- 172.21.15.66:0/4003556019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d40cc0d0 con 0x7f46d40cabb0 2024-03-30T00:51:07.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.495+0000 7f46dc600640 1 -- 172.21.15.66:0/4003556019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d4066c30 con 0x7f46d40cabb0 2024-03-30T00:51:07.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.495+0000 7f46d1600640 1 -- 172.21.15.66:0/4003556019 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f46c80bf480 con 0x7f46d40cabb0 2024-03-30T00:51:07.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.495+0000 7f46d1600640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46b4066690 0x7f46b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.495+0000 7f46d1600640 1 -- 172.21.15.66:0/4003556019 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f46c80ba070 con 0x7f46d40cabb0 2024-03-30T00:51:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.495+0000 7f46d9200640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46b4066690 0x7f46b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.499+0000 7f46d9200640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46b4066690 0x7f46b4068b50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f46d40c4e00 tx=0x7f46c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:07.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.499+0000 7f46d1600640 1 -- 172.21.15.66:0/4003556019 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f46c80ec790 con 0x7f46d40cabb0 2024-03-30T00:51:07.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.659+0000 7f46dc600640 1 -- 172.21.15.66:0/4003556019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f46d4165a00 con 0x7f46d40cabb0 2024-03-30T00:51:07.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.659+0000 7f46d1600640 1 -- 172.21.15.66:0/4003556019 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46c80efdb0 con 0x7f46d40cabb0 2024-03-30T00:51:07.668 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.659+0000 7f46dc600640 1 -- 172.21.15.66:0/4003556019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46b4066690 msgr2=0x7f46b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.659+0000 7f46dc600640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46b4066690 0x7f46b4068b50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f46d40c4e00 tx=0x7f46c0009040 comp rx=0 tx=0).stop 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.659+0000 7f46dc600640 1 -- 172.21.15.66:0/4003556019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d40cabb0 msgr2=0x7f46d4152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.659+0000 7f46dc600640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d40cabb0 0x7f46d4152030 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f46c801ab60 tx=0x7f46c8019250 comp rx=0 tx=0).stop 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.663+0000 7f46dc600640 1 -- 172.21.15.66:0/4003556019 shutdown_connections 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.663+0000 7f46dc600640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46d415a640 0x7f46d40c1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.663+0000 7f46dc600640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46d40ce520 0x7f46d4097960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.663+0000 7f46dc600640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f46b4066690 0x7f46b4068b50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.663+0000 7f46dc600640 1 --2- 172.21.15.66:0/4003556019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46d40cabb0 0x7f46d4152030 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.663+0000 7f46dc600640 1 -- 172.21.15.66:0/4003556019 >> 172.21.15.66:0/4003556019 conn(0x7f46d4069f60 msgr2=0x7f46d415c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.663+0000 7f46dc600640 1 -- 172.21.15.66:0/4003556019 shutdown_connections 2024-03-30T00:51:07.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.663+0000 7f46dc600640 1 -- 172.21.15.66:0/4003556019 wait complete. 2024-03-30T00:51:07.689 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:51:07.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.959+0000 7fa850a00640 1 Processor -- start 2024-03-30T00:51:07.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.959+0000 7fa850a00640 1 -- start start 2024-03-30T00:51:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa850a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa84c0d0b50 0x7fa84c0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa850a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c0cd130 con 0x7fa84c0d31b0 2024-03-30T00:51:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa850a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c0d0970 con 0x7fa84c0d0160 2024-03-30T00:51:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa850a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84c0cd360 con 0x7fa84c0d0b50 2024-03-30T00:51:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa84c0d31b0 0x7fa84c0d0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36470/0 (socket says 172.21.15.66:36470) 2024-03-30T00:51:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849e00640 1 -- 172.21.15.66:0/2862459678 learned_addr learned my addr 172.21.15.66:0/2862459678 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa84a800640 1 --2- 172.21.15.66:0/2862459678 >> v2:172.21.15.156:3300/0 conn(0x7fa84c0d0b50 0x7fa84c0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019037393 0 0) 0x7fa84c0cd130 con 0x7fa84c0d31b0 2024-03-30T00:51:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa830003530 con 0x7fa84c0d31b0 2024-03-30T00:51:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591143903 0 0) 0x7fa84c0d0970 con 0x7fa84c0d0160 2024-03-30T00:51:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa84c0cd130 con 0x7fa84c0d0160 2024-03-30T00:51:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 813740697 0 0) 0x7fa830003530 con 0x7fa84c0d31b0 2024-03-30T00:51:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 >> v2:172.21.15.156:3300/0 conn(0x7fa84c0d0b50 msgr2=0x7fa84c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 --2- 172.21.15.66:0/2862459678 >> v2:172.21.15.156:3300/0 conn(0x7fa84c0d0b50 0x7fa84c0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 >> v1:172.21.15.156:6789/0 conn(0x7fa84c0d0160 legacy=0x7fa84c06ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c0cd640 con 0x7fa84c0d31b0 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fa838001680 con 0x7fa84c0d31b0 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 >> v1:172.21.15.66:6789/0 conn(0x7fa84c0d31b0 legacy=0x7fa84c0d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa830003b10 0x7fa830003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 0x7fa830004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa830005200 0x7fa83000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c0cd360 con 0x7fa8300047d0 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83000c160 con 0x7fa830005200 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83000c320 con 0x7fa830003b10 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849e00640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 0x7fa830004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa84a800640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa830003b10 0x7fa830003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa84b200640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa830005200 0x7fa83000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849e00640 1 -- 172.21.15.66:0/2862459678 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa830003b10 msgr2=0x7fa830003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849e00640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa830003b10 0x7fa830003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849e00640 1 -- 172.21.15.66:0/2862459678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa830005200 msgr2=0x7fa83000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849e00640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa830005200 0x7fa83000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa849e00640 1 -- 172.21.15.66:0/2862459678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa830003530 con 0x7fa8300047d0 2024-03-30T00:51:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa84b200640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa830005200 0x7fa83000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.963+0000 7fa84b200640 1 -- 172.21.15.66:0/2862459678 reap_dead start 2024-03-30T00:51:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa849e00640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 0x7fa830004c50 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c160 tx=0x7fa83801cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214e387ca785f74 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8380058e0 con 0x7fa8300047d0 2024-03-30T00:51:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa838005a60 con 0x7fa8300047d0 2024-03-30T00:51:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa849400640 1 -- 172.21.15.66:0/2862459678 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838005e50 con 0x7fa8300047d0 2024-03-30T00:51:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa850a00640 1 -- 172.21.15.66:0/2862459678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 msgr2=0x7fa830004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa850a00640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 0x7fa830004c50 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c160 tx=0x7fa83801cb90 comp rx=0 tx=0).stop 2024-03-30T00:51:07.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa850a00640 1 -- 172.21.15.66:0/2862459678 shutdown_connections 2024-03-30T00:51:07.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa850a00640 1 --2- 172.21.15.66:0/2862459678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 0x7fa830004c50 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa850a00640 1 -- 172.21.15.66:0/2862459678 >> 172.21.15.66:0/2862459678 conn(0x7fa84c09b8c0 msgr2=0x7fa84c0d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:07.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa850a00640 1 -- 172.21.15.66:0/2862459678 shutdown_connections 2024-03-30T00:51:07.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa850a00640 1 -- 172.21.15.66:0/2862459678 wait complete. 2024-03-30T00:51:07.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.967+0000 7fa850a00640 1 Processor -- start 2024-03-30T00:51:07.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa850a00640 1 -- start start 2024-03-30T00:51:07.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa850a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 0x7fa84c153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa850a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa84c0b0cf0 0x7fa84c0b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa850a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa84c15e4c0 0x7fa84c15e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa850a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c0b1690 con 0x7fa8300047d0 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa850a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c15d940 con 0x7fa84c0b0cf0 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa850a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c15cd30 con 0x7fa84c15e4c0 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa84a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 0x7fa84c153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa84a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 0x7fa84c153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48822/0 (socket says 172.21.15.66:48822) 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa84a800640 1 -- 172.21.15.66:0/2144616974 learned_addr learned my addr 172.21.15.66:0/2144616974 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa849e00640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa84c0b0cf0 0x7fa84c0b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa84b200640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa84c15e4c0 0x7fa84c15e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa84a800640 1 -- 172.21.15.66:0/2144616974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa84c15e4c0 msgr2=0x7fa84c15e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa84a800640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa84c15e4c0 0x7fa84c15e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa84a800640 1 -- 172.21.15.66:0/2144616974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa84c0b0cf0 msgr2=0x7fa84c0b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa84a800640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa84c0b0cf0 0x7fa84c0b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:07.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa84a800640 1 -- 172.21.15.66:0/2144616974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c15d010 con 0x7fa8300047d0 2024-03-30T00:51:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa84b200640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa84c15e4c0 0x7fa84c15e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa84a800640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 0x7fa84c153c50 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fa83c011900 tx=0x7fa83c011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa842a00640 1 -- 172.21.15.66:0/2144616974 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0095b0 con 0x7fa8300047d0 2024-03-30T00:51:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa842a00640 1 -- 172.21.15.66:0/2144616974 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c004240 con 0x7fa8300047d0 2024-03-30T00:51:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa850a00640 1 -- 172.21.15.66:0/2144616974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c15d2f0 con 0x7fa8300047d0 2024-03-30T00:51:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa842a00640 1 -- 172.21.15.66:0/2144616974 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00f8e0 con 0x7fa8300047d0 2024-03-30T00:51:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa850a00640 1 -- 172.21.15.66:0/2144616974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c154190 con 0x7fa8300047d0 2024-03-30T00:51:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.971+0000 7fa850a00640 1 -- 172.21.15.66:0/2144616974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c073e00 con 0x7fa8300047d0 2024-03-30T00:51:07.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.975+0000 7fa842a00640 1 -- 172.21.15.66:0/2144616974 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa83c00fa40 con 0x7fa8300047d0 2024-03-30T00:51:07.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.975+0000 7fa842a00640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa82c066690 0x7fa82c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:07.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.975+0000 7fa842a00640 1 -- 172.21.15.66:0/2144616974 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa83c084820 con 0x7fa8300047d0 2024-03-30T00:51:07.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.975+0000 7fa849e00640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa82c066690 0x7fa82c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:07.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.975+0000 7fa849e00640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa82c066690 0x7fa82c068b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa8380aa000 tx=0x7fa838001a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:07.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:07.979+0000 7fa842a00640 1 -- 172.21.15.66:0/2144616974 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa83c051af0 con 0x7fa8300047d0 2024-03-30T00:51:08.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 -- 172.21.15.66:0/2144616974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa84c0cfcb0 con 0x7fa8300047d0 2024-03-30T00:51:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa842a00640 1 -- 172.21.15.66:0/2144616974 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa83c019020 con 0x7fa8300047d0 2024-03-30T00:51:08.147 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:51:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 -- 172.21.15.66:0/2144616974 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa82c066690 msgr2=0x7fa82c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa82c066690 0x7fa82c068b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa8380aa000 tx=0x7fa838001a00 comp rx=0 tx=0).stop 2024-03-30T00:51:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 -- 172.21.15.66:0/2144616974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 msgr2=0x7fa84c153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 0x7fa84c153c50 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fa83c011900 tx=0x7fa83c011dc0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 -- 172.21.15.66:0/2144616974 shutdown_connections 2024-03-30T00:51:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa84c15e4c0 0x7fa84c15e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa84c0b0cf0 0x7fa84c0b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa82c066690 0x7fa82c068b50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 --2- 172.21.15.66:0/2144616974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8300047d0 0x7fa84c153c50 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 -- 172.21.15.66:0/2144616974 >> 172.21.15.66:0/2144616974 conn(0x7fa84c09b8c0 msgr2=0x7fa84c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 -- 172.21.15.66:0/2144616974 shutdown_connections 2024-03-30T00:51:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.139+0000 7fa850a00640 1 -- 172.21.15.66:0/2144616974 wait complete. 2024-03-30T00:51:08.168 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:51:08.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.299+0000 7f95fd800640 1 Processor -- start 2024-03-30T00:51:08.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.299+0000 7f95fd800640 1 -- start start 2024-03-30T00:51:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95fd800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95f815a640 0x7f95f806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95fd800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f95f80cabb0 0x7f95f80cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95fd800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f806d070 con 0x7f95f80ce520 2024-03-30T00:51:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95fd800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f80cbba0 con 0x7f95f80cabb0 2024-03-30T00:51:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95fd800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f806e2c0 con 0x7f95f815a640 2024-03-30T00:51:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95f6000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f95f80ce520 0x7f95f80ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:36480/0 (socket says 172.21.15.66:36480) 2024-03-30T00:51:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95f6000640 1 -- 172.21.15.66:0/1095919701 learned_addr learned my addr 172.21.15.66:0/1095919701 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95f7400640 1 --2- 172.21.15.66:0/1095919701 >> v2:172.21.15.66:3300/0 conn(0x7f95f80cabb0 0x7f95f80cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95f6a00640 1 --2- 172.21.15.66:0/1095919701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95f815a640 0x7f95f806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95f7400640 1 -- 172.21.15.66:0/1095919701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95f815a640 msgr2=0x7f95f806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95f7400640 1 --2- 172.21.15.66:0/1095919701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95f815a640 0x7f95f806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95f7400640 1 -- 172.21.15.66:0/1095919701 >> v1:172.21.15.66:6789/0 conn(0x7f95f80ce520 legacy=0x7f95f80ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95f7400640 1 -- 172.21.15.66:0/1095919701 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f80cb8c0 con 0x7f95f80cabb0 2024-03-30T00:51:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95f5600640 1 -- 172.21.15.66:0/1095919701 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911844378 0 0) 0x7f95f806d070 con 0x7f95f80ce520 2024-03-30T00:51:08.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.303+0000 7f95f7400640 1 --2- 172.21.15.66:0/1095919701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f80cabb0 0x7f95f80cb040 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009d90 tx=0x7f95f001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1632e0fe73ff1b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:08.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95f5600640 1 -- 172.21.15.66:0/1095919701 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00a5a00 con 0x7f95f80cabb0 2024-03-30T00:51:08.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95f5600640 1 -- 172.21.15.66:0/1095919701 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f00a5b60 con 0x7f95f80cabb0 2024-03-30T00:51:08.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95f5600640 1 -- 172.21.15.66:0/1095919701 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00ad440 con 0x7f95f80cabb0 2024-03-30T00:51:08.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 -- 172.21.15.66:0/1095919701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f80cabb0 msgr2=0x7f95f80cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:08.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 --2- 172.21.15.66:0/1095919701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f80cabb0 0x7f95f80cb040 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009d90 tx=0x7f95f001c940 comp rx=0 tx=0).stop 2024-03-30T00:51:08.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 -- 172.21.15.66:0/1095919701 shutdown_connections 2024-03-30T00:51:08.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 --2- 172.21.15.66:0/1095919701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f80cabb0 0x7f95f80cb040 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 --2- 172.21.15.66:0/1095919701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95f815a640 0x7f95f806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 -- 172.21.15.66:0/1095919701 >> 172.21.15.66:0/1095919701 conn(0x7f95f8069f60 msgr2=0x7f95f806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:08.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 -- 172.21.15.66:0/1095919701 shutdown_connections 2024-03-30T00:51:08.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 -- 172.21.15.66:0/1095919701 wait complete. 2024-03-30T00:51:08.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 Processor -- start 2024-03-30T00:51:08.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 -- start start 2024-03-30T00:51:08.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95f80cabb0 0x7f95f80cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:08.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f80ce520 0x7f95f815c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:08.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f815a640 0x7f95f815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80cd180 con 0x7f95f80ce520 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80cc2d0 con 0x7f95f815a640 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95fd800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80cc570 con 0x7f95f80cabb0 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95f6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f80ce520 0x7f95f815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95f6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f80ce520 0x7f95f815c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48842/0 (socket says 172.21.15.66:48842) 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95f6000640 1 -- 172.21.15.66:0/3139204562 learned_addr learned my addr 172.21.15.66:0/3139204562 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95f6a00640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95f80cabb0 0x7f95f80cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.307+0000 7f95f7400640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f815a640 0x7f95f815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95f6000640 1 -- 172.21.15.66:0/3139204562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95f80cabb0 msgr2=0x7f95f80cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95f6000640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95f80cabb0 0x7f95f80cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95f6000640 1 -- 172.21.15.66:0/3139204562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f815a640 msgr2=0x7f95f815c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95f6000640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f815a640 0x7f95f815c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95f6000640 1 -- 172.21.15.66:0/3139204562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f0009a70 con 0x7f95f80ce520 2024-03-30T00:51:08.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95f6000640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f80ce520 0x7f95f815c330 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f95e40011f0 tx=0x7f95e400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:08.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95eea00640 1 -- 172.21.15.66:0/3139204562 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e40074e0 con 0x7f95f80ce520 2024-03-30T00:51:08.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95eea00640 1 -- 172.21.15.66:0/3139204562 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e4007640 con 0x7f95f80ce520 2024-03-30T00:51:08.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95eea00640 1 -- 172.21.15.66:0/3139204562 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4004a10 con 0x7f95f80ce520 2024-03-30T00:51:08.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95fd800640 1 -- 172.21.15.66:0/3139204562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f80cc850 con 0x7f95f80ce520 2024-03-30T00:51:08.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95fd800640 1 -- 172.21.15.66:0/3139204562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f80cce10 con 0x7f95f80ce520 2024-03-30T00:51:08.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95fd800640 1 -- 172.21.15.66:0/3139204562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f8066c30 con 0x7f95f80ce520 2024-03-30T00:51:08.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95eea00640 1 -- 172.21.15.66:0/3139204562 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f95e4004b70 con 0x7f95f80ce520 2024-03-30T00:51:08.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95eea00640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f95d8066690 0x7f95d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:08.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95eea00640 1 -- 172.21.15.66:0/3139204562 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f95e4083580 con 0x7f95f80ce520 2024-03-30T00:51:08.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.311+0000 7f95f6a00640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f95d8066690 0x7f95d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:08.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.315+0000 7f95f6a00640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f95d8066690 0x7f95d8068b50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009ac0 tx=0x7f95e8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:08.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.315+0000 7f95eea00640 1 -- 172.21.15.66:0/3139204562 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f95e40507c0 con 0x7f95f80ce520 2024-03-30T00:51:08.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.487+0000 7f95fd800640 1 -- 172.21.15.66:0/3139204562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f95f80cd2c0 con 0x7f95f80ce520 2024-03-30T00:51:08.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.487+0000 7f95eea00640 1 -- 172.21.15.66:0/3139204562 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) v1 ==== 80+0+0 (secure 0 0 0) 0x7f95e4018080 con 0x7f95f80ce520 2024-03-30T00:51:08.497 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T00:51:08.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 -- 172.21.15.66:0/3139204562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f95d8066690 msgr2=0x7f95d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f95d8066690 0x7f95d8068b50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009ac0 tx=0x7f95e8009190 comp rx=0 tx=0).stop 2024-03-30T00:51:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 -- 172.21.15.66:0/3139204562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f80ce520 msgr2=0x7f95f815c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f80ce520 0x7f95f815c330 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f95e40011f0 tx=0x7f95e400c9b0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 -- 172.21.15.66:0/3139204562 shutdown_connections 2024-03-30T00:51:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95f815a640 0x7f95f815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f95d8066690 0x7f95d8068b50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f80ce520 0x7f95f815c330 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 --2- 172.21.15.66:0/3139204562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95f80cabb0 0x7f95f80cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 -- 172.21.15.66:0/3139204562 >> 172.21.15.66:0/3139204562 conn(0x7f95f8069f60 msgr2=0x7f95f8159890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 -- 172.21.15.66:0/3139204562 shutdown_connections 2024-03-30T00:51:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:08.491+0000 7f95fd800640 1 -- 172.21.15.66:0/3139204562 wait complete. 2024-03-30T00:51:08.518 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:51:08.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:08.518 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:08.536 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[54282]: fuse finished with error 0 and tester_r 0 2024-03-30T00:51:08.541 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:51:14.642 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:51:14.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:14.643 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:51:14.654 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:51:14.654 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:14.706 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:51:14.706 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi066.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:14.706 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:51:14.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:14.707 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:51:14.758 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-30T00:51:14.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:51:14.759 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:51:14.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5944000640 1 Processor -- start 2024-03-30T00:51:14.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5944000640 1 -- start start 2024-03-30T00:51:14.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5944000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f593c15c780 0x7f593c0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:14.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5944000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593c0b1d00 con 0x7f593c160bb0 2024-03-30T00:51:14.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5944000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593c0cff70 con 0x7f593c15cb60 2024-03-30T00:51:14.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5944000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593c0cd820 con 0x7f593c15c780 2024-03-30T00:51:14.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5942000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f593c160bb0 0x7f593c0d0290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46486/0 (socket says 172.21.15.66:46486) 2024-03-30T00:51:14.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5942000640 1 -- 172.21.15.66:0/1788044559 learned_addr learned my addr 172.21.15.66:0/1788044559 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:14.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5941600640 1 --2- 172.21.15.66:0/1788044559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f593c15c780 0x7f593c0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:14.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f593be00640 1 -- 172.21.15.66:0/1788044559 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59471382 0 0) 0x7f593c0b1d00 con 0x7f593c160bb0 2024-03-30T00:51:14.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f593be00640 1 -- 172.21.15.66:0/1788044559 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5924003530 con 0x7f593c160bb0 2024-03-30T00:51:14.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f593be00640 1 -- 172.21.15.66:0/1788044559 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223499694 0 0) 0x7f593c0cff70 con 0x7f593c15cb60 2024-03-30T00:51:14.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f593be00640 1 -- 172.21.15.66:0/1788044559 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f593c0b1d00 con 0x7f593c15cb60 2024-03-30T00:51:14.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5941600640 1 -- 172.21.15.66:0/1788044559 >> v1:172.21.15.156:6789/0 conn(0x7f593c15cb60 legacy=0x7f593c1604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:14.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5941600640 1 -- 172.21.15.66:0/1788044559 >> v1:172.21.15.66:6789/0 conn(0x7f593c160bb0 legacy=0x7f593c0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:14.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.947+0000 7f5941600640 1 -- 172.21.15.66:0/1788044559 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c0cdb00 con 0x7f593c15c780 2024-03-30T00:51:14.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f5941600640 1 --2- 172.21.15.66:0/1788044559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f593c15c780 0x7f593c0cfa30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f59300097f0 tx=0x7f593001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd5e63868a3a6e68 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:14.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f593be00640 1 -- 172.21.15.66:0/1788044559 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59300ac620 con 0x7f593c15c780 2024-03-30T00:51:14.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f593be00640 1 -- 172.21.15.66:0/1788044559 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59300ac780 con 0x7f593c15c780 2024-03-30T00:51:14.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f593be00640 1 -- 172.21.15.66:0/1788044559 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59300acb10 con 0x7f593c15c780 2024-03-30T00:51:14.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f5944000640 1 -- 172.21.15.66:0/1788044559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f593c15c780 msgr2=0x7f593c0cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:14.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f5944000640 1 --2- 172.21.15.66:0/1788044559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f593c15c780 0x7f593c0cfa30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f59300097f0 tx=0x7f593001c4b0 comp rx=0 tx=0).stop 2024-03-30T00:51:14.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f5944000640 1 -- 172.21.15.66:0/1788044559 shutdown_connections 2024-03-30T00:51:14.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f5944000640 1 --2- 172.21.15.66:0/1788044559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f593c15c780 0x7f593c0cfa30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:14.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f5944000640 1 -- 172.21.15.66:0/1788044559 >> 172.21.15.66:0/1788044559 conn(0x7f593c09ba30 msgr2=0x7f593c089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:14.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f5944000640 1 -- 172.21.15.66:0/1788044559 shutdown_connections 2024-03-30T00:51:14.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f5944000640 1 -- 172.21.15.66:0/1788044559 wait complete. 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f5944000640 1 Processor -- start 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.951+0000 7f5944000640 1 -- start start 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5944000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f593c15c780 0x7f593c0c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5944000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f593c15cb60 0x7f593c151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5944000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593c160bb0 0x7f593c151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5944000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593c1524f0 con 0x7f593c160bb0 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5944000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593c0b31d0 con 0x7f593c15c780 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5944000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593c0b34b0 con 0x7f593c15cb60 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5942000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593c160bb0 0x7f593c151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5942000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593c160bb0 0x7f593c151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32868/0 (socket says 172.21.15.66:32868) 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5942000640 1 -- 172.21.15.66:0/1745976889 learned_addr learned my addr 172.21.15.66:0/1745976889 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5941600640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f593c15c780 0x7f593c0c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5940c00640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f593c15cb60 0x7f593c151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:14.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5942000640 1 -- 172.21.15.66:0/1745976889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f593c15cb60 msgr2=0x7f593c151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:14.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5942000640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f593c15cb60 0x7f593c151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:14.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5942000640 1 -- 172.21.15.66:0/1745976889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f593c15c780 msgr2=0x7f593c0c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:14.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5942000640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f593c15c780 0x7f593c0c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:14.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5942000640 1 -- 172.21.15.66:0/1745976889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5930009420 con 0x7f593c160bb0 2024-03-30T00:51:14.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5941600640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f593c15c780 0x7f593c0c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:51:14.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5942000640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593c160bb0 0x7f593c151fb0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f593400caa0 tx=0x7f593400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:14.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5939600640 1 -- 172.21.15.66:0/1745976889 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593400ac00 con 0x7f593c160bb0 2024-03-30T00:51:14.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5939600640 1 -- 172.21.15.66:0/1745976889 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593400ad60 con 0x7f593c160bb0 2024-03-30T00:51:14.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5939600640 1 -- 172.21.15.66:0/1745976889 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934012600 con 0x7f593c160bb0 2024-03-30T00:51:14.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5944000640 1 -- 172.21.15.66:0/1745976889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593c0b3790 con 0x7f593c160bb0 2024-03-30T00:51:14.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5944000640 1 -- 172.21.15.66:0/1745976889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593c0b3d50 con 0x7f593c160bb0 2024-03-30T00:51:14.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.955+0000 7f5944000640 1 -- 172.21.15.66:0/1745976889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593c0b3f40 con 0x7f593c160bb0 2024-03-30T00:51:14.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.959+0000 7f5939600640 1 -- 172.21.15.66:0/1745976889 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5934007070 con 0x7f593c160bb0 2024-03-30T00:51:14.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.959+0000 7f5939600640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f591c066690 0x7f591c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:14.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.959+0000 7f5939600640 1 -- 172.21.15.66:0/1745976889 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f59340841d0 con 0x7f593c160bb0 2024-03-30T00:51:14.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.959+0000 7f5941600640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f591c066690 0x7f591c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:14.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.959+0000 7f5941600640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f591c066690 0x7f591c068b50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f59300ae000 tx=0x7f59300a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:14.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:14.963+0000 7f5939600640 1 -- 172.21.15.66:0/1745976889 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5934051410 con 0x7f593c160bb0 2024-03-30T00:51:15.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.159+0000 7f5944000640 1 -- 172.21.15.66:0/1745976889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f593c066c30 con 0x7f593c160bb0 2024-03-30T00:51:15.168 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:51:15.159+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:51:15.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.167+0000 7f5939600640 1 -- 172.21.15.66:0/1745976889 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v96) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5934054a30 con 0x7f593c160bb0 2024-03-30T00:51:15.176 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:51:15.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 -- 172.21.15.66:0/1745976889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f591c066690 msgr2=0x7f591c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f591c066690 0x7f591c068b50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f59300ae000 tx=0x7f59300a7000 comp rx=0 tx=0).stop 2024-03-30T00:51:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 -- 172.21.15.66:0/1745976889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593c160bb0 msgr2=0x7f593c151fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593c160bb0 0x7f593c151fb0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f593400caa0 tx=0x7f593400cf60 comp rx=0 tx=0).stop 2024-03-30T00:51:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 -- 172.21.15.66:0/1745976889 shutdown_connections 2024-03-30T00:51:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f591c066690 0x7f591c068b50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f593c160bb0 0x7f593c151fb0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f593c15cb60 0x7f593c151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 --2- 172.21.15.66:0/1745976889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f593c15c780 0x7f593c0c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 -- 172.21.15.66:0/1745976889 >> 172.21.15.66:0/1745976889 conn(0x7f593c09ba30 msgr2=0x7f593c089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 -- 172.21.15.66:0/1745976889 shutdown_connections 2024-03-30T00:51:15.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.171+0000 7f5944000640 1 -- 172.21.15.66:0/1745976889 wait complete. 2024-03-30T00:51:15.188 INFO:tasks.ceph.mds.a.smithi066.stderr: -7933> 2024-03-30T00:50:43.699+0000 7f5857000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:51:15.200 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T00:51:15.203 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T00:51:15.241 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:51:15.245 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-03-30T00:51:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-03-30T00:51:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T00:51:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-03-30T00:51:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-03-30T00:51:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T00:51:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-03-30T00:51:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-03-30T00:51:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-03-30T00:51:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 20 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 20 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 10 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: mon op complaint time = 120 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 600 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-03-30T00:51:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds = 20 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds balancer = 20 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug frag = True 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug scatterstat = True 2024-03-30T00:51:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: mds op complaint time = 180 2024-03-30T00:51:15.256 INFO:teuthology.orchestra.run.smithi066.stdout: mds verify scatter = False 2024-03-30T00:51:15.256 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T00:51:15.256 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T00:51:15.256 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T00:51:15.256 INFO:teuthology.orchestra.run.smithi156.stdout:[global] 2024-03-30T00:51:15.256 INFO:teuthology.orchestra.run.smithi156.stdout: chdir = "" 2024-03-30T00:51:15.256 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T00:51:15.256 INFO:teuthology.orchestra.run.smithi156.stdout: auth supported = cephx 2024-03-30T00:51:15.256 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: filestore xattr use omap = true 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: mon clock drift allowed = 1.000 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: osd crush chooseleaf type = 0 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: auth debug = true 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on old message = true 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on bug = true 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: mon pg warn max object skew = 0 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.257 INFO:teuthology.orchestra.run.smithi156.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default size = 2 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow primary affinity = true 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow pg remap = true 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on legacy crush tunables = false 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on crush straw calc version zero = false 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on no sortbitwise = false 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on osd down out interval zero = false 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on too few osds = false 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T00:51:15.258 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: mon_allow_pool_size_one = true 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: osd default data pool replay window = 5 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: mon allow pool delete = true 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: mon cluster log file level = debug 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: debug asserts on shutdown = true 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: mon health detail to clog = false 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: mon host = "172.21.15.66,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: fsid = 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.259 INFO:teuthology.orchestra.run.smithi156.stdout:[osd] 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: osd journal size = 100 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub load threshold = 5.0 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub max interval = 600 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: osd mclock profile = high_recovery_ops 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: osd recover clone overlap = true 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: osd recovery max chunk = 1048576 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug shutdown = true 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug op order = true 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug verify stray on activate = true 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: osd open classes on start = true 2024-03-30T00:51:15.260 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug pg log writeout = true 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: osd deep scrub update digest min age = 30 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: osd map max advance = 10 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: journal zero on create = true 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: filestore ondisk finisher threads = 3 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: filestore apply finisher threads = 3 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: bdev debug aio = true 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug misdirected ops = true 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore block size = 96636764160 2024-03-30T00:51:15.261 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore fsck on mount = True 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluefs = 20 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluestore = 20 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: debug osd = 20 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: debug rocksdb = 10 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd full ratio = 0.9 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd nearfull ratio = 0.8 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: osd failsafe full ratio = 0.95 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: osd objectstore = bluestore 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr] 2024-03-30T00:51:15.262 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgr = 20 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: mgr/telemetry/nag = false 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout:[mon] 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: debug paxos = 20 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: mon data avail warn = 5 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: mon mgr mkfs grace = 240 2024-03-30T00:51:15.263 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd reporter subtree level = osd 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd prime pg temp = true 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.264 INFO:teuthology.orchestra.run.smithi156.stdout: # 1m isn't quite enough 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: mon_down_mkfs_grace = 2m 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_filestore_osds = false 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: mon op complaint time = 120 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout:[client] 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: rgw cache enabled = true 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable ops log = true 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable usage log = true 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: client mount timeout = 600 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2024-03-30T00:51:15.265 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = "" 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.a] 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.b] 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.c] 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout:[mds] 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds = 20 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds balancer = 20 2024-03-30T00:51:15.266 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:15.267 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug frag = True 2024-03-30T00:51:15.267 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug scatterstat = True 2024-03-30T00:51:15.267 INFO:teuthology.orchestra.run.smithi156.stdout: mds op complaint time = 180 2024-03-30T00:51:15.267 INFO:teuthology.orchestra.run.smithi156.stdout: mds verify scatter = False 2024-03-30T00:51:15.267 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2024-03-30T00:51:15.267 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2024-03-30T00:51:15.267 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2024-03-30T00:51:15.269 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T00:51:15.312 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T00:51:15.351 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-03-30T00:51:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-03-30T00:51:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-03-30T00:51:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-03-30T00:51:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 20 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 20 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 10 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-03-30T00:51:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T00:51:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-03-30T00:51:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T00:51:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-03-30T00:51:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-03-30T00:51:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-03-30T00:51:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-03-30T00:51:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: mon op complaint time = 120 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:15.360 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 600 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-03-30T00:51:15.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds = 20 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds balancer = 20 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug frag = True 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug scatterstat = False 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: mds op complaint time = 180 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: mds verify scatter = False 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T00:51:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T00:51:15.363 INFO:teuthology.orchestra.run.smithi156.stdout:[global] 2024-03-30T00:51:15.363 INFO:teuthology.orchestra.run.smithi156.stdout: chdir = "" 2024-03-30T00:51:15.363 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T00:51:15.363 INFO:teuthology.orchestra.run.smithi156.stdout: auth supported = cephx 2024-03-30T00:51:15.363 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.363 INFO:teuthology.orchestra.run.smithi156.stdout: filestore xattr use omap = true 2024-03-30T00:51:15.363 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.363 INFO:teuthology.orchestra.run.smithi156.stdout: mon clock drift allowed = 1.000 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: osd crush chooseleaf type = 0 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: auth debug = true 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on old message = true 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on bug = true 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: mon pg warn max object skew = 0 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default size = 2 2024-03-30T00:51:15.364 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow primary affinity = true 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow pg remap = true 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on legacy crush tunables = false 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on crush straw calc version zero = false 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on no sortbitwise = false 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on osd down out interval zero = false 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on too few osds = false 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: mon_allow_pool_size_one = true 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: osd default data pool replay window = 5 2024-03-30T00:51:15.365 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: mon allow pool delete = true 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: mon cluster log file level = debug 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: debug asserts on shutdown = true 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: mon health detail to clog = false 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: mon host = "172.21.15.66,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: fsid = 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout:[osd] 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: osd journal size = 100 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub load threshold = 5.0 2024-03-30T00:51:15.366 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub max interval = 600 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: osd mclock profile = high_recovery_ops 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: osd recover clone overlap = true 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: osd recovery max chunk = 1048576 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug shutdown = true 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug op order = true 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug verify stray on activate = true 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: osd open classes on start = true 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug pg log writeout = true 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: osd deep scrub update digest min age = 30 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.367 INFO:teuthology.orchestra.run.smithi156.stdout: osd map max advance = 10 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: journal zero on create = true 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: filestore ondisk finisher threads = 3 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: filestore apply finisher threads = 3 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: bdev debug aio = true 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug misdirected ops = true 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore block size = 96636764160 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore fsck on mount = True 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluefs = 20 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluestore = 20 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:15.368 INFO:teuthology.orchestra.run.smithi156.stdout: debug osd = 20 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: debug rocksdb = 10 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd full ratio = 0.9 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd nearfull ratio = 0.8 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: osd failsafe full ratio = 0.95 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: osd objectstore = bluestore 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr] 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgr = 20 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2024-03-30T00:51:15.369 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: mgr/telemetry/nag = false 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout:[mon] 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: debug paxos = 20 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: mon data avail warn = 5 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: mon mgr mkfs grace = 240 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd reporter subtree level = osd 2024-03-30T00:51:15.370 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd prime pg temp = true 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: # 1m isn't quite enough 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: mon_down_mkfs_grace = 2m 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_filestore_osds = false 2024-03-30T00:51:15.371 INFO:teuthology.orchestra.run.smithi156.stdout: mon op complaint time = 120 2024-03-30T00:51:15.372 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:15.372 INFO:teuthology.orchestra.run.smithi156.stdout:[client] 2024-03-30T00:51:15.372 INFO:teuthology.orchestra.run.smithi156.stdout: rgw cache enabled = true 2024-03-30T00:51:15.372 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable ops log = true 2024-03-30T00:51:15.372 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable usage log = true 2024-03-30T00:51:15.372 INFO:teuthology.orchestra.run.smithi156.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T00:51:15.372 INFO:teuthology.orchestra.run.smithi156.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T00:51:15.372 INFO:teuthology.orchestra.run.smithi156.stdout: client mount timeout = 600 2024-03-30T00:51:15.372 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2024-03-30T00:51:15.372 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:15.372 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = "" 2024-03-30T00:51:15.373 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2024-03-30T00:51:15.373 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2024-03-30T00:51:15.373 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.a] 2024-03-30T00:51:15.373 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.b] 2024-03-30T00:51:15.373 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.c] 2024-03-30T00:51:15.373 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout:[mds] 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds = 20 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds balancer = 20 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug frag = True 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug scatterstat = False 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout: mds op complaint time = 180 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout: mds verify scatter = False 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2024-03-30T00:51:15.374 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2024-03-30T00:51:15.376 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-03-30T00:51:15.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 Processor -- start 2024-03-30T00:51:15.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 -- start start 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e2d0b20 0x55c67e2a0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c67e2e9bf0 0x55c67e409120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c67e35f040 0x55c67e409660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67e1e8960 con 0x55c67e2d0b20 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67e1e8600 con 0x55c67e2e9bf0 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c67e1df690 con 0x55c67e35f040 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f3855800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e2d0b20 0x55c67e2a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f3855800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e2d0b20 0x55c67e2a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32896/0 (socket says 172.21.15.66:32896) 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f3855800640 1 -- 172.21.15.66:0/1508613119 learned_addr learned my addr 172.21.15.66:0/1508613119 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f3855800640 1 -- 172.21.15.66:0/1508613119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c67e35f040 msgr2=0x55c67e409660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f3855800640 1 --2- 172.21.15.66:0/1508613119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c67e35f040 0x55c67e409660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f3855800640 1 -- 172.21.15.66:0/1508613119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c67e2e9bf0 msgr2=0x55c67e409120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f3855800640 1 --2- 172.21.15.66:0/1508613119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c67e2e9bf0 0x55c67e409120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f3855800640 1 -- 172.21.15.66:0/1508613119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385000c3b0 con 0x55c67e2d0b20 2024-03-30T00:51:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f3855800640 1 --2- 172.21.15.66:0/1508613119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e2d0b20 0x55c67e2a0930 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f38480097d0 tx=0x7f3848005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a87b52a7d088bd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f384f400640 1 -- 172.21.15.66:0/1508613119 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384800f040 con 0x55c67e2d0b20 2024-03-30T00:51:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f384f400640 1 -- 172.21.15.66:0/1508613119 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3848008d40 con 0x55c67e2d0b20 2024-03-30T00:51:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f384f400640 1 -- 172.21.15.66:0/1508613119 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848005e20 con 0x55c67e2d0b20 2024-03-30T00:51:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 -- 172.21.15.66:0/1508613119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e2d0b20 msgr2=0x55c67e2a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 --2- 172.21.15.66:0/1508613119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e2d0b20 0x55c67e2a0930 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f38480097d0 tx=0x7f3848005a10 comp rx=0 tx=0).stop 2024-03-30T00:51:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 -- 172.21.15.66:0/1508613119 shutdown_connections 2024-03-30T00:51:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 --2- 172.21.15.66:0/1508613119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c67e35f040 0x55c67e409660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 --2- 172.21.15.66:0/1508613119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c67e2e9bf0 0x55c67e409120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 --2- 172.21.15.66:0/1508613119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e2d0b20 0x55c67e2a0930 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 -- 172.21.15.66:0/1508613119 >> 172.21.15.66:0/1508613119 conn(0x55c67e2a31a0 msgr2=0x55c67e2a3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:15.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 -- 172.21.15.66:0/1508613119 shutdown_connections 2024-03-30T00:51:15.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.455+0000 7f38585c4080 1 -- 172.21.15.66:0/1508613119 wait complete. 2024-03-30T00:51:15.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f38585c4080 1 Processor -- start 2024-03-30T00:51:15.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f38585c4080 1 -- start start 2024-03-30T00:51:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f38585c4080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c67e2e9bf0 0x55c67e374e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f38585c4080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e35f040 0x55c67e37bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f38585c4080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c67e375360 0x55c67e37c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f38585c4080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67e37dac0 con 0x55c67e35f040 2024-03-30T00:51:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f38585c4080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67e2e64e0 con 0x55c67e375360 2024-03-30T00:51:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f38585c4080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c67e2e5a60 con 0x55c67e2e9bf0 2024-03-30T00:51:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f3854e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e35f040 0x55c67e37bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f3854e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e35f040 0x55c67e37bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32898/0 (socket says 172.21.15.66:32898) 2024-03-30T00:51:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f3854e00640 1 -- 172.21.15.66:0/2777975797 learned_addr learned my addr 172.21.15.66:0/2777975797 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f3855800640 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c67e2e9bf0 0x55c67e374e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f3856200640 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c67e375360 0x55c67e37c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f3854e00640 1 -- 172.21.15.66:0/2777975797 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c67e2e9bf0 msgr2=0x55c67e374e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f3854e00640 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c67e2e9bf0 0x55c67e374e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f3854e00640 1 -- 172.21.15.66:0/2777975797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c67e375360 msgr2=0x55c67e37c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f3854e00640 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c67e375360 0x55c67e37c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f3854e00640 1 -- 172.21.15.66:0/2777975797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848009420 con 0x55c67e35f040 2024-03-30T00:51:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f3854e00640 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e35f040 0x55c67e37bc90 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f384400be10 tx=0x7f3844009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f384e000640 1 -- 172.21.15.66:0/2777975797 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384400e480 con 0x55c67e35f040 2024-03-30T00:51:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f384e000640 1 -- 172.21.15.66:0/2777975797 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384400e5e0 con 0x55c67e35f040 2024-03-30T00:51:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f384e000640 1 -- 172.21.15.66:0/2777975797 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384400e990 con 0x55c67e35f040 2024-03-30T00:51:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f38585c4080 1 -- 172.21.15.66:0/2777975797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c67e1f78c0 con 0x55c67e35f040 2024-03-30T00:51:15.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.459+0000 7f38585c4080 1 -- 172.21.15.66:0/2777975797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c67e1ead00 con 0x55c67e35f040 2024-03-30T00:51:15.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f384e000640 1 -- 172.21.15.66:0/2777975797 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f384400a040 con 0x55c67e35f040 2024-03-30T00:51:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f384e000640 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f38380650a0 0x7f3838067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f384e000640 1 -- 172.21.15.66:0/2777975797 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f3844082f00 con 0x55c67e35f040 2024-03-30T00:51:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f3855800640 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f38380650a0 0x7f3838067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f38585c4080 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55c67e37e8b0 0x55c67e37ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f38585c4080 1 -- 172.21.15.66:0/2777975797 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x55c67e2e3fb0 con 0x55c67e37e8b0 2024-03-30T00:51:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f3856200640 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55c67e37e8b0 0x55c67e37ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f3855800640 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f38380650a0 0x7f3838067560 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55c67e2eb180 tx=0x7f3848011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:15.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f3856200640 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55c67e37e8b0 0x55c67e37ecc0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f3856200640 1 -- 172.21.15.66:0/2777975797 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v126'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f38500ab070 con 0x55c67e37e8b0 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f38585c4080 1 -- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55c67e37e8b0 msgr2=0x55c67e37ecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f38585c4080 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x55c67e37e8b0 0x55c67e37ecc0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f38585c4080 1 -- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f38380650a0 msgr2=0x7f3838067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f38585c4080 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f38380650a0 0x7f3838067560 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55c67e2eb180 tx=0x7f3848011000 comp rx=0 tx=0).stop 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f38585c4080 1 -- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e35f040 msgr2=0x55c67e37bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f38585c4080 1 --2- 172.21.15.66:0/2777975797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c67e35f040 0x55c67e37bc90 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f384400be10 tx=0x7f3844009650 comp rx=0 tx=0).stop 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f3856200640 1 -- 172.21.15.66:0/2777975797 reap_dead start 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f38585c4080 1 -- 172.21.15.66:0/2777975797 shutdown_connections 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.463+0000 7f38585c4080 1 -- 172.21.15.66:0/2777975797 >> 172.21.15.66:0/2777975797 conn(0x55c67e2a31a0 msgr2=0x55c67e2e36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.467+0000 7f38585c4080 1 -- 172.21.15.66:0/2777975797 shutdown_connections 2024-03-30T00:51:15.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.467+0000 7f38585c4080 1 -- 172.21.15.66:0/2777975797 wait complete. 2024-03-30T00:51:15.478 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-30T00:51:15.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.611+0000 7f6b20a00640 1 Processor -- start 2024-03-30T00:51:15.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.611+0000 7f6b20a00640 1 -- start start 2024-03-30T00:51:15.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b20a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6b1c15cb20 0x7f6b1c06c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b20a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b1c0b0450 0x7f6b1c0d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b20a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1c0b0890 0x7f6b1c0d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b20a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c15cf00 con 0x7f6b1c15cb20 2024-03-30T00:51:15.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b20a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c0ce890 con 0x7f6b1c0b0450 2024-03-30T00:51:15.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b20a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b0da0 con 0x7f6b1c0b0890 2024-03-30T00:51:15.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b19e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6b1c15cb20 0x7f6b1c06c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b19e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6b1c15cb20 0x7f6b1c06c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32904/0 (socket says 172.21.15.66:32904) 2024-03-30T00:51:15.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b19e00640 1 -- 172.21.15.66:0/1431485960 learned_addr learned my addr 172.21.15.66:0/1431485960 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:15.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b13e00640 1 --2- 172.21.15.66:0/1431485960 >> v2:172.21.15.156:3300/0 conn(0x7f6b1c0b0450 0x7f6b1c0d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b1a800640 1 --2- 172.21.15.66:0/1431485960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1c0b0890 0x7f6b1c0d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b19e00640 1 -- 172.21.15.66:0/1431485960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1c0b0890 msgr2=0x7f6b1c0d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:15.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b19e00640 1 --2- 172.21.15.66:0/1431485960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1c0b0890 0x7f6b1c0d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b19e00640 1 -- 172.21.15.66:0/1431485960 >> v2:172.21.15.156:3300/0 conn(0x7f6b1c0b0450 msgr2=0x7f6b1c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:15.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b19e00640 1 --2- 172.21.15.66:0/1431485960 >> v2:172.21.15.156:3300/0 conn(0x7f6b1c0b0450 0x7f6b1c0d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.615+0000 7f6b19e00640 1 -- 172.21.15.66:0/1431485960 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0b1080 con 0x7f6b1c15cb20 2024-03-30T00:51:15.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b19e00640 1 --2- 172.21.15.66:0/1431485960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b1c15cb20 0x7f6b1c06c020 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0b1f10 tx=0x7f6b0c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2929ee46db0da166 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:15.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b19400640 1 -- 172.21.15.66:0/1431485960 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0aa610 con 0x7f6b1c15cb20 2024-03-30T00:51:15.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b19400640 1 -- 172.21.15.66:0/1431485960 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c0a6040 con 0x7f6b1c15cb20 2024-03-30T00:51:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b19400640 1 -- 172.21.15.66:0/1431485960 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0aa990 con 0x7f6b1c15cb20 2024-03-30T00:51:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 -- 172.21.15.66:0/1431485960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b1c15cb20 msgr2=0x7f6b1c06c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 --2- 172.21.15.66:0/1431485960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b1c15cb20 0x7f6b1c06c020 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0b1f10 tx=0x7f6b0c01c4f0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 -- 172.21.15.66:0/1431485960 shutdown_connections 2024-03-30T00:51:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 --2- 172.21.15.66:0/1431485960 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1c0b0890 0x7f6b1c0d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 --2- 172.21.15.66:0/1431485960 >> v2:172.21.15.156:3300/0 conn(0x7f6b1c0b0450 0x7f6b1c0d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 --2- 172.21.15.66:0/1431485960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b1c15cb20 0x7f6b1c06c020 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 -- 172.21.15.66:0/1431485960 >> 172.21.15.66:0/1431485960 conn(0x7f6b1c089fc0 msgr2=0x7f6b1c08a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 -- 172.21.15.66:0/1431485960 shutdown_connections 2024-03-30T00:51:15.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 -- 172.21.15.66:0/1431485960 wait complete. 2024-03-30T00:51:15.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 Processor -- start 2024-03-30T00:51:15.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 -- start start 2024-03-30T00:51:15.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1c0b0450 0x7f6b1c14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b1c0b0890 0x7f6b1c15f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b1c15cb20 0x7f6b1c15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1559c0 con 0x7f6b1c15cb20 2024-03-30T00:51:15.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c155c10 con 0x7f6b1c0b0890 2024-03-30T00:51:15.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.619+0000 7f6b20a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c155ef0 con 0x7f6b1c0b0450 2024-03-30T00:51:15.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b1a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b1c15cb20 0x7f6b1c15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b1a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b1c15cb20 0x7f6b1c15bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32908/0 (socket says 172.21.15.66:32908) 2024-03-30T00:51:15.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b1a800640 1 -- 172.21.15.66:0/3627930138 learned_addr learned my addr 172.21.15.66:0/3627930138 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:15.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b19e00640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1c0b0450 0x7f6b1c14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b13e00640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b1c0b0890 0x7f6b1c15f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b1a800640 1 -- 172.21.15.66:0/3627930138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1c0b0450 msgr2=0x7f6b1c14a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:15.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b1a800640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1c0b0450 0x7f6b1c14a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b1a800640 1 -- 172.21.15.66:0/3627930138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b1c0b0890 msgr2=0x7f6b1c15f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:15.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b1a800640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b1c0b0890 0x7f6b1c15f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:15.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b1a800640 1 -- 172.21.15.66:0/3627930138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c0094d0 con 0x7f6b1c15cb20 2024-03-30T00:51:15.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b1a800640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b1c15cb20 0x7f6b1c15bcc0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400efc0 tx=0x7f6b1400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:15.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b12000640 1 -- 172.21.15.66:0/3627930138 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14009ba0 con 0x7f6b1c15cb20 2024-03-30T00:51:15.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b12000640 1 -- 172.21.15.66:0/3627930138 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1400f040 con 0x7f6b1c15cb20 2024-03-30T00:51:15.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b12000640 1 -- 172.21.15.66:0/3627930138 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b140146c0 con 0x7f6b1c15cb20 2024-03-30T00:51:15.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b20a00640 1 -- 172.21.15.66:0/3627930138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c15f130 con 0x7f6b1c15cb20 2024-03-30T00:51:15.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b20a00640 1 -- 172.21.15.66:0/3627930138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c15fe70 con 0x7f6b1c15cb20 2024-03-30T00:51:15.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b20a00640 1 -- 172.21.15.66:0/3627930138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c15f650 con 0x7f6b1c15cb20 2024-03-30T00:51:15.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b12000640 1 -- 172.21.15.66:0/3627930138 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6b14014820 con 0x7f6b1c15cb20 2024-03-30T00:51:15.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.623+0000 7f6b12000640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6af8066690 0x7f6af8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:15.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.627+0000 7f6b12000640 1 -- 172.21.15.66:0/3627930138 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f6b14084ca0 con 0x7f6b1c15cb20 2024-03-30T00:51:15.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.627+0000 7f6b19e00640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6af8066690 0x7f6af8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:15.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.627+0000 7f6b19e00640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6af8066690 0x7f6af8068b50 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0ae000 tx=0x7f6b0c0b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:15.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.631+0000 7f6b12000640 1 -- 172.21.15.66:0/3627930138 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6b1c15f650 con 0x7f6b1c15cb20 2024-03-30T00:51:15.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:15.827+0000 7f6b20a00640 1 -- 172.21.15.66:0/3627930138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f6b1c15f650 con 0x7f6b1c15cb20 2024-03-30T00:51:16.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.171+0000 7f6b12000640 1 -- 172.21.15.66:0/3627930138 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v97) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6b1c15f650 con 0x7f6b1c15cb20 2024-03-30T00:51:16.179 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-30T00:51:16.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.171+0000 7f6b20a00640 1 -- 172.21.15.66:0/3627930138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6af8066690 msgr2=0x7f6af8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:16.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.171+0000 7f6b20a00640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6af8066690 0x7f6af8068b50 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0ae000 tx=0x7f6b0c0b5040 comp rx=0 tx=0).stop 2024-03-30T00:51:16.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.171+0000 7f6b20a00640 1 -- 172.21.15.66:0/3627930138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b1c15cb20 msgr2=0x7f6b1c15bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:16.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.171+0000 7f6b20a00640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b1c15cb20 0x7f6b1c15bcc0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400efc0 tx=0x7f6b1400c490 comp rx=0 tx=0).stop 2024-03-30T00:51:16.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.171+0000 7f6b20a00640 1 -- 172.21.15.66:0/3627930138 shutdown_connections 2024-03-30T00:51:16.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.171+0000 7f6b20a00640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6af8066690 0x7f6af8068b50 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.171+0000 7f6b20a00640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b1c15cb20 0x7f6b1c15bcc0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.171+0000 7f6b20a00640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b1c0b0890 0x7f6b1c15f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.171+0000 7f6b20a00640 1 --2- 172.21.15.66:0/3627930138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b1c0b0450 0x7f6b1c14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.171+0000 7f6b20a00640 1 -- 172.21.15.66:0/3627930138 >> 172.21.15.66:0/3627930138 conn(0x7f6b1c089fc0 msgr2=0x7f6b1c0af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:16.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.175+0000 7f6b20a00640 1 -- 172.21.15.66:0/3627930138 shutdown_connections 2024-03-30T00:51:16.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.175+0000 7f6b20a00640 1 -- 172.21.15.66:0/3627930138 wait complete. 2024-03-30T00:51:16.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:51:16.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf45c00640 1 Processor -- start 2024-03-30T00:51:16.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf45c00640 1 -- start start 2024-03-30T00:51:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf45c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcf4015cc00 0x7fcf400cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf45c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf400cb1d0 con 0x7fcf400ccd10 2024-03-30T00:51:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf45c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf400cfcb0 con 0x7fcf4006c100 2024-03-30T00:51:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf45c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf400cab80 con 0x7fcf4015cc00 2024-03-30T00:51:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3ea00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fcf400ccd10 0x7fcf400cd0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46498/0 (socket says 172.21.15.66:46498) 2024-03-30T00:51:16.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3ea00640 1 -- 172.21.15.66:0/745361651 learned_addr learned my addr 172.21.15.66:0/745361651 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:16.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf37e00640 1 --2- 172.21.15.66:0/745361651 >> v2:172.21.15.156:3300/0 conn(0x7fcf4015cc00 0x7fcf400cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:16.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819641398 0 0) 0x7fcf400cb1d0 con 0x7fcf400ccd10 2024-03-30T00:51:16.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf24003530 con 0x7fcf400ccd10 2024-03-30T00:51:16.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1699396127 0 0) 0x7fcf24003530 con 0x7fcf400ccd10 2024-03-30T00:51:16.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 >> v2:172.21.15.156:3300/0 conn(0x7fcf4015cc00 msgr2=0x7fcf400cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:16.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 --2- 172.21.15.66:0/745361651 >> v2:172.21.15.156:3300/0 conn(0x7fcf4015cc00 0x7fcf400cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 >> v1:172.21.15.156:6789/0 conn(0x7fcf4006c100 legacy=0x7fcf400cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:16.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf400cae60 con 0x7fcf400ccd10 2024-03-30T00:51:16.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fcf30001680 con 0x7fcf400ccd10 2024-03-30T00:51:16.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 >> v1:172.21.15.66:6789/0 conn(0x7fcf400ccd10 legacy=0x7fcf400cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcf240039d0 0x7fcf24003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 0x7fcf24004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf24005190 0x7fcf2400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400cab80 con 0x7fcf240047d0 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2400c090 con 0x7fcf24005190 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.339+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2400c250 con 0x7fcf240039d0 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf37e00640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 0x7fcf24004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf3ea00640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcf240039d0 0x7fcf24003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf3f400640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf24005190 0x7fcf2400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf37e00640 1 -- 172.21.15.66:0/745361651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcf240039d0 msgr2=0x7fcf24003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf37e00640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcf240039d0 0x7fcf24003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf37e00640 1 -- 172.21.15.66:0/745361651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf24005190 msgr2=0x7fcf2400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:16.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf37e00640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf24005190 0x7fcf2400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf3f400640 1 -- 172.21.15.66:0/745361651 reap_dead start 2024-03-30T00:51:16.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf37e00640 1 -- 172.21.15.66:0/745361651 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf24003530 con 0x7fcf240047d0 2024-03-30T00:51:16.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf3ea00640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcf240039d0 0x7fcf24003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:16.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf37e00640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 0x7fcf24004c50 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009ae0 tx=0x7fcf2801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10771ea8371f271a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:16.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280a5550 con 0x7fcf240047d0 2024-03-30T00:51:16.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf280a56b0 con 0x7fcf240047d0 2024-03-30T00:51:16.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf3e000640 1 -- 172.21.15.66:0/745361651 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280a5a60 con 0x7fcf240047d0 2024-03-30T00:51:16.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf45c00640 1 -- 172.21.15.66:0/745361651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 msgr2=0x7fcf24004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:16.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.343+0000 7fcf45c00640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 0x7fcf24004c50 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009ae0 tx=0x7fcf2801ca70 comp rx=0 tx=0).stop 2024-03-30T00:51:16.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 -- 172.21.15.66:0/745361651 shutdown_connections 2024-03-30T00:51:16.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 --2- 172.21.15.66:0/745361651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 0x7fcf24004c50 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 -- 172.21.15.66:0/745361651 >> 172.21.15.66:0/745361651 conn(0x7fcf400d0490 msgr2=0x7fcf400d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:16.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 -- 172.21.15.66:0/745361651 shutdown_connections 2024-03-30T00:51:16.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 -- 172.21.15.66:0/745361651 wait complete. 2024-03-30T00:51:16.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 Processor -- start 2024-03-30T00:51:16.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 -- start start 2024-03-30T00:51:16.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 0x7fcf401551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:16.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf4006f250 0x7fcf4015f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:16.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcf4015f8c0 0x7fcf401f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:16.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400b6170 con 0x7fcf240047d0 2024-03-30T00:51:16.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf401556f0 con 0x7fcf4006f250 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf45c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400b5e90 con 0x7fcf4015f8c0 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf3ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 0x7fcf401551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf3ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 0x7fcf401551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32914/0 (socket says 172.21.15.66:32914) 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf3ea00640 1 -- 172.21.15.66:0/2477479137 learned_addr learned my addr 172.21.15.66:0/2477479137 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf3f400640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcf4015f8c0 0x7fcf401f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf37e00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf4006f250 0x7fcf4015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf3ea00640 1 -- 172.21.15.66:0/2477479137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcf4015f8c0 msgr2=0x7fcf401f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf3ea00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcf4015f8c0 0x7fcf401f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf3ea00640 1 -- 172.21.15.66:0/2477479137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf4006f250 msgr2=0x7fcf4015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf3ea00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf4006f250 0x7fcf4015f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf3ea00640 1 -- 172.21.15.66:0/2477479137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4015d800 con 0x7fcf240047d0 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf3ea00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 0x7fcf401551b0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fcf30004880 tx=0x7fcf30011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf36a00640 1 -- 172.21.15.66:0/2477479137 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000fb00 con 0x7fcf240047d0 2024-03-30T00:51:16.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.347+0000 7fcf36a00640 1 -- 172.21.15.66:0/2477479137 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf30011e90 con 0x7fcf240047d0 2024-03-30T00:51:16.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.351+0000 7fcf36a00640 1 -- 172.21.15.66:0/2477479137 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000fe60 con 0x7fcf240047d0 2024-03-30T00:51:16.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.351+0000 7fcf45c00640 1 -- 172.21.15.66:0/2477479137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4015dae0 con 0x7fcf240047d0 2024-03-30T00:51:16.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.351+0000 7fcf45c00640 1 -- 172.21.15.66:0/2477479137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4015fd40 con 0x7fcf240047d0 2024-03-30T00:51:16.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.351+0000 7fcf45c00640 1 -- 172.21.15.66:0/2477479137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf400b30b0 con 0x7fcf240047d0 2024-03-30T00:51:16.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.351+0000 7fcf36a00640 1 -- 172.21.15.66:0/2477479137 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcf3001f460 con 0x7fcf240047d0 2024-03-30T00:51:16.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.351+0000 7fcf36a00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcf1c0666c0 0x7fcf1c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:16.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.351+0000 7fcf36a00640 1 -- 172.21.15.66:0/2477479137 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fcf30087460 con 0x7fcf240047d0 2024-03-30T00:51:16.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.351+0000 7fcf37e00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcf1c0666c0 0x7fcf1c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:16.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.351+0000 7fcf37e00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcf1c0666c0 0x7fcf1c068b80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcf280af000 tx=0x7fcf280046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:16.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.355+0000 7fcf36a00640 1 -- 172.21.15.66:0/2477479137 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcf400b30b0 con 0x7fcf240047d0 2024-03-30T00:51:16.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf45c00640 1 -- 172.21.15.66:0/2477479137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf400b30b0 con 0x7fcf240047d0 2024-03-30T00:51:16.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf36a00640 1 -- 172.21.15.66:0/2477479137 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4741 (secure 0 0 0) 0x7fcf400b30b0 con 0x7fcf240047d0 2024-03-30T00:51:16.560 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:16.560 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:51:16.173431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9732":{"gid":9732,"name":"a","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.66:6836/3460522956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3460522956},{"type":"v1","addr":"172.21.15.66:6836","nonce":3460522956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:51:16.560 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 98 2024-03-30T00:51:16.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf45c00640 1 -- 172.21.15.66:0/2477479137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcf1c0666c0 msgr2=0x7fcf1c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:16.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf45c00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcf1c0666c0 0x7fcf1c068b80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcf280af000 tx=0x7fcf280046d0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf45c00640 1 -- 172.21.15.66:0/2477479137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 msgr2=0x7fcf401551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:16.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf45c00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 0x7fcf401551b0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fcf30004880 tx=0x7fcf30011ca0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf45c00640 1 -- 172.21.15.66:0/2477479137 shutdown_connections 2024-03-30T00:51:16.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf45c00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcf4015f8c0 0x7fcf401f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf45c00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf4006f250 0x7fcf4015f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf45c00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fcf1c0666c0 0x7fcf1c068b80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf45c00640 1 --2- 172.21.15.66:0/2477479137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf240047d0 0x7fcf401551b0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:16.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.551+0000 7fcf45c00640 1 -- 172.21.15.66:0/2477479137 >> 172.21.15.66:0/2477479137 conn(0x7fcf400d0490 msgr2=0x7fcf4006ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:16.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.555+0000 7fcf45c00640 1 -- 172.21.15.66:0/2477479137 shutdown_connections 2024-03-30T00:51:16.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:16.555+0000 7fcf45c00640 1 -- 172.21.15.66:0/2477479137 wait complete. 2024-03-30T00:51:16.983 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:50:38.040525+0000', 'modified': '2024-03-30T00:51:16.173431+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9732}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9732': {'gid': 9732, 'name': 'a', 'rank': 0, 'incarnation': 98, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.66:6836/3460522956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 3460522956}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 3460522956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:51:16.984 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9732:up:replay 2024-03-30T00:51:17.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:51:17.756 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:51:17.985 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:51:18.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.131+0000 7f5d94600640 1 Processor -- start 2024-03-30T00:51:18.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.131+0000 7f5d94600640 1 -- start start 2024-03-30T00:51:18.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.131+0000 7f5d94600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5d8c0b2c10 0x7f5d8c0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:18.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.131+0000 7f5d94600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5d8c0cb3a0 0x7f5d8c0b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:18.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.131+0000 7f5d94600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8c0ce010 con 0x7f5d8c15ed40 2024-03-30T00:51:18.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.131+0000 7f5d94600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8c0b2ff0 con 0x7f5d8c0cb3a0 2024-03-30T00:51:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.131+0000 7f5d94600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8c06cfc0 con 0x7f5d8c0b2c10 2024-03-30T00:51:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.131+0000 7f5d92600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5d8c0cb3a0 0x7f5d8c0b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.135+0000 7f5d92600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5d8c0cb3a0 0x7f5d8c0b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32928/0 (socket says 172.21.15.66:32928) 2024-03-30T00:51:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.135+0000 7f5d92600640 1 -- 172.21.15.66:0/3818326680 learned_addr learned my addr 172.21.15.66:0/3818326680 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.135+0000 7f5d91c00640 1 --2- 172.21.15.66:0/3818326680 >> v2:172.21.15.156:3300/0 conn(0x7f5d8c0b2c10 0x7f5d8c0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.135+0000 7f5d92600640 1 -- 172.21.15.66:0/3818326680 >> v2:172.21.15.156:3300/0 conn(0x7f5d8c0b2c10 msgr2=0x7f5d8c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:18.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.135+0000 7f5d92600640 1 --2- 172.21.15.66:0/3818326680 >> v2:172.21.15.156:3300/0 conn(0x7f5d8c0b2c10 0x7f5d8c0cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:18.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.135+0000 7f5d92600640 1 -- 172.21.15.66:0/3818326680 >> v1:172.21.15.156:6789/0 conn(0x7f5d8c15ed40 legacy=0x7f5d8c0cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:18.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.135+0000 7f5d92600640 1 -- 172.21.15.66:0/3818326680 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c06d260 con 0x7f5d8c0cb3a0 2024-03-30T00:51:18.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.135+0000 7f5d92600640 1 --2- 172.21.15.66:0/3818326680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d8c0cb3a0 0x7f5d8c0b10f0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0cc6c0 tx=0x7f5d8401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3d45ee8e825364 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:18.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.135+0000 7f5d8be00640 1 -- 172.21.15.66:0/3818326680 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840a64c0 con 0x7f5d8c0cb3a0 2024-03-30T00:51:18.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.135+0000 7f5d8be00640 1 -- 172.21.15.66:0/3818326680 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d840a6620 con 0x7f5d8c0cb3a0 2024-03-30T00:51:18.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.135+0000 7f5d8be00640 1 -- 172.21.15.66:0/3818326680 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840a69b0 con 0x7f5d8c0cb3a0 2024-03-30T00:51:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 -- 172.21.15.66:0/3818326680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d8c0cb3a0 msgr2=0x7f5d8c0b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 --2- 172.21.15.66:0/3818326680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d8c0cb3a0 0x7f5d8c0b10f0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0cc6c0 tx=0x7f5d8401c9c0 comp rx=0 tx=0).stop 2024-03-30T00:51:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 -- 172.21.15.66:0/3818326680 shutdown_connections 2024-03-30T00:51:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 --2- 172.21.15.66:0/3818326680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d8c0cb3a0 0x7f5d8c0b10f0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 --2- 172.21.15.66:0/3818326680 >> v2:172.21.15.156:3300/0 conn(0x7f5d8c0b2c10 0x7f5d8c0cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 -- 172.21.15.66:0/3818326680 >> 172.21.15.66:0/3818326680 conn(0x7f5d8c069cf0 msgr2=0x7f5d8c06a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 -- 172.21.15.66:0/3818326680 shutdown_connections 2024-03-30T00:51:18.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 -- 172.21.15.66:0/3818326680 wait complete. 2024-03-30T00:51:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 Processor -- start 2024-03-30T00:51:18.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 -- start start 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c0b2c10 0x7f5d8c15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d8c0cb3a0 0x7f5d8c0aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d8c15ed40 0x7f5d8c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c15a840 con 0x7f5d8c15ed40 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0af200 con 0x7f5d8c0b2c10 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d94600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0c5e20 con 0x7f5d8c0cb3a0 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d92600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d8c15ed40 0x7f5d8c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d91c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c0b2c10 0x7f5d8c15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d91c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c0b2c10 0x7f5d8c15a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:36782/0 (socket says 172.21.15.66:36782) 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d92600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d8c15ed40 0x7f5d8c15aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:32938/0 (socket says 172.21.15.66:32938) 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d91c00640 1 -- 172.21.15.66:0/3390107878 learned_addr learned my addr 172.21.15.66:0/3390107878 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:18.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d91200640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d8c0cb3a0 0x7f5d8c0aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d92600640 1 -- 172.21.15.66:0/3390107878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d8c0cb3a0 msgr2=0x7f5d8c0aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d92600640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d8c0cb3a0 0x7f5d8c0aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d92600640 1 -- 172.21.15.66:0/3390107878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c0b2c10 msgr2=0x7f5d8c15a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d92600640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c0b2c10 0x7f5d8c15a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:18.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d92600640 1 -- 172.21.15.66:0/3390107878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84009b30 con 0x7f5d8c15ed40 2024-03-30T00:51:18.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.139+0000 7f5d92600640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d8c15ed40 0x7f5d8c15aae0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f5d840ab000 tx=0x7f5d84019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:18.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.143+0000 7f5d89600640 1 -- 172.21.15.66:0/3390107878 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840b4070 con 0x7f5d8c15ed40 2024-03-30T00:51:18.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.143+0000 7f5d89600640 1 -- 172.21.15.66:0/3390107878 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84009150 con 0x7f5d8c15ed40 2024-03-30T00:51:18.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.143+0000 7f5d89600640 1 -- 172.21.15.66:0/3390107878 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840a69f0 con 0x7f5d8c15ed40 2024-03-30T00:51:18.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.143+0000 7f5d94600640 1 -- 172.21.15.66:0/3390107878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c0c60c0 con 0x7f5d8c15ed40 2024-03-30T00:51:18.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.143+0000 7f5d94600640 1 -- 172.21.15.66:0/3390107878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c0c6680 con 0x7f5d8c15ed40 2024-03-30T00:51:18.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.143+0000 7f5d94600640 1 -- 172.21.15.66:0/3390107878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8c0c6870 con 0x7f5d8c15ed40 2024-03-30T00:51:18.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.143+0000 7f5d89600640 1 -- 172.21.15.66:0/3390107878 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5d840b8070 con 0x7f5d8c15ed40 2024-03-30T00:51:18.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.143+0000 7f5d89600640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d6c066710 0x7f5d6c068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:18.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.143+0000 7f5d89600640 1 -- 172.21.15.66:0/3390107878 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5d840aa070 con 0x7f5d8c15ed40 2024-03-30T00:51:18.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.143+0000 7f5d91c00640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d6c066710 0x7f5d6c068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:18.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.147+0000 7f5d91c00640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d6c066710 0x7f5d6c068bd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c09c3e0 tx=0x7f5d80009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:18.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.147+0000 7f5d89600640 1 -- 172.21.15.66:0/3390107878 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5d8411fe60 con 0x7f5d8c15ed40 2024-03-30T00:51:18.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.347+0000 7f5d94600640 1 -- 172.21.15.66:0/3390107878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d8c066c40 con 0x7f5d8c15ed40 2024-03-30T00:51:18.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.347+0000 7f5d89600640 1 -- 172.21.15.66:0/3390107878 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4743 (secure 0 0 0) 0x7f5d840bf090 con 0x7f5d8c15ed40 2024-03-30T00:51:18.355 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:18.356 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:51:18.177265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9732":{"gid":9732,"name":"a","rank":0,"incarnation":98,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.66:6836/3460522956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3460522956},{"type":"v1","addr":"172.21.15.66:6836","nonce":3460522956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:51:18.356 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 100 2024-03-30T00:51:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.347+0000 7f5d94600640 1 -- 172.21.15.66:0/3390107878 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d6c066710 msgr2=0x7f5d6c068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.347+0000 7f5d94600640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d6c066710 0x7f5d6c068bd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c09c3e0 tx=0x7f5d80009190 comp rx=0 tx=0).stop 2024-03-30T00:51:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.347+0000 7f5d94600640 1 -- 172.21.15.66:0/3390107878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d8c15ed40 msgr2=0x7f5d8c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.347+0000 7f5d94600640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d8c15ed40 0x7f5d8c15aae0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f5d840ab000 tx=0x7f5d84019760 comp rx=0 tx=0).stop 2024-03-30T00:51:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.347+0000 7f5d94600640 1 -- 172.21.15.66:0/3390107878 shutdown_connections 2024-03-30T00:51:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.347+0000 7f5d94600640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5d6c066710 0x7f5d6c068bd0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.347+0000 7f5d94600640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d8c15ed40 0x7f5d8c15aae0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.347+0000 7f5d94600640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d8c0cb3a0 0x7f5d8c0aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.347+0000 7f5d94600640 1 --2- 172.21.15.66:0/3390107878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d8c0b2c10 0x7f5d8c15a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.351+0000 7f5d94600640 1 -- 172.21.15.66:0/3390107878 >> 172.21.15.66:0/3390107878 conn(0x7f5d8c069cf0 msgr2=0x7f5d8c071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:18.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.351+0000 7f5d94600640 1 -- 172.21.15.66:0/3390107878 shutdown_connections 2024-03-30T00:51:18.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:18.351+0000 7f5d94600640 1 -- 172.21.15.66:0/3390107878 wait complete. 2024-03-30T00:51:18.779 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:50:38.040525+0000', 'modified': '2024-03-30T00:51:18.177265+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9732}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9732': {'gid': 9732, 'name': 'a', 'rank': 0, 'incarnation': 98, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.66:6836/3460522956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 3460522956}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 3460522956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:51:18.779 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9732:up:rejoin 2024-03-30T00:51:19.780 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:51:19.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.923+0000 7f4895e00640 1 Processor -- start 2024-03-30T00:51:19.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.923+0000 7f4895e00640 1 -- start start 2024-03-30T00:51:19.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.923+0000 7f4895e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4890159860 0x7f48900ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:19.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.923+0000 7f4895e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f48900ce940 0x7f48900b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:19.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.923+0000 7f4895e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48900cd0f0 con 0x7f489015d3f0 2024-03-30T00:51:19.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.923+0000 7f4895e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48900b1830 con 0x7f4890159860 2024-03-30T00:51:19.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.923+0000 7f4895e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489015d9b0 con 0x7f48900ce940 2024-03-30T00:51:19.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4890159860 0x7f48900ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:19.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4890159860 0x7f48900ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33044/0 (socket says 172.21.15.66:33044) 2024-03-30T00:51:19.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488ea00640 1 -- 172.21.15.66:0/2661293453 learned_addr learned my addr 172.21.15.66:0/2661293453 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:19.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f4886000640 1 --1- 172.21.15.66:0/2661293453 >> v1:172.21.15.66:6789/0 conn(0x7f489015d3f0 0x7f48900ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:32974/0 (socket says 172.21.15.66:32974) 2024-03-30T00:51:19.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488f400640 1 --2- 172.21.15.66:0/2661293453 >> v2:172.21.15.156:3300/0 conn(0x7f48900ce940 0x7f48900b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:19.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488ea00640 1 -- 172.21.15.66:0/2661293453 >> v2:172.21.15.156:3300/0 conn(0x7f48900ce940 msgr2=0x7f48900b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:19.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488ea00640 1 --2- 172.21.15.66:0/2661293453 >> v2:172.21.15.156:3300/0 conn(0x7f48900ce940 0x7f48900b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:19.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488ea00640 1 -- 172.21.15.66:0/2661293453 >> v1:172.21.15.66:6789/0 conn(0x7f489015d3f0 legacy=0x7f48900ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:19.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488ea00640 1 -- 172.21.15.66:0/2661293453 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489015dc90 con 0x7f4890159860 2024-03-30T00:51:19.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488ea00640 1 --2- 172.21.15.66:0/2661293453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4890159860 0x7f48900ccbb0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f48900ad530 tx=0x7f488001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62e5e13e38f3c70 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:19.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488e000640 1 -- 172.21.15.66:0/2661293453 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488000b2d0 con 0x7f4890159860 2024-03-30T00:51:19.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488e000640 1 -- 172.21.15.66:0/2661293453 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488000b430 con 0x7f4890159860 2024-03-30T00:51:19.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.927+0000 7f488e000640 1 -- 172.21.15.66:0/2661293453 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48800a7c20 con 0x7f4890159860 2024-03-30T00:51:19.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 -- 172.21.15.66:0/2661293453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4890159860 msgr2=0x7f48900ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:19.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 --2- 172.21.15.66:0/2661293453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4890159860 0x7f48900ccbb0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f48900ad530 tx=0x7f488001eb30 comp rx=0 tx=0).stop 2024-03-30T00:51:19.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 -- 172.21.15.66:0/2661293453 shutdown_connections 2024-03-30T00:51:19.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 --2- 172.21.15.66:0/2661293453 >> v2:172.21.15.156:3300/0 conn(0x7f48900ce940 0x7f48900b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:19.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 --2- 172.21.15.66:0/2661293453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4890159860 0x7f48900ccbb0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:19.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 -- 172.21.15.66:0/2661293453 >> 172.21.15.66:0/2661293453 conn(0x7f4890069cf0 msgr2=0x7f489006a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:19.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 -- 172.21.15.66:0/2661293453 shutdown_connections 2024-03-30T00:51:19.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 -- 172.21.15.66:0/2661293453 wait complete. 2024-03-30T00:51:19.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 Processor -- start 2024-03-30T00:51:19.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 -- start start 2024-03-30T00:51:19.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48900ce940 0x7f489006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:19.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890159860 0x7f48901628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:19.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f489015d3f0 0x7f489015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:19.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489015c3a0 con 0x7f489015d3f0 2024-03-30T00:51:19.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4890096160 con 0x7f4890159860 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4895e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4890096390 con 0x7f48900ce940 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f488f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f489015d3f0 0x7f489015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f488f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f489015d3f0 0x7f489015b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33058/0 (socket says 172.21.15.66:33058) 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f488f400640 1 -- 172.21.15.66:0/1506028536 learned_addr learned my addr 172.21.15.66:0/1506028536 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f488ea00640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48900ce940 0x7f489006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4886000640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890159860 0x7f48901628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f488f400640 1 -- 172.21.15.66:0/1506028536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48900ce940 msgr2=0x7f489006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f488f400640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48900ce940 0x7f489006d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f488f400640 1 -- 172.21.15.66:0/1506028536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890159860 msgr2=0x7f48901628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f488f400640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890159860 0x7f48901628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f488f400640 1 -- 172.21.15.66:0/1506028536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488000bd40 con 0x7f489015d3f0 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.931+0000 7f4886000640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890159860 0x7f48901628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:19.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.935+0000 7f488f400640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f489015d3f0 0x7f489015b920 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f488800d800 tx=0x7f488800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:19.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.935+0000 7f4887e00640 1 -- 172.21.15.66:0/1506028536 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488800b6c0 con 0x7f489015d3f0 2024-03-30T00:51:19.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.935+0000 7f4887e00640 1 -- 172.21.15.66:0/1506028536 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488800b820 con 0x7f489015d3f0 2024-03-30T00:51:19.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.935+0000 7f4887e00640 1 -- 172.21.15.66:0/1506028536 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488800bbd0 con 0x7f489015d3f0 2024-03-30T00:51:19.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.935+0000 7f4895e00640 1 -- 172.21.15.66:0/1506028536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48900966d0 con 0x7f489015d3f0 2024-03-30T00:51:19.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.935+0000 7f4895e00640 1 -- 172.21.15.66:0/1506028536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4890070ae0 con 0x7f489015d3f0 2024-03-30T00:51:19.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.935+0000 7f4895e00640 1 -- 172.21.15.66:0/1506028536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4890066c40 con 0x7f489015d3f0 2024-03-30T00:51:19.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.935+0000 7f4887e00640 1 -- 172.21.15.66:0/1506028536 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f488800bd30 con 0x7f489015d3f0 2024-03-30T00:51:19.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.935+0000 7f4887e00640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c066690 0x7f486c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:19.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.935+0000 7f4887e00640 1 -- 172.21.15.66:0/1506028536 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f4888084800 con 0x7f489015d3f0 2024-03-30T00:51:19.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.935+0000 7f488ea00640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c066690 0x7f486c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:19.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.939+0000 7f488ea00640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c066690 0x7f486c068b50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f48800b0000 tx=0x7f48800a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:19.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:19.939+0000 7f4887e00640 1 -- 172.21.15.66:0/1506028536 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f48880519c0 con 0x7f489015d3f0 2024-03-30T00:51:20.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.135+0000 7f4895e00640 1 -- 172.21.15.66:0/1506028536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48900705f0 con 0x7f489015d3f0 2024-03-30T00:51:20.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4887e00640 1 -- 172.21.15.66:0/1506028536 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4743 (secure 0 0 0) 0x7f4888054fe0 con 0x7f489015d3f0 2024-03-30T00:51:20.146 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:20.146 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":101,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:51:19.184013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9732":{"gid":9732,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/3460522956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3460522956},{"type":"v1","addr":"172.21.15.66:6836","nonce":3460522956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:51:20.146 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 101 2024-03-30T00:51:20.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 -- 172.21.15.66:0/1506028536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c066690 msgr2=0x7f486c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c066690 0x7f486c068b50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f48800b0000 tx=0x7f48800a9070 comp rx=0 tx=0).stop 2024-03-30T00:51:20.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 -- 172.21.15.66:0/1506028536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f489015d3f0 msgr2=0x7f489015b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f489015d3f0 0x7f489015b920 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f488800d800 tx=0x7f488800dcc0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 -- 172.21.15.66:0/1506028536 shutdown_connections 2024-03-30T00:51:20.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f486c066690 0x7f486c068b50 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f489015d3f0 0x7f489015b920 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890159860 0x7f48901628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 --2- 172.21.15.66:0/1506028536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48900ce940 0x7f489006d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 -- 172.21.15.66:0/1506028536 >> 172.21.15.66:0/1506028536 conn(0x7f4890069cf0 msgr2=0x7f489014d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:20.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 -- 172.21.15.66:0/1506028536 shutdown_connections 2024-03-30T00:51:20.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.139+0000 7f4895e00640 1 -- 172.21.15.66:0/1506028536 wait complete. 2024-03-30T00:51:20.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:50:38.040525+0000', 'modified': '2024-03-30T00:51:19.184013+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9732}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9732': {'gid': 9732, 'name': 'a', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.66:6836/3460522956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 3460522956}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 3460522956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:51:20.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:51:20.570 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-30T00:51:20.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f031b400640 1 Processor -- start 2024-03-30T00:51:20.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f031b400640 1 -- start start 2024-03-30T00:51:20.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f031b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f031415aa30 0x7f03140cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f031b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f03140ad080 0x7f03140cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f031b400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031406e280 con 0x7f03140cc1f0 2024-03-30T00:51:20.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f031b400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031406e070 con 0x7f03140ad080 2024-03-30T00:51:20.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f031b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031406e560 con 0x7f031415aa30 2024-03-30T00:51:20.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f030be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f03140ad080 0x7f03140cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f030be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f03140ad080 0x7f03140cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33064/0 (socket says 172.21.15.66:33064) 2024-03-30T00:51:20.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f030be00640 1 -- 172.21.15.66:0/3545309342 learned_addr learned my addr 172.21.15.66:0/3545309342 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:20.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f0318a00640 1 --2- 172.21.15.66:0/3545309342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f031415aa30 0x7f03140cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f030be00640 1 -- 172.21.15.66:0/3545309342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f031415aa30 msgr2=0x7f03140cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:20.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f030be00640 1 --2- 172.21.15.66:0/3545309342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f031415aa30 0x7f03140cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f030be00640 1 -- 172.21.15.66:0/3545309342 >> v1:172.21.15.156:6789/0 conn(0x7f03140cc1f0 legacy=0x7f03140ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f030be00640 1 -- 172.21.15.66:0/3545309342 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031406e840 con 0x7f03140ad080 2024-03-30T00:51:20.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.707+0000 7f0318a00640 1 --2- 172.21.15.66:0/3545309342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f031415aa30 0x7f03140cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:20.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f030be00640 1 --2- 172.21.15.66:0/3545309342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03140ad080 0x7f03140cea70 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009870 tx=0x7f02fc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0715eb7643fb7a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:20.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f0313e00640 1 -- 172.21.15.66:0/3545309342 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0ab640 con 0x7f03140ad080 2024-03-30T00:51:20.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f0313e00640 1 -- 172.21.15.66:0/3545309342 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc0a7070 con 0x7f03140ad080 2024-03-30T00:51:20.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f0313e00640 1 -- 172.21.15.66:0/3545309342 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0ab9c0 con 0x7f03140ad080 2024-03-30T00:51:20.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f031b400640 1 -- 172.21.15.66:0/3545309342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03140ad080 msgr2=0x7f03140cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f031b400640 1 --2- 172.21.15.66:0/3545309342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03140ad080 0x7f03140cea70 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009870 tx=0x7f02fc01c530 comp rx=0 tx=0).stop 2024-03-30T00:51:20.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f031b400640 1 -- 172.21.15.66:0/3545309342 shutdown_connections 2024-03-30T00:51:20.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f031b400640 1 --2- 172.21.15.66:0/3545309342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03140ad080 0x7f03140cea70 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f031b400640 1 --2- 172.21.15.66:0/3545309342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f031415aa30 0x7f03140cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f031b400640 1 -- 172.21.15.66:0/3545309342 >> 172.21.15.66:0/3545309342 conn(0x7f03140d0490 msgr2=0x7f03140d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:20.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f031b400640 1 -- 172.21.15.66:0/3545309342 shutdown_connections 2024-03-30T00:51:20.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.711+0000 7f031b400640 1 -- 172.21.15.66:0/3545309342 wait complete. 2024-03-30T00:51:20.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f031b400640 1 Processor -- start 2024-03-30T00:51:20.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f031b400640 1 -- start start 2024-03-30T00:51:20.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f031b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03140ad080 0x7f03141505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f031b400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03140cc1f0 0x7f031406b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f031b400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f031415aa30 0x7f031414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f031b400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031406b860 con 0x7f03140ad080 2024-03-30T00:51:20.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f031b400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031406bc70 con 0x7f031415aa30 2024-03-30T00:51:20.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f031b400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031406bf50 con 0x7f03140cc1f0 2024-03-30T00:51:20.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0318a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03140ad080 0x7f03141505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0318a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03140ad080 0x7f03141505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33070/0 (socket says 172.21.15.66:33070) 2024-03-30T00:51:20.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0318a00640 1 -- 172.21.15.66:0/2858972268 learned_addr learned my addr 172.21.15.66:0/2858972268 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:20.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0319400640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f031415aa30 0x7f031414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f030be00640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03140cc1f0 0x7f031406b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0318a00640 1 -- 172.21.15.66:0/2858972268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03140cc1f0 msgr2=0x7f031406b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0318a00640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03140cc1f0 0x7f031406b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0318a00640 1 -- 172.21.15.66:0/2858972268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f031415aa30 msgr2=0x7f031414fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0318a00640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f031415aa30 0x7f031414fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0318a00640 1 -- 172.21.15.66:0/2858972268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc0094d0 con 0x7f03140ad080 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0319400640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f031415aa30 0x7f031414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0318a00640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03140ad080 0x7f03141505e0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f030400d800 tx=0x7f030400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0311600640 1 -- 172.21.15.66:0/2858972268 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03040096c0 con 0x7f03140ad080 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0311600640 1 -- 172.21.15.66:0/2858972268 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0304009820 con 0x7f03140ad080 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f0311600640 1 -- 172.21.15.66:0/2858972268 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304009bd0 con 0x7f03140ad080 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f031b400640 1 -- 172.21.15.66:0/2858972268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031406c230 con 0x7f03140ad080 2024-03-30T00:51:20.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.715+0000 7f031b400640 1 -- 172.21.15.66:0/2858972268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0314150dd0 con 0x7f03140ad080 2024-03-30T00:51:20.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f0311600640 1 -- 172.21.15.66:0/2858972268 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0304009d30 con 0x7f03140ad080 2024-03-30T00:51:20.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f0311600640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02f0066690 0x7f02f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f0311600640 1 -- 172.21.15.66:0/2858972268 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f0304084a40 con 0x7f03140ad080 2024-03-30T00:51:20.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f030be00640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02f0066690 0x7f02f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f031e7f7000 1 Processor -- start 2024-03-30T00:51:20.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f031e7f7000 1 -- start start 2024-03-30T00:51:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f030be00640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02f0066690 0x7f02f0068b50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0af000 tx=0x7f02fc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164cc4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 0x558164c79f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164c7a4c0 0x558164c7a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f031e7f7000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164b425a0 con 0x558164cc3db0 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f031e7f7000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164acfaf0 con 0x558164cc46d0 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f031e7f7000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558164acd0c0 con 0x558164c7a4c0 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f0318a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164cc4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f0318a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164cc4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33084/0 (socket says 172.21.15.66:33084) 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f0318a00640 1 -- 172.21.15.66:0/1728296240 learned_addr learned my addr 172.21.15.66:0/1728296240 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f030be00640 1 --2- 172.21.15.66:0/1728296240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 0x558164c79f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.719+0000 7f0319400640 1 --2- 172.21.15.66:0/1728296240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164c7a4c0 0x558164c7a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f0318a00640 1 -- 172.21.15.66:0/1728296240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164c7a4c0 msgr2=0x558164c7a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f0318a00640 1 --2- 172.21.15.66:0/1728296240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164c7a4c0 0x558164c7a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f0318a00640 1 -- 172.21.15.66:0/1728296240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 msgr2=0x558164c79f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f0318a00640 1 --2- 172.21.15.66:0/1728296240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 0x558164c79f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f0318a00640 1 -- 172.21.15.66:0/1728296240 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581647c4890 con 0x558164cc3db0 2024-03-30T00:51:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f0318a00640 1 --2- 172.21.15.66:0/1728296240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164cc4190 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f0304051dc0 tx=0x7f0304055350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cddd0e27a02fa4b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f0309600640 1 -- 172.21.15.66:0/1728296240 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03040596f0 con 0x558164cc3db0 2024-03-30T00:51:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f0309600640 1 -- 172.21.15.66:0/1728296240 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030405dcb0 con 0x558164cc3db0 2024-03-30T00:51:20.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f0309600640 1 -- 172.21.15.66:0/1728296240 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030406d1c0 con 0x558164cc3db0 2024-03-30T00:51:20.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 -- 172.21.15.66:0/1728296240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 msgr2=0x558164cc4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 --2- 172.21.15.66:0/1728296240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164cc4190 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f0304051dc0 tx=0x7f0304055350 comp rx=0 tx=0).stop 2024-03-30T00:51:20.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 -- 172.21.15.66:0/1728296240 shutdown_connections 2024-03-30T00:51:20.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 --2- 172.21.15.66:0/1728296240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164c7a4c0 0x558164c7a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 --2- 172.21.15.66:0/1728296240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 0x558164c79f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 --2- 172.21.15.66:0/1728296240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164cc4190 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 -- 172.21.15.66:0/1728296240 >> 172.21.15.66:0/1728296240 conn(0x558164c80830 msgr2=0x558164c78860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 -- 172.21.15.66:0/1728296240 shutdown_connections 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 -- 172.21.15.66:0/1728296240 wait complete. 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 -- start start 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 Processor -- start 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164c7a4c0 0x558164c7e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164c7e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 0x558164c7ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164ac89b0 con 0x558164cc3db0 2024-03-30T00:51:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164a67c00 con 0x558164cc46d0 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558164a86070 con 0x558164c7a4c0 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f031e7f7000 10 client.0 _ll_register_callbacks cb 0x558164c660e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f030be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164c7e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f030be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164c7e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33100/0 (socket says 172.21.15.66:33100) 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f030be00640 1 -- 172.21.15.66:0/3688542077 learned_addr learned my addr 172.21.15.66:0/3688542077 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f0318a00640 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164c7a4c0 0x558164c7e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.723+0000 7f0319400640 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 0x558164c7ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f030be00640 1 -- 172.21.15.66:0/3688542077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164c7a4c0 msgr2=0x558164c7e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f030be00640 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164c7a4c0 0x558164c7e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f030be00640 1 -- 172.21.15.66:0/3688542077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 msgr2=0x558164c7ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f030be00640 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 0x558164c7ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f030be00640 1 -- 172.21.15.66:0/3688542077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0314150dd0 con 0x558164cc3db0 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f030be00640 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164c7e770 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0ad040 tx=0x7f02fc01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c0a455bfcb4e00 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0302000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0302000640 1 -- 172.21.15.66:0/3688542077 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0ca640 con 0x558164cc3db0 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 -- 172.21.15.66:0/3688542077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558164a855f0 con 0x558164cc3db0 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0302000640 1 -- 172.21.15.66:0/3688542077 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f02fc0c0410 con 0x558164cc3db0 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0302000640 1 -- 172.21.15.66:0/3688542077 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02fc0c05d0 con 0x558164cc3db0 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 10 client.9783 fetch_fsmap learned FSMap version 101 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 -- 172.21.15.66:0/3688542077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) v3 -- 0x558164a8dab0 con 0x558164cc3db0 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0302000640 1 -- 172.21.15.66:0/3688542077 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558164a855f0 con 0x558164cc3db0 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0302000640 1 -- 172.21.15.66:0/3688542077 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 101) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f02fc0ca7a0 con 0x558164cc3db0 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 10 client.9783 fetch_fsmap finished waiting for FSMap version 101 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 10 client.9783 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:51:20.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 20 client.9783 populate_metadata read hostname 'smithi066' 2024-03-30T00:51:20.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x558164c7f6c0 0x558164cd4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 4 client.9783 mds_command: new command op to 9732 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:51:20.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 -- 172.21.15.66:0/3688542077 --> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558164ac5790 con 0x558164c7f6c0 2024-03-30T00:51:20.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0318a00640 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x558164c7f6c0 0x558164cd4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0302000640 10 client.9783 ms_handle_connect on v2:172.21.15.66:6834/3460522956 2024-03-30T00:51:20.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0318a00640 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x558164c7f6c0 0x558164cd4e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8254d97f17d52cf1 server_cookie=ab907f2788d8b517 in_seq=0 out_seq=0 2024-03-30T00:51:20.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0302000640 1 -- 172.21.15.66:0/3688542077 <== mds.0 v2:172.21.15.66:6834/3460522956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558164ac5790 con 0x558164c7f6c0 2024-03-30T00:51:20.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0302000640 10 client.9783 handle_command_reply: tid=0 2024-03-30T00:51:20.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 client.9783 shutdown 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 -- 172.21.15.66:0/3688542077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 msgr2=0x558164c7e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164c7e770 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0ad040 tx=0x7f02fc01ad70 comp rx=0 tx=0).stop 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 -- 172.21.15.66:0/3688542077 shutdown_connections 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x558164c7f6c0 0x558164cd4e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 0x558164c7ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164cc3db0 0x558164c7e770 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f0302000640 0 client.9783 ms_handle_reset on v2:172.21.15.66:6834/3460522956 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 --2- 172.21.15.66:0/3688542077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164c7a4c0 0x558164c7e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 -- 172.21.15.66:0/3688542077 >> 172.21.15.66:0/3688542077 conn(0x558164c80830 msgr2=0x558164c78f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 -- 172.21.15.66:0/3688542077 shutdown_connections 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 1 -- 172.21.15.66:0/3688542077 wait complete. 2024-03-30T00:51:20.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.727+0000 7f031e7f7000 20 client.9783 trim_cache size 0 max 16384 2024-03-30T00:51:20.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.743+0000 7f031e7f7000 1 Processor -- start 2024-03-30T00:51:20.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.743+0000 7f031e7f7000 1 -- start start 2024-03-30T00:51:20.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.743+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.743+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc3db0 0x558164d1d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.743+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164cc46d0 0x558164d1dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.743+0000 7f031e7f7000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164785940 con 0x558164c7a4c0 2024-03-30T00:51:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164b993e0 con 0x558164cc3db0 2024-03-30T00:51:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558164a8fc60 con 0x558164cc46d0 2024-03-30T00:51:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33112/0 (socket says 172.21.15.66:33112) 2024-03-30T00:51:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 -- 172.21.15.66:0/2694460830 learned_addr learned my addr 172.21.15.66:0/2694460830 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0319400640 1 --2- 172.21.15.66:0/2694460830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164cc46d0 0x558164d1dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f030be00640 1 --2- 172.21.15.66:0/2694460830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc3db0 0x558164d1d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 -- 172.21.15.66:0/2694460830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164cc46d0 msgr2=0x558164d1dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:20.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 --2- 172.21.15.66:0/2694460830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164cc46d0 0x558164d1dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 -- 172.21.15.66:0/2694460830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc3db0 msgr2=0x558164d1d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:20.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 --2- 172.21.15.66:0/2694460830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc3db0 0x558164d1d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 -- 172.21.15.66:0/2694460830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558164a73c00 con 0x558164c7a4c0 2024-03-30T00:51:20.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 --2- 172.21.15.66:0/2694460830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1d460 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f0304055850 tx=0x7f030406a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0c1e1478e78ba6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:20.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f030a000640 1 -- 172.21.15.66:0/2694460830 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03040596f0 con 0x558164c7a4c0 2024-03-30T00:51:20.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f030a000640 1 -- 172.21.15.66:0/2694460830 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0304053c60 con 0x558164c7a4c0 2024-03-30T00:51:20.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f030a000640 1 -- 172.21.15.66:0/2694460830 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304051fd0 con 0x558164c7a4c0 2024-03-30T00:51:20.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 -- 172.21.15.66:0/2694460830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 msgr2=0x558164d1d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 --2- 172.21.15.66:0/2694460830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1d460 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f0304055850 tx=0x7f030406a000 comp rx=0 tx=0).stop 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 -- 172.21.15.66:0/2694460830 shutdown_connections 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 --2- 172.21.15.66:0/2694460830 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164cc46d0 0x558164d1dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 --2- 172.21.15.66:0/2694460830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc3db0 0x558164d1d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 --2- 172.21.15.66:0/2694460830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1d460 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 -- 172.21.15.66:0/2694460830 >> 172.21.15.66:0/2694460830 conn(0x558164c80830 msgr2=0x558164c7d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 -- 172.21.15.66:0/2694460830 shutdown_connections 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 -- 172.21.15.66:0/2694460830 wait complete. 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 -- start start 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 Processor -- start 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164cc3db0 0x558164cc1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 0x558164cc1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164acc9b0 con 0x558164c7a4c0 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558164b0bac0 con 0x558164cc46d0 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558164ac61c0 con 0x558164cc3db0 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f031e7f7000 10 client.0 _ll_register_callbacks cb 0x558164a8d590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33124/0 (socket says 172.21.15.66:33124) 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.747+0000 7f0318a00640 1 -- 172.21.15.66:0/2705758142 learned_addr learned my addr 172.21.15.66:0/2705758142 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0318a00640 1 -- 172.21.15.66:0/2705758142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164cc3db0 msgr2=0x558164cc1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0318a00640 1 --2- 172.21.15.66:0/2705758142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164cc3db0 0x558164cc1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0318a00640 1 -- 172.21.15.66:0/2705758142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 msgr2=0x558164cc1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0318a00640 1 --2- 172.21.15.66:0/2705758142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 0x558164cc1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0318a00640 1 -- 172.21.15.66:0/2705758142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581647c4890 con 0x558164c7a4c0 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0318a00640 1 --2- 172.21.15.66:0/2705758142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1e170 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f0304055850 tx=0x7f0304000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4d3ab1df5fe056 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0302000640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0302000640 1 -- 172.21.15.66:0/2705758142 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03040596f0 con 0x558164c7a4c0 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 -- 172.21.15.66:0/2705758142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558164a98460 con 0x558164c7a4c0 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0302000640 1 -- 172.21.15.66:0/2705758142 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f0304084040 con 0x558164c7a4c0 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0302000640 1 -- 172.21.15.66:0/2705758142 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0304051fd0 con 0x558164c7a4c0 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 10 client.9789 fetch_fsmap learned FSMap version 101 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 -- 172.21.15.66:0/2705758142 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) v3 -- 0x5581647c4cc0 con 0x558164c7a4c0 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0302000640 1 -- 172.21.15.66:0/2705758142 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558164a98460 con 0x558164c7a4c0 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0302000640 1 -- 172.21.15.66:0/2705758142 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 101) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f030407a030 con 0x558164c7a4c0 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 10 client.9789 fetch_fsmap finished waiting for FSMap version 101 2024-03-30T00:51:20.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 10 client.9789 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 20 client.9789 populate_metadata read hostname 'smithi066' 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 --2- 172.21.15.66:0/2705758142 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x558164cc1e80 0x558164cc2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 4 client.9789 mds_command: new command op to 9732 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 -- 172.21.15.66:0/2705758142 --> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558164a919c0 con 0x558164cc1e80 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f030be00640 1 --2- 172.21.15.66:0/2705758142 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x558164cc1e80 0x558164cc2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f030be00640 1 --2- 172.21.15.66:0/2705758142 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x558164cc1e80 0x558164cc2260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f88e1e07927655c2 server_cookie=bca8157e8b98c0c5 in_seq=0 out_seq=0 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0302000640 10 client.9789 ms_handle_connect on v2:172.21.15.66:6834/3460522956 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0302000640 1 -- 172.21.15.66:0/2705758142 <== mds.0 v2:172.21.15.66:6834/3460522956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558164a919c0 con 0x558164cc1e80 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0302000640 10 client.9789 handle_command_reply: tid=0 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 client.9789 shutdown 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 -- 172.21.15.66:0/2705758142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 msgr2=0x558164d1e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 --2- 172.21.15.66:0/2705758142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1e170 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f0304055850 tx=0x7f0304000bd0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 -- 172.21.15.66:0/2705758142 shutdown_connections 2024-03-30T00:51:20.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 --2- 172.21.15.66:0/2705758142 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x558164cc1e80 0x558164cc2260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 --2- 172.21.15.66:0/2705758142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558164cc46d0 0x558164cc1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f0302000640 0 client.9789 ms_handle_reset on v2:172.21.15.66:6834/3460522956 2024-03-30T00:51:20.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 --2- 172.21.15.66:0/2705758142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558164cc3db0 0x558164cc1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 --2- 172.21.15.66:0/2705758142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558164c7a4c0 0x558164d1e170 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 -- 172.21.15.66:0/2705758142 >> 172.21.15.66:0/2705758142 conn(0x558164c80830 msgr2=0x558164c7dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:20.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 -- 172.21.15.66:0/2705758142 shutdown_connections 2024-03-30T00:51:20.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 1 -- 172.21.15.66:0/2705758142 wait complete. 2024-03-30T00:51:20.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031e7f7000 20 client.9789 trim_cache size 0 max 16384 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 9732, 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 4.575745263, 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 101, 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 126, 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 126, 2024-03-30T00:51:20.766 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 5.5156406970000003 2024-03-30T00:51:20.767 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:20.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 -- 172.21.15.66:0/2858972268 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02f0066690 msgr2=0x7f02f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02f0066690 0x7f02f0068b50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0af000 tx=0x7f02fc0a7000 comp rx=0 tx=0).stop 2024-03-30T00:51:20.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 -- 172.21.15.66:0/2858972268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03140ad080 msgr2=0x7f03141505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03140ad080 0x7f03141505e0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f030400d800 tx=0x7f030400dcc0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 -- 172.21.15.66:0/2858972268 shutdown_connections 2024-03-30T00:51:20.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f031415aa30 0x7f031414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03140cc1f0 0x7f031406b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02f0066690 0x7f02f0068b50 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 --2- 172.21.15.66:0/2858972268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03140ad080 0x7f03141505e0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 -- 172.21.15.66:0/2858972268 >> 172.21.15.66:0/2858972268 conn(0x7f03140d0490 msgr2=0x7f031415ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:20.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 -- 172.21.15.66:0/2858972268 shutdown_connections 2024-03-30T00:51:20.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.751+0000 7f031b400640 1 -- 172.21.15.66:0/2858972268 wait complete. 2024-03-30T00:51:20.780 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:51:20.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.911+0000 7f9217600640 1 Processor -- start 2024-03-30T00:51:20.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.911+0000 7f9217600640 1 -- start start 2024-03-30T00:51:20.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9217600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9210159860 0x7f92100ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9217600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f92100ce940 0x7f92100b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9217600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92100cd0f0 con 0x7f921015d3f0 2024-03-30T00:51:20.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9217600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92100b1830 con 0x7f92100ce940 2024-03-30T00:51:20.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9217600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921015d9b0 con 0x7f9210159860 2024-03-30T00:51:20.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9215600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f92100ce940 0x7f92100b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9215600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f92100ce940 0x7f92100b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33132/0 (socket says 172.21.15.66:33132) 2024-03-30T00:51:20.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9215600640 1 -- 172.21.15.66:0/1963510600 learned_addr learned my addr 172.21.15.66:0/1963510600 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:20.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f920fe00640 1 --1- 172.21.15.66:0/1963510600 >> v1:172.21.15.66:6789/0 conn(0x7f921015d3f0 0x7f92100ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:32980/0 (socket says 172.21.15.66:32980) 2024-03-30T00:51:20.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9214c00640 1 --2- 172.21.15.66:0/1963510600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9210159860 0x7f92100ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9215600640 1 -- 172.21.15.66:0/1963510600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9210159860 msgr2=0x7f92100ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:20.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9215600640 1 --2- 172.21.15.66:0/1963510600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9210159860 0x7f92100ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9215600640 1 -- 172.21.15.66:0/1963510600 >> v1:172.21.15.66:6789/0 conn(0x7f921015d3f0 legacy=0x7f92100ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9215600640 1 -- 172.21.15.66:0/1963510600 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921015dc90 con 0x7f92100ce940 2024-03-30T00:51:20.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.915+0000 7f9215600640 1 --2- 172.21.15.66:0/1963510600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92100ce940 0x7f92100b1cc0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f9208009e50 tx=0x7f920801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcae179e206ed1e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:20.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f920f400640 1 -- 172.21.15.66:0/1963510600 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92080a5850 con 0x7f92100ce940 2024-03-30T00:51:20.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f920f400640 1 -- 172.21.15.66:0/1963510600 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92080a59b0 con 0x7f92100ce940 2024-03-30T00:51:20.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f920f400640 1 -- 172.21.15.66:0/1963510600 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92080a5d40 con 0x7f92100ce940 2024-03-30T00:51:20.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 -- 172.21.15.66:0/1963510600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92100ce940 msgr2=0x7f92100b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 --2- 172.21.15.66:0/1963510600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92100ce940 0x7f92100b1cc0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f9208009e50 tx=0x7f920801c970 comp rx=0 tx=0).stop 2024-03-30T00:51:20.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 -- 172.21.15.66:0/1963510600 shutdown_connections 2024-03-30T00:51:20.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 --2- 172.21.15.66:0/1963510600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92100ce940 0x7f92100b1cc0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 --2- 172.21.15.66:0/1963510600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9210159860 0x7f92100ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 -- 172.21.15.66:0/1963510600 >> 172.21.15.66:0/1963510600 conn(0x7f9210069cf0 msgr2=0x7f921006a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:20.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 -- 172.21.15.66:0/1963510600 shutdown_connections 2024-03-30T00:51:20.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 -- 172.21.15.66:0/1963510600 wait complete. 2024-03-30T00:51:20.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 Processor -- start 2024-03-30T00:51:20.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 -- start start 2024-03-30T00:51:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92100ce940 0x7f921015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9210159860 0x7f9210097a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921015d3f0 0x7f9210152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92101f3530 con 0x7f9210159860 2024-03-30T00:51:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92100c1c90 con 0x7f921015d3f0 2024-03-30T00:51:20.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9217600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9210152d40 con 0x7f92100ce940 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f920fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9210159860 0x7f9210097a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f920fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9210159860 0x7f9210097a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33136/0 (socket says 172.21.15.66:33136) 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f920fe00640 1 -- 172.21.15.66:0/281473311 learned_addr learned my addr 172.21.15.66:0/281473311 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9214c00640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92100ce940 0x7f921015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9215600640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921015d3f0 0x7f9210152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f920fe00640 1 -- 172.21.15.66:0/281473311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92100ce940 msgr2=0x7f921015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f920fe00640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92100ce940 0x7f921015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f920fe00640 1 -- 172.21.15.66:0/281473311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921015d3f0 msgr2=0x7f9210152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f920fe00640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921015d3f0 0x7f9210152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f920fe00640 1 -- 172.21.15.66:0/281473311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9208009b30 con 0x7f9210159860 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9214c00640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92100ce940 0x7f921015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:20.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.919+0000 7f9215600640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921015d3f0 0x7f9210152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:20.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.923+0000 7f920fe00640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9210159860 0x7f9210097a50 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c400 tx=0x7f91fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:20.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.923+0000 7f920cc00640 1 -- 172.21.15.66:0/281473311 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc0074e0 con 0x7f9210159860 2024-03-30T00:51:20.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.923+0000 7f920cc00640 1 -- 172.21.15.66:0/281473311 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc007660 con 0x7f9210159860 2024-03-30T00:51:20.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.923+0000 7f920cc00640 1 -- 172.21.15.66:0/281473311 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc004a30 con 0x7f9210159860 2024-03-30T00:51:20.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.923+0000 7f9217600640 1 -- 172.21.15.66:0/281473311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9210152fc0 con 0x7f9210159860 2024-03-30T00:51:20.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.923+0000 7f9217600640 1 -- 172.21.15.66:0/281473311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9210153580 con 0x7f9210159860 2024-03-30T00:51:20.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.923+0000 7f9217600640 1 -- 172.21.15.66:0/281473311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9210153f60 con 0x7f9210159860 2024-03-30T00:51:20.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.923+0000 7f920cc00640 1 -- 172.21.15.66:0/281473311 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f91fc004b90 con 0x7f9210159860 2024-03-30T00:51:20.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.923+0000 7f920cc00640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91f0066690 0x7f91f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:20.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.923+0000 7f920cc00640 1 -- 172.21.15.66:0/281473311 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f91fc0841d0 con 0x7f9210159860 2024-03-30T00:51:20.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.923+0000 7f9214c00640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91f0066690 0x7f91f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:20.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.927+0000 7f9214c00640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91f0066690 0x7f91f0068b50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9204009ae0 tx=0x7f9204008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:20.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:20.927+0000 7f920cc00640 1 -- 172.21.15.66:0/281473311 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f91fc051480 con 0x7f9210159860 2024-03-30T00:51:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.123+0000 7f9217600640 1 -- 172.21.15.66:0/281473311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f92100cd0f0 con 0x7f9210159860 2024-03-30T00:51:21.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.123+0000 7f920cc00640 1 -- 172.21.15.66:0/281473311 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) v1 ==== 53+0+73 (secure 0 0 0) 0x7f92100cd0f0 con 0x7f9210159860 2024-03-30T00:51:21.132 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:51:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 -- 172.21.15.66:0/281473311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91f0066690 msgr2=0x7f91f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91f0066690 0x7f91f0068b50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9204009ae0 tx=0x7f9204008040 comp rx=0 tx=0).stop 2024-03-30T00:51:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 -- 172.21.15.66:0/281473311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9210159860 msgr2=0x7f9210097a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:21.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9210159860 0x7f9210097a50 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c400 tx=0x7f91fc00c8c0 comp rx=0 tx=0).stop 2024-03-30T00:51:21.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 -- 172.21.15.66:0/281473311 shutdown_connections 2024-03-30T00:51:21.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921015d3f0 0x7f9210152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:21.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f91f0066690 0x7f91f0068b50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:21.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9210159860 0x7f9210097a50 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:21.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 --2- 172.21.15.66:0/281473311 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92100ce940 0x7f921015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:21.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 -- 172.21.15.66:0/281473311 >> 172.21.15.66:0/281473311 conn(0x7f9210069cf0 msgr2=0x7f92100ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:21.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 -- 172.21.15.66:0/281473311 shutdown_connections 2024-03-30T00:51:21.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:21.127+0000 7f9217600640 1 -- 172.21.15.66:0/281473311 wait complete. 2024-03-30T00:51:21.154 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:51:21.154 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:51:21.154 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:51:21.154 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:51:21.154 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:21.154 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:51:21.154 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:51:21.154 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:51:21.154 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:51:21.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:21.155 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:51:21.164 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:51:21.164 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:51:21.165 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:21.166 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:51:21.166 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:21.166 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:51:21.166 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41797sec preferred_lft 41797sec 2024-03-30T00:51:21.166 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:51:21.166 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:21.166 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:51:21.166 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:21.166 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:51:21.166 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:21.166 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:51:21.167 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:21.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:21.168 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:51:21.186 INFO:tasks.ceph.mds.a.smithi066.stderr:2024-03-30T00:51:21.179+0000 7f091d800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:51:21.203 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:51:21.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:21.206 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:51:21.260 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:51:21.260 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:51:21.260 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:51:21.261 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:51:21.261 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:51:21.261 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:51:21.261 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:51:21.261 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:51:21.261 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:51:21.261 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:51:21.261 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:51:21.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:21.261 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:51:21.261 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:51:21.262 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:51:21.262 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:51:21.262 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:51:21.262 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:51:21.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:21.351 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:51:21.359 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:51:21.360 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:51:21.360 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:51:21.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:21.360 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:51:21.421 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:51:21.421 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:51:21.421 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:21.473 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:51:21.473 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:51:21.473 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:21.524 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:51:21.586 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:51:21.586 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:51:21.648 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:51:21.650 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:51:21.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:21.650 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:51:21.702 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:51:21.702 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:51:21.748 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:51:21.770 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:51:21.770 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:51:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:51:21.833 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:51:21.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:21.834 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:51:21.842 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:51:21.835+0000 7fde95455580 -1 init, newargv = 0x55a2f0c30f60 newargc=15 2024-03-30T00:51:21.843 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[54919]: starting ceph client 2024-03-30T00:51:21.850 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[54919]: starting fuse 2024-03-30T00:51:21.852 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:51:21.852 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:51:21.852 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:21.852 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:51:21.852 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:51:21.852 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:21.852 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:51:21.852 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:21.852 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:51:21.852 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:51:21.852 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:51:21.853 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:51:21.854 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:21.854 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:51:21.854 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:21.974 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.54919.asok 2024-03-30T00:51:22.159 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54919.asok 2024-03-30T00:51:22.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:22.159 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54919.asok status 2024-03-30T00:51:22.337 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:22.337 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:51:22.337 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:51:22.337 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "54919", 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 9807, 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:51:22.338 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 9807 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 4035468943 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 4035468943 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.9807 192.168.0.1:0/4035468943", 2024-03-30T00:51:22.339 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/4035468943", 2024-03-30T00:51:22.340 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:51:22.340 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 101, 2024-03-30T00:51:22.340 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 126, 2024-03-30T00:51:22.340 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:51:22.340 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:51:22.340 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:51:22.340 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:22.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:22.352 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:51:22.352 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:22.362 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:51:22.362 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:22.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:22.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:22.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:22.425 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-30T00:51:22.486 INFO:teuthology.orchestra.run.smithi066.stdout:./ 2024-03-30T00:51:22.486 INFO:teuthology.orchestra.run.smithi066.stdout:./parent 2024-03-30T00:51:22.486 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/flushed 2024-03-30T00:51:22.486 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/flushed/charlie 2024-03-30T00:51:22.486 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/flushed/alpha 2024-03-30T00:51:22.486 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/unflushed 2024-03-30T00:51:22.486 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/unflushed/jfile 2024-03-30T00:51:22.487 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:22.488 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:22.488 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:22.488 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:22.488 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-30T00:51:22.488 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:22.592 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:51:22.782 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:22.782 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:22.782 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:22.782 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:22.782 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-30T00:51:22.782 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:22.894 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627776 2024-03-30T00:51:23.084 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:23.084 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:23.084 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:23.084 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:23.084 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-30T00:51:23.084 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:23.190 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627777 2024-03-30T00:51:23.379 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:23.379 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:23.379 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:23.379 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:23.379 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-30T00:51:23.379 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:23.490 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628280 2024-03-30T00:51:23.679 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:23.679 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:23.680 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:23.680 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:23.680 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-30T00:51:23.680 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:23.786 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627778 2024-03-30T00:51:23.975 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:23.975 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:23.975 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:23.975 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:23.976 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-30T00:51:23.976 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:24.085 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627779 2024-03-30T00:51:24.274 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:24.274 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:24.274 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:24.274 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:24.274 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-30T00:51:24.274 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:24.382 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627780 2024-03-30T00:51:24.571 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:51:24.571 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:24.626 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:51:24.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.799+0000 7fb50b000640 1 Processor -- start 2024-03-30T00:51:24.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.799+0000 7fb50b000640 1 -- start start 2024-03-30T00:51:24.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.799+0000 7fb50b000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb50415a370 0x7fb50415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:24.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.799+0000 7fb50b000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb50406ad00 0x7fb50406b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:24.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.799+0000 7fb50b000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5041612b0 con 0x7fb504160350 2024-03-30T00:51:24.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.799+0000 7fb50b000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50406b6d0 con 0x7fb50415a370 2024-03-30T00:51:24.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.799+0000 7fb50b000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50415a750 con 0x7fb50406ad00 2024-03-30T00:51:24.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb503e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb50415a370 0x7fb50415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:24.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb503e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb50415a370 0x7fb50415d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33144/0 (socket says 172.21.15.66:33144) 2024-03-30T00:51:24.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb503e00640 1 -- 172.21.15.66:0/1700589154 learned_addr learned my addr 172.21.15.66:0/1700589154 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:24.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb4fb400640 1 --2- 172.21.15.66:0/1700589154 >> v2:172.21.15.156:3300/0 conn(0x7fb50406ad00 0x7fb50406b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:24.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb503e00640 1 -- 172.21.15.66:0/1700589154 >> v2:172.21.15.156:3300/0 conn(0x7fb50406ad00 msgr2=0x7fb50406b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:24.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb503e00640 1 --2- 172.21.15.66:0/1700589154 >> v2:172.21.15.156:3300/0 conn(0x7fb50406ad00 0x7fb50406b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:24.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb503e00640 1 -- 172.21.15.66:0/1700589154 >> v1:172.21.15.156:6789/0 conn(0x7fb504160350 legacy=0x7fb504160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:24.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb503e00640 1 -- 172.21.15.66:0/1700589154 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504160fd0 con 0x7fb50415a370 2024-03-30T00:51:24.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb4fb400640 1 --2- 172.21.15.66:0/1700589154 >> v2:172.21.15.156:3300/0 conn(0x7fb50406ad00 0x7fb50406b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:24.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb503e00640 1 --2- 172.21.15.66:0/1700589154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb50415a370 0x7fb50415d9c0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fb5040719d0 tx=0x7fb4f401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9af7a7b1869b7ec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:24.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb503400640 1 -- 172.21.15.66:0/1700589154 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40a89d0 con 0x7fb50415a370 2024-03-30T00:51:24.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb503400640 1 -- 172.21.15.66:0/1700589154 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f40a8b30 con 0x7fb50415a370 2024-03-30T00:51:24.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.803+0000 7fb503400640 1 -- 172.21.15.66:0/1700589154 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40b0440 con 0x7fb50415a370 2024-03-30T00:51:24.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 -- 172.21.15.66:0/1700589154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb50415a370 msgr2=0x7fb50415d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:24.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 --2- 172.21.15.66:0/1700589154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb50415a370 0x7fb50415d9c0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fb5040719d0 tx=0x7fb4f401ecd0 comp rx=0 tx=0).stop 2024-03-30T00:51:24.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 -- 172.21.15.66:0/1700589154 shutdown_connections 2024-03-30T00:51:24.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 --2- 172.21.15.66:0/1700589154 >> v2:172.21.15.156:3300/0 conn(0x7fb50406ad00 0x7fb50406b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:24.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 --2- 172.21.15.66:0/1700589154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb50415a370 0x7fb50415d9c0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:24.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 -- 172.21.15.66:0/1700589154 >> 172.21.15.66:0/1700589154 conn(0x7fb5040706e0 msgr2=0x7fb504070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:24.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 -- 172.21.15.66:0/1700589154 shutdown_connections 2024-03-30T00:51:24.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 -- 172.21.15.66:0/1700589154 wait complete. 2024-03-30T00:51:24.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 Processor -- start 2024-03-30T00:51:24.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 -- start start 2024-03-30T00:51:24.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb50406ad00 0x7fb504152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:24.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb50415a370 0x7fb5040b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:24.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb504160350 0x7fb504154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:24.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb504154180 con 0x7fb50415a370 2024-03-30T00:51:24.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040b0550 con 0x7fb504160350 2024-03-30T00:51:24.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb50b000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb504153570 con 0x7fb50406ad00 2024-03-30T00:51:24.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb4fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb50415a370 0x7fb5040b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:24.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb4fb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb50415a370 0x7fb5040b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33156/0 (socket says 172.21.15.66:33156) 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb4fb400640 1 -- 172.21.15.66:0/2305314591 learned_addr learned my addr 172.21.15.66:0/2305314591 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb503e00640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb50406ad00 0x7fb504152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb509000640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb504160350 0x7fb504154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb4fb400640 1 -- 172.21.15.66:0/2305314591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb50406ad00 msgr2=0x7fb504152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb4fb400640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb50406ad00 0x7fb504152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb4fb400640 1 -- 172.21.15.66:0/2305314591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb504160350 msgr2=0x7fb504154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb4fb400640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb504160350 0x7fb504154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb4fb400640 1 -- 172.21.15.66:0/2305314591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f40094d0 con 0x7fb50415a370 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.807+0000 7fb509000640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb504160350 0x7fb504154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.811+0000 7fb4fb400640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb50415a370 0x7fb5040b0010 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c7f0 tx=0x7fb4ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.811+0000 7fb500c00640 1 -- 172.21.15.66:0/2305314591 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec00cea0 con 0x7fb50415a370 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.811+0000 7fb500c00640 1 -- 172.21.15.66:0/2305314591 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec007a30 con 0x7fb50415a370 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.811+0000 7fb50b000640 1 -- 172.21.15.66:0/2305314591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb504153850 con 0x7fb50415a370 2024-03-30T00:51:24.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.811+0000 7fb500c00640 1 -- 172.21.15.66:0/2305314591 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec007de0 con 0x7fb50415a370 2024-03-30T00:51:24.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.811+0000 7fb50b000640 1 -- 172.21.15.66:0/2305314591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb504153e10 con 0x7fb50415a370 2024-03-30T00:51:24.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.811+0000 7fb50b000640 1 -- 172.21.15.66:0/2305314591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb504154330 con 0x7fb50415a370 2024-03-30T00:51:24.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.811+0000 7fb500c00640 1 -- 172.21.15.66:0/2305314591 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb4ec0113f0 con 0x7fb50415a370 2024-03-30T00:51:24.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.811+0000 7fb500c00640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4e0066690 0x7fb4e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:24.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.811+0000 7fb500c00640 1 -- 172.21.15.66:0/2305314591 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fb4ec0842f0 con 0x7fb50415a370 2024-03-30T00:51:24.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.811+0000 7fb503e00640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4e0066690 0x7fb4e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:24.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.815+0000 7fb503e00640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4e0066690 0x7fb4e0068b50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40b2000 tx=0x7fb4f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:24.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.819+0000 7fb500c00640 1 -- 172.21.15.66:0/2305314591 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb504154330 con 0x7fb50415a370 2024-03-30T00:51:24.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.979+0000 7fb50b000640 1 -- 172.21.15.66:0/2305314591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb504154330 con 0x7fb50415a370 2024-03-30T00:51:24.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb500c00640 1 -- 172.21.15.66:0/2305314591 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fb504154330 con 0x7fb50415a370 2024-03-30T00:51:24.990 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:24.990 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":126,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:51:15.162483+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6836/2683418851":"2024-03-31T00:51:15.162460+0000","172.21.15.66:6834/2683418851":"2024-03-31T00:51:15.162460+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:51:24.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 -- 172.21.15.66:0/2305314591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4e0066690 msgr2=0x7fb4e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:24.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4e0066690 0x7fb4e0068b50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40b2000 tx=0x7fb4f4009040 comp rx=0 tx=0).stop 2024-03-30T00:51:24.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 -- 172.21.15.66:0/2305314591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb50415a370 msgr2=0x7fb5040b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:24.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb50415a370 0x7fb5040b0010 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c7f0 tx=0x7fb4ec00ccb0 comp rx=0 tx=0).stop 2024-03-30T00:51:24.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 -- 172.21.15.66:0/2305314591 shutdown_connections 2024-03-30T00:51:24.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb504160350 0x7fb504154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:24.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb4e0066690 0x7fb4e0068b50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:24.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb50415a370 0x7fb5040b0010 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:24.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 --2- 172.21.15.66:0/2305314591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb50406ad00 0x7fb504152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:24.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 -- 172.21.15.66:0/2305314591 >> 172.21.15.66:0/2305314591 conn(0x7fb5040706e0 msgr2=0x7fb50406d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:24.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 -- 172.21.15.66:0/2305314591 shutdown_connections 2024-03-30T00:51:24.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:24.983+0000 7fb50b000640 1 -- 172.21.15.66:0/2305314591 wait complete. 2024-03-30T00:51:25.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:50:35.696965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:50:37.037165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:51:25.012 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:51:25.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f167de00640 1 Processor -- start 2024-03-30T00:51:25.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f167de00640 1 -- start start 2024-03-30T00:51:25.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f167de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f16780b1810 0x7f16780c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f167de00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f16780c4780 0x7f16780c9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f167de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780c8970 con 0x7f16781596a0 2024-03-30T00:51:25.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f167de00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16780cceb0 con 0x7f16780b1810 2024-03-30T00:51:25.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f167de00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16780cbb40 con 0x7f16780c4780 2024-03-30T00:51:25.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f166fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f16780b1810 0x7f16780c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f166fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f16780b1810 0x7f16780c4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33170/0 (socket says 172.21.15.66:33170) 2024-03-30T00:51:25.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f166fe00640 1 -- 172.21.15.66:0/1531342075 learned_addr learned my addr 172.21.15.66:0/1531342075 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:25.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f1676a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f16781596a0 0x7f16780cc7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:32984/0 (socket says 172.21.15.66:32984) 2024-03-30T00:51:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f1677400640 1 --2- 172.21.15.66:0/1531342075 >> v2:172.21.15.156:3300/0 conn(0x7f16780c4780 0x7f16780c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f166fe00640 1 -- 172.21.15.66:0/1531342075 >> v2:172.21.15.156:3300/0 conn(0x7f16780c4780 msgr2=0x7f16780c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f166fe00640 1 --2- 172.21.15.66:0/1531342075 >> v2:172.21.15.156:3300/0 conn(0x7f16780c4780 0x7f16780c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f166fe00640 1 -- 172.21.15.66:0/1531342075 >> v1:172.21.15.66:6789/0 conn(0x7f16781596a0 legacy=0x7f16780cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f166fe00640 1 -- 172.21.15.66:0/1531342075 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780b1ca0 con 0x7f16780b1810 2024-03-30T00:51:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.147+0000 7f1676000640 1 -- 172.21.15.66:0/1531342075 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776895111 0 0) 0x7f16780c8970 con 0x7f16781596a0 2024-03-30T00:51:25.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f166fe00640 1 --2- 172.21.15.66:0/1531342075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16780b1810 0x7f16780c4240 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f16780cbfb0 tx=0x7f166001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb91318f2457e1a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:25.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f1676000640 1 -- 172.21.15.66:0/1531342075 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16600a7ac0 con 0x7f16780b1810 2024-03-30T00:51:25.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f1676000640 1 -- 172.21.15.66:0/1531342075 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16600a7c20 con 0x7f16780b1810 2024-03-30T00:51:25.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f1676000640 1 -- 172.21.15.66:0/1531342075 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16600af440 con 0x7f16780b1810 2024-03-30T00:51:25.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f167de00640 1 -- 172.21.15.66:0/1531342075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16780b1810 msgr2=0x7f16780c4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f167de00640 1 --2- 172.21.15.66:0/1531342075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16780b1810 0x7f16780c4240 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f16780cbfb0 tx=0x7f166001ebf0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f167de00640 1 -- 172.21.15.66:0/1531342075 shutdown_connections 2024-03-30T00:51:25.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f167de00640 1 --2- 172.21.15.66:0/1531342075 >> v2:172.21.15.156:3300/0 conn(0x7f16780c4780 0x7f16780c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f167de00640 1 --2- 172.21.15.66:0/1531342075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16780b1810 0x7f16780c4240 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f167de00640 1 -- 172.21.15.66:0/1531342075 >> 172.21.15.66:0/1531342075 conn(0x7f1678069cf0 msgr2=0x7f167806a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:25.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f167de00640 1 -- 172.21.15.66:0/1531342075 shutdown_connections 2024-03-30T00:51:25.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.151+0000 7f167de00640 1 -- 172.21.15.66:0/1531342075 wait complete. 2024-03-30T00:51:25.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f167de00640 1 Processor -- start 2024-03-30T00:51:25.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f167de00640 1 -- start start 2024-03-30T00:51:25.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f167de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16780b1810 0x7f16780af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f167de00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16780c4780 0x7f16781625c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f167de00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1678162b00 0x7f1678154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f167de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678162f90 con 0x7f16780b1810 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f167de00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780afcc0 con 0x7f1678162b00 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f167de00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167806e2a0 con 0x7f16780c4780 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f1676a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16780b1810 0x7f16780af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f1676a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16780b1810 0x7f16780af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33172/0 (socket says 172.21.15.66:33172) 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f1676a00640 1 -- 172.21.15.66:0/939639648 learned_addr learned my addr 172.21.15.66:0/939639648 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f166fe00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16780c4780 0x7f16781625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f1677400640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1678162b00 0x7f1678154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f1676a00640 1 -- 172.21.15.66:0/939639648 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16780c4780 msgr2=0x7f16781625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f1676a00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16780c4780 0x7f16781625c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f1676a00640 1 -- 172.21.15.66:0/939639648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1678162b00 msgr2=0x7f1678154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f1676a00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1678162b00 0x7f1678154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f1676a00640 1 -- 172.21.15.66:0/939639648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166000b770 con 0x7f16780b1810 2024-03-30T00:51:25.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f1676a00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16780b1810 0x7f16780af5c0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f166800c480 tx=0x7f166800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:25.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f166f400640 1 -- 172.21.15.66:0/939639648 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680074e0 con 0x7f16780b1810 2024-03-30T00:51:25.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f166f400640 1 -- 172.21.15.66:0/939639648 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1668007660 con 0x7f16780b1810 2024-03-30T00:51:25.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f167de00640 1 -- 172.21.15.66:0/939639648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16780ab340 con 0x7f16780b1810 2024-03-30T00:51:25.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f166f400640 1 -- 172.21.15.66:0/939639648 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668004a30 con 0x7f16780b1810 2024-03-30T00:51:25.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.155+0000 7f167de00640 1 -- 172.21.15.66:0/939639648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16780ab900 con 0x7f16780b1810 2024-03-30T00:51:25.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.159+0000 7f167de00640 1 -- 172.21.15.66:0/939639648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167804e0b0 con 0x7f16780b1810 2024-03-30T00:51:25.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.159+0000 7f166f400640 1 -- 172.21.15.66:0/939639648 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1668004b90 con 0x7f16780b1810 2024-03-30T00:51:25.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.159+0000 7f166f400640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1654066690 0x7f1654068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.159+0000 7f166f400640 1 -- 172.21.15.66:0/939639648 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f16680837b0 con 0x7f16780b1810 2024-03-30T00:51:25.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.159+0000 7f166fe00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1654066690 0x7f1654068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.159+0000 7f166fe00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1654066690 0x7f1654068b50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f16600b1000 tx=0x7f16600a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:25.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.163+0000 7f166f400640 1 -- 172.21.15.66:0/939639648 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1668050a20 con 0x7f16780b1810 2024-03-30T00:51:25.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 -- 172.21.15.66:0/939639648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1678071f40 con 0x7f16780b1810 2024-03-30T00:51:25.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f166f400640 1 -- 172.21.15.66:0/939639648 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1668018020 con 0x7f16780b1810 2024-03-30T00:51:25.334 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:51:25.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 -- 172.21.15.66:0/939639648 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1654066690 msgr2=0x7f1654068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1654066690 0x7f1654068b50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f16600b1000 tx=0x7f16600a9070 comp rx=0 tx=0).stop 2024-03-30T00:51:25.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 -- 172.21.15.66:0/939639648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16780b1810 msgr2=0x7f16780af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16780b1810 0x7f16780af5c0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f166800c480 tx=0x7f166800c940 comp rx=0 tx=0).stop 2024-03-30T00:51:25.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 -- 172.21.15.66:0/939639648 shutdown_connections 2024-03-30T00:51:25.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1678162b00 0x7f1678154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16780c4780 0x7f16781625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f1654066690 0x7f1654068b50 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 --2- 172.21.15.66:0/939639648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16780b1810 0x7f16780af5c0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 -- 172.21.15.66:0/939639648 >> 172.21.15.66:0/939639648 conn(0x7f1678069cf0 msgr2=0x7f16780b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:25.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 -- 172.21.15.66:0/939639648 shutdown_connections 2024-03-30T00:51:25.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.327+0000 7f167de00640 1 -- 172.21.15.66:0/939639648 wait complete. 2024-03-30T00:51:25.355 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:51:25.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.491+0000 7f488f400640 1 Processor -- start 2024-03-30T00:51:25.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.491+0000 7f488f400640 1 -- start start 2024-03-30T00:51:25.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.491+0000 7f488f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880c44a0 0x7f48880ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.491+0000 7f488f400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f48880b2870 0x7f48880b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.491+0000 7f488f400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48880cd210 con 0x7f48880cb1a0 2024-03-30T00:51:25.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.491+0000 7f488f400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48880ce030 con 0x7f48880b2870 2024-03-30T00:51:25.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.491+0000 7f488f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48880b3220 con 0x7f48880c44a0 2024-03-30T00:51:25.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.491+0000 7f488ca00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f48880cb1a0 0x7f48880c3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:32996/0 (socket says 172.21.15.66:32996) 2024-03-30T00:51:25.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f488ca00640 1 -- 172.21.15.66:0/1738342371 learned_addr learned my addr 172.21.15.66:0/1738342371 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:25.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f487fe00640 1 --2- 172.21.15.66:0/1738342371 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880c44a0 0x7f48880ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f488d400640 1 --2- 172.21.15.66:0/1738342371 >> v2:172.21.15.156:3300/0 conn(0x7f48880b2870 0x7f48880b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f4887e00640 1 -- 172.21.15.66:0/1738342371 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713446532 0 0) 0x7f48880cd210 con 0x7f48880cb1a0 2024-03-30T00:51:25.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f4887e00640 1 -- 172.21.15.66:0/1738342371 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f486c003530 con 0x7f48880cb1a0 2024-03-30T00:51:25.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f487fe00640 1 -- 172.21.15.66:0/1738342371 >> v2:172.21.15.156:3300/0 conn(0x7f48880b2870 msgr2=0x7f48880b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:25.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f4887e00640 1 -- 172.21.15.66:0/1738342371 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3169617344 0 0) 0x7f486c003530 con 0x7f48880cb1a0 2024-03-30T00:51:25.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f487fe00640 1 --2- 172.21.15.66:0/1738342371 >> v2:172.21.15.156:3300/0 conn(0x7f48880b2870 0x7f48880b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f487fe00640 1 -- 172.21.15.66:0/1738342371 >> v1:172.21.15.66:6789/0 conn(0x7f48880cb1a0 legacy=0x7f48880c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f487fe00640 1 -- 172.21.15.66:0/1738342371 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48880cd4b0 con 0x7f48880c44a0 2024-03-30T00:51:25.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f488d400640 1 --2- 172.21.15.66:0/1738342371 >> v2:172.21.15.156:3300/0 conn(0x7f48880b2870 0x7f48880b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:51:25.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f487fe00640 1 --2- 172.21.15.66:0/1738342371 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880c44a0 0x7f48880ab1a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48700097c0 tx=0x7f487001c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ae8d03209d673c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:25.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f4887e00640 1 -- 172.21.15.66:0/1738342371 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878001680 con 0x7f48880c44a0 2024-03-30T00:51:25.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f4887e00640 1 -- 172.21.15.66:0/1738342371 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48700a7050 con 0x7f48880c44a0 2024-03-30T00:51:25.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.495+0000 7f4887e00640 1 -- 172.21.15.66:0/1738342371 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700ab620 con 0x7f48880c44a0 2024-03-30T00:51:25.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 -- 172.21.15.66:0/1738342371 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880c44a0 msgr2=0x7f48880ab1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 --2- 172.21.15.66:0/1738342371 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880c44a0 0x7f48880ab1a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48700097c0 tx=0x7f487001c390 comp rx=0 tx=0).stop 2024-03-30T00:51:25.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 -- 172.21.15.66:0/1738342371 shutdown_connections 2024-03-30T00:51:25.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 --2- 172.21.15.66:0/1738342371 >> v2:172.21.15.156:3300/0 conn(0x7f48880b2870 0x7f48880b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 --2- 172.21.15.66:0/1738342371 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880c44a0 0x7f48880ab1a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 -- 172.21.15.66:0/1738342371 >> 172.21.15.66:0/1738342371 conn(0x7f4888069cf0 msgr2=0x7f488806a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:25.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 -- 172.21.15.66:0/1738342371 shutdown_connections 2024-03-30T00:51:25.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 -- 172.21.15.66:0/1738342371 wait complete. 2024-03-30T00:51:25.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 Processor -- start 2024-03-30T00:51:25.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 -- start start 2024-03-30T00:51:25.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880b2870 0x7f4888158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48880c44a0 0x7f488815bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48880cb1a0 0x7f488814e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48880722c0 con 0x7f48880cb1a0 2024-03-30T00:51:25.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488815ac60 con 0x7f48880c44a0 2024-03-30T00:51:25.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48881507d0 con 0x7f48880b2870 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48880cb1a0 0x7f488814e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48880cb1a0 0x7f488814e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33182/0 (socket says 172.21.15.66:33182) 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488d400640 1 -- 172.21.15.66:0/1650727222 learned_addr learned my addr 172.21.15.66:0/1650727222 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488ca00640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880b2870 0x7f4888158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f487fe00640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48880c44a0 0x7f488815bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488d400640 1 -- 172.21.15.66:0/1650727222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880b2870 msgr2=0x7f4888158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488d400640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880b2870 0x7f4888158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488d400640 1 -- 172.21.15.66:0/1650727222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48880c44a0 msgr2=0x7f488815bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488d400640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48880c44a0 0x7f488815bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488d400640 1 -- 172.21.15.66:0/1650727222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4880009a70 con 0x7f48880cb1a0 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.499+0000 7f488ca00640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880b2870 0x7f4888158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:25.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f488d400640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48880cb1a0 0x7f488814e390 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f488000efc0 tx=0x7f4880009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:25.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f4885600640 1 -- 172.21.15.66:0/1650727222 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880009d70 con 0x7f48880cb1a0 2024-03-30T00:51:25.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f4885600640 1 -- 172.21.15.66:0/1650727222 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488000f040 con 0x7f48880cb1a0 2024-03-30T00:51:25.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f4885600640 1 -- 172.21.15.66:0/1650727222 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48800039a0 con 0x7f48880cb1a0 2024-03-30T00:51:25.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f488f400640 1 -- 172.21.15.66:0/1650727222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4870009420 con 0x7f48880cb1a0 2024-03-30T00:51:25.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f488f400640 1 -- 172.21.15.66:0/1650727222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488815af00 con 0x7f48880cb1a0 2024-03-30T00:51:25.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f488f400640 1 -- 172.21.15.66:0/1650727222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4888150990 con 0x7f48880cb1a0 2024-03-30T00:51:25.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f4885600640 1 -- 172.21.15.66:0/1650727222 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f48800144a0 con 0x7f48880cb1a0 2024-03-30T00:51:25.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f4885600640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48640666e0 0x7f4864068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f4885600640 1 -- 172.21.15.66:0/1650727222 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f4880084f90 con 0x7f48880cb1a0 2024-03-30T00:51:25.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f488ca00640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48640666e0 0x7f4864068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.503+0000 7f488ca00640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48640666e0 0x7f4864068ba0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4878007a10 tx=0x7f4878009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:25.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.507+0000 7f4885600640 1 -- 172.21.15.66:0/1650727222 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4880052140 con 0x7f48880cb1a0 2024-03-30T00:51:25.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 -- 172.21.15.66:0/1650727222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f488804e0b0 con 0x7f48880cb1a0 2024-03-30T00:51:25.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f4885600640 1 -- 172.21.15.66:0/1650727222 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4880055760 con 0x7f48880cb1a0 2024-03-30T00:51:25.684 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:51:25.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 -- 172.21.15.66:0/1650727222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48640666e0 msgr2=0x7f4864068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48640666e0 0x7f4864068ba0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4878007a10 tx=0x7f4878009040 comp rx=0 tx=0).stop 2024-03-30T00:51:25.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 -- 172.21.15.66:0/1650727222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48880cb1a0 msgr2=0x7f488814e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48880cb1a0 0x7f488814e390 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f488000efc0 tx=0x7f4880009f90 comp rx=0 tx=0).stop 2024-03-30T00:51:25.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 -- 172.21.15.66:0/1650727222 shutdown_connections 2024-03-30T00:51:25.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f48640666e0 0x7f4864068ba0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48880cb1a0 0x7f488814e390 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48880c44a0 0x7f488815bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 --2- 172.21.15.66:0/1650727222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48880b2870 0x7f4888158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 -- 172.21.15.66:0/1650727222 >> 172.21.15.66:0/1650727222 conn(0x7f4888069cf0 msgr2=0x7f4888159020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:25.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 -- 172.21.15.66:0/1650727222 shutdown_connections 2024-03-30T00:51:25.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.675+0000 7f488f400640 1 -- 172.21.15.66:0/1650727222 wait complete. 2024-03-30T00:51:25.705 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:51:25.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9ab200640 1 Processor -- start 2024-03-30T00:51:25.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9ab200640 1 -- start start 2024-03-30T00:51:25.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9ab200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9ac06a400 0x7fa9ac070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9ab200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa9ac15cbc0 0x7fa9ac160580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9ab200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ac06abf0 con 0x7fa9ac15c7e0 2024-03-30T00:51:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9ab200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ac15d090 con 0x7fa9ac15cbc0 2024-03-30T00:51:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9ab200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac15d370 con 0x7fa9ac06a400 2024-03-30T00:51:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9aa800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa9ac15cbc0 0x7fa9ac160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9aa800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa9ac15cbc0 0x7fa9ac160580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33196/0 (socket says 172.21.15.66:33196) 2024-03-30T00:51:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9aa800640 1 -- 172.21.15.66:0/2638479623 learned_addr learned my addr 172.21.15.66:0/2638479623 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9a3e00640 1 --2- 172.21.15.66:0/2638479623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9ac06a400 0x7fa9ac070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9aa800640 1 -- 172.21.15.66:0/2638479623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9ac06a400 msgr2=0x7fa9ac070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9aa800640 1 --2- 172.21.15.66:0/2638479623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9ac06a400 0x7fa9ac070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9aa800640 1 -- 172.21.15.66:0/2638479623 >> v1:172.21.15.156:6789/0 conn(0x7fa9ac15c7e0 legacy=0x7fa9ac069cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.843+0000 7fa9aa800640 1 -- 172.21.15.66:0/2638479623 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac15d650 con 0x7fa9ac15cbc0 2024-03-30T00:51:25.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9aa800640 1 --2- 172.21.15.66:0/2638479623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9ac15cbc0 0x7fa9ac160580 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009e50 tx=0x7fa9a401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a117b2f0316d3bce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:25.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9a9400640 1 -- 172.21.15.66:0/2638479623 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40af400 con 0x7fa9ac15cbc0 2024-03-30T00:51:25.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9a9400640 1 -- 172.21.15.66:0/2638479623 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a40ab050 con 0x7fa9ac15cbc0 2024-03-30T00:51:25.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9a9400640 1 -- 172.21.15.66:0/2638479623 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40af780 con 0x7fa9ac15cbc0 2024-03-30T00:51:25.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9ab200640 1 -- 172.21.15.66:0/2638479623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9ac15cbc0 msgr2=0x7fa9ac160580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9ab200640 1 --2- 172.21.15.66:0/2638479623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9ac15cbc0 0x7fa9ac160580 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009e50 tx=0x7fa9a401f210 comp rx=0 tx=0).stop 2024-03-30T00:51:25.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9ab200640 1 -- 172.21.15.66:0/2638479623 shutdown_connections 2024-03-30T00:51:25.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9ab200640 1 --2- 172.21.15.66:0/2638479623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9ac15cbc0 0x7fa9ac160580 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9ab200640 1 --2- 172.21.15.66:0/2638479623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9ac06a400 0x7fa9ac070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9ab200640 1 -- 172.21.15.66:0/2638479623 >> 172.21.15.66:0/2638479623 conn(0x7fa9ac0cbd00 msgr2=0x7fa9ac0cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:25.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9ab200640 1 -- 172.21.15.66:0/2638479623 shutdown_connections 2024-03-30T00:51:25.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.847+0000 7fa9ab200640 1 -- 172.21.15.66:0/2638479623 wait complete. 2024-03-30T00:51:25.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9ab200640 1 Processor -- start 2024-03-30T00:51:25.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9ab200640 1 -- start start 2024-03-30T00:51:25.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9ab200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9ac06a400 0x7fa9ac0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9ab200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9ac15c7e0 0x7fa9ac0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9ab200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9ac15cbc0 0x7fa9ac14a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9ab200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0c5900 con 0x7fa9ac15c7e0 2024-03-30T00:51:25.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9ab200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0c4cf0 con 0x7fa9ac15cbc0 2024-03-30T00:51:25.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9ab200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0c4fd0 con 0x7fa9ac06a400 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9ac15c7e0 0x7fa9ac0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9ac15c7e0 0x7fa9ac0c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33210/0 (socket says 172.21.15.66:33210) 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a3e00640 1 -- 172.21.15.66:0/3699047418 learned_addr learned my addr 172.21.15.66:0/3699047418 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a9e00640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9ac06a400 0x7fa9ac0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9aa800640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9ac15cbc0 0x7fa9ac14a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a3e00640 1 -- 172.21.15.66:0/3699047418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9ac06a400 msgr2=0x7fa9ac0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a3e00640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9ac06a400 0x7fa9ac0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a3e00640 1 -- 172.21.15.66:0/3699047418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9ac15cbc0 msgr2=0x7fa9ac14a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a3e00640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9ac15cbc0 0x7fa9ac14a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a3e00640 1 -- 172.21.15.66:0/3699047418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a4009b30 con 0x7fa9ac15c7e0 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a3e00640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9ac15c7e0 0x7fa9ac0c4280 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fa99400d800 tx=0x7fa99400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:25.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a2000640 1 -- 172.21.15.66:0/3699047418 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99400b6c0 con 0x7fa9ac15c7e0 2024-03-30T00:51:25.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a2000640 1 -- 172.21.15.66:0/3699047418 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99400b820 con 0x7fa9ac15c7e0 2024-03-30T00:51:25.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9a2000640 1 -- 172.21.15.66:0/3699047418 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99400bb90 con 0x7fa9ac15c7e0 2024-03-30T00:51:25.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9ab200640 1 -- 172.21.15.66:0/3699047418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ac0c52b0 con 0x7fa9ac15c7e0 2024-03-30T00:51:25.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9ab200640 1 -- 172.21.15.66:0/3699047418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ac0b4de0 con 0x7fa9ac15c7e0 2024-03-30T00:51:25.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.851+0000 7fa9ab200640 1 -- 172.21.15.66:0/3699047418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ac04e040 con 0x7fa9ac15c7e0 2024-03-30T00:51:25.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.855+0000 7fa9a2000640 1 -- 172.21.15.66:0/3699047418 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa99400bcf0 con 0x7fa9ac15c7e0 2024-03-30T00:51:25.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.855+0000 7fa9a2000640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa988066690 0x7fa988068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:25.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.855+0000 7fa9a2000640 1 -- 172.21.15.66:0/3699047418 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fa9940847f0 con 0x7fa9ac15c7e0 2024-03-30T00:51:25.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.855+0000 7fa9a9e00640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa988066690 0x7fa988068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:25.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.855+0000 7fa9a9e00640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa988066690 0x7fa988068b50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa99c007fd0 tx=0x7fa99c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:25.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:25.859+0000 7fa9a2000640 1 -- 172.21.15.66:0/3699047418 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa9940519d0 con 0x7fa9ac15c7e0 2024-03-30T00:51:26.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 -- 172.21.15.66:0/3699047418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9ac071f80 con 0x7fa9ac15c7e0 2024-03-30T00:51:26.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9a2000640 1 -- 172.21.15.66:0/3699047418 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa994054ff0 con 0x7fa9ac15c7e0 2024-03-30T00:51:26.027 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:51:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 -- 172.21.15.66:0/3699047418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa988066690 msgr2=0x7fa988068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa988066690 0x7fa988068b50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa99c007fd0 tx=0x7fa99c009040 comp rx=0 tx=0).stop 2024-03-30T00:51:26.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 -- 172.21.15.66:0/3699047418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9ac15c7e0 msgr2=0x7fa9ac0c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9ac15c7e0 0x7fa9ac0c4280 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fa99400d800 tx=0x7fa99400dcc0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 -- 172.21.15.66:0/3699047418 shutdown_connections 2024-03-30T00:51:26.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9ac15cbc0 0x7fa9ac14a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa988066690 0x7fa988068b50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9ac15c7e0 0x7fa9ac0c4280 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 --2- 172.21.15.66:0/3699047418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9ac06a400 0x7fa9ac0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 -- 172.21.15.66:0/3699047418 >> 172.21.15.66:0/3699047418 conn(0x7fa9ac0cbd00 msgr2=0x7fa9ac15dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:26.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 -- 172.21.15.66:0/3699047418 shutdown_connections 2024-03-30T00:51:26.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.019+0000 7fa9ab200640 1 -- 172.21.15.66:0/3699047418 wait complete. 2024-03-30T00:51:26.049 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:51:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.183+0000 7f71bc400640 1 Processor -- start 2024-03-30T00:51:26.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.183+0000 7f71bc400640 1 -- start start 2024-03-30T00:51:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.183+0000 7f71bc400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f71b40ce940 0x7f71b40ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.183+0000 7f71bc400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b40cd0f0 con 0x7f71b4159860 2024-03-30T00:51:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.183+0000 7f71bc400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b40b1830 con 0x7f71b415d3f0 2024-03-30T00:51:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.183+0000 7f71bc400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b415d9b0 con 0x7f71b40ce940 2024-03-30T00:51:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.183+0000 7f71b9a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f71b4159860 0x7f71b40ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33000/0 (socket says 172.21.15.66:33000) 2024-03-30T00:51:26.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.183+0000 7f71b9a00640 1 -- 172.21.15.66:0/1118641887 learned_addr learned my addr 172.21.15.66:0/1118641887 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b9000640 1 --2- 172.21.15.66:0/1118641887 >> v2:172.21.15.156:3300/0 conn(0x7f71b40ce940 0x7f71b40ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b3e00640 1 -- 172.21.15.66:0/1118641887 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412789783 0 0) 0x7f71a8001120 con 0x7f71b4159860 2024-03-30T00:51:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b3e00640 1 -- 172.21.15.66:0/1118641887 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f719c003530 con 0x7f71b4159860 2024-03-30T00:51:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b3e00640 1 -- 172.21.15.66:0/1118641887 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089183685 0 0) 0x7f71b40b1830 con 0x7f71b415d3f0 2024-03-30T00:51:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b3e00640 1 -- 172.21.15.66:0/1118641887 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71a8001120 con 0x7f71b415d3f0 2024-03-30T00:51:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b3e00640 1 -- 172.21.15.66:0/1118641887 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3791065926 0 0) 0x7f719c003530 con 0x7f71b4159860 2024-03-30T00:51:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b9000640 1 -- 172.21.15.66:0/1118641887 >> v1:172.21.15.156:6789/0 conn(0x7f71b415d3f0 legacy=0x7f71b40b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b9000640 1 -- 172.21.15.66:0/1118641887 >> v1:172.21.15.66:6789/0 conn(0x7f71b4159860 legacy=0x7f71b40ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b9000640 1 -- 172.21.15.66:0/1118641887 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b415dc90 con 0x7f71b40ce940 2024-03-30T00:51:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b9000640 1 --2- 172.21.15.66:0/1118641887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71b40ce940 0x7f71b40ccbb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f71a00097c0 tx=0x7f71a001c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=969411b84be0eed5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b3e00640 1 -- 172.21.15.66:0/1118641887 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8001810 con 0x7f71b40ce940 2024-03-30T00:51:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b3e00640 1 -- 172.21.15.66:0/1118641887 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a00a7050 con 0x7f71b40ce940 2024-03-30T00:51:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.187+0000 7f71b3e00640 1 -- 172.21.15.66:0/1118641887 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00ab620 con 0x7f71b40ce940 2024-03-30T00:51:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 -- 172.21.15.66:0/1118641887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71b40ce940 msgr2=0x7f71b40ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 --2- 172.21.15.66:0/1118641887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71b40ce940 0x7f71b40ccbb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f71a00097c0 tx=0x7f71a001c300 comp rx=0 tx=0).stop 2024-03-30T00:51:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 -- 172.21.15.66:0/1118641887 shutdown_connections 2024-03-30T00:51:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 --2- 172.21.15.66:0/1118641887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71b40ce940 0x7f71b40ccbb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 -- 172.21.15.66:0/1118641887 >> 172.21.15.66:0/1118641887 conn(0x7f71b4069cf0 msgr2=0x7f71b406a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 -- 172.21.15.66:0/1118641887 shutdown_connections 2024-03-30T00:51:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 -- 172.21.15.66:0/1118641887 wait complete. 2024-03-30T00:51:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 Processor -- start 2024-03-30T00:51:26.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 -- start start 2024-03-30T00:51:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71b40ce940 0x7f71b406fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b4159860 0x7f71b40979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71b415d3f0 0x7f71b41628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b40ad3c0 con 0x7f71b4159860 2024-03-30T00:51:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b4097f20 con 0x7f71b40ce940 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71bc400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b406d000 con 0x7f71b415d3f0 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71b9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b4159860 0x7f71b40979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71b9000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b4159860 0x7f71b40979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33212/0 (socket says 172.21.15.66:33212) 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71b9000640 1 -- 172.21.15.66:0/1997345255 learned_addr learned my addr 172.21.15.66:0/1997345255 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71b9a00640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71b40ce940 0x7f71b406fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71ba400640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71b415d3f0 0x7f71b41628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71b9000640 1 -- 172.21.15.66:0/1997345255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71b415d3f0 msgr2=0x7f71b41628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71b9000640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71b415d3f0 0x7f71b41628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71b9000640 1 -- 172.21.15.66:0/1997345255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71b40ce940 msgr2=0x7f71b406fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71b9000640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71b40ce940 0x7f71b406fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71b9000640 1 -- 172.21.15.66:0/1997345255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0009420 con 0x7f71b4159860 2024-03-30T00:51:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.191+0000 7f71b9000640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b4159860 0x7f71b40979e0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f71a001c7e0 tx=0x7f71a001ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.195+0000 7f71b1600640 1 -- 172.21.15.66:0/1997345255 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00bc070 con 0x7f71b4159860 2024-03-30T00:51:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.195+0000 7f71b1600640 1 -- 172.21.15.66:0/1997345255 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a00abcd0 con 0x7f71b4159860 2024-03-30T00:51:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.195+0000 7f71b1600640 1 -- 172.21.15.66:0/1997345255 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0004240 con 0x7f71b4159860 2024-03-30T00:51:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.195+0000 7f71bc400640 1 -- 172.21.15.66:0/1997345255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b40ac5f0 con 0x7f71b4159860 2024-03-30T00:51:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.195+0000 7f71bc400640 1 -- 172.21.15.66:0/1997345255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b40acb20 con 0x7f71b4159860 2024-03-30T00:51:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.195+0000 7f71bc400640 1 -- 172.21.15.66:0/1997345255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b4066c40 con 0x7f71b4159860 2024-03-30T00:51:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.195+0000 7f71b1600640 1 -- 172.21.15.66:0/1997345255 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f71a00043a0 con 0x7f71b4159860 2024-03-30T00:51:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.195+0000 7f71b1600640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7194066690 0x7f7194068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.195+0000 7f71b1600640 1 -- 172.21.15.66:0/1997345255 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f71a011e4c0 con 0x7f71b4159860 2024-03-30T00:51:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.195+0000 7f71b9a00640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7194066690 0x7f7194068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.199+0000 7f71b9a00640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7194066690 0x7f7194068b50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f71a8000f70 tx=0x7f71a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.199+0000 7f71b1600640 1 -- 172.21.15.66:0/1997345255 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f71a00eb560 con 0x7f71b4159860 2024-03-30T00:51:26.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.371+0000 7f71bc400640 1 -- 172.21.15.66:0/1997345255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f71b40c9e30 con 0x7f71b4159860 2024-03-30T00:51:26.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.371+0000 7f71b1600640 1 -- 172.21.15.66:0/1997345255 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v126) v1 ==== 80+0+122 (secure 0 0 0) 0x7f71a00b0090 con 0x7f71b4159860 2024-03-30T00:51:26.382 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6836/2683418851 2024-03-31T00:51:15.162460+0000 2024-03-30T00:51:26.382 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6834/2683418851 2024-03-31T00:51:15.162460+0000 2024-03-30T00:51:26.383 INFO:teuthology.orchestra.run.smithi066.stderr:listed 2 entries 2024-03-30T00:51:26.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 -- 172.21.15.66:0/1997345255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7194066690 msgr2=0x7f7194068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7194066690 0x7f7194068b50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f71a8000f70 tx=0x7f71a8009040 comp rx=0 tx=0).stop 2024-03-30T00:51:26.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 -- 172.21.15.66:0/1997345255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b4159860 msgr2=0x7f71b40979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b4159860 0x7f71b40979e0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f71a001c7e0 tx=0x7f71a001ab40 comp rx=0 tx=0).stop 2024-03-30T00:51:26.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 -- 172.21.15.66:0/1997345255 shutdown_connections 2024-03-30T00:51:26.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f71b415d3f0 0x7f71b41628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7194066690 0x7f7194068b50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71b4159860 0x7f71b40979e0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 --2- 172.21.15.66:0/1997345255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71b40ce940 0x7f71b406fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 -- 172.21.15.66:0/1997345255 >> 172.21.15.66:0/1997345255 conn(0x7f71b4069cf0 msgr2=0x7f71b406d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 -- 172.21.15.66:0/1997345255 shutdown_connections 2024-03-30T00:51:26.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.375+0000 7f71bc400640 1 -- 172.21.15.66:0/1997345255 wait complete. 2024-03-30T00:51:26.403 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:51:26.403 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:26.412 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:51:26.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.587+0000 7f80ac200640 1 Processor -- start 2024-03-30T00:51:26.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.587+0000 7f80ac200640 1 -- start start 2024-03-30T00:51:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.587+0000 7f80ac200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a40ccd10 0x7f80a40cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.587+0000 7f80ac200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80a415cc00 0x7f80a40cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.587+0000 7f80ac200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a40cb1d0 con 0x7f80a406c100 2024-03-30T00:51:26.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.587+0000 7f80ac200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a40cd630 con 0x7f80a415cc00 2024-03-30T00:51:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.587+0000 7f80ac200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40cfcb0 con 0x7f80a40ccd10 2024-03-30T00:51:26.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80a9800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a40ccd10 0x7f80a40cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80a8e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80a415cc00 0x7f80a40cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80aa200640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f80a406c100 0x7f80a406b010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:33656/0 (socket says 172.21.15.66:33656) 2024-03-30T00:51:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80a8e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80a415cc00 0x7f80a40cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:39486/0 (socket says 172.21.15.66:39486) 2024-03-30T00:51:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80a9800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a40ccd10 0x7f80a40cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:46390/0 (socket says 172.21.15.66:46390) 2024-03-30T00:51:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80aa200640 1 -- 172.21.15.66:0/2667823787 learned_addr learned my addr 172.21.15.66:0/2667823787 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80a3e00640 1 -- 172.21.15.66:0/2667823787 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280626288 0 0) 0x7f80a40cb1d0 con 0x7f80a406c100 2024-03-30T00:51:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80a9800640 1 -- 172.21.15.66:0/2667823787 >> v2:172.21.15.156:3300/0 conn(0x7f80a415cc00 msgr2=0x7f80a40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80a9800640 1 --2- 172.21.15.66:0/2667823787 >> v2:172.21.15.156:3300/0 conn(0x7f80a415cc00 0x7f80a40cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80a9800640 1 -- 172.21.15.66:0/2667823787 >> v1:172.21.15.156:6789/0 conn(0x7f80a406c100 legacy=0x7f80a406b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80a9800640 1 -- 172.21.15.66:0/2667823787 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a40cab80 con 0x7f80a40ccd10 2024-03-30T00:51:26.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.591+0000 7f80a9800640 1 --2- 172.21.15.66:0/2667823787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a40ccd10 0x7f80a40cd0f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f80980097c0 tx=0x7f809801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb3020e79cd6b6d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:26.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80a3e00640 1 -- 172.21.15.66:0/2667823787 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980ab610 con 0x7f80a40ccd10 2024-03-30T00:51:26.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80a3e00640 1 -- 172.21.15.66:0/2667823787 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80980ab770 con 0x7f80a40ccd10 2024-03-30T00:51:26.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80a3e00640 1 -- 172.21.15.66:0/2667823787 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980abb00 con 0x7f80a40ccd10 2024-03-30T00:51:26.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 -- 172.21.15.66:0/2667823787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a40ccd10 msgr2=0x7f80a40cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 --2- 172.21.15.66:0/2667823787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a40ccd10 0x7f80a40cd0f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f80980097c0 tx=0x7f809801c470 comp rx=0 tx=0).stop 2024-03-30T00:51:26.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 -- 172.21.15.66:0/2667823787 shutdown_connections 2024-03-30T00:51:26.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 --2- 172.21.15.66:0/2667823787 >> v2:172.21.15.156:3300/0 conn(0x7f80a415cc00 0x7f80a40cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 --2- 172.21.15.66:0/2667823787 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a40ccd10 0x7f80a40cd0f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 -- 172.21.15.66:0/2667823787 >> 172.21.15.66:0/2667823787 conn(0x7f80a40d0490 msgr2=0x7f80a40d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:26.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 -- 172.21.15.66:0/2667823787 shutdown_connections 2024-03-30T00:51:26.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 -- 172.21.15.66:0/2667823787 wait complete. 2024-03-30T00:51:26.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 Processor -- start 2024-03-30T00:51:26.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 -- start start 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a406c100 0x7f80a4151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80a40ccd10 0x7f80a40b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a415cc00 0x7f80a40c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a415aa70 con 0x7f80a406c100 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a415ad50 con 0x7f80a40ccd10 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80ac200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a415b030 con 0x7f80a415cc00 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80a9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a406c100 0x7f80a4151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80a9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a406c100 0x7f80a4151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33224/0 (socket says 172.21.15.66:33224) 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80a9800640 1 -- 172.21.15.66:0/1292950006 learned_addr learned my addr 172.21.15.66:0/1292950006 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.595+0000 7f80a8e00640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80a40ccd10 0x7f80a40b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80aa200640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a415cc00 0x7f80a40c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80a9800640 1 -- 172.21.15.66:0/1292950006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a415cc00 msgr2=0x7f80a40c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80a9800640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a415cc00 0x7f80a40c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80a9800640 1 -- 172.21.15.66:0/1292950006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80a40ccd10 msgr2=0x7f80a40b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80a9800640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80a40ccd10 0x7f80a40b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80a9800640 1 -- 172.21.15.66:0/1292950006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8098009420 con 0x7f80a406c100 2024-03-30T00:51:26.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80a9800640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a406c100 0x7f80a4151ca0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f80980ad040 tx=0x7f809801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80a1600640 1 -- 172.21.15.66:0/1292950006 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980bf610 con 0x7f80a406c100 2024-03-30T00:51:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80a1600640 1 -- 172.21.15.66:0/1292950006 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80980bf7b0 con 0x7f80a406c100 2024-03-30T00:51:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80a1600640 1 -- 172.21.15.66:0/1292950006 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980bfb60 con 0x7f80a406c100 2024-03-30T00:51:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80ac200640 1 -- 172.21.15.66:0/1292950006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a415b310 con 0x7f80a406c100 2024-03-30T00:51:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80ac200640 1 -- 172.21.15.66:0/1292950006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a41521e0 con 0x7f80a406c100 2024-03-30T00:51:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80ac200640 1 -- 172.21.15.66:0/1292950006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a4152370 con 0x7f80a406c100 2024-03-30T00:51:26.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80a1600640 1 -- 172.21.15.66:0/1292950006 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f80980b5400 con 0x7f80a406c100 2024-03-30T00:51:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.599+0000 7f80a1600640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8084066690 0x7f8084068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.603+0000 7f80a1600640 1 -- 172.21.15.66:0/1292950006 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f809811f890 con 0x7f80a406c100 2024-03-30T00:51:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.603+0000 7f80a8e00640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8084066690 0x7f8084068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.603+0000 7f80a8e00640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8084066690 0x7f8084068b50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8090009ac0 tx=0x7f8090009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.603+0000 7f80a1600640 1 -- 172.21.15.66:0/1292950006 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f80980ecb80 con 0x7f80a406c100 2024-03-30T00:51:26.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.767+0000 7f80ac200640 1 -- 172.21.15.66:0/1292950006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80a4066960 con 0x7f80a406c100 2024-03-30T00:51:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80a1600640 1 -- 172.21.15.66:0/1292950006 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f80980f01a0 con 0x7f80a406c100 2024-03-30T00:51:26.778 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:26.778 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":126,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:51:15.162483+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6836/2683418851":"2024-03-31T00:51:15.162460+0000","172.21.15.66:6834/2683418851":"2024-03-31T00:51:15.162460+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:51:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 -- 172.21.15.66:0/1292950006 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8084066690 msgr2=0x7f8084068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8084066690 0x7f8084068b50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8090009ac0 tx=0x7f8090009190 comp rx=0 tx=0).stop 2024-03-30T00:51:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 -- 172.21.15.66:0/1292950006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a406c100 msgr2=0x7f80a4151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a406c100 0x7f80a4151ca0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f80980ad040 tx=0x7f809801acb0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 -- 172.21.15.66:0/1292950006 shutdown_connections 2024-03-30T00:51:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80a415cc00 0x7f80a40c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80a40ccd10 0x7f80a40b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f8084066690 0x7f8084068b50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 --2- 172.21.15.66:0/1292950006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a406c100 0x7f80a4151ca0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 -- 172.21.15.66:0/1292950006 >> 172.21.15.66:0/1292950006 conn(0x7f80a40d0490 msgr2=0x7f80a4158830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 -- 172.21.15.66:0/1292950006 shutdown_connections 2024-03-30T00:51:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.771+0000 7f80ac200640 1 -- 172.21.15.66:0/1292950006 wait complete. 2024-03-30T00:51:26.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:50:35.696965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:50:37.037165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:51:26.800 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:51:26.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.935+0000 7f53e7600640 1 Processor -- start 2024-03-30T00:51:26.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.935+0000 7f53e7600640 1 -- start start 2024-03-30T00:51:26.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e7600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e01596a0 0x7f53e00b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e7600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53e00c4950 0x7f53e00c9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e7600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e00c8970 con 0x7f53e00cc7a0 2024-03-30T00:51:26.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e7600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e00cbb40 con 0x7f53e00c4950 2024-03-30T00:51:26.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e7600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00ca850 con 0x7f53e01596a0 2024-03-30T00:51:26.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e5600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53e00c4950 0x7f53e00c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e5600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f53e00c4950 0x7f53e00c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33240/0 (socket says 172.21.15.66:33240) 2024-03-30T00:51:26.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e5600640 1 -- 172.21.15.66:0/1027732776 learned_addr learned my addr 172.21.15.66:0/1027732776 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:26.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e4c00640 1 --2- 172.21.15.66:0/1027732776 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e01596a0 0x7f53e00b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e5600640 1 -- 172.21.15.66:0/1027732776 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e01596a0 msgr2=0x7f53e00b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:26.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e5600640 1 --2- 172.21.15.66:0/1027732776 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e01596a0 0x7f53e00b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e5600640 1 -- 172.21.15.66:0/1027732776 >> v1:172.21.15.156:6789/0 conn(0x7f53e00cc7a0 legacy=0x7f53e00c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:26.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e5600640 1 -- 172.21.15.66:0/1027732776 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00cb860 con 0x7f53e00c4950 2024-03-30T00:51:26.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53e5600640 1 --2- 172.21.15.66:0/1027732776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e00c4950 0x7f53e00c9680 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009e50 tx=0x7f53d801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd290ee0b086f75 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:26.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53dfe00640 1 -- 172.21.15.66:0/1027732776 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d80a5870 con 0x7f53e00c4950 2024-03-30T00:51:26.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.939+0000 7f53dfe00640 1 -- 172.21.15.66:0/1027732776 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d80a59d0 con 0x7f53e00c4950 2024-03-30T00:51:26.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53dfe00640 1 -- 172.21.15.66:0/1027732776 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d80a5d60 con 0x7f53e00c4950 2024-03-30T00:51:26.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 -- 172.21.15.66:0/1027732776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e00c4950 msgr2=0x7f53e00c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 --2- 172.21.15.66:0/1027732776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e00c4950 0x7f53e00c9680 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009e50 tx=0x7f53d801c9f0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 -- 172.21.15.66:0/1027732776 shutdown_connections 2024-03-30T00:51:26.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 --2- 172.21.15.66:0/1027732776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e00c4950 0x7f53e00c9680 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 --2- 172.21.15.66:0/1027732776 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e01596a0 0x7f53e00b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 -- 172.21.15.66:0/1027732776 >> 172.21.15.66:0/1027732776 conn(0x7f53e0069cf0 msgr2=0x7f53e006a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:26.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 -- 172.21.15.66:0/1027732776 shutdown_connections 2024-03-30T00:51:26.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 -- 172.21.15.66:0/1027732776 wait complete. 2024-03-30T00:51:26.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 Processor -- start 2024-03-30T00:51:26.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 -- start start 2024-03-30T00:51:26.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e00c4950 0x7f53e00acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e00cc7a0 0x7f53e014acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e0157b40 0x7f53e00af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00ad270 con 0x7f53e00c4950 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0157fd0 con 0x7f53e0157b40 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e7600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01582b0 con 0x7f53e00cc7a0 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e00c4950 0x7f53e00acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e00c4950 0x7f53e00acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33250/0 (socket says 172.21.15.66:33250) 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e4c00640 1 -- 172.21.15.66:0/1984015487 learned_addr learned my addr 172.21.15.66:0/1984015487 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53d7e00640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e00cc7a0 0x7f53e014acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e5600640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e0157b40 0x7f53e00af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e4c00640 1 -- 172.21.15.66:0/1984015487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e00cc7a0 msgr2=0x7f53e014acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e4c00640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e00cc7a0 0x7f53e014acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e4c00640 1 -- 172.21.15.66:0/1984015487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e0157b40 msgr2=0x7f53e00af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:26.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e4c00640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e0157b40 0x7f53e00af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:26.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.943+0000 7f53e4c00640 1 -- 172.21.15.66:0/1984015487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d8009b30 con 0x7f53e00c4950 2024-03-30T00:51:26.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.947+0000 7f53e4c00640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e00c4950 0x7f53e00acd30 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f53d000d5f0 tx=0x7f53d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:26.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.947+0000 7f53dd600640 1 -- 172.21.15.66:0/1984015487 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00125f0 con 0x7f53e00c4950 2024-03-30T00:51:26.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.947+0000 7f53dd600640 1 -- 172.21.15.66:0/1984015487 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d0012750 con 0x7f53e00c4950 2024-03-30T00:51:26.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.947+0000 7f53dd600640 1 -- 172.21.15.66:0/1984015487 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0012ae0 con 0x7f53e00c4950 2024-03-30T00:51:26.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.947+0000 7f53e7600640 1 -- 172.21.15.66:0/1984015487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e0158590 con 0x7f53e00c4950 2024-03-30T00:51:26.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.947+0000 7f53e7600640 1 -- 172.21.15.66:0/1984015487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e014b200 con 0x7f53e00c4950 2024-03-30T00:51:26.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.947+0000 7f53e7600640 1 -- 172.21.15.66:0/1984015487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e00b1680 con 0x7f53e00c4950 2024-03-30T00:51:26.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.947+0000 7f53dd600640 1 -- 172.21.15.66:0/1984015487 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f53d0012c40 con 0x7f53e00c4950 2024-03-30T00:51:26.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.947+0000 7f53dd600640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53bc066690 0x7f53bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:26.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.947+0000 7f53dd600640 1 -- 172.21.15.66:0/1984015487 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f53d0084680 con 0x7f53e00c4950 2024-03-30T00:51:26.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.947+0000 7f53d7e00640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53bc066690 0x7f53bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:26.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.951+0000 7f53d7e00640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53bc066690 0x7f53bc068b50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f53e00c9870 tx=0x7f53c8007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:26.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:26.951+0000 7f53dd600640 1 -- 172.21.15.66:0/1984015487 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f53d0051900 con 0x7f53e00c4950 2024-03-30T00:51:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 -- 172.21.15.66:0/1984015487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53e004e0b0 con 0x7f53e00c4950 2024-03-30T00:51:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53dd600640 1 -- 172.21.15.66:0/1984015487 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53d0007380 con 0x7f53e00c4950 2024-03-30T00:51:27.123 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:51:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 -- 172.21.15.66:0/1984015487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53bc066690 msgr2=0x7f53bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53bc066690 0x7f53bc068b50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f53e00c9870 tx=0x7f53c8007e40 comp rx=0 tx=0).stop 2024-03-30T00:51:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 -- 172.21.15.66:0/1984015487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e00c4950 msgr2=0x7f53e00acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e00c4950 0x7f53e00acd30 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f53d000d5f0 tx=0x7f53d000dab0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 -- 172.21.15.66:0/1984015487 shutdown_connections 2024-03-30T00:51:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e0157b40 0x7f53e00af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e00cc7a0 0x7f53e014acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f53bc066690 0x7f53bc068b50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 --2- 172.21.15.66:0/1984015487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53e00c4950 0x7f53e00acd30 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 -- 172.21.15.66:0/1984015487 >> 172.21.15.66:0/1984015487 conn(0x7f53e0069cf0 msgr2=0x7f53e006a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 -- 172.21.15.66:0/1984015487 shutdown_connections 2024-03-30T00:51:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.115+0000 7f53e7600640 1 -- 172.21.15.66:0/1984015487 wait complete. 2024-03-30T00:51:27.146 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:51:27.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.279+0000 7f68fa400640 1 Processor -- start 2024-03-30T00:51:27.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.279+0000 7f68fa400640 1 -- start start 2024-03-30T00:51:27.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68fa400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68f415a890 0x7f68f40cb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68fa400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f68f40ce3b0 0x7f68f40d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68fa400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f68f40cb6d0 0x7f68f40d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68fa400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f40d0d80 con 0x7f68f40ce3b0 2024-03-30T00:51:27.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68fa400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f40cbc70 con 0x7f68f40cb6d0 2024-03-30T00:51:27.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68fa400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40cbf50 con 0x7f68f415a890 2024-03-30T00:51:27.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68f2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f68f40ce3b0 0x7f68f40d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68f2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f68f40ce3b0 0x7f68f40d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33258/0 (socket says 172.21.15.66:33258) 2024-03-30T00:51:27.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68f2a00640 1 -- 172.21.15.66:0/1446748012 learned_addr learned my addr 172.21.15.66:0/1446748012 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:27.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68f3e00640 1 --2- 172.21.15.66:0/1446748012 >> v2:172.21.15.156:3300/0 conn(0x7f68f40cb6d0 0x7f68f40d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68f3400640 1 --2- 172.21.15.66:0/1446748012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68f415a890 0x7f68f40cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68f2a00640 1 -- 172.21.15.66:0/1446748012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68f415a890 msgr2=0x7f68f40cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:27.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68f2a00640 1 --2- 172.21.15.66:0/1446748012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68f415a890 0x7f68f40cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68f2a00640 1 -- 172.21.15.66:0/1446748012 >> v2:172.21.15.156:3300/0 conn(0x7f68f40cb6d0 msgr2=0x7f68f40d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:27.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68f2a00640 1 --2- 172.21.15.66:0/1446748012 >> v2:172.21.15.156:3300/0 conn(0x7f68f40cb6d0 0x7f68f40d0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68f2a00640 1 -- 172.21.15.66:0/1446748012 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40cc230 con 0x7f68f40ce3b0 2024-03-30T00:51:27.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.283+0000 7f68f3e00640 1 --2- 172.21.15.66:0/1446748012 >> v2:172.21.15.156:3300/0 conn(0x7f68f40cb6d0 0x7f68f40d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:27.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68f2a00640 1 --2- 172.21.15.66:0/1446748012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68f40ce3b0 0x7f68f40d0300 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f68f40b01e0 tx=0x7f68e001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13186799023ce91 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:27.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68f2000640 1 -- 172.21.15.66:0/1446748012 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00a89d0 con 0x7f68f40ce3b0 2024-03-30T00:51:27.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68f2000640 1 -- 172.21.15.66:0/1446748012 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e00a8b30 con 0x7f68f40ce3b0 2024-03-30T00:51:27.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68f2000640 1 -- 172.21.15.66:0/1446748012 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00b0440 con 0x7f68f40ce3b0 2024-03-30T00:51:27.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68fa400640 1 -- 172.21.15.66:0/1446748012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68f40ce3b0 msgr2=0x7f68f40d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68fa400640 1 --2- 172.21.15.66:0/1446748012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68f40ce3b0 0x7f68f40d0300 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f68f40b01e0 tx=0x7f68e001ed10 comp rx=0 tx=0).stop 2024-03-30T00:51:27.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68fa400640 1 -- 172.21.15.66:0/1446748012 shutdown_connections 2024-03-30T00:51:27.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68fa400640 1 --2- 172.21.15.66:0/1446748012 >> v2:172.21.15.156:3300/0 conn(0x7f68f40cb6d0 0x7f68f40d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68fa400640 1 --2- 172.21.15.66:0/1446748012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68f40ce3b0 0x7f68f40d0300 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68fa400640 1 --2- 172.21.15.66:0/1446748012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68f415a890 0x7f68f40cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68fa400640 1 -- 172.21.15.66:0/1446748012 >> 172.21.15.66:0/1446748012 conn(0x7f68f4089fc0 msgr2=0x7f68f408a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:27.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68fa400640 1 -- 172.21.15.66:0/1446748012 shutdown_connections 2024-03-30T00:51:27.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68fa400640 1 -- 172.21.15.66:0/1446748012 wait complete. 2024-03-30T00:51:27.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68fa400640 1 Processor -- start 2024-03-30T00:51:27.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.287+0000 7f68fa400640 1 -- start start 2024-03-30T00:51:27.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68fa400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68f40cb6d0 0x7f68f41f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68fa400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68f40ce3b0 0x7f68f40c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68fa400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68f415a890 0x7f68f4158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68fa400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40b5c10 con 0x7f68f415a890 2024-03-30T00:51:27.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68fa400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f41e3710 con 0x7f68f40ce3b0 2024-03-30T00:51:27.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68fa400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40c7fb0 con 0x7f68f40cb6d0 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68f415a890 0x7f68f4158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68f415a890 0x7f68f4158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33266/0 (socket says 172.21.15.66:33266) 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f3e00640 1 -- 172.21.15.66:0/2022577196 learned_addr learned my addr 172.21.15.66:0/2022577196 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f3400640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68f40cb6d0 0x7f68f41f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f2a00640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68f40ce3b0 0x7f68f40c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f3e00640 1 -- 172.21.15.66:0/2022577196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68f40cb6d0 msgr2=0x7f68f41f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f3e00640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68f40cb6d0 0x7f68f41f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f3e00640 1 -- 172.21.15.66:0/2022577196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68f40ce3b0 msgr2=0x7f68f40c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f3e00640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68f40ce3b0 0x7f68f40c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f3e00640 1 -- 172.21.15.66:0/2022577196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e00094d0 con 0x7f68f415a890 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f2a00640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68f40ce3b0 0x7f68f40c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f3400640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68f40cb6d0 0x7f68f41f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68f3e00640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68f415a890 0x7f68f4158100 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f68ec007ef0 tx=0x7f68ec00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68eb400640 1 -- 172.21.15.66:0/2022577196 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec003ba0 con 0x7f68f415a890 2024-03-30T00:51:27.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68eb400640 1 -- 172.21.15.66:0/2022577196 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec00f040 con 0x7f68f415a890 2024-03-30T00:51:27.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68eb400640 1 -- 172.21.15.66:0/2022577196 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00de80 con 0x7f68f415a890 2024-03-30T00:51:27.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68fa400640 1 -- 172.21.15.66:0/2022577196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f40c8290 con 0x7f68f415a890 2024-03-30T00:51:27.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68fa400640 1 -- 172.21.15.66:0/2022577196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f40c8850 con 0x7f68f415a890 2024-03-30T00:51:27.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68fa400640 1 -- 172.21.15.66:0/2022577196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f41e3880 con 0x7f68f415a890 2024-03-30T00:51:27.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.291+0000 7f68eb400640 1 -- 172.21.15.66:0/2022577196 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f68ec004030 con 0x7f68f415a890 2024-03-30T00:51:27.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.295+0000 7f68eb400640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f68d40666c0 0x7f68d4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.295+0000 7f68eb400640 1 -- 172.21.15.66:0/2022577196 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f68ec0845e0 con 0x7f68f415a890 2024-03-30T00:51:27.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.295+0000 7f68f3400640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f68d40666c0 0x7f68d4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.295+0000 7f68f3400640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f68d40666c0 0x7f68d4068b80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f68e400b970 tx=0x7f68e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:27.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.299+0000 7f68eb400640 1 -- 172.21.15.66:0/2022577196 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f68ec0517c0 con 0x7f68f415a890 2024-03-30T00:51:27.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.459+0000 7f68fa400640 1 -- 172.21.15.66:0/2022577196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68f404e0b0 con 0x7f68f415a890 2024-03-30T00:51:27.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.459+0000 7f68eb400640 1 -- 172.21.15.66:0/2022577196 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68ec054de0 con 0x7f68f415a890 2024-03-30T00:51:27.469 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:51:27.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 -- 172.21.15.66:0/2022577196 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f68d40666c0 msgr2=0x7f68d4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f68d40666c0 0x7f68d4068b80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f68e400b970 tx=0x7f68e4008040 comp rx=0 tx=0).stop 2024-03-30T00:51:27.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 -- 172.21.15.66:0/2022577196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68f415a890 msgr2=0x7f68f4158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68f415a890 0x7f68f4158100 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f68ec007ef0 tx=0x7f68ec00d4a0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 -- 172.21.15.66:0/2022577196 shutdown_connections 2024-03-30T00:51:27.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f68d40666c0 0x7f68d4068b80 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68f415a890 0x7f68f4158100 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68f40ce3b0 0x7f68f40c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 --2- 172.21.15.66:0/2022577196 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f68f40cb6d0 0x7f68f41f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 -- 172.21.15.66:0/2022577196 >> 172.21.15.66:0/2022577196 conn(0x7f68f4089fc0 msgr2=0x7f68f406e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:27.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 -- 172.21.15.66:0/2022577196 shutdown_connections 2024-03-30T00:51:27.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.463+0000 7f68fa400640 1 -- 172.21.15.66:0/2022577196 wait complete. 2024-03-30T00:51:27.491 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:51:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd4600640 1 Processor -- start 2024-03-30T00:51:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd4600640 1 -- start start 2024-03-30T00:51:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd4600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0bcc0af6c0 0x7f0bcc0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd4600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0bcc0b03d0 0x7f0bcc0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd4600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bcc15cff0 con 0x7f0bcc15cc10 2024-03-30T00:51:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd4600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bcc0cf2d0 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd4600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0b0200 con 0x7f0bcc0af6c0 2024-03-30T00:51:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd2600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0bcc0b03d0 0x7f0bcc0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd2600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0bcc0b03d0 0x7f0bcc0b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33276/0 (socket says 172.21.15.66:33276) 2024-03-30T00:51:27.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd2600640 1 -- 172.21.15.66:0/2754001125 learned_addr learned my addr 172.21.15.66:0/2754001125 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:27.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd1200640 1 --2- 172.21.15.66:0/2754001125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0bcc0af6c0 0x7f0bcc0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd2600640 1 -- 172.21.15.66:0/2754001125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0bcc0af6c0 msgr2=0x7f0bcc0c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:27.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd2600640 1 --2- 172.21.15.66:0/2754001125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0bcc0af6c0 0x7f0bcc0c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd2600640 1 -- 172.21.15.66:0/2754001125 >> v1:172.21.15.156:6789/0 conn(0x7f0bcc15cc10 legacy=0x7f0bcc0cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.627+0000 7f0bd2600640 1 -- 172.21.15.66:0/2754001125 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc0b0d40 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bd2600640 1 --2- 172.21.15.66:0/2754001125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc0b03d0 0x7f0bcc0b0800 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009e50 tx=0x7f0bc401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaedad49a89c2d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:27.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bcbe00640 1 -- 172.21.15.66:0/2754001125 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40a5860 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bcbe00640 1 -- 172.21.15.66:0/2754001125 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc40a59c0 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bcbe00640 1 -- 172.21.15.66:0/2754001125 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40a5d50 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bd4600640 1 -- 172.21.15.66:0/2754001125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc0b03d0 msgr2=0x7f0bcc0b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bd4600640 1 --2- 172.21.15.66:0/2754001125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc0b03d0 0x7f0bcc0b0800 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009e50 tx=0x7f0bc401c9a0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bd4600640 1 -- 172.21.15.66:0/2754001125 shutdown_connections 2024-03-30T00:51:27.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bd4600640 1 --2- 172.21.15.66:0/2754001125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc0b03d0 0x7f0bcc0b0800 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bd4600640 1 --2- 172.21.15.66:0/2754001125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0bcc0af6c0 0x7f0bcc0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bd4600640 1 -- 172.21.15.66:0/2754001125 >> 172.21.15.66:0/2754001125 conn(0x7f0bcc0cf420 msgr2=0x7f0bcc0cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:27.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bd4600640 1 -- 172.21.15.66:0/2754001125 shutdown_connections 2024-03-30T00:51:27.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bd4600640 1 -- 172.21.15.66:0/2754001125 wait complete. 2024-03-30T00:51:27.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bd4600640 1 Processor -- start 2024-03-30T00:51:27.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.631+0000 7f0bd4600640 1 -- start start 2024-03-30T00:51:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd4600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bcc0af6c0 0x7f0bcc1532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd4600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc0b03d0 0x7f0bcc14f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd4600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0bcc15cc10 0x7f0bcc155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd4600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc14f5b0 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd4600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc14ebe0 con 0x7f0bcc0af6c0 2024-03-30T00:51:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd4600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc14e2f0 con 0x7f0bcc15cc10 2024-03-30T00:51:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc0b03d0 0x7f0bcc14f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd1200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc0b03d0 0x7f0bcc14f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33292/0 (socket says 172.21.15.66:33292) 2024-03-30T00:51:27.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd1200640 1 -- 172.21.15.66:0/992990555 learned_addr learned my addr 172.21.15.66:0/992990555 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd1c00640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bcc0af6c0 0x7f0bcc1532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd2600640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0bcc15cc10 0x7f0bcc155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd1200640 1 -- 172.21.15.66:0/992990555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0bcc15cc10 msgr2=0x7f0bcc155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd1200640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0bcc15cc10 0x7f0bcc155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd1200640 1 -- 172.21.15.66:0/992990555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bcc0af6c0 msgr2=0x7f0bcc1532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd1200640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bcc0af6c0 0x7f0bcc1532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd1200640 1 -- 172.21.15.66:0/992990555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc4009b30 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd1200640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc0b03d0 0x7f0bcc14f070 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800e930 tx=0x7f0bb800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bc9600640 1 -- 172.21.15.66:0/992990555 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8009690 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bc9600640 1 -- 172.21.15.66:0/992990555 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb800ca00 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd4600640 1 -- 172.21.15.66:0/992990555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc14e590 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bc9600640 1 -- 172.21.15.66:0/992990555 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800cd70 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd4600640 1 -- 172.21.15.66:0/992990555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc0b2650 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.635+0000 7f0bd4600640 1 -- 172.21.15.66:0/992990555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc14ed50 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.639+0000 7f0bc9600640 1 -- 172.21.15.66:0/992990555 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0bb8013470 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.639+0000 7f0bc9600640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0bac066690 0x7f0bac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.639+0000 7f0bc9600640 1 -- 172.21.15.66:0/992990555 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f0bb8084560 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.639+0000 7f0bd1c00640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0bac066690 0x7f0bac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.639+0000 7f0bd1c00640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0bac066690 0x7f0bac068b50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc15a990 tx=0x7f0bc0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:27.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.643+0000 7f0bc9600640 1 -- 172.21.15.66:0/992990555 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0bb8051890 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.803+0000 7f0bd4600640 1 -- 172.21.15.66:0/992990555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0bcc04e0b0 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.803+0000 7f0bc9600640 1 -- 172.21.15.66:0/992990555 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0bb8054eb0 con 0x7f0bcc0b03d0 2024-03-30T00:51:27.812 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:51:27.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 -- 172.21.15.66:0/992990555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0bac066690 msgr2=0x7f0bac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0bac066690 0x7f0bac068b50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc15a990 tx=0x7f0bc0009040 comp rx=0 tx=0).stop 2024-03-30T00:51:27.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 -- 172.21.15.66:0/992990555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc0b03d0 msgr2=0x7f0bcc14f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc0b03d0 0x7f0bcc14f070 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800e930 tx=0x7f0bb800edf0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 -- 172.21.15.66:0/992990555 shutdown_connections 2024-03-30T00:51:27.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0bcc15cc10 0x7f0bcc155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0bac066690 0x7f0bac068b50 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bcc0b03d0 0x7f0bcc14f070 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 --2- 172.21.15.66:0/992990555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bcc0af6c0 0x7f0bcc1532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 -- 172.21.15.66:0/992990555 >> 172.21.15.66:0/992990555 conn(0x7f0bcc0cf420 msgr2=0x7f0bcc0ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:27.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 -- 172.21.15.66:0/992990555 shutdown_connections 2024-03-30T00:51:27.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.807+0000 7f0bd4600640 1 -- 172.21.15.66:0/992990555 wait complete. 2024-03-30T00:51:27.834 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:51:27.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.967+0000 7f8209800640 1 Processor -- start 2024-03-30T00:51:27.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.967+0000 7f8209800640 1 -- start start 2024-03-30T00:51:27.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.967+0000 7f8209800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f82040cb3a0 0x7f82041656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.967+0000 7f8209800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040ce010 con 0x7f82040b2c10 2024-03-30T00:51:27.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.967+0000 7f8209800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040b2ff0 con 0x7f82040cda30 2024-03-30T00:51:27.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.967+0000 7f8209800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820406cfc0 con 0x7f82040cb3a0 2024-03-30T00:51:27.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.967+0000 7f8202a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f82040b2c10 0x7f820415ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33012/0 (socket says 172.21.15.66:33012) 2024-03-30T00:51:27.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202a00640 1 -- 172.21.15.66:0/698874218 learned_addr learned my addr 172.21.15.66:0/698874218 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8203400640 1 --2- 172.21.15.66:0/698874218 >> v2:172.21.15.156:3300/0 conn(0x7f82040cb3a0 0x7f82041656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652732429 0 0) 0x7f81f40014a0 con 0x7f82040b2c10 2024-03-30T00:51:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81e8003530 con 0x7f82040b2c10 2024-03-30T00:51:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2390279920 0 0) 0x7f81e8003530 con 0x7f82040b2c10 2024-03-30T00:51:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 >> v2:172.21.15.156:3300/0 conn(0x7f82040cb3a0 msgr2=0x7f82041656c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 --2- 172.21.15.66:0/698874218 >> v2:172.21.15.156:3300/0 conn(0x7f82040cb3a0 0x7f82041656c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 >> v1:172.21.15.156:6789/0 conn(0x7f82040cda30 legacy=0x7f82040cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820406d260 con 0x7f82040b2c10 2024-03-30T00:51:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f81f4001b90 con 0x7f82040b2c10 2024-03-30T00:51:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 >> v1:172.21.15.66:6789/0 conn(0x7f82040b2c10 legacy=0x7f820415ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e8003980 0x7f81e8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81e80047d0 0x7f81e8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81e8005190 0x7f81e800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820406cfc0 con 0x7f81e80047d0 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e800c030 con 0x7f81e8003980 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e800c1f0 con 0x7f81e8005190 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f81fa000640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81e80047d0 0x7f81e8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8202a00640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e8003980 0x7f81e8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8203400640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81e8005190 0x7f81e800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f81fa000640 1 -- 172.21.15.66:0/698874218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81e8005190 msgr2=0x7f81e800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f81fa000640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81e8005190 0x7f81e800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f81fa000640 1 -- 172.21.15.66:0/698874218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e8003980 msgr2=0x7f81e8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f81fa000640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e8003980 0x7f81e8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f81fa000640 1 -- 172.21.15.66:0/698874218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8003530 con 0x7f81e80047d0 2024-03-30T00:51:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8203400640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81e8005190 0x7f81e800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:27.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.971+0000 7f8203400640 1 -- 172.21.15.66:0/698874218 reap_dead start 2024-03-30T00:51:27.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f81fa000640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81e80047d0 0x7f81e8004c50 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f81ec001780 tx=0x7f81ec01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84ecfdc5b29599b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:27.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0050d0 con 0x7f81e80047d0 2024-03-30T00:51:27.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec0a7050 con 0x7f81e80047d0 2024-03-30T00:51:27.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8202000640 1 -- 172.21.15.66:0/698874218 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec005490 con 0x7f81e80047d0 2024-03-30T00:51:27.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8209800640 1 -- 172.21.15.66:0/698874218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81e80047d0 msgr2=0x7f81e8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8209800640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81e80047d0 0x7f81e8004c50 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f81ec001780 tx=0x7f81ec01c400 comp rx=0 tx=0).stop 2024-03-30T00:51:27.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8209800640 1 -- 172.21.15.66:0/698874218 shutdown_connections 2024-03-30T00:51:27.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8209800640 1 --2- 172.21.15.66:0/698874218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81e80047d0 0x7f81e8004c50 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8209800640 1 -- 172.21.15.66:0/698874218 >> 172.21.15.66:0/698874218 conn(0x7f8204069cf0 msgr2=0x7f820406a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:27.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8209800640 1 -- 172.21.15.66:0/698874218 shutdown_connections 2024-03-30T00:51:27.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8209800640 1 -- 172.21.15.66:0/698874218 wait complete. 2024-03-30T00:51:27.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8209800640 1 Processor -- start 2024-03-30T00:51:27.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.975+0000 7f8209800640 1 -- start start 2024-03-30T00:51:27.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8209800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e80047d0 0x7f82040abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8209800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82040c5370 0x7f82041597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8209800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82040c57f0 0x7f8204159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8209800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820415a230 con 0x7f82040c57f0 2024-03-30T00:51:27.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8209800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82040c5dc0 con 0x7f81e80047d0 2024-03-30T00:51:27.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8209800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82040c60a0 con 0x7f82040c5370 2024-03-30T00:51:27.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8203400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82040c57f0 0x7f8204159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8203400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82040c57f0 0x7f8204159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33312/0 (socket says 172.21.15.66:33312) 2024-03-30T00:51:27.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8203400640 1 -- 172.21.15.66:0/203533643 learned_addr learned my addr 172.21.15.66:0/203533643 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:27.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8202a00640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e80047d0 0x7f82040abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f81fa000640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82040c5370 0x7f82041597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8203400640 1 -- 172.21.15.66:0/203533643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82040c5370 msgr2=0x7f82041597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8203400640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82040c5370 0x7f82041597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8203400640 1 -- 172.21.15.66:0/203533643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e80047d0 msgr2=0x7f82040abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:27.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8203400640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e80047d0 0x7f82040abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:27.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8203400640 1 -- 172.21.15.66:0/203533643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82040c6380 con 0x7f82040c57f0 2024-03-30T00:51:27.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8203400640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82040c57f0 0x7f8204159cf0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f820406c540 tx=0x7f81fc011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:27.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f81fb400640 1 -- 172.21.15.66:0/203533643 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc004230 con 0x7f82040c57f0 2024-03-30T00:51:27.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f81fb400640 1 -- 172.21.15.66:0/203533643 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc0043b0 con 0x7f82040c57f0 2024-03-30T00:51:27.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8209800640 1 -- 172.21.15.66:0/203533643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82040c6660 con 0x7f82040c57f0 2024-03-30T00:51:27.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f81fb400640 1 -- 172.21.15.66:0/203533643 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc015970 con 0x7f82040c57f0 2024-03-30T00:51:27.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8209800640 1 -- 172.21.15.66:0/203533643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82040c6e60 con 0x7f82040c57f0 2024-03-30T00:51:27.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f8209800640 1 -- 172.21.15.66:0/203533643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8204066c40 con 0x7f82040c57f0 2024-03-30T00:51:27.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.979+0000 7f81fb400640 1 -- 172.21.15.66:0/203533643 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f81fc015ad0 con 0x7f82040c57f0 2024-03-30T00:51:27.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.983+0000 7f81fb400640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f81e0066690 0x7f81e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:27.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.983+0000 7f81fb400640 1 -- 172.21.15.66:0/203533643 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f81fc084660 con 0x7f82040c57f0 2024-03-30T00:51:27.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.983+0000 7f8202a00640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f81e0066690 0x7f81e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:27.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.983+0000 7f8202a00640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f81e0066690 0x7f81e0068b50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f81f4001720 tx=0x7f81f4001380 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:27.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:27.987+0000 7f81fb400640 1 -- 172.21.15.66:0/203533643 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f81fc0518d0 con 0x7f82040c57f0 2024-03-30T00:51:28.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 -- 172.21.15.66:0/203533643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f82040ccb60 con 0x7f82040c57f0 2024-03-30T00:51:28.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f81fb400640 1 -- 172.21.15.66:0/203533643 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v126) v1 ==== 80+0+122 (secure 0 0 0) 0x7f81fc054ef0 con 0x7f82040c57f0 2024-03-30T00:51:28.167 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6836/2683418851 2024-03-31T00:51:15.162460+0000 2024-03-30T00:51:28.167 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6834/2683418851 2024-03-31T00:51:15.162460+0000 2024-03-30T00:51:28.167 INFO:teuthology.orchestra.run.smithi066.stderr:listed 2 entries 2024-03-30T00:51:28.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 -- 172.21.15.66:0/203533643 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f81e0066690 msgr2=0x7f81e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f81e0066690 0x7f81e0068b50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f81f4001720 tx=0x7f81f4001380 comp rx=0 tx=0).stop 2024-03-30T00:51:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 -- 172.21.15.66:0/203533643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82040c57f0 msgr2=0x7f8204159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82040c57f0 0x7f8204159cf0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f820406c540 tx=0x7f81fc011920 comp rx=0 tx=0).stop 2024-03-30T00:51:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 -- 172.21.15.66:0/203533643 shutdown_connections 2024-03-30T00:51:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f81e0066690 0x7f81e0068b50 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82040c57f0 0x7f8204159cf0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82040c5370 0x7f82041597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 --2- 172.21.15.66:0/203533643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e80047d0 0x7f82040abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 -- 172.21.15.66:0/203533643 >> 172.21.15.66:0/203533643 conn(0x7f8204069cf0 msgr2=0x7f82040b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 -- 172.21.15.66:0/203533643 shutdown_connections 2024-03-30T00:51:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.159+0000 7f8209800640 1 -- 172.21.15.66:0/203533643 wait complete. 2024-03-30T00:51:28.188 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:51:28.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:28.188 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:28.206 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[54919]: fuse finished with error 0 and tester_r 0 2024-03-30T00:51:28.209 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:51:28.232 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:51:28.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:28.232 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:51:28.282 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:51:28.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.455+0000 7f41b4e00640 1 Processor -- start 2024-03-30T00:51:28.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.455+0000 7f41b4e00640 1 -- start start 2024-03-30T00:51:28.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.459+0000 7f41b4e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f41b00b1a00 0x7f41b00cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.459+0000 7f41b4e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b00cacc0 con 0x7f41b00cea80 2024-03-30T00:51:28.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.459+0000 7f41b4e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b015bc30 con 0x7f41b015a720 2024-03-30T00:51:28.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.459+0000 7f41b4e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b015b0b0 con 0x7f41b00b1a00 2024-03-30T00:51:28.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.459+0000 7f41afe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f41b00b1a00 0x7f41b00cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.459+0000 7f41aea00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f41b00cea80 0x7f41b00cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33024/0 (socket says 172.21.15.66:33024) 2024-03-30T00:51:28.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.459+0000 7f41aea00640 1 -- 172.21.15.66:0/2804851607 learned_addr learned my addr 172.21.15.66:0/2804851607 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:28.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.459+0000 7f41afe00640 1 -- 172.21.15.66:0/2804851607 >> v1:172.21.15.156:6789/0 conn(0x7f41b015a720 legacy=0x7f41b00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:28.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.459+0000 7f41ae000640 1 -- 172.21.15.66:0/2804851607 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441847312 0 0) 0x7f41b00cacc0 con 0x7f41b00cea80 2024-03-30T00:51:28.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.459+0000 7f41afe00640 1 -- 172.21.15.66:0/2804851607 >> v1:172.21.15.66:6789/0 conn(0x7f41b00cea80 legacy=0x7f41b00cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.459+0000 7f41afe00640 1 -- 172.21.15.66:0/2804851607 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b015b390 con 0x7f41b00b1a00 2024-03-30T00:51:28.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41afe00640 1 --2- 172.21.15.66:0/2804851607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b00b1a00 0x7f41b00cb840 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009d90 tx=0x7f41a801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb14af0f89ca2e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:28.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41ae000640 1 -- 172.21.15.66:0/2804851607 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80a5a00 con 0x7f41b00b1a00 2024-03-30T00:51:28.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41ae000640 1 -- 172.21.15.66:0/2804851607 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a80a5b60 con 0x7f41b00b1a00 2024-03-30T00:51:28.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41ae000640 1 -- 172.21.15.66:0/2804851607 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80ad440 con 0x7f41b00b1a00 2024-03-30T00:51:28.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41b4e00640 1 -- 172.21.15.66:0/2804851607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b00b1a00 msgr2=0x7f41b00cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41b4e00640 1 --2- 172.21.15.66:0/2804851607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b00b1a00 0x7f41b00cb840 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009d90 tx=0x7f41a801c940 comp rx=0 tx=0).stop 2024-03-30T00:51:28.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41b4e00640 1 -- 172.21.15.66:0/2804851607 shutdown_connections 2024-03-30T00:51:28.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41b4e00640 1 --2- 172.21.15.66:0/2804851607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b00b1a00 0x7f41b00cb840 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41b4e00640 1 -- 172.21.15.66:0/2804851607 >> 172.21.15.66:0/2804851607 conn(0x7f41b0098a30 msgr2=0x7f41b00b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:28.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41b4e00640 1 -- 172.21.15.66:0/2804851607 shutdown_connections 2024-03-30T00:51:28.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41b4e00640 1 -- 172.21.15.66:0/2804851607 wait complete. 2024-03-30T00:51:28.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41b4e00640 1 Processor -- start 2024-03-30T00:51:28.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.463+0000 7f41b4e00640 1 -- start start 2024-03-30T00:51:28.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41b4e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b00b1a00 0x7f41b014c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41b4e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41b00cea80 0x7f41b00aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41b4e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41b015a720 0x7f41b014dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41b4e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00b0220 con 0x7f41b00b1a00 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41b4e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00af5a0 con 0x7f41b015a720 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41b4e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00af880 con 0x7f41b00cea80 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41af400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b00b1a00 0x7f41b014c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41af400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b00b1a00 0x7f41b014c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33340/0 (socket says 172.21.15.66:33340) 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41af400640 1 -- 172.21.15.66:0/607958419 learned_addr learned my addr 172.21.15.66:0/607958419 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41aea00640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41b00cea80 0x7f41b00aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41afe00640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41b015a720 0x7f41b014dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41af400640 1 -- 172.21.15.66:0/607958419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41b00cea80 msgr2=0x7f41b00aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41af400640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41b00cea80 0x7f41b00aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41af400640 1 -- 172.21.15.66:0/607958419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41b015a720 msgr2=0x7f41b014dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41af400640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41b015a720 0x7f41b014dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41af400640 1 -- 172.21.15.66:0/607958419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a8009a70 con 0x7f41b00b1a00 2024-03-30T00:51:28.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41af400640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b00b1a00 0x7f41b014c240 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f41a000c370 tx=0x7f41a000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:28.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41a7400640 1 -- 172.21.15.66:0/607958419 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a00052d0 con 0x7f41b00b1a00 2024-03-30T00:51:28.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41a7400640 1 -- 172.21.15.66:0/607958419 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a0005430 con 0x7f41b00b1a00 2024-03-30T00:51:28.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41b4e00640 1 -- 172.21.15.66:0/607958419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b00afb60 con 0x7f41b00b1a00 2024-03-30T00:51:28.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41a7400640 1 -- 172.21.15.66:0/607958419 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a0007260 con 0x7f41b00b1a00 2024-03-30T00:51:28.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41b4e00640 1 -- 172.21.15.66:0/607958419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b00af170 con 0x7f41b00b1a00 2024-03-30T00:51:28.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41b4e00640 1 -- 172.21.15.66:0/607958419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b004e040 con 0x7f41b00b1a00 2024-03-30T00:51:28.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.467+0000 7f41a7400640 1 -- 172.21.15.66:0/607958419 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f41a00073c0 con 0x7f41b00b1a00 2024-03-30T00:51:28.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.471+0000 7f41a7400640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41900666c0 0x7f4190068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.471+0000 7f41a7400640 1 -- 172.21.15.66:0/607958419 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f41a0083660 con 0x7f41b00b1a00 2024-03-30T00:51:28.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.471+0000 7f41aea00640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41900666c0 0x7f4190068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.471+0000 7f41aea00640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41900666c0 0x7f4190068b80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f419c007fd0 tx=0x7f419c007ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:28.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.471+0000 7f41a7400640 1 -- 172.21.15.66:0/607958419 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f41a0050820 con 0x7f41b00b1a00 2024-03-30T00:51:28.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.667+0000 7f41b4e00640 1 -- 172.21.15.66:0/607958419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41b0159d60 con 0x7f41b00b1a00 2024-03-30T00:51:28.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.667+0000 7f41a7400640 1 -- 172.21.15.66:0/607958419 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4743 (secure 0 0 0) 0x7f41a0053e40 con 0x7f41b00b1a00 2024-03-30T00:51:28.677 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:28.678 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":101,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:51:19.184013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9732":{"gid":9732,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/3460522956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3460522956},{"type":"v1","addr":"172.21.15.66:6836","nonce":3460522956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:51:28.678 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 101 2024-03-30T00:51:28.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 -- 172.21.15.66:0/607958419 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41900666c0 msgr2=0x7f4190068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41900666c0 0x7f4190068b80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f419c007fd0 tx=0x7f419c007ba0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 -- 172.21.15.66:0/607958419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b00b1a00 msgr2=0x7f41b014c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b00b1a00 0x7f41b014c240 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f41a000c370 tx=0x7f41a000c830 comp rx=0 tx=0).stop 2024-03-30T00:51:28.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 -- 172.21.15.66:0/607958419 shutdown_connections 2024-03-30T00:51:28.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41b015a720 0x7f41b014dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41b00cea80 0x7f41b00aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f41900666c0 0x7f4190068b80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 --2- 172.21.15.66:0/607958419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41b00b1a00 0x7f41b014c240 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 -- 172.21.15.66:0/607958419 >> 172.21.15.66:0/607958419 conn(0x7f41b0098a30 msgr2=0x7f41b00cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:28.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 -- 172.21.15.66:0/607958419 shutdown_connections 2024-03-30T00:51:28.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.671+0000 7f41b4e00640 1 -- 172.21.15.66:0/607958419 wait complete. 2024-03-30T00:51:28.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-03-30T00:51:28.862 INFO:teuthology.orchestra.run.smithi066.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-03-30T00:51:28.863 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-03-30T00:51:28.863 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-03-30T00:51:28.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.899+0000 7fc53c3b2080 1 Processor -- start 2024-03-30T00:51:28.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.899+0000 7fc53c3b2080 1 -- start start 2024-03-30T00:51:28.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555913037e40 0x555913049f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559130c81c0 0x55591304a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559130c85a0 0x55591304a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555912f50600 con 0x555913037e40 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555912f47690 con 0x5559130c81c0 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559130c69b0 con 0x5559130c85a0 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc539600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555913037e40 0x555913049f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc539600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555913037e40 0x555913049f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40062/0 (socket says 172.21.15.66:40062) 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc539600640 1 -- 172.21.15.66:0/4204633304 learned_addr learned my addr 172.21.15.66:0/4204633304 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc538c00640 1 --2- 172.21.15.66:0/4204633304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559130c81c0 0x55591304a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53a000640 1 --2- 172.21.15.66:0/4204633304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559130c85a0 0x55591304a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc539600640 1 -- 172.21.15.66:0/4204633304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559130c85a0 msgr2=0x55591304a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc539600640 1 --2- 172.21.15.66:0/4204633304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559130c85a0 0x55591304a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc539600640 1 -- 172.21.15.66:0/4204633304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559130c81c0 msgr2=0x55591304a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc539600640 1 --2- 172.21.15.66:0/4204633304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559130c81c0 0x55591304a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc539600640 1 -- 172.21.15.66:0/4204633304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53400a1c0 con 0x555913037e40 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc539600640 1 --2- 172.21.15.66:0/4204633304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555913037e40 0x555913049f20 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00a620 tx=0x7fc52c00aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33310243e0542b77 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc533400640 1 -- 172.21.15.66:0/4204633304 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c008ad0 con 0x555913037e40 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc533400640 1 -- 172.21.15.66:0/4204633304 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c008c50 con 0x555913037e40 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc533400640 1 -- 172.21.15.66:0/4204633304 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c013440 con 0x555913037e40 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- 172.21.15.66:0/4204633304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555913037e40 msgr2=0x555913049f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 --2- 172.21.15.66:0/4204633304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555913037e40 0x555913049f20 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00a620 tx=0x7fc52c00aae0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- 172.21.15.66:0/4204633304 shutdown_connections 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 --2- 172.21.15.66:0/4204633304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5559130c85a0 0x55591304a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 --2- 172.21.15.66:0/4204633304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559130c81c0 0x55591304a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 --2- 172.21.15.66:0/4204633304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555913037e40 0x555913049f20 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- 172.21.15.66:0/4204633304 >> 172.21.15.66:0/4204633304 conn(0x55591300b1a0 msgr2=0x555913033680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- 172.21.15.66:0/4204633304 shutdown_connections 2024-03-30T00:51:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- 172.21.15.66:0/4204633304 wait complete. 2024-03-30T00:51:28.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 Processor -- start 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- start start 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555913037e40 0x5559130d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559130c81c0 0x5559130c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559130c85a0 0x5559130c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559130c3890 con 0x5559130c81c0 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559130c3b20 con 0x5559130c85a0 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.903+0000 7fc53c3b2080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559130c3dc0 con 0x555913037e40 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc538c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559130c81c0 0x5559130c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc538c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559130c81c0 0x5559130c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40068/0 (socket says 172.21.15.66:40068) 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc538c00640 1 -- 172.21.15.66:0/2248001473 learned_addr learned my addr 172.21.15.66:0/2248001473 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc539600640 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555913037e40 0x5559130d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc53a000640 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559130c85a0 0x5559130c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc538c00640 1 -- 172.21.15.66:0/2248001473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555913037e40 msgr2=0x5559130d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc538c00640 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555913037e40 0x5559130d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc538c00640 1 -- 172.21.15.66:0/2248001473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559130c85a0 msgr2=0x5559130c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc538c00640 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5559130c85a0 0x5559130c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc538c00640 1 -- 172.21.15.66:0/2248001473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c005d80 con 0x5559130c81c0 2024-03-30T00:51:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc538c00640 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559130c81c0 0x5559130c2e10 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fc528006690 tx=0x7fc528006b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc532000640 1 -- 172.21.15.66:0/2248001473 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52800d8b0 con 0x5559130c81c0 2024-03-30T00:51:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc532000640 1 -- 172.21.15.66:0/2248001473 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52800f040 con 0x5559130c81c0 2024-03-30T00:51:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc532000640 1 -- 172.21.15.66:0/2248001473 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52800dc30 con 0x5559130c81c0 2024-03-30T00:51:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc53c3b2080 1 -- 172.21.15.66:0/2248001473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555912f79a60 con 0x5559130c81c0 2024-03-30T00:51:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc53c3b2080 1 -- 172.21.15.66:0/2248001473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555912f5f8c0 con 0x5559130c81c0 2024-03-30T00:51:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc532000640 1 -- 172.21.15.66:0/2248001473 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc52800dd90 con 0x5559130c81c0 2024-03-30T00:51:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc532000640 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc51c065070 0x7fc51c067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.907+0000 7fc532000640 1 -- 172.21.15.66:0/2248001473 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fc528083090 con 0x5559130c81c0 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc539600640 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc51c065070 0x7fc51c067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5559130ceeb0 0x55591304b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 -- 172.21.15.66:0/2248001473 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) v8 -- 0x55591304bd90 con 0x5559130ceeb0 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53a000640 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5559130ceeb0 0x55591304b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc539600640 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc51c065070 0x7fc51c067530 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00a5c0 tx=0x7fc52c002000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53a000640 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5559130ceeb0 0x55591304b850 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53a000640 1 -- 172.21.15.66:0/2248001473 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc5340a9940 con 0x5559130ceeb0 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 -- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5559130ceeb0 msgr2=0x55591304b850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5559130ceeb0 0x55591304b850 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 -- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc51c065070 msgr2=0x7fc51c067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc51c065070 0x7fc51c067530 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00a5c0 tx=0x7fc52c002000 comp rx=0 tx=0).stop 2024-03-30T00:51:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 -- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559130c81c0 msgr2=0x5559130c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:28.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 --2- 172.21.15.66:0/2248001473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5559130c81c0 0x5559130c2e10 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fc528006690 tx=0x7fc528006b50 comp rx=0 tx=0).stop 2024-03-30T00:51:28.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53a000640 1 -- 172.21.15.66:0/2248001473 reap_dead start 2024-03-30T00:51:28.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 -- 172.21.15.66:0/2248001473 shutdown_connections 2024-03-30T00:51:28.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 -- 172.21.15.66:0/2248001473 >> 172.21.15.66:0/2248001473 conn(0x55591300b1a0 msgr2=0x5559130c6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:28.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 -- 172.21.15.66:0/2248001473 shutdown_connections 2024-03-30T00:51:28.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:28.911+0000 7fc53c3b2080 1 -- 172.21.15.66:0/2248001473 wait complete. 2024-03-30T00:51:28.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:51:28.924 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-03-30T00:51:29.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578f73f080 1 Processor -- start 2024-03-30T00:51:29.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578f73f080 1 -- start start 2024-03-30T00:51:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578f73f080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892b27240 0x558892b1bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578f73f080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558892bca6c0 0x558892af4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578f73f080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558892bcaaa0 0x558892b1fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578f73f080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558892a3a600 con 0x558892b27240 2024-03-30T00:51:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578f73f080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558892a31690 con 0x558892bcaaa0 2024-03-30T00:51:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578f73f080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558892b20970 con 0x558892bca6c0 2024-03-30T00:51:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892b27240 0x558892b1bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892b27240 0x558892b1bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40086/0 (socket says 172.21.15.66:40086) 2024-03-30T00:51:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578ca00640 1 -- 172.21.15.66:0/1337845170 learned_addr learned my addr 172.21.15.66:0/1337845170 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578d400640 1 --2- 172.21.15.66:0/1337845170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558892bcaaa0 0x558892b1fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f5787e00640 1 --2- 172.21.15.66:0/1337845170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558892bca6c0 0x558892af4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578ca00640 1 -- 172.21.15.66:0/1337845170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558892bca6c0 msgr2=0x558892af4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578ca00640 1 --2- 172.21.15.66:0/1337845170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558892bca6c0 0x558892af4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578ca00640 1 -- 172.21.15.66:0/1337845170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558892bcaaa0 msgr2=0x558892b1fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578ca00640 1 --2- 172.21.15.66:0/1337845170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558892bcaaa0 0x558892b1fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578ca00640 1 -- 172.21.15.66:0/1337845170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5780005df0 con 0x558892b27240 2024-03-30T00:51:29.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578ca00640 1 --2- 172.21.15.66:0/1337845170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892b27240 0x558892b1bcb0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f57800970d0 tx=0x7f5780003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273b8626ccdb262b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f5786a00640 1 -- 172.21.15.66:0/1337845170 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57800a7af0 con 0x558892b27240 2024-03-30T00:51:29.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f5786a00640 1 -- 172.21.15.66:0/1337845170 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57800b5040 con 0x558892b27240 2024-03-30T00:51:29.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f5786a00640 1 -- 172.21.15.66:0/1337845170 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57800a7e90 con 0x558892b27240 2024-03-30T00:51:29.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578f73f080 1 -- 172.21.15.66:0/1337845170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892b27240 msgr2=0x558892b1bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.007+0000 7f578f73f080 1 --2- 172.21.15.66:0/1337845170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892b27240 0x558892b1bcb0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f57800970d0 tx=0x7f5780003030 comp rx=0 tx=0).stop 2024-03-30T00:51:29.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 -- 172.21.15.66:0/1337845170 shutdown_connections 2024-03-30T00:51:29.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 --2- 172.21.15.66:0/1337845170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558892bcaaa0 0x558892b1fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 --2- 172.21.15.66:0/1337845170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558892bca6c0 0x558892af4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 --2- 172.21.15.66:0/1337845170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892b27240 0x558892b1bcb0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 -- 172.21.15.66:0/1337845170 >> 172.21.15.66:0/1337845170 conn(0x558892b23550 msgr2=0x558892b1ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 -- 172.21.15.66:0/1337845170 shutdown_connections 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 -- 172.21.15.66:0/1337845170 wait complete. 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 Processor -- start 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 -- start start 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558892b27240 0x558892baf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892bca6c0 0x558892baf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558892bcaaa0 0x558892b383c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558892b1b590 con 0x558892bca6c0 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558892bb02e0 con 0x558892bcaaa0 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x558892bb0580 con 0x558892b27240 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5787e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892bca6c0 0x558892baf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5787e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892bca6c0 0x558892baf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40102/0 (socket says 172.21.15.66:40102) 2024-03-30T00:51:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5787e00640 1 -- 172.21.15.66:0/2461970809 learned_addr learned my addr 172.21.15.66:0/2461970809 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558892b27240 0x558892baf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578d400640 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558892bcaaa0 0x558892b383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5787e00640 1 -- 172.21.15.66:0/2461970809 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558892b27240 msgr2=0x558892baf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5787e00640 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x558892b27240 0x558892baf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5787e00640 1 -- 172.21.15.66:0/2461970809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558892bcaaa0 msgr2=0x558892b383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5787e00640 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558892bcaaa0 0x558892b383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5787e00640 1 -- 172.21.15.66:0/2461970809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558892a3c970 con 0x558892bca6c0 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5787e00640 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892bca6c0 0x558892baf590 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f5778008710 tx=0x7f5778008bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5785600640 1 -- 172.21.15.66:0/2461970809 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778006b70 con 0x558892bca6c0 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5785600640 1 -- 172.21.15.66:0/2461970809 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5778010040 con 0x558892bca6c0 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 -- 172.21.15.66:0/2461970809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558892a63a60 con 0x558892bca6c0 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f5785600640 1 -- 172.21.15.66:0/2461970809 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57780143f0 con 0x558892bca6c0 2024-03-30T00:51:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.011+0000 7f578f73f080 1 -- 172.21.15.66:0/2461970809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558892a498c0 con 0x558892bca6c0 2024-03-30T00:51:29.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f5785600640 1 -- 172.21.15.66:0/2461970809 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5778014550 con 0x558892bca6c0 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f5785600640 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f576c0650a0 0x7f576c067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f5785600640 1 -- 172.21.15.66:0/2461970809 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5778083340 con 0x558892bca6c0 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578ca00640 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f576c0650a0 0x7f576c067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x558892bb0cc0 0x558892bb10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 -- 172.21.15.66:0/2461970809 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) v8 -- 0x558892bb15e0 con 0x558892bb0cc0 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578d400640 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x558892bb0cc0 0x558892bb10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578ca00640 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f576c0650a0 0x7f576c067560 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f57800970d0 tx=0x7f57800a8000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578d400640 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x558892bb0cc0 0x558892bb10a0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578d400640 1 -- 172.21.15.66:0/2461970809 <== osd.2 v2:172.21.15.66:6816/27464 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f5788002970 con 0x558892bb0cc0 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 -- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x558892bb0cc0 msgr2=0x558892bb10a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x558892bb0cc0 0x558892bb10a0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 -- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f576c0650a0 msgr2=0x7f576c067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f576c0650a0 0x7f576c067560 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f57800970d0 tx=0x7f57800a8000 comp rx=0 tx=0).stop 2024-03-30T00:51:29.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 -- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892bca6c0 msgr2=0x558892baf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 --2- 172.21.15.66:0/2461970809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558892bca6c0 0x558892baf590 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f5778008710 tx=0x7f5778008bd0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578d400640 1 -- 172.21.15.66:0/2461970809 reap_dead start 2024-03-30T00:51:29.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 -- 172.21.15.66:0/2461970809 shutdown_connections 2024-03-30T00:51:29.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 -- 172.21.15.66:0/2461970809 >> 172.21.15.66:0/2461970809 conn(0x558892b23550 msgr2=0x558892b1d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 -- 172.21.15.66:0/2461970809 shutdown_connections 2024-03-30T00:51:29.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.015+0000 7f578f73f080 1 -- 172.21.15.66:0/2461970809 wait complete. 2024-03-30T00:51:29.030 INFO:teuthology.orchestra.run.smithi066.stdout:mytag123 2024-03-30T00:51:29.030 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-03-30T00:51:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660e6e2080 1 Processor -- start 2024-03-30T00:51:29.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660e6e2080 1 -- start start 2024-03-30T00:51:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660e6e2080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf120b450 0x559cf121c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660e6e2080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559cf129a1c0 0x559cf121ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660e6e2080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559cf129a5a0 0x559cf121fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660e6e2080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf11871e0 con 0x559cf129a1c0 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660e6e2080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf1122960 con 0x559cf129a5a0 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660e6e2080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cf1122600 con 0x559cf120b450 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660ba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf120b450 0x559cf121c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559cf129a1c0 0x559cf121ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660ba00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf120b450 0x559cf121c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:43468/0 (socket says 172.21.15.66:43468) 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660ba00640 1 -- 172.21.15.66:0/1374965615 learned_addr learned my addr 172.21.15.66:0/1374965615 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660c400640 1 --2- 172.21.15.66:0/1374965615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559cf129a5a0 0x559cf121fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660ba00640 1 -- 172.21.15.66:0/1374965615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559cf129a5a0 msgr2=0x559cf121fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660ba00640 1 --2- 172.21.15.66:0/1374965615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559cf129a5a0 0x559cf121fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660ba00640 1 -- 172.21.15.66:0/1374965615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559cf129a1c0 msgr2=0x559cf121ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660ba00640 1 --2- 172.21.15.66:0/1374965615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559cf129a1c0 0x559cf121ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.067+0000 7f660ba00640 1 -- 172.21.15.66:0/1374965615 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc005d30 con 0x559cf120b450 2024-03-30T00:51:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660ba00640 1 --2- 172.21.15.66:0/1374965615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf120b450 0x559cf121c950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00a530 tx=0x7f65fc003030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb54d81e1121a69c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f6609c00640 1 -- 172.21.15.66:0/1374965615 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0a47d0 con 0x559cf120b450 2024-03-30T00:51:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f6609c00640 1 -- 172.21.15.66:0/1374965615 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc00a230 con 0x559cf120b450 2024-03-30T00:51:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f6609c00640 1 -- 172.21.15.66:0/1374965615 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0a4ce0 con 0x559cf120b450 2024-03-30T00:51:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 -- 172.21.15.66:0/1374965615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf120b450 msgr2=0x559cf121c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 --2- 172.21.15.66:0/1374965615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf120b450 0x559cf121c950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00a530 tx=0x7f65fc003030 comp rx=0 tx=0).stop 2024-03-30T00:51:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 -- 172.21.15.66:0/1374965615 shutdown_connections 2024-03-30T00:51:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 --2- 172.21.15.66:0/1374965615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559cf129a5a0 0x559cf121fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 --2- 172.21.15.66:0/1374965615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559cf129a1c0 0x559cf121ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 --2- 172.21.15.66:0/1374965615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf120b450 0x559cf121c950 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 -- 172.21.15.66:0/1374965615 >> 172.21.15.66:0/1374965615 conn(0x559cf11dd1a0 msgr2=0x559cf12b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 -- 172.21.15.66:0/1374965615 shutdown_connections 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 -- 172.21.15.66:0/1374965615 wait complete. 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 Processor -- start 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 -- start start 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559cf120b450 0x559cf1348900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559cf129a1c0 0x559cf121f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf129a5a0 0x559cf134b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf1119690 con 0x559cf120b450 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf12971f0 con 0x559cf129a1c0 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660e6e2080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cf1297490 con 0x559cf129a5a0 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559cf120b450 0x559cf1348900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559cf120b450 0x559cf1348900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40128/0 (socket says 172.21.15.66:40128) 2024-03-30T00:51:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660ba00640 1 -- 172.21.15.66:0/38561009 learned_addr learned my addr 172.21.15.66:0/38561009 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660b000640 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559cf129a1c0 0x559cf121f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660c400640 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf129a5a0 0x559cf134b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660ba00640 1 -- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf129a5a0 msgr2=0x559cf134b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660ba00640 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf129a5a0 0x559cf134b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660ba00640 1 -- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559cf129a1c0 msgr2=0x559cf121f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660ba00640 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559cf129a1c0 0x559cf121f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660ba00640 1 -- 172.21.15.66:0/38561009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cf11318c0 con 0x559cf120b450 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660c400640 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559cf129a5a0 0x559cf134b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.071+0000 7f660ba00640 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559cf120b450 0x559cf1348900 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0b9040 tx=0x7f65fc018490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f6603e00640 1 -- 172.21.15.66:0/38561009 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0b6850 con 0x559cf120b450 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f6603e00640 1 -- 172.21.15.66:0/38561009 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc0b69b0 con 0x559cf120b450 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f660e6e2080 1 -- 172.21.15.66:0/38561009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559cf1124d00 con 0x559cf120b450 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f6603e00640 1 -- 172.21.15.66:0/38561009 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0b6d40 con 0x559cf120b450 2024-03-30T00:51:29.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f660e6e2080 1 -- 172.21.15.66:0/38561009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cf1124760 con 0x559cf120b450 2024-03-30T00:51:29.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f6603e00640 1 -- 172.21.15.66:0/38561009 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f65fc0bd050 con 0x559cf120b450 2024-03-30T00:51:29.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f6603e00640 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65ec065070 0x7f65ec067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f6603e00640 1 -- 172.21.15.66:0/38561009 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f65fc11efd0 con 0x559cf120b450 2024-03-30T00:51:29.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f660b000640 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65ec065070 0x7f65ec067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f660e6e2080 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x559cf121f5f0 0x559cf12969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f660e6e2080 1 -- 172.21.15.66:0/38561009 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) v8 -- 0x559cf1298110 con 0x559cf121f5f0 2024-03-30T00:51:29.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f660c400640 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x559cf121f5f0 0x559cf12969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.075+0000 7f660b000640 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65ec065070 0x7f65ec067530 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f65f8010420 tx=0x7f65f8001720 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660c400640 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x559cf121f5f0 0x559cf12969f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660c400640 1 -- 172.21.15.66:0/38561009 <== osd.4 v2:172.21.15.156:6800/27641 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f66040044d0 con 0x559cf121f5f0 2024-03-30T00:51:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660e6e2080 1 -- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x559cf121f5f0 msgr2=0x559cf12969f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660e6e2080 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x559cf121f5f0 0x559cf12969f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660e6e2080 1 -- 172.21.15.66:0/38561009 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65ec065070 msgr2=0x7f65ec067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660e6e2080 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65ec065070 0x7f65ec067530 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f65f8010420 tx=0x7f65f8001720 comp rx=0 tx=0).stop 2024-03-30T00:51:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660e6e2080 1 -- 172.21.15.66:0/38561009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559cf120b450 msgr2=0x559cf1348900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660e6e2080 1 --2- 172.21.15.66:0/38561009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559cf120b450 0x559cf1348900 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0b9040 tx=0x7f65fc018490 comp rx=0 tx=0).stop 2024-03-30T00:51:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660c400640 1 -- 172.21.15.66:0/38561009 reap_dead start 2024-03-30T00:51:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660e6e2080 1 -- 172.21.15.66:0/38561009 shutdown_connections 2024-03-30T00:51:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660e6e2080 1 -- 172.21.15.66:0/38561009 >> 172.21.15.66:0/38561009 conn(0x559cf11dd1a0 msgr2=0x559cf1221b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660e6e2080 1 -- 172.21.15.66:0/38561009 shutdown_connections 2024-03-30T00:51:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.079+0000 7f660e6e2080 1 -- 172.21.15.66:0/38561009 wait complete. 2024-03-30T00:51:29.091 INFO:teuthology.orchestra.run.smithi066.stdout:mytag123 2024-03-30T00:51:29.092 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-03-30T00:51:29.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f99887b4080 1 Processor -- start 2024-03-30T00:51:29.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f99887b4080 1 -- start start 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f99887b4080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591d82f4e40 0x5591d830ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f99887b4080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d83851c0 0x5591d82c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f99887b4080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591d83855a0 0x5591d8306bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f99887b4080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d820d960 con 0x5591d83851c0 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f99887b4080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d820d600 con 0x5591d83855a0 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f99887b4080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591d8204690 con 0x5591d82f4e40 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f9985000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d83851c0 0x5591d82c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f9985a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591d82f4e40 0x5591d830ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f9986400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591d83855a0 0x5591d8306bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f9985000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d83851c0 0x5591d82c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40144/0 (socket says 172.21.15.66:40144) 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f9985000640 1 -- 172.21.15.66:0/3226995517 learned_addr learned my addr 172.21.15.66:0/3226995517 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f9985000640 1 -- 172.21.15.66:0/3226995517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591d82f4e40 msgr2=0x5591d830ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f9985000640 1 --2- 172.21.15.66:0/3226995517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591d82f4e40 0x5591d830ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f9985000640 1 -- 172.21.15.66:0/3226995517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591d83855a0 msgr2=0x5591d8306bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f9985000640 1 --2- 172.21.15.66:0/3226995517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591d83855a0 0x5591d8306bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f9985000640 1 -- 172.21.15.66:0/3226995517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9978005df0 con 0x5591d83851c0 2024-03-30T00:51:29.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f9985000640 1 --2- 172.21.15.66:0/3226995517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d83851c0 0x5591d82c4c90 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f99740097d0 tx=0x7f9974005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e1c13b5126c5e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f997f400640 1 -- 172.21.15.66:0/3226995517 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997400f040 con 0x5591d83851c0 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f997f400640 1 -- 172.21.15.66:0/3226995517 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9974008da0 con 0x5591d83851c0 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.171+0000 7f997f400640 1 -- 172.21.15.66:0/3226995517 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974005e20 con 0x5591d83851c0 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 -- 172.21.15.66:0/3226995517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d83851c0 msgr2=0x5591d82c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 --2- 172.21.15.66:0/3226995517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d83851c0 0x5591d82c4c90 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f99740097d0 tx=0x7f9974005a10 comp rx=0 tx=0).stop 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 -- 172.21.15.66:0/3226995517 shutdown_connections 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 --2- 172.21.15.66:0/3226995517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591d83855a0 0x5591d8306bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 --2- 172.21.15.66:0/3226995517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d83851c0 0x5591d82c4c90 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 --2- 172.21.15.66:0/3226995517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591d82f4e40 0x5591d830ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 -- 172.21.15.66:0/3226995517 >> 172.21.15.66:0/3226995517 conn(0x5591d82c81a0 msgr2=0x5591d82c8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 -- 172.21.15.66:0/3226995517 shutdown_connections 2024-03-30T00:51:29.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 -- 172.21.15.66:0/3226995517 wait complete. 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 Processor -- start 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 -- start start 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d82f4e40 0x5591d82c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591d83851c0 0x5591d83837c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591d83855a0 0x5591d8383d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d8388a60 con 0x5591d82f4e40 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591d8388d00 con 0x5591d83851c0 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591d8388fa0 con 0x5591d83855a0 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9985a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d82f4e40 0x5591d82c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9985a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d82f4e40 0x5591d82c4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40146/0 (socket says 172.21.15.66:40146) 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9985a00640 1 -- 172.21.15.66:0/2791450101 learned_addr learned my addr 172.21.15.66:0/2791450101 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9986400640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591d83855a0 0x5591d8383d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9985000640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591d83851c0 0x5591d83837c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9985a00640 1 -- 172.21.15.66:0/2791450101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591d83855a0 msgr2=0x5591d8383d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9985a00640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591d83855a0 0x5591d8383d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9985a00640 1 -- 172.21.15.66:0/2791450101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591d83851c0 msgr2=0x5591d83837c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9985a00640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591d83851c0 0x5591d83837c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9985a00640 1 -- 172.21.15.66:0/2791450101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9974009420 con 0x5591d82f4e40 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9985000640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591d83851c0 0x5591d83837c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9986400640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5591d83855a0 0x5591d8383d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f9985a00640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d82f4e40 0x5591d82c4aa0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f997801b840 tx=0x7f99780185a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f997e000640 1 -- 172.21.15.66:0/2791450101 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99780ac990 con 0x5591d82f4e40 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f997e000640 1 -- 172.21.15.66:0/2791450101 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99780acaf0 con 0x5591d82f4e40 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f997e000640 1 -- 172.21.15.66:0/2791450101 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99780acea0 con 0x5591d82f4e40 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 -- 172.21.15.66:0/2791450101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5591d821c8c0 con 0x5591d82f4e40 2024-03-30T00:51:29.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.175+0000 7f99887b4080 1 -- 172.21.15.66:0/2791450101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591d820fd00 con 0x5591d82f4e40 2024-03-30T00:51:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f997e000640 1 -- 172.21.15.66:0/2791450101 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f99780b7420 con 0x5591d82f4e40 2024-03-30T00:51:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f997e000640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f99680650c0 0x7f9968067580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f997e000640 1 -- 172.21.15.66:0/2791450101 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f997811e5e0 con 0x5591d82f4e40 2024-03-30T00:51:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f9985000640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f99680650c0 0x7f9968067580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f99887b4080 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5591d82c4fe0 0x5591d83859b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f99887b4080 1 -- 172.21.15.66:0/2791450101 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) v8 -- 0x5591d8385ef0 con 0x5591d82c4fe0 2024-03-30T00:51:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f9986400640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5591d82c4fe0 0x5591d83859b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f9985000640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f99680650c0 0x7f9968067580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9974000c10 tx=0x7f9974011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f9986400640 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5591d82c4fe0 0x5591d83859b0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f9986400640 1 -- 172.21.15.66:0/2791450101 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f9980002710 con 0x5591d82c4fe0 2024-03-30T00:51:29.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f99887b4080 1 -- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5591d82c4fe0 msgr2=0x5591d83859b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f99887b4080 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5591d82c4fe0 0x5591d83859b0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f99887b4080 1 -- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f99680650c0 msgr2=0x7f9968067580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f99887b4080 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f99680650c0 0x7f9968067580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9974000c10 tx=0x7f9974011000 comp rx=0 tx=0).stop 2024-03-30T00:51:29.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f99887b4080 1 -- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d82f4e40 msgr2=0x5591d82c4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f99887b4080 1 --2- 172.21.15.66:0/2791450101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5591d82f4e40 0x5591d82c4aa0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f997801b840 tx=0x7f99780185a0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f9986400640 1 -- 172.21.15.66:0/2791450101 reap_dead start 2024-03-30T00:51:29.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f99887b4080 1 -- 172.21.15.66:0/2791450101 shutdown_connections 2024-03-30T00:51:29.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.179+0000 7f99887b4080 1 -- 172.21.15.66:0/2791450101 >> 172.21.15.66:0/2791450101 conn(0x5591d82c81a0 msgr2=0x5591d8306d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.183+0000 7f99887b4080 1 -- 172.21.15.66:0/2791450101 shutdown_connections 2024-03-30T00:51:29.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.183+0000 7f99887b4080 1 -- 172.21.15.66:0/2791450101 wait complete. 2024-03-30T00:51:29.194 INFO:teuthology.orchestra.run.smithi066.stdout:mytag123 2024-03-30T00:51:29.194 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-03-30T00:51:29.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f6821600640 1 Processor -- start 2024-03-30T00:51:29.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f6821600640 1 -- start start 2024-03-30T00:51:29.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f6821600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f681c1533b0 0x7f681c0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f6821600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f681c159d80 0x7f681c0ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f6821600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f681c0afe50 0x7f681c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f6821600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c0b0840 con 0x7f681c1533b0 2024-03-30T00:51:29.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f6821600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c15a2d0 con 0x7f681c159d80 2024-03-30T00:51:29.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f6821600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c153790 con 0x7f681c0afe50 2024-03-30T00:51:29.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f681aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f681c1533b0 0x7f681c0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f681aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f681c1533b0 0x7f681c0cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40156/0 (socket says 172.21.15.66:40156) 2024-03-30T00:51:29.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f681aa00640 1 -- 172.21.15.66:0/2956941050 learned_addr learned my addr 172.21.15.66:0/2956941050 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f6812000640 1 --2- 172.21.15.66:0/2956941050 >> v2:172.21.15.156:3300/0 conn(0x7f681c159d80 0x7f681c0ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.371+0000 7f681b400640 1 --2- 172.21.15.66:0/2956941050 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f681c0afe50 0x7f681c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.375+0000 7f681aa00640 1 -- 172.21.15.66:0/2956941050 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f681c0afe50 msgr2=0x7f681c0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.375+0000 7f681aa00640 1 --2- 172.21.15.66:0/2956941050 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f681c0afe50 0x7f681c0b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.375+0000 7f681aa00640 1 -- 172.21.15.66:0/2956941050 >> v2:172.21.15.156:3300/0 conn(0x7f681c159d80 msgr2=0x7f681c0ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.375+0000 7f681aa00640 1 --2- 172.21.15.66:0/2956941050 >> v2:172.21.15.156:3300/0 conn(0x7f681c159d80 0x7f681c0ab1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.375+0000 7f681aa00640 1 -- 172.21.15.66:0/2956941050 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c0cbec0 con 0x7f681c1533b0 2024-03-30T00:51:29.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.375+0000 7f681aa00640 1 --2- 172.21.15.66:0/2956941050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f681c1533b0 0x7f681c0cda70 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f681c0b0ba0 tx=0x7f680c01ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554e61ddc4be4246 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.375+0000 7f681a000640 1 -- 172.21.15.66:0/2956941050 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c0a79e0 con 0x7f681c1533b0 2024-03-30T00:51:29.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.375+0000 7f681a000640 1 -- 172.21.15.66:0/2956941050 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c0a7b40 con 0x7f681c1533b0 2024-03-30T00:51:29.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.375+0000 7f681a000640 1 -- 172.21.15.66:0/2956941050 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c0af3f0 con 0x7f681c1533b0 2024-03-30T00:51:29.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 -- 172.21.15.66:0/2956941050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f681c1533b0 msgr2=0x7f681c0cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 --2- 172.21.15.66:0/2956941050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f681c1533b0 0x7f681c0cda70 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f681c0b0ba0 tx=0x7f680c01ebd0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 -- 172.21.15.66:0/2956941050 shutdown_connections 2024-03-30T00:51:29.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 --2- 172.21.15.66:0/2956941050 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f681c0afe50 0x7f681c0b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 --2- 172.21.15.66:0/2956941050 >> v2:172.21.15.156:3300/0 conn(0x7f681c159d80 0x7f681c0ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 --2- 172.21.15.66:0/2956941050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f681c1533b0 0x7f681c0cda70 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 -- 172.21.15.66:0/2956941050 >> 172.21.15.66:0/2956941050 conn(0x7f681c069cf0 msgr2=0x7f681c06a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 -- 172.21.15.66:0/2956941050 shutdown_connections 2024-03-30T00:51:29.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 -- 172.21.15.66:0/2956941050 wait complete. 2024-03-30T00:51:29.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 Processor -- start 2024-03-30T00:51:29.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 -- start start 2024-03-30T00:51:29.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f681c0afe50 0x7f681c0ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f681c1533b0 0x7f681c0b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f681c159d80 0x7f681c0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c140820 con 0x7f681c1533b0 2024-03-30T00:51:29.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0adff0 con 0x7f681c159d80 2024-03-30T00:51:29.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6821600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0ae2d0 con 0x7f681c0afe50 2024-03-30T00:51:29.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6812000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f681c1533b0 0x7f681c0b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6812000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f681c1533b0 0x7f681c0b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40162/0 (socket says 172.21.15.66:40162) 2024-03-30T00:51:29.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6812000640 1 -- 172.21.15.66:0/565513884 learned_addr learned my addr 172.21.15.66:0/565513884 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f681aa00640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f681c0afe50 0x7f681c0ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f681b400640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f681c159d80 0x7f681c0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6812000640 1 -- 172.21.15.66:0/565513884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f681c0afe50 msgr2=0x7f681c0ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6812000640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f681c0afe50 0x7f681c0ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6812000640 1 -- 172.21.15.66:0/565513884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f681c159d80 msgr2=0x7f681c0aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6812000640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f681c159d80 0x7f681c0aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.379+0000 7f6812000640 1 -- 172.21.15.66:0/565513884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c009b30 con 0x7f681c1533b0 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f6812000640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f681c1533b0 0x7f681c0b2180 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f680400e900 tx=0x7f680400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f6813400640 1 -- 172.21.15.66:0/565513884 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804009690 con 0x7f681c1533b0 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f6813400640 1 -- 172.21.15.66:0/565513884 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680400ca00 con 0x7f681c1533b0 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f6813400640 1 -- 172.21.15.66:0/565513884 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680400cd90 con 0x7f681c1533b0 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f6821600640 1 -- 172.21.15.66:0/565513884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681c0ae5b0 con 0x7f681c1533b0 2024-03-30T00:51:29.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f6821600640 1 -- 172.21.15.66:0/565513884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c0adce0 con 0x7f681c1533b0 2024-03-30T00:51:29.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f6813400640 1 -- 172.21.15.66:0/565513884 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6804013470 con 0x7f681c1533b0 2024-03-30T00:51:29.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f6813400640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67f8066690 0x7f67f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f6813400640 1 -- 172.21.15.66:0/565513884 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f68040844a0 con 0x7f681c1533b0 2024-03-30T00:51:29.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f681aa00640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67f8066690 0x7f67f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f6824ac6000 1 Processor -- start 2024-03-30T00:51:29.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.383+0000 7f6824ac6000 1 -- start start 2024-03-30T00:51:29.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f681aa00640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67f8066690 0x7f67f8068b50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f680c0b0000 tx=0x7f680c00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd2a70 0x557706bd2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706b864c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b86de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557706a51630 con 0x557706b86a00 2024-03-30T00:51:29.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577069debd0 con 0x557706bd2a70 2024-03-30T00:51:29.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577069dc1a0 con 0x557706bd3390 2024-03-30T00:51:29.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f681b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b86de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f681b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b86de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40168/0 (socket says 172.21.15.66:40168) 2024-03-30T00:51:29.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f681b400640 1 -- 172.21.15.66:0/2202125609 learned_addr learned my addr 172.21.15.66:0/2202125609 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6812000640 1 --2- 172.21.15.66:0/2202125609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706b864c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f681aa00640 1 --2- 172.21.15.66:0/2202125609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd2a70 0x557706bd2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f681b400640 1 -- 172.21.15.66:0/2202125609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 msgr2=0x557706b864c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f681b400640 1 --2- 172.21.15.66:0/2202125609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706b864c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f681b400640 1 -- 172.21.15.66:0/2202125609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd2a70 msgr2=0x557706bd2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f681b400640 1 --2- 172.21.15.66:0/2202125609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd2a70 0x557706bd2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f681b400640 1 -- 172.21.15.66:0/2202125609 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577066d3e20 con 0x557706b86a00 2024-03-30T00:51:29.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f681b400640 1 --2- 172.21.15.66:0/2202125609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b86de0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f6814009b80 tx=0x7f6814011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b35faa05cdcee3c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f680b400640 1 -- 172.21.15.66:0/2202125609 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814011ec0 con 0x557706b86a00 2024-03-30T00:51:29.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f680b400640 1 -- 172.21.15.66:0/2202125609 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814009e80 con 0x557706b86a00 2024-03-30T00:51:29.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f680b400640 1 -- 172.21.15.66:0/2202125609 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814014640 con 0x557706b86a00 2024-03-30T00:51:29.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 -- 172.21.15.66:0/2202125609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 msgr2=0x557706b86de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 --2- 172.21.15.66:0/2202125609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b86de0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f6814009b80 tx=0x7f6814011690 comp rx=0 tx=0).stop 2024-03-30T00:51:29.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 -- 172.21.15.66:0/2202125609 shutdown_connections 2024-03-30T00:51:29.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 --2- 172.21.15.66:0/2202125609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b86de0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 --2- 172.21.15.66:0/2202125609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706b864c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 --2- 172.21.15.66:0/2202125609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd2a70 0x557706bd2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 -- 172.21.15.66:0/2202125609 >> 172.21.15.66:0/2202125609 conn(0x557706987160 msgr2=0x557706bb5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 -- 172.21.15.66:0/2202125609 shutdown_connections 2024-03-30T00:51:29.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 -- 172.21.15.66:0/2202125609 wait complete. 2024-03-30T00:51:29.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 -- start start 2024-03-30T00:51:29.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.387+0000 7f6824ac6000 1 Processor -- start 2024-03-30T00:51:29.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b8f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd2a70 0x557706bb6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706bb6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577069d7a90 con 0x557706b86a00 2024-03-30T00:51:29.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557706976d40 con 0x557706bd2a70 2024-03-30T00:51:29.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x557706994f30 con 0x557706bd3390 2024-03-30T00:51:29.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 10 client.0 _ll_register_callbacks cb 0x557706a39810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:51:29.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f681aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b8f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f681aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b8f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40176/0 (socket says 172.21.15.66:40176) 2024-03-30T00:51:29.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f681aa00640 1 -- 172.21.15.66:0/28221173 learned_addr learned my addr 172.21.15.66:0/28221173 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f681b400640 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706bb6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6812000640 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd2a70 0x557706bb6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f681aa00640 1 -- 172.21.15.66:0/28221173 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 msgr2=0x557706bb6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f681aa00640 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706bb6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f681aa00640 1 -- 172.21.15.66:0/28221173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd2a70 msgr2=0x557706bb6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f681aa00640 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd2a70 0x557706bb6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f681aa00640 1 -- 172.21.15.66:0/28221173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6814009860 con 0x557706b86a00 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f681b400640 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706bb6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f681aa00640 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b8f7b0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f680c01f470 tx=0x7f680c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90e79a9267456a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f67ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f67ffe00640 1 -- 172.21.15.66:0/28221173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c0ba930 con 0x557706b86a00 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f67ffe00640 1 -- 172.21.15.66:0/28221173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f680c0baa90 con 0x557706b86a00 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 1 -- 172.21.15.66:0/28221173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577069d63e0 con 0x557706b86a00 2024-03-30T00:51:29.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f67ffe00640 1 -- 172.21.15.66:0/28221173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f680c0bac50 con 0x557706b86a00 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 10 client.9921 fetch_fsmap learned FSMap version 101 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 1 -- 172.21.15.66:0/28221173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) v3 -- 0x55770699caa0 con 0x557706b86a00 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f67ffe00640 1 -- 172.21.15.66:0/28221173 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5577069d63e0 con 0x557706b86a00 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f67ffe00640 1 -- 172.21.15.66:0/28221173 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 101) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f680c0c5070 con 0x557706b86a00 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 10 client.9921 fetch_fsmap finished waiting for FSMap version 101 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 10 client.9921 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 20 client.9921 populate_metadata read hostname 'smithi066' 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x557706bda2d0 0x557706bda6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 4 client.9921 mds_command: new command op to 9732 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6824ac6000 1 -- 172.21.15.66:0/28221173 --> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577069a7d00 con 0x557706bda2d0 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6812000640 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x557706bda2d0 0x557706bda6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f67ffe00640 10 client.9921 ms_handle_connect on v2:172.21.15.66:6834/3460522956 2024-03-30T00:51:29.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.391+0000 7f6812000640 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x557706bda2d0 0x557706bda6b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e891e113e076f99 server_cookie=b5ffcdf9825afe7f in_seq=0 out_seq=0 2024-03-30T00:51:29.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f67ffe00640 1 -- 172.21.15.66:0/28221173 <== mds.0 v2:172.21.15.66:6834/3460522956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5577069a7d00 con 0x557706bda2d0 2024-03-30T00:51:29.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f67ffe00640 10 client.9921 handle_command_reply: tid=0 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 1 client.9921 shutdown 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 1 -- 172.21.15.66:0/28221173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 msgr2=0x557706b8f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b8f7b0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f680c01f470 tx=0x7f680c004a50 comp rx=0 tx=0).stop 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 1 -- 172.21.15.66:0/28221173 shutdown_connections 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x557706bda2d0 0x557706bda6b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706bb6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd2a70 0x557706bb6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 1 --2- 172.21.15.66:0/28221173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706b86a00 0x557706b8f7b0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 1 -- 172.21.15.66:0/28221173 >> 172.21.15.66:0/28221173 conn(0x557706987160 msgr2=0x557706b85bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f67ffe00640 0 client.9921 ms_handle_reset on v2:172.21.15.66:6834/3460522956 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 1 -- 172.21.15.66:0/28221173 shutdown_connections 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 1 -- 172.21.15.66:0/28221173 wait complete. 2024-03-30T00:51:29.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.395+0000 7f6824ac6000 20 client.9921 trim_cache size 0 max 16384 2024-03-30T00:51:29.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6824ac6000 1 Processor -- start 2024-03-30T00:51:29.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6824ac6000 1 -- start start 2024-03-30T00:51:29.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706b86a00 0x557706b85c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b86170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd3390 0x557706b86de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6824ac6000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557706694aa0 con 0x557706bd2a70 2024-03-30T00:51:29.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6824ac6000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577069d9020 con 0x557706bd3390 2024-03-30T00:51:29.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6824ac6000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x557706be0910 con 0x557706b86a00 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6812000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b86170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6812000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b86170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40188/0 (socket says 172.21.15.66:40188) 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6812000640 1 -- 172.21.15.66:0/13263148 learned_addr learned my addr 172.21.15.66:0/13263148 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f681aa00640 1 --2- 172.21.15.66:0/13263148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706b86a00 0x557706b85c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f681b400640 1 --2- 172.21.15.66:0/13263148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd3390 0x557706b86de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6812000640 1 -- 172.21.15.66:0/13263148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706b86a00 msgr2=0x557706b85c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6812000640 1 --2- 172.21.15.66:0/13263148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706b86a00 0x557706b85c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6812000640 1 -- 172.21.15.66:0/13263148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd3390 msgr2=0x557706b86de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6812000640 1 --2- 172.21.15.66:0/13263148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd3390 0x557706b86de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6812000640 1 -- 172.21.15.66:0/13263148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557706982df0 con 0x557706bd2a70 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f681b400640 1 --2- 172.21.15.66:0/13263148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd3390 0x557706b86de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f6812000640 1 --2- 172.21.15.66:0/13263148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b86170 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f6804051840 tx=0x7f6804084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee94ce451cb9619 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f680be00640 1 -- 172.21.15.66:0/13263148 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680406cea0 con 0x557706bd2a70 2024-03-30T00:51:29.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f680be00640 1 -- 172.21.15.66:0/13263148 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680406b040 con 0x557706bd2a70 2024-03-30T00:51:29.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.407+0000 7f680be00640 1 -- 172.21.15.66:0/13263148 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040536e0 con 0x557706bd2a70 2024-03-30T00:51:29.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 -- 172.21.15.66:0/13263148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 msgr2=0x557706b86170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 --2- 172.21.15.66:0/13263148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b86170 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f6804051840 tx=0x7f6804084040 comp rx=0 tx=0).stop 2024-03-30T00:51:29.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 -- 172.21.15.66:0/13263148 shutdown_connections 2024-03-30T00:51:29.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 --2- 172.21.15.66:0/13263148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706bd3390 0x557706b86de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 --2- 172.21.15.66:0/13263148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b86170 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 --2- 172.21.15.66:0/13263148 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706b86a00 0x557706b85c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 -- 172.21.15.66:0/13263148 >> 172.21.15.66:0/13263148 conn(0x557706987160 msgr2=0x557706b8e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 -- 172.21.15.66:0/13263148 shutdown_connections 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 -- 172.21.15.66:0/13263148 wait complete. 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 -- start start 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 Processor -- start 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706b86a00 0x557706c32680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b89ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706b8a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557706a2f920 con 0x557706bd2a70 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577069d6920 con 0x557706b86a00 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x557706b80930 con 0x557706bd3390 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 10 client.0 _ll_register_callbacks cb 0x7f6814009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6812000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b89ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6812000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b89ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40200/0 (socket says 172.21.15.66:40200) 2024-03-30T00:51:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6812000640 1 -- 172.21.15.66:0/1373070774 learned_addr learned my addr 172.21.15.66:0/1373070774 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f681aa00640 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706b86a00 0x557706c32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f681b400640 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706b8a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6812000640 1 -- 172.21.15.66:0/1373070774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 msgr2=0x557706b8a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6812000640 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706b8a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6812000640 1 -- 172.21.15.66:0/1373070774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706b86a00 msgr2=0x557706c32680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6812000640 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706b86a00 0x557706c32680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6812000640 1 -- 172.21.15.66:0/1373070774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c0adce0 con 0x557706bd2a70 2024-03-30T00:51:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f681aa00640 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706b86a00 0x557706c32680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:29.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f681b400640 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706b8a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6812000640 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b89ba0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f680400c520 tx=0x7f6804082660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fd309b63121127 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f67ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f67ffe00640 1 -- 172.21.15.66:0/1373070774 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680406cea0 con 0x557706bd2a70 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f67ffe00640 1 -- 172.21.15.66:0/1373070774 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f68040840b0 con 0x557706bd2a70 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 -- 172.21.15.66:0/1373070774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557706994970 con 0x557706bd2a70 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f67ffe00640 1 -- 172.21.15.66:0/1373070774 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6804073290 con 0x557706bd2a70 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 10 client.9927 fetch_fsmap learned FSMap version 101 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 1 -- 172.21.15.66:0/1373070774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) v3 -- 0x557706976960 con 0x557706bd2a70 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f67ffe00640 1 -- 172.21.15.66:0/1373070774 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557706994970 con 0x557706bd2a70 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f67ffe00640 1 -- 172.21.15.66:0/1373070774 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 101) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6804011070 con 0x557706bd2a70 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 10 client.9927 fetch_fsmap finished waiting for FSMap version 101 2024-03-30T00:51:29.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 10 client.9927 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-30T00:51:29.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.411+0000 7f6824ac6000 20 client.9927 populate_metadata read hostname 'smithi066' 2024-03-30T00:51:29.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.415+0000 7f6824ac6000 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x557706bee1e0 0x557706bee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.415+0000 7f6824ac6000 4 client.9927 mds_command: new command op to 9732 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:51:29.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.415+0000 7f6824ac6000 1 -- 172.21.15.66:0/1373070774 --> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557706a06a20 con 0x557706bee1e0 2024-03-30T00:51:29.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.415+0000 7f681aa00640 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x557706bee1e0 0x557706bee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.415+0000 7f681aa00640 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x557706bee1e0 0x557706bee5f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c635f7e2eb95d4 server_cookie=fdf771d588cb4081 in_seq=0 out_seq=0 2024-03-30T00:51:29.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.415+0000 7f67ffe00640 10 client.9927 ms_handle_connect on v2:172.21.15.66:6834/3460522956 2024-03-30T00:51:29.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f67ffe00640 1 -- 172.21.15.66:0/1373070774 <== mds.0 v2:172.21.15.66:6834/3460522956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557706a06a20 con 0x557706bee1e0 2024-03-30T00:51:29.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f67ffe00640 10 client.9927 handle_command_reply: tid=0 2024-03-30T00:51:29.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 1 client.9927 shutdown 2024-03-30T00:51:29.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 1 -- 172.21.15.66:0/1373070774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 msgr2=0x557706b89ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b89ba0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f680400c520 tx=0x7f6804082660 comp rx=0 tx=0).stop 2024-03-30T00:51:29.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 1 -- 172.21.15.66:0/1373070774 shutdown_connections 2024-03-30T00:51:29.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.66:6834/3460522956,v1:172.21.15.66:6836/3460522956] conn(0x557706bee1e0 0x557706bee5f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x557706bd3390 0x557706b8a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557706bd2a70 0x557706b89ba0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f67ffe00640 0 client.9927 ms_handle_reset on v2:172.21.15.66:6834/3460522956 2024-03-30T00:51:29.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 1 --2- 172.21.15.66:0/1373070774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557706b86a00 0x557706c32680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 1 -- 172.21.15.66:0/1373070774 >> 172.21.15.66:0/1373070774 conn(0x557706987160 msgr2=0x557706b84c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 1 -- 172.21.15.66:0/1373070774 shutdown_connections 2024-03-30T00:51:29.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 1 -- 172.21.15.66:0/1373070774 wait complete. 2024-03-30T00:51:29.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6824ac6000 20 client.9927 trim_cache size 0 max 16384 2024-03-30T00:51:29.439 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:29.439 INFO:teuthology.orchestra.run.smithi066.stdout: "message": "", 2024-03-30T00:51:29.439 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0 2024-03-30T00:51:29.439 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:29.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 -- 172.21.15.66:0/565513884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67f8066690 msgr2=0x7f67f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67f8066690 0x7f67f8068b50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f680c0b0000 tx=0x7f680c00d040 comp rx=0 tx=0).stop 2024-03-30T00:51:29.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 -- 172.21.15.66:0/565513884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f681c1533b0 msgr2=0x7f681c0b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f681c1533b0 0x7f681c0b2180 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f680400e900 tx=0x7f680400edc0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 -- 172.21.15.66:0/565513884 shutdown_connections 2024-03-30T00:51:29.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f681c159d80 0x7f681c0aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f67f8066690 0x7f67f8068b50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f681c1533b0 0x7f681c0b2180 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 --2- 172.21.15.66:0/565513884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f681c0afe50 0x7f681c0ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 -- 172.21.15.66:0/565513884 >> 172.21.15.66:0/565513884 conn(0x7f681c069cf0 msgr2=0x7f681c162fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 -- 172.21.15.66:0/565513884 shutdown_connections 2024-03-30T00:51:29.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.427+0000 7f6821600640 1 -- 172.21.15.66:0/565513884 wait complete. 2024-03-30T00:51:29.455 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:51:29.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.587+0000 7ff7f1c00640 1 Processor -- start 2024-03-30T00:51:29.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.587+0000 7ff7f1c00640 1 -- start start 2024-03-30T00:51:29.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.587+0000 7ff7f1c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7ec0cda30 0x7ff7ec0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.587+0000 7ff7f1c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7ec0cb1d0 0x7ff7ec0b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.587+0000 7ff7f1c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ec0ce010 con 0x7ff7ec0b2c10 2024-03-30T00:51:29.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.587+0000 7ff7f1c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ec0cb620 con 0x7ff7ec0cda30 2024-03-30T00:51:29.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.587+0000 7ff7f1c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0b2ff0 con 0x7ff7ec0cb1d0 2024-03-30T00:51:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.587+0000 7ff7e2000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7ec0cda30 0x7ff7ec0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.587+0000 7ff7e2000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7ec0cda30 0x7ff7ec0cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40210/0 (socket says 172.21.15.66:40210) 2024-03-30T00:51:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.587+0000 7ff7eaa00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff7ec0b2c10 0x7ff7ec15ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:59592/0 (socket says 172.21.15.66:59592) 2024-03-30T00:51:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.587+0000 7ff7e2000640 1 -- 172.21.15.66:0/2373783883 learned_addr learned my addr 172.21.15.66:0/2373783883 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7eb400640 1 --2- 172.21.15.66:0/2373783883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7ec0cb1d0 0x7ff7ec0b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7e2000640 1 -- 172.21.15.66:0/2373783883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7ec0cb1d0 msgr2=0x7ff7ec0b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7e2000640 1 --2- 172.21.15.66:0/2373783883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7ec0cb1d0 0x7ff7ec0b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7e2000640 1 -- 172.21.15.66:0/2373783883 >> v1:172.21.15.66:6789/0 conn(0x7ff7ec0b2c10 legacy=0x7ff7ec15ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7e2000640 1 -- 172.21.15.66:0/2373783883 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec06cfc0 con 0x7ff7ec0cda30 2024-03-30T00:51:29.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7e2000640 1 --2- 172.21.15.66:0/2373783883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ec0cda30 0x7ff7ec0cac90 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009870 tx=0x7ff7d401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67dd77945a69011 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7ea000640 1 -- 172.21.15.66:0/2373783883 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40ab600 con 0x7ff7ec0cda30 2024-03-30T00:51:29.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7ea000640 1 -- 172.21.15.66:0/2373783883 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d40a7050 con 0x7ff7ec0cda30 2024-03-30T00:51:29.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7ea000640 1 -- 172.21.15.66:0/2373783883 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40ab980 con 0x7ff7ec0cda30 2024-03-30T00:51:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2373783883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ec0cda30 msgr2=0x7ff7ec0cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7f1c00640 1 --2- 172.21.15.66:0/2373783883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ec0cda30 0x7ff7ec0cac90 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009870 tx=0x7ff7d401c530 comp rx=0 tx=0).stop 2024-03-30T00:51:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2373783883 shutdown_connections 2024-03-30T00:51:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7f1c00640 1 --2- 172.21.15.66:0/2373783883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7ec0cb1d0 0x7ff7ec0b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7f1c00640 1 --2- 172.21.15.66:0/2373783883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ec0cda30 0x7ff7ec0cac90 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2373783883 >> 172.21.15.66:0/2373783883 conn(0x7ff7ec069cf0 msgr2=0x7ff7ec06a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2373783883 shutdown_connections 2024-03-30T00:51:29.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.591+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2373783883 wait complete. 2024-03-30T00:51:29.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7f1c00640 1 Processor -- start 2024-03-30T00:51:29.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7f1c00640 1 -- start start 2024-03-30T00:51:29.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7f1c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7ec0b2c10 0x7ff7ec0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7f1c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec0cb1d0 0x7ff7ec0c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7f1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ec0cda30 0x7ff7ec0c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7f1c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0c81f0 con 0x7ff7ec0cda30 2024-03-30T00:51:29.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7f1c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0c5370 con 0x7ff7ec0cb1d0 2024-03-30T00:51:29.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7f1c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec1576e0 con 0x7ff7ec0b2c10 2024-03-30T00:51:29.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7eb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ec0cda30 0x7ff7ec0c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7eb400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ec0cda30 0x7ff7ec0c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40216/0 (socket says 172.21.15.66:40216) 2024-03-30T00:51:29.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7eb400640 1 -- 172.21.15.66:0/2064032890 learned_addr learned my addr 172.21.15.66:0/2064032890 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7eb400640 1 -- 172.21.15.66:0/2064032890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7ec0b2c10 msgr2=0x7ff7ec0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7eb400640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7ec0b2c10 0x7ff7ec0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7eb400640 1 -- 172.21.15.66:0/2064032890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec0cb1d0 msgr2=0x7ff7ec0c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7eb400640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec0cb1d0 0x7ff7ec0c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7eb400640 1 -- 172.21.15.66:0/2064032890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40094d0 con 0x7ff7ec0cda30 2024-03-30T00:51:29.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7eb400640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ec0cda30 0x7ff7ec0c6370 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4007e40 tx=0x7ff7e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7e3400640 1 -- 172.21.15.66:0/2064032890 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4014620 con 0x7ff7ec0cda30 2024-03-30T00:51:29.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7e3400640 1 -- 172.21.15.66:0/2064032890 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e400ee90 con 0x7ff7ec0cda30 2024-03-30T00:51:29.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2064032890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ec1579c0 con 0x7ff7ec0cda30 2024-03-30T00:51:29.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7e3400640 1 -- 172.21.15.66:0/2064032890 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40149c0 con 0x7ff7ec0cda30 2024-03-30T00:51:29.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.595+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2064032890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ec0c5910 con 0x7ff7ec0cda30 2024-03-30T00:51:29.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.599+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2064032890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ec0c5b00 con 0x7ff7ec0cda30 2024-03-30T00:51:29.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.599+0000 7ff7e3400640 1 -- 172.21.15.66:0/2064032890 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff7e4014b20 con 0x7ff7ec0cda30 2024-03-30T00:51:29.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.599+0000 7ff7e3400640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff7c80666c0 0x7ff7c8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.599+0000 7ff7e3400640 1 -- 172.21.15.66:0/2064032890 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff7e4084ce0 con 0x7ff7ec0cda30 2024-03-30T00:51:29.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.599+0000 7ff7eaa00640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff7c80666c0 0x7ff7c8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.599+0000 7ff7eaa00640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff7c80666c0 0x7ff7c8068b80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0ccb30 tx=0x7ff7dc007990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.603+0000 7ff7e3400640 1 -- 172.21.15.66:0/2064032890 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff7e4051eb0 con 0x7ff7ec0cda30 2024-03-30T00:51:29.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.803+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2064032890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff7ec066c40 con 0x7ff7ec0cda30 2024-03-30T00:51:29.810 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:51:29.803+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:51:29.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7e3400640 1 -- 172.21.15.66:0/2064032890 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff7e40554d0 con 0x7ff7ec0cda30 2024-03-30T00:51:29.819 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:51:29.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2064032890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff7c80666c0 msgr2=0x7ff7c8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff7c80666c0 0x7ff7c8068b80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0ccb30 tx=0x7ff7dc007990 comp rx=0 tx=0).stop 2024-03-30T00:51:29.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2064032890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ec0cda30 msgr2=0x7ff7ec0c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ec0cda30 0x7ff7ec0c6370 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4007e40 tx=0x7ff7e400e4b0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2064032890 shutdown_connections 2024-03-30T00:51:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff7c80666c0 0x7ff7c8068b80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ec0cda30 0x7ff7ec0c6370 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec0cb1d0 0x7ff7ec0c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 --2- 172.21.15.66:0/2064032890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7ec0b2c10 0x7ff7ec0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2064032890 >> 172.21.15.66:0/2064032890 conn(0x7ff7ec069cf0 msgr2=0x7ff7ec0c4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2064032890 shutdown_connections 2024-03-30T00:51:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.811+0000 7ff7f1c00640 1 -- 172.21.15.66:0/2064032890 wait complete. 2024-03-30T00:51:29.827 INFO:tasks.ceph.mds.a.smithi066.stderr: -2123> 2024-03-30T00:51:21.179+0000 7f091d800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:51:29.843 DEBUG:teuthology.orchestra.run.smithi066:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-03-30T00:51:29.875 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:51:29.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a89b6400 1 -- start start 2024-03-30T00:51:29.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a89b6400 1 Processor -- start 2024-03-30T00:51:29.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f21b7800 0x5650f2242580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f21b7400 0x5650f2242b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f21b7000 0x5650f2243080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a89b6400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f2284a00 con 0x5650f21b7400 2024-03-30T00:51:29.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a89b6400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f2284780 con 0x5650f21b7000 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a89b6400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650f2284640 con 0x5650f21b7800 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f21b7400 0x5650f2242b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f21b7400 0x5650f2242b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40248/0 (socket says 172.21.15.66:40248) 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a6000640 1 -- 172.21.15.66:0/2095934284 learned_addr learned my addr 172.21.15.66:0/2095934284 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a6a00640 1 --2- 172.21.15.66:0/2095934284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f21b7800 0x5650f2242580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a7400640 1 --2- 172.21.15.66:0/2095934284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f21b7000 0x5650f2243080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a6000640 1 -- 172.21.15.66:0/2095934284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f21b7800 msgr2=0x5650f2242580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a6000640 1 --2- 172.21.15.66:0/2095934284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f21b7800 0x5650f2242580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a6000640 1 -- 172.21.15.66:0/2095934284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f21b7000 msgr2=0x5650f2243080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a6000640 1 --2- 172.21.15.66:0/2095934284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f21b7000 0x5650f2243080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.883+0000 7f43a6000640 1 -- 172.21.15.66:0/2095934284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f21ec9c0 con 0x5650f21b7400 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a6000640 1 --2- 172.21.15.66:0/2095934284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f21b7400 0x5650f2242b00 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x5650f302ac90 tx=0x5650f30b6d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f46d88a2b6e74a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a3800640 1 -- 172.21.15.66:0/2095934284 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f220b600 con 0x5650f21b7400 2024-03-30T00:51:29.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a3800640 1 -- 172.21.15.66:0/2095934284 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650f2214480 con 0x5650f21b7400 2024-03-30T00:51:29.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a3800640 1 -- 172.21.15.66:0/2095934284 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f220b1e0 con 0x5650f21b7400 2024-03-30T00:51:29.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 -- 172.21.15.66:0/2095934284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650f21ecb60 con 0x5650f21b7400 2024-03-30T00:51:29.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 -- 172.21.15.66:0/2095934284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5650f21ec340 con 0x5650f21b7400 2024-03-30T00:51:29.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 4 waiting for MDS map... 2024-03-30T00:51:29.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a3800640 1 -- 172.21.15.66:0/2095934284 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x5650f21f6c40 con 0x5650f21b7400 2024-03-30T00:51:29.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a3800640 1 -- 172.21.15.66:0/2095934284 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 102) v1 ==== 2573+0+0 (secure 0 0 0) 0x5650f2204f00 con 0x5650f21b7400 2024-03-30T00:51:29.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 4 Got MDS map 102 2024-03-30T00:51:29.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 10 main: JournalTool::main 2024-03-30T00:51:29.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 4 main: JournalTool: connecting to RADOS... 2024-03-30T00:51:29.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 Processor -- start 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 -- start start 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cec00 0x5650f2243600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cf000 0x5650f2243b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f30cf400 0x5650f2244100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f21fde00 con 0x5650f30cf000 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f2284780 con 0x5650f30cf400 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650f2284500 con 0x5650f30cec00 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cf000 0x5650f2243b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a7400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cec00 0x5650f2243600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cf000 0x5650f2243b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40258/0 (socket says 172.21.15.66:40258) 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a6a00640 1 -- 172.21.15.66:0/469436575 learned_addr learned my addr 172.21.15.66:0/469436575 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a6000640 1 --2- 172.21.15.66:0/469436575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f30cf400 0x5650f2244100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a6a00640 1 -- 172.21.15.66:0/469436575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cec00 msgr2=0x5650f2243600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a6a00640 1 --2- 172.21.15.66:0/469436575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cec00 0x5650f2243600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a6a00640 1 -- 172.21.15.66:0/469436575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f30cf400 msgr2=0x5650f2244100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a6a00640 1 --2- 172.21.15.66:0/469436575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f30cf400 0x5650f2244100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a6a00640 1 -- 172.21.15.66:0/469436575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f21ec000 con 0x5650f30cf000 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a6a00640 1 --2- 172.21.15.66:0/469436575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cf000 0x5650f2243b80 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x5650f313c0f0 tx=0x5650f30e4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4806696e1a0b995 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a1000640 1 -- 172.21.15.66:0/469436575 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f220b600 con 0x5650f30cf000 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a1000640 1 -- 172.21.15.66:0/469436575 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650f2f8ea80 con 0x5650f30cf000 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a1000640 1 -- 172.21.15.66:0/469436575 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f220b1e0 con 0x5650f30cf000 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 -- 172.21.15.66:0/469436575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cf000 msgr2=0x5650f2243b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.887+0000 7f43a89b6400 1 --2- 172.21.15.66:0/469436575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cf000 0x5650f2243b80 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x5650f313c0f0 tx=0x5650f30e4c00 comp rx=0 tx=0).stop 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 -- 172.21.15.66:0/469436575 shutdown_connections 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 --2- 172.21.15.66:0/469436575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f30cf400 0x5650f2244100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 --2- 172.21.15.66:0/469436575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cf000 0x5650f2243b80 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 --2- 172.21.15.66:0/469436575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cec00 0x5650f2243600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 -- 172.21.15.66:0/469436575 >> 172.21.15.66:0/469436575 conn(0x5650f21b6c00 msgr2=0x5650f21e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 -- 172.21.15.66:0/469436575 shutdown_connections 2024-03-30T00:51:29.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 -- 172.21.15.66:0/469436575 wait complete. 2024-03-30T00:51:29.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 Processor -- start 2024-03-30T00:51:29.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 -- start start 2024-03-30T00:51:29.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cec00 0x5650f2245180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cf000 0x5650f2244c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f30cf400 0x5650f2244680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f22843c0 con 0x5650f30cec00 2024-03-30T00:51:29.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f2284280 con 0x5650f30cf400 2024-03-30T00:51:29.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650f2284140 con 0x5650f30cf000 2024-03-30T00:51:29.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cec00 0x5650f2245180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a7400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cec00 0x5650f2245180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40262/0 (socket says 172.21.15.66:40262) 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a7400640 1 -- 172.21.15.66:0/1985420298 learned_addr learned my addr 172.21.15.66:0/1985420298 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a6000640 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f30cf400 0x5650f2244680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a6a00640 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cf000 0x5650f2244c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a7400640 1 -- 172.21.15.66:0/1985420298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cf000 msgr2=0x5650f2244c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a7400640 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cf000 0x5650f2244c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a7400640 1 -- 172.21.15.66:0/1985420298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f30cf400 msgr2=0x5650f2244680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a7400640 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f30cf400 0x5650f2244680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a7400640 1 -- 172.21.15.66:0/1985420298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f21ec9c0 con 0x5650f30cec00 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a7400640 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cec00 0x5650f2245180 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x5650f313cd20 tx=0x5650f2291860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f439f200640 1 -- 172.21.15.66:0/1985420298 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f220b080 con 0x5650f30cec00 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f439f200640 1 -- 172.21.15.66:0/1985420298 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650f2f8e300 con 0x5650f30cec00 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f439f200640 1 -- 172.21.15.66:0/1985420298 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f220b600 con 0x5650f30cec00 2024-03-30T00:51:29.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 -- 172.21.15.66:0/1985420298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5650f21ecb60 con 0x5650f30cec00 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 1 -- 172.21.15.66:0/1985420298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650f21ecea0 con 0x5650f30cec00 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.891+0000 7f43a89b6400 4 main: JournalTool: resolving pool 17 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f439f200640 1 -- 172.21.15.66:0/1985420298 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x5650f2283080 con 0x5650f30cec00 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f439f200640 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5650f249ec00 0x5650f2245700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f439f200640 1 -- 172.21.15.66:0/1985420298 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x5650f21f6a80 con 0x5650f30cec00 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a89b6400 4 main: JournalTool: creating IoCtx.. 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a89b6400 4 main: Executing for rank 0 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a6a00640 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5650f249ec00 0x5650f2245700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a6a00640 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5650f249ec00 0x5650f2245700 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5650f313c0f0 tx=0x5650f30b6c60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a89b6400 1 -- start start 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a89b6400 1 Processor -- start 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cfc00 0x5650f2244100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f249f000 0x5650f2243b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a89b6400 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f249f400 0x5650f2243600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a89b6400 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f2284280 con 0x5650f249f000 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a89b6400 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f22843c0 con 0x5650f249f400 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a89b6400 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650f2284500 con 0x5650f30cfc00 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f249f000 0x5650f2243b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f249f000 0x5650f2243b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40272/0 (socket says 172.21.15.66:40272) 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a7400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cfc00 0x5650f2244100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a6a00640 1 -- 172.21.15.66:0/921736396 learned_addr learned my addr 172.21.15.66:0/921736396 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.895+0000 7f43a6000640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f249f400 0x5650f2243600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a7400640 1 -- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f249f400 msgr2=0x5650f2243600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a7400640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f249f400 0x5650f2243600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a7400640 1 -- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f249f000 msgr2=0x5650f2243b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a7400640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f249f000 0x5650f2243b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a7400640 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f24e3520 con 0x5650f30cfc00 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a6a00640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f249f000 0x5650f2243b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a7400640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cfc00 0x5650f2244100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5650f2502ba0 tx=0x5650f30e53e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ee7fce72579f2c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f439b600640 1 -- 172.21.15.66:0/921736396 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f220b1e0 con 0x5650f30cfc00 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f439b600640 1 -- 172.21.15.66:0/921736396 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650f2f8ef00 con 0x5650f30cfc00 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f439b600640 1 -- 172.21.15.66:0/921736396 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f220a420 con 0x5650f30cfc00 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650f24e3380 con 0x5650f30cfc00 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5650f24e31e0 con 0x5650f30cfc00 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a89b6400 4 waiting for MDS map... 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f439b600640 1 -- 172.21.15.66:0/921736396 <== mon.2 v2:172.21.15.156:3301/0 4 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x5650f21f68c0 con 0x5650f30cfc00 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f439b600640 1 -- 172.21.15.66:0/921736396 <== mon.2 v2:172.21.15.156:3301/0 5 ==== fsmap(e 102) v1 ==== 2573+0+0 (secure 0 0 0) 0x5650f25f8000 con 0x5650f30cfc00 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a89b6400 4 Got MDS map 102 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a89b6400 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5650f249f800 0x5650f2608000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x5650f249fc00 con 0x5650f249f800 2024-03-30T00:51:29.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a6a00640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5650f249f800 0x5650f2608000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a6a00640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5650f249f800 0x5650f2608000 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=b015e16c98b070ac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a6a00640 1 -- 172.21.15.66:0/921736396 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v127'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5650f30b2b40 con 0x5650f249f800 2024-03-30T00:51:29.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a89b6400 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x5650f25f7400 0x5650f2608580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.899+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x5650f25f7800 con 0x5650f25f7400 2024-03-30T00:51:29.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.903+0000 7f43a6000640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x5650f25f7400 0x5650f2608580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.903+0000 7f43a6000640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x5650f25f7400 0x5650f2608580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=3403caf6f9f5c90d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a6000640 1 -- 172.21.15.66:0/921736396 <== osd.3 v2:172.21.15.66:6808/27471 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v127'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5650f30b2d80 con 0x5650f25f7400 2024-03-30T00:51:29.914 INFO:teuthology.orchestra.run.smithi066.stdout:writing EResetJournal entry 2024-03-30T00:51:29.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x5650f25f7c00 con 0x5650f25f7400 2024-03-30T00:51:29.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a89b6400 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x5650f2618000 0x5650f2608b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x5650f2618400 con 0x5650f2618000 2024-03-30T00:51:29.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a89b6400 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5650f2618800 0x5650f2609080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x5650f2618c00 con 0x5650f2618800 2024-03-30T00:51:29.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a89b6400 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5650f2619000 0x5650f2609600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:29.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x5650f2619400 con 0x5650f2619000 2024-03-30T00:51:29.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x5650f2619800 con 0x5650f2618800 2024-03-30T00:51:29.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x5650f2619c00 con 0x5650f249f800 2024-03-30T00:51:29.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a6a00640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5650f2618800 0x5650f2609080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a7400640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x5650f2618000 0x5650f2608b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a6000640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5650f2619000 0x5650f2609600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:29.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a6a00640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5650f2618800 0x5650f2609080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=527c447f6efa6690 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a7400640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x5650f2618000 0x5650f2608b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=71c397ba79300859 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a6000640 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5650f2619000 0x5650f2609600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=222b281af5ffcd81 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:29.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a6000640 1 -- 172.21.15.66:0/921736396 <== osd.3 v2:172.21.15.66:6808/27471 2 ==== osd_op_reply(3 200.00000001 [delete] v127'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5650f30b2d80 con 0x5650f25f7400 2024-03-30T00:51:29.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a6a00640 1 -- 172.21.15.66:0/921736396 <== osd.1 v2:172.21.15.66:6821/27458 2 ==== osd_op_reply(8 200.00000006 [delete] v127'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5650f30b2b40 con 0x5650f249f800 2024-03-30T00:51:29.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f43a7400640 1 -- 172.21.15.66:0/921736396 <== osd.4 v2:172.21.15.156:6800/27641 1 ==== osd_op_reply(4 200.00000002 [delete] v127'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5650f30b3440 con 0x5650f2618000 2024-03-30T00:51:29.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.907+0000 7f4399800640 1 -- 172.21.15.66:0/921736396 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x5650f30ce800 con 0x5650f25f7400 2024-03-30T00:51:29.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a6a00640 1 -- 172.21.15.66:0/921736396 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== osd_op_reply(5 200.00000003 [delete] v127'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5650f30b2b40 con 0x5650f2618800 2024-03-30T00:51:29.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a6a00640 1 -- 172.21.15.66:0/921736396 <== osd.0 v2:172.21.15.66:6800/27456 2 ==== osd_op_reply(7 200.00000005 [delete] v127'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5650f30b2b40 con 0x5650f2618800 2024-03-30T00:51:29.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a6000640 1 -- 172.21.15.66:0/921736396 <== osd.3 v2:172.21.15.66:6808/27471 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v127'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5650f30b2d80 con 0x5650f25f7400 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a6000640 1 -- 172.21.15.66:0/921736396 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(6 200.00000004 [delete] v127'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5650f30b2d80 con 0x5650f2619000 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 4 Successfully wrote new journal pointer and header for rank 8:0 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5650f2618800 msgr2=0x5650f2609080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5650f2618800 0x5650f2609080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5650f249f800 msgr2=0x5650f2608000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5650f249f800 0x5650f2608000 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x5650f25f7400 msgr2=0x5650f2608580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x5650f25f7400 0x5650f2608580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x5650f2618000 msgr2=0x5650f2608b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x5650f2618000 0x5650f2608b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5650f2619000 msgr2=0x5650f2609600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5650f2619000 0x5650f2609600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a6000640 1 -- 172.21.15.66:0/921736396 reap_dead start 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cfc00 msgr2=0x5650f2244100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/921736396 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cfc00 0x5650f2244100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5650f2502ba0 tx=0x5650f30e53e0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a6000640 1 -- 172.21.15.66:0/921736396 reap_dead start 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a6000640 1 -- 172.21.15.66:0/921736396 reap_dead start 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 shutdown_connections 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 >> 172.21.15.66:0/921736396 conn(0x5650f30cf800 msgr2=0x5650f25e4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 shutdown_connections 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/921736396 wait complete. 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/1985420298 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5650f249ec00 msgr2=0x5650f2245700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5650f249ec00 0x5650f2245700 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5650f313c0f0 tx=0x5650f30b6c60 comp rx=0 tx=0).stop 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/1985420298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cec00 msgr2=0x5650f2245180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cec00 0x5650f2245180 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x5650f313cd20 tx=0x5650f2291860 comp rx=0 tx=0).stop 2024-03-30T00:51:29.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/1985420298 shutdown_connections 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f30cf400 0x5650f2244680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f30cf000 0x5650f2244c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5650f249ec00 0x5650f2245700 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/1985420298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f30cec00 0x5650f2245180 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/1985420298 >> 172.21.15.66:0/1985420298 conn(0x5650f21b6c00 msgr2=0x5650f21e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/1985420298 shutdown_connections 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/1985420298 wait complete. 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 -- 172.21.15.66:0/2095934284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f21b7400 msgr2=0x5650f2242b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.911+0000 7f43a89b6400 1 --2- 172.21.15.66:0/2095934284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f21b7400 0x5650f2242b00 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x5650f302ac90 tx=0x5650f30b6d20 comp rx=0 tx=0).stop 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.915+0000 7f43a89b6400 1 -- 172.21.15.66:0/2095934284 shutdown_connections 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.915+0000 7f43a89b6400 1 --2- 172.21.15.66:0/2095934284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650f21b7000 0x5650f2243080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.915+0000 7f43a89b6400 1 --2- 172.21.15.66:0/2095934284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650f21b7400 0x5650f2242b00 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.915+0000 7f43a89b6400 1 --2- 172.21.15.66:0/2095934284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5650f21b7800 0x5650f2242580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.915+0000 7f43a89b6400 1 -- 172.21.15.66:0/2095934284 >> 172.21.15.66:0/2095934284 conn(0x5650f21b7c00 msgr2=0x5650f21e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:29.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.915+0000 7f43a89b6400 1 -- 172.21.15.66:0/2095934284 shutdown_connections 2024-03-30T00:51:29.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:29.915+0000 7f43a89b6400 1 -- 172.21.15.66:0/2095934284 wait complete. 2024-03-30T00:51:29.926 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.083300, result: writing EResetJournal entry 2024-03-30T00:51:29.926 DEBUG:teuthology.orchestra.run.smithi066:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-03-30T00:51:30.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dae3208c0 1 -- start start 2024-03-30T00:51:30.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dae3208c0 1 Processor -- start 2024-03-30T00:51:30.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dae3208c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6a759800 0x562f6a7e4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dae3208c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6a759400 0x562f6a7e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dae3208c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6a759000 0x562f6a7e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dae3208c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6a826a00 con 0x562f6a759000 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dae3208c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6a826780 con 0x562f6a759400 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dae3208c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f6a826640 con 0x562f6a759800 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dace00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6a759000 0x562f6a7e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dace00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6a759000 0x562f6a7e5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40278/0 (socket says 172.21.15.66:40278) 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dace00640 1 -- 172.21.15.66:0/1316138910 learned_addr learned my addr 172.21.15.66:0/1316138910 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dace00640 1 -- 172.21.15.66:0/1316138910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6a759800 msgr2=0x562f6a7e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dace00640 1 --2- 172.21.15.66:0/1316138910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6a759800 0x562f6a7e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dace00640 1 -- 172.21.15.66:0/1316138910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6a759400 msgr2=0x562f6a7e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dace00640 1 --2- 172.21.15.66:0/1316138910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6a759400 0x562f6a7e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dace00640 1 -- 172.21.15.66:0/1316138910 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f6a78c9c0 con 0x562f6a759000 2024-03-30T00:51:30.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dace00640 1 --2- 172.21.15.66:0/1316138910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6a759000 0x562f6a7e5080 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x562f6b5bf350 tx=0x562f6b664000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5460aeb2dd5975e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9da9200640 1 -- 172.21.15.66:0/1316138910 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562f6a7ad600 con 0x562f6a759000 2024-03-30T00:51:30.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9da9200640 1 -- 172.21.15.66:0/1316138910 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f6a7b6480 con 0x562f6a759000 2024-03-30T00:51:30.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9da9200640 1 -- 172.21.15.66:0/1316138910 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562f6a7ad1e0 con 0x562f6a759000 2024-03-30T00:51:30.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1316138910 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f6a78cb60 con 0x562f6a759000 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1316138910 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562f6a78c340 con 0x562f6a759000 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9dae3208c0 4 waiting for MDS map... 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.003+0000 7f9da9200640 1 -- 172.21.15.66:0/1316138910 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x562f6a798c40 con 0x562f6a759000 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9da9200640 1 -- 172.21.15.66:0/1316138910 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 102) v1 ==== 2573+0+0 (secure 0 0 0) 0x562f6a7a6f00 con 0x562f6a759000 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 4 Got MDS map 102 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 4 datascan.main: Using metadata pool output 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 4 datascan.main: connecting to RADOS... 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 Processor -- start 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 -- start start 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6b666c00 0x562f6a7e5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b667000 0x562f6a7e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6b667400 0x562f6a7e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6a79fe00 con 0x562f6b667000 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6a826780 con 0x562f6b667400 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f6a826500 con 0x562f6b666c00 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dac400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b667000 0x562f6a7e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dac400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b667000 0x562f6a7e5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40294/0 (socket says 172.21.15.66:40294) 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dac400640 1 -- 172.21.15.66:0/3410024970 learned_addr learned my addr 172.21.15.66:0/3410024970 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9daba00640 1 --2- 172.21.15.66:0/3410024970 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6b666c00 0x562f6a7e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9daba00640 1 --2- 172.21.15.66:0/3410024970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6b667400 0x562f6a7e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dac400640 1 -- 172.21.15.66:0/3410024970 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6b666c00 msgr2=0x562f6a7e5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dac400640 1 --2- 172.21.15.66:0/3410024970 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6b666c00 0x562f6a7e5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dac400640 1 -- 172.21.15.66:0/3410024970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6b667400 msgr2=0x562f6a7e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dac400640 1 --2- 172.21.15.66:0/3410024970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6b667400 0x562f6a7e6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dac400640 1 -- 172.21.15.66:0/3410024970 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f6a78c000 con 0x562f6b667000 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dac400640 1 --2- 172.21.15.66:0/3410024970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b667000 0x562f6a7e5b80 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x562f6b680690 tx=0x562f6b684d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8639c62edd1f6a2a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9da6a00640 1 -- 172.21.15.66:0/3410024970 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562f6a7ad600 con 0x562f6b667000 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9da6a00640 1 -- 172.21.15.66:0/3410024970 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f6a7b6900 con 0x562f6b667000 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9da6a00640 1 -- 172.21.15.66:0/3410024970 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562f6a7ad1e0 con 0x562f6b667000 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 -- 172.21.15.66:0/3410024970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b667000 msgr2=0x562f6a7e5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/3410024970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b667000 0x562f6a7e5b80 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x562f6b680690 tx=0x562f6b684d20 comp rx=0 tx=0).stop 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 -- 172.21.15.66:0/3410024970 shutdown_connections 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/3410024970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6b667400 0x562f6a7e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/3410024970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b667000 0x562f6a7e5b80 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/3410024970 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6b666c00 0x562f6a7e5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 -- 172.21.15.66:0/3410024970 >> 172.21.15.66:0/3410024970 conn(0x562f6a758c00 msgr2=0x562f6a785800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 -- 172.21.15.66:0/3410024970 shutdown_connections 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.007+0000 7f9dae3208c0 1 -- 172.21.15.66:0/3410024970 wait complete. 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dae3208c0 1 Processor -- start 2024-03-30T00:51:30.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dae3208c0 1 -- start start 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dae3208c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b666c00 0x562f6a7e7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dae3208c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6b667000 0x562f6a7e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dae3208c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6b667400 0x562f6a7e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dae3208c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6a8263c0 con 0x562f6b666c00 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dae3208c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6a826280 con 0x562f6b667400 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dae3208c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f6a826140 con 0x562f6b667000 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9daba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b666c00 0x562f6a7e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9daba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b666c00 0x562f6a7e7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40296/0 (socket says 172.21.15.66:40296) 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 learned_addr learned my addr 172.21.15.66:0/1371700619 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dac400640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6b667000 0x562f6a7e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9daba00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6b667400 0x562f6a7e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6b667000 msgr2=0x562f6a7e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9daba00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6b667000 0x562f6a7e6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6b667400 msgr2=0x562f6a7e6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9daba00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6b667400 0x562f6a7e6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f6a78c9c0 con 0x562f6b666c00 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dac400640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6b667000 0x562f6a7e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9daba00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b666c00 0x562f6a7e7180 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x562f6b5be9f0 tx=0x562f6b4e20c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9da4c00640 1 -- 172.21.15.66:0/1371700619 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562f6a7ad080 con 0x562f6b666c00 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9da4c00640 1 -- 172.21.15.66:0/1371700619 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f6b532300 con 0x562f6b666c00 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9da4c00640 1 -- 172.21.15.66:0/1371700619 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562f6a7ad600 con 0x562f6b666c00 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562f6a78cb60 con 0x562f6b666c00 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f6a78cea0 con 0x562f6b666c00 2024-03-30T00:51:30.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.011+0000 7f9dae3208c0 4 datascan.init: resolving metadata pool 17 2024-03-30T00:51:30.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9da4c00640 1 -- 172.21.15.66:0/1371700619 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x562f6a791080 con 0x562f6b666c00 2024-03-30T00:51:30.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9da4c00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562f6b6e2c00 0x562f6a7e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9da4c00640 1 -- 172.21.15.66:0/1371700619 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x562f6a798a80 con 0x562f6b666c00 2024-03-30T00:51:30.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9dae3208c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-30T00:51:30.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9dae3208c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-03-30T00:51:30.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9dae3208c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-30T00:51:30.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x562f6b667800 0x562f6a7e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b667c00 con 0x562f6b667800 2024-03-30T00:51:30.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9dac400640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562f6b6e2c00 0x562f6a7e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9daba00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x562f6b667800 0x562f6a7e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9dac400640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562f6b6e2c00 0x562f6a7e7700 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x562f6b680690 tx=0x562f6b684c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9daba00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x562f6b667800 0x562f6a7e6100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== osd_op_reply(1 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6b667800 2024-03-30T00:51:30.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9da6000640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x562f6b6e3000 0x562f6ab40000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e3400 con 0x562f6b6e3000 2024-03-30T00:51:30.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9dace00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x562f6b6e3000 0x562f6ab40000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.015+0000 7f9dace00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x562f6b6e3000 0x562f6ab40000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.019+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== osd_op_reply(2 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b6e3000 2024-03-30T00:51:30.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.019+0000 7f9da6a00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x562f6b6e3c00 0x562f6ab40580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.019+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab46000 con 0x562f6b6e3c00 2024-03-30T00:51:30.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.019+0000 7f9dac400640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x562f6b6e3c00 0x562f6ab40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.019+0000 7f9dac400640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x562f6b6e3c00 0x562f6ab40580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.019+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(3 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65afc0 con 0x562f6b6e3c00 2024-03-30T00:51:30.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.019+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e3800 con 0x562f6b667800 2024-03-30T00:51:30.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.019+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.0 v2:172.21.15.66:6800/27456 2 ==== osd_op_reply(4 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6b667800 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.019+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab46400 con 0x562f6b667800 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.019+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.0 v2:172.21.15.66:6800/27456 3 ==== osd_op_reply(5 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6b667800 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.019+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab46800 con 0x562f6b6e3000 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.5 v2:172.21.15.156:6808/27644 2 ==== osd_op_reply(6 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b6e3000 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9da6a00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x562f6ab47400 0x562f6ab40b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab47800 con 0x562f6ab47400 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9daba00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x562f6ab47400 0x562f6ab40b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9daba00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x562f6ab47400 0x562f6ab40b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(7 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9da6000640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x562f6ab46c00 0x562f6ab41600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab47000 con 0x562f6ab46c00 2024-03-30T00:51:30.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9dace00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x562f6ab46c00 0x562f6ab41600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9dace00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x562f6ab46c00 0x562f6ab41600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.3 v2:172.21.15.66:6808/27471 1 ==== osd_op_reply(8 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6ab46c00 2024-03-30T00:51:30.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab47c00 con 0x562f6b667800 2024-03-30T00:51:30.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.0 v2:172.21.15.66:6800/27456 4 ==== osd_op_reply(9 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6b667800 2024-03-30T00:51:30.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b666800 con 0x562f6b667800 2024-03-30T00:51:30.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.0 v2:172.21.15.66:6800/27456 5 ==== osd_op_reply(10 [pgnls start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6b667800 2024-03-30T00:51:30.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab46800 con 0x562f6b6e3c00 2024-03-30T00:51:30.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 <== osd.6 v2:172.21.15.156:6816/27642 2 ==== osd_op_reply(11 [pgnls start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65afc0 con 0x562f6b6e3c00 2024-03-30T00:51:30.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.023+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b666400 con 0x562f6b667800 2024-03-30T00:51:30.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.0 v2:172.21.15.66:6800/27456 6 ==== osd_op_reply(12 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6b667800 2024-03-30T00:51:30.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab47000 con 0x562f6b6e3000 2024-03-30T00:51:30.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.5 v2:172.21.15.156:6808/27644 3 ==== osd_op_reply(13 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b6e3000 2024-03-30T00:51:30.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9da6000640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x562f6b667c00 0x562f6ab41b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b52b400 con 0x562f6b667c00 2024-03-30T00:51:30.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9dac400640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x562f6b667c00 0x562f6ab41b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9dac400640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x562f6b667c00 0x562f6ab41b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 <== osd.2 v2:172.21.15.66:6816/27464 1 ==== osd_op_reply(14 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65afc0 con 0x562f6b667c00 2024-03-30T00:51:30.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e3400 con 0x562f6ab47400 2024-03-30T00:51:30.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 2 ==== osd_op_reply(15 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9da6000640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562f6ab46000 0x562f6ab42100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab46800 con 0x562f6ab46000 2024-03-30T00:51:30.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9daba00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562f6ab46000 0x562f6ab42100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9daba00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562f6ab46000 0x562f6ab42100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== osd_op_reply(16 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab46000 2024-03-30T00:51:30.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e3800 con 0x562f6ab46000 2024-03-30T00:51:30.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.7 v2:172.21.15.156:6824/27643 2 ==== osd_op_reply(17 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab46000 2024-03-30T00:51:30.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab46400 con 0x562f6ab46c00 2024-03-30T00:51:30.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.027+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.3 v2:172.21.15.66:6808/27471 2 ==== osd_op_reply(18 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6ab46c00 2024-03-30T00:51:30.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab47c00 con 0x562f6b667c00 2024-03-30T00:51:30.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 <== osd.2 v2:172.21.15.66:6816/27464 2 ==== osd_op_reply(19 [pgnls start_epoch 127 out=81b] v126'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562f6b65afc0 con 0x562f6b667c00 2024-03-30T00:51:30.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab46800 con 0x562f6b667c00 2024-03-30T00:51:30.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 <== osd.2 v2:172.21.15.66:6816/27464 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562f6b65afc0 con 0x562f6b667c00 2024-03-30T00:51:30.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dae3208c0 10 datascan.operator(): handling object 1099511627778.0 2024-03-30T00:51:30.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab47800 con 0x562f6b667c00 2024-03-30T00:51:30.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 <== osd.2 v2:172.21.15.66:6816/27464 4 ==== osd_op_reply(21 10000000002.00000000 [call] v127'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562f6b65afc0 con 0x562f6b667c00 2024-03-30T00:51:30.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab68000 con 0x562f6ab46c00 2024-03-30T00:51:30.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.3 v2:172.21.15.66:6808/27471 3 ==== osd_op_reply(22 [pgnls start_epoch 127 out=49b] v125'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6ab46c00 2024-03-30T00:51:30.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e3400 con 0x562f6ab46c00 2024-03-30T00:51:30.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.3 v2:172.21.15.66:6808/27471 4 ==== osd_op_reply(23 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6ab46c00 2024-03-30T00:51:30.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9da6000640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x562f6b52b400 0x562f6ab42680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e2800 con 0x562f6b52b400 2024-03-30T00:51:30.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dace00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x562f6b52b400 0x562f6ab42680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dace00640 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x562f6b52b400 0x562f6ab42680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.4 v2:172.21.15.156:6800/27641 1 ==== osd_op_reply(24 [pgnls start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b52b400 2024-03-30T00:51:30.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.031+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e3800 con 0x562f6ab46c00 2024-03-30T00:51:30.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.3 v2:172.21.15.66:6808/27471 5 ==== osd_op_reply(25 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6ab46c00 2024-03-30T00:51:30.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e3400 con 0x562f6b667800 2024-03-30T00:51:30.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.0 v2:172.21.15.66:6800/27456 7 ==== osd_op_reply(26 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6b667800 2024-03-30T00:51:30.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b666800 con 0x562f6ab46000 2024-03-30T00:51:30.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.7 v2:172.21.15.156:6824/27643 3 ==== osd_op_reply(27 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab46000 2024-03-30T00:51:30.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e2800 con 0x562f6b667c00 2024-03-30T00:51:30.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 <== osd.2 v2:172.21.15.66:6816/27464 5 ==== osd_op_reply(28 [pgnls start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65afc0 con 0x562f6b667c00 2024-03-30T00:51:30.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b666400 con 0x562f6ab46c00 2024-03-30T00:51:30.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.3 v2:172.21.15.66:6808/27471 6 ==== osd_op_reply(29 [pgnls start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6ab46c00 2024-03-30T00:51:30.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e3800 con 0x562f6ab47400 2024-03-30T00:51:30.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 3 ==== osd_op_reply(30 [pgnls start_epoch 127 out=81b] v125'33 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab68400 con 0x562f6ab47400 2024-03-30T00:51:30.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 4 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9dae3208c0 10 datascan.operator(): handling object 1099511628279.0 2024-03-30T00:51:30.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab68800 con 0x562f6ab47400 2024-03-30T00:51:30.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 5 ==== osd_op_reply(32 100000001f7.00000000 [call] v127'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab68c00 con 0x562f6b667800 2024-03-30T00:51:30.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.0 v2:172.21.15.66:6800/27456 8 ==== osd_op_reply(33 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6b667800 2024-03-30T00:51:30.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.035+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab68000 con 0x562f6ab46c00 2024-03-30T00:51:30.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.3 v2:172.21.15.66:6808/27471 7 ==== osd_op_reply(34 [pgnls start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6ab46c00 2024-03-30T00:51:30.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6a000 con 0x562f6b6e3000 2024-03-30T00:51:30.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.5 v2:172.21.15.156:6808/27644 4 ==== osd_op_reply(35 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b6e3000 2024-03-30T00:51:30.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab47000 con 0x562f6ab46c00 2024-03-30T00:51:30.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.3 v2:172.21.15.66:6808/27471 8 ==== osd_op_reply(36 [pgnls start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6ab46c00 2024-03-30T00:51:30.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6a400 con 0x562f6ab46c00 2024-03-30T00:51:30.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.3 v2:172.21.15.66:6808/27471 9 ==== osd_op_reply(37 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6ab46c00 2024-03-30T00:51:30.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab46400 con 0x562f6b6e3000 2024-03-30T00:51:30.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.5 v2:172.21.15.156:6808/27644 5 ==== osd_op_reply(38 [pgnls start_epoch 127 out=49b] v125'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b6e3000 2024-03-30T00:51:30.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6a800 con 0x562f6ab47400 2024-03-30T00:51:30.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 6 ==== osd_op_reply(39 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6a800 con 0x562f6b52b400 2024-03-30T00:51:30.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.4 v2:172.21.15.156:6800/27641 2 ==== osd_op_reply(40 [pgnls start_epoch 127 out=81b] v125'15 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b52b400 2024-03-30T00:51:30.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab69000 con 0x562f6b52b400 2024-03-30T00:51:30.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.4 v2:172.21.15.156:6800/27641 3 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b52b400 2024-03-30T00:51:30.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9dae3208c0 10 datascan.operator(): handling object 1099511627780.1 2024-03-30T00:51:30.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.039+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab69400 con 0x562f6ab47400 2024-03-30T00:51:30.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 7 ==== osd_op_reply(42 10000000004.00000000 [call] v127'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab69800 con 0x562f6ab46c00 2024-03-30T00:51:30.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.3 v2:172.21.15.66:6808/27471 10 ==== osd_op_reply(43 [pgnls start_epoch 127 out=49b] v125'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6ab46c00 2024-03-30T00:51:30.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab68400 con 0x562f6ab46000 2024-03-30T00:51:30.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.7 v2:172.21.15.156:6824/27643 4 ==== osd_op_reply(44 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab46000 2024-03-30T00:51:30.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6ac00 con 0x562f6b52b400 2024-03-30T00:51:30.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.4 v2:172.21.15.156:6800/27641 4 ==== osd_op_reply(45 [pgnls start_epoch 127 out=81b] v126'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b52b400 2024-03-30T00:51:30.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab69c00 con 0x562f6b52b400 2024-03-30T00:51:30.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.4 v2:172.21.15.156:6800/27641 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b52b400 2024-03-30T00:51:30.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9dae3208c0 10 datascan.operator(): handling object 1099511628280.0 2024-03-30T00:51:30.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6a000 con 0x562f6b52b400 2024-03-30T00:51:30.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.4 v2:172.21.15.156:6800/27641 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v127'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b52b400 2024-03-30T00:51:30.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6a400 con 0x562f6b667800 2024-03-30T00:51:30.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.0 v2:172.21.15.66:6800/27456 9 ==== osd_op_reply(48 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6b667800 2024-03-30T00:51:30.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6b000 con 0x562f6ab47400 2024-03-30T00:51:30.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 8 ==== osd_op_reply(49 [pgnls start_epoch 127 out=81b] v127'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.043+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6a800 con 0x562f6ab47400 2024-03-30T00:51:30.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 9 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9dae3208c0 10 datascan.operator(): handling object 1099511627780.0 2024-03-30T00:51:30.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6b800 con 0x562f6ab47400 2024-03-30T00:51:30.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 10 ==== osd_op_reply(51 10000000004.00000000 [call] v127'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6bc00 con 0x562f6b6e3c00 2024-03-30T00:51:30.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 <== osd.6 v2:172.21.15.156:6816/27642 3 ==== osd_op_reply(52 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65afc0 con 0x562f6b6e3c00 2024-03-30T00:51:30.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6b400 con 0x562f6b6e3000 2024-03-30T00:51:30.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.5 v2:172.21.15.156:6808/27644 6 ==== osd_op_reply(53 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b6e3000 2024-03-30T00:51:30.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab68800 con 0x562f6b52b400 2024-03-30T00:51:30.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.4 v2:172.21.15.156:6800/27641 7 ==== osd_op_reply(54 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b52b400 2024-03-30T00:51:30.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6bc00 con 0x562f6b52b400 2024-03-30T00:51:30.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.4 v2:172.21.15.156:6800/27641 8 ==== osd_op_reply(55 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b52b400 2024-03-30T00:51:30.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab68c00 con 0x562f6b6e3c00 2024-03-30T00:51:30.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 <== osd.6 v2:172.21.15.156:6816/27642 4 ==== osd_op_reply(56 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65afc0 con 0x562f6b6e3c00 2024-03-30T00:51:30.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e2800 con 0x562f6b667800 2024-03-30T00:51:30.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.0 v2:172.21.15.66:6800/27456 10 ==== osd_op_reply(57 [pgnls start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6b667800 2024-03-30T00:51:30.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e3400 con 0x562f6ab46c00 2024-03-30T00:51:30.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.3 v2:172.21.15.66:6808/27471 11 ==== osd_op_reply(58 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6ab46c00 2024-03-30T00:51:30.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.047+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab47c00 con 0x562f6ab47400 2024-03-30T00:51:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 11 ==== osd_op_reply(59 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b6e3800 con 0x562f6b667c00 2024-03-30T00:51:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 <== osd.2 v2:172.21.15.66:6816/27464 6 ==== osd_op_reply(60 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65afc0 con 0x562f6b667c00 2024-03-30T00:51:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab46800 con 0x562f6ab46000 2024-03-30T00:51:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.7 v2:172.21.15.156:6824/27643 5 ==== osd_op_reply(61 [pgnls start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab46000 2024-03-30T00:51:30.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6b666800 con 0x562f6ab47400 2024-03-30T00:51:30.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 12 ==== osd_op_reply(62 [pgnls start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab47800 con 0x562f6b6e3000 2024-03-30T00:51:30.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.5 v2:172.21.15.156:6808/27644 7 ==== osd_op_reply(63 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b6e3000 2024-03-30T00:51:30.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab69400 con 0x562f6ab46000 2024-03-30T00:51:30.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.7 v2:172.21.15.156:6824/27643 6 ==== osd_op_reply(64 [pgnls start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab46000 2024-03-30T00:51:30.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6a000 con 0x562f6ab47400 2024-03-30T00:51:30.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 13 ==== osd_op_reply(65 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab68400 con 0x562f6b52b400 2024-03-30T00:51:30.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.4 v2:172.21.15.156:6800/27641 9 ==== osd_op_reply(66 [pgnls start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b52b400 2024-03-30T00:51:30.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6a400 con 0x562f6b667c00 2024-03-30T00:51:30.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 <== osd.2 v2:172.21.15.66:6816/27464 7 ==== osd_op_reply(67 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65afc0 con 0x562f6b667c00 2024-03-30T00:51:30.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.051+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab69000 con 0x562f6b6e3000 2024-03-30T00:51:30.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.5 v2:172.21.15.156:6808/27644 8 ==== osd_op_reply(68 [pgnls start_epoch 127 out=49b] v125'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b6e3000 2024-03-30T00:51:30.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6a800 con 0x562f6b6e3000 2024-03-30T00:51:30.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.5 v2:172.21.15.156:6808/27644 9 ==== osd_op_reply(69 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b6e3000 2024-03-30T00:51:30.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab69800 con 0x562f6ab46000 2024-03-30T00:51:30.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.7 v2:172.21.15.156:6824/27643 7 ==== osd_op_reply(70 [pgnls start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab46000 2024-03-30T00:51:30.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6ac00 con 0x562f6b6e3000 2024-03-30T00:51:30.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.5 v2:172.21.15.156:6808/27644 10 ==== osd_op_reply(71 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b6e3000 2024-03-30T00:51:30.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab69c00 con 0x562f6ab47400 2024-03-30T00:51:30.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.1 v2:172.21.15.66:6821/27458 14 ==== osd_op_reply(72 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab47400 2024-03-30T00:51:30.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9da6a00640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab6b000 con 0x562f6ab46000 2024-03-30T00:51:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9daba00640 1 -- 172.21.15.66:0/1371700619 <== osd.7 v2:172.21.15.156:6824/27643 8 ==== osd_op_reply(73 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ab40 con 0x562f6ab46000 2024-03-30T00:51:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9da6000640 1 -- 172.21.15.66:0/1371700619 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562f6ab68800 con 0x562f6b52b400 2024-03-30T00:51:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dace00640 1 -- 172.21.15.66:0/1371700619 <== osd.4 v2:172.21.15.156:6800/27641 10 ==== osd_op_reply(74 [pgnls start_epoch 127 out=49b] v125'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f6b65ad80 con 0x562f6b52b400 2024-03-30T00:51:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x562f6b667800 msgr2=0x562f6a7e6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x562f6b667800 0x562f6a7e6100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x562f6ab47400 msgr2=0x562f6ab40b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x562f6ab47400 0x562f6ab40b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x562f6b667c00 msgr2=0x562f6ab41b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x562f6b667c00 0x562f6ab41b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x562f6ab46c00 msgr2=0x562f6ab41600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x562f6ab46c00 0x562f6ab41600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x562f6b52b400 msgr2=0x562f6ab42680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x562f6b52b400 0x562f6ab42680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 reap_dead start 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x562f6b6e3000 msgr2=0x562f6ab40000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x562f6b6e3000 0x562f6ab40000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x562f6b6e3c00 msgr2=0x562f6ab40580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x562f6b6e3c00 0x562f6ab40580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562f6ab46000 msgr2=0x562f6ab42100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562f6ab46000 0x562f6ab42100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 reap_dead start 2024-03-30T00:51:30.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dac400640 1 -- 172.21.15.66:0/1371700619 reap_dead start 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562f6b6e2c00 msgr2=0x562f6a7e7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562f6b6e2c00 0x562f6a7e7700 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x562f6b680690 tx=0x562f6b684c00 comp rx=0 tx=0).stop 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b666c00 msgr2=0x562f6a7e7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b666c00 0x562f6a7e7180 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x562f6b5be9f0 tx=0x562f6b4e20c0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 shutdown_connections 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562f6b6e2c00 0x562f6a7e7700 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1371700619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6b666c00 0x562f6a7e7180 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 >> 172.21.15.66:0/1371700619 conn(0x562f6a758c00 msgr2=0x562f6a785800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 shutdown_connections 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.055+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1371700619 wait complete. 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.059+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1316138910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6a759000 msgr2=0x562f6a7e5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.059+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1316138910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6a759000 0x562f6a7e5080 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x562f6b5bf350 tx=0x562f6b664000 comp rx=0 tx=0).stop 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.059+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1316138910 shutdown_connections 2024-03-30T00:51:30.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.059+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1316138910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562f6a759000 0x562f6a7e5080 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.059+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1316138910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f6a759400 0x562f6a7e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.059+0000 7f9dae3208c0 1 --2- 172.21.15.66:0/1316138910 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562f6a759800 0x562f6a7e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.059+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1316138910 >> 172.21.15.66:0/1316138910 conn(0x562f6a759c00 msgr2=0x562f6a785000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.059+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1316138910 shutdown_connections 2024-03-30T00:51:30.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.059+0000 7f9dae3208c0 1 -- 172.21.15.66:0/1316138910 wait complete. 2024-03-30T00:51:30.070 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.144150, result: 2024-03-30T00:51:30.071 DEBUG:teuthology.orchestra.run.smithi066:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-03-30T00:51:30.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f477c4228c0 1 -- start start 2024-03-30T00:51:30.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f477c4228c0 1 Processor -- start 2024-03-30T00:51:30.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f477c4228c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9b5e5800 0x562a9b670580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f477c4228c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9b5e5400 0x562a9b670b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f477c4228c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9b5e5000 0x562a9b671080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f477c4228c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9b6b2780 con 0x562a9b5e5400 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f477c4228c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9b6b2a00 con 0x562a9b5e5000 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f477c4228c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a9b6b2640 con 0x562a9b5e5800 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f4779a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9b5e5400 0x562a9b670b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f4779a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9b5e5400 0x562a9b670b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40306/0 (socket says 172.21.15.66:40306) 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f4779a00640 1 -- 172.21.15.66:0/2342761661 learned_addr learned my addr 172.21.15.66:0/2342761661 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f477ae00640 1 --2- 172.21.15.66:0/2342761661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9b5e5000 0x562a9b671080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f477a400640 1 --2- 172.21.15.66:0/2342761661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9b5e5800 0x562a9b670580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f4779a00640 1 -- 172.21.15.66:0/2342761661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9b5e5800 msgr2=0x562a9b670580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f4779a00640 1 --2- 172.21.15.66:0/2342761661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9b5e5800 0x562a9b670580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f4779a00640 1 -- 172.21.15.66:0/2342761661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9b5e5000 msgr2=0x562a9b671080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f4779a00640 1 --2- 172.21.15.66:0/2342761661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9b5e5000 0x562a9b671080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.151+0000 7f4779a00640 1 -- 172.21.15.66:0/2342761661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a9b6189c0 con 0x562a9b5e5400 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f4779a00640 1 --2- 172.21.15.66:0/2342761661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9b5e5400 0x562a9b670b00 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x562a9c453320 tx=0x562a9c4f43c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4404e04ca83147fb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f4777200640 1 -- 172.21.15.66:0/2342761661 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a9b639600 con 0x562a9b5e5400 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f4777200640 1 -- 172.21.15.66:0/2342761661 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562a9b642480 con 0x562a9b5e5400 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f4777200640 1 -- 172.21.15.66:0/2342761661 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a9b6391e0 con 0x562a9b5e5400 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 1 -- 172.21.15.66:0/2342761661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a9b618b60 con 0x562a9b5e5400 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 1 -- 172.21.15.66:0/2342761661 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562a9b618340 con 0x562a9b5e5400 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 4 waiting for MDS map... 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f4777200640 1 -- 172.21.15.66:0/2342761661 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x562a9b624c40 con 0x562a9b5e5400 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f4777200640 1 -- 172.21.15.66:0/2342761661 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 102) v1 ==== 2573+0+0 (secure 0 0 0) 0x562a9b632f00 con 0x562a9b5e5400 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 4 Got MDS map 102 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 4 datascan.main: Using metadata pool output 2024-03-30T00:51:30.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 4 datascan.main: connecting to RADOS... 2024-03-30T00:51:30.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 1 Processor -- start 2024-03-30T00:51:30.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 1 -- start start 2024-03-30T00:51:30.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 0x562a9b671600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9c3c1000 0x562a9b671b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9c3c1400 0x562a9b672100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9b62be00 con 0x562a9c3c0c00 2024-03-30T00:51:30.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9b6b2a00 con 0x562a9c3c1400 2024-03-30T00:51:30.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477c4228c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a9b6b2500 con 0x562a9c3c1000 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 0x562a9b671600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 0x562a9b671600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40308/0 (socket says 172.21.15.66:40308) 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477ae00640 1 -- 172.21.15.66:0/852205170 learned_addr learned my addr 172.21.15.66:0/852205170 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477a400640 1 --2- 172.21.15.66:0/852205170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9c3c1000 0x562a9b671b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f4779a00640 1 --2- 172.21.15.66:0/852205170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9c3c1400 0x562a9b672100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477ae00640 1 -- 172.21.15.66:0/852205170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9c3c1000 msgr2=0x562a9b671b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477ae00640 1 --2- 172.21.15.66:0/852205170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9c3c1000 0x562a9b671b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477ae00640 1 -- 172.21.15.66:0/852205170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9c3c1400 msgr2=0x562a9b672100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477ae00640 1 --2- 172.21.15.66:0/852205170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9c3c1400 0x562a9b672100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477ae00640 1 -- 172.21.15.66:0/852205170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a9b618000 con 0x562a9c3c0c00 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f477ae00640 1 --2- 172.21.15.66:0/852205170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 0x562a9b671600 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x562a9c4529f0 tx=0x562a9c4f4120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ed2fab2bf6089c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f4774a00640 1 -- 172.21.15.66:0/852205170 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a9b639080 con 0x562a9c3c0c00 2024-03-30T00:51:30.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f4774a00640 1 -- 172.21.15.66:0/852205170 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562a9c3be900 con 0x562a9c3c0c00 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.155+0000 7f4774a00640 1 -- 172.21.15.66:0/852205170 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a9b639600 con 0x562a9c3c0c00 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 -- 172.21.15.66:0/852205170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 msgr2=0x562a9b671600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 --2- 172.21.15.66:0/852205170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 0x562a9b671600 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x562a9c4529f0 tx=0x562a9c4f4120 comp rx=0 tx=0).stop 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 -- 172.21.15.66:0/852205170 shutdown_connections 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 --2- 172.21.15.66:0/852205170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9c3c1400 0x562a9b672100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 --2- 172.21.15.66:0/852205170 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9c3c1000 0x562a9b671b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 --2- 172.21.15.66:0/852205170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 0x562a9b671600 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 -- 172.21.15.66:0/852205170 >> 172.21.15.66:0/852205170 conn(0x562a9b5e4c00 msgr2=0x562a9b611800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 -- 172.21.15.66:0/852205170 shutdown_connections 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 -- 172.21.15.66:0/852205170 wait complete. 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 Processor -- start 2024-03-30T00:51:30.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 -- start start 2024-03-30T00:51:30.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 0x562a9b673180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9c3c1000 0x562a9b672c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9c3c1400 0x562a9b672680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9b6b23c0 con 0x562a9c3c0c00 2024-03-30T00:51:30.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9b6b2280 con 0x562a9c3c1400 2024-03-30T00:51:30.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a9b6b2140 con 0x562a9c3c1000 2024-03-30T00:51:30.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 0x562a9b673180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 0x562a9b673180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40316/0 (socket says 172.21.15.66:40316) 2024-03-30T00:51:30.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 learned_addr learned my addr 172.21.15.66:0/2002882004 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:30.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f4779a00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9c3c1400 0x562a9b672680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477a400640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9c3c1000 0x562a9b672c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9c3c1000 msgr2=0x562a9b672c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f4779a00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9c3c1000 0x562a9b672c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 msgr2=0x562a9b673180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f4779a00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 0x562a9b673180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a9b6181a0 con 0x562a9c3c1400 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477ae00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9c3c0c00 0x562a9b673180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f4779a00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9c3c1400 0x562a9b672680 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x562a9c55f380 tx=0x562a9c36e180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f4772c00640 1 -- 172.21.15.66:0/2002882004 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a9b638f20 con 0x562a9c3c1400 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f4772c00640 1 -- 172.21.15.66:0/2002882004 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562a9c3bea80 con 0x562a9c3c1400 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562a9b618000 con 0x562a9c3c1400 2024-03-30T00:51:30.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f4772c00640 1 -- 172.21.15.66:0/2002882004 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a9b639080 con 0x562a9c3c1400 2024-03-30T00:51:30.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a9b6189c0 con 0x562a9c3c1400 2024-03-30T00:51:30.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.159+0000 7f477c4228c0 4 datascan.init: resolving metadata pool 17 2024-03-30T00:51:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f4772c00640 1 -- 172.21.15.66:0/2002882004 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x562a9b61c840 con 0x562a9c3c1400 2024-03-30T00:51:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f4772c00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562a9b8d3000 0x562a9b673700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f4772c00640 1 -- 172.21.15.66:0/2002882004 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x562a9b624a80 con 0x562a9c3c1400 2024-03-30T00:51:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477c4228c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-30T00:51:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477c4228c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-03-30T00:51:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477c4228c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-30T00:51:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x562a9c3c1800 0x562a9b672100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477ae00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562a9b8d3000 0x562a9b673700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9c3c1c00 con 0x562a9c3c1800 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477a400640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x562a9c3c1800 0x562a9b672100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477ae00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562a9b8d3000 0x562a9b673700 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x562a9c55e810 tx=0x562a9c4f40c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477a400640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x562a9c3c1800 0x562a9b672100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv36 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x562a9b8d3400 0x562a9b671b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b8d3800 con 0x562a9b8d3400 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477ae00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x562a9b8d3400 0x562a9b671b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.163+0000 7f477ae00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x562a9b8d3400 0x562a9b671b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b8d3400 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x562a9b8d3c00 0x562a9b671600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c4000 con 0x562a9b8d3c00 2024-03-30T00:51:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f477a400640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x562a9b8d3c00 0x562a9b671600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f477a400640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x562a9b8d3c00 0x562a9b671600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4774000640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x562a9c3c1c00 0x562a9b9d2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c4000 con 0x562a9c3c1c00 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4779a00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x562a9c3c1c00 0x562a9b9d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4779a00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x562a9c3c1c00 0x562a9b9d2000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e0000 con 0x562a9b8d3400 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.6 v2:172.21.15.156:6816/27642 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b8d3400 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e0400 con 0x562a9b8d3c00 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e1000 con 0x562a9b8d3c00 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e0800 con 0x562a9c3c1c00 2024-03-30T00:51:30.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4774a00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x562a9b9e1400 0x562a9b9d2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.167+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e1800 con 0x562a9b9e1400 2024-03-30T00:51:30.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f477ae00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x562a9b9e1400 0x562a9b9d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f477ae00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x562a9b9e1400 0x562a9b9d2580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e0c00 con 0x562a9c3c1800 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e1c00 con 0x562a9b8d3c00 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e0400 con 0x562a9b8d3c00 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4000 con 0x562a9b8d3400 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.6 v2:172.21.15.156:6816/27642 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b8d3400 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e1000 con 0x562a9b8d3c00 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4400 con 0x562a9c3c1c00 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f4774000640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x562a9b9e0c00 0x562a9b9d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b8d2c00 con 0x562a9b9e0c00 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f477a400640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x562a9b9e0c00 0x562a9b9d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f477a400640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x562a9b9e0c00 0x562a9b9d2b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.2 v2:172.21.15.66:6816/27464 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b9e0c00 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.171+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4800 con 0x562a9b9e1400 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4774000640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562a9b8d2800 0x562a9b9d3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b8d2400 con 0x562a9b8d2800 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4779a00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562a9b8d2800 0x562a9b9d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4779a00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562a9b8d2800 0x562a9b9d3080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9b8d2800 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4c00 con 0x562a9b8d2800 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.7 v2:172.21.15.156:6824/27643 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9b8d2800 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9c3c5400 con 0x562a9c3c1800 2024-03-30T00:51:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e1c00 con 0x562a9b9e0c00 2024-03-30T00:51:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.2 v2:172.21.15.66:6816/27464 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 127 out=81b] v127'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b9e0c00 2024-03-30T00:51:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c4400 con 0x562a9b8d3c00 2024-03-30T00:51:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e0400 con 0x562a9c3c1c00 2024-03-30T00:51:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c4000 con 0x562a9b8d3400 2024-03-30T00:51:30.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.6 v2:172.21.15.156:6816/27642 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b8d3400 2024-03-30T00:51:30.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e1000 con 0x562a9b8d3c00 2024-03-30T00:51:30.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.175+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4400 con 0x562a9b8d3c00 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b8d2c00 con 0x562a9c3c1c00 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4000 con 0x562a9b9e1400 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b8d3800 con 0x562a9c3c1800 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e5000 con 0x562a9b8d3c00 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4400 con 0x562a9b8d3c00 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e0800 con 0x562a9b8d3400 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.6 v2:172.21.15.156:6816/27642 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b8d3400 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e5400 con 0x562a9b8d3c00 2024-03-30T00:51:30.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e0000 con 0x562a9c3c1c00 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e1c00 con 0x562a9b9e0c00 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.2 v2:172.21.15.66:6816/27464 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b9e0c00 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e1800 con 0x562a9b9e1400 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.179+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e1000 con 0x562a9b8d2800 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.7 v2:172.21.15.156:6824/27643 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9b8d2800 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e5800 con 0x562a9b8d2800 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.7 v2:172.21.15.156:6824/27643 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9b8d2800 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c4400 con 0x562a9c3c1800 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e5c00 con 0x562a9b9e0c00 2024-03-30T00:51:30.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.2 v2:172.21.15.66:6816/27464 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 127 out=49b] v127'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b9e0c00 2024-03-30T00:51:30.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9c3c5400 con 0x562a9c3c1800 2024-03-30T00:51:30.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 127 out=49b] v125'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e1c00 con 0x562a9c3c1800 2024-03-30T00:51:30.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4774a00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x562a9b9e5000 0x562a9b9d3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4400 con 0x562a9b9e5000 2024-03-30T00:51:30.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f477ae00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x562a9b9e5000 0x562a9b9d3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f477ae00640 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x562a9b9e5000 0x562a9b9d3600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.4 v2:172.21.15.156:6800/27641 1 ==== osd_op_reply(43 [pgnls-filter start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e5000 2024-03-30T00:51:30.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e0400 con 0x562a9c3c1800 2024-03-30T00:51:30.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e5400 con 0x562a9b8d3c00 2024-03-30T00:51:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 13 ==== osd_op_reply(45 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.183+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e0000 con 0x562a9b8d2800 2024-03-30T00:51:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.7 v2:172.21.15.156:6824/27643 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9b8d2800 2024-03-30T00:51:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4c00 con 0x562a9b9e0c00 2024-03-30T00:51:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.2 v2:172.21.15.66:6816/27464 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b9e0c00 2024-03-30T00:51:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e1000 con 0x562a9c3c1800 2024-03-30T00:51:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e5800 con 0x562a9b9e1400 2024-03-30T00:51:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 127 out=81b] v127'34 uv34 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 10 datascan.operator(): handling object 100000001f7.0 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c4800 con 0x562a9b9e1400 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 6 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c4c00 con 0x562a9b8d3c00 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 14 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c5000 con 0x562a9c3c1800 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 10 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+524 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-03-30T00:51:30.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-03-30T00:51:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 20 datascan.get_frag_of: frag is * 2024-03-30T00:51:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c5400 con 0x562a9b8d3c00 2024-03-30T00:51:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 15 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 10 datascan.find_or_create_dirfrag: read_version = 8 2024-03-30T00:51:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-03-30T00:51:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c5800 con 0x562a9b8d3c00 2024-03-30T00:51:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 16 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-03-30T00:51:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 20 datascan.read_dentry: bravo_head not found in result 2024-03-30T00:51:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-03-30T00:51:30.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.187+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c5c00 con 0x562a9b8d3c00 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 17 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v127'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f477c4228c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e5c00 con 0x562a9b8d3c00 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4c00 con 0x562a9c3c1800 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b8d3800 con 0x562a9c3c1c00 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4800 con 0x562a9c3c1800 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b8d2400 con 0x562a9c3c1800 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4000 con 0x562a9c3c1c00 2024-03-30T00:51:30.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 127 out=49b] v125'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b8d2c00 con 0x562a9b9e1400 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4400 con 0x562a9b9e5000 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.4 v2:172.21.15.156:6800/27641 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 127 out=49b] v125'15 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e5000 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c4400 con 0x562a9c3c1800 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 127 out=49b] v125'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.191+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e5c00 con 0x562a9b8d2800 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.7 v2:172.21.15.156:6824/27643 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9b8d2800 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fa000 con 0x562a9b9e5000 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.4 v2:172.21.15.156:6800/27641 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 127 out=49b] v127'24 uv24 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e5000 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4c00 con 0x562a9b8d3c00 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fa400 con 0x562a9b9e1400 2024-03-30T00:51:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 127 out=49b] v127'15 uv15 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4800 con 0x562a9b8d3400 2024-03-30T00:51:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.6 v2:172.21.15.156:6816/27642 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b8d3400 2024-03-30T00:51:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fa800 con 0x562a9c3c1c00 2024-03-30T00:51:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c4c00 con 0x562a9b9e5000 2024-03-30T00:51:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.4 v2:172.21.15.156:6800/27641 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e5000 2024-03-30T00:51:30.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fac00 con 0x562a9b9e5000 2024-03-30T00:51:30.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.4 v2:172.21.15.156:6800/27641 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e5000 2024-03-30T00:51:30.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c5000 con 0x562a9b8d3400 2024-03-30T00:51:30.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.6 v2:172.21.15.156:6816/27642 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b8d3400 2024-03-30T00:51:30.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fb000 con 0x562a9b8d3c00 2024-03-30T00:51:30.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.0 v2:172.21.15.66:6800/27456 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b8d3c00 2024-03-30T00:51:30.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.195+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c5400 con 0x562a9c3c1800 2024-03-30T00:51:30.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.3 v2:172.21.15.66:6808/27471 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9c3c1800 2024-03-30T00:51:30.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fb400 con 0x562a9b9e1400 2024-03-30T00:51:30.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c5800 con 0x562a9b9e0c00 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.2 v2:172.21.15.66:6816/27464 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b9e0c00 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fb800 con 0x562a9b8d2800 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.7 v2:172.21.15.156:6824/27643 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9b8d2800 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9c5c00 con 0x562a9b9e1400 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fbc00 con 0x562a9c3c1c00 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e5800 con 0x562a9b8d2800 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.7 v2:172.21.15.156:6824/27643 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9b8d2800 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4000 con 0x562a9b9e1400 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4400 con 0x562a9b9e5000 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.4 v2:172.21.15.156:6800/27641 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e5000 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e5c00 con 0x562a9b9e0c00 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 <== osd.2 v2:172.21.15.66:6816/27464 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecb40 con 0x562a9b9e0c00 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e4800 con 0x562a9c3c1c00 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 127 out=49b] v125'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.199+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9e5800 con 0x562a9c3c1c00 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fa000 con 0x562a9b8d2800 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.7 v2:172.21.15.156:6824/27643 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9b8d2800 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fa800 con 0x562a9c3c1c00 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.5 v2:172.21.15.156:6808/27644 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9c3c1c00 2024-03-30T00:51:30.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fa400 con 0x562a9b9e1400 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.1 v2:172.21.15.66:6821/27458 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e1400 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f4774000640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b9fb800 con 0x562a9b8d2800 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f4779a00640 1 -- 172.21.15.66:0/2002882004 <== osd.7 v2:172.21.15.156:6824/27643 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecfc0 con 0x562a9b8d2800 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f4774a00640 1 -- 172.21.15.66:0/2002882004 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x562a9b8d2c00 con 0x562a9b9e5000 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477ae00640 1 -- 172.21.15.66:0/2002882004 <== osd.4 v2:172.21.15.156:6800/27641 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 127 out=49b] v125'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a9c4ecd80 con 0x562a9b9e5000 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x562a9b8d3c00 msgr2=0x562a9b671600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x562a9b8d3c00 0x562a9b671600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x562a9b9e1400 msgr2=0x562a9b9d2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x562a9b9e1400 0x562a9b9d2580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x562a9b9e0c00 msgr2=0x562a9b9d2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x562a9b9e0c00 0x562a9b9d2b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x562a9c3c1800 msgr2=0x562a9b672100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x562a9c3c1800 0x562a9b672100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x562a9b9e5000 msgr2=0x562a9b9d3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x562a9b9e5000 0x562a9b9d3600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x562a9c3c1c00 msgr2=0x562a9b9d2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x562a9c3c1c00 0x562a9b9d2000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x562a9b8d3400 msgr2=0x562a9b671b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 reap_dead start 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x562a9b8d3400 0x562a9b671b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562a9b8d2800 msgr2=0x562a9b9d3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x562a9b8d2800 0x562a9b9d3080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 reap_dead start 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477a400640 1 -- 172.21.15.66:0/2002882004 reap_dead start 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562a9b8d3000 msgr2=0x562a9b673700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562a9b8d3000 0x562a9b673700 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x562a9c55e810 tx=0x562a9c4f40c0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9c3c1400 msgr2=0x562a9b672680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9c3c1400 0x562a9b672680 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x562a9c55f380 tx=0x562a9c36e180 comp rx=0 tx=0).stop 2024-03-30T00:51:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 shutdown_connections 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9c3c1400 0x562a9b672680 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2002882004 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x562a9b8d3000 0x562a9b673700 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 >> 172.21.15.66:0/2002882004 conn(0x562a9b5e4c00 msgr2=0x562a9b611800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 shutdown_connections 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2002882004 wait complete. 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 -- 172.21.15.66:0/2342761661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9b5e5400 msgr2=0x562a9b670b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.203+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2342761661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9b5e5400 0x562a9b670b00 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x562a9c453320 tx=0x562a9c4f43c0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.207+0000 7f477c4228c0 1 -- 172.21.15.66:0/2342761661 shutdown_connections 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.207+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2342761661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9b5e5000 0x562a9b671080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.207+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2342761661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a9b5e5400 0x562a9b670b00 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.207+0000 7f477c4228c0 1 --2- 172.21.15.66:0/2342761661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562a9b5e5800 0x562a9b670580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.207+0000 7f477c4228c0 1 -- 172.21.15.66:0/2342761661 >> 172.21.15.66:0/2342761661 conn(0x562a9b5e5c00 msgr2=0x562a9b611000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.207+0000 7f477c4228c0 1 -- 172.21.15.66:0/2342761661 shutdown_connections 2024-03-30T00:51:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.207+0000 7f477c4228c0 1 -- 172.21.15.66:0/2342761661 wait complete. 2024-03-30T00:51:30.222 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.151696, result: 2024-03-30T00:51:30.223 DEBUG:teuthology.orchestra.run.smithi066:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-03-30T00:51:30.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83d8718c0 1 -- start start 2024-03-30T00:51:30.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83d8718c0 1 Processor -- start 2024-03-30T00:51:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83d8718c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558e90d7800 0x5558e9162580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83d8718c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e90d7400 0x5558e9162b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83d8718c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e90d7000 0x5558e9163080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83d8718c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e911de00 con 0x5558e90d7400 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83d8718c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e91a4a00 con 0x5558e90d7000 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83d8718c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558e91a4640 con 0x5558e90d7800 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e90d7400 0x5558e9162b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e90d7400 0x5558e9162b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40322/0 (socket says 172.21.15.66:40322) 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83b000640 1 -- 172.21.15.66:0/4284962883 learned_addr learned my addr 172.21.15.66:0/4284962883 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83ba00640 1 --2- 172.21.15.66:0/4284962883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558e90d7800 0x5558e9162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83c400640 1 --2- 172.21.15.66:0/4284962883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e90d7000 0x5558e9163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83b000640 1 -- 172.21.15.66:0/4284962883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558e90d7800 msgr2=0x5558e9162580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83b000640 1 --2- 172.21.15.66:0/4284962883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558e90d7800 0x5558e9162580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83b000640 1 -- 172.21.15.66:0/4284962883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e90d7000 msgr2=0x5558e9163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83b000640 1 --2- 172.21.15.66:0/4284962883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e90d7000 0x5558e9163080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83b000640 1 -- 172.21.15.66:0/4284962883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558e910a9c0 con 0x5558e90d7400 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83ba00640 1 --2- 172.21.15.66:0/4284962883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558e90d7800 0x5558e9162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.299+0000 7fd83c400640 1 --2- 172.21.15.66:0/4284962883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e90d7000 0x5558e9163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83b000640 1 --2- 172.21.15.66:0/4284962883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e90d7400 0x5558e9162b00 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x5558e9f3a9f0 tx=0x5558e9e59f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce5d1f274de26e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd838800640 1 -- 172.21.15.66:0/4284962883 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558e912b080 con 0x5558e90d7400 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd838800640 1 -- 172.21.15.66:0/4284962883 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5558e9f48180 con 0x5558e90d7400 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd838800640 1 -- 172.21.15.66:0/4284962883 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558e912b600 con 0x5558e90d7400 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 -- 172.21.15.66:0/4284962883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558e910a1a0 con 0x5558e90d7400 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 -- 172.21.15.66:0/4284962883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5558e910a9c0 con 0x5558e90d7400 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 4 waiting for MDS map... 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd838800640 1 -- 172.21.15.66:0/4284962883 <== mon.0 v2:172.21.15.66:3300/0 4 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x5558e9116c40 con 0x5558e90d7400 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd838800640 1 -- 172.21.15.66:0/4284962883 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 102) v1 ==== 2573+0+0 (secure 0 0 0) 0x5558e9124f00 con 0x5558e90d7400 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 4 Got MDS map 102 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 4 datascan.main: Using metadata pool output 2024-03-30T00:51:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 4 datascan.main: connecting to RADOS... 2024-03-30T00:51:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 Processor -- start 2024-03-30T00:51:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 -- start start 2024-03-30T00:51:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558e9ea5800 0x5558e9163600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5c00 0x5558e9163b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558ea050000 0x5558e9164100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e91a4500 con 0x5558e9ea5c00 2024-03-30T00:51:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e91a43c0 con 0x5558ea050000 2024-03-30T00:51:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558e91a4640 con 0x5558e9ea5800 2024-03-30T00:51:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5c00 0x5558e9163b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5c00 0x5558e9163b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40324/0 (socket says 172.21.15.66:40324) 2024-03-30T00:51:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83ba00640 1 -- 172.21.15.66:0/3679258632 learned_addr learned my addr 172.21.15.66:0/3679258632 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83c400640 1 --2- 172.21.15.66:0/3679258632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558e9ea5800 0x5558e9163600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83b000640 1 --2- 172.21.15.66:0/3679258632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558ea050000 0x5558e9164100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83ba00640 1 -- 172.21.15.66:0/3679258632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558e9ea5800 msgr2=0x5558e9163600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83ba00640 1 --2- 172.21.15.66:0/3679258632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558e9ea5800 0x5558e9163600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83ba00640 1 -- 172.21.15.66:0/3679258632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558ea050000 msgr2=0x5558e9164100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83ba00640 1 --2- 172.21.15.66:0/3679258632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558ea050000 0x5558e9164100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83ba00640 1 -- 172.21.15.66:0/3679258632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558e910ab60 con 0x5558e9ea5c00 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83ba00640 1 --2- 172.21.15.66:0/3679258632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5c00 0x5558e9163b80 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x5558e9fde900 tx=0x5558e9fe8f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69de4f935d2f732 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd836000640 1 -- 172.21.15.66:0/3679258632 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558e912b080 con 0x5558e9ea5c00 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd836000640 1 -- 172.21.15.66:0/3679258632 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5558e9134900 con 0x5558e9ea5c00 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd836000640 1 -- 172.21.15.66:0/3679258632 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558e912b600 con 0x5558e9ea5c00 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 -- 172.21.15.66:0/3679258632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5c00 msgr2=0x5558e9163b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/3679258632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5c00 0x5558e9163b80 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x5558e9fde900 tx=0x5558e9fe8f60 comp rx=0 tx=0).stop 2024-03-30T00:51:30.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 -- 172.21.15.66:0/3679258632 shutdown_connections 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/3679258632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558ea050000 0x5558e9164100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/3679258632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5c00 0x5558e9163b80 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/3679258632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558e9ea5800 0x5558e9163600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 -- 172.21.15.66:0/3679258632 >> 172.21.15.66:0/3679258632 conn(0x5558e90d6c00 msgr2=0x5558e9101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.303+0000 7fd83d8718c0 1 -- 172.21.15.66:0/3679258632 shutdown_connections 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 1 -- 172.21.15.66:0/3679258632 wait complete. 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 1 Processor -- start 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 1 -- start start 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e9ea5c00 0x5558e9165180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5800 0x5558e9164c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558ea051800 0x5558e9164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e91a4280 con 0x5558e9ea5800 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e91a4140 con 0x5558e9ea5c00 2024-03-30T00:51:30.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558e91a4000 con 0x5558ea051800 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5800 0x5558e9164c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5800 0x5558e9164c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40338/0 (socket says 172.21.15.66:40338) 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 learned_addr learned my addr 172.21.15.66:0/1727438477 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83c400640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e9ea5c00 0x5558e9165180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558ea051800 0x5558e9164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558ea051800 msgr2=0x5558e9164680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83ba00640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558ea051800 0x5558e9164680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e9ea5c00 msgr2=0x5558e9165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83ba00640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e9ea5c00 0x5558e9165180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558e910a340 con 0x5558e9ea5800 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558ea051800 0x5558e9164680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83ba00640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5800 0x5558e9164c00 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x5558e9fdf5c0 tx=0x5558e9fe8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd834200640 1 -- 172.21.15.66:0/1727438477 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558e912adc0 con 0x5558e9ea5800 2024-03-30T00:51:30.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd834200640 1 -- 172.21.15.66:0/1727438477 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5558e9f49500 con 0x5558e9ea5800 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5558e910a000 con 0x5558e9ea5800 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd834200640 1 -- 172.21.15.66:0/1727438477 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558e912b080 con 0x5558e9ea5800 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558e910aea0 con 0x5558e9ea5800 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.307+0000 7fd83d8718c0 4 datascan.init: resolving metadata pool 17 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd834200640 1 -- 172.21.15.66:0/1727438477 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x5558e910e840 con 0x5558e9ea5800 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd834200640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5558e93b6800 0x5558e9165700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd834200640 1 -- 172.21.15.66:0/1727438477 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x5558e9116a80 con 0x5558e9ea5800 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83d8718c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83d8718c0 4 datascan.main: resolving metadata pool 17 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83c400640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5558e93b6800 0x5558e9165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x5558ea051400 0x5558e9164100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea051000 con 0x5558ea051400 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x5558ea051400 0x5558e9164100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83c400640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5558e93b6800 0x5558e9165700 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x5558e91ea750 tx=0x5558e949e2a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x5558ea051400 0x5558e9164100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== osd_op_reply(1 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x5558ea051000 0x5558e94be000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e9ea5400 con 0x5558ea051000 2024-03-30T00:51:30.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83c400640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x5558ea051000 0x5558e94be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83c400640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x5558ea051000 0x5558e94be000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 1 ==== osd_op_reply(2 [pgnls start_epoch 127 out=177b] v126'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.311+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e9ea5400 con 0x5558ea051400 2024-03-30T00:51:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 2 ==== osd_op_reply(3 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5558e9ea5400 0x5558e94be580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e93b7800 con 0x5558e9ea5400 2024-03-30T00:51:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83ba00640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5558e9ea5400 0x5558e94be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83ba00640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5558e9ea5400 0x5558e94be580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== osd_op_reply(4 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e93b7800 con 0x5558e9ea5400 2024-03-30T00:51:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 2 ==== osd_op_reply(5 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e93b7800 con 0x5558ea051000 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 2 ==== osd_op_reply(6 [pgnls start_epoch 127 out=149b] v127'43 uv43 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e93b7800 con 0x5558ea051400 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 3 ==== osd_op_reply(7 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5558e93b7800 0x5558e94beb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e93b7c00 con 0x5558e93b7800 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5558e93b7800 0x5558e94beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5558e93b7800 0x5558e94beb00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== osd_op_reply(8 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e93b7800 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x5558e93b7c00 0x5558e94bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc000 con 0x5558e93b7c00 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.315+0000 7fd83c400640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x5558e93b7c00 0x5558e94bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83c400640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x5558e93b7c00 0x5558e94bf080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.4 v2:172.21.15.156:6800/27641 1 ==== osd_op_reply(9 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558e93b7c00 2024-03-30T00:51:30.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc000 con 0x5558ea051000 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 3 ==== osd_op_reply(10 [pgnls start_epoch 127 out=81b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc000 con 0x5558ea051400 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 4 ==== osd_op_reply(11 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc000 con 0x5558e9ea5400 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 3 ==== osd_op_reply(12 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83ba00640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5558e94cc000 0x5558e94c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cd400 con 0x5558e94cc000 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83ba00640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5558e94cc000 0x5558e94c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83ba00640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5558e94cc000 0x5558e94c0100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== osd_op_reply(13 [pgnls start_epoch 127 out=77b] v126'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e94cc000 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83ba00640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5558e94cd400 0x5558e94c0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cd800 con 0x5558e94cd400 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5558e94cd400 0x5558e94c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5558e94cd400 0x5558e94c0680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== osd_op_reply(14 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e94cd400 2024-03-30T00:51:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83b000640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x5558e94cd800 0x5558e94bf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.319+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e94cd800 2024-03-30T00:51:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83c400640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x5558e94cd800 0x5558e94bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83c400640 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x5558e94cd800 0x5558e94bf600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.2 v2:172.21.15.66:6816/27464 1 ==== osd_op_reply(15 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558e94cd800 2024-03-30T00:51:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e93b7800 2024-03-30T00:51:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.1 v2:172.21.15.66:6821/27458 2 ==== osd_op_reply(16 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e93b7800 2024-03-30T00:51:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e9ea5400 2024-03-30T00:51:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 4 ==== osd_op_reply(17 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558ea051000 2024-03-30T00:51:30.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 4 ==== osd_op_reply(18 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558ea051000 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 5 ==== osd_op_reply(19 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e94cd400 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.5 v2:172.21.15.156:6808/27644 2 ==== osd_op_reply(20 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e94cd400 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e94cc000 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.6 v2:172.21.15.156:6816/27642 2 ==== osd_op_reply(21 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e94cc000 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e94cd400 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.5 v2:172.21.15.156:6808/27644 3 ==== osd_op_reply(22 [pgnls start_epoch 127 out=97b] v126'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e94cd400 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558ea051400 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 5 ==== osd_op_reply(23 [pgnls start_epoch 127 out=74b] v124'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.323+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558ea051400 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 6 ==== osd_op_reply(24 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e93b7c00 2024-03-30T00:51:30.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.4 v2:172.21.15.156:6800/27641 2 ==== osd_op_reply(25 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558e93b7c00 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558ea051000 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 6 ==== osd_op_reply(26 [pgnls start_epoch 127 out=98b] v126'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e9ea5400 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 5 ==== osd_op_reply(27 [pgnls start_epoch 127 out=76b] v127'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e93b7c00 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.4 v2:172.21.15.156:6800/27641 3 ==== osd_op_reply(28 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558e93b7c00 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e9ea5400 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 6 ==== osd_op_reply(29 [pgnls start_epoch 127 out=105b] v127'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e94cc000 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.6 v2:172.21.15.156:6816/27642 3 ==== osd_op_reply(30 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e94cc000 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e93b7800 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.1 v2:172.21.15.66:6821/27458 3 ==== osd_op_reply(31 [pgnls start_epoch 127 out=73b] v127'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e93b7800 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94cc400 con 0x5558e94cc000 2024-03-30T00:51:30.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.6 v2:172.21.15.156:6816/27642 4 ==== osd_op_reply(32 [pgnls start_epoch 127 out=101b] v127'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e94cc000 2024-03-30T00:51:30.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-03-30T00:51:30.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050c00 con 0x5558ea051000 2024-03-30T00:51:30.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050800 con 0x5558ea051000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.327+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050000 con 0x5558ea051000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050c00 con 0x5558ea051000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-30T00:51:30.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050800 con 0x5558ea051000 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e93b7800 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.1 v2:172.21.15.66:6821/27458 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e93b7800 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e93b7400 con 0x5558ea051000 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 13 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-30T00:51:30.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0000 con 0x5558e94cd800 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.2 v2:172.21.15.66:6816/27464 2 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558e94cd800 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0400 con 0x5558e94cd400 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.5 v2:172.21.15.156:6808/27644 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e94cd400 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0800 con 0x5558ea051400 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-03-30T00:51:30.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0c00 con 0x5558ea051000 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e1000 con 0x5558e9ea5400 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e1400 con 0x5558e9ea5400 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 20 datascan.scan_links: injected ino detected 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-03-30T00:51:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.331+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e1800 con 0x5558e94cc000 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.6 v2:172.21.15.156:6816/27642 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e94cc000 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e1c00 con 0x5558ea051400 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 8 ==== osd_op_reply(48 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051000 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 15 ==== osd_op_reply(49 [pgnls start_epoch 127 out=177b] v126'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051400 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 9 ==== osd_op_reply(50 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e9ea5400 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 9 ==== osd_op_reply(51 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e9ea5400 2024-03-30T00:51:30.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 10 ==== osd_op_reply(52 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051000 2024-03-30T00:51:30.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 16 ==== osd_op_reply(53 [pgnls start_epoch 127 out=149b] v127'43 uv43 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051400 2024-03-30T00:51:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 10 ==== osd_op_reply(54 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e93b7800 2024-03-30T00:51:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.1 v2:172.21.15.66:6821/27458 5 ==== osd_op_reply(55 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e93b7800 2024-03-30T00:51:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e93b7c00 2024-03-30T00:51:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.4 v2:172.21.15.156:6800/27641 4 ==== osd_op_reply(56 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558e93b7c00 2024-03-30T00:51:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.335+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051000 2024-03-30T00:51:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 17 ==== osd_op_reply(57 [pgnls start_epoch 127 out=81b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051400 2024-03-30T00:51:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 11 ==== osd_op_reply(58 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e9ea5400 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 11 ==== osd_op_reply(59 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e94cc000 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.6 v2:172.21.15.156:6816/27642 6 ==== osd_op_reply(60 [pgnls start_epoch 127 out=77b] v126'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e94cc000 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e94cd400 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.5 v2:172.21.15.156:6808/27644 5 ==== osd_op_reply(61 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e94cd400 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e94cd800 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.2 v2:172.21.15.66:6816/27464 3 ==== osd_op_reply(62 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558e94cd800 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e93b7800 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.1 v2:172.21.15.66:6821/27458 6 ==== osd_op_reply(63 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e93b7800 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e9ea5400 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 12 ==== osd_op_reply(64 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051000 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 18 ==== osd_op_reply(65 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051000 2024-03-30T00:51:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 19 ==== osd_op_reply(66 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e94cd400 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.5 v2:172.21.15.156:6808/27644 6 ==== osd_op_reply(67 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e94cd400 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e94cc000 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.6 v2:172.21.15.156:6816/27642 7 ==== osd_op_reply(68 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e94cc000 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.339+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e94cd400 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.5 v2:172.21.15.156:6808/27644 7 ==== osd_op_reply(69 [pgnls start_epoch 127 out=97b] v126'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e94cd400 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051400 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 12 ==== osd_op_reply(70 [pgnls start_epoch 127 out=74b] v124'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051400 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 13 ==== osd_op_reply(71 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e93b7c00 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.4 v2:172.21.15.156:6800/27641 5 ==== osd_op_reply(72 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558e93b7c00 2024-03-30T00:51:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558ea051000 2024-03-30T00:51:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 20 ==== osd_op_reply(73 [pgnls start_epoch 127 out=98b] v126'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e9ea5400 2024-03-30T00:51:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 13 ==== osd_op_reply(74 [pgnls start_epoch 127 out=76b] v127'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e93b7c00 2024-03-30T00:51:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.4 v2:172.21.15.156:6800/27641 6 ==== osd_op_reply(75 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558e93b7c00 2024-03-30T00:51:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e9ea5400 2024-03-30T00:51:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 14 ==== osd_op_reply(76 [pgnls start_epoch 127 out=105b] v127'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e94cc000 2024-03-30T00:51:30.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.6 v2:172.21.15.156:6816/27642 8 ==== osd_op_reply(77 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e94cc000 2024-03-30T00:51:30.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e93b7800 2024-03-30T00:51:30.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.1 v2:172.21.15.66:6821/27458 7 ==== osd_op_reply(78 [pgnls start_epoch 127 out=73b] v127'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e93b7800 2024-03-30T00:51:30.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050400 con 0x5558e94cc000 2024-03-30T00:51:30.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.6 v2:172.21.15.156:6816/27642 9 ==== osd_op_reply(79 [pgnls start_epoch 127 out=101b] v127'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e94cc000 2024-03-30T00:51:30.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0400 con 0x5558ea051000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.343+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0800 con 0x5558ea051000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e1c00 con 0x5558ea051000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e1000 con 0x5558ea051000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e1400 con 0x5558ea051000 2024-03-30T00:51:30.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e1800 con 0x5558ea051000 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0000 con 0x5558e93b7800 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.1 v2:172.21.15.66:6821/27458 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e93b7800 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-03-30T00:51:30.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e93b7400 con 0x5558ea051000 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 27 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050c00 con 0x5558e94cd800 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.2 v2:172.21.15.66:6816/27464 4 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558e94cd800 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050800 con 0x5558e94cd400 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.5 v2:172.21.15.156:6808/27644 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558e94cd400 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-03-30T00:51:30.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0c00 con 0x5558ea051400 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0400 con 0x5558ea051000 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0800 con 0x5558e9ea5400 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-03-30T00:51:30.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.347+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e1c00 con 0x5558e9ea5400 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 20 datascan.scan_links: injected ino detected 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e1000 con 0x5558e94cc000 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.6 v2:172.21.15.156:6816/27642 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e94cc000 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 datascan.scan_links: processing 1 injected_inos 2024-03-30T00:51:30.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e1400 con 0x5558e9ea5400 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 20 datascan.scan_links: first is now 2 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0000 con 0x5558e9ea5400 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 <== osd.0 v2:172.21.15.66:6800/27456 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v127'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5558e9fd8fc0 con 0x5558e9ea5400 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 datascan.scan_links: updating inotable 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558e94e0c00 con 0x5558ea051000 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 29 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-03-30T00:51:30.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.351+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050000 con 0x5558ea051000 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83c400640 1 -- 172.21.15.66:0/1727438477 <== osd.3 v2:172.21.15.66:6808/27471 30 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v127'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5558e9fd8d80 con 0x5558ea051000 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 10 datascan.scan_links: updating snaptable 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x5558ea050800 con 0x5558ea051400 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83b000640 1 -- 172.21.15.66:0/1727438477 <== osd.7 v2:172.21.15.156:6824/27643 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x5558e9fd8b40 con 0x5558ea051400 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5558e9ea5400 msgr2=0x5558e94be580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x5558e9ea5400 0x5558e94be580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5558e93b7800 msgr2=0x5558e94beb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x5558e93b7800 0x5558e94beb00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x5558e94cd800 msgr2=0x5558e94bf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x5558e94cd800 0x5558e94bf600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x5558ea051000 msgr2=0x5558e94be000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x5558ea051000 0x5558e94be000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x5558e93b7c00 msgr2=0x5558e94bf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x5558e93b7c00 0x5558e94bf080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5558e94cd400 msgr2=0x5558e94c0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x5558e94cd400 0x5558e94c0680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 reap_dead start 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5558e94cc000 msgr2=0x5558e94c0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x5558e94cc000 0x5558e94c0100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83ba00640 1 -- 172.21.15.66:0/1727438477 reap_dead start 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x5558ea051400 msgr2=0x5558e9164100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x5558ea051400 0x5558e9164100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5558e93b6800 msgr2=0x5558e9165700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5558e93b6800 0x5558e9165700 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x5558e91ea750 tx=0x5558e949e2a0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5800 msgr2=0x5558e9164c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5800 0x5558e9164c00 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x5558e9fdf5c0 tx=0x5558e9fe8000 comp rx=0 tx=0).stop 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 shutdown_connections 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x5558ea051400 0x5558e9164100 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x5558e93b6800 0x5558e9165700 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/1727438477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e9ea5800 0x5558e9164c00 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 >> 172.21.15.66:0/1727438477 conn(0x5558e90d6c00 msgr2=0x5558e9101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 shutdown_connections 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/1727438477 wait complete. 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/4284962883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e90d7400 msgr2=0x5558e9162b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/4284962883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e90d7400 0x5558e9162b00 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x5558e9f3a9f0 tx=0x5558e9e59f20 comp rx=0 tx=0).stop 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/4284962883 shutdown_connections 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/4284962883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e90d7000 0x5558e9163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/4284962883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558e90d7400 0x5558e9162b00 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 --2- 172.21.15.66:0/4284962883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5558e90d7800 0x5558e9162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/4284962883 >> 172.21.15.66:0/4284962883 conn(0x5558e90d7c00 msgr2=0x5558e9101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/4284962883 shutdown_connections 2024-03-30T00:51:30.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.355+0000 7fd83d8718c0 1 -- 172.21.15.66:0/4284962883 wait complete. 2024-03-30T00:51:30.374 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.151862, result: 2024-03-30T00:51:30.376 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T00:51:30.420 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-03-30T00:51:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-03-30T00:51:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-03-30T00:51:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-03-30T00:51:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 20 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 20 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 10 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-03-30T00:51:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T00:51:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T00:51:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-03-30T00:51:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-03-30T00:51:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-03-30T00:51:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: mon op complaint time = 120 2024-03-30T00:51:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 600 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-03-30T00:51:30.470 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-03-30T00:51:30.471 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-03-30T00:51:30.471 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-03-30T00:51:30.471 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-03-30T00:51:30.471 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-03-30T00:51:30.471 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-03-30T00:51:30.471 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-03-30T00:51:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds = 20 2024-03-30T00:51:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds balancer = 20 2024-03-30T00:51:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug frag = True 2024-03-30T00:51:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug scatterstat = False 2024-03-30T00:51:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: mds op complaint time = 180 2024-03-30T00:51:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: mds verify scatter = True 2024-03-30T00:51:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T00:51:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T00:51:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T00:51:30.472 INFO:teuthology.orchestra.run.smithi156.stdout:[global] 2024-03-30T00:51:30.472 INFO:teuthology.orchestra.run.smithi156.stdout: chdir = "" 2024-03-30T00:51:30.472 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: auth supported = cephx 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: filestore xattr use omap = true 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: mon clock drift allowed = 1.000 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: osd crush chooseleaf type = 0 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: auth debug = true 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on old message = true 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on bug = true 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T00:51:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: mon pg warn max object skew = 0 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default size = 2 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow primary affinity = true 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow pg remap = true 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on legacy crush tunables = false 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on crush straw calc version zero = false 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on no sortbitwise = false 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on osd down out interval zero = false 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on too few osds = false 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T00:51:30.474 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: mon_allow_pool_size_one = true 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: osd default data pool replay window = 5 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: mon allow pool delete = true 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: mon cluster log file level = debug 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: debug asserts on shutdown = true 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: mon health detail to clog = false 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: mon host = "172.21.15.66,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: fsid = 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:51:30.475 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout:[osd] 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: osd journal size = 100 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub load threshold = 5.0 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub max interval = 600 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: osd mclock profile = high_recovery_ops 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: osd recover clone overlap = true 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: osd recovery max chunk = 1048576 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug shutdown = true 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug op order = true 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug verify stray on activate = true 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.476 INFO:teuthology.orchestra.run.smithi156.stdout: osd open classes on start = true 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug pg log writeout = true 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: osd deep scrub update digest min age = 30 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: osd map max advance = 10 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: journal zero on create = true 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: filestore ondisk finisher threads = 3 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: filestore apply finisher threads = 3 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: bdev debug aio = true 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug misdirected ops = true 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore block size = 96636764160 2024-03-30T00:51:30.477 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore fsck on mount = True 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluefs = 20 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluestore = 20 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: debug osd = 20 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: debug rocksdb = 10 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd full ratio = 0.9 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd nearfull ratio = 0.8 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: osd failsafe full ratio = 0.95 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: osd objectstore = bluestore 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.478 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr] 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgr = 20 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: mgr/telemetry/nag = false 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout:[mon] 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: debug paxos = 20 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: mon data avail warn = 5 2024-03-30T00:51:30.479 INFO:teuthology.orchestra.run.smithi156.stdout: mon mgr mkfs grace = 240 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd reporter subtree level = osd 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd prime pg temp = true 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.480 INFO:teuthology.orchestra.run.smithi156.stdout: # 1m isn't quite enough 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: mon_down_mkfs_grace = 2m 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_filestore_osds = false 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: mon op complaint time = 120 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout:[client] 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: rgw cache enabled = true 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable ops log = true 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable usage log = true 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: client mount timeout = 600 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:30.481 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = "" 2024-03-30T00:51:30.482 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2024-03-30T00:51:30.482 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2024-03-30T00:51:30.482 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.a] 2024-03-30T00:51:30.482 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.b] 2024-03-30T00:51:30.482 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.c] 2024-03-30T00:51:30.482 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2024-03-30T00:51:30.482 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2024-03-30T00:51:30.482 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2024-03-30T00:51:30.482 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2024-03-30T00:51:30.482 INFO:teuthology.orchestra.run.smithi156.stdout:[mds] 2024-03-30T00:51:30.483 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds = 20 2024-03-30T00:51:30.483 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds balancer = 20 2024-03-30T00:51:30.483 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:30.483 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug frag = True 2024-03-30T00:51:30.483 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug scatterstat = False 2024-03-30T00:51:30.483 INFO:teuthology.orchestra.run.smithi156.stdout: mds op complaint time = 180 2024-03-30T00:51:30.483 INFO:teuthology.orchestra.run.smithi156.stdout: mds verify scatter = True 2024-03-30T00:51:30.483 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2024-03-30T00:51:30.483 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2024-03-30T00:51:30.483 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2024-03-30T00:51:30.485 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T00:51:30.529 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T00:51:30.568 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-03-30T00:51:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-03-30T00:51:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T00:51:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-03-30T00:51:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-03-30T00:51:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2024-03-30T00:51:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-03-30T00:51:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-03-30T00:51:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 20 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 20 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 10 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T00:51:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-03-30T00:51:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T00:51:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: mon op complaint time = 120 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T00:51:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 600 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-03-30T00:51:30.578 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-03-30T00:51:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds = 20 2024-03-30T00:51:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds balancer = 20 2024-03-30T00:51:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T00:51:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug frag = True 2024-03-30T00:51:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug scatterstat = True 2024-03-30T00:51:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: mds op complaint time = 180 2024-03-30T00:51:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: mds verify scatter = True 2024-03-30T00:51:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T00:51:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T00:51:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T00:51:30.579 INFO:teuthology.orchestra.run.smithi156.stdout:[global] 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: chdir = "" 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: auth supported = cephx 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: filestore xattr use omap = true 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: mon clock drift allowed = 1.000 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: osd crush chooseleaf type = 0 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: auth debug = true 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on old message = true 2024-03-30T00:51:30.580 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on bug = true 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: mon pg warn max object skew = 0 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default size = 2 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow primary affinity = true 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow pg remap = true 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on legacy crush tunables = false 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on crush straw calc version zero = false 2024-03-30T00:51:30.581 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on no sortbitwise = false 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on osd down out interval zero = false 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on too few osds = false 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: mon_allow_pool_size_one = true 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: osd default data pool replay window = 5 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: mon allow pool delete = true 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.582 INFO:teuthology.orchestra.run.smithi156.stdout: mon cluster log file level = debug 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: debug asserts on shutdown = true 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: mon health detail to clog = false 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: mon host = "172.21.15.66,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: fsid = 0aec3283-d125-4c04-9eb7-d2c2f49c35a9 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout:[osd] 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: osd journal size = 100 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub load threshold = 5.0 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub max interval = 600 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: osd mclock profile = high_recovery_ops 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: osd recover clone overlap = true 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: osd recovery max chunk = 1048576 2024-03-30T00:51:30.583 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug shutdown = true 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug op order = true 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug verify stray on activate = true 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: osd open classes on start = true 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug pg log writeout = true 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: osd deep scrub update digest min age = 30 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: osd map max advance = 10 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: journal zero on create = true 2024-03-30T00:51:30.584 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: filestore ondisk finisher threads = 3 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: filestore apply finisher threads = 3 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: bdev debug aio = true 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug misdirected ops = true 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore block size = 96636764160 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore fsck on mount = True 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluefs = 20 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluestore = 20 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: debug osd = 20 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: debug rocksdb = 10 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T00:51:30.585 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd full ratio = 0.9 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd nearfull ratio = 0.8 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout: osd failsafe full ratio = 0.95 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout: osd objectstore = bluestore 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr] 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgr = 20 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:30.586 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: mgr/telemetry/nag = false 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout:[mon] 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: debug paxos = 20 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: mon data avail warn = 5 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: mon mgr mkfs grace = 240 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd reporter subtree level = osd 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd prime pg temp = true 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2024-03-30T00:51:30.587 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: # 1m isn't quite enough 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: mon_down_mkfs_grace = 2m 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_filestore_osds = false 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: mon op complaint time = 120 2024-03-30T00:51:30.588 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-30T00:51:30.589 INFO:teuthology.orchestra.run.smithi156.stdout:[client] 2024-03-30T00:51:30.589 INFO:teuthology.orchestra.run.smithi156.stdout: rgw cache enabled = true 2024-03-30T00:51:30.589 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable ops log = true 2024-03-30T00:51:30.589 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable usage log = true 2024-03-30T00:51:30.589 INFO:teuthology.orchestra.run.smithi156.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T00:51:30.589 INFO:teuthology.orchestra.run.smithi156.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T00:51:30.589 INFO:teuthology.orchestra.run.smithi156.stdout: client mount timeout = 600 2024-03-30T00:51:30.589 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2024-03-30T00:51:30.589 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:30.589 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = "" 2024-03-30T00:51:30.590 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2024-03-30T00:51:30.590 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2024-03-30T00:51:30.590 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.a] 2024-03-30T00:51:30.590 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.b] 2024-03-30T00:51:30.590 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.c] 2024-03-30T00:51:30.590 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2024-03-30T00:51:30.590 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2024-03-30T00:51:30.590 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2024-03-30T00:51:30.590 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2024-03-30T00:51:30.590 INFO:teuthology.orchestra.run.smithi156.stdout:[mds] 2024-03-30T00:51:30.591 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds = 20 2024-03-30T00:51:30.591 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds balancer = 20 2024-03-30T00:51:30.591 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2024-03-30T00:51:30.591 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug frag = True 2024-03-30T00:51:30.591 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug scatterstat = True 2024-03-30T00:51:30.591 INFO:teuthology.orchestra.run.smithi156.stdout: mds op complaint time = 180 2024-03-30T00:51:30.591 INFO:teuthology.orchestra.run.smithi156.stdout: mds verify scatter = True 2024-03-30T00:51:30.591 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2024-03-30T00:51:30.591 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2024-03-30T00:51:30.591 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2024-03-30T00:51:30.593 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-30T00:51:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.775+0000 7f57b2400640 1 Processor -- start 2024-03-30T00:51:30.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.775+0000 7f57b2400640 1 -- start start 2024-03-30T00:51:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.775+0000 7f57b2400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f57ac0cec30 0x7f57ac159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.775+0000 7f57b2400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57ac0cde70 0x7f57ac15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.775+0000 7f57b2400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ac15a6e0 con 0x7f57ac0cda90 2024-03-30T00:51:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.775+0000 7f57b2400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ac0b1b80 con 0x7f57ac0cec30 2024-03-30T00:51:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.775+0000 7f57b2400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0cd3e0 con 0x7f57ac0cde70 2024-03-30T00:51:30.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.775+0000 7f57a2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f57ac0cec30 0x7f57ac159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.775+0000 7f57a2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f57ac0cec30 0x7f57ac159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40352/0 (socket says 172.21.15.66:40352) 2024-03-30T00:51:30.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.775+0000 7f57a2a00640 1 -- 172.21.15.66:0/1110188976 learned_addr learned my addr 172.21.15.66:0/1110188976 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:30.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.775+0000 7f57abe00640 1 --2- 172.21.15.66:0/1110188976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57ac0cde70 0x7f57ac15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57a2a00640 1 -- 172.21.15.66:0/1110188976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57ac0cde70 msgr2=0x7f57ac15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:30.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57a2a00640 1 --2- 172.21.15.66:0/1110188976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57ac0cde70 0x7f57ac15a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57a2a00640 1 -- 172.21.15.66:0/1110188976 >> v1:172.21.15.156:6789/0 conn(0x7f57ac0cda90 legacy=0x7f57ac0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57a2a00640 1 -- 172.21.15.66:0/1110188976 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac15a990 con 0x7f57ac0cec30 2024-03-30T00:51:30.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57a2a00640 1 --2- 172.21.15.66:0/1110188976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57ac0cec30 0x7f57ac159c60 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f5794009870 tx=0x7f579401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd2f3781d938620 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57aaa00640 1 -- 172.21.15.66:0/1110188976 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57940aa430 con 0x7f57ac0cec30 2024-03-30T00:51:30.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57aaa00640 1 -- 172.21.15.66:0/1110188976 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57940aa590 con 0x7f57ac0cec30 2024-03-30T00:51:30.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57aaa00640 1 -- 172.21.15.66:0/1110188976 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57940aa920 con 0x7f57ac0cec30 2024-03-30T00:51:30.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57b2400640 1 -- 172.21.15.66:0/1110188976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57ac0cec30 msgr2=0x7f57ac159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57b2400640 1 --2- 172.21.15.66:0/1110188976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57ac0cec30 0x7f57ac159c60 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f5794009870 tx=0x7f579401c4d0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57b2400640 1 -- 172.21.15.66:0/1110188976 shutdown_connections 2024-03-30T00:51:30.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57b2400640 1 --2- 172.21.15.66:0/1110188976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57ac0cde70 0x7f57ac15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57b2400640 1 --2- 172.21.15.66:0/1110188976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57ac0cec30 0x7f57ac159c60 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57b2400640 1 -- 172.21.15.66:0/1110188976 >> 172.21.15.66:0/1110188976 conn(0x7f57ac069f60 msgr2=0x7f57ac06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:30.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57b2400640 1 -- 172.21.15.66:0/1110188976 shutdown_connections 2024-03-30T00:51:30.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.779+0000 7f57b2400640 1 -- 172.21.15.66:0/1110188976 wait complete. 2024-03-30T00:51:30.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57b2400640 1 Processor -- start 2024-03-30T00:51:30.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57b2400640 1 -- start start 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57b2400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57ac0cda90 0x7f57ac152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57b2400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57ac0cde70 0x7f57ac163250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57b2400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57ac0cec30 0x7f57ac159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57b2400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac1532a0 con 0x7f57ac0cda90 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57b2400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac159150 con 0x7f57ac0cec30 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57b2400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0abfd0 con 0x7f57ac0cde70 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57ab400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57ac0cda90 0x7f57ac152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57ab400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57ac0cda90 0x7f57ac152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40356/0 (socket says 172.21.15.66:40356) 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57ab400640 1 -- 172.21.15.66:0/1467655963 learned_addr learned my addr 172.21.15.66:0/1467655963 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57a2a00640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57ac0cde70 0x7f57ac163250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57abe00640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57ac0cec30 0x7f57ac159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57ab400640 1 -- 172.21.15.66:0/1467655963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57ac0cde70 msgr2=0x7f57ac163250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57ab400640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57ac0cde70 0x7f57ac163250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57ab400640 1 -- 172.21.15.66:0/1467655963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57ac0cec30 msgr2=0x7f57ac159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:30.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57ab400640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57ac0cec30 0x7f57ac159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:30.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57ab400640 1 -- 172.21.15.66:0/1467655963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57940094d0 con 0x7f57ac0cda90 2024-03-30T00:51:30.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57ab400640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57ac0cda90 0x7f57ac152d60 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f579c00c500 tx=0x7f579c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57a3e00640 1 -- 172.21.15.66:0/1467655963 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c005220 con 0x7f57ac0cda90 2024-03-30T00:51:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57a3e00640 1 -- 172.21.15.66:0/1467655963 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c005380 con 0x7f57ac0cda90 2024-03-30T00:51:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57b2400640 1 -- 172.21.15.66:0/1467655963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ac0ab980 con 0x7f57ac0cda90 2024-03-30T00:51:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.783+0000 7f57a3e00640 1 -- 172.21.15.66:0/1467655963 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c007270 con 0x7f57ac0cda90 2024-03-30T00:51:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.787+0000 7f57b2400640 1 -- 172.21.15.66:0/1467655963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ac1f60f0 con 0x7f57ac0cda90 2024-03-30T00:51:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.787+0000 7f57b2400640 1 -- 172.21.15.66:0/1467655963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ac066c30 con 0x7f57ac0cda90 2024-03-30T00:51:30.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.787+0000 7f57a3e00640 1 -- 172.21.15.66:0/1467655963 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f579c0073d0 con 0x7f57ac0cda90 2024-03-30T00:51:30.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.787+0000 7f57a3e00640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5788066690 0x7f5788068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:30.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.787+0000 7f57a3e00640 1 -- 172.21.15.66:0/1467655963 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f579c083b30 con 0x7f57ac0cda90 2024-03-30T00:51:30.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.787+0000 7f57a2a00640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5788066690 0x7f5788068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:30.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.791+0000 7f57a2a00640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5788066690 0x7f5788068b50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f57940ae000 tx=0x7f57940b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:30.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.791+0000 7f57a3e00640 1 -- 172.21.15.66:0/1467655963 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f579c050d70 con 0x7f57ac0cda90 2024-03-30T00:51:30.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:30.991+0000 7f57b2400640 1 -- 172.21.15.66:0/1467655963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f57ac0b3170 con 0x7f57ac0cda90 2024-03-30T00:51:31.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.823+0000 7f57a3e00640 1 -- 172.21.15.66:0/1467655963 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v104) v1 ==== 162+0+0 (secure 0 0 0) 0x7f579c054390 con 0x7f57ac0cda90 2024-03-30T00:51:31.832 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 -- 172.21.15.66:0/1467655963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5788066690 msgr2=0x7f5788068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5788066690 0x7f5788068b50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f57940ae000 tx=0x7f57940b5040 comp rx=0 tx=0).stop 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 -- 172.21.15.66:0/1467655963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57ac0cda90 msgr2=0x7f57ac152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57ac0cda90 0x7f57ac152d60 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f579c00c500 tx=0x7f579c00c9c0 comp rx=0 tx=0).stop 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 -- 172.21.15.66:0/1467655963 shutdown_connections 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57ac0cec30 0x7f57ac159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57ac0cde70 0x7f57ac163250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5788066690 0x7f5788068b50 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 --2- 172.21.15.66:0/1467655963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57ac0cda90 0x7f57ac152d60 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 -- 172.21.15.66:0/1467655963 >> 172.21.15.66:0/1467655963 conn(0x7f57ac069f60 msgr2=0x7f57ac06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 -- 172.21.15.66:0/1467655963 shutdown_connections 2024-03-30T00:51:31.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.827+0000 7f57b2400640 1 -- 172.21.15.66:0/1467655963 wait complete. 2024-03-30T00:51:31.862 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:51:32.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9937200640 1 Processor -- start 2024-03-30T00:51:32.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9937200640 1 -- start start 2024-03-30T00:51:32.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9937200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f993015ca30 0x7f993006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:32.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9937200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f993006d710 0x7f9930069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:32.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9937200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9930160590 con 0x7f99300cda40 2024-03-30T00:51:32.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9937200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993006db90 con 0x7f993006d710 2024-03-30T00:51:32.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9937200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9930160850 con 0x7f993015ca30 2024-03-30T00:51:32.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9935400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f993006d710 0x7f9930069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:32.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9935e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f993015ca30 0x7f993006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:32.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9936800640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f99300cda40 0x7f99300b0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:55710/0 (socket says 172.21.15.66:55710) 2024-03-30T00:51:32.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9936800640 1 -- 172.21.15.66:0/1067757976 learned_addr learned my addr 172.21.15.66:0/1067757976 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:32.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9935400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f993006d710 0x7f9930069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:47260/0 (socket says 172.21.15.66:47260) 2024-03-30T00:51:32.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9935e00640 1 -- 172.21.15.66:0/1067757976 >> v2:172.21.15.156:3300/0 conn(0x7f993006d710 msgr2=0x7f9930069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:32.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9935e00640 1 --2- 172.21.15.66:0/1067757976 >> v2:172.21.15.156:3300/0 conn(0x7f993006d710 0x7f9930069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:32.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9935e00640 1 -- 172.21.15.66:0/1067757976 >> v1:172.21.15.156:6789/0 conn(0x7f99300cda40 legacy=0x7f99300b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:32.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9935e00640 1 -- 172.21.15.66:0/1067757976 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930160b30 con 0x7f993015ca30 2024-03-30T00:51:32.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9935400640 1 --2- 172.21.15.66:0/1067757976 >> v2:172.21.15.156:3300/0 conn(0x7f993006d710 0x7f9930069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:51:32.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:31.999+0000 7f9934a00640 1 -- 172.21.15.66:0/1067757976 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246666953 0 0) 0x7f9930160590 con 0x7f99300cda40 2024-03-30T00:51:32.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9935e00640 1 --2- 172.21.15.66:0/1067757976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f993015ca30 0x7f993006d1d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f99240097c0 tx=0x7f992401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cdb70bde221993d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:32.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9934a00640 1 -- 172.21.15.66:0/1067757976 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240ad680 con 0x7f993015ca30 2024-03-30T00:51:32.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9934a00640 1 -- 172.21.15.66:0/1067757976 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99240ad7e0 con 0x7f993015ca30 2024-03-30T00:51:32.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9934a00640 1 -- 172.21.15.66:0/1067757976 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240adb70 con 0x7f993015ca30 2024-03-30T00:51:32.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9937200640 1 -- 172.21.15.66:0/1067757976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f993015ca30 msgr2=0x7f993006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:32.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9937200640 1 --2- 172.21.15.66:0/1067757976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f993015ca30 0x7f993006d1d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f99240097c0 tx=0x7f992401c470 comp rx=0 tx=0).stop 2024-03-30T00:51:32.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9937200640 1 -- 172.21.15.66:0/1067757976 shutdown_connections 2024-03-30T00:51:32.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9937200640 1 --2- 172.21.15.66:0/1067757976 >> v2:172.21.15.156:3300/0 conn(0x7f993006d710 0x7f9930069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:32.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9937200640 1 --2- 172.21.15.66:0/1067757976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f993015ca30 0x7f993006d1d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:32.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9937200640 1 -- 172.21.15.66:0/1067757976 >> 172.21.15.66:0/1067757976 conn(0x7f99300cbdc0 msgr2=0x7f99300cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:32.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9937200640 1 -- 172.21.15.66:0/1067757976 shutdown_connections 2024-03-30T00:51:32.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.003+0000 7f9937200640 1 -- 172.21.15.66:0/1067757976 wait complete. 2024-03-30T00:51:32.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9937200640 1 Processor -- start 2024-03-30T00:51:32.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9937200640 1 -- start start 2024-03-30T00:51:32.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9937200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f993006d710 0x7f99301548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9937200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99300cda40 0x7f99300b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9937200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f993015ca30 0x7f99301527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9937200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993015b720 con 0x7f99300cda40 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9937200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300b4e30 con 0x7f993006d710 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9937200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993015aaa0 con 0x7f993015ca30 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9935400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99300cda40 0x7f99300b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9935400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99300cda40 0x7f99300b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40370/0 (socket says 172.21.15.66:40370) 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9935400640 1 -- 172.21.15.66:0/2036168025 learned_addr learned my addr 172.21.15.66:0/2036168025 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9935e00640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f993006d710 0x7f99301548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9936800640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f993015ca30 0x7f99301527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9935400640 1 -- 172.21.15.66:0/2036168025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f993015ca30 msgr2=0x7f99301527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:32.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9935400640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f993015ca30 0x7f99301527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:32.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9935400640 1 -- 172.21.15.66:0/2036168025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f993006d710 msgr2=0x7f99301548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:32.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9935400640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f993006d710 0x7f99301548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:32.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9935400640 1 -- 172.21.15.66:0/2036168025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9920009420 con 0x7f99300cda40 2024-03-30T00:51:32.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f9935400640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99300cda40 0x7f99300b2ca0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f992000ebb0 tx=0x7f992000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:32.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f992a000640 1 -- 172.21.15.66:0/2036168025 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992000eeb0 con 0x7f99300cda40 2024-03-30T00:51:32.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f992a000640 1 -- 172.21.15.66:0/2036168025 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992000cc30 con 0x7f99300cda40 2024-03-30T00:51:32.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.007+0000 7f992a000640 1 -- 172.21.15.66:0/2036168025 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920013470 con 0x7f99300cda40 2024-03-30T00:51:32.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.011+0000 7f9937200640 1 -- 172.21.15.66:0/2036168025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9924009420 con 0x7f99300cda40 2024-03-30T00:51:32.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.011+0000 7f9937200640 1 -- 172.21.15.66:0/2036168025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993015b060 con 0x7f99300cda40 2024-03-30T00:51:32.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.011+0000 7f9937200640 1 -- 172.21.15.66:0/2036168025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9930067110 con 0x7f99300cda40 2024-03-30T00:51:32.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.011+0000 7f992a000640 1 -- 172.21.15.66:0/2036168025 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f99200135d0 con 0x7f99300cda40 2024-03-30T00:51:32.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.011+0000 7f992a000640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9914066690 0x7f9914068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:32.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.011+0000 7f992a000640 1 -- 172.21.15.66:0/2036168025 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f9920084640 con 0x7f99300cda40 2024-03-30T00:51:32.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.011+0000 7f9935e00640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9914066690 0x7f9914068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:32.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.011+0000 7f9935e00640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9914066690 0x7f9914068b50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f99240af000 tx=0x7f9924008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:32.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.015+0000 7f992a000640 1 -- 172.21.15.66:0/2036168025 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9920051860 con 0x7f99300cda40 2024-03-30T00:51:32.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.207+0000 7f9937200640 1 -- 172.21.15.66:0/2036168025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9930073cf0 con 0x7f99300cda40 2024-03-30T00:51:32.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.207+0000 7f992a000640 1 -- 172.21.15.66:0/2036168025 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4744 (secure 0 0 0) 0x7f9920054e80 con 0x7f99300cda40 2024-03-30T00:51:32.217 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:32.217 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:51:31.826619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9942":{"gid":9942,"name":"a","rank":0,"incarnation":105,"state":"up:replay","state_seq":1,"addr":"172.21.15.66:6836/2774791553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2774791553},{"type":"v1","addr":"172.21.15.66:6836","nonce":2774791553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:51:32.218 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 105 2024-03-30T00:51:32.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 -- 172.21.15.66:0/2036168025 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9914066690 msgr2=0x7f9914068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:32.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9914066690 0x7f9914068b50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f99240af000 tx=0x7f9924008040 comp rx=0 tx=0).stop 2024-03-30T00:51:32.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 -- 172.21.15.66:0/2036168025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99300cda40 msgr2=0x7f99300b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:32.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99300cda40 0x7f99300b2ca0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f992000ebb0 tx=0x7f992000c520 comp rx=0 tx=0).stop 2024-03-30T00:51:32.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 -- 172.21.15.66:0/2036168025 shutdown_connections 2024-03-30T00:51:32.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f993015ca30 0x7f99301527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:32.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9914066690 0x7f9914068b50 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:32.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99300cda40 0x7f99300b2ca0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:32.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 --2- 172.21.15.66:0/2036168025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f993006d710 0x7f99301548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:32.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 -- 172.21.15.66:0/2036168025 >> 172.21.15.66:0/2036168025 conn(0x7f99300cbdc0 msgr2=0x7f99300c9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:32.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 -- 172.21.15.66:0/2036168025 shutdown_connections 2024-03-30T00:51:32.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:32.211+0000 7f9937200640 1 -- 172.21.15.66:0/2036168025 wait complete. 2024-03-30T00:51:32.641 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:50:38.040525+0000', 'modified': '2024-03-30T00:51:31.826619+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9942}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9942': {'gid': 9942, 'name': 'a', 'rank': 0, 'incarnation': 105, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.66:6836/2774791553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 2774791553}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 2774791553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:51:32.641 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9942:up:replay 2024-03-30T00:51:33.642 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:51:33.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.787+0000 7f89f5c00640 1 Processor -- start 2024-03-30T00:51:33.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.787+0000 7f89f5c00640 1 -- start start 2024-03-30T00:51:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89f5c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f89f00cec30 0x7f89f0159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89f5c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f89f00cde70 0x7f89f015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89f5c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f015a6e0 con 0x7f89f00cda90 2024-03-30T00:51:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89f5c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f00b1b80 con 0x7f89f00cec30 2024-03-30T00:51:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89f5c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f00cd3e0 con 0x7f89f00cde70 2024-03-30T00:51:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89eea00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f89f00cda90 0x7f89f00ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:59620/0 (socket says 172.21.15.66:59620) 2024-03-30T00:51:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89eea00640 1 -- 172.21.15.66:0/440537065 learned_addr learned my addr 172.21.15.66:0/440537065 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:33.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89e7e00640 1 --2- 172.21.15.66:0/440537065 >> v2:172.21.15.66:3300/0 conn(0x7f89f00cec30 0x7f89f0159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:33.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89ef400640 1 --2- 172.21.15.66:0/440537065 >> v2:172.21.15.156:3300/0 conn(0x7f89f00cde70 0x7f89f015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:33.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89e7e00640 1 -- 172.21.15.66:0/440537065 >> v2:172.21.15.156:3300/0 conn(0x7f89f00cde70 msgr2=0x7f89f015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:33.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89e7e00640 1 --2- 172.21.15.66:0/440537065 >> v2:172.21.15.156:3300/0 conn(0x7f89f00cde70 0x7f89f015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:33.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89e7e00640 1 -- 172.21.15.66:0/440537065 >> v1:172.21.15.66:6789/0 conn(0x7f89f00cda90 legacy=0x7f89f00ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:33.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89ee000640 1 -- 172.21.15.66:0/440537065 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418722027 0 0) 0x7f89f015a6e0 con 0x7f89f00cda90 2024-03-30T00:51:33.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89e7e00640 1 -- 172.21.15.66:0/440537065 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f015a990 con 0x7f89f00cec30 2024-03-30T00:51:33.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89ef400640 1 --2- 172.21.15.66:0/440537065 >> v2:172.21.15.156:3300/0 conn(0x7f89f00cde70 0x7f89f015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:33.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89e7e00640 1 --2- 172.21.15.66:0/440537065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f00cec30 0x7f89f0159c60 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f89f00adcf0 tx=0x7f89d801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb46764230e50403 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:33.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.791+0000 7f89ee000640 1 -- 172.21.15.66:0/440537065 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d80ab5f0 con 0x7f89f00cec30 2024-03-30T00:51:33.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89ee000640 1 -- 172.21.15.66:0/440537065 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d80ab750 con 0x7f89f00cec30 2024-03-30T00:51:33.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89ee000640 1 -- 172.21.15.66:0/440537065 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d80abae0 con 0x7f89f00cec30 2024-03-30T00:51:33.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 -- 172.21.15.66:0/440537065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f00cec30 msgr2=0x7f89f0159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 --2- 172.21.15.66:0/440537065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f00cec30 0x7f89f0159c60 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f89f00adcf0 tx=0x7f89d801c3f0 comp rx=0 tx=0).stop 2024-03-30T00:51:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 -- 172.21.15.66:0/440537065 shutdown_connections 2024-03-30T00:51:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 --2- 172.21.15.66:0/440537065 >> v2:172.21.15.156:3300/0 conn(0x7f89f00cde70 0x7f89f015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 --2- 172.21.15.66:0/440537065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f00cec30 0x7f89f0159c60 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 -- 172.21.15.66:0/440537065 >> 172.21.15.66:0/440537065 conn(0x7f89f0069f60 msgr2=0x7f89f006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 -- 172.21.15.66:0/440537065 shutdown_connections 2024-03-30T00:51:33.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 -- 172.21.15.66:0/440537065 wait complete. 2024-03-30T00:51:33.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 Processor -- start 2024-03-30T00:51:33.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 -- start start 2024-03-30T00:51:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89f00cda90 0x7f89f00718d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f89f00cde70 0x7f89f0097960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f00cec30 0x7f89f015b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0097ea0 con 0x7f89f00cec30 2024-03-30T00:51:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c4c30 con 0x7f89f00cda90 2024-03-30T00:51:33.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89f5c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c4390 con 0x7f89f00cde70 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89ef400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f00cec30 0x7f89f015b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89ef400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f00cec30 0x7f89f015b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40392/0 (socket says 172.21.15.66:40392) 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89ef400640 1 -- 172.21.15.66:0/1072482362 learned_addr learned my addr 172.21.15.66:0/1072482362 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89e7e00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f89f00cde70 0x7f89f0097960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.795+0000 7f89eea00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89f00cda90 0x7f89f00718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89ef400640 1 -- 172.21.15.66:0/1072482362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f89f00cde70 msgr2=0x7f89f0097960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89ef400640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f89f00cde70 0x7f89f0097960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89ef400640 1 -- 172.21.15.66:0/1072482362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89f00cda90 msgr2=0x7f89f00718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89ef400640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89f00cda90 0x7f89f00718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89ef400640 1 -- 172.21.15.66:0/1072482362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8009420 con 0x7f89f00cec30 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89eea00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89f00cda90 0x7f89f00718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89ef400640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f00cec30 0x7f89f015b1b0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f89e800cef0 tx=0x7f89e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89e6a00640 1 -- 172.21.15.66:0/1072482362 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e800d840 con 0x7f89f00cec30 2024-03-30T00:51:33.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89e6a00640 1 -- 172.21.15.66:0/1072482362 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e800d9c0 con 0x7f89f00cec30 2024-03-30T00:51:33.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89e6a00640 1 -- 172.21.15.66:0/1072482362 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e800dd70 con 0x7f89f00cec30 2024-03-30T00:51:33.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89f5c00640 1 -- 172.21.15.66:0/1072482362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f00c4670 con 0x7f89f00cec30 2024-03-30T00:51:33.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89f5c00640 1 -- 172.21.15.66:0/1072482362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f01626c0 con 0x7f89f00cec30 2024-03-30T00:51:33.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89f5c00640 1 -- 172.21.15.66:0/1072482362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f01628b0 con 0x7f89f00cec30 2024-03-30T00:51:33.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89e6a00640 1 -- 172.21.15.66:0/1072482362 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f89e801e3b0 con 0x7f89f00cec30 2024-03-30T00:51:33.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89e6a00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f89cc066690 0x7f89cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:33.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89e6a00640 1 -- 172.21.15.66:0/1072482362 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f89e8084e30 con 0x7f89f00cec30 2024-03-30T00:51:33.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.799+0000 7f89eea00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f89cc066690 0x7f89cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:33.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.803+0000 7f89eea00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f89cc066690 0x7f89cc068b50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f89e0007fd0 tx=0x7f89e0007c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:33.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.803+0000 7f89e6a00640 1 -- 172.21.15.66:0/1072482362 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f89e8052080 con 0x7f89f00cec30 2024-03-30T00:51:34.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.999+0000 7f89f5c00640 1 -- 172.21.15.66:0/1072482362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89f0066c30 con 0x7f89f00cec30 2024-03-30T00:51:34.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:33.999+0000 7f89e6a00640 1 -- 172.21.15.66:0/1072482362 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4744 (secure 0 0 0) 0x7f89e801a020 con 0x7f89f00cec30 2024-03-30T00:51:34.010 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:34.010 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":107,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:51:33.833334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9942":{"gid":9942,"name":"a","rank":0,"incarnation":105,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.66:6836/2774791553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2774791553},{"type":"v1","addr":"172.21.15.66:6836","nonce":2774791553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:51:34.010 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 107 2024-03-30T00:51:34.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 -- 172.21.15.66:0/1072482362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f89cc066690 msgr2=0x7f89cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:34.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f89cc066690 0x7f89cc068b50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f89e0007fd0 tx=0x7f89e0007c20 comp rx=0 tx=0).stop 2024-03-30T00:51:34.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 -- 172.21.15.66:0/1072482362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f00cec30 msgr2=0x7f89f015b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:34.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f00cec30 0x7f89f015b1b0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f89e800cef0 tx=0x7f89e800a710 comp rx=0 tx=0).stop 2024-03-30T00:51:34.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 -- 172.21.15.66:0/1072482362 shutdown_connections 2024-03-30T00:51:34.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f89cc066690 0x7f89cc068b50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:34.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89f00cec30 0x7f89f015b1b0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:34.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f89f00cde70 0x7f89f0097960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:34.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 --2- 172.21.15.66:0/1072482362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89f00cda90 0x7f89f00718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:34.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 -- 172.21.15.66:0/1072482362 >> 172.21.15.66:0/1072482362 conn(0x7f89f0069f60 msgr2=0x7f89f015c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:34.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 -- 172.21.15.66:0/1072482362 shutdown_connections 2024-03-30T00:51:34.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:34.003+0000 7f89f5c00640 1 -- 172.21.15.66:0/1072482362 wait complete. 2024-03-30T00:51:34.433 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:50:38.040525+0000', 'modified': '2024-03-30T00:51:33.833334+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9942}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9942': {'gid': 9942, 'name': 'a', 'rank': 0, 'incarnation': 105, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.66:6836/2774791553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 2774791553}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 2774791553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:51:34.433 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9942:up:rejoin 2024-03-30T00:51:35.434 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:51:35.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.575+0000 7fded9c00640 1 Processor -- start 2024-03-30T00:51:35.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded9c00640 1 -- start start 2024-03-30T00:51:35.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded9c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fded415a640 0x7fded406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:35.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded9c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fded40ce520 0x7fded40ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:35.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded9c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded406d070 con 0x7fded40cabb0 2024-03-30T00:51:35.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded9c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded40cbba0 con 0x7fded415a640 2024-03-30T00:51:35.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded9c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded406e2c0 con 0x7fded40ce520 2024-03-30T00:51:35.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fded415a640 0x7fded406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:35.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fded415a640 0x7fded406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40396/0 (socket says 172.21.15.66:40396) 2024-03-30T00:51:35.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded2a00640 1 -- 172.21.15.66:0/1785119811 learned_addr learned my addr 172.21.15.66:0/1785119811 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:35.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fdecbe00640 1 --2- 172.21.15.66:0/1785119811 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fded40ce520 0x7fded40ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:35.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded2a00640 1 -- 172.21.15.66:0/1785119811 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fded40ce520 msgr2=0x7fded40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:35.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded2a00640 1 --2- 172.21.15.66:0/1785119811 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fded40ce520 0x7fded40ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:35.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded2a00640 1 -- 172.21.15.66:0/1785119811 >> v1:172.21.15.156:6789/0 conn(0x7fded40cabb0 legacy=0x7fded40cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:35.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.579+0000 7fded2a00640 1 -- 172.21.15.66:0/1785119811 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded40cee60 con 0x7fded415a640 2024-03-30T00:51:35.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded2a00640 1 --2- 172.21.15.66:0/1785119811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fded415a640 0x7fded406dd80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009870 tx=0x7fdec401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b8e7c85411adfc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:35.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded2000640 1 -- 172.21.15.66:0/1785119811 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec40ab650 con 0x7fded415a640 2024-03-30T00:51:35.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded2000640 1 -- 172.21.15.66:0/1785119811 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec40ab7b0 con 0x7fded415a640 2024-03-30T00:51:35.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded2000640 1 -- 172.21.15.66:0/1785119811 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec40abb40 con 0x7fded415a640 2024-03-30T00:51:35.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded9c00640 1 -- 172.21.15.66:0/1785119811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fded415a640 msgr2=0x7fded406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:35.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded9c00640 1 --2- 172.21.15.66:0/1785119811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fded415a640 0x7fded406dd80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009870 tx=0x7fdec401c560 comp rx=0 tx=0).stop 2024-03-30T00:51:35.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded9c00640 1 -- 172.21.15.66:0/1785119811 shutdown_connections 2024-03-30T00:51:35.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded9c00640 1 --2- 172.21.15.66:0/1785119811 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fded40ce520 0x7fded40ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:35.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded9c00640 1 --2- 172.21.15.66:0/1785119811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fded415a640 0x7fded406dd80 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:35.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded9c00640 1 -- 172.21.15.66:0/1785119811 >> 172.21.15.66:0/1785119811 conn(0x7fded4069f60 msgr2=0x7fded406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:35.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded9c00640 1 -- 172.21.15.66:0/1785119811 shutdown_connections 2024-03-30T00:51:35.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded9c00640 1 -- 172.21.15.66:0/1785119811 wait complete. 2024-03-30T00:51:35.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded9c00640 1 Processor -- start 2024-03-30T00:51:35.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.583+0000 7fded9c00640 1 -- start start 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded9c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fded40cabb0 0x7fded4155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded9c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fded40ce520 0x7fded41f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fded415a640 0x7fded40c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded9c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41f2cd0 con 0x7fded415a640 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded9c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41f2f90 con 0x7fded40ce520 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded9c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded41f3270 con 0x7fded40cabb0 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fded415a640 0x7fded40c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded3400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fded415a640 0x7fded40c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40398/0 (socket says 172.21.15.66:40398) 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded2a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fded40cabb0 0x7fded4155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded3400640 1 -- 172.21.15.66:0/3084175596 learned_addr learned my addr 172.21.15.66:0/3084175596 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fdecbe00640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fded40ce520 0x7fded41f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded3400640 1 -- 172.21.15.66:0/3084175596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fded40cabb0 msgr2=0x7fded4155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:35.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded3400640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fded40cabb0 0x7fded4155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:35.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded3400640 1 -- 172.21.15.66:0/3084175596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fded40ce520 msgr2=0x7fded41f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:35.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded3400640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fded40ce520 0x7fded41f2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:35.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded3400640 1 -- 172.21.15.66:0/3084175596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec40094d0 con 0x7fded415a640 2024-03-30T00:51:35.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded3400640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fded415a640 0x7fded40c9440 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00c770 tx=0x7fdecc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:35.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fdecaa00640 1 -- 172.21.15.66:0/3084175596 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc003b80 con 0x7fded415a640 2024-03-30T00:51:35.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fdecaa00640 1 -- 172.21.15.66:0/3084175596 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc003ce0 con 0x7fded415a640 2024-03-30T00:51:35.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fdecaa00640 1 -- 172.21.15.66:0/3084175596 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00aa90 con 0x7fded415a640 2024-03-30T00:51:35.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded9c00640 1 -- 172.21.15.66:0/3084175596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded41f3550 con 0x7fded415a640 2024-03-30T00:51:35.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded9c00640 1 -- 172.21.15.66:0/3084175596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded4097d80 con 0x7fded415a640 2024-03-30T00:51:35.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fded9c00640 1 -- 172.21.15.66:0/3084175596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded4066c30 con 0x7fded415a640 2024-03-30T00:51:35.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.587+0000 7fdecaa00640 1 -- 172.21.15.66:0/3084175596 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdecc00abf0 con 0x7fded415a640 2024-03-30T00:51:35.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.591+0000 7fdecaa00640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdeb00666c0 0x7fdeb0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:35.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.591+0000 7fdecaa00640 1 -- 172.21.15.66:0/3084175596 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fdecc084050 con 0x7fded415a640 2024-03-30T00:51:35.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.591+0000 7fded2a00640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdeb00666c0 0x7fdeb0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:35.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.591+0000 7fded2a00640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdeb00666c0 0x7fdeb0068b80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdec40af000 tx=0x7fdec40b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:35.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.595+0000 7fdecaa00640 1 -- 172.21.15.66:0/3084175596 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdecc051280 con 0x7fded415a640 2024-03-30T00:51:35.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 -- 172.21.15.66:0/3084175596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fded406a4e0 con 0x7fded415a640 2024-03-30T00:51:35.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fdecaa00640 1 -- 172.21.15.66:0/3084175596 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4744 (secure 0 0 0) 0x7fdecc0548a0 con 0x7fded415a640 2024-03-30T00:51:35.799 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:35.799 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":108,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:51:34.842265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9942":{"gid":9942,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/2774791553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2774791553},{"type":"v1","addr":"172.21.15.66:6836","nonce":2774791553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:51:35.799 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 108 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 -- 172.21.15.66:0/3084175596 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdeb00666c0 msgr2=0x7fdeb0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdeb00666c0 0x7fdeb0068b80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdec40af000 tx=0x7fdec40b6040 comp rx=0 tx=0).stop 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 -- 172.21.15.66:0/3084175596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fded415a640 msgr2=0x7fded40c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fded415a640 0x7fded40c9440 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00c770 tx=0x7fdecc00cc30 comp rx=0 tx=0).stop 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 -- 172.21.15.66:0/3084175596 shutdown_connections 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdeb00666c0 0x7fdeb0068b80 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fded415a640 0x7fded40c9440 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fded40ce520 0x7fded41f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 --2- 172.21.15.66:0/3084175596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fded40cabb0 0x7fded4155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 -- 172.21.15.66:0/3084175596 >> 172.21.15.66:0/3084175596 conn(0x7fded4069f60 msgr2=0x7fded40ad7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 -- 172.21.15.66:0/3084175596 shutdown_connections 2024-03-30T00:51:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:35.791+0000 7fded9c00640 1 -- 172.21.15.66:0/3084175596 wait complete. 2024-03-30T00:51:36.222 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T00:50:38.040525+0000', 'modified': '2024-03-30T00:51:34.842265+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9942}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9942': {'gid': 9942, 'name': 'a', 'rank': 0, 'incarnation': 105, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.66:6836/2774791553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 2774791553}, {'type': 'v1', 'addr': '172.21.15.66:6836', 'nonce': 2774791553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:51:36.223 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T00:51:36.223 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-30T00:51:36.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07fd400640 1 Processor -- start 2024-03-30T00:51:36.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07fd400640 1 -- start start 2024-03-30T00:51:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07fd400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f07f80cda90 0x7f07f80ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07fd400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f07f815a370 0x7f07f80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07fd400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f815a990 con 0x7f07f80cde70 2024-03-30T00:51:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07fd400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f80b1b80 con 0x7f07f815a370 2024-03-30T00:51:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07fd400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f80cd3e0 con 0x7f07f80cda90 2024-03-30T00:51:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07f7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f07f815a370 0x7f07f80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07f7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f07f815a370 0x7f07f80cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40414/0 (socket says 172.21.15.66:40414) 2024-03-30T00:51:36.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07f7200640 1 -- 172.21.15.66:0/2181967617 learned_addr learned my addr 172.21.15.66:0/2181967617 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07f6800640 1 --2- 172.21.15.66:0/2181967617 >> v2:172.21.15.156:3300/0 conn(0x7f07f80cda90 0x7f07f80ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07f7200640 1 -- 172.21.15.66:0/2181967617 >> v2:172.21.15.156:3300/0 conn(0x7f07f80cda90 msgr2=0x7f07f80ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07f7200640 1 --2- 172.21.15.66:0/2181967617 >> v2:172.21.15.156:3300/0 conn(0x7f07f80cda90 0x7f07f80ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07f7200640 1 -- 172.21.15.66:0/2181967617 >> v1:172.21.15.156:6789/0 conn(0x7f07f80cde70 legacy=0x7f07f8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07f7200640 1 -- 172.21.15.66:0/2181967617 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f80b11e0 con 0x7f07f815a370 2024-03-30T00:51:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.359+0000 7f07f6800640 1 --2- 172.21.15.66:0/2181967617 >> v2:172.21.15.156:3300/0 conn(0x7f07f80cda90 0x7f07f80ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07f7200640 1 --2- 172.21.15.66:0/2181967617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f815a370 0x7f07f80cea60 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f07f80cf3f0 tx=0x7f07f001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948825699a8230e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07f5e00640 1 -- 172.21.15.66:0/2181967617 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00a5730 con 0x7f07f815a370 2024-03-30T00:51:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07f5e00640 1 -- 172.21.15.66:0/2181967617 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f00a5890 con 0x7f07f815a370 2024-03-30T00:51:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07f5e00640 1 -- 172.21.15.66:0/2181967617 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00a5c20 con 0x7f07f815a370 2024-03-30T00:51:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07fd400640 1 -- 172.21.15.66:0/2181967617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f815a370 msgr2=0x7f07f80cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07fd400640 1 --2- 172.21.15.66:0/2181967617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f815a370 0x7f07f80cea60 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f07f80cf3f0 tx=0x7f07f001c960 comp rx=0 tx=0).stop 2024-03-30T00:51:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07fd400640 1 -- 172.21.15.66:0/2181967617 shutdown_connections 2024-03-30T00:51:36.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07fd400640 1 --2- 172.21.15.66:0/2181967617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f815a370 0x7f07f80cea60 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07fd400640 1 --2- 172.21.15.66:0/2181967617 >> v2:172.21.15.156:3300/0 conn(0x7f07f80cda90 0x7f07f80ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07fd400640 1 -- 172.21.15.66:0/2181967617 >> 172.21.15.66:0/2181967617 conn(0x7f07f8069f60 msgr2=0x7f07f806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:36.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07fd400640 1 -- 172.21.15.66:0/2181967617 shutdown_connections 2024-03-30T00:51:36.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07fd400640 1 -- 172.21.15.66:0/2181967617 wait complete. 2024-03-30T00:51:36.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07fd400640 1 Processor -- start 2024-03-30T00:51:36.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.363+0000 7f07fd400640 1 -- start start 2024-03-30T00:51:36.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07fd400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f80cda90 0x7f07f8167b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07fd400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f80cde70 0x7f07f81634e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07fd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f815a370 0x7f07f8163a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07fd400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8158410 con 0x7f07f815a370 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07fd400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f81646e0 con 0x7f07f80cde70 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07fd400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f81580e0 con 0x7f07f80cda90 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07f7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f815a370 0x7f07f8163a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07f7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f815a370 0x7f07f8163a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40420/0 (socket says 172.21.15.66:40420) 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07f7200640 1 -- 172.21.15.66:0/4237364494 learned_addr learned my addr 172.21.15.66:0/4237364494 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07ede00640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f80cde70 0x7f07f81634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07f6800640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f80cda90 0x7f07f8167b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07f7200640 1 -- 172.21.15.66:0/4237364494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f80cda90 msgr2=0x7f07f8167b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07f7200640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f80cda90 0x7f07f8167b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07f7200640 1 -- 172.21.15.66:0/4237364494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f80cde70 msgr2=0x7f07f81634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07f7200640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f80cde70 0x7f07f81634e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07f7200640 1 -- 172.21.15.66:0/4237364494 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0009b30 con 0x7f07f815a370 2024-03-30T00:51:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07f7200640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f815a370 0x7f07f8163a20 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f07f00af040 tx=0x7f07f0019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07ef400640 1 -- 172.21.15.66:0/4237364494 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00b8670 con 0x7f07f815a370 2024-03-30T00:51:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07ef400640 1 -- 172.21.15.66:0/4237364494 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f00b87d0 con 0x7f07f815a370 2024-03-30T00:51:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07fd400640 1 -- 172.21.15.66:0/4237364494 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f8155ee0 con 0x7f07f815a370 2024-03-30T00:51:36.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07ef400640 1 -- 172.21.15.66:0/4237364494 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00b8b80 con 0x7f07f815a370 2024-03-30T00:51:36.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07fd400640 1 -- 172.21.15.66:0/4237364494 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f8157bc0 con 0x7f07f815a370 2024-03-30T00:51:36.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.367+0000 7f07ef400640 1 -- 172.21.15.66:0/4237364494 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f07f00b8ce0 con 0x7f07f815a370 2024-03-30T00:51:36.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07ef400640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07d4066690 0x7f07d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07ef400640 1 -- 172.21.15.66:0/4237364494 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f07f011f8c0 con 0x7f07f815a370 2024-03-30T00:51:36.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07f6800640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07d4066690 0x7f07d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 Processor -- start 2024-03-30T00:51:36.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 -- start start 2024-03-30T00:51:36.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07f6800640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07d4066690 0x7f07d4068b50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f07f8165470 tx=0x7f07e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:36.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b84db0 0x55af40b85190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b856d0 0x55af40b3af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b3b4c0 0x55af40b3b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af40a035a0 con 0x55af40b856d0 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af40990af0 con 0x55af40b3b4c0 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af4098e0c0 con 0x55af40b84db0 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07ede00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b856d0 0x55af40b3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07ede00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b856d0 0x55af40b3af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40434/0 (socket says 172.21.15.66:40434) 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07ede00640 1 -- 172.21.15.66:0/3595358486 learned_addr learned my addr 172.21.15.66:0/3595358486 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07f7200640 1 --2- 172.21.15.66:0/3595358486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b3b4c0 0x55af40b3b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07f6800640 1 --2- 172.21.15.66:0/3595358486 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b84db0 0x55af40b85190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07ede00640 1 -- 172.21.15.66:0/3595358486 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b84db0 msgr2=0x55af40b85190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07ede00640 1 --2- 172.21.15.66:0/3595358486 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b84db0 0x55af40b85190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07ede00640 1 -- 172.21.15.66:0/3595358486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b3b4c0 msgr2=0x55af40b3b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07ede00640 1 --2- 172.21.15.66:0/3595358486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b3b4c0 0x55af40b3b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07ede00640 1 -- 172.21.15.66:0/3595358486 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af40685890 con 0x55af40b856d0 2024-03-30T00:51:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07f6800640 1 --2- 172.21.15.66:0/3595358486 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b84db0 0x55af40b85190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07f7200640 1 --2- 172.21.15.66:0/3595358486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b3b4c0 0x55af40b3b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07ede00640 1 --2- 172.21.15.66:0/3595358486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b856d0 0x55af40b3af80 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f07e0001550 tx=0x7f07e0001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ffb43b9525da19 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07e7400640 1 -- 172.21.15.66:0/3595358486 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00050d0 con 0x55af40b856d0 2024-03-30T00:51:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07e7400640 1 -- 172.21.15.66:0/3595358486 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e0005230 con 0x55af40b856d0 2024-03-30T00:51:36.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f07e7400640 1 -- 172.21.15.66:0/3595358486 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e000e6e0 con 0x55af40b856d0 2024-03-30T00:51:36.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 -- 172.21.15.66:0/3595358486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b856d0 msgr2=0x55af40b3af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 --2- 172.21.15.66:0/3595358486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b856d0 0x55af40b3af80 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f07e0001550 tx=0x7f07e0001a10 comp rx=0 tx=0).stop 2024-03-30T00:51:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 -- 172.21.15.66:0/3595358486 shutdown_connections 2024-03-30T00:51:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 --2- 172.21.15.66:0/3595358486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b3b4c0 0x55af40b3b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 --2- 172.21.15.66:0/3595358486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b856d0 0x55af40b3af80 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 --2- 172.21.15.66:0/3595358486 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b84db0 0x55af40b85190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 -- 172.21.15.66:0/3595358486 >> 172.21.15.66:0/3595358486 conn(0x55af40b41830 msgr2=0x55af40b39860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 -- 172.21.15.66:0/3595358486 shutdown_connections 2024-03-30T00:51:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.371+0000 7f0800947000 1 -- 172.21.15.66:0/3595358486 wait complete. 2024-03-30T00:51:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 -- start start 2024-03-30T00:51:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 Processor -- start 2024-03-30T00:51:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b3b4c0 0x55af40b3fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b84db0 0x55af40b40260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b856d0 0x55af40b85ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af409899b0 con 0x55af40b84db0 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af40928c00 con 0x55af40b3b4c0 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af40947070 con 0x55af40b856d0 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 10 client.0 _ll_register_callbacks cb 0x55af40b270e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07ede00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b84db0 0x55af40b40260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07ede00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b84db0 0x55af40b40260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40448/0 (socket says 172.21.15.66:40448) 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07ede00640 1 -- 172.21.15.66:0/4099585620 learned_addr learned my addr 172.21.15.66:0/4099585620 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07f6800640 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b3b4c0 0x55af40b3fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07f7200640 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b856d0 0x55af40b85ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07ede00640 1 -- 172.21.15.66:0/4099585620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b856d0 msgr2=0x55af40b85ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07ede00640 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b856d0 0x55af40b85ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07ede00640 1 -- 172.21.15.66:0/4099585620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b3b4c0 msgr2=0x55af40b3fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07ede00640 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b3b4c0 0x55af40b3fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07ede00640 1 -- 172.21.15.66:0/4099585620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e000f040 con 0x55af40b84db0 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07ede00640 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b84db0 0x55af40b40260 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f07e0013000 tx=0x7f07e0000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7205de177659f67f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07dbe00640 1 -- 172.21.15.66:0/4099585620 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0022070 con 0x55af40b84db0 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07dbe00640 1 -- 172.21.15.66:0/4099585620 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f07e0005480 con 0x55af40b84db0 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 -- 172.21.15.66:0/4099585620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af409465f0 con 0x55af40b84db0 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07dbe00640 1 -- 172.21.15.66:0/4099585620 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07e000e8f0 con 0x55af40b84db0 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 10 client.10026 fetch_fsmap learned FSMap version 108 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 -- 172.21.15.66:0/4099585620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x55af4094eab0 con 0x55af40b84db0 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07dbe00640 1 -- 172.21.15.66:0/4099585620 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af409465f0 con 0x55af40b84db0 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f07dbe00640 1 -- 172.21.15.66:0/4099585620 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 108) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f07e0025020 con 0x55af40b84db0 2024-03-30T00:51:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 10 client.10026 fetch_fsmap finished waiting for FSMap version 108 2024-03-30T00:51:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 10 client.10026 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:51:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 20 client.10026 populate_metadata read hostname 'smithi066' 2024-03-30T00:51:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.66:6834/2774791553,v1:172.21.15.66:6836/2774791553] conn(0x55af40b93e40 0x55af40b96230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 4 client.10026 mds_command: new command op to 9942 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:51:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.375+0000 7f0800947000 1 -- 172.21.15.66:0/4099585620 --> [v2:172.21.15.66:6834/2774791553,v1:172.21.15.66:6836/2774791553] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af40986790 con 0x55af40b93e40 2024-03-30T00:51:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f07f6800640 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.66:6834/2774791553,v1:172.21.15.66:6836/2774791553] conn(0x55af40b93e40 0x55af40b96230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f07dbe00640 10 client.10026 ms_handle_connect on v2:172.21.15.66:6834/2774791553 2024-03-30T00:51:36.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f07f6800640 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.66:6834/2774791553,v1:172.21.15.66:6836/2774791553] conn(0x55af40b93e40 0x55af40b96230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a11c4fe88f15c528 server_cookie=5fbd8e5faca7b08e in_seq=0 out_seq=0 2024-03-30T00:51:36.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f07dbe00640 1 -- 172.21.15.66:0/4099585620 <== mds.0 v2:172.21.15.66:6834/2774791553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55af40986790 con 0x55af40b93e40 2024-03-30T00:51:36.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f07dbe00640 10 client.10026 handle_command_reply: tid=0 2024-03-30T00:51:36.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 1 client.10026 shutdown 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 1 -- 172.21.15.66:0/4099585620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b84db0 msgr2=0x55af40b40260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b84db0 0x55af40b40260 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f07e0013000 tx=0x7f07e0000ee0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 1 -- 172.21.15.66:0/4099585620 shutdown_connections 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.66:6834/2774791553,v1:172.21.15.66:6836/2774791553] conn(0x55af40b93e40 0x55af40b96230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b856d0 0x55af40b85ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b84db0 0x55af40b40260 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 1 --2- 172.21.15.66:0/4099585620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b3b4c0 0x55af40b3fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f07dbe00640 0 client.10026 ms_handle_reset on v2:172.21.15.66:6834/2774791553 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 1 -- 172.21.15.66:0/4099585620 >> 172.21.15.66:0/4099585620 conn(0x55af40b41830 msgr2=0x55af40b39f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 1 -- 172.21.15.66:0/4099585620 shutdown_connections 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 1 -- 172.21.15.66:0/4099585620 wait complete. 2024-03-30T00:51:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.379+0000 7f0800947000 20 client.10026 trim_cache size 0 max 16384 2024-03-30T00:51:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 Processor -- start 2024-03-30T00:51:36.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 -- start start 2024-03-30T00:51:36.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40b837a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b84db0 0x55af40b83ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b856d0 0x55af40b94220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af40646940 con 0x55af40b3b4c0 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4098af40 con 0x55af40b856d0 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af409ccac0 con 0x55af40b84db0 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07f6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40b837a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07f6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40b837a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40464/0 (socket says 172.21.15.66:40464) 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07f6800640 1 -- 172.21.15.66:0/523564600 learned_addr learned my addr 172.21.15.66:0/523564600 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07ede00640 1 --2- 172.21.15.66:0/523564600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b84db0 0x55af40b83ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07f7200640 1 --2- 172.21.15.66:0/523564600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b856d0 0x55af40b94220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07f6800640 1 -- 172.21.15.66:0/523564600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b84db0 msgr2=0x55af40b83ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07f6800640 1 --2- 172.21.15.66:0/523564600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b84db0 0x55af40b83ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07f6800640 1 -- 172.21.15.66:0/523564600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b856d0 msgr2=0x55af40b94220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:36.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07f6800640 1 --2- 172.21.15.66:0/523564600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b856d0 0x55af40b94220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07f6800640 1 -- 172.21.15.66:0/523564600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af40934c00 con 0x55af40b3b4c0 2024-03-30T00:51:36.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07f6800640 1 --2- 172.21.15.66:0/523564600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40b837a0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x55af4098aef0 tx=0x7f07e8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1affa7f2a0d84297 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:36.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07e7e00640 1 -- 172.21.15.66:0/523564600 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8028070 con 0x55af40b3b4c0 2024-03-30T00:51:36.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07e7e00640 1 -- 172.21.15.66:0/523564600 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e801c040 con 0x55af40b3b4c0 2024-03-30T00:51:36.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f07e7e00640 1 -- 172.21.15.66:0/523564600 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8015650 con 0x55af40b3b4c0 2024-03-30T00:51:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 -- 172.21.15.66:0/523564600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 msgr2=0x55af40b837a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 --2- 172.21.15.66:0/523564600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40b837a0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x55af4098aef0 tx=0x7f07e8015000 comp rx=0 tx=0).stop 2024-03-30T00:51:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 -- 172.21.15.66:0/523564600 shutdown_connections 2024-03-30T00:51:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 --2- 172.21.15.66:0/523564600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b856d0 0x55af40b94220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 --2- 172.21.15.66:0/523564600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b84db0 0x55af40b83ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 --2- 172.21.15.66:0/523564600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40b837a0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.395+0000 7f0800947000 1 -- 172.21.15.66:0/523564600 >> 172.21.15.66:0/523564600 conn(0x55af40b41830 msgr2=0x55af40b3f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 -- 172.21.15.66:0/523564600 shutdown_connections 2024-03-30T00:51:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 -- 172.21.15.66:0/523564600 wait complete. 2024-03-30T00:51:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 -- start start 2024-03-30T00:51:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 Processor -- start 2024-03-30T00:51:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40be3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b84db0 0x55af40be38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b856d0 0x55af40b96a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af409c9260 con 0x55af40b3b4c0 2024-03-30T00:51:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af40926c10 con 0x55af40b84db0 2024-03-30T00:51:36.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af40946320 con 0x55af40b856d0 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 10 client.0 _ll_register_callbacks cb 0x55af4094e590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07f6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40be3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07f6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40be3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40474/0 (socket says 172.21.15.66:40474) 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07f6800640 1 -- 172.21.15.66:0/3783228496 learned_addr learned my addr 172.21.15.66:0/3783228496 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07f7200640 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b856d0 0x55af40b96a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07ede00640 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b84db0 0x55af40be38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07f6800640 1 -- 172.21.15.66:0/3783228496 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b856d0 msgr2=0x55af40b96a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07f6800640 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b856d0 0x55af40b96a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07f6800640 1 -- 172.21.15.66:0/3783228496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b84db0 msgr2=0x55af40be38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07f6800640 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b84db0 0x55af40be38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07f6800640 1 -- 172.21.15.66:0/3783228496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e800e040 con 0x55af40b3b4c0 2024-03-30T00:51:36.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07f6800640 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40be3380 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x55af40935980 tx=0x7f07e8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b5c76b8d995512 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07dbe00640 1 -- 172.21.15.66:0/3783228496 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8005ba0 con 0x55af40b3b4c0 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07dbe00640 1 -- 172.21.15.66:0/3783228496 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f07e8005d00 con 0x55af40b3b4c0 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 -- 172.21.15.66:0/3783228496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af40959460 con 0x55af40b3b4c0 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07dbe00640 1 -- 172.21.15.66:0/3783228496 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07e8006040 con 0x55af40b3b4c0 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 10 client.10032 fetch_fsmap learned FSMap version 108 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 -- 172.21.15.66:0/3783228496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x55af40685cc0 con 0x55af40b3b4c0 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07dbe00640 1 -- 172.21.15.66:0/3783228496 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af40959460 con 0x55af40b3b4c0 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07dbe00640 1 -- 172.21.15.66:0/3783228496 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 108) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f07e8021040 con 0x55af40b3b4c0 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 10 client.10032 fetch_fsmap finished waiting for FSMap version 108 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 10 client.10032 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 20 client.10032 populate_metadata read hostname 'smithi066' 2024-03-30T00:51:36.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.66:6834/2774791553,v1:172.21.15.66:6836/2774791553] conn(0x55af40bea250 0x55af40bec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 4 client.10032 mds_command: new command op to 9942 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:51:36.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f0800947000 1 -- 172.21.15.66:0/3783228496 --> [v2:172.21.15.66:6834/2774791553,v1:172.21.15.66:6836/2774791553] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55af409529c0 con 0x55af40bea250 2024-03-30T00:51:36.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.399+0000 7f07ede00640 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.66:6834/2774791553,v1:172.21.15.66:6836/2774791553] conn(0x55af40bea250 0x55af40bec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07ede00640 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.66:6834/2774791553,v1:172.21.15.66:6836/2774791553] conn(0x55af40bea250 0x55af40bec640 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f0c0f96c582c2c server_cookie=13bd1625f6fe2ee2 in_seq=0 out_seq=0 2024-03-30T00:51:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07dbe00640 10 client.10032 ms_handle_connect on v2:172.21.15.66:6834/2774791553 2024-03-30T00:51:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07dbe00640 1 -- 172.21.15.66:0/3783228496 <== mds.0 v2:172.21.15.66:6834/2774791553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55af409529c0 con 0x55af40bea250 2024-03-30T00:51:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07dbe00640 10 client.10032 handle_command_reply: tid=0 2024-03-30T00:51:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 1 client.10032 shutdown 2024-03-30T00:51:36.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 1 -- 172.21.15.66:0/3783228496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 msgr2=0x55af40be3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40be3380 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x55af40935980 tx=0x7f07e8004740 comp rx=0 tx=0).stop 2024-03-30T00:51:36.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 1 -- 172.21.15.66:0/3783228496 shutdown_connections 2024-03-30T00:51:36.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.66:6834/2774791553,v1:172.21.15.66:6836/2774791553] conn(0x55af40bea250 0x55af40bec640 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55af40b856d0 0x55af40b96a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af40b84db0 0x55af40be38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 1 --2- 172.21.15.66:0/3783228496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af40b3b4c0 0x55af40be3380 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07dbe00640 0 client.10032 ms_handle_reset on v2:172.21.15.66:6834/2774791553 2024-03-30T00:51:36.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 1 -- 172.21.15.66:0/3783228496 >> 172.21.15.66:0/3783228496 conn(0x55af40b41830 msgr2=0x55af40b3fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:36.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 1 -- 172.21.15.66:0/3783228496 shutdown_connections 2024-03-30T00:51:36.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 1 -- 172.21.15.66:0/3783228496 wait complete. 2024-03-30T00:51:36.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f0800947000 20 client.10032 trim_cache size 0 max 16384 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "0aec3283-d125-4c04-9eb7-d2c2f49c35a9", 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 9942, 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 4.5741923929999997, 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 108, 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 127, 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 127, 2024-03-30T00:51:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 6.531743144 2024-03-30T00:51:36.417 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 -- 172.21.15.66:0/4237364494 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07d4066690 msgr2=0x7f07d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07d4066690 0x7f07d4068b50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f07f8165470 tx=0x7f07e8008040 comp rx=0 tx=0).stop 2024-03-30T00:51:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 -- 172.21.15.66:0/4237364494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f815a370 msgr2=0x7f07f8163a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f815a370 0x7f07f8163a20 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f07f00af040 tx=0x7f07f0019700 comp rx=0 tx=0).stop 2024-03-30T00:51:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 -- 172.21.15.66:0/4237364494 shutdown_connections 2024-03-30T00:51:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07d4066690 0x7f07d4068b50 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f815a370 0x7f07f8163a20 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f80cde70 0x7f07f81634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 --2- 172.21.15.66:0/4237364494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f80cda90 0x7f07f8167b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 -- 172.21.15.66:0/4237364494 >> 172.21.15.66:0/4237364494 conn(0x7f07f8069f60 msgr2=0x7f07f8159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 -- 172.21.15.66:0/4237364494 shutdown_connections 2024-03-30T00:51:36.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.403+0000 7f07fd400640 1 -- 172.21.15.66:0/4237364494 wait complete. 2024-03-30T00:51:36.430 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T00:51:36.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dc5200640 1 Processor -- start 2024-03-30T00:51:36.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dc5200640 1 -- start start 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dc5200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9dc006f8e0 0x7f9dc00d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dc5200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc006b0b0 con 0x7f9dc006e800 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dc5200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc006b390 con 0x7f9dc00cc560 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dc5200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc0160640 con 0x7f9dc006f8e0 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9db5400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9dc006e800 0x7f9dc00ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:59634/0 (socket says 172.21.15.66:59634) 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9db5400640 1 -- 172.21.15.66:0/1533696166 learned_addr learned my addr 172.21.15.66:0/1533696166 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dbe800640 1 --2- 172.21.15.66:0/1533696166 >> v2:172.21.15.156:3300/0 conn(0x7f9dc006f8e0 0x7f9dc00d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dbd400640 1 -- 172.21.15.66:0/1533696166 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421657411 0 0) 0x7f9dc006b0b0 con 0x7f9dc006e800 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dbd400640 1 -- 172.21.15.66:0/1533696166 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da4003530 con 0x7f9dc006e800 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dbe800640 1 -- 172.21.15.66:0/1533696166 >> v1:172.21.15.156:6789/0 conn(0x7f9dc00cc560 legacy=0x7f9dc006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dbe800640 1 -- 172.21.15.66:0/1533696166 >> v1:172.21.15.66:6789/0 conn(0x7f9dc006e800 legacy=0x7f9dc00ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.563+0000 7f9dbe800640 1 -- 172.21.15.66:0/1533696166 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc006b670 con 0x7f9dc006f8e0 2024-03-30T00:51:36.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.567+0000 7f9dbe800640 1 --2- 172.21.15.66:0/1533696166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc006f8e0 0x7f9dc00d1060 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097c0 tx=0x7f9db801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1596deb471cafce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.567+0000 7f9dbd400640 1 -- 172.21.15.66:0/1533696166 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db80ab670 con 0x7f9dc006f8e0 2024-03-30T00:51:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.567+0000 7f9dbd400640 1 -- 172.21.15.66:0/1533696166 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db80ab7d0 con 0x7f9dc006f8e0 2024-03-30T00:51:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.567+0000 7f9dbd400640 1 -- 172.21.15.66:0/1533696166 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db80abb40 con 0x7f9dc006f8e0 2024-03-30T00:51:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.567+0000 7f9dc5200640 1 -- 172.21.15.66:0/1533696166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc006f8e0 msgr2=0x7f9dc00d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.567+0000 7f9dc5200640 1 --2- 172.21.15.66:0/1533696166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc006f8e0 0x7f9dc00d1060 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097c0 tx=0x7f9db801c470 comp rx=0 tx=0).stop 2024-03-30T00:51:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.567+0000 7f9dc5200640 1 -- 172.21.15.66:0/1533696166 shutdown_connections 2024-03-30T00:51:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.567+0000 7f9dc5200640 1 --2- 172.21.15.66:0/1533696166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc006f8e0 0x7f9dc00d1060 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.567+0000 7f9dc5200640 1 -- 172.21.15.66:0/1533696166 >> 172.21.15.66:0/1533696166 conn(0x7f9dc00d0540 msgr2=0x7f9dc00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.567+0000 7f9dc5200640 1 -- 172.21.15.66:0/1533696166 shutdown_connections 2024-03-30T00:51:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.567+0000 7f9dc5200640 1 -- 172.21.15.66:0/1533696166 wait complete. 2024-03-30T00:51:36.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dc5200640 1 Processor -- start 2024-03-30T00:51:36.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dc5200640 1 -- start start 2024-03-30T00:51:36.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dc5200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9dc006e800 0x7f9dc00c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dc5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc006f8e0 0x7f9dc00c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dc5200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc00cc560 0x7f9dc0157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dc5200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00b0310 con 0x7f9dc006f8e0 2024-03-30T00:51:36.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dc5200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00b05f0 con 0x7f9dc00cc560 2024-03-30T00:51:36.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dc5200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00b08d0 con 0x7f9dc006e800 2024-03-30T00:51:36.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc006f8e0 0x7f9dc00c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db5400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc006f8e0 0x7f9dc00c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40480/0 (socket says 172.21.15.66:40480) 2024-03-30T00:51:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db5400640 1 -- 172.21.15.66:0/3002068612 learned_addr learned my addr 172.21.15.66:0/3002068612 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dbde00640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9dc006e800 0x7f9dc00c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dbe800640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc00cc560 0x7f9dc0157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db5400640 1 -- 172.21.15.66:0/3002068612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9dc006e800 msgr2=0x7f9dc00c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db5400640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9dc006e800 0x7f9dc00c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db5400640 1 -- 172.21.15.66:0/3002068612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc00cc560 msgr2=0x7f9dc0157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db5400640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc00cc560 0x7f9dc0157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db5400640 1 -- 172.21.15.66:0/3002068612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db8009420 con 0x7f9dc006f8e0 2024-03-30T00:51:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db5400640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc006f8e0 0x7f9dc00c43b0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f9da800cbc0 tx=0x7f9da800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:36.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db6a00640 1 -- 172.21.15.66:0/3002068612 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da800ace0 con 0x7f9dc006f8e0 2024-03-30T00:51:36.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db6a00640 1 -- 172.21.15.66:0/3002068612 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da800ae40 con 0x7f9dc006f8e0 2024-03-30T00:51:36.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9db6a00640 1 -- 172.21.15.66:0/3002068612 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8012620 con 0x7f9dc006f8e0 2024-03-30T00:51:36.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dc5200640 1 -- 172.21.15.66:0/3002068612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc00b0bb0 con 0x7f9dc006f8e0 2024-03-30T00:51:36.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.571+0000 7f9dc5200640 1 -- 172.21.15.66:0/3002068612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc00c6e40 con 0x7f9dc006f8e0 2024-03-30T00:51:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.575+0000 7f9dc5200640 1 -- 172.21.15.66:0/3002068612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc0066c30 con 0x7f9dc006f8e0 2024-03-30T00:51:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.575+0000 7f9db6a00640 1 -- 172.21.15.66:0/3002068612 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9da8007070 con 0x7f9dc006f8e0 2024-03-30T00:51:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.575+0000 7f9db6a00640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d9c0666c0 0x7f9d9c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.575+0000 7f9db6a00640 1 -- 172.21.15.66:0/3002068612 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f9da80843a0 con 0x7f9dc006f8e0 2024-03-30T00:51:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.575+0000 7f9dbde00640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d9c0666c0 0x7f9d9c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.575+0000 7f9dbde00640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d9c0666c0 0x7f9d9c068b80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9db0000f70 tx=0x7f9db0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:36.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.579+0000 7f9db6a00640 1 -- 172.21.15.66:0/3002068612 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9da8051480 con 0x7f9dc006f8e0 2024-03-30T00:51:36.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.779+0000 7f9dc5200640 1 -- 172.21.15.66:0/3002068612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9dc006e190 con 0x7f9dc006f8e0 2024-03-30T00:51:36.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.779+0000 7f9db6a00640 1 -- 172.21.15.66:0/3002068612 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v108) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9da8054aa0 con 0x7f9dc006f8e0 2024-03-30T00:51:36.788 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:51:36.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 -- 172.21.15.66:0/3002068612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d9c0666c0 msgr2=0x7f9d9c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d9c0666c0 0x7f9d9c068b80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9db0000f70 tx=0x7f9db0009040 comp rx=0 tx=0).stop 2024-03-30T00:51:36.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 -- 172.21.15.66:0/3002068612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc006f8e0 msgr2=0x7f9dc00c43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:36.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc006f8e0 0x7f9dc00c43b0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f9da800cbc0 tx=0x7f9da800a430 comp rx=0 tx=0).stop 2024-03-30T00:51:36.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 -- 172.21.15.66:0/3002068612 shutdown_connections 2024-03-30T00:51:36.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc00cc560 0x7f9dc0157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d9c0666c0 0x7f9d9c068b80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc006f8e0 0x7f9dc00c43b0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 --2- 172.21.15.66:0/3002068612 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9dc006e800 0x7f9dc00c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:36.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 -- 172.21.15.66:0/3002068612 >> 172.21.15.66:0/3002068612 conn(0x7f9dc00d0540 msgr2=0x7f9dc015f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:36.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 -- 172.21.15.66:0/3002068612 shutdown_connections 2024-03-30T00:51:36.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:36.783+0000 7f9dc5200640 1 -- 172.21.15.66:0/3002068612 wait complete. 2024-03-30T00:51:36.810 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:51:36.810 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:51:36.810 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T00:51:36.811 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:51:36.811 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:36.811 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:51:36.811 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:51:36.811 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:51:36.811 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:51:36.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:36.811 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T00:51:36.821 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:51:36.821 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:51:36.821 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:51:36.821 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T00:51:36.822 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41781sec preferred_lft 41781sec 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:42:15:92:5a:bf brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:36.823 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:51:36.824 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:51:36.824 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T00:51:36.824 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:36.824 INFO:teuthology.orchestra.run.smithi066.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T00:51:36.824 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:51:36.824 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T00:51:36.824 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:51:36.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:36.824 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:51:36.883 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:51:36.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:36.885 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T00:51:36.941 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T00:51:36.941 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:51:36.942 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T00:51:36.942 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:51:36.942 INFO:teuthology.orchestra.run.smithi066.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T00:51:36.942 INFO:teuthology.orchestra.run.smithi066.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T00:51:36.942 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:51:36.942 INFO:teuthology.orchestra.run.smithi066.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:51:36.942 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:51:36.942 INFO:teuthology.orchestra.run.smithi066.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T00:51:36.942 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:51:36.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:36.943 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2024-03-30T00:51:36.943 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T00:51:36.943 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:51:36.943 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:51:36.943 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:51:36.943 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T00:51:37.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:37.035 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:51:37.044 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:51:37.044 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:51:37.045 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:51:37.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:37.045 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T00:51:37.105 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T00:51:37.106 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:51:37.106 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:37.158 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:51:37.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:51:37.158 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:37.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:51:37.269 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:51:37.270 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:51:37.331 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:51:37.333 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:51:37.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:37.333 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:51:37.385 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:51:37.385 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T00:51:37.436 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T00:51:37.456 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:51:37.457 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:51:37.515 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:51:37.517 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:51:37.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:37.518 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T00:51:37.532 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2024-03-30T00:51:37.523+0000 7f24f1a47580 -1 init, newargv = 0x562aebf9df60 newargc=15 2024-03-30T00:51:37.532 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[55825]: starting ceph client 2024-03-30T00:51:37.539 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[55825]: starting fuse 2024-03-30T00:51:37.543 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-03-30T00:51:37.543 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-30T00:51:37.543 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:37.543 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T00:51:37.544 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:51:37.545 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:51:37.545 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:51:37.545 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T00:51:37.545 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T00:51:37.545 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T00:51:37.545 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:51:37.545 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:37.545 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2024-03-30T00:51:37.545 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:37.670 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.55825.asok 2024-03-30T00:51:37.854 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55825.asok 2024-03-30T00:51:37.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:37.854 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55825.asok status 2024-03-30T00:51:38.029 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:38.029 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T00:51:38.029 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:51:38.029 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:51:38.029 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2024-03-30T00:51:38.029 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T00:51:38.029 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:51:38.029 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "55825", 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 10041, 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 10041 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:51:38.030 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3751427317 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3751427317 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.10041 192.168.0.1:0/3751427317", 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/3751427317", 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 108, 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 127, 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T00:51:38.031 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T00:51:38.032 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T00:51:38.032 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:38.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:38.044 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T00:51:38.044 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:38.053 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T00:51:38.053 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:38.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:38.054 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:38.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:38.119 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-30T00:51:38.187 INFO:teuthology.orchestra.run.smithi066.stdout:./ 2024-03-30T00:51:38.187 INFO:teuthology.orchestra.run.smithi066.stdout:./parent 2024-03-30T00:51:38.187 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/flushed 2024-03-30T00:51:38.187 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/flushed/charlie 2024-03-30T00:51:38.187 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/flushed/alpha 2024-03-30T00:51:38.187 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/flushed/bravo 2024-03-30T00:51:38.187 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/unflushed 2024-03-30T00:51:38.187 INFO:teuthology.orchestra.run.smithi066.stdout:./parent/unflushed/jfile 2024-03-30T00:51:38.188 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:38.189 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:38.189 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:38.189 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:38.189 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-30T00:51:38.189 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:38.293 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T00:51:38.483 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:38.520 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:38.520 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:38.520 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:38.520 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-30T00:51:38.520 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:38.598 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627776 2024-03-30T00:51:38.787 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:38.787 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:38.787 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:38.787 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:38.787 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-30T00:51:38.787 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:38.899 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627777 2024-03-30T00:51:39.087 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:39.087 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:39.087 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:39.087 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:39.087 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-30T00:51:39.087 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:39.193 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628280 2024-03-30T00:51:39.383 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:39.383 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:39.383 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:39.383 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:39.383 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-30T00:51:39.383 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:39.490 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627778 2024-03-30T00:51:39.679 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:39.680 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:39.680 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:39.680 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:39.680 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-03-30T00:51:39.680 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:39.790 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628279 2024-03-30T00:51:39.979 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:39.979 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:39.979 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:39.979 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:39.979 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-30T00:51:39.979 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:40.086 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627779 2024-03-30T00:51:40.275 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:51:40.275 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T00:51:40.275 DEBUG:teuthology.orchestra.run.smithi066:> import stat 2024-03-30T00:51:40.275 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T00:51:40.275 DEBUG:teuthology.orchestra.run.smithi066:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-30T00:51:40.275 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T00:51:40.382 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627780 2024-03-30T00:51:40.571 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T00:51:40.572 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:51:40.700 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T00:51:40.783 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T00:51:40.783 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:51:40.784 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:40.793 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:51:40.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1920a00640 1 Processor -- start 2024-03-30T00:51:40.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1920a00640 1 -- start start 2024-03-30T00:51:40.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1920a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f191c15e5d0 0x7f191c0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:40.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1920a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f191c15ae20 0x7f191c0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:40.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1920a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191c15eab0 con 0x7f191c15aa40 2024-03-30T00:51:40.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1920a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191c0cec50 con 0x7f191c15e5d0 2024-03-30T00:51:40.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1920a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c15ed90 con 0x7f191c15ae20 2024-03-30T00:51:40.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f191a800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f191c15aa40 0x7f191c0ce540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40158/0 (socket says 172.21.15.66:40158) 2024-03-30T00:51:40.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f191a800640 1 -- 172.21.15.66:0/1944622989 learned_addr learned my addr 172.21.15.66:0/1944622989 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:40.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805459061 0 0) 0x7f191c15eab0 con 0x7f191c15aa40 2024-03-30T00:51:40.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f191b200640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f191c15ae20 0x7f191c0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919e00640 1 --2- 172.21.15.66:0/1944622989 >> v2:172.21.15.156:3300/0 conn(0x7f191c15e5d0 0x7f191c0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1900003530 con 0x7f191c15aa40 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 633122421 0 0) 0x7f1900003530 con 0x7f191c15aa40 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f191c15ae20 msgr2=0x7f191c0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f191c15ae20 0x7f191c0ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 >> v2:172.21.15.156:3300/0 conn(0x7f191c15e5d0 msgr2=0x7f191c0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 --2- 172.21.15.66:0/1944622989 >> v2:172.21.15.156:3300/0 conn(0x7f191c15e5d0 0x7f191c0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c15f070 con 0x7f191c15aa40 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f190c001680 con 0x7f191c15aa40 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 >> v1:172.21.15.66:6789/0 conn(0x7f191c15aa40 legacy=0x7f191c0ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1900003980 0x7f1900003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19000047d0 0x7f1900004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f191c15ae20 0x7f1900005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0cec50 con 0x7f1900003980 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c15ed90 con 0x7f19000047d0 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190000ba10 con 0x7f191c15ae20 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f191a800640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1900003980 0x7f1900003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f191b200640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f191c15ae20 0x7f1900005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f1919e00640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19000047d0 0x7f1900004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f191a800640 1 -- 172.21.15.66:0/1944622989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f191c15ae20 msgr2=0x7f1900005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f191a800640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f191c15ae20 0x7f1900005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f191a800640 1 -- 172.21.15.66:0/1944622989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19000047d0 msgr2=0x7f1900004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f191a800640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19000047d0 0x7f1900004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.971+0000 7f191a800640 1 -- 172.21.15.66:0/1944622989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1900003530 con 0x7f1900003980 2024-03-30T00:51:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.975+0000 7f191a800640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1900003980 0x7f1900003da0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c110 tx=0x7f190c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ad03de38ff563f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.975+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c0058e0 con 0x7f1900003980 2024-03-30T00:51:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.975+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c005a60 con 0x7f1900003980 2024-03-30T00:51:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.975+0000 7f1919400640 1 -- 172.21.15.66:0/1944622989 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c005e10 con 0x7f1900003980 2024-03-30T00:51:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.975+0000 7f1920a00640 1 -- 172.21.15.66:0/1944622989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1900003980 msgr2=0x7f1900003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.975+0000 7f1920a00640 1 --2- 172.21.15.66:0/1944622989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1900003980 0x7f1900003da0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c110 tx=0x7f190c01cbc0 comp rx=0 tx=0).stop 2024-03-30T00:51:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.975+0000 7f191b200640 1 -- 172.21.15.66:0/1944622989 reap_dead start 2024-03-30T00:51:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.975+0000 7f1920a00640 1 -- 172.21.15.66:0/1944622989 shutdown_connections 2024-03-30T00:51:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.975+0000 7f1920a00640 1 -- 172.21.15.66:0/1944622989 >> 172.21.15.66:0/1944622989 conn(0x7f191c06e0d0 msgr2=0x7f191c06e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:40.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.975+0000 7f1920a00640 1 -- 172.21.15.66:0/1944622989 shutdown_connections 2024-03-30T00:51:40.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.975+0000 7f1920a00640 1 -- 172.21.15.66:0/1944622989 wait complete. 2024-03-30T00:51:40.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1920a00640 1 Processor -- start 2024-03-30T00:51:40.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1920a00640 1 -- start start 2024-03-30T00:51:40.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1920a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1900003980 0x7f191c155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1920a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f191c0b00f0 0x7f191c0b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1920a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f191c0b4710 0x7f191c0b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1920a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0b50d0 con 0x7f191c0b00f0 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1920a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0b1870 con 0x7f191c0b4710 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1920a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0b0c60 con 0x7f1900003980 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1919e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f191c0b00f0 0x7f191c0b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1919e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f191c0b00f0 0x7f191c0b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37946/0 (socket says 172.21.15.66:37946) 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1919e00640 1 -- 172.21.15.66:0/3377431381 learned_addr learned my addr 172.21.15.66:0/3377431381 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f191a800640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1900003980 0x7f191c155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f191b200640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f191c0b4710 0x7f191c0b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1919e00640 1 -- 172.21.15.66:0/3377431381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1900003980 msgr2=0x7f191c155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1919e00640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1900003980 0x7f191c155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1919e00640 1 -- 172.21.15.66:0/3377431381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f191c0b4710 msgr2=0x7f191c0b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1919e00640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f191c0b4710 0x7f191c0b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1919e00640 1 -- 172.21.15.66:0/3377431381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c0b0f40 con 0x7f191c0b00f0 2024-03-30T00:51:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1919e00640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f191c0b00f0 0x7f191c0b0570 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f1908011940 tx=0x7f1908011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1912a00640 1 -- 172.21.15.66:0/3377431381 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908004240 con 0x7f191c0b00f0 2024-03-30T00:51:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1912a00640 1 -- 172.21.15.66:0/3377431381 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19080043a0 con 0x7f191c0b00f0 2024-03-30T00:51:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1912a00640 1 -- 172.21.15.66:0/3377431381 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190800fb00 con 0x7f191c0b00f0 2024-03-30T00:51:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1920a00640 1 -- 172.21.15.66:0/3377431381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191c0b1220 con 0x7f191c0b00f0 2024-03-30T00:51:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1920a00640 1 -- 172.21.15.66:0/3377431381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191c0c4270 con 0x7f191c0b00f0 2024-03-30T00:51:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.979+0000 7f1920a00640 1 -- 172.21.15.66:0/3377431381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191c073cf0 con 0x7f191c0b00f0 2024-03-30T00:51:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.983+0000 7f1912a00640 1 -- 172.21.15.66:0/3377431381 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f190800fc60 con 0x7f191c0b00f0 2024-03-30T00:51:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.983+0000 7f1912a00640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18fc0666c0 0x7f18fc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.983+0000 7f1912a00640 1 -- 172.21.15.66:0/3377431381 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f19080849b0 con 0x7f191c0b00f0 2024-03-30T00:51:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.983+0000 7f191a800640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18fc0666c0 0x7f18fc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:40.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.983+0000 7f191a800640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18fc0666c0 0x7f18fc068b80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c110 tx=0x7f190c0019b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:40.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:40.987+0000 7f1912a00640 1 -- 172.21.15.66:0/3377431381 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1908051b40 con 0x7f191c0b00f0 2024-03-30T00:51:41.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.147+0000 7f1920a00640 1 -- 172.21.15.66:0/3377431381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f191c0af8e0 con 0x7f191c0b00f0 2024-03-30T00:51:41.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1912a00640 1 -- 172.21.15.66:0/3377431381 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16575 (secure 0 0 0) 0x7f1908023090 con 0x7f191c0b00f0 2024-03-30T00:51:41.159 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:41.159 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":127,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:51:29.804902+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6834/3460522956":"2024-03-31T00:51:29.804880+0000","172.21.15.66:6836/3460522956":"2024-03-31T00:51:29.804880+0000","172.21.15.66:6836/2683418851":"2024-03-31T00:51:15.162460+0000","172.21.15.66:6834/2683418851":"2024-03-31T00:51:15.162460+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:51:41.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 -- 172.21.15.66:0/3377431381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18fc0666c0 msgr2=0x7f18fc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18fc0666c0 0x7f18fc068b80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c110 tx=0x7f190c0019b0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 -- 172.21.15.66:0/3377431381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f191c0b00f0 msgr2=0x7f191c0b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f191c0b00f0 0x7f191c0b0570 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f1908011940 tx=0x7f1908011e00 comp rx=0 tx=0).stop 2024-03-30T00:51:41.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 -- 172.21.15.66:0/3377431381 shutdown_connections 2024-03-30T00:51:41.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f191c0b4710 0x7f191c0b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f18fc0666c0 0x7f18fc068b80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f191c0b00f0 0x7f191c0b0570 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 --2- 172.21.15.66:0/3377431381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1900003980 0x7f191c155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 -- 172.21.15.66:0/3377431381 >> 172.21.15.66:0/3377431381 conn(0x7f191c06e0d0 msgr2=0x7f191c06b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:41.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 -- 172.21.15.66:0/3377431381 shutdown_connections 2024-03-30T00:51:41.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.151+0000 7f1920a00640 1 -- 172.21.15.66:0/3377431381 wait complete. 2024-03-30T00:51:41.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:50:35.696965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:50:37.037165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:51:41.180 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:51:41.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.315+0000 7fd1c7e00640 1 Processor -- start 2024-03-30T00:51:41.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.315+0000 7fd1c7e00640 1 -- start start 2024-03-30T00:51:41.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1c00cad00 0x7fd1c00c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c7e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd1c00cc8d0 0x7fd1c006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c7e00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c00c8e80 con 0x7fd1c00cd2d0 2024-03-30T00:51:41.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c7e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c00cbec0 con 0x7fd1c00cad00 2024-03-30T00:51:41.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c7e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c00ca660 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c5400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1c00cad00 0x7fd1c00c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:41.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c5400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1c00cad00 0x7fd1c00c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37962/0 (socket says 172.21.15.66:37962) 2024-03-30T00:51:41.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c5400640 1 -- 172.21.15.66:0/1123128249 learned_addr learned my addr 172.21.15.66:0/1123128249 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:41.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c5400640 1 -- 172.21.15.66:0/1123128249 >> v2:172.21.15.156:3300/0 conn(0x7fd1c00cc8d0 msgr2=0x7fd1c006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:41.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c5400640 1 --2- 172.21.15.66:0/1123128249 >> v2:172.21.15.156:3300/0 conn(0x7fd1c00cc8d0 0x7fd1c006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c5400640 1 -- 172.21.15.66:0/1123128249 >> v1:172.21.15.156:6789/0 conn(0x7fd1c00cd2d0 legacy=0x7fd1c00cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:41.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.319+0000 7fd1c5400640 1 -- 172.21.15.66:0/1123128249 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c00ca940 con 0x7fd1c00cad00 2024-03-30T00:51:41.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c5400640 1 --2- 172.21.15.66:0/1123128249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c00cad00 0x7fd1c00c8940 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd1c006e4b0 tx=0x7fd1b401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acfe52d62604ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:41.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1bfe00640 1 -- 172.21.15.66:0/1123128249 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b40ac600 con 0x7fd1c00cad00 2024-03-30T00:51:41.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1bfe00640 1 -- 172.21.15.66:0/1123128249 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b40a8050 con 0x7fd1c00cad00 2024-03-30T00:51:41.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1bfe00640 1 -- 172.21.15.66:0/1123128249 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b40ac980 con 0x7fd1c00cad00 2024-03-30T00:51:41.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 -- 172.21.15.66:0/1123128249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c00cad00 msgr2=0x7fd1c00c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 --2- 172.21.15.66:0/1123128249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c00cad00 0x7fd1c00c8940 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd1c006e4b0 tx=0x7fd1b401c4b0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 -- 172.21.15.66:0/1123128249 shutdown_connections 2024-03-30T00:51:41.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 --2- 172.21.15.66:0/1123128249 >> v2:172.21.15.156:3300/0 conn(0x7fd1c00cc8d0 0x7fd1c006c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 --2- 172.21.15.66:0/1123128249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c00cad00 0x7fd1c00c8940 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 -- 172.21.15.66:0/1123128249 >> 172.21.15.66:0/1123128249 conn(0x7fd1c0069f60 msgr2=0x7fd1c006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:41.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 -- 172.21.15.66:0/1123128249 shutdown_connections 2024-03-30T00:51:41.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 -- 172.21.15.66:0/1123128249 wait complete. 2024-03-30T00:51:41.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 Processor -- start 2024-03-30T00:51:41.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 -- start start 2024-03-30T00:51:41.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c00cad00 0x7fd1c0162120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c00cc8d0 0x7fd1c00715b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1c00cd2d0 0x7fd1c00978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0156870 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c01633a0 con 0x7fd1c00cad00 2024-03-30T00:51:41.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.323+0000 7fd1c7e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0071af0 con 0x7fd1c00cd2d0 2024-03-30T00:51:41.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c00cc8d0 0x7fd1c00715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:41.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c00cc8d0 0x7fd1c00715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37966/0 (socket says 172.21.15.66:37966) 2024-03-30T00:51:41.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c4a00640 1 -- 172.21.15.66:0/349505652 learned_addr learned my addr 172.21.15.66:0/349505652 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:41.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c5400640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c00cad00 0x7fd1c0162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:41.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c5e00640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1c00cd2d0 0x7fd1c00978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c4a00640 1 -- 172.21.15.66:0/349505652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1c00cd2d0 msgr2=0x7fd1c00978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c4a00640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1c00cd2d0 0x7fd1c00978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c4a00640 1 -- 172.21.15.66:0/349505652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c00cad00 msgr2=0x7fd1c0162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c4a00640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c00cad00 0x7fd1c0162120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c4a00640 1 -- 172.21.15.66:0/349505652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b40094d0 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c5e00640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1c00cd2d0 0x7fd1c00978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c4a00640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c00cc8d0 0x7fd1c00715b0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00c400 tx=0x7fd1ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1bd600640 1 -- 172.21.15.66:0/349505652 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac003600 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1bd600640 1 -- 172.21.15.66:0/349505652 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ac003760 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1bd600640 1 -- 172.21.15.66:0/349505652 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac003b10 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c7e00640 1 -- 172.21.15.66:0/349505652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c0162790 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c7e00640 1 -- 172.21.15.66:0/349505652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c0162d50 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1c7e00640 1 -- 172.21.15.66:0/349505652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c0066c30 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1bd600640 1 -- 172.21.15.66:0/349505652 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd1ac003c70 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1bd600640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1a0066690 0x7fd1a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.327+0000 7fd1bd600640 1 -- 172.21.15.66:0/349505652 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fd1ac083530 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.331+0000 7fd1c5400640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1a0066690 0x7fd1a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:41.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.331+0000 7fd1c5400640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1a0066690 0x7fd1a0068b50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40ae000 tx=0x7fd1b40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:41.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.331+0000 7fd1bd600640 1 -- 172.21.15.66:0/349505652 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd1ac050750 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.495+0000 7fd1c7e00640 1 -- 172.21.15.66:0/349505652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1c00afb30 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1bd600640 1 -- 172.21.15.66:0/349505652 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd1ac017020 con 0x7fd1c00cc8d0 2024-03-30T00:51:41.506 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:51:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 -- 172.21.15.66:0/349505652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1a0066690 msgr2=0x7fd1a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1a0066690 0x7fd1a0068b50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40ae000 tx=0x7fd1b40a7000 comp rx=0 tx=0).stop 2024-03-30T00:51:41.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 -- 172.21.15.66:0/349505652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c00cc8d0 msgr2=0x7fd1c00715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c00cc8d0 0x7fd1c00715b0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00c400 tx=0x7fd1ac00c8c0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 -- 172.21.15.66:0/349505652 shutdown_connections 2024-03-30T00:51:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1c00cd2d0 0x7fd1c00978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd1a0066690 0x7fd1a0068b50 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1c00cc8d0 0x7fd1c00715b0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 --2- 172.21.15.66:0/349505652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c00cad00 0x7fd1c0162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 -- 172.21.15.66:0/349505652 >> 172.21.15.66:0/349505652 conn(0x7fd1c0069f60 msgr2=0x7fd1c00af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 -- 172.21.15.66:0/349505652 shutdown_connections 2024-03-30T00:51:41.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.499+0000 7fd1c7e00640 1 -- 172.21.15.66:0/349505652 wait complete. 2024-03-30T00:51:41.527 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:51:41.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01b1400640 1 Processor -- start 2024-03-30T00:51:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01b1400640 1 -- start start 2024-03-30T00:51:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01b1400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f01ac0ce520 0x7f01ac0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01b1400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01ac0cabb0 0x7f01ac0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01b1400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac06d070 con 0x7f01ac15a640 2024-03-30T00:51:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01b1400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac0cbba0 con 0x7f01ac0ce520 2024-03-30T00:51:41.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01b1400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0cee70 con 0x7f01ac0cabb0 2024-03-30T00:51:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01a3e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f01ac0ce520 0x7f01ac0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01ab200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01ac0cabb0 0x7f01ac0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01a3e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f01ac0ce520 0x7f01ac0ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:56218/0 (socket says 172.21.15.66:56218) 2024-03-30T00:51:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01a3e00640 1 -- 172.21.15.66:0/592537359 learned_addr learned my addr 172.21.15.66:0/592537359 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01a3e00640 1 -- 172.21.15.66:0/592537359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01ac0cabb0 msgr2=0x7f01ac0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01a3e00640 1 --2- 172.21.15.66:0/592537359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01ac0cabb0 0x7f01ac0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01a3e00640 1 -- 172.21.15.66:0/592537359 >> v1:172.21.15.156:6789/0 conn(0x7f01ac15a640 legacy=0x7f01ac06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01a3e00640 1 -- 172.21.15.66:0/592537359 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac0cb8c0 con 0x7f01ac0ce520 2024-03-30T00:51:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.667+0000 7f01a9e00640 1 -- 172.21.15.66:0/592537359 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883653158 0 0) 0x7f01ac06d070 con 0x7f01ac15a640 2024-03-30T00:51:41.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01a3e00640 1 --2- 172.21.15.66:0/592537359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0ce520 0x7f01ac0ce930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0194009870 tx=0x7f019401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4dc3f92dc3614bc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:41.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01a9e00640 1 -- 172.21.15.66:0/592537359 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01940a95f0 con 0x7f01ac0ce520 2024-03-30T00:51:41.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01a9e00640 1 -- 172.21.15.66:0/592537359 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01940a9750 con 0x7f01ac0ce520 2024-03-30T00:51:41.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01a9e00640 1 -- 172.21.15.66:0/592537359 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01940a9ae0 con 0x7f01ac0ce520 2024-03-30T00:51:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01b1400640 1 -- 172.21.15.66:0/592537359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0ce520 msgr2=0x7f01ac0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01b1400640 1 --2- 172.21.15.66:0/592537359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0ce520 0x7f01ac0ce930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0194009870 tx=0x7f019401c530 comp rx=0 tx=0).stop 2024-03-30T00:51:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01b1400640 1 -- 172.21.15.66:0/592537359 shutdown_connections 2024-03-30T00:51:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01b1400640 1 --2- 172.21.15.66:0/592537359 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01ac0cabb0 0x7f01ac0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01b1400640 1 --2- 172.21.15.66:0/592537359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0ce520 0x7f01ac0ce930 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01b1400640 1 -- 172.21.15.66:0/592537359 >> 172.21.15.66:0/592537359 conn(0x7f01ac069f60 msgr2=0x7f01ac06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01b1400640 1 -- 172.21.15.66:0/592537359 shutdown_connections 2024-03-30T00:51:41.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.671+0000 7f01b1400640 1 -- 172.21.15.66:0/592537359 wait complete. 2024-03-30T00:51:41.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01b1400640 1 Processor -- start 2024-03-30T00:51:41.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01b1400640 1 -- start start 2024-03-30T00:51:41.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01b1400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01ac0cabb0 0x7f01ac153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01b1400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0ce520 0x7f01ac0c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01b1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01ac15a640 0x7f01ac0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01b1400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac070500 con 0x7f01ac15a640 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01b1400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac06fed0 con 0x7f01ac0ce520 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01b1400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0c0f80 con 0x7f01ac0cabb0 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01ab200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01ac15a640 0x7f01ac0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01ab200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01ac15a640 0x7f01ac0ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37972/0 (socket says 172.21.15.66:37972) 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01ab200640 1 -- 172.21.15.66:0/1558448735 learned_addr learned my addr 172.21.15.66:0/1558448735 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01a3e00640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0ce520 0x7f01ac0c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01aa800640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01ac0cabb0 0x7f01ac153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01ab200640 1 -- 172.21.15.66:0/1558448735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01ac0cabb0 msgr2=0x7f01ac153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01ab200640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01ac0cabb0 0x7f01ac153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01ab200640 1 -- 172.21.15.66:0/1558448735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0ce520 msgr2=0x7f01ac0c2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01ab200640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0ce520 0x7f01ac0c2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01ab200640 1 -- 172.21.15.66:0/1558448735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940094d0 con 0x7f01ac15a640 2024-03-30T00:51:41.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01a3e00640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0ce520 0x7f01ac0c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:41.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01ab200640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01ac15a640 0x7f01ac0ada60 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f01a400efc0 tx=0x7f01a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01a2a00640 1 -- 172.21.15.66:0/1558448735 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4014480 con 0x7f01ac15a640 2024-03-30T00:51:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01a2a00640 1 -- 172.21.15.66:0/1558448735 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a400f040 con 0x7f01ac15a640 2024-03-30T00:51:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01a2a00640 1 -- 172.21.15.66:0/1558448735 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4014820 con 0x7f01ac15a640 2024-03-30T00:51:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01b1400640 1 -- 172.21.15.66:0/1558448735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac0c1260 con 0x7f01ac15a640 2024-03-30T00:51:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01b1400640 1 -- 172.21.15.66:0/1558448735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac153880 con 0x7f01ac15a640 2024-03-30T00:51:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.675+0000 7f01b1400640 1 -- 172.21.15.66:0/1558448735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac070670 con 0x7f01ac15a640 2024-03-30T00:51:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.679+0000 7f01a2a00640 1 -- 172.21.15.66:0/1558448735 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f01a4014980 con 0x7f01ac15a640 2024-03-30T00:51:41.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.679+0000 7f01a2a00640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01880666c0 0x7f0188068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:41.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.679+0000 7f01a2a00640 1 -- 172.21.15.66:0/1558448735 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f01a4084ea0 con 0x7f01ac15a640 2024-03-30T00:51:41.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.679+0000 7f01aa800640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01880666c0 0x7f0188068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:41.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.679+0000 7f01aa800640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01880666c0 0x7f0188068b80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f019c000f70 tx=0x7f019c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:41.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.683+0000 7f01a2a00640 1 -- 172.21.15.66:0/1558448735 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f01a4052010 con 0x7f01ac15a640 2024-03-30T00:51:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 -- 172.21.15.66:0/1558448735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01ac066c30 con 0x7f01ac15a640 2024-03-30T00:51:41.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01a2a00640 1 -- 172.21.15.66:0/1558448735 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f01a40180a0 con 0x7f01ac15a640 2024-03-30T00:51:41.851 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 -- 172.21.15.66:0/1558448735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01880666c0 msgr2=0x7f0188068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01880666c0 0x7f0188068b80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f019c000f70 tx=0x7f019c009040 comp rx=0 tx=0).stop 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 -- 172.21.15.66:0/1558448735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01ac15a640 msgr2=0x7f01ac0ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01ac15a640 0x7f01ac0ada60 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f01a400efc0 tx=0x7f01a400c490 comp rx=0 tx=0).stop 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 -- 172.21.15.66:0/1558448735 shutdown_connections 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f01880666c0 0x7f0188068b80 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01ac15a640 0x7f01ac0ada60 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0ce520 0x7f01ac0c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 --2- 172.21.15.66:0/1558448735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01ac0cabb0 0x7f01ac153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 -- 172.21.15.66:0/1558448735 >> 172.21.15.66:0/1558448735 conn(0x7f01ac069f60 msgr2=0x7f01ac0afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 -- 172.21.15.66:0/1558448735 shutdown_connections 2024-03-30T00:51:41.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:41.843+0000 7f01b1400640 1 -- 172.21.15.66:0/1558448735 wait complete. 2024-03-30T00:51:41.872 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:51:42.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c5be00640 1 Processor -- start 2024-03-30T00:51:42.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c5be00640 1 -- start start 2024-03-30T00:51:42.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c5be00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6c5415c780 0x7f6c540cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c5be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c5415cb60 0x7f6c541604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c5be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c540cd260 0x7f6c541609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c5be00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c540b1d00 con 0x7f6c5415cb60 2024-03-30T00:51:42.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c5be00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c540cff70 con 0x7f6c5415c780 2024-03-30T00:51:42.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c5be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540cd820 con 0x7f6c540cd260 2024-03-30T00:51:42.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c58a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c5415cb60 0x7f6c541604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c58a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c5415cb60 0x7f6c541604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37980/0 (socket says 172.21.15.66:37980) 2024-03-30T00:51:42.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c58a00640 1 -- 172.21.15.66:0/1206524998 learned_addr learned my addr 172.21.15.66:0/1206524998 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:42.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c59e00640 1 --2- 172.21.15.66:0/1206524998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c540cd260 0x7f6c541609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c59400640 1 --2- 172.21.15.66:0/1206524998 >> v2:172.21.15.156:3300/0 conn(0x7f6c5415c780 0x7f6c540cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c58a00640 1 -- 172.21.15.66:0/1206524998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c540cd260 msgr2=0x7f6c541609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:42.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c58a00640 1 --2- 172.21.15.66:0/1206524998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c540cd260 0x7f6c541609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c58a00640 1 -- 172.21.15.66:0/1206524998 >> v2:172.21.15.156:3300/0 conn(0x7f6c5415c780 msgr2=0x7f6c540cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:42.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c58a00640 1 --2- 172.21.15.66:0/1206524998 >> v2:172.21.15.156:3300/0 conn(0x7f6c5415c780 0x7f6c540cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.007+0000 7f6c58a00640 1 -- 172.21.15.66:0/1206524998 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c540cdb00 con 0x7f6c5415cb60 2024-03-30T00:51:42.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c58a00640 1 --2- 172.21.15.66:0/1206524998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5415cb60 0x7f6c541604a0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f6c5409c120 tx=0x7f6c4001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3db31b907d1c656 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:42.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c53e00640 1 -- 172.21.15.66:0/1206524998 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400ac620 con 0x7f6c5415cb60 2024-03-30T00:51:42.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c53e00640 1 -- 172.21.15.66:0/1206524998 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c400ac780 con 0x7f6c5415cb60 2024-03-30T00:51:42.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c53e00640 1 -- 172.21.15.66:0/1206524998 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400acb10 con 0x7f6c5415cb60 2024-03-30T00:51:42.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c5be00640 1 -- 172.21.15.66:0/1206524998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5415cb60 msgr2=0x7f6c541604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:42.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c5be00640 1 --2- 172.21.15.66:0/1206524998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5415cb60 0x7f6c541604a0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f6c5409c120 tx=0x7f6c4001c4f0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c5be00640 1 -- 172.21.15.66:0/1206524998 shutdown_connections 2024-03-30T00:51:42.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c5be00640 1 --2- 172.21.15.66:0/1206524998 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c540cd260 0x7f6c541609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c5be00640 1 --2- 172.21.15.66:0/1206524998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5415cb60 0x7f6c541604a0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c5be00640 1 --2- 172.21.15.66:0/1206524998 >> v2:172.21.15.156:3300/0 conn(0x7f6c5415c780 0x7f6c540cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c5be00640 1 -- 172.21.15.66:0/1206524998 >> 172.21.15.66:0/1206524998 conn(0x7f6c5409ba30 msgr2=0x7f6c54089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:42.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c5be00640 1 -- 172.21.15.66:0/1206524998 shutdown_connections 2024-03-30T00:51:42.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.011+0000 7f6c5be00640 1 -- 172.21.15.66:0/1206524998 wait complete. 2024-03-30T00:51:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c5be00640 1 Processor -- start 2024-03-30T00:51:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c5be00640 1 -- start start 2024-03-30T00:51:42.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c5be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c540cd260 0x7f6c540b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c5be00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c5415c780 0x7f6c5414caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c5be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5415cb60 0x7f6c5414d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c5be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5414ec80 con 0x7f6c5415cb60 2024-03-30T00:51:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c5be00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c54159e20 con 0x7f6c5415c780 2024-03-30T00:51:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c5be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541527f0 con 0x7f6c540cd260 2024-03-30T00:51:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c59e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5415cb60 0x7f6c5414d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c59e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5415cb60 0x7f6c5414d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37990/0 (socket says 172.21.15.66:37990) 2024-03-30T00:51:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c59e00640 1 -- 172.21.15.66:0/3400187861 learned_addr learned my addr 172.21.15.66:0/3400187861 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c59400640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c540cd260 0x7f6c540b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c58a00640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c5415c780 0x7f6c5414caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c59e00640 1 -- 172.21.15.66:0/3400187861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c540cd260 msgr2=0x7f6c540b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:42.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c59e00640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c540cd260 0x7f6c540b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c59e00640 1 -- 172.21.15.66:0/3400187861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c5415c780 msgr2=0x7f6c5414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c59e00640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c5415c780 0x7f6c5414caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c59e00640 1 -- 172.21.15.66:0/3400187861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400094d0 con 0x7f6c5415cb60 2024-03-30T00:51:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c59e00640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5415cb60 0x7f6c5414d3b0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00e890 tx=0x7f6c4c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c51600640 1 -- 172.21.15.66:0/3400187861 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c009600 con 0x7f6c5415cb60 2024-03-30T00:51:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c51600640 1 -- 172.21.15.66:0/3400187861 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c00c940 con 0x7f6c5415cb60 2024-03-30T00:51:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c51600640 1 -- 172.21.15.66:0/3400187861 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00ccd0 con 0x7f6c5415cb60 2024-03-30T00:51:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c5be00640 1 -- 172.21.15.66:0/3400187861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c540b57f0 con 0x7f6c5415cb60 2024-03-30T00:51:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.015+0000 7f6c5be00640 1 -- 172.21.15.66:0/3400187861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c54152510 con 0x7f6c5415cb60 2024-03-30T00:51:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.019+0000 7f6c5be00640 1 -- 172.21.15.66:0/3400187861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c54066c30 con 0x7f6c5415cb60 2024-03-30T00:51:42.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.019+0000 7f6c51600640 1 -- 172.21.15.66:0/3400187861 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6c4c013470 con 0x7f6c5415cb60 2024-03-30T00:51:42.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.019+0000 7f6c51600640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 0x7f6c34068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.019+0000 7f6c51600640 1 -- 172.21.15.66:0/3400187861 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f6c4c0848a0 con 0x7f6c5415cb60 2024-03-30T00:51:42.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.019+0000 7f6c59400640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 0x7f6c34068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.019+0000 7f6c59400640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 0x7f6c34068b50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6c48003ab0 tx=0x7f6c480098d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:42.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.023+0000 7f6c51600640 1 -- 172.21.15.66:0/3400187861 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6c4c051a30 con 0x7f6c5415cb60 2024-03-30T00:51:42.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 -- 172.21.15.66:0/3400187861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c54159f60 con 0x7f6c5415cb60 2024-03-30T00:51:42.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c51600640 1 -- 172.21.15.66:0/3400187861 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c4c055050 con 0x7f6c5415cb60 2024-03-30T00:51:42.195 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:51:42.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 -- 172.21.15.66:0/3400187861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 msgr2=0x7f6c34068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 0x7f6c34068b50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6c48003ab0 tx=0x7f6c480098d0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 -- 172.21.15.66:0/3400187861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5415cb60 msgr2=0x7f6c5414d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5415cb60 0x7f6c5414d3b0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00e890 tx=0x7f6c4c00ed50 comp rx=0 tx=0).stop 2024-03-30T00:51:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 -- 172.21.15.66:0/3400187861 shutdown_connections 2024-03-30T00:51:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 0x7f6c34068b50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5415cb60 0x7f6c5414d3b0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c5415c780 0x7f6c5414caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 --2- 172.21.15.66:0/3400187861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c540cd260 0x7f6c540b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 -- 172.21.15.66:0/3400187861 >> 172.21.15.66:0/3400187861 conn(0x7f6c5409ba30 msgr2=0x7f6c54160400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 -- 172.21.15.66:0/3400187861 shutdown_connections 2024-03-30T00:51:42.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.187+0000 7f6c5be00640 1 -- 172.21.15.66:0/3400187861 wait complete. 2024-03-30T00:51:42.216 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:51:42.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.347+0000 7f544e600640 1 Processor -- start 2024-03-30T00:51:42.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.347+0000 7f544e600640 1 -- start start 2024-03-30T00:51:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f544e600640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f544815a640 0x7f544806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f544e600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54480ce520 0x7f54480ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f544e600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544806d070 con 0x7f54480cabb0 2024-03-30T00:51:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f544e600640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54480cbba0 con 0x7f544815a640 2024-03-30T00:51:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f544e600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544806e2c0 con 0x7f54480ce520 2024-03-30T00:51:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f5447e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f54480cabb0 0x7f54480cb000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40170/0 (socket says 172.21.15.66:40170) 2024-03-30T00:51:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f5447e00640 1 -- 172.21.15.66:0/3394824142 learned_addr learned my addr 172.21.15.66:0/3394824142 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f543fe00640 1 --2- 172.21.15.66:0/3394824142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54480ce520 0x7f54480ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f5447400640 1 --2- 172.21.15.66:0/3394824142 >> v2:172.21.15.156:3300/0 conn(0x7f544815a640 0x7f544806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f5446a00640 1 -- 172.21.15.66:0/3394824142 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510727491 0 0) 0x7f544806d070 con 0x7f54480cabb0 2024-03-30T00:51:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f5446a00640 1 -- 172.21.15.66:0/3394824142 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f542c003530 con 0x7f54480cabb0 2024-03-30T00:51:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f5447400640 1 -- 172.21.15.66:0/3394824142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54480ce520 msgr2=0x7f54480ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f5447400640 1 --2- 172.21.15.66:0/3394824142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54480ce520 0x7f54480ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f5447400640 1 -- 172.21.15.66:0/3394824142 >> v1:172.21.15.66:6789/0 conn(0x7f54480cabb0 legacy=0x7f54480cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.351+0000 7f5447400640 1 -- 172.21.15.66:0/3394824142 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54480cee60 con 0x7f544815a640 2024-03-30T00:51:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f5447400640 1 --2- 172.21.15.66:0/3394824142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f544815a640 0x7f544806dd80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f54380097c0 tx=0x7f543801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee52cef143d8eb42 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f5446a00640 1 -- 172.21.15.66:0/3394824142 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380ad640 con 0x7f544815a640 2024-03-30T00:51:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f5446a00640 1 -- 172.21.15.66:0/3394824142 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54380ad7a0 con 0x7f544815a640 2024-03-30T00:51:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f5446a00640 1 -- 172.21.15.66:0/3394824142 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380adb10 con 0x7f544815a640 2024-03-30T00:51:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f544e600640 1 -- 172.21.15.66:0/3394824142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f544815a640 msgr2=0x7f544806dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f544e600640 1 --2- 172.21.15.66:0/3394824142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f544815a640 0x7f544806dd80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f54380097c0 tx=0x7f543801c4a0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f544e600640 1 -- 172.21.15.66:0/3394824142 shutdown_connections 2024-03-30T00:51:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f544e600640 1 --2- 172.21.15.66:0/3394824142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54480ce520 0x7f54480ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f544e600640 1 --2- 172.21.15.66:0/3394824142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f544815a640 0x7f544806dd80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f544e600640 1 -- 172.21.15.66:0/3394824142 >> 172.21.15.66:0/3394824142 conn(0x7f5448069f60 msgr2=0x7f544806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:42.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f544e600640 1 -- 172.21.15.66:0/3394824142 shutdown_connections 2024-03-30T00:51:42.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f544e600640 1 -- 172.21.15.66:0/3394824142 wait complete. 2024-03-30T00:51:42.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f544e600640 1 Processor -- start 2024-03-30T00:51:42.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.355+0000 7f544e600640 1 -- start start 2024-03-30T00:51:42.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f544e600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54480cabb0 0x7f54480cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f544e600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54480ce520 0x7f5448154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f544e600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f544815a640 0x7f5448149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f544e600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54480717f0 con 0x7f54480cabb0 2024-03-30T00:51:42.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f544e600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5448071360 con 0x7f544815a640 2024-03-30T00:51:42.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f544e600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54480ccaf0 con 0x7f54480ce520 2024-03-30T00:51:42.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f5447400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54480cabb0 0x7f54480cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f5447400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54480cabb0 0x7f54480cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38004/0 (socket says 172.21.15.66:38004) 2024-03-30T00:51:42.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f5447400640 1 -- 172.21.15.66:0/595513115 learned_addr learned my addr 172.21.15.66:0/595513115 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:42.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f543fe00640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54480ce520 0x7f5448154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f5447e00640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f544815a640 0x7f5448149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f5447400640 1 -- 172.21.15.66:0/595513115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54480ce520 msgr2=0x7f5448154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:42.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f5447400640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54480ce520 0x7f5448154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f5447400640 1 -- 172.21.15.66:0/595513115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f544815a640 msgr2=0x7f5448149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:42.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f5447400640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f544815a640 0x7f5448149610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f5447400640 1 -- 172.21.15.66:0/595513115 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5438009420 con 0x7f54480cabb0 2024-03-30T00:51:42.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f5447e00640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f544815a640 0x7f5448149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:42.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f5447400640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54480cabb0 0x7f54480cc5b0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f5438000c10 tx=0x7f5438019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:42.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f543f400640 1 -- 172.21.15.66:0/595513115 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380adb10 con 0x7f54480cabb0 2024-03-30T00:51:42.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f543f400640 1 -- 172.21.15.66:0/595513115 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54380a6070 con 0x7f54480cabb0 2024-03-30T00:51:42.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f543f400640 1 -- 172.21.15.66:0/595513115 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380ade70 con 0x7f54480cabb0 2024-03-30T00:51:42.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f544e600640 1 -- 172.21.15.66:0/595513115 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54480cbff0 con 0x7f54480cabb0 2024-03-30T00:51:42.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f544e600640 1 -- 172.21.15.66:0/595513115 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54480ccd60 con 0x7f54480cabb0 2024-03-30T00:51:42.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f544e600640 1 -- 172.21.15.66:0/595513115 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5448072c00 con 0x7f54480cabb0 2024-03-30T00:51:42.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.359+0000 7f543f400640 1 -- 172.21.15.66:0/595513115 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f54380b4040 con 0x7f54480cabb0 2024-03-30T00:51:42.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.363+0000 7f543f400640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5424066690 0x7f5424068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:42.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.363+0000 7f543f400640 1 -- 172.21.15.66:0/595513115 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f543811d760 con 0x7f54480cabb0 2024-03-30T00:51:42.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.363+0000 7f543fe00640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5424066690 0x7f5424068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:42.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.363+0000 7f543fe00640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5424066690 0x7f5424068b50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f544806a770 tx=0x7f5430008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:42.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.367+0000 7f543f400640 1 -- 172.21.15.66:0/595513115 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f54380ea8d0 con 0x7f54480cabb0 2024-03-30T00:51:42.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.535+0000 7f544e600640 1 -- 172.21.15.66:0/595513115 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5448066c30 con 0x7f54480cabb0 2024-03-30T00:51:42.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.535+0000 7f543f400640 1 -- 172.21.15.66:0/595513115 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v127) v1 ==== 80+0+244 (secure 0 0 0) 0x7f54380edef0 con 0x7f54480cabb0 2024-03-30T00:51:42.544 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6834/3460522956 2024-03-31T00:51:29.804880+0000 2024-03-30T00:51:42.545 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6836/3460522956 2024-03-31T00:51:29.804880+0000 2024-03-30T00:51:42.545 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6836/2683418851 2024-03-31T00:51:15.162460+0000 2024-03-30T00:51:42.545 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6834/2683418851 2024-03-31T00:51:15.162460+0000 2024-03-30T00:51:42.545 INFO:teuthology.orchestra.run.smithi066.stderr:listed 4 entries 2024-03-30T00:51:42.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 -- 172.21.15.66:0/595513115 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5424066690 msgr2=0x7f5424068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5424066690 0x7f5424068b50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f544806a770 tx=0x7f5430008040 comp rx=0 tx=0).stop 2024-03-30T00:51:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 -- 172.21.15.66:0/595513115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54480cabb0 msgr2=0x7f54480cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54480cabb0 0x7f54480cc5b0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f5438000c10 tx=0x7f5438019240 comp rx=0 tx=0).stop 2024-03-30T00:51:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 -- 172.21.15.66:0/595513115 shutdown_connections 2024-03-30T00:51:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f544815a640 0x7f5448149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54480ce520 0x7f5448154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5424066690 0x7f5424068b50 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 --2- 172.21.15.66:0/595513115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54480cabb0 0x7f54480cc5b0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 -- 172.21.15.66:0/595513115 >> 172.21.15.66:0/595513115 conn(0x7f5448069f60 msgr2=0x7f54480afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 -- 172.21.15.66:0/595513115 shutdown_connections 2024-03-30T00:51:42.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:42.539+0000 7f544e600640 1 -- 172.21.15.66:0/595513115 wait complete. 2024-03-30T00:51:42.566 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T00:51:42.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:42.567 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:51:42.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[55825]: fuse finished with error 0 and tester_r 0 2024-03-30T00:51:42.587 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:51:42.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:42.587 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:51:42.720 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:51:42.720 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:42.730 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:51:42.730 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:51:42.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:42.730 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T00:51:42.780 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-30T00:51:42.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:51:42.781 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:51:42.782 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:42.791 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:51:42.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:51:42.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:42.792 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-30T00:51:42.841 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-30T00:51:42.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:51:42.843 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:51:42.843 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:42.853 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:51:42.853 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:51:42.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:42.853 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-30T00:51:42.904 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-30T00:51:42.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:51:42.906 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:51:43.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df2000640 1 Processor -- start 2024-03-30T00:51:43.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df2000640 1 -- start start 2024-03-30T00:51:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df2000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9dec15aa80 0x7f9dec0d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df2000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9dec0d0870 0x7f9dec0ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df2000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dec0d0cb0 con 0x7f9dec15a6a0 2024-03-30T00:51:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df2000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dec0cdf90 con 0x7f9dec0d0870 2024-03-30T00:51:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df2000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dec15eea0 con 0x7f9dec15aa80 2024-03-30T00:51:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df1600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9dec0d0870 0x7f9dec0ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df0c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9dec15a6a0 0x7f9dec15e3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40184/0 (socket says 172.21.15.66:40184) 2024-03-30T00:51:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df0c00640 1 -- 172.21.15.66:0/3576457295 learned_addr learned my addr 172.21.15.66:0/3576457295 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9debe00640 1 --2- 172.21.15.66:0/3576457295 >> v2:172.21.15.156:3300/0 conn(0x7f9dec15aa80 0x7f9dec0d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df1600640 1 -- 172.21.15.66:0/3576457295 >> v2:172.21.15.156:3300/0 conn(0x7f9dec15aa80 msgr2=0x7f9dec0d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df1600640 1 --2- 172.21.15.66:0/3576457295 >> v2:172.21.15.156:3300/0 conn(0x7f9dec15aa80 0x7f9dec0d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df1600640 1 -- 172.21.15.66:0/3576457295 >> v1:172.21.15.66:6789/0 conn(0x7f9dec15a6a0 legacy=0x7f9dec15e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9df1600640 1 -- 172.21.15.66:0/3576457295 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec15f180 con 0x7f9dec0d0870 2024-03-30T00:51:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9debe00640 1 --2- 172.21.15.66:0/3576457295 >> v2:172.21.15.156:3300/0 conn(0x7f9dec15aa80 0x7f9dec0d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.039+0000 7f9deb400640 1 -- 172.21.15.66:0/3576457295 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798593090 0 0) 0x7f9dec0d0cb0 con 0x7f9dec15a6a0 2024-03-30T00:51:43.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9df1600640 1 --2- 172.21.15.66:0/3576457295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec0d0870 0x7f9dec0ce9a0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f9dec06d8e0 tx=0x7f9de401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36670519716440f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:43.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9deb400640 1 -- 172.21.15.66:0/3576457295 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40a5930 con 0x7f9dec0d0870 2024-03-30T00:51:43.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9deb400640 1 -- 172.21.15.66:0/3576457295 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de40a5a90 con 0x7f9dec0d0870 2024-03-30T00:51:43.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9deb400640 1 -- 172.21.15.66:0/3576457295 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40a5e20 con 0x7f9dec0d0870 2024-03-30T00:51:43.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9df2000640 1 -- 172.21.15.66:0/3576457295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec0d0870 msgr2=0x7f9dec0ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9df2000640 1 --2- 172.21.15.66:0/3576457295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec0d0870 0x7f9dec0ce9a0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f9dec06d8e0 tx=0x7f9de401c870 comp rx=0 tx=0).stop 2024-03-30T00:51:43.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9df2000640 1 -- 172.21.15.66:0/3576457295 shutdown_connections 2024-03-30T00:51:43.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9df2000640 1 --2- 172.21.15.66:0/3576457295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec0d0870 0x7f9dec0ce9a0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9df2000640 1 --2- 172.21.15.66:0/3576457295 >> v2:172.21.15.156:3300/0 conn(0x7f9dec15aa80 0x7f9dec0d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9df2000640 1 -- 172.21.15.66:0/3576457295 >> 172.21.15.66:0/3576457295 conn(0x7f9dec06e0f0 msgr2=0x7f9dec06e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:43.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9df2000640 1 -- 172.21.15.66:0/3576457295 shutdown_connections 2024-03-30T00:51:43.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9df2000640 1 -- 172.21.15.66:0/3576457295 wait complete. 2024-03-30T00:51:43.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9df2000640 1 Processor -- start 2024-03-30T00:51:43.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.043+0000 7f9df2000640 1 -- start start 2024-03-30T00:51:43.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df2000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec0d0870 0x7f9dec1436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec15a6a0 0x7f9dec154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df2000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9dec15aa80 0x7f9dec140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df2000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0ca350 con 0x7f9dec15a6a0 2024-03-30T00:51:43.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df2000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0c5ff0 con 0x7f9dec0d0870 2024-03-30T00:51:43.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df2000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec143bf0 con 0x7f9dec15aa80 2024-03-30T00:51:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9debe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec15a6a0 0x7f9dec154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9debe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec15a6a0 0x7f9dec154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38018/0 (socket says 172.21.15.66:38018) 2024-03-30T00:51:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9debe00640 1 -- 172.21.15.66:0/3469403383 learned_addr learned my addr 172.21.15.66:0/3469403383 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df0c00640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec0d0870 0x7f9dec1436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9debe00640 1 -- 172.21.15.66:0/3469403383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9dec15aa80 msgr2=0x7f9dec140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df1600640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9dec15aa80 0x7f9dec140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9debe00640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9dec15aa80 0x7f9dec140c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9debe00640 1 -- 172.21.15.66:0/3469403383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec0d0870 msgr2=0x7f9dec1436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9debe00640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec0d0870 0x7f9dec1436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9debe00640 1 -- 172.21.15.66:0/3469403383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de4009a70 con 0x7f9dec15a6a0 2024-03-30T00:51:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df0c00640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec0d0870 0x7f9dec1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df1600640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9dec15aa80 0x7f9dec140c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9debe00640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec15a6a0 0x7f9dec154570 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c7f0 tx=0x7f9dd800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9de8c00640 1 -- 172.21.15.66:0/3469403383 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8007820 con 0x7f9dec15a6a0 2024-03-30T00:51:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9de8c00640 1 -- 172.21.15.66:0/3469403383 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd8007980 con 0x7f9dec15a6a0 2024-03-30T00:51:43.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9de8c00640 1 -- 172.21.15.66:0/3469403383 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8007d30 con 0x7f9dec15a6a0 2024-03-30T00:51:43.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df2000640 1 -- 172.21.15.66:0/3469403383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec15ec70 con 0x7f9dec15a6a0 2024-03-30T00:51:43.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df2000640 1 -- 172.21.15.66:0/3469403383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec0ca070 con 0x7f9dec15a6a0 2024-03-30T00:51:43.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9df2000640 1 -- 172.21.15.66:0/3469403383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dec0c6130 con 0x7f9dec15a6a0 2024-03-30T00:51:43.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.047+0000 7f9de8c00640 1 -- 172.21.15.66:0/3469403383 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9dd80113f0 con 0x7f9dec15a6a0 2024-03-30T00:51:43.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.051+0000 7f9de8c00640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9dcc066690 0x7f9dcc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.051+0000 7f9de8c00640 1 -- 172.21.15.66:0/3469403383 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f9dd80847f0 con 0x7f9dec15a6a0 2024-03-30T00:51:43.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.051+0000 7f9df0c00640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9dcc066690 0x7f9dcc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.051+0000 7f9df0c00640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9dcc066690 0x7f9dcc068b50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f70 tx=0x7f9de0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:43.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.055+0000 7f9de8c00640 1 -- 172.21.15.66:0/3469403383 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9dec0c6130 con 0x7f9dec15a6a0 2024-03-30T00:51:43.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 -- 172.21.15.66:0/3469403383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dec0c6130 con 0x7f9dec15a6a0 2024-03-30T00:51:43.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9de8c00640 1 -- 172.21.15.66:0/3469403383 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4744 (secure 0 0 0) 0x7f9dec0c6130 con 0x7f9dec15a6a0 2024-03-30T00:51:43.260 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:43.261 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":108,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:51:34.842265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9942":{"gid":9942,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/2774791553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2774791553},{"type":"v1","addr":"172.21.15.66:6836","nonce":2774791553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:51:43.262 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 108 2024-03-30T00:51:43.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 -- 172.21.15.66:0/3469403383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9dcc066690 msgr2=0x7f9dcc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9dcc066690 0x7f9dcc068b50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f70 tx=0x7f9de0009040 comp rx=0 tx=0).stop 2024-03-30T00:51:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 -- 172.21.15.66:0/3469403383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec15a6a0 msgr2=0x7f9dec154570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec15a6a0 0x7f9dec154570 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c7f0 tx=0x7f9dd800ccb0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 -- 172.21.15.66:0/3469403383 shutdown_connections 2024-03-30T00:51:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9dec15aa80 0x7f9dec140c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9dcc066690 0x7f9dcc068b50 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec15a6a0 0x7f9dec154570 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 --2- 172.21.15.66:0/3469403383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec0d0870 0x7f9dec1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 -- 172.21.15.66:0/3469403383 >> 172.21.15.66:0/3469403383 conn(0x7f9dec06e0f0 msgr2=0x7f9dec06b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:43.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 -- 172.21.15.66:0/3469403383 shutdown_connections 2024-03-30T00:51:43.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.251+0000 7f9df2000640 1 -- 172.21.15.66:0/3469403383 wait complete. 2024-03-30T00:51:43.281 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:51:43.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.411+0000 7f6c5ca00640 1 Processor -- start 2024-03-30T00:51:43.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.411+0000 7f6c5ca00640 1 -- start start 2024-03-30T00:51:43.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c5ca00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6c581570a0 0x7f6c5815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c5ca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c5806cd70 0x7f6c580cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c5ca00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5806dca0 con 0x7f6c580ccc90 2024-03-30T00:51:43.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c5ca00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5806d220 con 0x7f6c5806cd70 2024-03-30T00:51:43.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c5ca00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c580cd0b0 con 0x7f6c581570a0 2024-03-30T00:51:43.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c4d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c5806cd70 0x7f6c580cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c4d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6c5806cd70 0x7f6c580cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38034/0 (socket says 172.21.15.66:38034) 2024-03-30T00:51:43.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c4d400640 1 -- 172.21.15.66:0/3460739551 learned_addr learned my addr 172.21.15.66:0/3460739551 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:43.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c55e00640 1 --2- 172.21.15.66:0/3460739551 >> v2:172.21.15.156:3300/0 conn(0x7f6c581570a0 0x7f6c5815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c4d400640 1 -- 172.21.15.66:0/3460739551 >> v2:172.21.15.156:3300/0 conn(0x7f6c581570a0 msgr2=0x7f6c5815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:43.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c4d400640 1 --2- 172.21.15.66:0/3460739551 >> v2:172.21.15.156:3300/0 conn(0x7f6c581570a0 0x7f6c5815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c4d400640 1 -- 172.21.15.66:0/3460739551 >> v1:172.21.15.156:6789/0 conn(0x7f6c580ccc90 legacy=0x7f6c581604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c4d400640 1 -- 172.21.15.66:0/3460739551 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5806d400 con 0x7f6c5806cd70 2024-03-30T00:51:43.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.415+0000 7f6c55e00640 1 --2- 172.21.15.66:0/3460739551 >> v2:172.21.15.156:3300/0 conn(0x7f6c581570a0 0x7f6c5815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:43.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c4d400640 1 --2- 172.21.15.66:0/3460739551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5806cd70 0x7f6c580cc750 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f6c580b4eb0 tx=0x7f6c4001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c029d6dcf50d81f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:43.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c55400640 1 -- 172.21.15.66:0/3460739551 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400ab660 con 0x7f6c5806cd70 2024-03-30T00:51:43.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c55400640 1 -- 172.21.15.66:0/3460739551 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c400ab7c0 con 0x7f6c5806cd70 2024-03-30T00:51:43.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c55400640 1 -- 172.21.15.66:0/3460739551 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400abb50 con 0x7f6c5806cd70 2024-03-30T00:51:43.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3460739551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5806cd70 msgr2=0x7f6c580cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c5ca00640 1 --2- 172.21.15.66:0/3460739551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5806cd70 0x7f6c580cc750 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f6c580b4eb0 tx=0x7f6c4001c460 comp rx=0 tx=0).stop 2024-03-30T00:51:43.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3460739551 shutdown_connections 2024-03-30T00:51:43.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c5ca00640 1 --2- 172.21.15.66:0/3460739551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5806cd70 0x7f6c580cc750 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c5ca00640 1 --2- 172.21.15.66:0/3460739551 >> v2:172.21.15.156:3300/0 conn(0x7f6c581570a0 0x7f6c5815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3460739551 >> 172.21.15.66:0/3460739551 conn(0x7f6c580ceb80 msgr2=0x7f6c580cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3460739551 shutdown_connections 2024-03-30T00:51:43.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3460739551 wait complete. 2024-03-30T00:51:43.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c5ca00640 1 Processor -- start 2024-03-30T00:51:43.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.419+0000 7f6c5ca00640 1 -- start start 2024-03-30T00:51:43.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c5ca00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c5806cd70 0x7f6c58155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c5ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c580ccc90 0x7f6c580b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c5ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c581570a0 0x7f6c58153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c5ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580b2910 con 0x7f6c580ccc90 2024-03-30T00:51:43.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c5ca00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580b2b40 con 0x7f6c5806cd70 2024-03-30T00:51:43.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c5ca00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580b2e20 con 0x7f6c581570a0 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c580ccc90 0x7f6c580b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c580ccc90 0x7f6c580b23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38044/0 (socket says 172.21.15.66:38044) 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4d400640 1 -- 172.21.15.66:0/3052877902 learned_addr learned my addr 172.21.15.66:0/3052877902 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c55e00640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c5806cd70 0x7f6c58155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c56800640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c581570a0 0x7f6c58153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4d400640 1 -- 172.21.15.66:0/3052877902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c581570a0 msgr2=0x7f6c58153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4d400640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c581570a0 0x7f6c58153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4d400640 1 -- 172.21.15.66:0/3052877902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c5806cd70 msgr2=0x7f6c58155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4d400640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c5806cd70 0x7f6c58155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4d400640 1 -- 172.21.15.66:0/3052877902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400094d0 con 0x7f6c580ccc90 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c55e00640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c5806cd70 0x7f6c58155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4d400640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c580ccc90 0x7f6c580b23d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f6c40019070 tx=0x7f6c4001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4ea00640 1 -- 172.21.15.66:0/3052877902 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400b4800 con 0x7f6c580ccc90 2024-03-30T00:51:43.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4ea00640 1 -- 172.21.15.66:0/3052877902 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c400b4980 con 0x7f6c580ccc90 2024-03-30T00:51:43.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4ea00640 1 -- 172.21.15.66:0/3052877902 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400b4d30 con 0x7f6c580ccc90 2024-03-30T00:51:43.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3052877902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c580b3100 con 0x7f6c580ccc90 2024-03-30T00:51:43.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3052877902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c580b36c0 con 0x7f6c580ccc90 2024-03-30T00:51:43.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3052877902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c58066c30 con 0x7f6c580ccc90 2024-03-30T00:51:43.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.423+0000 7f6c4ea00640 1 -- 172.21.15.66:0/3052877902 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6c400a9070 con 0x7f6c580ccc90 2024-03-30T00:51:43.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.427+0000 7f6c4ea00640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 0x7f6c34068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.427+0000 7f6c4ea00640 1 -- 172.21.15.66:0/3052877902 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f6c4011ed90 con 0x7f6c580ccc90 2024-03-30T00:51:43.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.427+0000 7f6c55e00640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 0x7f6c34068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.427+0000 7f6c55e00640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 0x7f6c34068b50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6c480044f0 tx=0x7f6c48003a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:43.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.431+0000 7f6c4ea00640 1 -- 172.21.15.66:0/3052877902 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6c400ec020 con 0x7f6c580ccc90 2024-03-30T00:51:43.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.591+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3052877902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c580b49f0 con 0x7f6c580ccc90 2024-03-30T00:51:43.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.591+0000 7f6c4ea00640 1 -- 172.21.15.66:0/3052877902 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16575 (secure 0 0 0) 0x7f6c400ef640 con 0x7f6c580ccc90 2024-03-30T00:51:43.600 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:43.600 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":127,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:51:29.804902+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6834/3460522956":"2024-03-31T00:51:29.804880+0000","172.21.15.66:6836/3460522956":"2024-03-31T00:51:29.804880+0000","172.21.15.66:6836/2683418851":"2024-03-31T00:51:15.162460+0000","172.21.15.66:6834/2683418851":"2024-03-31T00:51:15.162460+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:51:43.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.591+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3052877902 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 msgr2=0x7f6c34068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.591+0000 7f6c5ca00640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 0x7f6c34068b50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6c480044f0 tx=0x7f6c48003a40 comp rx=0 tx=0).stop 2024-03-30T00:51:43.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.591+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3052877902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c580ccc90 msgr2=0x7f6c580b23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.591+0000 7f6c5ca00640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c580ccc90 0x7f6c580b23d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f6c40019070 tx=0x7f6c4001aca0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.591+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3052877902 shutdown_connections 2024-03-30T00:51:43.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.591+0000 7f6c5ca00640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c581570a0 0x7f6c58153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.591+0000 7f6c5ca00640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6c34066690 0x7f6c34068b50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.591+0000 7f6c5ca00640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c580ccc90 0x7f6c580b23d0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.591+0000 7f6c5ca00640 1 --2- 172.21.15.66:0/3052877902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c5806cd70 0x7f6c58155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.595+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3052877902 >> 172.21.15.66:0/3052877902 conn(0x7f6c580ceb80 msgr2=0x7f6c580ab9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:43.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.595+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3052877902 shutdown_connections 2024-03-30T00:51:43.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.595+0000 7f6c5ca00640 1 -- 172.21.15.66:0/3052877902 wait complete. 2024-03-30T00:51:43.622 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:39:38.912947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:50:35.696965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:50:37.037165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:51:43.623 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:51:43.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadff200640 1 Processor -- start 2024-03-30T00:51:43.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadff200640 1 -- start start 2024-03-30T00:51:43.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadff200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fadf80ce520 0x7fadf80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadff200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fadf80cabb0 0x7fadf80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadff200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf806d070 con 0x7fadf815a640 2024-03-30T00:51:43.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadff200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf80cbba0 con 0x7fadf80cabb0 2024-03-30T00:51:43.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadff200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf80cee70 con 0x7fadf80ce520 2024-03-30T00:51:43.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadfd200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fadf80cabb0 0x7fadf80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadf7e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fadf815a640 0x7fadf806dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40198/0 (socket says 172.21.15.66:40198) 2024-03-30T00:51:43.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadf7e00640 1 -- 172.21.15.66:0/2644371494 learned_addr learned my addr 172.21.15.66:0/2644371494 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:43.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadfd200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fadf80cabb0 0x7fadf80cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38056/0 (socket says 172.21.15.66:38056) 2024-03-30T00:51:43.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.763+0000 7fadefe00640 1 --2- 172.21.15.66:0/2644371494 >> v2:172.21.15.156:3300/0 conn(0x7fadf80ce520 0x7fadf80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadfd200640 1 -- 172.21.15.66:0/2644371494 >> v2:172.21.15.156:3300/0 conn(0x7fadf80ce520 msgr2=0x7fadf80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:43.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadfd200640 1 --2- 172.21.15.66:0/2644371494 >> v2:172.21.15.156:3300/0 conn(0x7fadf80ce520 0x7fadf80ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadfd200640 1 -- 172.21.15.66:0/2644371494 >> v1:172.21.15.66:6789/0 conn(0x7fadf815a640 legacy=0x7fadf806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadfd200640 1 -- 172.21.15.66:0/2644371494 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80cb8c0 con 0x7fadf80cabb0 2024-03-30T00:51:43.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadf7400640 1 -- 172.21.15.66:0/2644371494 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898745074 0 0) 0x7fadf806d070 con 0x7fadf815a640 2024-03-30T00:51:43.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadfd200640 1 --2- 172.21.15.66:0/2644371494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadf80cabb0 0x7fadf80cafe0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fadf80c88d0 tx=0x7fadf001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82795895cfa14de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:43.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadf7400640 1 -- 172.21.15.66:0/2644371494 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00a59c0 con 0x7fadf80cabb0 2024-03-30T00:51:43.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadf7400640 1 -- 172.21.15.66:0/2644371494 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf00a5b20 con 0x7fadf80cabb0 2024-03-30T00:51:43.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadf7400640 1 -- 172.21.15.66:0/2644371494 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00ad440 con 0x7fadf80cabb0 2024-03-30T00:51:43.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadff200640 1 -- 172.21.15.66:0/2644371494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadf80cabb0 msgr2=0x7fadf80cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadff200640 1 --2- 172.21.15.66:0/2644371494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadf80cabb0 0x7fadf80cafe0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fadf80c88d0 tx=0x7fadf001c900 comp rx=0 tx=0).stop 2024-03-30T00:51:43.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadff200640 1 -- 172.21.15.66:0/2644371494 shutdown_connections 2024-03-30T00:51:43.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadff200640 1 --2- 172.21.15.66:0/2644371494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadf80cabb0 0x7fadf80cafe0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadff200640 1 --2- 172.21.15.66:0/2644371494 >> v2:172.21.15.156:3300/0 conn(0x7fadf80ce520 0x7fadf80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.767+0000 7fadff200640 1 -- 172.21.15.66:0/2644371494 >> 172.21.15.66:0/2644371494 conn(0x7fadf8069f60 msgr2=0x7fadf806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:43.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadff200640 1 -- 172.21.15.66:0/2644371494 shutdown_connections 2024-03-30T00:51:43.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadff200640 1 -- 172.21.15.66:0/2644371494 wait complete. 2024-03-30T00:51:43.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadff200640 1 Processor -- start 2024-03-30T00:51:43.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadff200640 1 -- start start 2024-03-30T00:51:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadff200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadf80cabb0 0x7fadf80c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadff200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadf80ce520 0x7fadf806fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadff200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadf815a640 0x7fadf80c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadff200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf8071ed0 con 0x7fadf815a640 2024-03-30T00:51:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadff200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80705d0 con 0x7fadf80ce520 2024-03-30T00:51:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadff200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80708b0 con 0x7fadf80cabb0 2024-03-30T00:51:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadfd200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadf815a640 0x7fadf80c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadfd200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadf815a640 0x7fadf80c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38064/0 (socket says 172.21.15.66:38064) 2024-03-30T00:51:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadfd200640 1 -- 172.21.15.66:0/892435858 learned_addr learned my addr 172.21.15.66:0/892435858 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadf7e00640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadf80cabb0 0x7fadf80c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadefe00640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadf80ce520 0x7fadf806fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadfd200640 1 -- 172.21.15.66:0/892435858 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadf80cabb0 msgr2=0x7fadf80c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadfd200640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadf80cabb0 0x7fadf80c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadfd200640 1 -- 172.21.15.66:0/892435858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadf80ce520 msgr2=0x7fadf806fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadfd200640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadf80ce520 0x7fadf806fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadfd200640 1 -- 172.21.15.66:0/892435858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf0009a70 con 0x7fadf815a640 2024-03-30T00:51:43.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadfd200640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadf815a640 0x7fadf80c9800 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fadf0019510 tx=0x7fadf00196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:43.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadf4c00640 1 -- 172.21.15.66:0/892435858 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00add70 con 0x7fadf815a640 2024-03-30T00:51:43.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadf4c00640 1 -- 172.21.15.66:0/892435858 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0009450 con 0x7fadf815a640 2024-03-30T00:51:43.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.771+0000 7fadf4c00640 1 -- 172.21.15.66:0/892435858 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00b9690 con 0x7fadf815a640 2024-03-30T00:51:43.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.775+0000 7fadff200640 1 -- 172.21.15.66:0/892435858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf8070b90 con 0x7fadf815a640 2024-03-30T00:51:43.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.775+0000 7fadff200640 1 -- 172.21.15.66:0/892435858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf8071150 con 0x7fadf815a640 2024-03-30T00:51:43.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.775+0000 7fadff200640 1 -- 172.21.15.66:0/892435858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf8071340 con 0x7fadf815a640 2024-03-30T00:51:43.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.775+0000 7fadf4c00640 1 -- 172.21.15.66:0/892435858 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fadf00b97f0 con 0x7fadf815a640 2024-03-30T00:51:43.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.775+0000 7fadf4c00640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fadd4066690 0x7fadd4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:43.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.775+0000 7fadf4c00640 1 -- 172.21.15.66:0/892435858 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fadf011fb70 con 0x7fadf815a640 2024-03-30T00:51:43.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.775+0000 7fadf7e00640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fadd4066690 0x7fadd4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:43.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.775+0000 7fadf7e00640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fadd4066690 0x7fadd4068b50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fade8000f70 tx=0x7fade8007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:43.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.779+0000 7fadf4c00640 1 -- 172.21.15.66:0/892435858 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fadf00ecec0 con 0x7fadf815a640 2024-03-30T00:51:43.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 -- 172.21.15.66:0/892435858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fadf8066c30 con 0x7fadf815a640 2024-03-30T00:51:43.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadf4c00640 1 -- 172.21.15.66:0/892435858 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadf00f04e0 con 0x7fadf815a640 2024-03-30T00:51:43.950 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T00:51:43.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 -- 172.21.15.66:0/892435858 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fadd4066690 msgr2=0x7fadd4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fadd4066690 0x7fadd4068b50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fade8000f70 tx=0x7fade8007df0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 -- 172.21.15.66:0/892435858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadf815a640 msgr2=0x7fadf80c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:43.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadf815a640 0x7fadf80c9800 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fadf0019510 tx=0x7fadf00196a0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 -- 172.21.15.66:0/892435858 shutdown_connections 2024-03-30T00:51:43.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fadd4066690 0x7fadd4068b50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadf815a640 0x7fadf80c9800 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadf80ce520 0x7fadf806fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 --2- 172.21.15.66:0/892435858 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadf80cabb0 0x7fadf80c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:43.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 -- 172.21.15.66:0/892435858 >> 172.21.15.66:0/892435858 conn(0x7fadf8069f60 msgr2=0x7fadf8157180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:43.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 -- 172.21.15.66:0/892435858 shutdown_connections 2024-03-30T00:51:43.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:43.943+0000 7fadff200640 1 -- 172.21.15.66:0/892435858 wait complete. 2024-03-30T00:51:43.972 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:51:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c9c00640 1 Processor -- start 2024-03-30T00:51:44.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c9c00640 1 -- start start 2024-03-30T00:51:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c9c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe6c415a640 0x7fe6c406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c9c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6c40ce520 0x7fe6c40ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c9c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6c40cabb0 0x7fe6c40cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c9c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c406d070 con 0x7fe6c40cabb0 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c9c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40cbba0 con 0x7fe6c415a640 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c9c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c406e2c0 con 0x7fe6c40ce520 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6c40cabb0 0x7fe6c40cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6c40cabb0 0x7fe6c40cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38068/0 (socket says 172.21.15.66:38068) 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c3400640 1 -- 172.21.15.66:0/3049024161 learned_addr learned my addr 172.21.15.66:0/3049024161 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6ba000640 1 --2- 172.21.15.66:0/3049024161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6c40ce520 0x7fe6c40ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c2a00640 1 --2- 172.21.15.66:0/3049024161 >> v2:172.21.15.156:3300/0 conn(0x7fe6c415a640 0x7fe6c406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c3400640 1 -- 172.21.15.66:0/3049024161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6c40ce520 msgr2=0x7fe6c40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c3400640 1 --2- 172.21.15.66:0/3049024161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6c40ce520 0x7fe6c40ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c3400640 1 -- 172.21.15.66:0/3049024161 >> v2:172.21.15.156:3300/0 conn(0x7fe6c415a640 msgr2=0x7fe6c406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c3400640 1 --2- 172.21.15.66:0/3049024161 >> v2:172.21.15.156:3300/0 conn(0x7fe6c415a640 0x7fe6c406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.107+0000 7fe6c3400640 1 -- 172.21.15.66:0/3049024161 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c40cee60 con 0x7fe6c40cabb0 2024-03-30T00:51:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c3400640 1 --2- 172.21.15.66:0/3049024161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c40cabb0 0x7fe6c40cb000 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40cf160 tx=0x7fe6bc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1001b38b7fa97d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c2000640 1 -- 172.21.15.66:0/3049024161 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0aa610 con 0x7fe6c40cabb0 2024-03-30T00:51:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c2000640 1 -- 172.21.15.66:0/3049024161 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc0a6040 con 0x7fe6c40cabb0 2024-03-30T00:51:44.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c2000640 1 -- 172.21.15.66:0/3049024161 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0aa990 con 0x7fe6c40cabb0 2024-03-30T00:51:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c9c00640 1 -- 172.21.15.66:0/3049024161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c40cabb0 msgr2=0x7fe6c40cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c9c00640 1 --2- 172.21.15.66:0/3049024161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c40cabb0 0x7fe6c40cb000 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40cf160 tx=0x7fe6bc01c4f0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c9c00640 1 -- 172.21.15.66:0/3049024161 shutdown_connections 2024-03-30T00:51:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c9c00640 1 --2- 172.21.15.66:0/3049024161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c40cabb0 0x7fe6c40cb000 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c9c00640 1 --2- 172.21.15.66:0/3049024161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6c40ce520 0x7fe6c40ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c9c00640 1 --2- 172.21.15.66:0/3049024161 >> v2:172.21.15.156:3300/0 conn(0x7fe6c415a640 0x7fe6c406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c9c00640 1 -- 172.21.15.66:0/3049024161 >> 172.21.15.66:0/3049024161 conn(0x7fe6c4069f60 msgr2=0x7fe6c406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c9c00640 1 -- 172.21.15.66:0/3049024161 shutdown_connections 2024-03-30T00:51:44.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.111+0000 7fe6c9c00640 1 -- 172.21.15.66:0/3049024161 wait complete. 2024-03-30T00:51:44.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c9c00640 1 Processor -- start 2024-03-30T00:51:44.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c9c00640 1 -- start start 2024-03-30T00:51:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c40cabb0 0x7fe6c40ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c9c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6c40ce520 0x7fe6c4153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c9c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6c415a640 0x7fe6c4097560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c9c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c4154390 con 0x7fe6c40cabb0 2024-03-30T00:51:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c9c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c4153af0 con 0x7fe6c40ce520 2024-03-30T00:51:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c9c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c4153dd0 con 0x7fe6c415a640 2024-03-30T00:51:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c40cabb0 0x7fe6c40ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c40cabb0 0x7fe6c40ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38084/0 (socket says 172.21.15.66:38084) 2024-03-30T00:51:44.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c2a00640 1 -- 172.21.15.66:0/469796480 learned_addr learned my addr 172.21.15.66:0/469796480 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c3400640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6c415a640 0x7fe6c4097560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6ba000640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6c40ce520 0x7fe6c4153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c2a00640 1 -- 172.21.15.66:0/469796480 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6c415a640 msgr2=0x7fe6c4097560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c2a00640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6c415a640 0x7fe6c4097560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c2a00640 1 -- 172.21.15.66:0/469796480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6c40ce520 msgr2=0x7fe6c4153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c2a00640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6c40ce520 0x7fe6c4153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c2a00640 1 -- 172.21.15.66:0/469796480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc0094d0 con 0x7fe6c40cabb0 2024-03-30T00:51:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6ba000640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6c40ce520 0x7fe6c4153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c2a00640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c40cabb0 0x7fe6c40ca140 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4007e40 tx=0x7fe6b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6bb400640 1 -- 172.21.15.66:0/469796480 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4014620 con 0x7fe6c40cabb0 2024-03-30T00:51:44.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6bb400640 1 -- 172.21.15.66:0/469796480 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b400ee90 con 0x7fe6c40cabb0 2024-03-30T00:51:44.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6bb400640 1 -- 172.21.15.66:0/469796480 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40149c0 con 0x7fe6c40cabb0 2024-03-30T00:51:44.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c9c00640 1 -- 172.21.15.66:0/469796480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c41540b0 con 0x7fe6c40cabb0 2024-03-30T00:51:44.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.115+0000 7fe6c9c00640 1 -- 172.21.15.66:0/469796480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c4159a30 con 0x7fe6c40cabb0 2024-03-30T00:51:44.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.119+0000 7fe6c9c00640 1 -- 172.21.15.66:0/469796480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c4066c30 con 0x7fe6c40cabb0 2024-03-30T00:51:44.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.119+0000 7fe6bb400640 1 -- 172.21.15.66:0/469796480 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe6b4014b20 con 0x7fe6c40cabb0 2024-03-30T00:51:44.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.119+0000 7fe6bb400640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6a0066690 0x7fe6a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.119+0000 7fe6bb400640 1 -- 172.21.15.66:0/469796480 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fe6b4084d30 con 0x7fe6c40cabb0 2024-03-30T00:51:44.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.119+0000 7fe6ba000640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6a0066690 0x7fe6a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.119+0000 7fe6ba000640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6a0066690 0x7fe6a0068b50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac004660 tx=0x7fe6ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.123+0000 7fe6bb400640 1 -- 172.21.15.66:0/469796480 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe6b4051e10 con 0x7fe6c40cabb0 2024-03-30T00:51:44.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.283+0000 7fe6c9c00640 1 -- 172.21.15.66:0/469796480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6c40c19d0 con 0x7fe6c40cabb0 2024-03-30T00:51:44.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.283+0000 7fe6bb400640 1 -- 172.21.15.66:0/469796480 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6b4055430 con 0x7fe6c40cabb0 2024-03-30T00:51:44.293 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T00:51:44.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 -- 172.21.15.66:0/469796480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6a0066690 msgr2=0x7fe6a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6a0066690 0x7fe6a0068b50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac004660 tx=0x7fe6ac008040 comp rx=0 tx=0).stop 2024-03-30T00:51:44.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 -- 172.21.15.66:0/469796480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c40cabb0 msgr2=0x7fe6c40ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c40cabb0 0x7fe6c40ca140 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4007e40 tx=0x7fe6b400e4b0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 -- 172.21.15.66:0/469796480 shutdown_connections 2024-03-30T00:51:44.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6c415a640 0x7fe6c4097560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6c40ce520 0x7fe6c4153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe6a0066690 0x7fe6a0068b50 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 --2- 172.21.15.66:0/469796480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c40cabb0 0x7fe6c40ca140 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 -- 172.21.15.66:0/469796480 >> 172.21.15.66:0/469796480 conn(0x7fe6c4069f60 msgr2=0x7fe6c40ac4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:44.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 -- 172.21.15.66:0/469796480 shutdown_connections 2024-03-30T00:51:44.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.287+0000 7fe6c9c00640 1 -- 172.21.15.66:0/469796480 wait complete. 2024-03-30T00:51:44.314 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:51:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.447+0000 7f7b9ea00640 1 Processor -- start 2024-03-30T00:51:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.447+0000 7f7b9ea00640 1 -- start start 2024-03-30T00:51:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b9ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7b980ceb60 0x7f7b9815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b9ea00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b9815d360 0x7f7b980cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b9ea00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b980cbec0 con 0x7f7b980ce780 2024-03-30T00:51:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b9ea00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9815d970 con 0x7f7b980ceb60 2024-03-30T00:51:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b9ea00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9815dc50 con 0x7f7b9815d360 2024-03-30T00:51:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b97e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7b980ce780 0x7f7b980cc910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40210/0 (socket says 172.21.15.66:40210) 2024-03-30T00:51:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b97e00640 1 -- 172.21.15.66:0/2543138092 learned_addr learned my addr 172.21.15.66:0/2543138092 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:44.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b8fe00640 1 --2- 172.21.15.66:0/2543138092 >> v2:172.21.15.66:3300/0 conn(0x7f7b980ceb60 0x7f7b9815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b9ca00640 1 --2- 172.21.15.66:0/2543138092 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b9815d360 0x7f7b980cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b97400640 1 -- 172.21.15.66:0/2543138092 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495784321 0 0) 0x7f7b980cbec0 con 0x7f7b980ce780 2024-03-30T00:51:44.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b97400640 1 -- 172.21.15.66:0/2543138092 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b7c003530 con 0x7f7b980ce780 2024-03-30T00:51:44.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b8fe00640 1 -- 172.21.15.66:0/2543138092 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b9815d360 msgr2=0x7f7b980cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:44.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b8fe00640 1 --2- 172.21.15.66:0/2543138092 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b9815d360 0x7f7b980cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b8fe00640 1 -- 172.21.15.66:0/2543138092 >> v1:172.21.15.66:6789/0 conn(0x7f7b980ce780 legacy=0x7f7b980cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b8fe00640 1 -- 172.21.15.66:0/2543138092 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980cc110 con 0x7f7b980ceb60 2024-03-30T00:51:44.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.451+0000 7f7b8fe00640 1 --2- 172.21.15.66:0/2543138092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b980ceb60 0x7f7b9815ce20 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f7b800097c0 tx=0x7f7b8001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312ba2620ab5cc10 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:44.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b97400640 1 -- 172.21.15.66:0/2543138092 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800ab600 con 0x7f7b980ceb60 2024-03-30T00:51:44.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b97400640 1 -- 172.21.15.66:0/2543138092 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b800a7050 con 0x7f7b980ceb60 2024-03-30T00:51:44.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b97400640 1 -- 172.21.15.66:0/2543138092 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800ab980 con 0x7f7b980ceb60 2024-03-30T00:51:44.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 -- 172.21.15.66:0/2543138092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b980ceb60 msgr2=0x7f7b9815ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 --2- 172.21.15.66:0/2543138092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b980ceb60 0x7f7b9815ce20 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f7b800097c0 tx=0x7f7b8001c3e0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 -- 172.21.15.66:0/2543138092 shutdown_connections 2024-03-30T00:51:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 --2- 172.21.15.66:0/2543138092 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b9815d360 0x7f7b980cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 --2- 172.21.15.66:0/2543138092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b980ceb60 0x7f7b9815ce20 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 -- 172.21.15.66:0/2543138092 >> 172.21.15.66:0/2543138092 conn(0x7f7b98069f60 msgr2=0x7f7b9806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 -- 172.21.15.66:0/2543138092 shutdown_connections 2024-03-30T00:51:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 -- 172.21.15.66:0/2543138092 wait complete. 2024-03-30T00:51:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 Processor -- start 2024-03-30T00:51:44.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 -- start start 2024-03-30T00:51:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b980ce780 0x7f7b98148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b980ceb60 0x7f7b981447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b9815d360 0x7f7b98144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b98145360 con 0x7f7b980ceb60 2024-03-30T00:51:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b98145640 con 0x7f7b980ce780 2024-03-30T00:51:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b9ea00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b98145920 con 0x7f7b9815d360 2024-03-30T00:51:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b8fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b980ceb60 0x7f7b981447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b8fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b980ceb60 0x7f7b981447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38098/0 (socket says 172.21.15.66:38098) 2024-03-30T00:51:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.455+0000 7f7b8fe00640 1 -- 172.21.15.66:0/936070138 learned_addr learned my addr 172.21.15.66:0/936070138 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b9ca00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b9815d360 0x7f7b98144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b97e00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b980ce780 0x7f7b98148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b8fe00640 1 -- 172.21.15.66:0/936070138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b9815d360 msgr2=0x7f7b98144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b8fe00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b9815d360 0x7f7b98144d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b8fe00640 1 -- 172.21.15.66:0/936070138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b980ce780 msgr2=0x7f7b98148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b8fe00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b980ce780 0x7f7b98148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b8fe00640 1 -- 172.21.15.66:0/936070138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b80009420 con 0x7f7b980ceb60 2024-03-30T00:51:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b97e00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b980ce780 0x7f7b98148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b8fe00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b980ceb60 0x7f7b981447f0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f7b8001a010 tx=0x7f7b80019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b94c00640 1 -- 172.21.15.66:0/936070138 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800b4410 con 0x7f7b980ceb60 2024-03-30T00:51:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b94c00640 1 -- 172.21.15.66:0/936070138 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b800b4570 con 0x7f7b980ceb60 2024-03-30T00:51:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b94c00640 1 -- 172.21.15.66:0/936070138 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800b4920 con 0x7f7b980ceb60 2024-03-30T00:51:44.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b9ea00640 1 -- 172.21.15.66:0/936070138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b98145c00 con 0x7f7b980ceb60 2024-03-30T00:51:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b9ea00640 1 -- 172.21.15.66:0/936070138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b98149800 con 0x7f7b980ceb60 2024-03-30T00:51:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b9ea00640 1 -- 172.21.15.66:0/936070138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b98146120 con 0x7f7b980ceb60 2024-03-30T00:51:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b94c00640 1 -- 172.21.15.66:0/936070138 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7b800b2040 con 0x7f7b980ceb60 2024-03-30T00:51:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b94c00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b74066690 0x7f7b74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b94c00640 1 -- 172.21.15.66:0/936070138 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f7b8011e3a0 con 0x7f7b980ceb60 2024-03-30T00:51:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.459+0000 7f7b97e00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b74066690 0x7f7b74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.463+0000 7f7b97e00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b74066690 0x7f7b74068b50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7b880011f0 tx=0x7f7b88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.463+0000 7f7b94c00640 1 -- 172.21.15.66:0/936070138 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7b98146120 con 0x7f7b980ceb60 2024-03-30T00:51:44.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 -- 172.21.15.66:0/936070138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b98146120 con 0x7f7b980ceb60 2024-03-30T00:51:44.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b94c00640 1 -- 172.21.15.66:0/936070138 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b98146120 con 0x7f7b980ceb60 2024-03-30T00:51:44.631 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 -- 172.21.15.66:0/936070138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b74066690 msgr2=0x7f7b74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b74066690 0x7f7b74068b50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7b880011f0 tx=0x7f7b88009040 comp rx=0 tx=0).stop 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 -- 172.21.15.66:0/936070138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b980ceb60 msgr2=0x7f7b981447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b980ceb60 0x7f7b981447f0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f7b8001a010 tx=0x7f7b80019180 comp rx=0 tx=0).stop 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 -- 172.21.15.66:0/936070138 shutdown_connections 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b9815d360 0x7f7b98144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7b74066690 0x7f7b74068b50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b980ceb60 0x7f7b981447f0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 --2- 172.21.15.66:0/936070138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b980ce780 0x7f7b98148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 -- 172.21.15.66:0/936070138 >> 172.21.15.66:0/936070138 conn(0x7f7b98069f60 msgr2=0x7f7b980ab710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 -- 172.21.15.66:0/936070138 shutdown_connections 2024-03-30T00:51:44.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.623+0000 7f7b9ea00640 1 -- 172.21.15.66:0/936070138 wait complete. 2024-03-30T00:51:44.653 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:51:44.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.783+0000 7fea56200640 1 Processor -- start 2024-03-30T00:51:44.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.783+0000 7fea56200640 1 -- start start 2024-03-30T00:51:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.787+0000 7fea56200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fea5006e800 0x7fea500d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.787+0000 7fea56200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5006b0b0 con 0x7fea5006f8e0 2024-03-30T00:51:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.787+0000 7fea56200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5006b390 con 0x7fea500cc560 2024-03-30T00:51:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.787+0000 7fea56200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea50160640 con 0x7fea5006e800 2024-03-30T00:51:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.787+0000 7fea46a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fea5006e800 0x7fea500d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.787+0000 7fea4fe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fea5006f8e0 0x7fea500ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40220/0 (socket says 172.21.15.66:40220) 2024-03-30T00:51:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.787+0000 7fea4fe00640 1 -- 172.21.15.66:0/1905182175 learned_addr learned my addr 172.21.15.66:0/1905182175 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.787+0000 7fea46a00640 1 -- 172.21.15.66:0/1905182175 >> v1:172.21.15.156:6789/0 conn(0x7fea500cc560 legacy=0x7fea5006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.787+0000 7fea46a00640 1 -- 172.21.15.66:0/1905182175 >> v1:172.21.15.66:6789/0 conn(0x7fea5006f8e0 legacy=0x7fea500ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.787+0000 7fea46a00640 1 -- 172.21.15.66:0/1905182175 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5006b670 con 0x7fea5006e800 2024-03-30T00:51:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.787+0000 7fea4ea00640 1 -- 172.21.15.66:0/1905182175 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495697210 0 0) 0x7fea5006b0b0 con 0x7fea5006f8e0 2024-03-30T00:51:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea46a00640 1 --2- 172.21.15.66:0/1905182175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5006e800 0x7fea500d1060 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fea380097c0 tx=0x7fea3801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37e39b65a088c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea4ea00640 1 -- 172.21.15.66:0/1905182175 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea380ab650 con 0x7fea5006e800 2024-03-30T00:51:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea4ea00640 1 -- 172.21.15.66:0/1905182175 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea380ab7b0 con 0x7fea5006e800 2024-03-30T00:51:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea4ea00640 1 -- 172.21.15.66:0/1905182175 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea380abb40 con 0x7fea5006e800 2024-03-30T00:51:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea56200640 1 -- 172.21.15.66:0/1905182175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5006e800 msgr2=0x7fea500d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea56200640 1 --2- 172.21.15.66:0/1905182175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5006e800 0x7fea500d1060 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fea380097c0 tx=0x7fea3801c430 comp rx=0 tx=0).stop 2024-03-30T00:51:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea56200640 1 -- 172.21.15.66:0/1905182175 shutdown_connections 2024-03-30T00:51:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea56200640 1 --2- 172.21.15.66:0/1905182175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5006e800 0x7fea500d1060 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea56200640 1 -- 172.21.15.66:0/1905182175 >> 172.21.15.66:0/1905182175 conn(0x7fea500d0540 msgr2=0x7fea500d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea56200640 1 -- 172.21.15.66:0/1905182175 shutdown_connections 2024-03-30T00:51:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea56200640 1 -- 172.21.15.66:0/1905182175 wait complete. 2024-03-30T00:51:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea56200640 1 Processor -- start 2024-03-30T00:51:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.791+0000 7fea56200640 1 -- start start 2024-03-30T00:51:44.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea56200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea5006e800 0x7fea500c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea56200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5006f8e0 0x7fea501519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea56200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea500cc560 0x7fea500aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea56200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea500b1390 con 0x7fea5006f8e0 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea56200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea500b0780 con 0x7fea500cc560 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea56200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea500b0a60 con 0x7fea5006e800 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea46a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5006f8e0 0x7fea501519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea46a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5006f8e0 0x7fea501519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38120/0 (socket says 172.21.15.66:38120) 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea46a00640 1 -- 172.21.15.66:0/4110913370 learned_addr learned my addr 172.21.15.66:0/4110913370 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea4f400640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea5006e800 0x7fea500c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea4fe00640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea500cc560 0x7fea500aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea46a00640 1 -- 172.21.15.66:0/4110913370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea5006e800 msgr2=0x7fea500c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea46a00640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea5006e800 0x7fea500c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea46a00640 1 -- 172.21.15.66:0/4110913370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea500cc560 msgr2=0x7fea500aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea46a00640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea500cc560 0x7fea500aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:44.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea46a00640 1 -- 172.21.15.66:0/4110913370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea38009420 con 0x7fea5006f8e0 2024-03-30T00:51:44.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea4fe00640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea500cc560 0x7fea500aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:44.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea46a00640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5006f8e0 0x7fea501519e0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fea38019040 tx=0x7fea3801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea47e00640 1 -- 172.21.15.66:0/4110913370 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38004220 con 0x7fea5006f8e0 2024-03-30T00:51:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea47e00640 1 -- 172.21.15.66:0/4110913370 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea380b9070 con 0x7fea5006f8e0 2024-03-30T00:51:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea47e00640 1 -- 172.21.15.66:0/4110913370 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea380ab610 con 0x7fea5006f8e0 2024-03-30T00:51:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea56200640 1 -- 172.21.15.66:0/4110913370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea500b0d40 con 0x7fea5006f8e0 2024-03-30T00:51:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea56200640 1 -- 172.21.15.66:0/4110913370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea500b0470 con 0x7fea5006f8e0 2024-03-30T00:51:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea56200640 1 -- 172.21.15.66:0/4110913370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea50066c30 con 0x7fea5006f8e0 2024-03-30T00:51:44.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.795+0000 7fea47e00640 1 -- 172.21.15.66:0/4110913370 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fea38003620 con 0x7fea5006f8e0 2024-03-30T00:51:44.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.799+0000 7fea47e00640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fea2c066690 0x7fea2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:44.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.799+0000 7fea47e00640 1 -- 172.21.15.66:0/4110913370 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fea380bb070 con 0x7fea5006f8e0 2024-03-30T00:51:44.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.799+0000 7fea4f400640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fea2c066690 0x7fea2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:44.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.799+0000 7fea4f400640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fea2c066690 0x7fea2c068b50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fea5015bfd0 tx=0x7fea40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:44.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.803+0000 7fea47e00640 1 -- 172.21.15.66:0/4110913370 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fea380ec1c0 con 0x7fea5006f8e0 2024-03-30T00:51:45.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.999+0000 7fea56200640 1 -- 172.21.15.66:0/4110913370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea500ca420 con 0x7fea5006f8e0 2024-03-30T00:51:45.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:44.999+0000 7fea47e00640 1 -- 172.21.15.66:0/4110913370 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4744 (secure 0 0 0) 0x7fea380c4020 con 0x7fea5006f8e0 2024-03-30T00:51:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:45.009 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":108,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:51:34.842265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9942":{"gid":9942,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/2774791553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2774791553},{"type":"v1","addr":"172.21.15.66:6836","nonce":2774791553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:51:45.009 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 108 2024-03-30T00:51:45.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 -- 172.21.15.66:0/4110913370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fea2c066690 msgr2=0x7fea2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fea2c066690 0x7fea2c068b50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fea5015bfd0 tx=0x7fea40008040 comp rx=0 tx=0).stop 2024-03-30T00:51:45.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 -- 172.21.15.66:0/4110913370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5006f8e0 msgr2=0x7fea501519e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5006f8e0 0x7fea501519e0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fea38019040 tx=0x7fea3801ac70 comp rx=0 tx=0).stop 2024-03-30T00:51:45.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 -- 172.21.15.66:0/4110913370 shutdown_connections 2024-03-30T00:51:45.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea500cc560 0x7fea500aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fea2c066690 0x7fea2c068b50 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5006f8e0 0x7fea501519e0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 --2- 172.21.15.66:0/4110913370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea5006e800 0x7fea500c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 -- 172.21.15.66:0/4110913370 >> 172.21.15.66:0/4110913370 conn(0x7fea500d0540 msgr2=0x7fea500ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 -- 172.21.15.66:0/4110913370 shutdown_connections 2024-03-30T00:51:45.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.003+0000 7fea56200640 1 -- 172.21.15.66:0/4110913370 wait complete. 2024-03-30T00:51:45.031 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:51:45.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa38e200640 1 Processor -- start 2024-03-30T00:51:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa38e200640 1 -- start start 2024-03-30T00:51:45.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa38e200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3880cda90 0x7fa3880ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa38e200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa38815a370 0x7fa3880cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa38e200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38815a990 con 0x7fa3880cde70 2024-03-30T00:51:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa38e200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3880b1b80 con 0x7fa38815a370 2024-03-30T00:51:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa38e200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880cd3e0 con 0x7fa3880cda90 2024-03-30T00:51:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa387e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa38815a370 0x7fa3880cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa387e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa38815a370 0x7fa3880cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38134/0 (socket says 172.21.15.66:38134) 2024-03-30T00:51:45.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa387e00640 1 -- 172.21.15.66:0/3963546132 learned_addr learned my addr 172.21.15.66:0/3963546132 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa387400640 1 --2- 172.21.15.66:0/3963546132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3880cda90 0x7fa3880ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa387e00640 1 -- 172.21.15.66:0/3963546132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3880cda90 msgr2=0x7fa3880ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa387e00640 1 --2- 172.21.15.66:0/3963546132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3880cda90 0x7fa3880ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa387e00640 1 -- 172.21.15.66:0/3963546132 >> v1:172.21.15.156:6789/0 conn(0x7fa3880cde70 legacy=0x7fa388159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa387e00640 1 -- 172.21.15.66:0/3963546132 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3880b11e0 con 0x7fa38815a370 2024-03-30T00:51:45.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.163+0000 7fa387400640 1 --2- 172.21.15.66:0/3963546132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3880cda90 0x7fa3880ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:45.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa387e00640 1 --2- 172.21.15.66:0/3963546132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa38815a370 0x7fa3880cea60 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa380009e50 tx=0x7fa38001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49deffa677496c00 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:45.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa386a00640 1 -- 172.21.15.66:0/3963546132 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800a56f0 con 0x7fa38815a370 2024-03-30T00:51:45.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa386a00640 1 -- 172.21.15.66:0/3963546132 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3800a5850 con 0x7fa38815a370 2024-03-30T00:51:45.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa386a00640 1 -- 172.21.15.66:0/3963546132 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800a5be0 con 0x7fa38815a370 2024-03-30T00:51:45.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa38e200640 1 -- 172.21.15.66:0/3963546132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa38815a370 msgr2=0x7fa3880cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa38e200640 1 --2- 172.21.15.66:0/3963546132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa38815a370 0x7fa3880cea60 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa380009e50 tx=0x7fa38001c960 comp rx=0 tx=0).stop 2024-03-30T00:51:45.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa38e200640 1 -- 172.21.15.66:0/3963546132 shutdown_connections 2024-03-30T00:51:45.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa38e200640 1 --2- 172.21.15.66:0/3963546132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa38815a370 0x7fa3880cea60 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa38e200640 1 --2- 172.21.15.66:0/3963546132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3880cda90 0x7fa3880ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa38e200640 1 -- 172.21.15.66:0/3963546132 >> 172.21.15.66:0/3963546132 conn(0x7fa388069f60 msgr2=0x7fa38806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:45.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa38e200640 1 -- 172.21.15.66:0/3963546132 shutdown_connections 2024-03-30T00:51:45.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.167+0000 7fa38e200640 1 -- 172.21.15.66:0/3963546132 wait complete. 2024-03-30T00:51:45.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa38e200640 1 Processor -- start 2024-03-30T00:51:45.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa38e200640 1 -- start start 2024-03-30T00:51:45.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa38e200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3880cda90 0x7fa3880aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa38e200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3880cde70 0x7fa388097830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa38e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa38815a370 0x7fa38815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa38e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa388159720 con 0x7fa38815a370 2024-03-30T00:51:45.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa38e200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3881520f0 con 0x7fa3880cda90 2024-03-30T00:51:45.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa38e200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa388159440 con 0x7fa3880cde70 2024-03-30T00:51:45.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa387e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa38815a370 0x7fa38815c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa387e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa38815a370 0x7fa38815c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38148/0 (socket says 172.21.15.66:38148) 2024-03-30T00:51:45.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa387e00640 1 -- 172.21.15.66:0/1178290940 learned_addr learned my addr 172.21.15.66:0/1178290940 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:45.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa387400640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3880cda90 0x7fa3880aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa37fe00640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3880cde70 0x7fa388097830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa387e00640 1 -- 172.21.15.66:0/1178290940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3880cde70 msgr2=0x7fa388097830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa387e00640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3880cde70 0x7fa388097830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa387e00640 1 -- 172.21.15.66:0/1178290940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3880cda90 msgr2=0x7fa3880aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa387e00640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3880cda90 0x7fa3880aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa387e00640 1 -- 172.21.15.66:0/1178290940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa380009b30 con 0x7fa38815a370 2024-03-30T00:51:45.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa387e00640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa38815a370 0x7fa38815c450 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fa3800af040 tx=0x7fa380019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:45.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa37f400640 1 -- 172.21.15.66:0/1178290940 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800b8640 con 0x7fa38815a370 2024-03-30T00:51:45.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa37f400640 1 -- 172.21.15.66:0/1178290940 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3800b87a0 con 0x7fa38815a370 2024-03-30T00:51:45.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa38e200640 1 -- 172.21.15.66:0/1178290940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa388157820 con 0x7fa38815a370 2024-03-30T00:51:45.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa37f400640 1 -- 172.21.15.66:0/1178290940 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800b8b10 con 0x7fa38815a370 2024-03-30T00:51:45.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.171+0000 7fa38e200640 1 -- 172.21.15.66:0/1178290940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3880abf60 con 0x7fa38815a370 2024-03-30T00:51:45.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.175+0000 7fa38e200640 1 -- 172.21.15.66:0/1178290940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa388157680 con 0x7fa38815a370 2024-03-30T00:51:45.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.175+0000 7fa37f400640 1 -- 172.21.15.66:0/1178290940 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa3800b8c70 con 0x7fa38815a370 2024-03-30T00:51:45.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.175+0000 7fa37f400640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa3640666c0 0x7fa364068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.175+0000 7fa37f400640 1 -- 172.21.15.66:0/1178290940 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fa38011f1e0 con 0x7fa38815a370 2024-03-30T00:51:45.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.175+0000 7fa387400640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa3640666c0 0x7fa364068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.175+0000 7fa387400640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa3640666c0 0x7fa364068b80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa3881568a0 tx=0x7fa378008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:45.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.179+0000 7fa37f400640 1 -- 172.21.15.66:0/1178290940 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa388157680 con 0x7fa38815a370 2024-03-30T00:51:45.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.343+0000 7fa38e200640 1 -- 172.21.15.66:0/1178290940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa388157680 con 0x7fa38815a370 2024-03-30T00:51:45.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.343+0000 7fa37f400640 1 -- 172.21.15.66:0/1178290940 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16575 (secure 0 0 0) 0x7fa388157680 con 0x7fa38815a370 2024-03-30T00:51:45.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:45.352 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":127,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:51:29.804902+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6834/3460522956":"2024-03-31T00:51:29.804880+0000","172.21.15.66:6836/3460522956":"2024-03-31T00:51:29.804880+0000","172.21.15.66:6836/2683418851":"2024-03-31T00:51:15.162460+0000","172.21.15.66:6834/2683418851":"2024-03-31T00:51:15.162460+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:51:45.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 -- 172.21.15.66:0/1178290940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa3640666c0 msgr2=0x7fa364068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa3640666c0 0x7fa364068b80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa3881568a0 tx=0x7fa378008040 comp rx=0 tx=0).stop 2024-03-30T00:51:45.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 -- 172.21.15.66:0/1178290940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa38815a370 msgr2=0x7fa38815c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa38815a370 0x7fa38815c450 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fa3800af040 tx=0x7fa380019700 comp rx=0 tx=0).stop 2024-03-30T00:51:45.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 -- 172.21.15.66:0/1178290940 shutdown_connections 2024-03-30T00:51:45.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa3640666c0 0x7fa364068b80 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa38815a370 0x7fa38815c450 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3880cde70 0x7fa388097830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 --2- 172.21.15.66:0/1178290940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3880cda90 0x7fa3880aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 -- 172.21.15.66:0/1178290940 >> 172.21.15.66:0/1178290940 conn(0x7fa388069f60 msgr2=0x7fa38815d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:45.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 -- 172.21.15.66:0/1178290940 shutdown_connections 2024-03-30T00:51:45.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.347+0000 7fa38e200640 1 -- 172.21.15.66:0/1178290940 wait complete. 2024-03-30T00:51:45.374 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T00:51:45.375 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:51:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f072bc00640 1 Processor -- start 2024-03-30T00:51:45.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f072bc00640 1 -- start start 2024-03-30T00:51:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f072bc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07240cc560 0x7f07240d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f072bc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f072406f8e0 0x7f072406e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f072bc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f072406e630 0x7f07240ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f072bc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0724160640 con 0x7f072406f8e0 2024-03-30T00:51:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f072bc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072406eaa0 con 0x7f072406e630 2024-03-30T00:51:45.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f072bc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072406b0b0 con 0x7f07240cc560 2024-03-30T00:51:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f071be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f072406f8e0 0x7f072406e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f071be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f072406f8e0 0x7f072406e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38152/0 (socket says 172.21.15.66:38152) 2024-03-30T00:51:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f071be00640 1 -- 172.21.15.66:0/2134106281 learned_addr learned my addr 172.21.15.66:0/2134106281 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f0729c00640 1 --2- 172.21.15.66:0/2134106281 >> v2:172.21.15.156:3300/0 conn(0x7f072406e630 0x7f07240ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f0729200640 1 --2- 172.21.15.66:0/2134106281 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07240cc560 0x7f07240d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f071be00640 1 -- 172.21.15.66:0/2134106281 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07240cc560 msgr2=0x7f07240d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f071be00640 1 --2- 172.21.15.66:0/2134106281 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07240cc560 0x7f07240d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f071be00640 1 -- 172.21.15.66:0/2134106281 >> v2:172.21.15.156:3300/0 conn(0x7f072406e630 msgr2=0x7f07240ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f071be00640 1 --2- 172.21.15.66:0/2134106281 >> v2:172.21.15.156:3300/0 conn(0x7f072406e630 0x7f07240ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f071be00640 1 -- 172.21.15.66:0/2134106281 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072406b390 con 0x7f072406f8e0 2024-03-30T00:51:45.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.507+0000 7f0729c00640 1 --2- 172.21.15.66:0/2134106281 >> v2:172.21.15.156:3300/0 conn(0x7f072406e630 0x7f07240ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:45.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f071be00640 1 --2- 172.21.15.66:0/2134106281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f072406f8e0 0x7f072406e0f0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f072406dac0 tx=0x7f070c01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bd115bc889d304 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:45.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f0723e00640 1 -- 172.21.15.66:0/2134106281 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0ab600 con 0x7f072406f8e0 2024-03-30T00:51:45.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f0723e00640 1 -- 172.21.15.66:0/2134106281 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c0a7050 con 0x7f072406f8e0 2024-03-30T00:51:45.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f0723e00640 1 -- 172.21.15.66:0/2134106281 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0ab980 con 0x7f072406f8e0 2024-03-30T00:51:45.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f072bc00640 1 -- 172.21.15.66:0/2134106281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f072406f8e0 msgr2=0x7f072406e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f072bc00640 1 --2- 172.21.15.66:0/2134106281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f072406f8e0 0x7f072406e0f0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f072406dac0 tx=0x7f070c01c230 comp rx=0 tx=0).stop 2024-03-30T00:51:45.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f072bc00640 1 -- 172.21.15.66:0/2134106281 shutdown_connections 2024-03-30T00:51:45.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f072bc00640 1 --2- 172.21.15.66:0/2134106281 >> v2:172.21.15.156:3300/0 conn(0x7f072406e630 0x7f07240ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f072bc00640 1 --2- 172.21.15.66:0/2134106281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f072406f8e0 0x7f072406e0f0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f072bc00640 1 --2- 172.21.15.66:0/2134106281 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07240cc560 0x7f07240d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f072bc00640 1 -- 172.21.15.66:0/2134106281 >> 172.21.15.66:0/2134106281 conn(0x7f07240d0540 msgr2=0x7f07240d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:45.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f072bc00640 1 -- 172.21.15.66:0/2134106281 shutdown_connections 2024-03-30T00:51:45.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.511+0000 7f072bc00640 1 -- 172.21.15.66:0/2134106281 wait complete. 2024-03-30T00:51:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f072bc00640 1 Processor -- start 2024-03-30T00:51:45.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f072bc00640 1 -- start start 2024-03-30T00:51:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f072bc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f072406e630 0x7f07240ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f072bc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f072406f8e0 0x7f072413fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f072bc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07240cc560 0x7f07240aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f072bc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0724152da0 con 0x7f07240cc560 2024-03-30T00:51:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f072bc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07241519c0 con 0x7f072406f8e0 2024-03-30T00:51:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f072bc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0724151ca0 con 0x7f072406e630 2024-03-30T00:51:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0729c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07240cc560 0x7f07240aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0729c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07240cc560 0x7f07240aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38154/0 (socket says 172.21.15.66:38154) 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0729c00640 1 -- 172.21.15.66:0/3836765490 learned_addr learned my addr 172.21.15.66:0/3836765490 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0729200640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f072406e630 0x7f07240ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f071be00640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f072406f8e0 0x7f072413fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0729c00640 1 -- 172.21.15.66:0/3836765490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f072406e630 msgr2=0x7f07240ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0729c00640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f072406e630 0x7f07240ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0729c00640 1 -- 172.21.15.66:0/3836765490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f072406f8e0 msgr2=0x7f072413fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0729c00640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f072406f8e0 0x7f072413fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0729c00640 1 -- 172.21.15.66:0/3836765490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c0094d0 con 0x7f07240cc560 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0729c00640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07240cc560 0x7f07240aff30 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f071c007e10 tx=0x7f071c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0721600640 1 -- 172.21.15.66:0/3836765490 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c003ba0 con 0x7f07240cc560 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0721600640 1 -- 172.21.15.66:0/3836765490 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c00f040 con 0x7f07240cc560 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f0721600640 1 -- 172.21.15.66:0/3836765490 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c00de80 con 0x7f07240cc560 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f072bc00640 1 -- 172.21.15.66:0/3836765490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0724151f80 con 0x7f07240cc560 2024-03-30T00:51:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f072bc00640 1 -- 172.21.15.66:0/3836765490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0724152540 con 0x7f07240cc560 2024-03-30T00:51:45.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.515+0000 7f072bc00640 1 -- 172.21.15.66:0/3836765490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0724152730 con 0x7f07240cc560 2024-03-30T00:51:45.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.519+0000 7f0721600640 1 -- 172.21.15.66:0/3836765490 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f071c004030 con 0x7f07240cc560 2024-03-30T00:51:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.519+0000 7f0721600640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07000666c0 0x7f0700068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.519+0000 7f0721600640 1 -- 172.21.15.66:0/3836765490 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f071c084660 con 0x7f07240cc560 2024-03-30T00:51:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.519+0000 7f0729200640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07000666c0 0x7f0700068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.519+0000 7f0729200640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07000666c0 0x7f0700068b80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0714009ac0 tx=0x7f0714009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:45.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.523+0000 7f0721600640 1 -- 172.21.15.66:0/3836765490 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f071c0517f0 con 0x7f07240cc560 2024-03-30T00:51:45.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.719+0000 7f072bc00640 1 -- 172.21.15.66:0/3836765490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0724066c30 con 0x7f07240cc560 2024-03-30T00:51:45.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.719+0000 7f0721600640 1 -- 172.21.15.66:0/3836765490 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4744 (secure 0 0 0) 0x7f071c054e10 con 0x7f07240cc560 2024-03-30T00:51:45.729 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:45.729 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":108,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:51:34.842265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9942":{"gid":9942,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/2774791553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2774791553},{"type":"v1","addr":"172.21.15.66:6836","nonce":2774791553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:51:45.729 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 108 2024-03-30T00:51:45.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 -- 172.21.15.66:0/3836765490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07000666c0 msgr2=0x7f0700068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07000666c0 0x7f0700068b80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0714009ac0 tx=0x7f0714009190 comp rx=0 tx=0).stop 2024-03-30T00:51:45.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 -- 172.21.15.66:0/3836765490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07240cc560 msgr2=0x7f07240aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07240cc560 0x7f07240aff30 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f071c007e10 tx=0x7f071c00d4a0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 -- 172.21.15.66:0/3836765490 shutdown_connections 2024-03-30T00:51:45.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f07000666c0 0x7f0700068b80 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07240cc560 0x7f07240aff30 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f072406f8e0 0x7f072413fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 --2- 172.21.15.66:0/3836765490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f072406e630 0x7f07240ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 -- 172.21.15.66:0/3836765490 >> 172.21.15.66:0/3836765490 conn(0x7f07240d0540 msgr2=0x7f07240cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:45.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 -- 172.21.15.66:0/3836765490 shutdown_connections 2024-03-30T00:51:45.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.723+0000 7f072bc00640 1 -- 172.21.15.66:0/3836765490 wait complete. 2024-03-30T00:51:45.750 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:51:45.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02e4000640 1 Processor -- start 2024-03-30T00:51:45.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02e4000640 1 -- start start 2024-03-30T00:51:45.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02e4000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02dc0cf2f0 0x7f02dc06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02e4000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f02dc0cb570 0x7f02dc0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02e4000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02dc0cb0e0 con 0x7f02dc15ca80 2024-03-30T00:51:45.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02e4000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02dc0cf7f0 con 0x7f02dc0cb570 2024-03-30T00:51:45.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02e4000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc0cfad0 con 0x7f02dc0cf2f0 2024-03-30T00:51:45.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02e1600640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f02dc15ca80 0x7f02dc06b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40236/0 (socket says 172.21.15.66:40236) 2024-03-30T00:51:45.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02e1600640 1 -- 172.21.15.66:0/164803261 learned_addr learned my addr 172.21.15.66:0/164803261 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:45.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02e0c00640 1 --2- 172.21.15.66:0/164803261 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02dc0cf2f0 0x7f02dc06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02e2000640 1 --2- 172.21.15.66:0/164803261 >> v2:172.21.15.156:3300/0 conn(0x7f02dc0cb570 0x7f02dc0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02dbe00640 1 -- 172.21.15.66:0/164803261 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174942324 0 0) 0x7f02dc0cb0e0 con 0x7f02dc15ca80 2024-03-30T00:51:45.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.883+0000 7f02dbe00640 1 -- 172.21.15.66:0/164803261 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02c4003530 con 0x7f02dc15ca80 2024-03-30T00:51:45.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.887+0000 7f02e2000640 1 -- 172.21.15.66:0/164803261 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02dc0cf2f0 msgr2=0x7f02dc06b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:45.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.887+0000 7f02dbe00640 1 -- 172.21.15.66:0/164803261 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1146369942 0 0) 0x7f02c4003530 con 0x7f02dc15ca80 2024-03-30T00:51:45.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.887+0000 7f02e2000640 1 --2- 172.21.15.66:0/164803261 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02dc0cf2f0 0x7f02dc06b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.887+0000 7f02e2000640 1 -- 172.21.15.66:0/164803261 >> v1:172.21.15.66:6789/0 conn(0x7f02dc15ca80 legacy=0x7f02dc06b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.887+0000 7f02e2000640 1 -- 172.21.15.66:0/164803261 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02dc0cad70 con 0x7f02dc0cb570 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.887+0000 7f02e2000640 1 --2- 172.21.15.66:0/164803261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02dc0cb570 0x7f02dc0cba20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009d90 tx=0x7f02d401f080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bda7b943a66cfc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.887+0000 7f02dbe00640 1 -- 172.21.15.66:0/164803261 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0001680 con 0x7f02dc0cb570 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.887+0000 7f02dbe00640 1 -- 172.21.15.66:0/164803261 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d40a8bd0 con 0x7f02dc0cb570 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.887+0000 7f02dbe00640 1 -- 172.21.15.66:0/164803261 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d40b0440 con 0x7f02dc0cb570 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.887+0000 7f02e4000640 1 -- 172.21.15.66:0/164803261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02dc0cb570 msgr2=0x7f02dc0cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.887+0000 7f02e4000640 1 --2- 172.21.15.66:0/164803261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02dc0cb570 0x7f02dc0cba20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009d90 tx=0x7f02d401f080 comp rx=0 tx=0).stop 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 -- 172.21.15.66:0/164803261 shutdown_connections 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 --2- 172.21.15.66:0/164803261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02dc0cb570 0x7f02dc0cba20 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 --2- 172.21.15.66:0/164803261 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02dc0cf2f0 0x7f02dc06b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 -- 172.21.15.66:0/164803261 >> 172.21.15.66:0/164803261 conn(0x7f02dc0b2090 msgr2=0x7f02dc0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 -- 172.21.15.66:0/164803261 shutdown_connections 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 -- 172.21.15.66:0/164803261 wait complete. 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 Processor -- start 2024-03-30T00:51:45.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 -- start start 2024-03-30T00:51:45.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc0cb570 0x7f02dc15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02dc0cf2f0 0x7f02dc0b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02dc15ca80 0x7f02dc09da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc09d590 con 0x7f02dc0cb570 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc09ccd0 con 0x7f02dc0cf2f0 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e4000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc09a6d0 con 0x7f02dc15ca80 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc0cb570 0x7f02dc15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc0cb570 0x7f02dc15a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38168/0 (socket says 172.21.15.66:38168) 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e1600640 1 -- 172.21.15.66:0/3421081524 learned_addr learned my addr 172.21.15.66:0/3421081524 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e0c00640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02dc0cf2f0 0x7f02dc0b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e2000640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02dc15ca80 0x7f02dc09da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e1600640 1 -- 172.21.15.66:0/3421081524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02dc15ca80 msgr2=0x7f02dc09da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e1600640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02dc15ca80 0x7f02dc09da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e1600640 1 -- 172.21.15.66:0/3421081524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02dc0cf2f0 msgr2=0x7f02dc0b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e1600640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02dc0cf2f0 0x7f02dc0b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e1600640 1 -- 172.21.15.66:0/3421081524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d4009a70 con 0x7f02dc0cb570 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.891+0000 7f02e1600640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc0cb570 0x7f02dc15a4f0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f02d000c780 tx=0x7f02d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:45.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.895+0000 7f02d9600640 1 -- 172.21.15.66:0/3421081524 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0005ae0 con 0x7f02dc0cb570 2024-03-30T00:51:45.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.895+0000 7f02d9600640 1 -- 172.21.15.66:0/3421081524 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d0005c40 con 0x7f02dc0cb570 2024-03-30T00:51:45.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.895+0000 7f02e4000640 1 -- 172.21.15.66:0/3421081524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02dc09cf40 con 0x7f02dc0cb570 2024-03-30T00:51:45.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.895+0000 7f02d9600640 1 -- 172.21.15.66:0/3421081524 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0007070 con 0x7f02dc0cb570 2024-03-30T00:51:45.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.895+0000 7f02e4000640 1 -- 172.21.15.66:0/3421081524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02dc09a310 con 0x7f02dc0cb570 2024-03-30T00:51:45.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.895+0000 7f02e4000640 1 -- 172.21.15.66:0/3421081524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02dc066c30 con 0x7f02dc0cb570 2024-03-30T00:51:45.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.895+0000 7f02d9600640 1 -- 172.21.15.66:0/3421081524 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f02d00071f0 con 0x7f02dc0cb570 2024-03-30T00:51:45.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.895+0000 7f02d9600640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02bc066690 0x7f02bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:45.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.895+0000 7f02d9600640 1 -- 172.21.15.66:0/3421081524 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f02d0083f80 con 0x7f02dc0cb570 2024-03-30T00:51:45.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.895+0000 7f02e0c00640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02bc066690 0x7f02bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:45.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.895+0000 7f02e0c00640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02bc066690 0x7f02bc068b50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0c61e0 tx=0x7f02c8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:45.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:45.899+0000 7f02d9600640 1 -- 172.21.15.66:0/3421081524 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f02d00511a0 con 0x7f02dc0cb570 2024-03-30T00:51:46.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.095+0000 7f02e4000640 1 -- 172.21.15.66:0/3421081524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02dc0b2bb0 con 0x7f02dc0cb570 2024-03-30T00:51:46.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.095+0000 7f02d9600640 1 -- 172.21.15.66:0/3421081524 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4744 (secure 0 0 0) 0x7f02d00547c0 con 0x7f02dc0cb570 2024-03-30T00:51:46.104 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:46.104 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":108,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T00:50:38.040525+0000","modified":"2024-03-30T00:51:34.842265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9942":{"gid":9942,"name":"a","rank":0,"incarnation":105,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6836/2774791553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2774791553},{"type":"v1","addr":"172.21.15.66:6836","nonce":2774791553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-30T00:51:46.105 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 108 2024-03-30T00:51:46.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 -- 172.21.15.66:0/3421081524 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02bc066690 msgr2=0x7f02bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02bc066690 0x7f02bc068b50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0c61e0 tx=0x7f02c8009190 comp rx=0 tx=0).stop 2024-03-30T00:51:46.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 -- 172.21.15.66:0/3421081524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc0cb570 msgr2=0x7f02dc15a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc0cb570 0x7f02dc15a4f0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f02d000c780 tx=0x7f02d000cc40 comp rx=0 tx=0).stop 2024-03-30T00:51:46.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 -- 172.21.15.66:0/3421081524 shutdown_connections 2024-03-30T00:51:46.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02dc15ca80 0x7f02dc09da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02dc0cf2f0 0x7f02dc0b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f02bc066690 0x7f02bc068b50 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 --2- 172.21.15.66:0/3421081524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02dc0cb570 0x7f02dc15a4f0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 -- 172.21.15.66:0/3421081524 >> 172.21.15.66:0/3421081524 conn(0x7f02dc0b2090 msgr2=0x7f02dc0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:46.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 -- 172.21.15.66:0/3421081524 shutdown_connections 2024-03-30T00:51:46.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.099+0000 7f02e4000640 1 -- 172.21.15.66:0/3421081524 wait complete. 2024-03-30T00:51:46.125 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:51:46.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.255+0000 7fab9e200640 1 Processor -- start 2024-03-30T00:51:46.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.255+0000 7fab9e200640 1 -- start start 2024-03-30T00:51:46.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab9e200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fab980cda90 0x7fab980ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab9e200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fab980cde70 0x7fab980cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab9e200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9815a990 con 0x7fab98159c60 2024-03-30T00:51:46.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab9e200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab980b1b80 con 0x7fab980cda90 2024-03-30T00:51:46.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab9e200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab980cd3e0 con 0x7fab980cde70 2024-03-30T00:51:46.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab97400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fab980cda90 0x7fab980ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab97400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fab980cda90 0x7fab980ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38172/0 (socket says 172.21.15.66:38172) 2024-03-30T00:51:46.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab97400640 1 -- 172.21.15.66:0/3546855980 learned_addr learned my addr 172.21.15.66:0/3546855980 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:46.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab8ea00640 1 --2- 172.21.15.66:0/3546855980 >> v2:172.21.15.156:3300/0 conn(0x7fab980cde70 0x7fab980cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab97400640 1 -- 172.21.15.66:0/3546855980 >> v2:172.21.15.156:3300/0 conn(0x7fab980cde70 msgr2=0x7fab980cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:46.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab97400640 1 --2- 172.21.15.66:0/3546855980 >> v2:172.21.15.156:3300/0 conn(0x7fab980cde70 0x7fab980cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab97400640 1 -- 172.21.15.66:0/3546855980 >> v1:172.21.15.156:6789/0 conn(0x7fab98159c60 legacy=0x7fab9815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab97400640 1 -- 172.21.15.66:0/3546855980 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab980b11e0 con 0x7fab980cda90 2024-03-30T00:51:46.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.259+0000 7fab97400640 1 --2- 172.21.15.66:0/3546855980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab980cda90 0x7fab980ce520 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fab980cc320 tx=0x7fab8801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84717609d8354ea4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:46.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab96a00640 1 -- 172.21.15.66:0/3546855980 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab880a5730 con 0x7fab980cda90 2024-03-30T00:51:46.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab96a00640 1 -- 172.21.15.66:0/3546855980 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab880a5890 con 0x7fab980cda90 2024-03-30T00:51:46.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab96a00640 1 -- 172.21.15.66:0/3546855980 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab880a5c20 con 0x7fab980cda90 2024-03-30T00:51:46.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 -- 172.21.15.66:0/3546855980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab980cda90 msgr2=0x7fab980ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 --2- 172.21.15.66:0/3546855980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab980cda90 0x7fab980ce520 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fab980cc320 tx=0x7fab8801c960 comp rx=0 tx=0).stop 2024-03-30T00:51:46.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 -- 172.21.15.66:0/3546855980 shutdown_connections 2024-03-30T00:51:46.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 --2- 172.21.15.66:0/3546855980 >> v2:172.21.15.156:3300/0 conn(0x7fab980cde70 0x7fab980cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 --2- 172.21.15.66:0/3546855980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab980cda90 0x7fab980ce520 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 -- 172.21.15.66:0/3546855980 >> 172.21.15.66:0/3546855980 conn(0x7fab98069f60 msgr2=0x7fab9806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:46.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 -- 172.21.15.66:0/3546855980 shutdown_connections 2024-03-30T00:51:46.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 -- 172.21.15.66:0/3546855980 wait complete. 2024-03-30T00:51:46.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 Processor -- start 2024-03-30T00:51:46.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 -- start start 2024-03-30T00:51:46.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab980cda90 0x7fab981558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab980cde70 0x7fab98163620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab98159c60 0x7fab98163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab98166150 con 0x7fab98159c60 2024-03-30T00:51:46.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab98165540 con 0x7fab980cde70 2024-03-30T00:51:46.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab9e200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab98165820 con 0x7fab980cda90 2024-03-30T00:51:46.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab97e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab98159c60 0x7fab98163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab97e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab98159c60 0x7fab98163b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38178/0 (socket says 172.21.15.66:38178) 2024-03-30T00:51:46.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.263+0000 7fab97e00640 1 -- 172.21.15.66:0/457844591 learned_addr learned my addr 172.21.15.66:0/457844591 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:46.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab97400640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab980cda90 0x7fab981558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab8ea00640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab980cde70 0x7fab98163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab97e00640 1 -- 172.21.15.66:0/457844591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab980cda90 msgr2=0x7fab981558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab97e00640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab980cda90 0x7fab981558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab97e00640 1 -- 172.21.15.66:0/457844591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab980cde70 msgr2=0x7fab98163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab97e00640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab980cde70 0x7fab98163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab97e00640 1 -- 172.21.15.66:0/457844591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab88009b30 con 0x7fab98159c60 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab8ea00640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab980cde70 0x7fab98163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab97e00640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab98159c60 0x7fab98163b60 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fab9000c430 tx=0x7fab9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab8fe00640 1 -- 172.21.15.66:0/457844591 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab90005220 con 0x7fab98159c60 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab8fe00640 1 -- 172.21.15.66:0/457844591 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab90005380 con 0x7fab98159c60 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab9e200640 1 -- 172.21.15.66:0/457844591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab98165b00 con 0x7fab98159c60 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab8fe00640 1 -- 172.21.15.66:0/457844591 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab90007270 con 0x7fab98159c60 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab9e200640 1 -- 172.21.15.66:0/457844591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab98155df0 con 0x7fab98159c60 2024-03-30T00:51:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab9e200640 1 -- 172.21.15.66:0/457844591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab98066c30 con 0x7fab98159c60 2024-03-30T00:51:46.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab8fe00640 1 -- 172.21.15.66:0/457844591 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fab900073d0 con 0x7fab98159c60 2024-03-30T00:51:46.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab8fe00640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab74066690 0x7fab74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.267+0000 7fab8fe00640 1 -- 172.21.15.66:0/457844591 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fab90083940 con 0x7fab98159c60 2024-03-30T00:51:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.271+0000 7fab97400640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab74066690 0x7fab74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.271+0000 7fab97400640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab74066690 0x7fab74068b50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fab880ae000 tx=0x7fab880a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:46.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.271+0000 7fab8fe00640 1 -- 172.21.15.66:0/457844591 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fab90050a20 con 0x7fab98159c60 2024-03-30T00:51:46.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 -- 172.21.15.66:0/457844591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab980b3170 con 0x7fab98159c60 2024-03-30T00:51:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab8fe00640 1 -- 172.21.15.66:0/457844591 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16575 (secure 0 0 0) 0x7fab90054040 con 0x7fab98159c60 2024-03-30T00:51:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:46.443 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":127,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:51:29.804902+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:50:35.696965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-30T00:50:37.037165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6834/3460522956":"2024-03-31T00:51:29.804880+0000","172.21.15.66:6836/3460522956":"2024-03-31T00:51:29.804880+0000","172.21.15.66:6836/2683418851":"2024-03-31T00:51:15.162460+0000","172.21.15.66:6834/2683418851":"2024-03-31T00:51:15.162460+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:51:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 -- 172.21.15.66:0/457844591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab74066690 msgr2=0x7fab74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab74066690 0x7fab74068b50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fab880ae000 tx=0x7fab880a7070 comp rx=0 tx=0).stop 2024-03-30T00:51:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 -- 172.21.15.66:0/457844591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab98159c60 msgr2=0x7fab98163b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab98159c60 0x7fab98163b60 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fab9000c430 tx=0x7fab9000c8f0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 -- 172.21.15.66:0/457844591 shutdown_connections 2024-03-30T00:51:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fab74066690 0x7fab74068b50 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab98159c60 0x7fab98163b60 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab980cde70 0x7fab98163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 --2- 172.21.15.66:0/457844591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab980cda90 0x7fab981558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 -- 172.21.15.66:0/457844591 >> 172.21.15.66:0/457844591 conn(0x7fab98069f60 msgr2=0x7fab980cb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 -- 172.21.15.66:0/457844591 shutdown_connections 2024-03-30T00:51:46.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.435+0000 7fab9e200640 1 -- 172.21.15.66:0/457844591 wait complete. 2024-03-30T00:51:46.465 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T00:51:46.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.595+0000 7fe858200640 1 Processor -- start 2024-03-30T00:51:46.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe858200640 1 -- start start 2024-03-30T00:51:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe858200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe8500cda90 0x7fe8500ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe858200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe85015a370 0x7fe8500cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe858200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85015a990 con 0x7fe8500cde70 2024-03-30T00:51:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe858200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8500b1b80 con 0x7fe8500cda90 2024-03-30T00:51:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe858200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500cd3e0 con 0x7fe85015a370 2024-03-30T00:51:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe855800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe8500cda90 0x7fe8500ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe855800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe8500cda90 0x7fe8500ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38186/0 (socket says 172.21.15.66:38186) 2024-03-30T00:51:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe855800640 1 -- 172.21.15.66:0/4012459377 learned_addr learned my addr 172.21.15.66:0/4012459377 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe856200640 1 --2- 172.21.15.66:0/4012459377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe85015a370 0x7fe8500cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe855800640 1 -- 172.21.15.66:0/4012459377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe85015a370 msgr2=0x7fe8500cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:46.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe855800640 1 --2- 172.21.15.66:0/4012459377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe85015a370 0x7fe8500cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe855800640 1 -- 172.21.15.66:0/4012459377 >> v1:172.21.15.156:6789/0 conn(0x7fe8500cde70 legacy=0x7fe850159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.599+0000 7fe855800640 1 -- 172.21.15.66:0/4012459377 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8500b11e0 con 0x7fe8500cda90 2024-03-30T00:51:46.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe855800640 1 --2- 172.21.15.66:0/4012459377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500cda90 0x7fe8500ce520 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe84400bb10 tx=0x7fe84401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68a9c0031e928ac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:46.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe84fe00640 1 -- 172.21.15.66:0/4012459377 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8440a7b80 con 0x7fe8500cda90 2024-03-30T00:51:46.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe84fe00640 1 -- 172.21.15.66:0/4012459377 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8440a7ce0 con 0x7fe8500cda90 2024-03-30T00:51:46.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe84fe00640 1 -- 172.21.15.66:0/4012459377 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8440af660 con 0x7fe8500cda90 2024-03-30T00:51:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe858200640 1 -- 172.21.15.66:0/4012459377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500cda90 msgr2=0x7fe8500ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe858200640 1 --2- 172.21.15.66:0/4012459377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500cda90 0x7fe8500ce520 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe84400bb10 tx=0x7fe84401eba0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe858200640 1 -- 172.21.15.66:0/4012459377 shutdown_connections 2024-03-30T00:51:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe858200640 1 --2- 172.21.15.66:0/4012459377 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe85015a370 0x7fe8500cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe858200640 1 --2- 172.21.15.66:0/4012459377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500cda90 0x7fe8500ce520 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe858200640 1 -- 172.21.15.66:0/4012459377 >> 172.21.15.66:0/4012459377 conn(0x7fe850069f60 msgr2=0x7fe85006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe858200640 1 -- 172.21.15.66:0/4012459377 shutdown_connections 2024-03-30T00:51:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe858200640 1 -- 172.21.15.66:0/4012459377 wait complete. 2024-03-30T00:51:46.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe858200640 1 Processor -- start 2024-03-30T00:51:46.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.603+0000 7fe858200640 1 -- start start 2024-03-30T00:51:46.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe858200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500cda90 0x7fe8500ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe858200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500cde70 0x7fe8500979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe858200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe85015a370 0x7fe850162620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe858200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe850097f30 con 0x7fe8500cda90 2024-03-30T00:51:46.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe858200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500acc00 con 0x7fe8500cde70 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe858200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500ad090 con 0x7fe85015a370 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe855800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500cda90 0x7fe8500ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe855800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500cda90 0x7fe8500ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38194/0 (socket says 172.21.15.66:38194) 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe855800640 1 -- 172.21.15.66:0/184149590 learned_addr learned my addr 172.21.15.66:0/184149590 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe854e00640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500cde70 0x7fe8500979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe856200640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe85015a370 0x7fe850162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe855800640 1 -- 172.21.15.66:0/184149590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe85015a370 msgr2=0x7fe850162620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe855800640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe85015a370 0x7fe850162620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe855800640 1 -- 172.21.15.66:0/184149590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500cde70 msgr2=0x7fe8500979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe855800640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500cde70 0x7fe8500979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe855800640 1 -- 172.21.15.66:0/184149590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84400b770 con 0x7fe8500cda90 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe855800640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500cda90 0x7fe8500ab010 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fe84400b2c0 tx=0x7fe84401b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe84d600640 1 -- 172.21.15.66:0/184149590 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8440afe30 con 0x7fe8500cda90 2024-03-30T00:51:46.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe84d600640 1 -- 172.21.15.66:0/184149590 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8440ba440 con 0x7fe8500cda90 2024-03-30T00:51:46.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe84d600640 1 -- 172.21.15.66:0/184149590 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8440ba7f0 con 0x7fe8500cda90 2024-03-30T00:51:46.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe858200640 1 -- 172.21.15.66:0/184149590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85015c7e0 con 0x7fe8500cda90 2024-03-30T00:51:46.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe858200640 1 -- 172.21.15.66:0/184149590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8501634d0 con 0x7fe8500cda90 2024-03-30T00:51:46.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe858200640 1 -- 172.21.15.66:0/184149590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8501636c0 con 0x7fe8500cda90 2024-03-30T00:51:46.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.607+0000 7fe84d600640 1 -- 172.21.15.66:0/184149590 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe8440b93e0 con 0x7fe8500cda90 2024-03-30T00:51:46.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.611+0000 7fe84d600640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8300666c0 0x7fe830068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:46.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.611+0000 7fe84d600640 1 -- 172.21.15.66:0/184149590 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fe84411e880 con 0x7fe8500cda90 2024-03-30T00:51:46.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.611+0000 7fe854e00640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8300666c0 0x7fe830068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:46.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.611+0000 7fe854e00640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8300666c0 0x7fe830068b80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe8500ac650 tx=0x7fe83c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:46.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.615+0000 7fe84d600640 1 -- 172.21.15.66:0/184149590 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe8501636c0 con 0x7fe8500cda90 2024-03-30T00:51:46.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.819+0000 7fe858200640 1 -- 172.21.15.66:0/184149590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe8501636c0 con 0x7fe8500cda90 2024-03-30T00:51:46.830 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T00:51:46.823+0000 7f1c7c800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T00:51:46.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.835+0000 7fe84d600640 1 -- 172.21.15.66:0/184149590 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v109) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe8501636c0 con 0x7fe8500cda90 2024-03-30T00:51:46.844 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T00:51:46.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 -- 172.21.15.66:0/184149590 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8300666c0 msgr2=0x7fe830068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8300666c0 0x7fe830068b80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe8500ac650 tx=0x7fe83c009040 comp rx=0 tx=0).stop 2024-03-30T00:51:46.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 -- 172.21.15.66:0/184149590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500cda90 msgr2=0x7fe8500ab010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:46.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500cda90 0x7fe8500ab010 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fe84400b2c0 tx=0x7fe84401b940 comp rx=0 tx=0).stop 2024-03-30T00:51:46.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 -- 172.21.15.66:0/184149590 shutdown_connections 2024-03-30T00:51:46.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe85015a370 0x7fe850162620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8500cde70 0x7fe8500979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fe8300666c0 0x7fe830068b80 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 --2- 172.21.15.66:0/184149590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500cda90 0x7fe8500ab010 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:46.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 -- 172.21.15.66:0/184149590 >> 172.21.15.66:0/184149590 conn(0x7fe850069f60 msgr2=0x7fe85006d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:46.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 -- 172.21.15.66:0/184149590 shutdown_connections 2024-03-30T00:51:46.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:46.839+0000 7fe858200640 1 -- 172.21.15.66:0/184149590 wait complete. 2024-03-30T00:51:46.867 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T00:51:46.895 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T00:51:47.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.003+0000 7f767ca00640 1 Processor -- start 2024-03-30T00:51:47.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.003+0000 7f767ca00640 1 -- start start 2024-03-30T00:51:47.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.003+0000 7f767ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f767815c6b0 0x7f76780d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:47.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.003+0000 7f767ca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f767806cb90 0x7f767815ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:47.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.003+0000 7f767ca00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767815cfd0 con 0x7f76780cdef0 2024-03-30T00:51:47.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.003+0000 7f767ca00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767806b1f0 con 0x7f767806cb90 2024-03-30T00:51:47.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.003+0000 7f767ca00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767815d250 con 0x7f767815c6b0 2024-03-30T00:51:47.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.003+0000 7f766fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f767806cb90 0x7f767815ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:47.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.003+0000 7f766fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f767806cb90 0x7f767815ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38218/0 (socket says 172.21.15.66:38218) 2024-03-30T00:51:47.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.003+0000 7f766fe00640 1 -- 172.21.15.66:0/895842936 learned_addr learned my addr 172.21.15.66:0/895842936 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:47.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.003+0000 7f7676800640 1 --2- 172.21.15.66:0/895842936 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f767815c6b0 0x7f76780d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:47.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f766fe00640 1 -- 172.21.15.66:0/895842936 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f767815c6b0 msgr2=0x7f76780d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:47.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f766fe00640 1 --2- 172.21.15.66:0/895842936 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f767815c6b0 0x7f76780d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:47.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f766fe00640 1 -- 172.21.15.66:0/895842936 >> v1:172.21.15.156:6789/0 conn(0x7f76780cdef0 legacy=0x7f767806b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:47.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f766fe00640 1 -- 172.21.15.66:0/895842936 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767815d530 con 0x7f767806cb90 2024-03-30T00:51:47.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f766fe00640 1 --2- 172.21.15.66:0/895842936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f767806cb90 0x7f767815ca90 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f7660009870 tx=0x7f766001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acf1ad53a9a65f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f7675e00640 1 -- 172.21.15.66:0/895842936 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600a95f0 con 0x7f767806cb90 2024-03-30T00:51:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f7675e00640 1 -- 172.21.15.66:0/895842936 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76600a9750 con 0x7f767806cb90 2024-03-30T00:51:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f7675e00640 1 -- 172.21.15.66:0/895842936 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600a9ae0 con 0x7f767806cb90 2024-03-30T00:51:47.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f767ca00640 1 -- 172.21.15.66:0/895842936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f767806cb90 msgr2=0x7f767815ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f767ca00640 1 --2- 172.21.15.66:0/895842936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f767806cb90 0x7f767815ca90 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f7660009870 tx=0x7f766001c530 comp rx=0 tx=0).stop 2024-03-30T00:51:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f767ca00640 1 -- 172.21.15.66:0/895842936 shutdown_connections 2024-03-30T00:51:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f767ca00640 1 --2- 172.21.15.66:0/895842936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f767806cb90 0x7f767815ca90 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f767ca00640 1 --2- 172.21.15.66:0/895842936 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f767815c6b0 0x7f76780d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f767ca00640 1 -- 172.21.15.66:0/895842936 >> 172.21.15.66:0/895842936 conn(0x7f7678069d90 msgr2=0x7f767806a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f767ca00640 1 -- 172.21.15.66:0/895842936 shutdown_connections 2024-03-30T00:51:47.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.007+0000 7f767ca00640 1 -- 172.21.15.66:0/895842936 wait complete. 2024-03-30T00:51:47.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f767ca00640 1 Processor -- start 2024-03-30T00:51:47.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f767ca00640 1 -- start start 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f767ca00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f767806cb90 0x7f767815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f767ca00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76780cdef0 0x7f767814fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f767ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f767815c6b0 0x7f76780b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f767ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767815a960 con 0x7f767815c6b0 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f767ca00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767815ad80 con 0x7f767806cb90 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f767ca00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7678159ce0 con 0x7f76780cdef0 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f7677200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f767815c6b0 0x7f76780b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f7676800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f767806cb90 0x7f767815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f7677200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f767815c6b0 0x7f76780b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38222/0 (socket says 172.21.15.66:38222) 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f7677200640 1 -- 172.21.15.66:0/1152810482 learned_addr learned my addr 172.21.15.66:0/1152810482 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f766fe00640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76780cdef0 0x7f767814fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:47.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f7677200640 1 -- 172.21.15.66:0/1152810482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76780cdef0 msgr2=0x7f767814fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:47.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f7677200640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76780cdef0 0x7f767814fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:47.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f7677200640 1 -- 172.21.15.66:0/1152810482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f767806cb90 msgr2=0x7f767815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:47.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f7677200640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f767806cb90 0x7f767815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:47.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f7677200640 1 -- 172.21.15.66:0/1152810482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76600094d0 con 0x7f767815c6b0 2024-03-30T00:51:47.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f7677200640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f767815c6b0 0x7f76780b0360 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f767000c770 tx=0x7f767000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:47.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f766ea00640 1 -- 172.21.15.66:0/1152810482 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670003b80 con 0x7f767815c6b0 2024-03-30T00:51:47.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f766ea00640 1 -- 172.21.15.66:0/1152810482 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7670003ce0 con 0x7f767815c6b0 2024-03-30T00:51:47.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f767ca00640 1 -- 172.21.15.66:0/1152810482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7678159fc0 con 0x7f767815c6b0 2024-03-30T00:51:47.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.011+0000 7f766ea00640 1 -- 172.21.15.66:0/1152810482 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767000aa90 con 0x7f767815c6b0 2024-03-30T00:51:47.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.015+0000 7f767ca00640 1 -- 172.21.15.66:0/1152810482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767815a580 con 0x7f767815c6b0 2024-03-30T00:51:47.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.015+0000 7f767ca00640 1 -- 172.21.15.66:0/1152810482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767815aaa0 con 0x7f767815c6b0 2024-03-30T00:51:47.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.015+0000 7f766ea00640 1 -- 172.21.15.66:0/1152810482 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f767000abf0 con 0x7f767815c6b0 2024-03-30T00:51:47.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.015+0000 7f766ea00640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7654066690 0x7f7654068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:47.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.015+0000 7f766ea00640 1 -- 172.21.15.66:0/1152810482 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f7670084040 con 0x7f767815c6b0 2024-03-30T00:51:47.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.015+0000 7f7676800640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7654066690 0x7f7654068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:47.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.015+0000 7f7676800640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7654066690 0x7f7654068b50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f76680079c0 tx=0x7f7668008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:47.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.019+0000 7f766ea00640 1 -- 172.21.15.66:0/1152810482 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f76700512b0 con 0x7f767815c6b0 2024-03-30T00:51:47.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.219+0000 7f767ca00640 1 -- 172.21.15.66:0/1152810482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7678073e00 con 0x7f767815c6b0 2024-03-30T00:51:47.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.839+0000 7f766ea00640 1 -- 172.21.15.66:0/1152810482 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v110) v1 ==== 104+0+0 (secure 0 0 0) 0x7f76700548d0 con 0x7f767815c6b0 2024-03-30T00:51:47.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.839+0000 7f767ca00640 1 -- 172.21.15.66:0/1152810482 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7654066690 msgr2=0x7f7654068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:47.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.839+0000 7f767ca00640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7654066690 0x7f7654068b50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f76680079c0 tx=0x7f7668008040 comp rx=0 tx=0).stop 2024-03-30T00:51:47.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.839+0000 7f767ca00640 1 -- 172.21.15.66:0/1152810482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f767815c6b0 msgr2=0x7f76780b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:47.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.839+0000 7f767ca00640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f767815c6b0 0x7f76780b0360 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f767000c770 tx=0x7f767000cc30 comp rx=0 tx=0).stop 2024-03-30T00:51:47.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.843+0000 7f767ca00640 1 -- 172.21.15.66:0/1152810482 shutdown_connections 2024-03-30T00:51:47.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.843+0000 7f767ca00640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7654066690 0x7f7654068b50 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:47.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.843+0000 7f767ca00640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f767815c6b0 0x7f76780b0360 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:47.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.843+0000 7f767ca00640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76780cdef0 0x7f767814fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:47.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.843+0000 7f767ca00640 1 --2- 172.21.15.66:0/1152810482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f767806cb90 0x7f767815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:47.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.843+0000 7f767ca00640 1 -- 172.21.15.66:0/1152810482 >> 172.21.15.66:0/1152810482 conn(0x7f7678069d90 msgr2=0x7f7678155970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:47.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.843+0000 7f767ca00640 1 -- 172.21.15.66:0/1152810482 shutdown_connections 2024-03-30T00:51:47.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:47.843+0000 7f767ca00640 1 -- 172.21.15.66:0/1152810482 wait complete. 2024-03-30T00:51:47.872 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T00:51:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.055+0000 7f6523e00640 1 Processor -- start 2024-03-30T00:51:48.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.055+0000 7f6523e00640 1 -- start start 2024-03-30T00:51:48.064 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:51:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.055+0000 7f6523e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f651c15ba00 0x7f651c0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.055+0000 7f6523e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f651c0cecf0 0x7f651c1579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.055+0000 7f6523e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c0cbf80 con 0x7f651c0acaf0 2024-03-30T00:51:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.055+0000 7f6523e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c0b2be0 con 0x7f651c15ba00 2024-03-30T00:51:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.055+0000 7f6523e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c0cc820 con 0x7f651c0cecf0 2024-03-30T00:51:48.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.055+0000 7f6521400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f651c0acaf0 0x7f651c0b24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:40248/0 (socket says 172.21.15.66:40248) 2024-03-30T00:51:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.055+0000 7f6521400640 1 -- 172.21.15.66:0/2518145236 learned_addr learned my addr 172.21.15.66:0/2518145236 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.055+0000 7f6520a00640 1 --2- 172.21.15.66:0/2518145236 >> v2:172.21.15.66:3300/0 conn(0x7f651c15ba00 0x7f651c0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6521e00640 1 --2- 172.21.15.66:0/2518145236 >> v2:172.21.15.156:3300/0 conn(0x7f651c0cecf0 0x7f651c1579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f651be00640 1 -- 172.21.15.66:0/2518145236 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408258312 0 0) 0x7f651c0cbf80 con 0x7f651c0acaf0 2024-03-30T00:51:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6520a00640 1 -- 172.21.15.66:0/2518145236 >> v2:172.21.15.156:3300/0 conn(0x7f651c0cecf0 msgr2=0x7f651c1579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6520a00640 1 --2- 172.21.15.66:0/2518145236 >> v2:172.21.15.156:3300/0 conn(0x7f651c0cecf0 0x7f651c1579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6520a00640 1 -- 172.21.15.66:0/2518145236 >> v1:172.21.15.66:6789/0 conn(0x7f651c0acaf0 legacy=0x7f651c0b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6520a00640 1 -- 172.21.15.66:0/2518145236 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c0cf110 con 0x7f651c15ba00 2024-03-30T00:51:48.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6521e00640 1 --2- 172.21.15.66:0/2518145236 >> v2:172.21.15.156:3300/0 conn(0x7f651c0cecf0 0x7f651c1579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:48.068 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6520a00640 1 --2- 172.21.15.66:0/2518145236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f651c15ba00 0x7f651c0ce7b0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f651c0ad9c0 tx=0x7f650801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572a235c4247ee3f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f651be00640 1 -- 172.21.15.66:0/2518145236 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65080ac620 con 0x7f651c15ba00 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f651be00640 1 -- 172.21.15.66:0/2518145236 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65080ac780 con 0x7f651c15ba00 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f651be00640 1 -- 172.21.15.66:0/2518145236 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65080acb10 con 0x7f651c15ba00 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6523e00640 1 -- 172.21.15.66:0/2518145236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f651c15ba00 msgr2=0x7f651c0ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6523e00640 1 --2- 172.21.15.66:0/2518145236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f651c15ba00 0x7f651c0ce7b0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f651c0ad9c0 tx=0x7f650801c3c0 comp rx=0 tx=0).stop 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6523e00640 1 -- 172.21.15.66:0/2518145236 shutdown_connections 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6523e00640 1 --2- 172.21.15.66:0/2518145236 >> v2:172.21.15.156:3300/0 conn(0x7f651c0cecf0 0x7f651c1579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6523e00640 1 --2- 172.21.15.66:0/2518145236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f651c15ba00 0x7f651c0ce7b0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6523e00640 1 -- 172.21.15.66:0/2518145236 >> 172.21.15.66:0/2518145236 conn(0x7f651c06a190 msgr2=0x7f651c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.059+0000 7f6523e00640 1 -- 172.21.15.66:0/2518145236 shutdown_connections 2024-03-30T00:51:48.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6523e00640 1 -- 172.21.15.66:0/2518145236 wait complete. 2024-03-30T00:51:48.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6523e00640 1 Processor -- start 2024-03-30T00:51:48.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6523e00640 1 -- start start 2024-03-30T00:51:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6523e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f651c0acaf0 0x7f651c159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6523e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c0cecf0 0x7f651c15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6523e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f651c15ba00 0x7f651c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6523e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0c9980 con 0x7f651c15ba00 2024-03-30T00:51:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6523e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c15d420 con 0x7f651c0cecf0 2024-03-30T00:51:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6523e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c156200 con 0x7f651c0acaf0 2024-03-30T00:51:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6521e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f651c15ba00 0x7f651c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6521e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f651c15ba00 0x7f651c155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38232/0 (socket says 172.21.15.66:38232) 2024-03-30T00:51:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6521e00640 1 -- 172.21.15.66:0/3554656548 learned_addr learned my addr 172.21.15.66:0/3554656548 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6520a00640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c0cecf0 0x7f651c15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:48.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6521400640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f651c0acaf0 0x7f651c159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6521e00640 1 -- 172.21.15.66:0/3554656548 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f651c0acaf0 msgr2=0x7f651c159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6521e00640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f651c0acaf0 0x7f651c159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6521e00640 1 -- 172.21.15.66:0/3554656548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c0cecf0 msgr2=0x7f651c15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6521e00640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c0cecf0 0x7f651c15dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6521e00640 1 -- 172.21.15.66:0/3554656548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6508009420 con 0x7f651c15ba00 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6520a00640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c0cecf0 0x7f651c15dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6521e00640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f651c15ba00 0x7f651c155cc0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f651400cd50 tx=0x7f651400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6519600640 1 -- 172.21.15.66:0/3554656548 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400f680 con 0x7f651c15ba00 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6519600640 1 -- 172.21.15.66:0/3554656548 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651400f800 con 0x7f651c15ba00 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6519600640 1 -- 172.21.15.66:0/3554656548 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400fbb0 con 0x7f651c15ba00 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.063+0000 7f6523e00640 1 -- 172.21.15.66:0/3554656548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c0ca180 con 0x7f651c15ba00 2024-03-30T00:51:48.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.067+0000 7f6523e00640 1 -- 172.21.15.66:0/3554656548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c0ca3d0 con 0x7f651c15ba00 2024-03-30T00:51:48.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.067+0000 7f6523e00640 1 -- 172.21.15.66:0/3554656548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c15d590 con 0x7f651c15ba00 2024-03-30T00:51:48.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.067+0000 7f6519600640 1 -- 172.21.15.66:0/3554656548 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f651400b040 con 0x7f651c15ba00 2024-03-30T00:51:48.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.067+0000 7f6519600640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f64fc066690 0x7f64fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:48.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.067+0000 7f6519600640 1 -- 172.21.15.66:0/3554656548 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f6514084db0 con 0x7f651c15ba00 2024-03-30T00:51:48.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.067+0000 7f6521400640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f64fc066690 0x7f64fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:48.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.067+0000 7f6521400640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f64fc066690 0x7f64fc068b50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6510000f70 tx=0x7f6510007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:48.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.071+0000 7f6519600640 1 -- 172.21.15.66:0/3554656548 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f651c15d590 con 0x7f651c15ba00 2024-03-30T00:51:48.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.235+0000 7f6523e00640 1 -- 172.21.15.66:0/3554656548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f651c15d590 con 0x7f651c15ba00 2024-03-30T00:51:48.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.851+0000 7f6519600640 1 -- 172.21.15.66:0/3554656548 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7f651c15d590 con 0x7f651c15ba00 2024-03-30T00:51:48.860 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-03-30T00:51:48.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 -- 172.21.15.66:0/3554656548 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f64fc066690 msgr2=0x7f64fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:48.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f64fc066690 0x7f64fc068b50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6510000f70 tx=0x7f6510007c10 comp rx=0 tx=0).stop 2024-03-30T00:51:48.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 -- 172.21.15.66:0/3554656548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f651c15ba00 msgr2=0x7f651c155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:48.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f651c15ba00 0x7f651c155cc0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f651400cd50 tx=0x7f651400a650 comp rx=0 tx=0).stop 2024-03-30T00:51:48.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 -- 172.21.15.66:0/3554656548 shutdown_connections 2024-03-30T00:51:48.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f64fc066690 0x7f64fc068b50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:48.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f651c15ba00 0x7f651c155cc0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:48.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c0cecf0 0x7f651c15dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:48.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 --2- 172.21.15.66:0/3554656548 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f651c0acaf0 0x7f651c159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:48.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 -- 172.21.15.66:0/3554656548 >> 172.21.15.66:0/3554656548 conn(0x7f651c06a190 msgr2=0x7f651c0ad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:48.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 -- 172.21.15.66:0/3554656548 shutdown_connections 2024-03-30T00:51:48.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:48.855+0000 7f6523e00640 1 -- 172.21.15.66:0/3554656548 wait complete. 2024-03-30T00:51:48.887 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T00:51:49.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad6d400640 1 Processor -- start 2024-03-30T00:51:49.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad6d400640 1 -- start start 2024-03-30T00:51:49.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad6d400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fad680cb050 0x7fad680b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:49.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad6d400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6806df80 con 0x7fad680ad590 2024-03-30T00:51:49.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad6d400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad680b2a20 con 0x7fad68158e80 2024-03-30T00:51:49.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad6d400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad680ada00 con 0x7fad680cb050 2024-03-30T00:51:49.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fad680ad590 0x7fad680ca940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35284/0 (socket says 172.21.15.66:35284) 2024-03-30T00:51:49.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65e00640 1 -- 172.21.15.66:0/518042913 learned_addr learned my addr 172.21.15.66:0/518042913 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:49.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad67200640 1 --2- 172.21.15.66:0/518042913 >> v2:172.21.15.156:3300/0 conn(0x7fad680cb050 0x7fad680b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:49.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373703004 0 0) 0x7fad6806df80 con 0x7fad680ad590 2024-03-30T00:51:49.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad4c003530 con 0x7fad680ad590 2024-03-30T00:51:49.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726647995 0 0) 0x7fad4c003530 con 0x7fad680ad590 2024-03-30T00:51:49.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 >> v2:172.21.15.156:3300/0 conn(0x7fad680cb050 msgr2=0x7fad680b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:49.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 --2- 172.21.15.66:0/518042913 >> v2:172.21.15.156:3300/0 conn(0x7fad680cb050 0x7fad680b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:49.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 >> v1:172.21.15.156:6789/0 conn(0x7fad68158e80 legacy=0x7fad6806bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:49.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad680c93d0 con 0x7fad680ad590 2024-03-30T00:51:49.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7fad54001680 con 0x7fad680ad590 2024-03-30T00:51:49.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 >> v1:172.21.15.66:6789/0 conn(0x7fad680ad590 legacy=0x7fad680ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:49.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 --2- 172.21.15.66:0/518042913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad4c003980 0x7fad4c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:49.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 --2- 172.21.15.66:0/518042913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad4c0047d0 0x7fad4c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:49.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 --2- 172.21.15.66:0/518042913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad4c005190 0x7fad4c00caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:49.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad680ada00 con 0x7fad4c005190 2024-03-30T00:51:49.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c00d150 con 0x7fad4c0047d0 2024-03-30T00:51:49.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c00d310 con 0x7fad4c003980 2024-03-30T00:51:49.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad67200640 1 --2- 172.21.15.66:0/518042913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad4c005190 0x7fad4c00caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:49.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad66800640 1 --2- 172.21.15.66:0/518042913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad4c003980 0x7fad4c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:49.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad65e00640 1 --2- 172.21.15.66:0/518042913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad4c0047d0 0x7fad4c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:49.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad67200640 1 -- 172.21.15.66:0/518042913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad4c003980 msgr2=0x7fad4c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:49.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad67200640 1 --2- 172.21.15.66:0/518042913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad4c003980 0x7fad4c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:49.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad67200640 1 -- 172.21.15.66:0/518042913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad4c0047d0 msgr2=0x7fad4c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:49.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad67200640 1 --2- 172.21.15.66:0/518042913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad4c0047d0 0x7fad4c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:49.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.027+0000 7fad67200640 1 -- 172.21.15.66:0/518042913 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c003530 con 0x7fad4c005190 2024-03-30T00:51:49.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad67200640 1 -- 172.21.15.66:0/518042913 reap_dead start 2024-03-30T00:51:49.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad67200640 1 --2- 172.21.15.66:0/518042913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad4c005190 0x7fad4c00caa0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fad60009f90 tx=0x7fad6001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59df5c7fc6e79fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:49.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600a5c90 con 0x7fad4c005190 2024-03-30T00:51:49.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6000f040 con 0x7fad4c005190 2024-03-30T00:51:49.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad65400640 1 -- 172.21.15.66:0/518042913 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600ad440 con 0x7fad4c005190 2024-03-30T00:51:49.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad6d400640 1 -- 172.21.15.66:0/518042913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad4c005190 msgr2=0x7fad4c00caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:49.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad6d400640 1 --2- 172.21.15.66:0/518042913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad4c005190 0x7fad4c00caa0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fad60009f90 tx=0x7fad6001cc80 comp rx=0 tx=0).stop 2024-03-30T00:51:49.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad6d400640 1 -- 172.21.15.66:0/518042913 shutdown_connections 2024-03-30T00:51:49.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad6d400640 1 --2- 172.21.15.66:0/518042913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad4c005190 0x7fad4c00caa0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:49.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad6d400640 1 -- 172.21.15.66:0/518042913 >> 172.21.15.66:0/518042913 conn(0x7fad6806a190 msgr2=0x7fad6806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:49.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad6d400640 1 -- 172.21.15.66:0/518042913 shutdown_connections 2024-03-30T00:51:49.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.031+0000 7fad6d400640 1 -- 172.21.15.66:0/518042913 wait complete. 2024-03-30T00:51:49.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad6d400640 1 Processor -- start 2024-03-30T00:51:49.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad6d400640 1 -- start start 2024-03-30T00:51:49.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad6d400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad4c005190 0x7fad680c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:49.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad6d400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad68150140 0x7fad68157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:49.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad6d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad680c69d0 0x7fad680c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:49.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad6d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad680c5900 con 0x7fad680c69d0 2024-03-30T00:51:49.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad6d400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad680c20c0 con 0x7fad4c005190 2024-03-30T00:51:49.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad6d400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad68150700 con 0x7fad68150140 2024-03-30T00:51:49.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad67200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad680c69d0 0x7fad680c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:49.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad67200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad680c69d0 0x7fad680c6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48642/0 (socket says 172.21.15.66:48642) 2024-03-30T00:51:49.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad67200640 1 -- 172.21.15.66:0/1357564752 learned_addr learned my addr 172.21.15.66:0/1357564752 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:49.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad66800640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad4c005190 0x7fad680c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:49.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad65e00640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad68150140 0x7fad68157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:49.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad67200640 1 -- 172.21.15.66:0/1357564752 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad68150140 msgr2=0x7fad68157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:49.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad67200640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad68150140 0x7fad68157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:49.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad67200640 1 -- 172.21.15.66:0/1357564752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad4c005190 msgr2=0x7fad680c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:49.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad67200640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad4c005190 0x7fad680c1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:49.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad67200640 1 -- 172.21.15.66:0/1357564752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad681509e0 con 0x7fad680c69d0 2024-03-30T00:51:49.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad65e00640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad68150140 0x7fad68157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:49.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad67200640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad680c69d0 0x7fad680c6e50 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fad6001d520 tx=0x7fad60009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:49.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad5ea00640 1 -- 172.21.15.66:0/1357564752 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600bd070 con 0x7fad680c69d0 2024-03-30T00:51:49.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad5ea00640 1 -- 172.21.15.66:0/1357564752 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad600adac0 con 0x7fad680c69d0 2024-03-30T00:51:49.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad5ea00640 1 -- 172.21.15.66:0/1357564752 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600ade70 con 0x7fad680c69d0 2024-03-30T00:51:49.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad6d400640 1 -- 172.21.15.66:0/1357564752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad68150cc0 con 0x7fad680c69d0 2024-03-30T00:51:49.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.035+0000 7fad6d400640 1 -- 172.21.15.66:0/1357564752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad681584e0 con 0x7fad680c69d0 2024-03-30T00:51:49.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.039+0000 7fad6d400640 1 -- 172.21.15.66:0/1357564752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad680c5a40 con 0x7fad680c69d0 2024-03-30T00:51:49.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.039+0000 7fad5ea00640 1 -- 172.21.15.66:0/1357564752 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fad600b4040 con 0x7fad680c69d0 2024-03-30T00:51:49.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.039+0000 7fad5ea00640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fad480666c0 0x7fad48068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:49.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.039+0000 7fad5ea00640 1 -- 172.21.15.66:0/1357564752 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5849+0+0 (secure 0 0 0) 0x7fad6011e9d0 con 0x7fad680c69d0 2024-03-30T00:51:49.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.039+0000 7fad66800640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fad480666c0 0x7fad48068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:49.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.039+0000 7fad66800640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fad480666c0 0x7fad48068b80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fad58007fd0 tx=0x7fad58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:49.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.043+0000 7fad5ea00640 1 -- 172.21.15.66:0/1357564752 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fad680c5a40 con 0x7fad680c69d0 2024-03-30T00:51:49.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.203+0000 7fad6d400640 1 -- 172.21.15.66:0/1357564752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad680c5a40 con 0x7fad680c69d0 2024-03-30T00:51:49.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.859+0000 7fad5ea00640 1 -- 172.21.15.66:0/1357564752 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) v1 ==== 169+0+0 (secure 0 0 0) 0x7fad680c5a40 con 0x7fad680c69d0 2024-03-30T00:51:49.870 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-03-30T00:51:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 -- 172.21.15.66:0/1357564752 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fad480666c0 msgr2=0x7fad48068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fad480666c0 0x7fad48068b80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fad58007fd0 tx=0x7fad58009040 comp rx=0 tx=0).stop 2024-03-30T00:51:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 -- 172.21.15.66:0/1357564752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad680c69d0 msgr2=0x7fad680c6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad680c69d0 0x7fad680c6e50 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fad6001d520 tx=0x7fad60009750 comp rx=0 tx=0).stop 2024-03-30T00:51:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 -- 172.21.15.66:0/1357564752 shutdown_connections 2024-03-30T00:51:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fad480666c0 0x7fad48068b80 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad680c69d0 0x7fad680c6e50 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad68150140 0x7fad68157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 --2- 172.21.15.66:0/1357564752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad4c005190 0x7fad680c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:49.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 -- 172.21.15.66:0/1357564752 >> 172.21.15.66:0/1357564752 conn(0x7fad6806a190 msgr2=0x7fad680c4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:49.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 -- 172.21.15.66:0/1357564752 shutdown_connections 2024-03-30T00:51:49.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:49.863+0000 7fad6d400640 1 -- 172.21.15.66:0/1357564752 wait complete. 2024-03-30T00:51:49.893 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-30T00:51:50.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.039+0000 7f6323000640 1 Processor -- start 2024-03-30T00:51:50.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.039+0000 7f6323000640 1 -- start start 2024-03-30T00:51:50.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.039+0000 7f6323000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f631c0b19a0 0x7f631c0c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:50.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.039+0000 7f6323000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f631c0c1800 0x7f631c0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:50.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f6323000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631c0c97f0 con 0x7f631c160c10 2024-03-30T00:51:50.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f6323000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631c0c1c40 con 0x7f631c0b19a0 2024-03-30T00:51:50.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f6323000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631c0c9a30 con 0x7f631c0c1800 2024-03-30T00:51:50.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f6321000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f631c0c1800 0x7f631c0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:50.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f6321000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f631c0c1800 0x7f631c0cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:39914/0 (socket says 172.21.15.66:39914) 2024-03-30T00:51:50.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f6321000640 1 -- 172.21.15.66:0/258615242 learned_addr learned my addr 172.21.15.66:0/258615242 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:50.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f631b400640 1 --2- 172.21.15.66:0/258615242 >> v2:172.21.15.66:3300/0 conn(0x7f631c0b19a0 0x7f631c0c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:50.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f6321000640 1 -- 172.21.15.66:0/258615242 >> v2:172.21.15.66:3300/0 conn(0x7f631c0b19a0 msgr2=0x7f631c0c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:50.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f6321000640 1 --2- 172.21.15.66:0/258615242 >> v2:172.21.15.66:3300/0 conn(0x7f631c0b19a0 0x7f631c0c12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f6321000640 1 -- 172.21.15.66:0/258615242 >> v1:172.21.15.156:6789/0 conn(0x7f631c160c10 legacy=0x7f631c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:50.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f6321000640 1 -- 172.21.15.66:0/258615242 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c0c9d10 con 0x7f631c0c1800 2024-03-30T00:51:50.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.043+0000 7f631b400640 1 --2- 172.21.15.66:0/258615242 >> v2:172.21.15.66:3300/0 conn(0x7f631c0b19a0 0x7f631c0c12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:50.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.047+0000 7f6321000640 1 --2- 172.21.15.66:0/258615242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f631c0c1800 0x7f631c0cc280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f631c0cf340 tx=0x7f631401c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c926008e9751f08 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:50.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.047+0000 7f631aa00640 1 -- 172.21.15.66:0/258615242 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63140a5880 con 0x7f631c0c1800 2024-03-30T00:51:50.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.047+0000 7f631aa00640 1 -- 172.21.15.66:0/258615242 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63140a59e0 con 0x7f631c0c1800 2024-03-30T00:51:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.051+0000 7f631aa00640 1 -- 172.21.15.66:0/258615242 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63140a5d70 con 0x7f631c0c1800 2024-03-30T00:51:50.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.051+0000 7f6323000640 1 -- 172.21.15.66:0/258615242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f631c0c1800 msgr2=0x7f631c0cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.051+0000 7f6323000640 1 --2- 172.21.15.66:0/258615242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f631c0c1800 0x7f631c0cc280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f631c0cf340 tx=0x7f631401c9c0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.051+0000 7f6323000640 1 -- 172.21.15.66:0/258615242 shutdown_connections 2024-03-30T00:51:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.051+0000 7f6323000640 1 --2- 172.21.15.66:0/258615242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f631c0c1800 0x7f631c0cc280 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.051+0000 7f6323000640 1 --2- 172.21.15.66:0/258615242 >> v2:172.21.15.66:3300/0 conn(0x7f631c0b19a0 0x7f631c0c12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.051+0000 7f6323000640 1 -- 172.21.15.66:0/258615242 >> 172.21.15.66:0/258615242 conn(0x7f631c069f60 msgr2=0x7f631c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.051+0000 7f6323000640 1 -- 172.21.15.66:0/258615242 shutdown_connections 2024-03-30T00:51:50.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.051+0000 7f6323000640 1 -- 172.21.15.66:0/258615242 wait complete. 2024-03-30T00:51:50.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.051+0000 7f6323000640 1 Processor -- start 2024-03-30T00:51:50.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.051+0000 7f6323000640 1 -- start start 2024-03-30T00:51:50.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6323000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f631c0b19a0 0x7f631c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:50.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6323000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f631c0c1800 0x7f631c0c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:50.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6323000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f631c160c10 0x7f631c15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:50.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6323000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c0ab010 con 0x7f631c160c10 2024-03-30T00:51:50.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6323000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c0aea20 con 0x7f631c0c1800 2024-03-30T00:51:50.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6323000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c0ab260 con 0x7f631c0b19a0 2024-03-30T00:51:50.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6321000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f631c160c10 0x7f631c15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:50.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6321000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f631c160c10 0x7f631c15f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48660/0 (socket says 172.21.15.66:48660) 2024-03-30T00:51:50.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6321000640 1 -- 172.21.15.66:0/3125680901 learned_addr learned my addr 172.21.15.66:0/3125680901 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:50.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f631b400640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f631c0c1800 0x7f631c0c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:50.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f631be00640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f631c0b19a0 0x7f631c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:50.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6321000640 1 -- 172.21.15.66:0/3125680901 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f631c0b19a0 msgr2=0x7f631c15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6321000640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f631c0b19a0 0x7f631c15af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6321000640 1 -- 172.21.15.66:0/3125680901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f631c0c1800 msgr2=0x7f631c0c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6321000640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f631c0c1800 0x7f631c0c1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6321000640 1 -- 172.21.15.66:0/3125680901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6314009b30 con 0x7f631c160c10 2024-03-30T00:51:50.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6321000640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f631c160c10 0x7f631c15f7e0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f63140af040 tx=0x7f6314019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:50.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6313e00640 1 -- 172.21.15.66:0/3125680901 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63140b8880 con 0x7f631c160c10 2024-03-30T00:51:50.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6313e00640 1 -- 172.21.15.66:0/3125680901 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63140b89e0 con 0x7f631c160c10 2024-03-30T00:51:50.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6323000640 1 -- 172.21.15.66:0/3125680901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c0ab540 con 0x7f631c160c10 2024-03-30T00:51:50.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6313e00640 1 -- 172.21.15.66:0/3125680901 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63140b8d90 con 0x7f631c160c10 2024-03-30T00:51:50.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6323000640 1 -- 172.21.15.66:0/3125680901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c0abb00 con 0x7f631c160c10 2024-03-30T00:51:50.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6323000640 1 -- 172.21.15.66:0/3125680901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631c066c30 con 0x7f631c160c10 2024-03-30T00:51:50.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.055+0000 7f6313e00640 1 -- 172.21.15.66:0/3125680901 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f63140c3460 con 0x7f631c160c10 2024-03-30T00:51:50.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.059+0000 7f6313e00640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62fc066690 0x7f62fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:50.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.059+0000 7f6313e00640 1 -- 172.21.15.66:0/3125680901 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f631411f730 con 0x7f631c160c10 2024-03-30T00:51:50.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.059+0000 7f631be00640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62fc066690 0x7f62fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:50.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.059+0000 7f631be00640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62fc066690 0x7f62fc068b50 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f630c007fd0 tx=0x7f630c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:50.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.063+0000 7f6313e00640 1 -- 172.21.15.66:0/3125680901 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f63140ecda0 con 0x7f631c160c10 2024-03-30T00:51:50.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.223+0000 7f6323000640 1 -- 172.21.15.66:0/3125680901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f631c0b2620 con 0x7f631c160c10 2024-03-30T00:51:50.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.227+0000 7f6313e00640 1 -- 172.21.15.66:0/3125680901 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f63140f03c0 con 0x7f631c160c10 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 -- 172.21.15.66:0/3125680901 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62fc066690 msgr2=0x7f62fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62fc066690 0x7f62fc068b50 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f630c007fd0 tx=0x7f630c009040 comp rx=0 tx=0).stop 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 -- 172.21.15.66:0/3125680901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f631c160c10 msgr2=0x7f631c15f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f631c160c10 0x7f631c15f7e0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f63140af040 tx=0x7f6314019780 comp rx=0 tx=0).stop 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 -- 172.21.15.66:0/3125680901 shutdown_connections 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f62fc066690 0x7f62fc068b50 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f631c160c10 0x7f631c15f7e0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f631c0c1800 0x7f631c0c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 --2- 172.21.15.66:0/3125680901 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f631c0b19a0 0x7f631c15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 -- 172.21.15.66:0/3125680901 >> 172.21.15.66:0/3125680901 conn(0x7f631c069f60 msgr2=0x7f631c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 -- 172.21.15.66:0/3125680901 shutdown_connections 2024-03-30T00:51:50.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.231+0000 7f6323000640 1 -- 172.21.15.66:0/3125680901 wait complete. 2024-03-30T00:51:50.260 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-03-30T00:51:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.391+0000 7f1733a00640 1 Processor -- start 2024-03-30T00:51:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.391+0000 7f1733a00640 1 -- start start 2024-03-30T00:51:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.391+0000 7f1733a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f172c0cde70 0x7f172c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.391+0000 7f1733a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172c0cd3e0 con 0x7f172c0cda90 2024-03-30T00:51:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.391+0000 7f1733a00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172c15a990 con 0x7f172c0cec30 2024-03-30T00:51:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.391+0000 7f1733a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0b1b80 con 0x7f172c0cde70 2024-03-30T00:51:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.391+0000 7f1731000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f172c0cda90 0x7f172c0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35288/0 (socket says 172.21.15.66:35288) 2024-03-30T00:51:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.391+0000 7f1731000640 1 -- 172.21.15.66:0/2265149632 learned_addr learned my addr 172.21.15.66:0/2265149632 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.391+0000 7f1731a00640 1 --2- 172.21.15.66:0/2265149632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f172c0cde70 0x7f172c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.395+0000 7f172b400640 1 -- 172.21.15.66:0/2265149632 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182201273 0 0) 0x7f172c0cd3e0 con 0x7f172c0cda90 2024-03-30T00:51:50.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.395+0000 7f172b400640 1 -- 172.21.15.66:0/2265149632 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1714003530 con 0x7f172c0cda90 2024-03-30T00:51:50.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.395+0000 7f172b400640 1 -- 172.21.15.66:0/2265149632 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365616442 0 0) 0x7f172c15a990 con 0x7f172c0cec30 2024-03-30T00:51:50.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.395+0000 7f172b400640 1 -- 172.21.15.66:0/2265149632 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f172c0cd3e0 con 0x7f172c0cec30 2024-03-30T00:51:50.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.395+0000 7f1731a00640 1 -- 172.21.15.66:0/2265149632 >> v1:172.21.15.156:6789/0 conn(0x7f172c0cec30 legacy=0x7f172c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.395+0000 7f1731a00640 1 -- 172.21.15.66:0/2265149632 >> v1:172.21.15.66:6789/0 conn(0x7f172c0cda90 legacy=0x7f172c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.395+0000 7f1731a00640 1 -- 172.21.15.66:0/2265149632 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c0b11e0 con 0x7f172c0cde70 2024-03-30T00:51:50.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.395+0000 7f1731a00640 1 --2- 172.21.15.66:0/2265149632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f172c0cde70 0x7f172c15a370 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1724009dc0 tx=0x7f172401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fe2261c90e04e27 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:50.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.395+0000 7f172b400640 1 -- 172.21.15.66:0/2265149632 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17240a59c0 con 0x7f172c0cde70 2024-03-30T00:51:50.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.395+0000 7f172b400640 1 -- 172.21.15.66:0/2265149632 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17240a5b20 con 0x7f172c0cde70 2024-03-30T00:51:50.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.395+0000 7f172b400640 1 -- 172.21.15.66:0/2265149632 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17240ad440 con 0x7f172c0cde70 2024-03-30T00:51:50.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 -- 172.21.15.66:0/2265149632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f172c0cde70 msgr2=0x7f172c15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 --2- 172.21.15.66:0/2265149632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f172c0cde70 0x7f172c15a370 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1724009dc0 tx=0x7f172401c900 comp rx=0 tx=0).stop 2024-03-30T00:51:50.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 -- 172.21.15.66:0/2265149632 shutdown_connections 2024-03-30T00:51:50.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 --2- 172.21.15.66:0/2265149632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f172c0cde70 0x7f172c15a370 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 -- 172.21.15.66:0/2265149632 >> 172.21.15.66:0/2265149632 conn(0x7f172c069f60 msgr2=0x7f172c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:50.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 -- 172.21.15.66:0/2265149632 shutdown_connections 2024-03-30T00:51:50.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 -- 172.21.15.66:0/2265149632 wait complete. 2024-03-30T00:51:50.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 Processor -- start 2024-03-30T00:51:50.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 -- start start 2024-03-30T00:51:50.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cda90 0x7f172c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:50.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f172c0cde70 0x7f172c1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:50.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f172c0cec30 0x7f172c1637a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:50.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c070540 con 0x7f172c0cda90 2024-03-30T00:51:50.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1f55c0 con 0x7f172c0cec30 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1733a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1f58a0 con 0x7f172c0cde70 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1731000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cda90 0x7f172c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1731000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cda90 0x7f172c155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48674/0 (socket says 172.21.15.66:48674) 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1731000640 1 -- 172.21.15.66:0/468331163 learned_addr learned my addr 172.21.15.66:0/468331163 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f172be00640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f172c0cde70 0x7f172c1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1731a00640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f172c0cec30 0x7f172c1637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1731000640 1 -- 172.21.15.66:0/468331163 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f172c0cde70 msgr2=0x7f172c1f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1731000640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f172c0cde70 0x7f172c1f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1731000640 1 -- 172.21.15.66:0/468331163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f172c0cec30 msgr2=0x7f172c1637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1731000640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f172c0cec30 0x7f172c1637a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.399+0000 7f1731000640 1 -- 172.21.15.66:0/468331163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1724009a70 con 0x7f172c0cda90 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.403+0000 7f1731000640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cda90 0x7f172c155ad0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f172000c410 tx=0x7f172000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.403+0000 7f1728c00640 1 -- 172.21.15.66:0/468331163 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720005cf0 con 0x7f172c0cda90 2024-03-30T00:51:50.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.403+0000 7f1728c00640 1 -- 172.21.15.66:0/468331163 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1720005e50 con 0x7f172c0cda90 2024-03-30T00:51:50.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.403+0000 7f1733a00640 1 -- 172.21.15.66:0/468331163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c1f5b80 con 0x7f172c0cda90 2024-03-30T00:51:50.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.403+0000 7f1728c00640 1 -- 172.21.15.66:0/468331163 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720007310 con 0x7f172c0cda90 2024-03-30T00:51:50.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.403+0000 7f1733a00640 1 -- 172.21.15.66:0/468331163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c1f8230 con 0x7f172c0cda90 2024-03-30T00:51:50.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.403+0000 7f1733a00640 1 -- 172.21.15.66:0/468331163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172c066c30 con 0x7f172c0cda90 2024-03-30T00:51:50.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.403+0000 7f1728c00640 1 -- 172.21.15.66:0/468331163 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1720007470 con 0x7f172c0cda90 2024-03-30T00:51:50.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.403+0000 7f1728c00640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f170c0666c0 0x7f170c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:50.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.403+0000 7f1728c00640 1 -- 172.21.15.66:0/468331163 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f1720083600 con 0x7f172c0cda90 2024-03-30T00:51:50.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.403+0000 7f172be00640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f170c0666c0 0x7f170c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:50.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.407+0000 7f172be00640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f170c0666c0 0x7f170c068b80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f172c15e080 tx=0x7f1718009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:50.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.407+0000 7f1728c00640 1 -- 172.21.15.66:0/468331163 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1720050b20 con 0x7f172c0cda90 2024-03-30T00:51:50.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.627+0000 7f1733a00640 1 -- 172.21.15.66:0/468331163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f172c0b3170 con 0x7f172c0cda90 2024-03-30T00:51:50.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1728c00640 1 -- 172.21.15.66:0/468331163 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v574) v1 ==== 144+0+0 (secure 0 0 0) 0x7f1720054140 con 0x7f172c0cda90 2024-03-30T00:51:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 -- 172.21.15.66:0/468331163 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f170c0666c0 msgr2=0x7f170c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f170c0666c0 0x7f170c068b80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f172c15e080 tx=0x7f1718009040 comp rx=0 tx=0).stop 2024-03-30T00:51:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 -- 172.21.15.66:0/468331163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cda90 msgr2=0x7f172c155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cda90 0x7f172c155ad0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f172000c410 tx=0x7f172000c8d0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 -- 172.21.15.66:0/468331163 shutdown_connections 2024-03-30T00:51:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f172c0cec30 0x7f172c1637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f172c0cde70 0x7f172c1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f170c0666c0 0x7f170c068b80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 --2- 172.21.15.66:0/468331163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cda90 0x7f172c155ad0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 -- 172.21.15.66:0/468331163 >> 172.21.15.66:0/468331163 conn(0x7f172c069f60 msgr2=0x7f172c0ad510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 -- 172.21.15.66:0/468331163 shutdown_connections 2024-03-30T00:51:50.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:50.859+0000 7f1733a00640 1 -- 172.21.15.66:0/468331163 wait complete. 2024-03-30T00:51:50.889 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-30T00:51:50.889 INFO:tasks.cephfs_test_runner: 2024-03-30T00:51:50.889 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-30T00:51:50.889 INFO:tasks.cephfs_test_runner:Ran 7 tests in 666.275s 2024-03-30T00:51:50.889 INFO:tasks.cephfs_test_runner: 2024-03-30T00:51:50.889 INFO:tasks.cephfs_test_runner:OK 2024-03-30T00:51:50.890 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-30T00:51:50.902 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-30T00:51:50.923 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-30T00:51:50.923 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:51:50.923 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:50.932 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:51:50.932 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T00:51:50.932 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:50.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354548k,nr_inodes=4088637,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2588 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T00:51:50.982 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:51:50.982 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:50.992 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:51:50.992 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-30T00:51:50.992 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:51:51.045 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354620k,nr_inodes=4088655,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3892 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T00:51:51.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:51.045 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2024-03-30T00:51:51.097 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T00:51:51.097 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-30T00:51:51.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:51.098 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T00:51:51.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:51.161 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T00:51:51.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:51.240 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2024-03-30T00:51:51.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:51.455 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T00:51:51.465 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:51:51.465 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T00:51:51.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:51.466 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:51:51.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:51.531 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T00:51:51.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:51:51.604 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-03-30T00:51:51.772 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-30T00:51:51.782 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-30T00:51:51.782 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:51:51.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.915+0000 7f6837a00640 1 Processor -- start 2024-03-30T00:51:51.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6837a00640 1 -- start start 2024-03-30T00:51:51.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6837a00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f68300af790 0x7f6830099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:51.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6837a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f683006bb90 0x7f6830098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:51.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6837a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f683006bff0 0x7f6830098f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:51.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6837a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68300b0940 con 0x7f683006bff0 2024-03-30T00:51:51.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6837a00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683006b140 con 0x7f68300af790 2024-03-30T00:51:51.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6837a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683006b700 con 0x7f683006bb90 2024-03-30T00:51:51.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6835a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f683006bff0 0x7f6830098f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:51.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6835a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f683006bff0 0x7f6830098f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48686/0 (socket says 172.21.15.66:48686) 2024-03-30T00:51:51.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6835a00640 1 -- 172.21.15.66:0/200724614 learned_addr learned my addr 172.21.15.66:0/200724614 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:51.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6835000640 1 --2- 172.21.15.66:0/200724614 >> v2:172.21.15.156:3300/0 conn(0x7f68300af790 0x7f6830099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:51.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f682fe00640 1 --2- 172.21.15.66:0/200724614 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f683006bb90 0x7f6830098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:51.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6835a00640 1 -- 172.21.15.66:0/200724614 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f683006bb90 msgr2=0x7f6830098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:51.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6835a00640 1 --2- 172.21.15.66:0/200724614 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f683006bb90 0x7f6830098a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:51.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6835a00640 1 -- 172.21.15.66:0/200724614 >> v2:172.21.15.156:3300/0 conn(0x7f68300af790 msgr2=0x7f6830099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:51.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6835a00640 1 --2- 172.21.15.66:0/200724614 >> v2:172.21.15.156:3300/0 conn(0x7f68300af790 0x7f6830099760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:51.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.919+0000 7f6835a00640 1 -- 172.21.15.66:0/200724614 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68300b0b90 con 0x7f683006bff0 2024-03-30T00:51:51.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f6835a00640 1 --2- 172.21.15.66:0/200724614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f683006bff0 0x7f6830098f60 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f6830072da0 tx=0x7f682801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d532f1ceb28cb4c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:51.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f682f400640 1 -- 172.21.15.66:0/200724614 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280a5840 con 0x7f683006bff0 2024-03-30T00:51:51.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f682f400640 1 -- 172.21.15.66:0/200724614 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68280a59a0 con 0x7f683006bff0 2024-03-30T00:51:51.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f682f400640 1 -- 172.21.15.66:0/200724614 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280a5d30 con 0x7f683006bff0 2024-03-30T00:51:51.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f6837a00640 1 -- 172.21.15.66:0/200724614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f683006bff0 msgr2=0x7f6830098f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:51.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f6837a00640 1 --2- 172.21.15.66:0/200724614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f683006bff0 0x7f6830098f60 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f6830072da0 tx=0x7f682801c9c0 comp rx=0 tx=0).stop 2024-03-30T00:51:51.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f6837a00640 1 -- 172.21.15.66:0/200724614 shutdown_connections 2024-03-30T00:51:51.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f6837a00640 1 --2- 172.21.15.66:0/200724614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f683006bff0 0x7f6830098f60 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:51.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f6837a00640 1 --2- 172.21.15.66:0/200724614 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f683006bb90 0x7f6830098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:51.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f6837a00640 1 --2- 172.21.15.66:0/200724614 >> v2:172.21.15.156:3300/0 conn(0x7f68300af790 0x7f6830099760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:51.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f6837a00640 1 -- 172.21.15.66:0/200724614 >> 172.21.15.66:0/200724614 conn(0x7f68300b20c0 msgr2=0x7f68300b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:51.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f6837a00640 1 -- 172.21.15.66:0/200724614 shutdown_connections 2024-03-30T00:51:51.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.923+0000 7f6837a00640 1 -- 172.21.15.66:0/200724614 wait complete. 2024-03-30T00:51:51.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6837a00640 1 Processor -- start 2024-03-30T00:51:51.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6837a00640 1 -- start start 2024-03-30T00:51:51.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6837a00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f683006bb90 0x7f683014fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:51.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6837a00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f683006bff0 0x7f683014d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:51.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6837a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68300af790 0x7f68300cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:51.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6837a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68300b5410 con 0x7f68300af790 2024-03-30T00:51:51.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6837a00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68300b4800 con 0x7f683006bff0 2024-03-30T00:51:51.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6837a00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68300b4ae0 con 0x7f683006bb90 2024-03-30T00:51:51.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6835a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68300af790 0x7f68300cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:51.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6835a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68300af790 0x7f68300cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48696/0 (socket says 172.21.15.66:48696) 2024-03-30T00:51:51.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6835a00640 1 -- 172.21.15.66:0/1789918562 learned_addr learned my addr 172.21.15.66:0/1789918562 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:51.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f682fe00640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f683006bff0 0x7f683014d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:51.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6835000640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f683006bb90 0x7f683014fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:51.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6835a00640 1 -- 172.21.15.66:0/1789918562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f683006bb90 msgr2=0x7f683014fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:51.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6835a00640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f683006bb90 0x7f683014fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:51.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6835a00640 1 -- 172.21.15.66:0/1789918562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f683006bff0 msgr2=0x7f683014d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:51.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6835a00640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f683006bff0 0x7f683014d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:51.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6835a00640 1 -- 172.21.15.66:0/1789918562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6828009b30 con 0x7f68300af790 2024-03-30T00:51:51.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6835a00640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68300af790 0x7f68300cf110 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f682801b070 tx=0x7f6828019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:51.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f682cc00640 1 -- 172.21.15.66:0/1789918562 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280b8670 con 0x7f68300af790 2024-03-30T00:51:51.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f682cc00640 1 -- 172.21.15.66:0/1789918562 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68280b87d0 con 0x7f68300af790 2024-03-30T00:51:51.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6837a00640 1 -- 172.21.15.66:0/1789918562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68300b4dc0 con 0x7f68300af790 2024-03-30T00:51:51.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f682cc00640 1 -- 172.21.15.66:0/1789918562 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280b8bc0 con 0x7f68300af790 2024-03-30T00:51:51.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.927+0000 7f6837a00640 1 -- 172.21.15.66:0/1789918562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6830158c20 con 0x7f68300af790 2024-03-30T00:51:51.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.931+0000 7f6837a00640 1 -- 172.21.15.66:0/1789918562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6830153990 con 0x7f68300af790 2024-03-30T00:51:51.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.931+0000 7f682cc00640 1 -- 172.21.15.66:0/1789918562 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f68280b8d20 con 0x7f68300af790 2024-03-30T00:51:51.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.931+0000 7f682cc00640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6810066690 0x7f6810068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:51.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.931+0000 7f682cc00640 1 -- 172.21.15.66:0/1789918562 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f682811f6e0 con 0x7f68300af790 2024-03-30T00:51:51.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.931+0000 7f6835000640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6810066690 0x7f6810068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:51.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.931+0000 7f6835000640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6810066690 0x7f6810068b50 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6830167ba0 tx=0x7f6824009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:51.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:51.935+0000 7f682cc00640 1 -- 172.21.15.66:0/1789918562 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6830153990 con 0x7f68300af790 2024-03-30T00:51:52.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.095+0000 7f6837a00640 1 -- 172.21.15.66:0/1789918562 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f68300afb70 con 0x7f6810066690 2024-03-30T00:51:52.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.095+0000 7f682cc00640 1 -- 172.21.15.66:0/1789918562 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50621 (secure 0 0 0) 0x7f6824011a60 con 0x7f6810066690 2024-03-30T00:51:52.106 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:52.106 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:51:52.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 -- 172.21.15.66:0/1789918562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6810066690 msgr2=0x7f6810068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6810066690 0x7f6810068b50 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6830167ba0 tx=0x7f6824009190 comp rx=0 tx=0).stop 2024-03-30T00:51:52.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 -- 172.21.15.66:0/1789918562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68300af790 msgr2=0x7f68300cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68300af790 0x7f68300cf110 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f682801b070 tx=0x7f6828019760 comp rx=0 tx=0).stop 2024-03-30T00:51:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 -- 172.21.15.66:0/1789918562 shutdown_connections 2024-03-30T00:51:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6810066690 0x7f6810068b50 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68300af790 0x7f68300cf110 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f683006bff0 0x7f683014d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 --2- 172.21.15.66:0/1789918562 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f683006bb90 0x7f683014fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 -- 172.21.15.66:0/1789918562 >> 172.21.15.66:0/1789918562 conn(0x7f68300b20c0 msgr2=0x7f68301564a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 -- 172.21.15.66:0/1789918562 shutdown_connections 2024-03-30T00:51:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.099+0000 7f6837a00640 1 -- 172.21.15.66:0/1789918562 wait complete. 2024-03-30T00:51:52.127 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-03-30T00:51:51.015428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278948,"kb_used_data":30032,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754695772,"statfs":{"total":773094113280,"available":772808470528,"internally_reserved":0,"allocated":30752768,"data_stored":23854800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392378,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-177,"num_write_kb":-8297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000995"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":277,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-03-30T00:51:42.965505+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:51:42.965505+0000","last_peered":"2024-03-30T00:51:42.965505+0000","last_clean":"2024-03-30T00:51:42.965505+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:51:42.965505+0000","last_undegraded":"2024-03-30T00:51:42.965505+0000","last_fullsized":"2024-03-30T00:51:42.965505+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574996,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":2207744,"data_stored":1330601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.65600000000000003},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84399999999999997,"5min":0.97599999999999998,"15min":1.0860000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66900000000000004,"15min":0.65300000000000002},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.98299999999999998,"15min":1.149},"last":0.65700000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.65800000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.03,"15min":1.03},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.46700000000000003,"5min":0.442,"15min":0.40400000000000003},"max":{"1min":0.81399999999999995,"5min":0.93000000000000005,"15min":1.3029999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.629,"5min":0.49399999999999999,"15min":0.45000000000000001},"max":{"1min":0.97499999999999998,"5min":1.085,"15min":1.6020000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.73699999999999999,"15min":0.75600000000000001},"min":{"1min":0.503,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93600000000000005,"5min":1.034,"15min":1.6619999999999999},"last":0.73599999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.755},"min":{"1min":0.47899999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":2.0150000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78000000000000003,"15min":0.79800000000000004},"min":{"1min":0.59999999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.96599999999999997,"5min":1.006,"15min":1.9490000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41099999999999998,"15min":0.37},"max":{"1min":0.92000000000000004,"5min":1.2,"15min":1.2},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.89700000000000002,"5min":0.95799999999999996,"15min":0.97899999999999998},"last":0.88900000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.45800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0649999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72599999999999998,"15min":0.74299999999999999},"min":{"1min":0.503,"5min":0.441,"15min":0.41699999999999998},"max":{"1min":0.98799999999999999,"5min":0.99199999999999999,"15min":1.026},"last":0.878}]},{"osd":6,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.54100000000000004,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.89200000000000002,"5min":0.94399999999999995,"15min":1.214},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.49099999999999999,"5min":0.45800000000000002,"15min":0.41199999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0309999999999999},"last":0.85499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5791744,"data_stored":4934336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69799999999999995,"15min":0.77900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.995,"15min":4.0490000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.48899999999999999,"5min":0.46400000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":1.091,"15min":1.47},"last":0.78500000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.78600000000000003},"min":{"1min":0.52000000000000002,"5min":0.438,"15min":0.36899999999999999},"max":{"1min":0.96199999999999997,"5min":1.204,"15min":4.3170000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72899999999999998,"15min":0.83099999999999996},"min":{"1min":0.52600000000000002,"5min":0.47199999999999998,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":1.1120000000000001,"15min":4.4160000000000004},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73599999999999999,"15min":0.83099999999999996},"min":{"1min":0.57899999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95099999999999996,"5min":1.0720000000000001,"15min":4.375},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70799999999999996,"15min":0.81999999999999995},"min":{"1min":0.47799999999999998,"5min":0.42999999999999999,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":1.0229999999999999,"15min":4.2960000000000003},"last":0.72999999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.749,"15min":0.84599999999999997},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":1.2390000000000001,"15min":4.3399999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71899999999999997,"15min":0.82699999999999996},"min":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.41999999999999998},"max":{"1min":0.91700000000000004,"5min":1.081,"15min":4.6020000000000003},"last":0.70799999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44600000000000001,"5min":0.39100000000000001,"15min":0.35999999999999999},"max":{"1min":0.871,"5min":1.0089999999999999,"15min":1.1870000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69599999999999995,"15min":0.72599999999999998},"min":{"1min":0.432,"5min":0.372,"15min":0.372},"max":{"1min":0.90500000000000003,"5min":0.98699999999999999,"15min":1.036},"last":0.83599999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61799999999999999,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91400000000000003,"5min":1.0449999999999999,"15min":1.0720000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68000000000000005,"15min":0.69099999999999995},"min":{"1min":0.46800000000000003,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.96799999999999997,"5min":1.0109999999999999,"15min":1.101},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66800000000000004,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.46200000000000002,"15min":0.45700000000000002},"max":{"1min":0.94999999999999996,"5min":1.103,"15min":1.103},"last":0.76200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5795840,"data_stored":4933914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":1.036,"5min":1.036,"15min":1.095},"last":1.01},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67600000000000005,"15min":0.72399999999999998},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.38800000000000001},"max":{"1min":1.0089999999999999,"5min":1.0449999999999999,"15min":1.7569999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67200000000000004,"15min":0.71299999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.43099999999999999},"max":{"1min":0.88600000000000001,"5min":0.97399999999999998,"15min":1.7330000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.44500000000000001,"5min":0.43099999999999999,"15min":0.42399999999999999},"max":{"1min":0.95099999999999996,"5min":0.98799999999999999,"15min":1.3260000000000001},"last":0.69499999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70799999999999996,"15min":0.71499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.024,"5min":1.131,"15min":1.131},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.47999999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.97899999999999998,"5min":1.0609999999999999,"15min":1.2789999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70099999999999996,"15min":0.74299999999999999},"min":{"1min":0.48399999999999999,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":1.024,"5min":1.0740000000000001,"15min":1.7},"last":0.755},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72599999999999998,"15min":0.74099999999999999},"min":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.999,"5min":1.1160000000000001,"15min":1.181},"last":0.79800000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.314,"15min":0.309},"max":{"1min":0.746,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.72699999999999998},"min":{"1min":0.53500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":1.615},"last":0.877}]},{"osd":6,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.97399999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.624,"15min":0.66400000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35199999999999998},"max":{"1min":0.84499999999999997,"5min":0.93700000000000006,"15min":1.569},"last":0.97299999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.70899999999999996},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.377},"max":{"1min":0.90500000000000003,"5min":0.92200000000000004,"15min":1.0149999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.78700000000000003},"min":{"1min":0.53600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92300000000000004,"5min":1.167,"15min":1.383},"last":0.89700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5795840,"data_stored":4930148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.42099999999999999,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.97099999999999997,"15min":1.4199999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.67900000000000005},"min":{"1min":0.502,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.877,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.48999999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.47699999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.93999999999999995,"5min":0.95399999999999996,"15min":1.2909999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.71499999999999997},"min":{"1min":0.49199999999999999,"5min":0.46600000000000003,"15min":0.40699999999999997},"max":{"1min":0.86199999999999999,"5min":1.113,"15min":1.113},"last":0.628}]},{"osd":2,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.51600000000000001,"5min":0.48999999999999999,"15min":0.48499999999999999},"max":{"1min":0.96999999999999997,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.748},"min":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":0.98499999999999999,"5min":1.163,"15min":1.228},"last":0.76700000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.71799999999999997},"min":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.46800000000000003},"max":{"1min":0.91800000000000004,"5min":1.0800000000000001,"15min":1.27},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.75600000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.44500000000000001},"max":{"1min":0.93200000000000005,"5min":0.99399999999999999,"15min":1.1739999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.629,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.329},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.65800000000000003,"15min":0.69299999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.33300000000000002},"max":{"1min":0.85899999999999999,"5min":0.99099999999999999,"15min":1.3819999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.2130000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.68700000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39000000000000001},"max":{"1min":0.89200000000000002,"5min":0.95799999999999996,"15min":1.3220000000000001},"last":0.73199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63,"15min":0.68200000000000005},"min":{"1min":0.438,"5min":0.42399999999999999,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":0.91500000000000004,"15min":1.6910000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72799999999999998,"15min":0.77000000000000002},"min":{"1min":0.51600000000000001,"5min":0.51400000000000001,"15min":0.46600000000000003},"max":{"1min":0.90200000000000002,"5min":0.97799999999999998,"15min":1.5009999999999999},"last":0.745}]}]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1597440,"data_stored":735866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.434,"15min":0.45300000000000001},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.68999999999999995,"5min":0.748,"15min":1.0529999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47799999999999998,"15min":0.50800000000000001},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.24199999999999999},"max":{"1min":0.72199999999999998,"5min":0.75,"15min":0.79100000000000004},"last":0.32500000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.5,"15min":0.504},"min":{"1min":0.42899999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74199999999999999,"5min":0.79100000000000004,"15min":0.80900000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.51900000000000002,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.32100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":1.2150000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.79900000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.434}]},{"osd":4,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75700000000000001,"15min":0.76200000000000001},"min":{"1min":0.59399999999999997,"5min":0.47799999999999998,"15min":0.46300000000000002},"max":{"1min":0.96199999999999997,"5min":1.1259999999999999,"15min":1.2470000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.59099999999999997,"5min":0.51600000000000001,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":1.079,"15min":1.0860000000000001},"last":0.64700000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76900000000000002,"15min":0.77700000000000002},"min":{"1min":0.58099999999999996,"5min":0.54600000000000004,"15min":0.51100000000000001},"max":{"1min":1.0169999999999999,"5min":1.0329999999999999,"15min":1.125},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.62,"5min":0.52200000000000002,"15min":0.48599999999999999},"max":{"1min":0.93000000000000005,"5min":1.133,"15min":1.2390000000000001},"last":0.63}]},{"osd":6,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76500000000000001,"15min":0.79300000000000004},"min":{"1min":0.58099999999999996,"5min":0.51400000000000001,"15min":0.499},"max":{"1min":1.0049999999999999,"5min":1.091,"15min":1.226},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.1120000000000001,"15min":1.167},"last":0.60299999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.67400000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0549999999999999,"5min":1.1910000000000001,"15min":1.202},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.81799999999999995,"15min":0.85299999999999998},"min":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.1639999999999999,"15min":1.3160000000000001},"last":0.74399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1585152,"data_stored":732882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.339,"5min":1.339,"15min":4.0999999999999996},"last":0.435},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.9239999999999999},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.36599999999999999,"5min":0.32800000000000001,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":3.1139999999999999},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":2.4159999999999999},"last":0.48799999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73599999999999999,"5min":0.86699999999999999,"15min":0.93000000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.907},"last":0.46100000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.129,"5min":1.129,"15min":1.9470000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77400000000000002,"15min":0.81699999999999995},"min":{"1min":0.56599999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.972},"last":0.53300000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77900000000000003,"15min":0.81200000000000006},"min":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.3,"5min":1.3,"15min":2.4569999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77600000000000002,"15min":0.80700000000000005},"min":{"1min":0.60399999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":2.8039999999999998},"last":0.55900000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0980000000000001,"15min":2.8849999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.81299999999999994},"min":{"1min":0.56499999999999995,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.867},"last":0.61799999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78500000000000003,"15min":0.83199999999999996},"min":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.495},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":2.3690000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79000000000000004,"15min":0.83299999999999996},"min":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.0489999999999999,"15min":2.851},"last":0.68700000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5795840,"data_stored":4933781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.27600000000000002,"5min":0.27200000000000002,"15min":0.253},"max":{"1min":0.66400000000000003,"5min":0.72199999999999998,"15min":0.80300000000000005},"last":0.505},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.70799999999999996,"15min":0.76500000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46000000000000002,"15min":0.45700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29199999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.373},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.84099999999999997,"15min":0.89400000000000002},"last":0.42299999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.42599999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.72599999999999998,"5min":0.76600000000000001,"15min":0.86899999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.50600000000000001},"min":{"1min":0.34899999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.83199999999999996,"15min":0.83999999999999997},"last":0.48199999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.746,"15min":0.73399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.38800000000000001},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.753,"15min":0.755},"min":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98199999999999998,"5min":0.999,"15min":1.2},"last":0.84499999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51600000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78300000000000003,"15min":0.77300000000000002},"min":{"1min":0.60499999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0329999999999999,"5min":1.129,"15min":1.2330000000000001},"last":0.91300000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.113,"5min":1.113,"15min":1.179},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.755,"15min":0.77100000000000002},"min":{"1min":0.56299999999999994,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.2150000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.57999999999999996,"5min":0.56200000000000006,"15min":0.48499999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.236},"last":0.86199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2183168,"data_stored":1323272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.442,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.29099999999999998,"15min":0.245},"max":{"1min":0.66200000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.311,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67500000000000004,"5min":0.77700000000000002,"15min":0.85099999999999998},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.30199999999999999},"max":{"1min":0.621,"5min":0.67100000000000004,"15min":1.02},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70999999999999996,"5min":0.75,"15min":1.006},"last":0.54500000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.754,"15min":0.84799999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56699999999999995,"15min":0.57599999999999996},"min":{"1min":0.45200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.67600000000000005,"5min":0.83999999999999997,"15min":0.96899999999999997},"last":0.52400000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.622,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.5169999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81499999999999995,"15min":0.81399999999999995},"min":{"1min":0.61799999999999999,"5min":0.52300000000000002,"15min":0.46400000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.252},"last":0.80100000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.55900000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0029999999999999,"5min":1.075,"15min":1.3080000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.462},"last":0.75}]},{"osd":6,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"max":{"1min":1.377,"5min":1.377,"15min":1.5920000000000001},"last":0.88},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79900000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.46999999999999997},"max":{"1min":1.139,"5min":1.1990000000000001,"15min":1.2230000000000001},"last":0.79000000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.80600000000000005,"15min":0.81299999999999994},"min":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.3600000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.79200000000000004,"15min":0.80100000000000005},"min":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.556},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:51:52.129 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:51:52.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f483d000640 1 Processor -- start 2024-03-30T00:51:52.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f483d000640 1 -- start start 2024-03-30T00:51:52.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f483d000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f48380ce520 0x7f48380ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f483d000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f48380cabb0 0x7f48380cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f483d000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483806d070 con 0x7f483815a640 2024-03-30T00:51:52.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f483d000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48380cbba0 con 0x7f48380cabb0 2024-03-30T00:51:52.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f483d000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48380cee70 con 0x7f48380ce520 2024-03-30T00:51:52.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f4836800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f48380cabb0 0x7f48380cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f4835e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f483815a640 0x7f483806dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35302/0 (socket says 172.21.15.66:35302) 2024-03-30T00:51:52.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f4836800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f48380cabb0 0x7f48380cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48708/0 (socket says 172.21.15.66:48708) 2024-03-30T00:51:52.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f4835e00640 1 -- 172.21.15.66:0/2885780337 learned_addr learned my addr 172.21.15.66:0/2885780337 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:52.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f482d400640 1 --2- 172.21.15.66:0/2885780337 >> v2:172.21.15.156:3300/0 conn(0x7f48380ce520 0x7f48380ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f4836800640 1 -- 172.21.15.66:0/2885780337 >> v2:172.21.15.156:3300/0 conn(0x7f48380ce520 msgr2=0x7f48380ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:52.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f4836800640 1 --2- 172.21.15.66:0/2885780337 >> v2:172.21.15.156:3300/0 conn(0x7f48380ce520 0x7f48380ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f4836800640 1 -- 172.21.15.66:0/2885780337 >> v1:172.21.15.66:6789/0 conn(0x7f483815a640 legacy=0x7f483806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f4835400640 1 -- 172.21.15.66:0/2885780337 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398499013 0 0) 0x7f483806d070 con 0x7f483815a640 2024-03-30T00:51:52.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.263+0000 7f4836800640 1 -- 172.21.15.66:0/2885780337 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48380cb8c0 con 0x7f48380cabb0 2024-03-30T00:51:52.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f4836800640 1 --2- 172.21.15.66:0/2885780337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48380cabb0 0x7f48380cafe0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f48380c88d0 tx=0x7f483001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63e2038f8764a32 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:52.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f4835400640 1 -- 172.21.15.66:0/2885780337 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300a59c0 con 0x7f48380cabb0 2024-03-30T00:51:52.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f4835400640 1 -- 172.21.15.66:0/2885780337 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48300a5b20 con 0x7f48380cabb0 2024-03-30T00:51:52.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f4835400640 1 -- 172.21.15.66:0/2885780337 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300ad440 con 0x7f48380cabb0 2024-03-30T00:51:52.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f483d000640 1 -- 172.21.15.66:0/2885780337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48380cabb0 msgr2=0x7f48380cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f483d000640 1 --2- 172.21.15.66:0/2885780337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48380cabb0 0x7f48380cafe0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f48380c88d0 tx=0x7f483001c900 comp rx=0 tx=0).stop 2024-03-30T00:51:52.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f483d000640 1 -- 172.21.15.66:0/2885780337 shutdown_connections 2024-03-30T00:51:52.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f483d000640 1 --2- 172.21.15.66:0/2885780337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48380cabb0 0x7f48380cafe0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f483d000640 1 --2- 172.21.15.66:0/2885780337 >> v2:172.21.15.156:3300/0 conn(0x7f48380ce520 0x7f48380ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f483d000640 1 -- 172.21.15.66:0/2885780337 >> 172.21.15.66:0/2885780337 conn(0x7f4838069f60 msgr2=0x7f483806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:52.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f483d000640 1 -- 172.21.15.66:0/2885780337 shutdown_connections 2024-03-30T00:51:52.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f483d000640 1 -- 172.21.15.66:0/2885780337 wait complete. 2024-03-30T00:51:52.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f483d000640 1 Processor -- start 2024-03-30T00:51:52.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.267+0000 7f483d000640 1 -- start start 2024-03-30T00:51:52.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f483d000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48380cabb0 0x7f4838152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f483d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48380ce520 0x7f4838163190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f483d000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f483815a640 0x7f48381636d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f483d000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48380c87b0 con 0x7f48380ce520 2024-03-30T00:51:52.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f483d000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48380c9050 con 0x7f48380cabb0 2024-03-30T00:51:52.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f483d000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48380c4540 con 0x7f483815a640 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48380ce520 0x7f4838163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48380ce520 0x7f4838163190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48710/0 (socket says 172.21.15.66:48710) 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482d400640 1 -- 172.21.15.66:0/2171788921 learned_addr learned my addr 172.21.15.66:0/2171788921 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f4835e00640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48380cabb0 0x7f4838152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f4836800640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f483815a640 0x7f48381636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482d400640 1 -- 172.21.15.66:0/2171788921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f483815a640 msgr2=0x7f48381636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482d400640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f483815a640 0x7f48381636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482d400640 1 -- 172.21.15.66:0/2171788921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48380cabb0 msgr2=0x7f4838152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482d400640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48380cabb0 0x7f4838152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482d400640 1 -- 172.21.15.66:0/2171788921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4830009a70 con 0x7f48380ce520 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f4836800640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f483815a640 0x7f48381636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f4835e00640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48380cabb0 0x7f4838152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482d400640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48380ce520 0x7f4838163190 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f482000e930 tx=0x7f482000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482ea00640 1 -- 172.21.15.66:0/2171788921 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820009690 con 0x7f48380ce520 2024-03-30T00:51:52.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482ea00640 1 -- 172.21.15.66:0/2171788921 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482000ca00 con 0x7f48380ce520 2024-03-30T00:51:52.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f483d000640 1 -- 172.21.15.66:0/2171788921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48380c8a00 con 0x7f48380ce520 2024-03-30T00:51:52.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482ea00640 1 -- 172.21.15.66:0/2171788921 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482000cd70 con 0x7f48380ce520 2024-03-30T00:51:52.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f483d000640 1 -- 172.21.15.66:0/2171788921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48380adbe0 con 0x7f48380ce520 2024-03-30T00:51:52.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f483d000640 1 -- 172.21.15.66:0/2171788921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4838162eb0 con 0x7f48380ce520 2024-03-30T00:51:52.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.271+0000 7f482ea00640 1 -- 172.21.15.66:0/2171788921 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4820013470 con 0x7f48380ce520 2024-03-30T00:51:52.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.275+0000 7f482ea00640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4814066690 0x7f4814068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.275+0000 7f482ea00640 1 -- 172.21.15.66:0/2171788921 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f4820084370 con 0x7f48380ce520 2024-03-30T00:51:52.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.275+0000 7f4835e00640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4814066690 0x7f4814068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.275+0000 7f4835e00640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4814066690 0x7f4814068b50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4828000f70 tx=0x7f4828007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:52.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.279+0000 7f482ea00640 1 -- 172.21.15.66:0/2171788921 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f48200518a0 con 0x7f48380ce520 2024-03-30T00:51:52.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.431+0000 7f483d000640 1 -- 172.21.15.66:0/2171788921 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f483815ba30 con 0x7f4814066690 2024-03-30T00:51:52.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.435+0000 7f482ea00640 1 -- 172.21.15.66:0/2171788921 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50621 (secure 0 0 0) 0x7f48280073a0 con 0x7f4814066690 2024-03-30T00:51:52.445 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:52.445 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:51:52.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 -- 172.21.15.66:0/2171788921 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4814066690 msgr2=0x7f4814068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4814066690 0x7f4814068b50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4828000f70 tx=0x7f4828007cd0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 -- 172.21.15.66:0/2171788921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48380ce520 msgr2=0x7f4838163190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48380ce520 0x7f4838163190 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f482000e930 tx=0x7f482000edf0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 -- 172.21.15.66:0/2171788921 shutdown_connections 2024-03-30T00:51:52.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f483815a640 0x7f48381636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f4814066690 0x7f4814068b50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48380ce520 0x7f4838163190 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 --2- 172.21.15.66:0/2171788921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48380cabb0 0x7f4838152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 -- 172.21.15.66:0/2171788921 >> 172.21.15.66:0/2171788921 conn(0x7f4838069f60 msgr2=0x7f4838158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:52.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 -- 172.21.15.66:0/2171788921 shutdown_connections 2024-03-30T00:51:52.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.439+0000 7f483d000640 1 -- 172.21.15.66:0/2171788921 wait complete. 2024-03-30T00:51:52.466 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-03-30T00:51:51.015428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278948,"kb_used_data":30032,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754695772,"statfs":{"total":773094113280,"available":772808470528,"internally_reserved":0,"allocated":30752768,"data_stored":23854800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392378,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-177,"num_write_kb":-8297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000995"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":277,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-03-30T00:51:42.965505+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:51:42.965505+0000","last_peered":"2024-03-30T00:51:42.965505+0000","last_clean":"2024-03-30T00:51:42.965505+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:51:42.965505+0000","last_undegraded":"2024-03-30T00:51:42.965505+0000","last_fullsized":"2024-03-30T00:51:42.965505+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574996,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":2207744,"data_stored":1330601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.65600000000000003},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84399999999999997,"5min":0.97599999999999998,"15min":1.0860000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66900000000000004,"15min":0.65300000000000002},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.98299999999999998,"15min":1.149},"last":0.65700000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.65800000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.03,"15min":1.03},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.46700000000000003,"5min":0.442,"15min":0.40400000000000003},"max":{"1min":0.81399999999999995,"5min":0.93000000000000005,"15min":1.3029999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.629,"5min":0.49399999999999999,"15min":0.45000000000000001},"max":{"1min":0.97499999999999998,"5min":1.085,"15min":1.6020000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.73699999999999999,"15min":0.75600000000000001},"min":{"1min":0.503,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93600000000000005,"5min":1.034,"15min":1.6619999999999999},"last":0.73599999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.755},"min":{"1min":0.47899999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":2.0150000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78000000000000003,"15min":0.79800000000000004},"min":{"1min":0.59999999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.96599999999999997,"5min":1.006,"15min":1.9490000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41099999999999998,"15min":0.37},"max":{"1min":0.92000000000000004,"5min":1.2,"15min":1.2},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.89700000000000002,"5min":0.95799999999999996,"15min":0.97899999999999998},"last":0.88900000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.45800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0649999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72599999999999998,"15min":0.74299999999999999},"min":{"1min":0.503,"5min":0.441,"15min":0.41699999999999998},"max":{"1min":0.98799999999999999,"5min":0.99199999999999999,"15min":1.026},"last":0.878}]},{"osd":6,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.54100000000000004,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.89200000000000002,"5min":0.94399999999999995,"15min":1.214},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.49099999999999999,"5min":0.45800000000000002,"15min":0.41199999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0309999999999999},"last":0.85499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5791744,"data_stored":4934336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69799999999999995,"15min":0.77900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.995,"15min":4.0490000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.48899999999999999,"5min":0.46400000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":1.091,"15min":1.47},"last":0.78500000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.78600000000000003},"min":{"1min":0.52000000000000002,"5min":0.438,"15min":0.36899999999999999},"max":{"1min":0.96199999999999997,"5min":1.204,"15min":4.3170000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72899999999999998,"15min":0.83099999999999996},"min":{"1min":0.52600000000000002,"5min":0.47199999999999998,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":1.1120000000000001,"15min":4.4160000000000004},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73599999999999999,"15min":0.83099999999999996},"min":{"1min":0.57899999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95099999999999996,"5min":1.0720000000000001,"15min":4.375},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70799999999999996,"15min":0.81999999999999995},"min":{"1min":0.47799999999999998,"5min":0.42999999999999999,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":1.0229999999999999,"15min":4.2960000000000003},"last":0.72999999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.749,"15min":0.84599999999999997},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":1.2390000000000001,"15min":4.3399999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71899999999999997,"15min":0.82699999999999996},"min":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.41999999999999998},"max":{"1min":0.91700000000000004,"5min":1.081,"15min":4.6020000000000003},"last":0.70799999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44600000000000001,"5min":0.39100000000000001,"15min":0.35999999999999999},"max":{"1min":0.871,"5min":1.0089999999999999,"15min":1.1870000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69599999999999995,"15min":0.72599999999999998},"min":{"1min":0.432,"5min":0.372,"15min":0.372},"max":{"1min":0.90500000000000003,"5min":0.98699999999999999,"15min":1.036},"last":0.83599999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61799999999999999,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91400000000000003,"5min":1.0449999999999999,"15min":1.0720000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68000000000000005,"15min":0.69099999999999995},"min":{"1min":0.46800000000000003,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.96799999999999997,"5min":1.0109999999999999,"15min":1.101},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66800000000000004,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.46200000000000002,"15min":0.45700000000000002},"max":{"1min":0.94999999999999996,"5min":1.103,"15min":1.103},"last":0.76200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5795840,"data_stored":4933914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":1.036,"5min":1.036,"15min":1.095},"last":1.01},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67600000000000005,"15min":0.72399999999999998},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.38800000000000001},"max":{"1min":1.0089999999999999,"5min":1.0449999999999999,"15min":1.7569999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67200000000000004,"15min":0.71299999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.43099999999999999},"max":{"1min":0.88600000000000001,"5min":0.97399999999999998,"15min":1.7330000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.44500000000000001,"5min":0.43099999999999999,"15min":0.42399999999999999},"max":{"1min":0.95099999999999996,"5min":0.98799999999999999,"15min":1.3260000000000001},"last":0.69499999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70799999999999996,"15min":0.71499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.024,"5min":1.131,"15min":1.131},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.47999999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.97899999999999998,"5min":1.0609999999999999,"15min":1.2789999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70099999999999996,"15min":0.74299999999999999},"min":{"1min":0.48399999999999999,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":1.024,"5min":1.0740000000000001,"15min":1.7},"last":0.755},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72599999999999998,"15min":0.74099999999999999},"min":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.999,"5min":1.1160000000000001,"15min":1.181},"last":0.79800000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.314,"15min":0.309},"max":{"1min":0.746,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.72699999999999998},"min":{"1min":0.53500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":1.615},"last":0.877}]},{"osd":6,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.97399999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.624,"15min":0.66400000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35199999999999998},"max":{"1min":0.84499999999999997,"5min":0.93700000000000006,"15min":1.569},"last":0.97299999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.70899999999999996},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.377},"max":{"1min":0.90500000000000003,"5min":0.92200000000000004,"15min":1.0149999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.78700000000000003},"min":{"1min":0.53600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92300000000000004,"5min":1.167,"15min":1.383},"last":0.89700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5795840,"data_stored":4930148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.42099999999999999,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.97099999999999997,"15min":1.4199999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.67900000000000005},"min":{"1min":0.502,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.877,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.48999999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.47699999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.93999999999999995,"5min":0.95399999999999996,"15min":1.2909999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.71499999999999997},"min":{"1min":0.49199999999999999,"5min":0.46600000000000003,"15min":0.40699999999999997},"max":{"1min":0.86199999999999999,"5min":1.113,"15min":1.113},"last":0.628}]},{"osd":2,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.51600000000000001,"5min":0.48999999999999999,"15min":0.48499999999999999},"max":{"1min":0.96999999999999997,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.748},"min":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":0.98499999999999999,"5min":1.163,"15min":1.228},"last":0.76700000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.71799999999999997},"min":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.46800000000000003},"max":{"1min":0.91800000000000004,"5min":1.0800000000000001,"15min":1.27},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.75600000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.44500000000000001},"max":{"1min":0.93200000000000005,"5min":0.99399999999999999,"15min":1.1739999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.629,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.329},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.65800000000000003,"15min":0.69299999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.33300000000000002},"max":{"1min":0.85899999999999999,"5min":0.99099999999999999,"15min":1.3819999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.2130000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.68700000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39000000000000001},"max":{"1min":0.89200000000000002,"5min":0.95799999999999996,"15min":1.3220000000000001},"last":0.73199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63,"15min":0.68200000000000005},"min":{"1min":0.438,"5min":0.42399999999999999,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":0.91500000000000004,"15min":1.6910000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72799999999999998,"15min":0.77000000000000002},"min":{"1min":0.51600000000000001,"5min":0.51400000000000001,"15min":0.46600000000000003},"max":{"1min":0.90200000000000002,"5min":0.97799999999999998,"15min":1.5009999999999999},"last":0.745}]}]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1597440,"data_stored":735866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.434,"15min":0.45300000000000001},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.68999999999999995,"5min":0.748,"15min":1.0529999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47799999999999998,"15min":0.50800000000000001},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.24199999999999999},"max":{"1min":0.72199999999999998,"5min":0.75,"15min":0.79100000000000004},"last":0.32500000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.5,"15min":0.504},"min":{"1min":0.42899999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74199999999999999,"5min":0.79100000000000004,"15min":0.80900000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.51900000000000002,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.32100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":1.2150000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.79900000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.434}]},{"osd":4,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75700000000000001,"15min":0.76200000000000001},"min":{"1min":0.59399999999999997,"5min":0.47799999999999998,"15min":0.46300000000000002},"max":{"1min":0.96199999999999997,"5min":1.1259999999999999,"15min":1.2470000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.59099999999999997,"5min":0.51600000000000001,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":1.079,"15min":1.0860000000000001},"last":0.64700000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76900000000000002,"15min":0.77700000000000002},"min":{"1min":0.58099999999999996,"5min":0.54600000000000004,"15min":0.51100000000000001},"max":{"1min":1.0169999999999999,"5min":1.0329999999999999,"15min":1.125},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.62,"5min":0.52200000000000002,"15min":0.48599999999999999},"max":{"1min":0.93000000000000005,"5min":1.133,"15min":1.2390000000000001},"last":0.63}]},{"osd":6,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76500000000000001,"15min":0.79300000000000004},"min":{"1min":0.58099999999999996,"5min":0.51400000000000001,"15min":0.499},"max":{"1min":1.0049999999999999,"5min":1.091,"15min":1.226},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.1120000000000001,"15min":1.167},"last":0.60299999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.67400000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0549999999999999,"5min":1.1910000000000001,"15min":1.202},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.81799999999999995,"15min":0.85299999999999998},"min":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.1639999999999999,"15min":1.3160000000000001},"last":0.74399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1585152,"data_stored":732882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.339,"5min":1.339,"15min":4.0999999999999996},"last":0.435},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.9239999999999999},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.36599999999999999,"5min":0.32800000000000001,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":3.1139999999999999},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":2.4159999999999999},"last":0.48799999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73599999999999999,"5min":0.86699999999999999,"15min":0.93000000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.907},"last":0.46100000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.129,"5min":1.129,"15min":1.9470000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77400000000000002,"15min":0.81699999999999995},"min":{"1min":0.56599999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.972},"last":0.53300000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77900000000000003,"15min":0.81200000000000006},"min":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.3,"5min":1.3,"15min":2.4569999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77600000000000002,"15min":0.80700000000000005},"min":{"1min":0.60399999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":2.8039999999999998},"last":0.55900000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0980000000000001,"15min":2.8849999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.81299999999999994},"min":{"1min":0.56499999999999995,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.867},"last":0.61799999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78500000000000003,"15min":0.83199999999999996},"min":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.495},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":2.3690000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79000000000000004,"15min":0.83299999999999996},"min":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.0489999999999999,"15min":2.851},"last":0.68700000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5795840,"data_stored":4933781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.27600000000000002,"5min":0.27200000000000002,"15min":0.253},"max":{"1min":0.66400000000000003,"5min":0.72199999999999998,"15min":0.80300000000000005},"last":0.505},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.70799999999999996,"15min":0.76500000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46000000000000002,"15min":0.45700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29199999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.373},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.84099999999999997,"15min":0.89400000000000002},"last":0.42299999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.42599999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.72599999999999998,"5min":0.76600000000000001,"15min":0.86899999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.50600000000000001},"min":{"1min":0.34899999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.83199999999999996,"15min":0.83999999999999997},"last":0.48199999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.746,"15min":0.73399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.38800000000000001},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.753,"15min":0.755},"min":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98199999999999998,"5min":0.999,"15min":1.2},"last":0.84499999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51600000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78300000000000003,"15min":0.77300000000000002},"min":{"1min":0.60499999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0329999999999999,"5min":1.129,"15min":1.2330000000000001},"last":0.91300000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.113,"5min":1.113,"15min":1.179},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.755,"15min":0.77100000000000002},"min":{"1min":0.56299999999999994,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.2150000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.57999999999999996,"5min":0.56200000000000006,"15min":0.48499999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.236},"last":0.86199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2183168,"data_stored":1323272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.442,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.29099999999999998,"15min":0.245},"max":{"1min":0.66200000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.311,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67500000000000004,"5min":0.77700000000000002,"15min":0.85099999999999998},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.30199999999999999},"max":{"1min":0.621,"5min":0.67100000000000004,"15min":1.02},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70999999999999996,"5min":0.75,"15min":1.006},"last":0.54500000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.754,"15min":0.84799999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56699999999999995,"15min":0.57599999999999996},"min":{"1min":0.45200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.67600000000000005,"5min":0.83999999999999997,"15min":0.96899999999999997},"last":0.52400000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.622,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.5169999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81499999999999995,"15min":0.81399999999999995},"min":{"1min":0.61799999999999999,"5min":0.52300000000000002,"15min":0.46400000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.252},"last":0.80100000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.55900000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0029999999999999,"5min":1.075,"15min":1.3080000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.462},"last":0.75}]},{"osd":6,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"max":{"1min":1.377,"5min":1.377,"15min":1.5920000000000001},"last":0.88},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79900000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.46999999999999997},"max":{"1min":1.139,"5min":1.1990000000000001,"15min":1.2230000000000001},"last":0.79000000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.80600000000000005,"15min":0.81299999999999994},"min":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.3600000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.79200000000000004,"15min":0.80100000000000005},"min":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.556},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:51:52.467 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-30T00:51:52.467 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:51:52.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9939c00640 1 Processor -- start 2024-03-30T00:51:52.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9939c00640 1 -- start start 2024-03-30T00:51:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9939c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9934098c80 0x7f99340ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9939c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9934099060 0x7f99340aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9939c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9934099760 con 0x7f99340b0470 2024-03-30T00:51:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9939c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99340cb8f0 con 0x7f9934098c80 2024-03-30T00:51:52.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9939c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99340999b0 con 0x7f9934099060 2024-03-30T00:51:52.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9932a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9934098c80 0x7f99340ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9933400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f99340b0470 0x7f99340cbd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35304/0 (socket says 172.21.15.66:35304) 2024-03-30T00:51:52.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9932a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9934098c80 0x7f99340ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48718/0 (socket says 172.21.15.66:48718) 2024-03-30T00:51:52.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9933400640 1 -- 172.21.15.66:0/2610049070 learned_addr learned my addr 172.21.15.66:0/2610049070 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:52.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f992be00640 1 --2- 172.21.15.66:0/2610049070 >> v2:172.21.15.156:3300/0 conn(0x7f9934099060 0x7f99340aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9932a00640 1 -- 172.21.15.66:0/2610049070 >> v2:172.21.15.156:3300/0 conn(0x7f9934099060 msgr2=0x7f99340aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:52.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9932a00640 1 --2- 172.21.15.66:0/2610049070 >> v2:172.21.15.156:3300/0 conn(0x7f9934099060 0x7f99340aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9932a00640 1 -- 172.21.15.66:0/2610049070 >> v1:172.21.15.66:6789/0 conn(0x7f99340b0470 legacy=0x7f99340cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.599+0000 7f9932a00640 1 -- 172.21.15.66:0/2610049070 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9934099c90 con 0x7f9934098c80 2024-03-30T00:51:52.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9932a00640 1 --2- 172.21.15.66:0/2610049070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9934098c80 0x7f99340ce3b0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f9934071940 tx=0x7f992401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64c5d5a65dbe94 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:52.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9932000640 1 -- 172.21.15.66:0/2610049070 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240ab650 con 0x7f9934098c80 2024-03-30T00:51:52.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9932000640 1 -- 172.21.15.66:0/2610049070 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99240ab7b0 con 0x7f9934098c80 2024-03-30T00:51:52.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9932000640 1 -- 172.21.15.66:0/2610049070 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240abb40 con 0x7f9934098c80 2024-03-30T00:51:52.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9939c00640 1 -- 172.21.15.66:0/2610049070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9934098c80 msgr2=0x7f99340ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9939c00640 1 --2- 172.21.15.66:0/2610049070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9934098c80 0x7f99340ce3b0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f9934071940 tx=0x7f992401c450 comp rx=0 tx=0).stop 2024-03-30T00:51:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9939c00640 1 -- 172.21.15.66:0/2610049070 shutdown_connections 2024-03-30T00:51:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9939c00640 1 --2- 172.21.15.66:0/2610049070 >> v2:172.21.15.156:3300/0 conn(0x7f9934099060 0x7f99340aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9939c00640 1 --2- 172.21.15.66:0/2610049070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9934098c80 0x7f99340ce3b0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9939c00640 1 -- 172.21.15.66:0/2610049070 >> 172.21.15.66:0/2610049070 conn(0x7f99340b20c0 msgr2=0x7f99340b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9939c00640 1 -- 172.21.15.66:0/2610049070 shutdown_connections 2024-03-30T00:51:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.603+0000 7f9939c00640 1 -- 172.21.15.66:0/2610049070 wait complete. 2024-03-30T00:51:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9939c00640 1 Processor -- start 2024-03-30T00:51:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9939c00640 1 -- start start 2024-03-30T00:51:52.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9939c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9934098c80 0x7f9934158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9939c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9934099060 0x7f99341f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9939c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99340b0470 0x7f99341f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9939c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993406b8e0 con 0x7f9934098c80 2024-03-30T00:51:52.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9939c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993406bbc0 con 0x7f9934099060 2024-03-30T00:51:52.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9939c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993406bea0 con 0x7f99340b0470 2024-03-30T00:51:52.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9932a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9934098c80 0x7f9934158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9932a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9934098c80 0x7f9934158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48720/0 (socket says 172.21.15.66:48720) 2024-03-30T00:51:52.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9932a00640 1 -- 172.21.15.66:0/2037027625 learned_addr learned my addr 172.21.15.66:0/2037027625 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f992be00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9934099060 0x7f99341f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9933400640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99340b0470 0x7f99341f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9932a00640 1 -- 172.21.15.66:0/2037027625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99340b0470 msgr2=0x7f99341f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9932a00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99340b0470 0x7f99341f92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9932a00640 1 -- 172.21.15.66:0/2037027625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9934099060 msgr2=0x7f99341f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9932a00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9934099060 0x7f99341f8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9932a00640 1 -- 172.21.15.66:0/2037027625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9924009420 con 0x7f9934098c80 2024-03-30T00:51:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9932a00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9934098c80 0x7f9934158050 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f9924019060 tx=0x7f992401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f992aa00640 1 -- 172.21.15.66:0/2037027625 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240bf610 con 0x7f9934098c80 2024-03-30T00:51:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f992aa00640 1 -- 172.21.15.66:0/2037027625 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99240bf770 con 0x7f9934098c80 2024-03-30T00:51:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f992aa00640 1 -- 172.21.15.66:0/2037027625 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240bfb20 con 0x7f9934098c80 2024-03-30T00:51:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9939c00640 1 -- 172.21.15.66:0/2037027625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993406c180 con 0x7f9934098c80 2024-03-30T00:51:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.607+0000 7f9939c00640 1 -- 172.21.15.66:0/2037027625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9934158590 con 0x7f9934098c80 2024-03-30T00:51:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.611+0000 7f9939c00640 1 -- 172.21.15.66:0/2037027625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9934066c30 con 0x7f9934098c80 2024-03-30T00:51:52.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.611+0000 7f992aa00640 1 -- 172.21.15.66:0/2037027625 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f99240bfc80 con 0x7f9934098c80 2024-03-30T00:51:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.611+0000 7f992aa00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9910066690 0x7f9910068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.611+0000 7f992aa00640 1 -- 172.21.15.66:0/2037027625 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f992411ea30 con 0x7f9934098c80 2024-03-30T00:51:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.611+0000 7f992be00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9910066690 0x7f9910068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.611+0000 7f992be00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9910066690 0x7f9910068b50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f99340af2d0 tx=0x7f991c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:52.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.615+0000 7f992aa00640 1 -- 172.21.15.66:0/2037027625 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f99240ebf60 con 0x7f9934098c80 2024-03-30T00:51:52.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.771+0000 7f9939c00640 1 -- 172.21.15.66:0/2037027625 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f993406bd30 con 0x7f9910066690 2024-03-30T00:51:52.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f992aa00640 1 -- 172.21.15.66:0/2037027625 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50621 (secure 0 0 0) 0x7f993406bd30 con 0x7f9910066690 2024-03-30T00:51:52.784 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:52.784 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:51:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f9939c00640 1 -- 172.21.15.66:0/2037027625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9910066690 msgr2=0x7f9910068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f9939c00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9910066690 0x7f9910068b50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f99340af2d0 tx=0x7f991c009190 comp rx=0 tx=0).stop 2024-03-30T00:51:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f9939c00640 1 -- 172.21.15.66:0/2037027625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9934098c80 msgr2=0x7f9934158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f9939c00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9934098c80 0x7f9934158050 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f9924019060 tx=0x7f992401ac90 comp rx=0 tx=0).stop 2024-03-30T00:51:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f9939c00640 1 -- 172.21.15.66:0/2037027625 shutdown_connections 2024-03-30T00:51:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f9939c00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f99340b0470 0x7f99341f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f9939c00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9934099060 0x7f99341f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f9939c00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9910066690 0x7f9910068b50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f9939c00640 1 --2- 172.21.15.66:0/2037027625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9934098c80 0x7f9934158050 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f9939c00640 1 -- 172.21.15.66:0/2037027625 >> 172.21.15.66:0/2037027625 conn(0x7f99340b20c0 msgr2=0x7f9934164110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:52.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.775+0000 7f9939c00640 1 -- 172.21.15.66:0/2037027625 shutdown_connections 2024-03-30T00:51:52.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.779+0000 7f9939c00640 1 -- 172.21.15.66:0/2037027625 wait complete. 2024-03-30T00:51:52.805 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-03-30T00:51:51.015428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278948,"kb_used_data":30032,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754695772,"statfs":{"total":773094113280,"available":772808470528,"internally_reserved":0,"allocated":30752768,"data_stored":23854800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392378,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-177,"num_write_kb":-8297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000995"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":277,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-03-30T00:51:42.965505+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:51:42.965505+0000","last_peered":"2024-03-30T00:51:42.965505+0000","last_clean":"2024-03-30T00:51:42.965505+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:51:42.965505+0000","last_undegraded":"2024-03-30T00:51:42.965505+0000","last_fullsized":"2024-03-30T00:51:42.965505+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574996,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":2207744,"data_stored":1330601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.65600000000000003},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84399999999999997,"5min":0.97599999999999998,"15min":1.0860000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66900000000000004,"15min":0.65300000000000002},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.98299999999999998,"15min":1.149},"last":0.65700000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.65800000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.03,"15min":1.03},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.46700000000000003,"5min":0.442,"15min":0.40400000000000003},"max":{"1min":0.81399999999999995,"5min":0.93000000000000005,"15min":1.3029999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.629,"5min":0.49399999999999999,"15min":0.45000000000000001},"max":{"1min":0.97499999999999998,"5min":1.085,"15min":1.6020000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.73699999999999999,"15min":0.75600000000000001},"min":{"1min":0.503,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93600000000000005,"5min":1.034,"15min":1.6619999999999999},"last":0.73599999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.755},"min":{"1min":0.47899999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":2.0150000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78000000000000003,"15min":0.79800000000000004},"min":{"1min":0.59999999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.96599999999999997,"5min":1.006,"15min":1.9490000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41099999999999998,"15min":0.37},"max":{"1min":0.92000000000000004,"5min":1.2,"15min":1.2},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.89700000000000002,"5min":0.95799999999999996,"15min":0.97899999999999998},"last":0.88900000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.45800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0649999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72599999999999998,"15min":0.74299999999999999},"min":{"1min":0.503,"5min":0.441,"15min":0.41699999999999998},"max":{"1min":0.98799999999999999,"5min":0.99199999999999999,"15min":1.026},"last":0.878}]},{"osd":6,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.54100000000000004,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.89200000000000002,"5min":0.94399999999999995,"15min":1.214},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.49099999999999999,"5min":0.45800000000000002,"15min":0.41199999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0309999999999999},"last":0.85499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5791744,"data_stored":4934336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69799999999999995,"15min":0.77900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.995,"15min":4.0490000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.48899999999999999,"5min":0.46400000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":1.091,"15min":1.47},"last":0.78500000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.78600000000000003},"min":{"1min":0.52000000000000002,"5min":0.438,"15min":0.36899999999999999},"max":{"1min":0.96199999999999997,"5min":1.204,"15min":4.3170000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72899999999999998,"15min":0.83099999999999996},"min":{"1min":0.52600000000000002,"5min":0.47199999999999998,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":1.1120000000000001,"15min":4.4160000000000004},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73599999999999999,"15min":0.83099999999999996},"min":{"1min":0.57899999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95099999999999996,"5min":1.0720000000000001,"15min":4.375},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70799999999999996,"15min":0.81999999999999995},"min":{"1min":0.47799999999999998,"5min":0.42999999999999999,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":1.0229999999999999,"15min":4.2960000000000003},"last":0.72999999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.749,"15min":0.84599999999999997},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":1.2390000000000001,"15min":4.3399999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71899999999999997,"15min":0.82699999999999996},"min":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.41999999999999998},"max":{"1min":0.91700000000000004,"5min":1.081,"15min":4.6020000000000003},"last":0.70799999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44600000000000001,"5min":0.39100000000000001,"15min":0.35999999999999999},"max":{"1min":0.871,"5min":1.0089999999999999,"15min":1.1870000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69599999999999995,"15min":0.72599999999999998},"min":{"1min":0.432,"5min":0.372,"15min":0.372},"max":{"1min":0.90500000000000003,"5min":0.98699999999999999,"15min":1.036},"last":0.83599999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61799999999999999,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91400000000000003,"5min":1.0449999999999999,"15min":1.0720000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68000000000000005,"15min":0.69099999999999995},"min":{"1min":0.46800000000000003,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.96799999999999997,"5min":1.0109999999999999,"15min":1.101},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66800000000000004,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.46200000000000002,"15min":0.45700000000000002},"max":{"1min":0.94999999999999996,"5min":1.103,"15min":1.103},"last":0.76200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5795840,"data_stored":4933914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":1.036,"5min":1.036,"15min":1.095},"last":1.01},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67600000000000005,"15min":0.72399999999999998},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.38800000000000001},"max":{"1min":1.0089999999999999,"5min":1.0449999999999999,"15min":1.7569999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67200000000000004,"15min":0.71299999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.43099999999999999},"max":{"1min":0.88600000000000001,"5min":0.97399999999999998,"15min":1.7330000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.44500000000000001,"5min":0.43099999999999999,"15min":0.42399999999999999},"max":{"1min":0.95099999999999996,"5min":0.98799999999999999,"15min":1.3260000000000001},"last":0.69499999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70799999999999996,"15min":0.71499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.024,"5min":1.131,"15min":1.131},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.47999999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.97899999999999998,"5min":1.0609999999999999,"15min":1.2789999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70099999999999996,"15min":0.74299999999999999},"min":{"1min":0.48399999999999999,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":1.024,"5min":1.0740000000000001,"15min":1.7},"last":0.755},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72599999999999998,"15min":0.74099999999999999},"min":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.999,"5min":1.1160000000000001,"15min":1.181},"last":0.79800000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.314,"15min":0.309},"max":{"1min":0.746,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.72699999999999998},"min":{"1min":0.53500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":1.615},"last":0.877}]},{"osd":6,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.97399999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.624,"15min":0.66400000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35199999999999998},"max":{"1min":0.84499999999999997,"5min":0.93700000000000006,"15min":1.569},"last":0.97299999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.70899999999999996},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.377},"max":{"1min":0.90500000000000003,"5min":0.92200000000000004,"15min":1.0149999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.78700000000000003},"min":{"1min":0.53600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92300000000000004,"5min":1.167,"15min":1.383},"last":0.89700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5795840,"data_stored":4930148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.42099999999999999,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.97099999999999997,"15min":1.4199999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.67900000000000005},"min":{"1min":0.502,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.877,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.48999999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.47699999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.93999999999999995,"5min":0.95399999999999996,"15min":1.2909999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.71499999999999997},"min":{"1min":0.49199999999999999,"5min":0.46600000000000003,"15min":0.40699999999999997},"max":{"1min":0.86199999999999999,"5min":1.113,"15min":1.113},"last":0.628}]},{"osd":2,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.51600000000000001,"5min":0.48999999999999999,"15min":0.48499999999999999},"max":{"1min":0.96999999999999997,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.748},"min":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":0.98499999999999999,"5min":1.163,"15min":1.228},"last":0.76700000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.71799999999999997},"min":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.46800000000000003},"max":{"1min":0.91800000000000004,"5min":1.0800000000000001,"15min":1.27},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.75600000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.44500000000000001},"max":{"1min":0.93200000000000005,"5min":0.99399999999999999,"15min":1.1739999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.629,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.329},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.65800000000000003,"15min":0.69299999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.33300000000000002},"max":{"1min":0.85899999999999999,"5min":0.99099999999999999,"15min":1.3819999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.2130000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.68700000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39000000000000001},"max":{"1min":0.89200000000000002,"5min":0.95799999999999996,"15min":1.3220000000000001},"last":0.73199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63,"15min":0.68200000000000005},"min":{"1min":0.438,"5min":0.42399999999999999,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":0.91500000000000004,"15min":1.6910000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72799999999999998,"15min":0.77000000000000002},"min":{"1min":0.51600000000000001,"5min":0.51400000000000001,"15min":0.46600000000000003},"max":{"1min":0.90200000000000002,"5min":0.97799999999999998,"15min":1.5009999999999999},"last":0.745}]}]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1597440,"data_stored":735866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.434,"15min":0.45300000000000001},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.68999999999999995,"5min":0.748,"15min":1.0529999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47799999999999998,"15min":0.50800000000000001},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.24199999999999999},"max":{"1min":0.72199999999999998,"5min":0.75,"15min":0.79100000000000004},"last":0.32500000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.5,"15min":0.504},"min":{"1min":0.42899999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74199999999999999,"5min":0.79100000000000004,"15min":0.80900000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.51900000000000002,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.32100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":1.2150000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.79900000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.434}]},{"osd":4,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75700000000000001,"15min":0.76200000000000001},"min":{"1min":0.59399999999999997,"5min":0.47799999999999998,"15min":0.46300000000000002},"max":{"1min":0.96199999999999997,"5min":1.1259999999999999,"15min":1.2470000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.59099999999999997,"5min":0.51600000000000001,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":1.079,"15min":1.0860000000000001},"last":0.64700000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76900000000000002,"15min":0.77700000000000002},"min":{"1min":0.58099999999999996,"5min":0.54600000000000004,"15min":0.51100000000000001},"max":{"1min":1.0169999999999999,"5min":1.0329999999999999,"15min":1.125},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.62,"5min":0.52200000000000002,"15min":0.48599999999999999},"max":{"1min":0.93000000000000005,"5min":1.133,"15min":1.2390000000000001},"last":0.63}]},{"osd":6,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76500000000000001,"15min":0.79300000000000004},"min":{"1min":0.58099999999999996,"5min":0.51400000000000001,"15min":0.499},"max":{"1min":1.0049999999999999,"5min":1.091,"15min":1.226},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.1120000000000001,"15min":1.167},"last":0.60299999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.67400000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0549999999999999,"5min":1.1910000000000001,"15min":1.202},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.81799999999999995,"15min":0.85299999999999998},"min":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.1639999999999999,"15min":1.3160000000000001},"last":0.74399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1585152,"data_stored":732882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.339,"5min":1.339,"15min":4.0999999999999996},"last":0.435},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.9239999999999999},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.36599999999999999,"5min":0.32800000000000001,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":3.1139999999999999},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":2.4159999999999999},"last":0.48799999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73599999999999999,"5min":0.86699999999999999,"15min":0.93000000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.907},"last":0.46100000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.129,"5min":1.129,"15min":1.9470000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77400000000000002,"15min":0.81699999999999995},"min":{"1min":0.56599999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.972},"last":0.53300000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77900000000000003,"15min":0.81200000000000006},"min":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.3,"5min":1.3,"15min":2.4569999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77600000000000002,"15min":0.80700000000000005},"min":{"1min":0.60399999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":2.8039999999999998},"last":0.55900000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0980000000000001,"15min":2.8849999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.81299999999999994},"min":{"1min":0.56499999999999995,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.867},"last":0.61799999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78500000000000003,"15min":0.83199999999999996},"min":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.495},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":2.3690000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79000000000000004,"15min":0.83299999999999996},"min":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.0489999999999999,"15min":2.851},"last":0.68700000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5795840,"data_stored":4933781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.27600000000000002,"5min":0.27200000000000002,"15min":0.253},"max":{"1min":0.66400000000000003,"5min":0.72199999999999998,"15min":0.80300000000000005},"last":0.505},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.70799999999999996,"15min":0.76500000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46000000000000002,"15min":0.45700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29199999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.373},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.84099999999999997,"15min":0.89400000000000002},"last":0.42299999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.42599999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.72599999999999998,"5min":0.76600000000000001,"15min":0.86899999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.50600000000000001},"min":{"1min":0.34899999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.83199999999999996,"15min":0.83999999999999997},"last":0.48199999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.746,"15min":0.73399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.38800000000000001},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.753,"15min":0.755},"min":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98199999999999998,"5min":0.999,"15min":1.2},"last":0.84499999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51600000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78300000000000003,"15min":0.77300000000000002},"min":{"1min":0.60499999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0329999999999999,"5min":1.129,"15min":1.2330000000000001},"last":0.91300000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.113,"5min":1.113,"15min":1.179},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.755,"15min":0.77100000000000002},"min":{"1min":0.56299999999999994,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.2150000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.57999999999999996,"5min":0.56200000000000006,"15min":0.48499999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.236},"last":0.86199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2183168,"data_stored":1323272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.442,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.29099999999999998,"15min":0.245},"max":{"1min":0.66200000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.311,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67500000000000004,"5min":0.77700000000000002,"15min":0.85099999999999998},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.30199999999999999},"max":{"1min":0.621,"5min":0.67100000000000004,"15min":1.02},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70999999999999996,"5min":0.75,"15min":1.006},"last":0.54500000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.754,"15min":0.84799999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56699999999999995,"15min":0.57599999999999996},"min":{"1min":0.45200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.67600000000000005,"5min":0.83999999999999997,"15min":0.96899999999999997},"last":0.52400000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.622,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.5169999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81499999999999995,"15min":0.81399999999999995},"min":{"1min":0.61799999999999999,"5min":0.52300000000000002,"15min":0.46400000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.252},"last":0.80100000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.55900000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0029999999999999,"5min":1.075,"15min":1.3080000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.462},"last":0.75}]},{"osd":6,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"max":{"1min":1.377,"5min":1.377,"15min":1.5920000000000001},"last":0.88},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79900000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.46999999999999997},"max":{"1min":1.139,"5min":1.1990000000000001,"15min":1.2230000000000001},"last":0.79000000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.80600000000000005,"15min":0.81299999999999994},"min":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.3600000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.79200000000000004,"15min":0.80100000000000005},"min":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.556},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:51:52.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:51:52.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.939+0000 7fb0c2200640 1 Processor -- start 2024-03-30T00:51:52.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.939+0000 7fb0c2200640 1 -- start start 2024-03-30T00:51:52.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.943+0000 7fb0c2200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb0bc15cb60 0x7fb0bc1604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.943+0000 7fb0c2200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc0b1d00 con 0x7fb0bc0cd260 2024-03-30T00:51:52.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.943+0000 7fb0c2200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc0cd820 con 0x7fb0bc15c780 2024-03-30T00:51:52.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.943+0000 7fb0c2200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc0cdb00 con 0x7fb0bc15cb60 2024-03-30T00:51:52.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.943+0000 7fb0b2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb0bc15cb60 0x7fb0bc1604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.943+0000 7fb0bbe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb0bc0cd260 0x7fb0bc0d0290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35308/0 (socket says 172.21.15.66:35308) 2024-03-30T00:51:52.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.943+0000 7fb0bbe00640 1 -- 172.21.15.66:0/1972364559 learned_addr learned my addr 172.21.15.66:0/1972364559 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:52.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.943+0000 7fb0b2a00640 1 -- 172.21.15.66:0/1972364559 >> v1:172.21.15.156:6789/0 conn(0x7fb0bc15c780 legacy=0x7fb0bc0cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:52.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.943+0000 7fb0b2a00640 1 -- 172.21.15.66:0/1972364559 >> v1:172.21.15.66:6789/0 conn(0x7fb0bc0cd260 legacy=0x7fb0bc0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.943+0000 7fb0b2a00640 1 -- 172.21.15.66:0/1972364559 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0cdde0 con 0x7fb0bc15cb60 2024-03-30T00:51:52.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.943+0000 7fb0baa00640 1 -- 172.21.15.66:0/1972364559 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640243162 0 0) 0x7fb0bc0b1d00 con 0x7fb0bc0cd260 2024-03-30T00:51:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0b2a00640 1 --2- 172.21.15.66:0/1972364559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0bc15cb60 0x7fb0bc1604a0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009d90 tx=0x7fb0a401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ce3135a4ca9ebf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0baa00640 1 -- 172.21.15.66:0/1972364559 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40a58d0 con 0x7fb0bc15cb60 2024-03-30T00:51:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0baa00640 1 -- 172.21.15.66:0/1972364559 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a40a5a30 con 0x7fb0bc15cb60 2024-03-30T00:51:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0baa00640 1 -- 172.21.15.66:0/1972364559 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40a5dc0 con 0x7fb0bc15cb60 2024-03-30T00:51:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 -- 172.21.15.66:0/1972364559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0bc15cb60 msgr2=0x7fb0bc1604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 --2- 172.21.15.66:0/1972364559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0bc15cb60 0x7fb0bc1604a0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009d90 tx=0x7fb0a401c900 comp rx=0 tx=0).stop 2024-03-30T00:51:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 -- 172.21.15.66:0/1972364559 shutdown_connections 2024-03-30T00:51:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 --2- 172.21.15.66:0/1972364559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0bc15cb60 0x7fb0bc1604a0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 -- 172.21.15.66:0/1972364559 >> 172.21.15.66:0/1972364559 conn(0x7fb0bc09ba30 msgr2=0x7fb0bc089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 -- 172.21.15.66:0/1972364559 shutdown_connections 2024-03-30T00:51:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 -- 172.21.15.66:0/1972364559 wait complete. 2024-03-30T00:51:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 Processor -- start 2024-03-30T00:51:52.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 -- start start 2024-03-30T00:51:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc0cd260 0x7fb0bc155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0bc15c780 0x7fb0bc0b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0bc15cb60 0x7fb0bc0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc15b1a0 con 0x7fb0bc15cb60 2024-03-30T00:51:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0b6c10 con 0x7fb0bc0cd260 2024-03-30T00:51:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0c2200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc153810 con 0x7fb0bc15c780 2024-03-30T00:51:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0bc15cb60 0x7fb0bc0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0bbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0bc15cb60 0x7fb0bc0c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48740/0 (socket says 172.21.15.66:48740) 2024-03-30T00:51:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0bbe00640 1 -- 172.21.15.66:0/1145052034 learned_addr learned my addr 172.21.15.66:0/1145052034 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.947+0000 7fb0bb400640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc0cd260 0x7fb0bc155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0b2a00640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0bc15c780 0x7fb0bc0b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0bbe00640 1 -- 172.21.15.66:0/1145052034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0bc15c780 msgr2=0x7fb0bc0b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0bbe00640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0bc15c780 0x7fb0bc0b66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0bbe00640 1 -- 172.21.15.66:0/1145052034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc0cd260 msgr2=0x7fb0bc155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0bbe00640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc0cd260 0x7fb0bc155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0bbe00640 1 -- 172.21.15.66:0/1145052034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a4009a70 con 0x7fb0bc15cb60 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0bb400640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc0cd260 0x7fb0bc155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0bbe00640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0bc15cb60 0x7fb0bc0c7670 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c450 tx=0x7fb0b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0b3e00640 1 -- 172.21.15.66:0/1145052034 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40074e0 con 0x7fb0bc15cb60 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0b3e00640 1 -- 172.21.15.66:0/1145052034 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b4007660 con 0x7fb0bc15cb60 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0b3e00640 1 -- 172.21.15.66:0/1145052034 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4004a30 con 0x7fb0bc15cb60 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0c2200640 1 -- 172.21.15.66:0/1145052034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc153af0 con 0x7fb0bc15cb60 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0c2200640 1 -- 172.21.15.66:0/1145052034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc1540b0 con 0x7fb0bc15cb60 2024-03-30T00:51:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0c2200640 1 -- 172.21.15.66:0/1145052034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc15b2e0 con 0x7fb0bc15cb60 2024-03-30T00:51:52.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0b3e00640 1 -- 172.21.15.66:0/1145052034 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb0b4004b90 con 0x7fb0bc15cb60 2024-03-30T00:51:52.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0b3e00640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb098066690 0x7fb098068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:52.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0b3e00640 1 -- 172.21.15.66:0/1145052034 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fb0b4083e50 con 0x7fb0bc15cb60 2024-03-30T00:51:52.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.951+0000 7fb0bb400640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb098066690 0x7fb098068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:52.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.955+0000 7fb0bb400640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb098066690 0x7fb098068b50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc160010 tx=0x7fb0ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:52.955+0000 7fb0b3e00640 1 -- 172.21.15.66:0/1145052034 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb0bc15b2e0 con 0x7fb0bc15cb60 2024-03-30T00:51:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 -- 172.21.15.66:0/1145052034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0bc15b2e0 con 0x7fb0bc15cb60 2024-03-30T00:51:53.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0b3e00640 1 -- 172.21.15.66:0/1145052034 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+12993 (secure 0 0 0) 0x7fb0bc15b2e0 con 0x7fb0bc15cb60 2024-03-30T00:51:53.131 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:53.131 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":130,"fsid":"0aec3283-d125-4c04-9eb7-d2c2f49c35a9","created":"2024-03-30T00:39:25.680933+0000","modified":"2024-03-30T00:51:49.849278+0000","last_up_change":"2024-03-30T00:39:37.600631+0000","last_in_change":"2024-03-30T00:39:29.050525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:39:38.912947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7c5db6f7-404b-4949-93f0-9378144404fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":27456},{"type":"v1","addr":"172.21.15.66:6801","nonce":27456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":27456},{"type":"v1","addr":"172.21.15.66:6803","nonce":27456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":27456},{"type":"v1","addr":"172.21.15.66:6807","nonce":27456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":27456},{"type":"v1","addr":"172.21.15.66:6805","nonce":27456}]},"public_addr":"172.21.15.66:6801/27456","cluster_addr":"172.21.15.66:6803/27456","heartbeat_back_addr":"172.21.15.66:6807/27456","heartbeat_front_addr":"172.21.15.66:6805/27456","state":["exists","up"]},{"osd":1,"uuid":"f6b429b0-c16f-44fb-b35d-ee6774a74577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6821","nonce":27458},{"type":"v1","addr":"172.21.15.66:6823","nonce":27458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6825","nonce":27458},{"type":"v1","addr":"172.21.15.66:6827","nonce":27458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":27458},{"type":"v1","addr":"172.21.15.66:6831","nonce":27458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":27458},{"type":"v1","addr":"172.21.15.66:6829","nonce":27458}]},"public_addr":"172.21.15.66:6823/27458","cluster_addr":"172.21.15.66:6827/27458","heartbeat_back_addr":"172.21.15.66:6831/27458","heartbeat_front_addr":"172.21.15.66:6829/27458","state":["exists","up"]},{"osd":2,"uuid":"17dced93-3dd9-42c1-9cc8-2ce56c7845fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":27464},{"type":"v1","addr":"172.21.15.66:6817","nonce":27464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":27464},{"type":"v1","addr":"172.21.15.66:6819","nonce":27464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":27464},{"type":"v1","addr":"172.21.15.66:6826","nonce":27464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":27464},{"type":"v1","addr":"172.21.15.66:6822","nonce":27464}]},"public_addr":"172.21.15.66:6817/27464","cluster_addr":"172.21.15.66:6819/27464","heartbeat_back_addr":"172.21.15.66:6826/27464","heartbeat_front_addr":"172.21.15.66:6822/27464","state":["exists","up"]},{"osd":3,"uuid":"38a5cdb5-b81c-42d3-80d3-239be5d4cccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":27471},{"type":"v1","addr":"172.21.15.66:6809","nonce":27471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":27471},{"type":"v1","addr":"172.21.15.66:6811","nonce":27471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":27471},{"type":"v1","addr":"172.21.15.66:6815","nonce":27471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":27471},{"type":"v1","addr":"172.21.15.66:6813","nonce":27471}]},"public_addr":"172.21.15.66:6809/27471","cluster_addr":"172.21.15.66:6811/27471","heartbeat_back_addr":"172.21.15.66:6815/27471","heartbeat_front_addr":"172.21.15.66:6813/27471","state":["exists","up"]},{"osd":4,"uuid":"56195e4d-4f62-45b8-9cca-9ab4eccec014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":27641},{"type":"v1","addr":"172.21.15.156:6801","nonce":27641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":27641},{"type":"v1","addr":"172.21.15.156:6803","nonce":27641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":27641},{"type":"v1","addr":"172.21.15.156:6807","nonce":27641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":27641},{"type":"v1","addr":"172.21.15.156:6805","nonce":27641}]},"public_addr":"172.21.15.156:6801/27641","cluster_addr":"172.21.15.156:6803/27641","heartbeat_back_addr":"172.21.15.156:6807/27641","heartbeat_front_addr":"172.21.15.156:6805/27641","state":["exists","up"]},{"osd":5,"uuid":"e2b93e58-a16b-409a-9980-3197387fe368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":27644},{"type":"v1","addr":"172.21.15.156:6809","nonce":27644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":27644},{"type":"v1","addr":"172.21.15.156:6811","nonce":27644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":27644},{"type":"v1","addr":"172.21.15.156:6815","nonce":27644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":27644},{"type":"v1","addr":"172.21.15.156:6813","nonce":27644}]},"public_addr":"172.21.15.156:6809/27644","cluster_addr":"172.21.15.156:6811/27644","heartbeat_back_addr":"172.21.15.156:6815/27644","heartbeat_front_addr":"172.21.15.156:6813/27644","state":["exists","up"]},{"osd":6,"uuid":"b930ed94-76bd-432e-8316-5114a343f8ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":27642},{"type":"v1","addr":"172.21.15.156:6817","nonce":27642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":27642},{"type":"v1","addr":"172.21.15.156:6819","nonce":27642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":27642},{"type":"v1","addr":"172.21.15.156:6823","nonce":27642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":27642},{"type":"v1","addr":"172.21.15.156:6821","nonce":27642}]},"public_addr":"172.21.15.156:6817/27642","cluster_addr":"172.21.15.156:6819/27642","heartbeat_back_addr":"172.21.15.156:6823/27642","heartbeat_front_addr":"172.21.15.156:6821/27642","state":["exists","up"]},{"osd":7,"uuid":"ba829def-38d8-4dd7-9d66-91dddaa8395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":27643},{"type":"v1","addr":"172.21.15.156:6825","nonce":27643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":27643},{"type":"v1","addr":"172.21.15.156:6827","nonce":27643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":27643},{"type":"v1","addr":"172.21.15.156:6831","nonce":27643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":27643},{"type":"v1","addr":"172.21.15.156:6829","nonce":27643}]},"public_addr":"172.21.15.156:6825/27643","cluster_addr":"172.21.15.156:6827/27643","heartbeat_back_addr":"172.21.15.156:6831/27643","heartbeat_front_addr":"172.21.15.156:6829/27643","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.508446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.681638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.760474+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.753237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.586453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.643994+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.755708+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:39:34.724481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6834/2774791553":"2024-03-31T00:51:46.824405+0000","172.21.15.66:6834/3460522956":"2024-03-31T00:51:29.804880+0000","172.21.15.66:6836/3460522956":"2024-03-31T00:51:29.804880+0000","172.21.15.66:6836/2683418851":"2024-03-31T00:51:15.162460+0000","172.21.15.66:6836/2774791553":"2024-03-31T00:51:46.824405+0000","172.21.15.66:6834/2683418851":"2024-03-31T00:51:15.162460+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:51:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 -- 172.21.15.66:0/1145052034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb098066690 msgr2=0x7fb098068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb098066690 0x7fb098068b50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc160010 tx=0x7fb0ac009040 comp rx=0 tx=0).stop 2024-03-30T00:51:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 -- 172.21.15.66:0/1145052034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0bc15cb60 msgr2=0x7fb0bc0c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0bc15cb60 0x7fb0bc0c7670 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c450 tx=0x7fb0b400c910 comp rx=0 tx=0).stop 2024-03-30T00:51:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 -- 172.21.15.66:0/1145052034 shutdown_connections 2024-03-30T00:51:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fb098066690 0x7fb098068b50 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0bc15cb60 0x7fb0bc0c7670 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0bc15c780 0x7fb0bc0b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:53.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 --2- 172.21.15.66:0/1145052034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc0cd260 0x7fb0bc155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:53.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 -- 172.21.15.66:0/1145052034 >> 172.21.15.66:0/1145052034 conn(0x7fb0bc09ba30 msgr2=0x7fb0bc0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:53.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 -- 172.21.15.66:0/1145052034 shutdown_connections 2024-03-30T00:51:53.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:53.123+0000 7fb0c2200640 1 -- 172.21.15.66:0/1145052034 wait complete. 2024-03-30T00:51:54.153 INFO:tasks.ceph:Scrubbing osd.0 2024-03-30T00:51:54.153 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-30T00:51:54.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.299+0000 7f65db200640 1 Processor -- start 2024-03-30T00:51:54.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.299+0000 7f65db200640 1 -- start start 2024-03-30T00:51:54.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65db200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d415cab0 0x7f65d406ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65db200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f65d40ca220 0x7f65d40ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65db200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d406b740 con 0x7f65d406b2e0 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65db200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d406d930 con 0x7f65d40ca220 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65db200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d415ce90 con 0x7f65d415cab0 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65d9200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f65d40ca220 0x7f65d40ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65d9200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f65d40ca220 0x7f65d40ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48756/0 (socket says 172.21.15.66:48756) 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65d9200640 1 -- 172.21.15.66:0/3957041929 learned_addr learned my addr 172.21.15.66:0/3957041929 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65d3e00640 1 --2- 172.21.15.66:0/3957041929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d415cab0 0x7f65d406ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65d9200640 1 -- 172.21.15.66:0/3957041929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d415cab0 msgr2=0x7f65d406ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65d9200640 1 --2- 172.21.15.66:0/3957041929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d415cab0 0x7f65d406ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65d9200640 1 -- 172.21.15.66:0/3957041929 >> v1:172.21.15.156:6789/0 conn(0x7f65d406b2e0 legacy=0x7f65d406ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.303+0000 7f65d9200640 1 -- 172.21.15.66:0/3957041929 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d40caf10 con 0x7f65d40ca220 2024-03-30T00:51:54.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65d9200640 1 --2- 172.21.15.66:0/3957041929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d40ca220 0x7f65d40ca690 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009e50 tx=0x7f65cc01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7552c4dd44e25ea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:54.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65d3400640 1 -- 172.21.15.66:0/3957041929 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0a58d0 con 0x7f65d40ca220 2024-03-30T00:51:54.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65d3400640 1 -- 172.21.15.66:0/3957041929 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc0a5a30 con 0x7f65d40ca220 2024-03-30T00:51:54.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65d3400640 1 -- 172.21.15.66:0/3957041929 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0a5dc0 con 0x7f65d40ca220 2024-03-30T00:51:54.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65db200640 1 -- 172.21.15.66:0/3957041929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d40ca220 msgr2=0x7f65d40ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65db200640 1 --2- 172.21.15.66:0/3957041929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d40ca220 0x7f65d40ca690 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009e50 tx=0x7f65cc01ca00 comp rx=0 tx=0).stop 2024-03-30T00:51:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65db200640 1 -- 172.21.15.66:0/3957041929 shutdown_connections 2024-03-30T00:51:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65db200640 1 --2- 172.21.15.66:0/3957041929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d40ca220 0x7f65d40ca690 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65db200640 1 --2- 172.21.15.66:0/3957041929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d415cab0 0x7f65d406ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65db200640 1 -- 172.21.15.66:0/3957041929 >> 172.21.15.66:0/3957041929 conn(0x7f65d40708d0 msgr2=0x7f65d4070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65db200640 1 -- 172.21.15.66:0/3957041929 shutdown_connections 2024-03-30T00:51:54.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65db200640 1 -- 172.21.15.66:0/3957041929 wait complete. 2024-03-30T00:51:54.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65db200640 1 Processor -- start 2024-03-30T00:51:54.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.307+0000 7f65db200640 1 -- start start 2024-03-30T00:51:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65db200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d406b2e0 0x7f65d40b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65db200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d40ca220 0x7f65d415e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65db200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65d415cab0 0x7f65d40af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65db200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40b4b60 con 0x7f65d406b2e0 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65db200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d415d9d0 con 0x7f65d415cab0 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65db200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d415d0a0 con 0x7f65d40ca220 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d406b2e0 0x7f65d40b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d406b2e0 0x7f65d40b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48768/0 (socket says 172.21.15.66:48768) 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d3e00640 1 -- 172.21.15.66:0/672538768 learned_addr learned my addr 172.21.15.66:0/672538768 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d9200640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65d415cab0 0x7f65d40af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65cb400640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d40ca220 0x7f65d415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d3e00640 1 -- 172.21.15.66:0/672538768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d40ca220 msgr2=0x7f65d415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d3e00640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65d40ca220 0x7f65d415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d3e00640 1 -- 172.21.15.66:0/672538768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65d415cab0 msgr2=0x7f65d40af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d3e00640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65d415cab0 0x7f65d40af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d3e00640 1 -- 172.21.15.66:0/672538768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc009b30 con 0x7f65d406b2e0 2024-03-30T00:51:54.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d3e00640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d406b2e0 0x7f65d40b4620 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f65c400d800 tx=0x7f65c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:54.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d0c00640 1 -- 172.21.15.66:0/672538768 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c400b6c0 con 0x7f65d406b2e0 2024-03-30T00:51:54.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d0c00640 1 -- 172.21.15.66:0/672538768 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c400b860 con 0x7f65d406b2e0 2024-03-30T00:51:54.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d0c00640 1 -- 172.21.15.66:0/672538768 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c400bc10 con 0x7f65d406b2e0 2024-03-30T00:51:54.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65db200640 1 -- 172.21.15.66:0/672538768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d415d380 con 0x7f65d406b2e0 2024-03-30T00:51:54.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65db200640 1 -- 172.21.15.66:0/672538768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d40c5520 con 0x7f65d406b2e0 2024-03-30T00:51:54.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65db200640 1 -- 172.21.15.66:0/672538768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65d40cabd0 con 0x7f65d406b2e0 2024-03-30T00:51:54.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.311+0000 7f65d0c00640 1 -- 172.21.15.66:0/672538768 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f65c401e460 con 0x7f65d406b2e0 2024-03-30T00:51:54.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.315+0000 7f65d0c00640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65b0066690 0x7f65b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.315+0000 7f65d0c00640 1 -- 172.21.15.66:0/672538768 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f65c4084ff0 con 0x7f65d406b2e0 2024-03-30T00:51:54.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.315+0000 7f65d0c00640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x7f65b006cad0 0x7f65b006ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.315+0000 7f65cb400640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65b0066690 0x7f65b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.315+0000 7f65d0c00640 1 -- 172.21.15.66:0/672538768 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65b006f5e0 con 0x7f65b006cad0 2024-03-30T00:51:54.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.315+0000 7f65d0c00640 1 -- 172.21.15.66:0/672538768 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65c40853a0 con 0x7f65d406b2e0 2024-03-30T00:51:54.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.315+0000 7f65d9200640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x7f65b006cad0 0x7f65b006ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.315+0000 7f65cb400640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65b0066690 0x7f65b0068b50 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f65d40b0310 tx=0x7f65bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:54.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.315+0000 7f65d9200640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x7f65b006cad0 0x7f65b006ef10 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:54.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.315+0000 7f65d0c00640 1 -- 172.21.15.66:0/672538768 <== osd.0 v2:172.21.15.66:6800/27456 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f65b006f5e0 con 0x7f65b006cad0 2024-03-30T00:51:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.327+0000 7f65db200640 1 -- 172.21.15.66:0/672538768 --> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f65d415d830 con 0x7f65b006cad0 2024-03-30T00:51:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.327+0000 7f65d0c00640 1 -- 172.21.15.66:0/672538768 <== osd.0 v2:172.21.15.66:6800/27456 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f65cc009150 con 0x7f65b006cad0 2024-03-30T00:51:54.337 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:54.337 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T00:51:54.337 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.327+0000 7f65db200640 1 -- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x7f65b006cad0 msgr2=0x7f65b006ef10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.327+0000 7f65db200640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:6800/27456,v1:172.21.15.66:6801/27456] conn(0x7f65b006cad0 0x7f65b006ef10 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.327+0000 7f65db200640 1 -- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65b0066690 msgr2=0x7f65b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.327+0000 7f65db200640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f65b0066690 0x7f65b0068b50 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f65d40b0310 tx=0x7f65bc009040 comp rx=0 tx=0).stop 2024-03-30T00:51:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.327+0000 7f65db200640 1 -- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d406b2e0 msgr2=0x7f65d40b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.327+0000 7f65db200640 1 --2- 172.21.15.66:0/672538768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65d406b2e0 0x7f65d40b4620 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f65c400d800 tx=0x7f65c400dcc0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.327+0000 7f65d9200640 1 -- 172.21.15.66:0/672538768 reap_dead start 2024-03-30T00:51:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.327+0000 7f65db200640 1 -- 172.21.15.66:0/672538768 shutdown_connections 2024-03-30T00:51:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.327+0000 7f65db200640 1 -- 172.21.15.66:0/672538768 >> 172.21.15.66:0/672538768 conn(0x7f65d40708d0 msgr2=0x7f65d406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.331+0000 7f65db200640 1 -- 172.21.15.66:0/672538768 shutdown_connections 2024-03-30T00:51:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.331+0000 7f65db200640 1 -- 172.21.15.66:0/672538768 wait complete. 2024-03-30T00:51:54.355 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-30T00:51:54.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.487+0000 7f0f83c00640 1 Processor -- start 2024-03-30T00:51:54.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f83c00640 1 -- start start 2024-03-30T00:51:54.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f83c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 0x7f0f7c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f83c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f7c0cde70 0x7f0f7c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f83c00640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7c15a6e0 con 0x7f0f7c0cda90 2024-03-30T00:51:54.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f83c00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7c0b1b80 con 0x7f0f7c0cde70 2024-03-30T00:51:54.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f83c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0cd3e0 con 0x7f0f7c0cec30 2024-03-30T00:51:54.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f7be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 0x7f0f7c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f81c00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f7c0cde70 0x7f0f7c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f81200640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f0f7c0cda90 0x7f0f7c0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.66:55820/0 (socket says 172.21.15.66:55820) 2024-03-30T00:51:54.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f7be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 0x7f0f7c159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:58300/0 (socket says 172.21.15.66:58300) 2024-03-30T00:51:54.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f81200640 1 -- 172.21.15.66:0/1126240228 learned_addr learned my addr 172.21.15.66:0/1126240228 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:54.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f7b400640 1 -- 172.21.15.66:0/1126240228 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977222509 0 0) 0x7f0f7c15a6e0 con 0x7f0f7c0cda90 2024-03-30T00:51:54.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f7be00640 1 -- 172.21.15.66:0/1126240228 >> v2:172.21.15.156:3300/0 conn(0x7f0f7c0cde70 msgr2=0x7f0f7c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:54.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f7be00640 1 --2- 172.21.15.66:0/1126240228 >> v2:172.21.15.156:3300/0 conn(0x7f0f7c0cde70 0x7f0f7c15a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f7be00640 1 -- 172.21.15.66:0/1126240228 >> v1:172.21.15.156:6789/0 conn(0x7f0f7c0cda90 legacy=0x7f0f7c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.491+0000 7f0f7be00640 1 -- 172.21.15.66:0/1126240228 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c15a990 con 0x7f0f7c0cec30 2024-03-30T00:51:54.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f7be00640 1 --2- 172.21.15.66:0/1126240228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 0x7f0f7c159c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f680097c0 tx=0x7f0f6801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea16744ba8040293 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:54.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f7b400640 1 -- 172.21.15.66:0/1126240228 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680ab610 con 0x7f0f7c0cec30 2024-03-30T00:51:54.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f7b400640 1 -- 172.21.15.66:0/1126240228 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f680ab770 con 0x7f0f7c0cec30 2024-03-30T00:51:54.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f7b400640 1 -- 172.21.15.66:0/1126240228 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680abb00 con 0x7f0f7c0cec30 2024-03-30T00:51:54.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f83c00640 1 -- 172.21.15.66:0/1126240228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 msgr2=0x7f0f7c159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f83c00640 1 --2- 172.21.15.66:0/1126240228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 0x7f0f7c159c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f680097c0 tx=0x7f0f6801c410 comp rx=0 tx=0).stop 2024-03-30T00:51:54.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f83c00640 1 -- 172.21.15.66:0/1126240228 shutdown_connections 2024-03-30T00:51:54.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f83c00640 1 --2- 172.21.15.66:0/1126240228 >> v2:172.21.15.156:3300/0 conn(0x7f0f7c0cde70 0x7f0f7c15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f83c00640 1 --2- 172.21.15.66:0/1126240228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 0x7f0f7c159c60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f83c00640 1 -- 172.21.15.66:0/1126240228 >> 172.21.15.66:0/1126240228 conn(0x7f0f7c069f60 msgr2=0x7f0f7c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:54.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f83c00640 1 -- 172.21.15.66:0/1126240228 shutdown_connections 2024-03-30T00:51:54.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f83c00640 1 -- 172.21.15.66:0/1126240228 wait complete. 2024-03-30T00:51:54.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f83c00640 1 Processor -- start 2024-03-30T00:51:54.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.495+0000 7f0f83c00640 1 -- start start 2024-03-30T00:51:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f83c00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f7c0cda90 0x7f0f7c157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f83c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7c0cde70 0x7f0f7c157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f83c00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 0x7f0f7c096c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f83c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0c4ab0 con 0x7f0f7c0cde70 2024-03-30T00:51:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f83c00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0c1a80 con 0x7f0f7c0cda90 2024-03-30T00:51:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f83c00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c15b0f0 con 0x7f0f7c0cec30 2024-03-30T00:51:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f7be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7c0cde70 0x7f0f7c157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f7be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7c0cde70 0x7f0f7c157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48776/0 (socket says 172.21.15.66:48776) 2024-03-30T00:51:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f7be00640 1 -- 172.21.15.66:0/75782857 learned_addr learned my addr 172.21.15.66:0/75782857 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:54.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f81200640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f7c0cda90 0x7f0f7c157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f81c00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 0x7f0f7c096c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f7be00640 1 -- 172.21.15.66:0/75782857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 msgr2=0x7f0f7c096c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f7be00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 0x7f0f7c096c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f7be00640 1 -- 172.21.15.66:0/75782857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f7c0cda90 msgr2=0x7f0f7c157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f7be00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f7c0cda90 0x7f0f7c157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f7be00640 1 -- 172.21.15.66:0/75782857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f68009420 con 0x7f0f7c0cde70 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f81c00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 0x7f0f7c096c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f81200640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f7c0cda90 0x7f0f7c157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f7be00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7c0cde70 0x7f0f7c157bb0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f0f6801a040 tx=0x7f0f680191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f78c00640 1 -- 172.21.15.66:0/75782857 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680bf450 con 0x7f0f7c0cde70 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f78c00640 1 -- 172.21.15.66:0/75782857 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f680bf5d0 con 0x7f0f7c0cde70 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f78c00640 1 -- 172.21.15.66:0/75782857 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680bf980 con 0x7f0f7c0cde70 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f83c00640 1 -- 172.21.15.66:0/75782857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7c0c4740 con 0x7f0f7c0cde70 2024-03-30T00:51:54.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f83c00640 1 -- 172.21.15.66:0/75782857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c071a30 con 0x7f0f7c0cde70 2024-03-30T00:51:54.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.499+0000 7f0f83c00640 1 -- 172.21.15.66:0/75782857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c0c4bf0 con 0x7f0f7c0cde70 2024-03-30T00:51:54.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.503+0000 7f0f78c00640 1 -- 172.21.15.66:0/75782857 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0f680b43f0 con 0x7f0f7c0cde70 2024-03-30T00:51:54.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.503+0000 7f0f78c00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f5c0666c0 0x7f0f5c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.503+0000 7f0f78c00640 1 -- 172.21.15.66:0/75782857 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f0f6811eb60 con 0x7f0f7c0cde70 2024-03-30T00:51:54.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.503+0000 7f0f81200640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f5c0666c0 0x7f0f5c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.503+0000 7f0f81200640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f5c0666c0 0x7f0f5c068b80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0f70007fd0 tx=0x7f0f70007c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:54.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.507+0000 7f0f78c00640 1 -- 172.21.15.66:0/75782857 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0f680ebfd0 con 0x7f0f7c0cde70 2024-03-30T00:51:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 -- 172.21.15.66:0/75782857 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f7c0b1fa0 con 0x7f0f5c0666c0 2024-03-30T00:51:54.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f78c00640 1 -- 172.21.15.66:0/75782857 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0f70005c90 con 0x7f0f5c0666c0 2024-03-30T00:51:54.676 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 0 to deep-scrub 2024-03-30T00:51:54.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 -- 172.21.15.66:0/75782857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f5c0666c0 msgr2=0x7f0f5c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f5c0666c0 0x7f0f5c068b80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0f70007fd0 tx=0x7f0f70007c20 comp rx=0 tx=0).stop 2024-03-30T00:51:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 -- 172.21.15.66:0/75782857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7c0cde70 msgr2=0x7f0f7c157bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7c0cde70 0x7f0f7c157bb0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f0f6801a040 tx=0x7f0f680191b0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 -- 172.21.15.66:0/75782857 shutdown_connections 2024-03-30T00:51:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f7c0cec30 0x7f0f7c096c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f0f5c0666c0 0x7f0f5c068b80 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f7c0cde70 0x7f0f7c157bb0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 --2- 172.21.15.66:0/75782857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f7c0cda90 0x7f0f7c157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 -- 172.21.15.66:0/75782857 >> 172.21.15.66:0/75782857 conn(0x7f0f7c069f60 msgr2=0x7f0f7c06d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 -- 172.21.15.66:0/75782857 shutdown_connections 2024-03-30T00:51:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.667+0000 7f0f83c00640 1 -- 172.21.15.66:0/75782857 wait complete. 2024-03-30T00:51:54.696 INFO:tasks.ceph:Scrubbing osd.1 2024-03-30T00:51:54.696 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-30T00:51:54.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.831+0000 7fdf7b200640 1 Processor -- start 2024-03-30T00:51:54.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.831+0000 7fdf7b200640 1 -- start start 2024-03-30T00:51:54.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.831+0000 7fdf7b200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdf7c15c730 0x7fdf7c15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.831+0000 7fdf7b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdf7c06cb20 0x7fdf7c0cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.831+0000 7fdf7b200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7c0d0970 con 0x7fdf7c0d03f0 2024-03-30T00:51:54.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf7b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7c0d0c50 con 0x7fdf7c06cb20 2024-03-30T00:51:54.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf7b200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7c0d0f30 con 0x7fdf7c15c730 2024-03-30T00:51:54.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf73e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdf7c06cb20 0x7fdf7c0cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf73e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdf7c06cb20 0x7fdf7c0cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48782/0 (socket says 172.21.15.66:48782) 2024-03-30T00:51:54.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf73e00640 1 -- 172.21.15.66:0/134231235 learned_addr learned my addr 172.21.15.66:0/134231235 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:54.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf79e00640 1 --2- 172.21.15.66:0/134231235 >> v2:172.21.15.156:3300/0 conn(0x7fdf7c15c730 0x7fdf7c15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf73e00640 1 -- 172.21.15.66:0/134231235 >> v2:172.21.15.156:3300/0 conn(0x7fdf7c15c730 msgr2=0x7fdf7c15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:54.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf73e00640 1 --2- 172.21.15.66:0/134231235 >> v2:172.21.15.156:3300/0 conn(0x7fdf7c15c730 0x7fdf7c15dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf73e00640 1 -- 172.21.15.66:0/134231235 >> v1:172.21.15.156:6789/0 conn(0x7fdf7c0d03f0 legacy=0x7fdf7c0cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:54.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf73e00640 1 -- 172.21.15.66:0/134231235 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c06aea0 con 0x7fdf7c06cb20 2024-03-30T00:51:54.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf79e00640 1 --2- 172.21.15.66:0/134231235 >> v2:172.21.15.156:3300/0 conn(0x7fdf7c15c730 0x7fdf7c15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:54.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf73e00640 1 --2- 172.21.15.66:0/134231235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7c06cb20 0x7fdf7c0cfeb0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c06f0c0 tx=0x7fdf6401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebadbe1608a51a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:54.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf79400640 1 -- 172.21.15.66:0/134231235 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640ab650 con 0x7fdf7c06cb20 2024-03-30T00:51:54.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf79400640 1 -- 172.21.15.66:0/134231235 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf640ab7b0 con 0x7fdf7c06cb20 2024-03-30T00:51:54.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.835+0000 7fdf79400640 1 -- 172.21.15.66:0/134231235 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640abb40 con 0x7fdf7c06cb20 2024-03-30T00:51:54.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 -- 172.21.15.66:0/134231235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7c06cb20 msgr2=0x7fdf7c0cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 --2- 172.21.15.66:0/134231235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7c06cb20 0x7fdf7c0cfeb0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c06f0c0 tx=0x7fdf6401c4b0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 -- 172.21.15.66:0/134231235 shutdown_connections 2024-03-30T00:51:54.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 --2- 172.21.15.66:0/134231235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7c06cb20 0x7fdf7c0cfeb0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 --2- 172.21.15.66:0/134231235 >> v2:172.21.15.156:3300/0 conn(0x7fdf7c15c730 0x7fdf7c15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 -- 172.21.15.66:0/134231235 >> 172.21.15.66:0/134231235 conn(0x7fdf7c0b1e80 msgr2=0x7fdf7c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:54.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 -- 172.21.15.66:0/134231235 shutdown_connections 2024-03-30T00:51:54.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 -- 172.21.15.66:0/134231235 wait complete. 2024-03-30T00:51:54.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 Processor -- start 2024-03-30T00:51:54.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 -- start start 2024-03-30T00:51:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7c06cb20 0x7fdf7c14fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7c0d03f0 0x7fdf7c14e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7c15c730 0x7fdf7c14e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c152430 con 0x7fdf7c15c730 2024-03-30T00:51:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c15a750 con 0x7fdf7c06cb20 2024-03-30T00:51:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7b200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c15d650 con 0x7fdf7c0d03f0 2024-03-30T00:51:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7c15c730 0x7fdf7c14e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf79e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7c06cb20 0x7fdf7c14fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf73e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7c0d03f0 0x7fdf7c14e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf79e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7c06cb20 0x7fdf7c14fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.66:40040/0 (socket says 172.21.15.66:40040) 2024-03-30T00:51:54.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf79e00640 1 -- 172.21.15.66:0/31694723 learned_addr learned my addr 172.21.15.66:0/31694723 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:54.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7a800640 1 -- 172.21.15.66:0/31694723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7c0d03f0 msgr2=0x7fdf7c14e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7a800640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7c0d03f0 0x7fdf7c14e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7a800640 1 -- 172.21.15.66:0/31694723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7c06cb20 msgr2=0x7fdf7c14fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7a800640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf7c06cb20 0x7fdf7c14fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.839+0000 7fdf7a800640 1 -- 172.21.15.66:0/31694723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640094d0 con 0x7fdf7c15c730 2024-03-30T00:51:54.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf7a800640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7c15c730 0x7fdf7c14e990 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400c780 tx=0x7fdf7400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:54.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf72000640 1 -- 172.21.15.66:0/31694723 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7400aa50 con 0x7fdf7c15c730 2024-03-30T00:51:54.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf72000640 1 -- 172.21.15.66:0/31694723 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7400abb0 con 0x7fdf7c15c730 2024-03-30T00:51:54.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf7b200640 1 -- 172.21.15.66:0/31694723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c0ce8a0 con 0x7fdf7c15c730 2024-03-30T00:51:54.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf72000640 1 -- 172.21.15.66:0/31694723 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7400ce30 con 0x7fdf7c15c730 2024-03-30T00:51:54.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf7b200640 1 -- 172.21.15.66:0/31694723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7c159eb0 con 0x7fdf7c15c730 2024-03-30T00:51:54.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf7b200640 1 -- 172.21.15.66:0/31694723 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf7c073cf0 con 0x7fdf7c15c730 2024-03-30T00:51:54.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf72000640 1 -- 172.21.15.66:0/31694723 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdf74012420 con 0x7fdf7c15c730 2024-03-30T00:51:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf72000640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf58066690 0x7fdf58068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf72000640 1 -- 172.21.15.66:0/31694723 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fdf740845d0 con 0x7fdf7c15c730 2024-03-30T00:51:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf79e00640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf58066690 0x7fdf58068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf72000640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x7fdf5806cad0 0x7fdf5806ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf72000640 1 -- 172.21.15.66:0/31694723 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf5806f5e0 con 0x7fdf5806cad0 2024-03-30T00:51:54.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf72000640 1 -- 172.21.15.66:0/31694723 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf74084940 con 0x7fdf7c15c730 2024-03-30T00:51:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.843+0000 7fdf73e00640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x7fdf5806cad0 0x7fdf5806ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.847+0000 7fdf79e00640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf58066690 0x7fdf58068b50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c006fd0 tx=0x7fdf6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:54.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.847+0000 7fdf73e00640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x7fdf5806cad0 0x7fdf5806ef10 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.847+0000 7fdf72000640 1 -- 172.21.15.66:0/31694723 <== osd.1 v2:172.21.15.66:6821/27458 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fdf5806f5e0 con 0x7fdf5806cad0 2024-03-30T00:51:54.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7b200640 1 -- 172.21.15.66:0/31694723 --> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdf7c1672d0 con 0x7fdf5806cad0 2024-03-30T00:51:54.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf72000640 1 -- 172.21.15.66:0/31694723 <== osd.1 v2:172.21.15.66:6821/27458 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdf7c1672d0 con 0x7fdf5806cad0 2024-03-30T00:51:54.868 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:54.868 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T00:51:54.868 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:54.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7b200640 1 -- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x7fdf5806cad0 msgr2=0x7fdf5806ef10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7b200640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:6821/27458,v1:172.21.15.66:6823/27458] conn(0x7fdf5806cad0 0x7fdf5806ef10 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:54.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7b200640 1 -- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf58066690 msgr2=0x7fdf58068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7b200640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdf58066690 0x7fdf58068b50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c006fd0 tx=0x7fdf6c008040 comp rx=0 tx=0).stop 2024-03-30T00:51:54.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7b200640 1 -- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7c15c730 msgr2=0x7fdf7c14e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:54.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7b200640 1 --2- 172.21.15.66:0/31694723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf7c15c730 0x7fdf7c14e990 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400c780 tx=0x7fdf7400cc40 comp rx=0 tx=0).stop 2024-03-30T00:51:54.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7a800640 1 -- 172.21.15.66:0/31694723 reap_dead start 2024-03-30T00:51:54.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7b200640 1 -- 172.21.15.66:0/31694723 shutdown_connections 2024-03-30T00:51:54.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7b200640 1 -- 172.21.15.66:0/31694723 >> 172.21.15.66:0/31694723 conn(0x7fdf7c0b1e80 msgr2=0x7fdf7c15da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:54.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7b200640 1 -- 172.21.15.66:0/31694723 shutdown_connections 2024-03-30T00:51:54.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:54.859+0000 7fdf7b200640 1 -- 172.21.15.66:0/31694723 wait complete. 2024-03-30T00:51:54.886 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-30T00:51:55.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e1d000640 1 Processor -- start 2024-03-30T00:51:55.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e1d000640 1 -- start start 2024-03-30T00:51:55.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e1d000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e180cc560 0x7f2e180d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e1d000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e1806e800 0x7f2e180ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e1d000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1806b0b0 con 0x7f2e1806f8e0 2024-03-30T00:51:55.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e1d000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e18160640 con 0x7f2e1806e800 2024-03-30T00:51:55.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e1d000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1806b390 con 0x7f2e180cc560 2024-03-30T00:51:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e16800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e1806e800 0x7f2e180ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e16800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e1806e800 0x7f2e180ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48804/0 (socket says 172.21.15.66:48804) 2024-03-30T00:51:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e16800640 1 -- 172.21.15.66:0/1432022916 learned_addr learned my addr 172.21.15.66:0/1432022916 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e15e00640 1 --2- 172.21.15.66:0/1432022916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e180cc560 0x7f2e180d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e16800640 1 -- 172.21.15.66:0/1432022916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e180cc560 msgr2=0x7f2e180d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e16800640 1 --2- 172.21.15.66:0/1432022916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e180cc560 0x7f2e180d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e16800640 1 -- 172.21.15.66:0/1432022916 >> v1:172.21.15.156:6789/0 conn(0x7f2e1806f8e0 legacy=0x7f2e1806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.019+0000 7f2e16800640 1 -- 172.21.15.66:0/1432022916 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1806b670 con 0x7f2e1806e800 2024-03-30T00:51:55.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e16800640 1 --2- 172.21.15.66:0/1432022916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e1806e800 0x7f2e180ce8d0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009870 tx=0x7f2e1001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf561d6a7ff56573 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e15400640 1 -- 172.21.15.66:0/1432022916 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e100ab640 con 0x7f2e1806e800 2024-03-30T00:51:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e15400640 1 -- 172.21.15.66:0/1432022916 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e100a7070 con 0x7f2e1806e800 2024-03-30T00:51:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e15400640 1 -- 172.21.15.66:0/1432022916 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e100ab9c0 con 0x7f2e1806e800 2024-03-30T00:51:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e1d000640 1 -- 172.21.15.66:0/1432022916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e1806e800 msgr2=0x7f2e180ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e1d000640 1 --2- 172.21.15.66:0/1432022916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e1806e800 0x7f2e180ce8d0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009870 tx=0x7f2e1001c530 comp rx=0 tx=0).stop 2024-03-30T00:51:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e1d000640 1 -- 172.21.15.66:0/1432022916 shutdown_connections 2024-03-30T00:51:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e1d000640 1 --2- 172.21.15.66:0/1432022916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e1806e800 0x7f2e180ce8d0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e1d000640 1 --2- 172.21.15.66:0/1432022916 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e180cc560 0x7f2e180d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e1d000640 1 -- 172.21.15.66:0/1432022916 >> 172.21.15.66:0/1432022916 conn(0x7f2e180d0540 msgr2=0x7f2e180d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e1d000640 1 -- 172.21.15.66:0/1432022916 shutdown_connections 2024-03-30T00:51:55.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.023+0000 7f2e1d000640 1 -- 172.21.15.66:0/1432022916 wait complete. 2024-03-30T00:51:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e1d000640 1 Processor -- start 2024-03-30T00:51:55.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e1d000640 1 -- start start 2024-03-30T00:51:55.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e1d000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e1806e800 0x7f2e1815c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e1d000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1806f8e0 0x7f2e181542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e1d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e180cc560 0x7f2e18154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e1d000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180b10c0 con 0x7f2e180cc560 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e1d000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1815fde0 con 0x7f2e1806f8e0 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e1d000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1815f820 con 0x7f2e1806e800 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e16800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e180cc560 0x7f2e18154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e16800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e180cc560 0x7f2e18154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48820/0 (socket says 172.21.15.66:48820) 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e16800640 1 -- 172.21.15.66:0/2842496038 learned_addr learned my addr 172.21.15.66:0/2842496038 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e15e00640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e1806e800 0x7f2e1815c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e0d400640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1806f8e0 0x7f2e181542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e16800640 1 -- 172.21.15.66:0/2842496038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e1806e800 msgr2=0x7f2e1815c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e16800640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e1806e800 0x7f2e1815c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e16800640 1 -- 172.21.15.66:0/2842496038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1806f8e0 msgr2=0x7f2e181542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e16800640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1806f8e0 0x7f2e181542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e16800640 1 -- 172.21.15.66:0/2842496038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e100094d0 con 0x7f2e180cc560 2024-03-30T00:51:55.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e16800640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e180cc560 0x7f2e18154800 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f2e100ad040 tx=0x7f2e1001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e0ea00640 1 -- 172.21.15.66:0/2842496038 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e100abe20 con 0x7f2e180cc560 2024-03-30T00:51:55.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e0ea00640 1 -- 172.21.15.66:0/2842496038 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e100be3f0 con 0x7f2e180cc560 2024-03-30T00:51:55.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e0ea00640 1 -- 172.21.15.66:0/2842496038 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e100be7a0 con 0x7f2e180cc560 2024-03-30T00:51:55.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e1d000640 1 -- 172.21.15.66:0/2842496038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1815fb00 con 0x7f2e180cc560 2024-03-30T00:51:55.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.027+0000 7f2e1d000640 1 -- 172.21.15.66:0/2842496038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e180b0d50 con 0x7f2e180cc560 2024-03-30T00:51:55.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.031+0000 7f2e1d000640 1 -- 172.21.15.66:0/2842496038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e180b1270 con 0x7f2e180cc560 2024-03-30T00:51:55.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.031+0000 7f2e0ea00640 1 -- 172.21.15.66:0/2842496038 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2e100b43f0 con 0x7f2e180cc560 2024-03-30T00:51:55.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.031+0000 7f2e0ea00640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2df4066690 0x7f2df4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.031+0000 7f2e0ea00640 1 -- 172.21.15.66:0/2842496038 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f2e1011e470 con 0x7f2e180cc560 2024-03-30T00:51:55.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.031+0000 7f2e15e00640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2df4066690 0x7f2df4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.031+0000 7f2e15e00640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2df4066690 0x7f2df4068b50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2e08011340 tx=0x7f2e0800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.035+0000 7f2e0ea00640 1 -- 172.21.15.66:0/2842496038 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2e180b1270 con 0x7f2e180cc560 2024-03-30T00:51:55.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.195+0000 7f2e1d000640 1 -- 172.21.15.66:0/2842496038 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e180cc940 con 0x7f2df4066690 2024-03-30T00:51:55.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.195+0000 7f2e0ea00640 1 -- 172.21.15.66:0/2842496038 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2e0800f040 con 0x7f2df4066690 2024-03-30T00:51:55.205 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 1 to deep-scrub 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 -- 172.21.15.66:0/2842496038 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2df4066690 msgr2=0x7f2df4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2df4066690 0x7f2df4068b50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2e08011340 tx=0x7f2e0800b040 comp rx=0 tx=0).stop 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 -- 172.21.15.66:0/2842496038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e180cc560 msgr2=0x7f2e18154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e180cc560 0x7f2e18154800 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f2e100ad040 tx=0x7f2e1001ad70 comp rx=0 tx=0).stop 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 -- 172.21.15.66:0/2842496038 shutdown_connections 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f2df4066690 0x7f2df4068b50 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e180cc560 0x7f2e18154800 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e1806f8e0 0x7f2e181542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 --2- 172.21.15.66:0/2842496038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2e1806e800 0x7f2e1815c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 -- 172.21.15.66:0/2842496038 >> 172.21.15.66:0/2842496038 conn(0x7f2e180d0540 msgr2=0x7f2e180af0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 -- 172.21.15.66:0/2842496038 shutdown_connections 2024-03-30T00:51:55.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.199+0000 7f2e1d000640 1 -- 172.21.15.66:0/2842496038 wait complete. 2024-03-30T00:51:55.227 INFO:tasks.ceph:Scrubbing osd.2 2024-03-30T00:51:55.227 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-30T00:51:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f649f400640 1 Processor -- start 2024-03-30T00:51:55.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f649f400640 1 -- start start 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f649f400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f649806e7f0 0x7f649806ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f649f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f64980cbd30 0x7f649806d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f649f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64980cb650 0x7f649806d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f649f400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f649806a860 con 0x7f64980cbd30 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f649f400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64980cd340 con 0x7f649806e7f0 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f649f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64980cbb50 con 0x7f64980cb650 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f6497e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f64980cbd30 0x7f649806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f6497e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f64980cbd30 0x7f649806d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48832/0 (socket says 172.21.15.66:48832) 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f6497e00640 1 -- 172.21.15.66:0/3863891847 learned_addr learned my addr 172.21.15.66:0/3863891847 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f649d400640 1 --2- 172.21.15.66:0/3863891847 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64980cb650 0x7f649806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f649ca00640 1 --2- 172.21.15.66:0/3863891847 >> v2:172.21.15.156:3300/0 conn(0x7f649806e7f0 0x7f649806ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f6497e00640 1 -- 172.21.15.66:0/3863891847 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64980cb650 msgr2=0x7f649806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f6497e00640 1 --2- 172.21.15.66:0/3863891847 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64980cb650 0x7f649806d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f6497e00640 1 -- 172.21.15.66:0/3863891847 >> v2:172.21.15.156:3300/0 conn(0x7f649806e7f0 msgr2=0x7f649806ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f6497e00640 1 --2- 172.21.15.66:0/3863891847 >> v2:172.21.15.156:3300/0 conn(0x7f649806e7f0 0x7f649806ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.363+0000 7f6497e00640 1 -- 172.21.15.66:0/3863891847 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649806f110 con 0x7f64980cbd30 2024-03-30T00:51:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f6497e00640 1 --2- 172.21.15.66:0/3863891847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64980cbd30 0x7f649806d220 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f649806fb50 tx=0x7f648401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78bd436dc06ea7c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f6497400640 1 -- 172.21.15.66:0/3863891847 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840ab640 con 0x7f64980cbd30 2024-03-30T00:51:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f6497400640 1 -- 172.21.15.66:0/3863891847 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64840a7070 con 0x7f64980cbd30 2024-03-30T00:51:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f6497400640 1 -- 172.21.15.66:0/3863891847 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840ab9c0 con 0x7f64980cbd30 2024-03-30T00:51:55.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f649f400640 1 -- 172.21.15.66:0/3863891847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64980cbd30 msgr2=0x7f649806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f649f400640 1 --2- 172.21.15.66:0/3863891847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64980cbd30 0x7f649806d220 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f649806fb50 tx=0x7f648401c420 comp rx=0 tx=0).stop 2024-03-30T00:51:55.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f649f400640 1 -- 172.21.15.66:0/3863891847 shutdown_connections 2024-03-30T00:51:55.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f649f400640 1 --2- 172.21.15.66:0/3863891847 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64980cb650 0x7f649806d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f649f400640 1 --2- 172.21.15.66:0/3863891847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64980cbd30 0x7f649806d220 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f649f400640 1 --2- 172.21.15.66:0/3863891847 >> v2:172.21.15.156:3300/0 conn(0x7f649806e7f0 0x7f649806ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f649f400640 1 -- 172.21.15.66:0/3863891847 >> 172.21.15.66:0/3863891847 conn(0x7f64980b1030 msgr2=0x7f64980b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:55.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f649f400640 1 -- 172.21.15.66:0/3863891847 shutdown_connections 2024-03-30T00:51:55.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.367+0000 7f649f400640 1 -- 172.21.15.66:0/3863891847 wait complete. 2024-03-30T00:51:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649f400640 1 Processor -- start 2024-03-30T00:51:55.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649f400640 1 -- start start 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f649806e7f0 0x7f6498099bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64980cb650 0x7f64980c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64980cbd30 0x7f6498155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64980c5ed0 con 0x7f649806e7f0 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649f400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6498156330 con 0x7f64980cbd30 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649809a590 con 0x7f64980cb650 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f649806e7f0 0x7f6498099bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f649806e7f0 0x7f6498099bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48838/0 (socket says 172.21.15.66:48838) 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649ca00640 1 -- 172.21.15.66:0/4153994571 learned_addr learned my addr 172.21.15.66:0/4153994571 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f6497e00640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64980cb650 0x7f64980c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649d400640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64980cbd30 0x7f6498155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649ca00640 1 -- 172.21.15.66:0/4153994571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64980cb650 msgr2=0x7f64980c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649ca00640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f64980cb650 0x7f64980c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649ca00640 1 -- 172.21.15.66:0/4153994571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64980cbd30 msgr2=0x7f6498155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649ca00640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64980cbd30 0x7f6498155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649ca00640 1 -- 172.21.15.66:0/4153994571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64840094d0 con 0x7f649806e7f0 2024-03-30T00:51:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649ca00640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f649806e7f0 0x7f6498099bf0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f648c00d800 tx=0x7f648c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f6494c00640 1 -- 172.21.15.66:0/4153994571 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c0096a0 con 0x7f649806e7f0 2024-03-30T00:51:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f6494c00640 1 -- 172.21.15.66:0/4153994571 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c009800 con 0x7f649806e7f0 2024-03-30T00:51:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f6494c00640 1 -- 172.21.15.66:0/4153994571 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c009990 con 0x7f649806e7f0 2024-03-30T00:51:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649f400640 1 -- 172.21.15.66:0/4153994571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64980aedc0 con 0x7f649806e7f0 2024-03-30T00:51:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649f400640 1 -- 172.21.15.66:0/4153994571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6498156660 con 0x7f649806e7f0 2024-03-30T00:51:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.371+0000 7f649f400640 1 -- 172.21.15.66:0/4153994571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64980aec40 con 0x7f649806e7f0 2024-03-30T00:51:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.375+0000 7f6494c00640 1 -- 172.21.15.66:0/4153994571 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f648c009af0 con 0x7f649806e7f0 2024-03-30T00:51:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.375+0000 7f6494c00640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6478066690 0x7f6478068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.375+0000 7f6494c00640 1 -- 172.21.15.66:0/4153994571 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f648c0843b0 con 0x7f649806e7f0 2024-03-30T00:51:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.375+0000 7f6497e00640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6478066690 0x7f6478068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.375+0000 7f6494c00640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x7f647806cad0 0x7f647806ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.375+0000 7f6494c00640 1 -- 172.21.15.66:0/4153994571 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f647806f5e0 con 0x7f647806cad0 2024-03-30T00:51:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.375+0000 7f6494c00640 1 -- 172.21.15.66:0/4153994571 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f648c086930 con 0x7f649806e7f0 2024-03-30T00:51:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.375+0000 7f649d400640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x7f647806cad0 0x7f647806ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.375+0000 7f6497e00640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6478066690 0x7f6478068b50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f64840af000 tx=0x7f64840a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.375+0000 7f649d400640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x7f647806cad0 0x7f647806ef10 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.375+0000 7f6494c00640 1 -- 172.21.15.66:0/4153994571 <== osd.2 v2:172.21.15.66:6816/27464 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f647806f5e0 con 0x7f647806cad0 2024-03-30T00:51:55.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.387+0000 7f649f400640 1 -- 172.21.15.66:0/4153994571 --> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f649806e0b0 con 0x7f647806cad0 2024-03-30T00:51:55.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f6494c00640 1 -- 172.21.15.66:0/4153994571 <== osd.2 v2:172.21.15.66:6816/27464 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f64900083a0 con 0x7f647806cad0 2024-03-30T00:51:55.398 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:55.398 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T00:51:55.398 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f649f400640 1 -- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x7f647806cad0 msgr2=0x7f647806ef10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f649f400640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:6816/27464,v1:172.21.15.66:6817/27464] conn(0x7f647806cad0 0x7f647806ef10 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f649f400640 1 -- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6478066690 msgr2=0x7f6478068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f649f400640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f6478066690 0x7f6478068b50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f64840af000 tx=0x7f64840a7000 comp rx=0 tx=0).stop 2024-03-30T00:51:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f649f400640 1 -- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f649806e7f0 msgr2=0x7f6498099bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f649f400640 1 --2- 172.21.15.66:0/4153994571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f649806e7f0 0x7f6498099bf0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f648c00d800 tx=0x7f648c00dcc0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f649d400640 1 -- 172.21.15.66:0/4153994571 reap_dead start 2024-03-30T00:51:55.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f649f400640 1 -- 172.21.15.66:0/4153994571 shutdown_connections 2024-03-30T00:51:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f649f400640 1 -- 172.21.15.66:0/4153994571 >> 172.21.15.66:0/4153994571 conn(0x7f64980b1030 msgr2=0x7f64980aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f649f400640 1 -- 172.21.15.66:0/4153994571 shutdown_connections 2024-03-30T00:51:55.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.391+0000 7f649f400640 1 -- 172.21.15.66:0/4153994571 wait complete. 2024-03-30T00:51:55.417 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-30T00:51:55.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.547+0000 7f841be00640 1 Processor -- start 2024-03-30T00:51:55.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f841be00640 1 -- start start 2024-03-30T00:51:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f841be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f841415a640 0x7f841406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f841be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f84140cabb0 0x7f84140cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f841be00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841406d070 con 0x7f84140ce520 2024-03-30T00:51:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f841be00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84140cbba0 con 0x7f84140cabb0 2024-03-30T00:51:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f841be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841406e2c0 con 0x7f841415a640 2024-03-30T00:51:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f8419e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f84140cabb0 0x7f84140cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f8418a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f84140ce520 0x7f84140ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35318/0 (socket says 172.21.15.66:35318) 2024-03-30T00:51:55.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f8418a00640 1 -- 172.21.15.66:0/624957197 learned_addr learned my addr 172.21.15.66:0/624957197 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:55.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f8419400640 1 --2- 172.21.15.66:0/624957197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f841415a640 0x7f841406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f8419e00640 1 -- 172.21.15.66:0/624957197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f841415a640 msgr2=0x7f841406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:55.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f8419e00640 1 --2- 172.21.15.66:0/624957197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f841415a640 0x7f841406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f8419e00640 1 -- 172.21.15.66:0/624957197 >> v1:172.21.15.66:6789/0 conn(0x7f84140ce520 legacy=0x7f84140ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f8419e00640 1 -- 172.21.15.66:0/624957197 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84140cb8c0 con 0x7f84140cabb0 2024-03-30T00:51:55.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.551+0000 7f8413e00640 1 -- 172.21.15.66:0/624957197 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261477112 0 0) 0x7f841406d070 con 0x7f84140ce520 2024-03-30T00:51:55.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f8419e00640 1 --2- 172.21.15.66:0/624957197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84140cabb0 0x7f84140cb040 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f840c009d90 tx=0x7f840c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fa9cd790b756ba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f8413e00640 1 -- 172.21.15.66:0/624957197 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0a5a00 con 0x7f84140cabb0 2024-03-30T00:51:55.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f8413e00640 1 -- 172.21.15.66:0/624957197 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c0a5b60 con 0x7f84140cabb0 2024-03-30T00:51:55.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f8413e00640 1 -- 172.21.15.66:0/624957197 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0ad440 con 0x7f84140cabb0 2024-03-30T00:51:55.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f841be00640 1 -- 172.21.15.66:0/624957197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84140cabb0 msgr2=0x7f84140cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f841be00640 1 --2- 172.21.15.66:0/624957197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84140cabb0 0x7f84140cb040 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f840c009d90 tx=0x7f840c01c940 comp rx=0 tx=0).stop 2024-03-30T00:51:55.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f841be00640 1 -- 172.21.15.66:0/624957197 shutdown_connections 2024-03-30T00:51:55.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f841be00640 1 --2- 172.21.15.66:0/624957197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84140cabb0 0x7f84140cb040 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f841be00640 1 --2- 172.21.15.66:0/624957197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f841415a640 0x7f841406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f841be00640 1 -- 172.21.15.66:0/624957197 >> 172.21.15.66:0/624957197 conn(0x7f8414069f60 msgr2=0x7f841406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:55.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f841be00640 1 -- 172.21.15.66:0/624957197 shutdown_connections 2024-03-30T00:51:55.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f841be00640 1 -- 172.21.15.66:0/624957197 wait complete. 2024-03-30T00:51:55.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f841be00640 1 Processor -- start 2024-03-30T00:51:55.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.555+0000 7f841be00640 1 -- start start 2024-03-30T00:51:55.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f841be00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cabb0 0x7f84140b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f841be00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84140ce520 0x7f84140cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f841be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f841415a640 0x7f841414ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f841be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841414f320 con 0x7f841415a640 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f841be00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84140cc9d0 con 0x7f84140ce520 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f841be00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841414f550 con 0x7f84140cabb0 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8419e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f841415a640 0x7f841414ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8419e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f841415a640 0x7f841414ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48852/0 (socket says 172.21.15.66:48852) 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8419e00640 1 -- 172.21.15.66:0/3524528836 learned_addr learned my addr 172.21.15.66:0/3524528836 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8419400640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cabb0 0x7f84140b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8418a00640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84140ce520 0x7f84140cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8419e00640 1 -- 172.21.15.66:0/3524528836 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cabb0 msgr2=0x7f84140b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8419e00640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cabb0 0x7f84140b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8419e00640 1 -- 172.21.15.66:0/3524528836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84140ce520 msgr2=0x7f84140cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8419e00640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84140ce520 0x7f84140cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8419e00640 1 -- 172.21.15.66:0/3524528836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c009a70 con 0x7f841415a640 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8419e00640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f841415a640 0x7f841414ede0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f840c01a570 tx=0x7f840c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8411600640 1 -- 172.21.15.66:0/3524528836 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0aee00 con 0x7f841415a640 2024-03-30T00:51:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8411600640 1 -- 172.21.15.66:0/3524528836 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c009450 con 0x7f841415a640 2024-03-30T00:51:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f841be00640 1 -- 172.21.15.66:0/3524528836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84140cc380 con 0x7f841415a640 2024-03-30T00:51:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8411600640 1 -- 172.21.15.66:0/3524528836 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c003ba0 con 0x7f841415a640 2024-03-30T00:51:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f841be00640 1 -- 172.21.15.66:0/3524528836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841414f830 con 0x7f841415a640 2024-03-30T00:51:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f841be00640 1 -- 172.21.15.66:0/3524528836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84140ccb80 con 0x7f841415a640 2024-03-30T00:51:55.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.559+0000 7f8411600640 1 -- 172.21.15.66:0/3524528836 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f840c0b4050 con 0x7f841415a640 2024-03-30T00:51:55.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.563+0000 7f8411600640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f83f4066690 0x7f83f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.563+0000 7f8411600640 1 -- 172.21.15.66:0/3524528836 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f840c11e560 con 0x7f841415a640 2024-03-30T00:51:55.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.563+0000 7f8419400640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f83f4066690 0x7f83f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.563+0000 7f8419400640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f83f4066690 0x7f83f4068b50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8408009ac0 tx=0x7f8408009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.567+0000 7f8411600640 1 -- 172.21.15.66:0/3524528836 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f84140ccb80 con 0x7f841415a640 2024-03-30T00:51:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.731+0000 7f841be00640 1 -- 172.21.15.66:0/3524528836 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f84140cb580 con 0x7f83f4066690 2024-03-30T00:51:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.731+0000 7f8411600640 1 -- 172.21.15.66:0/3524528836 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f84080119e0 con 0x7f83f4066690 2024-03-30T00:51:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 2 to deep-scrub 2024-03-30T00:51:55.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 -- 172.21.15.66:0/3524528836 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f83f4066690 msgr2=0x7f83f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f83f4066690 0x7f83f4068b50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8408009ac0 tx=0x7f8408009190 comp rx=0 tx=0).stop 2024-03-30T00:51:55.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 -- 172.21.15.66:0/3524528836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f841415a640 msgr2=0x7f841414ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f841415a640 0x7f841414ede0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f840c01a570 tx=0x7f840c0196e0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 -- 172.21.15.66:0/3524528836 shutdown_connections 2024-03-30T00:51:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f83f4066690 0x7f83f4068b50 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f841415a640 0x7f841414ede0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84140ce520 0x7f84140cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 --2- 172.21.15.66:0/3524528836 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cabb0 0x7f84140b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 -- 172.21.15.66:0/3524528836 >> 172.21.15.66:0/3524528836 conn(0x7f8414069f60 msgr2=0x7f84140ac430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 -- 172.21.15.66:0/3524528836 shutdown_connections 2024-03-30T00:51:55.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.735+0000 7f841be00640 1 -- 172.21.15.66:0/3524528836 wait complete. 2024-03-30T00:51:55.761 INFO:tasks.ceph:Scrubbing osd.3 2024-03-30T00:51:55.761 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-30T00:51:55.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7fe000640 1 Processor -- start 2024-03-30T00:51:55.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7fe000640 1 -- start start 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7fe000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7f80d13b0 0x7ff7f815c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7fe000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7f815cb50 0x7ff7f80cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7fe000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f80cd330 con 0x7ff7f80d0fd0 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7fe000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f815cfe0 con 0x7ff7f80d13b0 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7fe000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f8160ac0 con 0x7ff7f815cb50 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7eea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7f80d13b0 0x7ff7f815c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7eea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff7f80d13b0 0x7ff7f815c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48864/0 (socket says 172.21.15.66:48864) 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7eea00640 1 -- 172.21.15.66:0/4224696185 learned_addr learned my addr 172.21.15.66:0/4224696185 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7f7e00640 1 --2- 172.21.15.66:0/4224696185 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7f815cb50 0x7ff7f80cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7eea00640 1 -- 172.21.15.66:0/4224696185 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7f815cb50 msgr2=0x7ff7f80cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7eea00640 1 --2- 172.21.15.66:0/4224696185 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7f815cb50 0x7ff7f80cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7eea00640 1 -- 172.21.15.66:0/4224696185 >> v1:172.21.15.156:6789/0 conn(0x7ff7f80d0fd0 legacy=0x7ff7f80d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.899+0000 7ff7eea00640 1 -- 172.21.15.66:0/4224696185 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f8160da0 con 0x7ff7f80d13b0 2024-03-30T00:51:55.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7eea00640 1 --2- 172.21.15.66:0/4224696185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f80d13b0 0x7ff7f815c610 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0006fd0 tx=0x7ff7e001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f0674ddc5e327c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7f6a00640 1 -- 172.21.15.66:0/4224696185 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00a9400 con 0x7ff7f80d13b0 2024-03-30T00:51:55.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7f6a00640 1 -- 172.21.15.66:0/4224696185 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e00a9560 con 0x7ff7f80d13b0 2024-03-30T00:51:55.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7f6a00640 1 -- 172.21.15.66:0/4224696185 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00a98f0 con 0x7ff7f80d13b0 2024-03-30T00:51:55.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7fe000640 1 -- 172.21.15.66:0/4224696185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f80d13b0 msgr2=0x7ff7f815c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7fe000640 1 --2- 172.21.15.66:0/4224696185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f80d13b0 0x7ff7f815c610 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0006fd0 tx=0x7ff7e001c540 comp rx=0 tx=0).stop 2024-03-30T00:51:55.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7fe000640 1 -- 172.21.15.66:0/4224696185 shutdown_connections 2024-03-30T00:51:55.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7fe000640 1 --2- 172.21.15.66:0/4224696185 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7f815cb50 0x7ff7f80cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7fe000640 1 --2- 172.21.15.66:0/4224696185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f80d13b0 0x7ff7f815c610 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7fe000640 1 -- 172.21.15.66:0/4224696185 >> 172.21.15.66:0/4224696185 conn(0x7ff7f809bb30 msgr2=0x7ff7f8089fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:55.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7fe000640 1 -- 172.21.15.66:0/4224696185 shutdown_connections 2024-03-30T00:51:55.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.903+0000 7ff7fe000640 1 -- 172.21.15.66:0/4224696185 wait complete. 2024-03-30T00:51:55.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7fe000640 1 Processor -- start 2024-03-30T00:51:55.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7fe000640 1 -- start start 2024-03-30T00:51:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7fe000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7f80d0fd0 0x7ff7f80cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7fe000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7f80d13b0 0x7ff7f814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7fe000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f815cb50 0x7ff7f814d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7fe000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f80c9990 con 0x7ff7f815cb50 2024-03-30T00:51:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7fe000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f814e2e0 con 0x7ff7f80d13b0 2024-03-30T00:51:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7fe000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f814e5c0 con 0x7ff7f80d0fd0 2024-03-30T00:51:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7f7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f815cb50 0x7ff7f814d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7f7400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7f80d0fd0 0x7ff7f80cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7eea00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7f80d13b0 0x7ff7f814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7f7400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7f80d0fd0 0x7ff7f80cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:58384/0 (socket says 172.21.15.66:58384) 2024-03-30T00:51:55.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7f7400640 1 -- 172.21.15.66:0/1855221402 learned_addr learned my addr 172.21.15.66:0/1855221402 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:55.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7f7e00640 1 -- 172.21.15.66:0/1855221402 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7f80d0fd0 msgr2=0x7ff7f80cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7f7e00640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff7f80d0fd0 0x7ff7f80cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7f7e00640 1 -- 172.21.15.66:0/1855221402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7f80d13b0 msgr2=0x7ff7f814eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7f7e00640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7f80d13b0 0x7ff7f814eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7f7e00640 1 -- 172.21.15.66:0/1855221402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e0009480 con 0x7ff7f815cb50 2024-03-30T00:51:55.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7f7e00640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f815cb50 0x7ff7f814d070 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0007d40 tx=0x7ff7f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7efe00640 1 -- 172.21.15.66:0/1855221402 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0018070 con 0x7ff7f815cb50 2024-03-30T00:51:55.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7efe00640 1 -- 172.21.15.66:0/1855221402 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f000ed70 con 0x7ff7f815cb50 2024-03-30T00:51:55.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7efe00640 1 -- 172.21.15.66:0/1855221402 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0003bc0 con 0x7ff7f815cb50 2024-03-30T00:51:55.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7fe000640 1 -- 172.21.15.66:0/1855221402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f814e8a0 con 0x7ff7f815cb50 2024-03-30T00:51:55.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7fe000640 1 -- 172.21.15.66:0/1855221402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f814f0c0 con 0x7ff7f815cb50 2024-03-30T00:51:55.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.907+0000 7ff7fe000640 1 -- 172.21.15.66:0/1855221402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7f8066c30 con 0x7ff7f815cb50 2024-03-30T00:51:55.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.911+0000 7ff7efe00640 1 -- 172.21.15.66:0/1855221402 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff7f0027430 con 0x7ff7f815cb50 2024-03-30T00:51:55.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.911+0000 7ff7efe00640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff7d4066690 0x7ff7d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.911+0000 7ff7efe00640 1 -- 172.21.15.66:0/1855221402 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7ff7f0084fa0 con 0x7ff7f815cb50 2024-03-30T00:51:55.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.911+0000 7ff7f7400640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff7d4066690 0x7ff7d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.911+0000 7ff7efe00640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x7ff7d406cad0 0x7ff7d406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:55.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.911+0000 7ff7efe00640 1 -- 172.21.15.66:0/1855221402 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7d406f5e0 con 0x7ff7d406cad0 2024-03-30T00:51:55.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.911+0000 7ff7efe00640 1 -- 172.21.15.66:0/1855221402 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7f00853f0 con 0x7ff7f815cb50 2024-03-30T00:51:55.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.911+0000 7ff7eea00640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x7ff7d406cad0 0x7ff7d406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:55.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.911+0000 7ff7f7400640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff7d4066690 0x7ff7d4068b50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8000fc0 tx=0x7ff7e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.911+0000 7ff7eea00640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x7ff7d406cad0 0x7ff7d406ef10 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:55.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.911+0000 7ff7efe00640 1 -- 172.21.15.66:0/1855221402 <== osd.3 v2:172.21.15.66:6808/27471 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ff7d406f5e0 con 0x7ff7d406cad0 2024-03-30T00:51:55.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7fe000640 1 -- 172.21.15.66:0/1855221402 --> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff7f80c9ad0 con 0x7ff7d406cad0 2024-03-30T00:51:55.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7efe00640 1 -- 172.21.15.66:0/1855221402 <== osd.3 v2:172.21.15.66:6808/27471 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff7f80c9ad0 con 0x7ff7d406cad0 2024-03-30T00:51:55.934 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:55.934 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T00:51:55.934 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:55.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7fe000640 1 -- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x7ff7d406cad0 msgr2=0x7ff7d406ef10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7fe000640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:6808/27471,v1:172.21.15.66:6809/27471] conn(0x7ff7d406cad0 0x7ff7d406ef10 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7fe000640 1 -- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff7d4066690 msgr2=0x7ff7d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7fe000640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7ff7d4066690 0x7ff7d4068b50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8000fc0 tx=0x7ff7e8009040 comp rx=0 tx=0).stop 2024-03-30T00:51:55.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7fe000640 1 -- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f815cb50 msgr2=0x7ff7f814d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:55.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7fe000640 1 --2- 172.21.15.66:0/1855221402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f815cb50 0x7ff7f814d070 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0007d40 tx=0x7ff7f000e4b0 comp rx=0 tx=0).stop 2024-03-30T00:51:55.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7f7e00640 1 -- 172.21.15.66:0/1855221402 reap_dead start 2024-03-30T00:51:55.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7fe000640 1 -- 172.21.15.66:0/1855221402 shutdown_connections 2024-03-30T00:51:55.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7fe000640 1 -- 172.21.15.66:0/1855221402 >> 172.21.15.66:0/1855221402 conn(0x7ff7f809bb30 msgr2=0x7ff7f80ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:55.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7fe000640 1 -- 172.21.15.66:0/1855221402 shutdown_connections 2024-03-30T00:51:55.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:55.927+0000 7ff7fe000640 1 -- 172.21.15.66:0/1855221402 wait complete. 2024-03-30T00:51:55.953 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-30T00:51:56.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc081800640 1 Processor -- start 2024-03-30T00:51:56.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc081800640 1 -- start start 2024-03-30T00:51:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc081800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc07c0cca40 0x7fc07c0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc081800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc07c099960 0x7fc07c06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc081800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07c098ac0 con 0x7fc07c06b140 2024-03-30T00:51:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc081800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07c06baf0 con 0x7fc07c0cca40 2024-03-30T00:51:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc081800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c0cf3f0 con 0x7fc07c099960 2024-03-30T00:51:56.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc07a000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc07c06b140 0x7fc07c099250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35332/0 (socket says 172.21.15.66:35332) 2024-03-30T00:51:56.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc07a000640 1 -- 172.21.15.66:0/1406943403 learned_addr learned my addr 172.21.15.66:0/1406943403 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:56.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc07aa00640 1 --2- 172.21.15.66:0/1406943403 >> v2:172.21.15.156:3300/0 conn(0x7fc07c0cca40 0x7fc07c0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc07b400640 1 --2- 172.21.15.66:0/1406943403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc07c099960 0x7fc07c06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc079600640 1 -- 172.21.15.66:0/1406943403 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962025882 0 0) 0x7fc07c098ac0 con 0x7fc07c06b140 2024-03-30T00:51:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc079600640 1 -- 172.21.15.66:0/1406943403 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc060003530 con 0x7fc07c06b140 2024-03-30T00:51:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc07aa00640 1 -- 172.21.15.66:0/1406943403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc07c099960 msgr2=0x7fc07c06b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc07aa00640 1 --2- 172.21.15.66:0/1406943403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc07c099960 0x7fc07c06b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc07aa00640 1 -- 172.21.15.66:0/1406943403 >> v1:172.21.15.66:6789/0 conn(0x7fc07c06b140 legacy=0x7fc07c099250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.087+0000 7fc07aa00640 1 -- 172.21.15.66:0/1406943403 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c099d90 con 0x7fc07c0cca40 2024-03-30T00:51:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc07aa00640 1 --2- 172.21.15.66:0/1406943403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c0cca40 0x7fc07c0cac90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0097c0 tx=0x7fc06c01c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a84f74154f66ab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc079600640 1 -- 172.21.15.66:0/1406943403 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0a7070 con 0x7fc07c0cca40 2024-03-30T00:51:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc079600640 1 -- 172.21.15.66:0/1406943403 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c0ab430 con 0x7fc07c0cca40 2024-03-30T00:51:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc079600640 1 -- 172.21.15.66:0/1406943403 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0ab7a0 con 0x7fc07c0cca40 2024-03-30T00:51:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc081800640 1 -- 172.21.15.66:0/1406943403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c0cca40 msgr2=0x7fc07c0cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc081800640 1 --2- 172.21.15.66:0/1406943403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c0cca40 0x7fc07c0cac90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0097c0 tx=0x7fc06c01c430 comp rx=0 tx=0).stop 2024-03-30T00:51:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc081800640 1 -- 172.21.15.66:0/1406943403 shutdown_connections 2024-03-30T00:51:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc081800640 1 --2- 172.21.15.66:0/1406943403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc07c099960 0x7fc07c06b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc081800640 1 --2- 172.21.15.66:0/1406943403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c0cca40 0x7fc07c0cac90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc081800640 1 -- 172.21.15.66:0/1406943403 >> 172.21.15.66:0/1406943403 conn(0x7fc07c0b2130 msgr2=0x7fc07c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc081800640 1 -- 172.21.15.66:0/1406943403 shutdown_connections 2024-03-30T00:51:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.091+0000 7fc081800640 1 -- 172.21.15.66:0/1406943403 wait complete. 2024-03-30T00:51:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc081800640 1 Processor -- start 2024-03-30T00:51:56.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc081800640 1 -- start start 2024-03-30T00:51:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc081800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c06b140 0x7fc07c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc081800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc07c099960 0x7fc07c0c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc081800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc07c0cca40 0x7fc07c1556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc081800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c151500 con 0x7fc07c0cca40 2024-03-30T00:51:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc081800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c151750 con 0x7fc07c06b140 2024-03-30T00:51:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc081800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c151a30 con 0x7fc07c099960 2024-03-30T00:51:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc07b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc07c0cca40 0x7fc07c1556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc07b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc07c0cca40 0x7fc07c1556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48870/0 (socket says 172.21.15.66:48870) 2024-03-30T00:51:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc07b400640 1 -- 172.21.15.66:0/4179114428 learned_addr learned my addr 172.21.15.66:0/4179114428 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc07b400640 1 -- 172.21.15.66:0/4179114428 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc07c099960 msgr2=0x7fc07c0c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc07aa00640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c06b140 0x7fc07c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc07b400640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc07c099960 0x7fc07c0c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc07b400640 1 -- 172.21.15.66:0/4179114428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c06b140 msgr2=0x7fc07c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc07b400640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c06b140 0x7fc07c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc07b400640 1 -- 172.21.15.66:0/4179114428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c009420 con 0x7fc07c0cca40 2024-03-30T00:51:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc07aa00640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c06b140 0x7fc07c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc07b400640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc07c0cca40 0x7fc07c1556d0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fc07400efc0 tx=0x7fc07400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc072a00640 1 -- 172.21.15.66:0/4179114428 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074014480 con 0x7fc07c0cca40 2024-03-30T00:51:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc072a00640 1 -- 172.21.15.66:0/4179114428 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07400f040 con 0x7fc07c0cca40 2024-03-30T00:51:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc072a00640 1 -- 172.21.15.66:0/4179114428 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074014820 con 0x7fc07c0cca40 2024-03-30T00:51:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc081800640 1 -- 172.21.15.66:0/4179114428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c151d10 con 0x7fc07c0cca40 2024-03-30T00:51:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc081800640 1 -- 172.21.15.66:0/4179114428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c1522d0 con 0x7fc07c0cca40 2024-03-30T00:51:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.095+0000 7fc081800640 1 -- 172.21.15.66:0/4179114428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c0677d0 con 0x7fc07c0cca40 2024-03-30T00:51:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.099+0000 7fc072a00640 1 -- 172.21.15.66:0/4179114428 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc074014980 con 0x7fc07c0cca40 2024-03-30T00:51:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.099+0000 7fc072a00640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc05c066690 0x7fc05c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.099+0000 7fc072a00640 1 -- 172.21.15.66:0/4179114428 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fc074084c60 con 0x7fc07c0cca40 2024-03-30T00:51:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.099+0000 7fc07aa00640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc05c066690 0x7fc05c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.099+0000 7fc07aa00640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc05c066690 0x7fc05c068b50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0af000 tx=0x7fc06c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.103+0000 7fc072a00640 1 -- 172.21.15.66:0/4179114428 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc074052040 con 0x7fc07c0cca40 2024-03-30T00:51:56.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 -- 172.21.15.66:0/4179114428 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc07c0cce20 con 0x7fc05c066690 2024-03-30T00:51:56.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc072a00640 1 -- 172.21.15.66:0/4179114428 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc06c0bb090 con 0x7fc05c066690 2024-03-30T00:51:56.275 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 3 to deep-scrub 2024-03-30T00:51:56.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 -- 172.21.15.66:0/4179114428 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc05c066690 msgr2=0x7fc05c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc05c066690 0x7fc05c068b50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0af000 tx=0x7fc06c008040 comp rx=0 tx=0).stop 2024-03-30T00:51:56.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 -- 172.21.15.66:0/4179114428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc07c0cca40 msgr2=0x7fc07c1556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc07c0cca40 0x7fc07c1556d0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fc07400efc0 tx=0x7fc07400c490 comp rx=0 tx=0).stop 2024-03-30T00:51:56.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 -- 172.21.15.66:0/4179114428 shutdown_connections 2024-03-30T00:51:56.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc05c066690 0x7fc05c068b50 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc07c0cca40 0x7fc07c1556d0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc07c099960 0x7fc07c0c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 --2- 172.21.15.66:0/4179114428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c06b140 0x7fc07c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 -- 172.21.15.66:0/4179114428 >> 172.21.15.66:0/4179114428 conn(0x7fc07c0b2130 msgr2=0x7fc07c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:56.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 -- 172.21.15.66:0/4179114428 shutdown_connections 2024-03-30T00:51:56.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.267+0000 7fc081800640 1 -- 172.21.15.66:0/4179114428 wait complete. 2024-03-30T00:51:56.295 INFO:tasks.ceph:Scrubbing osd.4 2024-03-30T00:51:56.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-30T00:51:56.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.431+0000 7f9d8fc00640 1 Processor -- start 2024-03-30T00:51:56.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.431+0000 7f9d8fc00640 1 -- start start 2024-03-30T00:51:56.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.431+0000 7f9d8fc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d880acaf0 0x7f9d880b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.431+0000 7f9d8fc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9d8815ba00 0x7f9d880ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.431+0000 7f9d8fc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9d880cecf0 0x7f9d881579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.431+0000 7f9d8fc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d880cf180 con 0x7f9d8815ba00 2024-03-30T00:51:56.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.431+0000 7f9d8fc00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d880cbf80 con 0x7f9d880cecf0 2024-03-30T00:51:56.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.431+0000 7f9d8fc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880cc820 con 0x7f9d880acaf0 2024-03-30T00:51:56.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d87e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9d8815ba00 0x7f9d880ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d8d200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d880acaf0 0x7f9d880b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d8dc00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9d880cecf0 0x7f9d881579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d87e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9d8815ba00 0x7f9d880ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48882/0 (socket says 172.21.15.66:48882) 2024-03-30T00:51:56.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d87e00640 1 -- 172.21.15.66:0/1706177737 learned_addr learned my addr 172.21.15.66:0/1706177737 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:56.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d87e00640 1 -- 172.21.15.66:0/1706177737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d880acaf0 msgr2=0x7f9d880b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:56.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d87e00640 1 --2- 172.21.15.66:0/1706177737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d880acaf0 0x7f9d880b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d87e00640 1 -- 172.21.15.66:0/1706177737 >> v2:172.21.15.156:3300/0 conn(0x7f9d880cecf0 msgr2=0x7f9d881579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:56.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d87e00640 1 --2- 172.21.15.66:0/1706177737 >> v2:172.21.15.156:3300/0 conn(0x7f9d880cecf0 0x7f9d881579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d87e00640 1 -- 172.21.15.66:0/1706177737 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d880b2dd0 con 0x7f9d8815ba00 2024-03-30T00:51:56.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d87e00640 1 --2- 172.21.15.66:0/1706177737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815ba00 0x7f9d880ce7b0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f9d880b1630 tx=0x7f9d7401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0074ece7fa87b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.435+0000 7f9d87400640 1 -- 172.21.15.66:0/1706177737 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d740ab620 con 0x7f9d8815ba00 2024-03-30T00:51:56.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d87400640 1 -- 172.21.15.66:0/1706177737 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d740ab780 con 0x7f9d8815ba00 2024-03-30T00:51:56.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d87400640 1 -- 172.21.15.66:0/1706177737 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d740abb10 con 0x7f9d8815ba00 2024-03-30T00:51:56.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 -- 172.21.15.66:0/1706177737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815ba00 msgr2=0x7f9d880ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 --2- 172.21.15.66:0/1706177737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815ba00 0x7f9d880ce7b0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f9d880b1630 tx=0x7f9d7401c460 comp rx=0 tx=0).stop 2024-03-30T00:51:56.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 -- 172.21.15.66:0/1706177737 shutdown_connections 2024-03-30T00:51:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 --2- 172.21.15.66:0/1706177737 >> v2:172.21.15.156:3300/0 conn(0x7f9d880cecf0 0x7f9d881579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 --2- 172.21.15.66:0/1706177737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815ba00 0x7f9d880ce7b0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 --2- 172.21.15.66:0/1706177737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d880acaf0 0x7f9d880b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 -- 172.21.15.66:0/1706177737 >> 172.21.15.66:0/1706177737 conn(0x7f9d8806a190 msgr2=0x7f9d8806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 -- 172.21.15.66:0/1706177737 shutdown_connections 2024-03-30T00:51:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 -- 172.21.15.66:0/1706177737 wait complete. 2024-03-30T00:51:56.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 Processor -- start 2024-03-30T00:51:56.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 -- start start 2024-03-30T00:51:56.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d880acaf0 0x7f9d881649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d880cecf0 0x7f9d88157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815ba00 0x7f9d8815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d88071310 con 0x7f9d8815ba00 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d88164f10 con 0x7f9d880acaf0 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8fc00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880afa80 con 0x7f9d880cecf0 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815ba00 0x7f9d8815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815ba00 0x7f9d8815ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48892/0 (socket says 172.21.15.66:48892) 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8dc00640 1 -- 172.21.15.66:0/2382425141 learned_addr learned my addr 172.21.15.66:0/2382425141 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d8d200640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d880acaf0 0x7f9d881649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.439+0000 7f9d87e00640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d880cecf0 0x7f9d88157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d8dc00640 1 -- 172.21.15.66:0/2382425141 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d880cecf0 msgr2=0x7f9d88157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d8dc00640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d880cecf0 0x7f9d88157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d8dc00640 1 -- 172.21.15.66:0/2382425141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d880acaf0 msgr2=0x7f9d881649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d8dc00640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d880acaf0 0x7f9d881649d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d8dc00640 1 -- 172.21.15.66:0/2382425141 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d740094d0 con 0x7f9d8815ba00 2024-03-30T00:51:56.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d8dc00640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815ba00 0x7f9d8815ce70 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000efc0 tx=0x7f9d8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d84c00640 1 -- 172.21.15.66:0/2382425141 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80009ba0 con 0x7f9d8815ba00 2024-03-30T00:51:56.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d84c00640 1 -- 172.21.15.66:0/2382425141 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8000f040 con 0x7f9d8815ba00 2024-03-30T00:51:56.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d8fc00640 1 -- 172.21.15.66:0/2382425141 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d880703f0 con 0x7f9d8815ba00 2024-03-30T00:51:56.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d84c00640 1 -- 172.21.15.66:0/2382425141 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d800146c0 con 0x7f9d8815ba00 2024-03-30T00:51:56.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d8fc00640 1 -- 172.21.15.66:0/2382425141 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d88070970 con 0x7f9d8815ba00 2024-03-30T00:51:56.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d8fc00640 1 -- 172.21.15.66:0/2382425141 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d880ab220 con 0x7f9d8815ba00 2024-03-30T00:51:56.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d84c00640 1 -- 172.21.15.66:0/2382425141 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9d80014820 con 0x7f9d8815ba00 2024-03-30T00:51:56.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d84c00640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d68066690 0x7f9d68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d84c00640 1 -- 172.21.15.66:0/2382425141 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f9d800856a0 con 0x7f9d8815ba00 2024-03-30T00:51:56.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d8d200640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d68066690 0x7f9d68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d84c00640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x7f9d6806cad0 0x7f9d6806ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d84c00640 1 -- 172.21.15.66:0/2382425141 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d6806f5e0 con 0x7f9d6806cad0 2024-03-30T00:51:56.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.443+0000 7f9d84c00640 1 -- 172.21.15.66:0/2382425141 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d80085a10 con 0x7f9d8815ba00 2024-03-30T00:51:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.447+0000 7f9d8d200640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d68066690 0x7f9d68068b50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009a40 tx=0x7f9d7c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.447+0000 7f9d87e00640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x7f9d6806cad0 0x7f9d6806ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.447+0000 7f9d87e00640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x7f9d6806cad0 0x7f9d6806ef10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.447+0000 7f9d84c00640 1 -- 172.21.15.66:0/2382425141 <== osd.4 v2:172.21.15.156:6800/27641 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f9d6806f5e0 con 0x7f9d6806cad0 2024-03-30T00:51:56.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.459+0000 7f9d8fc00640 1 -- 172.21.15.66:0/2382425141 --> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9d88071450 con 0x7f9d6806cad0 2024-03-30T00:51:56.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.459+0000 7f9d84c00640 1 -- 172.21.15.66:0/2382425141 <== osd.4 v2:172.21.15.156:6800/27641 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9d88071450 con 0x7f9d6806cad0 2024-03-30T00:51:56.469 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:56.469 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T00:51:56.469 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:56.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.459+0000 7f9d8fc00640 1 -- 172.21.15.66:0/2382425141 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x7f9d6806cad0 msgr2=0x7f9d6806ef10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.459+0000 7f9d8fc00640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.156:6800/27641,v1:172.21.15.156:6801/27641] conn(0x7f9d6806cad0 0x7f9d6806ef10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.459+0000 7f9d8fc00640 1 -- 172.21.15.66:0/2382425141 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d68066690 msgr2=0x7f9d68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.459+0000 7f9d8fc00640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f9d68066690 0x7f9d68068b50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009a40 tx=0x7f9d7c009190 comp rx=0 tx=0).stop 2024-03-30T00:51:56.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.459+0000 7f9d8fc00640 1 -- 172.21.15.66:0/2382425141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815ba00 msgr2=0x7f9d8815ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.459+0000 7f9d8fc00640 1 --2- 172.21.15.66:0/2382425141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815ba00 0x7f9d8815ce70 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000efc0 tx=0x7f9d8000c490 comp rx=0 tx=0).stop 2024-03-30T00:51:56.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.459+0000 7f9d8dc00640 1 -- 172.21.15.66:0/2382425141 reap_dead start 2024-03-30T00:51:56.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.463+0000 7f9d8fc00640 1 -- 172.21.15.66:0/2382425141 shutdown_connections 2024-03-30T00:51:56.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.463+0000 7f9d8fc00640 1 -- 172.21.15.66:0/2382425141 >> 172.21.15.66:0/2382425141 conn(0x7f9d8806a190 msgr2=0x7f9d8806a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:56.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.463+0000 7f9d8fc00640 1 -- 172.21.15.66:0/2382425141 shutdown_connections 2024-03-30T00:51:56.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.463+0000 7f9d8fc00640 1 -- 172.21.15.66:0/2382425141 wait complete. 2024-03-30T00:51:56.487 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-30T00:51:56.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f2308e00640 1 Processor -- start 2024-03-30T00:51:56.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f2308e00640 1 -- start start 2024-03-30T00:51:56.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f2308e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f230415ca80 0x7f230406b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f2308e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23040cb570 0x7f23040cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f2308e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23040cb0e0 con 0x7f23040cf2f0 2024-03-30T00:51:56.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f2308e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23040cf7f0 con 0x7f230415ca80 2024-03-30T00:51:56.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f2308e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23040cfad0 con 0x7f23040cb570 2024-03-30T00:51:56.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f2301e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f230415ca80 0x7f230406b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f22f9400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f23040cf2f0 0x7f230406b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35342/0 (socket says 172.21.15.66:35342) 2024-03-30T00:51:56.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f22f9400640 1 -- 172.21.15.66:0/1784726927 learned_addr learned my addr 172.21.15.66:0/1784726927 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:56.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f2302800640 1 --2- 172.21.15.66:0/1784726927 >> v2:172.21.15.156:3300/0 conn(0x7f23040cb570 0x7f23040cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.619+0000 7f2301400640 1 -- 172.21.15.66:0/1784726927 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565471132 0 0) 0x7f23040cb0e0 con 0x7f23040cf2f0 2024-03-30T00:51:56.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2301400640 1 -- 172.21.15.66:0/1784726927 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22e8003530 con 0x7f23040cf2f0 2024-03-30T00:51:56.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2301e00640 1 -- 172.21.15.66:0/1784726927 >> v2:172.21.15.156:3300/0 conn(0x7f23040cb570 msgr2=0x7f23040cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:56.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2301e00640 1 --2- 172.21.15.66:0/1784726927 >> v2:172.21.15.156:3300/0 conn(0x7f23040cb570 0x7f23040cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2301e00640 1 -- 172.21.15.66:0/1784726927 >> v1:172.21.15.66:6789/0 conn(0x7f23040cf2f0 legacy=0x7f230406b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2301e00640 1 -- 172.21.15.66:0/1784726927 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23040cad70 con 0x7f230415ca80 2024-03-30T00:51:56.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2301e00640 1 --2- 172.21.15.66:0/1784726927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f230415ca80 0x7f230406b0b0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f2304071d80 tx=0x7f22f401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef21926f4da72a28 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2301400640 1 -- 172.21.15.66:0/1784726927 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40a8970 con 0x7f230415ca80 2024-03-30T00:51:56.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2301400640 1 -- 172.21.15.66:0/1784726927 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f40a8ad0 con 0x7f230415ca80 2024-03-30T00:51:56.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2301400640 1 -- 172.21.15.66:0/1784726927 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40a8e60 con 0x7f230415ca80 2024-03-30T00:51:56.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2308e00640 1 -- 172.21.15.66:0/1784726927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f230415ca80 msgr2=0x7f230406b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2308e00640 1 --2- 172.21.15.66:0/1784726927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f230415ca80 0x7f230406b0b0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f2304071d80 tx=0x7f22f401ec70 comp rx=0 tx=0).stop 2024-03-30T00:51:56.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2308e00640 1 -- 172.21.15.66:0/1784726927 shutdown_connections 2024-03-30T00:51:56.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2308e00640 1 --2- 172.21.15.66:0/1784726927 >> v2:172.21.15.156:3300/0 conn(0x7f23040cb570 0x7f23040cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2308e00640 1 --2- 172.21.15.66:0/1784726927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f230415ca80 0x7f230406b0b0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2308e00640 1 -- 172.21.15.66:0/1784726927 >> 172.21.15.66:0/1784726927 conn(0x7f23040b2090 msgr2=0x7f23040b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:56.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2308e00640 1 -- 172.21.15.66:0/1784726927 shutdown_connections 2024-03-30T00:51:56.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.623+0000 7f2308e00640 1 -- 172.21.15.66:0/1784726927 wait complete. 2024-03-30T00:51:56.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2308e00640 1 Processor -- start 2024-03-30T00:51:56.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2308e00640 1 -- start start 2024-03-30T00:51:56.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2308e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23040cb570 0x7f230414b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2308e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23040cf2f0 0x7f23040c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2308e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f230415ca80 0x7f23040b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2308e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23040c9c00 con 0x7f23040cb570 2024-03-30T00:51:56.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2308e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23040c7670 con 0x7f23040cf2f0 2024-03-30T00:51:56.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2308e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230414bad0 con 0x7f230415ca80 2024-03-30T00:51:56.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2301e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23040cb570 0x7f230414b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2301e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23040cb570 0x7f230414b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48904/0 (socket says 172.21.15.66:48904) 2024-03-30T00:51:56.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2301e00640 1 -- 172.21.15.66:0/321979364 learned_addr learned my addr 172.21.15.66:0/321979364 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2301e00640 1 -- 172.21.15.66:0/321979364 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f230415ca80 msgr2=0x7f23040b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2301e00640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f230415ca80 0x7f23040b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2301e00640 1 -- 172.21.15.66:0/321979364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23040cf2f0 msgr2=0x7f23040c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2301e00640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23040cf2f0 0x7f23040c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2301e00640 1 -- 172.21.15.66:0/321979364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f4009420 con 0x7f23040cb570 2024-03-30T00:51:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2301e00640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23040cb570 0x7f230414b590 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f22f401b880 tx=0x7f22f401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f22faa00640 1 -- 172.21.15.66:0/321979364 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40b1cb0 con 0x7f23040cb570 2024-03-30T00:51:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f22faa00640 1 -- 172.21.15.66:0/321979364 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f40b1e10 con 0x7f23040cb570 2024-03-30T00:51:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f22faa00640 1 -- 172.21.15.66:0/321979364 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40b9630 con 0x7f23040cb570 2024-03-30T00:51:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2308e00640 1 -- 172.21.15.66:0/321979364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230414bf50 con 0x7f23040cb570 2024-03-30T00:51:56.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.627+0000 7f2308e00640 1 -- 172.21.15.66:0/321979364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23040b5f60 con 0x7f23040cb570 2024-03-30T00:51:56.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.631+0000 7f2308e00640 1 -- 172.21.15.66:0/321979364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23040b6150 con 0x7f23040cb570 2024-03-30T00:51:56.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.631+0000 7f22faa00640 1 -- 172.21.15.66:0/321979364 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f22f40b7040 con 0x7f23040cb570 2024-03-30T00:51:56.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.631+0000 7f22faa00640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f22e0066690 0x7f22e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.631+0000 7f22faa00640 1 -- 172.21.15.66:0/321979364 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f22f4121b30 con 0x7f23040cb570 2024-03-30T00:51:56.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.631+0000 7f22f9400640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f22e0066690 0x7f22e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.631+0000 7f22f9400640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f22e0066690 0x7f22e0068b50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f22ec007fd0 tx=0x7f22ec007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.635+0000 7f22faa00640 1 -- 172.21.15.66:0/321979364 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f23040b6150 con 0x7f23040cb570 2024-03-30T00:51:56.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 -- 172.21.15.66:0/321979364 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f230406bd00 con 0x7f22e0066690 2024-03-30T00:51:56.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f22faa00640 1 -- 172.21.15.66:0/321979364 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f22ec0058a0 con 0x7f22e0066690 2024-03-30T00:51:56.803 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 4 to deep-scrub 2024-03-30T00:51:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 -- 172.21.15.66:0/321979364 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f22e0066690 msgr2=0x7f22e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f22e0066690 0x7f22e0068b50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f22ec007fd0 tx=0x7f22ec007900 comp rx=0 tx=0).stop 2024-03-30T00:51:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 -- 172.21.15.66:0/321979364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23040cb570 msgr2=0x7f230414b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23040cb570 0x7f230414b590 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f22f401b880 tx=0x7f22f401ba10 comp rx=0 tx=0).stop 2024-03-30T00:51:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 -- 172.21.15.66:0/321979364 shutdown_connections 2024-03-30T00:51:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f230415ca80 0x7f23040b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23040cf2f0 0x7f23040c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f22e0066690 0x7f22e0068b50 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 --2- 172.21.15.66:0/321979364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23040cb570 0x7f230414b590 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 -- 172.21.15.66:0/321979364 >> 172.21.15.66:0/321979364 conn(0x7f23040b2090 msgr2=0x7f23040ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 -- 172.21.15.66:0/321979364 shutdown_connections 2024-03-30T00:51:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.795+0000 7f2308e00640 1 -- 172.21.15.66:0/321979364 wait complete. 2024-03-30T00:51:56.824 INFO:tasks.ceph:Scrubbing osd.5 2024-03-30T00:51:56.824 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-30T00:51:56.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.959+0000 7fa58e200640 1 Processor -- start 2024-03-30T00:51:56.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58e200640 1 -- start start 2024-03-30T00:51:56.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58e200640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa5880ae840 0x7fa58815cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58e200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa58815d330 0x7fa588069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58e200640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58806e3d0 con 0x7fa588156d00 2024-03-30T00:51:56.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58e200640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa588071830 con 0x7fa5880ae840 2024-03-30T00:51:56.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58e200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5881570e0 con 0x7fa58815d330 2024-03-30T00:51:56.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58d800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa58815d330 0x7fa588069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa587e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa5880ae840 0x7fa58815cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58d800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa58815d330 0x7fa588069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.66:58444/0 (socket says 172.21.15.66:58444) 2024-03-30T00:51:56.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58d800640 1 -- 172.21.15.66:0/1947160863 learned_addr learned my addr 172.21.15.66:0/1947160863 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:56.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58d800640 1 -- 172.21.15.66:0/1947160863 >> v2:172.21.15.156:3300/0 conn(0x7fa5880ae840 msgr2=0x7fa58815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:56.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58d800640 1 --2- 172.21.15.66:0/1947160863 >> v2:172.21.15.156:3300/0 conn(0x7fa5880ae840 0x7fa58815cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58d800640 1 -- 172.21.15.66:0/1947160863 >> v1:172.21.15.156:6789/0 conn(0x7fa588156d00 legacy=0x7fa5880ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.963+0000 7fa58d800640 1 -- 172.21.15.66:0/1947160863 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58815db70 con 0x7fa58815d330 2024-03-30T00:51:56.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa58d800640 1 --2- 172.21.15.66:0/1947160863 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa58815d330 0x7fa588069d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa580009e50 tx=0x7fa58001ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=995df3e8e8fb2536 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa587400640 1 -- 172.21.15.66:0/1947160863 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5800a58d0 con 0x7fa58815d330 2024-03-30T00:51:56.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa587400640 1 -- 172.21.15.66:0/1947160863 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5800a5a30 con 0x7fa58815d330 2024-03-30T00:51:56.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa587400640 1 -- 172.21.15.66:0/1947160863 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5800a5dc0 con 0x7fa58815d330 2024-03-30T00:51:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa58e200640 1 -- 172.21.15.66:0/1947160863 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa58815d330 msgr2=0x7fa588069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa58e200640 1 --2- 172.21.15.66:0/1947160863 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa58815d330 0x7fa588069d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa580009e50 tx=0x7fa58001ca00 comp rx=0 tx=0).stop 2024-03-30T00:51:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa58e200640 1 -- 172.21.15.66:0/1947160863 shutdown_connections 2024-03-30T00:51:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa58e200640 1 --2- 172.21.15.66:0/1947160863 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa58815d330 0x7fa588069d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa58e200640 1 --2- 172.21.15.66:0/1947160863 >> v2:172.21.15.156:3300/0 conn(0x7fa5880ae840 0x7fa58815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa58e200640 1 -- 172.21.15.66:0/1947160863 >> 172.21.15.66:0/1947160863 conn(0x7fa5880cf2a0 msgr2=0x7fa5880cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa58e200640 1 -- 172.21.15.66:0/1947160863 shutdown_connections 2024-03-30T00:51:56.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa58e200640 1 -- 172.21.15.66:0/1947160863 wait complete. 2024-03-30T00:51:56.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa58e200640 1 Processor -- start 2024-03-30T00:51:56.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.967+0000 7fa58e200640 1 -- start start 2024-03-30T00:51:56.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5880ae840 0x7fa58814c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58e200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa588156d00 0x7fa5880ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58e200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa58815d330 0x7fa5881556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58814d8c0 con 0x7fa5880ae840 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58e200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58814c550 con 0x7fa588156d00 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58e200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58814ccb0 con 0x7fa58815d330 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5880ae840 0x7fa58814c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5880ae840 0x7fa58814c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48916/0 (socket says 172.21.15.66:48916) 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58ce00640 1 -- 172.21.15.66:0/4168298861 learned_addr learned my addr 172.21.15.66:0/4168298861 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa587e00640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa588156d00 0x7fa5880ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58d800640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa58815d330 0x7fa5881556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58ce00640 1 -- 172.21.15.66:0/4168298861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa58815d330 msgr2=0x7fa5881556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58ce00640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa58815d330 0x7fa5881556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58ce00640 1 -- 172.21.15.66:0/4168298861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa588156d00 msgr2=0x7fa5880ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58ce00640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa588156d00 0x7fa5880ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58ce00640 1 -- 172.21.15.66:0/4168298861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa580009b30 con 0x7fa5880ae840 2024-03-30T00:51:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58ce00640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5880ae840 0x7fa58814c010 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00c530 tx=0x7fa57c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa584c00640 1 -- 172.21.15.66:0/4168298861 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c005220 con 0x7fa5880ae840 2024-03-30T00:51:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa584c00640 1 -- 172.21.15.66:0/4168298861 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c005380 con 0x7fa5880ae840 2024-03-30T00:51:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58e200640 1 -- 172.21.15.66:0/4168298861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58815d940 con 0x7fa5880ae840 2024-03-30T00:51:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa584c00640 1 -- 172.21.15.66:0/4168298861 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c007270 con 0x7fa5880ae840 2024-03-30T00:51:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58e200640 1 -- 172.21.15.66:0/4168298861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58814d270 con 0x7fa5880ae840 2024-03-30T00:51:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.971+0000 7fa58e200640 1 -- 172.21.15.66:0/4168298861 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa58815d7c0 con 0x7fa5880ae840 2024-03-30T00:51:56.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.975+0000 7fa584c00640 1 -- 172.21.15.66:0/4168298861 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa57c0073d0 con 0x7fa5880ae840 2024-03-30T00:51:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.975+0000 7fa584c00640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa5680666c0 0x7fa568068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.975+0000 7fa584c00640 1 -- 172.21.15.66:0/4168298861 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fa57c083890 con 0x7fa5880ae840 2024-03-30T00:51:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.975+0000 7fa587e00640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa5680666c0 0x7fa568068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.975+0000 7fa584c00640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x7fa56806cb00 0x7fa56806ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.975+0000 7fa584c00640 1 -- 172.21.15.66:0/4168298861 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa56806f610 con 0x7fa56806cb00 2024-03-30T00:51:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.975+0000 7fa584c00640 1 -- 172.21.15.66:0/4168298861 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa57c0882a0 con 0x7fa5880ae840 2024-03-30T00:51:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.975+0000 7fa58d800640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x7fa56806cb00 0x7fa56806ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.975+0000 7fa587e00640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa5680666c0 0x7fa568068b80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa58815cef0 tx=0x7fa574009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.975+0000 7fa58d800640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x7fa56806cb00 0x7fa56806ef40 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:56.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.975+0000 7fa584c00640 1 -- 172.21.15.66:0/4168298861 <== osd.5 v2:172.21.15.156:6808/27644 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa56806f610 con 0x7fa56806cb00 2024-03-30T00:51:56.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.987+0000 7fa58e200640 1 -- 172.21.15.66:0/4168298861 --> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa588071c50 con 0x7fa56806cb00 2024-03-30T00:51:56.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa584c00640 1 -- 172.21.15.66:0/4168298861 <== osd.5 v2:172.21.15.156:6808/27644 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa5800090d0 con 0x7fa56806cb00 2024-03-30T00:51:56.998 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T00:51:56.998 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:56.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa58e200640 1 -- 172.21.15.66:0/4168298861 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x7fa56806cb00 msgr2=0x7fa56806ef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa58e200640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.156:6808/27644,v1:172.21.15.156:6809/27644] conn(0x7fa56806cb00 0x7fa56806ef40 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:56.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa58e200640 1 -- 172.21.15.66:0/4168298861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa5680666c0 msgr2=0x7fa568068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa58e200640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fa5680666c0 0x7fa568068b80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa58815cef0 tx=0x7fa574009190 comp rx=0 tx=0).stop 2024-03-30T00:51:56.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa58e200640 1 -- 172.21.15.66:0/4168298861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5880ae840 msgr2=0x7fa58814c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:56.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa58e200640 1 --2- 172.21.15.66:0/4168298861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5880ae840 0x7fa58814c010 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00c530 tx=0x7fa57c00c9f0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa58d800640 1 -- 172.21.15.66:0/4168298861 reap_dead start 2024-03-30T00:51:57.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa58e200640 1 -- 172.21.15.66:0/4168298861 shutdown_connections 2024-03-30T00:51:57.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa58e200640 1 -- 172.21.15.66:0/4168298861 >> 172.21.15.66:0/4168298861 conn(0x7fa5880cf2a0 msgr2=0x7fa58806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:57.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa58e200640 1 -- 172.21.15.66:0/4168298861 shutdown_connections 2024-03-30T00:51:57.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:56.991+0000 7fa58e200640 1 -- 172.21.15.66:0/4168298861 wait complete. 2024-03-30T00:51:57.016 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-30T00:51:57.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fcea2600640 1 Processor -- start 2024-03-30T00:51:57.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fcea2600640 1 -- start start 2024-03-30T00:51:57.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fcea2600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fce9c0cec30 0x7fce9c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fcea2600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce9c0cd3e0 con 0x7fce9c0cda90 2024-03-30T00:51:57.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fcea2600640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce9c15a990 con 0x7fce9c0cde70 2024-03-30T00:51:57.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fcea2600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9c0b1b80 con 0x7fce9c0cec30 2024-03-30T00:51:57.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fce9b400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fce9c0cda90 0x7fce9c0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35354/0 (socket says 172.21.15.66:35354) 2024-03-30T00:51:57.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fce9b400640 1 -- 172.21.15.66:0/4109172354 learned_addr learned my addr 172.21.15.66:0/4109172354 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:57.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fce92a00640 1 --2- 172.21.15.66:0/4109172354 >> v2:172.21.15.66:3300/0 conn(0x7fce9c0cec30 0x7fce9c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fce92a00640 1 -- 172.21.15.66:0/4109172354 >> v1:172.21.15.156:6789/0 conn(0x7fce9c0cde70 legacy=0x7fce9c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fce92a00640 1 -- 172.21.15.66:0/4109172354 >> v1:172.21.15.66:6789/0 conn(0x7fce9c0cda90 legacy=0x7fce9c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.151+0000 7fce92a00640 1 -- 172.21.15.66:0/4109172354 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c0b11e0 con 0x7fce9c0cec30 2024-03-30T00:51:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fce92a00640 1 --2- 172.21.15.66:0/4109172354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c0cec30 0x7fce9c159c60 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fce840097c0 tx=0x7fce8401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e49b63f1992a495 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fce9aa00640 1 -- 172.21.15.66:0/4109172354 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce840ab610 con 0x7fce9c0cec30 2024-03-30T00:51:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fce9aa00640 1 -- 172.21.15.66:0/4109172354 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce840ab770 con 0x7fce9c0cec30 2024-03-30T00:51:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fce9aa00640 1 -- 172.21.15.66:0/4109172354 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce840abb00 con 0x7fce9c0cec30 2024-03-30T00:51:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fcea2600640 1 -- 172.21.15.66:0/4109172354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c0cec30 msgr2=0x7fce9c159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fcea2600640 1 --2- 172.21.15.66:0/4109172354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c0cec30 0x7fce9c159c60 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fce840097c0 tx=0x7fce8401c410 comp rx=0 tx=0).stop 2024-03-30T00:51:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fcea2600640 1 -- 172.21.15.66:0/4109172354 shutdown_connections 2024-03-30T00:51:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fcea2600640 1 --2- 172.21.15.66:0/4109172354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c0cec30 0x7fce9c159c60 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fcea2600640 1 -- 172.21.15.66:0/4109172354 >> 172.21.15.66:0/4109172354 conn(0x7fce9c069f60 msgr2=0x7fce9c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fcea2600640 1 -- 172.21.15.66:0/4109172354 shutdown_connections 2024-03-30T00:51:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fcea2600640 1 -- 172.21.15.66:0/4109172354 wait complete. 2024-03-30T00:51:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fcea2600640 1 Processor -- start 2024-03-30T00:51:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.155+0000 7fcea2600640 1 -- start start 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fcea2600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c0cda90 0x7fce9c096d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fcea2600640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce9c0cde70 0x7fce9c154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fcea2600640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce9c0cec30 0x7fce9c154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fcea2600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c155300 con 0x7fce9c0cda90 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fcea2600640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c0b0c50 con 0x7fce9c0cec30 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fcea2600640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c0cb860 con 0x7fce9c0cde70 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce9b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c0cda90 0x7fce9c096d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce9b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c0cda90 0x7fce9c096d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48946/0 (socket says 172.21.15.66:48946) 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce9b400640 1 -- 172.21.15.66:0/2501847014 learned_addr learned my addr 172.21.15.66:0/2501847014 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce92a00640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce9c0cde70 0x7fce9c154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce9be00640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce9c0cec30 0x7fce9c154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce9b400640 1 -- 172.21.15.66:0/2501847014 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce9c0cde70 msgr2=0x7fce9c154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce9b400640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce9c0cde70 0x7fce9c154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce9b400640 1 -- 172.21.15.66:0/2501847014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce9c0cec30 msgr2=0x7fce9c154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce9b400640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce9c0cec30 0x7fce9c154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce9b400640 1 -- 172.21.15.66:0/2501847014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce84009420 con 0x7fce9c0cda90 2024-03-30T00:51:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce9b400640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c0cda90 0x7fce9c096d00 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00c430 tx=0x7fce8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce93e00640 1 -- 172.21.15.66:0/2501847014 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c005cf0 con 0x7fce9c0cda90 2024-03-30T00:51:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce93e00640 1 -- 172.21.15.66:0/2501847014 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c005e50 con 0x7fce9c0cda90 2024-03-30T00:51:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fce93e00640 1 -- 172.21.15.66:0/2501847014 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c0072d0 con 0x7fce9c0cda90 2024-03-30T00:51:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fcea2600640 1 -- 172.21.15.66:0/2501847014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9c0978c0 con 0x7fce9c0cda90 2024-03-30T00:51:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fcea2600640 1 -- 172.21.15.66:0/2501847014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c0975e0 con 0x7fce9c0cda90 2024-03-30T00:51:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.159+0000 7fcea2600640 1 -- 172.21.15.66:0/2501847014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce9c066c30 con 0x7fce9c0cda90 2024-03-30T00:51:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.163+0000 7fce93e00640 1 -- 172.21.15.66:0/2501847014 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fce8c007430 con 0x7fce9c0cda90 2024-03-30T00:51:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.163+0000 7fce93e00640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce780666c0 0x7fce78068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.163+0000 7fce93e00640 1 -- 172.21.15.66:0/2501847014 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fce8c083510 con 0x7fce9c0cda90 2024-03-30T00:51:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.163+0000 7fce92a00640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce780666c0 0x7fce78068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.163+0000 7fce92a00640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce780666c0 0x7fce78068b80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fce840af000 tx=0x7fce840a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:57.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.167+0000 7fce93e00640 1 -- 172.21.15.66:0/2501847014 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fce8c050a30 con 0x7fce9c0cda90 2024-03-30T00:51:57.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.327+0000 7fcea2600640 1 -- 172.21.15.66:0/2501847014 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fce9c072000 con 0x7fce780666c0 2024-03-30T00:51:57.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.327+0000 7fce93e00640 1 -- 172.21.15.66:0/2501847014 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fce9c072000 con 0x7fce780666c0 2024-03-30T00:51:57.335 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 5 to deep-scrub 2024-03-30T00:51:57.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 -- 172.21.15.66:0/2501847014 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce780666c0 msgr2=0x7fce78068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce780666c0 0x7fce78068b80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fce840af000 tx=0x7fce840a7000 comp rx=0 tx=0).stop 2024-03-30T00:51:57.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 -- 172.21.15.66:0/2501847014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c0cda90 msgr2=0x7fce9c096d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c0cda90 0x7fce9c096d00 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00c430 tx=0x7fce8c00c8f0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 -- 172.21.15.66:0/2501847014 shutdown_connections 2024-03-30T00:51:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce9c0cec30 0x7fce9c154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce9c0cde70 0x7fce9c154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fce780666c0 0x7fce78068b80 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 --2- 172.21.15.66:0/2501847014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce9c0cda90 0x7fce9c096d00 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 -- 172.21.15.66:0/2501847014 >> 172.21.15.66:0/2501847014 conn(0x7fce9c069f60 msgr2=0x7fce9c15b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 -- 172.21.15.66:0/2501847014 shutdown_connections 2024-03-30T00:51:57.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.331+0000 7fcea2600640 1 -- 172.21.15.66:0/2501847014 wait complete. 2024-03-30T00:51:57.358 INFO:tasks.ceph:Scrubbing osd.6 2024-03-30T00:51:57.358 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-30T00:51:57.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.491+0000 7fd4f7e00640 1 Processor -- start 2024-03-30T00:51:57.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f7e00640 1 -- start start 2024-03-30T00:51:57.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f7e00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd4f8156d00 0x7fd4f80ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd4f815d500 0x7fd4f8069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f7e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f806e3d0 con 0x7fd4f80ae670 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f7e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f8071830 con 0x7fd4f815d500 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f7e00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f80aeae0 con 0x7fd4f8156d00 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd4f815d500 0x7fd4f8069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4efe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd4f80ae670 0x7fd4f815cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35360/0 (socket says 172.21.15.66:35360) 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4efe00640 1 -- 172.21.15.66:0/1742964813 learned_addr learned my addr 172.21.15.66:0/1742964813 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f6a00640 1 --2- 172.21.15.66:0/1742964813 >> v2:172.21.15.156:3300/0 conn(0x7fd4f8156d00 0x7fd4f80ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f7400640 1 -- 172.21.15.66:0/1742964813 >> v2:172.21.15.156:3300/0 conn(0x7fd4f8156d00 msgr2=0x7fd4f80ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f7400640 1 --2- 172.21.15.66:0/1742964813 >> v2:172.21.15.156:3300/0 conn(0x7fd4f8156d00 0x7fd4f80ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f7400640 1 -- 172.21.15.66:0/1742964813 >> v1:172.21.15.66:6789/0 conn(0x7fd4f80ae670 legacy=0x7fd4f815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f7400640 1 -- 172.21.15.66:0/1742964813 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f81570e0 con 0x7fd4f815d500 2024-03-30T00:51:57.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.495+0000 7fd4f6a00640 1 --2- 172.21.15.66:0/1742964813 >> v2:172.21.15.156:3300/0 conn(0x7fd4f8156d00 0x7fd4f80ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:57.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f7400640 1 --2- 172.21.15.66:0/1742964813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4f815d500 0x7fd4f8069d90 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8072af0 tx=0x7fd4f001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4baceb8a5c8549c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:57.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f6000640 1 -- 172.21.15.66:0/1742964813 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00a58e0 con 0x7fd4f815d500 2024-03-30T00:51:57.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f6000640 1 -- 172.21.15.66:0/1742964813 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f00a5a40 con 0x7fd4f815d500 2024-03-30T00:51:57.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f6000640 1 -- 172.21.15.66:0/1742964813 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00a5dd0 con 0x7fd4f815d500 2024-03-30T00:51:57.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1742964813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4f815d500 msgr2=0x7fd4f8069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f7e00640 1 --2- 172.21.15.66:0/1742964813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4f815d500 0x7fd4f8069d90 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8072af0 tx=0x7fd4f001c930 comp rx=0 tx=0).stop 2024-03-30T00:51:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1742964813 shutdown_connections 2024-03-30T00:51:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f7e00640 1 --2- 172.21.15.66:0/1742964813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4f815d500 0x7fd4f8069d90 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f7e00640 1 --2- 172.21.15.66:0/1742964813 >> v2:172.21.15.156:3300/0 conn(0x7fd4f8156d00 0x7fd4f80ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1742964813 >> 172.21.15.66:0/1742964813 conn(0x7fd4f80cf2a0 msgr2=0x7fd4f80cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1742964813 shutdown_connections 2024-03-30T00:51:57.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1742964813 wait complete. 2024-03-30T00:51:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f7e00640 1 Processor -- start 2024-03-30T00:51:57.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.499+0000 7fd4f7e00640 1 -- start start 2024-03-30T00:51:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f7e00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4f80ae670 0x7fd4f80ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4f8156d00 0x7fd4f8151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f7e00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4f815d500 0x7fd4f8155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f8149160 con 0x7fd4f8156d00 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f7e00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f8149440 con 0x7fd4f815d500 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f7e00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f8149720 con 0x7fd4f80ae670 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4efe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4f8156d00 0x7fd4f8151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4efe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4f8156d00 0x7fd4f8151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48974/0 (socket says 172.21.15.66:48974) 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4efe00640 1 -- 172.21.15.66:0/1792407690 learned_addr learned my addr 172.21.15.66:0/1792407690 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f7400640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4f815d500 0x7fd4f8155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f6a00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4f80ae670 0x7fd4f80ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4efe00640 1 -- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4f80ae670 msgr2=0x7fd4f80ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4efe00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4f80ae670 0x7fd4f80ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4efe00640 1 -- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4f815d500 msgr2=0x7fd4f8155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4efe00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4f815d500 0x7fd4f8155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4efe00640 1 -- 172.21.15.66:0/1792407690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f0009a70 con 0x7fd4f8156d00 2024-03-30T00:51:57.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f6a00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4f80ae670 0x7fd4f80ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:57.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4efe00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4f8156d00 0x7fd4f8151cb0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000c420 tx=0x7fd4e000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:57.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4eea00640 1 -- 172.21.15.66:0/1792407690 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0005cf0 con 0x7fd4f8156d00 2024-03-30T00:51:57.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4eea00640 1 -- 172.21.15.66:0/1792407690 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e0005e50 con 0x7fd4f8156d00 2024-03-30T00:51:57.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4eea00640 1 -- 172.21.15.66:0/1792407690 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00072f0 con 0x7fd4f8156d00 2024-03-30T00:51:57.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1792407690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f8149a00 con 0x7fd4f8156d00 2024-03-30T00:51:57.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1792407690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f814a840 con 0x7fd4f8156d00 2024-03-30T00:51:57.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1792407690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd4f8071c50 con 0x7fd4f8156d00 2024-03-30T00:51:57.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.503+0000 7fd4eea00640 1 -- 172.21.15.66:0/1792407690 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd4e0007450 con 0x7fd4f8156d00 2024-03-30T00:51:57.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.507+0000 7fd4eea00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd4d4066690 0x7fd4d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.507+0000 7fd4eea00640 1 -- 172.21.15.66:0/1792407690 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd4e0083dd0 con 0x7fd4f8156d00 2024-03-30T00:51:57.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.507+0000 7fd4f6a00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd4d4066690 0x7fd4d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.507+0000 7fd4eea00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x7fd4d406cad0 0x7fd4d406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.507+0000 7fd4eea00640 1 -- 172.21.15.66:0/1792407690 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4d406f5e0 con 0x7fd4d406cad0 2024-03-30T00:51:57.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.507+0000 7fd4eea00640 1 -- 172.21.15.66:0/1792407690 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4e0084140 con 0x7fd4f8156d00 2024-03-30T00:51:57.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.507+0000 7fd4f7400640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x7fd4d406cad0 0x7fd4d406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.507+0000 7fd4f6a00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd4d4066690 0x7fd4d4068b50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8006fd0 tx=0x7fd4e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:57.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.507+0000 7fd4f7400640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x7fd4d406cad0 0x7fd4d406ef10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:57.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.507+0000 7fd4eea00640 1 -- 172.21.15.66:0/1792407690 <== osd.6 v2:172.21.15.156:6816/27642 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fd4d406f5e0 con 0x7fd4d406cad0 2024-03-30T00:51:57.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.519+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1792407690 --> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd4f8071450 con 0x7fd4d406cad0 2024-03-30T00:51:57.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4eea00640 1 -- 172.21.15.66:0/1792407690 <== osd.6 v2:172.21.15.156:6816/27642 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd4f00a5bb0 con 0x7fd4d406cad0 2024-03-30T00:51:57.530 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:57.530 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T00:51:57.530 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:57.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x7fd4d406cad0 msgr2=0x7fd4d406ef10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4f7e00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.156:6816/27642,v1:172.21.15.156:6817/27642] conn(0x7fd4d406cad0 0x7fd4d406ef10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1792407690 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd4d4066690 msgr2=0x7fd4d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4f7e00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fd4d4066690 0x7fd4d4068b50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8006fd0 tx=0x7fd4e8008040 comp rx=0 tx=0).stop 2024-03-30T00:51:57.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1792407690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4f8156d00 msgr2=0x7fd4f8151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4f7e00640 1 --2- 172.21.15.66:0/1792407690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4f8156d00 0x7fd4f8151cb0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000c420 tx=0x7fd4e000c8e0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4f7400640 1 -- 172.21.15.66:0/1792407690 reap_dead start 2024-03-30T00:51:57.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1792407690 shutdown_connections 2024-03-30T00:51:57.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1792407690 >> 172.21.15.66:0/1792407690 conn(0x7fd4f80cf2a0 msgr2=0x7fd4f815f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:57.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1792407690 shutdown_connections 2024-03-30T00:51:57.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.523+0000 7fd4f7e00640 1 -- 172.21.15.66:0/1792407690 wait complete. 2024-03-30T00:51:57.549 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-30T00:51:57.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.683+0000 7f3a01800640 1 Processor -- start 2024-03-30T00:51:57.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.683+0000 7f3a01800640 1 -- start start 2024-03-30T00:51:57.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.683+0000 7f3a01800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f39fc0cc580 0x7f39fc167630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.683+0000 7f3a01800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc0cb0e0 con 0x7f39fc0cf2f0 2024-03-30T00:51:57.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.683+0000 7f3a01800640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc06bed0 con 0x7f39fc15ca80 2024-03-30T00:51:57.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.683+0000 7f3a01800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39fc0cf7f0 con 0x7f39fc0cc580 2024-03-30T00:51:57.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.683+0000 7f39f3e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f39fc0cf2f0 0x7f39fc06b7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35370/0 (socket says 172.21.15.66:35370) 2024-03-30T00:51:57.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.683+0000 7f39f3e00640 1 -- 172.21.15.66:0/1225795904 learned_addr learned my addr 172.21.15.66:0/1225795904 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:57.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fb400640 1 --2- 172.21.15.66:0/1225795904 >> v2:172.21.15.156:3300/0 conn(0x7f39fc0cc580 0x7f39fc167630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404413101 0 0) 0x7f39fc0cb0e0 con 0x7f39fc0cf2f0 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e0003530 con 0x7f39fc0cf2f0 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1847996569 0 0) 0x7f39e0003530 con 0x7f39fc0cf2f0 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 >> v2:172.21.15.156:3300/0 conn(0x7f39fc0cc580 msgr2=0x7f39fc167630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 --2- 172.21.15.66:0/1225795904 >> v2:172.21.15.156:3300/0 conn(0x7f39fc0cc580 0x7f39fc167630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 >> v1:172.21.15.156:6789/0 conn(0x7f39fc15ca80 legacy=0x7f39fc06b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0cfad0 con 0x7f39fc0cf2f0 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1694807146 0 0) 0x7f39e4001680 con 0x7f39fc0cf2f0 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 >> v1:172.21.15.66:6789/0 conn(0x7f39fc0cf2f0 legacy=0x7f39fc06b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 0x7f39e0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39e00047d0 0x7f39e0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39e0005190 0x7f39e000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc0cf7f0 con 0x7f39e0003980 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e000c040 con 0x7f39e0005190 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e000c200 con 0x7f39e00047d0 2024-03-30T00:51:57.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39faa00640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 0x7f39e0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fb400640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39e0005190 0x7f39e000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39f3e00640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39e00047d0 0x7f39e0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39faa00640 1 -- 172.21.15.66:0/1225795904 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39e00047d0 msgr2=0x7f39e0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39faa00640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39e00047d0 0x7f39e0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39faa00640 1 -- 172.21.15.66:0/1225795904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39e0005190 msgr2=0x7f39e000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39faa00640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39e0005190 0x7f39e000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39faa00640 1 -- 172.21.15.66:0/1225795904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0003530 con 0x7f39e0003980 2024-03-30T00:51:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39fb400640 1 -- 172.21.15.66:0/1225795904 reap_dead start 2024-03-30T00:51:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.687+0000 7f39f3e00640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39e00047d0 0x7f39e0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:51:57.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f39faa00640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 0x7f39e0003da0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f39ec001780 tx=0x7f39ec01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5211d106d7532f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:57.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0052d0 con 0x7f39e0003980 2024-03-30T00:51:57.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec0a7050 con 0x7f39e0003980 2024-03-30T00:51:57.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f39fa000640 1 -- 172.21.15.66:0/1225795904 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec003600 con 0x7f39e0003980 2024-03-30T00:51:57.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 -- 172.21.15.66:0/1225795904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 msgr2=0x7f39e0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 0x7f39e0003da0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f39ec001780 tx=0x7f39ec01c400 comp rx=0 tx=0).stop 2024-03-30T00:51:57.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 -- 172.21.15.66:0/1225795904 shutdown_connections 2024-03-30T00:51:57.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 --2- 172.21.15.66:0/1225795904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 0x7f39e0003da0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 -- 172.21.15.66:0/1225795904 >> 172.21.15.66:0/1225795904 conn(0x7f39fc0b2090 msgr2=0x7f39fc0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:57.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 -- 172.21.15.66:0/1225795904 shutdown_connections 2024-03-30T00:51:57.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 -- 172.21.15.66:0/1225795904 wait complete. 2024-03-30T00:51:57.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 Processor -- start 2024-03-30T00:51:57.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 -- start start 2024-03-30T00:51:57.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 0x7f39fc1f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39fc150650 0x7f39fc1f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39fc14aa20 0x7f39fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc15f5c0 con 0x7f39e0003980 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc0ca100 con 0x7f39fc150650 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f3a01800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc0ca3e0 con 0x7f39fc14aa20 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.691+0000 7f39faa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 0x7f39fc1f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39faa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 0x7f39fc1f15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49002/0 (socket says 172.21.15.66:49002) 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39faa00640 1 -- 172.21.15.66:0/974184673 learned_addr learned my addr 172.21.15.66:0/974184673 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39f3e00640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39fc150650 0x7f39fc1f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39faa00640 1 -- 172.21.15.66:0/974184673 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39fc14aa20 msgr2=0x7f39fc0b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39fb400640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39fc14aa20 0x7f39fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39faa00640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39fc14aa20 0x7f39fc0b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39faa00640 1 -- 172.21.15.66:0/974184673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39fc150650 msgr2=0x7f39fc1f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39faa00640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39fc150650 0x7f39fc1f1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39faa00640 1 -- 172.21.15.66:0/974184673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc15f880 con 0x7f39e0003980 2024-03-30T00:51:57.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39f3e00640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39fc150650 0x7f39fc1f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:57.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39fb400640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39fc14aa20 0x7f39fc0b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:51:57.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39faa00640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 0x7f39fc1f15b0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f39ec01c230 tx=0x7f39e0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:57.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39f2a00640 1 -- 172.21.15.66:0/974184673 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec003e40 con 0x7f39e0003980 2024-03-30T00:51:57.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39f2a00640 1 -- 172.21.15.66:0/974184673 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec007a40 con 0x7f39e0003980 2024-03-30T00:51:57.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39f2a00640 1 -- 172.21.15.66:0/974184673 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec007270 con 0x7f39e0003980 2024-03-30T00:51:57.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f3a01800640 1 -- 172.21.15.66:0/974184673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc15fb60 con 0x7f39e0003980 2024-03-30T00:51:57.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f3a01800640 1 -- 172.21.15.66:0/974184673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc0b54d0 con 0x7f39e0003980 2024-03-30T00:51:57.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f3a01800640 1 -- 172.21.15.66:0/974184673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39fc0ca5a0 con 0x7f39e0003980 2024-03-30T00:51:57.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39f2a00640 1 -- 172.21.15.66:0/974184673 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f39ec0073d0 con 0x7f39e0003980 2024-03-30T00:51:57.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.695+0000 7f39f2a00640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f39d8066690 0x7f39d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:57.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.699+0000 7f39f2a00640 1 -- 172.21.15.66:0/974184673 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f39ec11e530 con 0x7f39e0003980 2024-03-30T00:51:57.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.699+0000 7f39f3e00640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f39d8066690 0x7f39d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:57.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.699+0000 7f39f3e00640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f39d8066690 0x7f39d8068b50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f39fc1f5540 tx=0x7f39e4001530 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:57.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.699+0000 7f39f2a00640 1 -- 172.21.15.66:0/974184673 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f39ec0ef3b0 con 0x7f39e0003980 2024-03-30T00:51:57.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 -- 172.21.15.66:0/974184673 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f39fc15ce60 con 0x7f39d8066690 2024-03-30T00:51:57.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f39f2a00640 1 -- 172.21.15.66:0/974184673 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f39e40128a0 con 0x7f39d8066690 2024-03-30T00:51:57.871 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 6 to deep-scrub 2024-03-30T00:51:57.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 -- 172.21.15.66:0/974184673 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f39d8066690 msgr2=0x7f39d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f39d8066690 0x7f39d8068b50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f39fc1f5540 tx=0x7f39e4001530 comp rx=0 tx=0).stop 2024-03-30T00:51:57.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 -- 172.21.15.66:0/974184673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 msgr2=0x7f39fc1f15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:57.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 0x7f39fc1f15b0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f39ec01c230 tx=0x7f39e0005800 comp rx=0 tx=0).stop 2024-03-30T00:51:57.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 -- 172.21.15.66:0/974184673 shutdown_connections 2024-03-30T00:51:57.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39fc14aa20 0x7f39fc0b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39fc150650 0x7f39fc1f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f39d8066690 0x7f39d8068b50 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 --2- 172.21.15.66:0/974184673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39e0003980 0x7f39fc1f15b0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:57.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 -- 172.21.15.66:0/974184673 >> 172.21.15.66:0/974184673 conn(0x7f39fc0b2090 msgr2=0x7f39fc06e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:57.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 -- 172.21.15.66:0/974184673 shutdown_connections 2024-03-30T00:51:57.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:57.863+0000 7f3a01800640 1 -- 172.21.15.66:0/974184673 wait complete. 2024-03-30T00:51:57.892 INFO:tasks.ceph:Scrubbing osd.7 2024-03-30T00:51:57.892 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-30T00:51:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5871800640 1 Processor -- start 2024-03-30T00:51:58.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5871800640 1 -- start start 2024-03-30T00:51:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5871800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c06b0b0 0x7f586c0cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5871800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f586c0ce460 0x7f586c0b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5871800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586c0b3a50 con 0x7f586c15c830 2024-03-30T00:51:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5871800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586c0ce8d0 con 0x7f586c0ce460 2024-03-30T00:51:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5871800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0b3d30 con 0x7f586c06b0b0 2024-03-30T00:51:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f586be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f586c15c830 0x7f586c0b3200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35372/0 (socket says 172.21.15.66:35372) 2024-03-30T00:51:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f586be00640 1 -- 172.21.15.66:0/49312520 learned_addr learned my addr 172.21.15.66:0/49312520 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5870e00640 1 --2- 172.21.15.66:0/49312520 >> v2:172.21.15.156:3300/0 conn(0x7f586c0ce460 0x7f586c0b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5863400640 1 --2- 172.21.15.66:0/49312520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c06b0b0 0x7f586c0cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f586b400640 1 -- 172.21.15.66:0/49312520 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187046535 0 0) 0x7f586c0b3a50 con 0x7f586c15c830 2024-03-30T00:51:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f586b400640 1 -- 172.21.15.66:0/49312520 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5850003530 con 0x7f586c15c830 2024-03-30T00:51:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5870e00640 1 -- 172.21.15.66:0/49312520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c06b0b0 msgr2=0x7f586c0cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:51:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5870e00640 1 --2- 172.21.15.66:0/49312520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c06b0b0 0x7f586c0cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5870e00640 1 -- 172.21.15.66:0/49312520 >> v1:172.21.15.66:6789/0 conn(0x7f586c15c830 legacy=0x7f586c0b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.031+0000 7f5870e00640 1 -- 172.21.15.66:0/49312520 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586c0b4010 con 0x7f586c0ce460 2024-03-30T00:51:58.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f5870e00640 1 --2- 172.21.15.66:0/49312520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c0ce460 0x7f586c0b4af0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5864009d90 tx=0x7f586401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3166a4c3cdfca2a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:58.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f586b400640 1 -- 172.21.15.66:0/49312520 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58640a5a00 con 0x7f586c0ce460 2024-03-30T00:51:58.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f586b400640 1 -- 172.21.15.66:0/49312520 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58640a5b60 con 0x7f586c0ce460 2024-03-30T00:51:58.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f586b400640 1 -- 172.21.15.66:0/49312520 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58640ad440 con 0x7f586c0ce460 2024-03-30T00:51:58.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f5871800640 1 -- 172.21.15.66:0/49312520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c0ce460 msgr2=0x7f586c0b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f5871800640 1 --2- 172.21.15.66:0/49312520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c0ce460 0x7f586c0b4af0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5864009d90 tx=0x7f586401c940 comp rx=0 tx=0).stop 2024-03-30T00:51:58.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f5871800640 1 -- 172.21.15.66:0/49312520 shutdown_connections 2024-03-30T00:51:58.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f5871800640 1 --2- 172.21.15.66:0/49312520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c0ce460 0x7f586c0b4af0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f5871800640 1 --2- 172.21.15.66:0/49312520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c06b0b0 0x7f586c0cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f5871800640 1 -- 172.21.15.66:0/49312520 >> 172.21.15.66:0/49312520 conn(0x7f586c06fad0 msgr2=0x7f586c06fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:58.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f5871800640 1 -- 172.21.15.66:0/49312520 shutdown_connections 2024-03-30T00:51:58.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f5871800640 1 -- 172.21.15.66:0/49312520 wait complete. 2024-03-30T00:51:58.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f5871800640 1 Processor -- start 2024-03-30T00:51:58.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.035+0000 7f5871800640 1 -- start start 2024-03-30T00:51:58.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5871800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c06b0b0 0x7f586c0c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5871800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c0ce460 0x7f586c06feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5871800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586c15c830 0x7f586c0703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5871800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0c4ca0 con 0x7f586c15c830 2024-03-30T00:51:58.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5871800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0c4020 con 0x7f586c06b0b0 2024-03-30T00:51:58.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5871800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0c4300 con 0x7f586c0ce460 2024-03-30T00:51:58.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5870e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586c15c830 0x7f586c0703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5870e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586c15c830 0x7f586c0703f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49012/0 (socket says 172.21.15.66:49012) 2024-03-30T00:51:58.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5870e00640 1 -- 172.21.15.66:0/2273592022 learned_addr learned my addr 172.21.15.66:0/2273592022 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:58.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f586be00640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c06b0b0 0x7f586c0c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5863400640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c0ce460 0x7f586c06feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5870e00640 1 -- 172.21.15.66:0/2273592022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c0ce460 msgr2=0x7f586c06feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5870e00640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f586c0ce460 0x7f586c06feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5870e00640 1 -- 172.21.15.66:0/2273592022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c06b0b0 msgr2=0x7f586c0c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5870e00640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f586c06b0b0 0x7f586c0c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5870e00640 1 -- 172.21.15.66:0/2273592022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5864009a70 con 0x7f586c15c830 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5870e00640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586c15c830 0x7f586c0703f0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f586401a570 tx=0x7f58640196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5868c00640 1 -- 172.21.15.66:0/2273592022 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58640bd070 con 0x7f586c15c830 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5868c00640 1 -- 172.21.15.66:0/2273592022 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5864004040 con 0x7f586c15c830 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5871800640 1 -- 172.21.15.66:0/2273592022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c0c45e0 con 0x7f586c15c830 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5868c00640 1 -- 172.21.15.66:0/2273592022 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864004430 con 0x7f586c15c830 2024-03-30T00:51:58.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5871800640 1 -- 172.21.15.66:0/2273592022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586c0c5600 con 0x7f586c15c830 2024-03-30T00:51:58.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.039+0000 7f5871800640 1 -- 172.21.15.66:0/2273592022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f586c073cf0 con 0x7f586c15c830 2024-03-30T00:51:58.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.043+0000 7f5868c00640 1 -- 172.21.15.66:0/2273592022 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f58640b4030 con 0x7f586c15c830 2024-03-30T00:51:58.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.043+0000 7f5868c00640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5848066690 0x7f5848068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.043+0000 7f5868c00640 1 -- 172.21.15.66:0/2273592022 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f586411f570 con 0x7f586c15c830 2024-03-30T00:51:58.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.043+0000 7f5868c00640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x7f584806cad0 0x7f584806ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.043+0000 7f586be00640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5848066690 0x7f5848068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.043+0000 7f5868c00640 1 -- 172.21.15.66:0/2273592022 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f584806f5e0 con 0x7f584806cad0 2024-03-30T00:51:58.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.043+0000 7f5868c00640 1 -- 172.21.15.66:0/2273592022 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f586411f8e0 con 0x7f586c15c830 2024-03-30T00:51:58.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.043+0000 7f5863400640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x7f584806cad0 0x7f584806ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.043+0000 7f586be00640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5848066690 0x7f5848068b50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f585c007fd0 tx=0x7f585c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:58.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.043+0000 7f5863400640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x7f584806cad0 0x7f584806ef10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:58.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.043+0000 7f5868c00640 1 -- 172.21.15.66:0/2273592022 <== osd.7 v2:172.21.15.156:6824/27643 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f584806f5e0 con 0x7f584806cad0 2024-03-30T00:51:58.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.055+0000 7f5871800640 1 -- 172.21.15.66:0/2273592022 --> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f586c0cd620 con 0x7f584806cad0 2024-03-30T00:51:58.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5868c00640 1 -- 172.21.15.66:0/2273592022 <== osd.7 v2:172.21.15.156:6824/27643 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f586c0cd620 con 0x7f584806cad0 2024-03-30T00:51:58.066 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T00:51:58.066 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T00:51:58.066 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T00:51:58.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5871800640 1 -- 172.21.15.66:0/2273592022 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x7f584806cad0 msgr2=0x7f584806ef10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5871800640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.156:6824/27643,v1:172.21.15.156:6825/27643] conn(0x7f584806cad0 0x7f584806ef10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5871800640 1 -- 172.21.15.66:0/2273592022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5848066690 msgr2=0x7f5848068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5871800640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f5848066690 0x7f5848068b50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f585c007fd0 tx=0x7f585c009040 comp rx=0 tx=0).stop 2024-03-30T00:51:58.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5871800640 1 -- 172.21.15.66:0/2273592022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586c15c830 msgr2=0x7f586c0703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5871800640 1 --2- 172.21.15.66:0/2273592022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586c15c830 0x7f586c0703f0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f586401a570 tx=0x7f58640196e0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5870e00640 1 -- 172.21.15.66:0/2273592022 reap_dead start 2024-03-30T00:51:58.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5871800640 1 -- 172.21.15.66:0/2273592022 shutdown_connections 2024-03-30T00:51:58.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5871800640 1 -- 172.21.15.66:0/2273592022 >> 172.21.15.66:0/2273592022 conn(0x7f586c06fad0 msgr2=0x7f586c158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:58.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5871800640 1 -- 172.21.15.66:0/2273592022 shutdown_connections 2024-03-30T00:51:58.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.059+0000 7f5871800640 1 -- 172.21.15.66:0/2273592022 wait complete. 2024-03-30T00:51:58.084 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-30T00:51:58.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.215+0000 7fc71f400640 1 Processor -- start 2024-03-30T00:51:58.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.215+0000 7fc71f400640 1 -- start start 2024-03-30T00:51:58.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc71f400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc7180ce780 0x7fc7180cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc71f400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7180cbec0 con 0x7fc7180ceb60 2024-03-30T00:51:58.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc71f400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7180cce50 con 0x7fc7180cd5e0 2024-03-30T00:51:58.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc71f400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71815d970 con 0x7fc7180ce780 2024-03-30T00:51:58.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc70fe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc7180ceb60 0x7fc71815ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35388/0 (socket says 172.21.15.66:35388) 2024-03-30T00:51:58.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc70fe00640 1 -- 172.21.15.66:0/306558981 learned_addr learned my addr 172.21.15.66:0/306558981 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:58.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc71ca00640 1 --2- 172.21.15.66:0/306558981 >> v2:172.21.15.156:3300/0 conn(0x7fc7180ce780 0x7fc7180cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc717e00640 1 -- 172.21.15.66:0/306558981 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624511384 0 0) 0x7fc7180cbec0 con 0x7fc7180ceb60 2024-03-30T00:51:58.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc717e00640 1 -- 172.21.15.66:0/306558981 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6fc003530 con 0x7fc7180ceb60 2024-03-30T00:51:58.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc71ca00640 1 -- 172.21.15.66:0/306558981 >> v1:172.21.15.156:6789/0 conn(0x7fc7180cd5e0 legacy=0x7fc7180c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc71ca00640 1 -- 172.21.15.66:0/306558981 >> v1:172.21.15.66:6789/0 conn(0x7fc7180ceb60 legacy=0x7fc71815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc717e00640 1 -- 172.21.15.66:0/306558981 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 202641699 0 0) 0x7fc6fc003530 con 0x7fc7180ceb60 2024-03-30T00:51:58.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.219+0000 7fc71ca00640 1 -- 172.21.15.66:0/306558981 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71815dc50 con 0x7fc7180ce780 2024-03-30T00:51:58.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71ca00640 1 --2- 172.21.15.66:0/306558981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7180ce780 0x7fc7180cc910 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc7080097c0 tx=0x7fc70801c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=745ef83ebc7dbed3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:58.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc717e00640 1 -- 172.21.15.66:0/306558981 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700001680 con 0x7fc7180ce780 2024-03-30T00:51:58.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc717e00640 1 -- 172.21.15.66:0/306558981 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7080a7050 con 0x7fc7180ce780 2024-03-30T00:51:58.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc717e00640 1 -- 172.21.15.66:0/306558981 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7080ab620 con 0x7fc7180ce780 2024-03-30T00:51:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 -- 172.21.15.66:0/306558981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7180ce780 msgr2=0x7fc7180cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 --2- 172.21.15.66:0/306558981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7180ce780 0x7fc7180cc910 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc7080097c0 tx=0x7fc70801c300 comp rx=0 tx=0).stop 2024-03-30T00:51:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 -- 172.21.15.66:0/306558981 shutdown_connections 2024-03-30T00:51:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 --2- 172.21.15.66:0/306558981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7180ce780 0x7fc7180cc910 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 -- 172.21.15.66:0/306558981 >> 172.21.15.66:0/306558981 conn(0x7fc718069f60 msgr2=0x7fc71806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 -- 172.21.15.66:0/306558981 shutdown_connections 2024-03-30T00:51:58.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 -- 172.21.15.66:0/306558981 wait complete. 2024-03-30T00:51:58.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 Processor -- start 2024-03-30T00:51:58.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 -- start start 2024-03-30T00:51:58.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc7180cd5e0 0x7fc718099b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7180ce780 0x7fc718098c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7180ceb60 0x7fc7180c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180c8220 con 0x7fc7180ce780 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7181f61d0 con 0x7fc7180ceb60 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.223+0000 7fc71f400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7181f64b0 con 0x7fc7180cd5e0 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc70fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7180ce780 0x7fc718098c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc70fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7180ce780 0x7fc718098c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49020/0 (socket says 172.21.15.66:49020) 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc70fe00640 1 -- 172.21.15.66:0/2312097162 learned_addr learned my addr 172.21.15.66:0/2312097162 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc71ca00640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc7180cd5e0 0x7fc718099b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc71d400640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7180ceb60 0x7fc7180c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc70fe00640 1 -- 172.21.15.66:0/2312097162 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc7180cd5e0 msgr2=0x7fc718099b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc70fe00640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc7180cd5e0 0x7fc718099b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc70fe00640 1 -- 172.21.15.66:0/2312097162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7180ceb60 msgr2=0x7fc7180c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc70fe00640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7180ceb60 0x7fc7180c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc70fe00640 1 -- 172.21.15.66:0/2312097162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc708009420 con 0x7fc7180ce780 2024-03-30T00:51:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc70fe00640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7180ce780 0x7fc718098c70 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fc70000c750 tx=0x7fc70000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc715600640 1 -- 172.21.15.66:0/2312097162 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700005ae0 con 0x7fc7180ce780 2024-03-30T00:51:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc715600640 1 -- 172.21.15.66:0/2312097162 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc700005c40 con 0x7fc7180ce780 2024-03-30T00:51:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc715600640 1 -- 172.21.15.66:0/2312097162 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7000072b0 con 0x7fc7180ce780 2024-03-30T00:51:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc71f400640 1 -- 172.21.15.66:0/2312097162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc718098190 con 0x7fc7180ce780 2024-03-30T00:51:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc71f400640 1 -- 172.21.15.66:0/2312097162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc718098750 con 0x7fc7180ce780 2024-03-30T00:51:58.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc71f400640 1 -- 172.21.15.66:0/2312097162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc718098940 con 0x7fc7180ce780 2024-03-30T00:51:58.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc715600640 1 -- 172.21.15.66:0/2312097162 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc700007410 con 0x7fc7180ce780 2024-03-30T00:51:58.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc715600640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc6f40666c0 0x7fc6f4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.227+0000 7fc715600640 1 -- 172.21.15.66:0/2312097162 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fc7000836b0 con 0x7fc7180ce780 2024-03-30T00:51:58.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.231+0000 7fc71ca00640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc6f40666c0 0x7fc6f4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.231+0000 7fc71ca00640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc6f40666c0 0x7fc6f4068b80 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc7080ad000 tx=0x7fc7080a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:58.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.231+0000 7fc715600640 1 -- 172.21.15.66:0/2312097162 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc700050c90 con 0x7fc7180ce780 2024-03-30T00:51:58.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.391+0000 7fc71f400640 1 -- 172.21.15.66:0/2312097162 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7180cde20 con 0x7fc6f40666c0 2024-03-30T00:51:58.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc715600640 1 -- 172.21.15.66:0/2312097162 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc708004020 con 0x7fc6f40666c0 2024-03-30T00:51:58.402 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 7 to deep-scrub 2024-03-30T00:51:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 -- 172.21.15.66:0/2312097162 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc6f40666c0 msgr2=0x7fc6f4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc6f40666c0 0x7fc6f4068b80 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc7080ad000 tx=0x7fc7080a6000 comp rx=0 tx=0).stop 2024-03-30T00:51:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 -- 172.21.15.66:0/2312097162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7180ce780 msgr2=0x7fc718098c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7180ce780 0x7fc718098c70 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fc70000c750 tx=0x7fc70000cc10 comp rx=0 tx=0).stop 2024-03-30T00:51:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 -- 172.21.15.66:0/2312097162 shutdown_connections 2024-03-30T00:51:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7180ceb60 0x7fc7180c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fc6f40666c0 0x7fc6f4068b80 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7180ce780 0x7fc718098c70 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 --2- 172.21.15.66:0/2312097162 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc7180cd5e0 0x7fc718099b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 -- 172.21.15.66:0/2312097162 >> 172.21.15.66:0/2312097162 conn(0x7fc718069f60 msgr2=0x7fc7180adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 -- 172.21.15.66:0/2312097162 shutdown_connections 2024-03-30T00:51:58.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.395+0000 7fc71f400640 1 -- 172.21.15.66:0/2312097162 wait complete. 2024-03-30T00:51:58.423 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:51:58.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fcce6000640 1 Processor -- start 2024-03-30T00:51:58.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fcce6000640 1 -- start start 2024-03-30T00:51:58.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fcce6000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcce00cda90 0x7fcce00ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fcce6000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce00cd3e0 con 0x7fcce00cde70 2024-03-30T00:51:58.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fcce6000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce015a990 con 0x7fcce015a370 2024-03-30T00:51:58.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fcce6000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00b1b80 con 0x7fcce00cda90 2024-03-30T00:51:58.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fccd6a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fcce00cde70 0x7fcce0159c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:35390/0 (socket says 172.21.15.66:35390) 2024-03-30T00:51:58.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fccd6a00640 1 -- 172.21.15.66:0/4021732055 learned_addr learned my addr 172.21.15.66:0/4021732055 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T00:51:58.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fccdf400640 1 --2- 172.21.15.66:0/4021732055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcce00cda90 0x7fcce00ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fccdea00640 1 -- 172.21.15.66:0/4021732055 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500686883 0 0) 0x7fcce00cd3e0 con 0x7fcce00cde70 2024-03-30T00:51:58.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fccdea00640 1 -- 172.21.15.66:0/4021732055 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccc4003530 con 0x7fcce00cde70 2024-03-30T00:51:58.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fccdea00640 1 -- 172.21.15.66:0/4021732055 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914833128 0 0) 0x7fcce015a990 con 0x7fcce015a370 2024-03-30T00:51:58.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fccdea00640 1 -- 172.21.15.66:0/4021732055 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce00cd3e0 con 0x7fcce015a370 2024-03-30T00:51:58.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fccdf400640 1 -- 172.21.15.66:0/4021732055 >> v1:172.21.15.156:6789/0 conn(0x7fcce015a370 legacy=0x7fcce0156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fccdf400640 1 -- 172.21.15.66:0/4021732055 >> v1:172.21.15.66:6789/0 conn(0x7fcce00cde70 legacy=0x7fcce0159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.555+0000 7fccdf400640 1 -- 172.21.15.66:0/4021732055 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce00b11e0 con 0x7fcce00cda90 2024-03-30T00:51:58.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.559+0000 7fccdf400640 1 --2- 172.21.15.66:0/4021732055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcce00cda90 0x7fcce00ce520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fccd00097f0 tx=0x7fccd001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e739204a920d3459 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:58.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.559+0000 7fccdea00640 1 -- 172.21.15.66:0/4021732055 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd00ab600 con 0x7fcce00cda90 2024-03-30T00:51:58.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.559+0000 7fccdea00640 1 -- 172.21.15.66:0/4021732055 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd00a7050 con 0x7fcce00cda90 2024-03-30T00:51:58.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.559+0000 7fccdea00640 1 -- 172.21.15.66:0/4021732055 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd00ab980 con 0x7fcce00cda90 2024-03-30T00:51:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.559+0000 7fcce6000640 1 -- 172.21.15.66:0/4021732055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcce00cda90 msgr2=0x7fcce00ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.559+0000 7fcce6000640 1 --2- 172.21.15.66:0/4021732055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcce00cda90 0x7fcce00ce520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fccd00097f0 tx=0x7fccd001c450 comp rx=0 tx=0).stop 2024-03-30T00:51:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.559+0000 7fcce6000640 1 -- 172.21.15.66:0/4021732055 shutdown_connections 2024-03-30T00:51:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.559+0000 7fcce6000640 1 --2- 172.21.15.66:0/4021732055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcce00cda90 0x7fcce00ce520 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.559+0000 7fcce6000640 1 -- 172.21.15.66:0/4021732055 >> 172.21.15.66:0/4021732055 conn(0x7fcce0069f60 msgr2=0x7fcce006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.559+0000 7fcce6000640 1 -- 172.21.15.66:0/4021732055 shutdown_connections 2024-03-30T00:51:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.559+0000 7fcce6000640 1 -- 172.21.15.66:0/4021732055 wait complete. 2024-03-30T00:51:58.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fcce6000640 1 Processor -- start 2024-03-30T00:51:58.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fcce6000640 1 -- start start 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fcce6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcce00cda90 0x7fcce00c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fcce6000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcce00cde70 0x7fcce00acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fcce6000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcce015a370 0x7fcce0159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fcce6000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce0158de0 con 0x7fcce00cda90 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fcce6000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00ad300 con 0x7fcce015a370 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fcce6000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00c4910 con 0x7fcce00cde70 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccdf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcce00cda90 0x7fcce00c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccdf400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcce00cda90 0x7fcce00c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49024/0 (socket says 172.21.15.66:49024) 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccdf400640 1 -- 172.21.15.66:0/2385173449 learned_addr learned my addr 172.21.15.66:0/2385173449 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccd6a00640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcce00cde70 0x7fcce00acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccdfe00640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcce015a370 0x7fcce0159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccdf400640 1 -- 172.21.15.66:0/2385173449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcce00cde70 msgr2=0x7fcce00acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccdf400640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcce00cde70 0x7fcce00acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccdf400640 1 -- 172.21.15.66:0/2385173449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcce015a370 msgr2=0x7fcce0159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccdf400640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcce015a370 0x7fcce0159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccdf400640 1 -- 172.21.15.66:0/2385173449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd0009420 con 0x7fcce00cda90 2024-03-30T00:51:58.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccdf400640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcce00cda90 0x7fcce00c42a0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fccd0019060 tx=0x7fccd00191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:58.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccd7e00640 1 -- 172.21.15.66:0/2385173449 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd00b4410 con 0x7fcce00cda90 2024-03-30T00:51:58.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccd7e00640 1 -- 172.21.15.66:0/2385173449 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd00b4590 con 0x7fcce00cda90 2024-03-30T00:51:58.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fccd7e00640 1 -- 172.21.15.66:0/2385173449 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd00b4940 con 0x7fcce00cda90 2024-03-30T00:51:58.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fcce6000640 1 -- 172.21.15.66:0/2385173449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce0158790 con 0x7fcce00cda90 2024-03-30T00:51:58.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fcce6000640 1 -- 172.21.15.66:0/2385173449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce015dd90 con 0x7fcce00cda90 2024-03-30T00:51:58.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.563+0000 7fcce6000640 1 -- 172.21.15.66:0/2385173449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce0066c30 con 0x7fcce00cda90 2024-03-30T00:51:58.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.567+0000 7fccd7e00640 1 -- 172.21.15.66:0/2385173449 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fccd00b2040 con 0x7fcce00cda90 2024-03-30T00:51:58.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.567+0000 7fccd7e00640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccbc066690 0x7fccbc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:51:58.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.567+0000 7fccd7e00640 1 -- 172.21.15.66:0/2385173449 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fccd011dfe0 con 0x7fcce00cda90 2024-03-30T00:51:58.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.567+0000 7fccd6a00640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccbc066690 0x7fccbc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:51:58.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.567+0000 7fccd6a00640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccbc066690 0x7fccbc068b50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fcce00c87f0 tx=0x7fccc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:51:58.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.571+0000 7fccd7e00640 1 -- 172.21.15.66:0/2385173449 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fccd00eb450 con 0x7fcce00cda90 2024-03-30T00:51:58.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.731+0000 7fcce6000640 1 -- 172.21.15.66:0/2385173449 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcce00b1fa0 con 0x7fccbc066690 2024-03-30T00:51:58.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.731+0000 7fccd7e00640 1 -- 172.21.15.66:0/2385173449 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50687 (secure 0 0 0) 0x7fccc8001eb0 con 0x7fccbc066690 2024-03-30T00:51:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:51:58.741 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:51:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 -- 172.21.15.66:0/2385173449 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccbc066690 msgr2=0x7fccbc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccbc066690 0x7fccbc068b50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fcce00c87f0 tx=0x7fccc8009040 comp rx=0 tx=0).stop 2024-03-30T00:51:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 -- 172.21.15.66:0/2385173449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcce00cda90 msgr2=0x7fcce00c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:51:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcce00cda90 0x7fcce00c42a0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fccd0019060 tx=0x7fccd00191f0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 -- 172.21.15.66:0/2385173449 shutdown_connections 2024-03-30T00:51:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcce015a370 0x7fcce0159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcce00cde70 0x7fcce00acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fccbc066690 0x7fccbc068b50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 --2- 172.21.15.66:0/2385173449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcce00cda90 0x7fcce00c42a0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:51:58.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 -- 172.21.15.66:0/2385173449 >> 172.21.15.66:0/2385173449 conn(0x7fcce0069f60 msgr2=0x7fcce006d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:51:58.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 -- 172.21.15.66:0/2385173449 shutdown_connections 2024-03-30T00:51:58.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:51:58.735+0000 7fcce6000640 1 -- 172.21.15.66:0/2385173449 wait complete. 2024-03-30T00:51:58.762 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-03-30T00:51:57.016342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262804,"kb_used_data":13864,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754711916,"statfs":{"total":773094113280,"available":772825001984,"internally_reserved":0,"allocated":14196736,"data_stored":7196752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.164190"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":279,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-03-30T00:51:48.857882+0000","last_change":"2024-03-30T00:39:41.626339+0000","last_active":"2024-03-30T00:51:48.857882+0000","last_peered":"2024-03-30T00:51:48.857882+0000","last_clean":"2024-03-30T00:51:48.857882+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:51:48.857882+0000","last_undegraded":"2024-03-30T00:51:48.857882+0000","last_fullsized":"2024-03-30T00:51:48.857882+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:39:39.606874+0000","last_clean_scrub_stamp":"2024-03-30T00:39:39.606874+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:33:04.429227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33284,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2220032,"data_stored":1342370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.65600000000000003},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84399999999999997,"5min":0.97599999999999998,"15min":1.0860000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66900000000000004,"15min":0.65300000000000002},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.98299999999999998,"15min":1.149},"last":0.749}]},{"osd":1,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.65800000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.03,"15min":1.03},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.46700000000000003,"5min":0.442,"15min":0.40400000000000003},"max":{"1min":0.81399999999999995,"5min":0.93000000000000005,"15min":1.3029999999999999},"last":0.71999999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.629,"5min":0.49399999999999999,"15min":0.45000000000000001},"max":{"1min":0.97499999999999998,"5min":1.085,"15min":1.6020000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.73699999999999999,"15min":0.75600000000000001},"min":{"1min":0.503,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93600000000000005,"5min":1.034,"15min":1.6619999999999999},"last":0.69999999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.755},"min":{"1min":0.47899999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":2.0150000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78000000000000003,"15min":0.79800000000000004},"min":{"1min":0.59999999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.96599999999999997,"5min":1.006,"15min":1.9490000000000001},"last":0.86099999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41099999999999998,"15min":0.37},"max":{"1min":0.92000000000000004,"5min":1.2,"15min":1.2},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.89700000000000002,"5min":0.95799999999999996,"15min":0.97899999999999998},"last":0.79300000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.45800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0649999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72599999999999998,"15min":0.74299999999999999},"min":{"1min":0.503,"5min":0.441,"15min":0.41699999999999998},"max":{"1min":0.98799999999999999,"5min":0.99199999999999999,"15min":1.026},"last":0.84599999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:50:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.54100000000000004,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.89200000000000002,"5min":0.94399999999999995,"15min":1.214},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.49099999999999999,"5min":0.45800000000000002,"15min":0.41199999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0309999999999999},"last":0.76800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69799999999999995,"15min":0.77900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.995,"15min":4.0490000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.48899999999999999,"5min":0.46400000000000002,"15min":0.27200000000000002},"max":{"1min":0.96199999999999997,"5min":1.091,"15min":1.47},"last":0.78500000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.78600000000000003},"min":{"1min":0.52000000000000002,"5min":0.438,"15min":0.36899999999999999},"max":{"1min":0.96199999999999997,"5min":1.204,"15min":4.3170000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72899999999999998,"15min":0.83099999999999996},"min":{"1min":0.52600000000000002,"5min":0.47199999999999998,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":1.1120000000000001,"15min":4.4160000000000004},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73599999999999999,"15min":0.83099999999999996},"min":{"1min":0.57899999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95099999999999996,"5min":1.0720000000000001,"15min":4.375},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70799999999999996,"15min":0.81999999999999995},"min":{"1min":0.47799999999999998,"5min":0.42999999999999999,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":1.0229999999999999,"15min":4.2960000000000003},"last":0.72999999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.749,"15min":0.84599999999999997},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":1.2390000000000001,"15min":4.3399999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71899999999999997,"15min":0.82699999999999996},"min":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.41999999999999998},"max":{"1min":0.91700000000000004,"5min":1.081,"15min":4.6020000000000003},"last":0.70799999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44600000000000001,"5min":0.39100000000000001,"15min":0.35999999999999999},"max":{"1min":0.871,"5min":1.0089999999999999,"15min":1.1870000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69599999999999995,"15min":0.72599999999999998},"min":{"1min":0.432,"5min":0.372,"15min":0.372},"max":{"1min":0.90500000000000003,"5min":0.98699999999999999,"15min":1.036},"last":0.83599999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61799999999999999,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91400000000000003,"5min":1.0449999999999999,"15min":1.0720000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68000000000000005,"15min":0.69099999999999995},"min":{"1min":0.46800000000000003,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.96799999999999997,"5min":1.0109999999999999,"15min":1.101},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66800000000000004,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.46200000000000002,"15min":0.45700000000000002},"max":{"1min":0.94999999999999996,"5min":1.103,"15min":1.103},"last":0.76200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":1.036,"5min":1.036,"15min":1.095},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67600000000000005,"15min":0.72399999999999998},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.38800000000000001},"max":{"1min":1.0089999999999999,"5min":1.0449999999999999,"15min":1.7569999999999999},"last":0.45700000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67200000000000004,"15min":0.71299999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.43099999999999999},"max":{"1min":0.88600000000000001,"5min":0.97399999999999998,"15min":1.7330000000000001},"last":0.746},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.44500000000000001,"5min":0.43099999999999999,"15min":0.42399999999999999},"max":{"1min":0.95099999999999996,"5min":0.98799999999999999,"15min":1.3260000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70799999999999996,"15min":0.71499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.024,"5min":1.131,"15min":1.131},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.47999999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.97899999999999998,"5min":1.0609999999999999,"15min":1.2789999999999999},"last":0.60499999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70099999999999996,"15min":0.74299999999999999},"min":{"1min":0.48399999999999999,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":1.024,"5min":1.0740000000000001,"15min":1.7},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72599999999999998,"15min":0.74099999999999999},"min":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.999,"5min":1.1160000000000001,"15min":1.181},"last":0.68300000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.314,"15min":0.309},"max":{"1min":0.746,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.72699999999999998},"min":{"1min":0.53500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":1.615},"last":0.79700000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.97399999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.624,"15min":0.66400000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35199999999999998},"max":{"1min":0.84499999999999997,"5min":0.93700000000000006,"15min":1.569},"last":0.77100000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.70899999999999996},"min":{"1min":0.41199999999999998,"5min":0.38900000000000001,"15min":0.377},"max":{"1min":0.90500000000000003,"5min":0.92200000000000004,"15min":1.0149999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.78700000000000003},"min":{"1min":0.53600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92300000000000004,"5min":1.167,"15min":1.383},"last":0.89600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.42099999999999999,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.97099999999999997,"15min":1.4199999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.67900000000000005},"min":{"1min":0.502,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.877,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.51300000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.47699999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.93999999999999995,"5min":0.95399999999999996,"15min":1.2909999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.71499999999999997},"min":{"1min":0.49199999999999999,"5min":0.46600000000000003,"15min":0.40699999999999997},"max":{"1min":0.86199999999999999,"5min":1.113,"15min":1.113},"last":0.70899999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.51600000000000001,"5min":0.48999999999999999,"15min":0.48499999999999999},"max":{"1min":0.96999999999999997,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.748},"min":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":0.98499999999999999,"5min":1.163,"15min":1.228},"last":0.80700000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.71799999999999997},"min":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.46800000000000003},"max":{"1min":0.91800000000000004,"5min":1.0800000000000001,"15min":1.27},"last":0.624},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.75600000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.44500000000000001},"max":{"1min":0.93200000000000005,"5min":0.99399999999999999,"15min":1.1739999999999999},"last":0.83799999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.629,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.329},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.65800000000000003,"15min":0.69299999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.33300000000000002},"max":{"1min":0.85899999999999999,"5min":0.99099999999999999,"15min":1.3819999999999999},"last":0.76500000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.2130000000000001},"last":0.87},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.68700000000000006},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39000000000000001},"max":{"1min":0.89200000000000002,"5min":0.95799999999999996,"15min":1.3220000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63,"15min":0.68200000000000005},"min":{"1min":0.438,"5min":0.42399999999999999,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":0.91500000000000004,"15min":1.6910000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72799999999999998,"15min":0.77000000000000002},"min":{"1min":0.51600000000000001,"5min":0.51400000000000001,"15min":0.46600000000000003},"max":{"1min":0.90200000000000002,"5min":0.97799999999999998,"15min":1.5009999999999999},"last":0.90200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.434,"15min":0.45300000000000001},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.68999999999999995,"5min":0.748,"15min":1.0529999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47799999999999998,"15min":0.50800000000000001},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.24199999999999999},"max":{"1min":0.72199999999999998,"5min":0.75,"15min":0.79100000000000004},"last":0.46000000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.5,"15min":0.504},"min":{"1min":0.42899999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74199999999999999,"5min":0.79100000000000004,"15min":0.80900000000000005},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.51900000000000002,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.32100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":1.2150000000000001},"last":0.503}]},{"osd":2,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.79900000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54100000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75700000000000001,"15min":0.76200000000000001},"min":{"1min":0.59399999999999997,"5min":0.47799999999999998,"15min":0.46300000000000002},"max":{"1min":0.96199999999999997,"5min":1.1259999999999999,"15min":1.2470000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.59099999999999997,"5min":0.51600000000000001,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":1.079,"15min":1.0860000000000001},"last":0.81100000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76900000000000002,"15min":0.77700000000000002},"min":{"1min":0.58099999999999996,"5min":0.54600000000000004,"15min":0.51100000000000001},"max":{"1min":1.0169999999999999,"5min":1.0329999999999999,"15min":1.125},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.62,"5min":0.52200000000000002,"15min":0.48599999999999999},"max":{"1min":0.93000000000000005,"5min":1.133,"15min":1.2390000000000001},"last":0.71399999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76500000000000001,"15min":0.79300000000000004},"min":{"1min":0.58099999999999996,"5min":0.51400000000000001,"15min":0.499},"max":{"1min":1.0049999999999999,"5min":1.091,"15min":1.226},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.1120000000000001,"15min":1.167},"last":0.85599999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.67400000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0549999999999999,"5min":1.1910000000000001,"15min":1.202},"last":0.873},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.81799999999999995,"15min":0.85299999999999998},"min":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.1639999999999999,"15min":1.3160000000000001},"last":0.83799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.339,"5min":1.339,"15min":4.0999999999999996},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.9239999999999999},"last":0.32200000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.36599999999999999,"5min":0.32800000000000001,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":3.1139999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":2.4159999999999999},"last":0.46200000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73599999999999999,"5min":0.86699999999999999,"15min":0.93000000000000005},"last":0.438},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.907},"last":0.48599999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.129,"5min":1.129,"15min":1.9470000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77400000000000002,"15min":0.81699999999999995},"min":{"1min":0.56599999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.972},"last":0.67100000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77900000000000003,"15min":0.81200000000000006},"min":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.3,"5min":1.3,"15min":2.4569999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77600000000000002,"15min":0.80700000000000005},"min":{"1min":0.60399999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":2.8039999999999998},"last":0.65900000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0980000000000001,"15min":2.8849999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.81299999999999994},"min":{"1min":0.56499999999999995,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.867},"last":0.68500000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:50:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78500000000000003,"15min":0.83199999999999996},"min":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.495},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":2.3690000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79000000000000004,"15min":0.83299999999999996},"min":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.0489999999999999,"15min":2.851},"last":0.69799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.27600000000000002,"5min":0.27200000000000002,"15min":0.253},"max":{"1min":0.66400000000000003,"5min":0.72199999999999998,"15min":0.80300000000000005},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.70799999999999996,"15min":0.76500000000000001},"last":0.44900000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46000000000000002,"15min":0.45700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29199999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.84099999999999997,"15min":0.89400000000000002},"last":0.40699999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.42599999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.72599999999999998,"5min":0.76600000000000001,"15min":0.86899999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.50600000000000001},"min":{"1min":0.34899999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.83199999999999996,"15min":0.83999999999999997},"last":0.49299999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.746,"15min":0.73399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.38800000000000001},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.753,"15min":0.755},"min":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.98199999999999998,"5min":0.999,"15min":1.2},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51600000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78300000000000003,"15min":0.77300000000000002},"min":{"1min":0.60499999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0329999999999999,"5min":1.129,"15min":1.2330000000000001},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.113,"5min":1.113,"15min":1.179},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.755,"15min":0.77100000000000002},"min":{"1min":0.56299999999999994,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.2150000000000001},"last":0.77000000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.57999999999999996,"5min":0.56200000000000006,"15min":0.48499999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.236},"last":0.76000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2220032,"data_stored":1342370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.442,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.29099999999999998,"15min":0.245},"max":{"1min":0.66200000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.311,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67500000000000004,"5min":0.77700000000000002,"15min":0.85099999999999998},"last":0.39200000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.30199999999999999},"max":{"1min":0.621,"5min":0.67100000000000004,"15min":1.02},"last":0.503},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70999999999999996,"5min":0.75,"15min":1.006},"last":0.435}]},{"osd":3,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.754,"15min":0.84799999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56699999999999995,"15min":0.57599999999999996},"min":{"1min":0.45200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.67600000000000005,"5min":0.83999999999999997,"15min":0.96899999999999997},"last":0.46500000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.622,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.5169999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81499999999999995,"15min":0.81399999999999995},"min":{"1min":0.61799999999999999,"5min":0.52300000000000002,"15min":0.46400000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.252},"last":0.68600000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.55900000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0029999999999999,"5min":1.075,"15min":1.3080000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.462},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"max":{"1min":1.377,"5min":1.377,"15min":1.5920000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79900000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.46999999999999997},"max":{"1min":1.139,"5min":1.1990000000000001,"15min":1.2230000000000001},"last":0.75900000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.80600000000000005,"15min":0.81299999999999994},"min":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.3600000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.79200000000000004,"15min":0.80100000000000005},"min":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.556},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:51:58.766 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-30T00:39:39.606874+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=30, tm_hour=0, tm_min=39, tm_sec=39, tm_wday=5, tm_yday=90, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=30, tm_hour=0, tm_min=51, tm_sec=53, tm_wday=5, tm_yday=90, tm_isdst=0) 2024-03-30T00:51:58.767 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-30T00:52:18.122 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:52:18.127 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:52:18.768 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:52:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f768ec00640 1 Processor -- start 2024-03-30T00:52:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f768ec00640 1 -- start start 2024-03-30T00:52:18.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f768ec00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f76880cda90 0x7f76880ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:18.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f768ec00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76880cde70 0x7f76880cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:18.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f768ec00640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7688159c60 0x7f768815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:18.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f768ec00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76880b1b80 con 0x7f76880cda90 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f768ec00640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768815a990 con 0x7f7688159c60 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f768ec00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76880b11e0 con 0x7f76880cde70 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f7687e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f76880cda90 0x7f76880ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f7687e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f76880cda90 0x7f76880ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44578/0 (socket says 172.21.15.66:44578) 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f7687e00640 1 -- 172.21.15.66:0/3441389030 learned_addr learned my addr 172.21.15.66:0/3441389030 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f768cc00640 1 --2- 172.21.15.66:0/3441389030 >> v2:172.21.15.156:3300/0 conn(0x7f7688159c60 0x7f768815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f767f400640 1 --2- 172.21.15.66:0/3441389030 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76880cde70 0x7f76880cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f7687e00640 1 -- 172.21.15.66:0/3441389030 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76880cde70 msgr2=0x7f76880cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f7687e00640 1 --2- 172.21.15.66:0/3441389030 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76880cde70 0x7f76880cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f7687e00640 1 -- 172.21.15.66:0/3441389030 >> v2:172.21.15.156:3300/0 conn(0x7f7688159c60 msgr2=0x7f768815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f7687e00640 1 --2- 172.21.15.66:0/3441389030 >> v2:172.21.15.156:3300/0 conn(0x7f7688159c60 0x7f768815a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.910+0000 7f7687e00640 1 -- 172.21.15.66:0/3441389030 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76880b14c0 con 0x7f76880cda90 2024-03-30T00:52:18.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f7687e00640 1 --2- 172.21.15.66:0/3441389030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76880cda90 0x7f76880ce520 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f76880724d0 tx=0x7f767801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84200ad896fc4d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:52:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f7687400640 1 -- 172.21.15.66:0/3441389030 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76780ab670 con 0x7f76880cda90 2024-03-30T00:52:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f7687400640 1 -- 172.21.15.66:0/3441389030 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76780ab7d0 con 0x7f76880cda90 2024-03-30T00:52:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f7687400640 1 -- 172.21.15.66:0/3441389030 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76780abb60 con 0x7f76880cda90 2024-03-30T00:52:18.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f768ec00640 1 -- 172.21.15.66:0/3441389030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76880cda90 msgr2=0x7f76880ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f768ec00640 1 --2- 172.21.15.66:0/3441389030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76880cda90 0x7f76880ce520 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f76880724d0 tx=0x7f767801c490 comp rx=0 tx=0).stop 2024-03-30T00:52:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f768ec00640 1 -- 172.21.15.66:0/3441389030 shutdown_connections 2024-03-30T00:52:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f768ec00640 1 --2- 172.21.15.66:0/3441389030 >> v2:172.21.15.156:3300/0 conn(0x7f7688159c60 0x7f768815a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f768ec00640 1 --2- 172.21.15.66:0/3441389030 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76880cde70 0x7f76880cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f768ec00640 1 --2- 172.21.15.66:0/3441389030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76880cda90 0x7f76880ce520 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f768ec00640 1 -- 172.21.15.66:0/3441389030 >> 172.21.15.66:0/3441389030 conn(0x7f7688069f60 msgr2=0x7f768806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:52:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f768ec00640 1 -- 172.21.15.66:0/3441389030 shutdown_connections 2024-03-30T00:52:18.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.914+0000 7f768ec00640 1 -- 172.21.15.66:0/3441389030 wait complete. 2024-03-30T00:52:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768ec00640 1 Processor -- start 2024-03-30T00:52:18.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768ec00640 1 -- start start 2024-03-30T00:52:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768ec00640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76880cda90 0x7f76880acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768ec00640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76880cde70 0x7f76881551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688159c60 0x7f76881584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76880ad0c0 con 0x7f7688159c60 2024-03-30T00:52:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768ec00640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76880abae0 con 0x7f76880cde70 2024-03-30T00:52:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768ec00640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76880abdc0 con 0x7f76880cda90 2024-03-30T00:52:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688159c60 0x7f76881584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688159c60 0x7f76881584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44594/0 (socket says 172.21.15.66:44594) 2024-03-30T00:52:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768cc00640 1 -- 172.21.15.66:0/3971489571 learned_addr learned my addr 172.21.15.66:0/3971489571 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:52:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f7687e00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76880cda90 0x7f76880acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:18.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f767f400640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76880cde70 0x7f76881551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:18.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768cc00640 1 -- 172.21.15.66:0/3971489571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76880cda90 msgr2=0x7f76880acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:18.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768cc00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76880cda90 0x7f76880acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:18.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768cc00640 1 -- 172.21.15.66:0/3971489571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76880cde70 msgr2=0x7f76881551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:18.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768cc00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76880cde70 0x7f76881551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:18.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768cc00640 1 -- 172.21.15.66:0/3971489571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76780094d0 con 0x7f7688159c60 2024-03-30T00:52:18.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768cc00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688159c60 0x7f76881584e0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f7680007e40 tx=0x7f768000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:52:18.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f7684c00640 1 -- 172.21.15.66:0/3971489571 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680014620 con 0x7f7688159c60 2024-03-30T00:52:18.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f7684c00640 1 -- 172.21.15.66:0/3971489571 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768000ee90 con 0x7f7688159c60 2024-03-30T00:52:18.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f7684c00640 1 -- 172.21.15.66:0/3971489571 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76800149e0 con 0x7f7688159c60 2024-03-30T00:52:18.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.918+0000 7f768ec00640 1 -- 172.21.15.66:0/3971489571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76880ac0a0 con 0x7f7688159c60 2024-03-30T00:52:18.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.922+0000 7f768ec00640 1 -- 172.21.15.66:0/3971489571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76880ac660 con 0x7f7688159c60 2024-03-30T00:52:18.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.922+0000 7f768ec00640 1 -- 172.21.15.66:0/3971489571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76880ac880 con 0x7f7688159c60 2024-03-30T00:52:18.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.922+0000 7f7684c00640 1 -- 172.21.15.66:0/3971489571 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7680014b40 con 0x7f7688159c60 2024-03-30T00:52:18.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.922+0000 7f7684c00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7664066690 0x7f7664068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:18.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.922+0000 7f7684c00640 1 -- 172.21.15.66:0/3971489571 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f76800848e0 con 0x7f7688159c60 2024-03-30T00:52:18.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.922+0000 7f7687e00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7664066690 0x7f7664068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:18.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.926+0000 7f7687e00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7664066690 0x7f7664068b50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f76780af000 tx=0x7f76780b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:52:18.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:18.926+0000 7f7684c00640 1 -- 172.21.15.66:0/3971489571 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7680051e10 con 0x7f7688159c60 2024-03-30T00:52:19.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.086+0000 7f768ec00640 1 -- 172.21.15.66:0/3971489571 --> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7688066c30 con 0x7f7664066690 2024-03-30T00:52:19.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.086+0000 7f7684c00640 1 -- 172.21.15.66:0/3971489571 <== mgr.4101 v2:172.21.15.66:6832/27382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50760 (secure 0 0 0) 0x7f7688066c30 con 0x7f7664066690 2024-03-30T00:52:19.098 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:52:19.099 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T00:52:19.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 -- 172.21.15.66:0/3971489571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7664066690 msgr2=0x7f7664068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7664066690 0x7f7664068b50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f76780af000 tx=0x7f76780b6040 comp rx=0 tx=0).stop 2024-03-30T00:52:19.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 -- 172.21.15.66:0/3971489571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688159c60 msgr2=0x7f76881584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688159c60 0x7f76881584e0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f7680007e40 tx=0x7f768000e4b0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 -- 172.21.15.66:0/3971489571 shutdown_connections 2024-03-30T00:52:19.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f7664066690 0x7f7664068b50 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688159c60 0x7f76881584e0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76880cde70 0x7f76881551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 --2- 172.21.15.66:0/3971489571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76880cda90 0x7f76880acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 -- 172.21.15.66:0/3971489571 >> 172.21.15.66:0/3971489571 conn(0x7f7688069f60 msgr2=0x7f768806d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:52:19.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 -- 172.21.15.66:0/3971489571 shutdown_connections 2024-03-30T00:52:19.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.090+0000 7f768ec00640 1 -- 172.21.15.66:0/3971489571 wait complete. 2024-03-30T00:52:19.119 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2024-03-30T00:52:19.021533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262792,"kb_used_data":13864,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754711928,"statfs":{"total":773094113280,"available":772825014272,"internally_reserved":0,"allocated":14196736,"data_stored":7196752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002885"},"pg_stats":[{"pgid":"1.0","version":"130'119","reported_seq":293,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-03-30T00:52:05.316228+0000","last_change":"2024-03-30T00:52:05.316333+0000","last_active":"2024-03-30T00:52:05.316228+0000","last_peered":"2024-03-30T00:52:05.316228+0000","last_clean":"2024-03-30T00:52:05.316228+0000","last_became_active":"2024-03-30T00:39:41.624868+0000","last_became_peered":"2024-03-30T00:39:41.624868+0000","last_unstale":"2024-03-30T00:52:05.316228+0000","last_undegraded":"2024-03-30T00:52:05.316228+0000","last_fullsized":"2024-03-30T00:52:05.316228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'119","last_scrub_stamp":"2024-03-30T00:52:05.316205+0000","last_deep_scrub":"130'119","last_deep_scrub_stamp":"2024-03-30T00:52:05.316205+0000","last_clean_scrub_stamp":"2024-03-30T00:52:05.316205+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:45:23.051817+0000","scrub_duration":0.0057846820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2220032,"data_stored":1342370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.40799999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.95899999999999996,"5min":0.97599999999999998,"15min":1.0860000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67700000000000005,"15min":0.65500000000000003},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92800000000000005,"5min":0.98299999999999998,"15min":1.149},"last":0.67500000000000004}]},{"osd":1,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68400000000000005,"15min":0.65900000000000003},"min":{"1min":0.48799999999999999,"5min":0.45700000000000002,"15min":0.436},"max":{"1min":0.89100000000000001,"5min":1.03,"15min":1.03},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.40400000000000003},"max":{"1min":0.86199999999999999,"5min":0.93000000000000005,"15min":1.3029999999999999},"last":0.64200000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.76000000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.45000000000000001},"max":{"1min":0.96499999999999997,"5min":1.0129999999999999,"15min":1.6020000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.55400000000000005,"5min":0.503,"15min":0.45400000000000001},"max":{"1min":0.89500000000000002,"5min":1.0249999999999999,"15min":1.6619999999999999},"last":0.71799999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.747},"min":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.0820000000000001,"15min":2.0150000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.78000000000000003,"15min":0.79300000000000004},"min":{"1min":0.61899999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.94099999999999995,"5min":1.006,"15min":1.9490000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.433,"5min":0.41099999999999998,"15min":0.37},"max":{"1min":0.86399999999999999,"5min":0.97299999999999998,"15min":1.2},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.88900000000000001,"5min":0.95799999999999996,"15min":0.97899999999999998},"last":0.46300000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67300000000000004,"15min":0.68899999999999995},"min":{"1min":0.50700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.999,"5min":1.0649999999999999,"15min":1.1519999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.72999999999999998,"15min":0.74099999999999999},"min":{"1min":0.52800000000000002,"5min":0.47299999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.026},"last":0.624}]},{"osd":6,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71299999999999997,"15min":0.71999999999999997},"min":{"1min":0.47699999999999998,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.214},"last":0.495},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68600000000000005,"15min":0.68899999999999995},"min":{"1min":0.46999999999999997,"5min":0.45800000000000002,"15min":0.41199999999999998},"max":{"1min":0.91200000000000003,"5min":1.0189999999999999,"15min":1.0309999999999999},"last":0.54200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:51:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69199999999999995,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.39800000000000002,"15min":0.35399999999999998},"max":{"1min":0.93100000000000005,"5min":0.995,"15min":4.0490000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.50600000000000001,"5min":0.46400000000000002,"15min":0.27200000000000002},"max":{"1min":0.88400000000000001,"5min":1.091,"15min":1.47},"last":0.69699999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:51:55 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.69699999999999995,"15min":0.76400000000000001},"min":{"1min":0.49099999999999999,"5min":0.438,"15min":0.36899999999999999},"max":{"1min":0.82199999999999995,"5min":1.0629999999999999,"15min":4.3170000000000002},"last":0.621},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71999999999999997,"15min":0.80700000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.878,"5min":1.1120000000000001,"15min":4.4160000000000004},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Mar 30 00:51:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.73099999999999998,"15min":0.80800000000000005},"min":{"1min":0.503,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89900000000000002,"5min":1.0720000000000001,"15min":4.375},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71299999999999997,"15min":0.79800000000000004},"min":{"1min":0.442,"5min":0.42999999999999999,"15min":0.38600000000000001},"max":{"1min":0.86499999999999999,"5min":1.0229999999999999,"15min":4.2960000000000003},"last":0.82999999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:51:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.82299999999999995},"min":{"1min":0.45800000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91700000000000004,"5min":1.1339999999999999,"15min":4.3399999999999999},"last":0.873},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.71499999999999997,"15min":0.80200000000000005},"min":{"1min":0.51900000000000002,"5min":0.49099999999999999,"15min":0.41999999999999998},"max":{"1min":0.91900000000000004,"5min":1.081,"15min":4.6020000000000003},"last":0.81399999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:51:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.63900000000000001,"15min":0.65300000000000002},"min":{"1min":0.48999999999999999,"5min":0.39100000000000001,"15min":0.35999999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.1870000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69799999999999995,"15min":0.72299999999999998},"min":{"1min":0.54200000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:51:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61199999999999999,"15min":0.64300000000000002},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94699999999999995,"5min":1.0449999999999999,"15min":1.0720000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.67500000000000004,"15min":0.68899999999999995},"min":{"1min":0.52800000000000002,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.97899999999999998,"5min":1.0109999999999999,"15min":1.101},"last":0.93400000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:51:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.54500000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96499999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":1.016},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69799999999999995,"15min":0.70899999999999996},"min":{"1min":0.47199999999999998,"5min":0.46200000000000002,"15min":0.45700000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.91200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.376},"max":{"1min":1.01,"5min":1.036,"15min":1.095},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67700000000000005,"15min":0.71199999999999997},"min":{"1min":0.45600000000000002,"5min":0.41099999999999998,"15min":0.38800000000000001},"max":{"1min":0.89300000000000002,"5min":1.0449999999999999,"15min":1.7569999999999999},"last":0.77800000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":0.97399999999999998,"15min":1.7330000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.51400000000000001,"5min":0.43099999999999999,"15min":0.42399999999999999},"max":{"1min":0.90800000000000003,"5min":0.98799999999999999,"15min":1.3260000000000001},"last":0.79500000000000004}]},{"osd":2,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.55200000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83499999999999996,"5min":1.131,"15min":1.131},"last":0.872},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70199999999999996,"15min":0.69799999999999995},"min":{"1min":0.5,"5min":0.45900000000000002,"15min":0.439},"max":{"1min":0.88400000000000001,"5min":1.0609999999999999,"15min":1.2789999999999999},"last":0.71299999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70999999999999996,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.82799999999999996,"5min":1.0740000000000001,"15min":1.7},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72999999999999998,"15min":0.73899999999999999},"min":{"1min":0.58099999999999996,"5min":0.50600000000000001,"15min":0.46999999999999997},"max":{"1min":0.85699999999999998,"5min":1.1160000000000001,"15min":1.181},"last":0.84399999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57599999999999996,"15min":0.59499999999999997},"min":{"1min":0.41799999999999998,"5min":0.314,"15min":0.309},"max":{"1min":0.92800000000000005,"5min":0.93999999999999995,"15min":1.0009999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68000000000000005,"15min":0.71899999999999997},"min":{"1min":0.47999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.97799999999999998,"15min":1.615},"last":0.61899999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60099999999999998,"15min":0.63500000000000001},"min":{"1min":0.47699999999999998,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.88600000000000001,"5min":0.94499999999999995,"15min":0.97399999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.63,"15min":0.65700000000000003},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35199999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.569},"last":0.51000000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:52:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.70099999999999996},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.377},"max":{"1min":0.91300000000000003,"5min":0.92200000000000004,"15min":1.0149999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.78100000000000003},"min":{"1min":0.54700000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91200000000000003,"5min":1.167,"15min":1.383},"last":0.69099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:51:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.68400000000000005},"min":{"1min":0.47899999999999998,"5min":0.42099999999999999,"15min":0.34899999999999998},"max":{"1min":0.89200000000000002,"5min":0.97099999999999997,"15min":1.4199999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71399999999999997,"15min":0.68100000000000005},"min":{"1min":0.48999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.1879999999999999},"last":0.80500000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:51:54 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.46999999999999997,"5min":0.441,"15min":0.441},"max":{"1min":0.91500000000000004,"5min":0.95399999999999996,"15min":1.2909999999999999},"last":0.751},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71599999999999997,"15min":0.71299999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.40699999999999997},"max":{"1min":0.83999999999999997,"5min":1.006,"15min":1.113},"last":0.76400000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:51:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72899999999999998,"15min":0.72199999999999998},"min":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48499999999999999},"max":{"1min":0.94799999999999995,"5min":1.0009999999999999,"15min":1.0600000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.56100000000000005,"5min":0.52200000000000002,"15min":0.46700000000000003},"max":{"1min":0.98199999999999998,"5min":0.98499999999999999,"15min":1.228},"last":0.84599999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:51:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73799999999999999,"15min":0.71699999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96699999999999997,"5min":1.0800000000000001,"15min":1.27},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.57099999999999995,"5min":0.52300000000000002,"15min":0.44500000000000001},"max":{"1min":0.879,"5min":0.99399999999999999,"15min":1.1739999999999999},"last":0.83099999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:51:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.65000000000000002,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":1.329},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65900000000000003,"15min":0.68799999999999994},"min":{"1min":0.435,"5min":0.371,"15min":0.33300000000000002},"max":{"1min":0.83099999999999996,"5min":0.99099999999999999,"15min":1.3819999999999999},"last":0.67600000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:51:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61299999999999999,"15min":0.64200000000000002},"min":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.87,"5min":0.90500000000000003,"15min":1.2130000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.63800000000000001,"15min":0.68000000000000005},"min":{"1min":0.46400000000000002,"5min":0.41799999999999998,"15min":0.39000000000000001},"max":{"1min":0.88400000000000001,"5min":0.95799999999999996,"15min":1.3220000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:51:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64200000000000002,"15min":0.67600000000000005},"min":{"1min":0.47399999999999998,"5min":0.42399999999999999,"15min":0.40799999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":1.6910000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.73599999999999999,"15min":0.76400000000000001},"min":{"1min":0.58099999999999996,"5min":0.51600000000000001,"15min":0.46600000000000003},"max":{"1min":0.90200000000000002,"5min":0.97799999999999998,"15min":1.5009999999999999},"last":0.68999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:52:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42199999999999999,"15min":0.44900000000000001},"min":{"1min":0.27400000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.65800000000000003,"5min":0.68999999999999995,"15min":1.0529999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.46300000000000002,"15min":0.499},"min":{"1min":0.316,"5min":0.28399999999999997,"15min":0.24199999999999999},"max":{"1min":0.57199999999999995,"5min":0.75,"15min":0.79100000000000004},"last":0.57199999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:52:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.49099999999999999,"15min":0.499},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.29999999999999999},"max":{"1min":0.628,"5min":0.74199999999999999,"15min":0.80900000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36099999999999999,"15min":0.32100000000000001},"max":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":1.2150000000000001},"last":0.68799999999999994}]},{"osd":2,"last update":"Sat Mar 30 00:52:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.5,"15min":0.503},"min":{"1min":0.35499999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.60699999999999998,"5min":0.77900000000000003,"15min":0.86499999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.39900000000000002,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.70999999999999996,"5min":0.81100000000000005,"15min":0.81699999999999995},"last":0.81299999999999994}]},{"osd":4,"last update":"Sat Mar 30 00:52:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73399999999999999,"15min":0.755},"min":{"1min":0.51900000000000002,"5min":0.47799999999999998,"15min":0.46300000000000002},"max":{"1min":0.93000000000000005,"5min":1.1259999999999999,"15min":1.2470000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.48499999999999999},"max":{"1min":0.83499999999999996,"5min":1.079,"15min":1.0860000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:52:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.755,"15min":0.77100000000000002},"min":{"1min":0.55700000000000005,"5min":0.54600000000000004,"15min":0.51100000000000001},"max":{"1min":0.92200000000000004,"5min":1.024,"15min":1.125},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.54200000000000004,"5min":0.52200000000000002,"15min":0.48599999999999999},"max":{"1min":0.996,"5min":1.133,"15min":1.2390000000000001},"last":0.86599999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:52:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.747,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.499},"max":{"1min":0.80200000000000005,"5min":1.0669999999999999,"15min":1.226},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77300000000000002,"15min":0.78400000000000003},"min":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.89300000000000002,"5min":1.1120000000000001,"15min":1.167},"last":0.79700000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:52:00 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.81399999999999995,"15min":0.82899999999999996},"min":{"1min":0.625,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.95599999999999996,"5min":1.1910000000000001,"15min":1.202},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.79000000000000004,"15min":0.83999999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.497},"max":{"1min":0.98399999999999999,"5min":1.1639999999999999,"15min":1.3160000000000001},"last":0.97099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:51:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44500000000000001,"15min":0.44900000000000001},"min":{"1min":0.29099999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66700000000000004,"5min":1.339,"15min":4.0999999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.495,"15min":0.501},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":1.0900000000000001,"15min":1.9239999999999999},"last":0.503}]},{"osd":1,"last update":"Sat Mar 30 00:51:58 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.499,"15min":0.51500000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70499999999999996,"5min":0.82199999999999995,"15min":3.1139999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72999999999999998,"5min":1.1830000000000001,"15min":2.4159999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:51:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.496,"15min":0.495},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69699999999999995,"5min":0.86699999999999999,"15min":0.93000000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.46100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.749,"5min":0.95599999999999996,"15min":1.907},"last":0.54000000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:51:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78100000000000003,"15min":0.79700000000000004},"min":{"1min":0.54500000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.155,"5min":1.155,"15min":1.9470000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.80800000000000005},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.2649999999999999,"15min":1.972},"last":0.65400000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:51:58 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79400000000000004,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.2010000000000001,"5min":1.3,"15min":2.4569999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":0.80300000000000005},"min":{"1min":0.55900000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":2.8039999999999998},"last":0.71799999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:51:58 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78000000000000003,"15min":0.80000000000000004},"min":{"1min":0.53900000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0489999999999999,"5min":1.0980000000000001,"15min":2.8849999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.58599999999999997,"5min":0.53700000000000003,"15min":0.52200000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.867},"last":0.81399999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:51:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.82399999999999995},"min":{"1min":0.59699999999999998,"5min":0.55600000000000005,"15min":0.495},"max":{"1min":1.0960000000000001,"5min":1.2270000000000001,"15min":2.3690000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80200000000000005,"15min":0.82799999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.335,"5min":1.335,"15min":2.851},"last":0.78800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1626112,"data_stored":752002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:52:09 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.43099999999999999,"15min":0.42799999999999999},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.253},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80300000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48199999999999998,"15min":0.47299999999999998},"min":{"1min":0.33000000000000002,"5min":0.28999999999999998,"15min":0.27000000000000002},"max":{"1min":0.64600000000000002,"5min":0.70799999999999996,"15min":0.76500000000000001},"last":0.751}]},{"osd":2,"last update":"Sat Mar 30 00:52:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45400000000000001,"15min":0.45500000000000002},"min":{"1min":0.33900000000000002,"5min":0.29399999999999998,"15min":0.29199999999999998},"max":{"1min":0.71199999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51100000000000001,"15min":0.51800000000000002},"min":{"1min":0.375,"5min":0.35199999999999998,"15min":0.33300000000000002},"max":{"1min":0.66800000000000004,"5min":0.69999999999999996,"15min":0.89400000000000002},"last":0.64800000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:52:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.39500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.86899999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.83199999999999996,"15min":0.83999999999999997},"last":0.68300000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:52:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.75800000000000001,"15min":0.73699999999999999},"min":{"1min":0.58099999999999996,"5min":0.48399999999999999,"15min":0.38800000000000001},"max":{"1min":1.0700000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.56299999999999994,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.2},"last":0.85599999999999998}]},{"osd":5,"last update":"Sat Mar 30 00:52:09 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.77600000000000002,"15min":0.76400000000000001},"min":{"1min":0.57399999999999995,"5min":0.52700000000000002,"15min":0.51600000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.77400000000000002},"min":{"1min":0.55400000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.94699999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:52:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.56499999999999995,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.77200000000000002,"15min":0.77300000000000002},"min":{"1min":0.59499999999999997,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.89100000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:52:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.79800000000000004},"min":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.55100000000000005},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80100000000000005,"15min":0.79500000000000004},"min":{"1min":0.60499999999999998,"5min":0.56200000000000006,"15min":0.48499999999999999},"max":{"1min":1.1240000000000001,"5min":1.1950000000000001,"15min":1.236},"last":0.86399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575003,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2220032,"data_stored":1342370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.43099999999999999,"15min":0.44600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.245},"max":{"1min":0.61699999999999999,"5min":0.71699999999999997,"15min":0.82999999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.48099999999999998,"15min":0.48699999999999999},"min":{"1min":0.30299999999999999,"5min":0.28000000000000003,"15min":0.25900000000000001},"max":{"1min":0.72199999999999998,"5min":0.77700000000000002,"15min":0.85099999999999998},"last":0.61299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48199999999999998,"15min":0.501},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.30199999999999999},"max":{"1min":0.56899999999999995,"5min":0.67100000000000004,"15min":1.02},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32200000000000001},"max":{"1min":0.71599999999999997,"5min":0.75,"15min":1.006},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.5,"15min":0.501},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32800000000000001},"max":{"1min":0.66500000000000004,"5min":0.754,"15min":0.84799999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.42599999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.68999999999999995,"5min":0.83999999999999997,"15min":0.96899999999999997},"last":0.70299999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98999999999999999,"5min":1.3939999999999999,"15min":1.5169999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.59399999999999997,"5min":0.52300000000000002,"15min":0.46400000000000002},"max":{"1min":0.94499999999999995,"5min":1.1739999999999999,"15min":1.252},"last":1.125}]},{"osd":5,"last update":"Sat Mar 30 00:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.012,"5min":1.075,"15min":1.3080000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89400000000000002,"5min":1.333,"15min":1.462},"last":0.96599999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.002,"5min":1.377,"15min":1.5920000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.79300000000000004,"15min":0.78700000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.89300000000000002,"5min":1.1990000000000001,"15min":1.2230000000000001},"last":1.1040000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80600000000000005,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.52300000000000002},"max":{"1min":0.96999999999999997,"5min":1.266,"15min":1.3600000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.79000000000000004,"15min":0.79600000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":0.92700000000000005,"5min":1.2210000000000001,"15min":1.556},"last":0.92400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:52:19.121 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-30T00:52:19.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4473800640 1 Processor -- start 2024-03-30T00:52:19.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4473800640 1 -- start start 2024-03-30T00:52:19.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4473800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f446c0ce520 0x7f446c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4473800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f446c0cabb0 0x7f446c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4473800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c06d070 con 0x7f446c15a640 2024-03-30T00:52:19.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4473800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446c0cbba0 con 0x7f446c0cabb0 2024-03-30T00:52:19.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4473800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446c0cee70 con 0x7f446c0ce520 2024-03-30T00:52:19.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4471800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f446c0cabb0 0x7f446c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4471800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f446c0cabb0 0x7f446c0cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44604/0 (socket says 172.21.15.66:44604) 2024-03-30T00:52:19.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4470e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f446c15a640 0x7f446c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:58272/0 (socket says 172.21.15.66:58272) 2024-03-30T00:52:19.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4471800640 1 -- 172.21.15.66:0/392338718 learned_addr learned my addr 172.21.15.66:0/392338718 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:52:19.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4463e00640 1 --2- 172.21.15.66:0/392338718 >> v2:172.21.15.156:3300/0 conn(0x7f446c0ce520 0x7f446c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4471800640 1 -- 172.21.15.66:0/392338718 >> v2:172.21.15.156:3300/0 conn(0x7f446c0ce520 msgr2=0x7f446c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:52:19.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4471800640 1 --2- 172.21.15.66:0/392338718 >> v2:172.21.15.156:3300/0 conn(0x7f446c0ce520 0x7f446c0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4471800640 1 -- 172.21.15.66:0/392338718 >> v1:172.21.15.66:6789/0 conn(0x7f446c15a640 legacy=0x7f446c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.250+0000 7f4471800640 1 -- 172.21.15.66:0/392338718 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0cb8c0 con 0x7f446c0cabb0 2024-03-30T00:52:19.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f4471800640 1 --2- 172.21.15.66:0/392338718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446c0cabb0 0x7f446c0cafe0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f446c0c88d0 tx=0x7f446401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74966ff6af043bc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:52:19.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f446be00640 1 -- 172.21.15.66:0/392338718 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44640a5880 con 0x7f446c0cabb0 2024-03-30T00:52:19.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f446be00640 1 -- 172.21.15.66:0/392338718 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44640a59e0 con 0x7f446c0cabb0 2024-03-30T00:52:19.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f446be00640 1 -- 172.21.15.66:0/392338718 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44640a5d70 con 0x7f446c0cabb0 2024-03-30T00:52:19.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f4473800640 1 -- 172.21.15.66:0/392338718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446c0cabb0 msgr2=0x7f446c0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f4473800640 1 --2- 172.21.15.66:0/392338718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446c0cabb0 0x7f446c0cafe0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f446c0c88d0 tx=0x7f446401c9c0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f4473800640 1 -- 172.21.15.66:0/392338718 shutdown_connections 2024-03-30T00:52:19.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f4473800640 1 --2- 172.21.15.66:0/392338718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446c0cabb0 0x7f446c0cafe0 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f4473800640 1 --2- 172.21.15.66:0/392338718 >> v2:172.21.15.156:3300/0 conn(0x7f446c0ce520 0x7f446c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f4473800640 1 -- 172.21.15.66:0/392338718 >> 172.21.15.66:0/392338718 conn(0x7f446c069f60 msgr2=0x7f446c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:52:19.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f4473800640 1 -- 172.21.15.66:0/392338718 shutdown_connections 2024-03-30T00:52:19.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.254+0000 7f4473800640 1 -- 172.21.15.66:0/392338718 wait complete. 2024-03-30T00:52:19.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4473800640 1 Processor -- start 2024-03-30T00:52:19.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4473800640 1 -- start start 2024-03-30T00:52:19.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4473800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446c0cabb0 0x7f446c097100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4473800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446c0ce520 0x7f446c14f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4473800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f446c15a640 0x7f446c14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4473800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c097640 con 0x7f446c0ce520 2024-03-30T00:52:19.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4473800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c070d70 con 0x7f446c0cabb0 2024-03-30T00:52:19.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4473800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c071050 con 0x7f446c15a640 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4463e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446c0ce520 0x7f446c14f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4463e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446c0ce520 0x7f446c14f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44620/0 (socket says 172.21.15.66:44620) 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4463e00640 1 -- 172.21.15.66:0/1911999932 learned_addr learned my addr 172.21.15.66:0/1911999932 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4470e00640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446c0cabb0 0x7f446c097100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4471800640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f446c15a640 0x7f446c14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4463e00640 1 -- 172.21.15.66:0/1911999932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f446c15a640 msgr2=0x7f446c14fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4463e00640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f446c15a640 0x7f446c14fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4463e00640 1 -- 172.21.15.66:0/1911999932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446c0cabb0 msgr2=0x7f446c097100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4463e00640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446c0cabb0 0x7f446c097100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4463e00640 1 -- 172.21.15.66:0/1911999932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464009b30 con 0x7f446c0ce520 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4463e00640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446c0ce520 0x7f446c14f7d0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f445400d800 tx=0x7f445400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4469600640 1 -- 172.21.15.66:0/1911999932 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445400b6c0 con 0x7f446c0ce520 2024-03-30T00:52:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4469600640 1 -- 172.21.15.66:0/1911999932 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445400b820 con 0x7f446c0ce520 2024-03-30T00:52:19.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4473800640 1 -- 172.21.15.66:0/1911999932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446c071330 con 0x7f446c0ce520 2024-03-30T00:52:19.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4469600640 1 -- 172.21.15.66:0/1911999932 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445400bb90 con 0x7f446c0ce520 2024-03-30T00:52:19.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.258+0000 7f4473800640 1 -- 172.21.15.66:0/1911999932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446c0718f0 con 0x7f446c0ce520 2024-03-30T00:52:19.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.262+0000 7f4473800640 1 -- 172.21.15.66:0/1911999932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446c071ab0 con 0x7f446c0ce520 2024-03-30T00:52:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.262+0000 7f4469600640 1 -- 172.21.15.66:0/1911999932 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f445400bcf0 con 0x7f446c0ce520 2024-03-30T00:52:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.262+0000 7f4469600640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44480666c0 0x7f4448068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.262+0000 7f4469600640 1 -- 172.21.15.66:0/1911999932 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f44540846c0 con 0x7f446c0ce520 2024-03-30T00:52:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.262+0000 7f4470e00640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44480666c0 0x7f4448068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.262+0000 7f4470e00640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44480666c0 0x7f4448068b80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f445c000f70 tx=0x7f445c007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:52:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.266+0000 7f4469600640 1 -- 172.21.15.66:0/1911999932 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4454051aa0 con 0x7f446c0ce520 2024-03-30T00:52:19.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.426+0000 7f4473800640 1 -- 172.21.15.66:0/1911999932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f446c066c30 con 0x7f446c0ce520 2024-03-30T00:52:19.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.430+0000 7f4469600640 1 -- 172.21.15.66:0/1911999932 <== mon.0 v2:172.21.15.66:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4454051aa0 con 0x7f446c0ce520 2024-03-30T00:52:19.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.430+0000 7f4469600640 1 -- 172.21.15.66:0/1911999932 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7f44540550c0 con 0x7f446c0ce520 2024-03-30T00:52:19.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 -- 172.21.15.66:0/1911999932 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44480666c0 msgr2=0x7f4448068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44480666c0 0x7f4448068b80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f445c000f70 tx=0x7f445c007cb0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 -- 172.21.15.66:0/1911999932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446c0ce520 msgr2=0x7f446c14f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446c0ce520 0x7f446c14f7d0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f445400d800 tx=0x7f445400dcc0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 -- 172.21.15.66:0/1911999932 shutdown_connections 2024-03-30T00:52:19.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f446c15a640 0x7f446c14fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7f44480666c0 0x7f4448068b80 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f446c0ce520 0x7f446c14f7d0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 --2- 172.21.15.66:0/1911999932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f446c0cabb0 0x7f446c097100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 -- 172.21.15.66:0/1911999932 >> 172.21.15.66:0/1911999932 conn(0x7f446c069f60 msgr2=0x7f446c159700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:52:19.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 -- 172.21.15.66:0/1911999932 shutdown_connections 2024-03-30T00:52:19.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.434+0000 7f4473800640 1 -- 172.21.15.66:0/1911999932 wait complete. 2024-03-30T00:52:19.464 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T00:52:19.464 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:52:19.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.594+0000 7fe011800640 1 Processor -- start 2024-03-30T00:52:19.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.594+0000 7fe011800640 1 -- start start 2024-03-30T00:52:19.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.594+0000 7fe011800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe00c157060 0x7fe00c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.594+0000 7fe011800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 0x7fe00c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.594+0000 7fe011800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe00c1609f0 0x7fe00c15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.594+0000 7fe011800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00c157440 con 0x7fe00c1609f0 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe011800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00c160ea0 con 0x7fe00c157060 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe011800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c06dc60 con 0x7fe00c1610c0 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe00c1609f0 0x7fe00c15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe00c1609f0 0x7fe00c15cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44630/0 (socket says 172.21.15.66:44630) 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00b400640 1 -- 172.21.15.66:0/868335097 learned_addr learned my addr 172.21.15.66:0/868335097 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe003e00640 1 --2- 172.21.15.66:0/868335097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 0x7fe00c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00aa00640 1 --2- 172.21.15.66:0/868335097 >> v2:172.21.15.156:3300/0 conn(0x7fe00c157060 0x7fe00c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00b400640 1 -- 172.21.15.66:0/868335097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 msgr2=0x7fe00c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00b400640 1 --2- 172.21.15.66:0/868335097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 0x7fe00c1604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00b400640 1 -- 172.21.15.66:0/868335097 >> v2:172.21.15.156:3300/0 conn(0x7fe00c157060 msgr2=0x7fe00c06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:52:19.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00b400640 1 --2- 172.21.15.66:0/868335097 >> v2:172.21.15.156:3300/0 conn(0x7fe00c157060 0x7fe00c06d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00b400640 1 -- 172.21.15.66:0/868335097 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c06df40 con 0x7fe00c1609f0 2024-03-30T00:52:19.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe003e00640 1 --2- 172.21.15.66:0/868335097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 0x7fe00c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:52:19.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00b400640 1 --2- 172.21.15.66:0/868335097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe00c1609f0 0x7fe00c15cec0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fe00c072710 tx=0x7fe00401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501f2c621cba531c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:52:19.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00a000640 1 -- 172.21.15.66:0/868335097 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0040a5640 con 0x7fe00c1609f0 2024-03-30T00:52:19.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00a000640 1 -- 172.21.15.66:0/868335097 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe0040a57a0 con 0x7fe00c1609f0 2024-03-30T00:52:19.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.598+0000 7fe00a000640 1 -- 172.21.15.66:0/868335097 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0040a5b50 con 0x7fe00c1609f0 2024-03-30T00:52:19.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 -- 172.21.15.66:0/868335097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe00c1609f0 msgr2=0x7fe00c15cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 --2- 172.21.15.66:0/868335097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe00c1609f0 0x7fe00c15cec0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fe00c072710 tx=0x7fe00401c8d0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 -- 172.21.15.66:0/868335097 shutdown_connections 2024-03-30T00:52:19.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 --2- 172.21.15.66:0/868335097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe00c1609f0 0x7fe00c15cec0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 --2- 172.21.15.66:0/868335097 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 0x7fe00c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 --2- 172.21.15.66:0/868335097 >> v2:172.21.15.156:3300/0 conn(0x7fe00c157060 0x7fe00c06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 -- 172.21.15.66:0/868335097 >> 172.21.15.66:0/868335097 conn(0x7fe00c071750 msgr2=0x7fe00c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:52:19.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 -- 172.21.15.66:0/868335097 shutdown_connections 2024-03-30T00:52:19.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 -- 172.21.15.66:0/868335097 wait complete. 2024-03-30T00:52:19.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 Processor -- start 2024-03-30T00:52:19.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 -- start start 2024-03-30T00:52:19.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe00c157060 0x7fe00c155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe00c1609f0 0x7fe00c0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 0x7fe00c0b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c14f710 con 0x7fe00c1609f0 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0b3c40 con 0x7fe00c157060 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe011800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0b3f20 con 0x7fe00c1610c0 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe003e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe00c1609f0 0x7fe00c0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe003e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe00c1609f0 0x7fe00c0b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44632/0 (socket says 172.21.15.66:44632) 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe003e00640 1 -- 172.21.15.66:0/875873969 learned_addr learned my addr 172.21.15.66:0/875873969 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe00aa00640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe00c157060 0x7fe00c155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe00b400640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 0x7fe00c0b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe003e00640 1 -- 172.21.15.66:0/875873969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 msgr2=0x7fe00c0b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe003e00640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 0x7fe00c0b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe003e00640 1 -- 172.21.15.66:0/875873969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe00c157060 msgr2=0x7fe00c155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe003e00640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe00c157060 0x7fe00c155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe003e00640 1 -- 172.21.15.66:0/875873969 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe004009b30 con 0x7fe00c1609f0 2024-03-30T00:52:19.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe00aa00640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe00c157060 0x7fe00c155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:52:19.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.602+0000 7fe00b400640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 0x7fe00c0b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:52:19.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.606+0000 7fe003e00640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe00c1609f0 0x7fe00c0b30a0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e810 tx=0x7fdff400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:52:19.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.606+0000 7fe002a00640 1 -- 172.21.15.66:0/875873969 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff400c6d0 con 0x7fe00c1609f0 2024-03-30T00:52:19.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.606+0000 7fe002a00640 1 -- 172.21.15.66:0/875873969 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdff400c830 con 0x7fe00c1609f0 2024-03-30T00:52:19.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.606+0000 7fe011800640 1 -- 172.21.15.66:0/875873969 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c0b4200 con 0x7fe00c1609f0 2024-03-30T00:52:19.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.606+0000 7fe002a00640 1 -- 172.21.15.66:0/875873969 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff400cc00 con 0x7fe00c1609f0 2024-03-30T00:52:19.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.606+0000 7fe011800640 1 -- 172.21.15.66:0/875873969 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c151530 con 0x7fe00c1609f0 2024-03-30T00:52:19.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.606+0000 7fe011800640 1 -- 172.21.15.66:0/875873969 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00c066c30 con 0x7fe00c1609f0 2024-03-30T00:52:19.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.606+0000 7fe002a00640 1 -- 172.21.15.66:0/875873969 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdff4013410 con 0x7fe00c1609f0 2024-03-30T00:52:19.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.606+0000 7fe002a00640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfe8066690 0x7fdfe8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:52:19.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.606+0000 7fe002a00640 1 -- 172.21.15.66:0/875873969 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fdff4083fe0 con 0x7fe00c1609f0 2024-03-30T00:52:19.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.606+0000 7fe00aa00640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfe8066690 0x7fdfe8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:52:19.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.610+0000 7fe00aa00640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfe8066690 0x7fdfe8068b50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009720 tx=0x7fdffc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:52:19.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.610+0000 7fe002a00640 1 -- 172.21.15.66:0/875873969 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdff4051350 con 0x7fe00c1609f0 2024-03-30T00:52:19.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.806+0000 7fe011800640 1 -- 172.21.15.66:0/875873969 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe00c06b510 con 0x7fe00c1609f0 2024-03-30T00:52:19.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe002a00640 1 -- 172.21.15.66:0/875873969 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+3535 (secure 0 0 0) 0x7fdff4021020 con 0x7fe00c1609f0 2024-03-30T00:52:19.818 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T00:52:19.818 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":110,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3464427470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3464427470},{"type":"v1","addr":"172.21.15.156:6835","nonce":3464427470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/806762639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":806762639},{"type":"v1","addr":"172.21.15.66:6837","nonce":806762639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":8767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2988832483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2988832483},{"type":"v1","addr":"172.21.15.156:6833","nonce":2988832483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/3353356197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3353356197},{"type":"v1","addr":"172.21.15.66:6836","nonce":3353356197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":110}],"filesystems":[]} 2024-03-30T00:52:19.818 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 110 2024-03-30T00:52:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 -- 172.21.15.66:0/875873969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfe8066690 msgr2=0x7fdfe8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfe8066690 0x7fdfe8068b50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009720 tx=0x7fdffc008040 comp rx=0 tx=0).stop 2024-03-30T00:52:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 -- 172.21.15.66:0/875873969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe00c1609f0 msgr2=0x7fe00c0b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:52:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe00c1609f0 0x7fe00c0b30a0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e810 tx=0x7fdff400ecd0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 -- 172.21.15.66:0/875873969 shutdown_connections 2024-03-30T00:52:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe00c1610c0 0x7fe00c0b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.66:6832/27382,v1:172.21.15.66:6833/27382] conn(0x7fdfe8066690 0x7fdfe8068b50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe00c1609f0 0x7fe00c0b30a0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 --2- 172.21.15.66:0/875873969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe00c157060 0x7fe00c155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:52:19.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 -- 172.21.15.66:0/875873969 >> 172.21.15.66:0/875873969 conn(0x7fe00c071750 msgr2=0x7fe00c15aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:52:19.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 -- 172.21.15.66:0/875873969 shutdown_connections 2024-03-30T00:52:19.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T00:52:19.810+0000 7fe011800640 1 -- 172.21.15.66:0/875873969 wait complete. 2024-03-30T00:52:19.840 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-30T00:52:19.840 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-30T00:52:19.840 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T00:52:19.840 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:19.851 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T00:52:19.852 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T00:52:19.852 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:19.909 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T00:52:19.909 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T00:52:19.909 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:19.924 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T00:52:19.924 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T00:52:19.924 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:19.977 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T00:52:19.977 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-30T00:52:19.977 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-30T00:52:19.977 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:20.076 INFO:tasks.ceph.osd.0:Stopped 2024-03-30T00:52:20.076 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-30T00:52:20.076 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:22.670 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-03-30T00:52:26.178 INFO:tasks.ceph.osd.1:Stopped 2024-03-30T00:52:26.178 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-30T00:52:26.178 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:26.266 INFO:tasks.ceph.osd.2:Stopped 2024-03-30T00:52:26.266 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-30T00:52:26.266 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:26.357 INFO:tasks.ceph.osd.3:Stopped 2024-03-30T00:52:26.357 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-30T00:52:26.357 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:26.421 INFO:tasks.ceph.osd.4:Stopped 2024-03-30T00:52:26.422 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-30T00:52:26.422 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:26.514 INFO:tasks.ceph.osd.5:Stopped 2024-03-30T00:52:26.514 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-30T00:52:26.514 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:26.597 INFO:tasks.ceph.osd.6:Stopped 2024-03-30T00:52:26.597 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-30T00:52:26.597 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:26.669 INFO:tasks.ceph.osd.7:Stopped 2024-03-30T00:52:26.669 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-30T00:52:26.669 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-30T00:52:26.669 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:26.730 INFO:tasks.ceph.mgr.y:Stopped 2024-03-30T00:52:26.730 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-30T00:52:26.730 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:26.783 INFO:tasks.ceph.mgr.x:Stopped 2024-03-30T00:52:26.784 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-30T00:52:26.784 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-30T00:52:26.784 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:26.804 INFO:tasks.ceph.mon.a:Stopped 2024-03-30T00:52:26.804 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-30T00:52:26.804 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:26.822 INFO:tasks.ceph.mon.b:Stopped 2024-03-30T00:52:26.822 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-30T00:52:26.822 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:52:26.880 INFO:tasks.ceph.mon.c:Stopped 2024-03-30T00:52:26.880 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-30T00:52:26.881 DEBUG:teuthology.orchestra.run.smithi066:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-30T00:52:26.913 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:44:42.408669+0000 mds.c (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-03-30T00:52:26.914 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-03-30T00:52:26.915 DEBUG:teuthology.orchestra.run.smithi066:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-30T00:52:26.994 DEBUG:teuthology.orchestra.run.smithi066:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-30T00:52:27.073 INFO:teuthology.orchestra.run.smithi066.stdout:2024-03-30T00:46:41.436638+0000 mds.a (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-30T00:52:27.074 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:52:27.074 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-30T00:52:27.542 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:52:27.542 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-30T00:52:27.671 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-03-30T00:52:27.767 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:52:27.768 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-30T00:52:27.955 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:52:27.955 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-30T00:52:28.130 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:52:28.130 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-30T00:52:28.470 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:52:28.471 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-30T00:52:28.638 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:52:28.639 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-30T00:52:28.830 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:52:28.830 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-30T00:52:28.996 INFO:tasks.ceph:Archiving mon data... 2024-03-30T00:52:28.997 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246/data/mon.a.tgz 2024-03-30T00:52:28.998 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-03-30T00:52:29.007 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.yzP3zXFSVQ 2024-03-30T00:52:29.007 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.yzP3zXFSVQ 2024-03-30T00:52:29.154 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.yzP3zXFSVQ 2024-03-30T00:52:29.234 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.yzP3zXFSVQ is 412KB 2024-03-30T00:52:29.300 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.yzP3zXFSVQ 2024-03-30T00:52:29.308 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246/data/mon.b.tgz 2024-03-30T00:52:29.308 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2024-03-30T00:52:29.317 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.ncYC4g6lOc 2024-03-30T00:52:29.317 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ncYC4g6lOc 2024-03-30T00:52:29.464 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.ncYC4g6lOc 2024-03-30T00:52:29.540 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.ncYC4g6lOc is 421KB 2024-03-30T00:52:29.607 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.ncYC4g6lOc 2024-03-30T00:52:29.615 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246/data/mon.c.tgz 2024-03-30T00:52:29.615 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2024-03-30T00:52:29.664 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.zOrpZDzBml 2024-03-30T00:52:29.664 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.zOrpZDzBml 2024-03-30T00:52:29.806 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.zOrpZDzBml 2024-03-30T00:52:29.880 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.zOrpZDzBml is 421KB 2024-03-30T00:52:29.943 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.zOrpZDzBml 2024-03-30T00:52:29.950 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-30T00:52:29.951 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-30T00:52:29.953 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-30T00:52:30.012 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-03-30T00:52:30.013 DEBUG:teuthology.orchestra.run.smithi066:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-30T00:52:30.020 DEBUG:teuthology.orchestra.run.smithi156:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-30T00:52:30.054 INFO:teuthology.orchestra.run.smithi066.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-30T00:52:30.088 INFO:teuthology.orchestra.run.smithi156.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-30T00:52:30.095 INFO:tasks.ceph:Archiving crash dumps... 2024-03-30T00:52:30.097 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246/remote/smithi066/crash 2024-03-30T00:52:30.098 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-30T00:52:30.130 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246/remote/smithi156/crash 2024-03-30T00:52:30.131 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-30T00:52:30.154 INFO:tasks.ceph:Shutting down logrotate 2024-03-30T00:52:30.154 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:52:30.176 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T00:52:30.244 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-30T00:52:30.265 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-30T00:52:30.283 INFO:tasks.ceph:Compressing logs... 2024-03-30T00:52:30.283 DEBUG:teuthology.orchestra.run.smithi066:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T00:52:30.308 DEBUG:teuthology.orchestra.run.smithi156:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T00:52:38.589 INFO:tasks.ceph:Archiving logs... 2024-03-30T00:52:38.589 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246/remote/smithi066/log 2024-03-30T00:52:38.590 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T00:52:39.985 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246/remote/smithi156/log 2024-03-30T00:52:39.986 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T00:52:40.703 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-30T00:52:40.714 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-30T00:52:40.714 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T00:52:40.718 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T00:52:40.796 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-30T00:52:40.804 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-30T00:52:40.805 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T00:52:40.808 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T00:52:40.821 INFO:teuthology.orchestra.run.smithi066.stdout: remote refid st t when poll reach delay offset jitter 2024-03-30T00:52:40.821 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================== 2024-03-30T00:52:40.821 INFO:teuthology.orchestra.run.smithi066.stdout:*hv01.front.sepi 44.4.53.2 2 u 8 64 377 0.071 -0.178 0.594 2024-03-30T00:52:40.821 INFO:teuthology.orchestra.run.smithi066.stdout:+hv02.front.sepi 96.245.170.99 3 u 8 64 377 0.083 -0.442 0.696 2024-03-30T00:52:40.821 INFO:teuthology.orchestra.run.smithi066.stdout:+hv03.front.sepi 74.6.168.72 3 u 8 64 377 0.080 +1.526 0.807 2024-03-30T00:52:40.821 INFO:teuthology.orchestra.run.smithi066.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-03-30T00:52:40.822 INFO:teuthology.orchestra.run.smithi156.stdout: remote refid st t when poll reach delay offset jitter 2024-03-30T00:52:40.822 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================== 2024-03-30T00:52:40.823 INFO:teuthology.orchestra.run.smithi156.stdout:*hv01.front.sepi 44.4.53.2 2 u 11 64 377 0.076 -2.532 1.139 2024-03-30T00:52:40.823 INFO:teuthology.orchestra.run.smithi156.stdout:+hv02.front.sepi 96.245.170.99 3 u 1 64 377 0.083 -4.725 1.327 2024-03-30T00:52:40.823 INFO:teuthology.orchestra.run.smithi156.stdout:+hv03.front.sepi 74.6.168.72 3 u 1 64 377 0.132 -1.941 0.749 2024-03-30T00:52:40.823 INFO:teuthology.orchestra.run.smithi156.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-03-30T00:52:40.824 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-30T00:52:40.833 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-30T00:52:40.833 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-30T00:52:40.843 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-30T00:52:40.852 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-30T00:52:40.862 INFO:teuthology.task.internal:Duration was 1464.258804 seconds 2024-03-30T00:52:40.863 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-30T00:52:40.873 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-30T00:52:40.873 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T00:52:40.876 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T00:52:40.967 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-30T00:52:40.968 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T00:52:40.968 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T00:52:40.988 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2024-03-30T00:52:40.989 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T00:52:41.009 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-30T00:52:41.009 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T00:52:41.036 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T00:52:41.092 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-30T00:52:41.092 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T00:52:41.096 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T00:52:41.216 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-30T00:52:41.232 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-30T00:52:41.232 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T00:52:41.252 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T00:52:41.272 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-30T00:52:41.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T00:52:41.300 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T00:52:41.315 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-03-30T00:52:41.333 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2024-03-30T00:52:41.350 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T00:52:41.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:52:41.380 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T00:52:41.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:52:41.402 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-30T00:52:41.413 INFO:teuthology.task.internal:Transferring archived files... 2024-03-30T00:52:41.414 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246/remote/smithi066 2024-03-30T00:52:41.414 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T00:52:41.449 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246/remote/smithi156 2024-03-30T00:52:41.449 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T00:52:41.475 INFO:teuthology.task.internal:Removing archive directory... 2024-03-30T00:52:41.475 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T00:52:41.488 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T00:52:41.524 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-30T00:52:41.538 INFO:teuthology.task.internal:Not uploading archives. 2024-03-30T00:52:41.538 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-30T00:52:41.588 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-30T00:52:41.589 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T00:52:41.594 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T00:52:41.599 INFO:teuthology.orchestra.run.smithi066.stdout: 658116 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 00:52 /home/ubuntu/cephtest 2024-03-30T00:52:41.603 INFO:teuthology.orchestra.run.smithi156.stdout: 658118 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 00:52 /home/ubuntu/cephtest 2024-03-30T00:52:41.605 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-30T00:52:41.616 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-30T00:52:41.650 INFO:teuthology.nuke:Checking targets against current locks 2024-03-30T00:52:41.674 DEBUG:teuthology.nuke:shortname: smithi066 2024-03-30T00:52:41.674 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T00:52:41.696 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 00:16:51.577550', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHquc1fXzOkR+Oc1lu0Vh5hvEbwh7hdzwryuzCoCkRz2kM9DTitUMYF4yCUJnGjeYGWwCNtdmZwlb/AZBCdkHzs='} 2024-03-30T00:52:41.700 DEBUG:teuthology.nuke:shortname: smithi156 2024-03-30T00:52:41.700 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T00:52:41.721 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 00:16:51.579006', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF3D/q5JOaLuZ73PSCANMnVcs6eIMU7C3S/m8GabWZjBMBkYYlRzUfN9LKxOP3IvKTZGl+HXiPoGXl3fFjGlbzA='} 2024-03-30T00:52:41.747 INFO:teuthology.orchestra.console.smithi066:Power off 2024-03-30T00:52:41.748 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T00:52:41.775 DEBUG:teuthology.orchestra.console.smithi066:power off output: Chassis Power Control: Down/Off 2024-03-30T00:52:41.775 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:52:41.790 INFO:teuthology.orchestra.console.smithi156:Power off 2024-03-30T00:52:41.790 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T00:52:41.805 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-03-30T00:52:41.818 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2024-03-30T00:52:41.818 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:52:41.842 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-03-30T00:52:45.807 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:52:45.844 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:52:45.918 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-03-30T00:52:45.954 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-03-30T00:52:49.919 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:52:49.955 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:52:50.030 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-03-30T00:52:50.065 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-03-30T00:52:54.033 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:52:54.066 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:52:54.143 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off 2024-03-30T00:52:54.177 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2024-03-30T00:52:54.245 INFO:teuthology.orchestra.console.smithi066:Power off completed 2024-03-30T00:52:54.278 INFO:teuthology.orchestra.console.smithi156:Power off completed 2024-03-30T00:52:54.384 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1464.258804321289 failure_reason: '"2024-03-30T00:46:41.436638+0000 mds.a (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default mon.a-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.b-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 owner: scheduled_rishabh@teuthology success: false 2024-03-30T00:52:54.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-30T00:52:54.481 INFO:teuthology.run:FAIL